Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@ release (0.x.y) only fixes bugs. Breaking changes are called out below.

## Unreleased

- `easy.validate_and_wait`: a validation job that ends FAILED or CANCELED_BY_ADMIN now
raises `JobError` naming the job and the dataset (`status` and `info` unchanged).
Both endings were checked live on the sandbox by cancelling a running job; the
docstring explains that a refused start is a `LockedError` (423), never a polled job.
- `dataflow` docstrings checked against the current backend source:
`dataflow_datasets` now says an unknown dataflow id answers 500 (not an empty
200), and lists the admin and national coordinator access; `dataflows` lists
Expand Down
24 changes: 24 additions & 0 deletions docs/live_testing_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -1030,3 +1030,27 @@ Dataflow `<sandbox dataflow>` answers `bigData: true`. For its datasets (`<sandb
`/dataset/exportFileDL`, `/dataset/{id}/downloadFileDL`. The validations export
(`/validation/export/{id}` then `/validation/downloadFile/{id}`) has no `DL`
variant in the UI config and gave no file on three tries for `<sandbox dataset>`.

## P10-19 — validation job endings (live, 2026-11-22, sandbox, Bearer)

`easy.validate_and_wait` on a data-lake dataset of `<sandbox dataflow>`, with the
validation job cancelled from a second thread through `PUT
/orchestrator/jobs/cancelJob/{jobId}?dataflowId=&datasetId=` (a route for dataflow
custodians, stewards and lead reporters; backend branch `MasterOneVersion`):

- The job is `QUEUED` for about the first 4 s, then `IN_PROGRESS`. A cancel at
4.6 s answered `200` but did not stick: the job went on to `FINISHED` and
`validate_and_wait` returned the normal result. Cancelling a queued job is not
reliable.
- Cancel at about 22 s with `jobShouldFail=true`: the poll answered `FAILED` (no
`info`) and `validate_and_wait` raised `JobError` with `status="FAILED"` at once.
- Cancel at about 22 s with `jobInfo=ERROR_VALIDATION_FAILURE`: `CANCELED_BY_ADMIN`
with `info` "Validation failed due to system error. Please contact the Service
Desk for support."; `JobError.status`/`.info` carried both.
- Both endings stayed as they were (re-polled minutes later). A new validation
started 5 s after each cancel was accepted, so no lock was left behind.
- A refused start never becomes a job to poll: `addValidationJob` stores the job
as `REFUSED`, sends the refused notification and answers `423`
(`LockedError`), as seen on 2026-11-21.
- Not produced: a plain `CANCELED` (only the backend's hourly cleanup of stuck
runs sets it) and `FINISHED` with an `info` note.
11 changes: 8 additions & 3 deletions docs/reference/easy.md
Original file line number Diff line number Diff line change
Expand Up @@ -650,9 +650,14 @@ with the job's `status` and `info`; a FINISHED job with an `info` text (for
example "There are canceled validation tasks", so the results may be partial)
warns with it and puts it in the result's `attrs["validation_note"]`. Read from the
backend source (`MasterOneVersion` branch) and checked live on the sandbox
(Bearer): the start answered a job id and polling it reached FINISHED. A start
while another job runs on the dataset gets `LockedError` (423) at once. The
FAILED/REFUSED/CANCELED and FINISHED-with-note endings are tested with mocks only.
(Bearer): the start answered a job id and polling it reached FINISHED. A
refused start (another job runs on the dataset) never becomes a job to poll:
the backend answers the start with `LockedError` (423) at once. A job
cancelled while running (`/orchestrator/jobs/cancelJob`) ends FAILED or
CANCELED_BY_ADMIN, with the `info` text if the cancel gave one; both were
checked live on the sandbox, and the dataset was not left locked. A plain
CANCELED (the backend's hourly cleanup of stuck runs) and the
FINISHED-with-note ending are tested with mocks only.

Without a job id (older backends), or when the poll route is refused, the
signal is the user notification the backend creates when a run ends
Expand Down
18 changes: 15 additions & 3 deletions reportnet_api/easy.py
Original file line number Diff line number Diff line change
Expand Up @@ -478,6 +478,13 @@ def _wait_validation_job(
f"Validation job {job_id} for dataset {dataset_id} did not finish within "
f"{timeout:g} seconds (it may still be running on the server)"
) from err
if err.status is not None:
raise JobError(
f"Validation job {job_id} for dataset {dataset_id} ended {err.status}"
+ (f": {err.info}" if err.info else ""),
status=err.status,
info=err.info,
) from err
raise


Expand Down Expand Up @@ -509,9 +516,14 @@ def validate_and_wait(
example "There are canceled validation tasks", so the results may be partial)
warns with it and puts it in the result's `attrs["validation_note"]`. Read from the
backend source (`MasterOneVersion` branch) and checked live on the sandbox
(Bearer): the start answered a job id and polling it reached FINISHED. A start
while another job runs on the dataset gets `LockedError` (423) at once. The
FAILED/REFUSED/CANCELED and FINISHED-with-note endings are tested with mocks only.
(Bearer): the start answered a job id and polling it reached FINISHED. A
refused start (another job runs on the dataset) never becomes a job to poll:
the backend answers the start with `LockedError` (423) at once. A job
cancelled while running (`/orchestrator/jobs/cancelJob`) ends FAILED or
CANCELED_BY_ADMIN, with the `info` text if the cancel gave one; both were
checked live on the sandbox, and the dataset was not left locked. A plain
CANCELED (the backend's hourly cleanup of stuck runs) and the
FINISHED-with-note ending are tested with mocks only.

Without a job id (older backends), or when the poll route is refused, the
signal is the user notification the backend creates when a run ends
Expand Down
24 changes: 24 additions & 0 deletions tests/test_easy.py
Original file line number Diff line number Diff line change
Expand Up @@ -381,6 +381,30 @@ def test_validate_and_wait_raises_when_the_job_fails(monkeypatch: pytest.MonkeyP
assert caught.value.status == "FAILED" and caught.value.info == info


@pytest.mark.parametrize(
("answer", "expected"),
[
# Both seen live on the sandbox after cancelling a running validation job.
({"status": "FAILED"}, "Validation job 44570 for dataset 9 ended FAILED"),
(
{"status": "CANCELED_BY_ADMIN", "info": "Stopped."},
"Validation job 44570 for dataset 9 ended CANCELED_BY_ADMIN: Stopped.",
),
],
)
@respx.mock
def test_validate_and_wait_job_failure_names_job_and_dataset(
monkeypatch: pytest.MonkeyPatch, answer: dict[str, str], expected: str
) -> None:
monkeypatch.setattr("reportnet_api.jobs.time.sleep", lambda s: None)
_job_backend([httpx.Response(200, json=answer)])
with pytest.raises(JobError) as caught:
easy.validate_and_wait(easy.connect("k"), 9, 5)
assert str(caught.value) == expected
assert caught.value.status == answer["status"]
assert caught.value.info == answer.get("info")


@respx.mock
def test_validate_and_wait_warns_on_finished_with_info(monkeypatch: pytest.MonkeyPatch) -> None:
note = "There are canceled validation tasks. Please review and address the issues."
Expand Down