diff --git a/CHANGELOG.md b/CHANGELOG.md index 3dd4b0b..a0b73dd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,9 @@ release (0.x.y) only fixes bugs. Breaking changes are called out below. ## Unreleased +- `datasets.import_big_file_data` docstring: on the current backend a data-lake import + answers a job map like any other import (checked live on the sandbox); the empty-body + `{}` case is for older backends. - `read_tables` (and `read_table`, `read_records`, `read_all_records`): when the export has no rows at all, the dataflow's `bigData` flag is looked up. A data-lake dataflow gets a definite warning pointing to `export_to_file()`; a dataflow that answers diff --git a/docs/reference/datasets.md b/docs/reference/datasets.md index 3f8dfcd..0c90af8 100644 --- a/docs/reference/datasets.md +++ b/docs/reference/datasets.md @@ -1114,11 +1114,14 @@ Import file to dataset data (Large files) | Design dataset | CUSTODIAN, STEWARD, EDITOR WRITE | | EU dataset | CUSTODIAN, STEWARD | -Overrides the generated method. An empty response body returns `{}` (data lake -datasets answer `void`: the import is done inside the request, MasterDataLakes `DatasetControllerImpl:391-422`; -other datasets answer a job map), and `timeout` overrides the client's timeout for -this call only (a lake import can run for minutes). `preparation_code` is sent as -`code`. +Overrides the generated method. The answer is a job map (`jobId`, `pollingUrl`; +poll it with `jobs.wait_for_job`), also for data lake datasets on the current +backend (`MasterOneVersion`; checked live on the sandbox, where a two-row lake +import finished in about 20 s). Older backends answer a lake import with an +empty body (`void`, the import is done inside the request; `MasterDataLakes` +`DatasetControllerImpl:391-422`), returned as `{}`. `timeout` overrides the +client's timeout for this call only (a lake import can run for minutes). +`preparation_code` is sent as `code`. **Live-tested notes:** diff --git a/reportnet_api/resources/datasets_ext.py b/reportnet_api/resources/datasets_ext.py index 62ed3ed..cab165a 100644 --- a/reportnet_api/resources/datasets_ext.py +++ b/reportnet_api/resources/datasets_ext.py @@ -265,11 +265,14 @@ def import_big_file_data( EU dataset: CUSTODIAN, STEWARD - Overrides the generated method. An empty response body returns `{}` (data lake - datasets answer `void`: the import is done inside the request, MasterDataLakes `DatasetControllerImpl:391-422`; - other datasets answer a job map), and `timeout` overrides the client's timeout for - this call only (a lake import can run for minutes). `preparation_code` is sent as - `code`.""" + Overrides the generated method. The answer is a job map (`jobId`, `pollingUrl`; + poll it with `jobs.wait_for_job`), also for data lake datasets on the current + backend (`MasterOneVersion`; checked live on the sandbox, where a two-row lake + import finished in about 20 s). Older backends answer a lake import with an + empty body (`void`, the import is done inside the request; `MasterDataLakes` + `DatasetControllerImpl:391-422`), returned as `{}`. `timeout` overrides the + client's timeout for this call only (a lake import can run for minutes). + `preparation_code` is sent as `code`.""" params = { "dataflowId": dataflow_id, "delimiter": delimiter, @@ -640,11 +643,14 @@ async def import_big_file_data( EU dataset: CUSTODIAN, STEWARD - Overrides the generated method. An empty response body returns `{}` (data lake - datasets answer `void`: the import is done inside the request, MasterDataLakes `DatasetControllerImpl:391-422`; - other datasets answer a job map), and `timeout` overrides the client's timeout for - this call only (a lake import can run for minutes). `preparation_code` is sent as - `code`.""" + Overrides the generated method. The answer is a job map (`jobId`, `pollingUrl`; + poll it with `jobs.wait_for_job`), also for data lake datasets on the current + backend (`MasterOneVersion`; checked live on the sandbox, where a two-row lake + import finished in about 20 s). Older backends answer a lake import with an + empty body (`void`, the import is done inside the request; `MasterDataLakes` + `DatasetControllerImpl:391-422`), returned as `{}`. `timeout` overrides the + client's timeout for this call only (a lake import can run for minutes). + `preparation_code` is sent as `code`.""" params = { "dataflowId": dataflow_id, "delimiter": delimiter,