Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ All notable changes are listed here. The project follows [Semantic Versioning](h
while the major version is `0`, a **minor** release (0.x.0) may change the API, and a patch
release (0.x.y) only fixes bugs. Breaking changes are called out below.

## Unreleased

- New `easy.list_representatives(client, dataflow_id)`: the countries assigned to a
dataflow as a table, with the country code and name joined from the provider
listing (one lookup per provider group). Lead reporter email addresses are left out
unless `include_contacts=True`. Also in `easy.guide()` and the glossary.

## 0.1.4 (2026-11-21)

- `datasets.import_big_file_data` docstring: on the current backend a data-lake import
Expand Down
5 changes: 3 additions & 2 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ integers; the most common mistake is passing a dataflow id where a dataset id be
`easy.list_tables` lists them and `read_table(..., table_name=...)` picks one.
- **Provider** (data provider): the reporting party, usually a country, that owns one
reporting dataset (`provider_id`). `easy.list_providers` lists the known ones.
- **Representative**: a provider assigned to a dataflow, with its lead reporters
(`client.representative`).
- **Representative**: a provider assigned to a dataflow, with its lead reporters.
`easy.list_representatives` lists them with the country code and name; lead reporter
emails only with `include_contacts=True`.
- **Lead reporter**: the person (by email) who reports for a provider.
- **Custodian / Steward / Reporter**: roles on a dataflow. A login (Bearer) token carries
the user's roles; an ApiKey carries Reporter-level access only.
Expand Down
22 changes: 22 additions & 0 deletions docs/reference/easy.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ Hand-written helpers in `reportnet_api.easy`. They are plain functions, not meth
- [list_dataflows](#list_dataflows)
- [list_datasets](#list_datasets)
- [list_providers](#list_providers)
- [list_representatives](#list_representatives)
- [list_tables](#list_tables)
- [read_all_records](#read_all_records)
- [read_many](#read_many)
Expand Down Expand Up @@ -448,6 +449,27 @@ A provider is a reporting country; `provider_id` is the `providerId` other
calls take and `code` its country code. Group `1` is the member
countries; `client.representative.data_provider_groups()` lists the groups.

### list_representatives

A dataflow's representatives (the countries assigned to it), one row each.

**Call:** `easy.list_representatives(...)`

| Argument | Type | Required | Default | Description |
|---|---|---|---|---|
| `client` | `ReportnetClient` | yes | | |
| `dataflow_id` | `int` | yes | | |
| `include_contacts` | `bool` | no | `False` | |

**Returns:** `pd.DataFrame`

Columns: representative_id, provider_id, code and label (the country, joined
from the provider listing), lead_reporters (how many), has_datasets,
receipt_downloaded, receipt_outdated and restrict_from_public. Lead reporter
email addresses are personal data, so they are left out unless
`include_contacts=True`, which adds a `lead_reporter_emails` list column. A
dataflow without representatives gives an empty table.

### list_tables

Tables defined in a dataset's schema: table_id and name.
Expand Down
1 change: 1 addition & 0 deletions docs/reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,7 @@ Every resource method also exists on `AsyncReportnetClient`, called with `await`
- [`list_dataflows`](easy.md#list_dataflows): All dataflows visible to the credential, one row each (id, name, status, ...).
- [`list_datasets`](easy.md#list_datasets): Datasets of a dataflow as a flat table: dataset_id, name, kind, provider_id.
- [`list_providers`](easy.md#list_providers): Data providers of a group as a table: provider_id, code, label, group_id.
- [`list_representatives`](easy.md#list_representatives): A dataflow's representatives (the countries assigned to it), one row each.
- [`list_tables`](easy.md#list_tables): Tables defined in a dataset's schema: table_id and name.
- [`read_all_records`](easy.md#read_all_records): Every table as ``{table name: list of row dicts}``, without pandas (see `read_records`).
- [`read_many`](easy.md#read_many): One table from several dataset handles, stacked (see `handles.read_many`).
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/representative.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,7 +294,7 @@ representatives (e.g. a design dataflow) gives an empty list.

**Live-tested notes:**

- Live-checked (ApiKey and Bearer) only on a dataflow with no representatives, so the empty list is verified and the item shape comes from the backend source. `leadReporters` include email addresses (personal data).
- Live-checked (ApiKey and Bearer) on a dataflow with no representatives (empty list) and with Bearer on a sandbox dataflow with one: the item shape matches the list above, and `leadReporters` items are `{id, email, representativeId, invalid}` (email addresses: personal data). `easy.list_representatives` adds the country.

### validate_lead_reporters

Expand Down
73 changes: 73 additions & 0 deletions reportnet_api/easy.py
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,78 @@ def list_providers(client: ReportnetClient, group_id: int = 1) -> pd.DataFrame:
return pd.DataFrame(rows, columns=["provider_id", "code", "label", "group_id"])


_REPRESENTATIVE_COLUMNS = [
"representative_id",
"provider_id",
"code",
"label",
"lead_reporters",
"has_datasets",
"receipt_downloaded",
"receipt_outdated",
"restrict_from_public",
]


def _providers_by_id(
client: ReportnetClient, reps: list[dict[str, Any]]
) -> dict[int, dict[str, Any]]:
"""Provider records for the representatives: one call per group, then per stray id."""
found: dict[int, dict[str, Any]] = {}
groups = {r.get("dataProviderGroupId") for r in reps}
for group in sorted(g for g in groups if isinstance(g, int)):
for p in client.representative.data_providers(group) or []:
if isinstance(p.get("id"), int):
found[p["id"]] = p
for rep in reps:
pid = rep.get("dataProviderId")
if isinstance(pid, int) and pid not in found:
found[pid] = client.representative.data_provider(pid) or {}
return found


def list_representatives(
client: ReportnetClient, dataflow_id: int, *, include_contacts: bool = False
) -> pd.DataFrame:
"""A dataflow's representatives (the countries assigned to it), one row each.

Columns: representative_id, provider_id, code and label (the country, joined
from the provider listing), lead_reporters (how many), has_datasets,
receipt_downloaded, receipt_outdated and restrict_from_public. Lead reporter
email addresses are personal data, so they are left out unless
`include_contacts=True`, which adds a `lead_reporter_emails` list column. A
dataflow without representatives gives an empty table.
"""
pd = _pandas()
_require_id(dataflow_id, "dataflow_id", _DATAFLOW_HINT)
try:
reps = client.representative.representatives(dataflow_id) or []
providers = _providers_by_id(client, reps)
except ForbiddenError as err:
raise _hinted(err, _FORBIDDEN_HINT) from err
rows = []
for rep in reps:
pid = rep.get("dataProviderId")
provider = providers.get(pid, {}) if isinstance(pid, int) else {}
leads = rep.get("leadReporters") or []
row: dict[str, Any] = {
"representative_id": rep.get("id"),
"provider_id": pid,
"code": provider.get("code"),
"label": provider.get("label"),
"lead_reporters": len(leads),
"has_datasets": rep.get("hasDatasets"),
"receipt_downloaded": rep.get("receiptDownloaded"),
"receipt_outdated": rep.get("receiptOutdated"),
"restrict_from_public": rep.get("restrictFromPublic"),
}
if include_contacts:
row["lead_reporter_emails"] = [lead.get("email") for lead in leads]
rows.append(row)
columns = _REPRESENTATIVE_COLUMNS + (["lead_reporter_emails"] if include_contacts else [])
return pd.DataFrame(rows, columns=columns)


def list_tables(client: ReportnetClient, dataset_id: int) -> pd.DataFrame:
"""Tables defined in a dataset's schema: table_id and name."""
pd = _pandas()
Expand Down Expand Up @@ -1570,6 +1642,7 @@ def check_access(
2. Find ids list_dataflows(client) # dataflow ids and names
list_datasets(client, dataflow_id) # dataset ids (NOT the dataflow id)
flow = find_dataflow(client, id_or_name); flow.dataset(id_or_country)
list_providers(client); list_representatives(client, dataflow_id)
3. Read list_tables(client, dataset_id); read_table(client, dataset_id, dataflow_id)
read_tables(...) for all tables; read_many(datasets, table) to stack
4. Validation validation_summary / validation_errors; validate_and_wait (WRITES)
Expand Down
7 changes: 4 additions & 3 deletions scripts/docs_caveats.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,9 +311,10 @@
# --- representative
"representatives": [
(
"Live-checked (ApiKey and Bearer) only on a dataflow with no representatives, so the "
"empty list is verified and the item shape comes from the backend source. "
"`leadReporters` include email addresses (personal data).",
"Live-checked (ApiKey and Bearer) on a dataflow with no representatives (empty "
"list) and with Bearer on a sandbox dataflow with one: the item shape matches the "
"list above, and `leadReporters` items are `{id, email, representativeId, invalid}` "
"(email addresses: personal data). `easy.list_representatives` adds the country.",
"",
)
],
Expand Down
98 changes: 98 additions & 0 deletions tests/test_representative.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,3 +361,101 @@ def test_lead_reporter_errors_propagate() -> None:
with pytest.raises(ReportnetAPIError) as excinfo:
client.representative.create_representative(5, data_provider_id=2)
assert excinfo.value.status_code == 400


GREECE = {"id": 60, "group": None, "label": "Greece", "code": "EL", "groupId": 2}
LIVE_REPS = [
{
"id": 7,
"dataProviderId": 60,
"dataProviderGroupId": 2,
"leadReporters": [
{"id": 1, "email": "a@example.org", "representativeId": 7, "invalid": False},
{"id": 2, "email": "b@example.org", "representativeId": 7, "invalid": True},
],
"receiptDownloaded": False,
"receiptOutdated": True,
"hasDatasets": True,
"restrictFromPublic": False,
},
{**REPS[0], "id": 8},
]
REP_COLUMNS = [
"representative_id",
"provider_id",
"code",
"label",
"lead_reporters",
"has_datasets",
"receipt_downloaded",
"receipt_outdated",
"restrict_from_public",
]


def _mock_live_reps(mock: respx.MockRouter) -> None:
mock.get("/representative/v1/dataflow/5").mock(return_value=httpx.Response(200, json=LIVE_REPS))
mock.get("/representative/dataProvider/2").mock(return_value=httpx.Response(200, json=[GREECE]))
mock.get("/representative/dataProvider/1").mock(
return_value=httpx.Response(200, json=PROVIDERS)
)


def test_easy_list_representatives_joins_countries_and_hides_emails() -> None:
pytest.importorskip("pandas")
with respx.mock(base_url=BASE_URL) as mock:
_mock_live_reps(mock)
with ReportnetClient(base_url=BASE_URL, token="t") as client:
df = easy.list_representatives(client, 5)
assert list(df.columns) == REP_COLUMNS
assert df["code"].tolist() == ["EL", "AT"]
assert df["label"].tolist() == ["Greece", "Austria"]
assert df["lead_reporters"].tolist() == [2, 0]
assert df["receipt_outdated"].tolist() == [True, False]
assert "@" not in df.to_csv()


def test_easy_list_representatives_include_contacts() -> None:
pytest.importorskip("pandas")
with respx.mock(base_url=BASE_URL) as mock:
_mock_live_reps(mock)
with ReportnetClient(base_url=BASE_URL, token="t") as client:
df = easy.list_representatives(client, 5, include_contacts=True)
assert list(df.columns) == [*REP_COLUMNS, "lead_reporter_emails"]
assert df["lead_reporter_emails"].tolist() == [["a@example.org", "b@example.org"], []]


def test_easy_list_representatives_one_lookup_per_group_and_fallback() -> None:
pytest.importorskip("pandas")
reps = [{**LIVE_REPS[0], "id": 7}, {**LIVE_REPS[0], "id": 9, "dataProviderId": 61}]
stray = {"id": 61, "group": None, "label": "Iceland", "code": "IS", "groupId": 2}
with respx.mock(base_url=BASE_URL) as mock:
mock.get("/representative/v1/dataflow/5").mock(return_value=httpx.Response(200, json=reps))
group = mock.get("/representative/dataProvider/2").mock(
return_value=httpx.Response(200, json=[GREECE])
)
single = mock.get("/representative/dataProvider/id/61").mock(
return_value=httpx.Response(200, json=stray)
)
with ReportnetClient(base_url=BASE_URL, token="t") as client:
df = easy.list_representatives(client, 5)
assert group.call_count == 1
assert single.call_count == 1
assert df["code"].tolist() == ["EL", "IS"]


def test_easy_list_representatives_empty() -> None:
pytest.importorskip("pandas")
with respx.mock(base_url=BASE_URL) as mock:
mock.get("/representative/v1/dataflow/5").mock(return_value=httpx.Response(200, json=[]))
with ReportnetClient(base_url=BASE_URL, token="t") as client:
df = easy.list_representatives(client, 5)
assert df.empty
assert list(df.columns) == REP_COLUMNS


def test_easy_list_representatives_rejects_bad_id() -> None:
pytest.importorskip("pandas")
with ReportnetClient(base_url=BASE_URL, token="t") as client:
with pytest.raises(ValueError, match="dataflow_id"):
easy.list_representatives(client, 0)