Skip to content

fix(rest): keep table tokens off the catalog session - #3976

Open
jaideeppyne wants to merge 2 commits into
apache:mainfrom
jaideeppyne:fix/commit-table-session-headers
Open

jaideeppyne wants to merge 2 commits into
apache:mainfrom
jaideeppyne:fix/commit-table-session-headers

Conversation

@jaideeppyne

@jaideeppyne jaideeppyne commented Sep 16, 2026 •

Copy link
Copy Markdown

Rationale for this change

RestCatalog.commit_table assigned into self._session.headers, which is the session's own mapping. A table-scoped token then stuck on the catalog session for later requests.

Session.auth would also overwrite a per-request Authorization header with the catalog token, so the table token is applied through a request-local auth callable and the session headers are left unchanged.

A second, related leak was custom S3 signers sharing one cached filesystem. Current main already registers the signer on a private aiobotocore session, so this branch keeps that fix instead of skip_instance_cache.

Closes #3970

Are these changes tested?

Unit tests in tests/catalog/test_rest.py and tests/io/test_fsspec.py.

Are there any user-facing changes?

Bug fix. Catalog sessions no longer retain a table-scoped token after commit_table. FileIO instances with custom S3 signers are no longer shared across catalogs via fsspec's instance cache.

commit_table wrote a table-scoped Authorization header into the live
session mapping, so later requests from the same RestCatalog carried it.
Send that token on the commit request only.

Also skip fsspec's S3FileSystem instance cache when a custom signer is
registered, so two catalogs cannot overwrite each other's before-sign
handler.

Closes apache#3970
@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Oct 17, 2026
@jaideeppyne

Copy link
Copy Markdown
Author

This is still relevant and up to date with main. I'd appreciate a review whenever someone has time, thanks!

@github-actions github-actions Bot removed the stale label Oct 18, 2026
@github-actions

Copy link
Copy Markdown

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.

@github-actions github-actions Bot added the stale label Nov 18, 2026
Upstream now sends mutation headers from commit_table and registers S3 signers on a private aiobotocore session. Keep the per-request table token so the catalog session headers stay unchanged.
@jaideeppyne

Copy link
Copy Markdown
Author

Resolved the merge conflict with current main in e4bd421.

commit_table still puts a table-scoped token on that request only, and it now also sends _mutation_headers(). The session headers are not written. Main already registers a custom S3 signer on a private aiobotocore session, so I kept that instead of the older skip_instance_cache workaround.

test_commit_table_does_not_mutate_session_headers, test_table_identifier_in_commit_table_request, and the two per-session signer tests pass locally. Please keep this open for review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Per-catalog auth state is shared: commit_table mutates the session, and cached S3FileSystem instances share a signer

1 participant