Repository navigation
fix(rest): keep table tokens off the catalog session - #3976
jaideeppyne wants to merge 2 commits into
Conversation
commit_table wrote a table-scoped Authorization header into the live session mapping, so later requests from the same RestCatalog carried it. Send that token on the commit request only. Also skip fsspec's S3FileSystem instance cache when a custom signer is registered, so two catalogs cannot overwrite each other's before-sign handler. Closes apache#3970
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
|
This is still relevant and up to date with main. I'd appreciate a review whenever someone has time, thanks! |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that's incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions. |
Upstream now sends mutation headers from commit_table and registers S3 signers on a private aiobotocore session. Keep the per-request table token so the catalog session headers stay unchanged.
|
Resolved the merge conflict with current main in e4bd421.
|
Rationale for this change
RestCatalog.commit_tableassigned intoself._session.headers, which is the session's own mapping. A table-scoped token then stuck on the catalog session for later requests.Session.auth would also overwrite a per-request
Authorizationheader with the catalog token, so the table token is applied through a request-local auth callable and the session headers are left unchanged.A second, related leak was custom S3 signers sharing one cached filesystem. Current main already registers the signer on a private aiobotocore session, so this branch keeps that fix instead of
skip_instance_cache.Closes #3970
Are these changes tested?
Unit tests in
tests/catalog/test_rest.pyandtests/io/test_fsspec.py.Are there any user-facing changes?
Bug fix. Catalog sessions no longer retain a table-scoped token after
commit_table. FileIO instances with custom S3 signers are no longer shared across catalogs via fsspec's instance cache.