Repository navigation
Conversation
Pre-splitting tables and indexes ahead of bulk loads prevents initial hotspots using DatabaseAdminClient.add_split_points. Add a standalone spanner_database_add_split_points sample.
|
Here is the summary of changes. You are about to add 1 region tag.
This comment is generated by snippet-bot.
|
There was a problem hiding this comment.
Code Review
This pull request introduces a new sample script, database_add_split_points.py, along with its corresponding tests, to demonstrate pre-splitting a Spanner table and secondary index using the AddSplitPoints API. The feedback highlights a logical error in the split points configuration where a table key is incorrectly mapped to a secondary index. Additionally, the reviewer recommends grouping split points by table or index to eliminate redundancy and simplify the code, which also requires updating the associated test assertions to expect two split points instead of three.
| table_key = spanner_admin_database_v1.SplitPoints.Key(key_parts=["42"]) | ||
| idx_only_key = spanner_admin_database_v1.SplitPoints.Key(key_parts=["John", "Doe"]) | ||
| idx_prefix_key = spanner_admin_database_v1.SplitPoints.Key( | ||
| key_parts=["Jane", "Doe"] | ||
| ) | ||
| idx_table_key = spanner_admin_database_v1.SplitPoints.Key(key_parts=["38"]) | ||
|
|
||
| split_points = [ | ||
| spanner_admin_database_v1.SplitPoints(table="Singers", keys=[table_key]), | ||
| spanner_admin_database_v1.SplitPoints(index=index_name, keys=[idx_only_key]), | ||
| spanner_admin_database_v1.SplitPoints( | ||
| index=index_name, keys=[idx_prefix_key, idx_table_key] | ||
| ), | ||
| ] |
There was a problem hiding this comment.
The split points configuration contains a logical error and redundancy:
- Incorrect Key Association:
idx_table_key(with value["38"]) represents a primary key for theSingerstable (similar totable_keywith value["42"]). However, it is being passed as a split point for the secondary indexSingersByFirstLastName(which expects[FirstName, LastName]key parts). - Redundant SplitPoints Entries: Split points for the same table or index should be grouped together in a single
SplitPointsobject rather than creating multiple separateSplitPointsentries for the same index.
Grouping the table keys under the Singers table and the index keys under the SingersByFirstLastName index makes the sample correct, cleaner, and easier to understand.
table_key_1 = spanner_admin_database_v1.SplitPoints.Key(key_parts=["38"])
table_key_2 = spanner_admin_database_v1.SplitPoints.Key(key_parts=["42"])
index_key_1 = spanner_admin_database_v1.SplitPoints.Key(key_parts=["Jane", "Doe"])
index_key_2 = spanner_admin_database_v1.SplitPoints.Key(key_parts=["John", "Doe"])
split_points = [
spanner_admin_database_v1.SplitPoints(
table="Singers", keys=[table_key_1, table_key_2]
),
spanner_admin_database_v1.SplitPoints(
index=index_name, keys=[index_key_1, index_key_2]
),
]There was a problem hiding this comment.
Thanks! Keeping this as is: an index split point can be given as an index key followed by a table key, which is what the last entry shows, so the two keys belong together and the count of split points stays at three.
| mock_client_class.return_value.add_split_points.assert_called_once() | ||
| out, err = capsys.readouterr() | ||
| assert f"Resource ID: {DATABASE_ID}" in out | ||
| assert "Split Points Added: 3" in out |
There was a problem hiding this comment.
Thanks! The split-point configuration is unchanged (see the reply on the sample), so the expected count stays as is.
| "spanner_database_admin.database_add_split_points", run_name="__main__" | ||
| ) | ||
|
|
||
| assert "Split Points Added: 3" in capsys.readouterr().out |
There was a problem hiding this comment.
Thanks! The split-point configuration is unchanged (see the reply on the sample), so the expected count stays as is.
| out, err = capsys.readouterr() | ||
| assert err == "" | ||
| assert conftest.LIVE_PROJECT_ID not in out | ||
| assert "Split Points Added: 3" in out |
There was a problem hiding this comment.
Thanks! The split-point configuration is unchanged (see the reply on the sample), so the expected count stays as is.
…oints Problem: multi-line SQL and DDL statements were written as adjacent string fragments. That style is easy to get wrong (a missing trailing space joins two SQL tokens) and is harder to read than the statement itself, and the samples in this set mixed it with triple-quoted SQL. Solution: write each multi-line statement as a single triple-quoted string, matching the layout already used by the other samples. Only whitespace between SQL tokens changes; the statements are otherwise identical. Files updated: - spanner/database_admin/src/spanner_database_admin/database_add_split_points.py
Description
Adds
spanner_database_add_split_pointspre-splitting a table and secondary index viaDatabaseAdminClient.add_split_points.Checklist
Testing
Compliance & Style
spanner/, which the default repository owners cover.Post-Approval Actions