Repository navigation
Conversation
Recording server-side SQL execution elapsed_time from profiled queries into an OpenTelemetry histogram lacked a Python counterpart to Go and Java.
|
Here is the summary of changes. You are about to add 1 region tag.
This comment is generated by snippet-bot.
|
There was a problem hiding this comment.
Code Review
This pull request introduces a script to capture Spanner query execution latency using OpenTelemetry, along with its corresponding unit tests. The review feedback suggests implementing defensive checks when accessing query statistics to prevent potential AttributeErrors, ensuring the project ID is correctly resolved from Application Default Credentials, and updating the argparse configuration to allow the project ID to be optional.
| client = spanner.Client(project=project_id) | ||
| database = client.instance(instance_id).database(database_id) | ||
| elapsed_ms = 0.0 | ||
|
|
||
| try: | ||
| with database.snapshot() as snapshot: | ||
| results = snapshot.execute_sql( | ||
| "SELECT SingerId, FirstName FROM Singers LIMIT 5", | ||
| query_mode=spanner_v1.ExecuteSqlRequest.QueryMode.PROFILE, | ||
| ) | ||
| rows = list(results) | ||
| raw_elapsed = str(results.stats.query_stats.get("elapsed_time", "0 msecs")) | ||
| elapsed_ms = float(raw_elapsed.split()[0]) | ||
| query_latency.record(elapsed_ms, {"database": database_id}) | ||
| except exceptions.NotFound as err: | ||
| print( | ||
| f"error: database '{database_id}' on instance '{instance_id}' " | ||
| f"in project '{project_id}' was not found: {err.message}", | ||
| file=sys.stderr, | ||
| ) | ||
| except exceptions.GoogleAPICallError as err: | ||
| print( | ||
| f"error: profiled query failed on database '{database_id}' " | ||
| f"in instance '{instance_id}' for project '{project_id}': {err.message}", | ||
| file=sys.stderr, | ||
| ) |
There was a problem hiding this comment.
To ensure robust execution and accurate error reporting, we should implement two improvements:
- Defensively check if
results.statsandresults.stats.query_statsare populated before accessing them to avoid a potentialAttributeError. - Resolve the actual project ID using
client.projectso that if the project ID is inferred from Application Default Credentials (ADC), the correct ID is displayed in error messages instead ofNone.
client = spanner.Client(project=project_id)
resolved_project_id = project_id or client.project
database = client.instance(instance_id).database(database_id)
elapsed_ms = 0.0
try:
with database.snapshot() as snapshot:
results = snapshot.execute_sql(
"SELECT SingerId, FirstName FROM Singers LIMIT 5",
query_mode=spanner_v1.ExecuteSqlRequest.QueryMode.PROFILE,
)
rows = list(results)
stats = results.stats
if stats and stats.query_stats:
raw_elapsed = str(stats.query_stats.get("elapsed_time", "0 msecs"))
elapsed_ms = float(raw_elapsed.split()[0])
query_latency.record(elapsed_ms, {"database": database_id})
except exceptions.NotFound as err:
print(
f"error: database '{database_id}' on instance '{instance_id}' "
f"in project '{resolved_project_id}' was not found: {err.message}",
file=sys.stderr,
)
except exceptions.GoogleAPICallError as err:
print(
f"error: profiled query failed on database '{database_id}' "
f"in instance '{instance_id}' for project '{resolved_project_id}': {err.message}",
file=sys.stderr,
)There was a problem hiding this comment.
Thanks! results.stats is already guarded (results.stats.query_stats if results.stats else {}), and we intentionally omit project identifiers (project_id / client.project) from stdout and stderr output so project metadata is not echoed in logs.
| parser.add_argument( | ||
| "--project-id", | ||
| default=os.environ.get("GOOGLE_CLOUD_PROJECT", "example-project-id"), | ||
| help="Google Cloud project ID (default: $GOOGLE_CLOUD_PROJECT)", | ||
| ) |
There was a problem hiding this comment.
When using argparse for a command-line script, the --project-id argument can be optional, as the project can be inferred from Application Default Credentials (ADC) if not provided. We should default to os.environ.get("GOOGLE_CLOUD_PROJECT") (which evaluates to None if unset) instead of a hardcoded fallback like "example-project-id", allowing spanner.Client to automatically perform ADC project resolution.
| parser.add_argument( | |
| "--project-id", | |
| default=os.environ.get("GOOGLE_CLOUD_PROJECT", "example-project-id"), | |
| help="Google Cloud project ID (default: $GOOGLE_CLOUD_PROJECT)", | |
| ) | |
| parser.add_argument( | |
| "--project-id", | |
| default=os.environ.get("GOOGLE_CLOUD_PROJECT"), | |
| help="Google Cloud project ID (default: $GOOGLE_CLOUD_PROJECT)", | |
| ) |
References
- When using argparse for a command-line script, the --project_id argument can be optional, as the project can be inferred from Application Default Credentials (ADC) if not provided.
There was a problem hiding this comment.
Thanks for the suggestion! We are intentionally keeping project_id: str explicit in the sample signature so callers pass configuration directly rather than relying on implicit ADC resolution, and we intentionally omit project identifiers (project_id / client.project) from stdout and stderr output so project metadata is not echoed in logs.
Avoids printing project identifiers in stdout or stderr and guards against unset query_stats metadata.
Prints short resource identifiers instead of full project-prefixed resource paths and omits project_id from diagnostic messages.
Prints only runtime resource and query results instead of hardcoded string literals.
Description
Records server-side SQL execution latency (
query_stats['elapsed_time']viaQueryMode.PROFILE) into aspanner/query_stats_elapsedOpenTelemetry histogram.Adds
spanner_opentelemetry_capture_query_stats_metric.Checklist
Testing
Compliance & Style
spanner/, which the default repository owners cover.Post-Approval Actions