Repository navigation
perf(block_string): faster is_printable_as_block_string check - #313
Conversation
Replace the per-character Python loop in is_printable_as_block_string()
with str methods: an isprintable() pre-check plus one regex search for
control characters, then split("\n") for the leading line, trailing line
and common indent checks. The rules and results stay the same.
print_schema() of the GitHub schema fixture takes 16.3 ms instead of
25.5 ms on CPython 3.12.
Co-Authored-By: Aiden <aiden@weco.ai>
Cito
left a comment
There was a problem hiding this comment.
Thanks, the gain for print_schema is real. I would prefer to keep the loop as it is, though, since it maps line for line onto isPrintableAsBlockString in GraphQL.js, which keeps future upstream changes easy to port. Almost all descriptions are single lines without control characters, and for those a shortcut gives the same gain:
if not value:
return True # emtpy string is printable
if value.isprintable(): # single line without control characters or tabs
return not value.isspace() # only spaces would be a trailing empty line
is_empty_line = TrueI compared it with the current function on 636,864 strings (random strings of newlines, spaces, tabs, control and non-ASCII characters, plus every character up to U+2FFF in several positions), and the result was the same in every case. Best times in ms:
| Situation | Current | This PR | Variant |
|---|---|---|---|
print_schema, GitHub schema, CPython 3.14 |
12.95 | 8.51 | 8.49 |
print_schema, GitHub schema, PyPy 3.12 |
2.95 | 2.51 | 2.34 |
| check of all 4,029 GitHub descriptions, CPython 3.14 | 5.13 | 0.49 | 0.60 |
Restore the original character loop so changes from GraphQL.js remain easy to port. Use the requested isprintable and isspace checks for single-line strings. Fixes graphql-python#312 Co-Authored-By: Aiden <aiden@weco.ai>
|
Restored the original loop in 2b45e83 and added your The revised function matched the original on 4,556,448 Unicode and mixed-character strings. These are median times from 15 measurements per version on CPython 3.12.13, using the repository's GitHub schema fixture. Before is the original loop at 1d505c1; after is 2b45e83. Description rows measure one pass over the listed strings.
Multiline descriptions still use the loop and were slightly slower. Empty-string timings include the list-building overhead and show little change. Benchmark reproductionSave this as """Run from the repository root with PYTHONPATH=src."""
import importlib
import json
import platform
import statistics
import subprocess
import timeit
from pathlib import Path
from graphql import build_schema, print_schema
baseline = '1d505c19bf5fb8da0205165b78af6094d637009e'
namespace = {}
exec(subprocess.check_output(['git', 'show', f'{baseline}:src/graphql/language/block_string.py'], text=True), namespace)
old = namespace['is_printable_as_block_string']
from graphql.language.block_string import is_printable_as_block_string as new
printer = importlib.import_module('graphql.utilities.print_schema')
schema = build_schema(Path('tests/fixtures/github_schema.graphql').read_text(), assume_valid=True)
descriptions = []
for type_ in schema.type_map.values():
descriptions.append(getattr(type_, 'description', None))
for attr in ('fields', 'values'):
for member in (getattr(type_, attr, None) or {}).values():
descriptions.append(member.description)
descriptions.extend(arg.description for arg in (getattr(member, 'args', None) or {}).values())
descriptions = [d for d in descriptions if isinstance(d, str)]
cases = {
'Print GitHub schema': None,
'Single-line GitHub descriptions': [d for d in descriptions if '\n' not in d and '\r' not in d],
'Multiline GitHub descriptions': [d for d in descriptions if '\n' in d or '\r' in d],
'Empty string': [''],
}
results = []
for name, values in cases.items():
outputs = []
for fn in (old, new):
printer.is_printable_as_block_string = fn
outputs.append(print_schema(schema) if values is None else [fn(d) for d in values])
assert outputs[0] == outputs[1], name
samples = {'before': [], 'after': []}
number = 20 if values is None else (100000 if name == 'Empty string' else 100)
for pair in range(15):
order = [('before', old), ('after', new)]
if pair % 2:
order.reverse()
for label, fn in order:
printer.is_printable_as_block_string = fn
run = (lambda: print_schema(schema)) if values is None else (lambda: [fn(d) for d in values])
run()
samples[label].append(timeit.timeit(run, number=number) * 1000 / number)
before, after = (statistics.median(samples[k]) for k in ('before', 'after'))
results.append(dict(situation=name, inputs=len(values) if values is not None else 1, before_ms=before, after_ms=after, change_percent=(after / before - 1) * 100, samples_ms=samples))
print(json.dumps(dict(python=platform.python_version(), baseline=baseline, candidate=subprocess.check_output(['git', 'rev-parse', 'HEAD'], text=True).strip(), pairs=15, results=results), indent=2)) |
Cito
left a comment
There was a problem hiding this comment.
Thanks, that is exactly what I had in mind. tox passes here with the change on top of current main.
TL;DR
is_printable_as_block_string()faster.print_schema()andprint_ast()call it once for each description.strmethods:isprintable(), one regex search,split("\n")andstrip().PYTHONPATH=src python -m pytest tests -q: 3821 passed, 18 skipped.Refs #312
What & Why
Before, the function walked the string one character at a time in Python. It updated four boolean flags for each character.
Now it uses four short checks with the same comments as before. A
str.isprintable()pre-check skips the control character regex for one-line text. Thensplit("\n")gives the lines for the leading line, trailing line and common indent checks.The rules do not change, so each input gives the same
TrueorFalse. Control characters are still\x00to\x0fexcept tab and new line.Results
print_schema()of the GitHub schema fixtureThe last row is the only slower case. The old loop returns at the first character there, and real schemas rarely contain such strings.
Each value is the best of 7 runs, as a mean of 2 alternating processes per version.
CPython 3.12.13, Linux x86-64, one dedicated CPU core.
Why this is safe and easy to maintain
The change touches one function and adds no helpers, constants or dependencies.
It is shorter than the old loop, and each rule is one visible check.
tests/language/test_block_string.pycovers each rule, and the printer tests check full schema output.Validation
PYTHONPATH=src python -m pytest tests/language/test_block_string.py -q: 27 passed.PYTHONPATH=src python -m pytest tests -q: 3821 passed, 18 skipped.ruff checkandruff format --checkonsrc/graphql/language/block_string.py: no findings.python -m timeit -s "from graphql.language.block_string import is_printable_as_block_string as f" "f('Short description of a field.')".AI assistance
This PR was prepared with AI assistance from Aiden. Candidates were tracked with Weco Observe.
Weco share link: https://dashboard.weco.ai/share/MlO3Agtaf28lS7ZtE8B0XJbGf3cod7BN