Skip to content

Reject Packer buffer exports while packing in the C extension - #774

Open
ravi0800 wants to merge 1 commit into
msgpack:mainfrom
ravi0800:packer-export-while-packing
Open

ravi0800 wants to merge 1 commit into
msgpack:mainfrom
ravi0800:packer-export-while-packing

Conversation

@ravi0800

Copy link
Copy Markdown

The Cython Packer checks for live buffer exports only on entry to pack(), pack_map_pairs() and pack_ext_type(). Python code that runs mid-pack (a default callback, the pairs iterable, a list or dict subclass) can call memoryview(packer), and the next msgpack_pack_write() that grows the buffer PyMem_Reallocs pk.buf under that view. With Packer(buf_size=16, autoreset=False, default=...) where default stores memoryview(packer) and returns b"x" * 65536, the stored view reads b' ' instead of the b'\x91' array header, i.e. freed heap memory.

__getbuffer__ now raises BufferError while one of those three methods is running, using a nesting counter so a reentrant pack() from a callback doesn't clear the outer guard. The pure Python Packer already raises BufferError here because BytesIO refuses to resize while exported, so the new test in test_buffer.py passes on both implementations and fails on the unpatched C extension.

Packer only checked for live buffer exports on entry to pack(),
pack_map_pairs() and pack_ext_type(). Python code that runs during
packing (a default callback, the pairs iterable, a list or dict
subclass) could call memoryview(packer), after which
msgpack_pack_write() reallocated pk.buf and left the export pointing
at freed memory.

Refuse exports from __getbuffer__ while one of those methods is
running. The pure Python Packer already raises BufferError in this
case because BytesIO cannot be resized while exported.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant