Skip to content

gh-106939: document ShareableList nul-strip quirk. #107266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gpshead merged 3 commits into from
Jul 25, 2023
Merged

gh-106939: document ShareableList nul-strip quirk. #107266

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ee01fff
gh-106939: document ShareableList nul-strip quirk.
gpshead Jul 25, 2023
1013e30
Mention the `int` size constraint.
gpshead Jul 25, 2023
59dea5a
fix the doctest
gpshead Jul 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
gh-106939: document ShareableList nul-strip quirk.
  • Loading branch information
@gpshead
gpshead committed Jul 25, 2023
commit ee01fffc077d12bfee07122f73961ef8c8600438
29 changes: 29 additions & 0 deletions Doc/library/multiprocessing.shared_memory.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,35 @@ shared memory blocks created using that manager are all released when the
existing ``ShareableList``, specify its shared memory block's unique
name while leaving ``sequence`` set to ``None``.

.. note::

A known issue exists for :class:`bytes` and :class:`str` values.
If they end with ``\x00`` nul bytes or characters, those may be
*silently stripped* when fetching them by index from the
:class:`ShareableList`. This ``.rstrip(b'\x00')`` behavior is
considered a bug and may go away in the future. See :gh:`106939`.

For applications where rstripping of trailing nulls is a problem,
work around it by always unconditionally appending an extra non-0
byte to the end of such values when storing and unconditionally
removing it when fetching:

.. doctest::

>>> from multiprocessing import shared_memory
>>> nul_bug_demo = shared_memory.ShareableList(['?\x00', b'\x03\x02\x01\x00\x00\x00'])
>>> nul_bug_demo[0]
'?'
>>> nul_bug_demo[1]
b'\x03\x02\x01'
>>> nul_bug_demo.shm.shutdown()
>>> padded = shared_memory.ShareableList(['?\x00\x07', b'\x03\x02\x01\x00\x00\x00\x07'])
>>> padded[0][:-1]
'?\x00'
>>> padded[1][:-1]
b'\x03\x02\x01\x00\x00\x00'
>>> padded.shm.shutdown()

.. method:: count(value)

Returns the number of occurrences of ``value``.
Expand Down