Skip to content

gh-95913: Edit, expand & format Bytecode sect in 3.11 WhatsNew #98559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iritkatriel merged 13 commits into from
Oct 24, 2022
Merged

gh-95913: Edit, expand & format Bytecode sect in 3.11 WhatsNew #98559

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
48a18d5
Fix Sphinx syntax in bytecode section and tweak phrasing
CAM-Gerlach Oct 20, 2022
4a3d830
Use clear, consistant phrasing on all Bytecode entries
CAM-Gerlach Oct 20, 2022
611a628
Re-order opcode changes to Added, Replaced, Removed, Changed
CAM-Gerlach Oct 20, 2022
68bef2a
Add missing added and removed opcodes to section
CAM-Gerlach Oct 21, 2022
f34698c
Add table with all added/replace/removed opcodes
CAM-Gerlach Oct 22, 2022
650b4e4
Merge opcode information from list into tables
CAM-Gerlach Oct 22, 2022
87e06bb
Add exception reprisentation change to Changed Opcodes
CAM-Gerlach Oct 23, 2022
9c165b8
Unify & simplify added and replaced tables & entries
CAM-Gerlach Oct 23, 2022
1e0d7cb
Move added opcodes out of table into a bulleted list instead
CAM-Gerlach Oct 23, 2022
a0b90d8
Re-order Replaced Opcodes table by replaced opcode
CAM-Gerlach Oct 23, 2022
4ed6cfb
Consolidate changed and removed sections to be more concise
CAM-Gerlach Oct 23, 2022
7c94ab0
Consolidate jump opcode changes further
CAM-Gerlach Oct 23, 2022
c1a7743
CHECK_EG_MATCH is new
iritkatriel Oct 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Move added opcodes out of table into a bulleted list instead
  • Loading branch information
@CAM-Gerlach
CAM-Gerlach committed Oct 23, 2022
commit 1e0d7cb939c7233ccd5d8b512c1a4fee5859d56e
129 changes: 66 additions & 63 deletions Doc/whatsnew/3.11.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1407,73 +1407,76 @@ contributors are volunteers from the community.
CPython bytecode changes
========================

The bytecode now contains inline cache entries,
which take the form of the newly-added :opcode:`CACHE` instructions.
Many opcodes expect to be followed by an exact number of caches,
and instruct the interpreter to skip over them at runtime.
Populated caches can look like arbitrary instructions,
so great care should be taken when reading or modifying
raw, adaptive bytecode containing quickened data.


.. _whatsnew311-added-opcodes:
Copy link
Copy Markdown
Member

@iritkatriel iritkatriel Oct 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The list we have now has all the jump related changes together, but if you split it into a "added" table and a "replaced" table then they are no longer together. I don't know if that's an improvement.

Copy link
Copy Markdown
Member

@iritkatriel iritkatriel Oct 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I created an alternative PR -#98566. It tries to consolidate related items into a high level descriptions, rather than enumerate all the changes. Maybe this is more useful for whatsnew. The full details, if you need them, as in the dis doc.

I added an item about the exc-info related changes. Didn't do anything about these:

GET_AWAITABLE

WITH_EXCEPT_START
If the low bit of namei is set, then a NULL is pushed to the stack before the global variable.

LOAD_GLOBAL
i is no longer offset by the length of co_varnames

LOAD_CLOSURE
LOAD_DEREF
LOAD_CLASSDEREF
STORE_DEREF
DELETE_DEREF

Copy link
Copy Markdown
Member Author

@CAM-Gerlach CAM-Gerlach Oct 23, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great points, thanks. I've incorporated the changes suggested via your PR, specifically:

  • Consolidated the jump opcode additions, replacements and changes (except for the NO_INTERRUPT opcode addition, which didn't appear to be closely related to the others) into a single Replaced table entry with a linked note
  • Just linked the high-level jump opcode name patterns, instead of every opcode.
  • Added a "Changed" note about the exception stack representation
  • (EXC/EG changes were already consolidated)

Beyond that, I've given this PR a major revamp based on your feedback above:

  • Converted the "Added" section back to (much shorter) bullets
  • Inlined the previous separate notes into the Replaced table
  • Combined the previous "Removed" and "Changed" sections into one, condensing the Removed items into a single bullet to take up much less space
  • Made additional textual refinements, particularly in the Changed entries, to be both more clear and more concise

In total, this retains nearly the same space consumption of the previous dense wall of flat bullets format (once allowing for the missing substantive additions, replacements and removals added here), while being much easier for the reader to quickly navigate, find what they're looking for and jump to more information in the dis docs.

Flat Bullets Initial PR Revised PR
image image image


New opcodes
-----------

* :opcode:`ASYNC_GEN_WRAP`, :opcode:`RETURN_GENERATOR` and :opcode:`SEND`,
used in generators and co-routines.

* :opcode:`COPY_FREE_VARS`,
which avoids needing special caller-side code for closures.

* :opcode:`JUMP_BACKWARD_NO_INTERRUPT`,
for use in certain loops where handling interrupts is undesirable.

* :opcode:`MAKE_CELL`, to create :ref:`cell-objects`.

* :opcode:`PREP_RERAISE_STAR`,
to handle the :ref:`new exception groups and except* <whatsnew311-pep654>`
added in :pep:`654`.

* :opcode:`PUSH_EXC_INFO`, for use in exception handlers.

* :opcode:`RESUME`, a no-op,
for internal tracing, debugging and optimization checks.


.. _whatsnew311-replaced-opcodes:
.. _whatsnew311-added-replaced-opcodes:

Added/Replaced Opcodes
----------------------
Replaced opcodes
----------------

+-------------------------------------------+------------------------------------+-----------------------------------------+
| New Opcode(s) | Replaced Opcode(s) | Notes |
+===========================================+====================================+=========================================+
| | :opcode:`ASYNC_GEN_WRAP` | | Used in generators and co-routines |
| | :opcode:`RETURN_GENERATOR` | | |
| | :opcode:`SEND` | | |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`BEFORE_WITH` | | :opcode:`!SETUP_WITH` | :keyword:`with` block setup |
| | | :opcode:`!SETUP_ASYNC_WITH` | |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`BINARY_OP` | | :opcode:`!BINARY_*` | Replaced all numeric binary/in-place |
| | | :opcode:`!INPLACE_*` | opcodes with a single opcode |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`CACHE` | | Inline cache entries [#opcode-cache]_ |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`CALL` | | :opcode:`!CALL_FUNCTION` | Decouples argument shifting for methods |
| | :opcode:`KW_NAMES` | | :opcode:`!CALL_FUNCTION_KW` | from handling of keyword arguments; |
| | :opcode:`PRECALL` | | :opcode:`!CALL_METHOD` | allows better specialization of calls |
| | :opcode:`PUSH_NULL` | | |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`CHECK_EXC_MATCH` | | :opcode:`!JUMP_IF_NOT_EXC_MATCH` | Now performs check |
| | :opcode:`CHECK_EG_MATCH` | | :opcode:`!JUMP_IF_NOT_EG_MATCH` | but doesn't jump |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`COPY` | | :opcode:`!DUP_TOP` | Stack manipulation instructions |
| | :opcode:`SWAP` | | :opcode:`!DUP_TOP_TWO` | |
| | | :opcode:`!ROT_TWO` | |
| | | :opcode:`!ROT_THREE` | |
| | | :opcode:`!ROT_FOUR` | |
| | | :opcode:`!ROT_N` | |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`COPY_FREE_VARS` | | Avoids needing special caller-side code |
| | | for closures |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`JUMP_BACKWARD_NO_INTERRUPT` | | For certain loops where |
| | | handling interrupts is undesirable |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`JUMP_BACKWARD` | | :opcode:`!JUMP_ABSOLUTE` | All jump opcodes now relative; |
| | :opcode:`POP_JUMP_BACKWARD_IF_* | | :opcode:`!POP_JUMP_IF_FALSE` | arg is offset from current instruction |
| <POP_JUMP_BACKWARD_IF_TRUE>` | | :opcode:`!POP_JUMP_IF_TRUE` | rather than absolute location |
| | :opcode:`POP_JUMP_FORWARD_IF_* | | and most op names contain direction; |
| <POP_JUMP_FORWARD_IF_TRUE>` | | added ``NONE`` & ``NOT_NONE`` variants |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`MAKE_CELL` | | Creates new :ref:`cell-objects` |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`PREP_RERAISE_STAR` | | Handles the :ref:`new exception groups |
| | | <whatsnew311-pep654>` |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`PUSH_EXC_INFO` | | For use in exception handlers |
+-------------------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`RESUME` | | No-op; performs internal tracing, |
| | | debugging and optimization checks |
+-------------------------------------------+------------------------------------+-----------------------------------------+

.. [#opcode-cache] The bytecode now contains inline cache entries,
which take the form of :opcode:`CACHE` instructions.
Many opcodes expect to be followed by an exact number of caches,
and instruct the interpreter to skip over them at runtime.
Populated caches can look like arbitrary instructions,
so great care should be taken when reading or modifying
raw, adaptive bytecode containing quickened data.
+-----------------------------------+------------------------------------+-----------------------------------------+
| New Opcode(s) | Replaced Opcode(s) | Notes |
+===================================+====================================+=========================================+
| :opcode:`BEFORE_WITH` | | :opcode:`!SETUP_WITH` | :keyword:`with` block setup |
| | | :opcode:`!SETUP_ASYNC_WITH` | |
+-----------------------------------+------------------------------------+-----------------------------------------+
| :opcode:`BINARY_OP` | | :opcode:`!BINARY_*` | Replaced all numeric binary/in-place |
| | | :opcode:`!INPLACE_*` | opcodes with a single opcode |
+-----------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`CALL` | | :opcode:`!CALL_FUNCTION` | Decouples argument shifting for methods |
| | :opcode:`KW_NAMES` | | :opcode:`!CALL_FUNCTION_KW` | from handling of keyword arguments; |
| | :opcode:`PRECALL` | | :opcode:`!CALL_METHOD` | allows better specialization of calls |
| | :opcode:`PUSH_NULL` | | |
+-----------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`CHECK_EXC_MATCH` | | :opcode:`!JUMP_IF_NOT_EXC_MATCH` | Now performs check |
| | :opcode:`CHECK_EG_MATCH` | | :opcode:`!JUMP_IF_NOT_EG_MATCH` | but doesn't jump |
+-----------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`COPY` | | :opcode:`!DUP_TOP` | Stack manipulation instructions |
| | :opcode:`SWAP` | | :opcode:`!DUP_TOP_TWO` | |
| | | :opcode:`!ROT_TWO` | |
| | | :opcode:`!ROT_THREE` | |
| | | :opcode:`!ROT_FOUR` | |
| | | :opcode:`!ROT_N` | |
+-----------------------------------+------------------------------------+-----------------------------------------+
| | :opcode:`JUMP_BACKWARD` | | :opcode:`!JUMP_ABSOLUTE` | All jump opcodes now relative; |
| | :opcode:`POP_JUMP_BACKWARD_IF_* | | :opcode:`!POP_JUMP_IF_FALSE` | arg is offset from current instruction |
| <POP_JUMP_BACKWARD_IF_TRUE>` | | :opcode:`!POP_JUMP_IF_TRUE` | rather than absolute location |
| | :opcode:`POP_JUMP_FORWARD_IF_* | | and most op names contain direction; |
| <POP_JUMP_FORWARD_IF_TRUE>` | | added ``NONE`` & ``NOT_NONE`` variants |
+-----------------------------------+------------------------------------+-----------------------------------------+


.. _whatsnew311-removed-opcodes:
Expand Down