gh-127303: Add docs for token.EXACT_TOKEN_TYPES

[lubaskinc0de]

• Issue: Document token.EXACT_TOKEN_TYPES #127303

---

📚 Documentation preview 📚: https://cpython-previews--127304.org.readthedocs.build/

[cpython-cla-bot]

All commit authors signed the Contributor License Agreement.

[Eclips4]

LGTM. Ilya and I checked that it was added in version 3.8. Although I'm a bit unsure about the wording, I think it could be improved, but neither I nor Ilya are native English speakers :(

[terryjreedy]

Now that it is documented, EXACT_TOKEN_TYPES needs to be added to __all__ at the top of token.py. In particular, line 4, which should then be wrapped to 2 lines.

__all__ = ['tok_name', 'ISTERMINAL', 'ISNONTERMINAL', 'ISEOF',
           'EXACT_TOKEN_TYPES']

EDIT: The complication is that token.py is auto-generated by Tools/build/generate_token.py, so it is the latter that needs revision. In particular, expand line 229 of that file to 2 lines.

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[lubaskinc0de]

I have made the requested changes; please review again

[bedevere-app]

Thanks for making the requested changes!

@terryjreedy, @Eclips4: please review the changes made to this pull request.

[picnixz]

While the attribute already existed its scope was not specifically public. Now we can really stress that it's public and documented.

[picnixz]

Can we have two blank lines here? (as above)

[Eclips4]

LGTM.

[miss-islington-app]

Thanks @lubaskinc0de for the PR, and @terryjreedy for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-127390 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-127391 is a backport of this pull request to the 3.12 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 MacOS Intel NoGIL 3.13 has failed when building commit 4cba0e6.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1449/builds/718) and take a look at the build logs.
4. Check if the failure is related to this commit (4cba0e6) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1449/builds/718

Failed tests:

• test_threading

Failed subtests:

• test_start_new_thread_failed - test.test_threading.ThreadTests.test_start_new_thread_failed

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/ec2-user/buildbot/buildarea/3.13.itamaro-macos-intel-aws.nogil/build/Lib/test/test_threading.py", line 1205, in test_start_new_thread_failed
    self.assertEqual(out, b'ok')
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^
AssertionError: b"skip\nshouldn't be printed" != b'ok'