Skip to content

Only honor second-line encoding comments after blank or comment lines - #1465

Open
ychampion wants to merge 1 commit into
python-babel:masterfrom
ychampion:fix/second-line-encoding-declaration
Open

ychampion wants to merge 1 commit into
python-babel:masterfrom
ychampion:fix/second-line-encoding-declaration

Conversation

@ychampion

Copy link
Copy Markdown

Fixes #1463

Summary

babel.util.parse_encoding() previously ran ast.parse() on line 1 to decide whether line 2 could contain a magic encoding comment. Single-line executable code (such as x = 1, """docstring""", or ;) succeeded under ast.parse(), causing Babel to honor a second-line # coding: comment that Python itself ignores per PEP 263 (and to raise a spurious SyntaxError if the file also began with a UTF-8 BOM).

This change replaces the ast.parse() check with _PYTHON_BLANK_OR_COMMENT_re (rb'[ \t\f]*(?:[#\r\n]|$)', matching tokenize.blank_re), so line 2 is checked only when line 1 is blank or a comment.

Changes

  • babel/util.py: Match line 1 against _PYTHON_BLANK_OR_COMMENT_re before reading line 2 for PYTHON_MAGIC_COMMENT_re in parse_encoding().
  • tests/test_util.py: Add unit tests covering valid blank/shebang/comment first lines, ignored second-line declarations after executable code/docstrings/;/\x0b, UTF-8 BOM + code on line 1, and stream position preservation.
  • tests/messages/test_extract_python.py: Add extraction tests verifying second-line # coding: behavior after code, blank/comment lines, and a UTF-8 BOM.

Verification

  • python -X utf8 -m pytest tests/test_util.py tests/messages/test_extract.py tests/messages/test_extract_python.py: 67 passed
  • python -X utf8 -m pytest (with imported CLDR data): 8225 passed, 5 skipped, 2 xfailed
  • ruff check .: all checks passed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python extraction honors a second-line encoding declaration after executable code

1 participant