_parse module

Python parsing utilities for pyflyby.

This module provides AST parsing and manipulation functionality. It includes PythonBlock and PythonStatement classes for working with Python code.

On newer versions of Python it is suggested to run

python -m pyflyby.check_parse <path/to/cpython>

This will parse all Python files in the given path and report any parsing issues

class pyflyby._parse.AnnotatedAst

body: Any

col_offset: int

endpos: FilePos

flags: CompilerFlags

lieneno: int

s: str

source_flags: CompilerFlags

startpos: FilePos

text: FileText

value: AnnotatedAst

class pyflyby._parse.AnnotatedModule

source_flags: CompilerFlags

class pyflyby._parse.IgnoreOptionsDocTestParser

_find_options(source, name, lineno)

Return a dictionary containing option overrides extracted from option directives in the given source string.

name is the string’s name, and lineno is the line number where the example starts; both are used for error messages.

Return type:: Dict[Any, Any]

class pyflyby._parse._DummyAst_Node

startpos: FilePos

pyflyby._parse._annotate_ast_nodes(ast_node)

Annotate every node in the tree rooted at ast_node with startpos and endpos attributes giving the node’s start and end FilePos within the source text.

Since Python 3.8 the built-in parser reports correct lineno/ col_offset and end_lineno/end_col_offset for every node – including multiline string literals, which historically misreported their position as the ending line with a col_offset of -1 – so positions can be read directly off each node rather than reconstructed by re-parsing candidate sub-ranges.

Parameters:: ast_node (AST) – AST node returned by _parse_ast_nodes
Return type:: AnnotatedAst ast.AST

pyflyby._parse._ast_str_literal_value(node)

Return type:: Any

pyflyby._parse._is_ast_bytes(node)

utility function that test if node is an ast.Str in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:: bool

pyflyby._parse._is_ast_str(node)

utility function that test if node is an ast.Str in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:: bool

pyflyby._parse._is_ast_str_or_byte(node)

utility function that test if node is an ast.Str|ast.Bytes in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:: bool

pyflyby._parse._is_comment_or_blank(line, /)

Returns whether a line of python code contains only a comment is blank.

>>> _is_comment_or_blank("foo\n")
False

>>> _is_comment_or_blank("  # blah\n")
True

Return type:: bool

pyflyby._parse._parse_ast_nodes(text, flags, mode)

Parse a block of lines into an AST.

Also annotate input_flags, source_flags, and flags on the resulting ast node.

Parameters:: mode (str) – Compilation mode: “exec”, “single”, or “eval”.
Return type:: AnnotatedModule ast.Module

pyflyby._parse._split_code_lines(ast_nodes, text)

Split the given ast_nodes and corresponding text by code/noncode statement.

Yield tuples of (nodes, subtext). nodes is a list of ast.AST nodes, length 0 or 1; subtext is a FileText sliced from text.

FileText(…))} for code lines and (None, FileText(...)) for non-code lines (comments and blanks).

Return type:: Iterator[Tuple[List[Any], Any]]

pyflyby._parse._test_parse_string_literal(text, flags)

Attempt to parse text. If it parses cleanly to a single string literal, return its value. Otherwise return None.

>>> _test_parse_string_literal(r'"foo\n" r"\nbar"', None)
'foo\n\\nbar'

Return type:: Any

pyflyby._parse.infer_compile_mode(arg)

Infer the mode needed to compile arg.

Return type:: Literal['exec', 'eval', 'single'] str