_parse module

class pyflyby._parse.AnnotatedAst

col_offset: int

endpos: FilePos

flags: str

lieneno: int

s: str

source_flags: CompilerFlags

startpos: FilePos

text: FileText

value: AnnotatedAst

class pyflyby._parse.AnnotatedModule

source_flags: CompilerFlags

class pyflyby._parse.AstNodeContext(parent, field, index)

_asdict(): Return a new dict which maps field names to their values.

_field_defaults = {}

_fields = ('parent', 'field', 'index')

classmethod _make(iterable): Make a new AstNodeContext object from a sequence or iterable

_replace(**kwds): Return a new AstNodeContext object replacing specified fields with new values

field: Alias for field number 1

index: Alias for field number 2

parent: Alias for field number 0

class pyflyby._parse.IgnoreOptionsDocTestParser

_find_options(source, name, lineno)

Return a dictionary containing option overrides extracted from option directives in the given source string.

name is the string’s name, and lineno is the line number where the example starts; both are used for error messages.

class pyflyby._parse._DummyAst_Node

pyflyby._parse._annotate_ast_context(ast_node)

Recursively annotate context on ast nodes, setting context to a AstNodeContext named tuple with values (parent, field, index). Each aast_node satisfies parent.<field>[<index>] is ast_node.

For non-list fields, the index part is None.

pyflyby._parse._annotate_ast_nodes(ast_node)

Annotate AST with:

startpos and endpos
[disabled for now: context as AstNodeContext ]

Parameters:: ast_node (AST) – AST node returned by _parse_ast_nodes
Return type:: AnnotatedAst
Returns:: None

pyflyby._parse._annotate_ast_startpos(ast_node, parent_ast_node, minpos, text, flags)

Annotate ast_node. Set ast_node.startpos to the starting position of the node within text.

For “typical” nodes, i.e. those other than multiline strings, this is simply FilePos(ast_node.lineno, ast_node.col_offset+1), but taking text.startpos into account.

For multiline string nodes, this function works by trying to parse all possible subranges of lines until finding the range that is syntactically valid and matches value. The candidate range is text[min_start_lineno:lineno+text.startpos.lineno+1].

This function is unfortunately necessary because of a flaw in the output produced by the Python built-in parser. For some crazy reason, the ast_node.lineno attribute represents something different for multiline string literals versus all other statements. For multiline string literal nodes and statements that are just a string expression (or more generally, nodes where the first descendant leaf node is a multiline string literal), the compiler attaches the ending line number as the value of the lineno attribute. For all other than AST nodes, the compiler attaches the starting line number as the value of the lineno attribute. This means e.g. the statement “’’’foonbar’’’” has a lineno value of 2, but the statement “x=’’’foonbar’’’” has a lineno value of 1.

Parameters:

minpos (FilePos) – Earliest position to check, in the number space of text.
text (FileText) – Source text that was used to parse the AST, whose startpos should be used in interpreting ast_node.lineno (which always starts at 1 for the subset that was parsed).
flags (CompilerFlags) – Compiler flags to use when re-compiling code.

Return type:

bool

Returns:

True if this node is a multiline string literal or the first child is such a node (recursively); False otherwise.

Raises:

ValueError – Could not find the starting line number.

pyflyby._parse._ast_node_is_in_docstring_position(ast_node)

Given a Str AST node, return whether its position within the AST makes it eligible as a docstring.

The main way a Str can be a docstring is if it is a standalone string at the beginning of a Module, FunctionDef, AsyncFucntionDef or ClassDef.

We also support variable docstrings per Epydoc:

If a variable assignment statement is immediately followed by a bare string literal, then that assignment is treated as a docstring for that variable.

Parameters:: ast_node (ast.Str) – AST node that has been annotated by _annotate_ast_nodes.
Return type:: bool
Returns:: Whether this string ast node is in docstring position.

pyflyby._parse._ast_str_literal_value(node)

pyflyby._parse._flags_to_try(source, flags, auto_flags, mode)

Flags to try for auto_flags.

If auto_flags is False, then only yield flags. If auto_flags is True, then yield flags and flags ^ print_function.

pyflyby._parse._flatten_ast_nodes(arg)

pyflyby._parse._is_ast_bytes(node)

utility function that test if node is an ast.Str in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:: bool

pyflyby._parse._is_ast_str(node)

utility function that test if node is an ast.Str in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:: bool

pyflyby._parse._is_ast_str_or_byte(node)

utility function that test if node is an ast.Str|ast.Bytes in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:: bool

pyflyby._parse._is_comment_or_blank(line, /)

Returns whether a line of python code contains only a comment is blank.

>>> _is_comment_or_blank("foo\n")
False

>>> _is_comment_or_blank("  # blah\n")
True

pyflyby._parse._iter_child_nodes_in_order(node)

Yield all direct child nodes of node, that is, all fields that are nodes and all items of fields that are lists of nodes.

_iter_child_nodes_in_order yields nodes in the same order that they appear in the source.

ast.iter_child_nodes does the same thing, but not in source order. e.g. for Dict s, it yields all key nodes before all value nodes.

pyflyby._parse._iter_child_nodes_in_order_internal_1(node)

pyflyby._parse._parse_ast_nodes(text, flags, auto_flags, mode)

Parse a block of lines into an AST.

Also annotate input_flags, source_flags, and flags on the resulting ast node.

Parameters:

auto_flags (bool) – Whether to guess different flags if text can’t be parsed with flags.
mode (str) – Compilation mode: “exec”, “single”, or “eval”.

Return type:

ast.Module

pyflyby._parse._split_code_lines(ast_nodes, text)

Split the given ast_nodes and corresponding text by code/noncode statement.

Yield tuples of (nodes, subtext). nodes is a list of ast.AST nodes, length 0 or 1; subtext is a FileText sliced from text.

FileText(…))} for code lines and (None, FileText(...)) for non-code lines (comments and blanks).

pyflyby._parse._test_parse_string_literal(text, flags)

Attempt to parse text. If it parses cleanly to a single string literal, return its value. Otherwise return None.

>>> _test_parse_string_literal(r'"foo\n" r"\nbar"', None)
'foo\n\\nbar'

pyflyby._parse._walk_ast_nodes_in_order(node)

Recursively yield all child nodes of node, in the same order that the node appears in the source.

ast.walk does the same thing, but yields nodes in an arbitrary order.

pyflyby._parse.infer_compile_mode(arg)

Infer the mode needed to compile arg.

Return type:: str