_parse module

class pyflyby._parse.AnnotatedAst
col_offset: int
endpos: FilePos
flags: str
lieneno: int
s: str
source_flags: CompilerFlags
startpos: FilePos
text: FileText
value: AnnotatedAst
class pyflyby._parse.AnnotatedModule
source_flags: CompilerFlags
class pyflyby._parse.AstNodeContext(parent, field, index)
_asdict()

Return a new dict which maps field names to their values.

_field_defaults = {}
_fields = ('parent', 'field', 'index')
classmethod _make(iterable)

Make a new AstNodeContext object from a sequence or iterable

_replace(**kwds)

Return a new AstNodeContext object replacing specified fields with new values

field

Alias for field number 1

index

Alias for field number 2

parent

Alias for field number 0

class pyflyby._parse.IgnoreOptionsDocTestParser
_find_options(source, name, lineno)

Return a dictionary containing option overrides extracted from option directives in the given source string.

name is the string’s name, and lineno is the line number where the example starts; both are used for error messages.

class pyflyby._parse._DummyAst_Node
pyflyby._parse._annotate_ast_nodes(ast_node)
Annotate AST with:

  • startpos and endpos

  • [disabled for now: context as AstNodeContext ]

Parameters:

ast_node (AST) – AST node returned by _parse_ast_nodes

Return type:

AnnotatedAst

Returns:

None

pyflyby._parse._annotate_ast_startpos(ast_node, parent_ast_node, minpos, text, flags)

Annotate ast_node. Set ast_node.startpos to the starting position of the node within text.

For “typical” nodes, i.e. those other than multiline strings, this is simply FilePos(ast_node.lineno, ast_node.col_offset+1), but taking text.startpos into account.

For multiline string nodes, this function works by trying to parse all possible subranges of lines until finding the range that is syntactically valid and matches value. The candidate range is text[min_start_lineno:lineno+text.startpos.lineno+1].

This function is unfortunately necessary because of a flaw in the output produced by the Python built-in parser. For some crazy reason, the ast_node.lineno attribute represents something different for multiline string literals versus all other statements. For multiline string literal nodes and statements that are just a string expression (or more generally, nodes where the first descendant leaf node is a multiline string literal), the compiler attaches the ending line number as the value of the lineno attribute. For all other than AST nodes, the compiler attaches the starting line number as the value of the lineno attribute. This means e.g. the statement “’’’foonbar’’’” has a lineno value of 2, but the statement “x=’’’foonbar’’’” has a lineno value of 1.

Parameters:

  • minpos (FilePos) – Earliest position to check, in the number space of text.

  • text (FileText) – Source text that was used to parse the AST, whose startpos should be used in interpreting ast_node.lineno (which always starts at 1 for the subset that was parsed).

  • flags (CompilerFlags) – Compiler flags to use when re-compiling code.

Return type:

bool

Returns:

True if this node is a multiline string literal or the first child is such a node (recursively); False otherwise.

Raises:

ValueError – Could not find the starting line number.

pyflyby._parse._ast_str_literal_value(node)
pyflyby._parse._flags_to_try(source, flags, auto_flags, mode)

Flags to try for auto_flags.

If auto_flags is False, then only yield flags. If auto_flags is True, then yield flags and flags ^ print_function.

pyflyby._parse._flatten_ast_nodes(arg)
pyflyby._parse._is_ast_bytes(node)

utility function that test if node is an ast.Str in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:

bool

pyflyby._parse._is_ast_str(node)

utility function that test if node is an ast.Str in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:

bool

pyflyby._parse._is_ast_str_or_byte(node)

utility function that test if node is an ast.Str|ast.Bytes in Python < 3.12, and if it is a ast.Constant, with node.value being a str in newer version.

Return type:

bool

pyflyby._parse._is_comment_or_blank(line, /)

Returns whether a line of python code contains only a comment is blank.

>>> _is_comment_or_blank("foo\n")
False

>>> _is_comment_or_blank("  # blah\n")
True
pyflyby._parse._iter_child_nodes_in_order(node)

Yield all direct child nodes of node, that is, all fields that are nodes and all items of fields that are lists of nodes.

_iter_child_nodes_in_order yields nodes in the same order that they appear in the source.

ast.iter_child_nodes does the same thing, but not in source order. e.g. for Dict s, it yields all key nodes before all value nodes.

pyflyby._parse._iter_child_nodes_in_order_internal_1(node)
pyflyby._parse._parse_ast_nodes(text, flags, auto_flags, mode)

Parse a block of lines into an AST.

Also annotate input_flags, source_flags, and flags on the resulting ast node.

Parameters:

  • auto_flags (bool) – Whether to guess different flags if text can’t be parsed with flags.

  • mode (str) – Compilation mode: “exec”, “single”, or “eval”.

Return type:

ast.Module

pyflyby._parse._split_code_lines(ast_nodes, text)

Split the given ast_nodes and corresponding text by code/noncode statement.

Yield tuples of (nodes, subtext). nodes is a list of ast.AST nodes, length 0 or 1; subtext is a FileText sliced from text.

FileText(…))} for code lines and (None, FileText(...)) for non-code lines (comments and blanks).

pyflyby._parse._test_parse_string_literal(text, flags)

Attempt to parse text. If it parses cleanly to a single string literal, return its value. Otherwise return None.

>>> _test_parse_string_literal(r'"foo\n" r"\nbar"', None)
'foo\n\\nbar'
pyflyby._parse._walk_ast_nodes_in_order(node)

Recursively yield all child nodes of node, in the same order that the node appears in the source.

ast.walk does the same thing, but yields nodes in an arbitrary order.

pyflyby._parse.infer_compile_mode(arg)

Infer the mode needed to compile arg.

Return type:

str