_autoimp module

exception pyflyby._autoimp.LoadSymbolError
class pyflyby._autoimp.ScopeStack(arg, _class_delayed=None)

A stack of namespace scopes, as a tuple of dict s.

Each entry is a dict.

Ordered from most-global to most-local. Builtins are always included. Duplicates are removed.

_abc_impl = <_abc._abc_data object>
_cached_has_star_import = False
_with_new_scope(*, include_class_scopes, new_class_scope, unhide_classdef)

Return a new ScopeStack with an additional empty scope.

Parameters:

  • include_class_scopes (bool) – Whether to include previous scopes that are meant for ClassDefs.

  • new_class_scope (bool) – Whether the new scope is for a ClassDef.

  • unhide_classdef (bool) – Unhide class definitiion scope (when we enter a method)

Return type:

ScopeStack

clone_top()

Return a new ScopeStack referencing the same namespaces as self, but cloning the topmost namespace (and aliasing the others).

has_star_import()

Return whether there are any star-imports in this ScopeStack. Only relevant in AST-based static analysis mode.

Return type:

bool

merged_to_two()

Return a 2-tuple of dicts.

These can be used for functions that take a globals and locals argument, such as eval.

If there is only one entry, then return it twice.

If there are more than two entries, then create a new dict that merges the more-global ones. The most-local stack will alias the dict from the existing ScopeStack.

Return type:

tuple of (dict, dict)

class pyflyby._autoimp._ClassScope
pyflyby._autoimp._IMPORT_FAILED: Set[Any] = {}

Set of imports we’ve already attempted and failed.

class pyflyby._autoimp._MissingImportFinder(scopestack, *, find_unused_imports, parse_docstrings)

A helper class to be used only by _find_missing_imports_in_ast.

This class visits every AST node and collects symbols that require importing. A symbol requires importing if it is not already imported or otherwise defined/assigned in this scope.

For attributes like “foo.bar.baz”, we need to be more sophisticated:

Suppose the user imports “foo.bar” and then accesses “foo.bar.baz.quux”. Baz may be already available just by importing foo.bar, or it may require further import. We decide as follows. If foo.bar is not a module, then we assume whatever’s under it can’t be imported. If foo.bar is a module but does not have a ‘baz’ attribute, then it does require import.

_NewScopeCtx(include_class_scopes=False, new_class_scope=False, unhide_classdef=False, check_unused_imports=True)

Context manager that temporarily pushes a new empty namespace onto the stack of namespaces.

_UpScopeCtx()

Context manager that temporarily moves up one in the scope stack

_check_load(fullname, scopestack, lineno)

Check if the symbol needs import. (As a side effect, if the object is a _UseChecker, this will mark it as used.

TODO: It would be better to refactor symbol_needs_import so that it just returns the object it found, and we mark it as used here.)

_deferred_load_checks: list[tuple[str, ScopeStack, Optional[int]]]
_finish_deferred_load_checks()
_get_scope_info()
_lineno: Optional[int]
_remove_from_missing_imports(fullname)
_scan_node(node)
_scan_unused_imports()
_visit_Load(fullname)
_visit_Load_defered(fullname)
_visit_Load_defered_global(fullname)

Some things will be resolved in global scope later.

_visit_Load_immediate(fullname)
_visit_Store(fullname, value=None)

Visit a Store action, check for unused import and add current value to the last scope.

_visit_StoreImport(node, modulename)
_visit__all__(node)
_visit_fullname(fullname, ctx)
_visit_typecomment(typecomment)

Warning, when a type comment the node is a string, not an ast node. We also get two types of type comments:

The signature one just after a function definition :rtype: None

def foo(a):

# type: int -> None pass

And the variable annotation ones:

def foo(a #type: int

): pass

ast parse “func_type” mode only support the first one.

find_missing_imports(node)
generic_visit(node)

Generic visitor that visits all of the node’s field values, in the order declared by node._fields.

Called if no explicit visitor function exists for a node.

missing_imports: List[Tuple[Optional[int], DottedIdentifier]]
parse_docstrings: bool
scan_for_import_issues(codeblock)
scopestack: ScopeStack
unused_imports: Optional[List[Tuple[int, str]]]
visit(node)

Visit a node.

visit_Assign(node)
visit_AsyncFunctionDef(node)
visit_Attribute(node)
visit_Call(node)
visit_ClassDef(node)
visit_Constant(node)
visit_Delete(node)
visit_Dict(node)
visit_DictComp(node)
visit_ExceptHandler(node)
Return type:

None

visit_Expr(node)
visit_FunctionDef(node)
visit_GeneratorExp(node)
visit_ImportFrom(node)
visit_Lambda(node)
visit_ListComp(node)
visit_Match(node)
visit_MatchAs(node)
visit_MatchMapping(node)
visit_Module(node)
visit_Name(node)
visit_Pass(node)
visit_SetComp(node)
visit_alias(node, modulename=None)
visit_arg(node)
visit_arguments(node)
Return type:

None

visit_comprehension(node)
visit_match_case(node)
class pyflyby._autoimp._UseChecker(name, source, lineno)

An object that can check whether it was used.

lineno: int
name: str
source: str
used: bool = False
pyflyby._autoimp._find_earliest_backjump_label(bytecode)

Find the earliest target of a backward jump.

These normally represent loops.

For example, given the source code:

>>> def f():
...     if foo1():
...         foo2()
...     else:
...         foo3()
...     foo4()
...     while foo5():  # L7
...         foo6()

The earliest target of a backward jump would be the ‘while’ loop at L7, at bytecode offset 38:

>>> _find_earliest_backjump_label(f.__code__.co_code) 
38

Note that in this example there are earlier targets of jumps at bytecode offsets 20 and 28, but those are targets of _forward_ jumps, and the clients of this function care about the earliest _backward_ jump.

If there are no backward jumps, return an offset that points after the end of the bytecode.

Parameters:

bytecode (bytes) – Compiled bytecode, e.g. function.__code__.co_code.

Return type:

int

Returns:

The earliest target of a backward jump, as an offset into the bytecode.

pyflyby._autoimp._find_loads_without_stores_in_code(co, loads_without_stores)

Find global LOADs without corresponding STOREs, by disassembling code. Recursive helper for _find_missing_imports_in_code.

Parameters:

  • co (types.CodeType) – Code object, e.g. function.__code__

  • loads_without_stores (set) – Mutable set to which we add loads without stores.

Returns:

None

pyflyby._autoimp._find_missing_imports_in_ast(node, namespaces)

Find missing imports in an AST node. Helper function to find_missing_imports.

>>> node = ast.parse("import numpy; numpy.arange(x) + arange(x)")
>>> _find_missing_imports_in_ast(node, [{}])
[DottedIdentifier('arange'), DottedIdentifier('x')]
Return type:

list of DottedIdentifier

pyflyby._autoimp._find_missing_imports_in_code(co, namespaces)

Find missing imports in a code object. Helper function to find_missing_imports.

>>> f = lambda: foo.bar(x) + baz(y)
>>> [str(m) for m in _find_missing_imports_in_code(f.__code__, [{}])]
['baz', 'foo.bar', 'x', 'y']

>>> f = lambda x: (lambda: x+y)
>>> _find_missing_imports_in_code(f.__code__, [{}])
[DottedIdentifier('y')]
Return type:

list of str

pyflyby._autoimp._try_import(imp, namespace)

Try to execute an import. Import the result into the namespace namespace.

Print to stdout what we’re about to do.

Only import into namespace if we won’t clobber an existing definition.

Parameters:

  • imp (Import or str) – The import to execute, e.g. “from numpy import arange”

  • namespace (dict) – Namespace to import into.

Returns:

True on success, False on failure

pyflyby._autoimp.auto_import_symbol(fullname, namespaces, db=None, autoimported=None, post_import_hook=None)

Try to auto-import a single name.

Parameters:

  • fullname (str) – Fully-qualified module name, e.g. “sqlalchemy.orm”.

  • namespaces (list of dict, e.g. [globals()].) – Namespaces to check. Namespace[-1] is the namespace to import into.

  • db (ImportDB) – Import database to use.

  • autoimported – If not None, then a dictionary of identifiers already attempted. auto_import will not attempt to auto-import symbols already in this dictionary, and will add attempted symbols to this dictionary, with value True if the autoimport succeeded, or False if the autoimport did not succeed.

  • post_import_hook (callable) – A callable that is invoked if an import was successfully made. It is invoked with the Import object representing the successful import

Return type:

bool

Returns:

True if the symbol was already in the namespace, or the auto-import succeeded; False if the auto-import failed.

pyflyby._autoimp.clear_failed_imports_cache()

Clear the cache of previously failed imports.

pyflyby._autoimp.get_known_import(fullname, db=None)

Get the deepest known import.

For example, suppose:

  • The user accessed “foo.bar.baz”,

  • We know imports for “foo”, “foo.bar”, and “foo.bar.quux”.

Then we return “import foo.bar”.

Parameters:

fullname (DottedIdentifier) – Fully-qualified name, such as “scipy.interpolate”

pyflyby._autoimp.load_symbol(fullname, namespaces, autoimport=False, db=None, autoimported=None)

Load the symbol fullname.

>>> import os
>>> load_symbol("os.path.join.__name__", {"os": os})
'join'

>>> load_symbol("os.path.join.asdf", {"os": os})
Traceback (most recent call last):
...
pyflyby._autoimp.LoadSymbolError: os.path.join.asdf: AttributeError: 'function' object has no attribute 'asdf'

>>> load_symbol("os.path.join", {})
Traceback (most recent call last):
...
pyflyby._autoimp.LoadSymbolError: os.path.join: NameError: os
Parameters:

  • fullname (str) – Fully-qualified symbol name, e.g. “os.path.join”.

  • namespaces (dict or list of dict) – Namespaces to check.

  • autoimport – If False (default), the symbol must already be imported. If True, then auto-import the symbol first.

  • db (ImportDB) – Import database to use when autoimport=True.

  • autoimported – If not None, then a dictionary of identifiers already attempted. auto_import will not attempt to auto-import symbols already in this dictionary, and will add attempted symbols to this dictionary, with value True if the autoimport succeeded, or False if the autoimport did not succeed.

Returns:

Object.

Raises:

LoadSymbolError – Object was not found or there was another exception.

pyflyby._autoimp.scan_for_import_issues(codeblock, find_unused_imports=True, parse_docstrings=False)

Find missing and unused imports, by lineno.

>>> arg = "import numpy, aa.bb as cc\nnumpy.arange(x)\narange(x)"
>>> missing, unused = scan_for_import_issues(arg)
>>> missing
[(2, DottedIdentifier('x')), (3, DottedIdentifier('arange')), (3, DottedIdentifier('x'))]
>>> unused
[(1, Import('from aa import bb as cc'))]
Parameters:

parse_docstrings (bool) –

Whether to parse docstrings. Compare the following examples. When parse_docstrings=True, ‘bar’ is not considered unused because there is a string that references it in braces:

>>> scan_for_import_issues("import foo as bar, baz\n'{bar}'\n")
([], [(1, Import('import baz')), (1, Import('import foo as bar'))])
>>> scan_for_import_issues("import foo as bar, baz\n'{bar}'\n", parse_docstrings=True)
([], [(1, Import('import baz'))])

pyflyby._autoimp.symbol_needs_import(fullname, namespaces)

Return whether fullname is a symbol that needs to be imported, given the current namespace scopes.

A symbol needs importing if it is not previously imported or otherwise assigned. namespaces normally includes builtins and globals as well as symbols imported/assigned locally within the scope.

If the user requested “foo.bar.baz”, and we see that “foo.bar” exists and is not a module, we assume nothing under foo.bar needs import. This is intentional because (1) the import would not match what is already in the namespace, and (2) we don’t want to do call getattr(foo.bar, “baz”), since that could invoke code that is slow or has side effects.

Parameters:

  • fullname (DottedIdentifier) – Fully-qualified symbol name, e.g. “os.path.join”.

  • namespaces (list of dict) – Stack of namespaces to search for existing items.

Return type:

bool

Returns:

True if fullname needs import, else False

pyflyby._autoimp.take_arg(op)