_livepatch module

livepatch/xreload: Alternative to reload().

xreload performs a “live patch” of the modules/classes/functions/etc that have already been loaded in memory. It does so by executing the module in a scratch namespace, and then patching classes, methods and functions in-place. New objects are copied into the target namespace.

This addresses cases where one module imported functions from another module.

For example, suppose m1.py contains:

from m2 import foo
def print_foo():
    return foo()

and m2.py contains:

def foo():
    return 42

If you edit m2.py and modify foo, then reload(m2) on its own would not do what you want. You would also need to reload(m1) after reload(m2). This is because the built-in reload affects the module being reloaded, but references to the old module remain. On the other hand, xreload() patches the existing m2.foo, so that live references to it are updated.

In table form:

Undesired effect:  reload(m2)
Undesired effect:  reload(m1);  reload(m2)
Desired effect:    reload(m2);  reload(m1)

Desired effect:   xreload(m2)
Desired effect:   xreload(m1); xreload(m2)
Desired effect:   xreload(m2); xreload(m1)

Even with just two modules, we can see that xreload() is an improvement. When working with a large set of interdependent modules, it becomes infeasible to know the precise sequence of reload() calls that would be necessary. xreload() really shines in that case.

This implementation of xreload() was originally based the following mailing-list post by Guido van Rossum:

Customizing behavior

If a class/function/module/etc has an attribute __livepatch__, then this function is called instead of performing the regular livepatch mechanism.

The __livepatch__() function is called with the following arguments:

  • old : The object to be updated with contents of new

  • new : The object whose contents to put into old

  • do_livepatch: A function that can be called to do the standard

    livepatch, replacing the contents of old with new. If it’s not possible to livepatch old, it returns new. The do_livepatch function takes no arguments. Calling the do_livepatch function is roughly equivalent to calling pyflyby.livepatch(old, new, modname=modname, heed_hook=False).

  • modnameThe module currently being updated. Recursively called

    updates should keep track of the module being updated to avoid touching other modules.

These arguments are matched by name and are passed only if the __livepatch__ function is declared to take such named arguments or it takes **kwargs. If the __livepatch__ function takes **kwargs, it should ignore unknown arguments, in case new parameters are added in the future.

If the object being updated is an object instance, and __livepatch__ is a method, then the function is bound to the new object, i.e. the self parameter is the same as new.

If the __livepatch__ function successfully patched the old object, then it should return old. If it is unable to patch, it should return new.

Examples

By default, any attributes on an existing function are updated with ones from the new function. If you want a memoized function to keep its cache across xreload, you could implement that like this:

def memoize(function):
    cache = {}
    def wrapped_fn(*args):
        try:
            return cache[args]
        except KeyError:
            result = function(*args)
            cache[args] = result
            return result
    wrapped_fn.cache = cache
    def my_livepatch(old, new, do_livepatch):
        keep_cache = dict(old.cache)
        result = do_livepatch()
        result.cache.update(keep_cache)
        return result
    wrapped_fn.__livepatch__ = my_livepatch
    return wrapped_fn

XXX change example b/c cache is already cleared by default XXX maybe global cache

class MyObj(…):
def __livepatch__(self, old):

self.__dict__.update(old.__dict__) return self

class MyObj(…):
def __init__(self):

self._my_cache = {}

def __livepatch__(self, old, do_livepatch):

keep_cache = dict(old._my_cache) result = do_livepatch() result._my_cache.update(keep_cache) return result

XXX test

exception pyflyby._livepatch.UnknownModuleError
pyflyby._livepatch._format_age(t)
pyflyby._livepatch._get_definition_module(obj)

Get the name of the module that an object is defined in, or None if unknown.

For classes and functions, this returns the __module__ attribute.

For object instances, this returns None, ignoring the __module__ attribute. The reason is that the __module__ attribute on an instance just gives the module that the class was defined in, which is not necessarily the module where the instance was constructed.

Return type:

str

pyflyby._livepatch._get_module_py_file(module)
pyflyby._livepatch._interpret_module(arg)
pyflyby._livepatch._livepatch__class(oldclass, newclass, modname, cache, visit_stack)

Livepatch a class.

This is similar to _livepatch__dict(oldclass.__dict__, newclass.__dict__). However, we can’t just operate on the dict, because class dictionaries are special objects that don’t allow setitem, even though we can setattr on the class.

pyflyby._livepatch._livepatch__dict(old_dict, new_dict, modname, cache, visit_stack)

Livepatch a dict.

pyflyby._livepatch._livepatch__function(old_func, new_func, modname, cache, visit_stack)

Livepatch a function.

pyflyby._livepatch._livepatch__method(old_method, new_method, modname, cache, visit_stack)

Livepatch a method.

pyflyby._livepatch._livepatch__module(old_mod, new_mod, modname, cache, visit_stack)

Livepatch a module.

pyflyby._livepatch._livepatch__object(oldobj, newobj, modname, cache, visit_stack)

Livepatch a general object.

pyflyby._livepatch._livepatch__setattr(oldobj, newobj, name, modname, cache, visit_stack)

Livepatch something via setattr, i.e.:

oldobj.{name} = livepatch(oldobj.{name}, newobj.{name}, ...)
pyflyby._livepatch._xreload_module(module, filename, force=False)

Reload a module in place, using livepatch.

Parameters:

  • module (ModuleType) – Module to reload.

  • force – Whether to reload even if the module has not been modified since the previous load. If False, then do nothing. If True, then reload.