CocoIsBuggy A small flapping bug

Unspeakable Python Metaprogramming

A short guide on how to commit devious python metaprogramming crimes and get away with it.

There are typically three reasons why you might write python code that can change its execution context while it is running. The first reason to do so happens trivially, and it occurs because you don’t know any better.

Metaprogramming By Accident

age = input("How old are you?")
print("you will be", eval(age) + 2, "in to years time")

I see this kind of thing often when people are new to python. They want the ‘24’ that the user enters to be used interchangeably with the 24 that they would type into their code editor. People don’t usually consider it to be metaprogramming but I’m going to claim a technicality. The programmer may not know what behaviors they have introduced, but they have written a program that can alter its own structure and behavior when it runs.

Metaprogramming For Evil

The other reason to use metaprogramming is when you hate your coworkers and you would like to prevent them from reasoning about the code that they write. Metaprogramming with a heart full of malice might be fun, but I encourage you to look within and restrain your perverted impulses.

import colins_helpful_logger
import math

print(math.sqrt(9))  # Outputs: 42

bad news friend, even though you did not call ANY functionality from colin’s helpful logger, it contained the following

import sys
import types

class EvilImporter:
    def find_spec(self, fullname, path, target=None):
        if fullname == "math":
            fake_math = types.ModuleType("math")
            fake_math.sqrt = lambda x: 42
            sys.modules[fullname] = fake_math
        return None

sys.meta_path.insert(0, EvilImporter())

This is an obvious way to get caught, I assure you that you can think of better ways to maintain plausible deniability. The problem with evil code of this kind is that it changes your program only at runtime, you get very few hints as to where the behavior comes from and there is no concrete static analysis that gets you from the misbehaving code to its runtime override.

Metaprogramming For Good

The final reason to use metaprogramming is when you love and care for your coworkers, and you want to protect them from themselves.

Consider this injector that I have used before when working in a project that has LOTS of concurrent worker functions. I want to force any functions in a certain context to be reliably cancelable at the cost of speed and timing reliability.

I can reasonably enforce that contract by changing the code written by users of my API. I love my co-workers, and so I will automatically fix their code whenever they try and run it.

class InterleaveHookInjector(ast.NodeTransformer):
    def body_sleeper_logic(
        self, node: ast.While | ast.For | ast.AsyncFor | ast.AsyncFunctionDef
    ) -> ast.AST:
        for idx, expr in enumerate([x for x in node.body]):
            with contextlib.suppress(IndexError):
                if ast.unparse(node.body[idx + 1]) == "await asyncio.sleep(0)":
                    continue

            node.body.insert(
                idx + idx + 1,
                ast.Expr(
                    ast.Await(
                        ast.Call(
                            func=ast.Attribute(
                                value=ast.Name(id="asyncio", ctx=ast.Load()),
                                attr="sleep",
                                ctx=ast.Load(),
                            ),
                            args=[
                                ast.Constant(0),
                            ],
                        ),
                    ),
                ),
            )

            ast.fix_missing_locations(node)

        self.generic_visit(node)
        return node

    def visit_AsyncFunctionDef(self, node: ast.AsyncFunctionDef) -> ast.AST:
        for idx, dec in enumerate([x for x in node.decorator_list]):
            if isinstance(dec, ast.Name) and dec.id == interleave.__name__:
                node.decorator_list.pop(idx)
                ast.fix_missing_locations(node)

        return self.body_sleeper_logic(node)

    def visit_While(self, node: ast.While) -> ast.AST:
        return self.body_sleeper_logic(node)

    def visit_For(self, node: ast.For) -> ast.AST:
        return self.body_sleeper_logic(node)

    def visit_AsyncFor(self, node: ast.AsyncFor) -> ast.AST:
        return self.body_sleeper_logic(node)

This effectively takes some async function like

@interleave
async def interruptable():
    start = time.monotonic()
    trip = False

    while time.monotonic() - start < 1:
        trip = not trip
        for x in range(4):
            print(trip)

And will unwind the syntax tree, alter it to yield after every expression, and then wind it back up.

async def interruptable():
    start = time.monotonic()
    await asyncio.sleep(0)
    trip = False
    await asyncio.sleep(0)
    while time.monotonic() - start < 1:
        trip = not trip
        await asyncio.sleep(0)
        for x in range(4):
            print(trip)
            await asyncio.sleep(0)
        await asyncio.sleep(0)

The @interleave decorator is responsible for this runtime-change to the code.

This has a relatively small performance overhead but gives my subsystem a much greater chance at running in the way I intend it to.

There is no practical difference between this and the evil metaprogramming, and usually it is fairly likely to end up as a paving stone on the road to hell.

With great power comes great responsibility.