Building a self-extending AI agent: how skull's skill system works

The idea

Most agent frameworks give a model a fixed toolbox: read this, write that, search this. skull, a terminal AI agent I built for a self-hosted Qwen model, does something a step further — when no existing tool solves a task, the model writes a new Python function itself, and it becomes a permanent, callable tool from that point on, in that session and every future one.

A skill is nothing exotic: a plain Python file with a top-level run(**kwargs) function, plus a SKILL.md describing what it does. create_skill writes both to skills/<name>/run.py and skills/<name>/SKILL.md, and registers the skill's name, description, and JSON-schema parameters in a lightweight skills/index.json so the tool list can be assembled without re-reading every skill file from disk on every turn.

Overwriting used to have no undo

The first version of this had an obvious hole: calling create_skill again with an existing name just overwrote run.py. If the rewrite was worse than the original — or just broken — the only working copy was gone.

The fix is version archiving. Before a skill's files get overwritten, _archive_current_version copies the current run.py and SKILL.md into skills/<name>/versions/<n>/, bounded to the 5 most recent versions:

def _archive_current_version(name: str, entry: dict) -> None:
    skill_dir = _skill_dir(name)
    run_py = skill_dir / "run.py"
    if not run_py.exists():
        return
    version_num = _next_version_number(name)
    version_dir = _versions_dir(name) / str(version_num)
    version_dir.mkdir(parents=True, exist_ok=True)
    shutil.copy2(run_py, version_dir / "run.py")
    # ...prune anything past MAX_VERSIONS_KEPT

rollback_skill restores an archived version as the live one — and it archives whatever was live first, so a rollback is itself undoable with another rollback. There's also a safety net inside create_skill: if the new code fails to even import, and this was an overwrite (not a brand-new skill), the just-archived version is restored automatically rather than leaving broken code live under the skill's name.

The stale-bytecode bug

A subtler bug showed up during actual use: after overwriting a skill with materially different logic, calling it kept returning the old result. The cause was Python's default import caching — every skill is always imported under the exact same module name (skills.<name>.run), and the default SourceFileLoader writes a .pyc next to run.py. Since the module name never changes even though the file's content does on every create_skill call, Python could serve stale compiled bytecode instead of the new source.

The fix disables bytecode writing entirely for skill imports:

loader = importlib.machinery.SourceFileLoader(f"skills.{name}.run", str(path))
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.set_data = lambda *a, **k: None  # never write a .pyc for skill code
loader.exec_module(module)

It's a small patch, but it's the kind of bug that's invisible in a quick manual test and only shows up once you're actually iterating on a skill's code across several create_skill calls in the same run.

Skills calling skills

call_skill lets one skill's code call another by name instead of duplicating logic:

from skull.tools.skill_composition import call_skill

def run(**kwargs):
    celsius = call_skill("fahrenheit_to_celsius", fahrenheit=98.6)["celsius"]
    ...

It raises a proper SkillError on failure instead of returning the {"error": ...} dict shape the model-facing run_skill tool uses — that shape is meant for the model to read, not for skill code to branch on with if "error" in outcome. call_skill also explicitly checks for a truncated result (skull caps every skill's return value at 40,000 characters, same ceiling every other content-returning tool enforces) and raises a clear SkillError rather than letting a bare KeyError on outcome["result"] surface with no context about which skill or why.

Proving a skill is safe, without running it

The most interesting piece is skill_analysis.py's static classifier, used to decide whether a skill is safe to expose in "plan mode" — a restricted mode where every mutating tool is withheld from the model's tool list entirely.

The obvious approach — a blacklist of dangerous calls (subprocess, os.remove, etc.) — turns out to be provably unsafe against skull's own real skills. One existing skill's actual file write happens inside a string that gets exec'd in a subprocess: the write itself never appears in that file's AST at all, just as string content. Once a skill imports subprocess, eval, exec, compile, or importlib, there's no way to prove what it actually does by reading the outer file's syntax tree — so the classifier treats any of those as mutating, full stop, no further analysis.

The classifier is a whitelist instead: a skill is read_only only if its entire AST can be proven to call nothing but an explicit, narrow set of safe (module, function) pairs — math.*, json.dumps/loads, re.*, shutil.disk_usage specifically (not the whole shutil module, which also contains rmtree), and a short list of read-only string/dict/list methods. Anything the analyzer can't statically resolve — a call through a subscript, a lambda, an arbitrary expression — defaults to mutating too:

def visit_Call(self, node):
    func_name, is_bare_method = self._resolve_call_name(node.func)
    if func_name is None:
        self._mark_mutating()  # can't prove it's safe
    elif func_name in DYNAMIC_EXECUTION_NAMES:
        self._mark_mutating()
    elif is_bare_method:
        if func_name not in SAFE_INSTANCE_METHODS:
            self._mark_mutating()
    elif not self._is_whitelisted(func_name):
        self._mark_mutating()

That "default to mutating on any uncertainty" stance is deliberate: a false "mutating" just hides a skill in plan mode unnecessarily, but a false "read_only" would be an actual safety hole. Every ambiguous case is designed to fail closed.

What this buys, and what it doesn't

The result is an agent that genuinely gets more capable the more you use it, without needing a code review step for every new tool — the classifier substitutes for that review for the read-only/mutating distinction specifically. It's not a general sandbox: a skill classified "mutating" can still do anything real Python code can do, because it is real Python code, running in the same process as everything else. The static analysis answers one narrow question — "can this be proven safe to run unattended in plan mode" — and answers it conservatively rather than trying to be a general security boundary.