In Brief
Teel is an open source Python runtime that overlaps independent IO-bound calls in code you wrote sequentially. No async, no await, no futures, no gather. Mark expensive functions with @cacheable, mark orchestration with @jit, and a magic shadow interpreter reads ahead through your bytecode, finds calls whose arguments are already known, and dispatches them on background threads before the main thread arrives. On Princeton’s Tree of Thoughts, unmodified: 83% less runtime on average, and 8.07x on the longest run. 19% faster on average than LLM-generated async JavaScript inside a Deep Agents coding agent. Check the repo: github.com/AI21Labs/teel
Why this matters for agents
Agent quality right now comes mostly from spending more compute at inference. Run an ensemble. Generate five candidate patches instead of one. Split a task across N subagents and merge what they come back with. Different techniques, one execution shape underneath: fan out a lot of independent, IO-bound work, then join it back.
Our recent results on BrowseComp-Plus, SWE-rebench and DeepResearch Bench II came from three unrelated techniques and drew the same picture at the compute level: many independent rollouts, retrievals, tool calls or whole agent runs, overlapping, then merged.
And more and more, the agent writes that orchestration itself. In LangChain’s Deep Agents, the model writes a short script that dispatches its own subagents, instead of calling them one tool call at a time. Whoever writes it, all that fan-out comes with a price. Agent swarms take it furthest: Moonshot’s Kimi Agent Swarm runs up to 300 sub-agents in parallel and reports finishing tasks about 4.5x faster than a single agent.
The problem
When we profiled our agents’ runs, most of the wall clock was a thread sitting on a socket: model calls, retrievals, tool calls, a shell exec in a sandbox. Shaving the main agent code barely moved the total. The leverage is in the waits that do not depend on each other.
In Python, overlapping those waits means rewriting your code first. To run two calls at once you make them async. Then every function that calls them has to become async too, and so does every function that calls those. A change you wanted in one place climbs the whole call chain, until orchestration logic and concurrency plumbing are the same lines of code. Every new fan-out means doing it again. And when a model writes that code, it has to get all of it right on every run.
Here’s a small code-review agent three ways: as written, rewritten with asyncio, and as written plus Teel’s decorators. It asks a model for the worst bug in each changed file, asks whether those findings should block the merge, then either requests changes or approves.
Versions 2 and 3 finish in about the same time, 3.0s and 2.9s against 7.1s sequential. Both get there the same way: they start both possible follow-ups before the verdict is in, and keep whichever one it picks. Teel does that on its own, by following both branches of the if while the verdict is still pending. The asyncio version does it by hand, with three tasks, two cancellations, a finally block, and an async twin of every function it calls. The difference isn’t the concurrency. It’s who has to find it and manage it.
That’s why we built Teel. Concurrency shouldn’t have to be part of the code a developer, or an agent, writes. It can be the runtime’s job. You write the sequential version. Teel reads your function’s bytecode ahead of the line currently running, spots calls that aren’t waiting on anything, and starts them early on background threads. By the time your code reaches them, the results are already there or on their way.
Results
Tree of Thoughts, unmodified. We ran Princeton’s tree-of-thoughts-llm on Game of 24 samples with GPT-4 without editing the repo: @jit, @cacheable and @inline were applied from the outside. It is a textbook fan-out written as textbook sequential code (generate candidates, evaluate, keep the best, recurse), and the original walks independent siblings one at a time. Average runtime reduction was 83%, from 148s down to 24s at the low end and 307s down to 38s at the high end.
Code the model writes at runtime. LangChain’s Deep Agents ships a dcode agent with a js_eval tool: the model solves a task by emitting a snippet that runs in a QuickJS sandbox. The snippet can call tools.web_search() and spawn subagents via task(), so the model is authoring a fan-out on the fly. But it authors it in async JavaScript, which hands the concurrency model to the model itself, and models are inconsistent at reaching for Promise.all.
We forked Deep Agents and swapped the snippet language for synchronous Python under Teel, expecting simpler generated code at some cost in speed. It came out faster instead: across ten mixed web-search-and-LLM prompts, 58s average against 71s for the async JS and 150s for the same Python run sequentially. Teel parallelizes the independent calls the model forgot to overlap, and speculatively runs fallback branches ahead of the decision that needs them. (The same prompt yields different code each time, so this compares distributions of generated programs, not two implementations of one.) Our fork is at github.com/AI21Labs/deepagents.
The API
There are three decorators and a context manager. Teel currently needs Python 3.12.13 exactly; Known limits explains why.
@cacheable marks the heavy functions worth dispatching early. Results are memoized on a key built from json.dumps of the arguments, so keep those arguments JSON-serializable and boring.
@jit marks the functions Teel is allowed to look inside. This is your orchestration layer.
@inline is optional. It marks cheap helpers (string formatting, dict digging) that Teel may actually execute during lookahead, which usually uncovers more @cacheable calls to dispatch ahead of time.
with Teel(): turns it on. Outside the context every decorated function runs as plain Python, untouched.
How it works
Why bytecode? Teel’s implementation is strange enough that the path there is worth 30 seconds, because each step explains the next. Every step kept the same execution idea, independent calls running in parallel, and changed only the interface you wrote orchestration in.
The first version ran orchestration on Celery. Canvas signatures gave us a real DAG the cluster could schedule: utilization up, runs resumable, orchestration finally separate from computation. It was also miserable to write, because orchestration is business logic and Celery makes you write business logic in a pipe-and-signature dialect that is not quite Python.
So we brought it back into Python, or into a restricted subset of Python. We parsed that subset with ast, compiled it into an execution graph and wrote our own executor. The graph paid off on its own terms: independent functions ran in parallel, if bodies could be entered speculatively before the test resolved, and a call like func(x, y) could push into its subcalls while x and y were still pending. All of it available only for the slice of the language we were willing to parse.
Giving up the restriction meant giving up ast. Bytecode is the level at which every Python program looks the same.
How does a runtime know what you are about to call? When a @jit function runs, Teel duplicates the execution state into a limited shadow interpreter and steps through the bytecode your main thread hasn’t reached yet. It works on its own copy, so your program’s state is left alone.
Does Teel build a dependency graph? No. The ast version did. Teel never needs the whole graph, only one answer per call: are this call’s arguments ready yet? Any value it can’t compute yet, like the result of a @cacheable call still in flight, is marked pending, and anything computed from a pending value is pending too:
When lookahead reaches a @cacheable call with no pending arguments, Teel dispatches it to a ThreadPoolExecutor and memoizes the result. When your code reaches that line for real, the answer is already there or on its way. Two calls that don’t consume each other’s results never wait on each other, which is why fan-out needs no special support: [answer(q) for q in sub_questions] dispatches every answer(q) whose question is already known. Parallelism is the default, and you opt out by creating a real data dependency.
How does it get at your program’s state? Teel grabs the frame stack with sys._getframe(), uses a small C extension (frameinspect) to pull each frame’s full value stack out of CPython, and reads f_lasti to find the last instruction that was actually executed. That’s where the shadow interpreter starts stepping forward.
Getting to that state is a cursed operation. Reading a live frame’s value stack is not something CPython offers you, which is why frameinspect compiles against CPython’s internal headers and the whole thing is pinned to one exact interpreter version. The full walkthrough, opcode by opcode, is in the README’s Under the hood section.
The contract
What you promise. Teel does three things a normal interpreter never does: it runs a function earlier than your code would, sometimes runs one your code would never reach, and hands back a saved result instead of running it again. Each is only legal if the function does not care when it runs or how often. Picture a @cacheable function that sends an email or charges a card: Teel might run it for a branch your program never takes, and nothing unsends the email. That is the @cacheable contract, and it is on you: no database writes, no reading the clock, no hidden state, and JSON-serializable arguments, because json.dumps of them is the memo key. @jit promises nothing. It is permission to read your bytecode, not a claim about what the function does.
What Teel promises. The same call runs once: pending and finished tasks are keyed by function plus arguments, so a second lookahead that discovers the same fetch(url) attaches to the first instead of firing another request. Failed calls are dropped rather than cached, on the assumption the failure was transient.
And a wrong guess costs time and tokens, not correctness. The shadow interpreter never produces your answer. CPython still executes your function to completion, instruction for instruction, exactly as it would without Teel, and the only difference is that some calls find their result already in the cache. Guess wrong and you burned a background call, plus its tokens if it was an LLM call. Guess nothing and you pay full sequential price. Break the purity contract, though, and you own the result. That is the one failure mode Teel cannot protect you from.
Observability: where the wall clock actually went
When the runtime decides what runs early, you need a way to see what it decided. Give the context a path and Teel records the run:
The output is a Chrome trace-event file, so Perfetto renders it as a timeline: one bar per @cacheable call, on the thread that ran it, with start time and duration, and overlapping work showing up as bars side by side.
Per-call is the granularity that starts to matter once you dial test-time compute up and down. A result on a hard agentic benchmark is rarely a single number anymore: our BrowseComp-Plus work reported a frontier of accuracy against cost and latency, so a team can read off the quality it can afford. You cannot chart a latency frontier you can only measure end to end.
To measure the speedup, run the same code twice. A @cacheable function runs untouched with no active context, so the baseline is not a separate sequential implementation, it is the identical program.
Where this goes
Parallelism and observability both follow from one decision: the run is a set of calls keyed by their arguments. Take that one step further and an agent run stops being a disposable process and becomes reusable data. Persist the results and a later run can be served from them: unchanged calls are hits, and any call an edited input flows into gets a new key and re-runs. That gives you crash resume, debugging against pinned outputs, and iterating on orchestration without re-paying for every tool call. It also gives you human-in-the-loop: an agent freezes its state at a decision point, a person reviews, and the run resumes hours later on a different host.
Known limits
This is a proof of concept with two sharp edges. It’s pinned to Python 3.12.13 exactly, because frameinspect compiles against CPython’s internal headers. And the GIL still applies: background calls run on a ThreadPoolExecutor, so IO, network calls and GIL-free work like NumPy overlap fine, but for CPU-bound, GIL-constrained work, no dice.
Clone it, decorate your own functions, wrap your entry point, and start with the README:
