Advanced Python 2 min read
TIL - Python 3.14 runs code in parallel with subinterpreters
InterpreterPoolExecutor gives you multi-core parallelism inside one process, each interpreter with its own GIL.
Python 3.14 added concurrent.futures.InterpreterPoolExecutor and the concurrent.interpreters module. Each subinterpreter has its own GIL, so CPU-bound Python code can run in parallel within a single process.
It’s a middle ground between threads and processes:
| Threads | Subinterpreters | Processes | |
|---|---|---|---|
| CPU parallelism (standard build) | shared GIL | one GIL each | |
| Startup cost | tiny | small | large |
| Shared Python objects | yes | no (isolated) | no |
| Isolation | none | strong | full |
Using it looks exactly like the other executors:
from concurrent.futures import InterpreterPoolExecutor
import time
def cpu_work(n):
return sum(i * i for i in range(n))
if __name__ == "__main__":
start = time.perf_counter()
with InterpreterPoolExecutor(max_workers=4) as pool:
results = list(pool.map(cpu_work, [2_000_000] * 4))
print(len(results), f"{time.perf_counter() - start:.2f}s")On a multi-core machine this scales like a process pool for pure-Python CPU work, but without spawning whole new processes.
The lower-level concurrent.interpreters module gives you direct control:
from concurrent import interpreters
interp = interpreters.create()
interp.exec("import sys; print('hello from', sys.implementation.name)")
print(interp.call(pow, 2, 10)) # 1024: run a function and get its result
interp.close()interpreters.create_queue() passes data between interpreters.
What to know before using it:
- Nothing is shared. Each interpreter imports modules separately and has its own globals. Arguments and results are copied between interpreters (usually via pickle), so the function has to be importable.
- Extension module support varies. C extensions must opt in to multi-interpreter support, and many popular ones haven’t yet, so imports may fail inside a subinterpreter.
- It’s new, and the ecosystem is still catching up. Treat it as promising rather than default.
Works on: Python 3.14+ (PEP 734).
Takeaway: for CPU-bound pure-Python work, try InterpreterPoolExecutor as a lighter alternative to ProcessPoolExecutor.