Python Extensions in Rust With PyO3 and Maturin Step by Step

Python is the right language for gluing things together and the wrong one for tight numeric loops, text crunching or anything you would like to run on all your cores at once. The standard fix is to write the hot function in a compiled language and call it from Python. In Rust, that is PyO3 for the bindings and maturin for building and packaging, and the result is a normal Python package: pip install, import, call. This guide builds one from an empty directory. You will write four Rust items (a function, a function that returns structured data, a function that releases Python's global interpreter lock, and a class), see Rust errors turn into Python exceptions, measure the speedup honestly, including the case where it is only 4 times and the case where it is 165 times, and build a wheel that installs on any Python from 3.9 up. Everything ran with Rust 1.99, PyO3 0.30.1, maturin 1.16.0 and CPython 3.14.6 on an Apple M1 Max. The performance numbers are from that machine; yours will differ in size, not in shape.

When this is worth doing

Decide before you write any Rust, because the boundary has a cost. A function call from Python into Rust takes tens of nanoseconds and converting arguments takes more, so wrapping a function that does a microsecond of work buys little. The win comes from work that stays inside Rust for a while: a loop over a million items, parsing a large string, compressing, hashing, geometry, a state machine, a CPU-bound task you want on several threads. If a profile of your Python shows one function eating most of the time and that function takes simple inputs (numbers, strings, byte buffers, lists of numbers) and returns a simple result, it is a good candidate. If the time goes into many small calls with Python objects, restructure first so that one call does a batch.

Set up an isolated environment

python3 -m venv .venv
. .venv/bin/activate
pip install maturin
maturin --version

The virtual environment matters for a practical reason: maturin develop installs the extension into the active environment and refuses to run without one. Outside a venv it stops with "Couldn't find a virtualenv or conda environment", which is a feature, because it keeps a half-built development package out of your system Python. pip install maturin gives you version 1.16.0; any 1.x release works with the configuration below.

The project: three files

[package]
name = "fastlib"
version = "0.1.0"
edition = "2024"

[lib]
name = "fastlib"
crate-type = ["cdylib"]

[dependencies]
pyo3 = { version = "0.30", features = ["abi3-py39"] }

[build-system]
requires = ["maturin>=1.7,<2.0"]
build-backend = "maturin"

[project]
name = "fastlib"
requires-python = ">=3.9"
dynamic = ["version"]

[tool.maturin]
features = ["pyo3/extension-module"]

Each setting has a reason. crate-type = ["cdylib"] builds a C-compatible dynamic library, which is what Python loads as an extension module; the [lib] name must match the Python module name you will import. The abi3-py39 feature builds against CPython's stable ABI, so one compiled wheel works on Python 3.9 and every later version, instead of one wheel per interpreter release. The cost is a restricted API (some PyO3 conveniences are unavailable and a few operations are slightly slower), and for most extensions it is the right trade. The extension-module feature, set through maturin, tells PyO3 not to link against libpython: the extension runs inside an already-running interpreter, which supplies the symbols. Putting it in pyproject.toml and not in Cargo.toml keeps cargo test working, since a test binary has no interpreter to provide them. The file pyproject.toml is what makes pip install . and maturin build the project; dynamic = ["version"] takes the version from Cargo.toml so it is written once.

The Rust module

use pyo3::exceptions::PyValueError;
use pyo3::prelude::*;
use std::collections::HashMap;

#[pymodule]
mod fastlib {
    use super::*;

    /// Iterative Fibonacci, wrapping on overflow like a u64.
    #[pyfunction]
    fn fib(n: u32) -> u64 {
        let (mut a, mut b) = (0u64, 1u64);
        for _ in 0..n {
            (a, b) = (b, a.wrapping_add(b));
        }
        a
    }

    /// Count words in a string. Returns the 'n' most common as (word, count).
    #[pyfunction]
    fn top_words(text: &str, n: usize) -> PyResult<Vec<(String, usize)>> {
        if n == 0 {
            return Err(PyValueError::new_err("n must be at least 1"));
        }
        let mut counts: HashMap<String, usize> = HashMap::new();
        for w in text.split(|c: char| !c.is_alphanumeric()).filter(|w| !w.is_empty()) {
            *counts.entry(w.to_lowercase()).or_default() += 1;
        }
        let mut pairs: Vec<_> = counts.into_iter().collect();
        pairs.sort_by(|a, b| b.1.cmp(&a.1).then_with(|| a.0.cmp(&b.0)));
        pairs.truncate(n);
        Ok(pairs)
    }

    /// Mix every value through a hash-like loop. The heavy part runs with the GIL released.
    #[pyfunction]
    fn churn(py: Python<'_>, values: Vec<u64>, rounds: u32) -> u64 {
        py.detach(|| {
            values.iter().fold(0u64, |acc, v| {
                let mut h = *v ^ 0xcbf29ce484222325;
                for _ in 0..rounds {
                    h = (h ^ (h >> 29)).wrapping_mul(0x100000001b3);
                }
                acc ^ h
            })
        })
    }

    /// Same work, but holding the GIL the whole time.
    #[pyfunction]
    fn churn_locked(values: Vec<u64>, rounds: u32) -> u64 {
        values.iter().fold(0u64, |acc, v| {
            let mut h = *v ^ 0xcbf29ce484222325;
            for _ in 0..rounds {
                h = (h ^ (h >> 29)).wrapping_mul(0x100000001b3);
            }
            acc ^ h
        })
    }

    #[pyclass]
    struct Counter {
        count: u64,
        step: u64,
    }

    #[pymethods]
    impl Counter {
        #[new]
        #[pyo3(signature = (step = 1))]
        fn new(step: u64) -> PyResult<Self> {
            if step == 0 {
                return Err(PyValueError::new_err("step must be non-zero"));
            }
            Ok(Counter { count: 0, step })
        }

        fn bump(&mut self) -> u64 {
            self.count += self.step;
            self.count
        }

        #[getter]
        fn count(&self) -> u64 {
            self.count
        }
    }
}

PyO3 0.30 uses a declarative style: a #[pymodule] mod fastlib { ... } block whose items become the module's contents. #[pyfunction] exposes a function, and PyO3 converts arguments and results automatically: u32 arrives from a Python int (out-of-range values raise OverflowError for you), &str borrows a Python string without copying, Vec<u64> converts from any list of ints, and a returned Vec<(String, usize)> becomes a Python list of tuples. Returning PyResult<T> lets a function raise: PyValueError::new_err("...") becomes a ValueError in Python with that message. The doc comment (///) becomes the Python docstring. #[pyclass] and #[pymethods] define a Python class backed by a Rust struct: #[new] is __init__, #[pyo3(signature = (step = 1))] gives the argument a default, and #[getter] exposes a read-only attribute.

Build, install, call

maturin develop --release

That compiles in release mode and installs the module into the active environment as an editable package; the first build took about eight seconds. Without --release you get a debug build that can be 10 to 100 times slower, which is the most common reason for "my Rust extension is not faster". Now drive it from Python:

import fastlib
import threading
import time

print(fastlib.fib(50), fastlib.fib(10))
print(fastlib.top_words("the cat and the hat and THE bat", 2))

try:
    fastlib.top_words("x", 0)
except ValueError as e:
    print("ValueError:", e)

try:
    fastlib.fib(-1)
except OverflowError as e:
    print("OverflowError:", e)

c = fastlib.Counter(step=5)
print(c.bump(), c.bump(), c.count)
try:
    fastlib.Counter(0)
except ValueError as e:
    print("ValueError:", e)

print(fastlib.churn([1, 2, 3], 10))
print(type(fastlib.fib), fastlib.fib.__doc__)

12586269025 55
[('the', 3), ('and', 2)]
ValueError: n must be at least 1
OverflowError: number too small to fit in target type
5 10 10
ValueError: step must be non-zero
13593663989309771750
<class 'builtin_function_or_method'> Iterative Fibonacci, wrapping on overflow like a u64.

Walk through the output. fib(50) is 12,586,269,025, and the wrapping arithmetic documented in the docstring is visible through fastlib.fib.__doc__. The word count returned the two most common words as tuples. The three exception lines show the conversion in action: top_words("x", 0) raised the ValueError from Rust, fib(-1) raised OverflowError without any code of ours, because a negative number does not fit a u32, and a Counter(0) raised ValueError from the constructor. The counter held its state in a Rust struct across calls (5, 10, then the count getter reports 10).

How much faster, measured

The benchmark compares each function with the obvious pure-Python version, taking the best of five runs:

import fastlib, time, collections, re, threading

def py_fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, (a + b) & 0xFFFFFFFFFFFFFFFF
    return a

def timeit(f, repeat=5):
    best = 1e9
    for _ in range(repeat):
        t = time.perf_counter(); f(); best = min(best, time.perf_counter() - t)
    return best

n = 1_000_000
p = timeit(lambda: py_fib(n)); r = timeit(lambda: fastlib.fib(n))
print(f"fib({n}): python {p*1e3:8.2f} ms | rust {r*1e3:8.3f} ms | {p/r:6.0f}x")

# tiny call: overhead dominated
tp = timeit(lambda: [py_fib(10) for _ in range(100000)]); tr = timeit(lambda: [fastlib.fib(10) for _ in range(100000)])
print(f"100k calls of fib(10): python {tp*1e3:7.1f} ms | rust {tr*1e3:7.1f} ms | {tp/tr:4.1f}x  (per call {tr/100000*1e9:.0f} ns)")

text = ("the quick brown fox jumps over the lazy dog and the cat " * 20000)
def py_top():
    c = collections.Counter(w.lower() for w in re.split(r"[^\w]+", text) if w)
    return sorted(c.items(), key=lambda kv: (-kv[1], kv[0]))[:3]
a = timeit(py_top); b = timeit(lambda: fastlib.top_words(text, 3))
print(f"top_words on {len(text)/1e6:.1f} MB: python {a*1e3:7.1f} ms | rust {b*1e3:7.1f} ms | {a/b:4.1f}x")
assert [list(x) for x in py_top()] == [list(x) for x in fastlib.top_words(text, 3)]

vals = list(range(2000))
def run_threads(k, fn=fastlib.churn):
    res = [0]*k; ts = [threading.Thread(target=lambda i=i: res.__setitem__(i, fn(vals, 20000))) for i in range(k)]
    t = time.perf_counter(); [x.start() for x in ts]; [x.join() for x in ts]; return time.perf_counter() - t
t1 = min(run_threads(1) for _ in range(3)); t4 = min(run_threads(4) for _ in range(3))
print(f"churn with the GIL released: 1 thread {t1*1e3:.0f} ms, 4 threads {t4*1e3:.0f} ms for 4x the work")
l1 = min(run_threads(1, fastlib.churn_locked) for _ in range(3)); l4 = min(run_threads(4, fastlib.churn_locked) for _ in range(3))
print(f"churn_locked (GIL held):       1 thread {l1*1e3:.0f} ms, 4 threads {l4*1e3:.0f} ms for 4x the work")

fib(1000000): python    53.26 ms | rust    0.322 ms |    165x
100k calls of fib(10): python    37.3 ms | rust     6.2 ms |  6.0x  (per call 62 ns)
top_words on 1.1 MB: python    52.5 ms | rust    12.1 ms |  4.3x
churn with the GIL released: 1 thread 64 ms, 4 threads 66 ms for 4x the work
churn_locked (GIL held):       1 thread 64 ms, 4 threads 258 ms for 4x the work

Four results, four lessons. Pure compute is where the gain is: a million iterations of Fibonacci took 53 ms in Python and 0.32 ms in Rust, 165 times faster, because the Python version pays interpreter overhead on every iteration. Tiny calls are not free: a hundred thousand calls of fib(10) was only 6 times faster, with a call costing about 62 ns of which almost none is the Fibonacci. Crossing the boundary is the cost; batch work so that each call does a lot. Library-heavy code shrinks the gain: counting words in a 1.1 MB string was 4.3 times faster, a modest win, because Python's re and Counter are themselves compiled C code, so you are comparing Rust with C plus some interpreter glue. The assertion in the script checks that both versions return the same answer, which is the check you always want in a benchmark. The GIL is the multiplier: see the next section.

Releasing the GIL to use all your cores

CPython's global interpreter lock lets only one thread run Python code at a time. A Rust function that holds the lock while it works blocks every other Python thread, exactly as a slow Python function would. Inside py.detach(|| ...) the lock is released, and other threads run, including other calls into your Rust code. The rule: copy what you need out of Python objects first (the Vec<u64> argument is already a Rust-owned copy by the time the function body starts), then detach for the heavy part. The last two benchmark lines run the same hashing loop, once with detach and once without, on four Python threads each doing the same work as a single thread:

With the lock released, four threads did four times the work in the same wall time: real parallelism, from a language with a global lock. With the lock held, the same work took four times as long, because the threads queued behind one another. That difference is the strongest reason to use Rust from Python for CPU-bound work, and it is a one-line change. A caution from the first attempt at this benchmark: an earlier version detached only around a trivial sum and converted a two-million-element list while holding the lock, and four threads took four times as long. The conversion from a Python list is itself work under the lock, so a function whose cost is mostly argument conversion gains nothing from detach. Pass big data as a buffer (bytes, or a NumPy array through the numpy crate) that Rust can read without converting element by element. Older PyO3 code spells it py.allow_threads; the name changed to detach in recent releases, along with with_gil becoming attach.

Big inputs, interruptions and panics

Three more behaviours decide whether an extension feels like a real Python library. First, large inputs. A &[u8] parameter borrows the contents of a Python bytes object directly, with no copy, so a checksum over 100 MB costs only the checksum. It accepts bytes and nothing else; a bytearray or str is a TypeError with an accurate message. Second, long loops should stay interruptible: a Rust loop that never returns to Python ignores Ctrl-C, so call py.check_signals()? every few thousand iterations, which raises KeyboardInterrupt when a signal is pending. Third, panics: a Rust panic must not unwind into the interpreter, so PyO3 catches it and raises PanicException, deriving from BaseException rather than Exception so that ordinary except Exception handlers do not swallow it. Three small functions added to the module show all three:

/// FNV-1a over a bytes-like object, without copying it out of Python.
#[pyfunction]
fn checksum(data: &[u8]) -> u64 {
    data.iter().fold(0xcbf29ce484222325u64, |h, b| (h ^ u64::from(*b)).wrapping_mul(0x100000001b3))
}

/// Busy loop that stays interruptible from Python.
#[pyfunction]
fn spin(py: Python<'_>, seconds: f64) -> PyResult<u64> {
    let end = std::time::Instant::now() + std::time::Duration::from_secs_f64(seconds);
    let mut n = 0u64;
    while std::time::Instant::now() < end {
        n += 1;
        if n % 1000 == 0 {
            py.check_signals()?;
        }
    }
    Ok(n)
}

/// Always panics, to show what Python sees.
#[pyfunction]
fn explode() -> u32 {
    let v: Vec<u32> = Vec::new();
    v[3]
}

import fastlib, signal, time

data = bytes(range(256)) * 400_000
t = time.perf_counter(); c = fastlib.checksum(data); dt = time.perf_counter() - t
print(f"checksum of {len(data)/1e6:.0f} MB bytes: {c:#018x} in {dt*1e3:.0f} ms")
for bad in (bytearray(b"abc"), "text"):
    try:
        fastlib.checksum(bad)
    except TypeError as e:
        print(f"TypeError for {type(bad).__name__}:", str(e).replace(chr(10), " "))

signal.signal(signal.SIGALRM, lambda *a: signal.raise_signal(signal.SIGINT))
signal.setitimer(signal.ITIMER_REAL, 0.2)
t = time.perf_counter()
try:
    fastlib.spin(5.0)
except KeyboardInterrupt:
    print(f"spin(5.0) interrupted after {time.perf_counter() - t:.1f} s: KeyboardInterrupt")

try:
    fastlib.explode()
except BaseException as e:
    print(type(e).__name__, "-", e, "| is an Exception:", isinstance(e, Exception))

checksum of 102 MB bytes: 0x07ec59baf399a325 in 134 ms
TypeError for bytearray: 'bytearray' object is not an instance of 'bytes'
TypeError for str: 'str' object is not an instance of 'bytes'
spin(5.0) interrupted after 0.2 s: KeyboardInterrupt
PanicException - index out of bounds: the len is 0 but the index is 3 | is an Exception: False

The checksum ran over 102 MB in 134 ms with no copy, about 760 MB per second for a byte-at-a-time hash. The interrupt test schedules a SIGINT after 0.2 s while Rust spins for five: the call stopped after 0.2 s with KeyboardInterrupt, exactly as a Python loop would. The panic produced PanicException with Rust's message and isinstance(e, Exception) was false; the panic message is also printed to standard error by Rust's hook. The practical rules follow directly: expected failures return PyResult (and become precise Python exceptions), panics are for bugs, and any loop longer than a few milliseconds checks for signals.

The direction that loses: Rust calling Python

Everything above sends work into Rust. The reverse, a Rust function that calls a Python callable for each item, is worth measuring before you design around it, because it is the case where the extension is slower than the code it replaced:

#[pyfunction]
fn apply(f: &Bound<'_, PyAny>, values: Vec<i64>) -> PyResult<Vec<i64>> {
    values.into_iter().map(|v| f.call1((v,))?.extract::<i64>()).collect()
}

1M callbacks: python list comprehension 47 ms | Rust calling the same lambda 97 ms
exception from the callback propagates: division by zero

A million calls of lambda v: v * 2 took 47 ms in a Python list comprehension and 97 ms from Rust: about twice as slow, because each callback builds an argument tuple, enters the interpreter's call machinery and converts the result back, while the comprehension runs on a fast path inside the interpreter. The error handling is the good news: an exception raised in the callback (ZeroDivisionError) came out of the Rust function as itself, because ? on a PyResult propagates it. The design rule is that the callback boundary is expensive, so give Rust data and let it work without calling back. If the user must supply behaviour, accept a small description of it (an enum or a string naming an operation) instead of a function, or make the callback process a whole batch per call.

A package that mixes Python and Rust

A pure extension module is fine for a small helper. A real library usually wants a Python layer on top: a friendlier API, docstrings, type hints, convenience functions that do not need Rust. Maturin supports that with a source directory for the Python code and a private compiled module underneath, conventionally named _core. The package imports from it, and users only see the public names:

[build-system]
requires = ["maturin>=1.7,<2.0"]
build-backend = "maturin"

[project]
name = "fastlib"
requires-python = ">=3.9"
dynamic = ["version"]

[tool.maturin]
python-source = "python"
module-name = "fastlib._core"
features = ["pyo3/extension-module"]

use pyo3::prelude::*;

#[pymodule]
mod _core {
    use super::*;

    #[pyfunction]
    fn fib(n: u32) -> u64 {
        let (mut a, mut b) = (0u64, 1u64);
        for _ in 0..n {
            (a, b) = (b, a.wrapping_add(b));
        }
        a
    }
}

"""Fast helpers backed by Rust."""
from ._core import fib

__all__ = ["fib", "fib_many"]


def fib_many(ns):
    """Pure Python convenience layer on top of the Rust core."""
    return [fib(n) for n in ns]

Three names must agree: the Cargo library name (_core), the #[pymodule] module name (_core) and the last component of module-name in pyproject.toml (fastlib._core). After maturin develop the package imported correctly and fastlib.fib_many([10, 20, 30]) returned [55, 6765, 832040], with the compiled module landing at fastlib/_core.abi3.so next to the Python files. Editing a Python file takes effect immediately in the editable install; editing Rust needs another maturin develop. This layout also gives you a place for the .pyi stub file and the py.typed marker, which together make type checkers understand the package.

Build wheels you can ship

maturin build --release
ls target/wheels
pip install target/wheels/fastlib-0.1.0-cp39-abi3-macosx_11_0_arm64.whl

The wheel produced here was named fastlib-0.1.0-cp39-abi3-macosx_11_0_arm64.whl and was 251 KB. The tags say it all: cp39-abi3 means "CPython 3.9 or newer, stable ABI", and the platform part says macOS on ARM. You need one wheel per operating system and CPU architecture, not per Python version. Build them in CI with the PyO3/maturin-action GitHub Action, which runs inside the manylinux containers so Linux wheels work on every distribution, and publish with maturin publish or twine. Linux wheels must be built against an old glibc (the manylinux tag), which is exactly what the action's containers provide; a wheel built on your newest laptop distribution will fail to import on older servers with a GLIBC_2.xx not found error.

What goes wrong

Test it like a Python package

Test the Rust logic with cargo test on plain functions (keep the algorithm in an ordinary function and make the #[pyfunction] a thin wrapper), and test the Python surface with pytest: import the module, call every function, assert on results and on the exact exception types. The most valuable kind of test compares the Rust implementation against a slow, obviously correct Python one over many inputs, including the edges:

import pytest
import fastlib


def python_fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, (a + b) & 0xFFFFFFFFFFFFFFFF
    return a


@pytest.mark.parametrize("n", [0, 1, 2, 10, 50, 93, 94, 1000])
def test_matches_python(n):
    assert fastlib.fib(n) == python_fib(n)


def test_negative_is_overflow_error():
    with pytest.raises(OverflowError):
        fastlib.fib(-1)


def test_fib_many_keeps_order():
    assert fastlib.fib_many([5, 1, 10]) == [5, 1, 55]

..........                                                               [100%]
10 passed in 0.01s

The parametrised cases include 93 and 94, the boundary where a Fibonacci number stops fitting in 64 bits and the Rust code wraps; the Python reference masks to 64 bits to match, so the test pins down that the behaviour is intentional and the same on both sides. The OverflowError test pins the conversion behaviour that PyO3 gives for free, which protects you from a future change of the argument type. Ship a fastlib.pyi stub file next to the module (or inside the package, with an empty py.typed marker) so editors and type checkers know the signatures; PyO3 does not generate stubs for you, and a hand-written 15-line file is enough. Add one benchmark to the suite that fails if the Rust path becomes slower than the Python fallback, so the reason for the dependency stays true.

Where this does not fit

If the bottleneck is I/O (network, disk), Rust will not help; use async Python. If it is a numeric array operation that NumPy, Polars or JAX already do, call those. If you need a command-line tool or a service, write a Rust binary and call it, avoiding the boundary altogether. And when the Python side needs to call back into Python many times from Rust, the cost of crossing in both directions can erase the gain, so keep the callbacks coarse. The same approach for JavaScript is in Node addons with napi-rs, and the C interface underneath both is explained in calling Rust from C.