Migrating a Python or Node Service to Rust Incrementally

A service that has been running for years contains decisions nobody wrote down: how it rounds a price, which characters it strips from a name, what it returns for a missing field. A rewrite throws that knowledge away and rediscovers it in production, one customer complaint at a time. An incremental migration keeps the service running and lets Rust take over one route at a time, with evidence at each step that the new code behaves the same. This guide runs that method on a small, real experiment. A Python service computes prices. A Rust gateway sits in front of it and moves traffic through four modes: pass-through, shadow comparison, a percentage canary and full cutover. The shadow step finds that a straightforward Rust rewrite disagrees with the old service on 244 of 600 requests, for a reason that no code review would have caught. Versions: rustc 1.99.0, axum 0.8, reqwest 0.13, Python 3 for the legacy side, on an Apple M1 Max, all on one machine over loopback. The numbers describe this experiment and not your system.

Step zero: decide why, and what done looks like

Do not start with a language. Start with a measurable reason that Rust would address: a p99 latency target the current service misses, a fleet that could shrink from twelve instances to four, a class of production bug (null handling, type confusion) that keeps recurring, or a library that exists only in Rust. Write the target as a number with a deadline. If you cannot name one, the migration may not be worth the cost, and "the team likes Rust" is a legitimate input but a poor success criterion. The target also tells you which route to move first: the one that dominates the cost you want to cut, not the easiest one. A route that is cheap, isolated and busy is the ideal first candidate.

The shape: a router in front, one route at a time

The structure that makes this safe is a router that sits in front of the legacy service and decides, per request, who answers. The legacy service does not change. Moving a route is a configuration edit, and so is moving it back, which turns a risky release into a reversible switch. This is called the strangler pattern: the new system grows around the old one, taking over route by route, until the old one has nothing left to serve and can be removed. In production the router is often nginx, Envoy or a cloud load balancer; here it is a Rust program, because that lets one binary do the routing, the comparison and the new implementation, and keeps the experiment self-contained.

The legacy service

The old service is a Python HTTP server with one endpoint, GET /price?sku=A100&qty=3. It stores unit prices as floating-point dollars and rounds the total with Python's round(). This is exactly the kind of code that gets written once and works for years:

import json
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib.parse import urlparse, parse_qs

PRICES = {"A100": 1.005, "B200": 2.675, "C300": 0.125}  # dollars per unit
TAX = 1.0

class H(BaseHTTPRequestHandler):
    def do_GET(self):
        u = urlparse(self.path)
        q = parse_qs(u.query)
        if u.path != "/price":
            self.send_response(404); self.end_headers(); return
        sku, qty = q["sku"][0], int(q["qty"][0])
        total = round(PRICES[sku] * qty * TAX, 2)
        body = json.dumps({"sku": sku, "qty": qty, "total": total}).encode()
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(body)))
        self.end_headers()
        self.wfile.write(body)
    def log_message(self, *a): pass

HTTPServer(("127.0.0.1", 9001), H).serve_forever()

Three spot checks against the running service show its behaviour. They are the reference answers, correct by definition, because customers have been billed these amounts:

GET /price?sku=A100&qty=3   {"sku": "A100", "qty": 3, "total": 3.01}
GET /price?sku=C300&qty=1   {"sku": "C300", "qty": 1, "total": 0.12}
GET /price?sku=B200&qty=1   {"sku": "B200", "qty": 1, "total": 2.67}

A person reading the table would expect different totals: 3 times 1.005 is 3.015, which rounds to 3.02 by the rule taught at school, and 0.125 rounds to 0.13. The service says 3.01 and 0.12. Two effects explain it, and both are properties of the old code that no document mentions. First, 1.005 and 2.675 cannot be stored exactly in binary floating point; the stored values are slightly below the decimal value, so the product lands just under the halfway point and rounds down. Second, 0.125 is exact, and Python's round() breaks exact ties toward the even digit (banker's rounding), so 0.125 becomes 0.12. A rewrite that does "the right arithmetic" will produce different invoices from the old system. Whether the old behaviour is a bug to fix or a contract to keep is a business decision, and the point of the migration method is to surface it before customers do.

The gateway and its four modes

The Rust side is an axum service with one route and a mode chosen by an environment variable. The modes are the migration steps in order, and moving between them is a restart, not a deployment of new code:

• legacy: forward every request to the old service. This proves the router is transparent before anything changes.
• shadow: forward to the old service and return its answer, but also compute the Rust answer and compare. Customers see no change; you collect disagreements.
• canary:N: serve N percent of requests from Rust, the rest from legacy.
• rust: Rust answers everything. The legacy service can be turned off once this has been quiet long enough.

The core of it is one function per side and one handler that dispatches on the mode. First, the first Rust implementation, written the way most people would write it, with exact integer arithmetic and school rounding:

/// Prices in thousandths of a dollar, so no floating point is involved.
fn unit_thousandths(sku: &str) -> Option<u64> {
    match sku {
        "A100" => Some(1005),
        "B200" => Some(2675),
        "C300" => Some(125),
        _ => None,
    }
}

/// The "obvious" rewrite: exact decimal arithmetic, round half up to the cent.
fn price_clean(sku: &str, qty: u32) -> Option<Price> {
    let thousandths = unit_thousandths(sku)? * qty as u64;
    let cents = (thousandths + 5) / 10;
    Some(Price { sku: sku.into(), qty, total: cents as f64 / 100.0 })
}

This is arguably the better arithmetic: no binary floating point in the calculation, and integers all the way to the final division for display. A code reviewer would approve it, and it would pass a unit test written from the specification. The handler, which dispatches on the mode and, in shadow mode, records disagreements, is where the migration actually happens:

async fn price(State(app): State<Arc<App>>, Query(q): Query<PriceQuery>) -> Result<Json<Price>, StatusCode> {
    match app.mode {
        Mode::Legacy => Ok(Json(price_legacy(&app, &q).await?)),
        Mode::Rust => price_rust(&app, &q).map(Json).ok_or(StatusCode::NOT_FOUND),
        Mode::Canary(pct) => {
            if bucket(&q) < pct {
                price_rust(&app, &q).map(Json).ok_or(StatusCode::NOT_FOUND)
            } else {
                Ok(Json(price_legacy(&app, &q).await?))
            }
        }
        Mode::Shadow => {
            let old = price_legacy(&app, &q).await?;
            app.compared.fetch_add(1, Ordering::Relaxed);
            let new = price_rust(&app, &q);
            if new.as_ref() != Some(&old) {
                app.mismatched.fetch_add(1, Ordering::Relaxed);
                let mut ex = app.examples.lock().unwrap();
                if ex.len() < 3 {
                    ex.push(json!({"sku": q.sku, "qty": q.qty, "legacy": old.total, "rust": new.map(|p| p.total)}));
                }
            }
            Ok(Json(old)) // the caller still gets the legacy answer
        }
    }
}

Notice the last line of the shadow arm: the caller always receives the legacy answer. Shadow mode is safe because the Rust result is computed, compared and thrown away. Comparison uses the typed Price struct and its derived PartialEq, so a difference in any field counts; in a real service you would normalise fields that legitimately vary, such as timestamps and request ids, before comparing. A /stats endpoint reports the counters and the first three disagreements, which is all the operator needs to see.

Shadow mode finds what review could not

To drive it, a client asked for every combination of the three SKUs and quantities 1 through 200, which is 600 requests. First with the gateway in pass-through mode as a control, then in shadow mode with the clean rewrite. The control showed 600 requests and no comparisons, as it should. Shadow mode reported this:

{"compared": 600,
 "mismatched": 244,
 "examples": [
   {"sku": "A100", "qty": 1, "legacy": 1.0,  "rust": 1.01},
   {"sku": "A100", "qty": 3, "legacy": 3.01, "rust": 3.02},
   {"sku": "A100", "qty": 5, "legacy": 5.02, "rust": 5.03}],
 "mode": "Shadow"}

Of 600 requests, 244 disagreed, which is 41%. The first example is the smallest: one unit of A100 costs $1.005, which the old service rounds to 1.0 and the clean Rust version to 1.01. None of this is exotic: it is the ordinary difference between binary floating point and decimal arithmetic, and the old code's behaviour is the correct answer by the only definition that matters, which is that it is what customers were charged. Switching straight over would have changed the price of 41% of these requests, silently, with every unit test green. That is the case for shadow mode in one number.

There are two honest responses, and the team that owns the product must pick one. Either the old rounding is a defect, in which case you fix it deliberately, announce it, and accept that the shadow disagreement is expected from now on. Or it is a contract, in which case the Rust code must reproduce it. For this experiment I chose compatibility, to see whether shadow mode would confirm a fix. The compatible implementation does what Python does, with floating-point arithmetic and a rounding step that works on the float's exact value:

/// Matches the legacy service: float arithmetic, then the float's exact decimal value rounded half to even.
fn price_compat(sku: &str, qty: u32) -> Option<Price> {
    let unit = unit_thousandths(sku)? as f64 / 1000.0;
    let total: f64 = format!("{:.2}", unit * qty as f64).parse().ok()?;
    Some(Price { sku: sku.into(), qty, total })
}

It relies on Rust's formatting of a float with two decimals, which rounds the exact binary value, the same quantity Python rounds. With this function swapped in and the same 600 requests, shadow mode reported:

{"compared": 600, "mismatched": 0, "examples": [], "mode": "Shadow"}

Zero disagreements across 600 requests. I want to be precise about what that does and does not show. It shows the two implementations agree on these 600 inputs: three SKUs and quantities up to 200. It does not prove they agree everywhere, because float rounding behaviour depends on the exact values, and a SKU with a price like 19.995 or a quantity in the thousands might expose another case. The practical rule is that shadow mode proves agreement on the traffic you send through it, so run it on real production traffic for long enough to cover the long tail (month-end, large orders, unusual SKUs), and treat a clean shadow run as a reason to proceed to a canary, not as a proof of equivalence.

Canary: move a percentage, keep it stable

Once shadow is quiet, let Rust answer for real, starting small. The gateway chooses the side by hashing the request's key into one of 100 buckets and comparing it with the canary percentage. The hash makes the choice stable: the same customer and the same request always land on the same side, so a customer does not see the price flip between two implementations from one reload to the next, and a bug report can be reproduced.

/// A stable bucket per request key, so the same customer always lands on the same side during a canary.
fn bucket(q: &PriceQuery) -> u8 {
    let mut h = DefaultHasher::new();
    (&q.sku, q.qty).hash(&mut h);
    (h.finish() % 100) as u8
}

A caution about that function. DefaultHasher is documented as unspecified: its algorithm can change between Rust releases, and a compiler upgrade could silently move customers from one side to the other in the middle of a canary. For a real rollout, hash the key with a fixed, named algorithm (FNV-1a or xxHash, for example) so the assignment is a property of your code and not of your toolchain. I used the default here because the experiment runs one binary once.

I ran 3,000 requests (the 600 distinct requests, five times each) in each mode with the compatible implementation and counted how many Rust answered:

gateway mode legacy      served_by_rust = 0 of 3000
gateway mode canary:10   served_by_rust = 340 of 3000
gateway mode canary:50   served_by_rust = 1470 of 3000
gateway mode rust        served_by_rust = 3000 of 3000

A 10% canary sent 340 requests (11.3%) to Rust and a 50% canary sent 1,470 (49.0%). The split is close to the target but not exact, because 600 distinct keys are hashed into 100 buckets: with so few keys, the share of keys that fall below the line varies. At production volumes it converges. Each key was repeated five times and always went to the same side, which is why the counts are not noisy. During a real canary you watch error rate, latency and the shadow-style comparison on the canary slice, and you raise the percentage in steps (1, 5, 25, 50, 100) with a pause at each. Moving back is the same edit in the other direction.

What the hop costs

An extra proxy hop is not free, and the migration should say what it costs before it argues what it saves. I timed sequential requests from a Python client (a new connection per request, which inflates every number) and kept the best of five rounds of 600:

legacy directly              185 us per request
gateway mode legacy          311 us per request
gateway mode canary:10       299 us per request
gateway mode canary:50       240 us per request
gateway mode rust            142 us per request

Putting the gateway in front of the legacy service added about 126 microseconds per request (185 to 311). Answering in Rust removed the Python service from the path altogether and came in at 142 microseconds, below even the direct call to legacy. The cost of the migration is therefore front-loaded: during the transition, requests that still go to legacy pay the extra hop, and the price falls as traffic moves. These figures mix client overhead, loopback networking and the cost of a new connection per request, so only the ordering and rough ratios mean anything; do not quote 142 microseconds as the speed of Rust. A real measurement would use a load generator with keep-alive connections against the deployed topology and compare percentiles, not a best-of-five mean.

Beyond one stateless route

This experiment moved a pure function, which is the easy case. Real services push back in four places, and each needs its own plan.

• Shared data. If both services write to the same database, they must agree on every invariant the old code enforced, including the ones in application code and not in the schema. Move reads first, since a read-only route has no write conflicts, and give a table a single writer at a time.
• Side effects cannot be shadowed. Shadow mode that sends an email or charges a card sends it twice. Shadow only routes whose effects are idempotent or can be stubbed. For writes, compare what each side would have written, or run the new path against a copy of the data.
• Sessions and authentication. The new service must accept the cookies, tokens and signatures the old one issues. Port the verification code and its test vectors before moving any authenticated route.
• Background jobs and queues. Consumers can move one queue at a time, and two consumers on one queue need care about ordering and duplicate delivery.

The gateway is not the only way in. When only a hot function is slow, replacing it inside the existing process with a native extension (PyO3 for Python, napi-rs for Node, both covered elsewhere on this site) is smaller than a new service, with no network hop and no new deployment unit. The trade-off is that the language boundary is now inside your process: a panic or a memory bug in the extension takes the whole service with it, and the build and packaging story gets harder. Choose the service when the logic is large or you want Rust's runtime for the whole route; choose the extension when one function dominates the profile.

Retiring the legacy route

A route is finished when it has been at 100% for long enough that you would notice a problem, the shadow comparison (kept running on a sample of traffic after cutover is a cheap safeguard) is clean, and nothing else calls the old endpoint. Check the old service's access logs, not just the gateway's: an internal batch job that bypasses the router is how a "retired" route comes back to life. Then delete the legacy code for that route, not just the routing rule, because dead code that nobody can run is documentation that lies. Keep the shadow comparison test and the sample inputs; they are now the specification.

Where this experiment stops

• One endpoint, three SKUs. The 600-request sweep is a toy. Its value is the shape of the method and one real, surprising disagreement, not coverage.
• Loopback on a laptop. No network, no TLS, no real concurrency. The latency table shows ordering, not production numbers, and the load client opened a new connection per request.
• No database, authentication or writes. The hardest parts of a migration are exactly the ones absent here.
• Shadow mode doubles the work. Computing both answers costs CPU on every request. At real volume, shadow a sample.
• The unspecified hash. The canary used DefaultHasher; see the caution above before copying it.
• Agreement is not correctness. A clean shadow run shows the new code matches the old, bugs included. If the old behaviour is wrong, matching it is a choice you made, and the fix is a separate, announced change.

The measured story fits in three numbers. A clean, defensible rewrite disagreed with the legacy service on 244 of 600 requests. A bug-compatible version disagreed on zero. And moving traffic through a stable-hashed canary took the request path from 311 microseconds through the gateway to 142 microseconds with Rust answering everything. The order of the steps is the safety: the route, then shadow, then canary, then cutover, then deletion, and the evidence collected at each one.