The Bypass Landscape

Before you start. This chapter assumes descriptor rings (the shared circular queues the NIC and driver exchange buffers through, ch00b), DMA (the card reading/writing RAM directly, no CPU copy, ch00b), sk_buff (the kernel’s per-packet metadata object, ch00b), syscalls (the cost of crossing into the kernel, ch00b), hugepages / TLB (big memory pages that keep address translation cached, ch00b), and busy-polling (spinning instead of blocking, ch00c). If any are new, read those first — 20 minutes there saves an hour here.

The kernel-tuning chapter shrank the kernel path. This chapter replaces it.

The mailroom, and the desk

Hold one picture for the whole chapter.

Your process is an office worker. The NIC is the building’s loading dock, where packages (packets) physically arrive. Between the dock and your desk sits the kernel: the building’s mailroom. Every arriving package gets logged (sk_buff allocated), sorted (protocol stack), placed in a pigeonhole (socket buffer), and then someone walks upstairs to wake you (scheduler wakeup) so you can come down to the counter (syscall) and collect it (copy). Each step exists for a reason — the mailroom serves every tenant in the building safely and fairly. But you’re paying for that generality on every single package: ~2–10µs on a tuned kernel, with occasional multi-µs hiccups whenever the mailroom staff gets busy with someone else.

Kernel bypass fires the mailroom — for your packages. The dock delivers straight to a tray on your desk, and you glance at the tray in a tight loop instead of waiting to be called. Concretely: the NIC’s RX/TX descriptor rings are mapped into your process’s memory, the card DMA-writes arriving frames directly into buffers you own, and your pinned thread polls those rings. No interrupts, no sk_buffs, no syscalls, no copies on the hot path.

   kernel path (the mailroom)          bypass path (tray on your desk)
   ───────────────────────────        ─────────────────────────────────
   NIC ──DMA──► kernel ring           NIC ──DMA──► rings mapped into
        IRQ → softirq → skb                        YOUR address space
        → stack → socket →            your thread polls descriptors
        wakeup → syscall → copy       in a loop; TX = write descriptor
        → your buffer                 + doorbell. No kernel on hot path.

Two things you buy:

• Latency: wire-to-app drops from ~2–10µs to ~1–2µs.
• Variance — the more important one: nothing in the path can schedule, sleep, or be preempted, so the tail collapses. The mailroom can have a bad day; a tray on your desk cannot.

And one thing you sell: every service the mailroom provided. Nobody sorts your other mail anymore (no sockets, no TCP), nobody keeps records (no tcpdump), nobody answers the building’s front door for you (no ARP). Every framework below is a different answer to three questions: who owns the dock, who rebuilds the mailroom’s services, and how much of the building you still get to use.

First, though, one framing that shapes how you should learn this chapter: you have not run these in production, and you should say so in interviews. That’s fine — most senior candidates haven’t. What’s tested is whether you understand the architecture, the tradeoffs, and where each tool fits. The strong position: “My production latency work was lock-free, zero-alloc userspace design plus kernel tuning at ~10ms budgets dominated by venue RTT; kernel bypass is the next lever down, and here’s exactly how I’d apply it and what it would cost us.” This chapter arms that sentence. Never bluff hands-on experience — a follow-up (“what did your ef_vi event loop look like?”) kills a bluff in seconds; architecture fluency paired with knowing exactly where your experience ends reads as senior.

DPDK: fire the mailroom, run the dock yourself

DPDK (Data Plane Development Kit — Intel-originated, now Linux Foundation) is the full takeover: your application takes the loading dock away from the building and staffs it with its own people.

Watch what physically happens when a DPDK app starts:

1. The kernel unbinds the NIC — vfio-pci, the kernel facility that detaches a device from its in-kernel driver and hands userspace safe access to it. The NIC vanishes from ip link. As far as Linux is concerned, that network card no longer exists.
2. Your process maps the card’s rings and control registers (the memory-mapped switches that command the hardware) into its own address space, using a PMD — a poll-mode driver, DPDK’s userspace driver for that NIC family. The driver now lives inside your app.
3. One or more of your threads — DPDK calls a thread pinned to its own dedicated core an lcore — start spinning: rte_eth_rx_burst() to harvest arrived frames, rte_eth_tx_burst() to send. Those cores run at 100% CPU forever. That’s not a bug; that’s the employee standing at the dock staring at the conveyor so nothing ever waits.

A packet’s life is now: wire → NIC → DMA into a buffer your process owns → your loop notices it on the next poll → your code parses it. The kernel never learns the packet existed.

// the whole idea in one loop
while (run) {
    struct rte_mbuf *bufs[BURST];
    uint16_t n = rte_eth_rx_burst(port, queue, bufs, BURST);
    for (uint16_t i = 0; i < n; i++) {
        handle_frame(rte_pktmbuf_mtod(bufs[i], uint8_t *), bufs[i]->data_len);
        rte_pktmbuf_free(bufs[i]);
    }
}

The programming model is run-to-completion: each packet is handled start-to-finish inside that one loop on that one core — like an Express middleware chain inlined into a single function. No handoffs, no awaits, no scheduler between the steps.

Vocabulary you’ll meet around this loop, one at a time:

• mbuf (rte_mbuf): DPDK’s packet buffer — the sk_buff replacement, except it’s basically just a pointer plus a length.
• mempool: hugepage-backed pools of those buffers, all pre-allocated at startup, so nothing mallocs on the hot path. Hugepages are mandatory — DMA needs pinned physical memory, and GB-scale pools would thrash the TLB on 4KB pages.
• burst APIs: every poll grabs up to 32 frames at once — amortize the poll over a batch.
• rings (rte_ring): lock-free queues (single- or multi-producer/consumer — SPSC/MPMC) that lcores use to pass work to each other.
• EAL (Environment Abstraction Layer): DPDK’s runtime, which initializes all of the above. It is a framework with opinions — it takes over your process layout, your cores, and your memory at startup.

Latency: ~1–2µs wire-to-app; RX poll detection at ~100ns scale; line rate at 100G+ on a few cores.

What you just lost — recite these. You fired the mailroom, so:

• No kernel TCP/IP stack. For UDP market data that’s fine: parsing an Ethernet+IP+UDP header yourself is ~50 lines. But order entry rides TCP (for its in-order delivery guarantees, the TCP/UDP chapter) — and TCP is a mailroom service. Rebuilding it means adopting a userspace TCP stack: F-Stack (FreeBSD’s stack ported atop DPDK), mTCP, Seastar’s, or a commercial one — each an adopted codebase with its own bugs. Many shops therefore split: market data via DPDK/UDP, order entry via tuned-kernel TCP. A pragmatic hybrid worth naming in interviews.
• No tcpdump (DPDK has dpdk-dumpcap/KNI-style reinjection, but it’s extra machinery you must build and run).
• No ARP. ARP is the who-has-this-IP broadcast protocol that maps IP→MAC. The mailroom used to answer the front door; now, if your process doesn’t reply to ARP queries itself, the rest of the network slowly forgets your address and stops delivering.
• Burned cores (100% spin by design), hugepage provisioning, a NIC dedicated to one process, ops/debug retraining, and PMD/firmware/NIC-model compatibility as a permanent maintenance line-item.
• Rust story: rust-dpdk-style bindings exist but are rough; realistically you write the datapath against C FFI or in C/C++.

Solarflare/Xilinx/AMD: Onload and ef_vi — the tradfi default

Solarflare NICs (acquired by Xilinx, then AMD; today’s X2/X3 series) plus their software are the incumbent default in traditional HFT. Join a tradfi desk and this is most likely what’s in the racks. Two layers, same card:

OpenOnload: the mailroom is secretly replaced, and you never notice. Your app keeps dropping letters through the same mail slots — socket(), send(), recv(), epoll_wait(), unchanged — but behind the wall, a faster private crew has replaced the building staff. Mechanically it’s an LD_PRELOAD shim: a loader trick that injects a library ahead of libc at process start, monkey-patching the socket API. TCP and UDP now run in a userspace stack inside your process, which talks to the NIC through mapped VIs (virtual interfaces — each VI is a private slice of the NIC’s rings). The result:

• onload ./your_engine — zero code changes, no recompile — and latency drops from ~5–10µs to ~2–3µs one-way.
• Anything the shim can’t accelerate silently falls back to the real kernel.
• It stays debuggable-ish (onload_stackdump instead of tcpdump).

That “recompile nothing” property is why it won tradfi: the entire estate of existing socket code — including third-party FIX engines you don’t have source for — accelerates overnight.

ef_vi: skip the mail slots, take packages raw off the belt. This is the raw layer beneath Onload: an event-queue + descriptor-ring API against the NIC. No protocol stack at all; you build Ethernet frames yourself. ~1µs-class. The model: post receive buffers with ef_vi_receive_post(), poll ef_eventq_poll() for events, and on TX write the frame and call ef_vi_transmit(). It sits at DPDK’s level of abstraction with one big difference: per-VI, not whole-NIC. You claim specific flows/queues; the kernel keeps the interface, and ordinary traffic (ssh, monitoring) flows normally alongside your bypass path. You took over one conveyor belt, not the whole dock.

Also know the name TCPDirect (Solarflare’s ~µs “zockets” TCP built on ef_vi), and the vendor’s slogan: “Onload for the estate, ef_vi for the crown jewels.”

Cost: proprietary-ish ecosystem tied to one NIC vendor; licenses; Onload’s stack has its own tuning surface (spin options, stack-per-process layout). But operationally it’s the lowest-friction bypass in existence — which is exactly why it’s the industry default.

AF_XDP: the mailroom installs an express chute

Upstream Linux’s own answer (since ~4.18). Nobody gets fired. Instead, the mailroom bolts an express chute onto the loading dock: a small rule that says “packages matching THIS description skip sorting and drop straight into that office’s bin.” Everything else goes through normal sorting, untouched.

The rule is an XDP program — eBPF, small verified code the kernel runs safely inside itself — executing in the NIC driver at the earliest possible point, before an sk_buff is even allocated. When it matches your packets, it redirects them into a UMEM: a chunk of your process’s memory organized into frame slots. The bin on your desk.

You and the kernel then coordinate through four rings — two request/response pairs, one pair per direction:

   RX side (two rings)                  TX side (two rings)
   you ──fill ring─────▶ kernel        you ──TX ring─────────▶ kernel
        "here are empty slots"              "send these frames"
   you ◀─RX ring─────── kernel        you ◀─completion ring── kernel
        "slots now hold packets"            "sent; slots reusable"

You post empty slots on the fill ring; the driver (in zero-copy mode, supported by modern Intel/Mellanox drivers) DMAs packets straight into your UMEM; you poll the RX ring. Like DPDK you get raw frames — no TCP for you. Rust support is decent (xsk-rs, plus libbpf/aya for the XDP program).

Latency/throughput: far better than the socket path; close-to-DPDK throughput in zero-copy mode; latency typically a hair above DPDK. The reason is visible in the picture: the mailroom still owns the dock. Ring refills and TX doorbells go through the kernel driver rather than your process touching hardware directly (kernel flags — “need-wakeup” and poll mode — tune whether that mediation costs a syscall or a spin). Realistic: ~2–4µs class wire-to-app, sub-2µs achievable with busy-polling zero-copy on good drivers.

Why choose it: no NIC takeover — the interface stays a normal kernel netdev (it still appears in ip link; tcpdump, ssh, and the rest of your traffic all still work); no vendor lock; no out-of-tree drivers; plays cleanly with containers and modern infra tooling. Why not: fewer absolute-lowest-latency guarantees than ef_vi/DPDK, zero-copy support varies by driver, and you’re still writing an L2/L3/UDP parser yourself. In one line: AF_XDP is what you pick when you want 80% of DPDK’s win while remaining a normal Linux citizen.

io_uring: not bypass — and worth knowing anyway

Be precise here; interviewers use io_uring as a trap. io_uring does not bypass the network stack. The mailroom is completely unchanged — every package still gets logged, sorted, and pigeonholed exactly as in the packet-path chapter. What changes is your trips to the counter. Instead of walking down for every operation (a syscall per send/recv), you and the mailroom share an inbox/outbox tray pair:

• The inbox is the SQ (submission queue) — a ring in shared memory where you write one entry per I/O request.
• The outbox is the CQ (completion queue) — where the kernel posts each result.
• With SQPOLL, a kernel thread watches your inbox for you, so steady-state operation needs zero syscalls.
• Registered buffers and files skip per-operation reference and mapping costs.
• Multishot recv keeps a standing receive armed — one submission keeps producing a completion per arriving message.

Where it fits in trading infra: exactly the places that are latency-sensitive but not latency-critical — order gateways handling thousands of TCP sessions, logging/journaling (its original file-IO home), drop-copy (the real-time duplicate stream of your own orders and fills that goes to risk and compliance systems), market data recording, web/API layers. It’s also the modern answer to “epoll event-loop overhead” in a crypto stack: a Rust gateway on io-uring/tokio-uring with registered buffers gets meaningful syscall-batching wins with zero exotic ops burden. Saying “I’d use io_uring for the gateway tier and reserve real bypass for the market-data hot path” is a well-calibrated senior sentence.

The exotic tier: names to know

FPGA NICs / tick-to-trade — a robot at the dock. The top of the market deletes the office worker entirely. An FPGA (Field-Programmable Gate Array — a chip whose logic circuits you rewire with code, so your “program” runs as hardware, not instructions) sits on the NIC itself, pre-programmed: “if a package matches this description, immediately drop this pre-written reply into the outgoing truck.” Feed parsing, book-delta extraction, and a (simple, pre-armed) trigger all evaluate in NIC silicon, and the response order goes out wire-to-wire in ~50–500ns — vs ~1–5µs for the best software. Software still exists, but as the “slow path” (now meaning: microseconds): it computes strategy state and arms the robot with conditions and order templates. The decision was made earlier; the hardware only executes it. One consequence follows immediately: the pre-trade risk checks (price bands, size caps, kill conditions) must live in the same hardware path — a robot that fires in 500ns can also mis-fire faster than any software can stop it, so a check that runs in software is a check that runs too late.

• Platforms: AMD/Xilinx Alveo, historically Exablaze (now Cisco), Enyx cores.
• Cost: HDL/HLS skillsets (hardware description languages / high-level synthesis — how FPGA logic is written), months-long dev cycles, painful iteration. A different engineering culture.
• Below that: full-custom ASICs (an FPGA design burned permanently into silicon) at a handful of top firms, and layer-1 switch tricks — a layer-1 switch rewires signals between ports without ever reading frames, and some can hold a pre-armed order and blast it onto the wire on a trigger, skipping the server entirely.

Kernel modules / custom drivers: the pre-DPDK-era approach (put strategy logic in a kernel module to skip user/kernel crossings). Obsolete — all downside (crash the box, GPL entanglement, undebuggable) now that userspace bypass exists — but you mention it to show you know why it’s obsolete.

Comparison table

Read it through the mailroom lens: the top rows keep the mailroom and shave trips to the counter; the middle rows dodge it — AF_XDP and Onload while keeping the rest of the building’s services, DPDK by seizing the dock outright; the bottom row replaces the worker.

How you’d actually choose (the interview arc)

1. Establish the budget: measure end-to-end and the kernel share (Lab I). If transit/venue dominates (most crypto): tuned kernel + io_uring at the edges; done.
2. Colo/in-AZ with µs races: split the flows. Market data (UDP, one-way, loss-tolerant-by-design): AF_XDP if staying vendor-neutral/cloud-adjacent, ef_vi or DPDK on owned metal. Order entry (TCP, correctness-critical): Onload if on Solarflare (zero-change TCP acceleration is unbeatable ROI), else tuned-kernel TCP until proven insufficient.
3. Only then the exotic tier, bought not built, when strategy economics prove that the last µs pays for an FPGA team.

Plain-English recap

If you remember nothing else from this chapter:

• The kernel is a mailroom; bypass is delivery straight to your desk. The NIC’s queues get mapped into your process; your loop polls them. No interrupts, no kernel objects, no syscalls — and also no mailroom services. You traded the staff for raw speed.
• DPDK fires the whole mailroom and runs the dock itself: the NIC leaves the OS, one core stands at the conveyor forever, and you get ~1–2µs and total control. Like leaving the npm ecosystem: no sockets, no tcpdump, no ARP — you reimplement or go without. TCP becomes bring-your-own-stack, which is why real shops often run UDP market data on DPDK and keep orders on tuned-kernel TCP.
• Onload secretly replaces the mailroom behind the same mail slots: LD_PRELOAD swaps the socket layer under your unmodified binary — onload ./engine and existing socket code (even a vendor’s FIX engine) drops to ~2–3µs. Zero code changes is why it’s the tradfi default; ef_vi is its raw ~1µs layer for paths that justify a rewrite.
• AF_XDP is the mailroom’s own express chute: a filter in the driver drops your packets straight into your process’s memory while everything else flows normally. ~80% of DPDK’s win while staying a normal Linux citizen — no vendor lock, observability intact.
• io_uring is NOT bypass — it batches your trips to the counter, not the sorting. Packets still walk the full kernel stack; io_uring replaces syscall-per-operation with shared submission/completion trays. Right for gateways and loggers; conflating it with DPDK is the junior tell.
• FPGAs are a pre-programmed robot at the dock, not application code: software decides the strategy and arms conditions; hardware fires a pre-built order in ~50–500ns when the condition hits. The decision was made earlier — the hardware only executes it.
• Choose by measured budget share, not coolness: if venue RTT dominates (most crypto), tuned kernel + io_uring at the edges wins. Bypass pays only when the kernel’s 2–10µs is the biggest remaining term.

Interviewer will ask

“Have you used DPDK or Onload in production?” Straight answer: “No. My production latency work was zero-allocation, lock-free userspace design and kernel tuning at budgets where venue RTT dominated — bypass wasn’t the binding constraint. I know the landscape well: what I’d reach for is Onload for estate-wide TCP because it’s zero-code-change, ef_vi or DPDK for a UDP feed handler, AF_XDP if I need to stay vendor-neutral — and I know the costs: burned cores, DIY protocol stacks, lost tcpdump-style observability.” Honest, specific, decision-shaped: that’s what passes.

“Why does DPDK need hugepages?” Two reasons. DMA: the NIC writes physical addresses, so packet memory must be pinned with stable physical mappings, and huge contiguous regions make buffer→physical translation trivial. TLB: mempools span GBs; 4KB pages would thrash the TLB on every packet touch, 2MB/1GB pages keep translations resident. Bonus: hugepages are also just good hot-path hygiene outside DPDK (the kernel-tuning chapter).

“You take over the NIC with DPDK. What just broke?” Everything the mailroom did — I just fired it, so I walk its services and read off the loss and the fix for each. Sorting: no kernel protocol stack, so no sockets on that interface — ssh and monitoring agents go dark, and order-entry TCP needs a userspace stack or another NIC. Records: no tcpdump — nobody logs the packages anymore; I mirror traffic or run DPDK’s own capture machinery. The front door: no ARP — nobody answers “who has this IP,” so peers’ ARP queries time out, the network forgets my address, and delivery stops unless my datapath answers ARP itself. Standard mitigation is the split design: a second (or flow-bifurcated) NIC keeps management and TCP in the kernel, ARP answered inline in the datapath. The question tests operational understanding, and the checklist is just “what did the mailroom do for me yesterday?”

“Onload vs ef_vi — when each?” “Onload for the estate, ef_vi for the crown jewels.” Onload keeps the mail slots: the app still calls socket()/send()/recv(), but LD_PRELOAD swaps a faster private crew in behind the wall — so existing socket code, including a vendor FIX engine you have no source for, drops to ~2–3µs with zero changes and full TCP semantics. ef_vi skips the slots and takes packages raw off one conveyor belt: no protocol stack, you build the frames yourself, ~1µs — worth it only for the few paths that justify a rewrite, typically market-data ingest and the final order-TX. They coexist on the same NIC — that layering is the standard tradfi deployment.

“Is AF_XDP as fast as DPDK? Why would you pick the slower one?” The gap falls out of who owns the dock. DPDK fires the mailroom and runs the loading dock itself, so your process touches the hardware directly. AF_XDP is the mailroom’s express chute: your packets skip the sorting, but the kernel driver still owns the dock — ring refills and TX doorbells go through it, and that mediation is the residual cost. So in zero-copy + busy-poll mode they’re close, with DPDK slightly ahead at the extreme. Pick AF_XDP anyway when the ops delta matters: the NIC stays a normal netdev — tcpdump works, other traffic flows, containers behave, no vendor lock. Pick DPDK when the last few hundred ns and total control of the device justify owning the dock outright. The framing: AF_XDP is ~80% of DPDK’s win while staying a normal Linux citizen.

“Where does io_uring fit — is it kernel bypass?” No — it batches your trips to the counter, not the sorting. Every packet still walks the full mailroom path from the packet-path chapter; what changes is that instead of a syscall per operation, you and the kernel share an inbox/outbox tray pair — the SQ and CQ rings — and with SQPOLL a kernel thread watches your inbox, so steady state needs zero syscalls. That makes it the right tool for tiers that are latency-sensitive but not latency-critical: order gateways holding thousands of TCP sessions, recorders, journaling. Conflating io_uring with DPDK is the exact junior tell this question screens for.

“Sketch the architecture of an FPGA tick-to-trade system.” Feed parsing, book-delta extraction, and pre-armed trigger evaluation in NIC-resident FPGA logic; on trigger, splice a pre-built order template (IDs, price/qty fields patched in hardware) onto the wire — ~50–500ns wire-to-wire. Host software runs the actual strategy asynchronously: computes desired triggers/prices, arms/disarms the FPGA, handles everything non-critical (risk recalc, cancels, recovery). Key design point: the FPGA executes decisions already made; software makes decisions. Risk checks must be in the hardware path too — a hardware bug can send orders faster than software can stop it.

Further reading

• DPDK Programmer’s Guide and Getting Started Guide (doc.dpdk.org) — EAL, mbufs, mempools, PMDs from the source; skim the l2fwd/l3fwd sample apps to internalize the loop.
• Cloudflare blog: “Kernel bypass” (Marek Majkowski) — the classic survey of why and how to escape the stack, with measurements; pairs with their “How to receive a million packets per second.”
• OpenOnload / ef_vi documentation (AMD/Xilinx Solarflare) — the Onload User Guide’s tuning chapters double as the best public description of the tradfi deployment model.
• LWN.net: “Accelerating networking with AF_XDP” (Jonathan Corbet) plus Karlsson & Töpel’s “The Path to DPDK Speeds for AF_XDP” (Linux Plumbers) — design and performance from the authors.
• Jens Axboe, “Efficient IO with io_uring” and the “Lord of the io_uring” tutorial site — canonical io_uring model docs; read alongside the io_uring man pages.

Where this goes next: NIC Internals (ch05) — every bypass framework programs the NIC itself, so what is that card actually doing in silicon (RSS, flow steering, hardware timestamps), and how much of it survives when your “colo” is an AWS VM?