[{"content":"1. The Estate, Divided 1.1 The Promise from Twenty Years Ago Upgrading the virtualization stack is one of the oldest pains in cloud operations: the dataplane and the device models live in the host\u0026rsquo;s kernel and userspace, and the host has never been able to answer one question — where exactly should the firmware/software boundary be drawn? In today\u0026rsquo;s host almost everything is software, so almost every upgrade can only be a machine-level event: a kernel upgrade means a host reboot, and reboots bring migrations and maintenance windows.\nIn 2005 the Xen team published at HotOS a paper whose title was the question: Are Virtual Machine Monitors Microkernels Done Right? The yardstick of the question is the microkernel\u0026rsquo;s lifelong pursuit: a tiny privileged layer, services in isolated domains, fault domains independent of one another. The question asks: is the VMM the carrier that finally gets these three done? If they are done, the service stack no longer shares the host\u0026rsquo;s fate — and the opening question gets an architectural answer.\nThe paper\u0026rsquo;s answer is yes, and the argument is a necessity claim: of the VMM\u0026rsquo;s duties, what must reside in the highest privilege layer is naturally a tiny sliver — CPU/memory virtualization and isolation itself; boundaries between guests are enforced by hardware; device drivers, device models, and the control plane need none of them live in the privilege layer, and in principle all can be pushed into ordinary domains.\nBut this yes redeemed only the shape. The isolation half holds naturally; the other half — services living in ordinary domains should be restartable and upgradable like ordinary processes — and the paper\u0026rsquo;s own Xen was the first to try. The VEE'08 Improving Xen Security through Disaggregation opened the driver-domain route; the SOSP'11 Breaking Up is Hard to Do (Xoar) broke dom0 into a set of minimally privileged service domains, each guest\u0026rsquo;s device model running alone in a stubdom; on paper, the architectural loop was closed.\nBut deployment history answered with a different word. Driver domains and stubdoms entered the mainline, yet never entered production clouds: Citrix\u0026rsquo;s Windsor wanted this split and died quietly; the knot was xenstore — this global registry stayed welded inside dom0, every guest\u0026rsquo;s device tree, configuration, and credentials hanging off it, and restarting it means rebuilding the whole machine\u0026rsquo;s guest control plane; thus a dom0 restart is operationally equivalent to a host reboot. The pure-software redemption at cloud scale remains suspended to this day.\nSo service-stack upgrades have no free option: what lives in userspace restarts in place, at the cost of one stall for tenants; what lives in the kernel can only ride the host reboot — tenant migration or outage. The migrations and maintenance windows spoken of at the opening come precisely from this. What this failure shows is not that the architecture was wrong, but that it could not withstand production. Why it could not — the four obstacles of chapter 2 are the answer.\n1.2 The Industry\u0026rsquo;s Actual Answers The cloud vendors spent fifteen years each writing their own answer — but they were not answering the same question.\nAWS: move dom0 into hardware. First it fled Xen for a minimal KVM-lineage VMM, then moved the entire infrastructure stack onto the Nitro card. The promise\u0026rsquo;s \u0026ldquo;isolatable, restartable service domains\u0026rdquo; were redeemed — only the carrier changed from process domains to PCIe endpoints. The architecture is right; the placement is silicon.\nMicrosoft: dataplane down, control plane stays. From the early Catapult FPGA SmartNICs to AccelNet: network policy into FPGA and NIC silicon, host software only the control plane. Half a microkernel solution: the dataplane got a life of its own; the service stack did not.\nGoogle: ran in software for a decade first. Andromeda\u0026rsquo;s VFP — a packet-processing engine in the host kernel — proved a software dataplane can carry a hyperscale cloud\u0026rsquo;s VPC: the dataplane need not be born in silicon. But in the end Google too co-developed an IPU with Intel — the staunchest standard-bearer of the software route moved both the network and the block-storage dataplane off the host — this offload silicon Google calls Titanium externally.\nThe KVM ecosystem: gradual peeling, not structural disaggregation. vhost moved the dataplane out of QEMU\u0026rsquo;s device model, landing in the kernel; vhost-user moved it into an independent userspace daemon, restartable on its own; vDPA then handed it to hardware. Three landings, one constant direction: QEMU\u0026rsquo;s device model hollowed out piece by piece. The microVM school (Firecracker, Cloud Hypervisor, rust-vmm) trimmed the VMM itself to a minimum and rewrote it in Rust — but what it trims is the VMM, not the service stack: storage, networking, and the control plane still share the host\u0026rsquo;s fate. Isolation won; restartability did not.\nFour answers, answering isolation, dataplane, attack surface; the only one that touched restartability, AWS, locked its answer in silicon that is not for sale. In pure software, none answered the original question: can the firmware/software boundary be drawn thin enough — can everything above the firmware layer be upgraded and restarted in place?\nLooking back, the promise was redeemed piecemeal: isolation went to microVMs — Firecracker won serverless; the dataplane went to silicon — eswitch flow tables and offload engines of all kinds, present in ordinary NICs too; the on-card general-purpose compute that truly separates DPUs from ordinary NICs took the last piece: discipline. Only the plainest clause, \u0026ldquo;the service stack restarts painlessly,\u0026rdquo; has no home in pure software.\nThis is exactly the most substantive residue left by the previous article\u0026rsquo;s audit of DPUs: an independently upgradable and restartable root compute domain — engineering discipline frozen into hardware, at too high a price in hardware. This article asks the other half: can the same boundary be held in pure software?\nIt can. And every part of the idea is already in service as someone\u0026rsquo;s equipment — what is missing is only someone to assemble them into that machine.\n2. The Four Gravities The promise\u0026rsquo;s redemption in pure software has been suspended for twenty years, not because no one thought of it. The difficulty lies in four layers, like four gravities pulling the architecture back toward the monolith: the first three are engineering problems, the last an organizational one.\nThe mechanisms exist; the orchestration is missing. Restart semantics were never absent: the PV (paravirtualized) frontend/backend protocol — guest frontend and service-domain backend talking over shared-memory rings — leaves a reconnect exit in its state machine; vhost-user made reconnect an explicit feature; live migration performs the existence proof daily — a guest moves to another machine and re-establishes every backend connection under that machine\u0026rsquo;s dom0. \u0026ldquo;Replace dom0\u0026rdquo; needs no new mechanism, only tearing down and rebuilding the whole web of relationships among guests, backends, and the registry in dependency order. The difficulty is the web: xenstore is its hub, backends depend on one another, and the ordering of teardown and rebuild is itself the problem. No ecosystem has ever productized this step.\nThe bootstrap loop. dom0 holds the driver of the device its own rootfs sits on. Restarting it requires handing hardware ownership away first — storage, network, console — yet the domains taking over that ownership must themselves be launched and managed by dom0. Driver-domaining is the precondition of bootstrapping; bootstrapping is where the driver domains live. Xoar tore down the serving side; the bootstrapping side stayed where it was.\nThe gravity of performance. Every extra domain boundary costs one more notification and one more copy. The gravity of engineering optimization always points at taking the dataplane back: back into the kernel, back into the same process, back into the same silicon. vhost was one such retrieval; SR-IOV a larger one. Isolation and performance renegotiate on every generation of hardware, and performance almost always wins — so holding the boundary is not enough; holding it must cost so little that performance has no motive to defect.\nThe organizational asymmetry. The benefit of dismantling dom0 diffuses across all future upgrades — every upgrade becomes a background operation instead of a fleet maintenance window; the risk concentrates on the proposer\u0026rsquo;s incident rate this one time. No release manager wants to take this year\u0026rsquo;s outage in exchange for \u0026ldquo;next year\u0026rsquo;s operations will be smoother.\u0026rdquo; The first three obstacles have technical solutions; the fourth does not — it can only be absorbed by design: make the assembly cheap enough, and each restart\u0026rsquo;s blast radius small enough.\n3. Assembling the Machine The goal is only one: draw the firmware/software boundary as thin as possible — the thin core is the system\u0026rsquo;s firmware, the equivalent of an in-house DPU\u0026rsquo;s hardened logic: not part of the software, not a hot-upgrade target, evolving by versioned releases; everything above the firmware layer — the service stack and the control plane — can be upgraded and restarted at any time, without rebooting the host and without migration.\nTwo disciplines follow:\nThe core is firmware, not software. The thin core is the equivalent of an in-house DPU\u0026rsquo;s hardened logic: not a hot-upgrade target, evolving by versioned releases, rare events on the scale of years; CVEs between releases are absorbed by livepatch as a safety valve. It owns only what is strictly its own — the smaller it is, the fewer the releases. Service domains must be stateless. State dies wherever it lives at restart time — so all state is externalized, and nothing that cannot be lost is kept inside a service domain. 3.1 The Architecture: Three Layers The architecture has three layers, as the figure shows: at the bottom, the thin core spans the whole node — trimmed Linux + KVM, holding only CPU/memory virtualization, IOMMU, interrupt injection, and boot media; above it, two zones side by side — the host zone houses the control plane: a special VM carries the body, while a zero-state daemon in thin-core userspace executes operations and writes flow tables; the guests zone houses the function-split service VMs and the user VMs they serve. Every VM gets one passthrough VF for its own connectivity; the tenant dataplane goes through VF passthrough, never through any service VM; cache is divided by partition, host services and tenants each in their own. The paragraphs below unfold each in turn.\nThe thin core. Trimmed Linux + KVM, owning only CPU/memory virtualization, IOMMU, interrupt injection, and boot media — no dataplane, no control plane. This layer is the system\u0026rsquo;s firmware: the equivalent of an in-house DPU\u0026rsquo;s hardened logic, evolving by versioned releases; KVM itself is a kernel module, squarely within livepatch\u0026rsquo;s reach — the smaller the attack surface, the fewer the CVEs and the fewer the releases, and what slips through between releases is absorbed by livepatch as a safety valve. What must not be in the thin core is clearer than what must: no storage drivers, no device models, no toolstack; on the network side only the PF driver remains — eswitch management goes to the control-plane daemon.\nStateless control plane: a special VM on the host side. Registries, configuration, and orchestration state are all externalized — the sole repository is the cluster-level control plane; the in-node control-plane VM carries the control plane\u0026rsquo;s body, zero-state, passively receiving the cluster\u0026rsquo;s scheduling arrangements, only issuing intent — landing it is the control-plane daemon\u0026rsquo;s job, and the two restart independently. The control-plane VM\u0026rsquo;s restart likewise never stops forwarding — the dataplane lives in eswitch flow tables, unrelated to it. The xenstore lesson stands: the global registry must never again be welded into anything that needs to restart.\nThe control-plane daemon. The NIC\u0026rsquo;s PF and its driver belong to the thin core; the real control operations the control-plane VM issues — creating VMs, deleting VMs, binding and unbinding devices — are landed by this zero-state process in thin-core userspace; eswitch flow tables and per-VF policies are also written by it. Its restart is not the kernel\u0026rsquo;s restart: flow tables live in silicon, so forwarding never stops during the process\u0026rsquo;s restart; what pauses is only change. This service does not live in an isolated domain but as a process beside the thin core — device ownership (the PF cannot leave the thin core without an accompanying reset) trumps domain isolation; what is given up is fault-isolation granularity, what is gained is that restart and dataplane lifetimes are no longer entangled.\nService VMs. Split by function, not by machine: block storage and file storage each form their own domain; how many per domain is set by need — one or many. Each service VM holds its own hardware via VFIO passthrough, dataplane and protocol termination running inside; every VM — service VMs and the control-plane VM alike — also gets one passthrough VF for its own connectivity. Every service VM is a dom0 — but a stateless one: a crash counts as a restart, an upgrade is a restart, both travel the same road. The finer the split, the smaller the restart blast radius and the freer the upgrade cadence. Using VMs rather than processes buys one more thing: restart domain, deployment domain, and failure domain collapse into a single boundary — a hardware-enforced one, free of charge.\nCache partitioning. Services living on host cores share L3 with tenants: left unmanaged, service-VM traffic and daemon scans pollute tenants\u0026rsquo; cache lines, and this interference is invisible and unbookable. Today\u0026rsquo;s multi-core L3s are mostly partitionable: Intel CAT allocates cache ways per CLOS, AMD\u0026rsquo;s L3 belongs to a CCX so dedicating one CCX is physical isolation, ARM MPAM partitions by PARTID; Linux\u0026rsquo;s resctrl unifies the three under one configuration interface. The response is configuration, not prayer: concentrate the CPUs used by host services — service VMs and each daemon — into one or two cache partitions (on AMD, literally one CCX), tenants use the rest. Pollution thus goes from an uncontrolled externality to a boundary drawn by configuration: how much cache services may occupy is written in config, not in luck.\n3.2 Networking: The Dataplane Belongs to the NIC Guests hold SR-IOV VFs; VPC rules are pushed down into the NIC\u0026rsquo;s eswitch flow tables; per-VF rate limiting and QoS are standard NIC capabilities. What remains for the control-plane daemon is only out-of-band work: programming rules, collecting state.\nThus the control-plane daemon\u0026rsquo;s restart becomes boring: rules stay in silicon, forwarding never stops; what pauses in its restart window is only \u0026ldquo;change\u0026rdquo; — config pushes, security-group edits, queued for tens of seconds. Azure\u0026rsquo;s Accelerated Networking has run this pattern for years: policy in hardware, host bypassed.\nThe expressiveness ceiling of eswitch flow tables and match-action pipelines decides that \u0026ldquo;dataplane to the NIC\u0026rdquo; is forever only partial: what pushes down is the stable and hot match-action — L2/L3 forwarding, encapsulation, per-VF rate limiting; what does not is the long tail — stateful connections, complex NAT and load-balancing semantics, new protocols. This is not a cost item but a design constraint: it dictates that the software path is not a transitional measure but a permanent component, the hardware fast path an accelerator rather than the sole execution point. The ceiling itself is unrelated to merchant vs in-house — in-house DPUs freeze at tape-out too, this being the embodiment in hardware of chapter 2\u0026rsquo;s \u0026ldquo;iteration agility\u0026rdquo; counter-argument; programmable pipelines (P4-class match-action, vendors\u0026rsquo; flow programming APIs) push the boundary outward, but the boundary does not disappear, and it moves with someone else\u0026rsquo;s roadmap.\nTwo points need further explanation:\nVF migration is now supported. mlx5\u0026rsquo;s vfio migration variant driver is merged into mainline (Add mlx5 live migration driver), and merchant NIC VFs can migrate from now on; the mature production practice is the synthetic fallback path: dynamically revoke and restore VFs around maintenance events, applications bound to the synthetic device to keep connectivity — the official documentation says so explicitly. Two caveats: first, what is standardized is the migration framework, not the state — the migration data stream is vendor-opaque, coverage limited to the same series; second, cross-vendor migration has no path — but this is not fatal, cross-CPU-model/vendor live migration is likewise constrained, and fleets are homogeneous by construction. The offload boundary is drawn by others. Which policies push down depends on NIC generation and vendor; the two sides of the boundary are heterogeneous — inside the card, the vendor\u0026rsquo;s flow API; outside, our software; the long tail pays one extra hop when crossing. In-house DPUs have ceilings too, but own both sides of the boundary and catch the long tail on on-card cores; this design trades that ownership for not building silicon. 3.3 Storage: Protocol Termination in a Service VM Guests see only virtio; the storage service VM terminates the guest\u0026rsquo;s block requests at the front and meets the storage cluster at the back with a storage protocol — RDMA or TCP, NVMe-oF or proprietary, deployment-dependent; credentials and topology invisible to tenants.\nIts restart window is the most engineered part of the whole design, yet needs no redundant instances. In most cases what restarts/upgrades is the userspace storage service daemon — process-level restart, seconds or faster; only when the service VM\u0026rsquo;s kernel itself must be swapped — rare — does kexec preload the new kernel and compress the restart to seconds. The stall the guest feels equals the restart duration; the guest side\u0026rsquo;s absorption capacity is far wider:\nvirtio-blk has no hard timeout, and the guest\u0026rsquo;s filesystem retries on its own, with an absorption ceiling on the order of tens of seconds — this is margin, not expected stall; on the backend side, the storage protocol\u0026rsquo;s reconnect and retry semantics provide the backstop. Multiple instances exist first for isolation: one instance\u0026rsquo;s crash or restart touches only its own share; second, to make restarts easier: rolling per instance, so a single window need not gamble the whole.\n3.4 No New Mechanisms, and Every Segment Walked Nothing in this design is new: the reconnect semantics cited in chapter 2\u0026rsquo;s first layer, plus VFIO\u0026rsquo;s unbind and rebind, are the entire inventory the assembly needs. What Xen lacked was never mechanisms, but turning \u0026ldquo;replace dom0\u0026rdquo; into one orchestrated action. And every segment of this road has been walked by someone:\nNitro is this architecture — only the service VMs moved onto the card. What it proves is that the architecture holds at cloud scale; whether the service domain lives in host memory or on-card silicon is a commercial question, not an architectural one. AccelNet proves the other half: a hardware dataplane plus a software control plane can serve for years. What no one has walked is assembling them back into pure software — that is this chapter\u0026rsquo;s work.\nConclusion The 2005 question remains open to this day, but the shape of the answer is already clear. This article\u0026rsquo;s assembly lets the host answer the opening question: the firmware/software boundary drawn as thin as possible — everything above the firmware layer can be upgraded and restarted at any time, without rebooting the host and without migration, what tenants feel at most an absorbable stall; the firmware layer itself, like an in-house DPU\u0026rsquo;s hardened logic, evolves by versioned releases. The microkernel VMM needs no new mechanisms; what it lacks is one serious assembly — and every block the assembly needs has already run in someone\u0026rsquo;s fleet for years: reconnect semantics, VFIO, kexec, eswitch flow tables, cache partitioning — none invented here.\nThe reasonableness of this architecture lies in making every boundary explicit and cheap: VM boundaries plus the IOMMU hold in software the boundary the DPU holds with a single PCIe bus — hardware-enforced, free of charge; the tax is booked on fungible host cores, not stranded on-card silicon; the offload boundary and the cache partitions are written in configuration, not in vendor roadmaps or luck. What software buys back is iteration: no tape-out freeze, new semantics need not wait for the next silicon.\nIsolation was never the hard part. Maintenance was — and what this article gives back to maintenance is its ordinariness: a crash, an upgrade, both count as one restart.\n","date":"2026-08-24T00:00:00Z","permalink":"/en/p/microkernel-vmm-done-right/","title":"The Microkernel VMM, Done Right"},{"content":"1. A Successful Pattern Doesn\u0026rsquo;t Redeem a Failed Market The DPU narrative has been told for years: \u0026ldquo;zero virtualization tax\u0026rdquo;, \u0026ldquo;performance gains\u0026rdquo;, \u0026ldquo;security isolation\u0026rdquo;, \u0026ldquo;bare-metal unification\u0026rdquo;, \u0026ldquo;iteration agility\u0026rdquo;, \u0026ldquo;energy efficiency\u0026rdquo;, culminating in its coronation as \u0026ldquo;the third pillar of computing\u0026rdquo; after CPU and GPU. But the market\u0026rsquo;s answer sits awkwardly with this narrative: it has rewarded the self-builders, but not the buyers. This article audits the narrative claim by claim — first the market, then the value propositions, then the costs, and finally what is left.\nEvery large-scale DPU success story comes from a hyperscaler\u0026rsquo;s in-house, vertically integrated program: AWS acquired Annapurna Labs in 2015, and the Nitro system made its full debut with C5 instances in 2017; Alibaba Cloud unveiled its X-Dragon architecture in 2017 and launched CIPU in 2022; Microsoft started with its in-house Catapult FPGA SmartNIC, acquired Fungible\u0026rsquo;s team in 2023, and shipped the in-house Azure Boost DPU in 2024. Huawei\u0026rsquo;s QingTian, Baidu\u0026rsquo;s Taihang, and Tencent all follow the same in-house path. The only top player that comes close to \u0026ldquo;procurement\u0026rdquo; is Google, which co-designed Mount Evans with Intel — but that is custom silicon built to Google\u0026rsquo;s spec, not an off-the-shelf purchase.\nThe merchant DPU market, meanwhile, is a complete wreck. Fungible raised over $300M, then was acquired by Microsoft in early 2023 — reportedly for only about $190M. Pensando was acquired by AMD in 2022 and has faded from view since. Nebulon quietly folded, with its team reportedly absorbed by NVIDIA. Intel\u0026rsquo;s IPU product line has been reworked repeatedly. NVIDIA BlueField\u0026rsquo;s public deployments concentrate in niches such as AI networking. For the enterprise market, vSphere DSE (vSphere on DPUs) never saw adoption at scale — and its DPU firmware update requires a host reboot, with vMotion evacuating the VMs to keep them alive. Replacing seamless upgrade with evacuation is itself the most honest vote.\nChinese independent DPU vendors are a copy of the same verdict, only starker. The furthest along, JaguarMicro, is now rushing toward a Hong Kong IPO as \u0026ldquo;China\u0026rsquo;s first DPU stock\u0026rdquo; — yet its prospectus shows three-year revenue growing from ¥170K to ¥370M against cumulative losses of nearly ¥2.5B, with over 90% of revenue from a single customer: Tencent — which is also its largest shareholder, and which builds its own DPU anyway. Largest shareholder plus 90% of revenue: that is not an independent vendor, it is one cloud\u0026rsquo;s department in all but name. As for the \u0026ldquo;truly independent\u0026rdquo; rest: Zuojiang\u0026rsquo;s DPU business was found by regulators to be \u0026ldquo;severely misrepresented\u0026rdquo; — confirmed financial fraud, delisting, and its controller detained. YUSUR is still living on venture rounds, its latest C+ round at a pre-money valuation of about ¥5B; its market has narrowed to financial low-latency trading — which is no longer cloud at all — while on the private-cloud side it has only project-scale cases like a bank\u0026rsquo;s cloud platform and a university edge cloud, with no access to the public-cloud mainstream; publicly it offers only unverifiable claims like \u0026ldquo;orders doubled\u0026rdquo;, and has never filed a prospectus — the silence about its financials is itself an answer. NebulaMatrix, YunSilicon, Dayu and the other startups from the same cohort have gone quiet. In other words, no independent Chinese DPU vendor has made it — the only one to reach the IPO door got there precisely by ceasing to be independent.\nIf the DPU\u0026rsquo;s technical value were substantive, it would have rewarded buyers as well as self-builders; in reality it has rewarded only self-builders. That means its margin is so thin that it turns positive only once all vendor margin is stripped out, at BOM cost. A successful pattern cannot excuse a failed market — on the contrary, the split itself is the market\u0026rsquo;s verdict.\n2. An Attribution Audit of the Value Propositions The DPU narrative consists of seven value propositions. Audited one by one, five do not survive.\n\u0026ldquo;Zero virtualization tax\u0026rdquo; — relocation, not elimination. The virtualization stack does consume some fraction of host compute — the so-called \u0026ldquo;virtualization tax\u0026rdquo;. But a DPU merely moves that tax from host cores to card cores, running the same protocol stack and the same device models; not one line of work disappears, only the ledger changes — from sellable host CPU to a one-time sunk card cost. That is at best a currency conversion, and what you convert into is stranded capital: host cores not doing infrastructure can still run tenant workloads, while card silicon can only ever run infrastructure.\n\u0026ldquo;Performance gains\u0026rdquo; — the hardware pipeline is not the DPU\u0026rsquo;s. Line-rate small-packet forwarding is indeed hardware territory; software suits only low-speed virtual networks. But the hardware pipeline exists in ordinary SR-IOV NICs. What a DPU actually adds — its programmable part — runs on weak ARM or RISC-V cores that are slower than the host CPU. Beneath the \u0026ldquo;hardware offload\u0026rdquo; packaging, it is the same software deployed onto the card — it even relies on SR-IOV, enumerating VFs exactly the way an ordinary NIC does. The accurate translation of \u0026ldquo;hardware offload\u0026rdquo; is: the same software moved onto weaker hardware. And that weakness only becomes more of a bottleneck as link speeds march into the hundreds of Gbps.\n\u0026ldquo;Security isolation\u0026rdquo; — the boundary is one-way. Virtio is just a pair of shared-memory ring queues; security depends on the device implementation, not on where it runs. Card firmware is closed-source, less audited, and slower to patch, while the QEMU device model it replaces is continuously fuzzed in public. More critical is DMA: as a PCIe device the card inherently holds access to host memory, and on bare metal the IOMMU cannot trim it — compromising the card means compromising the host. The DPU\u0026rsquo;s security boundary stops the tenant from touching the platform; it never stops the card from harming the tenant.\n\u0026ldquo;Bare-metal unification\u0026rdquo; — real, but worth only half a claim. When the host is untrusted, cloud-disk and cloud-network protocols must be terminated on a card — this is the DPU\u0026rsquo;s hardest-to-replace function. But it is being eroded from both ends. Cardless bare metal (switch-scoped VXLANs, iSCSI/NVMe-oF remote boot) is a legitimate, merely thinner product, as SoftLayer and Equinix Metal proved long ago. And for customers demanding full VPC/EBS semantics, a VM occupying (nearly) the whole host approaches bare-metal performance while inheriting all cloud semantics. The true irreducible demand shrinks to three cases: bring-your-own hypervisor stacks, licenses bound to physical hardware, and compliance clauses that literally mandate dedicated hardware.\n\u0026ldquo;Iteration agility\u0026rdquo; — actually the opposite. Pipeline features on the card are frozen at tape-out; new protocols wait for the next silicon generation; firmware and drivers upgrade in lockstep. The DPU does not decouple software from hardware — it couples iteration to hardware generations. Meanwhile, upgrading the dataplane without disturbing tenants is something host software does just as well.\n\u0026ldquo;The third pillar of computing\u0026rdquo; — negated by its own name. DPU is a rebranding NVIDIA pushed after acquiring Mellanox. The only defensible reading is \u0026ldquo;a fixed menu of domain-specific acceleration\u0026rdquo; — and domain-specific self-refutes \u0026ldquo;general\u0026rdquo;.\n\u0026ldquo;Energy efficiency\u0026rdquo; — the ledger is bankrupt. The perf-per-watt of modern x86 and ARM host CPUs voids the premise that the card\u0026rsquo;s ARM cores save power; the card itself draws 75W+ against roughly 25W for an ordinary NIC — a net increase in power.\n3. But What\u0026rsquo;s the Cost? Even if you doubt the entire audit above, the DPU\u0026rsquo;s cost list stands on its own, and it is hard:\nHardware is irreversible. Once taped out, nothing can be changed: how many I/O queues, how many VFs — fixed at manufacture; any design flaw ships as an erratum and stays for the product\u0026rsquo;s whole generation. A software defect is a hotfix; a hardware defect is a generation.\nOne more failure domain per server. Merchant DPUs generally run a full Linux distribution on-card — kernel CVEs and firmware updates are all still there, just moved onto a device with worse tooling and a slower release cadence. A card-firmware bug is a fleet-wide correlated event — the failure domains multiply exactly with the fleet.\nCreating a single point of failure on bare metal. With cardless bare metal, cloud-side maintenance is transparent to the tenant: switches are redundant, storage is multipathed, upgrades never touch the tenant\u0026rsquo;s machine. The DPU inserts the only cloud-controlled failure domain that needs recurring maintenance into the tenant\u0026rsquo;s dedicated chassis — and bare metal has no live migration as a fallback, so a card firmware reboot lands directly on the tenant\u0026rsquo;s SLA. \u0026ldquo;Seamless restart\u0026rdquo; is supposed to be the answer, yet its delivery is: real inside two or three closed hyperscaler fleets, unauditable; slideware in the open market; and on bare metal — where it is needed most — backed by no public data at all.\nOrganizational costs are systematically undercounted. Entire silicon and firmware teams, ASIC tape-outs, fleet incidents — these costs are real, yet no public ROI story has ever disclosed them, while the benefits can be booked precisely at vCPU prices. More subtly, the card\u0026rsquo;s existence freezes the basis of comparison: the pure-software alternative was never built, the controlled experiment can never be run, and sunk NRE makes \u0026ldquo;keep funding the card\u0026rdquo; look rational forever.\n4. What Remains At the end of the audit, one genuine thing remains: an independently upgradable and restartable root compute domain — the infrastructure service stack gains a lifecycle independent of both the host kernel and the tenants, and its upgrades and restarts disturb no one. This is the most substantive residue after all seven claims are sifted, and it is the twenty-year-old promise of microkernel architecture: the title of the Xen team\u0026rsquo;s 2005 paper, Are Virtual Machine Monitors Microkernels Done Right?, was a question — deployment history answered \u0026ldquo;no\u0026rdquo;, as dom0 grew irresistibly into a monolith and could not resist architectural rot. But Xen\u0026rsquo;s failure to build it does not mean software cannot.\nWith one PCIe bus, the DPU turns a boundary that software discipline could not hold into one that is physically uncrossable. That is its most respectable technical footnote: engineering discipline frozen in hardware — organizational value, not technical value. It is just that the price of achieving this with hardware is too high.\n5. Conclusion The full structure of the DPU myth: seven value propositions, five fail the attribution audit; one is real but software-replaceable — and replaceable means it does not stand; half a claim is real but niche. Seven claims, half left standing, and behind them a non-trivial bill of costs. Its success belongs to two or three hyperscalers in a particular historical window — as much chance as design; its failure belongs to the whole market\u0026rsquo;s open appraisal of its technical value — and the latter is the truth of pricing. \u0026ldquo;Everyone is using it\u0026rdquo; was never evidence of technical value; it may only be evidence of historical decisions, organizational inertia, and a signaling equilibrium.\nThin value wrapped in a thick narrative is the standard recipe for a myth. And this myth should end.\nSupplement on 2026.08.31 In-house cards make VM migration easy — and for a long time that was true. The obstacle was never \u0026ldquo;in-house versus merchant\u0026rdquo;; it is whether device state can stay consistent across migration: with bare VF passthrough, queue contexts live inside the NIC, and exporting them completely and restoring them on the new host\u0026rsquo;s card was beyond what merchant hardware could do — high performance and migratability thus did not go together. The in-house solution was to have a program hold the state: the guest\u0026rsquo;s device is provided by firmware on the card (ena under Nitro, virtio under X-Dragon), and at migration time the new host\u0026rsquo;s card simply instantiates it again, and the guest notices nothing. And it need not be a DPU: any card that can impersonate the guest\u0026rsquo;s device will do. For years only the in-house builders could walk this path.\nThat page has turned, on both roads. The earlier one took the software-protocol interface path — mlx5\u0026rsquo;s vDPA mode (VDPA support for Mellanox ConnectX devices) has presented virtio devices to guests since 2020, with the data path in hardware; and for those who stayed with bare VF passthrough, the mlx5 vfio migration variant driver landed in the mainline kernel in 2022 (Add mlx5 live migration driver), and NVIDIA\u0026rsquo;s official documentation now provides full SR-IOV Live Migration support — with state export and restore in place, merchant NIC VFs can migrate too. This once-exclusive value has been worn flat by the merchant path — what remains is the head start and roadmap ownership: procurement, not technology.\n","date":"2026-08-20T00:00:00Z","permalink":"/en/p/end-of-dpu-myth/","title":"Time to End the DPU Myth"},{"content":"In the design of randomly readable compressed storage, the data layout must first answer one question: where to draw the compression boundary. Fixed-input splits the input into fixed-size units, compresses each, and packs the variable-length outputs back to back; fixed-output goes the opposite way — variable-length input is \u0026ldquo;stuffed\u0026rdquo; into fixed-size output blocks, so that compression units align naturally with storage blocks.\nFixed-output was proposed in [1], and it arrived with a string of fine stories: eliminating read amplification, aligning I/O, improving compression ratio, saving memory. This essay checks the fate of each: some of these stories credit the wrong owner, some are staged only inside a restricted framing, and some cannot be found in measurement at all — while the costs stay in effect forever. As a format design decision, fixed-output has no reason to exist. This essay rejects the technique\u0026rsquo;s value as a general design choice; it says nothing about any particular system built on it. A system\u0026rsquo;s success is decided by implementation quality, operations, and ecosystem together, and an unsound mechanism does not necessarily prevent a successful system.\n1. Read Amplification and Alignment: Two Fairy Tales The mechanism of read amplification is simple: decompression must start from the head of a compressed stream (algorithms like LZ4 cannot seek into the middle of a stream), so the larger the unit, the more I/O and decompression a small request costs. It depends only on the unit size, not on whether the input or the output is fixed-length.\nStart with [1]\u0026rsquo;s own measurements (16MB read, actual I/O issued; the stride read touches only the first 4KB of every 128KB):\nLayout Unit size Random read Stride read fixed-input 128KB 165.27 MB 203.91 MB fixed-input 4KB 26.19 MB 26.23 MB fixed-output 4KB (output) 26.12 MB 25.93 MB At 4KB granularity the two layouts issue nearly identical I/O; the six-to-eightfold difference in amplification comes entirely from unit size. Here the first fairy tale takes shape: the elimination of read amplification is granularity\u0026rsquo;s credit, not the layout\u0026rsquo;s — and the comparison that proves it was given by the very paper that proposed the technique.\nThe same table hides a second fairy tale. Fixed-output\u0026rsquo;s blocks align naturally with storage blocks, while fixed-input\u0026rsquo;s variable-length blocks pack tightly and inevitably straddle block boundaries, so a read must fetch two partially-used physical blocks at the ends — a theoretical \u0026ldquo;rubble\u0026rdquo; loss. The theory holds; the dividend was never paid: the two layouts issue identical total I/O — the rubble bytes belong to neighboring units and are equally useful to later reads, and the cache irons this theoretical loss flat.\nThe key at the mechanism level is that the cache picks up data fetched along the way but not yet used, and the two layouts cache different things. Fixed-input only needs to cache compressed data, while fixed-output must cache decompressed data, or the decompression work is wasted. The cache layer not only fails to favor fixed-output — it sides with fixed-input.\n2. Compression Ratio: A Victory in a Restricted Framing Against fixed-input of the same unit size (4KB vs 4KB), fixed-output does win about 10% in image size [1] — a great victory, and the mechanism is clear: \u0026ldquo;stuffing\u0026rdquo; makes each compressed stream cover a longer input, and longer streams compress better.\nBut this victory avoids the realistic opponent. Fixed-output with 4KB output produces images 11% to 29% larger than large-window fixed-input (128KB units) [1] (two corpora measured: 0.52GB vs 0.47GB; 100.9MB vs 78.0MB). To recover that gap, fixed-output can only enlarge its output blocks — that is, turn the granularity knob back up and invite read amplification back in. The technique welds compression ratio and random-read performance onto a single knob, and turning it either way is a concession.\nThere is also a small bill on the storage side: fixed-input packs back to back without wasting a byte; every fixed-output block leaves its tail unfilled — internal fragmentation built in.\n3. Apples to Apples: The Tale of Twin Brothers The first two sections hid a framing trap: fixed-input\u0026rsquo;s label measures the input, fixed-output\u0026rsquo;s label measures the output — both tagged \u0026ldquo;4KB\u0026rdquo;, yet the streams are not the same length. A true like-for-like comparison gives every compressed stream the same logical coverage. At a 2:1 ratio, for example, fixed-input\u0026rsquo;s 256KB unit and fixed-output\u0026rsquo;s 128KB output block are two cuttings of the same stream. Recount line by line (serving one 4KB random read):\nfixed-input, 256KB unit fixed-output, 128KB output One stream 256KB input → ~128KB compressed ~256KB input → 128KB compressed Read I/O whole stream, ~128KB whole stream, 128KB Decompression from head to target output, half a stream on average from head to target output, half a stream on average Read amplification ~32× ~32× Compression ratio identical identical Everything zeroes out: between twin brothers, there is no question of who stands higher. One corollary follows — Section 2\u0026rsquo;s 10% \u0026ldquo;advantage\u0026rdquo; is itself a framing illusion: under the \u0026ldquo;4KB vs 4KB\u0026rdquo; label, fixed-output\u0026rsquo;s streams are actually twice as long. The source of the advantage is the stream-length difference, not the layout; level the stream length and the ratio gap zeroes out. A 128KB fixed-output block ≡ a 256KB fixed-input unit plus alignment — and the alignment dividend was never paid (Section 1).\nWhat remains under equal framing is entirely structural: fixed-input has a sparse index, arithmetic-free logical locating, and zero storage waste; fixed-output has block alignment and a priori known I/O sizes (Section 6). In a fair comparison, the performance and compression ledgers hold no surplus that belongs to the layout.\n4. The Arithmetic of Locating a Read: Flowers in the Mirror Locating a read takes two mappings: logical offset → compression unit, and compression unit → physical address. Each layout gets exactly one of the two steps for free, in opposite directions:\nfixed-input: offset ──[÷C, arithmetic]──→ unit ──[lookup]──→ physical fixed-output: offset ──[÷4K]──→ logical block ──[?]──→ unit ──[×4K]──→ physical The point: fixed-output\u0026rsquo;s free multiply step sits at the end of the chain, while queries always start from the logical offset — this arithmetic convenience is flowers in a mirror, visible but forever out of the read path\u0026rsquo;s reach. The missing \u0026ldquo;logical → unit\u0026rdquo; mapping is data-dependent and cannot be computed arithmetically; two choices remain: a dense index (one 8-byte entry per logical block [1], an order of magnitude denser than fixed-input\u0026rsquo;s per-unit index for large windows), or a sparse index plus binary search (O(log n)).\nFixed-input has no such problem: subtracting two adjacent entries of the offset table yields the compressed unit\u0026rsquo;s length, so position and length arrive in a single lookup. On the locating axis, fixed-output\u0026rsquo;s ledger records only outlays.\n5. Addressability: A Legend Beyond [1], a certain expectation of fixed-size alignment still circulates in discussions of distribution formats: on-demand distribution, lazy loading, block-level deduplication all seem to presuppose fixed-size blocks — and this is exactly where the legend begins to distort. Fixed-input plus an offset table makes variable-length blocks equally addressable — the address is the (offset, length) pair in the index entry, which combined with HTTP range requests forms a complete on-demand distribution. Production-grade fixed-input distribution schemes have long existed (OverlayBD\u0026rsquo;s ZFile [2]; stargz\u0026rsquo;s TOC [3]), at enormous scale.\nThe legend\u0026rsquo;s true source is now located: addressability comes from the existence of an index, not from fixed-length output.\n6. Memory Savings: A Medal for the Wrong Recipient [1] carries one more claim, of memory efficiency: fixed-output makes in-place decompression possible, each decompression reads at most two compressed blocks whose sizes are known in advance, so memory use is bounded. This claim likewise fails the attribution test — every key memory-saving technique is credited to the decompressor.\nFirst, partial decompression: decode from the stream head and stop at the target output, leaving the bytes beyond untouched. This is a direct corollary of sequential decoding and holds identically for both layouts. Second, rolling decompression: with the algorithm\u0026rsquo;s sliding window (e.g., LZ4\u0026rsquo;s 64KB window), only a window\u0026rsquo;s worth of history pages must be kept to sustain decoding — the window is a property of the algorithm, not the layout. Third, filling target pages directly: decompression output is written straight into the destination page cache, eliminating a temporary output buffer; fixed-input\u0026rsquo;s read path can do exactly the same. As for \u0026ldquo;sizes known in advance\u0026rdquo;, Section 4 already supplied it — the compressed unit\u0026rsquo;s length is the difference of two adjacent offset-table entries, hardly a monopoly of fixed-length output.\nThe reverse ledger is worth recording too: fixed-output pays more complexity for this path. It requires the compression algorithm to offer a fixed-output interface (destSize) — a format-level constraint; locating needs a dense per-logical-block index; and [1]\u0026rsquo;s in-place decompression is itself not free, requiring the image builder to simulate decompression and rule on feasibility block by block. This medal belongs to the decompressor, not the layout; the complexity fixed-output carries for it is not one whit less.\n7. The Costs: The Format\u0026rsquo;s Tattoo The first six sections audited the stories on the benefit side; the cost side\u0026rsquo;s ledger is kept in this one.\nFirst, engineering complexity, whose final form is code volume. At both ends fixed-input is a short loop: on the build side — split, compress, append, record the offset; on the read side — one table lookup, one decompression call. Fixed-output has no such straight line: every structural difference in the preceding sections ultimately lands as code — a state machine on the build side, a trail of branches on the read side, and the code volume multiplies severalfold. This overhead is not billed per use but per implementation: kernel driver, userspace tools, language bindings — every adopter writes it all over again.\nSecond, build speed. Mainstream compression libraries shape their trunk interface for fixed-input: fixed-size input, variable-size output, one call — and all their optimization effort concentrates there. The fixed-output (destSize) interface exists only as a side-branch variant in some algorithms; to avoid it, one must trial-compress with the generic interface — feed some input, watch whether the output crosses the block boundary, roll back and recompress when it does. Both roads are slow; and every library upgrade lands its dividend on the trunk first — fixed-output watches from across the shore.\nThird, the evolution burden. Fixed-input keeps its freedom outside the format: the unit size is a parameter each image chooses for itself, and indexing and decompression strategies are implementation details, adjustable at will. Fixed-output writes the key decisions into the format: fixed-size output, block alignment, per-block in-place flags — all clauses of the contract. Every future format extension, every algorithm swap, every read-path optimization must carry this contract along; it cannot be revised.\nA story\u0026rsquo;s flaw lives in the telling — change the framing or run one measurement and it shows itself; the costs\u0026rsquo; flaw lives in the definition — as long as the format is fixed-output, every image and every implementation pays. Implementations can iterate; a published format can only be complied with: internal fragmentation will not vanish for a cleverer builder, the locating lookup will not be saved by a more diligent cache, and the build-time interface mismatch will not be healed by library upgrades. These costs compound with every use of the format — this is what \u0026ldquo;format-level and permanent\u0026rdquo; means.\nAnd so the whole account closes: every claimed benefit, the opponent can obtain for free by implementation means — granularity, index, decompressor, none of which needs fixed-length output; while the costs are all carved into the format itself. Stories fade; tattoos do not.\nConclusion With attribution done, the myths bow out one by one: read amplification goes to granularity, memory efficiency to the decompressor, the alignment dividend cannot be found anywhere, the compression-ratio victory is staged only in a framing that avoids the realistic opponent; the addressability expectation beyond [1] likewise goes to the index — every claimed advantage either vanishes or changes owner. The costs all stay on fixed-output\u0026rsquo;s own ledger: locating loses its arithmetic, output blocks carry internal fragmentation, compression algorithms are constrained to a destSize interface, engineering complexity on both the build and read sides, slower builds, and an evolution contract that cannot be revised — format-level, permanent.\nOur conclusion is therefore polite but total: this technique is not worth adopting. In fact, since its proposal in 2019, no second system has adopted it — its only carrier remains the filesystem that proposed it. When designing a read-only compressed layout, the right move is to keep the freedom on the granularity axis (choose unit sizes per workload), build the index well, and use fixed-input.\nTwo methodology notes to close; they will outlast this essay\u0026rsquo;s specific conclusions:\nThe attribution test. When you see a claimed advantage, first ask \u0026ldquo;can an equally well-implemented opponent obtain it?\u0026rdquo; Being able to answer this question makes most technical marketing show its true form on the spot. The time test. The theses of most papers and techniques fade with time; that is the norm. The survival of an artifact is not the survival of its argument — a system may be thriving while its central claim has long been quietly replaced by its own evolution. References [1] Xiang Gao, Mingkai Dong, Xie Miao, Wei Du, Chao Yu, Haibo Chen. \u0026ldquo;EROFS: A Compression-friendly Read-only File System for Resource-scarce Devices.\u0026rdquo; USENIX Annual Technical Conference (ATC \u0026lsquo;19), 2019. https://www.usenix.org/conference/atc19/presentation/gao\n[2] Huiba Li, Yifan Yuan, Rui Du, Kai Ma, Lanzheng Liu, Windsor Hsu. \u0026ldquo;DADI: Block-Level Image Service for Agile and Elastic Application Deployment.\u0026rdquo; USENIX Annual Technical Conference (ATC \u0026lsquo;20), 2020. https://www.usenix.org/conference/atc20/presentation/li-huiba\n[3] stargz Snapshotter: eStargz (seekable tar.gz) lazy-pulling image support. https://github.com/containerd/stargz-snapshotter\n","date":"2026-08-17T00:00:00Z","permalink":"/en/p/against-fixed-output/","title":"The Myth of Fixed-Output Compression"},{"content":"Containers, VMs, and agent sandboxes all start the same way: from an image. Yet the mainstream image formats were each designed for exactly one of them — OCI tar+gzip layers for containers, qcow2/VHD/VMDK for VMs — and each carries structural costs that only grow with scale.\nOverlaybd takes a different route: a layered, lazily-loaded, seekably-compressed block-device image format. One format serves containers, VM-isolated secure containers, agent sandboxes, and full virtual machines alike — and it has been proven for years in some of the largest production fleets in the world. This post synthesizes the case.\nThe Two Camps, and Why Both Fall Short Existing image stacks fall into two architectural camps:\nFilesystem-based images. The dominant OCI tar+gzip format served through overlayfs; lazy-pull variants (eStargz, SOCI) that defer extraction but keep tar and filesystem semantics; single-filesystem images (squashfs, EROFS); and FUSE-based formats (Nydus/RAFS). Block-based VM disks. qcow2, VHD/VHDX, and VMDK — each a virtual block device plus a fixed-granularity allocation table, chained one file per snapshot. Measured against what these workloads actually need — sub-second cold start, efficiency at thousands of concurrent instances, stability and security as business-critical infrastructure, and one format across runtimes and guest OSes — both camps fail structurally.\nCold start The standard OCI image must be downloaded, decompressed, and extracted in full before the process can run — even though only a small fraction of image data is touched during startup. For multi-gigabyte images that means tens of seconds to minutes. P2P tools (Dragonfly, Kraken) speed up distribution but cannot skip the download-and-decompress steps. Lazy-pull variants defer extraction yet remain bound to archive structure and filesystem semantics.\nO(n) lookup with depth Filesystem stacking resolves every file access — open, stat, readdir — by walking each layer\u0026rsquo;s directory tree top-down: O(n) in the number of layers. The VM camp has the same disease in a different shape: a read miss in a qcow2/VHD/VMDK backing chain falls through parent files until some snapshot holds the data. Vendor guidance says the quiet part out loud — VMware supports at most 32 snapshots per chain and recommends 2–3; Microsoft flags \u0026ldquo;more than 50 checkpoints\u0026rdquo; as a problem; QEMU ships block-commit/block-stream to collapse chains. The distribution model demands deep stacks (base OS, runtime, dependencies, app, per-tenant customization); the formats only tolerate shallow ones.\nIndex memory, multiplied by depth qcow2\u0026rsquo;s two-level L1/L2 table runs ~12.5 MB per 100 GB of virtual disk at the default 64 KB cluster size — per file, and every link in a backing chain carries its own table and cache. A 30-snapshot chain has a 30 × 12.5 MB index problem. QEMU\u0026rsquo;s default L2 cache is only ~1 MB, forcing a choice between wasting host memory or wasting I/O on table misses. VMDK and VHD/VHDX sit on the same trade-off curve at different points (smaller tables bought with coarser grains).\nThe granularity dilemma For fixed-granularity tables, index size is proportional to virtual size ÷ cluster size, while copy-on-write cost is proportional to cluster size. Small clusters: cheap writes, huge tables. Large clusters: tiny tables, but a 4 KB write can trigger a multi-megabyte copy. No fixed-granularity format escapes this curve — you can move along it, but not off it. Filesystem formats pay the analogous tax as file-level copy-up: the first modification to lower-layer data copies the entire file.\nServing-path complexity and security Filesystem-based formats must parse complex metadata (xattrs, ACLs, symlinks, device nodes) in host or hypervisor context — a large attack surface facing untrusted guest content. Served into a VM, every metadata operation becomes a FUSE/virtio-fs round-trip; a single open() on a nested path may cost five or more cross-VM round-trips. Worse, virtio-fs/9p have been shown to let guests bypass memory-cgroup and ephemeral-storage accounting entirely (kata-containers#12203). And when the userspace FUSE daemon dies, the mounted filesystem typically hangs — a serious liability for infrastructure that serves around the clock.\nOverlaybd\u0026rsquo;s Advantages Overlaybd presents each image as a virtual block device backed by a stack of block-level layers. The design choices that follow from that one decision resolve each problem above.\nA merged extent index: O(1) at any depth At load time, overlaybd merges all layer indices into a single LSMT index of variable-length extent records (16 bytes each). Chain depth never enters the data path: a read resolves against one index whether the image carries two layers or fifty. Index size tracks fragmentation, not virtual size — in Alibaba\u0026rsquo;s production environment, images larger than 50 GB average under 300 KB of merged index, small enough to stay fully memory-resident across thousands of concurrent instances.\nA query engine built for speed LBA lookup is a segment-search problem over a sorted set of non-overlapping intervals. Overlaybd replaced the original std::lower_bound binary search with a linearized B+ tree vectorized with AVX-512 batch comparison — over 10× faster, sustaining hundreds of millions of lookups per second on a single core:\nSegment count B+tree + AVX-512 B+tree + bitmask loop std::lower_bound 1K 220 M/s 42.2 M/s 18.3 M/s 10K 160 M/s 30.7 M/s 12.8 M/s 100K 108 M/s 21.8 M/s 8.6 M/s 1M 57.4 M/s 15.2 M/s 5.6 M/s The algorithm is so fast it is even applicable to IP address lookup in backbone core routers (see my paper PlanB, NSDI \u0026lsquo;26).\nSector-granularity writes: no copy-on-write, ever The writable layer is part of the format itself, indexed at 512-byte sector granularity — the smallest unit any block I/O can cover — so a write always covers whole sectors and never triggers a copy. Write cost is proportional to write size, not file or cluster size. This is what breaks the granularity dilemma: the mapping is decoupled from write granularity, so the index stays small and writes stay cheap.\nOn-demand fetching with seekable compression Containers and VMs start by reading image data remotely as needed — a 1 GB+ image launches in under a second. Data is compressed in small, independently addressable units (ZFile, LZ4/zstd), so a random read fetches and decompresses only the relevant unit; because the compressed transfer is smaller, compressed random reads are often faster than uncompressed ones. Trace-based or file-list-based prefetch warms the cache ahead of demand, and P2P distribution spreads block fetches across peers when thousands of instances start at once.\nA stateless, minimal serving path The host sees only a sequence of block reads and writes. It never parses filesystem structures, never interprets symlinks or xattrs, and keeps no session state shared with the guest — every request carries all information needed to serve it. The consequences compound: the attack surface is the block interface itself, one of the oldest and most battle-tested boundaries in computing; the service recovers from crashes by simply re-attaching the device; and snapshot, clone, cross-host restore, and migration reduce to a disk sync plus access to the same backing layers — a decisive feature for agent sandboxes that fork on every speculative branch. The userspace block server is built on PhotonLibOS, a coroutine runtime that handles tens of thousands of concurrent I/O streams without thread-per-connection overhead.\nOne Format, Every Workload A block device is the most universal storage abstraction in computing — anything that can attach one can use overlaybd:\nContainers (runc) see a normal ext4 filesystem mounted from the device on the host, delivered through a containerd snapshotter and the OCI registry ecosystem. Secure containers (Kata, Firecracker) attach the device via virtio-blk: all metadata operations complete in-guest with zero cross-VM round-trips, and only data I/O crosses the boundary. Agent sandboxes get sub-second cold start, cheap snapshot/fork, strong isolation for untrusted code, and massive concurrency from shared base images. Virtual machines replace per-file allocation tables with the merged extent index, gaining O(1) lookup at any chain depth and OCI-style layered distribution through existing registries. Any guest OS — Linux, Windows, Android, macOS — understands a block device, with no guest agent or kernel module required. The filesystem inside remains a free choice: ext4, XFS, Btrfs, EROFS, NTFS. Notably, EROFS\u0026rsquo;s answer to the O(n) layer walk — a pre-built merged view flattened onto a single block device for VM pass-through — is an explicit acknowledgment of the same conclusion: the way out of filesystem stacking is the block-device route.\nProduction Evidence This is not a research prototype:\nAlibaba has run overlaybd for years across its entire application portfolio — Taobao, TMall, AlibabaCloud and more — and commercialized it on AlibabaCloud as the container image acceleration offering. Function Compute runs function microVMs on the same architecture. Azure built AKS Artifact Streaming on overlaybd. Databricks reported 7× faster VM startup for serverless compute; Superhuman built a 200K-QPS inference platform on the same infrastructure. DeepSeek Elastic Compute runs its agent execution environment on overlaybd-format images. fly.io boots Firecracker microVMs from overlaybd images; hocus.dev backs microVM development environments with it. Google Colab starts a 27 GB runtime image in ~170 ms warm, ~5.6 s cold. Kimi AgentEnv (Moonshot AI) independently re-implemented the open format as its sandbox image substrate, achieving sub-second launch latency. The design is peer-reviewed: DADI (ATC \u0026lsquo;20) describes the large-scale deployment at Alibaba, and FaaSNet (ATC \u0026lsquo;21) applies it to Function Compute.\nOpen Source Overlaybd is an open-source sub-project of containerd (CNCF graduated), Apache-2.0 licensed, with fully documented specifications (LSMT, ZFile) open for independent implementation:\nData I/O path: containerd/overlaybd Snapshotter \u0026amp; conversion tools: containerd/accelerated-container-image P2P distribution: data-accelerator/dadi-p2proxy Conclusion Conventional formats ask you to choose: fast cold start or small metadata or cheap writes or runtime universality. Overlaybd\u0026rsquo;s block-level design — a merged extent index, sector-granularity writes, on-demand seekably-compressed fetch, and a stateless block-device serving path — removes those trade-offs together, and does it with one format for every consumer. For infrastructure that starts, snapshots, and distributes workloads at scale, it is the stronger foundation.\n","date":"2026-08-10T00:00:00Z","permalink":"/en/p/overlaybd-the-ultimate-image-format/","title":"Overlaybd — The Ultimate Image Format for Every Workload"},{"content":"The blog is live.\nBuilt with Hugo and the Stack theme, hosted on GitHub Pages, with comments powered by Giscus. Published bilingually in Chinese and English.\npackage main import \u0026#34;fmt\u0026#34; func main() { fmt.Println(\u0026#34;Hello, blog!\u0026#34;) } ","date":"2026-08-09T00:00:00Z","permalink":"/en/p/hello-world/","title":"Hello World"},{"content":"Abstract Stackful coroutine, also known as user-space cooperative thread, offers the promise of more intuitive and accessible concurrent programming. With the growing demand for highly concurrent programs, stackful coroutine has gained increasing interest in recent years. It has, however, been much maligned for poor performance compared to stackless coroutine because of its heavy reliance on context switching. In this paper we perform in-depth measurement and analysis of several advanced implementations of stackful and stackless coroutine. Our analysis indicates that although current implementations of stackful coroutines are indeed significantly slower than their stackless counterparts, stackful coroutine is not intrinsically slow. Rather, stackful coroutine performs poorly mainly because the current implementations do not fully leverage the fact that control flow is cooperatively passed among stackful coroutines. Based on this observation, we propose context-aware context switching (CACS) among stackful coroutines. Instead of a full set of registers, CACS saves (restores) only the minimum necessary set of registers according to the caller (callee) context. It also enables the branch to be inlined at the caller site so that branch prediction is more accurate. We have implemented CACS in Photon, a highly optimized libOS based on coroutine. Performance measurements show that with the optimizations proposed in this paper, stackful coroutine out-performs stackless coroutine in most cases, and ties in the generator paradigm which is an especially challenging scenario for stackful coroutine. We also suggest a few supporting changes in computer architecture, programming language, compiler and OS that can further improve the performance of stackful coroutine. Our work is open-sourced on GitHub.\nIntroduction Modern servers may have network connectivity with bandwidth that is on the same order as that of its memory or CPU interconnect, and host dozens of SSDs each offering more than 10GB/s of throughput. With such advances in the hardware I/O capability, software stacks must become both highly concurrent and efficient to unleash the growing performance potential of the modern server. Recent software improvements to this effect include the development of high-performance I/O frameworks, such as DPDK and SPDK, that budget to process a single packet or request in terms of CPU cycles.\nCoroutine has also been gaining increasing attention in recent years to handle concurrent programming efficiently and effectively. Programming languages that have or are embracing coroutine include C++20, Rust 1.39 in 2019, C# 5.0 in 2012, Python 3.5 in 2015, JavaScript ES2017, Swift 5.5 in 2021, Java 21 in 2023, Dragonwell 8 in 2019, etc. Golang has provided coroutine (goroutine) as a first-class construct since its initial design.\nThere are two types of coroutine — stackless and stackful. The former shares a default stack among all the coroutines while the latter assigns a separate stack to each coroutine. With stackless coroutine, the code is transformed into event handlers at compile time, and driven by an event engine at run time, i.e. the scheduler of stackless coroutine. Transferring control of CPU to a stackless coroutine is merely a function call with an argument pointing to its context. Conversely, transferring CPU control to a stackful coroutine requires a context switch. This context switch is widely regarded as a heavy-weight operation when compared to the function call. In reality, this context switch is much more efficient than a kernel task switch because it does not incur the overhead of a round-trip transition from user-space to kernel-space, and it is also possible to perform optimizations by making use of the cooperative nature of the coroutines within a single program.\nNevertheless, due primarily to the perceived performance concern (among other issues), more and more systems are abandoning stackful coroutine for stackless coroutine, especially systems that emphasize performance such as C++20, Rust, C#, Swift, etc. There was once a heated debate in the C++ standards committee over stackless or stackful coroutine for C++20. Although stackful coroutine is generally easier to use, more compatible with existing codebases and more efficient in many scenarios, the proponents of stackless coroutine constructed a microbenchmark to show that \u0026ldquo;fibers (stackful coroutines) have 20 times larger context switch overhead\u0026rdquo; than stackless coroutine. The defendants of stackful coroutine did not give a direct response to the challenge, and ultimately the committee adopted stackless coroutine for C++20. We believe that the demonstrated outsized difference in overhead played an important role in this decision, as well as similar decisions for other systems.\nIn this paper we argue that stackful coroutine is not intrinsically slow. It just has not been implemented to fully exploit the cooperative nature of stackful coroutine, and there are opportunities to make context switching more efficient. We perform in-depth measurement and analysis of several current coroutine implementations. Based on the analysis, we propose context-aware context switching (CACS) to improve the efficiency of register saving, accuracy of branch prediction, and hit rate of CPU cache. CACS leverages the fact that context switches among stackful coroutines occur only at specific caller sites. With CACS, each context switching saves only the registers that will be needed after it switches back, and these registers are determined by the compiler at the caller site. CACS also enables the branching to be inlined at the caller site so that branch prediction is more accurate. We apply CACS to optimize stackful asymmetric coroutine by designing in-stack generator. With CACS, it performs as efficiently as the corresponding stackless coroutine implementation. We also introduce a new function calling convention named preserve_none as an expansion of CACS for all switching functions to further improve performance. With our work the cost of yield operation (scheduling and switching to the next coroutine) is greatly reduced. CACS is implemented in Photon, a sophisticated libOS based on coroutine. The proposed calling convention is implemented in Clang.\nOn the other hand, we demonstrate that stackless coroutine is inefficient in handling multi-level invocation, an inevitable pattern in real-world programs. It even incurs an overhead proportional to the length of the call chain when dealing with recursion. Despite there are optimizations that can reduce this overhead to a constant in some scenarios, it is still much higher than the corresponding overhead with stackful coroutine. Our results suggest that stackful coroutine is the better choice for efficient concurrency.\nThe contributions of this paper are as follows:\nWe conduct an in-depth performance characterization of state-of-the-art implementations of both stackless and stackful coroutines, analyzing the root causes for various measured differences. We observe that there are untapped opportunities to improve the performance of stackful coroutine by exploiting the fact that coroutines are cooperatively scheduled user-space threads. We propose and implement CACS to optimize the performance of stackful coroutine, and demonstrate that it can effectively eliminate the performance concern of context switching, thereby raising the performance upper bound of stackful coroutine. CACS makes stackful coroutine even feasible for challenging scenarios such as the generator paradigm. The overall result is promising: a single Xeon CPU core can perform a yield operation in ~1.52 ns or ~3.34 cycles, comparable to the cost of a function call, and out-performing state-of-the-art result by several times. While we demonstrate promising results with CACS, the current implementation is still constrained by existing architecture, programming language, compiler and OS. We suggest several supporting changes in these areas that can further improve the performance of stackful coroutine. Read the full paper ","date":"2024-10-14T00:00:00Z","permalink":"/en/p/stackful-coroutine-made-fast/","title":"Stackful Coroutine Made Fast"},{"content":"IPMI allows system administrators to control server hardware remotely, no matter what state the operating system is in — even when the machine is powered off, as long as the power supply is connected. This makes IPMI extremely valuable for servers used in experiments. Our cluster had always used a dedicated IPMI network, which required extra switches and cables, extra installation and maintenance work, and made the wiring messy and hard to manage.\nFor our newly deployed cluster, we planned to use IPMI over the shared port and cabling. The basic principle is to multiplex the eth0 port through VLANs: the production network and the IPMI network run in separate VLANs. When a packet arrives at the eth0 port, its VLAN ID is checked first; if it matches the IPMI VLAN ID, the packet is forwarded to the IPMI subsystem, otherwise it goes through the normal packet processing path.\nWhen I eagerly entered the BIOS to configure the IPMI network, I found that I could only change the IP address, subnet mask, and gateway — the network port option was stuck on \u0026ldquo;dedicated\u0026rdquo; and grayed out. As shown below:\nI suspected the \u0026ldquo;IPMI LAN Selection\u0026rdquo; option was locked because the motherboard integrated a relatively new 10GbE NIC whose IPMI features were not yet complete, so I gave up for the moment. However, while searching for information online that evening, I stumbled upon the fact that this NIC supports \u0026ldquo;IPMI passthrough\u0026rdquo; — judging from the name, the related functionality seemed to be complete, so I decided to give it another try.\nThis time, I first set a static IP address, subnet mask, and gateway for IPMI in the BIOS, then connected to the network through the dedicated IPMI port and opened the IPMI web configuration interface in a browser. On the Configuration / Networking page, I found the shared-port settings, as shown below. I set VLAN to \u0026ldquo;enable\u0026rdquo;, VLAN ID to \u0026ldquo;100\u0026rdquo;, and Lan Interface to \u0026ldquo;Share\u0026rdquo; — and then successfully accessed IPMI through the eth0 port on VLAN 100.\nAfter configuring the shared port, entering the BIOS again showed that the IPMI network port had changed to \u0026ldquo;Share LAN\u0026rdquo;, while the dedicated network connection status became \u0026ldquo;No Connect\u0026rdquo;.\n","date":"2014-03-26T22:51:49+08:00","permalink":"/en/p/ipmi-shared-lan-setup/","title":"Setting Up IPMI on a Shared LAN Port"}]