面向并行开发的 Nix 构建一次构建处处复用 Nix Builds for Parallel Development Build Once,Reuse Everywhere

Oct.29

IndustrialOpenSource

Oct.30

OpenSourceRobotics

Agenda

NixBuildsfor

ParallelDevelopment

BuildOnce,ReuseEverywhere

29October2026

MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAG

Motivation:WhyNot10xFaster?

AImakescoding10xfaster.Whydoesn’ttheteamship10xfaster?

Theorem(Amdahl’sLaw).Overallspeedupislimitedbythefractionofworkthatremainsunchanged:

p=fractionaccelerated,s=speeduponthatfraction

s=10(illustrative)

•p=30%(coding):1.37x

BuildandCItimeistheconstraint

•p=50%(coding):1.8x

Everythingafterthecodeisunchanged

•Reviewqueuesgrow

•CIandbuildstakethesamewalltime

Speedingupthecodegenerationisonepart.Next:speedupthebuild.

Page2MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Page3MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

TraditionalSolutions

CI-providercaches

Provider-specificAPIsandcachekeys;switchingCIproviderscanmeanreworkingthesetup.Mightnotworklocallyacrossallsystems.

Build-toolcaches

Reusetaskoutputs,locallyorremotely;eachtoolislimitedtoitsownecosystem.

Compilercaches

Hermeticbuildsystems

ccache,sccache:cachecompileroutputs;sccachesupportsremotestorage.Neithercachestoolchainsorsystemlibraries.

Bazel,Buck2:cacheeverything,butadoptionmeansrewritingthebuild;third-partydependenciesare

painful.

SiloedorCostly

Wanted:onecacheandbuild-offloadmodelforalldependencies;wrapexistingbuildsinsteadofrewritingthem.

Page4MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

CanWeDoBetter?

IntroducingNix:Theuniversal,composablebuildsystem

Input-AddressedBuildsruninasandboxBuilds

withoutnetworkaccess;onlydeclaredinputsarevisible.

RichEcosystem

Hermetic

~150,000

packages

;existingbuildsarewrapped,not

rewritten.

Outputpathsderivefromthederivationanditsinputs;thestorepathisthecachekey

(trivialinvalidation).

Reuseinsteadofrebuild

Nixeitherfetchesmatchingstorepathsfromabinarycache

Page5MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

NixLanguage,Tool,andEcosystem

Lazy,purelyfunctionallanguage.

Evaluatesexpressionsintoderivations(buildrecipes:inputs,builder,env→output

paths);managesthestoreandrunsbuilds.

Collectionofpackageexpressionsandbuildfunctions.

Nixlanguage

evaluatedby

nix+nix-daemon

evaluates

Nixpkgs

LinuxdistributionconfiguredwithNixmod-ules.

providesmodulesfor

NixOS

Page6MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

NixLanguage:AQuickPrimer

TheNixlanguageconsistsofexpressionsthatevaluatetovalues:

let

add=a:b:a+b;

libs=["openssl""zlib"];

src=./.;#caveat:

copies.to

storein{

inheritsrc;#src=src;

sum=add12;

names=map(l:"lib-"+l)libs;}

#=>{src=/path/here;sum=3;

#names=["lib-openssl"

#"lib-zlib"];}

•letbindsnames;expressionafterinistheresult

•a:b:exprdefinesafunctionoftwoarguments;add12appliesit

•[...]isaspace-separatedlist;mapappliesafunctiontoeachitem

•./.isapathliteral,notastring;copiedtothestorewhencoercedtoastring(e.g.,inaderivation)

•A{}isanattributeset,themaindatastructureofNix

Page7MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

NixLanguage:AttributeSetsandMerging

let

common={pname="app";version="1";};

mkPkg={pname,version,...}:{inheritpnameversion;

description="${pname}${version}";};

inmkPkg(common//{version="2";})

#=>{pname="app";version="2";

#description="app2";}

{hello="ok";huge=throw"no";}.hello#=>"ok"

•x={a=1;}isaset;x.aselects

•//mergessets(shallow);therightsidewins

•{a,b,}:takesaset

anddestructuresit;...ignoresextraattributes

•inherita;meansa=a;

•"${a}"isstringinterpolation

•Evaluationislazy:hugeisneverevaluated,sonoerror

Page8MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

NixPackages:It'sAllAboutFunctions

ANixpackageisafunction,withtheargumentsbeingitsdependencies.

{craneLib,pkg-config,openssl}:#depsareNixexpressions

let

commonArgs={

src=craneLib.cleanCargoSource./.;#FiltersourcetoCargo-relevantfiles

nativeBuildInputs=[pkg-config];#Build-platformtools

buildInputs=[openssl];#Host-platformdeps(e.g.,libraries)

};

cargoArtifacts=craneLib.buildDepsOnlycommonArgs;#depsonly(release)cargoDebugArtifacts=craneLib.buildDepsOnly(commonArgs//{

env.CARGO_PROFILE="dev";#debugdepstopre-populatedevShelltarget/});

in

craneLib.buildPackage(commonArgs//{

inheritcargoArtifacts;#onlyworkspacecodebuilthere

passthru={inheritcargoDebugArtifacts;};#exposeforCIcache+devShell})

Page9MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Evaluation:FromNixExpressiontoArtifact

Derivationgraph

.drvfiles

Checklocalstore

andbinarycaches

Cachemiss

Buildlocallyoron

remotebuilder

Nixexpressione.g.,package

Cachehit

Reuselocallyorsub-

stitute(=down-

load)fromcache

Immutablestoreoutput

Evaluate

Page10MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Flakes:It'sAllAboutComposition

Nixflakespininputsanddefinepackagesanddevshells:

{inputs.nixpkgs.url="github:NixOS/nixpkgs/nixos-unstable";inputs.crane.url="github:ipetkov/crane";

outputs={nixpkgs,crane,...}:letsystem="x86_64-linux";

pkgs=importnixpkgs{inheritsystem;};

craneLib=crane.mkLibpkgs;

my-app=pkgs.callPackage./package.nix{inheritcraneLib;};in{

packages.${system}={

inheritmy-app;

deps-debug=my-app.cargoDebugArtifacts;#forCItopopulatecache};

devShells.${system}.default=craneLib.devShell{inputsFrom=[my-app];};};

}

Page11MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Innerloop

•Enterdevshell:nixdevelop.

•Runtoolsdirectly:cargocheck,cargotest

•Pre-populatewithCI-builtdependencies

DX:TwoFeedbackLoops

Outerloop

•Verifybuild:nixbuild.#my-app

•Isolatedderivationcatchesundeclaredinputs

Page12MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

InnerLoop:Pre-populateRusttarget/fromaBinaryCache

•

•

•

•

•

Buildalldependenciesonce(e.g.,inCI)andreusethemforlocalbuilds:

let

deps=my-app.cargoDebugArtifacts;

prepopulate=''

STAMP="target/.deps-stamp"

if["$(cat"$STAMP"2>/dev/null)"\!="${deps}"];then

mkdir-ptarget

tar-Izstd-Ctarget-xf\${deps}/target.tar.zst

chmod-Ru+wtarget

echo-n"${deps}">"$STAMP"fi

#+pointCARGO_HOMEatvendoredsources'';

in

devShells.${system}.default=craneLib.devShell{inputsFrom=[my-app];

packages=[pkgs.zstd];

shellHook=prepopulate;};

nixdevelopsubstitutes

cargoDebugArtifacts(publishedvia.#deps-debug)andunpacksitintotarget/

cargobuildcompilesworkspacecrates

Deletetarget/freely,depsstaycachedin/nix/store

Needsidenticalrustc/flagsandvendoredstorepaths,elsefullrebuild

Forper-cratecachereuse,usecrate2nix

Page13MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

CachingStrategy

substituteBinaryCachessign+push

DevelopersCISystems

substitute

Remote

offloadbuildsBuilder

Signedstorepaths

BinaryCaches

•Upstream:

(read-only,S3-backed)

•Mirrors:e.g.,USTC,TUNA,SJTU(read-only)

•Self-hosted:Attic,Harmonia

•SaaS:Cachix,FlakeHubCache

KeyMechanics

•Trust:artifactsarecryptographicallysigned

•Builders:accessviaSSH

•CIsavings:cachehitscutbuildtimeandCIminutes

•Caching:optimization,notharddependency

Page14MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

MeasuredResults

Scenario

PlainCargo

Warmsccache

Nix,pre-populated

Freshtarget,walltime

7.59s

5.06s

2.36s

Freshtarget,CPUtime

42.13s

20.28s

4.69s

4parallelbuilds,lastfinish

18.57s

9.67s

4.71s

4parallelbuilds,totalCPU

249.85s

116.76s

28.87s

4parallelbuilds,peakRSS

4,045MiB

2,787MiB

1,780MiB

Incrementalleaf,walltime

0.51s

0.52s

0.51s

Incrementalleaf,CPUtime

0.82s

0.67s

0.72s

Takeaway

•Fresh-buildCPU:9×faster

•4parallelbuilds,CPU:8.7×faster

•Incrementalrebuilds:comparable

Amdahl:end-to-endgain

Speedupon4parallelbuilds:s=18.57/4.71≈3.9

•Builds+CIwaitp=20%ofcycle:1.18x

•p=40%:1.42xMeasureyourownp.

Five-runmedians,16CPUs,32GiBRAM.Smallone-cratelocalcheck,noremotecacheorbuilder;notascaleforecast.

Page15MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Conclusion

Onepinnedflakedefinespackagesanddevshells;binarycacheandremotebuildersreuseitsoutputs.

Benefits

•Samedeclareddependenciesindevelopment,agents,andCI

•Matchingoutputssharedacrossmachinesinsteadofrebuilt

•Buildscanbetriviallyoffloadedtoremotebuilders

Costs

•Steeplearningcurve(thoughLLMsgreatlylowerit)

•Coarseinvalidation:anyCargo.lockchangerebuildsallofcargoDebugArtifacts

•nixbuildisnotincremental:alatefailurererunsthederivation

•Pre-populatingbuilddirectoriesis

per-languagework;Nixevaladdsoverhead

Page16MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Thanks

Questions?

MichaelAdler

michael.adler@

Page17MichaelAdler|SeniorKeyExpert|FTRPDCPCOES-DE|SiemensAGSIEMENS

Backup:SandboxedAgen