Hudson J1 2009

1、Continuous Integrationin the Cloudwith HudsonKohsuke KawaguchiJesse GlickSun Microsystems, Inc.Hudson committersRise of Continuous IntegrationOffload from people, push to computers2$timecomputersusSpend more CPU power to help you even if it only helps a littleFirst on your laptops and workstationslI

2、DEs are at the forefrontAnd then to the serversla.k.a. “Continuous Integration”lMore frequent build/test executionslStatic code analysis toolslAnd more to come3HudsonOpen-source CI server at Emphasis on ease of installation and usel“java -jar hudson.war” executionlConfigure everything from browsersE

3、xtensibilityl140+ community-developed public pluginslBy 150+ contributorsEstimated 13,000 installations4https:/ basically does builds and testsCheck out the source codelSubversion, Perforce, Git, Mercurial, CVS, Do builds and/or testslAnt, Maven, MSBuild, shell script, Record resultslBinary, test re

4、sults, code coverage, static analysisNotify peoplelE-mail, IM, RSS, tray apps, IDEs5Localized to 8 languages6And hopefully more to come78Adoption in all kinds of businesses9Why Distributed Builds?You need to use multiple computers becauselYou need different environmentslYou need isolationTheres only

5、 so much you can do with 1 computer10Before we talk about cloudsGoing virtual doesnt solvelSoftware installation problemlNode failure problemlRemote maintenance problemlWhat does Hudson do to help you with these?11Installing new slavesFor first 20 or so slaves, we did it manuallylInsert CD, click, t

6、ype, click, type, click, lBut that doesnt scaleThen we automatedlAvailable as “Hudson PXE Plugin”12Automated System Installations13SlaveslPower on, hit F12lPC boots from network (PXE)Hudson + PXE pluginlISO images of OSAutomated System Installations14SlaveslPower on, hit F12lPC boots from network (P

7、XE)lChoose OS from menulInstalls non-interactivelyHudson + PXE pluginlISO images of OSYour corporate IT guy & his DHCP serverAutomated System InstallationsSupports OpenSolaris, Ubuntu, CentOS, FedoralTrivial with most LinuxCooperate with Windows, tooQuite useful outside Hudson, toolNo more broke

8、n CD driveslNo more CD-Rs15Distributed builds with HudsonMasterlServes HTTP requestslStores all important infoSlavesl170KB single JARlAssumed to be unreliablelScale to at least 100LinklSingle bi-di byte streamlNo other requirements16Masterslave1slave2slave3slave4slave5slave6slave7slave8How master an

9、d slaves start talkingFor Unix slaves, via SSHlOnly need SSH and JRE on slaveslWe just need a host name17How master and slaves start talkingFor Windows, DCOMlWe just need admin user name and passwordlNo manual interventionlWorks even from Unix masters18How master and slaves start talkingVia Java Web

10、 StartlWhen master cannot see slaveslA separate socket connection is made19Automating JNLP launchOnce started, can be installed as Windows service20Automating JNLP launchEmulate the JNLP client headless21$ java -jar slave.jar -jnlpUrl URLAutomated Tool Installation - JDKJDK from http:/ automatically

11、 chooses the right bundlelAlways up to date with new releases22Automated Tool Installation - ApacheAnt and Maven from Apache23Automated Tool Installation - CustomDownload arbitrary archive and unpack24Automated Tool Installation - CustomRun arbitrary shell commandslCan have variants by OS25Automated

12、 Tool Installation - ExtensibleWrite your ownlSimple Hudson extensionlJust write Java code to create tool on slaveIn progress:lSCMs Mercurial, lInstall from Subversion26Heterogeneous Cluster ChallengeYour builds/tests need to run in specific environmentDependency on individual nodes hurts utilizatio

13、n27WombatWindows testHudson Windows testWindows #1jobsslavesGlassFishWindows testWindows #2Solaris#1Hudson Solaris testLabels to rescueLabel is a group of slavesTie jobs to labels28WombatWindows testHudson Windows testWindows #1jobsslavesGlassFishWindows testWindows #2Solaris#1Hudson Solaris testWin

14、dowsSolarisWindows #3Making builds stickyWe want jobs to be mostly on the same slavelFaster check outlConsistent resultslMinimizes disk consumptionBut does it softlyHudson uses consistent hash* to achieve thisMore schedule controls become possible:lUse faster machines more frequentlylSlowly ramp up

15、newly installed slaves29Coming to your Hudson soon* /wiki/Consistent_hashingForecasting failuresHudson monitors key health metrics of slaveslLow disk space, insufficient swaplClock out of synchlExtensibleSlaves go offline automaticallyCatch problems before they break builds30Cl

16、ean up mess after buildsKill runaway processeslDaemons, background processes left by your buildlWorks on Windows, Linux, Mac, and Solaris31Load Statistics Monitoring32When its time to add more slaves33Hudson made this extensibleHudson detects excessive workloadHudson notifies pluginsPlugins can prov

17、ision more slavesl assuming that you have that infrastructure3435Amazon EC2: The GoodPay as you go (10/h or so)lLoads on Hudson tend to be spikyProgrammable APIInstances launch at machine-speedEC2 instances are forgetful36Amazon EC2: The BadYour data is still inside your firewalllTakes time to check

18、 out codel or to archive build artifactslSome data just cant be movedEC2 instances are forgetfulCan your tests run in parallel?37Hudson EC2 pluginBuilt on top of typica*What does it do?lAutomatically provisions slaves on EC2 on demandlPicks the right AMI depending on demandlStarts slave agentlShuts

19、down unused instances38* http:/ it all together39time# of executorscapacityusagequeue lengthHudson “Appliance” on EC2Run the master in the cloud too, if you likelHudson on stock OpenSolaris AMIlData stored persistently in Elastic Block StoragelDynamically expandable thanks to ZFSlOnline, tooPackaged

20、 as a wizard404142Hudson Hadoop pluginlJust a few mouse clicks to installlTurn every Hudson slave into a Hadoop nodeDistributed file systemlAutomatic data replication (fault tolerant)lNice for storing old artifacts, logs, test records, Map/reduce frameworklLarge scale test results analysis / datamin

21、inglMore interesting work to be done in the future43Hudson Selenium PluginSeleniumlTests webapps by scripting browsersSelenium GridlRuns Selenium over a grid of computersAllow Hudson labels to specify where to start browsersHudson & Selenium both need heterogeneous cluster44Selenium Grid45Hudson master(selenium hub)Hudson slavesConclusionCI is here to staylWell continue to push more workload to serversHudson makes this easy for youReap the benefit of a cluster in multiple ways46Resourceshttp:/ “Hudson Community Meet u