高盛如何在大数据和云计算中使用JavaPPT课件.pptx

TECHNOLOGYDIVISION HowGoldmanSachsworkswithJava 3 18 2020 1 TECHNOLOGYDIVISION 2 Agenda EngineeringatGoldmanSachsUsingJavaandOpenJDKinGoldmanSachsEclipseCollections What sEclipseCollections BenefitofEclipseCollectionscomparedtoStreamsandJCF LearningEclipseCollectionsbysolvingproblems 123 3 18 2020 2 TECHNOLOGYDIVISION 3 EngineeringatGoldmanSachs 3 18 2020 3 TECHNOLOGYDIVISION 4 GoldmanSachs TheGoldmanSachsGroup Inc isaleadingglobalinvestmentbanking securitiesandinvestmentmanagementfirmthatprovidesawiderangeoffinancialservicestoasubstantialanddiversifiedclientbasethatincludescorporations financialinstitutions governmentsandindividuals Foundedin1869 thefirmisheadquarteredinNewYorkandmaintainsofficesinallmajorfinancialcentersaroundtheworld GoldmanSachsTechnologyDivisionWebuildsolutionstosomeofthemostcomplexproblems creatingtransformationaltechnologiesanddevelopingsystemsthatdrivebusinessandfinancialmarketsworldwide 3 18 2020 4 TECHNOLOGYDIVISION 5 EngineeringatGoldmanSachs Leadinglatestopentechnology globalinitiatives OpenComputeProject Aprojecttobuildeffectivecomputationinfrastructureatthelowestpossiblecost resultingwithoptimizeddatacenteroperation OpenContainerInitiative Aprojecttodefineformatandruntimeofcontainertechnology Morethan30organizationsincludingDockerareparticipatingintheproject JavaCommunityProcess ExecutiveCommittee JCPExecutiveCommitteeisresponsibleofimportantspecificationprocessforJavaplatform GoldmanSachswaselectedtothecommitteein2011forthefirsttimeandcontinuetobesincethen 3 18 2020 5 TECHNOLOGYDIVISION 6 EngineeringatGoldmanSachs 3 18 2020 6 TECHNOLOGYDIVISION 7 UsingJavainGoldmanSachs 3 18 2020 7 TECHNOLOGYDIVISION 8 199820002000 2004 BeganinternalJavaevaluation Javabecameoneofthestrategicplatforms ContinuouslyworkingcloselywithSun OracletoimproveJava ReadyforLambdasinGSCollectionsFrameworkpredecessor ElectedintoJCPExecutiveCommittee2012 GSCollectionsFrameworkOpen sourced SignedOCA2013 StartedcontributingtoOpenJDK 2015 MigratedGSCollectionstoEclipseFoundation HistoryofGoldmanSachsandJava 3 18 2020 8 TECHNOLOGYDIVISION 9 JavainGS JCPExecutiveCommitteemember PartoftheExpertGroupsforthefollowingJSRs JSR107 JCACHE JavaTemporaryCachingAPI JSR335 LambdaExpressionsfortheJavaProgrammingLanguage JSR351 JavaIdentityAPI Over3000JavaDevelopers Maintain 130millionlinesofJavacode Onaverage 125 000Javaprocessesrunningatanygivenhoureveryday 3 18 2020 9 TECHNOLOGYDIVISION 10 CuttingEdgeJavainGSWedopushtheboundaryofJavatechnology o Verylargeheap Someapplicationsuseasmuchas190Gheap IntegeroverflowissueswithGrowableArraycausedissuewithMarkStackoJVMstressedunderhighvolumeconditionsorunusualusagepattern CodecachecapacityOpenSourceinternallydevelopedframeworkoEclipseCollections CollectionframeworkthatisricherandmorefluentthanStreamAPI 3 18 2020 10 TECHNOLOGYDIVISION 11 Sourceaccessisimportant Timelyresolutionforproductionissues Identifypotentialareaforimprovement Goodforsecurity Transparency Howisthecodebeingdeveloped Bugdatabase ContributepatchesforbugsandenhancementsthatbenefitusandthewiderJavacommunity WhyWeUseOpenJDK 3 18 2020 11 TECHNOLOGYDIVISION 12 TypicalUsageofOpenJDK Troubleshooting Resolveproductionissues Usefulduringdevelopment Researchnewfeatures IdentifyoptimizationopportunitiesintheJDK Howtheycanbeappliedandbeneficialtoapplications 3 18 2020 12 TECHNOLOGYDIVISION 13 TroubleshootingHavingaccesstoJDKsourcecodeisimportant TimelyDiagnostics Crashdumpanalysis Debugbuilds Identifyworkaroundsandpotentialfixes Createpatchedbuildsfordeveloperstoverifyfixes IncludingpatchesnotyetavailabletoGAbuilds 3 18 2020 13 TECHNOLOGYDIVISION 14 Research Understandingnewfeaturesinhotspot Howdoesitwork Doesitbenefitourapplications Ifitdoes howcouldapplicationsexploitit Identifyperformancebottleneck Potentialworkaroundandfeaturestoimproveapplicationperformance 3 18 2020 14 15 TECHNOLOGYDIVISIONImproveArray sort logic DualPivotQuicksort forprimitivecollectionsHowChangedthealgorithmto reducenumberof run a run ishowlongyougothroughthearraywithitbeinginorder StopsortinginthecaseofdescendingnumbersTestcase Additionalfunctionalunittests PerformancebenchmarkusingJMHPatchcontribution Discussedpatchat 3 18 2020 15 TECHNOLOGYDIVISION 16 EclipseCollectionsOpenforcontributions collections blob master CONTRIBUTING md 3 18 2020 16 TECHNOLOGYDIVISION 17 WhatisEclipseCollections GSCollections OpensourceJavacollectionsframeworkdevelopedinGoldmanSachsInspiredbySmalltalkCollectionsFrameworkIndevelopmentsince2004HostedonGitHubw Apache2 0LicenseinJanuary2012 EclipseCollections7 0 Project EclipseFoundationhttps www eclipse org collections FeaturesetisthesameasGSCollections7 0PublicreleasedonJan20th Packagesrenamedcom gs org eclipse 3 18 2020 17 TECHNOLOGYDIVISION 18 JavaOne2015 EclipseCollectionsbyExample 3 18 2020 18 TECHNOLOGYDIVISION 19 EclipseCollections Richsetoffeatures Java8StreamAPI FunctionalAPIsLazyonlySingleuseSerial ParallelPrimitivestreams 3types ExtensibleCollectors EclipseCollections7 0 3 18 2020 19 TECHNOLOGYDIVISION 20 ReadableInterface MutableInterface ImmutableInterface Bag Set List Stack Map etc MutableBag MutableList etc ImmutableBag ImmutableList etc DesignConceptRichIterable 3 18 2020 20 TECHNOLOGYDIVISION 21 Inheritancediagram Immutability 3 18 2020 21 TECHNOLOGYDIVISION 22 CollectionFrameworkComparisons 3 18 2020 22 TECHNOLOGYDIVISION 23 BenefitofEclipseCollectionscomparedtoStreamsandJCF 3 18 2020 23 StreamAPI EclipseCollections TECHNOLOGYDIVISION 24 booleanresult this people stream anyMatch person person hasPet PetType CAT longresult this people stream filter person person hasPet PetType CAT count ListpeopleWithCats this people stream filter person person hasPet PetType CAT collect Collectors toList booleanresult this people anySatisfy person person hasPet PetType CAT intresult this people count person person hasPet PetType CAT MutableListpeopleWithCats this people select person person hasPet PetType CAT WriteConciseCode 3 18 2020 24 StreamAPI EclipseCollections TECHNOLOGYDIVISION 25 Leveragemethodreferences booleanresult this people stream anyMatch person person hasPet PetType CAT longresult this people stream filter person person hasPet PetType CAT count ListpeopleWithCats this people stream filter person person hasPet PetType CAT collect Collectors toList booleanresult this people anySatisfyWith Person hasPet PetType CAT intresult this people countWith Person hasPet PetType CAT MutableListpeopleWithCats this people selectWith Person hasPet PetType CAT 3 18 2020 25 StreamAPI EclipseCollections TECHNOLOGYDIVISION 26 ListpeopleWithCats this people stream filter person person hasPet PetType CAT collect Collectors toList ListpeopleWithoutCats this people stream filter person person hasPet PetType CAT collect Collectors toList Map peopleWithcatsAndNoCats this people stream collect Collectors partitioningBy person person hasPet PetType CAT peopleWithcatsAndNoCats get true peopleWithcatsAndNoCats get false MutableListpeopleWithCats this people select person person hasPet PetType CAT MutableListpeopleWithoutCats this people reject person person hasPet PetType CAT PartitionMutableListpeopleWithcatsAndNoCats this people partition person person hasPet PetType CAT peopleWithcatsAndNoCats getSelected peopleWithcatsAndNoCats getRejected CodewithBetterReadability 3 18 2020 26 StreamAPI EclipseCollections 27 LazyIterablelazy Lists mutable with 1 2 3 asLazy Assert assertEquals 1 lazy collectInt i i min Assert assertEquals 3 lazy collectInt i i max TECHNOLOGYDIVISION throws ReuseCodeStreamstream Lists mutable with 1 2 3 stream Assert assertEquals 1 stream mapToInt i i min getAsInt Assert assertEquals 3 stream mapToInt i i max getAsInt java lang IllegalStateException streamhasalreadybeenoperateduponorclosedatjava util stream AbstractPipeline AbstractPipeline java 203 atjava util stream IntPipeline IntPipeline java 91 atjava util stream IntPipeline StatelessOp IntPipeline java 592 atjava util stream ReferencePipeline 4 ReferencePipeline java 204 atjava util stream ReferencePipeline mapToInt ReferencePipeline java 203 3 18 2020 27 Size Mb TECHNOLOGY DIVISION 28 6050403020100 Elements Datacollectedby MemoryEfficient HashSetvsUnifiedSet 3 18 2020 28 Size Mb 29 454035302520151050 Elements Datacollectedby TECHNOLOGYDIVISIONMemoryEfficient HashMapvsUnifiedMap 3 18 2020 29 Size Mb 30 2520151050 Elements Datacollectedby TECHNOLOGYDIVISIONMemoryEfficient primitivecollections 3 18 2020 30 Java Collections Eclipse Collections TECHNOLOGYDIVISION 31 ImmutableListeclipseList Lists immutable with two eclipseList each Procedures throwing appendable append Assert assertEquals onetwo appendable toString Appendableappendable newStringBuilder ListjdkList Arrays asList one jdkList forEach appendable append CompileError incompatiblethrowntypesjava io IOExceptioninmethodreferencejdkList forEach each try appendable append each catch IOExceptione thrownewRuntimeException e Assert assertEquals one appendable toString ConciseExceptionHandling 3 18 2020 31 TECHNOLOGYDIVISION 32 Appendix 3 18 2020 32 TECHNOLOGYDIVISION 33 OpenJDKEnhancedArrays sort JMHResults TestSampleAnarrayofintwithsize10 000 000 32pairflips arunofzeroesinthemiddle and32pairflipsattheend by JMHthe 3 18 2020 33 TECHNOLOGYDIVISION 34 Eageriterationpatter