构建容器化的集群管理系统

BuildingContainer-basedClusterManagement

Systems构建容器化的集群管理系统About

Me电子科大：计算机、算法、控制理论、机器人卡内基梅陇大学：计算机、操作系统、分布式系统、存储系统FusionIO：存储系统谷歌：集群管理系统及生态ArchitectureUser

browserNginxRate

LimitingSSL

TerminationSolution

ManagerPublic

APIAuthenticationTransformationPagingPublic

APIAuthenticationLoggingCluster

ManagerPublic

APIAuthenticationAuthorizationSolution

ManagerPrivate

API…PagingPrivate

API…Cluster

ManagerPrivate

API…PagingThirdParty

API…AuthorizationPublic

APITransformationLoggingAuthorizationPublic

API…Problem:Duplicate

FunctionalitiesSystemtendstobe

monolithicComplexfrontendlogicduetovaried

APIArchitecture

RevampSolution

Managercloud

native)API

GatewayUserbrowserCargo(registry)Circle(release

mgmt)KubernetesAPI

callsCaicloudadminbrowserAdmin

Console…

…UserKubernetesClusterMonitoringLoggingUserKubernetesClusterMonitoringLoggingUserKubernetesClusterMonitoringLoggingUserKubernetesClusterMonitoringLogging…

…Ap(pcloicmatpionsitMeaanpapgse,

rClusterManager(feCdluesrtaetrioMn,ainsaoglaetrion,logging,HA,

lifecycle)PPaaggiinnggAAccccoouunnttiinnggloggingauthCircle:

GoalCluster…

…ClusterClusterWhereismy-awesome-app

running?What

is

the

latest

version

of

my-awesome-app?What

is

the

live

version

of

my-awesome-app?Is

version

Y

running

long

enough

to

roll

out

(and

upgrade

version

X)?Can

I

continuously

deploy

my-awesome-app

to

test

cluster.How

can

I

upgrade

my-awesome-app

with

his-xxx-app

now

that

Ihave

to

depend

on

it?Circle:

GoalStatic

ConfigurationEasy

but

‘static’,

works

well

in

most

casesDynamic

trackingRecord

status

while

deployingUse

kubernetes

annotation

for

trackingDynamic

dependency

management

remains

unsolvedHypervisorHypervisorHypervisorCI

ModuleLogging

ModuleDeploymentModuleSolution

ManagerCircle:

POCRequestsAsyncManagerDockerDockerManagerVcsManagerCI/CDManagerAPIKubernetesLog

ManagerAPI:Handleuserrequests,validation,

etcAsyncManager:Asynchronouslyexecuting

operationsgochannel->message

queueVcsManager:Handlesversioncontrol

toolsStateless:runand

goLogManagerBuildlog

streaminglocalfile->kafka,web

socketDockerManagerManagedocker

build/push?Multiple

Docker•Lightweight:2cores,4G=>50daemons+concurrent

pushesSingleDocker

DaemonLimitedconcurrentbuild/push:2cores,4G=>15concurrent

pushesIsolationformultiple

build/runRaceconditionfor

imagesRiskofleakingprivate

imagesDINDNotthat

confidentVirtual

MachineComplete

isolationSlow,andlackofvirtualizationsupportoncloud

environmentProvisioningoverheadVMbasedcontainerisfaster,butdoesn’tsolveimage

problemMultipleDocker

DaemonComplete

isolationMultiple

DockerUsedifferentworkingdirectoryfordocker

daemonayerInit

LCMD

[“./main”]ADDdata.txt/VOLUME

/dataFROM

ubuntu/etc/hostname/etc/hosts

/etc/resolv.confRead-WriteLayerMultiple

DockerDifferentnetworksettingfordocker

daemon-APOSTROUTING-s172.18.2.0/24!-odocker2-j

MASQUERADE-APOSTROUTING-s172.18.1.0/24!-odocker1-j

MASQUERADE-AFORWARD-odocker2-j

DOCKER-AFORWARD-odocker1-j

DOCKER-AFORWARD-odocker2-mconntrack--ctstateRELATED,ESTABLISHED-j

ACCEPT-AFORWARD-odocker1-mconntrack--ctstateRELATED,ESTABLISHED-j

ACCEPT-AFORWARD-idocker2!-odocker2-j

ACCEPT-AFORWARD-idocker2-odocker2-j

ACCEPT-AFORWARD-idocker1!-odocker1-j

ACCEPT-AFORWARD-idocker1-odocker1-j

ACCEPTContainer1172.18.1.4/24docker1172.18.1.0/24Container2172.18.1.5/24Container1172.18.2.4/24docker2172.18.2.0/24Container2172.18.2.5/24eth0Graceful

TerminationRollingupdateisgreat,

when:Youwanttotestmultipleversionsofcodeor

configurationYouwanttoupdateapplicationwithoutserviceinterruptionHowever,youwanttomakesurethereisreallyno“service

interruption”ComponentOSconGraceful

LibrarysignalTask

OffloadingCluster

ManagerClEuxseteucrtMoranagerClEuxseteucrtMoranagerExecutoretcd

clusterClusterManagerValidatorClusterManagerCluSsetervreMr

anagerCluSsetervreMranagerCluSsetervreMr

anagerServerAservicewithalotof

states:Operation

statusExecution

statusCluster

statusState

transformationSolutionDistributed

transaction?Astate

machine?State

reconstruction?Graceful

termination?Orjustuse

k8s!Task

OffloadingController

framework:Listoperationtoreconcile

cacheWatchoperationtoreflectchange

ASAPEventsystemand

hooksCachereconstructionduring

restartKube