ScyllaDB NoSQL数据库的应用加速实践

1、SCYLLA:NoSQL at Ludicrous SpeedScyllaDB NoSQL数据库的应用加速实践Today we will cover:+ Intro: Who we are, what we do, who uses it+ Why we started ScyllaDB+ Why should you care+ How we made design decisions to achieve no-compromise performance and availability+Founded by KVM hypervisor creators+Q2 2014 - Pivot

2、 to the database world+Q3 2015 - Decloak during Cassandra Summit 2015, Beta+Q1 2016 - General Availability+Q3 2016 - First Scylla Summit: 100+ Attendees+Q1 2017 - Completed B round+$25MM in funding+HQs: Palo Alto, CA; Herzelia, Israel+42+ employees, hiring!IntroductionWhy ?#$%$%?Scylla benchmark by

3、Samsungop/sWhat we do: Scylla, towards the best NoSQL+ 1 million OPS per node+ C* gets full volumeRead MicroserviceWrite ProcessScylla/Cassandra side by side deploymentWrites are written in parallel to C* and ScyllaReads are done in parallel:Memcached + Cassandra 2) Scylla (no cache at all)Read Micr

4、oserviceWrite ProcessScylla (w/o cache) vs Cassandra + MemcachedScyllaCassandraDiff%Requests/ Minute12,000,000500,000(memcache handles 11,500,000)24XAVGLatency4 ms8 ms2XMax Latency8 ms35 ms3.5XHardware9 machines30+9machines4.3XCassandras latencyScyllas latencyWhat does it mean for a non Cassandra us

5、er?+Throughput, latency and scale benefits+Wide range of big data integration: Kariosdb, Spark, JanusGraph, Presto, Kafka, Elastic+Best HA/DR in the industry.+Stop using caches in front of the database+Consolidate HBase, Redis, MySQL, Mongo and othersAssorted QuotesToday we will cover:+ Intro: Who w

6、e are, what we do, who uses it+ Why we started ScyllaDB+ Why should you care+ How we made design decisions to achieve no-compromise performance and availabilityDesign decisions: #1 The trivialsSSTable file formatConfiguration file formatCQL languageCQL native protocolJMX management protocolManagemen

7、tDesign decisions: #2 CompatibilityDouble cluster - Migration w/o downtimeAppCassandraScyllaCQLproxyDesign decisions: #3 All things asyncDesign decisions: #4 Shard per coreThreadsShardsSCYLLA DB: Network ComparisonKernelCassandraTCP/IPSchedulerqueuequeuequeuequeuequeuethreadsKernelTraditional stackS

8、cylla sharded stackMemoryApplicationCP/ITask SchedulerqueuequeuequeuequeuequeueNIQuDPDKKernel(isnt nvolved)UserspaceApplicationCP/ITask Schedulerqueuequeuequeuequeuequeuesmp queueNICQu eueeDPDKKernel(isnt nvolved)UserspaceP/IPApplicationTCT T P PTask Schedulerqueuequeuequeuequeuequeuesmp queuesNI CQ

9、euueiueDPDKKernel(isntUserspaceCore DatabaseTask Schedulerqueuequeuequeuequeuequeuemp queuesmp queueNICCQueiue involved)UserspaceScylla has its own task schedulerCPUCPUPromiseTask TaskTaskPromiseTask TaskTaskPromiseTask TaskTaskPromiseTask TaskTaskCPUPromiseTaskPromisePromiseTaskPromiseTaskPromiseTa

10、skCPUPromise PromisePromiseTaskPromisePromiseTaskPromise PromisePromiseTaskPromise PromisePromiseTaskCPUPromise is a pointer to eventually computed valueTask is a pointer to a lambda functionScheduleSrcheduleSrcheduleSrcheduleSrchedulerCPU CPU CPU CPU CPUThreadStackThreadThreadStackStackThreadStackT

11、hreadStackThreadStackThreadThreadStackStackThread is a function pointerStack is a byte array from 64k to megabytesSCYLLA IS DIFFERENTDMALog structured merge treeDBaware cacheUserspace I/O schedulerNUMA friendlyLog structured allocatorZero copyThread per coreLock-freeTask schedulerReactor programingC

12、+14Multi queuePoll modeUserspace TCP/IPScylla vs C* latency by KenshooDesign Decision: #5 Unified cacheCassandraScyllaKey cacheRow cacheOn-heap / Off-heapLinux page cacheSSTablesUnified cacheSSTablesComplex TuningCassandra Streaming configurationDesign decisions: #6 I/O schedulerScylla I/O Schedulin

13、gQueryCommitlogCompactionQueueQueueQueueUserspace I/O SchedulerDiskMax useful disk concurrencyI/O queued in FS/deviceNo queuesI/O scheduler result by KenshooMemtableSeastar SchedulerCompactionQueryRepairCommitlogSSDCompaction Backlog MonitorMemory MonitorAdjust priorityAdjust priorityWANCPUDesign Decision: #7 Workload conditioningWorkload Conditioning in practiceDisk cant keep up:workload conditioning will figure out the right request rateUpcoming releases+Enterprise release, based on 1.6+1.7 - May 2017CountersNew intra-node sharding algorithmSStableloader from 2.2/3.xDebian+2.0 S