Mongodb事务介绍与最佳实践

1、MongoDB事务介绍与最佳实践目录 CONTENTS事务核心原理分析案例分享multi-document transactionsfor shard clusterMongoDB 4.2multi-document transactions for replica sets MongoDB v4.0single-document transactions Two-phase commit MongoDB v3.6事务的前世今生wiredTiger supportMongoDB v3.0写代码，看效果：事务的隔离client = MongoClient(mongodb:/28:60001/)

2、db = client.crminventory = db.inventorysession = client.start_session() #开启一个sessionsession.start_transaction() #在session内部，开启一个事务 inventory.insert_one(_id: 4, model:switch, count: 200, session=session) doc1 = inventory.find_one(_id: 4, session=session)pprint.pprint(doc1)doc2 = inventory.find_one(_i

3、d: 4) pprint.pprint(doc2)mit_transaction()#提交事务 doc3 = inventory.find_one(_id: 4) pprint.pprint(doc3)session.end_session()#结束session在session内部查询，可以访问到前面插入的数据在session外部查询，事务 没有提交，访问不到前面插 入的数据在session外部查询，事务 已提交，可以访问到前面插 入的数据写代码，看效果：事务的冲突client = MongoClient(mongodb:/28:60001/) db = client.crminventor

4、y = db.inventorysession1 = client.start_session()#开启一个session1 session1.start_transaction()# 开 启 一 个 事 务 1 inventory.delete_one(_id:4, session=session1)doc1 = inventory.find_one(_id: 4,session=session1)pprint.pprint(doc1)#输出none，说明在事务里面已经删除session2 = client.start_session()#再开启一个session2session2.star

5、t_transaction()# 再 开 启 一 个 事 务 2 inventory.delete_one(_id:4, session=session2)#会发生事务冲突session1.abort_transaction()#终止事务1session1.end_session()#结束session1session2.abort_transaction()#终止事务2session2.end_session()#结束session2 doc2 = inventory.find_one(_id: 4)pprint.pprint(doc2)#在事务外可以找到，说明事务1被终止后，回滚了在ses

6、sion内部查询，数据 已被删除，访问不到两个未提交的事务，同时修 改相同的文档，产生冲突事务总结不属于你的，看不到（Snapshot隔离）做了不该做的，会冲突（MVCC机制）Snapshot是什么？Snapshot is a view of a data source from Wired Tiger.ABCDEFGH9.Committed Data in WT1.P1(A)2.P2(B)3.P3(C)4.P4(D)Snapshot1 in t11.P1(A)2.P2(B)3.P3(C)4.P4(D)Snapshot2 in t2Update D to D1ABCD1EFGH9.Commit

7、ted Data in WT1.P1(A)2.P2(B)3.P3(C)4.P4(D1)Snapshot3 in t3多版本机制内存里面会保存同一条记录的多个版本比较版本号的大小，判断是否冲突并发时如何发现冲突？123A事务首先从表中读取到要修改的行数据，读取到库存值为100，行记录的版本号为1B事务也从中读取到要修改的相同行数据，读取库存值为100，行记录版本号为1。A事务修改库存值后提交，同时行记录版本 号加1，即变为2，大于一开始读取到的版本 号1，A事务可以提交。4但B事务提交时发现此时行记录版本号已经 变为2，产生冲突，B事务提交失败。5B事务尝试重新提交，此时读取到的版本号为2，加1

8、，即变为3，不会产生冲突正常提交。事务在内存里面的数据结构是什么样子？wt_transaction transaction_id, snapshot_object, operation_array, log_buf, transaction_state, timestampstransaction_id为事务的唯一标识一直递增，由存储引擎内部的全局事务管理模块进行分配snapshot_object为事务快照operation_array为本次事务中执行的操作动作，用于事务回滚。log_buf为事务对应的事务日志缓冲区，会被持久化，利用其重做事务transaction_state为事物的当前状态

9、写代码，看效果：事务与复制集client = MongoClient(mongodb:/28:60001/)db = client.crm inventory = db.inventorysession1 = client.start_session()# 开 启 一 个 session session1.start_transaction()#开启一个事务 inventory.insert_one(_id: 5, model:switch, count: 300, session=session1)i = 0while (i1):print(“inside the transaction,

10、before commit.”) #让程序无限循 mit_transaction()#提交事务session1.end_session()#结束session事务提交前，观察oplog集合、待插入数据的集合，都没 有数据写入事务提交后，观察oplog集合、待插入数据的集合，有数 据写入Oplog在复制集比较重要，放在一个事务里面，确保“原子性”Oplog是什么样子？ ts : Timestamp(1573482598, 1), t : NumberLong(5), h : NumberLong(0), v : 2, op :c, ns : admin.$cmd, wall : ISODate(

11、2019-11-11T14:29:58.918Z), lsid : id : UUID(3c762cf4-0be6-4791-84a1-05e1c1533ed3), uid :BinData(0,47DEQpj8HBSa+/TImW+5JCeuQeRkm5N MpJWZG3hSuFU=) ,txnNumber : NumberLong(1), prevOpTime : ts : Timestamp(0, 0), t : NumberLong(-1) ,o : applyOps : op : i, ns : crm.inventory, ui : UUID(0cbc8551-dc06-4d74-

12、 b3bf-cd35faf2377d),o : _id : 5, model : switch, count : 300 ts：操作时间lsid：session的标识符 txnNumber：事务编号 applyOps：具体的操作动作Oplog在同步的过程中产生的几个问题思考？时序问题？选举发生后数据不一致？全局锁vs并发全局锁，效率低并行apply，通过时间戳保证一致性重新选举新的Primary节点后，如何保证数据一致性？少数服从多数的原则Read Concern解决“脏读”You can specify a readConcern of majority to read data that

13、has been written to a majority of nodes and thus cannot be rolled back.By default, MongoDB uses a readConcern of local to return the most recent data available to the node at the time of the query, even if the data has not been persisted to a majority of nodes and may be rolled back.整体认识：一次完整的写流程Pyt

14、hon APIC APIJAVA APITransactionsRow StorageColumn StorageSnapshotsCacheRePage ad/WriteLoggingDatabase files12 3 4Block management2.5事务提交前WiredTiger Engine Schema &Cursors事务提交后Transaction login bufferJouLrnogalfilloegs files事务日志journalCheckpoint探究写操作发生后内存的变化RootpageInternalpageleafpageInternalpagedir

15、tyleafpageleafpageCacheReconciling&CheckpointupdatelistinsertlistNew RootpageInternalpageleafpageInternalpageleafpageNewleafpagePage on diskWT Block management1234When reconciling a row store leaf page we traverse the in- memory page. We:Create a buffer that can hold a page worth of dataStart traver

16、sing the in-memory page, by first looking at the head of the insert list, then alternating between looking at the key/values read from disk (and any updates that were done to them), and the next entry in the insert list, until we get to the end of the insert list.按行存储的叶子page的数据结构struct wt_pageWT_ROW

17、 *row; uint32_t entries;WT_PAGE_MODIFY *modify;/* Key/value pairs */* Leaf page entries */* modified information. */* Dirty bytes added to the cache. */*insert; /* Inserted items for row-store. */*update; /* Updated items for row-stores. */struct wt_page_modify size_t bytes_dirty; WT_INSERT_HEAD WT_

18、UPDATE/* update transaction */* forward-linked list */* update length */*updata data*/struct wt_update uint64_t txnid; WT_UPDATE *next; uint32_t size;upd;struct wt_insert WT_UPDATE *upd;/* value */ uint32_t offset;/* row-store key data start */uint32_t size;/* row-store key data size */ WT_INSERT *n

19、ext0;/* forward-linked skip list */一个page的生命周期总结pages are read into memory from disk and their state set to WT_REF_MEM.When the page is selected for eviction, the page state is set to WT_REF_LOCKED. In all cases, evicting threads reset the pages state when finished with the page: if eviction was suc

20、cessful (a clean page was discarded, and a dirty page was written to disk and then discarded), the page state is set to WT_REF_DISK; if eviction failed because the page was busy, page state is reset to WT_REF_MEM.Readers check the state field and if its WT_REF_MEM, they set a hazard pointer to the page, flush memory and re-confirm th