lvorySQL 2025生态大会暨PostgreSQL高峰论坛：Patroni究竟是什么

What

is

Patroni,

really?Polina

Bungina,

AlexanderKukushkinHOW2025PostgreSQL&IvorySQL

EcoConferenceAbout

usAlexander

Kukushkin

Polina

Bungina•PrincipalSoftware

Engineer

@Microsoft•

The

Patroniguy•

akukushkin@•Software

Engineer@Zalando•polina.bungina@zalando.de●What

is

it,

really?●

Automaticfailoverdonewrong●Patroni

overview○how

it

works?○notablefeaturesAgenda3Whatis

it,

really?Whatis

it,

really?●

Originatedfrom

GovernorprojectbyCompose,in

2015●

Mainfunctions:○

Automaticfailover○

Clustercreationand

initialsetup○

Clustermanagement○

~

Monitoring5AutomaticfailoverdonewrongprimaryWALstreamstandbyhealth-checkAutomaticfailoverdonewrongRunningtwonodes

only7primaryWALstreamstandby

health-checkShould

I

…promote?AutomaticfailoverdonewrongRunningtwonodes

only8primaryprimaryAutomaticfailoverdonewrongRunningtwonodes

only9standby

primaryhealth-checkWALstreamShould

I

…promote?Avoidingsplit-brainAutomaticfailoverdonewrongSTONITH10Avoidingsplit-brain●STONITH(shootthe

other

node

inthe

head)●Must

use

a

secondary

network●Almost

impossible

to

get

it

rightAutomaticfailoverdonewrong11primaryVWALstreamstandbywitness

node

(arbiter)health-checkhealth-checkSinglewitnessnodeAutomaticfailoverdonewrong12primaryVWALstreamstandbyhealth-checkhealth-checkSinglewitnessnodeAutomaticfailoverdonewrong13primaryWALstreamstandbywitness

node

(arbiter)health-checkPromotestandby! health-checkSinglewitnessnodeSTONITHAutomaticfailoverdonewrong14witness

node

(arbiter)primaryprimaryVhealth-checkSinglewitnessnodeAutomaticfailoverdonewrong15Thingstoconsider●Think

about

network

partition●

Preventsplit-brain→fencing○

STONITH●

Shutdown●Kill

old

connections,

re-configure

proxy○

Self-fencing(locally)●

WatchdogAutomaticfailoverdonewrong16primaryagentWALstreamstandbyagentwitness

node

(arbiter)LocalagentsAutomaticfailoverdonewrong17DC2DC3standbyagentstandbyagentprimary

agent

witness

node

(arbiter)LocalagentsAutomaticfailoverdonewrongDC118AVDC2DC3standbyagentstandbyagentprimary

agent

witness

node

(arbiter)LocalagentsDC1(isolated)Automaticfailoverdonewrong19AVstandbyprimarywitness

node

(arbiter)Buthowto

do

it

right?agentagentWALstreamQuorum20standbyprimaryButhowto

do

it

right?agentagentWALstreamQuorum21standbyprimaryWALstreamButhowto

do

it

right?agentQuorum22Patroni:how

itworks?How

itworks?General

idea●

Statestored

in

DistributedConfigurationStore(DCS)○Etcd,ZooKeeper,

Consul,

Kubernetes

control-plane●Built-indistributedconsensus

(RAFT,Zab)●

Key-valuestore●Atomic

CAS

(compare-and-swap)

operations●

Lease/Session/TTLtoexpiredata○

/leader,/members/*●

Watchesfor

keys24primary

node

Astandby

node

B4WATCHES/leaderPatronioverview/leader:

“A”,ttl:30UPDATES

/leader,/status,

…UPDATES

/members/BHow

itworks?AV25Aprimary

nodeAstandby

node

Cstandby

node

BprevValue=”A”)SUCCESS/leader:

“A”,ttl:30UPDATE(“/leader”,“A”,ttl=30,Leader

raceWATCH/leaderWATCH/leaderHow

itworks?26primary

nodeAstandby

node

Cstandby

node

B/leader:

“A”,ttl:

1Leader

raceWATCH/leaderWATCH/leaderHow

itworks?27standby

node

Cstandby

node

BNOTIFY(“/leader”,expired=true)NOTIFY(“/leader”,expired=true)Leader

raceHow

itworks?28standby

node

B1)GETA:8008/patroni->timeout2)GETC:8008/patroni

->

wal_position:

1001)GETA:8008/patroni->timeout2)GETC:8008/patroni

->

wal_position:

100standby

node

CLeader

raceHow

itworks?29standby

node

Cstandby

node

BCREATE(“/leader”,“B”,ttl=30,prevExists=False)/leader:

“B”,ttl:30CREATE(“/leader”,“C”,ttl=30,Leader

raceprevExists=False)SUCCESSHow

itworks?FAIL30Astandby

node

BAVUPDATE

/members/B

4WATCH/leaderprimary

node

Aread-only

instanceUPDATE

/leader/leader:

“A”,ttl:

19Self-fencingHow

itworks?31standby

node

Ademotestandby

node

Bread-only

instance4WATCH/leader/leader:

“A”,ttl:

9Self-fencingUPDATE

/leaderUPDATE

/members/BHow

itworks?AV32standby

node

A

1.

NOTIFIES/leaderexpiredstandby

node

Bpromoteread-only

instance/leader:

“A”,ttl:

0Self-fencing2.

CREATES

/leaderHow

itworks?3.AV33update/statuswrite/failoverupdate/sync…update/leaderget

/Communicationwith

DCS–leader(wholecluster)sleep

for

loop_waitHA

loopretriableHow

itworks?34(10)

(10)

(30)loop_wait

+2*retry_timeout

<=ttlget

/

update

/leaderttl,loop_wait,retry_timeout(wholecluster)How

itworks?35/*

global(dynamic)configuration*/

/*

cluster

identifier

*//*

whoistheprimary?

*//*

discovery

*//*

failover

history

*//*

manual

failover/switchover

*/

/*synchronousmode*//service/demo/config/service/demo/initialize/service/demo/leader/service/demo/members/patroni1/service/demo/members/patroni2/service/demo/members/patroni3/service/demo/status/service/demo/history/service/demo/failover/service/demo/syncDatastored

in

DCS$

etcdctl

get

--keys-only

--prefix

/service/demo}How

itworks?36+patroni1+|+|Leader+|running+|1+|+patroni2||Replica|streaming|1|0patroni3+|+|+Replica|+streaming|+1|+0+Datastored

in

DCSdata

retrievedfrom

DCS$

patronictl

list+Cluster:demo

(7497665970948870167)--------+----+-----------+|Member|Host|Role

|State|TL|Lag

in

MB|How

itworks?37Datastored

in

DCS$

etcdctl

get

--print-value-only

--prefix

/service/demo/leader

patroni1$

etcdctl

get

--print-value-only

--prefix

/service/demo/initialize7497665970948870167How

itworks?38Datastored

in

DCS$

etcdctl

get

--keys-only

--prefix/service/demo/members/patroni2{"conn_url":"postgres://:5432/postgres","api_url":

":8008/patroni","state":

"running","role":

"replica","version":

"4.0.5","xlog_location":

67425896,

/*

max(receive_lsnor0,replay_lsn

or

0)*/

"replication_state":"streaming","timeline":

1}How

itworks?39Datastored

in

DCS$

etcdctl

get

--print-value-only

--prefix

/service/demo/status

{"optime":

67425896,

/*

pg_current_wal_flush_lsn()

*/

"slots":

{},"retain_slots":["patroni1","patroni2",

/*

member_slots_ttl

*/"patroni3"]}"patroni2":

67425896,"patroni3":

67425896,/*

members

slots*/"patroni1":

67425896,“my_logical_slot:

67425700/*

permanent

slots

*/How

itworks?40Datastored

in

DCS$

etcdctl

get

--print-value-only

--prefix

/service/demo/config{"loop_wait":

10,"ttl":

30,"retry_timeout":

10,"maximum_lag_on_failover":1048576,"postgresql":{"parameters":{"max_connections":100},

/*

applied

to

all

members(global)*/"use_pg_rewind":

true},"synchronous_mode":

"quorum"}How

itworks?41Whatelse?Whatelse?Notablefeatures●Standbycluster–

runningcascading

replicationto

a

remote

datacenter

(reg