Moon Light Box

Time is Money

Linux - Recovery DRBD Split Brain

| Comments

When occorred split brain, you can see below message (Primary/Unknown) or (Secondary/Unknown).

Node A $ cat /proc/drbd

version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@, 2014-08-17 22:54:26
 0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown   r-----
    ns:0 nr:0 dw:886517 dr:2106580 al:368 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:5770156

Node B $ cat /proc/drbd
version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@, 2014-08-17 22:54:26
 0: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown   r-----
    ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:16044

Node A (Primary): Assumed your main data in Node A.

First, check your DRBD partition is mounted.

Node A $ sudo drbdadm primary all
Node A $ sudo drbdadm connect all

Synchronizing data.

Node A $ cat /proc/drbd

version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@, 2014-08-17 22:54:26
 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r-----
    ns:379080 nr:0 dw:886698 dr:2495004 al:370 bm:0 lo:2 pe:0 ua:2 ap:0 ep:1 wo:f oos:5391140
        [>...................] sync'ed:  6.7% (5264/5632)M
        finish: 0:03:04 speed: 29,160 (29,160) K/sec

Synchronization Result.

Node A $ cat /proc/drbd

version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@, 2014-08-17 22:54:26
 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r-----
    ns:233565451 nr:0 dw:233561279 dr:1788854 al:51494 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

Node B (Secondary):

First, check your DRBD partition is unmounted.

Node B $ sudo drbdadm disconnect all
Node B $ sudo drbdadm secondary all
Node B $ sudo drbdadm -- --discard-my-data connect all

Synchronizing data.

Node B $ cat /proc/drbd

version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@, 2014-08-17 22:54:26
 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r-----
    ns:0 nr:4400804 dw:4400804 dr:0 al:0 bm:0 lo:0 pe:6 ua:0 ap:0 ep:1 wo:f oos:1369416
        [==============>.....] sync'ed: 76.3% (1336/5632)M
        finish: 0:00:37 speed: 36,028 (34,380) want: 102,400 K/sec

Synchronization Result.

Node B $ cat /proc/drbd

version: 8.4.5 (api:1/proto:86-101)
GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by mockbuild@, 2014-08-17 22:54:26
 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r-----
    ns:0 nr:233565357 dw:233565357 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0

Reference:https://drbd.linbit.com/users-guide/s-resolve-split-brain.html

Comments

comments powered by Disqus