[ClusterLabs] DRBD sync stalled at 100% ?

Sun Jun 28 14:16:51 EDT 2020

i was thinking about  a github issue, but it seems  that only 'linstor-server' has an issue section.

Best Regards,
Strahil Nikolov

На 28 юни 2020 г. 20:13:21 GMT+03:00, Eric Robinson <eric.robinson at psmnv.com> написа:
>I could if linbit had per-incident pricing. Unfortunately, they only
>offer yearly contracts, which is way more than I need.
>
>Get Outlook for Android<https://aka.ms/ghei36>
>
>________________________________
>From: Strahil Nikolov <hunter86_bg at yahoo.com>
>Sent: Sunday, June 28, 2020 4:11:47 AM
>To: Eric Robinson <eric.robinson at psmnv.com>; Cluster Labs - All topics
>related to open-source clustering welcomed <users at clusterlabs.org>
>Subject: RE: [ClusterLabs] DRBD sync stalled at 100% ?
>
>I guess you  can open an issue to linbit, as you still have the logs.
>
>Best Regards,
>Strahil Nikolov
>
>На 28 юни 2020 г. 8:19:59 GMT+03:00, Eric Robinson
><eric.robinson at psmnv.com> написа:
>>I fixed it with a drbd down/up.
>>
>>From: Users <users-bounces at clusterlabs.org> On Behalf Of Eric Robinson
>>Sent: Saturday, June 27, 2020 4:32 PM
>>To: Cluster Labs - All topics related to open-source clustering
>>welcomed <users at clusterlabs.org>; Strahil Nikolov
>><hunter86_bg at yahoo.com>
>>Subject: Re: [ClusterLabs] DRBD sync stalled at 100% ?
>>
>>Thanks for the feedback. I was hoping for a non-downtime solution. No
>>way to do that?
>>Get Outlook for Android<https://aka.ms/ghei36>
>>
>>________________________________
>>From: Strahil Nikolov
>><hunter86_bg at yahoo.com<mailto:hunter86_bg at yahoo.com>>
>>Sent: Saturday, June 27, 2020 2:40:38 PM
>>To: Cluster Labs - All topics related to open-source clustering
>>welcomed <users at clusterlabs.org<mailto:users at clusterlabs.org>>; Eric
>>Robinson <eric.robinson at psmnv.com<mailto:eric.robinson at psmnv.com>>
>>Subject: Re: [ClusterLabs] DRBD sync stalled at 100% ?
>>
>>I've  seen this  on a  test setup  after multiple  network
>disruptions.
>>I managed  to fix it by stopping drbd on all  nodes  and starting it
>>back.
>>
>>I guess  you can get downtime  and try that approach.
>>
>>
>>Best Regards,
>>Strahil Nikolov
>>
>>
>>
>>На 27 юни 2020 г. 16:36:10 GMT+03:00, Eric Robinson
>><eric.robinson at psmnv.com<mailto:eric.robinson at psmnv.com>> написа:
>>>I'm not seeing anything on Google about this. Two DRBD nodes lost
>>>communication with each other, and then reconnected and started sync.
>>>But then it got to 100% and is just stalled there.
>>>
>>>The nodes are 001db03a, 001db03b.
>>>
>>>On 001db03a:
>>>
>>>[root at 001db03a ~]# drbdadm status
>>>ha01_mysql role:Primary
>>>  disk:UpToDate
>>>  001db03b role:Secondary
>>>    replication:SyncSource peer-disk:Inconsistent done:100.00
>>>
>>>ha02_mysql role:Secondary
>>>  disk:UpToDate
>>>  001db03b role:Primary
>>>    peer-disk:UpToDate
>>>
>>>On 001drbd03b:
>>>
>>>[root at 001db03b ~]# drbdadm status
>>>ha01_mysql role:Secondary
>>>  disk:Inconsistent
>>>  001db03a role:Primary
>>>    replication:SyncTarget peer-disk:UpToDate done:100.00
>>>
>>>ha02_mysql role:Primary
>>>  disk:UpToDate
>>>  001db03a role:Secondary
>>>    peer-disk:UpToDate
>>>
>>>
>>>On 001db03a, here are the DRBD messages from the onset of the problem
>>>until now.
>>>
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b: PingAck
>did
>>>not arrive in time.
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Connected -> NetworkFailure ) peer( Primary -> Unknown )
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>UpToDate -> Consistent )
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b:
>>ack_receiver
>>>terminated
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b:
>Terminating
>>>ack_recv thread
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql: Preparing
>>>cluster-wide state change 2946943372 (1->-1 0/0)
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql: Committing
>>>cluster-wide state change 2946943372 (6ms)
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>Consistent -> UpToDate )
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b: Connection
>>>closed
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>NetworkFailure -> Unconnected )
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b: Restarting
>>>receiver thread
>>>Jun 26 22:34:27 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Unconnected -> Connecting )
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b: PingAck
>did
>>>not arrive in time.
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b: conn(
>>>Connected -> NetworkFailure ) peer( Secondary -> Unknown )
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b:
>>ack_receiver
>>>terminated
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b:
>Terminating
>>>ack_recv thread
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql/0 drbd0: new current
>>>UUID: D07A3D4B2F99832D weak: FFFFFFFFFFFFFFFD
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b: Connection
>>>closed
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b: conn(
>>>NetworkFailure -> Unconnected )
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b: Restarting
>>>receiver thread
>>>Jun 26 22:34:30 001db03a kernel: drbd ha01_mysql 001db03b: conn(
>>>Unconnected -> Connecting )
>>>Jun 26 22:34:33 001db03a pengine[1474]:  notice:  * Start
>>>p_drbd0:1        (                 001db03b )
>>>Jun 26 22:34:33 001db03a crmd[1475]:  notice: Initiating notify
>>>operation p_drbd0_pre_notify_start_0 locally on 001db03a
>>>Jun 26 22:34:33 001db03a crmd[1475]:  notice: Result of notify
>>>operation for p_drbd0 on 001db03a: 0 (ok)
>>>Jun 26 22:34:33 001db03a crmd[1475]:  notice: Initiating start
>>>operation p_drbd0_start_0 on 001db03b
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql 001db03b: Handshake
>>to
>>>peer 0 successful: Agreed network protocol version 113
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql 001db03b: Feature
>>>flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
>>>WRITE_ZEROES.
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql 001db03b: Starting
>>>ack_recv thread (from drbd_r_ha02_mys [2116])
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql 001db03b: Preparing
>>>remote state change 3920461435
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql 001db03b: Committing
>>>remote state change 3920461435 (primary_nodes=1)
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Connecting -> Connected ) peer( Unknown -> Primary )
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>UpToDate -> Outdated )
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>drbd_sync_handshake:
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>self
>>>492F8D33A72A8E08:0000000000000000:659DC04F5C85B6E4:8254EEA2EC50AD7C
>>>bits:0 flags:120
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>peer
>>>5A6B1EBE80500C39:492F8D33A72A8E09:659DC04F5C85B6E4:51A00A23ED88187A
>>>bits:1 flags:120
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>uuid_compare()=-2 by rule 50
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>pdsk( DUnknown -> UpToDate ) repl( Off -> WFBitMapT )
>>>Jun 26 22:34:34 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 23(1), total
>>23;
>>>compression: 100.0%
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>send
>>>bitmap stats [Bytes(packets)]: plain 0(0), RLE 23(1), total 23;
>>>compression: 100.0%
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>helper command: /sbin/drbdadm before-resync-target
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>helper command: /sbin/drbdadm before-resync-target exit code 0 (0x0)
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>Outdated -> Inconsistent )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>repl( WFBitMapT -> SyncTarget )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>Began resync as SyncTarget (will sync 4 KB [1 bits set]).
>>>Jun 26 22:34:35 001db03a crmd[1475]:  notice: Initiating notify
>>>operation p_drbd0_post_notify_start_0 locally on 001db03a
>>>Jun 26 22:34:35 001db03a crmd[1475]:  notice: Initiating notify
>>>operation p_drbd0_post_notify_start_0 on 001db03b
>>>Jun 26 22:34:35 001db03a crmd[1475]:  notice: Transition aborted by
>>>status-2-master-p_drbd0 doing create master-p_drbd0=10000: Transient
>>>attribute change
>>>Jun 26 22:34:35 001db03a crmd[1475]:  notice: Result of notify
>>>operation for p_drbd0 on 001db03a: 0 (ok)
>>>Jun 26 22:34:35 001db03a crmd[1475]:  notice: Initiating monitor
>>>operation p_drbd0_monitor_60000 on 001db03b
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>Resync done (total 1 sec; paused 0 sec; 4 K/sec)
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>updated UUIDs
>>>5A6B1EBE80500C38:0000000000000000:492F8D33A72A8E08:659DC04F5C85B6E4
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>Inconsistent -> UpToDate )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>repl( SyncTarget -> Established )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>helper command: /sbin/drbdadm after-resync-target
>>>Jun 26 22:34:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>helper command: /sbin/drbdadm after-resync-target exit code 0 (0x0)
>>>Jun 26 22:34:35 001db03a crmd[1475]:  notice: Transition aborted by
>>>status-2-master-p_drbd0 doing modify master-p_drbd0=1000: Transient
>>>attribute change
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql 001db03b: Handshake
>>to
>>>peer 0 successful: Agreed network protocol version 113
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql 001db03b: Feature
>>>flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
>>>WRITE_ZEROES.
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql 001db03b: Starting
>>>ack_recv thread (from drbd_r_ha01_mys [2110])
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql 001db03b: Preparing
>>>remote state change 3458191960
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql 001db03b: Committing
>>>remote state change 3458191960 (primary_nodes=2)
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql 001db03b: conn(
>>>Connecting -> Connected ) peer( Unknown -> Secondary )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>drbd_sync_handshake:
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>self
>>>D07A3D4B2F99832D:50AE57670FCB98C3:7DDDDEEEEEA477C4:B75C5B6B7AAFBB6A
>>>bits:22 flags:120
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>peer
>>>50AE57670FCB98C2:0000000000000000:7DDDDEEEEEA477C4:D2AAA82A5FF6EE84
>>>bits:0 flags:20
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>uuid_compare()=2 by rule 70
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>pdsk( DUnknown -> Consistent ) repl( Off -> WFBitMapS )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>send
>>>bitmap stats [Bytes(packets)]: plain 0(0), RLE 83(1), total 83;
>>>compression: 100.0%
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>pdsk( Consistent -> Outdated )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>receive bitmap stats [Bytes(packets)]: plain 0(0), RLE 83(1), total
>>83;
>>>compression: 100.0%
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>helper command: /sbin/drbdadm before-resync-source
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>helper command: /sbin/drbdadm before-resync-source exit code 0 (0x0)
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>pdsk( Outdated -> Inconsistent ) repl( WFBitMapS -> SyncSource )
>>>Jun 26 22:34:35 001db03a kernel: drbd ha01_mysql/0 drbd0 001db03b:
>>>Began resync as SyncSource (will sync 212 KB [53 bits set]).
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql 001db03b: sock was
>>>shut down by peer
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Connected -> BrokenPipe ) peer( Primary -> Unknown )
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>UpToDate -> Consistent )
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql 001db03b: meta
>>>connection shut down by peer.
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql 001db03b:
>>ack_receiver
>>>terminated
>>>Jun 26 22:36:35 001db03a kernel: drbd ha02_mysql 001db03b:
>Terminating
>>>ack_recv thread
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql: Preparing
>>>cluster-wide state change 2546365252 (1->-1 0/0)
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql: Committing
>>>cluster-wide state change 2546365252 (9ms)
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>Consistent -> UpToDate )
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql 001db03b: Connection
>>>closed
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>BrokenPipe -> Unconnected )
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql 001db03b: Restarting
>>>receiver thread
>>>Jun 26 22:36:39 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Unconnected -> Connecting )
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql 001db03b: Handshake
>>to
>>>peer 0 successful: Agreed network protocol version 113
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql 001db03b: Feature
>>>flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
>>>WRITE_ZEROES.
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql 001db03b: Starting
>>>ack_recv thread (from drbd_r_ha02_mys [2116])
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql 001db03b: Preparing
>>>remote state change 1109150886
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql 001db03b: Committing
>>>remote state change 1109150886 (primary_nodes=1)
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Connecting -> Connected ) peer( Unknown -> Primary )
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>drbd_sync_handshake:
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>self
>>>5A6B1EBE80500C38:0000000000000000:492F8D33A72A8E08:659DC04F5C85B6E4
>>>bits:0 flags:120
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>peer
>>>5A6B1EBE80500C39:0000000000000000:492F8D33A72A8E08:659DC04F5C85B6E4
>>>bits:0 flags:120
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>uuid_compare()=0 by rule 38
>>>Jun 26 22:36:40 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql 001db03b: PingAck
>did
>>>not arrive in time.
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Connected -> NetworkFailure ) peer( Primary -> Unknown )
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>UpToDate -> Consistent )
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql 001db03b:
>>ack_receiver
>>>terminated
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql 001db03b:
>Terminating
>>>ack_recv thread
>>>Jun 26 22:39:41 001db03a kernel: drbd ha02_mysql 001db03b: sock was
>>>shut down by peer
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql: Preparing
>>>cluster-wide state change 3067178175 (1->-1 0/0)
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql: Committing
>>>cluster-wide state change 3067178175 (8ms)
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql/0 drbd1: disk(
>>>Consistent -> UpToDate )
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Connection
>>>closed
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>NetworkFailure -> Unconnected )
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Restarting
>>>receiver thread
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Unconnected -> Connecting )
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Handshake
>>to
>>>peer 0 successful: Agreed network protocol version 113
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Feature
>>>flags enabled on protocol level: 0xf TRIM THIN_RESYNC WRITE_SAME
>>>WRITE_ZEROES.
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Starting
>>>ack_recv thread (from drbd_r_ha02_mys [2116])
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Preparing
>>>remote state change 2747304939
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: Committing
>>>remote state change 2747304939 (primary_nodes=1)
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql 001db03b: conn(
>>>Connecting -> Connected ) peer( Unknown -> Primary )
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>drbd_sync_handshake:
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>self
>>>5A6B1EBE80500C38:0000000000000000:492F8D33A72A8E08:659DC04F5C85B6E4
>>>bits:0 flags:120
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>peer
>>>5A6B1EBE80500C39:0000000000000000:492F8D33A72A8E08:659DC04F5C85B6E4
>>>bits:0 flags:120
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>uuid_compare()=0 by rule 38
>>>Jun 26 22:39:45 001db03a kernel: drbd ha02_mysql/0 drbd1 001db03b:
>>>pdsk( DUnknown -> UpToDate ) repl( Off -> Established )
>>>
>>>[cid:image001.png at 01D64C5C.FA76F310]
>>>
>>>Disclaimer : This email and any files transmitted with it are
>>>confidential and intended solely for intended recipients. If you are
>>>not the named addressee you should not disseminate, distribute, copy
>>or
>>>alter this email. Any views or opinions presented in this email are
>>>solely those of the author and might not represent those of Physician
>>>Select Management. Warning: Although Physician Select Management has
>>>taken reasonable precautions to ensure no viruses are present in this
>>>email, the company cannot accept responsibility for any loss or
>damage
>>>arising from the use of this email or attachments.
>>Disclaimer : This email and any files transmitted with it are
>>confidential and intended solely for intended recipients. If you are
>>not the named addressee you should not disseminate, distribute, copy
>or
>>alter this email. Any views or opinions presented in this email are
>>solely those of the author and might not represent those of Physician
>>Select Management. Warning: Although Physician Select Management has
>>taken reasonable precautions to ensure no viruses are present in this
>>email, the company cannot accept responsibility for any loss or damage
>>arising from the use of this email or attachments.
>>Disclaimer : This email and any files transmitted with it are
>>confidential and intended solely for intended recipients. If you are
>>not the named addressee you should not disseminate, distribute, copy
>or
>>alter this email. Any views or opinions presented in this email are
>>solely those of the author and might not represent those of Physician
>>Select Management. Warning: Although Physician Select Management has
>>taken reasonable precautions to ensure no viruses are present in this
>>email, the company cannot accept responsibility for any loss or damage
>>arising from the use of this email or attachments.
>Disclaimer : This email and any files transmitted with it are
>confidential and intended solely for intended recipients. If you are
>not the named addressee you should not disseminate, distribute, copy or
>alter this email. Any views or opinions presented in this email are
>solely those of the author and might not represent those of Physician
>Select Management. Warning: Although Physician Select Management has
>taken reasonable precautions to ensure no viruses are present in this
>email, the company cannot accept responsibility for any loss or damage
>arising from the use of this email or attachments.