Broadcast message from root@itafgfsxen02 (/dev/ttyS0) at 12:29 ... The system is going down for reboot NOW! Mar 22 12:29:28 debug [TOTEM ] entering GATHER state from 12(join during operational state). Mar 22 12:29:31 debug [TOTEM ] entering GATHER state from 11(merge during join). Mar 22 12:29:35 debug [TOTEM ] entering GATHER state from 0(consensus timeout). Mar 22 12:29:35 debug [TOTEM ] Creating commit token because I am the rep. Mar 22 12:29:35 debug [TOTEM ] Saving state aru 4a high seq received 4a Mar 22 12:29:35 debug [TOTEM ] Storing new sequence id for ring d94 Mar 22 12:29:35 debug [TOTEM ] entering COMMIT state. Mar 22 12:29:35 debug [TOTEM ] got commit token Mar 22 12:29:35 debug [TOTEM ] entering RECOVERY state. Mar 22 12:29:35 debug [TOTEM ] TRANS [0] member 172.17.0.35: Mar 22 12:29:35 debug [TOTEM ] position [0] member 172.17.0.35: Mar 22 12:29:35 debug [TOTEM ] previous ring seq d90 rep 172.17.0.34 Mar 22 12:29:35 debug [TOTEM ] aru 4a high delivered 4a received flag 1 Mar 22 12:29:35 debug [TOTEM ] Did not need to originate any messages in recovery. Mar 22 12:29:35 debug [TOTEM ] got commit token Mar 22 12:29:35 debug [TOTEM ] Sending initial ORF token Mar 22 12:29:35 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Mar 22 12:29:35 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:35 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Mar 22 12:29:35 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:35 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Mar 22 12:29:35 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:35 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Mar 22 12:29:35 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:35 debug [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0 Mar 22 12:29:35 debug [TOTEM ] Resetting old ring state Mar 22 12:29:35 debug [TOTEM ] recovery to regular 1-0 Mar 22 12:29:35 debug [MAIN ] Member left: r(0) ip(172.17.0.34) Mar 22 12:29:35 debug [MAIN ] Member left: r(0) ip(172.17.0.42) Mar 22 12:29:35 debug [TOTEM ] waiting_trans_ack changed to 1 Mar 22 12:29:35 debug [TOTEM ] entering OPERATIONAL state. Mar 22 12:29:35 notice [TOTEM ] A new membership (172.17.0.35:3476) was formed. Members left: 2016001 2016003 Mar 22 12:29:35 debug [SYNC ] Committing synchronization for corosync configuration map access Mar 22 12:29:35 debug [QB ] Not first sync -> no action Mar 22 12:29:35 debug [CPG ] comparing: sender r(0) ip(172.17.0.35) ; members(old:3 left:2) Mar 22 12:29:35 debug [CPG ] chosen downlist: sender r(0) ip(172.17.0.35) ; members(old:3 left:2) Mar 22 12:29:35 debug [CPG ] left_list_entries:2 Mar 22 12:29:35 debug [CPG ] left_list[0] group:dlm:controld\x00, ip:r(0) ip(172.17.0.34) , pid:6312 Mar 22 12:29:35 debug [CPG ] left_list[1] group:dlm:controld\x00, ip:r(0) ip(172.17.0.42) , pid:6302 Mar 22 12:29:35 debug [CPG ] left_list_entries:2 Mar 22 12:29:35 debug [CPG ] left_list[0] group:dlm:ls:GFS\x00, ip:r(0) ip(172.17.0.34) , pid:6312 Mar 22 12:29:35 debug [CPG ] left_list[1] group:dlm:ls:GFS\x00, ip:r(0) ip(172.17.0.42) , pid:6302 Mar 22 12:29:35 debug [CPG ] got joinlist message from node 1ec302 Mar 22 12:29:35 debug [SYNC ] Committing synchronization for corosync cluster closed process group service v1.01 Mar 22 12:29:35 debug [CPG ] joinlist_messages[0] group:dlm:ls:GFS\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:35 debug [CPG ] joinlist_messages[1] group:dlm:controld\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:35 debug [QUORUM] flags: quorate: Yes Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:35 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:35 debug [QUORUM] nodeinfo message[2016002]: votes: 1, expected: 3 flags: 1 Mar 22 12:29:35 debug [QUORUM] flags: quorate: Yes Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:35 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:35 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:35 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:35 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:35 debug [QUORUM] quorum lost, blocking activity Mar 22 12:29:35 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:35 debug [QUORUM] nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Mar 22 12:29:35 debug [SYNC ] Committing synchronization for corosync vote quorum service v1.0 Mar 22 12:29:35 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:35 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:35 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:35 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:35 notice [QUORUM] This node is within the non-primary component and will NOT provide any services. Mar 22 12:29:35 notice [QUORUM] Members[1]: 2016002 Mar 22 12:29:35 debug [QUORUM] sending quorum notification to (nil), length = 52 Mar 22 12:29:35 notice [MAIN ] Completed service synchronization, ready to provide service. Mar 22 12:29:35 debug [TOTEM ] waiting_trans_ack changed to 0 Mar 22 12:29:35 debug [TOTEM ] entering GATHER state from 9(merge during operational state). Mar 22 12:29:39 debug [TOTEM ] entering GATHER state from 0(consensus timeout). Mar 22 12:29:39 debug [TOTEM ] Creating commit token because I am the rep. Mar 22 12:29:39 debug [TOTEM ] Saving state aru 5 high seq received 5 Mar 22 12:29:39 debug [TOTEM ] Storing new sequence id for ring d98 Mar 22 12:29:39 debug [TOTEM ] entering COMMIT state. Mar 22 12:29:39 debug [TOTEM ] got commit token Mar 22 12:29:39 debug [TOTEM ] entering RECOVERY state. Mar 22 12:29:39 debug [TOTEM ] TRANS [0] member 172.17.0.35: Mar 22 12:29:39 debug [TOTEM ] position [0] member 172.17.0.35: Mar 22 12:29:39 debug [TOTEM ] previous ring seq d94 rep 172.17.0.35 Mar 22 12:29:39 debug [TOTEM ] aru 5 high delivered 5 received flag 1 Mar 22 12:29:39 debug [TOTEM ] Did not need to originate any messages in recovery. Mar 22 12:29:39 debug [TOTEM ] got commit token Mar 22 12:29:39 debug [TOTEM ] Sending initial ORF token Mar 22 12:29:39 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Mar 22 12:29:39 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:39 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Mar 22 12:29:39 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:39 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Mar 22 12:29:39 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:39 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Mar 22 12:29:39 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:39 debug [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0 Mar 22 12:29:39 debug [TOTEM ] Resetting old ring state Mar 22 12:29:39 debug [TOTEM ] recovery to regular 1-0 Mar 22 12:29:39 debug [TOTEM ] waiting_trans_ack changed to 1 Mar 22 12:29:39 debug [TOTEM ] entering OPERATIONAL state. Mar 22 12:29:39 notice [TOTEM ] A new membership (172.17.0.35:3480) was formed. Members Mar 22 12:29:39 debug [SYNC ] Committing synchronization for corosync configuration map access Mar 22 12:29:39 debug [QB ] Not first sync -> no action Mar 22 12:29:39 debug [CPG ] comparing: sender r(0) ip(172.17.0.35) ; members(old:1 left:0) Mar 22 12:29:39 debug [CPG ] chosen downlist: sender r(0) ip(172.17.0.35) ; members(old:1 left:0) Mar 22 12:29:39 debug [CPG ] got joinlist message from node 1ec302 Mar 22 12:29:39 debug [SYNC ] Committing synchronization for corosync cluster closed process group service v1.01 Mar 22 12:29:39 debug [CPG ] joinlist_messages[0] group:dlm:ls:GFS\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:39 debug [CPG ] joinlist_messages[1] group:dlm:controld\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:39 debug [QUORUM] flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:39 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:39 debug [QUORUM] nodeinfo message[2016002]: votes: 1, expected: 3 flags: 0 Mar 22 12:29:39 debug [QUORUM] flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:39 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:39 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:39 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:39 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:39 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:39 debug [QUORUM] nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Mar 22 12:29:39 debug [SYNC ] Committing synchronization for corosync vote quorum service v1.0 Mar 22 12:29:39 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:39 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:39 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:39 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:39 notice [QUORUM] Members[1]: 2016002 Mar 22 12:29:39 debug [QUORUM] sending quorum notification to (nil), length = 52 Mar 22 12:29:39 notice [MAIN ] Completed service synchronization, ready to provide service. Mar 22 12:29:39 debug [TOTEM ] waiting_trans_ack changed to 0 Mar 22 12:29:39 debug [TOTEM ] entering GATHER state from 9(merge during operational state). Mar 22 12:29:42 debug [TOTEM ] entering GATHER state from 0(consensus timeout). Mar 22 12:29:42 debug [TOTEM ] Creating commit token because I am the rep. Mar 22 12:29:42 debug [TOTEM ] Saving state aru 5 high seq received 5 Mar 22 12:29:42 debug [TOTEM ] Storing new sequence id for ring d9c Mar 22 12:29:42 debug [TOTEM ] entering COMMIT state. Mar 22 12:29:42 debug [TOTEM ] got commit token Mar 22 12:29:42 debug [TOTEM ] entering RECOVERY state. Mar 22 12:29:42 debug [TOTEM ] TRANS [0] member 172.17.0.35: Mar 22 12:29:42 debug [TOTEM ] position [0] member 172.17.0.35: Mar 22 12:29:42 debug [TOTEM ] previous ring seq d98 rep 172.17.0.35 Mar 22 12:29:42 debug [TOTEM ] aru 5 high delivered 5 received flag 1 Mar 22 12:29:42 debug [TOTEM ] Did not need to originate any messages in recovery. Mar 22 12:29:42 debug [TOTEM ] got commit token Mar 22 12:29:42 debug [TOTEM ] Sending initial ORF token Mar 22 12:29:42 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Mar 22 12:29:42 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:42 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Mar 22 12:29:42 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:42 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Mar 22 12:29:42 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:42 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Mar 22 12:29:42 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:42 debug [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0 Mar 22 12:29:42 debug [TOTEM ] Resetting old ring state Mar 22 12:29:42 debug [TOTEM ] recovery to regular 1-0 Mar 22 12:29:42 debug [TOTEM ] waiting_trans_ack changed to 1 Mar 22 12:29:42 debug [TOTEM ] entering OPERATIONAL state. Mar 22 12:29:42 notice [TOTEM ] A new membership (172.17.0.35:3484) was formed. Members Mar 22 12:29:42 debug [SYNC ] Committing synchronization for corosync configuration map access Mar 22 12:29:42 debug [QB ] Not first sync -> no action Mar 22 12:29:42 debug [CPG ] comparing: sender r(0) ip(172.17.0.35) ; members(old:1 left:0) Mar 22 12:29:42 debug [CPG ] chosen downlist: sender r(0) ip(172.17.0.35) ; members(old:1 left:0) Mar 22 12:29:42 debug [CPG ] got joinlist message from node 1ec302 Mar 22 12:29:42 debug [SYNC ] Committing synchronization for corosync cluster closed process group service v1.01 Mar 22 12:29:42 debug [CPG ] joinlist_messages[0] group:dlm:ls:GFS\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:42 debug [CPG ] joinlist_messages[1] group:dlm:controld\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:42 debug [QUORUM] flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:42 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:42 debug [QUORUM] nodeinfo message[2016002]: votes: 1, expected: 3 flags: 0 Mar 22 12:29:42 debug [QUORUM] flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:42 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:42 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:42 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:42 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:42 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:42 debug [QUORUM] nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Mar 22 12:29:42 debug [SYNC ] Committing synchronization for corosync vote quorum service v1.0 Mar 22 12:29:42 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:42 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:42 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:42 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:42 notice [QUORUM] Members[1]: 2016002 Mar 22 12:29:42 debug [QUORUM] sending quorum notification to (nil), length = 52 Mar 22 12:29:42 notice [MAIN ] Completed service synchronization, ready to provide service. Mar 22 12:29:42 debug [TOTEM ] waiting_trans_ack changed to 0 Mar 22 12:29:43 debug [TOTEM ] entering GATHER state from 9(merge during operational state). Mar 22 12:29:46 debug [TOTEM ] entering GATHER state from 0(consensus timeout). Mar 22 12:29:46 debug [TOTEM ] Creating commit token because I am the rep. Mar 22 12:29:46 debug [TOTEM ] Saving state aru 5 high seq received 5 Mar 22 12:29:46 debug [TOTEM ] Storing new sequence id for ring da0 Mar 22 12:29:46 debug [TOTEM ] entering COMMIT state. Mar 22 12:29:46 debug [TOTEM ] got commit token Mar 22 12:29:46 debug [TOTEM ] entering RECOVERY state. Mar 22 12:29:46 debug [TOTEM ] TRANS [0] member 172.17.0.35: Mar 22 12:29:46 debug [TOTEM ] position [0] member 172.17.0.35: Mar 22 12:29:46 debug [TOTEM ] previous ring seq d9c rep 172.17.0.35 Mar 22 12:29:46 debug [TOTEM ] aru 5 high delivered 5 received flag 1 Mar 22 12:29:46 debug [TOTEM ] Did not need to originate any messages in recovery. Mar 22 12:29:46 debug [TOTEM ] got commit token Mar 22 12:29:46 debug [TOTEM ] Sending initial ORF token Mar 22 12:29:46 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 0, aru 0 Mar 22 12:29:46 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:46 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 1, aru 0 Mar 22 12:29:46 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:46 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 2, aru 0 Mar 22 12:29:46 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:46 debug [TOTEM ] token retrans flag is 0 my set retrans flag0 retrans queue empty 1 count 3, aru 0 Mar 22 12:29:46 debug [TOTEM ] install seq 0 aru 0 high seq received 0 Mar 22 12:29:46 debug [TOTEM ] retrans flag count 4 token aru 0 install seq 0 aru 0 0 Mar 22 12:29:46 debug [TOTEM ] Resetting old ring state Mar 22 12:29:46 debug [TOTEM ] recovery to regular 1-0 Mar 22 12:29:46 debug [TOTEM ] waiting_trans_ack changed to 1 Mar 22 12:29:46 debug [TOTEM ] entering OPERATIONAL state. Mar 22 12:29:46 notice [TOTEM ] A new membership (172.17.0.35:3488) was formed. Members Mar 22 12:29:46 debug [SYNC ] Committing synchronization for corosync configuration map access Mar 22 12:29:46 debug [QB ] Not first sync -> no action Mar 22 12:29:46 debug [CPG ] comparing: sender r(0) ip(172.17.0.35) ; members(old:1 left:0) Mar 22 12:29:46 debug [CPG ] chosen downlist: sender r(0) ip(172.17.0.35) ; members(old:1 left:0) Mar 22 12:29:46 debug [CPG ] got joinlist message from node 1ec302 Mar 22 12:29:46 debug [SYNC ] Committing synchronization for corosync cluster closed process group service v1.01 Mar 22 12:29:46 debug [CPG ] joinlist_messages[0] group:dlm:ls:GFS\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:46 debug [CPG ] joinlist_messages[1] group:dlm:controld\x00, ip:r(0) ip(172.17.0.35) , pid:4725 Mar 22 12:29:46 debug [QUORUM] flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:46 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:46 debug [QUORUM] nodeinfo message[2016002]: votes: 1, expected: 3 flags: 0 Mar 22 12:29:46 debug [QUORUM] flags: quorate: No Leaving: No WFA Status: No First: No Qdevice: No QdeviceAlive: No QdeviceCastVote: No QdeviceMasterWins: No Mar 22 12:29:46 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:46 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:46 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:46 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:46 debug [QUORUM] got nodeinfo message from cluster node 2016002 Mar 22 12:29:46 debug [QUORUM] nodeinfo message[0]: votes: 0, expected: 0 flags: 0 Mar 22 12:29:46 debug [SYNC ] Committing synchronization for corosync vote quorum service v1.0 Mar 22 12:29:46 debug [QUORUM] total_votes=1, expected_votes=3 Mar 22 12:29:46 debug [QUORUM] node 2016001 state=2, votes=1, expected=3 Mar 22 12:29:46 debug [QUORUM] node 2016002 state=1, votes=1, expected=3 Mar 22 12:29:46 debug [QUORUM] node 2016003 state=2, votes=1, expected=3 Mar 22 12:29:46 notice [QUORUM] Members[1]: 2016002 Mar 22 12:29:46 debug [QUORUM] sending quorum notification to (nil), length = 52 Mar 22 12:29:46 notice [MAIN ] Completed service synchronization, ready to provide service. Mar 22 12:29:46 debug [TOTEM ] waiting_trans_ack changed to 0 Mar 22 12:29:46 debug [TOTEM ] entering GATHER state from 9(merge during operational state).