[ClusterLabs] Antw: Rebooting a standby node triggers lots of transitions

Kadlecsik József kadlecsik.jozsef at wigner.mta.hu
Wed Sep 5 20:34:17 UTC 2018


On Wed, 5 Sep 2018, Ken Gaillot wrote:

> > > For testing purposes one of our nodes was put in standby node and 
> > > then  rebooted several times. When the standby node started up, it 
> > > joined the  cluster as a new member and it resulted in transitions 
> > > between the online  nodes. However, when the standby node was 
> > > rebooted in mid‑transitions, it  triggered another transitions 
> > > again. As a result, live migrations was  aborted and guests 
> > > stopped/started.
> > > 
> > > How can one make sure that join/leave operations of standby nodes
> > > do not 
> > > affect the location of the running resources?
> > > 
> > > It's pacemaker 1.1.16‑1 with corosync 2.4.2‑3+deb9u1 on debian
> > > stretch 
> > > nodes.
> 
> Node joins/leaves do and should trigger new transitions, but that should 
> not result in any actions if the node is in standby.
>
> The cluster will wait for any actions in progress (such as a live 
> migration) to complete before beginning a new transition, so there is 
> likely something else going on that is affecting the migration.
> 
> > Logs and more details, please!
> 
> Particularly the detail log on the DC should be helpful. It will have
> "pengine:" messages with "saving inputs" at each transition.

I attached the log file.

There are log lines like this

Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Transition aborted by 
w2-utilization-cpu doing modify cpu=1: Configuration change 

which I don't understand: in the configuration the cpu utilization is 
explicitly set to cpu=2 for w2.

Nothing changed, just the node atlas0 (in standby mode) was halted/started 
several times. Still, resources were migrated, like in this case:

Sep  5 12:22:31 atlas4 VirtualDomain(mail0)[61781]: INFO: mail0: Starting 
live migration to atlas3 (using: virsh --connect=qemu:///system --quiet 
migrate --live  mail0 qemu+tls://atlas3/system ).

And besides the successful migrations, sometimes the guest was 
stopped/started instead of migration:

Sep  5 12:25:22 atlas4 crmd[32776]:   notice: Result of stop operation for 
mail0 on atlas4: 0 (ok) 
Sep  5 12:25:22 atlas4 crmd[32776]:   notice: Initiating start operation 
mail0_start_0 on atlas3 

Best regards,
Jozsef
--
E-mail : kadlecsik.jozsef at wigner.mta.hu
PGP key: http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address: Wigner Research Centre for Physics, Hungarian Academy of Sciences
         H-1525 Budapest 114, POB. 49, Hungary
-------------- next part --------------
Sep  5 12:17:08 atlas4 attrd[32774]:   notice: Node atlas0 state is now lost 
Sep  5 12:17:08 atlas4 attrd[32774]:   notice: Removing all atlas0 attributes for peer loss
Sep  5 12:17:08 atlas4 attrd[32774]:   notice: Lost attribute writer atlas0
Sep  5 12:17:08 atlas4 attrd[32774]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:17:08 atlas4 stonith-ng[32772]:   notice: Node atlas0 state is now lost 
Sep  5 12:17:08 atlas4 stonith-ng[32772]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:17:08 atlas4 cib[32771]:   notice: Node atlas0 state is now lost 
Sep  5 12:17:08 atlas4 cib[32771]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:17:09 atlas4 crmd[32776]:  warning: Input I_ELECTION_DC received in state S_INTEGRATION from do_election_check
Sep  5 12:17:09 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.2:156) was formed. Members left: 1084762113
Sep  5 12:17:09 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762114 1084762115 1084762116 1084762117 1084762118 1084762\
Sep  5 12:17:09 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762119
Sep  5 12:17:09 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 12:17:09 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now lost 
Sep  5 12:17:09 atlas4 crmd[32776]:   notice: Node atlas0 state is now lost 
Sep  5 12:17:09 atlas4 crmd[32776]:  warning: No reason to expect node 1084762113 to be down
Sep  5 12:17:09 atlas4 crmd[32776]:   notice: Stonith/shutdown of atlas0 not matched
Sep  5 12:17:11 atlas4 pengine[32775]:   notice: Calculated transition 0, saving inputs in /var/lib/pacemaker/pengine/pe-input-0.bz2
Sep  5 12:17:11 atlas4 crmd[32776]:   notice: Transition 0 (Complete=7, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-0.bz2): Complete
Sep  5 12:17:11 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 12:22:25 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.1:160) was formed. Members joined: 1084762113
Sep  5 12:22:25 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762113 1084762114 1084762115 1084762116 1084762117 1084762\
Sep  5 12:22:25 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762118 1084762119
Sep  5 12:22:25 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 12:22:25 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now member 
Sep  5 12:22:25 atlas4 crmd[32776]:   notice: Node atlas0 state is now member 
Sep  5 12:22:26 atlas4 cib[32771]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:22:26 atlas4 cib[32771]:   notice: Node (null) state is now member 
Sep  5 12:22:26 atlas4 attrd[32774]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:22:26 atlas4 attrd[32774]:   notice: Node (null) state is now member 
Sep  5 12:22:26 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_INTEGRATION 
Sep  5 12:22:26 atlas4 stonith-ng[32772]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:22:26 atlas4 stonith-ng[32772]:   notice: Node (null) state is now member 
Sep  5 12:22:30 atlas4 pengine[32775]:   notice: Calculated transition 1, saving inputs in /var/lib/pacemaker/pengine/pe-input-1.bz2
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas0 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Transition aborted by w2-utilization-cpu doing modify cpu=1: Configuration change 
Sep  5 12:22:30 atlas4 crmd[32776]:   notice: Transition 1 (Complete=41, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-1.bz2): Complete
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas3 -> atlas5)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas2 -> atlas3)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas5 -> atlas2)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas5)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas3 -> atlas1)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas4 -> atlas3)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas6 -> atlas4)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas2 -> atlas5)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas6)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas3 -> atlas1)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas3 -> atlas1)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas2)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas6 -> atlas3)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas2 -> atlas5)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas6 -> atlas2)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas6 -> atlas3)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas6 -> atlas4)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas6 -> atlas5)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas6 -> atlas5)
Sep  5 12:22:31 atlas4 pengine[32775]:   notice: Calculated transition 2, saving inputs in /var/lib/pacemaker/pengine/pe-input-2.bz2
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas3 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eprints_migrate_to_0 on atlas2 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating migrate_to operation gitlab_migrate_to_0 on atlas5 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas3 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 locally on atlas4 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mailman_migrate_to_0 on atlas2 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas6 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas2 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas6 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas6 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas6 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas6 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas6 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas3 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas2 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas3 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 locally on atlas4 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas5 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas5 
Sep  5 12:22:31 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas5 
Sep  5 12:22:31 atlas4 VirtualDomain(mail0)[61781]: INFO: mail0: Starting live migration to atlas3 (using: virsh --connect=qemu:///system --quiet migrate --live  mail0 qemu+tls://atlas3/system ).
Sep  5 12:22:32 atlas4 crmd[32776]:   notice: Result of start operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 12:22:34 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas5 
Sep  5 12:22:34 atlas4 crmd[32776]:   notice: Initiating migrate_from operation idp_migrate_from_0 on atlas1 
Sep  5 12:22:34 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas3 
Sep  5 12:22:34 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas3 
Sep  5 12:22:35 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mailman_migrate_from_0 on atlas5 
Sep  5 12:22:36 atlas4 crmd[32776]:   notice: Initiating stop operation mailman_stop_0 on atlas2 
Sep  5 12:22:36 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eprints_migrate_from_0 on atlas3 
Sep  5 12:22:36 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas2 
Sep  5 12:22:36 atlas4 crmd[32776]:   notice: Initiating migrate_to operation ykval_migrate_to_0 on atlas5 
Sep  5 12:22:38 atlas4 crmd[32776]:   notice: Transition aborted by w2-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 12:22:41 atlas4 VirtualDomain(mail0)[61781]: INFO: mail0: live migration to atlas3 succeeded.
Sep  5 12:22:41 atlas4 crmd[32776]:   notice: Result of migrate_to operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:22:41 atlas4 crmd[32776]:   notice: Transition 2 (Complete=31, Pending=0, Fired=0, Skipped=3, Incomplete=51, Source=/var/lib/pacemaker/pengine/pe-input-2.bz2): Stopped
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Resource gitlab can no longer migrate to atlas2. Stopping on atlas5 too
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Resource mail0 can no longer migrate to atlas3. Stopping on atlas4 too
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Resource ykval can no longer migrate to atlas2. Stopping on atlas5 too
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas5 -> atlas3)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas3 -> atlas2)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    gitlab#011(Started atlas2 -> atlas5)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas1 -> atlas3)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    mail0#011(Started atlas3 -> atlas4)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas5 -> atlas2)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    ykval#011(Started atlas2 -> atlas5)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas3 -> atlas6)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas5 -> atlas2)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas2 -> atlas6)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas3 -> atlas6)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas4 -> atlas6)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas5 -> atlas6)
Sep  5 12:22:41 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas5 -> atlas6)
Sep  5 12:22:42 atlas4 pengine[32775]:   notice: Calculated transition 3, saving inputs in /var/lib/pacemaker/pengine/pe-input-3.bz2
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas5 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas2 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 locally on atlas4 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 on atlas3 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas5 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas2 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas3 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas5 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas2 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas3 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 locally on atlas4 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas5 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas5 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Result of stop operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas6 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas6 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas6 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas2 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas6 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas6 
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas6 
Sep  5 12:22:42 atlas4 VirtualDomain(mail0)[62023]: INFO: Domain mail0 already stopped.
Sep  5 12:22:42 atlas4 crmd[32776]:   notice: Result of stop operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:22:56 atlas4 crmd[32776]:   notice: Initiating start operation mail0_start_0 locally on atlas4 
Sep  5 12:22:57 atlas4 crmd[32776]:   notice: Result of start operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:22:57 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_10000 locally on atlas4 
Sep  5 12:22:59 atlas4 crmd[32776]:   notice: Transition aborted by transient_attributes.1084762113 'create': Transient attribute change 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Transition 3 (Complete=26, Pending=0, Fired=0, Skipped=2, Incomplete=28, Source=/var/lib/pacemaker/pengine/pe-input-3.bz2): Stopped
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Scheduling Node atlas0 for shutdown
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas5 -> atlas3)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas3 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas6 -> atlas5)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas6)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas4 -> atlas3)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas6 -> atlas4)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas6)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas3 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas3 -> atlas1)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas6 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas6 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas6 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas6 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas6 -> atlas3)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas6 -> atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Start   gitlab#011(atlas2)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Start   ykval#011(atlas5)
Sep  5 12:23:18 atlas4 pengine[32775]:   notice: Calculated transition 4, saving inputs in /var/lib/pacemaker/pengine/pe-input-4.bz2
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas5 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation hexaa_migrate_to_0 on atlas1 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_10000 on atlas1 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 locally on atlas4 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_10000 on atlas5 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas6 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas6 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas6 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas6 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas6 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas6 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eprints_migrate_to_0 on atlas3 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation wiki_migrate_to_0 on atlas3 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation gitlab_start_0 on atlas2 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 12:23:18 atlas4 attrd[32774]:   notice: Node atlas0 state is now lost 
Sep  5 12:23:18 atlas4 attrd[32774]:   notice: Removing all atlas0 attributes for peer loss
Sep  5 12:23:18 atlas4 attrd[32774]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:23:18 atlas4 stonith-ng[32772]:   notice: Node atlas0 state is now lost 
Sep  5 12:23:18 atlas4 stonith-ng[32772]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas2 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas2 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas2 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas2 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas3 
Sep  5 12:23:18 atlas4 cib[32771]:   notice: Node atlas0 state is now lost 
Sep  5 12:23:18 atlas4 cib[32771]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas2 
Sep  5 12:23:18 atlas4 VirtualDomain(mail0)[62780]: INFO: mail0: Starting live migration to atlas3 (using: virsh --connect=qemu:///system --quiet migrate --live  mail0 qemu+tls://atlas3/system ).
Sep  5 12:23:18 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.2:164) was formed. Members left: 1084762113
Sep  5 12:23:18 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762114 1084762115 1084762116 1084762117 1084762118 1084762\
Sep  5 12:23:18 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762119
Sep  5 12:23:18 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 12:23:18 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now lost 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: Node atlas0 state is now lost 
Sep  5 12:23:18 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 12:23:19 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_10000 on atlas2 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating migrate_from operation hexaa_migrate_from_0 on atlas6 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas3 
Sep  5 12:23:20 atlas4 VirtualDomain(mail0)[62780]: INFO: mail0: live migration to atlas3 succeeded.
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Result of migrate_to operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail0_migrate_from_0 on atlas3 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas1 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas5 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating migrate_to operation kerberos_migrate_to_0 on atlas6 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating start operation ykval_start_0 on atlas5 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 locally on atlas4 
Sep  5 12:23:20 atlas4 VirtualDomain(mail0)[62870]: INFO: Domain mail0 already stopped.
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Result of stop operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:23:20 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail2_migrate_to_0 on atlas6 
Sep  5 12:23:21 atlas4 crmd[32776]:   notice: Initiating migrate_from operation wiki_migrate_from_0 on atlas2 
Sep  5 12:23:21 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas5 
Sep  5 12:23:22 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas3 
Sep  5 12:23:22 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_10000 on atlas2 
Sep  5 12:23:22 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eprints_migrate_from_0 on atlas2 
Sep  5 12:23:23 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas3 
Sep  5 12:23:23 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_10000 on atlas2 
Sep  5 12:23:23 atlas4 crmd[32776]:   notice: Initiating migrate_from operation kerberos_migrate_from_0 on atlas5 
Sep  5 12:23:23 atlas4 crmd[32776]:   notice: Initiating stop operation kerberos_stop_0 on atlas6 
Sep  5 12:23:23 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_10000 on atlas5 
Sep  5 12:23:26 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail2_migrate_from_0 locally on atlas4 
Sep  5 12:23:26 atlas4 VirtualDomain(mail2)[63124]: INFO: mail2: live migration from atlas6 succeeded.
Sep  5 12:23:26 atlas4 crmd[32776]:   notice: Result of migrate_from operation for mail2 on atlas4: 0 (ok) 
Sep  5 12:23:26 atlas4 crmd[32776]:   notice: Initiating stop operation mail2_stop_0 on atlas6 
Sep  5 12:23:26 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_10000 locally on atlas4 
Sep  5 12:23:26 atlas4 crmd[32776]:   notice: Initiating migrate_to operation seafile_migrate_to_0 on atlas1 
Sep  5 12:23:26 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_10000 on atlas6 
Sep  5 12:23:35 atlas4 crmd[32776]:   notice: Initiating migrate_from operation seafile_migrate_from_0 on atlas6 
Sep  5 12:23:35 atlas4 crmd[32776]:   notice: Initiating stop operation seafile_stop_0 on atlas1 
Sep  5 12:23:35 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_10000 on atlas6 
Sep  5 12:23:35 atlas4 crmd[32776]:   notice: Initiating migrate_to operation webmail_migrate_to_0 on atlas3 
Sep  5 12:23:42 atlas4 crmd[32776]:   notice: Initiating migrate_from operation webmail_migrate_from_0 on atlas1 
Sep  5 12:23:43 atlas4 crmd[32776]:   notice: Initiating stop operation webmail_stop_0 on atlas3 
Sep  5 12:23:43 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_10000 on atlas1 
Sep  5 12:23:43 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_10000 on atlas3 
Sep  5 12:23:43 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_10000 on atlas3 
Sep  5 12:23:43 atlas4 crmd[32776]:   notice: Transition 4 (Complete=72, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-4.bz2): Complete
Sep  5 12:23:43 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 12:25:08 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.1:168) was formed. Members joined: 1084762113
Sep  5 12:25:08 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762113 1084762114 1084762115 1084762116 1084762117 1084762\
Sep  5 12:25:08 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762118 1084762119
Sep  5 12:25:08 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 12:25:08 atlas4 crmd[32776]:   notice: Node atlas0 state is now member 
Sep  5 12:25:08 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now member 
Sep  5 12:25:08 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 12:25:08 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_INTEGRATION 
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas2 -> atlas5)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas5 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas6 -> atlas1)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas1 -> atlas3)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas3 -> atlas4)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas4 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas5 -> atlas2)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas6 -> atlas1)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas2 -> atlas3)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas1 -> atlas3)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas2 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas2 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas2 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas2 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas3 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas2 -> atlas6)
Sep  5 12:25:10 atlas4 pengine[32775]:   notice: Calculated transition 5, saving inputs in /var/lib/pacemaker/pengine/pe-input-5.bz2
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating migrate_to operation gitlab_migrate_to_0 on atlas2 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating migrate_to operation hexaa_migrate_to_0 on atlas6 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas1 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 on atlas3 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas2 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas2 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas2 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas2 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas3 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas2 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas6 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas6 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas6 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas6 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas6 
Sep  5 12:25:10 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas6 
Sep  5 12:25:10 atlas4 attrd[32774]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:25:10 atlas4 attrd[32774]:   notice: Node (null) state is now member 
Sep  5 12:25:10 atlas4 cib[32771]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:25:10 atlas4 cib[32771]:   notice: Node (null) state is now member 
Sep  5 12:25:11 atlas4 stonith-ng[32772]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:25:11 atlas4 stonith-ng[32772]:   notice: Node (null) state is now member 
Sep  5 12:25:11 atlas4 crmd[32776]:   notice: Transition aborted: Peer Halt 
Sep  5 12:25:12 atlas4 crmd[32776]:   notice: Transition aborted: Node join 
Sep  5 12:25:13 atlas4 crmd[32776]:   notice: Transition 5 (Complete=17, Pending=0, Fired=0, Skipped=4, Incomplete=53, Source=/var/lib/pacemaker/pengine/pe-input-5.bz2): Stopped
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas2 -> atlas5)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas5 -> atlas6)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas6 -> atlas1)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas1 -> atlas3)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas3 -> atlas4)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas4 -> atlas6)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas5 -> atlas2)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas6 -> atlas1)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas2 -> atlas3)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas1 -> atlas3)
Sep  5 12:25:14 atlas4 pengine[32775]:   notice: Calculated transition 6, saving inputs in /var/lib/pacemaker/pengine/pe-input-6.bz2
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas0 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating migrate_from operation idp_migrate_from_0 on atlas3 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating migrate_from operation hexaa_migrate_from_0 on atlas1 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail0_migrate_from_0 locally on atlas4 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating migrate_from operation gitlab_migrate_from_0 on atlas5 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas1 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas6 
Sep  5 12:25:14 atlas4 VirtualDomain(mail0)[65082]: INFO: mail0: live migration from atlas3 succeeded.
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Result of migrate_from operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:25:14 atlas4 crmd[32776]:   notice: Transition aborted by w2-utilization-cpu doing modify cpu=1: Configuration change 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Transition 6 (Complete=41, Pending=0, Fired=0, Skipped=2, Incomplete=47, Source=/var/lib/pacemaker/pengine/pe-input-6.bz2): Stopped
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas3 -> atlas5)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas2 -> atlas3)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    gitlab#011(Started atlas5 -> atlas2)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas6)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas3 -> atlas1)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    mail0#011(Started atlas4 -> atlas3)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas2 -> atlas1)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas2)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas6 -> atlas3)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas2 -> atlas5)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas6 -> atlas2)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas6 -> atlas3)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas6 -> atlas4)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas6 -> atlas5)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas6 -> atlas5)
Sep  5 12:25:15 atlas4 pengine[32775]:   notice: Calculated transition 7, saving inputs in /var/lib/pacemaker/pengine/pe-input-7.bz2
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas3 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas5 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas2 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 locally on atlas4 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 on atlas3 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas6 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas2 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas6 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas6 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas6 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas6 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas6 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating migrate_to operation hexaa_migrate_to_0 on atlas1 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas5 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas3 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas2 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas3 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 locally on atlas4 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas5 
Sep  5 12:25:15 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas5 
Sep  5 12:25:15 atlas4 VirtualDomain(mail0)[65202]: INFO: Issuing graceful shutdown request for domain mail0.
Sep  5 12:25:16 atlas4 crmd[32776]:   notice: Result of start operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation hexaa_migrate_from_0 on atlas6 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas5 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas1 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas3 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_10000 on atlas6 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas3 
Sep  5 12:25:17 atlas4 crmd[32776]:   notice: Initiating migrate_to operation wiki_migrate_to_0 on atlas2 
Sep  5 12:25:19 atlas4 crmd[32776]:   notice: Initiating migrate_from operation idp_migrate_from_0 on atlas1 
Sep  5 12:25:20 atlas4 crmd[32776]:   notice: Initiating migrate_from operation wiki_migrate_from_0 on atlas1 
Sep  5 12:25:20 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas3 
Sep  5 12:25:20 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas2 
Sep  5 12:25:20 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eprints_migrate_to_0 on atlas2 
Sep  5 12:25:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_10000 on atlas1 
Sep  5 12:25:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_10000 on atlas1 
Sep  5 12:25:22 atlas4 crmd[32776]:   notice: Result of stop operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:25:22 atlas4 crmd[32776]:   notice: Initiating start operation mail0_start_0 on atlas3 
Sep  5 12:25:23 atlas4 crmd[32776]:   notice: Transition aborted by w2-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Transition 7 (Complete=43, Pending=0, Fired=0, Skipped=2, Incomplete=17, Source=/var/lib/pacemaker/pengine/pe-input-7.bz2): Stopped
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Resource eprints can no longer migrate to atlas3. Stopping on atlas2 too
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas5 -> atlas3)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Move    eprints#011(Started atlas3 -> atlas2)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas1 -> atlas2)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas2)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas2 -> atlas3)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas5 -> atlas2)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Start   gitlab#011(atlas4)
Sep  5 12:25:32 atlas4 pengine[32775]:   notice: Calculated transition 8, saving inputs in /var/lib/pacemaker/pengine/pe-input-8.bz2
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas2 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas3 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_10000 on atlas3 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas2 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas5 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating start operation gitlab_start_0 locally on atlas4 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas2 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas3 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating migrate_to operation wiki_migrate_to_0 on atlas1 
Sep  5 12:25:32 atlas4 crmd[32776]:   notice: Initiating migrate_to operation ykval_migrate_to_0 on atlas5 
Sep  5 12:25:34 atlas4 crmd[32776]:   notice: Result of start operation for gitlab on atlas4: 0 (ok) 
Sep  5 12:25:34 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_10000 locally on atlas4 
Sep  5 12:25:35 atlas4 crmd[32776]:   notice: Initiating migrate_from operation ykval_migrate_from_0 on atlas2 
Sep  5 12:25:35 atlas4 crmd[32776]:   notice: Initiating migrate_from operation wiki_migrate_from_0 on atlas2 
Sep  5 12:25:35 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas5 
Sep  5 12:25:35 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas2 
Sep  5 12:25:35 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas1 
Sep  5 12:25:35 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_10000 on atlas2 
Sep  5 12:25:41 atlas4 crmd[32776]:   notice: Initiating start operation eprints_start_0 on atlas2 
Sep  5 12:25:41 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas5 
Sep  5 12:25:42 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_10000 on atlas2 
Sep  5 12:25:43 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas3 
Sep  5 12:25:43 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas5 
Sep  5 12:25:43 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_10000 on atlas3 
Sep  5 12:25:44 atlas4 crmd[32776]:   notice: Transition 8 (Complete=34, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-8.bz2): Complete
Sep  5 12:25:44 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_POLICY_ENGINE 
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Scheduling Node atlas0 for shutdown
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas4 -> atlas5)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas5 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas6 -> atlas1)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas1 -> atlas3)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas3 -> atlas4)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas4 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas5 -> atlas2)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas6 -> atlas1)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas2 -> atlas3)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas1 -> atlas3)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas2 -> atlas5)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas3 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas5 -> atlas2)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas3 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas3 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas4 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas5 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas2 -> atlas6)
Sep  5 12:38:42 atlas4 pengine[32775]:   notice: Calculated transition 9, saving inputs in /var/lib/pacemaker/pengine/pe-input-9.bz2
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating migrate_to operation gitlab_migrate_to_0 locally on atlas4 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating migrate_to operation hexaa_migrate_to_0 on atlas6 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas1 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 on atlas3 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mailman_migrate_to_0 on atlas5 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas3 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas5 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas3 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas3 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 locally on atlas4 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas5 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas2 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Result of stop operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas6 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas6 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas6 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas6 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas6 
Sep  5 12:38:42 atlas4 attrd[32774]:   notice: Node atlas0 state is now lost 
Sep  5 12:38:42 atlas4 attrd[32774]:   notice: Removing all atlas0 attributes for peer loss
Sep  5 12:38:42 atlas4 attrd[32774]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas2 
Sep  5 12:38:42 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas6 
Sep  5 12:38:42 atlas4 VirtualDomain(gitlab)[17606]: INFO: gitlab: Starting live migration to atlas5 (using: virsh --connect=qemu:///system --quiet migrate --live  gitlab qemu+tls://atlas5/system ).
Sep  5 12:38:42 atlas4 stonith-ng[32772]:   notice: Node atlas0 state is now lost 
Sep  5 12:38:42 atlas4 stonith-ng[32772]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:38:42 atlas4 cib[32771]:   notice: Node atlas0 state is now lost 
Sep  5 12:38:42 atlas4 cib[32771]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 12:38:43 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.2:172) was formed. Members left: 1084762113
Sep  5 12:38:43 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762114 1084762115 1084762116 1084762117 1084762118 1084762\
Sep  5 12:38:43 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762119
Sep  5 12:38:43 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 12:38:43 atlas4 crmd[32776]:   notice: Node atlas0 state is now lost 
Sep  5 12:38:43 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now lost 
Sep  5 12:38:43 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 12:38:44 atlas4 crmd[32776]:   notice: Initiating migrate_from operation idp_migrate_from_0 on atlas3 
Sep  5 12:38:45 atlas4 crmd[32776]:   notice: Initiating migrate_from operation hexaa_migrate_from_0 on atlas1 
Sep  5 12:38:45 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas1 
Sep  5 12:38:45 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas6 
Sep  5 12:38:45 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail0_migrate_from_0 locally on atlas4 
Sep  5 12:38:46 atlas4 VirtualDomain(mail0)[17838]: INFO: mail0: live migration from atlas3 succeeded.
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Result of migrate_from operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 on atlas3 
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Initiating migrate_to operation wiki_migrate_to_0 on atlas2 
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Initiating migrate_to operation webmail_migrate_to_0 on atlas1 
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_10000 on atlas3 
Sep  5 12:38:46 atlas4 VirtualDomain(gitlab)[17606]: INFO: gitlab: live migration to atlas5 succeeded.
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Result of migrate_to operation for gitlab on atlas4: 0 (ok) 
Sep  5 12:38:46 atlas4 crmd[32776]:   notice: Initiating migrate_from operation gitlab_migrate_from_0 on atlas5 
Sep  5 12:38:47 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 locally on atlas4 
Sep  5 12:38:47 atlas4 VirtualDomain(gitlab)[17911]: INFO: Domain gitlab already stopped.
Sep  5 12:38:47 atlas4 crmd[32776]:   notice: Result of stop operation for gitlab on atlas4: 0 (ok) 
Sep  5 12:38:47 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mailman_migrate_from_0 on atlas2 
Sep  5 12:38:48 atlas4 crmd[32776]:   notice: Initiating stop operation mailman_stop_0 on atlas5 
Sep  5 12:38:48 atlas4 crmd[32776]:   notice: Initiating migrate_from operation wiki_migrate_from_0 on atlas3 
Sep  5 12:38:48 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas2 
Sep  5 12:38:49 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_10000 on atlas3 
Sep  5 12:38:53 atlas4 crmd[32776]:   notice: Initiating migrate_from operation webmail_migrate_from_0 on atlas3 
Sep  5 12:38:53 atlas4 crmd[32776]:   notice: Initiating stop operation webmail_stop_0 on atlas1 
Sep  5 12:38:54 atlas4 crmd[32776]:   notice: Initiating migrate_to operation seafile_migrate_to_0 on atlas6 
Sep  5 12:38:54 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_10000 on atlas3 
Sep  5 12:38:54 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_10000 on atlas1 
Sep  5 12:39:02 atlas4 crmd[32776]:   notice: Initiating migrate_from operation seafile_migrate_from_0 on atlas1 
Sep  5 12:39:02 atlas4 crmd[32776]:   notice: Initiating stop operation seafile_stop_0 on atlas6 
Sep  5 12:39:02 atlas4 crmd[32776]:   notice: Initiating migrate_to operation kerberos_migrate_to_0 on atlas5 
Sep  5 12:39:02 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail2_migrate_to_0 locally on atlas4 
Sep  5 12:39:02 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_10000 on atlas1 
Sep  5 12:39:02 atlas4 VirtualDomain(mail2)[18209]: INFO: mail2: Starting live migration to atlas6 (using: virsh --connect=qemu:///system --quiet migrate --live  mail2 qemu+tls://atlas6/system ).
Sep  5 12:39:05 atlas4 crmd[32776]:   notice: Initiating migrate_from operation kerberos_migrate_from_0 on atlas6 
Sep  5 12:39:05 atlas4 crmd[32776]:   notice: Initiating stop operation kerberos_stop_0 on atlas5 
Sep  5 12:39:05 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_10000 on atlas6 
Sep  5 12:39:05 atlas4 crmd[32776]:   notice: Initiating migrate_to operation ykval_migrate_to_0 on atlas2 
Sep  5 12:39:05 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_10000 on atlas5 
Sep  5 12:39:07 atlas4 crmd[32776]:   notice: Initiating migrate_from operation ykval_migrate_from_0 on atlas5 
Sep  5 12:39:07 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas2 
Sep  5 12:39:07 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas5 
Sep  5 12:39:07 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_10000 on atlas2 
Sep  5 12:39:08 atlas4 VirtualDomain(mail2)[18209]: INFO: mail2: live migration to atlas6 succeeded.
Sep  5 12:39:08 atlas4 crmd[32776]:   notice: Result of migrate_to operation for mail2 on atlas4: 0 (ok) 
Sep  5 12:39:08 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail2_migrate_from_0 on atlas6 
Sep  5 12:39:08 atlas4 crmd[32776]:   notice: Initiating stop operation mail2_stop_0 locally on atlas4 
Sep  5 12:39:08 atlas4 VirtualDomain(mail2)[18338]: INFO: Domain mail2 already stopped.
Sep  5 12:39:08 atlas4 crmd[32776]:   notice: Result of stop operation for mail2 on atlas4: 0 (ok) 
Sep  5 12:39:08 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_10000 on atlas6 
Sep  5 12:39:08 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_10000 locally on atlas4 
Sep  5 12:39:09 atlas4 crmd[32776]:   notice: Transition 9 (Complete=78, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-9.bz2): Complete
Sep  5 12:39:09 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 12:54:09 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_POLICY_ENGINE 
Sep  5 12:54:09 atlas4 pengine[32775]:   notice: Calculated transition 10, saving inputs in /var/lib/pacemaker/pengine/pe-input-10.bz2
Sep  5 12:54:09 atlas4 crmd[32776]:   notice: Transition 10 (Complete=7, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-10.bz2): Complete
Sep  5 12:54:09 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 12:56:07 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.1:176) was formed. Members joined: 1084762113
Sep  5 12:56:07 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762113 1084762114 1084762115 1084762116 1084762117 1084762\
Sep  5 12:56:07 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762118 1084762119
Sep  5 12:56:07 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 12:56:07 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now member 
Sep  5 12:56:07 atlas4 crmd[32776]:   notice: Node atlas0 state is now member 
Sep  5 12:56:09 atlas4 stonith-ng[32772]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:56:09 atlas4 stonith-ng[32772]:   notice: Node (null) state is now member 
Sep  5 12:56:09 atlas4 attrd[32774]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:56:09 atlas4 attrd[32774]:   notice: Node (null) state is now member 
Sep  5 12:56:09 atlas4 cib[32771]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 12:56:09 atlas4 cib[32771]:   notice: Node (null) state is now member 
Sep  5 12:56:10 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_INTEGRATION 
Sep  5 12:56:12 atlas4 pengine[32775]:   notice: Calculated transition 11, saving inputs in /var/lib/pacemaker/pengine/pe-input-11.bz2
Sep  5 12:56:12 atlas4 pengine[32775]:   notice: Calculated transition 12, saving inputs in /var/lib/pacemaker/pengine/pe-input-12.bz2
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas0 
Sep  5 12:56:12 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas0 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Transition aborted by w2-utilization-cpu doing modify cpu=1: Configuration change 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Transition 12 (Complete=41, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-12.bz2): Complete
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas3 -> atlas5)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas2 -> atlas3)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas5 -> atlas2)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas5)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas3 -> atlas1)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas4 -> atlas3)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas6 -> atlas4)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas2 -> atlas5)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas6)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas3 -> atlas1)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas3 -> atlas1)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas2)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas6 -> atlas3)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas2 -> atlas5)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas6 -> atlas2)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas6 -> atlas3)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas6 -> atlas4)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas6 -> atlas5)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas6 -> atlas5)
Sep  5 12:56:13 atlas4 pengine[32775]:   notice: Calculated transition 13, saving inputs in /var/lib/pacemaker/pengine/pe-input-13.bz2
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas3 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eprints_migrate_to_0 on atlas2 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating migrate_to operation gitlab_migrate_to_0 on atlas5 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas3 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 locally on atlas4 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mailman_migrate_to_0 on atlas2 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas6 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas2 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas6 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas6 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas6 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas6 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas6 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas3 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas2 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas3 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 locally on atlas4 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas5 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas5 
Sep  5 12:56:13 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas5 
Sep  5 12:56:13 atlas4 VirtualDomain(mail0)[36056]: INFO: mail0: Starting live migration to atlas3 (using: virsh --connect=qemu:///system --quiet migrate --live  mail0 qemu+tls://atlas3/system ).
Sep  5 12:56:14 atlas4 crmd[32776]:   notice: Result of start operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 12:56:16 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eprints_migrate_from_0 on atlas3 
Sep  5 12:56:16 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas2 
Sep  5 12:56:16 atlas4 crmd[32776]:   notice: Initiating migrate_from operation idp_migrate_from_0 on atlas1 
Sep  5 12:56:16 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas5 
Sep  5 12:56:16 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas3 
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas3 
Sep  5 12:56:17 atlas4 VirtualDomain(mail0)[36056]: INFO: mail0: live migration to atlas3 succeeded.
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Result of migrate_to operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail0_migrate_from_0 on atlas3 
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 locally on atlas4 
Sep  5 12:56:17 atlas4 VirtualDomain(mail0)[36214]: INFO: Domain mail0 already stopped.
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Result of stop operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail2_migrate_to_0 on atlas6 
Sep  5 12:56:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation gitlab_migrate_from_0 on atlas2 
Sep  5 12:56:18 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas5 
Sep  5 12:56:18 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mailman_migrate_from_0 on atlas5 
Sep  5 12:56:19 atlas4 crmd[32776]:   notice: Initiating stop operation mailman_stop_0 on atlas2 
Sep  5 12:56:19 atlas4 crmd[32776]:   notice: Initiating migrate_to operation ykval_migrate_to_0 on atlas5 
Sep  5 12:56:19 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_10000 on atlas2 
Sep  5 12:56:20 atlas4 crmd[32776]:   notice: Transition aborted by w2-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Transition 13 (Complete=39, Pending=0, Fired=0, Skipped=2, Incomplete=43, Source=/var/lib/pacemaker/pengine/pe-input-13.bz2): Stopped
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Resource mail2 can no longer migrate to atlas4. Stopping on atlas6 too
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Resource ykval can no longer migrate to atlas2. Stopping on atlas5 too
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas5 -> atlas3)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas3 -> atlas2)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas2 -> atlas5)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas1 -> atlas3)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas3 -> atlas4)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    mail2#011(Started atlas4 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas5 -> atlas2)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    ykval#011(Started atlas2 -> atlas5)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas3 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas5 -> atlas2)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas2 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas3 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas4 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas5 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas5 -> atlas6)
Sep  5 12:56:22 atlas4 pengine[32775]:   notice: Calculated transition 14, saving inputs in /var/lib/pacemaker/pengine/pe-input-14.bz2
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas5 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating migrate_to operation gitlab_migrate_to_0 on atlas2 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation mail2_stop_0 on atlas6 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation mail2_stop_0 locally on atlas4 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas5 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas2 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas3 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas5 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas2 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas3 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 locally on atlas4 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas5 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas5 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Result of stop operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas6 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas6 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas6 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas6 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas2 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas6 
Sep  5 12:56:22 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas6 
Sep  5 12:56:22 atlas4 VirtualDomain(mail2)[36423]: INFO: Issuing graceful shutdown request for domain mail2.
Sep  5 12:56:24 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas3 
Sep  5 12:56:25 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas5 
Sep  5 12:56:26 atlas4 crmd[32776]:   notice: Initiating migrate_from operation gitlab_migrate_from_0 on atlas5 
Sep  5 12:56:26 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas2 
Sep  5 12:56:29 atlas4 crmd[32776]:   notice: Result of stop operation for mail2 on atlas4: 0 (ok) 
Sep  5 12:56:29 atlas4 crmd[32776]:   notice: Initiating start operation mail2_start_0 on atlas6 
Sep  5 12:56:29 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 on atlas3 
Sep  5 12:56:30 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_10000 on atlas6 
Sep  5 12:56:32 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mail0_migrate_from_0 locally on atlas4 
Sep  5 12:56:32 atlas4 VirtualDomain(mail0)[36737]: INFO: mail0: live migration from atlas3 succeeded.
Sep  5 12:56:32 atlas4 crmd[32776]:   notice: Result of migrate_from operation for mail0 on atlas4: 0 (ok) 
Sep  5 12:56:32 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 on atlas3 
Sep  5 12:56:32 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_10000 locally on atlas4 
Sep  5 12:56:58 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eprints_migrate_to_0 on atlas3 
Sep  5 12:56:58 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mailman_migrate_to_0 on atlas5 
Sep  5 12:57:00 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eprints_migrate_from_0 on atlas2 
Sep  5 12:57:01 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas3 
Sep  5 12:57:01 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_10000 on atlas2 
Sep  5 12:57:01 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas1 
Sep  5 12:57:01 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_10000 on atlas3 
Sep  5 12:57:03 atlas4 crmd[32776]:   notice: Initiating migrate_from operation idp_migrate_from_0 on atlas3 
Sep  5 12:57:03 atlas4 crmd[32776]:   notice: Initiating migrate_from operation mailman_migrate_from_0 on atlas2 
Sep  5 12:57:03 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas1 
Sep  5 12:57:03 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_10000 on atlas3 
Sep  5 12:57:04 atlas4 crmd[32776]:   notice: Initiating stop operation mailman_stop_0 on atlas5 
Sep  5 12:57:04 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_10000 on atlas2 
Sep  5 12:57:04 atlas4 crmd[32776]:   notice: Initiating start operation ykval_start_0 on atlas5 
Sep  5 12:57:04 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_10000 on atlas5 
Sep  5 12:57:05 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas5 
Sep  5 12:57:05 atlas4 crmd[32776]:   notice: Transition 14 (Complete=60, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-14.bz2): Complete
Sep  5 12:57:05 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 13:01:39 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_POLICY_ENGINE 
Sep  5 13:01:39 atlas4 pengine[32775]:   notice: Scheduling Node atlas0 for shutdown
Sep  5 13:01:39 atlas4 pengine[32775]:   notice: Calculated transition 15, saving inputs in /var/lib/pacemaker/pengine/pe-input-15.bz2
Sep  5 13:01:39 atlas4 crmd[32776]:   notice: Transition 15 (Complete=8, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-15.bz2): Complete
Sep  5 13:01:39 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 13:01:39 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 13:01:39 atlas4 attrd[32774]:   notice: Node atlas0 state is now lost 
Sep  5 13:01:39 atlas4 attrd[32774]:   notice: Removing all atlas0 attributes for peer loss
Sep  5 13:01:39 atlas4 attrd[32774]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 13:01:39 atlas4 stonith-ng[32772]:   notice: Node atlas0 state is now lost 
Sep  5 13:01:39 atlas4 stonith-ng[32772]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 13:01:39 atlas4 cib[32771]:   notice: Node atlas0 state is now lost 
Sep  5 13:01:39 atlas4 cib[32771]:   notice: Purged 1 peers with id=1084762113 and/or uname=atlas0 from the membership cache
Sep  5 13:01:40 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.2:180) was formed. Members left: 1084762113
Sep  5 13:01:40 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762114 1084762115 1084762116 1084762117 1084762118 1084762\
Sep  5 13:01:40 atlas4 corosync[29705]:   [QUORUM] Members[6]: 1084762119
Sep  5 13:01:40 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 13:01:40 atlas4 crmd[32776]:   notice: Node atlas0 state is now lost 
Sep  5 13:01:40 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now lost 
Sep  5 13:01:40 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 13:08:16 atlas4 corosync[29705]:   [TOTEM ] A new membership (192.168.40.1:188) was formed. Members joined: 1084762113
Sep  5 13:08:16 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762113 1084762114 1084762115 1084762116 1084762117 1084762\
Sep  5 13:08:16 atlas4 corosync[29705]:   [QUORUM] Members[7]: 1084762118 1084762119
Sep  5 13:08:16 atlas4 corosync[29705]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep  5 13:08:16 atlas4 crmd[32776]:   notice: Node atlas0 state is now member 
Sep  5 13:08:16 atlas4 pacemakerd[32769]:   notice: Node atlas0 state is now member 
Sep  5 13:08:16 atlas4 crmd[32776]:   notice: do_shutdown of peer atlas0 is complete 
Sep  5 13:08:16 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_INTEGRATION 
Sep  5 13:08:17 atlas4 pengine[32775]:   notice: Calculated transition 16, saving inputs in /var/lib/pacemaker/pengine/pe-input-16.bz2
Sep  5 13:08:17 atlas4 crmd[32776]:   notice: Transition 16 (Complete=7, Pending=0, Fired=0, Skipped=0, Incomplete=0, Source=/var/lib/pacemaker/pengine/pe-input-16.bz2): Complete
Sep  5 13:08:17 atlas4 crmd[32776]:   notice: State transition S_TRANSITION_ENGINE -> S_IDLE 
Sep  5 13:08:17 atlas4 stonith-ng[32772]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 13:08:17 atlas4 stonith-ng[32772]:   notice: Node (null) state is now member 
Sep  5 13:08:17 atlas4 cib[32771]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 13:08:17 atlas4 cib[32771]:   notice: Node (null) state is now member 
Sep  5 13:08:17 atlas4 attrd[32774]:   notice: Could not obtain a node name for corosync nodeid 1084762113
Sep  5 13:08:17 atlas4 attrd[32774]:   notice: Node (null) state is now member 
Sep  5 13:08:18 atlas4 crmd[32776]:   notice: State transition S_IDLE -> S_INTEGRATION 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Syncing the Cluster Information Base from atlas0 to rest of cluster 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Requested version   <generation_tuple crm_feature_set="3.0.11" validate-with="pacemaker-2.6" epoch="329" num_updates="1" admin_epoch="0" cib-last-written="Wed Sep  5 13:05:25 2018" update-origin="atlas0" update-client="crm_resource" update-user="root" have-quorum="0"/>
Sep  5 13:08:20 atlas4 attrd[32774]:   notice: Updating all attributes after cib_refresh_notify event
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Migrate w1#011(Started atlas4 -> atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Migrate git#011(Started atlas4 -> atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas4 -> atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   netfilter#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   people#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   indico#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   wignerdc#011(atlas4)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   lx1#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   lx0#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   w0#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   w2#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   blue1#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   caladan#011(atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   eduid#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   eprints#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   gitlab#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   kerberos#011(atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   hexaa#011(atlas4)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   idp#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   mail2#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   mailman#011(atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   seafile#011(atlas4)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   wiki#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   webmail#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   ykval#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas0#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas6#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas5#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas4#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas3#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas2#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas1#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Calculated transition 17, saving inputs in /var/lib/pacemaker/pengine/pe-input-17.bz2
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Migrate w1#011(Started atlas4 -> atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Migrate git#011(Started atlas4 -> atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas4 -> atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   netfilter#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   people#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   indico#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   wignerdc#011(atlas4)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   lx1#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   lx0#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   w0#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   w2#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   blue1#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   caladan#011(atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   eduid#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   eprints#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   gitlab#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   kerberos#011(atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   hexaa#011(atlas4)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   idp#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   mail2#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   mailman#011(atlas3)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   seafile#011(atlas4)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   wiki#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   webmail#011(atlas2)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   ykval#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas0#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas6#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas5#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas4#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas3#011(atlas1)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas2#011(atlas5)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Start   stonith-atlas1#011(atlas6)
Sep  5 13:08:20 atlas4 pengine[32775]:   notice: Calculated transition 18, saving inputs in /var/lib/pacemaker/pengine/pe-input-18.bz2
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx2_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w1_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail0_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation template00_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation idp_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas0_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas6_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas5_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas4_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas3_monitor_0 on atlas0 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas6 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas5 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas2 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas1 
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas3 
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 210 (stonith-atlas0_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:   notice: Transition aborted by operation stonith-atlas0_monitor_0 'create' on atlas6: Event failed 
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 210 (stonith-atlas0_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 212 (stonith-atlas5_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 212 (stonith-atlas5_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 213 (stonith-atlas4_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 213 (stonith-atlas4_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 214 (stonith-atlas3_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 214 (stonith-atlas3_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 215 (stonith-atlas2_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 215 (stonith-atlas2_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 216 (stonith-atlas1_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:20 atlas4 crmd[32776]:  warning: Action 216 (stonith-atlas1_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 109 (stonith-atlas6_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 109 (stonith-atlas6_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 53 (people_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 53 (people_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 58 (w0_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 58 (w0_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 88 (indico_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 88 (indico_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 70 (seafile_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 70 (seafile_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:   notice: Transition aborted by webmail-utilization-cpu doing modify cpu=1: Configuration change 
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 66 (hexaa_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 66 (hexaa_monitor_0) on atlas1 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 103 (mailman_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 103 (mailman_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 94 (blue1_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 94 (blue1_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 97 (eprints_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 97 (eprints_monitor_0) on atlas2 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 154 (netfilter_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 154 (netfilter_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 166 (gitlab_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 166 (gitlab_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 123 (wignerdc_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 123 (wignerdc_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 192 (lx1_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 192 (lx1_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 193 (lx0_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 193 (lx0_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 175 (ykval_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 175 (ykval_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 130 (eduid_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:21 atlas4 crmd[32776]:  warning: Action 130 (eduid_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 197 (caladan_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 197 (caladan_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 135 (idp_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 135 (idp_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 161 (w2_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 161 (w2_monitor_0) on atlas5 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 201 (kerberos_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 201 (kerberos_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 139 (wiki_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 139 (wiki_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 204 (mail2_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 204 (mail2_monitor_0) on atlas6 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 140 (webmail_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:  warning: Action 140 (webmail_monitor_0) on atlas3 failed (target: 7 vs. rc: 0): Error
Sep  5 13:08:22 atlas4 crmd[32776]:   notice: Transition 18 (Complete=205, Pending=0, Fired=0, Skipped=8, Incomplete=73, Source=/var/lib/pacemaker/pengine/pe-input-18.bz2): Stopped
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate caladan#011(Started atlas6 -> atlas2)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas3 -> atlas4)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate eprints#011(Started atlas2 -> atlas6)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate git#011(Started atlas4 -> atlas1)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate gitlab#011(Started atlas5 -> atlas3)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas6 -> atlas2)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas5)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate idp#011(Started atlas3 -> atlas4)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas4 -> atlas1)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas2 -> atlas3)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas2)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas3 -> atlas5)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas3 -> atlas4)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas2 -> atlas1)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas6 -> atlas1)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas6 -> atlas2)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas6 -> atlas5)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas6 -> atlas3)
Sep  5 13:08:23 atlas4 pengine[32775]:   notice: Calculated transition 19, saving inputs in /var/lib/pacemaker/pengine/pe-input-19.bz2
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation netfilter_monitor_10000 on atlas5 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation people_monitor_10000 on atlas1 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation indico_monitor_10000 on atlas2 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation wignerdc_monitor_10000 on atlas3 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation lx1_monitor_10000 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation lx0_monitor_10000 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation w0_monitor_10000 on atlas1 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation w2_monitor_10000 on atlas5 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation blue1_monitor_10000 on atlas2 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation caladan_migrate_to_0 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas3 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eprints_migrate_to_0 on atlas2 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation git_migrate_to_0 locally on atlas4 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation gitlab_migrate_to_0 on atlas5 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation idp_migrate_to_0 on atlas3 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 locally on atlas4 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_10000 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas5 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas2 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas2_monitor_0 on atlas0 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas3 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas2 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas1 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating monitor operation stonith-atlas1_monitor_0 on atlas0 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas1 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas2 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas1 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas6 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas5 
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas3 
Sep  5 13:08:23 atlas4 VirtualDomain(git)[49019]: INFO: git: Starting live migration to atlas1 (using: virsh --connect=qemu:///system --quiet migrate --live  git qemu+tls://atlas1/system ).
Sep  5 13:08:23 atlas4 VirtualDomain(mail0)[49020]: INFO: mail0: Starting live migration to atlas1 (using: virsh --connect=qemu:///system --quiet migrate --live  mail0 qemu+tls://atlas1/system ).
Sep  5 13:08:23 atlas4 crmd[32776]:   notice: Transition aborted by indico-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 13:08:25 atlas4 VirtualDomain(git)[49019]: INFO: git: live migration to atlas1 succeeded.
Sep  5 13:08:25 atlas4 crmd[32776]:   notice: Result of migrate_to operation for git on atlas4: 0 (ok) 
Sep  5 13:08:26 atlas4 VirtualDomain(mail0)[49020]: INFO: mail0: live migration to atlas1 succeeded.
Sep  5 13:08:26 atlas4 crmd[32776]:   notice: Result of migrate_to operation for mail0 on atlas4: 0 (ok) 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Transition 19 (Complete=34, Pending=0, Fired=0, Skipped=7, Incomplete=65, Source=/var/lib/pacemaker/pengine/pe-input-19.bz2): Stopped
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource caladan can no longer migrate to atlas2. Stopping on atlas6 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource eduid can no longer migrate to atlas4. Stopping on atlas3 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource eprints can no longer migrate to atlas6. Stopping on atlas2 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource git can no longer migrate to atlas1. Stopping on atlas4 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource gitlab can no longer migrate to atlas3. Stopping on atlas5 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource idp can no longer migrate to atlas4. Stopping on atlas3 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Resource mail0 can no longer migrate to atlas1. Stopping on atlas4 too
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    caladan#011(Started atlas2 -> atlas4)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    eduid#011(Started atlas4 -> atlas6)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    eprints#011(Started atlas6 -> atlas3)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    git#011(Started atlas1 -> atlas2)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    gitlab#011(Started atlas3 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas4)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    idp#011(Started atlas4 -> atlas1)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    mail0#011(Started atlas1 -> atlas3)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas6 -> atlas2)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas2 -> atlas4)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas6)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas3 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas3 -> atlas1)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas3)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas6 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas1 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas6 -> atlas2)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas1 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas2 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas3 -> atlas5)
Sep  5 13:08:27 atlas4 pengine[32775]:   notice: Calculated transition 20, saving inputs in /var/lib/pacemaker/pengine/pe-input-20.bz2
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation caladan_stop_0 on atlas6 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation caladan_stop_0 on atlas2 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas3 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 locally on atlas4 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas2 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation eprints_stop_0 on atlas6 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation git_stop_0 locally on atlas4 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation git_stop_0 on atlas1 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas5 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation gitlab_stop_0 on atlas3 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_10000 on atlas6 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 on atlas3 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation idp_stop_0 locally on atlas4 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 locally on atlas4 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 on atlas1 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating migrate_to operation wiki_migrate_to_0 on atlas3 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas6 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas1 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas6 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas1 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas2 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas3 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas5 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas2 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas5 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas5 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas5 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas5 
Sep  5 13:08:27 atlas4 VirtualDomain(eduid)[49363]: INFO: Issuing graceful shutdown request for domain eduid.
Sep  5 13:08:27 atlas4 VirtualDomain(idp)[49365]: INFO: Issuing graceful shutdown request for domain idp.
Sep  5 13:08:27 atlas4 VirtualDomain(mail0)[49367]: INFO: Domain mail0 already stopped.
Sep  5 13:08:27 atlas4 VirtualDomain(git)[49364]: INFO: Domain git already stopped.
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Result of stop operation for mail0 on atlas4: 0 (ok) 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Result of stop operation for git on atlas4: 0 (ok) 
Sep  5 13:08:27 atlas4 crmd[32776]:   notice: Transition aborted by w1-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 13:08:33 atlas4 crmd[32776]:   notice: Result of stop operation for idp on atlas4: 0 (ok) 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Result of stop operation for eduid on atlas4: 0 (ok) 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Transition 20 (Complete=30, Pending=0, Fired=0, Skipped=4, Incomplete=54, Source=/var/lib/pacemaker/pengine/pe-input-20.bz2): Stopped
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Resource wiki can no longer migrate to atlas5. Stopping on atlas3 too
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas1 -> atlas3)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas6 -> atlas2)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Migrate mailman#011(Started atlas2 -> atlas4)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas5)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    wiki#011(Started atlas5 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas1)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas5 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas5 -> atlas1)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas2 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas5 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas5 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas5 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas5 -> atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   caladan#011(atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   eduid#011(atlas6)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   eprints#011(atlas2)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   git#011(atlas4)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   gitlab#011(atlas1)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   idp#011(atlas5)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Start   mail0#011(atlas3)
Sep  5 13:08:43 atlas4 pengine[32775]:   notice: Calculated transition 21, saving inputs in /var/lib/pacemaker/pengine/pe-input-21.bz2
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating migrate_to operation seafile_migrate_to_0 on atlas1 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas3 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_10000 on atlas3 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas2 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas5 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mailman_migrate_to_0 on atlas2 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating start operation git_start_0 locally on atlas4 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas6 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas6 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas1 
Sep  5 13:08:43 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas6 
Sep  5 13:08:44 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas6 
Sep  5 13:08:44 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas6 
Sep  5 13:08:44 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas6 
Sep  5 13:08:44 atlas4 crmd[32776]:   notice: Initiating migrate_to operation hexaa_migrate_to_0 on atlas1 
Sep  5 13:08:44 atlas4 crmd[32776]:   notice: Initiating start operation mail0_start_0 on atlas3 
Sep  5 13:08:44 atlas4 crmd[32776]:   notice: Transition aborted by mail0-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 13:08:45 atlas4 crmd[32776]:   notice: Result of start operation for git on atlas4: 0 (ok) 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Transition 21 (Complete=25, Pending=0, Fired=0, Skipped=5, Incomplete=41, Source=/var/lib/pacemaker/pengine/pe-input-21.bz2): Stopped
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Resource hexaa can no longer migrate to atlas3. Stopping on atlas1 too
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Resource mailman can no longer migrate to atlas4. Stopping on atlas2 too
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Resource seafile can no longer migrate to atlas5. Stopping on atlas1 too
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Migrate git#011(Started atlas4 -> atlas6)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas6 -> atlas3)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    hexaa#011(Started atlas3 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Migrate mail0#011(Started atlas3 -> atlas4)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Migrate mail2#011(Started atlas6 -> atlas5)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    mailman#011(Started atlas4 -> atlas6)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    seafile#011(Started atlas5 -> atlas1)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas5 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas6 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas1 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas6 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas6 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas6 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas2#011(Started atlas6 -> atlas5)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas6 -> atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Start   caladan#011(atlas2)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Start   eduid#011(atlas5)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Start   eprints#011(atlas6)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Start   gitlab#011(atlas4)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Start   idp#011(atlas3)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Start   wiki#011(atlas1)
Sep  5 13:08:56 atlas4 pengine[32775]:   notice: Calculated transition 22, saving inputs in /var/lib/pacemaker/pengine/pe-input-22.bz2
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating migrate_to operation git_migrate_to_0 locally on atlas4 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas1 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas3 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation mailman_stop_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation mailman_stop_0 locally on atlas4 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation seafile_stop_0 on atlas1 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation seafile_stop_0 on atlas5 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas6 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas1 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas6 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas6 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas6 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas2_stop_0 on atlas6 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas6 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas2_start_0 on atlas5 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 on atlas2 
Sep  5 13:08:56 atlas4 VirtualDomain(mailman)[50193]: INFO: Issuing graceful shutdown request for domain mailman.
Sep  5 13:08:56 atlas4 VirtualDomain(git)[50192]: INFO: git: Starting live migration to atlas6 (using: virsh --connect=qemu:///system --quiet migrate --live  git qemu+tls://atlas6/system ).
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating migrate_to operation ykval_migrate_to_0 on atlas5 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation caladan_start_0 on atlas2 
Sep  5 13:08:56 atlas4 crmd[32776]:   notice: Initiating start operation wiki_start_0 on atlas1 
Sep  5 13:08:57 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_10000 on atlas1 
Sep  5 13:08:57 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_10000 on atlas2 
Sep  5 13:08:57 atlas4 VirtualDomain(git)[50192]: INFO: git: live migration to atlas6 succeeded.
Sep  5 13:08:57 atlas4 crmd[32776]:   notice: Result of migrate_to operation for git on atlas4: 0 (ok) 
Sep  5 13:08:57 atlas4 crmd[32776]:   notice: Initiating migrate_from operation git_migrate_from_0 on atlas6 
Sep  5 13:08:58 atlas4 crmd[32776]:   notice: Initiating migrate_from operation ykval_migrate_from_0 on atlas2 
Sep  5 13:08:58 atlas4 crmd[32776]:   notice: Initiating stop operation git_stop_0 locally on atlas4 
Sep  5 13:08:58 atlas4 VirtualDomain(git)[50321]: INFO: Domain git already stopped.
Sep  5 13:08:58 atlas4 crmd[32776]:   notice: Result of stop operation for git on atlas4: 0 (ok) 
Sep  5 13:08:58 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas5 
Sep  5 13:08:58 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas2 
Sep  5 13:09:02 atlas4 crmd[32776]:   notice: Initiating start operation hexaa_start_0 on atlas2 
Sep  5 13:09:03 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_10000 on atlas2 
Sep  5 13:09:06 atlas4 crmd[32776]:   notice: Result of stop operation for mailman on atlas4: 0 (ok) 
Sep  5 13:09:06 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail0_migrate_to_0 on atlas3 
Sep  5 13:09:06 atlas4 crmd[32776]:   notice: Initiating start operation gitlab_start_0 locally on atlas4 
Sep  5 13:09:06 atlas4 crmd[32776]:   notice: Initiating start operation seafile_start_0 on atlas1 
Sep  5 13:09:06 atlas4 crmd[32776]:   notice: Initiating migrate_to operation mail2_migrate_to_0 on atlas6 
Sep  5 13:09:06 atlas4 crmd[32776]:   notice: Initiating start operation eduid_start_0 on atlas5 
Sep  5 13:09:07 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_10000 on atlas1 
Sep  5 13:09:07 atlas4 crmd[32776]:   notice: Transition aborted by gitlab-utilization-cpu doing modify cpu=2: Configuration change 
Sep  5 13:09:07 atlas4 crmd[32776]:   notice: Result of start operation for gitlab on atlas4: 0 (ok) 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Transition 22 (Complete=45, Pending=0, Fired=0, Skipped=4, Incomplete=26, Source=/var/lib/pacemaker/pengine/pe-input-22.bz2): Stopped
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Resource mail0 can no longer migrate to atlas4. Stopping on atlas3 too
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Resource mail2 can no longer migrate to atlas5. Stopping on atlas6 too
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate caladan#011(Started atlas2 -> atlas6)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate eduid#011(Started atlas5 -> atlas3)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate git#011(Started atlas6 -> atlas2)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate kerberos#011(Started atlas6 -> atlas5)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate hexaa#011(Started atlas2 -> atlas6)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    mail0#011(Started atlas4 -> atlas1)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    mail2#011(Started atlas5 -> atlas3)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate seafile#011(Started atlas1 -> atlas2)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate wiki#011(Started atlas1 -> atlas5)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate webmail#011(Started atlas3 -> atlas6)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Migrate ykval#011(Started atlas2 -> atlas5)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    stonith-atlas0#011(Started atlas2 -> atlas3)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    stonith-atlas6#011(Started atlas2 -> atlas4)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    stonith-atlas5#011(Started atlas2 -> atlas3)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    stonith-atlas4#011(Started atlas2 -> atlas3)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    stonith-atlas3#011(Started atlas2 -> atlas4)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Move    stonith-atlas1#011(Started atlas2 -> atlas4)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Start   eprints#011(atlas3)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Start   idp#011(atlas4)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Start   mailman#011(atlas1)
Sep  5 13:09:08 atlas4 pengine[32775]:   notice: Calculated transition 23, saving inputs in /var/lib/pacemaker/pengine/pe-input-23.bz2
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating migrate_to operation caladan_migrate_to_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating migrate_to operation eduid_migrate_to_0 on atlas5 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating migrate_to operation git_migrate_to_0 on atlas6 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating monitor operation gitlab_monitor_10000 locally on atlas4 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 on atlas3 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation mail0_stop_0 locally on atlas4 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation mail2_stop_0 on atlas6 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation mail2_stop_0 on atlas5 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas0_stop_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas6_stop_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas5_stop_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas4_stop_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas3_stop_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating stop operation stonith-atlas1_stop_0 on atlas2 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas0_start_0 on atlas3 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas6_start_0 locally on atlas4 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas5_start_0 on atlas3 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas4_start_0 on atlas3 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas3_start_0 locally on atlas4 
Sep  5 13:09:08 atlas4 crmd[32776]:   notice: Initiating start operation stonith-atlas1_start_0 locally on atlas4 
Sep  5 13:09:08 atlas4 VirtualDomain(mail0)[50752]: INFO: Issuing graceful shutdown request for domain mail0.
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Result of start operation for stonith-atlas6 on atlas4: 0 (ok) 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Result of start operation for stonith-atlas3 on atlas4: 0 (ok) 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Result of start operation for stonith-atlas1 on atlas4: 0 (ok) 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Initiating migrate_from operation eduid_migrate_from_0 on atlas3 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Initiating migrate_from operation caladan_migrate_from_0 on atlas6 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Initiating migrate_from operation git_migrate_from_0 on atlas2 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Initiating stop operation eduid_stop_0 on atlas5 
Sep  5 13:09:09 atlas4 crmd[32776]:   notice: Initiating stop operation caladan_stop_0 on atlas2 
Sep  5 13:09:10 atlas4 crmd[32776]:   notice: Initiating stop operation git_stop_0 on atlas6 
Sep  5 13:09:15 atlas4 crmd[32776]:   notice: Initiating migrate_to operation kerberos_migrate_to_0 on atlas6 
Sep  5 13:09:15 atlas4 crmd[32776]:   notice: Initiating migrate_to operation wiki_migrate_to_0 on atlas1 
Sep  5 13:09:15 atlas4 crmd[32776]:   notice: Initiating migrate_to operation ykval_migrate_to_0 on atlas2 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation wiki_migrate_from_0 on atlas5 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation ykval_migrate_from_0 on atlas5 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating stop operation wiki_stop_0 on atlas1 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating stop operation ykval_stop_0 on atlas2 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating monitor operation wiki_monitor_10000 on atlas5 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating monitor operation ykval_monitor_10000 on atlas5 
Sep  5 13:09:17 atlas4 crmd[32776]:   notice: Initiating migrate_from operation kerberos_migrate_from_0 on atlas5 
Sep  5 13:09:18 atlas4 crmd[32776]:   notice: Initiating stop operation kerberos_stop_0 on atlas6 
Sep  5 13:09:18 atlas4 crmd[32776]:   notice: Initiating monitor operation kerberos_monitor_10000 on atlas5 
Sep  5 13:09:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation hexaa_migrate_to_0 on atlas2 
Sep  5 13:09:18 atlas4 crmd[32776]:   notice: Initiating migrate_to operation webmail_migrate_to_0 on atlas3 
Sep  5 13:09:18 atlas4 crmd[32776]:   notice: Initiating monitor operation caladan_monitor_10000 on atlas6 
Sep  5 13:09:19 atlas4 crmd[32776]:   notice: Initiating migrate_from operation hexaa_migrate_from_0 on atlas6 
Sep  5 13:09:20 atlas4 crmd[32776]:   notice: Initiating stop operation hexaa_stop_0 on atlas2 
Sep  5 13:09:20 atlas4 crmd[32776]:   notice: Initiating monitor operation hexaa_monitor_10000 on atlas6 
Sep  5 13:09:20 atlas4 crmd[32776]:   notice: Initiating migrate_to operation seafile_migrate_to_0 on atlas1 
Sep  5 13:09:20 atlas4 crmd[32776]:   notice: Initiating monitor operation git_monitor_10000 on atlas2 
Sep  5 13:09:22 atlas4 crmd[32776]:   notice: Initiating migrate_from operation seafile_migrate_from_0 on atlas2 
Sep  5 13:09:22 atlas4 crmd[32776]:   notice: Initiating stop operation seafile_stop_0 on atlas1 
Sep  5 13:09:22 atlas4 crmd[32776]:   notice: Initiating monitor operation seafile_monitor_10000 on atlas2 
Sep  5 13:09:22 atlas4 crmd[32776]:   notice: Initiating start operation mailman_start_0 on atlas1 
Sep  5 13:09:23 atlas4 crmd[32776]:   notice: Initiating monitor operation mailman_monitor_10000 on atlas1 
Sep  5 13:09:25 atlas4 crmd[32776]:   notice: Initiating migrate_from operation webmail_migrate_from_0 on atlas6 
Sep  5 13:09:25 atlas4 crmd[32776]:   notice: Initiating stop operation webmail_stop_0 on atlas3 
Sep  5 13:09:25 atlas4 crmd[32776]:   notice: Initiating start operation mail2_start_0 on atlas3 
Sep  5 13:09:25 atlas4 crmd[32776]:   notice: Initiating monitor operation webmail_monitor_10000 on atlas6 
Sep  5 13:09:25 atlas4 crmd[32776]:   notice: Initiating start operation eprints_start_0 on atlas3 
Sep  5 13:09:25 atlas4 crmd[32776]:   notice: Initiating monitor operation eduid_monitor_10000 on atlas3 
Sep  5 13:09:26 atlas4 crmd[32776]:   notice: Initiating monitor operation eprints_monitor_10000 on atlas3 
Sep  5 13:09:26 atlas4 crmd[32776]:   notice: Initiating monitor operation mail2_monitor_10000 on atlas3 
Sep  5 13:10:23 atlas4 crmd[32776]:   notice: Transition aborted by status-1084762113-shutdown doing modify shutdown=1536145823: Transient attribute change 
Sep  5 13:11:03 atlas4 VirtualDomain(mail0)[50752]: INFO: Issuing forced shutdown (destroy) request for domain mail0.
Sep  5 13:11:03 atlas4 lrmd[32773]:   notice: mail0_stop_0:50752:stderr [  ]
Sep  5 13:11:03 atlas4 crmd[32776]:   notice: Result of stop operation for mail0 on atlas4: 0 (ok) 
Sep  5 13:11:03 atlas4 crmd[32776]:   notice: Transition 23 (Complete=76, Pending=0, Fired=0, Skipped=2, Incomplete=4, Source=/var/lib/pacemaker/pengine/pe-input-23.bz2): Stopped


More information about the Users mailing list