<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
      charset=windows-1252">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">On 09/13/2017 10:26 AM, CART Andreas
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:2F7CCDA58EE1AA49BA591FE26F2FAB04E8C35A7B@neptun.vie.sonorys.at">
      <meta http-equiv="Content-Type" content="text/html;
        charset=windows-1252">
      <meta name="Generator" content="Microsoft Exchange Server">
      <!-- converted from rtf -->
      <style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
      <font size="2" face="Calibri"><span style="font-size:11pt;">
          <div><font color="#0070C0">Hello</font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">We have a basic 2 node
              active/passive cluster with Pacemaker (1.1.16 , pcs:
              0.9.148) / CMAN (3.0.12.1) / Corosync (1.4.7) on RHEL 6.8.</font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">On the occasion of testing the
              cluster we noticed that dependent resources are stopped
              when calling cleanup for a resource lower down the order
              chain.</font></div>
          <div><font color="#0070C0">But for the production system we
              have to avoid this by all means.</font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">Actually  we have a master/slave
              resource and dependent on that and colocated with the
              master a clustered IP address.</font></div>
          <div><font color="#0070C0">Here is the relevant part of the
              cluster config:</font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Cluster Name:
                bam-cluster</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Corosync Nodes:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> bam1-backend
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Pacemaker Nodes:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> bam1-backend
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> </span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Resources:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> Master: mvno-100-master</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">  Meta Attrs:
                master-node-max=1 clone-max=2 notify=true master-max=1
                clone-node-max=1</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">  Resource: mvno-100
                (class=ocf provider=sonorys type=sba)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">   Operations: start
                interval=0s timeout=120s (mvno-100-start-interval-0s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">               stop
                interval=0s timeout=120s (mvno-100-stop-interval-0s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">               promote
                interval=0s timeout=5s (mvno-100-promote-interval-0s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">               demote
                interval=0s timeout=5s (mvno-100-demote-interval-0s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">               monitor
                interval=9 role=Master (mvno-100-monitor-interval-9)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">               monitor
                interval=10 role=Slave (mvno-100-monitor-interval-10)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> Resource:
                IPaddrAdminToolMvno100 (class=ocf provider=heartbeat
                type=IPaddr2)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">  Attributes:
                ip=xxx.xxx.xxx.xxx cidr_netmask=yy</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">  Operations: start
                interval=0s timeout=20s
                (IPaddrAdminToolMvno100-start-interval-0s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">              stop
                interval=0s timeout=20s
                (IPaddrAdminToolMvno100-stop-interval-0s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">              monitor
                interval=10s timeout=20s
                (IPaddrAdminToolMvno100-monitor-interval-10s)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Ordering Constraints:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">  start
                IPaddrAdminToolMvno100 then promote mvno-100-master
                (kind:Mandatory)
                (id:order-IPaddrAdminToolMvno100-mvno-100-master-mandatory)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Colocation Constraints:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">  IPaddrAdminToolMvno100
                with mvno-100-master (score:INFINITY) (rsc-role:Started)
                (with-rsc-role:Master)
                (id:colocation-IPaddrAdminToolMvno100-mvno-100-master-INFINITY)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Cluster Properties:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> cluster-infrastructure:
                cman</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">
                default-resource-stickiness: 100</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> no-quorum-policy:
                ignore</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"> symmetric-cluster: true</span></font></div>
        </span></font></blockquote>
    <br>
    Have you tried the other way round?<br>
    like: promote mvno-100 then start IPaddrAdminToolMvno100<br>
    <br>
    Regards,<br>
    Klaus<br>
    <font size="1"><font face="Courier New"></font></font><br>
    <blockquote type="cite"
cite="mid:2F7CCDA58EE1AA49BA591FE26F2FAB04E8C35A7B@neptun.vie.sonorys.at"><font
        size="2" face="Calibri"><span style="font-size:11pt;">
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">For example we had the following
              failure notice:</font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Failed Actions:</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">* mvno-100_monitor_10000
                on bam1-backend 'not running' (7): call=40,
                status=complete, exitreason='none',</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">    last-rc-change='Tue
                Sep 12 09:55:49 2017', queued=0ms, exec=0ms</span></font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">And cleared it by calling:</font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;"># pcs resource cleanup
                mvno-100</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Wed Sep 13 06:43:53 UTC
                2017</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Cleaning up mvno-100:0
                on bam1-backend, removing fail-count-mvno-100</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Cleaning up mvno-100:0
                on bam2-backend, removing fail-count-mvno-100</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Waiting for 2 replies
                from the CRMd.. OK</span></font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">This resulted in stopping
              IPaddrAdminToolMvno100 and starting it again, which
              actually means a (short) interruption of service … which
              we would like to avoid.</font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">Here is the corresponding logfile
              for the cluster:</font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3829]
                bam1-omc      attrd:   notice: attrd_cs_dispatch:       
                Update relayed from bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3829]
                bam1-omc      attrd:   notice: attrd_trigger_update:    
                Sending flush op to all hosts for: fail-count-mvno-100
                (<null>)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.167.0 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.167.1 590cae41f6fad2b00a8cf067302756e4</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   --
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='mvno-100']</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=1</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_delete operation for section
                //node_state[@uname='bam1-backend']//lrm_resource[@id='mvno-10</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">0']: OK (rc=0,
                origin=local/crmd/1485, version=0.167.0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.167.0 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.167.1 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   --
/cib/status/node_state[@id='bam2-backend']/lrm[@id='bam2-backend']/lrm_resources/lrm_resource[@id='mvno-100']</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=1</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_delete operation for section
                //node_state[@uname='bam2-backend']//lrm_resource[@id='mvno-10</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">0']: OK (rc=0,
                origin=bam2-backend/crmd/39, version=0.167.0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: delete_resource: 
                Removing resource mvno-100 for
                1c0903b8-1dcc-432a-b6a8-e409ce917dda (root) on
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3828]
                bam1-omc       lrmd:     info: cancel_recurring_action: 
                Cancelling ocf operation mvno-100_monitor_9000</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3829]
                bam1-omc      attrd:   notice: attrd_perform_update:    
                Sent delete 57: node=bam1-backend,
                attr=fail-count-mvno-100, id=<n/a>, set=(null),
                section=status</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: lrm_remove_deleted_op:   
                Removing op mvno-100_monitor_9000:95 for deleted
                resource mvno-100</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: notify_deleted:  
                Notifying 1c0903b8-1dcc-432a-b6a8-e409ce917dda on
                bam2-backend that mvno-100 was deleted</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.167.0 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.167.1 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   --
/cib/status/node_state[@id='bam1-backend']/transient_attributes[@id='bam1-backend']/instance_attributes[@id='s</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">tatus-bam1-backend']/nvpair[@id='status-bam1-backend-fail-count-mvno-100']</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=1</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_delete operation for section status: OK
                (rc=0, origin=local/attrd/57,
                version=0.167.1)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.167.1 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.167.2 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   --
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='mvno-100']</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_delete operation for section
                //node_state[@uname='bam1-backend']//lrm_resource[@id='mvno-10</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">0']: OK (rc=0,
                origin=local/crmd/1486, version=0.167.2)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by deletion of
                nvpair[@id='status-bam1-backend-fail-count-mvno-100']:
                Transient attrib</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">ute change | cib=0.167.1
                source=abort_unless_down:345
path=/cib/status/node_state[@id='bam1-backend']/transient_attributes[@id='bam1-backend']/instance_attributes[@id='status-bam1-backe</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">nd']/nvpair[@id='status-bam1-backend-fail-count-mvno-100']
                complete=true</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by deletion of
                lrm_resource[@id='mvno-100']: Resource state removal
                | cib=0.167.2 sour</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">ce=abort_unless_down:345
path=/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='mvno-100']
                complete=true</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3828]
                bam1-omc       lrmd:     info:
                process_lrmd_get_rsc_info:        Resource 'mvno-100'
                not found (3 active resources)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: process_lrm_event:       
                Result of monitor operation for mvno-100 on
                bam1-backend: Cancelled | call=95
                key=mvno-100_monitor_9000
                c</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">onfirmed=true</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: update_history_cache:    
                Resource mvno-100 no longer exists, not updating cache</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:   notice: do_state_transition:     
                State transition S_IDLE -> S_POLICY_ENGINE |
                input=I_PE_CALC cause=C_FSA_INTERNAL
                origin=abort_transition</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">_graph</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.167.2 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.0 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @epoch=168, @num_updates=0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-opti</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">ons-last-lrm-refresh']: 
                @value=1505285033</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section crm_config:
                OK (rc=0, origin=local/crmd/1488,
                version=0.168.0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by
                cib-bootstrap-options-last-lrm-refresh doing modify
                last-lrm-refresh=1505285033:
                Configuration change | cib=0.168.0
                source=te_update_diff:444
path=/cib/configuration/crm_config/cluster_property_set[@id='cib-bootstrap-options']/nvpair[@id='cib-bootstrap-options-last-lrm-refresh']
                complete=true</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_file_backup: 
                Archived previous version as
                /var/lib/pacemaker/cib/cib-65.raw</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.0 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.1 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   --
/cib/status/node_state[@id='bam2-backend']/lrm[@id='bam2-backend']/lrm_resources/lrm_resource[@id='mvno-100']</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=1</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_delete operation for section
                //node_state[@uname='bam2-backend']//lrm_resource[@id='mvno-100']:
                OK (rc=0, origin=bam2-backend/crmd/40, version=0.168.1)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section crm_config:
                OK (rc=0, origin=bam2-backend/crmd/42,
                version=0.168.1)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info:
                cib_file_write_with_digest:       Wrote version 0.168.0
                of the CIB to disk (digest:
                7d2e14eee5a440883e9659536e70260f)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:53 [3826]
                bam1-omc        cib:     info:
                cib_file_write_with_digest:       Reading cluster
                configuration file /var/lib/pacemaker/cib/cib.BuFPbI
                (digest:
                /var/lib/pacemaker/cib/cib.jHzYAc)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: parse_notifications:     
                No optional alerts section in cib</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by deletion of
                lrm_resource[@id='mvno-100']: Resource state removal
                | cib=0.168.1 source=abort_unless_down:345
path=/cib/status/node_state[@id='bam2-backend']/lrm[@id='bam2-backend']/lrm_resources/lrm_resource[@id='mvno-100']
                complete=true</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:   notice: unpack_config:    On loss
                of CCM Quorum: Ignore</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info:
                determine_online_status_fencing:  Node bam2-backend is
                active</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: determine_online_status: 
                Node bam2-backend is online</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info:
                determine_online_status_fencing:  Node bam1-backend is
                active</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: determine_online_status: 
                Node bam1-backend is online</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: determine_op_status:     
                Operation monitor found resource IPaddrAdminToolMvno100
                active on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: clone_print:      
                Master/Slave Set: mvno-100-master [mvno-100]</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: short_print:          
                Stopped: [ bam1-backend bam2-backend ]</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: native_print:    
                IPaddrAdminToolMvno100  (ocf::heartbeat:IPaddr2):      
                Started bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: native_print:    
                IPMI_fence_bam1 (stonith:fence_ipmilan):        Started
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: native_print:    
                IPMI_fence_bam2 (stonith:fence_ipmilan):        Started
                bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: master_color:    
                mvno-100-master: Promoted 0 instances of a possible 1 to
                master</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: native_color:    
                Resource IPaddrAdminToolMvno100 cannot run anywhere</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for mvno-100:0 on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for mvno-100:1 on bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for mvno-100:0 on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for mvno-100:1 on bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:   notice: LogActions:       Start  
                mvno-100:0      (bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:   notice: LogActions:       Start  
                mvno-100:1      (bam2-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:   notice: LogActions:       Stop   
                IPaddrAdminToolMvno100  (bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^</span></font></div>
          <div><font color="#0070C0">I suspect the cluster “forgets” all
              status information about the cleaned up resource and
              therefore supposes it is in stopped state and has to be
              started.</font></div>
          <div><font color="#0070C0">And due to the order constraint the
              dependent IP address has to be stopped as long as the
              cluster does not see the first resource in state started.</font></div>
          <div><font color="#0070C0">Shouldn’t there be a monitor
              operation to determine the actual state before performing
              any actions based on unconfirmed assumptions of state?</font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: LogActions:       Leave  
                IPMI_fence_bam1 (Started bam2-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:     info: LogActions:       Leave  
                IPMI_fence_bam2 (Started bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3830]
                bam1-omc    pengine:   notice: process_pe_message:      
                Calculated transition 1379, saving inputs in
                /var/lib/pacemaker/pengine/pe-input-31.bz2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: do_state_transition:     
                State transition S_POLICY_ENGINE ->
                S_TRANSITION_ENGINE | input=I_PE_SUCCESS
                cause=C_IPC_MESSAGE
                origin=handle_response</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: do_te_invoke:    
                Processing graph 1379 (ref=pe_calc-dc-1505285034-1562)
                derived from /var/lib/pacemaker/pengine/pe-input-31.bz2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating monitor operation mvno-100:0_monitor_0
                locally on bam1-backend | action 5</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3828]
                bam1-omc       lrmd:     info:
                process_lrmd_get_rsc_info:        Resource 'mvno-100'
                not found (3 active resources)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3828]
                bam1-omc       lrmd:     info:
                process_lrmd_get_rsc_info:        Resource 'mvno-100:0'
                not found (3 active resources)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3828]
                bam1-omc       lrmd:     info:
                process_lrmd_rsc_register:        Added 'mvno-100' to
                the rsc list (4 active resources)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=mvno-100_monitor_0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating monitor operation mvno-100:1_monitor_0 on
                bam2-backend | action 6</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating stop operation IPaddrAdminToolMvno100_stop_0
                locally on bam1-backend | action
                35</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3828]
                bam1-omc       lrmd:     info: cancel_recurring_action: 
                Cancelling ocf operation
                IPaddrAdminToolMvno100_monitor_10000</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=35:1379:0:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=IPaddrAdminToolMvno100_stop_0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3828]
                bam1-omc       lrmd:     info: log_execute:     
                executing - rsc:IPaddrAdminToolMvno100 action:stop
                call_id:104</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: process_lrm_event:       
                Result of monitor operation for IPaddrAdminToolMvno100
                on bam1-backend: Cancelled
                | call=90 key=IPaddrAdminToolMvno100_monitor_10000
                confirmed=true</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[56850]:  
                2017/09/13_06:43:54 INFO: mvno-100 monitor started</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[56850]:  
                2017/09/13_06:43:54 INFO: Check status</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">IPaddr2(IPaddrAdminToolMvno100)[56851]:
                2017/09/13_06:43:54 INFO: IP status = ok, IP_CIP=</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3828]
                bam1-omc       lrmd:     info: log_finished:    
                finished - rsc:IPaddrAdminToolMvno100 action:stop
                call_id:104 pid:56851 exit-code:0 exec-time:72ms
                queue-time:0ms</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:   notice: process_lrm_event:       
                Result of stop operation for IPaddrAdminToolMvno100 on
                bam1-backend: 0 (ok) | call=104
                key=IPaddrAdminToolMvno100_stop_0 confirmed=true
                cib-update=1494</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.1 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.2 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='IPaddrAdminToolMvno100']/lrm_rsc_op[@id='IPaddrAdminToolMvno100_last_0']: @operation_key=IPaddrAdminToolMvno100_stop_0,
                @operation=stop,
                @transition-key=35:1379:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
@transition-magic=0:0;35:1379:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
                @call-id=104, @last-run=1505285034,
                @last-rc-change=1505285034,
                @ex</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=local/crmd/1494,
                version=0.168.2)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action IPaddrAdminToolMvno100_stop_0 (35) confirmed on
                bam1-backend (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[56850]:  
                2017/09/13_06:43:54 INFO: mvno-100 monitor returned 8</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:   notice: process_lrm_event:       
                Result of probe operation for mvno-100 on bam1-backend:
                8 (master) | call=102 key=mvno-100_monitor_0
                confirmed=true cib-update=1495</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.2 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.3 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=3</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:   ++
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources: 
                <lrm_resource
                id="mvno-100" type="sba" class="ocf"
                provider="sonorys"/></span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:  
++                                                                                     <lrm_rsc_op
                id="mvno-100_last_failure_0"
                operation_key="mvno-100_monitor_0" operation="monitor"
                crm-debug-origin="do_update_resource"
                crm_feature_set="3.0.11"
                transition-key="5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
transition-magic="0:8;5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
                on_node="bam1-backend" call-id="102" rc-code="8"
                op-status="0"</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:  
++                                                                                     <lrm_rsc_op
                id="mvno-100_last_0" operation_key="mvno-100_monitor_0"
                operation="monitor"
                crm-debug-origin="do_update_resource"
                crm_feature_set="3.0.11"
                transition-key="5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
transition-magic="0:8;5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
                on_node="bam1-backend" call-id="102" rc-code="8"
                op-status="0" interva</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_perform_op:  
++                                                                                   
                </lrm_resource></span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=local/crmd/1495,
                version=0.168.3)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:  warning: status_from_rc:   Action
                5 (mvno-100:0_monitor_0) on bam1-backend failed (target:
                7 vs. rc: 8): Error</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:   notice: abort_transition_graph:  
                Transition aborted by operation mvno-100_monitor_0
                'create' on bam1-backend: Event
                failed |
                magic=0:8;5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c
                cib=0.168.3 source=match_graph_event:310 complete=false</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_monitor_0 (5) confirmed on bam1-backend
                (rc=8)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: process_graph_event:     
                Detected action (1379.5) mvno-100_monitor_0.102=master:
                failed</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:  warning: status_from_rc:   Action
                5 (mvno-100:0_monitor_0) on bam1-backend failed (target:
                7 vs. rc: 8): Error</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by operation mvno-100_monitor_0
                'create' on bam1-backend: Event
                failed |
                magic=0:8;5:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c
                cib=0.168.3 source=match_graph_event:310 complete=false</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_monitor_0 (5) confirmed on bam1-backend
                (rc=8)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:54 [3831]
                bam1-omc       crmd:     info: process_graph_event:     
                Detected action (1379.5) mvno-100_monitor_0.102=master:
                failed</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.3 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.4 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=4</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   ++
/cib/status/node_state[@id='bam2-backend']/lrm[@id='bam2-backend']/lrm_resources: 
                <lrm_resource
                id="mvno-100" type="sba" class="ocf"
                provider="sonorys"/></span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:  
++                                                                                     <lrm_rsc_op
                id="mvno-100_last_failure_0"
                operation_key="mvno-100_monitor_0" operation="monitor"
                crm-debug-origin="do_update_resource"
                crm_feature_set="3.0.11"
                transition-key="6:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
transition-magic="0:0;6:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
                on_node="bam2-backend" call-id="82" rc-code="0"
                op-status="0"</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:  
++                                                                                     <lrm_rsc_op
                id="mvno-100_last_0" operation_key="mvno-100_monitor_0"
                operation="monitor"
                crm-debug-origin="do_update_resource"
                crm_feature_set="3.0.11"
                transition-key="6:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
transition-magic="0:0;6:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c"
                on_node="bam2-backend" call-id="82" rc-code="0"
                op-status="0" interval</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:  
++                                                                                   
                </lrm_resource></span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=bam2-backend/crmd/44,
                version=0.168.4)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:  warning: status_from_rc:   Action
                6 (mvno-100:1_monitor_0) on bam2-backend failed (target:
                7 vs. rc: 0): Error</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by operation mvno-100_monitor_0
                'create' on bam2-backend: Event
                failed |
                magic=0:0;6:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c
                cib=0.168.4 source=match_graph_event:310 complete=false</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_monitor_0 (6) confirmed on bam2-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: process_graph_event:     
                Detected action (1379.6) mvno-100_monitor_0.82=ok:
                failed</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:  warning: status_from_rc:   Action
                6 (mvno-100:1_monitor_0) on bam2-backend failed (target:
                7 vs. rc: 0): Error</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: abort_transition_graph:  
                Transition aborted by operation mvno-100_monitor_0
                'create' on bam2-backend: Event
                failed |
                magic=0:0;6:1379:7:e2c19428-0707-4677-a89a-ff1c19ebe57c
                cib=0.168.4 source=match_graph_event:310 complete=false</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_monitor_0 (6) confirmed on bam2-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: process_graph_event:     
                Detected action (1379.6) mvno-100_monitor_0.82=ok:
                failed</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: run_graph:       
                Transition 1379 (Complete=6, Pending=0, Fired=0,
                Skipped=0, Incomplete=10,
                Source=/var/lib/pacemaker/pengine/pe-input-31.bz2):
                Complete</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_state_transition:     
                State transition S_TRANSITION_ENGINE ->
                S_POLICY_ENGINE | input=I_PE_CALC cause=C_FSA_INTERNAL
                origin=notify_crmd</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:   notice: unpack_config:    On loss
                of CCM Quorum: Ignore</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info:
                determine_online_status_fencing:  Node bam2-backend is
                active</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_online_status: 
                Node bam2-backend is online</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info:
                determine_online_status_fencing:  Node bam1-backend is
                active</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_online_status: 
                Node bam1-backend is online</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_op_status:     
                Operation monitor found resource mvno-100:0 active on
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_op_status:     
                Operation monitor found resource mvno-100:0 active on
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_op_status:     
                Operation monitor found resource IPaddrAdminToolMvno100
                active on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_op_status:     
                Operation monitor found resource mvno-100:1 active in
                master mode on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: determine_op_status:     
                Operation monitor found resource mvno-100:1 active in
                master mode on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: clone_print:      
                Master/Slave Set: mvno-100-master [mvno-100]</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: short_print:          
                Masters: [ bam1-backend ]</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: short_print:          
                Slaves: [ bam2-backend ]</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: native_print:    
                IPaddrAdminToolMvno100  (ocf::heartbeat:IPaddr2):      
                Stopped</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: native_print:    
                IPMI_fence_bam1 (stonith:fence_ipmilan):        Started
                bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: native_print:    
                IPMI_fence_bam2 (stonith:fence_ipmilan):        Started
                bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: master_color:    
                Promoting mvno-100:1 (Master bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: master_color:    
                mvno-100-master: Promoted 1 instances of a possible 1 to
                master</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for mvno-100:0 on bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (9s) for mvno-100:1 on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for mvno-100:0 on bam2-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (9s) for mvno-100:1 on bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: RecurringOp:       Start
                recurring monitor (10s) for IPaddrAdminToolMvno100 on
                bam1-backend</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: LogActions:       Leave  
                mvno-100:0      (Slave bam2-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: LogActions:       Leave  
                mvno-100:1      (Master bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:   notice: LogActions:       Start  
                IPaddrAdminToolMvno100  (bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: LogActions:       Leave  
                IPMI_fence_bam1 (Started bam2-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:     info: LogActions:       Leave  
                IPMI_fence_bam2 (Started bam1-backend)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3830]
                bam1-omc    pengine:   notice: process_pe_message:      
                Calculated transition 1380, saving inputs in
                /var/lib/pacemaker/pengine/pe-input-32.bz2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_state_transition:     
                State transition S_POLICY_ENGINE ->
                S_TRANSITION_ENGINE | input=I_PE_SUCCESS
                cause=C_IPC_MESSAGE
                origin=handle_response</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_te_invoke:    
                Processing graph 1380 (ref=pe_calc-dc-1505285035-1566)
                derived from /var/lib/pacemaker/pengine/pe-input-32.bz2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating start operation
                IPaddrAdminToolMvno100_start_0 locally on bam1-backend |
                action
                36</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=36:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=IPaddrAdminToolMvno100_start_0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_execute:     
                executing - rsc:IPaddrAdminToolMvno100 action:start
                call_id:105</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating notify operation
                mvno-100_pre_notify_promote_0 on bam2-backend | action
                52</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating notify operation
                mvno-100_pre_notify_promote_0 locally on bam1-backend |
                action
                54</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=54:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=mvno-100_notify_0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_execute:     
                executing - rsc:mvno-100 action:notify call_id:106</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[56964]:  
                2017/09/13_06:43:55 INFO: mvno-100 notify started</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_finished:    
                finished - rsc:mvno-100 action:notify call_id:106
                pid:56964 exit-code:3 exec-time:21ms queue-time:0ms</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_notify_0 (54) confirmed on bam1-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: process_lrm_event:       
                Result of notify operation for mvno-100 on bam1-backend:
                0 (ok) | call=106 key=mvno-100_notify_0
                confirmed=true cib-update=0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_notify_0 (52) confirmed on bam2-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">IPaddr2(IPaddrAdminToolMvno100)[56963]:
                2017/09/13_06:43:55 INFO: Adding inet address
                xxx.xxx.xxx.xxx/yy with broadcast address
                zzz.zzz.zzz.zzz to device bond2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">IPaddr2(IPaddrAdminToolMvno100)[56963]:
                2017/09/13_06:43:55 INFO: Bringing device bond2 up</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">IPaddr2(IPaddrAdminToolMvno100)[56963]:
                2017/09/13_06:43:55 INFO:
                /usr/libexec/heartbeat/send_arp -i 200 -r 5 -p
                /var/run/resource-agents/send_arp-xxx.xxx.xxx.xxx
                bond2 xxx.xxx.xxx.xxx auto not_used not_used</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_finished:    
                finished - rsc:IPaddrAdminToolMvno100 action:start
                call_id:105 pid:56963 exit-code:0 exec-time:91ms
                queue-time:0ms</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: action_synced_wait:      
                Managed IPaddr2_meta-data_0 process 57044 exited with
                rc=0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: process_lrm_event:       
                Result of start operation for IPaddrAdminToolMvno100 on
                bam1-backend: 0 (ok) | call=105
                key=IPaddrAdminToolMvno100_start_0 confirmed=true
                cib-update=1497</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.4 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.5 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=5</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='IPaddrAdminToolMvno100']/lrm_rsc_op[@id='IPaddrAdminToolMvno100_last_0']: @operation_key=IPaddrAdminToolMvno100_start_0,
                @operation=start,
                @transition-key=36:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
@transition-magic=0:0;36:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
                @call-id=105, @last-run=1505285035,
                @last-rc-change=1505285035,
                @</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=local/crmd/1497,
                version=0.168.5)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action IPaddrAdminToolMvno100_start_0 (36) confirmed on
                bam1-backend (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating monitor operation
                IPaddrAdminToolMvno100_monitor_10000 locally on
                bam1-backend
                | action 37</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=37:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=IPaddrAdminToolMvno100_monitor_10000</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating promote operation mvno-100_promote_0 locally
                on bam1-backend | action 10</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=10:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=mvno-100_promote_0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_execute:     
                executing - rsc:mvno-100 action:promote call_id:108</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57049]:  
                2017/09/13_06:43:55 INFO: mvno-100 promote started</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57049]:  
                2017/09/13_06:43:55 INFO: Check status</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: process_lrm_event:       
                Result of monitor operation for IPaddrAdminToolMvno100
                on bam1-backend: 0 (ok) |
                call=107 key=IPaddrAdminToolMvno100_monitor_10000
                confirmed=false cib-update=1498</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.5 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.6 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=6</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='IPaddrAdminToolMvno100']/lrm_rsc_op[@id='IPaddrAdminToolMvno100_monitor_10000']: @transition-key=37:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
@transition-magic=0:0;37:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
                @call-id=107, @last-rc-change=1505285035, @exec-time=65</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=local/crmd/1498,
                version=0.168.6)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action IPaddrAdminToolMvno100_monitor_10000 (37)
                confirmed on bam1-backend (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57049]:  
                2017/09/13_06:43:55 INFO: Resource is already running as
                Master</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57049]:  
                2017/09/13_06:43:55 INFO: mvno-100 promote returned 0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_finished:    
                finished - rsc:mvno-100 action:promote call_id:108
                pid:57049 exit-code:0 exec-time:287ms
                queue-time:0ms</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: process_lrm_event:       
                Result of promote operation for mvno-100 on
                bam1-backend: 0 (ok) | call=108 key=mvno-100_promote_0
                confirmed=true cib-update=1499</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.6 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.7 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=7</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='mvno-100']/lrm_rsc_op[@id='mvno-100_last_0']: @operation_key=mvno-100_promote_0,
                @operation=promote,
                @transition-key=10:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
@transition-magic=0:0;10:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c,
                @call-id=108, @rc-code=0, @last-run=1505285035,
                @last-rc-change=1505285035,
                @exec-time=287</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=local/crmd/1499,
                version=0.168.7)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_promote_0 (10) confirmed on bam1-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating notify operation
                mvno-100_post_notify_promote_0 on bam2-backend | action
                53</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating notify operation
                mvno-100_post_notify_promote_0 locally on bam1-backend |
                action
                55</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=55:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=mvno-100_notify_0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_execute:     
                executing - rsc:mvno-100 action:notify call_id:109</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57157]:  
                2017/09/13_06:43:55 INFO: mvno-100 notify started</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_notify_0 (53) confirmed on bam2-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3828]
                bam1-omc       lrmd:     info: log_finished:    
                finished - rsc:mvno-100 action:notify call_id:109
                pid:57157 exit-code:3 exec-time:25ms queue-time:0ms</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_notify_0 (55) confirmed on bam1-backend
                (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: process_lrm_event:       
                Result of notify operation for mvno-100 on bam1-backend:
                0 (ok) | call=109 key=mvno-100_notify_0
                confirmed=true cib-update=0</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating monitor operation mvno-100_monitor_10000 on
                bam2-backend | action 6</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: te_rsc_command:  
                Initiating monitor operation mvno-100_monitor_9000
                locally on bam1-backend | action 11</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_lrm_rsc_op:   
                Performing
                key=11:1380:8:e2c19428-0707-4677-a89a-ff1c19ebe57c
                op=mvno-100_monitor_9000</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57166]:  
                2017/09/13_06:43:55 INFO: mvno-100 monitor started</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57166]:  
                2017/09/13_06:43:55 INFO: Check status</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.7 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.8 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=8</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   ++
/cib/status/node_state[@id='bam2-backend']/lrm[@id='bam2-backend']/lrm_resources/lrm_resource[@id='mvno-100']: <lrm_rsc_op
                id="mvno-100_monitor_10000"
                operation_key="mvno-100_monitor_10000"
                operation="monitor"
                crm-debug-origin="do_update_resource"
                crm_feature_set="3.0.11"
                transition-key="6:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c"
transition-magic="0:0;6:1380:0:e2c19428-0707-4677-a89a-ff1c19ebe57c"
                on_node="bam2-backend" call-id="</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=bam2-backend/crmd/45,
                version=0.168.8)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_monitor_10000 (6) confirmed on
                bam2-backend (rc=0)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">sba(mvno-100)[57166]:  
                2017/09/13_06:43:55 INFO: mvno-100 monitor returned 8</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: process_lrm_event:       
                Result of monitor operation for mvno-100 on
                bam1-backend: 8 (master) | call=110
                key=mvno-100_monitor_9000 confirmed=false
                cib-update=1500</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                --- 0.168.8 2</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   Diff:
                +++ 0.168.9 (null)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   + 
                /cib:  @num_updates=9</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_perform_op:   ++
/cib/status/node_state[@id='bam1-backend']/lrm[@id='bam1-backend']/lrm_resources/lrm_resource[@id='mvno-100']: <lrm_rsc_op
                id="mvno-100_monitor_9000"
                operation_key="mvno-100_monitor_9000"
                operation="monitor"
                crm-debug-origin="do_update_resource"
                crm_feature_set="3.0.11"
                transition-key="11:1380:8:e2c19428-0707-4677-a89a-ff1c19ebe57c"
transition-magic="0:8;11:1380:8:e2c19428-0707-4677-a89a-ff1c19ebe57c"
                on_node="bam1-backend" call-id="</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3826]
                bam1-omc        cib:     info: cib_process_request:     
                Completed cib_modify operation for section status: OK
                (rc=0, origin=local/crmd/1500,
                version=0.168.9)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: match_graph_event:       
                Action mvno-100_monitor_9000 (11) confirmed on
                bam1-backend (rc=8)</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: run_graph:       
                Transition 1380 (Complete=15, Pending=0, Fired=0,
                Skipped=0, Incomplete=0,
                Source=/var/lib/pacemaker/pengine/pe-input-32.bz2):
                Complete</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:     info: do_log:   Input
                I_TE_SUCCESS received in state S_TRANSITION_ENGINE from
                notify_crmd</span></font></div>
          <div style="padding-left:35.4pt;"><font size="1" face="Courier
              New"><span style="font-size:8pt;">Sep 13 06:43:55 [3831]
                bam1-omc       crmd:   notice: do_state_transition:     
                State transition S_TRANSITION_ENGINE -> S_IDLE |
                input=I_TE_SUCCESS cause=C_FSA_INTERNAL
                origin=notify_crmd</span></font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">How can we avoid this interruption
              of service on cleanup?</font></div>
          <div><font color="#0070C0"> </font></div>
          <div><font color="#0070C0">Kind regards</font></div>
          <div><font color="#0070C0">Andreas Cart</font></div>
          <div><font color="#0070C0"> </font></div>
        </span></font>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list: <a class="moz-txt-link-abbreviated" href="mailto:Users@clusterlabs.org">Users@clusterlabs.org</a>
<a class="moz-txt-link-freetext" href="http://lists.clusterlabs.org/mailman/listinfo/users">http://lists.clusterlabs.org/mailman/listinfo/users</a>

Project Home: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org">http://www.clusterlabs.org</a>
Getting started: <a class="moz-txt-link-freetext" href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a>
Bugs: <a class="moz-txt-link-freetext" href="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>