[ClusterLabs] Pending Fencing Actions shown in pcs status

Steffen Vinther Sørensen svinther at gmail.com
Thu Jan 7 03:04:52 EST 2021


Hi Hideo,

Here CIB settings attached (pcs config show) for all 3 of my nodes
(all 3 seems 100% identical), node03 is the DC.

Regards
Steffen

On Thu, Jan 7, 2021 at 8:06 AM <renayama19661014 at ybb.ne.jp> wrote:
>
> Hi Steffen,
> Hi Reid,
>
> I also checked the Centos source rpm and it seems to include a fix for the problem.
>
> As Steffen suggested, if you share your CIB settings, I might know something.
>
> If this issue is the same as the fix, the display will only be displayed on the DC node and will not affect the operation.
> The pending actions shown will remain for a long time, but will not have a negative impact on the cluster.
>
> Best Regards,
> Hideo Yamauchi.
>
>
> ----- Original Message -----
> > From: Reid Wahl <nwahl at redhat.com>
> > To: Cluster Labs - All topics related to open-source clustering welcomed <users at clusterlabs.org>
> > Cc:
> > Date: 2021/1/7, Thu 15:58
> > Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs status
> >
> > It's supposedly fixed in that version.
> >   - https://bugzilla.redhat.com/show_bug.cgi?id=1787749
> >   - https://access.redhat.com/solutions/4713471
> >
> > So you may be hitting a different issue (unless there's a bug in the
> > pcmk 1.1 backport of the fix).
> >
> > I may be a little bit out of my area of knowledge here, but can you
> > share the CIBs from nodes 1 and 3? Maybe Hideo, Klaus, or Ken has some
> > insight.
> >
> > On Wed, Jan 6, 2021 at 10:53 PM Steffen Vinther Sørensen
> > <svinther at gmail.com> wrote:
> >>
> >>  Hi Hideo,
> >>
> >>  If the fix is not going to make it into the CentOS7 pacemaker version,
> >>  I guess the stable approach to take advantage of it is to build the
> >>  cluster on another OS than CentOS7 ? A little late for that in this
> >>  case though :)
> >>
> >>  Regards
> >>  Steffen
> >>
> >>
> >>
> >>
> >>  On Thu, Jan 7, 2021 at 7:27 AM <renayama19661014 at ybb.ne.jp> wrote:
> >>  >
> >>  > Hi Steffen,
> >>  >
> >>  > The fix pointed out by Reid is affecting it.
> >>  >
> >>  > Since the fencing action requested by the DC node exists only in the
> > DC node, such an event occurs.
> >>  > You will need to take advantage of the modified pacemaker to resolve
> > the issue.
> >>  >
> >>  > Best Regards,
> >>  > Hideo Yamauchi.
> >>  >
> >>  >
> >>  >
> >>  > ----- Original Message -----
> >>  > > From: Reid Wahl <nwahl at redhat.com>
> >>  > > To: Cluster Labs - All topics related to open-source clustering
> > welcomed <users at clusterlabs.org>
> >>  > > Cc:
> >>  > > Date: 2021/1/7, Thu 15:07
> >>  > > Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs
> > status
> >>  > >
> >>  > > Hi, Steffen. Are your cluster nodes all running the same
> > Pacemaker
> >>  > > versions? This looks like Bug 5401[1], which is fixed by upstream
> >>  > > commit df71a07[2]. I'm a little bit confused about why it
> > only shows
> >>  > > up on one out of three nodes though.
> >>  > >
> >>  > > [1] https://bugs.clusterlabs.org/show_bug.cgi?id=5401
> >>  > > [2] https://github.com/ClusterLabs/pacemaker/commit/df71a07
> >>  > >
> >>  > > On Tue, Jan 5, 2021 at 8:31 AM Steffen Vinther Sørensen
> >>  > > <svinther at gmail.com> wrote:
> >>  > >>
> >>  > >>  Hello
> >>  > >>
> >>  > >>  node 1 is showing this in 'pcs status'
> >>  > >>
> >>  > >>  Pending Fencing Actions:
> >>  > >>  * reboot of kvm03-node02.avigol-gcs.dk pending:
> > client=crmd.37819,
> >>  > >>  origin=kvm03-node03.avigol-gcs.dk
> >>  > >>
> >>  > >>  node 2 and node 3 outputs no such thing (node 3 is DC)
> >>  > >>
> >>  > >>  Google is not much help, how to investigate this further and
> > get rid
> >>  > >>  of such terrifying status message ?
> >>  > >>
> >>  > >>  Regards
> >>  > >>  Steffen
> >>  > >>  _______________________________________________
> >>  > >>  Manage your subscription:
> >>  > >>  https://lists.clusterlabs.org/mailman/listinfo/users
> >>  > >>
> >>  > >>  ClusterLabs home: https://www.clusterlabs.org/
> >>  > >>
> >>  > >
> >>  > >
> >>  > > --
> >>  > > Regards,
> >>  > >
> >>  > > Reid Wahl, RHCA
> >>  > > Senior Software Maintenance Engineer, Red Hat
> >>  > > CEE - Platform Support Delivery - ClusterHA
> >>  > >
> >>  > > _______________________________________________
> >>  > > Manage your subscription:
> >>  > > https://lists.clusterlabs.org/mailman/listinfo/users
> >>  > >
> >>  > > ClusterLabs home: https://www.clusterlabs.org/
> >>  > >
> >>  >
> >>  > _______________________________________________
> >>  > Manage your subscription:
> >>  > https://lists.clusterlabs.org/mailman/listinfo/users
> >>  >
> >>  > ClusterLabs home: https://www.clusterlabs.org/
> >>  _______________________________________________
> >>  Manage your subscription:
> >>  https://lists.clusterlabs.org/mailman/listinfo/users
> >>
> >>  ClusterLabs home: https://www.clusterlabs.org/
> >
> >
> >
> > --
> > Regards,
> >
> > Reid Wahl, RHCA
> > Senior Software Maintenance Engineer, Red Hat
> > CEE - Platform Support Delivery - ClusterHA
> >
> > _______________________________________________
> > Manage your subscription:
> > https://lists.clusterlabs.org/mailman/listinfo/users
> >
> > ClusterLabs home: https://www.clusterlabs.org/
> >
>
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/
-------------- next part --------------
Cluster Name: kvm03
Corosync Nodes:
 kvm03-node01.avigol-gcs.dk kvm03-node02.avigol-gcs.dk kvm03-node03.avigol-gcs.dk
Pacemaker Nodes:
 kvm03-node01.avigol-gcs.dk kvm03-node02.avigol-gcs.dk kvm03-node03.avigol-gcs.dk

Resources:
 Clone: dlm-clone
  Meta Attrs: interleave=true ordered=true
  Resource: dlm (class=ocf provider=pacemaker type=controld)
   Operations: monitor interval=30s on-fail=fence (dlm-monitor-interval-30s)
               start interval=0s timeout=90 (dlm-start-interval-0s)
               stop interval=0s timeout=100 (dlm-stop-interval-0s)
 Clone: clvmd-clone
  Meta Attrs: interleave=true ordered=true
  Resource: clvmd (class=ocf provider=heartbeat type=clvm)
   Operations: monitor interval=30s on-fail=fence (clvmd-monitor-interval-30s)
               start interval=0s timeout=90s (clvmd-start-interval-0s)
               stop interval=0s timeout=90s (clvmd-stop-interval-0s)
 Clone: sharedfs01-clone
  Meta Attrs: interleave=true
  Resource: sharedfs01 (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/clustervg01/sharedfs01 directory=/usr/local/sharedfs01 fstype=gfs2 options=noatime,nodiratime,nobarrier
   Operations: monitor interval=10s on-fail=fence (sharedfs01-monitor-interval-10s)
               notify interval=0s timeout=60s (sharedfs01-notify-interval-0s)
               start interval=0s timeout=60s (sharedfs01-start-interval-0s)
               stop interval=0s timeout=60s (sharedfs01-stop-interval-0s)
 Clone: libvirtd-clone
  Meta Attrs: interleave=true
  Resource: libvirtd (class=systemd type=libvirtd)
   Operations: monitor interval=10s on-fail=fence (libvirtd-monitor-interval-10s)
               start interval=0s timeout=100 (libvirtd-start-interval-0s)
               stop interval=0s timeout=100 (libvirtd-stop-interval-0s)
 Clone: virtlockd-clone
  Meta Attrs: interleave=true
  Resource: virtlockd (class=systemd type=virtlockd)
   Operations: monitor interval=10s on-fail=fence (virtlockd-monitor-interval-10s)
               start interval=0s timeout=100 (virtlockd-start-interval-0s)
               stop interval=0s timeout=100 (virtlockd-stop-interval-0s)
 Clone: vmbootimages01-clone
  Meta Attrs: interleave=true
  Resource: vmbootimages01 (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/vmbootimages/vmbootimages01 directory=/usr/local/vmbootimages01 fstype=gfs2 options=noatime,nodiratime,nobarrier
   Operations: monitor interval=10s on-fail=fence (vmbootimages01-monitor-interval-10s)
               notify interval=0s timeout=60s (vmbootimages01-notify-interval-0s)
               start interval=0s timeout=60s (vmbootimages01-start-interval-0s)
               stop interval=0s timeout=60s (vmbootimages01-stop-interval-0s)
 Resource: dns-master (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/dns-master/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (dns-master-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (dns-master-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (dns-master-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (dns-master-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (dns-master-stop-interval-0s)
 Resource: dns-slave01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/dns-slave01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (dns-slave01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (dns-slave01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (dns-slave01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (dns-slave01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (dns-slave01-stop-interval-0s)
 Resource: httpdfrontend01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/httpdfrontend01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (httpdfrontend01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (httpdfrontend01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (httpdfrontend01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (httpdfrontend01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (httpdfrontend01-stop-interval-0s)
 Resource: httpdfrontend02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/httpdfrontend02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (httpdfrontend02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (httpdfrontend02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (httpdfrontend02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (httpdfrontend02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (httpdfrontend02-stop-interval-0s)
 Resource: highk29 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk29/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk29-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk29-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk29-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk29-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk29-stop-interval-0s)
 Resource: highk30 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk30/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk30-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk30-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk30-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk30-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk30-stop-interval-0s)
 Resource: highk31 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk31/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk31-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk31-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk31-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk31-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk31-stop-interval-0s)
 Resource: highk33 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk33/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk33-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk33-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk33-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk33-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk33-stop-interval-0s)
 Resource: highk34 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk34/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk34-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk34-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk34-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk34-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk34-stop-interval-0s)
 Resource: highk35 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk35/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk35-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk35-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk35-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk35-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk35-stop-interval-0s)
 Resource: stunnel01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/stunnel01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (stunnel01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (stunnel01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (stunnel01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (stunnel01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (stunnel01-stop-interval-0s)
 Resource: stunnel02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/stunnel02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (stunnel02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (stunnel02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (stunnel02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (stunnel02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (stunnel02-stop-interval-0s)
 Resource: crllists01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/crllists01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (crllists01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (crllists01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (crllists01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (crllists01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (crllists01-stop-interval-0s)
 Resource: crllists02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/crllists02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (crllists02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (crllists02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (crllists02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (crllists02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (crllists02-stop-interval-0s)
 Resource: antivirus01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus01-stop-interval-0s)
 Resource: antivirus02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus02-stop-interval-0s)
 Resource: antivirus03 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus03/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus03-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus03-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus03-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus03-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus03-stop-interval-0s)
 Resource: postfixrelay01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/postfixrelay01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (postfixrelay01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (postfixrelay01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (postfixrelay01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (postfixrelay01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (postfixrelay01-stop-interval-0s)
 Resource: fedora-fcos-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/fedora-fcos-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (fedora-fcos-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (fedora-fcos-demo01-stop-interval-0s)
 Resource: centos7-cloud-init-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/centos7-cloud-init-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (centos7-cloud-init-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (centos7-cloud-init-demo01-stop-interval-0s)
 Resource: centos7-virt-builder-docker-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/centos7-virt-builder-docker-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (centos7-virt-builder-docker-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (centos7-virt-builder-docker-demo01-stop-interval-0s)
 Resource: highk32 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk32/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk32-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk32-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk32-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk32-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk32-stop-interval-0s)

Stonith Devices:
 Resource: ipmi-fencing-node01 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node01-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node01.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node01-monitor-interval-60s)
 Resource: ipmi-fencing-node02 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node02-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node02.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node02-monitor-interval-60s)
 Resource: ipmi-fencing-node03 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node03-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node03.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node03-monitor-interval-60s)
Fencing Levels:

Location Constraints:
Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory) (id:order-dlm-clone-clvmd-clone-mandatory)
  start clvmd-clone then start sharedfs01-clone (kind:Mandatory) (id:order-clvmd-clone-sharedfs01-clone-mandatory)
  start sharedfs01-clone then start virtlockd-clone (kind:Mandatory) (id:order-sharedfs01-clone-virtlockd-clone-mandatory)
  start sharedfs01-clone then start libvirtd-clone (kind:Mandatory) (id:order-sharedfs01-clone-libvirtd-clone-mandatory)
  start virtlockd-clone then start libvirtd-clone (kind:Mandatory) (id:order-virtlockd-clone-libvirtd-clone-mandatory)
  start clvmd-clone then start vmbootimages01-clone (kind:Mandatory) (id:order-clvmd-clone-vmbootimages01-clone-mandatory)
  start vmbootimages01-clone then start virtlockd-clone (kind:Mandatory) (id:order-vmbootimages01-clone-virtlockd-clone-mandatory)
  start vmbootimages01-clone then start libvirtd-clone (kind:Mandatory) (id:order-vmbootimages01-clone-libvirtd-clone-mandatory)
  start libvirtd-clone then start dns-master (kind:Mandatory) (id:order-libvirtd-clone-dns-master-mandatory)
  start libvirtd-clone then start dns-slave01 (kind:Mandatory) (id:order-libvirtd-clone-dns-slave01-mandatory)
  start libvirtd-clone then start httpdfrontend01 (kind:Mandatory) (id:order-libvirtd-clone-httpdfrontend01-mandatory)
  start libvirtd-clone then start httpdfrontend02 (kind:Mandatory) (id:order-libvirtd-clone-httpdfrontend02-mandatory)
  start libvirtd-clone then start highk29 (kind:Mandatory) (id:order-libvirtd-clone-highk29-mandatory)
  start libvirtd-clone then start highk30 (kind:Mandatory) (id:order-libvirtd-clone-highk30-mandatory)
  start libvirtd-clone then start highk31 (kind:Mandatory) (id:order-libvirtd-clone-highk31-mandatory)
  start libvirtd-clone then start highk33 (kind:Mandatory) (id:order-libvirtd-clone-highk33-mandatory)
  start libvirtd-clone then start highk34 (kind:Mandatory) (id:order-libvirtd-clone-highk34-mandatory)
  start libvirtd-clone then start highk35 (kind:Mandatory) (id:order-libvirtd-clone-highk35-mandatory)
  start libvirtd-clone then start stunnel01 (kind:Mandatory) (id:order-libvirtd-clone-stunnel01-mandatory)
  start libvirtd-clone then start stunnel02 (kind:Mandatory) (id:order-libvirtd-clone-stunnel02-mandatory)
  start libvirtd-clone then start crllists01 (kind:Mandatory) (id:order-libvirtd-clone-crllists01-mandatory)
  start libvirtd-clone then start crllists02 (kind:Mandatory) (id:order-libvirtd-clone-crllists02-mandatory)
  start libvirtd-clone then start antivirus01 (kind:Mandatory) (id:order-libvirtd-clone-antivirus01-mandatory)
  start libvirtd-clone then start antivirus02 (kind:Mandatory) (id:order-libvirtd-clone-antivirus02-mandatory)
  start libvirtd-clone then start antivirus03 (kind:Mandatory) (id:order-libvirtd-clone-antivirus03-mandatory)
  start libvirtd-clone then start postfixrelay01 (kind:Mandatory) (id:order-libvirtd-clone-postfixrelay01-mandatory)
  start libvirtd-clone then start fedora-fcos-demo01 (kind:Mandatory) (id:order-libvirtd-clone-fedora-fcos-demo01-mandatory)
  start libvirtd-clone then start centos7-cloud-init-demo01 (kind:Mandatory) (id:order-libvirtd-clone-centos7-cloud-init-demo01-mandatory)
  start libvirtd-clone then start centos7-virt-builder-docker-demo01 (kind:Mandatory) (id:order-libvirtd-clone-centos7-virt-builder-docker-demo01-mandatory)
  start libvirtd-clone then start highk32 (kind:Mandatory) (id:order-libvirtd-clone-highk32-mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY) (id:colocation-clvmd-clone-dlm-clone-INFINITY)
  sharedfs01-clone with clvmd-clone (score:INFINITY) (id:colocation-sharedfs01-clone-clvmd-clone-INFINITY)
  vmbootimages01-clone with clvmd-clone (score:INFINITY) (id:colocation-vmbootimages01-clone-clvmd-clone-INFINITY)
Ticket Constraints:

Alerts:
 No alerts defined

Resources Defaults:
 resource-stickiness=100
Operations Defaults:
 No defaults set

Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: kvm03
 dc-version: 1.1.23-1.el7_9.1-9acf116022
 have-watchdog: false
 last-lrm-refresh: 1609784788
 migration-limit: 2
 no-quorum-policy: freeze
 stonith-enabled: true

Quorum:
  Options:
-------------- next part --------------
Cluster Name: kvm03
Corosync Nodes:
 kvm03-node01.avigol-gcs.dk kvm03-node02.avigol-gcs.dk kvm03-node03.avigol-gcs.dk
Pacemaker Nodes:
 kvm03-node01.avigol-gcs.dk kvm03-node02.avigol-gcs.dk kvm03-node03.avigol-gcs.dk

Resources:
 Clone: dlm-clone
  Meta Attrs: interleave=true ordered=true
  Resource: dlm (class=ocf provider=pacemaker type=controld)
   Operations: monitor interval=30s on-fail=fence (dlm-monitor-interval-30s)
               start interval=0s timeout=90 (dlm-start-interval-0s)
               stop interval=0s timeout=100 (dlm-stop-interval-0s)
 Clone: clvmd-clone
  Meta Attrs: interleave=true ordered=true
  Resource: clvmd (class=ocf provider=heartbeat type=clvm)
   Operations: monitor interval=30s on-fail=fence (clvmd-monitor-interval-30s)
               start interval=0s timeout=90s (clvmd-start-interval-0s)
               stop interval=0s timeout=90s (clvmd-stop-interval-0s)
 Clone: sharedfs01-clone
  Meta Attrs: interleave=true
  Resource: sharedfs01 (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/clustervg01/sharedfs01 directory=/usr/local/sharedfs01 fstype=gfs2 options=noatime,nodiratime,nobarrier
   Operations: monitor interval=10s on-fail=fence (sharedfs01-monitor-interval-10s)
               notify interval=0s timeout=60s (sharedfs01-notify-interval-0s)
               start interval=0s timeout=60s (sharedfs01-start-interval-0s)
               stop interval=0s timeout=60s (sharedfs01-stop-interval-0s)
 Clone: libvirtd-clone
  Meta Attrs: interleave=true
  Resource: libvirtd (class=systemd type=libvirtd)
   Operations: monitor interval=10s on-fail=fence (libvirtd-monitor-interval-10s)
               start interval=0s timeout=100 (libvirtd-start-interval-0s)
               stop interval=0s timeout=100 (libvirtd-stop-interval-0s)
 Clone: virtlockd-clone
  Meta Attrs: interleave=true
  Resource: virtlockd (class=systemd type=virtlockd)
   Operations: monitor interval=10s on-fail=fence (virtlockd-monitor-interval-10s)
               start interval=0s timeout=100 (virtlockd-start-interval-0s)
               stop interval=0s timeout=100 (virtlockd-stop-interval-0s)
 Clone: vmbootimages01-clone
  Meta Attrs: interleave=true
  Resource: vmbootimages01 (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/vmbootimages/vmbootimages01 directory=/usr/local/vmbootimages01 fstype=gfs2 options=noatime,nodiratime,nobarrier
   Operations: monitor interval=10s on-fail=fence (vmbootimages01-monitor-interval-10s)
               notify interval=0s timeout=60s (vmbootimages01-notify-interval-0s)
               start interval=0s timeout=60s (vmbootimages01-start-interval-0s)
               stop interval=0s timeout=60s (vmbootimages01-stop-interval-0s)
 Resource: dns-master (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/dns-master/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (dns-master-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (dns-master-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (dns-master-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (dns-master-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (dns-master-stop-interval-0s)
 Resource: dns-slave01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/dns-slave01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (dns-slave01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (dns-slave01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (dns-slave01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (dns-slave01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (dns-slave01-stop-interval-0s)
 Resource: httpdfrontend01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/httpdfrontend01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (httpdfrontend01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (httpdfrontend01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (httpdfrontend01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (httpdfrontend01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (httpdfrontend01-stop-interval-0s)
 Resource: httpdfrontend02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/httpdfrontend02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (httpdfrontend02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (httpdfrontend02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (httpdfrontend02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (httpdfrontend02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (httpdfrontend02-stop-interval-0s)
 Resource: highk29 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk29/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk29-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk29-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk29-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk29-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk29-stop-interval-0s)
 Resource: highk30 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk30/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk30-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk30-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk30-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk30-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk30-stop-interval-0s)
 Resource: highk31 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk31/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk31-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk31-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk31-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk31-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk31-stop-interval-0s)
 Resource: highk33 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk33/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk33-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk33-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk33-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk33-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk33-stop-interval-0s)
 Resource: highk34 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk34/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk34-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk34-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk34-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk34-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk34-stop-interval-0s)
 Resource: highk35 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk35/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk35-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk35-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk35-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk35-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk35-stop-interval-0s)
 Resource: stunnel01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/stunnel01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (stunnel01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (stunnel01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (stunnel01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (stunnel01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (stunnel01-stop-interval-0s)
 Resource: stunnel02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/stunnel02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (stunnel02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (stunnel02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (stunnel02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (stunnel02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (stunnel02-stop-interval-0s)
 Resource: crllists01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/crllists01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (crllists01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (crllists01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (crllists01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (crllists01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (crllists01-stop-interval-0s)
 Resource: crllists02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/crllists02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (crllists02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (crllists02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (crllists02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (crllists02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (crllists02-stop-interval-0s)
 Resource: antivirus01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus01-stop-interval-0s)
 Resource: antivirus02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus02-stop-interval-0s)
 Resource: antivirus03 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus03/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus03-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus03-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus03-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus03-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus03-stop-interval-0s)
 Resource: postfixrelay01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/postfixrelay01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (postfixrelay01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (postfixrelay01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (postfixrelay01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (postfixrelay01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (postfixrelay01-stop-interval-0s)
 Resource: fedora-fcos-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/fedora-fcos-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (fedora-fcos-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (fedora-fcos-demo01-stop-interval-0s)
 Resource: centos7-cloud-init-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/centos7-cloud-init-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (centos7-cloud-init-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (centos7-cloud-init-demo01-stop-interval-0s)
 Resource: centos7-virt-builder-docker-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/centos7-virt-builder-docker-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (centos7-virt-builder-docker-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (centos7-virt-builder-docker-demo01-stop-interval-0s)
 Resource: highk32 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk32/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk32-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk32-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk32-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk32-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk32-stop-interval-0s)

Stonith Devices:
 Resource: ipmi-fencing-node01 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node01-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node01.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node01-monitor-interval-60s)
 Resource: ipmi-fencing-node02 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node02-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node02.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node02-monitor-interval-60s)
 Resource: ipmi-fencing-node03 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node03-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node03.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node03-monitor-interval-60s)
Fencing Levels:

Location Constraints:
Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory) (id:order-dlm-clone-clvmd-clone-mandatory)
  start clvmd-clone then start sharedfs01-clone (kind:Mandatory) (id:order-clvmd-clone-sharedfs01-clone-mandatory)
  start sharedfs01-clone then start virtlockd-clone (kind:Mandatory) (id:order-sharedfs01-clone-virtlockd-clone-mandatory)
  start sharedfs01-clone then start libvirtd-clone (kind:Mandatory) (id:order-sharedfs01-clone-libvirtd-clone-mandatory)
  start virtlockd-clone then start libvirtd-clone (kind:Mandatory) (id:order-virtlockd-clone-libvirtd-clone-mandatory)
  start clvmd-clone then start vmbootimages01-clone (kind:Mandatory) (id:order-clvmd-clone-vmbootimages01-clone-mandatory)
  start vmbootimages01-clone then start virtlockd-clone (kind:Mandatory) (id:order-vmbootimages01-clone-virtlockd-clone-mandatory)
  start vmbootimages01-clone then start libvirtd-clone (kind:Mandatory) (id:order-vmbootimages01-clone-libvirtd-clone-mandatory)
  start libvirtd-clone then start dns-master (kind:Mandatory) (id:order-libvirtd-clone-dns-master-mandatory)
  start libvirtd-clone then start dns-slave01 (kind:Mandatory) (id:order-libvirtd-clone-dns-slave01-mandatory)
  start libvirtd-clone then start httpdfrontend01 (kind:Mandatory) (id:order-libvirtd-clone-httpdfrontend01-mandatory)
  start libvirtd-clone then start httpdfrontend02 (kind:Mandatory) (id:order-libvirtd-clone-httpdfrontend02-mandatory)
  start libvirtd-clone then start highk29 (kind:Mandatory) (id:order-libvirtd-clone-highk29-mandatory)
  start libvirtd-clone then start highk30 (kind:Mandatory) (id:order-libvirtd-clone-highk30-mandatory)
  start libvirtd-clone then start highk31 (kind:Mandatory) (id:order-libvirtd-clone-highk31-mandatory)
  start libvirtd-clone then start highk33 (kind:Mandatory) (id:order-libvirtd-clone-highk33-mandatory)
  start libvirtd-clone then start highk34 (kind:Mandatory) (id:order-libvirtd-clone-highk34-mandatory)
  start libvirtd-clone then start highk35 (kind:Mandatory) (id:order-libvirtd-clone-highk35-mandatory)
  start libvirtd-clone then start stunnel01 (kind:Mandatory) (id:order-libvirtd-clone-stunnel01-mandatory)
  start libvirtd-clone then start stunnel02 (kind:Mandatory) (id:order-libvirtd-clone-stunnel02-mandatory)
  start libvirtd-clone then start crllists01 (kind:Mandatory) (id:order-libvirtd-clone-crllists01-mandatory)
  start libvirtd-clone then start crllists02 (kind:Mandatory) (id:order-libvirtd-clone-crllists02-mandatory)
  start libvirtd-clone then start antivirus01 (kind:Mandatory) (id:order-libvirtd-clone-antivirus01-mandatory)
  start libvirtd-clone then start antivirus02 (kind:Mandatory) (id:order-libvirtd-clone-antivirus02-mandatory)
  start libvirtd-clone then start antivirus03 (kind:Mandatory) (id:order-libvirtd-clone-antivirus03-mandatory)
  start libvirtd-clone then start postfixrelay01 (kind:Mandatory) (id:order-libvirtd-clone-postfixrelay01-mandatory)
  start libvirtd-clone then start fedora-fcos-demo01 (kind:Mandatory) (id:order-libvirtd-clone-fedora-fcos-demo01-mandatory)
  start libvirtd-clone then start centos7-cloud-init-demo01 (kind:Mandatory) (id:order-libvirtd-clone-centos7-cloud-init-demo01-mandatory)
  start libvirtd-clone then start centos7-virt-builder-docker-demo01 (kind:Mandatory) (id:order-libvirtd-clone-centos7-virt-builder-docker-demo01-mandatory)
  start libvirtd-clone then start highk32 (kind:Mandatory) (id:order-libvirtd-clone-highk32-mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY) (id:colocation-clvmd-clone-dlm-clone-INFINITY)
  sharedfs01-clone with clvmd-clone (score:INFINITY) (id:colocation-sharedfs01-clone-clvmd-clone-INFINITY)
  vmbootimages01-clone with clvmd-clone (score:INFINITY) (id:colocation-vmbootimages01-clone-clvmd-clone-INFINITY)
Ticket Constraints:

Alerts:
 No alerts defined

Resources Defaults:
 resource-stickiness=100
Operations Defaults:
 No defaults set

Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: kvm03
 dc-version: 1.1.23-1.el7_9.1-9acf116022
 have-watchdog: false
 last-lrm-refresh: 1609784788
 migration-limit: 2
 no-quorum-policy: freeze
 stonith-enabled: true

Quorum:
  Options:
-------------- next part --------------
Cluster Name: kvm03
Corosync Nodes:
 kvm03-node01.avigol-gcs.dk kvm03-node02.avigol-gcs.dk kvm03-node03.avigol-gcs.dk
Pacemaker Nodes:
 kvm03-node01.avigol-gcs.dk kvm03-node02.avigol-gcs.dk kvm03-node03.avigol-gcs.dk

Resources:
 Clone: dlm-clone
  Meta Attrs: interleave=true ordered=true
  Resource: dlm (class=ocf provider=pacemaker type=controld)
   Operations: monitor interval=30s on-fail=fence (dlm-monitor-interval-30s)
               start interval=0s timeout=90 (dlm-start-interval-0s)
               stop interval=0s timeout=100 (dlm-stop-interval-0s)
 Clone: clvmd-clone
  Meta Attrs: interleave=true ordered=true
  Resource: clvmd (class=ocf provider=heartbeat type=clvm)
   Operations: monitor interval=30s on-fail=fence (clvmd-monitor-interval-30s)
               start interval=0s timeout=90s (clvmd-start-interval-0s)
               stop interval=0s timeout=90s (clvmd-stop-interval-0s)
 Clone: sharedfs01-clone
  Meta Attrs: interleave=true
  Resource: sharedfs01 (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/clustervg01/sharedfs01 directory=/usr/local/sharedfs01 fstype=gfs2 options=noatime,nodiratime,nobarrier
   Operations: monitor interval=10s on-fail=fence (sharedfs01-monitor-interval-10s)
               notify interval=0s timeout=60s (sharedfs01-notify-interval-0s)
               start interval=0s timeout=60s (sharedfs01-start-interval-0s)
               stop interval=0s timeout=60s (sharedfs01-stop-interval-0s)
 Clone: libvirtd-clone
  Meta Attrs: interleave=true
  Resource: libvirtd (class=systemd type=libvirtd)
   Operations: monitor interval=10s on-fail=fence (libvirtd-monitor-interval-10s)
               start interval=0s timeout=100 (libvirtd-start-interval-0s)
               stop interval=0s timeout=100 (libvirtd-stop-interval-0s)
 Clone: virtlockd-clone
  Meta Attrs: interleave=true
  Resource: virtlockd (class=systemd type=virtlockd)
   Operations: monitor interval=10s on-fail=fence (virtlockd-monitor-interval-10s)
               start interval=0s timeout=100 (virtlockd-start-interval-0s)
               stop interval=0s timeout=100 (virtlockd-stop-interval-0s)
 Clone: vmbootimages01-clone
  Meta Attrs: interleave=true
  Resource: vmbootimages01 (class=ocf provider=heartbeat type=Filesystem)
   Attributes: device=/dev/vmbootimages/vmbootimages01 directory=/usr/local/vmbootimages01 fstype=gfs2 options=noatime,nodiratime,nobarrier
   Operations: monitor interval=10s on-fail=fence (vmbootimages01-monitor-interval-10s)
               notify interval=0s timeout=60s (vmbootimages01-notify-interval-0s)
               start interval=0s timeout=60s (vmbootimages01-start-interval-0s)
               stop interval=0s timeout=60s (vmbootimages01-stop-interval-0s)
 Resource: dns-master (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/dns-master/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (dns-master-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (dns-master-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (dns-master-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (dns-master-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (dns-master-stop-interval-0s)
 Resource: dns-slave01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/dns-slave01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (dns-slave01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (dns-slave01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (dns-slave01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (dns-slave01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (dns-slave01-stop-interval-0s)
 Resource: httpdfrontend01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/httpdfrontend01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (httpdfrontend01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (httpdfrontend01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (httpdfrontend01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (httpdfrontend01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (httpdfrontend01-stop-interval-0s)
 Resource: httpdfrontend02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/httpdfrontend02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (httpdfrontend02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (httpdfrontend02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (httpdfrontend02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (httpdfrontend02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (httpdfrontend02-stop-interval-0s)
 Resource: highk29 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk29/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk29-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk29-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk29-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk29-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk29-stop-interval-0s)
 Resource: highk30 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk30/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk30-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk30-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk30-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk30-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk30-stop-interval-0s)
 Resource: highk31 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk31/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk31-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk31-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk31-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk31-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk31-stop-interval-0s)
 Resource: highk33 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk33/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk33-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk33-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk33-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk33-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk33-stop-interval-0s)
 Resource: highk34 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk34/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk34-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk34-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk34-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk34-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk34-stop-interval-0s)
 Resource: highk35 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk35/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk35-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk35-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk35-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk35-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk35-stop-interval-0s)
 Resource: stunnel01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/stunnel01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (stunnel01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (stunnel01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (stunnel01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (stunnel01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (stunnel01-stop-interval-0s)
 Resource: stunnel02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/stunnel02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (stunnel02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (stunnel02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (stunnel02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (stunnel02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (stunnel02-stop-interval-0s)
 Resource: crllists01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/crllists01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (crllists01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (crllists01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (crllists01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (crllists01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (crllists01-stop-interval-0s)
 Resource: crllists02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/crllists02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (crllists02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (crllists02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (crllists02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (crllists02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (crllists02-stop-interval-0s)
 Resource: antivirus01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus01-stop-interval-0s)
 Resource: antivirus02 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus02/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus02-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus02-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus02-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus02-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus02-stop-interval-0s)
 Resource: antivirus03 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/antivirus03/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (antivirus03-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (antivirus03-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (antivirus03-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (antivirus03-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (antivirus03-stop-interval-0s)
 Resource: postfixrelay01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/postfixrelay01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (postfixrelay01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (postfixrelay01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (postfixrelay01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (postfixrelay01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (postfixrelay01-stop-interval-0s)
 Resource: fedora-fcos-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/fedora-fcos-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (fedora-fcos-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (fedora-fcos-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (fedora-fcos-demo01-stop-interval-0s)
 Resource: centos7-cloud-init-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/centos7-cloud-init-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (centos7-cloud-init-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (centos7-cloud-init-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (centos7-cloud-init-demo01-stop-interval-0s)
 Resource: centos7-virt-builder-docker-demo01 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/centos7-virt-builder-docker-demo01/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (centos7-virt-builder-docker-demo01-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (centos7-virt-builder-docker-demo01-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (centos7-virt-builder-docker-demo01-stop-interval-0s)
 Resource: highk32 (class=ocf provider=heartbeat type=VirtualDomain)
  Attributes: config=/usr/local/vmbootimages01/vmdefinitions/highk32/vm.xml hypervisor=qemu:///system migration_transport=ssh
  Meta Attrs: allow-migrate=true
  Operations: migrate_from interval=0s on-fail=block timeout=120s (highk32-migrate_from-interval-0s)
              migrate_to interval=0s on-fail=block timeout=120s (highk32-migrate_to-interval-0s)
              monitor interval=10s on-fail=block timeout=30s (highk32-monitor-interval-10s)
              start interval=0s on-fail=block timeout=120s (highk32-start-interval-0s)
              stop interval=0s on-fail=block timeout=180s (highk32-stop-interval-0s)

Stonith Devices:
 Resource: ipmi-fencing-node01 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node01-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node01.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node01-monitor-interval-60s)
 Resource: ipmi-fencing-node02 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node02-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node02.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node02-monitor-interval-60s)
 Resource: ipmi-fencing-node03 (class=stonith type=fence_ipmilan)
  Attributes: ipaddr=kvm03-node03-console.avigol-gcs.dk login=admin passwd=admin pcmk_host_check=static-list pcmk_host_list=kvm03-node03.avigol-gcs.dk
  Operations: monitor interval=60s (ipmi-fencing-node03-monitor-interval-60s)
Fencing Levels:

Location Constraints:
Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory) (id:order-dlm-clone-clvmd-clone-mandatory)
  start clvmd-clone then start sharedfs01-clone (kind:Mandatory) (id:order-clvmd-clone-sharedfs01-clone-mandatory)
  start sharedfs01-clone then start virtlockd-clone (kind:Mandatory) (id:order-sharedfs01-clone-virtlockd-clone-mandatory)
  start sharedfs01-clone then start libvirtd-clone (kind:Mandatory) (id:order-sharedfs01-clone-libvirtd-clone-mandatory)
  start virtlockd-clone then start libvirtd-clone (kind:Mandatory) (id:order-virtlockd-clone-libvirtd-clone-mandatory)
  start clvmd-clone then start vmbootimages01-clone (kind:Mandatory) (id:order-clvmd-clone-vmbootimages01-clone-mandatory)
  start vmbootimages01-clone then start virtlockd-clone (kind:Mandatory) (id:order-vmbootimages01-clone-virtlockd-clone-mandatory)
  start vmbootimages01-clone then start libvirtd-clone (kind:Mandatory) (id:order-vmbootimages01-clone-libvirtd-clone-mandatory)
  start libvirtd-clone then start dns-master (kind:Mandatory) (id:order-libvirtd-clone-dns-master-mandatory)
  start libvirtd-clone then start dns-slave01 (kind:Mandatory) (id:order-libvirtd-clone-dns-slave01-mandatory)
  start libvirtd-clone then start httpdfrontend01 (kind:Mandatory) (id:order-libvirtd-clone-httpdfrontend01-mandatory)
  start libvirtd-clone then start httpdfrontend02 (kind:Mandatory) (id:order-libvirtd-clone-httpdfrontend02-mandatory)
  start libvirtd-clone then start highk29 (kind:Mandatory) (id:order-libvirtd-clone-highk29-mandatory)
  start libvirtd-clone then start highk30 (kind:Mandatory) (id:order-libvirtd-clone-highk30-mandatory)
  start libvirtd-clone then start highk31 (kind:Mandatory) (id:order-libvirtd-clone-highk31-mandatory)
  start libvirtd-clone then start highk33 (kind:Mandatory) (id:order-libvirtd-clone-highk33-mandatory)
  start libvirtd-clone then start highk34 (kind:Mandatory) (id:order-libvirtd-clone-highk34-mandatory)
  start libvirtd-clone then start highk35 (kind:Mandatory) (id:order-libvirtd-clone-highk35-mandatory)
  start libvirtd-clone then start stunnel01 (kind:Mandatory) (id:order-libvirtd-clone-stunnel01-mandatory)
  start libvirtd-clone then start stunnel02 (kind:Mandatory) (id:order-libvirtd-clone-stunnel02-mandatory)
  start libvirtd-clone then start crllists01 (kind:Mandatory) (id:order-libvirtd-clone-crllists01-mandatory)
  start libvirtd-clone then start crllists02 (kind:Mandatory) (id:order-libvirtd-clone-crllists02-mandatory)
  start libvirtd-clone then start antivirus01 (kind:Mandatory) (id:order-libvirtd-clone-antivirus01-mandatory)
  start libvirtd-clone then start antivirus02 (kind:Mandatory) (id:order-libvirtd-clone-antivirus02-mandatory)
  start libvirtd-clone then start antivirus03 (kind:Mandatory) (id:order-libvirtd-clone-antivirus03-mandatory)
  start libvirtd-clone then start postfixrelay01 (kind:Mandatory) (id:order-libvirtd-clone-postfixrelay01-mandatory)
  start libvirtd-clone then start fedora-fcos-demo01 (kind:Mandatory) (id:order-libvirtd-clone-fedora-fcos-demo01-mandatory)
  start libvirtd-clone then start centos7-cloud-init-demo01 (kind:Mandatory) (id:order-libvirtd-clone-centos7-cloud-init-demo01-mandatory)
  start libvirtd-clone then start centos7-virt-builder-docker-demo01 (kind:Mandatory) (id:order-libvirtd-clone-centos7-virt-builder-docker-demo01-mandatory)
  start libvirtd-clone then start highk32 (kind:Mandatory) (id:order-libvirtd-clone-highk32-mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY) (id:colocation-clvmd-clone-dlm-clone-INFINITY)
  sharedfs01-clone with clvmd-clone (score:INFINITY) (id:colocation-sharedfs01-clone-clvmd-clone-INFINITY)
  vmbootimages01-clone with clvmd-clone (score:INFINITY) (id:colocation-vmbootimages01-clone-clvmd-clone-INFINITY)
Ticket Constraints:

Alerts:
 No alerts defined

Resources Defaults:
 resource-stickiness=100
Operations Defaults:
 No defaults set

Cluster Properties:
 cluster-infrastructure: corosync
 cluster-name: kvm03
 dc-version: 1.1.23-1.el7_9.1-9acf116022
 have-watchdog: false
 last-lrm-refresh: 1609784788
 migration-limit: 2
 no-quorum-policy: freeze
 stonith-enabled: true

Quorum:
  Options:


More information about the Users mailing list