[ClusterLabs] Pending Fencing Actions shown in pcs status
renayama19661014 at ybb.ne.jp
renayama19661014 at ybb.ne.jp
Thu Jan 7 03:54:25 EST 2021
Hi Steffen,
> Unfortunately not sure about the exact scenario. But I have been doing
> some recent experiments with node standby/unstandby stop/start. This
> to get procedures right for updating node rpms etc.
>
> Later I noticed the uncomforting "pending fencing actions" status msg.
Okay!
Repeat the standby and unstandby steps in the same way to check.
We will start checking after tomorrow, so I think it will take some time until next week.
Many thanks,
Hideo Yamauchi.
----- Original Message -----
> From: "renayama19661014 at ybb.ne.jp" <renayama19661014 at ybb.ne.jp>
> To: Reid Wahl <nwahl at redhat.com>; Cluster Labs - All topics related to open-source clustering welcomed <users at clusterlabs.org>
> Cc:
> Date: 2021/1/7, Thu 17:51
> Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs status
>
> Hi Steffen,
> Hi Reid,
>
> The fencing history is kept inside stonith-ng and is not written to cib.
> However, getting the entire cib and getting it sent will help you to reproduce
> the problem.
>
> Best Regards,
> Hideo Yamauchi.
>
>
> ----- Original Message -----
>> From: Reid Wahl <nwahl at redhat.com>
>> To: renayama19661014 at ybb.ne.jp; Cluster Labs - All topics related to
> open-source clustering welcomed <users at clusterlabs.org>
>> Date: 2021/1/7, Thu 17:39
>> Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs status
>>
>>
>> Hi, Steffen. Those attachments don't contain the CIB. They contain the
> `pcs config` output. You can get the cib with `pcs cluster cib >
> $(hostname).cib.xml`.
>>
>>
>> Granted, it's possible that this fence action information wouldn't
> be in the CIB at all. It might be stored in fencer memory.
>>
>>
>> On Thu, Jan 7, 2021 at 12:26 AM <renayama19661014 at ybb.ne.jp> wrote:
>>
>> Hi Steffen,
>>>
>>>> Here CIB settings attached (pcs config show) for all 3 of my nodes
>>>> (all 3 seems 100% identical), node03 is the DC.
>>>
>>>
>>> Thank you for the attachment.
>>>
>>> What is the scenario when this situation occurs?
>>> In what steps did the problem appear when fencing was performed (or
> failed)?
>>>
>>>
>>> Best Regards,
>>> Hideo Yamauchi.
>>>
>>>
>>> ----- Original Message -----
>>>> From: Steffen Vinther Sørensen <svinther at gmail.com>
>>>> To: renayama19661014 at ybb.ne.jp; Cluster Labs - All topics related
> to open-source clustering welcomed <users at clusterlabs.org>
>>>> Cc:
>>>> Date: 2021/1/7, Thu 17:05
>>>> Subject: Re: [ClusterLabs] Pending Fencing Actions shown in pcs
> status
>>>>
>>>> Hi Hideo,
>>>>
>>>> Here CIB settings attached (pcs config show) for all 3 of my nodes
>>>> (all 3 seems 100% identical), node03 is the DC.
>>>>
>>>> Regards
>>>> Steffen
>>>>
>>>> On Thu, Jan 7, 2021 at 8:06 AM <renayama19661014 at ybb.ne.jp>
> wrote:
>>>>>
>>>>> Hi Steffen,
>>>>> Hi Reid,
>>>>>
>>>>> I also checked the Centos source rpm and it seems to include a
> fix for the
>>>> problem.
>>>>>
>>>>> As Steffen suggested, if you share your CIB settings, I might
> know
>>>> something.
>>>>>
>>>>> If this issue is the same as the fix, the display will only be
> displayed on
>>>> the DC node and will not affect the operation.
>>>>> The pending actions shown will remain for a long time, but
> will not have a
>>>> negative impact on the cluster.
>>>>>
>>>>> Best Regards,
>>>>> Hideo Yamauchi.
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> > From: Reid Wahl <nwahl at redhat.com>
>>>>> > To: Cluster Labs - All topics related to open-source
> clustering
>>>> welcomed <users at clusterlabs.org>
>>>>> > Cc:
>>>>> > Date: 2021/1/7, Thu 15:58
>>>>> > Subject: Re: [ClusterLabs] Pending Fencing Actions shown
> in pcs status
>>>>> >
>>>>> > It's supposedly fixed in that version.
>>>>> > - https://bugzilla.redhat.com/show_bug.cgi?id=1787749
>>>>> > - https://access.redhat.com/solutions/4713471
>>>>> >
>>>>> > So you may be hitting a different issue (unless
> there's a bug in
>>>> the
>>>>> > pcmk 1.1 backport of the fix).
>>>>> >
>>>>> > I may be a little bit out of my area of knowledge here,
> but can you
>>>>> > share the CIBs from nodes 1 and 3? Maybe Hideo, Klaus, or
> Ken has some
>>>>> > insight.
>>>>> >
>>>>> > On Wed, Jan 6, 2021 at 10:53 PM Steffen Vinther Sørensen
>>>>> > <svinther at gmail.com> wrote:
>>>>> >>
>>>>> >> Hi Hideo,
>>>>> >>
>>>>> >> If the fix is not going to make it into the CentOS7
> pacemaker
>>>> version,
>>>>> >> I guess the stable approach to take advantage of it
> is to build
>>>> the
>>>>> >> cluster on another OS than CentOS7 ? A little late
> for that in
>>>> this
>>>>> >> case though :)
>>>>> >>
>>>>> >> Regards
>>>>> >> Steffen
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Thu, Jan 7, 2021 at 7:27 AM
> <renayama19661014 at ybb.ne.jp>
>>>> wrote:
>>>>> >> >
>>>>> >> > Hi Steffen,
>>>>> >> >
>>>>> >> > The fix pointed out by Reid is affecting it.
>>>>> >> >
>>>>> >> > Since the fencing action requested by the DC
> node exists
>>>> only in the
>>>>> > DC node, such an event occurs.
>>>>> >> > You will need to take advantage of the modified
> pacemaker to
>>>> resolve
>>>>> > the issue.
>>>>> >> >
>>>>> >> > Best Regards,
>>>>> >> > Hideo Yamauchi.
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > ----- Original Message -----
>>>>> >> > > From: Reid Wahl <nwahl at redhat.com>
>>>>> >> > > To: Cluster Labs - All topics related to
> open-source
>>>> clustering
>>>>> > welcomed <users at clusterlabs.org>
>>>>> >> > > Cc:
>>>>> >> > > Date: 2021/1/7, Thu 15:07
>>>>> >> > > Subject: Re: [ClusterLabs] Pending Fencing
> Actions
>>>> shown in pcs
>>>>> > status
>>>>> >> > >
>>>>> >> > > Hi, Steffen. Are your cluster nodes all
> running the
>>>> same
>>>>> > Pacemaker
>>>>> >> > > versions? This looks like Bug 5401[1],
> which is fixed
>>>> by upstream
>>>>> >> > > commit df71a07[2]. I'm a little bit
> confused about
>>>> why it
>>>>> > only shows
>>>>> >> > > up on one out of three nodes though.
>>>>> >> > >
>>>>> >> > > [1]
> https://bugs.clusterlabs.org/show_bug.cgi?id=5401
>>>>> >> > > [2]
>>>> https://github.com/ClusterLabs/pacemaker/commit/df71a07
>>>>> >> > >
>>>>> >> > > On Tue, Jan 5, 2021 at 8:31 AM Steffen
> Vinther Sørensen
>>>>> >> > > <svinther at gmail.com> wrote:
>>>>> >> > >>
>>>>> >> > >> Hello
>>>>> >> > >>
>>>>> >> > >> node 1 is showing this in 'pcs
> status'
>>>>> >> > >>
>>>>> >> > >> Pending Fencing Actions:
>>>>> >> > >> * reboot of
> kvm03-node02.avigol-gcs.dk pending:
>>>>> > client=crmd.37819,
>>>>> >> > >> origin=kvm03-node03.avigol-gcs.dk
>>>>> >> > >>
>>>>> >> > >> node 2 and node 3 outputs no such
> thing (node 3 is
>>>> DC)
>>>>> >> > >>
>>>>> >> > >> Google is not much help, how to
> investigate this
>>>> further and
>>>>> > get rid
>>>>> >> > >> of such terrifying status message ?
>>>>> >> > >>
>>>>> >> > >> Regards
>>>>> >> > >> Steffen
>>>>> >> > >>
> _______________________________________________
>>>>> >> > >> Manage your subscription:
>>>>> >> > >>
>>>> https://lists.clusterlabs.org/mailman/listinfo/users
>>>>> >> > >>
>>>>> >> > >> ClusterLabs home:
> https://www.clusterlabs.org/
>>>>> >> > >>
>>>>> >> > >
>>>>> >> > >
>>>>> >> > > --
>>>>> >> > > Regards,
>>>>> >> > >
>>>>> >> > > Reid Wahl, RHCA
>>>>> >> > > Senior Software Maintenance Engineer, Red
> Hat
>>>>> >> > > CEE - Platform Support Delivery -
> ClusterHA
>>>>> >> > >
>>>>> >> > >
> _______________________________________________
>>>>> >> > > Manage your subscription:
>>>>> >> > >
> https://lists.clusterlabs.org/mailman/listinfo/users
>>>>> >> > >
>>>>> >> > > ClusterLabs home:
> https://www.clusterlabs.org/
>>>>> >> > >
>>>>> >> >
>>>>> >> > _______________________________________________
>>>>> >> > Manage your subscription:
>>>>> >> >
> https://lists.clusterlabs.org/mailman/listinfo/users
>>>>> >> >
>>>>> >> > ClusterLabs home: https://www.clusterlabs.org/
>>>>> >> _______________________________________________
>>>>> >> Manage your subscription:
>>>>> >> https://lists.clusterlabs.org/mailman/listinfo/users
>
>>>>> >>
>>>>> >> ClusterLabs home: https://www.clusterlabs.org/
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Regards,
>>>>> >
>>>>> > Reid Wahl, RHCA
>>>>> > Senior Software Maintenance Engineer, Red Hat
>>>>> > CEE - Platform Support Delivery - ClusterHA
>>>>> >
>>>>> > _______________________________________________
>>>>> > Manage your subscription:
>>>>> > https://lists.clusterlabs.org/mailman/listinfo/users
>>>>> >
>>>>> > ClusterLabs home: https://www.clusterlabs.org/
>>>>> >
>>>>>
>>>>> _______________________________________________
>>>>> Manage your subscription:
>>>>> https://lists.clusterlabs.org/mailman/listinfo/users
>>>>>
>>>>> ClusterLabs home: https://www.clusterlabs.org/
>>>>
>>>
>>> _______________________________________________
>>> Manage your subscription:
>>> https://lists.clusterlabs.org/mailman/listinfo/users
>>>
>>> ClusterLabs home: https://www.clusterlabs.org/
>>>
>>
>> --
>>
>> Regards,
>>
>> Reid Wahl, RHCA
>>
>> Senior Software Maintenance Engineer, Red Hat
>> CEE - Platform Support Delivery - ClusterHA
>>
>>
>
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
>
> ClusterLabs home: https://www.clusterlabs.org/
>
More information about the Users
mailing list