<div dir="ltr">Illumos might have getpeerucred, which can also set errno to ENOTSUP.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jul 26, 2020 at 3:25 AM Reid Wahl <<a href="mailto:nwahl@redhat.com">nwahl@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hmm. If it's reading PCMK_ipc_type and matching the server type to QB_IPC_SOCKET, then the only other place I see it could be coming from is qb_ipc_auth_creds.</div><div><br></div><div>qb_ipcs_run -> qb_ipcs_us_publish -> qb_ipcs_us_connection_acceptor -> qb_ipcs_uc_recv_and_auth -> process_auth -> qb_ipc_auth_creds -> <br></div><div><br></div><div style="margin-left:40px"><span style="font-family:monospace">static int32_t<br>qb_ipc_auth_creds(struct ipc_auth_data *data)<br>{</span></div><div style="margin-left:40px"><span style="font-family:monospace">...</span></div><div style="margin-left:40px"><span style="font-family:monospace">#ifdef HAVE_GETPEERUCRED<br> /*<br> * Solaris and some BSD systems</span></div><div style="margin-left:40px"><span style="font-family:monospace">...</span></div><div style="margin-left:40px"><span style="font-family:monospace">#elif defined(HAVE_GETPEEREID)<br> /*<br> * Usually MacOSX systems</span></div><div style="margin-left:40px"><span style="font-family:monospace">...</span></div><div style="margin-left:40px"><span style="font-family:monospace">#elif defined(SO_PASSCRED)<br> /*<br> * Usually Linux systems</span></div><div style="margin-left:40px"><span style="font-family:monospace">...<br></span></div><div style="margin-left:40px"><span style="font-family:monospace">#else /* no credentials */<br> data->ugp.pid = 0;<br> data->ugp.uid = 0;<br> data->ugp.gid = 0;<br> res = -ENOTSUP;<br>#endif /* no credentials */<br><br> return res;</span></div><div style="margin-left:40px"><br></div>I'll leave it to Ken to say whether that's likely and what it implies if so.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jul 26, 2020 at 2:53 AM Gabriele Bulfon <<a href="mailto:gbulfon@sonicle.com" target="_blank">gbulfon@sonicle.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">Sorry, actually the problem is not gone yet.</div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">Now corosync and pacemaker are running happily, but those IPC errors are coming out of heartbeat and crmd as soon as I start it.</div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">The pacemakerd process has PCMK_ipc_type=socket, what's wrong with heartbeat or crmd?</div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)"> </div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">Here's the env of the process:</div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)"> </div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">sonicle@xstorage1:/sonicle/etc/cluster/ha.d# penv 4222<br>4222: /usr/sbin/pacemakerd<br>envp[0]: PCMK_respawned=true<br>envp[1]: PCMK_watchdog=false<br>envp[2]: HA_LOGFACILITY=none<br>envp[3]: HA_logfacility=none<br>envp[4]: PCMK_logfacility=none<br>envp[5]: HA_logfile=/sonicle/var/log/cluster/corosync.log<br>envp[6]: PCMK_logfile=/sonicle/var/log/cluster/corosync.log<br>envp[7]: HA_debug=0<br>envp[8]: PCMK_debug=0<br>envp[9]: HA_quorum_type=corosync<br>envp[10]: PCMK_quorum_type=corosync<br>envp[11]: HA_cluster_type=corosync<br>envp[12]: PCMK_cluster_type=corosync<br>envp[13]: HA_use_logd=off<br>envp[14]: PCMK_use_logd=off<br>envp[15]: HA_mcp=true<br>envp[16]: PCMK_mcp=true<br>envp[17]: HA_LOGD=no<br>envp[18]: LC_ALL=C<br>envp[19]: PCMK_service=pacemakerd<br>envp[20]: PCMK_ipc_type=socket<br>envp[21]: SMF_ZONENAME=global<br>envp[22]: PWD=/<br>envp[23]: SMF_FMRI=svc:/sonicle/xstream/cluster/pacemaker:default<br>envp[24]: _=/usr/sbin/pacemakerd<br>envp[25]: TZ=Europe/Rome<br>envp[26]: LANG=en_US.UTF-8<br>envp[27]: SMF_METHOD=start<br>envp[28]: SHLVL=2<br>envp[29]: PATH=/usr/sbin:/usr/bin<br>envp[30]: SMF_RESTARTER=svc:/system/svc/restarter:default<br>envp[31]: A__z="*SHLVL</div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)"> </div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)"> </div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">Here are crmd complaints:</div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)"> </div>
<div style="font-family:Tahoma;font-size:14px;color:rgb(0,0,0)">Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.notice] notice: Node xstorage1 state is now member<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.error] error: Could not start crmd IPC server: Operation not supported (-48)<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.error] error: Failed to create IPC server: shutting down and inhibiting respawn<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.notice] notice: The local CRM is operational<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.error] error: Input I_ERROR received in state S_STARTING from do_started<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.notice] notice: State transition S_STARTING -> S_RECOVERY<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.warning] warning: Fast-tracking shutdown in response to errors<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.warning] warning: Input I_PENDING received in state S_RECOVERY from do_started<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.error] error: Input I_TERMINATE received in state S_RECOVERY from do_recover<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.notice] notice: Disconnected from the LRM<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.error] error: Child process pengine exited (pid=4316, rc=100)<br>Jul 26 11:39:07 xstorage1 crmd[4315]: [ID 702911 daemon.error] error: Could not recover from internal error<br>Jul 26 11:39:07 xstorage1 heartbeat: [ID 996084 daemon.warning] [4275]: WARN: Managed /usr/libexec/pacemaker/crmd process 4315 exited with return code 201.<br><br><br>
<div id="gmail-m_-6007181615830748374gmail-m_7204354136475660423wt-mailcard">
<div> </div>
<div> </div>
<div><span style="font-size:14px;font-family:Helvetica"><strong>Sonicle S.r.l. </strong>: <a href="http://www.sonicle.com/" target="_blank">http://www.sonicle.com</a></span></div>
<div><span style="font-size:14px;font-family:Helvetica"><strong>Music: </strong><a href="http://www.gabrielebulfon.com/" target="_blank">http://www.gabrielebulfon.com</a></span></div>
<div><span style="font-size:14px;font-family:Helvetica"><strong>Quantum Mechanics : </strong><a href="http://www.cdbaby.com/cd/gabrielebulfon" target="_blank">http://www.cdbaby.com/cd/gabrielebulfon</a></span></div>
</div>
<tt><br><br><br>----------------------------------------------------------------------------------<br><br>Da: Ken Gaillot <<a href="mailto:kgaillot@redhat.com" target="_blank">kgaillot@redhat.com</a>><br>A: Cluster Labs - All topics related to open-source clustering welcomed <<a href="mailto:users@clusterlabs.org" target="_blank">users@clusterlabs.org</a>> <br>Data: 25 luglio 2020 0.46.52 CEST<br>Oggetto: Re: [ClusterLabs] pacemaker startup problem<br><br></tt>
<blockquote style="border-left:2px solid rgb(0,0,128);margin-left:5px;padding-left:5px"><tt>On Fri, 2020-07-24 at 18:34 +0200, Gabriele Bulfon wrote:<br>> Hello,<br>> <br>> after a long time I'm back to run heartbeat/pacemaker/corosync on our<br>> XStreamOS/illumos distro.<br>> I rebuilt the original components I did in 2016 on our latest release<br>> (probably a bit outdated, but I want to start from where I left).<br>> Looks like pacemaker is having trouble starting up showin this logs:<br>> <br>> Set r/w permissions for uid=401, gid=401 on /var/log/pacemaker.log<br>> Set r/w permissions for uid=401, gid=401 on /var/log/pacemaker.log<br>> Jul 24 18:21:32 [971] crmd: info: crm_log_init: Changed active<br>> directory to /sonicle/var/cluster/lib/pacemaker/cores<br>> Jul 24 18:21:32 [971] crmd: info: main: CRM Git Version: 1.1.15<br>> (e174ec8)<br>> Jul 24 18:21:32 [971] crmd: info: do_log: Input I_STARTUP received in<br>> state S_STARTING from crmd_init<br>> Jul 24 18:21:32 [969] lrmd: info: crm_log_init: Changed active<br>> directory to /sonicle/var/cluster/lib/pacemaker/cores<br>> Jul 24 18:21:32 [968] stonith-ng: info: crm_log_init: Changed active<br>> directory to /sonicle/var/cluster/lib/pacemaker/cores<br>> Jul 24 18:21:32 [968] stonith-ng: info: get_cluster_type: Verifying<br>> cluster type: 'heartbeat'<br>> Jul 24 18:21:32 [968] stonith-ng: info: get_cluster_type: Assuming an<br>> active 'heartbeat' cluster<br>> Jul 24 18:21:32 [968] stonith-ng: notice: crm_cluster_connect:<br>> Connecting to cluster infrastructure: heartbeat<br><br><br>> Jul 24 18:21:32 [969] lrmd: error: mainloop_add_ipc_server: Could not<br>> start lrmd IPC server: Operation not supported (-48)<br><br>This is repeated for all the subdaemons ... the error is coming from<br>qb_ipcs_run(), which looks like the issue is an invalid PCMK_ipc_type<br>for illumos. If you set it to "socket" it should work.<br><br><br>> Jul 24 18:21:32 [969] lrmd: error: main: Failed to create IPC server:<br>> shutting down and inhibiting respawn<br>> Jul 24 18:21:32 [969] lrmd: info: crm_xml_cleanup: Cleaning up memory<br>> from libxml2<br>> Jul 24 18:21:32 [971] crmd: info: get_cluster_type: Verifying cluster<br>> type: 'heartbeat'<br>> Jul 24 18:21:32 [971] crmd: info: get_cluster_type: Assuming an<br>> active 'heartbeat' cluster<br>> Jul 24 18:21:32 [971] crmd: info: start_subsystem: Starting sub-<br>> system "pengine"<br>> Jul 24 18:21:32 [968] stonith-ng: info: crm_get_peer: Created entry<br>> 25bc5492-a49e-40d7-ae60-fd8f975a294a/80886f0 for node xstorage1/0 (1<br>> total)<br>> Jul 24 18:21:32 [968] stonith-ng: info: crm_get_peer: Node 0 has uuid<br>> d426a730-5229-6758-853a-99d4d491514a<br>> Jul 24 18:21:32 [968] stonith-ng: info: register_heartbeat_conn:<br>> Hostname: xstorage1<br>> Jul 24 18:21:32 [968] stonith-ng: info: register_heartbeat_conn:<br>> UUID: d426a730-5229-6758-853a-99d4d491514a<br>> Jul 24 18:21:32 [970] attrd: notice: crm_cluster_connect: Connecting<br>> to cluster infrastructure: heartbeat<br>> Jul 24 18:21:32 [970] attrd: error: mainloop_add_ipc_server: Could<br>> not start attrd IPC server: Operation not supported (-48)<br>> Jul 24 18:21:32 [970] attrd: error: attrd_ipc_server_init: Failed to<br>> create attrd servers: exiting and inhibiting respawn.<br>> Jul 24 18:21:32 [970] attrd: warning: attrd_ipc_server_init: Verify<br>> pacemaker and pacemaker_remote are not both enabled.<br>> Jul 24 18:21:32 [972] pengine: info: crm_log_init: Changed active<br>> directory to /sonicle/var/cluster/lib/pacemaker/cores<br>> Jul 24 18:21:32 [972] pengine: error: mainloop_add_ipc_server: Could<br>> not start pengine IPC server: Operation not supported (-48)<br>> Jul 24 18:21:32 [972] pengine: error: main: Failed to create IPC<br>> server: shutting down and inhibiting respawn<br>> Jul 24 18:21:32 [972] pengine: info: crm_xml_cleanup: Cleaning up<br>> memory from libxml2<br>> Jul 24 18:21:33 [971] crmd: info: do_cib_control: Could not connect<br>> to the CIB service: Transport endpoint is not connected<br>> Jul 24 18:21:33 [971] crmd: warning: do_cib_control: Couldn't<br>> complete CIB registration 1 times... pause and retry<br>> Jul 24 18:21:33 [971] crmd: error: crmd_child_exit: Child process<br>> pengine exited (pid=972, rc=100)<br>> Jul 24 18:21:35 [971] crmd: info: crm_timer_popped: Wait Timer<br>> (I_NULL) just popped (2000ms)<br>> Jul 24 18:21:36 [971] crmd: info: do_cib_control: Could not connect<br>> to the CIB service: Transport endpoint is not connected<br>> Jul 24 18:21:36 [971] crmd: warning: do_cib_control: Couldn't<br>> complete CIB registration 2 times... pause and retry<br>> Jul 24 18:21:38 [971] crmd: info: crm_timer_popped: Wait Timer<br>> (I_NULL) just popped (2000ms)<br>> Jul 24 18:21:39 [971] crmd: info: do_cib_control: Could not connect<br>> to the CIB service: Transport endpoint is not connected<br>> Jul 24 18:21:39 [971] crmd: warning: do_cib_control: Couldn't<br>> complete CIB registration 3 times... pause and retry<br>> Jul 24 18:21:41 [971] crmd: info: crm_timer_popped: Wait Timer<br>> (I_NULL) just popped (2000ms)<br>> Jul 24 18:21:42 [971] crmd: info: do_cib_control: Could not connect<br>> to the CIB service: Transport endpoint is not connected<br>> Jul 24 18:21:42 [971] crmd: warning: do_cib_control: Couldn't<br>> complete CIB registration 4 times... pause and retry<br>> Jul 24 18:21:42 [968] stonith-ng: error: setup_cib: Could not connect<br>> to the CIB service: Transport endpoint is not connected (-134)<br>> Jul 24 18:21:42 [968] stonith-ng: error: mainloop_add_ipc_server:<br>> Could not start stonith-ng IPC server: Operation not supported (-48)<br>> Jul 24 18:21:42 [968] stonith-ng: error: stonith_ipc_server_init:<br>> Failed to create stonith-ng servers: exiting and inhibiting respawn.<br>> Jul 24 18:21:42 [968] stonith-ng: warning: stonith_ipc_server_init:<br>> Verify pacemaker and pacemaker_remote are not both enabled.<br>> <br>> Any idea what's happening?<br>> Gabriele<br>> <br>> <br>> <br>> <br>> Sonicle S.r.l. : <a href="http://www.sonicle.com" target="_blank">http://www.sonicle.com</a><br>> Music: <a href="http://www.gabrielebulfon.com" target="_blank">http://www.gabrielebulfon.com</a><br>> Quantum Mechanics : <a href="http://www.cdbaby.com/cd/gabrielebulfon" target="_blank">http://www.cdbaby.com/cd/gabrielebulfon</a><br>> _______________________________________________<br>> Manage your subscription:<br>> <a href="https://lists.clusterlabs.org/mailman/listinfo/users" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>> <br>> ClusterLabs home: <a href="https://www.clusterlabs.org/" target="_blank">https://www.clusterlabs.org/</a><br>-- <br>Ken Gaillot <<a href="mailto:kgaillot@redhat.com" target="_blank">kgaillot@redhat.com</a>><br><br>_______________________________________________<br>Manage your subscription:<br><a href="https://lists.clusterlabs.org/mailman/listinfo/users" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br><br>ClusterLabs home: <a href="https://www.clusterlabs.org/" target="_blank">https://www.clusterlabs.org/</a><br><br><br></tt></blockquote>
</div></div>_______________________________________________<br>
Manage your subscription:<br>
<a href="https://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">https://lists.clusterlabs.org/mailman/listinfo/users</a><br>
<br>
ClusterLabs home: <a href="https://www.clusterlabs.org/" rel="noreferrer" target="_blank">https://www.clusterlabs.org/</a><br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div>Regards,<br><br></div>Reid Wahl, RHCA<br></div><div>Software Maintenance Engineer, Red Hat<br></div>CEE - Platform Support Delivery - ClusterHA</div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div>Regards,<br><br></div>Reid Wahl, RHCA<br></div><div>Software Maintenance Engineer, Red Hat<br></div>CEE - Platform Support Delivery - ClusterHA</div></div></div></div></div></div></div></div></div></div></div></div></div></div>