<div dir="ltr">The reason for --force is:<br>
Error: missing required option(s): 'ipaddr, login, plug' for resource type: stonith:fence_vbox (use --force to override)<br>
<br>
I have selinux disabled on both nodes:<br>
[root@nfsnode1 ~]# cat /etc/sysconfig/selinux<br>
SELINUX=disabled<br>
<br>
pcs stonith update vbox-fencing verbose=true<br>
Error: resource option(s): 'verbose', are not recognized for resource type: 'stonith::fence_vbox' (use --force to override)<br>
It worked with --force:<br>
<br>
Jul 7 13:37:49 nfsnode1 fence_vbox: Unable to connect/login to fencing device<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ Running command: /usr/bin/ssh -4 <a href="mailto:AW23321@10.0.2.2">AW23321@10.0.2.2</a> -i /root/.ssh/id_rsa -p 22 -t '/bin/bash -c "PS1=\\[EXPECT\\]#\ /bin/bash --noprofile --norc"' ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ Timeout exceeded in read_nonblocking(). ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ <fencing.fspawn object at 0x7f882e3c3d50> ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ version: 2.3 ($Revision: 399 $) ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ command: /usr/bin/ssh ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ args: ['/usr/bin/ssh', '-4', '<a href="mailto:AW23321@10.0.2.2">AW23321@10.0.2.2</a>', '-i', '/root/.ssh/id_rsa', '-p', '22', '-t', '/bin/bash -c "PS1=\\\\[EXPECT\\\\]#\\ /bin/bash --noprofile --norc"'] ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ searcher: searcher_re: ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765]
stderr: [ 0: re.compile("Enter passphrase for key
'/root/.ssh/id_rsa':") ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765]
stderr: [ 1: re.compile("Are you sure you want to continue
connecting (yes/no)?") ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ 2: re.compile("\[EXPECT\]#\ ") ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ buffer (last 100 chars): ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ before (last 100 chars): ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ after: <class 'pexpect.TIMEOUT'> ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ match: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ match_index: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ exitstatus: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ flag_eof: False ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ pid: 4779 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ child_fd: 5 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ closed: False ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ timeout: 30 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ delimiter: <class 'pexpect.EOF'> ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ logfile: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ logfile_read: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ logfile_send: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ maxread: 2000 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ ignorecase: False ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ searchwindowsize: None ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ delaybeforesend: 0.05 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ delayafterclose: 0.1 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ delayafterterminate: 0.1 ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ Unable to connect/login to fencing device ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ ]<br>
Jul 7 13:37:49 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4765] stderr: [ ]<br>
Jul 7 13:37:50 nfsnode1 fence_vbox: Running command: /usr/bin/ssh -4 <a href="mailto:AW23321@10.0.2.2">AW23321@10.0.2.2</a> -i /root/.ssh/id_rsa -p 22 -t '/bin/bash -c "PS1=\\[EXPECT\\]#\ /bin/bash --noprofile --norc"'<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ Running command: /usr/bin/ssh -4 <a href="mailto:AW23321@10.0.2.2">AW23321@10.0.2.2</a> -i /root/.ssh/id_rsa -p 22 -t '/bin/bash -c "PS1=\\[EXPECT\\]#\ /bin/bash --noprofile --norc"' ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ Received: [EXPECT]# ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984]
stderr: [ Sent: /cygdrive/c/Program\ Files/Oracle/VirtualBox/<wbr>VBoxManage list vms ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984]
stderr: [ Received: /cygdrive/c/Program\ Files/Oracle/VirtualBox/<wbr>VBoxManage list vms#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ "centos1" {03991a96-bbdc-4a8d-8123-<wbr>298e41c24342}#015#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ "centos2" {f7eb05e5-7cfb-4ef3-a97f-<wbr>e6cafa612b0f}#015#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ "CentOSTest1" {75f0f309-28ea-4314-9d87-<wbr>9227d78ab8c0}#015#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ "CentOSTest2" {d99df7ee-9a2e-4c2a-ac4a-<wbr>902a56055d15}#015#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ [EXPECT]# ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984]
stderr: [ Sent: /cygdrive/c/Program\ Files/Oracle/VirtualBox/<wbr>VBoxManage list runningvms ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ Received: #015< Files/Oracle/VirtualBox/<wbr>VBoxManage list runningvms#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ "centos1" {03991a96-bbdc-4a8d-8123-<wbr>298e41c24342}#015#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ "centos2" {f7eb05e5-7cfb-4ef3-a97f-<wbr>e6cafa612b0f}#015#015 ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ [EXPECT]# ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ Sent: quit ]<br>
Jul 7 13:37:53 nfsnode1 stonith-ng[2045]: warning: fence_vbox[4984] stderr: [ ]<br>
<br>
It seems like there is has some problem but eventually it gets runningvms. What does it mean?</div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>Pozdrawiam,<br>Arek</div></div></div></div>
<br><div class="gmail_quote">2017-07-07 10:25 GMT+02:00 Oyvind Albrigtsen <span dir="ltr"><<a href="mailto:oalbrigt@redhat.com" target="_blank">oalbrigt@redhat.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">On 07/07/17 10:07 +0200, Marek Grac wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
On Fri, Jul 7, 2017 at 8:02 AM, ArekW <<a href="mailto:arkaduis@gmail.com" target="_blank">arkaduis@gmail.com</a>> wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
I did a small research on the scripts<br>
<br>
/usr/sbin/fence_vbox<br>
def main():<br>
...<br>
conn = fence_login(options)<br>
<br>
The fence_loging is scripted in the fencing.py and it should invoke<br>
function: _login_ssh_with_identity_file<br>
<br>
/usr/share/fence/fencing.py<br>
def _login_ssh_with_identity_file:<br>
...<br>
command = '%s %s %s@%s -i %s -p %s' % \<br>
(options["--ssh-path"], force_ipvx, options["--username"],<br>
options["--ip"], \<br>
options["--identity-file"], options["--ipport"])<br>
<br>
There are username and ip parameter used here (not login and ipaddr as in<br>
fence description) so I used:<br>
<br>
</blockquote>
<br>
You have noticed this right, this is due to backward compatibility. And we<br>
are working towards ability to use command-line options everywhere (it is<br>
already in upstream but it is not yet supported in pcs).<br>
<br>
So 'login=FOO' is same as '--username FOO/-l FOO'. Misleading at least. The<br>
mapping between those systems was available on our wiki pages, it is<br>
available in documentation and in (somewhat less readable way) in manual<br>
page.<br>
</blockquote></div></div>
You can run "fence_vbox -o metadata" to see what the different<br>
parameters are named.<div class="HOEnZb"><div class="h5"><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
pcs stonith create vbox-fencing fence_vbox ip=10.0.2.2 username=AW23321<br>
identity_file=/root/.ssh/id_rs<wbr>a host_os=windows<br>
vboxmanage_path="/cygdrive/c/P<wbr>rogram\ Files/Oracle/VirtualBox/VBoxMa<wbr>nage"<br>
pcmk_host_map="nfsnode1:centos<wbr>1;nfsnode2:centos2" ssh=true<br>
inet4_only=true op monitor interval=5 -force<br>
<br>
</blockquote>
<br>
* Why are you using -force?<br>
<br>
* ssh=true is not a valid option (=> it is ignored and warning should be in<br>
the logs) and fence_vbox can use ssh only. [secure=true will do what you<br>
want]<br>
<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I still got the same warning in messages:<br>
Jul 7 07:52:24 nfsnode1 stonith-ng[6244]: warning: fence_vbox[21564]<br>
stderr: [ Unable to connect/login to fencing device ]<br>
Jul 7 07:52:24 nfsnode1 stonith-ng[6244]: warning: fence_vbox[21564]<br>
stderr: [ ]<br>
Jul 7 07:52:24 nfsnode1 stonith-ng[6244]: warning: fence_vbox[21564]<br>
stderr: [ ]<br>
<br>
"Standalone" test is working with the same parameters:<br>
[root@nfsnode1 nfsinfo]# fence_vbox --ip 10.0.2.2 --username=AW23321<br>
--identity-file=/root/.ssh/id_<wbr>rsa --plug=centos2 --host-os=windows<br>
--action=status --vboxmanage-path="/cygdrive/c<wbr>/Program\<br>
Files/Oracle/VirtualBox/VBoxMa<wbr>nage" -4 -x<br>
Status: ON<br>
<br>
</blockquote>
<br>
This looks like SELinux for me. From the command line, you are in<br>
unconfined domain so no checks are performed. Try to look at SELinux<br>
boolean "fenced_can_ssh"<br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I could use more debug in the scripts.<br>
<br>
</blockquote>
You can use verbose=true (-v) and it will display all input/output<br>
operations. In case of the fence_vbox you will see what we attempt to run<br>
and what is the output of these commands. If there is need for more detail<br>
output, please let me know and I will try to add it.<br>
<br>
m,<br>
</blockquote>
<br>
</div></div><div class="HOEnZb"><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank">Users@clusterlabs.org</a><br>
<a href="http://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.clusterlabs.org/m<wbr>ailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/doc<wbr>/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
</blockquote>
<br>
<br>
______________________________<wbr>_________________<br>
Users mailing list: <a href="mailto:Users@clusterlabs.org" target="_blank">Users@clusterlabs.org</a><br>
<a href="http://lists.clusterlabs.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.clusterlabs.org/m<wbr>ailman/listinfo/users</a><br>
<br>
Project Home: <a href="http://www.clusterlabs.org" rel="noreferrer" target="_blank">http://www.clusterlabs.org</a><br>
Getting started: <a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf" rel="noreferrer" target="_blank">http://www.clusterlabs.org/doc<wbr>/Cluster_from_Scratch.pdf</a><br>
Bugs: <a href="http://bugs.clusterlabs.org" rel="noreferrer" target="_blank">http://bugs.clusterlabs.org</a><br>
</div></div></blockquote></div><br></div>