<html><body><p><font size="2">Thanks Ken for prompt response. </font><br><br><font size="2">Yes.. It was at system boot. I am still to find out a reason as to what caused the reboot. There was no Stonith OR any other error in pacemaker log. </font><br><br><font size="2" face="Arial">Thanks & Regards<br></font><b><font size="2" face="Arial"><br>Dileep Nair</font></b><font size="2" face="Arial"><br>Squad Lead - SAP Base </font><br><font size="2" face="Arial">Togaf Certified Enterprise Architect<br>IBM Services for Managed Applications<br>+91 98450 22258 Mobile<br>dilenair@in.ibm.com<br><br>IBM </font><b><font size="2" face="Arial">Services</font></b><br><br><br><img width="16" height="16" src="cid:1__=EABB0903DFCBDEEF8f9e8a93df938690918cEAB@" border="0" alt="Inactive hide details for Ken Gaillot ---01/28/2019 09:18:11 PM---On Mon, 2019-01-28 at 18:04 +0530, Dileep V Nair wrote: > Hi,"><font size="2" color="#424282">Ken Gaillot ---01/28/2019 09:18:11 PM---On Mon, 2019-01-28 at 18:04 +0530, Dileep V Nair wrote: > Hi,</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Ken Gaillot <kgaillot@redhat.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">Cluster Labs - All topics related to open-source clustering welcomed <users@clusterlabs.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">01/28/2019 09:18 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [ClusterLabs] Pacemaker log showing time mismatch after</font><br><font size="2" color="#5F5F5F">Sent by: </font><font size="2">"Users" <users-bounces@clusterlabs.org></font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><tt><font size="2">On Mon, 2019-01-28 at 18:04 +0530, Dileep V Nair wrote:<br>> Hi,<br>> <br>> I am seeing that there is a log entry showing Recheck Timer popped<br>> and the time in pacemaker.log went back in time. After sometime, the<br>> time issue Around the same time the resources also failed over (Slave<br>> became master). Do anyone know why this behavior ? <br>> <br>> Jan 23 01:16:48 [9383] pn4ushleccp1 lrmd: notice: operation_finished:<br>> db_cp1_monitor_20000:32476:stderr [ /usr/bin/.: Permission denied. ]<br>> Jan 23 01:16:48 [9383] pn4ushleccp1 lrmd: notice: operation_finished:<br>> db_cp1_monitor_20000:32476:stderr [ /usr/bin/.: Permission denied. ]<br>> Jan 22 20:17:03 [9386] pn4ushleccp1 crmd: info: crm_timer_popped:<br>> PEngine Recheck Timer (I_PE_CALC) just popped (900000ms)<br><br>Pacemaker can handle the clock jumping forward, but not backward. The<br>recheck timer here is unrelated to the clock jump, it's just the first<br>log message to appear since it jumped.<br><br>You definitely want to find out what's changing the clock.<br><br>If this is at system boot, likely the hardware clock is wrong and some<br>time manager (ntp, etc.) is adjusting it. Pacemaker's systemd unit file<br>has "After=time-sync.target" to try to ensure that it doesn't start<br>until after this has happened, but unfortunately you often have to take<br>extra steps to make time managers use that target (e.g. enable chronyd-<br>wait.service if you're using chronyd), and of course if you're not<br>using systemd it's not any help. But the basic idea is you want to<br>ensure pacemaker starts after the time has been adjusted at boot.<br><br>If this isn't at boot, then your host has something weird going on.<br>Check the system log around the time of the jump, etc.<br><br>> Jan 22 20:17:03 [9386] pn4ushleccp1 crmd: notice:<br>> do_state_transition: State transition S_IDLE -> S_POLICY_ENGINE |<br>> input=I_PE_CALC cause=C_TIMER_POPPED origin=crm_timer_popped<br>> Jan 22 20:17:03 [9386] pn4ushleccp1 crmd: info: do_state_transition:<br>> Progressed to state S_POLICY_ENGINE after C_TIMER_POPPED<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> process_pe_message: Input has not changed since last time, not saving<br>> to disk<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: notice: unpack_config:<br>> Relying on watchdog integration for fencing<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_online_status_fencing: Node pn4us7leccp1 is active<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_online_status: Node pn4us7leccp1 is online<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_online_status_fencing: Node pn4ushleccp1 is active<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_online_status: Node pn4ushleccp1 is online<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource db_cp1:0 active<br>> on pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource TSM_DB2 active<br>> on pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource TSM_DB2 active<br>> on pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource ip_cp1 active<br>> on pn4ushleccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource db_cp1:1 active<br>> in master mode on pn4ushleccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource TSM_DB2log<br>> active on pn4ushleccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info:<br>> determine_op_status: Operation monitor found resource KUD_DB2 active<br>> on pn4ushleccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: native_print:<br>> stonith-sbd (stonith:external/sbd): Started pn4ushleccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: native_print:<br>> ip_cp1 (ocf::heartbeat:IPaddr2): Started pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: clone_print:<br>> Master/Slave Set: ms_db2_cp1 [db_cp1]<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: short_print:<br>> Masters: [ pn4us7leccp1 ]<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: short_print:<br>> Slaves: [ pn4ushleccp1 ]<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: native_print:<br>> TSM_DB2 (systemd:dsmcad_db2): Started pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: native_print:<br>> TSM_DB2log (systemd:dsmcad_db2log): Started pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: native_print:<br>> KUD_DB2 (systemd:kuddb2_db2): Started pn4us7leccp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: rsc_merge_weights:<br>> ms_db2_cp1: Breaking dependency loop at ms_db2_cp1<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: master_color:<br>> Promoting db_cp1:0 (Master pn4us7leccp1)<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: master_color:<br>> ms_db2_cp1: Promoted 1 instances of a possible 1 to master<br>> Jan 22 20:17:03 [9385] pn4ushleccp1 pengine: info: LogActions: Leave<br>> ip_cp1 (Started pn4us7leccp1)<br>> <br>> <br>> After the transition, the date was shifted back to normal<br>> <br>> Jan 22 20:47:03 [9386] pn4ushleccp1 crmd: info: do_log: Input<br>> I_TE_SUCCESS received in state S_TRANSITION_ENGINE from notify_crmd<br>> Jan 22 20:47:03 [9386] pn4ushleccp1 crmd: notice:<br>> do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE |<br>> input=I_TE_SUCCESS cause=C_FSA_INTERNAL origin=notify_crmd<br>> Jan 23 01:47:22 [9383] pn4ushleccp1 lrmd: notice: operation_finished:<br>> db_cp1_monitor_20000:19518:stderr [ /usr/bin/.: Permission denied. ]<br>> Jan 23 01:47:22 [9383] pn4ushleccp1 lrmd: notice: operation_finished:<br>> db_cp1_monitor_20000:19518:stderr [ /usr/bin/.: Permission denied. ]<br>> <br>> <br>> <br>> Thanks & Regards<br>> <br>> Dileep Nair<br>> Squad Lead - SAP Base <br>> Togaf Certified Enterprise Architect<br>> IBM Services for Managed Applications<br>> +91 98450 22258 Mobile<br>> dilenair@in.ibm.com<br>> <br>> IBM Services<br>-- <br>Ken Gaillot <kgaillot@redhat.com><br><br>_______________________________________________<br>Users mailing list: Users@clusterlabs.org<br></font></tt><tt><font size="2"><a href="https://lists.clusterlabs.org/mailman/listinfo/users">https://lists.clusterlabs.org/mailman/listinfo/users</a></font></tt><tt><font size="2"><br><br>Project Home: </font></tt><tt><font size="2"><a href="http://www.clusterlabs.org">http://www.clusterlabs.org</a></font></tt><tt><font size="2"><br>Getting started: </font></tt><tt><font size="2"><a href="http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf">http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf</a></font></tt><tt><font size="2"><br>Bugs: </font></tt><tt><font size="2"><a href="http://bugs.clusterlabs.org">http://bugs.clusterlabs.org</a></font></tt><tt><font size="2"><br><br></font></tt><br><br><BR>
</body></html>