Cpha status / cphaprob stat down problem

Problem Description

1) cphaprob stat shows the partner firewall status is down on BOTH nodes of an HA pair.

Example:

 firewall-1[admin]# cphaprob stat

 Cluster Mode:   Sync only (IPSO cluster)

 Number     Unique Address  Firewall State (*)

 1 (local)  10.10.30.2      Active
 2          10.10.30.3      Down

2) fw ctl pstat shows zero packets recieved on BOTH nodes

Example:

 firewall-2[admin]# fw ctl pstat|grep -C 1 Sync

 Sync:
       Version: new
       Status: Able to Send/Receive sync packets
       Sync packets sent:
        total : 326990,  retransmitted : 0, retrans reqs : 0,  acks : 0
       Sync packets received:
        total : 0,  were queued : 0, dropped by net : 0

3) tcpdumps on the sync interface only show OUTBOUND packets, no INBOUND packets (2nd field O=outbound packet)

23:46:26.358170 O 00:00:00:00:fe:00 (oui XEROX CORPORATION) > 01:00:5e:0a:1e:fa (oui Unknown), ethertype NOK sync (0x7005), length 78: CPHA 0.0.0.0.8116 > 10.10.30.0.8116: ifc 0 smach 0 dmach 65534 op ifc-cfg-resp

23:46:26.358173 O 00:00:00:00:fe:00 (oui XEROX CORPORATION) > 01:00:5e:0a:1e:fa (oui Unknown), ethertype NOK sync (0x7005), length 78: CPHA 0.0.0.0.8116 > 10.10.30.0.8116: ifc 0 smach 0 dmach 65534 op ifc-cfg-resp

23:46:26.459135 O 00:00:00:00:fe:00 (oui XEROX CORPORATION) > 01:00:5e:0a:1e:fa (oui Unknown), ethertype NOK sync (0x7005), length 218: CPHA 0.0.0.0.8116 > 10.10.30.0.8116: ifc 0 smach 0 dmach 65534 op new-sync

4) Sync mode set to multicast

 crx-dev1[admin]# cphaprob -a if

 eth4c0          sync(secured), broadcast
 eth2c1          non sync(non secured)
 eth2c0          non sync(non secured)
 eth3c0          sync(secured), broadcast
 eth1c1          non sync(non secured)

Possible Causes:

Switch problem, physical NIC / cabling problem.

Solution:

for this case... the sync mode was changed from multicast to broadcast

firewall-1[admin]# cphaconf set_ccp broadcast

run "cphaprob stat" again and it will show active/active if this fix worked.