Problem - fwm start failure on mds
From cpwiki.net
Problem description:
- mds fails to fuly start up
- problem with guis connecting to CMA/CLM from MDG
- mdsstat shows MDS fwm down after mdsstart
Troubleshooting steps
start fwm in debug mode (from mds environment)
# mdsenv # fwm -d mds [Expert@r65_mdshost]# fwm -d mds [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] cpobj_get_plugin_conf_info: Could not open file (/opt/CPPIconnectra-R65/conf/plugin_groups.conf). [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_create: version 5301. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_add_name_to_group: finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_set_local_names: () names. finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_create: finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_read (/opt/CPshrd-R65/conf/sic_policy.conf): finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_set_external_host_groups: 43 names. finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_add_name_to_group: finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_set_local_names: (local_sic_name) names. finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_add_name_to_group: finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_set_local_names: (171.186.108.253) names. finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_add_name_to_group: finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_policy_set_local_names: ("CN=cp_mgmt_r65_mdshost,O=iproricNGX2..rsyqv9") names. finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_apply_default_dn: ca_dn = [O=iproricNGX2..rsyqv9]. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_apply_default_dn: calling PM_policy_DN_conversion .. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] PM_apply_default_dn: finished successfully. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 12 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Error opening file /opt/CPshrd-R65/database//authkeys.C:: No such file or directory [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 12 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 12 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 32 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 12 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 12 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 32 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 32 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 11 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 31 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 11 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 11 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 31 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] ckpSSLctx_New: prefs = 31 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] peers addresses are [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] 171.186.108.253 [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] sic_client_do_connect: using server local sic name. [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] is_command_no_need_for_license: it's ok to run this command, without special checking [ 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] is_msp_environment_set_correctly> YES [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] CPPRODIS_init_error_logging_ex: initialized error logging for product 'FW1' application 'MDS'. Log file is not set. [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] raise_file_limit: raising limit from 1024 to 1024 [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] FW Cleaner: Adding cleanup function FwmIsAliveMutex_Destroy() (0x80fb840, 0x1849) [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Env Configuration: ( :type (opsec_info) )
[MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Could not find info for ...opsec_sic_name... [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Could not find info for ...opsec_sslca_file... [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Could not find info for ...opsec_shared_local_path... [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Could not find info for ...opsec_sic_policy_file... [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] Could not find info for ...opsec_mt... [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] opsec_init: multithread safety is not initialized [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:37] FW Cleaner: Adding cleanup function FwmDestroyOpsecEnv() (0x824afb0, 0x0) [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:38] fwa_db_init_with_scope: called [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:38] do_links_getver: strncmp failed. Returning -2 [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] port found in reg 1024 [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] cplog_localtcpip: found port 1024 [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] Failed to connect to FWD (log connection). [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] resolver_gethostbyname: Performing gethostbyname for r65_mdshost [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] resolver_gethostbyname: Failed to resolve hostname 'r65_mdshost' fw_ipaddr: Unable to resolve ipaddr for r65_mdshost: Resource temporarily unavailable [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] FW Cleaner: calling cleanup function FwmIsAliveMutex_Destroy() (0x80fb840, 0x1849) [MDS 6217 2002609888]@r65_mdshost[20 Jul 15:41:39] FW Cleaner: calling cleanup function FwmDestroyOpsecEnv() (0x824afb0, 0x0)
hosts file doesn't contain mds hostname
# cat /etc/hosts 192.168.1.100 -n # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost
Solution
fix hosts file
mdsstop -m; mdsstart -m
[Expert@r65_mdshost]# mdsstat |grep MDS
MDS | 192.168.1.100 | up 6474 | up 6473 | up 6472 | N/R |