When installing Oracle Grid Infrastructure (GI) 12c or higher version (18c, 19c, 21c), multiple network interfaces (NIC) can be used as private interconnection to implement inter-communication load balance and failover. However, if this configuration is adopted on Oracle Linux (OL) 6 or Red Hat Enterprise Linux (RHEL) 6 or higher, the GI installation prerequists checking shows error
Verifying Reverse path filter setting ...FAILED rac01: PRVE-0456 : Reverse path filter parameter "rp_filter" for private interconnect network interfaces "eno1,ens1f1,eno2,ens2f1" is not configured to the value of 0 or 2 in file /etc/sysctl.conf on node "rac01". rac02: PRVE-0456 : Reverse path filter parameter "rp_filter" for private interconnect network interfaces "eno1,ens1f1,eno2,ens2f1" is not configured to the value of 0 or 2 in file /etc/sysctl.conf on node "rac02".
The Linux kernel parameter "rp_filter" is defined for applying Strict Reverse Path Forwarding. When the strict filtering is enabled, for a given remote IP, the system will only communicate with it via a specific interface. Unfortunately, the strict reverse patch forwarding may potentially block/discard Oracle GI interconnect communication packets. Therefore, it has to be disabled on NICs used for GI interconnection.
The parameter "rp_filter" can be set to one of three values to control the Reverse Path Forwarding,
0 - No source validation. The filter is disabled.
1 - Strict mode as defined in RFC3704 Strict Reverse Path. Each incoming packet is tested against the Forwarding Information Base (FIB) and if the interface is not the best reverse path the packet check will fail. By default failed packets are discarded.
2 - Loose mode as defined in RFC3704 Loose Reverse Path. Each incoming packet's source address is also tested against the FIB and if the source address is not reachable via any interface the packet check will fail.
GI requires the value to be 0 or 2. The parameter can be set at three different level/scope
1. Default setting with parameter
net.ipv4.conf.default.rp_filter
This parameter only affect new added NIC, does not affect the configuration of existing NICs. It is recommended to be 1 on GI installation.
2. NIC-specific setting with parameter
net.ipv4.conf.<NIC_NAME>.rp_filter
Here, <NIC_NAME> is the device name of the NIC. The setting only affect the specific NIC unless following globle setting is enabled.
3. Global setting with parameter
net.ipv4.conf.all.rp_filter
This setting has priority over default and NIC-specific. It means, if this value is 1 or 2, all NICs' behaviour will be 1 or 2 no matter what their specific settings are. Usually, in most of IT languages, specific (local) setting overrides global setting. But here, it is walking on opposit way, a werid logic. Another more interesting thing is that you cannot set it to 0 to override all NICs. Yes, when it is 0, all NICs will take their own setting which may not be 0.
Regarding GI requirements, we only need disable strict filtering on private interconnection NICs. Therefore the parameter should be
net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.<private-nic-1>.rp_filter = 2 net.ipv4.conf.<private-nic-2>.rp_filter = 2 net.ipv4.conf.<private-nic-3>.rp_filter = 2 net.ipv4.conf.<private-nic-4>.rp_filter = 2
The parameters can be added to file "/etc/sysctl.conf" or create a new conf file under directory "/etc/sysctl.d". Creating a new conf is recommended because sysctl.conf file is deprecated in OL 8 / RHEL 8. The new file name should start with 99 and file name extension should be ".conf" (e.g. 99-sysctl.conf), and make sure the setting will be loaded as late as possible in case they are overriden by the setting in other conf files.
Note: If you create new file under "/etc/sysctl.d" instead of using "/etc/sysctl.conf", GI prerequists may complain that strict filter is enabled though it is disabled. The reason is that GI is looking for global setting "net.ipv4.conf.all.rp_filter" from file "/etc/sysctl.conf", not from new file under "/etc/sysctl.d". As a workaround, add following line to "/etc/sysctl.conf"
net.ipv4.conf.all.rp_filter = 0
To let new setting take effect, run following command with new created conf file name
sysctl -p /etc/sysctl.d/<new-conf-filename>
If setting in "/etc/sysctl.conf", just run the command without file name as following
sysctl -p
To validate if the new setting is loaded, run command "sysctl -a" as following
[root@rac01]# sysctl -a | grep '\.rp_filter' net.ipv4.conf.all.rp_filter = 0 net.ipv4.conf.default.rp_filter = 1 net.ipv4.conf.eno0.rp_filter = 1 net.ipv4.conf.eno1.rp_filter = 2 net.ipv4.conf.eno2.rp_filter = 2 net.ipv4.conf.ens1f0.rp_filter = 1 net.ipv4.conf.ens1f1.rp_filter = 2 net.ipv4.conf.ens2f0.rp_filter = 1 net.ipv4.conf.ens2f1.rp_filter = 2 net.ipv4.conf.lo.rp_filter = 0 net.ipv4.conf.team0.rp_filter = 1 net.ipv4.conf.team1.rp_filter = 1
No comments:
Post a Comment