BIG-IP LTM was configured to send email alerts when node or pool member was down. All was working fine until one day I had to configure Remote High-Speed Log (HSL) destination for SPLUNK. HSL worked as expected but node up/down email alerts stopped working. Since I do not get alerts every day it took a while to link the two together. I’ve rebuild alert.conf file, created custom user_alert.conf file with custom SNMP strings, restarted alertd service without any luck. What was confusing is that I could send test emails to my email account and emails alerts from cron job were coming in ok.
F5 BIG-IP: 11.5.1 Build 6.0.159 Hotfix HF6
After several weeks of troubleshooting, it came down to this.
HSL log level was set too low and was preventing messages from being logged locally when node or pool member was going down. Since the log message was not written locally email alert could not be generated. The fix was as follows:
Under System > Logs > Configuration > Log Filters > Log_Filter hyperlink change Severity pulldown menu from ‘Notice’ to ‘Warning’.
Once this setting was changed node status email alerts started to work properly.
- Log level does matter when you start sending logs to multiple destinations.