Splunk Processing Language (SPL) serves as the foundation for data analysis within the Splunk platform. Among the numerous commands available in SPL, the anomalousvalue command stands out as a critical tool for statistical analysis. This command identifies statistical outliers within your data sets, proving invaluable for detecting unusual patterns, potential security threats, and performance anomalies. Security operations centers rely heavily on this functionality to spot suspicious activities that deviate from normal baselines. The command integrates seamlessly with existing security workflows and incident response procedures. The practical applications span across multiple use cases, from network security monitoring to system performance analysis.
Understanding the anomalousvalue Command
The anomalousvalue command operates by calculating statistical measures for specified fields in your data, identifying values that fall inside/outside normal distribution parameters using various statistical methods. The command employs algorithms like those found in the Machine Learning Toolkit (MLTK). Although using this in tandem with the MLTK maximizes its capabilities, it is not necessary to make use of this command.
If you’d like to learn more about the Machine Learning Toolkit, check out our blog post here.
This command makes statistical analysis accessible to users without extensive statistical backgrounds. Splunk automatically handles the complex mathematical calculations required for outlier detection. The command generates additional fields in your search results to indicate anomalous values. These generated fields include anomaly scores and threshold indicators that simplify the analysis process.
Benefits of Using the anomalousvalue Command
Implementing the anomalousvalue command in your regular Splunk operations provides several significant advantages:
- Enhanced Threat Detection Capabilities: The command automatically identifies suspicious activities that might indicate security breaches or malicious behavior. This automation reduces the time required for manual analysis while improving detection accuracy across large data volumes.
- Improved Operational Efficiency: Analysts can focus their attention on genuinely anomalous events rather than sifting through normal operational data. A targeted approach streamlines investigation workflows and reduces alert fatigue among security teams.
- Proactive Performance Monitoring: System administrators can detect performance degradation before it impacts end users significantly. The command helps identify capacity planning requirements and potential infrastructure issues through statistical analysis of historical performance data.
Basic Syntax
The fundamental syntax structure for the anomalousvalue command follows this pattern:
| anomalousvalue [av-options] [action] [field-list] [pthresh]
- Anomalousvalue: This command can be used without any arguments. When this is used, it will create new fields called Anomaly_Score_Cat(<field>) for non-numeric fields and Anomaly_Score_Num(<field>) if it is numerical. If anomalousvalues read it as both Num and Cat, it will create a field for both.
- av-options: This argument provides ways to remove fields from consideration based on a set frequency.
- minsupcount: The number of times a field must appear to be included. If you set the <int> as 70, then a field must appear in at least 70 events or it will be ignored. This can help filter out fields that do not show up enough to be useful for analysis or could affect accuracy.
- minsupfreq: The minimum frequency a field must appear in a dataset. If the <float> is set to 0.15, then that field must appear in 15% or more of the dataset to be considered.
- maxanofreq: The maximum frequency a field can be anomalous before it’s ignored. If you set the <float> to 0.05, then any field showing more than 5% anomalous is dropped. This removes data not useful to your analysis since it is too unreliable.
- minnormfreq: The minimum frequency for anomalous values. If a field doesn’t have enough anomalous values, it is excluded. This leverages a <float> data type, so if set to 0.04 would mean any field with less than 4% of its values being anomalous are excluded.
- action: This controls what the command does with the anomaly scores. There are three options: filter, annotate, and summary.
- filter: This will only return only fields that contain anomalous values.
- annotate: Adds anomaly scores to each event as seen in filter, except it does not filter out any events.
- summary: This command works in the same vein as the fieldsummary command where it returns a comprehensive table of statistics about your fields. This includes the fieldname, count, distinct count, and percentage frequencies based on either its number of normal fields (like with catNormFreq%). Conversely it also has percentages for the frequencies of anomalies (like with catAnoFreq%). This is an excellent tool for analyzing your data and ensuring its normalization with fields like supportFreq% which tells you the percentage that field appears across all events returned.
If you’d like to learn more about fieldsummary, check out our other blog post here.
- fieldlist: This is the number of fields that anomalousvalue will analyze.
- pthresh: Also called probability threshold is the likelihood a value is anomalous. This score is calculated based on how frequently it appears in the dataset for categorical values and how far it is from the mean for numerical values. This defaults to 0.01 or 1% and raising it to 0.05 would mean that each value would have to be at least 5% off the norm to be considered anomalous.
The depth of the arguments and functionality we’ve covered is a testament to the flexibility for this search command to suit many different use cases.
Usage Examples & Practical Applications
Example #1: Tracking Private Network Infiltration
Scenario: I want to monitor the MAC addresses of devices connecting to a private network. With anomalous values, I want to look for any MAC address of a device not normally found on the network. This could be indicative of an unwanted device being connected to a secure network.
index=network sourcetype=cisco:router
| anomolousvalue action=filter src_mac
| table src_mac Anomaly_Score_Cat(src_mac)
| sort - Anomaly_Score_Cat(src_mac)
This would return a table of MAC addresses and their anomalous score. From here, we can analyze the anomalous nature of the higher-scoring MAC addresses. This could be an Indicator of Compromise (IoC) in our network that we should further investigate.
Example #2: Authentication Failure Detection
Scenario: I want to monitor failed login attempts. Specifically, I want to look for indicators of potential brute force attacks. This example shows how to identify users experiencing anomalous authentication failure rates.
index=security sourcetype=authentication action=failure
| bucket _time span=1h
| stats count as failed_attempts by user, _time
| anomalousvalue action=filter failed_attempts pthresh=0.02
| table _time, user, failed_attempts, Anomaly_Score_Num(failed_attempts)
This search is made to identify users who appear to fail authentication more than once in each timespan. But not all users are logging in at the same rate, or some users may be more consistent at entering their passwords than others. This removes that blind spot by using an anomaly score to rate how unusual a user’s number of failed logon attempts is across the searched time range.
Example #3: Analyze Scheduled Searches
Scenario: I believe the scheduled searches in my environment are not being consistent. I want to get a general idea of how they’re performing over a span of time.
index=_internal search_type=scheduled
| anomalousvalue action=summary
This search helps identify potential oddities in my scheduled searches. We can look at fields like app, user, cpuTime, skipped, and others for statistical anomalies worth further investigation. For example, if I wanted to see how often a search is marked as “skipped” across the timespan, then I can look at the skipped_count field under the supportFreq% column to see how often a scheduled search was marked as skipping. If we check for this every day and notice a considerable increase over time, then that could be indicative of the Search Head performance degradation.
Conclusion
The anomalousvalue command represents a powerful tool for statistical analysis within the Splunk ecosystem. Through its automated outlier detection capabilities, organizations can significantly enhance their security monitoring and operational efficiency. The command’s versatility makes it applicable across various use cases, from security threat detection to performance monitoring.
Key points to remember about the anomalousvalue command include:
- Statistical Foundation: The command leverages robust mathematical algorithms to identify outliers automatically, eliminating the need for manual threshold management and reducing false positive rates in security monitoring workflows.
- Operational Integration: Organizations can incorporate this command into existing security operations and system monitoring procedures, enhancing detection capabilities.
- Scalable Analysis: The command efficiently processes large data volumes while maintaining analytical accuracy, making it suitable for enterprise-scale deployments and high-velocity data environments where manual analysis becomes impractical.
To access more Splunk searches, check out Atlas Search Library, which is part of the Atlas Platform. Specifically, Atlas Search Library offers a curated list of optimized searches. These searches empower Splunk users without requiring SPL knowledge. Furthermore, you can create, customize, and maintain your own search library. By doing so, you ensure your users get the most from using Splunk.
