Introduction to SPL & the foreach Command
Splunk’s Search Processing Language (SPL) is a powerful query language designed to analyze data at scale. It enables users to retrieve, transform, and visualize vast amounts of data from multiple sources.
Among its many commands, foreach stands out. It provides a way to iterate over multiple fields, values within a multivalue field, or elements in a JSON array field and apply the same operation to each one. This command is especially helpful when working with structured data that contains repeated patterns.
Understanding the foreach Command
The foreach command allows you to loop through:
- A group of fields matching a wildcard pattern
- Each value in a multivalue field
- Elements in a single field that contains a JSON array
You can perform operations on each field or value individually. This makes the command highly versatile for metric processing, field normalization, and data cleanup. Instead of manually writing the same calculation for each target, you can apply one logic in bulk. This saves time and reduces errors.
Proper Syntax
The syntax of foreach is straightforward, but it’s crucial to understand the placeholders it uses.
... | foreach <field-pattern or field> [mode=<mode>] <operation>
- <field-pattern>: Wildcard pattern to match field names (e.g., metric_*), a single multivalue field, or a field containing a JSON array
- mode=<mode> (optional): Defines how the foreach loop behaves — either over fields (mode=multifield) or values (mode=multivalue). If not specified, it will default to multifield mode.
- <operation>: The operation or expression to perform using placeholder variables
Placeholders within <operation> include:
- <<FIELD>>: Replaced with the actual field name or value
- <<MATCHSTR>>: The variable portion of a wildcard match
- << ITEM >>: Each element in a multivalue field or JSON array
Example with multiple fields:
... | foreach metric_* [eval <<FIELD>> = round(<<FIELD>>/100, 2)]
Why Use foreach in Daily Splunking?
Using foreach streamlines repeated logic across fields and values. It enhances flexibility and efficiency.
- Reduces Repetition: Apply operations across fields or values without duplicating logic.
- Supports Complex Data Structures: Easily work with multivalue fields and JSON arrays.
- Improves Readability: Consolidate repetitive eval statements into a single block.
Example Use Cases
Example #1: Iterating Over a Wildcard Field List
Use Case: You have fields like cpu_user, cpu_system, and cpu_idle. You want to convert them from percentages to fractions for normalization.
Before:
host | cpu_user | cpu_system | cpu_idle |
|---|---|---|---|
A | 85 | 10 | 5 |
Sample SPL:
index=os sourcetype=perfmon:cpu
| foreach cpu_* [eval <<FIELD>> = round(<<FIELD>> / 100, 3)]
After:
host | cpu_user | cpu_system | cpu_idle |
|---|---|---|---|
A | 0.85 | 0.1 | 0.05 |
Explanation: This search matches all fields starting with cpu_. It divides each one by 100 and rounds the result to three decimal places. The eval function applies the logic individually to each matched field, eliminating repetitive code.
Example #2: Iterating Over a Mutivalue Field
Use Case: A field user_roles contains multiple values like “admin”, “editor”, and “viewer”. You want to tag each role with a prefix.
Before:
user | user_roles |
|---|---|
john | admin,editor,viewer |
Sample SPL
index=access sourcetype=web_logs
| eval user_roles=split(user_roles, ",")
| foreach user_roles mode=multivalue
[eval roles=mvappend(roles, "role_".<<FIELD>>)]
| rename roles as user_roles
After:
user | user_roles |
|---|---|
john | role_admin role_editor role_viewer |
Explanation: This search splits the comma separated values in the user_roles field into a multivalue list. The mode=multivalue argument tells foreach to loop over each value instead of field names. Each value is prefixed with “role_” and appended into a new field allied roles to avoid duplicating results. the roles field then gets renamed back into user_roles for consistency.
Example #3: Iterating Through JSON Array Field
Use Case: A field alerts_json contains a JSON array of objects with different severity levels. You want to extract the severity of each alert and normalize it to lowercase.
Before:
event_id | alerts_json |
|---|---|
001 | [{“type”:”network”,”severity”:”HIGH”}, |
{“type”:”malware”,”severity”:”CRITICAL”}] |
Sample SPL:
index=threatintel sourcetype=json_logs
| foreach alerts_json mode=json_array
[eval normalized_severities=mvappend(normalized_severities, lower('<<ITEM.severity>>'))]
| mvexpand normalized_severities
After:
event_id | normalized_severities |
|---|---|
001 | High |
001 | Critical |
Explanation: This search parses through a JSON format in the alerts_json field with mode=json_array, foreach iterates over each object in the array. It extracts the severity field from each object, converts it to lowercase, and appends it to a new multivalue field called normalized_severities.
Conclusion
The foreach command is a vital tool in your SPL toolkit. It brings automation and flexibility to your searches by allowing bulk operations over fields, multivalue entries, or array elements.
When used properly, foreach enhances clarity and saves time. It is especially useful in CIM-based datasets or when working with structured data formats.
Key Takeaways:
- foreach works on field groups, multivalue fields, and JSON arrays
- Use mode=multivalue to loop through values instead of field names
- Simplifies repeated logic across structured data
Let foreach simplify your Splunk searches—one element at a time.
To access more Splunk searches, check out Atlas Search Library, which is part of the Atlas Platform. Specifically, Atlas Search Library offers a curated list of optimized searches. These searches empower Splunk users without requiring SPL knowledge. Furthermore, you can create, customize, and maintain your own search library. By doing so, you ensure your users get the most from using Splunk.
