With Splunk, getting data in is hard enough. After uploading a CSV, monitoring a log file, or forwarding data for indexing, more often than not the data does not look as expected. These large blocks of unseparated data are hard to read and unable to be searched. If the data is not separated into events, you may be wondering how to correctly parse, and perform advanced search commands using fields.
This is where field extraction comes in handy.
Field Extraction via the GUI
Field extractions in Splunk are the function and result of extracting fields from your event data for both default and custom fields. Basically, organize your data with field extractions in order to see the results you’re looking for.
Pictured above is one of Splunk’s solutions to extracting searchable fields out of your data via Splunk Web. Within the Search and Reporting App, users will see this button available upon search. After clicking, a sample of the file is presented for you to define from events the data. The image below demonstrates this feature of Splunk’s Field Extractor in the GUI, after selecting an event from the sample data.
From here, you have two options: use a regular expression to separate patterns in your event data into fields, and the ability to separate fields by delimiter. Delimiters are characters used to separate values such as commas, pipes, tabs and colons.
If you have selected a delimiter to separate your fields, Splunk will automatically create a tabular view in order to allow you to see what all events properly parsed would look like compared to its _raw data pictured above.
Functionality is provided to rename all fields parsed by the selected delimiter. After saving, you will be able to search upon these fields, perform mathematical operations, and advanced SPL commands.
What’s Next? Rex and Erex Commands
After extracting fields, you may find that some fields contain specific data you would like to manipulate, use for calculations, or display by itself. You can use the Rex and Erex commands to help you out.
The Rex command is perfect for these situations. With a working knowledge of regex, you can utilize the Rex command to create a new field out of any existing field which you have previously defined. This new field will appear in the field sidebar on the Search and Reporting app to be utilized like any other extracted field.
| rex [field=<field>] (<regex-expression>)
For those who would like to use the Rex command, and would like resources to learn, please utilize websites such as https://regex101.com/ to further your development.
In order to define what your new field name will be called in Splunk, use the following syntax:
| rex [field=<field>] (?<field_name>”regex”)
Many Splunk users have found the benefit of implementing Regex for field extraction, masking values, and the ability to narrow results. Rather than learning the “ins and outs” of Regex, Splunk provides the erex command, which allows users to generate regular expressions. Unlike Splunk’s rex and regex commands, erex does not require knowledge of Regex, and instead allows a user to define examples and counterexamples of the data to be matched.
| erex <field_name> examples="<example, <example>" counterexamples="<example, <example>"
Here’s an example of the syntax in action:
| erex Port_Used examples=”Port 8000, Port 3182”
That’s a Wrap
There is a ton of incredible work that can be done with your data in Splunk. When it comes to extracting and manipulating fields in your data, I hope you found this information useful. We have plenty of Splunk tips to share with you. Fill out the form below if you’d like to talk with us about how to make your Splunk environment the best it can be.