My Strange Use Case: Interesting Fields in Splunk

Enriching data is a key outcome Splunk delivers with utmost consistency. In my work with users, this becomes the main objective in easing or expanding the use of your data. When it comes to the “who, what, when, where, and why” of enriching data in Splunk, communication is key in delivering on client engagements.

Let’s see where communication plays in as a consultant…

The “Why”

In this generalized use case, I’ll touch on some key points for nailing down what data we want to enrich and why…

Let’s say, you want to have your Splunk version as an interesting field in your searches for Cisco ISE controllers. In this case, my question first question is… “Why?” Let’s get down to the value you can get out of having your Splunk version as an interesting field. As a consultant, I know it’s possible to have different versions of Splunk from a forwarding environment to the indexing and search layers — which may not result in the ideal environment for you as the customer. This may make this a strange use case, but it sure produces some interesting results.

The “How”

Let’s take a look at how we can get this done for you as the customer. To begin, we’ll take a look at how can you get the current version of Splunk using CLI with the command:

| rest splunk_server=local count=1 /services/server/info | table version

The best (and possibly only) way you could make this an interesting field, according to the requirements, is to make a lookup table that automatically associates with the sourcetype of “cisco:ise”.

How do we get that done? Here’s an overview:

  • Implement a saved search that creates the lookup table
  • Set the lookup table permissions and definitions
  • Set the lookup table as an automatic lookup based on a sourcetype

The Steps

Step 1: Make a saved search that checks the version of Splunk on a regular basis.

In this case, I made a saved search on the Search Head that fires off every hour creating an output lookup table named version.csv. Now, set the permissions to All Apps. Here’s the search.

| rest splunk_server=local count=1 /services/server/info|eval versionnum=1|rename version AS SplunkVersion|table versionnum SplunkVersion|outputlookup version.csv

You’ll see two columns in this table.

  1. versionnum – a numeric number to “key off” of
  2. SplunkVersion – the running version of Splunk

Contents of “version.csv”, versionnum,SplunkVersion, 1,”8.0.2”

Figure 1 - aves the version.csv table in the $SPLUNK_HOME/etc/apps/search/lookups directory
Figure 1 -Searches, Alerts, and Reports table

 

Then, save the version.csv table in the $SPLUNK_HOME/etc/apps/search/lookups directory. Select “Run” to initiate the table.

Step 2: Make the lookup table known and available for use.

Set the lookup table permissions in the Splunk UI: Settings ==> Lookups ==> Lookup table files. Choose “version.csv”

Set Permissions to All Apps, admin write, everyone read.

Figure 2 - Set up lookup table permission in Splunk UI
Figure 2 – Set up lookup table permission in Splunk UI

Step 3: Define the lookup table in the Splunk UI.

Head to Settings > Lookups > Lookup Definitions. Then, Destination App > Search.

Name – choose “version.csv”

Set Permissions to All Apps, admin write, everyone read.

Figure 3 - Define the lookup table in the Splunk UI
Figure 3 – Define the lookup table in the Splunk UI

Step 4: Make it an automatic lookup.

Head to Settings > Lookups > Automatic Lookups.

Set Permissions to All Apps, admin write, everyone read.

Figure 4 - Add new automatic lookup
Figure 4 – Add a new automatic lookup
Figure 5 - Set automatic lookup permissions
Figure 5 – Set automatic lookup permissions

 

Step 5: Add to or create a “props.conf.”

Look at the example of cisco_ise as the sourcetype via TA or in /opt/splunk/etc/system/local.

[cisco_ise]

Then, set EVAL-versionnum = “1”

Step 6: Restart Splunk.

Step 7: Enter in the Search App.

[index=main sourcetype="cisco_ise"| table versionnum SplunkVersion

Figure 6 - New Search in Splunk
Figure 6 – New Search in Splunk

Now, you’ve got your  “Interesting Fields”

 

Figure 7 - Interesting Fields in Splunk
Figure 7 – Interesting Fields in Splunk

Interested?

In this strange use case, Splunk was able to deliver quality results. Although this may seem like a strange need from Splunk, ask yourself this – how are you proving the software matched with versions for reporting? Maybe it’s for an audit check, maybe it’s a request from an executive, but this is a great case on making Splunk work for you.  Now, this example can be replicated for other sourcetypes in the future. We’re all about making Splunk work for you.

Is this thing on? A quick and easy Splunk dashboard status tip

Many clients request some sort of “up or down” status indicator for their customized dashboards. There are many potential uses for such a solution (a simplified result for checking server status, for example; or changing a complex numerical result into an easy-to-read text visualization), and since this is a common question in the Splunk user community, I wanted to share my go-to approach.

Exploiting the Rangemap Command

“Up or Down” functionality isn’t native to Splunk, so for this example we’re going to “exploit” the rangemap command, used extensively in ITSI, and modify the dashboard XML to get the desired result.

Let’s consider the following search:

index=_internal sourcetype=splunkd earliest=-30m latest=now
|eval CountStatus="No Activity"
|stats count
|eval CountStatus=if(count==0,"Down","Up")
|eval alert_level = case(CountStatus=="Up",1,CountStatus=="Down",2)
|rangemap field=alert_level
low=1-1 severe=2-2

This will yield results along the following lines:

Figure 1 – The Rangemap feature results

The Single Value visualization will display the count:

Figure 2 – The Single Value visualization display

What we really want to show, however, is the countStatus of “Up” or “No Data”

To do this, we must get into the XML so we need to save the search as a dashboard and single value in Visualizations.

Figure 3 – Save the search as a dashboard and single value in Visualizations

Then, edit the XML and add the following two lines:

<option name="classField">range</option>
<option name="field">CountStatus</option>

And there you have it!

Figure 5 – Up Status

What do you need to get done with Splunk? We’d love to help!

Kinney Group’s Expertise on Demand (EOD) for Splunk service provides immediate access to our team of Splunk-certified professionals with experience delivering 500+ Splunk engagements worldwide. Contact us below to get started or for more information.