A Lesson on Macros in Splunk (Part Two)

Let’s talk about macros (again).  Macros in Splunk are built into a lot of apps found on Splunk Base and heavily used in the Monitoring Console.  In part one, we talked through all the prep work and foundation Splunk macros. Now in part two, let’s jump into some methods to create macros, and talk a little about context and sharing. Let’s get cooking!

 

Making Macros

Let’s go ahead and make some macros.   We’ll create a few macros through the web interface, then I’ll take you behind the scenes to see what actually happens in the conf files.

In my lab, I’m going to switch to the Search & Reporting app and add a macro for a search I personally run quite often.  As I’m experimenting and developing in the lab, I always run this search when I get unexpected behavior as a starting point for root cause analysis.

 

index=_internal AND earliest=-5m AND (log_level=WARN* OR log_level=ERROR) AND sourcetype=splunkd

 

But, it’s kind of long and I get lazy so I’m going to set it as a short macro.  So now that we’re in the Search & Reporting app, I’m going to go back to Settings > Advanced Search and click on the “+ Add new” button on the Search macros line.

Figure 1 - Add new Splunk macro
Figure 1 – Add new Splunk macro

The destination app is already set correctly.  I’ll name the macro “myissues” – that should be a unique name that’s descriptive of its purpose.  In the Definition field, I’ll paste in my search from above.  We’ll keep it simple for now and not use another of the options to include arguments.  Click Save.

Now we’ll go back to the Search & Reporting app and use our macro to run a search.  Enter `myissues` (remember the backticks) and click the search button.

Figure 2 - Use macro to run a search in Splunk
Figure 2 – Use macro to run a search in Splunk

There you have it!  A short macro name in the search bar and I have my results.  And with way less typing that, to be honest, usually includes a typo or two.

Add Parameters to Your Macro

It’s kind of inflexible.  What if the issue isn’t caught in Splunk?  What if it occurred more than 15 minutes ago?  Or less than 15 minutes ago in a large environment so I want to restrict the time and speed up results?  Sure, you can use the time picker, but where’s the macro fun in that?  Let’s add some parameters to our macro to make it more useful.

Going back to the Advanced Search settings, I’m going t0 click the Clone button to create a copy of my macro, then edit that clone.  This time, I’m going to give my search a unique name and add “(2)” to the end of the name, indicating that it will expect two arguments.  Then, in the Definition field, I’m going to tokenize the search so Splunk knows where to place the arguments in the search.  In the Arguments field, I’m going to list my arguments, separated by commas.

Now, I’m going to add a little validation to this macro.  The timeframe submitted should be a number here.  Any text would cause the search to fail, so before running the search we’ll validate that field is in fact numbers.  In the Validation Expression box, I’m going to put a simple eval statement that should return TRUE if the input is correct.  If that validation fails, I can write a custom error message to show when the macro runs.  Once set, click Save.

Figure 3 - Add validation to your macro in Splunk
Figure 3 – Add validation to your macro in Splunk

Now we’ll go back to the Search app and test it.  I’ll search the following, to find web errors in the last 30 minutes: `myissues2(30,splunk_web_service)`

And we get results!

Figure 4 - Splunk validation of macro results
Figure 4 – Splunk validation of macro results

And a look at the Job Inspector shows the search expanded with the tokens replaced by my parameters.

Figure 5 - Review job inspector
Figure 5 – Review job inspector

And if I use something other than numbers for my $earliest$ token, I get an error with the message we just set.

Figure 5 - Watch out for this error in Splunk
Figure 6 – Watch out for this error in Splunk

Storing Macros in Splunk

Great!  Now, if you’re a fan of the command line and get tired of GUI’s, let’s look behind the scenes.  If you’re not interested in how Splunk actually stores macros, then jump ahead.

OK, we know that most of Splunk’s knowledge objects and settings are stored in .conf files, and so it’s no surprise that macros are in a file called macros.conf.  Macros are user-level knowledge objects, at least when you create them in the web interface.  Since I was logged in as admin and working in the Search & Reporting app, I’ll navigate to /opt/splunk/etc/users/admin/search/local to find my personal configs for that app.

cat’ing the macros.conf file, we’ll see both of my macros in their own stanzas.  The settings we provided are now in alphabetical order rather than how we saw them on the web interface, but it should look familiar.

Figure 6 - Find your personal configs for the app
Figure 7 – Find your personal configs for the app

Permissions and Contexts

As mentioned briefly above, macros are privately owned by default – they’re only available to the user that created them and only in that app.  In the image below, I’ve logged in as a regular user and tried to run the admin macro created above.

Figure 7 - Error running macro in Splunk
Figure 8 – Error running macros in Splunk

The error is actually really descriptive.  We’ll follow instructions and go share this macro as the admin user.

Now, even though this particular macro is more admin oriented and not really useful to most users, I still want to be able to access it across apps.  That way I can troubleshoot from anywhere.  So it’s back to the Search Macros page under Settings > Advanced Search.  Filter for my macros and click the Permissions button.

Figure 8 - Filter for your macros in Splunk
Figure 9 – Filter for your macros in Splunk

To make sure I can use this macro in any app context, I’m going to select “All apps” under the “Object should appear in” section.  I’ll go ahead and leave the Permissions the same here because a regular user won’t have the right to read the _internal index anyways.  But that gets into another discussion of user roles and permissions that’s best left for another post.

Just to check that I can access my macro from anywhere, I’ll switch to another app and try accessing it again.  (Don’t mind the blanked out names, this is experimental and not ready for release….yet).

Figure 9 - Test your macros from anywhere
Figure 10 – Access your macro from anywhere

If you still have a terminal open where you just cat’ed macros.conf, hit the up arrow and enter.  You can see that the macro we just shared globally is gone.  It’s been moved to the search app, in the macros.conf in the local directory.  It hasn’t changed at all, but it did find a new home.

Figure 9 - Find your macros with open terminal
Figure 11 – Find your macros with an open terminal

One other advanced tip, if you want to see all the macros available to a given user, you can use a simple rest search:

 

| rest /servicesNS/-/-/admin/macros

 

This may be more information than you need to see but could help with some admin down the line, so I thought I’d share it.

Good luck with your Splunking!

Until Next Time…

Hopefully, this helps your understanding of macros: what they are, how to create and use them, and how to share them with other apps and users.  I’d love to give you a list of custom macros that everybody should have, but every Splunk customer and the user has different needs and different environments.

A Lesson on Macros in Splunk (Part One)

Let’s talk about macros.  Not the scripts in MS Office often used to execute malicious code, but the little commands that can do big things in Splunk.  There’s a good chance you’re already using macros – they’re built into a lot of apps found on Splunk Base and heavily used in the Monitoring Console as we’ll see later.  In part one, I want to provide a little primer on what a macro is, show some examples of using macros. In part two, I’ll demonstrate methods to create macros, and talk a little about context and sharing.

 

What Are Macros?

In Splunk terms, macros are Knowledge Objects.  The Splexicon defines a search macro as “A knowledge object …  that contains a portion of a search or a search function”, and a Knowledge Object as “A user-defined entity that enriches the existing data in Splunk Enterprise.”

Put another way, a macro is a short command that can be used to replace parts of or all of search strings to make your SPL searches shorter and easier to understand.  Maybe this is to search multiple indexes without having to enter “index=a OR index=b OR index=r…” every time.  Or maybe it’s a way to ensure consistency by defining the span used on timecharts.

Macros are more powerful than just being a substitute for part of search.  You can also define parameters on a macro, allowing you to call the macro and pass in other fields or values.  This gives you the flexibility to, for instance, to calculate current sales in different currencies by setting the exchange rate as a parameter that you enter when running the search.  Or filtering to different office locations in the BY field of a stats command.

 

Let’s look at some examples…

All right, now that we’ve briefly acquainted you with macros as a concept, let’s look at some examples.  I like to use examples that Splunk users can access right now, without having to download a specific app or dataset, so we’re going to look at the Monitoring Console for most of our examples.  If you’re a User or Power User Role, you may not have access to the Monitoring Console.  But don’t worry, I’ll use plenty of images so you can see what’s going on and show some other examples you may be able to access as well.

Note: I’m running a simple set up of a single indexer and a single search head, running on docker on my laptop.  I’m a huge fan of the Splunk-Ansible project as a tool for Splunk users and developers to experiment with admin rights and no risk to an operational environment.

In the MC, I’m going to click on the Search pulldown, then Scheduler Activity and select Scheduler Activity: Deployment.

Figure 1 - Using scheduler activity: deployment for Splunk macros
Figure 1 – Using scheduler activity: deployment for Splunk macros

Find the Skip Ratio Across All Instances panel.  Should be second from the bottom on the right side.  Mouse over that panel and click the Open in Search button on the bottom right.

Figure 2 - Finding skip ratio for Splunk macros example
Figure 2 – Finding skip ratio for Splunk macros example

That should open a new tab with the search that populates the Skip Ratio panel.  The search you see should look like this:

Figure 3 - search results for skip ratio panel
Figure 3 – search results for skip ratio panel

There are two macros you’ll see here and they’re pretty easy to pick out.  Each macro, when called in an SPL search string, has to have a backtick at the beginning and the end.  Do note that’s not the same as a single quote; this key is usually found at the top left of your keyboard, surrounded by the ESC, Tab, and numeral one (1) keys.  So, the first is on line 1 and the second on line 3, `dmc_set_index_internal` and `dmc_timechart` respectively.

But what’s it actually doing?  To see that, I’m going to click on Settings > Advanced Search, then choose Search Macros on the page.

Figure 4 - check out your search macros in advanced search
Figure 4 – Check out your search macros in advanced search

While writing this, my page loads with the Monitoring Console already chosen as the app filter, but you may need to manually set that.  Just looking at the summary here, we can see 128 macros owned by the Monitoring Console app (apparently, Splunk Inc. is a fan of search macros).

Searching for the first macro set above, we can see this is simply setting the search to start with “index=_internal”.  A very simple macro, I know.  But if you filter by the macro name, you’ll see that it’s used in 10 other macros (yes, you can embed one or more macros in another macro).  Let’s suppose, just for a moment, that Splunk decides to change that index from “_internal” to “_internals_” for some unknown reason in the future.  Changing the definition of this one macro to “index=_internals_” would then update the other 10 macros and the slew of panels that use them.  It’s a simple macro that substitutes a small part of a search but can make updating lots of searches and dashboards really simple.  Pretty neat.

Figure 5 - Setting the search to start with “index=_internal”
Figure 5 – Setting the search to start with “index=_internal”

Now we’ll look at the second macro found above.  `dmc_timechart` is a little different than our last example.

Figure 6 - Setting the search to start with “index=_internal” results
Figure 6 – Setting the search to start with “index=_internal” results

It’s another simple macro but used here to set the minspan option of the timechart.  As I mentioned previously, this provides a way to have consistency across different panels and searches without having to explicitly set the option in each.  And, as simple as it, it appears this macro is used 27 different times[i] in the MC itself.  As before, changing this one macro can do a lot.

Now that we’re looking at the macros themselves, scroll until you find the macro called `dmc_tcp_throughput_split_by(2)`.  You’ll notice right away this one is a little different, ending with a number in parenthesis.  This is an implementation of the parameters that I mentioned above.  The number 2 indicates that the macro expects two parameters, and the Arguments column shows that those are “field” and “group”.  You can also see the tokens “$field$” and “$group$” in the definition.

Figure 7 - `dmc_timechart` macro results
Figure 7 – `dmc_timechart` macro results

So, if you were to use this macro manually, it would look something like this:

`dmc_tcp_throughput_split_by(<field to replace $field$ token>, <group to replace $group$ token)`.

Until Next Time…

That wraps up the first part of the Splunk macros breakdown. I hope you could take back some tips from understanding and exploring some macros examples. The real juicy content will be coming shortly in Part Two, where I’ll show you how to make macros in Splunk. Stay tuned for the next part, as well as upcoming blogs. Until then, shoot us your information below if you’ve got any guidance on using Splunk or want to connect on topics like this.