Forgot My pass4SymmKey

It’s 11 PM on a Thursday, and you’re ready to complete your approved change of adding another Splunk instance to your cluster. And then the realization hits you: you’ve forgotten your pass4SymmKey. We’ve all been there. No worries, you’ve landed on the perfect blog post to help you decrypt and recover that pass4SymmKey quickly.

What is pass4SymmKey?

The pass4SymmKey is a password that controls authentication between Splunk instances, and not only for clusters. There can be a few unique pass4SymmKeys on a single instance of Splunk such as one for control authentication between an indexer cluster, search head cluster, or license master.

Sometimes, your pass4SymmKey may look different from what you initially typed. After restarting Splunk, the pass4SymmKey is encrypted using the unique splunk.secret file on the Splunk instance and is then replaced in the .conf file in its encrypted form.

For example, you may add “pass4SymmKey = SuperS3cur3pa55w0rd” to the .conf file. Then, Splunk restarts and it now shows as “pass4SymmKey = $7$hAE4mKMz396uoI7CarEWWaj86zbHgsFAcz1KQ73PBvKsR5dBfnPbsQ==”.

Decrypt your pass4SymmKey on Splunk Enterprise 7.2.2 and newer

When you’re working on Splunk Enterprise 7.2.2 and newer, follow these steps:

Step 1. Log into the Splunk server’s CLI that stores the pass4SymmKey. Then, decrypt the pass4SymmKey on the same instance that encrypted it.

Step 2. Copy the pass4SymmKey, place it in this command, run the command, and Splunk will decrypt it and tell you what it is:

splunk show-decrypted --value ‘pass4SymmKeyGoesHere’

Follow this example:

splunk show-decrypted --value ‘$7$hAE4mKMz396uoI7CarEWWaj86zbHgsFAcz1KQ73PBvKsR5dBfnPbsQ==’

Step 3. Document the pass4SymmKey in a secure password manager.

Decrypt your pass4SymmKey on Splunk Enterprise Pre 7.2.2

When you’re working on Splunk Enterprise Pre 7.2.2 (this will also work with Splunk 7.2.2 and newer), follow these steps:

Step 1. Log into the Splunk server’s CLI that stores the pass4SymmKey. You must decrypt the pass4SymmKey on the same instance that encrypted it.

Step 2. Create a directory in the Splunk apps directory and call it “decrypt.”

/opt/splunk/etc/apps/decrypt

Step 3. Create a directory called “local” under the “decrypt” directory.

/opt/splunk/etc/apps/decrypt/local

Step 4. Within the “local” directory create a file called “passwords.conf” and place the following contents within that file and save it.

[credential:p4SK]

password = yourPass4SymmKeyGoesHere

Follow this example:

[credential:p4SK]

password = $7$hAE4mKMz396uoI7CarEWWaj86zbHgsFAcz1KQ73PBvKsR5dBfnPbsQ==

Step 5. Restart Splunk. You could alternatively run a “debug refresh” on the Splunk instance if you cannot restart Splunk. This will force Splunk to read the new configuration we added without restarting it. To complete a debug refresh, you’ll have to…

1. Visit http(s)://yourSplunkServerAndWebPortNumber/en-US/debug/refresh in your web browser.
   192. Here’s an example search: https://192.168.1.10:8000/en-US/debug/refresh
2. Click the “Refresh” button and wait for Splunk to read the new configuration. It will take a few seconds then return information on what it did. You can safely close the web browser after that.

Step 6. Run the following command, enter your Splunk web admin login info, and look through the output for <code><s:key name=”clear_password”></code> – the value to the right of that is your decrypted pass4SymmKey in clear text:
splunk _internal call /servicesNS/nobody/decrypt/storage/passwords

Here’s an example Output Snippet. Please note, SuperS3cur3pa55w0rd” is the decrypted password for this example:

<s:key name="clear_password">SuperS3cur3pa55w0rd</s:key>

 

Alternatively, you could run the following Splunk search in the web GUI and it will output the decrypted pass4SymmKey in clear text as well.

| rest /servicesNS/nobody/decrypt/storage/passwords | fields clear_password

Step 7. Delete the “decrypt” directory we created and all its contents. Then, restart Splunk or run a “debug refresh” to reduce access to the pass4SymmKey in clear text.

Or, an alternative method is a rest call via your web browser. Visit your Splunk Enterprise server’s URL: http(s)://yourSplunkServerAndManagementPortNumber/servicesNS/nobody/decrypt/storage/passwords

Step 8. Document the pass4SymmKey in a secure password manager.

Splunk Pro Tip: This type of work can be a considerable resource expense when executing it in-house. The experts at Kinney Group have several years of experience architecting, creating, and solving in Splunk. With Expertise on Demand, you’ll have access to some of the best and brightest minds to walk you through simple and tough problems as they come up.

See for Yourself

Decryption, Done!

And there you have it. Your pass4SymmKey is recovered. When you add these tips to your Splunk toolshed, remember you can also use the above steps to decrypt the “sslPassword” encrypted password on a Splunk Enterprise instance. Stay tuned for decrypting the pass4SymmKey on another Splunk Enterprise instance when you can’t touch production.

If you need help with decrypting your Splunk pass4SymmKey, or just need help decrypting how to effectively use or grow your Splunk environment, we are here to make you your organization’s Splunk hero. Not to be cryptic, if you want to chat with one of our expert Splunk consultants, fill out the for below.