Over the years I have searched for a tool that will allow me to size a customer’s Splunk license quickly and accurately. I have even attempted to manually and accurately complete this task several times in the past, but I have failed.
Estimating the Splunk data volume within an environment is not an easy task due to several factors: number of devices, logging level set on devices, data types collected per device, user levels on devices, load volumes on devices, volatility of all data sources, not knowing what the end logging level will be, not knowing which events can be discarded, and many more.
As you begin the process of planning and implementing the Splunk environment, understand that the license size can be increased and the Splunk environment can be expanded quickly and easily if Splunk best practices are followed.
Here is my tested and approved, 7-step process on how to determine what size Splunk license is needed:
- Identify and prioritize the data types within the environment.
- Install the free license version of Splunk.
- Take the highest priority data type and start ingesting its data into Splunk, making sure to start adding servers/devices slowly so the data volume does not exceed the license. If data volumes are too high, pick a couple of servers/devices from the different types, areas, or locations to get a good representation of the servers/devices.
- Review the data to ensure that the correct data is coming in. If there is unnecessary data being ingested, that data can be dropped to further optimize the Splunk implementation.
- Make any adjustments to the Splunk configurations needed, and then watch the data volume over the next week to see the high, low, and average size of the data per server/device.
- Take these numbers and calculate them against the total number of servers/devices to find the total data volume for this data type.
- Repeat this process for the other data types listed until you are completed.
If you would like to accelerate this process you can work with Splunk or a Splunk partner to get a larger, temporary license to do your testing.
Good luck with your Splunk implementing and continue Splunking.