Unlocking the Power of Data Analytics with Splunk Machine Learning

Table of Contents

1. Introduction
2. Splunk Machine Learning Toolkit 2.1 What is the Splunk Machine Learning Toolkit? 2.2 Features and Benefits 2.3 Licensing Information
3. Meet the Experts 3.1 Jessica Davlin - Senior Product Marketing Manager 3.2 Kanandra Rana - Product Manager for AI/ML
4. Agenda
5. The Power of Machine Learning in Splunk 5.1 Tracking and Monitoring with Splunk 5.2 Unlocking Insights with Machine Learning
6. Use Cases for Splunk Machine Learning 6.1 Outlier and Anomaly Detection 6.2 Predictive Analytics 6.3 Data Clustering
7. Building Machine Learning Models in Splunk 7.1 Supervised vs Unsupervised Learning 7.2 Machine Learning Workflow
8. Demo: Splunk Machine Learning Toolkit in Action 8.1 Use Case 1: Smart Forecasting 8.2 Use Case 2: Smart Outlier Detection
9. Operationalizing Models and Continuous Training
10. Additional Resources for Machine Learning in Splunk 10.1 Documentation and Blogs 10.2 Downloading the App
11. Conclusion

Splunk Machine Learning Toolkit: Unleashing the Power of Data Analytics

In today's data-driven world, organizations are constantly seeking ways to derive more insights and make informed decisions faster. This is where machine learning comes into play, revolutionizing how we analyze and interpret data. Splunk, a leading data analytics platform, offers the Splunk Machine Learning Toolkit (MLTK) to help organizations leverage their existing data and derive valuable insights.

Introduction

Welcome to Tech Talks! In this edition, we dive into the world of Splunk Machine Learning Toolkit and explore its capabilities. Tech Talks is a series of short webinars that focus on features and best practices within various use cases. Our aim is to provide you, our valued customer, with tips and tricks to help you leverage Splunk's machine learning capabilities in your daily role.

Allow me to introduce myself. I'm Jessica Davlin, a Senior Product Marketing Manager for the Splunk platform, with a specific focus on machine learning. I'm thrilled to be here today to share with you the exciting possibilities offered by the latest release of Splunk's Machine Learning Toolkit (MLTK).

But first, let me introduce my colleague, Kanandra Rana. Kanandra is a Product Manager for AI/ML and has been working with Splunk for the past five years. She will be diving into more depth and providing a demo later in this Tech Talk.

Splunk Machine Learning Toolkit

What is the Splunk Machine Learning Toolkit?

The Splunk Machine Learning Toolkit (MLTK) is an app that unleashes the power of machine learning within Splunk's robust data analytics platform. It enables organizations to operationalize machine learning models on their data, transforming raw data into actionable insights. The MLTK is available with both Splunk Enterprise and Splunk Cloud licenses, ensuring that organizations of all sizes can take advantage of its capabilities.

Features and Benefits

With the Splunk Machine Learning Toolkit, organizations can explore, experiment, and model their data using a wide range of machine learning tools. The toolkit brings together a collection of algorithms, techniques, and visualizations that simplify the process of building machine learning models.

Some of the key features and benefits of the MLTK include:

• Smart Assistants: The MLTK features smart assistants that guide users through the process of building machine learning models. These assistants provide step-by-step guidance, making it easier for users to explore different algorithms, evaluate results, and deploy models into production.

• Effortless Workflow: The MLTK provides a Simplified machine learning workflow that removes complexities associated with exploration and experimentation. Users can easily define their goals, clean the data, apply algorithms, and evaluate the results, all within a user-friendly interface.

• Customizable Interface: The MLTK offers an intuitive and customizable interface that allows users to tailor their machine learning models to specific use cases. Through point-and-click setup, interactive customization, and visualizations, users can fine-tune their models and achieve accurate predictions.

Licensing Information

The Splunk Machine Learning Toolkit is available with any Splunk Enterprise or Splunk Cloud license. Organizations can take advantage of the MLTK's capabilities without any additional cost. If you're already using Splunk, you have the power to unlock the potential of machine learning and gain valuable insights from your data.

Meet the Experts

Allow me to introduce the experts who have worked tirelessly to develop the Splunk Machine Learning Toolkit and bring its capabilities to you.

Jessica Davlin - Senior Product Marketing Manager

As a Senior Product Marketing Manager for the Splunk platform, I have had the privilege of working closely with the Machine Learning Toolkit team. With a deep understanding of both the product and customer needs, I am here to guide you through the exciting world of Splunk machine learning.

Kanandra Rana - Product Manager for AI/ML

Kanandra has been part of the Splunk family for the past five years, specializing in Artificial Intelligence and Machine Learning. With her extensive knowledge and experience, Kanandra will be diving into the technical aspects of the MLTK and providing a live demo to showcase its capabilities.

Agenda

Let's briefly cover the agenda for today's Tech Talk. We will be addressing the following topics:

1. Experimenting and Modeling your Splunk Data using Machine Learning Tools.
2. The Possibilities with Splunk's Machine Learning Capabilities.
3. Live Demo of the Splunk Machine Learning Toolkit.
4. Additional Resources to Help You Leverage Machine Learning and Splunk.

Throughout the Tech Talk, our team will be available to answer any questions you may have through the chat feature. If you are watching a Recording of this webinar, please continue the conversation through the Splunk Community website for any follow-up questions.

Now, let's dive in and explore the power of machine learning in Splunk.

The Power of Machine Learning in Splunk

Many organizations are already leveraging Splunk to track and monitor their event and metric data. However, what they may not know is that Splunk offers the ability to experiment and model data with machine learning, enhancing their IT, security, and DevOps use cases.

Machine learning in Splunk enables organizations to unlock deeper insights from their data, allowing them to make faster and more informed decisions. What's more, Splunk's smart assistants guide users through the process of experimentation, helping them build different models to forecast events, detect anomalies, and more.

Let's explore some of the possibilities offered by machine learning in Splunk.

Outlier and Anomaly Detection

Outlier and anomaly detection involves identifying deviating behaviors or unusual changes in data. Machine learning can help organizations pinpoint these anomalies, enabling them to take proactive actions. Whether it's detecting suspicious network behavior, identifying fraudulent transactions, or spotting irregular system performance, machine learning in Splunk can be a valuable asset in ensuring the security and stability of your systems.

Predictive Analytics

Predictive analytics is all about forecasting and predicting future events based on historical data. With machine learning in Splunk, organizations can leverage their existing data to forecast upcoming trends, spikes in behavior, or even predict failures in the context of predictive maintenance. By harnessing the power of machine learning algorithms, organizations can gain valuable insights into their operations, enabling them to plan resources effectively and mitigate potential risks.

Data Clustering

Data clustering involves grouping similar events or reducing the noise of alerts by grouping them together. Machine learning in Splunk enables organizations to correlate events, identify Patterns, and reduce the overwhelming number of alerts. By doing so, organizations can focus on the significant events that matter and reduce the time and effort spent on investigating false positives or non-critical alerts.

In today's demo, we will explore both outlier detection and forecasting use cases in more detail. Now, before we dive into building these models in Splunk, let's briefly touch upon Supervised and unsupervised machine learning.

Building Machine Learning Models in Splunk

When it comes to building machine learning models, you may come across terms like supervised and unsupervised learning. These terms essentially refer to whether the data used to train the model contains a matched set of questions and answers.

Supervised Learning: In supervised learning, the data set used to train the model contains labeled or matched data. This labeled data helps the model learn the relationship between the input (question) and output (answer), enabling it to propose potential outcomes to new questions.

Unsupervised Learning: In unsupervised learning, the data set used to train the model consists of one-sided data. This data is analyzed to determine similarities, which can then be recognized in future data inputs.

Now, let's HAND it over to Kanandra for a live demo of building machine learning models using the Splunk Machine Learning Toolkit.

Demo: Splunk Machine Learning Toolkit in Action

Thank you, Jessica. In this demo, I will showcase the capabilities of the Splunk Machine Learning Toolkit with two specific use cases: smart forecasting and smart outlier detection. These use cases highlight how organizations can harness the power of machine learning to make accurate predictions and identify outliers in their data.

Use Case 1: Smart Forecasting

Imagine an e-commerce platform that wants to forecast the number of user logins for the next 30, 60, or 90 days. By accurately predicting the expected number of logins, the platform can ensure it has enough resources to handle peak usage.

To demonstrate this use case, we will open the Smart Forecast Assistant in the MLTK. The assistant provides a step-by-step guided process to help users build accurate forecasting models.

1. Define the Data: Users can define the data they want to use for forecasting, either through search or by selecting datasets and metrics. In this case, we will define the data using search.

2. Data Cleaning: Cleaning the data is crucial for accurate forecasting. The assistant guides users through the process of cleaning the data, ensuring that any null values or outliers are appropriately handled.

3. Feature Engineering: In some cases, special events or holidays may impact the data. In our example, we have created a Lookup file that contains all the special dates related to sales and holidays. By joining this lookup file with the logon data, we can account for the impact of these events on the forecasting model.

4. Forecasting: Once the data is cleaned and preprocessed, users can select the target field for forecasting. Additionally, users can specify the holdback period and the forecast horizon. The Smart Assistant provides tooltips and explanations to guide users in making the right selections.

5. Review the Results: After running the forecast, users can review the forecasted values and statistics. The assistant provides an interactive interface where users can explore the forecasted values and compare them to the actual data.

6. Identify Violations: In this use case, one of the critical points is identifying when the logon count exceeds a certain threshold, such as 100 logins per day. The smart assistant provides alerts and notifications to help users identify these violations and take appropriate actions.

7. Operationalize the Model: Once the forecasting model is built and validated, users can operationalize it to run on incoming data continuously. This ensures that the model stays up to date with new data and provides real-time insights.

Now that we have explored the smart forecasting use case, let's move on to the next use case: smart outlier detection.

Use Case 2: Smart Outlier Detection

In this use case, we will focus on a system administrator who wants to identify outliers in the call volume at a call center. The goal is to identify outliers by the day of the week and the source of the calls.

1. Data Exploration: We start by exploring the data, which contains information about the call volume, time, and sources. The MLTK provides an interactive interface to Visualize and understand the data.

2. Feature Extraction: To identify outliers by the day of the week, we use the MLTK's feature extraction capabilities. We extract features such as the hour of the day, the day of the week, the day of the month, and the month of the year.

3. Outlier Detection: Using the Splunk SPL (Search Processing Language), the MLTK constructs the entire outlier detection process. This eliminates the need for users to write complex SPL queries manually. The smart assistant takes care of the entire process, making it easier for users to identify outliers.

4. Interpret the Results: The MLTK presents the results of the outlier detection, providing a summary of the outliers by day of the week and source. Users can visualize the outliers and gain insights into the patterns and trends associated with them.

By leveraging the Splunk Machine Learning Toolkit's smart assistants, users can build accurate forecasting and outlier detection models with ease. These smart assistants simplify the machine learning workflow and provide step-by-step guidance.

Operationalizing Models and Continuous Training

Building machine learning models is just the first step. To ensure the models remain accurate and up to date, organizations need to continuously train and update them with new data. The Splunk Machine Learning Toolkit provides features to operationalize models and automate the training process.

Once a model is built and validated, users can schedule model training to keep the model up to date with incoming data. By specifying the frequency of training, users can ensure that their models are continuously trained on real-time data.

Additionally, the MLTK allows users to set up alerts and notifications based on model predictions or violations of specified thresholds. This way, organizations can take immediate action whenever anomalies or outliers are detected.

Additional Resources for Machine Learning in Splunk

To help you further explore and leverage the capabilities of the Splunk Machine Learning Toolkit, we have several resources available:

1. Documentation and Blogs: Visit Splunk's official documentation and blog posts to learn more about the MLTK's features and best practices for machine learning in Splunk.
2. Download the App: You can download the Splunk Machine Learning Toolkit app from the Splunkbase and start exploring its capabilities on your own data.

By utilizing these resources, you can take your machine learning journey in Splunk to the next level and unlock even more powerful insights from your data.

Conclusion

In conclusion, the Splunk Machine Learning Toolkit is a powerful tool that allows organizations to leverage their data and unlock valuable insights. Whether it's forecasting future trends, identifying outliers, or clustering events, machine learning in Splunk is transforming how organizations analyze and interpret their data.

We hope this Tech Talk has provided you with a glimpse into the possibilities offered by the Splunk Machine Learning Toolkit. Whether you're just starting or already exploring machine learning in Splunk, our team is here to support you. Please continue the conversation on our Splunk Community website, where you can find additional resources and connect with other Splunk users.

Thank you for joining us today, and be sure to stay tuned for future Tech Talks as we continue to explore the exciting world of machine learning and data analytics.