Boost ML Model Performance with Advanced Observability
Table of Contents
- Introduction
- Setting Up a Data Pipeline in Arise
- Understanding Embeddings
- What Are Embeddings?
- Importance of Embeddings
- Using UMAP for Visualization
- Detecting Drift in Embeddings
- Analyzing Unstructured Workflows
- Utilizing Computer Vision Data Set
- Best Practices in Unstructured Workflows
- Exploring the Hot Topic of Generative Models
- Creating Custom Business Metrics
- Associating ML Models with Business Value
- Understanding Statistical Metrics vs. Business Metrics
- Building Custom Metrics with SQL-Like Syntax
- Programmatic Access with Arise
- Using SDKs to Push Inference Logs
- Integrating Arise with Your Architecture
- Automating Actions with Arise APIs
- Monitoring and Alerting with Graphql
- Introduction to Graphql Interface
- Querying and Mutating Data with Graphql
- Setting Up Performance Monitors with Graphql
- Deploying Auto-Thresholding Monitors
Introduction
Hello everyone! Welcome to this workshop on advanced Arise workflows. My name is Dat, and I am a ML Solutions Architect here at Arise. In this workshop, we will be diving deeper into the world of Arise and exploring some more advanced user workflows. Whether You are already familiar with Arise or just getting started, this presentation will provide valuable insights into how to leverage the ML observability capabilities of Arise.
Throughout this workshop, we will cover various topics including setting up a data pipeline in Arise, understanding embeddings and detecting drift, analyzing unstructured workflows using computer vision data, creating custom business metrics, and utilizing programmatic access with Arise. We will also explore how to set up monitors and alerts using Graphql.
So let's get started with our agenda for today.
Setting Up a Data Pipeline in Arise
To begin, we will walk through the process of setting up a data pipeline in Arise. Whether you are working with data connectors or SDKs, Arise provides various ways to integrate with your Current stack and ingest data. We will specifically focus on setting up a data pipeline using a data connector, specifically for a data bucket in an AWS S3 account. We will explore the necessary configurations and permissions required to establish this connection and push data into Arise. By the end of this section, you will have a clear understanding of how to set up a data pipeline in Arise and seamlessly integrate it with your existing architecture.
Understanding Embeddings
The next topic we will cover is embeddings and their importance in ML models. We will start by explaining what embeddings are and how they serve as representations of unstructured data. We will Delve into the significance of embeddings and why they are foundational to most unstructured models. To better Visualize and comprehend embeddings, we will utilize the UMAP algorithm to project embeddings into a 3D space. This will enable us to analyze embeddings drift and understand how embeddings change over time. We will then dive into various unstructured workflows using a computer vision data set. Throughout this section, we will discuss best practices and explore the space of generative models, which has gained significant Attention in recent times.
Analyzing Unstructured Workflows
In this segment, we will delve deeper into unstructured workflows in the Context of our computer vision data set. We will discuss the challenges and considerations of working with unstructured data, particularly in the field of computer vision. By examining the data set and understanding the best practices, we will gain insights into what is happening with our data. We will explore techniques and approaches to effectively analyze and interpret unstructured data. This section will provide valuable guidance for data scientists and ML practitioners working with unstructured data and generative models.
Creating Custom Business Metrics
As data scientists and ML practitioners, it is crucial to associate our ML models with some sort of business value. In this section, we will explore the concept of custom business metrics and their significance in evaluating the impact of ML models on business outcomes. We will discuss the difference between statistical metrics and business metrics, emphasizing the importance of aligning ML goals with business objectives. We will demonstrate how to Create custom business metrics using SQL-like syntax. By establishing Meaningful business metrics, we can better showcase the value of ML models and make informed decisions for our organizations.
Programmatic Access with Arise
Arise offers programmatic access through SDKs, allowing users to perform various actions and access items within the platform programmatically. In this section, we will explore how to leverage Arise SDKs to push inference logs and data, integrating Arise seamlessly into your architecture. We will provide insights into the architectural Patterns and considerations when using Arise SDKs. Additionally, we will showcase how to automate actions and perform custom business metrics using Arise APIs. By the end of this section, you will have a comprehensive understanding of programmatic access with Arise and how to harness its power to optimize your ML observability capabilities.
Monitoring and Alerting with Graphql
In the final section of this workshop, we will explore how to set up monitors and alerts using Graphql. Graphql offers a powerful interface within Arise, enabling users to query and mutate data efficiently. We will demonstrate how to create performance monitors and configure auto-thresholding using the Graphql interface. By leveraging Graphql, you can automate monitoring processes and receive alerts Based on predefined thresholds. We will walk through examples and discuss the relevance of monitoring in ensuring the stability and performance of ML models.
That concludes our agenda for today's workshop. We hope you find these topics insightful and valuable in enhancing your ML observability capabilities with Arise. Feel free to ask questions as we proceed, and let's dive into the first topic: setting up a data pipeline in Arise.
Article:
Introduction
Hello everyone! Welcome to this workshop on advanced Arise workflows. My name is Dat, and I am a ML Solutions Architect here at Arise. In this workshop, we will be diving deeper into the world of Arise and exploring some more advanced user workflows. Whether you are already familiar with Arise or just getting started, this presentation will provide valuable insights into how to leverage the ML observability capabilities of Arise.
Throughout this workshop, we will cover various topics including setting up a data pipeline in Arise, understanding embeddings and detecting drift, analyzing unstructured workflows using computer vision data, creating custom business metrics, and utilizing programmatic access with Arise. We will also explore how to set up monitors and alerts using Graphql.
So let's get started with our agenda for today.
Setting Up a Data Pipeline in Arise
To begin, we will walk through the process of setting up a data pipeline in Arise. Whether you are working with data connectors or SDKs, Arise provides various ways to integrate with your current stack and ingest data. We will specifically focus on setting up a data pipeline using a data connector, specifically for a data bucket in an AWS S3 account. We will explore the necessary configurations and permissions required to establish this connection and push data into Arise. By the end of this section, you will have a clear understanding of how to set up a data pipeline in Arise and seamlessly integrate it with your existing architecture.
Understanding Embeddings
The next topic we will cover is embeddings and their importance in ML models. We will start by explaining what embeddings are and how they serve as representations of unstructured data. We will delve into the significance of embeddings and why they are foundational to most unstructured models. To better visualize and comprehend embeddings, we will utilize the UMAP algorithm to project embeddings into a 3D space. This will enable us to analyze embeddings drift and understand how embeddings change over time. We will then dive into various unstructured workflows using a computer vision data set. Throughout this section, we will discuss best practices and explore the space of generative models, which has gained significant attention in recent times.
Analyzing Unstructured Workflows
In this segment, we will delve deeper into unstructured workflows in the context of our computer vision data set. We will discuss the challenges and considerations of working with unstructured data, particularly in the field of computer vision. By examining the data set and understanding the best practices, we will gain insights into what is happening with our data. We will explore techniques and approaches to effectively analyze and interpret unstructured data. This section will provide valuable guidance for data scientists and ML practitioners working with unstructured data and generative models.
Creating Custom Business Metrics
As data scientists and ML practitioners, it is crucial to associate our ML models with some sort of business value. In this section, we will explore the concept of custom business metrics and their significance in evaluating the impact of ML models on business outcomes. We will discuss the difference between statistical metrics and business metrics, emphasizing the importance of aligning ML goals with business objectives. We will demonstrate how to create custom business metrics using SQL-like syntax. By establishing meaningful business metrics, we can better showcase the value of ML models and make informed decisions for our organizations.
Programmatic Access with Arise
Arise offers programmatic access through SDKs, allowing users to perform various actions and access items within the platform programmatically. In this section, we will explore how to leverage Arise SDKs to push inference logs and data, integrating Arise seamlessly into your architecture. We will provide insights into the architectural patterns and considerations when using Arise SDKs. Additionally, we will showcase how to automate actions and perform custom business metrics using Arise APIs. By the end of this section, you will have a comprehensive understanding of programmatic access with Arise and how to harness its power to optimize your ML observability capabilities.
Monitoring and Alerting with Graphql
In the final section of this workshop, we will explore how to set up monitors and alerts using Graphql. Graphql offers a powerful interface within Arise, enabling users to query and mutate data efficiently. We will demonstrate how to create performance monitors and configure auto-thresholding using the Graphql interface. By leveraging Graphql, you can automate monitoring processes and receive alerts based on predefined thresholds. We will walk through examples and discuss the relevance of monitoring in ensuring the stability and performance of ML models.
That concludes our agenda for today's workshop. We hope you find these topics insightful and valuable in enhancing your ML observability capabilities with Arise. Feel free to ask questions as we proceed, and let's dive into the first topic: setting up a data pipeline in Arise.
