Home AI News Unlock the Power of AI Integration in the Browser with AirJS SDK

Unlock the Power of AI Integration in the Browser with AirJS SDK

Introduction
The Last Mile of AI Integration
1. The Problem
2. Model Optimization
3. Challenges in AI Integration
Getting AI Models in the Browser
1. Introduction to AirJS SDK
2. Chrome Extension for Model Integration
  1. Drag and Drop Interface
  2. Model Playlist
  3. Customization Options
Deep Dive into AirJS SDK
1. Harvesting Data from the DOM
2. Pre-processing Data
3. Inference Engine
4. Post-processing Results
5. Rendering Model Outputs
Experimenting with the Extension
1. Installation and Setup
2. Adding and Running Models
3. Customizing Model Integration
4. Sharing and Removing Models
Conclusion

📰 Article: Enabling AI Integration in the Browser for Seamless Model Deployment

Every day, artificial intelligence (AI) is becoming increasingly powerful and prevalent. From assisting us with complex tasks to making intelligent predictions, AI has the potential to revolutionize multiple industries. However, one challenge many organizations face is the last mile of AI integration, which involves getting AI models into the hands of users in a useful manner. This article will explore how AI models can be deployed in the browser while customizing how insights from these models are displayed over existing web applications.

The Last Mile of AI Integration

The Problem

The last mile of AI integration refers to the final step of deploying AI models and making them accessible to end users. It involves addressing challenges related to model optimization and ensuring that the models can be used effectively in an operational workflow. However, this step often poses several difficulties, such as the complexity of model deployment and the need for specialized hardware and computational power.

Model Optimization

To address the challenges of the last mile of AI integration, the AI Squared team has developed a model optimization framework called AI Squared. This framework supports the optimization of AI models for deployment in the browser. By using the AI Squared framework, organizations can arrive at a version of their model that can be effectively used in the browser.

Challenges in AI Integration

The challenges involved in the last mile of AI integration are vast and varied. One of the main challenges is the complexity of many state-of-the-art model types, such as neural networks. These models can have millions or even hundreds of millions of parameters, making their deployment and utilization extremely difficult. Additionally, the traditional methodology for deploying AI models involves multiple moving parts and can take weeks or even months to complete.

Getting AI Models in the Browser

To overcome the challenges of the last mile of AI integration, the AI Squared team has developed the AirJS SDK and an accompanying Chrome extension. The AirJS SDK enables the integration of AI models in the browser and provides users with a drag and drop interface for model deployment. With the AirJS SDK, users can easily customize how their models are used and displayed within existing web applications.

Introduction to AirJS SDK

The AirJS SDK is a powerful JavaScript library that serves as an inference engine for AI models. It supports the implementation of models locally in the browser and provides seamless integration for TensorFlow.js models, with support for scikit-learn models coming soon. The SDK also offers the capability to invoke models deployed on remote endpoints or even pre-computed model results.

Chrome Extension for Model Integration

The AirJS SDK is accompanied by a user-friendly Chrome extension that streamlines the process of model integration. The extension provides users with a model playlist, which allows them to easily manage and switch between different models. With the extension's drag and drop interface, users can effortlessly add their models and configure them to suit their needs.

Drag and Drop Interface

Adding models to the extension is as simple as dragging and dropping the model's dot air file into the interface. Once added, the model is immediately available for integration into web applications. The extension supports various model types, including computer vision models and natural language models.

Model Playlist

The model playlist within the extension allows users to have multiple models at their fingertips. Users can easily switch between models and experiment with different AI capabilities. The extension stores all the necessary information and metadata about the models, making it easy to manage and organize.

Customization Options

One of the key features of the AirJS extension is its customization options. Users can edit and adjust various aspects of model integration, such as data harvesting, pre-processing steps, model inference, and post-processing of results. This level of customization helps tailor the AI experience to specific requirements and use cases.

Deep Dive into AirJS SDK

To further understand the capabilities of the AirJS SDK, let's explore its key components and functionalities in more detail:

Harvesting Data from the DOM

The AirJS SDK provides a streamlined way to scrape data from the Document Object Model (DOM) of web pages. Using the SDK, developers can easily define selectors and retrieve Relevant data for model inputs. This capability allows for real-time data acquisition for seamless integration with AI models, ensuring up-to-date insights.

Pre-processing Data

Pre-processing of data plays a vital role in preparing inputs for AI models. The AirJS SDK supports various pre-processing techniques to ensure the compatibility and quality of data before feeding it into the models. These techniques can include resizing images, normalizing inputs, or applying any form of data transformation required by the specific model.

Inference Engine

The AirJS SDK acts as a powerful inference engine, capable of invoking AI models directly within the browser. It seamlessly integrates with TensorFlow.js models and provides support for invoking pre-trained models deployed on remote endpoints. The SDK handles all the necessary computations and optimizations required to run models efficiently and effectively.

Post-processing Results

After obtaining model predictions, post-processing steps may be necessary to interpret and display the outputs appropriately. The AirJS SDK offers flexibility in defining post-processing routines, allowing users to customize how the model's results are presented. This step ensures that the insights derived from AI models are easily understandable and actionable.

Rendering Model Outputs

The AirJS SDK provides a user-friendly approach to rendering model outputs within web applications. Developers can easily define the rendering options, such as displaying bounding boxes around objects or adding labels to images. This feature enables users to Visualize AI model outputs directly in the browser, making it easier to interpret and analyze the results.

Experimenting with the Extension

Now that we have explored the capabilities of the AirJS SDK, let's walk through the installation and usage of the Chrome extension for model integration:

Installation and Setup

To get started, download the provided Chrome extension build and unzip it. Then, load the extension manually by navigating to the extensions management page in Chrome and selecting the unzipped build folder.

Adding and Running Models

Once the extension is installed, you can add your customized dot air models to the model playlist. Simply drag and drop the dot air file into the extension interface or use the file selection feature. The model will appear in the playlist, ready to be run on web pages.

Customizing Model Integration

The extension offers a convenient way to customize model integration. By clicking the "Edit Model" button, users can modify various configuration elements, such as data harvesting methods, pre-processing steps, and rendering options. All changes take effect immediately and can be saved or removed as desired.

Sharing and Removing Models

If you want to share your customized model configuration with others, the extension allows you to download the modified dot air file. Conversely, if you wish to remove a model from the extension, simply click the "Remove" button associated with that model.

Conclusion

The integration of AI models into web applications has never been easier, thanks to the AI Squared team's work on the AirJS SDK and Chrome extension. This powerful combination enables seamless model deployment and customization directly in the browser. By utilizing the extension's drag and drop interface, users can easily manage and experiment with different AI models. With the AirJS SDK, developers have full control over data harvesting, pre-processing, model inference, and post-processing, making it a versatile solution for AI integration. By breaking down the barriers of the last mile of AI integration, organizations can effectively bring AI capabilities to users and empower them with valuable insights.

We encourage you to explore the possibilities of AI integration in the browser using the AirJS SDK and extension. Whether you are an experienced AI practitioner or just starting your journey, the AI Squared team is here to support you. Feel free to reach out for further guidance or to become a beta tester for upcoming releases. Together, we can Shape the future of AI integration and unlock its full potential.

🔗 Resources:

AI Squared: https://aisquared.ai/
AirJS SDK Documentation: https://docs.square.ai/airjs-sdk/
Chrome Extension Build: [Link shared during the workshop]

Highlights:

The Last Mile of AI Integration: Challenges in deploying AI models effectively
Introducing the AirJS SDK: Enabling AI model integration in the browser
Customization Options: Tailoring AI integration to specific requirements
Real-time Data Acquisition: Scraping data from the DOM for model inputs
Streamlined Pre-processing: Transforming and normalizing data for AI models
Seamless Inference Engine: Invoking AI models in the browser with optimized performance
User-Friendly Rendering: Visualizing model outputs within web applications
Experimenting with the Chrome Extension: Adding, customizing, and running AI models in the browser
Empowering Users with Valuable Insights: Breaking down barriers in the last mile of AI integration

FAQ

Q: Can I use my own AI model with the AirJS SDK? A: Absolutely! The AirJS SDK allows you to integrate and run your own AI models in the browser, providing a customizable and user-friendly experience.

Q: Is the AirJS SDK limited to specific model types? A: No, the AirJS SDK supports various model types, including computer vision models, natural language models, and more. It provides flexibility in integrating different AI capabilities into web applications.

Q: Is the Chrome extension compatible with other browsers? A: Currently, the AirJS Chrome extension is designed for Google Chrome. However, we are continually exploring options to expand compatibility with other browsers in the future.

Q: How can I share my customized model configuration with others? A: The AirJS extension allows you to download the modified dot air file containing your customized model configuration. You can then share this file with others who can import it into their extension.

Q: Can I use the extension in conjunction with other AI frameworks? A: Yes, the AirJS extension can be used alongside other AI frameworks. It provides seamless integration for TensorFlow.js models and supports invoking models deployed on remote endpoints.

🔗 Resources: