The Battle for Generative AI: Cloud vs. On-Prem

Table of Contents

1. Introduction
2. Overall Spending Climate
3. The Adoption of Large Language Models
4. Spending on Generative AI
5. Challenges and Concerns about Generative AI
6. Gen AI Usage in Production Environments
7. Business Case and ROI of Gen AI
8. Deployment Options: On-prem vs Cloud
9. Cloud Capabilities for AI
10. Market Presence of Cloud Players
11. Revenue Growth of Cloud Players
12. Power Law Distribution of Large Language Models
13. Success Factors for On-prem Incumbents
14. The Role of Enterprise AI and Headcount Reduction
15. Edge Inference and its Disruptive Force
16. Conclusion

Article

Introduction

The adoption of large language models and generative AI has been a topic of discussion among enterprise customers. The data shows that while 94% of customers are spending more on AI this year, they are doing so with budget constraints that may impact other initiatives. Moreover, the choice between public cloud and on-premises/edge deployments for AI usage is split almost equally. This article will Delve into the factors that organizations need to consider when adopting large language models and address the challenges and concerns surrounding its implementation.

Overall Spending Climate

Before delving into the specifics of large language models, it's important to understand the overall spending climate for enterprise technology budgets. Senior IT decision makers had initially expected a budget increase between four and 5%, but the actual increase has been lower, currently standing at 2.9%. Budget constraints, coupled with the urgency to adopt generative AI, have forced organizations to reprioritize their spending. The spending velocity for AI has decreased, and some sectors, like the AI segment, have been particularly impacted by budget constraints.

The Adoption of Large Language Models

While the spending velocity for AI has decreased, the adoption of large language models, specifically generative AI, has seen an acceleration. A majority of customers (94%) actively investing in generative AI report an increase in their AI spending for 2023. Around 36% of these customers expect their spending to increase by double digits. This top-down pressure to adopt generative AI is driven by the belief that it will bring value and productivity to organizations. However, the actual implementation and evaluation of generative AI pose challenges and complexities.

Challenges and Concerns about Generative AI

Organizations face several challenges and concerns when it comes to adopting and implementing generative AI. A study by tech analysis firm highlights the top concerns of IT decision makers, including compliance, IP leakage, copyright infringement, bias, and data and tools quality. These concerns highlight the need for caution and careful consideration when using generative AI. As a result, many organizations are opting for on-premises deployment to mitigate these risks. While public cloud offers attractive features and tooling, concerns about data privacy and security drive the demand for private infrastructure.

Gen AI Usage in Production Environments

When looking at the actual usage of generative AI in production environments, it becomes evident that most organizations are still in evaluation mode. The use cases for generative AI are primarily centered around chatbots, code generation, text summarization, and writing marketing copy. These use cases reflect the straightforward applications of generative AI in improving productivity and efficiency. However, for generative AI to be successful in production environments, organizations need a clear understanding of the business case and ROI.

Business Case and ROI of Gen AI

The success of generative AI lies in establishing a clear business case and identifying its return on investment. The primary driver of ROI for generative AI is the reduction in labor costs. By leveraging generative AI to automate tasks and processes, organizations can minimize their headcount requirements. This shift in value creation will undoubtedly impact the future job market and the skills needed for employment. Organizations must carefully evaluate and quantify the economic value of generative AI to ensure its long-term viability.

Deployment Options: On-prem vs Cloud

The choice between on-premises and cloud deployment is a critical factor in the adoption of large language models. While the public cloud offers superior tooling and ease of integration for AI, concerns about data security and IP leakage drive organizations to choose on-premises deployment. The data shows that organizations have an equal mix of private and public infrastructure deployments. The allure of the cloud, coupled with the availability of data, leads developers to favor cloud-Based AI capabilities. However, private infrastructure is expected to be in demand due to the need for enhanced security and control.

Cloud Capabilities for AI

Developers report that the public cloud offers several capabilities that drive the adoption of AI. These capabilities include the pace of innovation in AI, the simplicity of integration, and the productivity it enables for developers. Additionally, the public cloud provides model optionality and diversity, ensuring access to various AI models and tools. Developers also value the ability to secure data and ensure privacy by leveraging cloud tools and governance choices. Cloud providers like AWS, Microsoft, and Google have established their presence in the AI market, offering reliable and scalable AI capabilities.

Market Presence of Cloud Players

When looking at the market presence of cloud players in the AI space, AWS, Microsoft, and Google dominate in terms of spending velocity. These cloud providers have a significant number of new customer logos and a strong market presence. In comparison, on-prem incumbents like Dell and HPE have lower spending velocity, indicating a slower adoption of AI. While Dell and HPE have a large market presence, the cloud players still hold a significant AdVantage in terms of spending Momentum.

Revenue Growth of Cloud Players

Analysts forecast varying revenue growth for cloud players in the AI market. AWS is expected to experience moderate growth, with a 14% increase in Q4 due to the tailwind of AI and generative AI. However, the macro environment and competition pose risks to this Scenario. While there are Talks of cloud optimization and repatriation, the numbers favor the cloud players, indicating their sustained growth in the market. This growth is driven by the combined forces of AI, data, and volume economics.

Power Law Distribution of Large Language Models

Large language models follow a modified power law distribution, indicating that a few companies dominate the creation of these models. We believe that the enterprise tech innovation is influenced by consumer volumes and the dominance of big cloud and consumer brands. The large internet giants will play a significant role in the creation and running of large language models, while open-source tools and third-party vendors will fill the gap. On-prem incumbents like Dell, HPE, and IBM can succeed if they leverage large language models and build a comprehensive ecosystem. However, the economics of edge inference will disrupt the enterprise IT landscape in the long run.

Conclusion

The adoption of large language models and generative AI presents both opportunities and challenges for enterprise customers. Budget constraints and concerns about compliance, security, and IP leakage influence the adoption and deployment decisions. The business case for generative AI lies in minimizing labor costs, which may redefine the employment landscape. While the cloud offers superior AI capabilities, on-premises deployment provides enhanced security and control. Cloud players dominate the market in terms of spending momentum and revenue growth. The future of AI and large language models will be Shaped by consumer volumes, data, and the economic value they bring to enterprises.