Revolutionizing Writing: Voice Composer Converts Speech into Polished Text

Table of Contents:

1. Introduction
2. The Big Picture of Voice Composer
3. The Writing Process and Grammarly's Role
4. Ideation Phase: Generating Ideas and Outlining
5. Drafting Stage: Getting Ideas on Paper
6. Revision Stage: Reviewing and Reviving Drafts
7. Editing Stage: Rewriting and Making Adjustments
8. Composition and Revision: Grammarly's Impact
9. Voice Composer's Features and User Requirements
10. How Voice Composer Works: An Overview
11. Challenges and Learnings in Developing Voice Composer
12. Evaluation Framework: Human Judgments and Metrics
13. Scaling Up and Optimizations for Voice Composer
14. Conclusion

Introduction

Voice Composer is a powerful AI-powered tool developed by Grammarly that allows users to convert conversational speech into polished and accurate compositions. In this article, we will explore the features, development process, and optimization techniques used in building Voice Composer. We will also delve into the challenges faced and takeaways from this project.

The Big Picture of Voice Composer

Voice Composer is designed to address a common challenge in the writing process: composition. While Grammarly has traditionally focused on revision and error correction, Voice Composer aims to assist users in the actual creation of written content. By leveraging advanced Speech Recognition and natural language processing algorithms, Voice Composer can transform conversational speech into well-formed and grammatically correct text.

The Writing Process and Grammarly's Role

To understand the motivation behind creating Voice Composer, we must first examine the typical writing process. The writing process involves several stages, including ideation, planning, drafting, revision, and editing. Grammarly has played a significant role in the revision stage, helping users correct grammar and spelling errors. However, the composition stage requires a considerable amount of time and cognitive effort.

Ideation Phase: Generating Ideas and Outlining

In the ideation phase, writers generate ideas, Gather information, and exclude potential topics. This phase sets the foundation for the writing process. Voice Composer aims to streamline this phase by enabling users to dictate their ideas or instructions, which will be converted into a written format.

Drafting Stage: Getting Ideas on Paper

The drafting stage is where users begin writing their content based on the Outline or plan created in the previous phase. During this stage, the focus is on getting the ideas on paper without worrying too much about grammar or perfection. Voice Composer allows users to dictate their content and convert it into a draft, providing a starting point for further refinement.

Revision Stage: Reviewing and Reviving Drafts

Once the initial draft is complete, the next step is the revision stage. Here, writers review and revise their drafts for Clarity, coherence, and flow. Grammarly has been instrumental in helping users in this stage, providing feedback and suggestions for improvement. With Voice Composer, users can receive assistance in refining their drafts, making the revision process more efficient.

Editing Stage: Rewriting and Making Adjustments

After revising the draft, the editing stage focuses on making significant structural changes and minor adjustments to ensure the text is ready to be sent. Writers may rewrite entire sections, fix grammar errors, and fine-tune their content to achieve the desired impact. Voice Composer can assist users in this stage by suggesting alternative phrasing and providing a fresh perspective on the text.

Composition and Revision: Grammarly's Impact

While Voice Composer primarily focuses on the composition stage, it is worth noting that Grammarly's revision capabilities have played a crucial role in the writing process. By offering feedback on grammar, spelling, and style, Grammarly has helped users enhance their writing and improve the overall quality of their drafts. Voice Composer complements Grammarly's revision capabilities by streamlining the composition stage.

Voice Composer's Features and User Requirements

Voice Composer caters to various user requirements by offering support for different writing applications, such as email, messaging, and note-taking. It provides both instructional and dictation input modes, allowing users to choose the most convenient option. Additionally, Voice Composer ensures flexibility by supporting both open-ended and closed-ended prompts, catering to different levels of information containment.

How Voice Composer Works: An Overview

Voice Composer utilizes advanced speech recognition and language comprehension models to transform spoken input into well-formed text. The system follows a multi-stage pipeline, including automatic speech recognition, normalization, and comprehension. It leverages pre-trained models and fine-tuned models to handle speech disfluencies, restore punctuation, and correct grammatical errors. Through this process, Voice Composer generates structured and coherent output, ready for further refinement.

Challenges and Learnings in Developing Voice Composer

Developing Voice Composer came with its fair share of challenges. Finding the right product experience and iterating based on user feedback was critical to improving the technology and ensuring a seamless user experience. Balancing trade-offs between accuracy and efficiency was also challenging, as user preferences and writing styles evolved over time. Additionally, handling errors that cascade through the pipeline and ensuring model robustness were ongoing challenges in the development process.

Evaluation Framework: Human Judgments and Metrics

Evaluating the performance of Voice Composer required a comprehensive framework that combined automated metrics and human judgments. While automated metrics helped measure certain aspects, such as grammaticality and coherence, human judgments played a crucial role in assessing naturalness and overall quality. Addressing subjectivity and variability in human judgments required careful guideline development, iterative testing, and quality checks. The evaluation framework evolved over time, ensuring comprehensive coverage and accurate results.

Scaling Up and Optimizations for Voice Composer

Scaling up Voice Composer to serve a large user base required optimizations at various levels. Choosing the optimal instance type for hosting the service, such as AWS G5 instances, balanced performance and cost-effectiveness. Leveraging the ONNX runtime library improved inference latency for the in-house model. However, the overall performance was more favorable using the native PyTorch model for the given latency and throughput requirements. Additionally, comparing the performance of in-house models and third-party hosted models highlighted the benefits of a hybrid system, ensuring quality and throughput while leveraging the strengths of each model.

Conclusion

Voice Composer is a powerful tool developed by Grammarly to assist users in the composition stage of the writing process. By leveraging advanced AI algorithms, Voice Composer converts conversational speech into well-formed and grammatically correct text. Through optimizations and a comprehensive evaluation framework, Grammarly ensures that Voice Composer meets the high standards of quality and efficiency expected by users. With ongoing enhancements and improvements, Voice Composer continues to evolve to cater to the diverse needs of writers worldwide.

Highlights:

• Voice Composer revolutionizes the composition stage of the writing process, allowing users to convert conversational speech into well-formed text.
• Grammarly's revision capabilities have traditionally focused on the revision stage, but Voice Composer complements these by streamlining the composition stage.
• The development of Voice Composer involved addressing challenges such as finding the right product experience and adapting to evolving user preferences.
• Evaluating the performance of Voice Composer required a comprehensive framework that combined automated metrics and human judgments.
• Optimizations, including instance type selection, model-level optimizations, and hybrid system design, have enabled the scaling up of Voice Composer to serve a large user base.

Resources:

• Grammarly
• AWS Elastic Container Service
• Microsoft Azure Speech Recognition
• ONNX Runtime