Introducing Pom 2: Google's Next-gen Language Model
Table of Contents:
I. Introduction
II. Pom 2 - An Overview
III. Pathways Language Model
IV. Multilingualism in Pom 2
V. Improved Reasoning Capabilities
VI. Pre-training of Pom 2
VII. Four Sizes of Pom 2 Models
VIII. Products and Features Powered by Pom 2
IX. Examples of Pom 2’s Specializations
X. Fine-tuning Pom 2 for Domain-specific Knowledge
XI. Availability of Pom 2
XII. Future of AI with Pom 2
Pom 2 - An Overview
Google has recently announced the release of Pom 2, which is a next-generation language model designed with improved multilingual reasoning and coding capabilities. This model is faster, more efficient, and powers 25 first-party products and features in 2022.
Pathways Language Model
Pom 2 features the Pathways Language Model or Palm with a Pathways AI architecture that can train a single model to do thousands or millions of things. This new architecture allows for a significant improvement in the ability to understand, generate, and translate nuanced text like idioms, poems, and riddles.
Multilingualism in Pom 2
Pom 2's data set includes scientific papers and web pages with mathematical expressions for improved capabilities in logic, commonsense reasoning, and mathematics. Furthermore, Pom 2 has been more heavily trained on multilingual text across over 100 languages. This results in a significantly improved ability to understand and generate nuanced text.
Improved Reasoning Capabilities
The Pom 2 model is trained on scientific and mathematical topics, which makes it stronger in logic and reasoning. It can also generate specialized code in prologue, Fortran, and verilog.
Pre-training of Pom 2
Lastly, Pom 2 was pre-trained on a large quantity of public source code data sets besides Python and JavaScript. It will be available in four sizes, with animal-inspired naming – Gecko, Otter, Bison, and Unicorn.
Four Sizes of Pom 2 Models
Gecko can process 20 tokens per Second and has been tested on the latest phones. Although Google didn't specify which particular devices, at i/o 2023, Google shared that Pom 2 is powering 25 products, including Bard for coding, expansion to Korean and Japanese, and med-Pom 2 to answer questions and summarize insights from dense medical Texts and Sec-Palm for cybersecurity.
Products and Features Powered by Pom 2
Palm 2 delivers excellent foundational capabilities across a wide range of sizes, including Gekko, Otter, Bison, and Unicorn. Gekko is so lightweight that it can work on mobile devices, fast enough for great interactive applications on the device, even when offline.
Examples of Pom 2’s Specializations
Pom 2 models combined with powerful coding capabilities can also help developers collaborate around the world. For instance, if You're working with a colleague in Seoul and debugging code, you can ask Pom 2 to fix a bug, add comments, and query the code. Pom 2 recognizes the code as recursive, suggests affects and explains the reasoning behind the fix. Furthermore, Pom 2 can communicate with a radiologist to interpret images and communicate the results.
Fine-tuning Pom 2 for Domain-specific Knowledge
Once Pom 2 is highly capable, it can be fine-tuned on domain-specific knowledge. For example, SECOM is a version of Palm to fine-tune for security use cases and better detect malicious scripts while metform2 is fine-tuned on medical knowledge. This fine-tuning achieved a 9x reduction in inaccurate reasoning when compared to the model, approaching the performance of clinician experts. In fact, metform2 was the first language model to perform an expert level on medical licensing exam style questions, and it is currently the state of the art.
Availability of Pom 2
Pom 2 is now available in preview, and it is the latest step in Google's decade-long Journey to bring AI in responsible ways to billions of people.
Future of AI with Pom 2
Pom 2 offers exciting possibilities to both developers and consumers alike. This language model can help in cross-border collaborations, fine-tuning for specific industries, and improving cybersecurity measures. With a focus on multilingualism and improved reasoning capabilities, Pom 2 opens doors to new applications that were merely unimaginable before.
Highlights:
- Pom 2 is a next-generation language model designed with improved multilingual reasoning and coding capabilities
- The model is pre-trained on a large quantity of public source code datasets besides Python and JavaScript
- Pom 2 is available in four sizes, with animal-inspired naming – Gecko, Otter, Bison, and Unicorn
- Pom 2 is now available in preview
- Fine-tuning Pom 2 for domain-specific knowledge achieved a 9x reduction in inaccurate reasoning compared to the model
- Palm 2 delivers excellent foundational capabilities across a wide range of sizes
FAQ:
Q: What are the four sizes of Pom 2 models?
A: Pom 2 models are available in four sizes, with animal-inspired naming – Gecko, Otter, Bison, and Unicorn.
Q: Can Pom 2 work on mobile devices?
A: Yes, the Gecko model of Pom 2 is so lightweight that it can work on mobile devices, even when offline.
Q: What are some applications of Pom 2 in specialized domains?
A: Pom 2 can be fine-tuned for specific industries, such as in medical knowledge and cybersecurity use cases.