Why do only a small percentage of GenAI projects actually make it into production?

June04, 2024

Jun 04, 2024

Only about 5% of GenAI projects lead to significant monetization of new product offerings.

Google Cloud’s director of AI/ML and generative AI, Miku Jha, estimated that in only about 15% of AI adoptions does the organization have a clear idea of what it wants to accomplish using GenAI

For only about 5% of cases, will GenAI projects lead to significant monetization of new product offerings.

Challenges include

Integrating traditional AI, GenAI’s specific use cases,
Successful GenAI projects focus on solving business problems, integrating other technologies, involving humans, and using reliable data.
Organizations often face misguided expectations and must critically assess their business goals to effectively leverage GenAI.

Building Meta’s GenAI Infrastructure

24k GPU clusters

Meta’s long-term vision is to build artificial general intelligence (AGI),that is open and built responsibly

The newer AI clusters build upon the successes and lessons learned from RSC

The efficiency of the high-performance network fabrics within these clusters, some of the key storage decisions, combined with the 24,576 NVIDIA Tensor Core H100 GPUs in each

Built one cluster with a remote direct memory access (RDMA) over converged Ethernet (RoCE) network fabric solution based on the Arista 7800 with Wedge400 and Minipack2 OCP rack switches

Social learning: Collaborative learning with large language models

Framework for social learning in which LLMs share knowledge with each other in a privacy-aware manner

Language models have shown a remarkable capacity to perform tasks given only a handful of examples–a process called few-shot learning. With this in mind, we provide human-labeled examples of a task that enables the teacher model to teach it to a student. One of the main use cases of social learning arises when these examples cannot be directly shared with the student due, for example, to privacy concerns.

To illustrate this, let’s look at a hypothetical example for a spam detection task. A teacher model is located on-device where some users volunteer to mark incoming messages they receive as either “spam” or “not spam”. This is useful data that could help train a student model to differentiate between spam and not spam, but sharing personal messages with other users is a breach of privacy and should be avoided. To prevent this, a social learning process can transfer the knowledge from the teacher model to the student so it learns what spam messages look like without needing to share the user’s personal text messages.

DigitalDrip

Discussion about this post