Using AI to preserve the languages of the world, run AI chat on your phone in private, generate truly multi-modal content with CoDI and numbers every LLM developer should know about.
Welcome to week 21 of The Generative Edge. Here is the gist in 4 bullet points:
Meta AI's MMS project supports over 1100 languages, significantly improving speech recognition technology and promoting global language diversity.
Open-source Large Language Models (LLMs) enable personalized AI chat applications on consumer devices like smartphones, addressing privacy and availability concerns.
Microsoft Research's CoDI is a multimodal generative model capable of handling various input-output combinations, including language, image, video, and audio.
A while back, OpenAI released Whisper, a multilingual text transcription model that is open source and state-of-the-art, at the same time Elevenlabs is pushing the voice generation frontier with life-like artificial voice cloning.
Last week, Meta AI has released a project called MMS (Massively Multilingual Speech), designed to address the limitations of speech recognition technology in terms of language coverage.
The preservation of cultural diversity is certainly a novel and interesting application of modern AI technology.
Supports over 1100 languages, including highly obscure rarely spoken languages
The model can transcribe as well as generate speech (think Whisper, and Elevenlabs).
Among other things, it utilizes old religious texts for diverse language coverage.
Outperforms existing models, covers 10x more languages.
As of now, OpenAI's GPT-4 is the best language model available. Nevertheless, the platform is closed-sourced, only accessible via API, and cannot be hosted internally.
As a result, legitimate concerns arise regarding privacy, availability, bias, etc. We've mentioned before that a large community effort is underway to create, modify, and improve open source language models.
Some of which you can now run in the browser or even on consumer devices like smartphones:
Open-source Large Language Models (LLMs) have proliferated greatly
Exciting opportunities for end users to explore and leverage personalized open LLMs
RedPajama is an example of open-source, high-performing LLMs that can be run at the edge, e.g. on Apple Silicon, AMD/NVIDIA GPUs, WebGPU, and iOS devices
Running AI chat locally on the phone
This could lead to a proliferation of personalized fine-tuned models.
Multi-model models, which can take and output a variety of inputs, have appeared on the scene. From the elusive, yet-to-be released multi-modal capabilities of GPT-4, to TaskMatrix and others. Microsoft Research has now pushed things further and released CoDI, a model that can handle multimodal input and generate combined multi modal outputs.
Generates language, image, video, audio from any input combination.
Not limited to specific input types like existing AI systems.
Works with input-output combinations not in training data.
Examples:
Input: Image of a creek and sound of birds
1×
0:00
-0:09
Output (sounds of water and birds):
1×
0:00
-0:10
Input: Image and Text
Eating on a coffee table
Output: Video
Delivers results on par with or better than specialized single-format models
