The Generative Edge Week 19
StarCoder could make you a better programmer, the family of open LLMs is growing larger, AI powered David Attenborough speaks perfect German and be careful what you name your AI tool.
Welcome to week 19 of The Generative Edge. Here is the gist in 5 bullets:
StarCoder is a new open LLM for coding, trained on code in 80+ programming languages
New open LLMs MPT-7B and RedPajama are released, including a storywriter variant which was trained on books and has a giant context window
Elevenlabs releases a multi-lingual generative AI voice model supporting 8 languages and showcasing the potential future of voice commoditization
OpenAI is addressing GPT "trademark" infringement, DEFCON is set to challenge hackers with LLMs, and questions about ChatGPT's GDPR compliance arise.
For all the details, let’s hop in!
StarCoder opens up AI assisted programming
Developers are embracing AI-assisted coding left and right, and while there can be significant caveats, productivity tends to soar when using these code models. Prominent AI coding solutions like OpenAI Codex, GitHub Copilot, CodeT5, and DeepMind's AlphaCode. None of the large models can be operated privately or locally, but now there is an option for that:
StarCoder is a new LLM trained on code in 80+ programming languages
Try it easily here: huggingface.co/chat/
It can integrate into VSCode as an alternative to Copilot
The model can be used commercially, it can be hosted internally and used inside of CI/CD stacks and such - that’s a big benefit, especially for privacy sensitive organisations.
More open LLMs hit the scene
There is no shortage of small, open language models these days, we’ve talked about them previously However, a lot of the existing models are marred in licensing issues, and either they can only be used for research purposes but not commercially, or they’ve been trained on problematic data.
This week we saw not just one, but two open model families release: MTP-7B and RedPajama.
MosaicML releases MPT-7B, a transformer model trained on 1T tokens of text and code, which is open and available for commercial use.
It can be trained, fine-tuned, and deployed privately and locally.
MPT-7B comes in different variants, including MPT-7B-Instruct, MPT-7B-Chat, and MPT-7B-StoryWriter-65k+. The latter has a context length of 65k tokens (which is a lot!), enabling it to generate longform stories.
Another model that was released is RedPajama, a family of models in various sizes
Don’t expect ChatGPT quality output from these models out of the box (yet), but they are an important stepping stone in integrating this technology into your tech stach while retaining full ownership.
Elevenlabs goes multi-lingual
As we’ve mentioned before, generative AI voice technology has come a long way. Elevenlabs is probably the best commercial vendor offering voice cloning right now and they’ve recently released their new multi-lingual model.
Ever wondered what David Attenborough sounds like if he spoke fluent German? Well, wonder no more:
It works surprisingly well, and while it has some rough edges here and there, you can absolutely see where this will be going.
Combined with other advances in this sector, expect voice to be commoditized in the same way as digital art.
… and what else?
OpenAI is kind of, sort of going after GPT “trademark” infringement so keep that in mind when naming your new shiny AI products, infamous hacking conference DEFCON to set thousands of hackers loose on LLMs, is ChatGPT GDPR compliant?
And that’s it for this week!
Find all of our updates on our Substack at thegenerativeedge.substack.com, get in touch via our office hours if you want to talk professionally about Generative AI and visit our website at contiamo.com.
Have a wonderful week everyone!
Daniel
Generative AI engineer at Contiamo