The Generative Edge Week 14
Financial AI with Bloomberg, did Google's Bard copy ChatGPT's homework, the Llama family keeps growing and generative 3D imagery learns how to walk.
We are back for week 14 of The Generative Edge. After an incessant torrent of news over the last few weeks, things have slowed down this week. Expect this to pick up again after Easter, but for now let’s check out what happened this last week.
Bloomberg GPT
There is a lot of development around language models that are customized and specialized for certain domains. Last week saw the release of an article by Bloomberg announcing Bloomberg GPT, a model trained on financial text data.
Bloomberg’s model is a 50 Billion parameter model (compared to ChatGPT’s 175B, and an unknown yet probably comparable size for GPT4).
It has been trained on financial text data from Bloomberg's financial data sources.
What would it be used for? Entity extraction, sentiment analysis, few-shot learning, text generation, conversational systems.. anything that other LLMs can do, but with extra expertise in the financial domain
The model hasn’t been released, so we cannot make any real statements about its performance. However, giving these large models a specialization seems sensible and is in line with what we’ve seen over the last weeks.
This may indicate a shift away from extremely broad foundational models towards more narrow specialist models
Did Google train Bard on ChatGPT?
As we’ve mentioned last week, Google opened Bard for early access and it has received mixed reviews. To add insult to injury, there is some evidence that Google used ChatGPT output to improve their own model.
Using output from stronger models is a functional way to imbue a lesser model with more capabilities. We know this works.
Allegedly, a senior AI researcher within Google warned leadership that Bard training data included ChatGPT output.
That output came from the popular ChatGPT sharing website ShareGPT which has since restricted access to ChatGPT logs.
While there may be license issues with the approach (OpenAI prohibits using their model output to train a competitor), the big news here is that Google is allegedly using data from a competitor to bolster their own product - not a good look.
Google has since denied the allegations, though a certain taste remains.
A new addition to the Llama family
Meta’s Llama continues to spur community development and a veritable zoo of llama derivatives. We’ve seen Alpaca by Stanford, Dolly by Databricks and now there's Vicuna.
It has been trained on ChatGPT output via the ShareGPT website (yes, the same thing Google is accused of doing with Bard).
This model seems to work exceptionally well, demonstrating the impact of transfer learning between models
Expect more specialized, custom models to enter the discussion soon.
As with the other Llama derivatives, you can also run this on your local machine. Check out OpenPlayground if you feel adventurous. It allows you to compare all the various language models, remote, local or otherwise.
A little 3D in your generated images?
As always, the generative image space marches ahead. We have seen a few published papers in the last weeks about creating 3D images.
This paper discusses creating 3D images by using the same 2D diffusion mechanism that also powers Stable Diffusion, Midjourney and Dall-E
Generating pseudo 3D images like that can have many applications, from e-commerce, video games to augmented reality.
… and what else?
Italy bans ChatGPT (and an alternative emerges), an open letter demands to stop AI development which turns out to be mostly hot air and Stanford releases a massive 300+ page report on the state of AI
And that’s it for this week! Find all of our updates on our Substack at thegenerativeedge.substack.com.
Get in touch via our office hours if you would like to talk professionally about Generative AI and visit our website at contiamo.com.
Have a wonderful week everyone!
Daniel
Generative AI engineer at Contiamo
