The Generative Edge by Contiamo

Share this post

The Generative Edge Week 15

thegenerativeedge.substack.com

The Generative Edge Week 15

Generative video continues to blow our minds, will language models finally be set free, segmenting anything and Westworld meets The Sims.

Contiamo
Apr 11, 2023
Share
Share this post

The Generative Edge Week 15

thegenerativeedge.substack.com

Welcome to The Generative Edge for week 15. Generative media has seen some exciting new things this week, as well as more projects pushing for truly open and free AI. Let’s dive right in!

Generative video is getting here fast!

Just a couple of weeks ago we discussed the first tentative steps into generative video, followed by even longer coherent generated videos just a week after that.

Things have really started to shape up and generative video has turned from a fun novelty meme machine to something that is much closer to being ready for prime time. Watch the following video, still plenty of weird AI artefacts, but so so much better than what came before:

  • RunwayML is a research lab focusing on multimodal creative tools

  • They have just released their generation 2 generative video model (paper here)

  • Runway’s approach involves adding depth information and a few other tricks, to really improve on the generated video output

    Multimodal generative video
  • From janky, barely recognizable toy to something that is actually close to usable - all in the matter of a few weeks.

  • As we said before, expect this to really blow up in the near future - this stuff will democratize video/movies like nothing else, and we will lead to an explosion of content.

A truly open and free LLM chatbot

Alternative models to GPT3/4 have been popping up, but they are either burdened by license restrictions or can only be queried via an API (and don’t quite stack up to ChatGPT). Open Assistant could be the answer.

  • Open Assistant is a project meant to give everyone access to a great chat-based large language model.

  • The vision of the project is to make a large language model that can run on a single high-end consumer GPU.

  • The project aims to collect high-quality human-generated Instruction-Fulfillment samples (the sort of dialog you have with ChatGPT) through a crowdsourced process

    Gamification and crowdsourcing

    You can already check out the model, as well as the gamified data collection process here open-assistant.io/dashboard

  • Keep an eye on this project, it might be the first that gives us a truly open, truly free LLM chatbot.

Generate images by segmenting anything

Last week Meta released Segment Anything, a system that can take any image and break it down into segments. For example, a picture of dogs running at the beach turns into 4 segments, two dogs, a stick and the beach.

The model is fully open source and incredibly robust and being able to reliably identify segments of an image is super useful for some generative image tasks, and naturally the generative AI community has already built on top of it:

  • Grounded Segment Anything is a combination of multiple models, including but not limited to Stable Diffusion

  • It can reliably identify and segment any object in an image and change it, remove it and add to it etc.

  • Change someone’s hair, clothing, furniture? It’s never been easier or more robust.

  • Expect this to find its way into creative pipelines almost immediately (similar to ControlNet)

… and what else?

  • Stanford/Google released a wild paper in which they simulate agents with GPT4 (think The Sims meets Westworld. Agents that live their lives, interact with-one another, even plan parties. Check out this thread, it’s worth it.

  • LAION releases a petition, demanding the opposite of last week’s open letter: a firm push forward and a kind of CERN for AI


And that’s it for this week!

Find all of our updates on our Substack at thegenerativeedge.substack.com, get in touch via our office hours if you want to talk professionally about Generative AI and visit our website at contiamo.com.

Have a wonderful week everyone!

Daniel
Generative AI engineer at Contiamo

Thanks for reading The Generative Edge by Contiamo! Subscribe for free to receive new posts.

Share
Share this post

The Generative Edge Week 15

thegenerativeedge.substack.com
Comments
Top
New

No posts

Ready for more?

© 2023 Contiamo
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing