The Generative Edge Week 30
The case for keeping humans in the AI loop (for now), agents are teaching each other SKILLs and detecting AI content is futile.
Welcome to week 30 of The Generative Edge. Here is the gist in 4 bullet points:
While generative AI systems like ChatGPT and Copilot are powerful, they still require human supervision to navigate their limitations and optimize their potential.
We can expect a surge in AI sidekicks that seamlessly integrate into our daily tasks, while the final decision-making remains in human hands.
The cutting-edge field of autonomous agents continues to advance, with the SKILL framework enabling these agents to learn and share acquired skills.
Plagiarism detection tools struggle to identify AI-generated text and educators must come up with new methods to assess their student’s knowledge.
A case for the human in the loop
Generative AI can be wonderful, feel close to magic even. As all of us have gotten more familiar with this new class of tools however, limitations and shortcomings have started to become apparent. Hallucinations, bias, confident incorrectness, subtle mistakes, and so on. This is not to say that generative AI tools can’t be incredibly useful, but at this point just letting them roam unsupervised invites tears and pain.
Generative AI tools really shine when they are being used in tandem with a human, who can steer, verify and improve the output these systems give us via feedback.
ChatGPT, Copilot and copilot-like integrations all work with a human in the loop (HiTL).
According to Gartner, by 2027 over 100 million humans could engage AI sidekicks to contribute to their work.
We do see more and more sidekick-style systems coming online (e.g. Shopify launching their "Sidekick" to much fanfare) which assist the user in what they’re doing, while leaving the final decision making to the human.
When implementing use-cases today, keeping the human in the driver’s seat is probably the safest bet to offset current day limitations of generative AI models and to avoid possible complex legal and compliance questions.
With technological progress and as humans continue to contribute their feedback in order to improve the generative AI systems of today, truly autonomous systems will eventually become ready for prime time.
Agents with SKILL
Autonomous agents continue to fascinate. Their ability to iteratively utilize large language models, apply tools, and strategize to achieve high-level objectives is a captivating one.
That said, at this point, agents are not ready for prime time, and in many ways are more of a curious research object than something that can really make a productive difference.
As with all things generative AI though, progress is being made swiftly and before long, autonomous agents might be ready to take over actual tasks. One relevant step in that direction is SKILL.
A paper proposes a new way to enable agents to learn from each other (source).
SKILL is a the name of a the proposed framework that can enable agents to teach each other new skills that they have previously acquired while completing objectives/tasks.
In isolation, an agent might pick up a new skill to solve a task, but that knowledge might fade over time and agents have to reacquire it from scratch.
With SKILL, a swarm of agents could quickly acquire and distribute new and diverse skill sets to enable them to tackle novel and/or complex tasks much more quickly.
Imagine a network of agents that can dynamically upskill themselves to tackle new tasks, in parallel, without burdening every agent to keep each skill around indefinitely.
More research and engineering will be needed to harden such a knowledge transfer system in order to keep the agent swarm aligned with human values.
While autonomous agents are still learning how to walk, more and more work is being done to enable them to run, and eventually fly.
Detection is futile
ChatGPT is a popular tool, including but not limited to students who use it for learning and research. However, its ability to generate entire essays has raised plagiarism concerns in educational institutions. To combat this, plagiarism detection tools, including OpenAI's own AI text classifier, emerged.
Yet, OpenAI recently (and quietly) discontinued their own tool, suggesting that reliably detecting AI-generated content may be an unachievable goal. This raises important questions about the future of academic integrity in the age of AI.
Classifying generative AI output as “AI generated” is a fool’s errand
False positives are plentiful and those are often more harmful than not having a detector at all
If someone were to develop a perfect detection system, it could be instantly utilized to enhance AI text generation, subsequently outsmarting and rendering the detection system obsolete.
Prompting models for style changes, using translators to flip the text, replacing words, students are resourceful in finding ways around detection tools as it is.
Assume everything you read is AI generated, unless proven otherwise. There is no reliable detection.
Educators will have to come up with better ways of validating the knowledge of students, as the old ways of yore will no longer cut it and no amount of AI detection snake oil is going to change that.
… and what else?
And that’s it for this week!
Find all of our updates on our Substack at thegenerativeedge.substack.com, get in touch via our office hours if you want to talk professionally about Generative AI and visit our website at contiamo.com.
Have a wonderful week everyone!
Daniel
Generative AI engineer at Contiamo
