Researchers receive funding to explore ambitious new ways of improving AI security

At a time when concerns about AI safety are hitting the headlines, and even some AI leaders themselves are urgently warning we need to develop the technology more carefully, three researchers here are receiving funding from a £50m UK government programme to explore ambitious new ways of improving AI security.

ProfessorsTom Gur,Martin Kleppmann andAmanda Prorok are among the researchers being awarded grants today (7 October) under the Advanced Research & Invention Agency (ARIA) 'Scaling Trust' programme.

They will be leading teams conducting cutting-edge research into how we could make AI safer.

Two of the new projects being carried out here will explore how we could use techniques from fields like cryptography to force AI systems to operate and interact with integrity and in a way that keeps users' data secure.

The third project is about applying a watermark – a security feature increasingly used to guard digital images and text – to the physical movement that robots display when they move.

While this movement, or 'jitter', is nearly imperceptible to the naked eye, it can be analysed from ordinary video footage by a custom app developed by Professor Amanda Prorok's team.

Revealing who is controlling robots' movements

By decoding the signal embedded within these filmed movements, the app identifies exactly who developed the robot's underlying control software. In this way, says Professor Prorok (right), "it introduces provenance to robots in the way that we've seen watermarks do in digital text and images."

Such a tool could have wide applications, she says, from giving a warning that a control system has been hacked, to informing investigators which control system was installed in a vehicle that was involved in a collision.

It would also protect the intellectual property of companies that develop robot movement control systems.

This concept has already been demonstrated by her team on stage at the United Nations AI Summit in Switzerland last July. But they are looking forward to taking this work forward, in particular scaling it up, making it more robust and encouraging more companies to use it.

ARIA is an R&D funding agency created to unlock technological breakthroughs that benefit everyone. It funds teams of scientists and engineers (that it calls R&D Creators) to pursue research at the edge of what is scientifically and technologically possible.

Its Scaling Trust programme, led by Programme Director Alex Obadia, backs the frontier infrastructure and fundamental research for AI agents to coordinate securely on our behalf, across digital and physical worlds.

But as such agents begin to be deployed at scale, issues of trust, safety and security have become increasingly important areas of interest and concern.

Using cryptography for AI

As Prof Tom Gur (left), who will be leading the new 'Advanced Cryptography for AI' project, puts it: "The AI models that are being developed and deployed at unprecedented speed hold great promise, but they also raise serious safety and privacy concerns. Addressing these risks is our highest priority."

And ARIA agrees, pointing out that it is "contributing to this global effort by funding the moonshots that could dramatically change the picture for multi-agentic security".

Here in this Department, those moonshots are:

• 'Advanced Cryptography for AI', led byTom Gur, Professor of Computer Science.
• 'The Centre for Cryptographic Trust Infrastructure', co-led by ourAssociate Professor Martin Kleppmann, along with Martin Pompéry from the SINE Foundation.
• 'Physical Watermarking: Robust Policy Certification for Embodied Multi-Agent Systems', led byAmanda Prorok, Professor of Collective Intelligence and Robotics.

The Centre for Cryptographic Trust Infrastructure, according to Dr Martin Kleppmann, will enable mutually untrusting AI agents (e.g. agents representing different companies) to establish trust by cryptographically proving facts about their companies and their physical-world processes to each other.

"For example, they could prove that a product has been produced according to a particular specification," he says.

Telling the truth without giving away sensitive information

Traditionally, he points out, this has been done "through trusted third parties such as certification bodies, or the slow, manual processes of vetting potential suppliers in person". But in a new, increasingly AI-facilitated world, we need to find ways to do this faster, more cheaply – and more reliably.

At CCTI, the hypothesis they're testing about how this can be done is through using cryptographic protocols such as zero-knowledge proofs. "These provide a way for one agent to prove to another that it is telling the truth, but without revealing commercially sensitive data," Martin Kleppmann (right) says.

For example, "it would allow a supplier to prove that its products were indeed manufactured using a particular process as it claims. And if this works, we believe that it will unlock huge economic opportunities."

Cryptography is also at the heart of the new project being led by Prof Tom Gur. Here the aim is to "apply techniques and tools from cryptography to controlling AI systems and trying to make them behave in prescribed ways".

This includes trying to enforce integrity of behaviour – so an AI system doesn’t go rogue and hack into a government agency's system – and also confidentiality.

"As we're giving AI models access to a great deal of data, we’d like to ensure we can maintain the integrity and privacy of the data."

Is such a thing at all possible? "That's the big question," says Tom Gur, "and a lot of people around the world are trying to answer just that. But I believe our approach is unique as it relies on new cryptographic proof systems that we have been developing in order to try and solve these big problems."

Projects in our department funded by the ARIA Scaling Trust programme:

Advanced Cryptography for AI

Team Lead: Tom Gur, University of Cambridge. Team: Nir Bitansky (NYU), Yuval Ishai (Technion), Sarah Meiklejohn (UCL/Google), Ron Rothblum (Succinct/Technion)

The project will be developing privacy techniques suited to AI workloads using cryptography techniques. This includes private retrieval for RAG, semantic search, secure computation, and methods for concealing queries or embeddings. Its aim is to let agents use shared memory and sensitive data without exposing commercially or personally confidential information.

The Centre for Cryptographic Trust Infrastructure (CCTI)

Team leads: Martin Kleppmann (University of Cambridge), Martin Pompéry (SINE Foundation). Team: grjte (Ink & Switch), Hossein Hafezi (University of Cambridge), Jonathan Heiß (SINE), Daniel Hugenroth (Light Squares), Alireza Kavousi (University of Cambridge), Ágnes Kiss (SINE), Arman Kolozyan (University of Cambridge), Mario Lins (Light Squares), Jessica Man (University of Cambridge), Aurel Stenzel (SINE)

CCTI will enable mutually untrusting AI agents (e.g. representing different companies) to establish trust by cryptographically proving facts about those companies and their physical-world processes to each other. For example, they could prove that a product has been produced according to a particular specification, or under what conditions an agent would be willing to reach an agreement. CCTI will also be the programme's integration partner, mapping how creators' work fits together, co-developing shared building blocks with interested teams, and proposing open-source benchmarks and challenges to the Scaling Trust Arena.

Physical Watermarking: Robust Policy Certification for Embodied Multi-Agent Systems

Team Lead: Amanda Prorok, University of Cambridge. Team: Manon Flageat, Mateusz Sypniewski, Sally Matthews, University of Cambridge.

This project develops a tamper-resistant physical watermarking framework to verify the provenance and safety compliance of control policies driving embodied robots. By extending the Colored Noise Coherency (CoNoCo) construction, the framework enables independent parties to remotely authenticate active controllers using commodity hardware, such as standard CCTV cameras or smartphones, without requiring direct access to the robot or specialized sensing equipment.