Salesforce AI Research recently published SFR-AutoR&D, a set of agents built to run the research loop end-to-end, autonomously improving codebases and training AI models. Given one goal, a system to work on, and a compute budget, SFR-AutoR&D agents formulate hypotheses, implement methods, and run controlled experiments against baselines. They retain improvements to the codebase only when supported by evidence and proven accurate and safe.
SFR-AutoR&D is now being used to train and improve production AI models. The system autonomously improved SFR-Embedding V3, the model Salesforce uses for retrieval in Data Cloud. Next, the Salesforce AI Research team pointed SFR-AutoR&D at the open source model EmbeddingGemma 2, and it produced a measurably better model in both quality and latency, SFR-Embedding v4. These are early examples of something larger: AI systems that improve the software and models an enterprise already runs on.
The system had already built a track record on improving public codebases. The first published results came from open source software that enterprise data teams rely on every day: pandas, the Python library most data scientists use to manipulate large tables of data. Other codebases SFR-AutoR&D has improved include hnswlib and USearch, two libraries that power the kind of similarity search behind AI retrieval. These are heavily optimized projects with years of expert tuning in them, and yet the agents found further gains in all three, each packaged with the benchmark evidence.
Embedding models are the workhorses of enterprise AI, doing the unglamorous work that keeps a business running. They convert text into numerical representations so that a system can find the right record, document, or case history out of millions. When an Agentforce agent answers a question grounded in a customer’s own data, it reaches through Data Cloud, and an embedding model decides what comes back. Retrieval quality — the measure of how well the model finds the right information — determines everything the agent says next.
From weeks to hours
Improving an embedding model has historically been a human process. Research scientists propose an approach, build the experiment, run it, read the results, and decide what to try next, often over a period of several weeks. With SFR-AutoR&D, the system did all of that work itself, completing the cycle in about 80 hours; just a little over three days.
“For most of the history of machine learning, progress has moved at the speed of human experimentation,” said Silvio Savarese, EVP and chief scientist at Salesforce. “What we are seeing now is a research system that forms a hypothesis, tests it, and holds itself to the evidence, applied to a model our customers depend on. The scientific method still governs, and the loop runs faster. This is self-improvement engineered for trusted enterprise deployment.”
What we are seeing now is a research system that forms a hypothesis, tests it, and holds itself to the evidence… The scientific method still governs, and the loop runs faster. Silvio Savarese, EVP and Chief Scientist, Salesforce
What we are seeing now is a research system that forms a hypothesis, tests it, and holds itself to the evidence… The scientific method still governs, and the loop runs faster.
The team built two agents to deliver this kind of improvement. AutoR&D-Engineer searches for algorithmic and systems improvements inside existing code. TrainForge explores new training methods, runs them, and evaluates candidates against held-out benchmarks that the system never trains on. Both agents run on AutoInfra, the training backbone that carries their experiments across shared GPU capacity.
“Our goal is to build AI agents that discover methodological improvements we wouldn’t have thought to explore, and back those discoveries with rigorous experimental evidence,” said Shafiq Joty, senior research director at Salesforce AI Research. “Rather than selecting from a predefined set of training recipes, SFR-AutoR&D agents independently formulate hypotheses, develop novel methods, and validate them through controlled experiments on held-out evaluations. Only improvements that demonstrate measurable gains over established baselines are retained. This emphasis on empirical validation is what makes autonomous AI research both powerful and trustworthy.”
Our goal is to build AI agents that discover methodological improvements we wouldn’t have thought to explore, and back those discoveries with rigorous experimental evidence. Shafiq Joty, Senior Research Director, Salesforce
Our goal is to build AI agents that discover methodological improvements we wouldn’t have thought to explore, and back those discoveries with rigorous experimental evidence.
Evaluated against 10 public retrieval tasks in MTEB, SFRAutoR&D improved the model’s ability to place relevant results near the top, raising its nDCG@10 score by 6.13 points. That metric measures the relevance of the top 10 results, giving more weight to those ranked higher.
The system then ran a second session on its own, roughly nine hours, this time aimed at how fast and cheaply the model runs. It came back with 36% more throughput and 17% lower latency, with almost no loss in quality. For a business, that is the difference between a model that performs well in a benchmark and one that is affordable to run across millions of customer records.
Building trust into training
An important aspect of training any model at Salesforce is trust. The models trained for product use draw only on datasets that have cleared ethical and legal review, a constraint the research team treats as part of the problem definition rather than a step after the fact. A model that performs well in evaluation still has to clear production requirements before any customer sees it, and that work is underway now to maintain the highest levels of trust and safety.
Autonomy here has limits by design. Researchers set the goal, the budget, the evaluation, and the data constraints, and the system explores inside them. Without a verified improvement over the baseline, the model stays in place.
What’s next in self-evolving systems
Self-improving agents and models will soon be the norm, and the role of researchers and engineers is already moving to set the goals and constraints for autonomous, self-improving frameworks. A related effort from Salesforce AI Research, CodeEvolve, uses evolutionary search with large language models to improve performance and code quality in Java and Salesforce Apex. An upcoming blog will cover that work in depth.
“The improvements are happening as we speak,” Joty said. “We used to count progress in weeks of human work. Now we count it in hours of compute.” He added, “It’s a really exciting time for research in enterprise AI.”
SFR-AutoR&D is an early look at how enterprise AI gets built from here, through systems that propose their own improvements and prove them before anything ships. Humans stay firmly at the helm, setting the constraints that earn the trust.
