Beats OpenAI’s O1 in Their Own MLE Benchmark with a Score of 26% Versus 16.9%
NEO Automates Machine Learning Engineering from End-to-End, Reducing Experimentation, Coding, and Ops Grunt Work by Thousands of Hours
PALO ALTO, CA / ACCESSWIRE / November 21, 2024 / A stealth startup announces NEO, the world’s first autonomous AI engineer to automate the entire machine learning workflow for ML engineering, giving ML developers superhuman abilities to build AI models in record time and saving thousands of hours in grunt labor. Some multi-billion dollar large enterprises are working with NEO as early design partners. A waitlist is open for early adopters.
NEO, powered by a proprietary multi-step reasoning algorithm to provide end-to-end automation, enables businesses to have a tool that actually competes with the best-in-class ML engineer. The system allows users to chat in natural language, performs data engineering, and can run hundreds of fine-tuning experiments. Demo video is available here.
Meet NEO – First autonomous Machine Learning Engineer
In a major breakthrough, the founding team tested NEO on the MLE bench, an OpenAI benchmark for measuring how well AI agents perform in machine learning engineering tasks. NEO scored 26% using smaller models like GPT 4o and Sonnet 3.5, compared to OpenAI’s 16.9% using the bigger and more expensive o1 model with the AIDE scaffolding, putting the NEO AI engineer at the level of a Kaggle Grandmaster.
Building high quality AI models and pipelines requires complex ML engineering and significant experimentation. Machine learning tasks include cleaning and merging data, fine-tuning models with different combinations of hyper-parameters and model types, and performing evaluations before finally deploying them. An optimized pipeline often involves a variety of frameworks, including vLLM, TensorRT, and Hugging Face transformers.
“Most existing ML developer tools solve just one part of the value chain-like training, fine-tuning or deployments,” said Saurabh Vij, CEO and co-founder. “Imagine having an ML engineer who works tirelessly, never takes a break, and costs a fraction of the price and handles all the repetitive experimentation and infrastructure level work. That’s what we’re building-the first autonomous ML engineer-a game-changer for companies struggling to scale their AI efforts.”
“AI automations are indispensable tools for all developers and companies looking to get the most value for their time and money,” Saurabh added. “It’s something machine learning doesn’t have-and desperately needs today. ML engineers are very scarce and expensive. In addition, the multiple elements of experimentation, retraining and redeployment suck the soul out of ML teams and slow down the entire innovation cycle.”
NEO is a multi-agent system that completes tasks in multiple stages:
Stage 1: The NEO understands the problem and explores multiple ideas to solve it.
Stage 2: Using its multi-step reasoning capabilities, NEO first explores all the possible solutions and then creates the plan for the most promising path and performs rigorous experimentation for different steps. These include:
Transforming datasets
Fine-tuning
Rag
Deployment
Framework integrations
Stage 3: NEO monitors the quality of the model in terms of latency and accuracy, as well as the health of the infrastructure in terms of load balancing of nodes and costs.
Co-founder Gaurav Vij said: “We created NEO to take on the tasks that distract the creative developers from innovating and waste so much time. As a result, this is not a solution to replace humans; they are always in the loop to guide it and control its behavior. Like a new employee, you can induct NEO and it will learn your processes, playbook and then act accordingly.”
“Today, there are less than 600 Kaggle Grandmasters,” Saurabh continued. Our long-term vision is to provide an AI ‘Kaggle Grandmaster’ to every company on the planet. That way a company could focus on innovation-what they want to achieve in space exploration, scientific discovery, genome sequencing, environmental efficiencies, healthcare, etc. AIs would supply the necessary machine learning to tell us how to make it happen.”
“Every once in a while, a technology comes that not only solves a problem for the present challenges but transforms the entire industry and takes them into the future,” Saurabh concluded. “NEO is going to do that for ML engineering. We’re on the cusp of something bigger than any of us can imagine.”
About NEO
NEO is a multi-agent system capable of solving complex machine learning problems-from data engineering to deployments of ML models-reducing the grunt work of ML engineers, with human guidance. The goal is that every company on the planet can have a best-in-class AI expert on the team, accelerating scientific breakthroughs for all of humanity. Two brothers-Saurabh and Gaurav Vij-with eight years of machine learning experience between the two of them lead the organization, now in stealth mode.
For more information, visit: heyneo.so.
Follow us on social media:
X: @withneo
Intro Post: NEO AI
LinkedIn: hey-neo
Media Contact:
Erica Zeidenberg
Hot Tomato Marketing
[email protected]
925.518.8159
SOURCE: NEO
