NEO Builds Full ML Pipelines in VS Code from One Prompt

The Breakthrough

NEO VS Code extension constructs complete machine learning pipelines that include synthetic dataset generation, baseline model training, inference API deployment, and frontend UI creation from a single prompt, such as 'build a chat moderation pipeline to detect profanity and harmful text in messages'.

What Actually Worked

• NEO scans the workspace for available resources, identifies missing data, proposes a detailed task plan, and awaits approval before execution; for chat moderation, it plans synthetic dataset generation covering English messages with profanity, hate speech, bullying, and threats.
• NEO generates synthetic datasets by writing a Python script that defines a schema, establishes annotation guidelines for label consistency, runs the script to produce a CSV file with thousands of balanced rows, and creates validation outputs.
• NEO analyzes data to select a baseline classifier, writes training code to split data into training and validation sets, trains the model locally, evaluates results, and provides detailed logs with timestamps, errors, recovery actions, and optional Weights & Biases integration.
• NEO deploys a real-time inference API by writing application code, setting up endpoints, configuring serialization, generating a requirements file, and resolving environment issues such as dependency conflicts, package versions, Python setup, or CUDA problems.
• NEO builds a frontend web interface for testing, spins up the application, and displays classification results with confidence scores for inputs like harmless messages or toxic text.

Context

Building machine learning agents from scratch requires sourcing and cleaning data, feature engineering, model selection and training, hyperparameter tuning, deployment, and UI development, which typically demands data scientists, backend engineers, and DevOps specialists. NEO operates as an execution-focused AI teammate inside VS Code, accessing local project files, datasets, and environments with workspace isolation for privacy; users install the free extension from the marketplace, sign in, and optionally connect AWS S3, Hugging Face, Weights & Biases, GitHub, or Kaggle via encrypted local storage.

NEO supports light mode for quick prototyping and pro mode for deeper logs and control, enables pausing or interrupting tasks, handles diverse workflows like tabular ML, forecasting, computer vision, OCR, speech, LLM fine-tuning, and RAG systems, and performs auto-refinement cycles based on results. This local-first approach eliminates black-box issues and tedious plumbing for applied ML engineering.

Notable Quotes

• "It scans the workspace looks at what is available and creates a task plan before execution."
• "It does not just hallucinate rows of text. It actually writes a Python script specifically designed to generate synthetic data defines a schema creates annotation guidelines so the labels stay consistent."
• "You get detailed execution logs timestamps errors recovery actions and performance information while the task is running."
• "Your code data and credentials stay on your machine and the extension keeps credentials encrypted locally."