What if your AI agent never had to phone home? On October 15, 2026, NVIDIA released Nemotron 3.5, a family of open AI models built for agents — and the headline member is a 74-billion-parameter model that fits in 24GB of memory. That's a single high-end graphics card. For the first time, a genuinely capable AI agent can live on your own desk instead of someone else's server.
The lineup splits the work three ways: Super VL, a 120B multimodal model for always-on enterprise agents; Bolt 74B-A9B, the 24GB local hero available now; and Bolt E4B, a 4GB featherweight arriving October 23, 2026. All three ship with open weights under a permissive licence.
Here is what each model does, what it really costs to run, and the catches you should know before switching off the cloud.
What is NVIDIA Nemotron 3.5?
Nemotron is NVIDIA's family of open AI models, and version 3.5 is the agentic generation. It joins Lightning (released August 2026, for high-throughput workloads) and Ultra (for the hardest reasoning tasks) — the 3.5 release is the tier that brings agent capabilities to local and edge hardware.
Agent models are a different breed from chatbots. They are trained to call tools, read documents, browse the web, write and debug code, and keep working through multi-step tasks that span days or weeks. Super VL was built in collaboration with the Nemotron Coalition and trained on roughly 26 trillion tokens, with about 3.8 million post-training samples and reinforcement learning across dozens of environments.
The release lands in a crowded October. It arrives days after Meta's DeepSeek V4.1-Flash-1017 update shook the open-model leaderboard and weeks after OpenAI's Decisions API pushed agent verdicts into the mainstream. The thread connecting them all: 2026 is the year AI stopped just answering and started doing. Nemotron 3.5 is NVIDIA's bid to make that doing happen on your own hardware.
Super VL 120B: the enterprise agent model
Nemotron 3.5 Super VL is the flagship: 120 billion total parameters with 12 billion active per token (a mixture-of-experts design), a 1-million-token context window, and full multimodal input — text, images, short videos and PDFs. NVIDIA positions it for always-on enterprise agents: finance document analysis, M&A due diligence, vendor credit review, medical claims processing, cybersecurity workflows.
The headline claims: NVIDIA says Super VL matches the accuracy of its largest reasoning model at roughly one-fifth the parameter count, and completes agent tasks up to 5x faster than the next-best open model in its class, measured on an Artificial Analysis agentic-task evaluation. NVIDIA also says organisations serving it on its Dynamo platform can reduce latency by 85% while serving up to 8 times more users, under its specified conditions.
An unusually honest model card tells a more careful story. Super VL beats Qwen3.5-122B on most agent tests but trails GLM-5.3-Flash and DeepSeek V4 Flash on coding and browsing, per independent analysis of the card's own numbers. And the 5x speed figure is a throughput measure that does not appear in the benchmark table — so read it as a vendor claim with a specific methodology, not a universal truth.
The early-adopter list is substantial: Scale AI, EY, Skydance, Atlassian, CrowdStrike, Salesforce and Perplexity are all named as post-training or integrating partners. Weights, datasets and training recipes are published, with the model served as an NVIDIA NIM microservice and a free OpenRouter endpoint.
Bolt 74B-A9B: a 74B model that fits in 24GB
This is the model most readers should care about. Nemotron 3.5 Bolt 74B-A9B packs 74 billion parameters into a 24GB footprint using native 2-bit quantisation (IQ2, trained with quantisation-aware training so the compression hurts less). Only 9 billion parameters are active on each token, so per-token compute is cheap; the trick is that the whole model still fits where a single high-end consumer GPU lives. It has a 256K-token context window and sees text, images and video.
NVIDIA's test hardware includes a PC with a GeForce RTX 5090 and a DGX Spark, and the model is tuned for llama.cpp, Ollama, LM Studio, vLLM and Unsloth — the standard local-LLM stack. NVIDIA says it sustains at least 80 tokens per second.
The benchmark card compares Bolt against smaller open models you could realistically run locally:
- BrowseComp (web-browsing agent tasks): 66.75 vs 48.74 for Qwen3.6-35B-A3B
- Terminal-Bench 2.1 (terminal coding tasks): 50.98 vs 44.38
- GPQA Diamond (hard science questions): 86.43 vs 83.40
- PinchBench: 87.82 vs 88.07 — Bolt narrowly loses this one
Note the last line: NVIDIA's announcement calls Bolt the "#1 open model in its class on PinchBench," but its own model card scores Qwen3.6-35B-A3B a quarter-point higher. A tiny gap, but a reminder to read the tables, not the press release.
What makes Bolt genuinely newsworthy is the privacy angle: an agent that browses, codes and debugs on your machine keeps your documents off the cloud entirely. For a law firm, a clinic or anyone under a compliance regime, "the data never leaves the device" is worth more than any benchmark point. (The usual honesty clause: local does not mean secure by default — access controls, logging and configuration still decide whether "local" actually means "protected.")
Bolt E4B: the 4GB edge agent (Oct 23)
The smallest sibling, Bolt E4B, targets a roughly 4GB footprint for fast conversational agents, voice assistants, interactive characters and physical-AI systems on GeForce RTX cards, Jetson modules and RTX/DGX Spark systems. It arrives on October 23, 2026. If Bolt 74B is the desktop agent, E4B is the one that could live in a kiosk, a robot or an always-listening home assistant.
How to run Nemotron 3.5 locally in 3 steps
- Download the weights from Hugging Face: the GGUF release
nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-GGUFis the ready-to-run local build; a full-precision BF16 variant exists for customisation and post-training. Download the one that matches your VRAM — the 24GB build is the one that runs on a single RTX 5090-class card. - Pick your runtime: for the simplest path, load it in Ollama or LM Studio; for more control, use llama.cpp with the supplied 28GB or 32GB recipes, which include DFlash speculative decoding for extra speed. NVIDIA also publishes vLLM recipes for multi-GPU servers.
- Point an agent framework at it: because Bolt speaks the standard tool-calling interfaces, agent runtimes that work with other open models can adopt it — keep the recommended sampling settings (temperature 1.0, top-p 0.95) from the model card, and note that reasoning mode is toggled through the chat template with
enable_thinking.
Hardware reality check: you need a 24GB+ GPU (RTX 5090-class or equivalent) or an RTX/DGX Spark system. NVIDIA's RTX Spark laptops went on sale the day after the release, starting at $2,599 — so the cheapest "official" door into local Nemotron agents is a new laptop. If you already own the GPU, the software is free.
License and commercial use
Both models ship under OpenMDW-1.1, a permissive licence published by the Linux Foundation that grants use, modification and redistribution "free of charge" and "without restriction," as long as you keep the licence and notices when redistributing. The Hugging Face model cards explicitly state the models are ready for commercial or non-commercial use — with the usual termination clause if you sue claiming the model infringes a patent or copyright.
NVIDIA also opened the training side: datasets and training recipes are published alongside the weights, which is rarer than it should be. One piece of context worth knowing: NVIDIA agreed to acquire Hugging Face in September 2026, so the company now sells the chip, publishes the model and is buying the platform where the models live — a vertical stack that should make at least sceptical observers pause before betting their whole toolchain on it.
Honest caveats
- The 5x claim is NVIDIA's own. The Super VL speed figure comes from an NVIDIA-run evaluation on an Artificial Analysis methodology — no independent replication exists yet. Treat it as a starting point for your own tests.
- "#1 on PinchBench" isn't. NVIDIA's own card scores Qwen3.6-35B-A3B slightly above Bolt on that test. The announcement's wording overreaches its table.
- 24GB is still a lot. "Runs locally" here means on a $1,500+ GPU or a $2,599+ laptop — not on the machine most people own today. The real accessibility story arrives with the 4GB E4B, and only once independent users confirm its quality.
- Local ≠ private by default. An agent that can browse and use tools on your machine needs real access controls and logging. A careless local setup can leak more than a well-run cloud one.
- October is busy. With DeepSeek V4.1-Flash-1017, Meta's Muse Wonder and the Nemotron line-up all landing within days, benchmark tables are comparing pre-release versions of each other. Expect revisions — and recheck numbers before making infrastructure bets.
Verdict: who should run it locally?
Developers and teams with 24GB+ NVIDIA hardware — yes, try Bolt 74B. It leads its local-class group on browsing and terminal benchmarks, the tooling (Ollama, LM Studio, llama.cpp) is the stack you already know, and the commercial licence removes the usual licensing anxiety.
Privacy-sensitive professionals — Bolt is the first open agent model that makes "the document never leaves the room" practical on a single workstation. Audit the setup properly, but the option is real now.
Enterprise agent builders — Super VL deserves a place on the shortlist next to the frontier APIs, especially if your workloads run for days and the 5x throughput claim holds up in your tests. Benchmark it against your actual tasks before believing any leaderboard.
Everyone else — the trend is what matters. Open models that once needed a data centre now fit in a desktop, and in a month the 4GB E4B may push them into devices you can hold. The cloud isn't dead; it's just no longer the only option. Compared with the other October heavyweights — our ChatGPT vs Claude vs Gemini breakdown and the GPT-6.1 Sol guide — Nemotron 3.5 is the one you can take home.
FAQ
Is NVIDIA Nemotron 3.5 free?
Yes — the weights, datasets and training recipes are free to download from Hugging Face, and the OpenMDW-1.1 licence permits commercial use. Your only cost is the hardware: a 24GB-class GPU or RTX Spark system for Bolt 74B.
Can I run Bolt 74B on a normal gaming PC?
If the PC has a 24GB or larger GPU — a GeForce RTX 5090-class card, for example — yes. Load the 24GB GGUF build in Ollama, LM Studio or llama.cpp. Cards with 16GB or less will need the 4GB Bolt E4B, due October 23, 2026.
Is OpenMDW-1.1 really open source?
It's a permissive, commercial-use-friendly licence published by the Linux Foundation — more open than most custom model licences — but it's not a classic OSI open-source licence, and your rights terminate if you sue over patent or copyright claims. Read the licence text for your use case.
When does Bolt E4B release?
NVIDIA lists October 23, 2026. It targets a 4GB memory footprint for conversational agents, voice assistants and physical-AI systems on GeForce RTX, Jetson and RTX/DGX Spark hardware.
Sources: TechRepublic, HPCwire AIwire (NVIDIA announcement), Hugging Face model card, CellCog, Blockchain.news.
No comments:
Post a Comment