Product
Industry News
NVIDIA RTX Spark for AI Teams: Worth It ?
NVIDIA's RTX Spark laptop runs 120B-parameter LLMs locally with 128GB unified memory. Specs, the DGX Spark mix-up, and whether your AI team should buy.
•12 min read
H100 on-demand rental rates are back down to roughly $2 to $2.53 an hour as of June 2026 That's a level last seen before the AI boom priced the same hardware at $8 an hour.
If you're pricing an AI feature off compute cost assumptions from even six months ago, that number is already stale. Here's what actually moved, why, and the math you should be redoing today.
Key Takeaways
- In June 2026, H100 on-demand pricing hit $2.01 to $2.53/hour, down from roughly $8/hour at the 2023 to 2024 peak (Spheron, June 2026).
- The drop is not demand destruction. It's Blackwell displacement: as B200/B300 supply ramps, H100 gets pushed to mid-tier pricing while frontier-chip demand stays tight.
- A back-of-envelope model shows compute cost per 100,000 inference requests falling from $444 to roughly $141 at these rates, a floor most AI feature pricing hasn't caught up to.
- The floor is generation-specific and reversible. A March 2026 shortage already pushed reserved H100 pricing up 38% in five months before this relief arrived (SemiAnalysis, H100 Rental Price Index).
H100 pricing has moved in three distinct swings, not one straight line, and each swing had a different cause. Missing that pattern is how teams end up building pricing models on top of a number that's already out of date.
In 2023, H100 rental rates climbed past $8 an hour at the peak, driven by startups racing to train foundation models on any capacity they could book (latent.space, $2 H100s: How the GPU Bubble Burst, October 2024). By August 2024, that same capacity was renting for $1 to $2 an hour. Companies locked into three- to five-year reserved contracts at $3 to $4/hour had finished their training runs and started reselling unused capacity through Runpod, Vast.ai, and Together.ai. That resale wave landed right as open models like Llama 3 shifted demand from full training to cheaper fine-tuning.
Then, as of March 2026, prices climbed again. SemiAnalysis's H100 1-Year Rental Price Index tracked reserved pricing rising nearly 40%, from $1.70/hour in October 2025 to $2.35/hour by March 2026, as reasoning and agentic models drove a fresh compute crunch (SemiAnalysis, The Great GPU Shortage, 2026). Anyone who called the 2024 crash a permanent floor got that shortage wrong.
That's the part most "GPU prices are crashing" takes miss: this market doesn't settle, it oscillates, and each swing has its own trigger. Treating any single data point as the new normal is how a pricing model built in March turns out wrong by June.
Because Blackwell showed up, not because AI demand slowed down. As B200 and B300 supply ramped through the first half of 2026, H100 shifted from top-tier to mid-tier hardware. Providers who once fought over scarce H100 inventory could finally offer it at thinner margins, or redirect new customers to Blackwell instead (Spheron, NVIDIA H100 News 2026, June 2026).
Reserved capacity is doing double duty here too. A wave of 2023 and 2024 three- to five-year H100 contracts is starting to expire. As early adopters migrate those workloads to B200 and GB300 systems, the freed-up inventory is landing back on the resale market (Thunder Compute, AI GPU Rental Market Trends, July 2026). Add competitive pressure (AWS reportedly cut H100, H200, and A100 instance pricing by up to 45% in the same window) and the mid-tier price floor drops fast.
What this is not: evidence that overall AI compute demand cooled off. Frontier-tier Blackwell capacity remains tight through 2026 into 2027, with hyperscalers absorbing most of it on multi-billion-dollar forward orders (businessengineer.ai, The State of the GPU Economy, 2026). This is a generational reshuffle inside the GPU market, not a demand collapse across it. If your pricing story assumes the second one, it's built on the wrong premise.
More expensive than H100, and still supply-constrained at the top end. As of June 2026, B200 on-demand rates on neo-cloud providers run about $3.70/hour, roughly 1.5 times H100's $2.53/hour rate. Spot B200 starts around $2.74/hour, while hyperscaler on-demand pricing (AWS) reaches $14.24/hour for the same chip (Spheron, NVIDIA B200 Cloud Pricing 2026, June 2026). Average B200 pricing across 25 tracked providers sits around $5.18/hour, roughly double what H100 costs today.
B300 (Blackwell Ultra) sits higher still. On-demand B300 rates had firmed to about $9.16/hour on Spheron by July 2026, with GB300 cloud instances starting from $3.02/hour on lower-cost providers (Spheron, NVIDIA B300 Blackwell Ultra Guide, 2026). GB300 shipments are projected to grow 129% year over year in 2026. Even so, the highest-memory 288GB configuration is still selling faster than it ships, and direct NVIDIA DGX B300 orders currently run 8 to 12 week lead times.
| Chip | On-demand rate (mid-2026) | Spot rate | vs. H100 |
|---|---|---|---|
| H100 (SXM/PCIe) | $2.01-2.53/hr | as low as $1.43/hr | baseline |
| B200 | ~$3.70/hr (avg $5.18/hr across providers) | ~$2.74/hr | 1.5-2x |
| B300 (Blackwell Ultra) | ~$9.16/hr | not widely quoted | ~3.6x |
Sources: Spheron H100/B200/B300 pricing guides, Thunder Compute market trends, June-July 2026.
Put those two facts together and the H100 story makes more sense. H100 didn't get cheap because Blackwell got cheap. It got cheap because Blackwell finally became available in volume, giving providers somewhere to route new frontier-tier demand instead of squeezing it onto aging H100 fleets. Blackwell itself is still priced at a 1.5x to 4x premium over H100 depending on tier and provider, and the top-spec configurations remain genuinely scarce. That's the generational reshuffle from the last section, with real numbers attached to it.
Let's do the math instead of gesturing at it. Assume a mid-size AI feature (a support-ticket assistant, a document summarizer, anything running one moderate LLM call per user action) uses roughly 2 seconds of H100-class compute per request. That's a reasonable planning number for a 7B to 30B-class model doing a single-turn generation, not a published benchmark, just the assumption most teams should be running when they price a feature.
At $8/hour, that request costs about $0.0044. At $2.53/hour, it costs about $0.0014, a 68% drop in raw compute cost per call. Scale that to 100,000 requests a month and the compute line drops from roughly $444 to $141.
That gap is the whole point. If your feature's price point, free-tier usage cap, or margin target was set when the compute line assumed $8/hour, you're either underpricing the value you deliver or sitting on margin you don't know you have. Neither is a good place to run a business from. Go rebuild that spreadsheet before your competitor does it first and undercuts you on price.
Treat it as temporary until proven otherwise. Spot rates already ticked up slightly between May and June 2026, Vast.ai's H100 rate moved from $1.93 to $2.01 an hour in that window (Thunder Compute, July 2026). It's a small move, but a reminder that this market corrects in both directions inside a single quarter.
There's a structural reason to think cheap previous-generation compute sticks around for inference even if it doesn't for training, though. Stratechery's Ben Thompson argues that "agentic inference" doesn't need frontier speed at all. These are workloads where an agent runs unattended, with no human waiting on the response, and as he puts it: "if latency isn't the top priority, then slower and cheaper memory, like traditional DRAM, makes a lot more sense" (Stratechery, The Inference Shift, May 2026). His argument is that agentic inference will become the largest compute market precisely because it isn't GPU-bound the way training is.
That matches what we see building AI features for clients: the batch summarization job, the overnight data-cleanup agent, and the async workflow step almost never need the fastest chip in the fleet. They need enough memory and enough reliability to finish the job. Routing that class of work to whatever previous-generation hardware is cheapest that month is a real lever, not a theoretical one.
Don't reprice around a single data point, and don't ignore the swing either. Three moves are worth making this quarter:
Both are true, at different points. Prices crashed to $1 to $2/hour by August 2024, then rose again to $2.35/hour by March 2026 during a shortage, then eased to $2.01 to $2.53/hour by June 2026 as Blackwell supply ramped (Spheron, SemiAnalysis, 2026).
No. It's generational displacement. Blackwell (B200/B300) capacity is absorbing new demand and pushing H100 into a mid-tier role, while frontier-chip demand from hyperscalers remains tight into 2027 (businessengineer.ai, 2026). Broad AI compute demand has not slowed.
It depends on how compute-heavy the feature is. But a 3 to 4x swing in GPU-hour pricing, roughly what happened between the 2023 peak and June 2026, changes the compute line in a unit-economics model more than almost any other input teams typically re-check. Most pricing models don't get revisited that often.
No, it's the opposite. B200 on-demand rates average around $5.18/hour and B300 has firmed to about $9.16/hour as of mid-2026, both well above H100's $2.01 to $2.53/hour (Spheron, 2026). Blackwell's availability, not its price, is what's driving H100 rates down.
Not as a permanent assumption. Build in a sensitivity range instead, model your unit economics at both the current rate and the prior shortage-era rate (~$2.35 to $2.50), so a future price swing doesn't silently erase your margin.
H100 compute is roughly a third of what it cost at the 2023 peak, but this market has already proven it can reverse in months, not years. The unit economics of your AI feature deserve a live number, not a number from your last planning cycle.
That's a quick, honest conversation to have before a competitor undercuts you on margin they don't know they have.
Continue exploring these related topics
NVIDIA's RTX Spark laptop runs 120B-parameter LLMs locally with 128GB unified memory. Specs, the DGX Spark mix-up, and whether your AI team should buy.
Explore the buzz around China's new autonomous agent, Manus, and its potential to revolutionize AI and global tech leadership.
June 30, 2026, Anthropic released Claude Sonnet 5 and pitched it as the model that finally makes cheap, reliable agents the default. The claim underneath the launch is simple: near-Opus intelligence at a Sonnet price.