top of page

AI Inference Providers vs. Neoclouds: Who Takes the Risk in the AI Compute Stack

Writer: Hans Stege
Hans Stege
5 days ago
12 min read

How the likes of Modal, Baseten, Fireworks and Together sit between debt-funded neoclouds and pay-as-you-go AI demand.


It’s been a tumultuous few weeks for AI infrastructure, even as Anthropic’s IPO approaches. Jensen Huang resigned as Nvidia’s CEO on November 12 amid reports of affairs, including one with a woman US officials reportedly suspected of being a Chinese intelligence asset, and Nvidia shares are down about 25% since the first report on October 29. That’s left the market asking what comes next for the company at the center of financing and backstopping the AI build-out. The same week, Nebius slid on delayed capacity and Fireworks disclosed a second breach. Yet AWS’s CEO said inference is still below 1% of its eventual scale. It’s a good moment to look at how the market underneath is put together.


The AI build-out runs on long-dated commitments. Labs sign multi-year, multi-billion-dollar compute contracts. Neoclouds borrow against those contracts to buy GPUs and lease data centers. But most of the companies actually building AI products can't commit to anything for five years, or even one.


Inference platforms like Modal, Baseten, Fireworks and Together sit in that gap. They buy capacity in one- to three-year blocks from many suppliers and resell it by the second or by the token to thousands of smaller customers. This year that middle layer has been one of the fastest-growing parts of AI infrastructure. Modal closed a $750M round at $15.75B and disclosed more than $1B of annualized revenue; Baseten and Fireworks are reportedly raising at $30B or more.


This piece places the cohort in the broader stack: what it commits to, what it sells, how its balance sheet differs from the neoclouds', where the lines are blurring, and what could go wrong.


Serving is profitable, and demand is growing


Anthropic's S-1 is the best public evidence yet that serving models at scale is profitable, and that demand is still growing fast. As I noted in my write-up of the filing, by my math gross margin on customer-serving compute rose from about 12% in H1 2025 to about 85% in H1 2026, and revenue per dollar of serving compute rose from 1.1x to 6.8x. The amended S-1 filed November 2 shows the serving margin holding at roughly the same level in Q3, by the same math. But total compute rose to 58% of revenue from 48% in Q2, mostly because Anthropic bought $5B of short-term compute that it says costs significantly more than its multi-year contracts, and it swung to a $567M operating loss. Even a buyer with close to $900B of commitments is paying up for capacity on short notice.


Anthropic serves its own models, so it is not a customer of the independent platforms. But it shows what inference economics can look like when the models are good enough, it sets the price umbrella open-weight models compete under, and by locking up hundreds of billions of dollars of compute it tightens GPU supply for everyone else. The filing also calls agentic products "structurally more compute-intensive."


If inference really is below 1% of its eventual scale, as AWS CEO Matt Garman put it in an interview released November 10, the bottleneck could very well come down to available supply: AWS is adding a GPU every 14 seconds and he still expects shortages for another couple of years. When compute is that scarce, who gets it comes down to who is willing to commit for it, and for how long.


A ladder of commitments


The easiest way to place the inference platforms is by how long each layer commits for, and who carries the balance sheet.



How compute, capital and model weights flow through the AI stack, simplified


At the top, labs and hyperscalers sign multi-year contracts measured in billions. For example, IREN's $9.7B GPU cloud deal with Microsoft struck last year runs five years, with 20% prepaid. In its Q3 results, CoreWeave reiterated this long-dated structure, saying it expects 74% of its $132.5B backlog to be recognized more than two years out and 36% more than four years out. Those contracts with highly-rated counterparties are what lets neoclouds borrow to buy GPUs.




As a group, only two of ten listed neoclouds are profitable, and their combined debt is about nine times annual revenue, per The Economist. That’s the cost of owning the rung: someone has to carry the GPUs and the buildings on a balance sheet.


Those long contracts may be less firm than they look, though. Microsoft walked away from anchoring Nscale’s first UK data center in Loughton after power delays, and reportedly decided not to renew a contract with Nscale in Texas. Anthropic's $41.3B agreement with Lambda became terminable when Lambda missed a lease deadline, though Anthropic says it will stay. Google took over Crusoe's planned Wyoming campus after raising concerns about its cost and timetable, and now plans to spend $17B there itself. Delivery is slipping too. Nebius fell as much as 16% after its Q3 report on November 11: revenue beat, but it trimmed its 2026 outlook because some third-party colocation capacity, including a New Jersey site hit by power problems, shifted into 2027, even as it guided 2027 capex to $40–45B. And the public market has pushed back on new listings: Firmus scrapped a roughly US$30B IPO in October, and Nscale has yet to list despite filing its S-1 back in September. Nscale amended it’s filing this week, which could be a sign it will push ahead with its process, though presumably dependent on market reaction to Anthropic and the latest news on Nvidia, a key backer.


The platforms in the middle


The inference platforms sit one rung down and carry a different kind of risk. For now they have no buildings for a departing tenant to strand and no asset-backed debt to refinance, but they are exposed to the price of the capacity they rent and to customers who can leave at any time. Modal, for example, reserves capacity in one- to three-year blocks from the major hyperscalers and dozens of neoclouds. It added 30 of those neoclouds in the past year after it couldn’t get Nvidia Blackwell chips at the price and quantity it wanted from the big clouds. Its CEO has described single commitments in the tens or hundreds of millions of dollars. It then resells that capacity by the GPU-second to a long tail of customers. So far, the inference platforms have funded themselves largely with venture equity rather than asset-backed debt, which is why their raises this year have generally been so large.


This middle rung has grown fast because it sells what the rungs above it don't: compute without a commitment. Fireworks, Together and Modal each reached roughly $1B of annualized revenue this year, and Baseten was at an estimated $600M by March and given its growth trajectory, is likely nearing that $1B figure if it hasn’t already surpassed it.




Company announcements (Fireworks, Modal) and Sacra estimates, Sep 2025 to Oct 2026


Their customers look different from the neoclouds'. Ramp's card and bill-pay data, which skews toward smaller companies, shows it: developer-friendly platforms such as RunPod and Modal are used by the most buyers, while the capacity neoclouds take most of the dollars through fewer, larger accounts.


Ramp Rate, Neocloud category, latest month as of Nov 10, 2026; Ramp's customer base skews toward smaller companies


The big four are only the most visible part of a long tail. GPU marketplaces like SF Compute, Shadeform and Vast.ai sell raw machines by the hour or week, often matching buyers with other providers’ spare capacity, and leave the software to the customer. Developer clouds like RunPod rent GPUs directly to individual developers. Smaller inference APIs such as DeepInfra and Wafer focus on competing on price. The platforms in the middle differ in what they add: they commit to capacity themselves and sell it back as finished compute.


A growing number of AI companies also use these platforms to fine-tune open models and cut what they spend on frontier models from OpenAI and Anthropic, Bloomberg reports. That customer base is the business model. One customer's demand is spiky; thousands of customers' demand, pooled across many clouds, is smoother. Smoother demand means higher utilization of the capacity the platform has committed to, and utilization is where the margin comes from. On top of that, each platform has been adding products that pure GPU rental doesn't offer: sandboxes for agents to run code, fine-tuning and reinforcement learning, batch jobs and low-latency serving. Modal CEO Erik Bernhardsson has called plain inference on its own "a race to the bottom"; the bundle is where the platforms hope to earn better margins.


The rungs are blurring


The ladder is a simplification, and every rung is reaching for the one next to it.

Chipmakers are becoming inference providers. Cerebras, which listed on Nasdaq this year, now earns most of its revenue from its own inference cloud rather than from selling systems. Cloud revenue was $126M in Q2, up 281% from a year earlier and about 70% of the total, with customers including OpenAI, Cognition and Lovable.


Neoclouds are moving up, and platforms are moving down. CoreWeave launched its own inference service in March, and some suppliers are getting choosier about renting to resellers: DigitalOcean's CEO says he prioritizes customers with real end-user demand over those reselling his capacity. And the platforms themselves are converging: as Huggingface’s Niels Rogge noted, Together, Fireworks, Prime Intellect and Modal all started in different places and now offer some mix of inference, sandboxes and RL training.


Some platforms are starting to buy their own chips. Tight supply is pushing the middle rung down the ladder. Fal is buying Nvidia GPUs for leased data centers with part of its new $1.65B raise, Together planned to have 250 MW of its own data center capacity by the end of 2026, and Baseten is looking to buy GPUs for data centers it would manage. Several providers have also talked to lenders about borrowing to pay for neocloud contracts. Others are holding back: Fireworks has no plans to buy hardware, and Bernhardsson says Modal will do whatever it takes to get capacity but that "you're just buying chips and reselling them," so the value is in the software. Owning hardware would change the economics: more control over supply, but also the capex and debt that define the rungs above.


Failed labs can become small neoclouds. A line making the rounds among observers: “You either die a neolab or live long enough to become a neocloud.” It has a real basis. GPU rental prices have jumped this year, with H200 rates up about 160% and B200 rates up about 120% since January, according to Ornn data cited by a16z, and one-year contracts are reportedly almost impossible to get. That turns early capacity reservations into a valuable asset. Yuchen Jin relayed one unnamed neolab founder saying the lab could "just resell the compute and print money," and Bernhardsson predicted "a bunch of neolab to neocloud pivots next year". The reports are anecdotal, but the incentive is clear: seed capital spent on compute can earn a return even when the research doesn't.


For the inference platforms, this cuts both ways. More capacity from more sellers helps an aggregator source cheaply. But it also means more competitors offering the same GPUs, and a resale margin that shrinks as soon as supply catches up.


What could go wrong


The middle of a ladder is exposed from both ends.


The duration gap. Platforms commit to capacity for one to three years and sell it by the second. If demand slows or GPU prices fall, they still owe for the capacity. That is the same mismatch Anthropic flags in its own filing, at a smaller scale but with thinner margins and no anchor contracts to lean on. IREN's chief commercial officer, Kent Draper, calls the inference providers "the WeWork of the GPU world": long leases resold short, which "works well while it works, until either demand or financing dries up." Though to be fair, he’s talking up his own book, including the company’s Token Factory concept.


Hyperscalers bundle the extras. On November 9, AWS launched Nitro Sandboxes, giving every container or VM its own hardware-isolated environment at no extra charge, explicitly for running AI agents and AI-written code. Sandboxes are one of the products the platforms are counting on for margin. Customers still have to build the orchestration themselves, and AWS didn't mention GPUs, but the free default raises the bar for what independents have to deliver.


Customers graduate. The largest AI-native customers can sign their own neocloud contracts once their workloads are predictable, which takes the best accounts out of the pool.


Trust is part of the product. Fireworks has disclosed two security incidents this fall. The second came through a small cloud provider it rented from, and the bad press was exacerbated when Notion told its customers it later learned data was being retained despite a zero-retention promise. That exposes a risk for the whole category, not just Fireworks: the same shortage that pushed platforms toward dozens of smaller neoclouds also adds suppliers with less security depth than the hyperscalers. The Information reports at least one customer weighing a move to its own data centers or the hyperscalers. It's another reason platforms are looking at owning hardware, which brings more control over security and better margins, but takes billions just as lending to smaller borrowers is tightening.


Where this leaves the middle


None of this means the middle layer is in trouble. It means the business is getting harder to do well. If Garman is right that inference is still below 1% of its eventual scale, there’s room for platforms to grow faster than their products get commoditized, at least for a while. That’s why they keep adding layers like sandboxes, fine-tuning and batch even as AWS gives isolation away for free. Demand for running models and agent-written code is growing quickly enough that a better-packaged, faster or more trusted version can still win the business, even when a free primitive exists underneath. OpenRouter’s open-model traffic alone has grown more than a thousandfold since early 2025, and agents multiply the compute behind every task.


The platforms best placed for that stretch are the ones that combine enough software, services and trust to keep customers from graduating, without taking on the financing risk of the rungs above. Modal is one example: it reached more than $1B of annualized revenue without owning hardware, and has said it would rather stay a lean software company than hold hard assets. That model has its own exposure. Bernhardsson himself has said that if prices and utilization fall while the market for reselling capacity dries up, “it could get very bad.” But it carries none of the debt that is now testing the neoclouds.

Timing matters too, benefiting those who have closed rounds before this fall’s turmoil, from the Nvidia leadership shock to the neocloud selloff. Platforms still raising, or counting on reported rounds to close, are doing it in a tougher market.


What we’re watching

  • Whether the capacity Nebius pushed into 2027 arrives on schedule, and how neocloud financing holds up through the Nscale and Lambda raises.

  • Whether rental prices keep rising, which rewards anyone holding early reservations, or start to fall, which squeezes the middle.

  • How reported rounds, including Baseten, Fireworks, Lambda and others close out after this month’s turmoil.

  • Whether platforms keep their customers as security and trust become part of the buying decision, and how quickly agent builders adopt AWS’s free sandboxes.



Disclosure


This post is for educational and research purposes only and is not an offer to sell or a solicitation of an offer to buy any security. PrePublic Equity Partners funds hold positions in, or exposure to, Modal Labs and Anthropic. Figures are drawn from public filings, company announcements, third-party data and press reports, some of which describe rounds that have not closed, and have not been independently verified.


Care to get more insights and access to our research? Fill out the form below.



RESEARCH DISCLOSURE This article is for informational and educational purposes only. It represents independent thematic analysis prepared by PrePublic Equity Partners ("PEP") and is intended to discuss industry trends and company dynamics in the private markets. This content does not constitute investment advice, a recommendation, or a solicitation to buy or sell any security or investment product. PEP is not a registered investment adviser or broker-dealer. Any offer or solicitation relating to securities will be made only through definitive offering documents to eligible investors. PEP and its affiliates may hold financial interests in companies discussed herein and reserve the right to trade such positions at any time without notice. Private market investing involves significant risk, including illiquidity and potential loss of principal. All data is sourced from publicly available information and has not been independently verified.


IMPORTANT DISCLOSURE This content is published by PrePublic Equity Partners, LLC (“PEP”) for informational and educational purposes only. It reflects independent research and the author’s views on industry trends and company developments in the private markets as of the date of publication. Those views may change without notice, and PEP has no obligation to update this content.


Nothing in this content is investment, financial, legal or tax advice, or a recommendation to buy, sell or hold any security. It is not an offer to sell or a solicitation of an offer to buy any security, including interests in any fund or vehicle managed by PEP. Any such offer will be made only through a confidential private placement memorandum or other definitive offering documents, and only to investors who meet the applicable eligibility requirements.


PEP is an exempt reporting adviser and is not registered as an investment adviser with the SEC or any state, or as a broker-dealer.


PEP, its funds and affiliates, and their officers and employees may hold financial interests in companies discussed in this content and may buy or sell those interests at any time without notice. Where funds managed by PEP have exposure to a company that is a main subject of a post, the post says so. PEP does not receive compensation from any company it writes about.


Information in this content comes from public sources that PEP believes to be reliable, but PEP has not independently verified it and makes no representation as to its accuracy or completeness. Statements about future events are forward-looking and subject to risks and uncertainties, and actual results may differ materially. Prior observations and analysis are not indicative of future results.


Private market and pre-IPO investing involves a high degree of risk. That includes illiquidity, limited and unaudited information about private companies, and the potential loss of an entire investment.


PrePublic Equity Partners, LLC is an independently operated investment manager that is majority owned by AVG AM Holdings, LLC, an affiliate of Alumni Ventures. PEP makes its own investment decisions and operates independently of the Alumni Ventures investment platform.

Comments


Commenting on this post isn't available anymore. Contact the site owner for more info.
bottom of page