LLM inference costs cheaper than traditional tools, business model viability uncertain
Problem
LLM inference costs are falling rapidly, potentially below traditional tool costs, but there is a lack of concrete profitability data, creating uncertainty about sustainable business models.
Pain points
- Inference costs may undercut traditional tools
- High profitability expectations with limited hard data
- Uncertainty about long‑term monetization of AI infrastructure
Questions to verify
- Is there public financial data supporting the 10% return requirement for AI infrastructure?
- Can the rate of LLM inference cost decline truly sustain below traditional tool costs?
- What factors will affect the long‑term profitability of AI models?
Evidence
quote · @Anonymous
Tokens become cheaper than tool callsThe author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
quote · @Anonymous
I found the OP insightful and worth a read. Thank you for sharing it on HN.The only aspect that is poorly analyzed by the OP is business model viability. All players are investing insane amounts of money in infrastructure with the expectation that their future profits will justify all that investment. The winner or winners in the AGI race, they believe, will find the proverbial "pot of gold at the end of the rainbow."The OP glosses over questions of business model viability with a brief qualitative discussion and very little hard data. For example, to earn an annual return > 10% on every trillion dollars of capital sunk into infrastructure, the owners of that infrastructure must earn free cash flow (operating profit less investment) in excess of $100 billion per year in perpetuity. Is that
quote · @Anonymous
A much deeper analysis on the falling price per task was published yesterday by Epoch AI [1]. It's a real statistical analysis and comes to more defensible and grounded conclusions. The headline takeaway is:The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA. Two years later, prices fall half as fast, at 32% per quarter (4.7× per year).
Disclaimer: this card is distilled by AI from community discussions as “pain point evidence”. It is a starting point for need analysis and validation, not a “validated market need”. AI has analytical limits; please open the original discussion above to manually verify before key decisions.