High cost of API prototyping and missing regression testing
Problem
Software engineers face difficulty in confidently designing APIs because building multiple prototypes is time‑consuming. Ephemeral testing approaches often end up as integration tests, offering little regression protection, and designers may miss hidden usage patterns which LLM‑generated code can reveal.
Pain points
- Writing multiple API prototypes to validate design requires significant time and effort, especially without low‑cost tools.
- Ephemeral testing solutions often degrade into integration tests, providing limited regression protection and risking undiscovered bugs later.
- Designing libraries or abstractions typically requires at least three client implementations, but hand‑writing them is costly and hinders rapid iteration.
- Designers often fail to anticipate diverse usage patterns, causing critical scenarios to be overlooked; LLM‑generated code can expose these blind spots.
Questions to verify
- Do you believe using LLMs to automatically generate API prototypes can significantly reduce design cost?
- In your workflow, how does the importance of regression testing compare to rapid prototype validation?
- Have you ever encountered design issues because you failed to anticipate certain usage scenarios? Would you be willing to let an LLM help uncover such blind spots?
Evidence
quote
The big challenge with designing an API is that the only way to be confident in the design is to build a bunch of different things on top of it. But why invest all that effort in an API that you don't think is ready yet? With coding agents the cost of building those prototypes drops to almost nothing. I can exercise a proposed API design five different ways before I commit to the shape.
quote
Unless I'm missing something, this doesn't help with preventing regressions. In the end, as the author already puts it, it's an integration test in the end, why not just write the integration tests directly?
quote
This is an application of the rule of three: you need at least three clients to write a good library, protocol, or other underlying abstraction. Except it proposes using a large language model to write the three clients and then throwing them away.
quote
I had good results with a similar technique this summer. I was designing a DSL to help make a colleague’s work easier, and I tested it by having a coding agent re-implement some of their notebooks using a draft of the DSL. The LLM output was a decent enough approximation of “typical” use, and it uncovered some warts I didn’t catch by testing it myself. As its designer, I simply wouldn’t have thought to try using it in some of the ways the LLM-generated code did.
Disclaimer: this card is distilled by AI from community discussions as “pain point evidence”. It is a starting point for need analysis and validation, not a “validated market need”. AI has analytical limits; please open the original discussion above to manually verify before key decisions.