Why Every AI Startup Eventually Builds the Same Thing

There is a pattern in AI startup development that investors have started calling the “RAG convergence.” It goes like this: a company starts with a specific AI-powered product. The product requires custom data. To make the product reliable, they build a retrieval system. To make the retrieval system work well, they build document processing pipelines. To make the document processing reliable, they build evaluation frameworks. Six months in, they have built most of the same infrastructure that every other AI startup has built.

The RAG convergence is a tax on the AI application layer. Every serious AI product company is spending 30-50% of its engineering resources on infrastructure that is not specific to their product, that does not create competitive differentiation, and that they are rebuilding from scratch because the available open-source alternatives don’t quite fit their use case.

Several companies are trying to solve this: LlamaIndex, LangChain, and the more recent Dust and Comet are all attempting to provide the infrastructure layer so that application companies can focus on their actual product. None of them has definitively solved it yet, which is why the rebuilding continues.

The economic argument for a standard RAG infrastructure layer is overwhelming. The technical argument is harder, because the performance characteristics that matter for one application — latency, recall precision, context window utilisation — differ enough between applications that the infrastructure needs to be tunable in ways that generic frameworks handle poorly.