Honestly, it's the training costs that will kill them. AFAIK, training costs have not come down anywhere near as much as inference costs.

And the models don't last long. So you have a rapidly depreciating capital asset that you need to provide your services, not really a recipe for a sustainable business (certainly not with the fat software margins tech companies are used to).