I teach computer science at a public university. Every semester I have kids who come to classes but never turn in any homework. They also don't withdraw either. I'm literally forced to fail them because I have nothing to grade them on.

Humans make such mistakes slowly. It's much harder to catch the "drift" introduced by LLM because it happens so quickly and silently. By the time you notice something is wrong, it has already become the foundation for more code. You are then looking at a full rewrite.

The rate of the mistakes versus the rate of consumers and testers finding them was a ratio we could deal with and we don’t have the facilities to deal with the new ratio.

It is likely over time that AI code will necessitate the use of more elaborate canary systems that increase the cost per feature quite considerably. Particularly for small and mid sized orgs where those costs are difficult to amortize.

Or maybe this is a SaaS opportunity for someone.

To borrow a concept of cloud server renting, there's also the factor of overselling. Most open source LLM operators probably oversell quite a bit - they don't scale up resources as fast as OpenAI/Anthropic when requests increase. I notice many openrouter providers are noticeably faster during off hours.

In other words, it's not just the model size, but also concurrent load and how many gpus do you turn on at any time. I bet the big players' cost is quite a bit higher than the numbers on openrouter, even for comparable model parameters.

Zed on Linux is buggy as hell. On macos it's somehow more stable. Maybe Zed is indeed a "good" example of Ai-coded products.

This is a thing you can credibly say only if you've never used a popular new non-mainline editor. Sublime Code was "buggy as hell" too; all new editors are. Editors are incredibly difficult to do well. And Zed is doing it on hard mode, cross-platform.

Bigger than Windows 98

Oh junior devs submit PRs that don't fully work all the time.

I have this DVD set in my basement. Technically, there are still methods for estimating the probability of unseen ngrams. Backoff (interpolating with lower grams) is an option. You can also impose prior distributions like a Bayesian so that you can make "rational" guesses.

Ngrams are surprisingly powerful for how little computation they require. They can be trained in seconds even with tons of data.

Thank you for this wonderful answer.

If models are profitable once trained, isn't it weird that chatgpt and Claude have $200 tiers that still have usage limits?