Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Wouldn't this increase your token usage because the tokenizer now can't process whole words, but it needs to go letter by letter?


It doesn't go letter by letter, so not with current tokenizers.

There will likely be some internal reasoning going "I wonder if the user meant spell check, I'm gonna go with that one".

And it'll also bias the reasoning and output to internet speak instead of what you'd usually want, such as code or scientific jargon, which used to decrease output quality. I'm not sure if it still does




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: