What about request caching? If you swap to a cheaper model mid execution it migh... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

k9294 55 days ago | parent | context | favorite | on: Show HN: Smart model routing directly in Claude, C...

What about request caching? If you swap to a cheaper model mid execution it might cost more that to make multiple requests to the already cached provider?

adchurch 55 days ago [–]

Yep 100%, mentioned this in another thread (https://news.ycombinator.com/item?id=48689448) but tl;dr we build the router to be cache aware

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact