AnythingLLM is great for this, it even enables you to setup RAG from your own set of docs, which it then retrieves via embedding and adds as context.

I use that + OpenRouter which gives me API access to more models as well. Huge fan of this approach.