“It’s just a chat bot Michael, how much can it cost?”

A philosophy degree later…

I ended up just generating a summary of each of our 1k docs, using the summaries for retrieval, running a filter to confirm the doc is relevant, and finally using the actual doc to generate an answers.