I've been working on something like this the last few months specifically around service quote analysis (repairs, construction, hvac, auto, etc.) and it's really cool. I think LLM analysis is the way to go because the amount of complexity is absolutely staggering - just to start the difference in quality and information available on a quote is drastically different between vendors within the SAME vertical. Then to do actual do analysis on local laws, the details of your property (not just photos/videos, but zoning and lot details), vendor analysis, etc.
On top of it all, the most important thing to consider is intent -> An emergency plumbing visit is often very different than a proactive upgrade.
edit: spelling
how do you handle the LLM hallucinations in analysis? I like it for data extraction but i never trust it to analyze anything
First, I've spent a ton of time becoming opinionated about a normalized data model that supports the product experience I'm trying to build. This applies both to the extraction (line items, warranty sections, vendors, etc.) and the analysis portion. The latter is imperfect, but aligns philosophically with what I'm willing to stand behind. For example
- building outputs for price fairness (based on publicly available labor data)
- scope match (is vendor over/under scoping user's intent)
- risk (vendor risk, timeline, price variability, etc.)
- value (some combination of price, service, longevity, etc.)
I don't get much hallucinations in my testing, but overall it's pretty complex pipeline since it is broken down into so many steps.