Cool! Yeah, that's the kind of UI/UX I meant.

I agree with the right abstraction and it's tough to find the balance- in our data pipeline app, what we did is make key core functionality of the app exposed so the assistant can use it, and implemented a handful of basic agents out of the box, including one default one that could shell out work to others. We also made it easy as an extension point for users to add a new agent that used the core functionality/tools, just by defining the agent in a markdown file.

We found starting small for critical use cases that saved the most time, but thinking about building blocks, was useful.

Because the responses of the AI assistant come back and are processed on the UI, we found we could give the LLM our UI docs as well as knowledge about UI element IDs, etc so it could respond with input commands that would drive the UI.

This way, we could do something like, provide the LLM with the input/prompt including the context of like - what page/view is the user on, what is their intent, what tools are available in general, what sub agents are available for specialized tasks, etc.

Please don't let my suggestions sway you away from core progress in your app (take with a grain of salt). But it's great you're already experimenting- keep your eyes open if you see a great use case where it accelerates workflow.

Another HNer mentioned people not reading docs- that's a low hanging fruit use case we had too - "how do I use this view?", "what does this field mean?", or retrieving information from other parts of the app without having to navigate away, etc. It can save having to find answers in a doc or navigate elsewhere.

Edit: perhaps a useful exercise - imagine a workflow of "talking to the app to achieve a task" as a way to explore.

"Hey ERP, open the part entry screen for part 12345"

"Hey ERP, can you update the description for part 12345 to correct the spelling error?"

"Hey ERP, how many of widget XYZ are in stock? If there are enough in stock, can you transfer quantity 10 from warehouse A to B?"

"Hey ERP, how do I cancel a sales order?"

"Hey ERP, how does this screen work?"

I think if you break these down, you'll find common abstractions that map to features, API endpoints, user interface sequences and interactions, triggering workflows, looking things up in docs, etc.