Thank you for the detailed and technical response. I’m not doubting it providers benefits at the technical level, and I’m keen to dive into it deeper, but I was hoping to understand what benefits this translates to at a user level. I’m imagining something like it can lead to faster switching between between agent calls, or even at a developer level, it allows you to run agents/llms in a more constrained environment, therefore you can run more agent containers in parallel?
I’m familiar with enough with Java that I can squint and follow some of it, but I’m hoping to understand more at a macro level before committing to diving into the technical details.
Again, thank you for the information already provided.
You're welcome, I appreciate the engagement!
Your intuitions are right, faster switching between agent calls, and tighter packing of agents in the same amount of compute space.
There's a second thing besides just efficiency that I provide here however. At a developer-user level, it automagically provides JVM developers (not just java) a frictionless way to add AI into their stack with a single dependency.
It makes using AI in java as easy as it is in python.
I'm providing the [what I've found to be missing] layer for AI in java in the most efficient way I could.
I'm not sure if by user you meant developer (user of this code) or user as in end-user. Let me know if I still haven't answered your question fully yet!
Great. I feel like this is something that I’m going to dig into more. At day job I work with a platform that’s built on OSGI and JCR. I’m keen to explore if this could enable any interesting interactions there (they might not necessarily be good interactions though haha).