I'm extremely interested in this sort of use case and am regularly disappointed to find that very few others are. And even fewer are interested in doing it in a hypermedia-first way.
There's two ways this could be approached.
1. Just cache the backend-generated html that has datastar attributes in it, and serve it from a service worker. Datastar library in the main thread won't know or care whether it came from service worker, edge worker or your backend.
2. If there's a need for dynamic data/templating (eg user-specific into), you can do so in the service worker - if you've stored the templates and data already. But that has little to do with datastar - you'd have to do that in any case.
However, service workers have limitations with regards to things like dynamic script imports, using es modules etc. You can send messages to a dedicated web worker which handles rendering and returns the html, but it does get complicated. Sharedworker would be even better but android chromium still doesn't support it - but they're finally trialing it and hopefully it'll be available in 6 months.
The only datastar-specific consideration is if you want to do something like stream sse events from the "backend" to the datastar library in the main thread/dom. The TypeScript/JavaScript sdk works just fine in the service worker as well, and can maintain a long-lived connection to periodically stream new messages. Of course, service worker lifecycle is somewhat of a black box and out of your control, but that has nothing to do with d*.
So, overall, yes you can use datastar offline. But it comes with all the issues that you'd face with another approach.