I think you are way too optimistic. Even with an antimatter drive and 100% conversion efficiency, such rocket would have a fuel to payload ratio of >1000.

That seems pretty promising actually.

Our moon landing missions had a similar ratio, so I assume we can do the engineering to make even a slightly worse ratio work for us a 100 years after it.

In practice it would be better with slingshot maneuvers and picking up mass on the way.