Yep. If you hear Chaucer read aloud by someone who knows the phonetics and the meaning it's instantly like 75% comprehensible. If they give you a bit (like, half a class-period) more help, you'll get to ~90% without any more effort than that. Chaucer is more or less as incomprehensible as Glaswegian - that is to say, it's a bit tricky, and you won't know all the slang, but you can get used to it pretty quickly.
What's really fun is that if you keep the dialect in your head, and try reading it out loud with your best "Chaucer accent" it feels like slipping on a pair of glasses: you immediately "get" stuff that you'd have thought was impenetrable just looking at the page.
Source: my medieval lit classes, with a teacher who was really good.
Replying to myself to add that that last ~10% is an enormous time-sink. You'll have to look up every unfamiliar word, which isn't a big deal for a short text, but becomes one for something longer. Worse, though, is that you'll constantly be led astray by words whose meanings have drifted over time: your interpretation (often) won't align exactly with that of Chaucer or his audience. What I said above is good enough for casual / undergraduate use - the stories are still funny, and / or have enough depth for a good discussion - but there's a reason "medievalist" is an actual speciality: you'll spend a whole career trying to get to 100%.