My experience of the book was similar: the first third was great. Great idea, brilliantly executed. Definitely worth it for the first third alone.
In a way, maybe it going off-piste is coherent with the idea of the first third. I'm sure this was not the author's intent, but fun from an ironic perspective.
It certainly invites that level of meta-commentary on its own structure, though I agree it's inadvertent. And I know at some point someone is going invoke that point in full sincerity as if its an answer, and whatever that is, the satisfied meta-commentary that makes too much of irony as if its a sincere insight, I feel like is just looming as a possible and frustratingly shallow justification of the book. There's an interesting question there of the scales of abstraction at which anti-memes could function, and that's fascinating but as you noted in this instance not necessarily intentional.
It makes me think of the movie Doubt, where I remember being sincerely confused as to the central accusation at the center of the movie (though retrospectively its obvious and I knew it was at least one possibility but I wasn't sure if there was perhaps a different interpretation), and was told that not being sure was the point and by expecting an answer I was missing the point since the whole movie is about "doubt". I felt this explanation was, frankly, just stupid. Just because you're going meta doesn't mean any point coherently registered in the form of meta-analysis is insightful. But anyway, I'm off the rails a bit now going after imaginary adversaries, but agree with everything you've said.