It's not really the size that matters in this case, because the video is loaded after the page is completed, and in theory, ought not be slowing down the page if at all (my cursory examining of the network tab gives me a sort of confirmation).

However, what is indeed slow is the initial load, and the lack of CDN for their static assets (css etc). When the HN effect started eating their resources, these static assets are what hurt their load time the most.