It is jarring to me that most of the dancing seems slightly out of sync with the music. It is like a music video uncanny valley - images look good, but the lack of sync to the sound shatters the illusion entirely.
It is jarring to me that most of the dancing seems slightly out of sync with the music. It is like a music video uncanny valley - images look good, but the lack of sync to the sound shatters the illusion entirely.
Well yeah, because music is not a modality of the models involved at all. It's literally just an LLM with the timestamps of each line injected into its context, and then making an API request to image / video generators. I don't really understand what the point of this project even was; stitching together dumber models with bespoke glue logic like this takes us further away from general intelligence and confuses the public.
While it's made huge improvement in just the past few years, AI still hasn't quite solved motion, especially human motion. The humans in these videos are rendered really well, but move unnaturally. Like someone did a motion capture of a real human, and then played it back with a very high quality 3d model--something hard to describe is lost in the processing. Everything seems to move just too smoothly, at too exactly constant a speed. Same for camera movement. A real life camera doesn't precisely follow some exact 3D Bézier curve at an exactly constant speed. AI doesn't get this yet.
To me nearly all of the real world dancing looks like that. The dancers could literally be olympic level and I still don't see the connection between their movements and the sounds of the music. Music videos sometimes do sync for me but only on very simple motions, and for whatever reason shuffle dance on video almost always looks great.
How many hours of your life would you say you’ve danced? Have you ever trained at a sport beyond the recreational level, especially highly rhythmic ones like basketball or soccer?
Genuinely curious, because I have friends without any sense of rhythm and it’s interesting how it manifests
> How many hours of your life would you say you’ve danced?
I'd say less than 20? Maybe even less than that. Even counting short bursts, to especially compelling music, that did't last more than few seconds.
The closest I ever felt, to what I imagine people might feel while dancing, was playing Beat Saber in VR. You could have your friends try, they might enjoy it. I did. I spent way more hours on Beat Saber than I ever spent dancing. But I don't see any improvement from that.
> Have you ever trained at a sport beyond the recreational level, especially highly rhythmic ones like basketball or soccer?
I don't think I ever did anything in my life beyond recreational level. Even coding feels recreational level regardless of how much I get paid.
Basketball is completely incomprehensible for me. The rule that you can't do more than one step while holding the ball is impossible and unreadable when somebody else does it. Dribbling is pretty hard too.
I played some soccer as a kid and at school but I wasn't good at this because of multiple factors. I mostly played defense when sometimes I could interrupt the attacking player mostly with a single action, not a sequence. I wouldn't even think about soccer as rhythmic sport, but I imagine it can be, if you are leading the ball which I pretty much never did successfully.
I hate repeatable movements done during exercises. Never participated in any exercise groups (beyond what was enforced at school). I like biking but I vary my pedaling style a lot while riding and I don't ride for too long, few hours at most. I like walking but I zone out completely during walks. I can't run due to rapidly diminishing lung capacity when I try but I imagine if I could it would resemble walking.
I don't play instruments and never had any success in that area. I can't do and don't enjoy trying souls like games. I'm not a fan of platformers with all the repeated jumping and timing their repeats to what happens in the level. From games that require skill I prefer (quake-like) FPPs.
On the plus side, I have decent full body reflexes and rarely ever fall. In falls reliably instincitvely shield myself with arms to prevent damage to the head and mostly body. I can't imagine getting hurt by stepping on lego brick because my feet react to what I'm stepping on quickly shifting the load around any sharp or uneven objects. The brain itself is decent as well. I'm in top <1% according to Mensa iq test results.
It feels like cyclical engine of my brain is very weak while the reactive one is fairly good.