I was a little shocked that they could get it fully working in a week to be honest. My side project is a very similar ambition (https://tsz.dev) but I am in no way claiming success. i keep adding more and more tests to ensure things works. Even after all of TypeScript's own tests pass I am finding bugs which I was totally expecting.
The bar for matching tsc's behavior is really _really_ high. see:
https://github.com/type-challenges/type-challenges
I'm not against using LLMs to write a lot of code. But verification should be 100x more robust now that we can output code at this rate.
I'm stunned that it went from 'this is an experiment' to merging a ~million lines of (likely) unreviewed code in a week. I have nothing against using agents but to rush something like this and leave the community blindsided seems extremely ameteurish. Like something you'd expect a bright eyed graduate engineer to do.
Blindsided? Has there even been a release yet?
Yes, they released a canary version.
True enough. Every merge to main releases a canary though
tsz for me is an experiment to see how can this kind of work be done better. With a slight difference that tsz is not a direct port and it's a different architecture. I'm also not claiming to have answers but I've learned a ton. A few things that works
- Test before code, Bun had lots of test so that's good but maybe they could start by asking Mythos to write like 20k additional tests that pass on Zig Bun first.
- Deterministic anti-slop features. LLMs love to solve the problem in the wrong abstraction layer or place. There are many ways to catch this with deterministic tests. I do this in tsz a lot
- Roadmap that constantly evolving by humans.
- Taking a pause and looking how the progress is going and undoing slop
- Fuzztest(https://github.com/google/fuzztest) style "trying to break things" with the powers of LLM
I suspect they've been planning this and experimenting for many months. Along with the large existing test suite, they have lots of tooling for parallelizing agents and an unlimited token budget. So don't feel too bad..
Are there any evidences which prove the process was done in a week?