If you use agents to generate formal verification, how do you tell if the properties that are verified are the correct ones?

In my view, AI currently relies primarily on human-developed tools for verification. Although AI has used some reinforcement learning (RL) algorithms to identify the paths humans have taken at each level, these paths are essentially still based on human mathematical tools and require confirmation by human experts. We need a more revolutionary framework to provide AI for mathematics with new methods and verification pathways.

Applications: implications, simulations, and experiments. Gobble the frontier; wring out the usable.

This reads as gibberish.

make no mistakes

mostly jk, although agents themselves are of course getting better at detecting whether a formal spec is faithful to a natural language spec

You don't. You place unwavering faith in the machine god.

This is why, for formal methods to really be the next step, we'll need deterministic source to CoC language compilers, and we'll need to go back again.

Lean is kind of an unfortunate, cargo cult choice because it only compiles to C. Really for software we should use a prover that has a lot more targets, like Danfy. Even Rocq (formerly Coq) can compile down to several functional languages, which is more than we can say for lean.

As usual, the AI crowd latched on to the first solution that came to mind, and thanks to its prominence in mathematics applications recently just lazily are starting to try and apply it to computing, without actually understanding anything or weighing trade offs. Of course, now we are basically stuck with Lean since the models will have been trained hard on it. At least it's a CoC language, but now we need to build a bunch of extra compilation around it for this to really work.

Most of this feels fairly orthogonal to the question asked. And the reasons lean are chosen are fairly simple: it is designed to also be used for programming and it's actually successful. Compiling to C is fairly obviously a bad choice (well I guess not obviously, people keep implementing languages this way for some reason) but arguably one of the least important factors in choosing which prover to use. The alternatives to lean would be Idris (which is both unsound and unpopular), ats (which is unpopular) or rocq (which is more reasonable but still is mostly an academic project).

Compiling to C seems obviously an excellent choice for compiling a language to. It is a good choice as a target language, and one can rely on a lot of existing tooling, a stable standard, good interoperability, and wide platform support. It is also simply enough to be modeled precisely.