The highly regulated, zero tolerance for error environment is a huge problem because anyone doing these jobs is more like a small sample size LORA than a general model.

It doesn't matter how smart someone is, they need specialized training to be good at these jobs. Specialized training in the area the company specializes in.

There is a category error in all this that is hard to think about because of the normal discourse and ordinary language. We say people work in "finance" but no one works in just "finance". They work at a company that has a specialization within "finance", inside a hierarchy that has specialization on top of specialization.

What we really need is exactly what we don't have and aren't going to get. A type of LORA that generalizes the task specific intelligence needed from a very small sample size and that in practice makes so many less mistakes in a highly regulated, zero tolerance for error environment that it is irresponsible to not use the model.

I have worked in this type of environment for 3 years and I have made zero mistakes in 3 years. The people that make even a small number of mistakes get fired.

Any real automation in this area is going to be incredibly slow and piecemeal over a long period of time because even an amazing model would need a long time to prove itself against what the human standards for error rates are.

Even the ensemble average error rate on a large number of tasks in space would not be good enough. It needs to be an average error rate over time.