Small steps of nondeterministic computation, checked thoroughly with deterministic computation every so often, and the sky is the limit.
That's when A.I. starts advancing itself and needs humans in the loop no more.
Small steps of nondeterministic computation, checked thoroughly with deterministic computation every so often, and the sky is the limit.
That's when A.I. starts advancing itself and needs humans in the loop no more.
> That's when A.I. starts advancing itself and needs humans in the loop no more.
You got to put the environment back in the loop though, it needs a source of discovery and validity feedback for ideas. For math and code is easy, for self driving cars doable but not easy, for business ideas - how would we test them without wasting money? It varies field by field, some allow automated testing, others are slow, expensive and rate limited to test.
Simulation is the answer. You just need a model that's decent at economics to independently judge the outcome, unless the model itself is smart enough. Then it becomes a self-reinforcing training environment.
Now, depending on how good your simulation is, it may or may not be useful, but still, that's how you do it. Something like https://en.wikipedia.org/wiki/MuZero
Electric dreams. Simulation of what?
Think scams and pure resource extraction. They won't consider many impacts outside of bottom line.
Simulated environment suggests the possibility of alignment during training but real time, real world, data streams are better.
But the larger point stands: you don't need an environment to explore the abstraction landscape prescribed by systems thinking. You only need the environment at the human interface.
The question is where should AI advance itself? Which direction? There are an infinite number of theorems that can be derived from a set of axioms. Infinite. AI can't prove them all. Somebody needs to tell it what it needs to do, and that is us.
Your checks don't have to be deterministic either.
Eg randomised quicksort works really well.
Couldn't disagree more.
Sorting a finite number of elements in a sequence, is a very narrow application of AI, akin to playing chess. Usually very simple approaches like RL work totally fine for problems like these, but auto-regression/diffusion models have to take steps that are not well defined at all, and the next step towards solving the problem is not obvious.
As an example, imagine a robot trying to grab a tomato from a table. It's arm extends across 1 meter maximum, and the tomato is placed 0.98 meters away. Is it able to grab the tomato from the point it stands, or it needs to move closer, and only then try to grab the tomato?
That computation should better be calculated deterministically. Deterministic computation is faster, cheaper and more secure. It has to prove that: $tomato_distance + $tomato_size < $arm_length. If this constraint is not satisfied, then: move_closer(); Calculate again:$tomato_distance + $tomato_size < $arm_length.
From the paper:
> Our system employs a custom interpreter that parses "LLM-Thoughts" (represented as DSL code snippets) to generate First Order Logic programs, which are then verified by a Z3 theorem prover.
> Sorting a finite number of elements in a sequence, is a very narrow application of AI, [...]
Sorry, I did not suggest you should use AI to sort numbers. I was solely replying to this:
> Small steps of nondeterministic computation, checked thoroughly with deterministic computation every so often, and the sky is the limit.
You don't necessarily need your checks to be deterministic.
In fact, it's often better for them to be not deterministic.
See also https://fsharpforfunandprofit.com/series/property-based-test...
I don't understand your claim about 'Deterministic computation is faster, cheaper and more secure.' That's not true at all.
In fact, for many problems the fastest and simplest known solutions are non-deterministic. And in eg cryptography you _need_ non-determinism to get any security at all.