FTA:
> Strange, isn't it? The AI hasn’t crashed. It’s still running.
As a human I answer a question because my time to do so is finite. Why can't we just ask an AI to give its best answer in due time ? As a human I can do that easily. Will my answer be optimal ? No of course, but every manager on earth do that all the time. We're all happy with approximate answers. (and I would add: approximation are sometimes based on our core values, instinct, consciousness, etc.. All things that make us humans, IOW not machines)
G. E. Moore (in his Principia Ethica, 1903) makes a very similar case to this relation to consequentialist ethics:
"The first difficulty in the way of establishing a probability that one course of action will give a better total result than another, lies in the fact that we have to take account of the effects of both throughout an infinite future. We have no certainty but that, if we do one action now, the Universe will, throughout all time, differ in some way from what it would have been, if we had done another; and, if there is such a permanent difference, it is certainly relevant to our calculation.
But it is quite certain that our causal knowledge is utterly insufficient to tell us what different effects will probably result from two different actions, except within a comparatively short space of time; we can certainly only pretend to calculate the effects of actions within what may be called an ‘immediate’ future. No one, when he proceeds upon what he considers a rational consideration of effects, would guide his choice by any forecast that went beyond a few centuries at most; and, in general, we consider that we have acted rationally, if we think we have secured a balance of good within a few years or months or days."
tldr: it's impossible to know for sure which choice is the absolute best.
In a sense, I get why they write verbosely, but...
The first and most important task of our lives is to determine what our goal is.
https://en.wikipedia.org/wiki/Alfred_North_Whitehead#God
> Why can't we just ask an AI to give its best answer in due time ?
Sure you can. One approach is https://arxiv.org/html/2505.11274v2 another is having a parallel "do you want to do more analysis?" agent, and I'm sure someone's already at least experimenting with building the confidence measurement into the layers as well.
You can go recursive though, the intrusive thought firing again and again, eating yourself in doubt and endless overthinking things. Which indicates which system chemically regulate and dampens and action/reaction in the human mind.