I feel like you're asking two different questions: what do we do about average criminals, and what do we do when our normal strategies don't work?
Most shoplifters stop after they get caught the first time, even if they don't face any further consequences. Most people who keep shoplifting after that stop if they are caught and face consequences like community service. In general, the system of escalating punishments before prison actually works to keep problems from developing to that level.
If someone is doing serious violent crimes then it seems obvious that they need to be imprisoned until we can figure out how to make them stop. But what if someone is doing minor property crimes where the total value of those crimes does not exceed the cost to imprison them? In that case we are essentially offering an incredibly inefficient subsidy to property owners.
The real answer is to directly address the factors that lead to crime and disorder, such as cost of living exceeding prevailing wages, unemployment, drug addiction, and mental illness.
Most criminals do not stop after they get caught the first time.
I can't find shoplifting-specific statistics, but then again, essentially nobody is being jailed for only shoplifting anyways, so it's a moot point in a conversation about incarceration.
On a broader discussion of the importance of incarceration in reducing continued victimization of society:
"In many states, up to 80 percent of the youth who are incarcerated are rearrested within 3 years of release" - https://csgjusticecenter.org/publications/reducing-juvenile-...
"According to the National Institute of Justice, almost 44 percent of the recently released return before the end of their first year out. About 68 percent of 405,000 prisoners released in 30 states in 2005 were arrested for a new crime within three years of their release from prison, and 77 percent were arrested within five years, and by year nine that number reaches 83 percent.
The solution for your example of minor property crimes is reducing the cost of incarceration, not letting them continue commit property crimes and therefore offload the costs of society's failure to police criminals onto citizens.
To your point of "how do we make them stop", incarceration/forced hospitalization is ironically the answer to some of your supposed "causes" of crime, like mental illness and drug addiction. Look to a place like California or Seattle to see the impact of these approaches, where every imaginable alternative has been tried to control homelessness, drug addiction, and nuisance crime, far exceeding the cost of just locking them up.
Recidivism in the USA is higher than almost every other nation [0], at about 65-70%.
You're absolutely right that they don't stop, the first time, in the US. However, the cause of that same recidivism, is overwhelmingly correlated to a lack of support on leaving. Rehabilitation is in the name - habeo, to have or possess.
Why do people reoffend? Because currently, they're fucked. Background checks in everything, screw them over. They can't get work, they can't rent, and so, when all other options are off the table, is it actually a surprise, that they reoffend?
It'd be a hell of a lot cheaper, not to lock them up, and not expose a first-time offender to a black hole where everyone says the world would be better off with them dead.
See for example, Norway, with the lowest recidivism rate in the world [1], where it actually is true, that most do not reoffend.
[0] https://bjs.ojp.gov/topics/recidivism-and-reentry
[1] https://digitalcommons.coastal.edu/bridges/vol10/iss10/2/