In 2024, the average US prison sentence was for a duration of 52 months, with over half the total months being for drug trafficking (average 82 months) https://www.ussc.gov/sites/default/files/pdf/research-and-pu... (page 11).
Also in 2024, Japanese prison sentences were mostly between 1 and 3 years, and about two thirds were fully suspended. https://www.moj.go.jp/content/001436547.pdf (page 20). Assuming an average of two years for the third not-fully-suspended, that works out to 8 months of time spent in prison per person handed a prison sentence.
That results in about a factor of 7 difference in prison population simply due to the difference in average sentence length. Maybe the US should sentence more people to prison but keep each of them there shorter.
The US court system already relies heavily on plea deals to not jam up the courts. Increasing the throughput of the justice system to facilitate your idea would require an order of magnitude investment into more courts, judges, public defenders, prosecutors, etc. I'm all for it, but the incredible cost does have to be considered