I can explain what's going on here. For context, you need to know how somebody becomes a doctor in the US.

1. You get a 4 year degree in college. You hopefully get a very good GPA. You need to do so-called pre-med classes that really don't have much to do with medical education but are known as "weed out" classes, particularly Organic Chemistry. If you don't do these in your 4 year degree, you can do a program afterwards called a post-BAC;

2. At some point you take the MCAT. You may need to take it multiple times to get a sufficient score;

3. You apply to med school with your transcripts, any relevant experience, your MCAT, a personal statement and letters of recommendation. This is an onerous process. Demand greatly exceeds supply. You will need to do an interview (if you get that far);

4. If you get accepted you will do a 4 year program that's broadly characterized as MD or DO. It's easier to get into a DO school but they have worse match rates into residency, particularly for competitive specialties. There's also the international option, particularly Caribbean schools. They have even worse match rates;

5. Now begins the US Medical License Exam ("USMLE") process to become a doctor. You take Step 1 as an M2 (second year medical student). Typically the first 2 years of med school are academic. The last 2 are mostly clinical where you do rotations in various specialties;

6. As an M4 you have to do these rotations as well as take Step 2 (of the USMLE) and do your residency applications. This is probably the most stressful part because you can end up unmatched and then you've spent $400-800k+ to not become a doctor, at least not immediately and probably not in your preferred specialty;

7. To apply for residency you apply to programs, hopefully get an interview and then submit an application for each program you're interested in. This again includes letters of recommendation (very important), transcripts, your Step 2 results (Step 1 is now pass/fail, more on this below), research, etc. Applicants rank their programs. Programs rank their applicants. A matchin algorithm compares the two and attempts to essentially place each applicant in their most preferred program. Not all specialties do this. You can also attempt to match as a couple (usually used by married people);

8. If you match you're now contractually obligated to do that program. Depending on the specialty it's going to be 3-7 years, more if you do a fellowship afterwards. You basically get paid minimum wage for that entire time. Somewhere in there you need to take Step 3 and at the end do your medical boards to be licensed to operate independently as a medical doctor.

9. If you don't match, it gets real awkward. You either scramble for an open spot (a process called the SOAP), extend medical school for a 5th year (so you don't have the stink of having failed to match, seriously) or do a research year to improve your odds next year. Note that you can match into incomplete programs (eg an intern year only program).

So, let's do the math. In a perfect world you graduate high school at 18, college at 22, get accepted immediately, graduate medical school at 26, match immediately and then complete residency at 33 (for a general surgery residency program). That's a lot of education and training. You likely have $400k=$1M in debt by this point. And only now do you earn a real income.

But it often doesn't go that way. You may fail to get into medical school the first time. You may not have realized you wanted to have been a doctor so you had to do 1-2 years of a post-BAC. So you might be 25-26 before you start medical school. You may fail to match or not try and do a research year. Or you might do an MD-PhD program and take a few extra years to graduate. Combined with a fellowship, that 33 years of age might turn into 40 years old.

So one thing that changed in the last few years is that Step 1 went from a score to pass/fail. This is ostensibly to reduce the stress of having a bad score. Some med schools are also pass/fail rather than having a class ranking. What this means in practice is that school reputation and ranking become more important. These are harder to get into obviously so it has a knock-on effect into undergrad. So if you go to Harvard undergrad, you'll generally have a better chance of going to a T20 med school. But how do you get into Harvard?

But let me bring this long-winded thing back to research. Over the past decade, the number of research items for each matched resident has massively increased, more than doubled in some cases. Some med schools are research-heavy so going to those has become a competitive advantage. It means people who successfully match into a competitive specialty are more likely to take a research year before applying. This is particularly true for neurosurgery.

Income potential and lifestyle massively vary. Primary care (family medicine) and pediatrics have awful earning potential. Any surgical specialty, dermatology (I honestly don't understand this one) and radiology have much higher earning potential. The difference can be 5x or more.

So I guess this is a really long way of saying that churning out low-quality research is resume-padding. Residency programs don't even tend to care about the quality of your research. It's just the number of research items you have. Increased competitiveness of certain programs combined with reduced signal in other areas (particularly Step 1 going pass/fail) may have exacerbated the situation.

So anyone who complains about how much doctors earn should look at the time it takes and the years of exploitation as a resident. Maybe a doctor wouldn't be so expensive if it wasn't so expensive to become a doctor. You will also find a large number of physicians who would take a big pay cut if they didn't have to deal with insurance.

do you know why residencies value the number of research items? Why would having a large number of garbage papers be seen as a positive signal at all?

Attendings and existing residents are consulted in the ranking process, they are picking people they will have to work pretty closely with for 4 years, they have skin in the game. Why does anyone put any weight on such a clearly bogus metric?

don't existing doctors aim to keep supply short? Seems like it's like the homeowner problem (existing homeowners always want their house prices to go up)

It's complicated. The AMA seeks to restrict supply. The number of medical schools has been cut dramatically from historic highs. The AMA relented in WW2 because of the need for combat physicians but then went right back to their old ways after WW2 ended. Doctors don't necessarily share that view.

But there are multiple issues that contribute to shortages. Just like with homes (as you brought up), there can be homes where nobody wants to live. Likewise, some specialties never fill all their places. So earning potential is a factor. Not wanting to live rural is often also a factor, despite efforts to attract people to both using things like PSLF. PSLF itself is on shaky ground under this administration and you will see physicians unwilling to sacrifice career potential for a program that won't trust will be there to forgive their debt.

And then there's burn out. Many doctors leave the profession in their 40s and 50s. And if you didn't really become a doctor until your 30s, that's a relatively short professional life. But why do they burn out? Insane hours, administration, insurance, work-life balance are all up there.

> dermatology (I honestly don't understand this one)

Botox and other cosmetic procedures. In any big city you can find swanky dermatology practices offering expensive cosmetic procedures to rich people.