Didn’t the Dutch basically take over the Portuguese trading empire from them?

Yes, to a significant but not total degree. Some of those losses were later recouped by Portugal (current Northeastern Brazil and Angola).

I think that the losses in Asia were more lasting, or permanent.