> it always seemed a bit backward to me for the importer to ask for lazy import, especially if you make it an import keyword rather than a Python flag
Exactly this. There must be zero side effects at module import time, not just for load times, but because the order of such effects is 1) undefined, 2) heavily dependent on a import protocol implementation, and 3) poses safety and security nightmares that Python devs don't seem to care much about until bad things happen at the most inconvenient time possible.
> Of course that doesn't solve the overhead of finding the modules, but that could be optimized without lazy import, for example by having a way to pre-compute the module locations at install time.
1) opt for https://docs.python.org/3/reference/import.html#replacing-th...
2) pre-compute everything in CI by using a solution from (1) and doing universal toplevel import of the entire Python monorepo (safe, given no side effects).
3) This step can be used to scan all toplevel definitions too, to gather extra code meta useful for various dynamic dispatch at runtime without complex lookups. See for example: https://docs.pylonsproject.org/projects/venusian/en/latest/i...
3) put the result of (2) and (3) as a machine-readable dump, read by (1) as the alternative optimised loading branch.
4) deploy (3) together with your program.
For optimizing the module finding, using a custom import hook was indeed what I had in mind!