If you use AWS Config with an organization wide aggregator, which Control Tower sets up by default these days, have your agent use it to find resources and related resources. You can build most of the graph from that alone without traversing accounts.