Neither does the GPLv3:
A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an “aggregate” if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.
If you'd include a GPLv3 gallery app in, say, a mobile OS, it does not mean that the rest of the OS has to be under the GPLv3. It merely means that you cannot limit the user's right when it comes to the GPLv3-part (the gallery app). They would still be allowed to redistribute/modify it and you have to provide the source code on request.
You only have to make other code GPLv3 if you somehow create a derivative work (e.g. linking against a GPLv3 library).
(IANAL blah blah)
The issue isn't that GPLv3 applies to non-derivative OS components but rather that the terms apply to the overall redistribution of the OS. Including any GPLv3 component means GrapheneOS cannot be used in products where AOSP can be used due to having additional restrictions beyond GPLv2. We don't want more restrictive licensing than AOSP. We're fine with using additional GPLv2 code as long as we're sure it's not going to move to GPLv3 or that if it does we're prepared to maintain it ourselves.
Name the restrictions.