More concretely, I'm asking this: why aren't applications compiled fully to native code before distribution rather than bytecode that runs on some virtual machine or runtime environment?
Implementation details aside, fundamentally, an Android application consists of bytecode, static resources, etc. In the Java world, I understand that the main appeal of having the JVM is to allow for enhanced portability and maybe also improved security. I know Android uses ART, but it remains that the applications are composed of processor-independent bytecode that leads to all this complex design to convert it into runnable code in some efficient manner. See: ART optimizing profiles, JIT compilation, JIT/AOT Hybrid Compilation... that's a lot of work to support this complex design.
Android only officially supports arm64 currently, so why the extra complexity? Is this a vestigial remnant of the past? If so, with the move up in minimum supported versions, I should think Android should be transitioning to a binary distribution model at a natural point where compatibility is breaking. What benefit is being realized from all this runtime complexity?
The Androids book by Chet Haase provides a good look at the early history and design decisions of the platform and how they came to be made.
At the time there was a debate inside the team over what language their app development framework should use, with native C++ and Java being the two main options (I think there might have been another option or two, I can't recall). In the end Java won out, and from memory one of the main reasons was to make it easier to make apps and not need to think about the lower level parts of the platform, i.e. the platform takes on the complexity internally in order to lower the barrier to entry for app developers. The idea being that a lower barrier to entry would result in more apps being developed for the platform. For a brand new platform that lives and dies by the apps available for it, that's a pretty sensible trade-off.
And yes, Android has a lot of vestigial remnants of the past, the Android framework team has been very particular about maintaining as much backwards compatibility as possible within the framework.