It would be nice if the detached signer and the normal wallet could both
verify the correctness of generated addresses before you cause coins to be
sent there.

e.g. the hardware wallet could give its master public key to Bitcoin Core
and you can thereafter generate your receiving addresses on Core, with the
option to have the HW wallet validate them.

One of my biggest fears about using any wallet is the "whoops, cosmic ray
flipped a bit while producing receiving address; SFYL!" possibility. For
high value cold storage, I always generate my addresses on two independent
machines using two different pieces of software. Am I nuts for doing that?

With the above scheme, you are pretty well protected from losing money if
your HW wallet is defective. You could still lose it if the HW wallet was
evil of course, but that strikes me as much more likely to be discovered
quickly.