Of course there is good technical consensus for CLTV by IsSuperMajority() in the same way as BIP66 was rolled out. I believe the only open question is whether we have to account for XT's use of versionbits (because the standard has not been finalised). One can take the view that it is a non issue given the almost negligible number of BIP101 blocks, but it certainly goes away if XT also merges BIP65/CLTV.

As for risks, I think we learned a lot from BIP66:

1. miners are now aware of the risks of SPV mining near activation and are financially incentivised not to during that period. 

2. As for SPV wallets need to handle awareness of the new blocks. BitcoinJ can play a pivotal role: as far as I am aware if we'd thought about adding handling to BitcoinJ before activation rather than after activation[1][2], the SPV issues would have been mitigated for the vast majority who rely on the library. To me, this particular issue highlights our collective failure to communicate the necessity for additional SPV handling requirements and other preparation the ecosystem should engage in during a soft fork. This is something we should definitely add to the release notes for the next soft fork and advertise widely. Certainly it MUST be well documented in the BIP65 deployment section, which it is currently not.

Lastly your objections came across very strongly (at least to my understanding) so I am curious: Peter stated Gavin is OK with adding CLTV support to XT, and assuming that is the case, will you object to merging it or similarly object to adding the necessary block handling to BitcoinJ?

[1] https://github.com/bitcoinj/bitcoinj/commit/6f03669fbd6c368961a25dfd772751d1ca2a1b5b

[2] https://github.com/bitcoinj/bitcoinj/commit/d3d11df6d71ff11cef2dc0caa8263daa641fe118