While Segwit's change from 1 mb size limit to 4 mb weight limit seems to be controversial among some users (I find that very often it is because they have been confused about what segwit does or even outright lied about it) I don't think it's very interesting to discuss further size increases.
I find more interesting to talk to the users and see how they think Segwit harms them, maybe we missed something in segwit that needs to be removed for segwit to become uncontroversial, or maybe it is just disinformation.
On the other hand, we may want to have our first uncontroversial hardfork asap, independently of block size. For example, we could do something as simple as fixing the timewarp attack as bip99 proposes. I cannot think of a hf that is easier to implement or has less potential for controversy than that.