I use bitcoin heavily (not from time to time) but I don't mine - can I
vote? The way I see it I cannot, and I am not saying it is a bad thing,
but I missed the argument explaining why users don't matter and only
miners do.

It is a reasonable question. Let me try and explain why it's done this way.

In theory, you do have a vote. If a majority of miners were to club together and decide to change the protocol against the wishes of the wider user base, then we'd get a fight between the hashpower majority and the so-called economic majority. And because bitcoins only have value because you can buy things with them, in theory, the wishes of the economic majority should always win.

In practice, the code we have today doesn't let us measure what the economic majority wants. It's not even really clear how that term is defined. Intuitively we can understand that people who are trading real goods and services for bitcoin have the final say, because they can always just refuse to accept BTC. But defining it precisely enough to put in an algorithm is tricky.

Then you have the question of how to express the vote? For miners, it's easy: they express their vote by switching to a different full node implementation that gives them different blocks. 

For users, it'd mean switching to a different wallet. If their wallet is fully validating and the decision is implemented via hard fork, this is sufficient. If the wallet is not fully validating and cannot detect the fork point by itself, then it'd need help in the form of checkpointing. Some months ago I pointed out this possibility and a whole bunch of people freaked out - then bitcoin.org decided to start censoring any wallet that said it'd ignore what miners wanted.

So if you want a user vote, that's an issue that'd have to be tackled: the people who admin the main communication channels Bitcoin users have vowed to censor any program that doesn't slavishly follow 51%+ hash power. That attempt to control the conversation is certainly not libertarian or democratic in nature, but there you go.

We can also imagine voting via proof-of-stake. This might be useful as a form of opinion poll, but wallet developers would have to actually add support for such a protocol to their apps, and then we're back to the same issue as with mining pools. Plus, of course, static wealth does not equal economic importance. 

Luckily the wallet market is a decentralisation success story. There are wallets everywhere these days. Man+dog make their own wallet, it seems. So it's not silly to think a coin voting protocol could one day happen.