At least in german law there is a difference between ‘Eigentum' means ownership and ‘Besitz’ means ability to deal with it.
Being able to deal with an asset does not make you the owner.
If you have the private keys for your users' bitcoins, then you are every bit as much the owner of those bitcoins as your users are. There is no custodial relationship, as you have both the ability and the right to spend those bitcoins. Possession of a private key is equivalent to ownership of the bitcoins controlled by that private key.