​The least reasonable thing I could expect is some claimed former holder of
some ordianls turning up to challenge me that it was their stolen Bitcoin
was some of what I received.


I think it's unlikely that this would come to pass. A previous owner of an
ordinal wouldn't have any particular reason to expect that they should own
it after they transfer it. Similar to how noting a dollar bill's serial
number doesn't give you a claim to it after you spend it. From the BIP:

​Since any ordinal can be sent to any address at any time, ordinals that
are transferred, even those with some public history, should be considered
to be fungible with other satoshis with no such history.
<https://github.com/casey/ord/blob/master/bip.mediawiki#backward-compatibility>