Unfortunately, when it comes to cross-transaction computations, it is infeasible to ban non-terminating computation.

The nature of recursive covenants is that the program "writes" the *source code* next step of the computation to the scriptPubKey to one of the outputs of its transaction. Technically speaking it verifies that the scriptPubKey is a commitment to the source code of the next step of the program, but morally that is the same as writing the source code.  Then the next step of the computation is invoked by someone "evaluating* that next step's source code by creating a valid transaction that spends the generated output.

The point is this ability to create new source code and then evaluate it leads to the ability to write universal (i.e non-terminating) computations.  The only way to prevent it is to ban source code manipulation, but since Bitcoin Script source code is just a string of bytes, it would mean banning the manipulation of strings of bytes.  But the entire Bitcoin Script language works by manipulating strings of bytes within a stack machine.  Indeed the most trivial of non-terminating programs can be implemented by extracting the current input's scriptPubKey from the sighash and "writing" the identical scriptPubKey to one of its outputs.  That example hardly takes any manipulation at all to implement.