Hi,

In the proposal below, any annex that begins with `0x00` is defined as
free-form. This isn't the most efficient format though because there is
always one byte lost on signalling. In a future where unstructured annex
data turns out to be the predominant use case, this may be relevant. Also
for very short annexes, the lost byte may weigh relatively heavy.

Without sacrificing future extensions to structured annex data, one could
also store the annex data as is except for the case where the data starts
with `0x21` (or any other 'uncommon' byte). If the data starts with `0x21`,
this byte needs to be repeated first and then followed by the remainder of
the data.

Examples:

Data: [01 02 03]
Encoding [01 02 03]

Data: [21 22 23]
Encoding: [21 21 22 23]

The prefixes [21 (not 21)] are available for future upgrades.

Joost

On Fri, Jun 2, 2023 at 5:00 PM Joost Jager <joost.jager@gmail.com> wrote:

> Hi,
>
> As it stands, the taproot annex is consensus valid but non-standard. The
> conversations around standardization seem to be leaning towards the
> adoption of a flexible Type-Length-Value (TLV) format [1]. There's no doubt
> that this approach has considerable potential. However, settling on an
> exact format may require a significant amount of time.
>
> In the interim, the benefits of making the annex available in a
> non-structured form are both evident and immediate. By allowing developers
> to utilize the taproot annex without delay, we can take advantage of its
> features today, without the need to wait for the finalization of a more
> lengthy standardization process.
>
> With this in view, I am proposing that we define any annex that begins
> with '0' as free-form, without any additional constraints. This strategy
> offers several distinct benefits:
>
> Immediate utilization: This opens the door for developers to make use of
> the taproot annex for a variety of applications straight away, thus
> eliminating the need to wait for the implementation of TLV or any other
> structured format.
>
> Future flexibility: Assigning '0'-beginning annexes as free-form keeps our
> options open for future developments and structure improvements. As we
> forge ahead in determining the best way to standardize the annex, this
> strategy ensures we do not limit ourselves by setting its structure in
> stone prematurely.
>
> Chainspace efficiency: Non-structured data may require fewer bytes
> compared to a probable TLV format, which would necessitate the encoding of
> length even when there's only a single field.
>
> In conclusion, adopting this approach will immediately broaden the
> utilization scope of the taproot annex while preserving the possibility of
> transitioning to a more structured format in the future. I believe this is
> a pragmatic and efficient route, one that can yield substantial benefits in
> both the short and long term.
>
> Joost
>
> [1] https://github.com/bitcoin/bips/pull/1381
>