Pre-UIP: Transparent Addresses

UIPs - Penumbra Improvement Proposals
1

As described in USDC Transfers Temporarily Affected by Noble Chain Upgrade , a recent Noble upgrade broke USDC withdrawals for ultimately pretty silly reasons: a change to a Bech32 library unintentionally added a new length limit, breaking compatibility with Penumbra addresses.

The correct fix for that issue is for a bugfix to be deployed to the Noble chain. However, this may take time (as of now, there is no concrete timeline for deployment of a fix). And, this isn’t the first compatibility issue that’s arisen with Penumbra addresses, and it may not be the last.

Rather than waiting for a third party fix, it may be better to perform “proactive interoperability”, and define a maximally compatible address format: truncated addresses. These addresses would be 32-byte Bech32 addresses, like this:

ptrunc16t45lct6x4w6an6hvv7zp9p49qx0teg78qyukcgfk76ch87leggq0wlmdc

Truncated addresses have compatibility both with external systems and with Penumbra. A truncated address can be directly decoded to an ordinary Penumbra address. The address above decodes to

penumbra169wv6gf7r2fp0p0z6dsdlr7ntjetp2v2s4q77l434vw2qsdg3lqzlpyy6krgueygk0z3m2c77galpszc6q053nd4tjyu03d6hawlw4992ahdasgazqy54k39g2ezu536etjzkr

so that the truncation is really just an encoding difference, and no changes are needed to any other part of Penumbra.

Unfortunately, truncated addresses achieve this compatibility at a cost:

  • There is only one truncated address per Penumbra wallet
  • That address points at a random, fixed sub-account
  • Reusing a truncated address creates linkability (e.g., using it for IBC deposits or withdrawals causes all deposits and withdrawals to be linked, as there is only one truncated address per wallet)

These make the UX much, much worse than normal Penumbra addresses in hard-to-explain ways, so the use of truncated addresses should always be discouraged. However, it may be useful to add the capability anyways as a kind of compatibility escape hatch: a user might prefer to at least be able to do withdrawals or deposits in the event of a current or future compatibility issue, rather than having stuck funds.

How Truncated Addresses Work

A Penumbra address has three components:

  • The diversifier, d;
  • The transmission key, pk_d;
  • The clue key, ck_d;

The controller of the address creates the diversifier (random bytes) by choosing a 16-byte address index, which encodes structured information about the address metadata. By convention, the address index is LE32(account index) || 12-byte randomizer. The address index is then symmetrically encrypted with the wallet’s viewing key. The output of that encryption is used as the diversifier. The diversifier is used to derive the transmission key, which a counterparty uses to encrypt messages to the user. The clue key allows fuzzy message detection, for future scaling, but this is not used in the client stack at this time. This design has two really useful properties:

  1. The wallet’s viewing key can scan messages encrypted to all possible transmission keys simultaneously (i.e., one scanning pass can detect messages sent to any address)
  2. The viewing key can “decode” the encrypted metadata in the diversifier, so it can statelessly determine which account the address corresponds to.

These components are 16 + 32 + 32 = 80 bytes, and are jumbled as part of the encoding (so it’s always safe to do the common user behavior of checking only the first N characters).

Truncated addresses truncate the address data to only 32 bytes, by:

  • Requiring the diversifier is [0; 16] and skipping its encoding (it is filled in with zero bytes on the decoding side);
  • Encoding only the transmission key;
  • Discarding the clue key, and filling it in with a dummy value (the identity element) on decoding.

Pros

  • Truncated addresses are as compatible with Cosmos or other systems as possible: 32 bytes of Bech32 encoded data
  • Because truncated addresses can be decoded to an ordinary address, the extra address format does not need to “propagate” through the rest of the client stack, and would require no changes to the core cryptography or protocol.
  • Penumbra could add truncated addresses without waiting for any other chain, and leave them as an ugly hack that allows compatibility workarounds for future issues.

Cons

  • There is only one truncated address per wallet, which breaks privacy when making IBC transfers.
    • Currently, IBC transfers can use unique addresses for each transfer automatically, so users never have to think about address reuse. Truncated addresses would force users to think about this issue, and they would probably get it wrong.
  • The truncated address points at a random account, which is weird and unexpected.
  • Truncated addresses are not compatible with FMD
    • This is probably not a big deal, since FMD is not currently used, and the model Penumbra is evolving towards with Spend Backreferences would mean that FMD could be used to detect any one transaction made by a user
  • Adding a third address format, even if only marginally used, adds to the already substantial conceptual UX burden

In spite of those downsides, I still think this could be worthwhile to implement, if only as an escape hatch. The right solution to compatibility issues with other chains is to have full support for normal Penumbra addresses. But in the meantime, having an ugly hack that allows users to transact is probably preferable, and having it available for the future seems desirable.

Proof of Concept

A PoC implementation can be found at [WIP]: Truncated address formats by hdevalence · Pull Request #4950 · penumbra-zone/penumbra · GitHub

2

First, I would suggest that for ease of use, we modify diversifier decryption in this case to always set the account index to 0, ignoring whatever encrypting the null ciphertext gives us. This avoids having the truncated address be pointing at some random account.

Having only one truncated address per account is a big downside, and is really hard to manage, and savvy users might be forced into the very awkward pattern of using multiple wallets, which is a UX that’s incredibly annoying.

However, if we allow for variable length addresses, I think we can elegantly allow for multiple addresses up to whatever length limitation we might run into, with the above truncation address as a fallback case. For example, with the accidental noble length limit, we can afford an extra 12 bytes (at least).

My idea is that we stick to the suggestion I had above of always using account index 0 for truncated addresses, ignoring whatever the ciphertext decrypts to, and then we also allow “extra” data beyond the minimal 32 bytes of address data.
The mapping from extra data to a ciphertext can be done as follows (python example):

4 * [0x00] + extra[:12] + max(0, len(extra) - 12) * [0x00]

In other words, put the extra bytes as the diversifier, then pad with zeros. If there are no extra bytes, we get the null ciphertext, but otherwise, we get as many bytes as allowed.

I think the additional complexity of having this additional slider is worth the privacy gain ; furthermore, if wallets paper over temporary breakages with certain chains by automatically selecting the correct compatibility address for a given transfer destination, then they can also select a maximally randomized address, or even use a counter system to prevent re-use, with a warning if the user does happen to attempt to re-use a given address: in this case, there’s no additional complexity to the user, and only privacy gains

1 Like
3

From a cryptographic point of view, I think this works out. From a product point of view, however, I don’t think it’s a good idea — it adds a lot of conceptual surface area.

Even “truncated” is a complex word.

Instead, I think we should align with the conceptual terminology used by other systems and call them “transparent” addresses: tpenumbra1…

While transparent addresses for Penumbra actually have much stronger privacy guarantees than those on Namada or Zcash (eg there are no transparent balances), choosing this terminology makes it clear to users that they’re not getting full privacy benefits. In other words, it’s kind of underselling it, but the error is in the right direction — it’s more private than a user might expect.

This also allows transparent addresses to remain marginalized from a product point of view. They are useful for this compatibility purpose but not otherwise.

4

This is a great idea though and definitely we should do it. Only thing is we should think through the implications of having decryption no longer be bijective. I think this is OK since we’d never generate a t-address from an address index.

5

If we have to do this, calling these addresses “transparent” to make the reduced privacy guarantees clear to the user is wise.

It’s a nice idea if we were to do this to always set the address index to zero instead of allowing whatever random account/randomizer combination we get from decryption of the null ciphertext. If we modify the diversifier encryption/decryption (i.e. the methods on DiversifierKey) to special case null ciphertexts in general we will enable the possibility of generating a null ciphertext that decrypts to the “wrong” ciphertext, even though this is an astronomically small (i.e. 1/2^128) probability. We could have a separate diversifier decryption path in the code specifically for Penumbra addresses with identity clue keys and null diversifiers, but that is pretty ugly.

6

I’m reminded of a code comment I read somewhere in one of Adam Langley’s crypto implementations to the effect of “we could check this, but that condition happens with probability 1/2^128, so we’re more likely to fault the CPU instructions we’d use to check than to hit the condition”. I think having an override in decryption only would be fine.

7

OK, another question is, what other changes should exist at the protocol level?

The one that comes to mind is ensuring that the Ics20Withdrawal action has a use_transparent_address boolean. The action is given an Address and the chain produces the packet, so the chain needs to be told to use the transparent address encoding (just like with compat addresses).

If transparent addresses exist, should compat addresses be deprecated? I think the answer is yes, though probably only in the sense of:

  • deprecate the use_compat_address in Ics20Withdrawal
  • deprecate anything that uses them

Since they exist in on-chain data I guess they’ll live on forever. Remember, protocols don’t have mistakes, just happy little accidents.

8

started writing up this proposed change here: UIP: Transparent addresses by redshiftzero · Pull Request #17 · penumbra-zone/UIPs · GitHub