Wanted to give an update from the Penumbra Labs side.
Noble Fix Status
We’ve been in communication with the Noble team, but there is unfortunately still no concrete timeline on deploying a fix. As a result, I think it’s worth considering changes on the Penumbra side that would allow proactive interoperability. One possibility would be to add a third address format: Pre-UIP: Truncated Addresses , which would maximize compatibility with other chains. This format is kind of a hack and has numerous UX downsides, but could act as an escape hatch (an address format with UX warts may still be better UX than stuck funds). This would need to be deployed as part of a Penumbra chain upgrade, and this could be discussed in this week’s Penumbra community call.
Weekend Osmosis Relaying Issues
Over the weekend, users reported that relaying to Osmosis was interrupted, causing transfers to fail. This was concerning, as the Noble issue is specific to USDC (other Noble assets like USDY, and the IBC channel itself, are unaffected). Discussions with relayer operators suggest that the issue was ultimately caused by an Osmosis RPC that returned a stale/incorrect/corrupted sequence number for the relayer account. This caused the relayer to be unable to make transactions on Osmosis, although it could make transactions on Penumbra. As a result, inbound transfers worked, but outbound transfers failed. Failed transfers would time out and be refunded on the Penumbra side, so users maintained both control over and access to their funds at all times. However, as described below, a UI issue in Prax wallet did not appropriately surface this information. Relayer operators restarted relayers and all packets are believed to be cleared.
Penumbra <> Osmosis relaying should be fully operational at this time.
Users can check the Range dashboard to view transfers in and out of Penumbra, though it does not have full support for indexing Penumbra itself.
To minimize the chance of this reoccuring, some next steps are:
- Upstream Penumbra-specific relayer code into Hermes, now that Penumbra’s code has stabilized post-mainnet;
- Improve monitoring and health-checking of IBC relaying;
- Engage the community to find near-term solutions to improve relayer diversity and resilience;
- Longer-term, work to make self-relaying practical rather than just technically possible, to eliminate the dependence on relayers entirely (this could ideally be done as part of the IBC Eureka design).
Prax Improvements for Failed IBC Transfers
While investigating relaying issues, the Prax team noticed a UI issue that creates a very bad user experience when a user’s IBC transfer fails. We believe this has been the cause of significant confusion (including the confusion reported above).
In short, although Prax correctly detected refunds and made them available to users, Prax did not correctly display the corresponding transactions.
This meant that while a user’s balance is silently updated with their refunded transfer, there is no indication that the transfer’s failure was correctly handled in the form of a refund, only that an outbound transfer has been initiated. This leads users to (justifiably) wonder what happened to their transfer and conclude that funds are missing (even though they are not).
That’s a terrible user experience, and a fix for this issue is in development and will ship in the next Prax release. I believe this has been a significant contributor and “negativity multiplier” for the bad experience users are currently experiencing, and I’m glad to have identified it so that it can be fixed.
.