Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: LP V2 message forwarding #1971

Merged
merged 25 commits into from
Aug 18, 2024
Merged

feat: LP V2 message forwarding #1971

merged 25 commits into from
Aug 18, 2024

Conversation

wischli
Copy link
Contributor

@wischli wischli commented Aug 16, 2024

Description

Adds feature to handle forwarding inbound and outbound messages

  • Adds new pallet pallet-liquidity-pools-forwarder which implements MessageReceiver and MessageSender
  • Adds new runtime type MessageSerializer which acts as entity to serialize and deserialze all incoming and outgoing messages
  • Instantiates router mock
  • Removes extrinsic receive_message from Gateway since this was only used by the axelar router
    • If it is necessary in the future, should be added to router instead

In order to set up forwarding for some Domain::Evm(123)

  1. In pallet-liquidity-pools-forwarder storage RouterForwarding: Store the source_domain and forwarding contract address for some router_id
  2. In pallet-liquidity-pools-gateway storage Allowlist: Store the (source_domain, DomainAddress::Evm(source_somain, forwarding_contract_address)

Sending messages

Before this PR

Gateway -> RouterDispatcher -> Router

After this PR

Gateway -> Forwarder -> Serializer -> RouterDispatcher -> Router

Receiving messages

Before this PR*

Router -> Gateway

After this PR

Router -> Serializer -> Forwarder -> Gateway

TODO

  • Refactor MessageEntry and InboundEntry to use Domain instead of DomainAddress
  • Wire into runtimes and with other pallets
  • Unit tests
  • Add docs
  • Fix Domain and SerializableDomain ATs of LpMessage
  • Move (de-)serialization to runtime type MessageSerializer

Checklist:

  • I have added Rust doc comments to structs, enums, traits and functions
  • I have made corresponding changes to the documentation
  • I have performed a self-review of my code
  • I have added tests that prove my fix is effective or that my feature works

@wischli wischli force-pushed the feat/lp-msg-relay branch from e53b41a to b4c4eda Compare August 16, 2024 17:03
@wischli wischli self-assigned this Aug 16, 2024
@wischli wischli added the I8-enhancement An additional feature. label Aug 16, 2024
Copy link
Contributor

@lemunozm lemunozm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really like that the serialization and deserialization is in the same place of the LP chain.

I like the design 🚀

libs/traits/src/liquidity_pools.rs Outdated Show resolved Hide resolved
libs/traits/src/liquidity_pools.rs Outdated Show resolved Hide resolved
libs/traits/src/liquidity_pools.rs Outdated Show resolved Hide resolved
libs/traits/src/liquidity_pools.rs Outdated Show resolved Hide resolved
@wischli wischli force-pushed the feat/lp-msg-relay branch from de41c47 to 006c5a3 Compare August 16, 2024 20:41
@lemunozm
Copy link
Contributor

Just a NIT thought. Seems like an entity called forwarder is not the entity in charge of serializing / deserializing, could be weird to find those methods here? 🤔

Could make sense a super thing wrapper as a type runtime that makes that serialization?

// To send
gateway -> forwarder -> serializer -> router-dispatcher -> router 

// To receive
router -> serializer -> forwarder -> gateway

The serializer could be just a MessageSerializer implementing MessageSender and MessageReceiver type who calls msg.serialize() and deserialize().

Just a thought, and maybe being a paranoic organizer 😆

@wischli
Copy link
Contributor Author

wischli commented Aug 17, 2024

Just a NIT thought. Seems like an entity called forwarder is not the entity in charge of serializing / deserializing, could be weird to find those methods here? 🤔

Could make sense a super thing wrapper as a type runtime that makes that serialization?


// To send

gateway -> forwarder -> serializer -> router-dispatcher -> router 



// To receive

router -> serializer -> forwarder -> gateway

The serializer could be just a MessageSerializer implementing MessageSender and MessageReceiver type who calls msg.serialize() and deserialize().

Just a thought, and maybe being a paranoic organizer 😆

I agree that your proposal would be the preferred design. Will look into it later after finishing the mandatory tasks for this feature.

@lemunozm
Copy link
Contributor

Great! but please, it is not mandatory at all for Monday 🙏🏻

@lemunozm lemunozm mentioned this pull request Aug 17, 2024
9 tasks
@wischli
Copy link
Contributor Author

wischli commented Aug 17, 2024

Great! but please, it is not mandatory at all for Monday 🙏🏻

Much cleaner now: b39fa47

@cdamian
Copy link
Contributor

cdamian commented Aug 17, 2024

Only a couple of minor nits, I'm happy we got to slay this one as well, thank you @wischli!

Copy link

codecov bot commented Aug 17, 2024

Codecov Report

Attention: Patch coverage is 56.00000% with 33 lines in your changes missing coverage. Please review.

Project coverage is 49.01%. Comparing base (03262c5) to head (5cf3752).

Files Patch % Lines
pallets/liquidity-pools/src/message.rs 0.00% 21 Missing ⚠️
runtime/common/src/routing.rs 0.00% 6 Missing ⚠️
pallets/liquidity-pools-forwarder/src/lib.rs 92.30% 3 Missing ⚠️
runtime/altair/src/lib.rs 0.00% 1 Missing ⚠️
runtime/centrifuge/src/lib.rs 0.00% 1 Missing ⚠️
runtime/development/src/lib.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1971      +/-   ##
==========================================
+ Coverage   48.92%   49.01%   +0.09%     
==========================================
  Files         183      184       +1     
  Lines       13133    13200      +67     
==========================================
+ Hits         6425     6470      +45     
- Misses       6708     6730      +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@wischli wischli requested a review from cdamian August 17, 2024 19:24
Copy link
Contributor

@lemunozm lemunozm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @wischli for bringing this super PR along with documentation chains so fast!! 🚀 I really like the result.

Just few NITs, some questions, and just one think that I would simplify about mocking the Message in both pallets.

But the code looks super great! Thanks

pallets/liquidity-pools-forwarder/src/lib.rs Show resolved Hide resolved
RouterForwarding::<T>::get(&router_id).is_some(),
message.clone().unwrap_forwarded(),
) {
(true, Some((_domain, _contract, lp_message))) => Ok(lp_message),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question. Why the _domain is not used?

Could make sense the new domain_addres to be: DomainAddress::Evm(domain, contract) or similar?

Copy link
Contributor Author

@wischli wischli Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the origin of the LP message was not _domain (say Celo) but the domain_address.domain() (say Ethereum). The gateway checks in the allowlist, if it contains Ethereum's domain_address.domain() in the allowlist. I don't believe that Celo's _domain should be in the allowlist because there is no direct connection configured for Celo, for that purpose we have the forwarder pallet which acts as an allowlist gatekeeper for forwarded domains whereas the gateway acts as gatekeeper for direct connections.

Copy link
Contributor Author

@wischli wischli Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though now I remember why I initially went with the RouterProvider! We cannot guarantee that the router_id of the receive signature here is the same for which we have stored forwarding info. Therefore, we actually need to iterate over all possible router ids!

cc @cdamian

Copy link
Contributor

@lemunozm lemunozm Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm... Ok, here we have two points:

First, I think we need to read _domain or _contract somehow because if the other side forwarder implementation works like ours, it will never forward a message.

Second, if we need to use the forwarder domain, how do we get the correct router?

Crazy idea. Could it makes sense that our chain never receives a Forwarder message? I mean, we're the final endpoint, right? The forwarder wrapper should have been unwrapped before reaching centrifuge. So, as a final endpoint, we should never receive a wrapped message.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think my above crazy idea is not as crazy, and it's related to this thread above with Jeroen about not using the contract.

TLDR: I think we should not receive forwarded messages.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forwarder X -> Ethereum -> Centrifuge - message that comes through router A (pallet-axelar-router)
Forwarder X -> Ethereum -> Centrifuge - proof that comes through router B (some other router pallet)

This is the counterexample to the current implementation of the forwarder pallet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your point now. The current design wouldn't handle messages and their proofs correclty as everything is mapped to the same router id. So we are on the same page.

Revisiting our inbound setup and my comment about correct forwarding configuration: Router -> Serializer -> Forwarder -> Gateway, it appears to me that we can actually rely on the correct router_id from the receive signature called by the Router and re-remove the RouterProvider from the forwarder pallet.

E.g., when we store the forwarding info for a router id, we need to do that for every supported router. This way, the forwarder pallet has no impact on handling proof- vs non-proof messages and leaves this up to the Gateway based on the router_id which is passed all the way from the MessageReceiver impl of the corresponding router.

WDYT @cdamian @lemunozm

Completely agree.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy pasting here the flows for a 2-router-setup you broke down to make this more explicit:

Flow of a non-forwarded message

1) Ethereum -> Centrifuge Router A -> LP Forwarder::receive(router A, domain address, message) -> noop -> LP Gateway::receive(router A, domain address, message)

2) Ethereum -> Centrifuge Router B -> LP Forwarder::receive(router A, domain address, message) - >noop -> LP Gateway::receive(router A, domain address, message)

Flow of a forwarded message

1) Celo(Ethereum) -> Centrifuge Router A -> LP Forwarder::receive(router A, domain address, message) -> unwrap message to (source_domain, contract, unwrapped_message) -> LP Gateway::receive(router A, (source domain, contract), unwrapped_message)

2) Celo(Ethereum) -> Centrifuge Router B -> LP Forwarder::receive(router B, domain address, message) -> unwrap message to (source_domain, contract, unwrapped_message) -> LP Gateway::receive(router B, (source domain, contract), unwrapped_message)

Copy link
Contributor Author

@wischli wischli Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we all have found consensus on this matter, here's the corresponding code: acc6f39

I really hope this was the last raid boss to slay and that we can merge this beast.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand the second line of non-forwaded message is router B in all places right? @wischli

pallets/liquidity-pools-forwarder/src/mock.rs Outdated Show resolved Hide resolved
pallets/liquidity-pools-forwarder/src/tests.rs Outdated Show resolved Hide resolved
Comment on lines +129 to +140

fn is_forwarded(&self) -> bool {
unimplemented!("out of scope")
}

fn unwrap_forwarded(self) -> Option<(Self::Domain, H160, Self)> {
unimplemented!("out of scope")
}

fn try_wrap_forward(_: Self::Domain, _: H160, _: Self) -> Result<Self, DispatchError> {
unimplemented!("out of scope")
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can simplify a lot this by having two traits. Note that previous LpMessage methods and these new methods are or-exclusively used. This means we can have two traits:

  • LpMessage as it was, used by the gateway.
  • ForwardableMessage (or a more, similar, cool name), used by the forwarder.

I think there is no need to merge all message methods under the same trait. This will allow us to remove all unimplemented! methods in the two mocks, reducing noise

Copy link
Contributor

@lemunozm lemunozm Aug 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a more deep thought, ideally we should have the following traits (names can be changed):

  • LpMessageSerializer with serialize/deserialize.
  • LpMessageBatch with batching methods.
  • LpMessageProof with proofs methods.
  • LpMessageHash with message_hash().
  • LpMessageForwarder with forwarding methods.

Note that all the above traits will NOT be in a hierarchy. They are just behaviors that you can add or remove to a T::Message to deal with.

That way, different entities can deal with different message behaviors, i.e. splitting tomorrow the gateway in different pallets with different concerns, etc.

If agree, I think we can refactor this later. But at least I will split now The LpMessage from the LpMessageForwarder to simplify the mocks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely agree! Luckily, this change won't influence the outcome of the audit. So we can ship it after the audit feature freeze without decreasing the effectiveness.

pallets/axelar-router/src/mock.rs Outdated Show resolved Hide resolved
runtime/common/src/routing.rs Show resolved Hide resolved
@wischli
Copy link
Contributor Author

wischli commented Aug 17, 2024

Thanks @wischli for bringing this super PR along with documentation chains so fast!! 🚀 I really like the result.

Just few NITs, some questions, and just one think that I would simplify about mocking the Message in both pallets.

But the code looks super great! Thanks

Absolutely agree! Sorry I didn't initiate like that but prio was to get it done. If you think, you can still fit something in, great, but entirely optional. Whatever we change might have to be updated/added to the docs.

@@ -416,7 +416,7 @@ pub mod pallet {
return Err(Error::<T>::InvalidMessageOrigin.into());
}

Self::receive(router_id, origin_address, msg.into())
Self::receive(router_id, origin_address, T::Message::deserialize(&msg)?)
Copy link
Contributor

@lemunozm lemunozm Aug 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to move this out from the gateway. If not, the caller of this extrinsic will skip the forwarding logic.

Still not sure where. Maybe on pallet-axelar-router or pallet-liquidity-pool-forwarder (and using Message instead of Vec<u8>).

Or maybe just removing it, because it's no longer used...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch! I propose to add a MessageDeserializer AT to the Gateway config which is implemented by our runtime type MessageSerializer. This will deserialize the message and pass it to the forwarder pallet which will pass it back to the MessageReiver impl of the Gateway.

So from now:

  • process extrinsic -> Gateway impl of MessageReceiver

To

  • process in Gateway -> receive in MessagSerializer which deserializes -> receive in forward pallet which unwraps any forwarded message -> receive in Gateway which does its thing

WDYT?

Copy link
Contributor Author

@wischli wischli Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that htis requires instantiating the router_message mock pallet, which I am working on right now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm... I am not sure about that walk...

As more I think on it as more I think we can remove this. We no longer need this since Axelar uses the trait to send message to the gateway. If we want to emulate receiving message explicitly, this extrinsic should be in axelar-router

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interested in your take on this @cdamian

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @lemunozm - the flow that you are describing above would mean that the forwarder can be called by the gateway, in an inbound context, which shouldn't happen? Also, this extrinsic can maybe ensure that no forwarded message is submitted since we have it for special cases?

Copy link
Contributor Author

@wischli wischli Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See here for code changes if we added the MessageDeserializer to the Gateway: a253a6f

However, I have reverted that change and applied @lemunozm's proposal of removing the receive_message extrinsic which apparently was only used by the Axelar precompile. I have refactored the existing Gateway UTs for checking the MessageReceiver implementation which was not tested: cbb75b4

pallets/liquidity-pools-forwarder/src/lib.rs Show resolved Hide resolved
RouterForwarding::<T>::get(&router_id).is_some(),
message.clone().unwrap_forwarded(),
) {
(true, Some((_domain, _contract, lp_message))) => Ok(lp_message),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that way, we're breaking the proof checks, because all messages (even proofs) are redirected to the first found router, right?

cc @cdamian

Given the flow that William described above, I think so.

@@ -416,7 +416,7 @@ pub mod pallet {
return Err(Error::<T>::InvalidMessageOrigin.into());
}

Self::receive(router_id, origin_address, msg.into())
Self::receive(router_id, origin_address, T::Message::deserialize(&msg)?)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @lemunozm - the flow that you are describing above would mean that the forwarder can be called by the gateway, in an inbound context, which shouldn't happen? Also, this extrinsic can maybe ensure that no forwarded message is submitted since we have it for special cases?

RouterForwarding::<T>::get(&router_id).is_some(),
message.clone().unwrap_forwarded(),
) {
(true, Some((_domain, _contract, lp_message))) => Ok(lp_message),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although, if we do have multiple routers set up, wouldn't it be possible to have smth like:

Forwarder X -> Ethereum -> Centrifuge - message that comes through router A (pallet-axelar-router)
Forwarder X -> Ethereum -> Centrifuge - proof that comes through router B (some other router pallet)

OR

Forwarder X -> Ethereum -> Centrifuge - message that comes through router A (pallet-axelar-router)
Forwarder Y -> Ethereum -> Centrifuge - proof that comes through router B (some other router pallet)

Do any of these make sense?

@wischli wischli force-pushed the feat/lp-msg-relay branch from e409fd5 to acc6f39 Compare August 18, 2024 12:09
@wischli wischli requested review from lemunozm and cdamian August 18, 2024 12:11
.expect("Domain not Centrifuge; qed"),
info.contract,
);
Ok((lp_message, domain_address))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit - this one is actually the forwarding_domain_address. Feel free to disregard tho, can live with this.

Copy link
Contributor Author

@wischli wischli Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No both are different:

  • forwarding_domain_address is some address of the forwarding domain Ethereum, e.g. DomainAddress::Evm(1, x)
  • domain_address here is the the domain address of the source domain and the forwarding contract, e.g. DomainAddress::Evm(42220, info.contract)

When setting up forwarding for some domain, we need to store two twings:

  1. In pallet-liquidity-pools-forwarder storage RouterForwarding: Store the source_domain and forwarding contract address for some router_id
  2. In pallet-liquidity-pools-gateway storage Allowlist: Store the (source_domain, DomainAddress::Evm(source_somain, forwarding_contract_address)

Copy link
Contributor

@cdamian cdamian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work @wischli!

Copy link
Contributor

@lemunozm lemunozm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Some comments that does not block the merge! Let's add this 💯

Comment on lines +250 to +252
info.source_domain
.get_evm_chain_id()
.expect("Domain not Centrifuge; qed"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT. we can have more domains in the future and break this silently. Just a tech-debt here to fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get the point but actually think we wont have an additional domain variant for the time being because that requires LP compatibility with that domain and that is not on the radar outside EVM. For XCM, the domain would actually be Centrifuge.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure! Not for the time being. Could only be an issue in the future

pallets/liquidity-pools/src/message.rs Show resolved Hide resolved
@@ -206,7 +239,7 @@ impl BatchMessages {
MaxEncodedLen,
Default,
)]
pub enum Message<BatchContent = BatchMessages> {
pub enum Message<BatchContent = BatchMessages, ForwardContent = NonForwardMessage> {
Copy link
Contributor

@lemunozm lemunozm Aug 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Am I curious about where a message with ForwardContent != NonForwardMessage is created? I think you always create instances with NonForwardMessage, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I suppose this can be reduced then? Would like to merge this PR as is assuming CI is green. Will clean up in a follow up PR. Thanks for spotting this!

@hieronx hieronx merged commit a1109d8 into main Aug 18, 2024
8 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
I8-enhancement An additional feature.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants