Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

De-dupe fills and trades from WebSocket. #413

Merged
merged 2 commits into from
Jun 5, 2024
Merged

Conversation

prashanDYDX
Copy link
Contributor

We're seeing quite a few crashes from trying to render fills and trades with duplicate IDs. After discussing with James, it sounds like it is to be expected that duplicate messages are received over the websocket.

The main issue here is that for Processors that return a List of items, we are just appending items blindly. To limit the scope of this PR, we won't change the API signature from List to Map, but rather will use a set to ensure that only a single item with the same ID is returned.

Note: mutableSetOf() is backed by a LinkedHashSet, so ordering is preserved.

This PR addresses the two main offenders: Fills and Trades. There are other processors that have the same potential pitfall, but I will tackle these in a follow-up.

@ruixhuang
Copy link
Contributor

Note: mutableSetOf() is backed by a LinkedHashSet, so ordering is preserved.

Not sure if it's safe to since the implementation can be differ on each platform, and future implementation might not maintain ordering. Might as well just keep the ids in a set, and appending the old to new if id is not in the set.

@prashanDYDX
Copy link
Contributor Author

prashanDYDX commented Jun 4, 2024

Note: mutableSetOf() is backed by a LinkedHashSet, so ordering is preserved.

Not sure if it's safe to since the implementation can be differ on each platform, and future implementation might not maintain ordering. Might as well just keep the ids in a set, and appending the old to new if id is not in the set.

I think it's pretty safe to assume that LinkedHashSet will continue to provide ordering. LinkedHashSet itself is in the Kotlin stdlib, this isn't just the JVM implementation. And changing the default of mutableSetOf() to a different set implementation seems like a pretty drastic breaking change.

But managing the IDs on their own does let us avoid the map at the end, so I'll do that.

@prashanDYDX prashanDYDX force-pushed the prashan/dedupe branch 2 times, most recently from d53e5ad to 57edf71 Compare June 4, 2024 18:46
@prashanDYDX prashanDYDX force-pushed the prashan/dedupe branch 4 times, most recently from 703f2c8 to 9af73c3 Compare June 4, 2024 22:07
@prashanDYDX prashanDYDX merged commit ef33a43 into main Jun 5, 2024
4 checks passed
@prashanDYDX prashanDYDX deleted the prashan/dedupe branch June 5, 2024 00:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants