Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Raschel sync #2555

Merged
merged 13 commits into from
Dec 20, 2024
Merged

Raschel sync #2555

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e6a0195
add army sync by position
aymericdelab Dec 17, 2024
3ff0406
add stamina
aymericdelab Dec 17, 2024
4fddc1a
limit sync because of useeffects
aymericdelab Dec 17, 2024
080f8ce
resource arrivals sync
aymericdelab Dec 17, 2024
40781d9
Merge branch 'next' into raschel-sync
aymericdelab Dec 19, 2024
c12696d
improve resource arrival component
aymericdelab Dec 19, 2024
f6b5866
add resource arrivals sync queries
aymericdelab Dec 19, 2024
27aeadd
add multiple loading states
aymericdelab Dec 19, 2024
dc94fac
fix
aymericdelab Dec 19, 2024
642dee4
add loading states and optimize queries
aymericdelab Dec 20, 2024
4548342
Merge branch 'next' into raschel-sync
aymericdelab Dec 20, 2024
de96dbf
revert event change
aymericdelab Dec 20, 2024
f44e971
merge
ponderingdemocritus Dec 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions client/src/App.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import "./index.css";
import { Toaster } from "./ui/components/Toaster";
import { TransactionNotification } from "./ui/components/TxEmit";
import { WorldLoading } from "./ui/components/WorldLoading";
import { World } from "./ui/layouts/World";

function App({ backgroundImage }: { backgroundImage: string }) {
Expand All @@ -9,6 +10,7 @@ function App({ backgroundImage }: { backgroundImage: string }) {
<Toaster />
<TransactionNotification />
<World backgroundImage={backgroundImage} />
<WorldLoading />
</>
);
}
Expand Down
53 changes: 44 additions & 9 deletions client/src/dojo/debouncedQueries.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { Component, Metadata, Schema } from "@dojoengine/recs";
import { ToriiClient } from "@dojoengine/torii-client";
import debounce from "lodash/debounce";
import {
addArrivalsSubscription,
addMarketSubscription,
addToSubscription,
addToSubscriptionOneKeyModelbyRealmEntityId,
Expand All @@ -16,8 +17,11 @@ class RequestQueue {
private batchSize = 3; // Number of concurrent requests
private batchDelayMs = 100; // Delay between batches

async add(request: () => Promise<void>) {
this.queue.push(request);
async add(request: () => Promise<void>, onComplete?: () => void) {
this.queue.push(async () => {
await request();
onComplete?.(); // Call onComplete after the request is processed
});
Comment on lines +20 to +24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Consider error handling for onComplete callback

The onComplete callback should be protected from errors to prevent queue processing interruption.

   async add(request: () => Promise<void>, onComplete?: () => void) {
     this.queue.push(async () => {
-      await request();
-      onComplete?.();
+      try {
+        await request();
+        onComplete?.();
+      } catch (error) {
+        console.error('Request failed:', error);
+        throw error; // Re-throw to be caught by processQueue
+      }
     });
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
async add(request: () => Promise<void>, onComplete?: () => void) {
this.queue.push(async () => {
await request();
onComplete?.(); // Call onComplete after the request is processed
});
async add(request: () => Promise<void>, onComplete?: () => void) {
this.queue.push(async () => {
try {
await request();
onComplete?.();
} catch (error) {
console.error('Request failed:', error);
throw error; // Re-throw to be caught by processQueue
}
});

if (!this.processing) {
this.processing = true;
this.processQueue();
Expand Down Expand Up @@ -54,8 +58,13 @@ const marketQueue = new RequestQueue();

// Debounced functions that add to queues
export const debouncedSyncPosition = debounce(
async <S extends Schema>(client: ToriiClient, components: Component<S, Metadata, undefined>[], entityID: string) => {
await positionQueue.add(() => syncPosition(client, components, entityID));
async <S extends Schema>(
client: ToriiClient,
components: Component<S, Metadata, undefined>[],
entityID: string,
onComplete?: () => void,
) => {
await positionQueue.add(() => syncPosition(client, components, entityID), onComplete);
},
100,
{ leading: true }, // Add leading: true to execute immediately on first call
Expand All @@ -66,8 +75,12 @@ export const debouncedAddToSubscriptionTwoKey = debounce(
client: ToriiClient,
components: Component<S, Metadata, undefined>[],
entityID: string[],
onComplete?: () => void,
) => {
await subscriptionQueue.add(() => addToSubscriptionTwoKeyModelbyRealmEntityId(client, components, entityID));
await subscriptionQueue.add(
() => addToSubscriptionTwoKeyModelbyRealmEntityId(client, components, entityID),
onComplete,
);
},
250,
{ leading: true },
Expand All @@ -78,8 +91,25 @@ export const debouncedAddToSubscriptionOneKey = debounce(
client: ToriiClient,
components: Component<S, Metadata, undefined>[],
entityID: string[],
onComplete?: () => void,
) => {
await subscriptionQueue.add(() => addToSubscriptionOneKeyModelbyRealmEntityId(client, components, entityID));
await subscriptionQueue.add(
() => addToSubscriptionOneKeyModelbyRealmEntityId(client, components, entityID),
onComplete,
);
},
250,
{ leading: true },
);

export const debounceAddResourceArrivals = debounce(
async <S extends Schema>(
client: ToriiClient,
components: Component<S, Metadata, undefined>[],
entityID: number[],
onComplete?: () => void,
) => {
await subscriptionQueue.add(() => addArrivalsSubscription(client, components, entityID), onComplete);
},
250,
{ leading: true },
Expand All @@ -91,16 +121,21 @@ export const debouncedAddToSubscription = debounce(
components: Component<S, Metadata, undefined>[],
entityID: string[],
position?: { x: number; y: number }[],
onComplete?: () => void,
) => {
await subscriptionQueue.add(() => addToSubscription(client, components, entityID, position));
await subscriptionQueue.add(() => addToSubscription(client, components, entityID, position), onComplete);
},
250,
{ leading: true },
);

export const debouncedAddMarketSubscription = debounce(
async <S extends Schema>(client: ToriiClient, components: Component<S, Metadata, undefined>[]) => {
await marketQueue.add(() => addMarketSubscription(client, components));
async <S extends Schema>(
client: ToriiClient,
components: Component<S, Metadata, undefined>[],
onComplete?: () => void,
) => {
await marketQueue.add(() => addMarketSubscription(client, components), onComplete);
},
500,
{ leading: true },
Expand Down
78 changes: 74 additions & 4 deletions client/src/dojo/queries.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,11 @@ export const addMarketSubscription = async <S extends Schema>(
await getEntities(
client,
{
Keys: {
keys: [undefined, undefined],
pattern_matching: "FixedLen",
models: ["s0_eternum-DetachedResource"],
Member: {
model: "s0_eternum-DetachedResource",
member: "resource_amount",
operator: "Gt",
value: { Primitive: { U128: "0" } },
},
},
components,
Expand All @@ -151,3 +152,72 @@ export const addMarketSubscription = async <S extends Schema>(
const end = performance.now();
console.log("MarketEnd", end - start);
};

export const addArrivalsSubscription = async <S extends Schema>(
client: ToriiClient,
components: Component<S, Metadata, undefined>[],
entityIds: number[],
) => {
const start = performance.now();
console.log("ArrivalsEnd: starting resource arrivals");
await getEntities(
client,
// todo: waiting on ghlim to check issue with this query
// {
// Composite: {
// operator: "And",
// clauses: [
// {
// Composite: {
// operator: "Or",
// clauses: entityIds.map((id) => ({
// Member: {
// model: "s0_eternum-EntityOwner",
// member: "entity_owner_id",
// operator: "Eq",
// value: { Primitive: { U32: id } },
// },
// })),
// },
// },
// {
// Member: {
// model: "s0_eternum-OwnedResourcesTracker",
// member: "resource_types",
// operator: "Neq",
// value: { Primitive: { U256: "0" } },
// },
// },
// ],
// },
// },
Comment on lines +165 to +193
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Based on the gathered evidence, I can now provide a final response:

The commented-out query with OwnedResourcesTracker filtering is necessary

The codebase analysis reveals that OwnedResourcesTracker is a crucial component used to track resource ownership across the system:

  • It's used in multiple places to filter entities that have resources (NotValue(OwnedResourcesTracker, { resource_types: 0n })), indicating it's an important filtering condition
  • The component maintains a bitmap of owned resource types and is actively used in resource transfer operations (as seen in contracts/src/models/resources.cairo)
  • The simpler query without the OwnedResourcesTracker check could include entities that don't actually have any resources, leading to incorrect resource tracking

Action items:

  • Restore the commented-out query to ensure proper resource tracking
  • Remove the TODO comment and implement the complete query with OwnedResourcesTracker filtering
  • If there's a specific issue with the query, it should be documented in a proper issue tracker rather than a TODO comment
🔗 Analysis chain

Verify if the simplified query is sufficient

The commented-out code shows a more complex query that includes additional filtering for OwnedResourcesTracker. While waiting for the issue to be resolved:

  1. Document why the simpler query was chosen as temporary solution
  2. Create a tracking issue for implementing the complete query once the blocker is resolved

Let's verify if the simpler query might miss important cases:

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if there are cases where OwnedResourcesTracker is crucial
rg -A 3 "OwnedResourcesTracker" --type typescript

Length of output: 84

Script:

#!/bin/bash
# Let's try without file type restriction since the ripgrep version might not support typescript
rg -A 3 "OwnedResourcesTracker"

Length of output: 16197

Script:

#!/bin/bash
# Search for any references to resources or tracking in the queries file
rg -A 5 "resource" client/src/dojo/queries.ts

Length of output: 549

Script:

#!/bin/bash
# Look for the actual query implementation
ast-grep --pattern 'addArrivalsSubscription($$$)'

Length of output: 197

Script:

#!/bin/bash
# Let's look at the actual implementation of addArrivalsSubscription
rg -B 5 -A 10 "export const addArrivalsSubscription" client/src/dojo/queries.ts

Length of output: 529

Script:

#!/bin/bash
# Check how OwnedResourcesTracker is used in the codebase
ast-grep --pattern 'NotValue(OwnedResourcesTracker, { resource_types: $_})'

Length of output: 303

{
Composite: {
operator: "Or",
clauses: entityIds.map((id) => ({
Member: {
model: "s0_eternum-EntityOwner",
member: "entity_owner_id",
operator: "Eq",
value: { Primitive: { U32: id } },
},
})),
},
},

components,
[],
[
"s0_eternum-Army",
"s0_eternum-Position",
"s0_eternum-EntityOwner",
"s0_eternum-Weight",
"s0_eternum-OwnedResourcesTracker",
"s0_eternum-ArrivalTime",
],
1000,
false,
);
const end = performance.now();
console.log("ArrivalsEnd", end - start);
};
66 changes: 40 additions & 26 deletions client/src/dojo/setup.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import { AppStore } from "@/hooks/store/useUIStore";
import { LoadingStateKey } from "@/hooks/store/useWorldLoading";
import {
BUILDING_CATEGORY_POPULATION_CONFIG_ID,
HYPERSTRUCTURE_CONFIG_ID,
Expand Down Expand Up @@ -71,10 +73,11 @@ export const syncEntitiesDebounced = async <S extends Schema>(
};
};

export async function setup({ ...config }: DojoConfig) {
export async function setup(config: DojoConfig & { state: AppStore }) {
const network = await setupNetwork(config);
const components = createClientComponents(network);
const systemCalls = createSystemCalls(network);
const setLoading = config.state.setLoading;

const configClauses: Clause[] = [
{
Expand Down Expand Up @@ -114,13 +117,37 @@ export async function setup({ ...config }: DojoConfig) {
},
];

await getEntities(
network.toriiClient,
{ Composite: { operator: "Or", clauses: configClauses } },
network.contractComponents as any,
);
setLoading(LoadingStateKey.Config, true);
try {
await Promise.all([
getEntities(
network.toriiClient,
{ Composite: { operator: "Or", clauses: configClauses } },
network.contractComponents as any,
),
getEntities(
network.toriiClient,
{
Keys: {
keys: [undefined, undefined],
pattern_matching: "FixedLen",
models: ["s0_eternum-CapacityConfigCategory", "s0_eternum-ResourceCost"],
},
},
network.contractComponents as any,
[],
[],
40_000,
false,
),
]);
} finally {
setLoading(LoadingStateKey.Config, false);
}

// fetch all existing entities from torii

setLoading(LoadingStateKey.SingleKey, true);
await getEntities(
network.toriiClient,
{
Expand All @@ -137,10 +164,8 @@ export async function setup({ ...config }: DojoConfig) {
"s0_eternum-BankConfig",
"s0_eternum-Bank",
"s0_eternum-Trade",
"s0_eternum-Army",
"s0_eternum-Structure",
"s0_eternum-Battle",
"s0_eternum-EntityOwner",
],
},
},
Expand All @@ -149,28 +174,15 @@ export async function setup({ ...config }: DojoConfig) {
[],
40_000,
false,
);

await getEntities(
network.toriiClient,
{
Keys: {
keys: [undefined, undefined],
pattern_matching: "FixedLen",
models: ["s0_eternum-CapacityConfigCategory", "s0_eternum-ResourceCost"],
},
},
network.contractComponents as any,
[],
[],
40_000,
false,
);
).finally(() => {
setLoading(LoadingStateKey.SingleKey, false);
});

const sync = await syncEntitiesDebounced(network.toriiClient, network.contractComponents as any, [], false);

configManager.setDojo(components);

setLoading(LoadingStateKey.Events, true);
const eventSync = getEvents(
network.toriiClient,
network.contractComponents.events as any,
Expand Down Expand Up @@ -198,7 +210,9 @@ export async function setup({ ...config }: DojoConfig) {
},
false,
false,
);
).finally(() => {
setLoading(LoadingStateKey.Events, false);
});

return {
network,
Expand Down
4 changes: 3 additions & 1 deletion client/src/hooks/store/useUIStore.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { ThreeStore, createThreeStoreSlice } from "./_threeStore";
import { BattleViewInfo } from "./types";
import { BlockchainStore, createBlockchainStore } from "./useBlockchainStore";
import { RealmStore, createRealmStoreSlice } from "./useRealmStore";
import { WorldStore, createWorldStoreSlice } from "./useWorldLoading";

type TooltipType = {
content: React.ReactNode;
Expand Down Expand Up @@ -70,7 +71,7 @@ interface UIStore {
setShowToS: (show: boolean) => void;
}

export type AppStore = UIStore & PopupsStore & ThreeStore & BuildModeStore & RealmStore & BlockchainStore;
export type AppStore = UIStore & PopupsStore & ThreeStore & BuildModeStore & RealmStore & BlockchainStore & WorldStore;

const useUIStore = create(
subscribeWithSelector<AppStore>((set, get) => ({
Expand Down Expand Up @@ -139,6 +140,7 @@ const useUIStore = create(
...createBuildModeStoreSlice(set),
...createRealmStoreSlice(set),
...createBlockchainStore(set),
...createWorldStoreSlice(set),
})),
);

Expand Down
Loading
Loading