Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use name hash for model identifiers #1670

Merged
merged 35 commits into from
Mar 25, 2024
Merged

Conversation

remybar
Copy link
Contributor

@remybar remybar commented Mar 15, 2024

Based on #1600

This is a backward incompatible change to migrate model identifiers to be name hash (keccak(name)) rather than the name literal. This will be necessary as we move to fully qualified model names and is necessary to support longer model names too. It aligns model identifiers with event identifiers which also uses keccak(event_name)

Ref DOJ-226, #1629

remybar and others added 14 commits March 10, 2024 22:33
…e#1643)

The function `handle_introspect_enum` handles the derive Introspect attribute for an enum.
At the moment, two cases are supported: tuples and type paths.

A type path may be a primitive type or a custom type (struct or enum).
A tuple may be composed of primitive and/or custom types.

This PR updates the function `handle_introspect_enum` to handle all these cases properly.
* fix: remove old reference to submodule

* Update submodule

* Will this now work?

* Removed old files

* Correctly updated gitmodules

* chore: update forge std

---------

Co-authored-by: glihm <[email protected]>
* fix: ensure warnings are not blocking the test build

* fix: update default world address for spawn and move example
* feat: check model name and address before granting write access

* Update crates/dojo-world/src/contracts/model.rs

Co-authored-by: glihm <[email protected]>

* Update bin/sozo/src/ops/auth.rs

Co-authored-by: glihm <[email protected]>

---------

Co-authored-by: glihm <[email protected]>
…e#1576)

* check existing class-hashes before registering model

* update checking if models registered logic

* apply review feedback

* add initial test fixture for reregister

* check if models need reregistering using remote manifest

* test ouput of the reregister command

* apply further review feedback

* allow dead code for test utils
…oengine#1651)

* feat: add transaction_hash and block_number in the manifest file

Store the `transaction_hash` and the `block_number` of the World deployment in the
manifest file.

* merge transaction_hash and block_number from previous deployed manifest if exists

* update spawn-and-move example

* fix fmt issues
…gine#1556)

* refacto: using cainome parser in order to parse events in sozo

* refacto: clean logs + add tests

* fix: clippy error

* refacto: best error handling

* update how felt are displayed + remove  unnecessary expect in tests

* fix tests

* fix: fmt and artifacts parsing for missing events

* fix: adjust event count as now world and base are included

* return error from parse_and_print_events function

* fix: ensure world address is always present to filter events

---------

Co-authored-by: glihm <[email protected]>
* add simulation to the RPC

* simulate transaction on sequencer

* add `new_query`

* add fee type

* lint

* update to `simulate_transactions`

* remove `simulation.rs`

* individual parameters for `simulate_transactions`

* lint rebase

* update to fit spec

* pin link
* feat(libp2p): start working on persistence & grpc

* feat: add relay proto and db migration

* feat(libp2p): relaystorage for message persistence

* feat(libp2p): relay subscription service

* refactor: start implementing models/ty for libp2p

* refactor(torii-core): generic set ty query

* feat: use models for libp2p messages

* feat: rpcs for subscring and retrieving messages

* refactor(torii-core): arc rwlock for sql database & work on messages for models

* refactor: tests & write message to db

* refactor(grpc): grpc service & subscription

* refactor: use entitiy type and models for messages

* chore: fmt & clippy

* refactor: unused async

* chore: world descriptor

* refactor: unused deps & fmt

* chore: remove post types

* chore: unused libp2p deps

* chore: non wasm deps

* chore

* chore: fmt

* fix: test

* refactor: check if entity already exists for libp2p messages

* chore: fmt & clippy

* feat(libp2p): start working on identity and signatures for messages

* refactor(libp2p): use array for signature type and check its ocmponents

* refactor: decouple logic

* refactor(libpo2p): extract identity & signature from validate message

* feat(libp2p): eip 712 typed data

* feat(libp2p): start workling on typed data structure snip-12

* refactor: change name

* feat: handle type encoding

* feat: typed data message encoding

* refactor: rework in progress for using snip-12

* feat(libp2p): testing typed data with mocks & tests

* fix: typed data passing test for enums

* feat: test for presets and refactor type primitives

* refactor: string parsing for felts

* fix: dont include type hash on children objects

* fix: json ordering issue with serde_json

* fix: typed data for shortstring

* fix: message encoding - passes all tests

* feat: start supporting enums

* feat: handle enum variant types

* chore: rebase and fix db rwlock

* refactor: encode all object types & start fixing enums

* fix: enum test

* fix: preset types. full compatibility with starknet js

* refactor: more optimized approach for preset types

* feat(libp2p): valid message signature & identity

* refactor: wip identity and verifying message

* refactor: get entity model identity & check

* wip: start working on encoding typed data to Ty

* refactor: implement ty types for encode typed data

* refactor: parse object to ty function

* refactor: set entity from relay w verification

* feat: sign last signature and check signed last sig against last sig in db

* chore: fmt

* fix: torii grpc

* refactor: sqlx onlyu server side

* chore: clippy

* chore: use char

* refactor: sql reference

* chore: fmt & clippy

* refactor: typed data payload

* fix: clippy
remybar and others added 4 commits March 15, 2024 10:26
…joengine#1668)

* fix(torii): Added timezone informations to naive datetimes

* fix(torii_naive_datetimes): Quick parameters renaming

* fix(torii_naive_datetimes): Used rust fmt script

* fix(torii_naive_datetimes): Naive datetime refacto

* fix(torii_naive_datetimes): Quick refacotring

* fix(torii_naive_datetimes): Fixed cargo clippy error

* fix(torii_naive_datetimes): Updated datetime retrieval
* refactor: write manifest file before trying to deploy them

* fix lints

* sort fs entries for deterministic order in manifest

* write migration output for partial migrations

* make suggested changes
* feat(sozo): add seed to manifest

* make suggested changes
crates/dojo-core/src/resource_metadata.cairo Outdated Show resolved Hide resolved
crates/dojo-lang/src/model.rs Show resolved Hide resolved
crates/dojo-lang/src/model.rs Outdated Show resolved Hide resolved
crates/dojo-world/src/manifest.rs Outdated Show resolved Hide resolved
broody and others added 4 commits March 18, 2024 21:46
…ngine#1672)

feat: detect and manage manifests and artifacts discrepancies

`sozo` uses several file types (previously generated by `sozo build` and `sozo migrate`)
as input for migration. All of these files may be corrupt or may have discrepancies due
to an older format version.

This PR introduces a new `sozo clean` command to clean the project of these files.
`sozo clean --manifest-abis` cleans only manifest and ABI files.
`sozo clean --artifacts` cleans only artifacts.

When an anomaly is detected in a file, the user is informed of the problem
and advised to run `sozo clean` to clean up the project.
* Add explorer flag to open World Explorer

* Log error message instead of unwrap when fail to open world explorer on browser

* fix typo

* fix typo 2

---------

Co-authored-by: glihm <[email protected]>
remybar and others added 3 commits March 20, 2024 13:32
…gine#1631)

* feat(torii): parallelize torii sync_range func

* refactor: retrieve events & store only relevant txns

* refactor: fix chunk size overflow & handle next events pages

* chore: fmt

* chore: clippy

* chore: fmt

* refactor: execute all txns at end of sync

* chore: code

* chroe: suggested changes

* feat: configure events page chunk size

* fmt

* refactor: include chink size in config

* fix
* Split sozo ops into its own crate

* fix: minor fixes and adjust commands documentation

* fix: add missing file

* fix: ensure migration failure is not silent

---------

Co-authored-by: Jonatan Chaverri <[email protected]>
Co-authored-by: glihm <[email protected]>
remybar and others added 2 commits March 21, 2024 10:52
…acro (dojoengine#1656)

* feat: add eventmessage for emitting models & start refactoring emit macro

* refactor: emit multiple models event

* feat: event message processor

* feat: emit_message world func emit event mssage

* refactor: catch all event processor

* refactor: check model key as name

* feat: model name keccak as id & rework model events

* fix: storing entities with model hash

* refactor: catch all event message  and store

* feat: emit model evrnt from spawn and move spawn

* fix: pass keys as array

* feat: fix emit macro and correctly index model events

* feat: event messages migrations & set

* feat: store events messages and new id system

* chore: fmt

* fix: keys array

* feat: add grpc endpoint for event messages

* feat: graphql schema for event messages

* cadd comments for moel name hash

* revert world.cario changes

* fix: graphql entity and model connection

* refactor: only test models name ordering

* fix: entity/modeldata relation

* refactor: remove event testing

* fix: cairo code

* refactor: add back event rxample

* fix: merge

* chore: migration

* fix: subscription test

* fix: tests

* fix: subscription test

* fix: sql test

* chore: format model id correctly in test

* fix: model subscription with id

* fix: schema for model

* chore: revert modelmmeber type

* fix: event message query
@glihm
Copy link
Collaborator

glihm commented Mar 21, 2024

@remybar thanks for the update here and the great work on that!

I'll finish the review once we unlock the indexing of ByteArray, as this PR directly depends on that.

Also, do we want to add a model version to the ModelRegistered event?

@remybar
Copy link
Contributor Author

remybar commented Mar 22, 2024

@remybar thanks for the update here and the great work on that!

I'll finish the review once we unlock the indexing of ByteArray, as this PR directly depends on that.

Ok thanks ! Do I have to look at Torii test errors in the CI or these tests will be fixed thanks to another PR ?

Also, do we want to add a model version to the ModelRegistered event?

For me, it depends on how we plan to use this version field. At the moment, we ignore old ModelRegistered events. That means, even with a version field, we will only process events of the latest version. I would prefer to add a new field when we really need it.

The overlays are managed by the user. We don't want to clean them.
The deployments are only output, and can't cause any conflict.
For this reason, only base should be clean.
@remybar remybar marked this pull request as ready for review March 22, 2024 07:00
@glihm
Copy link
Collaborator

glihm commented Mar 25, 2024

Do I have to look at Torii test errors in the CI or these tests will be fixed thanks to another PR ?

They are related to the support of String by Torii. We should tackle that with @Larkooo this week in an other PR before merging this one.

For me, it depends on how we plan to use this version field. At the moment, we ignore old ModelRegistered events. That means, even with a version field, we will only process events of the latest version. I would prefer to add a new field when we really need it.

Then we could keep it like so for now. I was thinking that if we start versioning the models, it may make sense to have the version to simplify off-chain component logic on filtering the models / supporting multiple version of them.
For now we skip old ones, but as worlds are permission-less, some models may not be updated with newer versions, and we have to check how this can be handled.

@glihm glihm merged commit 73461ed into dojoengine:namehash Mar 25, 2024
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.