Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DON'T MERGE][DEV] feat: introduce file_table with sqlite3, re-implement create_standby.rebuild_mode #453

Draft
wants to merge 184 commits into
base: main
Choose a base branch
from
Draft

[DON'T MERGE][DEV] feat: introduce file_table with sqlite3, re-implement create_standby.rebuild_mode #453

wants to merge 184 commits into from

Conversation

Bodong-Yang
Copy link
Member

@Bodong-Yang Bodong-Yang commented Dec 11, 2024 •
edited
Loading

Note

Due to the size of this PR is large, some sub PRs will be split from this dev PR.

Bodong-Yang and others added 30 commits December 11, 2024 07:18
@Bodong-Yang
add msgpack to deps
08e6f76
@Bodong-Yang
ota_metadata: implement file_table
24dd822
@Bodong-Yang
ota_metadata.legacy: add rs_table
8cc3378
@Bodong-Yang
minor update
a88584b
@Bodong-Yang
implement csv_parser
5639201
@Bodong-Yang
deps: bump pydantic to >=2.10,<3 (#451)
9688360 
This PR bumps pydantic version to >=2.10,<3.
With pydantic >=2.10, typing issues related to IPAddress and URL has been resolved, update the ecu_info and proxy_info modules accordingly.
@Bodong-Yang
bump to use simple-sqlite3-orm<0.4,>=0.3
5c55be7
@Bodong-Yang
minor update
fb374c8
@Bodong-Yang
WIP: parser: add OTAMeta2
17793ba
@Bodong-Yang
WIP: OTAMeta2: add connect_fstable and connect_rstable API
9f33dea
@Bodong-Yang
cleanup ota_metadata.legacy.parser
da1a9f2
@Bodong-Yang
finish cleanup ota_metadata.legacy
d5fa7fd
@Bodong-Yang
file_table: record table name in tablespec
26dbcb5
@Bodong-Yang
ota_metadata orms: embbeded table name
b72fc71
@Bodong-Yang
minor update
483a5d1
@Bodong-Yang
parser: add db fpath properties
948e91b
@Bodong-Yang
rs_table: add size field
4f1e5a5
@Bodong-Yang
ota_metadata.parser: split OTAMetadata class to a new module metadata
5fda42d
@Bodong-Yang
move DownloadInfo to ota_metadata.utils.__init__
4c15dfe
@Bodong-Yang
finish up ota_metadata.legacy refactoring
e51e616
@Bodong-Yang
finish up create_standby.delta_gen
1afc03b
@Bodong-Yang
fstable: add copy_to_target
a2c4894
@Bodong-Yang
ota_metadata.file_table: use NamedTuple for InodeTable to reduce size
eeeb7d7
@Bodong-Yang
file_table.types: fix Xattr and InodeTable validator
e512525
@Bodong-Yang
minor update
3969d36
@Bodong-Yang
create_standby._delta_gen: remove hardlink group related implementation
7b49c5b
@Bodong-Yang
delta_gen: refine calculate_delta
3d0794b
@Bodong-Yang
file_table: now support handling hardlink files
d6ae559
@Bodong-Yang
table.copy_to_target: minor refinement
adb7328
@Bodong-Yang
table: add support for xattr
3abb339
@Bodong-Yang
cleanup unused code
8715805
@Bodong-Yang
orm_base.ORMPoolBase: add a helper method
a4ee23b
@Bodong-Yang
orm_base: minor fix to check_entry
c2a21f1
@Bodong-Yang
metadata: use the original connections to write
b7ac912
@Bodong-Yang
orm_base: check_entry
e2977da
@Bodong-Yang
working for now
1d2f2a9
@Bodong-Yang
minor update
9ff7fcf
@Bodong-Yang
first working version
0659e06
@Bodong-Yang
delta_gen: minor refinement
79a12c9
@Bodong-Yang
minor cleanup
f267656
@Bodong-Yang
no need to enable wal mode for in-memory db
9def53d
@Bodong-Yang
metadata: remember all the conns made, and provide APIs to close all …
5a1ebcd 
…the conns
@Bodong-Yang
ota_core: release db connection resources
ecab819
@Bodong-Yang
fix sqlite3 connection cannot be weakrefed
e7bc6b9
@Bodong-Yang
minor update
708c793
@Bodong-Yang
minor update
b6c011d
@Bodong-Yang
delta_gen: still use the previous method
2789ade
@Bodong-Yang
delta_gen: still use the old but fast method
c6e157b
@Bodong-Yang
minor update
318b44f
@Bodong-Yang
fix _delta_gen
efe52d5
@Bodong-Yang
minor update
23e92d4
@Bodong-Yang
rebuild_mode: no need to use threadpool for configuring dirs and syml…
3370cc0 
…inks
@Bodong-Yang
minor update and refine
21aa019
@Bodong-Yang
remove duplicated logic
c562b28
@Bodong-Yang
remove duplicated logic
499fc2e
@Bodong-Yang
minor update
a32e6c2
@Bodong-Yang
file_table: fix chown reset sticky bit. always do chown before chmod!
e395b20
@Bodong-Yang
Revert "rebuild_mode: no need to use threadpool for configuring dirs …
7998d63 
…and symlinks"

This reverts commit 3370cc0.
@Bodong-Yang
delta_gen: minor optimization
9f0f2ad
@Bodong-Yang
minor fix
056e518
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
don't merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@Bodong-Yang