Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

General Issue: Improve derive_*_joined_*() functions #2590

Closed
bundfussr opened this issue Dec 2, 2024 · 1 comment · Fixed by #2599 or #2624
Closed

General Issue: Improve derive_*_joined_*() functions #2590

bundfussr opened this issue Dec 2, 2024 · 1 comment · Fixed by #2599 or #2624
Assignees
@bundfussr
Labels
general issue

Comments

@bundfussr
Copy link
Collaborator

Background Information

The derive_*_joined_*() functions require too much memory.

They should be updated such that the intermediate datasets created in the functions are not bigger than the input data and output data.

Definition of Done

The functions work with big datasets.
@bundfussr bundfussr self-assigned this Dec 2, 2024
@bundfussr bundfussr moved this from Backlog to In Progress in admiral (sdtm/adam, dev, ci, template, core) Dec 3, 2024
bundfussr added a commit that referenced this issue Dec 6, 2024
@bundfussr
#2590 improve_joinded: experimenting
dfb5b76
bundfussr added a commit that referenced this issue Dec 6, 2024
@bundfussr
#2590 improve_joinded: try temporary database
5834a48
bundfussr added a commit that referenced this issue Dec 6, 2024
@bundfussr
#2590 improve_joinded: try split and run in parallel
f1093f6
bundfussr added a commit that referenced this issue Dec 9, 2024
@bundfussr
#2590 improve_joinded: handle nr cores limit in check()
d3300c1
@bundfussr bundfussr linked a pull request Dec 9, 2024 that will close this issue
Closes #2590 improve_joined: improve performance of *joined* functions #2599
Merged
15 tasks
bundfussr added a commit that referenced this issue Dec 9, 2024
@bundfussr
#2590 improve_joined: style files
5e1bb4e
bundfussr added a commit that referenced this issue Dec 12, 2024
@bundfussr
#2590 improve_joined: do not use parallel computing, update documenta…
eca0025 
…tion and NEWS
bundfussr added a commit that referenced this issue Dec 12, 2024
@bundfussr
#2590 improve_joined: fix lintr
d3addb7
@bms63 bms63 closed this as completed in bb40ca5 Dec 13, 2024
@bundfussr
Copy link
Collaborator Author

The new memory saving code should be optional because it increases the run-time.

@bundfussr bundfussr reopened this Jan 7, 2025
bundfussr added a commit that referenced this issue Jan 7, 2025
@bundfussr
#2590 improve_joined: make new code optional
ce9b4fa
bundfussr added a commit that referenced this issue Jan 7, 2025
@bundfussr
#2590 improve_joined: style files
02bd560
@bms63 bms63 linked a pull request Jan 7, 2025 that will close this issue
Closes #2590 improve_joined: make new code optional #2624
Merged
15 tasks
bundfussr added a commit that referenced this issue Jan 7, 2025
@bundfussr
#2590 improve_joined: add link to set_admiral_options()
bb76a78
@bms63 bms63 closed this as completed in a98db65 Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bundfussr bundfussr

Labels
general issue
Projects
admiral (sdtm/adam, dev, ci, template, core)
Status: Archive
Milestone
No milestone
1 participant
@bundfussr