Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up NetworkTMLE #1

Open
pzivich opened this issue Sep 22, 2022 · 2 comments
Open

Speed up NetworkTMLE #1

pzivich opened this issue Sep 22, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@pzivich
Copy link
Owner

pzivich commented Sep 22, 2022

Right now the bottleneck is the creation of the data for the IPW numerator / Monte Carlo integration. This should be easy to speed up (via Pool) or using some clever tricks in generation. The real issue is the creation of all the summary measures for each copy (lots of matrix multiplications)

@pzivich pzivich added the enhancement New feature or request label Sep 22, 2022
@pzivich
Copy link
Owner Author

pzivich commented Oct 13, 2022

So, the packing in Pool is expensive. However, when calculating all the different summary measures, this initial expense becomes worthwhile. I would like to find something that is quicker than Pool with the packing into a list. If I can find something that is, then everything will be far faster

@pzivich
Copy link
Owner Author

pzivich commented Oct 13, 2022

namespace looks like a promising alternative to access all the objects without copying

https://superfastpython.com/multiprocessing-manager-namespace/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant