Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues with retrieving/caching addresses during sync #798

Closed
Tracked by #1660 ...
mahalakshme opened this issue Sep 25, 2024 · 3 comments
Closed
Tracked by #1660 ...

Issues with retrieving/caching addresses during sync #798

mahalakshme opened this issue Sep 25, 2024 · 3 comments
Assignees

Comments

@mahalakshme
Copy link
Contributor

mahalakshme commented Sep 25, 2024

Need:

  • On calling POST /syncDetails for syncing by catchment, we are retrieving all addresses of a catchment that takes more time
  • Also we are caching the above retrieved addresses in cache which is bloating the cache memory

Technical suggestions:

  • using ltree of address_level table will be faster to retrieve addresses of a catchment,
  • no need to retrieve from virtual_catchment_address_mapping table, and might not need to cache as well if the above is fast enough
  • But using ltree doesn't work with RLS, might need to do as openchs user

AC:

  • Check if using ltree makes it faster enough equivalent to using cache
  • If so, remove using cache for this purpose.
@mahalakshme mahalakshme changed the title Retreiving addresses from catchment_address_mapping table is slow Issues with retrieving addresses from catchment_address_mapping table Sep 25, 2024
@mahalakshme mahalakshme changed the title Issues with retrieving addresses from catchment_address_mapping table Issues with retrieving addresses from view Sep 25, 2024
@mahalakshme mahalakshme changed the title Issues with retrieving addresses from view Issues with retrieving/caching addresses during sync Sep 25, 2024
@1t5j0y 1t5j0y self-assigned this Sep 26, 2024
@1t5j0y 1t5j0y transferred this issue from avniproject/avni-client Sep 26, 2024
1t5j0y added a commit that referenced this issue Sep 26, 2024
@1t5j0y 1t5j0y moved this from In Progress to Code Review Ready in Avni Product Sep 26, 2024
@petmongrels
Copy link
Contributor

We should introduce reset role in this case as the performance gain seems quite a lot.

https://explain.dalibo.com/plan/1ef87e3b4dd65c31
https://explain.dalibo.com/plan/d9d2be477gb8e2c5

@petmongrels petmongrels moved this from In Code Review to Code Review with Comments in Avni Product Sep 27, 2024
1t5j0y added a commit that referenced this issue Sep 30, 2024
…nce gain. Use join with catchment_address_mapping instead of virtual_catchment_address_mapping while checking for access to subject in DEA.
1t5j0y added a commit that referenced this issue Sep 30, 2024
@1t5j0y
Copy link
Contributor

1t5j0y commented Sep 30, 2024

Pending fixing the cache related tests introduced in #794 to run predictably.

@1t5j0y
Copy link
Contributor

1t5j0y commented Oct 8, 2024

QA Notes:
will have to test following flows to ensure there is no functional change:

  • sync
  • task assignment
  • DEA save
  • reset sync

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

4 participants