Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tlog] Remove all local storage caches #1369

Closed
marcopeereboom opened this issue Mar 19, 2021 · 1 comment
Closed

[tlog] Remove all local storage caches #1369

marcopeereboom opened this issue Mar 19, 2021 · 1 comment

Comments

@marcopeereboom
Copy link
Member

#1180 adds tlog functionality to replace the no longer viable git backend. This PR unfortunately uses the local filesystem to generate various caches. These caches can easily become stale or corrupt and are inherently non-scalable. What needs to happen is that all local caches need to be killed and replaced with a global memory cache of sorts. Either redis or memcached are exactly meant for this use case.

the tlog backend should manage the cache which means it should inherently be coherent so long additions and removals are called in the proper order. Thus every call should be made directly to tlog and let tlog deal with caching. The pseudo code looks like this:

func GetTlogValue(x) y {
  y, err := getFromCache(x)
  if err != nil {
    y, err = getFromTlog(x)
    if err != nil {
      return err
    }
    insertCache(x, y)
  }
  return y
}

func DelValue(x) error {
  delFromCache(x)
  return delFromWherever(x)
}
@lukebp
Copy link
Member

lukebp commented Apr 12, 2021

Whoops, I added a duplicate of this in #1391. Closing this one since we've decided to handle this another way and the above comment no longer applies.

@lukebp lukebp closed this as completed Apr 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants