Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: retry logic for pepr store call #1109

Merged
merged 1 commit into from
Dec 11, 2024
Merged

fix: retry logic for pepr store call #1109

merged 1 commit into from
Dec 11, 2024

Conversation

mjnagel
Copy link
Contributor

@mjnagel mjnagel commented Dec 11, 2024

Description

The primary change in this PR is to properly handle undefined errors in our retryWithDelay function. The jest tests added provide coverage of the retry function testing a number of scenarios.

Related Issue

No written issue, but token set failure seen on this CI run and no retries observed in the logs.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Other (security config, docs update, etc)

Steps to Validate

This one is harder to reproduce the problem on, so the jest tests are a good place to start for validating the expected functionality. To reproduce almost the exact scenario:

# Deploy slim dev, with the unicorn flavor to avoid rate limits
uds run slim-dev --set flavor=unicorn
# Delete pepr store to artifically add some store failure
kubectl delete peprstores.pepr.dev -n pepr-system pepr-uds-core-store
# Add a test namespace and package
kubectl create ns test
kubectl apply -f - <<EOF
apiVersion: uds.dev/v1alpha1
kind: Package
metadata:
  name: test
  namespace: test
spec:
  sso:
    - name: Test
      clientId: test
      redirectUris:
        - "https://test.uds.dev"
EOF
# Check the pepr watcher logs, you should see several lines like this: "Attempt <some number> of setStoreToken failed, retrying in 2000ms."
# This is where previously no retries would be attempted
kubectl logs -n pepr-system -l app=pepr-uds-core-watcher --tail -1 | grep "failed, retrying"

Checklist before merging

@mjnagel mjnagel self-assigned this Dec 11, 2024
@mjnagel mjnagel marked this pull request as ready for review December 11, 2024 21:50
@mjnagel mjnagel requested a review from a team as a code owner December 11, 2024 21:50
@mjnagel mjnagel merged commit e4c0f61 into main Dec 11, 2024
21 checks passed
@mjnagel mjnagel deleted the proper-store-retry branch December 11, 2024 23:32
mjnagel pushed a commit that referenced this pull request Dec 17, 2024
🤖 I have created a release *beep* *boop*
---


##
[0.33.0](v0.32.1...v0.33.0)
(2024-12-17)


### Features

* configurable authentication flows
([#1102](#1102))
([498574c](498574c))
* experimental opt-in classification banner
([#1127](#1127))
([d701067](d701067))
* set Istio gateway TLS from Kubernetes secret
([#982](#982))
([2711209](2711209))


### Bug Fixes

* kubeapi netpol initialization / support for ingress policies
([#1097](#1097))
([620e6b2](620e6b2))
* retry logic for pepr store call
([#1109](#1109))
([e4c0f61](e4c0f61))


### Miscellaneous

* add additional step to pr request template
([#1104](#1104))
([7370ab1](7370ab1))
* allow separate configuration of admin domain name
([#1114](#1114))
([c331ec1](c331ec1))
* bump aks sku from free to standard to address API server perfo…
([#1121](#1121))
([bcb8848](bcb8848))
* **deps:** update curl to v8.11.1
([#1110](#1110))
([39a656c](39a656c))
* **deps:** update grafana
([#1126](#1126))
([056a6ee](056a6ee))
* **deps:** update grafana to 11.4.0
([#1053](#1053))
([77aa0b4](77aa0b4))
* **deps:** update identity-config to v0.9.0
([#1129](#1129))
([da720b2](da720b2))
* **deps:** update istio to v1.24.1
([#962](#962))
([8ecd5ff](8ecd5ff))
* **deps:** update loki to 3.3.1
([#1022](#1022))
([42d5bda](42d5bda))
* **deps:** update pepr to 0.42.0
(#1095)
([3ebae7b](3ebae7b))
* **deps:** update pepr to v0.42.1
([#1116](#1116))
([bde01da](bde01da))
* **deps:** update playwright to v1.49.1
([#1103](#1103))
([658ad0d](658ad0d))
* **deps:** update support-deps
([#1076](#1076))
([2fa010f](2fa010f))
* **deps:** update support-deps
([#1100](#1100))
([777387b](777387b))
* **deps:** update support-deps
([#1105](#1105))
([18472ea](18472ea))
* **deps:** update support-deps
([#1117](#1117))
([5b2e3a4](5b2e3a4))
* **deps:** update support-deps
([#1125](#1125))
([4a1bdfb](4a1bdfb))
* **deps:** update vector to 0.43.1
([#1107](#1107))
([2f6c8b5](2f6c8b5))
* **deps:** update velero kubectl to v1.31.4
([#1108](#1108))
([bd8ee0e](bd8ee0e))
* **deps:** update velero to v1.32.0
([#1128](#1128))
([669ebe5](669ebe5))
* **docs:** replace promtail reference with vector in prerequisites
([#1098](#1098))
([33cee59](33cee59))
* remove loki peerauth exception
([#1106](#1106))
([f87a96d](f87a96d))
* update arch diagrams
([#1120](#1120))
([e8a1beb](e8a1beb))
* update doc-gen output_dir
([#1123](#1123))
([496ea40](496ea40))
* update infra ci to run weekly and on release pr
([#1124](#1124))
([79534c9](79534c9))
* update README to explicitly indicate the need for a running co…
([#1113](#1113))
([6426c5a](6426c5a))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants