Skip to content

Commit

Permalink
Merge main into release-1.1 for v1.1.2 RC image (#268)
Browse files Browse the repository at this point in the history 
* Remove callSkip to prevent Logger.check error (#254)

* Refactor conntrack cleanup - v4 and v6 (#256)

* Refactor conntrack cleanup

* Minor optimization

* minor change

* Bump google.golang.org/grpc from 1.62.0 to 1.63.2 (#253)

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.62.0 to 1.63.2.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](grpc/grpc-go@v1.62.0...v1.63.2)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump golang.org/x/sys from 0.18.0 to 0.19.0 (#252)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.18.0 to 0.19.0.
- [Commits](golang/sys@v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/onsi/gomega from 1.31.1 to 1.33.0 (#261)

Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.31.1 to 1.33.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](onsi/gomega@v1.31.1...v1.33.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Handle PolicyEndpoint Slice(s) deletion gracefully (#259)

* Handle PE slices during delete

* Handle PE slice cleanup - Pod selector churn

* Remove stale code

---------

Co-authored-by: Jayanth Varavani <[email protected]>

* Added support for network policies applied to pods with a '.' in their pod name (#225)

* Bump github.com/aws/amazon-vpc-cni-k8s from 1.18.0 to 1.18.1 (#262)

Bumps [github.com/aws/amazon-vpc-cni-k8s](https://github.com/aws/amazon-vpc-cni-k8s) from 1.18.0 to 1.18.1.
- [Release notes](https://github.com/aws/amazon-vpc-cni-k8s/releases)
- [Changelog](https://github.com/aws/amazon-vpc-cni-k8s/blob/v1.18.1/CHANGELOG.md)
- [Commits](aws/amazon-vpc-cni-k8s@v1.18.0...v1.18.1)

---
updated-dependencies:
- dependency-name: github.com/aws/amazon-vpc-cni-k8s
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/onsi/ginkgo/v2 from 2.17.1 to 2.17.2 (#263)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.17.1 to 2.17.2.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.17.1...v2.17.2)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update eBPF SDK (#269)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Ryota Sakamoto <[email protected]>
Co-authored-by: Jayanth Varavani <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Apurup Chevuru <[email protected]>
Co-authored-by: Zach Dorame-Barajas <[email protected]>
  • Loading branch information
@jchen6585 @ryota-sakamoto
@jayanthvn @dependabot @achevuru @zachdorame
6 people authored May 8, 2024
1 parent 7fdfc7a commit 309d343
Show file tree
Hide file tree
Showing 15 changed files with 561 additions and 313 deletions.
61 changes: 51 additions & 10 deletions controllers/policyendpoints_controller.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,8 @@ type PolicyEndpointsReconciler struct {
podIdentifierToPolicyEndpointMapMutex sync.Mutex
// Maps PolicyEndpoint resource with a list of local pods
policyEndpointSelectorMap sync.Map
// Maps a Network Policy to list of selected pod Identifiers
networkPolicyToPodIdentifierMap sync.Map
//BPF Client instance
ebpfClient ebpf.BpfClient

Expand Down Expand Up @@ -186,6 +188,12 @@ func (r *PolicyEndpointsReconciler) cleanUpPolicyEndpoint(ctx context.Context, r
policyTearDownLatency.WithLabelValues(req.NamespacedName.Name, req.NamespacedName.Namespace).Observe(duration)
}

for _, podToBeCleanedUp := range podsToBeCleanedUp {
podIdentifier := utils.GetPodIdentifier(podToBeCleanedUp.Name, podToBeCleanedUp.Namespace, r.log)
//Delete this policyendpoint resource against the current PodIdentifier
r.deletePolicyEndpointFromPodIdentifierMap(ctx, podIdentifier, req.NamespacedName.Name)
}

return nil
}

Expand All @@ -201,7 +209,7 @@ func (r *PolicyEndpointsReconciler) updatePolicyEnforcementStatusForPods(ctx con
r.log.Info("Updating Pod: ", "Name: ", targetPod.Name, "Namespace: ", targetPod.Namespace)

deletePinPath := true
podIdentifier := utils.GetPodIdentifier(targetPod.Name, targetPod.Namespace)
podIdentifier := utils.GetPodIdentifier(targetPod.Name, targetPod.Namespace, r.log)
r.log.Info("Derived ", "Pod identifier to check if update is needed : ", podIdentifier)
//Derive the podIdentifier and check if there is another pod in the same replicaset using the pinpath
if found, ok := podIdentifiers[podIdentifier]; ok {
Expand Down Expand Up @@ -281,7 +289,7 @@ func (r *PolicyEndpointsReconciler) configureeBPFProbes(ctx context.Context, pod
for _, pod := range targetPods {
r.log.Info("Processing Pod: ", "name:", pod.Name, "namespace:", pod.Namespace, "podIdentifier: ", podIdentifier)

currentPodIdentifier := utils.GetPodIdentifier(pod.Name, pod.Namespace)
currentPodIdentifier := utils.GetPodIdentifier(pod.Name, pod.Namespace, r.log)
if currentPodIdentifier != podIdentifier {
r.log.Info("Target Pod doesn't belong to the current pod Identifier: ", "Name: ", pod.Name, "Pod ID: ", podIdentifier)
continue
Expand Down Expand Up @@ -314,9 +322,7 @@ func (r *PolicyEndpointsReconciler) cleanupeBPFProbes(ctx context.Context, targe
var isIngressIsolated, isEgressIsolated bool
noActiveIngressPolicies, noActiveEgressPolicies := false, false

podIdentifier := utils.GetPodIdentifier(targetPod.Name, targetPod.Namespace)
// Delete this policyendpoint resource against the current PodIdentifier
r.deletePolicyEndpointFromPodIdentifierMap(ctx, podIdentifier, policyEndpoint)
podIdentifier := utils.GetPodIdentifier(targetPod.Name, targetPod.Namespace, r.log)

// Detach eBPF probes attached to the local pods (if required). We should detach eBPF probes if this
// is the only PolicyEndpoint resource that applies to this pod. If not, just update the Ingress/Egress Map contents
Expand Down Expand Up @@ -467,6 +473,7 @@ func (r *PolicyEndpointsReconciler) updateeBPFMaps(ctx context.Context, podIdent
func (r *PolicyEndpointsReconciler) deriveTargetPodsForParentNP(ctx context.Context,
parentNP, resourceNamespace, resourceName string) ([]types.NamespacedName, map[string]bool, []types.NamespacedName) {
var targetPods, podsToBeCleanedUp, currentPods []types.NamespacedName
var targetPodIdentifiers []string
podIdentifiers := make(map[string]bool)
currentPE := &policyk8sawsv1.PolicyEndpoint{}

Expand Down Expand Up @@ -509,9 +516,13 @@ func (r *PolicyEndpointsReconciler) deriveTargetPodsForParentNP(ctx context.Cont
targetPods = append(targetPods, currentTargetPods...)
for podIdentifier, _ := range currentPodIdentifiers {
podIdentifiers[podIdentifier] = true
targetPodIdentifiers = append(targetPodIdentifiers, podIdentifier)
}
}

//Update active podIdentifiers selected by the current Network Policy
stalePodIdentifiers := r.deriveStalePodIdentifiers(ctx, resourceName, targetPodIdentifiers)

for _, policyEndpointResource := range parentPEList {
policyEndpointIdentifier := utils.GetPolicyEndpointIdentifier(policyEndpointResource,
resourceNamespace)
Expand All @@ -522,11 +533,16 @@ func (r *PolicyEndpointsReconciler) deriveTargetPodsForParentNP(ctx context.Cont
r.log.Info("No more target pods so deleting the entry in PE selector map for ", "Name ", policyEndpointResource)
r.policyEndpointSelectorMap.Delete(policyEndpointIdentifier)
}

for _, podIdentifier := range stalePodIdentifiers {
r.deletePolicyEndpointFromPodIdentifierMap(ctx, podIdentifier, policyEndpointResource)
}
}

//Update active podIdentifiers selected by the current Network Policy
r.networkPolicyToPodIdentifierMap.Store(utils.GetParentNPNameFromPEName(resourceName), targetPodIdentifiers)

if len(currentPods) > 0 {
podsToBeCleanedUp = r.getPodListToBeCleanedUp(currentPods, targetPods)
podsToBeCleanedUp = r.getPodListToBeCleanedUp(currentPods, targetPods, podIdentifiers)
}
return targetPods, podIdentifiers, podsToBeCleanedUp
}
Expand All @@ -543,7 +559,7 @@ func (r *PolicyEndpointsReconciler) deriveTargetPods(ctx context.Context,
// by the Host IP value.
nodeIP := net.ParseIP(r.nodeIP)
for _, pod := range policyEndpoint.Spec.PodSelectorEndpoints {
podIdentifier := utils.GetPodIdentifier(pod.Name, pod.Namespace)
podIdentifier := utils.GetPodIdentifier(pod.Name, pod.Namespace, r.log)
if nodeIP.Equal(net.ParseIP(string(pod.HostIP))) {
r.log.Info("Found a matching Pod: ", "name: ", pod.Name, "namespace: ", pod.Namespace)
targetPods = append(targetPods, types.NamespacedName{Name: pod.Name, Namespace: pod.Namespace})
Expand All @@ -556,22 +572,24 @@ func (r *PolicyEndpointsReconciler) deriveTargetPods(ctx context.Context,
}

func (r *PolicyEndpointsReconciler) getPodListToBeCleanedUp(oldPodSet []types.NamespacedName,
newPodSet []types.NamespacedName) []types.NamespacedName {
newPodSet []types.NamespacedName, podIdentifiers map[string]bool) []types.NamespacedName {
var podsToBeCleanedUp []types.NamespacedName

for _, oldPod := range oldPodSet {
activePod := false
oldPodIdentifier := utils.GetPodIdentifier(oldPod.Name, oldPod.Namespace, r.log)
for _, newPod := range newPodSet {
if oldPod == newPod {
activePod = true
break
}
}
if !activePod {
if !activePod && !podIdentifiers[oldPodIdentifier] {
r.log.Info("Pod to cleanup: ", "name: ", oldPod.Name, "namespace: ", oldPod.Namespace)
podsToBeCleanedUp = append(podsToBeCleanedUp, oldPod)
}
}

return podsToBeCleanedUp
}

Expand Down Expand Up @@ -606,6 +624,29 @@ func (r *PolicyEndpointsReconciler) updatePodIdentifierToPEMap(ctx context.Conte
return
}

func (r *PolicyEndpointsReconciler) deriveStalePodIdentifiers(ctx context.Context, resourceName string,
targetPodIdentifiers []string) []string {

var stalePodIdentifiers []string
if currentPodIdentifiers, ok := r.networkPolicyToPodIdentifierMap.Load(utils.GetParentNPNameFromPEName(resourceName)); ok {
for _, podIdentifier := range currentPodIdentifiers.([]string) {
r.log.Info("podIdentifier", "name", podIdentifier)
stalePodIdentifier := true
for _, pe := range targetPodIdentifiers {
if pe == podIdentifier {
//Nothing to do if this PE is already tracked against this podIdentifier
stalePodIdentifier = false
break
}
}
if stalePodIdentifier {
stalePodIdentifiers = append(stalePodIdentifiers, podIdentifier)
}
}
}
return stalePodIdentifiers
}

func (r *PolicyEndpointsReconciler) deletePolicyEndpointFromPodIdentifierMap(ctx context.Context, podIdentifier string,
policyEndpoint string) {
r.podIdentifierToPolicyEndpointMapMutex.Lock()
Expand Down
28 changes: 14 additions & 14 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,25 +3,25 @@ module github.com/aws/aws-network-policy-agent
go 1.21

require (
github.com/aws/amazon-vpc-cni-k8s v1.18.0
github.com/aws/aws-ebpf-sdk-go v1.0.7
github.com/aws/amazon-vpc-cni-k8s v1.18.1
github.com/aws/aws-ebpf-sdk-go v1.0.8
github.com/aws/aws-sdk-go v1.50.30
github.com/go-logr/logr v1.4.1
github.com/go-logr/zapr v1.3.0
github.com/golang/mock v1.6.0
github.com/google/go-cmp v0.6.0
github.com/google/uuid v1.6.0
github.com/onsi/ginkgo/v2 v2.15.0
github.com/onsi/gomega v1.31.1
github.com/onsi/ginkgo/v2 v2.17.2
github.com/onsi/gomega v1.33.0
github.com/pkg/errors v0.9.1
github.com/prometheus/client_golang v1.19.0
github.com/spf13/cobra v1.8.0
github.com/spf13/pflag v1.0.5
github.com/stretchr/testify v1.9.0
github.com/vishvananda/netlink v1.2.1-beta.2
go.uber.org/zap v1.27.0
golang.org/x/sys v0.18.0
google.golang.org/grpc v1.62.0
golang.org/x/sys v0.19.0
google.golang.org/grpc v1.63.2
gopkg.in/natefinch/lumberjack.v2 v2.2.1
k8s.io/api v0.29.1
k8s.io/apimachinery v0.29.3
Expand All @@ -39,13 +39,13 @@ require (
github.com/go-openapi/jsonpointer v0.19.6 // indirect
github.com/go-openapi/jsonreference v0.20.2 // indirect
github.com/go-openapi/swag v0.22.3 // indirect
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
github.com/go-task/slim-sprig/v3 v3.0.0 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/gnostic-models v0.6.9-0.20230804172637-c7be7c783f49 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/pprof v0.0.0-20230323073829-e72429f035bd // indirect
github.com/google/pprof v0.0.0-20240424215950-a892ee059fd6 // indirect
github.com/imdario/mergo v0.3.13 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
Expand All @@ -57,20 +57,20 @@ require (
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.48.0 // indirect
github.com/prometheus/common v0.52.2 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/vishvananda/netns v0.0.4 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/exp v0.0.0-20230315142452-642cacee5cc0 // indirect
golang.org/x/net v0.21.0 // indirect
golang.org/x/oauth2 v0.16.0 // indirect
golang.org/x/exp v0.0.0-20231006140011-7918f672742d // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/oauth2 v0.18.0 // indirect
golang.org/x/term v0.18.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/time v0.3.0 // indirect
golang.org/x/tools v0.16.1 // indirect
golang.org/x/tools v0.20.0 // indirect
gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect
google.golang.org/appengine v1.6.8 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240123012728-ef4313101c80 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240227224415-6ceb2ff114de // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
Expand Down
Loading

0 comments on commit 309d343

Please sign in to comment.