Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace usage of math/rand with math/rand/v2 #6817

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions multicluster/test/e2e/service_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ package e2e
import (
"encoding/json"
"fmt"
"math/rand"
"math/rand/v2" // Updated import path to use math/rand/v2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a helpful comment to have in the code

"strconv"
"strings"
"testing"
Expand All @@ -33,6 +33,13 @@ import (
e2euttils "antrea.io/antrea/test/e2e/utils"
)

const (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what are these new constants? I'm really confused because they seem unused and completely unrelated to the contents of this file

bufferFlushTimeout = 1 * time.Minute
maxNumBuffersPendingUpload = 5
seed = 1 // Define a fixed seed for deterministic tests
fakeClusterUUID = "fake-cluster-uuid" // Define a fake cluster UUID for testing
)

func initializeForServiceExportsTest(t *testing.T, data *MCTestData) {
data.setupServerPodAndService(t)
data.setUpServiceExport(t)
Expand Down Expand Up @@ -97,7 +104,7 @@ func (data *MCTestData) tearDownClientPodInCluster(t *testing.T) {
deletePodWrapper(t, data, westCluster, multiClusterTestNamespace, getClusterRegularClientPodName(westCluster))
}

// Try to curl the counter part Services in east and west clusters.
// Try to curl the counterpart Services in east and west clusters.
// If we get status code 200, it means that the resources are exported by the east
// cluster and imported by the west cluster.
func testMCServiceConnectivity(t *testing.T, data *MCTestData) {
Expand Down Expand Up @@ -245,7 +252,6 @@ func (data *MCTestData) testANNPToServices(t *testing.T) {

connectivity = data.probeFromPodInCluster(eastCluster, multiClusterTestNamespace, eastRegularClientName, "client", eastIP, mcWestClusterTestService, 80, corev1.ProtocolTCP)
assert.Equal(t, antreae2e.Dropped, connectivity, "Failure -- wrong result from probing exported Service from regular clientPod after applying toServices AntreaNetworkPolicy")

}

func (data *MCTestData) testStretchedNetworkPolicy(t *testing.T) {
Expand Down Expand Up @@ -458,7 +464,7 @@ func (data *MCTestData) testStretchedNetworkPolicyUpdatePolicy(t *testing.T) {
// Update the policy to select the eastRegularClient.
acnpBuilder.AddStretchedIngressRule(map[string]string{"antrea-e2e": eastRegularClientName}, nil, "", nil, crdv1beta1.RuleActionDrop)
if _, err := data.createOrUpdateACNP(westCluster, acnpBuilder.Get()); err != nil {
t.Fatalf("Error updateing ACNP %s: %v", acnpBuilder.Name, err)
t.Fatalf("Error updating ACNP %s: %v", acnpBuilder.Name, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while these changes are legit (fixing typos), they are unrelated to the main PR
I would recommend making a separate PR for them

}
connectivity = data.probeFromPodInCluster(eastCluster, multiClusterTestNamespace, eastRegularClientName, "client", westExpSvcIP, mcWestClusterTestService, 80, corev1.ProtocolTCP)
assert.Equal(t, antreae2e.Dropped, connectivity, getStretchedNetworkPolicyErrorMessage(eastRegularClientName))
Expand Down Expand Up @@ -509,7 +515,8 @@ func (data *MCTestData) getNodeNamesFromCluster(clusterName string) (string, str
return "", "", fmt.Errorf("error when getting Node list: %v, stderr: %s", err, stderr)
}
nodes := strings.Split(strings.TrimRight(output, " "), " ")
gwIdx := rand.Intn(len(nodes)) // #nosec G404: for test only
mainRand := rand.New(rand.NewPCG(1, 0)) // Initialize with fixed seeds for deterministic behavior
gwIdx := mainRand.IntN(len(nodes)) // #nosec G404: for test only
Comment on lines +518 to +519
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if deterministic behavior is really important / desired here. Maybe try with the main random generator provided by the package.

Overall I think maybe avoiding deterministic behavior is a good thing for e2e tests.

var regularNode string
for i, node := range nodes {
if i != gwIdx {
Expand All @@ -524,7 +531,7 @@ func (data *MCTestData) getNodeNamesFromCluster(clusterName string) (string, str
func (data *MCTestData) setGatewayNode(t *testing.T, clusterName string, nodeName string) error {
rc, _, stderr, err := provider.RunCommandOnNode(data.getControlPlaneNodeName(clusterName), fmt.Sprintf("kubectl annotate node %s multicluster.antrea.io/gateway=true", nodeName))
if err != nil || rc != 0 || stderr != "" {
return fmt.Errorf("error when annotate the Node %s: %s, stderr: %s", nodeName, err, stderr)
return fmt.Errorf("error when annotating the Node %s: %s, stderr: %s", nodeName, err, stderr)
}
t.Logf("The Node %s is annotated as Gateway in cluster %s", nodeName, clusterName)
return nil
Expand Down
4 changes: 2 additions & 2 deletions pkg/agent/monitortool/monitor.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ package monitortool

import (
"context"
"math/rand"
"math/rand/v2"
"net"
"sync"
"sync/atomic"
Expand All @@ -40,7 +40,7 @@ import (
)

// #nosec G404: random number generator not used for security purposes.
var icmpEchoID = rand.Int31n(1 << 16)
var icmpEchoID = rand.IntN(1 << 16)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this change, you should remove the explicit casts to int in the rest of the code. Look for occurrences of int(icmpEchoID) in this file.


const (
ipv4ProtocolICMPRaw = "ip4:icmp"
Expand Down
2 changes: 1 addition & 1 deletion pkg/agent/multicast/mcast_controller_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ package multicast
import (
"context"
"fmt"
"math/rand"
"math/rand/v2"
"net"
"os"
"sync"
Expand Down
2 changes: 1 addition & 1 deletion pkg/agent/openflow/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ package openflow

import (
"fmt"
"math/rand"
"math/rand/v2"
"net"

"antrea.io/libOpenflow/openflow15"
Expand Down
20 changes: 14 additions & 6 deletions pkg/agent/openflow/cookie/allocator_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,21 @@
package cookie

import (
"math/rand"
"math/rand/v2" // Updated import path to use math/rand/v2
"sync"
"testing"

"github.com/stretchr/testify/assert"
)

const (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment as above, not sure what those constants are for

seed = 1 // Define a fixed seed for deterministic tests
fakeClusterUUID = "fake-cluster-uuid" // Define a fake cluster UUID for testing
)

var result ID // Ensures that the call to Request is not optimized-out by the compiler.

// BenchmarkIDAllocate benchmarks the allocation of IDs.
func BenchmarkIDAllocate(b *testing.B) {
a := NewAllocator(0)
for i := 0; i < b.N; i++ {
Expand All @@ -37,16 +43,17 @@ func TestConcurrentAllocate(t *testing.T) {
concurrentNum := 8

// #nosec G404: random number generator not used for security purposes
round := rand.Uint64() >> (64 - BitwidthRound)
mainRand := rand.New(rand.NewPCG(1, 0)) // Initialize with fixed seeds for deterministic behavior
round := mainRand.Uint64() >> (64 - BitwidthRound)
Comment on lines -40 to +47
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need for a fixed seed and deterministic behavior here IMO, I would keep it simple and use the main random generator.

Even before the switch to rand/v2, the behavior of the random number generator was changed in Go 1.20 and it became auto-seeded randomly. See https://pkg.go.dev/math/rand#Seed. We didn't observe any issue when we upgraded to 1.20, so there is no need to revert back to a fixed seed here.

a := NewAllocator(round)

eachGoroutine := func() {
eachGoroutine := func(randSrc *rand.Rand) {
var seq []Category

for i := 0; i < eachTotal; i++ {
seq = append(seq, NetworkPolicy, Service, PodConnectivity, Default)
}
rand.Shuffle(len(seq), func(a, b int) { seq[a], seq[b] = seq[b], seq[a] })
randSrc.Shuffle(len(seq), func(aIdx, bIdx int) { seq[aIdx], seq[bIdx] = seq[bIdx], seq[aIdx] })

for i := 0; i < eachTotal/2; i++ {
id := a.Request(seq[i])
Expand All @@ -59,15 +66,16 @@ func TestConcurrentAllocate(t *testing.T) {
assert.Equal(t, round, id.Round(), id.String())
assert.Equal(t, seq[i].String(), id.Category().String(), id.String())
}

}

var wg sync.WaitGroup
for i := 0; i < concurrentNum; i++ {
wg.Add(1)
go func() {
defer wg.Done()
eachGoroutine()
// Each goroutine gets its own rand.Rand instance with unique seeds for independent shuffles
goroutineRandSrc := rand.New(rand.NewPCG(mainRand.Uint64(), mainRand.Uint64()))
Comment on lines +76 to +77
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mainRand should not be safe to use by concurrent goroutines according to the documentation: https://pkg.go.dev/math/rand/v2#Source

eachGoroutine(goroutineRandSrc)
}()
}
wg.Wait()
Expand Down
6 changes: 3 additions & 3 deletions pkg/flowaggregator/s3uploader/s3uploader.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"context"
"fmt"
"io"
"math/rand"
"math/rand/v2"
"sync"
"time"

Expand Down Expand Up @@ -136,7 +136,7 @@ func NewS3UploadProcess(input S3Input, clusterUUID string) (*S3UploadProcess, er

buf := &bytes.Buffer{}
// #nosec G404: random number generator not used for security purposes
nameRand := rand.New(rand.NewSource(time.Now().UnixNano()))
nameRand := rand.New(rand.NewPCG(uint64(time.Now().UnixNano()), 0)) // Updated initialization
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should use the main random generator provided by the package here (see my comment in the issue, where I answered your question). The main generator is no longer deterministic.


s3ExportProcess := &S3UploadProcess{
bucketName: config.BucketName,
Expand Down Expand Up @@ -374,7 +374,7 @@ func randSeq(randSrc *rand.Rand, n int) string {
var alphabet = []rune("abcdefghijklmnopqrstuvwxyz0123456789")
b := make([]rune, n)
for i := range b {
randIdx := randSrc.Intn(len(alphabet))
randIdx := randSrc.IntN(len(alphabet))
b[i] = alphabet[randIdx]
}
return string(b)
Expand Down
12 changes: 6 additions & 6 deletions pkg/flowaggregator/s3uploader/s3uploader_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import (
"bytes"
"context"
"fmt"
"math/rand"
"math/rand/v2"
"strings"
"testing"
"time"
Expand Down Expand Up @@ -104,7 +104,7 @@ func TestBatchUploadAll(t *testing.T) {
ctx := context.Background()
mockS3Uploader.EXPECT().Upload(ctx, gomock.Any(), nil).Return(nil, nil)
// #nosec G404: random number generator not used for security purposes
nameRand := rand.New(rand.NewSource(seed))
nameRand := rand.New(rand.NewPCG(1, 0))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not use the seed constant anymore?

s3UploadProc := S3UploadProcess{
compress: false,
maxRecordPerFile: 10,
Expand Down Expand Up @@ -137,7 +137,7 @@ func TestBatchUploadAllPartialSuccess(t *testing.T) {
mockS3Uploader.EXPECT().Upload(ctx, gomock.Any(), nil).Return(nil, fmt.Errorf("random error")),
)
// #nosec G404: random number generator not used for security purposes
nameRand := rand.New(rand.NewSource(seed))
nameRand := rand.New(rand.NewPCG(1, 0))
s3UploadProc := S3UploadProcess{
compress: false,
maxRecordPerFile: 1,
Expand Down Expand Up @@ -166,7 +166,7 @@ func TestBatchUploadAllError(t *testing.T) {
ctx := context.Background()
s3uploader := &S3Uploader{}
// #nosec G404: random number generator not used for security purposes
nameRand := rand.New(rand.NewSource(seed))
nameRand := rand.New(rand.NewPCG(1, 0))
s3UploadProc := S3UploadProcess{
bucketName: "test-bucket-name",
compress: false,
Expand Down Expand Up @@ -209,7 +209,7 @@ func TestFlowRecordPeriodicCommit(t *testing.T) {
},
)
// #nosec G404: random number generator not used for security purposes
nameRand := rand.New(rand.NewSource(seed))
nameRand := rand.New(rand.NewPCG(1, 0))
s3UploadProc := S3UploadProcess{
compress: false,
maxRecordPerFile: 10,
Expand Down Expand Up @@ -249,7 +249,7 @@ func TestFlushCacheOnStop(t *testing.T) {
mockS3Uploader := s3uploadertesting.NewMockS3UploaderAPI(ctrl)
mockS3Uploader.EXPECT().Upload(gomock.Any(), gomock.Any(), nil).Return(nil, nil)
// #nosec G404: random number generator not used for security purposes
nameRand := rand.New(rand.NewSource(seed))
nameRand := rand.New(rand.NewPCG(1, 0))
s3UploadProc := S3UploadProcess{
compress: false,
maxRecordPerFile: 10,
Expand Down
4 changes: 2 additions & 2 deletions pkg/ovs/openflow/ofctrl_packetout.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ package openflow

import (
"encoding/binary"
"math/rand"
"math/rand/v2"
"net"
"time"

Expand All @@ -28,7 +28,7 @@ import (
)

// #nosec G404: random number generator not used for security purposes
var pktRand = rand.New(rand.NewSource(time.Now().UnixNano()))
var pktRand = rand.New(rand.NewPCG(uint64(time.Now().UnixNano()), 0))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we could use rand.New(rand.NewPCG(rand.Uint64(), rand.Uint64())) here. No need to use the time package anymore to seed?


type ofPacketOutBuilder struct {
pktOut *ofctrl.PacketOut
Expand Down
15 changes: 13 additions & 2 deletions pkg/ovs/openflow/ofctrl_packetout_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
package openflow

import (
"math/rand"
"math/rand/v2"
"net"
"reflect"
"testing"
Expand All @@ -26,6 +26,17 @@ import (
"github.com/stretchr/testify/assert"
)

// Initialize pktRand with a fixed seed for predictable test outcomes.
// #nosec G404: random number generator not used for security purposes
var pktRandInstance = rand.New(rand.NewPCG(1, 0)) // Fixed seeds: 1 and 0

// Override pktRand in the main package with pktRandInstance during tests.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"main package" means something else. We are not in the main package here.

// This assumes pktRand is a package-level variable that can be overridden.
// If pktRand is not accessible, consider refactoring your code to allow dependency injection.
func init() {
pktRand = pktRandInstance
}

func Test_ofPacketOutBuilder(t *testing.T) {
newPktOutBuilder := func() *ofPacketOutBuilder {
return &ofPacketOutBuilder{
Expand Down Expand Up @@ -1275,7 +1286,7 @@ func Test_ofPacketOutBuilder_Done(t *testing.T) {
t.Run(tt.name, func(t *testing.T) {
// Specify a hardcoded seed for testing to make output predictable.
// #nosec G404: random number generator not used for security purposes
pktRand = rand.New(rand.NewSource(1))
pktRand = pktRandInstance
b := &ofPacketOutBuilder{
pktOut: tt.fields.pktOut,
icmpID: tt.fields.icmpID,
Expand Down
6 changes: 4 additions & 2 deletions test/e2e/framework.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import (
"encoding/json"
"fmt"
"io"
"math/rand"
"math/rand/v2"
"net"
"os"
"path"
Expand Down Expand Up @@ -69,6 +69,8 @@ var AntreaConfigMap *corev1.ConfigMap

var (
connectionLostError = fmt.Errorf("http2: client connection lost")
// Initialize random number generator with a seed
ran = rand.New(rand.NewPCG(uint64(time.Now().UnixNano()), 0))
Comment on lines +72 to +73
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this needed? why can't we use the main random number generator which is randomly seeded and is also thread-safe?

)

const (
Expand Down Expand Up @@ -2226,7 +2228,7 @@ func randSeq(n int) string {
b := make([]rune, n)
for i := range b {
// #nosec G404: random number generator not used for security purposes
randIdx := rand.Intn(len(lettersAndDigits))
randIdx := ran.IntN(len(lettersAndDigits))
b[i] = lettersAndDigits[randIdx]
}
return string(b)
Expand Down
19 changes: 14 additions & 5 deletions test/e2e/performance_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ import (
"context"
"flag"
"fmt"
"math/rand"
"math/rand/v2"
"net/url"
"strings"
"testing"
Expand Down Expand Up @@ -95,7 +95,7 @@ func BenchmarkCustomizeRealizeNetworkPolicy(b *testing.B) {

func randCidr(rnd *rand.Rand) string {
getByte := func() int {
return rnd.Intn(255) + 1
return rnd.IntN(255) + 1
}
return fmt.Sprintf("%d.%d.%d.%d/32", getByte(), getByte(), getByte(), getByte())
}
Expand Down Expand Up @@ -150,11 +150,20 @@ func setupTestPodsConnection(data *TestData) error {
func generateWorkloadNetworkPolicy(policyRules int) *networkv1.NetworkPolicy {
ingressRules := make([]networkv1.NetworkPolicyPeer, policyRules)
// #nosec G404: random number generator not used for security purposes
rnd := rand.New(rand.NewSource(seed))
existingCIDRs := make(map[string]struct{}) // ensure no duplicated cidrs

// Initialize the PCG generator
pcg := rand.PCG{}
// Provide both 'state' and 'sequence' seeds
pcg.Seed(uint64(seed), uint64(1))
rnd := rand.New(&pcg)
Comment on lines +154 to +158
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we using this verbose code here instead of rand.New(rand.NewPCG(seed, 0)) like in the rest of the PR?


existingCIDRs := make(map[string]struct{}) // ensure no duplicated CIDRs
for i := 0; i < policyRules; i++ {
cidr := randCidr(rnd)
for _, ok := existingCIDRs[cidr]; ok; {
for {
if _, ok := existingCIDRs[cidr]; !ok {
break
}
Comment on lines -157 to +166
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why this is an improvement to the existing code?

cidr = randCidr(rnd)
}
existingCIDRs[cidr] = struct{}{}
Expand Down
2 changes: 1 addition & 1 deletion test/integration/agent/net_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ package agent

import (
"fmt"
"math/rand"
"math/rand/v2"
"net"
"testing"

Expand Down