ci: test-unit job matrix for win/macos/ubuntu #2206

crazy-max · 2024-01-24T12:59:11Z

crazy-max · 2024-01-24T13:52:21Z

Some tests look to fail on windows runner: https://github.com/docker/buildx/actions/runs/7640538748/job/20815891223#step:5:419

crazy-max · 2024-01-25T09:26:39Z

This one is interesting when running tests on macOS runner when temp dir is the default one: https://github.com/docker/buildx/actions/runs/7652363539/job/20851977017?pr=2206#step:6:863

=== Failed
=== FAIL: controller/pb TestResolvePaths/contextpath (0.00s)
    path_test.go:244: expected pb.BuildOptions{ContextPath:"/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/testresolvepaths759994391/test", DockerfileName:"", PrintFunc:(*pb.PrintFunc)(nil), NamedContexts:map[string]string(nil), Allow:[]string(nil), Attests:[]*pb.Attest(nil), BuildArgs:map[string]string(nil), CacheFrom:[]*pb.CacheOptionsEntry(nil), CacheTo:[]*pb.CacheOptionsEntry(nil), CgroupParent:"", Exports:[]*pb.ExportEntry(nil), ExtraHosts:[]string(nil), Labels:map[string]string(nil), NetworkMode:"", NoCacheFilter:[]string(nil), Platforms:[]string(nil), Secrets:[]*pb.Secret(nil), ShmSize:0, SSH:[]*pb.SSH(nil), Tags:[]string(nil), Target:"", Ulimits:(*pb.UlimitOpt)(nil), Builder:"", NoCache:false, Pull:false, ExportPush:false, ExportLoad:false, SourcePolicy:(*moby_buildkit_v1_sourcepolicy.Policy)(nil), Ref:"", GroupRef:"", Annotations:[]string(nil), XXX_NoUnkeyedLiteral:struct {}{}, XXX_unrecognized:[]uint8(nil), XXX_sizecache:0}, got pb.BuildOptions{ContextPath:"/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/testresolvepaths759994391/test", DockerfileName:"", PrintFunc:(*pb.PrintFunc)(nil), NamedContexts:map[string]string(nil), Allow:[]string(nil), Attests:[]*pb.Attest(nil), BuildArgs:map[string]string(nil), CacheFrom:[]*pb.CacheOptionsEntry(nil), CacheTo:[]*pb.CacheOptionsEntry(nil), CgroupParent:"", Exports:[]*pb.ExportEntry(nil), ExtraHosts:[]string(nil), Labels:map[string]string(nil), NetworkMode:"", NoCacheFilter:[]string(nil), Platforms:[]string(nil), Secrets:[]*pb.Secret(nil), ShmSize:0, SSH:[]*pb.SSH(nil), Tags:[]string(nil), Target:"", Ulimits:(*pb.UlimitOpt)(nil), Builder:"", NoCache:false, Pull:false, ExportPush:false, ExportLoad:false, SourcePolicy:(*moby_buildkit_v1_sourcepolicy.Policy)(nil), Ref:"", GroupRef:"", Annotations:[]string(nil), XXX_NoUnkeyedLiteral:struct {}{}, XXX_unrecognized:[]uint8(nil), XXX_sizecache:0}
    --- FAIL: TestResolvePaths/contextpath (0.00s)

Note the diff with ContextPath:

/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/testresolvepaths759994391/test
vs
/private/var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/testresolvepaths759994391/test

cc @tonistiigi @jedevc

Signed-off-by: CrazyMax <[email protected]>

crazy-max · 2024-01-25T14:04:23Z

build/git_windows.go

+// getLongPathName converts Windows short pathnames to full pathnames.
+// For example C:\Users\ADMIN~1 --> C:\Users\Administrator.
+func getLongPathName(path string) (string, error) {
+	// See https://groups.google.com/forum/#!topic/golang-dev/1tufzkruoTg
+	p, err := windows.UTF16FromString(path)
+	if err != nil {
+		return "", err
+	}
+	b := p // GetLongPathName says we can reuse buffer
+	n, err := windows.GetLongPathName(&p[0], &b[0], uint32(len(b)))
+	if err != nil {
+		return "", err
+	}
+	if n > uint32(len(b)) {
+		b = make([]uint16, n)
+		_, err = windows.GetLongPathName(&p[0], &b[0], uint32(len(b)))
+		if err != nil {
+			return "", err
+		}
+	}
+	return windows.UTF16ToString(b), nil
+}


Logic is the same as moby one https://github.com/moby/moby/blob/e8346c53d93d2cddf3d09910e166cdd8874070b3/integration-cli/utils_windows_test.go#L5-L27

crazy-max · 2024-01-25T14:05:00Z

util/gitutil/path_windows_test.go

+	expected := "C:\\Users\\foobar"
+	if isGitBash() {
+		expected = "C:/Users/foobar"
+	}


As we are using Git bash on ci to run tests, the behavior is not the same.

Signed-off-by: CrazyMax <[email protected]>

crazy-max · 2024-01-25T14:20:03Z

Second commit is a bug on Windows. Let me know if you want this in a dedicated PR. Should be backported as well.

crazy-max · 2024-01-25T14:22:59Z

build/git_windows.go

@@ -0,0 +1,26 @@
+package build


Not sure if we should put this in this package or util one. Maybe on buildkit as it might be useful for Windows support (cc @TBBle @gabriel-samfira)

crazy-max · 2024-01-25T14:25:55Z

.github/workflows/build.yml

-        include:
-          - pkg: ./...
-            skip-integration-tests: 1


@jedevc I removed this one as it's already handled in test-unit job through ubuntu-latest otherwise we have dup testing. Let me know if that sounds good.

crazy-max · 2024-01-25T14:26:40Z

.github/workflows/build.yml

      -
        name: Send to Codecov
        if: always()
        uses: codecov/codecov-action@v3
        with:
          directory: ./bin/testreports
+          flags: integration


Added some flags for codecov so we have a clear distinction between unit and integration tests

crazy-max · 2024-01-26T15:59:30Z

Needs follow-up for client-side integration tests on windows/macos

crazy-max added kind/tests area/ci labels Jan 24, 2024

crazy-max force-pushed the test-os branch from c3a5358 to 1936d07 Compare January 24, 2024 13:00

crazy-max changed the title ~~ci: run unit tests on windows~~ ci: test-os job for unit tests Jan 25, 2024

crazy-max force-pushed the test-os branch 4 times, most recently from 61852c8 to 173a44d Compare January 25, 2024 08:45

crazy-max changed the title ~~ci: test-os job for unit tests~~ ci: test-unit job matrix for win/macos/ubuntu Jan 25, 2024

crazy-max force-pushed the test-os branch 3 times, most recently from 5137a6c to 947498a Compare January 25, 2024 09:20

crazy-max force-pushed the test-os branch 3 times, most recently from f3c264e to b04b304 Compare January 25, 2024 09:39

ci: test-unit job matrix for win/macos/ubuntu

eabbee7

Signed-off-by: CrazyMax <[email protected]>

crazy-max force-pushed the test-os branch from 8d6b2a8 to eabbee7 Compare January 25, 2024 09:53

crazy-max commented Jan 25, 2024

View reviewed changes

crazy-max and others added 2 commits January 25, 2024 15:06

build: resolve 8.3 filename format to long one on Windows

fb2c62a

Signed-off-by: CrazyMax <[email protected]>

gitutil: check git bash env when testing

703c765

Signed-off-by: CrazyMax <[email protected]>

crazy-max force-pushed the test-os branch from 8f7dfde to 703c765 Compare January 25, 2024 14:06

crazy-max requested review from tonistiigi, jsternberg, daghack and jedevc January 25, 2024 14:18

crazy-max marked this pull request as ready for review January 25, 2024 14:18

crazy-max commented Jan 25, 2024

View reviewed changes

crazy-max added the area/windows label Jan 25, 2024

crazy-max commented Jan 25, 2024

View reviewed changes

This was referenced Jan 25, 2024

Use Cobra's command.Context(), cancel execution when CLI is signalled #2184

Merged

build: resolve 8.3 filename format to long one on Windows #2209

Closed

tonistiigi approved these changes Jan 26, 2024

View reviewed changes

crazy-max added the needs-cherry-pick/v0.12 label Jan 26, 2024

crazy-max merged commit 8276224 into docker:master Jan 26, 2024
63 checks passed

crazy-max deleted the test-os branch January 26, 2024 08:11

crazy-max added this to the v0.13.0 milestone Jan 26, 2024

This was referenced Jan 26, 2024

vendor: update compose-go to v2.0.0-rc.3 #2205

Merged

vendor: github.com/spf13/cobra v1.8.0 #2215

Merged

crazy-max added the status/needs-follow-up label Jan 26, 2024

crazy-max mentioned this pull request Feb 1, 2024

[v0.12] cherry-picks for v0.12.2 #2233

Closed

crazy-max removed the needs-cherry-pick/v0.12 label Feb 1, 2024

crazy-max mentioned this pull request Feb 2, 2024

vendor: github.com/containerd/console 8f6c4e4 #2238

Merged

crazy-max mentioned this pull request Feb 10, 2024

retrieve root path from FS instances tonistiigi/fsutil#179

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: test-unit job matrix for win/macos/ubuntu #2206

ci: test-unit job matrix for win/macos/ubuntu #2206

crazy-max commented Jan 24, 2024

crazy-max commented Jan 24, 2024

crazy-max commented Jan 25, 2024 •

edited

Loading

crazy-max Jan 25, 2024

crazy-max Jan 25, 2024

crazy-max commented Jan 25, 2024

crazy-max Jan 25, 2024 •

edited

Loading

crazy-max Jan 25, 2024 •

edited

Loading

crazy-max Jan 25, 2024

crazy-max commented Jan 26, 2024

ci: test-unit job matrix for win/macos/ubuntu #2206

ci: test-unit job matrix for win/macos/ubuntu #2206

Conversation

crazy-max commented Jan 24, 2024

crazy-max commented Jan 24, 2024

crazy-max commented Jan 25, 2024 • edited Loading

crazy-max Jan 25, 2024

Choose a reason for hiding this comment

crazy-max Jan 25, 2024

Choose a reason for hiding this comment

crazy-max commented Jan 25, 2024

crazy-max Jan 25, 2024 • edited Loading

Choose a reason for hiding this comment

crazy-max Jan 25, 2024 • edited Loading

Choose a reason for hiding this comment

crazy-max Jan 25, 2024

Choose a reason for hiding this comment

crazy-max commented Jan 26, 2024

crazy-max commented Jan 25, 2024 •

edited

Loading

crazy-max Jan 25, 2024 •

edited

Loading

crazy-max Jan 25, 2024 •

edited

Loading