Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

smoke-test failures: Win32 #258

Open
jkeenan opened this issue Sep 1, 2020 · 13 comments
Open

smoke-test failures: Win32 #258

jkeenan opened this issue Sep 1, 2020 · 13 comments
Assignees
Labels
blocker-to-strict-by-default Blocks completion of Objective 2, strict on by default

Comments

@jkeenan
Copy link
Collaborator

jkeenan commented Sep 1, 2020

We pay for our failure/inability to test on Windows so far.

Smoke test report: http://perl5.test-smoke.org/report/117911
Log file: http://m-l.org/~perl/smoke/perl/mswin32-win2000/smoke-me/James%20E%20Keenan/log005643c4ee4b5d0cd729c01372d3b3fc59752d3a.log.gz

$ zgrep -n -E '(FAILED|Configuration:)' log005643c4ee4b5d0cd729c01372d3b3fc59752d3a.log.gz
12650:[2020-08-31 20:00:33Z] Configuration: -Dusedevel
34095:[2020-08-31 20:56:20Z] ../t/op/magic.t.............................................FAILED
34097:[2020-08-31 20:56:20Z] ../t/win32/runenv.t.........................................FAILED
34099:[2020-08-31 20:56:20Z] ../t/win32/runenv.t.........................................FAILED
34101:[2020-08-31 20:56:20Z] ../t/win32/signal.t.........................................FAILED
34103:[2020-08-31 20:56:20Z] ../t/win32/signal.t.........................................FAILED
34105:[2020-08-31 20:56:20Z] ../t/win32/system.t.........................................FAILED
34107:[2020-08-31 20:56:20Z] ../t/win32/system.t.........................................FAILED
34109:[2020-08-31 20:56:20Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
34111:[2020-08-31 20:56:20Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
34113:[2020-08-31 20:56:20Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
34115:[2020-08-31 20:56:20Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
34117:[2020-08-31 20:56:20Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
34119:[2020-08-31 20:56:20Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
34123:[2020-08-31 20:56:20Z] ../cpan/Win32API-File/t/file.t..............................FAILED
34125:[2020-08-31 20:56:20Z] ../cpan/Win32API-File/t/file.t..............................FAILED
34129:[2020-08-31 20:56:20Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
34131:[2020-08-31 20:56:20Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
34136:[2020-08-31 20:56:20Z] Configuration: -Dusedevel -DDEBUGGING
56310:[2020-08-31 22:38:20Z] ../t/op/magic.t.............................................FAILED
56312:[2020-08-31 22:38:20Z] ../t/win32/runenv.t.........................................FAILED
56314:[2020-08-31 22:38:20Z] ../t/win32/runenv.t.........................................FAILED
56316:[2020-08-31 22:38:20Z] ../t/win32/signal.t.........................................FAILED
56318:[2020-08-31 22:38:20Z] ../t/win32/signal.t.........................................FAILED
56320:[2020-08-31 22:38:20Z] ../t/win32/system.t.........................................FAILED
56322:[2020-08-31 22:38:20Z] ../t/win32/system.t.........................................FAILED
56324:[2020-08-31 22:38:20Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
56326:[2020-08-31 22:38:20Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
56328:[2020-08-31 22:38:20Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
56330:[2020-08-31 22:38:20Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
56332:[2020-08-31 22:38:20Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
56334:[2020-08-31 22:38:20Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
56338:[2020-08-31 22:38:20Z] ../cpan/Win32API-File/t/file.t..............................FAILED
56340:[2020-08-31 22:38:20Z] ../cpan/Win32API-File/t/file.t..............................FAILED
56344:[2020-08-31 22:38:20Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
56346:[2020-08-31 22:38:20Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
56351:[2020-08-31 22:38:20Z] Configuration: -Dusedevel -Duselargefiles
77794:[2020-08-31 23:36:10Z] ../t/op/magic.t.............................................FAILED
77796:[2020-08-31 23:36:10Z] ../t/win32/runenv.t.........................................FAILED
77798:[2020-08-31 23:36:10Z] ../t/win32/runenv.t.........................................FAILED
77800:[2020-08-31 23:36:10Z] ../t/win32/signal.t.........................................FAILED
77802:[2020-08-31 23:36:10Z] ../t/win32/signal.t.........................................FAILED
77804:[2020-08-31 23:36:10Z] ../t/win32/system.t.........................................FAILED
77806:[2020-08-31 23:36:10Z] ../t/win32/system.t.........................................FAILED
77808:[2020-08-31 23:36:10Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
77810:[2020-08-31 23:36:10Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
77812:[2020-08-31 23:36:10Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
77814:[2020-08-31 23:36:10Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
77816:[2020-08-31 23:36:10Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
77818:[2020-08-31 23:36:10Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
77822:[2020-08-31 23:36:10Z] ../cpan/Win32API-File/t/file.t..............................FAILED
77824:[2020-08-31 23:36:10Z] ../cpan/Win32API-File/t/file.t..............................FAILED
77828:[2020-08-31 23:36:10Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
77830:[2020-08-31 23:36:10Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
77835:[2020-08-31 23:36:10Z] Configuration: -Dusedevel -Duselargefiles -DDEBUGGING
100009:[2020-09-01 01:09:58Z] ../t/op/magic.t.............................................FAILED
100011:[2020-09-01 01:09:58Z] ../t/win32/runenv.t.........................................FAILED
100013:[2020-09-01 01:09:58Z] ../t/win32/runenv.t.........................................FAILED
100015:[2020-09-01 01:09:58Z] ../t/win32/signal.t.........................................FAILED
100017:[2020-09-01 01:09:58Z] ../t/win32/signal.t.........................................FAILED
100019:[2020-09-01 01:09:58Z] ../t/win32/system.t.........................................FAILED
100021:[2020-09-01 01:09:58Z] ../t/win32/system.t.........................................FAILED
100023:[2020-09-01 01:09:58Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
100025:[2020-09-01 01:09:58Z] ../cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t.................FAILED
100027:[2020-09-01 01:09:58Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
100029:[2020-09-01 01:09:58Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
100031:[2020-09-01 01:09:58Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
100033:[2020-09-01 01:09:58Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
100037:[2020-09-01 01:09:58Z] ../cpan/Win32API-File/t/file.t..............................FAILED
100039:[2020-09-01 01:09:58Z] ../cpan/Win32API-File/t/file.t..............................FAILED
100043:[2020-09-01 01:09:58Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
100045:[2020-09-01 01:09:58Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
100050:[2020-09-01 01:09:58Z] Configuration: -Dusedevel -Duseithreads
121317:[2020-09-01 02:18:10Z] ../t/op/magic.t.............................................FAILED
121319:[2020-09-01 02:18:10Z] ../t/win32/runenv.t.........................................FAILED
121321:[2020-09-01 02:18:10Z] ../t/win32/runenv.t.........................................FAILED
121323:[2020-09-01 02:18:10Z] ../t/win32/signal.t.........................................FAILED
121325:[2020-09-01 02:18:10Z] ../t/win32/signal.t.........................................FAILED
121327:[2020-09-01 02:18:10Z] ../t/win32/system.t.........................................FAILED
121329:[2020-09-01 02:18:10Z] ../t/win32/system.t.........................................FAILED
121331:[2020-09-01 02:18:10Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
121333:[2020-09-01 02:18:10Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
121335:[2020-09-01 02:18:10Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
121337:[2020-09-01 02:18:10Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
121341:[2020-09-01 02:18:10Z] ../cpan/Win32API-File/t/file.t..............................FAILED
121343:[2020-09-01 02:18:10Z] ../cpan/Win32API-File/t/file.t..............................FAILED
121347:[2020-09-01 02:18:10Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
121349:[2020-09-01 02:18:10Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
121354:[2020-09-01 02:18:10Z] Configuration: -Dusedevel -Duseithreads -DDEBUGGING
143348:[2020-09-01 04:19:40Z] ../t/op/magic.t.............................................FAILED
143350:[2020-09-01 04:19:40Z] ../t/win32/runenv.t.........................................FAILED
143352:[2020-09-01 04:19:40Z] ../t/win32/runenv.t.........................................FAILED
143354:[2020-09-01 04:19:40Z] ../t/win32/signal.t.........................................FAILED
143356:[2020-09-01 04:19:40Z] ../t/win32/signal.t.........................................FAILED
143358:[2020-09-01 04:19:40Z] ../t/win32/system.t.........................................FAILED
143360:[2020-09-01 04:19:40Z] ../t/win32/system.t.........................................FAILED
143362:[2020-09-01 04:19:40Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
143364:[2020-09-01 04:19:40Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
143366:[2020-09-01 04:19:40Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
143368:[2020-09-01 04:19:40Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
143372:[2020-09-01 04:19:40Z] ../cpan/Win32API-File/t/file.t..............................FAILED
143374:[2020-09-01 04:19:40Z] ../cpan/Win32API-File/t/file.t..............................FAILED
143378:[2020-09-01 04:19:40Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
143380:[2020-09-01 04:19:40Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
143385:[2020-09-01 04:19:40Z] Configuration: -Dusedevel -Duseithreads -Duselargefiles
164653:[2020-09-01 05:45:01Z] ../t/op/magic.t.............................................FAILED
164655:[2020-09-01 05:45:01Z] ../t/win32/runenv.t.........................................FAILED
164657:[2020-09-01 05:45:01Z] ../t/win32/runenv.t.........................................FAILED
164659:[2020-09-01 05:45:01Z] ../t/win32/signal.t.........................................FAILED
164661:[2020-09-01 05:45:01Z] ../t/win32/signal.t.........................................FAILED
164663:[2020-09-01 05:45:01Z] ../t/win32/system.t.........................................FAILED
164665:[2020-09-01 05:45:01Z] ../t/win32/system.t.........................................FAILED
164667:[2020-09-01 05:45:01Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
164669:[2020-09-01 05:45:01Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
164671:[2020-09-01 05:45:01Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
164673:[2020-09-01 05:45:01Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
164677:[2020-09-01 05:45:01Z] ../cpan/Win32API-File/t/file.t..............................FAILED
164679:[2020-09-01 05:45:01Z] ../cpan/Win32API-File/t/file.t..............................FAILED
164683:[2020-09-01 05:45:01Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
164685:[2020-09-01 05:45:01Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
164690:[2020-09-01 05:45:01Z] Configuration: -Dusedevel -Duseithreads -Duselargefiles -DDEBUGGING
186685:[2020-09-01 07:35:12Z] ../t/op/magic.t.............................................FAILED
186687:[2020-09-01 07:35:12Z] ../t/win32/runenv.t.........................................FAILED
186689:[2020-09-01 07:35:12Z] ../t/win32/runenv.t.........................................FAILED
186691:[2020-09-01 07:35:12Z] ../t/win32/signal.t.........................................FAILED
186693:[2020-09-01 07:35:12Z] ../t/win32/signal.t.........................................FAILED
186695:[2020-09-01 07:35:12Z] ../t/win32/system.t.........................................FAILED
186697:[2020-09-01 07:35:12Z] ../t/win32/system.t.........................................FAILED
186699:[2020-09-01 07:35:12Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
186701:[2020-09-01 07:35:12Z] ../cpan/ExtUtils-MakeMaker/t/miniperl.t.....................FAILED
186703:[2020-09-01 07:35:12Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
186705:[2020-09-01 07:35:12Z] ../cpan/ExtUtils-MakeMaker/t/MM_Win32.t.....................FAILED
186709:[2020-09-01 07:35:12Z] ../cpan/Win32API-File/t/file.t..............................FAILED
186711:[2020-09-01 07:35:12Z] ../cpan/Win32API-File/t/file.t..............................FAILED
186715:[2020-09-01 07:35:12Z] ../ext/Win32CORE/t/win32core.t..............................FAILED
186717:[2020-09-01 07:35:12Z] ../ext/Win32CORE/t/win32core.t..............................FAILED

A gzipped file containing the relevant part of the log for the non-threaded, non-debugging configuration can be found here.

@atoomic, @toddr please review.

Thank you very much.
Jim Keenan

@jkeenan jkeenan added the blocker-to-strict-by-default Blocks completion of Objective 2, strict on by default label Sep 1, 2020
@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 1, 2020

We pay for our failure/inability to test on Windows so far.

[snip]

A gzipped file containing the relevant part of the log for the non-threaded, non-debugging configuration can be found here.

[snip]

My very preliminary analysis of the failure output from the Win32 smoke-test run suggests that some of the failures are for reasons similar to what we've encountered in our development on *nix.

  • Programs compose and then execute other Perl programs, but the latter are not yet strict-compliant.

  • BEGIN blocks need some standarization and touch-ups to properly locate dependencies.

  • We didn't notice that *.t files were not strict-compliant because they weren't being run on *nix.

Some diagnostic hunts that may be useful.

$ ack 'show-?(argv|shebang)' .
t/win32/system.t
31:my $plxname = "showargv";

t/win32/system_tests
18:my $plxname = "showargv";

t/op/magic.t
309:    my $script = "$wd/show-shebang";
314:	$script = "$wd/show-shebang.bat";
329:      $script = "./show-shebang";
332:      $script = "[]show-shebang";

@toddr toddr self-assigned this Sep 1, 2020
@atoomic
Copy link
Owner

atoomic commented Sep 1, 2020

Poke at it with a few minor fixes
c1fe38f

@toddr
Copy link
Collaborator

toddr commented Sep 1, 2020

This was my result on a interactive run:

Test Summary Report
-------------------
io/openpid.t                                                     (Wstat: 256 Tests: 8 Failed: 4)
  Failed tests:  5-8
  Non-zero exit status: 1
  Parse errors: Bad plan.  You planned 10 tests but ran 8.
op/magic.t                                                       (Wstat: 0 Tests: 192 Failed: 1)
  Failed test:  94
op/taint.t                                                       (Wstat: 0 Tests: 1052 Failed: 2)
  Failed tests:  1, 463
win32/signal.t                                                   (Wstat: 512 Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
win32/system.t                                                   (Wstat: 0 Tests: 1428 Failed: 168)
  Failed tests:  1261-1428
  Parse errors: Bad plan.  You planned 1512 tests but ran 1428.
../cpan/ExtUtils-MakeMaker/t/miniperl.t                          (Wstat: 512 Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
../cpan/ExtUtils-MakeMaker/t/MM_Win32.t                          (Wstat: 256 Tests: 60 Failed: 1)
  Failed test:  8
  Non-zero exit status: 1
../cpan/Module-Metadata/t/extract-version.t                      (Wstat: 0 Tests: 224 Failed: 0)
  TODO passed:   204
../dist/Net-Ping/t/450_service.t                                 (Wstat: 256 Tests: 26 Failed: 1)
  Failed test:  9
  Non-zero exit status: 1
../ext/IPC-Open3/t/IPC-Open3.t                                   (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2678, Tests=1154750, 1989 wallclock secs (73.84 usr +  6.58 sys = 80.42 CPU)
Result: FAIL
gmake: *** [GNUmakefile:1930: test] Error 177

@atoomic
Copy link
Owner

atoomic commented Sep 1, 2020

note that @rem is a windows command, view https://jpsoft.com/help/rem.htm
we cannot use my @rem to fix the issue, the only thing I can think of is using -m-strict when calling perl %0 in a .bat file

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 1, 2020

This was my result on a interactive run:

What do you mean by an "interactive" run?

Test Summary Report
-------------------
io/openpid.t                                                     (Wstat: 256 Tests: 8 Failed: 4)
  Failed tests:  5-8
  Non-zero exit status: 1
  Parse errors: Bad plan.  You planned 10 tests but ran 8.
op/magic.t                                                       (Wstat: 0 Tests: 192 Failed: 1)
  Failed test:  94
op/taint.t                                                       (Wstat: 0 Tests: 1052 Failed: 2)
  Failed tests:  1, 463
win32/signal.t                                                   (Wstat: 512 Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
win32/system.t                                                   (Wstat: 0 Tests: 1428 Failed: 168)
  Failed tests:  1261-1428
  Parse errors: Bad plan.  You planned 1512 tests but ran 1428.
../cpan/ExtUtils-MakeMaker/t/miniperl.t                          (Wstat: 512 Tests: 0 Failed: 0)
  Non-zero exit status: 2
  Parse errors: No plan found in TAP output
../cpan/ExtUtils-MakeMaker/t/MM_Win32.t                          (Wstat: 256 Tests: 60 Failed: 1)
  Failed test:  8
  Non-zero exit status: 1
../cpan/Module-Metadata/t/extract-version.t                      (Wstat: 0 Tests: 224 Failed: 0)
  TODO passed:   204
../dist/Net-Ping/t/450_service.t                                 (Wstat: 256 Tests: 26 Failed: 1)
  Failed test:  9
  Non-zero exit status: 1
../ext/IPC-Open3/t/IPC-Open3.t                                   (Wstat: 0 Tests: 45 Failed: 0)
  TODO passed:   25
Files=2678, Tests=1154750, 1989 wallclock secs (73.84 usr +  6.58 sys = 80.42 CPU)
Result: FAIL
gmake: *** [GNUmakefile:1930: test] Error 177

If you can post verbose output for any of those, I can stare at them overnight.

Thanks for your investigation.
jimk

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 3, 2020

@toddr did a lot of work which went into #259 and which has just been merged into the current target branch. Let's see what smoke-testers have to say about this.

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 4, 2020

@toddr did a lot of work which went into #259 and which has just been merged into the current target branch. Let's see what smoke-testers have to say about this.

Results look good. See attachment.

greer-win32-20200904.txt

The bottom report is what I believe is the most recent smoke-test run on Perl 5 blead run on this OS/OS version/compiler/compiler version combination.

The middle report is the next-to-most-recent smoke-test run on the simulation of Perl 7 branch run on this host. cpan/ExtUtils-MakeMaker/t/02-xsdynamic.t and dist/Net-Ping/t/450_service.t are frequently observed failures on this host, but all the other are attributable to our previous lack of testing on Win32.

The top report is the most recent smoke-test report on the Perl 7 simulation branch. Note that now the only files failing are ones also always or intermittently failing on Perl 5 blead.

We, like Perl 5, are handicapped by the fact that this host is the only one steadily submitting smoke-test reports on MSWin32.

Thank you very much.
Jim Keenan

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 5, 2020 via email

@atoomic
Copy link
Owner

atoomic commented Sep 14, 2020

@jkeenan can we close this case or are we viewing more failures on windows?

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 14, 2020

@jkeenan can we close this case or are we viewing more failures on windows?

@atoomic I suspect we aren't going to get any more smoke-test reports for this particular Merge Candidate tag. I'd like to do that after we resolve #267.

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 14, 2020 via email

@jkeenan
Copy link
Collaborator Author

jkeenan commented Sep 14, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker-to-strict-by-default Blocks completion of Objective 2, strict on by default
Projects
None yet
Development

No branches or pull requests

3 participants