"make testios" fails with Xcode 16+ due to changes in xcresulttool #126925

ned-deily · 2024-11-17T06:51:46Z

As of Xcode 16, Apple has changed the command interface to xcresultool such that the way it is invoked when running the testios recipe in Makefile.pre.in is no longer valid, causing the iOS test step to fail while attempting to extract the results of the test run.

[...]
# Regardless of success or failure, extract and print the test output
xcrun xcresulttool get --path iOSTestbed.arm64-iphonesimulator.1731824398/arm64-iphonesimulator.xcresult \
		--id $( xcrun xcresulttool get --path iOSTestbed.arm64-iphonesimulator.1731824398/arm64-iphonesimulator.xcresult --format json | _PYTHON_PROJECT_BASE=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator _PYTHON_HOST_PLATFORM=ios-13.0-arm64-iphonesimulator PYTHONPATH=../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__ios_arm64-iphonesimulator _PYTHON_SYSCONFIGDATA_PATH=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator/build/lib.ios-13.0-arm64-iphonesimulator-3.14 /Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-darwin/root-arm64-apple-darwin/bin/python3 -c "import sys, json; result = json.load(sys.stdin); print(result['actions']['_values'][0]['actionResult']['logRef']['id']['_value'])" ) \
		--format json | \
		_PYTHON_PROJECT_BASE=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator _PYTHON_HOST_PLATFORM=ios-13.0-arm64-iphonesimulator PYTHONPATH=../Lib _PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata__ios_arm64-iphonesimulator _PYTHON_SYSCONFIGDATA_PATH=/Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-ios-simulator/build/lib.ios-13.0-arm64-iphonesimulator-3.14 /Users/nad/Projects/PyDev/active/dev/3x/source/build-arm64-apple-darwin/root-arm64-apple-darwin/bin/python3 -c "import sys, json; result = json.load(sys.stdin); print(result['subsections']['_values'][1]['subsections']['_values'][0]['emittedOutput']['_value'])"
Error: This command is deprecated and will be removed in a future release, --legacy flag is required to use it.
Usage: xcresulttool get object [--legacy] --path <path> [--id <id>] [--version <version>] [--format <format>]
  See 'xcresulttool get object --help' for more information.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import sys, json; result = json.load(sys.stdin); print(result['actions']['_values'][0]['actionResult']['logRef']['id']['_value'])
                               ~~~~~~~~~^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 293, in load
    return loads(fp.read(),
        cls=cls, object_hook=object_hook,
        parse_float=parse_float, parse_int=parse_int,
        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Error: Missing value for '--id <id>'
Help:  --id <id>  The ID of the object [optional, assumes rootID if not specified].
Usage: xcresulttool get object [--legacy] --path <path> [--id <id>] [--version <version>] [--format <format>]
  See 'xcresulttool get object --help' for more information.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import sys, json; result = json.load(sys.stdin); print(result['subsections']['_values'][1]['subsections']['_values'][0]['emittedOutput']['_value'])
                               ~~~~~~~~~^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 293, in load
    return loads(fp.read(),
        cls=cls, object_hook=object_hook,
        parse_float=parse_float, parse_int=parse_int,
        parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 345, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nad/Projects/PyDev/active/dev/3x/source/Lib/json/decoder.py", line 363, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
make: *** [testios] Error 1

Linked PRs

The text was updated successfully, but these errors were encountered:

freakboy3742 · 2024-11-18T01:16:07Z

See #126821 for related issue/feature. Adding stdout/stderr redirection will enable real-time log streaming, which means we don't need to extract stdout/err from the XCresult file, and will also allow progress tracking of a running test suite.

Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite.

…honGH-127592) Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite. (cherry picked from commit 2041a95) Co-authored-by: Russell Keith-Magee <[email protected]>

#127754) Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite. (cherry picked from commit 2041a95)

…onGH-127592) (python#127754) Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite. (cherry picked from commit 2041a95)

…honGH-127592) (python#127754) Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite. (cherry picked from commit 2041a95)

) Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite.

…honGH-127592) (python#127754) Adds a `use_system_log` config item to enable stdout/stderr redirection for Apple platforms. This log streaming is then used by a new iOS test runner script, allowing the display of test suite output at runtime. The iOS test runner script can be used by any Python project, not just the CPython test suite. (cherry picked from commit 2041a95)

ned-deily added OS-ios needs backport to 3.13 bugs and security fixes labels Nov 17, 2024

ned-deily assigned freakboy3742 Nov 17, 2024

ned-deily mentioned this issue Nov 17, 2024

gh-126167: Modify iOS Testbed to read arguments from Info.plist #126169

Merged

Eclips4 added 3.13 bugs and security fixes 3.14 new features, bugs and security fixes and removed needs backport to 3.13 bugs and security fixes labels Nov 17, 2024

freakboy3742 mentioned this issue Nov 18, 2024

Add option for redirecting stdout and stderr to the system log #126821

Closed

github-actions bot mentioned this issue Dec 1, 2024

Monthly issue metrics report hugovk/test#88

Closed

freakboy3742 mentioned this issue Dec 4, 2024

gh-126925: Modify how iOS test results are gathered #127592

Merged

picnixz added tests Tests in the Lib/test dir build The build process and cross-build labels Dec 5, 2024

freakboy3742 closed this as completed in #127592 Dec 9, 2024

bedevere-app bot mentioned this issue Dec 9, 2024

[3.13] gh-126925: Modify how iOS test results are gathered (GH-127592) #127754

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

"make testios" fails with Xcode 16+ due to changes in xcresulttool #126925

"make testios" fails with Xcode 16+ due to changes in xcresulttool #126925

ned-deily commented Nov 17, 2024 •

edited by bedevere-app bot

Loading

freakboy3742 commented Nov 18, 2024

"make testios" fails with Xcode 16+ due to changes in xcresulttool #126925

"make testios" fails with Xcode 16+ due to changes in xcresulttool #126925

Comments

ned-deily commented Nov 17, 2024 • edited by bedevere-app bot Loading

Linked PRs

freakboy3742 commented Nov 18, 2024

ned-deily commented Nov 17, 2024 •

edited by bedevere-app bot

Loading