show stdout/stderr from subprocess calls in failed tests

recipe/meta.yaml

@@ -63,6 +63,8 @@ source:
     - patches/0014-Fix-FindOpenBLAS.patch
     # backport https://github.com/pytorch/pytorch/pull/138095
     - patches/0015-CD-Enable-Python-3.13-on-windows-138095.patch
+    # helps debug test failures that compile custom modules
+    - patches/0016-show-stdout-stderr-from-subprocess-call-in-_run_ninj.patch
 
 build:
   number: {{ build }}

recipe/patches/0001-Force-usage-of-python-3-and-error-without-numpy.patch

@@ -1,7 +1,7 @@
 From f3a0f9aab6dce56eea590b946f60256014b61bf7 Mon Sep 17 00:00:00 2001
 From: Mark Harfouche <[email protected]>
 Date: Sun, 1 Sep 2024 17:35:40 -0400
-Subject: [PATCH 01/15] Force usage of python 3 and error without numpy
+Subject: [PATCH 01/16] Force usage of python 3 and error without numpy
 
 ---
  cmake/Dependencies.cmake | 6 +++---

recipe/patches/0002-Help-find-numpy.patch

@@ -1,7 +1,7 @@
 From 21c30036b5b86f403c0cf4426165d9a6a50edb1a Mon Sep 17 00:00:00 2001
 From: Mark Harfouche <[email protected]>
 Date: Tue, 1 Oct 2024 00:28:40 -0400
-Subject: [PATCH 02/15] Help find numpy
+Subject: [PATCH 02/16] Help find numpy
 
 ---
  tools/setup_helpers/cmake.py | 6 ++++++

recipe/patches/0003-Add-USE_SYSTEM_NVTX-option-138287.patch

@@ -1,7 +1,7 @@
 From d1826af525db41eda5020a1404f5d5521d67a5dc Mon Sep 17 00:00:00 2001
 From: Jeongseok Lee <[email protected]>
 Date: Sat, 19 Oct 2024 04:26:01 +0000
-Subject: [PATCH 03/15] Add USE_SYSTEM_NVTX option (#138287)
+Subject: [PATCH 03/16] Add USE_SYSTEM_NVTX option (#138287)
 
 ## Summary
 

recipe/patches/0004-Update-sympy-version.patch

@@ -1,7 +1,7 @@
 From e3219c5fe8834753b0cf9e92be4d1ef1e874f370 Mon Sep 17 00:00:00 2001
 From: Jeongseok Lee <[email protected]>
 Date: Thu, 17 Oct 2024 15:04:05 -0700
-Subject: [PATCH 04/15] Update sympy version
+Subject: [PATCH 04/16] Update sympy version
 
 ---
  setup.py | 2 +-

recipe/patches/0005-Fix-duplicate-linker-script.patch

@@ -1,7 +1,7 @@
 From 08a1f44fbc81324aa98d720dfb7b87a261923ac2 Mon Sep 17 00:00:00 2001
 From: Jeongseok Lee <[email protected]>
 Date: Sun, 3 Nov 2024 01:12:36 -0700
-Subject: [PATCH 05/15] Fix duplicate linker script
+Subject: [PATCH 05/16] Fix duplicate linker script
 
 ---
  setup.py | 4 +++-

recipe/patches/0006-fix-3.13-pickle-error-in-serialization.py-136034.patch

@@ -1,7 +1,7 @@
 From 15df314a41c69a31c0443254d5552aa1b39d708d Mon Sep 17 00:00:00 2001
 From: William Wen <[email protected]>
 Date: Fri, 13 Sep 2024 13:02:33 -0700
-Subject: [PATCH 06/15] fix 3.13 pickle error in serialization.py (#136034)
+Subject: [PATCH 06/16] fix 3.13 pickle error in serialization.py (#136034)
 
 Error encountered when adding dynamo 3.13 support.
 Pull Request resolved: https://github.com/pytorch/pytorch/pull/136034

recipe/patches/0007-Allow-users-to-overwrite-ld-with-environment-variabl.patch

@@ -1,7 +1,7 @@
 From 655f694854c3eafdd631235b60bc6c1b279218ed Mon Sep 17 00:00:00 2001
 From: Mark Harfouche <[email protected]>
 Date: Thu, 3 Oct 2024 22:49:56 -0400
-Subject: [PATCH 07/15] Allow users to overwrite ld with environment variables
+Subject: [PATCH 07/16] Allow users to overwrite ld with environment variables
 
 This should help in the case of cross compilation.
 

recipe/patches/0008-Allow-libcufile-for-conda-builds.patch

@@ -1,7 +1,7 @@
 From 2578bf017b0453c8eb028a09b7523fb150429eca Mon Sep 17 00:00:00 2001
 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= <[email protected]>
 Date: Fri, 22 Nov 2024 17:50:53 +0100
-Subject: [PATCH 08/15] Allow libcufile for conda builds
+Subject: [PATCH 08/16] Allow libcufile for conda builds
 
 ---
  CMakeLists.txt | 2 +-

recipe/patches/0009-Allow-overriding-CUDA-related-paths.patch

@@ -1,7 +1,7 @@
 From 52e530f222f2d30531c8da889695ac2674964245 Mon Sep 17 00:00:00 2001
 From: =?UTF-8?q?Micha=C5=82=20G=C3=B3rny?= <[email protected]>
 Date: Wed, 27 Nov 2024 13:47:23 +0100
-Subject: [PATCH 09/15] Allow overriding CUDA-related paths
+Subject: [PATCH 09/16] Allow overriding CUDA-related paths
 
 ---
  cmake/Modules/FindCUDAToolkit.cmake | 2 +-

recipe/patches/0010-Fix-test-test_linalg.py-for-NumPy-2-136800.patch

@@ -1,7 +1,7 @@
 From db0902077a0ef68d0a48ae42f3b60f9cff7b2248 Mon Sep 17 00:00:00 2001
 From: Haifeng Jin <[email protected]>
 Date: Tue, 1 Oct 2024 07:53:24 +0000
-Subject: [PATCH 10/15] Fix test/test_linalg.py for NumPy 2 (#136800)
+Subject: [PATCH 10/16] Fix test/test_linalg.py for NumPy 2 (#136800)
 
 Related to  #107302.
 

recipe/patches/0011-Fixes-NumPy-2-test-failures-in-test_torch.py-137740.patch

@@ -1,7 +1,7 @@
 From a1ced1daa37fdcf5345aa0c3095cab8674c1cded Mon Sep 17 00:00:00 2001
 From: Haifeng Jin <[email protected]>
 Date: Sat, 12 Oct 2024 02:40:17 +0000
-Subject: [PATCH 11/15] Fixes NumPy 2 test failures in test_torch.py (#137740)
+Subject: [PATCH 11/16] Fixes NumPy 2 test failures in test_torch.py (#137740)
 
 Related to #107302
 

recipe/patches/0012-Use-BLAS_USE_CBLAS_DOT-for-OpenBLAS-builds.patch

@@ -1,7 +1,7 @@
 From e74a39d09953d4c4dd293e167121cfe83fb172ec Mon Sep 17 00:00:00 2001
 From: Isuru Fernando <[email protected]>
 Date: Wed, 18 Dec 2024 03:59:00 +0000
-Subject: [PATCH 12/15] Use BLAS_USE_CBLAS_DOT for OpenBLAS builds
+Subject: [PATCH 12/16] Use BLAS_USE_CBLAS_DOT for OpenBLAS builds
 
 There are two calling conventions for *dotu functions
 

recipe/patches/0013-fix-issue-142484.patch

@@ -1,7 +1,7 @@
 From 67b122f715e93592f1d7913ab970619f7b571b96 Mon Sep 17 00:00:00 2001
 From: "Zheng, Zhaoqiong" <[email protected]>
 Date: Fri, 27 Dec 2024 13:49:36 +0800
-Subject: [PATCH 13/15] fix issue 142484
+Subject: [PATCH 13/16] fix issue 142484
 
 From https://github.com/pytorch/pytorch/pull/143894
 ---

recipe/patches/0014-Fix-FindOpenBLAS.patch

@@ -1,7 +1,7 @@
 From a6924f47769d3d068b6d8e40686733b7ff32544e Mon Sep 17 00:00:00 2001
 From: Bas Zalmstra <[email protected]>
 Date: Thu, 16 May 2024 10:46:49 +0200
-Subject: [PATCH 14/15] Fix FindOpenBLAS
+Subject: [PATCH 14/16] Fix FindOpenBLAS
 
 ---
  cmake/Modules/FindOpenBLAS.cmake | 15 +++++++++------

recipe/patches/0015-CD-Enable-Python-3.13-on-windows-138095.patch

@@ -1,7 +1,7 @@
 From 66fa234eccfe98339bfa2bba6eae425ce27a037d Mon Sep 17 00:00:00 2001
 From: atalman <[email protected]>
 Date: Tue, 12 Nov 2024 12:28:10 +0000
-Subject: [PATCH 15/15] CD Enable Python 3.13 on windows (#138095)
+Subject: [PATCH 15/16] CD Enable Python 3.13 on windows (#138095)
 
 Adding CD windows. Part of: https://github.com/pytorch/pytorch/issues/130249
 Builder PR landed with smoke test: https://github.com/pytorch/builder/pull/2035

recipe/patches/0016-show-stdout-stderr-from-subprocess-call-in-_run_ninj.patch

@@ -0,0 +1,25 @@
+From cec7ebe3568429c2c73fa8cbba100aac7ecf9496 Mon Sep 17 00:00:00 2001
+From: "H. Vetinari" <[email protected]>
+Date: Thu, 23 Jan 2025 13:34:19 +1100
+Subject: [PATCH 16/16] show stdout/stderr from subprocess call in
+ _run_ninja_build
+
+---
+ torch/utils/cpp_extension.py | 4 ++++
+ 1 file changed, 4 insertions(+)
+
+diff --git a/torch/utils/cpp_extension.py b/torch/utils/cpp_extension.py
+index aaa45ea4c90..8bdb911b0ff 100644
+--- a/torch/utils/cpp_extension.py
++++ b/torch/utils/cpp_extension.py
+@@ -2117,6 +2117,10 @@ def _run_ninja_build(build_directory: str, verbose: bool, error_prefix: str) ->
+         # mypy thinks it's Optional[BaseException] and doesn't narrow
+         if hasattr(error, 'output') and error.output:  # type: ignore[union-attr]
+             message += f": {error.output.decode(*SUBPROCESS_DECODE_ARGS)}"  # type: ignore[union-attr]
++        if e.stdout:
++            message += f"STDOUT:\n{e.stdout}\n"
++        if e.stderr:
++            message += f"STDERR:\n{e.stderr}\n"
+         raise RuntimeError(message) from e
+
+