pbrt_exe (Release) crashing

[kyamant]

This is in addition to the issues reported in #429 !
Build fails for VS 17.11.2 with CUDA 12.6, --allow-unsupported-compiler does not help.
Using platform toolset Vısual Studio 2019 and CUDA 11.7 build succeeds.
Debug runs OK with arguments: --gpu --display-server localhost:14160 D:\Personal\Projects\ProjPhysicallyBasedRendering\pbrt-v4-scenes\killeroos\killeroo-simple.pbrt --spp 256
Release fails with the same arguments:

[ tid 78100 @ 0.000s D:\Personal\Projects\ProjPhysicallyBasedRendering\pbrt-v4\src\pbrt\gpu\aggregate.cpp:1026 ] ERROR OptiX: COMPILER: COMPILE ERROR: Invalid PTX input: ptx2llvm-module-001: error: Failed to parse input PTX string
ptx2llvm-module-001, line 94161; fatal : Parsing error near ': syntax error
Cannot parse input PTX string

[ tid 78100 @ 0.000s D:\Personal\Projects\ProjPhysicallyBasedRendering\pbrt-v4\src\pbrt\gpu\aggregate.cpp:1105 ] FATAL OptiX call OPTIX_MODULE_CREATE_FN( optixContext, &moduleCompileOptions, &pipelineCompileOptions, ptx, strlen(ptx), log, &logSize, &optixModule ) failed with code 7200: "Invalid input"
Logs: COMPILE ERROR: Invalid PTX input: ptx2llvm-module-001: error: Failed to parse input PTX string
ptx2llvm-module-001, line 94161; fatal : Parsing error near ': syntax error
Cannot parse input PTX string

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00010D60000B6C80 - ►{

(unknown ) 0x00007FFC21142560 - BaseThreadInitThunk
(unknown ) 0x00007FFC22F4AF00 - RtlUserThreadStart

[NicNel]

windows 10, cuda 12.6 Update 1, VS 2022 Community with latest update, optix 7.3/8.0, gpu:GTX 1070(sm_60) - compiling without errors, render goes normal, but output image is completely black

[vincentsiu]

@NicNel Have you tried this?

#428

[NicNel]

@vincentsiu, thank you for pointing to the post, I missed it. Compilation and rendering are now working for me with setup:

Windows 10
GeForce GTX 1070
Optix 8.0.0
Cuda 12.6.68
VS 2022 with MSVC v143 Buildtools (v14.39 -17.9)

But optix denoising fails with the message:

pbrt-v4\src\pbrt\gpu\denoiser.cpp:62 ] FATAL OptiX call optixDenoiserComputeMemoryResources(denoiserHandle, resolution.x, resolution.y, &memorySizes) failed with code 7001: "Invalid value"
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x0001918000010EA0 - ░Ц☺
(unknown ) 0x00007FFD13327360 - BaseThreadInitThunk
(unknown ) 0x00007FFD1433CC70 - RtlUserThreadStart

@kyamant, it seems that there is some problem with the light source in the killeroo-simple scene. I changed it to infinite light and scene rendering was done without crashing

[kyamant]

@NicNel : I just pulled from [email protected]:mmp/pbrt-v4-scenes.git. Did not pick up a new killeroo-simple. Why would it cause to fail with release but not with debug version anyway?

[NicNel]

@kyamant, I didn't change the scene file in the repository. I don't know why this happens, maybe there are some bugs. Try to replace text in killeroo-simple.pbrt file with this:

LookAt 400 20 30
    0 63 -110
    0 0 1
Rotate -5 0 0 1
Camera "perspective"
    "float fov" [ 39 ]
# zoom in by feet
# "integer xresolution" [1500] "integer yresolution" [1500]
#	"float cropwindow" [ .34 .49  .67 .8 ]
Film "rgb"
    "string filename" [ "killeroo-simple.exr" ]
    "integer yresolution" [ 700 ]
    "integer xresolution" [ 700 ]
Sampler "halton"
    "integer pixelsamples" [ 256 ]

WorldBegin

AttributeBegin
    Rotate 0.0 1 0 0
    Rotate 0.0 0 1 0
    Rotate 0.0 0 0 1
    LightSource "infinite"
        "rgb L" [ 1.0 1.0 1.0 ]
        "float scale" [ 1.0 ]
AttributeEnd

AttributeBegin
    Material "diffuse"
        "rgb reflectance" [ 0.5 0.5 0.8 ]
    Translate 0 0 -140
    Shape "trianglemesh"
        "point2 uv" [ 0 0 5 0 5 5 0 5 
            ]
        "integer indices" [ 0 1 2 2 3 0 ]
        "point3 P" [ -1000 -1000 0 1000 -1000 0 1000 1000 0 -1000 1000 0 ]
    Shape "trianglemesh"
        "point2 uv" [ 0 0 5 0 5 5 0 5 
            ]
        "integer indices" [ 0 1 2 2 3 0 ]
        "point3 P" [ -400 -1000 -1000 -400 1000 -1000 -400 1000 1000 -400 -1000 1000 ]
AttributeEnd

AttributeBegin
    Scale 0.5 0.5 0.5
    Rotate -60 0 0 1
    Material "coateddiffuse"
        "float roughness" [ 0.025 ]
        "rgb reflectance" [ 0.4 0.2 0.2 ]
    Translate 100 200 -140
    Include "geometry/killeroo.pbrt"
    Material "coateddiffuse"
        "float roughness" [ 0.15 ]
        "rgb reflectance" [ 0.4 0.5 0.4 ]
    Translate -200 0 0
    Include "geometry/killeroo.pbrt"
AttributeEnd

[kyamant]

@NicNel : Your fix did not solve anything. I am including my original text. The problem is with Optix compilation! not fixable by your scene file!

This is in addition to the issues reported in #429 ! Build fails for VS 17.11.2 with CUDA 12.6, --allow-unsupported-compiler does not help. Using platform toolset Vısual Studio 2019 and CUDA 11.7 build succeeds.
Debug runs OK with arguments: --gpu --display-server localhost:14160 D:\Personal\Projects\ProjPhysicallyBasedRendering\pbrt-v4-scenes\killeroos\killeroo-simple.pbrt --spp 256
Release fails with the same arguments:

[ tid 78100 @ 0.000s D:\Personal\Projects\ProjPhysicallyBasedRendering\pbrt-v4\src\pbrt\gpu\aggregate.cpp:1026 ] ERROR OptiX: COMPILER: COMPILE ERROR: Invalid PTX input: ptx2llvm-module-001: error: Failed to parse input PTX string
ptx2llvm-module-001, line 94161; fatal : Parsing error near ': syntax error
Cannot parse input PTX string
[ tid 78100 @ 0.000s D:\Personal\Projects\ProjPhysicallyBasedRendering\pbrt-v4\src\pbrt\gpu\aggregate.cpp:1105 ] FATAL OptiX call OPTIX_MODULE_CREATE_FN( optixContext, &moduleCompileOptions, &pipelineCompileOptions, ptx, strlen(ptx), log, &logSize, &optixModule ) failed with code 7200: "Invalid input"
Logs: COMPILE ERROR: Invalid PTX input: ptx2llvm-module-001: error: Failed to parse input PTX string
ptx2llvm-module-001, line 94161; fatal : Parsing error near ': syntax error
Cannot parse input PTX string
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00010D60000B6C80 - ►{
(unknown ) 0x00007FFC21142560 - BaseThreadInitThunk
(unknown ) 0x00007FFC22F4AF00 - RtlUserThreadStart

[NicNel]

@kyamant, ok, adding extra gpu synchronization inside render loop fix issue for me. Try to add this code in pbrt-v4/src/pbrt/wavefront/integrator.cpp and rebuild the solution

#ifdef PBRT_BUILD_GPU_RENDERER
        if (Options->useGPU && !gui)
            GPUWait();
#endif

[kyamant]

@NicNel : The fix you've provided made the problem go away. Could you make your fix official please?

[NicNel]

@mmp, what do you think about this fix, is it acceptable? Can I or someone else make a pull request?

[mmp]

Is there any theory about what this is fixing why is it necessary? If it's just a random change that makes the bug go away then it doesn't make sense to merge it without understanding what the bug actually is.

(I am also surprised that it would fix a bug with PTX parsing.)

[NicNel]

On my side, crashing goes with the message:

pbrt-v4\src\pbrt\gpu\util.cpp:148 ] FATAL CUDA error: invalid argument
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x0001918000010EA0 - ░Ц☺
(unknown                                 )      0x00007FFEDDA37360 - BaseThreadInitThunk
(unknown                                 )      0x00007FFEDF97CC70 - RtlUserThreadStart

In util.cpp line 148: CUDA_CHECK(cudaEventSynchronize(start));
Crashing does not appear if --interactive rendering option is used.
The theory is that between samples calculation in render loop cpu doesn't wait while gpu finishes the current sample and goes to the next one, which leads to incorrect memory access.

[kyamant]

On my machine (RTX 3070ti) the Release version crashes but Debug works fine; which may point to a speed/synchronization issue.