Problem with hardcoded typo3conf/ext path inside bootstrap.php when using composer-installers >= version 4

[ktallafus]

Bug Report

Current Behavior
When running "vendor/bin/typo3 crawler:processQueue" I get this error:
PHP Warning: include(/var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php/index.php): Failed to open stream: No such file or directory in /var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php on line 84

This prevents the indexer from indexing content.

Expected behavior/output
The crawler/indexer should run without this error.

Steps to reproduce
Use typo3/cms-composer-installers >= 4 (eg. v4.0.0-RC1), so that EXT:crawler is installed into vendor folder
run vendor/bin/typo3 crawler:buildQueue
run vendor/bin/typo3 crawler:processQueue

Environment

• Crawler version(s): 11.0.7
• TYPO3 version(s): 11.5.30
• Is your TYPO3 installation set up with Composer (Composer Mode): YES

Possible Solution
The error seems to originate from this line:
https://github.com/tomasnorre/crawler/blob/main/cli/bootstrap.php#L24

In our installation, extensions are installed directly inside vendor folder (using typo3/cms-composer-installers:v4.0.0-RC1). typo3conf/ext does not even exist. The bootstrap script should take that into account.

[github-actions]

Hi there, thank you for taking your time to create your first issue. Please give us a bit of time to review it.

[tomasnorre]

This is very much related, not a duplicate, but both issues should be into consideration when creating a solution.

#932

[tomasnorre]

Hi @ktallafus

I have looked a little into this, to see if I can create some sort of quick fix, that doesn't break anything.

The patch below works on my system with compose, and default vendor-dir and web-dir [1].
The problem is the second one starts to customize vendor-dir or web-dir, or not using composer, then it's not working anymore.

If you have any suggestion on how to fix this. We don't have the DI present, so using the ExtensionManagementUtility::extPath('crawler') is also not an option.

I already have a function for getting the bin-dir from composer.json, https://github.com/tomasnorre/crawler/blob/main/Classes/Service/ProcessService.php#L93, but would prefer not to have duplicate code in the bootstrap.php

Diff

}
 
 $typo3Root = preg_replace('#typo3conf/ext/crawler/cli/bootstrap.php$#', '', $tempPathThisScript);
+$typo3Root = preg_replace('#vendor/tomasnorre/crawler/cli/bootstrap.php$#', '', $typo3Root);
 
 /**
  * Second parameter is a base64 encoded serialized array of header data
@@ -81,7 +81,7 @@
 }
 
 chdir($typo3Root);
-include($typo3Root . '/index.php');
+include($typo3Root . '/public/index.php');

1. https://github.com/TYPO3/CmsComposerInstallers?tab=readme-ov-file#web-dir

[tomasnorre]

I just got this error locally when running with direct request.

-- error
Warning: chdir(): Not a directory (errno 20) in /var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php on line 83 Warning: include(/var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php/index.php): Failed to open stream: No such file or directory in /var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php on line 84 Warning: include(): Failed opening '/var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php/index.php' for inclusion (include_path='.:/usr/share/php') in /var/www/html/vendor/tomasnorre/crawler/cli/bootstrap.php on line 84

which is of cause an error, as bootstrap.php/index.php cannot be valid.

[tomasnorre]

Workaround

If you are not limited to direct request, the crawler will still work with direct requests disabled.

I know it's not a proper solution, but at least it's a temporary fix.