Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JEF Monitor causes crawl failures on Windows #9

Open
essiembre opened this issue Sep 8, 2018 · 2 comments
Open

JEF Monitor causes crawl failures on Windows #9

essiembre opened this issue Sep 8, 2018 · 2 comments
Labels

Comments

@essiembre
Copy link
Contributor

From collector-http created by rustyx : Norconex/crawlers#519

Due to file locking, a running JEF Monitor causes intermittent crawler start failures with an exception like this:

java.io.IOException: Could not move "D:\work\logs\latest\logs\test1.log" to "D:\work\logs\backup\2018\09\05\logs\201809052154420193__test1.log".
        at com.norconex.commons.lang.file.FileUtil.moveFile(FileUtil.java:179)
        at com.norconex.jef4.log.FileLogManager.backup(FileLogManager.java:186)
        at com.norconex.jef4.suite.JobSuite.backupSuite(JobSuite.java:543)
        at com.norconex.jef4.suite.JobSuite.initialize(JobSuite.java:473)
        at com.norconex.jef4.suite.JobSuite.doExecute(JobSuite.java:277)
        at com.norconex.jef4.suite.JobSuite.execute(JobSuite.java:168)
        at com.norconex.collector.core.AbstractCollector.start(AbstractCollector.java:131)
        at com.norconex.collector.core.AbstractCollectorLauncher.launch(AbstractCollectorLauncher.java:95)
        at com.norconex.collector.http.HttpCollector.main(HttpCollector.java:74)

Windows error message is: The process cannot access the file because it is being used by another process

@truezjz
Copy link

truezjz commented Jun 24, 2019

Hi Pascal, for this issue, instead of using file logs, using RDB to log will be a solution, just like ManifoldCF did.

@essiembre
Copy link
Contributor Author

Something to consider for sure. Right now, it is possible to write your own LogManager to store logs however you want. Probably not the easiest. The next major version will use SLF4J and you will be able to plug your own logging implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants