@pponnuvel For the files that import FileSearcher from hotsos.core.search, i.e. from hotsos.core.search import (FileSearcher), we can pass decode_errors directly to the super().__init__ :

@pponnuvel For the files that import FileSearcher from hotsos.core.search, i.e. from hotsos.core.search import (FileSearcher), we can pass decode_errors directly to the super().__init__ :

hotsos/hotsos/core/search.py

Lines 24 to 27 in 4121edf

	super().__init__(*args,
	max_parallel_tasks=HotSOSConfig.max_parallel_tasks,
	max_logrotate_depth=max_logrotate_depth,
	**kwargs)

Ah, I didn't realise there's another FileSearcher in hotsos itself! So that would simplify this change. But is there any situation where we do want to see the decode errors and/or handle it? I don't envisage any - just wonder as this woud prevent such things.

@dosaboy pponnuvel@97b7d05 is the other way. But I've a question.

I had had issues in the past when using decode error handlers since the injected characters can yield unexpected results from the constraints logic. Having said that I think with the current implementation is reasonably safe. I have however in recent times very seldom comes across this kind of error, would you be able to share anything about what file/output it was that failed? I wonder if a more selective approach would be better i.e. not doing this globally.

@dosaboy
Changing these two is sufficient for the specific sosreport I've issues with:

 hotsos/core/plugins/kernel/kernlog/common.py
 hotsos/core/plugins/lxd/common.py           

So we can limit to just to these two too.

As a "safer" alternative what do you think about putting this behind a config option and making it optional e.g. --handle-utf8-decode-errors. Overkill?

As a "safer" alternative what do you think about putting this behind a config option and making it optional e.g. --handle-utf8-decode-errors. Overkill?

That's an option. But you need to know it first to ignore it. What you'd see is "failed-parts" in hotsos output which could be due to anything - not necessarily because "decode-errors".

Another question is what could be "unsafe" if we ignore it globally. Do we expect only UTF-8 and ignore anything else? or do we want to support multiple encodings?

It depends on what we could do when we hit decode errors . If the answer is "nothing as we only "grep" for ASCII text in all log/config files, so ignore it" then I'd prefer the global change (pponnuvel@97b7d05).

@dosaboy Can we get this in? I hit this with every sosreport from a user...