Malformed UTF-8 character errors when accessing Mojo::UserAgent DOM with binary response

[tim-2]

Hi, when you download an URL with Mojo::UserAgent and the response is binary e.g. image or pdf and you access the dom, I (sometimes) get many "Malformed UTF-8 character" errors:

UTF-8 "\xA1" does not map to Unicode at /usr/local/share/perl/5.36.0/Mojo/Util.pm line 122.
Malformed UTF-8 character: \xde\xed (unexpected non-continuation byte 0xed, immediately after start byte 0xde; need 2 bytes, got 1) in reverse at /usr/local/share/perl/5.36.0/Mojo/Util.pm line 459.

Minimal example to reproduce:

use Mojo::UserAgent;
my $ua = Mojo::UserAgent->new;
my $dom = $ua->get('https://www.google.de/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png')->result->dom;
die;

I think it is because $@ is not localized in eval in Mojo::Util.

Of course I could check the content-type before accessing the dom, but do you think we could also catch these errors in Mojo::Util itself?

[Grinnz]

dom is only valid on text responses, and assumes a character encoding of UTF-8 by default as any other access to ->text does, so attempting to access it will of course cause errors when called on non-text responses. Mojolicious chooses to avoid checking for every possible thing a user may do (for performance and complexity reasons), instead you should only call methods in cases that make sense.