Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large XMLDocument: Print( XMLPrinter* ) throws std::bad_alloc, XMLDocument::SaveFile does not. #932

Open
RL-S opened this issue Jan 19, 2023 · 2 comments

Comments

@RL-S
Copy link

RL-S commented Jan 19, 2023

I have a large XML document (ca. 1.9GB on disk).
Using XMLDocument::Print to write this document to a file always causes a std::bad_alloc to be thrown:

namespace xml = tinyxml2;
void toStream( const xml::XMLDocument* doc, std::ostream& os ){
	xml::XMLPrinter streamer;
	doc->Print( &streamer );
	os << streamer.CStr();
}

The exception was thrown before XMLDocument::Print returned.

On the other hand, using XMLDocument::SaveFile did not throw:

namespace xml = tinyxml2;
namespace fs = std::filesystem;
void toFile( xml::XMLDocument* doc, const fs::path& filePath ){
	xml::XMLError ret { doc->SaveFile( filePath.c_str() ) };
	if ( ret != xml::XML_SUCCESS ){
		// error handling here
	}
}

Maybe it would make sense to unify these functions.

@cugone
Copy link
Contributor

cugone commented Jan 20, 2023

Out of curiosity are you compiling for x86, i.e. 32-bit, targets? It's my understanding that processes run in 32-bit mode are only allowed 2 GB of memory. 1.9 GB is really close to that limit and if you were to have a lot of heap allocated memory elsewhere you can easily go over that limit and the program can't allocate any more.

Since XMLPrinter works entirely in-memory it would need to take up 1.9 GB of memory. Hence, the bad_alloc throw.

@RL-S
Copy link
Author

RL-S commented Jan 20, 2023

Output of file command for my tinyxml2.so:
ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, BuildID[sha1]=..., not stripped

It's 64-bit. Same goes for the whole project, which definitely can use more than 2GB of memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants