Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct handling of MAP_FULL exception #73

Open
kk00ss opened this issue Jul 22, 2016 · 2 comments
Open

Correct handling of MAP_FULL exception #73

kk00ss opened this issue Jul 22, 2016 · 2 comments

Comments

@kk00ss
Copy link

kk00ss commented Jul 22, 2016

I've set initial size to 4Gb, and when I attempt to reopen environment I get exception which for some reason is null, and after restart I see
Exception in thread "main" java.lang.ExceptionInInitializerError at rhinodog.Run.main.main(main.scala) Caused by: org.fusesource.lmdbjni.LMDBException: MDB_INVALID: File is not an LMDB file
and cannot reopen LMDB.
I'm blocking all the read threads that are trying to access Environment.

        this.environment = new Env()
        logger.debug("openEnvironment mapSize = {}", newSize)
        environment.setMapSize(newSize)
        environment.setMaxDbs(numberOfDatabases)
        //TODO: VERIFY ASSUMPTION - Constants.NORDAHEAD hurts single threaded performance
        //TODO: - improves multithreaded theoretically
        val flag = if (this.storageMode == storageModeEnum.READ_ONLY) Constants.RDONLY else 0
        environment.open(newDir.getPath, flag) // EXCEPTION == NULL IS THROWN HERE
        this.postingsDB = environment.openDatabase("postings")
        this.metadataDB = environment.openDatabase("metadata")
        this.documentsDB = environment.openDatabase("documents")
        this.numberOfDeletedDB = environment.openDatabase("numberOfDeletedByBlock")
        this.roaringBitmapsDB = environment.openDatabase("roaringBitmaps")
        this.term2ID_DB = environment.openDatabase("term2ID_DB")
        this.ID2Term_DB = environment.openDatabase("ID2Term_DB")
@kk00ss
Copy link
Author

kk00ss commented Jul 22, 2016

New detail - exception at reopening is org.fusesource.lmdbjni.LMDBException: MDB_INVALID: File is not an LMDB file
Sorta fixed by changing the original MAP_SIZE and the step of size increase to 1GB. My code for increasing MAP_SIZE works until it reaches 5GB and fails miserably. I cannot even open the database after that.

@krisskross
Copy link
Member

Could be a LMDB bug, have you tried the same in C?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants