Not building for android

[recoilme]

jni build succesfully, but .so files - not(
I run mvn install -P android with no luck
Error in logs:
checking whether the C compiler works... no
[INFO] configure: error: in /Users/recoilme/asp/lmdbjni/lmdbjni-android/target/native-build': [INFO] configure: error: C compiler cannot create executables [INFO] Seeconfig.log' for more details

In config.log - wrong directory name

• android-ndk-r10e/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64
  linux-x86_64 is missing in ndk.
  I try rename darwin-x86_64 to linux-x86_64 with no luck. Now errors command line options not supported and so on.

May you provide some information how to build for android?

[krisskross]

I built it on Linux last time. You may need to adjust the android pom with the correct --host option. Also you need to set the NDK path variable.

There is also a release in maven central, if that works for you.

<dependency>
    <groupId>org.deephacks.lmdbjni</groupId>
    <artifactId>lmdbjni-android</artifactId>
    <version>0.4.0</version>
</dependency>

[recoilme]

Thx for the answer. So i add in gradle maven dependency:

compile group: 'org.deephacks.lmdbjni', name: 'lmdbjni-android', version: '0.4.0'

And project build well, but i have error native library missing at run:

Could not load library. Reasons: [...,nativeLibraryDirectories=[/data/app/ru.test.test-2/lib/arm, /vendor/lib, /system/lib]]] couldn't find "liblmdbjni-1-0.0.so",...couldn't find "liblmdbjni.so"]

May you provide prebuild version liblmdbjni.so and may be somethink like libmdb.so will needed too?

[recoilme]

I try reproduce step by step how i try to build:
git clone https://github.com/deephacks/lmdbjni.git
cd lmdbjni/
mvn package (success)
cd lmdbjni-android/
export NDK=/Users/recoilme/android-ndk-r10e
mvn install -P android
-- configure: error: C compiler cannot create executables
See `config.log' for more details
config.log:
target='x86_64-unknown-linux-gnu'
target_alias=''
target_cpu='x86_64'
target_os='linux-gnu'
target_vendor='unknown'

For project https://github.com/deephacks/lmdb - this steps work well - and i have liblmdb.so and liblmdb.a)

[recoilme]

Ops, i extract *so and put manualy and now all works good, thank you

I create repo with builded lmdb jni - for simple test lmdb vs leveldb vs filesystem here:
https://github.com/recoilme/testLevelDb

Just put & get some byte array to db 50 times in one circle:

    for (int i = 0; i < n; i++) {
        lmdb.put((""+i).getBytes(), byteArray);
        byte[] bytes = lmdb.get((""+i).getBytes());
    }

It's work, and better then Level db, but perfomance not so good as i expected(

File (write, read)*50 : 1620ms
LevelDB (get, put)*50 : 6723ms
Lmdb (get, put)*50 : 3919ms

I expect what its must work like memory, but it work like filesystem^^
Please, may you give me right direction how tune speed on?

Source test: https://github.com/recoilme/testLevelDb/blob/master/app/src/main/java/ru/test/test/MainActivity.java

[krisskross]

There are a number of thing you can do to increase write speed.

• If keys are sorted you can use the APPEND mode. Be careful, if keys are unsorted you can corrupt the database.
• Don't fsync metapage after commit using NOMETASYNC.
• Use asynchronous msync MAPASYNC.
• Don't fsync after commit using NOSYNC.

All of these options are available in the Constants class.

[recoilme]

Thank you, Kristoffer

With this options speeds is more close to file system and, sometime, faster >>

File open, (write, read)*5 : 126ms
LevelDB (get, put)*5 : 875ms
Lmdb (put, get)*5 : 95ms

And one more question, pls: what do you think about try build lmdb specialy for android with this commit:
https://github.com/recoilme/LMDB_0.9.15_32x/commit/6fc390dec8580aa5f49f9cf704870ca247f5682d
If i understand right, with this commit direct buffer must work on android, or all not so easy?

Right now i dont see some effective and simple key/value storage for byte arrays with good speed and small overhead, and lmdb looks very promising, but we need more speed on mobile devices. I want try this, but have no luck with build from source for android...

[krisskross]

It would be very nice to have direct buffer support on Android. We tried to get it working before but failed, see #21. I must admit that neither my android nor c skills is good enough to make a fair judgement.

Have you tried direct buffers with this commit?

[recoilme]

Sorry^ i am so lame and dont may build from source (i extract *so from jar from maven and put manualy) but i will be happy to do some tests if you may try build with this commit and give me *so

[krisskross]

The clone you're referring to looks like the vl32 branch on LMDB?

https://github.com/LMDB/lmdb/tree/vl32

[recoilme]

Yes

[krisskross]

I built this from latest https://github.com/LMDB/lmdb/tree/vl32 using the following agcc script.

#!/bin/bash
$NDK/android-ndk-r10e/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi-gcc --sysroot=$NDK/android-ndk-r10e/platforms/android-19/arch-arm -DMDB_DSYNC=O_SYNC $@

make CC="./agcc"

https://dl.dropboxusercontent.com/u/39196712/liblmdb.so

Let me know how it goes.

[recoilme]

Kristoffer,
but how test it without jni?

[krisskross]

You can replace the so file in lmdbjni-android-0.4.0. jar.

[recoilme]

Sorry, i try it, but lmdbjni-android contains only solid liblmdbjni.so (1.2Mb)

[krisskross]

Sorry, of course you need the proper jar built. Silly me.

https://dl.dropboxusercontent.com/u/39196712/lmdbjni-android-0.4.3-SNAPSHOT.jar

[recoilme]

I am not sure what all do right
At first - i am replace *so and add this code in test:

    //add some array
    long t0 = System.currentTimeMillis();
    Transaction tx = env.createWriteTransaction();
    for (int i = 0; i < n; i++) {
        lmdb.put(tx, ("" + i).getBytes(), byteArray);
        byte[] bytes = lmdb.get(tx,("" + i).getBytes());
    }
    tx.commit();
    String s = "Lmdb (put, get)*" + n + " : " + (System.currentTimeMillis() - t0) + "ms\n";

    textView.append(s);
    Log.w(TAG,s);
    // Buffer cursor 
    tx = env.createReadTransaction();
    try {
        BufferCursor cursor = lmdb.bufferCursor(tx);
        cursor.first();
        while(cursor.next()) {
            // read a position in buffer
            cursor.keyByte(0);
            cursor.valByte(0);
        }
    }
    catch (Exception e) {
        e.printStackTrace();
    }

And programm fail with sigsegv:

08-11 11:49:05.997 4369-4369/ru.test.test A/libc﹕ Fatal signal 11 (SIGSEGV), code 1, fault addr 0x10 in tid 4369 (ru.test.test)
08-11 11:49:06.099 191-191/? I/DEBUG﹕ *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-11 11:49:06.099 191-191/? I/DEBUG﹕ Build fingerprint: 'google/hammerhead/hammerhead:5.1/LMY47I/1767468:user/release-keys'
08-11 11:49:06.099 191-191/? I/DEBUG﹕ Revision: '11'
08-11 11:49:06.099 191-191/? I/DEBUG﹕ ABI: 'arm'
08-11 11:49:06.099 191-191/? I/DEBUG﹕ pid: 4369, tid: 4369, name: ru.test.test >>> ru.test.test <<<
08-11 11:49:06.099 191-191/? I/DEBUG﹕ signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x10
08-11 11:49:06.106 191-191/? I/DEBUG﹕ r0 00000013 r1 95483fac r2 0013507b r3 00000009
08-11 11:49:06.106 191-191/? I/DEBUG﹕ r4 00000010 r5 b4aff560 r6 00000010 r7 00000010
08-11 11:49:06.107 191-191/? I/DEBUG﹕ r8 00000000 r9 b4827800 sl 00000000 fp 12c5f460
08-11 11:49:06.107 191-191/? I/DEBUG﹕ ip 95483000 sp bea79e80 lr b3586180 pc b35839b0 cpsr 200b0010
08-11 11:49:06.107 191-191/? I/DEBUG﹕ backtrace:
08-11 11:49:06.107 191-191/? I/DEBUG﹕ #00 pc 000139b0 /data/app/ru.test.test-2/lib/arm/liblmdbjni.so (mdb_node_read+92)
08-11 11:49:06.107 191-191/? I/DEBUG﹕ #1 pc 0001617c /data/app/ru.test.test-2/lib/arm/liblmdbjni.so (mdb_cursor_first+276)
08-11 11:49:06.107 191-191/? I/DEBUG﹕ #2 pc 000150d0 /data/app/ru.test.test-2/lib/arm/liblmdbjni.so (mdb_cursor_get+612)
08-11 11:49:06.107 191-191/? I/DEBUG﹕ #3 pc 000d0b8b /data/dalvik-cache/arm/data@[email protected]@[email protected]

• without Buffer cursor all work

[recoilme]

More detail message:
this command:lmdb.bufferCursor(env.createReadTransaction()).first()
org.fusesource.lmdbjni.LMDBException: MDB_BAD_RSLOT: Invalid reuse of reader locktable slot

[krisskross]

Yes, i'm afraid that BufferCursor will not work without taking 32-bit addresses into account.

I don't have time to spend on this right now. But if you want to take a stab at it, read up on #21 and try something similar. If you get a prototype working i'll be happy to implement the rest.

[recoilme]

Whats sad but not critical. I dont know C( Feel free to close this issue..
Last question: i am right understand what jnilmdb will work only on android 19+?

[krisskross]

There is nothing in lmdbjni that enforce a particular android version. I would expect the NDK to be backward compatible.

[krisskross]

Ok, closing.