"failed to fill whole buffer" error message when trying to sync

[ConorOkus]

I'm trying to sync LDK using as_Confirm().transactions_confirmed in Kotlin/Android here but when called the app crashes with the following error:

E/LDK: thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Error { kind: UnexpectedEof, message: "failed to fill whole buffer" })', src/c_types/mod.rs:402:116
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 32055 (DefaultDispatch), pid 31807 (e.umlandowallet)

Below is a screenshot of what I think is being passed to this method:

[TheBlueMatt]

That indicates the transaction was not valid. Specifically, we needed to read more bytes than were available.

[ConorOkus]

Thanks I had to make a call to GET /tx/:txid/raw instead of just GET /tx/:txid and turning the response into a ByteArray.

[ConorOkus]

I'm now getting a crash on the GET /tx/:txid/merkle-proof call:

2022-11-01 20:31:06.985 6552-6804/com.example.umlandowallet A/libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 6804 (DefaultDispatch), pid 6552 (e.umlandowallet)

Some more logs:

2022-11-01 20:31:07.075 7016-7016/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2022-11-01 20:31:07.075 7016-7016/? A/DEBUG: Build fingerprint: 'google/sdk_gphone_x86_64_arm64/generic_x86_64_arm64:11/RSR1.210722.013/7800151:userdebug/dev-keys'
2022-11-01 20:31:07.075 7016-7016/? A/DEBUG: Revision: '0'
2022-11-01 20:31:07.075 7016-7016/? A/DEBUG: ABI: 'x86_64'
2022-11-01 20:31:07.076 7016-7016/? A/DEBUG: Timestamp: 2022-11-01 20:31:07+0000
2022-11-01 20:31:07.076 7016-7016/? A/DEBUG: pid: 6552, tid: 6804, name: DefaultDispatch  >>> com.example.umlandowallet <<<
2022-11-01 20:31:07.076 7016-7016/? A/DEBUG: uid: 10168
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG: Cause: null pointer dereference
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG:     rax 0000000000000000  rbx 0000000000000000  rcx 0000000000000000  rdx 0000000000000000
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG:     r8  000073650c3cbfb8  r9  000073650fadd1b9  r10 000000000001f36e  r11 000000000000f0ff
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG:     r12 0000000000000000  r13 0080736599765610  r14 000073650c3cc0d0  r15 000073659973f490
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG:     rdi 0000000000000000  rsi 000073650c3cbfd0
2022-11-01 20:31:07.077 7016-7016/? A/DEBUG:     rbp 00807366d973e030  rsp 000073650c3cc050  rip 000073650e79af76
2022-11-01 20:31:07.197 7016-7016/? A/DEBUG: backtrace:
2022-11-01 20:31:07.197 7016-7016/? A/DEBUG:       #00 pc 0000000000714f76  /data/app/~~dzTzButcHcEWAMqhb5vTBQ==/com.example.umlandowallet-jTP6PzdAsGXwoZ7w5c0fhA==/base.apk (offset 0xd28000) (update_persisted_channel_LDKPersist_jcall+406)
2022-11-01 20:31:07.197 7016-7016/? A/DEBUG:       #01 pc 000000000065a746  /data/app/~~dzTzButcHcEWAMqhb5vTBQ==/com.example.umlandowallet-jTP6PzdAsGXwoZ7w5c0fhA==/base.apk (offset 0xd28000)
2022-11-01 20:31:07.197 7016-7016/? A/DEBUG:       #02 pc 000000000065e04d  /data/app/~~dzTzButcHcEWAMqhb5vTBQ==/com.example.umlandowallet-jTP6PzdAsGXwoZ7w5c0fhA==/base.apk (offset 0xd28000)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #03 pc 0000000000714a62  /data/app/~~dzTzButcHcEWAMqhb5vTBQ==/com.example.umlandowallet-jTP6PzdAsGXwoZ7w5c0fhA==/base.apk (offset 0xd28000) (Java_org_ldk_impl_bindings_Confirm_1transactions_1confirmed+370)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #04 pc 00000000001840c7  /apex/com.android.art/lib64/libart.so (art_quick_generic_jni_trampoline+215) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #05 pc 000000000017080a  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #06 pc 00000000000dca5c  [anon:dalvik-classes8.dex extracted in memory from /data/app/~~dzTzButcHcEWAMqhb5vTBQ==/com.example.umlandowallet-jTP6PzdAsGXwoZ7w5c0fhA==/base.apk!classes8.dex] (org.ldk.structs.Confirm.transactions_confirmed+52)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #07 pc 0000000000170d5d  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #08 pc 0000000000002c48  /data/data/com.example.umlandowallet/code_cache/.overlay/base.apk/classes3.dex (com.example.umlandowallet.data.remote.AccessImpl.syncTransactionConfirmed+1448)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #09 pc 0000000000170d5d  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2022-11-01 20:31:07.198 7016-7016/? A/DEBUG:       #10 pc 000000000000242a  /data/data/com.example.umlandowallet/code_cache/.overlay/base.apk/classes3.dex (com.example.umlandowallet.data.remote.AccessImpl$syncTransactionConfirmed$1.invokeSuspend+30)

[TheBlueMatt]

Do you have any more context to the second log fragment?

[TheBlueMatt]

Ah, the issue is here - https://github.com/ConorOkus/uMlando-wallet/blob/c92968d31b7936b41fd3ff24b9d5abd8a6627a99/app/src/main/java/com/example/umlandowallet/Start.kt#L239. update is null when we're persisting a ChannelMonitor because of block/transaction connection, but we still need to update and get a valid return value. Returning null there is causing us to try to figure out what the return value is from null, which causes both of the above issues.

[ConorOkus]

I moved those checks and that error seems to have gone. Now getting UnsignedChannelAnnouncement error when opening an outbound channel to the LDK sample node (regtest):
EVENT: Channel 7ed35345d8a762168611c378b7b8cb35b638be3047814e63e43e4074504237e4 closed due to: ProcessingError { err: "Bad announcement_signatures. Failed to verify node_signature. UnsignedChannelAnnouncement used for verification is UnsignedChannelAnnouncement { features: [], chain_hash: 0f9188f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e2206, short_channel_id: 223200860504065, node_id_1: PublicKey(e9c2dd6e3bfd4576738239d05c38575f3a5663ddf92c0156aa9be86b21bd41f04210a3ada8f0db4da67f2d569325d2ac98fb8f7bad385bbed6677fdf0d486955), node_id_2: PublicKey(0748bd1f3659697db53a44972eb3328ba18dd64679b14d89facf6ce6494d13eccbb9a787be357439dd1863e3f3b8ba4eb78575306c78cc6987246c7c54d54bea), bitcoin_key_1: PublicKey(6badfa7a479933028e9975b9ffe6436e26fabd364b6817f6b9e56e8678af7268a3939587cbe05f514c9747802ca9935dd15ede3f3579e596af157ef86efb0851), bitcoin_key_2: PublicKey(d0df6c0de0936980a2e739541fd498313cd32b46182451cebfb98891a131a2b98a7bcdd1b207af6a4f06704f42adad2d1c4156bbe761450369a9cd20bb45c2b2), excess_data: [] }. their_node_key is PublicKey(0748bd1f3659697db53a44972eb3328ba18dd64679b14d89facf6ce6494d13eccbb9a787be357439dd1863e3f3b8ba4eb78575306c78cc6987246c7c54d54bea)" }

Opening a private channel works fine PR here. Anyway, the original issue was fixed so will create a new discussion if the problem still continues.

Thanks @TheBlueMatt

[tnull]

As just discussed it is rather strange to see this error (originating here), which could only happen if the two nodes either don't agree on each's other public keys or on the chain hash (or have them in the wrong format). We just briefly went through both of those, but you might check that again.

[TheBlueMatt]

Is it possible there's a missing block hash endian reverse somewhere? Keep in mind that block hashes (and txids) are almost always printed in byte-reversed order, but stored in memory (both in LDK and generally across the bitcoin ecosystem) they are not byte-reversed. Thus, if you see a block hash and you parse it from hex into a 32-byte array you need to byte-swap it.