Switched to using emurgo/cardano-serialization-lib

[bladyjoker]

Fixes (#488)

TODO:

• Update CTL to use the latest emurgo/cardano-serialization-library
  • STATUS: Done
• Fix any broken tests
  • STATUS: Done
  • Updates:
    • Fix the fixtures
    • Discovered new issues with CSL with new tests
    • purs-bridge PR checks for Cbor level encoding compatibility now
    • The incompatible datums were probably created with an older version of cardano libs.
• Add regression tests for zero serialization
  • STATUS: DONE
• Add a comprehensive test suite using Plutus data samples gathered from the Ogmios Datum Cache testing Aeson and hashing
  • STATUS: Done:

STASH:

• Unskip the plutusDataAesonRoundtripTest when Make honest Aeson Eq instance mlabs-haskell/purescript-aeson#7 is fixed
• Switch to https://github.com/dcSpark/cardano-multiplatform-lib
• Remove Cbor coding in Plutus.ToFromPlutusType
• Consolidate CML boundary in a separate and tested purescript-package, introduce consistent API interaction that could lead to code-generated modules (from CDDL or Flow?)
• We use BigInt where BigNum should be used, consider reflecting the types used in CML more closely
• Deserialization.FromBytes exists but there's not a dual typeclass in Serialization, let's introduce it
• Combine Serialization/Deserialization modules into a single one

[bladyjoker]

deserialization » WitnessSet - deserialization is inverse to serialization                                                                                                                                                                    
  ✗ fixture #1:                                                                                                                                                                                                                               
                                                                                                                                                                                                                                              
  (hexToByteArrayUnsafe "a40081825820096092b8515d75c2a2f75d6aa7c5191996755840e81deaa403dba5b690f091b6584089ed6f628b02ed3c79f1b3508e35ea772aea916e7c88010f34cc57ee619a9b6ec3cadd9bb4aeb14374121111db4f75fc8c8fc8772ba6b82b599743abf6fa3a0503815
909b75909b401000033233223322323233322232333222323333333322222222323332223233332222323233223233322232333222323233223322323233333222223322332233223322332233222222323253353031333006375c00a6eb8010cccd5cd19b8735573aa004900011980499191919191919
191919191999ab9a3370e6aae754029200023333333333017335025232323333573466e1cd55cea8012400046603a60706ae854008c0a8d5d09aba250022350573530583357389201035054310005949926135573ca00226ea8004d5d0a80519a8128131aba150093335502c75ca0566ae854020ccd540
b1d728159aba1500733502504135742a00c66a04a66aa0a4094eb4d5d0a8029919191999ab9a3370e6aae7540092000233501f3232323333573466e1cd55cea80124000466a04e66a080eb4d5d0a80118229aba135744a00446a0b66a60b866ae712401035054310005d49926135573ca00226ea8004d5
d0a8011919191999ab9a3370e6aae7540092000233502533504075a6ae854008c114d5d09aba2500223505b35305c3357389201035054310005d49926135573ca00226ea8004d5d09aba250022350573530583357389201035054310005949926135573ca00226ea8004d5d0a80219a812bae35742a006
66a04a66aa0a4eb88004d5d0a801181b9aba135744a00446a0a66a60a866ae71241035054310005549926135744a00226ae8940044d5d1280089aba25001135744a00226ae8940044d5d1280089aba25001135573ca00226ea8004d5d0a8011919191999ab9a3370ea00290031180e181c9aba135573ca
00646666ae68cdc3a801240084603660866ae84d55cf280211999ab9a3370ea00690011180d98171aba135573ca00a46666ae68cdc3a802240004603c6eb8d5d09aab9e500623504e35304f3357389201035054310005049926499264984d55cea80089baa001357426ae8940088d411cd4c120cd5ce24
90350543100049499261048135046353047335738920103505435000484984d55cf280089baa0012212330010030022001222222222212333333333300100b00a0090080070060050040030022001221233001003002200112212330010030021200112212330010030021200112212330010030021200
1212222300400521222230030052122223002005212222300100520011232230023758002640026aa068446666aae7c004940388cd4034c010d5d080118019aba200203323232323333573466e1cd55cea801a4000466600e6464646666ae68cdc39aab9d5002480008cc034c0c4d5d0a80119a8098169
aba135744a00446a06c6a606e66ae712401035054310003849926135573ca00226ea8004d5d0a801999aa805bae500a35742a00466a01eeb8d5d09aba25002235032353033335738921035054310003449926135744a00226aae7940044dd5000911091998008020018011000910919800801801100089
9aa800bae75a224464460046eac004c8004d540b888c8cccd55cf80112804919a80419aa81898031aab9d5002300535573ca00460086ae8800c0b84d5d08008891001091091198008020018900089119191999ab9a3370ea002900011a80418029aba135573ca00646666ae68cdc3a801240044a01046a
0526a605466ae712401035054310002b499264984d55cea80089baa001121223002003112200112001232323333573466e1cd55cea8012400046600c600e6ae854008dd69aba135744a00446a0466a604866ae71241035054310002549926135573ca00226ea80048848cc00400c00880048c8cccd5cd1
9b8735573aa002900011bae357426aae7940088d407cd4c080cd5ce24810350543100021499261375400224464646666ae68cdc3a800a40084a00e46666ae68cdc3a8012400446a014600c6ae84d55cf280211999ab9a3370ea00690001280511a8111a981199ab9c49010350543100024499264992613
5573aa00226ea8004484888c00c0104488800844888004480048c8cccd5cd19b8750014800880188cccd5cd19b8750024800080188d4068d4c06ccd5ce249035054310001c499264984d55ce9baa0011220021220012001232323232323333573466e1d4005200c200b23333573466e1d4009200a200d2
3333573466e1d400d200823300b375c6ae854014dd69aba135744a00a46666ae68cdc3a8022400c46601a6eb8d5d0a8039bae357426ae89401c8cccd5cd19b875005480108cc048c050d5d0a8049bae357426ae8940248cccd5cd19b875006480088c050c054d5d09aab9e500b23333573466e1d401d20
00230133016357426aae7940308d407cd4c080cd5ce2481035054310002149926499264992649926135573aa00826aae79400c4d55cf280109aab9e500113754002424444444600e01044244444446600c012010424444444600a010244444440082444444400644244444446600401201044244444446
600201201040024646464646666ae68cdc3a800a400446660106eb4d5d0a8021bad35742a0066eb4d5d09aba2500323333573466e1d400920002300a300b357426aae7940188d4040d4c044cd5ce2490350543100012499264984d55cea80189aba25001135573ca00226ea80048488c00800c888488cc
c00401401000c80048c8c8cccd5cd19b875001480088c018dd71aba135573ca00646666ae68cdc3a80124000460106eb8d5d09aab9e500423500a35300b3357389201035054310000c499264984d55cea80089baa001212230020032122300100320011122232323333573466e1cd55cea80124000466a
a016600c6ae854008c014d5d09aba25002235007353008335738921035054310000949926135573ca00226ea8004498480048004448848cc00400c008448004448c8c00400488cc00cc008008004cccc888ccc888cccccccc88888888cc88ccccc88888ccc888cccc8888cc88cc88cc88ccc888cc88cc8
8ccc888cc88cc88cc88cc8888894cd4c084ccd5cd19b8f00337240040460442046266ae71240109426164206775657373000222212330010030022001222222222212333333333300100b00a00900800700600500400300220012212330010030022001222123330010040030022001112200212212233
0010040031200111221233001003002112001221233001003002200112122300200311220011200112212330010030021200112212330010030021200112212330010030021200112122230030041122200211222001120011220021220012001212222300400521222230030052122223002005212222
300100520012212330010030022001212222222300700822122222223300600900821222222230050081222222200412222222003221222222233002009008221222222233001009008200121223002003222122333001005004003200121223002003212230010032001480101049f58202bb80d537b1
da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25bff058184000046736563726574821a00065cd41a0af4845c") ≠ (hexToByteArrayUnsafe "a40081825820096092b8515d75c2a2f75d6aa7c5191996755840e81deaa403dba5b690f091b6584089ed6f628b02ed3c79f1b3508e35e
a772aea916e7c88010f34cc57ee619a9b6ec3cadd9bb4aeb14374121111db4f75fc8c8fc8772ba6b82b599743abf6fa3a0503815909b75909b4010000332332233223232333222323332223233333333222222223233322232333322223232332232333222323332223232332233223232333332222233
22332233223322332233222222323253353031333006375c00a6eb8010cccd5cd19b8735573aa004900011980499191919191919191919191999ab9a3370e6aae754029200023333333333017335025232323333573466e1cd55cea8012400046603a60706ae854008c0a8d5d09aba2500223505735305
83357389201035054310005949926135573ca00226ea8004d5d0a80519a8128131aba150093335502c75ca0566ae854020ccd540b1d728159aba1500733502504135742a00c66a04a66aa0a4094eb4d5d0a8029919191999ab9a3370e6aae7540092000233501f3232323333573466e1cd55cea8012400
0466a04e66a080eb4d5d0a80118229aba135744a00446a0b66a60b866ae712401035054310005d49926135573ca00226ea8004d5d0a8011919191999ab9a3370e6aae7540092000233502533504075a6ae854008c114d5d09aba2500223505b35305c3357389201035054310005d49926135573ca00226
ea8004d5d09aba250022350573530583357389201035054310005949926135573ca00226ea8004d5d0a80219a812bae35742a00666a04a66aa0a4eb88004d5d0a801181b9aba135744a00446a0a66a60a866ae71241035054310005549926135744a00226ae8940044d5d1280089aba25001135744a002
26ae8940044d5d1280089aba25001135573ca00226ea8004d5d0a8011919191999ab9a3370ea00290031180e181c9aba135573ca00646666ae68cdc3a801240084603660866ae84d55cf280211999ab9a3370ea00690011180d98171aba135573ca00a46666ae68cdc3a802240004603c6eb8d5d09aab9
e500623504e35304f3357389201035054310005049926499264984d55cea80089baa001357426ae8940088d411cd4c120cd5ce2490350543100049499261048135046353047335738920103505435000484984d55cf280089baa0012212330010030022001222222222212333333333300100b00a00900
800700600500400300220012212330010030022001122123300100300212001122123300100300212001122123300100300212001212222300400521222230030052122223002005212222300100520011232230023758002640026aa068446666aae7c004940388cd4034c010d5d080118019aba20020
3323232323333573466e1cd55cea801a4000466600e6464646666ae68cdc39aab9d5002480008cc034c0c4d5d0a80119a8098169aba135744a00446a06c6a606e66ae712401035054310003849926135573ca00226ea8004d5d0a801999aa805bae500a35742a00466a01eeb8d5d09aba2500223503235
3033335738921035054310003449926135744a00226aae7940044dd50009110919980080200180110009109198008018011000899aa800bae75a224464460046eac004c8004d540b888c8cccd55cf80112804919a80419aa81898031aab9d5002300535573ca00460086ae8800c0b84d5d080088910010
91091198008020018900089119191999ab9a3370ea002900011a80418029aba135573ca00646666ae68cdc3a801240044a01046a0526a605466ae712401035054310002b499264984d55cea80089baa001121223002003112200112001232323333573466e1cd55cea8012400046600c600e6ae854008d
d69aba135744a00446a0466a604866ae71241035054310002549926135573ca00226ea80048848cc00400c00880048c8cccd5cd19b8735573aa002900011bae357426aae7940088d407cd4c080cd5ce24810350543100021499261375400224464646666ae68cdc3a800a40084a00e46666ae68cdc3a80
12400446a014600c6ae84d55cf280211999ab9a3370ea00690001280511a8111a981199ab9c490103505431000244992649926135573aa00226ea8004484888c00c0104488800844888004480048c8cccd5cd19b8750014800880188cccd5cd19b8750024800080188d4068d4c06ccd5ce249035054310
001c499264984d55ce9baa0011220021220012001232323232323333573466e1d4005200c200b23333573466e1d4009200a200d23333573466e1d400d200823300b375c6ae854014dd69aba135744a00a46666ae68cdc3a8022400c46601a6eb8d5d0a8039bae357426ae89401c8cccd5cd19b87500548
0108cc048c050d5d0a8049bae357426ae8940248cccd5cd19b875006480088c050c054d5d09aab9e500b23333573466e1d401d2000230133016357426aae7940308d407cd4c080cd5ce2481035054310002149926499264992649926135573aa00826aae79400c4d55cf280109aab9e500113754002424
444444600e01044244444446600c012010424444444600a010244444440082444444400644244444446600401201044244444446600201201040024646464646666ae68cdc3a800a400446660106eb4d5d0a8021bad35742a0066eb4d5d09aba2500323333573466e1d400920002300a300b357426aae7
940188d4040d4c044cd5ce2490350543100012499264984d55cea80189aba25001135573ca00226ea80048488c00800c888488ccc00401401000c80048c8c8cccd5cd19b875001480088c018dd71aba135573ca00646666ae68cdc3a80124000460106eb8d5d09aab9e500423500a35300b33573892010
35054310000c499264984d55cea80089baa001212230020032122300100320011122232323333573466e1cd55cea80124000466aa016600c6ae854008c014d5d09aba25002235007353008335738921035054310000949926135573ca00226ea8004498480048004448848cc00400c008448004448c8c0
0400488cc00cc008008004cccc888ccc888cccccccc88888888cc88ccccc88888ccc888cccc8888cc88cc88cc88ccc888cc88cc88ccc888cc88cc88cc88cc8888894cd4c084ccd5cd19b8f00337240040460442046266ae712401094261642067756573730002222123300100300220012222222222123
33333333300100b00a0090080070060050040030022001221233001003002200122212333001004003002200111220021221223300100400312001112212330010030021120012212330010030022001121223002003112200112001122123300100300212001122123300100300212001122123300100
3002120011212223003004112220021122200112001122002122001200121222230040052122223003005212222300200521222230010052001221233001003002200121222222230070082212222222330060090082122222223005008122222220041222222200322122222223300200900822122222
2233001009008200121223002003222122333001005004003200121223002003212230010032001480101048158202bb80d537b1da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25b058184000046736563726574821a00065cd41a0af4845c")                                 
  ✓︎ fixture #2                                                                                                                                                                                                                                
  ✗ fixture #3:                                                                                                                                                                                                                               
                                                                                                                                                                                                                                              
  (hexToByteArrayUnsafe "a20081825820096092b8515d75c2a2f75d6aa7c5191996755840e81deaa403dba5b690f091b65840c7f77418c5c956aab848b6f101d6bc014f22c699642531c8f39f91979c0313f686999402c4769117857c08fcceb57b60478a590ecae5190317f7abff7cd38000049f5
8202bb80d537b1da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25bff") ≠ (hexToByteArrayUnsafe "a20081825820096092b8515d75c2a2f75d6aa7c5191996755840e81deaa403dba5b690f091b65840c7f77418c5c956aab848b6f101d6bc014f22c699642531c8f39f91979c031
3f686999402c4769117857c08fcceb57b60478a590ecae5190317f7abff7cd38000048158202bb80d537b1da3e38bd30361aa855686bde0eacd7162fef6a25fe97bf527a25b")                                                                                                 
  ✗ fixture #4:                                                                                                                                                                                                                               
                                                                                                                                                                                                                                              
  (hexToByteArrayUnsafe "a30081825820096092b8515d75c2a2f75d6aa7c5191996755840e81deaa403dba5b690f091b658400d91f7ab723ed0adb9f7ec06bba5cb99b4dcbbe8fb6ce45fb3fcab31ddf57ca085437d7ec4e6fea8d10d0c455fdfb2fdbcf1d89643f635841da0e2593f6dd50a01818
204187b049f02ff") ≠ (hexToByteArrayUnsafe "a30081825820096092b8515d75c2a2f75d6aa7c5191996755840e81deaa403dba5b690f091b658400d91f7ab723ed0adb9f7ec06bba5cb99b4dcbbe8fb6ce45fb3fcab31ddf57ca085437d7ec4e6fea8d10d0c455fdfb2fdbcf1d89643f635841da
0e2593f6dd50a01818204187b048102")                                                                                                                                                                                                             

[bladyjoker]

New failure

  ✗ Plutus data samples should have a compatible hash:                                                                                                                                                                                        
                                                                                                                                                                                                                                              
  (hexToByteArrayUnsafe "17115bfbd63fd9c90c3041e31fd28d0c1a5cce37ca337a0f016d25f8469fe0d1") ≠ (hexToByteArrayUnsafe "1e3c6202b98673b34c00933ea6b6d808d5b1015f22787d44ed69b3bdea577d3c")                                                       

for the following sample

  {
    "hash": "17115bfbd63fd9c90c3041e31fd28d0c1a5cce37ca337a0f016d25f8469fe0d1",
    "plutusData": {
      "constr": 0,
      "fields": [
        "da13ed22b9294f1d86bbd530e99b1456884c7364bf16c90edc1ae41e",
        500000000,
        "82325cbfc20b85bd1ca12e5d12b44b83f68662d8395167b45f1ff7fa",
        "746f6e6573206f6620736b7920",
        "da13ed22b9294f1d86bbd530e99b1456884c7364bf16c90edc1ae41e",
        45
      ]
    }
  },

[ngua]

Thanks for taking care of this @bladyjoker! Let's get this merged today

[ngua]

I'm going to run the integration tests and all the examples (I'm waiting for the node to sync). If all is well we can merge this

[ngua]

Looks like everything works!

[ngua]

Thanks @bladyjoker!

[ngua]

There's an issue with Cachix and Hydra, so the unit test check won't complete. They pass locally though, so we can go ahead and merge this

[bladyjoker]

Thanks @ngua @klntsky for the review. Also thank you for fixing the code and sorry for the implicit imports, this is LSP being annoying and doing that by default. Just so you know, I probably should have mentioned, I always run CTL against our testnet infrastructure before asking for reviews!