Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a faster encoding algoritm for varint #98

Merged
merged 1 commit into from
Jul 12, 2024

Implement a faster encoding algoritm for varint

440a2c5
Select commit
Loading
Failed to load commit list.
Refresh
Sign in for the full log view
Merged

Implement a faster encoding algoritm for varint #98

Implement a faster encoding algoritm for varint
440a2c5
Select commit
Loading
Failed to load commit list.
GitHub Actions / Unit Tests net8.0 BMI2 succeeded Jul 12, 2024 in 1s

190 passed, 0 failed and 0 skipped

Tests passed successfully

✅ Snappier.Tests/TestResults/results.trx

190 tests were completed in 2s with 190 passed, 0 failed and 0 skipped.

Test suite Passed Failed Skipped Time
Snappier.Tests.HelpersTests 36✅ 9ms
Snappier.Tests.Internal.Crc32CAlgorithmTests 4✅ 17ms
Snappier.Tests.Internal.SnappyCompressorTests 45✅ 12ms
Snappier.Tests.Internal.SnappyDecompressorTests 6✅ 24ms
Snappier.Tests.Internal.SnappyStreamCompressorTests 11✅ 1s
Snappier.Tests.Internal.VarIntEncodingWriteTests 26✅ 39ms
Snappier.Tests.SnappyStreamTests 34✅ 636ms
Snappier.Tests.SnappyTests 28✅ 1s

✅ Snappier.Tests.HelpersTests

✅ LeftShiftOverflows_False(value: 0, shift: 31)
✅ LeftShiftOverflows_False(value: 1, shift: 31)
✅ LeftShiftOverflows_False(value: 255, shift: 24)
✅ LeftShiftOverflows_True(value: 2, shift: 31)
✅ LeftShiftOverflows_True(value: 255, shift: 25)
✅ Log2FloorNonZero(value: 1)
✅ Log2FloorNonZero(value: 10)
✅ Log2FloorNonZero(value: 11)
✅ Log2FloorNonZero(value: 12)
✅ Log2FloorNonZero(value: 13)
✅ Log2FloorNonZero(value: 14)
✅ Log2FloorNonZero(value: 15)
✅ Log2FloorNonZero(value: 16)
✅ Log2FloorNonZero(value: 17)
✅ Log2FloorNonZero(value: 18)
✅ Log2FloorNonZero(value: 19)
✅ Log2FloorNonZero(value: 2)
✅ Log2FloorNonZero(value: 20)
✅ Log2FloorNonZero(value: 21)
✅ Log2FloorNonZero(value: 22)
✅ Log2FloorNonZero(value: 23)
✅ Log2FloorNonZero(value: 24)
✅ Log2FloorNonZero(value: 25)
✅ Log2FloorNonZero(value: 26)
✅ Log2FloorNonZero(value: 27)
✅ Log2FloorNonZero(value: 28)
✅ Log2FloorNonZero(value: 29)
✅ Log2FloorNonZero(value: 3)
✅ Log2FloorNonZero(value: 30)
✅ Log2FloorNonZero(value: 31)
✅ Log2FloorNonZero(value: 4)
✅ Log2FloorNonZero(value: 5)
✅ Log2FloorNonZero(value: 6)
✅ Log2FloorNonZero(value: 7)
✅ Log2FloorNonZero(value: 8)
✅ Log2FloorNonZero(value: 9)

✅ Snappier.Tests.Internal.Crc32CAlgorithmTests

✅ Compute(asciiChars: "123456789", expectedResult: 3808858755)
✅ Compute(asciiChars: "1234567890123456", expectedResult: 2594449535)
✅ Compute(asciiChars: "123456789012345612345678901234", expectedResult: 3972483380)
✅ Compute(asciiChars: "12345678901234561234567890123456", expectedResult: 3444075339)

✅ Snappier.Tests.Internal.SnappyCompressorTests

✅ FindMatchLength(expectedResult: 0, s1String: "01234567", s2String: "?1234567", length: 8)
✅ FindMatchLength(expectedResult: 0, s1String: "01234567xxxxxxxx", s2String: "?1234567xxxxxxxx", length: 16)
✅ FindMatchLength(expectedResult: 1, s1String: "01234567", s2String: "0?234567", length: 8)
✅ FindMatchLength(expectedResult: 1, s1String: "01234567xxxxxxxx", s2String: "0?234567xxxxxxxx", length: 16)
✅ FindMatchLength(expectedResult: 10, s1String: "xxxxxxabcd?", s2String: "xxxxxxabcd?", length: 10)
✅ FindMatchLength(expectedResult: 10, s1String: "xxxxxxabcd", s2String: "xxxxxxabcd", length: 10)
✅ FindMatchLength(expectedResult: 10, s1String: "xxxxxxabcd0123", s2String: "xxxxxxabcd?123", length: 14)
✅ FindMatchLength(expectedResult: 10, s1String: "xxxxxxabcd0123xx", s2String: "xxxxxxabcd?123xx", length: 16)
✅ FindMatchLength(expectedResult: 11, s1String: "01234567abc!", s2String: "01234567abc!", length: 11)
✅ FindMatchLength(expectedResult: 11, s1String: "01234567abc!", s2String: "01234567abc?", length: 11)
✅ FindMatchLength(expectedResult: 11, s1String: "01234567abc", s2String: "01234567abc", length: 11)
✅ FindMatchLength(expectedResult: 11, s1String: "xxxxxx0123abc", s2String: "xxxxxx0123axc", length: 13)
✅ FindMatchLength(expectedResult: 11, s1String: "xxxxxxabcd0123", s2String: "xxxxxxabcd0?23", length: 14)
✅ FindMatchLength(expectedResult: 11, s1String: "xxxxxxabcd0123xx", s2String: "xxxxxxabcd0?23xx", length: 16)
✅ FindMatchLength(expectedResult: 12, s1String: "abcdefgh01234567xxxxxxxx", s2String: "abcdefgh01237654xxxxxxxx", length: 24)
✅ FindMatchLength(expectedResult: 12, s1String: "xxxxxx0123abc!", s2String: "xxxxxx0123abc!", length: 12)
✅ FindMatchLength(expectedResult: 12, s1String: "xxxxxx0123abc!", s2String: "xxxxxx0123abc?", length: 12)
✅ FindMatchLength(expectedResult: 12, s1String: "xxxxxxabcd0123", s2String: "xxxxxxabcd0132", length: 14)
✅ FindMatchLength(expectedResult: 12, s1String: "xxxxxxabcd0123xx", s2String: "xxxxxxabcd0132xx", length: 16)
✅ FindMatchLength(expectedResult: 13, s1String: "xxxxxxabcd0123", s2String: "xxxxxxabcd012?", length: 14)
✅ FindMatchLength(expectedResult: 13, s1String: "xxxxxxabcd0123xx", s2String: "xxxxxxabcd012?xx", length: 16)
✅ FindMatchLength(expectedResult: 13, s1String: "xxxxxxabcdef", s2String: "xxxxxxabcdefx", length: 13)
✅ FindMatchLength(expectedResult: 15, s1String: "abcdefgh01234567xxxxxxxx", s2String: "abcdefgh0123456?xxxxxxxx", length: 24)
✅ FindMatchLength(expectedResult: 2, s1String: "01234567", s2String: "01?34567", length: 8)
✅ FindMatchLength(expectedResult: 3, s1String: "01234567", s2String: "012?4567", length: 8)
✅ FindMatchLength(expectedResult: 4, s1String: "01234567", s2String: "0123?567", length: 8)
✅ FindMatchLength(expectedResult: 4, s1String: "01234567xxxxxxxx", s2String: "01237654xxxxxxxx", length: 16)
✅ FindMatchLength(expectedResult: 5, s1String: "01234567", s2String: "01234?67", length: 8)
✅ FindMatchLength(expectedResult: 6, s1String: "012345", s2String: "012345", length: 6)
✅ FindMatchLength(expectedResult: 6, s1String: "01234567", s2String: "012345?7", length: 8)
✅ FindMatchLength(expectedResult: 6, s1String: "xxxxxx0123", s2String: "xxxxxx?123", length: 10)
✅ FindMatchLength(expectedResult: 6, s1String: "xxxxxx0123xxxxxxxx", s2String: "xxxxxx?123xxxxxxxx", length: 18)
✅ FindMatchLength(expectedResult: 7, s1String: "01234567!", s2String: "0123456??", length: 7)
✅ FindMatchLength(expectedResult: 7, s1String: "01234567", s2String: "0123456?", length: 7)
✅ FindMatchLength(expectedResult: 7, s1String: "01234567", s2String: "0123456?", length: 8)
✅ FindMatchLength(expectedResult: 7, s1String: "01234567xxxxxxxx", s2String: "0123456?xxxxxxxx", length: 16)
✅ FindMatchLength(expectedResult: 7, s1String: "xxxxxx0123", s2String: "xxxxxx0?23", length: 10)
✅ FindMatchLength(expectedResult: 7, s1String: "xxxxxx0123xxxxxxxx", s2String: "xxxxxx0?23xxxxxxxx", length: 18)
✅ FindMatchLength(expectedResult: 8, s1String: "abcdefgh01234567xxxxxxxx", s2String: "abcdefgh?1234567xxxxxxxx", length: 24)
✅ FindMatchLength(expectedResult: 8, s1String: "xxxxxx0123", s2String: "xxxxxx0132", length: 10)
✅ FindMatchLength(expectedResult: 8, s1String: "xxxxxx0123xxxxxxxx", s2String: "xxxxxx0132xxxxxxxx", length: 18)
✅ FindMatchLength(expectedResult: 9, s1String: "01234567abc", s2String: "01234567axc", length: 9)
✅ FindMatchLength(expectedResult: 9, s1String: "abcdefgh01234567xxxxxxxx", s2String: "abcdefgh0?234567xxxxxxxx", length: 24)
✅ FindMatchLength(expectedResult: 9, s1String: "xxxxxx0123", s2String: "xxxxxx012?", length: 10)
✅ FindMatchLength(expectedResult: 9, s1String: "xxxxxx0123xxxxxxxx", s2String: "xxxxxx012?xxxxxxxx", length: 18)

✅ Snappier.Tests.Internal.SnappyDecompressorTests

✅ DecompressAllTags_ShortInputBufferWhichCopiesToScratch_DoesNotReadPastEndOfScratch
✅ ExtractData_NoLength_InvalidOperationException
✅ ExtractData_NotFullDecompressed_InvalidOperationException
✅ ExtractData_SomeData_DoesResetForReuse
✅ ExtractData_SomeData_Memory
✅ ExtractData_ZeroLength_EmptyMemory

✅ Snappier.Tests.Internal.SnappyStreamCompressorTests

✅ Write(resourceName: "alice29.txt")
✅ Write(resourceName: "asyoulik.txt")
✅ Write(resourceName: "fireworks.jpeg")
✅ Write(resourceName: "geo.protodata")
✅ Write(resourceName: "html_x_4")
✅ Write(resourceName: "html")
✅ Write(resourceName: "kppkn.gtb")
✅ Write(resourceName: "lcet10.txt")
✅ Write(resourceName: "paper-100k.pdf")
✅ Write(resourceName: "plrabn12.txt")
✅ Write(resourceName: "urls.10K")

✅ Snappier.Tests.Internal.VarIntEncodingWriteTests

✅ Test_Write(value: 0, expected: [0])
✅ Test_Write(value: 1, expected: [1])
✅ Test_Write(value: 127, expected: [127])
✅ Test_Write(value: 128, expected: [128, 1])
✅ Test_Write(value: 1365, expected: [213, 10])
✅ Test_Write(value: 212856576, expected: [128, 222, 191, 101])
✅ Test_Write(value: 21845, expected: [213, 170, 1])
✅ Test_Write(value: 32767, expected: [255, 255, 1])
✅ Test_Write(value: 32768, expected: [128, 128, 2])
✅ Test_Write(value: 3405705229, expected: [141, 224, 251, 215, 12])
✅ Test_Write(value: 4294967295, expected: [255, 255, 255, 255, 15])
✅ Test_Write(value: 49151, expected: [255, 255, 2])
✅ Test_Write(value: 65535, expected: [255, 255, 3])
✅ Test_WriteWithPadding(value: 0, expected: [0])
✅ Test_WriteWithPadding(value: 1, expected: [1])
✅ Test_WriteWithPadding(value: 127, expected: [127])
✅ Test_WriteWithPadding(value: 128, expected: [128, 1])
✅ Test_WriteWithPadding(value: 1365, expected: [213, 10])
✅ Test_WriteWithPadding(value: 212856576, expected: [128, 222, 191, 101])
✅ Test_WriteWithPadding(value: 21845, expected: [213, 170, 1])
✅ Test_WriteWithPadding(value: 32767, expected: [255, 255, 1])
✅ Test_WriteWithPadding(value: 32768, expected: [128, 128, 2])
✅ Test_WriteWithPadding(value: 3405705229, expected: [141, 224, 251, 215, 12])
✅ Test_WriteWithPadding(value: 4294967295, expected: [255, 255, 255, 255, 15])
✅ Test_WriteWithPadding(value: 49151, expected: [255, 255, 2])
✅ Test_WriteWithPadding(value: 65535, expected: [255, 255, 3])

✅ Snappier.Tests.SnappyStreamTests

✅ CompressAndDecompress(filename: "alice29.txt")
✅ CompressAndDecompress(filename: "asyoulik.txt")
✅ CompressAndDecompress(filename: "fireworks.jpeg")
✅ CompressAndDecompress(filename: "geo.protodata")
✅ CompressAndDecompress(filename: "html_x_4")
✅ CompressAndDecompress(filename: "html")
✅ CompressAndDecompress(filename: "kppkn.gtb")
✅ CompressAndDecompress(filename: "lcet10.txt")
✅ CompressAndDecompress(filename: "paper-100k.pdf")
✅ CompressAndDecompress(filename: "plrabn12.txt")
✅ CompressAndDecompress(filename: "urls.10K")
✅ CompressAndDecompressAsync(filename: "alice29.txt")
✅ CompressAndDecompressAsync(filename: "asyoulik.txt")
✅ CompressAndDecompressAsync(filename: "fireworks.jpeg")
✅ CompressAndDecompressAsync(filename: "geo.protodata")
✅ CompressAndDecompressAsync(filename: "html_x_4")
✅ CompressAndDecompressAsync(filename: "html")
✅ CompressAndDecompressAsync(filename: "kppkn.gtb")
✅ CompressAndDecompressAsync(filename: "lcet10.txt")
✅ CompressAndDecompressAsync(filename: "paper-100k.pdf")
✅ CompressAndDecompressAsync(filename: "plrabn12.txt")
✅ CompressAndDecompressAsync(filename: "urls.10K")
✅ CompressAndDecompressChunkStressTest(filename: "alice29.txt")
✅ CompressAndDecompressChunkStressTest(filename: "asyoulik.txt")
✅ CompressAndDecompressChunkStressTest(filename: "fireworks.jpeg")
✅ CompressAndDecompressChunkStressTest(filename: "geo.protodata")
✅ CompressAndDecompressChunkStressTest(filename: "html_x_4")
✅ CompressAndDecompressChunkStressTest(filename: "html")
✅ CompressAndDecompressChunkStressTest(filename: "kppkn.gtb")
✅ CompressAndDecompressChunkStressTest(filename: "lcet10.txt")
✅ CompressAndDecompressChunkStressTest(filename: "paper-100k.pdf")
✅ CompressAndDecompressChunkStressTest(filename: "plrabn12.txt")
✅ CompressAndDecompressChunkStressTest(filename: "urls.10K")
✅ Known8192ByteChunkStressTest

✅ Snappier.Tests.SnappyTests

✅ BadData_FromFile_ThrowsInvalidDataException(filename: "baddata1.snappy")
✅ BadData_FromFile_ThrowsInvalidDataException(filename: "baddata2.snappy")
✅ BadData_FromFile_ThrowsInvalidDataException(filename: "baddata3.snappy")
✅ BadData_InsufficentOutputBuffer_ThrowsArgumentException
✅ BadData_LongLength_ThrowsInvalidDataException
✅ BadData_SimpleCorruption_ThrowsInvalidDataException
✅ CompressAndDecompressFile(filename: "alice29.txt")
✅ CompressAndDecompressFile(filename: "asyoulik.txt")
✅ CompressAndDecompressFile(filename: "fireworks.jpeg")
✅ CompressAndDecompressFile(filename: "geo.protodata")
✅ CompressAndDecompressFile(filename: "html_x_4")
✅ CompressAndDecompressFile(filename: "html")
✅ CompressAndDecompressFile(filename: "kppkn.gtb")
✅ CompressAndDecompressFile(filename: "lcet10.txt")
✅ CompressAndDecompressFile(filename: "paper-100k.pdf")
✅ CompressAndDecompressFile(filename: "plrabn12.txt")
✅ CompressAndDecompressFile(filename: "urls.10K")
✅ CompressAndDecompressString(str: "")
✅ CompressAndDecompressString(str: "a")
✅ CompressAndDecompressString(str: "aaaaaaabbbbbbbbbbbbbbbbaaaaaabc")
✅ CompressAndDecompressString(str: "aaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"···)
✅ CompressAndDecompressString(str: "aaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"···)
✅ CompressAndDecompressString(str: "aaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"···)
✅ CompressAndDecompressString(str: "ab")
✅ CompressAndDecompressString(str: "abc")
✅ CompressAndDecompressString(str: "abcaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"···)
✅ DecompressToMemory
✅ RandomData
View more details on GitHub Actions