HPCC-32875 Ensure rowservice can process fields stored as blobs

[ghalliday]

Type of change:

• This change is a bug fix (non-breaking change which fixes an issue).
• This change is a new feature (non-breaking change which adds functionality).
• This change improves the code (refactor or other change that does not change the functionality)
• This change fixes warnings (the fix does not alter the functionality or the generated code)
• This change is a breaking change (fix or feature that will cause existing behavior to change).
• This change alters the query API (existing queries will have to be recompiled)

Checklist:

• My code follows the code style of this project.
  • My code does not create any new warnings from compiler, build system, or lint.
• The commit message is properly formatted and free of typos.
  • The commit message title makes sense in a changelog, by itself.
  • The commit is signed.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly, or...
  • I have created a JIRA ticket to update the documentation.
  • Any new interfaces or exported functions are appropriately commented.
• I have read the CONTRIBUTORS document.
• The change has been fully tested:
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • I have checked that this change does not introduce memory leaks.
  • I have used Valgrind or similar tools to check for potential issues.
• I have given due consideration to all of the following potential concerns:
  • Scalability
  • Performance
  • Security
  • Thread-safety
  • Cloud-compatibility
  • Premature optimization
  • Existing deployed queries will not be broken
  • This change fixes the problem, not just the symptom
  • The target branch of this pull request is appropriate for such a change.
• There are no similar instances of the same problem that should be addressed
  • I have addressed them here
  • I have raised JIRA issues to address them separately
• This is a user interface / front-end modification
  • I have tested my changes in multiple modern browsers
  • The component(s) render as expected

Smoketest:

• Send notifications about my Pull Request position in Smoketest queue.
• Test my draft Pull Request.

Testing:

[github-actions]

Jira Issue: https://hpccsystems.atlassian.net//browse/HPCC-32875

Jirabot Action Result:
Assigning user: [email protected]
Workflow Transition To: Merge Pending
Updated PR

[AttilaVamos]

Perhaps it is not relatd, but unittests crashed on this change:

Testing two processes with a port collision on an instance...Test Complete.
/home/runner/work/_temp/2f68e2d3-8d1f-443d-942a-5bbba156eb39.sh: line 3:  4969 Segmentation fault      sudo /opt/HPCCSystems/bin/unittests

and the core trace shows:

Core was generated by `/opt/HPCCSystems/bin/unittests -e IKeyManagerTest'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f0b688135fe in RtlRecord::RtlRecord(RtlFieldInfo const* const*, bool) () from /opt/HPCCSystems/lib/libeclrtl.so

Thread 1 (Thread 0x7f0b610e8300 (LWP 39553)):
#0  0x00007f0b688135fe in RtlRecord::RtlRecord(RtlFieldInfo const* const*, bool) () from /opt/HPCCSystems/lib/libeclrtl.so
#1  0x00007f0b68813e00 in COutputMetaData::queryRecordAccessor(bool) const () from /opt/HPCCSystems/lib/libeclrtl.so
#2  0x00007f0b68db52e9 in ?? () from /opt/HPCCSystems/lib/libjhtree.so
#3  0x00007f0b68dc27f9 in IKeyManagerTest::testKeys() () from /opt/HPCCSystems/lib/libjhtree.so
#4  0x00007f0b6b9c6fa2 in CppUnit::TestCaseMethodFunctor::operator()() const () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#5  0x00007f0b6b9bcf0e in CppUnit::DefaultProtector::protect(CppUnit::Functor const&, CppUnit::ProtectorContext const&) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#6  0x00007f0b6b9c4110 in CppUnit::ProtectorChain::protect(CppUnit::Functor const&, CppUnit::ProtectorContext const&) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#7  0x00007f0b6b9ce794 in CppUnit::TestResult::protect(CppUnit::Functor const&, CppUnit::Test*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#8  0x00007f0b6b9c6c59 in CppUnit::TestCase::run(CppUnit::TestResult*) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#9  0x00007f0b6b9c7233 in CppUnit::TestComposite::doRunChildTests(CppUnit::TestResult*) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#10 0x00007f0b6b9c72b7 in CppUnit::TestComposite::run(CppUnit::TestResult*) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#11 0x00007f0b6b9ce6f2 in CppUnit::TestResult::runTest(CppUnit::Test*) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#12 0x00007f0b6b9d0f9d in CppUnit::TestRunner::run(CppUnit::TestResult&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#13 0x00007f0b6b9d2ff2 in CppUnit::TextTestRunner::run(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, bool, bool) () from /opt/HPCCSystems/lib/libcppunit-1.15.so.1
#14 0x000000000044c0e3 in ?? ()
#15 0x00007f0b630ccd85 in __libc_start_main () from /lib64/libc.so.6
#16 0x000000000044e8be in ?? ()

[ghalliday]

It almost certainly is related. I will debug.

[ghalliday]

Fixed a bug in the unittest, and improved the error checking.

[jakesmith]

@ghalliday - looks sensible, 1 question.

[jakesmith]

is it necessary to check in dtor in case of exception? or will they be cleared up as the index is released anyway?

[jakesmith]

@ghalliday - looks good.

[github-actions]

Jirabot Action Result:
Added fix version: 9.6.62
Added fix version: 9.8.36
Workflow Transition: 'Resolve issue'