Support for .gpkg geopackage

[arpita0911patel]

Currently, SQLITE is set to OFF as below:

#16 178.5 --     NGEN_WITH_MPI: ON
#16 178.5 --     NGEN_WITH_NETCDF: ON
#16 178.5 --     NGEN_WITH_SQLITE: OFF
#16 178.5 --     NGEN_WITH_UDUNITS: ON
#16 178.5 --     NGEN_WITH_BMI_FORTRAN: ON
#16 178.5 --     NGEN_WITH_BMI_C: ON
#16 178.5 --     NGEN_WITH_PYTHON: ON
#16 178.5 --     NGEN_WITH_ROUTING: ON
#16 178.5 --     NGEN_WITH_TESTS: ON
#16 178.5 --     NGEN_QUIET: OFF

Current behavior

Once the input data package is created with .gpkg we will have to update the functionality to add this support.

Expected behavior

NGEN_WITH_SQLITE: ON

[arpita0911patel]

@ZacharyWills @TrupeshKumarPatel @jameshalgren

[ZacharyWills]

In pursuit of updating NGIAB to assist with the ngen-datastream, I'm turning on GPKG support and adding a few other features (for example options to get example data automatically and run the detected data automatically).

[ZacharyWills]

@arpita0911patel When the NGIAB image gets rebuilt from the new ngen-deps, it should have GPKG support. You can test with ./ngen --info taking the 3 option when inside the guide script.

[ZacharyWills]

I believe that this is a cacheing issue on the runner. I have built a fresh AWS instance and run the builds there, and locally and both have this feature when run. The image build from the runner is the only image missing it.

[benlee0423]

This is what I got with awiciroh/ciroh-ngen-image:latest

bash-5.1# /dmod/bin/ngen --info
NGen version: 0.1.0
Build configuration summary:
  Generator: Unix Makefiles
  Build type: 
  System: Linux
  C Compiler: /usr/bin/cc
  CXX Compiler: /usr/bin/c++
  Flags:
    NGEN_WITH_MPI: ON
    NGEN_WITH_NETCDF: ON
    NGEN_WITH_SQLITE: OFF
    NGEN_WITH_UDUNITS: ON
    NGEN_WITH_BMI_FORTRAN: ON
    NGEN_WITH_BMI_C: ON
    NGEN_WITH_PYTHON: ON
    NGEN_WITH_ROUTING: ON
    NGEN_WITH_TESTS: ON
    NGEN_QUIET: OFF

I will do the same test once image can build in PR branch.

[benlee0423]

./ngen --info

/dmod/bin/ngen --info

[benlee0423]

It is missing sqlite header and executable.

bash-5.1# find / -name *sqlite*
/usr/local/lib64/python3.9/site-packages/pyproj.libs/libsqlite3-b8fab44b.so.0.8.6
/usr/lib64/libsqlite3.so.0.8.6
/usr/lib64/libsqlite3.so.0
/usr/lib64/python3.9/sqlite3
/usr/lib64/python3.9/lib-dynload/_sqlite3.cpython-39-aarch64-linux-gnu.so
/usr/lib64/qt5/plugins/sqldrivers/libqsqlite.so
/var/lib/rpm/rpmdb.sqlite-wal
/var/lib/rpm/rpmdb.sqlite-shm
/var/lib/rpm/rpmdb.sqlite

[benlee0423]

I found some issue with CI that misses tag-name in Dockerfiles. Those are fixed, and I build again. It fixes the issue. I verified the result below.

bash-5.1# /dmod/bin/ngen --info
NGen version: 0.1.0
Build configuration summary:
  Generator: Unix Makefiles
  Build type: 
  System: Linux
  C Compiler: /usr/bin/cc
  CXX Compiler: /usr/bin/c++
  Flags:
    NGEN_WITH_MPI: ON
    NGEN_WITH_NETCDF: ON
    NGEN_WITH_SQLITE: ON
    NGEN_WITH_UDUNITS: ON
    NGEN_WITH_BMI_FORTRAN: ON
    NGEN_WITH_BMI_C: ON
    NGEN_WITH_PYTHON: ON
    NGEN_WITH_ROUTING: ON
    NGEN_WITH_TESTS: ON
    NGEN_QUIET: OFF

I have created one PR that fixes build issue at #78.

[arpita0911patel]

Thank you Ben for getting this new image and testing it. Thank you Zach for working on this PR and getting SQLITE enabled in the image.

[arpita0911patel]

Addressed by

• Add gpkg support #79
• Ngen update #80

Testing blocked due to : #87

[arpita0911patel]

Retest once #93 is fixed.

[arpita0911patel]

Closing this ticket as it has been resolved. Worked fine on my end.

bash-5.1# /dmod/bin/ngen --info
NGen version: 0.1.0
Build configuration summary:
Generator: Unix Makefiles
Build type:
System: Linux
C Compiler: /usr/bin/cc
C Flags:
CXX Compiler: /usr/bin/c++
CXX Flags:
Flags:
NGEN_WITH_MPI: ON
NGEN_WITH_NETCDF: ON
NGEN_WITH_SQLITE: ON