Ingestion bug fixes

• Fixes issues where ingestion function would throw exceptions for non-data files.
• Adds better support for newly created paths, archive, and non-data files.