Connector: directory connector [Source/Destination]

[alarbada]

Resource name

directory

Resource link

https://en.wikipedia.org/wiki/Directory_(computing)

Connector type

Source

Expected connector behavior

We already have the file connector, so this is the logical next step. Just point to a directory and let conduit do its job.

Usecases:

• Backups mainly
• People that use something like obsidian, and want to backup all files into S3
• Move files all files from an S3 bucket into a folder.

[nickchomey]

I was looking into golang recursive directory watchers recently, so here's some info that might be helpful for this

fsnotify is probably the most popular golang filewatchers, but doesnt do recursive watching. fsnotify/fsnotify#18

frankenphp (a fantastic new golang-based php webserver, based in caddy) added such capabilities recently. They started with a PR from a contributor, then rolled their own golang watcher, then ended up using e-dant/watcher (which is also mentioned in the fsnotify thread above. Here's where they discovered e-dant and continued the discussion about and implementation of it. the e-dant dev seems to be quite happy to help others integrate it - you can see him communicating in both the fsnotify and frankenphp threads

Hope this helps

[alarbada]

Oh @nickchomey hello! I was thinking about fsnotify aswell, sad that doesn't include adding new files. I remember reading somewhere that air also had good filewatching capabilities.

I'd like to do this connector soon at some point, looks very fun to program!

[nickchomey]

air uses fsnotify, so one of the other options would probably be better.

[hariso]

FYI I've added this (and other connector requests) to our internal connectors' project where we prioritize all connector requests. We'll let you know once we figure when we're going to work on this.