ZenFS vs streamich's libraries (e.g. memfs)

[vassudanagunta]

I've been researching libraries that provide a unified file system API (not necessarily Node fs compatible) that works across a variety of backends, e.g. local file system, browser-based file systems, remote file systems, etc.

The one with the most usage that supports the Node fs API seems to be the set of libs by @streamich.

• memfs has 13 million weekly NPM downloads and 9 million repos and 2000 packages that depend on it on GitHub, including many with a lot of stars.
• some compelling example demos, including three ways to run git (in browser on local fs, in browser on OPFS, entirely in memory)

I haven't had time to really explore it, or ZenFS or its predecessor BrowserFS, but I figure the author of ZenFS some of its existing users might have some insight.

At the very least, if I were writing ZenFS I'd want to know about @streamich's libraries and what they do well. I've tagged streamich as they may want to comment as well.

[james-pre]

Briefly looking through some of the memfs code, it looks like there are many parallels. I find it interesting that the way both handle Stats with bigints is similar and yet developed independently.

As for the Git demos, ZenFS and BrowserFS are fully compatible with isomorphic-git (see isomorphic-git#1910)

A nice bonus is that every ZenFS backend supports synchronous operations.

Here are the major differences as far as I know:

Thing	ZenFS	memfs	BrowserFS
Integration with other APIs (FS Access, IndexedDB, etc.)	backends	volumes	backends with FileSystem classes
Node FS emulation	functional	object-oriented, only partial support	object-oriented
Stability/status	Mostly stable, no 1.0.0 release yet	Stable	Deprecated
Configuration/steup	configure function with Backend options	creating instances of *NodeFs classes directly, use as is	configure function with a callback
Support for multiple FSes at the same time	Yes, no additional config needed	No	Yes, though a MoutableFileSystem must be configured
unified internal API	FileSystem	none	FileSystem
Popularity: downloads	729	12,907,104	9,814
Popularity: stars	52 (entire organization)	1.6k	3k

API integrations:

API integration	ZenFS	memfs	BrowserFS
in-memory	✓	✓	✓
File System Access	✓	✓	✓
Content-addressable storage	✓2	✓	✓2
Indexed DB	✓	✗	✓
ZIP archives	✓	✗	✓
ISO images	✓	✗	✓
HTTP Requests	✓	✗	✓
Copy-on-write readonly overlay	✓	✗	✓
CRUD1	✓	✗ (async only)	✓
Storage	✓	✗	✗ (localStorage only)
Worker	✓	✗	✗ (WebWorker only)
MessagePort	✓	✗	✗

¹Create, Read, Update, Delete
²using SyncStore/AsyncStore

Please correct me if I make a mistake with memfs on the tables.

In my opinion:

• The way ZenFS and BrowserFS handle API integrations is better, since they use a unified internal API which all of the different API integrations use rather than converting to and from the in-memory API calls.
• ZenFS is much more flexible with how you can configure and extend it.
• memfs has become so popular due to performance and also the timing with when it was released.

I hope I've been of some help, feel free to ask more questions and I will do my best to answer them.

[vassudanagunta]

Thanks for that extensive reply!

I myself have developed my own file system abstraction layer, one that by design does not emulate the Node fs API. I think there are both pros and cons to Node fs emulation. If I decide the pros outweigh the cons, or that the cons are not even significant or real, then I need to decide whether memFS or ZenFS is better.

From your table ZenFS seems clearly better, but it is created by the ZenFS author, so can be biased, or unaware of memFS advantages. I'm hoping that @streamich sees this and replies as well. Maybe over time other users who have experience with both will weigh in. At the moment, this discussion is the only thing across all of GitHub that mentions both "streamich" and "ZenFS".

I would much rather use ZenFS or memFS or some other existing, well-maintained project than continue to develop my own. I will likely have more questions as I do this evaluation.

Again, thank you!

[james-pre]

@vassudanagunta:

I think there are both pros and cons to Node fs emulation.

Handling all of the different function signatures in the emulation is a pain, though everything else I find to be enjoyable to work with.

From your table ZenFS seems clearly better, but it is created by the ZenFS author, so can be biased, or unaware of memFS advantages.

I agree, and I would prefer if you choose ZenFS =). In all seriousness, I tried to be as unbiased as possible, though I did include some opinions at the end.

At the moment, this discussion is the only thing across all of GitHub that mentions both "streamich" and "ZenFS".

ZenFS is relatively new, only being 2 months old, so I'm not surprised. I hope that it can become popular though, especially considering that the fs module ZenFS exports is sub-type compatible with Node's fs module. This means that they are interchangeable to Typescript, so you can actually use ZenFS's exported fs exactly like Node's fs as a namespace import (i.e. import * as fs from ...).