Proof of concept: Synchronous ds <-> dstopic IPC

[zkat]

One big challenge for the ds/dstopic divide is this idea that there will be a single, long-lived dstopic binary that ds will then communicate with to request data be warmed up. But this comes with a tricky bit: There’s no easy way to do synchronous IPC in Node, and require is a synchronous operation.

So here’s a challenge: We need at -least- a proof of concept of how to send requests from ds to dstopic synchronously. This includes a way to wait for dstopic to be done with its task and send a success response back before the call is over.

Hint: one way to possibly do this is by using fs.writeSync/fs.readSync with a local socket of some sort. I forget the details, but that’s the first avenue I’d explore to accomplish this.

(original thread)

[zkat]

cc @ysz are you still interested in giving this a shot and coming back to this where you left off?

[daubaris]

@zkat isn't dstopic archived?

[zkat]

@daubaris it lives in this repo now!

[daubaris]

@zkat Oh, got it. I would like to tackle it (if possible of course (and would need to familiarize myself with the codebase))!

[zkat]

Please go ahead! That would be really awesome! Let me know if you have any questions, it's a bit of a hard problem

[daubaris]

@zkat I was thinking about creating a separate RPC that would communicate locally via http. But I might be naive and not understanding how the it should work (lack of docs).

In addition, dstropic is a rust client for entropic and ds is the entropic client. What is the difference (besides language)? Are there any docs that I could read just to get to know the project and its working parts?