-
Notifications
You must be signed in to change notification settings - Fork 590
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Discussion: removing or combining ports used in the standalone mode #12086
Comments
I just think we don't have to expose such many ports. More or less, it will ease the pain to deploy the standalone mode. |
Add some comments from @BugenZhao :
|
Something else I thought about: Even though meta, CN and frontend share the same process, is it possible that other nodes will need to communicate directly to meta / frontend via gRPC? In that case we need to handle routing logic, and the RPC messages need to be tagged. |
As for refactoring work, I think the main hurdle is gRPC, I'm not sure how tightly coupled our system is with it, whether it's as simple as not starting the gRPC service. It would require some investigation on my part. Other refactoring work would be just adding a flag to each rw node, to indicate they are executing in standalone mode, and do not need to start gRPC service. |
I guess avoiding exposing unnecessary ports should be enough. There's not much benefit for thoroughly removing them and switching to an in-memory channel for most RPC calls, as explained above. |
As the connector node is going to be embedded into CN, will it still rely on gRPC to communicate with CN? Or will it be totally replace with JNI? |
Probably replace with JNI. |
There would be 2 phases to replace the connector node with the embedded one. In the first phase, only part of RPCs is replaced. But in the end, all of them would be JNI. |
This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned. |
Just to move some discussion in the Slack to Github
Right now, the standalone mode opens dozens of ports. And for cloud deployment, we'd like to reduce the number of ports. cc @arkbriar could you elaborate more on the benefits of having fewer ports in the standalone mode?
For Prometheus, I suppose three ports can be combined into one only.
And for gRPC communication among meta, frontend, and compute, we probably can do the same as they used to be different processes but now they are all in the same process.
How much work does it take?
Which set of ports are to be combined/removed most easily first?
The text was updated successfully, but these errors were encountered: