Op-conductor can operate several op-nodes, but what if op-conductor is down

[askucher]

Did you check the documentation and specs?

• [ *] Yes

Was there anything unclear or missing?

I have read

• https://docs.optimism.io/builders/chain-operators/tools/op-conductor
• https://www.chaindrop.com/library/high-availability-sequencing-for-op-chains-using-op-conductor

Questions:

We have a question regarding the mainnet’s security. To avoid a single point of failure due to disk issues or server downtime, you’ve introduced the op-conductor. However, what happens if the op-conductor itself fails? We considered using a load balancer and deploying multiple op-conductors, but we’re unsure how to manage the database that the op-conductor relies on. How should we back up this database, and how large might it become over time? What if we remove the database entirely and start fresh?

We haven’t found clear guidance on these points in the documentation. Could you please explain how to properly use the op-conductor in this scenario?
.
from https://www.chaindrop.com/library/high-availability-sequencing-for-op-chains-using-op-conductor :

op-conductor stores new blocks in the raft log, which means it gets replicated across the cluster. This ensures that all nodes have the same view of the chain and can start sequencing from the last block produced by the previous sequencer.

Questions:

• So, can we considered raft log as a separate alt-da?
• Why op-conductor cannot work with alt-da, why it's needs the own raft log
• What happens if we have lost db, how op-conductor can sync it again?
• Why not just make p2p communication between op-node and if master op-node is down then blocks can be produced by another node?

It's quite obvious design

op-node1 (producing blocks)
op-node2 (health check of node1)
op-node3 (health check of node2)

it's also the raft consensus

• Also if it solves the problem of centralized op-node, what about op-proposer, op-batcher ? logically all of them should be distributed.

Honestly, I am thinking that regular private ETH network with custom bridge can be considered more fault protected than l2/l3

Did you check for duplicate questions?

• [ *] Yes

Similar Questions or Issues Found

#313

Please select the type of request

• Bug Report
• [ *] Feature Request

Steps to Reproduce

Just try to make some logical questions and try to find answers in documentation

Expected vs. Actual Behaviour

Op-conductor should be launched on several servers too
Op-conductor should work with alt-da

Software Versions

v1.9.1

Operating System

Any

Hardware Resources

Any

Configuration Files, Startup Flags & Environment Variables

op-conductor enabled

Error Messages / Logs

it's not an error, but design question

Feature Description

What if conductor is down?

Purpose and Benefits

We need to be able to make l2,l3 without single point of failures

Relevant Context or Examples

Docs

[hash1go]

Hello. Thank you for posting on the forum.
I'd like to confirm - is this a question or feedback?

[askucher]

There are few questions. Please read the text, all questions inside

[hash1go]

Understood!
Here are the answers.

So, can we considered raft log as a separate alt-da?

The raft log is not an alt-da. It's an internal consensus mechanism that the op-conductor uses. In the case of Rollups, the single state is derived from transaction data stored on Ethereum.

Why op-conductor cannot work with alt-da, why it's needs the own raft log

The op-conductor is an auxiliary service that uses Raft consensus for leader election and state management in high-availability setups. It does not function as an alt-da because its purpose is operational, not data storage or propagation.

What happens if we have lost db, how op-conductor can sync it again?

The op-conductor stores its Raft log in a persistent volume and is designed to be deployed as a statefulset in production environments. If the database is lost, the cluster needs to be reinitialized or rebuilt. However, because Rollups derive their single state from Ethereum transaction data, losing the op-conductor database will not result in data loss for the Rollup chain itself.

Why not just make p2p communication between op-node and if master op-node is down then blocks can be produced by another node?

The op-conductor is designed to provide deterministic guarantees, such as No Unsafe Reorgs and 100% Uptime with No More Than 1 Node Failure. While a P2P approach could allow nodes to communicate directly, it does not inherently provide the same level of deterministic leader election and log consistency as Raft. Raft ensures that all nodes have a consistent view of the chain state and prevents unsafe block production during network partitions, which is critical for high availability.

Also if it solves the problem of centralized op-node, what about op-proposer, op-batcher ? logically all of them should be distributed.

This is my opinion, but considering the goal of providing trustless blockspace, the logical distribution of these components is a lower priority. While it makes sense for L1 to focus on liveness as one of its security characteristics, liveness is not as high a priority for L2.

[askucher]

Thank you for the detailed answer. But do we need to restore the raft log when op-conductor is rebuilt? And what happens when conductor is down? Who is leader?

[askucher]

@hash1go

[hash1go]

I have forwarded your inquiry to the OP team. Please wait for a while.
As it's the holiday season, it may take some time to get a response.
cc: @sbvegan