run conflux v2.3.0 , but blocks are out of sync

[xt-com]

Hi everyone.
I'm running a full node and right now the blocks are out of sync,
Calling rpc cfx_epochNumber always returns: {"jsonrpc":"2.0","result":"0x4b6350f","id":1}
Look at the running log as follows. Please help me find out what is causing it and how can I fix it? Thanks

os: ubuntu:20.04
conflux :
https://github.com/Conflux-Chain/conflux-rust/releases/download/v2.3.0/conflux_linux_glibc2.27_x64_v2.3.0.zip

`curl -H "Content-Type:application/json" -X POST -d '{"jsonrpc":"2.0","method":"cfx_epochNumber","params":[],"id":1}' http://localhost:12539

{"jsonrpc":"2.0","result":"0x4b6350f","id":1}
`

log-info:

2023-09-10T09:05:55.511460591+00:00 WARN Socket IO Worker #0 cfxcore::syn - Error while handling message, peer=0x48b3…23e1, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:55.693788435+00:00 INFO http.worker3570 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:05:55.798457665+00:00 WARN Socket IO Worker #0 cfxcore::syn - Error while handling message, peer=0x7b46…1c79, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:55.901340342+00:00 INFO http.worker8600 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:05:55.901743760+00:00 INFO http.worker8640 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:05:55.930133820+00:00 WARN Socket IO Worker #0 cfxcore::syn - Error while handling message, peer=0x2b72…a34f, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:56.000781465+00:00 WARN Socket IO Worker #2 cfxcore::syn - Error while handling message, peer=0xc65a…6efd, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:56.049173082+00:00 WARN Socket IO Worker #3 cfxcore::syn - Error while handling message, peer=0xb7ae…f44e, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:56.467967982+00:00 INFO IO Worker #3 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:05:56.572951901+00:00 WARN Socket IO Worker #1 cfxcore::syn - Error while handling message, peer=0x28d3…f81b, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:56.689739225+00:00 INFO http.worker4940 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:05:56.792249924+00:00 WARN Socket IO Worker #3 cfxcore::syn - Error while handling message, peer=0x48b3…23e1, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:56.901315093+00:00 INFO http.worker8860 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:05:56.901364127+00:00 INFO http.worker7020 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:05:56.919500872+00:00 WARN Socket IO Worker #1 cfxcore::syn - Error while handling message, peer=0xc22a…7efb, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:56.996287067+00:00 WARN Socket IO Worker #2 cfxcore::syn - Error while handling message, peer=0x4d5f…8f2b, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:57.028152454+00:00 WARN Socket IO Worker #3 cfxcore::syn - Error while handling message, peer=0xfea7…048b, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:57.132367428+00:00 WARN Socket IO Worker #1 cfxcore::syn - Error while handling message, peer=0x838e…7cd8, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:57.182869774+00:00 WARN Socket IO Worker #2 cfxcore::syn - Error while handling message, peer=0x70db…5433, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:57.405207755+00:00 WARN Socket IO Worker #3 cfxcore::syn - Error while handling message, peer=0x07fa…9b64, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:57.468848289+00:00 INFO IO Worker #2 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:05:57.687895614+00:00 INFO http.worker5740 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:05:57.901503512+00:00 INFO http.worker8500 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:05:57.905333224+00:00 INFO http.worker3660 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:05:58.010110499+00:00 WARN Socket IO Worker #1 cfxcore::syn - Error while handling message, peer=0x2cbd…9d06, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:58.104513539+00:00 WARN Socket IO Worker #2 cfxcore::syn - Error while handling message, peer=0xe7f1…c2a0, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })
2023-09-10T09:05:58.104550860+00:00 WARN Socket IO Worker #0 cfxcore::syn - Error while handling message, peer=0xcac5…a891, msgid=6, error=Error(InvalidBlock, State { next_error: None, backtrace: InternalBacktrace { backtrace: None } })

[xt-com]

Sorry,
Now the log output becomes as follows, but the block height returned by RPC is always
{"jsonrpc":"2.0","result":"0x4b6350f","id":1}

log-info:

2023-09-10T09:27:50.438111390+00:00 INFO IO Worker #0 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:50.687493910+00:00 INFO http.worker600 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:50.901232676+00:00 INFO http.worker1730 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:50.901513981+00:00 INFO http.worker3430 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:51.438638185+00:00 INFO IO Worker #1 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:51.692193300+00:00 INFO http.worker3480 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:51.901414574+00:00 INFO http.worker8210 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:51.901489747+00:00 INFO http.worker5350 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:52.439348891+00:00 INFO IO Worker #3 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:52.687679508+00:00 INFO http.worker9140 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:52.901434926+00:00 INFO http.worker9530 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:52.903211840+00:00 INFO http.worker9010 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:53.440258526+00:00 INFO IO Worker #0 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:53.689152706+00:00 INFO http.worker2410 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:53.901490895+00:00 INFO http.worker3450 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:53.901576899+00:00 INFO http.worker1170 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:54.441111853+00:00 INFO IO Worker #1 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:54.687627374+00:00 INFO http.worker1740 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:54.901640863+00:00 INFO http.worker30 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:54.901730894+00:00 INFO http.worker9960 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:55.441772794+00:00 INFO IO Worker #3 cfxcore::sta - Statistics: StatisticsInner { sync_graph: SyncGraphStatistics { inserted_block_count: 42, inserted_header_count: 263629 }, consensus_graph: ConsensusGraphStatistics { inserted_block_count: 170189, activated_block_count: 170188, processed_block_count: 170194 } }
2023-09-10T09:27:55.441875480+00:00 INFO IO Worker #2 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:55.687734350+00:00 INFO http.worker2550 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:55.901451164+00:00 INFO http.worker7340 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:55.901571564+00:00 INFO http.worker8270 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:56.442605663+00:00 INFO IO Worker #3 cfxcore::syn - Catch-up mode: true, latest epoch: 79049999 missing_bodies: 0
2023-09-10T09:27:56.687700879+00:00 INFO http.worker2400 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-10T09:27:56.901418645+00:00 INFO http.worker4150 client::rpc: - RPC Request: eth_blockNumber()
2023-09-10T09:27:56.905250412+00:00 INFO http.worker710 client::rpc: - RPC Request: eth_blockNumber()

[peilun-conflux]

Are you replacing the binary version to v2.3.0 after the hardfork height 79050000? If that's the case, you'll need to clean the database and resync the node completely (possibly using the snapshot file https://github.com/conflux-fans/archive-tool).

[xt-com]

Are you replacing the binary version to v2.3.0 after the hardfork height 79050000? If that's the case, you'll need to clean the database and resync the node completely (possibly using the snapshot file https://github.com/conflux-fans/archive-tool).

Thanks. peilun-conflux

block height:79220679
Then it went out of sync again.
The log is as follows:
2023-09-14T04:03:27.736658831+00:00 INFO http.worker3490 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:27.903811395+00:00 INFO http.worker4290 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:27.903969386+00:00 INFO http.worker2350 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:28.540656177+00:00 INFO http.worker4010 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:28.553817572+00:00 INFO http.worker5270 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:28.689212526+00:00 INFO http.worker7590 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:28.902865578+00:00 INFO http.worker9620 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:28.904034805+00:00 INFO http.worker1480 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:28.905097008+00:00 INFO http.worker5010 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:29.321226628+00:00 INFO http.worker4090 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:29.688735592+00:00 INFO http.worker6060 client::rpc: - RPC Request: cfx_getBlocksByEpoch epoch_number=Num(79050052)
2023-09-14T04:03:29.690146940+00:00 INFO http.worker5430 client::rpc: - RPC Request: cfx_getBlockByHash hash=0xe966ae588b00815078c7a15e9979848753224e44aa2975a52bea887424f593ad include_txs=true
2023-09-14T04:03:29.754964442+00:00 INFO http.worker5650 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:29.775853484+00:00 INFO http.worker6840 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:29.901867762+00:00 INFO http.worker2980 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:29.905692757+00:00 INFO http.worker2460 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:30.698313056+00:00 INFO http.worker4290 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:30.787766857+00:00 INFO http.worker9320 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:30.901723522+00:00 INFO http.worker5710 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:30.902555607+00:00 INFO http.worker2840 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:31.692424516+00:00 INFO http.worker1660 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:31.901885986+00:00 INFO http.worker600 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:31.909062974+00:00 INFO http.worker9110 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:32.498152739+00:00 INFO http.worker8170 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:32.689739598+00:00 INFO http.worker7290 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:32.903400405+00:00 INFO http.worker6990 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:32.907230219+00:00 INFO http.worker2690 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:32.913656397+00:00 INFO http.worker6620 client::rpc: - RPC Request: eth_blockNumber()
^X2023-09-14T04:03:33.310212471+00:00 INFO http.worker3960 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:33.688056968+00:00 INFO http.worker510 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:33.806614305+00:00 INFO http.worker8200 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:33.830865751+00:00 INFO http.worker8260 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:33.902054617+00:00 INFO http.worker2890 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:33.905678679+00:00 INFO http.worker9720 client::rpc: - RPC Request: eth_blockNumber()
^X2023-09-14T04:03:34.718744345+00:00 INFO http.worker4880 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:34.797619719+00:00 INFO http.worker5290 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:34.902024113+00:00 INFO http.worker3600 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:34.902323032+00:00 INFO http.worker490 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:34.915707152+00:00 INFO http.worker4790 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:35.313002131+00:00 INFO http.worker4150 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:35.689241261+00:00 INFO http.worker5340 client::rpc: - RPC Request: cfx_epochNumber(LatestMined)
2023-09-14T04:03:35.770425804+00:00 INFO http.worker850 client::rpc: - RPC Request: eth_getBlockByNumber block_number=Num(79050006) include_txs=true
2023-09-14T04:03:35.904666709+00:00 INFO http.worker8970 client::rpc: - RPC Request: eth_blockNumber()
2023-09-14T04:03:35.908493157+00:00 INFO http.worker1560 client::rpc: - RPC Request: eth_blockNumber()