Arrangement backfill fails recovery test due to read version mapping not empty when clear

[kwannoel]

Branch to reproduce: #14930 (please fork the branch if you want to do any debugging, to make sure we always have a reproducible case).

2022-08-12T15:56:56.432961Z  INFO node{id=11 name="compute-3"}:task{id=1016977}: risingwave_storage::hummock::event_handler::hummock_event_handler: handle clear event. max_committed_epoch: 2823593286107136, max_synced_epoch: 2823593286107136, max_sealed_epoch: 2823593286107136
thread '<unnamed>' panicked at src/storage/src/hummock/event_handler/hummock_event_handler.rs:418:9:
read version mapping not empty when clear. remaining tables: [TableId { table_id: 96015 }, TableId { table_id: 99001 }, TableId { table_id: 96031 }, TableId { table_id: 96030 }, TableId { table_id: 94003 }, TableId { table_id: 96032 }, TableId { table_id: 99005 }, TableId { table_id: 98004 }, TableId { table_id: 99002 }, TableId { table_id: 94002 }, TableId { table_id: 96028 }, TableId { table_id: 96034 }, TableId { table_id: 96027 }, TableId { table_id: 98008 }, TableId { table_id: 94003 }, TableId { table_id: 95002 }, TableId { table_id: 98005 }, TableId { table_id: 94002 }, TableId { table_id: 99008 }, TableId { table_id: 98007 }, TableId { table_id: 97009 }, TableId { table_id: 99006 }, TableId { table_id: 97024 }, TableId { table_id: 94003 }, TableId { table_id: 94003 }, TableId { table_id: 95001 }, TableId { table_id: 98003 }, TableId { table_id: 94003 }, TableId { table_id: 96035 }, TableId { table_id: 99003 }, TableId { table_id: 98002 }, TableId { table_id: 96029 }, TableId { table_id: 99004 }]
stack backtrace:
   0: rust_begin_unwind
   1: core::panicking::panic_fmt
   2: risingwave_storage::hummock::event_handler::hummock_event_handler::HummockEventHandler::handle_clear
             at ./src/storage/src/hummock/event_handler/hummock_event_handler.rs:418:9
             at ./src/storage/src/hummock/event_handler/hummock_event_handler.rs:584:17
             at ./src/storage/src/hummock/event_handler/hummock_event_handler.rs:532:24
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
context: node=11 "compute-3", task=1016977 (spawned at /risingwave/src/storage/src/hummock/store/hummock_storage.rs:224:9)
note: run with `MADSIM_TEST_SEED=3` environment variable to reproduce this error

[kwannoel]

Resolved by @wenym1 's PR: #14737

[kwannoel]

Reason for failure from @wenym1:

The other non-empty read version map assertion is a previous known issue. It is caused by concurrent build_actors request and recovery request. An actor starts holding the version when it is being created, but the recovery request only stops the spawned actors, so actors created but not yet spawned will not be dropped. It won't happen after 5db21ec.