Skip to content
This repository has been archived by the owner on Apr 1, 2023. It is now read-only.

Handle persistence failure better #177

Open
iafuture opened this issue May 19, 2021 · 0 comments
Open

Handle persistence failure better #177

iafuture opened this issue May 19, 2021 · 0 comments

Comments

@iafuture
Copy link
Contributor

Currently, we just hard-stop the process if we detect persistence failure:

    return _persistence->flush()
        .then([resp=std::move(resp)] (auto&& flushStatus) mutable {
            if (!flushStatus.is2xxOK()) {
                K2LOG_E(log::skvsvr, "Persistence failed with status {}", flushStatus);
                // TODO gracefully fail to aid in faster recovery.
                seastar::engine().exit(1);
            }

            K2LOG_D(log::skvsvr, "persistence flush succeeded. Sending response to client");
            return seastar::make_ready_future<std::tuple<Status, ResponseT>>(std::move(resp));
        });

We should handle this more gracefully, probably going via Monitor to the CPO so that we can take cluster-wide action if needed

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant