Allow snapshots to be optional when recovering a persistent actor

[chrisjhoare]

Is your feature request related to a problem? Please describe.
If snapshot loading fails (for example due to serialization changes) the current behavior is to stop the actor.

akka.net/src/core/Akka.Persistence/Eventsourced.Recovery.cs

Lines 121 to 130 in 902daf5

	case LoadSnapshotFailed failed:
	timeoutCancelable.Cancel();
	try
	{
	OnRecoveryFailure(failed.Cause);
	}
	finally
	{
	Context.Stop(Self);
	}

I have an actor that derives its state from queries to a 3rd party API. Persistence snapshots help improve the start up time by skipping some of the queries needed, but in the case of a failing snapshot I would much prefer the start up took a bit longer than the actor couldn't start at all.

I could also see this being useful for event sourcing, where you'd prefer to fall back to just reading all events if snapshots could not be retrieved.

Describe the solution you'd like
Akka on the JVM already has this feature so it can be ported directly. It would be opt in configuration for the snapshot storage
https://doc.akka.io/libraries/akka-core/current//typed/persistence-snapshot.html#optional-snapshots

Describe alternatives you've considered
One thing I did consider is if this something you would want to configure at the actor level rather than per-snapshot store. For example you could make it a setting on the Recovery class which is already responsible for controlling which snapshots are retrieved for a particular actor. This would however be a departure from the JVM implementation.

Additional context
I've ported the JVM implementation as close as I can here chrisjhoare@1d66bcc

Would be happy to raise this as a PR if this is a solution you'd accept!

[to11mtm]

I've ported the JVM implementation as close as I can here chrisjhoare@1d66bcc

Would be happy to raise this as a PR if this is a solution you'd accept!

Was this ported from Akka or Pekko? And if it was from Akka, What commit?

We have to be -very- careful about making sure we do not take code that may have been derived from 'Modern' JVM Akka.

That said it's not a bad idea at all.

[chrisjhoare]

I ported from Akka. This is the commit where it was added akka/akka@e09de4f

Apologies, I hadn't thought about the change to the Akka licence (Presume that would be the issue?). I think this this is ok though looking at the date. Might be worth adding a small clarification here about porting JVM features (https://getakka.net/community/contributing/index.html#port-a-missing-feature-or-add-a-new-one)