bootstrapping (#176)

* thoth grpc service.

an unfortunate reformatting event, also, too

* clean up dep declarations

* Provide action upon Thoth inception.

clean up parameters/etc
dep clean up for kq/ep madness, yet again

* Inline action upon Thoth inception.

clean up close errors with KerlSpace

* decent amount of logging/clean up on thoth dynamic membership

For the moment, just disable the rebalancing test until I can pay some much needed attention to it.

* Refactor Seal to have Any message type value()

Allows Seals of arbitrary self describing protobuf messages

* Fix mispelling of proto packages - d'oh

Add leyden module, new proto services for such.

* basic skeleton of Binder and Reconciliation service infra

* MVStore

Finally a use for the ProtobufDatatype - lol

* Further flesh on the bone

* Further flesh on the bone

* The general idea.

Flesh out the remaining comms, validation strategy, etc.

Replication pattern established.

* Flesh out get/unbind API

* Woohoo! First successful replication.

* correct reconciliation logic.  track digests of bound.

* General cleanup.  Add README.md's

button up Leyden for now

* Moar cleanup. Expose dht in proc domain

* test comms cleanup

* refactor out subdomain behaviour from top level ProcessDomain with ProcessContainerDomain

* correctly attach sources

* moar cleanup

* remove

* cache?

* process domain uses the event validation of the DHT's Ani

* properly parameterize ProcessDomain

Geebus :)

* use active majority bootstrapping

* revert RingIterator bootstrap majority

* logging

* use controlled identifier directly for seed from Node

* override additional

* amp logging

* amp logging

* targeted loggin

* better config.  just model for now

* arg

* again?

* moar logging

* decouple context

* only unique

* add BootstrapVerifiers to provide key state resolution via delegation to successors in joined group

* fix compilation error

* use stereotomy msg, add BootstrapVerifiersTest

* Rename to Bootstrapper, add EventValidation functionality.  Basically Ani for bootstrapping the view member's join

* interim

cannot seem to connect to entrance server for get key state.  Weirdly intermittent connect (and subsequent fail on the retrieval) if debug/single stepping is happening.

* Handle view join failure.

Reseed if we cannot join the view.

* logging

* Bootstrapping Fireflies.

Revert back to olden daze where the EstablishmentEvent was used in ye Note. Use this as the verifier for the Participant who's Note this is.

This locks the View into the current Note of the Participant, as there's only that event as the Verifier.

Validation is used in the Gateway admission, to ensure only valid member IDs join.

* Cleanup and extend SwarmTest timeout

* moar clean up

* Use non virtual threads on inbound servers

* halve swarm test for the poor build hamsters. Moar view cleanup

* Moar view stuff

Spawn a ViewManagement.populate using the supplied initial seed set to help ensure population coverage and prevent successor loops with other members who just happened to joined in the same view.

* The problem child

* This seems to make everything all right again.

Fix/revert a lot of threading changes made that effed things up.  Scheduled execs do not work well with VThreads, so always fork the running.

* Reinstitute delegated event validation and verification to Fireflies. Segregate append from read only KERL/KEL

Revert back to EventCoordinates for Fireflies note.

* fix isolates

* Work around the Verifiers

* Minimize sig verification in replication. Re-enable sig verification in Firefly tests.

Whoooo boy.

* add keyState api for bootstrap verification

* readd Bootstrapper

* Use delegated Verifiers and EventValidation in the View

* Moar cleanup wrt scheduling

Also remove delegated verifier/validation as I don't believe that will be necessary

.github/workflows/maven.yml

@@ -1,13 +1,20 @@
 name: Java CI
 
-on: [push]
+on: [ push ]
 
 jobs:
   build:
 
     runs-on: ubuntu-latest
 
     steps:
+      - name: Cache local Maven repository
+        uses: actions/cache@v3
+        with:
+          path: ~/.m2/repository
+          key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
+          restore-keys: |
+            ${{ runner.os }}-maven-
       - name: Set up Maven
         uses: stCarolas/[email protected]
         with:

choam/src/main/java/com/salesforce/apollo/choam/CHOAM.java

@@ -11,27 +11,27 @@
 import com.google.protobuf.ByteString;
 import com.google.protobuf.InvalidProtocolBufferException;
 import com.google.protobuf.Message;
-import com.salesfoce.apollo.choam.proto.*;
-import com.salesfoce.apollo.choam.proto.SubmitResult.Result;
-import com.salesfoce.apollo.cryptography.proto.PubKey;
-import com.salesfoce.apollo.messaging.proto.AgedMessageOrBuilder;
 import com.salesforce.apollo.archipelago.RouterImpl.CommonCommunications;
 import com.salesforce.apollo.bloomFilters.BloomFilter;
 import com.salesforce.apollo.choam.comm.*;
 import com.salesforce.apollo.choam.fsm.Combine;
 import com.salesforce.apollo.choam.fsm.Combine.Merchantile;
+import com.salesforce.apollo.choam.proto.*;
+import com.salesforce.apollo.choam.proto.SubmitResult.Result;
 import com.salesforce.apollo.choam.support.*;
 import com.salesforce.apollo.choam.support.Bootstrapper.SynchronizedState;
 import com.salesforce.apollo.choam.support.HashedCertifiedBlock.NullBlock;
 import com.salesforce.apollo.cryptography.*;
 import com.salesforce.apollo.cryptography.Signer.SignerImpl;
+import com.salesforce.apollo.cryptography.proto.PubKey;
 import com.salesforce.apollo.membership.Context;
 import com.salesforce.apollo.membership.GroupIterator;
 import com.salesforce.apollo.membership.Member;
 import com.salesforce.apollo.membership.RoundScheduler;
 import com.salesforce.apollo.membership.messaging.rbc.ReliableBroadcaster;
 import com.salesforce.apollo.membership.messaging.rbc.ReliableBroadcaster.MessageAdapter;
 import com.salesforce.apollo.membership.messaging.rbc.ReliableBroadcaster.Msg;
+import com.salesforce.apollo.messaging.proto.AgedMessageOrBuilder;
 import com.salesforce.apollo.utils.Utils;
 import io.grpc.StatusRuntimeException;
 import org.h2.mvstore.MVMap;
@@ -75,9 +75,9 @@ public class CHOAM {
     private final        AtomicReference<HashedCertifiedBlock>                 genesis               = new AtomicReference<>();
     private final        AtomicReference<HashedCertifiedBlock>                 head                  = new AtomicReference<>();
     private final        ExecutorService                                       linear;
-    private final        AtomicReference<nextView> next       = new AtomicReference<>();
-    private final        AtomicReference<Digest>   nextViewId = new AtomicReference<>();
-    private final        Parameters                params;
+    private final        AtomicReference<nextView>                             next                  = new AtomicReference<>();
+    private final        AtomicReference<Digest>                               nextViewId            = new AtomicReference<>();
+    private final        Parameters                                            params;
     private final        PriorityBlockingQueue<HashedCertifiedBlock>           pending               = new PriorityBlockingQueue<>();
     private final        RoundScheduler                                        roundScheduler;
     private final        Session                                               session;
@@ -587,6 +587,10 @@ private void genesisInitialization(final HashedBlock h, final List<Transaction>
         }
     }
 
+    private String getLabel() {
+        return "CHOAM" + params.member().getId() + params.context().getId();
+    }
+
     private boolean isNext(HashedBlock next) {
         if (next == null) {
             return false;
@@ -938,10 +942,6 @@ private void synchronizedProcess(CertifiedBlock certifiedBlock) {
         pending.add(hcb);
     }
 
-    private String getLabel() {
-        return "CHOAM" + params.member().getId() + params.context().getId();
-    }
-
     public interface BlockProducer {
         Block checkpoint();
 

choam/src/main/java/com/salesforce/apollo/choam/Committee.java

@@ -15,12 +15,12 @@
 
 import org.slf4j.Logger;
 
-import com.salesfoce.apollo.choam.proto.Certification;
-import com.salesfoce.apollo.choam.proto.Reconfigure;
-import com.salesfoce.apollo.choam.proto.SubmitResult;
-import com.salesfoce.apollo.choam.proto.SubmitResult.Result;
-import com.salesfoce.apollo.choam.proto.Transaction;
-import com.salesfoce.apollo.choam.proto.ViewMember;
+import com.salesforce.apollo.choam.proto.Certification;
+import com.salesforce.apollo.choam.proto.Reconfigure;
+import com.salesforce.apollo.choam.proto.SubmitResult;
+import com.salesforce.apollo.choam.proto.SubmitResult.Result;
+import com.salesforce.apollo.choam.proto.Transaction;
+import com.salesforce.apollo.choam.proto.ViewMember;
 import com.salesforce.apollo.choam.support.HashedCertifiedBlock;
 import com.salesforce.apollo.cryptography.Digest;
 import com.salesforce.apollo.cryptography.DigestAlgorithm;
@@ -33,21 +33,19 @@
 
 /**
  * @author hal.hildebrand
- *
  */
 public interface Committee {
 
     static Map<Member, Verifier> validatorsOf(Reconfigure reconfigure, Context<Member> context) {
         return reconfigure.getJoinsList()
                           .stream()
                           .collect(Collectors.toMap(e -> context.getMember(new Digest(e.getMember().getId())),
-                                                    e -> new DefaultVerifier(publicKey(e.getMember()
-                                                                                        .getConsensusKey()))));
+                                                    e -> new DefaultVerifier(
+                                                    publicKey(e.getMember().getConsensusKey()))));
     }
 
     /**
-     * Create a view based on the cut of the supplied hash across the rings of the
-     * base context
+     * Create a view based on the cut of the supplied hash across the rings of the base context
      */
     static Context<Member> viewFor(Digest hash, Context<? super Member> baseContext) {
         Context<Member> newView = new ContextImpl<>(hash, baseContext.getRingCount(),

choam/src/main/java/com/salesforce/apollo/choam/GenesisAssembly.java

@@ -9,8 +9,8 @@
 import com.chiralbehaviors.tron.Fsm;
 import com.google.protobuf.ByteString;
 import com.google.protobuf.InvalidProtocolBufferException;
-import com.salesfoce.apollo.choam.proto.*;
-import com.salesfoce.apollo.cryptography.proto.PubKey;
+import com.salesforce.apollo.choam.proto.*;
+import com.salesforce.apollo.cryptography.proto.PubKey;
 import com.salesforce.apollo.archipelago.RouterImpl.CommonCommunications;
 import com.salesforce.apollo.choam.comm.Terminal;
 import com.salesforce.apollo.choam.fsm.Genesis;

choam/src/main/java/com/salesforce/apollo/choam/GenesisContext.java

@@ -8,7 +8,7 @@
 
 import java.util.Collections;
 
-import com.salesfoce.apollo.choam.proto.Validate;
+import com.salesforce.apollo.choam.proto.Validate;
 import com.salesforce.apollo.choam.CHOAM.BlockProducer;
 import com.salesforce.apollo.cryptography.Signer;
 import com.salesforce.apollo.cryptography.Verifier;
@@ -17,7 +17,6 @@
 
 /**
  * @author hal.hildebrand
- *
  */
 public class GenesisContext extends ViewContext {
 

choam/src/main/java/com/salesforce/apollo/choam/Parameters.java

@@ -11,10 +11,10 @@
 import com.netflix.concurrency.limits.limit.AIMDLimit;
 import com.netflix.concurrency.limits.limiter.LifoBlockingLimiter;
 import com.netflix.concurrency.limits.limiter.SimpleLimiter;
-import com.salesfoce.apollo.choam.proto.FoundationSeal;
-import com.salesfoce.apollo.choam.proto.Join;
-import com.salesfoce.apollo.choam.proto.Transaction;
-import com.salesfoce.apollo.stereotomy.event.proto.KERL_;
+import com.salesforce.apollo.choam.proto.FoundationSeal;
+import com.salesforce.apollo.choam.proto.Join;
+import com.salesforce.apollo.choam.proto.Transaction;
+import com.salesforce.apollo.stereotomy.event.proto.KERL_;
 import com.salesforce.apollo.archipelago.Router;
 import com.salesforce.apollo.choam.CHOAM.TransactionExecutor;
 import com.salesforce.apollo.choam.support.CheckpointState;
@@ -60,32 +60,36 @@ public static Builder newBuilder() {
         return new Builder();
     }
 
-    public int majority() {
-        return runtime.context.majority();
+    public Function<ULong, File> checkpointer() {
+        return runtime.checkpointer;
     }
 
-    public SigningMember member() {
-        return runtime.member;
+    public Router communications() {
+        return runtime.communications;
     }
 
     public Context<Member> context() {
         return runtime.context;
     }
 
-    public Router communications() {
-        return runtime.communications;
+    public Function<Map<Member, Join>, List<Transaction>> genesisData() {
+        return runtime.genesisData;
     }
 
-    public ChoamMetrics metrics() {
-        return runtime.metrics;
+    public Supplier<KERL_> kerl() {
+        return runtime.kerl;
     }
 
-    public Function<ULong, File> checkpointer() {
-        return runtime.checkpointer;
+    public int majority() {
+        return runtime.context.majority();
     }
 
-    public Function<Map<Member, Join>, List<Transaction>> genesisData() {
-        return runtime.genesisData;
+    public SigningMember member() {
+        return runtime.member;
+    }
+
+    public ChoamMetrics metrics() {
+        return runtime.metrics;
     }
 
     public TransactionExecutor processor() {
@@ -96,10 +100,6 @@ public BiConsumer<HashedBlock, CheckpointState> restorer() {
         return runtime.restorer;
     }
 
-    public Supplier<KERL_> kerl() {
-        return runtime.kerl;
-    }
-
     public static class MvStoreBuilder implements Cloneable {
         private int     autoCommitBufferSize = -1;
         private int     autoCompactFillRate  = -1;
@@ -580,10 +580,6 @@ public Limiter<Void> build(String name, MetricRegistry metrics) {
                                       .build();
         }
 
-        public Duration getbacklogDuration() {
-            return backlogDuration;
-        }
-
         public int getBacklogSize() {
             return backlogSize;
         }
@@ -638,6 +634,10 @@ public LimiterBuilder setTimeout(Duration timeout) {
             return this;
         }
 
+        public Duration getbacklogDuration() {
+            return backlogDuration;
+        }
+
         public LimiterBuilder setBacklogDuration(Duration backlogDuration) {
             this.backlogDuration = backlogDuration;
             return this;
@@ -679,14 +679,6 @@ public static class Builder implements Cloneable {
         private SignatureAlgorithm               viewSigAlgorithm      = SignatureAlgorithm.DEFAULT;
         private int                              crowns                = 2;
 
-        public int getCrowns() {
-            return crowns;
-        }
-
-        public void setCrowns(int crowns) {
-            this.crowns = crowns;
-        }
-
         public Parameters build(RuntimeParameters runtime) {
             return new Parameters(runtime, combine, gossipDuration, maxCheckpointSegments, submitTimeout, genesisViewId,
                                   checkpointBlockDelta, crowns, digestAlgorithm, viewSigAlgorithm,
@@ -739,6 +731,14 @@ public Builder setCombine(ReliableBroadcaster.Parameters combine) {
             return this;
         }
 
+        public int getCrowns() {
+            return crowns;
+        }
+
+        public void setCrowns(int crowns) {
+            this.crowns = crowns;
+        }
+
         public DigestAlgorithm getDigestAlgorithm() {
             return digestAlgorithm;
         }