[indexer][writer] Add first_tx_sequence_number to epochs table to dec…

…ouple from checkpoints table (#19773) ## Description The `epoch_total_transactions` field on `epochs` table is calculated today from `checkpoints.network_total_transactions`, which is not ideal since the latter is prunable and `epochs` will not be pruned for the foreseeable future. To remove this dependency, we add `first_tx_sequence_number` to the `epochs` table at epoch boundary. That is, the network total transaction count from the final checkpoint of the epoch becomes the first tx sequence number of the new epoch. This also means that at epoch boundary, the current-to-be-previous epoch's `epoch_total_transactions` is derived from the checkpoint's network total transactions - the epoch's `first_tx_sequence_number`. Consequently, this will also help in the pruner implementation, as given an epoch we'd like to know the corresponding cp and tx. This encompasses just the writer change. Before updating the read path, we will need to backfill the instance. ## Test plan How did you test the new or updated feature? --- ## Release notes Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required. For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates. - [ ] Protocol: - [ ] Nodes (Validators and Full nodes): - [ ] Indexer: - [ ] JSON-RPC: - [ ] GraphQL: - [ ] CLI: - [ ] Rust SDK: - [ ] REST API:
MystenLabs · Oct 16, 2024 · 6cc663c · 6cc663c
1 parent 71f9203
commit 6cc663c
Show file tree

Hide file tree

Showing 15 changed files with 638 additions and 240 deletions.
diff --git a/crates/sui-graphql-e2e-tests/tests/stable/epoch/epoch_start_to_end.exp b/crates/sui-graphql-e2e-tests/tests/stable/epoch/epoch_start_to_end.exp
@@ -0,0 +1,231 @@
+processed 13 tasks
+
+init:
+C: object(0,0)
+
+task 1, line 6:
+//# create-checkpoint
+Checkpoint created: 1
+
+task 2, line 8:
+//# advance-epoch
+Epoch advanced: 0
+
+task 3, lines 10-12:
+//# programmable --sender C --inputs 10000000000 @C
+//> SplitCoins(Gas, [Input(0)]);
+//> TransferObjects([Result(0)], Input(1));
+created: object(3,0)
+mutated: object(0,0)
+gas summary: computation_cost: 1000000, storage_cost: 1976000,  storage_rebate: 0, non_refundable_storage_fee: 0
+
+task 4, lines 14-16:
+//# programmable --sender C --inputs 5000000000 @C
+//> SplitCoins(Gas, [Input(0)]);
+//> TransferObjects([Result(0)], Input(1));
+created: object(4,0)
+mutated: object(0,0)
+gas summary: computation_cost: 1000000, storage_cost: 1976000,  storage_rebate: 978120, non_refundable_storage_fee: 9880
+
+task 5, line 18:
+//# run 0x3::sui_system::request_add_stake --args object(0x5) object(3,0) @validator_0 --sender C
+events: Event { package_id: sui_system, transaction_module: Identifier("sui_system"), sender: C, type_: StructTag { address: sui_system, module: Identifier("validator"), name: Identifier("StakingRequestEvent"), type_params: [] }, contents: [135, 141, 242, 35, 38, 24, 124, 195, 86, 219, 178, 127, 110, 40, 201, 151, 112, 169, 166, 183, 93, 180, 71, 210, 141, 37, 35, 151, 110, 94, 69, 29, 218, 131, 22, 109, 1, 175, 215, 221, 207, 138, 245, 248, 68, 244, 90, 170, 83, 244, 133, 72, 229, 17, 124, 35, 245, 162, 151, 140, 253, 66, 34, 68, 252, 204, 154, 66, 27, 187, 19, 193, 166, 106, 26, 169, 143, 10, 215, 80, 41, 237, 233, 72, 87, 119, 156, 105, 21, 180, 79, 148, 6, 139, 146, 30, 1, 0, 0, 0, 0, 0, 0, 0, 0, 228, 11, 84, 2, 0, 0, 0] }
+created: object(5,0)
+mutated: object(_), 0x0000000000000000000000000000000000000000000000000000000000000005, object(0,0)
+deleted: object(3,0)
+gas summary: computation_cost: 1000000, storage_cost: 15078400,  storage_rebate: 1956240, non_refundable_storage_fee: 19760
+
+task 6, line 20:
+//# create-checkpoint
+Checkpoint created: 3
+
+task 7, lines 22-72:
+//# run-graphql
+Response: {
+  "data": {
+    "epoch": {
+      "epochId": 1,
+      "referenceGasPrice": "1000",
+      "validatorSet": {
+        "totalStake": "20000000000000000",
+        "activeValidators": {
+          "nodes": [
+            {
+              "name": "validator-0"
+            }
+          ]
+        }
+      },
+      "startTimestamp": "1970-01-01T00:00:00Z",
+      "totalCheckpoints": 0,
+      "totalTransactions": null,
+      "totalGasFees": null,
+      "totalStakeRewards": null,
+      "totalStakeSubsidies": null,
+      "fundSize": "0",
+      "netInflow": null,
+      "fundInflow": null,
+      "fundOutflow": null,
+      "storageFund": {
+        "totalObjectStorageRebates": "0",
+        "nonRefundableBalance": "0"
+      },
+      "safeMode": {
+        "enabled": false
+      },
+      "systemStateVersion": 2,
+      "systemParameters": {
+        "stakeSubsidyStartEpoch": 0
+      },
+      "systemStakeSubsidy": {
+        "balance": "9949400000000000000",
+        "currentDistributionAmount": "1000000000000000"
+      },
+      "checkpoints": {
+        "nodes": [
+          {
+            "sequenceNumber": 3
+          }
+        ]
+      },
+      "transactionBlocks": {
+        "nodes": [
+          {
+            "digest": "HqhN74u19oG1oVeEjLJx9Z5RdYxT7XWoGYHMmM1QNQ2q"
+          }
+        ]
+      },
+      "endTimestamp": null
+    }
+  }
+}
+
+task 8, line 74:
+//# create-checkpoint
+Checkpoint created: 4
+
+task 9, line 76:
+//# advance-epoch
+Epoch advanced: 1
+
+task 10, line 78:
+//# create-checkpoint
+Checkpoint created: 6
+
+task 11, lines 80-130:
+//# run-graphql
+Response: {
+  "data": {
+    "epoch": {
+      "epochId": 1,
+      "referenceGasPrice": "1000",
+      "validatorSet": {
+        "totalStake": "20000000000000000",
+        "activeValidators": {
+          "nodes": [
+            {
+              "name": "validator-0"
+            }
+          ]
+        }
+      },
+      "startTimestamp": "1970-01-01T00:00:00Z",
+      "totalCheckpoints": 2,
+      "totalTransactions": 4,
+      "totalGasFees": "3000000",
+      "totalStakeRewards": "3000000",
+      "totalStakeSubsidies": "0",
+      "fundSize": "0",
+      "netInflow": "16096040",
+      "fundInflow": "19030400",
+      "fundOutflow": "2934360",
+      "storageFund": {
+        "totalObjectStorageRebates": "0",
+        "nonRefundableBalance": "0"
+      },
+      "safeMode": {
+        "enabled": false
+      },
+      "systemStateVersion": 2,
+      "systemParameters": {
+        "stakeSubsidyStartEpoch": 0
+      },
+      "systemStakeSubsidy": {
+        "balance": "9949400000000000000",
+        "currentDistributionAmount": "1000000000000000"
+      },
+      "checkpoints": {
+        "nodes": [
+          {
+            "sequenceNumber": 5
+          }
+        ]
+      },
+      "transactionBlocks": {
+        "nodes": [
+          {
+            "digest": "BjA5MhqTJWEkp2gyPhBBBYcER2nYoYevdGSmagrrvVRb"
+          }
+        ]
+      },
+      "endTimestamp": "1970-01-01T00:00:00Z"
+    }
+  }
+}
+
+task 12, lines 132-182:
+//# run-graphql
+Response: {
+  "data": {
+    "epoch": {
+      "epochId": 2,
+      "referenceGasPrice": "1000",
+      "validatorSet": {
+        "totalStake": "20000010003000000",
+        "activeValidators": {
+          "nodes": [
+            {
+              "name": "validator-0"
+            }
+          ]
+        }
+      },
+      "startTimestamp": "1970-01-01T00:00:00Z",
+      "totalCheckpoints": 0,
+      "totalTransactions": null,
+      "totalGasFees": null,
+      "totalStakeRewards": null,
+      "totalStakeSubsidies": null,
+      "fundSize": "16096040",
+      "netInflow": null,
+      "fundInflow": null,
+      "fundOutflow": null,
+      "storageFund": {
+        "totalObjectStorageRebates": "16066400",
+        "nonRefundableBalance": "29640"
+      },
+      "safeMode": {
+        "enabled": false
+      },
+      "systemStateVersion": 2,
+      "systemParameters": {
+        "stakeSubsidyStartEpoch": 0
+      },
+      "systemStakeSubsidy": {
+        "balance": "9949400000000000000",
+        "currentDistributionAmount": "1000000000000000"
+      },
+      "checkpoints": {
+        "nodes": [
+          {
+            "sequenceNumber": 6
+          }
+        ]
+      },
+      "transactionBlocks": {
+        "nodes": []
+      },
+      "endTimestamp": null
+    }
+  }
+}