Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update performance tracker and rewards #318

Merged
merged 73 commits into from
Sep 21, 2023
Merged
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
5864664
Update perf factor calculation, to watch for sub-slots activity, add …
daria305 Aug 14, 2023
c95f679
Merge branch 'develop' into feat/rewards
daria305 Aug 14, 2023
dfb7a1c
Move VersionAndHash to iota.go
daria305 Aug 16, 2023
15c3acc
Performance tracker stores versionAndHash, use serix instead marshalutil
daria305 Aug 16, 2023
acb4e0a
Merge branch 'develop' into feat/rewards
daria305 Aug 16, 2023
50be3f7
Update profit margin calc and related params
daria305 Aug 17, 2023
beb36dc
Reevaluate and update pool reward calculations
daria305 Aug 18, 2023
721520f
Clean up validator and delegator rewards calculations
daria305 Aug 18, 2023
0d3c30e
Handle no activity in a slot
daria305 Aug 21, 2023
45d0224
Move validators params to API parameters
daria305 Aug 21, 2023
3d97618
Merge branch 'develop' into feat/rewards2
daria305 Aug 22, 2023
3a71880
Add ActivityWindowDuration param for activity tracking in committee s…
daria305 Aug 22, 2023
7e61052
Merge branch 'develop' into feat/rewards
daria305 Aug 23, 2023
5238cee
Track activity and registered validators draft
daria305 Aug 23, 2023
82558ad
Add missing store
daria305 Aug 23, 2023
9197fe2
Safemath, safemath everywhere
daria305 Aug 24, 2023
968fab8
Test fixes, handle nil performance factor
daria305 Aug 24, 2023
ef8e074
Update rewards tests
daria305 Aug 25, 2023
4117caa
Revert candidate activity tracking in performanceTracker
daria305 Aug 30, 2023
23e0f93
Rewards test changes
daria305 Aug 30, 2023
f9f6143
Merge branch 'develop' into feat/rewards
daria305 Aug 30, 2023
24fc560
After merge fixes and updates for validatorPerformance prunable storage
daria305 Aug 30, 2023
8d2a821
Merge branch 'develop' into feat/rewards
daria305 Aug 31, 2023
74f21ec
Fix compilation errors
daria305 Aug 31, 2023
959d950
Fix test: use validation blocks
daria305 Aug 31, 2023
88adf16
Fix test: missing serix tags
daria305 Aug 31, 2023
2956cd2
Fix overflow in target reward
daria305 Aug 31, 2023
2ee1599
Solve TODOs: limit rewards export loop to the latest pruned epoch
daria305 Aug 31, 2023
30801d5
Fix delegation reward API: endEpoch is filled in only for delayed cla…
daria305 Aug 31, 2023
8f3fde4
Remove nonexisting param from presets
daria305 Aug 31, 2023
d2efabb
Fix: poolRewards goes for delegators only if fixed cost is too high
daria305 Aug 31, 2023
887e5a6
Update reward test to cover some edge cases
daria305 Aug 31, 2023
e651f8f
Fix reward overflow, with updating param, and revert previous changes
daria305 Aug 31, 2023
74c27cd
Fix mistake in profit margin factor calc
daria305 Aug 31, 2023
0aa10be
Update test scenarios
daria305 Aug 31, 2023
62b7da2
Fix dropping off in test
daria305 Aug 31, 2023
0a5f1f5
Merge branch 'develop' into feat/rewards
daria305 Aug 31, 2023
3a753ca
Use model.VersionAndHash
daria305 Aug 31, 2023
6d79ead
Update iotago go mod ref
daria305 Aug 31, 2023
2fe2a68
Merge remote-tracking branch 'origin/develop' into feat/rewards
karimodm Sep 4, 2023
70ef2e8
Point to compatible iota.go
karimodm Sep 4, 2023
ecd809c
Address review comments
karimodm Sep 4, 2023
3f4b813
Make doggo happy
karimodm Sep 4, 2023
00988e6
serix tags on version and hash for performance factors snapshot
karimodm Sep 5, 2023
9d83d6d
codestyle
karimodm Sep 5, 2023
1279d46
Start subslot indexing from 0 to properly fill the performance bitmap
karimodm Sep 5, 2023
6b0585e
Method ordering, exported first
karimodm Sep 5, 2023
db2eede
Update pkg/protocol/sybilprotection/sybilprotectionv1/performance/rew…
karimodm Sep 5, 2023
9bba504
Use safe left shift from safemath
karimodm Sep 6, 2023
6113fda
Shift epoch's performance factor by SlotsPerEpochExponent
karimodm Sep 6, 2023
178f30e
Code style
karimodm Sep 6, 2023
1a840fc
Bumped iota.go: compute initial and final rewards only once
karimodm Sep 6, 2023
994a25d
Merge remote-tracking branch 'origin/develop' into feat/rewards
karimodm Sep 6, 2023
ed8a47b
go.mod
karimodm Sep 6, 2023
617b4f9
Merge remote-tracking branch 'origin/develop' into feat/rewards
karimodm Sep 7, 2023
a262931
Update with merged iota.go
karimodm Sep 7, 2023
5b98921
Merge branch 'develop' into feat/rewards
jkrvivian Sep 15, 2023
f01f1cc
Update iota.go version
jkrvivian Sep 15, 2023
12b476c
Fix epochPerformanceFactor in test
jkrvivian Sep 15, 2023
604bdb1
Clean up and do not set issuedCount more than needed
daria305 Sep 19, 2023
ca1f893
Clean up the code according to review
daria305 Sep 19, 2023
99edf33
Propagate errors instead of panic
daria305 Sep 19, 2023
67bf7b5
Merge branch 'develop' into feat/rewards
daria305 Sep 19, 2023
57a2745
Comment should end with a period.
daria305 Sep 19, 2023
39d97dd
Merge remote-tracking branch 'origin/develop' into feat/rewards
karimodm Sep 20, 2023
ba02048
SlotsPerEpochExponent moved to TimeProvider
karimodm Sep 20, 2023
513beb1
Merge branch 'feat/rewards' of github.com:iotaledger/iota-core into f…
karimodm Sep 20, 2023
f0d891e
Fix validator punishment, rewards should be slashed only for one epoch
daria305 Sep 20, 2023
be09a2f
Update fixed cost subtraction in poolRewards
daria305 Sep 20, 2023
084afe9
Rewards are saved with fixed cost included
daria305 Sep 21, 2023
29afbb9
Merge branch 'develop' into feat/rewards
daria305 Sep 21, 2023
672197b
Update go mod
daria305 Sep 21, 2023
02376ee
Point to correct iota.go
karimodm Sep 21, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions components/restapi/core/accounts.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,6 @@ import (
"github.com/iotaledger/iota.go/v4/nodeclient/apimodels"
)

const (
DefaultPageSize = 50
RequestsMemoryCacheGranularity = 10
MaxRequestedSlotAge = 10
)

func congestionForAccountID(c echo.Context) (*apimodels.CongestionResponse, error) {
accountID, err := httpserver.ParseAccountIDParam(c, restapipkg.ParameterAccountID)
if err != nil {
Expand Down Expand Up @@ -79,13 +73,13 @@ func validators(c echo.Context) (*apimodels.ValidatorsResponse, error) {
}

// do not respond to really old requests
if requestedSlotIndex+MaxRequestedSlotAge < latestCommittedSlot {
if requestedSlotIndex+iotago.SlotIndex(restapi.ParamsRestAPI.MaxRequestedSlotAge) < latestCommittedSlot {
return nil, ierrors.Errorf("request is too old, request started at %d, latest committed slot index is %d", requestedSlotIndex, latestCommittedSlot)
}

nextEpoch := deps.Protocol.APIForSlot(latestCommittedSlot).TimeProvider().EpochFromSlot(latestCommittedSlot) + 1

slotRange := uint32(requestedSlotIndex / RequestsMemoryCacheGranularity)
slotRange := uint32(requestedSlotIndex) / restapi.ParamsRestAPI.RequestsMemoryCacheGranularity
registeredValidators, exists := deps.Protocol.MainEngineInstance().Retainer.RegisteredValidatorsCache(slotRange)
if !exists {
registeredValidators, err = deps.Protocol.MainEngineInstance().SybilProtection.OrderedRegisteredCandidateValidatorsList(nextEpoch)
Expand Down Expand Up @@ -175,11 +169,17 @@ func rewardsByOutputID(c echo.Context) (*apimodels.ManaRewardsResponse, error) {
case iotago.OutputDelegation:
//nolint:forcetypeassert
delegationOutput := utxoOutput.Output().(*iotago.DelegationOutput)
latestCommittedSlot := deps.Protocol.MainEngineInstance().SyncManager.LatestCommitment().Index()
stakingEnd := delegationOutput.EndEpoch
// the output is in delayed calaiming state if endEpoch is set, otherwise we use latest possible epoch
if delegationOutput.EndEpoch == 0 {
stakingEnd = deps.Protocol.APIForSlot(latestCommittedSlot).TimeProvider().EpochFromSlot(deps.Protocol.MainEngineInstance().SyncManager.LatestCommitment().Index())
}
reward, actualStart, actualEnd, err = deps.Protocol.MainEngineInstance().SybilProtection.DelegatorReward(
delegationOutput.ValidatorID,
delegationOutput.DelegatedAmount,
delegationOutput.StartEpoch,
delegationOutput.EndEpoch,
stakingEnd,
)
}
if err != nil {
Expand Down
4 changes: 4 additions & 0 deletions components/restapi/params.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@ type ParametersRestAPI struct {
AllowIncompleteBlock bool `default:"false" usage:"whether the node allows to fill in incomplete block and issue it for user"`
// MaxPageSize defines the maximum number of results per page.
MaxPageSize uint32 `default:"100" usage:"the maximum number of results per page"`
// RequestsMemoryCacheGranularity defines per how many slots a cache is created for big API requests.
RequestsMemoryCacheGranularity uint32 `default:"10" usage:"defines per how many slots a cache is created for big API requests"`
// MaxRequestedSlotAge defines the maximum age of a request that will be processed.
MaxRequestedSlotAge uint32 `default:"10" usage:"the maximum age of a request that will be processed"`

JWTAuth struct {
// salt used inside the JWT tokens for the REST API. Change this to a different value to invalidate JWT tokens not matching this new value
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ require (
github.com/iotaledger/hive.go/stringify v0.0.0-20230906114834-b50190b9f9c2
github.com/iotaledger/inx-app v1.0.0-rc.3.0.20230829161228-3f4eb50a4d14
github.com/iotaledger/inx/go v1.0.0-rc.2.0.20230829160617-69b96c7c9f9b
github.com/iotaledger/iota.go/v4 v4.0.0-20230906064556-75f5be378284
github.com/iotaledger/iota.go/v4 v4.0.0-20230907173112-861f7ed7eed6
github.com/labstack/echo/v4 v4.11.1
github.com/labstack/gommon v0.4.0
github.com/libp2p/go-libp2p v0.30.0
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,8 @@ github.com/iotaledger/inx/go v1.0.0-rc.2.0.20230829160617-69b96c7c9f9b h1:EPB/+i
github.com/iotaledger/inx/go v1.0.0-rc.2.0.20230829160617-69b96c7c9f9b/go.mod h1:B7gyJP6GshCSlEmY3CxEk5TZdsMs3UNz5U92hkFDdMs=
github.com/iotaledger/iota.go/v4 v4.0.0-20230906064556-75f5be378284 h1:SmPyo08i2/ymMZkmIQX/7EexuWWFl7CKDZjJr47zn8Q=
github.com/iotaledger/iota.go/v4 v4.0.0-20230906064556-75f5be378284/go.mod h1:MM3RLtTEsfT6Wh0EhpgmzVO/HM0/NOw+E7+mnGTnyA0=
github.com/iotaledger/iota.go/v4 v4.0.0-20230907173112-861f7ed7eed6 h1:kKSImID+de7k27+1BcQejmb0s1cTfruauvmPfY5xVf4=
github.com/iotaledger/iota.go/v4 v4.0.0-20230907173112-861f7ed7eed6/go.mod h1:MM3RLtTEsfT6Wh0EhpgmzVO/HM0/NOw+E7+mnGTnyA0=
github.com/ipfs/boxo v0.10.0 h1:tdDAxq8jrsbRkYoF+5Rcqyeb91hgWe2hp7iLu7ORZLY=
github.com/ipfs/boxo v0.10.0/go.mod h1:Fg+BnfxZ0RPzR0nOodzdIq3A7KgoWAOWsEIImrIQdBM=
github.com/ipfs/go-cid v0.4.1 h1:A/T3qGvxi4kpKWWcPC/PgbvDA2bjVLO7n4UeVwnbs/s=
Expand Down
37 changes: 37 additions & 0 deletions pkg/model/validator_performance.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
package model

import (
iotago "github.com/iotaledger/iota.go/v4"
)

type ValidatorPerformance struct {
// works if ValidatorBlocksPerSlot is less than 32 because we use it as bit vector
SlotActivityVector uint32 `serix:"0"`
// can be uint8 because max count per slot is maximally ValidatorBlocksPerSlot + 1
BlockIssuedCount uint8 `serix:"1"`
HighestSupportedVersionAndHash VersionAndHash `serix:"2"`
}

func NewValidatorPerformance() *ValidatorPerformance {
return &ValidatorPerformance{
SlotActivityVector: 0,
BlockIssuedCount: 0,
HighestSupportedVersionAndHash: VersionAndHash{},
}
}

func ValidatorPerformanceFromBytes(decodeAPI iotago.API) func([]byte) (*ValidatorPerformance, int, error) {
return func(bytes []byte) (*ValidatorPerformance, int, error) {
validatorPerformance := new(ValidatorPerformance)
consumedBytes, err := decodeAPI.Decode(bytes, validatorPerformance)
if err != nil {
return nil, 0, err
}

return validatorPerformance, consumedBytes, nil
}
}

func (p *ValidatorPerformance) Bytes(api iotago.API) ([]byte, error) {
return api.Encode(p)
}
4 changes: 2 additions & 2 deletions pkg/model/version_and_hash.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ import (
const VersionAndHashSize = iotago.IdentifierLength + iotago.VersionLength

type VersionAndHash struct {
Version iotago.Version
Hash iotago.Identifier
Version iotago.Version `serix:"0"`
Hash iotago.Identifier `serix:"1"`
}

func (v VersionAndHash) Bytes() ([]byte, error) {
Expand Down
Loading