V21.0RC1

CHANGELOG

Release V21.0RC1

Full Changelog

Major Changes

Pull Request	Title
#2545	Sideband upgrade using epoch byte for is_send/receive/epoch
#2487	IPC 2.0
#2535	Election state refactor
#2446	Node telemetry

---

Protocol Changes

Pull Request	Title
#2667	Validate work difficulty during ledger processing
#2683	Increase minimum supported protocol version to 17
#2701	Dont peer with v20 and earlier after epoch 2 block is seen

---

Node Configuration Updates

Pull Request	Title
#2588	LMDB sync options and new config settings
#2724	Clarify nano_pow_server configs are not in use
#2727	Safely read override values when no config file is present
#2787	Double default bandwidth limit config

---

RPC Updates

Pull Request	Title
#2689	BREAKING Change response for RPC work_validate with implicit difficulty
#2671	BREAKING Handle epoch_2 work thresholds in the wallet and most RPCs
#2499	BREAKING Bootstrap attempts and connections/pulls separation
#2560	Request telemetry data for local node
#2569	Work version concept
#2619	Election refactor follow up
#2684	Epoch 2 started flag in ledger cache
#2703	Difficulty calculation for RPC block_create
#2704	Asynchronous epoch upgrade RPC
#2718	Epoch upgrader as an async task
#2752	Result difficulty in RPC block_create
#2753	Flood difficulty updates from RPC process
#2754	Optional "block" given to RPC "work_generate" to infer difficulty
#2804	Make RPC name consistent with stats, websocket & protocol naming for telemetry

---

IPC Updates

Pull Request	Title
#2644	Add Flatbuffers schema evolution rules to IDL

---

Websocket Updates

Pull Request	Title
#2562	Fake websocket client to test the node websocket server
#2566	Incremental options for ws confirmation subscription
#2634	Add telemetry response websocket callbacks
#2729	Websocket new_unconfirmed_block
#2734	Websocket notification for RPC work_generate without peers
#2774	Enable WebSocket server by default in Docker image

---

CLI Updates

Pull Request	Title
#2579	Configurable inactive votes cache size
#2575	Add flag for vote_processor capacity and tests
#2606	Launch flag --allow_bootstrap_peers_duplicates
#2594	Allow CLI --config values for inactive node tests
#2637	Read config file for CLI commands
#2707	Add difficulty and multiplier to CLI work generation commands
#2719	CLI compare_rep_weights to compare ledger and hardcoded weights
#2725	CLI command for a frontier confirmation speed test
#2749	Multithreaded --validate_blocks
#2769	Deprecate --batch_size/debug_mass_acitvity CLI options
#2794	CLI commands incorrect ledger cache setup

---

Deprecation/Removal

Pull Request	Title
#2555	Disable UDP by default

---

Developer/Debug Options

Pull Request	Title
#2540	Lower beta network work threshold to 1/64x base
#2591	friendly backtraces in actions
#2731	New stats for elections
#2746	Fix Xcode warnings
#2762	Add cemented block log timings
#2791	Fix insufficient work logging
#2793	Add logging section to rpc config to optionally disable logging
#2802	Clarify election difficulty update log message

---

Fixed Bugs

Pull Request	Title
#2527	Allow rep crawler targets with ephemeral ports
#2530	Remove representatives with closed channels
#2546	Restore max block processor signature verification batch size
#2553	Inactive votes cache confirmation status
#2563	Fix election calling confirm_if_quorum after destruction
#2582	Check against op aborted on secure rpc acceptor shutdown
#2593	Start vote generator for changed winner only if voting is enabled
#2609	[RocksDB] Only delete from unchecked when existence is guaranteed
#2648	Fix wrong number of representatives in confirmation solicitor
#2650	Telemetry results not correctly utilising cache timeouts
#2659	Signature checker blocking fix
#2675	Prevent getting stuck in block processor flush
#2674	Modify inactive cache to prevent multiple insertions for confirmed
#2652	Consistently add conflicting block to election
#2696	Remove assert if a delayed work cache request is not found
#2651	Using relaxed atomics for counts not involved in control flow in conf height processor
#2706	Prevent more rare deadlocks due to races for condition variables
#2664	Incorrect cemented count during conf height algo transition
#2723	Some confirmed block observer callbacks being missed
#2733	Remove invalid uses of epoch_1 work threshold
#2744	Tally votes on conflicting block with no inactive votes
#2766	Votes from local representatives should not be flooded on processing
#2767	Fix previous balance in active_transaction::insert ()
#2781	Union std::hash coverage
#2782	Stuck uncemented blocks after heavy load
#2796	Retrieve block when activating dependencies
#2799	Slow test fixes
#2800	Fix open blocks not getting activated through dependencies

---

Implemented Enhancements

Pull Request	Title
#2522	Optionally disable data in responses to telemetry requests with CLI option
#2518	Active_transactions updates blocks when updating difficulty.
#2524	Extend telemetry data with more node versioning details
#2528	Improve initial TCP server keepalive
#2512	Cache genesis block hash in ledger constants
#2536	Cache hash for multiple block->hash () calls
#2531	Bounded memory and redesign in the confirmation height processor
#2541	Handle legacy confirm_req using the aggregator
#2521	Poll all nodes, remove outliers, ddos protection & amend RPC response with endpoint
#2543	Probabilistic network packet filter
#2552	Encapsulating election::confirmed so its implementation can vary.
#2548	Add wallet-processed block to work watcher via the block processor
#2554	[ASAN] Access node through a weak_ptr on distributed_work dtor
#2551	Return created election when adding to active_transactions
#2542	Separate inactive votes cache from gap cache
#2557	Add launch flag --disable_block_processor_unchecked_deletion
#2571	Use only the necessary protocols in UPnP
#2559	Buffer drop policies
#2567	Add GSL-style narrow_cast
#2572	Move back timer comments to header
#2544	Use a different confirmation height algorithm when ledger is almost fully cemented
#2574	Add tests for vote_processor
#2534	Log stable filename
#2583	warnings: Remove some unused locals and captures
#2568	Output stacktrace with custom debug assert
#2585	Remove vote cache & generate new vote if election winner is changed
#2573	Add peer timestamp to telemetry responses
#2592	[RocksDB] Limit write locks to necessary tables
#2589	Reworking confirmation_height.dependent_election test
#2590	Remove dropped_election_cache in preparation for election refactor.
#2599	Block work version
#2577	Minimize work validation calls
#2600	Reduce active mutex locking with election winner details
#2602	Don't use active mutex lock during confirmation solicitor.prepare ()
#2549	Aggressive flooding for local blocks
#2601	Block difficulty and work validation cleanup
#2605	Utility: nano::optional_ptr
#2608	Lock before stopping when notifying other threads
#2598	Simplify telemetry data processing
#2616	Update preconfigured_peers comment regarding port
#2596	Attach sideband to block
#2614	Simplify request aggregator mutex lock behavior
#2620	Use the sideband when available in ledger.is_send
#2626	Improve --debug_profile_bootstrap performance
#2604	Update adjusted difficulty in batches
#2640	Read transaction scope in active
#2581	Use attempts list for TCP channels
#2646	Bandwidth considerations following election refactor
#2641	Increase active elections capacity with periodic full checks
#2643	Network duplicate filter for publish messages
#2649	Erase representatives with full queues when adding to confirmation solicitor
#2653	Include requesting telemetry metrics from temporary channels
#2570	Parallelize state block signature verification with block processor
#2631	Delay inactive/gap cache bootstrap start for 30 seconds
#2625	Improve confirmation consistency
#2661	Increase minimum time to log block processing
#2665	Output current function in assert diagnostics
#2618	Sign telemetry messages
#2638	Definitions for work thresholds with epoch_2
#2663	Check if a vote is for a recently confirmed block
#2662	Move all request aggregator operations out of the mutex hold scope
#2610	Remove telemetry message versions
#2505	Directed block broadcasting for long elections
#2603	Remove peers with different genesis block or invalid telemetry signature
#2676	Optimize mutex access when adding blocks to block processor
#2666	Delay voting for non-priority elections under saturation
#2672	Refactor work thresholds as nano::work_thresholds
#2627	Optimize vote post-processing operations
#2686	Improve automatic frontiers confirmation
#2680	Delay wallet work caching to allow using lower difficulty on demand
#2690	Work version parameter in default_difficulty and use it in more places
#2693	Move excluded_peers to network
#2692	Wrapper for RPC worker tasks
#2673	Epoch open blocks should have corresponding pending entries
#2688	Move vote generator calls into election code
#2645	Rate limiting using token buckets
#2694	Prevent reconnecting to excluded peers with sufficient score
#2691	Active difficulty normalization
#2705	Ensure max_work_generate_difficulty is updated when changing the default difficulty
#2669	Improve telemetry request/response under load
#2702	Vote generator session for batch insertions
#2709	Ensure propagation and removal for the work watcher
#2710	Difficulty updates for elections with multiple blocks
#2714	Improve batching of writes in unbounded conf height processor
#2613	Move TCP messages processing to network threads
#2720	Separate election state for the broadcasting block fallback
#2721	Remove confirmation requests for a new representative
#2716	Release write_guard lock when no longer required
#2715	Allow restarting elections with higher work
#2728	Add active difficulty to node telemetry
#2730	Use a multi-index container to allow fifo queue for pending confirmations
#2751	Serialize telemetry as big endian
#2759	Perform wallet representative action without holding any mutex
#2772	Republishing a vote to principal representatives
#2768	Force node exit if ledger inconsistency in the conf height processor is found
#2565	Push front blocks from unchecked
#2784	Confirmation requests and broadcasts if available vote is for a conflicting block
#2778	Bisected election backtracking
#2797	Check executable paths in load_test
#2785	Sequential voting
#2801	Sequential elections

---

Build, Test, Automation, & Chores

Pull Request	Title
#2525	Define GCC sanitizers builds
#2537	Update beta network bootstrap weights for v21, cutoff 7M
#2538	ASAN error with database transaction tracker json serialization
#2547	BOOST_1.69
#2550	guard policy setting by version checks
#2561	Disable node.fork_open_flip test on Windows actions
#2564	Update --debug_profile_process CLI test
#2556	Update --debug_profile_bootstrap CLI test
#2576	Fix intermittent node telemetry tests with tsan/valgrind
#2584	tsan: race in telemetry::ongoing_single_request_cleanup
#2586	[TSAN] confirmation_height.cemented_gap_below_receive test
#2587	[TSAN] vote_processor.flush test
#2580	Improve confirmation_solicitor.batches test
#2595	Fix rpc telemetry test timestamps
#2597	Use C++17 locally, C++14 on CI
#2607	CMakeLists backwards compatibility
#2612	Fix intermittent send_node_id_handshake unit test failures
#2617	update url for sourceforge
#2622	Host qt assets on s3
#2623	[RocksDB] Tests not reading account count from store correctly
#2621	Fix confirmation_height.gap_live intermittent test failure
#2624	confirmation_height.modified_chain test fails on a non-debug build
#2615	tsan fix: rpc.wallet_destroy
#2628	Make confirmation_solicitor.batches more robust under tsan
#2630	Make network.replace_port more robust under tsan
#2633	Add ASSERT_TIMELY
#2636	split docker artifacts to separate job
#2632	Fix request_aggregator unit tests
#2647	gather sha256 hashes of artifacts and upload to s3 with artifacts
#2655	Update tick count in timer::update
#2656	Fix node.aggressive_flooding
#2660	typo on windows sha256 Out-File
#2670	clang included now
#2677	Fix intermittent wallet.work_watcher_update failure
#2682	Fix intermittent failure in active_transactions.confirmation_consistency
#2687	set timeout to 1hr to catch tests that hang
#2695	Fix qt test wallet.seed_work_generation
#2697	updates for fuzzer
#2699	Fix qt tests failing to click radio buttons
#2711	[TSAN] lock order inversion in active transactions / wallet
#2712	Add node sequence for tests
#2708	Fix failure in ledger.work_validation due to random work being above threshold
#2700	Add Security Policy file
#2698	[TSAN] start_time data race in bootstrap_client
#2611	Preparation for building with shared boost
#2717	Fix logic in active_transactions.prioritize_chains test
#2722	changelog_generator refactored as changelog.py
#2732	record_rep_weights to py3
#2735	Allow starting more that max_peers_per_ip test nodes
#2737	Fix intermittently failing rpc.confirmation_height_currently_processing test
#2738	Fix intermittent node_telemetry.remove_peer_different_genesis test
#2739	Fix rpc.wallet_history failures
#2742	Fix system.generate_send_new intermittent failures
#2745	Fix intermittently failing conflicts.adjusted_multiplier test
#2743	Fix node.fork_invalid_block_signature intermittent failures, re-enable on windows CI
#2748	Fix minor test-specific intermittent failures
#2750	Fix network.tcp_no_connect_excluded_peers test failure
#2757	[TSAN] race for system in test websocket.bootstrap_exited
#2760	Fix intermittent failure in test wallet.work_cache_delayed
#2779	Set boost min to 1.69
#2783	enable shared boost for tests
#2786	[TSAN] Fix off-by-one in socket.drop_policy test
#2790	use full cache for PR's requesting modules not currently specified in…
#2792	update bundled FindBoost.cmake
#2789	Epoch2 signer
#2803	add label "logging" to Developer/Debug Options Sections