diff --git a/.github/workflows/code_freeze.yml b/.github/workflows/code_freeze.yml index 1bb74feee50..efe1d6e8856 100644 --- a/.github/workflows/code_freeze.yml +++ b/.github/workflows/code_freeze.yml @@ -9,4 +9,4 @@ jobs: steps: - name: Fail if Code Freeze is enabled run: | - exit 1 + exit 0 diff --git a/doc/releasenotes/16_0_0_changelog.md b/doc/releasenotes/16_0_0_changelog.md index 53b503bc6e1..0849e143a05 100644 --- a/doc/releasenotes/16_0_0_changelog.md +++ b/doc/releasenotes/16_0_0_changelog.md @@ -1,4 +1,4 @@ -# Changelog of Vitess v16.0.0-rc1 +# Changelog of Vitess v16.0.0 ### Announcement #### Governance @@ -21,9 +21,17 @@ * Fix files changes filtering in CI [#11714](https://github.com/vitessio/vitess/pull/11714) * Fix `codeql` workflow timeout issue [#11760](https://github.com/vitessio/vitess/pull/11760) * Trigger OnlineDDL workflow when test data changes [#11827](https://github.com/vitessio/vitess/pull/11827) +#### CLI + * [release-16.0] vtctldclient: Format GetKeyspace output using cli.MarshalJSON (#12495) [#12503](https://github.com/vitessio/vitess/pull/12503) #### Cluster management * BugFix: Errant GTID detection for a single replica [#12024](https://github.com/vitessio/vitess/pull/12024) * BugFix: Fix race in `IsFlagProvided` [#12042](https://github.com/vitessio/vitess/pull/12042) +#### Examples + * Examples: Add lib functions to wait for shard states [#12239](https://github.com/vitessio/vitess/pull/12239) + * Examples: Correct VTAdmin Discovery File Path And Add Check [#12415](https://github.com/vitessio/vitess/pull/12415) + * examples: Fix Vtop example and release script [#12440](https://github.com/vitessio/vitess/pull/12440) + * Fix vtctldclient command in vtop test script and the fix operator example [#12458](https://github.com/vitessio/vitess/pull/12458) + * Open vtadmin-api port in docker-local example to enable vtadmin access [#12467](https://github.com/vitessio/vitess/pull/12467) #### General * [main] Stats Flags: include stats flags in the correct binaries (#11450) [#11453](https://github.com/vitessio/vitess/pull/11453) * Test flags: Update logic for parsing test flags to run unit tests within GoLand and to parse test flags in vtgate to allow running unit tests [#11551](https://github.com/vitessio/vitess/pull/11551) @@ -32,7 +40,8 @@ #### Observability * stats/prometheus: normalize labels for single-label implementations [#12057](https://github.com/vitessio/vitess/pull/12057) #### Online DDL - * Parse binlog variable length encoded columns [#11871](https://github.com/vitessio/vitess/pull/11871) + * Parse binlog variable length encoded columns [#11871](https://github.com/vitessio/vitess/pull/11871) + * Allow zero (in) date when setting up internal _vt schema [#12262](https://github.com/vitessio/vitess/pull/12262) #### Operator * [main] Increase the memory limit of the vitess-operator (#11548) [#11550](https://github.com/vitessio/vitess/pull/11550) * Fix rbac config in the vtop example [#12034](https://github.com/vitessio/vitess/pull/12034) @@ -85,13 +94,22 @@ * VTGate: Ensure HealthCheck Cache Secondary Maps Stay in Sync With Authoritative Map on Tablet Delete [#12178](https://github.com/vitessio/vitess/pull/12178) * schemadiff: fix scenario where no tables exist in schema and with just views reading from DUAL [#12189](https://github.com/vitessio/vitess/pull/12189) * Fix parsing of bitnum values larger than 64 bit [#12191](https://github.com/vitessio/vitess/pull/12191) - * Online DDL: fix deadlock by releasing mutex before calling callback [#12211](https://github.com/vitessio/vitess/pull/12211) + * Online DDL: fix deadlock by releasing mutex before calling callback [#12211](https://github.com/vitessio/vitess/pull/12211) + * fix: transaction_isolation to be applied at session level [#12281](https://github.com/vitessio/vitess/pull/12281) + * Fix aggregation on outer joins [#12298](https://github.com/vitessio/vitess/pull/12298) + * [release-16.0] fix: added null safe operator precendence rule (#12297) [#12307](https://github.com/vitessio/vitess/pull/12307) + * Fix for USING when column names not lower cased [#12379](https://github.com/vitessio/vitess/pull/12379) + * [release-16.0] Fix bug in vtexplain around JOINs (#12376) [#12384](https://github.com/vitessio/vitess/pull/12384) + * Fix scalar aggregation engine primitive for column truncation [#12468](https://github.com/vitessio/vitess/pull/12468) + * [release-16.0] BugFix: Unsharded query using a derived table and a dual table [#12484](https://github.com/vitessio/vitess/pull/12484) #### VReplication * VReplication: escape identifiers when renaming source tables [#11670](https://github.com/vitessio/vitess/pull/11670) * VReplication: Prevent Orphaned VDiff2 Jobs [#11768](https://github.com/vitessio/vitess/pull/11768) * VDiff2: Properly Apply --only_pks Report Flag [#12025](https://github.com/vitessio/vitess/pull/12025) * VReplication: Improve Error/Status Reporting [#12052](https://github.com/vitessio/vitess/pull/12052) - * VReplication: Propagate Binlog Stream Errors [#12095](https://github.com/vitessio/vitess/pull/12095) + * VReplication: Propagate Binlog Stream Errors [#12095](https://github.com/vitessio/vitess/pull/12095) + * VReplication: ignore GC tables in schema analysis [#12320](https://github.com/vitessio/vitess/pull/12320) + * Sidecar schema init: use COPY algorithm while altering sidecardb tables [#12436](https://github.com/vitessio/vitess/pull/12436) #### VTorc * Fix VTOrc holding locks after shutdown [#11442](https://github.com/vitessio/vitess/pull/11442) * [15.0] Fix VTOrc to handle multiple failures [#11489](https://github.com/vitessio/vitess/pull/11489) @@ -129,7 +147,8 @@ * consolidating OnlineDDL 'declarative' tests into 'scheduler' tests: part 1 [#12061](https://github.com/vitessio/vitess/pull/12061) * OnlineDDL CI: remove 'revertible' tests (part 2) [#12192](https://github.com/vitessio/vitess/pull/12192) * Update vtadmin dependencies [#12201](https://github.com/vitessio/vitess/pull/12201) - * Update Go dependencies [#12215](https://github.com/vitessio/vitess/pull/12215) + * Update Go dependencies [#12215](https://github.com/vitessio/vitess/pull/12215) + * Download mariadb from vitess-resources [#12271](https://github.com/vitessio/vitess/pull/12271) #### Cluster management * Endtoend cluster improvements [#11859](https://github.com/vitessio/vitess/pull/11859) * CI, tabletmanager throttler topo tests: polling until status received [#12107](https://github.com/vitessio/vitess/pull/12107) @@ -143,7 +162,8 @@ * Add more codeowners to the `/test` directory [#11762](https://github.com/vitessio/vitess/pull/11762) #### Query Serving * Consistent sorting in Online DDL Vrepl suite test [#11821](https://github.com/vitessio/vitess/pull/11821) - * Flakes: Properly Test HealthCheck Cache Response Handling [#12226](https://github.com/vitessio/vitess/pull/12226) + * Flakes: Properly Test HealthCheck Cache Response Handling [#12226](https://github.com/vitessio/vitess/pull/12226) + * Fix `SHOW VSCHEMA TABLES` tests using v17 vtgate that expected `dual` [#12381](https://github.com/vitessio/vitess/pull/12381) #### TabletManager * Update throttler-topo workflow file [#11784](https://github.com/vitessio/vitess/pull/11784) * Fix closing the body for HTTP requests [#11842](https://github.com/vitessio/vitess/pull/11842) @@ -152,6 +172,9 @@ #### VTorc * Move vtorc runners back to normal github runners [#11482](https://github.com/vitessio/vitess/pull/11482) ### Dependabot +#### Build/CI + * [release-16.0] Bump golang.org/x/net from 0.5.0 to 0.7.0 (#12390) [#12400](https://github.com/vitessio/vitess/pull/12400) + * [release-16.0] Update additional Go dependencies (#12401) [#12402](https://github.com/vitessio/vitess/pull/12402) #### Java * build(deps): Bump protobuf-java from 3.19.4 to 3.19.6 in /java [#11439](https://github.com/vitessio/vitess/pull/11439) #### VTAdmin @@ -167,6 +190,8 @@ #### Documentation * Upgrades the release notes for v15.0.0 [#11567](https://github.com/vitessio/vitess/pull/11567) * Copy design docs over from website [#12071](https://github.com/vitessio/vitess/pull/12071) +#### Query Serving + * Add release notes summary for views [#12422](https://github.com/vitessio/vitess/pull/12422) #### VReplication * Mark VDiff V2 as GA in v16 [#12084](https://github.com/vitessio/vitess/pull/12084) ### Enhancement @@ -185,7 +210,8 @@ * Allow override of build git env in docker/base builds [#11968](https://github.com/vitessio/vitess/pull/11968) * Add vtorc port to vitess local docker run [#12001](https://github.com/vitessio/vitess/pull/12001) * Update the MySQL version used by our Docker images [#12054](https://github.com/vitessio/vitess/pull/12054) - * Fail CI when a PR is labeled with `NeedsWebsiteDocsUpdate` or `NeedsDescriptionUpdate` [#12062](https://github.com/vitessio/vitess/pull/12062) + * Fail CI when a PR is labeled with `NeedsWebsiteDocsUpdate` or `NeedsDescriptionUpdate` [#12062](https://github.com/vitessio/vitess/pull/12062) + * Revert default MySQL 80 version to `8.0.30` [#12252](https://github.com/vitessio/vitess/pull/12252) #### CLI * Add GenerateShardRanges to vtctldclient [#11492](https://github.com/vitessio/vitess/pull/11492) * Properly deprecate flags and fix default for `--cell` [#11501](https://github.com/vitessio/vitess/pull/11501) @@ -207,7 +233,8 @@ * Add default lower stack limit [#11569](https://github.com/vitessio/vitess/pull/11569) * Upgrade to `go1.19.4` [#11905](https://github.com/vitessio/vitess/pull/11905) * Add structure logging to Vitess [#11960](https://github.com/vitessio/vitess/pull/11960) - * Revert changes made in #11960 [#12219](https://github.com/vitessio/vitess/pull/12219) + * Revert changes made in #11960 [#12219](https://github.com/vitessio/vitess/pull/12219) + * Upgrade `release-16.0` to `go1.20.1` [#12398](https://github.com/vitessio/vitess/pull/12398) #### Governance * Add manan and florent to Docker files CODEOWNERS [#11981](https://github.com/vitessio/vitess/pull/11981) #### Query Serving @@ -263,7 +290,13 @@ * schemadiff: TableCharsetCollateStrategy hint [#12137](https://github.com/vitessio/vitess/pull/12137) * Support BETWEEN in the evalengine [#12150](https://github.com/vitessio/vitess/pull/12150) * Use schema for the information_schema views [#12171](https://github.com/vitessio/vitess/pull/12171) - * vtgateconn: add DeregisterDialer hook [#12213](https://github.com/vitessio/vitess/pull/12213) + * vtgateconn: add DeregisterDialer hook [#12213](https://github.com/vitessio/vitess/pull/12213) + * add database name to _vt.views table [#12368](https://github.com/vitessio/vitess/pull/12368) + * Schema RPC to fetch table/view definition [#12375](https://github.com/vitessio/vitess/pull/12375) + * Change `GetSchema` RPC to return `CreateView` instead of `SelectStmt` [#12421](https://github.com/vitessio/vitess/pull/12421) + * GetSchema rpc to streaming api [#12447](https://github.com/vitessio/vitess/pull/12447) +#### TabletManager + * SidecarDB Init: don't fail on schema init errors [#12328](https://github.com/vitessio/vitess/pull/12328) #### VReplication * VReplication Copy Phase: Parallelize Bulk Inserts [#10828](https://github.com/vitessio/vitess/pull/10828) * VSCopy: Resume the copy phase consistently from given GTID and lastpk [#11103](https://github.com/vitessio/vitess/pull/11103) @@ -273,7 +306,8 @@ * Allow users to control VReplication DDL handling [#11532](https://github.com/vitessio/vitess/pull/11532) * VReplication: Defer Secondary Index Creation [#11700](https://github.com/vitessio/vitess/pull/11700) * VSCopy: Send COPY_COMPLETED events when the copy operation is done [#11740](https://github.com/vitessio/vitess/pull/11740) - * Add `VStreamerCount` stat to `vttablet` [#11978](https://github.com/vitessio/vitess/pull/11978) + * Add `VStreamerCount` stat to `vttablet` [#11978](https://github.com/vitessio/vitess/pull/11978) + * VReplication: Use MariaDB Compat JSON Functions [#12420](https://github.com/vitessio/vitess/pull/12420) #### VTAdmin * [VTAdmin] `Validate`, `ValidateShard`, `ValidateVersionShard`, `GetFullStatus` [#11438](https://github.com/vitessio/vitess/pull/11438) * Full Status tab improvements for VTAdmin [#11470](https://github.com/vitessio/vitess/pull/11470) @@ -291,8 +325,9 @@ * Add support for views in vtgate [#11195](https://github.com/vitessio/vitess/pull/11195) * Add support for Views DDL [#11896](https://github.com/vitessio/vitess/pull/11896) * notify view change to vtgate [#12115](https://github.com/vitessio/vitess/pull/12115) - * Views Support: Updating Views in VSchema for query serving [#12124](https://github.com/vitessio/vitess/pull/12124) -#### Admin Web UI + * Views Support: Updating Views in VSchema for query serving [#12124](https://github.com/vitessio/vitess/pull/12124) + * Create Views allowed for same keyspace [#12409](https://github.com/vitessio/vitess/pull/12409) +#### web UI * [VTAdmin] Topology Browser [#11496](https://github.com/vitessio/vitess/pull/11496) ### Internal Cleanup #### Backup and Restore @@ -316,7 +351,11 @@ * Remove example script that caused some confusion [#11529](https://github.com/vitessio/vitess/pull/11529) * Remove unused ioutil2 code [#11661](https://github.com/vitessio/vitess/pull/11661) * Fix some linter errors [#11773](https://github.com/vitessio/vitess/pull/11773) - * Remove Deprecated flags, code and stats. [#12083](https://github.com/vitessio/vitess/pull/12083) + * Remove Deprecated flags, code and stats. [#12083](https://github.com/vitessio/vitess/pull/12083) + * Fix release notes on release-16.0 [#12276](https://github.com/vitessio/vitess/pull/12276) + * Fix summary and release notes [#12283](https://github.com/vitessio/vitess/pull/12283) + * Reorder summary in order of importance [#12433](https://github.com/vitessio/vitess/pull/12433) + * release notes: mark dynamic throttler configuration as experimental [#12475](https://github.com/vitessio/vitess/pull/12475) #### Governance * Correct minor inaccuracies in governing docs [#11933](https://github.com/vitessio/vitess/pull/11933) #### Online DDL @@ -343,7 +382,7 @@ * VReplication: Remove Deprecated V1 Client Commands [#11705](https://github.com/vitessio/vitess/pull/11705) #### VTAdmin * move react-scripts to dev dependencies [#11767](https://github.com/vitessio/vitess/pull/11767) -#### Admin Web UI +#### web UI * [vtctld2] Remove vtctld2 UI and vtctld server components that serve the app UI [#11851](https://github.com/vitessio/vitess/pull/11851) ### Performance #### Cluster management @@ -373,6 +412,10 @@ * Release notes for 15.0.1 [#11850](https://github.com/vitessio/vitess/pull/11850) * updating summary and release notes for v15.0.1 [#11852](https://github.com/vitessio/vitess/pull/11852) * [main] Update the release `15.0.2` summary doc (#11954) [#11956](https://github.com/vitessio/vitess/pull/11956) + * Release of v16.0.0-rc1 [#12235](https://github.com/vitessio/vitess/pull/12235) + * Back to dev mode after v16.0.0-rc1 [#12277](https://github.com/vitessio/vitess/pull/12277) + * [release-16.0] Fix release script for the version in the docker script (#12284) [#12289](https://github.com/vitessio/vitess/pull/12289) + * Code freeze of release-16.0 for v16.0.0 [#12469](https://github.com/vitessio/vitess/pull/12469) ### Testing #### Backup and Restore * go/vt/mysqlctl: add compression benchmarks [#11994](https://github.com/vitessio/vitess/pull/11994) @@ -385,9 +428,12 @@ * Remove the semgrep action [#12148](https://github.com/vitessio/vitess/pull/12148) * CI cleanup: remove onlineddl_declarative, onlineddl_singleton (cleanup part 2) [#12182](https://github.com/vitessio/vitess/pull/12182) * Online DDL CI: consolidated revertible and revert CI tests (part 1) [#12183](https://github.com/vitessio/vitess/pull/12183) - * Allow manually kicking off CodeQL [#12200](https://github.com/vitessio/vitess/pull/12200) + * Allow manually kicking off CodeQL [#12200](https://github.com/vitessio/vitess/pull/12200) + * Don't keep data in upgrade-downgrade tests [#12462](https://github.com/vitessio/vitess/pull/12462) #### General * endtoend: fix dbconfig initialization for endtoend tests [#11609](https://github.com/vitessio/vitess/pull/11609) +#### Online DDL + * Backport to v16: onlineddl_vrepl flakiness and subsequent fixes [#12426](https://github.com/vitessio/vitess/pull/12426) #### Query Serving * Add additional unit test with state changes swapped [#11192](https://github.com/vitessio/vitess/pull/11192) * Use JSON for plan tests [#11430](https://github.com/vitessio/vitess/pull/11430) diff --git a/doc/releasenotes/16_0_0_release_notes.md b/doc/releasenotes/16_0_0_release_notes.md index 36996f95f82..b860a907031 100644 --- a/doc/releasenotes/16_0_0_release_notes.md +++ b/doc/releasenotes/16_0_0_release_notes.md @@ -1,30 +1,33 @@ -# Release of Vitess v16.0.0-rc1 +# Release of Vitess v16.0.0 ## Summary ### Table of Contents - **[Major Changes](#major-changes)** - - **[VReplication](#vreplication)** - - [VStream Copy Resume](#vstream-copy-resume) - - [VDiff2 GA](#vdiff2-ga) - - **[Tablet throttler](#tablet-throttler)** - - **[Incremental backup and point in time recovery](#inc-backup)** - - **[Replication manager removal and VTOrc becomes mandatory](#repl-manager-removal)** - **[Breaking Changes](#breaking-changes)** - [VTGate Advertised MySQL Version](#advertised-mysql-version) - [Default MySQL version on Docker](#default-mysql-version) - - [Running Vitess on the Operator](#running-vitess-on-the-operator) - - [vtctld UI Removal](#vtcltd-ui-removal) - - [vtctld Flag Deprecation & Deletions](#vtctld-flag-deprecations) + - [⚠️ Upgrading to this release with vitess-operator](#upgrading-to-this-release-with-vitess-operator) + - [Flag Deletions and Deprecations](#flag-deletions-and-deprecations) + - [VTCtld](#vtctld-flag-deletions-deprecations) + - [MySQLCtl](#mysqlctl-flag-deletions-deprecations) + - [VTTablet](#vttablet-flag-deletions-deprecations) + - [VTBackup](#vtbackup-flag-deletions-deprecations) + - [VTOrc](#vtorc-flag-deletions-deprecations) + - [`lock-timeout` and `remote_operation_timeout` Changes](#lock-timeout-introduction) - [Orchestrator Integration Deletion](#orc-integration-removal) - - [mysqlctl Flags](#mysqlctl-flags) + - [vtctld UI Removal](#vtcltd-ui-removal) - [Query Serving Errors](#qs-errors) - [Logstats Table and Keyspace removed](#logstats-table-keyspace) - [Removed Stats](#removed-stats) - [Deprecated Stats](#deprecated-stats) - - [Removed flag](#removed-flag) - - [`lock-timeout` and `remote_operation_timeout` Changes](#lock-timeout-introduction) - [Normalized labels in the Prometheus Exporter](#normalized-lables) + - **[Replication manager removal and VTOrc becomes mandatory](#repl-manager-removal)** + - **[VReplication](#vreplication)** + - [VStream Copy Resume](#vstream-copy-resume) + - [VDiff2 GA](#vdiff2-ga) + - **[Tablet throttler](#tablet-throttler)** + - **[Incremental backup and point in time recovery](#inc-backup)** - **[New command line flags and behavior](#new-flag)** - [VTGate: Support query timeout --query-timeout](#vtgate-query-timeout) - [VTTablet: VReplication parallel insert workers --vreplication-parallel-insert-workers](#vrepl-parallel-workers) @@ -41,8 +44,6 @@ - [Transaction Isolation Level](#transaction-isolation-level) - [Transaction Access Mode](#transaction-access-mode) - [Support for views](#support-views) - - **[VTOrc](#vtorc)** - - [Flag Deprecations](#flag-deprecations) - **[VTTestServer](#vttestserver)** - [Performance Improvement](#perf-improvement) - **[Minor Changes](#minor-changes)** @@ -52,54 +53,21 @@ ## Major Changes -### VReplication - -#### VStream Copy Resume - -In [PR #11103](https://github.com/vitessio/vitess/pull/11103) we introduced the ability to resume a `VTGate` [`VStream` copy operation](https://vitess.io/docs/design-docs/vreplication/vstream/vscopy/). This is useful when a [`VStream` copy operation](https://vitess.io/docs/design-docs/vreplication/vstream/vscopy/) is interrupted due to e.g. a network failure or a server restart. The `VStream` copy operation can be resumed by specifying each table's last seen primary key value in the `VStream` request. Please see the [`VStream` docs](https://vitess.io/docs/16.0/reference/vreplication/vstream/) for more details. - -#### VDiff2 GA - -We are marking [VDiff v2](https://vitess.io/docs/16.0/reference/vreplication/vdiff2/) as Generally Available or production-ready in v16. We now recommend that you use v2 rather than v1 going forward. V1 will be deprecated and eventually removed in future releases. -If you wish to use v1 for any reason, you will now need to specify the `--v1` flag. - -### Tablet throttler - -The tablet throttler can now be configured dynamically. Configuration is now found in the topo service, and applies to all tablets in all shards and cells of a given keyspace. For backwards compatibility `v16` still supports `vttablet`-based command line flags for throttler ocnfiguration. - -It is possible to enable/disable, to change throttling threshold as well as the throttler query. - -See https://github.com/vitessio/vitess/pull/11604 - -### Incremental backup and point in time recovery - -In [PR #11097](https://github.com/vitessio/vitess/pull/11097) we introduced native incremental backup and point in time recovery: - -- It is possible to take an incremental backup, starting with last known (full or incremental) backup, and up to either a specified (GTID) position, or current ("auto") position. -- The backup is done by copying binary logs. The binary logs are rotated as needed. -- It is then possible to restore a backup up to a given point in time (GTID position). This involves finding a restore path consisting of a full backup and zero or more incremental backups, applied up to the given point in time. -- A server restored to a point in time remains in `DRAINED` tablet type, and does not join the replication stream (thus, "frozen" in time). -- It is possible to take incremental backups from different tablets. It is OK to have overlaps in incremental backup contents. The restore process chooses a valid path, and is valid as long as there are no gaps in the backed up binary log content. - -### Replication manager removal and VTOrc becomes mandatory -VTOrc is now a **required** component of Vitess starting from v16. If the users want VTOrc to manage replication, then they must run VTOrc. -Replication manager is removed from vttablets since the responsibility of fixing replication lies entirely with VTOrc now. -The flag `disable-replication-manager` is deprecated and will be removed in a later release. - ### Breaking Changes #### VTGate Advertised MySQL Version -VTGate now advertises MySQL version 8.0.30. This is a breaking change for clients that rely on the VTGate advertised MySQL version and still use MySQL 5.7. +Since [Pull Request #11989](https://github.com/vitessio/vitess/pull/11989), VTGate advertises MySQL version 8.0.30. This is a breaking change for clients that rely on the VTGate advertised MySQL version and still use MySQL 5.7. The users can set the `mysql_server_version` flag to advertise the correct version. #### Default MySQL version on Docker The default major MySQL version used by our `vitess/lite:latest` image is going from `5.7` to `8.0`. Additionally, the patch version of MySQL80 has been upgraded from `8.0.23` to `8.0.30`. +This change was brought by [Pull Request #12252](https://github.com/vitessio/vitess/pull/12252). -#### Running Vitess on the Operator +#### ⚠️Upgrading to this release with vitess-operator -If you are using the vitess-operator and want to remain on MySQL 5.7, we invite you to use the `vitess/lite:v16.0.0-mysql57` Docker Image. +If you are using the vitess-operator and want to remain on MySQL 5.7, **you are required** to use the `vitess/lite:v16.0.0-mysql57` Docker Image, otherwise the `vitess/lite:v16.0.0` image will be on MySQL 80. However, if you are running MySQL 8.0 on the vitess-operator, with for instance `vitess/lite:v15.0.2-mysql80`, considering that we are bumping the patch version of MySQL 80 from `8.0.23` to `8.0.30`, you will have to manually upgrade: @@ -110,12 +78,9 @@ However, if you are running MySQL 8.0 on the vitess-operator, with for instance 5. Remove `innodb_fast_shutdown=0` from your extra cnf in your YAML file. 6. Apply this file. -#### vtctld web UI Removal -In v13, the vtctld UI was deprecated. As of this release, the `web/vtctld2` directory is deleted and the UI will no longer be included in any Vitess images going forward. All build scripts and the Makefile have been updated to reflect this change. +#### Flag Deletions and Deprecations -However, the vtctld HTTP API will remain at `{$vtctld_web_port}/api`. - -#### vtctld Flag Deprecation & Deletions +##### VTCtld With the removal of the vtctld UI, the following vtctld flags have been deprecated: - `--vtctld_show_topology_crud`: This was a flag that controlled the display of CRUD topology actions in the vtctld UI. The UI is removed, so this flag is no longer necessary. @@ -128,19 +93,51 @@ The following deprecated flags have also been removed: - `--workflow_manager_use_election` - `--workflow_manager_disable` +##### MySQLCtld + +The [`mysqlctl` command-line client](https://vitess.io/docs/16.0/reference/programs/mysqlctl/) had some leftover (ignored) server flags after the [v15 pflag work](https://github.com/vitessio/enhancements/blob/main/veps/vep-4.md). Those unused flags have now been removed. If you are using any of the following flags with `mysqlctl` in your scripts or other tooling, they will need to be removed prior to upgrading to v16: +`--port --grpc_auth_static_client_creds --grpc_compression --grpc_initial_conn_window_size --grpc_initial_window_size --grpc_keepalive_time --grpc_keepalive_timeout` + +##### VTTablet + +The following flags were removed in v16: +- `--enable_semi_sync` +- `--backup_storage_hook`, use one of the builtin compression algorithms or `--external-compressor` and `--external-decompressor` instead. +- `--init_populate_metadata`, since we have deleted the `local_metadata` and `shard_metadata` sidecar database tables. + +The flag `--disable-replication-manager` is deprecated and will be removed in a future release. + +##### VTBackup + +The VTBackup flag `--backup_storage_hook` has been removed, use one of the builtin compression algorithms or `--external-compressor` and `--external-decompressor` instead. + + +##### VTOrc + +The flag `--lock-shard-timeout` has been deprecated. Please use the newly introduced `--lock-timeout` flag instead. More detail [here](#lock-timeout-introduction). + +#### `lock-timeout` and `remote_operation_timeout` Changes + +Before the changes made in [Pull Request #11881](https://github.com/vitessio/vitess/pull/11881), the shard and keyspace locks used to be capped by the `remote_operation_timeout`. This is no longer the case and instead a new flag called `lock-timeout` is introduced. +For backward compatibility, if `lock-timeout` is unspecified and `remote_operation_timeout` flag is provided, then its value will also be used for `lock-timeout`. +The default value for `remote_operation_timeout` has also changed from 30 seconds to 15 seconds. The default for the new flag `lock-timeout` is 45 seconds. + +During upgrades, if the users want to preserve the same behaviour as previous releases, then they should provide the `remote_operation_timeout` flag explicitly before upgrading. +After the upgrade, they should then alter their configuration to also specify `lock-timeout` explicitly. + #### Orchestrator Integration Deletion Orchestrator integration in `vttablet` was deprecated in the previous release and is deleted in this release. -Consider using `VTOrc` instead of `Orchestrator`. +`VTOrc` should be deployed instead. You can read more on [how VTOrc is designed](https://vitess.io/docs/16.0/reference/vtorc/) and on [how to run VTOrc in production](https://vitess.io/docs/16.0/user-guides/configuration-basic/vtorc/). -#### mysqlctl Flags +#### vtctld web UI Removal +In v13, the vtctld UI was deprecated. As of this release, the `web/vtctld2` directory is deleted and the UI will no longer be included in any Vitess images going forward. All build scripts and the Makefile have been updated to reflect this change, which was done in [Pull Request #11851](https://github.com/vitessio/vitess/pull/11851) -The [`mysqlctl` command-line client](https://vitess.io/docs/16.0/reference/programs/mysqlctl/) had some leftover (ignored) server flags after the [v15 pflag work](https://github.com/vitessio/enhancements/blob/main/veps/vep-4.md). Those unused flags have now been removed. If you are using any of the following flags with `mysqlctl` in your scripts or other tooling, they will need to be removed prior to upgrading to v16: - `--port --grpc_auth_static_client_creds --grpc_compression --grpc_initial_conn_window_size --grpc_initial_window_size --grpc_keepalive_time --grpc_keepalive_timeout` +However, the vtctld HTTP API will remain at `{$vtctld_web_port}/api`. #### Query Serving Errors -In this release, we are introducing a new way to report errors from Vitess through the query interface. +In [Pull Request #10738](https://github.com/vitessio/vitess/pull/10738) we are introducing a new way to report errors from Vitess through the query interface. Errors will now have an error code for each error, which will make it easy to search for more information on the issue. For instance, the following error: @@ -163,33 +160,69 @@ If your application is searching for specific errors, you might need to update y #### Logstats Table and Keyspace removed Information about which tables are used is now reported by the field TablesUsed added in v15, that is a string array, listing all tables and which keyspace they are in. -The Table/Keyspace fields were deprecated in v15 and are now removed in the v16 release of Vitess. +The Table/Keyspace fields were deprecated in v15 and are now removed in the v16 release, more information can be found on [Pull Request #12083](https://github.com/vitessio/vitess/pull/12083). #### Removed Stats -The stat `QueryRowCounts` is removed in v16. `QueryRowsAffected` and `QueryRowsReturned` can be used instead to gather the same information. +The stat `QueryRowCounts` is removed in v16 as part of [Pull Request #12083](https://github.com/vitessio/vitess/pull/12083). `QueryRowsAffected` and `QueryRowsReturned` can be used instead to gather the same information. #### Deprecated Stats -The stats `QueriesProcessed` and `QueriesRouted` are deprecated in v16. The same information can be inferred from the stats `QueriesProcessedByTable` and `QueriesRoutedByTable` respectively. These stats will be removed in the next release. +The stats `QueriesProcessed` and `QueriesRouted` are deprecated in v16 as part of [Pull Request #12083](https://github.com/vitessio/vitess/pull/12083). The same information can be inferred from the stats `QueriesProcessedByTable` and `QueriesRoutedByTable` respectively. These stats will be removed in the next release. -#### Removed flag +#### Normalized labels in the Prometheus Exporter -The following flag is removed in v16: -- `enable_semi_sync` +The Prometheus metrics exporter now properly normalizes _all_ label names into their `snake_case` form, as it is idiomatic for Prometheus metrics. Previously, Vitess instances were emitting inconsistent labels for their metrics, with some of them being `CamelCase` and others being `snake_case`. +More information about this change can be found on [Pull Request #12057](https://github.com/vitessio/vitess/pull/12057). -#### `lock-timeout` and `remote_operation_timeout` Changes +For example, `vtgate_topology_watcher_errors{Operation="GetTablet"} 0` will become `vtgate_topology_watcher_errors{operation="GetTablet"} 0` -Earlier, the shard and keyspace locks used to be capped by the `remote_operation_timeout`. This is no longer the case and instead a new flag called `lock-timeout` is introduced. -For backward compatibility, if `lock-timeout` is unspecified and `remote_operation_timeout` flag is provided, then its value will also be used for `lock-timeout` as well. -The default value for `remote_operation_timeout` has also changed from 30 seconds to 15 seconds. The default for the new flag `lock-timeout` is 45 seconds. +Some more of these changes are listed here - -During upgrades, if the users want to preserve the same behaviour as previous releases, then they should provide the `remote_operation_timeout` flag explicitly before upgrading. -After the upgrade, they should then alter their configuration to also specify `lock-timeout` explicitly. +| Previous metric | New Metric | +|-------------------------------------------------------------|-------------------------------------------------------------| +| vtgate_topology_watcher_operations{Operation="AddTablet"} | vtgate_topology_watcher_operations{operation="AddTablet"} | +| vtgate_queries_processed{Plan="Reference"} | vtgate_queries_processed{plan="Reference"} | +| vtgate_queries_routed{Plan="Reference"} | vtgate_queries_routed{plan="Reference"} | +| vttablet_table_allocated_size{Table="corder"} | vttablet_table_allocated_size{table="corder"} | +| vttablet_table_file_size{Table="corder"} | vttablet_table_file_size{table="corder"} | +| vttablet_topology_watcher_errors{Operation="GetTablet"} | vttablet_topology_watcher_errors{operation="GetTablet"} | +| vttablet_topology_watcher_operations{Operation="AddTablet"} | vttablet_topology_watcher_operations{operation="AddTablet"} | -#### Normalized labels in the Prometheus Exporter +### Replication manager removal and VTOrc becomes mandatory +VTOrc is now a **required** component of Vitess starting from v16. If the users want Vitess to manage replication, then they must run VTOrc. +Replication manager is removed from vttablets since the responsibility of fixing replication lies entirely with VTOrc now. +The flag `disable-replication-manager` is deprecated and will be removed in a future release. -The Prometheus metrics exporter now properly normalizes _all_ label names into their `snake_case` form, as it is idiomatic for Prometheus metrics. Previously, Vitess instances were emitting inconsistent labels for their metrics, with some of them being `CamelCase` and others being `snake_case`. +### VReplication + +#### VStream Copy Resume + +In [Pull Request #11103](https://github.com/vitessio/vitess/pull/11103) we introduced the ability to resume a `VTGate` [`VStream` copy operation](https://vitess.io/docs/16.0/reference/vreplication/vstream/). This is useful when a [`VStream` copy operation](https://vitess.io/docs/16.0/reference/vreplication/vstream/) is interrupted due to e.g. a network failure or a server restart. The `VStream` copy operation can be resumed by specifying each table's last seen primary key value in the `VStream` request. Please see the [`VStream` docs](https://vitess.io/docs/16.0/reference/vreplication/vstream/) for more details. + +#### VDiff2 GA + +We are marking [VDiff v2](https://vitess.io/docs/16.0/reference/vreplication/vdiff2/) as production-ready in v16. We now recommend that you use v2 rather than v1 going forward. V1 will be deprecated and eventually removed in future releases. +If you wish to use v1 for any reason, you will now need to specify the `--v1` flag. + +### Tablet throttler + +The tablet throttler can now be configured dynamically. Configuration is now found in the topo service, and applies to all tablets in all shards and cells of a given keyspace. +It is possible to enable or disable throttling, and to change the throttling threshold as well as the throttler query. + +Please note that this feature is considered experimental in this release. For backwards compatibility `v16` still supports `vttablet`-based command line flags for throttler configuration. + +More information can be found on [Pull Request #11604](https://github.com/vitessio/vitess/pull/11604). + +### Incremental backup and point in time recovery + +In [Pull Request #11097](https://github.com/vitessio/vitess/pull/11097) we introduced native incremental backup and point in time recovery: + +- It is possible to take an incremental backup, starting with last known (full or incremental) backup, and up to either a specified (GTID) position, or current ("auto") position. +- The backup is done by copying binary logs. The binary logs are rotated as needed. +- It is then possible to restore a backup up to a given point in time (GTID position). This involves finding a restore path consisting of a full backup and zero or more incremental backups, applied up to the given point in time. +- A server restored to a point in time remains in `DRAINED` tablet type, and does not join the replication stream (thus, "frozen" in time). +- It is possible to take incremental backups from different tablets. It is OK to have overlaps in incremental backup contents. The restore process chooses a valid path, and is valid as long as there are no gaps in the backed up binary log content. ### New command line flags and behavior @@ -197,7 +230,7 @@ The Prometheus metrics exporter now properly normalizes _all_ label names into t `--query-timeout` allows you to specify a timeout for queries. This timeout is applied to all queries. It can be overridden by setting the `query_timeout` session variable. -Setting it as command line directive with `QUERY_TIMEOUT_MS` will override other values. +Setting it as query comment directive with `QUERY_TIMEOUT_MS` will override other values. #### VTTablet: VReplication parallel insert workers --vreplication-parallel-insert-workers @@ -262,9 +295,9 @@ $ vtctlclient -- Backup --incremental_from_pos "auto" zone1-0000000102 When the value is `auto`, the position is evaluated as the last successful backup's `Position`. The idea with incremental backups is to create a contiguous (overlaps allowed) sequence of backups that store all changes from last full backup. -The incremental backup copies binary log files. It does not take MySQL down nor places any locks. It does not interrupt traffic on the MySQL server. The incremental backup copies comlete binlog files. It initially rotates binary logs, then copies anything from the requested position and up to the last completed binary log. +The incremental backup copies binary log files. It does not take MySQL down nor places any locks. It does not interrupt traffic on the MySQL server. The incremental backup copies complete binlog files. It initially rotates binary logs, then copies anything from the requested position and up to the last completed binary log. -The backup thus does not necessarily start _exactly_ at the requested position. It starts with the first binary log that has newer entries than requested position. It is OK if the binary logs include transactions prior to the equested position. The restore process will discard any duplicates. +The backup thus does not necessarily start _exactly_ at the requested position. It starts with the first binary log that has newer entries than requested position. It is OK if the binary logs include transactions prior to the requested position. The restore process will discard any duplicates. Normally, you can expect the backups to be precisely contiguous. Consider an `auto` value: due to the nature of log rotation and the fact we copy complete binlog files, the next incremental backup will start with the first binay log not covered by the previous backup, which in itself copied the one previous binlog file in full. Again, it is completely valid to enter any good position. @@ -301,13 +334,12 @@ The `RestoreFromBackup --restore_to_pos` ends with: - tablet type is `DRAINED` #### New `vexplain` command -A new `vexplain` command has been introduced with the following syntax - +A new `vexplain` command has been introduced with the following syntax: ``` VEXPLAIN [ALL|QUERIES|PLAN] explainable_stmt ``` -This command will help the users look at the plan that vtgate comes up with for the given query (`PLAN` type), see all the queries that are executed on all the MySQL instances (`QUERIES` type), -and see the vtgate plan along with the MySQL explain output for the executed queries (`ALL` type). +This command will help users look at the plan that vtgate comes up with for the given query (`PLAN` type), see all the queries that are executed on all the MySQL instances (`QUERIES` type), and see the vtgate plan along with the MySQL explain output for the executed queries (`ALL` type). The formats `VTEXPLAIN` and `VITESS` for `EXPLAIN` queries are deprecated, and these newly introduced commands should be used instead. @@ -330,22 +362,15 @@ is now fixed. The full issue can be found [here](https://github.com/vitessio/vit - `vtctlclient OnlineDDL ... [complete|retry|cancel|cancel-all]` returns empty result on success instead of number of shard affected. -- VTTablet flag `--backup_storage_hook` has been removed, use one of the builtin compression algorithms or `--external-compressor` and `--external-decompressor` instead. - -- vtbackup flag `--backup_storage_hook` has been removed, use one of the builtin compression algorithms or `--external-compressor` and `--external-decompressor` instead. - -- The VTTablet flag `--init_populate_metadata` has been deprecated, since we have deleted the `local_metadata` and `shard_metadata` sidecar database tables. - - The dead legacy Workflow Manager related code was removed in [#12085](https://github.com/vitessio/vitess/pull/12085). This included the following `vtctl` client commands: `WorkflowAction`, `WorkflowCreate`, `WorkflowWait`, `WorkflowStart`, `WorkflowStop`, `WorkflowTree`, `WorkflowDelete`. - VTAdmin's `VTExplain` endpoint has been deprecated. Users can use the new `vexplain` query format instead. The endpoint will be deleted in a future release. - ### MySQL Compatibility #### Transaction Isolation Level -Support added for `set [session] transaction isolation level ` +In [Pull Request #11704](https://github.com/vitessio/vitess/pull/11704) we are adding support for `set [session] transaction isolation level ` ```sql transaction_characteristic: { @@ -361,12 +386,12 @@ level: { } ``` -This will set the transaction isolation level for the current session. +This will set the transaction isolation level for the current session. This will be applied to any shard where the session will open a transaction. #### Transaction Access Mode -Support added for `start transaction` with transaction characteristic. +In [Pull Request #11704](https://github.com/vitessio/vitess/pull/11704) we are adding support for `start transaction` with transaction characteristic. ```sql START TRANSACTION @@ -383,36 +408,19 @@ This will allow users to start a transaction with these characteristics. #### Support For Views -Vitess now supports views in sharded keyspace. Views are not created on the underlying database but are logically stored -in vschema. -Any query using a view will get re-written as a derived table during query planning. -VSchema Example - -```json -{ - "sharded": true, - "vindexes": {}, - "tables": {}, - "views": { - "view1": "select * from t1", - "view2": "select * from t2", - } -} -``` - -### VTOrc - -#### Flag Deprecations +Views sharded support is released as an experimental feature in `v16.0.0`. +Views are not enabled by default in your Vitess cluster, but they can be turned on using the `--enable-views` flag on VTGate, and `--queryserver-enable-views` flag on VTTablet. -The flag `lock-shard-timeout` has been deprecated. Please use the newly introduced `lock-timeout` instead. More detail [here](#lock-timeout-introduction). +To read more on how views are implemented you can read the [Views Support RFC](https://github.com/vitessio/vitess/issues/11559). +And if you want to learn more on how to use views and its current limitations, you can read the [Views Documentation](https://vitess.io/docs/16.0/reference/compatibility/mysql-compatibility/#views). ### VTTestServer #### Performance Improvement Creating a database with vttestserver was taking ~45 seconds. This can be problematic in test environments where testcases do a lot of `create` and `drop` database. -In an effort to minimize the database creation time, we have changed the value of `tablet_refresh_interval` to 10s while instantiating vtcombo during vttestserver initialization. We have also made this configurable so that it can be reduced further if desired. -For any production cluster the default value of this flag is still [1 minute](https://vitess.io/docs/15.0/reference/programs/vtgate/). Reducing this value might put more stress on Topo Server (since we now read from Topo server more often) but for testing purposes +In an effort to minimize the database creation time, in [Pull Request #11918](https://github.com/vitessio/vitess/pull/11918) we have changed the value of `tablet_refresh_interval` to 10s while instantiating vtcombo during vttestserver initialization. We have also made this configurable so that it can be reduced further if desired. +For any production cluster the default value of this flag is still [1 minute](https://vitess.io/docs/16.0/reference/programs/vtgate/). Reducing this value might put more stress on Topo Server (since we now read from Topo server more often) but for testing purposes this shouldn't be a concern. ## Minor changes @@ -458,19 +466,19 @@ BenchmarkCompressLz4Builtin This is an internal refactor and should not change the behavior of Vitess as seen by users. Developers will see a difference though: v16 changes the way we maintain vttablet's sidecar database schema (also referred to as the `_vt` -database). Instead of using the `WithDDL` package, introduced in #6348, we use a declarative approach. Users will now have to update -the desired schema in the `go/vt/sidecardb/schema` directory. +database). Instead of using the `WithDDL` package, introduced in [PR #6348](https://github.com/vitessio/vitess/pull/6348), we use a +declarative approach. Developers will now have to update the desired schema in the `go/vt/sidecardb/schema` directory. The desired schema is specified, one per table. A new module `sidecardb`, compares this to the existing schema and performs the required `create` or `alter` to reach it. This is done whenever a primary vttablet starts up. The sidecar tables `local_metadata` and `shard_metadata` are no longer in use and all references to them are removed as -part of this refactor. There were used previously for Orchestrator support, which has been superseded by `vtorc`. +part of this refactor. They were used previously for Orchestrator support, which has been superseded by `vtorc`. ------------ The entire changelog for this release can be found [here](https://github.com/vitessio/vitess/blob/main/doc/releasenotes/16_0_0_changelog.md). -The release includes 331 commits (excluding merges) +The release includes 378 commits (excluding merges) Thanks to all our contributors: @EmadMokhtar, @GuptaManan100, @Weijun-H, @WilliamLu99, @ajm188, @arthurschreiber, @arvind-murty, @brendar, @brirams, @dbussink, @deepthi, @dependabot[bot], @draftcode, @ejortegau, @frouioui, @harshit-gangal, @jjh-kim, @johanoskarsson, @kbslvsk, @mattlord, @maxenglander, @mdlayher, @notfelineit, @pbibra, @pudiva, @rohit-nayak-ps, @rsajwani, @shlomi-noach, @systay, @timvaillancourt, @vitess-bot[bot], @vmg, @yoheimuta diff --git a/doc/releasenotes/16_0_0_summary.md b/doc/releasenotes/16_0_0_summary.md index 7b692b2f128..59fac06aaa5 100644 --- a/doc/releasenotes/16_0_0_summary.md +++ b/doc/releasenotes/16_0_0_summary.md @@ -174,6 +174,20 @@ The stats `QueriesProcessed` and `QueriesRouted` are deprecated in v16 as part o The Prometheus metrics exporter now properly normalizes _all_ label names into their `snake_case` form, as it is idiomatic for Prometheus metrics. Previously, Vitess instances were emitting inconsistent labels for their metrics, with some of them being `CamelCase` and others being `snake_case`. More information about this change can be found on [Pull Request #12057](https://github.com/vitessio/vitess/pull/12057). +For example, `vtgate_topology_watcher_errors{Operation="GetTablet"} 0` will become `vtgate_topology_watcher_errors{operation="GetTablet"} 0` + +Some more of these changes are listed here - + +| Previous metric | New Metric | +|-------------------------------------------------------------|-------------------------------------------------------------| +| vtgate_topology_watcher_operations{Operation="AddTablet"} | vtgate_topology_watcher_operations{operation="AddTablet"} | +| vtgate_queries_processed{Plan="Reference"} | vtgate_queries_processed{plan="Reference"} | +| vtgate_queries_routed{Plan="Reference"} | vtgate_queries_routed{plan="Reference"} | +| vttablet_table_allocated_size{Table="corder"} | vttablet_table_allocated_size{table="corder"} | +| vttablet_table_file_size{Table="corder"} | vttablet_table_file_size{table="corder"} | +| vttablet_topology_watcher_errors{Operation="GetTablet"} | vttablet_topology_watcher_errors{operation="GetTablet"} | +| vttablet_topology_watcher_operations{Operation="AddTablet"} | vttablet_topology_watcher_operations{operation="AddTablet"} | + ### Replication manager removal and VTOrc becomes mandatory VTOrc is now a **required** component of Vitess starting from v16. If the users want Vitess to manage replication, then they must run VTOrc. Replication manager is removed from vttablets since the responsibility of fixing replication lies entirely with VTOrc now. @@ -215,7 +229,7 @@ In [Pull Request #11097](https://github.com/vitessio/vitess/pull/11097) we intro `--query-timeout` allows you to specify a timeout for queries. This timeout is applied to all queries. It can be overridden by setting the `query_timeout` session variable. -Setting it as command line directive with `QUERY_TIMEOUT_MS` will override other values. +Setting it as query comment directive with `QUERY_TIMEOUT_MS` will override other values. #### VTTablet: VReplication parallel insert workers --vreplication-parallel-insert-workers @@ -280,9 +294,9 @@ $ vtctlclient -- Backup --incremental_from_pos "auto" zone1-0000000102 When the value is `auto`, the position is evaluated as the last successful backup's `Position`. The idea with incremental backups is to create a contiguous (overlaps allowed) sequence of backups that store all changes from last full backup. -The incremental backup copies binary log files. It does not take MySQL down nor places any locks. It does not interrupt traffic on the MySQL server. The incremental backup copies comlete binlog files. It initially rotates binary logs, then copies anything from the requested position and up to the last completed binary log. +The incremental backup copies binary log files. It does not take MySQL down nor places any locks. It does not interrupt traffic on the MySQL server. The incremental backup copies complete binlog files. It initially rotates binary logs, then copies anything from the requested position and up to the last completed binary log. -The backup thus does not necessarily start _exactly_ at the requested position. It starts with the first binary log that has newer entries than requested position. It is OK if the binary logs include transactions prior to the equested position. The restore process will discard any duplicates. +The backup thus does not necessarily start _exactly_ at the requested position. It starts with the first binary log that has newer entries than requested position. It is OK if the binary logs include transactions prior to the requested position. The restore process will discard any duplicates. Normally, you can expect the backups to be precisely contiguous. Consider an `auto` value: due to the nature of log rotation and the fact we copy complete binlog files, the next incremental backup will start with the first binay log not covered by the previous backup, which in itself copied the one previous binlog file in full. Again, it is completely valid to enter any good position. @@ -319,13 +333,12 @@ The `RestoreFromBackup --restore_to_pos` ends with: - tablet type is `DRAINED` #### New `vexplain` command -A new `vexplain` command has been introduced with the following syntax - +A new `vexplain` command has been introduced with the following syntax: ``` VEXPLAIN [ALL|QUERIES|PLAN] explainable_stmt ``` -This command will help the users look at the plan that vtgate comes up with for the given query (`PLAN` type), see all the queries that are executed on all the MySQL instances (`QUERIES` type), -and see the vtgate plan along with the MySQL explain output for the executed queries (`ALL` type). +This command will help users look at the plan that vtgate comes up with for the given query (`PLAN` type), see all the queries that are executed on all the MySQL instances (`QUERIES` type), and see the vtgate plan along with the MySQL explain output for the executed queries (`ALL` type). The formats `VTEXPLAIN` and `VITESS` for `EXPLAIN` queries are deprecated, and these newly introduced commands should be used instead. @@ -452,11 +465,11 @@ BenchmarkCompressLz4Builtin This is an internal refactor and should not change the behavior of Vitess as seen by users. Developers will see a difference though: v16 changes the way we maintain vttablet's sidecar database schema (also referred to as the `_vt` -database). Instead of using the `WithDDL` package, introduced in #6348, we use a declarative approach. Users will now have to update -the desired schema in the `go/vt/sidecardb/schema` directory. +database). Instead of using the `WithDDL` package, introduced in [PR #6348](https://github.com/vitessio/vitess/pull/6348), we use a +declarative approach. Developers will now have to update the desired schema in the `go/vt/sidecardb/schema` directory. The desired schema is specified, one per table. A new module `sidecardb`, compares this to the existing schema and performs the required `create` or `alter` to reach it. This is done whenever a primary vttablet starts up. The sidecar tables `local_metadata` and `shard_metadata` are no longer in use and all references to them are removed as -part of this refactor. There were used previously for Orchestrator support, which has been superseded by `vtorc`. +part of this refactor. They were used previously for Orchestrator support, which has been superseded by `vtorc`. diff --git a/docker/release.sh b/docker/release.sh index 9d97a58b7b7..d73b4ec2054 100755 --- a/docker/release.sh +++ b/docker/release.sh @@ -1,7 +1,7 @@ #!/bin/bash set -ex -vt_base_version='16.0.0-SNAPSHOT' +vt_base_version='v16.0.0' debian_versions='buster bullseye' default_debian_version='bullseye' diff --git a/examples/compose/docker-compose.beginners.yml b/examples/compose/docker-compose.beginners.yml index 00b0ca0ef5e..caed2f8b2aa 100644 --- a/examples/compose/docker-compose.beginners.yml +++ b/examples/compose/docker-compose.beginners.yml @@ -58,7 +58,7 @@ services: - "3306" vtctld: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15000:$WEB_PORT" - "$GRPC_PORT" @@ -81,7 +81,7 @@ services: condition: service_healthy vtgate: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15099:$WEB_PORT" - "$GRPC_PORT" @@ -111,7 +111,7 @@ services: condition: service_healthy schemaload: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 command: - sh - -c @@ -144,12 +144,12 @@ services: environment: - KEYSPACES=$KEYSPACE - GRPC_PORT=15999 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script vttablet100: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15100:$WEB_PORT" - "$GRPC_PORT" @@ -181,7 +181,7 @@ services: retries: 15 vttablet101: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15101:$WEB_PORT" - "$GRPC_PORT" @@ -213,7 +213,7 @@ services: retries: 15 vttablet102: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15102:$WEB_PORT" - "$GRPC_PORT" @@ -245,7 +245,7 @@ services: retries: 15 vttablet103: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15103:$WEB_PORT" - "$GRPC_PORT" @@ -277,7 +277,7 @@ services: retries: 15 vtorc: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 command: ["sh", "-c", "/script/vtorc-up.sh"] depends_on: - vtctld @@ -307,7 +307,7 @@ services: retries: 15 vreplication: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - ".:/script" environment: diff --git a/examples/compose/docker-compose.yml b/examples/compose/docker-compose.yml index 5d5750caf42..9dc6af7132f 100644 --- a/examples/compose/docker-compose.yml +++ b/examples/compose/docker-compose.yml @@ -75,7 +75,7 @@ services: - SCHEMA_FILES=lookup_keyspace_schema_file.sql - POST_LOAD_FILE= - EXTERNAL_DB=0 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script schemaload_test_keyspace: @@ -101,7 +101,7 @@ services: - SCHEMA_FILES=test_keyspace_schema_file.sql - POST_LOAD_FILE= - EXTERNAL_DB=0 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script set_keyspace_durability_policy: @@ -115,7 +115,7 @@ services: environment: - KEYSPACES=test_keyspace lookup_keyspace - GRPC_PORT=15999 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script vreplication: @@ -129,7 +129,7 @@ services: - TOPOLOGY_FLAGS=--topo_implementation consul --topo_global_server_address consul1:8500 --topo_global_root vitess/global - EXTERNAL_DB=0 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script vtctld: @@ -143,7 +143,7 @@ services: depends_on: external_db_host: condition: service_healthy - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15000:8080 - "15999" @@ -160,7 +160,7 @@ services: --normalize_queries=true ' depends_on: - vtctld - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15099:8080 - "15999" @@ -182,7 +182,7 @@ services: - EXTERNAL_DB=0 - DB_USER= - DB_PASS= - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 13000:8080 volumes: @@ -217,7 +217,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15101:8080 - "15999" @@ -254,7 +254,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15102:8080 - "15999" @@ -291,7 +291,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15201:8080 - "15999" @@ -328,7 +328,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15202:8080 - "15999" @@ -365,7 +365,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15301:8080 - "15999" @@ -402,7 +402,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15302:8080 - "15999" diff --git a/examples/compose/vtcompose/docker-compose.test.yml b/examples/compose/vtcompose/docker-compose.test.yml index 1f53ce21c37..f31665ade16 100644 --- a/examples/compose/vtcompose/docker-compose.test.yml +++ b/examples/compose/vtcompose/docker-compose.test.yml @@ -79,7 +79,7 @@ services: - SCHEMA_FILES=test_keyspace_schema_file.sql - POST_LOAD_FILE= - EXTERNAL_DB=0 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script schemaload_unsharded_keyspace: @@ -103,7 +103,7 @@ services: - SCHEMA_FILES=unsharded_keyspace_schema_file.sql - POST_LOAD_FILE= - EXTERNAL_DB=0 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script set_keyspace_durability_policy_test_keyspace: @@ -117,7 +117,7 @@ services: environment: - GRPC_PORT=15999 - KEYSPACES=test_keyspace - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script set_keyspace_durability_policy_unsharded_keyspace: @@ -130,7 +130,7 @@ services: environment: - GRPC_PORT=15999 - KEYSPACES=unsharded_keyspace - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script vreplication: @@ -144,7 +144,7 @@ services: - TOPOLOGY_FLAGS=--topo_implementation consul --topo_global_server_address consul1:8500 --topo_global_root vitess/global - EXTERNAL_DB=0 - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - .:/script vtctld: @@ -159,7 +159,7 @@ services: depends_on: external_db_host: condition: service_healthy - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15000:8080 - "15999" @@ -176,7 +176,7 @@ services: ''grpc-vtgateservice'' --normalize_queries=true ' depends_on: - vtctld - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15099:8080 - "15999" @@ -199,7 +199,7 @@ services: - EXTERNAL_DB=0 - DB_USER= - DB_PASS= - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 13000:8080 volumes: @@ -234,7 +234,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15101:8080 - "15999" @@ -271,7 +271,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15102:8080 - "15999" @@ -308,7 +308,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15201:8080 - "15999" @@ -345,7 +345,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15202:8080 - "15999" @@ -382,7 +382,7 @@ services: - CMD-SHELL - curl -s --fail --show-error localhost:8080/debug/health timeout: 10s - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - 15301:8080 - "15999" diff --git a/examples/compose/vtcompose/vtcompose.go b/examples/compose/vtcompose/vtcompose.go index 2543d7ac7e0..b0c182b8765 100644 --- a/examples/compose/vtcompose/vtcompose.go +++ b/examples/compose/vtcompose/vtcompose.go @@ -533,7 +533,7 @@ func generateDefaultShard(tabAlias int, shard string, keyspaceData keyspaceInfo, - op: add path: /services/init_shard_primary%[2]d value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 command: ["sh", "-c", "/vt/bin/vtctlclient %[5]s InitShardPrimary -force %[4]s/%[3]s %[6]s-%[2]d "] %[1]s `, dependsOn, aliases[0], shard, keyspaceData.keyspace, opts.topologyFlags, opts.cell) @@ -565,7 +565,7 @@ func generateExternalPrimary( - op: add path: /services/vttablet%[1]d value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15%[1]d:%[3]d" - "%[4]d" @@ -627,7 +627,7 @@ func generateDefaultTablet(tabAlias int, shard, role, keyspace string, dbInfo ex - op: add path: /services/vttablet%[1]d value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15%[1]d:%[4]d" - "%[5]d" @@ -665,7 +665,7 @@ func generateVtctld(opts vtOptions) string { - op: add path: /services/vtctld value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15000:%[1]d" - "%[2]d" @@ -696,7 +696,7 @@ func generateVtgate(opts vtOptions) string { - op: add path: /services/vtgate value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 ports: - "15099:%[1]d" - "%[2]d" @@ -738,7 +738,7 @@ func generateVTOrc(dbInfo externalDbInfo, keyspaceInfoMap map[string]keyspaceInf - op: add path: /services/vtorc value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - ".:/script" environment: @@ -763,7 +763,7 @@ func generateVreplication(dbInfo externalDbInfo, opts vtOptions) string { - op: add path: /services/vreplication value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - ".:/script" environment: @@ -791,7 +791,7 @@ func generateSetKeyspaceDurabilityPolicy( - op: add path: /services/set_keyspace_durability_policy_%[3]s value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - ".:/script" environment: @@ -828,7 +828,7 @@ func generateSchemaload( - op: add path: /services/schemaload_%[7]s value: - image: vitess/lite:v16.0.0-rc1 + image: vitess/lite:v16.0.0 volumes: - ".:/script" environment: diff --git a/examples/operator/101_initial_cluster.yaml b/examples/operator/101_initial_cluster.yaml index 5f15ae0087a..325e1291515 100644 --- a/examples/operator/101_initial_cluster.yaml +++ b/examples/operator/101_initial_cluster.yaml @@ -8,14 +8,14 @@ metadata: name: example spec: images: - vtctld: vitess/lite:v16.0.0-rc1 - vtadmin: vitess/vtadmin:v16.0.0-rc1 - vtgate: vitess/lite:v16.0.0-rc1 - vttablet: vitess/lite:v16.0.0-rc1 - vtbackup: vitess/lite:v16.0.0-rc1 - vtorc: vitess/lite:v16.0.0-rc1 + vtctld: vitess/lite:v16.0.0 + vtadmin: vitess/vtadmin:v16.0.0 + vtgate: vitess/lite:v16.0.0 + vttablet: vitess/lite:v16.0.0 + vtbackup: vitess/lite:v16.0.0 + vtorc: vitess/lite:v16.0.0 mysqld: - mysql80Compatible: vitess/lite:v16.0.0-rc1 + mysql80Compatible: vitess/lite:v16.0.0 mysqldExporter: prom/mysqld-exporter:v0.11.0 cells: - name: zone1 diff --git a/examples/operator/201_customer_tablets.yaml b/examples/operator/201_customer_tablets.yaml index e2b1ce56f5a..448c8a7190d 100644 --- a/examples/operator/201_customer_tablets.yaml +++ b/examples/operator/201_customer_tablets.yaml @@ -4,14 +4,14 @@ metadata: name: example spec: images: - vtctld: vitess/lite:v16.0.0-rc1 - vtadmin: vitess/vtadmin:v16.0.0-rc1 - vtgate: vitess/lite:v16.0.0-rc1 - vttablet: vitess/lite:v16.0.0-rc1 - vtbackup: vitess/lite:v16.0.0-rc1 - vtorc: vitess/lite:v16.0.0-rc1 + vtctld: vitess/lite:v16.0.0 + vtadmin: vitess/vtadmin:v16.0.0 + vtgate: vitess/lite:v16.0.0 + vttablet: vitess/lite:v16.0.0 + vtbackup: vitess/lite:v16.0.0 + vtorc: vitess/lite:v16.0.0 mysqld: - mysql80Compatible: vitess/lite:v16.0.0-rc1 + mysql80Compatible: vitess/lite:v16.0.0 mysqldExporter: prom/mysqld-exporter:v0.11.0 cells: - name: zone1 diff --git a/examples/operator/302_new_shards.yaml b/examples/operator/302_new_shards.yaml index 6491c242f32..d05d790565a 100644 --- a/examples/operator/302_new_shards.yaml +++ b/examples/operator/302_new_shards.yaml @@ -4,14 +4,14 @@ metadata: name: example spec: images: - vtctld: vitess/lite:v16.0.0-rc1 - vtadmin: vitess/vtadmin:v16.0.0-rc1 - vtgate: vitess/lite:v16.0.0-rc1 - vttablet: vitess/lite:v16.0.0-rc1 - vtbackup: vitess/lite:v16.0.0-rc1 - vtorc: vitess/lite:v16.0.0-rc1 + vtctld: vitess/lite:v16.0.0 + vtadmin: vitess/vtadmin:v16.0.0 + vtgate: vitess/lite:v16.0.0 + vttablet: vitess/lite:v16.0.0 + vtbackup: vitess/lite:v16.0.0 + vtorc: vitess/lite:v16.0.0 mysqld: - mysql80Compatible: vitess/lite:v16.0.0-rc1 + mysql80Compatible: vitess/lite:v16.0.0 mysqldExporter: prom/mysqld-exporter:v0.11.0 cells: - name: zone1 diff --git a/examples/operator/306_down_shard_0.yaml b/examples/operator/306_down_shard_0.yaml index 921c2ec269e..9d31fa0d730 100644 --- a/examples/operator/306_down_shard_0.yaml +++ b/examples/operator/306_down_shard_0.yaml @@ -4,14 +4,14 @@ metadata: name: example spec: images: - vtctld: vitess/lite:v16.0.0-rc1 - vtadmin: vitess/vtadmin:v16.0.0-rc1 - vtgate: vitess/lite:v16.0.0-rc1 - vttablet: vitess/lite:v16.0.0-rc1 - vtbackup: vitess/lite:v16.0.0-rc1 - vtorc: vitess/lite:v16.0.0-rc1 + vtctld: vitess/lite:v16.0.0 + vtadmin: vitess/vtadmin:v16.0.0 + vtgate: vitess/lite:v16.0.0 + vttablet: vitess/lite:v16.0.0 + vtbackup: vitess/lite:v16.0.0 + vtorc: vitess/lite:v16.0.0 mysqld: - mysql80Compatible: vitess/lite:v16.0.0-rc1 + mysql80Compatible: vitess/lite:v16.0.0 mysqldExporter: prom/mysqld-exporter:v0.11.0 cells: - name: zone1 diff --git a/examples/operator/operator.yaml b/examples/operator/operator.yaml index b840fa9fc19..9429889a686 100644 --- a/examples/operator/operator.yaml +++ b/examples/operator/operator.yaml @@ -6145,7 +6145,7 @@ spec: fieldPath: metadata.name - name: OPERATOR_NAME value: vitess-operator - image: planetscale/vitess-operator:v2.9.0-rc1 + image: planetscale/vitess-operator:v2.9.0 name: vitess-operator resources: limits: diff --git a/go/vt/servenv/version.go b/go/vt/servenv/version.go index 9018096ab32..16db2e1373c 100644 --- a/go/vt/servenv/version.go +++ b/go/vt/servenv/version.go @@ -19,4 +19,4 @@ package servenv // THIS FILE IS AUTO-GENERATED DURING NEW RELEASES BY ./tools/do_releases.sh // DO NOT EDIT -const versionName = "16.0.0-SNAPSHOT" +const versionName = "16.0.0" diff --git a/java/client/pom.xml b/java/client/pom.xml index efa409db760..a286b6b6cdf 100644 --- a/java/client/pom.xml +++ b/java/client/pom.xml @@ -5,7 +5,7 @@ io.vitess vitess-parent - 16.0.0-SNAPSHOT + 16.0.0 vitess-client diff --git a/java/example/pom.xml b/java/example/pom.xml index 3436995a228..697f7ffdf46 100644 --- a/java/example/pom.xml +++ b/java/example/pom.xml @@ -5,7 +5,7 @@ io.vitess vitess-parent - 16.0.0-SNAPSHOT + 16.0.0 vitess-example diff --git a/java/grpc-client/pom.xml b/java/grpc-client/pom.xml index 9a5fd6ca49a..60be34c5152 100644 --- a/java/grpc-client/pom.xml +++ b/java/grpc-client/pom.xml @@ -5,7 +5,7 @@ io.vitess vitess-parent - 16.0.0-SNAPSHOT + 16.0.0 vitess-grpc-client diff --git a/java/jdbc/pom.xml b/java/jdbc/pom.xml index 54321a065e5..7b02cc70cbf 100644 --- a/java/jdbc/pom.xml +++ b/java/jdbc/pom.xml @@ -5,7 +5,7 @@ io.vitess vitess-parent - 16.0.0-SNAPSHOT + 16.0.0 vitess-jdbc diff --git a/java/pom.xml b/java/pom.xml index 4db3730f541..a327638d810 100644 --- a/java/pom.xml +++ b/java/pom.xml @@ -11,7 +11,7 @@ io.vitess vitess-parent - 16.0.0-SNAPSHOT + 16.0.0 pom Vitess Java Client libraries [Parent]