Skip to content

HAProxy Data Plane API v3.0.0
Compare
Choose a tag to compare
@github-actions github-actions released this 10 Sep 16:52
· 16 commits to master since this release
v3.0.0
1491b08

Release Notes

HAProxy Data Plane API v3.0.0 1491b08

Build from: [email protected]:haproxytech/dataplaneapi.git
Build date: 2024-09-10T15:39:08Z

Key changes in the HAProxy Data Plane API 3.0 release include the following. For more information checkout our blog post announcement and release notes with all the examples.

Breaking changes

Data Plane API moves to v3, and our API resources now start with /v3 instead of /v2. In v3 of the API we introduce a number of breaking changes to improve API usability and readability. These are all based on feedback in the last four years since the release of v2.0.0. Working on that feedback we changed things around and organized it better to make the API more usable for everyone.

Removed _version and data wrapper fields in responses

Data Plane API uses versions for optimistic locking of the configuration, and in the initial design we returned data on each configuration endpoint with _version and data fields as top keys in the response object. Sometime later we also included the Configuration-Version header. In v3 we are getting rid of the redundant configuration version in the response, making the response more straightforward.

Rename of the defaults resource

In the beginning of the Data Plane API we wanted to have only one defaults section, and that resided in the /v2/services/haproxy/configuration/defaults resource. That was due to the fact that the name had no meaning in the functionality of the HAProxy instance. With HAproxy 2.4, from keywords were introduced to frontend and backend sections, and that gave more sense to the functionality of having multiple named defaults. Since the defaults resource was returning an object and not a list, to keep backward compatibility we implemented another resource called named_defaults that returned a list of defaults resources that had a name and could be referenced in backend and frontend sections. The defaults resource was kept for backward compatibility as an object. With v3 of the API, we are migrating named_defaults to simply defaults, while breaking the old defaults behavior. Now the defaults resource returns a list instead of an object.

Moving child resources as nested resources

In the v2 API we have some resources that represent lines inside a section of HAProxy configuration, making them effectively child resources of the parent section resource. One example is the acl resource that could be a child of a frontend, backend or server resource. In v2 two you could access these resources on the API by identifying them with their index (such as for acl) or name (such as for servers) and specifying their parents by using parent_type and parent_name query-string parameters.

Over the years, users found this confusing so now we are redesigning this into having nested resources in the URL. One child resource cannot simultaneously exist in multiple parent resources so it makes more sense to have this as part of the URL of the resource and also helps with many external tools for the future.

Example:

v2/services/haproxy/configuration/acls?parent_type=backend&parent_name=my_bck is now at v3/services/haproxy/configuration/backends/my_bck/acls

Removing explicit index field on index based child resources

In v2 of the API all index-based resources, which represent lines in a configuration that can be repeated inside a section and are distinguished by the order in which they appear in the section, had an index field that told us the position where this specific resource appeared. For example, the acls resource. For v3 we are removing that field as we find it redundant and it is complicated to maintain when positions change. You can infer the index from the actual position in the list returned, and access it at this position.

Remove support for HAProxy in multi-process mode

HAProxy had a feature that allowed it to run in multi-process mode. When HAProxy 2.0 introducing multi-thread mode, this was discouraged. It was deprecated in 2.3 and in 2.5 it was completely removed. In v2 of the API we supported all the keywords related to multi-process mode, and also supported multiple runtime APIs, one for each process mode, so all /runtime resources worked with multiple process support. In v3 of the API we are removing this support altogether. We removed the following fields from the the respective sections:

  • bind_process from the backend resource
  • process from the bind resource
  • bind_process from the defaults resource
  • bind_process from the frontend resource
  • nbproc from the global resource

In addition to removing these fields, /runtime endpoints received changes.

Stats endpoint change

The statistics resource at /v2/services/haproxy/stats/native used to return two layers of arrays, the top layer is stats per process, where then each item contains an array of stats lines per object: server, backend, and frontend. Now, as we don't support multi-process mode, /v3/services/haproxy/stats/native returns an object with only one level of stats arrays per object.

Process Info endpoint change

This change is similar to stats info change where we are not supporting multiple processes anymore. In v3 we removed one layer of arrays, which returned process infos per process.

Runtime Stick Table change

This change affected /v2/services/haproxy/runtime/stick_table in a way that the process field is removed from the responses, which used to return data from the process from which we were fetching the stick table information. In addition to this, the query-string parameter process was removed for both the /v2/services/haproxy/runtime/stick_tables and /v2/services/haproxy/runtime/stick_table_entries resources.

Reorganize Global resource fields

We found a need to logically reorganize the global fields. Global is our biggest resource if you measure it by number of fields, and it was mostly flat structured. To improve readability of this resource on the API, we decided to reorganize the fields into logical nested objects.

The API adheres to the field naming in HAProxy and some documentation hints in the HAProxy documentation, while trying to keep only one level of nested objects so as not to unnecessarily make the object more complex. There are still some fields left in the root level of the object, and some options are split into debug_options and performance_options. Also, we've added an ssl_options nested object to cover all SSL options, in addition to adding the tune_options to cover all tune.* options. There are additional nested objects like tune_ssl_options, tune_vars_options, tune_zlib_options, etc. which are all derived from tune.*.* options. Also following the dot notation in the configuration, options now exist under http_client_options, fifty_one_degrees_options, etc. For a full, detailed specification that describes the fields please see the global model specification defined in the Swagger definition.

Removal of deprecated fields and resources

During the development of Data Plane API, we deprecated some fields and resources in order to change or improve the behavior and support configuration keywords. Now we are removing those deprecated keywords. Here is an extensive list of removed deprecations in v3:

  • Global section: Removed deprecated tune_ssl_default_dh_param that can now be found in tune_ssl_options as default_dh_param.

  • Backend section: Removed deprecated http-check. You can now specify multiple http_check resources on /v3/services/haproxy/configuration/backends/{parent_name}/http_checks. Removed http-keep-alive, http-server-close, and httpclose fields as they are mutually exclusive and can be set using http_connection_mode field.

  • HTTP Request/Response Rules and TCP Request Rule resource: Removed deprecated track_sc<stick_counter> fields. In v2 we supported only 0, 1 and 2 stick counter indexes by having those hard-coded in the field names. Now with this change we added the track_sc_stick_counter field, which specifies the stick counter index and can be more than 2 if configured using the tune.stick_counters keyword in global (it defaults to 3), which was introduced in HAProxy 2.8.

  • Service Discovery resources: Removed the deprecated service-whitelist and service-blacklist, which were replaced by service_allowlist and service_denylist respectively.

Further API Improvements

This section describes other API improvements.

Fetching a parent resource with all children embedded

We've found that our granular approach to creating resources on the API can be helpful, but also in some cases complex. In v3 we've added an extra query-string parameter full_section on all the section resources (such as frontend and backend resources) which allows you to fetch, create, or replace a complete section with all of its child resources.

This query-string parameter works on both PUT and POST endpoints, meaning that you can now edit or create the whole parent section in just one API call. An important thing to note here is that we have two different types of child resources. One is the named child resource (in this example the bind resource). These are all the resources that can be uniquely identified by their name inside a parent section, and then that name is used as a key and those resources are represented as a map. The other child resource type is index-based as its unique identifier is the position in the configuration where the directive appears. It is represented as an index of a zero-based array. Those have a suffix _list.

This change helps you read and change the HAProxy configuration in less API calls.

Adding PUT operation on index based resources

While using the v2 API, if you wanted to make automated changes on a list of index based resources, it proved to be complex for a multitude of reasons. First, you had to reconcile all the index fields in each resource, and for each update it was hard to find the resource if something was deleted or added as you had to reconcile the indexes of each resource. Also, if the list was bigger with multiple changes, you were stuck with making multiple API calls.

With v3, we added PUT operation on the list endpoints, which allows you to atomically replace the complete list of resources.

Handle timeout options with preferred units

In the API we handle timeout options as integers, as we want to represent numeric values as numbers wherever possible. However, HAProxy supports strings by accepting time unit suffixes and in v2 we would recalculate the user's input values to the default unit (in most cases ms) and write it as an integer in the configuration file. This proved challenging for anyone using and reading the raw configuration file, as it changed their preferred inputs in the file, making it less readable.

In v3 we are introducing a new option in the Data Plane API configuration file called preffered_time_suffix where you can specify one of the options:

  • nearest
  • none
  • ms
  • s
  • m
  • h
  • d

Data Plane API will then honor your preferred time suffix. By default it's nearest, which means that it will be calculated to the smallest possible value with the according suffix. When using none, your timeout keywords in the configuration file will be written without the suffix using the default value for the respective keyword.

For further information, we've added some documentation to our Open API specification by annotating such fields with "x-duration": true and x-default-unit which indicates the default time unit for that specific field.

Keep Data Plane API config file unchanged

Since the Data Plane API doesn't have its own storage, it wrote its state to the configuration file. Notably, it wrote service discovery and users state there. This is problematic when the Data Plane API is used in combinations with automation software like Ansible, Puppet, etc.

In v3, Data Plane API will no longer write to its own configuration file. We are introducing something called dataplane_storage_dir, where Data Plane API's state will be stored, for now in specific JSON files for each feature. So now, you'll find users.json, service_discovery/consul.json and service_discovery/aws.json. It will all be migrated on the first start, and used from there in the future. We opted for using plain JSON files so it's easier for users to read and debug those files.

HAProxy Configuration Keyword Support

In addition to all the changes above, with this release we are fully supporting HAProxy 3.0 configuration with all the newly added keywords and features. You can find out more about that in the release blog post

Changelog

  • f36e1ef BUG/MAJOR: cluster: fix autogenerated cluster username
  • b66268a BUG/MAJOR: reconfigure runtime client and HAPorxy if path changes in the configuration file
  • 7d27cab BUG/MAJOR: reload runtime client if the configuration is changed with global endpoint
  • cf3d28d BUG/MEDIUM: change some storage api types to work with gorm
  • 2c99b56 BUG/MEDIUM: change track-sc<0|1|2> to track-sc
  • 83b7655 BUG/MEDIUM: configuration: Fix nil pointer dereference
  • 8fda41a BUG/MEDIUM: configuration: fix using backups_dir option
  • 8dc2875 BUG/MEDIUM: consul: fix health check if policy is not set
  • 0e3db1c BUG/MEDIUM: fix add server endpoint for other resources than backend.
  • 14ab8aa BUG/MEDIUM: log: fix blocking on syslog target communication errors
  • 2d436bd BUG/MEDIUM: retry opening the runtime socket and allow delay start at startup
  • b1084f8 BUG/MEDIUM: support multiple force-persist statements in backends
  • 692c06d BUG/MEDIUM: support multiple ignore-persist statements in backends
  • 6161c18 BUG/MINOR: allow all HTTP methods in httpchk_params and http-check
  • 2be281b BUG/MINOR: aws: sanity checking for instance service tags
  • f7d93b3 BUG/MINOR: cluster_sync: properly store storage dirs to file
  • 28f17ff BUG/MINOR: configuration: fix missing logs during configuration loading
  • eed09d0 BUG/MINOR: configuration: maintain backward compatibility for scheme
  • 150525f BUG/MINOR: consul: fix health check policy validation in create handler
  • c75026a BUG/MINOR: consul: fix validation for service changed check
  • 298772f BUG/MINOR: consuls: fix ns option to be set as query param instead of header
  • a412d0f BUG/MINOR: detect when a raw configuration is uploaded incorrectly
  • cf24d66 BUG/MINOR: fix panic in configuration when setting reload command
  • 9bc95ee BUG/MINOR: fix panic when trying to start with hcl configuration file
  • 48f0d03 BUG/MINOR: log_forwards: fix proper parent name and type checks
  • 5fa91c1 BUG/MINOR: prevent possible panics when generating random string
  • 76ccd72 BUG/MINOR: raw: return full raw config on GET and POST responses
  • ef54422 BUG/MINOR: reloads: trigger callback in force reload even when next reload exists
  • d4475ab BUG/MINOR: resolve-net, resolve_opt: e2e tests to allow comma separated list of arguments
  • 417ea79 BUG/MINOR: table: Add support for table in peers section
  • c346f34 BUILD/MAJOR: client-native: upgrade client-native to v5
  • bb3751c BUILD/MEDIUM: go.mod: upgrade client-native and generate
  • a7866a1 BUILD/MEDIUM: go: upgrade golang to 1.22
  • 0264138 BUILD/MEDIUM: goreleaser: Build apk, rpm and deb packages
  • e0c05d7 BUILD/MEDIUM: lint: apply stricter rules for linting
  • 805e0d1 BUILD/MEDIUM: update linters, use local version
  • cfc7d1a BUILD/MINOR: add checks if the code is formatted
  • 930572d BUILD/MINOR: add test for ca-verify-file bind parameter
  • 95f39e3 BUILD/MINOR: bind e2e tests >2.8 were not executed due to missing import lib and wrong fixture
  • 8e2ba9d BUILD/MINOR: bump go version to 1.21
  • 6758dae BUILD/MINOR: ci: add newer lint image
  • 723cb5c BUILD/MINOR: ci: remove only and use rules for running
  • 22c6b6b BUILD/MINOR: client-native: upgrade client native for new keywords
  • 2896f49 BUILD/MINOR: client-native: upgrade client-native version
  • 84f71d4 BUILD/MINOR: client-native: upgrade client-native version
  • 1491b08 BUILD/MINOR: cn: update client-native and generate spec
  • 44d49e0 BUILD/MINOR: do not rely on $PWD to be defined
  • 41332bd BUILD/MINOR: e2e: upgrade to 2.9 haproxy as default
  • 086fa7c BUILD/MINOR: git: ignore Goland ide configuration directory
  • 6e4a9fb BUILD/MINOR: github: add manual release action
  • c334dd7 BUILD/MINOR: github: use proper go version as a string
  • 1fbc47b BUILD/MINOR: global e2e tests >2.8 were not executed due to missing import lib
  • 809d8bf BUILD/MINOR: go.mo: upgrade client-native and generate new spec
  • 3a5dd5a BUILD/MINOR: go.mod: update Go indirect packages
  • 2dbfce7 BUILD/MINOR: go.mod: update Go indirect packages
  • 3b1038c BUILD/MINOR: go.mod: update Go indirect packages
  • bb50e04 BUILD/MINOR: go.mod: update Go packages
  • 0961699 BUILD/MINOR: go.mod: update Go packages
  • 4c14425 BUILD/MINOR: go.mod: update Go packages
  • 52af6e5 BUILD/MINOR: go.mod: update Go packages
  • d4621df BUILD/MINOR: go.mod: update Go packages
  • 811cf0c BUILD/MINOR: go.mod: update Go packages
  • ab3976b BUILD/MINOR: go.mod: update Go packages
  • 3f2e600 BUILD/MINOR: go.mod: update Go packages
  • e612bfb BUILD/MINOR: go.mod: update Go packages
  • 97e699e BUILD/MINOR: go.mod: update Go packages
  • 514233d BUILD/MINOR: go.mod: update Go packages
  • cac59ac BUILD/MINOR: go.mod: update client-native and config-parser
  • 2744c23 BUILD/MINOR: go.mod: update client-native and generate spec
  • 8805d97 BUILD/MINOR: go.mod: update client-native and specification
  • ac5ea55 BUILD/MINOR: go.mod: update client-native and specification
  • 6a94565 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 70e0171 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • ccb90e4 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 11698db BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • b3ea931 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • f4d31c2 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 55954b6 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 3ab8533 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • cb61979 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 86ad400 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 8cd5030 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 0608772 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2
  • 2f26199 BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2/service/ec2 v1.92.0
  • f33821c BUILD/MINOR: go.mod: update github.com/aws/aws-sdk-go-v2/service/ec2 v1.92.1
  • b2d53c1 BUILD/MINOR: go.mod: update github.com/dustinkirkland/golang-petname, go.mongodb.org/mongo-driver
  • 6a16ca2 BUILD/MINOR: go.mod: update github.com/perimeterx/marshmallow v1.1.5
  • 783a763 BUILD/MINOR: go.mod: update golang packages
  • c24efbc BUILD/MINOR: go.mod: update golang.org/x packages
  • 9857cc4 BUILD/MINOR: go.mod: update golang.org/x packages
  • eb0c1a8 BUILD/MINOR: go.mod: update golang.org/x packages
  • b3e834b BUILD/MINOR: go.mod: update golang.org/x packages
  • 784f008 BUILD/MINOR: go.mod: update golang.org/x packages
  • 6908e4c BUILD/MINOR: go.mod: update golang.org/x packages
  • efb5f29 BUILD/MINOR: go.mod: update golang.org/x packages
  • 3504781 BUILD/MINOR: go.mod: update golang.org/x packages
  • f40ca70 BUILD/MINOR: go.mod: update golang.org/x packages
  • 4190d7d BUILD/MINOR: go.mod: update golang.org/x packages
  • 0b8447e BUILD/MINOR: go.mod: update golang.org/x packages
  • 234ab2d BUILD/MINOR: go.mod: update golang.org/x packages
  • 3daa28c BUILD/MINOR: go.mod: update golang.org/x packages
  • d9d269b BUILD/MINOR: go.mod: update golang.org/x packages
  • ede4d54 BUILD/MINOR: go.mod: update golang.org/x packages
  • 866fa44 BUILD/MINOR: go.mod: update golang.org/x packages
  • dbf42e8 BUILD/MINOR: go.mod: upgrade client-native
  • 6af9a1b BUILD/MINOR: go.mod: upgrade client-native
  • b5ce490 BUILD/MINOR: go.mod: upgrade client-native
  • f9953df BUILD/MINOR: go.mod: upgrade client-native
  • 7a0c055 BUILD/MINOR: go.mod: upgrade dependencies
  • 279dd42 BUILD/MINOR: go.mod: upgrade dependencies
  • 8994aef BUILD/MINOR: go.mod: upgrade dependencies
  • 88d4a2a BUILD/MINOR: go.mod: upgrade dependencies
  • 2e6d658 BUILD/MINOR: go.mod: upgrade dependencies and generate specification
  • 5e8f0c0 BUILD/MINOR: go.mod: upgrade to latest client-native
  • 0341077 BUILD/MINOR: go.mod: upgrade to the latest client-native
  • 1f986da BUILD/MINOR: go: upgrade go to 1.20
  • 6d210be BUILD/MINOR: goreleaser: Upgrade to goreleaser v2
  • 7f691eb BUILD/MINOR: goreleaser: always use latest go version
  • 5aadfce BUILD/MINOR: goreleaser: fix deprecation and release notes generation
  • 6596800 BUILD/MINOR: quic: update libraries to enable quic keywords
  • 79e2f42 BUILD/MINOR: ring e2e test >2.8 were not executed due to missing import lib
  • f6f3645 CLEANUP/MAJOR: lint: upgrade linter for go 1.22
  • dd3075d CLEANUP/MINOR: ci: remove deprecated gitlab token
  • 0c7a4a3 DOC/MINOR: aws: only one port is allowed
  • 6ae4387 MAJOR: hcl: remove support
  • ad9347d MAJOR: reload configuration after creating a new Storage SSL Certificate
  • 783cf25 MAJOR: upgrade client-native to v6 and swagger version to /v3
  • 5e0afb7 MEDIUM : sticky-counter, tune ocsp: Add e2e tests for sticky-counters to http/tcp and global ocsp
  • e2389be MEDIUM: API consolidation- add PUT on index based lists and remove index in objects
  • 25676bc MEDIUM: API consolidation: remove support for multi-process
  • d0600ef MEDIUM: Add size and certificates info to the storage API
  • ce9fab6 MEDIUM: Add support for the crt-store section
  • fb848fd MEDIUM: Remove state and data from dataplaneapi configuration file
  • dab0faa MEDIUM: Update to the latest version of client-native
  • e1e03d4 MEDIUM: add support for structured operations on sections
  • f956754 MEDIUM: api consolitation- Move child resources as nested resources
  • 78a4e05 MEDIUM: remove defaults and rename named_defaults
  • 956226e MINOR: Add configuration option for preferred Time suffix
  • 4ad3136 MINOR: Remove dependencies on hashicorp
  • ca4b0f6 MINOR: add command socket for debugging
  • 5ed2686 MINOR: add deprecation warning log to Httpclose HTTPKeepAlive and HTTPServerClose
  • 482bc7d MINOR: bind: Add e2e test cases for thread
  • c43512a MINOR: cluster: removing storage upon cluster removal
  • 3524234 MINOR: cmd: Improve container compatibility
  • 62dadaa MINOR: consul: add support for from field for created backends
  • e5709f9 MINOR: consul: add support for health check policy
  • 0bf9805 MINOR: consul: add support for regexp filtering service names
  • 1a7aaea MINOR: env: add option to load environment variables from .env file
  • 25e5a01 MINOR: lint: increase timeout for a linter
  • 661163b MINOR: remove deprecated in the spec
  • c5ce613 MINOR: service_discovery: add mode handling for consul api requests
  • 72bcaac MINOR: update client-native dependency
  • 4a5c714 MINOR: update client-native dependency
  • 13ba120 MINOR: utils: decoding configuration errors to HTTP status code
  • 2eda323 OPTIM/MINOR: go: Optimize struct field alignments
  • 68ab09f OPTIM/MINOR: go: Update x/net and x/sys to latest versions
  • 0226e75 REORG/MAJOR: returning flat object in response
  • d1cda1e REORG/MINOR: client-native configuration API changes
  • 33c2786 TEST/MAJOR: e2e: fix proper version checking in all tests
  • 7b6d352 TEST/MEDIUM: haproxy.cfg: fix test fixtures to pass reloads
  • a3d9943 TEST/MEDIUM: haproxy: add haproxy 2.7 to tests and set as default
  • bdf972b TEST/MINOR: docker: upgrade docker for dind and add seccomp option
  • d0bc060 TEST/MINOR: e2e: add tune.h2 and bind e2e tests
  • 9a90c61 TEST/MINOR: e2e: fix tests for debian
  • e5f1cc1 TEST/MINOR: e2e: remove unsupported versions from testing
  • 84a438c TEST/MINOR: e2e: save the logs of HAProxy and dpapi on error
  • f3281a8 TEST/MINOR: e2e: swap kill -SIGUSR with kill -s 12 invocations
  • 7996549 TEST/MINOR: fixtures: add scheme to the fixture configuration files
  • b47d1f0 TEST/MINOR: global: fix tests to accomodate new global structs
  • 112c9bc TEST/MINOR: groups: fix expected error code for fetching group from non-existing userlist
  • 56c34bb TEST/MINOR: set_uid: fix set_uid test to match regexp
Assets 33
Loading