api.err.InvalidPayload for everything 7.1.66

[matutter]

I'm using the latest unifi-controller version, 7.1.66, from linuxserver/unifi-controller and nothing seems to work. Is the latest version supported?

error:

Plan: 1 to add, 0 to change, 0 to destroy.
unifi_network.vlan8: Creating...
╷
│ Error: api.err.InvalidPayload (400 ) for POST https://myunifi:8443/api/s/default/rest/networkconf
│ 
│   with unifi_network.vlan8,
│   on main.tf line 17, in resource "unifi_network" "vlan8":
│   17: resource "unifi_network" "vlan8" {

main.tf:

terraform {
  required_providers {
    unifi = {
      source = "paultyng/unifi"
      version = "0.34.0"
    }
  }
}

provider "unifi" {
  username = "testadmin"
  password = "ChangeMe123"
  api_url = "https://myunifi:8443"
  allow_insecure = true
}

resource "unifi_network" "vlan8" {
  name    = "vlan8"
  purpose = "corporate"
  vlan_id = 8

  subnet       = "10.20.8.1/24"
  dhcp_start   = "10.20.8.6"
  dhcp_stop    = "10.20.8.254"
  dhcp_enabled = true
}

docker-compose.yml

version: "3.9"

volumes:
  unify-controller-config:

networks:
  unify-test:
    # name: unify-test
    # attachable: true
    external: true

services:
  unifi-controller:
    image: lscr.io/linuxserver/unifi-controller:latest
    networks:
      unify-test:
        aliases:
        - myunifi
    container_name: unifi-controller
    environment:
      - PUID=1000
      - PGID=1000
      - MEM_LIMIT=1024 #optional
      - MEM_STARTUP=1024 #optional
    volumes:
      - unify-controller-config:/config
    ports:
      - 8443:8443
      - 8080:8080
      #- 3478:3478/udp
      #- 10001:10001/udp
      #- 1900:1900/udp #optional
      #- 8843:8843 #optional
      #- 8880:8880 #optional
      #- 6789:6789 #optional
      #- 5514:5514/udp #optional
    restart: unless-stopped

[matutter]

Using mitmproxy I pulled this request body out of the POST /api/s/default/rest/networkconf request:

{
  "auto_scale_enabled": false,
  "dhcpd_boot_enabled": false,
  "dhcpd_boot_server": "",
  "dhcpd_dns_1": "",
  "dhcpd_dns_2": "",
  "dhcpd_dns_3": "",
  "dhcpd_dns_4": "",
  "dhcpd_dns_enabled": false,
  "dhcpd_enabled": true,
  "dhcpd_gateway": "",
  "dhcpd_gateway_enabled": false,
  "dhcpd_ip_1": "",
  "dhcpd_ip_2": "",
  "dhcpd_ip_3": "",
  "dhcpd_leasetime": 86400,
  "dhcpd_mac_1": "",
  "dhcpd_mac_2": "",
  "dhcpd_mac_3": "",
  "dhcpd_ntp_1": "",
  "dhcpd_ntp_2": "",
  "dhcpd_ntp_enabled": false,
  "dhcpd_start": "10.20.8.6",
  "dhcpd_stop": "10.20.8.254",
  "dhcpd_time_offset_enabled": false,
  "dhcpd_unifi_controller": "",
  "dhcpdv6_dns_auto": false,
  "dhcpdv6_enabled": false,
  "dhcpd_wins_1": "",
  "dhcpd_wins_2": "",
  "dhcpd_wins_enabled": false,
  "dhcp_relay_enabled": false,
  "dhcpguard_enabled": false,
  "dpi_enabled": false,
  "dpigroup_id": "",
  "domain_name": "",
  "enabled": true,
  "exposed_to_site_vpn": false,
  "gateway_device": "",
  "igmp_fastleave": false,
  "igmp_querier": "",
  "igmp_snooping": false,
  "igmp_supression": false,
  "ipsec_dynamic_routing": false,
  "ipsec_pfs": false,
  "ip_subnet": "10.20.8.1/24",
  "ipv6_interface_type": "none",
  "ipv6_pd_prefixid": "",
  "ipv6_ra_enabled": false,
  "is_nat": false,
  "lte_lan_enabled": false,
  "mac_override": "",
  "mac_override_enabled": false,
  "name": "vlan8",
  "networkgroup": "LAN",
  "pptpc_require_mppe": false,
  "purpose": "corporate",
  "radiusprofile_id": "",
  "remote_site_id": "",
  "report_wan_event": false,
  "require_mschapv2": false,
  "upnp_lan_enabled": false,
  "usergroup_id": "",
  "vlan": 8,
  "vlan_enabled": true,
  "vpn_client_default_route": false,
  "vpn_client_pull_dns": false,
  "wan_dns1": "",
  "wan_dns2": "",
  "wan_dns3": "",
  "wan_dns4": "",
  "wan_gateway": "",
  "wan_gateway_v6": "",
  "wan_ipv6": "",
  "wan_provider_capabilities": {},
  "wan_smartq_enabled": false,
  "wan_vlan_enabled": false
}

However a valid request would look like this:

{
	"auto_scale_enabled": true,
	"dhcp_relay_enabled": false,
	"dhcpd_boot_enabled": false,
	"dhcpd_dns_enabled": false,
	"dhcpd_enabled": true,
	"dhcpd_gateway_enabled": false,
	"dhcpd_leasetime": 86400,
	"dhcpd_ntp_enabled": false,
	"dhcpd_start": "192.168.2.6",
	"dhcpd_stop": "192.168.2.254",
	"dhcpd_tftp_server": "",
	"dhcpd_time_offset_enabled": false,
	"dhcpd_unifi_controller": "",
	"dhcpd_wpad_url": "",
	"dhcpguard_enabled": false,
	"domain_name": "",
	"enabled": true,
	"gateway_type": "default",
	"igmp_snooping": false,
	"ip_subnet": "192.168.2.1/24",
	"ipv6_interface_type": "none",
	"ipv6_ra_enabled": false,
	"lte_lan_enabled": true,
	"mdns_enabled": true,
	"name": "test123",
	"networkgroup": "LAN",
	"purpose": "corporate",
	"setting_preference": "manual",
	"upnp_lan_enabled": false,
	"vlan": 7,
	"vlan_enabled": true
}

It seems all the omitempty flags used to marshal the structs to JSON are not working. Should I file this under the go/unify package instead?

[EDIT] Here is the specific response from the server:

{
    "data": [
        {
            "msg": "api.err.InvalidValue",
            "rc": "error",
            "validationError": {
                "field": "wan_gateway",
                "pattern": "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])$"
            }
        }
    ],
    "meta": {
        "msg": "api.err.InvalidPayload",
        "rc": "error"
    }
}

And I can confirm that modifying the request to completely omit wan_gateway does make the request succeed.

[joshuaspence]

Yeah, v7 isn't supported properly at the moment but should be partially working. I am trying to get time to fix this but haven't quite gotten to it yet. Feel free to submit a PR or I will try to get to it soonish

[joshuaspence]

It seems all the omitempty flags used to marshal the structs to JSON are not working

What do you mean by this? I see dhcpd_boot_filename (which is omitempty) is absent, for example

[matutter]

It seems all the omitempty flags used to marshal the structs to JSON are not working

What do you mean by this? I see dhcpd_boot_filename (which is omitempty) is absent, for example

I am specifically looking at the wan_gateway tags json:"wan_gateway,omitempty". It ends up in the JSON with the value "" instead of being absent even if set to null explicitly.

[joshuaspence]

Ok yeah, not immediately sure what's going on there. I'm not a fan of the omitempty logic we have at the moment. I think a better approach, albeit messier, would be for us to serialise the object based on its type. E.g. none of the WAN fields should be set unless the network type is WAN

[paultyng]

I think at least some of these issues where we were expecting specific errors will be fixed in #254

That only will fix the error reporting though, not any marshalling issues.

With this additional information in the error message, I'll have to see if I can also surface validation errors in a nicer way.

Still probably quite a bit of work to find all these issues in 7.1 though, the release notes are not very explicit.

[paultyng]

FWIW my home network is on 7.1 and I'm about to start parental leave so I may be motivated to work through this soon!

[paultyng]

I think that omitempty was only merged in the last few months, at least looking in commit history, although I may be misreading when it landed in the go.mod, hard to trace since its multiple repos. This test seems to pass on 7.1 (with dhcp and vlan id, but no wan gateway) though so it looks like this may be working now?