Unable to attach static IP (non-FQDN) node to a pool

[asmajlovic]

Environment

• TMOS/Bigip Version: BIG-IP 17.1.0.3 Build 0.0.4 Point Release 3
• Terraform Version: 1.3.3
• Terraform bigip provider Version: 1.22.4

Summary

Pool attachments appear to be always treated as FQDN nodes, despite the fact that a given node definition is making use of a static IP address. The result is that any static IP node pool attachment attempt will result in an error, stating that: "FQDN property cannot be used with static IP nodes".

Steps To Reproduce

Steps to reproduce the behavior:

variables.tf

variable "f5_partition" {
  type    = string
  default = "Common"
}

variable "nodes" {
  type = map(object({
    address           = optional(string)
    description       = optional(string)
    fqdn_autopopulate = optional(string, "enabled")
    fqdn_interval     = optional(number, 300)
    monitor           = optional(string, "default")
  }))
  default = {}
}

variable "pools" {
  type = map(object({
    allow_nat           = optional(string, "yes")
    allow_snat          = optional(string, "yes")
    description         = optional(string)
    load_balancing_mode = optional(string, "round-robin")
    monitors            = optional(list(string), ["http"])
    node_attachments    = optional(map(object({
      fqdn_autopopulate = optional(string, "enabled")
      priority_group    = optional(number, 0)
      ratio             = optional(number, 1)
      state             = optional(string, "enabled")
    })))
  }))
  default = {}
}

node.tf

resource "bigip_ltm_node" "node" {
  for_each    = var.nodes
  name        = "/${var.f5_partition}/${each.key}"
  address     = each.value.address == null ? each.key : each.value.address
  description = each.value.description
  monitor     = each.value.monitor

  // Consider entries without an address as an FQDN node
  dynamic "fqdn" {
    for_each = each.value.address == null ? [true] : []
    content {
      autopopulate = each.value.fqdn_autopopulate
      interval     = each.value.fqdn_interval
    }
  }
}

pool.tf

locals {
  pool_attachments = flatten([
    for pool_key, pool_values in var.pools : [
      for attachment_key, attachment_values in pool_values.node_attachments : {
        pool_name              = pool_key
        node_name              = attachment_key
        node_fqdn_autopopulate = attachment_values.fqdn_autopopulate
        node_priority_group    = attachment_values.priority_group
        node_ratio             = attachment_values.ratio
        node_state             = attachment_values.state
      }
    ]
  ])
}

resource "bigip_ltm_pool" "pool" {
  for_each            = var.pools
  name                = "/${var.f5_partition}/${each.key}"
  description         = each.value.description
  load_balancing_mode = each.value.load_balancing_mode
  monitors            = formatlist("/%s/%s", var.f5_partition, each.value.monitors)
  allow_nat           = each.value.allow_nat
  allow_snat          = each.value.allow_snat
  depends_on          = [bigip_ltm_monitor.monitor]
}

resource "bigip_ltm_pool_attachment" "attach_node" {
  for_each = {
    for entry in local.pool_attachments : "${entry.pool_name} : ${entry.node_name} : ${entry.node_fqdn_autopopulate} : ${entry.node_priority_group} : ${entry.node_ratio} : ${entry.node_state}" => entry
  }
  node              = each.value.node_name
  pool              = "/${var.f5_partition}/${each.value.pool_name}"
  state             = each.value.node_state
  ratio             = each.value.node_ratio
  priority_group    = each.value.node_priority_group
  fqdn_autopopulate = each.value.node_fqdn_autopopulate
  depends_on        = [bigip_ltm_pool.pool, bigip_ltm_node.node]
}

misc.auto.tfvars

nodes = {
  "cerastes" = {
    description = "Static IP test node #1"
    address     = "172.20.1.10"
  }
}

pools = {
  "myth" = {
    node_attachments = {
      "cerastes:5000" = {}
    }
    monitors = [
      "tcp"
    ]
  }
}

Terraform plan

The plan for the above resources is generated without any issues:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create                                         
                                                   
Terraform will perform the following actions:      
                                         
  # module.F5_automation.bigip_ltm_node.node["cerastes"] will be created
  + resource "bigip_ltm_node" "node" {             
      + address          = "172.20.1.10"       
      + connection_limit = (known after apply)
      + description      = "Static IP test node #1"
      + dynamic_ratio    = (known after apply)
      + id               = (known after apply)
      + monitor          = "default"
      + name             = "/Common/cerastes"
      + rate_limit       = (known after apply)
      + ratio            = (known after apply)
      + session          = (known after apply)
      + state            = (known after apply)
    }                                                                               
                                         
  # module.F5_automation.bigip_ltm_pool.pool["myth"] will be created
  + resource "bigip_ltm_pool" "pool" {
      + allow_nat              = "yes"                                                                                                    
      + allow_snat             = "yes"   
      + id                     = (known after apply)
      + load_balancing_mode    = "round-robin"
      + minimum_active_members = (known after apply)
      + monitors               = [      
          + "/Common/tcp",
        ]
      + name                   = "/Common/myth"                                                                                                                        
      + reselect_tries         = (known after apply)
      + service_down_action    = (known after apply)
      + slow_ramp_time         = (known after apply)
    }

  # module.F5_automation.bigip_ltm_pool_attachment.attach_node["myth : cerastes:5000 : enabled : 0 : 1 : enabled"] will be created
  + resource "bigip_ltm_pool_attachment" "attach_node" {
      + connection_limit      = (known after apply)
      + connection_rate_limit = (known after apply)
      + dynamic_ratio         = (known after apply)
      + fqdn_autopopulate     = "enabled"
      + id                    = (known after apply)
      + monitor               = (known after apply)
      + node                  = "cerastes:5000"
      + pool                  = "/Common/myth"
      + priority_group        = 0
      + ratio                 = 1
      + state                 = "enabled"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

Attempting to apply the changes results in a pool attachment error:

module.F5_automation.bigip_ltm_node.node["cerastes"]: Creating...
module.F5_automation.bigip_ltm_node.node["cerastes"]: Creation complete after 0s [id=/Common/cerastes]
module.F5_automation.bigip_ltm_pool.pool["myth"]: Creating...
module.F5_automation.bigip_ltm_pool.pool["myth"]: Creation complete after 1s [id=/Common/myth]
module.F5_automation.bigip_ltm_pool_attachment.attach_node["myth : cerastes:5000 : enabled : 0 : 1 : enabled"]: Creating...
╷
│ Error: failure adding node cerastes:5000 to pool /Common/myth: 01070734:3: Configuration error: Node (/Common/cerastes) is configured using static IP (172.20.1.10). FQDN property cannot be used with static IP nodes. Delete the node if you want to reconfigure it as FQDN.
│ 
│   with module.F5_automation.bigip_ltm_pool_attachment.attach_node["myth : cerastes:5000 : enabled : 0 : 1 : enabled"],
│   on ../modules/f5/pool.tf line 27, in resource "bigip_ltm_pool_attachment" "attach_node":
│   27: resource "bigip_ltm_pool_attachment" "attach_node" {

Expected Behavior

Static IP address node is identified as an existing resource and attached to a pool without any FQDN considerations. The API / UI permits the creation of an arbitrarily named node and preferred IPv4 / IPv6 address - there appears to be no obvious reason why the Terraform provider would not provide the same functionality.

Actual Behavior

Static IP address node pool attachment fails with an FQDN property error (see error output above).

Current workaround

For the time being, we have updated our tfvars source file to reference an IPv4 address for both the node name and address values:

 nodes = {
  // cerastes
  "172.20.1.10" = {
    description = "Static IP test node #1"
    address     = "172.20.1.10"
  }
}

 pools = {
  "myth" = {
    node_attachments = {
      // cerastes
      "172.20.1.10:5000" = {}
    }
    monitors = [
      "tcp"
    ]
  }
}

This feels like unnecessary duplication of node name and address, as well as a resulting resource that has no obvious reference in the node name (we leave a comment in the variables source file to indicate its purpose), but it works as expected:

# terraform apply
module.F5_automation.bigip_ltm_node.node["172.20.1.10"]: Creating...                                                                         
module.F5_automation.bigip_ltm_pool.pool["myth"]: Creating...                                                                                 
module.F5_automation.bigip_ltm_node.node["172.20.1.10"]: Creation complete after 0s [id=/Common/172.20.1.10]                                  
module.F5_automation.bigip_ltm_pool.pool["myth"]: Creation complete after 1s [id=/Common/myth]                                                    
module.F5_automation.bigip_ltm_pool_attachment.attach_node["myth : 172.20.1.10:5000 : enabled : 0 : 1 : enabled"]: Creating...                    
module.F5_automation.bigip_ltm_pool_attachment.attach_node["myth : 172.20.1.10:5000 : enabled : 0 : 1 : enabled"]: Creation complete after 0s [id=/Common/myth]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

Additional information

I have noted several previous Github issues that appear to be similar / related to what has been outlined above:

• bigip_ltm_pool_attachment Can't upgrade to 1.3.3 from 1.3.2 #361
• 1.3.3 completely breaks bigip_ltm_pool_attachment #363
• Unable to attach existing node to a pool in version 1.4.0 #380
• Unable to set priority group on pool member #381
• bigip_ltm_pool_attachment - not able to assign name #395

The last entry in the list is the closest duplicate I have been able to find. It seems that this was functional in version 1.6.0 of the F5 bigip Terraform provider and I'm afraid that I do not know when exactly the behaviour changed. It is also unclear why there have been occasional statements to this being "legacy" pool attachment behaviour. As already mentioned above, the API / UI supports the feature - while that remains true, I see no reason why the Terraform provider should not as well.

Any input, clarification, or suggestions (should we be doing something incorrectly) on the above would be very much appreciated. Thank you.

[pgouband]

Hi @asmajlovic,

I tested the following with success.
The issue is on node value as partition is needed.

resource "bigip_ltm_pool_attachment" "attach_node" {
  for_each = {
    for entry in local.pool_attachments : "${entry.pool_name} : ${entry.node_name} : ${entry.node_fqdn_autopopulate} : ${entry.node_priority_group} : ${entry.node_ratio} : ${entr
y.node_state}" => entry
  }
  node              = "/${var.f5_partition}/${each.value.node_name}"
  pool              = "/${var.f5_partition}/${each.value.pool_name}"
  state             = each.value.node_state
  ratio             = each.value.node_ratio
  priority_group    = each.value.node_priority_group
  depends_on        = [bigip_ltm_pool.pool, bigip_ltm_node.node]
}

$ terraform plan -out poolattach

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # bigip_ltm_node.node["cerastes"] will be created
  + resource "bigip_ltm_node" "node" {
      + address          = "172.20.1.10"
      + connection_limit = (known after apply)
      + description      = "Static IP test node #1"
      + dynamic_ratio    = (known after apply)
      + id               = (known after apply)
      + monitor          = "default"
      + name             = "/Common/cerastes"
      + rate_limit       = (known after apply)
      + ratio            = (known after apply)
      + session          = (known after apply)
      + state            = (known after apply)
    }

  # bigip_ltm_pool.pool["myth"] will be created
  + resource "bigip_ltm_pool" "pool" {
      + allow_nat              = "yes"
      + allow_snat             = "yes"
      + id                     = (known after apply)
      + load_balancing_mode    = "round-robin"
      + minimum_active_members = (known after apply)
      + monitors               = [
          + "/Common/tcp",
        ]
      + name                   = "/Common/myth"
      + reselect_tries         = (known after apply)
      + service_down_action    = (known after apply)
      + slow_ramp_time         = (known after apply)
    }

  # bigip_ltm_pool_attachment.attach_node["myth : cerastes:5000 : enabled : 0 : 1 : enabled"] will be created
  + resource "bigip_ltm_pool_attachment" "attach_node" {
      + connection_limit      = (known after apply)
      + connection_rate_limit = (known after apply)
      + dynamic_ratio         = (known after apply)
      + id                    = (known after apply)
      + monitor               = (known after apply)
      + node                  = "/Common/cerastes:5000"
      + pool                  = "/Common/myth"
      + priority_group        = 0
      + ratio                 = 1
      + state                 = "enabled"
    }

Plan: 3 to add, 0 to change, 0 to destroy.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

Saved the plan to: poolattach

To perform exactly these actions, run the following command to apply:
    terraform apply "poolattach"


$ terraform apply "poolattach"
bigip_ltm_node.node["cerastes"]: Creating...
bigip_ltm_pool.pool["myth"]: Creating...
bigip_ltm_node.node["cerastes"]: Creation complete after 0s [id=/Common/cerastes]
bigip_ltm_pool.pool["myth"]: Creation complete after 0s [id=/Common/myth]
bigip_ltm_pool_attachment.attach_node["myth : cerastes:5000 : enabled : 0 : 1 : enabled"]: Creating...
bigip_ltm_pool_attachment.attach_node["myth : cerastes:5000 : enabled : 0 : 1 : enabled"]: Creation complete after 0s [id=/Common/myth-/Common/cerastes:5000]

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

[asmajlovic]

Now I feel a bit stupid for raising the "issue" in the first place 😄 #rtfm

Thank you for your prompt response and code debugging - much obliged. I'll happily finance a coffee or beer for you (just send a suitable, non-crypto, link). Other than that, we can close this off.

[pgouband]

Hi @asmajlovic,

no pb.