Dnsdist is bypassing some traffic that must be blocked

[ahmedhkhalil]

Dears,
we have some blocking roles based on number of nxdomain responses from NS server configure on dnsdist , what we have discoverred is that most of IPs is blocked , but some of them is not blocked despite when check logs found that they hit limit and must be blocked , how to troubleshoot that may i know if there is limit on how many IPs by default dnsdist should block

[rgacogne]

Please share the version of dnsdist that you are using, the configuration and the logs showing the issue, otherwise there is nothing we can do to help you.

[ahmedhkhalil]

dnsdist 1.8.4

[ahmedhkhalil]

below image show number of requests that has been recieved at assoicated PowerDNS authroties server from only one of our DNsdist server as you can see that number of requests is larger than 2 and also during 20 seconds we exceed 6 requests , so according to our threshold it must be blocked since minute 1

[rgacogne]

Please share your configuration, otherwise there is no way for us to know what you are trying to achieve and whether this is working as expected.

[ahmedhkhalil]

below is sample of role that must hit
dbr:setRCodeRate(DNSRCode.NXDOMAIN, 4, 20, "exceeding", 70000 ,DNSAction.Drop, 3)****

[rgacogne]

Please don't do this: https://blog.powerdns.com/2016/01/18/open-source-support-out-in-the-open If you don't give us the full configuration we are spending a lot of time guessing, and this means wasting our time and yours.

[ahmedhkhalil]

-- dnsdist configuration

---------------------------------------------------
-- Dns services
---------------------------------------------------
addAction(OrRule({OpcodeRule(DNSOpcode.Notify), OpcodeRule(DNSOpcode.Update), QTypeRule(DNSQType.AXFR), QTypeRule(DNSQType.IXFR), QTypeRule(DNSQType.ANY),  OpcodeRule(DNSOpcode.Update), QTypeRule(DNSQType.SOA)}), RCodeAction(DNSRCode.REFUSED))

-- udp/tcp dns listening
setLocal("0.0.0.0:53",  {doTCP = true})


pc = newPacketCache(500000, {maxTTL=86400, minTTL=0, temporaryFailureTTL=300, staleTTL=300, dontAge=true})  -- 500k entries, 1-day TTL

-- Pools
---------------------------------------------------

pool_auth = "AUTH"

-- members definition
newServer({
  name = "NS01",
  address = "",
  useProxyProtocol=true,
  pool = pool_auth,
  qps = 4000,
})

newServer({
  name = "NS02",
  address = "",
  useProxyProtocol=true,
  pool = pool_auth,
  qps = 4000,
})

-- set the load balacing policy to use
setPoolServerPolicy(roundrobin, pool_auth)

-- enable cache for the pool
getPool(pool_auth):setCache(pc)

---------------------------------------------------
-- Rules
---------------------------------------------------

-- matches all incoming traffic and send-it to the pool of resolvers
addAction(
  AllRule(),
  PoolAction(pool_auth)
)


setACL({'0.0.0.0/0'})
bpf = newBPFFilter({ipv4MaxItems=30000, ipv6MaxItems=30000, qnamesMaxItems=60240})
setDefaultBPFFilter(bpf)


local dbr = dynBlockRulesGroup()
dbr:setQueryRate(100, 5, "Exceeded query rate", 600,DNSAction.Drop, 50)
dbr:setRCodeRate(DNSRCode.NXDOMAIN, 1, 1, "Exceeded NXD rate", 1800,DNSAction.Drop, 1)
dbr:setRCodeRate(DNSRCode.NXDOMAIN, 4, 20, "Exceeded NXD rate2", 1800,DNSAction.Drop, 2)
dbr:setRCodeRate(DNSRCode.SERVFAIL, 10, 1, "Exceeded ServFail rate", 1200,DNSAction.Drop, 5)
dbr:setQTypeRate(DNSQType.ANY, 5, 1, "Exceeded ANY rate", 1200,DNSAction.Drop, 2)
dbr:setResponseByteRate(10000, 5, "Exceeded resp BW rate", 600,DNSAction.Drop, 5000)
function maintenance()
  dbr:apply()
end
setRingBuffersSize(1000000, 100)
setMaxTCPClientThreads(20)
webserver("0.0.0.0:8083")
setWebserverConfig({acl="0.0.0.0/0"})
setSyslogFacility(8)

[ahmedhkhalil]

sorry for late reply

[ahmedhkhalil]

@rgacogne hello and happy new year did you get chance to review above configuration

[rgacogne]

The configuration looks valid at a quick glance, but I still don't understand what is not working as expected. You state:

below image show number of requests that has been recieved at assoicated PowerDNS authroties server from only one of our DNsdist server as you can see that number of requests is larger than 2 and also during 20 seconds we exceed 6 requests , so according to our threshold it must be blocked since minute 1

but I'm afraid I lack the context to understand what the graph is actually displaying. With the rules that you have specified above, for a specific client IP to be blocked it would take:

• dbr:setQueryRate(100, 5, "Exceeded query rate", 600,DNSAction.Drop, 50): 100 qps sustained for at least 5 seconds
• dbr:setRCodeRate(DNSRCode.NXDOMAIN, 1, 1, "Exceeded NXD rate", 1800,DNSAction.Drop, 1): 1 NXDOMAIN response per second, sustained for at least one second BUT
• this is immediately overridden by dbr:setRCodeRate(DNSRCode.NXDOMAIN, 4, 20, "Exceeded NXD rate2", 1800,DNSAction.Drop, 2) which means 4 NXDOMAIN responses per second, for at least 20 seconds
• dbr:setRCodeRate(DNSRCode.SERVFAIL, 10, 1, "Exceeded ServFail rate", 1200,DNSAction.Drop, 5) 10 SERVFAIL responses per second, sustained for at least 1 second
• dbr:setQTypeRate(DNSQType.ANY, 5, 1, "Exceeded ANY rate", 1200,DNSAction.Drop, 2): 5 ANY queries per second, for at least 1 second
• or dbr:setResponseByteRate(10000, 5, "Exceeded resp BW rate", 600,DNSAction.Drop, 5000): 10k bytes per seconds for at least 5 seconds

It's impossible for me to determine from the graph you shared whether any of these conditions are met by at least one client, so you will have to be give us more information if we are to be of any help. Possible ways of doing that:

• telling us exactly how you are sending queries, if you are the one generating the traffic that should be blocked
• sharing a network packet capture (PCAP) showing that a given client IP is indeed exceeding one of the configured limits
• sharing up the output of grepq("") from dnsdist's console, showing that a client IP is indeed exceeding one of the configured limits
• running dnsdist in verbose mode and sharing the logs, showing that a client IP is indeed exceeding one of the configured limits