From 774ec30b61c5e10918927964a8169f4cea5c86b3 Mon Sep 17 00:00:00 2001 From: Steve Lawrence Date: Mon, 29 Jan 2024 09:31:54 -0500 Subject: [PATCH] Disable cache TTL by default The current cache TTL default is 30 minutes, which is relatively short and could easily lead to hiccups if there is a lull in data, a schema is removed from the cache and Daffodil needs to recompile it when data starts flowing again. To avoid these hiccups, this changes the default to 0 seconds, which means to never remove schemas from the cache. In most cases, if a user uses a schema once they are probably going to use it again, so this avoids recompilations that could lead to unexpected latencies. --- .../nifi/processors/AbstractDaffodilProcessor.java | 4 ++-- .../additionalDetails.html | 5 ++--- .../additionalDetails.html | 5 ++--- 3 files changed, 6 insertions(+), 8 deletions(-) diff --git a/nifi-daffodil-processors/src/main/java/com/owlcyberdefense/nifi/processors/AbstractDaffodilProcessor.java b/nifi-daffodil-processors/src/main/java/com/owlcyberdefense/nifi/processors/AbstractDaffodilProcessor.java index bcc618a..fe1afdc 100644 --- a/nifi-daffodil-processors/src/main/java/com/owlcyberdefense/nifi/processors/AbstractDaffodilProcessor.java +++ b/nifi-daffodil-processors/src/main/java/com/owlcyberdefense/nifi/processors/AbstractDaffodilProcessor.java @@ -138,9 +138,9 @@ public abstract class AbstractDaffodilProcessor extends AbstractProcessor { public static final PropertyDescriptor CACHE_TTL_AFTER_LAST_ACCESS = new PropertyDescriptor.Builder() .name("cache-ttl-after-last-access") .displayName("Cache TTL After Last Access") - .description("The cache TTL (time-to-live) or how long to keep compiled DFDL schemas in the cache after last access.") + .description("Defines how long keep unused compiled DFDL schemas in the cache before removing them to free memory. Defaults to '0 seconds' which means to never remove cached schemas.") .required(true) - .defaultValue("30 mins") + .defaultValue("0 seconds") .addValidator(StandardValidators.TIME_PERIOD_VALIDATOR) .build(); diff --git a/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilParse/additionalDetails.html b/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilParse/additionalDetails.html index f79e76e..5f10172 100644 --- a/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilParse/additionalDetails.html +++ b/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilParse/additionalDetails.html @@ -139,9 +139,8 @@

Compiled DFDL Schema Cache

Cached compiled DFDL schemas that go unused for a specified amount of time are removed from the cache to save memory. This time is defind by the Cache TTL after last access property, with the format of <duration> <time_unit>, where <duration> is a non-negative integer and - <time_unit> is a supported unit of time, such as nanos, millis, secs, mins, hrs, days. If set - to zero (e.g. "0 seconds"), cached compiled DFDL schemas are never removed from the cache. The default value - is 30 minutes. + <time_unit> is a supported unit of time, such as nanos, millis, secs, mins, hrs, days. The + default is "0 seconds", which means compiled DFDL schemas are never removed from the cache.

For example, if a schema is used occasionally (once a day perhaps), then set this to 24 hours to diff --git a/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilUnparse/additionalDetails.html b/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilUnparse/additionalDetails.html index a44e0bd..52a74c4 100644 --- a/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilUnparse/additionalDetails.html +++ b/nifi-daffodil-processors/src/main/resources/docs/com.owlcyberdefense.nifi.processors.DaffodilUnparse/additionalDetails.html @@ -140,9 +140,8 @@

Compiled DFDL Schema Cache

Cached compiled DFDL schemas that go unused for a specified amount of time are removed from the cache to save memory. This time is defind by the Cache TTL after last access property, with the format of <duration> <time_unit>, where <duration> is a non-negative integer and - <time_unit> is a supported unit of time, such as nanos, millis, secs, mins, hrs, days. If set - to zero (e.g. "0 seconds"), cached compiled DFDL schemas are never removed from the cache. The default value - is 30 minutes. + <time_unit> is a supported unit of time, such as nanos, millis, secs, mins, hrs, days. The + default is "0 seconds", which means compiled DFDL schemas are never removed from the cache.

For example, if a schema is used occasionally (once a day perhaps), then set this to 24 hours to