merge rc1280-mb-5 into master branch

F5Networks · Apr 18, 2022 · f10f0e5 · f10f0e5
1 parent be6441c
commit f10f0e5
Show file tree

Hide file tree

Showing 49 changed files with 6,768 additions and 119 deletions.
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
@@ -194,10 +194,11 @@ test_node14:
     image: ${ARTIFACTORY_DOCKER_HUB}/node:14
 
 # Node 16 LTS (active, maintenance)
+# Pinned to 16.14.0, as later versions include versions of npm that are incompatible with our package-lock.json file version
 test_node16:
     extends:
         - .run_unittest
-    image: ${ARTIFACTORY_DOCKER_HUB}/node:16
+    image: ${ARTIFACTORY_DOCKER_HUB}/node:16.14.0
 
 # mostly for containers, unittests only (without coverage check)
 # disabled for now, npm has one issue with DNS (IPv6), probably

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,6 +1,22 @@
 # Changelog
 Changes to this project are documented in this file. More detail and links can be found in the Telemetry Streaming [Document Revision History](https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/revision-history.html).
 
+## 1.28.0
+### Added
+### Fixed
+- AUTOTOOL-3101: [GitHub #191](https://github.com/F5Networks/f5-telemetry-streaming/issues/191): Increased number of messages in a single PutLogEvents command for AWS Logs consumer and added retry logic for InvalidSequenceTokenException.
+### Changed
+- AUTOTOOL-2975: Update npm packages (@grpc/proto-loader from 0.6.4 to 0.6.9). Removed unneeded "long" dependency
+- AUTOTOOL-3122: Qkview should contain Declaration History file
+### Removed
+
+## 1.27.1
+### Added
+### Fixed
+### Changed
+- Promoted to LTS
+### Removed
+
 ## 1.27.0
 ### Added
 - AUTOTOOL-2925: Save recent declarations to a file

diff --git a/SUPPORT.md b/SUPPORT.md
@@ -18,9 +18,10 @@ Currently supported versions:
 | Software Version | Release Type  | First Customer Ship | End of Support  |
 |------------------|---------------|---------------------|-----------------|
 | TS 1.20.1        | LTS           |  30-Jun-2021        | 30-Jun-2022     |
-| TS 1.25.0        | Feature       |  14-Dec-2021        | 14-Mar-2022     |
 | TS 1.26.0        | Feature       |  25-Jan-2022        | 25-Apr-2022     |
-| TS 1.27.0        | Feature       |  09-Mar-2022        | 09-Jun-2022     |
+| TS 1.27.0        | Feature       |  08-Mar-2022        | 08-Jun-2022     |
+| TS 1.27.1        | LTS           |  19-Apr-2022        | 19-Apr-2023     |
+| TS 1.28.0        | Feature       |  19-Apr-2022        | 19-Jul-2022     |
 
 Versions no longer supported:
 
@@ -50,5 +51,6 @@ Versions no longer supported:
 | TS 1.22.0        | Feature       |  09-Aug-2021        | 09-Nov-2021     |
 | TS 1.23.0        | Feature       |  21-Sep-2021        | 21-Dec-2021     |
 | TS 1.24.0        | Feature       |  02-Nov-2021        | 02-Feb-2022     |
+| TS 1.25.0        | Feature       |  14-Dec-2021        | 14-Mar-2022     |
 
 See the [Release notes](https://github.com/F5Networks/f5-telemetry-streaming/releases) and [Telemetry Streaming documentation](https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/revision-history.html) for new features and issues resolved for each release.
diff --git a/contributing/README.md b/contributing/README.md
@@ -108,7 +108,7 @@ How does the project handle a typical `POST` request?
             "trace": false,
             "format": "default"
         },
-        "schemaVersion": "1.27.0"
+        "schemaVersion": "1.28.0"
     }
 }
 ```

diff --git a/contributing/process_release.md b/contributing/process_release.md
@@ -63,6 +63,7 @@
     * 1.25.0 - 17.7 MB
     * 1.26.0 - 17.8 MB
     * 1.27.0 - 17.7 MB
+    * 1.28.0 - 17.7 MB
   * Install build to BIG-IP, navigate to folder `/var/config/rest/iapps/f5-telemetry/` and check following:
     * Run `du -sh` and check that folder's size (shouldn't be much greater than previous versions):
       * 1.4.0 - 65 MB
@@ -89,6 +90,7 @@
       * 1.25.0 - 130 MB
       * 1.26.0 - 127 MB
       * 1.27.0 - 127 MB
+      * 1.28.0 - 127 MB
     * Check `node_modules` folder - if you see `eslint`, `mocha` or something else from [package.json](package.json) `devDependencies` section - something wrong with build process. Probably some `npm` flags are work as not expected and it MUST BE FIXED before publishing.
 * Ensure that all tests (unit tests and functional tests passed)
 * Optional: Ensure that your local tags match remote. If not, remove all and re-fetch:
@@ -169,4 +171,4 @@ Following list of changes should be done for both master/develop and LTS branche
     * `.rpm` file
     * `.sha256` file
 
-# ATTENTION: DO NOT FORGET TO MERGE 'MASTER' BRANCH INTO 'DEVELOP' WHEN YOU ARE DONE WITH RELEASE PROCESS
+# ATTENTION: DO NOT FORGET TO MERGE 'MASTER' BRANCH INTO 'DEVELOP' WHEN YOU ARE DONE WITH RELEASE PROCESS
diff --git a/docs/conf.py b/docs/conf.py
@@ -69,7 +69,7 @@
 
 # General information about the project.
 project = u'Telemetry Streaming'
-copyright = u'2021 F5 Networks Inc'
+copyright = u'2022 F5 Networks Inc'
 author = u'F5 Networks'
 
 # The version info for the project you're documenting, acts as replacement for
@@ -79,7 +79,7 @@
 # The short X.Y version.
 version = u''
 # The full version, including alpha/beta/rc tags.
-release = u'1.27.0'
+release = u'1.28.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

diff --git a/docs/index.rst b/docs/index.rst
@@ -130,10 +130,10 @@ You can click the following links to go directly to a specific :doc:`setting-up-
    data-modification
    namespaces
    troubleshooting
+   revision-history
    schema-reference
 
    poller-default-output-reference
-   revision-history
 
 
 

diff --git a/docs/output-example.rst b/docs/output-example.rst
@@ -38,7 +38,7 @@ The following properties were introduced in recent Telemetry Streaming versions.
 
       * - 1.23
         - System
-        - **throughtputPerformance**, which shows throughput performance information. 
+        - **throughputPerformance**, which shows throughput performance information. 
 
       * - 1.25
         - Pool Member
@@ -47,6 +47,10 @@ The following properties were introduced in recent Telemetry Streaming versions.
       * - 1.26
         - Pool Member
         - **fqdn**, which shows FQDN information for LTM pool member nodes that are FQDN nodes.
+
+      * - 1.27
+        - System
+        - **asmAttackSignatures**, which shows ASM attack signature information.
 
 
 

diff --git a/docs/revision-history.rst b/docs/revision-history.rst
@@ -11,6 +11,10 @@ Document Revision History
         - Description
         - Date
 
+      * - 1.27.0
+        - Updated the documentation for Telemetry Streaming v1.27.0. This release contains the following changes: |br| * Telemetry Streaming now stores up to 30 of the most recent declarations (see :ref:`Stored Declarations<save>`) |br| * Verified support for ElasticSearch 8 (see :ref:`ElasticSearch<elasticsearch-ref>`) |br| * Removed the experimental label from the OpenTelemetry Exporter consumer (see :ref:`OpenTelemetry<opent>`) |br| * Updated the DataDog consumer to use the v2 logs API |br| * TS now closes the tracer by timeout when inactive |br| |br| Issues Resolved: |br| * Added additional sanitation to StatsD metric and tag naming, `GitHub #184 <https://github.com/F5Networks/f5-telemetry-streaming/issues/184>`_ |br| * RegEx catastrophic backtracking on attempt to mask secrets in JSON data |br| * Added info about installed ASM attack signatures to output (see :ref:`System Information<sysinfo>`), `GitHub #174 <https://github.com/F5Networks/f5-telemetry-streaming/issues/174>`_)
+        - 3-8-22
+
       * - 1.26.0
         - Updated the documentation for Telemetry Streaming v1.26.0. This release contains the following changes: |br| * Added the **fqdn** property to the output for pool members that use FQDNs (see :ref:`System Information<sysinfo>`) |br| * Added two new sections to :doc:`troubleshooting` for logging and tracing 
         - 1-25-22

diff --git a/docs/setting-up-consumer.rst b/docs/setting-up-consumer.rst
@@ -477,7 +477,7 @@ ElasticSearch
 -------------
 |ElasticSearch|
 
-.. NOTE:: TS 1.24 adds support for sending data to ElasticSearch 7. 
+.. NOTE:: TS 1.24 added support for sending data to ElasticSearch 7 and ElasticSearch 8.
 
 Required Information:
  - Host: The address of the ElasticSearch system.
@@ -915,19 +915,17 @@ Example Declaration:
 
 .. _opent:
 
-OpenTelemetry Exporter (EXPERIMENTAL)
--------------------------------------
+OpenTelemetry Exporter
+----------------------
 |opentelemetry|
 
 .. sidebar:: :fonticon:`fa fa-info-circle fa-lg` Version Notice:
 
-    The OpenTelemetry Exporter consumer was introduced as an EXPERIMENTAL consumer in TS 1.23. 
+    The OpenTelemetry Exporter consumer was introduced as an EXPERIMENTAL consumer in TS 1.23, and is supported in DO 1.28 and later.
 
 The OpenTelemetry Exporter Consumer exports telemetry data to an OpenTelemetry Collector, or OpenTelemetry Protocol compatible API.
 
 
-.. IMPORTANT:: Be aware of the following before deploying this consumer: |br| * The OpenTelemetry Exporter consumer is an experimental feature and will change in future releases based on feedback. |br| * There is a possibility this consumer might crash Telemetry Streaming, and send incorrectly formatted data or incomplete metrics. |br| * Some metrics might lack tags or context (such as iRule events) that will be addressed in future updates.
-
 Required Information:
  - Host: The address of the OpenTelemetry Collector, or OpenTelemetry Protocol compatible API
  - Port: The port of the OpenTelemetry Collector, or OpenTelemetry Protocol compatible API

diff --git a/docs/troubleshooting.rst b/docs/troubleshooting.rst
@@ -97,12 +97,29 @@ The following is an example that enables tracing on the Telemetry_System and Tel
         }
     }
 
+
 |
 
 
 Specific troubleshooting entries
 --------------------------------
 
+.. _save:
+
+I need to access declarations I previously sent to Telemetry Streaming
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+As of Telemetry Streaming 1.27, TS stores up to 30 recent declarations at **/shared/tmp/telemetry/declarationHistory**. 
+
+This includes declarations submitted upon following events:
+
+- TS start
+- Declarations submitted to **/shared/telemetry/declare**
+- Declarations submitted to **/shared/telemetry/namespace/<namespace>/declare**
+
+.. NOTE:: The data contains information about the submitted declaration and the processed declaration (contains expanded references, default values and so on)
+
+|
+
 I'm receiving a path not registered error when I try to post a declaration
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  
 

diff --git a/examples/declarations/aws_cloudwatch_logs.json b/examples/declarations/aws_cloudwatch_logs.json
@@ -6,6 +6,7 @@
         "region": "us-west-1",
         "logGroup": "f5telemetry",
         "logStream": "default",
+        "maxAwsLogBatchSize" : 100,
         "username": "accesskey",
         "passphrase": {
             "cipherText": "secretkey"