Merge branch 'v1.25.0-RC-1' into 'master'

v1.25.0 release

See merge request automation-toolchain/f5-telemetry!538

.gitignore

@@ -9,6 +9,8 @@ _build/
 # coverage
 coverage/
 .nyc_output/
+# mutation
+reports/
 # local test directory/files
 test/functional/logs/
 test/functional/*_harness_file.json

.gitlab-ci.yml

@@ -29,6 +29,7 @@ include:
 .install_unittest_packages_cmd: &install_unittest_packages_cmd
 - nodeFullVer=$(node --version)
 - echo "Node.js version - $nodeFullVer"
+- echo "NPM origin version - $(npm --version)"
 - ver=$(echo $nodeFullVer | head -c 3)
 - if [[ "$ver" == "v4." ]]; then
 -   npm install --global [email protected]
@@ -45,6 +46,7 @@ include:
 - else
 -   npm run install-test
 - fi
+- echo "NPM updated version - $(npm --version)"
 
 .run_unittest_cmd: &run_unittest_cmd
 - npm run test-only
@@ -149,7 +151,7 @@ include:
 lint:
     extends:
         - .test_job_definition
-    image: ${ARTIFACTORY_DOCKER_HUB}/node:8
+    image: ${ARTIFACTORY_DOCKER_HUB}/node:12
     stage: lint
     script:
         - *install_unittest_packages_cmd
@@ -185,8 +187,22 @@ test_node12:
         - .run_unittest
     image: ${ARTIFACTORY_DOCKER_HUB}/node:12
 
+# Node 14 LTS (active, maintenance)
+test_node14:
+    extends:
+        - .run_unittest
+    image: ${ARTIFACTORY_DOCKER_HUB}/node:14
+
+# Node 16 LTS (active, maintenance)
+test_node16:
+    extends:
+        - .run_unittest
+    image: ${ARTIFACTORY_DOCKER_HUB}/node:16
+
 # mostly for containers, unittests only (without coverage check)
-test_node_latest:
+# disabled for now, npm has one issue with DNS (IPv6), probably
+# docker image issue. Node version 17.x, npm version 8.1.0
+.test_node_latest:
     extends:
         - .run_unittest
     image: ${ARTIFACTORY_DOCKER_HUB}/node:latest

CHANGELOG.md

@@ -1,6 +1,19 @@
 # Changelog
 Changes to this project are documented in this file. More detail and links can be found in the Telemetry Streaming [Document Revision History](https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/revision-history.html).
 
+## 1.25.0
+### Added
+- AUTOTOOL-2798: [GitHub #154](https://github.com/F5Networks/f5-telemetry-streaming/issues/154): Allow keyless (service account based) authorization for GCP Metrics and Logging consumers.
+- AUTOTOOL-2794: [GitHub #158](https://github.com/F5Networks/f5-telemetry-streaming/issues/158): Added "metricPrefix" property to DataDog consumer, which allows a custom prefix to be added to each DataDog metric
+- AUTOTOOL-2815: Added the "convertBooleansToMetrics" property to the DataDog, Statsd and OpenTelemetry_Exporter consumers, which determines whether boolean values are converted to metrics
+- AUTOTOOL-2842: [GitHub #173](https://github.com/F5Networks/f5-telemetry-streaming/issues/173): Added optional service endpoint URL property to AWS consumers. Used to overwrite the implicit endpoints based on the region.
+- AUTOTOOL-2795: [GitHub #160](https://github.com/F5Networks/f5-telemetry-streaming/issues/160): Added "customTags" property to DataDog consumer, which allows specifying custom tags to attach to DataDog metrics and logs
+- AUTOTOOL-2800: [GitHub #170](https://github.com/F5Networks/f5-telemetry-streaming/issues/170): Added "poolName" tag to pool member metrics.
+### Fixed
+### Changed
+- AUTOTOOL-2769: Update npm packages (@grpc/grpc-js from v1.3.4 to v1.4.2, @opentelemetry/exporter-collector-proto from v0.24.0 to 0.25.0, @opentelemetry/sdk-metrics-base from 0.24.1-alpha.4 to 0.26.0,  aws-sdk from v2.991.0 to v2.1018.0, eventemitter2 from v6.4.4 to v6.4.5)
+### Removed
+
 ## 1.24.0
 ### Added
 - AUTOTOOL-2752: [GitHub #159](https://github.com/F5Networks/f5-telemetry-streaming/issues/159) and [GitHub #162](https://github.com/F5Networks/f5-telemetry-streaming/issues/162): Added "region" property to DataDog consumer

SUPPORT.md

@@ -18,9 +18,9 @@ Currently supported versions:
 | Software Version | Release Type  | First Customer Ship | End of Support  |
 |------------------|---------------|---------------------|-----------------|
 | TS 1.20.1        | LTS           |  30-Jun-2021        | 30-Jun-2022     |
-| TS 1.22.0        | Feature       |  09-Aug-2021        | 09-Nov-2021     |
 | TS 1.23.0        | Feature       |  21-Sep-2021        | 21-Dec-2021     |
 | TS 1.24.0        | Feature       |  02-Nov-2021        | 02-Feb-2022     |
+| TS 1.25.0        | Feature       |  14-Dec-2021        | 14-Mar-2022     |
 
 Versions no longer supported:
 
@@ -47,5 +47,6 @@ Versions no longer supported:
 | TS 1.19.0        | Feature       |  06-Apr-2021        | 06-Jul-2021     |
 | TS 1.20.0        | Feature       |  18-May-2021        | 18-Aug-2021     |
 | TS 1.21.0        | Feature       |  28-Jun-2021        | 28-Sep-2021     |
+| TS 1.22.0        | Feature       |  09-Aug-2021        | 09-Nov-2021     |
 
 See the [Release notes](https://github.com/F5Networks/f5-telemetry-streaming/releases) and [Telemetry Streaming documentation](https://clouddocs.f5.com/products/extensions/f5-telemetry-streaming/latest/revision-history.html) for new features and issues resolved for each release.

contributing/README.md

@@ -108,7 +108,7 @@ How does the project handle a typical `POST` request?
             "trace": false,
             "format": "default"
         },
-        "schemaVersion": "1.24.0"
+        "schemaVersion": "1.25.0"
     }
 }
 ```

contributing/process_release.md

@@ -58,6 +58,8 @@
     * 1.21.0 - 15.5 MB
     * 1.22.0 - 15.6 MB
     * 1.23.0 - 17.8 MB (NOTE: inclusion of OpenTelemetry libraries)
+    * 1.24.0 - 19.2 MB
+    * 1.25.0 - 17.7 MB
   * Install build to BIG-IP, navigate to folder `/var/config/rest/iapps/f5-telemetry/` and check following:
     * Run `du -sh` and check that folder's size (shouldn't be much greater than previous versions):
       * 1.4.0 - 65 MB
@@ -80,6 +82,8 @@
       * 1.21.0 - 111 MB
       * 1.22.0 - 112 MB
       * 1.23.0 - 132 MB (NOTE: inclusion of OpenTelemetry libraries)
+      * 1.24.0 - 134 MB
+      * 1.25.0 - 130 MB
     * Check `node_modules` folder - if you see `eslint`, `mocha` or something else from [package.json](package.json) `devDependencies` section - something wrong with build process. Probably some `npm` flags are work as not expected and it MUST BE FIXED before publishing.
 * Ensure that all tests (unit tests and functional tests passed)
 * Optional: Ensure that your local tags match remote. If not, remove all and re-fetch:

docs/conf.py

@@ -79,7 +79,7 @@
 # The short X.Y version.
 version = u''
 # The full version, including alpha/beta/rc tags.
-release = u'1.24.0'
+release = u'1.25.0'
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.

docs/output-example.rst

@@ -21,6 +21,9 @@ Telemetry Streaming 1.23 introduced the following, which do not appear in earlie
 - Virtual server output: **isAvailable** and **isEnabled** (both boolean), which monitor virtual server availability, particularly for the Prometheus consumer.  
 - System output: **throughtputPerformance**, which provides throughput performance information.
 
+In Telemetry Streaming 1.24 and later, properties that require the bash endpoint are now skipped if bash is not available on the target BIG-IP: system.diskStorage, system.diskLatency, system.apmState, and all tmstats properties (used with “Splunk legacy” format).
+
+
 .. literalinclude:: ../examples/output/system_poller/output.json
     :language: json
 

docs/revision-history.rst

@@ -11,12 +11,16 @@ Document Revision History
         - Description
         - Date
 
+      * - 1.25.0
+        - Updated the documentation for Telemetry Streaming v1.25.0. This release contains the following changes: |br| * Added support for IAM roles for Google Cloud Platform (see Google :ref:`Cloud Monitoring<gcmiam>` and :ref:`Cloud Logging<gcliam>`) |br| * Added the **metricPrefix** and **customTags** properties to the DataDog consumer (see :ref:`DataDog<datadog>`) |br| * Added the **convertBooleansToMetrics** property to the DataDog, StatsD, and OpenTelemetry consumers (see :ref:`DataDog<datadog>`, :ref:`StatsD<statsd-ref>`, and :ref:`OpenTelemetry<opent>`) |br| * Added the **endpointUrl** property to AWS S3 and CloudWatch consumers (see :ref:`AWS S3<awss3-ref>` and :ref:`AWS CloudWatch<awscloud-ref>`) |br| * Removed the experimental label from the autoTag property for StatsD (see :ref:`StatsD addTags<addtags>`) |br| * Removed the experimental label from Splunk multi-metric format (see :ref:`Splunk multi-metric<multi-metric>`) |br| |br| Issues Resolved: |br| * 
+        - 12-14-21
+
       * - 1.24.0
-        - Updated the documentation for Telemetry Streaming v1.24.0. This release contains the following changes: |br| * Added support for the **region** and **service** properties for the Experimental DataDog consumer (see :ref:`DataDog<datadog>`) |br| * Added support for the **format** property for Azure Log Analytics (see :ref:`Azure Log Analytics<azure-ref>`) |br| |br| Issues Resolved: |br| * 
+        - Updated the documentation for Telemetry Streaming v1.24.0. This release contains the following changes: |br| * Added the **region** and **service** properties for the DataDog consumer (see :ref:`DataDog<datadog>`) |br| * Removed the Experimental label from the DataDog consumer (see :ref:`DataDog<datadog>`) |br| * Added support for the **format** property for Azure Log Analytics (see :ref:`Azure Log Analytics<azure-ref>`) |br| * Added support for ElasticSearch 7 (see :ref:`ElasticSearch<elasticsearch-ref>`) |br| |br| Issues Resolved: |br| * Fixed an issue where Telemetry Streaming would not collect System Poller data if **bash** was disabled on the BIG-IP device. Properties that require the bash endpoint are now skipped if bash is not available on the target BIG-IP (see :ref:`System information<sysinfo>`).
         - 11-2-21
 
       * - 1.23.0
-        - Updated the documentation for Telemetry Streaming v1.23.0. This release contains the following changes: |br| * Added a new EXPERIMENTAL Push consumer for OpenTelemetry Exporter (see :ref:`OpenTelemetry Exporter<opent>`) |br| * Added  **isAvailable** and **isEnabled** to virtual server output (see :ref:`System information output<sysinfo>`), `GitHub #152 <https://github.com/F5Networks/f5-telemetry-streaming/issues/152>`_ |br| * Added  **throughtputPerformance** to System output (see :ref:`System information output<sysinfo>`), `GitHub #129 <https://github.com/F5Networks/f5-telemetry-streaming/issues/129>`_ |br| * Added the **compressionType** property to the experimental DataDog consumer (see :ref:`DataDog<datadog>`), `GitHub #157 <https://github.com/F5Networks/f5-telemetry-streaming/issues/157>`_  |br| * Added functionality to handle responses from iControlREST that contain duplicate JSON keys |br| * Added support for TS Namespaces (was experimental), see :doc:`namespaces` |br| * Added support for specifying fallback hosts for Generic HTTP consumers (was experimental) :ref:`Fallback hosts<fallback>` |br| |br| Issues Resolved: |br| * Fixed issue where Prometheus consumer did not return the correct Content-Type HTTP Header, `GitHub #148 <https://github.com/F5Networks/f5-telemetry-streaming/issues/148>`_ |br| * Fixed issue where asmState could report incorrect state value. asmState and lastAsmChange properties are now retrieved from iControlREST, `GitHub #151 <https://github.com/F5Networks/f5-telemetry-streaming/issues/151>`_
+        - Updated the documentation for Telemetry Streaming v1.23.0. This release contains the following changes: |br| * Added a new EXPERIMENTAL Push consumer for OpenTelemetry Exporter (see :ref:`OpenTelemetry Exporter<opent>`) |br| * Added  **isAvailable** and **isEnabled** to virtual server output (see :ref:`System information output<sysinfo>`), `GitHub #152 <https://github.com/F5Networks/f5-telemetry-streaming/issues/152>`_ |br| * Added  **throughputPerformance** to System output (see :ref:`System information output<sysinfo>`), `GitHub #129 <https://github.com/F5Networks/f5-telemetry-streaming/issues/129>`_ |br| * Added the **compressionType** property to the experimental DataDog consumer (see :ref:`DataDog<datadog>`), `GitHub #157 <https://github.com/F5Networks/f5-telemetry-streaming/issues/157>`_  |br| * Added functionality to handle responses from iControlREST that contain duplicate JSON keys |br| * Added support for TS Namespaces (was experimental), see :doc:`namespaces` |br| * Added support for specifying fallback hosts for Generic HTTP consumers (was experimental) :ref:`Fallback hosts<fallback>` |br| |br| Issues Resolved: |br| * Fixed issue where Prometheus consumer did not return the correct Content-Type HTTP Header, `GitHub #148 <https://github.com/F5Networks/f5-telemetry-streaming/issues/148>`_ |br| * Fixed issue where asmState could report incorrect state value. asmState and lastAsmChange properties are now retrieved from iControlREST, `GitHub #151 <https://github.com/F5Networks/f5-telemetry-streaming/issues/151>`_
         - 9-21-21
 
       * - 1.22.0