Skip to content

Commit

Permalink
Added automated upload for BIG-IQ Collector
Browse files Browse the repository at this point in the history
  • Loading branch information
fabriziofiorucci committed Mar 10, 2023
1 parent 784cebe commit 6cc275a
Show file tree
Hide file tree
Showing 2 changed files with 77 additions and 17 deletions.
62 changes: 54 additions & 8 deletions contrib/bigiq-collect/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ the .tgz file can then be processed offline by Second Sight to build all target

`sampledata.tgz` is provided for testing purposes

## Usage - Data collection
## Script installation

- Copy (scp) `bigIQCollect.sh` from your local host to your BIG-IQ CM instance, under `/tmp/`

Expand All @@ -28,17 +28,30 @@ Last login: Fri Nov 19 00:00:05 2021 from 192.168.1.18
[root@bigiq:Active:Standalone] config # /tmp/bigIQCollect.sh
Second Sight - https://github.com/F5Networks/SecondSight
This tool collects usage tracking data from BIG-IQ for offline postprocessing.
=== Usage:
./bigIQCollect.sh [options]
-h - This help
-i - Interactive mode
-u [username] - BIG-IQ username (batch mode)
-p [password] - BIG-IQ password (batch mode)
=== Options:
-h - This help
-i - Interactive mode
-u [username] - BIG-IQ username (batch mode)
-p [password] - BIG-IQ password (batch mode)
-s [http(s)://address] - Upload data to Second Sight (optional)
=== Examples:
Interactive mode: ./bigIQCollect.sh -i
Batch mode: ./bigIQCollect.sh -u [username] -p [password]
Interactive mode: ./bigIQCollect.sh -i
Interactive mode + upload: ./bigIQCollect.sh -i -s https://<SECOND_SIGHT_FQDN_OR_IP>
Batch mode: ./bigIQCollect.sh -u [username] -p [password]
Batch mode: ./bigIQCollect.sh -u [username] -p [password] -s https://<SECOND_SIGHT_FQDN_OR_IP>
```

## Usage - Data collection with manual upload

- On BIG-IQ CM run the collection script using "admin" as the authentication username and its password

```
Expand Down Expand Up @@ -80,6 +93,40 @@ $ scp [email protected]:/tmp/20220113-0005-bigIQCollect.tgz .
$
```

## Usage - Data collection with automated upload

- On BIG-IQ CM run the collection script using "admin" as the authentication username and its password, add the `-s` switch to specify Second Sight GUI URL

```
$ ssh [email protected]
([email protected]) Password:
Last login: Fri Mar 10 18:30:03 2023 from 192.168.1.18
[root@bigiq:Active:Standalone] tmp # ./bigIQCollect.sh -i -s https://secondsight.acme.lan
Username: admin
Password:
-> Reading device list
-> Reading system provisioning
-> Reading device inventory details
-> Found 68 inventories
-> Inventories summary
1 FINISHED
-> Using inventory [3a892dde-f61d-412c-a243-42fcd13b8945]
-> Reading device info for [1346a1f5-49aa-422f-8c4d-19b0119b3927]
-> Reading device info for [aabcaca7-6986-4d0b-b9fe-b72e3473144e]
-> Reading device telemetry
-> Collecting utility billing for regkey [XXXXX-XXXXX-XXXXX-XXXXX-XXXXXXX]
-> Collecting utility billing for regkey [XXXXX-XXXXX-XXXXX-XXXXX-XXXXXXX]
-> Data collection completed, building tarfile
-> Uploading /tmp/20230310-1831-bigIQCollect.tgz to Second Sight at https://secondsight.acme.lan
{"success":true,"status":"File upload completed","filename":"20230310-1831-bigIQCollect.tgz","content-type":"application/octet-stream","description":"20230310-1831-bigIQCollect.tgz","uid":"23b8fdd3-ee3c-4cf3-89ea-5396ca467915"}
-> Upload complete
[root@bigiq:Active:Standalone] config # exit
logout
Connection to bigiq.f5 closed.
$
```

## Usage - CLI-based data postprocessing

- Start Second Sight and the additional fileserver script to process the tgz file and generate the target JSON file
Expand Down Expand Up @@ -177,4 +224,3 @@ Compressed output:
```
$ curl -s -H "Accept-Encoding: gzip" http://127.0.0.1:5000/instances --output output-json.gz
```

32 changes: 23 additions & 9 deletions contrib/bigiq-collect/bigIQCollect.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,16 +11,19 @@ This tool collects usage tracking data from BIG-IQ for offline postprocessing.\n
=== Usage:\n\n
$0 [options]\n\n
=== Options:\n\n
-h\t\t- This help\n
-i\t\t- Interactive mode\n
-u [username]\t- BIG-IQ username (batch mode)\n
-p [password]\t- BIG-IQ password (batch mode)\n\n
-h\t\t\t- This help\n
-i\t\t\t- Interactive mode\n
-u [username]\t\t- BIG-IQ username (batch mode)\n
-p [password]\t\t- BIG-IQ password (batch mode)\n
-s [http(s)://address]\t- Upload data to Second Sight (optional)\n\n
=== Examples:\n\n
Interactive mode:\t$0 -i\n
Batch mode:\t\t$0 -u [username] -p [password]\n
Interactive mode:\t\t$0 -i\n
Interactive mode + upload:\t$0 -i -s https://<SECOND_SIGHT_FQDN_OR_IP>\n
Batch mode:\t\t\t$0 -u [username] -p [password]\n
Batch mode:\t\t\t$0 -u [username] -p [password] -s https://<SECOND_SIGHT_FQDN_OR_IP>\n
"

while getopts 'hiu:p:' OPTION
while getopts 'hiu:p:s:' OPTION
do
case "$OPTION" in
h)
Expand All @@ -38,6 +41,9 @@ do
p)
BIGIQ_PASSWORD=$OPTARG
;;
s)
UPLOAD_SS=$OPTARG
;;
esac
done

Expand Down Expand Up @@ -277,8 +283,16 @@ done


echo "-> Data collection completed, building tarfile"
TARFILE=$OUTPUTROOT/`date +"%Y%m%d-%H%M"`-bigIQCollect.tgz
TARFILEBASENAME=`date +"%Y%m%d-%H%M"`-bigIQCollect.tgz
TARFILE=$OUTPUTROOT/$TARFILEBASENAME
tar zcmf $TARFILE $OUTPUTDIR 2>/dev/null
rm -rf $OUTPUTDIR

echo "-> All done, copy $TARFILE to your local host using scp"
if [ "$UPLOAD_SS" = "" ]
then
echo "-> All done, copy $TARFILE to your local host using scp"
else
echo "-> Uploading $TARFILE to Second Sight at $UPLOAD_SS"
curl -X POST -sk $UPLOAD_SS/api/v1/archive -F "file=@$TARFILE" -F "description=$TARFILEBASENAME"
echo "-> Upload complete"
fi

0 comments on commit 6cc275a

Please sign in to comment.