Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Show GET/PUT, etc Count By Satellite #6

Open
gsxryan opened this issue Jul 19, 2019 · 7 comments
Open

Show GET/PUT, etc Count By Satellite #6

gsxryan opened this issue Jul 19, 2019 · 7 comments
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@gsxryan
Copy link
Owner

gsxryan commented Jul 19, 2019

Specs Organized by Sat.

GETs
Sat1: 160
sat 2: 15

@gsxryan gsxryan added enhancement New feature or request help wanted Extra attention is needed labels Jul 19, 2019
@gsxryan
Copy link
Owner Author

gsxryan commented Jul 20, 2019

from robertstansfield:

`
#!/bin/bash
audits=$(mktemp)
sats=$(mktemp)

docker logs storagenode 2>&1 | grep GET_AUDIT > $audits

cat $audits | cut -f5 | cut -f5 -d " " | cut -f2 -d """ > ${sats}
known_sats=$(sort ${sats} | uniq)

for sat in $known_sats; do
echo -e "Sat ID:\t\t\t\t$sat"
echo -e "Unrecoverable Failed Audits:\t$(cat "$audits" | grep failed | grep open | grep -c "$sat")"
echo -e "Recoverable Failed Audits:\t$(cat "$audits" | grep failed | grep -v open | grep -c "$sat")"
echo -e "Successful Audits:\t\t$(cat "$audits" | grep downloaded | grep -c "$sat")"
done

rm $sats
rm $audits
`

@juergschwarz
Copy link

cut -f2 -d """ > ${sats}

should read as

cut -f2 -d " " > ${sats}

@AndreiG6
Copy link
Contributor

Idk where yall get these bash scripts from but they make anyone who knows how to bash cringe :)
here, use this script instead:

#!/bin/bash
audits=$(mktemp)

docker logs storagenode 2>&1 | grep GET_AUDIT > $audits
known_sats=($(grep -oP '(?<=Satellite ID": ")\w+' $audits|sort -u))
for sat in "${known_sats[@]}";do
echo -e "Sat ID:\t\t\t\t$sat
Unrecoverable Failed Audits:\t$(grep -c "failed.*${sat}.*open" $audits)
Recoverable Failed Audits:\t$(grep -Pc "failed.*${sat}(?!.*open)" $audits)
Successful Audits:\t\t$(grep -c "downloaded.*${sat}" $audits)"
done

rm -f $audits

Sample output:

# bash audits.sh
Sat ID:				118UWpMCHzs6CvSgWd9BfFVjw5K9pZbJjkfZJexMtSkmKxvvAW
Unrecoverable Failed Audits:	0
Recoverable Failed Audits:	0
Successful Audits:		2

@gsxryan
Copy link
Owner Author

gsxryan commented Nov 21, 2019

Haha. I don't work with bash much. I do migrate quite a bit of bash to powershell. I appreciate the input and will learn some from your pull.

@AndreiG6
Copy link
Contributor

AndreiG6 commented Nov 21, 2019

regex goes a long way :)
now I just need to find that page on storj again where they suggested madness like grep GET|grep|stuff|awk.. I stumbled on it by accident when reviewing the log entries for that pull (#23)

@gsxryan
Copy link
Owner Author

gsxryan commented Nov 24, 2019

It was sourced from Alexey on Storj's RocketChat. And we just built on it from here. My goal was a working dashboard. Glad that someone is looking into performance and optimization now. I usually only use Regex when absolutely required, because it doesn't come natural for me.

@AndreiG6
Copy link
Contributor

Glad to help, optimizing is my thing :) I'll have to keep digging for Alexey's post and suggest some corrections

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants