Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added more testdata json's #188

Merged
merged 1 commit into from
Mar 2, 2024
Merged

Conversation

k0ste
Copy link
Contributor

@k0ste k0ste commented Dec 28, 2023

Added more testdata. Produced & redacted via scripts

Current prometheus query to ignore (this models testdata present)

sum by (model_name, netbox_fqdn)
(smartctl_device{model_name!~"(.*)(WUS721010ALE6L4|HDS721010CLA332|HUS724020ALA640|HUS722T2TALA604|HUS724040ALA640|HUS726060ALE614|HUS726020ALE614|HUS726T4TALE6L4|HUS726040ALA614|HUS726040ALE614|HUS728T8TALE6L4|HUS722T1TALA604|HUSMR7632BHP301|WUH721818ALN604|WUH721816ALE6L4|WUH721818ALE6L4|HUH721008ALE604|HUA722010CLA630|HUA722010CLA330|HDWE160|HDWR160|MG09ACA18TE|MG06ACA800E|MG08ADA800E|MG08ACA16TE|MG06ACA800E|MG06ACA10TE|MG05ACA800E|MG04ACA100N|MG04ACA400E|MG03ACA100|MG04ACA200E|ST1000NM0033|ST18000NM000J|ST16000NM000J|ST16000NM001G|ST10000NM0016|ST8000NM017B|ST8000NM0016|ST8000NM0055|ST8000NM000A|ST1000NM000A|ST1000NM0008|ST4000NM0035|ST32000644NS|ST2000NM0033|ST2000NM0008|ST3000NM0033|ST500NM0011|SSDPEDMD016T4|SSDPE2KX020T8|SSDPE2KX020T7|SSDPEDKE020T7|SSDPF2KE032T1|SSDPE2KE032T8|SSDPE2KX040T8|SSDSC2KB019T7|SSDSC2KB019T8|SSDPEDMD400G4|SSDPF2KX038T1|SSDSC2KB960G8|SSDSC2KB960G7|SSDSCKJB240G7R|SSDPF2KX038TZ|MTFDKCB1T6TFC|MTFDHBE3T2TDG|MTFDKCB3T8TFR|MTFDHAX1T6MCE|MTFDKCB3T8TDZ|MTFDDAK960QDE|MTFDHBE1T9TDF|MTFDKCB3T2TFC|MTFDDAV240TDS|MTFDHAL3T8TDP|MTFDKCC3T8TFR|MTFDDAK960TDS|MTFDHBE960TDF|MTFDDAK240TDS|MTFDDAK3T8TDS|MTFDKCB1T9TDZ|MTFDDAK480TDS|MTFDHBE3T8TDF|MTFDKCC3T2TFS|MTFDHBE1T6TDG|MTFDDAK1T9TDS|MTFDDAV480TDS|SSDPEK1A058GA|SSDSC2KB480G8|SSDSCKKB480G8|SSDPE2MX450G7|SSDPEDMD800G4|SSDPE2KX010T8|SSDSC2KB480G7|SSDSC2KG480G8|SSDSC2BB480G6|SSDSC2BB016T7|SSDSC2KB240G8|SSDSC2KB240G7|SSDSC2BB016T4|SSDSC2BB480G7|SSDSC2BB480G4|SSDSC2BB800G7|SSDSC2BB800G6|SSDSC2BB800G4|SSDPE2KE020T7|SSDPE2KE016T8|SSDSC2BB240G7|SSDSC2BB240G4|SSDSC2BB240G6|SSDSC2KB019TZ|SSDSC2KG240G8|MEMPEK1W032GA|MZPLJ6T4HALA|MZPLL1T6HEHP|MZ7LH1T9HMLT|MZ7LH480HAHQ|MZ7L3240HCHQ|MZPLL3T2HAJQ|MZQL21T9HCJR|MZQL2960HCJR|MZ7L3480HCHQ|MZQL23T8HCLS|MZWLL3T2HAJQ|MZPLJ3T2HBJR|MZPLL1T6HAJQ|MZPLL3T2HMLS|WD3000FYYZ|WD2000FYYZ|WD2003FYYS|WD1004FBYZ|WD1003FBYZ|WD1003FBYX|WD1002FBYS|WD5003ABYX|TS32GMTS400|SA400M8120G|SKC400S371T|CT250P2SSD8|SSD 970 EVO)(.*)"}
)

@k0ste
Copy link
Contributor Author

k0ste commented Dec 28, 2023

@robbat2 hope this helps for CI

@robbat2
Copy link
Contributor

robbat2 commented Dec 29, 2023

@k0ste that's amazing data, thanks! 139 different drive models!

I see we need some tweaks to the scripts:
1.

testdata/sat-Toshiba_3.5"_MG03ACAxxx(Y)_Enterprise_HDD-TOSHIBA_MG03ACA100-sdv.json
testdata/sat-Toshiba_3.5"_HDD_MK.002TSKB-TOSHIBA_MK1002TSKB-sdb.json

I think the filenames shouldn't contain " or ( / ); this will break some systems.

Was the model_family getting the three dots here a redaction bug? Seems to affect only Toshiba drives.
Data shows:

"model_family": "Toshiba MG04ACA... Enterprise HDD",
"model_name": "TOSHIBA MG04ACA200E",

Filenames:

testdata/sat-Toshiba_MG04ACA..._Enterprise_HDD-TOSHIBA_MG04ACA200E-sdd.json
testdata/sat-Toshiba_MG04ACA..._Enterprise_HDD-TOSHIBA_MG04ACA400E-sdg.json
testdata/sat-Toshiba_MG05ACA..._Enterprise_Capacity_HDD-TOSHIBA_MG05ACA800E-sdw.json
testdata/sat-Toshiba_MG06ACA..._Enterprise_Capacity_HDD-TOSHIBA_MG06ACA10TE-sda.json
testdata/sat-Toshiba_MG06ACA..._Enterprise_Capacity_HDD-TOSHIBA_MG06ACA800E-sda.json
testdata/sat-Toshiba_MG08ACA..._Enterprise_Capacity_HDD-TOSHIBA_MG08ACA16TE-sda.json
testdata/sat-Toshiba_MG09ACA..._Enterprise_Capacity_HDD-TOSHIBA_MG09ACA18TE-sdd.json
  1. Do we need to consider redacting the device into sdX, to avoid revealing how many drives a submitter might have?
testdata/sat-Western_Digital_Ultrastar_DC_HC550-WDC__WUH721816ALE6L4-sdx.json

Tells me you have at least one very large disk chassis.

k0ste added a commit to k0ste/smartctl_exporter that referenced this pull request Dec 30, 2023
* Remove `"`, `(` and `)` from file names (see prometheus-community#188)
* Added regex ignore for some devices, fixes:

```
Collecting data for '/dev/sda'...       Saving to sat-Intel_S4510_S4610_S4500_S4600_Series_SSDs-INTEL_SSDSC2KB240G8-sda.json
Collecting data for '/dev/sdb'...       Saving to sat-Intel_S4510_S4610_S4500_S4600_Series_SSDs-INTEL_SSDSC2KB240G8-sdb.json
Collecting data for '/dev/bus/0'...jq: error (at <stdin>:34): null (null) cannot be matched, as it is not a string
        Saving to null-null--0.json
Collecting data for '/dev/bus/0'...jq: error (at <stdin>:34): null (null) cannot be matched, as it is not a string
        Saving to null-null--0.json
Collecting data for '/dev/nvme0'...     Saving to nvme-null-HUSMR7632BHP301-nvme0.json
Collecting data for '/dev/nvme1'...     Saving to nvme-null-HUSMR7632BHP301-nvme1.json
Collecting data for '/dev/nvme2'...     Saving to nvme-null-HUSMR7632BHP301-nvme2.json
Collecting data for '/dev/nvme3'...     Saving to nvme-null-HUSMR7632BHP301-nvme3.json
Collecting data for '/dev/nvme4'...     Saving to nvme-null-HUSMR7632BHP301-nvme4.json
```

The `/dev/bus/*` is some pseudo-device comes with Broadcom driver, after fix:

```
Collecting data for '/dev/sda'...       Saving to sat-Intel_S4510_S4610_S4500_S4600_Series_SSDs-INTEL_SSDSC2KB240G8-sda.json
Collecting data for '/dev/sdb'...       Saving to sat-Intel_S4510_S4610_S4500_S4600_Series_SSDs-INTEL_SSDSC2KB240G8-sdb.json
Collecting data for '/dev/nvme0'...     Saving to nvme-null-HUSMR7632BHP301-nvme0.json
Collecting data for '/dev/nvme1'...     Saving to nvme-null-HUSMR7632BHP301-nvme1.json
Collecting data for '/dev/nvme2'...     Saving to nvme-null-HUSMR7632BHP301-nvme2.json
Collecting data for '/dev/nvme3'...     Saving to nvme-null-HUSMR7632BHP301-nvme3.json
Collecting data for '/dev/nvme4'...     Saving to nvme-null-HUSMR7632BHP301-nvme4.json
```

Signed-off-by: Konstantin Shalygin <[email protected]>
@k0ste
Copy link
Contributor Author

k0ste commented Dec 30, 2023

I think the filenames shouldn't contain " or ( / ); this will break some systems.

#189

Was the model_family getting the three dots here a redaction bug? Seems to affect only Toshiba drives.

This isn't a redaction bug. This is Toshiba. I think the SMART table have some limitations, and vendor reduced number of symbols this way

Tells me you have at least one very large disk chassis.

From 12/24/36 devices in chassis - on your choose

Signed-off-by: Konstantin Shalygin <[email protected]>
@k0ste
Copy link
Contributor Author

k0ste commented Dec 31, 2023

Rebased: removed " or ( / ) from file names by hand

@k0ste
Copy link
Contributor Author

k0ste commented Feb 28, 2024

@NiceGuyIT

@NiceGuyIT
Copy link
Member

@k0ste I'll have a look this weekend. Thanks!

@NiceGuyIT NiceGuyIT merged commit 0dc8fb3 into prometheus-community:master Mar 2, 2024
2 checks passed
NiceGuyIT added a commit that referenced this pull request May 5, 2024
collect-smartctl-json: shell fixes for script (#188)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants