Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fio: fix device path issue for vpmem device type #2674

Merged
merged 1 commit into from
Aug 28, 2023

Conversation

abdhaleegit
Copy link
Collaborator

The last changes has error in getting path of the device in case if the device type is nvdimm as the fio test do not expect disk input instead it creates its own vpmem device so this changes make sure test do not fail if disk input is empty

Copy link
Member

@AYUSHJAIN951 AYUSHJAIN951 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide run logs for failure and pass case please

@abdhaleegit
Copy link
Collaborator Author

abdhaleegit commented Aug 23, 2023

vpmem device test

avocado run --max-parallel-tasks=1 fiotest.py --mux-yaml fiotest.py.data/fio-pmem.yaml 
JOB ID     : 4240ebb8067b6b5534e3e61916b67f6cd7cd95fa
JOB LOG    : /root/avocado-fvt-wrapper/results/job-2023-08-23T13.27-4240ebb/job.log
 (1/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-xfs-fefe: STARTED
 (1/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-xfs-fefe: WARN: Test passed but there were warnings during execution. Check the log for details. (61.86 s)
 (2/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-ext4-54b7: STARTED
 (2/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-ext4-54b7: WARN: Test passed but there were warnings during execution. Check the log for details. (39.36 s)
 (3/3) fiotest.py:FioTest.test;run-setup-disk-run_type-devdax-ab53: STARTED
 (3/3) fiotest.py:FioTest.test;run-setup-disk-run_type-devdax-ab53: PASS (80.15 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 2 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /root/avocado-fvt-wrapper/results/job-2023-08-23T13.27-4240ebb/results.html
JOB TIME   : 187.24 s

no pmem and with no disk in yaml

 avocado run --max-parallel-tasks=1 fiotest.py --mux-yaml fiotest.py.data/fio-pmem.yaml
JOB ID     : 31382f5d85a7830f647400b03b1d51dfc8a872b8
JOB LOG    : /home/OpTest/avocado-fvt-wrapper/results/job-2023-08-23T14.10-31382f5/job.log
 (1/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-xfs-fefe: STARTED
 (1/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-xfs-fefe: CANCEL: There are no pmem devices to test (3.76 s)
 (2/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-ext4-54b7: STARTED
 (2/3) fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-ext4-54b7: CANCEL: There are no pmem devices to test (3.35 s)
 (3/3) fiotest.py:FioTest.test;run-setup-disk-run_type-devdax-ab53: STARTED
 (3/3) fiotest.py:FioTest.test;run-setup-disk-run_type-devdax-ab53: CANCEL: There are no pmem devices to test (3.62 s)
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 3
JOB HTML   : /home/OpTest/avocado-fvt-wrapper/results/job-2023-08-23T14.10-31382f5/results.html
JOB TIME   : 16.95 s

if disk is given as input to yaml

 avocado run --max-parallel-tasks=1 fiotest.py --mux-yaml fiotest.py.data/fio.yaml 
JOB ID     : d1e0b1c00e8aeea8239834865fc92746f1f03333
JOB LOG    : /home/OpTest/avocado-fvt-wrapper/results/job-2023-08-23T14.22-d1e0b1c/job.log
 (01/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-lv-raid-raid-0c7a: STARTED
 (01/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-lv-raid-raid-0c7a: PASS (66.66 s)
 (02/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-lv-raid-no_raid-b03f: STARTED
 (02/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-lv-raid-no_raid-b03f: PASS (55.23 s)
 (03/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-no_lv-raid-raid-4ff4: STARTED
 (03/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-no_lv-raid-raid-4ff4: PASS (51.29 s)
 (04/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-no_lv-raid-no_raid-bce5: STARTED
 (04/16) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-no_lv-raid-no_raid-bce5: PASS (49.61 s)
 (05/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-lv-raid-raid-953e: STARTED
 (05/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-lv-raid-raid-953e: PASS (62.27 s)
 (06/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-lv-raid-no_raid-6f73: STARTED
 (06/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-lv-raid-no_raid-6f73: PASS (56.40 s)
 (07/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-no_lv-raid-raid-baab: STARTED
 (07/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-no_lv-raid-raid-baab: PASS (50.39 s)
 (08/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-no_lv-raid-no_raid-cb7f: STARTED
 (08/16) fiotest.py:FioTest.test;run-dir-fs-xfs-lv-no_lv-raid-no_raid-cb7f: PASS (47.38 s)
 (09/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-lv-raid-raid-6eec: STARTED
 (09/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-lv-raid-raid-6eec: CANCEL: btrfs is not supported with                                 RHEL 7.4 onwards (0.01 s)
 (10/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-lv-raid-no_raid-81d7: STARTED
 (10/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-lv-raid-no_raid-81d7: CANCEL: btrfs is not supported with                                 RHEL 7.4 onwards (0.01 s)
 (11/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-no_lv-raid-raid-9c47: STARTED
 (11/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-no_lv-raid-raid-9c47: CANCEL: btrfs is not supported with                                 RHEL 7.4 onwards (0.01 s)
 (12/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-no_lv-raid-no_raid-8163: STARTED
 (12/16) fiotest.py:FioTest.test;run-dir-fs-btrfs-lv-no_lv-raid-no_raid-8163: CANCEL: btrfs is not supported with                                 RHEL 7.4 onwards (0.01 s)
 (13/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-lv-raid-raid-6630: STARTED
 (13/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-lv-raid-raid-6630: WARN: Test passed but there were warnings during execution. Check the log for details. (51.45 s)
 (14/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-lv-raid-no_raid-70d9: STARTED
 (14/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-lv-raid-no_raid-70d9: WARN: Test passed but there were warnings during execution. Check the log for details. (49.54 s)
 (15/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-no_lv-raid-raid-9101: STARTED
 (15/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-no_lv-raid-raid-9101: WARN: Test passed but there were warnings during execution. Check the log for details. (41.61 s)
 (16/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-no_lv-raid-no_raid-1dd4: STARTED
 (16/16) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-no_lv-raid-no_raid-1dd4: WARN: Test passed but there were warnings during execution. Check the log for details. (47.88 s)
RESULTS    : PASS 8 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 4 | INTERRUPT 0 | CANCEL 4
JOB HTML   : /home/OpTest/avocado-fvt-wrapper/results/job-2023-08-23T14.22-d1e0b1c/results.html
JOB TIME   : 672.73 s

@abdhaleegit
Copy link
Collaborator Author

@AYUSHJAIN951 Please review

@abdhaleegit
Copy link
Collaborator Author

failure case

07:06:43 INFO    : Running Host Tests Suite nvdimm_fiotest_fio-pmem
07:06:43 INFO    : Running: /usr/local/bin/avocado run --max-parallel-tasks=1 /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py -m /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py.data/fio-pmem.yaml --force-job-id 839ddbac7a328b3a37c2535c97ef1909a8495e11  --job-results-dir /home/geet/avocado-fvt-wrapper/results
JOB ID     : 839ddbac7a328b3a37c2535c97ef1909a8495e11
JOB LOG    : /home/geet/avocado-fvt-wrapper/results/job-2023-07-01T07.06-839ddba/job.log
 (1/3) /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-xfs-fefe: STARTED
 (1/3) /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-xfs-fefe:  ERROR: 'FioTest' object has no attribute 'fio_file' (0.01 s)
 (2/3) /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-ext4-54b7: STARTED
 (2/3) /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py:FioTest.test;run-setup-disk-run_type-libpmem-filesystem-ext4-54b7:  ERROR: 'FioTest' object has no attribute 'fio_file' (0.01 s)
 (3/3) /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py:FioTest.test;run-setup-disk-run_type-devdax-ab53: STARTED
 (3/3) /home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py:FioTest.test;run-setup-disk-run_type-devdax-ab53:  ERROR: 'FioTest' object has no attribute 'fio_file' (0.01 s)
RESULTS    : PASS 0 | ERROR 3 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /home/geet/avocado-fvt-wrapper/results/job-2023-07-01T07.06-839ddba/results.html
JOB TIME   : 9.87 s
[stdlog] 2023-07-01 07:06:46,051 avocado.test ERROR| Traceback (most recent call last):
[stdlog] 2023-07-01 07:06:46,051 avocado.test ERROR|   File "/home/geet/avocado-fvt-wrapper/tests/avocado-misc-tests/io/disk/fiotest.py", line 62, in setUp
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|     self.disk = disk.get_absolute_disk_path(device)
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|   File "/usr/local/lib/python3.6/site-packages/avocado_framework-102.0-py3.6.egg/avocado/utils/disk.py", line 168, in get_absolute_disk_path
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|     if not os.path.exists(device):
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|   File "/usr/lib64/python3.6/genericpath.py", line 19, in exists
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|     os.stat(path)
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR| TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|
[stdlog] 2023-07-01 07:06:46,052 avocado.test ERROR|

io/disk/fiotest.py Outdated Show resolved Hide resolved
@Naresh-ibm
Copy link
Collaborator

@AYUSHJAIN951 do you have any comments on this?

@abdhaleegit
Copy link
Collaborator Author

with this #2695 added a todo to move pmem code out of fio, this can be targeted as next todo on fio

io/disk/fiotest.py Outdated Show resolved Hide resolved
The last changes has error in getting path of the device
in case if the device type is nvdimm as the fio test do
not expect disk input instead it creates its own vpmem
device so this changes make sure test do not fail if
disk input is empty

Also cancel test if either disk not given or if there is
no vpmem device exists in lpar

Signed-off-by: Abdul Haleem <[email protected]>
@Naresh-ibm
Copy link
Collaborator

@AYUSHJAIN951 please ACK.

@abdhaleegit
Copy link
Collaborator Author

with normal disk

avocado run --max-parallel-tasks=1 fiotest.py --mux-yaml fiotest.py.data/fio.yaml 
JOB ID     : b119977467e31eca60f4d8f19422f031fe172ead
JOB LOG    : /home/OpTest/avocado-fvt-wrapper/results/job-2023-08-28T12.32-b119977/job.log
 (1/2) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-lv-raid-raid-0c7a: STARTED
 (1/2) fiotest.py:FioTest.test;run-dir-fs-ext4-lv-lv-raid-raid-0c7a: PASS (75.96 s)
 (2/2) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-lv-raid-raid-6630: STARTED
 (2/2) fiotest.py:FioTest.test;run-dir-fs-no_fs-lv-lv-raid-raid-6630: WARN: Test passed but there were warnings during execution. Check the log for details. (66.30 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 1 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /home/OpTest/avocado-fvt-wrapper/results/job-2023-08-28T12.32-b119977/results.html
JOB TIME   : 147.22 s

with vpmem device

avocado run --max-parallel-tasks=1 fiotest.py --mux-yaml fiotest.py.data/f
JOB ID     : 4ba1d3179b7bf4364cdd3ec6ae5ee1d0bc0cf62d
JOB LOG    : /root/avocado-fvt-wrapper/results/job-2023-08-28T12.33-4ba1d31/job.log
 (1/3) fiotest.py:FioTest.test;run-setup-run_type-libpmem-filesystem-xfs-1190: STARTED
 (1/3) fiotest.py:FioTest.test;run-setup-run_type-libpmem-filesystem-xfs-1190: WARN: Test passed but there were warnings during execution. Check the log for details. (65.75 s)
 (2/3) fiotest.py:FioTest.test;run-setup-run_type-libpmem-filesystem-ext4-13e9: STARTED
 (2/3) fiotest.py:FioTest.test;run-setup-run_type-libpmem-filesystem-ext4-13e9: WARN: Test passed but there were warnings during execution. Check the log for details. (39.57 s)
 (3/3) fiotest.py:FioTest.test;run-setup-run_type-devdax-99cf: STARTED
 (3/3) fiotest.py:FioTest.test;run-setup-run_type-devdax-99cf: PASS (80.32 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 2 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /root/avocado-fvt-wrapper/results/job-2023-08-28T12.33-4ba1d31/results.html
JOB TIME   : 193.51 s

else part

avocado run --max-parallel-tasks=1  fiotest.py --muxio.yaml otest.py.data/fi
JOB ID     : d0592cb8bedac4c32effdfb27c5a47b319380eca
JOB LOG    : /home/ioci/logs/io_network/results/job-2023-08-28T02.04-d0592cb/job.log
 (01/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-lv-raid-raid-5880: STARTED
 (01/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-lv-raid-raid-5880: CANCEL: Please Provide valid disk (0.01 s)
 (02/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-lv-raid-no_raid-0d9a: STARTED
 (02/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-lv-raid-no_raid-0d9a: CANCEL: Please Provide valid disk (0.01 s)
 (03/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-no_lv-raid-raid-f626: STARTED
 (03/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-no_lv-raid-raid-f626: CANCEL: Please Provide valid disk (0.01 s)
 (04/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-no_lv-raid-no_raid-1f29: STARTED
 (04/16) fiotest.py:FioTest.test;run-dir-disk-fs-ext4-lv-no_lv-raid-no_raid-1f29: CANCEL: Please Provide valid disk (0.01 s)
 (05/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-lv-raid-raid-fc6a: STARTED
 (05/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-lv-raid-raid-fc6a: CANCEL: Please Provide valid disk (0.01 s)
 (06/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-lv-raid-no_raid-fbce: STARTED
 (06/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-lv-raid-no_raid-fbce: CANCEL: Please Provide valid disk (0.01 s)
 (07/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-no_lv-raid-raid-2829: STARTED
 (07/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-no_lv-raid-raid-2829: CANCEL: Please Provide valid disk (0.01 s)
 (08/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-no_lv-raid-no_raid-e836: STARTED
 (08/16) fiotest.py:FioTest.test;run-dir-disk-fs-xfs-lv-no_lv-raid-no_raid-e836: CANCEL: Please Provide valid disk (0.01 s)
 (09/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-lv-raid-raid-1b4f: STARTED
 (09/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-lv-raid-raid-1b4f:CANCEL: Please Provide valid disk (0.01 s)
 (10/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-lv-raid-no_raid-2302: STARTED
 (10/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-lv-raid-no_raid-2302: CANCEL: Please Provide valid disk (0.01 s)
 (11/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-no_lv-raid-raid-69bb: STARTED
 (11/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-no_lv-raid-raid-69bb: CANCEL: Please Provide valid disk (0.01 s)
 (12/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-no_lv-raid-no_raid-1f01: STARTED
 (12/16) fiotest.py:FioTest.test;run-dir-disk-fs-btrfs-lv-no_lv-raid-no_raid-1f01: CANCEL: Please Provide valid disk (0.01 s)
 (13/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-lv-raid-raid-96fa: STARTED
 (13/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-lv-raid-raid-96fa:CANCEL: Please Provide valid disk (0.01 s)
 (14/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-lv-raid-no_raid-3607: STARTED
 (14/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-lv-raid-no_raid-3607: CANCEL: Please Provide valid disk (0.01 s)
 (15/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-no_lv-raid-raid-44fa: STARTED
 (15/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-no_lv-raid-raid-44fa: CANCEL: Please Provide valid disk (0.01 s)
 (16/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-no_lv-raid-no_raid-c5d3: STARTED
 (16/16) fiotest.py:FioTest.test;run-dir-disk-fs-no_fs-lv-no_lv-raid-no_raid-c5d3: CANCEL: Please Provide valid disk (0.01 s)
RESULTS    : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 16
JOB HTML   : /home/ioci/logs/io_network/results/job-2023-08-28T02.04-d0592cb/results.html
JOB TIME   : 23.29 s

Copy link
Collaborator

@Naresh-ibm Naresh-ibm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Naresh-ibm Naresh-ibm merged commit 365f882 into avocado-framework-tests:master Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants