Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to scrub from last scrubbed txg #16301

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

oshogbo
Copy link
Contributor

@oshogbo oshogbo commented Jun 26, 2024

Motivation and Context

Sometimes it is useful to only scrub a particular range of txg. Some users might want to scrub only new data because they would like to know if the new write wasn't corrupted. This PR adds possiblity scrub only newly flushed data.

Description

Users who would like to scrub only new data, a new zpool property was introduced, that stores last scrubbed txg, so users can run scrub only from the last saved point.
We use a scn_max_txg and scn_min_txg which are already built into scrub, to accomplish that.

How Has This Been Tested?

Via test cases.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

@oshogbo oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 68edb6e to 42d967f Compare June 26, 2024 15:29
man/man7/zpoolprops.7 Show resolved Hide resolved
man/man7/zpoolprops.7 Outdated Show resolved Hide resolved
man/man7/zpoolprops.7 Outdated Show resolved Hide resolved
man/man8/zpool-scrub.8 Show resolved Hide resolved
module/zfs/spa.c Show resolved Hide resolved
module/zfs/dsl_scan.c Show resolved Hide resolved
@oshogbo oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 42d967f to 8d59c2c Compare August 16, 2024 11:22
@behlendorf behlendorf added the Status: Code Review Needed Ready for review and testing label Aug 16, 2024
@behlendorf behlendorf self-requested a review August 16, 2024 23:53
@tonyhutter
Copy link
Contributor

I noticed a cannot scrub tank: delegated administration is disabled on pool error when trying to continue (-C) on a paused scrub:

$ sudo ./zpool scrub tank && sleep 1 && sudo ./zpool scrub -p tank
$ sudo ./zpool status
  pool: tank
 state: ONLINE
  scan: scrub paused since Tue Aug 27 16:02:26 2024
	scrub started on Tue Aug 27 16:02:25 2024
	14.7G / 14.7G scanned, 3.22G / 14.7G issued
	0B repaired, 21.95% done
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  vdb       ONLINE       0     0     0

errors: No known data errors

$ sudo ./zpool scrub -C tank
cannot scrub tank: delegated administration is disabled on pool

$ echo $?
1

$ sudo ./zpool status
  pool: tank
 state: ONLINE
  scan: scrub repaired 0B in 00:00:14 with 0 errors on Tue Aug 27 16:02:39 2024
config:

	NAME        STATE     READ WRITE CKSUM
	tank        ONLINE       0     0     0
	  vdb       ONLINE       0     0     0

errors: No known data errors

It returns an error code, but then the scrub seems to complete successfully. Any thoughts?

@oshogbo
Copy link
Contributor Author

oshogbo commented Aug 29, 2024

@tonyhutter This is after the applied patch, but without it, it works fine?

@oshogbo oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from 8d59c2c to cc72197 Compare August 29, 2024 10:07
@oshogbo
Copy link
Contributor Author

oshogbo commented Aug 29, 2024

I have added another feature just to scrub "recent data".

@oshogbo oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch from cc72197 to 4823cc2 Compare August 29, 2024 10:22
cmd/zpool/zpool_main.c Show resolved Hide resolved
cmd/zpool/zpool_main.c Outdated Show resolved Hide resolved
cmd/zpool/zpool_main.c Outdated Show resolved Hide resolved
man/man8/zpool-scrub.8 Outdated Show resolved Hide resolved
module/zfs/dsl_scan.c Outdated Show resolved Hide resolved
@oshogbo oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch 2 times, most recently from 2a34659 to d9c7d38 Compare October 16, 2024 14:39
@amotin
Copy link
Member

amotin commented Oct 31, 2024

@oshogbo Would you rebase and resolve the test failures?

@oshogbo
Copy link
Contributor Author

oshogbo commented Oct 31, 2024

@amotin Yes, let my try to fix that.

cmd/zpool/zpool_main.c Show resolved Hide resolved
module/zfs/dsl_scan.c Outdated Show resolved Hide resolved
@oshogbo oshogbo force-pushed the zfs_scrub_last_scrubbed_txg branch 3 times, most recently from e5635df to 77f2265 Compare November 14, 2024 21:17
Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I have no objections for this variant.

Copy link
Contributor

@allanjude allanjude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed-by: Allan Jude <[email protected]>

Copy link
Contributor

@behlendorf behlendorf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small comments but nothing major. This seems like a good first step for an interface and something we can build on.

man/man7/zpoolprops.7 Outdated Show resolved Hide resolved
man/man7/zpoolprops.7 Outdated Show resolved Hide resolved
man/man7/zpoolprops.7 Outdated Show resolved Hide resolved
include/sys/dmu.h Show resolved Hide resolved
@oshogbo
Copy link
Contributor Author

oshogbo commented Nov 18, 2024

@behlendorf @allanjude @amotin Thank you for the review.

@behlendorf behlendorf added Status: Accepted Ready to integrate (reviewed, tested) and removed Status: Code Review Needed Ready for review and testing labels Nov 19, 2024
@behlendorf
Copy link
Contributor

@oshogbo it looks like this change unexpectedly breaks ztest. I've resubmitted the ztest CI build multiple times now and haven't been able to get a clean pass. Can you take a look at the CI logs from the failures:

https://github.com/openzfs/zfs/actions/runs/11891031862?pr=16301

@github-actions github-actions bot removed the Status: Accepted Ready to integrate (reviewed, tested) label Nov 24, 2024
Sponsored-By: Wasabi Technology, Inc.
Sponsored-By: Klara Inc.
Signed-off-by: Mariusz Zaborski <[email protected]>
The `last_scrubbed_txg` property indicates the transaction
group (TXG) up to which the most recent scrub operation has
checked and repaired the dataset. This provides administrators
with insight into the data integrity status of their pool at a
specific point in time.

Sponsored-By: Wasabi Technology, Inc.
Sponsored-By: Klara Inc.
Signed-off-by: Mariusz Zaborski <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants