Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Fix naming and docs for retention configuration parameters #200

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions docs/docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,24 +177,24 @@ downloaded if you know the package's id and version. You can override this behav

## Package auto-deletion

If your build server generates many nuget packages, your BaGet server can quickly run out of space. Bagetter leverages [SemVer 2](https://semver.org/) and has logic to keep a history of packages based on the version numbering such as `<major>.<minor>.<patch>-<prerelease tag>.<prerelease build number>`.
If your build server generates many nuget packages, your BaGetter server can quickly run out of space. Bagetter leverages [SemVer 2](https://semver.org/) and has logic to keep a history of packages based on the version numbering such as `<major>.<minor>.<patch>-<prerelease tag>.<prerelease build number>`.

There is an optional section for `Retention` and the following parameters can be enabled to limit history for each level of the version. If none of these are set, there are no cleaning rules enforced. Each parameter is optional, e.g. if you specify only a `MaxHistoryPerPatch`, the package limit will only enforced for each major and minor version combination.
There is an optional config section for `Retention` and the following parameters can be enabled to limit history for each level of the version. If none of these are set, there are no cleaning rules enforced. Each parameter is optional, e.g. if you specify only a `MaxPatchVersions`, the package limit will only enforced within each major and minor version combination.
Packages deleted are always the oldest based on version numbers.

- MaxHistoryPerMajorVersion: Maximum number of major versions
- MaxHistoryPerMinorVersion: Maximum number of minor versions for each major version
- MaxHistoryPerPatch: Maximum number of patch versions for each major + minor version
- MaxHistoryPerPrerelease: Maximum number of prerelease versions for each major + minor + patch version and prerelease type. if you have `beta` and `alpha` this will keep `MaxHistoryPerPrerelease` versions for both `beta` and `alpha`.
- MaxMajorVersions: Maximum number of major versions for each package
- MaxMinorVersions: Maximum number of minor versions for each major version
- MaxPatchVersions: Maximum number of patch versions for each major + minor version
- MaxPrereleaseVersions: Maximum number of prerelease builds for each major + minor + patch version and prerelease type. If you have `beta` and `alpha` this will keep `MaxPrereleaseVersions` versions for both `beta` and `alpha`.

```json
{
...
"Retention": {
"MaxHistoryPerMajorVersion": 5,
"MaxHistoryPerMinorVersion": 5,
"MaxHistoryPerPatch": 5,
"MaxHistoryPerPrerelease": 5,
"MaxMajorVersions": 5,
"MaxMinorVersions": 5,
"MaxPatchVersions": 5,
"MaxPrereleaseVersions": 5,
}
...
}
Expand Down
34 changes: 19 additions & 15 deletions src/BaGetter.Core/Configuration/RetentionOptions.cs
Original file line number Diff line number Diff line change
Expand Up @@ -3,31 +3,35 @@ namespace BaGetter.Core;
public class RetentionOptions
{
/// <summary>
/// If this is set to a value, it will limit the number of versions that can be pushed for a package.
/// The limit is applied to each major version of the package, and if the limit is exceeded,
/// If this is set to a value, it will limit the number of versions that will be retained for a package.
/// The limit is applied to all major version of the package, and if the limit is exceeded,
/// the older versions will be deleted.
/// For a limit of 5, if there are versions 1.*.* through 5.*.* and a package version 6.0.0 is pushed, versions 1.*.* will be deleted, including all minor, patch and prerelease versions from that major version.
/// </summary>
public uint? MaxHistoryPerMajorVersion { get; set; } = null;
public uint? MaxMajorVersions { get; set; } = null;

/// <summary>
/// This corresponds to the maximum number of minor versions for each major version.
/// If this is set to a value, it will limit the number of versions that can be pushed for a package.
/// The limit is applied to each minor version of the package, and if the limit is exceeded,
/// If this is set to a value, it will limit the number of versions that will be retained for a package.
/// The limit is applied within each major version of the package, and if the limit of minor versions within a major version is exceeded,
/// the older versions will be deleted.
/// For a limit of 5, if there are versions 1.0.* through 1.5.* and a package version 1.6.0 is pushed, versions 1.0.* will be deleted, including all patch and prerelease versions from that minor version.
/// </summary>
public uint? MaxHistoryPerMinorVersion { get; set; }
public uint? MaxMinorVersions { get; set; }

/// <summary>
/// If this is set to a value, it will limit the number of versions that can be pushed for a package.
/// The limit is applied to each patch number of the package, and if the limit is exceeded,
/// If this is set to a value, it will limit the number of versions that will be retained for a package.
/// The limit is applied within each minor version of the package, and if the limit of patches within a minor version is exceeded,
/// the older versions will be deleted.
/// For a limit of 5, if there are versions 1.0.0 through 1.0.5 and a package version 1.0.6 is pushed, version 1.0.0 will be deleted, including all prerelease versions from that patch version.
/// </summary>
public uint? MaxHistoryPerPatch { get; set; }
public uint? MaxPatchVersions { get; set; }

/// <summary>
/// If this is set to a value, it will limit the number of versions that can be pushed for a package.
/// The limit is applied to each pre-release of the package, and if the limit is exceeded,
/// the older versions will be deleted.
/// </summary>
public uint? MaxHistoryPerPrerelease { get; set; }
/// <summary>
/// If this is set to a value, it will limit the number of versions that will be retained for a package.
/// The limit is applied within each prerelease label of the package, and if the limit of prerelease builds within a label is exceeded,
/// the older versions will be deleted.
/// For a limit of 5, if there are versions 1.0.0-alpha.1 through 1.0.0-alpha.5 and a package version 1.0.0-alpha.6 is pushed, version 1.0.0-alpha.0 will be deleted.
/// </summary>
public uint? MaxPrereleaseVersions { get; set; }
}
13 changes: 7 additions & 6 deletions src/BaGetter.Core/Indexing/PackageIndexingService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ public PackageIndexingService(
#pragma warning disable CS0618 // Type or member is obsolete
if (_options.Value.MaxVersionsPerPackage > 0)
{
_logger.LogError("MaxVersionsPerPackage is deprecated and is not used. Please use MaxHistoryPerMajorVersion, MaxHistoryPerMinorVersion, MaxHistoryPerPatch, and MaxHistoryPerPrerelease instead.");
_logger.LogError("MaxVersionsPerPackage is deprecated and is not used. Please use MaxMajorVersions, MaxMinorVersions, MaxPatchVersions, and MaxPrereleaseVersions instead.");
}
#pragma warning restore CS0618 // Type or member is obsolete
}
Expand Down Expand Up @@ -165,19 +165,20 @@ await _storage.SavePackageContentAsync(

await _search.IndexAsync(package, cancellationToken);

if (_retentionOptions.Value.MaxHistoryPerMajorVersion.HasValue)
if (_retentionOptions.Value.MaxMajorVersions.HasValue)
{
try {
_logger.LogInformation(
"Deleting older packages for package {PackageId} {PackageVersion}",
package.Id,
package.NormalizedVersionString);

var deleted = await _packageDeletionService.DeleteOldVersionsAsync(
package,
_retentionOptions.Value.MaxHistoryPerMajorVersion,
_retentionOptions.Value.MaxHistoryPerMinorVersion,
_retentionOptions.Value.MaxHistoryPerPatch,
_retentionOptions.Value.MaxHistoryPerPrerelease,
_retentionOptions.Value.MaxMajorVersions,
_retentionOptions.Value.MaxMinorVersions,
_retentionOptions.Value.MaxPatchVersions,
_retentionOptions.Value.MaxPrereleaseVersions,
cancellationToken);
if (deleted > 0)
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -240,10 +240,10 @@ public async Task IndexIMAsync_WithValidPackage_CleansOldVersions()
// Arrange
_options.AllowPackageOverwrites = PackageOverwriteAllowed.False;

_retentionOptions.MaxHistoryPerMajorVersion = 2;
_retentionOptions.MaxHistoryPerMinorVersion = 2;
_retentionOptions.MaxHistoryPerPatch = 5;
_retentionOptions.MaxHistoryPerPrerelease = 5;
_retentionOptions.MaxMajorVersions = 2;
_retentionOptions.MaxMinorVersions = 2;
_retentionOptions.MaxPatchVersions = 5;
_retentionOptions.MaxPrereleaseVersions = 5;
// Add 10 packages
for (var major = 1; major < 4; major++)
{
Expand All @@ -262,24 +262,24 @@ public async Task IndexIMAsync_WithValidPackage_CleansOldVersions()

var packageVersions = await _packages.FindAsync(builder.Id, true, default);
var majorCount = packageVersions.Select(p => p.Version.Major).Distinct().Count();
Assert.Equal(majorCount, Math.Min(major, (int)_retentionOptions.MaxHistoryPerMajorVersion));
Assert.True(majorCount <= _retentionOptions.MaxHistoryPerMajorVersion, $"Major version {major} has {majorCount} packages");
Assert.Equal(majorCount, Math.Min(major, (int)_retentionOptions.MaxMajorVersions));
Assert.True(majorCount <= _retentionOptions.MaxMajorVersions, $"Major version {major} has {majorCount} packages");

// validate maximum number of minor versions for each major version.
var minorVersions = packageVersions.GroupBy(m => m.Version.Major)
.Select(gp => (version: gp.Key, versionCount: gp.Select(p => p.Version.Major + "." + p.Version.Minor).Distinct().Count())).ToList();
Assert.All(minorVersions, g => Assert.True(g.versionCount <= _retentionOptions.MaxHistoryPerMinorVersion, $"Minor version {g.version} has {g.versionCount} packages"));
Assert.All(minorVersions, g => Assert.True(g.versionCount <= _retentionOptions.MaxMinorVersions, $"Minor version {g.version} has {g.versionCount} packages"));

// validate maximum number of minor versions for each major version.
var patches = packageVersions.GroupBy(m => (m.Version.Major, m.Version.Minor))
.Select(gp => (version: gp.Key, versionCount: gp.Select(p => p.Version.Major + "." + p.Version.Minor + "." + p.Version.Patch).Distinct().Count())).ToList();
Assert.All(patches, g => Assert.True(g.versionCount <= _retentionOptions.MaxHistoryPerPatch, $"Patch version {g.version} has {g.versionCount} packages"));
Assert.All(patches, g => Assert.True(g.versionCount <= _retentionOptions.MaxPatchVersions, $"Patch version {g.version} has {g.versionCount} packages"));

// validate maximum number of beta versions for each major,minor,patch version.
var betaVersions = packageVersions.Where(p => p.IsPrerelease && p.Version.ReleaseLabels.First() == "beta")
.GroupBy(m => (m.Version.Major, m.Version.Minor, m.Version.Patch))
.Select(gp => (version: gp.Key, versionCount: gp.Select(p => p.Version.Major + "." + p.Version.Minor + "." + p.Version.Patch).Distinct().Count())).ToList();
Assert.All(betaVersions, g => Assert.True(g.versionCount <= _retentionOptions.MaxHistoryPerPatch, $"Pre-Release version {g.version} has {g.versionCount} packages"));
Assert.All(betaVersions, g => Assert.True(g.versionCount <= _retentionOptions.MaxPatchVersions, $"Pre-Release version {g.version} has {g.versionCount} packages"));


}
Expand Down
Loading