Paging settings. (#51)

RefData policy settings.

CHANGELOG.md

@@ -2,6 +2,10 @@
 
 Represents the **NuGet** versions.
 
+## v2.3.0
+- *Enhancement:* `PagingArgs.MaxTake` default set by `SettingsBase.PagingMaxTake`.
+- *Enhancement:* Reference data `ICacheEntry` policy configuration can now be defined in settings.
+
 ## v2.2.0
 - *Fixed:* Entity Framework `EfDb.UpdateAsync` resolved error where the instance of entity type cannot be tracked because another instance with the same key value is already being tracked.
 - *Fixed:* The `CollectionMapper` was incorrectly appending items to an existing collection, versus performing a replacement operation.

Common.targets

@@ -1,6 +1,6 @@
 <Project>
 	<PropertyGroup>
-		<Version>2.2.0</Version>
+		<Version>2.3.0</Version>
 		<LangVersion>preview</LangVersion>
 		<Authors>Avanade</Authors>
 		<Company>Avanade</Company>

samples/My.Hr/My.Hr.Api/appsettings.json

@@ -4,5 +4,13 @@
       "Default": "Information",
       "Microsoft.AspNetCore": "Warning"
     }
+  },
+  "RefDataCache": {
+    "AbsoluteExpirationRelativeToNow": "01:45:00",
+    "SlidingExpiration": "00:15:00",
+    "Gender": {
+      "AbsoluteExpirationRelativeToNow": "03:00:00",
+      "SlidingExpiration": "00:45:00"
+    }
   }
 }

samples/My.Hr/My.Hr.Business/Data/HrDb.cs

@@ -1,8 +1,4 @@
-using CoreEx.Database;
-using CoreEx.Database.SqlServer;
-using Microsoft.Data.SqlClient;
-
-namespace My.Hr.Business.Data
+namespace My.Hr.Business.Data
 {
     public class HrDb : SqlServerDatabase
     {

samples/My.Hr/My.Hr.Business/Data/HrDbContext.cs

@@ -1,6 +1,3 @@
-using CoreEx.Database;
-using CoreEx.EntityFrameworkCore;
-
 namespace My.Hr.Business.Data;
 
 public class HrDbContext : DbContext, IEfDbContext

samples/My.Hr/My.Hr.Business/Data/HrEfDb.cs

@@ -1,6 +1,4 @@
-using CoreEx.Mapping;
-
-namespace My.Hr.Business.Data
+namespace My.Hr.Business.Data
 {
     /// <summary>
     /// Enables the <b>My.Hr</b> database using Entity Framework.

samples/My.Hr/My.Hr.Business/ImplicitUsings.cs

@@ -1,11 +1,15 @@
 global using CoreEx;
 global using CoreEx.Configuration;
+global using CoreEx.Database;
+global using CoreEx.Database.SqlServer;
 global using CoreEx.Entities;
 global using CoreEx.EntityFrameworkCore;
 global using CoreEx.Events;
 global using CoreEx.Http;
 global using CoreEx.Json;
+global using CoreEx.Mapping;
 global using CoreEx.RefData;
+global using Microsoft.Data.SqlClient;
 global using Microsoft.EntityFrameworkCore;
 global using Microsoft.EntityFrameworkCore.Metadata.Builders;
 global using Microsoft.Extensions.Configuration;

src/CoreEx.EntityFrameworkCore/README.md

@@ -0,0 +1,112 @@
+# CoreEx
+
+The `CoreEx.EntityFrameworkCore` namespace provides extended [_Entity Framework Core (EF)_](https://learn.microsoft.com/en-us/ef/core/) capabilities. 
+
+<br/>
+
+## Motivation
+
+The motivation is to provide supporting EF Core capabilities for [CRUD](https://en.wikipedia.org/wiki/Create,_read,_update_and_delete) related access that support standardized _CoreEx_ data access patterns. This for the most part will simplify and unify the approach to ensure consistency of implementation where needed.
+
+<br/>
+
+## Requirements
+
+The requirements for usage are as follows.
+- An **entity** (DTO) that represents the data that must as a minimum implement [`IEntityKey`](../CoreEx/Entities/IEntityKey.cs); generally via either the implementation of [`IIdentifier`](../CoreEx/Entities/IIdentifierT.cs) or [`IPrimaryKey`](../CoreEx/Entities/IPrimaryKey.cs).
+- A **model** being the underlying configured EF Core [data source model](https://learn.microsoft.com/en-us/ef/core/modeling/).
+- An [`IMapper`](../CoreEx/Mapping/IMapper.cs) that contains the mapping logic to map to and from the **entity** and **model**.
+
+The **entity** and **model** are different types to encourage separation between the externalized **entity** representation and the underlying **model**; which may be shaped differently, and have different property to column naming conventions, etc.
+
+<br/>
+
+## CRUD capabilities
+
+The [`IEfDb`](./IEfDb.cs) and corresponding [`EfDb`](./EfDb.cs) provides the base CRUD capabilities as follows.
+
+<br/>
+
+### Query (read)
+
+A query is actioned using the [`EfDbQuery`](./EfDbQuery.cs) which is obstensibly a lightweight wrapper over an `IQueryable<TModel>` that automatically maps from the **model** to the **entity**.
+
+Queried entities are not tracked by default; internally uses [`AsNoTracking`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.entityframeworkqueryableextensions.asnotracking); this behaviour can be overridden using [`EfDbArgs.QueryNoTracking`](./EfDbArgs.cs).
+
+Note: a consumer should also consider using [`IgnoreAutoIncludes`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.entityframeworkqueryableextensions.ignoreautoincludes) to exclude related data, where not required, to improved query performance.
+
+The following methods provide additional capabilities.
+
+Method | Description
+-|-
+`WithPaging` | Adds `Skip` and `Take` paging to the query.
+`SelectSingleAsync` | Selects a single item.
+`SelectSingleOrDefaultAsync` | Selects a single item or default.
+`SelectFirstAsync` | Selects first item.
+`SelectSingleOrDefaultAsync` | Selects first item or default.
+`SelectQuery` | Select items into or creating a resultant collection.
+`SelectResultAsync` | Select items creating a [`ICollectionResult`](../CoreEx/Entities/ICollectionResultT2.cs) which also contains corresponding [`PagingResult`](../CoreEx/Entities/PagingResult.cs).
+
+<br/>
+
+### Get (Read)
+
+Gets the **entity** for the specified key mapping from the **model**. Uses the [`DbContext.Find`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.find) internally for the **model** and specified key.
+
+Where the data is not found, then a `null` will be returned. Where the **model** implements [`ILogicallyDeleted`](../CoreEx/Entities/ILogicallyDeleted.cs) and `IsDeleted` then this acts as if not found and returns a `null`.
+
+<br/>
+
+### Create
+
+Creates the **entity** by firstly mapping to the **model**. Uses the [`DbContext.Add`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.add) to begin tracking the **model** which will be inserted into the database when [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called.
+
+Where the **entity** implements [`IChangeLogAuditLog`](../CoreEx/Entities/IChangeLogAuditLog.cs) generally via [`ChangeLog`](../CoreEx/Entities/IChangeLog.cs) or [`ChangeLogEx`](../CoreEx/Entities/Extended/IChangeLogEx.cs), then the `CreatedBy` and `CreatedDate` properties will be automatically set from the [`ExecutionContext`](../CoreEx/ExecutionContext.cs).
+
+Where the **entity** and/or **model** implements [`ITenantId`](../CoreEx/Entities/ITenantId.cs) then the `TenantId` property will be automatically set from the [`ExecutionContext`](../CoreEx/ExecutionContext.cs).
+
+Generally, the [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called to perform the _insert_; unless [`EfDbArgs.SaveChanges`](./EfDbArgs.cs) is set to `false` (defaults to `true`).
+
+The inserted **model** is then re-mapped to the **entity** and returned where [`EfDbArgs.Refresh`](./EfDbArgs.cs) is set to `true` (default); this will ensure all properties updated as part of the _insert_ are included in the refreshed **entity**.
+
+<br/>
+
+### Update
+
+Updates the **entity** by firstly mapping to the **model**. Uses the [`DbContext.Update`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.add) to begin tracking the **model** which will be updated within the database when [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called.
+
+First will check existence of the **model** by performing a `DbContext.Find`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.find). Where the data is not found, then a [`NotFoundException`](../CoreEx/NotFoundException.cs) will be thrown. Where the **model** implements [`ILogicallyDeleted`](../CoreEx/Entities/ILogicallyDeleted.cs) and `IsDeleted` then this acts as if not found and will also result in a `NotFoundException`.
+
+Where the entity implements [`IETag`](../CoreEx/Entities/IETag.cs) this will be checked against the just read version, and where not matched a  [`ConcurrencyException`](../CoreEx/ConcurrencyException.cs) will be thrown. Also, any [`DbUpdateConcurrencyException`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbupdateconcurrencyexception) thrown will be converted to a corresponding `ConcurrencyException` for consistency.
+
+Where the **entity** implements [`IChangeLogAuditLog`](../CoreEx/Entities/IChangeLogAuditLog.cs) generally via [`ChangeLog`](../CoreEx/Entities/IChangeLog.cs) or [`ChangeLogEx`](../CoreEx/Entities/Extended/IChangeLogEx.cs), then the `UpdatedBy` and `UpdatedDate` properties will be automatically set from the [`ExecutionContext`](../CoreEx/ExecutionContext.cs).
+
+Where the **entity** and/or **model** implements [`ITenantId`](../CoreEx/Entities/ITenantId.cs) then the `TenantId` property will be automatically set from the [`ExecutionContext`](../CoreEx/ExecutionContext.cs).
+
+Generally, the [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called to perform the _update_; unless [`EfDbArgs.SaveChanges`](./EfDbArgs.cs) is set to `false` (defaults to `true`).
+
+The updated **model** is then re-mapped to the **entity** and returned where [`EfDbArgs.Refresh`](./EfDbArgs.cs) is set to `true` (default); this will ensure all properties updated as part of the _update_ are included in the refreshed **entity**.
+
+<br/>
+
+### Delete
+
+Deletes the **entity** either physically or logically,
+
+First will check existence of the **model** by performing a `DbContext.Find`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.find). Where the data is not found, then a [`NotFoundException`](../CoreEx/NotFoundException.cs) will be thrown. Where the **model** implements [`ILogicallyDeleted`](../CoreEx/Entities/ILogicallyDeleted.cs) and `IsDeleted` then this acts as if not found and will also result in a `NotFoundException`.
+
+Where the **model** implements [`ILogicallyDeleted`](../CoreEx/Entities/ILogicallyDeleted.cs) then an update will occur after setting `IsDeleted` to `true`. Uses the [`DbContext.Update`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.add) to begin tracking the **model** which will be updated within the database when [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called.
+
+Otherwise, will physically delete. Uses the [`DbContext.Remove`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.remove) to begin tracking the **model** which will be deleted from the database when [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called.
+
+Generally, the [`DbContext.SaveChanges`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext.savechanges) is called to perform the _update_; unless [`EfDbArgs.SaveChanges`](./EfDbArgs.cs) is set to `false` (defaults to `true`).
+
+<br/>
+
+## Usage
+
+To use `EfDB` relationships to the EF Core [`DbContext`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.dbcontext) must be established as follows.
+
+- [`Database`](../CoreEx.Database/Database.cs) must be defined; see [example](../../samples/My.Hr/My.Hr.Business/Data/HrDb.cs).
+- `DbContext` and [`Database`](../CoreEx.Database/Database.cs) relationship must be defined; see [example](../../samples/My.Hr/My.Hr.Business/Data/HrDbContext.cs).
+- [`EfDb`](./EfDb.cs) and `DbContext` relationship must be defined; see [example](../../samples/My.Hr/My.Hr.Business/Data/HrEfDb.cs).

src/CoreEx/Configuration/SettingsBase.cs

@@ -7,8 +7,8 @@
 using System.Reflection;
 using System.Runtime.CompilerServices;
 using System.Threading;
-using CoreEx.Entities;
 using CoreEx.Http;
+using Microsoft.Extensions.Caching.Memory;
 using Microsoft.Extensions.Configuration;
 
 namespace CoreEx.Configuration
@@ -41,11 +41,6 @@ public SettingsBase(IConfiguration? configuration, params string[] prefixes)
             }
 
             _allProperties = GetType().GetProperties(BindingFlags.Public | BindingFlags.Instance | BindingFlags.FlattenHierarchy).ToDictionary(p => p.Name, p => p);
-
-            // Configure (override) any standard settings.
-            var take = PagingDefaultTake;
-            if (take != null)
-                PagingArgs.DefaultTake = take.Value;
         }
 
         /// <summary>
@@ -197,6 +192,21 @@ public T GetRequiredValue<T>([CallerMemberName] string key = "")
         /// <summary>
         /// Gets the <see cref="Entities.PagingArgs.DefaultTake"/>; i.e. page size.
         /// </summary>
-        public long? PagingDefaultTake => GetValue<long?>(nameof(PagingDefaultTake), null);
+        public long PagingDefaultTake => GetValue<long>(nameof(PagingDefaultTake), 100);
+
+        /// <summary>
+        /// Gets the <see cref="Entities.PagingArgs.MaxTake"/>; i.e. absolute maximum page size.
+        /// </summary>
+        public long PagingMaxTake => GetValue<long>(nameof(PagingMaxTake), 1000);
+
+        /// <summary>
+        /// Gets the default <see cref="RefData.ReferenceDataOrchestrator"/> <see cref="ICacheEntry.AbsoluteExpirationRelativeToNow"/>. Defaults to <c>2</c> hours.
+        /// </summary>
+        public TimeSpan? RefDataCacheAbsoluteExpirationRelativeToNow => GetValue($"RefDataCache__{nameof(ICacheEntry.AbsoluteExpirationRelativeToNow)}", TimeSpan.FromHours(2));
+
+        /// <summary>
+        /// Gets the default <see cref="RefData.ReferenceDataOrchestrator"/> <see cref="ICacheEntry.SlidingExpiration"/>. Defaults to <c>30</c> minutes.
+        /// </summary>
+        public TimeSpan? RefDataCacheSlidingExpiration => GetValue($"RefDataCache__{nameof(ICacheEntry.SlidingExpiration)}", TimeSpan.FromMinutes(30));
     }
 }

src/CoreEx/Entities/PagingArgs.cs

@@ -14,15 +14,15 @@ namespace CoreEx.Entities
     public class PagingArgs
     {
         private static long? _defaultTake;
-        private static long _maxTake = 1000;
+        private static long? _maxTake;
 
         /// <summary>
         /// Gets or sets the default <see cref="Take"/> size.
         /// </summary>
         /// <remarks>Defaults to <see cref="SettingsBase.PagingDefaultTake"/> where specified; otherwise, <c>100</c>.</remarks>
         public static long DefaultTake
         {
-            get => _defaultTake ??= ExecutionContext.GetService<SettingsBase>()?.PagingDefaultTake ?? 100;
+            get => _defaultTake ?? ExecutionContext.GetService<SettingsBase>()?.PagingDefaultTake ?? 100;
 
             set
             {
@@ -32,12 +32,12 @@ public static long DefaultTake
         }
 
         /// <summary>
-        /// Gets or sets the maximum <see cref="Take"/> size allowed (defaults to 1000).
+        /// Gets or sets the absolute maximum <see cref="Take"/> size allowed.
         /// </summary>
-        /// <remarks>Defaults to <c>false</c>.</remarks>
+        /// <remarks>Defaults to <see cref="SettingsBase.PagingMaxTake"/> where specified; otherwise, <c>1000</c>.</remarks>
         public static long MaxTake
         {
-            get => _maxTake;
+            get => _maxTake ?? ExecutionContext.GetService<SettingsBase>()?.PagingMaxTake ?? 1000;
 
             set
             {

src/CoreEx/README.md

@@ -24,7 +24,7 @@ Namespace | Description
 `Http` | Provides capabilities to enable extended [typed](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/http-requests) [`HttpClient`](./Http/TypedHttpClientBaseT.cs) functionality providing a fluent-style method-chaining to enable the likes of `WithRetry`, `EnsureSuccess`, `Timeout`, and `ThrowTransientException`, etc. to improve the per invocation experience. Additionally, [`HttpRequestOptions`](./Http/HttpRequestOptions.cs) enable additional standardized options to be specified per request where applicable.
 `Json` | Whilst .NET recently added [`System.Text.Json`](https://docs.microsoft.com/en-us/dotnet/api/system.text.json) there is still extensive usage of [`Newtonsoft.Json`](https://www.newtonsoft.com/json), and there can be [challenges migrating](https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-migrate-from-newtonsoft-how-to) from the latter to the former. To aid the transition, and to support each, _CoreEX_ introduces [`IJsonSerializer`](./Json/IJsonSerializer.cs) which is used almost exclusively within _CoreEx_ to encapsulate usage -  [`CoreEx.Text.Json.JsonSerializer`](./Text/Json/JsonSerializer.cs) and [`CoreEx.Newtonsoft.Json.JsonSerializer`](../CoreEx.Newtonsoft/Json/JsonSerializer.cs) implementations are provided.
 `Localization` | To enable a simple and consistent [localization](https://docs.microsoft.com/en-us/dotnet/core/extensions/globalization-and-localization) experience, the [`LText`](./Localization/LText.cs) struct provides a light-weight wrapper over a [`ITextProvider`](./Localization/ITextProvider.cs) that implements the string [localization](https://docs.microsoft.com/en-us/dotnet/core/extensions/localization) replacement. 
-`RefData` | Provides standardized and enriched capabilities for reference data.
+[`RefData`](./RefData) | Provides standardized and enriched capabilities for reference data.
 `Text.Json` | Provides [`System.Text.Json`](https://docs.microsoft.com/en-us/dotnet/api/system.text.json) implementation of the [`IJsonSerializer`](./Json/IJsonSerializer.cs).
 `Validation` | Provides for implementation agnostic [`IValidator<T>`](./Validation/IValidatorT.cs).
 [`WebApis`](./WebApis) | Provides extended capabilities to build Web APIs, for the likes of [ASP.NET](https://dotnet.microsoft.com/en-us/apps/aspnet/apis) or [HTTP-triggered Azure functions](https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-http-webhook-trigger). The [WebApi](./WebApis/WebApi.cs) and [WebApiPublisher](./WebApis/WebApiPublisher.cs) capabilities within encapsulate the consistent handling of the HTTP request and corresponding response, whilst also providing additional capabilities that are not available out-of-the-box within the .NET runtime.

src/CoreEx/RefData/README.md

@@ -64,6 +64,29 @@ Each `IReferenceDataProvider` (typically only one) instance is registered via th
 
 <br/>
 
+### Cache policy configuration
+
+The default implementation for the `IMemoryCache` is that each _reference data_ type's collection [`ICacheEntry`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.caching.memory.icacheentry) is set in a standardized manner; being `AbsoluteExpirationRelativeToNow` and `SlidingExpiration` properties defaulted to `02:00:00` (2 hours) and `00:30:00` (30 minutes) respectively.
+
+These defaults can be overridden within the configuration [settings](../Configuration/SettingsBase.cs); as can specific _reference data_ types. The following is an example of setting the defaults and then specifically overridding the `Gender` policy (the `Type.Name` is used as the property name).
+
+``` json
+{
+  "RefDataCache": {
+    "AbsoluteExpirationRelativeToNow": "01:45:00",
+    "SlidingExpiration": "00:15:00",
+    "Gender": {
+      "AbsoluteExpirationRelativeToNow": "03:00:00",
+      "SlidingExpiration": "00:45:00"
+    }
+  }
+}
+```
+
+Where the above is not sufficient then the virtual `OnCreateCacheEntry` method can be overridden to fully customize (override) the default behaviour.
+
+<br/>
+
 ## Reference data properties and serialization
 
 It is recommended that the rich _reference data_ types themselves where included within an entity are not JSON serialized/deserialized over the wire from the likes of an API; in these instances only the `Code` is generally used. This is to minimize the resulting payload size of the owning entity.