1936 add api log to log all api calls #2319
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduce new classes and extensions to facilitate logging of HTTP requests, responses, and exceptions in a web API project. - Add `ExceptionModel` in `ExceptionModel.cs` to capture exception details. - Add `HeaderDictionaryExtensions` in `HeaderDictionaryExtensions.cs` to convert `IHeaderDictionary` to `Dictionary<string, string>`. - Add `HttpLog` in `HttpLog.cs` to represent a log entry for an HTTP transaction. - Add `HttpRequestLog` in `HttpRequestLog.cs` to capture HTTP request details. - Add `HttpResponseLog` in `HttpResponseLog.cs` to capture HTTP response details.
Refactored `ExceptionModel`, `HttpLog`, `HttpRequestLog`, `HttpResponseLog`, and `IApiExecutionContext` to new namespace `FluentCMS.Entities.Logging`. Removed `HeaderDictionaryExtensions.cs`. Added `HttpLoggingMiddleware` for logging HTTP requests and responses. Introduced `IHttpLogRepository` and `HttpLogRepository` for logging to LiteDB.
Removed ExceptionModel and introduced HttpException. Refactored HttpLoggingMiddleware for better exception handling and logging. Updated HttpLog to use HttpException. Added HttpLogService and IHttpLogService for logging HTTP logs. Updated DI to include IHttpLogRepository service.
Added ApiTokenKey to IApiExecutionContext and ApiExecutionContext. Refactored HttpLog class to include new properties and removed Context and Exception properties. Introduced HttpLoggingMiddleware to log HTTP requests and responses. Enhanced HttpLoggingMiddleware with additional logging details. Simplified HttpLogRepository and added it to MongoDBServiceExtensions for dependency injection.
- Modify constructor to initialize `_liteDatabase` and `_liteDbContext` fields using `liteDbContext` parameter. - Introduce static boolean `_isInitialized` to track database initialization. - Add check in constructor to set `_isInitialized` if database has at least one collection. - Update `Create` method to check `_isInitialized` before log creation. - Rename `LiteDatabase` and `LiteDbContext` fields to `_liteDatabase` and `_liteDbContext`, and change access modifiers from `protected` to `private`. - Ensure `Create` method only inserts log if `_isInitialized` is `true`.
Added pragma directives to disable and restore the IDE0290 warning around the constructor of the JwtAuthorizationMiddleware class. This ensures the existing constructor format is maintained without triggering the IDE warning.
Added a new `Url` property to `HttpLog` and renamed `DisplayUrl` to `Url` in `HttpRequestLog`. Updated `HttpLoggingMiddleware` to use the new `Url` property. Modified `HttpLogRepository` to log only if initialized. Refactored `GlobalSettingsRepository` for constructor initialization and updated methods to use the new `apiExecutionContext`.
The `HttpLoggingMiddleware` class has been modified to temporarily disable the logging of request and response bodies. The `ReadRequestBody` and `ReadResponseBody` methods have been commented out, and their invocations within the `Invoke` method have been replaced with placeholder strings. The original response stream and memory stream handling code has also been commented out. A `TODO` comment has been added to reconsider when to log the request and response bodies.
Modified the GetCollectionName method in HttpLogRepository.cs to update collection names based on status codes. Changed "HttpLog400", "HttpLog300", "HttpLog500", and "HttpLog200" to "httplog400", "httplog300", "httplog500", and "httplog200" respectively. Corrected a typo where "httphog400" should be "httplog400".
Added a static boolean field `_isInitialized` to the `HttpLogRepository` class to check if the database is initialized. This is a temporary workaround, and comments indicate uncertainty about its long-term validity.
Removed HttpException, HttpRequestLog, and HttpResponseLog classes. Refactored HttpLog class to handle request, response, and exception details directly. Updated HttpLoggingMiddleware to use IOptions for HttpLogConfig and refactored logging logic. Updated ApiServiceExtensions to bind HttpLogConfig. Added new HttpLogConfig class and updated appsettings.json for HTTP logging settings.
Moved HttpLog class to FluentCMS.Entities namespace. Updated HttpLogService, HttpLoggingMiddleware, IHttpLogRepository, and HttpLogRepository files to remove obsolete using directives. Refactored HttpLoggingMiddleware to use local process variable. Added using System.Collections directive to HttpLog.cs.
Updated README.md to include sections for "Documentation" and "Logging" with a link to the new logging documentation. Expanded HttpLog.cs with XML documentation comments for each property, capturing extensive HTTP request, response, and exception details. Added Logging.md to provide comprehensive documentation on HTTP logging, including middleware overview, configuration options, and usage examples.
A new table named `HttpLogs` has been created to store detailed logs of HTTP requests and responses. This includes metadata such as status code, duration, assembly details, process and thread information, memory usage, machine and environment details, user information, request and response details, and exception details. This addition will aid in tracking and analyzing HTTP interactions within the system.
A new table named `HttpLogs` has been created in the `Setup.sql` file to store detailed HTTP log information. This table includes columns for request and response details, user and environment information, and exception details. Additionally, the comment `-- Table: ApiTokenPolicies` was moved to a different position in the file, but the `ApiTokenPolicies` table definition remains unchanged.
Updated FluentCmsDbContext.cs to include JSON serialization and deserialization for HttpLog entity properties (ReqHeaders, ResHeaders, ExData) using JsonSerializer. Added necessary using directives for System.Collections and System.Text.Json. Ensured base class OnModelCreating method is called.
- Register `IHttpLogRepository` as a scoped service in `EFCoreServiceExtensions`. - Add `DbSet<HttpLog>` to `FluentCmsDbContext` for managing HTTP logs. - Update `appsettings.json` to use `sqlite` instead of `LiteDb` and adjust connection string. - Implement `HttpLogRepository` in `FluentCMS.Repositories.EFCore` to handle HTTP log entries.
Refactored `HttpLoggingMiddleware` to use `IHttpLogService` instead of `IHttpLogRepository` for logging HTTP requests. Updated `appsettings.json` to change the database configuration from SQLite to LiteDb, including modifying the connection string to match LiteDb's format.
Refactored HttpLogService to use a background processing queue for logging HTTP logs. Updated IHttpLogService interface to reflect non-blocking logging. Modified HttpLoggingMiddleware to call Log method without awaiting. Updated IHttpLogRepository interface and repository implementations to support batch processing of log entries, improving performance and efficiency.
Refactored `HttpLogService` to use `IHttpLogChannel` and removed the original implementation and interface. Commented out HTTP logging middleware in `ApiServiceExtensions.cs` and `HttpLoggingMiddleware.cs`. Updated `ServiceExtensions.cs` to comment out `BackgroundHttpLogProcessor` and `HttpLogChannel` registrations. Commented out cancellation token checks and log insertion logic in `HttpLogRepository.cs` for EFCore and LiteDb. Added `BackgroundHttpLogProcessor.cs` for batch processing of HTTP logs and `HttpLogChannel.cs` for HTTP log channel implementation.
Uncommented and restored the `HttpLogService`, `HttpLoggingMiddleware`, and `HttpLogRepository` across multiple files. Re-enabled the registration of related services and middleware in the service collection and middleware pipeline. Simplified the `ReadAllAsync` method in `HttpLogChannel.cs` by removing the try-catch block.
Updated BackgroundHttpLogProcessor and HttpLogChannel to use IOptions<HttpLogConfig> for configuration settings from appsettings.json. Modified ServiceExtensions to bind HttpLogConfig settings and removed binding from ApiServiceExtensions. Moved HttpLogConfig to FluentCMS.Services namespace and added BatchSize property. Updated appsettings.json to include BatchSize.
Replaced the `HttpLogConfig` class configuration with a new `HttpLogging` section in the `appsettings.json` file. Added a table detailing the new configuration properties, including `Enable`, `EnableRequestBody`, `EnableResponseBody`, and `BatchSize`. Updated the example JSON configuration to reflect these changes.
Refactored ApiExecutionContext to use a new constant for the API auth header key. Updated ApiResultActionFilter and ApiResultExceptionFilter to use constructor injection for IApiExecutionContext. Introduced new interfaces and classes for API results, including IApiPagingResult, ApiPagingResult, IApiResult, and ApiResult. Updated appsettings.json for database configuration and logging. Added new PersonController and Person entity. Removed AppApiClientException and added AppError and AppException classes.
Added using directive for FluentCMS.Services.Models and removed Microsoft.AspNetCore.Http directive. Simplified IApiExecutionContext registration in the DI container from a lambda expression to a direct type mapping.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.