Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for placeholder keywords for dynamic changes in texts based on recipient #545

Closed
14 of 15 tasks
Tracked by #88
SandGrainOne opened this issue Jun 11, 2024 · 9 comments
Closed
14 of 15 tasks
Tracked by #88
Labels
kind/feature-request New feature or request

Comments

@SandGrainOne
Copy link
Member

SandGrainOne commented Jun 11, 2024

Description

We want the Notifications service to have solid support for having multiple independent recipients in a single order. This means the texts needs to have some basic support for personalization based on recipient. Like addressing the recipient by name.

There needs to be way for service owners to make smaller recipient specific adjustments to the content of notifications. The feature here should be a search and replace in texts

Keywords copied from Altinn 2 and adjusted for Altinn 3:

Keyword Description
$recipientFirstName$ The first name of a person *
$recipientMiddleName$ The middle name of a person *
$recipientLastName$ The surname of a person *
$recipientName$ The full name of the recipient, support both person and organization.
$recipientNumber$ The organization number when recipient is an organization. Left empty if recipient is a person.
  • Support for first and middle name is postponed untill we have a propper data source.

Example

A simple request with personalized texts:

{
    "subject": "Happy birthday $recipientFirstName$!",
    "body": "Hi $recipientFirstName$, remember to never click any links in emails!",
    "recipients": [
        {
            "nationalIdentityNumber": "18874198354"
        }
    ]
}

There is a new specialized endpoint in Register that should provide what we need:
register/api/v1/parties/nameslookup
See issue: #593

Additional Information

The source of the keyword list above is inspired by Altinn 2 documentation: List of Varselmakroer. The list is curated down to a list of keywords actually being used in Altinn 2.

Refinement

  • We can perform the lookup and macro replacements during the order processing. The final texts is added to the generated notifications being written to Kafka topics.
  • It's difficult to validate that all keywords in texts are actual keywords for replacement. We'll assume that the application owners will test their texts and fix mistakes like missing $ symbols or typos.
  • Investigate if there is a library we can use. Templating/macro.

Tasks

  • Verify that the $recipientEmail$
    $recipientMobileNumber$ are actually in use before implementing support
  • Investigate available templating/macro libraries
    • Library needs to add value outside search and replace of text
    • Can we use the defined keywords or does the library require a different format
  • Implement necessary clients for obtaining the Party/Person/Organization objects
    • Some data might already be available through existing clients
    • Add caching of the data where appropriate for easy reuse between lookup and keyword replacement
  • Unit/integration tests
  • Include a keyword in one of the existing use-case/regression tests
  • Documentation
  • Review
  • Prepare and concuct demo (Altinn internal demo, Friday 13th)

Acceptance criteria

  • Original texts is stored without changes including keywords
  • Keywords in notification texts are being replaced by the correct values in the final notification(s)
  • Documentation includes information about keywords and how they can be used
@SandGrainOne SandGrainOne added the kind/feature-request New feature or request label Jun 11, 2024
@olebhansen olebhansen added the status/blocked Further work depending on the completion of some other task/PoC/issue label Aug 12, 2024
@olebhansen
Copy link

olebhansen commented Aug 13, 2024

Pending a suitable endpoint to surce data from (#589 #593 )

@SandGrainOne
Copy link
Member Author

Updated the list of keywords we're going to support after the #604 analysis.

@olebhansen olebhansen removed the status/blocked Further work depending on the completion of some other task/PoC/issue label Oct 22, 2024
@Ahmed-Ghanam Ahmed-Ghanam self-assigned this Nov 13, 2024
@Ahmed-Ghanam
Copy link
Contributor

Ahmed-Ghanam commented Nov 26, 2024

26.11.2024
During a meeting to assess our requirements, we made the following decisions:

  • Initially, we will support only the $recipientName$ and $recipientNumber$ placeholders. Other keywords can be implemented in the future if required.

Action points:

@olebhansen
Copy link

FYI, some tests from team "Melding og Formidling": https://digdir.atlassian.net/wiki/spaces/BF1/pages/2798256130/Melding+med+varsling

@Ahmed-Ghanam
Copy link
Contributor

The documentation has been updated and here is the pull request Altinn/altinn-studio-docs#1951

@Ahmed-Ghanam
Copy link
Contributor

The planning and implementation stages of this task took longer than expected because of the need to handle edge cases. I updated the size from L to XL

Ahmed-Ghanam added a commit that referenced this issue Jan 9, 2025
* Improve the documentation

* Fill in the recipient's name based on whether or not placeholder keywords are used in the template.

* Implemented a new function to retrieve party details for organizations.

* Extend the lookup logic to check for the recipientNumber keyword

* Improve the documentation

* Add a new property to check whether the template contains any recipient number placeholders

* Improve the documentation

* Eliminate the functions responsible for retrieving the individual name components.

* Allow the user to send the components of recipient name when creating an order of any type

* Simplify the keywords detection logic

* Remove the RecipientNameComponents type.

* Remove the name components

* Remove the unnecessary ConfigureAwait

* Create a new keyword service to inject actual value wherever a keyword is found

* Update the test data

* Update two functions to retrieve recipient number and name

* Update the sender email address.

* Save the customized email subject and body in the database

* Pass customized body and subject when trying to send Emails again

* Add support for placeholder keywords in SMS

* Improve the keyword replacement logic

* Replace Regex checks with string.Contains

* Improve the request that we use to retrieve unit details.

* Add validation logic to ensure that either organizationNumber or socialSecurityNumber is set at a time.

* Added a constructor for batch creation from existing lists to simplify usage in client code.

* Update the function that is used to retrieve emails

* Fix build errors

* Pass a default value for the new parameters

* Breaking a large method into smaller, more focused private methods

* Make a method static

* Code refactoring

* Use explicit JOIN instead of the implicit join in the FROM clause and improve indentation for better readability.

* Use SELECT INTO for assigning values to the __orderid variable and improve indentation for better readability.

* Add a script to alter tables

* Undo changes made on the auto generated file and move them to new scripts

* Remove two unnecessary values.

* Add a missing parenthesis

* Remove two unnecessary properties.

* Remove two unnecessary values.

* Enhance the logic to ensure all existing unit tests execute successfully.

* Save and retrieve the customized subject and body to and from the database

* Improve the documentation

* Update the keywords service to handle data batches

* Remove the logic used to drop and create both functions and procedures

* Update the Email sending and retry logic

* Code refactoring

* Fix the parameter of some unit tests

* Adjust the code to run unit test successfully

* Handle batch of organization and national identity numbers at one

* Update the test units to reflect latest changes

* Improve the readability

* Update the SQL query used to retrieve recipients

* Implement simple test units to text keyword replacement logic

* Update unit tests and test data

* Fix a test unit

* Update the order logic to handle orders sent to recipients without using national identity and organization numbers

* Added test units to test retrieving party details using organization and national identity numbers.

* Allow the user to retrieve party detail for both organization and national identity number in the same request.

* Refactor test units

* Improve the SMS order processing logic

* Remove the organization number

* Refactor: Remove unused PersonNameComponents type.

* #545 Added tests to increase coverage.

* #545 Validate the constructors and JSON serialization behavior of the PartyDetailsLookupRequest class across diverse scenarios.

* Change the type of a field to improve performance

* Add three test units to cover more use cases

* Remove test cases that check for the nullability of parameters that are defined as non-nullable.

* Replace the national identity number with an empty string

* Do not replace the $recipientNumber$ with the national identity number

* Compare the keywords using ordinal sort rules

* Optimize the logic to consolidate the retrieval of party details

* Remove unused methods and update test scenarios.

* Mock the GetPartyDetails function

* Update the use cases to match the new logic.

* Update the error message and add a check to ensure that data is correctly populated according to the provided lists.

* Improve the tests logic

* Code refactoring

* Ensure a property is non-nullable.

* Increase test coverage.

* Remove a null check

* Move a type to the integrations namespace

* Code refactoring

* Move two files to the Integrations project

* Move two files to the Integrations project

* Use a dummy endpoint.
Ahmed-Ghanam added a commit that referenced this issue Jan 10, 2025
* Improve the documentation

* Improve the documentation and initialization

* Improve the documentation

* Fill in the recipient's name based on whether or not placeholder keywords are used in the template.

* Implemented a new function to retrieve party details for organizations.

* Extend the lookup logic to check for the recipientNumber keyword

* Improve the members documentation

* Add a new property to whether the template contains any recipient number placeholders

* Improve type documentation

* Eliminate the functions responsible for retrieving the individual name components.

* Allow the user to send in the components of recipient name when creating an order of any type

* Simplify the keywords detection logic

* Remove the RecipientNameComponents type.

* Remove the name components

* Remove the unnecessary ConfigureAwait

* Create a new keyword service to inject actual value wherever a keyword is found

* Update the test data

* Update two functions to retrieve recipient number and recipient name

* Update the sender email address.

* Save the customized email subject and body in the database

* Pass customized body and subject when trying to send Emails again

* Add support for placeholder keywords in SMS

* Improve the keyword replacement logic

* Replace Regex checks with string.Contains

* Improve the request that we use to retrieve unit details.

* Add validation logic to ensure that either organizationNumber or socialSecurityNumber is set at a time.

* Added a constructor for batch creation from existing lists to simplify usage in client code.

* Update the function that is used to retrieve emails

* Fix build errors

* Pass a default value for the new parameters

* Breaking a large method into smaller, more focused private methods

* Make a method static

* Code refactoring

* Code refactoring

* Fix typos

* Use explicit JOIN instead of the implicit join in the FROM clause and improve indentation for better readability.

* Use explicit JOIN instead of the implicit join in the FROM clause and ensure consistent indentation for better readability.

* Use SELECT INTO for assigning values to the __orderid variable and improve indentation for better readability.

* Use SELECT INTO for assigning values to the __orderid variable and improve indentation for better readability.

* Add a script to alter tables

* Undo changes made on the auto generated file and move them to new scripts

* Remove two unnecessary values.

* Add a missing parenthesis

* Remove two unnecessary properties.

* Remove two unnecessary values.

* Enhance the logic to ensure all existing unit tests execute successfully.

* Save and retrieve the customized subject and body to and from the database

* Improve the XML documentation

* Update the keywords service to handle data batches

* Remove the logic used to drop and create both functions and procedures

* Update the Email sending and retry sending logic

* Code refactoring

* Fix the parameter of some unit tests

* Adjust the code to run unit test successfully

* Handle batch of organization and national identity numbers at one

* Update the test units to reflect latest changes

* Improve the readability

* Update the SQL query used to retrieve recipients

* Implement simple test units to text keyword replacement logic

* Update unit tests and test data

* Fix a test unit

* Update the order logic to handle orders sent to recipients without using national identity and organization numbers

* Added test units to test retrieving party details using organization and national identity numbers.

* Allow the user to retrieve party detail for both organization and national identity number in the same request.

* Refactor some test units

* Improve the SMS order processing logic

* Remove the organization number

* Refactor: Remove unused PersonNameComponents type.

* #545 Added tests to increase coverage.

* #545 Validate the constructors and JSON serialization behavior of the PartyDetailsLookupRequest class across diverse scenarios.

* Change the type of a field to improve performance

* Add three test units to cover more use cases

* Remove test cases that check for the nullability of parameters that are defined as non-nullable.

* Replace the national identity number with an empty string

* Do not replace the $recipientNumber$ with the national identity number

* Compare the keywords using ordinal sort rules

* Update target framework to 9.0

* Update NuGet packages and remove unused references

* Update NuGet packages and remove unused references.

* Remove unused references.

* Remove unused references and update NuGet packages

* Remove unused references and update NuGet packages

* Remove unused references and update NuGet packages

* Update the Docker image name.

* Install Confluent.Kafka package

* Install the Microsoft.NET.Test.Sdk package

* Add the minor version number

* Update the .NET framework the GitHub Work-flows target

* Change the tag name

* Increase type safety, readability by prefer the generic overload of Enum.Parse

* Update the .NET version number

* Remove two types.

* Specify the minor version
@Ahmed-Ghanam
Copy link
Contributor

Test Outcome: 15.01.2025

Use Case Tested:

The following notification order was tested:

{
  "sendersReference": "order-email-with-keywords",
  "recipients": [
    {
      "organizationNumber": "313600947"
    },
    {
      "nationalIdentityNumber": "01124801679"
    },
    {
      "emailAddress": "[email protected]"
    }
  ],
  "ignoreReservation": true,
  "notificationChannel": "Email",
  "emailTemplate": {
    "fromAddress": "[email protected]",
    "subject": "Hello $recipientName$! Test Email",
    "body": "Hi $recipientName$, this Email is tailored for $recipientNumber$.",
    "contentType": "Plain"
  }
}

Expected Outcome:

Notifications should be sent via email with keywords $recipientName$ and $recipientNumber$ replaced with recipient-specific data.

Actual Outcome:

The results were as expected:

  1. Recipient - organizationNumber: 313600947
    Outcome Image

  2. Recipient - nationalIdentityNumber: 01124801679
    Outcome Image

  3. Recipient - emailAddress: [email protected]
    Outcome Image

@Ahmed-Ghanam Ahmed-Ghanam self-assigned this Jan 15, 2025
@SandGrainOne
Copy link
Member Author

SandGrainOne commented Jan 15, 2025

  1. Tested lookup for person name and organization name for both email and sms. Looks good.
  2. The keyword $recipientNumber$ is correctly replaced with empty.string when recipient is a person.
  3. The keyword $recipientNumber$ is correctly replaced by the organization number when recipient is an organization.
  4. The customized texts are correctly stored in the notification tables.

@Ahmed-Ghanam
Copy link
Contributor

Ahmed-Ghanam commented Jan 15, 2025

Test Outcome: 15.01.2025

Use Case Tested:

The following notification order was tested:

{
  "sendersReference": "order-sms-keywords",
  "recipients": [
    {
      "mobileNumber": "+4746269164"
    },
    {
      "organizationNumber": "313600947"
    },
    {
      "nationalIdentityNumber": "01124801679"
    }
  ],
  "ignoreReservation": true,
  "notificationChannel": "SMS",
  "smsTemplate": {
    "senderNumber": "+4746269164",
    "body": "Hi $recipientName$, this SMS is customized for $recipientNumber$."
  }
}

Expected Outcome:

Notifications should be sent via SMS with keywords $recipientName$ and $recipientNumber$ replaced with recipient-specific data.

Actual Outcome:

The results were as expected:

  1. Recipient - organizationNumber: 313600947
  2. Recipient - nationalIdentityNumber: 01124801679
  3. Recipient - mobileNumber: +4746269164
    Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature-request New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants