Backend V2.0

This is the official release of backend v2.0, this release include:

• All the features previewed in the beta releases
• Clean up the database (all test and deleted records are removed)
• Clean up publishers types
• Move database migrations to a new system
• Run 2 version of the backend (production and testing - marked by a red bar)
• More form validations have been added
• Introduction to Breadcrumbs
• Add titles to the history (to be enhanced)
• New user and history functions and triggers (to track changes and identify users)

Global Serial Number, Review Status, IDU, Global Lookup

• The previous Serial No. has been removed, it caused a side effect after adding delete to the system (facts and documents fail to save), it's no more generated!
• New "Global Serial No" has been introduced for all entities, it has the format 13 char "TTXXXXXXXXXXX", where TT is the type of entity and the remainder is a timestamp+global counter represented in hexadecimal which allow for 10000 record per second, the type represented by TT is as follows:
  • Communications begins with 'CM'
  • Contacts begins with 'CN'
  • Contextual Analysis begins with 'CA'
  • Documents begins with 'DC'
  • Events begins with 'EV'
  • Facts begins with 'FC'
  • Monitoring Challenges begins with 'MC'
  • Publishers begins with 'PB'
  • Significant Updates begins with 'SU'
• Add a global serial number lookup, users can directly type the gsn in the URL in this format "backend.idmcdb.org/FCDD234DEF1F4", and the system will redirect the user to the required entity.
• Add a legacy redirect link using @ in this format "backend.idmcdb.org/@F2017-AFG-000001"
• Add review status to the fact table (with colors Sylvian)
• Add review status search in fact index page
• Move the IDU toggle to the analysis tab
• Add a new text box to the IDU in the the analysis tab

Fixes

• Add excel document type
• Fix strata data form in facts
• Add IDU toggle flag to facts
• Add a delay after fact submit to insure that the user can't click send twice by mistake
• Temporary remove of the Geolocation component - just to push the required fixes

Bug Fixes

• Date: Date & Accuracy
  fix a bug that allowed the system to store an empty date with accuracy, which caused search views to fail.
• Delete: Fix delete function
  Refactor delete function to properly check for user (moved it as a trigger on references on the database)
• Forms: Update & delete buttons color code
  All "Update" buttons are "Blue", All "Delete" buttons are "Red"
• Quick fact: Delete button confirmation
  Quick fact delete button show a confirmation message now before it deletes the fact.
• Search: Store last search
  Fact, Document and Event search forms remember their last setting on the same browsers (not portable between browsers
• Forms textarea: Autosize up to 36 lines
  All text area fields auto-resize to accommodate their content - min 6 lines, up to 36 lines.
• Events: Added multi-year search field
  Events can now be search by multiple values of years.
• Fact: Reported figure & reported_figure
  Figure & reported_figure are now required fields, they both must be a positive integer and start with value 0.
• Fact: Household Size default & error
  Household Size defaults now to lookup, and shows in red if the value is not found.
• Events: Hazard type selection
  Hazard type can be selected after text search
• Fact: Theme field bug
  Theme had a bug which made it show multiple tags with the same name. Fixed.
• Household Size:
  Updated household size for 2018 are included now.

Fixes to Fact form

• Bug fix: Figures with unit “Household” do not update when country, year or household size change.
• Unit and Term fields in facts are now mandatory fields

Geolocation + November GitHub Issues

• Geolocation component added to the fact, the component store (name, lat, lon) in the database as a JOSN field
• Contact: Fix the gender column duplication issue and size
• Quick Facts: Key fields are no more pre-populated and validation has been added to each field to ensure that the use fill in the required information before submisiion
• Fact: Fix an issue that prevented facts from being stored (this affected both creation and update)
• Date Range: Search forms that include date range have a field hint now to instruct the user to double click a date if a single date selection is needed (to overcome a limitation in the UX library)
• Document: Publishers/Sources field has a search threshold of 3 characters, this prevented something like "UN" from being selected, the characters has been reduced to 2
• Country fields: M.E reported missing countries. Fixed, all country fields use now idmc_short_name instead of the older name field.
• Contact: When creating a new contact an error message keep showing up, Fixed.
• Login: Users can press return when they finish filling in login information to login.
• Document: Upload by drag and drop didn't work, now it does. Fixed.
• Date Select: Restrict "some" date field from choosing a date in the future.
• Country Profile: "Contextual Analysis", "Significant Updates" and "Monitoring Challenges" have now the ability to expand the row (arrow on the left side) to view the content (comments) of the entity. eliminate the need to move away from the country page.
• Link back (multiple): Multiple entities have now a link to their parent entity in the title to go back (instead of adding breadcrumb - for now). e.g. a communication entity will have a link to the contact it relate to.
• Communication: "Conducted At" has been replaced with "Date"
• Contact: A new field "Comment" has been added to contact form.
• Document: File download link has been fixed (prevented the use from open/download the resource)
• Country Profile: "Contextual Analysis", "Significant Updates" and "Monitoring Challenges" now automatically return to the related country once they are created/updated.
• Communication: time has been removed from communication.
• Communication: fix a bug in the communication form which resulted in having the field reset itself.
• Communication: Added "Personal Meeting" as a new tool
• Contact: Remove "Madam" from titles
• Events: Hazard type cascade select no more require user to click to expand categories, it works by hover.
• Documents: Fix a problem in the file upload path
• Backend: Setup configuration file for easy migration
• Forms: Add validation script for all form (validation rules need to be discussed for each entity)
• User session: Increase user session from 30 min. to 2 hours from last request (the system will keep the user logged in 2 hour after their last action)
• Move user authentication/authorization to an API (this will help in iDetect in the future)
• Add a country lookup script (to simplify data retrieval - no joins are necessary)
• Country Profile: Country profiles page URL accept ISO3 codes to redirect to the desired country profile.
• Fact: locations field in the database has been converted into a JSON array + UX tag field (all country data will be converted into iso3 code in the next update)
• Restructure SQL migration files to be by version instead of by function.

New Modules + Fixes

This update introduce new modules, data restructure and fixes.

1. Contacts: Add new modules to manage contacts & country association
2. Communications: Add modules to manage communications associated with a contact & country
3. Portfolios: Add table and component to manage portfolios (on the country profile page)
4. Contextual Analysis: Add new module to the country page
5. Significant Updates: Add new module to the country page
6. Monitoring Challenges: Add new module to the country page
7. Country Profile: include table views to list associated contacts, communications, etc...
8. Country Tag Field: Reworked to store iso3 instead of only country name
9. Country Tag Field: Add a check function to insure valid iso3 from idmc's geo_entities table
10. User session: Modify user session layer to insure that user state persist cross multiple tabs
11. Users: Add function & triggers to assign and check users on create and update
12. Documents: Fix the document view to properly list countries, sources and publishers
13. Events: Events can store multiple glide numbers (+migrate the content of the old field to the new one)
14. ISO3: assign iso3 as a primary key to geo entities and a ISO3 validation function
15. Loading: change database read callbacks to be at loading time (created instead of mounted)
16. association: Rework association component based on user feedback (fix a bug where the number of the record show instead of the title)

Incremental update + Bug fixes

• Top Bar: Add a GitHub link
• Login: Redirect users to a user home page (later to be user activity dashboard)
• Fact Form: Move the Term field just after the figure to put context to the number
• Fact Form: Household Size only be visible when selecting the Unit “household”
• Fact Form: disable "Facts" tab for facts and sub-facts, it's only active with master facts
• Fact Form: Fix a bug in event association field, it didn't show the name of the event although the association is there.
• Country Profile: Fix documents table not showing data
• All forms: Forms will not show before the record is fully loaded from the database
• All index pages: Add a new button to add new entities
• Document Form: Add a new button to add a new fact associated with the document
• Document Form: Fix quick facts not saving
• Tables: fix sorting (function naming conflict)
• Contacts: Add contacts module (moved to the next update)
• Countries: Context + Monitoring analysis module (moved to the next update)
• Login: a user may be logged out by the system if they open more than a browser tab, a fix was introduced to the user session management.

Update on Fact and Document

Fact form

• Master fact association now always show the clear button (bug fix)

Documents:

• Improvements to the document data model, forms and API
• publication_date field is now a date field not timestamp (tech)
• Documents now have an extra checkbox next to the URL field
• Documents will no more grab the content of a URL as file by default
• Users have to check "Attach the content of the URL to the document”
• Users can replace a file by changing the URL and check the "Attach" check box again
• More format are allowed as an attached like JPEG for example (you can thank Luisa)

Facts Index page:

• "Displacement type" and "Fact type" radio boxes have been changed to dropdown fields
• New dropdown for year as a search field has been added
• New dropdown for the "Results Limit” has been added
• Please use carefully "Results Limit”, as choosing the wrong size could mean long download time
• A button has been added to download the search results into an excel sheet file
• The excel file contains extra fields (Adrian and Sylvain will set the final format)
• The excel file also Includes the facts and events URLs
• “Refresh” & “Clear” has been changed to “Search” and “Reset”
• Sub Fact names no more get truncated (you can thank Ivanna)

V2.05

• Review section is back to the facts
• Add Clone function to facts
• Quick Facts: Fix association role
• Facts: Fix document link
• Events: Added the ability to sort event tables by start date
• Fix dates on Facts:
  • Fix: Fact won't save because start/stock date is empty without warning
  • Fix: Required dates don't have a requireds red sign and doesn't show validation error message
• Modifications to the database to accommodate reviews (reviews are now stored separately)
• Remove the floating header temporary as it causes some error with the map