Prevent publishing if duplicate identifier_field

[TidyIQ]

Is your feature request related to a problem? Please describe.

Identifier fields should be unique. Duplicates should not be allowed to be published.

Describe the solution you'd like

Throw error if attempting to publish with an identifier field value that already exists.

Describe alternatives you've considered

There are no easy to use alternatives.

Additional context

I'm using a relation widget in a collection where the display/search/value field is set to the identifier_field of the collection it's related to. If a user has duplicate identifier_fields in that collection then the relation widget is useless as I can't lookup the correct field (since I have no idea which duplicate is the correct one).

Also, I'm unable to access the value of a field with duplicate identifier_fields as I have no idea which one is correct.

ID fields should always be unique. It doesn't make sense to allow duplicates.

[vai0]

@TidyIQ have you found a solution to this? I'm experiencing the same issue - it doesn't stop me from publishing another entry with the same identifier field value. I'm using a gitlab backend

[erquhart]

I thought we fixed this in the past, might be a regression.

cc/ @erezrokah

[erezrokah]

@TidyIQ, @vai0 can you share your config.yml so I can try to create a reproduction?
The CMS should generate a unique slug for each entry (postfixing an index in case of collisions).

[erezrokah]

Closing this, please re-open with more information if still experiencing the issue

[vai0]

@erezrokah

Here is my config.yml

- label: "LP Event Activity"
  name: "lp-event-activity"
  folder: "frontend/src/content/lp-event-activity"
  create: true
  description: "Create an activity event landing page"
  extension: "yml"
  identifier_field: slug
  fields:
    - label: "slug"
      name: "slug"
      widget: "string"
      hint: "/events/[slug]. e.g., strata-2019"
    - ...
...
backend:
  name: "gitlab"
  branch: "production"
  repo: "engineering/bifrost"
  auth_type: "implicit"
  app_id: ...
  api_root: .../api/v4
  base_url: ...
  auth_endpoint: "oauth/authorize"

Using

• gatsby-plugin-netlify-cms: 4.1.33
• netlify-cms-app: 2.10.1
• backend: gitlab

Steps to reproduce

1. Click on an existing entry in folder collection
2. Click "duplicate" in submenu
   
3. Entry is successfully duplicated, with a unique filename - however the slug, which is set as the identifer_field is same across both entries in gitlab. Below is a screenshot of the duplicated entry in gitlab.
   

What should ideally happen is the identifier field value should change after duplication (e.g., [original-slug]-1), and if the user attempts to publish, it should prevent the commit and instead throw an error, something like Entry could not be published - [slug-field-value] already exists, [identifier_field] must be unique

[barthc]

@vai0 So if the identifier_field is the title string field, the CMS should also prevent commit if the duplicate title is the same with the original post?

[traianrdev6]

I am also having this issue. We are trying to have comments that live in a database to be tied to a post so we need a unique identifier on the posts and we are seeing that the default identifier_field 'title' is allowing duplicates. We can't find any way to work around this either, if anyone knows a workaround that would be great.

[saschatu]

I faced the same issue and solved the Id creation issue like mentioned in this post at the end (widget generates ID, not editable for user - used as identifier field instead of title): https://jozefm.dev/articles/2019/06/18/gridsome-netlify-cms-collection-relations/

BUT you must not use duplicate - there I am stuck at the moment because there is no way to identify this situation within the widget. It would be great if the identifier field would not be duplicated - if it would be just empty the component update could fill the field with new ID

[vai0]

@saschatu we've used this same exact uuid solution for some of our collections, but for others collections where we'd like the user to add a slug, and have that slug be what's unique - since we don't want one page overwriting another, the auto-uuid generation doesn't really work :/

[saschatu]

Hi @vai0 ,
I agree with you and see the general need for defining unique fields - but would it be possible to use the ID as slug? So nothing "user defined" is the point. Checking that something manually defined by the user (like a title) is unqiue is probably more tricky from a technical point of view because you would need to pull all collection files from the repo and check the existing values. Maybe it is easier to rely on generation of UUIDs because you don't need to check if they are unique.

But also with a generated ID I did not manage yet to ensure that after duplication a new ID is stored :( - issue #3205
UPDATE: custom ID widget handling duplicate as well is solved - see #3205

[erezrokah]

This #1348 might serve as a solution once we enable modifying entry data in the hooks

[vai0]

@saschatu

Unfortunately - using ID's or appending them to the slug would be a no-go given the importance of SEO for our use-case.

Also - another big downside of setting a generated ID as the identifer_field is that it makes for a poor UX - a collection page displaying its entries would just be a list of uid's instead of something descriptive.

We can work a temp solution around a unique slug by setting a generated uid as the identifier field, and do a check for unique slugs when generating our static site; however - it'd be amazing if

1. it errors and prevents the commit if it finds another entry in the collection with the same identifier
2. if implementing 1 is too hard - there was an option in config.yml to set a field as the entry's display name that appears in the page that renders all the entries in a collection - this would mitigate the UX issue of an unintelligible list of random uid's.

this issue (along with a couple others, e.g., sorting entries by date / slow search) is what's holding us back from migrating our blog to netlify-cms

[vai0]

@vai0 So if the identifier_field is the title string field, the CMS should also prevent commit if the duplicate title is the same with the original post?

@barthc yes - IMO its an expectation that the identifier field be unique across all entries in the same collection.

[saschatu]

@vai0 check the summary field (https://www.netlifycms.org/docs/configuration-options/) its purpose is defining the text to be shown in the list ;)

[lifeofbrian]

It seems sensible to add a native uuid widget to Netlify CMS to help here.

[absentees]

@saschatu thanks for your link. Is this still the only way to achieve a relationship using Gridsome (and Gatsby, which i'm not as familiar but looks like it also needs an ID)?

[erezrokah]

Updates

1. You can now use the slug template variable in the relation widget which I think is preferred over an id widget (since it doesn't require an additional field on the referenced collection). Regardless we can track having an id widget in this dedicated issue.
2. As for auto generating a slug field when saving an entry, you can now use the preSave event to do so https://www.netlifycms.org/docs/beta-features/#registering-to-cms-events

As the original issue was to avoid duplication when using the relation widget, I think that issue is resolved.

@vai0 and others, if your use case is still relevant either comment (if it's the same as the original issue) or open a new issue (if different than the original issue).

[saschatu]

@erezrokah
Thanks for the update!
Regarding Slug: nice feature, but it is easier to have an explicit ID as a field (for e.g. a gridsome parser) but Slug is only persisted as file name. Which makes the whole entity creation with ID + referencing in relation more complicated.

However, regarding CMS Events: nice, but I miss some Event related to Duplication. That was the only open point regarding the ID: if an entity is duplicated we somehow need an option to avoid duplication of ID! But at the same time this ID must not change for the same entity (because relations will be broken). If I create the uuid during preSave, the entity get's a new ID with each save. Ok, checking for empty value would prevent this - but the "duplicate" case will lead to duplicate ID as well :(

EDIT: just realized that the slug is always unique, even in duplicate case (-1 is added in slug for first duplicate) -> now I get the point of using slug as identifier!
To ensure that the ID always stay the same you can than use

1. the preSave method: generate id for a field (e.g. "generated_id" ) not being listed in yaml BUT you can use it for slug generation
2. Or even easier using the hidden widget as field for slug generatin: default is a constant value and Slug creation simply numerates them :)
3. use {{year}}{{month}}{{day}}{{hour}}{{minute}}{{second}} as slug - always creation dateTime. It changes with duplication

[sleepingkiwi]

For anyone who finds this through search like I did - a good way to ensure unique IDs are generated for duplicates during preSave is to check for the existence of entry.path as well as the existence of an existing uid.

Brand new entries (i.e. a duplicate) don't have a path during their first preSave.

I ended up using something like this:

CMS.registerEventListener({
  name: 'preSave',
  handler: ({ entry }) => {
    // path is blank for brand new entries
    const path = entry.get('path');
    
    let uid = entry.get('data').get('uid');

    if (!uid || !path) {
      uid = someMethodOfGeneratingUID();
    }

    return entry.get('data').set('uid', uid);
  },
});