Can't show more than one message from a custom function, even though the response is an array

[RobPhippen]

Describe the bug
Custom functions can return an array of objects, but only the first element can be displayed.

To Reproduce

1. Create this function, called functions/fail_func.js

module.exports = (targetVal) => {
    return [
      {
        message: 'Message 1',
      },
      {
        message: 'Message 2',
      },
    ]
};

2. Create this function definition in .spectral.yml

functions:
  - fail_func

rules:
  fail-rule:
    description: Hope to show two errors
    given: $
    severity: error
    message: "{{error}}"
    then:
      function: fail_func

3. Run spectral lint myopenapi.yaml
   Get message

 1:1  error  fail-rule   Message 1

Expected behavior
at step 3, I expect to see both 'Message 1' and 'Message 2'

Comment
If that is not the expected behaviour, then perhaps the response from a custom function should not be an array?

[nulltoken]

Hey! Thanks for this bug report.

This reminds me a lot of a discussion that happened in #920.

Could you see if the explanation in #920 (comment) and the proposed resolution solves the issue you're facing?

[RobPhippen]

Hi @nulltoken many thanks for the response. I have taken a look at #920 .

I am fairly new to spectral so I find it a little hard to decide if the fix proposed to #920 will work for me.

Perhaps it would help if I explain my use-case. We already have a fairly comprehensive set of syntactic rules, and I am working on some custom functions that help to enforce some more 'structural/semantic consistency' rules. This means that, for example, the 'name' of a rest resource is used consistently across;

• resource collection final path segment name (is the plural of the resource name)
• individual resource path parameter name
• resource schema name in #/components/schemas

This means that;

• the rule needs access to $ (specifically, paths: and #/components/schemas:)
• this single consistency rule (as implemented by a custom function) can result in multiple messages

[RobPhippen]

@nulltoken OK, I have done a little more reading, and I think that this comment from #920 is the key proposal:

If it's specified in some path definitions and nothing is set at the root level, then raise an error for each path definition which hasn't been decorated, specifying the inner path of each path definition.

I think this refers to setting the path part of the response from the function, so I would need to have

return [
      {
        message: 'Message 1',
        path: [ ...x... ]
      },
      {
        message: 'Message 2',
        path: [ ...y... ]
      },
    ]

Is that correct? Could you give an example?

I also believe it to be the case that another way for me to work around this would be to use a custom deduplication strategy: https://stoplight.io/p/docs/gh/stoplightio/spectral/docs/guides/javascript.md#using-custom-de-duplication-strategy

[nulltoken]

Is that correct? Could you give an example?

@RobPhippen You're spot on! The typedEnum.ts function might be a good piece of code to glance at.

I've also just opened #1035 to try and document that.

I also believe it to be the case that another way for me to work around this would be to use a custom deduplication strategy: https://stoplight.io/p/docs/gh/stoplightio/spectral/docs/guides/javascript.md#using-custom-de-duplication-strategy

I wouldn't advise you to tweak this function to solve this kind of issue. Returning a different path per result is the way to go.

[RobPhippen]

Hi @nulltoken and thanks for taking the time to look at this - I really appreciate it.

Having read the example from typedEnum.ts I've got to the point where I can make different paths for different messages, and they do indeed show up.

However, the consistency rules I mentioned above really do legitimately result in more than one violation relating to a particular path.

I attempted to cope with this by constructing further 'virtual/logical' path elements. Probably with very good reason, spectral currently rejects those (I assume, because they are 'paths that do not exist in the current document')

Do you have any thoughts on what I could do? In the worst case I will split them in to separate rules.

Reading the new doc, I wonder if it would be possible to allow a function to optionally return a tag with each response - or possibly a sub-tag that would qualify the tag defined at the level of the rule.

[nulltoken]

In the worst case I will split them in to separate rules.

My experience around that would favor this approach. It's usually easier to deal with simple rules that do one thing rather than a big one that performs lots of checks.

Simple rules are usually easier to manage, describe and document.

Also, it happens that for specific use cases you'd like to ignore a specific finding (#939) and dealing with atomic rules makes that also easier.