Re-instate .lazy(..) to support Mutually Recursive Schemas?

[lukehesluke]

Support plan

• is this issue currently blocking your project? (yes/no): yes
• is this issue affecting a production system? (yes/no): no

Context

• node version: 14.15.0
• module version: 17.4.0
• environment (e.g. node, browser, native): node
• used with (e.g. hapi application, another framework, standalone, ...):standalone
• any other relevant information:

What problem are you trying to solve?

I would like to use JOI for the validation logic for a set of structural types which include some mutually recursive references.

A limited example just to demonstrate the point:

const X = Joi.object({ a: Y });
const Y = Joi.object({ b: X });

This obviously won't work as Y will be undefined in the first line. This also wouldn't be achievable using .link(..) as .link(..) (as far as I can tell) only works for referencing things within the one schema's boundaries itself. i.e. I initially tried:

const X = Joi.object({ a: Joi.link('#Y') }).id('X');
const Y = Joi.object({ b: Joi.link('#X') }).id('Y');

But, upon, trying to validate something, received the error Error: "a" contains link reference "ref:local:Y" which is outside of schema boundaries. This agrees with the docs here.

In v15, there was a function .lazy(..) which supports this approach. It would allow:

const X = Joi.object({ a: Joi.lazy(() => Y) });
const Y = Joi.object({ b: X });

.lazy(..) was deleted in v16 (release notes)

To help give more context, I'm writing something that generates JOI validation schemas out of schema.org. schema.org models have many mutually recursive references e.g. https://schema.org/Enumeration references https://schema.org/Class (in supersededBy), which references https://schema.org/Enumeration again, etc etc.

Do you have a new or modified API suggestion to solve the problem?

Re-introducing .lazy(..) as it was in v15 (https://joi.dev/api/?v=15.1.1#lazyfn-options---inherits-from-any).

[hueniverse]

Lazy cannot be described and therefore is not going to be added back. Maybe @Marsup has ideas for you no how to implement what you need.

[Marsup]

If you can write it in a single pass, this could be an option.

[lukehesluke]

Thanks @Marsup . This works well for self-recursion, but it doesn't seem to work for the mutual recursion example above. To use schema.org as an example, you can implement the fact that Enumeration can take an Enumeration in its supersededBy field e.g.

const Enumeration = Joi.object({
  supersededBy: Joi.link('/'),
})

But schema.org also allows values of type Class to exist in supersededBy. Class also references Enumeration, which is where the mutual recursion of references comes in. This is not expressible in Joi as far as I can tell:

const Enumeration = Joi.object({
  supersededBy: Joi.alternatives().try(
    Joi.link('/'), // Enumeration
    Class, // This won't work as Class hasn't been defined yet
  ),
});
const Class = Joi.object({
  supersededBy: Joi.alternatives().try(
    Enumeration,
    Joi.link('/'), // Class
  ),
});

For now, I have downgraded to Joi v14, where I'm doing this as follows:

const Enumeration = Joi.object({
  supersededBy: Joi.alternatives().try(
    Joi.lazy(() => Enumeration),
    Joi.lazy(() => Class),
  ),
});
const Class = Joi.object({
  supersededBy: Joi.alternatives().try(
    Joi.lazy(() => Enumeration),
    Joi.lazy(() => Class),
  ),
});

[Josh-Cena]

I'm trying to validate the following structure:

const linkSchema = Joi.object({
  type: 'link',
  target: Joi.string(),
});

const categorySchema = Joi.object({
  type: 'category',
  items: Joi.array().items(itemSchema),
});

const itemSchema = Joi.object().when('.type', {
  switch: [
    {is: 'link', then: linkSchema},
    {is: 'category', then: categorySchema},
  ],
});

The problem is, both categorySchema and itemSchema are reused, so I can't write it in one go (I'm also not sure if link('/') works well in switch clause—I hope yes). Is there another workaround?

/ping @Marsup

[Josh-Cena]

@hueniverse Do you have any idea if the pattern above is supported by Joi now?

[max-kahnt-keylight]

I think I was running into a similar issue when writing a validation generator for OpenApi 3(.1) definitions.
I believe the original error can be resolved as described in #2492:

let X = Joi.object({ a: Joi.link('#Y') }).id('X');
let Y = Joi.object({ b: Joi.link('#X') }).id('Y');
X = X.shared(Y);
Y = Y.shared(X);

It seems to be a bit cumbersome if you want X and Y separated in different modules (I think), since the import interdependecy can only be resolved properly by putting each statement above in its own file (I can give details if anybody is interested or wants to invalidate this claim 😅).

Alternatively, custom (sync) or external (async) could help for this use-case. I played around with the following draft which also seems to work and can probably be ported to external (async) usage with e.g. #2773. My use-case required to carry over artifacts in particular. I haven't yet looked into error handling.

export const lazy = getValidator => (value, helpers) => {
  const validator = getValidator();
  const { state } = helpers;
  const nested = validator.validate(value, { stripUnknown: true });

  if (nested.artifacts) {
    state.mainstay.artifacts = state.mainstay.artifacts ?? new Map();
    for (const [artifact, paths] of nested.artifacts) {
      if (!state.mainstay.artifacts.has(artifact)) {
        state.mainstay.artifacts.set(artifact, []);
      }
      for (const path of paths) {
        state.mainstay.artifacts.get(artifact).push([...state.path, ...path]);
      }
    }
  }

  return nested.value;
};

and

import * as Joi from 'joi';
import { validateIndirect2Schema } from './validate-indirect2-schema';
import { lazy } from './lazy';

export const validateIndirectSchema = Joi.object({
  indirect2: Joi.any()
    .custom(lazy(() => validateIndirect2Schema))
    .allow(null)
    .optional(),
  artifact: (Joi.any() as any).artifact('artifact'),
});

and validate-indirect2-schema accordingly linking back to validate-indirect-schema.

This can probably be put into a custom extension instead, which is also why I don't quite get the "Lazy cannot be described" argument in this respect.