Generate OpenSearch OpenAPI Spec from Legacy JsonSchema spec

[nhtruong]

This topic originated from this discussion on OpenSearch-Java

I've generated the OpenAPI spec from the Legacy JsonSchema:

• YAML
• JSON

(Note that the OpenApi Spec conversion is only 95% done. Still need to deal with some edge cases regarding reusable query parameters)

A few things I've learned while working on this conversion and the Ruby Code Generation POC:

1. Endpoint Grouping

Take a look at this excerpt of the legacy index.json file:

{
  "index":{
    "documentation":{
      "url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-index_.html",
      "description":"Creates or updates a document in an index."
    },
    "stability":"stable",
    "url":{
      "paths":[
        {
          "path":"/{index}/_doc/{id}",
          "methods":[
            "PUT",
            "POST"
          ],
          "parts":{
            "id":{
              "type":"string",
              "description":"Document ID"
            },
            "index":{
              "type":"string",
              "description":"The name of the index"
            }
          }
        },
        {
          "path":"/{index}/_doc",
          "methods":[
            "POST"
          ],
          "parts":{
            "index":{
              "type":"string",
              "description":"The name of the index"
            }
          }
        }
      ]
    },
    "params":{...},
    "body":{
      "description":"The document",
      "required":true
    }
  }
}

This spec when translated to OpenAPI (as well as Smithy) will result in 3 different endpoints:

• PUT /{index}/_doc/{id}
• POST /{index}/_doc/{id}
• POST /{index}/_doc

Since {id} is a path parameter, it's always required. Hence, we can't combine POST /{index}/_doc/{id} and POST /{index}/_doc into POST /{index}/_doc/{id} with optional {id}.

The clients combine these 3 endpoints into 1 action. For example, this is how Ruby handles it:

def index(arguments = {})
  raise ArgumentError, "Required argument 'body' missing" unless arguments[:body]
  raise ArgumentError, "Required argument 'index' missing" unless arguments[:index]

  headers = arguments.delete(:headers) || {}

  arguments = arguments.clone

  _id = arguments.delete(:id)

  _index = arguments.delete(:index)

  method = _id ? OpenSearch::API::HTTP_PUT : OpenSearch::API::HTTP_POST
  path   = if _index && _id
             "#{Utils.__listify(_index)}/_doc/#{Utils.__listify(_id)}"
           else
             "#{Utils.__listify(_index)}/_doc"
           end
  params = Utils.__validate_and_extract_params arguments, ParamsRegistry.get(__method__)

  body = arguments[:body]
  perform_request(method, path, params, body, headers).body
end

As you can see, it treats id as an optional parameter. When id is present, it hits PUT /{index}/_doc/{id} endpoint, and when id is absent, it hits POST /{index}/_doc endpoint.

This means we must provide this grouping information for the client code generator to combine these endpoints into one action. I did so through the x-endpoint-group extension for the generated spec mentioned above.

2. Namespace

In most clients, the API actions are divided into namespaces like cat, indices, and cluster. For example, here's the file structure of the Ruby client:

The user invokes these actions through their namespaces:

client.cat.health
client.ingest.delete_pipeline(...)
client.count(...)
client.index(...)

This namespace info is also provided by the legacy spec through each file's name and the top level key like cat.health.json. Again I add x-endpoint-group (e.g. "x-endpoint-group": "cat.health") for each endpoint to store this info in the rendered OpenAPI spec.

3. Comma Separated List of String Values

Many path parameters like index and alias sometimes accept a list of values presented as a string like indexA,indexB,indexC. The clients are aware of this and allows the users to pass in an array for such a parameter. The legacy spec denotes this as type: list. This datatype is translated the following OpenAPI Schema:

string_array:
  type: array
  items:
    type: string
  minItems: 1

Note that the current smithy spec is missing this bit of info (i.e. whether the client should accept an array then turn it into a string)

[nhtruong]

This has already been done. Closing