Implement graph query in API v2 (#1009)

* feature (api-v2): Refactor v1 graph query code for v2.

* test (api-v2): Refactor v1 graph tests for v2.

* feature (api-v2): Represent graph query results as normal JSON-LD graphs (ongoing).

* test (api-v2): Add graph query e2e tests.

- Add configurable limits to graph queries.
- Make results deterministic for tests.

* docs (api-v2): Document graph route.

- Clean up code a bit.

* style (api-v2): Clean up code a bit more.

docs/src/paradox/00-release-notes/v2.x.x.md

@@ -31,14 +31,16 @@ description.
 - trouble with xml-checker and/or consistency-checker during bulk import (@github[#978](#978))
 - ontology API error with link values (@github[#988](#988))
 
-## v2.0.1 (not release yet)
+## v2.0.1 (not released yet)
 
 ### Bugfixes:
 
 - sipi container config / sipi not able to talk to knora (@github[#988](#994))
 
-## v2.1.0 (not release yet)
+## v2.1.0 (not released yet)
 
 ### New features:
 
-### Bugfixes:
+- Implement graph query in API v2 (@github[#1009](#1009))
+
+### Bugfixes:

docs/src/paradox/03-apis/api-v2/reading-and-searching-resources.md

@@ -100,6 +100,80 @@ the path segment `resourcespreview`:
 HTTP GET to http://host/v2/resourcespreview/resourceIRI(/anotherResourceIri)*
 ```
 
+## Get a Graph of Resources
+
+Knora can return a graph of connections between resources, e.g. for generating
+a network diagram.
+
+```
+HTTP GET to http://host/v2/graph/resourceIRI[depth=Integer]
+[direction=outbound|inbound|both][excludeProperty=propertyIri]
+```
+
+The first parameter must be preceded by a question mark `?`, any
+following parameter by an ampersand `&`.
+
+- `depth` must be at least 1. The maximum depth is an Knora configuration setting.
+  The default is 4.
+- `direction` specifies the direction of the links to be queried, i.e. links to
+  and/or from the given resource. The default is `outbound`.
+- `excludeProperty` is an optional link property to be excluded from the
+  results.
+
+To accommodate large graphs, the graph response format is very concise, and is therefore
+simpler than the usual resources response format. Each resource represented only by its IRI,
+class, and label. Direct links are shown instead of link values. For example:
+
+```jsonld
+{
+  "@graph" : [ {
+    "@id" : "http://rdfh.ch/0001/0C-0L1kORryKzJAJxxRyRQ",
+    "@type" : "anything:Thing",
+    "rdfs:label" : "Sierra"
+  }, {
+    "@id" : "http://rdfh.ch/0001/A67ka6UQRHWf313tbhQBjw",
+    "@type" : "anything:Thing",
+    "rdfs:label" : "Victor"
+  }, {
+    "@id" : "http://rdfh.ch/0001/Lz7WEqJETJqqsUZQYexBQg",
+    "@type" : "anything:Thing",
+    "rdfs:label" : "Foxtrot"
+  }, {
+    "@id" : "http://rdfh.ch/0001/WLSHxQUgTOmG1T0lBU2r5w",
+    "@type" : "anything:Thing",
+    "anything:hasOtherThing" : {
+      "@id" : "http://rdfh.ch/0001/A67ka6UQRHWf313tbhQBjw"
+    },
+    "rdfs:label" : "Tango"
+  }, {
+    "@id" : "http://rdfh.ch/0001/start",
+    "@type" : "anything:Thing",
+    "anything:hasOtherThing" : [ {
+      "@id" : "http://rdfh.ch/0001/0C-0L1kORryKzJAJxxRyRQ"
+    }, {
+      "@id" : "http://rdfh.ch/0001/WLSHxQUgTOmG1T0lBU2r5w"
+    }, {
+      "@id" : "http://rdfh.ch/0001/tPfZeNMvRVujCQqbIbvO0A"
+    } ],
+    "rdfs:label" : "Romeo"
+  }, {
+    "@id" : "http://rdfh.ch/0001/tPfZeNMvRVujCQqbIbvO0A",
+    "@type" : "anything:Thing",
+    "anything:hasOtherThing" : {
+      "@id" : "http://rdfh.ch/0001/Lz7WEqJETJqqsUZQYexBQg"
+    },
+    "rdfs:label" : "Echo"
+  } ],
+  "@context" : {
+    "rdf" : "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
+    "knora-api" : "http://api.knora.org/ontology/knora-api/v2#",
+    "rdfs" : "http://www.w3.org/2000/01/rdf-schema#",
+    "xsd" : "http://www.w3.org/2001/XMLSchema#",
+    "anything" : "http://0.0.0.0:3333/ontology/0001/anything/v2#"
+  }
+}
+```
+
 ## Search for Resources
 
 ### Search for a Resource by its `rdfs:label`
@@ -122,8 +196,10 @@ specific. The first term should at least contain four characters. To
 make this kind of "search as you type" possible, a wildcard character is
 automatically added to the last search term.
 
-    HTTP GET to http://host/v2/searchbylabel/searchValue[limitToResourceClass=resourceClassIRI]
-    [limitToProject=projectIRI][offset=Integer]
+```
+HTTP GET to http://host/v2/searchbylabel/searchValue[limitToResourceClass=resourceClassIRI]
+[limitToProject=projectIRI][offset=Integer]
+```
 
 The first parameter must be preceded by a question mark `?`, any
 following parameter by an ampersand `&`.

webapi/src/main/resources/application.conf

@@ -332,6 +332,11 @@ app {
         },
         fulltext-search {
             search-value-min-length = 3
+        },
+        graph-route {
+            default-graph-depth = 4
+            max-graph-depth = 10
+            max-graph-breadth = 50
         }
     }
 

webapi/src/main/scala/org/knora/webapi/Settings.scala

@@ -127,10 +127,13 @@ class SettingsImpl(config: Config) extends Extension {
     val defaultIconSizeDimX: Int = config.getInt("app.gui.default-icon-size.dimX")
     val defaultIconSizeDimY: Int = config.getInt("app.gui.default-icon-size.dimY")
 
-
     val v2ResultsPerPage: Int = config.getInt("app.v2.resources-sequence.results-per-page")
     val searchValueMinLength: Int = config.getInt("app.v2.fulltext-search.search-value-min-length")
 
+    val defaultGraphDepth: Int = config.getInt("app.v2.graph-route.default-graph-depth")
+    val maxGraphDepth: Int = config.getInt("app.v2.graph-route.max-graph-depth")
+    val maxGraphBreadth: Int = config.getInt("app.v2.graph-route.max-graph-breadth")
+
     val triplestoreType: String = config.getString("app.triplestore.dbtype")
     val triplestoreHost: String = config.getString("app.triplestore.host")
 

webapi/src/main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/KnoraApiV2Simple.scala

@@ -571,43 +571,6 @@ object KnoraApiV2Simple {
         )
     }
 
-    /**
-      * Makes a [[ReadClassInfoV2]] representing an owl:Class.
-      *
-      * @param classIri               the IRI of the class.
-      * @param subClassOf             the set of direct superclasses of this class.
-      * @param predicates             the predicates of the class.
-      * @param directCardinalities    the direct cardinalities of the class.
-      * @param inheritedCardinalities the inherited cardinalities of the class.
-      * @return a [[ReadClassInfoV2]].
-      */
-    private def makeClass(classIri: IRI,
-                          subClassOf: Set[IRI] = Set.empty[IRI],
-                          predicates: Seq[PredicateInfoV2] = Seq.empty[PredicateInfoV2],
-                          directCardinalities: Map[IRI, Cardinality.Value] = Map.empty[IRI, Cardinality.Value],
-                          inheritedCardinalities: Map[SmartIri, KnoraCardinalityInfo] = Map.empty[SmartIri, KnoraCardinalityInfo]): ReadClassInfoV2 = {
-
-        val rdfType = OntologyConstants.Rdf.Type.toSmartIri -> PredicateInfoV2(
-            predicateIri = OntologyConstants.Rdf.Type.toSmartIri,
-            objects = Seq(SmartIriLiteralV2(OntologyConstants.Owl.Class.toSmartIri))
-        )
-
-        ReadClassInfoV2(
-            entityInfoContent = ClassInfoContentV2(
-                classIri = classIri.toSmartIri,
-                predicates = predicates.map {
-                    pred => pred.predicateIri -> pred
-                }.toMap + rdfType,
-                directCardinalities = directCardinalities.map {
-                    case (propertyIri, cardinality) => propertyIri.toSmartIri -> KnoraCardinalityInfo(cardinality)
-                },
-                subClassOf = subClassOf.map(_.toSmartIri),
-                ontologySchema = ApiV2Simple
-            ),
-            inheritedCardinalities = inheritedCardinalities
-        )
-    }
-
     /**
       * Makes a [[ReadClassInfoV2]] representing an rdfs:Datatype.
       *

webapi/src/main/scala/org/knora/webapi/messages/v2/responder/ontologymessages/KnoraApiV2WithValueObjects.scala

@@ -1577,49 +1577,4 @@ object KnoraApiV2WithValueObjects {
             isLinkValueProp = isLinkValueProp
         )
     }
-
-    /**
-      * Makes a [[ReadClassInfoV2]].
-      *
-      * @param classIri               the IRI of the class.
-      * @param subClassOf             the set of direct superclasses of this class.
-      * @param predicates             the predicates of the class.
-      * @param isResourceClass        true if this is a subclass of `knora-api:Resource`.
-      * @param canBeInstantiated      true if this is a Knora resource class that can be instantiated via the Knora API.
-      * @param directCardinalities    the direct cardinalities of the class.
-      * @param inheritedCardinalities the inherited cardinalities of the class.
-      * @return a [[ReadClassInfoV2]].
-      */
-    private def makeClass(classIri: IRI,
-                          subClassOf: Set[IRI] = Set.empty[IRI],
-                          predicates: Seq[PredicateInfoV2] = Seq.empty[PredicateInfoV2],
-                          isResourceClass: Boolean = false,
-                          canBeInstantiated: Boolean = false,
-                          isValueClass: Boolean = false,
-                          directCardinalities: Map[IRI, Cardinality.Value] = Map.empty[IRI, Cardinality.Value],
-                          inheritedCardinalities: Map[SmartIri, KnoraCardinalityInfo] = Map.empty[SmartIri, KnoraCardinalityInfo]): ReadClassInfoV2 = {
-        val rdfType = OntologyConstants.Rdf.Type.toSmartIri -> PredicateInfoV2(
-            predicateIri = OntologyConstants.Rdf.Type.toSmartIri,
-            objects = Seq(SmartIriLiteralV2(OntologyConstants.Owl.Class.toSmartIri))
-        )
-
-        ReadClassInfoV2(
-            entityInfoContent = ClassInfoContentV2(
-                classIri = classIri.toSmartIri,
-                predicates = predicates.map {
-                    pred => pred.predicateIri -> pred
-                }.toMap + rdfType,
-                directCardinalities = directCardinalities.map {
-                    case (propertyIri, cardinality) => propertyIri.toSmartIri -> KnoraCardinalityInfo(cardinality)
-                },
-                subClassOf = subClassOf.map(iri => iri.toSmartIri),
-                ontologySchema = ApiV2WithValueObjects
-            ),
-            inheritedCardinalities = inheritedCardinalities,
-            canBeInstantiated = canBeInstantiated,
-            isResourceClass = isResourceClass,
-            isValueClass = isValueClass
-        )
-    }
-
 }

webapi/src/main/scala/org/knora/webapi/messages/v2/responder/resourcemessages/ResourceMessagesV2.scala

@@ -536,7 +536,6 @@ case class ReadResourcesSequenceV2(numberOfResources: Int, resources: Seq[ReadRe
         toOntologySchema(targetSchema).generateJsonLD(targetSchema, settings)
     }
 
-
     /**
       * Checks that a [[ReadResourcesSequenceV2]] contains exactly one resource, and returns that resource. If the resource
       * is not present, or if it's `ForbiddenResource`, throws an exception.
@@ -562,3 +561,143 @@ case class ReadResourcesSequenceV2(numberOfResources: Int, resources: Seq[ReadRe
         resourceInfo
     }
 }
+
+/**
+  * Requests a graph of resources that are reachable via links to or from a given resource. A successful response
+  * will be a [[GraphDataGetResponseV2]].
+  *
+  * @param resourceIri     the IRI of the initial resource.
+  * @param depth           the maximum depth of the graph, counting from the initial resource.
+  * @param inbound         `true` to query inbound links.
+  * @param outbound        `true` to query outbound links.
+  * @param excludeProperty the IRI of a link property to exclude from the results.
+  * @param requestingUser  the user making the request.
+  */
+case class GraphDataGetRequestV2(resourceIri: IRI,
+                                 depth: Int,
+                                 inbound: Boolean,
+                                 outbound: Boolean,
+                                 excludeProperty: Option[SmartIri],
+                                 requestingUser: UserADM) extends ResourcesResponderRequestV2 {
+    if (!(inbound || outbound)) {
+        throw BadRequestException("No link direction selected")
+    }
+}
+
+/**
+  * Represents a node (i.e. a resource) in a resource graph.
+  *
+  * @param resourceIri      the IRI of the resource.
+  * @param resourceLabel    the label of the resource.
+  * @param resourceClassIri the IRI of the resource's OWL class.
+  */
+case class GraphNodeV2(resourceIri: IRI, resourceClassIri: SmartIri, resourceLabel: String) extends KnoraReadV2[GraphNodeV2] {
+    override def toOntologySchema(targetSchema: ApiV2Schema): GraphNodeV2 = {
+        copy(resourceClassIri = resourceClassIri.toOntologySchema(targetSchema))
+    }
+}
+
+/**
+  * Represents an edge (i.e. a link) in a resource graph.
+  *
+  * @param source      the resource that is the source of the link.
+  * @param propertyIri the link property that links the source to the target.
+  * @param target      the resource that is the target of the link.
+  */
+case class GraphEdgeV2(source: IRI, propertyIri: SmartIri, target: IRI) extends KnoraReadV2[GraphEdgeV2] {
+    override def toOntologySchema(targetSchema: ApiV2Schema): GraphEdgeV2 = {
+        copy(propertyIri = propertyIri.toOntologySchema(targetSchema))
+    }
+}
+
+/**
+  * Represents a graph of resources.
+  *
+  * @param nodes the nodes in the graph.
+  * @param edges the edges in the graph.
+  */
+case class GraphDataGetResponseV2(nodes: Seq[GraphNodeV2], edges: Seq[GraphEdgeV2], ontologySchema: OntologySchema) extends KnoraResponseV2 with KnoraReadV2[GraphDataGetResponseV2] {
+    private def generateJsonLD(targetSchema: ApiV2Schema, settings: SettingsImpl): JsonLDDocument = {
+        implicit val stringFormatter: StringFormatter = StringFormatter.getGeneralInstance
+
+        val sortedNodesInTargetSchema: Seq[GraphNodeV2] = nodes.map(_.toOntologySchema(targetSchema)).sortBy(_.resourceIri)
+        val edgesInTargetSchema: Seq[GraphEdgeV2] = edges.map(_.toOntologySchema(targetSchema))
+
+        // Make JSON-LD prefixes for the project-specific ontologies used in the response.
+
+        val resourceOntologiesUsed: Set[SmartIri] = sortedNodesInTargetSchema.map(_.resourceClassIri.getOntologyFromEntity).toSet.filter(!_.isKnoraBuiltInDefinitionIri)
+        val propertyOntologiesUsed: Set[SmartIri] = edgesInTargetSchema.map(_.propertyIri.getOntologyFromEntity).toSet.filter(!_.isKnoraBuiltInDefinitionIri)
+        val projectSpecificOntologiesUsed = resourceOntologiesUsed ++ propertyOntologiesUsed
+
+        // Make the knora-api prefix for the target schema.
+
+        val knoraApiPrefixExpansion = targetSchema match {
+            case ApiV2Simple => OntologyConstants.KnoraApiV2Simple.KnoraApiV2PrefixExpansion
+            case ApiV2WithValueObjects => OntologyConstants.KnoraApiV2WithValueObjects.KnoraApiV2PrefixExpansion
+        }
+
+        // Make the JSON-LD context.
+
+        val context = JsonLDUtil.makeContext(
+            fixedPrefixes = Map(
+                "rdf" -> OntologyConstants.Rdf.RdfPrefixExpansion,
+                "rdfs" -> OntologyConstants.Rdfs.RdfsPrefixExpansion,
+                "xsd" -> OntologyConstants.Xsd.XsdPrefixExpansion,
+                OntologyConstants.KnoraApi.KnoraApiOntologyLabel -> knoraApiPrefixExpansion
+            ),
+            knoraOntologiesNeedingPrefixes = projectSpecificOntologiesUsed
+        )
+
+        // Group the edges by source IRI and add them to the nodes.
+
+        val groupedEdges: Map[IRI, Seq[GraphEdgeV2]] = edgesInTargetSchema.groupBy(_.source)
+
+        val nodesWithEdges: Seq[JsonLDObject] = sortedNodesInTargetSchema.map {
+            node: GraphNodeV2 =>
+                // Convert the node to JSON-LD.
+                val jsonLDNodeMap = Map(
+                    JsonLDConstants.ID -> JsonLDString(node.resourceIri),
+                    JsonLDConstants.TYPE -> JsonLDString(node.resourceClassIri.toString),
+                    OntologyConstants.Rdfs.Label -> JsonLDString(node.resourceLabel)
+                )
+
+                // Is this node the source of any edges?
+                groupedEdges.get(node.resourceIri) match {
+                    case Some(nodeEdges: Seq[GraphEdgeV2]) =>
+                        // Yes. Convert them to JSON-LD and add them to the node.
+
+                        val nodeEdgesGroupedAndSortedByProperty: Vector[(SmartIri, Seq[GraphEdgeV2])] = nodeEdges.groupBy(_.propertyIri).toVector.sortBy(_._1)
+
+                        val jsonLDNodeEdges: Map[IRI, JsonLDArray] = nodeEdgesGroupedAndSortedByProperty.map {
+                            case (propertyIri: SmartIri, propertyEdges: Seq[GraphEdgeV2]) =>
+                                val sortedPropertyEdges = propertyEdges.sortBy(_.target)
+                                propertyIri.toString -> JsonLDArray(sortedPropertyEdges.map(propertyEdge => JsonLDUtil.iriToJsonLDObject(propertyEdge.target)))
+                        }.toMap
+
+                        JsonLDObject(jsonLDNodeMap ++ jsonLDNodeEdges)
+
+                    case None =>
+                        // This node isn't the source of any edges.
+                        JsonLDObject(jsonLDNodeMap)
+                }
+        }
+
+        // Make the JSON-LD document.
+
+        val body = JsonLDObject(Map(JsonLDConstants.GRAPH -> JsonLDArray(nodesWithEdges)))
+
+        JsonLDDocument(body = body, context = context)
+    }
+
+    override def toJsonLDDocument(targetSchema: ApiV2Schema, settings: SettingsImpl): JsonLDDocument = {
+        toOntologySchema(targetSchema).generateJsonLD(targetSchema, settings)
+    }
+
+    override def toOntologySchema(targetSchema: ApiV2Schema): GraphDataGetResponseV2 = {
+        GraphDataGetResponseV2(
+            nodes = nodes.map(_.toOntologySchema(targetSchema)),
+            edges = edges.map(_.toOntologySchema(targetSchema)),
+            ontologySchema = targetSchema
+        )
+    }
+}