How the Text2Sparql Model in Tutorial10_Knowledge_Graph was trained

[eboraks]

Hello haystack,

I am interested to learn more about how the Text2Sparql model in the Tutorial10_Knowledge_Graph tutorial was trained. If you can direct me to the right place, I will appreciate it.

Eliran

[julian-risch]

Hi @eboraks the training of such a model is not supported in Haystack but we give some hints how to do it here in the last paragraph at the very bottom of the page: https://haystack.deepset.ai/docs/latest/knowledgegraphmd You would need to train a BART model with transformers to translate text questions to queries in SPARQL format. There are only a few training datasets available to train such a model. One is the LC-QuAD dataset. Hope this helps! 🙂

[eboraks]

Thank you for the response @julian-risch. I ended up reviewing this example - https://github.com/NVIDIA/NeMo/blob/main/tutorials/nlp/Neural_Machine_Translation-Text2Sparql.ipynb to understand how to build Text2Sparql model. I am interested in learning how the Harry Potter Fandom data was created and trained. I understand that this isn't part of the Haystack code, but I just interested to learn how it was done.

[julian-risch]

Let me briefly summarize the main steps that we took to train a Text2SPARQL model on the Harry Potter Fandom data.

1. We started with the LC-QuAD dataset, which includes text questions and corresponding SPARQL queries, for example:

"sparql_wikidata": " select distinct ?obj where { wd:Q127998 wdt:P27 ?obj . ?obj wdt:P31 wd:Q6256 } ",
"paraphrased_question": "Which country is Mahmoud Abbas from?"

2. To better make use of the LC-QuAD dataset, we replaced all numeric identifiers, e.g., wd:Q127998, wdt:P27, wdt:P31, wd:Q6256, with something the model could come up with based on the question text without any other context. To this end, we looked up the name that refers to each identifier and created a dictionary in the form {"P35": "head of state", "Q127998": "Mahmoud Abbas", "P31": "instance of", "Q6256": "country", "Q1045": "Somalia", "P1082": "population", .... Using this dictionary and the LC-QuAD dataset leads to the transformed training sample:

"sparql_wikidata": "select distinct ?a { hp:Mahmoud_abbas hp:nationality ?a . ?a hp:instance_of hp:Country }"
"paraphrased_question": "Which country is Mahmoud Abbas from?"

We also mapped some of the names so that they better fit what we use in the Fandom dataset and our graph, e.g., "country_of_citizenship" -> "nationality" or "characters" -> "harrypotterrole".
4. We pre-trained the model on this transformed LC-QuAD dataset.
5. Finally, we fine-tuned the model on a small set of training samples from the Harry Potter Fandom data. These samples were manually created. This step is more important for learning to map relations correctly. The mapping of the name "Harry Potter" in the text question to "hp:Harry_potter" in SPARQL should already work quite well even before this step.

[julian-risch]

Hi afermi, I used code similar to the following lines. Note that the code builds a cache so that the number of request to wikidata.org is reduced.

import json
import urllib.request as urllib
from bs4 import BeautifulSoup

cache = dict()
# initiliaze the cache with the file provided in s3 to drastically speedup the process by preventing web requests
#cache = json.load(open("lcquad_identifier_to_label_cache.json"))

def identifier_to_label(token):
	if not token.startswith("wd:Q") and not token.startswith("wdt:P") and not token.startswith("ps:P") and not token.startswith("pq:P") and not token.startswith("p:P"):
		return token
	identifier = token.split(":")[1].rstrip("}")
	if identifier[0] == "Q":
		url = "https://www.wikidata.org/wiki/"+identifier
	elif identifier[0] == "P":
		url = "https://www.wikidata.org/wiki/Property:"+identifier

	if identifier in cache:
		return cache[identifier]
	try:
		soup = BeautifulSoup(urllib.urlopen(url), "lxml")
		title = soup.title.string.split(" - Wikidata")[0]
		cache[identifier] = title
		return title
	except Exception:
		print(f"Could not open {url}")
		cache[identifier] = ""
		return ""

[julian-risch]

@afermi see above 🙂

[afermi]

Hi @julian-risch,
thank you very much for your answer!
It is really very helpful to me!

[IliaGamov]

Hi @julian-risch, It's been a long time, but there are still questions,
What if i'd like to use text2sparql using different language (German or Russian), i'll have to create dataset that maps "Text" with it's "Sparql query"?

[julian-risch]

Hi @IliaGamov interesting project idea! 👍 Yes, that's right. You would need to create a new dataset that maps German or Russian text questions to their SPARQL query. As a simple start, you could use machine translation on the English LC-QuAD dataset to translate the English questions to German or Russian.