HuggingFaceQaInference EngineException on Longformer models

[xxx24xxx]

I implemented "Question-Answering" using "deepset/bert-base-cased-squad2" model from Huggingface.
The code below works and produces the expected result.

public class HuggingFaceLongformQaInference {
	public static void main(String[] args) throws IOException, TranslateException, ModelException {
		String question = "Where is my house?";
		String paragraph = "My house is in London.";

		QAInput input = new QAInput(question, paragraph);
		String answer = HuggingFaceLongformQaInference.qa_predict(input);
		// --> London
	}

	public static String qa_predict(QAInput input) throws IOException, TranslateException, ModelException {
		Criteria<QAInput, String> criteria = Criteria.builder()
				.setTypes(QAInput.class, String.class)
				.optModelPath(Paths.get("./model/bert-base-cased-squad2/bert-base-cased-squad2.pt"))
				//.optModelPath(Paths.get("./model/longformer-base-4096-finetuned-squadv2/longformer-base-4096-finetuned-squadv2.pt"))
				.optTranslatorFactory(new QuestionAnsweringTranslatorFactory())
				.optEngine("PyTorch")
				.optArgument("includeTokenTypes", "true")
				.optProgress(new ProgressBar())
				.build();

		ZooModel<QAInput, String> model = criteria.loadModel();
		try (Predictor<QAInput, String> predictor = model.newPredictor()) {
			return predictor.predict(input);
		}
	}
}

However, if I change the huggingface model to 'mrm8488/longformer-base-4096-finetuned-squadv2', which is a Longformer model that takes input paragraphs up to 4096 tokens, I get the following error:

Caused by: ai.djl.engine.EngineException: Expected at most 3 argument(s) for operator 'forward', but received 4 argument(s). Declaration: forward(__torch__.transformers.models.longformer.modeling_longformer.LongformerForQuestionAnswering self, Tensor input_ids, Tensor attention_mask) -> Dict(str, Tensor)

How can I solve this problem? Any help is welcome.

[frankfliu]

I think the problem is:

.optArgument("includeTokenTypes", "true")

The model doesn't take token_type_ids as input. If you convert the model with djl-convert, all necessary arguments should be set properly. You only used .optArgument() when you understand the impact.

djl-convert -m mrm8488/longformer-base-4096-finetuned-squadv2