1. First steps

Julgamento is an enviroment to evaluate annotated corpora in the CoNLL-U format. Several different tools are available for the evaluation, as described in Table of contents.

In order to begin an evaluation, first confirm you have already installed Julgamento in a local server by following the 4-steps Tutorial, or Deploying with Ubuntu/Apache. Once you can open the Julgamento homepage, you may want to upload your own corpus. Notice that it is important that the corpora are uploaded strictly through the Julgamento interface, since some additional checks are made within the app context.

First, click in the main menu button "manage corpora". Here you are able to see what corpora are already uploaded to your environment, you can delete them or upload new ones. Usually, the corpus you want to evaluate is a "golden" one, with manual corrections or entirely made by humans, so that you can evaluate if its quality is satisfactory and you can see in which points the annotation must be enhanced. Besides, you can also upload another version of the same corpus annotated by any pos-tagger and parser, if the final output is in the CoNLL-U format. That way, extra evaluations methods will appear, such as confusion matrices, to compare the golden and the system version of the corpus. Some Additional configuration may enable alternative ways of uploading the golden or the system corpora.

Once you upload whether only the golden version or the golden plus the system version, you will be able to go to the Julgamento homepage and start judging the corpus.