Why there are repeated matches?

[wjfjw]

Hello,
I can successfully run the deduplicate code with Duke, but the matched two records repeat twice in the match result, why?
For example, if records of ID1 and ID2 match, there will be two matches. The first one is ID1 and ID2, and the second one is ID2 and ID1. They are the same match despite of the different order.
I am looking forward to your reply. Thank you!

Here is the XML Configuration code:
`

0.8

<property type="id">
  <name>ID</name>
</property>

<property>
  <name>URL</name>
  <comparator>StringComparator</comparator>
  <low>0.49</low>
  <high>0.9</high>
</property>
<property>
  <name>TITLE</name> 
  <comparator>StringComparator</comparator>
  <low>0.3</low>
  <high>0.7</high>
</property>    
<property>
  <name>SOURCE</name> 
  <comparator>StringComparator</comparator>
  <low>0.49</low>
  <high>0.51</high>
</property>
<property type="ignore">
  <name>TIME</name> 
</property>
<property>
  <name>CONTENT</name> 
  <comparator>StringComparator</comparator>
  <low>0.2</low>
  <high>0.9</high>
</property>    

<column name="1" property="ID"/>
<column name="2" 
		property="URL"
		cleaner="no.priv.garshol.duke.cleaners.LowerCaseNormalizeCleaner"/>
<column name="3"
        property="TITLE"/>
<column name="4"
        property="SOURCE"/>
<column name="5"
        property="TIME"/>
<column name="6"
        property="CONTENT"/>

`

[ibuda]

This is only natural. When Duke analyses the batch with records ID1 and compares it to the record ID2, it spots that they are duplicates and save this result to output. Analogously, when it analyses the batch with record ID2 and compares it against the already indexed records ID2, in your case it also spots that they are duplicates.
I mentioned "in your case", because not always the distances (comparators) you are using are metrics, i.e. they are not always symmetric. In one word, you will not always get that:
Comparator(ID1, ID2) == Comparator(ID2, ID1)
This is why some users get these repeated matches, and others don't.