build: fix flaky test

[ndr-brt]

What this PR changes/adds

Set http sink partition size to 1 for the EndToEndKafkaTransferTest

Why it does that

Avoid test flakiness, the default partition size to 5 caused messages to be sent to the consumer slightly after termination, causing test failure.

Further notes

• changed the implementation of the next() method in the ConsumerRecordsIterator because most of the time was doing duplicated pulls for a single next() call.

Linked Issue(s)

Closes # <-- insert Issue number if one exists

Please be sure to take a look at the contributing guidelines and our etiquette for pull requests.

[codecov-commenter]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (0d6086d) 72.09% compared to head (086f02b) 72.10%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3601   +/-   ##
=======================================
  Coverage   72.09%   72.10%           
=======================================
  Files         897      897           
  Lines       17920    17919    -1     
  Branches     1018     1018           
=======================================
  Hits        12920    12920           
+ Misses       4563     4562    -1     
  Partials      437      437           

Files	Coverage Δ
.../edc/dataplane/kafka/pipeline/KafkaDataSource.java	0.00% <0.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.