ListShards Failure With Large Number of Shards

[jasonpepper]

If you have a stream with a large number of shards, you get an error that says, "[BROKER] ListShards error: InvalidArgumentException: NextToken and StreamName cannot be provided together."

The AWS docs say, "Don't specify StreamName or StreamCreationTimestamp if you specify NextToken because the latter unambiguously identifies the stream."

https://docs.aws.amazon.com/kinesis/latest/APIReference/API_ListShards.html

This shouldn't happen unless the number of shards is greater than the default value for MaxResults which is 10,000. But I'm seeing it in a stream with 1,266 shards (including closed ones).

I'm using version 0.3.2 of the library.

[slydon]

The documentation is wrong, it is definitely 1,000 shards.

[LyleScott]

To anyone reading this, maxresults is indeed max 10k. It Defaults to 1k. So, to fix this, supply MaxResults > the number of shards. Or a smaller MaxResults and paginate that many at a time using NextToken.