serializedSize(): Always return a double (to lower the risk for integer overflow downstream)

[HenrikBengtsson]

Background

The serializedSize() function returns the number of bytes it would take to serialize an object. It returns an integer, unless it is greater than .Machine$integer.max (= ~2.15 GB), in case it returns a double (

parallelly/src/calc-serialized-size.c

Lines 74 to 78 in 88783a6

	if (count > R_INT_MAX) {
	return(ScalarReal((double) count));
	} else {
	return(ScalarInteger(count));
	}

).

Issue

This agility of serializedSize() to return an integer, if sufficient, can lead to silent errors downstream. For instance, two objects might be small enough to return an integer;

size1 <- serializedSize(x1)
size2 <- serializedSize(x2)

but large enough for their sum to overflow;

> size1 + size2
NA
Warning message:
In size1 + size2:
  NAs produced by integer overflow

This problem was first reported by @ycl6 in futureverse/future#755.

Suggestion

Change serializedSize() to always return a double. FWIW, that is what utils::object.size() does.

/cc @coolbutuseless

[HenrikBengtsson]

Implemented in parallelly (>= 1.40.1-9007).

[HenrikBengtsson]

@ycl6, parallelly 1.41.0 fixing this is now on CRAN.

[ycl6]

Thanks @HenrikBengtsson for updating the CRAN package so swiftly!
I've been using parallelly 1.40.1-9007 and works perfectly 👍

[HenrikBengtsson]

@ycl6 , thanks again for reporting and working with downstream packages and issue trackers.