Skip to content

Commit

Permalink
Fix quality and performance issues, add monadic API
Browse files Browse the repository at this point in the history
This patch is mostly backwards compatible. See "Breaking Changes" below
for the full list of backwards incompatible changes.

This patch fixes quality and performance issues, addresses additional
miscellaneous issues, and introduces a monadic API.

Issues addressed
================

Priority issues fixed in this patch:

- Title: "The seeds generated by split are not independent"
  Link:  haskell#25
  Fixed: changed algorithm to SplitMix, which provides a robust 'split'
  operation

- Title: "Very low throughput"
  Link:  haskell#51
  Fixed: see "Performance" below

Additional issues addressed in this patch:

- Title: "Add Random instances for tuples"
  Link:  haskell#26
  Addressed: added 'Uniform' instances for up to 6-tuples

- Title: "Add Random instance for Natural"
  Link:  haskell#44
  Addressed: added 'UniformRange' instance for 'Natural'

- Title: "incorrect distribution of randomR for floating-point numbers"
  Link:  haskell#53
  Addressed: see "Regarding floating-point numbers" below

- Title: "System/Random.hs:43:1: warning: [-Wtabs]"
  Link:  haskell#55
  Fixed: no more tabs

- Title: "Why does random for Float and Double produce exactly 24 or 53 bits?"
  Link:  haskell#58
  Fixed: see "Regarding floating-point numbers" below

- Title: "read :: StdGen fails for strings longer than 6"
  Link:  haskell#59
  Addressed: 'StdGen' is no longer an instance of 'Read'

Regarding floating-point numbers: with this patch, the relevant
instances for 'Float' and 'Double' sample more bits than before but do
not sample every possible representable value. The documentation now
clearly spells out what this means for users.

Quality (issue 25)
==================

The algorithm [1] in version 1.1 of this library fails empirical PRNG
tests when used to generate "split sequences" as proposed in [3].

SplitMix [2] passes the same tests. This patch changes 'StdGen' to use
the SplitMix implementation provided by the splitmix package.

Test batteries used: dieharder, TestU1, PractRand.

[1]: P. L'Ecuyer, "Efficient and portable combined random number
generators". https://doi.org/10.1145/62959.62969

[2]: G. L. Steele, D. Lea, C. H. Flood, "Fast splittable pseudorandom
number generators". https://doi.org/10.1145/2714064.2660195

[3]: H. G. Schaathun, "Evaluation of splittable pseudo-random
generators". https://doi.org/10.1017/S095679681500012X

Performance (issue 51)
======================

The "improvement" column in the following table is a multiplier: the
improvement for 'random' for type 'Float' is 1038, so this operation is
1038 times faster with this patch.

| Name                    | Mean (1.1) | Mean (patch) | Improvement|
| ----------------------- | ---------- | ------------ | ---------- |
| pure/random/Float       |         30 |         0.03 |        1038|
| pure/random/Double      |         52 |         0.03 |        1672|
| pure/random/Integer     |         43 |         0.33 |         131|
| pure/uniform/Word       |         44 |         0.03 |        1491|
| pure/uniform/Int        |         43 |         0.03 |        1512|
| pure/uniform/Char       |         17 |         0.49 |          35|
| pure/uniform/Bool       |         18 |         0.03 |         618|

API changes
===========

StatefulGen
-----------

This patch adds a class 'StatefulGen':

    -- | 'StatefulGen' is an interface to monadic pseudo-random number generators.
    class Monad m => StatefulGen g m where
      uniformWord32 :: g -> m Word32 -- default implementation in terms of uniformWord64
      uniformWord64 :: g -> m Word64 -- default implementation in terms of uniformWord32
      -- plus methods for other word sizes and for byte strings
      -- all have default implementations so the MINIMAL pragma holds

In 'StatefulGen g m', 'g' is the type of the generator and 'm' the underlying
monad.

Four 'StatefulGen' instances ("monadic adapters") are provided for pure
generators to enable their use in monadic code. The documentation
describes them in detail.

FrozenGen
---------

This patch also introduces a class 'FrozenGen':

    -- | 'FrozenGen' is designed for stateful pseudo-random number generators
    -- that can be saved as and restored from an immutable data type.
    class StatefulGen (MutableGen f m) m => FrozenGen f m where
      type MutableGen f m = (g :: Type) | g -> f
      freezeGen :: MutableGen f m -> m f
      thawGen :: f -> m (MutableGen f m)

'f' is the type of the generator's state "at rest" and 'm' the underlying
monad. 'MutableGen' is defined as an injective type family via 'g -> f' so for
any generator 'g', the type 'f' of its at-rest state is well-defined.

Both 'StatefulGen' and 'FrozenGen' are generic enough to accommodate, for
example, the 'Gen' type from the 'mwc-random' package, which itself abstracts
over the underlying primitive monad and state token. The documentation shows
the full instances.

'Uniform' and 'UniformRange'
----------------------------

The 'Random' typeclass has conceptually been split into 'Uniform' and
'UniformRange'. The 'Random' typeclass is still included for backwards
compatibility. 'Uniform' is for types where it is possible to sample
from the type's entire domain; 'UniformRange' is for types where one can
sample from a specified range.

Breaking Changes
================

This patch introduces these breaking changes:

* requires 'base >= 4.8' (GHC-7.10)
* 'StdGen' is no longer an instance of 'Read'
* 'randomIO' and 'randomRIO' where extracted from the 'Random' class into
  separate functions

In addition, there may be import clashes with new functions, e.g. 'uniform' and
'uniformR'.

Deprecations
============

This patch introduces 'genWord64', 'genWord32' and similar methods to
the 'RandomGen' class. The significantly slower method 'next' and its
companion 'genRange' are now deprecated.

Co-authored-by: Alexey Kuleshevich <[email protected]>
Co-authored-by: idontgetoutmuch <[email protected]>
Co-authored-by: Leonhard Markert <[email protected]>
  • Loading branch information
3 people committed Jun 23, 2020
1 parent 8bfda8a commit b45bd28
Show file tree
Hide file tree
Showing 38 changed files with 3,741 additions and 1,632 deletions.
5 changes: 0 additions & 5 deletions .darcs-boring

This file was deleted.

15 changes: 3 additions & 12 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,3 @@
*~

Thumbs.db
.DS_Store

GNUmakefile
dist-install/
ghc.mk

dist
.cabal-sandbox
cabal.sandbox.config
stack.yaml.lock
.stack-work/
cabal.project.local
197 changes: 192 additions & 5 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,192 @@
language: haskell
ghc:
- 7.4
- 7.6
- 7.8
# This Travis job script has been generated by a script via
#
# haskell-ci '--travis-patches' '.travis.yml.patch' 'random.cabal'
#
# To regenerate the script (for example after adjusting tested-with) run
#
# haskell-ci regenerate
#
# For more information, see https://github.com/haskell-CI/haskell-ci
#
# version: 0.10
#
version: ~> 1.0
language: c
os: linux
dist: xenial
git:
# whether to recursively clone submodules
submodules: false
cache:
directories:
- $HOME/.cabal/packages
- $HOME/.cabal/store
- $HOME/.hlint
- $HOME/.stack
- $TRAVIS_BUILD_DIR/.stack-work
before_cache:
- rm -fv $CABALHOME/packages/hackage.haskell.org/build-reports.log
# remove files that are regenerated by 'cabal update'
- rm -fv $CABALHOME/packages/hackage.haskell.org/00-index.*
- rm -fv $CABALHOME/packages/hackage.haskell.org/*.json
- rm -fv $CABALHOME/packages/hackage.haskell.org/01-index.cache
- rm -fv $CABALHOME/packages/hackage.haskell.org/01-index.tar
- rm -fv $CABALHOME/packages/hackage.haskell.org/01-index.tar.idx
- rm -rfv $CABALHOME/packages/head.hackage
jobs:
include:
- compiler: ghc-8.10.1
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.10.1","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.8.2
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.8.2","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.8.1
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.8.1","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.6.5
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.6.5","cabal-install-3.2"]}}
env: COVERALLS_STACK_YAML="stack-coveralls.yaml"
os: linux
- compiler: ghc-8.6.4
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.6.4","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.6.3
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.6.3","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.4.4
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.4.4","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.4.3
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.4.3","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.2.2
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.2.2","cabal-install-3.2"]}}
os: linux
- compiler: ghc-8.0.2
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.0.2","cabal-install-3.2"]}}
os: linux
- compiler: ghc-7.10.3
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-7.10.3","cabal-install-3.2"]}}
os: linux
- compiler: ghc-7.10.2
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-7.10.2","cabal-install-3.2"]}}
os: linux
before_install:
- HC=$(echo "/opt/$CC/bin/ghc" | sed 's/-/\//')
- WITHCOMPILER="-w $HC"
- HADDOCK=$(echo "/opt/$CC/bin/haddock" | sed 's/-/\//')
- HCPKG="$HC-pkg"
- unset CC
- CABAL=/opt/ghc/bin/cabal
- CABALHOME=$HOME/.cabal
- export PATH="$CABALHOME/bin:$PATH"
- TOP=$(pwd)
- "HCNUMVER=$(${HC} --numeric-version|perl -ne '/^(\\d+)\\.(\\d+)\\.(\\d+)(\\.(\\d+))?$/; print(10000 * $1 + 100 * $2 + ($3 == 0 ? $5 != 1 : $3))')"
- echo $HCNUMVER
- CABAL="$CABAL -vnormal+nowrap"
- set -o pipefail
- TEST=--enable-tests
- BENCH=--enable-benchmarks
- HEADHACKAGE=false
- rm -f $CABALHOME/config
- |
echo "verbose: normal +nowrap +markoutput" >> $CABALHOME/config
echo "remote-build-reporting: anonymous" >> $CABALHOME/config
echo "write-ghc-environment-files: always" >> $CABALHOME/config
echo "remote-repo-cache: $CABALHOME/packages" >> $CABALHOME/config
echo "logs-dir: $CABALHOME/logs" >> $CABALHOME/config
echo "world-file: $CABALHOME/world" >> $CABALHOME/config
echo "extra-prog-path: $CABALHOME/bin" >> $CABALHOME/config
echo "symlink-bindir: $CABALHOME/bin" >> $CABALHOME/config
echo "installdir: $CABALHOME/bin" >> $CABALHOME/config
echo "build-summary: $CABALHOME/logs/build.log" >> $CABALHOME/config
echo "store-dir: $CABALHOME/store" >> $CABALHOME/config
echo "install-dirs user" >> $CABALHOME/config
echo " prefix: $CABALHOME" >> $CABALHOME/config
echo "repository hackage.haskell.org" >> $CABALHOME/config
echo " url: http://hackage.haskell.org/" >> $CABALHOME/config
# Download and unpack the stack executable
- mkdir -p ~/.local/bin
- |
if [ -n "${COVERALLS_STACK_YAML}" ]; then
if [ `uname` = "Darwin" ]
then
travis_retry curl --insecure -L https://get.haskellstack.org/stable/osx-x86_64.tar.gz | tar xz --strip-components=1 --include '*/stack' -C ~/.local/bin
else
travis_retry curl -L https://get.haskellstack.org/stable/linux-x86_64.tar.gz | tar xz --wildcards --strip-components=1 -C ~/.local/bin '*/stack'
fi
fi
install:
- ${CABAL} --version
- echo "$(${HC} --version) [$(${HC} --print-project-git-commit-id 2> /dev/null || echo '?')]"
- |
echo "program-default-options" >> $CABALHOME/config
echo " ghc-options: $GHCJOBS +RTS -M6G -RTS" >> $CABALHOME/config
- cat $CABALHOME/config
- rm -fv cabal.project cabal.project.local cabal.project.freeze
- travis_retry ${CABAL} v2-update -v
# Generate cabal.project
- rm -rf cabal.project cabal.project.local cabal.project.freeze
- touch cabal.project
- |
echo "packages: ." >> cabal.project
- if [ $HCNUMVER -ge 80200 ] ; then echo 'package random' >> cabal.project ; fi
- "if [ $HCNUMVER -ge 80200 ] ; then echo ' ghc-options: -Werror=missing-methods' >> cabal.project ; fi"
- |
- "for pkg in $($HCPKG list --simple-output); do echo $pkg | sed 's/-[^-]*$//' | (grep -vE -- '^(random)$' || true) | sed 's/^/constraints: /' | sed 's/$/ installed/' >> cabal.project.local; done"
- cat cabal.project || true
- cat cabal.project.local || true
- if [ -f "./configure.ac" ]; then (cd "." && autoreconf -i); fi
- ${CABAL} v2-freeze $WITHCOMPILER ${TEST} ${BENCH}
- "cat cabal.project.freeze | sed -E 's/^(constraints: *| *)//' | sed 's/any.//'"
- rm cabal.project.freeze
- travis_wait 40 ${CABAL} v2-build $WITHCOMPILER ${TEST} ${BENCH} --dep -j2 all
- travis_wait 40 ${CABAL} v2-build $WITHCOMPILER --disable-tests --disable-benchmarks --dep -j2 all
script:
- DISTDIR=$(mktemp -d /tmp/dist-test.XXXX)
# Coverage
- |
if [ -n "${COVERALLS_STACK_YAML}" ]; then
stack --stack-yaml $COVERALLS_STACK_YAML test --coverage
stack --stack-yaml $COVERALLS_STACK_YAML hpc report --all
travis_retry curl -L https://github.com/lehins/stack-hpc-coveralls/releases/download/0.0.5.0/shc.tar.gz | tar xz shc
STACK_YAML="$COVERALLS_STACK_YAML" ./shc combined custom
fi
# Packaging...
- ${CABAL} v2-sdist all
# Unpacking...
- mv dist-newstyle/sdist/*.tar.gz ${DISTDIR}/
- cd ${DISTDIR} || false
- find . -maxdepth 1 -type f -name '*.tar.gz' -exec tar -xvf '{}' \;
- find . -maxdepth 1 -type f -name '*.tar.gz' -exec rm '{}' \;
- PKGDIR_random="$(find . -maxdepth 1 -type d -regex '.*/random-[0-9.]*')"
# Generate cabal.project
- rm -rf cabal.project cabal.project.local cabal.project.freeze
- touch cabal.project
- |
echo "packages: ${PKGDIR_random}" >> cabal.project
- if [ $HCNUMVER -ge 80200 ] ; then echo 'package random' >> cabal.project ; fi
- "if [ $HCNUMVER -ge 80200 ] ; then echo ' ghc-options: -Werror=missing-methods' >> cabal.project ; fi"
- |
- "for pkg in $($HCPKG list --simple-output); do echo $pkg | sed 's/-[^-]*$//' | (grep -vE -- '^(random)$' || true) | sed 's/^/constraints: /' | sed 's/$/ installed/' >> cabal.project.local; done"
- cat cabal.project || true
- cat cabal.project.local || true
# Building...
# this builds all libraries and executables (without tests/benchmarks)
- ${CABAL} v2-build $WITHCOMPILER --disable-tests --disable-benchmarks all
# Building with tests and benchmarks...
# build & run tests, build benchmarks
- ${CABAL} v2-build $WITHCOMPILER ${TEST} ${BENCH} all
# Testing...
- ${CABAL} v2-test $WITHCOMPILER ${TEST} ${BENCH} all
# cabal check...
- (cd ${PKGDIR_random} && ${CABAL} -vnormal check)
# haddock...
- ${CABAL} v2-haddock $WITHCOMPILER --with-haddock $HADDOCK ${TEST} ${BENCH} all
# Building without installed constraints for packages in global-db...
- rm -f cabal.project.local
- ${CABAL} v2-build $WITHCOMPILER --disable-tests --disable-benchmarks all

# REGENDATA ("0.10",["--travis-patches",".travis.yml.patch","random.cabal"])
# EOF
50 changes: 50 additions & 0 deletions .travis.yml.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
@@ -22,6 +22,8 @@ cache:
- $HOME/.cabal/packages
- $HOME/.cabal/store
- $HOME/.hlint
+ - $HOME/.stack
+ - $TRAVIS_BUILD_DIR/.stack-work
before_cache:
- rm -fv $CABALHOME/packages/hackage.haskell.org/build-reports.log
# remove files that are regenerated by 'cabal update'
@@ -44,6 +49,7 @@ jobs:
os: linux
- compiler: ghc-8.6.5
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.6.5","cabal-install-3.2"]}}
+ env: COVERALLS_STACK_YAML="stack-coveralls.yaml"
os: linux
- compiler: ghc-8.6.4
addons: {"apt":{"sources":[{"sourceline":"deb http://ppa.launchpad.net/hvr/ghc/ubuntu xenial main","key_url":"https://keyserver.ubuntu.com/pks/lookup?op=get&search=0x063dab2bdc0b3f9fcebc378bff3aeacef6f88286"}],"packages":["ghc-8.6.4","cabal-install-3.2"]}}
@@ -103,6 +109,17 @@ before_install:
echo " prefix: $CABALHOME" >> $CABALHOME/config
echo "repository hackage.haskell.org" >> $CABALHOME/config
echo " url: http://hackage.haskell.org/" >> $CABALHOME/config
+ # Download and unpack the stack executable
+ - mkdir -p ~/.local/bin
+ - |
+ if [ -n "${COVERALLS_STACK_YAML}" ]; then
+ if [ `uname` = "Darwin" ]
+ then
+ travis_retry curl --insecure -L https://get.haskellstack.org/stable/osx-x86_64.tar.gz | tar xz --strip-components=1 --include '*/stack' -C ~/.local/bin
+ else
+ travis_retry curl -L https://get.haskellstack.org/stable/linux-x86_64.tar.gz | tar xz --wildcards --strip-components=1 -C ~/.local/bin '*/stack'
+ fi
+ fi
install:
- ${CABAL} --version
- echo "$(${HC} --version) [$(${HC} --print-project-git-commit-id 2> /dev/null || echo '?')]"
@@ -131,6 +148,14 @@ install:
- travis_wait 40 ${CABAL} v2-build $WITHCOMPILER --disable-tests --disable-benchmarks --dep -j2 all
script:
- DISTDIR=$(mktemp -d /tmp/dist-test.XXXX)
+ # Coverage
+ - |
+ if [ -n "${COVERALLS_STACK_YAML}" ]; then
+ stack --stack-yaml $COVERALLS_STACK_YAML test --coverage
+ stack --stack-yaml $COVERALLS_STACK_YAML hpc report --all
+ travis_retry curl -L https://github.com/lehins/stack-hpc-coveralls/releases/download/0.0.5.0/shc.tar.gz | tar xz shc
+ STACK_YAML="$COVERALLS_STACK_YAML" ./shc combined custom
+ fi
# Packaging...
- ${CABAL} v2-sdist all
# Unpacking...
Loading

0 comments on commit b45bd28

Please sign in to comment.