The triangle program benchmark

Used with minimal 14 tests covering all the code in "Efficient Multi Objective Higher Order Mutation Testing with Genetic Programming", Journal of Systems and Software, 83 (2010) pp2416-2430 doi:10.1016/j.jss.2010.07.027

See

• jss/triangle.c
  Original C source for the Triangle Program
• jss/testcases_oracle.txt
  14 tests (inputs and expected return value) for the Triangle Program

Fitness Landscape of the Triangle Program, William B. Langdon and Mark Harman presented at the PPSN-2016 Workshop on Landscape-Aware Heuristic Search, Nadarajen Veerapen and Gabriela Ochoa (editors), 17-21 September http://www.cs.ucl.ac.uk/fileadmin/UCL-CS/research/Research_Notes/rn1605.pdf

Data set http://www.cs.ucl.ac.uk/staff/W.Langdon/ppsn2016/triangle/

Visualising the Search Landscape of the Triangle Program, William B. Langdon and Nadarajen Veerapen and Gabriela Ochoa, in EuroGP-2017, M. Castelli and J. McDermott and L. Sekanina, Eds., LNCS 10196, Amsterdam, 19-21 April, Springer. http://dx.doi.org/10.1007/978-3-319-55696-3_7 https://dspace.stir.ac.uk/bitstream/1893/24848/1/triangle.pdf

Data set http://www.cs.ucl.ac.uk/staff/W.Langdon/egp2017/triangle/

Original code distribution http://www.cs.ucl.ac.uk/staff/W.Langdon/ftp/gp-code/triangle.tar.gz

---

Code for slides in http://www.cs.ucl.ac.uk/staff/W.Langdon/triangle_4-nov-2021

• information.awk gawk script to process huge volume of data generated by main2.c and calculate mutual information between each input and
  • the Triangle Program output
  • the internal triang variable at various lines inside the program after triang is (conditionally) updated.

  ./a.out | gawk -f information.awk > information.dat
  

  (takes 4.4GB and a several minutes)
• information.dat
  Output of information.awk (not in github)
• information_filter.awk
  preprocess output of main2.c for information.gnu

  ./a.out | gawk -f information_filter.awk > information_filter.out
  

• information_filter.out
  Output of information_filter.awk (not in github) for use by information.gnu
• information.gnu
  gnuplot script to show distribution of values held by the internal triang variable at various lines inside the program after each time it is conditionally updated. Note lines are horizontal. This shows that not only is the distribution the same for each of the three inputs but also there is no variation with input value.
• information.out
  More recent version of output of information_filter.awk (again not in github) for use by information_return.gnu
• information_return.gnu
  like information.gnu a gnuplot script to show distribution of values output by the Triangle Program. Again the distribution for each of the three inputs are identical. Note almost all return values are 4 (not a triangle) and the other three values only occur if the input is positive.
• isosceles.gnu
  Example gnuplot script to show 3D plot of three planes in 3D input space of the Triangle Program corresponding to outputs 2 isosceles and 3 Equilateral on the user's screen. User interaction via mouse etc. (Other values not plotted). Assumes 8120601 test cases for the Triangle Program.
• main2.c
  C file for instrumented version the Triangle Program Generates 8120601 test outputs using triang.c
• README.md
  This file
• triang.c
  Instrumented version the Triangle Program and support routines for main2.c
• triangle.gnu
  gnuplot script to show three dimensional distribution of values output by the Triangle Program. Value 2 for isosceles triangles are shown as three planes. To avoid swamping the display, value 1 for scalene triangles are sampled only every 5 (one in 125 in three dimensions).
• triangle_h.awk
  Example of calculating Entropy. Assumes 8120601 test cases for the Triangle Program

  gawk -f triangle_h.awk /dev/null
  Equilateral     3     100 0.000012 0.000200839
  Isosceles       2   22200 0.002734 0.0232779
  not_a_triangle  4 7605851 0.936612 0.0884881
  Scalene         1  492450 0.060642 0.245208
  entropy in 22.9532     entropy out 0.357175

• triangle.scalene
  data file generated from triangle.out by triangle.gnu (not in github)
• triang.out
  Distribution of values of internal variable triang after each time it is (conditionally) updated. (Not in github.)

  0

  line 11 (in jss/triangle.c), after triang = 0;

  1

  line 15, after triang conditionally incremented

  2

  line 18, after triang conditionally increased by 2

  3

  line 21, after triang conditionally increased by 3

  Assumes 8120601 test cases for the Triangle Program.

  0   0 1000000
  0 tot 1000000
  1   0  990000
  1   1   10000
  1 tot 1000000
  2   0  980100
  2   1    9900
  2   2    9900
  2   3     100
  2 tot 1000000
  3   0  970200
  3   1    9900
  3   2    9900
  3   3    9900
  3   6     100
  3 tot 1000000