Skip to content

Commit

Permalink
Improvements in PairwiseSetOperations.
Browse files Browse the repository at this point in the history
Many other minor javadoc edits and updates.
  • Loading branch information
Lee Rhodes committed Oct 5, 2016
1 parent 5d9ffe5 commit 955b018
Show file tree
Hide file tree
Showing 27 changed files with 407 additions and 321 deletions.
3 changes: 2 additions & 1 deletion memory/src/main/java/com/yahoo/memory/MemoryRegionR.java
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@
*/
public class MemoryRegionR extends MemoryRegion {

public MemoryRegionR(Memory memory, long memOffsetBytes, long capacityBytes, MemoryRequest memReq) {
public MemoryRegionR(Memory memory, long memOffsetBytes, long capacityBytes,
MemoryRequest memReq) {
super(memory, memOffsetBytes, capacityBytes, memReq);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@

/**
* Methods of serializing and deserializing arrays of Double.
*
* @author Alex Saydakov
*/
public class ArrayOfDoublesSerDe extends ArrayOfItemsSerDe<Double> {

Expand Down
16 changes: 9 additions & 7 deletions sketches/src/main/java/com/yahoo/sketches/ArrayOfItemsSerDe.java
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,16 @@
/**
* Base class for serializing and deserializing custom types.
* @param <T> Type of item
*
* @author Alex Saydakov
*/
public abstract class ArrayOfItemsSerDe<T> {

/**
* Serialize an array of items to byte array.
* The size of the array doesn't need to be serialized.
* This method is called by the sketch serialization process.
*
*
* @param items array of items to be serialized
* @return serialized representation of the given array of items
*/
Expand All @@ -26,7 +28,7 @@ public abstract class ArrayOfItemsSerDe<T> {
/**
* Deserialize an array of items from a given Memory object.
* This method is called by the sketch deserialization process.
*
*
* @param mem Memory containing a serialized array of items
* @param numItems number of items in the serialized array
* @return deserialized array of items
Expand All @@ -35,22 +37,22 @@ public abstract class ArrayOfItemsSerDe<T> {

/**
* Provides a simple mechanism to check compatibility between SerDe implementations.
*
*
* <p>You will need to override this in the following cases:
* <ul><li>If you want to rename the class or change its package hierarchy and keep the ID the
* <ul><li>If you want to rename the class or change its package hierarchy and keep the ID the
* same. This enables compatible deserialization of binary images that were serialized with a
* different class name or package hierarchy.</li>
* <li>If you wish to change the binary layout of the serialization and don't want to change the
* <li>If you wish to change the binary layout of the serialization and don't want to change the
* class name or package hierarchy, you will need to change the returned code.</li>
* </ul>
* @return a unique identifier of this SerDe
*/
public short getId() {
/*
* Note that the hashCode() of a String is strictly a function of the content of the String
* and will be the same across different JVMs. This is not the case for Object.hashCode(),
* and will be the same across different JVMs. This is not the case for Object.hashCode(),
* which generally computes the hash code from the native internal address of the object and
* will be DIFFERENT when computed on different JVMs. So if you override this method, make
* will be DIFFERENT when computed on different JVMs. So if you override this method, make
* sure it will be repeatable across JVMs.
*/
return (short) getClass().getName().hashCode();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@

/**
* Methods of serializing and deserializing arrays of Long.
*
* @author Alex Saydakov
*/
public class ArrayOfLongsSerDe extends ArrayOfItemsSerDe<Long> {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,15 @@

/**
* Methods of serializing and deserializing arrays of the object version of primitive types of Number.
*
* <p>This class serializes numbers with a leading byte (ASCII character) indicating the type. The class keeps
* the values byte aligned, even though only 3 bits are strictly necessary to encode one of
* the 6 different primitives with object types that extend Number.</p>
*
* <p>Classes handled are: <tt>Long</tt>, <tt>Integer</tt>, <tt>Short</tt>, <tt>Byte</tt>, <tt>Double</tt>, and
* <tt>Float</tt>.</p>
*
* @author Alex Saydakov
*/
public class ArrayOfNumbersSerDe extends ArrayOfItemsSerDe<Number> {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@
* {@link ArrayOfUtf16StringsSerDe}. In an extreme case when all strings are in ASCII,
* this method is 2 times more compact, but it takes more time to encode and decode
* by a factor of 1.5 to 2.
*
* @author Alex Saydakov
*/
public class ArrayOfStringsSerDe extends ArrayOfItemsSerDe<String> {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
* is a 16-bit code. The result is larger than one from {@link ArrayOfStringsSerDe}.
* In an extreme case when all strings are in ASCII, the size is doubled. However it takes
* less time to serialize and deserialize by a factor of 1.5 to 2.
*
* @author Alex Saydakov
*/
public class ArrayOfUtf16StringsSerDe extends ArrayOfItemsSerDe<String> {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,28 +11,27 @@
* This class is used to compute the bounds on the estimate of the ratio <i>B / A</i>, where:
* <ul>
* <li><i>A</i> is a Theta Sketch of population <i>PopA</i>.</li>
* <li><i>B</i> is a Theta Sketch of population <i>PopB</i> that is a proper subset of <i>A</i>,
* obtained by an intersection of <i>A</i> with some other Theta Sketch <i>C</i>,
* <li><i>B</i> is a Theta Sketch of population <i>PopB</i> that is a proper subset of <i>A</i>,
* obtained by an intersection of <i>A</i> with some other Theta Sketch <i>C</i>,
* which acts like a predicate or selection clause.</li>
* <li>The estimate of the ratio <i>PopB/PopA</i> is
* BoundsOnRatiosInThetaSketchedSets.getEstimateOfBoverA(<i>A, B</i>).</li>
* <li>The Upper Bound estimate on the ratio PopB/PopA is
* <li>The Upper Bound estimate on the ratio PopB/PopA is
* BoundsOnRatiosInThetaSketchedSets.getUpperBoundForBoverA(<i>A, B</i>).</li>
* <li>The Lower Bound estimate on the ratio PopB/PopA is
* <li>The Lower Bound estimate on the ratio PopB/PopA is
* BoundsOnRatiosInThetaSketchedSets.getLowerBoundForBoverA(<i>A, B</i>).</li>
* </ul>
* Note: The theta of <i>A</i> cannot be greater than the theta of <i>B</i>.
* If <i>B</i> is formed as an intersection of <i>A</i> and some other set <i>C</i>,
* Note: The theta of <i>A</i> cannot be greater than the theta of <i>B</i>.
* If <i>B</i> is formed as an intersection of <i>A</i> and some other set <i>C</i>,
* then the theta of <i>B</i> is guaranteed to be less than or equal to the theta of <i>B</i>.
*
* @version 0.5.1
*
* @author Kevin Lang
* @author Lee Rhodes
*/
public final class BoundsOnRatiosInThetaSketchedSets {

private BoundsOnRatiosInThetaSketchedSets() {}

/**
* Gets the approximate lower bound for B over A based on a 95% confidence interval
* @param sketchA the sketch A
Expand All @@ -43,16 +42,16 @@ public static double getLowerBoundForBoverA(Sketch sketchA, Sketch sketchB) {
double thetaA = sketchA.getTheta();
double thetaB = sketchB.getTheta();
checkThetas(thetaA, thetaB);

int countB = sketchB.getRetainedEntries(true);
int countA = (thetaB == thetaA) ? sketchA.getRetainedEntries(true)
int countA = (thetaB == thetaA) ? sketchA.getRetainedEntries(true)
: sketchA.getCountLessThanTheta(thetaB);

if (countA <= 0) return 0;

return BoundsOnRatiosInSampledSets.getLowerBoundForBoverA(countA, countB, thetaB);
}

/**
* Gets the approximate upper bound for B over A based on a 95% confidence interval
* @param sketchA the sketch A
Expand All @@ -65,14 +64,14 @@ public static double getUpperBoundForBoverA(Sketch sketchA, Sketch sketchB) {
checkThetas(thetaA, thetaB);

int countB = sketchB.getRetainedEntries(true);
int countA = (thetaB == thetaA) ? sketchA.getRetainedEntries(true)
int countA = (thetaB == thetaA) ? sketchA.getRetainedEntries(true)
: sketchA.getCountLessThanTheta(thetaB);

if (countA <= 0) return 1.0;

return BoundsOnRatiosInSampledSets.getUpperBoundForBoverA(countA, countB, thetaB);
}

/**
* Gets the estimate for B over A
* @param sketchA the sketch A
Expand All @@ -85,14 +84,14 @@ public static double getEstimateOfBoverA(Sketch sketchA, Sketch sketchB) {
checkThetas(thetaA, thetaB);

int countB = sketchB.getRetainedEntries(true);
int countA = (thetaB == thetaA) ? sketchA.getRetainedEntries(true)
int countA = (thetaB == thetaA) ? sketchA.getRetainedEntries(true)
: sketchA.getCountLessThanTheta(thetaB);

if (countA <= 0) return 0.5;

return (double) countB / (double) countA;
}

static void checkThetas(double thetaA, double thetaB) {
if (thetaB > thetaA) {
throw new SketchesArgumentException("ThetaB cannot be > ThetaA.");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,21 @@

/**
* Illegal Arguments Exception class for the library
*
* @author Lee Rhodes
*/
public class SketchesArgumentException extends SketchesException {
private static final long serialVersionUID = 1L;

//other constructors to be added as needed.

/**
* Constructs a new runtime exception with the specified detail message. The cause is not
* initialized, and may subsequently be initialized by a call to
* Constructs a new runtime exception with the specified detail message. The cause is not
* initialized, and may subsequently be initialized by a call to
* Throwable.initCause(java.lang.Throwable).
*
* @param message the detail message. The detail message is saved for later retrieval by the
* Throwable.getMessage() method.
*
* @param message the detail message. The detail message is saved for later retrieval by the
* Throwable.getMessage() method.
*/
public SketchesArgumentException(String message) {
super(message);
Expand Down
30 changes: 16 additions & 14 deletions sketches/src/main/java/com/yahoo/sketches/SketchesException.java
Original file line number Diff line number Diff line change
Expand Up @@ -7,37 +7,39 @@

/**
* Exception class for the library
*
* @author Lee Rhodes
*/
public class SketchesException extends RuntimeException {
private static final long serialVersionUID = 1L;

//other constructors to be added as needed.

/**
* Constructs a new runtime exception with the specified detail message. The cause is not
* initialized, and may subsequently be initialized by a call to
* Constructs a new runtime exception with the specified detail message. The cause is not
* initialized, and may subsequently be initialized by a call to
* Throwable.initCause(java.lang.Throwable).
*
* @param message the detail message. The detail message is saved for later retrieval by the
* Throwable.getMessage() method.
*
* @param message the detail message. The detail message is saved for later retrieval by the
* Throwable.getMessage() method.
*/
public SketchesException(String message) {
super(message);
}

/**
* Constructs a new runtime exception with the specified detail message and cause.
*
* <p>Note that the detail message associated with cause is not automatically incorporated
*
* <p>Note that the detail message associated with cause is not automatically incorporated
* in this runtime exception's detail message.</p>
*
* @param message the detail message (which is saved for later retrieval by the
*
* @param message the detail message (which is saved for later retrieval by the
* Throwable.getMessage() method).
* @param cause the cause (which is saved for later retrieval by the Throwable.getCause()
* @param cause the cause (which is saved for later retrieval by the Throwable.getCause()
* method). (A null value is permitted, and indicates that the cause is nonexistent or unknown.)
*/
public SketchesException(String message, Throwable cause) {
super(message, cause);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,21 @@

/**
* Illegal State Exception class for the library
*
* @author Lee Rhodes
*/
public class SketchesStateException extends SketchesException {
private static final long serialVersionUID = 1L;

//other constructors to be added as needed.

/**
* Constructs a new runtime exception with the specified detail message. The cause is not
* initialized, and may subsequently be initialized by a call to
* Constructs a new runtime exception with the specified detail message. The cause is not
* initialized, and may subsequently be initialized by a call to
* Throwable.initCause(java.lang.Throwable).
*
* @param message the detail message. The detail message is saved for later retrieval by the
* Throwable.getMessage() method.
*
* @param message the detail message. The detail message is saved for later retrieval by the
* Throwable.getMessage() method.
*/
public SketchesStateException(String message) {
super(message);
Expand Down
9 changes: 5 additions & 4 deletions sketches/src/main/java/com/yahoo/sketches/package-info.java
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
/*
* Copyright 2016, Yahoo! Inc.
* Licensed under the terms of the Apache License 2.0. See LICENSE file at the project root
* Licensed under the terms of the Apache License 2.0. See LICENSE file at the project root
* for terms.
*/

/**
* <p>This package is the parent package for all sketch algorithms as well as support packages
* such as the <i>memory</i> and <i>hash</i> packages. Classes at this level are used by more than
* one sub-package.
* <p>This package is the parent package for all sketch algorithms.
* Classes at this level are used by more than one sub-package.
*
* @author Lee Rhodes
*/
package com.yahoo.sketches;
Loading

0 comments on commit 955b018

Please sign in to comment.