org.apache.sysml.runtime.compress

Class ColGroupValue

java.lang.Object

org.apache.sysml.runtime.compress.ColGroup

org.apache.sysml.runtime.compress.ColGroupValue

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

ColGroupDDC, ColGroupOffset

public abstract class ColGroupValue
extends ColGroup

Base class for column groups encoded with value dictionary.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysml.runtime.compress.ColGroup

ColGroup.CompressionType

Field Summary

Fields

Modifier and Type	Field and Description
protected double[]	_values Distinct values associated with individual bitmaps.
static boolean	LOW_LEVEL_OPT
static boolean	SORT_VALUES_BY_LENGTH

Fields inherited from class org.apache.sysml.runtime.compress.ColGroup

_colIndexes, _numRows

Constructor Summary

Constructors

Modifier	Constructor and Description
	ColGroupValue()
protected	ColGroupValue(int[] colIndices, int numRows, double[] values) Constructor for subclass methods that need to create shallow copies
	ColGroupValue(int[] colIndices, int numRows, UncompressedBitmap ubm) Stores the headers for the individual bitmaps.

Method Summary

Modifier and Type	Method and Description
protected static double[]	allocDVector(int len, boolean reset)
protected static int[]	allocIVector(int len, boolean reset)
protected double[]	applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op) Method for use by subclasses.
protected double[]	applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op, double newVal, int numCols)
static void	cleanupThreadLocalMemory()
protected void	computeColMxx(MatrixBlock result, org.apache.sysml.runtime.functionobjects.Builtin builtin, boolean zeros) NOTE: Shared across OLE/RLE/DDC because value-only computation.
protected void	computeMxx(MatrixBlock result, org.apache.sysml.runtime.functionobjects.Builtin builtin, boolean zeros) NOTE: Shared across OLE/RLE/DDC because value-only computation.
protected int	containsAllZeroValue()
long	estimateInMemorySize() Note: Must be overridden by child classes to account for additional data and metadata
int	getNumValues() Obtain number of distrinct sets of values associated with the bitmaps in this column group.
double[]	getValues()
protected double[]	preaggValues(int numVals, double[] b)
protected double[]	preaggValues(int numVals, double[] b, boolean allocNew)
static void	setupThreadLocalMemory(int len)
protected double[]	sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus, org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)
protected double[]	sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus, org.apache.sysml.runtime.instructions.cp.KahanObject kbuff, boolean allocNew)
protected double	sumValues(int valIx)
protected double	sumValues(int valIx, double[] b)
protected double	sumValues(int valIx, org.apache.sysml.runtime.functionobjects.KahanFunction kplus, org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)
void	unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op, MatrixBlock result)
abstract void	unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op, MatrixBlock result, int rl, int ru)

Methods inherited from class org.apache.sysml.runtime.compress.ColGroup

countNonZerosPerRow, decompressToBlock, decompressToBlock, decompressToBlock, get, getColIndex, getColIndices, getCompType, getExactSizeOnDisk, getNumCols, getNumRows, leftMultByRowVector, readFields, rightMultByVector, scalarOperation, shiftColIndices, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOW_LEVEL_OPT

public static boolean LOW_LEVEL_OPT

SORT_VALUES_BY_LENGTH

public static final boolean SORT_VALUES_BY_LENGTH

See Also:

Constant Field Values

_values

protected double[] _values

Distinct values associated with individual bitmaps.

Constructor Detail

ColGroupValue

public ColGroupValue()

ColGroupValue

public ColGroupValue(int[] colIndices,
                     int numRows,
                     UncompressedBitmap ubm)

Stores the headers for the individual bitmaps.

Parameters:

colIndices - indices (within the block) of the columns included in this column

numRows - total number of rows in the parent block

ubm - Uncompressed bitmap representation of the block

ColGroupValue

protected ColGroupValue(int[] colIndices,
                        int numRows,
                        double[] values)

Constructor for subclass methods that need to create shallow copies

Parameters:

colIndices - raw column index information

numRows - number of rows in the block

values - set of distinct values for the block (associated bitmaps are kept in the subclass)

Method Detail

estimateInMemorySize

public long estimateInMemorySize()

Description copied from class: ColGroup

Note: Must be overridden by child classes to account for additional data and metadata

Overrides:

estimateInMemorySize in class ColGroup

Returns:

an upper bound on the number of bytes used to store this ColGroup in memory.

getNumValues

public int getNumValues()

Obtain number of distrinct sets of values associated with the bitmaps in this column group.

Returns:

the number of distinct sets of values associated with the bitmaps in this column group

getValues

public double[] getValues()

containsAllZeroValue

protected int containsAllZeroValue()

sumValues

protected final double sumValues(int valIx)

sumValues

protected final double sumValues(int valIx,
                                 org.apache.sysml.runtime.functionobjects.KahanFunction kplus,
                                 org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)

sumAllValues

protected final double[] sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus,
                                      org.apache.sysml.runtime.instructions.cp.KahanObject kbuff)

sumAllValues

protected final double[] sumAllValues(org.apache.sysml.runtime.functionobjects.KahanFunction kplus,
                                      org.apache.sysml.runtime.instructions.cp.KahanObject kbuff,
                                      boolean allocNew)

sumValues

protected final double sumValues(int valIx,
                                 double[] b)

preaggValues

protected final double[] preaggValues(int numVals,
                                      double[] b)

preaggValues

protected final double[] preaggValues(int numVals,
                                      double[] b,
                                      boolean allocNew)

computeMxx

protected void computeMxx(MatrixBlock result,
                          org.apache.sysml.runtime.functionobjects.Builtin builtin,
                          boolean zeros)

NOTE: Shared across OLE/RLE/DDC because value-only computation.

Parameters:

result - output matrix block

builtin - function object

zeros - indicator if column group contains zero values

computeColMxx

protected void computeColMxx(MatrixBlock result,
                             org.apache.sysml.runtime.functionobjects.Builtin builtin,
                             boolean zeros)

NOTE: Shared across OLE/RLE/DDC because value-only computation.

Parameters:

result - output matrix block

builtin - function object

zeros - indicator if column group contains zero values

applyScalarOp

protected double[] applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op)
                          throws DMLRuntimeException

Method for use by subclasses. Applies a scalar operation to the value metadata stored in the superclass.

Parameters:

op - scalar operation to perform

Returns:

transformed copy of value metadata for this column group

Throws:

DMLRuntimeException - if DMLRuntimeException occurs

applyScalarOp

protected double[] applyScalarOp(org.apache.sysml.runtime.matrix.operators.ScalarOperator op,
                                 double newVal,
                                 int numCols)
                          throws DMLRuntimeException

Throws:

DMLRuntimeException

unaryAggregateOperations

public void unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
                                     MatrixBlock result)
                              throws DMLRuntimeException

Specified by:

unaryAggregateOperations in class ColGroup

Throws:

DMLRuntimeException

unaryAggregateOperations

public abstract void unaryAggregateOperations(org.apache.sysml.runtime.matrix.operators.AggregateUnaryOperator op,
                                              MatrixBlock result,
                                              int rl,
                                              int ru)
                                       throws DMLRuntimeException

Parameters:

op - aggregation operator

result - output matrix block

rl - row lower index, inclusive

ru - row upper index, exclusive

Throws:

DMLRuntimeException - on invalid inputs

setupThreadLocalMemory

public static void setupThreadLocalMemory(int len)

cleanupThreadLocalMemory

public static void cleanupThreadLocalMemory()

allocDVector

protected static double[] allocDVector(int len,
                                       boolean reset)

allocIVector

protected static int[] allocIVector(int len,
                                    boolean reset)