public class FrameBlock extends Object implements org.apache.hadoop.io.Writable, org.apache.sysml.runtime.controlprogram.caching.CacheBlock, Externalizable
Modifier and Type | Class and Description |
---|---|
static class |
FrameBlock.ColumnMetadata |
Modifier and Type | Field and Description |
---|---|
static int |
BUFFER_SIZE |
Constructor and Description |
---|
FrameBlock() |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema) |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema,
String[] names) |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema,
String[][] data) |
FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema,
String[] names,
String[][] data) |
FrameBlock(FrameBlock that)
Copy constructor for frame blocks, which uses a shallow copy for
the schema (column types and names) but a deep copy for meta data
and actual column data.
|
FrameBlock(int ncols,
org.apache.sysml.parser.Expression.ValueType vt) |
Modifier and Type | Method and Description |
---|---|
void |
appendColumn(boolean[] col)
Append a column of value type BOOLEAN as the last column of
the data frame.
|
void |
appendColumn(double[] col)
Append a column of value type DOUBLE as the last column of
the data frame.
|
void |
appendColumn(long[] col)
Append a column of value type INT as the last column of
the data frame.
|
void |
appendColumn(String[] col)
Append a column of value type STRING as the last column of
the data frame.
|
void |
appendColumns(double[][] cols)
Append a set of column of value type DOUBLE at the end of the frame
in order to avoid repeated allocation with appendColumns.
|
FrameBlock |
appendOperations(FrameBlock that,
FrameBlock ret,
boolean cbind)
Appends the given argument frameblock 'that' to this frameblock by
creating a deep copy to prevent side effects.
|
void |
appendRow(Object[] row)
Append a row to the end of the data frame, where all row fields
are boxed objects according to the schema.
|
void |
appendRow(String[] row)
Append a row to the end of the data frame, where all row fields
are string encoded.
|
void |
compactEmptyBlock()
Free unnecessarily allocated empty block.
|
void |
copy(FrameBlock src) |
void |
copy(int rl,
int ru,
int cl,
int cu,
FrameBlock src) |
static String |
createColName(int i) |
static String[] |
createColNames(int size) |
static String[] |
createColNames(int off,
int size) |
void |
ensureAllocatedColumns(int numRows)
Allocate column data structures if necessary, i.e., if schema specified
but not all column data structures created yet.
|
void |
ensureColumnCompatibility(int newlen)
Checks for matching column sizes in case of existing columns.
|
Object |
get(int r,
int c)
Gets a boxed object of the value in position (r,c).
|
Object |
getColumn(int c) |
FrameBlock.ColumnMetadata[] |
getColumnMetadata() |
FrameBlock.ColumnMetadata |
getColumnMetadata(int c) |
String |
getColumnName(int c)
Returns the column name for the requested column.
|
Map<String,Integer> |
getColumnNameIDMap()
Creates a mapping from column names to column IDs, i.e.,
1-based column indexes
|
String[] |
getColumnNames()
Returns the column names of the frame block.
|
String[] |
getColumnNames(boolean alloc)
Returns the column names of the frame block.
|
long |
getExactSerializedSize()
Get the exact serialized size in bytes of the cache block.
|
long |
getInMemorySize()
Get the in-memory size in bytes of the cache block.
|
int |
getNumColumns()
Get the number of columns of the frame block, that is
the number of columns defined in the schema.
|
int |
getNumRows()
Get the number of rows of the frame block.
|
Iterator<Object[]> |
getObjectRowIterator()
Get a row iterator over the frame where all fields are encoded
as boxed objects according to their value types.
|
Iterator<Object[]> |
getObjectRowIterator(int rl,
int ru)
Get a row iterator over the frame where all fields are encoded
as boxed objects according to their value types.
|
HashMap<String,Long> |
getRecodeMap(int col)
This function will split every Recode map in the column using delimiter Lop.DATATYPE_PREFIX,
as Recode map generated earlier in the form of Code+Lop.DATATYPE_PREFIX+Token and store it in a map
which contains token and code for every unique tokens.
|
org.apache.sysml.parser.Expression.ValueType[] |
getSchema()
Returns the schema of the frame block.
|
Iterator<String[]> |
getStringRowIterator()
Get a row iterator over the frame where all fields are encoded
as strings independent of their value types.
|
Iterator<String[]> |
getStringRowIterator(int rl,
int ru)
Get a row iterator over the frame where all fields are encoded
as strings independent of their value types.
|
boolean |
isColNameDefault(int i) |
boolean |
isColNamesDefault() |
boolean |
isColumnMetadataDefault() |
boolean |
isColumnMetadataDefault(int c) |
boolean |
isShallowSerialize()
Indicates if the cache block is subject to shallow serialized,
which is generally true if in-memory size and serialized size
are almost identical allowing to avoid unnecessary deep serialize.
|
FrameBlock |
leftIndexingOperations(FrameBlock rhsFrame,
org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
leftIndexingOperations(FrameBlock rhsFrame,
int rl,
int ru,
int cl,
int cu,
FrameBlock ret) |
void |
merge(org.apache.sysml.runtime.controlprogram.caching.CacheBlock that,
boolean bDummy)
Merge the given block into the current block.
|
void |
merge(FrameBlock that) |
void |
readExternal(ObjectInput in) |
void |
readFields(DataInput in) |
void |
recomputeColumnCardinality() |
void |
reset() |
void |
reset(int nrow,
boolean clearMeta) |
void |
set(int r,
int c,
Object val)
Sets the value in position (r,c), where the input is assumed
to be a boxed object consistent with the schema definition.
|
void |
setColumnMetadata(FrameBlock.ColumnMetadata[] colmeta) |
void |
setColumnMetadata(int c,
FrameBlock.ColumnMetadata colmeta) |
void |
setColumnNames(String[] colnames) |
void |
setNumRows(int numRows) |
void |
setSchema(org.apache.sysml.parser.Expression.ValueType[] schema)
Sets the schema of the frame block.
|
void |
sliceOperations(ArrayList<Pair<Long,FrameBlock>> outlist,
org.apache.sysml.runtime.util.IndexRange range,
int rowCut) |
FrameBlock |
sliceOperations(org.apache.sysml.runtime.util.IndexRange ixrange,
FrameBlock ret) |
FrameBlock |
sliceOperations(int rl,
int ru,
int cl,
int cu,
org.apache.sysml.runtime.controlprogram.caching.CacheBlock retCache)
Right indexing operations to slice a subframe out of this frame block.
|
void |
write(DataOutput out) |
void |
writeExternal(ObjectOutput out) |
FrameBlock |
zeroOutOperations(FrameBlock result,
org.apache.sysml.runtime.util.IndexRange range,
boolean complementary,
int iRowStartSrc,
int iRowStartDest,
int brlen,
int iMaxRowsToCopy)
This function ZERO OUT the data in the slicing window applicable for this block.
|
public static final int BUFFER_SIZE
public FrameBlock()
public FrameBlock(FrameBlock that)
that
- frame blockpublic FrameBlock(int ncols, org.apache.sysml.parser.Expression.ValueType vt)
public FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema)
public FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema, String[] names)
public FrameBlock(org.apache.sysml.parser.Expression.ValueType[] schema, String[][] data)
public int getNumRows()
getNumRows
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public void setNumRows(int numRows)
public int getNumColumns()
getNumColumns
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public org.apache.sysml.parser.Expression.ValueType[] getSchema()
public void setSchema(org.apache.sysml.parser.Expression.ValueType[] schema)
schema
- schema as array of ValueTypespublic String[] getColumnNames()
public String[] getColumnNames(boolean alloc)
alloc
- if true, create column namespublic String getColumnName(int c)
c
- column indexpublic void setColumnNames(String[] colnames)
public FrameBlock.ColumnMetadata[] getColumnMetadata()
public FrameBlock.ColumnMetadata getColumnMetadata(int c)
public boolean isColumnMetadataDefault()
public boolean isColumnMetadataDefault(int c)
public void setColumnMetadata(FrameBlock.ColumnMetadata[] colmeta)
public void setColumnMetadata(int c, FrameBlock.ColumnMetadata colmeta)
public Map<String,Integer> getColumnNameIDMap()
public void ensureAllocatedColumns(int numRows)
numRows
- number of rowspublic void ensureColumnCompatibility(int newlen)
newlen
- number of rows to compare with existing number of rowspublic static String[] createColNames(int size)
public static String[] createColNames(int off, int size)
public static String createColName(int i)
public boolean isColNamesDefault()
public boolean isColNameDefault(int i)
public void recomputeColumnCardinality()
public Object get(int r, int c)
r
- row index, 0-basedc
- column index, 0-basedpublic void set(int r, int c, Object val)
r
- row indexc
- column indexval
- value to set at specified positionpublic void reset(int nrow, boolean clearMeta)
public void reset()
public void appendRow(Object[] row)
row
- array of objectspublic void appendRow(String[] row)
row
- array of stringspublic void appendColumn(String[] col)
col
- array of stringspublic void appendColumn(boolean[] col)
col
- array of booleanspublic void appendColumn(long[] col)
col
- array of longspublic void appendColumn(double[] col)
col
- array of doublespublic void appendColumns(double[][] cols)
cols
- 2d array of doublespublic Object getColumn(int c)
public Iterator<String[]> getStringRowIterator()
public Iterator<String[]> getStringRowIterator(int rl, int ru)
rl
- lower row indexru
- upper row indexpublic Iterator<Object[]> getObjectRowIterator()
public Iterator<Object[]> getObjectRowIterator(int rl, int ru)
rl
- lower row indexru
- upper row indexpublic void write(DataOutput out) throws IOException
write
in interface org.apache.hadoop.io.Writable
IOException
public void readFields(DataInput in) throws IOException
readFields
in interface org.apache.hadoop.io.Writable
IOException
public void writeExternal(ObjectOutput out) throws IOException
writeExternal
in interface Externalizable
IOException
public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException
readExternal
in interface Externalizable
IOException
ClassNotFoundException
public long getInMemorySize()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
getInMemorySize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public long getExactSerializedSize()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
getExactSerializedSize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public boolean isShallowSerialize()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
isShallowSerialize
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public void compactEmptyBlock()
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
compactEmptyBlock
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
public FrameBlock leftIndexingOperations(FrameBlock rhsFrame, org.apache.sysml.runtime.util.IndexRange ixrange, FrameBlock ret) throws DMLRuntimeException
DMLRuntimeException
public FrameBlock leftIndexingOperations(FrameBlock rhsFrame, int rl, int ru, int cl, int cu, FrameBlock ret) throws DMLRuntimeException
DMLRuntimeException
public FrameBlock sliceOperations(org.apache.sysml.runtime.util.IndexRange ixrange, FrameBlock ret) throws DMLRuntimeException
DMLRuntimeException
public FrameBlock sliceOperations(int rl, int ru, int cl, int cu, org.apache.sysml.runtime.controlprogram.caching.CacheBlock retCache) throws DMLRuntimeException
sliceOperations
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
rl
- row lower index, inclusive, 0-basedru
- row upper index, inclusive, 0-basedcl
- column lower index, inclusive, 0-basedcu
- column upper index, inclusive, 0-basedretCache
- cache blockDMLRuntimeException
- if DMLRuntimeException occurspublic void sliceOperations(ArrayList<Pair<Long,FrameBlock>> outlist, org.apache.sysml.runtime.util.IndexRange range, int rowCut)
public FrameBlock appendOperations(FrameBlock that, FrameBlock ret, boolean cbind) throws DMLRuntimeException
that
- frame block to append to current frame blockret
- frame block to return, can be nullcbind
- if true, column appendDMLRuntimeException
- if DMLRuntimeException occurspublic void copy(FrameBlock src)
public void copy(int rl, int ru, int cl, int cu, FrameBlock src)
public HashMap<String,Long> getRecodeMap(int col)
col
- is the column # from frame data which contains Recode map generated earlier.public void merge(org.apache.sysml.runtime.controlprogram.caching.CacheBlock that, boolean bDummy) throws DMLRuntimeException
org.apache.sysml.runtime.controlprogram.caching.CacheBlock
merge
in interface org.apache.sysml.runtime.controlprogram.caching.CacheBlock
that
- cache blockbDummy
- ?DMLRuntimeException
- if DMLRuntimeException occurspublic void merge(FrameBlock that) throws DMLRuntimeException
DMLRuntimeException
public FrameBlock zeroOutOperations(FrameBlock result, org.apache.sysml.runtime.util.IndexRange range, boolean complementary, int iRowStartSrc, int iRowStartDest, int brlen, int iMaxRowsToCopy) throws DMLRuntimeException
result
- frame blockrange
- index rangecomplementary
- ?iRowStartSrc
- ?iRowStartDest
- ?brlen
- ?iMaxRowsToCopy
- ?DMLRuntimeException
- if DMLRuntimeException occursCopyright © 2017 The Apache Software Foundation. All rights reserved.