org.apache.sysml.runtime.instructions

Class MRJobInstruction

java.lang.Object

org.apache.sysml.runtime.instructions.Instruction

org.apache.sysml.runtime.instructions.MRJobInstruction

public class MRJobInstruction
extends Instruction

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.sysml.runtime.instructions.Instruction

Instruction.INSTRUCTION_TYPE

Field Summary

Fields inherited from class org.apache.sysml.runtime.instructions.Instruction

beginCol, beginLine, DATATYPE_PREFIX, endCol, endLine, GPU_INST_PREFIX, instOpcode, INSTRUCTION_DELIM, instString, LITERAL_PREFIX, LOG, OPERAND_DELIM, SP_INST_PREFIX, type, VALUETYPE_PREFIX

Constructor Summary

Constructors

Constructor and Description
MRJobInstruction(org.apache.sysml.lops.compile.JobType type)
MRJobInstruction(MRJobInstruction that) (deep) Copy constructor, primarily used in parfor.

Method Summary

Modifier and Type	Method and Description
org.apache.sysml.runtime.controlprogram.caching.MatrixObject[]	extractInputMatrices(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec) Extracts input variables with MATRIX data type, and stores references to corresponding matrix objects in inputMatrices.
org.apache.sysml.runtime.controlprogram.caching.MatrixObject[]	extractOutputMatrices(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec) Extracts MatrixObject references to output variables, all of which will be of MATRIX data type, and stores them in outputMatrices.
boolean	findMRInstructions(int lineNum) Search whether or not this MR job contains at least one MR instruction with specified line number parameter
int[]	getBclens()
int[]	getBrlens()
long[]	getClens()
String	getDimsUnknownFilePrefix()
String	getGraphString()
InputInfo[]	getInputInfos()
org.apache.sysml.runtime.controlprogram.caching.MatrixObject[]	getInputMatrices()
String[]	getInputs()
String[]	getInputVars()
String	getIv_aggInstructions()
String	getIv_instructionsInMapper()
int	getIv_numReducers()
String	getIv_otherInstructions()
String	getIv_randInstructions()
String	getIv_recordReaderInstructions()
int	getIv_replication()
byte[]	getIv_resultIndices()
String	getIv_shuffleInstructions()
org.apache.sysml.lops.compile.JobType	getJobType()
double	getMemoryRequirements()
ArrayList<Integer>	getMRJobInstructionsLineNumbers() Getter for MRJobInstructionslineNumbers
String	getMRString(boolean debug) Method for displaying MR instructions interspersed with source code ONLY USED IN DEBUG MODE
OutputInfo[]	getOutputInfos()
org.apache.sysml.runtime.controlprogram.caching.MatrixObject[]	getOutputMatrices()
String[]	getOutputs()
String[]	getOutputVars()
boolean[]	getPartitioned()
org.apache.sysml.runtime.controlprogram.ParForProgramBlock.PDataPartitionFormat[]	getPformats()
int[]	getPsizes()
long[]	getRlens()
String	getString(byte[] arr)
boolean	isMapOnly()
boolean	isMergableMRJobInstruction(MRJobInstruction that)
void	mergeMRJobInstruction(MRJobInstruction that)
void	printCompleteMRJobInstruction(MatrixCharacteristics[] resultStats)
void	printMe()
void	processInstruction(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec) This method should be used to execute the instruction.
void	setAggregateInstructionsInReducer(String aggInstructions)
void	setDimsUnknownFilePrefix(String prefix)
void	setGMRInstructions(String[] inLabels, String recordReaderInstructions, String mapperInstructions, String aggInstructions, String otherInstructions, String[] outLabels, byte[] resultIndex, int numReducers, int replication)
void	setInputOutputLabels(String[] inputLabels, String[] outputLabels) Method to set input and output labels for a MapReduce instruction.
void	setIv_aggInstructions(String inst)
void	setIv_instructionsInMapper(String inst)
void	setIv_otherInstructions(String inst)
void	setIv_randInstructions(String inst)
void	setIv_recordReaderInstructions(String inst)
void	setIv_shuffleInstructions(String inst)
void	setMapperInstructions(String mapperInstructions)
void	setMemoryRequirements(double mem)
void	setMRJobInstructionsLineNumbers(ArrayList<Integer> MRJobLineNumbers) Setter for MRJobInstructionslineNumbers field
void	setNumberOfReducers(int numReducers) Method to set the number of reducers for a MapReducer instruction.
void	setOtherInstructionsInReducer(String otherInstructions)
void	setOutputs(byte[] outputIndices) Method to set outputs (output indices) for a MapReduce instruction.
void	setPartitioned(boolean[] partitioned)
void	setPformats(org.apache.sysml.runtime.controlprogram.ParForProgramBlock.PDataPartitionFormat[] pformats)
void	setPsizes(int[] psizes)
void	setRandInstructions(String randInstructions)
void	setReBlockInstructions(String[] inLabels, String mapperInstructions, String reblockInstructions, String otherInstructions, String[] outLabels, byte[] resultIndex, int numReducers, int replication)
void	setRecordReaderInstructions(String rrInstructions)
void	setReplication(int replication) Method to set the replication factor for outputs produced from a MapReduce instruction.
void	setShuffleInstructions(String shuffleInstructions)
String	toString()
void	updateInstructionThreadID(String pattern, String replace) All instructions that have thread-specific filenames or names encoded in it should overwrite this method in order to update (1) the in-memory instruction and (2) the instruction string

Methods inherited from class org.apache.sysml.runtime.instructions.Instruction

getExtendedOpcode, getInstID, getLineNum, getOpcode, getType, postprocessInstruction, preprocessInstruction, requiresLabelUpdate, setInstID, setLocation, setLocation, setLocation, setLocation, setType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

MRJobInstruction

public MRJobInstruction(org.apache.sysml.lops.compile.JobType type)

MRJobInstruction

public MRJobInstruction(MRJobInstruction that)
                 throws IllegalArgumentException,
                        IllegalAccessException

(deep) Copy constructor, primarily used in parfor. Additionally, replace all occurrences of srcPattern with targetPattern

Parameters:

that - MR job instruction

Throws:

IllegalArgumentException - if IllegalArgumentException occurs

IllegalAccessException - if IllegalAccessException occurs

Method Detail

getJobType

public org.apache.sysml.lops.compile.JobType getJobType()

getIv_instructionsInMapper

public String getIv_instructionsInMapper()

setIv_instructionsInMapper

public void setIv_instructionsInMapper(String inst)

getIv_recordReaderInstructions

public String getIv_recordReaderInstructions()

setIv_recordReaderInstructions

public void setIv_recordReaderInstructions(String inst)

getIv_randInstructions

public String getIv_randInstructions()

setIv_randInstructions

public void setIv_randInstructions(String inst)

getIv_shuffleInstructions

public String getIv_shuffleInstructions()

setIv_shuffleInstructions

public void setIv_shuffleInstructions(String inst)

getIv_aggInstructions

public String getIv_aggInstructions()

setIv_aggInstructions

public void setIv_aggInstructions(String inst)

getIv_otherInstructions

public String getIv_otherInstructions()

setIv_otherInstructions

public void setIv_otherInstructions(String inst)

getIv_resultIndices

public byte[] getIv_resultIndices()

getIv_numReducers

public int getIv_numReducers()

getIv_replication

public int getIv_replication()

getMemoryRequirements

public double getMemoryRequirements()

setMemoryRequirements

public void setMemoryRequirements(double mem)

getDimsUnknownFilePrefix

public String getDimsUnknownFilePrefix()

setDimsUnknownFilePrefix

public void setDimsUnknownFilePrefix(String prefix)

getInputVars

public String[] getInputVars()

getOutputVars

public String[] getOutputVars()

getMRJobInstructionsLineNumbers

public ArrayList<Integer> getMRJobInstructionsLineNumbers()

Getter for MRJobInstructionslineNumbers

Returns:

list containing all instructions indexed by line number

setOutputs

public void setOutputs(byte[] outputIndices)

Method to set outputs (output indices) for a MapReduce instruction.

Parameters:

outputIndices - output indices

setNumberOfReducers

public void setNumberOfReducers(int numReducers)

Method to set the number of reducers for a MapReducer instruction.

Parameters:

numReducers - number of reducers

setReplication

public void setReplication(int replication)

Method to set the replication factor for outputs produced from a MapReduce instruction.

Parameters:

replication - replication factor

setInputOutputLabels

public void setInputOutputLabels(String[] inputLabels,
                                 String[] outputLabels)

Method to set input and output labels for a MapReduce instruction.

Parameters:

inputLabels - input labels

outputLabels - output labels

setRecordReaderInstructions

public void setRecordReaderInstructions(String rrInstructions)

setMapperInstructions

public void setMapperInstructions(String mapperInstructions)

setShuffleInstructions

public void setShuffleInstructions(String shuffleInstructions)

setAggregateInstructionsInReducer

public void setAggregateInstructionsInReducer(String aggInstructions)

setOtherInstructionsInReducer

public void setOtherInstructionsInReducer(String otherInstructions)

setRandInstructions

public void setRandInstructions(String randInstructions)

setMRJobInstructionsLineNumbers

public void setMRJobInstructionsLineNumbers(ArrayList<Integer> MRJobLineNumbers)

Setter for MRJobInstructionslineNumbers field

Parameters:

MRJobLineNumbers - Line numbers for each instruction in this MRJob

setGMRInstructions

public void setGMRInstructions(String[] inLabels,
                               String recordReaderInstructions,
                               String mapperInstructions,
                               String aggInstructions,
                               String otherInstructions,
                               String[] outLabels,
                               byte[] resultIndex,
                               int numReducers,
                               int replication)

setReBlockInstructions

public void setReBlockInstructions(String[] inLabels,
                                   String mapperInstructions,
                                   String reblockInstructions,
                                   String otherInstructions,
                                   String[] outLabels,
                                   byte[] resultIndex,
                                   int numReducers,
                                   int replication)

findMRInstructions

public boolean findMRInstructions(int lineNum)

Search whether or not this MR job contains at least one MR instruction with specified line number parameter

Parameters:

lineNum - Line number in DML script

Returns:

Return true if found, otherwise return false

getString

public String getString(byte[] arr)

toString

public String toString()

Overrides:

toString in class Instruction

getMRString

public String getMRString(boolean debug)

Method for displaying MR instructions interspersed with source code ONLY USED IN DEBUG MODE

Parameters:

debug - Flag for displaying instructions in debugger test integration

Returns:

MR string

printMe

public void printMe()

Overrides:

printMe in class Instruction

getGraphString

public String getGraphString()

Overrides:

getGraphString in class Instruction

isMapOnly

public boolean isMapOnly()

getInputs

public String[] getInputs()

getInputInfos

public InputInfo[] getInputInfos()

getRlens

public long[] getRlens()

getClens

public long[] getClens()

getBrlens

public int[] getBrlens()

getBclens

public int[] getBclens()

getOutputs

public String[] getOutputs()

getOutputInfos

public OutputInfo[] getOutputInfos()

getInputMatrices

public org.apache.sysml.runtime.controlprogram.caching.MatrixObject[] getInputMatrices()

getPartitioned

public boolean[] getPartitioned()

setPartitioned

public void setPartitioned(boolean[] partitioned)

getPformats

public org.apache.sysml.runtime.controlprogram.ParForProgramBlock.PDataPartitionFormat[] getPformats()

setPformats

public void setPformats(org.apache.sysml.runtime.controlprogram.ParForProgramBlock.PDataPartitionFormat[] pformats)

getPsizes

public int[] getPsizes()

setPsizes

public void setPsizes(int[] psizes)

extractInputMatrices

public org.apache.sysml.runtime.controlprogram.caching.MatrixObject[] extractInputMatrices(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec)

Extracts input variables with MATRIX data type, and stores references to corresponding matrix objects in inputMatrices. Also, stores the data types in inputDataTypes.

Parameters:

ec - execution context

Returns:

array of matrix objects

getOutputMatrices

public org.apache.sysml.runtime.controlprogram.caching.MatrixObject[] getOutputMatrices()

extractOutputMatrices

public org.apache.sysml.runtime.controlprogram.caching.MatrixObject[] extractOutputMatrices(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec)
                                                                                     throws DMLRuntimeException

Extracts MatrixObject references to output variables, all of which will be of MATRIX data type, and stores them in outputMatrices. Also, populates auxiliary data structures.

Parameters:

ec - execution context

Returns:

array of matrix objects

Throws:

DMLRuntimeException - if DMLRuntimeException occurs

printCompleteMRJobInstruction

public void printCompleteMRJobInstruction(MatrixCharacteristics[] resultStats)
                                   throws DMLRuntimeException

Throws:

DMLRuntimeException

updateInstructionThreadID

public void updateInstructionThreadID(String pattern,
                                      String replace)
                               throws DMLRuntimeException

Description copied from class: Instruction

All instructions that have thread-specific filenames or names encoded in it should overwrite this method in order to update (1) the in-memory instruction and (2) the instruction string

Overrides:

updateInstructionThreadID in class Instruction

Parameters:

pattern - ?

replace - ?

Throws:

DMLRuntimeException - if DMLRuntimeException occurs

isMergableMRJobInstruction

public boolean isMergableMRJobInstruction(MRJobInstruction that)

mergeMRJobInstruction

public void mergeMRJobInstruction(MRJobInstruction that)

processInstruction

public void processInstruction(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec)
                        throws DMLRuntimeException

Description copied from class: Instruction

This method should be used to execute the instruction.

Specified by:

processInstruction in class Instruction

Parameters:

ec - execution context

Throws:

DMLRuntimeException - if DMLRuntimeException occurs