org.apache.sysml.runtime.instructions.spark

Class MatrixIndexingSPInstruction



  • public class MatrixIndexingSPInstruction
    extends IndexingSPInstruction
    This class implements the matrix indexing functionality inside CP. Example instructions: rangeReIndex:mVar1:Var2:Var3:Var4:Var5:mVar6 input=mVar1, output=mVar6, bounds = (Var2,Var3,Var4,Var5) rowindex_lower: Var2, rowindex_upper: Var3 colindex_lower: Var4, colindex_upper: Var5 leftIndex:mVar1:mVar2:Var3:Var4:Var5:Var6:mVar7 triggered by "mVar1[Var3:Var4, Var5:Var6] = mVar2" the result is stored in mVar7
    • Constructor Detail

      • MatrixIndexingSPInstruction

        public MatrixIndexingSPInstruction(org.apache.sysml.runtime.matrix.operators.Operator op,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand in,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand rl,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand ru,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand cl,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand cu,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand out,
                                           org.apache.sysml.hops.AggBinaryOp.SparkAggType aggtype,
                                           String opcode,
                                           String istr)
      • MatrixIndexingSPInstruction

        public MatrixIndexingSPInstruction(org.apache.sysml.runtime.matrix.operators.Operator op,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand lhsInput,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand rhsInput,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand rl,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand ru,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand cl,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand cu,
                                           org.apache.sysml.runtime.instructions.cp.CPOperand out,
                                           org.apache.sysml.lops.LeftIndex.LixCacheType type,
                                           String opcode,
                                           String istr)
    • Method Detail

      • processInstruction

        public void processInstruction(org.apache.sysml.runtime.controlprogram.context.ExecutionContext ec)
                                throws DMLRuntimeException
        Description copied from class: Instruction
        This method should be used to execute the instruction.
        Specified by:
        processInstruction in class SPInstruction
        Parameters:
        ec - execution context
        Throws:
        DMLRuntimeException - if DMLRuntimeException occurs
      • isSingleBlockLookup

        public static boolean isSingleBlockLookup(MatrixCharacteristics mcIn,
                                                  org.apache.sysml.runtime.util.IndexRange ixrange)
        Indicates if the given index range only covers a single blocks of the inputs matrix. In this case, we perform a key lookup which is very efficient in case of existing partitioner, especially for out-of-core datasets.
        Parameters:
        mcIn - matrix characteristics
        ixrange - index range
        Returns:
        true if index range covers a single block of the input matrix
      • isMultiBlockLookup

        public static boolean isMultiBlockLookup(org.apache.spark.api.java.JavaPairRDD<?,?> in,
                                                 MatrixCharacteristics mcIn,
                                                 MatrixCharacteristics mcOut,
                                                 org.apache.sysml.runtime.util.IndexRange ixrange)
        Indicates if the given index range and input matrix exhibit the following properties: (1) existing hash partitioner, (2) out-of-core input matrix (larger than aggregated memory), (3) aligned indexing range (which does not required aggregation), and (4) the output fits twice in memory (in order to collect the result).
        Parameters:
        in - input matrix
        mcIn - input matrix characteristics
        mcOut - output matrix characteristics
        ixrange - index range
        Returns:
        true if index range requires a multi-block lookup

Copyright © 2017 The Apache Software Foundation. All rights reserved.