org.apache.sysml.runtime.instructions.spark.utils

Class RDDSortUtils

  • java.lang.Object
    • org.apache.sysml.runtime.instructions.spark.utils.RDDSortUtils


  • public class RDDSortUtils
    extends Object
    • Constructor Detail

      • RDDSortUtils

        public RDDSortUtils()
    • Method Detail

      • sortDataByValMemSort

        public static org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> sortDataByValMemSort(org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> val,
                                                                                                            org.apache.spark.api.java.JavaPairRDD<MatrixIndexes,MatrixBlock> data,
                                                                                                            boolean asc,
                                                                                                            long rlen,
                                                                                                            long clen,
                                                                                                            int brlen,
                                                                                                            int bclen,
                                                                                                            org.apache.sysml.runtime.controlprogram.context.SparkExecutionContext sec,
                                                                                                            org.apache.sysml.runtime.matrix.operators.ReorgOperator r_op)
                                                                                                     throws DMLRuntimeException
        This function collects and sorts value column in memory and then broadcasts it.
        Parameters:
        val - value as JavaPairRDD<MatrixIndexes, MatrixBlock>
        data - data as JavaPairRDD<MatrixIndexes, MatrixBlock>
        asc - if true, sort ascending
        rlen - number of rows
        clen - number of columns
        brlen - number of rows in a block
        bclen - number of columns in a block
        sec - spark execution context
        r_op - reorg operator
        Returns:
        data as JavaPairRDD<MatrixIndexes, MatrixBlock>
        Throws:
        DMLRuntimeException - if DMLRuntimeException occurs

Copyright © 2017 The Apache Software Foundation. All rights reserved.