org.apache.sysml.api.mlcontext

Class MLContext



  • public class MLContext
    extends Object
    The MLContext API offers programmatic access to SystemML on Spark from languages such as Scala, Java, and Python.
    • Field Summary

      Fields 
      Modifier and Type Field and Description
      protected static org.apache.log4j.Logger log
      Logger for MLContext
    • Constructor Summary

      Constructors 
      Constructor and Description
      MLContext(org.apache.spark.api.java.JavaSparkContext javaSparkContext)
      Create an MLContext based on a JavaSparkContext for interaction with SystemML on Spark.
      MLContext(org.apache.spark.SparkContext sparkContext)
      Create an MLContext based on a SparkContext for interaction with SystemML on Spark.
      MLContext(org.apache.spark.sql.SparkSession spark)
      Create an MLContext based on a SparkSession for interaction with SystemML on Spark.
    • Field Detail

      • log

        protected static org.apache.log4j.Logger log
        Logger for MLContext
    • Constructor Detail

      • MLContext

        public MLContext(org.apache.spark.sql.SparkSession spark)
        Create an MLContext based on a SparkSession for interaction with SystemML on Spark.
        Parameters:
        spark - SparkSession
      • MLContext

        public MLContext(org.apache.spark.SparkContext sparkContext)
        Create an MLContext based on a SparkContext for interaction with SystemML on Spark.
        Parameters:
        sparkContext - SparkContext
      • MLContext

        public MLContext(org.apache.spark.api.java.JavaSparkContext javaSparkContext)
        Create an MLContext based on a JavaSparkContext for interaction with SystemML on Spark.
        Parameters:
        javaSparkContext - JavaSparkContext
    • Method Detail

      • getActiveMLContext

        public static MLContext getActiveMLContext()
        Retrieve the currently active MLContext. This is used internally by SystemML via MLContextProxy.
        Returns:
        the active MLContext
      • resetConfig

        public void resetConfig()
        Reset configuration settings to default settings.
      • setConfigProperty

        public void setConfigProperty(String propertyName,
                                      String propertyValue)
        Set configuration property, such as setConfigProperty("localtmpdir", "/tmp/systemml").
        Parameters:
        propertyName - property name
        propertyValue - property value
      • execute

        public MLResults execute(Script script)
        Execute a DML or PYDML Script.
        Parameters:
        script - The DML or PYDML Script object to execute.
        Returns:
        the results as a MLResults object
      • execute

        public MLResults execute(Script script,
                                 ScriptExecutor scriptExecutor)
        Execute a DML or PYDML Script object using a ScriptExecutor. The ScriptExecutor class can be extended to allow the modification of the default execution pathway.
        Parameters:
        script - the DML or PYDML Script object
        scriptExecutor - the ScriptExecutor that defines the script execution pathway
        Returns:
        the results as a MLResults object
      • setExecutionScript

        public void setExecutionScript(Script executionScript)
        Sets the script that is being executed
        Parameters:
        executionScript - script that is being executed
      • setConfig

        public void setConfig(String configFilePath)
        Set SystemML configuration based on a configuration file.
        Parameters:
        configFilePath - path to the configuration file
      • getSparkSession

        public org.apache.spark.sql.SparkSession getSparkSession()
        Obtain the SparkSession associated with this MLContext.
        Returns:
        the SparkSession associated with this MLContext.
      • isExplain

        public boolean isExplain()
        Whether or not an explanation of the DML/PYDML program should be output to standard output.
        Returns:
        true if explanation should be output, false otherwise
      • setExplain

        public void setExplain(boolean explain)
        Whether or not an explanation of the DML/PYDML program should be output to standard output.
        Parameters:
        explain - true if explanation should be output, false otherwise
      • isMaintainSymbolTable

        public boolean isMaintainSymbolTable()
        Obtain whether or not all values should be maintained in the symbol table after execution.
        Returns:
        true if all values should be maintained in the symbol table, false otherwise
      • setMaintainSymbolTable

        public void setMaintainSymbolTable(boolean maintainSymbolTable)
        Set whether or not all values should be maintained in the symbol table after execution.
        Parameters:
        maintainSymbolTable - true if all values should be maintained in the symbol table, false otherwise
      • setExplainLevel

        public void setExplainLevel(MLContext.ExplainLevel explainLevel)
        Set the level of program explanation that should be displayed if explain is set to true.
        Parameters:
        explainLevel - the level of program explanation
      • setExplainLevel

        public void setExplainLevel(String explainLevel)
        Set the level of program explanation that should be displayed if explain is set to true.
        Parameters:
        explainLevel - string denoting program explanation
      • setGPU

        public void setGPU(boolean enable)
        Whether or not to use (an available) GPU on the driver node. If a GPU is not available, and the GPU mode is set, SystemML will crash when the program is run.
        Parameters:
        enable - true if needs to be enabled, false otherwise
      • setForceGPU

        public void setForceGPU(boolean enable)
        Whether or not to explicitly "force" the usage of GPU. If a GPU is not available, and the GPU mode is set or if available memory on GPU is less, SystemML will crash when the program is run.
        Parameters:
        enable - true if needs to be enabled, false otherwise
      • isGPU

        public boolean isGPU()
        Whether or not the GPU mode is enabled.
        Returns:
        true if enabled, false otherwise
      • isForceGPU

        public boolean isForceGPU()
        Whether or not the "force" GPU mode is enabled.
        Returns:
        true if enabled, false otherwise
      • getInternalProxy

        public MLContext.InternalProxy getInternalProxy()
        Used internally by MLContextProxy.
        Returns:
        InternalProxy object used by MLContextProxy
      • isStatistics

        public boolean isStatistics()
        Whether or not statistics of the DML/PYDML program execution should be output to standard output.
        Returns:
        true if statistics should be output, false otherwise
      • setStatistics

        public void setStatistics(boolean statistics)
        Whether or not statistics of the DML/PYDML program execution should be output to standard output.
        Parameters:
        statistics - true if statistics should be output, false otherwise
      • setStatisticsMaxHeavyHitters

        public void setStatisticsMaxHeavyHitters(int maxHeavyHitters)
        Sets the maximum number of heavy hitters that are printed out as part of the statistics.
        Parameters:
        maxHeavyHitters - maximum number of heavy hitters to print
      • close

        public void close()
        Closes the mlcontext, which includes the cleanup of static and local state as well as scratch space and buffer pool cleanup. Note that the spark context is not explicitly closed to allow external reuse.
      • info

        public ProjectInfo info()
        Obtain information about the project such as version and build time from the manifest in the SystemML jar file.
        Returns:
        information about the project
      • version

        public String version()
        Obtain the SystemML version number.
        Returns:
        the SystemML version number
      • buildTime

        public String buildTime()
        Obtain the SystemML jar file build time.
        Returns:
        the SystemML jar file build time
      • getStatisticsMaxHeavyHitters

        public int getStatisticsMaxHeavyHitters()
        Obtain the maximum number of heavy hitters that are printed out as part of the statistics.
        Returns:
        maximum number of heavy hitters to print
      • isInitBeforeExecution

        public boolean isInitBeforeExecution()
        Whether or not the default ScriptExecutor should be initialized before execution.
        Returns:
        true if ScriptExecutor should be initialized before execution, false otherwise
      • setInitBeforeExecution

        public void setInitBeforeExecution(boolean initBeforeExecution)
        Whether or not the default ScriptExecutor should be initialized before execution.
        Parameters:
        initBeforeExecution - true if ScriptExecutor should be initialized before execution, false otherwise
      • getExecutionType

        public MLContext.ExecutionType getExecutionType()
        Obtain the current execution environment.
        Returns:
        the execution environment
      • setExecutionType

        public void setExecutionType(MLContext.ExecutionType executionType)
        Set the execution environment.
        Parameters:
        executionType - the execution environment

Copyright © 2017 The Apache Software Foundation. All rights reserved.