Package scriptella.execution

Class EtlExecutor

java.lang.Object
scriptella.execution.EtlExecutor
All Implemented Interfaces:
Runnable, Callable<ExecutionStatistics>
Direct Known Subclasses:
EtlExecutorBean
public class EtlExecutor extends Object implements Runnable, Callable<ExecutionStatistics>
Executes a Scriptella ETL configuration loaded from XML or created in memory.

The simplest way to run an ETL XML file is:


 EtlExecutor executor = EtlExecutor.newExecutor(new File("etl.xml"));
 ExecutionStatistics statistics = executor.execute();

newExecutor(File) and newExecutor(URL) use the current system properties as external ETL parameters. To supply parameters explicitly, use newExecutor(URL, Map). External parameters take precedence over properties declared in the ETL file.

Direct SQL file execution

For a single SQL file, use newSqlFileExecutor(File, String, String, String) to create a normal executor with one JDBC connection and one script in memory. No ETL XML file is needed:


 EtlExecutor executor = EtlExecutor.newSqlFileExecutor(
     new File("schema.sql"), "jdbc:postgresql://localhost/app", "app", "secret");
 ExecutionStatistics statistics = executor.execute();
 long updatedRows = statistics.getUpdateCount();

SQL files are read as UTF-8 at execution time. JDBC driver JARs must be on the classpath. As with the default XML factories, SQL variables come from JVM system properties, captured when the factory is called. Scriptella's normal SQL substitution is enabled by default. Set substitution to false to preserve dollar and question-mark expressions literally. Use ?name bindings for data values; text substitutions such as ${name} do not escape SQL.

The returned executor uses the same transactions, cleanup, progress, statistics, optional JMX, and cancellation as an XML-based executor. Execution commits on success and attempts rollback on failure; database DDL may not support rollback. SQL files support updates and DDL. Use an ETL query element when results need to be processed.

For more control, use ConfigurationFactory to parse the ETL file, then pass the resulting ConfigurationEl to EtlExecutor(ConfigurationEl).

ETL Cancellation

Scriptella uses the standard Java Thread.interrupt() mechanism. Interrupt the thread that is running execute(); the engine then attempts to roll back changes made during the ETL operation. A cancellation is reported as an EtlExecutorException for which EtlExecutorException.isCancelled() returns true.

ExecutorService and Future can also be used to submit and cancel an execution.

Integration with third-party systems

For convenience, EtlExecutor implements Runnable and Callable. Use call() when execution statistics or the checked EtlExecutorException are needed. run() discards the statistics and wraps execution failures in a SystemException.

Version:
1.0
Author:
Fyodor Kupolov

  • Constructor Details

    • EtlExecutor

      public EtlExecutor()
      Creates an ETL executor without a configuration. Call setConfiguration(ConfigurationEl) before executing it.

    • EtlExecutor

      public EtlExecutor(ConfigurationEl configuration)
      Creates an ETL executor for a parsed or programmatically constructed configuration.
      Parameters:
      configuration - ETL configuration.

  • Method Details

    • getConfiguration

      public ConfigurationEl getConfiguration()
      Returns ETL configuration for this executor.
      Returns:
      ETL configuration.

    • setConfiguration

      public void setConfiguration(ConfigurationEl configuration)
      Sets ETL configuration.
      Parameters:
      configuration - ETL configuration.

    • isJmxEnabled

      public boolean isJmxEnabled()
      Returns true if monitoring/management via JMX is enabled.

      If jmxEnabled=true the executor registers MBeans for executed ETL files. The object names of the mbeans have the following form: scriptella: type=etl,url="ETL_FILE_URL"

      Returns:
      true if monitoring/management via JMX is enabled.

    • setJmxEnabled

      public void setJmxEnabled(boolean jmxEnabled)
      Enables or disables ETL monitoring/management via JMX.

      If jmxEnabled=true the executor registers MBeans for executed ETL files. The object names of the mbeans have the following form: scriptella: type=etl,url="ETL_FILE_URL"

      Parameters:
      jmxEnabled - true if monitoring/management via JMX is enabled.
      See Also:

    • isSuppressStatistics

      public boolean isSuppressStatistics()
      Getter for suppressStatistics property.
      Returns:
      true if statistics collection is disabled. Default value is false.

    • setSuppressStatistics

      public void setSuppressStatistics(boolean suppressStatistics)
      Enables or disables collecting of statistics. Default value is false, which means statistics is collected.

      Setting this option to true may improve performance in some cases.

      Parameters:
      suppressStatistics - true if statistics collection should be disabled.

    • execute

      public ExecutionStatistics execute() throws EtlExecutorException
      Executes ETL based on a specified configuration.
      Returns:
      execution statistics for ETL execution.
      Throws:
      EtlExecutorException - if ETL fails.
      See Also:

    • execute

      public ExecutionStatistics execute(ProgressIndicator indicator) throws EtlExecutorException
      Executes ETL based on a specified configuration.
      Parameters:
      indicator - progress indicator to use, or null if progress reporting is not required.
      Returns:
      execution statistics for ETL execution.
      Throws:
      EtlExecutorException - if ETL fails.

    • prepare

      protected EtlContext prepare(ProgressIndicator indicator)
      Prepares the scripts context.
      Parameters:
      indicator - progress indicator to use.
      Returns:
      prepared scripts context.

    • newExecutor

      public static EtlExecutor newExecutor(File scriptFile)
      Converts file to URL and invokes newExecutor(java.net.URL).
      Parameters:
      scriptFile - ETL file.
      Returns:
      configured instance of script executor.
      See Also:

    • newExecutor

      public static EtlExecutor newExecutor(URL scriptFileUrl)
      Creates an ETL executor for the specified script URL.

      Calls newExecutor(java.net.URL, java.util.Map) and passes System properties as external properties.

      Parameters:
      scriptFileUrl - URL of the ETL file.
      Returns:
      configured instance of script executor.

    • newExecutor

      public static EtlExecutor newExecutor(URL scriptFileUrl, Map<String,?> externalProperties)
      Creates an ETL executor for the specified script URL and external parameters.
      Parameters:
      scriptFileUrl - URL of the ETL file.
      externalProperties - external ETL parameters, or null for none; these take precedence over properties in the ETL file.
      Returns:
      configured instance of script executor.
      See Also:

    • newSqlFileExecutor

      public static EtlExecutor newSqlFileExecutor(File file, String url)
      Creates a SQL file executor without explicit JDBC credentials.
      Parameters:
      file - UTF-8 SQL file
      url - JDBC connection URL
      Returns:
      a normal ETL executor
      See Also:

    • newSqlFileExecutor

      public static EtlExecutor newSqlFileExecutor(File file, String url, String user, String password)
      Creates an executor for a UTF-8 SQL file and a JDBC URL, without ETL XML. System properties are available for SQL substitution. No connection is opened until execution. JDBC driver JARs must be on the classpath.
      Parameters:
      file - UTF-8 SQL file
      url - JDBC connection URL
      user - optional username
      password - optional password
      Returns:
      a normal ETL executor
      See Also:

    • newSqlFileExecutor

      public static EtlExecutor newSqlFileExecutor(File file, String url, String user, String password, String driver, boolean substitution)
      Creates a SQL file executor using the normal ETL execution lifecycle. Execution commits on success and attempts rollback on failure; DDL rollback depends on the database. SELECT results are unsupported; use an ETL query. SQL variables come from a snapshot of JVM system properties. Text substitution does not escape SQL; use ?name bindings for data values. The JDBC URL, user, password, and driver accept the reserved ${env.NAME} namespace. Connection settings expand even when substitution is false.
      Parameters:
      file - UTF-8 SQL file, read at execution time
      url - JDBC connection URL
      user - optional username
      password - optional password
      driver - optional JDBC driver class; null uses JDBC automatic registration
      substitution - whether Scriptella SQL substitution is enabled
      Returns:
      a normal ETL executor supporting execute(), progress, JMX, and cancellation
      Throws:
      IllegalArgumentException - if file is null or the expanded URL is missing or does not start with jdbc:

    • run

      public void run() throws SystemException
      A Runnable adapter for execute().

      Because Runnable.run() cannot declare checked exceptions, this method wraps an EtlExecutorException in a SystemException.

      Specified by:
      run in interface Runnable
      Throws:
      SystemException - a wrapped EtlExecutorException.
      See Also:

    • call

      public ExecutionStatistics call() throws EtlExecutorException
      A Callable adapter for execute().
      Specified by:
      call in interface Callable<ExecutionStatistics>
      Returns:
      execution statistics for the ETL execution.
      Throws:
      EtlExecutorException - if ETL execution fails.