Ingestion scheduler reference¶

Database objects and procedures¶

The following procedures are created by the file scheduler/scheduler.sql.

PUBLIC.CREATE_SCHEDULER()¶

This procedure acts as the entry point between SQL and Java. It will create a task running according to the schedule available in APP_STATE table. This task will execute below PUBLIC.RUN_SCHEDULER_ITERATION() procedure when executed.

PUBLIC.RUN_SCHEDULER_ITERATION()¶

This procedure is an entry point to the Java implementation of the actual scheduling task. It will be invoked whenever the scheduler task is executed.

PUBLIC.ON_INGESTION_SCHEDULED (process_id VARCHAR)¶

This procedure defines the ingestion flow for a single process that was taken by the scheduler for execution. The default implementation does nothing. We recommend implementing this in Java using the OnIngestionScheduledCallback interface.

Custom handler¶

Ingestion scheduler feature consists of two different handlers acting as entry point from SQL to Java:

CreateSchedulerHandler
RunSchedulerIterationHandler

We recommend customizing only the latter one.

Builder approach¶

RunSchedulerIterationHandler can be customized using RunSchedulerIterationHandlerBuilder. This helper objects allows for custom implementations of the underlying interfaces:

ConnectorErrorHelper
OnIngestionScheduledCallback

In case they are not provided the default implementations will be used.

class CustomOnIngestionScheduledCallback implements OnIngestionScheduledCallback {
    @Override
    public void onIngestionScheduled(String processId) {
        // CUSTOM LOGIC
    }
}

class CustomHandler {

    // Path to this method needs to be specified in the PUBLIC.RUN_SCHEDULER_ITERATION procedure using SQL
    public static Variant runIteration(Session session) {
        return RunSchedulerIterationHandler.builder(session)
            .withOnIngestionScheduledCallback(new CustomOnIngestionScheduledCallback())
            .build()
            .runIteration()
            .toVariant();
    }
}

Copy