Renta / Docs / Pipelines

Scheduling

Renta provides flexible and powerful capabilities for automating your data pipeline runs. You can configure a schedule that fits your business requirements—from manual runs to complex schedules using cron expressions.

Schedule types

Renta offers four types of schedules to automate your pipeline runs: Manual, Daily, Interval, and Cron Expression. Each type is designed for specific use cases and provides different levels of automation and flexibility.

Manual

The pipeline is triggered only manually through the interface or via API.

When to use:

  1. For pipelines that run on demand.
  2. For integration with external data orchestration systems (Airflow, Prefect, etc.).
  3. For testing and debugging pipelines.
  4. For one-time data migrations.

Daily

The pipeline runs once a day at a specified time.

When to use:

  1. For daily data synchronization.
  2. To update dashboards and reports at a specific time.
  3. For nightly data loads.

Settings:

ParameterDescription
Update timeThe exact time the pipeline starts (selected from a dropdown list). From 12 AM to 11 PM in 1-hour increments. A total of 24 time options (every hour of the day)

Interval

The pipeline runs at a specified time interval.

When to use:

  1. For frequent data updates (every 15 minutes, every hour).
  2. For streaming analytics and monitoring.
  3. For real-time data synchronization.

Settings:

ParameterDescription
Set intervalThe numeric value of the interval (e.g., 15, 3, 1)
Time unitThe unit of time measurement: Minute(s) — for high-frequency updates, Hour(s) — for updates several times a day, Day(s) — for weekly synchronizations, Week(s) — for rare updates

The minimum recommended interval is 15 minutes. For more frequent updates, consider using streaming solutions.

Cron Expression

The pipeline runs according to a schedule defined by a cron expression. This provides maximum flexibility for setting up complex schedules.

When to use:

  1. For complex schedules (e.g., specific days of the week).
  2. For runs on the first day of the month.
  3. For non-standard timing patterns.

Settings:

ParameterDescription
Cron expressionA cron expression in standard format

Cron expression format:

* * * * *
│ │ │ │ │
│ │ │ │ └─── Day of week (0-7, where 0 and 7 = Sunday)
│ │ │ └───── Month (1-12)
│ │ └─────── Day of month (1-31)
│ └───────── Hour (0-23)
└─────────── Minute (0-59)

Cron expression examples:

ExpressionDescription
0 /3 Every 3 hours (at 0:00, 3:00, 6:00, etc.)
0 0 /3 *Every 3 days at midnight
0 9 1Every Monday at 9:00 AM
0 0 1 On the first day of every month at midnight
0 0 0Every Sunday at midnight
30 14 1-5Monday through Friday at 2:30 PM
0 0 1 1 *January 1st of every year at midnight
/15 *Every 15 minutes

Recommendations for choosing a schedule type

Choosing the right schedule type depends on your business requirements and data update frequency. The table below provides recommendations to help you select the optimal schedule type for different use cases.

RequirementRecommended TypeAlternative
Daily updates at the same timeDailyCron Expression
Updates several times a day with equal intervalsIntervalCron Expression
Complex schedules (specific days of week/month)Cron Expression—
Trigger on demand or from an external systemManual—
Very frequent updates (every 15-30 minutes)Interval—
Updates on business daysCron Expression—

Configuring a pipeline schedule

Follow these steps to set up an automatic run for your pipeline.

Open pipeline settingsStep 1

In the Pipelines section, find the desired pipeline in the list. Click the three-dot menu button (⋮) to the right of the pipeline and select Edit from the dropdown menu.

Open pipeline settings
Configure schedule and saveStep 2

Scroll down to the Schedule section. Choose one of the four schedule types: Manual, Daily, Interval, or Cron Expression. Fill in the necessary parameters (for example, for Interval type, specify the interval value and time unit). After configuration, click the Update pipeline button at the bottom of the page.

Configure schedule and save

Integration with external systems

Renta allows integration with external data orchestration systems such as Apache Airflow, Prefect, or other tools. You can trigger pipelines programmatically via REST API or embed them into your existing workflows.

Trigger via API

If you are using Manual mode, you can trigger the pipeline programmatically via the Renta REST API:

curl --location 'https://api.eu.renta.im/v1/etl/source/{source_name}/run_integration' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer YOUR_API_TOKEN' \
--data '{
    "integration_id": "your-integration-id",
    "full_refresh": false,
    "run_type": "manual"
}'

To use the API, you need to generate an API key first. Replace YOUR_API_TOKEN with your actual API key.

Integration with Airflow

You can create a DAG in Apache Airflow that calls the Renta API to trigger pipelines:

from airflow import DAG
from airflow.operators.http_operator import SimpleHttpOperator
import json

with DAG('renta_pipeline_dag', schedule_interval='@daily') as dag:
    trigger_renta = SimpleHttpOperator(
        task_id='trigger_renta_pipeline',
        http_conn_id='renta_api',
        endpoint='/v1/etl/source/google_ads/run_integration',
        method='POST',
        headers={
            "Authorization": "Bearer YOUR_API_TOKEN",
            "Content-Type": "application/json"
        },
        data=json.dumps({
            "integration_id": "your-integration-id",
            "full_refresh": False,
            "run_type": "manual"
        })
    )
Frequently asked questions

Ready to get started?

Build your data pipeline today or get a personalized demo. Start free!

Need help?

Get expert support to ensure your project succeeds. We're here to help!

Feature requests?

Help shape our product! Share your ideas for new features and integrations.