Automations
An automation is a recurring task defined on an asset. An automation computes forecasts, schedules or reports. Plugins can register additional types, such as data ingestion, with their own data generators, validation schemas and worker queues. See Adding your own automation type for the plugin contract.
On each run, the automation queues jobs (so make sure a worker is processing the forecasting, scheduling or reporting queue, whichever the automation needs, see Redis Queues).
The parameters of the task were stored when the automation was created, and validated with the same schema that the CLI and API use.
Timing parameters are resolved on each run — for instance, the forecast or schedule start defaults to the time the automation runs, so each run produces fresh results (see Timing each run).
Creating an automation
Here is how you create an automation in the CLI, asking for daily (at 6 AM) forecasts of sensor 12:
flexmeasures add automation --asset 3 --name "Daily PV forecasts" --type forecasting \
--cron "0 6 * * *" --timezone Europe/Amsterdam --sensor 12
--type says which registered task to automate and defaults to forecasting.
Built-in types are forecasting, scheduling and reporting; plugin types use the identifiers registered by the plugin.
The remaining options are the ones the task itself needs: a forecast automation accepts everything flexmeasures add forecast accepts, such as --forecaster to pick the forecaster and --config to configure it (see Forecasting).
The forecaster and its configuration are stored on a data source, so you can also pass --source to reuse the data source of an existing forecaster, in which case --forecaster and --config (and the individual configuration options) are not needed — the data source already determines them.
That data source is required while the automation exists, so it cannot be deleted until the automation is removed.
The recurrence is defined by a standard five-field cron string (minute, hour, day of month, month, and day of week), which defaults to "0 0 * * *" (daily at midnight).
It is interpreted in the automation’s IANA timezone.
If --timezone is omitted, the current FLEXMEASURES_TIMEZONE value is copied to the automation.
Changing that configuration later does not change existing automations.
Cron aliases and optional seconds or year fields are not supported.
Automations are active by default (use --inactive to create them in deactivated state).
Use flexmeasures edit automation to rename, re-schedule (--cron), change the timezone, activate or deactivate an automation, and flexmeasures delete automation to remove one.
These changes are recorded in the asset’s audit log.
For forecast automations, the sensor on which forecasts are saved (sensor-to-save, falling back to sensor) must belong to the automation’s asset or one of its descendants.
This relationship is checked both when the automation is created and immediately before each run.
Timing each run
An automation runs again and again, so its parameters cannot pin a moment in time:
a fixed start or end would have every run compute the same period,
and a fixed prior would have every run ignore the data recorded since then.
All three are refused when the automation is created, whatever its type.
A forecast automation can still fix train-start, where its training data begins, which is part of the forecaster’s configuration.
Instead, say how the period a run covers relates to that run, with two of these fields:
start-offset: where the period starts, as an offset chain (see below);end-offset: where the period ends, as an offset chain;duration: how long the period lasts, as an ISO 8601 duration.
On the command line, --start-offset, --end-offset and --duration set the same fields, so no parameters file is needed for them.
An offset chain is a comma-separated list of steps, applied from left to right. Each step is a Pandas offset alias, optionally preceded by a count and a sign, or one of two steps FlexMeasures adds. These are the ones most useful for automations, applied to a run due on Friday 27 March 2026 at noon:
Step |
Moves to |
Example result |
|---|---|---|
|
the beginning of the day (FlexMeasures) |
27 March, 00:00 |
|
the beginning of the hour (FlexMeasures) |
27 March, 12:00 |
|
minutes later, or earlier with a minus sign |
|
|
hours later or earlier |
|
|
days later or earlier, of 24 hours each |
|
|
business days later or earlier |
|
|
the next Sunday, so use |
|
|
the start of the next month, or of this one with |
|
|
the end of this month |
|
|
the start of the next quarter |
|
|
the start of the next year |
|
Steps that move to a boundary, such as MS, keep the time of day, so follow them with DB to start at midnight: -1MS,DB is the start of the current month.
Older aliases such as H, M and T still work, but Pandas warns that they will be removed; use h, ME and min instead.
The offsets apply to the time the run was due, on the automation’s own clock, the one its cron string is read in.
For instance, an automation due every day at noon, with start-offset: "1D,DB" and duration: "P1D", covers the whole of the next day.
A run that only happens after midnight, because the runner was delayed, still covers the day after the one it was due on.
A duration counts real time, so on the day the clocks go forward, P1D from midnight ends at 1 AM;
two offsets, such as "1D,DB" and "2D,DB", follow the calendar day instead.
A duration in months or years follows the calendar, so P1M from midnight ends at midnight, a month later.
An automation’s forecast is believed at the time it is computed, also when the period it covers starts later.
A single field can also be given alone, where the automation has a default for the rest.
A start-offset alone gives a forecast or schedule its default duration, and has a report end at the time of the run.
A duration alone has a forecast or schedule start at the time of the run, and is not enough for a report.
An end-offset alone is for reports only, which then start where the last successful report ended.
Before the first successful report, such a report covers the cron period before that end.
When the last successful report already reaches that end, as it does for an hourly report up to midnight after its first run of the day, the run has nothing new to report on, and queues no job.
Without offsets, a forecast starts at the time of the run, rounded down to its sensor’s resolution,
a schedule starts at the time of the run, rounded down to its resolution or else to the minute,
and a report covers the period since the last successful report ended (see Automating reports).
The difference shows after the runner was down.
It catches up with only the latest missed run (see Running automations), so offsets then cover the period around that run alone,
while a report without offsets covers everything since the last successful report.
Automating schedules
A schedule automation’s parameters form a schedule trigger message, as accepted by the [POST] /assets/(id)/schedules/trigger API endpoint (without the asset id).
Use the canonical API field names, including flex-model, flex-context and force-new-job-creation.
The message is passed in a file, through --parameters, and validated when the automation is created.
The forecaster options above configure a forecaster, so they do not apply here, and are refused when combined with --type scheduling.
A schedule automation has a data generator too, but you do not name it separately. It is put together from choices you have already made: the flex config in the trigger message, the flex config saved on the asset tree, and the scheduler that the asset resolves to. Because those live in two places, and the asset can be edited without touching the automation, the runner puts the generator together again on every run, and moves the automation to another data source when the combination has changed. Editing an asset’s flex-model is therefore a configuration change, and shows up as one: the schedules computed before and after it carry different data sources.
Because the schedule is recomputed on every run, the flex config may only describe the site and its devices, not one moment.
A field with a fixed moment in it, such as soc-at-start or a soc-targets entry with a datetime, is refused when the automation is created, and the error names the field.
Refer to a sensor instead, which says where to look rather than what was true once.
Without offsets (see Timing each run), the start is calculated afresh from the server time on each run.
It is floored to the fixed, positive resolution when given, or otherwise to the minute.
The duration must be positive; resolution does not accept nominal durations such as a month.
As usual, the flex-context and flex-model can also (partly) live on the asset itself, in which case a minimal trigger message suffices.
For example, this automation queues a scheduling job every hour, each time scheduling the next 12 hours:
echo 'duration: "PT12H"' > trigger-message.yml
flexmeasures add automation --asset 3 --name "Hourly schedules" --cron "0 * * * *" --type scheduling --parameters trigger-message.yml
And this one schedules the whole of the next day, every day at noon, as for a day-ahead market:
flexmeasures add automation --asset 3 --name "Day-ahead schedules" --cron "0 12 * * *" --timezone Europe/Amsterdam \
--type scheduling --start-offset 1D,DB --duration P1D
Plugin-defined automations
Create a plugin-defined automation with its registered type identifier and JSON or YAML files:
flexmeasures add automation --asset 3 --name "Import site measurements" \
--type site-ingestion --cron "*/15 * * * *" --timezone Europe/Amsterdam \
--config ingestion-config.yml --parameters ingestion-parameters.yml
The registered handler determines the data generator and worker queue.
--data-generator can explicitly name the registered generator; --source reuses an existing generator configuration and cannot be combined with --data-generator or --config.
Forecast-specific command-line options do not apply to plugin types.
Plugin schemas validate configuration and parameters, and unknown fields are rejected.
Run a worker for the queue declared by the handler, in addition to the automation dispatcher.
The UI lists registered plugin types in their own tabs and shows their parameters, data source, input sensors, output sensors and recent jobs. Existing actions, including editing recurrence, activation, deletion and Run now, also apply to these automations. Creation forms for plugin types are a follow-up: use the CLI or API to create them for now.
API-created automations remember the user who created them. The dispatcher and worker recheck that the user is active and can write to the output sensors. CLI-created automations run as trusted deployment operations. Every output sensor must still belong to the automation’s asset or one of its descendants.
Automating reports
A report automation’s parameters are report parameters, as flexmeasures add report accepts them (see Reporting), passed in a file through --parameters.
Name the reporter with --reporter, and configure it with --config.
As for a forecast automation, the reporter and its configuration are stored on a data source, so --source can reuse the data source of an existing reporter instead.
The sensors a report is recorded on must belong to the automation’s asset or one of its descendants. This is checked when the automation is created and immediately before each run. A report job only records on those sensors, so a reporter that returns results for any other sensor is refused.
Its period follows Timing each run, with offsets or one offset and a duration.
For instance, start-offset: "-1D,DB" with end-offset: "DB" reports on the whole of the previous day.
Without offsets, a report covers the period since the last successful report ended, falling back to the previous cron period on the first run.
That end is recorded by the reporting job itself, once it has succeeded, so a failed report leaves no gap: the next run starts where the last successful one ended.
For example, this automation reports on the previous day, with the offsets above in report-parameters.yml.
It runs every night at 1 AM, an hour after midnight, so that the day’s last readings have had time to arrive:
flexmeasures add automation --asset 3 --name "Daily self-consumption report" --type reporting \
--cron "0 1 * * *" --timezone Europe/Amsterdam \
--reporter PandasReporter --config reporter-config.yml --parameters report-parameters.yml
Running automations
For automations to run on schedule, invoke the dispatcher once per minute.
The provided Docker Compose stack does this with its automation-runner service, which starts after the web server is ready.
If you host FlexMeasures without that service, set up a cron job instead:
* * * * * flexmeasures jobs run-automations
Use one dispatcher for a deployment; the Docker Compose service already runs the command, so it does not need a host cron job as well.
Each due automation then queues its jobs. If the runner misses runs, because it was down or overloaded, it catches up when it resumes: it queues only the latest missed run of each automation, rather than replaying stale ones. Timing parameters that default to the run time are resolved when that catch-up run is queued, so it produces a current forecast, schedule or report.
Each scheduled run a runner picks up is recorded durably, so a queueing attempt which fails can be retried without duplicating the jobs it already created. See Runs and retries.
The jobs record how they were created, which is shown on the asset’s status page (UI), where recent jobs are listed.
Runs and retries
Every scheduled run a runner picks up gets a record in the database, which outlives the jobs it creates (jobs in Redis expire). The runner claims the run before doing any work, and the database allows only one record per automation, scheduled time and schedule revision, so two runners started in the same minute cannot both execute it. A claim comes with a lease: while one runner holds a live lease on a run, no other runner touches it, and once that lease expires the run is up for grabs again, which is how a runner that died mid-queueing hands its work over.
Before queueing anything, the runner writes down the plan for the run: the parameters it will use and the individual jobs it intends to create, each with its own logical name and a job ID derived from the run. This is what makes a retry safe. A run which failed before queueing anything is dispatched again in full. A run which queued only some of its jobs resumes from the same plan, recognizes the jobs already in Redis by their IDs, and queues only the ones still missing, so a retry never duplicates work, and never silently drops it either. Because the plan is stored, a retry hours later still uses the parameters the run was planned with, even if the automation has been edited since. Timings the automation left to the run time are not part of those parameters, so they are resolved afresh on each attempt: a resumed run’s jobs can therefore cover a later window than the ones its first attempt queued.
A dispatch which fails is attempted again, each time after a longer wait, and stops being attempted after five attempts. A dispatch that keeps failing usually fails for a reason no retry fixes, such as a sensor that was deleted, and the run’s record says what went wrong on each attempt.
Retrying a failed dispatch this way is what a forecast run does. A schedule run is recorded, claimed and reported in just the same way, but is left where it failed rather than dispatched again, because its jobs get a fresh ID on every dispatch, so a retry could not tell an already queued schedule from a missing one.
A forecast run also records each job it created, and how that job ended, which is what its execution state describes.
A schedule or report run records its dispatch in the same way, while its execution state stays pending, because the jobs of those runs are not recorded individually yet.
A run tracks two things separately: how far its dispatch got (pending, claimed, partially_queued, queued or failed), and how its execution by the workers ended (pending, running, succeeded, failed or canceled).
Each attempt to dispatch a run is recorded too, with the runner which made it, what it queued, and why it failed if it did.
This is what an operator needs to tell a run which failed before queueing anything, one which queued half its work, and one which queued everything but then failed while computing, apart from each other.
Editing an automation’s cron string or timezone, or reactivating it, counts up its schedule revision. Runs of the old and the new schedule therefore stay distinct, even when they fall on the same scheduled UTC time.
Running one automation on demand
Besides its recurring runs, a single automation can be run now, once. This is useful to try out a new automation, to re-run one after fixing what made it fail, or to refresh its results after late input data arrived.
flexmeasures jobs run-automation --automation 4
The same is available in the API, as [POST] /assets/(id)/automations/(automation-id)/trigger, and in the UI, as the Run now button on the asset’s Automations page.
The automation runs with the parameters it was created with, and the jobs it queues are recorded as its jobs, just like the jobs of a recurring run. An on-demand run does not affect the automation’s recurrence: its cursor (see Appendix: how the runner decides what is due) stays where it was, so the next recurring run still happens as scheduled, and a run missed while the runner was down is still caught up. Inactive automations can be run this way, too, which is how you can try one out before activating it.
Unlike a recurring run, an on-demand run is not protected against being started twice: asking for two runs in a row queues two runs.
Viewing automations
Automations defined on an asset can be viewed on the asset’s Automations page in the UI, and listed with the API endpoint [GET] /assets/(id)/automations.
Automations are usually defined on a sub-asset, so the page lists what runs anywhere below the asset as well, naming the asset each automation belongs to.
Turn Include automations of sub-assets off to see only the automations defined on the asset itself; the choice is remembered for your next visit.
The API endpoint does the same, and takes include-child-assets=false to narrow the listing.
Either way, only the assets you may read are included.
The page shows how far off each automation’s next scheduled run is, such as “in 6 minutes” or “tomorrow” (excluding any pending catch-up run). Hovering it gives the exact time, read on the automation’s own timezone, together with the recurrence it follows. Created At reads on that same clock. The page brings itself up to date once a minute, so runs and job counts appear without reloading; it holds off while the tab is in the background, or while a panel or menu is open.
An automation’s Info panel shows the sensors it reads from and writes to, linking to each sensor’s page, and the data source it records under, together with the configuration that data source was created with. It also summarizes the automation’s recent runs and their outcomes (see Runs and retries). Conversely, a sensor’s page lists the automations that write data to it.
Appendix: how the runner decides what is due
This section describes the bookkeeping behind the catch-up behaviour above. You do not need it to use automations.
The runner is a stateless command, executed once a minute by Docker Compose or cron, so it needs a durable record of how far each automation has gotten. That record is one timestamp per automation, its cursor: the scheduled time of the most recent run the automation has committed to. Runs at or before the cursor are never queued again. Before queueing any jobs, the runner advances the cursor to the run it is about to queue, and saves it. The cursor therefore records that a run was claimed, not that queueing or the task itself succeeded.
Keeping a single moving timestamp is what makes the catch-up behaviour above fall out: a runner that has been down catches up by moving the cursor straight to the latest due run, rather than replaying every run it missed. The cursor also decides who may claim a newly due run, because it is advanced with a conditional update which only one of two runners started in the same minute can win. What happened to a run once it is claimed is kept in its own record instead (see Runs and retries), which is why the cursor alone says nothing about whether queueing or the task succeeded.
A new automation starts from its creation minute and does not replay runs from before it existed. Changing its cron expression or timezone, or reactivating it, restarts from the time of that change. Deactivated automations do not accumulate catch-up work. After upgrading an existing installation, runs scheduled before the upgrade are not replayed.
Daylight-saving-time transitions follow wall-clock semantics. If the clock skips a scheduled local time in spring, that run happens once at the transition boundary. If a scheduled local time occurs twice in autumn, the first instance is the canonical run and the repeated instance is not queued again.