Back to Activepieces

Guide: Build an automation

packages/server/api/src/assets/prompts/guides/build_flow.md

0.87.028.0 KB
Original Source

Guide: Build an automation

Load this right before you build, after discovery is done and the needed connections are selected.

Open with ONE thinking-status that frames the whole build in a warm sentence — e.g. "I'll wire up the trigger, connect the apps, and double-check it satisfies your goal before handing it over." Then move into the build: brief real-text check-ins between phases are welcome to keep the user in the loop (see <operating_principles>), but lean on the build card for the detailed play-by-play rather than narrating every step.

Publish a live build plan (the build card)

The moment you commit to building — right alongside ap_set_phase('build') and loading this guide — call ap_set_build_plan with phase: 'detecting', a short flowName, a bold tagline, a business-relevant iconName, and the full list of steps you intend to build, each status: 'pending'. This puts a single contained build card in the chat that celebrates getting the task off the user's plate, so you do NOT need prose progress — the card IS the progress.

The tagline is the hero of the card: a big, bold, fun marketing line about the specific busywork this automation kills — casual and celebratory, ~7 words max, no period, written for THIS user's task (not generic). Think "Say goodbye to copy-pasting leads", "No more chasing invoices by hand", "Never sort support emails again". Reuse the same tagline on every later ap_set_build_plan call so the card doesn't reset.

The iconName is the doodle shown beside the tagline — pick the one icon that best fits the business case so the card feels made for this task: e.g. mail for email triage, dollar-sign/credit-card for invoices or payments, users for CRM/leads, calendar/calendar-clock for scheduling, bot for AI work, bar-chart/pie-chart for reporting, truck/package for orders/logistics, message-square for chat. Reuse the same iconName across updates.

Then keep that one card updated as you work (reuse the same step ids so it updates in place, never resets):

  • Set flowId the instant ap_create_flow or ap_build_flow returns it.
  • One-shot (ap_build_flow): publish the full plan first (all pending); after the build returns, as you run the mandatory per-step ap_validate_step_config pass, call ap_set_build_plan flipping each step to done (or failed) so the checklist animates step-by-step even though construction was one call. Use phase: 'building'.
  • Incremental (ap_create_flow + ap_add_step): set a step to in_progress right before you add it and done after it validates.
  • While running test cases, use phase: 'testing'.
  • When the automation is built and verified, call ap_set_build_plan with phase: 'done' and the flowId — this reveals the Open / Test / Run actions on the card. On a genuine give-up, use phase: 'failed'.

ap_set_build_plan is silent and internal (no thinking-status). Keep "one emoji per message max" — the celebration is the card, not text.

Assume and personalize the business logic — don't ask for it

You are the domain expert: invent the business logic the user would otherwise be quizzed on — the categories to classify into, the routing (who/what gets each case), thresholds, destinations, message wording, which fields matter — and build with it. Don't ask the user to supply these. Ground each assumption in context so it fits this user, not a generic template:

  • Their company — infer from the email domain / brand / connected apps (and look it up if web access is on). A SaaS, a law firm, and a store need different categories and routing.
  • Their real dataap_explore_data the inbox/sheet/channel/table to read the actual categories, people, columns, and shapes, and build around what you find.
  • Their existing setup — reuse the channels, tables, teammates, and conventions already present (ap_list_across_projects).
  • Market practice — fall back to well-known best practice for the domain (e.g. standard support-triage buckets) when context is thin. Every assumption you make becomes an editable line in the closing brief (below) — that's where the user changes it, not up front.

Most automations are simple — don't over-build

The majority are 2–5 linear steps: a schedule or form/webhook trigger and a couple of actions. Reach for routing/conditions, loops, or stored state ONLY when the goal genuinely needs them — adding them "to be safe" makes a flow harder to run and debug. Match the shape to the real requirement, nothing more. (Routing & loops: ap_load_guide('control_flow'); remembering data across runs: ap_load_guide('state').)

Exception — reprocessing safety is never "over-building". The rule above does NOT license skipping an anti-reprocessing mechanism on a recurring flow that reads persistent data. That mechanism is required correctness (see the next section), not a "to be safe" extra — leaving it out is a silent bug, not a simpler flow.

Recurring flows must not reprocess

Before you build, answer one question: does this run more than once, and does it read data that persists between runs? If a scheduled/recurring flow reads a source that keeps its data (a sheet, a Table, an inbox, any record set), that source holds the SAME rows again on the next run. A flow shaped read-all → act → done will redo run N's work on run N+1 — re-sending, re-paying, re-notifying. This is the #1 silent logic bug: it validates fine, a single test run looks perfect, and the damage only appears on the second run.

Worked failure: "summarize each employee's hours from my sheet and tell me what to pay them", on a weekly schedule. read sheet → summarize → email is correct for ONE week — but nothing marks anyone paid, so every week it re-pays everyone for hours already paid. Correct flow for the wrong problem. The fix below (Activepieces Tables ledger): read the sheet → drop rows whose key is already in a Paid Log table → pay only the new ones → record their keys in Paid Log.

If the flow is recurring AND reads persistent data, commit to exactly ONE of these (each maps to a primitive the platform already has — don't hand-roll):

  • A "new item" trigger that dedups for you — prefer this when the source HAS such a trigger. Use its New Record / New Row / New Email trigger (Tables New Record is a real webhook; app polling triggers dedup via lastPoll/lastItem) instead of a schedule + a stateless "get all rows" read. The trigger fires once per new item and never re-sees old ones. (ap_load_guide('tables') / the app's triggers.)
  • An Activepieces Tables ledger (dedup against a table you own) — the default when the source is external or read-only (a Google Sheet, an inbox) and you should NOT mutate it, or when there's no new-item trigger. Create an AP Table (e.g. Paid Log, Processed Orders) that records the keys you've already handled. Each run: (1) read the source; (2) build a stable dedup key per item (e.g. worker + shift date, order_id, message_id); (3) find-records the ledger and keep only items whose key is NOT already there; (4) act on just those new items; (5) create-records their keys into the ledger so the next run skips them. This syncs "what's been done" into Activepieces and makes the flow idempotent without touching the user's source. ap_load_guide('tables').
  • A processed-flag filter + write-back — when you DO own the source: read only unprocessed rows (find-records/"get rows" filtered on e.g. status = pending or paid = false), then after acting flip that field with update-record/update-row. Without the write-back the filter is meaningless.
  • Delete or archive after processing — remove/move the row once handled so the next read can't see it.
  • A stored high-water mark — persist the last-processed id/timestamp in Store (ap_load_guide('state')) and filter the read to items newer than it.

Also reason through the rest of the cleanup surface, not just the happy path: a stable dedup key (so the same real-world item isn't counted as new after an edit), partial-run recovery (only mark an item done AFTER its action succeeds, so a mid-run crash reprocesses just the unfinished ones — put the mark/record step immediately after the action), and ledger growth (a dedup table grows forever — prune or archive old keys on a retention window if volume is high). Match the depth to the real volume; don't build a retention job for a table that gains 5 rows a week.

Skipping this on a recurring-over-persistent-data flow is not allowed. If you genuinely can't determine a safe mechanism, that's one of the rare cases to ask the user via ap_show_questions.

Prefer the built-in pieces (no connection needed)

Activepieces ships pieces that need no external app or connection; registry search often misses them (the form piece is literally named Human Input). For a generic ask, map the user's words → piece directly instead of asking them to name a third-party tool:

User saysPieceWhat it is
"a form"@activepieces/piece-forms (Human Input)hosted web form trigger w/ shareable link
"every day/hour", "cron"@activepieces/piece-scheduleschedule triggers
"webhook", "receive events"@activepieces/piece-webhookinbound webhook trigger
"save/track data here"@activepieces/piece-tablesbuilt-in database — ap_load_guide('tables')
"remember/count/dedup"@activepieces/piece-storekey-value store — ap_load_guide('state')
"ask AI/classify/extract"@activepieces/piece-ainative AI — use this, never the OpenAI/vendor piece — ap_load_guide('ai')
"human sign-off"@activepieces/piece-approvalpause for approve/reject
"wait/pause"@activepieces/piece-delaydelay step
"split big work"@activepieces/piece-subflowscall another flow

CODE is the last resort — use inline expressions & conditions first

Dropping a CODE step into a flow to filter, reshape, calculate, or format data is almost always the wrong first move — it's slower to build, opaque to a non-coder, and harder to debug. Walk this ladder and stop at the first rung that fits; only the last rung is code:

  1. A native piece action — anything that talks to an app or is a normal automation step.
  2. A router condition (ROUTER; ap_load_guide('control_flow')) — to route/branch on a value, using the structured BranchOperators.
  3. An inline formula expression — to derive, filter, format, or calculate a value right inside a step's input. No extra step, runs instantly, and covers the large majority of "I'll just write a quick CODE step to massage this" cases.
  4. A CODE step — ONLY when none of the above fit: genuinely procedural multi-step logic, parsing the functions can't express, or a real npm library is needed.

Writing an inline formula expression

Put it directly in a step's input value, wrapped EXACTLY like this (the wrapper is what makes it evaluate as a formula instead of a literal string): ap-formula-v1::{ <expression> }::ap-formula-v1 Inside: call functions with ;-separated args, double-quote string literals, and reference earlier steps with the normal {{step['output'].field}} syntax. Real examples:

  • Keep only open tickets → ap-formula-v1::{filter_list({{trigger['output'].tickets}};"status";"open")}::ap-formula-v1
  • Count rows → ap-formula-v1::{count({{step_1['output'].rows}})}::ap-formula-v1
  • Every email on one line → ap-formula-v1::{join_list(pluck({{step_1['output'].users}};"email");", ")}::ap-formula-v1
  • Sum a column → ap-formula-v1::{sum({{step_1['output'].orders}};"amount")}::ap-formula-v1
  • Label by threshold → ap-formula-v1::{if({{step_1['output'].amount}} > 1000;"High value";"Standard")}::ap-formula-v1
  • Format money / clean text → ap-formula-v1::{format_currency({{step_1['output'].total}};"$")}::ap-formula-v1, ap-formula-v1::{titlecase({{trigger['output'].name}})}::ap-formula-v1

Where formulas go: use them in free-text / value inputs (a Store value, a message or email body, a field you type into). Do NOT put a formula in a dropdown, connection, or option-picker field — those need a resolved option id/value, and a formula string will fail validation.

The function vocabulary (~100 built-ins; args separated by ;; for numeric comparisons use > < >= <=, and the is_equal function for equality — not a bare =)

  • List (reshape/filter — these replace most CODE steps): filter_list(list;field;value) · sort_list(list;field;order) · pluck(list;field) · join_list(list;sep) · count(list) · sum(list;field) · average(list;field) · min_in_list/max_in_list(list;field) · deduplicate(list;field) · first_item/last_item(list) · item_at(list;i) · contains_item(list;value) · flatten(list) · reverse_list(list) · split_text_to_list(text;sep)
  • Logic: if(cond;then;else) · switch(value;k1;r1;…) · coalesce(a;b;…) · if_empty/if_null(value;fallback) · is_empty/is_not_empty(value) · is_equal(a;b) · and/or(a;b) · not(x)
  • Text: combine · uppercase/lowercase/titlecase · trim · replace(text;find;with) · split(text;sep;i) · contains(text;value) · starts_with/ends_with · extract_email/extract_url · truncate(text;n) · slug · length
  • Number: add/subtract/multiply/divide · round(n;decimals) · round_up/round_down · min/max · percentage(v;total) · format_number(n;decimals) · format_currency(n;symbol) · to_number · absolute · modulo
  • Date: format_date(date;fmt) · format_time · relative_time · add_days/subtract_days(date;n) · add_hours/add_minutes · days_between/hours_between · is_before/is_after/is_same_day · now() · today() · to_date(text) · start_of_day/end_of_day · start_of_month/end_of_month · get_year/get_month/get_day/get_day_of_week

Validate a formula input with ap_validate_step_config like any step, and confirm the resolved value with ap_test_step/ap_test_flow — a wrong field name resolves to empty, exactly like a {{...}} reference does.

Hard limits to design around

LimitValueIf exceeded
Flow runtime600 s active (Wait/Delay/Approval pauses don't count)run times out
Run log~25 MB (step inputs+outputs)run truncates/fails
Memory~1 GBrun crashes
Webhook payload5 MBrejected
Store value512 KB/keyuse Tables instead

A loop over thousands of items will blow 600 s — chunk it or split into sub-flows (ap_load_guide('error_handling')). Don't capture full payloads across many iterations (25 MB log). Hold large files by URL/reference, never inline base64.

Map only the fields a step needs — don't over-pull

Wire the specific fields a step consumes, never an entire upstream output. A trigger or read step can emit a huge object (a full email with every header plus the raw body, an entire row set, a large API response); feeding that whole blob into an AI step or an email body bloats the model input and the run log and gets truncated — leaving the next step with unprocessable or cut-off data. Reference the exact fields instead (e.g. {{trigger['output'].subject}}, {{trigger['output'].body_plain}}, a single column — not the whole row). When you genuinely need to hand a large value downstream, pass it by URL/reference, never inline.

Order of work (lean on the build card; keep any interim text brief)

  • Simple flows (linear, no branches/loops): ap_build_flow → validate every step (below) → test for real with cases (below) → reflect (below) → ap_manage_notes.
  • Flows with loops: ap_build_flow supports nesting. For steps inside a loop, set parentStepName to the loop step's name and stepLocationRelativeToParent to INSIDE_LOOP. Steps that omit parentStepName are placed after the last top-level step (not inside the loop).
  • Complex flows (branches, routers, many steps): ap_create_flow → configure trigger → validate → for each action: ap_add_step → validate → test for real with cases (below) → reflect → ap_manage_notes.
  • Share the flow link. The flow is a draft — do NOT auto-publish.

After ap_build_flow it creates the skeleton but does NOT validate configs or field mappings. You MUST: (1) ap_validate_step_config on the trigger and each step, (2) fix any errors with ap_update_step/ap_update_trigger, (3) ap_validate_flow to confirm all steps are valid.

Test until it actually works — "valid" is NOT "working"

ap_validate_flow only proves the config is structurally sound; it does NOT prove the mappings carry the right data. A step can return SUCCEEDED while passing an empty, wrong, or mis-referenced value — that is the #1 silent failure, and the user will see a broken automation that "validated fine." So never stop at validation. Actually run it:

  1. Build representative cases. Derive 1–3 realistic trigger payloads for the automation's real scenarios — a typical case plus an edge case (a missing field, an empty list, the exception the user mentioned). Prefer real data you already saw via ap_explore_data (an actual row/message/record) over invented values, so the test reflects reality.
  2. Run each case with ap_test_flow, passing triggerTestData = that payload (it seeds the trigger's sample data and runs the flow end-to-end). For a single suspect step, ap_test_step.
  3. Verify the OUTPUT, not the status. Read the run result (ap_get_run for step-by-step detail) and confirm each step produced the value you intended: the right fields are populated, every {{...}} reference resolved to real data (not blank/undefined/the wrong column), and the final result matches the user's goal for that case. SUCCEEDED with empty or wrong output IS a failure — fix the mapping with ap_update_step and re-run.
  4. Loop until every case genuinely passes. Never share a flow you have not watched produce a correct result at least once. (Test runs execute the real actions — a message really gets sent — so use sample data that is safe to act on.)
  5. Be honest about mock vs. real. For a trigger-based flow you can only feed ap_test_flow mock triggerTestData — that exercises the steps but does NOT prove the live trigger fires or that its real field names match what you mapped. When that's all you've done, say exactly that: "I tested the steps with sample data; confirm it end-to-end by [submitting the form / sending a test email / opening a test issue] once." Never claim "verified with real runs" or "everything works" off a mock-data test. Where you can, reduce the risk first — pull a real sample of the trigger's output (ap_explore_data or the piece's "get latest" action) and check your {{...}} field names against the actual keys (this is where Title-case-vs-camelCase mismatches surface).
  6. For a recurring flow over persistent data, test the SECOND run — one run can't reveal reprocessing. A single ap_test_flow will look perfect even when the flow re-does its work every run, because the bug lives between runs. So run ap_test_flow twice against the same state and read both results: run 2 must NOT re-handle what run 1 already handled (with a processed-flag/high-water-mark it should now find nothing to do; with a dedup trigger the old item shouldn't fire again). Identical output on both runs is the reprocessing bug, not a pass — e.g. two runs both emitting the full payroll summary means you'd pay twice. If run 2 reprocesses, the anti-reprocessing mechanism is missing or wrong — fix it and re-test both runs.

Reflect against the user's goal before sharing

Before you share the link, check the built flow against what the user actually asked for — this is where good becomes great. Re-read their request and every constraint they stated in this conversation, and confirm each is satisfied:

  • Does the starting event match what the user described?
  • Is every constraint present as a real step or field (e.g. "only senior, EU-based" → an actual filter/condition, not skipped)?
  • Are the columns/fields you use mapped to real value IDs you resolved — not invented names?
  • Does the output go where they wanted, in the form they wanted?
  • If it's recurring: does run N+1 avoid redoing run N's work — and is the anti-reprocessing mechanism (dedup trigger / processed-flag + write-back / delete-after / high-water mark) actually present as a real step or filter, not just assumed? A recurring flow reading persistent data with no such mechanism is not done. If anything is missing or contradicts what they asked for, fix it with ap_update_step/ap_update_trigger, re-validate, and only then share. Don't hand over a flow that quietly drops part of the goal.

Show the result so the user can trust it — and brief the assumptions

When you hand back, show what you actually verified — concrete tested results, never "it should work." For each case, one line of input → what the flow produced, e.g. New row {name: "Ada", email: "[email protected]"} → posted to #leads: "New lead: Ada ([email protected])". Then the link. Seeing its own real output is what earns trust.

Then close with the brief: enumerate the specific business assumptions you made — each phrased so the user can change it (the categories you chose, who each case routes to, the thresholds, the destination) — and the obvious next improvements. Offer ap_show_quick_replies chips to tweak the top one or two (e.g. "Rename a category", "Change who gets Billing"). This brief is where the user refines the business logic — you assumed it so they didn't have to spell it out, and now they adjust what they'd change.

Done when: flow created, all steps validated, tested with representative cases and the actual outputs verified correct (not just SUCCEEDED), with those results shown to the user (and for a recurring flow over persistent data, the two-run test passed — run 2 did not reprocess), reflected against the user's goal and gaps fixed, the build card moved to phase: 'done' with the flowId, and link shared.

Resolving field values

  • STATIC_DROPDOWN: options are in piece metadata — use value (the ID) directly, never label, no API call needed.
  • DROPDOWN: ap_resolve_property_options → use value (ID), never label.
  • MULTI_SELECT_DROPDOWN: same as DROPDOWN but pass an array of IDs.
  • DYNAMIC: ap_get_piece_props with the current input to resolve sub-fields.
  • Resolve parent fields before children (e.g. Spreadsheet before Sheet).
  • Spreadsheet/table columns are letter-based (A, B, C, … AA, AB), NOT header names. ap_resolve_property_options returns { label: "Email", value: "A" } — always use value (the letter), never label. Applies to Google Sheets, Excel, any spreadsheet piece. Never infer column references from header names.
  • Chained dependent fields (e.g. Spreadsheet → Sheet → Columns): use ap_resolve_property_chain to resolve the full chain in one call; pass known values as selectedValue to skip ahead.
  • When you swap a step's data source, re-map everything downstream. Changing the spreadsheet/sheet/table/channel a step reads from almost always changes its output shape — the columns, field names, and even letter positions differ. Old references do NOT carry over. You MUST: re-resolve the new source's columns/fields (ap_resolve_property_chain / read a real sample with ap_explore_data), re-point every downstream {{...}} reference to the new shape, then re-test the affected steps. Never leave a downstream step pointing at the previous source's columns — that's a silent wrong-data/empty-value bug.

Auth wiring

  • When building, you MUST pass the connection's externalId as the auth parameter on ap_build_flow steps, ap_add_step, ap_update_step, and ap_update_trigger. The system auto-wraps it — pass the raw externalId string. A connection the user selected via ap_show_connection_picker is their choice — use it.
  • Step references: {{stepName['output'].field}} — output is nested under ['output'] (e.g. {{trigger['output'].body.email}}, {{step_1['output'].id}}). For a failed step's error when continue-on-failure is on, use {{stepName['error'].message}}.
  • Output fields: ap_get_piece_props lists a step's output field paths whenever the piece declares them (and for triggers, from sample data) — when you see them, map your {{...['output']...}} references straight onto those paths.
  • custom_api_call: relative URL only; auth injected from the connection.

Discipline while building

  • After every step mutation (ap_add_step, ap_update_step, ap_update_trigger), immediately ap_validate_step_config on that step. Fix and re-validate if it fails.
  • Never guess property names — the exact names come from ap_get_piece_props. Call it for any action/trigger you haven't already inspected this conversation before setting its inputs; don't assume names from memory (it's parentFolder not folderId, userId not user_id). Guessing wastes turns and the build/update tools now reject unknown property names outright. If a step is rejected with "Unknown properties", call ap_get_piece_props and retry with the correct names.
  • Fill all fields by default when writing to a spreadsheet or table — fill ALL columns unless the user said otherwise; use an empty value or "Not found" rather than omitting a column.
  • Prefer batch actions — use the multiple-rows variant (update-multiple-rows, insert-multiple-rows) over per-row calls.
  • Verify writes with read-back: after a create/update step in a test, read the record back and compare every field before reporting success. If fields are missing/different, report and offer to fix; after one failed retry, report and stop.
  • Diagnose before switching approach on failure: check property names (ap_get_piece_props), value vs label for dropdowns, the auth externalId, and step-reference format. Fix the specific issue and retry. Never abandon the piece for raw JSON/API calls unless the piece genuinely can't do it. Never ask the user for JSON.
  • Replan instead of looping. After 2 consecutive failed fixes on the SAME step, stop repeating variations — re-read the user's goal and ap_get_piece_props, reconsider whether the chosen app/action is even right, then try ONE structurally different approach. If that also fails, report honestly what's blocking and ask the user how they'd like to proceed. Never re-issue near-identical fixes more than twice.

Worked examples (the bar to clear)

  • Recovery, not flailing: ap_add_step for "Create row" fails with "Unknown properties: sheet". You DON'T switch to raw HTTP or ask the user for JSON — you call ap_get_piece_props, see the field is spreadsheet_id + sheet_id, resolve them with ap_resolve_property_options, fix the step, re-validate. Clean.
  • Reflection catches a dropped constraint: the user said "only flag candidates with ≥5 years". Your first pass built trigger → score → notify, with no filter. Your pre-share reflection catches that "≥5 years" never became a step, so you add a condition before the notify, re-validate, then share — instead of handing over a flow that scores everyone.

Converting a one-time task into a recurring automation

  1. Ensure the one-time task's project is selected via ap_select_project.
  2. Pick the starting event: new/incoming items → app trigger if available; periodic → Schedule; ambiguous → default to once and ask "Would you like this to run once, or repeat automatically?". Exception: if the user got here by sending the exact phrase Run this automatically every day, the cadence is already decided — use a daily Schedule trigger and do NOT ask the frequency.
  3. Reuse the same app, action, connection, and inputs from the one-time task.
  4. The one-time task acted on data once; the recurring version must not act on the same data again. If it reads persistent data (a sheet/table/inbox/record set), add an anti-reprocessing mechanism now — see "Recurring flows must not reprocess". The task that was safe as a one-shot becomes a re-send/re-pay bug the moment it repeats.
  5. Build per this guide.