docs/v3/release-notes/oss/version-3-1.mdx
Released on January 30, 2025
Enhancements ➕➕
schedules in prefect.yaml by @zzstoatzz in #16867run_flow_in_subprocess utility by @desertaxle in #16802prefect.runtime.task_run by @zzstoatzz in #16879python -m prefect by @sky-code in #16790Bug Fixes 🐞
validation_alias to SnowflakeConnector by @desertaxle in #16854prefect-dask usage docs by @zzstoatzz in #16872site-packages from scanned dirs in prefect deploy by @zzstoatzz in #16885Task.__call__ by @desertaxle in #16891kubernetes-asyncio version by @jeanluciano in #16841Integrations & Dependencies 🤝
<0.14.0,>=0.12.0 to >=0.12.0,!=0.12.2,<0.16.0 by @dependabot in #16205prefect_aws tasks by @desertaxle in #16851PrefectDbtSettings to prefect-dbt by @kevingrismore in #16834Development & Tidiness 🧹
Flow.ato_deployment, Flow.afrom_source, and Runner.afrom_storage by @desertaxle in #16897pyright checks to CI by @desertaxle in #16843schedule key by @zzstoatzz in #16847run_flow_in_subprocess by @desertaxle in #16883[@sync](https://github.com/sync)_compatible from prefect.artifacts by @desertaxle in #16877Service base class to cover loop and other services by @chrisguidry in #16898Experimental 🔬
Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.14...3.1.15
Released on January 23, 2025
Bug Fixes 🐞
PREFECT_SERVER_LOGGING_LEVEL in prefect server start by @zzstoatzz in #16765prefect dashboard open by @zzstoatzz in #16768Inputs cache policies are consolidated when used in a CompoundCachePolicy by @zzstoatzz in #16785FlowRunContext for flow runs kicked off from a deployment by @desertaxle in #16831Development & Tidiness 🧹
cli.deploy by @zzstoatzz in #16767prefect.runner.runner by @desertaxle in #16797TaskRunRecorder by @zzstoatzz in #16829pendulum consolidation by @zzstoatzz in #16808prefect-redis to the Prefect image by @chrisguidry in #16818Documentation 📓
Schedule a flow tutorial by @kevingrismore in #16796Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.13...3.1.14
Released on January 17, 2025
We're thrilled to announce a major milestone: prefect is now 100% type complete! 🎉 Every public interface in the library now includes comprehensive type annotations, bringing significant improvements to the developer experience. This means you'll enjoy:
mypy and pyright, catching potential issues before runtimeThis achievement represents our commitment to building a more maintainable, reliable, and developer-friendly Python library.
Enhancements ➕➕
prefect server services CLI commands by @zzstoatzz in #16706Bug Fixes 🐞
SendgridEmail code example by @jakekaplan in #16677wait_for by @chrisguidry in #16709drain_all call to be called from new thread instead of global loop thread by @jeanluciano in #16739QueueService instances are unique within types by @chrisguidry in #16751Integrations & Dependencies 🤝
Development & Tidiness 🧹
_on_start by @cicdw in #16687prefect.server.events by @desertaxle in #16692prefect.server.services by @desertaxle in #16701prefect.results by @desertaxle in #16714prefect.settings by @desertaxle in #16719prefect.inputs to 100% by @desertaxle in #16740prefect.server to 100% by @desertaxle in #16729CoiledPushProvisioner by @desertaxle in #16745JsonPatch implementation by @desertaxle in #16743Documentation 📓
gcloud example command by @cicdw in #16738Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.12...3.1.13
Released on January 09, 2025
Please consider joining us virtually January 22-23 for our Winter Summit, where we'll be giving some exciting updates on our OSS community growth along with customer stories and updates on Prefect's roadmap! Check out the agenda and register here.
Enhancements ➕➕
sync_compatible from automations SDK by @zzstoatzz in #16584Bug Fixes 🐞
enforce_parameter_schema in prefect.yaml by @zzstoatzz in #16607FlowRunContext.model_rebuild() in hydrate_context by @zzstoatzz in #16628Integrations & Dependencies 🤝
<6.0.0,>=1.0.12 to >=1.0.12,<7.0.0 by @dependabot in #16431Development & Tidiness 🧹
prefect.blocks by @desertaxle in #16630Documentation 📓
otel extras in docker image and remove experimental flag setting by @bunchesofdonald in #16412Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.11...3.1.12
Released on January 02, 2025
Enhancements ➕➕
Bug Fixes 🐞
flow_run_name by @zzstoatzz in #16517GitRepository by @jakekaplan in #16518NO_PROXY for websocket connections by @jbw-vtl in #16538Path interpreting . in deployment names as the beginning of a file extension by @zzstoatzz in #16553Integrations & Dependencies 🤝
TypeError in SnowflakeConnector.fetch_all by @zzstoatzz in #16511fetch_one and fetch_many in prefect-snowflake by @zzstoatzz in #16554redis-streams by @zzstoatzz in #16432Development & Tidiness 🧹
prefect.cli part 1 by @desertaxle in #16546wait_for and return_state in Task.__call__ by @zzstoatzz in #16547Documentation 📓
v2 UI
Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.10...3.1.11
Released on December 25, 2024
'Twas the night before Christmas, when through every cloud Not a CPU was stirring, not even in prod
Bug Fixes 🐞
getattr by @zzstoatzz in #16498Development & Tidiness 🧹
prefect.futures by @zzstoatzz in #16381Documentation 📓
serve docs by @zzstoatzz in #16475Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.9...3.1.10
Released on December 20, 2024
Enhancements ➕➕
prefect.flow decorator by @mjpieters in #16405Bug Fixes 🐞
enforce_parameter_schema in prefect deploy by @zzstoatzz in #16418from_source safe to use concurrently by @zzstoatzz in #16458/csrf by @aaazzam in #16464Integrations & Dependencies 🤝
MattermostWebhook tests by @zzstoatzz in #16433OTLPLogExporter and OTLPSpanExporter in a QueueService to pull them out of the main thread by @bunchesofdonald in #16439Development & Tidiness 🧹
Documentation 📓
Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.8...3.1.9
Released on December 17, 2024
Runner Heartbeats for Flow Run Monitoring
Flow runs can now emit heartbeat events to detect infrastructure failures (crashed machines, evicted containers, etc). When enabled, an automation can automatically mark flows as crashed when heartbeats stop, preventing stuck "zombie" flows in the running state.
Enable with:
PREFECT_RUNNER_HEARTBEAT_FREQUENCY (requires Prefect 3.1.8+)from prefect.automations import Automation
from prefect.client.schemas.objects import StateType
from prefect.events.actions import ChangeFlowRunState
from prefect.events.schemas.automations import EventTrigger, Posture
my_automation = Automation(
name="Crash zombie flows",
trigger=EventTrigger(
after={"prefect.flow-run.heartbeat"},
expect={
"prefect.flow-run.heartbeat",
"prefect.flow-run.Completed",
"prefect.flow-run.Failed",
"prefect.flow-run.Cancelled",
"prefect.flow-run.Crashed",
},
match={"prefect.resource.id": ["prefect.flow-run.*"]},
for_each={"prefect.resource.id"},
posture=Posture.Proactive,
threshold=1,
within=90,
),
actions=[
ChangeFlowRunState(
state=StateType.CRASHED,
message="Flow run marked as crashed due to missing heartbeats.",
)
],
)
my_automation.create()
Basic Authentication
We have also added a long requested feature: basic authentication support for the Prefect API. Enable by setting an auth string:
Start a protected server:
PREFECT_SERVER_API_AUTH_STRING="admin:admin" prefect server start
Authenticate client operations:
PREFECT_API_AUTH_STRING="admin:admin" python flow_script.py
Enhancements ➕➕
Runner to send heartbeat events for flow runs by @desertaxle in #16410Integrations & Dependencies 🤝
prefect-redis by @zzstoatzz in #16417Documentation 📓
Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.7...3.1.8
Released on December 16, 2024
Enhancements ➕➕
Bug Fixes 🐞
[@task](https://github.com/task) decorator by @zzstoatzz in #16386websocket_connect utility in Subscriptions by @jakekaplan in #16388Development & Tidiness 🧹
Block.aload method and remove [@sync](https://github.com/sync)_compatible from Block.load by @desertaxle in #16341sync_compatible from git_clone by @zzstoatzz in #16340prefect.serializers by @desertaxle in #16331sync_compatible from prefect.variables by @zzstoatzz in #16349DateTime to prefect.types by @zzstoatzz in #16356Dockerfile build time and add CI to catch future slow downs by @zzstoatzz in #16348prefect.tasks and prefect.task_worker by @zzstoatzz in #16332entrypoint.sh to use uv for EXTRA_PIP_PACKAGES by @zzstoatzz in #16291prefect work-pool inspect by @zzstoatzz in #16378Documentation 📓
RunTelemetry class by @jeanluciano in #16233Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.6...3.1.7
Released on December 11, 2024
This release begins a concerted effort to improve our public interface's type completeness; we hope that this will make the experience of authoring Prefect flows within IDEs more robust and allow for better autocomplete. We welcome all contributions to this effort - follow along with this issue to get involved!
Enhancements ➕➕
job_variables in runtime.flow_run by @zzstoatzz in #16124aserve utility for serving multiple flows from an asynchronous context by @GitAlexxx in #15972sync_compatible from build_server by @zzstoatzz in #16314Bug Fixes 🐞
__getattr__ conflict in prefect.docker.__init__ by @zzstoatzz in #16172Integrations & Dependencies 🤝
Development & Tidiness 🧹
from_profiles_yml to dbt TargetConfigs by @kevingrismore in #16178prefect.server.database.migrations to prefect.server.database._migrations by @desertaxle in #16214blocks.abstract and concurrency by @zzstoatzz in #16212husky to pre-commit by @desertaxle in #16246flow and task decorators pass strict typing by @zzstoatzz in #16251uv to install dependencies in the Dockerfile by @zzstoatzz in #16235NODE_VERSION to .nvmrc by @desertaxle in #16282prefect.main by @zzstoatzz in #16288debug.mdx tutorial by @zzstoatzz in #16296Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.5...3.1.6
Released on December 03, 2024
Enhancements ➕➕
any_ by @zzstoatzz in #16072prefect_aws.s3 async tasks to async_dispatch by @zzstoatzz in #16096Bug Fixes 🐞
Integrations & Dependencies 🤝
prefect_aws.client_waiter off sync_compatible by @zzstoatzz in #16082prefect_aws.lambda_function off sync_compatible by @zzstoatzz in #16083pydantic==2.10.0 workaround given upstream fix by @zzstoatzz in #16087prefect_aws.secrets_manager off sync_compatible by @zzstoatzz in #16169Development & Tidiness 🧹
labels field on Flow, Deployment, FlowRun, and TaskRun by @bunchesofdonald in #16050BaseFlowRunEngine and AsyncFlowRunEngine shell by @desertaxle in #16091incidents by @aaazzam in #16139Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.4...3.1.5
Released on November 21, 2024
Enhancements ➕➕
RunnerDeployment by @zzstoatzz in #16075Bug Fixes 🐞
None -> {} in KeyValueLabels by @zzstoatzz in #16067pydantic==2.10 compatibility by @zzstoatzz in #16076Integrations & Dependencies 🤝
prefect pin to >=3.1.3 in prefect-aws by @zzstoatzz in #16064AzureBlobStorageCredentials to close default credentials when used as a context manager by @desertaxle in #16071Development & Tidiness 🧹
/docs/3.0 by @daniel-prefect in #16058Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.3...3.1.4
Released on November 19, 2024
Enhancements ➕➕
new action to OpsGenieWebhook for apprise==1.9.0 compatibility by @kevingrismore in #16030Bug Fixes 🐞
Integrations & Dependencies 🤝
prefect_redis use of async by @teocns in #15967<5.0.0,>=1.0.12 to >=1.0.12,<6.0.0 by @dependabot in #15856async_dispatch to enable forcing behavior for instance methods by @zzstoatzz in #15996Development & Tidiness 🧹
prefect.yaml by @discdiver in #15812prefect_aws.batch off sync_compatible by @zzstoatzz in #16053New OSS UI 📊
Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.2...3.1.3
Released on November 12, 2024
Bug Fixes 🐞
list and set settings values when marshaling to environment variables by @GalLadislav in #15979.env files load only Prefect environment variables by @desertaxle in #15975url_for by @sam-phinizy in #15974Integrations & Dependencies 🤝
async_dispatch for slack collection by @zzstoatzz in #15973Development & Tidiness 🧹
None value by @zzstoatzz in #15966asyncio_default_fixture_loop_scope in all collections by @zzstoatzz in #15976Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.1...3.1.2
Released on November 08, 2024
Enhancements ➕➕
prefect-kubernetes into setting framework by @desertaxle in #15888Bug Fixes 🐞
wait_for for DaskTaskRunner by @zzstoatzz in #15909prefect config view not showing .env file settings by @GalLadislav in #15922Integrations & Dependencies 🤝
<0.13.0,>=0.12.0 to >=0.12.0,!=0.12.2,<0.14.0 by @dependabot in #15949Development & Tidiness 🧹
async_dispatch util by @zzstoatzz in #15813COVERAGE_CORE=sysmon to speed up test on 3.12 by @desertaxle in #15905Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.1.0...3.1.1
Released on October 31, 2024
We're excited to unveil a major overhaul of Prefect's configuration system. At the heart of this release is a complete refactor of our settings architecture, now powered by pydantic-settings for enhanced organization and clarity. You can now define your settings in either a dedicated prefect.toml file or your existing pyproject.toml file, streamlining the configuration experience and making it more portable across environments.
To learn more about new settings capabilities, check out the updated settings documentation or watch @zzstoatzz's instructional demo on YouTube.
New Features 🎉
prefect.toml file by @desertaxle in #15788pyproject.toml file by @desertaxle in #15875Enhancements ➕➕
run_shell_process by @zzstoatzz in #15817jit="off" setting for Postgres DBs by @desertaxle in #15854Bug Fixes 🐞
Settings.copy_with_update to ignore sources when restoring default settings by @desertaxle in #15884Integrations & Dependencies 🤝
prefect-dbt
Dependencies
<4.0.0,>=1.0.12 to >=1.0.12,<5.0.0 by @dependabot in #15839Development & Tidiness 🧹
prefect.toml to .gitignore by @zzstoatzz in #15846soft_wrap link in prefect block register by @zzstoatzz in #15876experimental_warn setting to experiments model and removes async_fetch_state_result by @desertaxle in #15885prefect.toml support by @desertaxle in #15847max_active_runs and catchup fields by @zzstoatzz in #15810pytest-markdown-docs by @bunchesofdonald in #15808Full Changelog: https://github.com/PrefectHQ/prefect/compare/3.0.11...3.1.0