Workflow Diagnostics scans a workflow's history, finds what went wrong, and tells you how to fix it. Until now, the results lived on their own page. You read an issue there, then went back to the history to find the event it was about.

Now Diagnostics is part of Workflow History. Issues show up on the events that caused them, so you can debug a workflow in one place.

Cadence is a multi-tenant workflow orchestration platform, where a single cluster can serve many domains running different types of workflows. When a domain has a sudden traffic spike, it can consume a disproportionate share of resources used for history task processing and degrade performance for other domains in the same cluster, which is a noisy neighbor problem.

As the scale and diversity of workloads in a Cadence cluster grow, isolation between domains becomes increasingly important for maintaining predictable performance and reliability. To address this challenge, we redesigned two key components of the Cadence History service (the history task scheduler and history queue) to provide stronger domain-level isolation.

What happens when a bad deploy sends thousands of workflows into a broken state? Or when you need to signal an entire cohort of workflows to advance past a checkpoint?

With Batch Actions, Cadence Web lets you select a group of workflows and act on all of them at once, right from the browser.

Your nightly ETL job takes 90 minutes. The schedule fires every hour. What should happen?

With traditional cron, the answer is: both run, you deal with the fallout. With CronSchedule on StartWorkflowOptions, the answer was always: the new fire is skipped. Neither answer is right for every situation, and neither lets you change your mind without cancelling and restarting the workflow entirely.

Cadence Schedules give you control.

You encrypted the payloads. Here is what you did not encrypt.

The AES-256-GCM DataConverter makes every workflow input, activity result, signal payload, and query response opaque to anyone without your key. That is the good part. The less obvious part: several Cadence data surfaces sit entirely outside the DataConverter path, and they are still plaintext regardless of what your converter does.

The 2 MB per-payload limit in Cadence does not come with a helpful error. Your workflow does not receive a graceful degradation notice. It just fails. And if you have never hit the limit before, the stack trace is not obvious about what happened.

The claim-check pattern solves this completely. Instead of compressing a large payload and hoping it fits, you offload it to an external blob store and write only a small reference into Cadence history. The limit no longer applies to your payload; only to the reference, which is always tiny.

That customer order you passed into your fulfillment activity (the one with the email address, the shipping details, and the internal pricing fields) is sitting in your workflow history as plaintext JSON right now. Anyone with read access to Cadence history can see it. So can anyone with access to your history storage backend. This is not a bug. It is how Cadence works by design, and it is the right default for most workloads. But three problems follow from it in production, and most teams hit at least one of them before they know the solution exists.

Signals are how the outside world communicates with a running Cadence workflow without interrupting it. Consider a document approval workflow that runs for days or weeks: at any moment, a manager clicks "Approve" on a dashboard and that signal needs to reach the workflow immediately — without restarting it or losing its place. Whether you're building human-in-the-loop approvals, broadcasting configuration changes to thousands of running workflows, coordinating multi-stage pipelines, or implementing liveness checks — signals are the right tool.

Signal handling feature in python client is now available in the latest cadence-python-client release, v0.2.2 — see the v0.2.2 release notes on GitHub and install from PyPI.

How the new Cadence Shard Manager Found and Mitigated Two Latent Deadlocks​

We're rolling out a new Shard Manager service for Cadence that replaces the existing hash-ring based routing, and it's coming to the open-source release soon. The new architecture gives us load balancing, graceful shard handovers, and the debuggability and observability we've used to find the two deadlocks in this post. During the rollout, the Shard Manager exposed two latent deadlocks in the Cadence Matching service. It moved traffic to the healthy instance and kept the system running while two engineers fixed them in a day, with six lines of code total.

To understand how this happened, we first need to understand the new architecture.