Bypass the 2 MB Limit Without Shrinking Your Workflow
The 2 MB per-payload limit in Cadence does not come with a helpful error. Your workflow does not receive a graceful degradation notice. It just fails. And if you have never hit the limit before, the stack trace is not obvious about what happened.
The claim-check pattern solves this completely. Instead of compressing a large payload and hoping it fits, you offload it to an external blob store and write only a small reference into Cadence history. The limit no longer applies to your payload; only to the reference, which is always tiny.
How it works
The DataConverter intercepts every payload before it reaches Cadence history. With claim-check, it makes a size decision on each one:
- At or below the threshold (4 KB in the demo): payload is stored inline in history, prefixed with
0x00. - Above the threshold: payload is written to an external blob store. Only a small JSON reference (the blob key) is stored in history, prefixed with
0x01.
On the decode side, the worker reads the prefix byte and either uses the inline payload directly or fetches it from the blob store.
Encode: large payload goes to the blob store; only a tiny reference enters Cadence history.
Threshold tuning
The default threshold in the samples is 4 KB. In production, profile your actual payload sizes first:
- Too low: nearly everything gets offloaded, meaning more blob store API calls and more latency per payload.
- Too high: large payloads still slip into Cadence history and approach the 2 MB cap.
A reasonable starting point for most teams is 64 KB–256 KB, then adjust based on observed payload distribution.
Claim-check stores only a ~100 B JSON reference in Cadence history regardless of payload size. The actual payload lives in the blob store.
Blob store options
Writes blobs to a local directory. No credentials needed. Works out of the box with the sample repos. Not suitable for production: blobs are not shared across machines and are lost on pod restart.
store := blobstore.NewFilesystemStore("/tmp/cadence-blobs")
dc := claimcheck.NewDataConverter(store, 4*1024)All backends implement the same BlobStore interface, so swapping is a one-line config change.
The one rule you cannot skip
Blob keys must be deterministic.
Cadence can replay a workflow from the beginning at any time (to recover from a crash, to apply a code change, or during normal task processing). When it does, ToData is called again with the same payload.
If your key is a UUID, every replay writes a new blob and orphans the old one. After enough replays, you have a storage bill full of blobs no one can find.
The fix is a single line: use SHA-256 of the serialized payload as the key. The same payload always produces the same key. The Put on replay is a no-op.
Common production pitfalls
What's next in this series
Week 3: Encrypt Cadence History Payloads (And Know What You Didn't Encrypt)
Key management, the CADENCE_ENCRYPTION_KEY environment variable, and a practical map of what AES-256-GCM encryption protects in Cadence versus what it leaves exposed. If you handle regulated data, this is the post to bookmark.