markov

Markov Documentation

Markov is a Go-based YAML workflow engine for Kubernetes. Define workflows declaratively, and Markov executes them as K8s Jobs, HTTP requests, or shell commands with built-in concurrency, conditionals, checkpoint/resume, and a rule engine.

Quick Links

• Getting Started — install, validate, run your first workflow
• Core Concepts — understand the execution model
• Workflow File Reference — complete YAML schema
• CLI Reference — all commands and flags

Getting Started

New to markov? Start here:

1. Getting Started — from zero to a running workflow
2. Core Concepts — the mental model behind markov
3. Writing Workflows — build a real workflow step by step

Language Reference

Complete, field-by-field documentation for the markov YAML DSL:

• Workflow File Schema — top-level fields, workflow, step, rule, artifact, and step_type schemas
• Step Types — all built-in types: shell_exec, script_exec, prompt, k8s_job, http_request, gate, set_fact, load_artifact, assert
• Custom Step Types — defining reusable step types with base/job/defaults/params
• Template Engine — Pongo2 templates, filters, expressions, and type coercion
• Rule Engine — rules, gates, condition syntax, FactStore, and Grule evaluation
• Fan-Out (for_each) — for_each, for_each_key, for_each_sort, concurrency
• Variables and Context — vars, register, set_fact, context resolution, and data flow
• Artifacts — loading YAML, markdown, and table files into the context
• Callbacks — event system, HTTP/gRPC/JSONL backends, event types
• CLI — all commands and flags
• Step-Through Debugging — breakpoints, step mode, and the stdin control protocol

• State Store — checkpoint/resume, SQLite and Postgres backends, resume flow

• Jev decisions — connections, native steps, inline conditions, and decision-backed gates

Guides

Practical how-to guides with complete examples:

• Testing with CPU Kev — recreate the pinned local server and verify authenticated Jev steps and offline resume
• Writing Workflows — anatomy of a workflow, variables, register, conditions, custom types
• Fan-Out Patterns — for_each concurrency, sub-workflow fan-outs, stable keys
• Resuming Workflows — checkpoint/resume workflow operations, caveats, and troubleshooting
• Using Postgres for Markov State — shared durable state store setup, Kubernetes wiring, and operations
• Looping with Gates — recursive workflows, retry loops, gate-controlled iteration
• Conditional Logic — when vs gates vs assert, decision routing patterns
• Running in Kubernetes — in-cluster auth, RBAC, state persistence, callbacks
• Debugging — verbose/debug output, status inspection, diagrams, common errors

Design Documents

Internal design docs and research notes (preserved for reference):

• MVP Design — original architecture and design decisions
• Research Summary — business case and market analysis
• Project Comparisons — vs Ansible, Argo, Jenkins, Tekton, Kestra, CrewAI, LangGraph
• Callbacks Design — callback system design and trade-offs
• Agent Runner — AI agent pipeline lifecycle
• K8s Job Support — K8s Job integration design
• Callback Jobs Startup — callback initialization patterns
• fromjson Support — JSON parsing filter design
• Key Sorting — stable iteration key design
• Proposed Idempotency and Reconciliation Workflow — design example for retry-safe external effects

How to Use These Docs

• Writing your first workflow? Start with Getting Started, then Writing Workflows.
• Looking up a specific field? Use the Workflow File Reference or Step Types Reference.
• Understanding a pattern? Read the relevant Guide.
• Debugging a problem? See Debugging.