Development Environment and CI
I strive to automate all project tasks and use the same set of solutions for both local development and CI. This includes not just test/build/deploy, but all other repetitive tasks, such as developer environment initialization, data preparation, documentation generation, and so on. This provides independence from the current CI implementation, avoids duplication of functions, ensures constant dogfooding by developers, and encourages teams to independently manage and develop the CI of their product. This solution also has an obvious downside — one has to satisfy the requirements for both worlds within a single solution. Most often, build speed, readability of results, and simplicity suffer from this.
Main requirements:
- Isolation: Running all tools in Docker containers
- Scripting: Support for modern languages (TS/Go/Py) instead of pure bash/YAML
- Open Source: No lock-in to SaaS, possibility of forking
- Language agnostic: One framework regardless of the project’s stack
- Seamless CI: Running the same scripts locally and in CI
- Cacheability: BuildKit cache, remote-cache, Depot?, parallelism
- Security: rootless-user, secrets-mounts
- Multi-platform: arm64/amd64, macOS + Linux (OrbStack/WSL-2)
- Low entry barrier: minimum concepts and necessary commands
I periodically return to this topic, and this time I came across Dagger.io. It satisfies most of the requirements on the list.
| Requirement | Dagger Fit |
|---|---|
| Isolation | ✅ All stages run in containers via BuildKit |
| Low entry barrier | ⚠️ Simple CLI, but new abstractions needed (Graph, functions) |
| Scripting (TS/Go/Py) | ✅ Support for Go, Python, TypeScript SDK |
| Open Source | ✅ Engine and SDK are MIT-licensed, can be forked |
| Language agnostic | ✅ Containerized approach is independent of the application stack |
| Seamless CI | ✅ Same code works locally and in most CIs (Docker-in-Docker or sibling) |
| Cacheability | ✅ BuildKit cache and automatic function caching; ⚠️ Remote-cache requires Dagger Cloud |
| Security | ⚠️ Has rootless-mode and secrets-mounts, but image scanning must be connected manually |
| Multi-platform | ✅ Works on macOS (arm64/amd64) and Linux; Windows is in beta |
Pros
- Unified code for CI and local development — “no more push & pray.”
- Programmability: full languages instead of YAML allow for conditions, loops, and reusable functions.
- Caching and parallelism out of the box thanks to BuildKit.
- Modularity: Daggerverse and custom modules simplify pipeline reuse.
- Good DX: interactive execution with
dagger run, built-in tracing.
Cons
- Learning curve: need to understand the graph model and BuildKit principles.
- Young project: API might still change, fewer examples than GitHub Actions.
- Docker dependency: won’t run without a container daemon (OrbStack/Colima/WSL-2).
- Windows: official support is in beta.
Pitfalls and Recommendations
- Remote Cache = Dagger Cloud. For a full remote cache, a SaaS account is needed; offline mode only supports local cache.
- Secrets. Separate logic for passing secrets (env/secrets API) is needed, otherwise they might accidentally end up in logs.
- Performance on macOS ARM. BuildKit in a container might run slower due to x86 emulation; be careful with multi-arch.
- Artifact delivery. Dagger itself doesn’t push to registries/storages — you need to explicitly describe steps or connect modules.
- Debugging. For errors,
dagger sessionwith--debugsometimes helps, otherwise the call stack is non-obvious.
In the end, Dagger covers most of my requirements and looks promising as a unified tool for dev-env and CI. Critical risks are the youth of the ecosystem and dependency on Dagger Cloud for remote cache. If the team is ready to invest time in learning new concepts and is okay with the current level of stability, Dagger could become the foundation of future infrastructure.