Bridging Local Development and Cloud Deployment with Docker Compose and Defang

1. Introduction

Modern software development demands rapid iteration, seamless collaboration, and reliable deployment across diverse cloud environments. Docker revolutionized containerization and local development workflows, empowering developers with reproducible environments. Yet, bridging the gap between local development and cloud deployment remains challenging.

Defang addresses this gap by extending the Docker Compose model to native deployments on a variety of popular cloud platforms such as AWS, GCP, and DigitalOcean. This enables developers to use familiar tooling and workflows to deploy multi-service applications to the cloud of their choice without having to learn a whole new set of concepts and tools.

This white paper explores how the combination of Docker and Defang streamlines development and deployment, empowering developers to build and deploy modern applications efficiently.

2. What Is Defang?

Defang is a deployment tool for developers designed to extend Docker Compose into the cloud. While Docker Compose excels at defining and orchestrating multi-container applications locally via the signature ‘docker compose up’ command, Defang enables those same Compose definitions to be deployed directly to cloud infrastructure — such as AWS, Google Cloud Platform (GCP), and DigitalOcean — using an analogous ‘defang compose up’ command.

Defang preserves the Compose developer abstraction, and adds cloud-native capabilities such as DNS, advanced networking, efficient and scalable compute (including auto-scaling and GPU support), managed storage services such as Postgres, MongoDB, and Redis, and even managed LLMs. Defang enables the same Compose file used in the local desktop environment to be deployed to development, staging, or production environments in the cloud. And all this while ensuring that each deployment is appropriately optimized for security, scalability, performance, observability, and cost according to the best practices for the specific target cloud.

3. The Power of Docker + Defang

By combining Docker and Defang, developers benefit from a unified, consistent workflow for both “inner-loop” local development and “outer-loop” cloud deployment:

  • Simple and Concise App Definition: Docker Compose is the simplest and most concise way to define the “what” of an application, including domain names, networks, application services, storage, LLMs, dependencies, etc. With Defang, developers don’t need to learn a whole other Infrastructure-as-Code (IaC) tool-set (such as Terraform, CloudFormation, or Pulumi) to describe the “how” of application deployment. All you need is Compose!
  • Seamless Local-to-Cloud Transition: Over 20M developers already develop locally with Docker and Compose. With Defang, these developers can now deploy the exact same app definition to the cloud of their choice with a single command. Developers no longer have to learn and reason about a completely separate set of abstractions such as load balancers, networking configurations, container orchestrators, managed storage and LLM implementations, security and observability considerations, and so on.
  • Simplified Multi-Cloud Support: Defang allows developers to deploy their Compose project to any of AWS, GCP, or DigitalOcean platforms (and more in the future), without rewriting their application code or their Compose application definition. Defang does all the heavy lifting of mapping the application components specified in the Compose file to an optimal implementation, compliant with the best practices for the target cloud. Defang even supports different deployment modes (affordable, balanced, and high_availability) which are optimized to be used for development, staging, and production environments respectively.
  • Integrated Developer Experience: Defang integrates directly into popular IDEs such as VS Code, Cursor, and Windsurf via the Defang MCP Server, enabling developers to deploy their applications with natural language commands, while preserving a formal Compose app definition as the underlying foundation. Defang even supports cost estimation, allowing developers to estimate their monthly cloud bill on different cloud platforms before choosing the platform that best suits their needs of their deployment.

For all these reasons, the combination of Docker Compose and Defang is a revolutionary step forward in making life simpler for developers of all skill and experience levels.

4. Docker Compose Support in Defang

Docker Compose is a well-established and simple yet powerful way to specify multi-container applications. Defang supports many aspects of the Compose specification, allowing developers to deploy a rich set of applications to the cloud of their choice. Defang even uses the Compose Extensions mechanism to provide additional cloud-specific capabilities that don’t make sense in a local environment. We will cover each of these briefly in the section below, linking to the appropriate sections of the Compose and Defang documentation for details as appropriate.

4.1. Service Definition and Orchestration

Specifying a multi-service application is at the heart of Compose. Defang provisions and sets up seamless orchestration of these services using the best-fit native container orchestration service in the target cloud (e.g. ECS/Fargate on AWS or CloudRun on GCP). Defang supports resource reservation for CPUs, memory, and GPUs, allowing developers to declaratively describe their application’s resource requirements. Defang even supports an auto-scaling option with an x-defang-autoscaling extension.

services:
  app:
    x-defang-autoscaling: true
    deploy:
      resources:
        reservations:
          cpus: '2.0'
          memory: 1024M
          devices:
            - capabilities: ["gpu"]

4.2. Domain Names

Compose allows specification of a domainname parameter to declare a custom domain name to use for the service. Defang leverages this property with its corresponding Domains feature, so you can hook up a custom domain (as well as aliases) to your cloud-deployed service. Defang also handles certificate generation if needed.

services:
  app:
    domainname: example.com
    networks:
      default:
        aliases:
          - www.example.com # a public alias

4.3. Networking

Compose allows the specification of named networks that can be reused across multiple services. Defang’s corresponding networking feature allows provisioning of both public and private networks. Defang takes care of the underlying implementation based on the specific networking infrastructure of the target cloud provider. For example, on AWS this results in different Security Groups with permissions to allow communication within the same project.

services:
  # this service can receive public traffic and communicate to private services
  web: 
    ports:
      - 80
    networks:
      default:
      private:
  # this service can only receive traffic from other services in the same network
  db: 
    ports:
      - 1234
    networks:
      private:
networks:
  private:
  # any network that's not "default" is considered private

4.4. Managed Storage Services

The Compose specification does not explicitly mention persistent storage services such as databases, object stores, or queues. Defang can of course deploy these like any other stateless services, but Defang has developed custom extensions that make it easy for developers to deploy these as managed storage services such as Postgres, MongoDB, and Redis. Defang takes care of mapping these to managed implementations on the target provider (e.g. RDS for Postgres on AWS) and seamlessly hooking things up so no app changes are needed.

services:	
  sql:
    image: postgres:14
    x-defang-postgres: true
  cache:
    image: redis:6.2
    x-defang-redis: true
  nosql:
    image: mongo:5
    x-defang-mongodb: true

4.5. Managed LLMs

Docker recently added support for running LLMs locally via the Docker Model Runner, and also added support for this in Compose via the provider service type. Defang natively supports this provider service type to deploy LLMs in the cloud, giving developers ergonomic access to native managed LLM capabilities (e.g. Bedrock on AWS, Vertex AI on GCP). Defang even maps the model name from the Docker model repository to the appropriate name on the target cloud. This makes it possible for the same application code and Compose file to be run locally and in the cloud with no changes, giving developers a powerful local and cloud environments for building AI applications.

# using openai access gateway
services:
  app:
    # [...]
  llm:
    x-defang-llm: true
    image: docker.io/defangio/openai-access-gateway:latest

# with Docker Compose provider syntax
services:
  app:
    # [...]
  llm:
    provider:
      type: model
      options:
        model: ai/mistral
    x-defang-llm: true

4.6. Interpolation

Values in a Compose file can be interpolated at runtime. Defang also supports such interpolation. In addition, Defang also allows you to securely store sensitive information such as API keys, passwords, and other credentials.

services:
  app:
    environment:
      DB_PASSWORD: # an environment variable without a value is assumed to be a secret, and will be loaded from defang config. 
      DB_URL: dbservice://username:${DB_PASSWORD}@example.com:9876/dbname

4.7. Advanced Compose Syntax

Defang uses the official Open Source compose-go library and supports advanced Compose options such as depends_on, include, extends, and profiles mirroring the behavior of Docker Desktop during local development.

4.8. Build

Defang supports the Compose build syntax. When using this syntax, Defang will upload the source code to the user’s cloud account, taking into consideration their .dockerignore. It will then build the container image using build arguments, provision a container registry, and push the image to the registry. All of this happens within the user’s cloud account.

services:
  app:
    build:
      context: ./app
      dockerfile: Dockerfile

To summarize, Defang supports much of the Compose specification for cloud deployments, and adds custom extensions where necessary to provide the best cloud deployment experience for Docker Compose users.

5. Additional Features

In this section, we cover some additional features of Defang, which, while not directly relating to Compose support, enhance the deployment capability and experience in important ways.

5.1. Deployment Modes

Defang supports 3 built-in deployment modes: affordable, balanced, and high_availability. Defang makes different performance, cost, and resiliency tradeoffs based on the deployment mode. For example, affordable mode deployments are optimized to reduce cost (suitable for development / testing), whereas high_availability mode deployments are optimized to maximize performance and resiliency (suitable for production).

# deployment mode
defang compose up --mode=[affordable, balanced, high_availability]

5.2. Cost Estimation

Defang allows the developer to estimate the cost of a cloud deployment of a Compose project before actually deploying it. This allows developers to ensure that the cost will be in line with their expectations, and also allows them to compare estimated costs across different cloud providers and choose the one that suits them the best.

# cost estimation
defang estimate --mode=[affordable|balanced|high_availability] --provider=[aws, gcp, do]

5.3. Managing Local versus Cloud Compose Files

While Defang encourages reusing a single Compose file, developers can maintain environment-specific overrides to handle cloud-specific parameters. Defang supports merging Compose files to accommodate local and cloud differences efficiently. This is done using standard Docker Compose syntax, like passing multiple compose files with the -f flag, or using the include syntax, or the extends syntax (per service). See this docs page on local development for an example.

5.4. Managing Environment Variables

Defang provides the ability to manage environment variables, including sensitive values. Environment variables are simply passed along to the cloud service as defined in the Compose configuration. Sensitive values can be set, edited, and removed. They are stored, encrypted, using the parameter store provided by the user’s cloud account.

5.5. Adding Defang into your Development Environment

As documented here, there are a variety of ways in which Defang can be easily added to your development environment using Codespaces or Dev Containers.

5.6. Integrating into CI/CD Pipelines

Defang can be easily integrated into GitHub-based CI/CD workflows using the Defang GitHub Action, allowing automated pipelines to build and deploy Compose-based apps with confidence.

5.7. Infrastructure-as-Code (IaC) Integration via Pulumi Provider

Sometimes there are aspects of an application that are outside the scope of a Compose specification. For such cases, Defang has implemented a Defang Pulumi Provider, which can be used to plug the deployment of a Compose project into a larger IaC deployment. This gives developers the ergonomics of compose with the full flexibility of bespoke infrastructure.

6. Docker + Defang in Action: Defang on Defang

An elegant demonstration of Defang’s power is Defang deploying itself. As detailed in a series of blog posts (Part 1, Part 2), Defang automates the provisioning and deployment of its own infrastructure and services (such as the Defang.io website, the Defang Portal, the Playground environment, as well at its CD back-end) across multiple clouds using Compose definitions - a testament to the simplicity, flexibility, and reliability of Docker + Defang deployments.

7. Future Enhancements

Defang’s roadmap includes important future enhancements such as:

  • Expanded storage options including (a) Object Storage (e.g. S3 in AWS) and (b) the Volumes element in Compose.
  • Ability for enterprises to provide more directional guidance for deployment configurations - e.g. to target (or avoid) Kubernetes-based infrastructure.
  • Advanced auto-tuning capabilities that optimize deployments based on run-time metrics
  • More granular deployment controls for Blue-Green and Canary releases.
  • Support for additional cloud platforms such as Microsoft Azure and others.

8. Conclusion

The combination of Docker and Defang offers a powerful, streamlined path from local development to cloud deployment. By leveraging Docker Compose’s simplicity and Defang’s cloud-native deployment capabilities, developers can accelerate delivery, reduce complexity, and embrace multi-cloud strategies without sacrificing familiar workflows. Developers evaluating deployment tools owe it to themselves to explore Docker + Defang - the future of multi-cloud application deployment built on the developer-centric foundation of Compose.