Skip to content

Commit

Permalink
Merge pull request #25 from depot/onboarding-updates
Browse files Browse the repository at this point in the history
Onboarding updates & clean up
  • Loading branch information
kylegalbraith authored Oct 11, 2023
2 parents 084b078 + 80c936b commit 17c3372
Show file tree
Hide file tree
Showing 4 changed files with 60 additions and 63 deletions.
35 changes: 19 additions & 16 deletions content/cli/reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,27 @@ description: A reference for the `depot` CLI, including all config, commands, fl

Below is a reference to the `depot` CLI, including all config, commands, flags, and options. To submit an issue or features please see our CLI repo over on [GitHub](https://github.com/depot/cli).

## Commands
## Specifying a Depot project

### `depot bake`
Some commands need to know which [project](/docs/core-concepts#projects) to route the build to.

Run a Docker build from a HCL, JSON, or Compose file using Depot's remote builder infrastructure. This command accepts all the command line flags as Docker's `docker buildx bake` command, you can run `depot bake --help` for the full list.
For interactive terminals calling [`build`](#depot-build) or [`bake`](#depot-bake), if don't specify a project, you will be prompted to choose a project when using an interactive prompt and given the option to save that project for future use in a `depot.json` file.

The `bake` command needs to know which [project](/docs/core-concepts#projects) id to route the build to. For passing the project id you have three options available to you:
Alternatively, you can specify the Depot project for any command using any of the following methods:

1. Run `depot init` at the root of your repository and commit the resulting `depot.json` file
2. Use the `--project` flag in your `depot bake` command
3. Set the `DEPOT_PROJECT_ID` environment variable which will be automatically detected
1. Use the `--project` flag with the ID of the project you want to use
2. Set the `DEPOT_PROJECT_ID` environment variable to the ID of the project you want to use

By default, `depot bake` will leave the built image in the remote builder cache. If you would like to download the image to your local Docker daemon (for instance, to `docker run` the result), you can use the `--load` flag.
## Commands

Alternatively, to push the image to a remote registry directly from the builder instance, you can use the `--push` flag.
### `depot bake`

The `bake` command allows you to define all of your build targets in a central file, either HCL, JSON, or Compose. You can then pass that file to the `bake` command and Depot will build all of the target images with all of their options (i.e. platforms, tags, build arguments, etc.).

By default, `depot bake` will leave the built image in the remote builder cache. If you would like to download the image to your local Docker daemon (for instance, to `docker run` the result), you can use the `--load` flag.

Alternatively, to push the image to a remote registry directly from the builder instance, you can use the `--push` flag.

**Example**

An example `docker-bake.hcl` file:
Expand Down Expand Up @@ -62,6 +65,8 @@ depot bake -f docker-bake.hcl original

#### Flags for `bake`

This command accepts all the command line flags as Docker's `docker buildx bake` command.

{/* <!-- prettier-ignore-start --> */}
| Name | Description |
| ---- | ----------- |
Expand All @@ -80,19 +85,14 @@ depot bake -f docker-bake.hcl original
| `pull` | Always attempt to pull all referenced images |
| `push` | Shorthand for "--set=\*.output=type=registry" |
| `sbom` | Shorthand for "--set=\*.attest=type=sbom" |
| `sbom-dir` | Directory to store SBOM attestations |
| `set` | Override target value (e.g., "targetpattern.key=value") |
| `token` | Depot API token |
{/* <!-- prettier-ignore-end --> */}

### `depot build`

Runs a Docker build using Depot's remote builder infrastructure. This command accepts all the command line flags as Docker's `docker buildx build` command, you can run `depot build --help` for the full list.

The `build` command needs to know which [project](/docs/core-concepts#projects) id to route the build to. For passing the project id you have three options available to you:

1. Run [`depot init`](/docs/cli/reference#depot-init) at the root of your repository and commit the resulting `depot.json` file
2. Use the `--project` flag in your `depot build` command
3. Set the `DEPOT_PROJECT_ID` environment variable which will be automatically detected
Runs a Docker build using Depot's remote builder infrastructure.

By default, `depot build` will leave the built image in the remote builder cache. If you would like to download the image to your local Docker daemon (for instance, to `docker run` the result), you can use the `--load` flag.

Expand Down Expand Up @@ -122,6 +122,8 @@ depot build -t repo/image:tag . --push

#### Flags for `build`

This command accepts all the command line flags as Docker's `docker buildx build` command.

{/* <!-- prettier-ignore-start --> */}
| Name | Description |
| ---- | ----------- |
Expand Down Expand Up @@ -154,6 +156,7 @@ depot build -t repo/image:tag . --push
| `push` | Shorthand for "--output=type=registry" |
| `quiet` | Suppress the build output and print image ID on success |
| `sbom` | Shorthand for "--attest=type=sbom" |
| `sbom-dir` | Directory to store SBOM attestations |
| `secret` | Secret to expose to the build (format: "id=mysecret[,src=/local/secret]") |
| `shm-size` | Size of "/dev/shm" |
| `ssh` | SSH agent socket or keys to expose to the build |
Expand Down
10 changes: 7 additions & 3 deletions content/overview/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,17 @@ You can use [`depot bake -f docker-compose.yml`](/docs/cli/reference#depot-bake)

We have all our authentication options documented for `depot` in our [CLI authentication documentation](/docs/cli/authentication).

### Can I build Docker images for M1 Macs?
### How do I push my images to a private registry?

Yes! Depot supports native Arm container builds out of the box. We detect the architecture of the machine requesting a build via `depot build`. If that architecture is Arm, we route the build to a builder running Arm natively. You can build Docker images for M1 Macs and run the resulting image immediately, as it is made specifically for your architecture. See our documentation on [Arm containers](guides/arm-containers) for more details.
You can use the `--push` flag to push your images to a private registry. Our `depot` CLI uses your local Docker credentials provider. So, any registry you've logged into with `docker login` or similar will be available when running a Depot build. See our guide on [private registries](/docs/guides/private-registries) for more details.

### Can I build Docker images for M1/M2 Macs?

Yes! Depot supports native Arm container builds out of the box. We detect the architecture of the machine requesting a build via `depot build`. If that architecture is Arm, we route the build to a builder running Arm natively. You can build Docker images for M1/M2 Macs and run the resulting image immediately, as it is made specifically for your architecture. See our documentation on [Arm containers](/docs/guides/arm-containers) for more details.

### Can I build multi-platform Docker images?

Yes! Check out our [integration guide](guides/arm-containers#what-about-multi-architecture-containers) on how we do it.
Yes! Check out our [integration guide](/docs/guides/arm-containers#what-about-multi-architecture-containers) on how we do it.

### Can I use Depot with my existing `docker build` or `docker buildx build` commands?

Expand Down
42 changes: 20 additions & 22 deletions content/overview/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,14 @@ import {DocsTOC} from '~/components/DocsTOC'

Welcome to Depot!

Depot is a remote container build service that makes image builds up to 20x faster than building Docker images inside generic CI providers (with some projects showing [even faster speedups](https://depot.dev/benchmark/mastodon)). Docker image builds get sent to a fast builder instance with a persistent cache. The resulting image can then be downloaded locally or pushed to a registry. Adopting Depot is easy, as the Depot CLI `depot build` command accepts the same arguments as `docker buildx build` or we have a command that points all `docker build` commands to Depot builders, [see our `docker build` guide](/docs/guides/docker-build).
Depot is a remote container build service that can build Docker images up to 20x faster than building Docker images inside generic CI providers (with some projects showing [even faster speedups](https://depot.dev/benchmark/temporal)).

A given Docker image build is routed to a fast builder instance with a persistent cache when using Depot. The resulting image can then be downloaded locally or pushed to a registry.

Adopting Depot is easy, and you can choose the path that works for you:

1. We have the Depot CLI with the `depot build` command that accepts the same arguments as `docker buildx build`
2. Or we have a command, `depot configure-docker` that points all `docker build` and `docker buildx build` commands to Depot builders, [see our Docker guide](/docs/guides/docker-build) for more details.

Best of all, Depot's build infrastructure requires zero configuration on your part; everything just works, including the build cache! You can think of Depot as a specialized CI service focusing on Docker containers.

Expand All @@ -36,29 +43,22 @@ Take a look at the [quickstart](/docs/quickstart) to get started.

First, you will create a **project** underneath an organization. Projects usually represent a single application, repository, or Dockerfile. Once you've made your project, you can leverage Depot builders from your local machine or an existing CI workflow in two ways:

1. Use the `depot build` command from your local machine or an existing CI workflow to execute the container build
2. Run `depot configure-docker` one time and then use `docker build` or `docker buildx build` as normal
1. Use [`depot build`](/docs/cli/reference#depot-build) from your local machine or an existing CI workflow
2. Run [`depot configure-docker`](/docs/cli/reference#depot-configure-docker) one time and then use `docker build` or `docker buildx build`

Builder instances come with 16 CPUs, 32GB of memory, and an SSD disk for layer caching (the default size is 50GB). In addition, they run the latest version of [BuildKit](https://github.com/moby/buildkit), the advanced build engine that backs Docker.
Builder instances come with 16 CPUs, 32GB of memory, and an SSD disk for layer caching (the default size is 50GB, but you can expand this up to 500GB). A builder instance runs an optimized version of [BuildKit](https://github.com/moby/buildkit), the advanced build engine that backs Docker.

We offer Intel and Arm builder instances for all projects, so both architectures build without slow emulation.
We offer native Intel and Arm builder instances for all projects. Hence, both architectures build with zero emulation, and you don't have to run your own build runners to get native multi-platform images.

Once built, the image can be left in the build cache (the default) or downloaded to the local Docker daemon with `--load` or pushed to a registry with `--push`. If `--push` is specified, the image is pushed to the registry directly from the remote builder via high-speed network links and does not use your local network connection. Example:

```shell
$ cd path/to/project
$ depot build -t repo/project:tag . --push # build and push to registry
```

For more information, see the [core concepts](/docs/core-concepts) page.
Once built, the image can be left in the build cache (the default) or downloaded to the local Docker daemon with `--load` or pushed to a registry with `--push`. If `--push` is specified, the image is pushed to the registry directly from the remote builder via high-speed network links and does not use your local network connection. See our [private registry guide]() for more details on pushing to private Docker registries like Amazon ECR or Docker Hub.

### General Architecture

![](/images/depot-overall-architecture.png)

The general architecture for Depot consists of our `depot` CLI, a control plane, an open-source `cloud-agent`, and builder virtual machines running our open-source `machine-agent` and BuildKit with associated cache volumes. This design provides faster Docker image builds with as little configuration change as possible.

The flow of a given Docker image build using `depot build`, or `docker build` when `depot configure-docker` has been used, looks like this:
The flow of a given Docker image build when using Depot looks like this:

1. The Depot CLI asks the Depot API for a new builder machine connection (with organization ID, project ID, and the required architecture) and polls the API for when a machine is ready
2. The Depot API stores that pending request for a builder
Expand All @@ -76,22 +76,20 @@ We built Depot based on our experience with Docker as both application and platf

Depot works best in the following scenarios:

1. **Building the Docker image is slow in CI** — common CI providers often do not have native support for Docker build cache, instead requiring cache to be saved to and loaded from tarballs, which can be extremely slow. In addition, CI providers typically offer limited resources, causing overall build time to be long.

For an example of what "slow" could mean, in many of our projects, we have seen [a 2-3 times build-time speedup](/#benchmarks) by switching to Depot. Docker builds that use optimized `Dockerfile`s can regularly achieve even greater speedups; some projects have reduced their build time from over 2 hours to just a few minutes. 🚀
1. **Building the Docker image is slow in CI** — common CI providers often do not have native support for the Docker build cache. Instead, they require layer cache to be saved to and loaded from tarballs over slow networks. Often, CI providers offer limited resources as well, causing overall build time to be long.

You do not have to switch to a different CI provider. Depot works within your existing workflow by swapping out the call to `docker build` with `depot build`. See [our guides](/docs/guides/continuous-integration) for more information.
Depot works within your existing CI workflow by swapping out the call to `docker build` with `depot build`. Or by configuring `docker` in your environment to leverage Depot. See [our continuous integration guides](/docs/guides/continuous-integration) for more information.

2. **You need to build images for multiple platforms (Intel and Arm)**Depot's Intel and Arm builder instances can build both CPU architectures natively without any slow emulation. This is a valuable feature if you need to build Docker images for a platform that differs from your current host. For instance, if you are on an M1 Mac and have to build an Intel image, or if you have to build an Arm image from your CI provider that only offers Intel runners.
2. **You need to build images for multiple platforms/multiple architectures (Intel and Arm)**today, you're often stuck with managing your own build runner or relying on slow emulation in order to build multi-platform images. For example, CI providers usually run their workflows on Intel machines. So, to create a Docker image for Arm, you either have to launch your own BuildKit builder for Arm and connect to it from your CI provider. Or build your Arm image with slow QEMU emulation.

Depot can [build **multi-architecture** images](/docs/guides/arm-containers) in a single pass, so if you have to build and push a multi-architecture image to your registry to be used by both CPU architectures, Depot can do this.
Depot can [build multi-platform and Arm images](/docs/guides/arm-containers) natively with zero-emulation and without running additional infrastructure.

3. **Building the Docker image on your local machine is slow or expensive**since Depot executes builds on remote compute infrastructure, it offloads the CPU, memory, disk, and network resources required to that remote builder. If builds on your local machine are slow due to constrained compute, disk, or network, `depot build` eliminates the need to rely on your local environment. This also applies to CPU architecture; if you need to build a Docker image without CPU emulation, offloading the build to Depot is the fastest approach.
3. **Building the Docker image on your local machine is slow or expensive**Docker can hog resources on developer machines, taking up valuable network, CPU, and memory resources. Depot executes builds on remote compute infrastructure; it offloads the CPU, memory, disk, and network resources required to that remote builder. If builds on your local machine are slow due to constrained compute, disk, or network, `depot build` eliminates the need to rely on your local environment.

Additionally, since the project build cache is available remotely, multiple people can send builds to the same project and benefit from the same cache. If your coworker has already built the same image, your `depot build` command will re-use the previous result. This is especially useful for very slow builds, or for example, in reviewing a coworker's branch, you can pull their Docker image from the cache without an expensive rebuild.

## When not to use Depot?

Depot is not a replacement for a CI provider. It is a specialized service for building Docker images in the most performant way possible. You will still need a CI provider if you need to run tests, deploy, or perform other CI tasks. Depot is designed to work with your existing CI provider, not replace it.
Depot is not a replacement for a CI provider. It is a specialized service for building Docker images in the most performant way possible. You will still need a CI provider to run tests, deploy, or perform other CI tasks. Depot is designed to work with your existing CI provider, not replace it.

See our [Continous Integration guide](/docs/guides/continuous-integration) for more information.
Loading

0 comments on commit 17c3372

Please sign in to comment.