When an image pull fails but the service has a `build` section, it
will be built, so it's not an unrecoverable error. It's now logged as
a warning - in situations where the image will _never_ exist in a
registry, `pull_policy: never` can & should be used, which will
prevent the error and avoid unnecessary pull attempts.
Signed-off-by: Risky Feryansyah Pribadi <riskypribadi24@gmail.com>
* logs: filter to services from current Compose file
When using the file model, only attach to services
referenced in the active Compose file.
For example, let's say you have `compose-base.yaml`
and `compose.yaml`, where the former only has a
subset of the services but are both run as part of
the same named project.
Project based command:
```
docker compose -p myproj logs
```
This should return logs for active services based
on the project name, regardless of Compose file
state on disk.
File based command:
```
docker compose --file compose-base.yaml logs
```
This should return logs for ONLY services that are
defined in `compose-base.yaml`. Any other services
are considered 'orphaned' within the context of the
command and should be ignored.
See also #9705.
Fixes#9801.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
When running `compose down`, the `--rmi` flag can be passed,
which currently supports two values:
* `local`: remove any _implicitly-named_ images that Compose
built
* `all` : remove any named images (locally-built or fetched
from a remote repo)
Removing images in the `local` case can be problematic, as it's
historically been done via a fair amount of inference over the
Compose model. Additionally, when using the "project-model"
(by passing `--project-name` instead of using a Compose file),
we're even more limited: if no containers for the project are
running, there's nothing to derive state from to perform the
inference on.
As a first pass, we started labeling _containers_ with the name
of the locally-built image associated with it (if any) in #9715.
Unfortunately, this still suffers from the aforementioned problems
around using actual state (i.e. the containers might no longer
exist) and meant that when operating in file mode (the default),
things did not behave as expected: the label is not available
in the project since it only exists at runtime.
Now, with these changes, Compose will label any images it builds
with project metadata. Upon cleanup during `down`, the engine
image API is queried for related images and matched up with the
services for the project. As a fallback for images built with
prior versions of Compose, the previous approach is still taken.
See also:
* https://github.com/docker/compose/issues/9655
* https://github.com/docker/compose/pull/9715
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
support DOCKER_DEFAULT_PLATFORM when 'compose up --build'
add tests to check behaviour when DOCKER_DEFAULT_PLATFORM is defined
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
The `compose-spec/compose-go` lib is written with `gopkg.in/yaml.v2`
as a target.
When marshalling via CLI (`compose convert` / `compose config`), we
were using a _different_ YAML lib, which was a fork of `go-yaml`,
which is what `gopkg.in/yaml.v2` is based off of.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This keeps parity with v1, where only the containers explicitly
passed to `up` are torn down when `Ctrl-C` is hit, so any
dependencies that got launched (or orphan containers hanging
around) should not be touched.
Fixes#9696.
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
This updates the format of various usage strings to be more consistent
with other parts of the CLI.
- Use `[OPTIONS]` to indicate where command-specific options should be added
- Use `[SERVICE...]` to indicate zero-or-more services
- Remove some usage strings for specific options (e.g. `-e NAME=VAL`), as that
option is part of the already mentioned `[OPTIONS]` and we don't provide usage
for each possible option that can be passed.
- Remove `[--]`, which (I think) was needed for the Python implementation, but is
a general feature to stop processing flag-options.
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
If named pipe mounts are added to the volumeMounts mapping, the docker daemon will report an error that it cannot be mapped.
Signed-off-by: Robert Schumacher <ras0219@outlook.com>
NetworkList API doesn't return the extact name match, so we can retrieve more than one network with a request
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
As of Go 1.16, the same functionality is now provided by package io or
package os, and those implementations should be preferred in new code.
So replacing all usage of ioutil pkg with io & os.
Signed-off-by: Abhinav Nair <11939846+abhinavnair@users.noreply.github.com>
When using the "classic" (non-BuildKit) builder, ensure that
services are iterated in dependency order for a build so that
it's possible to guarantee the presence of a base image that's
been added as a dependency with `depends_on`. This is a very
common pattern when using base images with Compose.
A fix for BuildKit is blocked currently until we can rely on a
newer version of the engine (see docker/compose#9324)[^1].
[^1]: https://github.com/docker/compose/issues/9232#issuecomment-1060389808
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Within the Docker API/engine, networks have unique IDs, and the
name is a friendly label/alias, which notably does NOT have any
guarantees around uniqueness (see moby/moby#18864 [^1]).
During day-to-day interactive/CLI Compose usage, this is rarely
an issue, as Compose itself isn't creating networks concurrently
across goroutines. However, if multiple Compose instances are
executed simultaneously (e.g. as part of a test suite that runs
in parallel), this can easily occur.
When it does happen, it's very confusing for users and resolving
it via the `docker` CLI is not straightforward either [^2].
There's two primary changes here:
* Pass `CheckDuplicates: true` to the Docker API when creating
networks to reduce the likelihood of Compose creating duplicates
in the first place
* On `down`, list networks using a name filter and then remove
them all by ID, as the Docker API will return an error if the
name alias is used and maps to more than one network
Hopefully, this provides a better UX, since the issue should be
less likely to occur, and if it does, it can now be resolved via
standard Compose workflow commands.
[^1]: https://github.com/moby/moby/issues/18864
[^2]: https://stackoverflow.com/questions/63776518/error-2-matches-found-based-on-name-network-nameofservice-default-is-ambiguo
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Also add e2e tests to ensure `compose up --wait` does not get stuck forever waiting for one-shot containers
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
In v1, links were sent alongside the rest of the container create request, as part of `HostConfig`. In v2, links are usually set on the connect container to network request that happens after the create. However, this only happens if the service has one or more networks defined for it. If the services are configured to use the default bridge network, this request is not made and so links are never configured.
Signed-off-by: Laura Brehm <laurabrehm@hey.com>
`NetworkInspect` will match a network ID by prefix. While rare,
it's possible that users might use a network name that is also
a valid network ID prefix for a pre-existing Docker network.
(In the reported case, the network was named `db`, for example.)
Fixes#9496.
Signed-off-by: Milas Bowman <milas@tilt.dev>
A command down without any resource to delete was
trying to remove a default network that doesn't exist
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
This commit adds progress output while waiting for `depends_on`
conditions to resolve. The initial output looks like so:
⠿ Container chbench-zookeeper-1 Waiting 0s
⠿ Container chbench-kafka-1 Waiting 0s
⠿ Container chbench-one-off Waiting 0s
Once all conditions have been resolved, the ouput looks like this:
⠿ Container chbench-zookeeper-1 Healthy 1.2s
⠿ Container chbench-kafka-1 Healthy 3.2s
⠿ Container chbench-schema-registry-1 Exited 4s
As shown above, `service_healthy` conditions result in a terminal status
of "Healthy" while `service_exited_successfully` conditions result in a
terminal status of "Exited".
Signed-off-by: Nikhil Benesch <nikhil.benesch@gmail.com>
The previous code would wait for dependencies to become healthy forever,
even if they'd become unhealthy in the meantime. I can't find an issue
report for this bug, but it was described in a comment on the PR that
introduced the `--wait` flag [0].
[0]: https://github.com/docker/compose/pull/8777#issuecomment-965643839
Signed-off-by: Nikhil Benesch <nikhil.benesch@gmail.com>
Added unsupported volume bind option SELinux label `:z` and `:Z` in v2.
It is a regression compared to v1 written in Python.
The v2 uses the compose-spec/compose-go to parse Compose YAML
specification files but there was missing support for volume bind option
SELinux label `:z` and `:Z` in parser. It is fixed in:
- https://github.com/compose-spec/compose-go/pull/213
It fixes#9072
References:
- https://docs.docker.com/storage/bind-mounts/#configure-the-selinux-label
Signed-off-by: Tymoteusz Blazejczyk <tymoteusz.blazejczyk@tymonx.com>
The `group_add` key is parsed correctly from a compose file, but it is not
passed into the `ContainerCreate` API call, thus the configuration does
not take effect. This commit fixes the issue by propagating the
configuration from Docker compose's ServiceConfig to Docker container's
HostConfig.
Signed-off-by: Lance Chen <hello@lancechen.tw>
Intermediate containers remain even when build succeeded
when building with classic way (i.e. not with buildkit).
Remove them when build succeeded like default behavior of docker build.
Signed-off-by: notok <noto.kazufumi@gmail.com>
When a service has 0 replicas to start, it don't try to create any
container. However `composeService.startService()` still gets executed.
This method checks if there's been containers created for that service
beforehand. If there's none, it returns the following error: `no
containers to start`.
This change checks if replicas == 0 and exits early from
`composeService.startService()` when that's the case.
Signed-off-by: Albin Kerouanton <albinker@gmail.com>
Fixes#8747
When the event is a container destroy, calling ContainerInspect returns
an error, because the container no longer exists. This causes both
`docker-compose up` and `docker-compose logs -f` to exit when removing a
stopped container.
This container has already emitted its die event, and has already been
cleaned up. I believe all that needs doing in this case is to early-out.
Signed-off-by: Stephen Thirlwall <sdt@dr.com>
Milas Bowman
Risky Feryansyah
Lucas Berg
Laura Brehm
Laura Brehm
Guillaume Lours
Risky Feryansyah Pribadi
Guillaume Lours
Ulysses Souza
Michael Irwin
Nick Sieger
Vedant Koditkar
Vedant Koditkar
Nicolas De Loof
ikedam
Sebastiaan van Stijn
Jeremiah Millay
Kenny Millington
Robert Schumacher
Pablo Rodriguez Quesada
Abhinav Nair
Bastian Venthur
Milas Bowman
Eric Fan
Mehrad Dadar
Mehrad Dadar
Ulysses Souza
Yuri Kanivetsky
Nikhil Benesch
Tymoteusz Blazejczyk
Lance Chen
notok
Kevin Roy
Kyungsik Park
Albin Kerouanton
Pierre-Antoine Guillaume
Stephen Thirlwall