Commit Graph

48 Commits

Author SHA1 Message Date
Milas Bowman 3b0742fd57
watch: batch & de-duplicate file events (#10865)
Adjust the debouncing logic so that it applies to all inbound file
events, regardless of whether they match a sync or rebuild rule.

When the batch is flushed out, if any event for the service is a
rebuild event, then the service is rebuilt and all sync events for
the batch are ignored. If _all_ events in the batch are sync events,
then a sync is triggered, passing the entire batch at once. This
provides a substantial performance win for the new `tar`-based
implementation, as it can efficiently transfer the changes in bulk.

Additionally, this helps with jitter, e.g. it's not uncommon for
there to be double-writes in quick succession to a file, so even if
there's not many files being modified at once, it can still prevent
some unnecessary transfers.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-08-03 14:53:02 -04:00
Guillaume Lours bb94ea034e add support of depends_on.required attribute
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-07-18 23:13:47 +02:00
Nicolas De Loof b5f5e27597 don't use unitialized cli to setup DryRunClient
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-07-06 14:27:57 +02:00
Milas Bowman 37850f7955 ci: upgrade to Go 1.20.5 and Moby v24.x
Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2023-06-07 10:41:49 -04:00
Nicolas De Loof cfe1a860ff
fix detection of swarm mode
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-22 08:26:22 +02:00
Nicolas De Loof 00f72cb553 report external network not found when swarm is disabled
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-15 20:57:19 +02:00
Nicolas De loof a14abb9044
cli: option to write status messages on stdout (#10549)
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-05-11 12:45:00 -04:00
Sebastiaan van Stijn 1892be8c70
Don't use "info.IndexServerAddress" for authentication
The IndexServerAddress field was  as part of the initial Windows implementation
of the engine. For legal reasons, Microsoft Windows (and thus Docker images
based on Windows) were not allowed to be distributed through non-Microsoft
infrastructure. As a temporary solution, a dedicated "registry-win-tp3.docker.io"
registry was created to serve Windows images.

Using separate registries was not an ideal solution, and a more permanent
solution was created by introducing "foreign image layers" in the distribution
spec, after which the "registry-win-tp3.docker.io" ceased to exist, and
removed from the engine.

This replaces the code that calls out to the "/info" endpoint to use the
GetAuthConfigKey() function instead.

Related PR in docker/cli:
b4ca1c7368

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
2023-04-11 23:10:22 +02:00
Guillaume Lours 3f7d3c2661 add dry-run support for pull command
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-03-05 22:04:32 +01:00
Guillaume Lours 3cfbac6624
restart only needed services by checking depends_on relations
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-03-01 18:22:40 +01:00
Guillaume Lours 2336d9fe35 support dry-run for cp command
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-02-03 22:44:26 +01:00
Nicolas De Loof 41e056341b rename `convert` to `config` to align with compose v1 UX
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2023-01-30 22:23:53 +01:00
Guillaume Lours 790712fa92 update tty and plain text writers to support dry run mode
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-01-30 09:27:17 +01:00
Guillaume Lours fb36f7fffd directly embed the orignal APIClient in the DryRunClient
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-01-16 11:11:35 +01:00
Guillaume Lours eb59b0e265 add alpha command to test dry-run
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-01-16 11:11:35 +01:00
Guillaume Lours 5081ab0507 create custom CLI when dry-run mode active
update documentation

Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-01-16 11:11:35 +01:00
Guillaume Lours fbf845c5f8 add dry-run flag
Signed-off-by: Guillaume Lours <705411+glours@users.noreply.github.com>
2023-01-16 11:11:35 +01:00
Nicolas De Loof a0acc20d88
introduce --parallel to limit concurrent engine calls
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-12-06 08:15:50 +01:00
Nicolas De Loof 8f2b747104 use DistributionInspect to resolve image digest
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-11-30 21:18:18 +01:00
Nicolas De Loof 9ac4f69918 move image digests resolution to backend
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-11-30 21:18:18 +01:00
Milas Bowman b49b9ffe7e Merge remote-tracking branch 'upstream/v2' into down-image-rm 2022-09-13 18:00:41 +01:00
Lucas Berg 7a8d157871
convert: do not escape $ into $$ when using the --no-interpolate option (#9703)
Signed-off-by: Lucas Berg <root.lucasberg@gmail.com>
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
Co-authored-by: Ulysses Souza <ulyssessouza@gmail.com>
2022-09-08 16:25:23 -04:00
Milas Bowman bc806da712 build: label built images for reliable cleanup on `down`
When running `compose down`, the `--rmi` flag can be passed,
which currently supports two values:
 * `local`: remove any _implicitly-named_ images that Compose
            built
 * `all`  : remove any named images (locally-built or fetched
            from a remote repo)

Removing images in the `local` case can be problematic, as it's
historically been done via a fair amount of inference over the
Compose model. Additionally, when using the "project-model"
(by passing `--project-name` instead of using a Compose file),
we're even more limited: if no containers for the project are
running, there's nothing to derive state from to perform the
inference on.

As a first pass, we started labeling _containers_ with the name
of the locally-built image associated with it (if any) in #9715.
Unfortunately, this still suffers from the aforementioned problems
around using actual state (i.e. the containers might no longer
exist) and meant that when operating in file mode (the default),
things did not behave as expected: the label is not available
in the project since it only exists at runtime.

Now, with these changes, Compose will label any images it builds
with project metadata. Upon cleanup during `down`, the engine
image API is queried for related images and matched up with the
services for the project. As a fallback for images built with
prior versions of Compose, the previous approach is still taken.

See also:
 * https://github.com/docker/compose/issues/9655
 * https://github.com/docker/compose/pull/9715

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-09-07 17:57:29 -04:00
Ulysses Souza 55cf579e02 Fix `down` with `--rmi`
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2022-08-29 19:48:23 +02:00
Milas Bowman 1ee44a0acb
config: use correct YAML marshal func (#9712)
The `compose-spec/compose-go` lib is written with `gopkg.in/yaml.v2`
as a target.

When marshalling via CLI (`compose convert` / `compose config`), we
were using a _different_ YAML lib, which was a fork of `go-yaml`,
which is what `gopkg.in/yaml.v2` is based off of.

Signed-off-by: Milas Bowman <milas.bowman@docker.com>
2022-08-03 20:41:43 +00:00
Nicolas De Loof f6e96dd783 if command is ran with a compose file, apply the compose model, not just project name
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-08-02 22:33:26 +02:00
Guillaume Lours 150fd4b8cf
use '-' as separator by default for image name
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
2022-07-29 18:55:22 +02:00
Nicolas De Loof e7f545907b
don't remove external volumes/networks
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-03-14 15:24:49 +01:00
Nicolas De Loof 5e8040ea18
don't fail trying to remove container with no candidate
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-03-09 16:24:16 +01:00
Guillaume Lours 7036cda306
Merge pull request #9198 from ndeloof/dockerCli_stdout
composeService to use dockerCli's In/Out/Err streams
2022-03-09 14:36:07 +01:00
Nicolas De Loof 158b5ff6a3 build full compose model from resources, then filter by services
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-03-09 11:29:30 +01:00
Nicolas De Loof 22b8c731c7
prevent getCanonicalContainerName to crash
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-03-07 14:58:39 +01:00
Nicolas De Loof dbe7de50a7 don't try to start dependencies when there are none
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-03-07 08:25:05 +01:00
Nicolas De Loof f86f252a66 composeService to use dockerCli's In/Out/Err streams
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2022-03-03 16:34:57 +01:00
Mehrad Dadar 9c68c76bea minor improvement and fix
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-27 03:40:46 +03:30
Mehrad Dadar aeb7448449 minor improvement and fix
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-25 06:26:05 +03:30
Mehrad Dadar 42c3adb236 project existence check: added error message
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-25 05:36:22 +03:30
Mehrad Dadar 35f37cd1f7 fix lint
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-25 00:00:36 +03:30
Mehrad Dadar c0465616bb check service existence in project
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-24 23:46:07 +03:30
Mehrad Dadar 32d44dfc25 added scale to toProjectName
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-24 22:39:38 +03:30
Mehrad Dadar 67f7b84829 modified com.docker.compose.depends_on label to contain dependency type
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-23 21:34:34 +03:30
Mehrad Dadar ced9eba940 refactor: removed redundant code
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-17 07:55:11 +03:30
Mehrad Dadar 65ed8cf4c2 Implemented #9147
Signed-off-by: Mehrad Dadar <mehrad.dadar@gmail.com>
2022-02-05 10:27:52 +03:30
Ulysses Souza 17354fcc99 Add standalone capacility
Signed-off-by: Ulysses Souza <ulyssessouza@gmail.com>
2021-09-30 02:16:55 +02:00
Nicolas De Loof 99e75639d6 define compose container names with a valid hostname
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2021-09-22 19:06:34 +02:00
Nicolas De Loof abb2ddba88 `compose config` to espace `$` signs so compose file is still valid
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2021-09-17 10:32:52 +02:00
Nicolas De Loof 1ae9b3cb5d
move compose-cli code into docker/compose/v2
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2021-08-31 19:09:19 +02:00
Nicolas De Loof 49e7f2d45d Move compose v2 implementation under pkg/compose with dependencies
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2021-06-15 15:52:48 +02:00