Commit Graph

158 Commits

Author SHA1 Message Date
Nicolas De Loof cb74f7924e
Don't define service resource name
if we do, CloudFormation can't update resource and changeset fail with
"CloudFormation cannot update a stack when a custom-named resource
requires replacing"

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:18 +02:00
Nicolas De Loof ed262a0461
Generate mock inside a container
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:17 +02:00
Nicolas De Loof c5895fe09a
Use `Project` from compose-go
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:16 +02:00
Nicolas De Loof dcf84f2499
Fix broken build on master
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:15 +02:00
aiordache 1bb95134f0
match docker/api signature for up and down methods
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:15 +02:00
aiordache 9e8ddb63cc
fix lint issue - renamed CompatibilityChecker to Checker
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:15 +02:00
aiordache bb98dae082
code restructure
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:14 +02:00
Nicolas De Loof d36b9b104e
Fix broken master
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:14 +02:00
Nicolas De Loof 0f1a362664
Set FailureThreshold
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:13 +02:00
Nicolas De Loof d957987471
Unit tests for cobra commands
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:13 +02:00
aiordache d597e55f22
fix rebase
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:10 +02:00
aiordache 2c190f11f7
LB Type tests
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:10 +02:00
Nicolas De Loof 1d11e847fb
Test we create the expected policy document for pull_credentials
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:09 +02:00
Nicolas De Loof e88b11bc26
Introduce test to check CloudFormation conversion
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:09 +02:00
aiordache c0f1a8bf18
create different methods to get lb type and security groups
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:08 +02:00
aiordache 45dc8eda80
update test data
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:08 +02:00
aiordache c04950cdac
improve lb security groups parsing
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:08 +02:00
aiordache dad36e09f9
set ALB and security groups for http(s) protocol
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:07 +02:00
aiordache 7337c7520f
rename LB field in the compose file
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:07 +02:00
aiordache 02cc644c5a
fix test data
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:06 +02:00
aiordache 2ea694a1c5
update test data
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:06 +02:00
aiordache 3194cc9b16
allow user defined LB
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:06 +02:00
Nicolas De Loof fbb5bdac6e
Fix resource naming
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:05 +02:00
Nicolas De Loof 37177e6d7a
Split long `Convert` func into smaller, focussed sub-func
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:05 +02:00
aiordache f71109be9e
update testdata
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:04 +02:00
aiordache eddaa70a9e
create NLB load balancer only
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:04 +02:00
aiordache 335806a179
create only one global load balancer - error out if exports port require different types
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:03 +02:00
aiordache e7f77ca3ef
add all service security groups to LB
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:03 +02:00
aiordache 92173eaf35
add SO link for issue if listener is not in service dependencies
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:02 +02:00
aiordache ae3101fe12
create unique load balancer per app and cleanup
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:02 +02:00
aiordache fc9b10fc91
add load balancer
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:02 +02:00
Nicolas De Loof b702065075
custom extension to select existing VPC and SecurityGroups
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:01 +02:00
Nicolas De Loof 1bf4bc9d46
Use distinct family per service definition
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:01 +02:00
Nicolas De Loof 7d4222a725
Implement depends_on using CloudFormation
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:00 +02:00
Nicolas De Loof ff88290302
Make `ps` order predictable
so one can run `watch docker ecs compose ps`

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:00 +02:00
Nicolas De Loof 5080a83242
prevent "Tasks cannot be empty" error
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:26:00 +02:00
Nicolas De Loof 5783b63556
Service can freely communicate within a network
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:59 +02:00
Nicolas De Loof 6c57fb9693
support deploy.replicas
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:59 +02:00
Nicolas De Loof be1c65d441
Get more from DescribeTask
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:59 +02:00
Nicolas De Loof da299f59e2
introduce 'ps' command
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:58 +02:00
Nicolas De Loof 564c369c3e
Compute resource names to avoid unsupported characters
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:58 +02:00
Nicolas De Loof 257f829679
Create service with project and service tags
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:57 +02:00
Nicolas De Loof 01e2b0c989
Present service logs with colored service prefix
This reproduce docker-compose behaviour to report logs with prefix
also moves log formating out from sdk.go

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:57 +02:00
Nicolas De Loof a798c95963
Register services with a known port with SRV record
see
https://github.com/docker/docker_aws/issues/15#issuecomment-634357859

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:56 +02:00
Nicolas De Loof 3283bceac6
Support pull from ECR
close #58

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:56 +02:00
Guillaume Lours e9fe3b2864
Add e2e test deploying a compose application to an ECS cluster
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:55 +02:00
Nicolas De Loof 08bd18231d
Introduce `Normalize` and `Check` in compose model lifecycle
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:55 +02:00
Nicolas De Loof a5a925173c
SDK methods to query service tasks and retrieve public IP
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:55 +02:00
Guillaume Lours 6798ad1245
Add security group declaration in cloudformation conversion tests
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:54 +02:00
Nicolas De Loof ae4dc2e0db
Reject compose file with unsupported features
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:54 +02:00
Nicolas De Loof 8cd4a6fe9b
Fix golden files after rebase
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:54 +02:00
Guillaume Lours 07a57469db
Add unit tests version of migration tests instead of e2e one
Signed-off-by: Guillaume Lours <guillaume.lours@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:53 +02:00
aiordache 3e5b118f26
add GetLogs to MockAPI
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:53 +02:00
aiordache 0492dacfee
remove redundant var init
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:52 +02:00
aiordache 9dbff1eb72
add logs command
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:52 +02:00
Nicolas De Loof 43d3d94c43
Create cluster by compose up
close #53

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:51 +02:00
Nicolas De Loof 1fdac494f3
Create CloudFormation template with parameters
so we don't need AWS API to resolve IDs and can run conversion offline

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:51 +02:00
Nicolas De Loof 4ab37f8229
Implement Hostname-only service discovery using LOCALDOMAIN
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:50 +02:00
aiordache 3a678fd7dc
cleanup
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:49 +02:00
aiordache d09c8c7236
add private images support
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:49 +02:00
aiordache 57d7474f7d
set secrets in cloudformation template
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:48 +02:00
aiordache f95bd4fdbf
mapping cpu and memory to fargate values
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:48 +02:00
Nicolas De Loof 1889d04d83
Implement "network" using SecurityGroups
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:47 +02:00
Nicolas De Loof 09871400ef
Register services within Cloud Map
close #35

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:46 +02:00
Nicolas De Loof 0eab586106
Create CloudMap private namespace and register services
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:46 +02:00
Nicolas De Loof 9a6fe86a86
Introduce "Validate" phase to check/make app ECS-compliant
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:45 +02:00
Nicolas De Loof 2544307f55
drop GetEcsTaskExecutionRole which is not in used anymore
We need to define a way for compose-user to declare additional Policies
to be added to TaskExecutionRole

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:45 +02:00
Nicolas De Loof 3e30f2cd1a
Create CloudWatch LogGroup and IAM TaskExecutionRole
As part of the CloudFormation template, create a LogGroup and configure
task with awslogs log-driver. Also create a dedicated IAM Role, with
AmazonECSTaskExecutionRolePolicy. This one will later be fine-tuned to
grant access to secrets/config and other AWS resources according to
custom extensions

close #42

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:45 +02:00
Nicolas De Loof 99b4ed0bfd
Capture first failure to report root error to user on command completion
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:44 +02:00
Nicolas De Loof a8e963a304
Query stack events by stack ID (not name)
This prevent a race condition on `down` as stack is deleted and we still
ask for stack events as we didn't recieved the DELETE_COMPLETE one

Use WaitUntilStack* to detect stack operation completion

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:44 +02:00
Nicolas De Loof 814259ae33
Also wait for deletion events
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:43 +02:00
Nicolas De Loof 5d61fc119a
Format stack events as a set of resources with progress status
This mimic how docker-compose report containers creation

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:43 +02:00
Nicolas De Loof 678f4018f0
Collect events while waiting for stack to complete
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:42 +02:00
aiordache 41aaf802e3
implement secret management
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:42 +02:00
aiordache 2ad9504d15
add secret interface
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:42 +02:00
Nicolas De Loof de365f41e9
Fix test by regenerating mock
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:41 +02:00
aiordache 4a6fec63d2
yet another rebase
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:41 +02:00
aiordache cec44fbb7b
move to sdk
Signed-off-by: aiordache <anca.iordache@docker.com>
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:41 +02:00
Nicolas De Loof b6be4a0ac3
Use `WithContext` SDK APIs so we can implement cancelation
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:40 +02:00
Nicolas De Loof 4642bfa172
Fix linter warnings
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:39 +02:00
Nicolas De Loof 30029fa701
ComposeDown only require stack name
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>

g Sur la branche api

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:39 +02:00
Nicolas De Loof 541bda3af8
Remove ALB related options to be defined on phase 2
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:38 +02:00
Nicolas De Loof 096c800c1b
use go:generate to automatically run mockgen on build
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-08-17 21:25:38 +02:00
Nicolas De Loof 4138dcfb5a Split API interface by required SDK func per command
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-28 10:59:00 +02:00
Nicolas De Loof 3d8d982d4a format
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-28 10:59:00 +02:00
Nicolas De Loof 52c6177ff7 API mock and a test case relying on it
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-28 10:59:00 +02:00
Nicolas De Loof 8c0fee5abf Define amazon.API as a simplified and currated interface over AWS SDK
This makes code simpler to read and easier to mock within tests

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-28 10:59:00 +02:00
Nicolas De Loof 52440a4732 Setup Github Action for CI
close #1

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-28 10:50:27 +02:00
aiordache d612a4ab89 Project name parameter as alternative to compose file on down
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-04-27 09:21:58 +02:00
Nicolas De Loof f8bf0078aa Use DescribeCluster as ListCluster is a Paginated API
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-24 10:13:38 +02:00
Nicolas De Loof ea6d35a927 Fix minor issue after merge conflit resolution
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-23 16:19:47 +02:00
Nicolas De Loof a1c91ac4c2 Merge branch 'cluster-create' of github.com:aiordache/ecs-plugin into aiordache-cluster-create
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-23 14:55:41 +02:00
aiordache 3d7e062215 add delete-cluster flag on down cmd
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-04-23 14:50:18 +02:00
Nicolas De Loof 48096eeed8 DescribeStacks fail with error if stack does not exists
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-23 11:19:59 +02:00
aiordache 55f2908c16 wait for stack removal on cluster delete
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-04-22 16:49:55 +02:00
Nicolas De Loof 87f053d710 Detect stack already exists
This will later be used to switch to ChangeSet logic

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-22 15:37:39 +02:00
aiordache 30fd37b6ca ecs cluster create
Signed-off-by: aiordache <anca.iordache@docker.com>
2020-04-22 15:06:01 +02:00
Nicolas De Loof 0972776e6d Ingress description to include service being exposed
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-21 14:48:51 +02:00
Nicolas De Loof b70f01d2f4 Adopt CloudFormation to create ECS app from compose.yaml
Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-21 11:38:52 +02:00
Nicolas De Loof 4e72d1892a Prefer AWS API interface over actual implementation
This will help introduce mock-based tests

Signed-off-by: Nicolas De Loof <nicolas.deloof@gmail.com>
2020-04-20 13:47:38 +02:00