Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

run links containers that are not dependencies #9459

Closed
michaeldiscala opened this issue May 12, 2022 · 8 comments · Fixed by #9558
Closed

run links containers that are not dependencies #9459

michaeldiscala opened this issue May 12, 2022 · 8 comments · Fixed by #9558
Assignees
@ndeloof

Comments

@michaeldiscala
Copy link

Description
Hi there! Starting in compose version 2.3.0, the run command is bringing up linked containers that are not strict dependencies of the service that is being started. This behavior only happens if I have previously used up with one of the unrelated services.

I don't see anything that's jumping out to me in the changelog that would cause this & I don't see any updates to the docs suggesting that the behavior of these has changed, which makes me think this is a bug. Is this new behavior intentional though?

Steps to reproduce the issue:
I can demonstrate this issue with a toy compose definition:

version: "3.6"

services:
  service_a:
    image: bash
    command: echo "a"
    depends_on:
      - shared_dep

  service_b:
    image: bash
    command: echo "b"
    depends_on:
      - shared_dep

  shared_dep:
    image: bash

Describe the results you received:

If I just execute run service_a, compose will start the shared_dep container and the service_a container as expected:

❯ docker-compose  run service_a
[+] Running 2/2
 ⠿ Network compose-bug-test_default         Created                                                                                                                                                               
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
[+] Running 1/1
 ⠿ Container compose-bug-test-shared_dep-1  Started                                                                                                                                                               
a

However, an docker version 2.3.0 and above if I (1) execute up service_b and then (2) execute run service_a, I see the unrelated service_b also get launched:

~panorama/compose-bug-test
❯ docker-compose  up service_b
[+] Running 3/2
 ⠿ Network compose-bug-test_default         Created                                                                                                                                                               
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
 ⠿ Container compose-bug-test-service_b-1   Created                                                                                                                                                               
Attaching to compose-bug-test-service_b-1
compose-bug-test-service_b-1  | b
compose-bug-test-service_b-1 exited with code 0

~panorama/compose-bug-test
❯ docker-compose run service_a
[+] Running 1/0
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
[+] Running 2/2
 ⠿ Container compose-bug-test-shared_dep-1  Started                                                                                                                                                               
 ⠿ Container compose-bug-test-service_b-1   Started                                                                                                                                                               
a

Describe the results you expected:
I would not expect run service_a to ever launch the service_b container, since they have no dependencies against one another

Additional information you deem important (e.g. issue happens only occasionally):

I have confirmed that this behavior was not present in compose 2.2.3:

❯ [email protected] up service_b
[+] Running 3/3
 ⠿ Network compose-bug-test_default         Created                                                                                                                                                               
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
 ⠿ Container compose-bug-test-service_b-1   Created                                                                                                                                                               
Attaching to compose-bug-test-service_b-1
compose-bug-test-service_b-1  | b
compose-bug-test-service_b-1 exited with code 0

~panorama/compose-bug-test
❯ [email protected] run service_a
[+] Running 1/0
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
[+] Running 1/1
 ⠿ Container compose-bug-test-shared_dep-1  Started                                                                                                                                                              
a

I do see the bug in the 2.3.0 release however:

❯ [email protected] up service_b
[+] Running 3/2
 ⠿ Network compose-bug-test_default         Created                                                                                                                                                               
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
 ⠿ Container compose-bug-test-service_b-1   Created                                                                                                                                                               
Attaching to compose-bug-test-service_b-1
compose-bug-test-service_b-1  | b
compose-bug-test-service_b-1 exited with code 0

~panorama/compose-bug-test
❯ [email protected] run service_a
[+] Running 1/0
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                               
[+] Running 2/2
 ⠿ Container compose-bug-test-shared_dep-1  Started                                                                                                                                                               
 ⠿ Container compose-bug-test-service_b-1   Started                                                                                                                                                               
a

Output of docker compose version:

❯ docker-compose --version
Docker Compose version 2.5.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 75
  Running: 18
  Paused: 0
  Stopped: 57
 Images: 434
 Server Version: 19.03.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.130-boot2docker
 Operating System: Boot2Docker 19.03.12 (TCL 10.1)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 7.789GiB
 Name: dinghy
 ID: CN3C:XYJB:SR6V:ISNK:5CRG:J4QB:KVQO:EK44:YHJV:U2P6:K2NZ:CSZT
 Docker Root Dir: /mnt/sda1/var/lib/docker
 Debug Mode: false
 Username: mdiscalapanorama
 Registry: https://index.docker.io/v1/
 Labels:
  provider=virtualbox
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Thanks so much for any help that ya'll can offer!
@michaeldiscala
Copy link
Author

michaeldiscala commented May 12, 2022 •
edited
Loading

I ran git bisect on the code to identify when this behavior was implemented and I believe that this merge commit introduced the issue 9d73cc8

I'm able to produce the issue on this branch but not on the parent it was originally merged into & also skimming the code, it does seem related to dependency management.

Unfortunately I'm not familiar enough with docker-compose & go to make much more progress on this.

@arhemd, I think you were the original author of this patch. Does this change in behavior seem familiar to you at all? Thank you!

@michaeldiscala michaeldiscala mentioned this issue May 13, 2022
Closed
@ms-ati
Copy link

ms-ati commented May 16, 2022

Is anyone able to look at this?

@ndeloof
Copy link
Contributor

ndeloof commented May 17, 2022

This issue should be fixed with latest release, by db69856

Please give 2.5.0 a try to confirm

@michaeldiscala
Copy link
Author

michaeldiscala commented May 17, 2022 •
edited
Loading

Hi @ndeloof! Thanks for taking a look at this - I am still able to reproduce this issue in 2.5.0 using the compose file shown above & the commands there:

~/panorama/compose-bug-test
❯ dc down
[+] Running 4/0
 ⠿ Container compose-bug-test-service_b-1                 Removed                                                                                                                                                    0.0s
 ⠿ Container compose-bug-test_service_a_run_97f1ebf7e816  Removed                                                                                                                                                    0.0s
 ⠿ Container compose-bug-test-shared_dep-1                Removed                                                                                                                                                    0.0s
 ⠿ Network compose-bug-test_default                       Removed                                                                                                                                                    0.0s

~/panorama/compose-bug-test
❯ dc up service_b
[+] Running 3/3
 ⠿ Network compose-bug-test_default         Created                                                                                                                                                                  0.1s
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                                  0.1s
 ⠿ Container compose-bug-test-service_b-1   Created                                                                                                                                                                  0.1s
Attaching to compose-bug-test-service_b-1
compose-bug-test-service_b-1  | b
compose-bug-test-service_b-1 exited with code 0

~/panorama/compose-bug-test
❯ dc run service_a
[+] Running 1/0
 ⠿ Container compose-bug-test-shared_dep-1  Created                                                                                                                                                                  0.0s
[+] Running 2/2
 ⠿ Container compose-bug-test-shared_dep-1  Started                                                                                                                                                                  0.3s
 ⠿ Container compose-bug-test-service_b-1   Started                                                                                                                                                                  0.8s
a

~/panorama/compose-bug-test
❯ dc --version
Docker Compose version 2.5.0

When I do dc run service_a in this trace I'm still seeing ⠿ Container compose-bug-test-service_b-1 Started which is not implied by the dependencies in the compose file. Note that this only happens if I first do a dc up service_b

@ndeloof
Copy link
Contributor

ndeloof commented May 17, 2022

ok thanks, I'll test this tomorrow

@qhenkart qhenkart mentioned this issue May 24, 2022
Closed
2 tasks
@qhenkart
Copy link

Thanks to Michael's excellent detective work, I was able to resolve this by removing every dependency in our docker-compose file, which is obviously not ideal and a very temporary workaround. The rest of our team is locked into Docker for mac v4.5 which bundles the older version of docker-compose before this bug was introduced. Our kryptonite was a shared dependency to mongo and redis which went up the dependency tree and exploded into all of the services listed in our docker compose starting at once and crashing

mongo-client:
    image: mongo:5.0.2
    stdin_open: true
    tty: true
    command: mongosh mongodb://mongo:27017/db
    depends_on:
      - mongo
mongo:
    image: mongo:5.0.2
    command: mongod --port 27017 
    expose:
      - "27017"
    ports:
      - "27017:27017"

  service:
    image: golang
    expose:
      - "3000"
    ports:
      - "3000:3000"
    depends_on:
      - mongo

@ndeloof ndeloof self-assigned this May 30, 2022
@michaeldiscala
Copy link
Author

Hi @ndeloof! I noticed that v2.6.0 was just released and so I reran my tests and confirmed that this issue is still present in that version.

Would an option be to revert the patch that seems to have introduced this behavior (9d73cc8)? Not sure if a revert would be easy to apply at this point or if it's okay to remove behavior like that. I could attempt to apply this and open up a PR if you think that's appropriate.

@michaeldiscala michaeldiscala mentioned this issue Jun 13, 2022
Closed
@laurazard laurazard mentioned this issue Jun 13, 2022
Merged
@michaeldiscala
Copy link
Author

Thanks @ndeloof and @laurazard for the work to fix this! It will make such a big impact for our teams 🎉

Do you have a sense of when a new version will be release that includes this great change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ndeloof ndeloof

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Only start direct dependencies of service on compose run ... laurazard/compose
4 participants
@ndeloof @ms-ati @michaeldiscala @qhenkart