kubernetes rbac rules (still) not enforced

[davidje13]

• I have tried with the latest version of my channel (Stable or Edge)
• I have uploaded Diagnostics

This is a refresh of #3694, as that issue was believed to be fixed but actually still exists in 2.3.0.3 (45519). The old issue has been locked due to age.

Run the following:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: test-sa

---

apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  serviceAccountName: test-sa
  containers:
  - image: alpine
    name: test-container
    command: [sh]
    args:
    - -ec
    - |
      apk add curl;
      KUBE_NAMESPACE="$(cat /var/run/secrets/kubernetes.io/serviceaccount/namespace)";
      curl \
        --cacert "/var/run/secrets/kubernetes.io/serviceaccount/ca.crt" \
        -H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
        "https://kubernetes.default.svc/api/v1/namespaces/$KUBE_NAMESPACE/services";
      while true; do sleep 1; done;

kubectl apply -f test.yml
kubectl logs test-pod

Expected behavior

Permissions error (403)

Actual behavior

API request succeeds

Information

kubectl get clusterrolebinding docker-for-desktop-binding -o yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: "2020-06-22T13:01:34Z"
  name: docker-for-desktop-binding
  resourceVersion: "478"
  selfLink: /apis/rbac.authorization.k8s.io/v1/clusterrolebindings/docker-for-desktop-binding
  uid: f4d43180-d35e-448c-ba41-db0a1330baff
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts
  namespace: kube-system

kubectl auth can-i --list --as system:serviceaccount

Resources                                       Non-Resource URLs   Resource Names   Verbs
*.*                                             []                  []               [*]
                                                [*]                 []               [*]
selfsubjectaccessreviews.authorization.k8s.io   []                  []               [create]
selfsubjectrulesreviews.authorization.k8s.io    []                  []               [create]
                                                [/api/*]            []               [get]
                                                [/api]              []               [get]
                                                [/apis/*]           []               [get]
                                                [/apis]             []               [get]
                                                [/healthz]          []               [get]
                                                [/healthz]          []               [get]
                                                [/livez]            []               [get]
                                                [/livez]            []               [get]
                                                [/openapi/*]        []               [get]
                                                [/openapi]          []               [get]
                                                [/readyz]           []               [get]
                                                [/readyz]           []               [get]
                                                [/version/]         []               [get]
                                                [/version/]         []               [get]
                                                [/version]          []               [get]
                                                [/version]          []               [get]

rnsv posted the necessary fix in the old issue here: #3694 (comment) (the namespace was applied to the wrong part of the spec)

Diagnostic logs

Docker for Mac: 2.3.0.3 (45519)

[davidje13]

In the meantime, users can patch this issue on their own deploys by running:

kubectl apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: docker-for-desktop-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:kube-system
EOF

(it will complain about "kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply" but it will work. I don't know if it will cause problems with docker-desktop upgrades in the future)

[sixeyed]

A patch might be a better workaround - it will fail if the binding doesn't exist:

 kubectl patch clusterrolebinding docker-for-desktop-binding --type=json --patch $'[{"op":"replace", "path":"/https/github.com/subjects/0/name", "value":"system:serviceaccounts:kube-system"}]'

• or in Powershell which needs the quotes to be escaped:

kubectl patch clusterrolebinding docker-for-desktop-binding --type=json --patch '[{\"op\":\"replace\", \"path\":\"/subjects/0/name\", \"value\":\"system:serviceaccounts:kube-system\"}]'

[sixeyed]

@nebuk89 this is quite an important issue, currently it gives all Service Accounts cluster admin role which pretty much defeats RBAC and makes Kube in Desktop (Windows and Mac) behave unlike every other distro :)

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[davidje13]

I have just tested this in 2.4.0.0 (48506) and the issue still exists, so I'm marking this not stale.

For users, the workaround posted by sixeyed seems to be the best option:

kubectl patch clusterrolebinding docker-for-desktop-binding --type=json --patch $'[{"op":"replace", "path":"/https/github.com/subjects/0/name", "value":"system:serviceaccounts:kube-system"}]'

Alternatively I've found that minikube is pretty easy to get going with and seems to create a system which is a bit closer to a "real" kubernetes environment.

/remove-lifecycle stale

[sixeyed]

Still exists in Desktop 3.0.0

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[sixeyed]

/remove-lifecycle stale

Still exists in 3.2.2

[docker-robott]

Issues go stale after 90 days of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30 days of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

[davidje13]

maybe it's worth pinging @guillaumerose on this one since it keeps going stale and he previously asked people to let him know if this issue continued after the original attempted fix.

/remove-lifecycle stale

[guillaumerose]

I am no longer working at Docker.
Maybe @djs55 can help you.

[cdlliuy]

the issue is still there in 3.4.0 version ..

[piyush-saurabh]

The issue is still in v4.0.1

[adfoster-r7]

Also hit this issue 👍

[davidje13]

I see pings on this every now and again but given how long it's been, I'm pretty sure it will never be fixed.

I'd strongly encourage anybody who wants a realistic Kubernetes test environment to switch to minikube like I did ages ago. It's easy to set up and gives you something which is closer to a real deploy than the setup you get via docker-for-mac (fixing more than just this one issue). As far as I've been able to tell, there's no disadvantage to switching.

[ncresswell]

just hit this same issue and was pulling my hair out trying to see why... any update Docker ?

[mikesir87]

This "feature" does make teaching others about RBAC in k8s a little interesting for sure! 😆 While it's easy enough to nuke the binding, I'd love to hear why it's there in the first place. Should it be scoped to a specific namespace?

[jpetazzo]

Pinging @thaJeztah - maybe he won't know what to do, but he will know who knows ♥

(I need to refresh my look-up table of Docker Inc. folks so that I know better who to ping for that kind of issue 😅)

Short recap:

• Docker Desktop has a ClusterRoleBinding called docker-for-desktop-binding that gives cluster-admin privileges to all ServiceAccounts
• this means that any Pod running on Docker Desktop has cluster-admin privileges
• it should probably be changed to give these privileges only to the ServiceAccount kube-system:default (or even removed completely)

This behavior is present at least until Docker Desktop 4.1.1 (69879).

[mstaicu]

Slightly off-topic but as a first layer of restrictions you could opt out of automounting API credentials for a particular pod via automountServiceAccountToken directive ( https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ ), especially if your pods don't need to communicate with the control plane

[chrisdoherty4]

This is painful, please fix it.

[jpetazzo]

@chrisdoherty4 Hi! Can you delete the clusterrolebinding and report if there is any side effect? That would help us all to figure out what the best solution would be. Thank you!

[chrisdoherty4]

@jpetazzo Done and it performs well. I can correctly define RBAC and kubectl auth can-i spits out the expected result (I ran various tests mixing and matching stuff).

I didn't explicitly try ClusterRole's but I assume it functions fine also.

[djs55]

I believe the binding should not be installed any more in development builds. If you have time to check, here is the latest one: https://desktop-stage.docker.com/mac/main/amd64/71478/Docker.dmg (replace amd64 with arm64 if you are using an M1). I don't think it will remove a binding that already exists-- it will probably only take effect when the cluster is next initialised.

[thaJeztah]

Pinging @thaJeztah - maybe he won't know what to do, but he will know who knows ♥

Oh! Sorry, looks like I didn't reply on this thread. I see it's being worked on though, so 👍 ❤️

[chrisdoherty4]

@djs55 Awesome work. Thanks!

[trungutt]

New release 4.3.0 (https://docs.docker.com/desktop/mac/release-notes/) is out and has a fix for this issue

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked