[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Pods randomly running as root



Judging by pod start times, it looks like everything that started before February 2 is not running as root, while everything else is.

On Mon, Feb 6, 2017 at 12:57 PM, Alex Wauck <alexwauck exosite com> wrote:
OK, this just got a lot more interesting:

$ oc -n some-project exec app1-45-3blnd -- id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video),1000370000
$ oc -n some-project exec app2-18-q2fwm -- id
uid=1000370000 gid=0(root) groups=1000370000
$ oc -n some-project exec app3-10-lhato -- id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video),1000370000
$ oc -n some-project exec app4-16-dl2r7 -- id
uid=1000370000 gid=0(root) groups=1000370000
$ oc -n some-project exec app5-36-2rfsq -- id
uid=0(root) gid=0(root) groups=0(root),1000370000
$ oc -n some-project exec app6-15-078fd -- id
uid=0(root) gid=0(root) groups=0(root),1000370000

All of these pods are running on the same node, and as you can see, they are in the same project.  Yet, some are running as root and some are not.  How weird is that?

On Mon, Feb 6, 2017 at 12:49 PM, Alex Wauck <alexwauck exosite com> wrote:
$ oc export -n some-project pod/good-pod | grep serviceAccount
  serviceAccount: default
  serviceAccountName: default
$ oc export -n some-project pod/bad-pod | grep serviceAccount
  serviceAccount: default
  serviceAccountName: default

Same serviceAccountName.  This problem seems to happen with any pod from any project that happens to run on these newer nodes.  I examined the output of `oc describe scc`, and I did not find any unexpected access to elevated privileges for a default serviceaccount.  The project were I'm currently seeing the problem is not mentioned at all.  Also, I've seen the problem happen with pods that are managed by the same replication controller.

On Mon, Feb 6, 2017 at 12:46 PM, Clayton Coleman <ccoleman redhat com> wrote:
Adding the list back

---------- Forwarded message ----------
From: Clayton Coleman <ccoleman redhat com>
Date: Mon, Feb 6, 2017 at 1:42 PM
Subject: Re: Pods randomly running as root
To: Alex Wauck <alexwauck exosite com>
Cc: users <users redhat com>


Do the pods running as root and not have the same serviceAccountName field or different ones?  IF different, you may have granted the service account access to a higher role - defaulting is determined by the SCC's that a service account can access, so an admin level service account will run as root by default unless you specify you don't want that.

On Mon, Feb 6, 2017 at 1:37 PM, Alex Wauck <alexwauck exosite com> wrote:
I'm looking at two nodes where one has the problem and the other doesn't, and I have confirmed that their node-config.yaml is the same for both (modulo IP addresses).  The generated kubeconfigs for these nodes on the master are also the same (modulo IP addresses and keys/certs).

On Mon, Feb 6, 2017 at 10:46 AM, Alex Wauck <alexwauck exosite com> wrote:
Oh, wait.  I was looking at the wrong section.  The non-root pod as a runAsUser attribute, but the root pod doesn't!

On Mon, Feb 6, 2017 at 10:44 AM, Alex Wauck <alexwauck exosite com> wrote:
A pod that IS running as root have this:

  securityContext:
    fsGroup: 1000370000
    seLinuxOptions:
      level: s0:c19,c14

Another pod in the same project that is NOT running as root has the exact same securityContext section.

On Mon, Feb 6, 2017 at 10:25 AM, Clayton Coleman <ccoleman redhat com> wrote:
Do the pods themselves have a user UID set on them?  Each pod should have the container "securityContext" field set and have an explicit user ID value set.

On Mon, Feb 6, 2017 at 11:23 AM, Alex Wauck <alexwauck exosite com> wrote:
These are completely normal app containers.  They are managed by deploy configs.  Whether they run as root or not seems to depend on which node they run on: the older nodes seem to run pods as random UIDs, while the newer ones run as root.  Our older nodes have docker-selinux-1.10.3 installed, while the newer ones do not.  They only have docker-selinux-1.9.1 available, since the 1.10.3 package seems to have been removed from the CentOS extras repo.

We are running OpenShift 1.2.1, since I haven't had time to upgrade it.

On Mon, Feb 6, 2017 at 8:31 AM, Clayton Coleman <ccoleman redhat com> wrote:
Are you running them directly (launching a pod)?  Or running them under another controller resource.

On Feb 6, 2017, at 2:00 AM, Alex Wauck <alexwauck exosite com> wrote:

Recently, I began to notice that some of my pods on OpenShift run as root instead of a random UID.  There does not seem to be any obvious cause (e.g. SCC).  Any idea how this could happen or where to look for clues?





--
Alex Wauck // DevOps Engineer

E X O S I T E 
www.exosite.com 

Making Machines More Human.



--
Alex Wauck // DevOps Engineer

E X O S I T E 
www.exosite.com 

Making Machines More Human.



--
Alex Wauck // DevOps Engineer

E X O S I T E 
www.exosite.com 

Making Machines More Human.

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]