Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Container identifiers should be globally unique #199

Closed
lavalamp opened this issue Jun 21, 2014 · 46 comments · Fixed by #334
Closed

Container identifiers should be globally unique #199

lavalamp opened this issue Jun 21, 2014 · 46 comments · Fixed by #334

Comments

@lavalamp
Copy link
Member

A temporary workaround is to seed our random number generators. But really, apiserver should assign a guaranteed-unique identifier upon resource creation.

@thockin
Copy link
Member

thockin commented Jun 24, 2014

Internally we use RFC4122 UUIDs for identifying pods. Any objections to making this part of the pod setup? I guess it would really be a string (like "id") but with the strong suggestion that it be an encoded UUID.

Or we could use docker-style 256 bit randoms, but that might get confusing.

If we further lock down container names to RFC1035 labels, we can use . as the docker container name, which seems much nicer than the current dashes and underscores :)

What think?

@jjhuff
Copy link
Contributor

jjhuff commented Jun 24, 2014

Both points sound great to me. These encoded names are just plain ugly:)

Id is also a required field (network containers break otherwise), so it seems weird to have them marked as 'omitempty'. This is probably more an issue for config files than anything else.

@smarterclayton
Copy link
Contributor

Was going to get familiar and try to fix this - sounds like the suggestion is to add "ID string" to Container, and fail PodRegistryStorage.Create() if Container.ID is empty? Or should PodRegistryStorage.Create() populate unset DesiredState.Manifest.Containers[].ID that are empty? Latter seems more flexible (server controls default UUID generation for clients)

@thockin
Copy link
Member

thockin commented Jun 26, 2014

I started in on some validation logic this morning.

It's not clear whether unique ID is something users should have to spec or
not. I lean towards not. That means the master (api server?) Has to
generate a uuid upon acceptance. That uuid would have to flow down to
Kubelet.

Is that in the same vein as you were thinking?
On Jun 25, 2014 6:35 PM, "Clayton Coleman" notifications@github.com wrote:

Was going to get familiar and try to fix this - sounds like the suggestion
is to add "ID string" to Container, and fail PodRegistryStorage.Create() if
Container.ID is empty? Or should PodRegistryStorage.Create() populate unset
DesiredState.Manifest.Containers[].ID that are empty?

Reply to this email directly or view it on GitHub
#199 (comment)
.

@smarterclayton
Copy link
Contributor

As an API consumer I like not having to specify things that the server can do for me - having to generate a UUID on the command line to curl a new pod into existence feels wrong. I started here but didn't pass down to kubelet yet.

@vmarmol
Copy link
Contributor

vmarmol commented Jun 26, 2014

One thing to note with global unique identifier is static containers. Today
we setup cAdvisor as a static container and have no way to assign it a
unique ID globally.

On Wed, Jun 25, 2014 at 7:31 PM, Clayton Coleman notifications@github.com
wrote:

As an API consumer I like not having to specify things that the server can
do for me - having to generate a UUID on the command line to curl a new pod
into existence feels wrong. I started here
smarterclayton@984334f#diff-89731e89e31105da32e30576a5939fb6R149
but didn't pass down to kubelet yet.


Reply to this email directly or view it on GitHub
#199 (comment)
.

@smarterclayton
Copy link
Contributor

Static as in "defined on each host via a config file"? Would it make sense for the Kubelet to auto assign a UUID for containers pulled from files based on the host MAC and the position in the file (or a SHA1 of the contents of the manifest plus the host MAC)?

@thockin
Copy link
Member

thockin commented Jun 26, 2014

Are you setting it up by config file?

Can we generate a new uuid when we write the config file, or do we want it
to be identical across machines (google prod style)?

We cod do something like: if Kubelet finds a config without a uuid, it will
assign it a uuid and log it.

Internally we go one step further and define "master space" where each
configuration originator can choose an ID and then manage ids within that
space, meaning the master doesn't have to generate a UUID at all just a
unique masterspace + pod name.

Do we need to go that far? Uuids are notoriously human-hostile.
On Jun 25, 2014 7:38 PM, "Victor Marmol" notifications@github.com wrote:

One thing to note with global unique identifier is static containers.
Today
we setup cAdvisor as a static container and have no way to assign it a
unique ID globally.

On Wed, Jun 25, 2014 at 7:31 PM, Clayton Coleman notifications@github.com

wrote:

As an API consumer I like not having to specify things that the server
can
do for me - having to generate a UUID on the command line to curl a new
pod
into existence feels wrong. I started here
<
smarterclayton@984334f#diff-89731e89e31105da32e30576a5939fb6R149%3E

but didn't pass down to kubelet yet.

Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/199#issuecomment-47181803>

.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@vmarmol
Copy link
Contributor

vmarmol commented Jun 26, 2014

Yes, it is a static file we leave on the machine. I don't think we have the
ability to assign it a unique ID (outside of running some custom init
script). Having the Kubelet assign it a UUID seems reasonable to me.

On Wed, Jun 25, 2014 at 7:43 PM, Clayton Coleman notifications@github.com
wrote:

Static as in "defined on each host via a config file"? Would it make sense
for the Kubelet to auto assign a UUID for containers pulled from files
based on the host MAC and the position in the file (or a SHA1 of the
contents of the manifest plus the host MAC)?


Reply to this email directly or view it on GitHub
#199 (comment)
.

@thockin
Copy link
Member

thockin commented Jun 26, 2014

To be clearer on masterspace. Each new pod gets fields such as:

Masterspace: kubernetes.google.com
Name: "a name unique within this masterspace". I am sultaneously trying to
argue that new should be DNS compatible, which lits them to 64 lowercase
alphanums, and gives names semantic meaning. Making then bad as unique
IDs. Need to think more about this...
On Jun 25, 2014 7:46 PM, "Tim Hockin" thockin@google.com wrote:

Are you setting it up by config file?

Can we generate a new uuid when we write the config file, or do we want it
to be identical across machines (google prod style)?

We cod do something like: if Kubelet finds a config without a uuid, it
will assign it a uuid and log it.

Internally we go one step further and define "master space" where each
configuration originator can choose an ID and then manage ids within that
space, meaning the master doesn't have to generate a UUID at all just a
unique masterspace + pod name.

Do we need to go that far? Uuids are notoriously human-hostile.
On Jun 25, 2014 7:38 PM, "Victor Marmol" notifications@github.com wrote:

One thing to note with global unique identifier is static containers.
Today
we setup cAdvisor as a static container and have no way to assign it a
unique ID globally.

On Wed, Jun 25, 2014 at 7:31 PM, Clayton Coleman <
notifications@github.com>
wrote:

As an API consumer I like not having to specify things that the server
can
do for me - having to generate a UUID on the command line to curl a new
pod
into existence feels wrong. I started here
<
smarterclayton@984334f#diff-89731e89e31105da32e30576a5939fb6R149%3E

but didn't pass down to kubelet yet.

Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/199#issuecomment-47181803>

.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@lavalamp
Copy link
Member Author

I want the container names from the different sources to have different namespaces, so there won't be any collisions. E.g., kubelet prepends/appends ".etcd" (or something) to etcd-sourced containers, ".cfg" to containers from the config file, ".http" to containers from the manifest url, etc. Then it is up to each source to stay unique.

Api server can stay unique via uuid or counting up. Config files & manifest url stay unique by humans not screwing up; kubelet rejects them otherwise.

This has the nice effect that the container name produced from a config file is predictable without having to do a lookup. This would be good for our own container vm image and anything like it.

@thockin
Copy link
Member

thockin commented Jun 26, 2014

This started structured and turned into a stream0-of-consiousness, sorry.

I agree that unique names are desirable, but I'm not sure "etcd" and
"httpd" are sufficient.

The reason we have masterspace internally is because we also need to attach
metadata about the master that created a pod (e.g. which cluster it thought
it was in, what magic number it had (generation number) and so on).

Here's what I think I have convinced myself of, so far.

  1. All pods have a required name string, which is human-friendly and DNS
    friendly (probably rfc1035 subdomain rather than label).

  2. All pods which are running have a uuid (suggest RFC4122 but could be
    something else). This of this as a cluster-wide PID. It has no semantic
    meaning and changes whenever a pod is started on a new host.

  3. The kubelet API includes this uuid. If the uuid field is not specified,
    the kubelet will assign a new UUID.

This does not have the property that Daniel wants - predictable container
names. The problem is that you putting semantic meaning into the unique
ID. There's a reason that databse best practices involve surrogate keys
and that UNIX syscalls operate on PIDs rather than command names. Consider
what happens if we get a phantom instance on a split network - both pods
end up with the same name - not unique any more.

That said, I could maybe be convinced. If we put the rule that the pod
name had to be unique within a masterspace, we could sort of punt the
problem a bit, for now.

E.g.

Pod {
masterspace = "k8s.mydomain.com"
id = "id8675309.tims-pod"
containers [
{
name = "apache"

...would be created with container name
apache.id8675309.tims-pod.k8s.mydomain.com

If the apiserver did not care about phantoms, it could leave off the
id8675309 noise. for Google people, this should look very familiar.

I still have a vague foreboding about making the uniqueness be the master's
problem, but it would get rid of the need for opaque UUIDs. Or would it?
We need to handle static pods (e.g. cAdvisor). If they are allowed to
collide, then some hyopthetical cluster-data aggregator can not use ID as
primary key, and we will need to disambiguate queries by hostname. Blech.

What if the rule is that masterspace is optional. If not specified,
kubelet will make something up derived from the source. So cAdvsor would
say:

Pod {
#no masterspace
id = "cadvisor"
containers [
{
name = "cadvisor"

...would be created with container name cadvisor.cadvisor.file.<host_fqdn>

This still sort of sucks in that you can't aggregate by masterspace, e.g.
"give me a list of all containers in the cadvisor masterspace". But maybe
that's better served by labels anyway. Yes, I think so.

Thoughts? I could go wither way (UUIDs or masterspace + unique name). Or
does someone have other ideas? I feel like this got complicated pretty
fast.

Tim

On Wed, Jun 25, 2014 at 8:09 PM, Daniel Smith notifications@github.com
wrote:

I want the container names from the different sources to have different
namespaces, so there won't be any collisions. E.g., kubelet
prepends/appends ".etcd" (or something) to etcd-sourced containers, ".cfg"
to containers from the config file, ".http" to containers from the manifest
url, etc. Then it is up to each source to stay unique.

Api server can stay unique via uuid or counting up. Config files &
manifest url stay unique by humans not screwing up; kubelet rejects them
otherwise.

This has the nice effect that the container name produced from a config
file is predictable without having to do a lookup. This would be good for
our own container vm image and anything like it.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@lavalamp
Copy link
Member Author

Just to be clear, I only think that the property of predictable names is desirable for containers that come from the manifest url and maybe the container, because without that property, docker ps is unhelpful. But you've got me somewhat convinced that maybe I shouldn't care so much about that.

I don't know if we want to wait to solve naming in general before we accept a better solution than what we do currently.

@thockin
Copy link
Member

thockin commented Jun 26, 2014

I could put my weight behind either approach. The problem with UUIDs (be
they RFC4122 or SHA or whatever) is that they suck for humans. The problem
with !UUIDs is that we have to count on the "master" to get uniqueness
right.

On Wed, Jun 25, 2014 at 10:12 PM, Daniel Smith notifications@github.com
wrote:

Just to be clear, I only think that the property of predictable names is
desirable for containers that come from the manifest url and maybe the
container, because without that property, docker ps is unhelpful. But
you've got me somewhat convinced that maybe I shouldn't care so much about
that.

I don't know if we want to wait to solve naming in general before we
accept a better solution than what we do currently.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@jjhuff
Copy link
Contributor

jjhuff commented Jun 26, 2014

"I feel like this got complicated pretty fast." Yup:)

[I think I've read & parsed all of the comments, but I could be mistaken]

I like the idea of DNS-styled hierarchical namespaces. I'm not super worried about multiple masters scheduling over the same cluster of minions, but it does make multi-tenant masters easier (i.e. a provider is running a master, and customers run their minions). It's also something that people are used to.

I think it's also reasonable to require that users produce unique names for managing in the system.

Machine-generated unique IDs are more useful for running containers (i.e. a global PID). Having this be globally unique is really handy for things like log aggregation, etc. Why not always have the kubelet generate that? The master can learn it when it lists the running containers.

It's worth noting that the only restriction of docker names is that they are unique to the host. We can still encode useful human data (manifest+container names) in addition to the unique ID into the name...even if we only parse out the unique ID!

FWIW, I'm just polishing my ID cleanups to remove dead (or dieing) code. My goal was to normalize on docker IDs for many/most things inside the kubelet. I should have it out for review tomorrow.

@jjhuff
Copy link
Contributor

jjhuff commented Jun 26, 2014

Also, clarifying some terminology might be handy. Here's how I think of things (could easily be wrong!):

  • ContainerManifest is a collection of Containers.
  • Pods are an instance of a ContainerManifest. They can be running (or not). They are assigned to a host.

Things that confuse me:

  • ContainerManifests have an Id. Containers have a name. Pods have an Id. Seems like the manifest should only have a name (but this gets to the discussion above about names vs ids).
  • ReplicationController has a PodTemplate which really just wraps ContainerManifest. Seems like overkill.

@thockin
Copy link
Member

thockin commented Jun 26, 2014

On Wed, Jun 25, 2014 at 10:32 PM, Justin Huff notifications@github.com wrote:

"I feel like this got complicated pretty fast." Yup:)

[I think I've read & parsed all of the comments, but I could be mistaken]

I like the idea of DNS-styled hierarchical namespaces. I'm not super worried about multiple masters scheduling over the same cluster of minions, but it does make multi-tenant masters easier (i.e. a provider is running a master, and customers run their minions). It's also something that people are used to.

I'm more worried about "masters" that are config files crafted by
humans without coordination. Those should not accidentally collide :)

I think it's also reasonable to require that users produce unique names for managing in the system.

Here "users" == apiserver?

Machine-generated unique IDs are more useful for running containers (i.e. a global PID). Having this be globally unique is really handy for things like log aggregation, etc. Why not always have the kubelet generate that? The master can learn it when it lists the running containers.

Yeah, this is doable (and is in fact closer to what we do internally)

It's worth noting that the only restriction of docker names is that they are unique to the host. We can still encode useful human data (manifest+container names) in addition to the unique ID into the name...even if we only parse out the unique ID!

I think it is useful, but not critical that docker-reported names be
human-friendly.

FWIW, I'm just polishing my ID cleanups to remove dead (or dieing) code. My goal was to normalize on docker IDs for many/most things inside the kubelet. I should have it out for review tomorrow.

I'm keenly interested in this, and I'd like to stabilize it all ASAP.
Please do not merge this change until I have had time to feedback on
it. Caveat: I am part-time the rest of this week.

Reply to this email directly or view it on GitHub.

@thockin
Copy link
Member

thockin commented Jun 26, 2014

On Wed, Jun 25, 2014 at 11:07 PM, Justin Huff notifications@github.com wrote:

Also, clarifying some terminology might be handy. Here's how I think of things (could easily be wrong!):

ContainerManifest is a collection of Containers.
Pods are an instance of a ContainerManifest. They can be running (or not). They are assigned to a host.

The names as defined in pkg/api are somewhat confusing. It all
depends on perspective. We really have two APIs here (user -> master;
master -> kubelet), and we're cramming the spec into one file. We
should consider whether we really want to do that.

From Kubelet's POV ContainerManifest == Pod.

Things that confuse me:

ContainerManifests have an Id. Containers have a name. Pods have an Id. Seems like the manifest should only have a name (but this gets to the discussion above about names vs ids).

Yes, let's settle the discussion about UUIDs vs Names before we rename
things. and then lets rename things.

ReplicationController has a PodTemplate which really just wraps ContainerManifest. Seems like overkill.

I think this was to leave room for growth. Kubelet should do the same
and define a Pod type that has an api.ContainerManifest and maybe the
UUID (if we do that).

BTW: I have a change pending (not sent yet) to do validation of a
ContainerManifest - so it will reject manifests that, for example, do
not have an ID. The core of it is done, but it needs a test and has
some open questions. Hope to have it out tomorrow.

Reply to this email directly or view it on GitHub.

@bgrant0607
Copy link
Member

Whew! This is long. This issue started with no context about what we're trying to do.

Starting with the end: Yes, we should clean up the identifiers. Currently, every object/resource includes JSONBase, which has an ID (which probably should be Id). ContainerManifest also has an Id. Container has Name. Port has Name. I'll point out that Container and Port are not standalone resources right now. ContainerManifest is the way it is due to compatibility with the container-vm release.

What are we trying to do? I saw several things mentioned in this issue:

  • Human-friendly, memorable, semantically meaningful, short-ish references in operations
  • Predictable references in operations and/or configuration files
  • Unambiguous references in space, such as pods/containers created by multiple masters or by static configuration
  • Unambiguous references in time, such as for logging
  • Idempotent creation (Idempotent creation #148)
  • Auto-generation of DNS names (DNS #146)
  • Identification of sets for aggregation

Both unique identifiers and human-friendly names have value. Human-friendly names can only be unique in space, not in time. We should use "Id" for the former and "Name" for the latter. Names could be used to ensure idempotent (at most once) creation, though a non-indexed resource value could be used for that, also.

The replicationController would treat "Name" in the template as a prefix and would append relatively short random numbers for uniqueness. Static pods could be treated similarly.

Label selectors should be used for set formation / aggregation.

If users are providing Names, we could also use them for DNS (#146). I'd use "domain" rather than "masterspace".

Argh, my laptop needs to be rebooted...

@bgrant0607
Copy link
Member

Continuing...

I agree we want a mechanism that permits unique id allocation by Kubelets and that doesn't require centralized and/or persistent state. I like UUIDs. There are the issues of ensuring unique MAC addresses in VMs and/or namespaces, and determinism for testing, but I think we've found ways around these issues.

I've considered not having human-friendly names for pods before and just using labels instead. Labels are predictable and human-friendly, but don't require uniqueness, and don't require concatenating lots of identifying info together in order to ensure uniqueness, which users WILL do for names (and they'll want to parameterize them). They aren't short, though. Also, I guess part of the problem is that Docker doesn't support labels. We should push on that.

Idempotence could be ensured by a client-generated cookie, such as a fingerprint/hash or PR number. It wouldn't be required for static configs.

DNS names for instances aren't super-useful if they aren't predictable and DNS-like human-friendly names aren't that friendly if they are long. Short nicknames don't have to be predictable (so they could have a short uniquifying suffix) and don't even need to be semantically relevant -- just memorable (hence Docker's silly auto-generated names, I guess).

Services need predictable DNS names, OTOH, and ports need predictable names (for DNS SRV lookup or ENV vars or whatever), and pod-relative hostnames of containers should be predictable, so they can communicate with each other, though I don't know that they actually need to be FQDNs. Other types of services (e.g., master-elected services) and groups will need predictable DNS names, also.

Is the main motivation for names for pods to be consistent across all resource types? If so, I could buy into DNS-like hierarchical names for them. I'd use something like "domain" or "namespace" instead of "masterspace".

@smarterclayton
Copy link
Contributor

Pod-relative hostnames and stable internal names are definitely valuable - and limiting "name" to rfc1035 subdomain has been extremely valuable in practice to us on OpenShift.

The replicationController would treat "Name" in the template as a prefix and would append relatively short random numbers for uniqueness. Static pods could be treated similarly.

Quasi-uniqueness I assume?

Other types of services (e.g., master-elected services) and groups will need predictable DNS names, also.

How predictable? As a concrete example, with something like Zookeeper you need the container to have an identifier/name that is stable across restarts / reschedules in a shared config (so not a pod ID). I'd assumed you'd model this with a set of replication controllers (vs a shared controller) so you had 3 replication controllers with 1 item each, and you'd be able to either set an ENV per pod, or use the name in order to apply that. If name changes over pod instances that rules out the reuse there.

@lavalamp
Copy link
Member Author

One last thought from me; it occurs to me that docker already generates a long container id. Perhaps we can consider using that directly as our spatial/temporal unique identifier, and use this hypothetical dns-style solution as the friendly human name. We'd need to investigate just how unique docker's id's are, and we may not want to depend on docker for that, but it would reduce the number of IDs needed.

Also, as a footnote, if we end up with both pod.ID and Manifest.ID, IMO they should be the same identifier.

@jjhuff
Copy link
Contributor

jjhuff commented Jun 26, 2014

One last thought from me; it occurs to me that docker already generates a
long container id. Perhaps we can consider using that directly as our
spatial/temporal unique identifier, and use this hypothetical dns-style
solution as the friendly human name. We'd need to investigate just how
unique docker's id's are, and we may not want to depend on docker for that,
but it would reduce the number of IDs needed.

Yes, I was starting to think the same thing.

Also, as a footnote, if we end up with both pod.ID and Manifest.ID, IMO
they should be the same identifier.

My only concern here is that starts to feel weird when a Manifest is used
as part of a ReplicationController. I think. You end up with N pods and a
Manifest each with ID fields.

@lavalamp
Copy link
Member Author

For clarity, let me suggest the convention that "name" means a friendly mostly human readable string, possibly in the style of dns names, and that "id" means an opaque, machine generated identifier, guaranteed unique at some level of resolution. Maybe everyone except for me is already using this convention. :) But I want to talk about ids and names in general without referring to a particular implementation.

My only concern here is that starts to feel weird when a Manifest is used as part of a ReplicationController. I think. You end up with N pods and a Manifest each with ID fields.

I think, in that case, the rep. controller itself has a name, which it can use to make names for the pods it creates (prepend/append indices or something). IDs for the pods could still be generated by the apisever or wherever we decide they need to be generated. I was trying to say that since there's a 1:1 relationship between pods and manifests, we shouldn't make different IDs for each.

@thockin
Copy link
Member

thockin commented Jun 26, 2014

Trying to collect thoughts and ideas into a proposal. It's tricky
considering it from all angles (users, apiserver, replication controllers,
kubelet). My background is node-centric, so I am counting on people to
smack me if I say something dumb for higher-levels :)

I have to run out right now, but I'm hoping we can distill the discussion
into a design O(soon).

NB: We spec names as RFC 1035 compatible, though we might extend that to
allow purely numeric tokens (e.g. 123.something.com) but I forget which RFC
that is. Docker names allow underscores, which kubelet reserves for use in
infrastructure containers.

From kubelet's point of view:

  1. All pods have a namespace string, which is human-friendly and DNS
    friendly (rfc1035 subdomain fragment). For example: "k8s.mydomain.com".
    This is used to indicate the provenance of the pod. If the namespace is
    not specified when creating a pod, kubelet will assign a namespace derived
    from the source. For example, a file "/etc/k8s/cadvsor.pod" on host "
    xyz123.mydomain.com" might get masterspace "
    file-f5sxiyzpnm4hgl3dmfshm2ltn5zc44dpmqfa.xyz123.mydomain.com" (the big
    mess is a base32 encoding of the path).

  2. All pods have a name string, which is human-friendly and DNS friendly
    (rfc1035 subdomain fragment). For example: "id8675309.myjob". These names
    are typically NOT user-provided, but are generated from infrastructure.
    For example, if the user asked for a job named "myjob" the apiserver must
    uniquify that into a pod name.

  3. The namespace + name pair is assumed to be unique. This provides simple
    idempotency, for example when re-reading a config file.

Open: Do we need UUIDs at all? For what purpose? The only argument I can
come up with is to protect against accidentally non-unique names at cluster
scope (e.g. in a giant database of namespace, name, uuid, stats) you could
disambiguate colliding namespace+names pairs with uuid. Other
justifications?

From the apiserver's point of view:

  1. The apiserver has a configured namespace.

  2. All incoming pods have a name. I don't know what the formatting and
    uniqueness rules are here.

  3. Upon acceptance, a pod is assigned a unique ID of some sort.

  4. Upon assignment to a minion, the name and unique ID become the pod name.

Open: Should the unique ID persist if the pod is moved to a new minion?
Maybe we need two IDs - one that sticks across moves and one that does not?

On Thu, Jun 26, 2014 at 9:46 AM, Daniel Smith notifications@github.com
wrote:

For clarity, let me suggest the convention that "name" means a friendly
mostly human readable string, possibly in the style of dns names, and that
"id" means an opaque, machine generated identifier, guaranteed unique at
some level of resolution. Maybe everyone except for me is already using
this convention. :) But I want to talk about ids and names in general
without referring to a particular implementation.

My only concern here is that starts to feel weird when a Manifest is used
as part of a ReplicationController. I think. You end up with N pods and a
Manifest each with ID fields.

I think, in that case, the rep. controller itself has a name, which it can
use to make names for the pods it creates (prepend/append indices or
something). IDs for the pods could still be generated by the apisever or
wherever we decide they need to be generated. I was trying to say that
since there's a 1:1 relationship between pods and manifests, we shouldn't
make different IDs for each.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@bgrant0607
Copy link
Member

Numeric host names: http://tools.ietf.org/html/rfc1123#page-13

@smarterclayton
Copy link
Contributor

Regarding the open question: is a pod on a new minion the same as the old pod? Is a move an action that is logically part of kubernetes, or is only "remove" and "create new" available? If it's the former, it seems like the ID should be the same, if it's the later it seems like it should be different.

We have a use case for being able to move a pod from minion to minion (and any volumes that come with it) - however, since this requires the volume data on disk to be in a resting state, it's not an operation that seems to lend itself well to the replication controller (since the move is inherently stateful). So I would expect this to be managed as an operation above the replication controller, vs part of it.

One namespace per apiserver can be limiting if the namespace is automatically bound to DNS and you're dealing with very large numbers of containers, but it doesn't sound unreasonable if you are using wildcard DNS.

@smarterclayton
Copy link
Contributor

One minor note regarding Docker ecosystem - "name" is currently being used in Docker for a lot of lightweight integrations (linking, sky dock dns, hostname in container). A side effect of any generated name for a Docker container is that those lightweight integrations may become more difficult for end admins. Is there a practical way to make the name appropriately unique on the minion without breaking those potential integrations (making the name a subdomain fragment by omitting '.' for instance)

@brendanburns
Copy link
Contributor

I would also like to see us drive labels all the way down into Docker. That
would enable us to abandon much of what we are encoding into the name today.

Brendan

On Fri, Jun 27, 2014, 8:39 AM, Clayton Coleman notifications@github.com
wrote:

One minor note regarding Docker ecosystem - "name" is currently being used
in Docker for a lot of lightweight integrations (linking, sky dock dns,
hostname in container). A side effect of any generated name for a Docker
container is that those lightweight integrations may become more difficult
for end admins. Is there a practical way to make the name appropriately
unique on the minion without breaking those potential integrations?


Reply to this email directly or view it on GitHub
#199 (comment)
.

@bgrant0607
Copy link
Member

+1 to what brendanburns@ wrote. Labels provide much cleaner solutions for
many use cases.

@thockin
Copy link
Member

thockin commented Jun 28, 2014

On Thu, Jun 26, 2014 at 10:51 PM, Justin Huff notifications@github.com wrote:

I think this seems like a good outline!
Two comments:

  1. For config files, maybe use a hash of the file vs a base64 encoding of the path as the component of the namespace. Just seems more natural.

This is an implementation detail - human-friendliness of docker's
names is a nice-to-have not critical, I think.

  1. "These names are typically NOT user-provided, but are generated by infrastructure." I'd consider being more explicit about that behavior, even if it's as simple as a "unique prefix will be added to the name". This sorta touches on the whole id/name discussion....

Yeah, where is the line between "ID" and "name"? The apiserver COULD
just send a UUID for the name, but I think we want the structure and
content of a pod spec to stay largely the same throughout the entire
system, at least for now. I don't think we've got any consensus on
diverging it, anyway. For that reason, I think "name" makes sense.
It is more human readable than not.

@thockin
Copy link
Member

thockin commented Jun 28, 2014

+1 to labels

On Fri, Jun 27, 2014 at 8:45 AM, brendanburns notifications@github.com
wrote:

I would also like to see us drive labels all the way down into Docker.
That
would enable us to abandon much of what we are encoding into the name
today.

Brendan

On Fri, Jun 27, 2014, 8:39 AM, Clayton Coleman notifications@github.com
wrote:

One minor note regarding Docker ecosystem - "name" is currently being
used
in Docker for a lot of lightweight integrations (linking, sky dock dns,
hostname in container). A side effect of any generated name for a Docker
container is that those lightweight integrations may become more
difficult
for end admins. Is there a practical way to make the name appropriately
unique on the minion without breaking those potential integrations?

Reply to this email directly or view it on GitHub
<
https://github.com/GoogleCloudPlatform/kubernetes/issues/199#issuecomment-47362231>

.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@thockin
Copy link
Member

thockin commented Jun 28, 2014

What would break on those? We're not proposing to do anything with name
that is not totally valid as per standalone Docker?

On Fri, Jun 27, 2014 at 8:39 AM, Clayton Coleman notifications@github.com
wrote:

One minor note regarding Docker ecosystem - "name" is currently being used
in Docker for a lot of lightweight integrations (linking, sky dock dns,
hostname in container). A side effect of any generated name for a Docker
container is that those lightweight integrations may become more difficult
for end admins. Is there a practical way to make the name appropriately
unique on the minion without breaking those potential integrations?

Reply to this email directly or view it on GitHub
#199 (comment)
.

@smarterclayton
Copy link
Contributor

It's not that the names aren't valid for Docker, its that something consuming those names (as a dns prefix a la skydock, or as a hostname) would break due to length or format on the Kube generated name. I don't think that is a blocker to the proposed naming patterns above, but it's a consideration when thinking about how other software that plays well with Docker might react.

Concrete ideas might be to reduce the generated names' length and avoid using '.' as a separator.

@thockin
Copy link
Member

thockin commented Jun 29, 2014

I would argue (for the sake of arguing) that anyone who was making
assumptions about container names deserves to be broken. But yeah, let's
keep an eye on it.

On Sat, Jun 28, 2014 at 2:55 PM, Clayton Coleman notifications@github.com
wrote:

It's not that the names aren't valid for Docker, its that something
consuming those names (as a dns prefix a la skydock, or as a hostname)
would break due to length or format on the Kube generated name. I don't
think that is a blocker to the proposed naming patterns above, but it's a
consideration when thinking about how other software that plays well with
Docker might react.

Concrete ideas might be to reduce the generated names' length and avoid
using '.' as a separator.

Reply to this email directly or view it on GitHub
#199 (comment)
.

@smarterclayton
Copy link
Contributor

Agree, naming is hard

@smarterclayton
Copy link
Contributor

Writing up a summary md - does this belong in api/doc, DESIGN.md, or another location?

@thockin
Copy link
Member

thockin commented Jul 1, 2014

I would go for docs/identifiers.md or something

On Mon, Jun 30, 2014 at 2:48 PM, Clayton Coleman notifications@github.com
wrote:

Writing up a summary md - does this belong in api/doc, DESIGN.md, or
another location?

Reply to this email directly or view it on GitHub
#199 (comment)
.

smarterclayton added a commit to smarterclayton/kubernetes that referenced this issue Jul 1, 2014
@smarterclayton
Copy link
Contributor

Referenced pull includes a summary of this discussion, open questions:

  • implication 3-1 (agree/disagree)?
  • I defined "pod unique name" and "pod unique identifier" to represent thockin's kubelet items 3 and 4

vishh pushed a commit to vishh/kubernetes that referenced this issue Apr 6, 2016
rphillips added a commit to rphillips/kubernetes that referenced this issue Mar 13, 2018
seans3 pushed a commit to seans3/kubernetes that referenced this issue Apr 10, 2019
…lease-blocking-signaling

update release blocking issue handling for 1.6
wking pushed a commit to wking/kubernetes that referenced this issue Jul 21, 2020
pjh pushed a commit to pjh/kubernetes that referenced this issue Jan 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants