I would vote for 2.

I also want to get to a state where a pod will always let me know the resources it could consume or always apply same defaults.

Sent from my iPhone

On Feb 13, 2015, at 6:52 PM, Ravi Sankar Penta notifications@github.com wrote:

Currently Node 'resources' field is optional. We may have few issues here:

When resources field is omitted for some nodes and not for others during node creation, nodes with empty resources is not preferred by the scheduler. So the pods are unevenly distributed. (One of my team member ran into this issue while testing the release)
When resources field is omitted for all nodes in the cluster. Scheduler will load balance the pods and this works well if all nodes are homogeneous (i.e. same cpu/memory limits) otherwise some nodes will be under utilized.
When resources field is present but has incorrect values (fat finger) can lead to over/under utilization on the node. We are not doing any resource validations in the platform.
Few options to avoid these above issues:

Make resources field mandatory for the node and validate the limits.
Auto generate the node resources if the field is omitted by the user and if the field is defined by the user, validate the limits (user given limits must be <= the actual limits on the node). If we prefer this option, kubelet can provide GET '/facts' resource that can list current cpu/memory limits on the node and POST /nodes on the master can fetch the node facts and can validate or populate the resources field before creating the node representation. Either option 1 or 2 might need migration tool to patch existing nodes in the running cluster.
Sheduler can treat empty resources field for node as infinite! resources and should be considered for scheduling pods.
If we think sheduler will be smart enough to reject pod creation based on the node resource limits at some point, I feel option 2 will be fool proof and gives better experience to the user.
Please let me know your thoughts on this: @bgrant0607 @ddysher @abhgupta

—
Reply to this email directly or view it on GitHub.

In general, Kubelet should report node status. See #4135 for more discussion.

I'm happy with Kubelet reporting total/nominally available resources in status, also. If you want to implement this soon, we can discuss what the API should be exactly.

As discussed in https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/resources.md, usage data should not be in node status, however. We're working on a stats collector which will provide that info #4057.

cc @rjnagal @vishh @dchen1107 @ddysher

@derekwaynecarr Please file a separate issue if you want to discuss defaulting and/or auto-sizing pod resource requests/limits.

Fixed in #5030
Node 'resources' field is moved from node spec to node status in v1beta3 and is no longer a config param to the user. Node controller or Kubelet(based on --sync_node_status flag) will populate 'resources' field and is periodically updated as part of node status.