Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rkt: Better intergration #8262

Closed
11 of 15 tasks
yifan-gu opened this issue May 14, 2015 · 19 comments
Closed
11 of 15 tasks

rkt: Better intergration #8262

yifan-gu opened this issue May 14, 2015 · 19 comments
Labels
area/rkt lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. priority/backlog Higher priority than priority/awaiting-more-evidence. sig/node Categorizes an issue or PR as relevant to SIG Node.

Comments

@yifan-gu
Copy link
Contributor

yifan-gu commented May 14, 2015

This issue tracks the some of the TODOs to make using rkt in k8s have better experience.

  • Add a README.md or doc.go in rkt package (update: README can be found here now)
  • Integrate the reference manager, readiness manager.
  • Per container logs
  • Stdin and TTY (Support Stdin and TTY in the kubelet #12168)
  • Integrate with the cAdvisor.
  • More message and reasons for events like pull failures.
  • Image GC (kubelet/rkt: add container/image gc for rkt. #14686)
  • Enforce container/image gc policy.
  • Run ACI, integrate rkt trust.
  • Being able to change stage1 image.
  • e2e tests.
  • Network plugins. Join existing network?
  • Per container lifecycle hooks
  • Replace cli calls with rkt API
  • security context

/cc @dchen1107 @vmarmol @yujuhong @jonboulle @bakins

@vmarmol vmarmol added priority/backlog Higher priority than priority/awaiting-more-evidence. sig/node Categorizes an issue or PR as relevant to SIG Node. area/rkt labels May 14, 2015
@yifan-gu yifan-gu changed the title Better rkt intergration rkt: Better intergration May 15, 2015
@kamalmarhubi
Copy link
Contributor

Should the "Stdin and TTY" item be checked off now that #12168 has been merged? Is the rest of the status up to date? I'm quite interested in using rkt over Docker and might be willing to do some work here.

@yifan-gu
Copy link
Contributor Author

@kamalmarhubi No jobs have been done for the Stdin & TTY yet, helps welcome :)
The image gc also should be easier than the others.
Enabling as many e2e tests as possible for rkt might be a little painful...

Besides, we are currently design/develop APIs for rkt, which should save us many effort maintaining all these in long term.

@kamalmarhubi
Copy link
Contributor

@yifan-gu when you say developing APIs, do you mean as a Go library, or something akin to the Docker daemon with, eg, an HTTP interface?

@yifan-gu
Copy link
Contributor Author

@kamalmarhubi No restful API at this time, we are going to use grpc. It should be easy to generate client bindings, and maybe we will do a more "docker client like" wrapper over the binding.

@kamalmarhubi
Copy link
Contributor

That makes sense. Does this mean there will be a new daemon to go along side the metadata service?

@yifan-gu
Copy link
Contributor Author

yifan-gu commented Sep 3, 2015

@kamalmarhubi Yes. but it's also optional, you don't require it to run rkt. It just provides easy access to the pod/image information.
That means it should run in non-root, and it's failover should not affect running pods.

@kamalmarhubi
Copy link
Contributor

@yifan-gu ah so it's not used for starting pods?

@yifan-gu
Copy link
Contributor Author

yifan-gu commented Sep 3, 2015

@kamalmarhubi No.

@yifan-gu
Copy link
Contributor Author

yifan-gu commented Sep 3, 2015

@kamalmarhubi FYI, here's the PR for the API interface rkt/rkt#1359

@kamalmarhubi
Copy link
Contributor

Awesome thanks!

@yifan-gu
Copy link
Contributor Author

yifan-gu commented Oct 9, 2015

Run e2e tests on local:

Ran 109 of 185 Specs in 7246.865 seconds
FAIL! -- 60 Passed | 49 Failed | 2 Pending | 74 Skipped --- FAIL: TestE2E (7246.87s)
FAIL

Not every failures are related to rkt. As I noticed I had bunch of failures running with docker locally as well.
Detailed logs are here:
https://gist.github.com/yifan-gu/da1d30bf077d99cfa71f

@yifan-gu
Copy link
Contributor Author

yifan-gu commented Oct 9, 2015

Trying to setup gce env for e2e tests next.

@yujuhong
Copy link
Contributor

yujuhong commented Oct 9, 2015

I've never run e2e in a non-gce environment, and I wouldn't be surprised if a lot of them fail. Yes, setting up a gce cluster would definitely help.

@ppalucki
Copy link

there a additional rather minor issues, I found related to rkt driver:

  • cannot start new pod with "runonce"
  • better error handling eg. "ppid file not found" message for not-defined but required volume scenario
  • "container runtime" (describe node) always says "docker" regardless --container-runtime
  • feature - rkt support for /cluster/* deployment scripts (we have that for libvirt)
  • rkt with kvm engine support: kubelet logs/exec, gc

some of them are under our investigation

@yifan-gu: my question is, are you aware some of those ? are you working on some of them ? we can/will help you with some of them, but want to make sure, that we dont overlap

@yifan-gu
Copy link
Contributor Author

@ppalucki I am setting gce e2e test environments for rkt/k8s, so will get to these later, but feel free to take them, thanks :)

@borqosky
Copy link

@yifan-gu : Can you tell, what's the current status of e2e tests with rkt, or is there any PR related with it ? I see only #7205

@yifan-gu
Copy link
Contributor Author

@woodbor Here is the brief result
FAIL! -- 65 Passed | 103 Failed | 0 Pending | 69 Skipped
Run last week
logs here: https://gist.github.com/yifan-gu/f970a0b3748543ee919f

FAIL! -- 109 Passed | 59 Failed | 0 Pending | 69 Skipped
Run yesterday with a hacked rkt that supports hostname /etc/hosts
logs here: https://gist.github.com/yifan-gu/51e29ef92428e4272d04

We are tracking more status in this google doc https://docs.google.com/document/d/1otDQ2LSubtBUaDfdM8ZcSdWkqRHup4Hqt1VX1jSxh6A/edit#

@yifan-gu yifan-gu added this to the rktnetes-v1.1 milestone Jun 24, 2016
@thockin thockin removed this from the rktnetes-v1.1 milestone May 27, 2017
@fejta-bot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

@k8s-ci-robot k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 25, 2017
@yujuhong
Copy link
Contributor

Closing this since relevant issues should be handled in CRI or moved to the rktlet repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/rkt lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. priority/backlog Higher priority than priority/awaiting-more-evidence. sig/node Categorizes an issue or PR as relevant to SIG Node.
Projects
None yet
Development

No branches or pull requests

9 participants