| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Implement a Kubernetes driver https://review.openstack.org/521356 | 08:56 |
|---|---|---|
| *** haint has quit IRC | 13:14 | |
| *** nguyentrihai has joined #zuul | 13:15 | |
| *** haint has joined #zuul | 13:25 | |
| *** nguyentrihai has quit IRC | 13:28 | |
| SpamapS | tristanC: interesting k8s driver there | 14:34 |
| SpamapS | tristanC: I'm not convinced I want an ssh-able container. | 14:34 |
| SpamapS | I'd much prefer to have a namespace and use something like this: https://github.com/ansible/ansible/pull/26668/commits/21083b6b789e314e0cd84a83c03029c8a0b092a7 | 14:41 |
| SpamapS | in fact I've been thinking more about nodepool, "cleanup jobs", and secrets in general. I'm convinced what we want is to turn nodepool into "thing pool" and have it call back to zuul to run setup and teardown jobs. | 15:05 |
| pabelanger | SpamapS: doesn't kubectl require admin access on k8s? Or can any user run it? | 16:01 |
| SpamapS | pabelanger: kubectl is just speaking the REST API. authz is pretty rich in k8s. | 16:04 |
| pabelanger | cool, thanks! | 16:04 |
| SpamapS | pabelanger: so you could have nodepool setup a namespace, and then hand creds that can access it to the job. | 16:04 |
| pabelanger | wasn't sure how it all worked | 16:04 |
| SpamapS | To your point, just like how we have to setup an SSH key to access the node for zuul.. we have to do that for k8s too | 16:05 |
| SpamapS | but the namespace would be sufficient isolation, and there are such things as namespace quotas, so you can also prevent the job from deploying too much in it namespace. What might be tricky is also giving each job a unique authn so that they can't steal resources from eachother by guessing namespaces (though they could be random at least) | 16:06 |
| SpamapS | anyway, time to make breakfast burritos | 16:06 |
| pabelanger | om nom nom | 16:10 |
| pabelanger | but, looking at the URL you posted, if we had some sort of generic driver for ansible in nodepool, it would be easy to swap between oci and k8s I think | 16:11 |
| tristanC | SpamapS: heh, that could work too :) | 16:20 |
| tristanC | though it doesn't feel much different than ssh, and right now it's more simple to make nodepool give a ssh port back to zuul | 16:21 |
| tristanC | pabelanger: it seems like a generic driver in nodepool would be great, let alone ansible | 16:23 |
| tristanC | or we could continue to refactor the generic bits out of the openstack driver so that's just easier to implement a new driver | 16:25 |
| tristanC | SpamapS: ssh is also probably a better bet for module like synchronize | 16:28 |
| pabelanger | maybe start with SSH then do something more k8s specific? | 16:29 |
| pabelanger | I admit, I don't know what that looks like | 16:29 |
| tristanC | fwiw that nodepool driver only was enough to make zuul-jobs run on kubernetes | 16:32 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Revert "Add ensure-reno and ensure-babel roles" https://review.openstack.org/521253 | 21:43 |
| *** threestrands has joined #zuul | 22:27 | |
| *** threestrands has joined #zuul | 22:27 | |
| SpamapS | tristanC: yeah I think what you're doing is bolting sshd onto a container model that actively rejects it. I'd rather see jobs that build container images with the code than push too hard making zuul-jobs work as-is. | 23:13 |
| SpamapS | So in a k8s world, instead of preparing workspaces by pushing code onto running nodes, you would build an image on top of a base image and upload it, booting the containers from that. | 23:14 |
| SpamapS | If you just want container booting to chop up vms or something, lxd is a better choice. | 23:15 |
| SpamapS | Anyway, cool that you can get something on k8s. :) | 23:18 |
| SpamapS | I have planned on doing something similar soon if I get any free time. | 23:19 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!