Tuesday, 2018-06-12

« Monday, 2018-06-11 Index Wednesday, 2018-06-13 »
*** eernst has quit IRC01:21
*** zerocoolback has joined #kata-dev02:31
*** zerocoolback has quit IRC04:34
*** zerocoolback has joined #kata-dev04:38
*** zerocoolback has quit IRC04:51
*** zerocoolback has joined #kata-dev04:59
*** zerocoolback has quit IRC05:39
*** zerocoolback has joined #kata-dev05:39
*** zerocoolback has quit IRC05:40
*** zerocoolback has joined #kata-dev05:40
*** zerocoolback has quit IRC05:40
*** zerocoolback has joined #kata-dev05:41
*** zerocoolback has quit IRC05:41
*** zerocoolback has joined #kata-dev05:41
*** zerocoolback has quit IRC05:42
*** zerocoolback has joined #kata-dev05:42
*** zerocoolback has quit IRC05:43
*** dklyle has quit IRC06:18
*** dklyle has joined #kata-dev06:18
*** david-lyle has joined #kata-dev06:20
*** dklyle has quit IRC06:23
*** david-lyle has quit IRC06:32
*** jodh has joined #kata-dev07:11
*** jodh has quit IRC07:11
*** jodh has joined #kata-dev07:11
*** dklyle has joined #kata-dev07:28
*** sameo has joined #kata-dev07:32
*** sjas has joined #kata-dev07:47
*** gwhaley has joined #kata-dev07:59
*** davidgiluk has joined #kata-dev08:03
kata-irc-bot<niteshkonkar007> (Just a thought) Should we put a note somewhere for first time Kata-users that they should try kata on a bare-metal or on a VM with nested-virtualization? I know "kata-runtime kata-check" will show the error but then that's more like a disappointment  after  following some set of steps.08:24
*** cdent has joined #kata-dev08:42
*** chuanchang has joined #kata-dev09:14
*** zerocoolback has joined #kata-dev09:44
*** zerocoolback has quit IRC09:51
*** zerocoolback has joined #kata-dev09:52
*** zerocoolback has quit IRC09:52
*** zerocoolback has joined #kata-dev09:53
*** zerocoolback has quit IRC09:53
*** zerocoolback has joined #kata-dev09:54
*** zerocoolback has quit IRC09:54
*** zerocoolback has joined #kata-dev09:54
*** zerocoolback has quit IRC09:55
*** zerocoolback has joined #kata-dev09:55
*** zerocoolback has quit IRC09:55
*** chuanchang has quit IRC10:11
*** chuanchang has joined #kata-dev10:12
*** chuanchang has quit IRC10:16
*** dlw has quit IRC10:46
*** chuanchang has joined #kata-dev10:48
*** gwhaley has quit IRC11:07
*** bgmccollum has joined #kata-dev11:44
*** devimc has joined #kata-dev11:45
*** gwhaley has joined #kata-dev12:15
*** zerocoolback has joined #kata-dev12:29
kata-irc-bot<salvador.fuentes> @kata ping12:37
*** dlw has joined #kata-dev13:02
*** lamego has joined #kata-dev13:05
*** chuanchang has quit IRC13:09
*** zerocoolback has quit IRC13:47
*** zerocoolback has joined #kata-dev14:18
*** zerocoolback has quit IRC14:19
*** dlw has quit IRC14:27
*** lamego1 has joined #kata-dev14:49
*** zerocoolback has joined #kata-dev14:49
*** lamego1 has quit IRC14:50
*** zerocoolback has quit IRC14:50
*** eernst has joined #kata-dev14:51
*** zerocoolback has joined #kata-dev14:51
*** sameo has quit IRC14:51
*** lamego has quit IRC14:51
*** lamego has joined #kata-dev14:51
kata-irc-bot<anne> @niteshkonkar007 the first line of the install guide calls this out. New addition as of last week14:52
*** lamego has joined #kata-dev14:53
kata-irc-bot<niteshkonkar007> @anne: Oh nice. I missed it somehow.14:53
*** mordred has quit IRC14:55
*** zerocoolback has quit IRC14:55
*** jodh has quit IRC15:04
*** mordred has joined #kata-dev15:09
*** devimc has quit IRC15:29
*** fiddletwix has quit IRC15:29
*** fiddletwix has joined #kata-dev15:31
*** lamego has quit IRC15:33
*** lamego has joined #kata-dev15:34
*** lamego1 has joined #kata-dev15:35
*** lamego has quit IRC15:39
*** mordred has quit IRC15:48
*** mordred has joined #kata-dev15:52
*** libregeekingkid[ has quit IRC16:08
*** devimc has joined #kata-dev16:11
*** devimc has left #kata-dev16:17
*** devimc has joined #kata-dev16:18
kata-irc-bot<jdandrea> Good hello all. I am trying to troubleshoot a kata-on-k8s installation. Has anyone perchance run into a problem where cni0 already has a different CIDR assignment than expected when bringing up an untrusted workload? If so, how did you work around it?16:19
kata-irc-bot<sebastien.boeuf> @jdandrea if you installed CNI plugins before, you might have some remaining things inside `/var/lib/cni/networks/`16:24
kata-irc-bot<sebastien.boeuf> running `rm -r /var/lib/cni/networks/*` helps16:24
kata-irc-bot<jdandrea> @sebastien.boeuf Yup, as part of troubleshooting I'm resetting kubeadm, stopping kubelet and docker, then removing everything under /var/lib/cni, /var/lib/kubelet, /etc/cni, bringing cni0, flannel.1 (we're using flannel), and docker0 down, then deleting cni0 and flannel.1 links besides.16:29
kata-irc-bot<jdandrea> Still happens. *scratches head*16:30
kata-irc-bot<sebastien.boeuf> here is my gist on how to tear everything down related to k8s+crio: https://gist.github.com/sboeuf/d84acfbaf2bccd528098d73a3b891cd1#file-run-k8s-cri-o-L7-L3716:31
kata-irc-bot<sebastien.boeuf> it's using flannel too16:31
*** libregeekingkid[ has joined #kata-dev16:32
kata-irc-bot<jdandrea> Thanks @sebastien.boeuf - appreciated! I will compare/contrast with what I'm doing so I can understand if/what I'm doing that's amiss. biab...16:37
kata-irc-bot<sebastien.boeuf> np !16:38
kata-irc-bot<jdandrea> @sebastien.boeuf Hmm. So the first thing I notice is your rm on line 5 won't work for me. Do I need to already have /var/lib/cni/networks owned by a non-root user?16:41
kata-irc-bot<jdandrea> The other difference is I don't use istio, but I suppose I can just bring up what I was bringing up previously.16:42
kata-irc-bot<sebastien.boeuf> no no, just follow lines 7 to 37, forget about the other ones16:46
kata-irc-bot<jdandrea> Got it16:46
kata-irc-bot<sebastien.boeuf> and yes, you don't care about istio, this is something I am working on, don't apply it in your case16:46
kata-irc-bot<jdandrea> @sebastien.boeuf Ah, I'm using containerd, not crio (gave up on crio for the time being).16:49
kata-irc-bot<jdandrea> So I will just do the equivalent bits for containerd.16:49
kata-irc-bot<sebastien.boeuf> you can follow this: https://github.com/kata-containers/documentation/blob/master/how-to/how-to-use-k8s-with-cri-containerd-and-kata.md#configure-containerd-to-use-kata-containers for details about CRI-containerd setup16:51
kata-irc-bot<jdandrea> @sebastien.boeuf Yes, that's what I was originally following. (I still appreciate the confirmation that I'm on the right track!)16:57
kata-irc-bot<jdandrea> Everything's fine up until that point with the cni0 conflict. :S16:57
kata-irc-bot<sebastien.boeuf> @jose.carlos.venegas.m thoughts ^^ ?17:02
kata-irc-bot<eric.ernst> That seems like a weird misconfiguration. Curious -- you are running nested -- does your host have similar mix of CNI installed?17:03
kata-irc-bot<jdandrea> @sebastien.boeuf You did it! It's working!17:43
kata-irc-bot<jdandrea> Now I have to retrace my steps once again and understand *why*.17:44
kata-irc-bot<jdandrea> ty tyvm17:44
kata-irc-bot<jdandrea> @eric.ernst It's possible, but anything in that regard that's on the host is disabled/stopped/deleted.17:45
kata-irc-bot<jdandrea> (This from before we went and tried the containerd steps.)17:45
kata-irc-bot<sebastien.boeuf> @jdandrea glad it worked :slightly_smiling_face:17:46
kata-irc-bot<sebastien.boeuf> don't hesitate to raise an issue if you run into a specific case where the documentation was not completely accurate !17:46
kata-irc-bot<jdandrea> @sebastien.boeuf Absolutely.17:46
kata-irc-bot<jdandrea> I am trying this within a ccloudvm instance as well.17:47
kata-irc-bot<jdandrea> To help make it easy to start over as I futz with getting things working.17:47
*** annabelleB has joined #kata-dev17:47
kata-irc-bot<eric.ernst> Also, let us know if it is accurate but insufficient!17:49
kata-irc-bot<jdandrea> Yes yes!17:49
kata-irc-bot<jdandrea> I am going to stop this VM, start a new one, and run the whoooole thing again, and compare vs. the instructions.17:49
*** annabelleB has quit IRC18:30
*** annabelleB has joined #kata-dev18:35
*** annabelleB has quit IRC19:04
*** fiddletwix has quit IRC19:20
*** fuentess has quit IRC19:24
kata-irc-bot<yonatan.gefen> Hello All,   While building kata-containers following the developer guide I noticed that in the _"Install guest kernel images"_ section you pull the kernel config from https://raw.githubusercontent.com/kata-containers/packaging/master/kernel/configs/x86_kata_kvm_4.14.x. It looks to me, based on the URL and some of the configuration parameters in this file, that this is specifically for an x86 target architecture.   Will this19:28
kata-irc-botconfiguration file and the resulting kernel also work if I would like to run Kata-Containers on top of an ARM target? How would you recommend approaching this goal?  Thank you very much for your input, Yonatan19:28
*** davidgiluk has quit IRC19:30
gwhaley@yonatan.gefen: is that the same thing as handled by this pending PR? :-) https://github.com/kata-containers/documentation/pull/15219:31
* gwhaley has to drop ...19:31
*** gwhaley has quit IRC19:32
kata-irc-bot<jdandrea> Is there a requirement to install a particular cri-tools version for use with Kata?19:33
kata-irc-bot<jdandrea> File uploaded https://katacontainers.slack.com/files/UAYMNLRK6/FB63U2RTN/-.sh / https://slack-files.com/T86U7NQTT-FB63U2RTN-4dfcf2f4f519:33
kata-irc-bot<jdandrea> ^^ what I'm doing now.19:33
kata-irc-bot<eric.ernst> the requirement would be with respect to the CRI shim, not kata.19:38
kata-irc-bot<eric.ernst> (for cri-tools)19:38
kata-irc-bot<yonatan.gefen> @graham.whaley Thank you. Looking through it now.19:45
kata-irc-bot<jdandrea> @eric.ernst Ok. This is what I was directed to use last Friday, so I want to be sure I should just stay on that version and not use the latest.19:46
kata-irc-bot<jdandrea> Also, since starting from scratch with the notes thus far, /etc/systemd/system/kubelet.service.d/10-kubeadm.conf is missing, hmm. Checking docs again.19:46
*** eernst has quit IRC19:47
*** eernst has joined #kata-dev19:48
*** fuentess has joined #kata-dev19:56
kata-irc-bot<jdandrea> @eric.ernst So, by following all the steps (and using containerd) I am now repeatedly bumping up against /etc/kubernetes/pki/ca.crt not being found.  Steps thus far: https://paste.ofcode.org/jmfAV5Wz6cpMe9yaD9pwhi20:15
kata-irc-bot<jdandrea> I may need to install a newer docker.20:15
kata-irc-bot<jdandrea> https://github.com/kubernetes/kubernetes/issues/5388920:17
kata-irc-bot<jdandrea> Ah. It's a catch-22 between kubelet and kubeadm init!20:19
kata-irc-bot<jdandrea> @eric.ernst So what I'm noticing is that kubelet doesn't have the cert in place so startup fails. kubeadm init, meanwhile, sets up those items but also requires kubelet to be running. Tenuous. Hmm...20:35
*** devimc has quit IRC20:45
*** libregeekingkid[ has quit IRC20:49
kata-irc-bot<jdandrea> So while kubeadm init DOES eventually succeed, kubelet is in fact not working for a while during that timeframe.20:53
kata-irc-bot<sebastien.boeuf> yes kubelet needs those certs, and kubeadm provide them to kubelet, which makes the service properly starting only when kubeadm is started20:56
kata-irc-bot<jdandrea> @sebastien.boeuf Indeed! So, what I'm wondering is: "Is it good form" to start kubelet knowing it won't startup successfully, and then use kubeadm to finish the job? (I suppose that's debatable...)20:57
kata-irc-bot<sebastien.boeuf> not sure, but you need to start kubelet so that it retries overtime and at some point the certs are there, which makes it succeeding20:58
kata-irc-bot<jdandrea> *nodnod*21:02
kata-irc-bot<jdandrea> Chicken-egg-ish. :)21:02
kata-irc-bot<jdandrea> But it does eventually work.21:02
kata-irc-botAction: jdandrea makes note...21:03
kata-irc-bot<sebastien.boeuf> definitely chicken-egg !21:05
kata-irc-bot<sebastien.boeuf> haha21:05
*** libregeekingkid[ has joined #kata-dev21:06
*** PagliaccisCloud has left #kata-dev21:13
kata-irc-bot<jdandrea> How long after kata-deploy.yaml is applied (on avg) "kubectl get nodes --show-labels" shows kata-runtime=true in the LABELS section?21:16
kata-irc-bot<jdandrea> File uploaded https://katacontainers.slack.com/files/UAYMNLRK6/FB6GKAJ5T/-.php / https://slack-files.com/T86U7NQTT-FB6GKAJ5T-6a6e5e8ae221:17
*** fuentess has quit IRC21:20
kata-irc-bot<eric.ernst> it takes a bit due to the large size of the container image being downloaded.21:51
kata-irc-bot<eric.ernst> I hope that it came up in last 30 minutes though.21:51
kata-irc-bot<eric.ernst> hey hackers (@jonolson, @tallclair) - any suggestions on a run once daemonset?  :slightly_smiling_face:22:25
kata-irc-bot<eric.ernst> and other ways to abuse kubernetes.22:26
kata-irc-bot<eric.ernst> Basically, the cleanup preStopHook for my kata-deploy has a hard time (I do systemctl restart <cri-shim> and restart kubelet), so I was going to do this as part of a cleanup daemonset.  But....... I don't want to have to apply and then delete a cleanup, run once daemonset22:27
kata-irc-bot<tallclair> I've wanted support for this before... basically you're looking for a "DaemonJob" (doesn't exist)22:29
kata-irc-bot<eric.ernst> yeah.22:31
kata-irc-bot<eric.ernst> I guess the directions are  apply -f cleanup-daemonset.yaml22:31
kata-irc-bot<eric.ernst> delete -f cleanup-daemonset.yaml22:31
kata-irc-bot<eric.ernst> Yeah, it'd be a nice feature (daemonjob)22:33
kata-irc-bot<tallclair> hmm, so here's a big hack: Do the work in an InitContainer, and make the actualy container "pause". (InitContainer needs to be idempotent, in case of a restart).22:38
kata-irc-bot<tallclair> Then, wait for numberReady == desiredNumberScheduled22:38
kata-irc-bot<tallclair> At that point, the work is completed, and it's safe to delete.22:38
*** cdent has quit IRC22:51
*** lamego1 has quit IRC22:53
*** dklyle has quit IRC22:55
*** dklyle has joined #kata-dev22:56
*** sameo has joined #kata-dev23:08
*** eernst has quit IRC23:16
*** libregeekingkid[ has quit IRC23:16
*** eernst has joined #kata-dev23:22
*** eernst has quit IRC23:25
*** eernst has joined #kata-dev23:25
*** annabelleB has joined #kata-dev23:25
*** eernst has quit IRC23:27
*** eernst has joined #kata-dev23:27
*** eernst_ has joined #kata-dev23:28
*** eernst has quit IRC23:29
*** eernst has joined #kata-dev23:32
*** libregeekingkid[ has joined #kata-dev23:32
*** eernst_ has quit IRC23:33
*** eernst has quit IRC23:36
*** annabelleB has quit IRC23:44
*** fuentess has joined #kata-dev23:47
*** zerocoolback has joined #kata-dev23:58
« Monday, 2018-06-11 Index Wednesday, 2018-06-13 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!