| *** pcaruana has quit IRC | 02:54 | |
| *** pcaruana has joined #kata-dev | 03:07 | |
| *** sameo has joined #kata-dev | 06:10 | |
| *** dklyle has quit IRC | 06:53 | |
| *** jodh has joined #kata-dev | 07:56 | |
| *** sgarzare has joined #kata-dev | 08:53 | |
| *** davidgiluk has joined #kata-dev | 09:03 | |
| *** gwhaley has joined #kata-dev | 09:05 | |
| *** amorenoz has quit IRC | 09:16 | |
| *** amorenoz has joined #kata-dev | 09:26 | |
| *** jugs has quit IRC | 11:04 | |
| *** jugs has joined #kata-dev | 11:04 | |
| *** errordeveloper has joined #kata-dev | 11:10 | |
| errordeveloper | could some one have quick look at https://github.com/kata-containers/runtime/issues/2564? | 11:21 |
|---|---|---|
| gwhaley | errordeveloper - a quick check - can you run 'kata-runtime kata-env', and post the results on the Issue. That will get kata to spit out details about which files it thinks it is looking for from where etc. | 11:32 |
| gwhaley | oh, doh, maybe it is already there in the kata-collect data - sorry if I duped... | 11:33 |
| errordeveloper | gwhaley: yeah, it is :) | 11:34 |
| errordeveloper | also good to know someone is around :) | 11:34 |
| gwhaley | I also wonder - are you maybe confusing the containerd config file with the kata runtime config file? | 11:35 |
| gwhaley | errordeveloper - most folks who can help either just went to bed (china) or are not quite up yet (USA) - so, you will likely get more responses later today | 11:36 |
| errordeveloper | gwhaley: nope, I don't think so :) | 11:36 |
| errordeveloper | I shared my containerd config there | 11:36 |
| errordeveloper | ok, it's in the details section | 11:36 |
| errordeveloper | but also here https://github.com/errordeveloper/kata-fc-eks/blob/e6ee6fc523228fd07a5770868db39928c9a5d4fc/cluster.yaml#L53 | 11:37 |
| errordeveloper | I also had question about networking, is there a doc I can read that details how CNI meant to work with kata? | 11:38 |
| errordeveloper | gwhaley: any thoughts so far? shall I try using new containerd config syntax? | 11:45 |
| errordeveloper | see https://github.com/containerd/cri/blob/master/docs/config.md | 11:45 |
| gwhaley | errordeveloper: thx, just peeking at that config file. I was going to run up a local kata-deploy and show what it generates in the config files for containerd. You can see the script that does the edits at https://github.com/kata-containers/packaging/blob/master/kata-deploy/scripts/kata-deploy.sh | 11:45 |
| errordeveloper | gwhaley: I'll check kata-deloy.sh, thanks! | 11:46 |
| gwhaley | iirc, what kata-deploy does is replace 'kata-runtime' with a shell script that appends the correct config file onto the end - basically a re-direct on execution. I'm not sure if you can set the kata config file path in the containerd yaml section ... | 11:46 |
| gwhaley | It'll be a couple of hours maybe before I can reproduce and check... | 11:47 |
| errordeveloper | according to the docs, I should be able to set do this: | 11:47 |
| errordeveloper | [plugins.cri.containerd.runtimes.kata] | 11:47 |
| errordeveloper | runtime_type = "io.containerd.kata.v2" | 11:47 |
| errordeveloper | [plugins.cri.containerd.runtimes.kata.options] | 11:47 |
| errordeveloper | ConfigPath = "/etc/kata-containers/config.toml | 11:47 |
| errordeveloper | (that's from https://github.com/kata-containers/documentation/blob/master/how-to/containerd-kata.md) | 11:48 |
| errordeveloper | there is also a suggestion to use a shell script, but that's only meant to be applied for older versions of kata and containerd | 11:48 |
| gwhaley | errordeveloper: thx. let's put that link on the Issue if not already. Just for more reference, our CI sets up containerd with this snippet: https://github.com/kata-containers/tests/blob/master/.ci/configure_containerd_for_kata.sh | 11:53 |
| gwhaley | I wonder if our CI or code tests setting up the kata config path via containerd setup - @fuentess, do you know? We probably should... @lifupan as well I think for shimv2 input | 11:54 |
| errordeveloper | ok, so I've been able to move on a litte | 12:09 |
| errordeveloper | let me update my script to show what I did | 12:10 |
| *** crobinso has joined #kata-dev | 12:13 | |
| errordeveloper | gwhaley: I've added a comment to the issue about progress so far :) | 12:29 |
| *** devimc has joined #kata-dev | 13:26 | |
| *** sameo_ has joined #kata-dev | 14:28 | |
| *** sameo has quit IRC | 14:30 | |
| *** dklyle has joined #kata-dev | 14:53 | |
| *** devimc has quit IRC | 14:55 | |
| *** devimc has joined #kata-dev | 15:45 | |
| *** crobinso has quit IRC | 16:21 | |
| *** jugs has quit IRC | 17:10 | |
| *** sgarzare has quit IRC | 17:11 | |
| *** jugs has joined #kata-dev | 17:13 | |
| *** openstack has quit IRC | 17:49 | |
| *** openstack has joined #kata-dev | 17:49 | |
| *** ChanServ sets mode: +o openstack | 17:49 | |
| *** gwhaley has quit IRC | 18:00 | |
| *** jodh has quit IRC | 18:04 | |
| *** errordeveloper has quit IRC | 18:32 | |
| *** errordeveloper has joined #kata-dev | 18:57 | |
| *** errordeveloper has quit IRC | 19:02 | |
| fidencio | interesting, I'm taking a look at dwalsh's patch for SELinux and I noticed something quite weird | 19:19 |
| fidencio | in (sandbox) startVM, s.config.HypervisorConfig.ProcessLabel is set with the proper value ... a few lines below that s.hypervisor.startSandbox(), which in my case is (qemu) startSandbox(), and there q.config.ProcessLabel is not set | 19:21 |
| fidencio | is there any specific reason that may cause this to happen? | 19:21 |
| fidencio | hmmm. debugging show s.config.HypervisorConfig (from sandbox) and q.config (from qemu) are totally different pointers | 19:32 |
| *** errordeveloper has joined #kata-dev | 19:59 | |
| *** errordeveloper has quit IRC | 20:04 | |
| *** davidgiluk has quit IRC | 20:20 | |
| *** devimc has quit IRC | 20:27 | |
| *** devimc has joined #kata-dev | 20:28 | |
| *** crobinso has joined #kata-dev | 20:33 | |
| *** errordeveloper has joined #kata-dev | 20:59 | |
| *** errordeveloper has quit IRC | 21:04 | |
| *** errordeveloper has joined #kata-dev | 21:22 | |
| *** errordeveloper has quit IRC | 21:26 | |
| kata-irc-bot | <crobinso> is there a consensus preferred option between `initrd=` and `image=`for the mini appliance OS, if initrd/image contents are the same? is one config expected to give smaller memory overhead, boot faster, or something objective like that? | 21:37 |
| devimc | crobinso: I prefer to use the rootfs image | 22:03 |
| devimc | better boot time and mem footprint | 22:03 |
| devimc | + agent as init = even better boot time and mem footprint | 22:04 |
| devimc | @crobinso | 22:04 |
| *** devimc has quit IRC | 22:04 | |
| kata-irc-bot | <crobinso> interesting. this wiki page says initrd should be fastest boot but I didn't attempt to confirm. maybe page is out of date? https://github.com/kata-containers/documentation/wiki/Kata-images | 22:05 |
| kata-irc-bot | <crobinso> @julio.montes I figured image= would give smaller memory footprint but in my testing it quite reproducibly did not. I did a super basic test though. this was fedora 31 kata packages and qemu and initrd+image, but using clearlinux kernel. | 22:07 |
| kata-irc-bot | <crobinso> but the expectation is that image= should have smaller memory footprint? | 22:07 |
| kata-irc-bot | <crobinso> partly why I'm asking is that supporting `image=` with fedora/rhel distro kernel is difficult. needs pieces compiled in that are currently modularized. on rhel/centos most problematic is a usable filesystem, which are all modules. on Fedora at least ext4 is built in so we don't have that problem, but both are missing nvdimm pieces. some discussion about those here FWIW: https://bugzilla.redhat.com/show_bug.cgi?id=1750581 | 22:09 |
| openstack | bugzilla.redhat.com bug 1750581 in kernel "enable nvdimm drivers for kata-containers usage?" [Unspecified,New] - Assigned to kernel-maint | 22:09 |
| kata-irc-bot | <salvador.fuentes> @crobinso I think devimc wanted to say initrd, which is the one that runs agent as init | 22:11 |
| *** sameo_ has quit IRC | 22:11 | |
| kata-irc-bot | <crobinso> @salvador.fuentes can't image= and initrd= both run agent as init? | 22:12 |
| kata-irc-bot | <salvador.fuentes> @jose.carlos.venegas.m may also have input ^ | 22:12 |
| kata-irc-bot | <crobinso> (i've only experimented with systemd init for both cases) | 22:13 |
| kata-irc-bot | <salvador.fuentes> I think both can, but currently image has systemd by default | 22:13 |
| kata-irc-bot | <crobinso> I see, this is in reference to the initrd and image that upstream kata packages ship. | 22:14 |
| kata-irc-bot | <fidencio> @crobinso, just for your information, I've done some tests using initrd + kata agent as init | 22:15 |
| kata-irc-bot | <fidencio> it does work | 22:15 |
| kata-irc-bot | <crobinso> @fidencio cool. I would like to explore that more too | 22:16 |
| kata-irc-bot | <crobinso> my goal is to see if we can rule out ever really needing to care about image= in Fedora land. the path to get it working at all is unclear, but if there's not any particular benefit (and maybe if it's even a bit worse than initrd), then we can just stop caring about generating it entirely, which simplifies quite a few things | 22:17 |
| kata-irc-bot | <crobinso> other possible benefits of only using initrd: possibly future minified qemu could drop the nvdimm device, which reduces attack surface. same with guest kernel. those are likely very small wins though | 22:18 |
| kata-irc-bot | <fidencio> Sincerely? I do believe rhbz#1750581 is a blocker that won't be solved anytime soon | 22:18 |
| kata-irc-bot | <fidencio> I'd just drop image= usage / generation on FEdora | 22:18 |
| kata-irc-bot | <crobinso> @fidencio I could probably push to get that fixed in Fedora. but RHEL is another matter. it would just put me at ease to know we aren't closing the door on something that's considered optimal :slightly_smiling_face: | 22:19 |
| kata-irc-bot | <crobinso> something being `image=` | 22:19 |
| kata-irc-bot | <jose.carlos.venegas.m> @crobinso yes, both image and initrd support agent as init | 22:19 |
| kata-irc-bot | <fidencio> About using agent as init, it was as simple to configure as passing a parameter to the scripts we have as part of the osbuilder Fedora package. I guess you're familiar with those. :slightly_smiling_face: | 22:20 |
| kata-irc-bot | <jose.carlos.venegas.m> @crobinso so what are the current blockers • Depend on kernel modules added in the disk/initrd • initrd vs image what is the way you are looking for integration | 22:20 |
| kata-irc-bot | <crobinso> @jose.carlos.venegas.m I might have misunderstood you, but the blocker for using `image=` with Fedora/RHEL kernels is that some kernel modules need to be de-modularized (changed to be built into the kernel itself) or the nvdimm device can't be used as the guest root fs. that's the primary blocker on our side | 22:22 |
| kata-irc-bot | <jose.carlos.venegas.m> Got it so initrd would be a way to go given that first blocker | 22:23 |
| kata-irc-bot | <crobinso> @jose.carlos.venegas.m but the question I'm trying to answer: is there any particular benefit to `image=` vs `initrd=` if the contents are the same (both using AGENT_INIT=yes for example)? because if there isn't, then we can just ignore `image=` entirely and side step the blockers | 22:23 |
| kata-irc-bot | <crobinso> @jose.carlos.venegas.m right, `initrd` is what we are currently using, and it works great. I'm trying to figure out if we even care about image= or not | 22:24 |
| kata-irc-bot | <jose.carlos.venegas.m> I see, well for image as boot rootfs the other bennefit you get is memory usage because of DAX, if that is not really relevant today for you you can keep on that way | 22:25 |
| kata-irc-bot | <jose.carlos.venegas.m> For snap packages for ubuntu we use initrd as far as I know \cc @julio.montes | 22:27 |
| kata-irc-bot | <crobinso> @jose.carlos.venegas.m ah, so if image= is used with DAX, then that can yield less memory footprint than initrd=? maybe I wasn't using DAX in my image= testing | 22:27 |
| kata-irc-bot | <jose.carlos.venegas.m> That is a good question, I remember @julio.montes took some metrics some time ago comparing DAX but just comparing `image=` (one virtual `.img` that had NVDIMM headers and other that did not have it) | 22:29 |
| kata-irc-bot | <jose.carlos.venegas.m> but `image=as nvdimm device` vs `initrd=` is something I am not aware we have compared | 22:30 |
| kata-irc-bot | <jose.carlos.venegas.m> @crobinso I think for integration even is better to use initrd over image because `image` cretion requires create loopdevices/mount that would require root and potencially a dirty env if the process fails | 22:34 |
| kata-irc-bot | <jose.carlos.venegas.m> or not really? | 22:34 |
| kata-irc-bot | <crobinso> @jose.carlos.venegas.m yes the image generation process is certainly more complicated. so far in Fedora we are generating the initrd+image on host bootup with a systemd service, and on demand at RPM install time. dropping image= would simplify runtime deps for that quite a bit | 22:36 |
| kata-irc-bot | <crobinso> okay thanks for the info everyone this definitely clarified some things. I think I will propose dropping image= generating in Fedora though. if we ever get to the point of wanting to really micro-optimize memory+io requirements we can revisit it, but at this point simplicity is more valuable | 22:37 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!