Tuesday, 2019-11-26

*** igordc has quit IRC01:13
*** auk has joined #kata-general02:16
*** auk has quit IRC06:50
*** sgarzare has joined #kata-general08:09
*** sameo has joined #kata-general08:15
*** sgarzare has quit IRC08:22
*** sgarzare has joined #kata-general08:32
*** gwhaley has joined #kata-general09:03
*** sgarzare has quit IRC10:02
*** sgarzare has joined #kata-general10:04
*** sameo has quit IRC10:42
*** sameo has joined #kata-general10:42
*** lpetrut has joined #kata-general10:52
*** gwhaley has quit IRC12:02
*** gwhaley has joined #kata-general13:10
*** sgarzare has quit IRC16:35
*** sameo has quit IRC16:51
*** igordc has joined #kata-general17:20
*** sameo has joined #kata-general17:54
*** sameo has quit IRC18:28
*** CeeMac has quit IRC18:57
*** lpetrut has quit IRC19:08
kata-irc-bot3<archana.m.shinde> zer0def just seeing this, I have not been able to replicate this, so would like to know details about your setup20:18
zer0def@archana.m.shinde arch-based, linux-hardened 5.3.13, qemu 4.1.1, let me quickly generate a log20:22
kata-irc-bot3<archana.m.shinde> thanks zer0def20:25
kata-irc-bot3<archana.m.shinde> what version of podman ? distro provided packages?20:25
zer0defhappens both on 1.6.3 and git master20:26
kata-irc-bot3<archana.m.shinde> ok20:26
zer0defgimme a second, i've apparently mis-symlinked a local build i was working on to cleanup the aur pkgbuild20:27
kata-irc-bot3<archana.m.shinde> sure, take your time20:27
kata-irc-bot3<archana.m.shinde> zer0def stepping away for a bit, would also appreciate if you could attach output of ` sudo kata-collect-data.sh`20:32
*** gwhaley has quit IRC20:34
zer0def@archana.m.shinde https://dpaste.de/8k5h/raw and https://dpaste.de/jZ8i/raw respectively20:48
zer0defto be fair, the archlinux repack is straight from f30 with just a few paths tweaks: https://raw.githubusercontent.com/zer0def/PKGBUILDs/kata-containers/PKGBUILD20:51
kata-irc-bot3<archana.m.shinde> zer0def thanks. Can you send me the kata journal logs as well ?21:32
kata-irc-bot3<archana.m.shinde> you will need to  ``` sudo sed -i -e 's/^# *\(enable_debug\).*=.*$/\1 = true/g' /usr/share/defaults/kata-containers/configuration.toml  sudo journactl | grep kata-runtime```21:33
*** igordc has quit IRC21:33
zer0def@archana.m.shinde from just now: https://dpaste.de/oChO/raw21:38
zer0defnothing that the command execution log doesn't have21:38
*** igordc has joined #kata-general21:39
zer0defhnh… compared to a rooted podman container run, for rootless conmon returns -1 for the PID, which is obviously wrong21:43
kata-irc-bot3<archana.m.shinde> zer0def  yeah, the main issue is `mkdir /var/run/netns: permission denied`21:46
kata-irc-bot3<archana.m.shinde> will have to see where and why that was invoked21:46
kata-irc-bot3<archana.m.shinde> zer0def  was this with host networking, whats the podman command that you ran?21:51
zer0def`podman --log-level debug run --runtime kata --net none -ti alpine:edge /bin/sh`21:53
kata-irc-bot3<archana.m.shinde> can you try without `--net none`21:53
kata-irc-bot3<archana.m.shinde> and give me the kata logs21:53
zer0defuh, read again? :)21:53
kata-irc-bot3<archana.m.shinde> so, `podman --log-level debug run --runtime kata -ti alpine:edge /bin/sh`21:54
zer0defthis error actually turns into: `time="2019-11-26T22:51:54+01:00" level=debug msg="ExitCode msg: \"open /var/run/netns/cni-aeeccd84-64b1-2ed6-7187-45a72fb63e37: permission denied: oci runtime permission denied error\""` once /var/run/netns is created by, for example, creating a rooted container21:54
zer0defwithout `--net none` i personally consistently get: `time="2019-11-26T22:54:46+01:00" level=debug msg="ExitCode msg: \"rpc error: code = unknown desc = open /run/kata-containers/sandbox/resolv.conf: no such file or directory: oci runtime command not found error\""`21:55
kata-irc-bot3<archana.m.shinde> oh ok21:56
kata-irc-bot3<archana.m.shinde> can I ask you to paste the entire sequence of logs like you did earlier21:56
kata-irc-bot3<archana.m.shinde> I want to see how far it gets21:56
zer0deffor `--net none` you're probably looking at https://github.com/containers/libpod/blob/master/libpod/networking_linux.go#L39521:57
zer0defsure looks like a permissions issue, let me just slap on go+w on it, see what happens21:58
kata-irc-bot3<archana.m.shinde> yeah, I dont want to look into --net=none case right now21:58
kata-irc-bot3<archana.m.shinde> I am interested in the one without it21:59
zer0defyou don't have to, errors out the same way with /run/kata-containers/sandbox/resolv.conf21:59
zer0defactually, sorry, I confused the commands - once I make /var/run/netns go+w, the container starts22:01
zer0def`podman --log-level debug run --runtime kata -ti alpine:edge /bin/sh` → https://dpaste.de/bHiu/raw22:02
zer0defto clear up: when i run *with* `--net none` *and* 0777 on /var/run/netns, the container starts22:04
kata-irc-bot3<archana.m.shinde> ok22:05
kata-irc-bot3<archana.m.shinde> that seems to be a bug on the podman side for --net=none22:06
zer0defit does, will bring it up there later22:06
*** MartinXu41 has joined #kata-general22:23
zer0defjournal log for `podman --log-level debug run --runtime kata -ti alpine:edge /bin/sh`: https://dpaste.de/4jsT/raw I'm sort of curious about the warning for devices.json on line 1922:25
zer0defactually, nevermind.22:26
kata-irc-bot3<archana.m.shinde> yeah thats a benign warning22:29
MartinXu41@all looks like the Slack link does not work. Some one can tell me whether Slack is still avaiable for Kata?  thanks!22:30
kata-irc-bot3<archana.m.shinde> it looks like it was able to start the VM successfully, error seems to be while cleanup22:30
kata-irc-bot3<archana.m.shinde> nvm22:32
zer0deflooks more like line 45 is causing the issue, which means it's an error bubbling up from the agent… and @james.o.hunt just trips on line 43 instead22:32
zer0defpossibly s/43/41/22:34
kata-irc-bot3<archana.m.shinde> could be22:43
kata-irc-bot3<archana.m.shinde> seems there are some missing logs from the agent itself22:43
kata-irc-bot3<archana.m.shinde> `sudo journalctl | grep kata` may provide something useful22:44
zer0defhttps://dpaste.de/HNzx/raw starting from line 105 upwards22:50
zer0defspecifically the "&Empty{XXX_unrecognized:[],}" response22:50
zer0deffunny thing, running `sudo podman --log-level debug run **--shm-size 0** --runtime kata -ti alpine:edge /bin/sh` achieves the same result, but that's rather obvious from the log22:57
zer0defok, that response I've drawn attention to doesn't differ from a successful rooted launch, however the rooted container's agent starts correctly (as long as there's an shm), compared to rootless23:02
zer0defmeanwhile, I'm clocking out, let me know how else I could be of help, will get to it the first opportunity I'll get23:06
kata-irc-bot3<archana.m.shinde> sure zer0def, will let you know23:09
kata-irc-bot3<archana.m.shinde> thanks for your help23:09
*** MartinXu41 has quit IRC23:12

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!