Friday, 2022-06-10

kata-irc-bot<anastassios.nanos> Hi again -- trying to reproduce your issue but I'm afraid we can't. We are getting consistent behavior for kata+fc on 2.1.1-latest main. I think we should step back and check all the moving parts of the worflow (eg. verify an FC VM can boot, check vhost support, check kata-qemu & then try to get logs from the VM execution env).  for instance, can you check if with the following config & command you get kernel logs? `rm05:50
kata-irc-bot/tmp/test.sock fc.sock && /opt/kata/bin/firecracker --config-file config.json --api-sock fc.sock`05:50
kata-irc-bot<anastassios.nanos> 05:50
kata-irc-bot<anastassios.nanos> don't worry if it panics with the rootfs error -- its irrelevant to our current debug attempt. BTW replace `/opt/kata` with the respective dir where your kata installation resides.05:51
kata-irc-bot<norbj.d_kata> thanks for your help and your time once again :pray: firecracker command is indeed giving me an error (with 2.4.2): ```[    1.889951] fuseblk: Unknown parameter 'data' [    1.893026] xfs: Unknown parameter 'data' [    1.896507] fuseblk: Unknown parameter 'data' [    1.899597] xfs: Unknown parameter 'data' [    1.902501] List of all partitions: [    1.904815] 0100            4096 ram0 [    1.904817]  (driver?) [    1.909123] 010112:17
kata-irc-bot           4096 ram1 [    1.909125]  (driver?) [    1.913170] 0102            4096 ram2 [    1.913172]  (driver?) [    1.917127] 0103            4096 ram3 [    1.917253]  (driver?) [    1.921442] 0104            4096 ram4 [    1.921444]  (driver?) [    1.925422] 0105            4096 ram5 [    1.925424]  (driver?) [    1.929460] 0106            4096 ram6 [    1.929462]  (driver?) [    1.933510] 0107            4096 ram7 [    1.933512] 12:17
kata-irc-bot(driver?) [    1.937513] 0108            4096 ram8 [    1.937515]  (driver?) [    1.941468] 0109            4096 ram9 [    1.941469]  (driver?) [    1.945441] 010a            4096 ram10 [    1.945443]  (driver?) [    1.949462] 010b            4096 ram11 [    1.949463]  (driver?) [    1.953494] 010c            4096 ram12 [    1.953496]  (driver?) [    1.957655] 010d            4096 ram13 [    1.957657]  (driver?) [    1.961765] 010e           12:17
kata-irc-bot4096 ram14 [    1.961766]  (driver?) [    1.966165] 010f            4096 ram15 [    1.966167]  (driver?) [    1.970804] fe00          131072 vda [    1.970807]  driver: virtio_blk [    1.975586]   fe01          126976 vda1 291f194e-01 [    1.975589] [    1.979818] No filesystem could mount root, tried: [    1.979820]  ext3 [    1.983005]  ext2 [    1.984315]  ext4 [    1.985653]  fuseblk [    1.987004]  xfs [    1.988490] [    1.990803] Kernel12:17
kata-irc-botpanic - not syncing: VFS: Unable to mount root fs on unknown-block(254,0) [    1.993722] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.15.26 #2 [    1.993722] Call Trace: [    2.000866]  <TASK> [    2.000866]  dump_stack_lvl+0x34/0x44 [    2.000866]  panic+0xe8/0x27a [    2.008852]  mount_block_root+0x1e1/0x1f0 [    2.008852]  prepare_namespace+0x131/0x160 [    2.008852]  ? rest_init+0xc0/0xc0 [    2.008852]  kernel_init+0x11/0x100 [   12:17
kata-irc-bot2.016853]  ret_from_fork+0x22/0x30 [    2.016853]  </TASK> [    2.016853] Kernel Offset: disabled [    2.016853] Rebooting in 1 seconds..``` 12:17
kata-irc-bot<norbj.d_kata> thanks for your help and your time again :pray: it indeed panics with this error: ```[    2.020225] No filesystem could mount root, tried: [    2.020225]  ext3 [    2.023784]  ext2 [    2.025236]  ext4 [    2.026698]  fuseblk [    2.028052]  xfs [    2.029677] [    2.032246] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(254,0) [    2.035064] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.15.26 #2 [   12:20
kata-irc-bot2.042499] Call Trace: [    2.042564]  <TASK> [    2.042564]  dump_stack_lvl+0x34/0x44 [    2.042564]  panic+0xe8/0x27a [    2.050582]  mount_block_root+0x1e1/0x1f0 [    2.050582]  prepare_namespace+0x131/0x160 [    2.050582]  ? rest_init+0xc0/0xc0 [    2.058569]  kernel_init+0x11/0x100 [    2.058569]  ret_from_fork+0x22/0x30 [    2.058569]  </TASK> [    2.066570] Kernel Offset: disabled [    2.066570] Rebooting in 1 seconds..```12:20
kata-irc-bot<norbj.d_kata> is this the rootfs error you were talking about?12:20
kata-irc-bot<anastassios.nanos> yeap, so we have established there's no issue with the firecracker boot or the virtualization support whatsoever -- can you check if you can spawn a container with kata qemu? (with the versions that firecracker does not work on your setup, eg 2.2.0 onwards)14:21
kata-irc-bot<anastassios.nanos> next step I would say is see if we can get a qemu container with rootfs & devmapper14:22
kata-irc-bot<anastassios.nanos> and finally enable all debug in the configuration-fc.toml to see where the issue comes from14:22
kata-irc-bot<norbj.d_kata> kata qemu works fine :slightly_smiling_face: i'll try qemu with devmapper16:04
kata-irc-bot<feng.wang> @linux We rely on cpu and memory for pod scheduling.22:53

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!