| ianw | cgoncalves / johnsom : YES!!! i can replicate this slowdown with a systemd upgrade on a stock image | 00:49 |
|---|---|---|
| johnsom | ianw Hmm, so updating the version of systemd is what tanks that startup time? | 00:49 |
| ianw | johnsom: updating systemd & regenerating the initramfs accounts for what seems to be a fairly large chunk | 00:50 |
| ianw | johnsom: pulling apart the two initramfs' ... it seems there's some new udev rules making it in | 00:51 |
| johnsom | ianw Kudos for your patience figuring this out.... | 00:51 |
| ianw | not quite there yet, but i have a narrower target now | 00:51 |
| ianw | going to take a brain break for a bit, but will update https://etherpad.openstack.org/p/dib-centos-slow with the details | 00:52 |
| *** mjturek has joined #openstack-dib | 01:37 | |
| *** mjturek has quit IRC | 01:43 | |
| *** hwoarang has quit IRC | 02:33 | |
| *** hwoarang has joined #openstack-dib | 02:34 | |
| *** hwoarang has quit IRC | 03:15 | |
| *** hwoarang has joined #openstack-dib | 03:16 | |
| *** hwoarang has quit IRC | 03:34 | |
| *** hwoarang has joined #openstack-dib | 03:35 | |
| ianw | :/ well, reverting the minor change inthe initrd doesn't seem to change things | 05:09 |
| ianw | i'm just going to replicate it again, i have enough for a decent bug report at least now | 05:09 |
| *** hwoarang has quit IRC | 06:09 | |
| *** hwoarang has joined #openstack-dib | 06:15 | |
| cgoncalves | ianw, awesome!!! thanks a lot | 07:33 |
| cgoncalves | johnsom, you see?! aren't you one of those that complain that EL always ship last-century software versions? now you get why :P | 07:34 |
| ianw | cgoncalves: i'm just writing it up now ... i didn't have much luck with systemd deubgging output at this point | 07:34 |
| cgoncalves | ianw, one thing though, why even with newer cloud image it takes more time to boot from a DIB-built image? systemd version is the same. is it iscsi? | 07:36 |
| ianw | cgoncalves: even the latest cloud image odesn't quite have the latest systemd | 07:37 |
| ianw | while the dib build images do. this seems to be the difference | 07:37 |
| cgoncalves | ooh! | 07:37 |
| *** hwoarang has quit IRC | 07:39 | |
| *** hwoarang has joined #openstack-dib | 07:39 | |
| ianw | ARRRRRGGGGHHHHHHHHHH!!!!!!!! | 07:49 |
| ianw | I just clicked "report issue" instead of "submit issue" and it wiped the whole thing out | 07:49 |
| ianw | cgoncalves: ok, attempt 2 -> https://bugs.centos.org/view.php?id=15735 | 08:05 |
| ianw | cgoncalves: so what i was attempting was to boot with "systemd.log_level=debug systemd.log_target=console" to try and see if something useful comes up there | 08:07 |
| ianw | i was getting output, but then the boot seemed to hang, and frankly i gave up (for now :) | 08:07 |
| ianw | but i think that's going to be necessary. as noted in the diff between the initramfs's, i don't think it's actually in there | 08:08 |
| ianw | somehow i suspect systemd is scheduling things differently; perhaps doing things sequentially it was otherwise running in parallel? | 08:08 |
| cgoncalves | ianw, thank you a zillion! I will be tracking it closely. a fix might take time to land, I believe, as it would need to go up to RHEL first | 08:11 |
| cgoncalves | ianw, I compared systemd services between upstream vs dib and didn't find systemd to be scheduling differently, no | 08:13 |
| ianw | cgoncalves: yeah, i saw the svg files ... but that wasn't accounting for the initrd right? | 08:13 |
| ianw | that was just a much longer black box | 08:13 |
| cgoncalves | ianw, right. svg outputted by systemd-analyze | 08:14 |
| ianw | yeah, i don't know how to get that same info from *inside* the initramfs? | 08:18 |
| ianw | before it switches | 08:21 |
| cgoncalves | ianw, doable with bootchart2, it seems: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741593 | 08:24 |
| openstack | Debian bug 741593 in bootchart2 "Support profiling of ramdisk (a.k.a. initramfs, initrd)" [Wishlist,Open] | 08:24 |
| ianw | interesting, something like that will be necessary i think | 08:27 |
| *** AndreasF has joined #openstack-dib | 10:16 | |
| *** AndreasF has quit IRC | 10:18 | |
| *** AndreasF has joined #openstack-dib | 10:19 | |
| *** hwoarang has quit IRC | 10:36 | |
| *** hwoarang has joined #openstack-dib | 10:36 | |
| *** mjturek has joined #openstack-dib | 14:29 | |
| *** jabadia has joined #openstack-dib | 18:53 | |
| *** jabadia has quit IRC | 19:39 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!