| ianw | pabelanger: ok ... i seem to get a completely silent hang presumably when it's hitting the initrd | 00:04 |
|---|---|---|
| ianw | booting with initrd commented out fails with no root disk, so that's something | 00:04 |
| pabelanger | ianw: something indeed | 00:22 |
| ianw | ok, i can get into an emergency console | 00:28 |
| ianw | i'm wondering if it can't see the root disk | 00:28 |
| ianw | there's no sda in /dev | 00:28 |
| ianw | oh, no, vda i guess | 00:28 |
| pabelanger | ianw: nice | 00:30 |
| ianw | mount /dev/vda1 ... "unknown file system type ext4" | 00:31 |
| ianw | that can't be good | 00:31 |
| ianw | ok, compared to an initramfs for existing boots, we're missing a ton of modules, including ext4 | 00:42 |
| pabelanger | bad build? | 00:45 |
| greghaynes | haha | 00:55 |
| greghaynes | sounds like the dracut failed hard | 00:55 |
| greghaynes | there may be some info in the build logs about it, or we may need to make dracut more verbose | 00:56 |
| ianw | dracut runs i guess just as part of the rpm post install | 00:56 |
| greghaynes | sounds right | 00:57 |
| greghaynes | I cant remember if we do an explicit initramfs-update or w/e the command is | 00:57 |
| ianw | ok, after regenerating it, guess what ... all modules seem to be there | 01:06 |
| greghaynes | hah | 01:07 |
| greghaynes | so I wonder if theres a package ordering problem or somesuch | 01:07 |
| greghaynes | like maybe we install whatever runs dracut before the kernel modules | 01:07 |
| ianw | well the kernel modules are part of the kernel package | 01:08 |
| greghaynes | hrm | 01:08 |
| greghaynes | although, we install the kernel package from outside the chroot I think | 01:08 |
| greghaynes | right? | 01:08 |
| ianw | no | 01:17 |
| ianw | i wonder if we're getting tight on disk during the build | 01:17 |
| ianw | well, somewhat reproducable, i can make dracut generate a initrd without ext4 | 01:42 |
| ianw | why .. that's another question :) | 01:42 |
| ianw | ok, i think i know what's going on ... dracut looks for fstab and without it, at the time we install the kernel, it doesn't probe / and decide it wants to include ext4 modules | 02:27 |
| *** jaianshu has joined #openstack-dib | 04:09 | |
| *** jaianshu_ has joined #openstack-dib | 04:09 | |
| *** vmud213 has joined #openstack-dib | 04:15 | |
| *** chhavi has joined #openstack-dib | 04:31 | |
| *** jaianshu__ has joined #openstack-dib | 05:05 | |
| *** jaianshu_ has quit IRC | 05:08 | |
| *** jaianshu has quit IRC | 05:08 | |
| *** jaianshu has joined #openstack-dib | 05:08 | |
| *** noama has quit IRC | 05:28 | |
| *** noama has joined #openstack-dib | 05:28 | |
| *** vmud213 has quit IRC | 05:38 | |
| *** vmud213 has joined #openstack-dib | 05:40 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Recreate initramfs within loopback image https://review.openstack.org/411130 | 06:22 |
| ianw | greghaynes pabelanger: ^ so i get a bootable image with that | 06:42 |
| ianw | i haven't got a change to test it will a full infra build; pabelanger if you get time to try building & booting that would be awesome | 06:42 |
| ianw | i'm gonna think on it a bit ... i'd love to root cause it as to why it worked before. it actually seems like it shouldn't | 06:43 |
| *** vmud213 has quit IRC | 06:46 | |
| *** vmud213 has joined #openstack-dib | 06:46 | |
| *** vmud213 has quit IRC | 07:01 | |
| *** jaianshu__ has quit IRC | 07:24 | |
| *** jaianshu has quit IRC | 07:24 | |
| *** zhugaoxiao has quit IRC | 07:39 | |
| *** hashar has joined #openstack-dib | 07:39 | |
| *** zhugaoxiao has joined #openstack-dib | 07:39 | |
| *** hashar has quit IRC | 08:04 | |
| *** vmud213 has joined #openstack-dib | 08:14 | |
| *** vmud213 has quit IRC | 08:50 | |
| *** vmud213 has joined #openstack-dib | 08:52 | |
| *** chhavi has quit IRC | 09:30 | |
| *** hashar has joined #openstack-dib | 09:35 | |
| *** zhugaoxiao has quit IRC | 09:50 | |
| *** zhugaoxiao has joined #openstack-dib | 09:51 | |
| *** chhavi has joined #openstack-dib | 09:52 | |
| *** openstackgerrit has quit IRC | 10:18 | |
| *** openstackgerrit has joined #openstack-dib | 10:28 | |
| openstackgerrit | Andrey Shestakov proposed openstack/diskimage-builder: Fix dhcp-all-interfaces for ubuntu-minimal xenial https://review.openstack.org/407725 | 10:28 |
| *** evrardjp has quit IRC | 12:08 | |
| *** evrardjp has joined #openstack-dib | 12:13 | |
| *** owalsh is now known as owalsh-afk | 12:28 | |
| *** sudipto has joined #openstack-dib | 13:37 | |
| *** sudipto_ has joined #openstack-dib | 13:37 | |
| *** vmud213 has quit IRC | 13:39 | |
| pabelanger | ianw: greghaynes: of course, I need to shave the yak first to see why I can't even build the centos-7 image now using that review | 13:59 |
| pabelanger | /bin/bash: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8) | 14:06 |
| pabelanger | did we update locales again for centos? | 14:06 |
| *** chhavi has quit IRC | 14:52 | |
| pabelanger | greghaynes: ianw: so, after much raging, I found the locale issue. | 15:33 |
| pabelanger | I had LANG=C for my build host | 15:34 |
| pabelanger | which is passed into the chrrot | 15:34 |
| pabelanger | chroo* | 15:34 |
| pabelanger | shockingly we don't clear all variables for the chroot | 15:35 |
| pabelanger | http://www.linuxfromscratch.org/lfs/view/6.4/chapter06/chroot.html | 15:35 |
| pabelanger | going to hack on that adding, env -i and see what breaks in a bit | 15:35 |
| pabelanger | after I finally test 411130 | 15:35 |
| *** owalsh-afk is now known as owalsj | 15:38 | |
| *** owalsj is now known as owalsh | 15:38 | |
| pabelanger | ianw: did not work: http://paste.openstack.org/show/592515/ | 16:20 |
| pabelanger | entered emergency mode | 16:20 |
| *** hashar has quit IRC | 16:35 | |
| *** hashar has joined #openstack-dib | 16:38 | |
| *** hashar has quit IRC | 16:42 | |
| cinerama | hey folks | 17:10 |
| greghaynes | cinerama: hey! | 17:24 |
| greghaynes | wb | 17:24 |
| cinerama | thanks! been traveling a bit and then trapped in many meetings | 17:24 |
| greghaynes | sounds like fun :) | 17:25 |
| cinerama | traveling yes, meetings...no comment :) | 17:27 |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder: Recreate initramfs within loopback image https://review.openstack.org/411130 | 17:37 |
| pabelanger | ianw: greghaynes: ^ first issue, naming it .sh doesn't actually make it run in finalise.d, for what ever reason | 17:37 |
| pabelanger | rebuilding image now that code ran | 17:38 |
| greghaynes | huh | 17:49 |
| greghaynes | I thought dib only would care if its set +x | 17:49 |
| greghaynes | its been a while since I looked at that code though | 17:49 |
| pabelanger | not sure, no time to shave that yak | 17:53 |
| greghaynes | wow so the big question I have is - is this a change in dracut behavior? | 17:53 |
| pabelanger | once I confirm the fix for 411130, I'll poke at it | 17:53 |
| greghaynes | where it used to just load in a bunch of modules | 17:53 |
| greghaynes | sounds like maybe we werent running in hostonly mode before | 17:55 |
| pabelanger | ya, not sure | 17:55 |
| greghaynes | any idea where the code that actually runs dracut is? I assume in some rpm source | 17:56 |
| greghaynes | would be nice to check the commit log there | 17:56 |
| pabelanger | not sure, ianw did the heavy lifting last night | 17:56 |
| pabelanger | but once I confirm the fix, will help dig into why it broke | 17:57 |
| greghaynes | awesome, thanks | 17:57 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Create ubuntu/fedora test for pip-and-virtualenv https://review.openstack.org/407769 | 18:00 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: DNM: Fixups for block device patch https://review.openstack.org/411462 | 18:58 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add multi partition func test https://review.openstack.org/383055 | 18:59 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder: Add multi partition func test https://review.openstack.org/383055 | 19:02 |
| openstackgerrit | Merged openstack/diskimage-builder: Recreate initramfs within loopback image https://review.openstack.org/411130 | 19:10 |
| *** sudipto_ has quit IRC | 19:36 | |
| *** sudipto has quit IRC | 19:36 | |
| *** sudipto has joined #openstack-dib | 19:41 | |
| *** sudipto has quit IRC | 19:49 | |
| ianw | pabelanger: ahh, cool | 20:17 |
| ianw | sorry, my testing consisted of manually stopping it and doing that call in a bash shell. i just put it into a script for the review | 20:17 |
| ianw | greghaynes: i'll examine the logs a little more in a bit; i think it's also running in "host only mode" in the bad version too | 20:18 |
| greghaynes | ianw: I think thats the issue, I assume that beforehand it was not running in host-only-mode | 20:47 |
| ianw | greghaynes: i wonder if dumping a config file might be able to make it run in generic mode always. that seems a lot safer, at the cost of some size i guess | 20:59 |
| greghaynes | yep, exactly | 21:01 |
| pabelanger | greghaynes: ianw: any reason not to gzip the tarball? was thinking of adding support for tgz output, so I can get a gzip + checksum all together | 21:13 |
| greghaynes | pabelanger: that sounds fine to me | 21:14 |
| pabelanger | k, will poke at it | 21:14 |
| ianw | what tarball? | 21:25 |
| greghaynes | ianw: I think -t tar | 21:25 |
| ianw | ohh, ok | 21:25 |
| pabelanger | ya | 21:27 |
| pabelanger | want to add both tar and tgz | 21:27 |
| pabelanger | going head deep into this container thing | 21:28 |
| greghaynes | pabelanger: last time I did that I came out with 'we really should make a root.d cache' | 21:29 |
| greghaynes | and then didnt really finish the idea | 21:29 |
| pabelanger | ya, that would speed up things a little | 21:30 |
| greghaynes | heh, in my testing it was a ton | 21:30 |
| pabelanger | I'm doing a talk in devconf.cz about dib + ansible. Hope to show people it's a better way to do builds, without touching a Dockerfile | 21:33 |
| greghaynes | \O/ | 21:33 |
| ianw | greghaynes: so dracut has shipped with a conf file that puts it in "hostonly" mode for a long time | 22:48 |
| ianw | we previously must have been kicking it *out* of hostonly mode somehow | 22:48 |
| greghaynes | hrm | 22:52 |
| greghaynes | I wonder if theres something in the rpm that might pass the non hostonly arg? | 22:52 |
| ianw | i think i see it | 23:05 |
| ianw | i think it's a bug in findmnt | 23:05 |
| ianw | if anyone needs me, i'll just be over here going down this rabbit hole :) | 23:06 |
| greghaynes | haha, gl | 23:06 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!