| *** rosmaita has joined #openstack-infra | 00:01 | |
| *** sgw has joined #openstack-infra | 00:02 | |
| *** prometheanfire has quit IRC | 00:29 | |
| *** prometheanfire has joined #openstack-infra | 00:35 | |
| *** pkopec has quit IRC | 00:58 | |
| *** yamamoto has joined #openstack-infra | 01:01 | |
| *** rfolco has quit IRC | 01:29 | |
| *** ysandeep|away is now known as ysandeep|rover | 01:44 | |
| *** dchen has quit IRC | 01:50 | |
| *** dchen has joined #openstack-infra | 01:52 | |
| *** yamamoto has quit IRC | 02:03 | |
| *** yamamoto has joined #openstack-infra | 02:16 | |
| *** ociuhandu has joined #openstack-infra | 02:23 | |
| *** ysandeep|rover is now known as ysandeep|rover|b | 02:36 | |
| *** andrewbonney has quit IRC | 02:44 | |
| *** yamamoto has quit IRC | 03:02 | |
| *** yamamoto has joined #openstack-infra | 03:04 | |
| *** yamamoto has quit IRC | 03:04 | |
| *** yamamoto has joined #openstack-infra | 03:05 | |
| *** yamamoto has quit IRC | 03:09 | |
| *** yamamoto has joined #openstack-infra | 03:09 | |
| *** apetrich has quit IRC | 03:10 | |
| *** dayou_ has quit IRC | 03:21 | |
| *** stevebaker has quit IRC | 03:21 | |
| *** stevebaker has joined #openstack-infra | 03:22 | |
| *** psachin has joined #openstack-infra | 03:25 | |
| *** stevebaker has quit IRC | 03:26 | |
| *** stevebaker_ has joined #openstack-infra | 03:26 | |
| *** dayou_ has joined #openstack-infra | 03:29 | |
| *** stevebaker_ has quit IRC | 03:32 | |
| *** stevebaker has joined #openstack-infra | 03:33 | |
| *** Goneri has quit IRC | 03:49 | |
| *** ykarel|away is now known as ykarel | 03:54 | |
| *** ysandeep|rover|b is now known as ysandeep|rover | 04:19 | |
| *** larainema has joined #openstack-infra | 04:27 | |
| *** evrardjp has quit IRC | 04:36 | |
| *** evrardjp has joined #openstack-infra | 04:37 | |
| *** ociuhandu has quit IRC | 04:45 | |
| *** ociuhandu has joined #openstack-infra | 04:46 | |
| *** ociuhandu has quit IRC | 04:52 | |
| prometheanfire | for dib, is there a way to keep an image around for debug (failed builds)? iirc there was | 05:11 |
|---|---|---|
| *** yamamoto has quit IRC | 05:13 | |
| prometheanfire | break=after-error found it | 05:19 |
| *** yamamoto has joined #openstack-infra | 05:22 | |
| clarkb | prometheanfire: I've usually added a bash exec to the element that is failing | 05:28 |
| clarkb | then you can run yhr command that fails and sort ouy why | 05:28 |
| prometheanfire | the break seemed to do it | 05:32 |
| *** yamamoto has quit IRC | 05:38 | |
| *** lmiccini has joined #openstack-infra | 05:44 | |
| *** udesale has joined #openstack-infra | 05:55 | |
| *** sgw has quit IRC | 06:01 | |
| *** dpawlik has joined #openstack-infra | 06:06 | |
| prometheanfire | hmm, | 06:06 |
| prometheanfire | 2020-04-03 05:47:48.493 | /var/tmp/portage/._portage_reinstall_.4b7qcxim/bin/ebuild.sh: line 609: /var/tmp/portage/dev-python/lxml-4.5.0/build-info/lxml-4.5.0.ebuild: No such file or directory | 06:06 |
| prometheanfire | 2020-04-03 05:47:48.494 | * ERROR: dev-python/lxml-4.5.0::gentoo failed: | 06:06 |
| prometheanfire | wrong window | 06:06 |
| *** jamesmcarthur has quit IRC | 06:07 | |
| *** lbragstad_ has joined #openstack-infra | 06:09 | |
| *** yamamoto has joined #openstack-infra | 06:09 | |
| *** lbragstad has quit IRC | 06:12 | |
| *** igordc has quit IRC | 06:21 | |
| *** yamamoto has quit IRC | 06:22 | |
| *** nightmare_unreal has joined #openstack-infra | 06:33 | |
| *** yamamoto has joined #openstack-infra | 06:35 | |
| *** ociuhandu has joined #openstack-infra | 06:46 | |
| *** iurygregory has joined #openstack-infra | 06:49 | |
| *** ociuhandu has quit IRC | 06:55 | |
| *** ociuhandu has joined #openstack-infra | 06:55 | |
| openstackgerrit | Albin Vass proposed zuul/zuul-jobs master: Adds roles to install and run hashicorp packer https://review.opendev.org/709292 | 06:57 |
| *** slaweq has joined #openstack-infra | 06:58 | |
| *** yamamoto has quit IRC | 07:02 | |
| *** yamamoto has joined #openstack-infra | 07:05 | |
| *** ociuhandu has quit IRC | 07:05 | |
| *** ociuhandu has joined #openstack-infra | 07:06 | |
| *** jcapitao has joined #openstack-infra | 07:06 | |
| *** Pandry has joined #openstack-infra | 07:07 | |
| *** dklyle has quit IRC | 07:08 | |
| *** ociuhandu has quit IRC | 07:11 | |
| *** jpena|off is now known as jpena | 07:16 | |
| *** bnemec has quit IRC | 07:20 | |
| *** ykarel is now known as ykarel|afk | 07:21 | |
| *** pgaxatte has joined #openstack-infra | 07:25 | |
| *** ociuhandu has joined #openstack-infra | 07:26 | |
| *** lmiccini has quit IRC | 07:30 | |
| *** ociuhandu has quit IRC | 07:32 | |
| *** ykarel|afk is now known as ykarel | 07:37 | |
| *** jamesmcarthur has joined #openstack-infra | 07:41 | |
| *** tosky has joined #openstack-infra | 07:44 | |
| *** jamesmcarthur has quit IRC | 07:47 | |
| *** lmiccini has joined #openstack-infra | 07:48 | |
| openstackgerrit | Daniel Pawlik proposed zuul/zuul-jobs master: [DNM] - testing updated software-factory images https://review.opendev.org/717210 | 07:48 |
| *** ccamacho has joined #openstack-infra | 07:50 | |
| *** ykarel is now known as ykarel|afk | 07:55 | |
| *** rpittau|afk is now known as rpittau | 07:56 | |
| *** ralonsoh has joined #openstack-infra | 07:57 | |
| *** ysandeep|rover is now known as ysandeep|rover|l | 08:03 | |
| *** tkajinam has quit IRC | 08:08 | |
| *** ykarel|afk is now known as ykarel|lunch | 08:09 | |
| *** dpawlik has quit IRC | 08:11 | |
| *** dpawlik has joined #openstack-infra | 08:11 | |
| *** xek has joined #openstack-infra | 08:20 | |
| *** dtantsur|afk is now known as dtantsur | 08:22 | |
| *** ociuhandu has joined #openstack-infra | 08:27 | |
| *** ysandeep|rover|l is now known as ysandeep|lunch | 08:32 | |
| *** jamesmcarthur has joined #openstack-infra | 08:43 | |
| *** ociuhandu has quit IRC | 08:44 | |
| openstackgerrit | Merged openstack/project-config master: Add hourly periodic pipeline https://review.opendev.org/717063 | 08:44 |
| openstackgerrit | Merged openstack/project-config master: Be clear that zone repos are owned by infra-core https://review.opendev.org/717121 | 08:44 |
| *** jamesmcarthur has quit IRC | 08:48 | |
| *** rmart04 has joined #openstack-infra | 08:49 | |
| *** ociuhandu has joined #openstack-infra | 08:54 | |
| *** tkajinam has joined #openstack-infra | 08:54 | |
| *** ccamacho has quit IRC | 08:59 | |
| *** hashar has joined #openstack-infra | 09:03 | |
| *** xek_ has joined #openstack-infra | 09:05 | |
| *** jamesmcarthur has joined #openstack-infra | 09:07 | |
| *** xek has quit IRC | 09:08 | |
| *** gshippey has joined #openstack-infra | 09:08 | |
| *** ysandeep|lunch is now known as ysandeep|rover | 09:09 | |
| *** jamesmcarthur has quit IRC | 09:12 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on install-podman https://review.opendev.org/716578 | 09:14 |
| *** ociuhandu has quit IRC | 09:15 | |
| *** ociuhandu has joined #openstack-infra | 09:15 | |
| *** ociuhandu has quit IRC | 09:20 | |
| *** pkopec has joined #openstack-infra | 09:21 | |
| *** ykarel|lunch is now known as ykarel | 09:26 | |
| *** xek__ has joined #openstack-infra | 09:27 | |
| *** xek_ has quit IRC | 09:30 | |
| *** ociuhandu has joined #openstack-infra | 09:43 | |
| *** ccamacho has joined #openstack-infra | 09:45 | |
| *** jamesmcarthur has joined #openstack-infra | 09:47 | |
| *** ociuhandu has quit IRC | 09:52 | |
| *** jamesmcarthur has quit IRC | 09:55 | |
| *** ociuhandu has joined #openstack-infra | 09:55 | |
| *** dtantsur is now known as dtantsur|brb | 09:58 | |
| *** gfidente|afk is now known as gfidente | 10:02 | |
| *** ccamacho has quit IRC | 10:03 | |
| *** ccamacho has joined #openstack-infra | 10:03 | |
| *** jamesmcarthur has joined #openstack-infra | 10:04 | |
| openstackgerrit | Thierry Carrez proposed openstack/project-config master: release-approval pipeline: fix zuul-excluding regexp https://review.opendev.org/717277 | 10:06 |
| *** ociuhandu has quit IRC | 10:09 | |
| *** tkajinam has quit IRC | 10:11 | |
| *** ociuhandu has joined #openstack-infra | 10:22 | |
| *** rpittau is now known as rpittau|bbl | 10:29 | |
| *** rcernin has quit IRC | 10:29 | |
| *** ociuhandu has quit IRC | 10:33 | |
| *** derekh has joined #openstack-infra | 10:40 | |
| *** jcapitao is now known as jcapitao_lunch | 10:40 | |
| *** udesale_ has joined #openstack-infra | 10:43 | |
| *** udesale has quit IRC | 10:46 | |
| *** jamesmcarthur has quit IRC | 10:47 | |
| *** ysandeep|rover is now known as ysandeep|break | 11:05 | |
| *** yamamoto has quit IRC | 11:09 | |
| *** jamesmcarthur has joined #openstack-infra | 11:14 | |
| *** ociuhandu has joined #openstack-infra | 11:17 | |
| *** jamesmcarthur has quit IRC | 11:20 | |
| *** yamamoto has joined #openstack-infra | 11:27 | |
| *** ociuhandu has quit IRC | 11:32 | |
| *** jpena is now known as jpena|lunch | 11:38 | |
| *** ociuhandu has joined #openstack-infra | 11:43 | |
| *** ysandeep|break is now known as ysandeep|rover | 11:48 | |
| *** ociuhandu has quit IRC | 11:49 | |
| *** dtantsur|brb is now known as dtantsur | 11:57 | |
| *** jcapitao_lunch is now known as jcapitao | 12:07 | |
| *** rpittau|bbl is now known as rpitau | 12:08 | |
| *** rpitau is now known as rpittau | 12:09 | |
| *** jamesmcarthur has joined #openstack-infra | 12:15 | |
| *** jamesmcarthur has quit IRC | 12:20 | |
| *** rlandy has joined #openstack-infra | 12:28 | |
| *** Lucas_Gray has joined #openstack-infra | 12:30 | |
| *** Lucas_Gray has quit IRC | 12:35 | |
| *** pgaxatte1 has joined #openstack-infra | 12:36 | |
| *** Lucas_Gray has joined #openstack-infra | 12:37 | |
| *** rh-jelabarre has joined #openstack-infra | 12:38 | |
| *** pgaxatte has quit IRC | 12:39 | |
| *** Lucas_Gray has quit IRC | 12:40 | |
| *** jpena|lunch is now known as jpena | 12:45 | |
| *** Lucas_Gray has joined #openstack-infra | 12:46 | |
| *** Lucas_Gray has quit IRC | 12:47 | |
| *** ociuhandu has joined #openstack-infra | 12:47 | |
| *** zigo has quit IRC | 12:48 | |
| *** jamesmcarthur has joined #openstack-infra | 12:51 | |
| *** zigo has joined #openstack-infra | 12:51 | |
| *** jamesmcarthur has quit IRC | 12:56 | |
| *** rfolco has joined #openstack-infra | 13:01 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add phoronix-test-suite job https://review.opendev.org/679082 | 13:03 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 13:11 |
| openstackgerrit | Merged openstack/project-config master: release-approval pipeline: fix zuul-excluding regexp https://review.opendev.org/717277 | 13:20 |
| *** ociuhandu has quit IRC | 13:42 | |
| *** jamesmcarthur has joined #openstack-infra | 13:52 | |
| *** zxiiro has joined #openstack-infra | 13:57 | |
| *** pgaxatte1 has quit IRC | 13:57 | |
| *** jamesmcarthur has quit IRC | 13:57 | |
| *** Goneri has joined #openstack-infra | 13:58 | |
| *** ysandeep|rover is now known as ysandeep|away | 13:59 | |
| *** pgaxatte has joined #openstack-infra | 14:08 | |
| *** bnemec has joined #openstack-infra | 14:13 | |
| *** bnemec is now known as beekneemech | 14:14 | |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: zuul-worker: remove python-apt & libselinux deps https://review.opendev.org/716785 | 14:20 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: zuul-worker: remove additional install of apt-transport-https https://review.opendev.org/716789 | 14:20 |
| *** pgaxatte has quit IRC | 14:21 | |
| openstackgerrit | Merged openstack/project-config master: Trigger infra-prod-service-nodepool on nodepool changes https://review.opendev.org/717135 | 14:24 |
| *** cdearborn has joined #openstack-infra | 14:25 | |
| *** ociuhandu has joined #openstack-infra | 14:25 | |
| *** jamesmcarthur has joined #openstack-infra | 14:30 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Add support for RedHat platforms on install-podman https://review.opendev.org/716578 | 14:35 |
| *** roman_g has joined #openstack-infra | 14:39 | |
| *** EmilienM is now known as EvilienM | 14:40 | |
| *** jamesmcarthur has quit IRC | 14:44 | |
| *** anteaya has joined #openstack-infra | 14:44 | |
| *** jamesmcarthur has joined #openstack-infra | 14:44 | |
| *** sgw has joined #openstack-infra | 14:49 | |
| *** ykarel is now known as ykarel|away | 14:52 | |
| *** jamesmcarthur has quit IRC | 15:00 | |
| *** yamamoto has quit IRC | 15:00 | |
| *** jamesmcarthur has joined #openstack-infra | 15:00 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: tox: allow tox to be upgraded https://review.opendev.org/690057 | 15:01 |
| *** dklyle has joined #openstack-infra | 15:09 | |
| *** jcapitao has quit IRC | 15:11 | |
| *** andrewbonney has joined #openstack-infra | 15:13 | |
| *** yoctozepto has quit IRC | 15:17 | |
| *** yoctozepto8 has joined #openstack-infra | 15:18 | |
| *** Wasaac has quit IRC | 15:18 | |
| *** Wasaac has joined #openstack-infra | 15:19 | |
| *** yamamoto has joined #openstack-infra | 15:20 | |
| artom | clarkb, o/ That GPU flavor we were talking about a few days ago - is it a VM or baremetal? | 15:22 |
| clarkb | VM | 15:22 |
| *** gyee has joined #openstack-infra | 15:22 | |
| artom | So it's passed through from the host into the VM running the tests | 15:22 |
| artom | IIUC that prevents us (Nova, at least) from writing any useful tests - we can't do 2 levels of PCI passthrough | 15:23 |
| artom | Though sean-k-mooney was saying libvirt supports it, we just have to hax Nova itself a bit? | 15:23 |
| artom | (Well, KVM supports it) | 15:23 |
| sean-k-mooney | clarkb: the gpu flavor that mnaser provided last year does not support vGPU | 15:25 |
| sean-k-mooney | clarkb: its doing a physical gpu passthough of an older generation | 15:25 |
| clarkb | we know | 15:25 |
| *** lmiccini has quit IRC | 15:25 | |
| clarkb | artom: oh 2 levels of pci passthrough isnt doable either? | 15:26 |
| artom | We know that - infra's argument was: start using it for tests, otherwise there's nothing to show interest in what *does* exist, before you ask for things that *don't yet exist* :) | 15:26 |
| *** happyhemant has joined #openstack-infra | 15:26 | |
| sean-k-mooney | clarkb: it is | 15:26 |
| clarkb | oh if kvm supports it then fix nova :) | 15:26 |
| sean-k-mooney | clarkb: but not with nova | 15:26 |
| sean-k-mooney | clarkb: we need to add vIOMMU support and then we need to change how we generate the xml to put each passthough deivce on its won pci bridge | 15:26 |
| sean-k-mooney | that will force each passhtough device into a sperate iommu group in the l1 guest | 15:27 |
| artom | sean-k-mooney, I think we'll have bit the bullet and try it, show good faith effort and all that :) | 15:27 |
| sean-k-mooney | allowing it to be added to the l2 guest | 15:27 |
| fungi | sounds like we have a way to test it though ;) | 15:27 |
| sean-k-mooney | artom: i have dont this in novmeber to test it | 15:27 |
| sean-k-mooney | manually with libvirt that is | 15:27 |
| sean-k-mooney | clarkb: the other thing is vexhost would have to be upgrade to a version of openstack that suppotes these changes | 15:28 |
| artom | fungi, sorta, for now :) But with that logic "write a kernel module to fake your hardware" is also "a way to test it" ;) | 15:28 |
| sean-k-mooney | before we could test wtih them | 15:28 |
| fungi | sure | 15:28 |
| artom | sean-k-mooney, doh, forgot about that part | 15:28 |
| artom | But anyways, let's start | 15:28 |
| artom | I go agree that we come off as spoiled kids a bit | 15:28 |
| sean-k-mooney | its on our downstream backlog | 15:28 |
| sean-k-mooney | i think assinged to me | 15:29 |
| clarkb | sean-k-mooney: mnaser has been great about updates | 15:29 |
| sean-k-mooney | yep | 15:29 |
| artom | Oh in the /dev/null blackhole that is our downstream backlog ;) | 15:29 |
| fungi | sounds about like my to do list | 15:29 |
| sean-k-mooney | clarkb: so i filed a bunch of backlog stuff to make this usecase work. its potetally planned for Victoria | 15:29 |
| sean-k-mooney | https://bugzilla.redhat.com/show_bug.cgi?id=1762087 | 15:30 |
| openstack | bugzilla.redhat.com bug 1762087 in openstack-nova "[RFE] support vIOMMU device in nova" [High,New] - Assigned to smooney | 15:30 |
| *** igordc has joined #openstack-infra | 15:32 | |
| sean-k-mooney | https://bugzilla.redhat.com/show_bug.cgi?id=1626130 | 15:32 |
| openstack | bugzilla.redhat.com bug 1626130 in openstack-nova "[RFE] Auto-allocate PCIe route complex per NUMA node" [Medium,New] - Assigned to nova-maint | 15:32 |
| sean-k-mooney | https://bugzilla.redhat.com/show_bug.cgi?id=1762119 | 15:33 |
| openstack | bugzilla.redhat.com bug 1762119 in openstack-nova "[RFE] create a virtual pci root complex per guest numa node when using q35" [High,New] - Assigned to nova-maint | 15:33 |
| sean-k-mooney | it think we have another one but in theory if i get some time in Victoria im going to start closing the nova gap for double passthough | 15:34 |
| sean-k-mooney | clarkb: also on # | 15:34 |
| sean-k-mooney | clarkb: also on #openstack-qa i mentioned my experiments with using the mdpy sample kernel module were going well | 15:35 |
| clarkb | one of the up and downsides to dogfooding our own software here | 15:35 |
| sean-k-mooney | clarkb: i can spawn a vm using qemu direcltly that has a vgpu created by the mdpy module | 15:35 |
| sean-k-mooney | so if i automate that we can test vGPU in upstream gate without any special hardware | 15:36 |
| sean-k-mooney | i do need to slightly alter nova to make that work but it shold be doable | 15:36 |
| sean-k-mooney | clarkb: yep but it motivates use to make it better | 15:36 |
| sean-k-mooney | *us | 15:36 |
| sean-k-mooney | artom: why were you asking? were you thinking of using it for pci_passthough_testing? | 15:39 |
| artom | sean-k-mooney, yeah | 15:40 |
| artom | And as a show of good faith - to then be able to make the point of "see, we're using it! Give us MOAR!" | 15:40 |
| artom | (of the correct thing, this time ;) | 15:40 |
| sean-k-mooney | lol ok am ill have to think about is there a way to make it work | 15:41 |
| sean-k-mooney | without viommu and having the device in a sperate iommu group we mgiht be able to fall back to legacy kvm pci passthough or explore using the kernel softiommu | 15:42 |
| *** rpittau is now known as rpittau|afk | 15:43 | |
| *** anteaya has quit IRC | 15:46 | |
| *** Pandry has quit IRC | 15:48 | |
| *** ysandeep|away is now known as ysandeep | 15:49 | |
| *** udesale_ has quit IRC | 15:50 | |
| dirk | is anything wrong with nb01.opensuse.org the nodepool image builder_ | 15:52 |
| dirk | is anything wrong with nb01.opensuse.org the nodepool image builder? | 15:52 |
| dirk | it seems to rebuild the opensuse-15 all the time and fail every time differently | 15:52 |
| *** psachin has quit IRC | 15:53 | |
| dirk | currently it fails with 'Failed to cache rpm database'. smells a bit like a concurrent run or disk space problem | 15:53 |
| dirk | I'm looking at https://nb01.openstack.org/opensuse-15-0000086197.log | 15:53 |
| *** yamamoto has quit IRC | 15:53 | |
| sean-k-mooney | dirk: have you tried building it locally | 15:53 |
| sean-k-mooney | dirk: it could be an issue with the disk image builder elements too | 15:53 |
| sean-k-mooney | nodepool is just invoking dib with the set of elements and args you set in its config | 15:54 |
| *** JayF is now known as JasonF | 15:55 | |
| sean-k-mooney | i also think it build the images serially on one node but i could likely be wrong about that. i have not looked at how its run upstream in a very long time | 15:55 |
| dirk | I'm aware. at least small images are buildable, but I don't build them on ubuntu | 15:55 |
| *** JasonF is now known as JayF | 15:55 | |
| clarkb | dirk: it has 682GB of disk available which is plenty | 15:55 |
| dirk | although I see it was droppedf rom the dib CI | 15:55 |
| dirk | I guess I have to push a review that reenables it | 15:55 |
| sean-k-mooney | clarkb: isnt there a graphana dashboard for that somewhere | 15:55 |
| fungi | dirk: looks like it's dying with "Failed to cache rpm database" (everything after that is cleanup/wind-down) | 15:56 |
| clarkb | sean-k-mooney: cacti at cacti.openstack.org | 15:56 |
| *** ysandeep is now known as ysandeep|away | 15:56 | |
| sean-k-mooney | ah yes that is what i was thinking of | 15:57 |
| sean-k-mooney | although i thought there was one tied into thise too http://grafana.openstack.org/d/3Bwpi5SZk/nodepool-fortnebula?orgId=1 | 15:57 |
| sean-k-mooney | http://grafana.openstack.org/d/NcFtwKriz/nodepool-dib-status?orgId=1 that seams broken | 15:59 |
| sean-k-mooney | at least the last build column | 15:59 |
| dirk | sean-k-mooney: I can try building it locally, but as I use python 3.x it fails completely different elsewhere | 16:01 |
| fungi | looks like the last images we built for opensuse-15 were around the first week of february: https://nb02.openstack.org/images/ | 16:02 |
| prometheanfire | https://review.opendev.org/717177 might fix the gentoo build failing | 16:02 |
| donnyd | sean-k-mooney: http://grafana.openstack.org/d/dYykaX_Wz/nodepool-openedge?orgId=1&from=now-24h&to=now | 16:02 |
| donnyd | fortnebula was pulled if I am not mistaken | 16:02 |
| prometheanfire | I think infra was failing on installing portage (and portage failing to reuse envvars when it re-inits itself) | 16:03 |
| openstackgerrit | Merged openstack/project-config master: zuul-worker: remove python-apt & libselinux deps https://review.opendev.org/716785 | 16:03 |
| fungi | the opensuse-15 builds on nb02 look like they may be hitting a stale flock though? https://nb02.openstack.org/opensuse-15-0000086199.log | 16:07 |
| fungi | "Did not get /opt/dib_cache/source-repositories/repositories_flock" | 16:07 |
| *** ociuhandu has quit IRC | 16:08 | |
| sean-k-mooney | donnyd: ill update my bookmark :) | 16:08 |
| clarkb | fungi: possible that when builds were crashing due to python dep issues we leaked the flock? | 16:09 |
| sean-k-mooney | donnyd: its useful to have if i need to check the status and your not around | 16:09 |
| clarkb | fungi: are there more than one disk-image-create process running? | 16:09 |
| clarkb | fungi: we may have also leaked builds entirely into the process table | 16:09 |
| fungi | i would have expected two parallel logs in that case | 16:10 |
| fungi | (in the case of multiple builds running at once) | 16:10 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Add update-system-config to promote list for project-config https://review.opendev.org/717328 | 16:10 |
| clarkb | ya we've got several process sets looks like | 16:10 |
| clarkb | we might want to update nodepool to waitpid? | 16:11 |
| *** Lucas_Gray has joined #openstack-infra | 16:11 | |
| *** cmurphy is now known as cmorpheus | 16:13 | |
| *** yamamoto has joined #openstack-infra | 16:16 | |
| prometheanfire | glean fails if /etc/resolv.conf is not found? | 16:17 |
| fungi | prometheanfire: maybe? if so, probably just due to not being tested on platforms which don't typically provide that | 16:18 |
| clarkb | it wants to manage the file if resolvers are set in cloud config iirc | 16:19 |
| clarkb | is it possible it is failing to create th file? | 16:19 |
| *** Lucas_Gray has quit IRC | 16:20 | |
| *** armax has joined #openstack-infra | 16:20 | |
| prometheanfire | fungi: systemd-resolved is started after glean is run, so the symlink points to a non-existent file | 16:21 |
| * prometheanfire is going to look at the service files | 16:22 | |
| prometheanfire | for some reason, even with the requres/wants for glean on networkd/(and added resoved), glean starts first | 16:25 |
| clarkb | I think glean has to start before the network daemons | 16:26 |
| clarkb | since it is writing configs for them | 16:26 |
| clarkb | fungi: on nb01 I wonder if we shouldn't stop the nodepool service, check if those extra dib processes go away then start it again if they do | 16:26 |
| prometheanfire | it could just restart networkd | 16:26 |
| clarkb | prometheanfire: it probably could but taht isn't universally true | 16:26 |
| clarkb | in the case of networkmanager it MUST run first | 16:26 |
| prometheanfire | https://gist.github.com/prometheanfire/572362aaf1abe0bf8a0bbd4c80dea980 | 16:26 |
| clarkb | so its simpler to be consistent | 16:26 |
| prometheanfire | right | 16:26 |
| fungi | right, that may be increasingly less necessary as a lot of them now bind to the all address and keep trying to do stuff until they have working routes and name resolution | 16:26 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Trigger nodepool run https://review.opendev.org/717330 | 16:26 |
| prometheanfire | yay networkmanager :P | 16:27 |
| fungi | but it used to be that, for example, ntpd would die a horrible death if started before you had networking configured | 16:27 |
| fungi | but yeah, the nm integration is an entirely separate problem | 16:27 |
| fungi | because nm will just take over if nothing's been configured yet | 16:28 |
| dirk | fungi: sean-k-mooney: I pushed https://review.opendev.org/#/c/716437/ which fixes the python2-six failure in the opensuse image build | 16:29 |
| fungi | dirk: an alternative may be to switch the opensuse builds to our new clean model, where we no longer use the pip-and-virtualenv element | 16:30 |
| fungi | right now fedora-31 is the only image we're building that way, i think, but the plan is to roll that out to all images soon | 16:31 |
| fungi | we could still merge 716437 as a short-term solution, just be aware it's going away anyway | 16:32 |
| prometheanfire | clarkb: fungi what do you think of https://gist.github.com/1fa27cd733b80d21806839c7ef2f52fa | 16:32 |
| * prometheanfire will try that in a sec | 16:32 | |
| prometheanfire | that seems to fix it for me | 16:34 |
| fungi | prometheanfire: looks like it might be a reasonable approach, but yeah, testing needed | 16:34 |
| prometheanfire | I did test locally at least | 16:36 |
| *** evrardjp has quit IRC | 16:36 | |
| prometheanfire | gonna throw it in a review | 16:36 |
| *** evrardjp has joined #openstack-infra | 16:37 | |
| openstackgerrit | Merged openstack/project-config master: Add update-system-config to promote list for project-config https://review.opendev.org/717328 | 16:37 |
| prometheanfire | fungi: another change is to have the glean service depend on resolved | 16:37 |
| clarkb | prometheanfire: I thought resolv.conf is just a symlink to resolved's conf? | 16:37 |
| fungi | but if resolved hasn't started at least once... | 16:38 |
| clarkb | oh | 16:38 |
| prometheanfire | started at boot | 16:38 |
| fungi | yeah | 16:38 |
| clarkb | then the symlink target doesnt exist | 16:38 |
| fungi | hence the idea to start it first | 16:38 |
| prometheanfire | since it's tmpfiles thing | 16:38 |
| clarkb | ok I think resolved will also read resolv.conf first | 16:38 |
| clarkb | so we can probably juts overwrite the symlink too if we want? | 16:38 |
| prometheanfire | it will, if it's a file | 16:38 |
| prometheanfire | we could | 16:38 |
| prometheanfire | but glean tries to integrate with resolved | 16:39 |
| prometheanfire | (look at the patch and then look at that 'stanza' | 16:39 |
| *** nightmare_unreal has quit IRC | 16:39 | |
| *** roman_g has quit IRC | 16:41 | |
| donnyd | sean-k-mooney: I agree. I depend on people reporting failures many days because I am not always watching it closely | 16:42 |
| donnyd | thankfully this time around I have had zero complaints | 16:42 |
| prometheanfire | something like https://review.opendev.org/717339 | 16:43 |
| *** yoctozepto8 is now known as yoctozepto | 16:43 | |
| *** gfidente is now known as gfidente|afk | 16:43 | |
| prometheanfire | hmm, I think, resolved still needs to be restarted, gonna reorder the needs/wants | 16:46 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Support multiple matchers when parsing tox output https://review.opendev.org/716263 | 16:46 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Don't silently ignore exceptions when parsing tox output https://review.opendev.org/716766 | 16:46 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Strip source dir from file comments https://review.opendev.org/716264 | 16:47 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Ignore absolute paths after stripping work dir https://review.opendev.org/717042 | 16:47 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Add test cases for tox line comment parsing https://review.opendev.org/717341 | 16:47 |
| prometheanfire | ok, should be better | 16:50 |
| openstackgerrit | Merged openstack/project-config master: Trigger nodepool run https://review.opendev.org/717330 | 16:58 |
| *** yamamoto has quit IRC | 17:00 | |
| *** Lucas_Gray has joined #openstack-infra | 17:06 | |
| *** jlvillal is now known as jlvillal_laptop | 17:10 | |
| *** jlvillal_laptop is now known as jlvill-travel | 17:10 | |
| *** jlvill-travel is now known as jlvacation | 17:11 | |
| *** jlvacation is now known as jlv-sick | 17:11 | |
| *** jlv-sick is now known as jlviva-viva | 17:11 | |
| *** jlviva-viva is now known as jlvillal | 17:11 | |
| *** ociuhandu has joined #openstack-infra | 17:12 | |
| *** jpena is now known as jpena|off | 17:14 | |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Update deps list for nodepool job https://review.opendev.org/717351 | 17:15 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Add a deploy pipeline https://review.opendev.org/717353 | 17:21 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Use deploy pipeline for project-config deployments https://review.opendev.org/717354 | 17:21 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Add test cases for tox line comment parsing https://review.opendev.org/717341 | 17:22 |
| openstackgerrit | Monty Taylor proposed openstack/project-config master: Add and use a deploy pipeline https://review.opendev.org/717353 | 17:25 |
| *** jamesmcarthur has quit IRC | 17:26 | |
| *** jamesmcarthur has joined #openstack-infra | 17:29 | |
| *** dpawlik has quit IRC | 17:30 | |
| *** jamesmcarthur has quit IRC | 17:33 | |
| *** jamesmcarthur has joined #openstack-infra | 17:34 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul-jobs master: Add test cases for tox line comment parsing https://review.opendev.org/717341 | 17:34 |
| *** yamamoto has joined #openstack-infra | 17:39 | |
| *** sgw has left #openstack-infra | 17:42 | |
| *** yamamoto has quit IRC | 17:47 | |
| *** slaweq has quit IRC | 17:47 | |
| *** bdodd has quit IRC | 17:48 | |
| *** derekh has quit IRC | 17:52 | |
| *** hashar is now known as hasharDinner | 17:55 | |
| *** sean-k-mooney has quit IRC | 17:56 | |
| *** smarcet has joined #openstack-infra | 18:08 | |
| *** ociuhandu has quit IRC | 18:17 | |
| *** ociuhandu has joined #openstack-infra | 18:17 | |
| *** andrewbonney has quit IRC | 18:23 | |
| *** ociuhandu has quit IRC | 18:23 | |
| *** dtantsur is now known as dtantsur|afk | 18:36 | |
| *** ralonsoh has quit IRC | 18:38 | |
| *** slaweq has joined #openstack-infra | 18:38 | |
| *** rmart04 has quit IRC | 18:38 | |
| *** jamesmcarthur has quit IRC | 18:39 | |
| *** hasharDinner has quit IRC | 18:55 | |
| *** hashar has joined #openstack-infra | 18:57 | |
| *** hemna has quit IRC | 18:59 | |
| *** hemna_ has joined #openstack-infra | 19:08 | |
| *** hemna_ has quit IRC | 19:08 | |
| *** hemna_ has joined #openstack-infra | 19:10 | |
| *** slaweq has quit IRC | 19:10 | |
| *** stevebaker has quit IRC | 19:11 | |
| *** hemna_ has quit IRC | 19:11 | |
| *** jamesmcarthur has joined #openstack-infra | 19:28 | |
| *** jamesmcarthur has quit IRC | 19:28 | |
| *** jamesmcarthur has joined #openstack-infra | 19:29 | |
| *** hemna has joined #openstack-infra | 19:34 | |
| *** hemna has quit IRC | 19:37 | |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: install-yarn: add coverage for all platforms https://review.opendev.org/717375 | 19:38 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add phoronix-test-suite job https://review.opendev.org/679082 | 19:46 |
| *** dpawlik has joined #openstack-infra | 19:48 | |
| openstackgerrit | Merged openstack/project-config master: Update deps list for nodepool job https://review.opendev.org/717351 | 19:51 |
| openstackgerrit | Merged openstack/project-config master: Add and use a deploy pipeline https://review.opendev.org/717353 | 19:51 |
| *** Lucas_Gray has quit IRC | 19:52 | |
| openstackgerrit | Mohammed Naser proposed zuul/zuul-jobs master: install-yarn: always install https://review.opendev.org/717375 | 19:56 |
| *** Lucas_Gray has joined #openstack-infra | 19:57 | |
| *** ociuhandu has joined #openstack-infra | 19:58 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Support multiple matchers when parsing tox output https://review.opendev.org/716263 | 20:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Don't silently ignore exceptions when parsing tox output https://review.opendev.org/716766 | 20:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Strip source dir from file comments https://review.opendev.org/716264 | 20:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Ignore absolute paths after stripping work dir https://review.opendev.org/717042 | 20:00 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add test cases for tox line comment parsing https://review.opendev.org/717341 | 20:02 |
| *** ccamacho has quit IRC | 20:10 | |
| *** Lucas_Gray has quit IRC | 20:13 | |
| openstackgerrit | Merged zuul/zuul-jobs master: helm: collect kubernetes logs in post https://review.opendev.org/715709 | 20:14 |
| *** Lucas_Gray has joined #openstack-infra | 20:17 | |
| tbarron | question w.r.t. hacking-3.0.0-py2.py3-none-any.whl from devstack mirrors -- we | 20:21 |
| tbarron | are trying to merge a fix for a stable/train blocker in manila and in some of the jobs our | 20:22 |
| tbarron | test-requirement for this package is met | 20:22 |
| tbarron | just fine | 20:22 |
| tbarron | example: https://zuul.opendev.org/t/openstack/build/e94e947780404b8b9e85f7021414a294/log/logs/devstacklog.txt#9299 | 20:22 |
| tbarron | but in others we it is not found and devstack setup fails, e.g. | 20:23 |
| tbarron | https://zuul.opendev.org/t/openstack/build/9032b60cb27d41fd90cfb427d0fa75a2/log/logs/devstacklog.txt#9095 | 20:23 |
| tbarron | not just missing one time, seems to be repeating | 20:24 |
| *** Lucas_Gray has quit IRC | 20:24 | |
| openstackgerrit | Clark Boylan proposed openstack/project-config master: Install gpg tooling on dib images https://review.opendev.org/717386 | 20:31 |
| clarkb | tbarron: is that a python version specific package? | 20:35 |
| * clarkb looks | 20:35 | |
| clarkb | tbarron: yup hacking actually requires python>=3.5 | 20:35 |
| tbarron | clarkb: ah, mebbe, this is a py2 job where I see the failures! | 20:35 |
| clarkb | now to check if hacking is configured to build a universal wheel | 20:36 |
| tbarron | d'oh , the py2.py3 in the name led me to think it was compatible | 20:36 |
| clarkb | tbarron: the wheel filename says it can work uner python2 | 20:36 |
| clarkb | but that is wrong according to pypi metadata | 20:36 |
| clarkb | https://opendev.org/openstack/hacking/src/branch/master/setup.cfg#L76-L77 that should be deleted and a new release made | 20:37 |
| tbarron | hmm, so we are getting this newer hacking requirement in a tempest plugin which means it shows up in stable/branches with py2 b/c tempest is branchless | 20:37 |
| *** ociuhandu has quit IRC | 20:46 | |
| mordred | clarkb: the python-requires isn't filtering it? | 20:47 |
| clarkb | mordred: not on the wheel side | 20:47 |
| *** ociuhandu has joined #openstack-infra | 20:47 | |
| clarkb | because the wheel says "I can python2" I think | 20:47 |
| mordred | oy | 20:47 |
| clarkb | though actually | 20:47 |
| mordred | oh - so this is just a problem on our wheel mirror? | 20:47 |
| clarkb | no thats wrong | 20:47 |
| clarkb | it shouldn't be in our wheel mirror because we push wheels to pypi | 20:47 |
| clarkb | (we only build wheels now for things that don't have wheels on pypi) | 20:48 |
| mordred | good point | 20:48 |
| clarkb | so I think its just complaining that the requested version isn't valid for the running python | 20:48 |
| clarkb | we should fix hacking but also tbarron and the tempest thing need to not request that version under ypthon2 | 20:48 |
| mordred | yeah | 20:48 |
| tbarron | clarkb: yeah | 20:48 |
| mordred | because it's requiesting a bad version in the version specifying | 20:48 |
| tbarron | AJaeger: seems like we need to revert https://review.opendev.org/#/c/716559/ ? | 20:49 |
| *** pkopec has quit IRC | 20:49 | |
| tbarron | late for him ... | 20:49 |
| mordred | no - there'sa. better way | 20:49 |
| mordred | one sec | 20:49 |
| tbarron | ah | 20:50 |
| tbarron | no way to write the requirement conditional on python version I guess | 20:50 |
| mordred | tbarron: https://review.opendev.org/#/c/716559/1 | 20:51 |
| mordred | tbarron: I just left a comment on how to do that | 20:51 |
| tbarron | mordred: awesome, I am very grateful to AJaeger for all those fixes and hate to back any of that stuff out | 20:52 |
| *** ociuhandu has quit IRC | 20:52 | |
| tbarron | yeah, that's what I was trying to remember | 20:52 |
| mordred | tbarron: it'll break the ability to run hacking in py2 - but who cares :) | 20:52 |
| fungi | clearly hacking doesn't! | 20:52 |
| tbarron | agree but can I just add the old line too with 'python_version <'3' ? | 20:53 |
| tbarron | fungi: hah | 20:53 |
| mordred | tbarron: you could if you wanted to | 20:54 |
| mordred | it's unlikley to hurt - but also is unlikely to be needed | 20:55 |
| mordred | since nothing is _using_ that hacking install except in your linter jobs | 20:55 |
| tbarron | true | 20:55 |
| tbarron | mordred: and that s/b 'python_version>='3' , right? | 20:56 |
| tbarron | rather than ">'3'" | 20:56 |
| mordred | yah | 20:56 |
| tbarron | kk | 20:56 |
| mordred | oh - yeah -either would be fine | 20:56 |
| tbarron | since we're not exactly 3 anyways, I see | 20:57 |
| tbarron | mordred: clarkb: thanks much! | 20:57 |
| clarkb | you're welcome | 20:58 |
| clarkb | is anyone pushing up a fix to hacking yet? | 20:58 |
| clarkb | if not I'll do that | 20:58 |
| mordred | clarkb: go for it | 20:58 |
| openstackgerrit | Merged openstack/project-config master: Install gpg tooling on dib images https://review.opendev.org/717386 | 20:59 |
| openstackgerrit | Clark Boylan proposed openstack/hacking master: Stop building universal wheels https://review.opendev.org/717390 | 21:00 |
| *** ociuhandu has joined #openstack-infra | 21:00 | |
| *** ociuhandu has quit IRC | 21:04 | |
| *** EvilienM is now known as EmilienM | 21:07 | |
| *** gshippey has quit IRC | 21:23 | |
| *** jamesmcarthur has quit IRC | 21:23 | |
| *** jamesmcarthur has joined #openstack-infra | 21:24 | |
| *** jamesmcarthur has quit IRC | 21:29 | |
| *** KeithMnemonic has quit IRC | 21:34 | |
| *** jamesmcarthur has joined #openstack-infra | 21:36 | |
| *** xek__ has quit IRC | 21:42 | |
| *** weifan has joined #openstack-infra | 21:44 | |
| *** yamamoto has joined #openstack-infra | 21:44 | |
| *** weifan has quit IRC | 21:48 | |
| *** yamamoto has quit IRC | 21:50 | |
| *** gfidente|afk has quit IRC | 21:53 | |
| *** hashar has quit IRC | 22:05 | |
| *** rlandy has quit IRC | 22:12 | |
| *** larainema has quit IRC | 22:25 | |
| *** dpawlik has quit IRC | 22:32 | |
| *** igordc has quit IRC | 22:36 | |
| *** igordc has joined #openstack-infra | 22:36 | |
| *** rkukura has quit IRC | 22:37 | |
| *** rh-jelabarre has quit IRC | 22:38 | |
| *** rkukura has joined #openstack-infra | 22:38 | |
| *** smarcet has quit IRC | 23:15 | |
| *** tosky has quit IRC | 23:16 | |
| *** jamesmcarthur has quit IRC | 23:19 | |
| *** jamesmcarthur has joined #openstack-infra | 23:20 | |
| *** jamesmcarthur has quit IRC | 23:25 | |
| *** jamesmcarthur has joined #openstack-infra | 23:29 | |
| *** ociuhandu has joined #openstack-infra | 23:36 | |
| *** ociuhandu has quit IRC | 23:41 | |
| *** tetsuro has joined #openstack-infra | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!