| donnyd | I use cloud-init for most of my instances | 00:00 |
|---|---|---|
| clarkb | it is entirely possible that this issue affects them too | 00:00 |
| clarkb | donnyd: do you run fedora and centos though? and if you do are they on the ipv6 only networks too? | 00:00 |
| donnyd | I just launched a 25 node hadoop cluster today without issue on centos | 00:00 |
| clarkb | (and did you double check they got ipv4 and ipv6 both?) | 00:00 |
| clarkb | that seems to be the key here | 00:00 |
| clarkb | we get one or the other | 00:00 |
| clarkb | but often not both | 00:00 |
| donnyd | I only come in via v6 | 00:00 |
| donnyd | 100% success rate | 00:01 |
| clarkb | (what these chagnes are attempting to fix is the no ipv4 problem, but ipv6 seems to work pretty reliably. However we've inverted that setup and now ipv4 works reliable and ipv6 is not) | 00:01 |
| clarkb | ya so we have successful ipv6 without these changes, but lack successful ipv4 | 00:01 |
| *** sthussey has quit IRC | 00:01 | |
| clarkb | with these chagnes have successful ipv4 and lack ipv6 :( | 00:02 |
| clarkb | (mostly pointint out you need to check both, one or the other working doesn't mean the other is happy) | 00:02 |
| *** goldyfruit_ has joined #openstack-infra | 00:03 | |
| clarkb | ok down to two non working hosts in fn. I'll leave those in place so people can poke at them | 00:04 |
| ianw | clarkb: did you turn on console-to-log? | 00:04 |
| ianw | although if you can get in via ipv6 i imagine you're seeing the logs | 00:04 |
| clarkb | ianw: I'm not sure what that is so probably not | 00:04 |
| clarkb | oh ya I'm seeing the logs | 00:04 |
| clarkb | (getting in via ipv4 on a host with working ipv6) | 00:04 |
| clarkb | nmcli device show eth0 shows no ipv6 configured | 00:05 |
| clarkb | restart NM and then its there | 00:05 |
| ianw | ok, right, yeah there's just a dib element to send all systemd logs to console which helps if you can't get in at all | 00:05 |
| clarkb | journalctl shows glean running to completion then NM starting | 00:05 |
| clarkb | I think the two next steps are try builds without the kernel RA workaround (maybe that is detrimental to us in this new case?) and also boot upstream centos 7 and fedora 29 images to see if they suffer from similar | 00:05 |
| *** markvoelker has quit IRC | 00:05 | |
| clarkb | can also compare the resulting sysconfigfiles that way | 00:06 |
| ianw | for comparision; have we noticed this with upstream images? | 00:06 |
| ianw | ahh, great minds think alike :) | 00:06 |
| *** markvoelker has joined #openstack-infra | 00:07 | |
| clarkb | the image names in fn should be apparent to | 00:07 |
| clarkb | its like clarkb-test-glean-centos and -fedora | 00:07 |
| clarkb | if others want to boot their own | 00:07 |
| clarkb | but I need to start calling it a day and I'm out tomorrow but can help pcik this up thurday | 00:07 |
| *** mattw4 has quit IRC | 00:07 | |
| ianw | hrm, i wonder what "nmcli general logging debug" actually does, and how to do it before nm starts | 00:08 |
| *** smarcet has joined #openstack-infra | 00:12 | |
| *** markvoelker has quit IRC | 00:14 | |
| *** markvoelker has joined #openstack-infra | 00:19 | |
| fungi | at this point i wouldn't be surprised if the problem is some sort of dbus-related race | 00:19 |
| ianw | i'm just running a final full all-infra-elements test for centos8 | 00:20 |
| *** hwoarang has quit IRC | 00:34 | |
| *** diablo_rojo has joined #openstack-infra | 00:37 | |
| *** hwoarang has joined #openstack-infra | 00:44 | |
| *** smarcet has quit IRC | 00:52 | |
| *** smarcet has joined #openstack-infra | 00:53 | |
| *** goldyfruit_ has quit IRC | 00:56 | |
| *** igordc has quit IRC | 01:24 | |
| *** tkajinam_ has joined #openstack-infra | 01:26 | |
| *** tkajinam has quit IRC | 01:28 | |
| *** dychen has joined #openstack-infra | 01:31 | |
| *** dchen has quit IRC | 01:33 | |
| *** dychen has quit IRC | 01:35 | |
| *** dchen has joined #openstack-infra | 01:35 | |
| *** smarcet has joined #openstack-infra | 01:36 | |
| *** dchen has quit IRC | 01:39 | |
| *** dchen has joined #openstack-infra | 01:39 | |
| *** ykarel|away has joined #openstack-infra | 01:45 | |
| *** dychen has joined #openstack-infra | 01:46 | |
| *** dchen has quit IRC | 01:49 | |
| *** goldyfruit_ has joined #openstack-infra | 02:05 | |
| *** whoami-rajat has joined #openstack-infra | 02:07 | |
| *** apetrich has quit IRC | 02:10 | |
| *** psachin has joined #openstack-infra | 02:11 | |
| *** smarcet has quit IRC | 02:16 | |
| *** ricolin has joined #openstack-infra | 02:21 | |
| *** roman_g has quit IRC | 02:34 | |
| *** goldyfruit_ has quit IRC | 02:35 | |
| *** ramishra has joined #openstack-infra | 02:37 | |
| openstackgerrit | Colleen Murphy proposed openstack/openstack-zuul-jobs master: Migrate ldap job to keystone repo https://review.opendev.org/687444 | 03:06 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: nodepool/elements : use abstracted commands https://review.opendev.org/686524 | 03:07 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: infra-pkg-needs: Update pkg-maps for CentOS 8, select chronyd https://review.opendev.org/687445 | 03:07 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: zuul-worker: no selinux python2 libs on CentOS 8 https://review.opendev.org/687446 | 03:07 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: infra-package-needs: fix haveged install for all CentOS releases https://review.opendev.org/687447 | 03:07 |
| *** markvoelker has quit IRC | 03:09 | |
| *** markvoelker has joined #openstack-infra | 03:10 | |
| *** factor has joined #openstack-infra | 03:11 | |
| *** psachin has quit IRC | 03:17 | |
| *** psachin has joined #openstack-infra | 03:18 | |
| *** markvoelker has quit IRC | 03:23 | |
| *** markvoelker has joined #openstack-infra | 03:24 | |
| *** gregoryo has joined #openstack-infra | 03:33 | |
| *** diablo_rojo has quit IRC | 03:40 | |
| *** ramishra has quit IRC | 03:43 | |
| *** udesale has joined #openstack-infra | 03:55 | |
| *** raukadah is now known as chandankumar | 03:58 | |
| *** ykarel|away has quit IRC | 04:01 | |
| *** dingyichen has joined #openstack-infra | 04:03 | |
| *** dychen has quit IRC | 04:05 | |
| *** dychen has joined #openstack-infra | 04:12 | |
| *** dingyichen has quit IRC | 04:14 | |
| *** ociuhandu has joined #openstack-infra | 04:16 | |
| *** smarcet has joined #openstack-infra | 04:18 | |
| *** ociuhandu has quit IRC | 04:20 | |
| *** ykarel|away has joined #openstack-infra | 04:22 | |
| *** ykarel|away is now known as ykarel | 04:22 | |
| *** smarcet has quit IRC | 04:23 | |
| *** markvoelker has quit IRC | 04:35 | |
| *** markvoelker has joined #openstack-infra | 04:35 | |
| *** rh-jelabarre has quit IRC | 04:51 | |
| *** ramishra has joined #openstack-infra | 04:59 | |
| *** yamamoto has quit IRC | 05:25 | |
| *** yamamoto has joined #openstack-infra | 05:30 | |
| *** exsdev0 has joined #openstack-infra | 05:48 | |
| *** exsdev has quit IRC | 05:48 | |
| *** exsdev0 is now known as exsdev | 05:48 | |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: Remove explicit set of DIB_SIMPLE_INIT_NETWORKMANAGER https://review.opendev.org/687452 | 05:53 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: CentOS 8 initial deployment https://review.opendev.org/687453 | 05:53 |
| openstackgerrit | Ian Wienand proposed openstack/project-config master: CentOS 8 initial deployment https://review.opendev.org/687453 | 06:02 |
| *** kopecmartin|off is now known as kopecmartin | 06:07 | |
| *** slaweq has joined #openstack-infra | 06:08 | |
| *** jbadiapa has joined #openstack-infra | 06:29 | |
| *** pgaxatte has joined #openstack-infra | 06:31 | |
| *** kjackal has joined #openstack-infra | 06:32 | |
| *** kjackal has quit IRC | 06:33 | |
| *** jaosorior has quit IRC | 06:33 | |
| *** kjackal has joined #openstack-infra | 06:37 | |
| *** xenos76 has joined #openstack-infra | 06:45 | |
| *** jtomasek has joined #openstack-infra | 06:50 | |
| *** jtomasek has quit IRC | 06:52 | |
| *** jtomasek has joined #openstack-infra | 07:03 | |
| *** jaosorior has joined #openstack-infra | 07:04 | |
| *** roman_g has joined #openstack-infra | 07:06 | |
| *** xek_ has joined #openstack-infra | 07:07 | |
| *** pcaruana has joined #openstack-infra | 07:10 | |
| *** tesseract has joined #openstack-infra | 07:13 | |
| *** tosky has joined #openstack-infra | 07:14 | |
| *** verdurin has quit IRC | 07:16 | |
| *** verdurin has joined #openstack-infra | 07:17 | |
| *** iurygregory has joined #openstack-infra | 07:21 | |
| *** apetrich has joined #openstack-infra | 07:24 | |
| *** ccamacho has joined #openstack-infra | 07:24 | |
| *** yamamoto has quit IRC | 07:30 | |
| *** yamamoto has joined #openstack-infra | 07:33 | |
| *** yamamoto has quit IRC | 07:34 | |
| *** yamamoto has joined #openstack-infra | 07:34 | |
| *** rpittau|afk is now known as rpittau | 07:35 | |
| *** surpatil has joined #openstack-infra | 07:41 | |
| *** slaweq has quit IRC | 07:42 | |
| *** ykarel is now known as ykarel|lunch | 07:44 | |
| *** jpena|off is now known as jpena | 07:46 | |
| *** lucasagomes has joined #openstack-infra | 07:49 | |
| *** dtantsur|afk is now known as dtantsur | 07:49 | |
| *** slaweq has joined #openstack-infra | 07:49 | |
| *** xenos76 has quit IRC | 07:51 | |
| *** xenos76 has joined #openstack-infra | 07:52 | |
| *** dpawlik has joined #openstack-infra | 07:53 | |
| *** tosky has quit IRC | 07:58 | |
| *** ralonsoh has joined #openstack-infra | 07:58 | |
| *** tosky has joined #openstack-infra | 07:59 | |
| *** tkajinam_ has quit IRC | 08:10 | |
| openstackgerrit | pengyuesheng proposed openstack/diskimage-builder master: Blacklist sphinx 2.1.0 (autodoc bug) https://review.opendev.org/687475 | 08:16 |
| openstackgerrit | pengyuesheng proposed openstack/hacking master: Sync Sphinx requirement https://review.opendev.org/687477 | 08:19 |
| *** ociuhandu has joined #openstack-infra | 08:20 | |
| *** ociuhandu has quit IRC | 08:20 | |
| *** ociuhandu has joined #openstack-infra | 08:21 | |
| *** ociuhandu has quit IRC | 08:23 | |
| *** ociuhandu has joined #openstack-infra | 08:23 | |
| *** pkopec has joined #openstack-infra | 08:24 | |
| *** ramishra_ has joined #openstack-infra | 08:27 | |
| *** ramishra has quit IRC | 08:29 | |
| *** derekh has joined #openstack-infra | 08:33 | |
| *** ykarel|lunch is now known as ykarel | 08:34 | |
| *** ociuhandu has quit IRC | 08:34 | |
| *** e0ne has joined #openstack-infra | 08:39 | |
| *** dychen has quit IRC | 08:40 | |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Sort waiting static nodes by creation time https://review.opendev.org/687271 | 08:43 |
| *** jaosorior has quit IRC | 08:53 | |
| *** gfidente|afk has quit IRC | 09:01 | |
| *** xek_ has quit IRC | 09:11 | |
| *** gfidente has joined #openstack-infra | 09:14 | |
| *** ociuhandu has joined #openstack-infra | 09:23 | |
| *** priteau has joined #openstack-infra | 09:26 | |
| *** yamamoto has quit IRC | 09:30 | |
| *** gregoryo has quit IRC | 09:30 | |
| *** jaosorior has joined #openstack-infra | 09:31 | |
| *** dpawlik has quit IRC | 09:55 | |
| tosky | AJaeger: I won't be around for the Summit/PTG, but count me in if the Zuul v3 goal is approved | 09:57 |
| *** surpatil has quit IRC | 09:59 | |
| *** yamamoto has joined #openstack-infra | 10:07 | |
| *** efried has joined #openstack-infra | 10:12 | |
| efried | o/ infra | 10:12 |
| efried | last couple of days I've noticed test_results.html.gz not being a link anymore, sometimes. E.g. https://zuul.opendev.org/t/openstack/build/fa0e838822cf41cab8418be1dd448515/logs | 10:12 |
| efried | It's nbd because the `raw` link still works, just a behavior change I figured someone might care to be aware of. | 10:13 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - add the enqueue_ref unit test https://review.opendev.org/687351 | 10:13 |
| *** jamesmcarthur has joined #openstack-infra | 10:15 | |
| *** jamesmcarthur has quit IRC | 10:19 | |
| *** ociuhandu has quit IRC | 10:23 | |
| frickler | efried: that seems to be triggered by https://opendev.org/zuul/zuul/commit/e6496faf406529b4003ce7ebaa22eb1f2fa78929 but I'm unsure whether that is the intended result or a bug | 10:23 |
| frickler | corvus: ^^ | 10:23 |
| *** ociuhandu has joined #openstack-infra | 10:25 | |
| efried | frickler: Thanks. That appears intentional based on the commit message. And if I'm remembering the previous behavior correctly, which is that I would end up on the actual results page (not remain in the zuul portal thingy) it's doing what's intended. Thanks for the response. | 10:25 |
| *** pgaxatte has quit IRC | 10:36 | |
| *** prometheanfire has quit IRC | 10:38 | |
| *** lpetrut has joined #openstack-infra | 10:39 | |
| *** prometheanfire has joined #openstack-infra | 10:39 | |
| *** tesseract has quit IRC | 10:44 | |
| AJaeger | tosky: I won't go to the summit/PTG either - and asked for somebody else to drive it. | 10:45 |
| *** tesseract has joined #openstack-infra | 10:46 | |
| *** rcernin has quit IRC | 10:48 | |
| *** yamamoto has quit IRC | 10:50 | |
| *** yamamoto has joined #openstack-infra | 10:59 | |
| *** yamamoto has quit IRC | 10:59 | |
| *** yamamoto has joined #openstack-infra | 11:00 | |
| *** yamamoto has quit IRC | 11:04 | |
| *** ociuhandu has quit IRC | 11:11 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - add support for git.tag.creation event https://review.opendev.org/679938 | 11:14 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - Support for branch creation/deletion https://review.opendev.org/685116 | 11:15 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - add the enqueue_ref unit test https://review.opendev.org/687351 | 11:15 |
| *** ociuhandu has joined #openstack-infra | 11:15 | |
| *** ociuhandu has quit IRC | 11:16 | |
| *** ociuhandu has joined #openstack-infra | 11:16 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Add optional support for circular dependencies https://review.opendev.org/685354 | 11:16 |
| *** yamamoto has joined #openstack-infra | 11:22 | |
| *** dtantsur is now known as dtantsur|brb | 11:22 | |
| *** udesale has quit IRC | 11:22 | |
| *** yamamoto has quit IRC | 11:23 | |
| openstackgerrit | Simon Westphahl proposed zuul/zuul master: Spec for allowing circular dependencies https://review.opendev.org/643309 | 11:23 |
| *** jpena is now known as jpena|lunch | 11:29 | |
| *** ramishra_ has quit IRC | 11:30 | |
| *** pgaxatte has joined #openstack-infra | 11:31 | |
| *** yamamoto has joined #openstack-infra | 11:33 | |
| *** yamamoto has quit IRC | 11:33 | |
| *** yamamoto has joined #openstack-infra | 11:34 | |
| *** yamamoto has quit IRC | 11:38 | |
| *** dave-mccowan has joined #openstack-infra | 11:49 | |
| *** ramishra has joined #openstack-infra | 11:49 | |
| *** apetrich has quit IRC | 11:50 | |
| *** auristor has quit IRC | 11:57 | |
| *** jaosorior has quit IRC | 11:58 | |
| *** auristor has joined #openstack-infra | 11:59 | |
| *** xek_ has joined #openstack-infra | 12:03 | |
| *** apetrich has joined #openstack-infra | 12:03 | |
| *** yamamoto has joined #openstack-infra | 12:07 | |
| *** goldyfruit_ has joined #openstack-infra | 12:12 | |
| *** goldyfruit___ has joined #openstack-infra | 12:14 | |
| *** ociuhandu has quit IRC | 12:15 | |
| *** ociuhandu has joined #openstack-infra | 12:15 | |
| *** goldyfruit_ has quit IRC | 12:17 | |
| openstackgerrit | Akihiro Motoki proposed openstack/project-config master: gerritbot: notify stable branch changes to #openstack-horizon https://review.opendev.org/687543 | 12:17 |
| *** apetrich has quit IRC | 12:18 | |
| *** goldyfruit___ has quit IRC | 12:19 | |
| *** ociuhandu has quit IRC | 12:20 | |
| *** roman_g has quit IRC | 12:21 | |
| *** jpena|lunch is now known as jpena | 12:23 | |
| *** jaosorior has joined #openstack-infra | 12:28 | |
| *** rh-jelabarre has joined #openstack-infra | 12:29 | |
| *** psachin has quit IRC | 12:31 | |
| *** ociuhandu has joined #openstack-infra | 12:33 | |
| *** mriedem has joined #openstack-infra | 12:44 | |
| *** yamamoto has quit IRC | 12:51 | |
| *** yamamoto has joined #openstack-infra | 12:53 | |
| *** yamamoto has quit IRC | 12:53 | |
| *** yamamoto has joined #openstack-infra | 12:53 | |
| *** ociuhandu has quit IRC | 12:54 | |
| *** ociuhandu has joined #openstack-infra | 12:55 | |
| *** ociuhandu has quit IRC | 12:56 | |
| *** ociuhandu has joined #openstack-infra | 12:56 | |
| *** yamamoto has quit IRC | 12:57 | |
| *** eharney has joined #openstack-infra | 13:01 | |
| *** apetrich has joined #openstack-infra | 13:05 | |
| *** slaweq has quit IRC | 13:07 | |
| tdasilva | corvus: hello, not sure if I have asked you before, but are you aware of any jobs that would allow us to update the description of a docker image on docker hub? ideally we would keep the description in our git repo and have it updated on changes | 13:07 |
| tdasilva | corvus: for reference: I'd like to add a description to https://hub.docker.com/r/openstackswift/saio | 13:08 |
| *** slaweq has joined #openstack-infra | 13:10 | |
| *** markvoelker has quit IRC | 13:10 | |
| *** markvoelker has joined #openstack-infra | 13:11 | |
| *** markvoelker has quit IRC | 13:11 | |
| *** markvoelker has joined #openstack-infra | 13:11 | |
| *** xenos76 has quit IRC | 13:12 | |
| *** slaweq_ has joined #openstack-infra | 13:13 | |
| *** slaweq has quit IRC | 13:14 | |
| *** jcoufal has joined #openstack-infra | 13:19 | |
| *** sreejithp has joined #openstack-infra | 13:26 | |
| *** goldyfruit___ has joined #openstack-infra | 13:28 | |
| *** aaronsheffield has joined #openstack-infra | 13:29 | |
| *** bnemec has quit IRC | 13:29 | |
| *** goldyfruit___ has quit IRC | 13:29 | |
| *** slaweq_ is now known as slaweq | 13:29 | |
| *** auristor has quit IRC | 13:30 | |
| *** sreejithp_ has joined #openstack-infra | 13:30 | |
| *** bnemec has joined #openstack-infra | 13:31 | |
| *** pcaruana has quit IRC | 13:31 | |
| *** goldyfruit has joined #openstack-infra | 13:32 | |
| *** sreejithp has quit IRC | 13:33 | |
| *** jaosorior has quit IRC | 13:33 | |
| *** pcaruana has joined #openstack-infra | 13:33 | |
| *** roman_g has joined #openstack-infra | 13:40 | |
| *** ianychoi has quit IRC | 13:41 | |
| *** auristor has joined #openstack-infra | 13:43 | |
| *** udesale has joined #openstack-infra | 13:45 | |
| *** bdodd has joined #openstack-infra | 13:49 | |
| *** rf0lc0 has joined #openstack-infra | 13:51 | |
| *** otherwiseguy has quit IRC | 13:51 | |
| *** otherwiseguy has joined #openstack-infra | 13:52 | |
| *** mnencia_ has joined #openstack-infra | 13:53 | |
| *** jklare has quit IRC | 13:56 | |
| *** gagehugo has quit IRC | 13:56 | |
| *** davidlenwell has quit IRC | 13:56 | |
| *** tonyb has quit IRC | 13:56 | |
| *** mnencia has quit IRC | 13:56 | |
| *** scas has quit IRC | 13:56 | |
| *** auristor has quit IRC | 13:56 | |
| *** rfolco has quit IRC | 13:56 | |
| *** SotK has quit IRC | 13:56 | |
| *** jmccrory has quit IRC | 13:56 | |
| *** ullbeking has quit IRC | 13:56 | |
| *** gothicmindfood has quit IRC | 13:56 | |
| *** apetrich has quit IRC | 13:56 | |
| *** lucasagomes has quit IRC | 13:56 | |
| *** exsdev has quit IRC | 13:56 | |
| *** rakhmerov has quit IRC | 13:56 | |
| *** ddurst has quit IRC | 13:56 | |
| *** mnencia_ is now known as mnencia | 13:56 | |
| *** SotK has joined #openstack-infra | 13:56 | |
| *** exsdev has joined #openstack-infra | 13:56 | |
| *** auristor has joined #openstack-infra | 13:56 | |
| *** ullbeking has joined #openstack-infra | 13:56 | |
| *** davidlenwell has joined #openstack-infra | 13:56 | |
| *** ddurst has joined #openstack-infra | 13:56 | |
| *** openstackstatus has quit IRC | 13:58 | |
| *** ociuhandu has quit IRC | 13:59 | |
| *** apetrich has joined #openstack-infra | 14:00 | |
| *** auristor has quit IRC | 14:00 | |
| *** auristor has joined #openstack-infra | 14:01 | |
| *** rf0lc0 is now known as rfolco | 14:02 | |
| *** jaosorior has joined #openstack-infra | 14:03 | |
| *** dtantsur|brb is now known as dtantsur | 14:05 | |
| *** fdegir has quit IRC | 14:06 | |
| *** georgk has quit IRC | 14:06 | |
| *** georgk has joined #openstack-infra | 14:06 | |
| *** fdegir has joined #openstack-infra | 14:07 | |
| *** slaweq has quit IRC | 14:07 | |
| *** rascasoft has quit IRC | 14:08 | |
| *** slaweq has joined #openstack-infra | 14:09 | |
| *** lpetrut has quit IRC | 14:09 | |
| *** lucasagomes has joined #openstack-infra | 14:11 | |
| *** rascasoft has joined #openstack-infra | 14:11 | |
| corvus | tdasilva: this is also a think i would like -- we don't have that implemnted yet, but i think we could make a zuul role out of https://github.com/peter-evans/dockerhub-description/blob/master/entrypoint.sh | 14:14 |
| beisner | hi AJaeger - are you available to push workflow on project-config fhttps://review.opendev.org/#/c/686251 ? I think we have all of the requirements sorted. | 14:14 |
| corvus | tdasilva: sorry for spelling implemented like i just woke up or something | 14:15 |
| tdasilva | corvus: hehehe, no worries, thanks for the pointer! | 14:15 |
| *** Goneri has joined #openstack-infra | 14:16 | |
| *** apetrich has quit IRC | 14:18 | |
| *** jamesmcarthur has joined #openstack-infra | 14:19 | |
| AJaeger | beisner: every change needs two core reviewers to +2 | 14:21 |
| *** xenos76 has joined #openstack-infra | 14:21 | |
| AJaeger | beisner: so, I will not by policy workflow a change as single core | 14:21 |
| mordred | corvus: oh! I didn't realize there was a dockerhub description setting api - that's great | 14:22 |
| beisner | AJaeger: ah, indeed. | 14:24 |
| mordred | beisner: +A | 14:27 |
| *** apetrich has joined #openstack-infra | 14:27 | |
| *** zul has quit IRC | 14:27 | |
| openstackgerrit | Merged openstack/project-config master: New charm and interface for placement split https://review.opendev.org/686251 | 14:38 |
| *** michael-beaver has joined #openstack-infra | 14:39 | |
| *** udesale has quit IRC | 14:41 | |
| beisner | tyvm :) | 14:42 |
| *** udesale has joined #openstack-infra | 14:54 | |
| *** yamamoto has joined #openstack-infra | 15:02 | |
| *** diablo_rojo has joined #openstack-infra | 15:06 | |
| openstackgerrit | Jan Gutter proposed zuul/nodepool master: Add port-cleanup-interval config option https://review.opendev.org/687024 | 15:07 |
| *** jaosorior has quit IRC | 15:11 | |
| *** udesale has quit IRC | 15:14 | |
| *** yamamoto has quit IRC | 15:14 | |
| *** ociuhandu has joined #openstack-infra | 15:19 | |
| *** iurygregory has quit IRC | 15:20 | |
| *** ramishra has quit IRC | 15:21 | |
| *** lpetrut has joined #openstack-infra | 15:21 | |
| *** ociuhandu has quit IRC | 15:23 | |
| *** slaweq has quit IRC | 15:26 | |
| *** ykarel is now known as ykarel|afk | 15:26 | |
| *** slaweq has joined #openstack-infra | 15:27 | |
| *** slaweq has quit IRC | 15:28 | |
| *** ociuhandu has joined #openstack-infra | 15:30 | |
| *** smarcet has joined #openstack-infra | 15:31 | |
| openstackgerrit | Merged openstack/project-config master: gerritbot: notify stable branch changes to #openstack-horizon https://review.opendev.org/687543 | 15:31 |
| *** slaweq has joined #openstack-infra | 15:35 | |
| *** lpetrut has quit IRC | 15:35 | |
| *** slaweq has quit IRC | 15:44 | |
| *** rpittau is now known as rpittau|afk | 15:45 | |
| *** iurygregory has joined #openstack-infra | 15:47 | |
| *** pgaxatte has quit IRC | 15:51 | |
| *** openstackgerrit has quit IRC | 15:52 | |
| *** toabctl has quit IRC | 15:53 | |
| *** smarcet has quit IRC | 16:00 | |
| *** ccamacho has quit IRC | 16:03 | |
| *** jangutter has joined #openstack-infra | 16:04 | |
| *** smarcet has joined #openstack-infra | 16:04 | |
| jangutter | slightly ontopic: I saw a post failure in the nodepool-build-image job. I presume there's a static node that needs tlc? | 16:05 |
| *** lucasagomes has quit IRC | 16:05 | |
| fungi | jangutter: we don't have any static nodes, but that could be the registry having problems. have a link to the buildset or know the project/ref? | 16:07 |
| jangutter | http://zuul.opendev.org/t/zuul/build/281ad240814a4a2fb8ab1d0dffd842a6 | 16:07 |
| *** smarcet has quit IRC | 16:08 | |
| jangutter | fungi: am I reading that right that the local registry is somehow busted? Or is port 9000 mapped through to something remote? | 16:11 |
| *** igordc has joined #openstack-infra | 16:12 | |
| fungi | looks to me like it's a local registry service | 16:13 |
| fungi | when trying to copy from the local registry to the remote one | 16:13 |
| *** ociuhandu_ has joined #openstack-infra | 16:18 | |
| *** openstackgerrit has joined #openstack-infra | 16:20 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Implement namespacing https://review.opendev.org/687613 | 16:20 |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Implement namespacing https://review.opendev.org/687613 | 16:21 |
| *** ociuhandu has quit IRC | 16:22 | |
| *** jamesmcarthur has quit IRC | 16:22 | |
| diablo_rojo | clarkb, for Infra/OpenDev and gitea at the PTG, are either group going to be doing onboarding? or just technical discussions? | 16:24 |
| *** ociuhandu_ has quit IRC | 16:25 | |
| *** gothicmindfood has joined #openstack-infra | 16:26 | |
| *** derekh has quit IRC | 16:28 | |
| fungi | note he's likely out on a boat hauling in some feisty salmon | 16:28 |
| mordred | mmm. tasty tasty salmon | 16:28 |
| fungi | i believe we said onboarding hadn't really worked in the past and it always just ended up being existing infra folks in the room | 16:28 |
| fungi | but i'd defer to his recollection | 16:29 |
| diablo_rojo | Oh right. I forgot. | 16:29 |
| diablo_rojo | about the fish :) | 16:29 |
| *** jangutter has quit IRC | 16:29 | |
| *** jamesmcarthur has joined #openstack-infra | 16:30 | |
| diablo_rojo | I will say no for now then and keep an eye out for changes this evening/tomorrow. | 16:30 |
| fungi | never forget the fish | 16:30 |
| diablo_rojo | fungi, nor my towel | 16:30 |
| * fungi approves, regardless of whether that's a south park or hitchhiker's guide to the galaxy reference | 16:31 | |
| diablo_rojo | hitchhiker :) | 16:33 |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Implement namespacing https://review.opendev.org/687613 | 16:33 |
| *** jamesmcarthur has quit IRC | 16:35 | |
| *** ykarel|afk is now known as ykarel | 16:35 | |
| *** ramishra has joined #openstack-infra | 16:37 | |
| *** jamesmcarthur has joined #openstack-infra | 16:39 | |
| *** bnemec has quit IRC | 16:45 | |
| *** bnemec has joined #openstack-infra | 16:47 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Fix authorization URL https://review.opendev.org/687622 | 16:48 |
| *** slaweq has joined #openstack-infra | 16:48 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Fix authorization URL https://review.opendev.org/687622 | 16:49 |
| *** jpena is now known as jpena|off | 16:53 | |
| openstackgerrit | James E. Blair proposed opendev/system-config master: Add public url to intermediate registry https://review.opendev.org/687623 | 16:53 |
| *** panda is now known as panda|off | 16:53 | |
| corvus | fungi, mordred: ^ | 16:53 |
| *** markvoelker has quit IRC | 16:53 | |
| *** markvoelker has joined #openstack-infra | 16:54 | |
| *** kjackal has quit IRC | 16:56 | |
| *** mriedem is now known as mriedem_afk | 16:58 | |
| *** gyee has joined #openstack-infra | 16:58 | |
| *** e0ne has quit IRC | 17:02 | |
| *** slaweq has quit IRC | 17:02 | |
| *** kjackal has joined #openstack-infra | 17:04 | |
| corvus | fungi: i'm going to manually put a local container image build of zuul-registry into production to get past this problem | 17:09 |
| *** kjackal has quit IRC | 17:16 | |
| *** priteau has quit IRC | 17:20 | |
| corvus | okay, local build is running; hopefully it will keep running long enough for 687622 to merge | 17:21 |
| fungi | noted, thanks for the heads up! | 17:21 |
| fungi | and sorry for disappearing, got sidetracked making lunch | 17:21 |
| corvus | time is an illusion | 17:22 |
| fungi | lunch time, doubly so! | 17:23 |
| * fungi places sign on his office door, "missing, presumed fed" | 17:23 | |
| corvus | hrm, it's getting errors from rax on startup | 17:24 |
| corvus | we may have 2 problems | 17:24 |
| corvus | added it to emergency file | 17:25 |
| corvus | DEBUG urllib3.connectionpool: https://identity.api.rackspacecloud.com:443 "POST /v2.0/tokens HTTP/1.1" 401 128 | 17:26 |
| fungi | huh, seems that shouldn't result in a 401 | 17:27 |
| *** ramishra has quit IRC | 17:27 | |
| fungi | supposed to be 201? | 17:27 |
| corvus | that looks like rackspace rejecting our attempt to authenticate to keystone | 17:28 |
| fungi | right | 17:28 |
| fungi | is that in the system log? | 17:28 |
| fungi | er, instance console log, whatever | 17:28 |
| corvus | it's in the container logs for zuul-registry | 17:29 |
| fungi | er | 17:29 |
| fungi | oh! for swift | 17:29 |
| * fungi is daft sometimes | 17:29 | |
| corvus | hrm. that's working for me locally | 17:31 |
| corvus | but maybe i have different sdk versions or something | 17:31 |
| fungi | and you're testing with the same credentials i guess | 17:32 |
| *** ykarel_ has joined #openstack-infra | 17:32 | |
| corvus | yep | 17:32 |
| *** ykarel_ has quit IRC | 17:33 | |
| fungi | and the funky swift tenant id which isn't the normal tenant id (unless they've finally fixed that) | 17:33 |
| mordred | corvus: there really shouldn't be anything related to auth that would be different across recent versions | 17:33 |
| corvus | wow um it fixed itself | 17:33 |
| corvus | it just kept restarting until it worked. | 17:34 |
| mordred | hooray! ... question mark? | 17:34 |
| fungi | huh | 17:35 |
| mordred | corvus: maybe rax was just having a moment? | 17:35 |
| fungi | maybe they were throttling due to a failure from earlier? | 17:35 |
| *** ykarel has quit IRC | 17:35 | |
| corvus | okay, it's up, i've re-approved 687622 | 17:38 |
| openstackgerrit | James E. Blair proposed opendev/system-config master: Add public url to intermediate registry https://review.opendev.org/687623 | 17:39 |
| corvus | fungi, mordred: ^ and i made a correction to that change ^ | 17:39 |
| fungi | ahh, thanks, i meant to check if it was listening on 443 | 17:39 |
| *** ricolin has quit IRC | 17:41 | |
| *** eernst has quit IRC | 17:43 | |
| *** smarcet has joined #openstack-infra | 17:47 | |
| *** gagehugo has joined #openstack-infra | 17:49 | |
| openstackgerrit | Merged zuul/zuul-registry master: Fix authorization URL https://review.opendev.org/687622 | 17:50 |
| corvus | i'll remove it from emergency some time after 687623 merges | 17:52 |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Implement namespacing https://review.opendev.org/687613 | 17:52 |
| *** igordc has quit IRC | 17:59 | |
| *** smarcet has quit IRC | 18:01 | |
| *** ociuhandu has joined #openstack-infra | 18:07 | |
| *** dtantsur is now known as dtantsur|afk | 18:12 | |
| openstackgerrit | Jeremy Stanley proposed opendev/irc-meetings master: Update StoryBoard meeting day/time https://review.opendev.org/687644 | 18:15 |
| *** ralonsoh has quit IRC | 18:15 | |
| *** mriedem_afk is now known as mriedem | 18:18 | |
| *** slaweq has joined #openstack-infra | 18:20 | |
| *** dpawlik has joined #openstack-infra | 18:22 | |
| *** slaweq has quit IRC | 18:25 | |
| *** e0ne has joined #openstack-infra | 18:25 | |
| openstackgerrit | Merged opendev/system-config master: Add public url to intermediate registry https://review.opendev.org/687623 | 18:28 |
| *** slaweq has joined #openstack-infra | 18:29 | |
| *** hwoarang has quit IRC | 18:37 | |
| *** slaweq has quit IRC | 18:39 | |
| *** hwoarang has joined #openstack-infra | 18:40 | |
| *** jamesmcarthur has quit IRC | 18:45 | |
| openstackgerrit | Monty Taylor proposed opendev/system-config master: Run a gerrit container (test only) https://review.opendev.org/630406 | 18:49 |
| *** e0ne has quit IRC | 19:00 | |
| *** factor has quit IRC | 19:00 | |
| *** pcaruana has quit IRC | 19:07 | |
| *** e0ne has joined #openstack-infra | 19:08 | |
| *** harlowja has joined #openstack-infra | 19:10 | |
| *** hwoarang has quit IRC | 19:11 | |
| *** mgagne has quit IRC | 19:11 | |
| *** hwoarang has joined #openstack-infra | 19:12 | |
| *** yamamoto has joined #openstack-infra | 19:12 | |
| *** yamamoto has quit IRC | 19:17 | |
| *** slaweq has joined #openstack-infra | 19:20 | |
| *** tesseract has quit IRC | 19:20 | |
| *** factor has joined #openstack-infra | 19:26 | |
| *** ociuhandu has quit IRC | 19:26 | |
| *** kopecmartin is now known as kopecmartin|off | 19:35 | |
| *** iurygregory has quit IRC | 19:37 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-registry master: Raise a 404 when we don't find a blob from swift https://review.opendev.org/687657 | 19:39 |
| *** e0ne has quit IRC | 19:56 | |
| *** dpawlik has quit IRC | 19:59 | |
| *** jcoufal has quit IRC | 20:00 | |
| *** slaweq has quit IRC | 20:06 | |
| *** slaweq has joined #openstack-infra | 20:11 | |
| *** slaweq has quit IRC | 20:16 | |
| *** pcaruana has joined #openstack-infra | 20:33 | |
| *** ociuhandu has joined #openstack-infra | 20:37 | |
| ianw | https://techcrunch.com/2019/10/09/suses-openstack-cloud-dissipates/ ... that's a bummer | 20:37 |
| *** ociuhandu has quit IRC | 20:39 | |
| fungi | yep, thinking happy thoughts for our valued colleagues whose work is funded by suse | 20:39 |
| *** pcaruana has quit IRC | 20:40 | |
| *** lpetrut has joined #openstack-infra | 20:41 | |
| *** Goneri has quit IRC | 20:42 | |
| ianw | project-core: if i could get some eyes on the project-config bits of https://review.opendev.org/#/q/topic:centos8+(status:open) for centos8 support -- i just released dib 2.28.0 which will have all the support required. i can babysit centos8 image generation | 20:45 |
| ianw | i have locally built centos8 images with all our project-config elements -- something always seems to happen IRL but I think it's gtg | 20:46 |
| *** lpetrut has quit IRC | 20:52 | |
| ianw | config-core: ^ even | 20:53 |
| openstackgerrit | James E. Blair proposed opendev/system-config master: Temporarily disable intermediate registry prune https://review.opendev.org/687673 | 20:56 |
| corvus | mordred, fungi: ^ | 20:57 |
| *** xek_ has quit IRC | 21:04 | |
| *** whoami-rajat has quit IRC | 21:06 | |
| openstackgerrit | James E. Blair proposed opendev/system-config master: Temporarily disable intermediate registry prune https://review.opendev.org/687673 | 21:11 |
| *** lpetrut has joined #openstack-infra | 21:13 | |
| corvus | mordred: now that i understand this situation better, i appreciate that your patch in 687657 still gives us a nicely greppable error message | 21:15 |
| mordred | yay error messages! | 21:16 |
| *** gfidente has quit IRC | 21:25 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-registry master: HEAD object after PUT https://review.opendev.org/687681 | 21:25 |
| mordred | corvus, fungi: ^^ that should trigger the head | 21:26 |
| fungi | mordred: you were a bit shy with the ) key | 21:27 |
| corvus | mordred: lgtm; if we want, we could get the results of that and debug log size or something, but just the result code will probably answer our current immediate question | 21:28 |
| corvus | i mean, once you add enough ) | 21:28 |
| * mordred likes being thrifty with ) | 21:28 | |
| *** rfolco has quit IRC | 21:28 | |
| openstackgerrit | Merged zuul/zuul-registry master: Raise a 404 when we don't find a blob from swift https://review.opendev.org/687657 | 21:29 |
| fungi | my ) key is almost as worn down as my ( key | 21:29 |
| *** lpetrut has quit IRC | 21:30 | |
| *** goldyfruit has quit IRC | 21:32 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-registry master: HEAD object after PUT https://review.opendev.org/687681 | 21:34 |
| mordred | fungi, corvus: how's that look? | 21:34 |
| *** goldyfruit has joined #openstack-infra | 21:38 | |
| *** sreejithp_ has quit IRC | 21:40 | |
| *** pkopec has quit IRC | 21:41 | |
| *** cloudnull is now known as cloudnull-afk | 21:42 | |
| openstackgerrit | Nate Johnston proposed openstack/project-config master: Update neutron-tempest-plugin grafana dashboard https://review.opendev.org/687686 | 21:42 |
| corvus | mordred: should that be content-length not size? | 21:44 |
| fungi | oh, yeah, probably so | 21:44 |
| *** diablo_rojo has quit IRC | 21:47 | |
| *** rh-jelabarre has quit IRC | 21:48 | |
| *** eharney has quit IRC | 21:49 | |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** rcernin has joined #openstack-infra | 22:14 | |
| *** slaweq has quit IRC | 22:15 | |
| openstackgerrit | Merged opendev/system-config master: Temporarily disable intermediate registry prune https://review.opendev.org/687673 | 22:16 |
| *** xenos76 has quit IRC | 22:22 | |
| *** slaweq has joined #openstack-infra | 22:25 | |
| *** efried has quit IRC | 22:26 | |
| *** markvoelker has quit IRC | 22:28 | |
| *** slaweq has quit IRC | 22:29 | |
| *** mriedem has quit IRC | 22:30 | |
| *** ociuhandu has joined #openstack-infra | 22:30 | |
| *** ociuhandu has quit IRC | 22:35 | |
| *** jamesmcarthur has joined #openstack-infra | 22:42 | |
| *** efried has joined #openstack-infra | 22:45 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Run docker and podman push/pull tests https://review.opendev.org/687692 | 22:46 |
| corvus | i have removed the registry from emergency | 22:50 |
| openstackgerrit | Merged zuul/nodepool master: Assign static 'building' nodes in cleanup handler https://review.opendev.org/687261 | 23:01 |
| *** tkajinam has joined #openstack-infra | 23:02 | |
| *** yamamoto has joined #openstack-infra | 23:10 | |
| ianw | so unattended-upgrades is actually broken on bionic (https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1823872) due to a qemu packaging error. | 23:14 |
| openstack | Launchpad bug 1823872 in unattended-upgrades (Ubuntu Trusty) "Fixing fsfreeze-hook can break unattended upgrades" [Medium,Confirmed] - Assigned to Balint Reczey (rbalint) | 23:14 |
| ianw | #status log manually installed latest unattended-upgrades on bridge.o.o to get it working again | 23:14 |
| ianw | it seems we have no statusbot | 23:16 |
| *** openstackstatus has joined #openstack-infra | 23:18 | |
| *** ChanServ sets mode: +v openstackstatus | 23:18 | |
| ianw | #status log restarted stausbot as it was missing in #openstack-infra | 23:19 |
| openstackstatus | ianw: finished logging | 23:19 |
| ianw | #status log manually installed latest unattended-upgrades on bridge.o.o to get it working again (https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1823872) | 23:19 |
| openstackstatus | ianw: finished logging | 23:19 |
| openstack | Launchpad bug 1823872 in unattended-upgrades (Ubuntu Trusty) "Fixing fsfreeze-hook can break unattended upgrades" [Medium,Confirmed] - Assigned to Balint Reczey (rbalint) | 23:19 |
| ianw | so i think it's only if you have a broken qemu-guest-agent + u-a versions in tadem; which means it has to be a fairly old bionic server. thus i think only bridge.o.o is affected | 23:26 |
| *** yamamoto has quit IRC | 23:27 | |
| *** gyee has quit IRC | 23:27 | |
| *** tosky has quit IRC | 23:30 | |
| *** yamamoto has joined #openstack-infra | 23:30 | |
| *** dchen has joined #openstack-infra | 23:32 | |
| fungi | ahh, and yeah who knows if the qemu-guest-agent on there was even in ubuntu, it could have been custom added by the provider as a dependency of their provider-specific agent shim | 23:35 |
| fungi | looks like we uninstalled rax-nova-agent on it at some point, but likely not the qemu-guest-agent which was on there as a dependency | 23:36 |
| *** dychen has joined #openstack-infra | 23:36 | |
| fungi | (rax-nova-agent is in a removed with configuration not yet purged state according to dpkg -l) | 23:37 |
| *** goldyfruit has quit IRC | 23:42 | |
| *** jamesmcarthur has quit IRC | 23:43 | |
| *** dklyle has quit IRC | 23:47 | |
| *** david-lyle has joined #openstack-infra | 23:47 | |
| *** markvoelker has joined #openstack-infra | 23:48 | |
| *** slaweq has joined #openstack-infra | 23:56 | |
| *** aaronsheffield has quit IRC | 23:58 | |
| *** markvoelker has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!