| *** gyee has quit IRC | 00:06 | |
| *** jamesmcarthur has joined #openstack-infra | 00:09 | |
| *** michael-beaver has quit IRC | 00:10 | |
| *** ianychoi has quit IRC | 00:12 | |
| *** rlandy has quit IRC | 00:13 | |
| *** ianychoi has joined #openstack-infra | 00:15 | |
| *** goldyfruit_ has joined #openstack-infra | 00:23 | |
| *** jamesmcarthur has quit IRC | 00:24 | |
| *** ianychoi has quit IRC | 00:32 | |
| *** jamesmcarthur has joined #openstack-infra | 00:35 | |
| *** armax has quit IRC | 00:46 | |
| *** ianychoi has joined #openstack-infra | 00:48 | |
| *** jamesmcarthur has quit IRC | 00:49 | |
| *** rosmaita has left #openstack-infra | 00:49 | |
| *** jamesmcarthur has joined #openstack-infra | 00:51 | |
| *** jamesmcarthur has quit IRC | 00:54 | |
| *** jamesmcarthur has joined #openstack-infra | 00:55 | |
| *** hongbin has joined #openstack-infra | 01:01 | |
| *** jamesmcarthur has quit IRC | 01:07 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal : update mirrors for Centos 8 https://review.opendev.org/684439 | 01:08 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: dib-python : handle centos 8 https://review.opendev.org/684441 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update locales for Centos 8 https://review.opendev.org/684442 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update redhat-common pkg-map for centos 8 https://review.opendev.org/684449 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add environment switch for centos8 to use dnf https://review.opendev.org/684459 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use $YUM instead of direct calls in more places https://review.opendev.org/684460 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal: Don't install yum, install libcurl https://review.opendev.org/684461 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: pip-and-virtualenv : deprecate source for CentOS 8, new variables https://review.opendev.org/684462 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: Use wrappers to call pip for glean install https://review.opendev.org/685643 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Fix networking for CentOS 8 https://review.opendev.org/684513 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add NetworkManager and dhcp-client for CentOS 8 https://review.opendev.org/686090 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: CentOS 8 minimal testing and support https://review.opendev.org/684860 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] use mirrors for functional test builds https://review.opendev.org/685885 | 01:08 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: default to NetworkManager for CentOS and Fedora https://review.opendev.org/686319 | 01:08 |
| *** whoami-rajat has joined #openstack-infra | 01:28 | |
| *** hwoarang has quit IRC | 01:42 | |
| *** hwoarang has joined #openstack-infra | 01:50 | |
| *** ykarel|away has joined #openstack-infra | 01:58 | |
| *** anteaya has quit IRC | 01:59 | |
| *** igordc has quit IRC | 02:01 | |
| *** apetrich has quit IRC | 02:10 | |
| *** jamesmcarthur has joined #openstack-infra | 02:18 | |
| *** yamamoto has quit IRC | 02:22 | |
| *** yamamoto has joined #openstack-infra | 02:29 | |
| *** roman_g has quit IRC | 02:34 | |
| *** rh-jelabarre has quit IRC | 02:36 | |
| *** factor has joined #openstack-infra | 02:47 | |
| *** tinwood has quit IRC | 02:50 | |
| *** hongbin has quit IRC | 02:51 | |
| *** tinwood has joined #openstack-infra | 02:52 | |
| *** jamesmcarthur has quit IRC | 02:56 | |
| *** jamesmcarthur has joined #openstack-infra | 02:58 | |
| *** markvoelker has joined #openstack-infra | 03:07 | |
| *** markvoelker has quit IRC | 03:11 | |
| *** eernst has joined #openstack-infra | 03:20 | |
| *** markvoelker has joined #openstack-infra | 03:21 | |
| *** markvoelker has quit IRC | 03:24 | |
| *** eernst has quit IRC | 03:26 | |
| *** eernst has joined #openstack-infra | 03:27 | |
| *** markvoelker has joined #openstack-infra | 03:30 | |
| *** markvoelker has quit IRC | 03:32 | |
| *** markvoelker has joined #openstack-infra | 03:32 | |
| *** eernst has quit IRC | 03:33 | |
| *** eernst has joined #openstack-infra | 03:35 | |
| *** markvoelker has quit IRC | 03:37 | |
| *** dave-mccowan has quit IRC | 03:37 | |
| *** eernst has quit IRC | 03:38 | |
| *** exsdev has quit IRC | 03:44 | |
| *** jamesmcarthur has quit IRC | 03:45 | |
| *** jamesmcarthur has joined #openstack-infra | 03:47 | |
| *** exsdev has joined #openstack-infra | 03:48 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add security mirror override for debian-minimal https://review.opendev.org/686328 | 03:50 |
| *** kjackal has joined #openstack-infra | 03:52 | |
| *** jamesmcarthur has quit IRC | 03:57 | |
| *** jamesmcarthur has joined #openstack-infra | 03:57 | |
| *** jamesmcarthur has quit IRC | 04:02 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: default to NetworkManager for CentOS and Fedora https://review.opendev.org/686319 | 04:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: simple-init: Use wrappers to call pip for glean install https://review.opendev.org/685643 | 04:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Fix networking for CentOS 8 https://review.opendev.org/684513 | 04:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add NetworkManager and dhcp-client for CentOS 8 https://review.opendev.org/686090 | 04:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: CentOS 8 minimal testing and support https://review.opendev.org/684860 | 04:06 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] use mirrors for functional test builds https://review.opendev.org/685885 | 04:06 |
| *** markvoelker has joined #openstack-infra | 04:07 | |
| *** udesale has joined #openstack-infra | 04:13 | |
| *** ociuhandu has joined #openstack-infra | 04:14 | |
| *** ociuhandu has quit IRC | 04:18 | |
| *** ykarel|away has quit IRC | 04:22 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] use mirrors for functional test builds https://review.opendev.org/685885 | 04:23 |
| *** odicha has joined #openstack-infra | 04:25 | |
| *** odicha has quit IRC | 04:28 | |
| *** rcernin has quit IRC | 04:29 | |
| *** surpatil has joined #openstack-infra | 04:29 | |
| *** odicha has joined #openstack-infra | 04:32 | |
| *** rcernin has joined #openstack-infra | 04:32 | |
| *** jamesmcarthur has joined #openstack-infra | 04:33 | |
| *** markvoelker has quit IRC | 04:37 | |
| *** soniya29 has joined #openstack-infra | 04:37 | |
| *** jamesmcarthur has quit IRC | 04:40 | |
| *** ykarel|away has joined #openstack-infra | 04:41 | |
| *** ykarel|away is now known as ykarel | 04:56 | |
| *** markvoelker has joined #openstack-infra | 05:01 | |
| *** spsurya has joined #openstack-infra | 05:05 | |
| *** markvoelker has quit IRC | 05:06 | |
| *** markvoelker has joined #openstack-infra | 05:11 | |
| *** markvoelker has quit IRC | 05:15 | |
| *** dpawlik has joined #openstack-infra | 05:17 | |
| *** jamesmcarthur has joined #openstack-infra | 05:17 | |
| openstackgerrit | Merged opendev/base-jobs master: Add docs templates https://review.opendev.org/686228 | 05:19 |
| *** markvoelker has joined #openstack-infra | 05:20 | |
| *** jamesmcarthur has quit IRC | 05:23 | |
| *** markvoelker has quit IRC | 05:24 | |
| openstackgerrit | Merged openstack/openstack-zuul-jobs master: Bye, bye readthedocs-failing https://review.opendev.org/685618 | 05:26 |
| *** markvoelker has joined #openstack-infra | 05:29 | |
| *** markvoelker has quit IRC | 05:34 | |
| *** markvoelker has joined #openstack-infra | 05:38 | |
| *** markvoelker has quit IRC | 05:43 | |
| *** ykarel is now known as ykarel|meeting | 05:44 | |
| *** markvoelker has joined #openstack-infra | 05:47 | |
| *** ykarel_ has joined #openstack-infra | 05:48 | |
| *** ykarel|meeting has quit IRC | 05:50 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add openstacksdk to opendev tenant https://review.opendev.org/686334 | 05:51 |
| *** markvoelker has quit IRC | 05:52 | |
| AJaeger | config-core, see http://zuul.opendev.org/t/opendev/config-errors ^ | 05:52 |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Add openstacksdk /ara to opendev tenant https://review.opendev.org/686334 | 05:53 |
| AJaeger | missed one ... | 05:54 |
| *** markvoelker has joined #openstack-infra | 05:57 | |
| *** markvoelker has quit IRC | 06:02 | |
| *** lpetrut has joined #openstack-infra | 06:02 | |
| *** lpetrut has quit IRC | 06:03 | |
| *** lpetrut has joined #openstack-infra | 06:03 | |
| *** markvoelker has joined #openstack-infra | 06:06 | |
| *** surpatil has quit IRC | 06:10 | |
| *** markvoelker has quit IRC | 06:10 | |
| *** surpatil has joined #openstack-infra | 06:10 | |
| *** surpatil has quit IRC | 06:12 | |
| *** surpatil has joined #openstack-infra | 06:13 | |
| *** markvoelker has joined #openstack-infra | 06:15 | |
| *** jamesmcarthur has joined #openstack-infra | 06:19 | |
| *** markvoelker has quit IRC | 06:20 | |
| *** ianychoi has quit IRC | 06:20 | |
| *** jaosorior has joined #openstack-infra | 06:21 | |
| *** ianychoi has joined #openstack-infra | 06:23 | |
| *** markvoelker has joined #openstack-infra | 06:24 | |
| *** markvoelker has quit IRC | 06:29 | |
| *** ianychoi has quit IRC | 06:29 | |
| *** jamesmcarthur has quit IRC | 06:30 | |
| *** pgaxatte has joined #openstack-infra | 06:30 | |
| *** surpatil has quit IRC | 06:30 | |
| *** ianychoi has joined #openstack-infra | 06:31 | |
| *** xek has joined #openstack-infra | 06:32 | |
| *** markvoelker has joined #openstack-infra | 06:33 | |
| *** surpatil has joined #openstack-infra | 06:34 | |
| *** markvoelker has quit IRC | 06:38 | |
| *** markvoelker has joined #openstack-infra | 06:43 | |
| *** markvoelker has quit IRC | 06:47 | |
| *** yamamoto has quit IRC | 06:47 | |
| *** jamesmcarthur has joined #openstack-infra | 06:49 | |
| *** yamamoto has joined #openstack-infra | 06:51 | |
| *** markvoelker has joined #openstack-infra | 06:52 | |
| *** markvoelker has quit IRC | 06:57 | |
| *** jamesmcarthur has quit IRC | 06:58 | |
| *** udesale has quit IRC | 06:58 | |
| *** udesale has joined #openstack-infra | 06:59 | |
| *** lpetrut has quit IRC | 06:59 | |
| *** lpetrut has joined #openstack-infra | 07:00 | |
| *** jamesmcarthur has joined #openstack-infra | 07:01 | |
| *** markvoelker has joined #openstack-infra | 07:01 | |
| *** udesale has quit IRC | 07:02 | |
| *** udesale has joined #openstack-infra | 07:02 | |
| *** markvoelker has quit IRC | 07:05 | |
| *** rcernin has quit IRC | 07:06 | |
| *** markvoelker has joined #openstack-infra | 07:10 | |
| *** tesseract has joined #openstack-infra | 07:12 | |
| *** markvoelker has quit IRC | 07:15 | |
| *** tosky has joined #openstack-infra | 07:16 | |
| *** hjensas has joined #openstack-infra | 07:17 | |
| *** markvoelker has joined #openstack-infra | 07:19 | |
| *** ociuhandu has joined #openstack-infra | 07:22 | |
| *** ociuhandu has quit IRC | 07:23 | |
| *** markvoelker has quit IRC | 07:24 | |
| *** ykarel_ is now known as ykarel|meeting | 07:24 | |
| *** pcaruana has joined #openstack-infra | 07:28 | |
| *** piotrowskim has joined #openstack-infra | 07:29 | |
| *** markvoelker has joined #openstack-infra | 07:29 | |
| *** markvoelker has quit IRC | 07:34 | |
| *** ociuhandu has joined #openstack-infra | 07:36 | |
| *** ykarel_ has joined #openstack-infra | 07:36 | |
| *** ykarel|meeting has quit IRC | 07:36 | |
| *** ykarel_ is now known as ykarel | 07:37 | |
| *** markvoelker has joined #openstack-infra | 07:38 | |
| *** iurygregory has quit IRC | 07:39 | |
| *** surpatil is now known as surpatil|lunch | 07:41 | |
| *** markvoelker has quit IRC | 07:43 | |
| *** ianychoi has quit IRC | 07:45 | |
| *** markvoelker has joined #openstack-infra | 07:47 | |
| *** udesale has quit IRC | 07:48 | |
| *** udesale has joined #openstack-infra | 07:48 | |
| *** lpetrut has quit IRC | 07:48 | |
| *** lpetrut has joined #openstack-infra | 07:49 | |
| *** jpena|off is now known as jpena | 07:49 | |
| *** pkopec has joined #openstack-infra | 07:49 | |
| *** ykarel is now known as ykarel|lunch | 07:49 | |
| *** markvoelker has quit IRC | 07:52 | |
| *** e0ne has joined #openstack-infra | 07:52 | |
| *** e0ne has quit IRC | 07:53 | |
| *** kopecmartin|off is now known as kopecmartin | 07:53 | |
| *** yamamoto has quit IRC | 07:54 | |
| *** rpittau|afk is now known as rpittau | 07:59 | |
| *** tkajinam has quit IRC | 08:00 | |
| *** ociuhandu has quit IRC | 08:04 | |
| *** ociuhandu has joined #openstack-infra | 08:04 | |
| *** ociuhandu has quit IRC | 08:05 | |
| *** ociuhandu has joined #openstack-infra | 08:05 | |
| *** hjensas has quit IRC | 08:05 | |
| *** xenos76 has joined #openstack-infra | 08:05 | |
| *** markvoelker has joined #openstack-infra | 08:06 | |
| *** slaweq has joined #openstack-infra | 08:06 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] use mirrors for functional test builds https://review.opendev.org/685885 | 08:09 |
| *** markvoelker has quit IRC | 08:11 | |
| *** yamamoto has joined #openstack-infra | 08:13 | |
| *** shardy has joined #openstack-infra | 08:14 | |
| *** shardy has quit IRC | 08:15 | |
| *** markvoelker has joined #openstack-infra | 08:15 | |
| *** jamesmcarthur has quit IRC | 08:17 | |
| *** yamamoto has quit IRC | 08:17 | |
| *** shardy has joined #openstack-infra | 08:19 | |
| *** jaosorior has quit IRC | 08:19 | |
| *** yamamoto has joined #openstack-infra | 08:19 | |
| *** shardy has quit IRC | 08:19 | |
| *** markvoelker has quit IRC | 08:20 | |
| *** panda|off is now known as panda | 08:20 | |
| *** iurygregory has joined #openstack-infra | 08:21 | |
| *** markvoelker has joined #openstack-infra | 08:24 | |
| *** dchen has quit IRC | 08:26 | |
| *** markvoelker has quit IRC | 08:29 | |
| *** ccamacho has joined #openstack-infra | 08:31 | |
| *** jbadiapa has joined #openstack-infra | 08:34 | |
| *** markvoelker has joined #openstack-infra | 08:34 | |
| openstackgerrit | Gabor Lekeny proposed zuul/zuul master: Fix github webhook authentication https://review.opendev.org/686355 | 08:34 |
| *** hjensas has joined #openstack-infra | 08:36 | |
| openstackgerrit | Gabor Lekeny proposed zuul/zuul master: Fix github webhook authentication https://review.opendev.org/686355 | 08:38 |
| *** markvoelker has quit IRC | 08:39 | |
| *** ykarel|lunch is now known as ykarel | 08:40 | |
| *** slaweq has quit IRC | 08:41 | |
| *** derekh has joined #openstack-infra | 08:42 | |
| *** yamamoto has quit IRC | 08:43 | |
| *** roman_g has joined #openstack-infra | 08:44 | |
| *** e0ne has joined #openstack-infra | 08:49 | |
| *** markvoelker has joined #openstack-infra | 08:52 | |
| *** slaweq has joined #openstack-infra | 08:52 | |
| *** markvoelker has quit IRC | 08:57 | |
| *** jbadiapa has quit IRC | 08:59 | |
| *** markvoelker has joined #openstack-infra | 09:01 | |
| openstackgerrit | Merged openstack/project-config master: Add openstacksdk /ara to opendev tenant https://review.opendev.org/686334 | 09:05 |
| *** markvoelker has quit IRC | 09:06 | |
| openstackgerrit | Merged zuul/zuul-registry master: Add tox configuration and fixe flake8 errors https://review.opendev.org/686230 | 09:08 |
| *** yamamoto has joined #openstack-infra | 09:08 | |
| *** markvoelker has joined #openstack-infra | 09:10 | |
| *** gfidente has joined #openstack-infra | 09:13 | |
| *** jamesmcarthur has joined #openstack-infra | 09:14 | |
| *** markvoelker has quit IRC | 09:15 | |
| *** ricolin has quit IRC | 09:15 | |
| *** diablo_rojo has joined #openstack-infra | 09:17 | |
| *** jamesmcarthur has quit IRC | 09:18 | |
| *** markvoelker has joined #openstack-infra | 09:20 | |
| mordred | AJaeger: now that train is out - do you know what the story is with python2/python3? I think I remember that U is where python2 support goes away, yes? | 09:23 |
| *** diablo_rojo has quit IRC | 09:24 | |
| *** markvoelker has quit IRC | 09:25 | |
| *** hjensas has quit IRC | 09:25 | |
| *** markvoelker has joined #openstack-infra | 09:29 | |
| *** markvoelker has quit IRC | 09:34 | |
| *** gfidente has quit IRC | 09:38 | |
| *** markvoelker has joined #openstack-infra | 09:38 | |
| mordred | AJaeger: nm - I have learned the infos | 09:40 |
| *** markvoelker has quit IRC | 09:43 | |
| *** xenos76 has quit IRC | 09:44 | |
| *** xenos76 has joined #openstack-infra | 09:45 | |
| openstackgerrit | Stephen Finucane proposed openstack/openstack-zuul-jobs master: Create openstack-tox-mypy job https://review.opendev.org/686366 | 09:46 |
| *** markvoelker has joined #openstack-infra | 09:48 | |
| *** ociuhandu has quit IRC | 09:50 | |
| *** ociuhandu has joined #openstack-infra | 09:51 | |
| *** markvoelker has quit IRC | 09:52 | |
| *** udesale has quit IRC | 09:53 | |
| *** udesale has joined #openstack-infra | 09:54 | |
| *** pkopec has quit IRC | 09:55 | |
| *** surpatil has joined #openstack-infra | 09:56 | |
| *** surpatil|lunch has quit IRC | 09:56 | |
| *** markvoelker has joined #openstack-infra | 09:57 | |
| *** jaosorior has joined #openstack-infra | 09:59 | |
| *** markvoelker has quit IRC | 10:01 | |
| *** yamamoto has quit IRC | 10:02 | |
| *** yamamoto has joined #openstack-infra | 10:05 | |
| *** markvoelker has joined #openstack-infra | 10:06 | |
| *** jaosorior has quit IRC | 10:06 | |
| *** markvoelker has quit IRC | 10:11 | |
| *** yamamoto has quit IRC | 10:13 | |
| openstackgerrit | Merged zuul/nodepool master: Add statsd_exporter mapping https://review.opendev.org/660473 | 10:14 |
| *** jamesmcarthur has joined #openstack-infra | 10:15 | |
| *** markvoelker has joined #openstack-infra | 10:15 | |
| *** diablo_rojo has joined #openstack-infra | 10:16 | |
| *** jamesmcarthur has quit IRC | 10:19 | |
| *** VedaAnnayappa has joined #openstack-infra | 10:20 | |
| *** markvoelker has quit IRC | 10:20 | |
| *** mgoddard has quit IRC | 10:23 | |
| *** jaosorior has joined #openstack-infra | 10:24 | |
| *** pkopec has joined #openstack-infra | 10:24 | |
| * pkopec back | 10:24 | |
| *** markvoelker has joined #openstack-infra | 10:24 | |
| *** mgoddard has joined #openstack-infra | 10:26 | |
| *** markvoelker has quit IRC | 10:29 | |
| *** yamamoto has joined #openstack-infra | 10:31 | |
| *** markvoelker has joined #openstack-infra | 10:34 | |
| *** pkopec has quit IRC | 10:35 | |
| *** pkopec has joined #openstack-infra | 10:36 | |
| *** markvoelker has quit IRC | 10:38 | |
| *** diablo_rojo has quit IRC | 10:40 | |
| *** markvoelker has joined #openstack-infra | 10:43 | |
| *** rcernin has joined #openstack-infra | 10:43 | |
| *** markvoelker has quit IRC | 10:47 | |
| *** jaosorior has quit IRC | 10:51 | |
| *** markvoelker has joined #openstack-infra | 10:52 | |
| *** pgaxatte has quit IRC | 10:55 | |
| *** markvoelker has quit IRC | 10:57 | |
| *** jpena is now known as jpena|lunch | 11:00 | |
| *** markvoelker has joined #openstack-infra | 11:01 | |
| *** ociuhandu has quit IRC | 11:02 | |
| *** ociuhandu has joined #openstack-infra | 11:02 | |
| *** markvoelker has quit IRC | 11:06 | |
| *** ociuhandu has quit IRC | 11:07 | |
| openstackgerrit | Gabor Lekeny proposed zuul/zuul master: Fix github webhook authentication https://review.opendev.org/686355 | 11:10 |
| *** markvoelker has joined #openstack-infra | 11:10 | |
| *** markvoelker has quit IRC | 11:15 | |
| *** jamesmcarthur has joined #openstack-infra | 11:16 | |
| *** udesale has quit IRC | 11:17 | |
| *** lucasagomes has joined #openstack-infra | 11:17 | |
| *** VedaAnnayappa has quit IRC | 11:18 | |
| *** markvoelker has joined #openstack-infra | 11:20 | |
| *** jamesmcarthur has quit IRC | 11:20 | |
| *** pkopec has quit IRC | 11:22 | |
| *** ociuhandu has joined #openstack-infra | 11:23 | |
| *** markvoelker has quit IRC | 11:24 | |
| *** ociuhandu has quit IRC | 11:25 | |
| *** jamesmcarthur has joined #openstack-infra | 11:26 | |
| *** ociuhandu has joined #openstack-infra | 11:26 | |
| openstackgerrit | Jeremy Stanley proposed openstack/project-config master: Override security mirror for Debian image builds https://review.opendev.org/686204 | 11:28 |
| *** markvoelker has joined #openstack-infra | 11:29 | |
| *** pkopec has joined #openstack-infra | 11:31 | |
| *** markvoelker has quit IRC | 11:34 | |
| *** rcernin has quit IRC | 11:34 | |
| jrosser | fungi: i made a hacky patch which works around that security mirror issue for now https://review.opendev.org/#/c/686348/3 | 11:35 |
| *** jamesmcarthur has quit IRC | 11:35 | |
| *** jamesmcarthur has joined #openstack-infra | 11:37 | |
| *** markvoelker has joined #openstack-infra | 11:38 | |
| fungi | jrosser: that works | 11:41 |
| jrosser | we just need to fix a couple more OSA bugs and we can get stuff merging again, then once the dib bits and images are better i'll revert that | 11:42 |
| *** markvoelker has quit IRC | 11:42 | |
| *** tosky_ has joined #openstack-infra | 11:44 | |
| *** dosaboy has quit IRC | 11:45 | |
| *** tosky has quit IRC | 11:46 | |
| *** soniya29 has quit IRC | 11:47 | |
| *** ociuhandu has quit IRC | 11:47 | |
| *** ociuhandu has joined #openstack-infra | 11:47 | |
| *** markvoelker has joined #openstack-infra | 11:47 | |
| *** ociuhandu has quit IRC | 11:48 | |
| *** ociuhandu has joined #openstack-infra | 11:49 | |
| *** jamesmcarthur has quit IRC | 11:51 | |
| *** markvoelker has quit IRC | 11:52 | |
| *** pgaxatte has joined #openstack-infra | 11:52 | |
| *** jamesmcarthur has joined #openstack-infra | 11:52 | |
| *** prometheanfire has quit IRC | 11:54 | |
| *** dosaboy has joined #openstack-infra | 11:54 | |
| *** ociuhandu has quit IRC | 11:54 | |
| *** prometheanfire has joined #openstack-infra | 11:55 | |
| *** markvoelker has joined #openstack-infra | 11:57 | |
| *** rh-jelabarre has joined #openstack-infra | 11:57 | |
| *** jamesmcarthur has quit IRC | 11:59 | |
| *** rh-jelabarre has quit IRC | 11:59 | |
| *** rh-jelabarre has joined #openstack-infra | 11:59 | |
| *** markvoelker has quit IRC | 12:01 | |
| *** tosky_ is now known as tosky | 12:01 | |
| *** gfidente has joined #openstack-infra | 12:02 | |
| *** dosaboy has quit IRC | 12:03 | |
| *** dosaboy has joined #openstack-infra | 12:03 | |
| *** ociuhandu has joined #openstack-infra | 12:03 | |
| *** xenos76 has quit IRC | 12:05 | |
| *** yamamoto has quit IRC | 12:05 | |
| *** markvoelker has joined #openstack-infra | 12:05 | |
| *** jamesmcarthur has joined #openstack-infra | 12:07 | |
| *** ociuhandu has quit IRC | 12:07 | |
| *** jpena|lunch is now known as jpena | 12:12 | |
| *** soniya29 has joined #openstack-infra | 12:15 | |
| *** markvoelker has quit IRC | 12:17 | |
| *** markvoelker has joined #openstack-infra | 12:17 | |
| *** markvoelker has quit IRC | 12:19 | |
| *** goldyfruit_ has quit IRC | 12:19 | |
| *** jamesmcarthur has quit IRC | 12:19 | |
| *** jamesmcarthur has joined #openstack-infra | 12:20 | |
| *** nhicher has quit IRC | 12:20 | |
| *** yamamoto has joined #openstack-infra | 12:20 | |
| *** pkopec has quit IRC | 12:21 | |
| openstackgerrit | Jeremy Stanley proposed openstack/project-config master: Override security mirror for Debian image builds https://review.opendev.org/686204 | 12:23 |
| *** pkopec has joined #openstack-infra | 12:27 | |
| *** nhicher has joined #openstack-infra | 12:27 | |
| *** ociuhandu has joined #openstack-infra | 12:29 | |
| *** jamesmcarthur has quit IRC | 12:33 | |
| *** rlandy has joined #openstack-infra | 12:34 | |
| *** ociuhandu has quit IRC | 12:36 | |
| *** ociuhandu has joined #openstack-infra | 12:38 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul master: Authorization rules: support YAML nested dictionaries https://review.opendev.org/684790 | 12:38 |
| *** sthussey has joined #openstack-infra | 12:43 | |
| *** ociuhandu has quit IRC | 12:44 | |
| openstackgerrit | Matthieu Huin proposed openstack/project-config master: Add sphinx-contrib/openapi to untrusted github projects https://review.opendev.org/686392 | 12:45 |
| *** jamesmcarthur has joined #openstack-infra | 12:52 | |
| *** pkopec has quit IRC | 12:53 | |
| *** pkopec has joined #openstack-infra | 12:55 | |
| *** markvoelker has joined #openstack-infra | 13:01 | |
| *** ociuhandu has joined #openstack-infra | 13:01 | |
| *** kjackal_v2 has joined #openstack-infra | 13:06 | |
| *** pkopec has quit IRC | 13:09 | |
| *** kjackal has quit IRC | 13:09 | |
| *** pkopec has joined #openstack-infra | 13:17 | |
| *** xenos76 has joined #openstack-infra | 13:18 | |
| *** rosmaita has joined #openstack-infra | 13:19 | |
| *** yamamoto has quit IRC | 13:20 | |
| *** mriedem has joined #openstack-infra | 13:22 | |
| *** pkopec has quit IRC | 13:22 | |
| *** pkopec has joined #openstack-infra | 13:23 | |
| *** goldyfruit_ has joined #openstack-infra | 13:23 | |
| *** ociuhandu has quit IRC | 13:29 | |
| *** ociuhandu has joined #openstack-infra | 13:30 | |
| *** pkopec has quit IRC | 13:31 | |
| *** udesale has joined #openstack-infra | 13:33 | |
| *** pkopec has joined #openstack-infra | 13:34 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-registry master: WIP Consume typing from openstacksdk and keystoneauth https://review.opendev.org/686403 | 13:35 |
| *** jamesmcarthur has quit IRC | 13:40 | |
| *** ociuhandu has quit IRC | 13:47 | |
| *** ociuhandu has joined #openstack-infra | 13:47 | |
| *** ociuhandu has quit IRC | 13:48 | |
| *** ociuhandu has joined #openstack-infra | 13:48 | |
| *** yamamoto has joined #openstack-infra | 13:51 | |
| *** surpatil has quit IRC | 13:54 | |
| *** soniya29 has quit IRC | 13:54 | |
| *** david-lyle has quit IRC | 13:55 | |
| *** david-lyle has joined #openstack-infra | 13:55 | |
| *** ociuhandu has quit IRC | 13:57 | |
| *** ociuhandu has joined #openstack-infra | 13:58 | |
| *** ociuhandu has quit IRC | 14:03 | |
| openstackgerrit | Christopher Dearborn proposed openstack/project-config master: Update python-dracclient gate python3 support https://review.opendev.org/686409 | 14:04 |
| *** yamamoto has quit IRC | 14:04 | |
| *** odicha has quit IRC | 14:05 | |
| *** ociuhandu has joined #openstack-infra | 14:06 | |
| *** otherwiseguy_ has joined #openstack-infra | 14:06 | |
| *** haleyb has quit IRC | 14:09 | |
| *** otherwiseguy has quit IRC | 14:09 | |
| *** ociuhandu has quit IRC | 14:10 | |
| *** jamesmcarthur has joined #openstack-infra | 14:14 | |
| *** lpetrut has quit IRC | 14:16 | |
| *** Goneri has joined #openstack-infra | 14:17 | |
| *** ricolin has joined #openstack-infra | 14:17 | |
| fungi | if any system-config reviewers have a spare moment to review 682417 and 682425 i'll take care of retiring those mailing lists | 14:28 |
| fungi | both already have a +2 | 14:29 |
| *** pgaxatte has quit IRC | 14:30 | |
| *** ociuhandu has joined #openstack-infra | 14:31 | |
| *** ykarel is now known as ykarel|afk | 14:33 | |
| *** ociuhandu has quit IRC | 14:33 | |
| *** rfolco is now known as rfolco|bbl | 14:35 | |
| *** jamesmcarthur has quit IRC | 14:35 | |
| *** jamesmcarthur has joined #openstack-infra | 14:40 | |
| *** jamesmcarthur has quit IRC | 14:41 | |
| corvus | infra-root: i think zuul-registry is ready to go into production as the intermediate registry whenever we are ready. if we don't mind a small risk of failing image build jobs if something goes wrong, we can merge https://review.opendev.org/686298 which will just make the switch -- we can (manually) revert that if there's an error. alternatively we could spin up a replacement host, but that's | 14:42 |
| corvus | only going to let us verify the deployment before we start using it -- operation under load is still going to be something that either works or doesn't. | 14:42 |
| *** jamesmcarthur has joined #openstack-infra | 14:44 | |
| fungi | we're already failing image build jobs intermittently anyway, which this is targeted at alleviating, so seems reasonable to press forward | 14:47 |
| *** eernst has joined #openstack-infra | 14:48 | |
| *** goldyfruit_ has quit IRC | 14:49 | |
| fungi | corvus: registry_swift_secretkey isn't used any longer? what was that for previously? | 14:50 |
| fungi | i'm assuming use of openstacksdk in the new registry is what has made registry_swift_authurl no longer necessary | 14:52 |
| *** iurygregory_ has joined #openstack-infra | 14:52 | |
| *** ociuhandu has joined #openstack-infra | 14:53 | |
| *** iurygregory has quit IRC | 14:54 | |
| corvus | fungi: yeah, the secret key was an implementation-specific part of the docker-registry server. and yes, we outsource that to sdk. | 14:56 |
| corvus | that=authurl | 14:56 |
| *** jamesmcarthur has quit IRC | 14:57 | |
| *** goldyfruit_ has joined #openstack-infra | 14:57 | |
| *** ociuhandu has quit IRC | 14:57 | |
| fungi | awesome, lgtm! | 14:58 |
| AJaeger | is using the zuul tenant for this third-party testing correct? https://review.opendev.org/#/c/686392/ | 14:59 |
| *** jpena is now known as jpena|off | 14:59 | |
| *** jamesmcarthur has joined #openstack-infra | 15:00 | |
| corvus | AJaeger: i think so -- zuul does have openapi docs rendering, so i imagine mhu wants to test enhancements to that | 15:00 |
| AJaeger | Ah, ok | 15:01 |
| corvus | http://zuul.opendev.org/openapi | 15:01 |
| corvus | and https://zuul-ci.org/docs/zuul/user/web.html | 15:01 |
| AJaeger | thanks, corvus | 15:03 |
| AJaeger | config-core, a few open reviews are : https://review.opendev.org/683028 https://review.opendev.org/685791 https://review.opendev.org/683994 , please | 15:09 |
| *** ykarel|afk is now known as ykarel|away | 15:13 | |
| openstackgerrit | Merged openstack/project-config master: Add sphinx-contrib/openapi to untrusted github projects https://review.opendev.org/686392 | 15:18 |
| *** xek has quit IRC | 15:18 | |
| *** jbadiapa has joined #openstack-infra | 15:19 | |
| *** jamesmcarthur has quit IRC | 15:21 | |
| *** eernst has quit IRC | 15:23 | |
| *** gyee has joined #openstack-infra | 15:24 | |
| *** goldyfruit_ has quit IRC | 15:25 | |
| *** ykarel|away has quit IRC | 15:26 | |
| *** goldyfruit_ has joined #openstack-infra | 15:28 | |
| openstackgerrit | Nicolas Hicher proposed zuul/zuul-jobs master: DNM: debug console issue when iptables-save is called https://review.opendev.org/686424 | 15:29 |
| *** jamesmcarthur has joined #openstack-infra | 15:30 | |
| *** mattw4 has joined #openstack-infra | 15:31 | |
| *** michael-beaver has joined #openstack-infra | 15:33 | |
| *** udesale has quit IRC | 15:35 | |
| *** udesale has joined #openstack-infra | 15:35 | |
| *** tesseract has quit IRC | 15:36 | |
| *** lucasagomes has quit IRC | 15:40 | |
| *** ykarel|away has joined #openstack-infra | 15:49 | |
| *** jbadiapa has quit IRC | 15:50 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add type annotations https://review.opendev.org/686249 | 15:59 |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: WIP: use openstacksdk wip annotation https://review.opendev.org/686434 | 15:59 |
| *** rfolco|bbl is now known as rfolco | 16:00 | |
| *** iurygregory_ has quit IRC | 16:07 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: WIP - Gitlab - Basic handling of merge_requests event https://review.opendev.org/685990 | 16:07 |
| *** jbadiapa has joined #openstack-infra | 16:10 | |
| *** eernst has joined #openstack-infra | 16:16 | |
| *** eernst has quit IRC | 16:20 | |
| *** ociuhandu has joined #openstack-infra | 16:24 | |
| *** goldyfruit___ has joined #openstack-infra | 16:24 | |
| *** jbadiapa has quit IRC | 16:27 | |
| *** goldyfruit_ has quit IRC | 16:27 | |
| *** rpittau is now known as rpittau|afk | 16:27 | |
| *** udesale has quit IRC | 16:28 | |
| *** ociuhandu has quit IRC | 16:30 | |
| *** ccamacho has quit IRC | 16:31 | |
| *** chandankumar is now known as raukadah | 16:34 | |
| *** e0ne has quit IRC | 16:36 | |
| *** lucasagomes has joined #openstack-infra | 16:37 | |
| *** eernst has joined #openstack-infra | 16:39 | |
| *** jpena|off is now known as jpena | 16:41 | |
| *** eernst has quit IRC | 16:44 | |
| *** david-lyle has quit IRC | 16:44 | |
| *** david-lyle has joined #openstack-infra | 16:45 | |
| *** david-lyle has quit IRC | 16:45 | |
| *** igordc has joined #openstack-infra | 16:46 | |
| *** dklyle has joined #openstack-infra | 16:46 | |
| *** gfidente is now known as gfidente|afk | 16:52 | |
| *** derekh has quit IRC | 16:55 | |
| *** goldyfruit___ has quit IRC | 17:08 | |
| openstackgerrit | Merged openstack/project-config master: Add ansible-role-zuul-registry role for windmill https://review.opendev.org/685791 | 17:11 |
| openstackgerrit | Merged openstack/project-config master: Set Review-Priority label sticky https://review.opendev.org/683994 | 17:11 |
| *** tosky has quit IRC | 17:14 | |
| *** jamesmcarthur has quit IRC | 17:18 | |
| *** jamesmcarthur has joined #openstack-infra | 17:19 | |
| *** openstackgerrit has quit IRC | 17:21 | |
| *** Garyx has quit IRC | 17:31 | |
| *** openstackgerrit has joined #openstack-infra | 17:34 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Add streaming download support https://review.opendev.org/686449 | 17:34 |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Update test script to match playbook https://review.opendev.org/686450 | 17:34 |
| *** ricolin has quit IRC | 17:35 | |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Add streaming download support https://review.opendev.org/686449 | 17:38 |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Update test script to match playbook https://review.opendev.org/686450 | 17:38 |
| *** Garyx has joined #openstack-infra | 17:39 | |
| *** ykarel|away has quit IRC | 17:43 | |
| *** jamesmcarthur has quit IRC | 17:51 | |
| *** jamesmcarthur has joined #openstack-infra | 17:51 | |
| *** rfolco has quit IRC | 17:52 | |
| *** jamesmcarthur has quit IRC | 17:52 | |
| AJaeger | pabelanger: want me to update the windmill changes to use the opendev build-docs jobs? | 17:53 |
| *** e0ne has joined #openstack-infra | 17:59 | |
| *** jpena is now known as jpena|off | 18:05 | |
| *** armax has joined #openstack-infra | 18:06 | |
| *** e0ne has quit IRC | 18:06 | |
| *** spsurya has quit IRC | 18:07 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/nodepool master: wip: add a standalone launcher object https://review.opendev.org/639632 | 18:10 |
| *** ralonsoh has quit IRC | 18:10 | |
| *** ykarel|away has joined #openstack-infra | 18:11 | |
| *** goldyfruit___ has joined #openstack-infra | 18:16 | |
| *** hjensas has joined #openstack-infra | 18:20 | |
| openstackgerrit | Matthieu Huin proposed zuul/zuul master: Add OpenAPI description for enqueue, dequeue, autohold https://review.opendev.org/674257 | 18:39 |
| *** jamesmcarthur has joined #openstack-infra | 18:47 | |
| *** rfolco has joined #openstack-infra | 18:57 | |
| *** lpetrut has joined #openstack-infra | 18:58 | |
| *** lpetrut has quit IRC | 18:59 | |
| *** lpetrut has joined #openstack-infra | 19:00 | |
| *** haleyb has joined #openstack-infra | 19:01 | |
| *** lucasagomes has quit IRC | 19:02 | |
| *** lbragstad has quit IRC | 19:11 | |
| *** rlandy is now known as rlandy|brb | 19:13 | |
| *** zbr|ruck is now known as zbr | 19:14 | |
| *** ykarel|away has quit IRC | 19:17 | |
| openstackgerrit | Christopher Dearborn proposed openstack/project-config master: python-dracclient zuulv3 changes https://review.opendev.org/686409 | 19:18 |
| *** gfidente|afk has quit IRC | 19:27 | |
| *** yamamoto has joined #openstack-infra | 19:30 | |
| *** ociuhandu has joined #openstack-infra | 19:31 | |
| *** yamamoto has quit IRC | 19:35 | |
| *** ociuhandu has quit IRC | 19:37 | |
| *** lpetrut has quit IRC | 19:41 | |
| openstackgerrit | Tristan Cacqueray proposed opendev/glean master: Set Type=Ethernet for RedHat system https://review.opendev.org/686474 | 19:48 |
| *** gmann is now known as gmann_afk | 19:52 | |
| openstackgerrit | Paul Belanger proposed zuul/zuul master: WIP: Support Ansible 2.9 https://review.opendev.org/674854 | 20:03 |
| openstackgerrit | Christopher Dearborn proposed openstack/project-config master: python-dracclient zuulv3 changes https://review.opendev.org/686409 | 20:09 |
| *** pcaruana has quit IRC | 20:18 | |
| *** jamesmcarthur has quit IRC | 20:21 | |
| *** whoami-rajat has quit IRC | 20:28 | |
| *** lbragstad has joined #openstack-infra | 20:29 | |
| *** kjackal_v2 has quit IRC | 20:37 | |
| *** rlandy|brb is now known as rlandy | 20:38 | |
| corvus | i have restored access to the infra-root google account using a prepaid sim card i was given. i don't really consider this a scalable solution, but this will keep our zuul-gerrit poc moving and we can continue to talk to the gerrit/google folks about improvements. | 20:40 |
| corvus | the http password i previously set up works now -- apparently it stopped working because google disabled the account | 20:41 |
| corvus | infra-root: https://review.opendev.org/686049 should be gtg | 20:42 |
| *** Goneri has quit IRC | 20:46 | |
| fungi | corvus: did google disable the account because they wanted it validated by phone within some maximum timeframe after creation? or for unrelated reasons? or do you know? | 20:48 |
| fungi | and i agree, needing a "burner phone" to create google accounts seems extreme | 20:48 |
| corvus | fungi: https://imgur.com/a/XkOIjvi | 20:49 |
| corvus | that's a listing of 3 events; i've expanded the middle one with details | 20:49 |
| *** factor has quit IRC | 20:49 | |
| *** tosky has joined #openstack-infra | 20:50 | |
| corvus | the details of the other events are more...detailed.... | 20:50 |
| corvus | apparently they decided later that evening that the account i created in the morning was bogus | 20:50 |
| fungi | "looked like it was created with multiple other accounts or created by a program" | 20:51 |
| fungi | neat | 20:51 |
| fungi | "device: unknown device" | 20:51 |
| fungi | maybe a web browser with "too much privacy" in place | 20:52 |
| corvus | i didn't create any other accounts that day (or, really, in quite some time). i did use a program to create it called firefox. | 20:52 |
| fungi | also you signed on from linux. that hacker operating system | 20:52 |
| fungi | no good can come of it | 20:52 |
| corvus | yeah, an optimistic reading of this is "insufficiently fingerprinted" | 20:53 |
| fungi | (oh, wait, they based their entire mobile operating system on it didn't they?) | 20:53 |
| corvus | fungi: well, for now. later there's https://en.wikipedia.org/wiki/Google_Fuchsia | 20:53 |
| * paladox sooooo happy about this comment https://github.com/google/guava/issues/3602#issuecomment-538119157 | 20:53 | |
| openstackgerrit | Tristan Cacqueray proposed opendev/glean master: Set Type=Ethernet for RedHat system https://review.opendev.org/686474 | 20:54 |
| fungi | paladox: that's in service of solving thread locks in gerrit? | 20:55 |
| paladox | yup | 20:55 |
| paladox | ericson had it so they have cafeine running on a forked 2.14 | 20:55 |
| paladox | but as we use 2.15 we carn't use caffeine until some issue is fixed :( | 20:56 |
| fungi | corvus: wow, so fuscia's microkernel is descended from haiku (the beos clone)? | 21:05 |
| fungi | i haven't thought about beos in ages | 21:05 |
| fungi | granted, openbeos/haiku has been under development for 18 years now and is still a beta test ;) | 21:07 |
| tristanC | fungi: not sure about the origin, but it does have an interesting ci system that can do cross repos testing using this tool: https://fuchsia.googlesource.com/jiri | 21:09 |
| tristanC | and it seems like the build report features a comment system, perhaps to do log-review? | 21:10 |
| *** slaweq has quit IRC | 21:14 | |
| *** goldyfruit___ has quit IRC | 21:15 | |
| *** goldyfruit has joined #openstack-infra | 21:15 | |
| openstackgerrit | Andrei Nistor proposed openstack/diskimage-builder master: bootloader: make serial console configurable https://review.opendev.org/686495 | 21:17 |
| openstackgerrit | Andrei Nistor proposed openstack/diskimage-builder master: bootloader: make serial console configurable https://review.opendev.org/686495 | 21:18 |
| fungi | that's an interesting idea... being able to leave comments/annotations on build logs | 21:23 |
| *** auristor has quit IRC | 21:25 | |
| mnaser | infra-root: i'm cleaning up the issues for sjc1 and i noticed a lot of stale volumes right now (undeleted in-use ones) | 21:33 |
| mnaser | does anyone have a script handy for getting orphans or should i get writing.. | 21:33 |
| fungi | mnaser: i mainly looked for volumes with an old creation time | 21:38 |
| fungi | and then tried to correlate the attachments via what instances nova reported as still existing | 21:39 |
| fungi | no script, was probably just lengthy one-liners of nested osc invocations | 21:39 |
| clarkb | they usually show up as availabke not in use | 21:40 |
| openstackgerrit | Merged zuul/zuul-registry master: Add streaming download support https://review.opendev.org/686449 | 21:41 |
| tristanC | fungi: actually, build comments are automatic annotations, it's implemented by this gerrit plugins: https://chromium.googlesource.com/infra/gerrit-plugins/tricium | 21:41 |
| fungi | clarkb: depends, i think. the leaked ones where server delete reported failure but eventually worked show up as available volumes | 21:41 |
| fungi | er, now i don't actually remember | 21:42 |
| fungi | if cinder thinks they're still attached they may have continued to show up as in-use | 21:42 |
| fungi | but then openstack volume show would list an attachment to a server instance uuid which was not in the openstack server list output | 21:43 |
| mnaser | yeah so id hae to do that lokup | 21:44 |
| fungi | right, i think i just did some shell loops and expansions | 21:44 |
| openstackgerrit | Andrei Nistor proposed openstack/diskimage-builder master: bootloader: make serial console configurable https://review.opendev.org/686495 | 21:46 |
| fungi | something like... for a in $(for v in $(openstack volume list|grep ...|cut ...);do openstack volume show $v|grep ...|cut ...;done);do openstack server list|grep $a||echo $v;done | 21:46 |
| openstackgerrit | James E. Blair proposed zuul/zuul-registry master: Fix merge error in streaming support https://review.opendev.org/686505 | 21:47 |
| fungi | i write a ton of terrible nested loop one-liners in shell | 21:47 |
| mnaser | fungi: meet -c and -f :p | 21:47 |
| fungi | osc options? | 21:47 |
| fungi | yeah | 21:47 |
| fungi | i should make more use of its output formatting | 21:48 |
| mnaser | reduced the amount of cuts and awks :p | 21:48 |
| *** auristor has joined #openstack-infra | 21:49 | |
| Shrews | mnaser: if you can find a safe way to do it with openstacksdk, we might could integrate it into nodepool | 21:50 |
| clarkb | I can never get -c to work | 21:50 |
| mnaser | -c ID -f value usually is the trick for example | 21:50 |
| mnaser | Shrews: ok so ill actually scrap my bash code to write it with openstacksdk | 21:50 |
| fungi | well, we can't really do much with that in nodepool other than use it to skip attempts at re-deleting the affected volumes | 21:51 |
| fungi | er, skip re-attempts at deleting the affected volumes | 21:51 |
| mnaser | maybe a friendly log msg | 21:51 |
| fungi | yeah | 21:52 |
| fungi | at least for the ones i was finding, we needed a cloud admin to perform the actual deletion | 21:52 |
| Shrews | oh, well, if it needs different perms... | 21:52 |
| fungi | since it's a confusion between nova and cinder | 21:52 |
| fungi | nova fails to tell cinder the volume attachment is no longer needed, so cinder fails to allow the user to delete the volume | 21:53 |
| fungi | er, refuses to allow the user to delete the volume | 21:53 |
| fungi | thinking it's still attached to something | 21:53 |
| mnaser | ^ yep | 21:53 |
| mnaser | an admin needs top reset it | 21:53 |
| openstackgerrit | Merged zuul/zuul-registry master: Update test script to match playbook https://review.opendev.org/686450 | 21:58 |
| ianw | donnyd: over in #zuul looking at a couple of nodepool functional tests that are failing on fortnebula with what looks very much like the inner test-vm hitting it's 10 minute boot timeout | 22:03 |
| ianw | are there known issues? perhaps with i/o? | 22:04 |
| fungi | i think he's said the nodes there are mainly cpu-bound | 22:04 |
| *** mattw4 has quit IRC | 22:05 | |
| *** mattw4 has joined #openstack-infra | 22:05 | |
| fungi | so non-accelerated vm nesting may just be slower there. or perhaps there's some recent performance issue which has cropped up | 22:06 |
| *** pkopec has quit IRC | 22:06 | |
| ianw | the boot log is basically -> https://paste.fedoraproject.org/paste/FjWgq95WA1QFrTh3Q0bkvA | 22:07 |
| fungi | i gather the storage is all nvme, with a ton of bandwidth | 22:07 |
| fungi | so wouldn't expect to see poor filesystem i/o performance | 22:08 |
| *** rcernin has joined #openstack-infra | 22:08 | |
| ianw | there's doesn't seem to be one particluar point in the boot where things take a big jump ... just lots of slow things | 22:08 |
| fungi | could it be thrashing swap on the job node and not managing any fs cache? | 22:09 |
| ianw | 2019-10-03 02:40:36,188 DEBUG nodepool.NodeLauncher: [node: 0000000000] [ 465.082869] growroot[444]: + growpart /dev/vda 1 | 22:10 |
| ianw | 2019-10-03 02:40:36,189 DEBUG nodepool.NodeLauncher: [node: 0000000000] [ 499.918863] growroot[444]: CHANGED: partition=1 start=2048 old: size=3616256 end=3618304 new: size=10483679,end=10485727 | 22:10 |
| ianw | that's the best part of 30s right? | 22:10 |
| fungi | yep, and some change | 22:10 |
| fungi | nearly 35 seconds | 22:11 |
| mnaser | can you check if it used kvm and not qemu | 22:11 |
| mnaser | because afaik growpart doesnt actaully do much of io, it just kinda does a little bit of black magic that remaps the partition | 22:11 |
| mnaser | if the io was that bad id think it would be a looot slower | 22:11 |
| fungi | that's also only growing it from 3mb to 10mb right? | 22:12 |
| ianw | yeah, and the image build, which is basically all io went from 2019-10-03 02:04:20.118 .. 2019-10-03 02:18:13.718 ... that's about right | 22:13 |
| fungi | so even if it were allocating additional blocks, that's a relatively small amount | 22:13 |
| ianw | it should not take longer to boot the vm than build the image :) | 22:13 |
| mnaser | ok so i got all the orphans | 22:15 |
| mnaser | i am ready for the self inflicted ddos | 22:15 |
| donnyd | ianw: all the hypervisors use local NVMEs so not likely to be IO bound | 22:16 |
| donnyd | http://grafana.openstack.org/d/3Bwpi5SZk/nodepool-fortnebula?orgId=1&from=now-6h&to=now | 22:16 |
| donnyd | Looking here I don't see any launch errors | 22:17 |
| donnyd | Its pretty normal however whenever a new image is loaded for it to take a bit to populate due to glance | 22:17 |
| ianw | donnyd: so this is booting a vm inside the job | 22:17 |
| donnyd | oh | 22:17 |
| fungi | yeah, growpart in the nested (cirros) vm is clocking in waaay slow | 22:18 |
| donnyd | yea my CPU's are not exactly new and without nested virt enabled I can't do much else | 22:18 |
| donnyd | https://grafana.fortnebula.com/d/9MMqh8HWk/openstack-utilization?orgId=2&refresh=30s | 22:18 |
| donnyd | Also looking here I see some pretty sizable spikes, so you may have fallen into one of those | 22:19 |
| fungi | yeah, i could see that happening if a significant cpu spike happened on the host due to load from other job nodes | 22:20 |
| ianw | such as ... a bunch of nodepool jobs all trying to boot inner vm's at the same time | 22:20 |
| fungi | ;) | 22:22 |
| ianw | i guess grafana is showing me that in local time | 22:22 |
| donnyd | I turned down the over-subscription to try and help... but they just aren't super fast CPU's | 22:22 |
| ianw | how long ago was 2019-10-03 02:28:27,937 UTC? | 22:22 |
| fungi | nearly 20 hours | 22:22 |
| *** rh-jelabarre has quit IRC | 22:23 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-registry master: Add type annotations https://review.opendev.org/686249 | 22:23 |
| ianw | maybe ~ https://grafana.fortnebula.com/d/9MMqh8HWk/openstack-utilization?orgId=2&from=1570072636623&to=1570075500375 | 22:28 |
| donnyd | ianw: that is possible. Is this a tripelo job? | 22:29 |
| *** markvoelker has quit IRC | 22:30 | |
| donnyd | do you have a link for logs from the job? | 22:30 |
| *** rlandy is now known as rlandy|bbl | 22:31 | |
| ianw | https://zuul.opendev.org/t/openstack/build/0cf5dfc8bde84aed8c88b0887a5d79bb | 22:31 |
| ianw | https://zuul.opendev.org/t/openstack/build/076d513990da4dd4922550acbf4adced | 22:31 |
| ianw | failed | 22:31 |
| ianw | https://zuul.opendev.org/t/openstack/build/eed91655b2f948ea974dd92fb8b7ba39 | 22:31 |
| ianw | passed ... all ran on fortnebula around the same time | 22:32 |
| ianw | corvus: is it intentional to not collect the devstack logs in the nodepool functional tests, or an oversight? | 22:32 |
| donnyd | it would be really handy to have a way to manually reproduce the job for these types of things | 22:34 |
| corvus | ianw: mostly intentional -- that might be a better thing for the dib child job | 22:34 |
| donnyd | http://logstash.openstack.org/#/dashboard/file/logstash.json?query=node_provider:%5C%22fortnebula-regionone%5C%22%20AND%20filename:%5C%22job-output.txt%5C%22%20AND%20message:%5C%22RUN%20END%20RESULT_TIMED_OUT%5C%22&from=24h | 22:35 |
| *** panda is now known as panda|off | 22:35 | |
| donnyd | hrmmm that seems a little high | 22:35 |
| donnyd | usually there are just the jobs that get close to the timeout anyways that don't run well on FN | 22:35 |
| donnyd | and the ones that tinker with the network that don't account for ipv6 being the primary mode of communications | 22:36 |
| donnyd | but it looks like all the DIB jobs failed | 22:37 |
| donnyd | would this be a good way to test our nested virt label? | 22:37 |
| donnyd | but running this query filename:"job-output.txt" AND message:"RUN END RESULT_TIMED_OUT" AND build_name:"dib-nodepool-functional-openstack-fedora-29-src" | 22:38 |
| donnyd | its not comical... but in the last 24 hours 100% of that job has run on FN... with 100% failure rate... | 22:39 |
| ianw | that sounds about right :) | 22:40 |
| donnyd | and in the last 7d there have only been 4 runs of that job.. all on FN... all Failed | 22:42 |
| ianw | hrm, there would have been many more than 4 ... i have a very large centos 8 stack | 22:42 |
| donnyd | just fedora-29 | 22:44 |
| donnyd | oh and I am only looking for timeouts | 22:44 |
| ianw | https://zuul.opendev.org/t/openstack/build/0c400fb1ac3c4decb7882159b22c1601 is another example of a different job | 22:46 |
| ianw | dib-functests-centos7-python2 | 22:46 |
| ianw | that doesn't actually boot anything, only build | 22:46 |
| ianw | usually a 30-40 minute job | 22:47 |
| donnyd | gonna take me a little bit to sift through the logs to figure out where the delays are at | 22:47 |
| ianw | i don't understand that one | 22:52 |
| ianw | 2019-10-03 09:32:50.440 | Build completed successfully | 22:52 |
| donnyd | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0c4/684860/18/check/dib-functests-centos7-python2/0c400fb/logs/ | 22:52 |
| donnyd | well it looks like they all built | 22:53 |
| ianw | 2019-10-03 09:25:00.513841 | centos-7 | [ubuntu-minimal/xenial-build-succeeds] Output logs going to: /home/zuul/logs/ubuntu-minimal_xenial-build-succeeds.log | 22:53 |
| ianw | 2019-10-03 09:41:59.496671 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/diskimage-builder/playbooks/dib-functests/run.yaml@master] | 22:53 |
| ianw | why did it stop between 09:25 -> 09:41? | 22:53 |
| donnyd | looks like the acutal meat of the job starts here | 22:54 |
| donnyd | 2019-10-03 08:16:45.712130 | centos-7 | [JOB-CONTROL] Starting new job | 22:54 |
| donnyd | ends here | 22:55 |
| donnyd | 2019-10-03 09:25:00.513841 | centos-7 | [ubuntu-minimal/xenial-build-succeeds] Output logs going to: /home/zuul/logs/ubuntu-minimal_xenial-build-succeeds.log | 22:55 |
| ianw | yeah i think it started on the bionic build, but we didn't capture anything for that | 22:55 |
| donnyd | which is right around the 40 minutes you were saying it takes to run right | 22:55 |
| donnyd | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0c4/684860/18/check/dib-functests-centos7-python2/0c400fb/logs/ubuntu-minimal_bionic-build-succeeds.PASS.log | 22:56 |
| ianw | so similar issue, was working, but just slow | 22:56 |
| donnyd | there is the bionic job ^^^ | 22:56 |
| donnyd | not super familar with the job | 22:56 |
| ianw | yeah i just loops though a bunch of different builds, but doesn't actually boot the results. just to make sure a .qcow2 comes out the other side | 22:57 |
| donnyd | and this job here has something else broken | 22:58 |
| donnyd | https://zuul.opendev.org/t/openstack/build/076d513990da4dd4922550acbf4adced | 22:58 |
| donnyd | https://1251a16130059a3c4478-a4f75ebbaac54881743ae326d50075b2.ssl.cf1.rackcdn.com/684460/10/check/dib-nodepool-functional-openstack-fedora-30-src/076d513/job-output.txt | 22:58 |
| donnyd | 2019-10-03 01:53:45.999760 | TASK [Check nodepool functionality] | 22:58 |
| donnyd | 2019-10-03 02:43:29.692653 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/zuul/nodepool/playbooks/nodepool-functional-openstack/run.yaml@master] | 22:59 |
| donnyd | seems to be stuck in a loop for 45 minutes | 22:59 |
| ianw | that's more or less right, it's waiting for the inner vm to come up | 23:00 |
| ianw | 2019-10-03 02:39:02,191 DEBUG nodepool.NodeLauncher: [node: 0000000000] [ 563.356060] systemd[1]: Reached target sshd-keygen.target. | 23:00 |
| ianw | again up near the 600s timeout; that's from the launch logs | 23:00 |
| ianw | https://1251a16130059a3c4478-a4f75ebbaac54881743ae326d50075b2.ssl.cf1.rackcdn.com/684460/10/check/dib-nodepool-functional-openstack-fedora-30-src/076d513/nodepool/nodepool-launcher.log | 23:00 |
| *** goldyfruit_ has joined #openstack-infra | 23:00 | |
| ianw | we wait 10 minutes for the node to appear, before we consider it dead. i think nodepool does that maybe 3 times or so | 23:00 |
| *** mattw4 has quit IRC | 23:01 | |
| *** tkajinam has joined #openstack-infra | 23:01 | |
| ianw | 2019-10-03 02:28:27,937 DEBUG nodepool.NodeLauncher: [node: 0000000000] [ 539.215851] systemd[1]: Started Grow root partition. | 23:01 |
| ianw | that was the first attempt, the ^^ was the second attempt, it didn't get to a 3rd | 23:02 |
| *** goldyfruit has quit IRC | 23:02 | |
| *** mattw4 has joined #openstack-infra | 23:02 | |
| *** bdodd has quit IRC | 23:02 | |
| ianw | ... so, not sure. i would really like to merge the centos 8 queue, but it does rely on these in gate | 23:04 |
| donnyd | 2019-10-03 01:53:47,866 DEBUG keystoneauth.session: RESP BODY: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} | 23:05 |
| donnyd | https://1251a16130059a3c4478-a4f75ebbaac54881743ae326d50075b2.ssl.cf1.rackcdn.com/684460/10/check/dib-nodepool-functional-openstack-fedora-30-src/076d513/nodepool/nodepool-launcher.log | 23:05 |
| donnyd | 2019-10-03 01:53:47,866 DEBUG keystoneauth.session: RESP BODY: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}} | 23:05 |
| donnyd | 2019-10-03 01:53:47,866 DEBUG keystoneauth.session: GET call to http://192.168.48.87/compute/v2.1 used request id req-7729ce9a-1551-45ab-8401-77650fb74d08 | 23:05 |
| donnyd | 2019-10-03 01:53:47,867 DEBUG keystoneauth.session: Request returned failure status: 401 | 23:05 |
| donnyd | Looks to me like something in the job is busted | 23:05 |
| *** jtomasek has quit IRC | 23:07 | |
| *** petevg has quit IRC | 23:09 | |
| ianw | i feel like that may be something to do with the openstacksdk calls ... similar appears on a good run | 23:09 |
| ianw | https://zuul.opendev.org/t/openstack/build/315196acac334c3282a18310fc59f58a/log/nodepool/nodepool-launcher.log | 23:09 |
| ianw | it does | 23:09 |
| ianw | GET call to http://162.242.237.62/compute/v2.1 | 23:09 |
| ianw | (401) | 23:09 |
| ianw | GET call to compute for http://162.242.237.62/compute/v2.1/servers/detail | 23:09 |
| ianw | (works) | 23:09 |
| fungi | yeah, i think if those were a problem they should be logged higher than debug loglevel | 23:11 |
| donnyd | I can say I have nested virt enabled, so maybe it could accelerate those workloads (probably by quite a bit) | 23:17 |
| donnyd | but that is about all I can really do | 23:18 |
| donnyd | ianw: do you have a method to run this outside the CI? | 23:18 |
| donnyd | maybe to could do more something else to accelerate this job | 23:18 |
| donnyd | its surely worth a try | 23:18 |
| ianw | it is moderately involved; first devstack runs to give the job a cloud environment, then nodepool is setup to run against it, a build triggered, nodepool uploads the images and then a test vm is started | 23:21 |
| donnyd | oh | 23:21 |
| ianw | but if that test vm is taking +10 minutes to boot, that's a bottleneck we can't really speed up. it's already really doing as little as it can do | 23:22 |
| ianw | (modulo nested virt i guess) | 23:23 |
| *** jamesmcarthur has joined #openstack-infra | 23:23 | |
| *** tosky has quit IRC | 23:23 | |
| donnyd | we created a custom label so people can give that a swing. would it be possible to create an experimental job that uses it | 23:23 |
| *** jamesmcarthur has quit IRC | 23:24 | |
| donnyd | i wouldn't think it should take 10 minutes to spin up a vm with qemu though | 23:24 |
| ianw | yes. does nova just get it right if it's enabled, or does devstack require further setup? | 23:24 |
| donnyd | seems a little steep | 23:24 |
| donnyd | i would think you would have to tell devstack to use kvm.. not sure if it will just use it automagically | 23:25 |
| *** mriedem has quit IRC | 23:26 | |
| ianw | This | 23:27 |
| ianw | can be verified by checking for the presence of the file ``/dev/kvm`` in | 23:27 |
| ianw | your VM. If it is present, DevStack will default to using the config | 23:27 |
| ianw | attribute ``virt_type = kvm`` in ``/etc/nova.conf``; | 23:27 |
| mnaser | ianw: i believe in devstack job it is pinned to not use nested virt tho | 23:27 |
| mnaser | i think | 23:27 |
| donnyd | i think mnaser is correct | 23:28 |
| donnyd | otherwise it would already be using it on a couple different clouds | 23:28 |
| ianw | # Gate jobs can't deal with nested virt. Disable it. | 23:28 |
| ianw | LIBVIRT_TYPE: qemu | 23:28 |
| donnyd | do you have the devstack build... i can manually spin up a instance and then check it to see how long an instance actually takes | 23:29 |
| *** sthussey has quit IRC | 23:33 | |
| ianw | i should be able to get some experimental jobs with it turned on, just a tick | 23:34 |
| *** igordc has quit IRC | 23:36 | |
| *** armax has quit IRC | 23:38 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [dnm] testing nested virt labels https://review.opendev.org/686510 | 23:39 |
| ianw | The nodeset "nested-virt-ubuntu-bionic" was not found. | 23:40 |
| ianw | oohh, we added the nodes to nodepool, probably not to config? | 23:41 |
| *** jtomasek has joined #openstack-infra | 23:41 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [dnm] testing nested virt labels https://review.opendev.org/686510 | 23:44 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [dnm] testing nested virt labels https://review.opendev.org/686510 | 23:45 |
| ianw | interesting, that didn't work the jobs are going to rax | 23:47 |
| *** michael-beaver has quit IRC | 23:47 | |
| *** mattw4 has quit IRC | 23:49 | |
| *** xenos76 has quit IRC | 23:50 | |
| ianw | hrm, no they bailed with NODE_FAILURE | 23:52 |
| *** jtomasek has quit IRC | 23:54 | |
| *** goldyfruit_ has quit IRC | 23:56 | |
| *** jtomasek has joined #openstack-infra | 23:56 | |
| ianw | arrgh, that's right, a bunch of jobs run on xenial because ... debootstrap bugs | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!