| openstackgerrit | Merged openstack-infra/puppet-logstash master: Parameterize JRE package https://review.openstack.org/473112 | 00:01 |
|---|---|---|
| *** dimtruck is now known as zz_dimtruck | 00:05 | |
| *** gmann has joined #openstack-infra | 00:05 | |
| fungi | clarkb: i'll check in on it tomorrow and see if it's done | 00:05 |
| ianw | mordred: in _upload_image_put ... image_data = open(filename, 'rb') that is being uploaded. if that was wrapped and knew how much it had been read, when wait=True you could do a callback with the info? | 00:06 |
| *** zz_dimtruck is now known as dimtruck | 00:06 | |
| mordred | ianw: yah - main thought in my brain is whether or not this thing we're talking about should be a generalized thing or is specific to image/object upload | 00:09 |
| *** jamesmcarthur has quit IRC | 00:10 | |
| *** dimtruck is now known as zz_dimtruck | 00:13 | |
| fungi | clarkb: analyzing the sample collected so far, the top offender is actually gate-tempest-dsvm-neutron-full-ubuntu-xenial | 00:15 |
| fungi | clarkb: of the top ten, eight are tempest jobs and two are tripleo jobs | 00:16 |
| fungi | but let's see what a larger sample size tells us | 00:16 |
| fungi | http://paste.openstack.org/show/612604 | 00:18 |
| fungi | tripleo jobs do seem to be bubbling a little higher up the larger the sample size gets | 00:22 |
| *** armax has quit IRC | 00:26 | |
| fungi | gate-tempest-dsvm-neutron-full-ubuntu-xenial logs seem to account for roughly 5% of the total logs saved for change-oriented pipelines right now | 00:32 |
| fungi | but so does gate-tripleo-ci-centos-7-ovb-containers-oooq-nv | 00:34 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/shade master: De-client-ify User Create https://review.openstack.org/474397 | 00:35 |
| fungi | http://paste.openstack.org/show/612605 | 00:36 |
| clarkb | fungi: and tripleo is fewer jobs? | 00:38 |
| *** zhurong has joined #openstack-infra | 00:39 | |
| fungi | clarkb: right at an order of magnitude fewer, yes | 00:40 |
| clarkb | I also wonder if we can drop the keystone v3 only jov | 00:40 |
| fungi | 59:6 counted so far | 00:40 |
| clarkb | and just make default job v3 only | 00:40 |
| fungi | so suggesting that tripleo job saves ~10x the logs on average as the tempest job it's tied with | 00:41 |
| *** Apoorva_ has joined #openstack-infra | 00:41 | |
| fungi | oh, that du finished far sooner than i expected! | 00:41 |
| *** tuanluong has joined #openstack-infra | 00:42 | |
| *** liujiong has joined #openstack-infra | 00:42 | |
| fungi | i'm going to loop a dozen or so more invocations appending to that file now | 00:43 |
| *** Apoorva has quit IRC | 00:45 | |
| *** Apoorva_ has quit IRC | 00:46 | |
| fungi | i'm adding another 20 rounds to it | 00:46 |
| *** rfolco has joined #openstack-infra | 00:49 | |
| *** rfolco has quit IRC | 00:49 | |
| fungi | scaling the single initial run out by 100, looks like change-oriented pipeline logs account for ~8tb of data on the fs (implying periodic and commit/tag-oriented pipelines occupy the remaining ~2.5tb or so) | 00:50 |
| fungi | er, i guess that volume is a little larger than i thought so the other pipeline types account for close to 3.5tb per that estimate | 00:52 |
| *** zz_dimtruck is now known as dimtruck | 00:52 | |
| *** abramley has joined #openstack-infra | 00:54 | |
| *** cuongnv has joined #openstack-infra | 00:57 | |
| *** sbezverk_ has joined #openstack-infra | 00:59 | |
| *** jaypipes has quit IRC | 00:59 | |
| *** sbezverk has quit IRC | 01:02 | |
| *** rfolco has joined #openstack-infra | 01:03 | |
| *** rfolco has quit IRC | 01:03 | |
| *** ramishra has joined #openstack-infra | 01:05 | |
| fungi | broadening the sample set, that tripleo job is now outpacing the tempest job by a good amount. i guess 1% of changes is not a statistically significant sample | 01:05 |
| *** amotoki_away is now known as amotoki | 01:10 | |
| *** raissa has quit IRC | 01:11 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Implement Depends-On for github [wip] https://review.openstack.org/474401 | 01:20 |
| *** ramishra has quit IRC | 01:20 | |
| *** raissa has joined #openstack-infra | 01:21 | |
| *** Curtis has joined #openstack-infra | 01:23 | |
| *** jamesdenton has joined #openstack-infra | 01:23 | |
| *** namnh has joined #openstack-infra | 01:25 | |
| *** xarses_ has joined #openstack-infra | 01:28 | |
| *** aeng has quit IRC | 01:30 | |
| *** gcb has joined #openstack-infra | 01:30 | |
| *** sree has joined #openstack-infra | 01:31 | |
| *** jamesdenton has quit IRC | 01:34 | |
| *** sree has quit IRC | 01:35 | |
| *** yamahata has quit IRC | 01:39 | |
| *** armax has joined #openstack-infra | 01:40 | |
| *** Curtis has quit IRC | 01:51 | |
| *** hongbin has joined #openstack-infra | 02:03 | |
| cmurphy | clarkb: I've been not around, I assume module version bumps went okay? | 02:07 |
| cmurphy | clarkb: ty for approving puppet changes | 02:07 |
| clarkb | yup looked god | 02:09 |
| *** sflanigan has quit IRC | 02:10 | |
| *** baoli has joined #openstack-infra | 02:10 | |
| *** jamesmcarthur has joined #openstack-infra | 02:11 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Disable horizon by default https://review.openstack.org/474283 | 02:15 |
| *** jamesmcarthur has quit IRC | 02:15 | |
| *** jiaohaolin has joined #openstack-infra | 02:17 | |
| jiaohaolin | hey guys | 02:18 |
| jiaohaolin | I want to add a page to wiki .when I save the page ,it told me 'Incorrect or missing CAPTCHA.' But there is no captcha shows up | 02:20 |
| jiaohaolin | what should I do ? | 02:20 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: Add support for zuul.d configuration split https://review.openstack.org/473764 | 02:27 |
| *** dhinesh has quit IRC | 02:30 | |
| *** yland has quit IRC | 02:32 | |
| *** markvoelker has quit IRC | 02:32 | |
| *** markvoelker has joined #openstack-infra | 02:33 | |
| *** aeng has joined #openstack-infra | 02:33 | |
| *** shu-mutou-AWAY is now known as shu-mutou | 02:34 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/shade master: [WIP/RFC] add progress callback to image upload https://review.openstack.org/474425 | 02:34 |
| *** sflanigan has joined #openstack-infra | 02:35 | |
| *** sflanigan has joined #openstack-infra | 02:35 | |
| *** markvoelker has quit IRC | 02:37 | |
| *** ramishra has joined #openstack-infra | 02:38 | |
| *** dimtruck is now known as zz_dimtruck | 02:46 | |
| *** baoli has quit IRC | 02:47 | |
| *** nicolasbock has joined #openstack-infra | 02:51 | |
| *** ramishra has quit IRC | 02:54 | |
| *** ramishra has joined #openstack-infra | 02:56 | |
| *** rossella_s has joined #openstack-infra | 02:57 | |
| *** dklyle has joined #openstack-infra | 03:01 | |
| *** aeng has quit IRC | 03:02 | |
| *** david-lyle has quit IRC | 03:04 | |
| *** yamahata has joined #openstack-infra | 03:12 | |
| *** aeng has joined #openstack-infra | 03:21 | |
| *** isaacb has joined #openstack-infra | 03:22 | |
| *** rossella_s has quit IRC | 03:23 | |
| *** isaacb has quit IRC | 03:24 | |
| *** jamesmcarthur has joined #openstack-infra | 03:27 | |
| *** jamesmcarthur has quit IRC | 03:40 | |
| *** gomarivera has joined #openstack-infra | 03:45 | |
| *** dave-mcc_ has quit IRC | 03:51 | |
| *** nicolasbock has quit IRC | 03:53 | |
| *** brucet has quit IRC | 03:53 | |
| *** hongbin has quit IRC | 03:55 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/shade master: [WIP/RFC] add progress callback to image upload https://review.openstack.org/474425 | 03:56 |
| *** nt has quit IRC | 03:59 | |
| *** nt has joined #openstack-infra | 04:00 | |
| *** sree has joined #openstack-infra | 04:07 | |
| *** dhinesh has joined #openstack-infra | 04:08 | |
| *** jamesmcarthur has joined #openstack-infra | 04:09 | |
| *** jamesmcarthur has quit IRC | 04:14 | |
| *** gomarivera has quit IRC | 04:16 | |
| *** sbezverk_ has quit IRC | 04:16 | |
| *** stakeda has quit IRC | 04:16 | |
| *** salv-orlando has joined #openstack-infra | 04:16 | |
| *** ykarel has joined #openstack-infra | 04:20 | |
| *** salv-orlando has quit IRC | 04:21 | |
| *** psachin has joined #openstack-infra | 04:22 | |
| *** gomarivera has joined #openstack-infra | 04:24 | |
| *** stakeda has joined #openstack-infra | 04:24 | |
| *** gouthamr has quit IRC | 04:24 | |
| *** zhurong has quit IRC | 04:28 | |
| *** salv-orlando has joined #openstack-infra | 04:28 | |
| *** mdrabe has quit IRC | 04:30 | |
| *** cody-somerville has joined #openstack-infra | 04:32 | |
| *** markvoelker has joined #openstack-infra | 04:33 | |
| *** csomerville has quit IRC | 04:35 | |
| *** zhurong has joined #openstack-infra | 04:41 | |
| *** pgadiya has joined #openstack-infra | 04:42 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder master: Update system_groovy to make use convert_mapping_to_xml() https://review.openstack.org/474125 | 04:43 |
| *** goutham has joined #openstack-infra | 04:51 | |
| *** jaosorior has quit IRC | 05:00 | |
| *** salv-orl_ has joined #openstack-infra | 05:05 | |
| *** markvoelker has quit IRC | 05:07 | |
| *** salv-orlando has quit IRC | 05:08 | |
| *** claudiub has joined #openstack-infra | 05:08 | |
| openstackgerrit | Omer Anson proposed openstack-infra/devstack-gate master: Allow manual request for overlay network in multinode https://review.openstack.org/437050 | 05:12 |
| *** abramley has quit IRC | 05:13 | |
| *** sree has quit IRC | 05:21 | |
| *** prateek has joined #openstack-infra | 05:23 | |
| *** gyee has quit IRC | 05:39 | |
| *** ijw has joined #openstack-infra | 05:39 | |
| *** florianf has quit IRC | 05:42 | |
| *** cshastri has joined #openstack-infra | 05:43 | |
| *** florianf has joined #openstack-infra | 05:43 | |
| *** ijw has quit IRC | 05:44 | |
| *** aeng has quit IRC | 05:47 | |
| *** rossella_s has joined #openstack-infra | 05:48 | |
| *** dtantsur|afk is now known as dtantsur | 05:49 | |
| *** pgadiya has quit IRC | 05:51 | |
| *** gomarivera has quit IRC | 05:51 | |
| *** gomarivera has joined #openstack-infra | 05:52 | |
| *** gomarivera has quit IRC | 05:56 | |
| *** sree has joined #openstack-infra | 05:58 | |
| *** makowals has joined #openstack-infra | 05:59 | |
| *** ramishra has quit IRC | 05:59 | |
| *** tumbarka has joined #openstack-infra | 06:00 | |
| *** ramishra has joined #openstack-infra | 06:01 | |
| *** salv-orl_ has quit IRC | 06:02 | |
| *** tushar has quit IRC | 06:02 | |
| *** jklare has quit IRC | 06:02 | |
| *** makowals has quit IRC | 06:03 | |
| *** pgadiya has joined #openstack-infra | 06:04 | |
| *** markvoelker has joined #openstack-infra | 06:04 | |
| *** gomarivera has joined #openstack-infra | 06:06 | |
| *** gomarivera has quit IRC | 06:06 | |
| *** eranrom has quit IRC | 06:11 | |
| *** dhajare has joined #openstack-infra | 06:11 | |
| *** rcernin has quit IRC | 06:11 | |
| *** ltomasbo|away is now known as ltomasbo | 06:13 | |
| *** tushar has joined #openstack-infra | 06:14 | |
| *** tumbarka has quit IRC | 06:14 | |
| *** gcb has quit IRC | 06:17 | |
| *** jklare has joined #openstack-infra | 06:18 | |
| *** rossella_s has quit IRC | 06:24 | |
| *** dhinesh has quit IRC | 06:26 | |
| *** rwsu has quit IRC | 06:26 | |
| *** dhinesh has joined #openstack-infra | 06:26 | |
| *** gcb has joined #openstack-infra | 06:28 | |
| *** markvoelker has quit IRC | 06:37 | |
| *** rcernin has joined #openstack-infra | 06:37 | |
| *** hashar has joined #openstack-infra | 06:39 | |
| *** markmcd has quit IRC | 06:39 | |
| *** makowals has joined #openstack-infra | 06:40 | |
| *** prateek_cell has joined #openstack-infra | 06:41 | |
| *** JudeC has joined #openstack-infra | 06:41 | |
| *** prateek_cell has quit IRC | 06:42 | |
| *** markmcd has joined #openstack-infra | 06:42 | |
| *** tesseract has joined #openstack-infra | 06:43 | |
| *** ArchiFleKs has quit IRC | 06:44 | |
| *** dims has quit IRC | 06:44 | |
| *** markmcd has quit IRC | 06:45 | |
| *** dims has joined #openstack-infra | 06:46 | |
| *** markmcd has joined #openstack-infra | 06:47 | |
| *** armax has quit IRC | 06:49 | |
| *** markmcd has quit IRC | 06:50 | |
| *** markmcd has joined #openstack-infra | 06:52 | |
| *** dhinesh has quit IRC | 06:53 | |
| *** liujiong_lj has joined #openstack-infra | 06:57 | |
| *** markmcd has quit IRC | 06:57 | |
| *** liujiong has quit IRC | 06:58 | |
| *** pcaruana has joined #openstack-infra | 06:58 | |
| *** goutham has quit IRC | 06:59 | |
| *** markmcd has joined #openstack-infra | 06:59 | |
| *** eranrom has joined #openstack-infra | 07:02 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: executor: add support for custom ansible_port https://review.openstack.org/468710 | 07:04 |
| *** armax has joined #openstack-infra | 07:10 | |
| *** ccamacho|brb has quit IRC | 07:12 | |
| *** jaosorior has joined #openstack-infra | 07:12 | |
| *** salv-orlando has joined #openstack-infra | 07:12 | |
| *** gildub has quit IRC | 07:13 | |
| *** ajafo has quit IRC | 07:13 | |
| *** ajafo has joined #openstack-infra | 07:13 | |
| *** amoralej|off is now known as amoralej | 07:15 | |
| *** armax has quit IRC | 07:15 | |
| *** ccamacho has joined #openstack-infra | 07:16 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: bubblewrap: adds --die-with-parent option https://review.openstack.org/473164 | 07:18 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: executor: run trusted playbook in a bubblewrap https://review.openstack.org/474460 | 07:18 |
| *** ralonsoh has joined #openstack-infra | 07:21 | |
| *** liujiong_lj has quit IRC | 07:22 | |
| *** liujiong has joined #openstack-infra | 07:22 | |
| *** rossella_s has joined #openstack-infra | 07:26 | |
| *** tushar has quit IRC | 07:27 | |
| *** tushar has joined #openstack-infra | 07:27 | |
| *** neiljerram has joined #openstack-infra | 07:30 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Move neutron upgrades meeting to Thu 1h earlier https://review.openstack.org/474347 | 07:31 |
| *** ltomasbo is now known as ltomasbo|away | 07:32 | |
| *** markvoelker has joined #openstack-infra | 07:34 | |
| *** tushar has quit IRC | 07:35 | |
| *** tumbarka has joined #openstack-infra | 07:35 | |
| *** shardy has joined #openstack-infra | 07:36 | |
| *** rossella_s has quit IRC | 07:38 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/irc-meetings master: Fix osops meeting handle https://review.openstack.org/474471 | 07:39 |
| *** jpena|off is now known as jpena | 07:43 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/irc-meetings master: Cleanup usused meeting slots https://review.openstack.org/474473 | 07:44 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: executor: run trusted playbook in a bubblewrap https://review.openstack.org/474460 | 07:49 |
| openstackgerrit | Jan Provaznik proposed openstack-infra/project-config master: Add new puppet-ganesha project https://review.openstack.org/474474 | 07:50 |
| *** tushar has joined #openstack-infra | 07:51 | |
| *** tumbarka has quit IRC | 07:51 | |
| *** mugsie has quit IRC | 07:51 | |
| *** ramishra has quit IRC | 07:54 | |
| *** mugsie has joined #openstack-infra | 07:54 | |
| *** mugsie has quit IRC | 07:54 | |
| *** mugsie has joined #openstack-infra | 07:54 | |
| *** ramishra has joined #openstack-infra | 07:56 | |
| *** neiljerram has quit IRC | 07:57 | |
| openstackgerrit | Jan Provaznik proposed openstack-infra/project-config master: Add new puppet-ganesha project https://review.openstack.org/474474 | 07:58 |
| *** sflanigan has quit IRC | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** rossella_s has joined #openstack-infra | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** mhickey has joined #openstack-infra | 08:00 | |
| *** neiljerram has joined #openstack-infra | 08:01 | |
| *** JudeC has quit IRC | 08:02 | |
| *** ramishra has quit IRC | 08:04 | |
| *** gildub has joined #openstack-infra | 08:05 | |
| *** rossella_s has quit IRC | 08:05 | |
| *** ramishra has joined #openstack-infra | 08:06 | |
| *** markvoelker has quit IRC | 08:08 | |
| *** jpich has joined #openstack-infra | 08:11 | |
| *** armax has joined #openstack-infra | 08:11 | |
| *** dosaboy has quit IRC | 08:11 | |
| *** sree has quit IRC | 08:12 | |
| *** sree has joined #openstack-infra | 08:13 | |
| *** armax has quit IRC | 08:16 | |
| *** sree has quit IRC | 08:17 | |
| *** ltomasbo|away is now known as ltomasbo | 08:17 | |
| *** mriedem has quit IRC | 08:17 | |
| *** lucas-afk is now known as lucasagomes | 08:18 | |
| *** sree has joined #openstack-infra | 08:18 | |
| *** dosaboy has joined #openstack-infra | 08:19 | |
| *** rossella_s has joined #openstack-infra | 08:22 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: config: refactor config get default https://review.openstack.org/474484 | 08:23 |
| *** yamahata has quit IRC | 08:23 | |
| *** tosky has joined #openstack-infra | 08:23 | |
| *** dingyichen has quit IRC | 08:24 | |
| *** dizquierdo has joined #openstack-infra | 08:27 | |
| *** efoley has joined #openstack-infra | 08:28 | |
| *** shardy has quit IRC | 08:31 | |
| *** dtantsur is now known as dtantsur|lunch | 08:33 | |
| *** rossella_s has quit IRC | 08:36 | |
| *** psachin has quit IRC | 08:40 | |
| *** jamesmcarthur has joined #openstack-infra | 08:41 | |
| *** psachin has joined #openstack-infra | 08:41 | |
| *** liujiong has quit IRC | 08:43 | |
| *** psachin has quit IRC | 08:43 | |
| *** liujiong has joined #openstack-infra | 08:43 | |
| *** psachin has joined #openstack-infra | 08:43 | |
| *** jamesmcarthur has quit IRC | 08:45 | |
| *** jaranovich_ has quit IRC | 08:48 | |
| *** dosaboy has quit IRC | 08:50 | |
| *** electrofelix has joined #openstack-infra | 08:51 | |
| *** dosaboy has joined #openstack-infra | 08:51 | |
| *** dosaboy has quit IRC | 08:53 | |
| *** florianf has quit IRC | 08:53 | |
| *** florianf has joined #openstack-infra | 08:54 | |
| *** jiaohaolin has left #openstack-infra | 08:55 | |
| *** yolanda_ has joined #openstack-infra | 08:59 | |
| *** mugsie has quit IRC | 08:59 | |
| openstackgerrit | Lajos Katona proposed openstack-infra/project-config master: Enable q-trunk in neutron dsvm api jobs https://review.openstack.org/474499 | 08:59 |
| *** mugsie has joined #openstack-infra | 09:00 | |
| *** mugsie has quit IRC | 09:00 | |
| *** mugsie has joined #openstack-infra | 09:00 | |
| *** waynr has quit IRC | 09:01 | |
| *** sree has quit IRC | 09:03 | |
| *** markvoelker has joined #openstack-infra | 09:05 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: can_promote function: fix case for oooq when ZUUL_CHANGES is not defined https://review.openstack.org/474504 | 09:07 |
| *** dosaboy has joined #openstack-infra | 09:08 | |
| *** panda|off is now known as panda | 09:10 | |
| *** armax has joined #openstack-infra | 09:12 | |
| *** sree has joined #openstack-infra | 09:12 | |
| *** e0ne has joined #openstack-infra | 09:12 | |
| *** sree has quit IRC | 09:16 | |
| *** armax has quit IRC | 09:17 | |
| *** ramishra has quit IRC | 09:19 | |
| *** ramishra has joined #openstack-infra | 09:19 | |
| *** mugsie has quit IRC | 09:22 | |
| *** mugsie has joined #openstack-infra | 09:22 | |
| *** mugsie has quit IRC | 09:22 | |
| *** mugsie has joined #openstack-infra | 09:22 | |
| *** psachin has quit IRC | 09:29 | |
| *** sree has joined #openstack-infra | 09:30 | |
| *** tumbarka has joined #openstack-infra | 09:36 | |
| *** tushar has quit IRC | 09:36 | |
| *** markvoelker has quit IRC | 09:38 | |
| *** shu-mutou is now known as shu-mutou-AWAY | 09:47 | |
| *** d0ugal has quit IRC | 09:48 | |
| *** d0ugal has joined #openstack-infra | 09:49 | |
| *** stakeda has quit IRC | 09:50 | |
| *** s-shiono has quit IRC | 09:52 | |
| *** dtantsur|lunch is now known as dtantsur | 09:56 | |
| *** xarses_ has quit IRC | 09:59 | |
| *** liujiong has quit IRC | 10:06 | |
| *** armax has joined #openstack-infra | 10:12 | |
| *** jascott1 has quit IRC | 10:15 | |
| *** jascott1 has joined #openstack-infra | 10:15 | |
| *** e0ne has quit IRC | 10:17 | |
| *** armax has quit IRC | 10:17 | |
| *** nicolasbock has joined #openstack-infra | 10:19 | |
| *** jascott1 has quit IRC | 10:19 | |
| *** cuongnv has quit IRC | 10:20 | |
| *** wolverineav has joined #openstack-infra | 10:20 | |
| *** sambetts|afk is now known as sambetts | 10:23 | |
| *** wolverineav has quit IRC | 10:24 | |
| *** wolverineav has joined #openstack-infra | 10:24 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Gzip ara directory files https://review.openstack.org/473569 | 10:26 |
| *** mugsie has quit IRC | 10:27 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/474147 | 10:28 |
| *** mugsie has joined #openstack-infra | 10:29 | |
| *** mugsie has quit IRC | 10:29 | |
| *** mugsie has joined #openstack-infra | 10:29 | |
| *** sflanigan has joined #openstack-infra | 10:30 | |
| *** sflanigan has joined #openstack-infra | 10:30 | |
| *** jpena is now known as jpena|lunch | 10:31 | |
| *** panda is now known as panda|lunch | 10:31 | |
| *** jkilpatr has quit IRC | 10:32 | |
| *** florianf has quit IRC | 10:32 | |
| *** markvoelker has joined #openstack-infra | 10:34 | |
| *** tumbarka has quit IRC | 10:35 | |
| *** florianf has joined #openstack-infra | 10:35 | |
| *** tushar has joined #openstack-infra | 10:35 | |
| *** abramley has joined #openstack-infra | 10:36 | |
| *** derekh has joined #openstack-infra | 10:40 | |
| *** sree has quit IRC | 10:43 | |
| *** katkapilatova has joined #openstack-infra | 10:43 | |
| *** florianf has quit IRC | 10:46 | |
| *** florianf has joined #openstack-infra | 10:47 | |
| *** sdague has joined #openstack-infra | 10:48 | |
| *** jkilpatr has joined #openstack-infra | 10:49 | |
| *** e0ne has joined #openstack-infra | 10:49 | |
| *** shardy has joined #openstack-infra | 10:50 | |
| *** salv-orlando has quit IRC | 10:56 | |
| *** amoralej is now known as amoralej|lunch | 11:02 | |
| *** mhickey has quit IRC | 11:04 | |
| openstackgerrit | Denis V. Meltsaykin proposed openstack-infra/project-config master: [fuel] Grant permissions to fuel-stable-core https://review.openstack.org/474541 | 11:06 |
| *** markvoelker has quit IRC | 11:08 | |
| *** e0ne has quit IRC | 11:09 | |
| *** ralonsoh has quit IRC | 11:10 | |
| *** ralonsoh has joined #openstack-infra | 11:10 | |
| *** armax has joined #openstack-infra | 11:13 | |
| *** armax has quit IRC | 11:18 | |
| *** ericyoung has joined #openstack-infra | 11:18 | |
| *** Hal has quit IRC | 11:24 | |
| *** amitk_ has joined #openstack-infra | 11:30 | |
| *** yolanda__ has joined #openstack-infra | 11:30 | |
| *** yolanda_ has quit IRC | 11:30 | |
| amitk_ | Hi All I am working on CICD for Manila | 11:31 |
| amitk_ | In manila job file I had given the "export DEVSTACK_GATE_TEMPEST_REGEX="^(?=manila_tempest_tests.tests.api).*"" | 11:31 |
| amitk_ | but none of the manila TCs invoke by tempest | 11:32 |
| *** chlong has joined #openstack-infra | 11:32 | |
| amitk_ | any idea | 11:32 |
| *** salv-orlando has joined #openstack-infra | 11:34 | |
| *** akscram has quit IRC | 11:39 | |
| *** rhallisey has joined #openstack-infra | 11:39 | |
| *** akscram has joined #openstack-infra | 11:40 | |
| *** tumbarka has joined #openstack-infra | 11:48 | |
| *** markvoelker has joined #openstack-infra | 11:51 | |
| *** tushar has quit IRC | 11:51 | |
| *** abramley has quit IRC | 11:54 | |
| *** abelur_ has quit IRC | 11:56 | |
| *** amitk_ has quit IRC | 12:02 | |
| *** dizquierdo has quit IRC | 12:04 | |
| *** yolanda__ is now known as yolanda | 12:05 | |
| *** panda|lunch is now known as panda | 12:06 | |
| *** jamesmcarthur has joined #openstack-infra | 12:09 | |
| *** jamesmcarthur has quit IRC | 12:09 | |
| *** jamesmcarthur has joined #openstack-infra | 12:09 | |
| *** martinkopec has joined #openstack-infra | 12:09 | |
| *** namnh has quit IRC | 12:10 | |
| *** tuanluong has quit IRC | 12:10 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 12:12 |
| *** e0ne has joined #openstack-infra | 12:12 | |
| openstackgerrit | Arx Cruz proposed openstack-infra/tripleo-ci master: Fix stackviz collect data https://review.openstack.org/474559 | 12:13 |
| *** armax has joined #openstack-infra | 12:14 | |
| *** ccamacho is now known as ccamacho|lunch | 12:15 | |
| *** amoralej|lunch is now known as amoralej | 12:17 | |
| *** armax has quit IRC | 12:19 | |
| *** prateek has quit IRC | 12:20 | |
| *** trown|outtypewww is now known as trown | 12:24 | |
| *** rlandy has joined #openstack-infra | 12:26 | |
| *** zhurong has quit IRC | 12:26 | |
| *** zhurong has joined #openstack-infra | 12:27 | |
| *** baoli has joined #openstack-infra | 12:31 | |
| *** gordc has joined #openstack-infra | 12:33 | |
| *** jamesdenton has joined #openstack-infra | 12:34 | |
| *** zhurong has quit IRC | 12:36 | |
| *** jamesmcarthur has quit IRC | 12:38 | |
| *** Julien-zte has joined #openstack-infra | 12:39 | |
| ssbarnea_ | any jjb guru around? https://storyboard.openstack.org/#!/story/2001071 | 12:39 |
| *** e0ne has quit IRC | 12:42 | |
| *** liusheng has quit IRC | 12:46 | |
| *** esberglu has joined #openstack-infra | 12:50 | |
| tumbarka | Hi All | 12:50 |
| tumbarka | Unable to trigger CI because of zuul error | 12:51 |
| *** mhickey has joined #openstack-infra | 12:51 | |
| tumbarka | Error message -- Job is not registered with Gearman | 12:52 |
| *** hrubi has quit IRC | 12:52 | |
| *** hrubi has joined #openstack-infra | 12:53 | |
| tumbarka | Log files - https://gist.github.com/anonymous/bd08796bdabcb116d1533e437bf7f897 | 12:54 |
| tumbarka | https://gist.github.com/anonymous/34a6ee0088f536563df3eb5da5e690d4 | 12:56 |
| *** Goneri has joined #openstack-infra | 12:56 | |
| tumbarka | Anybody have any idea? | 12:56 |
| sfinucan | dhellmann: I spotted a note in a Sphinx extension you wrote about how Boolean options | 12:57 |
| sfinucan | work | 12:57 |
| sfinucan | for reference, the attribute either exists (True) or it does not (False). Bit odd, but that's how they do it | 12:57 |
| sfinucan | just fyi :) | 12:57 |
| dhellmann | hmm, yeah, I think I expected them to have a value but you have to just test their presence | 12:57 |
| sfinucan | 'zactly. Very odd design choice but it'll do | 12:57 |
| dhellmann | I just sort of hard-coded the default behavior, and left a note to fix it | 12:58 |
| sfinucan | Yeah, I noticed that | 12:58 |
| sfinucan | Also, your blog from way back on how to write Sphinx extensions - super helpful | 12:58 |
| sfinucan | For a documentation tools, the Sphinx docs are kind of sucky | 12:59 |
| dhellmann | yes, and reno has terrible release notes | 12:59 |
| sfinucan | dhellmann: For the reno thing though, I was going to structure it as a setuptools plugin and then just call that, if available, from pbr | 12:59 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/tripleo-ci master: gzip the ARA report after generating it https://review.openstack.org/468156 | 12:59 |
| dhellmann | cobbler's children, etc. | 12:59 |
| sfinucan | :) | 12:59 |
| sfinucan | plumber's leaky taps | 12:59 |
| dhellmann | sfinucan: stand by, I may have tried that approach | 13:00 |
| dhellmann | sfinucan : did I give you a link to https://review.openstack.org/#/c/350827/3 already? or did I give you a pbr link? | 13:01 |
| dhellmann | putting the plugin in reno resulted in a circular dependency on itself. maybe you'll have better luck resolving that | 13:01 |
| ssbarnea_ | anyone able to vote on https://review.openstack.org/#/c/463729/ ? | 13:02 |
| dhellmann | I was planning to just have a module in reno to do the work and call that from pbr, but not actually make it a setuptools plugin | 13:02 |
| sfinucan | dhellmann: Because reno uses pbr | 13:02 |
| sfinucan | ? | 13:02 |
| sfinucan | I hadn't considered that. I'll see what I can come up with | 13:03 |
| *** sshnaidm is now known as sshnaidm|afkj | 13:03 | |
| *** sshnaidm|afkj is now known as sshnaidm|afk | 13:03 | |
| sfinucan | Also, I'd only got the pbr link. Thanks for that | 13:03 |
| * dhellmann dusts the cobwebs off his memory | 13:03 | |
| dhellmann | yeah, I think, so. There was some issue getting reno to install, because pbr wants setuptools to invoke the plugin in reno | 13:03 |
| dhellmann | so if installation only worked properly every other time | 13:03 |
| dhellmann | the first time it would seem to fail, but put enough of the files in place that the next time it would work | 13:04 |
| dhellmann | I suppose one fix would be to make reno not use pbr, but it also seemed like just having pbr call reno explicitly would be simpler for everyone to understand | 13:04 |
| *** hrubi has quit IRC | 13:06 | |
| sfinucan | Hmm, that's a chicken-egg situation | 13:07 |
| *** gomarivera has joined #openstack-infra | 13:07 | |
| sfinucan | Personally, I'd be fine with not using pbr for reno, given that we don't need most of the features it provides | 13:07 |
| dhellmann | right | 13:07 |
| dhellmann | pbr can see if reno exists by trying to import it | 13:07 |
| dhellmann | it does that with sphinx, too | 13:07 |
| dhellmann | so the pattern is there | 13:07 |
| dhellmann | that would mean having to solve the problem of putting reno's release notes into its tarballs some other way | 13:07 |
| dhellmann | so we'd solve the same problem twice :-) | 13:08 |
| dhellmann | but it would be interesting to explore | 13:08 |
| *** mriedem has joined #openstack-infra | 13:08 | |
| sfinucan | yup - once with pbr and once with plain old setuptools | 13:08 |
| dhellmann | the main thing we take advantage of now is using the git tag to set the version; maybe there's another setuptools plugin that would do that for us | 13:08 |
| sfinucan | doesn't sound like an awful thing to have to do | 13:08 |
| sfinucan | there is | 13:08 |
| sfinucan | https://pypi.python.org/pypi/setuptools_scm | 13:09 |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Add experimental py35 job to barbican-tempest-plugin https://review.openstack.org/474572 | 13:09 |
| *** eharney has joined #openstack-infra | 13:09 | |
| sfinucan | setuptools seems to have been refactored to allow plugins like that | 13:09 |
| dhellmann | ok. that seems like a reasonable avenue to explore, then | 13:09 |
| dhellmann | I think the thing that blocked me on the existing pbr patch was figuring out how to add the file to the set of things to include, rather than generating it | 13:09 |
| dhellmann | but I haven't looked at that patch in a long time, so I could be wrong | 13:10 |
| *** hrubi has joined #openstack-infra | 13:10 | |
| sfinucan | cool. I'll give it a shot and see what happens | 13:10 |
| dhellmann | ok | 13:10 |
| sfinucan | I kind of want to learn how setuptools is working too, so this helps | 13:10 |
| smcginnis | Any idea when reno 2.4.1 will get picked up in the gate? Still getting a bunch of releasenotes jobs timing out. | 13:10 |
| *** jcoufal has joined #openstack-infra | 13:10 | |
| dhellmann | smcginnis : there's a constraints update patch to add it, but it failed a job for some reason so had to be rechecked | 13:10 |
| dhellmann | smcginnis : https://review.openstack.org/#/c/474328/ | 13:11 |
| *** gomarivera has quit IRC | 13:11 | |
| dhellmann | smcginnis : looks like the cinder unit test jobs failed, heh | 13:11 |
| smcginnis | dhellmann: Thanks, I was just about to look for it. | 13:11 |
| smcginnis | irony not lost | 13:11 |
| dhellmann | oh, weird, it failed to install reno | 13:11 |
| dhellmann | couldn't find 2.4.1 | 13:12 |
| *** gouthamr has joined #openstack-infra | 13:12 | |
| dhellmann | must have been an issue with the mirrors | 13:12 |
| fungi | given all the chicken-and-egg problems pbr's solved already, i wouldn't be surprised if someone comes up with a way to pbrize reno effectively | 13:13 |
| *** armax has joined #openstack-infra | 13:13 | |
| smcginnis | dhellmann: Looks like it's finishing the last tempest run in the check queue right now and is happy. | 13:13 |
| dhellmann | smcginnis : cool | 13:13 |
| dhellmann | fungi: I'm sure it's doable, by someone with the time to spend on it. | 13:14 |
| *** ykarel has quit IRC | 13:15 | |
| *** armax has quit IRC | 13:17 | |
| ssbarnea_ | fungi: hi! what should I do get the +2 vote? I have a strong interest in contributing on few infra projects like jenkin-job-builder and gerrit, which I used daily. I observed that these could use some help when it comes to reviewing power. | 13:18 |
| *** hrubi has quit IRC | 13:18 | |
| *** jamesmcarthur has joined #openstack-infra | 13:19 | |
| *** ihrachys has joined #openstack-infra | 13:20 | |
| *** dprince has joined #openstack-infra | 13:20 | |
| *** mdrabe has joined #openstack-infra | 13:20 | |
| *** mdrabe has quit IRC | 13:21 | |
| *** hrubi has joined #openstack-infra | 13:21 | |
| *** mdrabe has joined #openstack-infra | 13:22 | |
| *** hrubi has quit IRC | 13:23 | |
| *** hrubi has joined #openstack-infra | 13:23 | |
| *** askb has quit IRC | 13:26 | |
| *** dizquierdo has joined #openstack-infra | 13:26 | |
| openstackgerrit | Sorin Sbârnea proposed openstack-infra/gerritlib master: Remove pyflakes environment https://review.openstack.org/208250 | 13:26 |
| *** dave-mccowan has joined #openstack-infra | 13:27 | |
| *** rbrndt has joined #openstack-infra | 13:27 | |
| *** jpena|lunch is now known as jpena | 13:27 | |
| fungi | ssbarnea_: zxiiro and electrofelix are your best bet for getting feedback on jjb design proposals these days, i think | 13:28 |
| fungi | the openstack infra team is phasing out our use of it (we're already pinned to an older version, pending zuul v3 which won't need it at all), so we're not really involved in deciding the direction of jjb and how new features or syntax should be implemented | 13:29 |
| ssbarnea_ | fungi: my question was a bit more generic because i observe that there are few projects that really lack the ability to merge incoming CRs due to lack of reviews. There are lots of trivial examples that are even more than 1yr old. | 13:30 |
| ssbarnea_ | fungi: if this is phased out, please lets others takeover the burden on maintained these projects. | 13:30 |
| *** ccamacho|lunch is now known as ccamacho | 13:30 | |
| fungi | ssbarnea_: i agree, that's the case across all of openstack. generally i think we do a good job of reviewing urgent fixes, but less urgent reviews can be tricky since there are a number of applications we consider feature complete and are trying not to overload with new complexity | 13:31 |
| ssbarnea_ | fungi: probably we will follow the route at some point but until there it would be much better for everyone if we could continue to maintain them. | 13:32 |
| *** mariusv has joined #openstack-infra | 13:33 | |
| fungi | ssbarnea_: sure, maintenance can mean a lot of things though. what specific instance did you have in mind? | 13:33 |
| ssbarnea_ | fungi: i could help reviewing (and even fixing bugs) on: jenkins-job-builder and gerrit. In fact I do have few others around me that could do the same bug they avoided it because "too much effort to get contributions in". | 13:35 |
| *** jamesden_ has joined #openstack-infra | 13:35 | |
| *** jamesdenton has quit IRC | 13:36 | |
| fungi | ssbarnea_: sure, as i said the jjb team is mostly autonomous at this point and i and the rest of the openstack infra team don't exert much pressure on their design decisions any longer since we're not actively consuming new versions of it | 13:36 |
| *** thorst has joined #openstack-infra | 13:36 | |
| fungi | though if they're having trouble keeping up (i know the 2.0 version they've been working on has been taking a while), then maybe they should consider expanding their team | 13:37 |
| openstackgerrit | Merged openstack-infra/gerritlib master: Remove pyflakes environment https://review.openstack.org/208250 | 13:37 |
| *** hrubi has quit IRC | 13:38 | |
| ssbarnea_ | fungi: TBH, I am not a big fun of JJB but using it is a huge part of my job, and will stay like this for quite some time (no removal plans yet). So, it makes sense to make it better. | 13:38 |
| fungi | sure, that seems fine to me too. the most active people on jjb last time i checked were zxiiro and electrofelix, which is why i suggested reaching out to them to get feedback on your proposal for parameter defaults | 13:39 |
| *** jamesmcarthur has quit IRC | 13:40 | |
| *** hrubi has joined #openstack-infra | 13:40 | |
| ssbarnea_ | fungi: I will contact them on this aspect. Someone from my team proposed a migration to jinja2 but I think that this sounds more like a v3 kind of feature. And we don't really know if there will ever be a v3. | 13:41 |
| *** eharney has quit IRC | 13:42 | |
| *** mugsie has quit IRC | 13:43 | |
| *** bobh has joined #openstack-infra | 13:43 | |
| *** mugsie has joined #openstack-infra | 13:46 | |
| *** zzzeek has quit IRC | 13:46 | |
| *** bobh has quit IRC | 13:46 | |
| *** hashar has quit IRC | 13:47 | |
| *** zzzeek has joined #openstack-infra | 13:47 | |
| *** bobh has joined #openstack-infra | 13:49 | |
| *** zhurong has joined #openstack-infra | 13:49 | |
| *** hashar has joined #openstack-infra | 13:54 | |
| *** mariusv has quit IRC | 13:55 | |
| openstackgerrit | Merged openstack-infra/shade master: Increase timeout for volume tests https://review.openstack.org/474388 | 13:55 |
| *** eharney has joined #openstack-infra | 13:57 | |
| *** jistr is now known as jistr|mtg | 13:59 | |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config master: neutron: Pass python version to gate hook https://review.openstack.org/474585 | 13:59 |
| *** spzala has joined #openstack-infra | 14:01 | |
| *** bnemec has joined #openstack-infra | 14:01 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for neutron functional test bug 1698145 https://review.openstack.org/474587 | 14:02 |
| openstack | bug 1698145 in neutron "functional test test_dvr_gateway_host_binding_is_set fails with "testtools.matchers._impl.MismatchError: u'host0' != u'standby'"" [Undecided,New] https://launchpad.net/bugs/1698145 | 14:02 |
| *** spzala has quit IRC | 14:03 | |
| *** gomarivera has joined #openstack-infra | 14:04 | |
| *** hashar has quit IRC | 14:04 | |
| *** eharney_ has joined #openstack-infra | 14:08 | |
| *** mrmartin has joined #openstack-infra | 14:09 | |
| *** eharney has quit IRC | 14:10 | |
| *** zz_dimtruck is now known as dimtruck | 14:10 | |
| *** eharney_ is now known as eharney | 14:11 | |
| *** hashar has joined #openstack-infra | 14:12 | |
| *** gildub has quit IRC | 14:16 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for neutron functional test bug 1698145 https://review.openstack.org/474587 | 14:16 |
| openstack | bug 1698145 in neutron "functional test test_dvr_gateway_host_binding_is_set fails with "testtools.matchers._impl.MismatchError: u'host0' != u'standby'"" [Undecided,New] https://launchpad.net/bugs/1698145 | 14:16 |
| *** jascott1 has joined #openstack-infra | 14:17 | |
| *** pgadiya has quit IRC | 14:18 | |
| *** jascott1 has quit IRC | 14:21 | |
| *** zhurong has quit IRC | 14:22 | |
| *** abramley has joined #openstack-infra | 14:22 | |
| *** ccamacho has quit IRC | 14:22 | |
| *** ccamacho has joined #openstack-infra | 14:22 | |
| *** jamesden_ has quit IRC | 14:27 | |
| *** jamesdenton has joined #openstack-infra | 14:29 | |
| *** dhajare has quit IRC | 14:30 | |
| *** yland has joined #openstack-infra | 14:34 | |
| *** kgiusti has joined #openstack-infra | 14:35 | |
| *** xarses_ has joined #openstack-infra | 14:41 | |
| *** mugsie has quit IRC | 14:41 | |
| *** andreww has joined #openstack-infra | 14:42 | |
| *** mugsie has joined #openstack-infra | 14:42 | |
| *** mugsie has quit IRC | 14:42 | |
| *** mugsie has joined #openstack-infra | 14:42 | |
| *** mdrabe has quit IRC | 14:43 | |
| *** mdrabe has joined #openstack-infra | 14:43 | |
| *** xarses_ has quit IRC | 14:46 | |
| *** andreww has quit IRC | 14:46 | |
| *** xarses_ has joined #openstack-infra | 14:46 | |
| *** makowals has quit IRC | 14:47 | |
| *** andreww has joined #openstack-infra | 14:48 | |
| *** Swami has joined #openstack-infra | 14:51 | |
| *** xarses_ has quit IRC | 14:51 | |
| *** cshastri has quit IRC | 14:51 | |
| *** Julien-zte has quit IRC | 14:52 | |
| *** andreww has quit IRC | 14:52 | |
| *** xarses_ has joined #openstack-infra | 14:52 | |
| *** makowals has joined #openstack-infra | 14:53 | |
| *** e0ne has joined #openstack-infra | 14:53 | |
| *** andreww has joined #openstack-infra | 14:53 | |
| *** yland has quit IRC | 14:56 | |
| *** xarses_ has quit IRC | 14:57 | |
| *** andreww has quit IRC | 14:57 | |
| *** xarses_ has joined #openstack-infra | 14:57 | |
| *** andreww has joined #openstack-infra | 14:59 | |
| *** sshnaidm|afk is now known as sshnaidm | 14:59 | |
| *** jistr|mtg is now known as jistr | 14:59 | |
| *** xarses_ has quit IRC | 15:02 | |
| *** dtantsur is now known as dtantsur|bbl | 15:02 | |
| *** andreww has quit IRC | 15:03 | |
| *** xarses_ has joined #openstack-infra | 15:03 | |
| *** burgerk has joined #openstack-infra | 15:04 | |
| *** tumbarka has quit IRC | 15:04 | |
| *** umbarkar has joined #openstack-infra | 15:04 | |
| *** mdrabe has quit IRC | 15:04 | |
| *** tushar has joined #openstack-infra | 15:06 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: Add boot-from-volume support for nodes https://review.openstack.org/474607 | 15:07 |
| zxiiro | ssbarnea_: let me know if you have patches that aren't getting reviewed, I'm usually pretty good at staying ontop of them but sometimes I might miss one. We need 2 people to +2 though before it will get merged. | 15:09 |
| *** umbarkar has quit IRC | 15:09 | |
| *** rcernin has quit IRC | 15:09 | |
| *** dklyle is now known as david-lyle | 15:10 | |
| *** gyee has joined #openstack-infra | 15:10 | |
| zxiiro | ssbarnea_: we use JJB heavily at LinuxFoundation so we plan to maintain it for the foreseeable future. JJB 2.0 release is being held up by some fixes that we need to get in. should ping electrofelix about where we're at. Personally I'd like to see JJB get released more frequently. | 15:10 |
| *** dhinesh has joined #openstack-infra | 15:11 | |
| *** mdrabe has joined #openstack-infra | 15:11 | |
| *** felipemonteiro has joined #openstack-infra | 15:18 | |
| *** waynr has joined #openstack-infra | 15:19 | |
| *** florianf has quit IRC | 15:24 | |
| *** jaypipes has joined #openstack-infra | 15:24 | |
| *** claudiub has quit IRC | 15:25 | |
| clarkb | fungi: do we have log consumption results? | 15:26 |
| fungi | yup, lemme paste an updated analysis | 15:26 |
| *** hongbin has joined #openstack-infra | 15:28 | |
| clarkb | fungi: when thats done maybe you can quickly rereview https://review.openstack.org/#/c/437050/4 there was a minor issue with the last patchset you +2'd since fixed (and cofirmed by CI to be working) | 15:28 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: config: refactor config get default https://review.openstack.org/474484 | 15:29 |
| *** hashar has quit IRC | 15:31 | |
| *** yamahata has joined #openstack-infra | 15:32 | |
| clarkb | http://logs.openstack.org/13/473113/2/gate/gate-puppet-logstash-puppet-beaker-rspec-centos-7/bee0100/console.html#_2017-06-15_00_03_25_895540 DNS resolution errors hit again | 15:37 |
| clarkb | also in osic | 15:37 |
| clarkb | cloudnull: hey if you are around it seems like we are having a higher than normal/expected error rate resolving names with DNS in osic. We set up local caching resolvers on our test machines to hit google dns via ipv6 first then ipv4. Any idea what may be going on? | 15:38 |
| cloudnull | not a clue, happy to go look though | 15:38 |
| *** florianf has joined #openstack-infra | 15:38 | |
| cloudnull | i was seeing that on cent7 images within the osic the other day, are we seeing this on more than just cent ? | 15:39 |
| clarkb | cloudnull: ya we have seen it on ubuntu xenial too so I don't think its a distro specific issue | 15:39 |
| cloudnull | ok | 15:39 |
| * cloudnull going to go look | 15:39 | |
| clarkb | cloudnull: I'll get you a logstash query link and specifics from our unbound config in images | 15:40 |
| *** jamesmcarthur has joined #openstack-infra | 15:40 | |
| *** jkilpatr_ has joined #openstack-infra | 15:40 | |
| fungi | clarkb: http://paste.openstack.org/show/612724 | 15:42 |
| clarkb | cloudnull: http://logstash.openstack.org/#/dashboard/file/logstash.json?query=message:%5C%22wget:%20unable%20to%20resolve%20host%20address%20%27tarballs.openstack.org%27%5C%22%20AND%20tags:%5C%22console%5C%22%20AND%20voting:1&from=864000s that shows a subset of the failures (just those from wget so not including the fail I linked above that was ruby bundler) | 15:43 |
| clarkb | cloudnull: the vast majority are on osic and ubuntu is represented too | 15:43 |
| *** jkilpatr has quit IRC | 15:43 | |
| clarkb | fungi: thats impressive for the tripleo jobs :) | 15:44 |
| *** jamesmcarthur has quit IRC | 15:44 | |
| *** jamesmca_ has joined #openstack-infra | 15:44 | |
| clarkb | fungi: its absically the top two non tripleo jobs combined :) | 15:44 |
| *** rcernin has joined #openstack-infra | 15:44 | |
| *** cshastri has joined #openstack-infra | 15:46 | |
| *** rwsu has joined #openstack-infra | 15:47 | |
| fungi | yup | 15:47 |
| fungi | so an easy solution would be to stop running those, or find ways to run them less often? | 15:47 |
| fungi | at least until they figure out what needs to be logged | 15:48 |
| clarkb | cloudnull: http://paste.openstack.org/show/612726/ | 15:48 |
| *** dtantsur|bbl is now known as dtantsur | 15:49 | |
| clarkb | and if we can get rid of the v3 only job (beacuse it is potentially redundant now or can be made redundant?) we'd save a good chunk too | 15:49 |
| clarkb | lbragstad: ^ do you know what the plan is for the keystone v3 api only tempest job? | 15:49 |
| *** e0ne has quit IRC | 15:51 | |
| *** sbezverk has joined #openstack-infra | 15:51 | |
| clarkb | fungi: cloudnull I want to say I've seen a different ipv6 dns addr in other places | 15:51 |
| clarkb | so I'm going to poke around on other hosts | 15:52 |
| *** jamesmca_ has quit IRC | 15:53 | |
| lbragstad | clarkb: i assume it would be run by default? | 15:53 |
| openstackgerrit | Merged openstack-infra/puppet-logstash master: Remove unnecessary beaker tests https://review.openstack.org/473113 | 15:54 |
| clarkb | lbragstad: I think the default job uses v3 but also has v2 enabled | 15:54 |
| lbragstad | clarkb: yeah - that sounds accurate | 15:54 |
| clarkb | but I would need to double check that, thinking we may finally be to a point where we can v3 only in the default job? | 15:54 |
| *** katkapilatova has left #openstack-infra | 15:54 | |
| clarkb | or maybe just run the v3 only job against keystone changes? | 15:54 |
| lbragstad | clarkb: i like that idea | 15:55 |
| lbragstad | clarkb: that is consistent with keystone's push to get folks to use v3 | 15:55 |
| lbragstad | clarkb: is there a process for removing the v3+v2 job or is that what we need to figure out? | 15:57 |
| clarkb | lbragstad: I think that is what we'd need to figure out. Likely we'd modify the defaults in devstack, see what breaks in check, then remove the v3 only job once that is in place | 15:59 |
| clarkb | and then run a v2 + v3 job in keystone or something? | 15:59 |
| lbragstad | clarkb: that sounds reasonable | 15:59 |
| lbragstad | clarkb: do we need to take specific steps from a communication perspective? | 16:00 |
| clarkb | ya we probably want ot make sure that doesn't break any QA plans and announce the switch so that folks like shade can set up their testing as appropriate (since they test specifically that they work with all the apis) | 16:01 |
| lbragstad | clarkb: so - a note to the dev and operator mailing list? | 16:03 |
| clarkb | maybe not even operator, since this is all development tooling (and ops have been told v2 is deprecated for years iirc) | 16:03 |
| clarkb | lbragstad: I just bring it up because it shows up hgih in fungi's list of logs disk consumers and I think its mostly redundant at this point? maybe? | 16:03 |
| *** eharney has quit IRC | 16:05 | |
| *** jpich has quit IRC | 16:06 | |
| lbragstad | clarkb: the v3 job shows up as high as a disk consumer or the v3+v2 job does? | 16:07 |
| cloudnull | clarkb: I'm not seeing anything that stands out that could be causing an issue. | 16:07 |
| *** lucasagomes is now known as lucas-afk | 16:07 | |
| *** derekh has quit IRC | 16:07 | |
| clarkb | lbragstad: well v3+v2 is most jobs so it does too. Just wondering if v3 only as a separate thing run on all the changes makes sense anymore (I thought it was supposed to be temporary to get v3 only working then switch everything to v3 only) | 16:08 |
| cloudnull | I've updated the ipv6 network to use opendns by default and googledns as a fallback. | 16:08 |
| clarkb | lbragstad: http://paste.openstack.org/show/612724/ is the list | 16:08 |
| cloudnull | idk if we could do the same with the unbound forwarders ? | 16:08 |
| *** aadams has joined #openstack-infra | 16:08 | |
| cloudnull | http://paste.openstack.org/show/612729/ | 16:08 |
| *** jamesmcarthur has joined #openstack-infra | 16:08 | |
| clarkb | cloudnull: ya its possible that having only a single google server there (listed via ipv6 then ipv4 address) opens us to problems if a server decides to blacklist us or throttle | 16:09 |
| clarkb | cloudnull: probably not a bad idea to add a fallback like opendns | 16:09 |
| clarkb | infra-root ^ thoughts? | 16:09 |
| cloudnull | maybe we could add something like http://paste.openstack.org/ to /etc/unbound/forwarding.conf ? | 16:10 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Ping test horizon https://review.openstack.org/472758 | 16:10 |
| aadams | Hello everybody. I have another problem with a failing review https://review.openstack.org/#/c/471134/ . Does anybody here have any advice or can point me in the right dirction? | 16:12 |
| *** rcernin has quit IRC | 16:12 | |
| jeblair | cloudnull: your last link went awry | 16:13 |
| cloudnull | opps http://paste.openstack.org/show/612730/ -- ^ clarkb/infra-root | 16:13 |
| clarkb | aadams: http://logs.openstack.org/34/471134/12/check/gate-monasca-common-python35/800a7c6/console.html#_2017-06-15_14_47_56_544261 is the problem. Your python35 job didn't actually run any tests so it got failed by the test framework | 16:13 |
| clarkb | aadams: you should be able to reproduce locally using tox -epy35 | 16:13 |
| aadams | correct. I can reporduce | 16:14 |
| clarkb | cloudnull: I think we likely want to have ipv6 listed first (assumed they are checked in order) just to prefer ipv6 particularly in osic where ipv4 to the outside world means NAT | 16:14 |
| clarkb | but we can fiddle with that over time if necessary | 16:14 |
| *** SumitNaiksatam has joined #openstack-infra | 16:15 | |
| clarkb | aadams: you will need to modify things so that at least one test runs and passes, it could be a trivial assert True test | 16:15 |
| jeblair | cloudnull, clarkb: yeah, wfm. the intent was "univesally accessible caching dns structure that we don't run and is not tied to any cloud" adding opendns to the mix seems good. | 16:15 |
| jeblair | cloudnull, cloudnull: having said that, i'm a little surprised there are problems? | 16:15 |
| *** dhinesh has quit IRC | 16:15 | |
| lbragstad | clarkb: sorry - just clarifying here, are we going to make the v3 only job mandatory then remove the v2+v3 job? It sounds like you want to remove the v3 job? | 16:15 |
| clarkb | aadams: but we have the no tests run check in there because we've found regressions where tests passed because no tests actually ran | 16:15 |
| cloudnull | jeblair: me too. i have no idea why there would be issues with it. | 16:16 |
| clarkb | jeblair: we are having quite a few dns name resolution errors, predominantly in osic but also in infracloud a bit | 16:16 |
| aadams | clarkb: Ok thanks. Ill see what I can do | 16:16 |
| *** jamesdenton has quit IRC | 16:16 | |
| cloudnull | I can reproduce it building a mess of vms | 16:16 |
| cloudnull | and I cant see anything off with the neutron etc | 16:16 |
| clarkb | lbragstad: typically what we do is we modify hte "default" job to test some "standard" rather than constantly replace it with with $specificthingjob | 16:16 |
| jeblair | clarkb: that's interesting. i wonder if google dns is under pressure or is having to be a little less generous... | 16:17 |
| clarkb | lbragstad: so what you do is test something works reliably in a job dedicated to that item, then modify the default job to use it as part of the standard, and delete the job that was created just for that item | 16:17 |
| *** martinkopec has quit IRC | 16:17 | |
| clarkb | lbragstad: otherwise its hard to keep track fo what jobs are what and what should pass and how to query logstash properly and so on | 16:17 |
| pabelanger | clarkb: jeblair: cloudnull: IIRC, we are only using 8.8.8.8 on images, maybe we should add 8.8.4.4 too? | 16:18 |
| clarkb | pabelanger: I think having diversity of provider is a good first step before adding additional hosts within a provider (to avoid the blacklist/throttling issues) | 16:18 |
| pabelanger | wfm | 16:19 |
| *** martinkopec has joined #openstack-infra | 16:19 | |
| *** eharney has joined #openstack-infra | 16:20 | |
| clarkb | I can work on a patch to add that ot our images in a bit | 16:20 |
| clarkb | but I need breakfast first | 16:20 |
| *** yamahata has quit IRC | 16:21 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack-infra/project-config master: Add OpenDNS resolvers and move google to fallback https://review.openstack.org/474630 | 16:21 |
| clarkb | oh cloudnull is way ahead of me :) | 16:21 |
| cloudnull | IDK if that's all we'd need | 16:21 |
| cloudnull | but maybe? | 16:21 |
| cloudnull | im still poking at osic to find some fault their | 16:22 |
| lbragstad | clarkb: ah - ok | 16:22 |
| clarkb | cloudnull: great, thanks for the help on this | 16:23 |
| lbragstad | clarkb: i think that makes sense - just trying to think of how the work would lay out so that nothing is missed test coverage wise | 16:23 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 16:23 |
| *** Apoorva has joined #openstack-infra | 16:24 | |
| *** jascott1 has joined #openstack-infra | 16:26 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 16:29 |
| *** Apoorva_ has joined #openstack-infra | 16:30 | |
| *** cody-somerville has quit IRC | 16:31 | |
| *** cody-somerville has joined #openstack-infra | 16:31 | |
| *** Apoorva has quit IRC | 16:32 | |
| aadams | clarkb: The problem seems to be with a file called .testrepository/times.dbm . When I deleted that file I pass tests locally. | 16:32 |
| *** abramley has quit IRC | 16:32 | |
| *** martinkopec has quit IRC | 16:33 | |
| clarkb | lbragstad: ya thats the bit where I have to defer to you :) | 16:35 |
| *** jamesmcarthur has quit IRC | 16:35 | |
| clarkb | aadams: I think that is because python3 and python2 have different default db types. Is tox -epy35 running any tests or just succeeding trivialy because no tests run? | 16:35 |
| *** jkilpatr has joined #openstack-infra | 16:36 | |
| aadams | clarkb: Locally it ran 59 tests and passed | 16:36 |
| *** jkilpatr_ has quit IRC | 16:37 | |
| pabelanger | Could not get lock /var/lib/dpkg/lock - open (11 Resource temporarily unavailable) | 16:38 |
| *** ijw has joined #openstack-infra | 16:38 | |
| *** arxcruz has quit IRC | 16:38 | |
| pabelanger | for some reason, I'm now seeing that on images in infracloud | 16:38 |
| *** dtantsur is now known as dtantsur|afk | 16:38 | |
| pabelanger | this is using nl01.o.o | 16:38 |
| clarkb | pabelanger: perhaps an image build left the lock held? | 16:39 |
| *** arxcruz has joined #openstack-infra | 16:40 | |
| openstackgerrit | Doug Hellmann proposed openstack-dev/pbr master: allow user to override the output location of api docs https://review.openstack.org/473983 | 16:40 |
| pabelanger | clarkb: that is my guess | 16:41 |
| pabelanger | but curious why we don't see it in production nodepool | 16:41 |
| *** eharney has quit IRC | 16:41 | |
| *** yland has joined #openstack-infra | 16:43 | |
| openstackgerrit | Doug Hellmann proposed openstack-dev/pbr master: allow user to override the output location of api docs https://review.openstack.org/473983 | 16:43 |
| pabelanger | wonder if our configure_mirror.sh script is doing something | 16:43 |
| clarkb | aadams: if I run it locally it runs no tests, can you paste what you are seeing somehwere? | 16:45 |
| *** bobh has quit IRC | 16:45 | |
| clarkb | aadams: so I can reproduce the zuul result | 16:45 |
| *** jpena is now known as jpena|off | 16:46 | |
| *** tesseract has quit IRC | 16:46 | |
| aadams | clarkb: Locally it runs all tests that run with -epy27. It runs and passes 59 tests after I delete that file. I would love to post what I am seeing. Is there anywhere in particular you want me to do it? | 16:47 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 16:47 |
| clarkb | aadams: paste.openstack.org is a service we host for it, but gist or pastie or any other paste service is fine | 16:48 |
| aadams | clarkb: ok, Im on it. Thanks for your help | 16:49 |
| *** jamesmcarthur has joined #openstack-infra | 16:50 | |
| clarkb | cloudnull: your change looked good, I have approved it | 16:51 |
| clarkb | I even checked I can resolve against both IP addresses :) | 16:51 |
| aadams | clarkb: http://paste.openstack.org/show/612733/ | 16:52 |
| *** jamesmcarthur has quit IRC | 16:53 | |
| *** jamesmcarthur has joined #openstack-infra | 16:53 | |
| clarkb | aadams: interesting, if I delete that file and rerun I get the zuul result again. Non-zero exit code (2) from test listing. and The test run didn't actually run any tests | 16:54 |
| clarkb | aadams: the non zero exit code from test listing is important though it telss us what specifically is failing. Now to run that manually and see if we can get better more verbose messages | 16:55 |
| *** dizquierdo has quit IRC | 16:55 | |
| aadams | clarkb: I do get the same error for the second test that gets ran, either py27 or py35 | 16:56 |
| *** ltomasbo is now known as ltomasbo|away | 16:56 | |
| *** yamahata has joined #openstack-infra | 16:57 | |
| *** dizquierdo has joined #openstack-infra | 16:58 | |
| clarkb | aadams: http://paste.openstack.org/show/612734/ | 16:58 |
| openstackgerrit | Merged openstack-infra/project-config master: Add OpenDNS resolvers and move google to fallback https://review.openstack.org/474630 | 16:59 |
| clarkb | aadams: thats with python36 not python35 but I expect it is the same problem in both places | 16:59 |
| aadams | clarkb: oh my, that's my bad | 16:59 |
| *** Swami has quit IRC | 16:59 | |
| clarkb | aadams: I sourced the tox virtualenv then ran testr list-tests to get the verbose error | 17:00 |
| *** eharney has joined #openstack-infra | 17:00 | |
| *** yamahata has quit IRC | 17:00 | |
| *** yamahata_ has joined #openstack-infra | 17:01 | |
| *** ralonsoh has quit IRC | 17:01 | |
| aadams | clarkb: Thanks. Now I see locally too. Thanks a million :) | 17:02 |
| clarkb | awesome | 17:02 |
| clarkb | glad I could help | 17:02 |
| *** mhickey has quit IRC | 17:02 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Explicitly set REVIEW_RELEASE on master https://review.openstack.org/451445 | 17:03 |
| *** neiljerram has quit IRC | 17:03 | |
| *** tushar1 has joined #openstack-infra | 17:03 | |
| clarkb | fungi: ^ maybe we should bug bnemec and EmilienM more about the logging situation >_> | 17:03 |
| EmilienM | hey | 17:03 |
| *** dhinesh has joined #openstack-infra | 17:04 | |
| clarkb | EmilienM: so fungi did a random but representative sampling of disk usage on the logs filesystem. And collected usage by job name. http://paste.openstack.org/show/612724/ is the result of that | 17:05 |
| clarkb | EmilienM: that one tripleo job is about as much disk as the next two jobs combined | 17:05 |
| clarkb | EmilienM: I think that shows that tripleo is a big consumer of our disk right now so more priority on addressing the logging issues would be much appreciated | 17:06 |
| * bnemec proposes shutting off the offending jobs until someone fixes them | 17:06 | |
| fungi | yeah, methodology here was to collect a representative sample of ~21% of all changes for which we have logs on logs.o.o, run du over each job for every patchset therein, and then add the sizes together by job name | 17:07 |
| *** ramishra has quit IRC | 17:07 | |
| *** abramley has joined #openstack-infra | 17:07 | |
| fungi | that gets us an accurate picture of, if we turned off some particular job(s) right now and/or removed their logs form the logserver, how much space we'd free up | 17:08 |
| EmilienM | sshnaidm: ^ | 17:08 |
| *** sambetts is now known as sambetts|afk | 17:08 | |
| *** dizquierdo has quit IRC | 17:08 | |
| EmilienM | sshnaidm: is the gzip patch for ara did help? | 17:08 |
| *** dizquierdo has joined #openstack-infra | 17:08 | |
| clarkb | EmilienM: the gzip patch for ara is nice and good, but its like 27MB -> 5MB ara consumption per job | 17:08 |
| *** dhinesh has quit IRC | 17:08 | |
| EmilienM | fungi: removing gate-tripleo-ci-centos-7-ovb-containers-oooq-nv logs right now : go for it | 17:08 |
| clarkb | EmilienM: when jobs take up to !GB each | 17:09 |
| sshnaidm | EmilienM, after it's merged it will save us about 15M, not critical | 17:09 |
| clarkb | er 1GB | 17:09 |
| *** vhosakot has joined #openstack-infra | 17:09 | |
| bnemec | TBH, I'm on the verge of proposing that we remove that ovb-containers job anyway since it's never been stable and nobody seems to care that much. | 17:09 |
| fungi | this was prompted by our need to further reduce our log retention this week from 45 days to 30 days to avoid filling up the filesystem mid-week | 17:09 |
| clarkb | the bigger thing imo is removing the logging of redundant /etc copies | 17:09 |
| *** abramley has quit IRC | 17:09 | |
| sshnaidm | bnemec, it's actually stable last time | 17:09 |
| EmilienM | bnemec: you and I know it's our critical feature this cycle, it's not that easy | 17:09 |
| bnemec | EmilienM: Right, and I want to make it clear to people that we need to be CI'ing it, which right now we're essentially not. | 17:10 |
| sshnaidm | EmilienM, after deep-dive today maybe we can start reduce logs size in containers job.. | 17:10 |
| bnemec | Because the job is non-voting and perpetually failing on a regular basis. | 17:10 |
| *** csomerville has joined #openstack-infra | 17:10 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Default bubblewrap to work_root https://review.openstack.org/473099 | 17:10 |
| fungi | to take the top offender there, logs for gate-tripleo-ci-centos-7-ovb-containers-oooq-nv runs account for 8% of the total ci logging for all changes | 17:11 |
| *** JudeC has joined #openstack-infra | 17:11 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Read layout from BuildSet in first merge scenario https://review.openstack.org/474064 | 17:12 |
| fungi | next worst, gate-tempest-dsvm-neutron-full-ubuntu-xenial logs are 5.5% | 17:12 |
| clarkb | fungi: but gate-tempest-dsvm-neutron-full-ubuntu-xenial has 10x the job runs or so right? | 17:12 |
| fungi | and in a separate analysis, i determined that we run that job 10x as often | 17:12 |
| fungi | yep | 17:12 |
| *** jtomasek has quit IRC | 17:13 | |
| *** cody-somerville has quit IRC | 17:13 | |
| fungi | so basically the average gate-tripleo-ci-centos-7-ovb-containers-oooq-nv job archives ~15x as much log data as the average gate-tempest-dsvm-neutron-full-ubuntu-xenial job | 17:13 |
| EmilienM | fungi: can you purge gate-tripleo-ci-centos-7-ovb-containers-oooq-nv now please? | 17:14 |
| *** bobh has joined #openstack-infra | 17:14 | |
| *** salv-orl_ has joined #openstack-infra | 17:14 | |
| inc0 | can I have missing review please?:) https://review.openstack.org/#/c/471428/ | 17:14 |
| clarkb | I think before we purge anything we need to fix it otherwise it will just reaccumulate | 17:14 |
| lbragstad | clarkb: ack - let me do some follow up with stevemar just to make sure i get the full context | 17:15 |
| lbragstad | of the v3-only job | 17:15 |
| clarkb | lbragstad: sounds good, thanks | 17:15 |
| fungi | EmilienM: "now" is a bit of a challenge. it takes days, literally, for us to groom our logs site's filesystem. but i can start it if desired | 17:16 |
| clarkb | fungi: also we did a pretty nice sized pruning of the gate-tempest-dsvm-neutron-full-ubuntu-xenial's logs so I expect its usage to fall as we rotate out the old logs | 17:16 |
| *** pcaruana has quit IRC | 17:16 | |
| fungi | yes, probably so | 17:16 |
| *** trown is now known as trown|lunch | 17:16 | |
| *** panda is now known as panda|ko | 17:16 | |
| EmilienM | sshnaidm: do we have any work in progress to stop saving the redundant /etc copies ? | 17:17 |
| EmilienM | weshay: ^ | 17:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 17:17 |
| *** salv-orlando has quit IRC | 17:17 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Limit github reporters to event types https://review.openstack.org/474300 | 17:18 |
| fungi | by my estimate, if we purge all gate-tripleo-ci-centos-7-ovb-containers-oooq-nv that will free up ~640gb which would give us at least a little more breathing room based on http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=717&rra_id=all | 17:18 |
| sshnaidm | EmilienM, not that I'm aware about | 17:18 |
| *** greghaynes is now known as greghayn1 | 17:18 | |
| *** greghaynes has joined #openstack-infra | 17:18 | |
| fungi | but you can also see from grouping the top offender jobs in http://paste.openstack.org/show/612724/ by framework where some other gains could be made too | 17:19 |
| *** tosky has quit IRC | 17:19 | |
| EmilienM | fungi: I suggest to start purging while we figure out the /etc thing | 17:19 |
| fungi | EmilienM: i can do that, thanks | 17:19 |
| *** aadams has quit IRC | 17:21 | |
| *** greghaynes has quit IRC | 17:22 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Allow manual request for overlay network in multinode https://review.openstack.org/437050 | 17:22 |
| bnemec | EmilienM: I opened an alert bug so this doesn't get swept under the rug again: https://bugs.launchpad.net/tripleo/+bug/1698172 | 17:22 |
| openstack | Launchpad bug 1698172 in tripleo "ovb-containers job logs excessive amounts of data" [Critical,Triaged] | 17:22 |
| *** greghaynes has joined #openstack-infra | 17:23 | |
| *** SumitNaiksatam has quit IRC | 17:23 | |
| bnemec | EmilienM: I also opened https://bugs.launchpad.net/tripleo/+bug/1698175 since that job is basically a waste of resources all around right now. | 17:24 |
| openstack | Launchpad bug 1698175 in tripleo "ovb-containers job is stuck in non-voting limbo" [Critical,Triaged] | 17:24 |
| *** jcoufal_ has joined #openstack-infra | 17:24 | |
| *** greghayn1 has quit IRC | 17:25 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:25 | |
| *** jcoufal has quit IRC | 17:25 | |
| *** jcoufal has joined #openstack-infra | 17:29 | |
| EmilienM | bnemec: thx | 17:29 |
| EmilienM | bnemec: I proposed it voting btw: https://review.openstack.org/#/c/472536/ | 17:30 |
| EmilienM | bnemec: but we need to fix the logging stuff first probably | 17:30 |
| *** JudeC has quit IRC | 17:30 | |
| *** Swami has joined #openstack-infra | 17:30 | |
| *** jcoufal_ has quit IRC | 17:31 | |
| *** SumitNaiksatam has quit IRC | 17:31 | |
| *** jamesmcarthur has quit IRC | 17:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 17:33 |
| *** SumitNaiksatam has joined #openstack-infra | 17:35 | |
| *** cshastri has quit IRC | 17:35 | |
| *** yland has quit IRC | 17:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 17:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Configure regional mirrors for our jobs https://review.openstack.org/473850 | 17:37 |
| fungi | infra-root: the certcheck e-mails were me testing to make sure they're working, sorry for the noise | 17:39 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Add missing cert checks https://review.openstack.org/474713 | 17:41 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 17:42 |
| *** dizquierdo has quit IRC | 17:42 | |
| tushar1 | Hi All | 17:42 |
| tushar1 | Unable to trigger CI because of zuul error | 17:43 |
| tushar1 | Error message -- Job is not registered with Gearman | 17:43 |
| tushar1 | Log files - https://gist.github.com/anonymous/bd08796bdabcb116d1533e437bf7f897 | 17:43 |
| tushar1 | https://gist.github.com/anonymous/34a6ee0088f536563df3eb5da5e690d4 | 17:43 |
| tushar1 | Anybody have any idea? | 17:43 |
| clarkb | tushar1: have you checked that jenkins has the gearman plugin installed and enabled and connected to the gearman server? | 17:44 |
| *** jascott1 has quit IRC | 17:46 | |
| tushar1 | clarkb: yes gearman plugin is installed ,enabled and connected | 17:46 |
| sshnaidm | any problem with dns on infra? curl: (6) Could not resolve host: download.cirros-cloud.net; Unknown error | 17:46 |
| *** jascott1 has joined #openstack-infra | 17:46 | |
| clarkb | tushar1: ok, for a job to register there has to be a jenkins slave with the same label as the job. That would be the next thing I woudl check | 17:46 |
| tushar1 | clarkb : infact it was working properly earlier, After I created the new DIB image with new nodepool elements, it stopped working | 17:47 |
| clarkb | sshnaidm: yes, its an issue we are tracking with elastic-recheck, and I recently approved a change to add opendns to our resolvers list | 17:47 |
| sshnaidm | clarkb, and what it was before? google dns? | 17:47 |
| clarkb | tushar1: double check that your job label matches the slave label | 17:47 |
| clarkb | sshnaidm: yes, google dns, so now we will have opendsn falling back to google | 17:47 |
| tushar1 | clarkb : okay , checking | 17:47 |
| *** shardy has quit IRC | 17:47 | |
| sshnaidm | clarkb, I see.. we had a same problem with google yesteday | 17:48 |
| *** JudeC has joined #openstack-infra | 17:48 | |
| *** jascott1 has quit IRC | 17:51 | |
| *** JudeC has quit IRC | 17:53 | |
| *** yland has joined #openstack-infra | 17:53 | |
| *** yland is now known as Guest64086 | 17:53 | |
| *** felipemonteiro has quit IRC | 17:55 | |
| *** tosky has joined #openstack-infra | 17:57 | |
| *** ijw has quit IRC | 18:01 | |
| *** efoley has quit IRC | 18:01 | |
| tushar1 | clarkb : I have doubt, how can the slave label and job name will be same? | 18:01 |
| tushar1 | clarkb: because the slaves (nodes) are created during runtime which has unique name , while jobs are created in jenkins during the setup time | 18:01 |
| clarkb | tushar1: the label name in nodepool is the label that gets assigned to the node in jenkisn. So while the node name is unique the label is not | 18:01 |
| clarkb | tushar1: the label is a way to aggregate like slaves in jenkins | 18:02 |
| *** cshastri has joined #openstack-infra | 18:02 | |
| *** ijw has joined #openstack-infra | 18:02 | |
| *** jascott1 has joined #openstack-infra | 18:03 | |
| clarkb | tushar1: then in your jobs you will often restrcit them to a specific label | 18:03 |
| *** ijw has quit IRC | 18:03 | |
| *** ijw has joined #openstack-infra | 18:04 | |
| *** zzzeek has quit IRC | 18:04 | |
| tushar1 | clarkb : oh okay | 18:05 |
| tushar1 | clarkb : where we specify the label of slave? | 18:07 |
| clarkb | tushar1: in your nodepool config, the label there is the label of the slave | 18:07 |
| clarkb | tushar1: you should see it in jenkins under the slaves config too | 18:08 |
| *** sflanigan has quit IRC | 18:09 | |
| *** gildub has joined #openstack-infra | 18:12 | |
| *** JudeC has joined #openstack-infra | 18:15 | |
| *** amoralej is now known as amoralej|off | 18:16 | |
| tushar1 | clarkb : okay | 18:16 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Make logging helper method in zuul_stream https://review.openstack.org/472963 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Special case shell logging on localhost https://review.openstack.org/474216 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Change log streaming link to finger protocol https://review.openstack.org/437764 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Extract get_playhosts listing in zuul_stream to a method https://review.openstack.org/472964 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Handle lists of streamers https://review.openstack.org/474230 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Direct streaming at delegated_to target https://review.openstack.org/474215 | 18:17 |
| *** JudeC has quit IRC | 18:19 | |
| *** electrofelix has quit IRC | 18:24 | |
| fungi | clarkb: mordred: jeblair: i have only the vaguest recollection of earlier discussions about how to set the "mirrored from" link and repo type in github like you see at the top of https://github.com/GNOME/gtk | 18:24 |
| fungi | do any of you recall what we figured out about that>? | 18:24 |
| clarkb | fungi: it requires you to email github directly to have them do a pull mirroring | 18:25 |
| fungi | oh, and that has to be set per repository? | 18:25 |
| clarkb | its not part of their public api as far as I could tell and they don't really publish any of the workings around it | 18:25 |
| clarkb | ya | 18:25 |
| fungi | that's not what https://help.github.com/articles/about-github-mirrors/ seemed to imply | 18:25 |
| clarkb | granted this could've chagned since then but that was my understadnign, you get it if you convince github to pull and mirror for you somehow | 18:25 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-roles master: Create configure-mirrors role https://review.openstack.org/473845 | 18:26 |
| *** abramley has joined #openstack-infra | 18:26 | |
| clarkb | fungi: the post receive hook is basically what we do, if you want their fancy "this is mirrored from" it requies they do it on their end iirc | 18:26 |
| *** Apoorva_ has quit IRC | 18:26 | |
| mordred | yah. that's my understanding | 18:27 |
| mordred | and given our large number of ever changing repos, we never really explored that further | 18:27 |
| *** lbragstad has quit IRC | 18:28 | |
| *** xarses_ has quit IRC | 18:28 | |
| fungi | yup, i agree that their post-receive hook suggestion on that page is effectively equivalent to what we're already doing | 18:28 |
| *** xarses_ has joined #openstack-infra | 18:29 | |
| pabelanger | fungi: the certcheck would be awesome to run on our gearmans SSL certs also. Will have to see how that works | 18:29 |
| fungi | though maybe normalizing all our project descriptions in gh to "mirrored from https://git.openstack.org/..." would make more sense | 18:29 |
| clarkb | ya we can certainly inject it into the description ourselves | 18:29 |
| clarkb | it will be formatted slightly different but meh | 18:30 |
| fungi | pabelanger: it's pretty simple. as long as the listening socket can be reached from cacti.o.o (and doesn't involve some pre-ssl negotiation like smtp+starttls i think) you can just add the hostname and port number to that list | 18:30 |
| fungi | pabelanger: and by "that list" i mean what i was doing in https://review.openstack.org/474713 right after i apologized in channel here | 18:30 |
| jeblair | fungi: the organization itself also has a description field which we have left empty. | 18:30 |
| jeblair | fungi: we could add something like "these are mirrors of projects hosted at... for the real story, see the project navigator: https://www.openstack.org/software/project-navigator" | 18:31 |
| fungi | jeblair: oh! that's easily addressed. thanks for pointing it out | 18:31 |
| *** Sihan has joined #openstack-infra | 18:33 | |
| Sihan | Hi guys, how to reload the zuul/layout.yaml without restarting the zuul? | 18:34 |
| *** cshastri has quit IRC | 18:34 | |
| jeblair | Sihan: sighup will do it: kill -1 PID | 18:34 |
| *** JudeC has joined #openstack-infra | 18:36 | |
| Sihan | cool | 18:36 |
| *** lbragstad has joined #openstack-infra | 18:36 | |
| Sihan | thx | 18:36 |
| *** dhinesh has joined #openstack-infra | 18:40 | |
| *** nicolasbock has quit IRC | 18:41 | |
| clarkb | smcginnis: tracking down other jobs that fungi's log filesystem analysis show and wondering what gate-cinder-dsvm-apache-ubuntu-xenial-nv is | 18:42 |
| smcginnis | clarkb: Need to check, but I think that is a job that runs cinder-api using Apache as a front end. | 18:42 |
| clarkb | isn't that supposed to be the default in devstack now? maybe we can just run cinder like that by default and drop the explicit job? | 18:44 |
| clarkb | sdague: ^ | 18:44 |
| *** Hal has joined #openstack-infra | 18:44 | |
| smcginnis | clarkb: We are working towards that. Two patches yet to land before we can default to the uWSGI approach. | 18:45 |
| smcginnis | clarkb: But yeah, that just sets CINDER_USE_MOD_WSGI=True: http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/cinder.yaml#n92 | 18:45 |
| smcginnis | clarkb: So I'd be fine removing that job now, knowing thhings are just about to change. | 18:45 |
| smcginnis | clarkb: Want to do that? Or should I? | 18:45 |
| clarkb | smcginnis: we can wait for things to land, I was just sort of surprised to see a special apache job | 18:46 |
| clarkb | smcginnis: just ping me when you've got it running by default udner uwsgi and we can clean it up | 18:46 |
| smcginnis | clarkb: I think that was added a few releases ago now. | 18:46 |
| smcginnis | clarkb: Sounds good. | 18:46 |
| fungi | smcginnis: clarkb: how about a project-config change removing the special-case job so we don't forget, but give it a depends-on the other (cinder?) changes making that standard? | 18:46 |
| clarkb | fungi: ++ | 18:47 |
| fungi | that way we don't turn the job off until we can do so safely | 18:47 |
| clarkb | kevinbenton: ihrachys related to ^ I notice that every neutron patch seems to run 3 DVR jobs. Do we need a dvr single node, dvr multinode, and dvr ha multinode job? can we just maybe run the dvr ha multinode? | 18:47 |
| *** Sihan_ has joined #openstack-infra | 18:47 | |
| Sihan_ | After I run kill -1 <pid>, zuul server process is gone ): | 18:48 |
| smcginnis | fungi: Good call. | 18:48 |
| tushar1 | clarkb : Even after making the slave label and job name same ,getting same error. Is there any issue of version mismatch of python package/ | 18:48 |
| tushar1 | ? | 18:48 |
| clarkb | kevinbenton: ihrachys and if we do that, do we still need the non dvr multinode job? that one doesn't really test as much multinode specific stuff | 18:48 |
| *** haleyb_ has joined #openstack-infra | 18:48 | |
| clarkb | tushar1: there shouldn't be a python issue here since jenkins is java | 18:48 |
| ihrachys | clarkb, I asked haleyb_ a while ago to look at jobs that we could trim from the neutron gate. I am not sure at which stage the analysis is. | 18:48 |
| ihrachys | I agree that we have redundant jobs | 18:49 |
| ihrachys | clarkb, I would be probably ok with one single node that is legacy, and one multinode that is dvr+ha | 18:49 |
| clarkb | ihrachys: ok good to know it is being looked at. We are under log server filesystem size pressure, so would be good if we can priortize that a bit | 18:49 |
| clarkb | ihrachys: ya the single node that is legacy is our "default" job that runs everywhere I think it has been a good one and likely should keep it | 18:49 |
| tushar1 | clarkb : jenkins is java , but the error which am getting is coming from python package zuul,thats why I am asking | 18:50 |
| haleyb_ | ihrachys: i will shoot for our meeting next week. I need to know what combinations are tested versus failure rate | 18:50 |
| ihrachys | haleyb_, cool. it seems like infra would be glad for this to get more attention from our side. | 18:51 |
| haleyb_ | but a single dvr+ha multinode was the plan, that job is currently non-voting | 18:51 |
| clarkb | tushar1: yes because the java program (jenkins) isn't registering the job with gearman. You can check what is registered by connecting to the gearman server and running 'status' | 18:51 |
| clarkb | tushar1: telnet $gearman_server $gearman_port then enter status | 18:51 |
| ihrachys | speaking of neutron gate size... I have smth to follow up with qa on keystonev3 job that we have sitting in the check queue -nv | 18:52 |
| clarkb | ihrachys: ya lbragstad and I were talking about that one earlier today. I think we may finally be to the point wheer we can stop running that and just have devstack do v3 only be default but lbragstad was going to talk to stevemar about it too | 18:53 |
| clarkb | ihrachys: haleyb_ http://paste.openstack.org/show/612724/ is the list we are looking at | 18:53 |
| clarkb | ihrachys: haleyb_ thats a repesentative sample of disk usage by job | 18:53 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Remove zuulv3-dev log publisher https://review.openstack.org/474726 | 18:54 |
| pabelanger | jeblair: mordred: ^ is our fix | 18:54 |
| clarkb | mriedem: gate-tempest-dsvm-cells-ubuntu-xenial is the cellsv1 job because all other jobs are cellsv2 now right? when do we get to delete cellsv1 testing? | 18:54 |
| mriedem | clarkb: correct | 18:55 |
| mriedem | clarkb: when we drop cells v1 support | 18:55 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove zuulv3-dev log publisher https://review.openstack.org/474726 | 18:55 |
| clarkb | mriedem: oh I thought it was already dropped? | 18:55 |
| mriedem | nope | 18:56 |
| mriedem | cells v1 is also why we can't drop nova-net yet | 18:56 |
| clarkb | mriedem: but it only runs on newton right? | 18:56 |
| clarkb | it being nova net | 18:56 |
| haleyb_ | clarkb: is that for one day? | 18:57 |
| tushar1 | clarkb : I found nothing is registered here , https://gist.github.com/anonymous/a73f6ff5f02521a7cb96fdcc28200469 | 18:57 |
| mriedem | clarkb: no, | 18:58 |
| clarkb | haleyb_: no that is for ~30 days and only a subsample (its not a complete listing, we just wanted to get a comparison between jobs) | 18:58 |
| mriedem | nova-net only starts under cells v1 | 18:58 |
| mriedem | everything else by default runs neutron in master and ocata | 18:58 |
| clarkb | tushar1: check your jenkins logs to see if gearman think it is register things | 18:58 |
| clarkb | tushar1: but the problem is going to be on the jenkins to gearman side | 18:58 |
| pabelanger | clarkb: fungi: mordred: jeblair: do you mind adding https://review.openstack.org/#/c/474255/ to your review pipeline. This is for our zuul CA we'd run on puppetmaster and generate SSL certs for gearman server / client | 18:59 |
| fungi | haleyb_: that's a random sample of the jobs run for any patchsets of ~21% of changes over the past 30-45 days | 18:59 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config master: Remove {pipeline}-cinder-dsvm-apache-{node}{suffix} https://review.openstack.org/474727 | 18:59 |
| fungi | haleyb_: i tallied up the kilobytes of disk used in aggregate by all runs of each job by name across the sample | 19:00 |
| fungi | haleyb_: thus the total kilobytes line at the bottom is key... divide the usage for a job into that to get an idea of what percentage of our log volume for testing changes is represented by any particular job | 19:01 |
| smcginnis | clarkb: See if that looks OK when you get a chance. ^^ | 19:01 |
| clarkb | smcginnis: also is gate-tempest-dsvm-lvm-multibackend-ubuntu-xenial-nv a strict superset of the default lvm backend? maybe we can just make that default depending on what it tests? | 19:01 |
| smcginnis | clarkb: Not sure of that one either. Let me take a look. | 19:02 |
| * smcginnis likes cleaning out old cruft | 19:02 | |
| fungi | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=717&rra_id=all says the mass gate-tripleo-ci-centos-7-ovb-containers-oooq-nv log deletion is starting to have an impact | 19:02 |
| fungi | we are actually deleting logs faster than they can accumulate | 19:03 |
| mriedem | smcginnis: clarkb: that's the job scottda created to have 2 lvm backends to do migration/retype testing | 19:03 |
| mriedem | so we can test things like swap volume in nova | 19:03 |
| mriedem | although we already test swap volume in nova in tempest w/o retype/volume migration... | 19:03 |
| smcginnis | mriedem: I wonder if we can just always run that way though. Even if we only ever use one of them. | 19:03 |
| haleyb_ | fungi: there's a few big neutron jobs at the bottom, we'll see what we can do... | 19:03 |
| clarkb | mriedem: so if we just made it default we'd get the same test coverage? | 19:03 |
| *** armax has joined #openstack-infra | 19:03 | |
| mriedem | i don't know if the test that he actually needed that job for ever landed in tempest... | 19:04 |
| *** zengchen has quit IRC | 19:04 | |
| mriedem | best to probably ask scottda | 19:04 |
| smcginnis | mriedem: Are you a betting man? | 19:04 |
| *** zengchen has joined #openstack-infra | 19:04 | |
| mriedem | nickel slots buddy | 19:04 |
| fungi | haleyb_: much appreciated! the smaller the logs, the longer we can keep logs for everybody (this week we had to drop retention _again_ so it's down to 30 days now, which is what prompted us to run this analysis overnight) | 19:04 |
| clarkb | we can probably also do a second pass at what devstack-gate is collecting to see if we can trim it more. The obvious thing to trim is the journald files... | 19:05 |
| *** ijw has quit IRC | 19:05 | |
| clarkb | I think my change to export them in export format results in smaller journald files in total too | 19:05 |
| clarkb | sdague: ^ I know you were resisting that but may help us collect more data overall | 19:05 |
| openstackgerrit | gordon chung proposed openstack-infra/project-config master: don't enable gnocchi starting in pike https://review.openstack.org/474731 | 19:06 |
| clarkb | sdague: http://logs.openstack.org/28/352228/46/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/486c254/logs/ is 68.6MB of journald. http://logs.openstack.org/05/468705/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/52a0f91/logs/ is 23MB | 19:07 |
| clarkb | so almost a third the size | 19:07 |
| clarkb | sdague: if you are willing to reconsider that change I can rebase to resolve the merge conflict. I think it would be beneficial for log fs reasons and you still get to grab them locally | 19:07 |
| *** jaosorior is now known as jaosorior_away | 19:08 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/project-config master: Removed a bunch of identity-v3-only jobs https://review.openstack.org/474733 | 19:12 |
| ihrachys | clarkb, I asked mtreinish in -qa channel about that, but for now I also posted this ^ | 19:12 |
| ihrachys | if you think we should wait for more opinions, feel free to WIP | 19:12 |
| clarkb | lbragstad: ^ you may want to weigh in with what you find | 19:12 |
| clarkb | ihrachys: I think we should wait on lbragstad at least since he said he would look into it more | 19:13 |
| ihrachys | ok lemme add him to reviewers and WIP | 19:13 |
| lbragstad | clarkb: ok - will do | 19:13 |
| lbragstad | ihrachys: thank you | 19:13 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: mirror-server manifest: remove mitaka promotion cron https://review.openstack.org/474735 | 19:15 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/project-config master: migrate periodic jobs on stable branches to oooq https://review.openstack.org/474736 | 19:16 |
| ihrachys | armax, kevinbenton: 2nd opinion please for https://review.openstack.org/#/c/474305/ we need it to unblock the gate for newton | 19:17 |
| ihrachys | oops wrong channel | 19:17 |
| *** abramley has quit IRC | 19:17 | |
| *** mriedem has quit IRC | 19:17 | |
| smcginnis | Hmmm... Could not resolve host: zm06.openstack.org' | 19:21 |
| EmilienM | clarkb: I'm still trying to get some help on the tripleo-quickstart issue with container jobs | 19:22 |
| *** trown|lunch is now known as trown | 19:22 | |
| fungi | smcginnis: we've been seeing what appear to be recursion issues when using google public dns (which is what we configure our test instances to use as a workaround to generally broken dns in some of our providers) | 19:23 |
| *** ericyoung has quit IRC | 19:23 | |
| fungi | seems to have increased significantly this week | 19:23 |
| smcginnis | Huh, that's a little scary. | 19:24 |
| clarkb | e-r is tracking but only a subset due to the query | 19:26 |
| *** Sukhdev has joined #openstack-infra | 19:26 | |
| clarkb | and we have modified pur dns records to have longer ttls amd are switvhing to opendns | 19:26 |
| sdague | clarkb: yeh, if there isn't space just do what you need to | 19:27 |
| fungi | clarkb: did you check the ttls for the zmxx.o.o records, or should i go do that now? | 19:30 |
| *** mriedem has joined #openstack-infra | 19:33 | |
| clarkb | fungi: I didn't | 19:34 |
| fungi | i'm already digging into them, but so far they're all 3600 | 19:35 |
| fungi | i checked all 16 a/aaaa records and they're all already 3600 (including those for zm06 mentioned in the error above) | 19:37 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate master: Store journal in export format not native https://review.openstack.org/468705 | 19:41 |
| clarkb | sdague: ^ I'll double check that new runs after reabse produce the same sort of log size savings | 19:41 |
| *** mtanino_ has joined #openstack-infra | 19:44 | |
| *** jcoufal_ has joined #openstack-infra | 19:44 | |
| *** Guest64086 has quit IRC | 19:44 | |
| *** mtanino_ is now known as mtanino | 19:46 | |
| *** jcoufal has quit IRC | 19:47 | |
| *** tushar1 has quit IRC | 19:48 | |
| *** dprince has quit IRC | 19:50 | |
| *** abramley has joined #openstack-infra | 19:56 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: convert glance to the new unified publishing jobs https://review.openstack.org/474761 | 19:57 |
| *** mhickey has joined #openstack-infra | 20:00 | |
| *** Guest11 has joined #openstack-infra | 20:00 | |
| *** Hal has quit IRC | 20:04 | |
| *** xyang1 has joined #openstack-infra | 20:07 | |
| *** sshnaidm is now known as sshnaidm|off | 20:07 | |
| *** eharney has quit IRC | 20:10 | |
| *** jcoufal has joined #openstack-infra | 20:14 | |
| *** jcoufal_ has quit IRC | 20:17 | |
| dhellmann | are there (or were there earlier today) issues with the pypi mirror? I'm seeing some jobs fail because they can't get the new oslo.log release 3.28.1, which does appear on pypi | 20:17 |
| dhellmann | http://logs.openstack.org/22/474622/1/check/gate-cross-cinder-python27-ubuntu-xenial/6fae1ba/console.html#_2017-06-15_16_03_18_647685 | 20:18 |
| fungi | lemme look into the timing there | 20:18 |
| *** florianf has quit IRC | 20:18 | |
| *** jkilpatr has quit IRC | 20:18 | |
| dhellmann | if you've resolved an issue earlier today, I can just recheck and let the bots sort it out | 20:18 |
| fungi | pypi upload happened at 16:00:47 according to http://logs.openstack.org/91/91f918a74ecde7f9c3c5d30463017118f2348eb6/release/oslo.log-pypi-both-upload/2be8486/console.html#_2017-06-15_16_00_47_553162 | 20:20 |
| *** yland has joined #openstack-infra | 20:20 | |
| dhellmann | so maybe we hit a race? those are pretty close together | 20:20 |
| clarkb | vos release happened at 16:06 | 20:21 |
| clarkb | so ya you raced | 20:21 |
| fungi | oslo.log index page was updated by bandersnatch within 5 minutes and the vos release completed at 16:06:47 | 20:21 |
| clarkb | vroom vroom | 20:21 |
| fungi | so total time between upload to pypi and appearance on mirrors was 6 minutes there | 20:21 |
| dhellmann | oh, I see dirk already rechecked that | 20:21 |
| jeblair | that's an autogenerated commit -- we should be able to do something about that, yeah? | 20:22 |
| *** jcoufal_ has joined #openstack-infra | 20:22 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: gzip the ARA report after generating it https://review.openstack.org/468156 | 20:22 |
| dhellmann | yeah, it does already wait until we push the package to pypi | 20:22 |
| dhellmann | this may be the first time since we added that delay that we've hit a race with the mirror | 20:22 |
| fungi | in the past we've pontificated on having the pypi upload job spin and check a mirror for up to, say, 10 minutes before completing | 20:23 |
| jeblair | dhellmann: maybe we can wait up to 10 more minutes .... | 20:23 |
| jeblair | fungi: right that. :) | 20:23 |
| dhellmann | well, it's not "waiting" so much has the constraints update job isn't triggered until the upload job is done | 20:23 |
| fungi | basically don't consider the upload job complete unless the package appears on a mirror | 20:23 |
| dhellmann | so we could add a sleep somewhere, I guess | 20:23 |
| *** neiljerram has joined #openstack-infra | 20:23 | |
| dhellmann | or that | 20:23 |
| *** jcoufal has quit IRC | 20:23 | |
| dhellmann | that's way less janky | 20:23 |
| fungi | however, we may want to consider how that plays into clarkb's new mechanism to only vos release if packages of significance are uploaded | 20:24 |
| fungi | so might get hairy if the package in question is not one tracked in upper-constraints on any requirements branches | 20:24 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 20:24 |
| fungi | as that might take 4 hours to appear on the mirrors | 20:25 |
| clarkb | it would haev to be in constraints for anything in the automated release processes to be affected right? | 20:25 |
| fungi | (well, upper-constraints.txt plus blacklist.txt) | 20:25 |
| clarkb | oh wiat | 20:25 |
| fungi | sure, more thinking if we added that wait in our generic pypi upload job, it's used for plenty of unofficial projects which aren't deps of ours too | 20:26 |
| clarkb | you mean you can't delay for it to show up ebcause eg nova isn't in constraints | 20:26 |
| clarkb | btu it also isn't in pypi | 20:26 |
| clarkb | ya | 20:26 |
| jeblair | if project in constraints: wait; else: return succes ? | 20:27 |
| fungi | so maybe the job includes an advisory delay of up to 10 minutes but doesn't consider a package missing from the mirror longer than that to be a job failure, just goes ahead and reports success | 20:27 |
| fungi | or jeblair's suggestion, which is faster (at the expense of consulting files in branches of an additional repo) | 20:27 |
| *** thorst has quit IRC | 20:27 | |
| jeblair | if the proposal job is the only one that hangs off of the upload job like this and has this requirement, we could put the delay at the start of that. | 20:31 |
| jeblair | (should already have the repo checked out, may be simpler) | 20:35 |
| fungi | true | 20:39 |
| fungi | sounded like dhellmann was suggesting it already has a (less intelligent) delay that's not quite long enough, but if so i don't see it here: http://logs.openstack.org/91/91f918a74ecde7f9c3c5d30463017118f2348eb6/release/propose-oslo.log-update-constraints/cf81716/console.html | 20:42 |
| fungi | oh, rereading what he said, i think he was simply referring to the job dependency ordering | 20:42 |
| dhellmann | yeah, just the dependency ordering | 20:43 |
| fungi | got it. so we could simply stick a `sleep 600` at the start of update_constraints.sh or add a polling loop looking at a specific mirror waiting for the index to include the package version involved | 20:45 |
| *** JudeC has quit IRC | 20:47 | |
| scottda | mriedem: smcginnis tempest/scenario/test_volume_migrate_attached.py merged: https://review.openstack.org/#/c/326681/ | 20:47 |
| scottda | This ^^^ will run in the experimental cinder multi-backend job | 20:48 |
| *** gildub has quit IRC | 20:48 | |
| mriedem | scottda: and that's a single node job right? | 20:48 |
| scottda | mriedem: no, it requires 2 lvm backends | 20:48 |
| mriedem | right, but you can have multiple lvm backends on a single host right? | 20:49 |
| clarkb | scottda: fwiw its not an experimental job it is in check for cinder | 20:49 |
| scottda | clarkb: OK | 20:49 |
| mriedem | it's in experimental for nova i think | 20:49 |
| *** JudeC has joined #openstack-infra | 20:49 | |
| clarkb | yup and for devstack and tempest it is experimental | 20:49 |
| scottda | mriedem: If you run it on a single host, i *think* there is an optimisation that short curcuits the nova swap volume. | 20:49 |
| mriedem | i thought the jobs had to be defined as multinode.. | 20:50 |
| * mriedem looks | 20:50 | |
| *** baoli has quit IRC | 20:50 | |
| clarkb | it is single node | 20:51 |
| clarkb | there is a separate scenario job that is multinode | 20:51 |
| *** JudeC has quit IRC | 20:53 | |
| smcginnis | Yeah, two things, multinode and multibackend. | 20:54 |
| mriedem | which backend node is on first now again? | 20:55 |
| smcginnis | :) | 20:55 |
| mriedem | ah ok tempest-dsvm-lvm-multibackend vs tempest-dsvm-neutron-scenario-multinode-lvm-multibackend | 20:56 |
| mriedem | o dodm | 20:56 |
| mriedem | i didn't realize there were 2 | 20:56 |
| *** rhallisey has quit IRC | 20:56 | |
| clarkb | the first is the one consuming all the disk relative to the second | 20:56 |
| clarkb | do they do distinct things? should they be collapsed? | 20:57 |
| mriedem | https://review.openstack.org/#/c/326681/ is a scenario test, so that test will only run in tempest-dsvm-neutron-scenario-multinode-lvm-multibackend | 20:57 |
| mriedem | i'm pretty sure with the NWO | 20:57 |
| *** hamzy has quit IRC | 20:58 | |
| *** trown is now known as trown|lunch | 20:58 | |
| *** thorst has joined #openstack-infra | 20:59 | |
| *** jkilpatr has joined #openstack-infra | 20:59 | |
| mriedem | heh they both run in nova experimental | 21:00 |
| *** andreww has joined #openstack-infra | 21:00 | |
| mriedem | we should fix that and remove the single node one from nova experimental | 21:00 |
| mriedem | i can do that | 21:00 |
| *** andreww has quit IRC | 21:00 | |
| clarkb | mriedem: both jobs run on multiple projcts so you'll want to celan those up too | 21:00 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Test with bonded public interface https://review.openstack.org/385562 | 21:01 |
| clarkb | I think both are a mix of check and experimental depending which project they run against too | 21:01 |
| mriedem | let me find a run of the single node job | 21:01 |
| *** andreww has joined #openstack-infra | 21:01 | |
| mriedem | hmm, tempest-dsvm-neutron-scenario-multinode-lvm-multibackend is listed as runing in cinder check | 21:02 |
| mriedem | but i dont' see it here https://review.openstack.org/#/c/472786/ | 21:02 |
| mriedem | oh here it is http://logs.openstack.org/86/472786/6/check/gate-tempest-dsvm-lvm-multibackend-ubuntu-xenial-nv/9796850/ | 21:02 |
| mriedem | yeah so ^ doesn't run TestVolumeMigrateRetypeAttached | 21:02 |
| mriedem | because it doesn't run scenario tests | 21:02 |
| mriedem | scottda: smcginnis: so is there value in running the single-node lvm multibackend job if it's not testing retype/volume migration? | 21:03 |
| mriedem | we have a compute api test for swap volume already | 21:03 |
| mriedem | http://logs.openstack.org/86/472786/6/check/gate-tempest-dsvm-lvm-multibackend-ubuntu-xenial-nv/9796850/console.html#_2017-06-14_22_18_38_439653 | 21:03 |
| *** xarses_ has quit IRC | 21:03 | |
| mriedem | i'd say we ditch the single-node job | 21:03 |
| mriedem | and promote the multinode lvm multibackend job | 21:03 |
| mriedem | and then eat oatmeal raisin cookies | 21:03 |
| scottda | If multinode is running, we shouldn't need single-node | 21:04 |
| scottda | Problem with afternoon cookies, it makes me want afternoon coffee.... | 21:04 |
| mriedem | i already did the latter | 21:04 |
| mriedem | ok i'll push up a change and we can take a look | 21:04 |
| clarkb | mriedem: thanks | 21:04 |
| *** kgiusti has left #openstack-infra | 21:05 | |
| *** jamesmcarthur has joined #openstack-infra | 21:05 | |
| openstackgerrit | James E. Blair proposed openstack-infra/openstack-zuul-jobs master: DNM: testing https://review.openstack.org/474780 | 21:06 |
| smcginnis | mriedem, scottda: I think it makes sense to always do multibackend. | 21:07 |
| smcginnis | mriedem, scottda: But do we need multinode all the time? | 21:07 |
| mriedem | smcginnis: you don't get the retype test w/o that | 21:07 |
| smcginnis | mriedem: Didn't you say it wasn't running that? | 21:07 |
| mriedem | tempest-dsvm-neutron-scenario-multinode-lvm-multibackend is the only one of the two that runs the retype test | 21:08 |
| mriedem | retype/volume migration | 21:08 |
| mriedem | b/c it's a tempest scenario test | 21:08 |
| mriedem | the base tempest/dvsm jobs stopped running scenario tests by default shortly after the PTG | 21:08 |
| smcginnis | mriedem: In that case, sure, carry on. | 21:08 |
| *** Sihan_ has quit IRC | 21:08 | |
| *** Sihan has quit IRC | 21:08 | |
| mriedem | so now we have special jobs to run just the scenario tests | 21:08 |
| clarkb | mriedem: I think it added them back in | 21:08 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: convert glanceclient to the new unified doc build https://review.openstack.org/474782 | 21:09 |
| clarkb | mriedem: it just runs them serially at the end not all in parallel | 21:09 |
| * clarkb checks a job log | 21:09 | |
| clarkb | mriedem: http://logs.openstack.org/70/474370/2/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/0f80b62/console.html#_2017-06-14_23_40_49_730371 ya | 21:09 |
| mriedem | http://logs.openstack.org/86/472786/6/check/gate-tempest-dsvm-lvm-multibackend-ubuntu-xenial-nv/9796850/console.html | 21:10 |
| mriedem | then why doesn't it show up in ^ | 21:10 |
| mriedem | TestVolumeMigrateRetypeAttached i mean | 21:10 |
| *** jamesmcarthur has quit IRC | 21:10 | |
| clarkb | looks like that job isn't running the scenario tests, I don't know why | 21:11 |
| clarkb | the "base" tempest job does though | 21:11 |
| mriedem | export DEVSTACK_GATE_TEMPEST_REGEX='(?!.*\[.*\bslow\b.*\])(^tempest\.(api|scenario)|(^cinder\.tests.tempest))' | 21:11 |
| clarkb | they runs a different env than the base job so maybe magic in the tempest tox.ini ? | 21:11 |
| *** Guest11 has quit IRC | 21:13 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Add some release notes we forgot to add https://review.openstack.org/474385 | 21:14 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Go back to deleting volumes serially https://review.openstack.org/474783 | 21:14 |
| *** Apoorva has joined #openstack-infra | 21:14 | |
| mriedem | it does show up in the multinode job http://logs.openstack.org/45/470645/5/experimental/gate-tempest-dsvm-neutron-scenario-multinode-lvm-multibackend-ubuntu-xenial-nv/0b421c5/console.html#_2017-06-06_02_21_43_757566 | 21:19 |
| mriedem | and fails of course | 21:19 |
| mriedem | ah i see the difference | 21:20 |
| mriedem | the multinode job allows slow tests | 21:20 |
| mriedem | the singlenode one doesn't | 21:20 |
| mriedem | and the test is marked as a slow test | 21:20 |
| mriedem | 241.428216s | 21:20 |
| mriedem | 4 minutes is pretty slow | 21:21 |
| mriedem | smcginnis: so looking at what else relies on multibackend in tempest, there are some other tests, and one says it does retype/migration http://logs.openstack.org/86/472786/6/check/gate-tempest-dsvm-lvm-multibackend-ubuntu-xenial-nv/9796850/console.html#_2017-06-14_22_50_27_723983 | 21:22 |
| mriedem | it's a volume api test, not scenario | 21:22 |
| *** jamesmcarthur has joined #openstack-infra | 21:22 | |
| mriedem | i think the only difference there is it's doing it w/o the volume being attached to an instance | 21:22 |
| *** oanson has quit IRC | 21:23 | |
| mriedem | the slow scenario test is a retype migration with the volume attached to an instance | 21:23 |
| mriedem | boot from volume no less | 21:23 |
| *** gouthamr has quit IRC | 21:23 | |
| *** dimtruck is now known as zz_dimtruck | 21:25 | |
| clarkb | mriedem: what is a retype migration? | 21:26 |
| clarkb | it moves the backing store from one backend to another? | 21:26 |
| *** oanson has joined #openstack-infra | 21:26 | |
| mriedem | moves the volumes from one volume type / backend to another | 21:26 |
| mriedem | beyond that you'd have to ask the cinder guys | 21:26 |
| *** jamesmcarthur has quit IRC | 21:27 | |
| fungi | infra-root: i took a stab at jeblair's suggestion of updating the github org profiles to clarify their provenance and state, particularly https://github.com/openstack and https://github.com/openstack-dev and https://github.com/openstack-infra (also updated logos on a couple of those while i was at it, and added descriptions to openstack-attic, openstack-ci, stackforge and stackforge-attic) | 21:27 |
| fungi | lemme know what you think | 21:27 |
| fungi | wording was somewhat limited by their max string length in the description field | 21:27 |
| clarkb | https://github.com/openstack lgtm | 21:28 |
| jeblair | fungi: i like it. what is the max? i think it would be a nice improvement if we could get words in about the project navigator. | 21:29 |
| *** Goneri has quit IRC | 21:29 | |
| fungi | jeblair: about 5-10 characters longer than what i have there, i think. propose alternate wording and i'll see what it'll take | 21:29 |
| clarkb | infra-root http://logs.openstack.org/05/468705/3/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/3a4f706/logs/ does indeed save about 30MB per job there | 21:29 |
| clarkb | and it affects all the d-g jobs so we see all of them get smaller. I will have a paste with comparative data shortly | 21:29 |
| fungi | jeblair: i _did_ put the project navigator as the url field, as a compromise | 21:30 |
| jeblair | fungi: yeah, i saw that, that's good. maybe we can add a blink tag and an unde rconstruction gif. | 21:31 |
| clarkb | http://paste.openstack.org/show/612751/ compares two recent sets of d-g job runs | 21:32 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/project-config master: Cleanup lvm-multibackend job usage for nova/cinder https://review.openstack.org/474786 | 21:34 |
| mriedem | clarkb: smcginnis: not sure how much this helps, but it's something ^ | 21:34 |
| *** burgerk_ has joined #openstack-infra | 21:35 | |
| *** burgerk has quit IRC | 21:38 | |
| *** burgerk_ has quit IRC | 21:40 | |
| *** aeng has joined #openstack-infra | 21:41 | |
| openstackgerrit | James E. Blair proposed openstack-infra/openstack-zuul-jobs master: DNM: testing https://review.openstack.org/474780 | 21:43 |
| *** gouthamr has joined #openstack-infra | 21:43 | |
| *** esberglu has quit IRC | 21:43 | |
| *** esberglu has joined #openstack-infra | 21:44 | |
| *** yolanda has quit IRC | 21:45 | |
| clarkb | fungi: not sure if saw earlier but https://review.openstack.org/#/c/468705/ reduces log space needed for d-g jobs on xenial | 21:46 |
| fungi | ooh, thanks for the heads-up, i missed that | 21:47 |
| *** esberglu has quit IRC | 21:48 | |
| *** mhickey has quit IRC | 21:50 | |
| *** jamesmcarthur has joined #openstack-infra | 21:51 | |
| *** jascott1 has quit IRC | 21:55 | |
| *** jamesmcarthur has quit IRC | 21:55 | |
| *** jascott1 has joined #openstack-infra | 21:55 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Make logging helper method in zuul_stream https://review.openstack.org/472963 | 21:56 |
| *** jascott1 has quit IRC | 21:56 | |
| *** jascott1 has joined #openstack-infra | 21:56 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Extract get_playhosts listing in zuul_stream to a method https://review.openstack.org/472964 | 21:57 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Change log streaming link to finger protocol https://review.openstack.org/437764 | 21:57 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Direct streaming at delegated_to target https://review.openstack.org/474215 | 21:58 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Special case shell logging on localhost https://review.openstack.org/474216 | 21:58 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Handle lists of streamers https://review.openstack.org/474230 | 21:58 |
| *** jcoufal has joined #openstack-infra | 22:00 | |
| *** mdrabe has quit IRC | 22:00 | |
| *** jcoufal_ has quit IRC | 22:02 | |
| *** gomarivera has quit IRC | 22:04 | |
| *** ijw has joined #openstack-infra | 22:04 | |
| *** bobh has quit IRC | 22:05 | |
| *** xarses_ has joined #openstack-infra | 22:05 | |
| *** gomarivera has joined #openstack-infra | 22:05 | |
| *** wolverineav has quit IRC | 22:05 | |
| *** wolverineav has joined #openstack-infra | 22:06 | |
| *** andreww has quit IRC | 22:07 | |
| *** wolverineav has quit IRC | 22:11 | |
| *** gomarivera has quit IRC | 22:11 | |
| *** askb has joined #openstack-infra | 22:12 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: convert horizon to the new unified doc build jobs https://review.openstack.org/474795 | 22:14 |
| *** thorst has quit IRC | 22:14 | |
| openstackgerrit | James E. Blair proposed openstack-infra/openstack-zuul-jobs master: DNM: testing https://review.openstack.org/474780 | 22:20 |
| *** vhosakot has quit IRC | 22:23 | |
| *** jkilpatr has quit IRC | 22:24 | |
| *** jcoufal has quit IRC | 22:24 | |
| *** zz_dimtruck is now known as dimtruck | 22:31 | |
| *** thorst has joined #openstack-infra | 22:33 | |
| *** thorst has quit IRC | 22:34 | |
| *** rlandy has quit IRC | 22:34 | |
| *** tosky has quit IRC | 22:36 | |
| *** oanson has quit IRC | 22:38 | |
| *** esberglu has joined #openstack-infra | 22:38 | |
| *** salv-orl_ has quit IRC | 22:39 | |
| *** oanson has joined #openstack-infra | 22:39 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/474798 | 22:40 |
| *** Swami has quit IRC | 22:40 | |
| *** andreww has joined #openstack-infra | 22:41 | |
| *** andreww has quit IRC | 22:41 | |
| *** esberglu has quit IRC | 22:43 | |
| *** xarses_ has quit IRC | 22:43 | |
| clarkb | pabelanger: if you are back can you rereview https://review.openstack.org/#/c/468705/3 it saves log disk space too http://paste.openstack.org/show/612751/ | 22:46 |
| *** neiljerram has quit IRC | 22:47 | |
| pabelanger | +3, thought we had merged that already | 22:48 |
| *** aeng has quit IRC | 22:52 | |
| *** esberglu has joined #openstack-infra | 22:54 | |
| *** nhandler has quit IRC | 22:56 | |
| *** nhandler has joined #openstack-infra | 22:57 | |
| *** dimtruck is now known as zz_dimtruck | 22:59 | |
| *** ijw has quit IRC | 23:00 | |
| *** Apoorva_ has joined #openstack-infra | 23:00 | |
| *** Apoorva has quit IRC | 23:03 | |
| *** gomarivera has joined #openstack-infra | 23:05 | |
| *** aeng has joined #openstack-infra | 23:05 | |
| *** neiljerram has joined #openstack-infra | 23:06 | |
| *** gordc has quit IRC | 23:06 | |
| *** ihrachys has quit IRC | 23:07 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Implement pipeline reject filter for github https://review.openstack.org/474001 | 23:08 |
| *** ijw has joined #openstack-infra | 23:09 | |
| *** gomarivera has quit IRC | 23:09 | |
| *** zzzeek has joined #openstack-infra | 23:09 | |
| *** trown|lunch is now known as trown|outtypewww | 23:12 | |
| *** ijw has quit IRC | 23:13 | |
| *** rbrndt has quit IRC | 23:15 | |
| *** Goneri has joined #openstack-infra | 23:16 | |
| *** yland has quit IRC | 23:18 | |
| *** yland has joined #openstack-infra | 23:20 | |
| *** yland is now known as Guest38554 | 23:20 | |
| *** jascott1 has quit IRC | 23:22 | |
| *** aeng has quit IRC | 23:22 | |
| *** jascott1 has joined #openstack-infra | 23:22 | |
| *** Guest38554 has quit IRC | 23:23 | |
| *** Hal has joined #openstack-infra | 23:24 | |
| *** jascott1 has quit IRC | 23:24 | |
| *** jascott1 has joined #openstack-infra | 23:24 | |
| *** jascott1 has quit IRC | 23:27 | |
| *** jascott1 has joined #openstack-infra | 23:27 | |
| *** sdague has quit IRC | 23:28 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Revert sync from latest command from ansible https://review.openstack.org/474808 | 23:30 |
| *** jascott1 has quit IRC | 23:31 | |
| *** gomarivera has joined #openstack-infra | 23:32 | |
| *** hyakuhei has joined #openstack-infra | 23:36 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Use python2 for ansible on remote hosts https://review.openstack.org/474810 | 23:39 |
| *** jkilpatr has joined #openstack-infra | 23:43 | |
| *** aeng has joined #openstack-infra | 23:46 | |
| *** jascott1 has joined #openstack-infra | 23:47 | |
| *** Swimingly has quit IRC | 23:47 | |
| *** Swimingly has joined #openstack-infra | 23:48 | |
| *** Swimingly has quit IRC | 23:48 | |
| *** Swimingly has joined #openstack-infra | 23:48 | |
| *** gildub has joined #openstack-infra | 23:49 | |
| *** stakeda has joined #openstack-infra | 23:50 | |
| *** xyang1 has quit IRC | 23:51 | |
| *** jamesmcarthur has joined #openstack-infra | 23:51 | |
| *** thorst has joined #openstack-infra | 23:53 | |
| *** dingyichen has joined #openstack-infra | 23:55 | |
| *** jamesmcarthur has quit IRC | 23:56 | |
| *** gomarivera has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!