| *** baoli has quit IRC | 00:00 | |
| *** jistr has quit IRC | 00:01 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move ppc block-device default to right $ARCH https://review.openstack.org/471170 | 00:01 |
|---|---|---|
| *** jistr has joined #openstack-infra | 00:02 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Mutex repo updates and merge ops in executor https://review.openstack.org/470571 | 00:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix reporting on changes dequeued for deps https://review.openstack.org/471143 | 00:06 |
| *** mriedem has quit IRC | 00:07 | |
| *** marst has quit IRC | 00:10 | |
| *** ramishra has joined #openstack-infra | 00:13 | |
| *** masber has joined #openstack-infra | 00:13 | |
| *** jamesmcarthur has quit IRC | 00:14 | |
| *** jamesmcarthur has joined #openstack-infra | 00:14 | |
| *** kzaitsev1pi has quit IRC | 00:14 | |
| *** Apoorva has quit IRC | 00:15 | |
| *** Apoorva has joined #openstack-infra | 00:20 | |
| *** ramishra has quit IRC | 00:21 | |
| *** wolverineav has quit IRC | 00:21 | |
| *** ramishra has joined #openstack-infra | 00:21 | |
| *** stakeda has joined #openstack-infra | 00:25 | |
| rm_work | does anyone here know if the hacking check for "no backslash-line-continuations" is no longer a thing? was it removed? | 00:25 |
| rm_work | I can't find it / get it to fail | 00:25 |
| *** gardlt has joined #openstack-infra | 00:26 | |
| *** rhallisey has quit IRC | 00:28 | |
| *** tphummel has quit IRC | 00:30 | |
| *** gardlt has quit IRC | 00:30 | |
| *** Apoorva has quit IRC | 00:34 | |
| *** tuanluong has joined #openstack-infra | 00:35 | |
| *** jamesmcarthur has quit IRC | 00:38 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Status branch protection checking for github https://review.openstack.org/471175 | 00:39 |
| *** thorst has joined #openstack-infra | 00:40 | |
| *** jamesmcarthur has joined #openstack-infra | 00:42 | |
| *** thorst has quit IRC | 00:44 | |
| *** gardlt has joined #openstack-infra | 00:48 | |
| *** sbezverk has quit IRC | 00:49 | |
| *** mriedem has joined #openstack-infra | 00:52 | |
| *** tinwood has quit IRC | 01:05 | |
| *** tinwood has joined #openstack-infra | 01:06 | |
| *** thorst has joined #openstack-infra | 01:07 | |
| *** LindaWang has joined #openstack-infra | 01:07 | |
| *** thorst has quit IRC | 01:07 | |
| *** jamesmcarthur has quit IRC | 01:11 | |
| *** Apoorva has joined #openstack-infra | 01:13 | |
| *** yamahata has quit IRC | 01:14 | |
| *** tphummel has joined #openstack-infra | 01:14 | |
| *** gongysh has joined #openstack-infra | 01:16 | |
| *** Apoorva has quit IRC | 01:17 | |
| *** jamielennox is now known as jamielennox|away | 01:17 | |
| *** Apoorva has joined #openstack-infra | 01:20 | |
| *** rkukura has quit IRC | 01:24 | |
| *** Apoorva has quit IRC | 01:25 | |
| *** mattmceuen has joined #openstack-infra | 01:27 | |
| *** liujiong has joined #openstack-infra | 01:29 | |
| *** bobh has joined #openstack-infra | 01:32 | |
| *** namnh has joined #openstack-infra | 01:33 | |
| *** jamielennox|away is now known as jamielennox | 01:34 | |
| *** gomarivera has joined #openstack-infra | 01:38 | |
| *** thorst has joined #openstack-infra | 01:41 | |
| *** armax has quit IRC | 01:44 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Write inventory as yaml not ini https://review.openstack.org/467634 | 01:44 |
| smcginnis | rm_work: I believe it was removed. For some reason (and I really shouldn't waste brain cells on this) I think it was something like H90x something. | 01:45 |
| *** dhinesh has quit IRC | 01:51 | |
| *** Guest42158 has quit IRC | 02:00 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Add github reporter status-url config option https://review.openstack.org/449794 | 02:01 |
| *** rkukura has joined #openstack-infra | 02:03 | |
| *** huanxie has joined #openstack-infra | 02:05 | |
| *** xinliang has quit IRC | 02:06 | |
| *** thorst has quit IRC | 02:06 | |
| *** knangia_ has quit IRC | 02:13 | |
| *** mdrabe has quit IRC | 02:15 | |
| *** xinliang has joined #openstack-infra | 02:19 | |
| *** xinliang has quit IRC | 02:19 | |
| *** xinliang has joined #openstack-infra | 02:19 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use picked nodes for later cmd_* calls https://review.openstack.org/469330 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Don't make image & loopdev functions static https://review.openstack.org/470845 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move global mount tracking into state https://review.openstack.org/469752 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update tracing in block_device_create_config_file https://review.openstack.org/471163 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move rollback into NodeBase object https://review.openstack.org/464105 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use global state to check for duplicate fs labels https://review.openstack.org/469751 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Remove 'state' argument from later cmd_* calls https://review.openstack.org/469750 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Pad state dump https://review.openstack.org/471162 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add state to NodeBase class https://review.openstack.org/469749 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move ppc block-device default to right $ARCH https://review.openstack.org/471170 | 02:20 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 02:20 |
| *** bobh has quit IRC | 02:21 | |
| *** bobh has joined #openstack-infra | 02:23 | |
| *** hongbin has joined #openstack-infra | 02:26 | |
| *** thorst has joined #openstack-infra | 02:26 | |
| *** thorst has quit IRC | 02:26 | |
| *** links has joined #openstack-infra | 02:27 | |
| *** bobh has quit IRC | 02:29 | |
| *** links has quit IRC | 02:30 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 02:30 |
| *** Apoorva has joined #openstack-infra | 02:30 | |
| *** links has joined #openstack-infra | 02:31 | |
| *** Apoorva has quit IRC | 02:39 | |
| *** thorst has joined #openstack-infra | 02:42 | |
| *** thorst has quit IRC | 02:42 | |
| *** thorst has joined #openstack-infra | 02:43 | |
| *** thorst has quit IRC | 02:47 | |
| *** ccamacho has quit IRC | 02:48 | |
| *** bramwelt has quit IRC | 02:49 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use picked nodes for later cmd_* calls https://review.openstack.org/469330 | 02:51 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 02:51 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Don't make image & loopdev functions static https://review.openstack.org/470845 | 02:51 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move global mount tracking into state https://review.openstack.org/469752 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update tracing in block_device_create_config_file https://review.openstack.org/471163 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move rollback into NodeBase object https://review.openstack.org/464105 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use global state to check for duplicate fs labels https://review.openstack.org/469751 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Remove 'state' argument from later cmd_* calls https://review.openstack.org/469750 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Pad state dump https://review.openstack.org/471162 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add state to NodeBase class https://review.openstack.org/469749 | 02:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Move ppc block-device default to right $ARCH https://review.openstack.org/471170 | 02:52 |
| *** mat128 has quit IRC | 02:56 | |
| *** rkukura has quit IRC | 02:57 | |
| *** ericyoung has quit IRC | 02:57 | |
| *** xarses_ has quit IRC | 02:57 | |
| *** dimtruck is now known as zz_dimtruck | 03:01 | |
| *** winggundamth has joined #openstack-infra | 03:05 | |
| *** gardlt has quit IRC | 03:07 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Generalize version discovery for re-use https://review.openstack.org/470770 | 03:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Migrate dns to new discovery method https://review.openstack.org/470993 | 03:08 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: First Pass: De-client-ify Users https://review.openstack.org/467784 | 03:08 |
| *** gardlt has joined #openstack-infra | 03:08 | |
| *** gardlt has quit IRC | 03:08 | |
| *** jamesmcarthur has joined #openstack-infra | 03:11 | |
| *** thorst has joined #openstack-infra | 03:14 | |
| *** mriedem has quit IRC | 03:14 | |
| *** jamesmcarthur has quit IRC | 03:15 | |
| *** gomarivera has quit IRC | 03:21 | |
| *** gomarivera has joined #openstack-infra | 03:22 | |
| *** gomarivera has quit IRC | 03:23 | |
| *** gomarivera has joined #openstack-infra | 03:23 | |
| *** yamamoto_ has joined #openstack-infra | 03:25 | |
| *** zhurong has joined #openstack-infra | 03:28 | |
| *** thorst has quit IRC | 03:32 | |
| *** gomarivera has quit IRC | 03:34 | |
| *** gomarivera has joined #openstack-infra | 03:34 | |
| *** gomarivera has quit IRC | 03:38 | |
| *** gomarivera has joined #openstack-infra | 03:44 | |
| openstackgerrit | Merged openstack-infra/shade master: Add pagination for the list_volumes call https://review.openstack.org/466927 | 03:45 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Add github reporter status-url config option https://review.openstack.org/449794 | 03:50 |
| *** vikrant has joined #openstack-infra | 03:51 | |
| *** vikrant is now known as vikrant|brb | 03:53 | |
| *** ihrachys has joined #openstack-infra | 03:53 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Add github reporter status-url config option https://review.openstack.org/449794 | 03:54 |
| openstackgerrit | Merged openstack-infra/shade master: Pass hints to Cinder scheduler in create_volume https://review.openstack.org/470968 | 03:55 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Expose ref to the executor and url formatter https://review.openstack.org/463457 | 03:56 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Add tenant to url formatting. https://review.openstack.org/463628 | 03:56 |
| *** ykarel has joined #openstack-infra | 04:04 | |
| *** ihrachys has quit IRC | 04:04 | |
| *** dhinesh has joined #openstack-infra | 04:05 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix race in gerrit+github test https://review.openstack.org/471161 | 04:06 |
| *** psachin has joined #openstack-infra | 04:06 | |
| *** bramwelt has joined #openstack-infra | 04:11 | |
| *** bramwelt is now known as Guest28014 | 04:11 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Write inventory as yaml not ini https://review.openstack.org/467634 | 04:13 |
| *** hongbin has quit IRC | 04:16 | |
| *** ihrachys has joined #openstack-infra | 04:23 | |
| *** salv-orlando has joined #openstack-infra | 04:26 | |
| *** zhurong has quit IRC | 04:27 | |
| *** gouthamr has quit IRC | 04:30 | |
| *** salv-orlando has quit IRC | 04:31 | |
| *** esikachev has joined #openstack-infra | 04:37 | |
| *** apetrich has quit IRC | 04:43 | |
| *** armax has joined #openstack-infra | 04:44 | |
| *** gongysh has quit IRC | 04:50 | |
| *** ihrachys has quit IRC | 04:50 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Expose ref to the executor and url formatter https://review.openstack.org/463457 | 04:53 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Add tenant to url formatting. https://review.openstack.org/463628 | 04:53 |
| *** apetrich has joined #openstack-infra | 04:54 | |
| *** zhurong has joined #openstack-infra | 04:54 | |
| *** salv-orlando has joined #openstack-infra | 05:00 | |
| *** MatthewLi has quit IRC | 05:00 | |
| *** MatthewLi has joined #openstack-infra | 05:01 | |
| *** ykarel_ has joined #openstack-infra | 05:01 | |
| *** ykarel has quit IRC | 05:04 | |
| *** aeng has quit IRC | 05:04 | |
| *** sflanigan has quit IRC | 05:04 | |
| *** aviau has quit IRC | 05:06 | |
| *** aviau has joined #openstack-infra | 05:07 | |
| *** dikonoor has joined #openstack-infra | 05:07 | |
| *** yamamoto_ has quit IRC | 05:07 | |
| *** tuanluong has quit IRC | 05:08 | |
| *** gildub_ has quit IRC | 05:09 | |
| *** tphummel has quit IRC | 05:10 | |
| *** ltomasbo|away is now known as ltomasbo | 05:10 | |
| *** tphummel has joined #openstack-infra | 05:10 | |
| *** tphummel has quit IRC | 05:11 | |
| *** esikachev has quit IRC | 05:11 | |
| *** tphummel has joined #openstack-infra | 05:11 | |
| *** tphummel has quit IRC | 05:11 | |
| *** tphummel has joined #openstack-infra | 05:12 | |
| *** gyee has quit IRC | 05:12 | |
| *** prateek has joined #openstack-infra | 05:12 | |
| *** tphummel has quit IRC | 05:12 | |
| *** tphummel has joined #openstack-infra | 05:12 | |
| *** tphummel has quit IRC | 05:13 | |
| *** tphummel has joined #openstack-infra | 05:13 | |
| *** gildub_ has joined #openstack-infra | 05:15 | |
| *** sree has joined #openstack-infra | 05:16 | |
| *** sflanigan has joined #openstack-infra | 05:21 | |
| *** sflanigan has joined #openstack-infra | 05:21 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config master: Add OSH-AddOns to IRC channel notification https://review.openstack.org/471207 | 05:22 |
| openstackgerrit | zhurong proposed openstack-infra/project-config master: Add devstack-plugin-container to solum gate https://review.openstack.org/471208 | 05:24 |
| *** thorst has joined #openstack-infra | 05:29 | |
| *** gongysh has joined #openstack-infra | 05:30 | |
| *** thorst has quit IRC | 05:34 | |
| *** deafult has joined #openstack-infra | 05:41 | |
| *** jaosorior_away is now known as jaosorior | 05:44 | |
| *** cshastri has joined #openstack-infra | 05:46 | |
| *** dhinesh has quit IRC | 05:50 | |
| *** yamamoto has joined #openstack-infra | 05:51 | |
| *** LindaWang has quit IRC | 05:55 | |
| *** MatthewLi has quit IRC | 05:55 | |
| *** LindaWang has joined #openstack-infra | 05:56 | |
| *** pbourke has quit IRC | 05:56 | |
| *** pbourke has joined #openstack-infra | 05:56 | |
| *** MatthewLi has joined #openstack-infra | 05:57 | |
| *** kzaitsev_pi has joined #openstack-infra | 05:57 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements https://review.openstack.org/470447 | 05:59 |
| *** aeng has joined #openstack-infra | 06:01 | |
| *** gomarivera has quit IRC | 06:01 | |
| *** gomarivera has joined #openstack-infra | 06:01 | |
| *** sflanigan has quit IRC | 06:03 | |
| *** tuanluong has joined #openstack-infra | 06:04 | |
| *** gomarivera has quit IRC | 06:06 | |
| *** CynicGalahad has joined #openstack-infra | 06:06 | |
| *** dikonoor has quit IRC | 06:09 | |
| *** rcernin has joined #openstack-infra | 06:10 | |
| *** esikachev has joined #openstack-infra | 06:10 | |
| *** salv-orlando has quit IRC | 06:13 | |
| *** salv-orlando has joined #openstack-infra | 06:13 | |
| *** sflanigan has joined #openstack-infra | 06:15 | |
| *** salv-orlando has quit IRC | 06:17 | |
| *** thorst has joined #openstack-infra | 06:30 | |
| *** zhurong has quit IRC | 06:32 | |
| *** dhajare|busy has joined #openstack-infra | 06:32 | |
| *** thorst has quit IRC | 06:35 | |
| *** pcaruana has joined #openstack-infra | 06:39 | |
| *** zhurong has joined #openstack-infra | 06:39 | |
| *** aeng_ has joined #openstack-infra | 06:39 | |
| *** pgadiya has joined #openstack-infra | 06:40 | |
| *** pgadiya has quit IRC | 06:40 | |
| *** aeng has quit IRC | 06:41 | |
| *** jaosorior is now known as jaosorior_away | 06:41 | |
| *** adriant has quit IRC | 06:42 | |
| *** LindaWang has quit IRC | 06:42 | |
| *** deafult has quit IRC | 06:42 | |
| *** LindaWang has joined #openstack-infra | 06:43 | |
| *** yamamoto has quit IRC | 06:46 | |
| *** yamamoto has joined #openstack-infra | 06:47 | |
| *** eventingmonkey has quit IRC | 06:47 | |
| *** yamahata has joined #openstack-infra | 06:47 | |
| *** amrith has quit IRC | 06:48 | |
| *** amrith has joined #openstack-infra | 06:49 | |
| *** patriciadomin has quit IRC | 06:49 | |
| *** zara_the_lemur__ has quit IRC | 06:49 | |
| *** zara_the_lemur__ has joined #openstack-infra | 06:49 | |
| *** markus_z has joined #openstack-infra | 06:50 | |
| *** danpawlik has quit IRC | 06:50 | |
| *** dpawlik has joined #openstack-infra | 06:50 | |
| *** jrist has quit IRC | 06:50 | |
| *** patriciadomin has joined #openstack-infra | 06:51 | |
| *** esikachev has quit IRC | 06:51 | |
| *** eventingmonkey has joined #openstack-infra | 06:52 | |
| *** deafult has joined #openstack-infra | 06:52 | |
| openstackgerrit | Merged openstack-infra/project-config master: remove manila mitaka lvm job https://review.openstack.org/470358 | 07:00 |
| *** salv-orlando has joined #openstack-infra | 07:01 | |
| openstackgerrit | Merged openstack-infra/project-config master: [sahara] grenade jobs changes https://review.openstack.org/470271 | 07:01 |
| *** aarefiev has joined #openstack-infra | 07:03 | |
| *** cshen has joined #openstack-infra | 07:05 | |
| *** deafult has quit IRC | 07:07 | |
| *** eliqiao has quit IRC | 07:09 | |
| *** eliqiao has joined #openstack-infra | 07:10 | |
| *** tesseract has joined #openstack-infra | 07:13 | |
| *** florianf has joined #openstack-infra | 07:13 | |
| *** CynicGalahad has quit IRC | 07:14 | |
| *** aeng_ has quit IRC | 07:16 | |
| *** hashar has joined #openstack-infra | 07:21 | |
| *** esikachev has joined #openstack-infra | 07:22 | |
| *** yamahata has quit IRC | 07:23 | |
| *** salv-orl_ has joined #openstack-infra | 07:28 | |
| *** armax has quit IRC | 07:29 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Install tripleo-common in buildimage jobs https://review.openstack.org/471052 | 07:29 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add override-branch property to job repos https://review.openstack.org/467375 | 07:29 |
| *** hashar_ has joined #openstack-infra | 07:30 | |
| *** salv-orlando has quit IRC | 07:31 | |
| *** nicolasbock has joined #openstack-infra | 07:31 | |
| *** thorst has joined #openstack-infra | 07:31 | |
| *** jpena|off is now known as jpena | 07:32 | |
| *** hashar has quit IRC | 07:33 | |
| *** thorst has quit IRC | 07:35 | |
| *** amoralej|off is now known as amoralej | 07:36 | |
| *** jrist has joined #openstack-infra | 07:36 | |
| *** jrist has quit IRC | 07:36 | |
| *** jrist has joined #openstack-infra | 07:36 | |
| *** jpich has joined #openstack-infra | 07:37 | |
| *** dtantsur|afk is now known as dtantsur | 07:38 | |
| *** oanson_ has joined #openstack-infra | 07:42 | |
| *** hrubi has quit IRC | 07:43 | |
| *** hrubi has joined #openstack-infra | 07:44 | |
| *** ccamacho has joined #openstack-infra | 07:46 | |
| *** Hal has joined #openstack-infra | 07:46 | |
| *** hrubi has quit IRC | 07:47 | |
| *** hrubi has joined #openstack-infra | 07:47 | |
| *** alexchadin has joined #openstack-infra | 07:48 | |
| *** hrubi has quit IRC | 07:49 | |
| *** hrubi has joined #openstack-infra | 07:49 | |
| *** oanson_ has quit IRC | 07:50 | |
| *** hrubi has quit IRC | 07:50 | |
| *** hrubi has joined #openstack-infra | 07:51 | |
| *** hashar_ is now known as hashar | 07:51 | |
| *** hrubi has quit IRC | 07:52 | |
| *** hrubi has joined #openstack-infra | 07:52 | |
| *** gildub_ has quit IRC | 07:53 | |
| *** hrubi has quit IRC | 07:53 | |
| *** hrubi has joined #openstack-infra | 07:54 | |
| openstackgerrit | Ian Y. Choi proposed openstack-infra/project-config master: Modified job for i18n repo to build translated doc https://review.openstack.org/471242 | 07:55 |
| openstackgerrit | Ian Y. Choi proposed openstack-infra/project-config master: Modifies job for i18n repo to build translated doc https://review.openstack.org/471242 | 07:56 |
| *** markvoelker has quit IRC | 07:57 | |
| *** markvoelker has joined #openstack-infra | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** links has quit IRC | 08:02 | |
| *** gomarivera has joined #openstack-infra | 08:02 | |
| *** markvoelker has quit IRC | 08:02 | |
| *** markmcd has quit IRC | 08:06 | |
| *** gomarivera has quit IRC | 08:07 | |
| *** markmcd has joined #openstack-infra | 08:08 | |
| *** jamesmcarthur has joined #openstack-infra | 08:12 | |
| *** e0ne has joined #openstack-infra | 08:13 | |
| *** kuboj has joined #openstack-infra | 08:14 | |
| *** jamesmcarthur has quit IRC | 08:16 | |
| *** eranrom has quit IRC | 08:17 | |
| *** tosky has joined #openstack-infra | 08:17 | |
| *** vponomaryov has joined #openstack-infra | 08:18 | |
| *** esikachev has quit IRC | 08:19 | |
| *** links has joined #openstack-infra | 08:19 | |
| *** dizquierdo has joined #openstack-infra | 08:20 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Pad state dump https://review.openstack.org/471162 | 08:21 |
| *** dtantsur is now known as dtantsur|bbl | 08:22 | |
| *** rossella_s has joined #openstack-infra | 08:22 | |
| *** gildub_ has joined #openstack-infra | 08:25 | |
| *** lucas-afk is now known as lucasagomes | 08:26 | |
| *** cshastri has quit IRC | 08:30 | |
| kuboj | Hi. Are there any plans on releasing new version of shade any time soon? | 08:31 |
| *** thorst has joined #openstack-infra | 08:32 | |
| *** cshastri has joined #openstack-infra | 08:35 | |
| *** zhurong has quit IRC | 08:35 | |
| *** cshastri has quit IRC | 08:35 | |
| *** cshastri has joined #openstack-infra | 08:36 | |
| *** zhurong has joined #openstack-infra | 08:41 | |
| *** gildub_ has quit IRC | 08:41 | |
| *** derekh has joined #openstack-infra | 08:43 | |
| *** priteau has joined #openstack-infra | 08:49 | |
| *** thorst has quit IRC | 08:51 | |
| *** oanson_ has joined #openstack-infra | 08:51 | |
| *** jaosorior_away is now known as jaosorior | 08:53 | |
| *** panda|off is now known as panda | 08:53 | |
| *** electrofelix has joined #openstack-infra | 08:56 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Fix ceph configuration in nonha job too https://review.openstack.org/471124 | 08:56 |
| *** neil__ has joined #openstack-infra | 09:01 | |
| *** nicolasbock has quit IRC | 09:03 | |
| *** jaosorior has quit IRC | 09:04 | |
| *** dizquierdo has quit IRC | 09:08 | |
| *** markmcd has quit IRC | 09:08 | |
| openstackgerrit | Merged openstack-infra/project-config master: Install tripleo-common in buildimage jobs https://review.openstack.org/471052 | 09:10 |
| *** deafult has joined #openstack-infra | 09:10 | |
| *** huanxie has quit IRC | 09:11 | |
| *** huanxie has joined #openstack-infra | 09:12 | |
| *** rossella_ has joined #openstack-infra | 09:15 | |
| *** rossella_s has quit IRC | 09:17 | |
| *** deafult has quit IRC | 09:22 | |
| *** deafult has joined #openstack-infra | 09:22 | |
| *** eranrom has joined #openstack-infra | 09:23 | |
| *** markmcd has joined #openstack-infra | 09:23 | |
| *** sambetts|afk is now known as sambetts | 09:24 | |
| *** links has quit IRC | 09:25 | |
| *** eranrom_ has joined #openstack-infra | 09:29 | |
| *** dizquierdo has joined #openstack-infra | 09:29 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: DNM: debug pingtest failure on promotion jobs https://review.openstack.org/471139 | 09:31 |
| *** eranrom has quit IRC | 09:31 | |
| *** zhurong has quit IRC | 09:33 | |
| *** links has joined #openstack-infra | 09:43 | |
| *** sdague has joined #openstack-infra | 09:44 | |
| *** ykarel__ has joined #openstack-infra | 09:47 | |
| *** alexchadin has quit IRC | 09:48 | |
| *** thorst has joined #openstack-infra | 09:48 | |
| *** alexchadin has joined #openstack-infra | 09:48 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Update Upstream Institute meeting slots to alternate https://review.openstack.org/471137 | 09:48 |
| *** ykarel_ has quit IRC | 09:50 | |
| *** thorst has quit IRC | 09:52 | |
| *** alexchadin has quit IRC | 09:53 | |
| *** shardy has joined #openstack-infra | 09:55 | |
| *** oanson_ has quit IRC | 09:55 | |
| *** mrmartin has joined #openstack-infra | 09:56 | |
| *** eranrom_ has quit IRC | 09:56 | |
| *** alexchadin has joined #openstack-infra | 09:57 | |
| *** markvoelker has joined #openstack-infra | 09:59 | |
| *** MatthewLi has quit IRC | 10:01 | |
| *** MatthewLi has joined #openstack-infra | 10:01 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/tripleo-ci master: DO NOT MERGE testing 450792 https://review.openstack.org/466322 | 10:02 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/470614 | 10:04 |
| *** dtantsur|bbl is now known as dtantsur | 10:05 | |
| *** jamielennox is now known as jamielennox|away | 10:10 | |
| *** jamielennox|away is now known as jamielennox | 10:16 | |
| *** dpawlik has quit IRC | 10:20 | |
| *** wolverineav has joined #openstack-infra | 10:21 | |
| *** dizquierdo has quit IRC | 10:22 | |
| *** liujiong has quit IRC | 10:23 | |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck master: Add ER query for bug #1696098 https://review.openstack.org/471303 | 10:24 |
| openstack | bug 1696098 in neutron "Race in deleting ports during Tempest Runs" [Undecided,New] https://launchpad.net/bugs/1696098 | 10:24 |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck master: Add new query script to boiler plate out new bugs https://review.openstack.org/471304 | 10:25 |
| *** gongysh has quit IRC | 10:26 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Fix path for tripleo-common https://review.openstack.org/471305 | 10:26 |
| *** markvoelker has quit IRC | 10:32 | |
| *** rossella_ has quit IRC | 10:34 | |
| *** stakeda has quit IRC | 10:34 | |
| *** dizquierdo has joined #openstack-infra | 10:38 | |
| *** jkilpatr has quit IRC | 10:41 | |
| *** sshnaidm is now known as sshnaidm|afk | 10:43 | |
| *** nicolasbock has joined #openstack-infra | 10:44 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix path for tripleo-common https://review.openstack.org/471305 | 10:49 |
| *** rossella_ has joined #openstack-infra | 10:49 | |
| *** ykarel_ has joined #openstack-infra | 10:50 | |
| *** gildub_ has joined #openstack-infra | 10:51 | |
| *** yamamoto has quit IRC | 10:52 | |
| *** ykarel__ has quit IRC | 10:53 | |
| *** dizquierdo has quit IRC | 10:55 | |
| *** sree has quit IRC | 10:58 | |
| *** jkilpatr has joined #openstack-infra | 10:59 | |
| *** rhallisey has joined #openstack-infra | 11:00 | |
| *** gildub_ has quit IRC | 11:01 | |
| *** deafult has quit IRC | 11:06 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add ER query for bug #1696098 https://review.openstack.org/471303 | 11:09 |
| openstack | bug 1696098 in neutron "Race in deleting ports during Tempest Runs" [Undecided,New] https://launchpad.net/bugs/1696098 | 11:09 |
| *** Hal has quit IRC | 11:09 | |
| *** salv-orl_ has quit IRC | 11:16 | |
| *** lucasagomes is now known as lucas-hungry | 11:16 | |
| *** shardy is now known as shardy_afk | 11:21 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Fix the elements_path in image build https://review.openstack.org/471321 | 11:23 |
| *** gildub_ has joined #openstack-infra | 11:23 | |
| *** markvoelker has joined #openstack-infra | 11:29 | |
| *** zhurong has joined #openstack-infra | 11:31 | |
| *** jamesdenton has joined #openstack-infra | 11:32 | |
| *** rlandy has joined #openstack-infra | 11:33 | |
| *** rlandy is now known as rlandy|bbl | 11:34 | |
| *** mat128 has joined #openstack-infra | 11:36 | |
| *** jamesdenton has quit IRC | 11:37 | |
| *** gouthamr has joined #openstack-infra | 11:40 | |
| *** jamesdenton has joined #openstack-infra | 11:44 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:46 | |
| *** jpena is now known as jpena|lunch | 11:47 | |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/project-config master: Use non-deprecated image building https://review.openstack.org/471085 | 11:48 |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/project-config master: Use non-deprecated image building https://review.openstack.org/471085 | 11:49 |
| *** rcernin has quit IRC | 11:52 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Use non-deprecated image building https://review.openstack.org/471085 | 11:52 |
| *** jzerebecki has quit IRC | 11:53 | |
| *** pcaruana has quit IRC | 11:53 | |
| *** thorst has joined #openstack-infra | 11:54 | |
| *** jzerebecki has joined #openstack-infra | 11:54 | |
| *** jaosorior has joined #openstack-infra | 11:56 | |
| *** yamamoto has joined #openstack-infra | 11:56 | |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/project-config master: Use non-deprecated image building https://review.openstack.org/471085 | 11:59 |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/project-config master: Use non-deprecated image building https://review.openstack.org/471085 | 12:01 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add subunit result to CI jobs https://review.openstack.org/468874 | 12:02 |
| *** markvoelker has quit IRC | 12:03 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack-infra/project-config master: jenkins: propose_update.sh: Use a single active repository for testing https://review.openstack.org/471326 | 12:04 |
| *** jamesdenton has quit IRC | 12:05 | |
| *** rcernin has joined #openstack-infra | 12:07 | |
| *** rfolco has joined #openstack-infra | 12:07 | |
| *** baoli has joined #openstack-infra | 12:08 | |
| *** baoli_ has joined #openstack-infra | 12:09 | |
| *** pcaruana has joined #openstack-infra | 12:09 | |
| *** namnh has quit IRC | 12:09 | |
| *** shardy_afk is now known as shardy | 12:10 | |
| *** gildub_ has quit IRC | 12:10 | |
| *** zhurong has quit IRC | 12:12 | |
| *** baoli has quit IRC | 12:13 | |
| *** jamesmcarthur has joined #openstack-infra | 12:13 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/470451 | 12:13 |
| *** kuboj has quit IRC | 12:14 | |
| *** ramishra has quit IRC | 12:14 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use non-deprecated image building https://review.openstack.org/471085 | 12:14 |
| *** dave-mccowan has joined #openstack-infra | 12:16 | |
| *** lucas-hungry is now known as lucasagomes | 12:16 | |
| *** tuanluong has quit IRC | 12:17 | |
| *** jamesmcarthur has quit IRC | 12:18 | |
| *** markvoelker has joined #openstack-infra | 12:20 | |
| *** winggundamth has quit IRC | 12:21 | |
| *** yamamoto has quit IRC | 12:21 | |
| *** jamesdenton has joined #openstack-infra | 12:22 | |
| *** dizquierdo has joined #openstack-infra | 12:25 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: executor: add support for custom ansible_port https://review.openstack.org/468710 | 12:25 |
| *** claudiub has joined #openstack-infra | 12:26 | |
| *** vikrant|brb has quit IRC | 12:31 | |
| *** hrybacki|afkish is now known as hrybacki | 12:33 | |
| *** trown|outtypewww is now known as trown | 12:33 | |
| *** ramishra has joined #openstack-infra | 12:34 | |
| *** amoralej is now known as amoralej|lunch | 12:37 | |
| *** yamamoto has joined #openstack-infra | 12:38 | |
| *** gordc has joined #openstack-infra | 12:39 | |
| *** gouthamr has quit IRC | 12:39 | |
| *** mandre|away is now known as mandre | 12:43 | |
| *** Goneri has joined #openstack-infra | 12:50 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Don't report start on unmanaged projects https://review.openstack.org/455711 | 12:50 |
| *** jpena|lunch is now known as jpena | 12:51 | |
| *** gouthamr has joined #openstack-infra | 12:51 | |
| *** Hal has joined #openstack-infra | 12:52 | |
| *** apetrich has quit IRC | 12:52 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix reporting on changes dequeued for deps https://review.openstack.org/471143 | 12:55 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Mutex repo updates and merge ops in executor https://review.openstack.org/470571 | 12:55 |
| *** links has quit IRC | 12:56 | |
| *** jamesdenton has quit IRC | 12:57 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config master: [sahara] put saharaclient in the projects list https://review.openstack.org/471336 | 12:59 |
| *** efoley has joined #openstack-infra | 12:59 | |
| *** felipemonteiro has joined #openstack-infra | 13:00 | |
| *** efoley has quit IRC | 13:00 | |
| *** efoley has joined #openstack-infra | 13:00 | |
| *** mattmceuen has quit IRC | 13:02 | |
| *** mdrabe has joined #openstack-infra | 13:02 | |
| *** LindaWang has quit IRC | 13:02 | |
| *** felipemonteiro has quit IRC | 13:04 | |
| *** flepied has quit IRC | 13:04 | |
| *** xarses has quit IRC | 13:05 | |
| *** jrist has quit IRC | 13:07 | |
| *** mriedem has joined #openstack-infra | 13:07 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Rename 'repos' job attribute to 'required-projects' https://review.openstack.org/467376 | 13:08 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add override-branch property to jobs https://review.openstack.org/467775 | 13:08 |
| *** ihrachys has joined #openstack-infra | 13:09 | |
| *** jrist has joined #openstack-infra | 13:10 | |
| *** yamamoto has quit IRC | 13:10 | |
| *** jamesdenton has joined #openstack-infra | 13:11 | |
| *** yamamoto has joined #openstack-infra | 13:11 | |
| *** eranrom has joined #openstack-infra | 13:14 | |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/project-config master: Fix tripleo-common path for buildimage https://review.openstack.org/471344 | 13:15 |
| *** gardlt has joined #openstack-infra | 13:16 | |
| *** gomarivera has joined #openstack-infra | 13:17 | |
| *** amoralej|lunch is now known as amoralej | 13:20 | |
| *** hichihara has joined #openstack-infra | 13:20 | |
| *** jamesdenton has quit IRC | 13:20 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: DNM: debug pingtest failure on promotion jobs https://review.openstack.org/471139 | 13:21 |
| *** yamamoto has quit IRC | 13:21 | |
| *** yamamoto has joined #openstack-infra | 13:23 | |
| *** flepied has joined #openstack-infra | 13:23 | |
| *** gardlt_ has joined #openstack-infra | 13:28 | |
| *** jcoufal has joined #openstack-infra | 13:28 | |
| *** isaacb has joined #openstack-infra | 13:29 | |
| *** gomarivera has quit IRC | 13:30 | |
| *** gardlt has quit IRC | 13:30 | |
| *** baoli_ has quit IRC | 13:31 | |
| *** xarses has joined #openstack-infra | 13:33 | |
| *** pcaruana has quit IRC | 13:34 | |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config master: [sahara] introduce py3 scenario jobs https://review.openstack.org/471350 | 13:36 |
| *** baoli has joined #openstack-infra | 13:36 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for vif detach bug 1696125 https://review.openstack.org/471351 | 13:36 |
| *** rcernin has quit IRC | 13:36 | |
| openstack | bug 1696125 in OpenStack Compute (nova) "Detach interface failed - Unable to detach from guest transient domain (pike)" [Medium,Confirmed] https://launchpad.net/bugs/1696125 | 13:36 |
| *** apetrich has joined #openstack-infra | 13:37 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix tripleo-common path for buildimage https://review.openstack.org/471344 | 13:37 |
| *** eranrom has quit IRC | 13:37 | |
| *** gardlt_ has quit IRC | 13:38 | |
| *** gardlt has joined #openstack-infra | 13:38 | |
| *** rlandy|bbl is now known as rlandy | 13:40 | |
| *** prateek has quit IRC | 13:40 | |
| *** zz_dimtruck is now known as dimtruck | 13:41 | |
| *** psachin has quit IRC | 13:41 | |
| *** thorst is now known as thorst_afk | 13:47 | |
| *** LindaWang has joined #openstack-infra | 13:47 | |
| *** rcernin has joined #openstack-infra | 13:49 | |
| *** jamesdenton has joined #openstack-infra | 13:51 | |
| *** pcaruana has joined #openstack-infra | 13:51 | |
| openstackgerrit | Brad P. Crochet proposed openstack-infra/project-config master: Use the same tripleo-common path for all branches in buildimage job https://review.openstack.org/471358 | 13:52 |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/tripleo-ci master: DNM testing something https://review.openstack.org/471359 | 13:52 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: DNM: debug pingtest failure on promotion jobs https://review.openstack.org/471139 | 13:53 |
| *** bobh has joined #openstack-infra | 13:53 | |
| *** psachin has joined #openstack-infra | 13:55 | |
| openstackgerrit | Ihar Hrachyshka proposed openstack-infra/tripleo-ci master: DNM: testing something for gate https://review.openstack.org/471361 | 13:55 |
| *** bh526r has joined #openstack-infra | 13:56 | |
| *** pcaruana has quit IRC | 13:59 | |
| *** pcaruana has joined #openstack-infra | 14:00 | |
| *** rbrndt has joined #openstack-infra | 14:02 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for vif detach bug 1696125 https://review.openstack.org/471351 | 14:02 |
| openstack | bug 1696125 in OpenStack Compute (nova) "Detach interface failed - Unable to detach from guest transient domain (pike)" [Medium,Confirmed] https://launchpad.net/bugs/1696125 | 14:02 |
| *** jamesdenton has quit IRC | 14:02 | |
| *** salv-orlando has joined #openstack-infra | 14:04 | |
| *** mat128 has quit IRC | 14:04 | |
| *** rcernin has quit IRC | 14:04 | |
| *** rcernin has joined #openstack-infra | 14:05 | |
| *** rcernin has quit IRC | 14:07 | |
| *** rcernin has joined #openstack-infra | 14:07 | |
| *** gardlt has quit IRC | 14:08 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use the same tripleo-common path for all branches in buildimage job https://review.openstack.org/471358 | 14:09 |
| *** jaypipes has quit IRC | 14:10 | |
| *** eharney has joined #openstack-infra | 14:11 | |
| *** mdrabe has quit IRC | 14:11 | |
| *** mdrabe has joined #openstack-infra | 14:12 | |
| *** links has joined #openstack-infra | 14:12 | |
| *** ldnunes has joined #openstack-infra | 14:14 | |
| *** r-daneel has joined #openstack-infra | 14:14 | |
| *** armax has joined #openstack-infra | 14:15 | |
| *** mat128 has joined #openstack-infra | 14:17 | |
| *** links has quit IRC | 14:17 | |
| *** jamesmcarthur has joined #openstack-infra | 14:18 | |
| *** jamesmcarthur has quit IRC | 14:18 | |
| *** jamesmcarthur has joined #openstack-infra | 14:19 | |
| *** bexelbie has quit IRC | 14:23 | |
| *** rossella__ has joined #openstack-infra | 14:23 | |
| *** bexelbie has joined #openstack-infra | 14:23 | |
| *** rossella_ has quit IRC | 14:25 | |
| *** alexchadin has quit IRC | 14:25 | |
| *** hongbin has joined #openstack-infra | 14:28 | |
| *** ihrachys is now known as ihrachys|afk | 14:31 | |
| sdague | anyone able to look at top of gate and figure out why the job isn't completing | 14:31 |
| openstackgerrit | Merged openstack-infra/infra-specs master: Add nodepool drivers storyboard link and assignees https://review.openstack.org/468623 | 14:31 |
| sdague | it's hung at SUCCESS | 14:31 |
| sdague | telnet://89.46.82.24:19885 | 14:31 |
| *** markmcd has quit IRC | 14:31 | |
| sdague | 2017-06-06 12:42:17.281251 | Generating static files at /home/jenkins/workspace/gate-tempest-dsvm-neutron-full-ubuntu-xenial/logs/ara... | 14:31 |
| sdague | 2017-06-06 12:42:18.402215 | Done. | 14:31 |
| sdague | 2017-06-06 12:42:19.936113 | SUCCESSFULLY FINISHED | 14:31 |
| sdague | 2017-06-06 12:42:19.938479 | [Zuul] Task exit code: 0 | 14:31 |
| sdague | 2017-06-06 12:43:04.418633 | [Zuul] Job complete, result: SUCCESS | 14:31 |
| *** smarcet has joined #openstack-infra | 14:32 | |
| *** pcrews has quit IRC | 14:33 | |
| sdague | it finished like nearly an hour ago, but zuul seems to have lost it | 14:33 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/infra-specs master: Add a help-wanted section https://review.openstack.org/471149 | 14:33 |
| *** eharney has quit IRC | 14:33 | |
| smarcet | fungi: morning ! when u get the change please review https://review.openstack.org/#/c/470466/ is just a missing = on property file thx! | 14:33 |
| *** rossella_ has joined #openstack-infra | 14:34 | |
| fungi | sdague: i'm looking now to see which launcher that was | 14:34 |
| *** akscram has quit IRC | 14:34 | |
| *** rossella__ has quit IRC | 14:35 | |
| sdague | fungi: ok, cool. I don't know if there is a way to manually get it to complete, if so that would be nice. Because there are 10 passed and completed patches behind it | 14:35 |
| sdague | and restarting all that work again would be sad | 14:35 |
| *** akscram has joined #openstack-infra | 14:36 | |
| *** Guest28014 is now known as bramwelt | 14:37 | |
| *** bramwelt has quit IRC | 14:37 | |
| *** bramwelt has joined #openstack-infra | 14:37 | |
| *** Julien-zte has joined #openstack-infra | 14:37 | |
| fungi | sdague: looks like one of the post-run ansible tasks reached a timeout and the launcher is attempting (unsuccessfully) to abort it. still digging deeper to see if i can tell why | 14:38 |
| fungi | teh node is still online | 14:39 |
| *** dizquierdo has quit IRC | 14:39 | |
| fungi | the only ansible process still running on it seems to be: | 14:40 |
| fungi | jenkins 1151 0.0 0.1 189084 9720 ? Sl 11:33 0:00 /usr/bin/python /tmp/ansible_Meb8XQ/ansible_module_zuul_console.py | 14:40 |
| openstackgerrit | Merged openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/470614 | 14:40 |
| fungi | stracing that pid seems to just hang, nothing going on there | 14:40 |
| *** xarses_ has joined #openstack-infra | 14:40 | |
| *** baoli has quit IRC | 14:41 | |
| sdague | fungi: so, if you manually kill it, what happens? reset? | 14:41 |
| *** ggnel_t has quit IRC | 14:41 | |
| sdague | or are those things safe fails? | 14:41 |
| fungi | i think zuul is already trying to kill it, so yes it's on its way to a POST_FAILURE result as far as i can tell | 14:41 |
| sdague | fungi: has it been trying to for an hour? | 14:42 |
| *** pcrews has joined #openstack-infra | 14:42 | |
| sdague | because this completed quite a long time ago | 14:42 |
| sdague | sucks to not have any idea why that wedged | 14:42 |
| fungi | 2017-06-06 13:12:54,553 DEBUG zuul.NodeWorker.ubuntu-xenial-citycloud-la1-9157839: Abort: sending kill signal to job process group | 14:42 |
| sdague | ok, so that was 30 minutes ago? | 14:43 |
| fungi | so yeah, about 1.5 hours ago is when it decided this timed out | 14:43 |
| sdague | could you do your own kill -9 on that to see what happens? | 14:43 |
| fungi | that's 13:12 utc, it's now 14:43 utc by my clock | 14:43 |
| sdague | ah, ok, so even longer than I had thought | 14:43 |
| fungi | i'm going to in a moment as soon as i finish trying to see if i can tell what's going on with it | 14:43 |
| sdague | ok, cool | 14:43 |
| pabelanger | odd, so watchdog failed in zuul-launcher? | 14:45 |
| fungi | i'm guessing so | 14:45 |
| pabelanger | first time I've seen that | 14:45 |
| fungi | care to see if you can spot anything wrong there briefly before i destructively alter its state? | 14:45 |
| sdague | fungi: hmm... it just reported final success | 14:45 |
| pabelanger | fungi: sure, 1 min | 14:45 |
| fungi | wow! | 14:45 |
| fungi | the ansible process is still going | 14:46 |
| sdague | could it be something else it was trying to talk back to? | 14:46 |
| pabelanger | telnet just died | 14:47 |
| sdague | pabelanger: yeh, it finally reported | 14:47 |
| fungi | yeah, i went to hold the node and nodepool had already deleted it | 14:47 |
| pabelanger | were we copying logs? | 14:47 |
| *** mdrabe has quit IRC | 14:47 | |
| sdague | pabelanger: no idea, if so, that's a really freaking long time | 14:47 |
| pabelanger | sdague: indeed | 14:47 |
| pabelanger | let me check was we setup post playbook timeout to again | 14:48 |
| fungi | well, we have ansible logs now we can look at at least | 14:48 |
| fungi | http://logs.openstack.org/46/468646/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/97ddf30/_zuul_ansible/ | 14:48 |
| *** eharney has joined #openstack-infra | 14:48 | |
| fungi | except they end abruptly a couple hours ago | 14:49 |
| pabelanger | no, don't get the last bits, because we use ansible to upload ansible logs | 14:50 |
| sdague | right, at the success that was expected no? | 14:50 |
| sdague | it's a citycloud node, any thoughts that something might be funny there? | 14:50 |
| sdague | I don't know how long those have been in rotation | 14:50 |
| *** dizquierdo has joined #openstack-infra | 14:50 | |
| *** cshastri has quit IRC | 14:51 | |
| pabelanger | checking zl01 now | 14:51 |
| sdague | we're definitely merging the patches now | 14:51 |
| fungi | looks like it woke up in the launcher log here: | 14:51 |
| fungi | 2017-06-06 14:44:27,786 DEBUG zuul.NodeWorker.ubuntu-xenial-citycloud-la1-9157839: Ansible post exit code: 0 | 14:51 |
| pabelanger | ya, that looks real odd. The post playbook was successful, but ansible-playbook seems to be still running | 14:53 |
| pabelanger | Oh | 14:53 |
| pabelanger | look at the load on zl01 | 14:53 |
| pabelanger | it is huge | 14:53 |
| fungi | gah, that is high | 14:54 |
| *** psachin has quit IRC | 14:54 | |
| sdague | so we're in mega merge now | 14:54 |
| *** rcernin has quit IRC | 14:54 | |
| sdague | including merge conflict checking with nova | 14:54 |
| fungi | lots of system load in addition to user load on zl01 | 14:54 |
| pabelanger | ya | 14:54 |
| fungi | compared to zl02 which is mostly idle | 14:54 |
| sdague | as I sent in 20+ nova patches to the gate this morning | 14:55 |
| *** yamamoto has quit IRC | 14:55 | |
| pabelanger | Wow | 14:55 |
| sdague | 10 of which are now merging | 14:55 |
| pabelanger | zl01 is handling like twice the amount of jobs as other launchers | 14:55 |
| pabelanger | http://grafana.openstack.org/dashboard/db/nodepool | 14:55 |
| *** ykarel_ has quit IRC | 14:55 | |
| pabelanger | I've never see a launcher do more then 160 before | 14:55 |
| *** marst has joined #openstack-infra | 14:55 | |
| pabelanger | maybe we should place it in graceful shutdown to attempt to rebalance | 14:55 |
| fungi | pabelanger: it's swapping a bunch. look at the memory utilization for zuul-launcher... 12.473g VIRT | 14:56 |
| *** markmcd has joined #openstack-infra | 14:56 | |
| *** Julien-zte has quit IRC | 14:56 | |
| pabelanger | ya, something is off with our job balancer for some reason | 14:56 |
| pabelanger | we do have a fair bit of launch failures too | 14:56 |
| *** Julien-zte has joined #openstack-infra | 14:57 | |
| pabelanger | ovh-bhs1 seems to be the issue | 14:57 |
| *** armax has quit IRC | 14:58 | |
| *** dimtruck is now known as zz_dimtruck | 14:58 | |
| fungi | on the memory utilization front, zl01 is using nearly 50% of its swap while zl02 is using less than 5% of its swap | 14:59 |
| openstackgerrit | Merged openstack-infra/shade master: Updated from global requirements https://review.openstack.org/470447 | 14:59 |
| pabelanger | fungi: ya, I think we just have too many ansible-playbook processes running | 14:59 |
| *** salv-orlando has quit IRC | 14:59 | |
| openstackgerrit | Merged openstack-infra/shade master: Handle ports with no 'created_at' attribute https://review.openstack.org/469869 | 14:59 |
| fungi | i need to hop on a conference call, but i'll keep digging into this too | 14:59 |
| *** salv-orlando has joined #openstack-infra | 15:00 | |
| pabelanger | we've dropped to below 188 jobs now, cpu looks to be recovering | 15:00 |
| *** armax has joined #openstack-infra | 15:00 | |
| sdague | pabelanger: yeh, the mega merge just completed about a minute ago | 15:00 |
| *** hichihara has quit IRC | 15:00 | |
| sdague | fungi / pabelanger thanks for looking into this | 15:01 |
| *** isaacb has quit IRC | 15:01 | |
| clarkb | the launchers don't handle the merge though | 15:01 |
| pabelanger | lets see if zuul-launchers even out | 15:02 |
| fungi | though swap utilization _has_ freed up a little on zl01 now | 15:02 |
| *** LindaWang has quit IRC | 15:03 | |
| sdague | clarkb: idk, just seems corolated | 15:03 |
| *** LindaWang has joined #openstack-infra | 15:03 | |
| pabelanger | but I am thinking our large amount of launcher failures might also be skewing our node assignments to launchers, and some reason zl01.o.o is accepting more then others | 15:03 |
| sdague | any chance it's a shared constraint like the git servers getting slow when everyone is pounding on them? | 15:03 |
| *** salv-orlando has quit IRC | 15:04 | |
| clarkb | possibly gerrit | 15:04 |
| fungi | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4682&rra_id=all shows swap on zl01 started to get slammed around 12:30 utc today | 15:04 |
| clarkb | at least on the merge side for both submit and preparing the speculative state | 15:04 |
| *** pbourke has quit IRC | 15:05 | |
| pabelanger | wonder if we need to add max-jobs setting for an launcher and cap them to 200 | 15:05 |
| *** gomarivera has joined #openstack-infra | 15:05 | |
| fungi | compare to memory usage on zl02 which didn't really spike http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4738&rra_id=all | 15:05 |
| pabelanger | other launchers seem to be picking up more jobs now, all other 6 are on the up tick | 15:08 |
| pabelanger | zl01 still climbing a little bit | 15:08 |
| fungi | established tcp connections graph envelopes are very different between the two as well... i expect the dragging performance on zl01 was resulting in a pileup there | 15:08 |
| *** LindaWang has quit IRC | 15:09 | |
| fungi | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4685&rra_id=all vs http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=4737&rra_id=all | 15:09 |
| *** Julien-zte has quit IRC | 15:09 | |
| *** sree has joined #openstack-infra | 15:09 | |
| *** Julien-zte has joined #openstack-infra | 15:09 | |
| fungi | similar climb on rootfs utilization for zl01 which is not present on zl02 | 15:10 |
| *** LindaWang has joined #openstack-infra | 15:11 | |
| *** ericyoung has joined #openstack-infra | 15:12 | |
| *** hashar has quit IRC | 15:13 | |
| *** mdrabe has joined #openstack-infra | 15:13 | |
| pabelanger | I still it might be worth forcing a rebalance by grafeful stop of ze01 | 15:14 |
| pabelanger | err | 15:14 |
| pabelanger | zl01 | 15:14 |
| *** sree has quit IRC | 15:14 | |
| fungi | yeah, i'm curious if letting zl01 wind down and then putting it back in rotation finds it still trend back to handling a larger job count | 15:16 |
| fungi | could indicate a flaw in the scheduling/distribution algorithm | 15:17 |
| *** pbourke has joined #openstack-infra | 15:17 | |
| pabelanger | ya, I think it is pretty basic now IIRC | 15:17 |
| clarkb | gearman distribution algorithm is "who is fast enough to win" | 15:18 |
| pabelanger | don't we sleep based on current node count or something like that | 15:18 |
| clarkb | which should sort of let things steady state based on load, unless we grab a bunch of work before we get into the costly aspects of it then hit the problems later | 15:18 |
| tumbarka | Hi All, In my CI setup zuul service is not coming up, zuul-merger is running properly | 15:19 |
| tumbarka | the /var/run/zuul/zuul.pid file is not there | 15:19 |
| pabelanger | clarkb: fungi: https://review.openstack.org/#/c/330786/ | 15:19 |
| tumbarka | Earlier I faced same issue at that time correct version of voluptuous package solved my problem... Any one has any idea about this? | 15:21 |
| pabelanger | tumbarka: if you run zuul-server -d, do you see anything on console? | 15:21 |
| *** gomarivera has quit IRC | 15:22 | |
| *** flepied has quit IRC | 15:22 | |
| *** gomarivera has joined #openstack-infra | 15:22 | |
| *** eranrom has joined #openstack-infra | 15:22 | |
| clarkb | pabelanger: so thats a 10 second sleep with 100 nodes | 15:23 |
| tumbarka | Pablenager : its throwing exception - https://pastebin.com/dPV0CgnQ | 15:24 |
| pabelanger | clarkb: right, I'm thinking we should maybe have a cap. I've never seen a zuul-launcher about 200 jobs before, and look like we just run our of resources to run ansible | 15:24 |
| pabelanger | tumbarka: looks like gearman is already running, do you have any hung processes | 15:24 |
| *** sbezverk has joined #openstack-infra | 15:25 | |
| pabelanger | tumbarka: Oh | 15:25 |
| pabelanger | --no-webapp | 15:25 |
| pabelanger | I think that is the issue | 15:25 |
| clarkb | pabelanger: thats a 40 second sleep at 200 nodes ... seems like that should be sufficient to give the other launchers priority. Perhaps there is a bug in that? | 15:25 |
| pabelanger | clarkb: possible we are overloading gearman again? | 15:25 |
| *** eranrom has quit IRC | 15:26 | |
| openstackgerrit | Merged openstack-infra/puppet-openstackid master: Fixed typo on session config https://review.openstack.org/470466 | 15:26 |
| pabelanger | tumbarka: that seems to me you have something listening on http port that zuul users, check to see if something is already running | 15:26 |
| clarkb | fungi: when you have a moment would you be willing to review https://review.openstack.org/#/c/470421/ and its depends on? trying to simplify the swift functional testing there | 15:27 |
| tumbarka | Pablanger : Yes there was a zuul-server process running already,but it can not find /var/run/zuul/zuul.pid, still the process running | 15:27 |
| pabelanger | fungi: clarkb: ya, we still have some jobs still hanging around on zl01: 471268,2 is up to 6 hours | 15:28 |
| tumbarka | If I killed all the process and restarted again ..it again deletes the file /var/run/zuul/zuul.pid within few seconds | 15:28 |
| pabelanger | tumbarka: which operating system | 15:28 |
| tumbarka | ubuntu 14.04 | 15:29 |
| pabelanger | tumbarka: if make sure you dump your queues, if you kill the process, then start it again. See if the pid is created | 15:29 |
| pabelanger | I remember seeing something like this before, but cannot remember the issue | 15:29 |
| *** chlong has joined #openstack-infra | 15:29 | |
| pabelanger | clarkb: I think we'll want to stop new job launches on zl01 until things level out. then bring it back into rotation | 15:30 |
| pabelanger | I also have an email out to OVH to see about bhs1 region | 15:31 |
| tumbarka | Pabelanger : After starting the service again , it creates the pid file and deletes it within few seconds | 15:31 |
| *** dizquierdo has quit IRC | 15:31 | |
| clarkb | ya putting it in graceful shutdown so things like 471268 can clear out seems like a good idea | 15:31 |
| pabelanger | tumbarka: how many zuul processes are left, 1? | 15:32 |
| pabelanger | clarkb: k, doing so now | 15:32 |
| tumbarka | Earlier I faced same issue at that time downgradig the voluptuous package solved the problem | 15:32 |
| *** e0ne has quit IRC | 15:32 | |
| pabelanger | okay, zl01 in graceful mode | 15:33 |
| pabelanger | tumbarka: strace might be next step | 15:33 |
| pabelanger | clarkb: the gate is busy today too, first time in a while I've seen us up to 1100 jobs | 15:34 |
| tumbarka | In ps -eaf it shows zuul-server as well as zuul-merger, but /var/run/zuul/zuul.pid is not there unlike /var/run/zuul-merger/zuul-merger.pid | 15:34 |
| pabelanger | tumbarka: just 1 server right? | 15:35 |
| tumbarka | Pabelnager : strace? | 15:35 |
| pabelanger | if so, that is likely gearman that is running, but scheduler stopped for some reason | 15:35 |
| pabelanger | zuul-server creates 2 processes | 15:35 |
| tumbarka | Pabelanger : yes its created 2 server - https://pastebin.com/A0vD1kV7 | 15:37 |
| *** flepied has joined #openstack-infra | 15:39 | |
| *** eranrom has joined #openstack-infra | 15:41 | |
| *** dpaterson has quit IRC | 15:41 | |
| *** panda is now known as panda|bbl | 15:42 | |
| clarkb | pabelanger: at ~1k peak nodes (according to graphite) that should come out to ~125 work nodes per launcher on an even distribution | 15:42 |
| clarkb | if 01 had ~200 then thats about 115 for the other nodes (which isn't terrible except that they all seem to have bunched up on 01) | 15:43 |
| clarkb | wondering if the sleep just effectively slowed down all the nodes such that the fastest one was still winning? I dunno would need more logs but we don't seem to log those sleeps | 15:43 |
| pabelanger | clarkb: agree, zl01 was pushing 250 jobs at peak load | 15:44 |
| clarkb | oh wait | 15:44 |
| clarkb | the sleep is only for launchnodes not for accepting jobs | 15:44 |
| clarkb | so if you end up with a distribution of longer jobs on one node then that would accrue more launch nodes I think | 15:44 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Use shade discovery for keystone https://review.openstack.org/471404 | 15:45 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Avoid keystoneclient making yet another discovery call https://review.openstack.org/471405 | 15:45 |
| clarkb | baically jobs run long enough that they are still running overcomes the ~40 second timeout to get more nodes which then themselves run really long jobs? | 15:45 |
| clarkb | its like a snowball going downhill picking up moss | 15:45 |
| pabelanger | oh, I see what you are saying | 15:46 |
| pabelanger | interesting | 15:46 |
| *** sbezverk has quit IRC | 15:46 | |
| tumbarka | Pabelanger : Is there possibility of python package version mismatch ? | 15:46 |
| *** jaosorior is now known as jaosorior_away | 15:47 | |
| pabelanger | tumbarka: not sure, if you stop all zuul-server processes, then run zuul-server -d again, what console output do you get. you can use --no-web-app also to stop binding to http for now | 15:47 |
| pabelanger | see if zuul servers | 15:47 |
| pabelanger | otherwise, you might need to use strace | 15:47 |
| *** markus_z has quit IRC | 15:47 | |
| *** sbezverk has joined #openstack-infra | 15:47 | |
| *** ericjuma has joined #openstack-infra | 15:48 | |
| tumbarka | Pabelanger : Can you please tell what is strace? | 15:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: First Pass: De-client-ify Users https://review.openstack.org/467784 | 15:48 |
| *** jamesdenton has joined #openstack-infra | 15:51 | |
| *** bobh_ has joined #openstack-infra | 15:51 | |
| *** ltomasbo is now known as ltomasbo|away | 15:52 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Write inventory as yaml not ini https://review.openstack.org/467634 | 15:52 |
| *** hamzy has quit IRC | 15:52 | |
| openstackgerrit | Alfredo Moralejo proposed openstack-infra/tripleo-ci master: DNM: testing something for gate https://review.openstack.org/471361 | 15:52 |
| tumbarka | Pablenger : stopped zuul-server processes and run zuul-server -d , got following output - https://pastebin.com/ajkrYcGN | 15:52 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add support for defining groups in nodesets https://review.openstack.org/467611 | 15:53 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Write inventory as yaml not ini https://review.openstack.org/467634 | 15:53 |
| *** bobh has quit IRC | 15:54 | |
| *** eranrom has quit IRC | 15:55 | |
| *** rfolco has quit IRC | 15:56 | |
| *** dizquierdo has joined #openstack-infra | 15:56 | |
| pabelanger | tumbarka: looks like you might be missing the includes file in your layout.yaml file | 15:56 |
| *** dizquierdo has quit IRC | 15:56 | |
| *** dizquierdo_ has joined #openstack-infra | 15:56 | |
| *** hashar has joined #openstack-infra | 15:57 | |
| pabelanger | is single_use_node is missing | 15:57 |
| pabelanger | likely in your functions.py file | 15:57 |
| ericjuma | Hi, I am wondering how to set up a multi-node Jenkins gate for our project (Mix and Match https://review.openstack.org/#/q/project:openstack/mixmatch) | 15:57 |
| *** gyee has joined #openstack-infra | 15:58 | |
| *** lucasagomes is now known as lucas-afk | 15:58 | |
| fungi | notmyname: are you on board with clarkb's change to speed up swift functional testing? https://review.openstack.org/470421 | 15:59 |
| *** dpaterson has joined #openstack-infra | 15:59 | |
| clarkb | EmilienM: bkero http://logs.openstack.org/72/470172/2/check/gate-tripleo-ci-centos-7-containers-multinode-nv/89cc705/logs/subnode-2/var/log/extra/docker/containers/ is saving about half a gig of logs per tripleo container job. Do we really need all of /etc from each container? | 16:00 |
| fungi | ericjuma: i guess you mean multi-node job (you're using the related term "gate" in ways which make little sense). can you elaborate on what you want to test with multiple job nodes which you can't effectively test using a single job node? | 16:01 |
| notmyname | fungi: yes. if we're running swift tests, no need to install/run nova, neutron, glance--anything other than swift+keystone actually | 16:01 |
| *** ihrachys|afk is now known as ihrachys | 16:01 | |
| fungi | notmyname: awesome, i didn't see a +1 from you on that change so just wanted to make sure you were cool with it | 16:01 |
| notmyname | fungi: I'll leave the same comment in gerrit | 16:01 |
| EmilienM | jistr, sshnaidm, panda|bbl ^ have you seen that before? | 16:01 |
| EmilienM | clarkb: thx | 16:01 |
| fungi | notmyname: thanks! | 16:01 |
| bkero | clarkb: No, in fact I believe we have some outstanding reviews about reducing that. | 16:01 |
| *** rfolco has joined #openstack-infra | 16:01 | |
| fungi | clarkb: is that a recent increase do you think? | 16:02 |
| *** mudpuppy has joined #openstack-infra | 16:02 | |
| fungi | that's a stunning log volume | 16:02 |
| clarkb | fungi: it came up when we first looked at reducing tripleo log sizes, Not sure if it ever got reduced after that (I doubt it) | 16:02 |
| EmilienM | bkero: which ones? | 16:02 |
| sshnaidm | EmilienM, clarkb I think it could be reduced | 16:03 |
| sshnaidm | clarkb, before it was only 3 containers there, so it wasn't problem | 16:03 |
| ericjuma | fungi: Yes, that's what I mean. We need to do keystone to keystone federation between two devstacks and test how they interact through the proxy we're developing. The purpose of our project is to provide an interface for multiple openstack deployments to interact | 16:03 |
| clarkb | well even at 3 containers we don't need to copy all of /etc | 16:04 |
| *** rossella_ has quit IRC | 16:04 | |
| clarkb | especially when they are identical across all of them | 16:04 |
| sshnaidm | clarkb, right | 16:04 |
| fungi | ericjuma: i suggest starting by looking at the multi-node support documented in the openstack-infra/devstack-gate repo. it currently deploys devstack on each job node in the multi-node group, sets up some overlay networking to connect them to each other, and configures them however you specify: https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/multinode_setup_info.txt | 16:06 |
| fungi | ericjuma: i'm not aware of any jobs testing federated keystone yet, but maybe check with the keystone team to see if they might be doing something like that already you could copy | 16:06 |
| *** Apoorva has joined #openstack-infra | 16:06 | |
| clarkb | worth noting that federated keystone doesn't strictly require multinode jobs | 16:07 |
| clarkb | you can run keystone on two different ports against two different dbs in mysql | 16:07 |
| *** jamesdenton has quit IRC | 16:08 | |
| *** jascott1 has joined #openstack-infra | 16:08 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Return resulting commits from merger https://review.openstack.org/467776 | 16:08 |
| *** dhinesh has joined #openstack-infra | 16:09 | |
| *** dhinesh has quit IRC | 16:11 | |
| tumbarka | Pabelanger : Its there in layout.yaml -- parameter-function: single_use_node | 16:11 |
| pabelanger | tumbarka: yup, that should be loaded from your includes file | 16:11 |
| bkero | EmilienM, clarkb: https://review.openstack.org/#/c/447176/ | 16:11 |
| EmilienM | bkero: looks staled, no? | 16:12 |
| *** jamesdenton has joined #openstack-infra | 16:12 | |
| bkero | EmilienM: definitely stale, yeah | 16:12 |
| bkero | This is what happened last time clarkb raised this though | 16:12 |
| clarkb | flaper87: you can't have the entire identical /etc from every container | 16:12 |
| flaper87 | clarkb: ? I'm confused, what did I do? | 16:13 |
| bkero | flaper87: https://review.openstack.org/#/c/447176/ | 16:13 |
| *** mamitchl has joined #openstack-infra | 16:13 | |
| clarkb | flaper87: 1) its identical 2) its not useful info, we don't need http://logs.openstack.org/72/470172/2/check/gate-tripleo-ci-centos-7-containers-multinode-nv/89cc705/logs/subnode-2/var/log/extra/docker/containers/novacompute/etc/DIR_COLORS.256color.gz and everything else 3) it is putting a burden on the lgo servers | 16:13 |
| tumbarka | Pablenger : I ts there - | 16:13 |
| tumbarka | includes: | 16:13 |
| tumbarka | - python-file: openstack_functions.py | 16:13 |
| pabelanger | EmilienM: clarkb: flaper87: bkero these are also containers, can you just download them externally and inspect them? | 16:13 |
| fungi | pick the files you know you need to copy for specific debugging reasons, and when a job fails and you see you're missing a file which would help in debugging add it to the list so it gets collected next time | 16:13 |
| fungi | copying everything on the assumption that you _might_ need files you don't know about yet is wasteful of available storage and reduces log retention for everyone sharing that resource | 16:14 |
| flaper87 | what fungi said but we'll basically need to do this per container | 16:15 |
| flaper87 | the files we're interested in are the config files for the service itself and the puppet files, I think | 16:15 |
| *** yamahata has joined #openstack-infra | 16:15 | |
| flaper87 | I'll remove my -1 and we can do this better in a follow-up patch | 16:16 |
| *** smarcet has quit IRC | 16:16 | |
| *** tosky__ has joined #openstack-infra | 16:16 | |
| flaper87 | I don't feel strong either way as soon as we have a way (and we do) collect the config files from the container. | 16:16 |
| *** jamesmcarthur has quit IRC | 16:16 | |
| *** hashar has quit IRC | 16:16 | |
| flaper87 | Actually, given how we're doing things now, we might not need to copy anything from the container at all | 16:16 |
| bkero | We're collecting /var/lib/kolla/config_files/config.json | 16:16 |
| *** jamesmcarthur has joined #openstack-infra | 16:16 | |
| *** tosky has quit IRC | 16:16 | |
| flaper87 | bkero: exactly | 16:16 |
| bkero | Would it be possible to recreate the configs from that? | 16:16 |
| *** tosky__ is now known as tosky | 16:17 | |
| flaper87 | bkero: it'd be enough to collect /var/lib/config-data/ | 16:17 |
| flaper87 | that has all we need | 16:17 |
| fungi | or at least enough of them to be able to figure out whether they're useful in debugging a particular class of failures | 16:17 |
| flaper87 | from the host | 16:17 |
| flaper87 | fungi: clarkb bkero pabelanger thanks for the ping, I had forgotten about this patch | 16:17 |
| flaper87 | approved | 16:18 |
| bkero | flaper87: I updated the patch to add the collection of /var/lib/config-data | 16:19 |
| flaper87 | actually, +2'd, let's see what the gate says | 16:19 |
| flaper87 | :) | 16:19 |
| EmilienM | flaper87: bkero pushed over, please review again | 16:19 |
| fungi | and while it seems like copying the contents of /etc shouldn't result in a lot of data, the fact that there are a number of files and even directories symlinked from /etc to other places like /usr end up recursively dragging in a bunch of large binaries and similar sorts of surprises | 16:19 |
| pabelanger | why do those containers have systemd? | 16:19 |
| *** trown is now known as trown|lunch | 16:20 | |
| clarkb | pabelanger: they look like full distro images. I mean why does a container care about dir color config :P | 16:20 |
| pabelanger | clarkb: agree | 16:20 |
| flaper87 | bkero: that path is in the host, not in the container :/ | 16:22 |
| bkero | flaper87: got it | 16:23 |
| openstackgerrit | Modestas Vainius proposed openstack-infra/jenkins-job-builder master: Support taking slack token from Jenkins credentials. https://review.openstack.org/471415 | 16:23 |
| ericjuma | fungi: Thank you, this is a good place to start. | 16:24 |
| bkero | flaper87: does something like the latest commit look better? | 16:25 |
| *** yamamoto has joined #openstack-infra | 16:26 | |
| *** jamielennox is now known as jamielennox|away | 16:26 | |
| *** jpich has quit IRC | 16:26 | |
| *** jamesdenton has quit IRC | 16:27 | |
| *** clenimar_ has joined #openstack-infra | 16:28 | |
| *** clenimar_ has quit IRC | 16:28 | |
| *** baoli has joined #openstack-infra | 16:29 | |
| *** yamamoto has quit IRC | 16:31 | |
| *** tesseract has quit IRC | 16:32 | |
| *** LindaWang has quit IRC | 16:32 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: WIP: add log streaming test https://review.openstack.org/471079 | 16:32 |
| *** pcaruana has quit IRC | 16:34 | |
| *** armax has quit IRC | 16:35 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Create zuul-executor hiera group https://review.openstack.org/471418 | 16:37 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 16:38 |
| *** armax has joined #openstack-infra | 16:38 | |
| lbragstad | odyssey4me: andymccr ^ my initial first stab (which could be completely wrong) | 16:39 |
| fungi | memory utilization has finally started to drop on zl01 | 16:39 |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Rename overcloud-sec-hard-full to overcloud-hardened-full https://review.openstack.org/471420 | 16:40 |
| pabelanger | fungi: ya, we should be in graceful mode now | 16:40 |
| pabelanger | we are also down about 50% of the jobs from this morning too | 16:40 |
| fungi | yep, it's taking a while to run all those out | 16:41 |
| pabelanger | maybe 40% | 16:41 |
| *** ldnunes has quit IRC | 16:41 | |
| *** vhosakot has joined #openstack-infra | 16:41 | |
| *** pbourke has quit IRC | 16:41 | |
| *** ldnunes has joined #openstack-infra | 16:42 | |
| lbragstad | quick question jjb question - is there a macro or required step to determine the location or clone the change from gerrit for a project? | 16:43 |
| *** sshnaidm is now known as sshnaidm|afk | 16:43 | |
| fungi | lbragstad: generally jobs either use the devstack-gate framework or the zuul-git-prep jjb macro to clone the appropriate repository with the corresponding change applied | 16:45 |
| lbragstad | fungi: oh - cool, let me check out the macro | 16:45 |
| fungi | devstack-gate, because it tracks and maintains a multitude of git repos, organizes them under /opt/stack/new/ while the zuul-git-prep macro does a little bit of a dance to ensure that the project identified by the $ZUUL_PROJECT envvar is checked out as the job $WORKSPACE path | 16:47 |
| lbragstad | fungi: oh - interesting | 16:47 |
| lbragstad | fungi: in case this helps with the context - i left a comment on line 226 with what i'm trying to do https://review.openstack.org/#/c/471419/1/jenkins/jobs/keystone.yaml | 16:47 |
| *** jpena is now known as jpena|off | 16:49 | |
| fungi | lbragstad: looks like that builder is basically a copy of zuul-git-prep-upper-constraints but overriding the project to hard-code it to openstack/openstack-ansible-os_keystone no matter what job uses that builder... is that the intent? | 16:50 |
| *** efoley has quit IRC | 16:51 | |
| *** eharney has quit IRC | 16:51 | |
| lbragstad | fungi: correct - what i'm trying to do is leverage the tox upgrade environment of openstack-ansible-os_keystone | 16:51 |
| lbragstad | https://github.com/openstack/openstack-ansible-os_keystone/blob/master/tox.ini#L105 | 16:51 |
| lbragstad | which is a series of playbooks that deploys keystone latest stable branch to a couple containers, then upgrades to another version of keystone | 16:52 |
| fungi | lbragstad: skimming, looks basically sane then. though you haven't actually proposed to run it anywhere that i can see | 16:52 |
| lbragstad | fungi: nope - it's brand new | 16:52 |
| lbragstad | the upgrade tox environment expects the version (or source) of keystone to upgrade to, to be in /opt/keystone/ | 16:53 |
| lbragstad | or it uses the keystone_dev_local_directory variable if it is defined | 16:54 |
| *** derekh has quit IRC | 16:54 | |
| *** ihrachys is now known as ihrachys|afk | 16:54 | |
| fungi | i'll look in the tox.ini for openstack-ansible-os_keystone to see how that checks out keystone (if it even does) | 16:54 |
| *** ihrachys|afk is now known as ihrachys | 16:55 | |
| lbragstad | fungi: this might help - https://github.com/openstack/openstack-ansible-os_keystone/blob/master/tests/tests-repo-clone.sh | 16:55 |
| fungi | yep, just wound up following the breadcrumb trail to there myself | 16:56 |
| fungi | that gets called as part of the upgrade env in the tox.ini | 16:56 |
| lbragstad | yep | 16:56 |
| *** LindaWang has joined #openstack-infra | 16:57 | |
| lbragstad | and test-keystone-upgrade.sh coordinates the rolling upgrade between the containers | 16:57 |
| fungi | looks like it clones openstack/openstack-ansible-tests but not keystone as far as i can tell | 16:57 |
| *** LindaWang has quit IRC | 16:57 | |
| lbragstad | fungi: so - we were hoping to leverage jjb or infra in general to clone keystone + the patch in review | 16:57 |
| yolanda | hi infra-cores, can i get a fast review? https://review.openstack.org/#/c/471420/ | 16:57 |
| lbragstad | then we could just point openstack-ansible-os_keystone to the location of that directory | 16:58 |
| lbragstad | and have it install from there | 16:58 |
| *** baoli has quit IRC | 16:58 | |
| lbragstad | i asked a similar question here - https://review.openstack.org/#/c/432449/8/tests/test-clone-repo.yml | 16:58 |
| fungi | lbragstad: got it. so you could add a builder which just uses zuul-cloner to create a checkout of openstack/keystone wherever the upgrade script expects it | 16:59 |
| *** baoli has joined #openstack-infra | 16:59 | |
| lbragstad | fungi: including the change in review? | 16:59 |
| *** felipemonteiro has joined #openstack-infra | 16:59 | |
| fungi | that's the point of using zuul-cloner. it knows to look at the $ZUUL_.* envvars to figure out what should be checked out | 17:00 |
| clarkb | pabelanger: ya so reading more of the ansible launcher code I think what is likely happening is even with the delay in the launch worker, if we accumulate enough long running jobs the difference in a 1 minute timeout against 1.5 hour jobs isn't that significant so we keep launching jobs. Then if we are unlucking and happen to get more long running jobs than the other launchers we can end up in that | 17:01 |
| clarkb | state | 17:01 |
| *** jamesmcarthur has quit IRC | 17:02 | |
| pabelanger | clarkb: fungi: jeblair: do you mind a review of https://review.openstack.org/#/c/471418/ creates zuul-executor hiera group to share data across our new servers | 17:02 |
| *** zz_dimtruck is now known as dimtruck | 17:02 | |
| lbragstad | fungi: awesome - checking that out quick | 17:03 |
| clarkb | s/unlucking/unlucky/ thats a weird typo | 17:03 |
| fungi | lbragstad: by default (without a clonemap) it creates a clone of the specified project with any proposed changes relevant to the job build in a directory named identically to the repository under the current working directory, so you could roughly `cd /opt; /usr/zuul-env/bin/zuul-cloner --cache-dir /opt/git git://git.openstack.org openstack/keystone` and it would create /opt/openstack/keystone | 17:03 |
| fungi | lbragstad: if you wanted that to be /opt/keystone instead, then you need a clonemap file specifying a dest matching the directory you want created | 17:04 |
| lbragstad | fungi: cool - trying that quick and i'll propose another patch | 17:04 |
| fungi | you could omit the cd and in the clonemap use " dest: /opt/keystone" or whatever | 17:04 |
| *** jamielennox|away is now known as jamielennox | 17:05 | |
| pabelanger | clarkb: that seems to make sense, I did see a lot of tripleo jobs running on that launcher too. They tend to be longer running then most | 17:05 |
| lbragstad | fungi: whenever i use clonemap - do i need https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ara.yaml#L26-L32 ? | 17:06 |
| *** dtantsur is now known as dtantsur|afk | 17:07 | |
| fungi | lbragstad: no, in that case it's specific to the constraints copying, to make sure that the temporary clone the upper-constraints.txt file is being obtained from gets properly cleaned up before the job proceeds | 17:07 |
| inc0 | hey, I need a refresher, sorry, so, 1. I need to create brand new kolla job (experimental now and cron later) to build ubuntu-source and copy it to temporary space | 17:08 |
| fungi | and also cleans up the clonemap file while at it | 17:08 |
| inc0 | 2. I create kolla-ansible experimental job for every scenerio I want that will pull from it and test it | 17:08 |
| *** kgiusti has quit IRC | 17:08 | |
| fungi | lbragstad: as long as your job won't care about the existence of whatever file you put your clonemap in, it's not necessary to delete it | 17:08 |
| inc0 | 3. create new job to publish it to dockerhub when all tests passed | 17:09 |
| inc0 | but since I can't pipeline in current zuul, last one will have to be stub? | 17:09 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 17:09 |
| lbragstad | fungi: ok - i can't imagine the existing of that file causing an issue from a keystone perspective | 17:09 |
| lbragstad | fungi: ^ i took another stab at it | 17:09 |
| *** jlvillal is now known as jlvacation | 17:10 | |
| fungi | inc0: not sure what you mean by "pipeline" in this case. it has multiple meanings in different contexts (zuul uses the term for a configuration construct around change queues, but jenkins started using the term to mean passing files between jobs) | 17:11 |
| openstackgerrit | Merged openstack-infra/project-config master: Rename overcloud-sec-hard-full to overcloud-hardened-full https://review.openstack.org/471420 | 17:11 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 17:11 |
| inc0 | fungi: well, passing file between jobs would work | 17:11 |
| fungi | lbragstad: i think we may have also added that cleanup in the generic zuul-cloner using macros because they also get run on long-lived job runners where we want to avoid leaving lots of cruft in /tmp | 17:12 |
| lbragstad | fungi: oh - that makes sense | 17:12 |
| inc0 | what I need is: step 1. build images, 2. use these built images to run multiple gate jobs in pararell, 3. when and if all of previous succeed, run publisher job | 17:12 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 17:13 |
| openstackgerrit | Merged openstack-infra/system-config master: Create zuul-executor hiera group https://review.openstack.org/471418 | 17:13 |
| inc0 | you know what? I'll start etherpad for notes on this | 17:13 |
| inc0 | https://etherpad.openstack.org/p/kolla-publisher-job-design | 17:14 |
| *** rbrndt has quit IRC | 17:14 | |
| fungi | inc0: the plan as i understood it was to run a job on a single-use node which updates a proposed image with a known location on tarballs.o.o, then run a series of jobs on single-use nodes which retrieve and use that image, and then finally run a job on a trusted long-lived worker which downloads that image and reuploads it to tarballs.o.o replacing a different file at another known location used for | 17:14 |
| fungi | whatever image has most recently passed (and could also upload to dockerhub) | 17:14 |
| inc0 | fungi, pabelanger any chances I could borrow some of your time this week for quick hangout or discussion here to create plan for it? | 17:15 |
| fungi | inc0: if by hangout you mean irc, i can probably make time. i don't use proprietary communication frameworks though (postal delivery and telephones notwithstanding) | 17:16 |
| pabelanger | job dependencies in zuulv3 (dag) would also be able to do this too | 17:16 |
| pabelanger | 1 job builds, next job uses artifact | 17:16 |
| inc0 | fungi: sure, irc works | 17:16 |
| pabelanger | but we still need to work on artifact storage piece. I like the idea that clarkb had about a cinder volume | 17:16 |
| inc0 | what time would work best for you? | 17:16 |
| *** sambetts is now known as sambetts|afk | 17:16 | |
| fungi | yeah, the zuul v3 dag would make the process faster because the multiple jobs in the middle could run in parallel rather than having to be sequenced | 17:16 |
| inc0 | yeah it's optimization we can add later | 17:17 |
| inc0 | right now it's 3 jobs really anyway | 17:17 |
| fungi | inc0: i don't have any meetings on thursday so that may be the best bet (i'm free any time) | 17:17 |
| clarkb | pabelanger: problem is we don't have mutliple readers/attachments in openstack yet :/ | 17:17 |
| inc0 | cool, I'll ping you then, thank you | 17:17 |
| inc0 | clarkb: do we have swift in nodepools? | 17:18 |
| pabelanger | clarkb: ya :( | 17:18 |
| *** hamzy has joined #openstack-infra | 17:19 | |
| *** eharney has joined #openstack-infra | 17:19 | |
| fungi | inc0: zuul 2.x has a swift publisher which i think is getting dropped (at least initially) in v3, but that doesn't really solve anything for this case anyway. the problem is not that we lack somewhere to stick a file, but that we lack a mechanism for one job to pass along information to another job about details like where a file got put | 17:20 |
| inc0 | I was referring to multiple readers | 17:20 |
| clarkb | inc0: yes, though we've found it to be hard to use for this use case. Because it requires maintain external indexes which require effort | 17:20 |
| *** rkukura has joined #openstack-infra | 17:21 | |
| fungi | and if you're maintaining an external index anyway, can just as easily stick apache in front of a file tree and copy things in the same way we store logs or other sorts of persistent artifacts | 17:21 |
| pabelanger | fungi: ya, that job handshake would be interesting to work on | 17:21 |
| *** baoli has quit IRC | 17:21 | |
| *** baoli has joined #openstack-infra | 17:22 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Put variables into the inventory https://review.openstack.org/467635 | 17:22 |
| *** jamesmcarthur has joined #openstack-infra | 17:23 | |
| fungi | also, jobs creating ephemeral artifacts have a retention/expiration issue to deal with. we need to remove files once they're no longer needed but avoid removing them before all the jobs which might use them finish | 17:23 |
| fungi | we work around that problem right now by just replacing files with new files using te same path/filename | 17:23 |
| fungi | though that in turn makes for atomicity challenges and races if you might have things using that file and expecting it to remain consistent for any particular period of time | 17:24 |
| *** sree has joined #openstack-infra | 17:25 | |
| *** dizquierdo_ has quit IRC | 17:26 | |
| fungi | or multiple jobs updating that same file which could run in parallel and finish in any order | 17:26 |
| *** smarcet has joined #openstack-infra | 17:27 | |
| fungi | (e.g., we have times where a branch tarball might end up built from not-the-latest commit because several commits merged and triggered a branch tarball job at the same time and the job for the last commit completes sooner than one of the earlier commits) | 17:28 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use the executor cached repos more often https://review.openstack.org/467777 | 17:29 |
| *** sree has quit IRC | 17:29 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Check out the appropriate branch in executor https://review.openstack.org/467778 | 17:30 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Enable test_one_branch cloner test https://review.openstack.org/467779 | 17:30 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Release executor jobs on test exit https://review.openstack.org/468553 | 17:31 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add LoggerAdapter for executor jobs https://review.openstack.org/468554 | 17:31 |
| *** eharney has quit IRC | 17:32 | |
| *** dhinesh has joined #openstack-infra | 17:35 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Make multinode gates regular https://review.openstack.org/471428 | 17:36 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Make multinode gates regular https://review.openstack.org/471428 | 17:38 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Make multinode gates regular https://review.openstack.org/471428 | 17:39 |
| *** xyang1 has joined #openstack-infra | 17:39 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Update the merger recent dict when saving the repo state https://review.openstack.org/469595 | 17:40 |
| *** nicolasbock has quit IRC | 17:41 | |
| *** nicolasbock has joined #openstack-infra | 17:42 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Actually check out the branch in the jobdir https://review.openstack.org/468556 | 17:42 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Enable test_multi_branch cloner/executor test https://review.openstack.org/468557 | 17:42 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added new filter params for members API * group_id ( operators : ==) * group_slug ( operators : == , =@) https://review.openstack.org/471429 | 17:44 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added new filter params for members API * group_id ( operators : ==) * group_slug ( operators : == , =@) https://review.openstack.org/471429 | 17:46 |
| *** baoli has quit IRC | 17:58 | |
| *** smarcet has quit IRC | 18:00 | |
| *** mriedem1 has joined #openstack-infra | 18:00 | |
| clarkb | I've just gone through and tried to review specs this morning. So if you have a spec up that isn't otherwise -1'd or WIPd you likely have comments from me | 18:01 |
| clarkb | pabelanger: fungi cmurphy mrmartin ^ | 18:01 |
| clarkb | I skipped monty's ingestors spec though | 18:01 |
| *** trown|lunch is now known as trown | 18:02 | |
| cmurphy | thanks clarkb | 18:03 |
| *** mriedem1 has quit IRC | 18:03 | |
| *** mriedem has quit IRC | 18:03 | |
| *** mriedem1 has joined #openstack-infra | 18:03 | |
| *** mriedem1 is now known as mriedem | 18:03 | |
| clarkb | sdague: I updated https://review.openstack.org/#/c/469642/3 to use the header size configured in lib/swift and applies the apache default to all other services | 18:04 |
| sdague | clarkb: +2 | 18:06 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Handle job or build is None in formatUrlPattern https://review.openstack.org/470415 | 18:07 |
| fungi | clarkb: i saw--thanks! awe inspiring | 18:07 |
| fungi | clarkb: if you like specs review, you'll love today's meeting agenda ;) | 18:07 |
| clarkb | fungi: taskwarrior reminding me this morning that I was overdue to review specs so uh they all gone done at once :) | 18:08 |
| *** baoli has joined #openstack-infra | 18:08 | |
| clarkb | cmurphy: ^ I'm liking the tool so far, but still trying to sort out priority and urgency | 18:08 |
| bkero | taskwarrior++ | 18:08 |
| clarkb | fungi: you beat me to the spam bug | 18:09 |
| fungi | i closed down burrow's bug tracker and made it more obvious it was abandoned too | 18:10 |
| cmurphy | clarkb: yeah i like it in general, my latest pet peeve is when it chides me for completing less-urgent tasks before top-urgent tasks | 18:11 |
| fungi | that would annoy me. i use less-urgent tasks to procrastinate | 18:11 |
| *** pvaneck has joined #openstack-infra | 18:12 | |
| *** gardlt has joined #openstack-infra | 18:13 | |
| *** rbrndt has joined #openstack-infra | 18:13 | |
| pabelanger | cmurphy: thanks will make changes shortly | 18:14 |
| cmurphy | clarkb: probably meant for you ^ | 18:14 |
| pvaneck | hey, infra team. I have a couple puppet-refstack patches that have been sitting a while: https://review.openstack.org/#/c/429117/ and https://review.openstack.org/#/c/460746/ | 18:16 |
| pvaneck | would be great if someone could take a look :) | 18:16 |
| pabelanger | clarkb: fungi: I am not sure I'll be at todays meeting, currently #dadops this afternoon while wife is running errands | 18:17 |
| fungi | pabelanger: you'll miss all the fun, but duty calls! | 18:17 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Removed queries for bugs 1349617 and 1668958 https://review.openstack.org/442033 | 18:18 |
| openstack | bug 1668958 in neutron "metadata service occasionally not returning keys" [Critical,Fix released] https://launchpad.net/bugs/1668958 - Assigned to Kevin Benton (kevinbenton) | 18:18 |
| openstack | bug 1668958 in neutron "duplicate for #1349617 metadata service occasionally not returning keys" [Critical,Fix released] https://launchpad.net/bugs/1668958 - Assigned to Kevin Benton (kevinbenton) | 18:18 |
| *** baoli has quit IRC | 18:23 | |
| *** baoli has joined #openstack-infra | 18:24 | |
| *** kaisers has quit IRC | 18:27 | |
| *** yamamoto has joined #openstack-infra | 18:29 | |
| *** clenimar has quit IRC | 18:29 | |
| *** yamamoto has quit IRC | 18:34 | |
| *** rossella_ has joined #openstack-infra | 18:37 | |
| *** jamesmcarthur has quit IRC | 18:37 | |
| *** jamesmcarthur has joined #openstack-infra | 18:38 | |
| openstackgerrit | Merged openstack-infra/shade master: Generalize version discovery for re-use https://review.openstack.org/470770 | 18:42 |
| openstackgerrit | Merged openstack-infra/shade master: Migrate dns to new discovery method https://review.openstack.org/470993 | 18:42 |
| openstackgerrit | Merged openstack-infra/shade master: Use shade discovery for keystone https://review.openstack.org/471404 | 18:42 |
| openstackgerrit | Merged openstack-infra/shade master: Avoid keystoneclient making yet another discovery call https://review.openstack.org/471405 | 18:45 |
| *** panda|bbl is now known as panda | 18:46 | |
| *** mordred has quit IRC | 18:47 | |
| *** mordred has joined #openstack-infra | 18:48 | |
| *** sbezverk has quit IRC | 18:52 | |
| *** kaisers has joined #openstack-infra | 18:53 | |
| *** eharney has joined #openstack-infra | 18:54 | |
| *** thorst_afk has quit IRC | 18:54 | |
| *** rockyg has joined #openstack-infra | 18:54 | |
| *** yee37921 has joined #openstack-infra | 18:56 | |
| *** pcaruana has joined #openstack-infra | 18:56 | |
| *** thorst_afk has joined #openstack-infra | 18:56 | |
| *** yee379 has quit IRC | 19:00 | |
| fungi | it's that (weekly infra team meeting) time again! find us in #openstack-meeting for the next hour | 19:00 |
| *** thorst_afk has quit IRC | 19:00 | |
| *** salv-orlando has joined #openstack-infra | 19:03 | |
| *** jkilpatr has quit IRC | 19:03 | |
| *** felipemonteiro_ has joined #openstack-infra | 19:07 | |
| *** hashar has joined #openstack-infra | 19:07 | |
| openstackgerrit | Merged openstack-infra/infra-specs master: Add a help-wanted section https://review.openstack.org/471149 | 19:10 |
| *** felipemonteiro has quit IRC | 19:10 | |
| *** gouthamr has quit IRC | 19:12 | |
| *** Qiming has quit IRC | 19:13 | |
| *** thorst_afk has joined #openstack-infra | 19:14 | |
| *** thorst_afk has quit IRC | 19:15 | |
| *** thorst_afk has joined #openstack-infra | 19:16 | |
| *** Qiming has joined #openstack-infra | 19:17 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Set worker_private_key_file for zuul-executor https://review.openstack.org/471448 | 19:20 |
| chandankumar | ianw: hello | 19:21 |
| chandankumar | ianw: please have a look on this https://review.openstack.org/470286 thanks :-) | 19:22 |
| *** trown is now known as trown|outtypewww | 19:22 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Put variables into the inventory https://review.openstack.org/467635 | 19:25 |
| *** amoralej is now known as amoralej|off | 19:26 | |
| pabelanger | #status log both zuulv3.o.o and ze01.o.o are online and under puppet cfgmgmt | 19:27 |
| openstackstatus | pabelanger: finished logging | 19:27 |
| pabelanger | jeblair: ^ | 19:28 |
| *** harlowja has quit IRC | 19:29 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Allow ze01.o.o access to zookeeper / add to cacti.o.o https://review.openstack.org/471450 | 19:30 |
| *** jamesmcarthur has quit IRC | 19:31 | |
| *** yamamoto has joined #openstack-infra | 19:31 | |
| *** eharney has quit IRC | 19:33 | |
| *** jamesmcarthur has joined #openstack-infra | 19:33 | |
| *** pcaruana has quit IRC | 19:33 | |
| *** yamamoto has quit IRC | 19:36 | |
| mordred | jeblair: did WIP land in 2.14??? https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#work-in-progress-input | 19:37 |
| fungi | mordred: i assume that's the one david linked in his post to the infra ml seeking input | 19:38 |
| mordred | fungi: I believe it is - and I believe it did. https://gerrit-review.googlesource.com/Documentation/rest-api-changes.html#set-work-in-pogress | 19:39 |
| mordred | it's totally just listed in the main docs | 19:40 |
| *** vhosakot has quit IRC | 19:41 | |
| *** vhosakot has joined #openstack-infra | 19:41 | |
| fungi | excellent! (and wow that will be some work to adapt to) | 19:45 |
| fungi | but excellent! | 19:45 |
| *** vhosakot has quit IRC | 19:46 | |
| *** mdrabe has quit IRC | 19:46 | |
| mordred | fungi: I'm thrilled that we'll finally be able to add WIP support to git-review! :) | 19:47 |
| *** eharney has joined #openstack-infra | 19:47 | |
| fungi | me^2 | 19:49 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul master: Move webui bits into zuul::webui https://review.openstack.org/471456 | 19:49 |
| *** Goneri has quit IRC | 19:50 | |
| *** harlowja has joined #openstack-infra | 19:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Disable webui for zuul-executors https://review.openstack.org/471458 | 19:50 |
| *** ericjuma has quit IRC | 19:51 | |
| rm_work | any Flake8 experts around? :P | 19:52 |
| *** sbezverk has joined #openstack-infra | 19:54 | |
| *** felipemonteiro has joined #openstack-infra | 19:55 | |
| *** vhosakot has joined #openstack-infra | 19:56 | |
| *** neil__ has quit IRC | 19:56 | |
| *** mdrabe has joined #openstack-infra | 19:56 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Put variables into the inventory https://review.openstack.org/467635 | 19:57 |
| *** felipemonteiro_ has quit IRC | 19:59 | |
| openstackgerrit | Chandan Kumar proposed openstack-infra/project-config master: Switch to bash script for getting tempest plugin list https://review.openstack.org/470286 | 20:00 |
| clarkb | pabelanger: dirk speaking of mirrors we may consider a wheel mirror for suse once jobs are running in more places | 20:00 |
| * clarkb finds lunch as he did not get a sandwich before the meeting | 20:00 | |
| pabelanger | clarkb: agree | 20:01 |
| fungi | rm_work: whatcha need? i won't claim to be an expert, but i use it some | 20:01 |
| rm_work | fungi: i'm trying to find lines that use backslash continuation | 20:01 |
| rm_work | but logical_line just ... gives me the whole assembled line (lol, ok, makes sense) | 20:02 |
| rm_work | but *physical_line* gives me ... | 20:02 |
| dirk | clarkb: what needs to be done for that? I have not a lot of clue about wheel's yet | 20:02 |
| rm_work | the second line? but just skips the first | 20:02 |
| rm_work | so like I have: | 20:02 |
| rm_work | import \ | 20:02 |
| rm_work | os | 20:02 |
| rm_work | physical_line first gets run on " os" | 20:02 |
| rm_work | and i never see the import show up anywhere | 20:02 |
| clarkb | dirk: we make a static suse zuul worker that runs our build wheels job and publishes to afs. Its mostly infra related tasksto make server and afs volume | 20:03 |
| dirk | clarkb: can I test that it would actually successfully build all wheels we normally build somehow first? | 20:05 |
| fungi | rm_work: looks like this pr added it as a default-disabled check in pep8: https://github.com/PyCQA/pycodestyle/pull/64 | 20:05 |
| fungi | might be worthwhile to first see if it still exists | 20:05 |
| *** shardy has quit IRC | 20:05 | |
| clarkb | dirk: should be able to, I think you install the bindep file and just run pip wherl against requirements.txt | 20:05 |
| rm_work | interesting, good find fungi | 20:06 |
| rm_work | i did some googling but didn't manage to find an existing one | 20:06 |
| clarkb | dirk: it should also gracefully handle errors and publish as much as it can | 20:06 |
| *** salv-orl_ has joined #openstack-infra | 20:06 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added new relation all afilliations to Member API https://review.openstack.org/471470 | 20:06 |
| *** armax has quit IRC | 20:07 | |
| *** priteau has quit IRC | 20:07 | |
| *** florianf has quit IRC | 20:07 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for rebuild vif plugging timeout bug 1694371 https://review.openstack.org/471471 | 20:07 |
| openstack | bug 1694371 in OpenStack Compute (nova) "Timeout while waiting for network-vif-plugged event during server rebuild" [Undecided,Confirmed] https://launchpad.net/bugs/1694371 | 20:07 |
| dirk | clarkb: ah, ok, so this isn't being tested as a check job somewhere already where I can post a review and wait for things to happen? :) | 20:07 |
| * dirk hates it when he has to do all the work | 20:07 | |
| rm_work | fungi: wait, where are you seeing that? I don't see anything about backslash continuation in there | 20:08 |
| *** salv-orlando has quit IRC | 20:08 | |
| dirk | clarkb: pabelanger : speaking of AFS, there isn't anything needing to be done for https://review.openstack.org/#/c/468130/ other than reviewing right? it still mirrors into the very same AFS volume that already exists? | 20:08 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added new relation all afilliations to Member API https://review.openstack.org/471470 | 20:08 |
| clarkb | dirk: oh ya making afs work on suse will be the other thing | 20:08 |
| clarkb | I'm not sure about leap but tumbleweed was devoid of afs last I checked | 20:09 |
| pabelanger | dirk: no, I should be able to monitor that this wewk | 20:12 |
| pabelanger | week* | 20:12 |
| *** rossella_ has quit IRC | 20:13 | |
| dirk | clarkb: I'm pretty sure that leap is worse in supporting AFS if tumbleweed doesn't do it | 20:13 |
| ianw | clarkb: we have the template out there of the periodic centos job building the dkms modules ... of course if they work is the question i guess | 20:13 |
| dirk | clarkb: any more details or pointers? I have never used AFS so far | 20:14 |
| clarkb | dirk: we have a document let me find it | 20:14 |
| clarkb | https://docs.openstack.org/infra/system-config/afs.html | 20:14 |
| fungi | rm_work: oh, sorry, that was just for redundant/unnecessary backslash continuations. skimming poorly while trying to attend the tc meeting | 20:15 |
| clarkb | has client info for debuntu, you can get read only access from any client which should be sufficient for testing | 20:15 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added mising ref to Member Class https://review.openstack.org/471473 | 20:16 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added mising ref to Member Class https://review.openstack.org/471473 | 20:17 |
| dirk | clarkb: would openafs 1.6.20 be a good version to try? | 20:17 |
| clarkb | 1.6.7 is what is on trusty which is what our servers run so I think 1.6.20 should be fine | 20:19 |
| rm_work | fungi: lol no worries | 20:20 |
| *** andreww has joined #openstack-infra | 20:21 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul master: Move webui bits into zuul::webui https://review.openstack.org/471456 | 20:21 |
| *** andreww has quit IRC | 20:21 | |
| *** xarses has quit IRC | 20:22 | |
| *** andreww has joined #openstack-infra | 20:22 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: First Pass: De-client-ify Users https://review.openstack.org/467784 | 20:22 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Add links to user list dict https://review.openstack.org/471474 | 20:22 |
| *** kgiusti has joined #openstack-infra | 20:22 | |
| *** dtantsur|afk has quit IRC | 20:23 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: De-client-ify User Ops https://review.openstack.org/467784 | 20:24 |
| *** abishop has joined #openstack-infra | 20:25 | |
| *** dtantsur has joined #openstack-infra | 20:26 | |
| *** Apoorva_ has joined #openstack-infra | 20:32 | |
| *** yamamoto has joined #openstack-infra | 20:32 | |
| openstackgerrit | Pete Birley proposed openstack-infra/project-config master: Add Multinode gate for OpenStack-Helm project https://review.openstack.org/471476 | 20:32 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Extend in-repo config update support to github [wip] https://review.openstack.org/471477 | 20:32 |
| *** eranrom has joined #openstack-infra | 20:33 | |
| *** jamielennox has quit IRC | 20:34 | |
| *** rocky_g has joined #openstack-infra | 20:34 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul master: Move webui bits into zuul::webui https://review.openstack.org/471456 | 20:34 |
| *** Apoorva has quit IRC | 20:34 | |
| *** eranrom has quit IRC | 20:34 | |
| *** eranrom has joined #openstack-infra | 20:35 | |
| *** nicolasbock has quit IRC | 20:35 | |
| *** yamamoto has quit IRC | 20:37 | |
| *** rockyg has quit IRC | 20:37 | |
| *** jkilpatr has joined #openstack-infra | 20:38 | |
| *** jamielennox has joined #openstack-infra | 20:38 | |
| *** gouthamr has joined #openstack-infra | 20:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul master: Move webui bits into zuul::webui https://review.openstack.org/471456 | 20:43 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Use a select loop on log streamer socket reads https://review.openstack.org/471480 | 20:45 |
| *** bh526r has quit IRC | 20:46 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Use new env vars for tripleo yaml build https://review.openstack.org/471481 | 20:46 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Use a select loop on log streamer socket reads https://review.openstack.org/471480 | 20:48 |
| *** armax has joined #openstack-infra | 20:48 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 20:50 |
| *** rockyg has joined #openstack-infra | 20:51 | |
| *** jcoufal has quit IRC | 20:51 | |
| *** rocky_g has quit IRC | 20:52 | |
| *** rocky_g has joined #openstack-infra | 20:53 | |
| *** rocky_g has quit IRC | 20:53 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 20:54 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 20:57 |
| lbragstad | fungi: i took another shot at add that to keystone experimental queue, thanks for the help! | 20:58 |
| *** Apoorva_ has quit IRC | 21:04 | |
| *** Apoorva has joined #openstack-infra | 21:05 | |
| *** Apoorva_ has joined #openstack-infra | 21:05 | |
| *** dave-mccowan has quit IRC | 21:06 | |
| *** abishop has quit IRC | 21:08 | |
| *** Apoorva has quit IRC | 21:09 | |
| *** rockyg has quit IRC | 21:10 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Start at block-device mounting/unmounting cleanup https://review.openstack.org/471183 | 21:13 |
| *** ldnunes has quit IRC | 21:13 | |
| *** Julien-zte has quit IRC | 21:19 | |
| *** thorst_afk has quit IRC | 21:19 | |
| *** thorst_afk has joined #openstack-infra | 21:20 | |
| *** Julien-zte has joined #openstack-infra | 21:20 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use a select loop on log streamer socket reads https://review.openstack.org/471480 | 21:20 |
| *** rloo has joined #openstack-infra | 21:22 | |
| *** thorst_afk has quit IRC | 21:24 | |
| *** sree has joined #openstack-infra | 21:25 | |
| *** mat128 has quit IRC | 21:26 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Replace equals None with is None https://review.openstack.org/471075 | 21:29 |
| *** sree has quit IRC | 21:29 | |
| *** yamamoto has joined #openstack-infra | 21:33 | |
| *** yamamoto has quit IRC | 21:37 | |
| *** Apoorva_ has quit IRC | 21:38 | |
| *** Apoorva has joined #openstack-infra | 21:39 | |
| *** gouthamr has quit IRC | 21:40 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Make multinode gates regular https://review.openstack.org/471428 | 21:40 |
| *** baoli has quit IRC | 21:41 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Factor out duplicate code in executor repo tests https://review.openstack.org/471492 | 21:42 |
| *** mdrabe has quit IRC | 21:44 | |
| *** hashar has quit IRC | 21:45 | |
| *** gordc has quit IRC | 21:45 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Verify executor branch checkout https://review.openstack.org/468564 | 21:47 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Enable test_project_override https://review.openstack.org/470049 | 21:48 |
| weshay | pabelanger, hey Paul we're in the process of preparing rdo-cloud to be a proper openstack hosted cloud. We're going to add you to some of the reviews involved if you don't mind :) | 21:50 |
| mriedem | clarkb: going back to https://review.openstack.org/#/c/469247/ are we indexing subnode logs yet? | 21:50 |
| mriedem | because ^ only hits in a calico voting job in the periodic queue | 21:50 |
| jeblair | clarkb, fungi, mordred, zaro_: https://review.openstack.org/469946 is a really interesting zuul + gerrit 2.13 thing. | 21:51 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Make multinode gates regular https://review.openstack.org/471428 | 21:51 |
| clarkb | mriedem: yes I think we are. So its likely that etcd issue was restricted to non voting/fixed by the general fixes around that | 21:51 |
| clarkb | mriedem: but now we can check :) | 21:51 |
| mriedem | ok so we can revert that query now right? | 21:52 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Enable test_periodic https://review.openstack.org/470053 | 21:52 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Enable test_post_checkout https://review.openstack.org/470381 | 21:53 |
| clarkb | mriedem: yup | 21:53 |
| pabelanger | weshay: is this for any jobs to run on? | 21:54 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Enable test_post_and_master_checkout https://review.openstack.org/470385 | 21:55 |
| weshay | pabelanger, to be honest I have not heard a definitive answer to that question. I think the first pass is to make it an equiv to rh1 | 21:55 |
| pabelanger | weshay: so, if this is a rh1 replacement, we should be looking at adding this as 3rdparty CI now. This would be the perfect time to do it | 21:56 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Remove query for bug 1696098 https://review.openstack.org/471496 | 21:56 |
| openstack | bug 1696125 in OpenStack Compute (nova) "duplicate for #1696098 Detach interface failed - Unable to detach from guest transient domain (pike)" [High,In progress] https://launchpad.net/bugs/1696125 - Assigned to Matt Riedemann (mriedem) | 21:56 |
| *** gardlt has quit IRC | 21:56 | |
| pabelanger | weshay: otherwise, if any jobs runs on this, we'll need to properly provision the cloud into our infrastructure | 21:56 |
| *** gomarivera has quit IRC | 21:57 | |
| *** wolverineav has quit IRC | 21:57 | |
| *** gomarivera has joined #openstack-infra | 21:58 | |
| *** wolverineav has joined #openstack-infra | 21:58 | |
| weshay | pabelanger, ok let me talk to some folks. Would you have the bandwith to join the tripleo-ci sync meetings on thrs for the first 10 or so minutes for the next couple weeks as we iron this stuff out? | 21:58 |
| *** wolverineav has quit IRC | 21:58 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix passing labels to Gerrit when they are not defined in All-Projects https://review.openstack.org/465420 | 21:58 |
| *** wolverineav has joined #openstack-infra | 21:58 | |
| pabelanger | weshay: yes, when is your meetings | 21:58 |
| *** dimtruck is now known as zz_dimtruck | 21:59 | |
| weshay | thrs.. 2:30pm utc | 21:59 |
| weshay | I can add you | 21:59 |
| pabelanger | okay | 21:59 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Revert "Add query for etcd failing to start" https://review.openstack.org/469247 | 21:59 |
| pabelanger | clarkb: jeblair: fungi: https://review.openstack.org/#/c/471448 and https://review.openstack.org/#/c/471450 are ready for reviews for ze01.o.o fixes | 21:59 |
| *** eharney has quit IRC | 22:02 | |
| *** gomarivera has quit IRC | 22:02 | |
| *** jascott1 has quit IRC | 22:02 | |
| *** jascott1 has joined #openstack-infra | 22:03 | |
| *** jamesmcarthur has quit IRC | 22:03 | |
| *** jascott1 has quit IRC | 22:05 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Use get_discovery from keystoneauth https://review.openstack.org/471501 | 22:05 |
| mriedem | yikes looks like everything is failing with | 22:05 |
| mriedem | error: command 'x86_64-linux-gnu-gcc' failed with exit status 1 | 22:05 |
| pabelanger | cinder looks to be missing libssl-dev from bindep | 22:05 |
| pabelanger | mriedem: yes, missing libssl-dev in bindep.txt | 22:06 |
| smcginnis | Are we enforcing something with bindep now? | 22:07 |
| mriedem | i saw it fail in a elastic-recheck pep8 job though | 22:07 |
| mriedem | which wouldn't be using cinder | 22:07 |
| mriedem | and a neutron pep8 job | 22:07 |
| pabelanger | pep8 jobs don't use wheels, so if they have to compile something they will need libraries in bindep.txt | 22:07 |
| *** bobh_ has quit IRC | 22:08 | |
| pabelanger | I think cinder just added a bindep.txt file a few days ago? | 22:08 |
| smcginnis | I thought it landed longer ago than that. | 22:08 |
| smcginnis | I can update with libssl-dev shortly though. | 22:08 |
| mriedem | why would this just start exploding now? | 22:09 |
| smcginnis | Any idea if that's the same package name across distros? | 22:09 |
| clarkb | didn't depend on cryptography before? | 22:09 |
| pabelanger | it is needed for cryptography | 22:09 |
| pabelanger | is that a new dependencies | 22:09 |
| pabelanger | smcginnis: different on platforms | 22:09 |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack-infra/project-config master: Make multinode gates regular https://review.openstack.org/471428 | 22:09 |
| smcginnis | Fairly new, but still in there longer than a day or two. | 22:09 |
| smcginnis | OK, so openssl-dev for rpm, libssl-dev for dpkg? | 22:11 |
| mriedem | neutron doesn't depend on cryptography, at least not directly | 22:11 |
| mriedem | seems it's just getting pulled in http://logs.openstack.org/94/471494/1/check/gate-neutron-pep8-ubuntu-xenial/5bfe3ee/console.html#_2017-06-06_21_49_20_684816 | 22:11 |
| pabelanger | smcginnis: yes that looks right | 22:13 |
| clarkb | smcginnis: openssl-devel on rpm | 22:13 |
| openstackgerrit | Lance Bragstad proposed openstack-infra/project-config master: Propose a job for keystone rolling upgrades https://review.openstack.org/471419 | 22:13 |
| smcginnis | pabelanger, clarkb: Thanks | 22:13 |
| clarkb | its actually libopenssl-devel on suse | 22:13 |
| smcginnis | OK, I'll add that too. | 22:13 |
| smcginnis | A lot of bindep.txt files I find on codesearch.o.o do not include all platforms. | 22:14 |
| pabelanger | right, most just default to ubuntu | 22:14 |
| pabelanger | we don't have a good way of forcing it yet | 22:14 |
| pabelanger | best we do today is run pep8 with no wheels to force jobs to compile | 22:15 |
| smcginnis | pabelanger: Guess we would probably have to have tests on each platform. | 22:15 |
| pabelanger | that is why pep8 mostly failing | 22:15 |
| clarkb | mriedem: have you run sdague's what broke script to find what broke? | 22:15 |
| mriedem | doing it now | 22:16 |
| *** wolverineav has quit IRC | 22:16 | |
| *** wolverineav has joined #openstack-infra | 22:16 | |
| *** sdague has quit IRC | 22:18 | |
| pabelanger | https://review.openstack.org/#/c/469026/ bumped cryptography to 1.9 | 22:18 |
| pabelanger | 7 days ago however | 22:18 |
| clarkb | pabelanger: and anyone already explicitly depending on constraints would have it in their bindep.txt | 22:19 |
| pabelanger | I wonder | 22:19 |
| clarkb | I don't think it was a requirements update that did it because then we'd catch it in the requirements update change to the project(s) | 22:19 |
| pabelanger | checking something | 22:19 |
| clarkb | possible that we were installing it in our base images and now are not? | 22:20 |
| pabelanger | yes, that is excatly it | 22:20 |
| pabelanger | I think ssl-dev was a dependency of npm | 22:20 |
| pabelanger | new images came online 3 hours go | 22:21 |
| pabelanger | they will be missing libssl-dev | 22:21 |
| *** wolverineav has quit IRC | 22:21 | |
| smcginnis | Cinder patch: https://review.openstack.org/#/c/471502 | 22:21 |
| *** Julien-zte has quit IRC | 22:21 | |
| clarkb | nodejs depended on libssl1.0.0 on ubuntu xenial, but not -devel. Maybe cryptography doesn't need the dev package if that lib is there? | 22:22 |
| pabelanger | we still have libssl getting installed | 22:23 |
| pabelanger | just not -dev | 22:23 |
| smcginnis | clarkb: I think -devel is actually needed . | 22:23 |
| smcginnis | Looks for a header file, IIRC. | 22:23 |
| *** Julien-zte has joined #openstack-infra | 22:23 | |
| pabelanger | npm / nodejs was likely pulling it in | 22:23 |
| clarkb | smcginnis: ya thats what I figured, so wondering why npm/nodejs would pull in dev package if they already pull in the lib | 22:23 |
| pabelanger | and we just removed them | 22:23 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add ssl packages to bindep https://review.openstack.org/471505 | 22:23 |
| *** jascott1 has joined #openstack-infra | 22:24 | |
| pabelanger | clarkb: I'll get a ML post going | 22:24 |
| clarkb | ok | 22:25 |
| pabelanger | bindep-fallback is not affected but projects without libssl-dev will fail | 22:25 |
| clarkb | maybe you can provide a standard set for people to just copy pasta in? | 22:25 |
| pabelanger | good idea | 22:25 |
| clarkb | smcginnis: I left a comment on your change, I don't think its worth a -1 since we don't currently test those jobs on opensuse | 22:27 |
| clarkb | but something to follow up on maybe | 22:27 |
| smcginnis | clarkb: Ah, thanks. | 22:27 |
| smcginnis | clarkb: From the examples I've seen out there, there are a lot of those that need to be cleaned up. | 22:27 |
| clarkb | ya I'm not surprised. This is the cost of no one agreeing on simple things like names :) | 22:29 |
| *** claudiub has quit IRC | 22:29 | |
| *** mat128 has joined #openstack-infra | 22:29 | |
| smcginnis | clarkb: This look right? http://paste.openstack.org/show/611565/ | 22:30 |
| pabelanger | sent | 22:30 |
| clarkb | smcginnis: yup | 22:30 |
| clarkb | smcginnis: let me test it locally really quick | 22:30 |
| *** lbragstad has quit IRC | 22:30 | |
| pabelanger | don't you want platform:redhat? I think suse will match on rpm? | 22:30 |
| smcginnis | clarkb: That would be great. | 22:31 |
| smcginnis | pabelanger: I was wondering about that. Seemed a little contradictory. | 22:31 |
| *** dhajare|busy has quit IRC | 22:31 | |
| clarkb | smcginnis: ya that seems to be happy on tumbleweed (suse) | 22:32 |
| smcginnis | So we don't need it to be suse or redhat? | 22:33 |
| clarkb | I don't think centos and fedora match on redhat? | 22:33 |
| clarkb | you can run bindep --profiles to see | 22:33 |
| *** yamamoto_ has joined #openstack-infra | 22:34 | |
| clarkb | oh centos does match redhat | 22:34 |
| clarkb | does fedora?? | 22:34 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add ssl packages to bindep https://review.openstack.org/471505 | 22:34 |
| smcginnis | clarkb: Good enough, it worked for you, I'll update to that last paste. | 22:35 |
| openstackgerrit | Merged openstack-infra/devstack-gate master: List swift and keystone in feature.yaml https://review.openstack.org/470383 | 22:39 |
| *** yamamoto_ has quit IRC | 22:40 | |
| *** felipemonteiro has quit IRC | 22:40 | |
| *** tosky has quit IRC | 22:41 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add github reporter status-url config option https://review.openstack.org/449794 | 22:42 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove test_git_cache_dir https://review.openstack.org/470389 | 22:42 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove test_upgrade https://review.openstack.org/470390 | 22:42 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove test_periodic_update https://review.openstack.org/470391 | 22:42 |
| *** rbrndt has quit IRC | 22:42 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Remove test_post_checkout https://review.openstack.org/470392 | 22:42 |
| *** gomarivera has joined #openstack-infra | 22:43 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Rename TestCloner TestExecutorRepos https://review.openstack.org/470393 | 22:43 |
| *** rbrndt has joined #openstack-infra | 22:45 | |
| *** gardlt has joined #openstack-infra | 22:45 | |
| mriedem | clarkb: pabelanger: want to approve https://review.openstack.org/#/c/471505/ ? | 22:46 |
| clarkb | mriedem: done | 22:47 |
| mriedem | thanks | 22:47 |
| *** gomarivera has quit IRC | 22:47 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Reject some untrusted config objects https://review.openstack.org/470442 | 22:48 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Factor out duplicate code in executor repo tests https://review.openstack.org/471492 | 22:48 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add ssl packages to bindep https://review.openstack.org/471505 | 22:50 |
| *** gardlt has quit IRC | 22:50 | |
| *** thorst_afk has joined #openstack-infra | 22:50 | |
| *** marst has quit IRC | 22:51 | |
| *** marst has joined #openstack-infra | 22:51 | |
| openstackgerrit | Merged openstack-infra/project-config master: Only run swift and keystone in swift func jobs https://review.openstack.org/470421 | 22:52 |
| *** rbrndt has quit IRC | 22:55 | |
| *** gouthamr has joined #openstack-infra | 22:57 | |
| *** marst has quit IRC | 22:57 | |
| *** hongbin has quit IRC | 22:59 | |
| clarkb | the number of affected projects actually doesn't seem to bad based on the zuul status page | 22:59 |
| clarkb | neutron cinder patrole e-r I've found so far | 22:59 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Factor out duplicate code in executor repo tests https://review.openstack.org/471492 | 22:59 |
| clarkb | fungi: if you are still around what needs to be done to finish that specs cleanup? | 23:01 |
| *** xarses_ has quit IRC | 23:02 | |
| *** aeng has joined #openstack-infra | 23:04 | |
| *** Julien-zte has quit IRC | 23:07 | |
| clarkb | pvaneck: re https://review.openstack.org/#/c/429117/1 why not just leave it in requirements, its optional but it requires no C deps and is licesned liberally. Basically it hurts nothing to just install it then people don't have to do anything extra to make your isntall work... | 23:08 |
| *** Julien-zte has joined #openstack-infra | 23:08 | |
| clarkb | pvaneck: I've approved the change, it just seems silly to remove it from requirements when it only helps | 23:08 |
| *** thorst_afk has quit IRC | 23:09 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Remove query for bug 1696098 https://review.openstack.org/471496 | 23:09 |
| openstack | bug 1696125 in OpenStack Compute (nova) "duplicate for #1696098 Detach interface failed - Unable to detach from guest transient domain (pike)" [High,In progress] https://launchpad.net/bugs/1696125 - Assigned to Matt Riedemann (mriedem) | 23:09 |
| pabelanger | clarkb: have time in the late day to look at 471448 and 471450. Will allow us to continue with ze01.o.o in the morning | 23:12 |
| clarkb | pabelanger: for 471448 we are putting the key in a non default location because ssh vs .ssh? | 23:14 |
| clarkb | I wonder if we shouldn't just use default location, but this is fine | 23:14 |
| clarkb | pabelanger: and https://review.openstack.org/#/c/471450/1 is access to gearman not zk right? want to just update the commit message real quick and I can approve? | 23:16 |
| *** Julien-zte has quit IRC | 23:18 | |
| openstackgerrit | Merged openstack-infra/puppet-refstack master: Ensure pymysql is installed https://review.openstack.org/429117 | 23:18 |
| *** gildub_ has joined #openstack-infra | 23:18 | |
| openstackgerrit | Merged openstack-infra/puppet-refstack master: Add exec to generate doc HTML templates https://review.openstack.org/460746 | 23:18 |
| *** xyang1 has quit IRC | 23:18 | |
| *** Julien-zte has joined #openstack-infra | 23:22 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Remove query for bug 1696098 https://review.openstack.org/471496 | 23:23 |
| openstack | bug 1696125 in OpenStack Compute (nova) "duplicate for #1696098 Detach interface failed - Unable to detach from guest transient domain (pike)" [High,In progress] https://launchpad.net/bugs/1696125 - Assigned to Matt Riedemann (mriedem) | 23:23 |
| openstackgerrit | Merged openstack-infra/system-config master: Set worker_private_key_file for zuul-executor https://review.openstack.org/471448 | 23:24 |
| *** david-lyle has quit IRC | 23:28 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for rebuild vif plugging timeout bug 1694371 https://review.openstack.org/471471 | 23:28 |
| openstack | bug 1694371 in OpenStack Compute (nova) "Timeout while waiting for network-vif-plugged event during server rebuild" [Undecided,Confirmed] https://launchpad.net/bugs/1694371 | 23:29 |
| pvaneck | clarkb: Yea, I agree, I would prefer to keep it in requirements.txt, too. A while back there a push to pass some requirements check where extra requirements would cause a failure. | 23:30 |
| clarkb | who else can I bug for tripleo review on https://review.openstack.org/#/c/447176/? EmilienM ? | 23:30 |
| pvaneck | but yea, I will bring it up with the team again | 23:30 |
| pvaneck | clarkb: thanks for merging! | 23:30 |
| clarkb | pvaneck: making everything optional is one of those "optimizations" that I feel make things harder. Unless its a pain to install the dep (licensing issues or needing to link against C) I would just include it. Wheels make it fast these days too | 23:30 |
| *** sulo has quit IRC | 23:30 | |
| *** sulo has joined #openstack-infra | 23:31 | |
| *** david-lyle has joined #openstack-infra | 23:32 | |
| pvaneck | clarkb: yea, will push to reinclude it. thanks for the insight | 23:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Convert some leftover changeish mentions to ref https://review.openstack.org/465718 | 23:34 |
| *** yamamoto_ has joined #openstack-infra | 23:36 | |
| *** Julien-zte has quit IRC | 23:38 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Reject some untrusted config objects https://review.openstack.org/470442 | 23:40 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Factor out duplicate code in executor repo tests https://review.openstack.org/471492 | 23:41 |
| *** gouthamr has quit IRC | 23:41 | |
| *** jkilpatr has quit IRC | 23:42 | |
| *** yamamoto_ has quit IRC | 23:42 | |
| *** jamesmcarthur has joined #openstack-infra | 23:43 | |
| Swimingly | hi all, looks like my pep8 check for cinder is failing because the openssl headers aren't present in the environment: http://logs.openstack.org/03/464003/6/check/gate-cinder-pep8-ubuntu-xenial/36c5610/tox/pep8-1.log | 23:43 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: sql-reporter: add support for Ref change https://review.openstack.org/466457 | 23:44 |
| *** gouthamr has joined #openstack-infra | 23:44 | |
| clarkb | Swimingly: yes smcginnis has pushed an update to cinders bindep.txt to address that | 23:44 |
| Swimingly | when should that go through? | 23:45 |
| Swimingly | clarkb ^ | 23:45 |
| *** sbezverk has quit IRC | 23:46 | |
| *** jamesmcarthur has quit IRC | 23:47 | |
| *** jkilpatr has joined #openstack-infra | 23:47 | |
| clarkb | Swimingly: whenever cinder cores approve it. You can depends-on the change id in your commit message in the mean time | 23:48 |
| *** jkilpatr has quit IRC | 23:50 | |
| *** pvaneck has quit IRC | 23:50 | |
| *** gouthamr has quit IRC | 23:51 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Remove ansible files at startup before copy https://review.openstack.org/471518 | 23:53 |
| *** jkilpatr has joined #openstack-infra | 23:53 | |
| Swimingly | clarkb: sounds good, I'm new, can you link to documentation about modifying depends-on? | 23:53 |
| clarkb | Swimingly: https://docs.openstack.org/infra/manual/developers.html#cross-repository-dependencies you can also rebase onto smcginnis change | 23:55 |
| openstackgerrit | Pete Birley proposed openstack-infra/system-config master: Add #openstack-helm to meetbot channels https://review.openstack.org/455742 | 23:56 |
| Swimingly | clarkb, awesome! thanks so much | 23:56 |
| *** jkilpatr has quit IRC | 23:57 | |
| *** david-lyle has quit IRC | 23:58 | |
| *** zz_dimtruck is now known as dimtruck | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!