| *** gyee has quit IRC | 00:21 | |
| *** altlogbot_2 has quit IRC | 00:23 | |
| *** irclogbot_1 has quit IRC | 00:23 | |
| *** mattw4 has quit IRC | 00:29 | |
| *** threestrands has quit IRC | 00:32 | |
| *** adriant7 has quit IRC | 00:32 | |
| *** adriant7 has joined #openstack-infra | 00:32 | |
| *** adriant7 has quit IRC | 00:32 | |
| *** yamamoto has joined #openstack-infra | 00:42 | |
| *** markvoelker has joined #openstack-infra | 00:55 | |
| *** nicolasbock has quit IRC | 00:57 | |
| *** markvoelker has quit IRC | 01:01 | |
| *** Goneri has quit IRC | 01:02 | |
| *** jamesmcarthur has joined #openstack-infra | 01:02 | |
| *** exsdev has joined #openstack-infra | 01:05 | |
| *** Roamer` has quit IRC | 01:09 | |
| *** zhurong has joined #openstack-infra | 01:11 | |
| *** jamesmcarthur has quit IRC | 01:12 | |
| *** jamesmcarthur has joined #openstack-infra | 01:14 | |
| *** armax has quit IRC | 01:15 | |
| *** igordc has quit IRC | 01:16 | |
| *** adriant has joined #openstack-infra | 01:21 | |
| *** jamesmcarthur has quit IRC | 01:24 | |
| *** irclogbot_2 has joined #openstack-infra | 01:34 | |
| *** irclogbot_2 has quit IRC | 01:39 | |
| *** irclogbot_2 has joined #openstack-infra | 01:40 | |
| *** diablo_rojo has quit IRC | 01:41 | |
| *** irclogbot_2 has quit IRC | 01:43 | |
| *** apetrich has quit IRC | 02:10 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:17 | |
| *** bhavikdbavishi has quit IRC | 02:21 | |
| *** smarcet has joined #openstack-infra | 02:21 | |
| *** smarcet has quit IRC | 02:30 | |
| *** smarcet has joined #openstack-infra | 02:34 | |
| *** smarcet has quit IRC | 02:38 | |
| *** prometheanfire has quit IRC | 02:39 | |
| *** prometheanfire has joined #openstack-infra | 02:40 | |
| *** spsurya has joined #openstack-infra | 02:45 | |
| *** armax has joined #openstack-infra | 02:50 | |
| *** irclogbot_0 has joined #openstack-infra | 02:54 | |
| *** irclogbot_0 has quit IRC | 02:59 | |
| *** markvoelker has joined #openstack-infra | 03:01 | |
| *** rh-jelabarre has quit IRC | 03:04 | |
| *** markvoelker has quit IRC | 03:05 | |
| *** notmyname has quit IRC | 03:06 | |
| *** ianychoi has quit IRC | 03:12 | |
| *** ianychoi has joined #openstack-infra | 03:13 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:19 | |
| *** jamesmcarthur has joined #openstack-infra | 03:23 | |
| *** irclogbot_2 has joined #openstack-infra | 03:34 | |
| *** irclogbot_2 has quit IRC | 03:39 | |
| *** signed8bit has quit IRC | 03:40 | |
| *** signed8bit has joined #openstack-infra | 03:41 | |
| *** ramishra has joined #openstack-infra | 03:48 | |
| *** jamesmcarthur has quit IRC | 03:54 | |
| *** jamesmcarthur has joined #openstack-infra | 03:55 | |
| *** larainema has joined #openstack-infra | 03:56 | |
| *** jamesmcarthur has quit IRC | 04:02 | |
| *** irclogbot_0 has joined #openstack-infra | 04:02 | |
| *** jamesmcarthur has joined #openstack-infra | 04:04 | |
| *** irclogbot_0 has quit IRC | 04:07 | |
| *** ricolin_ has joined #openstack-infra | 04:08 | |
| *** ricolin_ has quit IRC | 04:08 | |
| *** ricolin has joined #openstack-infra | 04:10 | |
| *** udesale has joined #openstack-infra | 04:19 | |
| *** irclogbot_2 has joined #openstack-infra | 04:22 | |
| *** irclogbot_2 has quit IRC | 04:33 | |
| *** jamesmcarthur has quit IRC | 04:37 | |
| *** soniya29 has joined #openstack-infra | 04:39 | |
| *** raukadah is now known as chandankumar | 04:55 | |
| *** jamesmcarthur has joined #openstack-infra | 05:04 | |
| *** gregoryo has joined #openstack-infra | 05:20 | |
| *** irclogbot_1 has joined #openstack-infra | 05:20 | |
| *** witek_ has joined #openstack-infra | 05:23 | |
| *** irclogbot_1 has quit IRC | 05:25 | |
| *** rcernin_ has joined #openstack-infra | 05:31 | |
| *** rcernin has quit IRC | 05:33 | |
| AJaeger | config-core, please review https://review.opendev.org/679850 and https://review.opendev.org/679856 | 05:40 |
|---|---|---|
| *** witek_ has quit IRC | 05:47 | |
| *** gfidente|afk is now known as gfidente | 05:49 | |
| *** soniya29 has quit IRC | 05:49 | |
| *** piotrowskim has joined #openstack-infra | 05:58 | |
| *** rfolco has quit IRC | 06:00 | |
| *** jamesmcarthur has quit IRC | 06:04 | |
| *** jtomasek has joined #openstack-infra | 06:06 | |
| *** ricolin_ has joined #openstack-infra | 06:15 | |
| *** udesale has quit IRC | 06:15 | |
| *** irclogbot_0 has joined #openstack-infra | 06:16 | |
| *** ricolin has quit IRC | 06:17 | |
| *** irclogbot_0 has quit IRC | 06:21 | |
| *** udesale has joined #openstack-infra | 06:22 | |
| *** irclogbot_1 has joined #openstack-infra | 06:24 | |
| *** irclogbot_1 has quit IRC | 06:29 | |
| *** kjackal has joined #openstack-infra | 06:34 | |
| *** markvoelker has joined #openstack-infra | 06:34 | |
| *** markvoelker has quit IRC | 06:39 | |
| *** kaiokmo has quit IRC | 06:39 | |
| *** sshnaidm|pto is now known as sshnaidm|afk | 06:45 | |
| *** pgaxatte has joined #openstack-infra | 06:51 | |
| *** slaweq has joined #openstack-infra | 06:51 | |
| *** kjackal has quit IRC | 06:52 | |
| *** diga has joined #openstack-infra | 06:54 | |
| *** ginopc has joined #openstack-infra | 06:56 | |
| *** markvoelker has joined #openstack-infra | 06:56 | |
| *** kjackal has joined #openstack-infra | 06:57 | |
| *** zbr has quit IRC | 06:58 | |
| *** pkopec has joined #openstack-infra | 06:59 | |
| *** markvoelker has quit IRC | 07:00 | |
| *** rcernin_ has quit IRC | 07:01 | |
| *** xenos76 has joined #openstack-infra | 07:05 | |
| *** zbr has joined #openstack-infra | 07:08 | |
| *** xenos76 has quit IRC | 07:10 | |
| *** toabctl has quit IRC | 07:11 | |
| *** xenos76 has joined #openstack-infra | 07:11 | |
| *** toabctl has joined #openstack-infra | 07:16 | |
| AJaeger | infra-root, I see random POST_FAILURES - haven't caught anything yet ;( | 07:18 |
| AJaeger | Example https://review.opendev.org/678747 and https://review.opendev.org/601659 | 07:19 |
| *** kaiokmo has joined #openstack-infra | 07:23 | |
| *** apetrich has joined #openstack-infra | 07:23 | |
| *** tosky has joined #openstack-infra | 07:25 | |
| AJaeger | these are not reproduceable ;( Tried to follow them on the console but those were all fine | 07:26 |
| *** jamesmcarthur has joined #openstack-infra | 07:29 | |
| *** irclogbot_1 has joined #openstack-infra | 07:30 | |
| *** jamesmcarthur has quit IRC | 07:34 | |
| *** irclogbot_1 has quit IRC | 07:35 | |
| *** jpena|off is now known as jpena | 07:39 | |
| *** kopecmartin|off is now known as kopecmartin | 07:40 | |
| *** pcaruana has joined #openstack-infra | 07:42 | |
| *** udesale has quit IRC | 07:42 | |
| *** gregoryo has quit IRC | 07:43 | |
| *** udesale has joined #openstack-infra | 07:43 | |
| *** roman_g has joined #openstack-infra | 07:47 | |
| *** lucasagomes has joined #openstack-infra | 07:47 | |
| *** roman_g_ has joined #openstack-infra | 07:47 | |
| *** roman_g has quit IRC | 07:48 | |
| *** roman_g_ has quit IRC | 07:48 | |
| *** roman_g has joined #openstack-infra | 07:48 | |
| *** irclogbot_1 has joined #openstack-infra | 07:50 | |
| *** irclogbot_1 has quit IRC | 07:53 | |
| *** ralonsoh has joined #openstack-infra | 07:55 | |
| *** ociuhandu has joined #openstack-infra | 07:57 | |
| *** e0ne has joined #openstack-infra | 07:59 | |
| *** ociuhandu has quit IRC | 07:59 | |
| *** ociuhandu has joined #openstack-infra | 08:00 | |
| *** hwoarang has quit IRC | 08:05 | |
| *** hwoarang has joined #openstack-infra | 08:07 | |
| *** e0ne has quit IRC | 08:07 | |
| *** tkajinam has quit IRC | 08:07 | |
| *** dchen has quit IRC | 08:12 | |
| *** dychen has quit IRC | 08:13 | |
| *** pkopec has quit IRC | 08:15 | |
| *** irclogbot_0 has joined #openstack-infra | 08:22 | |
| *** ricolin_ is now known as ricolin | 08:25 | |
| *** irclogbot_0 has quit IRC | 08:25 | |
| *** xinranwang has joined #openstack-infra | 08:29 | |
| *** e0ne has joined #openstack-infra | 08:35 | |
| *** markvoelker has joined #openstack-infra | 08:36 | |
| *** derekh has joined #openstack-infra | 08:36 | |
| *** markvoelker has quit IRC | 08:40 | |
| *** priteau has joined #openstack-infra | 08:45 | |
| *** pkopec has joined #openstack-infra | 08:51 | |
| *** ociuhandu has quit IRC | 08:55 | |
| *** ociuhandu has joined #openstack-infra | 08:56 | |
| *** ociuhandu has quit IRC | 09:00 | |
| *** trident has quit IRC | 09:01 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - add support for git.tag.creation event https://review.opendev.org/679938 | 09:02 |
| *** pkopec has quit IRC | 09:03 | |
| *** irclogbot_1 has joined #openstack-infra | 09:08 | |
| *** trident has joined #openstack-infra | 09:10 | |
| *** irclogbot_1 has quit IRC | 09:11 | |
| *** Roamer` has joined #openstack-infra | 09:16 | |
| *** slaweq has quit IRC | 09:16 | |
| *** mpjetta has quit IRC | 09:17 | |
| *** mpjetta has joined #openstack-infra | 09:18 | |
| *** beagles has quit IRC | 09:19 | |
| *** dtantsur|afk is now known as dtantsur | 09:22 | |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Pagure - fix wrong commit gitweb url https://review.opendev.org/679946 | 09:30 |
| *** slaweq has joined #openstack-infra | 09:33 | |
| *** ricolin has quit IRC | 09:34 | |
| *** ricolin has joined #openstack-infra | 09:35 | |
| *** pgaxatte has quit IRC | 09:40 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 09:41 | |
| *** bhavikdbavishi has quit IRC | 09:44 | |
| *** pgaxatte has joined #openstack-infra | 09:46 | |
| *** yamamoto has quit IRC | 09:47 | |
| *** roman_g has quit IRC | 09:48 | |
| *** yamamoto has joined #openstack-infra | 09:50 | |
| *** diga has quit IRC | 09:54 | |
| *** pkopec has joined #openstack-infra | 10:01 | |
| *** yamamoto has quit IRC | 10:10 | |
| *** yamamoto has joined #openstack-infra | 10:13 | |
| *** pkopec has quit IRC | 10:15 | |
| *** yamamoto has quit IRC | 10:18 | |
| *** pkopec has joined #openstack-infra | 10:24 | |
| *** pkopec has quit IRC | 10:28 | |
| *** irclogbot_0 has joined #openstack-infra | 10:28 | |
| *** pkopec has joined #openstack-infra | 10:30 | |
| *** udesale has quit IRC | 10:32 | |
| *** udesale has joined #openstack-infra | 10:32 | |
| *** irclogbot_0 has quit IRC | 10:33 | |
| *** yamamoto has joined #openstack-infra | 10:35 | |
| *** markvoelker has joined #openstack-infra | 10:41 | |
| *** irclogbot_2 has joined #openstack-infra | 10:42 | |
| *** xenos76 has quit IRC | 10:44 | |
| *** markvoelker has quit IRC | 10:45 | |
| *** irclogbot_2 has quit IRC | 10:45 | |
| *** dave-mccowan has joined #openstack-infra | 10:47 | |
| *** irclogbot_2 has joined #openstack-infra | 10:58 | |
| *** ociuhandu has joined #openstack-infra | 11:01 | |
| *** irclogbot_2 has quit IRC | 11:02 | |
| fungi | infra-root: keep an eye out for critical exim security patches on friday: https://www.openwall.com/lists/oss-security/2019/09/04/1 | 11:02 |
| *** bhavikdbavishi has joined #openstack-infra | 11:03 | |
| *** nicolasbock has joined #openstack-infra | 11:04 | |
| *** trident has quit IRC | 11:04 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 11:06 | |
| *** bhavikdbavishi has quit IRC | 11:07 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 11:07 | |
| *** udesale has quit IRC | 11:12 | |
| *** trident has joined #openstack-infra | 11:13 | |
| *** zbr is now known as zbr|swimming | 11:23 | |
| *** lucasagomes has quit IRC | 11:25 | |
| *** lucasagomes has joined #openstack-infra | 11:25 | |
| *** jpena is now known as jpena|lunch | 11:28 | |
| *** rh-jelabarre has joined #openstack-infra | 11:29 | |
| *** soniya29 has joined #openstack-infra | 11:36 | |
| *** irclogbot_0 has joined #openstack-infra | 11:42 | |
| *** xek has joined #openstack-infra | 11:45 | |
| *** irclogbot_0 has quit IRC | 11:47 | |
| *** yamamoto has quit IRC | 11:47 | |
| *** happyhemant has joined #openstack-infra | 11:48 | |
| *** yamamoto has joined #openstack-infra | 11:49 | |
| *** xenos76 has joined #openstack-infra | 11:51 | |
| *** electrofelix has joined #openstack-infra | 11:55 | |
| *** xinranwang has quit IRC | 11:58 | |
| *** tesseract has joined #openstack-infra | 12:03 | |
| *** kmarc has quit IRC | 12:06 | |
| donnyd | Just incase anyone is interested in swift usage, I put some charts up on the FN dashboard | 12:07 |
| *** rfolco has joined #openstack-infra | 12:08 | |
| *** jamesmcarthur has joined #openstack-infra | 12:09 | |
| *** larainema has quit IRC | 12:09 | |
| *** markvoelker has joined #openstack-infra | 12:13 | |
| *** jamesmcarthur has quit IRC | 12:13 | |
| *** pkopec has quit IRC | 12:14 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 12:15 |
| *** pkopec has joined #openstack-infra | 12:20 | |
| *** xek has quit IRC | 12:21 | |
| pabelanger | AJaeger: yah, I think we need a better retries option for upload-logs-swift, for example, on upload failure to random provider, try again but with another provider | 12:31 |
| pabelanger | was going to propose a ML topic about it / and work on code | 12:31 |
| pabelanger | but going to be tricky, and maybe an interface change on role paramaters | 12:31 |
| *** jpena|lunch is now known as jpena | 12:32 | |
| *** soniya29 has quit IRC | 12:32 | |
| *** pkopec has quit IRC | 12:35 | |
| *** iurygregory has joined #openstack-infra | 12:36 | |
| *** pkopec has joined #openstack-infra | 12:37 | |
| *** zbr|swimming is now known as zbr | 12:42 | |
| *** eharney has quit IRC | 12:46 | |
| *** udesale has joined #openstack-infra | 12:48 | |
| *** ociuhandu has quit IRC | 12:50 | |
| *** markvoelker has quit IRC | 12:54 | |
| *** ociuhandu has joined #openstack-infra | 12:55 | |
| *** markvoelker has joined #openstack-infra | 13:00 | |
| *** mriedem has joined #openstack-infra | 13:04 | |
| *** markvoelker has quit IRC | 13:05 | |
| *** kmarc has joined #openstack-infra | 13:06 | |
| *** yamamoto has quit IRC | 13:07 | |
| *** markvoelker has joined #openstack-infra | 13:27 | |
| zbr | AJaeger: fungi : can we merge https://review.opendev.org/#/c/673893/2 (DISK_FULL on dev) so we can later merge on production https://review.opendev.org/#/c/631509/ ? | 13:29 |
| zbr | i can abandon if these are not desired, just wanted to do some autumn-cleaning :D | 13:31 |
| *** lpetrut has joined #openstack-infra | 13:31 | |
| *** smarcet has joined #openstack-infra | 13:31 | |
| AJaeger | zbr: I'm not a system-config core - and fungi might not be around much today... | 13:32 |
| AJaeger | zbr: best wait for clarkb to wake up | 13:32 |
| zbr | i can wait, sorry that I forget that you are not a core, maybe you should ;) | 13:33 |
| openstackgerrit | Oliver Walsh proposed openstack/diskimage-builder master: WIP: ensure nouveau is blacklisted in initramfs too https://review.opendev.org/679992 | 13:35 |
| *** yamamoto has joined #openstack-infra | 13:37 | |
| *** sgw has joined #openstack-infra | 13:38 | |
| *** irclogbot_1 has joined #openstack-infra | 13:38 | |
| *** yamamoto has quit IRC | 13:39 | |
| *** udesale has quit IRC | 13:39 | |
| *** udesale has joined #openstack-infra | 13:39 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/diskimage-builder master: Fixed use of flake8 https://review.opendev.org/677059 | 13:40 |
| *** irclogbot_1 has quit IRC | 13:42 | |
| zbr | pabelanger: are you still working on https://review.opendev.org/#/c/650431/ ? any blockers? | 13:42 |
| pabelanger | zbr: will loop back to it this week, once 2.9 is uploaded to pypi | 13:46 |
| zbr | pabelanger: i can rebased it if you want. i was curious if other issues were present. | 13:46 |
| openstackgerrit | Masayuki Igawa proposed openstack/cookiecutter master: Update URLs https://review.opendev.org/679998 | 13:47 |
| pabelanger | zbr: the whole stack needs rebased, but I need to dig into failures | 13:47 |
| pabelanger | just been busy with 2.9 release | 13:47 |
| pabelanger | I'll loop back tomorrow or friday | 13:48 |
| zbr | pabelanger: anything related to 2.9 which could give me nightmares on molecule? A prerelease on pypi would be cool | 13:48 |
| pabelanger | I think pre-release is going out tomorrow? Not 100% | 13:49 |
| pabelanger | not sure about molecule specific, likely collection related things | 13:49 |
| *** prometheanfire has quit IRC | 13:53 | |
| *** prometheanfire has joined #openstack-infra | 13:54 | |
| *** pkopec has quit IRC | 13:55 | |
| dmsimard | donnyd: where is that dashboard ? | 13:56 |
| donnyd | https://grafana.fortnebula.com/d/9MMqh8HWk/openstack-utilization?orgId=2&refresh=30s | 13:56 |
| donnyd | Let me know if you have issues getting to it | 13:56 |
| dmsimard | donnyd: works for me, thanks | 13:57 |
| *** yamamoto has joined #openstack-infra | 13:57 | |
| *** jamesmcarthur has joined #openstack-infra | 13:58 | |
| donnyd | Trying to gather info that is useful so we can see where the loads are at and then make adjustments accordingly | 13:58 |
| *** irclogbot_1 has joined #openstack-infra | 13:58 | |
| *** eharney has joined #openstack-infra | 14:00 | |
| *** bexelbie_ has joined #openstack-infra | 14:01 | |
| *** irclogbot_1 has quit IRC | 14:02 | |
| *** bexelbie_ is now known as bexelbie | 14:02 | |
| *** smarcet has quit IRC | 14:02 | |
| *** jamesmcarthur has quit IRC | 14:03 | |
| *** irclogbot_1 has joined #openstack-infra | 14:04 | |
| *** happyhemant has quit IRC | 14:07 | |
| *** irclogbot_1 has quit IRC | 14:08 | |
| *** irclogbot_2 has joined #openstack-infra | 14:10 | |
| *** irclogbot_2 has quit IRC | 14:14 | |
| *** pkopec has joined #openstack-infra | 14:17 | |
| *** radez has quit IRC | 14:18 | |
| *** lpetrut has quit IRC | 14:19 | |
| *** radez has joined #openstack-infra | 14:19 | |
| *** jamesmcarthur has joined #openstack-infra | 14:19 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/project-config master: Add tooling to update python jobs on branch creation https://review.opendev.org/673019 | 14:24 |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 14:27 |
| *** pgaxatte has quit IRC | 14:28 | |
| *** tesseract has quit IRC | 14:30 | |
| *** kjackal_v2 has joined #openstack-infra | 14:32 | |
| *** kjackal has quit IRC | 14:34 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/project-config master: Add tooling to update python jobs on branch creation https://review.opendev.org/673019 | 14:36 |
| *** irclogbot_3 has joined #openstack-infra | 14:37 | |
| *** tesseract has joined #openstack-infra | 14:39 | |
| *** irclogbot_3 has quit IRC | 14:40 | |
| *** markvoelker has quit IRC | 14:41 | |
| *** jamesmcarthur has quit IRC | 14:43 | |
| *** tesseract has quit IRC | 14:44 | |
| *** yamamoto has quit IRC | 14:46 | |
| *** yamamoto has joined #openstack-infra | 14:47 | |
| *** yamamoto has quit IRC | 14:47 | |
| *** irclogbot_1 has joined #openstack-infra | 14:51 | |
| *** virendra-sharma has joined #openstack-infra | 14:51 | |
| *** mattw4 has joined #openstack-infra | 14:53 | |
| virendra-sharma | anyone can help on provider minicloud step on ubuntu16.04 ? | 14:53 |
| virendra-sharma | v | 14:53 |
| virendra-sharma | getting error on setup "Error: Syntax error at 'Hash'; expected ')' at /etc/puppet/modules/apt/manifests/init.pp:119 on node" | 14:53 |
| *** irclogbot_1 has quit IRC | 14:54 | |
| *** eharney has quit IRC | 14:54 | |
| *** chandankumar is now known as raukadah | 15:02 | |
| *** smarcet has joined #openstack-infra | 15:08 | |
| *** smarcet has quit IRC | 15:09 | |
| *** Goneri has joined #openstack-infra | 15:11 | |
| *** jamesmcarthur has joined #openstack-infra | 15:13 | |
| *** tesseract has joined #openstack-infra | 15:14 | |
| *** smarcet has joined #openstack-infra | 15:15 | |
| *** irclogbot_2 has joined #openstack-infra | 15:19 | |
| *** ddurst has quit IRC | 15:19 | |
| *** irclogbot_2 has quit IRC | 15:22 | |
| *** smarcet has quit IRC | 15:23 | |
| clarkb | virendra-sharma: we don't run an openstack cloud here, however that looks like a bug in your apt puppet module | 15:24 |
| *** kjackal_v2 has quit IRC | 15:24 | |
| *** kjackal has joined #openstack-infra | 15:24 | |
| *** zbr is now known as zbr|out | 15:25 | |
| *** jamesmcarthur has quit IRC | 15:25 | |
| *** mattw4 has quit IRC | 15:25 | |
| *** xarses has quit IRC | 15:25 | |
| openstackgerrit | Scott Little proposed openstack/project-config master: StarlingX retructuring: Ten new repos https://review.opendev.org/679848 | 15:26 |
| virendra-sharma | clarkb any alternative for cloud ? | 15:26 |
| *** xarses has joined #openstack-infra | 15:26 | |
| virendra-sharma | I am creating new nodepool provider with ubuntu16.06 | 15:26 |
| virendra-sharma | 16.04*** | 15:26 |
| clarkb | virendra-sharma: what is failing the deployment of a cloud or the deployment of nodepool? | 15:26 |
| virendra-sharma | on one of bare blade I am installing "mini cloud" script, but it is ending with error | 15:27 |
| virendra-sharma | it is trying to install site.pp, but failing with syntax error | 15:28 |
| clarkb | I don't know what the mini cloud script is | 15:28 |
| clarkb | but I know a common deployment tool is openstack ansible | 15:28 |
| virendra-sharma | Do we have any steps for nodepool provider creation steps ? | 15:28 |
| virendra-sharma | for ubuntu 16.04 | 15:29 |
| *** smarcet has joined #openstack-infra | 15:29 | |
| *** e0ne has quit IRC | 15:29 | |
| *** e0ne_ has joined #openstack-infra | 15:29 | |
| clarkb | no, nodepool assumes resources exist for it (so we don't have a significant portion of docs on running a cloud). Instead you can look at deployment docs for openstack | 15:30 |
| slittle1 | The seed repos for the StarlingX restructure have been populated on github. We are ready to go! | 15:32 |
| *** e0ne_ is now known as e0ne | 15:33 | |
| *** irclogbot_2 has joined #openstack-infra | 15:33 | |
| *** jamesmcarthur has joined #openstack-infra | 15:33 | |
| *** ociuhandu has quit IRC | 15:34 | |
| *** rpioso is now known as rpioso|afk | 15:35 | |
| virendra-sharma | clarkb can you help me with deployment docs ? | 15:35 |
| clarkb | slittle1: great. AJaeger ^ maybe you can sanity check it and I'll review as soon as I finish updating these changes I am working on? | 15:36 |
| *** irclogbot_2 has quit IRC | 15:36 | |
| *** irclogbot_2 has joined #openstack-infra | 15:37 | |
| *** virendra-sharma has quit IRC | 15:37 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 15:38 |
| *** altlogbot_3 has joined #openstack-infra | 15:39 | |
| AJaeger | clarkb: waiting for tests to run... | 15:39 |
| openstackgerrit | Clark Boylan proposed opendev/base-jobs master: Test improvements to swift hosted logs https://review.opendev.org/679819 | 15:40 |
| clarkb | AJaeger: thanks | 15:40 |
| clarkb | AJaeger: corvus ^ made the generic name updates | 15:40 |
| AJaeger | slittle1: I would appreciate review from some other starlingx cores. | 15:41 |
| AJaeger | dtroyer: are you around and can review https://review.opendev.org/#/c/679848, please? | 15:41 |
| *** markvoelker has joined #openstack-infra | 15:45 | |
| AJaeger | clarkb, slittle1, change itself is fine - I wait for test runs to finish before I +2 - and also would like to see some indication from starlingx folks that this change is desired by starlingx team. | 15:45 |
| *** yamamoto has joined #openstack-infra | 15:47 | |
| *** lucasagomes has quit IRC | 15:48 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Document globally unique container name requirement https://review.opendev.org/679814 | 15:50 |
| AJaeger | slittle1: checks fail with "ERROR: project starlingx/openstack-armada-app, description 'StarlingX Openstack Armada App': contains wrong word 'Openstack', it should be 'OpenStack'" | 15:50 |
| AJaeger | slittle1: please fix | 15:50 |
| *** sthussey has joined #openstack-infra | 15:54 | |
| slittle1 | I'm back | 15:55 |
| slittle1 | checking into it | 15:55 |
| *** priteau has quit IRC | 15:55 | |
| *** dtantsur is now known as dtantsur|afk | 15:55 | |
| *** yamamoto has quit IRC | 15:57 | |
| *** virendra-sharma has joined #openstack-infra | 15:58 | |
| *** kjackal has quit IRC | 15:59 | |
| *** gyee has joined #openstack-infra | 15:59 | |
| openstackgerrit | Scott Little proposed openstack/project-config master: StarlingX retructuring: Ten new repos https://review.opendev.org/679848 | 15:59 |
| dtroyer | slittle1, AJaeger: +1 on ^^ | 16:03 |
| AJaeger | dtroyer: thanks for confirming... | 16:03 |
| AJaeger | slittle1: do you have changes up to remove the content moved to the new repos? | 16:04 |
| slittle1 | I have them ready to go. | 16:05 |
| AJaeger | dtroyer, slittle1, do you need IRC notifications? See https://review.opendev.org/679850 | 16:05 |
| AJaeger | note that changes for "master" are not published - I can add that as well if you want | 16:05 |
| slittle1 | We've not been using IRC notification to date, so no. | 16:06 |
| *** smarcet has quit IRC | 16:06 | |
| *** ociuhandu has joined #openstack-infra | 16:06 | |
| dtroyer | AJaeger: there were complaints about the notification 'noise' when we originally had them enabled, traffic is low enough in there actual comments got lost. | 16:07 |
| AJaeger | dtroyer: so, want to remove them instead? | 16:08 |
| AJaeger | slittle1: you used and a change we did disabled them by accident... | 16:08 |
| * AJaeger updates | 16:09 | |
| dtroyer | AJaeger: yes. I was under the impression we (someone?) explicitly removed them, so go ahead and do that now | 16:09 |
| dtroyer | thanks | 16:10 |
| slittle1 | Yes, disable them please | 16:10 |
| *** smarcet has joined #openstack-infra | 16:10 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Remove broken notification for starlingx https://review.opendev.org/679850 | 16:10 |
| AJaeger | slittle1, dtroyer , could you +1, please? ^ | 16:10 |
| *** ociuhandu has quit IRC | 16:11 | |
| dtroyer | AJaeger: done | 16:11 |
| AJaeger | thanks | 16:11 |
| slittle1 | done | 16:12 |
| AJaeger | clarkb: So, https://review.opendev.org/679848 LGTM (the new starlingx repos) - want to +2A once tests pass? | 16:12 |
| AJaeger | config-core, please put https://review.opendev.org/679850 and https://review.opendev.org/679856 on your review queue | 16:13 |
| * AJaeger needs to step out for a bit | 16:13 | |
| logan- | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_01/658701/3/check/build-openstack-releasenotes/3688467/ | 16:18 |
| *** kopecmartin is now known as kopecmartin|off | 16:18 | |
| pabelanger | eep | 16:18 |
| logan- | 402 Payment Required - Access was denied for financial reasons. | 16:18 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 16:18 |
| clarkb | neat | 16:19 |
| *** zbr|out is now known as zbr | 16:19 | |
| clarkb | corvus: ^ is that something you are able to look at? | 16:20 |
| clarkb | looking at our infra root imap ovh folder we somehow got charged 10 euros? | 16:22 |
| *** jaosorior has joined #openstack-infra | 16:23 | |
| clarkb | amorin: are you on IRC ^ | 16:23 |
| openstackgerrit | James E. Blair proposed opendev/base-jobs master: Stop storing logs in OVH https://review.opendev.org/680077 | 16:23 |
| corvus | clarkb: ^ | 16:23 |
| slittle1 | uh oh | 16:24 |
| clarkb | amorin: we've got two billing emails this moring about 15 minutes ago. One for ~10 euros and one for ~15 euros. I'm not sure how the billing comping is handled but may be our use of swift has tripped over something? | 16:24 |
| tbarron | I'm getting that "Payment Required: Access was denied for financial reasons" when trying to inspect logs too. | 16:24 |
| clarkb | tbarron: ya I expect we'll get that on all ovh urls until we sort it out | 16:25 |
| tbarron | oh I see you are talking about unexpected billing, nm | 16:25 |
| corvus | tbarron: we're taking ovh out of rotation; once 680077 lands you can recheck and the logs will go somewhere else | 16:25 |
| slittle1 | we had one subdirectory relocation between two pre-existing repos | 16:25 |
| tbarron | clarkb; corvus ty | 16:25 |
| slittle1 | gerrit is refusing to accept the merge | 16:25 |
| *** smarcet has quit IRC | 16:25 | |
| slittle1 | remote: ERROR: In commit e12bd0561ce734b28132ce2ed2e999256cb55163 | 16:26 |
| slittle1 | remote: ERROR: committer email address dtroyer@gmail.com | 16:26 |
| slittle1 | remote: ERROR: does not match your user account. | 16:26 |
| slittle1 | remote: ERROR: | 16:26 |
| slittle1 | remote: ERROR: The following addresses are currently registered: | 16:26 |
| slittle1 | remote: ERROR: scott.little@windriver.com | 16:26 |
| slittle1 | gotta step out for a doctor apt. Back in an hour. | 16:26 |
| *** eharney has joined #openstack-infra | 16:27 | |
| corvus | slittle1: in the future, you can use paste.openstack.org for large messages like that. it's likely that you're pushing up a series of changes which you didn't write. you can perform a rebase and rewrite all of the commits with yourself as the committer and it should work. | 16:27 |
| clarkb | I'm drafting up an email to damien, jean daniel and amorin now | 16:28 |
| openstackgerrit | James E. Blair proposed opendev/base-jobs master: Retain swift logs for only 1 month https://review.opendev.org/680080 | 16:29 |
| corvus | clarkb: ^ a followup which may be prudent | 16:29 |
| clarkb | +2 thanks | 16:30 |
| clarkb | actually approved it because I think there may be few other roots around for a bit | 16:30 |
| corvus | i was just about to ask that :) | 16:30 |
| *** e0ne has quit IRC | 16:31 | |
| *** smarcet has joined #openstack-infra | 16:32 | |
| *** sshnaidm|ruck is now known as sshnaidm|bbl | 16:32 | |
| mnaser | ouch, infra was doing 6 months retention right now? | 16:33 |
| clarkb | mnaser: with the old logs server it was only 30 days | 16:33 |
| clarkb | (I don't know what the role default is if there is one | 16:33 |
| mnaser | the role default is 6 months | 16:34 |
| mnaser | https://zuul-ci.org/docs/zuul-jobs/log-roles.html#rolevar-upload-logs-swift.zuul_log_delete_after | 16:34 |
| openstackgerrit | Merged opendev/base-jobs master: Stop storing logs in OVH https://review.opendev.org/680077 | 16:34 |
| clarkb | ok so we were at 30 days then went to 6 months for a couple weeks and are now at 30 days again :) | 16:34 |
| *** virendra-sharma8 has joined #openstack-infra | 16:35 | |
| *** virendra-sharma8 has quit IRC | 16:36 | |
| *** virendra-sharma8 has joined #openstack-infra | 16:36 | |
| *** virendra-sharma has quit IRC | 16:37 | |
| clarkb | email sent to ovh | 16:42 |
| *** kjackal has joined #openstack-infra | 16:42 | |
| *** smarcet has quit IRC | 16:45 | |
| mnaser | infra-root: probably worth of a #status to update users? i feel like a lot of people will come in asking | 16:47 |
| clarkb | #status notice Zuul job logs stored in OVH may fail. We have updated the base job to remove OVH from our storage location. If you have POST_FAILURES a recheck should fix them at this point. | 16:50 |
| openstackstatus | clarkb: sending notice | 16:50 |
| clarkb | mnaser: ^ there | 16:50 |
| *** stewie925 has joined #openstack-infra | 16:51 | |
| -openstackstatus- NOTICE: Zuul job logs stored in OVH may fail. We have updated the base job to remove OVH from our storage location. If you have POST_FAILURES a recheck should fix them at this point. | 16:52 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 16:52 |
| *** jpena is now known as jpena|off | 16:53 | |
| openstackstatus | clarkb: finished sending notice | 16:54 |
| stewie925 | thanks clarkb - I did get a POST FAILURE and your notice just came out in time :) | 16:54 |
| clarkb | mnaser: did you see https://review.opendev.org/679145 ? | 16:55 |
| clarkb | mnaser: that is aimed at making ceph object storage more viable | 16:55 |
| clarkb | mnaser: it will distribute all objects over 4k containers and due to random distributions it should be pretty evenly spread | 16:56 |
| clarkb | corvus: that is excellent gerrit upgrade info | 16:58 |
| clarkb | makes it feel far less daunting | 16:58 |
| zbr | corvus: https://review.opendev.org/#/c/430393/ | 16:59 |
| *** derekh has quit IRC | 17:00 | |
| corvus | clarkb: yeah, i feel a lot better about it too | 17:00 |
| *** electrofelix has quit IRC | 17:06 | |
| *** mattw4 has joined #openstack-infra | 17:06 | |
| *** udesale has quit IRC | 17:07 | |
| *** kaisers has quit IRC | 17:09 | |
| *** smarcet has joined #openstack-infra | 17:18 | |
| *** rfolco is now known as rfolco|bank | 17:18 | |
| openstackgerrit | Merged opendev/base-jobs master: Retain swift logs for only 1 month https://review.opendev.org/680080 | 17:20 |
| *** ociuhandu has joined #openstack-infra | 17:22 | |
| *** diablo_rojo has joined #openstack-infra | 17:23 | |
| *** smarcet has left #openstack-infra | 17:24 | |
| *** ociuhandu has quit IRC | 17:26 | |
| *** virendra-sharma8 has quit IRC | 17:26 | |
| openstackgerrit | Clark Boylan proposed opendev/base-jobs master: Remove ovh from the base-test and -minimal jobs https://review.opendev.org/680101 | 17:27 |
| clarkb | corvus: ^ that should fix it for the opendev jobs that do special things | 17:27 |
| *** kaisers has joined #openstack-infra | 17:28 | |
| *** gfidente is now known as gfidente|afk | 17:30 | |
| *** diablo_rojo has quit IRC | 17:31 | |
| clarkb | slittle1: fwiw I'm digging out my logs hosted in swift hole then will have time to help with your project moves and stuff. Feel free to ping when you are back | 17:32 |
| *** rpioso|afk is now known as rpioso | 17:33 | |
| clarkb | slittle1: for the pushing of those other commits are you trying to push them for review and merge or trying to just append them to the repo bypassing reivew? if pushing for review you could rewrite the committer on the commits (leave the author alone) and push | 17:33 |
| clarkb | slittle1: if wanting to just push that will require a gerrit admin to do for you | 17:33 |
| *** rledisez has joined #openstack-infra | 17:34 | |
| *** ociuhandu has joined #openstack-infra | 17:38 | |
| slittle1 | I'm back | 17:40 |
| openstackgerrit | Merged opendev/base-jobs master: Remove ovh from the base-test and -minimal jobs https://review.opendev.org/680101 | 17:41 |
| *** ociuhandu has quit IRC | 17:42 | |
| slittle1 | clarkb: The best outcome would be to avoid re-review and retain authorship for the content being relocated between gits | 17:43 |
| clarkb | slittle1: note changing the committer does not change the author | 17:43 |
| clarkb | git tracks those two entries separately | 17:43 |
| clarkb | gerrit wants you to push code as the committer but someone else can be marked as the author | 17:44 |
| slittle1 | clarkb: ok, what's the command syntax for that? | 17:44 |
| slittle1 | clarkb: I | 17:45 |
| AJaeger | slittle1: git commit --author ? | 17:45 |
| clarkb | AJaeger: thatwill change the attribute we don't want to update | 17:46 |
| slittle1 | That's what I was looking at. Seems to change author not commiter. | 17:46 |
| clarkb | I think you can set GIT_COMMITTER_NAME and GIT_COMMITTER_EMAIL as env vars then rebase and edit the commits | 17:46 |
| clarkb | there are probably other appraoches too | 17:47 |
| slittle1 | K, I can try that | 17:47 |
| clarkb | in fact it may just do the right thing if you rebase and edit each commit | 17:47 |
| AJaeger | clarkb: Ah, sorry for mixing it up | 17:49 |
| *** bdodd has joined #openstack-infra | 17:50 | |
| slittle1 | 'git rebase -i <sha>' doesn't like going deeper into the history than the merge | 17:50 |
| slittle1 | oops | 17:51 |
| slittle1 | wrong git | 17:51 |
| *** diablo_rojo has joined #openstack-infra | 17:53 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Web improvement ideas https://review.opendev.org/680109 | 17:54 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Specify open source projects using Zuul https://review.opendev.org/680110 | 17:54 |
| *** ociuhandu has joined #openstack-infra | 17:54 | |
| clarkb | AJaeger: thank you for rechecking my zuul-jobs change | 17:54 |
| *** ociuhandu has quit IRC | 17:58 | |
| slittle1 | ok, committer is now me, author is unchanged. | 18:01 |
| clarkb | you should be able to push to gerrit now | 18:01 |
| clarkb | if I'm remembering gerrit's requirements proprely | 18:01 |
| slittle1 | will a 'git review' result in a re-review of all the imported commits ? | 18:01 |
| *** tesseract has quit IRC | 18:01 | |
| clarkb | yes | 18:02 |
| clarkb | how many commits are we talking? | 18:02 |
| slittle1 | ~25 | 18:02 |
| clarkb | One upside to going through gerrit is you get that record of the move with the other records of changing | 18:02 |
| clarkb | if you really want to force push you'll need to put the repo somewhere where a gerrit admin can get it and push for you | 18:03 |
| clarkb | should we hold off on approving the new project repo creations until we sort out ^ | 18:03 |
| slittle1 | no, they won't be referenced | 18:03 |
| slittle1 | so don't hold off | 18:03 |
| clarkb | ok reviewing the project addition change now | 18:04 |
| slittle1 | let me check if there are any other cases like this | 18:06 |
| clarkb | slittle1: dtroyer any concern with having repos called 'kubernetes' and 'puppet'? neither appear to be forks of the proper noun versions of those repos so I'm fine with the contents just double checking you aren't worried about confusion there | 18:07 |
| *** iurygregory has quit IRC | 18:08 | |
| *** iurygregory has joined #openstack-infra | 18:09 | |
| slittle1 | two other repos have a similar issue. Amn additional ~15 re-reviews would be triggered | 18:09 |
| clarkb | ya I don't mean to bikeshed just pointing out that that could be seen as "now starlingx has forked kubernetes and puppet" | 18:10 |
| clarkb | (even though you haven't) | 18:10 |
| clarkb | I'm fine with it myself | 18:10 |
| *** markvoelker has quit IRC | 18:12 | |
| *** markvoelker has joined #openstack-infra | 18:12 | |
| slittle1 | hmmm... here is where dropping the stx- prefix might bite us | 18:12 |
| AJaeger | want to add it back in? | 18:15 |
| slittle1 | give us a moment | 18:17 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 18:18 |
| slittle1 | perhaps rename 'kubernetes' as 'containers' | 18:18 |
| slittle1 | rename 'puppet' as 'stx-puppet'. | 18:19 |
| slittle1 | leave rest as is | 18:19 |
| slittle1 | Dean, Saul... thoughts? | 18:20 |
| *** smarcet has joined #openstack-infra | 18:25 | |
| corvus | sgw, dtroyer: ^ | 18:25 |
| clarkb | AJaeger: corvus fwiw I +1'd as is with a note about ^ I'm fine with the change if starlingx is fine with the names (so it can be approved as is in that case). I need to step out for a bit now so feel free to approve in my absense if we stick with it | 18:26 |
| *** smarcet has quit IRC | 18:28 | |
| openstackgerrit | Clark Boylan proposed opendev/base-jobs master: Test improvements to swift hosted logs https://review.opendev.org/679819 | 18:28 |
| clarkb | opendev base jobs edited to match zuul-jobs latest patchset | 18:28 |
| clarkb | and now I must pop out for a bit | 18:28 |
| weshay | just in case you guys did not see it already.. seems to be an above average number of post_failures atm https://zuul.opendev.org/t/openstack/builds?result=POST_FAILURE although tbh I don't have a good baseline for post_failures | 18:34 |
| zbr | weshay: check https://twitter.com/openstackinfra/status/1169292579533443072 | 18:35 |
| weshay | Zuul job logs stored in OVH may fail. We have updated the base job to remove OVH from our storage location. If you have POST_FAILURES a recheck should fix them at this point. | 18:35 |
| weshay | roger that | 18:35 |
| slittle1 | crickets | 18:36 |
| zbr | can someone abandon bashate change https://review.opendev.org/#/c/335520/ ? | 18:37 |
| slittle1 | ok, if we request the rename... I need to revise https://review.opendev.org/679848 again... correct? | 18:38 |
| AJaeger | slittle1: yes, just update that change... | 18:39 |
| AJaeger | let's rename *now* ;) | 18:39 |
| *** jamesmcarthur has quit IRC | 18:43 | |
| *** jamesmcarthur has joined #openstack-infra | 18:44 | |
| openstackgerrit | Scott Little proposed openstack/project-config master: StarlingX retructuring: Ten new repos https://review.opendev.org/679848 | 18:46 |
| AJaeger | slittle1: stc-puppet or stx? | 18:47 |
| AJaeger | slittle1: typo in commit message only I hope - we can ignore that... | 18:47 |
| slittle1 | stx-puppet | 18:48 |
| *** kaisers has quit IRC | 18:49 | |
| AJaeger | just in the commit message - I'm fine ignoring that typo. +2 with comment | 18:49 |
| slittle1 | ok | 18:49 |
| *** ricolin has quit IRC | 18:50 | |
| slittle1 | now about merging without re-review .... | 18:52 |
| AJaeger | slittle1: an admin can force push new content - and that means that every user needs to reset their repo... | 18:54 |
| *** irclogbot_2 has quit IRC | 18:54 | |
| slittle1 | I imported a sub-directory from another repo with history. Ideally that has all been previouslt reviewed. But then I add several new commits to fix up various build config files. | 18:55 |
| *** irclogbot_2 has joined #openstack-infra | 18:55 | |
| slittle1 | right | 18:55 |
| slittle1 | reseting repos is bad | 18:55 |
| slittle1 | I think I'll go with re-review | 18:55 |
| *** ralonsoh has quit IRC | 18:55 | |
| *** e0ne has joined #openstack-infra | 18:59 | |
| slittle1 | hmmm... if folks are respecting the freeze, is it really a reset? | 19:00 |
| *** bhavikdbavishi has quit IRC | 19:00 | |
| AJaeger | if you just add stuff - it might work. Best to discuss with clarkb, he has more experience here... | 19:01 |
| AJaeger | so, could work in this special case. | 19:01 |
| *** bhavikdbavishi has joined #openstack-infra | 19:02 | |
| *** irclogbot_2 has quit IRC | 19:02 | |
| *** irclogbot_3 has joined #openstack-infra | 19:03 | |
| *** ramishra has quit IRC | 19:03 | |
| clarkb | depends on whether or not history changes | 19:12 |
| clarkb | if you are fast forward only then its fine | 19:12 |
| *** bhavikdbavishi has quit IRC | 19:16 | |
| slittle1 | you'll b e able to detect that? | 19:17 |
| slittle1 | what's the process to request the merge? | 19:19 |
| *** exsdev has quit IRC | 19:21 | |
| *** jamesmcarthur has quit IRC | 19:23 | |
| clarkb | slittle1: do your edits create new commits at the tip of master or do they edit history | 19:26 |
| clarkb | (that is how you tell) | 19:26 |
| *** exsdev has joined #openstack-infra | 19:27 | |
| slittle1 | clarkb: what's the process to request an admin to do the merge? | 19:28 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 19:29 |
| clarkb | slittle1: you'll need to create a repo with the content you want pushed, then point us to it and tell us where you want it pushed. We should probably get signoff by whoever is core on the target repo too (since we are bypassing review) | 19:30 |
| clarkb | note that we typically prefer to have you just push things for review if the number of hcanges is small | 19:30 |
| clarkb | ~25 is probably on the border of small and not small | 19:30 |
| *** jamesmcarthur has joined #openstack-infra | 19:33 | |
| slaweq | hi infra root, qq: I got "Payment Required" error on https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/logs_28/651828/44/check/tobiko-tripleo-update-standalone/b1a8641/ | 19:33 |
| slaweq | are You aware of it? | 19:33 |
| clarkb | slaweq: yes we sent a notice earlier | 19:34 |
| slaweq | clarkb: ok, thx a lot | 19:34 |
| clarkb | bsaically OVH billing got weird and we removed it from the rotation | 19:34 |
| clarkb | amorin has escalated it for us | 19:34 |
| slaweq | thx for info | 19:34 |
| clarkb | we'll add it back once sorted, but existing log links will probably return that until happy again | 19:34 |
| slaweq | so now I have to recheck my patch to get access to logs from job, right? | 19:35 |
| slaweq | or is there any other way to access those existing logs? | 19:35 |
| clarkb | AJaeger: should we have dtroyer ack https://review.opendev.org/#/c/679848/5 again? I think it is ready otherwise | 19:35 |
| clarkb | slaweq: I think the two options are wait for ovh problem to be sorted out or recheck | 19:36 |
| slaweq | clarkb: ok, thx | 19:36 |
| *** exsdev has quit IRC | 19:36 | |
| slaweq | I will recheck :) | 19:36 |
| *** kaisers has joined #openstack-infra | 19:37 | |
| aspiers | is it just me or is gitea painfully slow? | 19:38 |
| aspiers | e.g. https://opendev.org/openstack/nova/src/branch/master/nova/api/openstack | 19:38 |
| clarkb | aspiers: it depends on the repo, but yes some repos (particularly nova) are slow | 19:38 |
| * aspiers wonders if I asked that last week | 19:38 | |
| aspiers | I can understand for big files, but why for directory listings? | 19:38 |
| clarkb | aspiers: we asked for volunteers to help on a openstack-discuss mailing list thread a few weeks back but got crickets as soon as we mentioned it | 19:38 |
| aspiers | ah | 19:38 |
| clarkb | because gitea checks when every file and directory is modified everytime it renders a page | 19:39 |
| clarkb | and doesn't cache any of that data (this is why nova is slow with its large history and large number of files) | 19:39 |
| aspiers | ah ok | 19:39 |
| dtroyer | clarkb, AJaeger, slittle1: +1 on https://review.opendev.org/#/c/679848/5 | 19:40 |
| AJaeger | clarkb: yes, let's ask dtroyer to confirm the changes in 679848 | 19:40 |
| dtroyer | jinx :) | 19:40 |
| AJaeger | dtroyer: that was fast ;) | 19:40 |
| AJaeger | thanks | 19:40 |
| * AJaeger approves | 19:41 | |
| *** ociuhandu has joined #openstack-infra | 19:47 | |
| *** eharney has quit IRC | 19:51 | |
| clarkb | aspiers: fwiw I've still been trying to improve gitea reliability as I have time which is why I haven't poked at performance. Would be extremely happy to have help in either or both places :) | 19:53 |
| *** exsdev has joined #openstack-infra | 19:53 | |
| aspiers | Wish I had more time :-( Maybe in a while | 19:54 |
| aspiers | although TBH I'd want to help with the Gerrit upgrade more than with gitea | 19:54 |
| *** jamesmcarthur has quit IRC | 19:54 | |
| clarkb | aspiers: corvus just reported back on what they learned at the gerrit user summit. I think that gives us what we need to know to then go and make those chanes | 19:54 |
| openstackgerrit | Merged openstack/project-config master: StarlingX retructuring: Ten new repos https://review.opendev.org/679848 | 19:54 |
| clarkb | aspiers: topic:gus2019 if you want to review related changes | 19:55 |
| *** rfolco|bank is now known as rfolco | 19:55 | |
| aspiers | cool | 19:55 |
| clarkb | I know I said yesterday I was going to try and do the gerrit connection timeout change today, but I remembred I need to file paperwork for visa thing sand then the swift stuff happened. Tomorrow looking more likely now | 19:56 |
| *** smarcet has joined #openstack-infra | 20:02 | |
| *** smarcet has quit IRC | 20:06 | |
| *** jamesmcarthur has joined #openstack-infra | 20:08 | |
| *** jtomasek has quit IRC | 20:08 | |
| *** ociuhandu has quit IRC | 20:09 | |
| *** ociuhandu has joined #openstack-infra | 20:10 | |
| *** smarcet has joined #openstack-infra | 20:14 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Fix "Set failed, unreachable, skipped statuses in json plugin" https://review.opendev.org/680159 | 20:14 |
| openstackgerrit | Merged zuul/zuul-jobs master: Add option for object store friendly log paths https://review.opendev.org/679145 | 20:15 |
| zbr | clarkb: can we merge https://review.opendev.org/#/c/536355/8 ? -- i checked it myself with https://hub.docker.com/_/oraclelinux/ | 20:17 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Fix "Set failed, unreachable, skipped statuses in json plugin" https://review.opendev.org/680159 | 20:17 |
| clarkb | zbr: yes and thank you for testing it | 20:18 |
| zbr | https://review.opendev.org/#/c/668740/ another nobrainer | 20:19 |
| zbr | and also https://review.opendev.org/#/c/602433/ but i confess that i did not test this one, but visually looks legit. | 20:20 |
| zbr | usually is hard to say no to something that improves testing. | 20:20 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:20 |
| *** markvoelker has quit IRC | 20:21 | |
| *** eernst has joined #openstack-infra | 20:22 | |
| clarkb | zbr: I feel like I'll need to test that apk one beforeI can merge it. I probably won't get to that today | 20:24 |
| zbr | no worry, its prio was close to zero, i was just cleaning up. | 20:24 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:27 |
| *** markvoelker has joined #openstack-infra | 20:30 | |
| *** exsdev has quit IRC | 20:33 | |
| zbr | https://review.opendev.org/#/c/677971/6 which makes the emit-job-header bit more informative. | 20:33 |
| *** pkopec has quit IRC | 20:35 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:37 |
| *** e0ne has quit IRC | 20:37 | |
| *** ociuhandu has quit IRC | 20:38 | |
| *** exsdev has joined #openstack-infra | 20:38 | |
| openstackgerrit | Merged opendev/base-jobs master: Test improvements to swift hosted logs https://review.opendev.org/679819 | 20:42 |
| openstackgerrit | Clark Boylan proposed zuul/zuul-jobs master: DO NOT MERGE test log path changes https://review.opendev.org/680178 | 20:46 |
| clarkb | testing the update to base-test in ^ | 20:46 |
| openstackgerrit | Merged opendev/bindep master: Add dependency version examples to README https://review.opendev.org/668740 | 20:48 |
| *** e0ne has joined #openstack-infra | 20:48 | |
| *** exsdev has quit IRC | 20:52 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul-website master: Add Zuul FAQ page https://review.opendev.org/679670 | 20:53 |
| *** e0ne has quit IRC | 20:59 | |
| *** exsdev has joined #openstack-infra | 21:00 | |
| *** munimeha1 has joined #openstack-infra | 21:01 | |
| *** sshnaidm|bbl is now known as sshnaidm|afk | 21:01 | |
| *** eharney has joined #openstack-infra | 21:07 | |
| *** slaweq has quit IRC | 21:08 | |
| *** slaweq has joined #openstack-infra | 21:11 | |
| *** smarcet has quit IRC | 21:12 | |
| *** snecklifter has joined #openstack-infra | 21:14 | |
| *** gfidente|afk has quit IRC | 21:14 | |
| *** slaweq has quit IRC | 21:15 | |
| *** snecker has joined #openstack-infra | 21:16 | |
| *** snecklifter has quit IRC | 21:19 | |
| *** snecker has quit IRC | 21:22 | |
| *** psachin has joined #openstack-infra | 21:23 | |
| *** jamesmcarthur has quit IRC | 21:25 | |
| *** jamesmcarthur has joined #openstack-infra | 21:27 | |
| *** exsdev has quit IRC | 21:27 | |
| *** smarcet has joined #openstack-infra | 21:29 | |
| *** kjackal has quit IRC | 21:29 | |
| *** jamesmcarthur has quit IRC | 21:30 | |
| *** jamesmcarthur has joined #openstack-infra | 21:30 | |
| *** exsdev has joined #openstack-infra | 21:32 | |
| *** notmyname has joined #openstack-infra | 21:32 | |
| *** ddurst has joined #openstack-infra | 21:39 | |
| *** exsdev has quit IRC | 21:47 | |
| *** markvoelker has quit IRC | 21:47 | |
| *** munimeha1 has quit IRC | 21:50 | |
| *** jdelaros1 has joined #openstack-infra | 21:54 | |
| *** eharney has quit IRC | 21:56 | |
| openstackgerrit | Merged zuul/zuul-jobs master: Only use RHEL8 deps repo on Red Hat systems newer than 7 https://review.opendev.org/679126 | 21:57 |
| *** exsdev has joined #openstack-infra | 21:57 | |
| *** iurygregory has quit IRC | 21:58 | |
| openstackgerrit | Merged zuul/zuul master: Fix "Set failed, unreachable, skipped statuses in json plugin" https://review.opendev.org/680159 | 22:02 |
| *** smarcet has quit IRC | 22:04 | |
| *** trident has quit IRC | 22:05 | |
| *** jamesmcarthur has quit IRC | 22:07 | |
| *** exsdev has quit IRC | 22:09 | |
| *** jamesmcarthur has joined #openstack-infra | 22:10 | |
| *** jamesmcarthur has quit IRC | 22:10 | |
| *** mriedem has quit IRC | 22:11 | |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** trident has joined #openstack-infra | 22:14 | |
| *** slaweq has quit IRC | 22:15 | |
| *** exsdev has joined #openstack-infra | 22:17 | |
| *** exsdev has quit IRC | 22:25 | |
| *** jhesketh has quit IRC | 22:27 | |
| *** psachin has quit IRC | 22:31 | |
| *** jhesketh has joined #openstack-infra | 22:33 | |
| *** markvoelker has joined #openstack-infra | 22:35 | |
| *** markvoelker has quit IRC | 22:40 | |
| *** KeithMnemonic has quit IRC | 22:43 | |
| *** KeithMnemonic1 has joined #openstack-infra | 22:43 | |
| *** dchen has joined #openstack-infra | 22:46 | |
| *** exsdev has joined #openstack-infra | 22:46 | |
| *** dmsimard1 has joined #openstack-infra | 22:47 | |
| *** dmsimard has quit IRC | 22:47 | |
| *** dmsimard1 is now known as dmsimard | 22:50 | |
| *** tkajinam has joined #openstack-infra | 22:58 | |
| *** exsdev has quit IRC | 23:02 | |
| *** tosky has quit IRC | 23:02 | |
| *** rcernin has joined #openstack-infra | 23:04 | |
| *** exsdev has joined #openstack-infra | 23:06 | |
| *** exsdev has quit IRC | 23:10 | |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** bdodd has quit IRC | 23:14 | |
| *** slaweq has quit IRC | 23:16 | |
| *** jamesmcarthur has joined #openstack-infra | 23:17 | |
| *** jamesmcarthur has quit IRC | 23:18 | |
| rm_work | Anyone know if speaker codes went out yet for Shanghai> | 23:22 |
| rm_work | ? | 23:22 |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard-webclient master: Adds Migration Docs to Dashboard https://review.opendev.org/680235 | 23:23 |
| rm_work | diablo_rojo: ^^ | 23:31 |
| pabelanger | ianw: is there any progress on fedora-30 DIBs? | 23:31 |
| *** dchen has quit IRC | 23:32 | |
| pabelanger | ianw: ah, looks like it might be working | 23:32 |
| johnsom | FYI, I'm still getting these DNS errors on jobs: | 23:33 |
| johnsom | 2019-09-04 22:06:04.582298 | controller | E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/universe/y/yum/yum_3.4.3-3_all.deb Temporary failure resolving 'mirror.regionone.fortnebula.opendev.org' | 23:33 |
| johnsom | Source log: https://79636ab1f0eaf6899dc0-686811860e75a35eabcecb5697905253.ssl.cf2.rackcdn.com/665029/30/check/octavia-v2-dsvm-scenario/51891fa/job-output.txt | 23:33 |
| *** dchen has joined #openstack-infra | 23:33 | |
| pabelanger | is /etc/resolv.conf setup to use localhost ? | 23:33 |
| johnsom | It looks like about 26 packages could not be downloaded. | 23:34 |
| johnsom | Yes, nameserver 127.0.0.1 | 23:34 |
| pabelanger | johnsom: you should also be using the regional mirror | 23:34 |
| pabelanger | why it is going to ubuntu.com directly? | 23:34 |
| pabelanger | https://79636ab1f0eaf6899dc0-686811860e75a35eabcecb5697905253.ssl.cf2.rackcdn.com/665029/30/check/octavia-v2-dsvm-scenario/51891fa/controller/logs/resolv_conf.txt.gz | 23:35 |
| pabelanger | yah resolv.conf looks okay | 23:35 |
| johnsom | It gave up E: Failed to fetch http://mirror.regionone.fortnebula.opendev.org/ubuntu/pool/main/l/lzo2/liblzo2-2_2.08-1.2_amd64.deb Temporary failure resolving 'mirror.regionone.fortnebula.opendev.org' | 23:35 |
| pabelanger | k, then next thing to check is unbound logs | 23:35 |
| pabelanger | wonder if forwards are blocked or something | 23:35 |
| pabelanger | https://79636ab1f0eaf6899dc0-686811860e75a35eabcecb5697905253.ssl.cf2.rackcdn.com/665029/30/check/octavia-v2-dsvm-scenario/51891fa/controller/logs/unbound_log.txt.gz | 23:36 |
| johnsom | Yeah, we have randomly seen these for a while. Last time the unbound log was disabled. It looks like there is one now. So that is good, I will take a look | 23:37 |
| pabelanger | I'm not sure how ipv6 is setup on fortnebula, if direct connection or nat | 23:37 |
| pabelanger | johnsom: http://paste.openstack.org/show/770958/ | 23:38 |
| pabelanger | looks like routing issue | 23:38 |
| johnsom | sendto failed: Network is unreachable Yep, looks like v6 was broken | 23:38 |
| pabelanger | unbound should fail back to ipv4, but can't remember | 23:38 |
| johnsom | https://www.irccloud.com/pastebin/2N4BJBOJ/ | 23:39 |
| johnsom | Same message for the regional mirro | 23:39 |
| *** jamesmcarthur has joined #openstack-infra | 23:40 | |
| pabelanger | yah, looks like issue with provider network | 23:42 |
| pabelanger | I'm out of the loop on fortnebula | 23:43 |
| pabelanger | doesn't look like any ipv4 requests are logged | 23:43 |
| johnsom | It looks like it tried a google DNS and a cloudflare DNS server. using IPv6 | 23:47 |
| pabelanger | yah, if cloud supports ipv6, we set that up first | 23:48 |
| pabelanger | but we can force ipv4, but usually means something is broken | 23:48 |
| pabelanger | will defer to clarkb on that | 23:48 |
| clarkb | aroo? | 23:49 |
| clarkb | give me a few minutes to catch up I'm doing visa things | 23:49 |
| johnsom | clarkb It's the same DNS issue we have chatted about before, just this time we have unbound logs. | 23:50 |
| clarkb | ipv6 is direct connection out of fortnebula via hurricane electric | 23:51 |
| clarkb | pabelanger: unbound does not fail back to ipv45 | 23:51 |
| clarkb | pabelanger: we thought it did but instead what it does is round robin all resolvers so we stopped trying to dual stack it | 23:51 |
| pabelanger | Oh, thought it did | 23:51 |
| pabelanger | ah | 23:51 |
| *** xenos76 has quit IRC | 23:51 | |
| pabelanger | odd, I can see dual stack dns servers in unbound log | 23:52 |
| clarkb | because FN is an ipv6 cloud we configure it with ipv6 only resolvers | 23:52 |
| clarkb | we use cloudflare and google ipv6 | 23:52 |
| pabelanger | 1.0.0.1 and 8.8.8.8 | 23:52 |
| clarkb | pabelanger: that is what we boot with but when the job starts it should switch over | 23:52 |
| pabelanger | ack | 23:52 |
| pabelanger | well, TIL | 23:52 |
| *** trident has quit IRC | 23:52 | |
| clarkb | and the reason for that is we found that ipv4 dns through NAT was super unreliable back when we had osic | 23:53 |
| clarkb | switching osic over to ipv6 only fixed all the dns trouble there | 23:53 |
| pabelanger | yah | 23:54 |
| clarkb | first thing we should do is increase the TTL on those records | 23:54 |
| pabelanger | I thought we exposed that | 23:54 |
| pabelanger | via job config | 23:54 |
| pabelanger | looking | 23:54 |
| pabelanger | Oh, or do you mean TTL on server side | 23:56 |
| pabelanger | yah, looks like 300 seconds | 23:57 |
| *** diablo_rojo has quit IRC | 23:57 | |
| openstackgerrit | Clark Boylan proposed opendev/zone-opendev.org master: Increase mirror ttls to an hour https://review.opendev.org/680236 | 23:58 |
| clarkb | thats a patch to address that | 23:58 |
| pabelanger | Oh, right. We no longer use rackspace for that | 23:58 |
| *** mattw4 has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!