| *** jamesmcarthur has quit IRC | 00:04 | |
| *** jamesmcarthur has joined #openstack-infra | 00:30 | |
| *** jamesmcarthur has quit IRC | 00:43 | |
| *** rfolco has joined #openstack-infra | 00:52 | |
| *** jamesmcarthur has joined #openstack-infra | 00:59 | |
| *** jamesmcarthur has quit IRC | 01:05 | |
| *** jamesmcarthur has joined #openstack-infra | 01:05 | |
| *** rfolco has quit IRC | 01:45 | |
| *** jamesmcarthur has quit IRC | 01:47 | |
| *** yamamoto has quit IRC | 01:48 | |
| *** jamesmcarthur has joined #openstack-infra | 01:49 | |
| *** jamesmcarthur has quit IRC | 01:54 | |
| openstackgerrit | Merged openstack/project-config master: Adapt branches for sdk projects in gerritbot https://review.opendev.org/704244 | 01:54 |
|---|---|---|
| *** auristor has quit IRC | 02:01 | |
| *** auristor has joined #openstack-infra | 02:02 | |
| *** openstackgerrit has quit IRC | 02:04 | |
| *** armax has joined #openstack-infra | 02:10 | |
| *** openstackgerrit has joined #openstack-infra | 02:15 | |
| openstackgerrit | Monty Taylor proposed zuul/zuul-jobs master: Add support for installing python with pyenv https://review.opendev.org/704266 | 02:15 |
| mordred | fungi, yoctozepto: ^^ there's a sake-of-argument patch to add support for installing arbitrary python versions with pyenv | 02:15 |
| *** jamesmcarthur has joined #openstack-infra | 02:18 | |
| fungi | neat! | 02:23 |
| fungi | presumably still fragile due to lack of caching and increased job runtime to build from source? | 02:23 |
| * fungi still hasn't tried pyenv | 02:23 | |
| *** jamesmcarthur has quit IRC | 02:24 | |
| *** hwoarang has quit IRC | 02:27 | |
| *** hwoarang has joined #openstack-infra | 02:27 | |
| mordred | fungi: yah - it would be more solid if we pre-installed a set of pythons with pyenv in the images and then simply used the role to "Activate" one of them | 02:32 |
| *** ociuhandu has joined #openstack-infra | 02:33 | |
| mordred | but - since it allows any arbitrary point release we'd wind up having to make calls about which specific versions we wanted to pre-install , etc ... | 02:33 |
| mordred | but - who knows - maybe it's worth it? | 02:33 |
| *** ociuhandu has quit IRC | 02:33 | |
| *** ociuhandu has joined #openstack-infra | 02:34 | |
| *** hwoarang has quit IRC | 02:37 | |
| *** hwoarang has joined #openstack-infra | 02:39 | |
| *** ociuhandu has quit IRC | 02:40 | |
| *** jamesmcarthur has joined #openstack-infra | 02:52 | |
| *** jamesmcarthur has quit IRC | 02:56 | |
| *** yamamoto has joined #openstack-infra | 02:57 | |
| *** jamesmcarthur has joined #openstack-infra | 03:31 | |
| *** jamesmcarthur has quit IRC | 03:32 | |
| *** jamesmcarthur has joined #openstack-infra | 03:32 | |
| *** psachin has joined #openstack-infra | 03:35 | |
| *** ramishra has joined #openstack-infra | 03:46 | |
| *** jhesketh has quit IRC | 03:56 | |
| *** ykarel|away is now known as ykarel | 04:07 | |
| *** udesale has joined #openstack-infra | 04:25 | |
| *** jamesmcarthur has quit IRC | 04:44 | |
| *** raukadah is now known as chkumar|rover | 04:45 | |
| *** jtomasek has joined #openstack-infra | 05:28 | |
| *** ociuhandu has joined #openstack-infra | 05:30 | |
| *** ociuhandu has quit IRC | 05:36 | |
| *** zxiiro has joined #openstack-infra | 05:51 | |
| *** jtomasek has quit IRC | 05:53 | |
| *** slaweq__ has joined #openstack-infra | 06:03 | |
| *** surpatil has joined #openstack-infra | 06:21 | |
| *** tkajinam has quit IRC | 07:02 | |
| *** lmiccini has joined #openstack-infra | 07:02 | |
| *** hwoarang has quit IRC | 07:03 | |
| *** hwoarang has joined #openstack-infra | 07:04 | |
| *** tkajinam has joined #openstack-infra | 07:04 | |
| *** AJaeger has quit IRC | 07:05 | |
| *** AJaeger has joined #openstack-infra | 07:10 | |
| *** pgaxatte has joined #openstack-infra | 07:22 | |
| *** ralonsoh has joined #openstack-infra | 07:24 | |
| *** dpawlik has joined #openstack-infra | 07:26 | |
| *** florianf has joined #openstack-infra | 07:32 | |
| *** ykarel is now known as ykarel|lunch | 07:33 | |
| *** piotrowskim has joined #openstack-infra | 07:38 | |
| *** dpawlik has quit IRC | 07:52 | |
| *** tkajinam_ has joined #openstack-infra | 07:52 | |
| *** tkajinam has quit IRC | 07:55 | |
| *** tosky has joined #openstack-infra | 07:56 | |
| *** dpawlik has joined #openstack-infra | 07:59 | |
| *** iurygregory has joined #openstack-infra | 08:01 | |
| *** jtomasek has joined #openstack-infra | 08:08 | |
| *** witek has joined #openstack-infra | 08:09 | |
| *** ykarel|lunch is now known as ykarel | 08:11 | |
| *** tesseract has joined #openstack-infra | 08:13 | |
| *** lbragstad has quit IRC | 08:15 | |
| *** lbragstad has joined #openstack-infra | 08:15 | |
| *** cjloader has quit IRC | 08:16 | |
| *** prometheanfire has quit IRC | 08:16 | |
| *** prometheanfire has joined #openstack-infra | 08:17 | |
| *** tkajinam_ has quit IRC | 08:18 | |
| *** rpittau|afk is now known as rpittau | 08:18 | |
| *** pcaruana has joined #openstack-infra | 08:25 | |
| openstackgerrit | Merged opendev/irc-meetings master: Move Neutron CI meeting to Wednesday https://review.opendev.org/704200 | 08:34 |
| *** ociuhandu has joined #openstack-infra | 08:47 | |
| *** jpena|off is now known as jpena | 08:51 | |
| *** lucasagomes has joined #openstack-infra | 08:54 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Optionally support mitogen for job execution https://review.opendev.org/657024 | 08:55 |
| *** apetrich has joined #openstack-infra | 08:56 | |
| *** vesper11 has quit IRC | 09:01 | |
| *** vesper11 has joined #openstack-infra | 09:05 | |
| *** sajauddin has joined #openstack-infra | 09:07 | |
| *** yamamoto has quit IRC | 09:13 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Enforce sql connections for scheduler and web https://review.opendev.org/630472 | 09:20 |
| *** ccamacho has joined #openstack-infra | 09:25 | |
| *** pkopec has joined #openstack-infra | 09:36 | |
| *** derekh has joined #openstack-infra | 09:37 | |
| *** dpawlik has quit IRC | 09:40 | |
| *** xek has joined #openstack-infra | 09:42 | |
| *** ociuhandu has quit IRC | 09:50 | |
| *** kevko has quit IRC | 09:51 | |
| *** sajauddin has quit IRC | 10:18 | |
| *** gfidente has joined #openstack-infra | 10:18 | |
| *** dtantsur|afk is now known as dtantsur | 10:23 | |
| *** sshnaidm is now known as sshnaidm|afk | 10:27 | |
| *** surpatil has quit IRC | 10:32 | |
| *** jtomasek has quit IRC | 10:50 | |
| *** aedc has quit IRC | 10:56 | |
| *** aedc has joined #openstack-infra | 10:57 | |
| *** udesale has quit IRC | 10:58 | |
| openstackgerrit | Thierry Carrez proposed openstack/project-config master: check-release-approval job improvements https://review.opendev.org/704306 | 11:10 |
| *** rpittau is now known as rpittau|bbl | 11:21 | |
| *** pkopec has quit IRC | 11:24 | |
| *** Lucas_Gray has joined #openstack-infra | 11:25 | |
| *** lpetrut has joined #openstack-infra | 11:32 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:34 | |
| *** gfidente is now known as gfidente|afj | 11:39 | |
| *** gfidente|afj is now known as gfidente|afk | 11:39 | |
| *** florianf has quit IRC | 11:40 | |
| *** zxiiro has quit IRC | 11:50 | |
| *** rfolco has joined #openstack-infra | 12:00 | |
| *** yamamoto has joined #openstack-infra | 12:04 | |
| *** yamamoto has quit IRC | 12:09 | |
| *** yamamoto_ has joined #openstack-infra | 12:09 | |
| *** ociuhandu has joined #openstack-infra | 12:11 | |
| *** ociuhandu has quit IRC | 12:19 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul-jobs master: Add a markdownlint job and role https://review.opendev.org/607691 | 12:23 |
| *** jpena is now known as jpena|lunch | 12:28 | |
| *** rh-jelabarre has joined #openstack-infra | 12:28 | |
| openstackgerrit | Thierry Carrez proposed openstack/project-config master: check-release-approval job improvements https://review.opendev.org/704306 | 12:29 |
| *** ociuhandu has joined #openstack-infra | 12:30 | |
| *** noama has joined #openstack-infra | 12:37 | |
| *** gfidente|afk is now known as gfidente | 12:42 | |
| *** rosmaita has joined #openstack-infra | 12:48 | |
| *** rlandy has joined #openstack-infra | 12:50 | |
| *** beekneemech has joined #openstack-infra | 12:51 | |
| *** bnemec has quit IRC | 12:51 | |
| *** Goneri has quit IRC | 12:53 | |
| *** rpittau|bbl is now known as rpittau | 12:55 | |
| *** weshay_ is now known as weshay|ruck | 13:05 | |
| *** ykarel is now known as ykarel|afk | 13:06 | |
| *** jamesmcarthur has joined #openstack-infra | 13:06 | |
| *** aedc has quit IRC | 13:13 | |
| *** aedc has joined #openstack-infra | 13:14 | |
| *** ociuhandu has quit IRC | 13:14 | |
| *** aedc has quit IRC | 13:17 | |
| *** ramishra has quit IRC | 13:20 | |
| *** aedc has joined #openstack-infra | 13:21 | |
| *** ramishra has joined #openstack-infra | 13:21 | |
| *** jamesmcarthur has quit IRC | 13:21 | |
| *** ramishra has quit IRC | 13:21 | |
| *** ramishra has joined #openstack-infra | 13:21 | |
| *** jamesmcarthur has joined #openstack-infra | 13:27 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Enforce sql connections for scheduler and web https://review.opendev.org/630472 | 13:28 |
| *** psachin has quit IRC | 13:29 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| *** ociuhandu has joined #openstack-infra | 13:34 | |
| *** ykarel|afk is now known as ykarel | 13:35 | |
| *** jamesmcarthur has quit IRC | 13:37 | |
| openstackgerrit | Lenny Verkhovsky proposed openstack/project-config master: Updated x/networking-mlnx project for pypi and neutron https://review.opendev.org/704321 | 13:39 |
| *** Goneri has joined #openstack-infra | 13:44 | |
| *** roman_g has joined #openstack-infra | 13:46 | |
| *** jtomasek has joined #openstack-infra | 13:48 | |
| *** jamesmcarthur has joined #openstack-infra | 13:49 | |
| openstackgerrit | Jan Kubovy proposed zuul/zuul master: Enforce sql connections for scheduler and web https://review.opendev.org/630472 | 13:50 |
| *** smcginnis has quit IRC | 13:50 | |
| *** smcginnis has joined #openstack-infra | 13:51 | |
| *** aedc has quit IRC | 13:52 | |
| *** zxiiro has joined #openstack-infra | 13:52 | |
| *** smcginnis is now known as smcginnis|FOSDEM | 13:53 | |
| *** jtomasek has quit IRC | 13:54 | |
| *** yamamoto_ has quit IRC | 13:56 | |
| *** aedc has joined #openstack-infra | 13:57 | |
| *** yamamoto has joined #openstack-infra | 13:58 | |
| EmilienM | hello infra, http://codesearch.openstack.org seems down to me | 13:58 |
| *** jtomasek has joined #openstack-infra | 14:01 | |
| *** udesale has joined #openstack-infra | 14:03 | |
| frickler | EmilienM: seems to work for me right now, do you still see it down? | 14:09 |
| *** Lucas_Gray has quit IRC | 14:11 | |
| EmilienM | frickler: mhh then it's on my side | 14:12 |
| *** haleyb has joined #openstack-infra | 14:14 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Don't reset file list to None in updateChange https://review.opendev.org/704328 | 14:20 |
| openstackgerrit | Merged zuul/nodepool master: kubernetes: add release note about service account token issue https://review.opendev.org/703869 | 14:27 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Annotate logs in launcher https://review.opendev.org/703558 | 14:29 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Annotate logs in node request handler https://review.opendev.org/703559 | 14:29 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Include event id in node request listings https://review.opendev.org/703560 | 14:29 |
| openstackgerrit | Simon Westphahl proposed zuul/nodepool master: Annotate logs in zk module https://review.opendev.org/703561 | 14:29 |
| *** bdodd has joined #openstack-infra | 14:32 | |
| openstackgerrit | sebastian marcet proposed opendev/puppet-openstackid master: Fixed artisan command permissions https://review.opendev.org/704329 | 14:35 |
| *** hashar has joined #openstack-infra | 14:38 | |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: Enhance some logging messages https://review.opendev.org/704331 | 14:39 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 14:39 |
| *** jtomasek has quit IRC | 14:40 | |
| *** jtomasek has joined #openstack-infra | 14:45 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Handle draft pull requests in canMerge https://review.opendev.org/704150 | 14:55 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.opendev.org/644557 | 14:55 |
| *** paladox is now known as paladox|UKInEU | 14:56 | |
| *** cjloader has joined #openstack-infra | 15:08 | |
| *** ykarel is now known as ykarel|away | 15:11 | |
| *** jamesmcarthur has quit IRC | 15:17 | |
| *** noama has quit IRC | 15:26 | |
| *** hashar has quit IRC | 15:30 | |
| *** artom has joined #openstack-infra | 15:31 | |
| *** pgaxatte has quit IRC | 15:38 | |
| *** jackedin has joined #openstack-infra | 15:42 | |
| fungi | ianw: not sure if you've clocked out already, but rackspace opened a ticket to let us know that they need to perform a reboot migration for static.openstack.org at 20:15 utc on the 30th | 15:48 |
| fungi | but said we can work with them to do it sooner if we want | 15:49 |
| fungi | mostly i want to be sure it doesn't impact your migration efforts | 15:49 |
| fungi | i wonder if we shouldn't try to scale off of and detach all but one of the cinder volumes before then, to reduce chances it chokes | 15:51 |
| fungi | there are 14x 1tb volumes currently in the main vg, but it has only ~625gb of logical volumes allocated on it now | 15:52 |
| fungi | er, i take that back, technically the logs volume is still there | 15:56 |
| fungi | it's effectively except for a robots.txt and three html help files we put there. we could easily move those into the rootfs (in case they're still needed), umount and lvremove it | 15:57 |
| fungi | then pvmove the remaining two volumes (tarballs and static) live to the first pv, vgreduce off the other 13 and detach them | 15:58 |
| *** jtomasek has quit IRC | 16:05 | |
| clarkb | fungi: except -> empty? | 16:06 |
| fungi | effectively empty except for, yeah | 16:07 |
| fungi | there's a 14tb filesystem with 4 tiny text files in it | 16:07 |
| clarkb | ya we should probably trim that down now that we've been on swift for a while | 16:08 |
| fungi | i think the hope was that we'd just get the last of the content migrated off and then delete it all at once | 16:11 |
| fungi | but with rackspace wanting to do a reboot migration i'm slightly worried | 16:12 |
| fungi | so willing to work on scaling this down to one cinder volume | 16:12 |
| fungi | there are also something like 14k empty directories on it, but i don't think they're needed | 16:13 |
| fungi | apparently our cleanup cronjob didn't do a perfect job of removing empty dirs | 16:13 |
| fungi | yeesh, we still have the old pypi mirror in /srv/static too | 16:15 |
| *** ociuhandu has quit IRC | 16:16 | |
| *** jackedin has quit IRC | 16:19 | |
| *** udesale has quit IRC | 16:19 | |
| fungi | okay, those files have been copied into a directory in the parent volume now and /srv/static/logs unmounted | 16:22 |
| fungi | i took the relevant vhosts out of apache's config briefly and reloaded it so it wasn't serving anything from that directory while i surgeried | 16:23 |
| fungi | now i'll work on deleting the old redundant empty logs volume from the vg | 16:24 |
| fungi | Logical volume "logs" successfully removed | 16:25 |
| clarkb | \o/ | 16:25 |
| fungi | now according to `lvdisplay --maps` we have the static volume on /dev/xvdm1 and the tarballs volume spread over /dev/xvdg1, /dev/xvdj1 and /dev/xvdn1 | 16:26 |
| fungi | i'll work on pvmoving those extents to /dev/xvdb1 (the highest numbered cinder device) | 16:28 |
| fungi | that can be done live in the background while it's serving content | 16:28 |
| *** gyee has joined #openstack-infra | 16:29 | |
| fungi | i've got `pvmove /dev/xvdg1 /dev/xvdb1` underway first, in a root screen session on static.o.o | 16:32 |
| *** ccamacho has quit IRC | 16:34 | |
| fungi | it's at 10% done already, so shouldn't take too long. once it finishes i'll do the other three and can then work on the vgreduce step | 16:38 |
| AJaeger | fungi: you can delete http://files.openstack.org/docs-old/ - that is ancient backup... | 16:39 |
| fungi | yep, same with pyip-old | 16:40 |
| fungi | unfortunately deleting things from a filesystem on a logical volume we're keeping doesn't really speed anything up | 16:40 |
| AJaeger | ;) | 16:41 |
| fungi | i could instead have created a smaller volume and just copied the things we're keeping into it, then removed the old volume, but that's more fiddly | 16:41 |
| fungi | pvmove is much easier and (theoretically) causes no downtime for the content, but since it operates at the block layer it has no knowledge of what's allocated by the filesystem | 16:42 |
| fungi | also all the content, even the cruft, easily fits into one cinder volume, so trying to shrink it doesn't really accomplish much at this stage | 16:43 |
| *** lmiccini has quit IRC | 16:44 | |
| *** jtomasek has joined #openstack-infra | 16:51 | |
| *** jamesmcarthur has joined #openstack-infra | 16:51 | |
| *** smarcet has joined #openstack-infra | 16:53 | |
| *** chkumar|rover is now known as raukadah | 16:54 | |
| smarcet | fungi: afternoon trying to merge this change https://review.opendev.org/#/c/661661/ but this job its started to failing https://zuul.opendev.org/t/openstack/build/6c6a13d97b574d9baaed408e73028881 not sure if i need to change something from my side , could you shed some light ? :) | 16:55 |
| AJaeger | smarcet: promote-tox-docs-infra is failing? Mmh... | 16:56 |
| *** dtantsur is now known as dtantsur|afk | 16:57 | |
| fungi | looking | 16:57 |
| AJaeger | https://zuul.opendev.org/t/openstack/build/6c6a13d97b574d9baaed408e73028881/log/job-output.txt#73 | 16:57 |
| AJaeger | fungi, smarcet ^ | 16:57 |
| smarcet | yes i saw that , but not sure | 16:57 |
| smarcet | download-artifact/tasks/main.yaml | 16:57 |
| smarcet | its a job that i need to change at zuul jobs repo ? | 16:58 |
| *** iurygregory has quit IRC | 16:58 | |
| AJaeger | smarcet: it will not work since promote-tox-docs-infra neeeds opendev-tox-docs in gate instead of tox-docs | 16:58 |
| AJaeger | Please change the docs jobs | 16:58 |
| smarcet | ok | 16:58 |
| smarcet | thx u for the info | 16:58 |
| *** jtomasek has quit IRC | 17:00 | |
| openstackgerrit | Andreas Jaeger proposed openstack/project-config master: Fix description for promote-tox-docs-infra https://review.opendev.org/704359 | 17:01 |
| AJaeger | oh, found a bug in the job description ^ | 17:01 |
| *** yamamoto has quit IRC | 17:06 | |
| *** hwoarang has quit IRC | 17:08 | |
| *** paladox|UKInEU is now known as paladox | 17:09 | |
| *** hwoarang has joined #openstack-infra | 17:09 | |
| openstackgerrit | Clark Boylan proposed zuul/zuul master: Be more specific with remaining time https://review.opendev.org/704191 | 17:13 |
| *** ociuhandu has joined #openstack-infra | 17:14 | |
| *** hashar has joined #openstack-infra | 17:18 | |
| clarkb | Looks like the problem with nb03 talking to the new linaro cloud persists. I imagine holidays are impacting rtt on that. | 17:24 |
| clarkb | I'm working through my reviews and reviewing changes from others then would like to start focuses on the trusty upgrades | 17:25 |
| *** ociuhandu_ has joined #openstack-infra | 17:32 | |
| *** lucasagomes has quit IRC | 17:33 | |
| *** ociuhandu has quit IRC | 17:36 | |
| *** ociuhandu_ has quit IRC | 17:37 | |
| *** tesseract has quit IRC | 17:38 | |
| fungi | the pvmove of xvdg1 has finished, starting on xvdj1 now | 17:38 |
| *** beekneemech is now known as bnemec-ooo | 17:39 | |
| *** yamamoto has joined #openstack-infra | 17:43 | |
| *** witek has quit IRC | 17:44 | |
| *** slaweq__ is now known as slaweq | 17:45 | |
| *** yamamoto has quit IRC | 17:52 | |
| *** mattw4 has joined #openstack-infra | 17:53 | |
| smarcet | fungi: Ajaeger: any way to re trigger the promote for this https://review.opendev.org/#/c/661661/ i tried the recheck but no dice | 17:54 |
| AJaeger | smarcet: there's no way - and no need | 17:55 |
| AJaeger | The docs are overwritten every time, you only want the latest version | 17:55 |
| smarcet | AJaeger: ok undertood thx for the info :) | 17:55 |
| clarkb | I think the job is buggy too? | 17:55 |
| AJaeger | smarcet: https://review.opendev.org/#/c/704360/ should have done the right thing - has it? Is everything published? | 17:56 |
| AJaeger | smarcet: https://docs.openstack.org/infra/openstackid/ is published 30 mins ok, so all fine. | 17:56 |
| smarcet | AJaeger: thx :) | 17:57 |
| *** jpena is now known as jpena|off | 17:57 | |
| AJaeger | smarcet: just wonder whether that is the right location for the documents, since your repo is not an official part of OpenStack according to the governance repo | 17:57 |
| smarcet | AJaeger: not sure | 17:58 |
| fungi | it's where we had been publishing those documents previously, so it's probably fine for now | 17:58 |
| fungi | maybe worth revisiting after the infra/opendev governance changes go through | 17:59 |
| AJaeger | yep | 17:59 |
| AJaeger | bbl | 17:59 |
| fungi | there's going to be a bunch of documentation which needs to get rehomed anyway | 17:59 |
| openstackgerrit | Merged zuul/nodepool master: Annotate logs in launcher https://review.opendev.org/703558 | 18:05 |
| fungi | the pvmove of xvdj1 has finished, starting on xvdm1 now | 18:09 |
| openstackgerrit | Merged zuul/nodepool master: Annotate logs in node request handler https://review.opendev.org/703559 | 18:14 |
| *** yamamoto has joined #openstack-infra | 18:14 | |
| openstackgerrit | David Shrewsbury proposed zuul/nodepool master: Remove duplicate log info from OpenStack driver https://review.opendev.org/704379 | 18:14 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: Add details to waitUntilSettled() debug log https://review.opendev.org/704380 | 18:17 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tests: throttle after all builds have reported https://review.opendev.org/704381 | 18:17 |
| *** rpittau is now known as rpittau|afk | 18:18 | |
| *** gfidente is now known as gfidente|afk | 18:19 | |
| *** rkukura has joined #openstack-infra | 18:21 | |
| openstackgerrit | Merged zuul/nodepool master: Include event id in node request listings https://review.opendev.org/703560 | 18:25 |
| yoctozepto | 03:32:08 <mordred> fungi: yah - it would be more solid if we pre-installed a set of pythons with pyenv in the images and then simply used the role to "Activate" one of them | 18:27 |
| yoctozepto | +1 | 18:27 |
| yoctozepto | 03:33:31 <mordred> but - since it allows any arbitrary point release we'd wind up having to make calls about which specific versions we wanted to pre-install , etc ... | 18:27 |
| yoctozepto | images could always install latest per 3.x | 18:27 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Update screenshot for gerrit profile in quick-start https://review.opendev.org/704222 | 18:28 |
| yoctozepto | then by activating e.g. 3.7 you would be getting latest 3.7.y | 18:28 |
| yoctozepto | this would make jobs independent from distro python though | 18:28 |
| yoctozepto | which may or may not be expected/desired | 18:28 |
| fungi | the pvmove of xvdm1 has finished, starting on xvdn1 now | 18:29 |
| yoctozepto | fwiw, I am not a fan of distro-provided programming platforms so I usually run pyenv or analogs for other langs | 18:29 |
| fungi | yoctozepto: i basically do the same, though i just git checkout the latest patch release tag for each python minor release and then do a build ending with make altinstall into a subtree of my homedir where i symlink each interpreter from ~/bin/ | 18:31 |
| yoctozepto | fungi: well, software like pyenv does management for you | 18:32 |
| openstackgerrit | James E. Blair proposed zuul/zuul master: Fix path in quick-start localtest https://review.opendev.org/704385 | 18:32 |
| fungi | in this case the "management" is so simple i haven't bothered to look at pyenv | 18:33 |
| yoctozepto | fungi: well, it can replace python/python3 with the chosen version | 18:33 |
| fungi | but it's a distinct choice projects need to make. are they testing whether their source code works with specific versions of python or a specific linux distribution. taking openstack as an example, it depends on far more from a given system than just the python interpreter, so it makes sense that they'd want to test "the ubuntu bionic python version" rather than specifically "python 3.6" | 18:34 |
| yoctozepto | fungi: indeed, that's my issue | 18:34 |
| *** sshnaidm is now known as sshnaidm|afk | 18:35 | |
| yoctozepto | otoh, xenial still generally works perfectly fine if you don't need latest and greatest lower-level features, and bionic will as well even if focal comes out | 18:35 |
| fungi | keeping in mind that rhel 6 and by extension centos 6 backported all manner of early python3 nonsense into their python2 | 18:35 |
| yoctozepto | yet they would provide different python versions | 18:35 |
| fungi | so "distro python" and "upstream python" can differ wildly in behavior even when they claim the same version numbers | 18:36 |
| yoctozepto | yes, unfortunately | 18:36 |
| yoctozepto | and it makes it harder to keep compat between distros | 18:36 |
| yoctozepto | but I guess it's still tc issue to hanlde where openstack should test against vanilla python or distro python | 18:37 |
| yoctozepto | whether* | 18:37 |
| fungi | well, for sure it means that "tested on upstream python 3.6" doesn't necessarily mean "works on the python 3.6 in ubuntu bionic" | 18:37 |
| yoctozepto | indeed | 18:37 |
| yoctozepto | though obviously it's braindead easy to get vanilla python 3.6 on ubuntu bionic | 18:37 |
| openstackgerrit | Merged zuul/nodepool master: Annotate logs in zk module https://review.opendev.org/703561 | 18:37 |
| fungi | yup | 18:38 |
| fungi | (and 3.5, and 3.7 and 3.8, and 3.9 alpha...) | 18:39 |
| fungi | building 3.4 on bionic is a challenge though due to needing older libssl | 18:39 |
| fungi | same on recent debian | 18:39 |
| fungi | granted, 3.4 is also eol since last year, so i'm not that inclined to care | 18:40 |
| yoctozepto | we stopped caring about 3.5 in master already | 18:43 |
| yoctozepto | well, most places | 18:43 |
| jrosser | I am fighting this stuff for my second week now trying to unbkreak osa stable branches in light of neutron-lib/tempest and now keystone-tempest-plug-in wanting python things that cannot be satisfied by the distro python | 18:47 |
| jrosser | having some official position on what’s supposed to work would be super helpful | 18:48 |
| clarkb | jrosser: can you be more specific? I'm not sure what isn't working | 18:48 |
| yoctozepto | jrosser: req/u-c were updated | 18:48 |
| yoctozepto | so at least neutron-lib/tempest should be fine atm | 18:49 |
| jrosser | yeah they were, and that is fixed, now I see sqlalchemy not new enough in bionic, for example | 18:49 |
| yoctozepto | lolz | 18:49 |
| *** rkukura has quit IRC | 18:49 | |
| yoctozepto | is it because of u-c? | 18:49 |
| yoctozepto | bionic is current target of support | 18:50 |
| clarkb | oh I see, I think this is a qusetion for the requirements team | 18:50 |
| yoctozepto | and it has 3.6 and stuff... | 18:50 |
| clarkb | my understanding of it is that master isn't beholden to what is in distros | 18:50 |
| jrosser | I don’t know yet so much broken I’m kinda losing th plot fixing this across all stable branches at the same time | 18:50 |
| clarkb | instead it can be forward looking and require newer dependencies | 18:50 |
| clarkb | stable however I think is expected to work on distros? | 18:50 |
| openstackgerrit | Jeremy Stanley proposed zuul/zuul master: Flesh out the glossary significantly https://review.opendev.org/704391 | 18:50 |
| clarkb | but ya somethign to bring up with the requirements team | 18:51 |
| yoctozepto | well, back to "we need latest and greatest for tempest due to its branchless nature" | 18:54 |
| yoctozepto | fungi, clarkb: https://review.opendev.org/703476 | 18:54 |
| yoctozepto | I coauthored this with gmann and still have mixed feelings | 18:54 |
| yoctozepto | one part says ok, another says wtos ("what the openstack?!") | 18:54 |
| *** lpetrut has quit IRC | 18:55 | |
| clarkb | right so if this is specific to tempest then aiui you are expected to use the tag version of tempest if master stops working | 18:55 |
| fungi | yeah, i suspect that anything which is branchless and is intended to work in older environments than ussuri supports may need to test on and be compatible with older stuff than openstack/requirements master branch demands | 18:56 |
| clarkb | another option (which is what devstack does) is to install tempest in a venv independent of the system under test | 18:56 |
| fungi | part of the problem is that tempest wants to support coinstallation onto the same platforms as supported stable branches, but depends on libraries (either directly or indirectly through plugins) which demand newer python interpreters than are contemporary with those platforms | 18:57 |
| fungi | it's all the libs declaring they require python>=3.6 so pip will no longer install them on xenial | 18:57 |
| fungi | a separate virtualenv/venv doesn't solve that nuance | 18:58 |
| fungi | and you can hardly blame the libs, they say they require python>=3.6 because they were told they could stop testing on 3.5 | 18:59 |
| *** yamamoto has quit IRC | 18:59 | |
| yoctozepto | 19:56:04 <clarkb> another option (which is what devstack does) is to install tempest in a venv independent of the system under test | 19:00 |
| yoctozepto | yeah but it requires python to exist first ;p | 19:00 |
| yoctozepto | honestly, I see the problem in tempest being branchless | 19:01 |
| yoctozepto | where everything else is not | 19:01 |
| yoctozepto | we are losing a lot of flexibility with that actually | 19:01 |
| yoctozepto | it might have been good initially | 19:01 |
| clarkb | yoctozepto: yes, but the goals are still worthwhile though | 19:01 |
| fungi | yay, pvmove of xvdn1 is completed. now `lvdisplay --maps` reports all logical volumes are served from the /dev/xvdb1 pv | 19:02 |
| fungi | i'll work on the vgreduce next | 19:02 |
| *** xek has quit IRC | 19:02 | |
| clarkb | basically tempest should be able to run against $cloud | 19:02 |
| clarkb | the problem here is that our testing tries to conflate the dependencies between the cloud and the tempest install | 19:02 |
| *** xek has joined #openstack-infra | 19:02 | |
| yoctozepto | clarkb: good point | 19:03 |
| *** yamamoto has joined #openstack-infra | 19:03 | |
| *** yamamoto has quit IRC | 19:03 | |
| yoctozepto | it is indeed due to installation of tempest, not all the rest | 19:03 |
| yoctozepto | so back to one of my ideas - package tempest in container? :D | 19:03 |
| fungi | right, tempest is intended to support a range of different service versions, however it hasn't had installation onto the same operating system version as those services as an explicit design goal (and that has only been emergently implicit through how it's tested in our ci) | 19:03 |
| *** yamamoto has joined #openstack-infra | 19:03 | |
| yoctozepto | (well, actually kolla does it already) | 19:03 |
| fungi | yeah, docker run tempest might be one workaround | 19:04 |
| yoctozepto | cleanest one I think | 19:04 |
| *** rkukura has joined #openstack-infra | 19:05 | |
| *** jamesmcarthur has quit IRC | 19:05 | |
| fungi | speaking of clean, i'm in awe of the lvm2 authors | 19:07 |
| fungi | `vgreduce main -a` to clear out all unused pvs | 19:07 |
| *** yamamoto has quit IRC | 19:08 | |
| fungi | `vgdisplay -v` now shows the "main" vg uses only the /dev/xvdb1 pv | 19:08 |
| clarkb | for the longest time I thought tempest avoided using openstack libs because it didn't want their assumptions to pollute the test results | 19:09 |
| clarkb | but I guess that is only true for the api clients and not stuff like oslo test/config/etc | 19:09 |
| clarkb | ? | 19:09 |
| fungi | well, also tempest plugins are apparently happy to use other stuff | 19:10 |
| fungi | for example, neutron-lib | 19:10 |
| fungi | okay, for hygiene i've done a pvremove on every unused pv, so now `pvs` reports only /dev/xvdb1 | 19:11 |
| clarkb | shoudl be able to detach the volumes on the cinder side now? | 19:11 |
| fungi | next i'll see if i can tell nova to detach them, yeah | 19:11 |
| fungi | nova, not cinder | 19:11 |
| fungi | nova owns the relationship | 19:12 |
| fungi | it tells cinder | 19:12 |
| fungi | cinder can (and generally does) report it, of course | 19:12 |
| fungi | "public endpoint for volumev2 service in DFW region not found" | 19:14 |
| yoctozepto | well, happens | 19:14 |
| fungi | do we need a different osc/sdk version for rackspace cinder? | 19:15 |
| fungi | or is another infra-root able to `openstack volume list` in openstackci-rax DFW from bridge.o.o and can tell me how they did it? | 19:16 |
| Shrews | fungi: i cannot | 19:18 |
| *** ralonsoh has quit IRC | 19:22 | |
| smarcet | fungi: sorry to bother u again ,but its seems that this job is not running to me https://zuul.opendev.org/t/openstack/job/openstackid-release-master | 19:22 |
| clarkb | catalog list doesn't return anything useful either | 19:22 |
| clarkb | catalog listing issue seems to maybe be a bug in osc (at least handling the response from rax) according to --debug output | 19:24 |
| *** KeithMnemonic has joined #openstack-infra | 19:26 | |
| clarkb | it seems to be using identity v3 stuff? but the cloud should be v2? | 19:26 |
| clarkb | we might have to force v2 identity? | 19:26 |
| clarkb | yes now I can see the catalog | 19:27 |
| *** jamesmcarthur has joined #openstack-infra | 19:28 | |
| clarkb | fungi: there is only 'volume' not 'volumev2' | 19:28 |
| clarkb | I think we nnow need to figure out how to force old cinder api | 19:28 |
| clarkb | fungi: I hae a fix for you. Give me a minute to put it in git and push | 19:31 |
| fungi | smarcet: that job seems to have no build history at all. seeing if i can tell when that should be triggered | 19:31 |
| yoctozepto | OS_VOLUME_API_VERSION ? | 19:32 |
| smarcet | fungi: last openstakid build was on 2020-01-16 04:15 ( http://tarballs.openstack.org/openstackid/?C=M;O=D) | 19:33 |
| smarcet | i do think that job create the tarball | 19:33 |
| fungi | smarcet: here is is: openstackid-release-master | 19:33 |
| fungi | er | 19:33 |
| fungi | https://opendev.org/openstack/project-config/src/branch/master/zuul.d/projects.yaml#L4723-L4727 | 19:33 |
| fungi | the openstackid-release-branch job runs in the "post" pipeline every time a change merges | 19:33 |
| fungi | the "openstackid-release-master" job runs in the "release" pipeline any time a new signed git tag is pushed | 19:34 |
| fungi | i don't see any new tags pushed for openstackid since last april | 19:34 |
| openstackgerrit | Clark Boylan proposed opendev/system-config master: Set rax api versions for volume and identity https://review.opendev.org/704401 | 19:35 |
| clarkb | fungi: ^ that should do it | 19:35 |
| smarcet | ok bc i merged this one https://review.opendev.org/#/c/661661/ and openstackid-release-branch dint ran | 19:35 |
| fungi | smarcet: https://zuul.opendev.org/t/openstack/builds?job_name=openstackid-release-branch shows the job began failing on 2020-01-16 | 19:35 |
| smarcet | oh ok i will take a look then | 19:36 |
| smarcet | thx | 19:36 |
| fungi | job failure says it's trying to run the old zuul-cloner command which isn't in its path | 19:36 |
| fungi | clarkb: thanks!!! | 19:36 |
| clarkb | yoctozepto: yup, but encoded in clouds.yaml | 19:37 |
| clarkb | yoctozepto: so that it just works (tm) | 19:37 |
| clarkb | After lunch I'm going to look at booting a new status.o.o. For that do we think it should remain openstack.org host or would it make sense to go opendev.org? Most of the services running there are pretty openstack specific at this point in time | 19:37 |
| smarcet | fungi: there is a new replacement for zuul-cloner ? | 19:37 |
| yoctozepto | clarkb: yup, +1 | 19:37 |
| openstackgerrit | Merged zuul/zuul master: Update Quick-Start Fedora Package Install Command https://review.opendev.org/704221 | 19:38 |
| fungi | clarkb: do you happen to know if there's a role jobs have to include to get the zuul-cloner shim? | 19:38 |
| clarkb | fungi: yes, one sec | 19:39 |
| fungi | that job is parented to legacy-publish-openstack-artifacts which i would have expected to provide it | 19:39 |
| clarkb | fetch-zuul-cloner in zuul-jobs | 19:39 |
| fungi | should legacy-publish-openstack-artifacts in ozj be augmented to include that, do you think? | 19:40 |
| clarkb | I thought legacy-base included it, does that legacy job not parent to legacy-base? | 19:41 |
| fungi | it's parented to publish-openstack-artifacts | 19:41 |
| fungi | for reasons i'm sure seemed entirely sane at the time | 19:42 |
| clarkb | in that case ya we probably just want to shim it back in | 19:42 |
| smarcet | fungi: well looking at openstackid zuul jobs definitiions i am seeing something weird | 19:44 |
| smarcet | openstackid-release-branch | 19:44 |
| smarcet | has its parent set to https://opendev.org/osf/openstackid/src/branch/master/.zuul.yaml#L3 | 19:44 |
| smarcet | publish-openstack-artifacts | 19:45 |
| smarcet | and openstackid-release-master to legacy-publish-openstack-artifacts | 19:45 |
| smarcet | https://opendev.org/osf/openstackid/src/branch/master/.zuul.yaml#L10 | 19:45 |
| smarcet | arent supposed to point to the same parent ? | 19:45 |
| fungi | smarcet: oh, good eye. also it's openstackid-release-branch which is failing so legacy-publish-openstack-artifacts doesn't come into the picture | 19:47 |
| fungi | i suspect you need to either update openstackid-release-branch to have legacy-publish-openstack-artifacts as its parent job, or alter openstackid-release-branch to not make use of zuul-cloner any longer | 19:48 |
| *** mattw4 has quit IRC | 19:49 | |
| smarcet | fungi: and will try the first and then will dig on how to replace the zuul-cloner if u agree :) | 19:50 |
| smarcet | and == i | 19:50 |
| fungi | for the record, here's where you're invoking zuul-cloner: https://opendev.org/osf/openstackid/src/branch/master/playbooks/openstackid-release-branch/run.yaml#L11-L38 | 19:50 |
| fungi | i suspect that just removing that task would result in the expected behavior since zuul should provide a checkout of the project at the current path... clarkb is that accurate? | 19:51 |
| fungi | worst case replacing that task with one which copies from the location zuul checks out the triggering ref should suffice | 19:51 |
| smarcet | fungi: understood thx u :) | 19:52 |
| clarkb | current path meaning cwd? | 20:01 |
| fungi | okay, volume list now shows the only remaining volume attached to static.o.o is static.openstack.org/main01 on /dev/xvdb | 20:01 |
| clarkb | it wont clone to cwd you have to go into the project dir | 20:01 |
| fungi | in order to exercise the attached volume and make sure it's really the one we expect, i'm doing some cleanup of oooooold content | 20:03 |
| *** jamesmcarthur has quit IRC | 20:03 | |
| fungi | removing /srv/static/old-logs, /srv/static/old-docs-draft and /srv/static/old-pypi | 20:04 |
| fungi | once that works i'll start telling cinder to delete the other 13 available volumes | 20:04 |
| *** smarcet has quit IRC | 20:05 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/project-config master: Add opendev/pre-commit-hooks https://review.opendev.org/704411 | 20:07 |
| fungi | also /srv/static/mirror which was entirely empty and not referenced from apache configuration at all | 20:11 |
| fungi | and /srv/static/release which only has files 1306 days old or older (/srv/static/releases plural is what apache serves) | 20:11 |
| fungi | anyway, all that cleanup went without error, so i'll start deleting available cinder volumes | 20:12 |
| *** jamesmcarthur has joined #openstack-infra | 20:16 | |
| fungi | we only have 144gb worth of files in logical volumes on static.o.o now, but i'm not going to try shifting those to a new smaller cinder volume | 20:16 |
| openstackgerrit | Antoine Musso proposed zuul/zuul-jobs master: Skip test-setup.sh in pep8 jobs https://review.opendev.org/704414 | 20:17 |
| clarkb | I'm booting status01.openstack.org now | 20:18 |
| clarkb | decided that being conservative there should be fine | 20:18 |
| *** rlandy is now known as rlandy|brb | 20:19 | |
| fungi | nice. some half the cinder volumes i tried to delete are hanging around in error_deleting state | 20:19 |
| openstackgerrit | Merged opendev/system-config master: Set rax api versions for volume and identity https://review.opendev.org/704401 | 20:19 |
| fungi | clarkb: we said tuesday and wednesday for our trusty-exorcism sprint right? | 20:20 |
| fungi | you're making me feel like i'm lagging | 20:21 |
| openstackgerrit | Merged zuul/nodepool master: add tags support for aws provider https://review.opendev.org/703651 | 20:21 |
| clarkb | fungi: tuesday and wednesday australia time :) | 20:21 |
| fungi | just making sure i didn't misremember the schedule | 20:21 |
| fungi | ahh | 20:21 |
| clarkb | however if it works better for others to do it tuesday and wednesday I don't mind | 20:21 |
| fungi | i don't leave for fosdem until thursday anyway | 20:21 |
| clarkb | fungi: we can shift if that is better on your calendar. I'll likely just keep working through things today and those days in that case | 20:21 |
| fungi | i do want to knuckle down and knock out the wiki server upgrade | 20:22 |
| fungi | even moreso because in addition to the static.o.o reboot migration ticket there was also one for the wiki server | 20:22 |
| fungi | would be nice to have it out of production by the time the reboot rolls around | 20:23 |
| fungi | but to be honest i was more worried about static.o.o | 20:23 |
| clarkb | ya this work is related I figured you were just prioritizing it over the wiki :) | 20:23 |
| fungi | at this point i'm going to ignore the error_deleting volumes and declare victory on static.o.o | 20:24 |
| fungi | #status log static.o.o lvm scaled back to just the main01 volume, and some ancient unused files deleted from remaining logical volumes | 20:25 |
| *** SotK has quit IRC | 20:25 | |
| fungi | for my next trick, i'll figure out wth happened to statusbot | 20:25 |
| *** SotK has joined #openstack-infra | 20:25 | |
| clarkb | new status now booted but completely inconfigured so shouldn't have affected statusbot (also that runs on eavesdrop iirc) | 20:26 |
| *** tobias-urdin has quit IRC | 20:26 | |
| clarkb | I'm going to sort out lunch now, then I'll work on dns and inventory updates assuming peopel don't want it to be status.opendev.org | 20:26 |
| fungi | 2020-01-23 21:20:50 <-- openstackstatus (~openstack@eavesdrop01.openstack.org) has quit (Ping timeout: 240 seconds) | 20:26 |
| fungi | ouch | 20:26 |
| *** gyee has quit IRC | 20:27 | |
| *** openstackstatus has joined #openstack-infra | 20:28 | |
| *** ChanServ sets mode: +v openstackstatus | 20:28 | |
| fungi | #status log statusbot restarted after 2020-01-23T21:20:50 irc ping timeout | 20:28 |
| openstackstatus | fungi: finished logging | 20:28 |
| fungi | #status log static.o.o lvm scaled back to just the main01 volume, and some ancient unused files deleted from remaining logical volumes | 20:28 |
| *** gyee has joined #openstack-infra | 20:28 | |
| openstackstatus | fungi: finished logging | 20:28 |
| ianw | clarkb: hey, sorry coming online, back in about 20 | 20:28 |
| *** rlandy|brb is now known as rlandy | 20:29 | |
| fungi | ianw: no rush, i'll catch you up when you're ready | 20:31 |
| *** artom has quit IRC | 20:31 | |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: rename pep8 to linters https://review.opendev.org/703635 | 20:34 |
| openstackgerrit | Antoine Musso proposed zuul/zuul master: tox: do not install bindep for linters https://review.opendev.org/703636 | 20:36 |
| *** lbragstad has quit IRC | 20:40 | |
| *** lbragstad has joined #openstack-infra | 20:41 | |
| *** smarcet has joined #openstack-infra | 20:44 | |
| *** lbragsta_ has joined #openstack-infra | 20:46 | |
| *** lbragstad has quit IRC | 20:50 | |
| *** lbragsta_ has quit IRC | 20:50 | |
| *** lbragstad has joined #openstack-infra | 20:50 | |
| *** jamesmcarthur has quit IRC | 20:59 | |
| *** mattw4 has joined #openstack-infra | 21:07 | |
| openstackgerrit | Clark Boylan proposed opendev/system-config master: Add new xenial status.openstack.org https://review.opendev.org/704427 | 21:09 |
| clarkb | I haven't updated dns yet because I half expect puppet to have problems and I might recycle this hose | 21:10 |
| clarkb | *host | 21:10 |
| clarkb | but first step is running puppet so starting there | 21:10 |
| fungi | i went ahead and approved the inventory addition | 21:10 |
| clarkb | thanks | 21:11 |
| *** mattw4 has quit IRC | 21:13 | |
| *** mattw4 has joined #openstack-infra | 21:14 | |
| ianw | fungi: ok, sounds like maybe some of https://storyboard.openstack.org/#!/story/2006598 is done? | 21:18 |
| fungi | ooh. i should look | 21:18 |
| fungi | mmm, no really none of those tasks is 100% done with what i was doing today | 21:19 |
| fungi | i got rid of the separate /srv/static/logs volume and scaled the main vg back to just one cinder volume | 21:20 |
| fungi | also deleted some long-stale content from /srv/static while i was there | 21:20 |
| fungi | nothing apache was still serving | 21:20 |
| fungi | the quick summary is that rackspace opened a ticket saying they want to cold-migrate static.o.o and so i was worried about problems which might arise with its giant pile of cinder volumes | 21:21 |
| fungi | so i effectively dropped 13 our of the 14 cinder volumes which had been attached to it | 21:22 |
| fungi | in hopes that increases the chances for a speedy and problem-free reboot when the time comes | 21:22 |
| ianw | nice; yeah it never really survived migration that well before | 21:23 |
| ianw | ok, so it seems like i should get static.opendev.org up (all the ansible bits are there) and we can POC governance and TC serving from AFS | 21:23 |
| fungi | sounds great to me | 21:23 |
| fungi | happy to review, just give me a heads-up on changes in case i miss them | 21:24 |
| ianw | is RAX/dfw still the place to start it up? | 21:24 |
| fungi | there or vexxhost sjc1 | 21:24 |
| ianw | i imagine so, since this is intended to be basically an ephemeral AFS mirror, so best to keep it close to the servers | 21:24 |
| fungi | sure | 21:24 |
| clarkb | ya I think network locality matters a lot for afs | 21:26 |
| *** xek has quit IRC | 21:29 | |
| fungi | i know it does | 21:30 |
| *** signed8bit has joined #openstack-infra | 21:32 | |
| *** Goneri has quit IRC | 21:39 | |
| *** jamesmcarthur has joined #openstack-infra | 21:41 | |
| *** jamesmcarthur has quit IRC | 21:45 | |
| *** jamesmcarthur has joined #openstack-infra | 21:48 | |
| *** ociuhandu has joined #openstack-infra | 21:50 | |
| *** lpetrut has joined #openstack-infra | 21:51 | |
| hashar | hi infra folks. x/gearman-plugin is a plugin for Jenkins which is hmm.. no more used for opendev | 21:53 |
| hashar | I thought about reviewing merging what is left over in Gerrit, and maybe then migrate that to the upstream Jenkins repositories | 21:54 |
| hashar | namely https://github.com/jenkinsci/ | 21:54 |
| hashar | maybe I should write about it on the mailing list ;D | 21:55 |
| clarkb | hashar: correct we haven't use it since I think 2016? | 21:55 |
| *** lpetrut has quit IRC | 21:55 | |
| hashar | yeah since Zuul 2.5 and the migration from jjb to ansible | 21:56 |
| clarkb | and it hasn't really been maintained since, so we decided to retire it and figured if people were using it they would show up and we could figure out next steps from there :0 | 21:56 |
| clarkb | er :) | 21:56 |
| hashar | and here we are :] | 21:56 |
| hashar | for Wikimedia, I had a fork maintained by cherry picking pending patches | 21:57 |
| hashar | I just thought all those could just get merged and one last release be cut from opendev.org gerrit | 21:57 |
| hashar | then I am volunteering to have it mirrored to the Jenkins github and then get it archived from opendev gerrit | 21:57 |
| clarkb | hashar: I think it would be better to host what we had merged on Jenkins' github and then merge any outstanding changes | 21:58 |
| clarkb | then it is clear you are making those changes rather than us since we are not in a spot to review those properly | 21:58 |
| hashar | my idea was to at least benefit from opendev ci for the few pending changes that got reviews etc over the last years | 22:00 |
| hashar | but haven't been merged since Zaro moved to another organization | 22:00 |
| clarkb | hashar: if you want to keep using the opendev ci you can keep using opendev | 22:00 |
| hashar | true ;] | 22:01 |
| hashar | then I guess there is more activity and routine maintenance on the Jenkins github organization | 22:01 |
| hashar | I will sleep and think about it, and reach out to some opendev list or the infra list | 22:01 |
| clarkb | ok | 22:02 |
| hashar | thank you clarkb ! | 22:02 |
| openstackgerrit | Merged opendev/system-config master: Add new xenial status.openstack.org https://review.opendev.org/704427 | 22:03 |
| ianw | that's not the repo i had to unilaterally move to bionic to get rid of trusty is it (i.e. testing may be broken?) | 22:05 |
| ianw | legacy-gearman-plugin-mavin-build-ubuntu-trusty | 22:05 |
| clarkb | ianw: ya it is | 22:06 |
| ianw | i feel like it's unlikely that same job will work on bionic unmodified ... i didn't test at all, we just needed the references to the trusty nodeset gone | 22:08 |
| ianw | so of course opendev testing can continue, but i think it will need some work | 22:09 |
| *** rcernin has joined #openstack-infra | 22:10 | |
| ianw | ok i feel like i've helped(?) as much as i can on the current devstack various brokenness ... moving on to static.opendev.org | 22:11 |
| *** smarcet has quit IRC | 22:11 | |
| ianw | if anyone would like a cable update ... the 4th person came who plugged in the cable and it all came up ... for me to realise my ISP requires a vlan tag and the crappy old-isp-firmware-installed router i brought with me has no option for this | 22:12 |
| ianw | so i am eagerly awaiting the postal person with some decent networking gear ... | 22:12 |
| fungi | wow, vlan tagged... as in 802.1q tagging? | 22:14 |
| fungi | if so, that's a rather neat implementation they have, though questionable since it only scales to around 1k vlans per link without some additional encapsulation | 22:15 |
| ianw | yep -> https://www.internode.on.net/support/guides/general_settings/#nbn | 22:15 |
| *** jamesmcarthur has quit IRC | 22:16 | |
| lxkong | hi, not sure i can find devstack guys here, but is this a known issue when installing devstack `ERROR: Cannot uninstall 'httplib2'. It is a distutils installed project`? | 22:16 |
| lxkong | OS: ubuntu bionic, devstack: Master as of just now | 22:16 |
| ianw | lxkong: openstack-qa probably more appropriate, but that's a pip error ... what platform you running on? | 22:16 |
| *** aedc has quit IRC | 22:17 | |
| lxkong | ianw: please see above | 22:17 |
| fungi | ianw: and you thought you'd helped as much as you could on the current devstack carioys brokenness | 22:17 |
| fungi | er, various | 22:17 |
| *** slaweq has quit IRC | 22:17 | |
| *** jamesmcarthur has joined #openstack-infra | 22:17 | |
| fungi | the short answer there is that something installed as a distro package on ubuntu is depending on their distro package of httplib2, which was built using distutils, and devstack is for some reason wanting to upgrade httplib2 with pip but pip refuses because it's not sure how to cleanly uninstall the ubuntu package of it | 22:18 |
| *** jamesmcarthur has quit IRC | 22:18 | |
| *** aedc has joined #openstack-infra | 22:19 | |
| fungi | the simplest workarounds i've seen involve deleting package metadata off the filesystem before invoking pip, scary as that is | 22:19 |
| fungi | but i agree that #openstack-qa is a better venue for devstack questions | 22:20 |
| *** ahosam has joined #openstack-infra | 22:21 | |
| *** jamesmcarthur has joined #openstack-infra | 22:25 | |
| *** jamesmcarthur has quit IRC | 22:27 | |
| *** jamesmcarthur has joined #openstack-infra | 22:29 | |
| *** jamesmcarthur has quit IRC | 22:31 | |
| ianw | https://github.com/pypa/pip/commit/e5212281299d18cdd56ebc137cc18f31c9a7605c ... sigh ... | 22:32 |
| *** slaweq has joined #openstack-infra | 22:34 | |
| *** jamesmcarthur has joined #openstack-infra | 22:34 | |
| *** hashar has quit IRC | 22:36 | |
| *** smarcet has joined #openstack-infra | 22:38 | |
| *** jamesmcarthur has quit IRC | 22:38 | |
| *** jamesmcarthur has joined #openstack-infra | 22:38 | |
| *** slaweq has quit IRC | 22:38 | |
| fungi | i'm missing the relevance of that commit | 22:40 |
| ianw | just the commit message ... i was just poking for possible changes to the ignore flags and you have to filter through stuff like that | 22:41 |
| *** diablo_rojo has quit IRC | 22:42 | |
| fungi | oh, sure that's almost as terrible a commit message as "fix stuff" | 22:42 |
| *** rh-jelabarre has quit IRC | 22:42 | |
| fungi | after staring at so many other projects i've become desensitized to the fact that most communities don't even understand the importance of much less block on good commit messages | 22:43 |
| fungi | "please explain the reason for this in the commit message" isn't something i often see in code review outside our community | 22:45 |
| fungi | except on lkml | 22:46 |
| *** gfidente|afk is now known as gfidente | 22:46 | |
| *** smarcet has left #openstack-infra | 22:48 | |
| *** ahosam has quit IRC | 22:49 | |
| *** ahosam has joined #openstack-infra | 22:49 | |
| *** smarcet has joined #openstack-infra | 22:52 | |
| *** tkajinam has joined #openstack-infra | 22:55 | |
| clarkb | I think new status01 has successfully had the base ansible run on it, now waiting for puppet to run | 22:59 |
| ianw | fungi: i feel like i've asked this before, but we only need the CAA records for the actual hostname, not he CNAMES (i.e. static01.o.o, not also for static.o.o)? | 23:11 |
| *** artom has joined #openstack-infra | 23:13 | |
| *** jamesmcarthur has quit IRC | 23:13 | |
| *** jamesmcarthur has joined #openstack-infra | 23:17 | |
| ianw | i'm going to add the acme records for the openstack.org domains pointing to acme.opendev.org in preparation for static server. i'll mention it all in system-config commit message | 23:19 |
| *** jamesmcarthur has quit IRC | 23:21 | |
| fungi | ianw: yeah, really only something a browser will care about benefits from a caa rr | 23:22 |
| fungi | however you can't have a caa with the same name as a cname | 23:22 |
| fungi | because a cname needs to be exclusive | 23:22 |
| fungi | so the caa must have the name of whatever that cname points at (unless that too is a cname, repeat until resolved) | 23:23 |
| fungi | so if static.o.o is a cname for static01.o.o then when the browser asks for a caa rr for static.o.o it gets the cname record which tells it that the answer it wants can be found in the caa record for static01.o.o | 23:24 |
| fungi | is that what you're asking? | 23:24 |
| fungi | however, try to push a caa rr for static.o.o sometime, bind should refuse to load the zone complaining about "cname and other data at line xyz" | 23:25 |
| fungi | bin cares that cnames do not share a name with any other record | 23:26 |
| fungi | er, bind cares | 23:26 |
| *** ociuhandu has quit IRC | 23:30 | |
| *** ociuhandu has joined #openstack-infra | 23:30 | |
| *** dchen has joined #openstack-infra | 23:34 | |
| *** dchen has quit IRC | 23:35 | |
| *** dchen has joined #openstack-infra | 23:35 | |
| clarkb | status.o.o ended up failing to minimize some js things beacuse the expected files are not present | 23:35 |
| clarkb | I expect this has been broken in produection and we are still using whatever the last set of files were that were present | 23:35 |
| clarkb | working to untangle this now | 23:36 |
| *** smarcet has quit IRC | 23:37 | |
| openstackgerrit | Clark Boylan proposed opendev/system-config master: Flot sources moved, update our minimization to accomodate https://review.opendev.org/704467 | 23:39 |
| *** ociuhandu has quit IRC | 23:39 | |
| clarkb | fungi: ianw ^ I'm going to put prod status.o.o in the emergency file so that this can run only on the new host while we sort out what is what | 23:39 |
| clarkb | should be safe to go ahead and merge that and we'll see where puppet ends up after | 23:40 |
| clarkb | there was a different separate autoconf error which happened later that I am now looking at | 23:41 |
| clarkb | I think we lack build-essential | 23:42 |
| *** yamamoto has joined #openstack-infra | 23:45 | |
| *** tosky has quit IRC | 23:45 | |
| *** smarcet has joined #openstack-infra | 23:48 | |
| fungi | ahh, yeah, like autoconf not existing? | 23:49 |
| ianw | i agree it looks like /opt/flot/source ... although i have no idea what that is :) | 23:49 |
| clarkb | fungi: ya | 23:49 |
| *** yamamoto has quit IRC | 23:49 | |
| ianw | yui is all dead too now isn't it? | 23:50 |
| clarkb | the last update to yuicompressor was may of last year | 23:51 |
| clarkb | I'm not too concerned about using an older minifier, but maybe I should be | 23:51 |
| clarkb | looks like we install gerritlib, which installs paramiko, which installs pycrypto which needs build-essential and likely the libssl-dev headers | 23:51 |
| fungi | an alternative could be just not minifying that stuff | 23:51 |
| fungi | (to the yui-compressor problem i mean) | 23:52 |
| ianw | yeah, i don't know if it's an issue right now, but i think longer term it will be | 23:52 |
| ianw | i used to use yui for everything, it was great in the day | 23:52 |
| clarkb | for the building problem I think we want to bump our paramiko dependency | 23:53 |
| clarkb | that will get us cryptography which is more secure and has wheels | 23:53 |
| clarkb | so rather than install build-essential I'll start poking at ^ | 23:53 |
| openstackgerrit | Ian Wienand proposed opendev/zone-opendev.org master: Add static01.opendev.org records https://review.opendev.org/704469 | 23:54 |
| fungi | i'm not really sold on minifiers anyway. those .js files will be compressed on transfer between the server and browser so it's really not buying a lot with modern bandwidth expectations | 23:54 |
| clarkb | looks like we capped paramiko to avoid needing libffi-dev which is no longer the case with cffi and cryptography publishing wheels | 23:54 |
| fungi | yeah, i'd buy that it all boils down to a temporary problem we semi-permanently worked around and is now coming back to bite us | 23:55 |
| fungi | par for the course | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!