| clarkb | gitea fixup is still running. I'm going to leave it alone for a bit while I sort out dinner | 00:01 |
|---|---|---|
| *** wolverineav has quit IRC | 00:02 | |
| ianw | fungi: i do understand, however, it becomes quite verbose when you mix multiple distros. i'm starting to think this might be a feature rather than a bug though. | 00:04 |
| ianw | python-devel [platform:redhat !platform:fedora !platform:centos-8 !platform:rhel-8] | 00:04 |
| ianw | python3-devel [platform:fedora platform:centos-8 platform:rhel-8] | 00:04 |
| ianw | maybe that *is* the correct way to do it ... | 00:04 |
| *** slaweq has joined #openstack-infra | 00:11 | |
| fungi | i'm unclear what the distinction is | 00:12 |
| fungi | what does platform:redhat convey? | 00:12 |
| fungi | or put another way, are there redhat platforms which do not use rom packages? | 00:12 |
| fungi | er, rpm packages | 00:13 |
| *** jamesmcarthur has joined #openstack-infra | 00:13 | |
| fungi | rpm/dpkg aren't meant to convey package managers, they're references to packaging formats | 00:14 |
| *** mpjetta has quit IRC | 00:14 | |
| fungi | so platform:rpm is "platforms using rpm packages generally name the package thusly" | 00:14 |
| fungi | the names chosen are for the local low-level package unpacking and listing tools which are being invoked to find out what's installed | 00:15 |
| *** slaweq has quit IRC | 00:15 | |
| *** jamesmcarthur has quit IRC | 00:16 | |
| *** jamesmcarthur has joined #openstack-infra | 00:16 | |
| mordred | corvus: folded subjects!!! :) | 00:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Run node jobs using version 4 on xenial nodes https://review.openstack.org/643196 | 00:19 |
| ianw | fungi: well, "redhat like distributions" | 00:23 |
| *** xek has joined #openstack-infra | 00:23 | |
| fungi | then do we change what platform:debian means to debian-like distributions rather than meaning actual debian? | 00:24 |
| ianw | which is defined within bindep/depends.py as basically a fallthrough from a static list | 00:25 |
| *** Swami has quit IRC | 00:26 | |
| ianw | s/fallthrough/fallthrough of things !suse/ | 00:26 |
| ianw | fungi: i don't know if i have an answer. maybe it's fair enough that bindep just exposes a whole bunch of "atoms" it can determine from the underlying system, and we let users worry about how they use them | 00:27 |
| *** wolverineav has joined #openstack-infra | 00:28 | |
| *** dklyle has quit IRC | 00:31 | |
| *** dklyle has joined #openstack-infra | 00:31 | |
| *** jamesmcarthur has quit IRC | 00:31 | |
| *** wolverineav has quit IRC | 00:35 | |
| *** gyee has quit IRC | 00:39 | |
| *** dklyle has quit IRC | 00:46 | |
| mordred | fungi: I made a topic with a pile of patches: https://review.openstack.org/#/q/topic:nodev4-xenial | 00:46 |
| *** jamesmcarthur has joined #openstack-infra | 00:57 | |
| *** ricolin has joined #openstack-infra | 00:57 | |
| *** jamesmcarthur has quit IRC | 00:57 | |
| *** jamesmcarthur has joined #openstack-infra | 00:57 | |
| *** rlandy|bbl is now known as rlandy | 01:00 | |
| *** mpjetta has joined #openstack-infra | 01:01 | |
| *** slaweq has joined #openstack-infra | 01:11 | |
| *** mriedem_afk has quit IRC | 01:11 | |
| *** rascasoft has joined #openstack-infra | 01:12 | |
| *** slaweq has quit IRC | 01:16 | |
| *** rascasoft has quit IRC | 01:19 | |
| *** jamesmcarthur has quit IRC | 01:23 | |
| *** jamesmcarthur has joined #openstack-infra | 01:24 | |
| *** jamesmcarthur has quit IRC | 01:26 | |
| *** jamesmcarthur has joined #openstack-infra | 01:27 | |
| *** ricolin has quit IRC | 01:33 | |
| *** snevi has quit IRC | 01:33 | |
| *** adriant has joined #openstack-infra | 01:33 | |
| *** rlandy has quit IRC | 01:38 | |
| *** whoami-rajat has joined #openstack-infra | 02:02 | |
| *** wolverineav has joined #openstack-infra | 02:10 | |
| *** hongbin has joined #openstack-infra | 02:15 | |
| *** wolverineav has quit IRC | 02:15 | |
| *** ricolin has joined #openstack-infra | 02:23 | |
| *** ricolin has quit IRC | 02:33 | |
| *** jamesmcarthur has quit IRC | 02:47 | |
| *** dave-mccowan has joined #openstack-infra | 02:50 | |
| *** dave-mccowan has quit IRC | 02:54 | |
| *** roman_g has quit IRC | 02:59 | |
| *** yamamoto has joined #openstack-infra | 03:00 | |
| *** psachin has joined #openstack-infra | 03:00 | |
| *** ykarel|away has joined #openstack-infra | 03:04 | |
| *** slaweq has joined #openstack-infra | 03:11 | |
| *** jamesmcarthur has joined #openstack-infra | 03:13 | |
| *** larainema has joined #openstack-infra | 03:15 | |
| *** slaweq has quit IRC | 03:15 | |
| *** jamesmcarthur has quit IRC | 03:17 | |
| clarkb | ok I figured out why the gitea fixup wasn't finishing up cleanly. OOMKiller is killing the processes under ansible-playbook | 03:19 |
| clarkb | so the playbook hasn't run to completion yet, but I believe it to be a working setup now. | 03:19 |
| clarkb | I guess that bumps up the urgency of doing a server resize | 03:19 |
| clarkb | mordred: corvus ^ fyi | 03:19 |
| *** mthrynn has joined #openstack-infra | 03:20 | |
| *** ykarel|away has quit IRC | 03:22 | |
| *** ramishra has joined #openstack-infra | 03:29 | |
| *** hongbin has quit IRC | 03:36 | |
| *** diablo_rojo has quit IRC | 03:47 | |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: Support logical AND searches https://review.openstack.org/643224 | 03:48 |
| *** ricolin has joined #openstack-infra | 04:02 | |
| *** yamamoto has quit IRC | 04:03 | |
| *** slaweq has joined #openstack-infra | 04:11 | |
| *** slaweq has quit IRC | 04:15 | |
| *** zhangfei has joined #openstack-infra | 04:20 | |
| *** ramishra has quit IRC | 04:21 | |
| *** ramishra has joined #openstack-infra | 04:21 | |
| *** udesale has joined #openstack-infra | 04:22 | |
| *** ykarel|away has joined #openstack-infra | 04:24 | |
| *** ykarel|away is now known as ykarel | 04:24 | |
| *** ramishra_ has joined #openstack-infra | 04:25 | |
| *** ramishra has quit IRC | 04:28 | |
| *** ramishra has joined #openstack-infra | 04:29 | |
| *** ramishra_ has quit IRC | 04:30 | |
| *** janki has joined #openstack-infra | 04:35 | |
| *** yamamoto has joined #openstack-infra | 04:46 | |
| *** lpetrut has joined #openstack-infra | 04:55 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [dmn] Stashing some scripts to make git:// -> https:// changes https://review.openstack.org/642314 | 05:04 |
| *** lpetrut has quit IRC | 05:19 | |
| *** armax has quit IRC | 05:35 | |
| *** raukadah is now known as chandankumar | 05:50 | |
| *** kjackal has joined #openstack-infra | 05:51 | |
| *** jbadiapa has quit IRC | 06:03 | |
| *** jbadiapa has joined #openstack-infra | 06:04 | |
| *** slaweq has joined #openstack-infra | 06:11 | |
| *** jtomasek has joined #openstack-infra | 06:14 | |
| *** slaweq has quit IRC | 06:16 | |
| *** wolverineav has joined #openstack-infra | 06:18 | |
| *** slaweq has joined #openstack-infra | 06:19 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: [dmn] Stashing some scripts to make git:// -> https:// changes https://review.openstack.org/642314 | 06:42 |
| *** wolverin_ has joined #openstack-infra | 06:42 | |
| *** shardy has quit IRC | 06:44 | |
| *** wolverineav has quit IRC | 06:45 | |
| *** wolverin_ has quit IRC | 06:49 | |
| *** shardy has joined #openstack-infra | 06:52 | |
| *** yamamoto has quit IRC | 06:55 | |
| *** kopecmartin|off is now known as kopecmartin | 07:00 | |
| *** yamamoto has joined #openstack-infra | 07:03 | |
| *** yamamoto has quit IRC | 07:04 | |
| *** lpetrut has joined #openstack-infra | 07:07 | |
| *** mthrynn is now known as nnyrhtm | 07:11 | |
| *** nnyrhtm has quit IRC | 07:11 | |
| *** yamamoto has joined #openstack-infra | 07:14 | |
| *** kmarc has joined #openstack-infra | 07:18 | |
| *** markvoelker has quit IRC | 07:19 | |
| *** dpawlik has joined #openstack-infra | 07:22 | |
| *** tosky has joined #openstack-infra | 07:23 | |
| *** pgaxatte has joined #openstack-infra | 07:25 | |
| *** wolverineav has joined #openstack-infra | 07:27 | |
| *** dpawlik has quit IRC | 07:27 | |
| *** dpawlik has joined #openstack-infra | 07:29 | |
| *** wolverineav has quit IRC | 07:31 | |
| *** lpetrut has quit IRC | 07:35 | |
| *** lpetrut has joined #openstack-infra | 07:42 | |
| *** e0ne has joined #openstack-infra | 07:45 | |
| chandankumar | ianw: Hello | 07:46 |
| chandankumar | ianw: is something wrong with https://logs.openstack.org/ | 07:46 |
| chandankumar | ? | 07:46 |
| chandankumar | it is openstack governance page | 07:46 |
| chandankumar | on each patchset, when I click on the job link to go to log url, it is showing 404 | 07:46 |
| chandankumar | please have a look! | 07:47 |
| chandankumar | ah sorry, it was due to https everywhere plugin | 07:47 |
| chandankumar | it is working now | 07:47 |
| *** pcaruana has joined #openstack-infra | 08:07 | |
| *** ginopc has joined #openstack-infra | 08:08 | |
| *** dpawlik has quit IRC | 08:09 | |
| *** dpawlik has joined #openstack-infra | 08:10 | |
| *** pcaruana has quit IRC | 08:11 | |
| *** tkajinam__ has quit IRC | 08:13 | |
| *** ccamacho has joined #openstack-infra | 08:20 | |
| *** pcaruana has joined #openstack-infra | 08:23 | |
| *** ginopc has quit IRC | 08:23 | |
| *** ginopc has joined #openstack-infra | 08:24 | |
| *** ruffian_sheep has joined #openstack-infra | 08:25 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Update to a newer version (8) of Node.js for testing https://review.openstack.org/643046 | 08:34 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Add a linter for our .scss files https://review.openstack.org/643025 | 08:34 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Add a linter for HTML templates https://review.openstack.org/643032 | 08:34 |
| *** helenafm has joined #openstack-infra | 08:44 | |
| *** kjackal has quit IRC | 08:46 | |
| *** kjackal has joined #openstack-infra | 08:47 | |
| ruffian_sheep | frickler:a job in your CI.I want to build a 3rd party CI of cinder.But I dont know what should it do,and what is the result it should be. | 08:49 |
| *** slaweq has quit IRC | 08:53 | |
| *** slaweq has joined #openstack-infra | 08:55 | |
| *** dtantsur|afk is now known as dtantsur | 08:59 | |
| *** jklare has quit IRC | 08:59 | |
| *** jpena|off is now known as jpena | 09:00 | |
| *** jklare has joined #openstack-infra | 09:01 | |
| frickler | ruffian_sheep: I'm no expert on that, but this page should have some information: https://docs.openstack.org/infra/system-config/third_party.html | 09:03 |
| dtantsur | morning folks! can this be a consequence of moving to bionic? http://logs.openstack.org/02/642402/6/check/openstack-tox-lower-constraints/ca317fe/job-output.txt.gz#_2019-03-14_01_51_01_006156 | 09:07 |
| *** panda|rover|off has quit IRC | 09:07 | |
| dtantsur | is it something we should fix in our bindep? oslo.db bindep? both, none? :) | 09:08 |
| *** ykarel is now known as ykarel|lunch | 09:09 | |
| * dtantsur tries | 09:12 | |
| mgoddard | dtantsur: having the same issue in kolla :( | 09:16 |
| dtantsur | mgoddard: I've put up a patch updating bindep, we'll see | 09:16 |
| mgoddard | dtantsur: hold up, jumped the gun there. We're having a different lower-constraints issue | 09:16 |
| dtantsur | more issues \o/ | 09:17 |
| dtantsur | I'm surprised our devstack jobs work. this is suspicious ;) | 09:17 |
| mgoddard | here's ours: http://logs.openstack.org/84/642984/1/gate/openstack-tox-lower-constraints/dc5ed46/job-output.txt.gz#_2019-03-14_08_06_24_130682 | 09:17 |
| dtantsur | wow | 09:17 |
| dtantsur | mgoddard: this sounds like some lower constraint is too low | 09:17 |
| mgoddard | last time I had a lower-constraints issue I ended up binary-chopping the deps | 09:17 |
| mgoddard | yeah | 09:17 |
| * mgoddard fetches the binary chopper | 09:18 | |
| mgoddard | can't repro this one locally though, do I have to upgrade my laptop to bionic? | 09:19 |
| frickler | dtantsur: mgoddard: the issue seems to be that there was a wheel for 16.04 that we used http://mirror.mtl01.inap.openstack.org/wheel/ubuntu-16.04-x86_64/psycopg2/psycopg2-2.6.2-cp35-cp35m-linux_x86_64.whl | 09:19 |
| frickler | and there doesn't seem to be one for 18.04 | 09:19 |
| dtantsur | hmmm.. so installing libpq-dev won't help, right? | 09:19 |
| dtantsur | mgoddard: oh, we have an earlier patch: https://review.openstack.org/#/c/643231/ | 09:21 |
| ruffian_sheep | frickler:Nooo!I mean I have set this CI.But I dont know what the job it should do.And what the result the community need. | 09:22 |
| mgoddard | dtantsur: I think it's a different issue - we don't depend on postgres in kolla | 09:22 |
| *** IvensZambrano has joined #openstack-infra | 09:23 | |
| ruffian_sheep | frickler:I read this link:https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers | 09:23 |
| *** IvensZambrano has quit IRC | 09:24 | |
| *** jpich has joined #openstack-infra | 09:24 | |
| ruffian_sheep | frickler:But I still confuesd about how to set the job to get the result the community need | 09:24 |
| *** sshnaidm is now known as sshnaidm|rover | 09:25 | |
| frickler | dtantsur: maybe installing the lib will help, but maybe it's also overkill because it will not be needed in a normal world where the proper wheel is available | 09:29 |
| dtantsur | yeah, maybe. I see that Nova does have libpq-dev in their bindep | 09:30 |
| frickler | ruffian_sheep: sorry I don't think I can help you, then. maybe you can find other CI operators in #openstack-cinder | 09:30 |
| frickler | dtantsur: hmm, o.k., if nova does it, it should be fine for ironic, too | 09:31 |
| openstackgerrit | Camila Moura proposed openstack-infra/storyboard master: Remove simple priority support from the API https://review.openstack.org/643260 | 09:36 |
| *** e0ne has quit IRC | 09:43 | |
| *** e0ne has joined #openstack-infra | 09:46 | |
| ruffian_sheep | frickler:Thx | 09:47 |
| *** pcaruana has quit IRC | 09:48 | |
| *** ykarel|lunch has quit IRC | 09:52 | |
| *** ykarel has joined #openstack-infra | 09:57 | |
| *** rpittau|afk is now known as rpittau | 10:13 | |
| *** luizbag has joined #openstack-infra | 10:30 | |
| *** wolverineav has joined #openstack-infra | 10:49 | |
| elod | hi, i went through the today's periodic job failures and it seems that the psycopg2 problem is causing failures on 16 projects' sphinx job on *ocata* | 10:51 |
| elod | and another 8 with cryptograpy package issue with missing opensslv.h | 10:53 |
| *** wolverineav has quit IRC | 10:54 | |
| elod | any approach how we should fix these? | 10:54 |
| *** udesale has quit IRC | 10:56 | |
| *** udesale has joined #openstack-infra | 10:56 | |
| frickler | elod: I think jobs for ocata should continue to run on xenial, if they don't, that would be the place to fix | 10:59 |
| *** jpich has quit IRC | 11:01 | |
| *** jpich has joined #openstack-infra | 11:02 | |
| *** yolanda has quit IRC | 11:02 | |
| *** yolanda has joined #openstack-infra | 11:03 | |
| elod | frickler: ok, then I try to make them use xenial | 11:03 |
| *** shardy has quit IRC | 11:11 | |
| *** udesale has quit IRC | 11:15 | |
| *** udesale has joined #openstack-infra | 11:15 | |
| *** zbr has quit IRC | 11:16 | |
| *** zhangfei has quit IRC | 11:18 | |
| *** udesale has quit IRC | 11:27 | |
| *** udesale has joined #openstack-infra | 11:28 | |
| *** derekh has joined #openstack-infra | 11:29 | |
| *** gfidente has joined #openstack-infra | 11:30 | |
| *** pcaruana has joined #openstack-infra | 11:32 | |
| *** roman_g has joined #openstack-infra | 11:44 | |
| *** dave-mccowan has joined #openstack-infra | 11:46 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Allow permissions to be set for teams in worklists and boards https://review.openstack.org/370312 | 11:50 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Rationalise code for filtering private Boards and Worklists https://review.openstack.org/643303 | 11:50 |
| *** ruffian_sheep has quit IRC | 11:57 | |
| *** gcerami_ has joined #openstack-infra | 11:57 | |
| *** zbr has joined #openstack-infra | 11:58 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Add more fields for SMTPReporter subject format https://review.openstack.org/643306 | 12:02 |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Add more fields for SMTPReporter subject format https://review.openstack.org/643306 | 12:05 |
| *** jpena is now known as jpena|lunch | 12:12 | |
| *** yamamoto has quit IRC | 12:13 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: wip: Spec for allowing circular dependencies https://review.openstack.org/643309 | 12:14 |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: wip: Spec for allowing circular dependencies https://review.openstack.org/643309 | 12:17 |
| *** yamamoto has joined #openstack-infra | 12:17 | |
| *** yamamoto has quit IRC | 12:17 | |
| *** yamamoto has joined #openstack-infra | 12:18 | |
| *** trown|outtypewww is now known as trown | 12:19 | |
| *** yamamoto has quit IRC | 12:22 | |
| *** yamamoto has joined #openstack-infra | 12:29 | |
| *** yamamoto has quit IRC | 12:30 | |
| *** yamamoto has joined #openstack-infra | 12:30 | |
| *** helenafm has quit IRC | 12:30 | |
| *** e0ne has quit IRC | 12:31 | |
| *** rlandy has joined #openstack-infra | 12:32 | |
| *** eharney has quit IRC | 12:32 | |
| *** gcerami_ is now known as panda | 12:40 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Rationalise code for filtering private Boards and Worklists https://review.openstack.org/643303 | 12:47 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Allow permissions to be set for teams in worklists and boards https://review.openstack.org/370312 | 12:47 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Pin tox-py35 jobs to ubuntu-xenial https://review.openstack.org/643318 | 12:47 |
| *** jamesmcarthur has joined #openstack-infra | 12:47 | |
| frickler | infra-root: IIUC we don't populate sources.list.available.d in our images anymore, right? so this code is obsolete and should be dropped? http://git.openstack.org/cgit/openstack/devstack-plugin-ceph/tree/devstack/lib/ceph#n129 | 12:48 |
| *** kgiusti has joined #openstack-infra | 12:55 | |
| *** hwoarang has quit IRC | 12:56 | |
| *** hwoarang has joined #openstack-infra | 12:57 | |
| *** jchhatbar has joined #openstack-infra | 13:00 | |
| *** janki has quit IRC | 13:00 | |
| *** dklyle has joined #openstack-infra | 13:01 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Pin tox-py35 jobs to ubuntu-xenial https://review.openstack.org/643318 | 13:02 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Rationalise code for filtering private Boards and Worklists https://review.openstack.org/643303 | 13:02 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Allow permissions to be set for teams in worklists and boards https://review.openstack.org/370312 | 13:02 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Clean up private story filtering code https://review.openstack.org/643323 | 13:02 |
| *** dpawlik has quit IRC | 13:07 | |
| *** mriedem has joined #openstack-infra | 13:08 | |
| *** jchhatbar has quit IRC | 13:10 | |
| *** janki has joined #openstack-infra | 13:10 | |
| dtantsur | folks, could someone take a quick look at https://review.openstack.org/#/c/636169/15/zuul.d/stable-jobs.yaml and check if the zuul bits look right? | 13:11 |
| dtantsur | we've been struggling for some time | 13:12 |
| *** yamamoto has quit IRC | 13:17 | |
| frickler | dtantsur: it doesn't look wrong to me per se, but these branch overrides always seem to need some kind of voodoo in order to work correctly | 13:18 |
| dtantsur | frickler: yeah, it seems that we don't know this kind of voodoo, because the jobs don't pass even though they do pass on the respective branches.. | 13:18 |
| frickler | dtantsur: but we also just had some fixes in devstack regarding proper installation of tempest plugin reqs | 13:18 |
| frickler | dtantsur: so maybe a recheck might look better now | 13:18 |
| dtantsur | trying | 13:19 |
| *** yamamoto has joined #openstack-infra | 13:20 | |
| *** yamamoto has quit IRC | 13:20 | |
| *** yamamoto has joined #openstack-infra | 13:21 | |
| *** eharney has joined #openstack-infra | 13:24 | |
| *** yamamoto has quit IRC | 13:25 | |
| *** jamesmcarthur has quit IRC | 13:27 | |
| *** e0ne has joined #openstack-infra | 13:32 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul-jobs master: Support ~/.gnupg subdirs in remove-gpgkey role https://review.openstack.org/643328 | 13:32 |
| *** yamamoto has joined #openstack-infra | 13:39 | |
| *** yamamoto has quit IRC | 13:39 | |
| *** Wryhder has joined #openstack-infra | 13:40 | |
| *** yamamoto has joined #openstack-infra | 13:40 | |
| *** Lucas_Gray has joined #openstack-infra | 13:43 | |
| *** yamamoto has quit IRC | 13:44 | |
| openstackgerrit | Nir Magnezi proposed openstack/diskimage-builder master: WIP: Add RHEL8 Element https://review.openstack.org/643332 | 13:46 |
| *** jamesmcarthur has joined #openstack-infra | 13:46 | |
| openstackgerrit | Nir Magnezi proposed openstack/diskimage-builder master: WIP: Add RHEL8 Element https://review.openstack.org/643332 | 13:47 |
| *** jpena|lunch is now known as jpena | 13:50 | |
| *** hwoarang has quit IRC | 13:53 | |
| *** hwoarang has joined #openstack-infra | 13:55 | |
| *** janki has quit IRC | 13:58 | |
| *** ricolin has quit IRC | 13:59 | |
| openstackgerrit | Nir Magnezi proposed openstack/diskimage-builder master: WIP: Add RHEL8 Element https://review.openstack.org/643332 | 14:02 |
| *** ykarel is now known as ykarel|away | 14:05 | |
| *** Wryhder has quit IRC | 14:12 | |
| *** rh-jelabarre has joined #openstack-infra | 14:13 | |
| *** ykarel|away has quit IRC | 14:14 | |
| *** helenafm has joined #openstack-infra | 14:16 | |
| *** rcernin has quit IRC | 14:17 | |
| clarkb | frickler: we may do it as part of the base job mirror setup | 14:23 |
| clarkb | ? | 14:23 |
| *** lujinluo has joined #openstack-infra | 14:27 | |
| elod | for the ocata vs bionic job failures i've created a sample patch (in nova), could someone have a quick look at it? https://review.openstack.org/#/c/643320/2 | 14:31 |
| elod | if it is ok, then i will create patches for the 20+ other projects | 14:32 |
| *** lujinluo has quit IRC | 14:32 | |
| *** hwoarang has quit IRC | 14:35 | |
| *** hwoarang has joined #openstack-infra | 14:36 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Ensure correct lexical sorting of node requests https://review.openstack.org/643352 | 14:37 |
| *** armax has joined #openstack-infra | 14:37 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Use include_role and delegate_to for upload-npm https://review.openstack.org/643353 | 14:38 |
| *** lpetrut has quit IRC | 14:38 | |
| *** lpetrut has joined #openstack-infra | 14:39 | |
| frickler | clarkb: I looked on the bionic node I held and didn't find the dir there. also codesearch only found the above and a place in kolla | 14:39 |
| *** gfidente has quit IRC | 14:39 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: Ensure correct lexical sorting of node requests https://review.openstack.org/643352 | 14:40 |
| frickler | elod: I'm thinking there might be a better solution, fixing things in the templates rather than in each project. please wait for feedback from gmann and maybe other config-cores before proceeding | 14:44 |
| *** hwoarang has quit IRC | 14:46 | |
| *** Swami has joined #openstack-infra | 14:47 | |
| gmann | frickler: elod checking. | 14:48 |
| elod | frickler: thanks, ok. actually i thought to do it per project as to fix the jobs in the templates would require to duplicate ~4 templates | 14:48 |
| gmann | frickler: elod this is only for publish-openstack-sphinx-docs and on ocata or on other stable branch too | 14:48 |
| elod | gmann: i only see failing failing sphinx job on ocata in the periodics | 14:49 |
| elod | so i think it's only ocata | 14:49 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul-jobs master: Support ~/.gnupg subdirs in remove-gpgkey role https://review.openstack.org/643328 | 14:50 |
| *** wolverineav has joined #openstack-infra | 14:50 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Run tox-py35 jobs on xenial https://review.openstack.org/643359 | 14:51 |
| *** Shrews has quit IRC | 14:51 | |
| *** mordred has quit IRC | 14:51 | |
| *** hwoarang has joined #openstack-infra | 14:52 | |
| *** lseki has joined #openstack-infra | 14:52 | |
| *** kjackal has quit IRC | 14:52 | |
| *** Shrews has joined #openstack-infra | 14:53 | |
| *** luizbag has quit IRC | 14:54 | |
| elod | gmann: there are 20+ failing stable periodic job on ocata in the today's run ( http://lists.openstack.org/pipermail/openstack-stable-maint/2019-March/date.html#start ) | 14:54 |
| *** wolverineav has quit IRC | 14:54 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Run tox-py35 jobs on xenial https://review.openstack.org/643359 | 14:54 |
| *** gfidente has joined #openstack-infra | 14:55 | |
| *** kjackal has joined #openstack-infra | 14:56 | |
| gmann | elod: ok, fungi ^^ that is not based on tox job ? | 14:56 |
| *** mordred has joined #openstack-infra | 14:57 | |
| *** yolanda has quit IRC | 14:57 | |
| *** dpawlik has joined #openstack-infra | 14:58 | |
| fungi | elod: gmann: i've just caught up in here (was focused on release job failures first) and am looking at the inheritance path for that job now | 15:01 |
| gmann | seems we have same name for job and template - publish-openstack-sphinx-docs | 15:02 |
| elod | gmann fungi : thanks. why i considered doing the fix directly in the projects' .zuul.yaml instead of project-templates: 2-4 templates to duplicate (in master of openstack-zuul-jobs, project-templates would look a bit less nicer); it is only necessary at the oldest branch: ocata; and the projects .zuul.yaml needs to be updated anyway (on stable/ocata only) | 15:03 |
| elod | but maybe i'm wrong | 15:03 |
| elod | :) | 15:03 |
| *** roman_g has quit IRC | 15:04 | |
| fungi | looks like most of our sphinx-based jobs are parented to build-openstack-sphinx-docs in openstack-zuul-jobs and it in turn is parented to build-sphinx-docs in zuul-jobs | 15:05 |
| fungi | so build-openstack-sphinx-docs probably needs two variants | 15:05 |
| *** roman_g has joined #openstack-infra | 15:06 | |
| fungi | we also likely need variants of publish-openstack-sphinx-docs-base | 15:06 |
| fungi | (in project-config) | 15:06 |
| fungi | since the publish jobs are all parented directly to that | 15:07 |
| *** roman_g_ has joined #openstack-infra | 15:07 | |
| gmann | yeah seems so | 15:07 |
| fungi | oh, and publish-openstack-tox-docs-base also doesn't have any explicit parents | 15:08 |
| *** luizbag has joined #openstack-infra | 15:08 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Update to a newer version (8) of Node.js for testing https://review.openstack.org/643046 | 15:08 |
| fungi | i'll get to work on patches to openstack-zuul-jobs and project-config for those three | 15:09 |
| *** apetrich has quit IRC | 15:10 | |
| *** luizbag has quit IRC | 15:10 | |
| gmann | fungi: thanks. | 15:11 |
| elod | fungi: thanks, let me know if i can help then e.g. with (i guess) adding the new "publish-openstack-sphinx-docs-ocata" to the projects... if that is needed anyway :) | 15:11 |
| *** roman_g has quit IRC | 15:11 | |
| *** lpetrut has quit IRC | 15:12 | |
| gmann | elod: thanks. i think fungi can handle it in base jobs via branch variant . we should not need new template or job for ocata. | 15:12 |
| elod | gmann: oh, great, thanks \o/ | 15:12 |
| *** rascasoft has joined #openstack-infra | 15:14 | |
| clarkb | I'm going to have a late start today. Taking the kids to school then grabbing a proper breakfast. Will be back after to help with node transition, puppet 4, and possible bridge resize? | 15:15 |
| *** ccamacho has quit IRC | 15:16 | |
| *** luizbag has joined #openstack-infra | 15:17 | |
| fungi | elod: it won't need a separate job name or any changes in leaf projects. we just create variants of these parent jobs which match on different sets of branches and then pin the nodeset to ubuntu-xenial on one of them | 15:17 |
| fungi | zuul knows which variant to inherit from based on the branch being tested | 15:18 |
| clarkb | dmsimard: mordred fungi do you know where we left off on the resize of bridge? are we still not satisfied that it is a safe operation? | 15:20 |
| fungi | clarkb: i think the delay was testing the snapshot | 15:21 |
| dmsimard | ianw had written https://etherpad.openstack.org/p/bridge-snapshot-bringup | 15:21 |
| fungi | which ianw worked through | 15:21 |
| clarkb | ya I used those steps to make the afs01.ord.o.o snapshot work as well | 15:21 |
| fungi | we just never circled back around to pull the trigger on the resize once that was nailed down | 15:21 |
| clarkb | got it | 15:21 |
| efried | fungi: Because you'll know this off the top, are https://review.openstack.org/#/c/643283/ py35 failures related to <thing you've been working on> and if so, safe to recheck yet and/or needing some other action? | 15:22 |
| dmsimard | yeah, we would've needed to do a new snapshot and restore so we were looking at another ~90mins of waiting and we needed to kick bridge.o.o back up | 15:22 |
| fungi | efried: hah, you have misplaced confidence in my knowledge, but i'll look ;) | 15:22 |
| dmsimard | I had already exhausted the bandwidth I had for that particular thing that day | 15:22 |
| *** mordred has quit IRC | 15:23 | |
| clarkb | dmsimard: the new snapshot and restore is to test ianw's process then if that looks good kick off the resuze? | 15:23 |
| clarkb | (wondering if we need to do the restore or if we are alreay happy that works given ianws notes) | 15:23 |
| fungi | efried: woof... okay so that's a lot of failures... those py35 jobs do indeed look like they tried to run on xenial so we probably need to find a common parent for them and make sure it's updated with nodeset: ubuntu-xenial | 15:23 |
| *** mriedem is now known as mriedem_away | 15:24 | |
| dmsimard | clarkb: something like that | 15:24 |
| openstackgerrit | Serena Ziviani proposed openstack/diskimage-builder master: Also use selinuxenabled to check selinux status https://review.openstack.org/643367 | 15:24 |
| dmsimard | clarkb: it's our safety net if the resize doesn't end up working for some reason | 15:25 |
| *** dpawlik has quit IRC | 15:25 | |
| efried | fungi: I'm afraid I didn't understand all of that. Mainly the things involving "parent" and "nodeset". Is there something I need to (/can) do? | 15:25 |
| fungi | efried: the npm failure, similarly, looks bionic-related (though a different npm problem than elsewhere i suspect... this looks like a package split going from xenial to bionic and our princess^W^W is in another castle^Wpackage now) | 15:26 |
| efried | fungi: I assume has nothing to do with the specific patch and will affect anything in requirements? <== prometheanfire | 15:26 |
| fungi | mordred would be interested if he hadn't fallen off irc a while back | 15:26 |
| fungi | efried: yes, i think this is a requirements-wide concern. need to track down what parent the requirements-tox-py35-check-uc and cross-.*-py35 jobs have we could force to use xenial (bionic doesn't have python 3.5) | 15:27 |
| *** mordred has joined #openstack-infra | 15:29 | |
| efried | fungi: okay, lmk if there's anything I can do to help | 15:30 |
| mordred | efried: my irc bouncer went away for a sec - is fungi helping with the held node? | 15:32 |
| efried | mordred: No, unrelated thing | 15:32 |
| efried | mordred: requirements is busted because py35 jobs are bastards | 15:32 |
| mordred | ok. looks like the thing failed - is your public key somewhere accessible? | 15:32 |
| efried | ...or something to do with wrong parent | 15:32 |
| corvus | fungi: openstack-tox is the nearest common ancestor | 15:34 |
| *** agopi has quit IRC | 15:34 | |
| efried | do we need a "genaeology" script for zuul? | 15:35 |
| corvus | wouldn't be hard to write; we have the api for it now. but it's not hard to click the parent fields in http://zuul.openstack.org/job/requirements-tox-py35-check-uc and http://zuul.openstack.org/job/cross-cinder-py35 | 15:36 |
| *** ccamacho has joined #openstack-infra | 15:36 | |
| corvus | requirements-tox-py35-check-uc -> openstack-tox <- requirements-cross-test <- cross-cinder-py35 | 15:36 |
| mordred | you can also go to http://zuul.openstack.org/jobs and put requirements-tox-py35-check-uc in the search box and see the tree fwiw | 15:37 |
| corvus | yeah, though that only supports one at a time | 15:38 |
| mordred | indeed | 15:38 |
| corvus | we should make it support >1 -- the status page does | 15:38 |
| mordred | ++ | 15:38 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/openstack-zuul-jobs master: Use ubuntu-xenial for build-openstack-sphinx-docs https://review.openstack.org/643371 | 15:39 |
| fungi | elod: gmann: actually it was simpler than i expected for that first one ^ | 15:39 |
| corvus | fungi: mordred left a comment on the gnupg patch | 15:39 |
| fungi | corvus: thanks for running down those requirements py35 ancestors, those are backlogged behind about three other jobs i'm still working out fixes for | 15:40 |
| corvus | ah, maybe i can keep working on that then :) | 15:41 |
| * corvus attempts to jump into fray | 15:41 | |
| fungi | appreciated if you have the time! | 15:41 |
| fungi | my guess is they just need to be reparented to openstack-tox-py35 given what you found | 15:41 |
| openstackgerrit | Merged openstack-infra/storyboard master: Pin tox-py35 jobs to ubuntu-xenial https://review.openstack.org/643318 | 15:41 |
| elod | fungi: yeah, that's quite simple :) i just thought we only want to force xenial for ocata and this will use xenial for up to rocky | 15:42 |
| elod | fungi: thanks again for the help :) | 15:43 |
| fungi | elod: in the case of build-openstack-sphinx-docs it's not supposed to be used for rocky anyway, but regardless stable/rocky jobs ought to use ubuntu-xenial too | 15:43 |
| fungi | we're transitioning to ubuntu-bionic for stable/stein | 15:43 |
| *** hamzy has quit IRC | 15:43 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Displayed tags should link to search view for the tag https://review.openstack.org/642262 | 15:44 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Pagination error in active/merged/invalid stories in projects resolved. https://review.openstack.org/642224 | 15:44 |
| corvus | efried: my read of this is that since there is no ancestor of the various -py35 jobs which is itself -py35 specific, we'll need to add a nodeset line to all of them individually. i can work up a patch for that real quick and show it to you and fungi for approval. | 15:45 |
| *** yolanda has joined #openstack-infra | 15:45 | |
| efried | ack | 15:46 |
| elod | fungi: ok, thanks again! :) | 15:46 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Show tags for all stories in a project group https://review.openstack.org/642867 | 15:46 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Enhance adding tags on story details page https://review.openstack.org/642643 | 15:46 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Make task ordering in Story view selectable https://review.openstack.org/641930 | 15:46 |
| corvus | efried, fungi (for your backlog): https://review.openstack.org/643375 Use xenial for all py35 jobs | 15:47 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul-jobs master: Support ~/.gnupg subdirs in remove-gpgkey role https://review.openstack.org/643328 | 15:47 |
| efried | corvus: If I make the req patch Depends-On that...? | 15:48 |
| corvus | efried: yep. | 15:49 |
| prometheanfire | I'm guessing https://review.openstack.org/643375 should help master | 15:50 |
| efried | corvus: done https://review.openstack.org/#/c/643283/ | 15:50 |
| efried | prometheanfire: FYI ^ | 15:50 |
| *** ramishra has quit IRC | 15:51 | |
| prometheanfire | efried: ah, probably part of the bot update, which is being ignored during the freeze, thanks | 15:52 |
| *** yamamoto has joined #openstack-infra | 15:57 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Sphinx publishing adjustments for ubuntu-bionic https://review.openstack.org/643381 | 16:00 |
| fungi | elod: gmann: those are the other related job adjustments ^ | 16:00 |
| openstackgerrit | Camila Moura proposed openstack-infra/storyboard master: Remove simple priority support from the API https://review.openstack.org/643260 | 16:02 |
| *** Lucas_Gray has quit IRC | 16:04 | |
| *** diablo_rojo has joined #openstack-infra | 16:05 | |
| fungi | mordred: i know you love trying to interpret obscure ansible-lint exceptions... i have no clue what the one on your https://review.openstack.org/643353 change is trying to tell us | 16:06 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: web: support more than one job in the filter https://review.openstack.org/643385 | 16:07 |
| corvus | efried, mordred: ^ :) | 16:07 |
| corvus | here's what that looks like: https://screenshots.firefox.com/9g4MgmP0fS1rFaQi/localhost | 16:09 |
| *** dpawlik has joined #openstack-infra | 16:09 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Support ~/.gnupg subdirs in remove-gpgkey role https://review.openstack.org/643328 | 16:10 |
| mordred | fungi: oh - it' because I'm dumb | 16:11 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Use include_role and delegate_to for upload-npm https://review.openstack.org/643353 | 16:12 |
| openstackgerrit | Camila Moura proposed openstack-infra/storyboard master: Remove simple priority support from the API https://review.openstack.org/643260 | 16:13 |
| *** dpawlik has quit IRC | 16:13 | |
| *** agopi has joined #openstack-infra | 16:14 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: use the new redirect target for constraints when branching https://review.openstack.org/643387 | 16:15 |
| *** ykarel has joined #openstack-infra | 16:16 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Add a linter for our .scss files https://review.openstack.org/643025 | 16:17 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Add a linter for HTML templates https://review.openstack.org/643032 | 16:17 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Add a way to create tasks in a non-master branch https://review.openstack.org/643388 | 16:17 |
| *** auristor has quit IRC | 16:18 | |
| gmann | fungi: we do not need branch variant for openstack-tox-docs because that is not supposed to be used by stable branch before stein right ? it is from stein onwards | 16:18 |
| *** mattw4 has joined #openstack-infra | 16:19 | |
| fungi | gmann: perhaps i misread the deprecation, looked like it was also used by rocky? | 16:19 |
| fungi | or meant to be used for rocky at least | 16:19 |
| gmann | humm, i checked nova and cinder and they use build-openstack-sphinx-docs but not sure about deprecation things | 16:20 |
| gmann | nova and cinder rocky patch | 16:20 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: use the new redirect target for constraints when branching https://review.openstack.org/643387 | 16:20 |
| fungi | gmann: in the related build-openstack-sphinx-docs description it says "This job is obsolete and should not be used anymore since rocky..." which i take to mean stable/rocky and later are expected to use the tox-based one | 16:21 |
| *** gfidente has quit IRC | 16:22 | |
| fungi | but i'm up for trying just dropping the variant and seeing how things go | 16:22 |
| gmann | yeah seems so. we can be safe in case if anyone using tox based in rocky | 16:24 |
| gmann | fungi: you are doing or want me to do | 16:24 |
| *** auristor has joined #openstack-infra | 16:24 | |
| elod | i also saw only publish-openstack-sphinx-docs-base fails, but i checked only the periodic-stable results | 16:24 |
| fungi | gmann: i'll update it now | 16:25 |
| elod | i mean publish-openstack-sphinx-docs fails | 16:25 |
| gmann | fungi: perfect thanks | 16:25 |
| fungi | i'm done with my double-booked meeting slot so only in a single meeting at the moment ;) | 16:25 |
| gmann | :), i am suppose to be in QA office hour. | 16:25 |
| *** kopecmartin is now known as kopecmartin|off | 16:26 | |
| gmann | elod: this will fix - https://review.openstack.org/#/c/643381/1 | 16:27 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Use Xenial for publish-openstack-sphinx-docs-base https://review.openstack.org/643381 | 16:30 |
| fungi | gmann: ^ updated to drop the variant | 16:30 |
| elod | gmann: got it, thanks | 16:31 |
| *** jtomasek has quit IRC | 16:35 | |
| fungi | any other config-core reviewers up for looking at mordred's upload-npm fix for bionic? https://review.openstack.org/643353 | 16:36 |
| *** e0ne has quit IRC | 16:38 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Add a role to mirror a git repository to a remote git server https://review.openstack.org/643394 | 16:38 |
| *** pgaxatte has quit IRC | 16:39 | |
| *** agopi has left #openstack-infra | 16:40 | |
| *** jamesmcarthur has quit IRC | 16:42 | |
| *** gfidente has joined #openstack-infra | 16:44 | |
| *** ykarel has quit IRC | 16:44 | |
| *** Swami has quit IRC | 16:45 | |
| *** lujinluo has joined #openstack-infra | 16:45 | |
| *** emine__ has quit IRC | 16:46 | |
| *** jamesmcarthur has joined #openstack-infra | 16:47 | |
| *** dpawlik has joined #openstack-infra | 16:48 | |
| slaweq | frickler: clarkb: and other infra cores, please check logs from job http://logs.openstack.org/84/643084/6/check/tobiko-devstack-neutron-fedora/d3d73d0/ | 16:50 |
| slaweq | frickler: clarkb: there is interesting thing, devstack on both compute-1 and compute-2 failed | 16:50 |
| slaweq | but job wasn't stopped then, tests were running and failed because there was no available compute nodes there | 16:51 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Pin nodejs to come from nodesource repo https://review.openstack.org/643397 | 16:51 |
| slaweq | but should job be stopped in case when on one subnodes is not deployed properly? | 16:51 |
| clarkb | fungi that npm change lgtm but I cant vote from my phone | 16:52 |
| clarkb | feel free to approve and note Im good with it? | 16:52 |
| *** dpawlik has quit IRC | 16:52 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: web: support more than one job in the filter https://review.openstack.org/643385 | 16:53 |
| *** mriedem_away is now known as mriedem | 16:54 | |
| *** jaosorior has quit IRC | 16:57 | |
| *** jaosorior has joined #openstack-infra | 16:58 | |
| *** lujinluo has quit IRC | 16:59 | |
| *** dpawlik has joined #openstack-infra | 16:59 | |
| *** wolverineav has joined #openstack-infra | 16:59 | |
| *** guimaluf has quit IRC | 16:59 | |
| *** rpioso is now known as rpioso|afk | 17:01 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/openstack-zuul-jobs master: nodeset adjustment for openstack-tox-docs https://review.openstack.org/643399 | 17:01 |
| gmann | fungi: ^^ i found few project using openstack-tox-docs on stable/rocky | 17:02 |
| *** wolverineav has quit IRC | 17:02 | |
| *** wolverin_ has joined #openstack-infra | 17:02 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 17:02 | |
| *** wolverin_ has quit IRC | 17:02 | |
| *** wolverineav has joined #openstack-infra | 17:02 | |
| *** dpawlik has quit IRC | 17:03 | |
| gmann | fungi: same for publish-openstack-tox-docs-base, we should keep two variant for this as both were added as part of new template on projetc side | 17:04 |
| *** rpittau is now known as rpittau|afk | 17:05 | |
| fungi | gmann: okay, so that just switches back to the original patchset basically? | 17:06 |
| gmann | yeah. | 17:06 |
| *** udesale has quit IRC | 17:06 | |
| *** trown is now known as trown|lunch | 17:06 | |
| *** gyee has joined #openstack-infra | 17:07 | |
| fungi | gmann: can we just switch 643381 back and not create yet another change? | 17:07 |
| fungi | gmann: oh! that's for the openstack-zuul-jobs one | 17:07 |
| fungi | now i get it | 17:07 |
| fungi | i'll switch 643381 back to an equivalent of patchset #1 in that case | 17:08 |
| elod | gmann: isn't that enough to wait if we see such failure? i haven't seen problem with bionic on stable/rocky (of course it doesn't mean there arent't any) | 17:08 |
| mriedem | maybe a crazy question, but is it possible to make variables in zuul jobs conditional based on branch? | 17:08 |
| mriedem | i'd think so since it's just ansible | 17:08 |
| mriedem | because https://bugs.launchpad.net/devstack-plugin-ceph/+bug/1820098 | 17:08 |
| openstack | Launchpad bug 1820098 in devstack-plugin-ceph "devstack-plugin-ceph-tempest jobs broken with multiattach tests on stable branches" [Undecided,Confirmed] | 17:08 |
| mordred | mriedem: yeah, you can make a variant for a branch with different variables | 17:09 |
| *** hamzy has joined #openstack-infra | 17:09 | |
| mriedem | ok i guess gmann knows the special sauce to do that so i'll let him do it | 17:10 |
| mriedem | it's really just thousand island dressing left out in the sun for awhile | 17:10 |
| mordred | mriedem: gmann is very smart and knows al the things | 17:10 |
| *** dpawlik has joined #openstack-infra | 17:11 | |
| gmann | :) | 17:11 |
| fungi | gmann: okay, switched back again ^ | 17:11 |
| gmann | elod: yea no failure but let's be consistent of using xenial on stable branch and bionic on stein onwards | 17:11 |
| fungi | er, i was gonna. looks like gerrit thinks it's too similar | 17:11 |
| gmann | fungi: thanks | 17:11 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Sphinx publishing adjustments for ubuntu-bionic https://review.openstack.org/643381 | 17:12 |
| fungi | there ^ we go | 17:12 |
| fungi | i tweaked the commit message so gerrit wouldn't refuse | 17:12 |
| *** dpawlik has quit IRC | 17:15 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: web: support more than one job in the filter https://review.openstack.org/643385 | 17:16 |
| *** dtantsur is now known as dtantsur|afk | 17:18 | |
| clarkb | back now | 17:20 |
| *** psachin has quit IRC | 17:21 | |
| clarkb | fungi: https://review.openstack.org/#/c/643381/3/zuul.d/jobs.yaml the branch matchers there will prevent the job from running on release pipelines aiui | 17:24 |
| clarkb | because refs/tags/foo doesn't match refs/heads/master etc | 17:25 |
| fungi | clarkb: but there's also a variant without | 17:26 |
| clarkb | ah | 17:26 |
| fungi | the tags will match the variant with the negative match (they may simply break if they don't work with bionic) | 17:26 |
| clarkb | oh I see, because its not these branches as the rule the tags should match that? | 17:27 |
| fungi | this is something i've already warned the release team about anyway. worst case we resurrect corvus's work on branch guessing for tag events | 17:27 |
| fungi | but yes, afaik a tag ought to match a variant with has a purely negative branch matcher. if that's not the case i'll need to rethink some things | 17:28 |
| fungi | at least the expression there will also match on a branch which is the empty string | 17:29 |
| fungi | anyway, i need to go grab some lunch. will see where we stand with the in-flight patches for bionic fallout when i get back and find out what else it's broken | 17:30 |
| clarkb | infra-root if I can get a second reviewer on https://review.openstack.org/#/c/634726/6 I can watch that go in and verify things are happy after it takes effect | 17:31 |
| clarkb | also for the bridge resize: We want to make a new snapshot, boot it using ianw's process, then run the resize today? | 17:32 |
| cmurphy | \o/ | 17:32 |
| corvus | clarkb: that sounds great | 17:32 |
| clarkb | ok I'll trigger a snapshot of bridge now | 17:32 |
| clarkb | snapshot called bridge.openstack.org-20190314 is in progress | 17:33 |
| *** chandankumar is now known as chkumar246 | 17:34 | |
| *** s0ru has joined #openstack-infra | 17:34 | |
| *** panda is now known as panda|off | 17:34 | |
| *** derekh has quit IRC | 17:38 | |
| *** jpich has quit IRC | 17:38 | |
| corvus | clarkb, fungi, gmann, mordred: until 643385 lands and is deployed, you can use the preview at http://logs.openstack.org/85/643385/3/check/zuul-build-dashboard/e55539e/npm/html/status to get nearest common ancestor jobs if you need it for the py35 work. | 17:38 |
| *** ginopc has quit IRC | 17:38 | |
| clarkb | that is a neat side effect of the pre merge hosting | 17:38 |
| mordred | you have to remove /status for it to work though | 17:39 |
| mordred | because of reasons | 17:39 |
| corvus | clarkb, fungi, gmann, mordred: until 643385 lands and is deployed, you can use the preview at http://logs.openstack.org/85/643385/3/check/zuul-build-dashboard/e55539e/npm/html/ to get nearest common ancestor jobs if you need it for the py35 work. | 17:39 |
| corvus | i didn't realize it self-modified the url | 17:39 |
| corvus | clarkb, fungi, gmann, mordred: just click the jobs page and put two job names in the search filter, space or comma separated | 17:41 |
| *** priteau has joined #openstack-infra | 17:45 | |
| * mordred afks for a little bit | 17:46 | |
| *** gfidente is now known as gfidente|afk | 17:46 | |
| *** e0ne has joined #openstack-infra | 17:46 | |
| gmann | corvus: thanks this is nice. | 17:50 |
| gmann | does it show all derived jobs also or few ? i did not find all for tempest-full case | 17:50 |
| *** dpawlik has joined #openstack-infra | 17:51 | |
| corvus | gmann: when filtering, it only shows the job and its parents | 17:52 |
| gmann | got it, found all jobs without filter, thanks | 17:53 |
| *** eharney has quit IRC | 17:54 | |
| *** dpawlik has quit IRC | 17:55 | |
| clarkb | ok that image is active now. infra-root there shouldn't be any problem booting the bridge image because we only allow root ssh into the nodes from a specific IP right | 17:55 |
| clarkb | if we feel comfortable with that I'll boot the image and run through ianw's notes | 17:55 |
| openstackgerrit | Riju Khatri proposed openstack-infra/storyboard-webclient master: recheck https://review.openstack.org/643412 | 17:59 |
| *** panda|off has quit IRC | 18:01 | |
| *** trown|lunch is now known as trown | 18:01 | |
| *** panda has joined #openstack-infra | 18:04 | |
| efried | mordred: Curiouser and curiouser. I put a breakpoint into _get_client and restarted the actual n-cpu on that node. When it comes up, my debug log fires with... | 18:11 |
| efried | ('EFRIED: endpoint: %s', None) | 18:11 |
| *** wolverineav has quit IRC | 18:16 | |
| dhellmann | config-core: The release team needs to land https://review.openstack.org/#/c/643387/ before we start creating more stable/stein branches next week. We would appreciate it if you put it in your review queues. Thanks! | 18:17 |
| clarkb | dont make the redirect a permanent one :) | 18:18 |
| *** wolverineav has joined #openstack-infra | 18:19 | |
| *** wolverineav has quit IRC | 18:21 | |
| mnaser | clarkb, dhellmann: added a small comment to that change | 18:22 |
| *** wolverineav has joined #openstack-infra | 18:22 | |
| clarkb | mnaser: ah good point | 18:23 |
| fungi | okay, i'm back from lunch and catching up again | 18:23 |
| *** rlandy is now known as rlandy|biab | 18:25 | |
| *** jpena is now known as jpena|off | 18:27 | |
| *** wolverineav has quit IRC | 18:28 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: use the new redirect target for constraints when branching https://review.openstack.org/643387 | 18:28 |
| efried | mordred: actually, looking at latest log: http://logs.openstack.org/99/642899/6/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/1bbc3f0/controller/logs/screen-n-cpu.txt.gz?level=ERROR you'll note the log hits twice - first time okay, second time None | 18:29 |
| dhellmann | mnaser , clarkb : I did a little more cleanup, including the commit message | 18:29 |
| *** wolverineav has joined #openstack-infra | 18:29 | |
| efried | so clearly something is getting blown up | 18:29 |
| *** jamesmcarthur has quit IRC | 18:30 | |
| *** jamesmcarthur has joined #openstack-infra | 18:33 | |
| *** e0ne has quit IRC | 18:33 | |
| *** notmyname has quit IRC | 18:38 | |
| mnaser | clarkb: btw it looks like the redirect goes to http and not https | 18:38 |
| openstackgerrit | Ankita Bansal proposed openstack-infra/storyboard-webclient master: Provide -Assign to self- button for story tasks. https://review.openstack.org/643427 | 18:38 |
| mnaser | im not sure if pypi or friends might complain | 18:38 |
| *** roman_g_ has quit IRC | 18:39 | |
| *** notmyname has joined #openstack-infra | 18:39 | |
| *** e0ne has joined #openstack-infra | 18:40 | |
| *** eharney has joined #openstack-infra | 18:43 | |
| fungi | config-core: the problem we hope will be fixed by 643353 is blocking release requests for horizon plug-ins if you have a chance to take a look | 18:47 |
| clarkb | fungi: approved. I noted earlier it looked fine but was on my phone at breakfast so couldn't vote on it | 18:48 |
| clarkb | fungi: care to review https://review.openstack.org/#/c/634726/ I can watch it go in | 18:49 |
| clarkb | and finally thoughts on booting off my bridge snapshot? I think it should be safe because our ansible ssh must come from a specific IP address | 18:49 |
| AJaeger | could I get a second +2 on https://review.openstack.org/#/c/643196 to unblock 643353 which is stacked on top? | 18:50 |
| AJaeger | fungi, clarkb ^ | 18:51 |
| *** gmann is now known as gmann_afk | 18:51 | |
| clarkb | AJaeger: done | 18:52 |
| AJaeger | thanks | 18:52 |
| fungi | clarkb: 634726 lgtm, approved now | 18:55 |
| clarkb | thanks | 18:55 |
| fungi | and yeah, a test boot of a bridge snapshot seems like it should be entirely safe | 18:55 |
| openstackgerrit | Merged openstack-infra/zuul master: Run tox-py35 jobs on xenial https://review.openstack.org/643359 | 18:56 |
| *** priteau has quit IRC | 18:56 | |
| clarkb | I guess we can disable the cron when attached to the recovery host | 18:57 |
| clarkb | to be extra cautiou | 18:57 |
| openstackgerrit | Merged openstack-infra/project-config master: Run node jobs using version 4 on xenial nodes https://review.openstack.org/643196 | 19:00 |
| openstackgerrit | Merged openstack-infra/project-config master: Use include_role and delegate_to for upload-npm https://review.openstack.org/643353 | 19:01 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: nodeset adjustment for openstack-tox-docs https://review.openstack.org/643399 | 19:02 |
| *** rlandy|biab is now known as rlandy | 19:02 | |
| efried | mordred: meanwhile I'm making progress by constructing the adapter-for-direct-use separately (with min_version=1.46) from the adapter-for-use-by-ironicclient | 19:03 |
| efried | s/46/0/ | 19:03 |
| openstackgerrit | Merged openstack-infra/project-config master: Sphinx publishing adjustments for ubuntu-bionic https://review.openstack.org/643381 | 19:04 |
| AJaeger | So, all Xenial/Bionic open changes reviewed. | 19:05 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Pin nodejs to come from nodesource repo https://review.openstack.org/643397 | 19:05 |
| AJaeger | config-core, another +2 for fungi's change https://review.openstack.org/643371 would be nice to close out open changes | 19:05 |
| AJaeger | config-core, also general review of repos would be nice to reduce backlog... | 19:06 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Add job and integration tests for the upload-git-mirror role https://review.openstack.org/643435 | 19:07 |
| *** wolverineav has quit IRC | 19:07 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Add a role to mirror a git repository to a remote git server https://review.openstack.org/643394 | 19:07 |
| openstackgerrit | Merged openstack-infra/project-config master: use the new redirect target for constraints when branching https://review.openstack.org/643387 | 19:10 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Do not merge: test zuul-jobs-upload-git-mirror job https://review.openstack.org/643437 | 19:12 |
| clarkb | was the 8GB flavor the one we want to transition bridge to? | 19:13 |
| clarkb | (I'll be sure to boot the snapshot on the flavor we want to go to so that if we have to recover we can recover on the flavor we want) | 19:13 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Do not merge: test zuul-jobs-upload-git-mirror job https://review.openstack.org/643437 | 19:14 |
| *** priteau has joined #openstack-infra | 19:16 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Use ubuntu-xenial for build-openstack-sphinx-docs https://review.openstack.org/643371 | 19:18 |
| fungi | clarkb: i think so, yes, so as to also make room to move the ara db local to the server | 19:18 |
| clarkb | fungi: heh we've broken our xenial testing for system-config http://logs.openstack.org/26/634726/6/gate/infra-puppet-apply-3-ubuntu-xenial/4e84d82/job-output.txt.gz#_2019-03-14_19_07_22_757620 | 19:19 |
| clarkb | fungi: any idea if ^ has patches to review yet? | 19:19 |
| fungi | woo! i guess that depended on the default nodeset? no fixes in flight yet that i'm aware | 19:19 |
| clarkb | ya that is my guess | 19:19 |
| dmsimard | unless I screwed up, https://review.openstack.org/#/c/643437/ is not picking up a new job I declared in https://review.openstack.org/#/c/643435/ ... should we be merging the tests, add the job to project-config and then we can use it ? | 19:19 |
| fungi | i was less concerned with running down all the random places we might have broken jobs for infra projects because i knew we'd be more able to fix them quickly | 19:20 |
| dmsimard | The parent patch of 643437 is the one that contains the Depends-On to the job addition, maybe I can try adding it on the child patch too. | 19:21 |
| clarkb | ok I'll get a patch up in a sec | 19:21 |
| *** priteau has quit IRC | 19:21 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Do not merge: test zuul-jobs-upload-git-mirror job https://review.openstack.org/643437 | 19:23 |
| dmsimard | Yeah, doesn't look like it gets picked up :/ | 19:24 |
| fungi | dmsimard: missing the depends-on: https://review.openstack.org/643435 in the commit message for 643437? | 19:24 |
| openstackgerrit | Clark Boylan proposed openstack-infra/openstack-zuul-jobs master: Run infra puppet apply xenial jobs on xenial nodes https://review.openstack.org/643441 | 19:24 |
| clarkb | fungi: AJaeger ^ fix for those infra puppet apply jobs | 19:24 |
| dmsimard | fungi: that was the only thing I could think of too -- I added it just now but it's not getting picked up either | 19:24 |
| *** rpioso|afk is now known as rpioso | 19:25 | |
| fungi | dmsimard: why would a proposed change pick up a job defined in another proposed change without an explicit depends-on anyway? | 19:25 |
| dmsimard | fungi: the parent patch had the depends-on | 19:25 |
| fungi | oh, sorry, i missed there was a parent of 643437 | 19:26 |
| dmsimard | so I looked at the logs and it seems I may have screwed up with the file matcher... 2019-03-14 19:23:39,603 DEBUG zuul.layout: Job <Job zuul-jobs-upload-git-mirror branches: None source: opendev/base-jobs/zuul.yaml@master#75> did not match files in <Change 0x7f26b9e20400 openstack-infra/zuul-jobs 643437,3> | 19:26 |
| dmsimard | OH | 19:26 |
| dmsimard | I know what's up | 19:27 |
| fungi | 643397 doesn't touch any of those files | 19:27 |
| dmsimard | there aren't changes to the locations in the patch I sent to test the jobs T_T | 19:27 |
| dmsimard | yeah | 19:27 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Do not merge: test zuul-jobs-upload-git-mirror job https://review.openstack.org/643437 | 19:28 |
| clarkb | er I probably should've rebased so I'll do that now | 19:28 |
| dmsimard | works now, \o/ | 19:29 |
| fungi | good deal. no new zuul bugs ;) | 19:29 |
| openstackgerrit | Clark Boylan proposed openstack-infra/openstack-zuul-jobs master: Run infra puppet apply xenial jobs on xenial nodes https://review.openstack.org/643441 | 19:29 |
| openstackgerrit | Clark Boylan proposed openstack-infra/openstack-zuul-jobs master: Run legacy puppet 3 syntax checks on xenial https://review.openstack.org/643446 | 19:29 |
| clarkb | there we go | 19:29 |
| dmsimard | In unrelated news, the multi-node-bridge role is failing tests for Suse -- not sure what's up with that: http://logs.openstack.org/35/643435/1/check/openstack-infra-multinode-integration-opensuse-tumbleweed/04a71ac/job-output.txt.gz#_2019-03-14_19_12_51_525109 | 19:30 |
| clarkb | ovs_package is undefined | 19:31 |
| clarkb | possible that tumbleweed doesn't match the other suse vars file matcher? | 19:31 |
| dmsimard | oh, that looks like it | 19:31 |
| dmsimard | just above, it includes "default.yaml" instead of "Suse.yaml" | 19:31 |
| clarkb | dmsimard: in the zuul info dir we capture the facts for all the hosts | 19:32 |
| *** kjackal has quit IRC | 19:32 | |
| clarkb | http://logs.openstack.org/35/643435/1/check/openstack-infra-multinode-integration-opensuse-tumbleweed/04a71ac/zuul-info/host-info.primary.yaml | 19:32 |
| *** rossella_s has quit IRC | 19:32 | |
| dmsimard | "ansible_distribution: openSUSE Tumbleweed Kubic" | 19:33 |
| dmsimard | yikes | 19:33 |
| dmsimard | "ansible_os_family: openSUSE Tumbleweed Kubic" | 19:33 |
| clarkb | ansible_distribution_file_variety: SUSE might be a more reliable thing? | 19:33 |
| dmsimard | I guess that used to return "Suse" | 19:33 |
| clarkb | (I don't know what that is trying to tell me though) | 19:34 |
| clarkb | dmsimard: for tumbleweed that is probably correct for distribution, but family should be suse or opensuse imo | 19:34 |
| clarkb | ansible_os_family: openSUSE Tumbleweed Kubic | 19:34 |
| dmsimard | AJaeger, evrardjp: are you aware that this might have changed in Ansible somewhat recently ? ^ | 19:34 |
| *** rossella_s has joined #openstack-infra | 19:34 | |
| clarkb | dmsimard: it could be a change in /etc/os-release for tumbleweed too | 19:34 |
| *** kjackal has joined #openstack-infra | 19:35 | |
| dmsimard | true | 19:35 |
| clarkb | (or whatever similar things ansible looks at for facts) | 19:35 |
| dmsimard | need to pick up kids at school, be back later | 19:35 |
| clarkb | clarkb-test-bridge-snapshot-boot is booting in dfw to test the bridge snapshot bring up | 19:36 |
| clarkb | fungi: can we get a review on https://review.openstack.org/#/c/643446/1 too? that should fix the other failing jobs on system-config changes | 19:38 |
| fungi | mordred: looks like the delegation fails to get access to the work tree? http://logs.openstack.org/b5/b5dbb90fb347db2876f61cae8c2623f9ce767dfe/release/release-openstack-javascript/1c22748/job-output.txt.gz#_2019-03-14_19_36_52_223039 | 19:41 |
| fungi | (i guess because it's on a different machine?) | 19:41 |
| *** wolverineav has joined #openstack-infra | 19:42 | |
| *** wolverineav has quit IRC | 19:43 | |
| *** wolverineav has joined #openstack-infra | 19:44 | |
| *** yolanda has quit IRC | 19:46 | |
| *** wolverineav has quit IRC | 19:47 | |
| fungi | did anybody happen to get to the bottom of the psycopg2/libresolv "symbol __res_maybe_init version GLIBC_PRIVATE not defined in file libc.so.6" errors yet? are we trying to use a wheel built against the wrong distro? | 19:51 |
| *** dpawlik has joined #openstack-infra | 19:51 | |
| *** wolverineav has joined #openstack-infra | 19:51 | |
| AJaeger | dmsimard: I'm not - hope evrardjp or cmurphy can help (and read backscroll later) | 19:52 |
| *** wolverineav has quit IRC | 19:52 | |
| AJaeger | "openSUSE Tumbleweed Kubic" looks quite wrong - which image is this? | 19:53 |
| *** wolverineav has joined #openstack-infra | 19:53 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Run infra puppet apply xenial jobs on xenial nodes https://review.openstack.org/643441 | 19:53 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Run legacy puppet 3 syntax checks on xenial https://review.openstack.org/643446 | 19:53 |
| clarkb | my new server is still in a build status | 19:54 |
| * clarkb waits patiently | 19:55 | |
| AJaeger | dmsimard: I pinged one of the Tumbleweed Kubic engineers, but he might be offline as well now ;( | 19:55 |
| *** dpawlik has quit IRC | 19:56 | |
| *** gfidente|afk is now known as gfidente | 19:57 | |
| dmsimard | clarkb: if you're booting off the bridge snapshot, it took over an hour last time | 20:02 |
| clarkb | dmsimard: good to know, thanks (and ya I made a new snapshot today and booting that) | 20:02 |
| *** dpawlik has joined #openstack-infra | 20:02 | |
| *** dpawlik has quit IRC | 20:07 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Add a role to mirror a git repository to a remote git server https://review.openstack.org/643394 | 20:09 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Add job and integration tests for the upload-git-mirror role https://review.openstack.org/643435 | 20:10 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Do not merge: test zuul-jobs-upload-git-mirror job https://review.openstack.org/643437 | 20:10 |
| mordred | fungi: hrm. I guess the delegate-to isn't transitive across the tasks in that role - or just doesn't work quite that way :( | 20:14 |
| mordred | fungi: pondering ... | 20:14 |
| *** Lucas_Gray has joined #openstack-infra | 20:17 | |
| clarkb | for reasons I don't yet comprehend our beaker jobs use this nodeset definition: http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/.zuul.yaml#n2 | 20:17 |
| clarkb | fungi: ^ any idea if that is planned to change? | 20:18 |
| clarkb | (we can get ahead of that change for our beaker jobs if so) | 20:18 |
| fungi | i have no idea. what does that end up running on? | 20:19 |
| fungi | oh, xenial | 20:19 |
| fungi | says so right there. i'm blind | 20:19 |
| clarkb | ya so our puppet jobs are happy with it being xenial :) | 20:20 |
| fungi | looks to me like it's only referenced by 4 job definitions in http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/jobs.yaml | 20:20 |
| clarkb | might be a candidate to replace the nodeset in those jobs with eg ubuntu-xenial then remove teh nodeset from d-g? | 20:21 |
| fungi | looks like legacy-dsvm-base uses it only on old branches | 20:22 |
| clarkb | ah so can't remove the nodeset from d-g but could rehome the puppet jobs | 20:22 |
| fungi | same goes with legacy-publish-openstack-artifacts | 20:22 |
| fungi | and puppet-beaker-rspec-infra as you already noted | 20:22 |
| fungi | oh, and legacy-puppet-openstack-integration | 20:23 |
| clarkb | probably not urgent given only old branch jobs use it with devstack gate | 20:23 |
| fungi | so the two which aren't using that nodeset exclusively on old branches are puppet-beaker-rspec-infra and legacy-puppet-openstack-integration | 20:23 |
| clarkb | something we can clean up once bionic transition is settled | 20:23 |
| *** kgiusti has left #openstack-infra | 20:24 | |
| fungi | i thnik, yes, we can update puppet-beaker-rspec-infra and see if that works on ubuntu-bionic | 20:24 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Add job and integration tests for the upload-git-mirror role https://review.openstack.org/643435 | 20:24 |
| fungi | and we should reach out to the puppet-openstack crowd with regards to the legacy-puppet-openstack-integration job | 20:24 |
| clarkb | fungi: oh I know it won't work on bionic | 20:25 |
| clarkb | the puppet versions we run aren't available on bionic | 20:25 |
| *** priteau has joined #openstack-infra | 20:25 | |
| clarkb | (and we want to test hte platforms we run puppet on) | 20:25 |
| fungi | ahh, well xenial then, but yeah the nodeset isn't going anywhere so maybe just don't worry about it and we can deprecate it in the near future hopefully | 20:26 |
| clarkb | ya | 20:26 |
| *** jamesmcarthur has quit IRC | 20:36 | |
| fungi | okay, starting to dig into the psycopg2 thing now and see if maybe we're serving problematic wheels | 20:37 |
| clarkb | I'm waiting for puppet 4 things to merge and bridge snapshot to boot | 20:38 |
| ianw | fungi: literally just turned on ... is it looking like a corrupt wheel again? | 20:38 |
| clarkb | I should maybe go find lunch, but I ate too much at breakfast | 20:38 |
| dmsimard | That's some pretty crazy patch chains for nova in the check pipeline | 20:40 |
| fungi | ianw: i really don't have the faintest clue yet. we saw it with a build-openstack-sphinx-docs build for openstack/tooz on stable/pike in periodic jobs, but also starlinx is hitting it pretty continuously on their sysinv-tox-py27 | 20:40 |
| dmsimard | I'm not going to count them.. but woah | 20:40 |
| clarkb | http://logs.openstack.org/26/634726/6/check/openstackci-beaker/f634030/job-output.txt.gz#_2019-03-14_20_29_25_512023 our qemu-img vhd ppa gpg key isn't being found? | 20:40 |
| fungi | ianw: according to message:"symbol __res_maybe_init version GLIBC_PRIVATE not defined in file libc.so.6" in logstash | 20:41 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Use a for loop to get the key from the hostvars https://review.openstack.org/643453 | 20:41 |
| mordred | fungi: ^^ | 20:41 |
| mordred | fungi: try not to look at it too closely, it might hurt | 20:41 |
| clarkb | I get service unavailable from keyserver.ubuntu.com which probably explains that :/ | 20:41 |
| clarkb | if this persists we may want to bake the pubkey for that gpg key into our puppet and add it that way | 20:42 |
| clarkb | mordred: ^ fyi since you set that ppa up | 20:42 |
| mordred | fungi: the construct worked for me in a local playbook for setting a fact on a host and then ina second play looking it up that way | 20:42 |
| ianw | fungi: hrm, ok, so not a straight up corrupt wheel -- we've still got custom logging in our wheel builder pip if we see issues on that again. i've got an outstanding change to upstream for it that seems to have gone to the place github pull requests go to die | 20:42 |
| mordred | clarkb: ++ | 20:42 |
| fungi | clarkb: yeah, i'm a fan of just sticking literal keys into the job. the only reason to fetch them from a keyserver is when bootstrapping from a key id/fingerprint | 20:42 |
| clarkb | I'm going to take this as a sign to step away from the computer and find lunch | 20:43 |
| clarkb | if anyone wants to write that patch I'll totally review it :0 | 20:43 |
| clarkb | er :) | 20:43 |
| pabelanger | mordred: left comment on 643453 | 20:43 |
| fungi | it's yet one more network connection to break on you, puts unnecessary load on the keyservers, and expands the attack surface area (slightly) | 20:43 |
| *** priteau has quit IRC | 20:43 | |
| ianw | dmsimard: did you have any more thoughts on the bridge.o.o resize? i'd actually like to setup all these changes for git:// -> https:// on it's /opt so we can all see them, but don't want to fiddle on there too much and make things worse | 20:44 |
| *** hamzy has quit IRC | 20:45 | |
| fungi | ianw: clarkb's preparing for doing the resize shortly | 20:45 |
| dmsimard | ianw: I haven't had the chance to spend more cycles on that since we initially looked at it -- I know that clarkb is working on it right now | 20:45 |
| fungi | just testing booting a fresh snapshot first | 20:45 |
| clarkb | ianw: clarkb-test-bridge-snapshot-boot is the server name booting in dfw | 20:45 |
| clarkb | it was still booting last I checked | 20:46 |
| clarkb | feel free to take over that if you like and I can focus on fixing puppet gpg when I get back from lunch | 20:46 |
| ianw | clarkb: oh sorry i really should read scrollback ... that's the backup you're booting and we resize the actual image? | 20:47 |
| mordred | pabelanger: it's the delegate_to that doesn't work | 20:47 |
| fungi | yeah, i'm trying to identify a good place to try setting up a virtualenv on a bionic server to check our our built wheels | 20:47 |
| pabelanger | mordred: Oh, that is because the syntax is wrong | 20:48 |
| pabelanger | mordred: http://git.openstack.org/cgit/openstack/windmill-backup/tree/playbooks/borg-client.yaml#n55 | 20:49 |
| pabelanger | you need to use the apply field for include_role | 20:49 |
| mordred | pabelanger: OH WOW | 20:49 |
| mordred | pabelanger: TIL | 20:49 |
| pabelanger | yah, that is new in 2.7 | 20:50 |
| pabelanger | to fix this issue | 20:50 |
| mordred | we're not running 2.7 | 20:50 |
| mordred | are we? | 20:50 |
| pabelanger | oh | 20:50 |
| mordred | we're on 2.5 right? | 20:50 |
| dmsimard | TIL about apply | 20:50 |
| fungi | for zuul jobs, yes | 20:50 |
| corvus | yep. until someone else reviews tobiash's patches :) | 20:50 |
| pabelanger | yah, zuul... | 20:50 |
| pabelanger | I think there is another way... | 20:50 |
| pabelanger | let me check notes | 20:51 |
| mordred | yeah. so - I think for now we should land the hacky fix there - but let me put in a TODO comment | 20:51 |
| mordred | (or, I'll wait for pabelanger to check his notes - because he's clearly an ansible god) | 20:51 |
| pabelanger | mordred: I think you can wrap it in a block | 20:52 |
| pabelanger | http://paste.openstack.org/show/747814/ | 20:52 |
| ianw | dmsimard: do you want the test-dmsimard-* servers any more? | 20:52 |
| pabelanger | mordred: but yah, note with 2.7 upgrade should also be fine | 20:53 |
| *** helenafm has quit IRC | 20:53 | |
| pabelanger | confirmed: https://github.com/ansible/ansible/issues/44341#issuecomment-414336504 for 2.5.2+ | 20:53 |
| clarkb | ianw: yes if we can make that backup work using your notes then next step is to trogger resize of actual server | 20:53 |
| ianw | clarkb: ok it's active -- want me to do those steps to make it boot? | 20:54 |
| dmsimard | ianw: nope, feel free to delete anything with my name on it | 20:54 |
| clarkb | ianw: sure maybe also disable crontab for root when you are in there to avoid unecessary ansible? | 20:54 |
| dmsimard | ianw: there's probably a snapshot with my name too. | 20:54 |
| ianw | clarkb: will do | 20:54 |
| ianw | i'll add notes on https://etherpad.openstack.org/p/bridge-snapshot-bringup for posterity | 20:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Wrap upload-npm invocation in a block https://review.openstack.org/643454 | 20:55 |
| mordred | pabelanger, fungi ^^ so like that? | 20:55 |
| dmsimard | mriedem: those mass rebases are kind of expensive: https://i.imgur.com/tikEXus.png | 20:56 |
| pabelanger | mordred: +2 | 20:56 |
| mriedem | dmsimard: i don't really have a choice | 20:58 |
| mriedem | i'm not rebasing 44 changes b/c of the fun of it | 20:59 |
| fungi | looks like we don't build a psycopg2 wheel? http://files.openstack.org/mirror/wheel/ubuntu-18.04-x86_64/ | 20:59 |
| clarkb | fungi is it in global reqs? | 20:59 |
| *** rh-jelabarre has quit IRC | 21:00 | |
| *** wolverineav has quit IRC | 21:00 | |
| fungi | yep, https://git.openstack.org/cgit/openstack/requirements/tree/global-requirements.txt#n423 | 21:00 |
| *** wolverineav has joined #openstack-infra | 21:01 | |
| fungi | oh, actually we do have a directory for it, no ide why i didn't see it a moment ago | 21:01 |
| fungi | http://mirror.bhs1.ovh.openstack.org/wheel/ubuntu-16.04-x86_64/psycopg2/ | 21:02 |
| dmsimard | mriedem: expected as much -- no stress. Thought I'd let you know that it has an impact | 21:02 |
| dmsimard | fungi: 16.04 or 18.04 ? | 21:02 |
| fungi | yeah, typo, http://mirror.bhs1.ovh.openstack.org/wheel/ubuntu-18.04-x86_64/psycopg2/ | 21:02 |
| dmsimard | fungi: what the issue with wheels ? | 21:03 |
| *** mattw4 has quit IRC | 21:03 | |
| *** mattw4 has joined #openstack-infra | 21:04 | |
| dmsimard | Had weird issues with wheels trying to stand up virtualenvs with ansible the other day | 21:04 |
| fungi | dmsimard: not sure, just a hunch. trying to get to the bottom of errors like http://logs.openstack.org/39/642639/3/check/sysinv-tox-py27/743d204/job-output.txt#_2019-03-14_18_45_57_616049 | 21:04 |
| dmsimard | Woops, that's nothing like what I've seen -- iirc my issue was pip not attempting to fetch a package if there was no wheel for it | 21:05 |
| fungi | dmsimard: also seen at http://logs.openstack.org/periodic-stable/git.openstack.org/openstack/tooz/stable/pike/build-openstack-sphinx-docs/11fb01e/job-output.txt#_2019-03-14_06_08_40_979055 | 21:06 |
| fungi | unfortunately neither of those jobs collect the relevant tox logs so i can't see what they installed from where | 21:09 |
| *** armax has quit IRC | 21:09 | |
| dmsimard | fungi: pip reports psycopg2==2.7.3 -- it looks like we want 2.7.3.1: http://initd.org/psycopg/articles/2017/08/26/psycopg-2731-released/ | 21:09 |
| dmsimard | claiming to fix a problem with wheels | 21:09 |
| dmsimard | found from upstream issue https://github.com/psycopg/psycopg2-wheels/issues/2 | 21:10 |
| fungi | yeah, the sysinv-tox-py27 log there claims it installed psycopg2==2.7.3 | 21:10 |
| fungi | so maybe they fixed one problem and broke another | 21:10 |
| fungi | oh, it needs pyscopg2==2.7.3.2 ? | 21:12 |
| dmsimard | .1* | 21:13 |
| dtroyer | fungi: stx version requirements tend to be exact due to the way it is packaged, so there are a lot of those leaking out where we would usually see a range or minimum | 21:13 |
| dmsimard | I'm not sure where 2.7.3 comes from to begin with ? Latest version is 2.7.7 and codesearch doesn't seem to turn up something obvious: http://codesearch.openstack.org/?q=psycopg2.*2%5C.7%5C.3&i=nope&files=&repos= | 21:14 |
| fungi | yeah, seeing that at https://pypi.org/project/psycopg2/#history | 21:14 |
| fungi | i suspect the pike failure for tooz was because that's what we set in global reqs on stable/pike (we've since pinned that job to xenial anyway so it won't recur) | 21:14 |
| dtroyer | that also may be connected to the centos 7.6 update we recently merged into master | 21:14 |
| fungi | i'll give the starlingx folks a heads up | 21:14 |
| pabelanger | I think zuul is too busy merging things to launch more jobs, executor queue is climbing, but still at 630+ jobs for mergers | 21:15 |
| pabelanger | with that, I'm going to afk for a bit until things recover | 21:15 |
| dmsimard | pabelanger: there was a 45-patch-long tree rebase in nova | 21:15 |
| pabelanger | dmsimard: yah, looks like on track for over a hour for mergers to catch back up | 21:16 |
| *** trown is now known as trown|outtypewww | 21:16 | |
| pabelanger | which is hindering executor to launch jobs | 21:17 |
| pabelanger | back in a bit | 21:17 |
| fungi | dtroyer: it's almost certainly triggered by the xenial to ubuntu change, since it looks like psycopg2 2.7.3 was built against libs with symbol changes between xenial and ubuntu | 21:17 |
| fungi | er, the xenial to bionic change | 21:17 |
| dtroyer | fungi: likely, yes. I've found a number of 'interesting' things like that working on the devstack plugins for starlingx too, centos75 -> bionic is a big jump in many things it turns out :) | 21:19 |
| dmsimard | Am I mistaken or should the "used ram (%)" graph at http://grafana.openstack.org/d/T6vSHcSik/zuul-status?orgId=1 read "available ram (%)" instead ? | 21:21 |
| dmsimard | Unless they all started to use less ram all of a sudden | 21:22 |
| *** dpawlik has joined #openstack-infra | 21:23 | |
| dmsimard | hrm, nevermind, looks like the graph is correct | 21:23 |
| fungi | dmsimard: we switched it around because graphing available ram was confusing people | 21:23 |
| fungi | so, yeah, that used to be the other way around | 21:24 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Add job and integration tests for the upload-git-mirror role https://review.openstack.org/643435 | 21:26 |
| *** mriedem is now known as mriedem_away | 21:27 | |
| *** dpawlik has quit IRC | 21:27 | |
| *** s0ru has quit IRC | 21:28 | |
| *** whoami-rajat has quit IRC | 21:31 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-diskimage_builder master: Vendor the vhd-util PPA's gpg key https://review.openstack.org/643465 | 21:33 |
| clarkb | fungi: ^ assuming I've puppeted correctly I think that should fix our keyserver sadness | 21:33 |
| clarkb | if I depends on that change it should test it too /me does that now | 21:33 |
| *** dpawlik has joined #openstack-infra | 21:34 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Upgrade some servers to puppet 4 https://review.openstack.org/634726 | 21:34 |
| ianw | clarkb / dmsimard : ok, i rekon 104.130.127.9 is a pretty good facsimile of bridge.o.o. it's cron disabled but otherwise everything looks in the right place to me | 21:34 |
| clarkb | cmurphy: fungi ^ thats just a depends on to verify the gpg stuff but will also get the chagne through | 21:34 |
| clarkb | ianw: cool so we should be good to do a bridge resize and can always fallback to rebuilding off my snapshot | 21:35 |
| fungi | thanks clarkb! catching up on the new changes nwo | 21:35 |
| fungi | now | 21:35 |
| ianw | clarkb: should we do it? stop the snapshot getting too far out? | 21:36 |
| clarkb | do we need to schedule the resize? like maybe quiesce the ansible cron prior to doing that as it may speed things up | 21:36 |
| clarkb | ianw: ya I think it would be great to do it today if we can | 21:36 |
| *** wolverineav has quit IRC | 21:36 | |
| ianw | clarkb: so disable the cron jobs on real bridge.o.o and let it go quiet? | 21:36 |
| *** wolverineav has joined #openstack-infra | 21:37 | |
| fungi | wfm | 21:37 |
| clarkb | ianw: ya then resize is my thinking since a resize is going to be done like a migration (I think) reducing change on teh server will probably make it go quicker | 21:37 |
| dmsimard | resize ends with a reboot as well | 21:38 |
| dmsimard | so probably better to make sure nothing is running prior | 21:38 |
| ianw | huh, it's live? i'd imagined we'd shutdown | 21:38 |
| *** dpawlik has quit IRC | 21:38 | |
| dmsimard | can you actually resize a VM if it's shutdown ? I'm not sure I've ever tried haha | 21:38 |
| clarkb | ya I thought it was more like the suspend -> start trick using acpi than a proper reboot | 21:38 |
| clarkb | if we disable the two ansible crons we should be pretty quiet on that server | 21:39 |
| dmsimard | fwiw the resize test i did the other day was live | 21:39 |
| clarkb | it doesn't run services | 21:39 |
| ianw | dmsimard: oh right, this was via cli right, as they removed the resize button? | 21:40 |
| ianw | one thought is maybe to not make the api call to resize it from the host being resized? | 21:40 |
| clarkb | ianw: heh ya that might be a good idea | 21:40 |
| dmsimard | I did it with the openstack cli | 21:40 |
| dmsimard | ianw: eh good point | 21:41 |
| clarkb | you should be able to do it from the snapshot host since it has the clouds.yaml for it | 21:41 |
| ianw | ++ sounds good. i'll disable cron on real bridge and queue up commands for review | 21:42 |
| * ianw colors a terminal red and the other green to avoid confusing the real and snapshot server ... | 21:43 | |
| ianw | made that mistake with this sort of thing before :/ | 21:43 |
| dmsimard | ianw: haven't we all ? it's called experience :p | 21:47 |
| fungi | clarkb: there's an even easier way than 643465 (apt-key add is basically deprecated in favor of just sticking a file in /etc/apt/trusted.gpg.d/ instead) | 21:47 |
| clarkb | fungi: so I looked in there and the files are not ascii armored or whatever that format is | 21:47 |
| fungi | you can keep it in ascii-armored format as long as you put a .asc file extension on it | 21:47 |
| clarkb | fungi: will it accept that format? | 21:47 |
| clarkb | ah | 21:47 |
| fungi | though... it's possible the apt-key manpage i'm looking at is too new for xenial | 21:48 |
| fungi | if you had somewhere you were testing, it's easy to double-check i guess | 21:49 |
| clarkb | "File fragments for the trusted keys, additional keyrings can be stored here (by other packages or the administrator). Configuration Item Dir::Etc::TrustedParts." | 21:49 |
| clarkb | says xenial apt-key manpage | 21:49 |
| clarkb | unfortunately no mention of ascii armor | 21:49 |
| clarkb | (And I'd like to avoid sticking binary files in puppet) | 21:49 |
| clarkb | (much easier for people to confirm the content is what tehy want and not nefarious if ascii) | 21:49 |
| fungi | yeah, not sure if the manpage was just improved later, or if apt itself was improved to support ascii keyrings | 21:49 |
| fungi | apt (1.4~beta1) unstable; urgency=medium | 21:50 |
| fungi | * add apt-key support for armored GPG key files (*.asc) | 21:50 |
| clarkb | xenial is 1.2.29 | 21:51 |
| fungi | looks like xenial's carrying 1.2.29ubuntu0.1 | 21:51 |
| fungi | yeah | 21:51 |
| fungi | so not until bionic i guess! | 21:51 |
| fungi | oh well | 21:51 |
| fungi | something to look forward to ;) | 21:51 |
| ianw | dmsimard / clarkb: ok, proposed resize commands on etherpad. have a root screen running on 104.130.127.9 | 21:54 |
| ianw | bridge.o.o doesn't look to be doing much, so i think we can proceed when we like | 21:57 |
| clarkb | ianw: uuid and target flavor lgtm. Whats the two step process for a resize? you queue it then confirm? | 21:58 |
| dmsimard | you queue, wait N minutes (where N can be long) and confirm when the instance boots back up | 21:59 |
| ianw | yeah, that's my understanding; dmsimard those commands are what you used on your test? | 22:00 |
| clarkb | ah | 22:00 |
| dmsimard | ianw: yeah lgtm | 22:00 |
| fungi | AH00526: Syntax error on line 57 of /etc/apache2/sites-enabled/50-wiki-dev.openstack.org.conf: Invalid command 'php_admin_flag', perhaps misspelled or defined by a module not included in the server configuration | 22:03 |
| fungi | i wonder what that changes to in xenial | 22:03 |
| fungi | the job fixes for the requirements repo master branch are finally into the gate | 22:04 |
| fungi | aha! | 22:07 |
| fungi | (/Stage[main]/Mediawiki/Package[libapache2-mod-php5]/ensure) E: Package 'libapache2-mod-php5' has no installation candidate | 22:07 |
| ianw | ok, bridge.o.o is quiet ... i'm going to do this in the screen on alternate bridge | 22:07 |
| ianw | bridge.openstack.org | RESIZE | 22:08 |
| ianw | looks promising | 22:08 |
| *** dpawlik has joined #openstack-infra | 22:09 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-mediawiki master: Switch from PHP5 to default PHP (PHP7 on Xenial) https://review.openstack.org/643476 | 22:12 |
| fungi | infra-puppet-core: ^ next step on xenial for wiki-dev | 22:12 |
| *** dpawlik has quit IRC | 22:13 | |
| *** pcaruana has quit IRC | 22:21 | |
| *** rlandy is now known as rlandy|bbl | 22:22 | |
| clarkb | ianw: I can ssh into bridge again | 22:24 |
| clarkb | has more cpus and 8GB memory | 22:25 |
| ianw | yep, looks right | 22:25 |
| ianw | /opt is there, not grown but i don't think that matters | 22:25 |
| ianw | i'll re-enable cron before i verify it? | 22:26 |
| clarkb | ianw: ++ | 22:26 |
| ianw | alright, re-enabled, will watch a run and make sure it's happy | 22:26 |
| clarkb | once we've verified I'll restart my gitea fixup playbook | 22:27 |
| *** e0ne has quit IRC | 22:28 | |
| fungi | and soon we can talk about moving its ara db out of trove | 22:28 |
| *** wolverineav has quit IRC | 22:31 | |
| ianw | flock: cannot open lock file /var/run/ansible/run_all.lock: No such file or directory ... hrm this might be a reboot issue for bridge.o.o | 22:31 |
| clarkb | is /var/run ephemeral? | 22:31 |
| fungi | i know /run is for sure, /var/run may be depending on how it's set up | 22:32 |
| ianw | /var/run -> /run | 22:32 |
| fungi | well there you have it | 22:32 |
| ianw | i think it's a systemd-ism | 22:32 |
| fungi | and /run is mounted on a tmpfs | 22:33 |
| fungi | so we need to recreate /var/run/ansible anytime we expect to write into it | 22:33 |
| ianw | i guess cron job could do "if [ ! -e /var/run/ansible/run_all.lock ]; then mkdir -p /var/run/ansible; touch /var/run/ansible.lock; fi ... rest of command" | 22:34 |
| ianw | oh it's probably just the directory it needs | 22:34 |
| clarkb | should be safe to always mkdir -p if we want to brute force it | 22:35 |
| fungi | yup | 22:36 |
| *** wolverineav has joined #openstack-infra | 22:36 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Ensure lockfile dir is created for bridge.o.o https://review.openstack.org/643480 | 22:37 |
| *** gmann_afk is now known as gmann | 22:40 | |
| clarkb | ianw: we'll have to manually correct that in the cron now though right? | 22:42 |
| clarkb | or run the playbook by hand? | 22:42 |
| ianw | clarkb: i just made the directory for now, waiting for next run in a few mins | 22:42 |
| openstackgerrit | Merged openstack-infra/project-config master: Wrap upload-npm invocation in a block https://review.openstack.org/643454 | 22:43 |
| fungi | time to try reenqueuing a tag again | 22:45 |
| openstackgerrit | Ankita Bansal proposed openstack-infra/storyboard-webclient master: Provide -Assign to self- button for story tasks. https://review.openstack.org/643427 | 22:46 |
| ianw | python3: can't open file 'tools/cloud-to-env.py': [Errno 2] No such file or directory | 22:46 |
| openstackgerrit | Clark Boylan proposed openstack-infra/openstack-zuul-jobs master: Run legacy-puppet-lint on Xenial https://review.openstack.org/643481 | 22:48 |
| clarkb | fungi: ^ yet another bionic transition fix | 22:48 |
| clarkb | ianw: what is the context for that? | 22:48 |
| ianw | i dunno, it just popped out in the run_all_logs ... i don't think it's related | 22:49 |
| *** gfidente has quit IRC | 22:50 | |
| clarkb | seems like something mordred might knwo about if it is converting clouds.yaml to env vars | 22:51 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-diskimage_builder master: Vendor the vhd-util PPA's gpg key https://review.openstack.org/643465 | 22:52 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Change to parent dir in run_k8s_ansible.sh https://review.openstack.org/643482 | 22:53 |
| clarkb | ianw: http://git.openstack.org/cgit/openstack-infra/system-config/tree/run_k8s_ansible.sh oh good you found it | 22:53 |
| ianw | clarkb / modred: ^ just a path issue i think | 22:53 |
| ianw | clarkb: are we ok with retaining the current 2gb swap? we could do something like grow /opt to 6gb short of the disk, then add more swap at the end, but doesn't seem worth it | 22:57 |
| ianw | i think we'd be better just growing out /opt to fill the disk and leaving it at that | 22:58 |
| clarkb | xvde1 is swap right? then xvde2 is the /opt partition? | 22:58 |
| clarkb | (this might be a reason to reverse the order of these partitions) | 22:58 |
| clarkb | in the future I mean | 22:58 |
| clarkb | ianw some swap is better than no swap. I suppose if we run into problems with only 2GB swap we can always repartition later | 22:58 |
| clarkb | so ya probably fine as is for now | 22:58 |
| fungi | convenience being we partition a fixed amount of swap and can allocate "whatever's left" to /opt | 22:59 |
| clarkb | fungi: ya less math to do | 22:59 |
| *** rcernin has joined #openstack-infra | 23:00 | |
| clarkb | ianw have a moment to review https://review.openstack.org/#/c/643481/ ? fixes puppet lint job after bionic default node change | 23:01 |
| clarkb | ianw: also let me know when you think it is safe for me to run my gitea fixup playbook | 23:01 |
| *** tkajinam has joined #openstack-infra | 23:02 | |
| ianw | clarkb: i think everything looks like it's working, let's do the verify step? i guess that really just means it deletes the rollback | 23:03 |
| *** wolverineav has quit IRC | 23:04 | |
| clarkb | if you're happy with it the things I've checked look good to me | 23:04 |
| clarkb | and we have my snapshot from today if we need it | 23:04 |
| ianw | ok, done, that was quick | 23:06 |
| tbarron | hmmmm, https://bugs.launchpad.net/manila/+bug/1820118 | 23:06 |
| openstack | Launchpad bug 1820118 in Manila "Order Tramadol Online to Cure Psoriatic Arthritis" [Undecided,New] | 23:06 |
| tbarron | note that the original bug report is legitimate ^^^ | 23:06 |
| clarkb | yay they are defacing bug titles now? | 23:06 |
| clarkb | ianw: cool, good to run the gitea fixup then? | 23:07 |
| tbarron | clarkb: and descriptions | 23:07 |
| ianw | clarkb: yep, i think we can declare bridge.o.o resized :) | 23:07 |
| ianw | #status log bridge.o.o resized to a 8gb instance | 23:07 |
| openstackstatus | ianw: finished logging | 23:07 |
| tbarron | priyanka is not an active member of any Launchpad teams (the updater) | 23:07 |
| clarkb | ok started running it in a root screen | 23:07 |
| tbarron | Email: | 23:08 |
| tbarron | No public address provided. | 23:08 |
| ianw | clarkb: i will shutdown your snapshot now to avoid any confusion. can remove later | 23:08 |
| clarkb | ianw: ++ | 23:08 |
| tbarron | just thought this may be the place to mention it but see you folks are dealing with an active situation :) | 23:08 |
| *** wolverineav has joined #openstack-infra | 23:08 | |
| corvus | tbarron: if no one has already, it would be good to report this to the launchpad admins. there's the #launchpad channel and https://bugs.launchpad.net/launchpad | 23:08 |
| tbarron | maybe the answer is: switch to storyboard :) | 23:09 |
| tbarron | corvus: thanks, will do | 23:11 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Run legacy-puppet-lint on Xenial https://review.openstack.org/643481 | 23:11 |
| *** wolverineav has quit IRC | 23:11 | |
| gouthamr | tbarron: https://answers.launchpad.net/launchpad has a lot of spam reported | 23:12 |
| tbarron | gouthamr: ty | 23:12 |
| *** wolverineav has joined #openstack-infra | 23:12 | |
| ianw | clarkb: ok, i grew /opt but left about 10gb at the end of the disk; if we want to add more swap, can do it there | 23:18 |
| fungi | mordred: i *think* this worked? http://logs.openstack.org/b5/b5dbb90fb347db2876f61cae8c2623f9ce767dfe/release/release-openstack-javascript/0312156/ara-report/result/44e53614-4975-4430-8b19-04643744f65c/ | 23:18 |
| clarkb | ianw: sounds good, thanks | 23:18 |
| mordred | fungi: yes - it looks like it did | 23:19 |
| fungi | huzzah! | 23:19 |
| *** lseki has quit IRC | 23:19 | |
| fungi | one more we can scratch off the list then | 23:19 |
| mordred | fungi: http://tarballs.openstack.org/tripleo-ui/tripleo-ui-10.3.0.tar.gz exists | 23:20 |
| fungi | mordred: oh, also infra-root received e-mail from npm notifying it of the uploaded release | 23:21 |
| fungi | so we're good | 23:21 |
| mordred | sweet | 23:21 |
| *** mriedem_away has quit IRC | 23:23 | |
| *** slaweq has quit IRC | 23:25 | |
| *** tosky has quit IRC | 23:26 | |
| *** gyee has quit IRC | 23:34 | |
| clarkb | ok I've got my vhd-util gpg key puppet fix rechecked as well as cmurphy's puppet-4 switcheroo back into check | 23:37 |
| clarkb | https://review.openstack.org/643465 and https://review.openstack.org/#/c/634726/7 if people can review them again. For the second one maybe don't approve it since that should be watched (I can do that tomorrow morning) | 23:37 |
| *** Lucas_Gray has quit IRC | 23:39 | |
| *** wolverineav has quit IRC | 23:41 | |
| clarkb | we are into the charm repos with gitea fixup | 23:41 |
| clarkb | been running for about half an hour. THis is much quicker than when we had only 2GB of memory | 23:41 |
| *** wolverineav has joined #openstack-infra | 23:42 | |
| *** wolverineav has quit IRC | 23:46 | |
| *** wolverineav has joined #openstack-infra | 23:49 | |
| *** wolverineav has quit IRC | 23:53 | |
| *** wolverineav has joined #openstack-infra | 23:55 | |
| *** wolverineav has quit IRC | 23:56 | |
| *** wolverineav has joined #openstack-infra | 23:56 | |
| *** sthussey has quit IRC | 23:57 | |
| *** wolverineav has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
