| *** agopi|out has quit IRC | 00:00 | |
| *** rascasoft has joined #oooq | 00:09 | |
| *** weshay|ruck has quit IRC | 00:11 | |
| *** rlandy is now known as rlandy|bbl | 00:13 | |
| *** rascasoft has quit IRC | 00:16 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (1 more message) | 00:56 |
|---|---|---|
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7 (1 more message) | 02:56 |
| *** rlandy|bbl is now known as rlandy | 03:10 | |
| *** rlandy has quit IRC | 03:13 | |
| *** ykarel|away has joined #oooq | 03:40 | |
| *** agopi|out has joined #oooq | 03:40 | |
| *** ykarel|away is now known as ykarel | 03:48 | |
| *** skramaja has joined #oooq | 03:54 | |
| *** skramaja_ has joined #oooq | 03:58 | |
| *** skramaja has quit IRC | 03:59 | |
| *** udesale has joined #oooq | 04:36 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 04:56 |
| *** ykarel has quit IRC | 04:57 | |
| *** ratailor has joined #oooq | 05:09 | |
| *** ykarel has joined #oooq | 05:11 | |
| *** saneax has joined #oooq | 05:30 | |
| *** saneax has quit IRC | 06:38 | |
| *** ccamacho has quit IRC | 06:41 | |
| *** jfrancoa has joined #oooq | 06:42 | |
| *** quiquell|off is now known as quiquell|rover | 06:47 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 06:56 |
| *** quiquell|rover is now known as quique|rover|brb | 07:31 | |
| *** jtomasek has joined #oooq | 07:34 | |
| *** apetrich has joined #oooq | 07:35 | |
| *** jtomasek has quit IRC | 07:39 | |
| *** ykarel is now known as ykarel|lunch | 07:42 | |
| *** jtomasek has joined #oooq | 07:44 | |
| *** saneax has joined #oooq | 08:06 | |
| *** gkadam has joined #oooq | 08:10 | |
| *** gkadam is now known as gkadam-brb | 08:11 | |
| *** quique|rover|brb is now known as quiquell|rover | 08:15 | |
| *** gkadam-brb is now known as gkadam | 08:20 | |
| *** ccamacho has joined #oooq | 08:22 | |
| *** ykarel|lunch is now known as ykarel | 08:29 | |
| *** jpena|off is now known as jpena | 08:38 | |
| *** amoralej|off is now known as amoralej | 08:39 | |
| *** tosky has joined #oooq | 08:47 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 08:56 |
| *** kopecmartin|off is now known as kopecmartin | 08:57 | |
| *** panda|off is now known as panda | 09:00 | |
| *** bogdando has joined #oooq | 09:00 | |
| *** holser_ has joined #oooq | 09:15 | |
| *** holser_ has quit IRC | 09:16 | |
| *** holser_ has joined #oooq | 09:17 | |
| zbr | oooq: newer ansible-lint has "shell task uses pipeline without pipefail" rule embedded, which means that once we start using it we can scrap our own custom implementation. | 09:19 |
| *** dtantsur|afk is now known as dtantsur | 09:20 | |
| *** derekh has joined #oooq | 09:37 | |
| zbr | rfolco: i am still working on the refactoring of these two roles, lots of changes but in less than an hour I should have it pushed. i think I will not override your change and create a new one instead due to the huge differences. | 09:42 |
| *** gkadam has quit IRC | 09:51 | |
| *** gkadam has joined #oooq | 09:51 | |
| zbr | rfolco: quiquell|rover : please check https://review.openstack.org/#/c/636905/ and give it a local try. | 10:27 |
| *** jbadiapa has quit IRC | 10:42 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci- (5 more messages) | 10:56 |
| quiquell|rover | zbr: ack will look | 10:57 |
| *** udesale has quit IRC | 10:58 | |
| panda | marios: can you put the tasks in ready for review ? | 11:01 |
| marios | panda: for the container build? | 11:03 |
| marios | panda: (yes but not yet i am tidying things up for the merge - remove WIP etc) | 11:03 |
| panda | marios: ok | 11:04 |
| marios | panda: the job is ok now but failing for the dpdk thing | 11:04 |
| quiquell|rover | marios: add a Depends-On to https://review.openstack.org/#/c/636860/ | 11:13 |
| quiquell|rover | marios: while new ovs is build | 11:14 |
| marios | quiquell|rover: ack i thought about it /saw your mail but isn't urgent/will update it and we can recheck later thanks | 11:21 |
| marios | quiquell|rover: i have what i need from the ci for now (i.e. i was just lokoing for this file to generate OK http://logs.openstack.org/22/635222/20/check/tripleo-build-containers-centos-7/64b166c/conf/kolla-build.conf.txt.gz | 11:21 |
| quiquell|rover | zbr: commented https://review.openstack.org/#/c/636905 | 11:25 |
| quiquell|rover | marios: ack, ovs will be ready 30min or so I am looking at the microwave rigth now | 11:25 |
| marios | quiquell|rover: ack enjoy ;) | 11:26 |
| zbr | quiquell|rover: haha, non relative link splipped in. Anyway, to move things faster I did split that change in two, and first step is at: https://review.openstack.org/#/c/636923/ | 11:27 |
| zbr | quiquell|rover: meanwhile I will implement your comments on the old one rebase it on top of the partial one. | 11:28 |
| quiquell|rover | zbr: changes are huge man, btw maybe you can use bindep to express packages dependencies | 11:31 |
| zbr | quiquell|rover: using bindep is not so easy because its upstream role is partially broken. i plan to fix that but until then, we cannot just use it without a lot of boilerplate before. | 11:37 |
| quiquell|rover | well we don't need the upstream role to install bindep | 11:38 |
| zbr | quiquell|rover: the main issue is that upstream made some assumptions about base-os and they use altered images: with pip and virtualenv preinstalled. this is against testing on pure-os images. that's the main issue causing lots of failures when testing roles outside the upstream doctored images. | 11:39 |
| quiquell|rover | zbr: and this affects bindep ? | 11:39 |
| zbr | quiquell|rover: obviously, there is no rpm for it and installing it from pip requires pip, for which there is no official pip rpm on centos7 (fedora28 is ok on this aspect). | 11:41 |
| zbr | 90% of problems I encoutered were caused about ensuring that the whole-trinity is available: pip/virtualenv/setuptools | 11:42 |
| quiquell|rover | zbr: ahh there is rpm for bindep | 11:42 |
| quiquell|rover | there is NOT i mean | 11:42 |
| quiquell|rover | zbr: there is a RPM for suse :-= https://build.opensuse.org/package/view_file/Cloud:OpenStack:Master/python-bindep/python-bindep.spec?expand=1 | 11:45 |
| quiquell|rover | Maybe we can put one for centos/fedora | 11:45 |
| zbr | quiquell|rover: sorry but I am not in packaging..., but jschlueter could help. Even if we would have a rpm published in EPEL we would not be officially qallowed to use it (remember that we would need one in centos7/fedora28/rhel8/centos8...). We didn't get a pip package in centos.... Building an rpm is not hard but publishing and keeping it out of conflicts is not an easy task. | 11:50 |
| quiquell|rover | Have to drop I will be reading this | 11:53 |
| *** quiquell|rover is now known as quique|rover|r-- | 11:53 | |
| *** jpena is now known as jpena|lunch | 11:58 | |
| *** amoralej is now known as amoralej|lunch | 11:59 | |
| *** ratailor has quit IRC | 12:06 | |
| jschlueter | zbr, quique|rover|r--: amoralej|lunch or others in rdo packaging would be able to help | 12:21 |
| *** skramaja has joined #oooq | 12:22 | |
| *** skramaja_ has quit IRC | 12:23 | |
| jschlueter | but interesting if bindep is being used more in packages ... we might want to talk with amoralej|lunch and jruzicka (rdopkg maintainer) about using that info more for keeping rpm .spec file in sync with the project deps if it isn't already | 12:23 |
| quique|rover|r-- | jschlueter: ack thanks, so it's doable | 12:33 |
| quique|rover|r-- | Changing build-test-packages is delicate task | 12:34 |
| jschlueter | if it's useful package/tool for keeping things working it is worth looking into | 12:34 |
| jschlueter | just need to make sure we don't get more false positive issues coming out of it than actual things it fixes | 12:35 |
| quique|rover|r-- | Ack | 12:35 |
| *** udesale has joined #oooq | 12:45 | |
| zbr | quique|rover|r--: rfolco panda: simple fix of my own mistake https://review.openstack.org/#/c/636944/2 | 12:47 |
| quique|rover|r-- | marios commented https://review.openstack.org/#/c/635222 | 12:49 |
| marios | quique|rover|r--: thanks | 12:49 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci- (5 more messages) | 12:56 |
| *** vinaykns has joined #oooq | 12:58 | |
| panda | zbr: commented | 13:00 |
| zbr | panda: thanks, replied. shortly "UX", the warning was looking more like a code issue than a warning from a cli tool. | 13:06 |
| rfolco | panda, do you see any issues with my bindep file ? https://review.openstack.org/#/c/636160/10/bindep.txt | 13:10 |
| *** skramaja_ has joined #oooq | 13:11 | |
| *** skramaja has quit IRC | 13:12 | |
| *** amoralej|lunch is now known as amoralej | 13:13 | |
| *** trown|outtypewww is now known as trown | 13:16 | |
| *** jbadiapa has joined #oooq | 13:17 | |
| panda | rfolco: no. Is it not working ? | 13:21 |
| rfolco | panda, nope http://logs.openstack.org/88/636188/12/check/tripleo-build-containers-fedora-28/bce1032/job-output.txt.gz#_2019-02-14_13_03_33_722876 | 13:21 |
| panda | also, does bindep support platform versioning ? because centos8 will need python3 | 13:21 |
| rfolco | panda, see above bindep for tripleoclient works | 13:22 |
| rfolco | https://review.openstack.org/#/c/636160/11/playbooks/tripleo-buildcontainers/run.yaml | 13:22 |
| rfolco | panda, yes, I think so... like fedora-28 | 13:23 |
| quique|rover|r-- | marios: nit https://review.rdoproject.org/r/#/c/18729 | 13:24 |
| *** jpena|lunch is now known as jpena | 13:24 | |
| quique|rover|r-- | panda: yep you can add version there | 13:24 |
| marios | quique|rover|r--: thanks | 13:24 |
| panda | rfolco: where is that task "Get list of packages" defined ? | 13:26 |
| rfolco | in bindep role from zuul-jobs | 13:26 |
| rfolco | above you see tripleoclient bindep just worked ... | 13:27 |
| rfolco | http://logs.openstack.org/88/636188/12/check/tripleo-build-containers-fedora-28/bce1032/job-output.txt.gz#_2019-02-14_13_03_18_719044 | 13:27 |
| rfolco | like 2019-02-14 13:03:20.438476 | primary | libffi-devel | 13:27 |
| *** agopi|out is now known as agopi|brb | 13:28 | |
| panda | rfolco: if that is skippe it means that all the packages are already there ... | 13:30 |
| zbr | @oooq: out of curiosity, do we really need to display the *huge* output of sudo yum-config-manager --save to build console? | 13:30 |
| rfolco | panda, openstack command not found -- http://logs.openstack.org/88/636188/12/check/tripleo-build-containers-fedora-28/bce1032/logs/build-err.log.txt.gz | 13:30 |
| zbr | ... got muscle soreness from scrolling through it ;) | 13:30 |
| rfolco | panda, I am forcing an install with package ansible module in my latest ps... will watch | 13:32 |
| rfolco | panda, thx anyway | 13:32 |
| *** agopi|brb has quit IRC | 13:32 | |
| *** weshay has joined #oooq | 13:34 | |
| panda | rfolco: maybe I see the problem | 13:40 |
| panda | rfolco: roles/bindep/tasks/main.yaml: bindep_run: "{{ bindep_command }} -b -f {{ bindep_file }} {{ bindep_profile }}" | 13:40 |
| panda | this is how bindep is run | 13:40 |
| panda | rfolco: by default bindep_profile is compile doc | 13:40 |
| rfolco | need profile ? | 13:40 |
| *** rlandy has joined #oooq | 13:41 | |
| rfolco | why tripleoclient with profile test works ? https://github.com/openstack/python-tripleoclient/blob/master/bindep.txt | 13:41 |
| weshay | rfolco, because the test machines are ubunutu | 13:43 |
| rfolco | no but it works in my job (f28) | 13:44 |
| panda | rfolco: oh yeah sorry, the default is compile test | 13:45 |
| panda | rfolco: I was lookign at the wrong default | 13:45 |
| panda | rfolco: in zuul-jobs/roles/bindep/defaults/main.yaml: bindep_profile: compile test | 13:45 |
| panda | rfolco: so you can either add "test" as profile | 13:46 |
| zbr | who is ruck? i am curious about the broken delorean-master-deps repo.... | 13:46 |
| panda | zbr: it's all in the past | 13:46 |
| panda | rfolco: or you can override bindep_profile: "" | 13:47 |
| *** weshay is now known as weshay|ruck | 13:47 | |
| rfolco | hmm | 13:47 |
| weshay|ruck | quique|rover|r--, howdy | 13:47 |
| rfolco | panda, https://github.com/openstack/python-tripleoclient/blob/master/bindep.txt#L7 | 13:47 |
| zbr | panda: got failure from like 20min ago, should I just recheck again? | 13:47 |
| rfolco | libffi-devel [platform:rpm] | 13:47 |
| *** jaosorior has quit IRC | 13:47 | |
| quique|rover|r-- | weshay|ruck: CI should be back again | 13:47 |
| rfolco | libffi-devel [platform:rpm] --> no test profile and it got installed here http://logs.openstack.org/88/636188/12/check/tripleo-build-containers-fedora-28/bce1032/job-output.txt.gz#_2019-02-14_13_03_20_438476 | 13:48 |
| weshay|ruck | yay! packages | 13:48 |
| quique|rover|r-- | weshay|ruck: let's monitor noop reviews | 13:48 |
| quique|rover|r-- | weshay|ruck: yep dpdk broke us | 13:48 |
| weshay|ruck | rfolco, panda why are you guys messing around w/ python-tripleoclient? | 13:48 |
| weshay|ruck | aye | 13:48 |
| rfolco | weshay|ruck, I am just getting help on bindep coz tripleoclient's bindep works and has no profiles... but tripleo-ci bindep (new) doesn't | 13:49 |
| weshay|ruck | rfolco, we def.. want profiles | 13:49 |
| zbr | rfolco: why not using bindep role from upstream? | 13:50 |
| weshay|ruck | zbr, that's called I think in any job | 13:50 |
| rfolco | the thing is libffi-devel [platform:rpm] has no profile and it got installed in tripleoclient bindep | 13:50 |
| rfolco | I imported upstream role zbr | 13:50 |
| rfolco | but it is ignoring my packages | 13:50 |
| weshay|ruck | rfolco, can I see ur change to tripleo-ci | 13:51 |
| zbr | rfolco: uhh, time to fix it. you scared me. i know bindep upstream has buggy but we can fix it, nothing major. i just wanted to be sure we do not.... fork the internet. | 13:51 |
| *** jaosorior has joined #oooq | 13:51 | |
| rfolco | weshay|ruck, https://review.openstack.org/#/c/636160/11/playbooks/tripleo-buildcontainers/run.yaml | 13:52 |
| weshay|ruck | rfolco, this wont work python-tripleoclient [platform:centos] | 13:52 |
| weshay|ruck | neither will python3-tripleoclient [platform:fedora] | 13:53 |
| weshay|ruck | not at the time bindep runs | 13:53 |
| *** quique|rover|r-- is now known as quiquell|rover | 13:53 | |
| rfolco | why weshay|ruck | 13:53 |
| weshay|ruck | rfolco, because that is provided by dlrn | 13:53 |
| weshay|ruck | not the standard upstream centos repos | 13:53 |
| rfolco | weshay|ruck, hmmm tripleo-repos ran before, so I thought the dlrn repos were there.... unless bindep installs only pure distro packages | 13:57 |
| sshnaidm | rfolco, do we have a mtg? | 13:59 |
| *** agopi|brb has joined #oooq | 14:00 | |
| rfolco | sshnaidm, yes, panda can you or weshay|ruck cover me | 14:01 |
| rfolco | need to drop | 14:01 |
| *** rfolco is now known as rfolco|off | 14:01 | |
| weshay|ruck | zbr, mtg | 14:03 |
| *** agopi_ has joined #oooq | 14:04 | |
| *** agopi|brb has quit IRC | 14:06 | |
| amoralej | jschlueter, we don't use bindeps to mantain .spec files | 14:17 |
| amoralej | or for anything else, tbh | 14:17 |
| amoralej | i'd say bindeps is not a very good authoritative source of info | 14:17 |
| amoralej | very devstack jobs oriented | 14:18 |
| *** agopi_ is now known as agopi | 14:18 | |
| *** ykarel is now known as ykarel|away | 14:18 | |
| marios | weshay|ruck: panda rfolco|off using "ci-squad-testing" ok? (push is also false for now - i'd rather merge first and then iterate to try a push even for the test tag) https://review.rdoproject.org/r/#/c/18729/11/zuul.d/jobs.yaml@499 | 14:19 |
| *** ykarel|away has quit IRC | 14:23 | |
| marios | http://logs.openstack.org/63/636563/1/check/tripleo-ci-centos-7-scenario001-standalone/77189ef/logs/undercloud/var/log/containers/mysql/mariadb.log.txt.gz | 14:34 |
| *** ykarel|away has joined #oooq | 14:40 | |
| *** ykarel|away is now known as ykarel | 14:45 | |
| *** skramaja_ has quit IRC | 14:46 | |
| zbr | weshay|ruck: idea about 3rd party voting: using a normal user and adding -1 vote. it could make people more careful. | 14:49 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci- (5 more messages) | 14:56 |
| *** bogdando has quit IRC | 14:57 | |
| weshay|ruck | zbr, right.. I agree .. the idea here is to explore options and talk about them w/ our tripleo community | 15:02 |
| weshay|ruck | sshnaidm, ^ | 15:02 |
| weshay|ruck | zbr, quick break and then we'll 1-1 | 15:02 |
| rlandy | marios: I'm am still going to try the pacemaker stuff - just to see where I land up | 15:04 |
| rlandy | for comparison | 15:04 |
| panda | rlandy: mmhh, then I should maintain the us on the sprint ? | 15:04 |
| rlandy | panda: up to you | 15:04 |
| rlandy | panda: I may get nowhere | 15:05 |
| rlandy | but I'd like to know that there | 15:05 |
| rlandy | is nothing else hiding behind pacemaker | 15:05 |
| rlandy | I think it's worth the investigation | 15:05 |
| weshay|ruck | zbr, k.. 1-1 | 15:06 |
| weshay|ruck | sshnaidm, do you have 5min to disucss third party voting in about 20min? | 15:07 |
| sshnaidm | weshay|ruck, yep | 15:13 |
| marios | rlandy: ack | 15:14 |
| *** udesale has quit IRC | 15:22 | |
| *** saneax has quit IRC | 15:24 | |
| *** gkadam has quit IRC | 15:30 | |
| weshay|ruck | sshnaidm, ok.. ready when you are in my blue.. topic is 3rd party voting | 15:30 |
| weshay|ruck | sshnaidm, zbr https://docs.google.com/document/d/1vDi386BaK9n_j1krMFjdcQPzfN2nnLuV-HD4X8SdpKY/edit | 16:01 |
| weshay|ruck | kopecmartin++ | 16:06 |
| hubbot1 | weshay|ruck: kopecmartin's karma is now 6 | 16:06 |
| weshay|ruck | arxcruz++ | 16:06 |
| hubbot1 | weshay|ruck: arxcruz's karma is now 12 | 16:06 |
| weshay|ruck | sshnaidm, how do I get on the ovb nodes from the reproducer? | 16:08 |
| weshay|ruck | sshnaidm, note.. I'm just running recheck from the local gerrit | 16:10 |
| *** bogdando has joined #oooq | 16:11 | |
| weshay|ruck | so my local inventory is not up to date | 16:11 |
| * weshay|ruck keeps hitting https://1ce.org/1#rJyB7f7HV | 16:11 | |
| weshay|ruck | panda, marios the f28 check job for building containers.. seems at risk to me.. let me know if we need to chat about that | 16:16 |
| weshay|ruck | zbr, please help w/ ^ | 16:16 |
| *** dtantsur is now known as dtantsur|afk | 16:16 | |
| zbr | weshay|ruck: panda marios : re f28 please help me review/merge: https://review.openstack.org/#/c/636944/ or https://review.openstack.org/#/c/636969/ | 16:21 |
| zbr | any of those two would unblock tasks | 16:22 |
| marios | zbr: ack noted | 16:23 |
| marios | weshay|ruck: maybe we can have a call tomorrow at usual scrum time to assess. if rfolco|off is around then too? | 16:24 |
| weshay|ruck | k | 16:24 |
| panda | marios: ack | 16:26 |
| weshay|ruck | zbr, is https://review.openstack.org/#/c/636944/2/tripleo_repos/main.py critical path? | 16:27 |
| weshay|ruck | or an enhancement? | 16:27 |
| zbr | weshay|ruck: depends how you see it: merging one of the two is on critical-path. there are not both mandatory: only one, any of them would be enough. | 16:28 |
| weshay|ruck | zbr, it would be smart to have that patch depend on a patch that kicks the f28 containers build job under review right? | 16:31 |
| weshay|ruck | zbr, atm.. I can only see the centos-7 run | 16:31 |
| zbr | weshay|ruck: this is what i am doing now. | 16:31 |
| weshay|ruck | zbr, k | 16:31 |
| weshay|ruck | patch looks fine, but we need f28 job before voting | 16:32 |
| panda | marios: still working on https://review.openstack.org/635222 ? | 16:37 |
| panda | marios: we should merge this before merging the job or it's indifferent ? | 16:37 |
| weshay|ruck | if someone has 5 min.. I have no idea why | 16:38 |
| weshay|ruck | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/templates/undercloud.conf.j2#L152 | 16:38 |
| weshay|ruck | # Network that will be masqueraded for external access, if required. | 16:39 |
| weshay|ruck | # This should be the subnet used for PXE booting. (string value) | 16:39 |
| weshay|ruck | masquerade_network = 192.168.24.0/24 | 16:39 |
| weshay|ruck | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-8483/undercloud/home/stack/undercloud.conf.gz | 16:39 |
| weshay|ruck | https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-master-delorean-quick-basic-8483/undercloud/home/stack/undercloud_install.log.gz | 16:39 |
| quiquell|rover | panda, marios: Can give +2 there if this helps my comment is just nit | 16:39 |
| marios | panda: well yeah we should merge the https://review.openstack.org/635222 first really otherwise the vars passed from the job have nowhere to go | 16:39 |
| marios | quiquell|rover: replied to your comment... re the defaults let me add comment now (had some issues with those cos it isn't a role) | 16:40 |
| quiquell|rover | marios: And I don't want to tell you to change it into a role, I want a beer at PTG :-P | 16:40 |
| marios | quiquell|rover: i mean yeah we could but no in scope for this :) | 16:41 |
| quiquell|rover | marios: +2 <- panda let's workflow it ? | 16:41 |
| quiquell|rover | marios: yep totally not | 16:41 |
| quiquell|rover | let's workflow that | 16:41 |
| quiquell|rover | well after recheck | 16:41 |
| panda | quiquell|rover: yes, then we can blame marios in case everything explodes | 16:41 |
| marios | panda: \o/ | 16:41 |
| quiquell|rover | panda: fine for me | 16:42 |
| panda | merge fest! | 16:42 |
| panda | marios: unfortunately I still don't have +2 on the config repo | 16:43 |
| panda | marios: so I can give only my humble +1 | 16:43 |
| quiquell|rover | panda: sshnaidm can | 16:43 |
| quiquell|rover | sshnaidm: we need your "config" repo super powers | 16:44 |
| marios | ack thanks panda i think we can ask sshnaidm rlandy weshay|ruck to check that https://review.rdoproject.org/r/#/c/18729/ please if you have time today or even tomorrow. its the new periodic which inherits from the upstream build containers. it shouldn't break anything, is a new job. not wired into any dependencies yet. we just want to see it running | 16:44 |
| *** agopi is now known as agopi|FOOD | 16:44 | |
| panda | marios: one question about eview.rdoproject.org/r/#/c/18729, you're not using the registry login anywhere ? | 16:45 |
| panda | marios: nevermind I see it | 16:46 |
| marios | https://review.rdoproject.org/r/#/c/18729/12/zuul.d/jobs.yaml@494 panda | 16:46 |
| *** ccamacho has quit IRC | 16:48 | |
| arxcruz | weshay|ruck: ? | 16:48 |
| weshay|ruck | arxcruz, what you don't like karma? | 16:49 |
| arxcruz | weshay|ruck: karma is a bitch :D | 16:49 |
| weshay|ruck | lolz | 16:51 |
| sshnaidm | marios, commented | 16:56 |
| marios | sshnaidm: thanks ! fixing | 16:56 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci- (5 more messages) | 16:56 |
| *** quiquell|rover is now known as quiquell|off | 16:57 | |
| marios | sshnaidm: nice catch done | 16:57 |
| arxcruz | marios: just comment on https://review.openstack.org/#/c/636579/ hope it clarifi | 16:58 |
| arxcruz | clarify* | 16:58 |
| rlandy | weshay|ruck: do you remember if we required any secret or auth to push to https://thirdparty.logs.rdoproject.org? | 17:01 |
| * weshay|ruck looks | 17:03 | |
| rlandy | I don;t think so | 17:04 |
| weshay|ruck | rlandy, I think we're going to need to handle this differently now | 17:06 |
| weshay|ruck | rlandy, internal sf is going to log to it's own log server | 17:07 |
| rlandy | weshay|ruck; the issue is basically this ... downstream zuul jobs have a post. to report to upshift logs | 17:07 |
| marios | arxcruz: looking | 17:07 |
| rlandy | weshay|ruck: also - if I parent of rdo - the reporting will try go to rdo | 17:07 |
| marios | arxcruz: ack thanks, i will have a look tomorrow morning again with fresh coffee :) /me almost out | 17:07 |
| arxcruz | marios: fair enough :) | 17:08 |
| weshay|ruck | rlandy, if they log to rdo that's fine by me :) | 17:08 |
| weshay|ruck | if they have access | 17:08 |
| rlandy | weshay|ruck: if they can | 17:08 |
| rlandy | will have to see what the response is | 17:08 |
| weshay|ruck | rlandy, if you have time.. I'd rather you spend it helping w/ f28 work | 17:08 |
| weshay|ruck | to be honest | 17:08 |
| rlandy | weshay|ruck: ack - just didn;t want to break momentum with the help we are getting from sf guys | 17:09 |
| * rlandy experiments in the backgroud | 17:09 | |
| *** kopecmartin is now known as kopecmartin|off | 17:16 | |
| sshnaidm | weshay|ruck, is it a known issue with introspection problems in ovb? | 17:38 |
| sshnaidm | weshay|ruck, http://logs.rdoproject.org/90/636690/9/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/0653da6/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz | 17:39 |
| weshay|ruck | sshnaidm, there was just an issue w/ the compute hosts locking up | 17:39 |
| weshay|ruck | not sure if it's related | 17:39 |
| sshnaidm | ok | 17:40 |
| weshay|ruck | no one has reported introspection errors yesterday or today | 17:40 |
| weshay|ruck | sshnaidm, hey qq re the reproducer | 17:40 |
| weshay|ruck | how does one ssh to the nodes? | 17:40 |
| sshnaidm | weshay|ruck, ssh to undercloud | 17:40 |
| sshnaidm | weshay|ruck, and then from ther | 17:40 |
| weshay|ruck | ya.. that's how I would normally do it | 17:41 |
| * weshay|ruck looks again | 17:41 | |
| weshay|ruck | hrm.. | 17:42 |
| weshay|ruck | panda, maybe we do need your patch for ovb jobs | 17:42 |
| weshay|ruck | re: failing the job | 17:42 |
| weshay|ruck | sshnaidm, nothing atm has a public 38. address | 17:42 |
| panda | weshay|ruck: and instructions on how make zuul hold the job | 17:43 |
| weshay|ruck | previously the nodepool node did | 17:43 |
| weshay|ruck | panda, aye | 17:43 |
| sshnaidm | weshay|ruck, openstack server list? | 17:43 |
| weshay|ruck | sshnahttp://pastebin.test.redhat.com/714678 | 17:44 |
| weshay|ruck | sshnaidm, https://1ce.org/1#rJyB7f7HV | 17:45 |
| weshay|ruck | hit this 3 times now | 17:45 |
| weshay|ruck | but can't debug | 17:45 |
| sshnaidm | weshay|ruck, you don't have undercloud, seems like it was deleted by nodepool | 17:46 |
| sshnaidm | weshay|ruck, you need to use autohold to keep it | 17:46 |
| weshay|ruck | ya | 17:46 |
| weshay|ruck | so if I'm on the nodepool node.. ssh heat-admin@foo? | 17:47 |
| weshay|ruck | sshnaidm, are you configuring autohold manually? | 17:47 |
| sshnaidm | weshay|ruck, or tripleo-admin | 17:47 |
| sshnaidm | weshay|ruck, currently yes | 17:48 |
| weshay|ruck | easy to explain? | 17:48 |
| sshnaidm | weshay|ruck, I just run ""docker-compose exec scheduler zuul autohold --tenant tripleo-ci-reproducer --job tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 --reason debug --project test1 | 17:48 |
| sshnaidm | in ~/tripleo-ci-reproducer directory | 17:49 |
| sshnaidm | weshay|ruck, just use your job name | 17:49 |
| weshay|ruck | oh cool | 17:49 |
| *** trown is now known as trown|lunch | 17:49 | |
| weshay|ruck | sshnaidm, that is cool | 17:49 |
| weshay|ruck | me wonders if we could add a config to zuul.. something like "recheck autohold" | 17:50 |
| sshnaidm | weshay|ruck, you can also use "docker-compose exec executor zuul-executor keep" for keeping your console in executor node | 17:51 |
| sshnaidm | weshay|ruck, so you can check it later if no logs | 17:51 |
| weshay|ruck | k.. thanks sshnaidm will give it a go | 17:52 |
| *** ykarel is now known as ykarel|away | 17:54 | |
| *** ykarel|away has quit IRC | 18:00 | |
| *** bogdando has quit IRC | 18:05 | |
| *** jpena is now known as jpena|off | 18:08 | |
| *** derekh has quit IRC | 18:09 | |
| *** chem has quit IRC | 18:16 | |
| *** agopi|FOOD is now known as agopi | 18:19 | |
| *** amoralej is now known as amoralej|off | 18:32 | |
| sshnaidm | weshay|ruck, rdo cloud doesn't feel good.. | 18:45 |
| sshnaidm | weshay|ruck, a lot of retry limits | 18:45 |
| weshay|ruck | ya.. | 18:46 |
| * weshay|ruck looks at rhos-ops again | 18:46 | |
| weshay|ruck | sshnaidm, it will probably blow up today | 18:46 |
| sshnaidm | ok, seems like no more ovb today) | 18:47 |
| *** sshnaidm is now known as sshnaidm|off | 18:47 | |
| *** ccamacho has joined #oooq | 18:47 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7 (5 more messages) | 18:56 |
| rlandy | 2019-02-14 18:49:19 | "Error: '/usr/bin/clustercheck >/dev/null' returned 1 instead of one of [0]", | 19:01 |
| rlandy | 2019-02-14 18:49:19 | "Error: /Stage[main]/Tripleo::Profile::Pacemaker::Database::Mysql_bundle/Exec[galera-ready]/returns: change from 'notrun' to ['0'] failed: '/usr/bin/clustercheck >/dev/null' returned 1 instead of one of [0]", | 19:01 |
| rlandy | 2019-02-14 18:49:19 | "Error: Failed to apply catalog: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)", | 19:01 |
| rlandy | still coming up with this error | 19:01 |
| weshay|ruck | rlandy, I still can't find slow, /me asking in channel | 19:08 |
| rlandy | weshay|ruck: yep - he ran away | 19:08 |
| rlandy | been looking for him all week | 19:08 |
| rlandy | weshay|ruck: going down adding the pacemaker stuff in the mean time to debug | 19:13 |
| rlandy | 2019-02-14 18:49:18 | "stderr: Error: unable to find resource 'redis-bundle'", | 19:15 |
| rlandy | since | 19:15 |
| rlandy | OS::TripleO::Services::Redis: ../../docker/services/pacemaker/database/redis.yaml | 19:15 |
| *** ccamacho has quit IRC | 19:17 | |
| *** holser_ has quit IRC | 19:19 | |
| rlandy | weshay|ruck: also starting to love this local zuul testing | 19:34 |
| rlandy | no waiting - quick feedback | 19:34 |
| weshay|ruck | rlandy, cool.. you running scen009 on it? | 19:34 |
| weshay|ruck | or something else? | 19:34 |
| rlandy | weshay|ruck: ack | 19:34 |
| weshay|ruck | PER FECT O | 19:34 |
| rlandy | I am getting somewhere with the services | 19:34 |
| rlandy | and which ones it's objecting to | 19:34 |
| rlandy | if nothing else, it's debug info | 19:35 |
| rlandy | can keep a running tail on logs | 19:36 |
| weshay|ruck | can probably merge https://review.openstack.org/#/c/636385/ | 19:37 |
| rlandy | voted | 19:37 |
| weshay|ruck | thanks.. brb | 19:40 |
| *** weshay|ruck has quit IRC | 19:40 | |
| *** jfrancoa has quit IRC | 19:46 | |
| * rlandy has doctor's appt ... back in about 1.5 hrs | 19:47 | |
| *** rlandy is now known as rlandy|afk | 19:47 | |
| *** weshay has joined #oooq | 20:05 | |
| *** trown|lunch is now known as trown | 20:17 | |
| *** weshay has quit IRC | 20:20 | |
| *** weshay has joined #oooq | 20:21 | |
| *** weshay has quit IRC | 20:29 | |
| *** weshay has joined #oooq | 20:30 | |
| *** weshay has quit IRC | 20:53 | |
| *** weshay has joined #oooq | 20:54 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7 (5 more messages) | 20:56 |
| *** test_weshay has joined #oooq | 21:20 | |
| *** jtomasek has quit IRC | 21:23 | |
| -openstackstatus- NOTICE: Jobs are failing due to ssh host key mismatches caused by duplicate IPs in a test cloud region. We are disabling the region and will let you know when jobs can be rechecked. | 21:30 | |
| *** jtomasek has joined #oooq | 21:32 | |
| *** agopi_ has joined #oooq | 21:39 | |
| *** agopi has quit IRC | 21:42 | |
| *** test_weshay has quit IRC | 21:44 | |
| *** weshay is now known as weshay|test | 21:45 | |
| *** weshay|test is now known as weshay|ruck | 21:45 | |
| *** jtomasek has quit IRC | 21:54 | |
| *** trown is now known as trown|outtypewww | 22:02 | |
| *** tosky has quit IRC | 22:05 | |
| *** tosky has joined #oooq | 22:05 | |
| *** rlandy|afk is now known as rlandy | 22:09 | |
| -openstackstatus- NOTICE: The test cloud region using duplicate IPs has been removed from nodepool. Jobs can be rechecked now. | 22:12 | |
| *** agopi_ has quit IRC | 22:43 | |
| rlandy | "image_exist isn't supported by docker" - hmmm | 22:46 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-scenario002-standalone, tripleo-ci-centos-7-ovb- (5 more messages) | 22:56 |
| *** rlandy is now known as rlandy|bbl | 23:44 | |
| *** tosky has quit IRC | 23:51 | |
| *** agopi_ has joined #oooq | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!