Thursday, 2019-03-14

« Wednesday, 2019-03-13 Index Friday, 2019-03-15 »
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone,  (4 more messages)00:19
*** holser_ has quit IRC00:56
*** rlandy|bbl is now known as rlandy01:00
*** rlandy has quit IRC01:38
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci- (4 more messages)02:19
*** ykarel|away has joined #oooq03:04
*** ykarel|away has quit IRC03:22
*** skramaja has joined #oooq03:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (4 more messages)04:19
*** udesale has joined #oooq04:22
*** ykarel|away has joined #oooq04:24
*** ykarel|away is now known as ykarel04:24
*** saneax has joined #oooq04:57
*** raukadah is now known as chandankumar05:50
*** jbadiapa has quit IRC06:03
*** jbadiapa has joined #oooq06:04
*** jtomasek has joined #oooq06:15
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (4 more messages)06:19
*** marios_|ruck has joined #oooq06:20
*** kopecmartin|off is now known as kopecmartin07:00
*** tosky has joined #oooq07:23
*** jfrancoa has joined #oooq07:34
*** gkadam_ has joined #oooq07:50
*** holser_ has joined #oooq08:03
*** holser_ has quit IRC08:17
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (4 more messages)08:19
*** ccamacho has joined #oooq08:20
*** amoralej|off is now known as amoralej08:21
*** holser_ has joined #oooq08:22
*** bogdando has joined #oooq08:34
marios_|rucksshnaidm: o/ /me lost trying to understand what the priorities are08:47
marios_|rucksshnaidm: been trying to make a list of bugs but using https://bugs.launchpad.net/tripleo/+bugs?orderby=-date_last_updated&start=0 is not possible cos they were all updated recently (like release stein3). i went to the prod chain board theres like 5/6 issues there08:48
marios_|rucksshnaidm: hopefully will be clearer when we talk later08:48
*** dtantsur|afk is now known as dtantsur08:59
*** jpena|off is now known as jpena09:00
*** panda|rover|off has quit IRC09:07
marios_|ruckexit09:09
*** marios_|ruck has quit IRC09:09
*** marios_|ruck has joined #oooq09:09
*** ykarel is now known as ykarel|lunch09:09
*** chem has joined #oooq09:15
*** sshnaidm is now known as sshnaidm|rover09:25
sshnaidm|rovermarios_|ruck, let's talk when you're ready09:30
marios_|rucksshnaidm|rover: sure you mean phone?09:34
sshnaidm|rovermarios_|ruck, if you want bluejeans, or whatever :)09:35
sshnaidm|roverphone is a little bit extreme09:36
marios_|rucksshnaidm|rover: yeah i meant bluejeans man sure09:36
sshnaidm|rovermarios_|ruck, https://bluejeans.com/u/sshnaidm/09:37
marios_|rucksshnaidm|rover: there09:38
marios_|ruckhttps://review.rdoproject.org/etherpad/p/ruckrover-sprint709:39
sshnaidm|rovermarios_|ruck, 741530509:41
marios_|ruckhttps://bugs.launchpad.net/tripleo/+bugs?orderby=-date_last_updated&start=009:41
marios_|ruckhttps://trello.com/b/j4IcIomh/production-chain-escalation09:42
marios_|ruckhttps://etherpad.openstack.org/p/ruckroversprint709:43
*** ykarel|lunch has quit IRC09:52
*** ykarel has joined #oooq09:57
arxcruzjaosorior: https://review.openstack.org/#/c/643237 when you have a chance, this is a promotion blocker :)09:57
chandankumararxcruz: I need some help here https://review.openstack.org/#/c/641304/15/test-vars.yml@37510:02
chandankumararxcruz: I need to set the default value there if condition does not get satisfied10:03
chandankumarplease have a look, thanks!10:08
*** panda has joined #oooq10:10
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (4 more messages)10:19
jaosoriorarxcruz: well, you have 2 +2s already; it could be merged.10:20
*** saneax has quit IRC10:21
pandamarios_|ruck: how's it going ?10:24
pandamarios: any progress on the dlrn tag ?10:25
pandamarios: want to chat ?10:25
marios_|ruckpanda: o/ trying to understand priorities/issues made this and spoke briefly with sshnaidm|rover https://etherpad.openstack.org/p/ruckroversprint710:34
marios_|ruckpanda: can you review that and add things that are missing in terms of critical issues you and rfolco were chasing please10:34
marios_|ruckpanda: we have a call in 1.5 hours10:34
marios_|ruckpanda: re the tags, there was a nit in what wes updated (posted v7 https://review.openstack.org/#/c/643095/)10:35
marios_|ruckpanda: so still trying to understand this ^10:36
pandayeah, I was suspecting something like this. My doubt here is "how is that command deciding what tag to use then?"10:39
pandamarios_|ruck: ^10:39
pandamarios_|ruck: ok reviewing etherpad10:39
arxcruzjaosorior: lol when i ping you i had only one +2 :)10:42
arxcruzchandankumar: is the (nodepool_dir is defined) correct with parenthesis ?10:43
arxcruzoh, you abandon10:45
arxcruzso ¯\_(ツ)_/¯10:45
*** udesale has quit IRC10:56
*** udesale has joined #oooq10:56
chandankumardoes anyone tried standalone manually on rdocloud?10:58
*** yolanda has quit IRC11:02
*** yolanda has joined #oooq11:03
pandamarios_|ruck: uploaded PS8 for https://review.openstack.org/64309511:09
pandamarios_|ruck: after speaking with mandre11:09
marios_|ruckpanda: k thx looking11:10
pandamarios_|ruck: I don't thnk you need handoff anyway, you already have all the bugs, and priorities change constantly11:10
marios_|ruckpanda: well its usually customary to hand off ruck/rover no?11:11
pandamarios_|ruck: yes, but you're really already on top of it11:11
pandamarios_|ruck: I'll join, but I think you don't need me at all :)11:11
pandazbr: question: how can you be sure the containers we are pulling in our f28 job are really f28 ?11:12
pandazbr: I mean, is there something set in the prep-containers role to pull fedora-binary- instead of centos-binary ?11:13
*** udesale has quit IRC11:15
ykarelpanda, just in case you forgot about mistralclient bug to handover, i didn't see it in ruck/rover etherpad so pointing11:15
marios_|ruckpanda: are you sure about that --set i still don't see it here https://github.com/openstack/python-tripleoclient/blob/969e01ad8820f516362ad730a0e7f3fce07c35a6/tripleoclient/v1/container_image.py#L606-L66811:15
*** udesale has joined #oooq11:15
*** zbr has quit IRC11:16
pandaykarel: not only I forgot, Power went out in my home this morning so my bouncer lost all the backlog history...11:16
pandaykarel: and since internet is down in my home, I can't access saved logs11:17
marios_|ruckpanda: commented https://review.openstack.org/#/c/643095/8/roles/overcloud-prep-containers/templates/overcloud-prep-containers.sh.j211:17
ykarelpanda, ack, anything needed from me?11:19
pandamarios_|ruck: that role was created and heavily mangled by upgrades, mandre and stevebaker. I think communication with them during this patch would be essential11:21
marios_|ruckpanda: sure but it stil won't change that there is no such parser option :D11:21
marios_|ruckpanda: are you referring to overcloud-prep-containers specifically i guess11:22
pandamarios_|ruck: yes, and I pecifically asked mandre and he said that --seet works for both. So maybe the parser option is inherited from somewhere else ?11:24
marios_|ruckpanda: yeah i thought it might be inheritance but i don't see it there https://github.com/openstack/python-tripleoclient/blob/969e01ad8820f516362ad730a0e7f3fce07c35a6/tripleoclient/v1/container_image.py#L60011:25
marios_|ruckpanda: i.e. if the class was inheriting it would get the parser but isn't afaics11:25
marios_|ruckpanda: if it works then i have no idea how it is working :D11:26
pandamarios_|ruck: like the rest of tripleo11:26
marios_|ruckpanda: i mean this is ridiculous how are we in such a situation11:27
*** saneax has joined #oooq11:27
marios_|ruckpanda: we have no idea what tags we are using where11:27
marios_|ruckits like voodoo11:27
marios_|ruckexcept its our code11:27
marios_|ruckwtf11:27
*** udesale has quit IRC11:27
*** udesale has joined #oooq11:28
*** derekh has joined #oooq11:29
pandaykarel: yes, so, since I'm stupid, can you point me to the bug again, plase ? :(11:29
ykarelpanda, sure11:29
ykarelpanda, https://review.openstack.org/#/c/554950/17/mistralclient/api/base.py@7711:30
ykarelpanda, and https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master11:31
ykarelshould be enough to report bug and take action11:31
pandamarios_|ruck: wow, we actually got the "STOP TALKING TO ME!" card :)11:32
marios_|ruckpanda: qe11:32
pandamarios_|ruck: I remember weshay saying yesterday taht for master we don't actually use that code, but some other code, so it's possibile for the tagging problem taht changing that part with the --set is not actually needed11:37
*** saneax has quit IRC11:38
*** saneax has joined #oooq11:39
marios_|ruckpanda: see my last comment11:39
marios_|ruckpanda: i don't think we need set11:40
marios_|ruckpanda: but instead we are already setting in the env file11:40
marios_|ruckpanda: but there is typo updating11:40
marios_|ruckpanda: ha wonder if the typo is the problem here, or just another problem11:42
marios_|ruckpanda: no maybe not a type11:45
pandaykarel: opened https://bugs.launchpad.net/tripleo/+bug/182004511:48
openstackLaunchpad bug 1820045 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master times out during deploy: Mistral client fails to initialize base action" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)11:48
pandamarios_|ruck: sshnaidm|rover ^ as last task for my ruck rovering, I actually had to open that yesterday11:49
ykarelpanda, ack and Thanks11:49
ykarelgood to open it against mistral also ^^11:49
ykarelas it's really a mistralclient bug11:50
pandamarios_|ruck: ah yes you're right11:50
marios_|ruckpanda: well not sure i mean see the latest comment there (is it a typo? we have it both ways singluar prepare-paramter for undercloud and parameters for standalone :/11:51
pandamarios_|ruck: /o\11:53
pandamarios_|ruck: so this never worked ?11:54
marios_|ruckpanda: well that was also my thought. or we miss something still...11:55
*** skramaja_ has joined #oooq11:57
*** gcerami_ has joined #oooq11:57
*** saneax has quit IRC11:57
*** saneax has joined #oooq11:58
*** zbr has joined #oooq11:58
*** skramaja has quit IRC11:58
marios_|ruckrfolco: sshnaidm|rover panda weshay call if you're joining ruck/rover handoff12:01
pandagcerami_: oh, hello, my internet at home is back12:03
*** amoralej is now known as amoralej|off12:04
*** amoralej|off is now known as amoralej|lunch12:04
rfolcomarios_|ruck, joining in one min12:06
*** skramaja has joined #oooq12:11
*** skramaja_ has quit IRC12:11
*** jpena is now known as jpena|lunch12:12
*** trown|outtypewww is now known as trown12:19
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-multinode-1ctlr-featureset010, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp- (4 more messages)12:19
*** rlandy has joined #oooq12:32
rfolcoarxcruz, do you have a fix for https://bugs.launchpad.net/tripleo/+bug/1819440 ? I hav an env if you want12:40
openstackLaunchpad bug 1819440 in tripleo "phase 1 failing on No such file or directory: '/home/stack/tempest'" [Critical,In progress] - Assigned to wes hayutin (weshayutin)12:40
*** panda has quit IRC12:40
*** gcerami_ is now known as panda12:40
arxcruzrfolco: https://review.openstack.org/#/c/643237/12:41
arxcruzrfolco: https://review.openstack.org/#/c/642420/ and https://review.openstack.org/#/c/643267/ will enable containerized tempest which will avoid this kind of errors in the future12:41
rfolcoarxcruz, good, thanks.12:42
marios_|rucksshnaidm|rover: https://bugs.launchpad.net/tripleo/+bug/182004512:43
openstackLaunchpad bug 1820045 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master times out during deploy: Mistral client fails to initialize base action" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)12:43
weshayzbr I'll reschedule our 1-112:50
weshaysorry I missed it12:50
zbrweshay: no pb, i imagined, me also busy with fixing few things locally.12:51
weshayarxcruz can you focus on the tempest bug re: the tempest dir12:53
weshayrfolco has a recreate for you ^12:53
arxcruzweshay: another ?12:53
rfolcoweshay, he has the fix already12:53
weshaysame one you have a patch on12:53
arxcruzweshay: https://review.openstack.org/#/c/643237/12:53
weshayarxcruz is the man12:53
arxcruzchandankumar did the backport for rocky12:53
arxcruzweshay: I also worked on the logic to have standalone with containerized tempest12:54
arxcruzweshay: https://review.openstack.org/#/c/642420/ and https://review.openstack.org/#/c/643267/12:54
*** skramaja has quit IRC13:01
weshay containers-default-parameters.yaml.txt.gz                                                                                                                              2019-03-14 09:00  1.1K13:01
weshay      containers-prepare-parameter.yaml.txt.gz13:01
weshaymarios_|ruck ^13:01
*** amoralej|lunch is now known as amoralej13:05
arxcruzchandankumar: is tripleo-os-tempest voting on os_tempest side already ?13:06
chandankumararxcruz: the job will be voting at all places13:13
arxcruzchandankumar++13:13
hubbot1arxcruz: chandankumar's karma is now 913:13
chandankumararxcruz: kopecmartin weshay can we have a os_tempest planning meeting tomorrow at the same time?13:13
chandankumarsame == this time13:13
kopecmartinsure13:14
*** vkapalav has joined #oooq13:17
chandankumarkopecmartin: arxcruz weshay invite sent!13:24
rfolcomarios_|ruck, whats ruck notes etherpad you'll be using ?13:38
marioshttps://etherpad.openstack.org/p/ruckroversprint7 rfolco13:40
rfolcomarios, I wish we define url pattern for this :) thx13:40
weshayarxcruz you probably need to put a depends on https://review.openstack.org/#/c/643212/13:40
weshayarxcruz see that?13:41
weshayit fixes scen1/413:41
weshayarxcruz and you have a tox issue13:41
weshayarxcruz re: https://review.openstack.org/#/c/642420/13:41
weshay2019-03-14 10:14:51.074351 | ubuntu-bionic | roles/validate-tempest/tasks/tempest-containers.yml:12:5: [error] wrong indentation: expected 6 but found 4 (indentation)13:42
weshay2019-03-14 10:14:51.074539 | ubuntu-bionic | roles/validate-tempest/tasks/tempest-containers.yml:18:5: [error] wrong indentation: expected 6 but found 4 (indentation)13:42
weshay2019-03-14 10:14:51.074669 | ubuntu-bionic | roles/validate-tempest/defaults/main.yml:24:26: [warning] too few spaces before comment (comments)13:42
rfolcopanda, rlandy: maybe we can w+ this quick one pls? https://review.openstack.org/#/c/641808/13:43
weshayrfolco probably will fail the gate atm13:44
rfolcoah13:44
rfolcok13:44
weshayrfolco unless you depends on https://review.openstack.org/#/c/643212/13:44
rfolcono hurry13:45
mariosweshay: just gonna post it all in one reivew for now so is simpler/we can see if it works. we can split it later if we want to13:46
weshaymarios agree13:46
weshaychandankumar thanks for creating that mtg13:47
weshaycan you make it reoccur every friday13:47
weshayand let myself, arx and martin modify it13:48
chandankumarweshay: ah, done :-)13:49
*** jpena|lunch is now known as jpena13:50
arxcruzweshay: got it13:50
*** jfrancoa has quit IRC13:53
marios_|ruckweshay: this what you meant https://review.openstack.org/#/c/643095/9/roles/overcloud-deploy/tasks/pre-deploy.yml13:56
rfolcomarios, quiquell, sshnaidm, weshay, panda, rlandy, arxcruz, mwhahaha, rfolco, chkumar, ssbarnea, kopecmartin14:00
rfolcoping planning mtg14:00
rfolcohttps://bluejeans.com/707186672814:00
weshaykopecmartin chandankumar you guys can skip14:00
weshaytempest planning mtg on fri14:01
chandankumarweshay: I wanted to join to discuss the collect-log split stuff if it is ok?14:01
weshaychandankumar yes sure14:02
weshayrfolco it's https://redhat.bluejeans.com/5878458097/14:04
pandarfolco: why are you using Matt's BJ# ?14:04
rfolcooh jesus14:04
rfolcocalendar link is wrong14:04
rfolcofor some reason14:04
rfolcosorry arxcruz zbr correct bj is https://bluejeans.com/587845809714:05
*** ykarel is now known as ykarel|away14:05
rfolcoarxcruz, zbr https://bluejeans.com/587845809714:06
marios_|rucksshnaidm|rover: thanks for filing the bug from harald14:11
marios_|rucksshnaidm|rover: https://bugs.launchpad.net/tripleo/+bug/1820065  i mean14:11
openstackLaunchpad bug 1820065 in tripleo "Tripleo client fails in unittests when is being built by DLRN" [Critical,Triaged] - Assigned to Kamil Sambor (ksambor)14:11
sshnaidm|rovermarios_|ruck, yeah, it's taken by ksambor on #tripleo already14:12
*** ykarel|away has quit IRC14:14
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq- (3 more messages)14:19
*** chem has quit IRC14:48
*** saneax has quit IRC14:52
*** holser_ has quit IRC14:53
*** yolanda has quit IRC14:57
chandankumarweshay: I need to leave, I will go through the collect-log us15:05
weshayk15:07
*** apetrich has quit IRC15:10
*** rascasoft has joined #oooq15:14
*** ccamacho has quit IRC15:16
weshayhttps://images.rdoproject.org/master/15:29
*** agopi has quit IRC15:34
*** ccamacho has joined #oooq15:36
*** hamzy has quit IRC15:43
*** yolanda has joined #oooq15:45
pandarfolco: https://github.com/openstack-infra/tripleo-ci/tree/master/scripts/emit_releases_file15:55
*** matbu has quit IRC15:59
*** matbu has joined #oooq16:06
*** agopi has joined #oooq16:14
*** ykarel has joined #oooq16:16
*** matbu has quit IRC16:16
*** matbu has joined #oooq16:18
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq- (3 more messages)16:19
*** kopecmartin is now known as kopecmartin|off16:26
*** jtomasek has quit IRC16:35
marios_|ruckweshay: o/ you blue?16:39
marios_|ruck(your)16:39
weshaymarios_|ruck ya16:39
marios_|ruckk sec joining16:39
chandankumarweshay: sshnaidm|rover panda https://tree.taiga.io/project/tripleo-ci-board/us/852 I am assigning this us to myself and will populate the tasks tomorrow!16:43
*** ykarel has quit IRC16:44
marios_|ruckhttps://hub.docker.com/r/tripleomaster/fedora-binary-base/tags16:47
marios_|rucke3f9cc7df7c87a2fce4e9ddfa05f8365ca63703d_4bfa368516:47
marios_|ruckweshay: ^16:47
*** gkadam_ has quit IRC16:50
*** jaosorior has quit IRC16:57
*** jaosorior has joined #oooq16:58
zbrsshnaidm|rover: can we wf https://review.openstack.org/#/c/615805/ ? thanks.17:00
sshnaidm|roverzbr, no17:02
sshnaidm|roverzbr, it fails in CI, on linters17:02
*** sshnaidm|rover is now known as sshnaidm|afk17:02
zbrsshnaidm|afk: haha... sorry, missed to see that.17:03
sshnaidm|afkzbr, linters(!)17:03
zbri know, bit ironic.17:03
pandazbr: how does the foot feel after shooting it by yourself :)17:06
*** udesale has quit IRC17:06
zbrpanda: quite good, got titan prosthetics17:06
*** trown is now known as trown|lunch17:06
*** matbu has quit IRC17:07
*** hamzy has joined #oooq17:09
*** matbu has joined #oooq17:09
weshayzbr sshnaidm|afk f28 is promoted17:11
weshayhttps://trunk.rdoproject.org/fedora/current-tripleo/delorean.repo17:11
rlandymarios_|ruck: hi - need to borrow your tht expertise ... http://pastebin.test.redhat.com/737906 - trying to figure out what this deploy is complaining about - ever seen this>17:12
zbrweshay: it would be so cool if we can include a comment with the date in hat repo file. hashes are so opaque...17:12
rlandythe deploy parameters should be the same as ovb (plus some env-specific values)17:12
marios_|ruckrlandy: that looks like mistral error17:15
marios_|ruckrlandy: i mean it may not be a tht issue17:15
rlandymarios_|ruck: hmmm ... just thought my config may be wrong17:16
*** bogdando has quit IRC17:16
marios_|ruckrlandy: utException: Invalid input [name=tripleo.parameters.update, class=tripleo_common.actions.parameters.UpdateParametersAction, unexpected=[u\'validate\']]\n'}17:16
*** dtantsur is now known as dtantsur|afk17:18
rlandyd0ugal: hi ... looking to debug this mistral error  http://pastebin.test.redhat.com/737906 - it's a baremetal config so I may have something set in correctly,17:21
weshayarxcruz  what's the status on https://review.openstack.org/#/c/642420/17:22
weshayI see only the standalone running..17:23
weshayare we close?17:23
arxcruzweshay: it's already ready for review, i'm checking the tox error now17:23
weshayarxcruz++17:23
hubbot1weshay: arxcruz's karma is now 1517:23
weshayarxcruz hand that issue off to me before you leave for the day17:23
arxcruzweshay: ok17:24
zbrsshnaidm|afk: does https://review.openstack.org/#/c/641451/ makes sense?17:31
*** chandankumar is now known as chkumar24617:34
*** panda is now known as panda|off17:34
weshayarxcruz which depends on should I use w/ https://review.openstack.org/#/c/643095/ to ensure tempest runs through?17:37
weshayor what should I be on top of17:37
*** derekh has quit IRC17:38
rlandyrfolco: I a looking for a bug I think was logged in the last sprint about deployment failing - not finding containers I think?17:43
rfolcorlandy, hmm...17:43
rfolcorlandy, pacemaker container/service ?17:44
rlandymaybe - anything17:44
rfolcolet me find the ones related17:44
rfolcorlandy, this is about pacemaker container not found - https://bugs.launchpad.net/tripleo/+bug/181899417:44
openstackLaunchpad bug 1818994 in tripleo "ovb jobs broken because pacemaker is unconfigured" [Critical,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)17:44
rfolcorlandy, this is about pulling container image - https://bugs.launchpad.net/tripleo/+bug/181963217:45
openstackLaunchpad bug 1819632 in tripleo "[gate] Error pulling image: invalid character '<' parsing 404 response" [Medium,In progress] - Assigned to Emilien Macchi (emilienm)17:45
rfolcorlandy, this happened once and just vanished, so closed -- https://bugs.launchpad.net/tripleo/+bug/181976617:47
openstackLaunchpad bug 1819766 in tripleo "containers build fails to build ironic-inspector and octavia-base" [Critical,Invalid] - Assigned to Gabriele Cerami (gcerami)17:47
rlandythanks- none of these :(17:47
rfolcorlandy, give me more context I can try finding17:49
rlandyrfolco: no worries - if the ovb master gets by this - so should the master bm17:50
rlandyunless I have a config mistake which I can't find17:50
rlandy http://pastebin.test.redhat.com/73790617:50
* rlandy compares again17:50
weshayfyi.. https://review.openstack.org/#/c/642578/  I need to update the footer-doc to point to the rename of the repdocer-readme but this looks good https://review.openstack.org/#/c/642578/17:52
rfolcorlandy, weshay: I know I was not involved in the last design for bm, but can you please take a look at molteniron for consideration ? https://docs.openstack.org/openstack-ansible-os_molteniron/latest/17:56
rfolcomolteniron is the openstack project for managing bm hw through nodepool17:56
weshayrfolco ya.. we don't want to manage iron in nodepool :)17:57
weshaywe don't give a crap whether or not the nodepool node is bm17:57
weshayrfolco we'll wait on baremetal as a service to be provided to change the workflow17:58
rlandyright now I really care why I can't deploy the overcloud :(17:58
*** panda|off has quit IRC18:01
*** trown|lunch is now known as trown18:01
rfolcorlandy, want to get started on bm, do you prefer me to help you debug something or start any particular task ?18:02
rfolcorlandy, please advise18:03
sshnaidm|afkweshay, https://github.com/sshnaidm/docker-ilo-client18:03
weshayOH YA18:03
*** panda has joined #oooq18:04
weshaywhaa hooo18:04
rlandyrfolco: weshay: right now getting master to deploy is our major blocker - so if you can help figure that out, we can complete that18:05
rlandyrocky is deploying and passing tempest18:05
rlandysomething is not liking the overcloud deploy for master18:05
rlandyirrespective of multinic or single18:06
weshaylog?18:06
rlandyI removed the only env-soecific pieces which is the network-environment.yaml18:06
rfolcorlandy, how I can reproduce what you facing ?18:06
rlandyso if yo can point o a mistral expert or something, that would help18:06
rlandylog ...18:06
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/25/165325/3/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/ac01acf/18:07
weshayrlandy often mistral is not the issue18:07
rlandyweshay: ^^18:07
weshaybut it looks like it18:07
rlandyweshay: rfolco: https://sf.hosted.upshift.rdu2.redhat.com/logs/25/165325/3/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/ac01acf/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz18:07
rlandyovb gets by that so I am pretty sure I am issue18:08
rlandybut I don;t know why18:08
rlandyso there we go18:08
rlandyyou can get onto the hardware with the rhos-jenkins key18:08
rlandyI have fiddled there so I can get the deploy back to its original form18:08
weshayrlandy  unexpected=[u\'validate\']]\n'}18:09
rlandyweshay: yep - I have no idea18:09
rlandyrfolco: ^^18:09
rlandyI am deploying multi-nic18:10
rlandyso the rest should be the same18:10
rlandythe deploy was missing  this ...18:10
weshayrlandy there was a bug on docker-ha and docker being in the wrong order18:10
rlandyhttps://github.com/openstack-infra/tripleo-ci/blob/master/test-environments/worker-config.yaml18:10
rlandyoh - can you paste pls18:10
rlandycompare with this though ... http://logs.rdoproject.org/15/641015/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/96146fc/logs/undercloud/home/zuul/overcloud-deploy.sh18:11
rlandyovb master is failing but much further along than that18:11
rlandyrfolco: if we can figure this out, I have a few review to merge and then I could let you use the hardware to get familiar with the work18:12
rfolcorlandy, ok looking18:13
weshayrlandy why is the bm job using ovb-ha?18:14
weshay-e /usr/share/openstack-tripleo-heat-templates/ci/environments/ovb-ha.yaml18:14
rlandybecause I am randomly copying now :)18:15
weshaylolz18:15
weshaythe issue is def.. w/ what you are passing18:15
rlandyfors ure18:15
rlandyI am the problem - no doubt18:16
weshayrlandy you may have a conflict w/ /home/zuul/network-env18:16
weshayand ovb-ha18:16
* rlandy looks at rocky bm to compare18:16
weshayrlandy my suggestion is to get on the undercloud and make adjustments18:18
rlandyweshay: rfolco: here is the rocky comparison ... https://sf.hosted.upshift.rdu2.redhat.com/logs/22/165122/4/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-rocky/39bc5d2/logs/undercloud/home/zuul/overcloud-deploy.sh. I could just be lucky there and it works out18:18
weshayand rerun18:18
rfolcovalidate_stack=False but then validate_stack=validate_stack18:18
rlandyrfolco: ^^ where is that18:18
rlandysounds promising18:18
rfolcohttps://sf.hosted.upshift.rdu2.redhat.com/logs/25/165325/3/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/ac01acf/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-03-14_16_28_1018:18
rfolco"/usr/lib/python2.7/site-packages/tripleoclient/v1/overcloud_deploy.py", line 391, in _deploy_tripleo_heat_templates18:19
rfolco2019-03-14 16:28:10 |     validate_stack=False)18:19
rfolcobut then...18:19
rfolco"/usr/lib/python2.7/site-packages/tripleoclient/workflows/plan_management.py", line 238, in update_plan_from_templates18:19
rfolco2019-03-14 16:28:10 |     validate_stack=validate_stack)18:19
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci- (3 more messages)18:19
weshayit's not ovb-ha https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/ovb-ha.yaml18:21
rlandyweshay: ContainerCli: docker - shouldn't that be podman in master?18:22
rlandyalthough ovb has the same thing18:23
rfolcoI think docker does not matter18:23
rlandyrfolco: I think we will need to do as weshay advises - get on the undercloud and fiddle the deploy args until we get rid of the one that causes the error18:23
rlandyrfolco: can ping you in about 20 mins to do that if you like18:24
rlandyneed to step away for a few18:24
rfolcorlandy, why do we need to adjust that? didn't it work on bm jobs with ovb-ha ?18:24
weshayrlandy yes18:24
rlandyrfolco: will ping you in a bit we can go through it together18:25
*** rlandy is now known as rlandy|biab18:25
rfolcorlandy, ok, yes pls ping me, we can tmate or bj also. I'll get a read on stories18:25
rfolcothx18:25
chkumar246weshay: can I create f28 os_tempest job also?18:26
chkumar246to see how things are working on os_tempest side18:27
*** jpena is now known as jpena|off18:27
weshayya.. that should work18:27
*** amoralej is now known as amoralej|off18:33
weshaysshnaidm++18:39
weshaysshnaidm++18:39
weshayrlandy|biab check it out when ur back :)18:40
weshayhttps://github.com/sshnaidm/docker-ilo-client18:40
*** vkapalav has quit IRC18:55
*** vinaykns has joined #oooq19:00
*** rlandy|biab is now known as rlandy19:02
rlandyrfolco: starting debug19:10
rfolcorlandy, do you mind in sharing with me ?19:11
* rlandy will tmate19:11
rlandyrfolco: actually we should probably bj for 5 so I can show you the repos19:12
rfolcorlandy, ok whats your room pls19:13
rlandyhttps://bluejeans.com/116727655619:14
rfolcorlandy, wait19:17
rfolcorlandy, connection issue19:17
rlandyrfolco: http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/doc/oooq-upstream-baremetal.rst19:18
weshayzbr marios_|ruck http://logs.openstack.org/51/637151/4/check/tripleo-ci-fedora-28-standalone/67e0fa6/logs/stackviz/#/testrepository.subunit19:29
weshay\0/19:29
weshayrlandy https://review.rdoproject.org/r/1938419:41
* rlandy needs to think about that19:45
*** yolanda has quit IRC19:46
zbrweshay: \0/ !!!!19:46
zbrweshay: rlandy : and while celebrating maybe you can help with this simple one https://review.openstack.org/#/c/640463/419:52
weshayzbr ok..20:02
weshayzbr holy shit that got a +1 from rdo20:03
zbrweshay: starting to wonder if you are dreaming? ;)20:03
weshayCode-Review20:04
weshayVerified20:04
weshay+1 RDO Third Party CI Zuul20:04
weshayWorkflow20:04
rfolcorlandy, semaphore... looks nice, did you test ?20:10
rlandyrfolco: quique did20:10
rfolconice20:10
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @  (3 more messages)20:20
*** dsneddon has quit IRC20:27
rlandyhttps://github.com/openstack/tripleo-common/blob/master/workbooks/plan_management.yaml#L12720:28
rlandy^^ error is there I think20:28
weshayrlandy so ovb should fail on the same20:30
weshayrlandy is there a hold option now in the repro?20:30
rlandyweshay: I removed practically every config I passed20:30
rlandy{u'status': u'FAILED', u'message': u'<% task(ensure_passwords_exist).result %>'}, u'on-success': u'send_message'}}, u'name': u'update_deployment_plan', u'version': u'2.0', u'tags': [u'tripleo-common-managed']}}, u'plan_name': u'overcloud', u'execution_id': u'e492c225-40b1-45f7-a7a0-e787bb01dba7', u'deployment_status': None}}] evaluate /usr/lib/python2.7/site-packages/mistral/expressions/yaql_expression.py:16820:31
weshayrlandy can you try to run openstack overcloud plan export <plan_name>20:33
weshayw/ overcloud20:33
rlandyhttps://10.12.149.2:13808/v1/AUTH_65a7182662214c459aee783a7a85f447/plan-exports/overcloud.tar.gz?temp_url_sig=ebae2038f5170b89a145df86a57959e6e00f9823&temp_url_expires=155259930920:35
rlandynot resolving for me20:36
weshaywhat is 10.12.149.2 ur undercloud20:36
weshaydid the command try that url?20:37
weshayand fail?20:37
rlandyyes20:38
rlandyundercloud20:38
rlandyvm undercloud20:38
rlandyit created that url20:38
rlandyweshay: do we have a mistral contact other than adriano?20:42
weshayrlandy anything funny w/ networking you can detect?20:43
weshayrlandy he IS the ONLY one20:43
rlandya hundred possibilities20:43
rlandyI can ping redhat.com20:43
weshayrlandy you could ask nicely toure20:43
rlandyand the dns server20:43
weshayrlandy you've hit this more than once/20:43
weshay?20:43
rlandyon two separate boxes20:44
weshayrlandy and ur hitting current-tripleo?20:44
rlandylet me check20:45
*** hamzy has quit IRC20:45
weshayhttps://review.rdoproject.org/r/#/c/19384/20:45
rlandyweshay: --extra-vars @/home/zuul/workspace/.quickstart/config/release/tripleo-ci/CentOS-7/master.yml20:52
rlandyovb has this:20:53
rlandybaseurl=http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8080/rdo/centos7/56/ee/56eedaf13617f1880ce5b1b268db53a774f453c1_a77b775220:53
rlandybm has:20:53
rlandybaseurl=https://trunk.rdoproject.org/centos7/56/ee/56eedaf13617f1880ce5b1b268db53a774f453c1_a77b775220:53
weshayoh that's od20:54
weshayzbr ur killing me20:54
rlandydelorean-current.repo is different20:54
weshaythat's normal20:55
weshaymy odd comments was the 808020:55
rlandyweshay: will adriano be returning anytime soon?20:56
weshaythen remembered.. mirror20:56
* weshay checks.. you should talk to his manager20:56
weshayk.. qq rlandy20:57
weshayyou run rocky on the same env and it passes right?20:57
rlandyweshay: ack on a similar env20:58
weshayhrm20:58
rlandyenvD vs endE20:58
weshayrun rocky on this one20:58
weshayif it works on rocky, and only fails on master20:58
weshaythen I20:58
weshayI'll grab him20:58
weshayrlandy you could also try previous-current-tripleo20:59
weshayrlandy but because ovb is working20:59
weshayand based on where it's failing20:59
weshayI'm inclined to think it's not a mistral bug20:59
rlandyok - let me try rocky21:00
weshayI could be wrong21:00
weshaybeen wrong before21:00
rlandyon the exact same system21:00
rlandyweshay: thing with bm is that we walk alone21:00
rlandynobody knows really21:00
rlandyif we get the system working once - at least we have leverage21:01
weshaywe would see the issue outside of bm21:01
weshayespecially since ur passing the same options as ovb21:01
weshayrlandy can you look at that config review please21:09
weshayhttps://review.rdoproject.org/r/#/c/19384/21:10
rlandyyes - just checking on fs002 to confirm dep21:13
*** vinaykns has quit IRC21:14
*** trown is now known as trown|outtypewww21:16
rlandyweshay: just confirming - you intended to also remove:  periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master:21:17
rlandyif yes .. will w+21:17
weshaythat is the mixed job21:17
weshayya.. nuke that21:17
rlandyk21:17
weshaymy drive went down on rhel box :(21:18
*** agopi has quit IRC21:20
rlandyoh dear21:27
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario004-standalone, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario001-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @  (3 more messages)22:20
*** rlandy is now known as rlandy|bbl22:22
*** tosky has quit IRC23:26
*** agopi has joined #oooq23:40
*** agopi has quit IRC23:40
*** dsneddon has joined #oooq23:55
*** dsneddon has quit IRC23:59
« Wednesday, 2019-03-13 Index Friday, 2019-03-15 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!