Friday, 2019-03-22

*** irclogbot_0 has joined #oooq00:16
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages)00:24
*** rascasoft has joined #oooq00:28
*** rascasoft has quit IRC00:36
*** aakarsh has quit IRC00:57
*** aakarsh has joined #oooq01:34
*** rascasoft has joined #oooq01:45
*** rascasoft has quit IRC01:54
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages)02:24
*** rascasoft has joined #oooq03:00
*** skramaja has joined #oooq03:08
*** rascasoft has quit IRC03:10
*** apetrich has quit IRC03:15
*** udesale has joined #oooq03:21
*** ykarel has joined #oooq04:16
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages)04:24
*** hamzy_ has joined #oooq04:31
*** hamzy has quit IRC04:33
*** udesale has quit IRC04:47
*** udesale has joined #oooq04:47
*** raukadah is now known as chandankumar05:06
*** rascasoft has joined #oooq05:17
*** rascasoft has quit IRC05:28
*** jdennis has quit IRC05:45
*** jdennis has joined #oooq05:48
*** saneax has joined #oooq06:09
*** jtomasek has joined #oooq06:17
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages)06:24
*** jfrancoa has joined #oooq06:48
*** brault has quit IRC06:50
*** kopecmartin|off is now known as kopecmartin07:37
*** ykarel_ has joined #oooq07:37
*** ykarel_ has quit IRC07:37
*** ykarel_ has joined #oooq07:39
*** ykarel_ has quit IRC07:39
*** ykarel has quit IRC07:40
*** apetrich has joined #oooq07:43
*** ykarel has joined #oooq07:47
*** tosky has joined #oooq07:53
*** amoralej|off is now known as amoralej08:15
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages)08:24
chandankumarmarios_|ruck: Hello08:25
chandankumarmarios_|ruck: please have a look at this card https://tree.taiga.io/project/tripleo-ci-board/task/771?kanban-status=144727508:25
chandankumarmarios_|ruck: I am trying to run new zuul based reproducer but playbook got terminated at TASK [ansible-role-tripleo-ci-reproducer : Wait for zuul tenant]08:26
chandankumarI am executing it from rdocloud vm08:26
chandankumarto reproduce a job on rdocloud itself08:26
marios_|ruckchandankumar: ok will check08:31
*** dtantsur|afk is now known as dtantsur08:33
marios_|ruckchandankumar: did you upload your keys etc to gerrits08:33
marios_|ruckchandankumar: hmm but is 500 error not sure its that08:35
chandankumarmarios_|ruck: it is the same one which is used upstream and in rdo which is in the my local box from where it is running08:35
chandankumaror do I need to manually pass it?08:35
marios_|ruckchandankumar: no its the same then fine08:36
marios_|ruckchandankumar: but i always have to generate new keys for that bug (i.e. -i PEM )08:36
chandankumarmarios_|ruck: sorry which bug?08:37
chandankumarlet me try with generating new keys08:38
marios_|ruckchandankumar: wait08:38
marios_|ruckchandankumar: yes generate but sec08:38
marios_|ruckchandankumar: https://github.com/openstack/tripleo-quickstart-extras/blob/da26295b013d86c833dd461ac096f82aa5792258/roles/create-zuul-based-reproducer/templates/README-reproducer-zuul-based-quickstart.html.j2#L25208:39
marios_|ruckchandankumar: looks nicer in the restult e.g. http://logs.openstack.org/27/644827/2/check/tripleo-ci-centos-7-containers-multinode/7e140d4/logs/README-reproducer-zuul-based-quickstart.html08:39
marios_|ruckchandankumar: click known issues08:40
chandankumaryes, got it08:40
*** ykarel is now known as ykarel|lunch08:48
*** jpena|off is now known as jpena08:51
chandankumarmarios_|ruck: awesome, It worked08:56
chandankumarmarios++08:56
hubbot1chandankumar: marios's karma is now 1008:56
marios_|ruckchandankumar: cool08:57
chandankumarmarios_|ruck: I learned new thing from your taiga ssh-keyscan09:01
chandankumarnever heard using for first time09:02
*** jbadiapa has joined #oooq09:02
marios_|ruckchandankumar: glad it was useful man its actually from one of the reproducer tasks i mean the ssh-keyscan it was failing there on beaker boxes09:05
*** holser_ has joined #oooq09:19
*** bogdando has joined #oooq09:41
*** derekh has joined #oooq09:42
*** jbadiapa has quit IRC10:00
*** saneax has quit IRC10:01
*** saneax has joined #oooq10:03
*** ykarel|lunch is now known as ykarel10:08
chandankumarmarios_|ruck: so we always needs to run reproducer with -f flag otherwise the vm will be wiped out10:18
chandankumarfor autohold the node10:19
marios_|ruckchandankumar: yes10:19
*** jbadiapa has joined #oooq10:19
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages)10:24
*** dtantsur is now known as dtantsur|brb10:28
*** holser_ has quit IRC10:45
*** holser_ has joined #oooq10:47
*** kopecmartin is now known as kopecmartin|lunc10:47
*** holser_ is now known as holser|lunch10:48
arxcruz|ptojaosorior: hey, which scenario deploys barbican? 012 right ?11:26
jaosorior00211:27
arxcruz|ptothanks11:27
arxcruz|ptojaosorior: do barbican have endpoint ?11:28
jaosorioryes11:28
arxcruz|pto\o/11:29
chandankumarweshay: sshnaidm|off here is the new imported git repo https://github.com/chkumar246/tripleo-collect-logs/commits/master11:29
arxcruz|ptojaosorior: btw, i'm working from here today: http://www.thecastleberlin.de/11:30
jaosoriorarxcruz|pto: you're quite on untappd though11:31
*** arxcruz|pto is now known as arxcruz11:31
arxcruzjaosorior: yeah, I can't drink very often...11:31
jaosoriornow is your chance :P11:31
jaosoriorthe place looks great11:31
chandankumarweshay: sshnaidm|off Here is the outlined steps https://tree.taiga.io/project/tripleo-ci-board/task/88911:32
chandankumarmarios_|ruck: how to see whether the node is holded or not?11:36
marios_|ruckchandankumar: can you see the nodes in rdo cloud dashboard for example11:37
chandankumarmarios_|ruck: it got wiped out, not hold11:40
chandankumarI am doign something wrong for sure here11:40
arxcruzweshay: sshnaidm|off marios_|ruck https://review.openstack.org/#/c/645540/11:41
arxcruzthis add barbican tests to skip in featureset02011:41
arxcruzsince we don't have it enabled there11:42
arxcruzit's one of the failures recently11:42
arxcruzhttp://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/fb2acb8/logs/tempest.html.gz11:42
marios_|ruckchandankumar: ack i haven't run that for well over a week worth noting in taiga/wherever and ask rlandy or sshnaidm|off when they're about11:42
arxcruznot sure if there is a bug opened, if so, let me know so I can add on the commit message11:42
marios_|ruckarxcruz: thanks looking11:42
arxcruzI just notice today, I was looking for something else...11:43
marios_|ruckarxcruz: ack i didn't file a bug but i will gimme a minute (i noted it in the etherpad https://etherpad.openstack.org/p/ruckroversprint7 saw it this morning)11:43
marios_|ruckarxcruz: https://review.openstack.org/#/c/645540/11:49
arxcruzmarios_|ruck: updated11:50
marios_|ruckarxcruz: thanks11:51
arxcruznp11:51
arxcruzbrb, lunch11:52
*** holser|lunch is now known as holser_11:54
*** vinaykns has joined #oooq12:02
arxcruzjaosorior: your question, i don't know which scenario is featureset020, but clearly by the error, barbican isn't running there, i asked because i'll add support on python-tempestconf to barbican, so i need an environment to test :)12:03
chandankumararxcruz: feel free to take a look when free https://tree.taiga.io/project/tripleo-ci-board/us/867 thanks for putting the design doc!12:04
*** dtantsur|brb is now known as dtantsur12:05
*** jschluet has joined #oooq12:08
*** kopecmartin|lunc is now known as kopecmartin12:14
*** rlandy has joined #oooq12:14
*** jschlueter has quit IRC12:14
weshayarxcruz can you please update https://review.openstack.org/#/c/565716/12:17
weshaymorning marios_|ruck12:17
*** weshay is now known as weshay|rover12:17
weshay|roverI am your rover12:17
arxcruzweshay|rover: not following... you want to enable ironic on fs001 ?12:18
weshay|roverseemed reasonable to me, what about you?12:19
weshay|roverwe also have scen012 w/ ironic12:19
arxcruzokay12:19
weshay|roverstandalone12:19
arxcruzmaybe add on scen012 directly12:19
arxcruzwithout if's12:19
weshay|roverit may be better to make sure that has the right ironic tempest tests12:19
weshay|roverarxcruz aye12:19
marios_|ruckweshay|rover: o/12:20
arxcruzweshay|rover: actually, scen012 is already running ironic https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/standalone-jobs.yaml#L515-L53412:21
weshay|roveraye.. tempest tests look good to you?12:22
weshay|roveron 1212:22
arxcruzyup12:23
rlandyrfolco: weshay|rover: panda|drappt: pls can we meet briefly about https://tree.taiga.io/project/tripleo-ci-board/task/885?kanban-status=1447276 - from comments on the review, there is overlap12:23
arxcruzmaybe dtantsur can take a look and see if we should add more tests12:23
marios_|ruckweshay|rover: waiting on https://review.openstack.org/#/c/644932/ to merge so we can try to https://review.rdoproject.org/r/19642 for rocky promotion12:23
*** trown|outtypewww is now known as trown|brb12:24
*** trown|brb is now known as trown12:24
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container @  (2 more messages)12:24
weshay|roverarxcruz k.. cool man.. ya.. so you can probably abandon the change on fs0112:24
arxcruzweshay|rover: done12:25
rfolcorlandy, there is some overlap with https://review.rdoproject.org/r/#/c/19451/ on branchful jobs12:26
dtantsurarxcruz: sorry, what's tl;dr?12:26
rlandyrfolco; yes _ would like to confirm with weshay|rover and panda|drappt what that task actually requires12:27
weshay|roverrlandy rfolco sure12:27
rfolcorlandy, only 3rd party I guess, no periodic, no branchful ones12:27
rlandyweshay|rover: pls see comments and questions on https://review.rdoproject.org/r/#/c/19580/12:28
rlandyrfolco: but your comments say to ad branches for steon on some I have not12:28
rlandyso what is it that needs to be changed here?12:28
rfolcorlandy, story description ? it was an initial comment during planning... these stories are split to very small piece of work, and some have overlaps12:30
weshay|roverhow can I help12:30
marios_|ruckweshay|rover: so etherpad is up to date ... 2 main issues at the minute are the rocky job (as above) and the https://bugs.launchpad.net/tripleo/+bug/1821025 otherwise nothing major right now12:31
openstackLaunchpad bug 1821025 in tripleo "tripleo-image-serve : Image-Serve | reload httpd, "Unable to start service httpd: Job for httpd.service " [Critical,Fix released] - Assigned to Cédric Jeanneret (cjeanner)12:31
weshay|rovermarios_|ruck aye man.. enjoy the stillness12:31
rlandyweshay|rover: let's start with this question ... https://review.rdoproject.org/r/#/c/19580/1/zuul.d/ovb-jobs.yaml lines 17 and 2412:32
weshay|roverblue12:32
*** jbadiapa has quit IRC12:33
rlandyjoining12:33
weshay|roverrfolco12:33
rfolcok joining12:34
marios_|ruckrfolco: weshay|rover panda|drappt rlandy zbr arxcruz chandankumar reviews please https://review.rdoproject.org/r/19484 https://review.openstack.org/642662 removes push_tag and re-tagging in the container build job dead code12:34
marios_|ruckplease12:34
marios_|ruckthank12:34
marios_|ruckyou12:34
marios_|ruckvery12:34
marios_|ruckmuch12:34
*** jpena is now known as jpena|lunch12:36
marios_|ruckweshay|rover: feeling brave? https://review.rdoproject.org/r/#/c/19642/ good to go https://review.openstack.org/#/c/644932/ merged12:36
weshay|rovermarios_|ruck aye I AM12:39
marios_|ruckweshay|rover: lets go :D rlandy sshnaidm|off panda|drappt please merge this https://review.rdoproject.org/r/#/c/19642/12:40
marios_|ruckand zuul have mercy12:41
weshay|rovermarios_|ruck let's pull in rlandy to help merge12:42
*** udesale has quit IRC12:47
arxcruzdtantsur: ironic jobs running on fs00212:48
*** udesale has joined #oooq12:48
dtantsursounds good :)12:49
dtantsurhttps://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/standalone-jobs.yaml#L515-L534 looks good to me, but I'm still not sure what the question is12:50
weshay|roverzbr any updates on using the config files?12:52
zbrweshay|rover: yeah, wait few minutes to update it. got into bug in our code.12:53
weshay|rovercool12:54
weshay|roverthanks12:54
zbrweshay|rover: updated, try it yourself and see.12:54
weshay|roverzbr I'm not sure how to12:54
zbrmolecule test -s repo-setup12:55
zbrweshay|rover:  my issue: f28 failed on: sudo dnf config-manager --set-disabled '*' with  No such command: config-manager12:56
chandankumarweshay|rover: frrel free to take a look at these cards https://tree.taiga.io/project/tripleo-ci-board/task/889?kanban-status=1447276 and https://tree.taiga.io/project/tripleo-ci-board/us/867?milestone=223049 when free, thanks!12:57
zbrnot sure which package provides this extension, because on my own f28 host, same cmd works. probably the container does not have it.12:57
weshay|roverzbr ya.. there might be an old fedora config in there we don't use12:57
weshay|rover99% sure the tripleo-ci f28 and promotion release have dnf config manager12:57
weshay|roverwhich iirc is the right command12:57
zbrweshay|rover: and thus is why testing uncovered dead bodies :p12:57
zbri am sure is missing an rpm, ... looking now to find why.12:58
weshay|roverconfig manager12:58
weshay|roveris built into dnf I think12:58
weshay|roverzbr ask the folks who worked on fedora12:58
rlandymarios_|ruck: looking at reviews now13:00
*** altlogbot_2 has quit IRC13:01
*** irclogbot_0 has quit IRC13:01
*** altlogbot_2 has joined #oooq13:01
*** irclogbot_2 has joined #oooq13:01
marios_|ruckgoing afk for a few mins13:03
rlandymarios_|ruck: weshay|rover: https://review.rdoproject.org/r/#/c/19642/ w+'ed13:04
marios_|ruckthanks rlandy13:06
rlandymarios_|ruck: pls rebase https://review.rdoproject.org/r/#/c/19484/13:07
marios_|ruckrlandy: ack thanks13:07
ykarelmarios_|ruck, /me looking at rocky-build patch13:11
ykarelhttps://review.openstack.org/#/c/644932/7/playbooks/tripleo-buildcontainers/templates/build.sh.j2@513:11
marios_|ruckykarel: ack whats up.13:12
ykarelso in promotion pipeline we run from master branch, don't think it will work there13:12
marios_|ruckykarel: ah crap. ok at least is upstream so we can change that conditional pretty easily if we have to13:12
marios_|ruckykarel: thanks for the heads up on that13:12
ykarelhmm was just checking on how we going with rocky promotion13:13
marios_|ruckykarel: well we are trying it with the new job now https://review.rdoproject.org/r/#/c/19642/13:13
ykarelmarios_|ruck, yes i started seeing post ^^ only13:14
ykareli saw that review, and got curious how it's solved without cherrypicks13:14
marios_|ruckykarel: plan is still to do cherrypicks13:14
marios_|ruckykarel: the sed is 'until cherrypicks'13:14
ykarelyup i saw commit message :)13:14
marios_|ruckcos thatneeds tripleo common merged and released etc13:14
marios_|ruckk13:14
marios_|ruckbut we need different conditionals than zuul branch :/13:15
ykarelyes13:15
marios_|ruckrelease will work we pass that from periodics13:15
ykarelyes that should possibly work13:15
*** jbadiapa has joined #oooq13:20
chandankumarzbr: hello13:24
chandankumarzbr: is there a way to use ansible-galaxy in quickstart.sh script?13:25
chandankumarzbr: Now I have requirements.yaml file in os_tempest13:25
zbris bash, you can do whatever you want there but i do not think it would make much sense.13:25
zbrfirst, galaxy is part of ansible, so you need to install ansible first. 2nd: galaxy is deprecated and pending removal in the future.13:26
rlandymarios_|ruck: wrt https://review.rdoproject.org/r/#/c/19484/ and https://review.openstack.org/#/c/642662/ - are they dependent or I can merge https://review.rdoproject.org/r/#/c/19484/ w/o https://review.openstack.org/#/c/642662/?13:28
chandankumarzbr: sorry where can I find the deprecation notice for galaxy?13:28
zbrchandankumar: try to raise a bug in github, it will be closed automatically13:29
chandankumarzbr: sure!13:31
marios_|ruckrlandy: they should be fine independently13:32
rlandymarios_|ruck: ok - so I'll merge https://review.rdoproject.org/r/#/c/19484/13:32
zbrchandankumar: i would personally focus on upgrading ansible, and by the time we finish this we will have ansilble collections and mazer read to use.13:33
*** skramaja has quit IRC13:33
zbrsomehow we are lucky that ansible team just delayed release of 2.8 until may, because on the day 2.8 is released, 2.5 with be EOL.13:35
*** jpena|lunch is now known as jpena13:36
zbrweshay|rover: i fixed the issue with f28 by adding missing rpm. we didn't see it because we used an image already having it.13:36
weshay|roverah k13:37
weshay|roverhtanks13:37
weshay|roverthanks13:37
rlandymarios_|ruck: wrt tht change https://review.openstack.org/#/c/644562/ - I added this as a depends-on change but the change never made it into /usr/share/openstack-tripleo-heat-templates/deployment/. ( it was rebased at the time).If I manually change the file, the overcloud deployment picks up the change. should tht changes be picked up from depends-on reviews - anything special there?13:37
marios_|ruckweshay|rover: we need this too for rocky kudos ykarel https://review.openstack.org/#/c/645562/ when you get a chance please13:39
marios_|ruckrlandy: depends on should work fine for tht13:39
zbrweshay|rover: what should I do about repo-test with mo'? based it on the change that adds release config for stein? current version tests master and rocky13:42
weshay|roverzbr hrm.. not 100% sure what you mean.. let's have a morning sync on blue :)13:49
* weshay|rover is getting very cozy w/ zbr 13:49
zbrjoining in 5min (need coffe) :D13:49
weshay|roverarxcruz can I postpone your 1-1 by 30 min?13:51
arxcruzweshay|rover: you're da boss :)13:53
weshay|roverzbr hold.. I have to chat w/ matbu13:57
zbrsure13:57
zbrweshay|rover: meanwhile you may want to wf https://review.openstack.org/#/c/642662/ to help marios13:59
*** jaosorior has quit IRC14:01
*** jbadiapa has quit IRC14:04
ykarelmarios_|ruck, weshay|rover, any idea why https://bugs.launchpad.net/tripleo/+bug/1820574 is not in CIX, it has promotion-blocker tag14:05
openstackLaunchpad bug 1820574 in tripleo "[rocky] periodic-rocky-containers-build fails for missing centos-binary-cron" [Critical,Triaged] - Assigned to Marios Andreou (marios-b)14:05
weshay|roverzbr k.. ready14:07
*** amoralej is now known as amoralej|lunch14:10
marios_|ruckykarel: i think they are primariy interested in master/stein now but sure we can add a card. really the bug is irrelevant there, i mean the decision is to not fix that but go with the new -push job14:23
marios_|ruckykarel: weshay|rover maybe thats why we didn't add a card14:23
ykarelmarios_|ruck, card creation process is automated, it's seems somehow autamation didn't worked14:24
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container @  (2 more messages)14:24
marios_|ruckykarel: ok must be relatively new at least since december when i was last ruck (the automation?)14:24
ykarelmarios_|ruck, no, automation is in place from long14:25
marios_|ruckykarel: weird we've been adding cards there (i mean myself and others from the ci team :D)14:25
ykarelthe bugs with tag: promotion-blocker have cards created automatically14:25
marios_|ruckrlandy: fyi ^ :D14:25
marios_|ruckykarel: cool good to know14:26
ykarelmarios_|ruck, though they are created after waiting for some hours14:26
ykarel^^ i know, may be things changed recently14:26
arxcruzweshay|rover: ready ?14:28
arxcruzweshay|rover: or we can postpone to monday ?14:29
zbra wf on https://review.openstack.org/#/c/640463/ please (already reviewed)14:30
*** matbu has quit IRC14:51
marios_|ruckrlandy: fyi hit that again http://pastebin.test.redhat.com/742708 running on a rdo vm today... yum update -y seems to have cleared it but wondering if we need to detect for jinja 2.8 (its issue 1 in https://tree.taiga.io/project/tripleo-ci-board/task/765 and you merged that https://review.rdoproject.org/r/#/c/18951/3/requirements.txt already14:57
weshay|roverarxcruz ready14:58
arxcruzweshay|rover: logging14:58
*** matbu has joined #oooq15:01
*** amoralej|lunch is now known as amoralej15:03
zbrmarios: rlandy can you please help with https://review.openstack.org/#/c/645337/ [stein]? safe to merge and 0 risk.15:12
zbrweshay|rover: are you aware of centos-ci issue https://ci.centos.org/job/tripleo-quickstart-gate-master-delorean-quick-basic/8642/console  ?15:15
*** dtrainor has quit IRC15:16
rlandyugh - who lost my change????15:17
zbrrlandy: maybe it went into /dev/zuul ? ;)15:18
rlandymarios_|ruck : sshnaidm|off: ^^ https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleoci.yaml#L3415:18
marios_|ruckrlandy: ?15:19
rlandyhttps://github.com/rdo-infra/review.rdoproject.org-config/blob/9a711a3ee3d31c4f9bce111d5d38e26e2356ef2d/zuul.d/jobs.yaml#L50315:20
rlandyand line 50515:20
rlandymissing       - playbooks/tripleo-ci-periodic-base/pre.yaml15:20
rlandy    post-run:15:20
rlandy- playbooks/tripleo-ci-periodic-base/post.yaml15:20
rlandywhen the jobs moved - those playbooks were lost15:21
*** altlogbot_2 has quit IRC15:21
rfolcorlandy, I knew something was missing there!!15:22
rfolcorlandy, btw wasn't me15:22
marios_|ruckrlandy: ah i see i think sshnaidm|off did that15:22
rlandyno kidding15:22
rlandymarios_|ruck: np - I will patch15:22
marios_|ruckrlandy: don't forget this bit please https://review.rdoproject.org/r/#/c/19584/2/zuul.d/jobs.yaml15:22
marios_|ruckrlandy: looks like that was lost too15:23
marios_|ruckwow15:23
marios_|ruckwtf15:23
rlandyoh joy15:23
rlandyweshay|rover: pls don't let sshnaidm|off  merge code on purim ^^15:24
*** ksambor has quit IRC15:24
rlandymarios_|ruck: ok - putting up patch - will ping you to review we have all the pieces15:24
marios_|ruckrlandy: thanks15:24
*** altlogbot_3 has joined #oooq15:25
weshay|roverarxcruz https://k3s.io/15:29
*** irclogbot_2 has quit IRC15:30
*** irclogbot_1 has joined #oooq15:31
rfolcoweshay|rover, nice ^ you know if can run a single-node-cluster ?15:31
zbrweshay|rover: but it does not have etcd, ... without etcd, k8s is not k8n... is not the same *experience* ;)15:32
rfolcorlandy, please add me to the review as I spent hours yesterday rebasing my patches and did not understand why these bits vanished15:33
*** saneax has quit IRC15:33
*** irclogbot_1 has quit IRC15:36
*** irclogbot_0 has joined #oooq15:37
chandankumararxcruz: I think we need to put deprecation message for validate-tempest in favor of os_tempest, what do you say?15:37
*** aakarsh has quit IRC15:38
rlandymarios_|ruck: rfolco: pls review https://review.rdoproject.org/r/#/c/19698/15:40
marios_|ruckrlandy: lgtm15:41
rlandyrfolco: ^^ pls vote15:48
rfolcorlandy, done thx15:51
*** aakarsh has joined #oooq16:02
rlandymerging16:03
weshay|roverhrm.. rocky is working for me as well16:03
weshay|roverrocky, stein, and master working for me w/ release rocky, stein, master vs.. tripleoc-ci/CentOS-7/$release16:04
weshay|roverzbr ^16:04
chandankumarrlandy: Hello16:05
weshay|roverrfolco I'm ready when ever16:05
chandankumarrlandy: below is my notes from ovb experiment with zuul reproducer https://tree.taiga.io/project/tripleo-ci-board/task/771?kanban-status=144727516:05
chandankumarrlandy: when I download reproducer tar file from a specific job which it does not overrides the particular job16:06
*** aakarsh|2 has joined #oooq16:06
rlandychandankumar: looking through16:06
chandankumarhttp://logs.openstack.org/78/642578/5/check/tripleo-ci-centos-7-standalone/e1eb07d/logs/reproducer-quickstart/launcher-playbook.yaml16:07
rfolcoweshay|rover, ok joining16:07
chandankumarI am thinking of one improvement, like creating clouds.yaml automatically if rdocloud creds is sourced instead of manually creating that file16:08
chandankumarand another finger needs to be installed on the system under test16:08
rlandychandankumar: going back to "which it does not overrides the particular job" - do you mean the dlrn hashes don't work?16:08
*** aakarsh has quit IRC16:09
rlandyor the depends-on does not work?16:09
rlandychandankumar: marios_|ruck also hit this error ... openstack_image|selectattr .. will look into that16:11
chandankumar zuul_yaml: >-16:11
chandankumar      - project:16:11
chandankumar          check:16:11
chandankumar            jobs:16:11
chandankumar              - tripleo-ci-centos-7-standalone-dlrn-hash-tag16:11
chandankumarI think it should be tripleo-ci-centos-7-standalone16:11
marios_|ruckchandankumar: try yum update it helped. the problem is jinja16:11
marios_|ruckchandankumar: issue 1 @ https://tree.taiga.io/project/tripleo-ci-board/task/76516:11
chandankumarrlandy: with reproducer I wanted to invoke this job tripleo-ci-centos-7-standalone not tripleo-ci-centos-7-standalone-dlrn-hash-tag not this16:12
chandankumarmarios_|ruck: you mean, yum update after post requirements stuff?16:14
chandankumarwhen docker service started16:14
marios_|ruckchandankumar: i just did sudo yum update -y after it failed16:14
marios_|ruckchandankumar: and reran16:14
rlandychandankumar: the diff between tripleo-ci-centos-7-standalone and tripleo-ci-centos-7-standalone-dlrn-hash-tag is that the *-dlrn-hash-tag job just specifies the same dlrn hashes as were run in the CI16:14
rlandywe do that so it's an exact reproducer16:14
rlandyif you ran with diff dlrn hashes, the result could be completely different16:15
rlandyall  other job factors remain the same16:15
rlandymake sense?16:15
chandankumarmarios_|ruck: ok, I will try again, I had done yum update early before running the reproducer16:15
chandankumarand one more thing, I think node auto hold should be enabled by default16:16
chandankumarotherwise the job got failed, it wipes out the env16:17
chandankumarrlandy: sorry I did not get that, you mean we need to manually set the delorean hash?16:17
rlandychandankumar: correct16:17
rlandyotherwise you would get the latest tripleo-ci hash16:18
*** ykarel_ has joined #oooq16:18
chandankumarrlandy: then we need to expose a variable in shell script to pass th delorean hash manadatoru16:18
chandankumarotherwise enduser gets confused16:18
rlandychandankumar: iiuc, you want to by pass setting the dlrn hash16:19
rlandyyou want to run latest?16:19
chandankumarrlandy: I want to reproduce the same job as runned in ci just it16:19
chandankumarrlandy: the zuul reproducer work was awesome by the way I liked it for rdocloud16:20
chandankumarThanks to everyone who involved in that :-)16:20
*** ykarel has quit IRC16:20
rlandychandankumar: so you would run the same job as CI ran it - we can bj if I am not understanding you properly16:21
*** holser_ is now known as holser|eod16:23
rlandychandankumar: lastly, pls add your suggestion about automatically creating the clouds.yaml to https://tree.taiga.io/project/tripleo-ci-board/task/771?kanban-status=1447275 so we know to include it for enhancements. thanks16:23
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container @  (2 more messages)16:24
chandankumarrlandy: bj not possible sorry, let me summarize it16:26
chandankumarnetwork issue16:26
chandankumarrlandy: I tried to reproduce this job on rdo-cloud using http://logs.openstack.org/78/642578/5/check/tripleo-ci-centos-7-standalone/e1eb07d/logs/reproducer-quickstart/16:26
chandankumardownloaded the tar file,16:26
chandankumarcreated a vm on rdocloud, copied ssh key and rdocloud creds and created clouds.yaml16:26
chandankumarthen runned reproduer script with rdo/upstream git review user name16:26
chandankumarthen faced few issue, added new sshkey and then updated it16:26
chandankumarthen rekicked the script and it went to standlone deployed with this job tripleo-ci-centos-7-standalone-dlrn-hash-tag16:26
chandankumarand it failed while deploying standlone then checked rdo-cloud node was wiped out16:26
chandankumarthen set autohold node but it does not worked, so left it there, thinking to try another day16:26
chandankumarrlandy: for clouds.yaml I will get the patches up soon by monday!16:26
chandankumarrlandy: can we get a ascinema rocording of the new zuul reproducer it will be easier for people to try?16:27
*** ykarel_ is now known as ykarel16:28
rlandyyeah16:28
chandankumarrlandy: great thanks!16:29
*** ykarel is now known as ykarel|away16:30
*** bogdando has quit IRC16:38
*** holser|eod has quit IRC16:45
marios_|ruckreminder i'm out on monday folks16:46
marios_|rucki'm sure you'll miss me terribly16:46
marios_|rucksshnaidm|off: weshay|rover rlandy panda|drappt zbr rfolco chandankumar arxcruz ^16:46
weshay|rovershit..16:49
weshay|roverrlandy panda|drappt we may be w/o ruck/rover on monday16:50
weshay|roverand myself16:50
marios_|ruckweshay|rover: is sshnaidm|off out?16:51
weshay|roverhe said he MIGHT have to take the day off16:51
weshay|roverI may have to email him to avoid it16:51
marios_|ruckweshay|rover: email me too if you get stuck16:51
weshay|roverI'm also on pto but bringing a laptop16:51
weshay|roverthe only thing we need to worry about is upstream going down16:52
marios_|ruckweshay|rover: fyi needed for the rocky job.. kinda ugly but looks like it does the job. waiting on fedora to report (rdo jobs) then we can consider merging it16:55
marios_|ruckweshay|rover: https://review.openstack.org/#/c/645562/2/playbooks/tripleo-buildcontainers/templates/build.sh.j216:55
marios_|ruckweshay|rover: problem is no 'zuul.branch' available for the periodic rocky job cos branch is always master there apparently (thanks ykarel|away )16:56
chandankumarmarios_|ruck: /me also out on monday!16:56
marios_|ruckchandankumar: nice16:57
chandankumarbut will be available on irc partially16:57
chandankumarneed to attend week day meetup https://www.meetup.com/Vadodara-Open-Infra-Meetup/events/258838945/16:57
*** udesale has quit IRC17:00
chandankumartime to log out, see ya guys Happy weekend! see ya next week!17:02
*** chandankumar is now known as raukadah17:02
weshay|roverrlandy next Friday I scheduled an early meeting w/ the new guy from PIDONE, his name is Luca.  I would like you join for maybe just the last 1/2 of that meeing17:03
weshay|rovermeeting to cover bm work17:03
weshay|roverso 20-30min17:03
rfolcorlandy, (and weshay|rover if you're not busy) can you chat for 5 min and clarify specifics about bm docs for other dfgs ?17:07
weshay|roversure17:09
rfolcomy dream is to travel to greek islands some day. Is this your backyard marios_|ruck ? your one-day-pto program ?17:09
marios_|ruckrfolco: unfortunately no travel on monday just having some family time. its bank holiday here17:10
rfolcomarios_|ruck, enjoy :)17:10
*** dtantsur is now known as dtantsur|afk17:11
*** trown is now known as trown|lunch17:11
weshay|roverbanks don't deserve holidays17:11
* marios_|ruck status and shutdown sequence17:13
weshay|rovermarios_|ruck /me checking dates in oracle17:13
marios_|ruckweshay|rover: thank you. if you reply on the email i'll do it later/tomorrow and send it back17:14
weshay|rovermarios_|ruck I think april 1st is right17:14
weshay|roverapril 1st 2018 - april 1 201917:15
marios_|ruckweshay|rover: ack cool (you mean 19-20 tho right()17:15
weshay|roverha ya17:16
weshay|roverfriggin time17:16
weshay|roverFLYS17:16
marios_|ruck:)17:16
rlandyrfolco: sure17:17
rlandyweshay|rover: sure - np for next Friday17:18
weshay|rovercool17:18
rlandyweshay|rover: so panda|drappt and I are ruck/rover next week?17:19
weshay|roverno17:19
weshay|roverwe have an issue on monday17:19
rlandyjust monday17:19
rlandysure17:20
rlandynp17:20
weshay|roverbut hopefully Sagi will be here17:20
rlandyif not, will cover17:20
weshay|roverand one is enough for monday17:20
weshay|roverthanks17:20
rfolcorlandy, weshay|rover now is a good time ? its quick17:22
rlandyyeah17:22
rfolcorlandy, weshay|rover https://bluejeans.com/587845809717:22
weshay|roverk.. blue?17:22
*** derekh has quit IRC17:26
weshay|roverhttps://docs.openstack.org/tripleo-docs/latest/ci/index.html17:33
weshay|roverrlandy rfolco ^17:33
zbrgood news: ansible 2.8 will no longer need hack for using python3 interpreter/17:42
rlandyrfolco: weshay|rover: https://docs.google.com/document/d/1nDw9vh9jtTh6yHb9pVPNJw8Igh6QjiXUUkwk7X9I0WU/edit#heading=h.obcw27sy3gkk17:44
weshay|roverhttps://docs.google.com/document/d/1nDw9vh9jtTh6yHb9pVPNJw8Igh6QjiXUUkwk7X9I0WU/edit#heading=h.obcw27sy3gkk17:45
weshay|roverhttps://access.redhat.com/documentation/en-us/red_hat_enterprise_linux_openstack_platform/7/html/director_installation_and_usage/sect-Planning_Networks17:49
*** ykarel_ has joined #oooq17:50
*** ykarel|away has quit IRC17:52
marios_|ruckweshay|rover: can you folks check this https://review.openstack.org/#/c/645562/2 we need it for rocky (cc ykarel_  rlandy rfolco  **) added comment at https://review.openstack.org/#/c/645562/2/playbooks/tripleo-buildcontainers/templates/build.sh.j2 looks like it does what we want and we need it for rocky periodic containers build thanks17:53
* marios_|ruck actually goes now bai folks17:53
weshay|roverhttps://code.engineering.redhat.com/gerrit/gitweb?p=tripleo-environments.git;a=blob;f=hardware_environments/dell_fc430_envB/network_configs/single_nic_vlans/env_settings.yml;h=a09cd1e06b949d02c1f4aabdb422e09f63a9c8f7;hb=HEAD17:56
zbrweshay|rover: marios_|ruck rlandy : i need to go now, but please help me merge https://review.openstack.org/#/c/645626/ -- i need it for the other stein related work. -- discovered the bug today.17:59
weshay|roverk17:59
*** trown|lunch is now known as trown18:02
*** amoralej is now known as amoralej|off18:13
*** jpena is now known as jpena|off18:13
*** ykarel_ has quit IRC18:18
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container @  (2 more messages)18:24
*** jfrancoa has quit IRC18:45
rlandyzbr: weshay|rover: w+1'ing https://review.openstack.org/#/c/64562618:59
zbrrlandy: no. found bug, fixing now.18:59
rlandyoh ok18:59
zbrrlandy: as usual upstram != rdo != homeenv ...18:59
zbrrlandy: something is VERY weird, can you have a look? see https://logs.rdoproject.org/26/645626/1/openstack-check/tripleo-build-containers-fedora-28/a68d963/job-output.txt.gz#_2019-03-22_16_12_07_482780 ?19:05
zbrnow, look at first task in https://review.openstack.org/#/c/645626/1/roles/tripleo-repos/tasks/main.yml19:06
zbrisinstalling python3-virtualenv package at that step, and at next task it fails to find virtualenv-3 executable, which is installed by this package!19:06
zbrsame code works upstream but fails on rdo ... ?! me bit perplexed19:07
zbrf*** i found reason in https://logs.rdoproject.org/26/645626/1/openstack-check/tripleo-build-containers-fedora-28/a68d963/logs/system/dnf.log.txt.gz19:10
zbr2019-03-22T16:09:40Z DEBUG Excludes in dnf.conf: python2-pip, python2-setuptools, python2-virtualenv, python3-pip, python3-setuptools, python3-virtualenv19:10
zbrthese kind of hacks make me mad... and funny, I didn't know package would return success if the package was in exclude list.19:11
zbrweshay|rover: still here?19:12
rlandyweshay|rover: tested upshift user, filled out username, domain on doc - works great19:17
*** saneax has joined #oooq19:17
*** weshay|rover is now known as weshay19:17
weshayzbr ya19:17
weshayrlandy thanks19:17
weshayrlandy dang it19:18
weshayrlandy we're going to have to change hostnames19:18
weshayit is going to mess us up here I KNOW IT19:18
rlandyweshay: hostnames?19:18
weshayrlandy just forwarded19:19
rlandyoh dear19:19
rlandyweshay: we only reference the hostnames once - it's the ip's that matter - which will probably change as well19:20
* rlandy wants to go back on vacatio19:20
rlandyvacation19:20
zbrweshay: time for a quick chat in 5 mins?19:22
*** jtomasek has quit IRC19:23
weshayzbr sure.. I can chat now19:23
weshayrlandy sent a follow up19:24
rlandyweshay: thanks - whatever it is - we will deal with it19:26
weshayzbr I'm in my channel19:28
zbrweshay: https://logs.rdoproject.org/26/645626/1/openstack-check/tripleo-build-containers-fedora-28/a68d963/logs/system/dnf.log.txt.gz19:30
zbrhttps://review.openstack.org/#/c/645626/1/roles/tripleo-repos/tasks/main.yml19:31
zbrhttps://review.openstack.org/#/c/645626/119:34
zbrhttps://review.openstack.org/#/c/645273/19:37
zbrweshay: https://logs.rdoproject.org/26/645626/1/openstack-check/tripleo-build-containers-fedora-28/a68d963/logs/system/dnf.log.txt.gz19:39
weshay2019-03-22T16:09:40Z DEBUG Excludes in dnf.conf: python2-pip, python2-setuptools, python2-virtualenv, python3-pip, python3-setuptools, python3-virtualenv19:39
rlandyweshay: oh well 'but we can't pick up the networks and move them as-is unfortunately.' - guess we are rebuilding those settings. Think it is worth asking them id we can enable multinic by default there?19:57
weshayrlandy I don't think they'll know what we're talking about19:58
weshayrlandy so .. ugh19:58
rlandymeeting on monday - we should move that until you are back19:58
weshayit's not a big deal if the network ips change.. if the networks are still broken out the same way19:58
weshaybut19:58
weshaythey always screw this stuff up19:59
rlandyweshay: sure - just some busy work19:59
rlandyok - we wont expect the worst19:59
weshayyou know the problem w/ the rhos-pto calendar19:59
weshayis that it doesn't block your calendar19:59
rlandyweshay: I could not even add an item to the rhos-pto calendar20:00
rlandygood that you reminded me20:00
weshayrlandy you just need to conact eliska for perm20:00
rlandyI think my permissions are off20:00
rlandyack - will do20:00
rlandyI used to be able to20:00
weshayrlandy .. how does the saying go.. grant me the wisdom to understand the difference between the things I can and can not control20:03
weshayrlandy glad we pulled up the network diagrams today20:03
rlandyweshay: lol20:03
weshayrlandy I'm afraid we can't reschedule that20:03
rlandyweshay: when are you back?20:03
weshayrlandy do we have anything from dan sneddon20:04
* rlandy thinking weshay is running away now20:04
weshayre: the networking layout20:04
* rlandy looks in email20:04
weshayrlandy heh20:04
weshayI wish20:04
rlandyhold on - three year search20:04
weshaylolz20:04
rlandyweshay: bunch of docs - sending in pvt as I find them20:07
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @  (2 more messages)20:24
*** holser_ has joined #oooq20:26
*** holser_ has quit IRC20:27
*** trown is now known as trown|outtypewww20:58
rlandyreprovisioning env A undercloud21:28
rlandyweshay: did you hear back from ansible guy re:backport to 2.6.14?21:29
rlandystill holding the workaround21:30
*** saneax has quit IRC21:32
*** aakarsh|2 has quit IRC21:40
*** aakarsh|2 has joined #oooq22:23
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @  (2 more messages)22:24
*** fultonj has quit IRC22:56
*** tosky has quit IRC23:01
*** rlandy has quit IRC23:11
*** chem has quit IRC23:36

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!