Friday, 2018-12-21

« Thursday, 2018-12-20 Index Saturday, 2018-12-22 »
*** rascasoft has joined #oooq00:04
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7 (1 more message)00:08
*** rascasoft has quit IRC00:10
*** tosky has quit IRC00:28
*** dsneddon has quit IRC01:21
*** dsneddon has joined #oooq01:23
*** dsneddon has quit IRC01:29
*** dsneddon has joined #oooq01:34
*** dsneddon has quit IRC01:42
*** dsneddon has joined #oooq01:45
*** dsneddon has quit IRC01:51
*** dsneddon has joined #oooq01:58
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7 (1 more message)02:08
*** dsneddon has quit IRC02:08
*** dsneddon has joined #oooq02:16
*** ykarel|away has joined #oooq02:59
*** hamzy_ has quit IRC03:10
*** dsneddon has quit IRC03:32
*** dsneddon has joined #oooq03:34
*** dsneddon has quit IRC03:39
*** dsneddon has joined #oooq03:55
*** dsneddon has quit IRC04:01
*** udesale has joined #oooq04:05
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp- (1 more message)04:08
*** saneax has joined #oooq04:23
*** dsneddon has joined #oooq04:34
*** dsneddon has quit IRC04:40
*** ykarel|away is now known as ykarel04:53
*** dsneddon has joined #oooq05:10
*** dsneddon has quit IRC05:15
*** ykarel has quit IRC05:35
chandankumarquiquell|off: https://review.rdoproject.org/r/#/admin/projects/rdo-infra/ansible-role-tripleo-ci-reproducer05:39
*** dsneddon has joined #oooq05:45
*** ratailor has joined #oooq05:49
*** dsneddon has quit IRC05:50
*** dsneddon has joined #oooq05:52
*** ykarel has joined #oooq05:53
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp- (1 more message)06:08
*** jfrancoa has joined #oooq06:14
quiquell|offchandankumar: o/07:21
quiquell|offchandankumar: thanks !07:21
*** quiquell|off is now known as quiquell07:24
*** quiquell is now known as quiquell|brb07:32
chandankumarquiquell|brb: there is some replication issue with github we will wait for jpena or amoralej to come07:41
*** ykarel is now known as ykarel|lunch07:50
*** kopecmartin|off is now known as kopecmartin07:55
*** quiquell|brb is now known as quiquell08:01
quiquellchandankumar: what replication ?08:01
chandankumarquiquell: above repo should be replicated here https://github.com/rdo-infra/08:03
quiquellchandankumar: Do I have to change something in the project config ?08:05
chandankumarquiquell: nope08:05
chandankumarquiquell: may be rdocloud outage caused something08:05
quiquellchandankumar: I think yesterday there were issues08:08
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (1 more message)08:08
*** rascasoft has joined #oooq08:11
*** tosky has joined #oooq08:27
*** dmellado has quit IRC08:44
*** ykarel|lunch is now known as ykarel08:45
*** dmellado has joined #oooq08:46
*** jpena|off is now known as jpena08:53
*** tosky has quit IRC08:54
*** tosky has joined #oooq08:55
*** dsneddon has quit IRC09:12
*** skramaja has joined #oooq09:13
*** dsneddon has joined #oooq09:20
*** derekh has joined #oooq09:35
*** dsneddon has quit IRC09:40
*** tosky has quit IRC09:54
*** tosky has joined #oooq09:55
*** holser_ has joined #oooq10:00
*** holser_ has quit IRC10:00
*** holser_ has joined #oooq10:01
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (1 more message)10:08
*** ssbarnea has joined #oooq10:13
*** dsneddon has joined #oooq10:16
*** dsneddon has quit IRC10:28
*** jtomasek has joined #oooq10:31
*** amoralej is now known as amoralej|afk10:31
*** saneax has quit IRC10:41
*** udesale has quit IRC10:46
*** gkadam has joined #oooq10:56
*** agopi has joined #oooq10:58
mariosrfolco: panda where can i add this to the board please  https://review.openstack.org/604768 and now associated https://review.rdoproject.org/r/#/c/17962/10:59
mariosrfolco: panda|off ssbarnea sshnaidm|off quiquell *  can you check https://review.rdoproject.org/r/#/c/17962/ when you have time thanks11:00
*** dsneddon has joined #oooq11:02
*** dsneddon has quit IRC11:11
quiquellmarios: +1 we have to merge this before the other (My comment in the review is wrong)11:11
*** jfrancoa has quit IRC11:25
*** dsneddon has joined #oooq11:26
chandankumarquiquell: Can I try the reproducer role locally? it is available there?11:33
*** dsneddon has quit IRC11:34
quiquellchandankumar: We have to wait for public images at RDO cloud11:34
chandankumarquiquell: ok11:34
*** holser_ is now known as holser|lunch11:50
*** saneax has joined #oooq11:56
panda|offmarios: quiquell  https://review.rdoproject.org/api/17745 this is adding important feature yet should be backwards compatible and ready to merge12:03
panda|offs/ready/safe12:04
quiquellpanda|off: ack, looking12:04
mariospanda|off: ack add comment please? /me going afk biab12:07
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (1 more message)12:08
quiquellpanda|off: Can we run this with a fake periodic job or anything ?12:08
*** dsneddon has joined #oooq12:10
quiquellAhh is easy to run it locally12:10
quiquellNot needed12:10
*** amoralej|afk is now known as amoralej12:12
panda|offquiquell:  marios this instead https://review.rdoproject.org/api/17744 cannot be tested before, adds feature but for the first pass I'm focusing to make it a non breaking noop for the current pipeline12:15
quiquellpanda|off: script is working fine, just tested it locally12:15
panda|offquiquell: I'm da man12:15
quiquellpanda|off: +212:16
quiquellWe can do some assertions here and there and also add defaults for RELEASE and the like, but let's not stop on this12:16
quiquellpanda|off: what else is needed to review ?12:17
panda|offI added release, but for example I have no default fro the promote name12:17
panda|offquiquell: https://review.rdoproject.org/api/1774412:18
*** dsneddon has quit IRC12:23
*** jpena is now known as jpena|lunch12:24
*** ratailor has quit IRC12:24
ykarelpanda|off, hey12:26
ykarelpanda|off, when u will be |on, i wanted to ask something12:26
*** panda|off is now known as panda12:28
pandaykarel: ask away12:28
ykarelpanda, so u was working on rdoinfo jobs translation to new workflow12:29
ykarelpanda, i wanted to check if that's finished12:29
ykareli see the review: https://review.rdoproject.org/r/#/c/17352/1812:29
ykarelbut can't get what's still pending there12:29
pandaykarel: define finished12:30
pandaykarel: everything is blocked by this https://review.openstack.org/60728812:30
ykarelpanda, can we use it12:30
pandauntil that merges, there's no easy way to reuse selected part of tripleo-ci workflow12:30
ykarelpanda, i mean can we use this job in place of legacy one we are using12:30
ykarelpanda, ack12:30
ykarellooking that review12:30
pandaykarel: the job works12:30
pandaykarel: but it needs the tasks in form of role12:31
pandaso it can be used in a different project12:31
ykarelpanda, Thanks for the info12:32
ykarelso https://review.openstack.org/#/c/607288/ just waiting for reviews, right?12:32
pandaykarel: it's waiting for planetary alignment12:32
pandaykarel: all gate must work, it must be a quiet moment to not create too much distruption, and people must be ok with this big change12:33
quiquellpanda: +1 https://review.rdoproject.org/r/#/c/17744 little nit12:33
pandaykarel: and also there must no change in run-v3 otherwise I have to rebase everything manually again/.12:34
pandaquiquell: thanks12:35
ykarelpanda, ack, so atleast jobs in the patch are working, good to test the other jobs not tested there + promotion ones12:35
ykarelatleast that will make people happy12:36
pandaquiquell: the name wasn't there before :) but yeah, we should leave the code a bit better than how we found it.12:36
quiquellpanda: I know, little boy scout there12:37
quiquellNow that we have mixed stuff better to have debuging info12:38
*** agopi has quit IRC12:41
pandarfolco: when is the followup packaging meeting ?12:42
rfolcopanda, ooops12:43
rfolcopanda, forgot that one12:43
rfolcochekcing calendar... whats works best for you guys ?12:43
quiquellpanda, rfolco: Have to take 24th too, going to update etherpad12:44
pandarfolco: ok, in 1:15 hour ?12:44
rfolcoquiquell, I was surprised 24th is official holiday at RedHat (br)12:45
quiquellrfolco: yep12:45
*** derekh has quit IRC12:47
pandaquiquell:  fixed12:48
quiquellpanda: thanks12:48
quiquellrfolco: spain is from 25th not from 24th the company shutdown thing12:49
*** derekh has joined #oooq12:51
rfolcoquiquell, panda marios ssbarnea weshay|ruck ci breakage/ packaging follow-up scheduled, invites sent (starts +1h from now).12:54
quiquellrfolco: thanks12:55
*** dsneddon has joined #oooq12:56
*** chem has quit IRC12:57
rfolcomarios, question about replacing multinode w/ standalone...12:58
rfolcoyesterday while talking to weshay|ruck, I raised the following question: if we replace/remove multinode containers and replace with standalone, what we'll run for stable branches ? none. standalone covers stein only.12:59
*** holser|lunch is now known as holser_12:59
rfolcostable branches will be uncovered ? I've added the question here, please clarify https://tree.taiga.io/project/tripleo-ci-board/task/448?kanban-status=144727613:00
*** dsneddon has quit IRC13:01
holser_ssbarnea https://review.openstack.org/#/c/626157/ and https://review.openstack.org/#/c/625997/ were merged13:03
holser_there shouldn't be any issues with linting anymore13:04
weshay|ruckrfolco, add this to the invite https://docs.google.com/document/d/1kxGQ6WNGROUdApeOuogFxo7Kr_bCVMon-Gn87PjLGys/edit#heading=h.dnzjtqczrcp813:05
mariosrfolco: o/ right already considered this13:05
weshay|ruckquiquell, panda marios ssbarnea fyi https://docs.google.com/document/d/1kxGQ6WNGROUdApeOuogFxo7Kr_bCVMon-Gn87PjLGys/edit#heading=h.dnzjtqczrcp813:06
*** rlandy has joined #oooq13:06
mariosrfolco: see for example in https://review.openstack.org/#/q/topic:replace-scen1 you can see the abandoned ones for the stable/brances13:06
rfolcomarios, yeah just realized and answered myself.13:06
rfolcothanks13:06
rfolcoweshay|ruck, k, adding13:06
mariosrfolco: https://tree.taiga.io/project/tripleo-ci-board/task/448 also noted on the description13:06
mariosrfolco: ack on the call thanks13:06
quiquellweshay|ruck: I have stupid questions for the meeting13:06
*** rlandy is now known as rlandy|rover13:07
rlandy|roverweshay|ruck: morning :013:08
weshay|ruckquiquell, well.. I am stupid so I should be able to field those questions :)13:08
weshay|ruckmorning13:09
weshay|ruckrlandy|rover, 1/2 merged13:09
weshay|ruckrlandy|rover, gate is nice and clear13:09
rlandy|roverI see - that's a start13:09
rlandy|roveryesterday scared everyone off13:09
rlandy|roverperiodic-tripleo-centos-7-master-containers-build failing13:09
rfolcoquiquell, https://etherpad.openstack.org/p/tripleo-ci-squad-meeting @L30, just added a Q&A there, feel free to add your questions13:09
rfolcoif weshay|ruck prefers to use only the gdocs, we can transfer there13:10
*** skramaja has quit IRC13:10
rlandy|roverhmmm ... same deal I think13:10
weshay|ruckrlandy|rover, I think you can ignore the current master container build job13:11
weshay|ruckhttps://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build/7caba3f/logs/undercloud/home/zuul/install-undercloud.log.txt.gz13:11
weshay|ruck    "2018-12-21 12:45:56,616 ERROR: 17420 -- Failed running docker-puppet.py for heat_api",13:11
weshay|ruck    "2018-12-21 12:45:56,618 ERROR: 17420 -- standard_init_linux.go:203: exec user process caused \"no such file or directory\"",13:11
rlandy|roveryeah - was just looking at it13:11
rlandy|roverpity - would have liked to have seen the shake out13:11
weshay|ruckrlandy|rover, the fix didn't make it in13:11
Tengugrumpf. bad podman.13:12
rlandy|roverhttps://review.openstack.org/#/c/626546/13:12
rlandy|roverweshay|ruck: ^^?13:12
rlandy|roverjust in I guess13:12
Tengurlandy|rover: that one won't correct the "standard_init_linux.go:203:" thing, but the relabelling of netns directory13:13
Tenguand yep, it just landed. Another one, by Yatin, is having troubles with the random podman failures: https://review.openstack.org/#/c/626511/13:14
Tengualthough apparently the patch for puppet-tripleo is "the" one correcting most of the netns issues. still, yatin's needed.13:14
weshay|ruckrlandy|rover, promotion started 3hrs ago13:15
weshay|ruckthat landed 2hrs ago13:15
weshay|ruckquiquell, let's 1-113:16
quiquellack13:17
weshay|rucklet's knock it out early.. I read my calendar incorrectly in the middle of the night13:17
* weshay|ruck gets ready13:18
quiquellconnecting13:18
weshay|ruckhttps://bluejeans.com/u/whayutin/13:19
*** dsneddon has joined #oooq13:20
weshay|ruckquiquell, http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&panelId=61&fullscreen13:29
*** dsneddon has quit IRC13:31
rfolcorlandy|rover, do you know what is the issue/bug/log that broke our reproducer ?13:40
rlandy|roverrfolco: which one - old one?13:40
rfolcoyep old one13:40
rlandy|roverrfolco: not sure if there was a bug but this is the fix ... https://review.openstack.org/#/c/626758/13:41
rlandy|roverweshay|ruck put it in and I am testing it now13:41
rlandy|roverrunning on my tenant13:41
rlandy|roverit was the removal of te-broker13:41
rfolcorlandy|rover, ah ok, can I assign to you the new task/story on regression testing for the reproducer on taiga ?13:42
rfolcorlandy|rover, oh you are rover... :)13:42
*** amoralej is now known as amoralej|lunch13:42
rfolcorlandy|rover, will just add the bug to the task13:43
rfolcofor now... thx13:43
rlandy|roverrfolco: probably - I think it's a rover task13:43
rlandy|roverto keep the old one in shape13:43
rlandy|roverbut I think the task is for the new one, no?13:43
rlandy|roverregression testing the new reproducer?13:43
rfolcorlandy|rover, probably yes. Let me do this. An issue for the old one, assigned to you. The new task for the new one.13:44
rfolcorlandy|rover, thx13:44
rlandy|roverok13:44
rfolcorlandy|rover, fyi https://tree.taiga.io/project/tripleo-ci-board/issue/536?kanban-status=202773313:47
*** trown|outtypewww is now known as trown13:48
quiquellImpos13:53
rlandy|roverquiquell: hi - what happened with the images discussion?13:55
weshay|ruckmarios, you have a minute? re: https://review.openstack.org/#/c/620305/13:55
rlandy|roversf not happy with the public image idea?13:55
*** jtomasek has quit IRC13:56
weshay|ruckwha?13:56
mariosweshay|ruck: o/ getting ready to talk in 2 mins on bluejeans with you?13:57
*** dsneddon has joined #oooq13:58
quiquellrlandy|rover: I was talking with kforde about making them public13:59
weshay|ruckmarios, heh.. ya.. bad timing :)13:59
quiquellrfolco: but there is a comment in the task13:59
quiquellrlandy|rover: https://tree.taiga.io/project/tripleo-ci-board/task/413?kanban-status=144727513:59
weshay|ruckmarios, I'll catch up w/ you later13:59
quiquellrlandy|rover: I am having a discussion at #sf-ops about it14:01
quiquellrlandy|rover: with paul, alan and kieran14:01
rlandy|roveryeah - following14:01
rlandy|roverdidn't know it was still in question14:01
quiquellrlandy|rover: paul just said that no problem with deletion14:02
*** dsneddon has quit IRC14:02
quiquellrlandy|rover: going to talk with kforde so at least our personal tenants at our team have them14:03
quiquellrlandy|rover: so we can continue with the plan14:03
rlandy|roverquiquell: yeah  I would think admins would have objected if so14:03
rlandy|roveralways complicated14:03
quiquellrlandy|rover: I am going to put all my energy at CI this before we start to implemend stuff14:03
weshay|ruckquiquell, mtg14:03
quiquellyep sorryuy14:03
rlandy|roverabsolutely14:04
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (1 more message)14:08
*** chem has joined #oooq14:12
*** saneax has quit IRC14:20
weshay|ruckchandankumar, paste a link to unit tests14:21
chandankumarweshay|ruck: sharing a link14:22
chandankumarweshay|ruck: https://trunk.rdoproject.org/centos7-master-head/3d/08/3d08aaff7f39b32bb067782c52fa2d3e68f0adf2_88bac5cd/rpmbuild.log14:23
chandankumarweshay|ruck: do a ctrl f14:23
chandankumarweshay|ruck: Executing(%check): /bin/sh -e14:23
*** amoralej|lunch is now known as amoralej14:24
chandankumarweshay|ruck: https://github.com/rdo-packages14:29
chandankumarweshay|ruck: https://github.com/rdo-common -> forked deps spec which screwed us in earlier from fedora/centos land14:29
chandankumarquiquell: weshay|ruck https://blogs.rdoproject.org/2016/07/how-to-build-new-openstack-packages/14:30
*** dsneddon has joined #oooq14:34
chandankumarweshay|ruck: quiquell https://src.fedoraproject.org/group/openstack-sig14:34
*** agopi has joined #oooq14:37
quiquellrlandy|rover: going to mark one of the nodepool image as shared wit the nodepool creds14:38
quiquellrlandy|rover: then we can get them into our tenants14:38
quiquellsuper cheap and easy solution14:38
rlandy|roverquiquell: sounds good14:38
rlandy|roverwhatever works at this point14:38
quiquellYep14:38
*** dsneddon has quit IRC14:39
*** agopi has quit IRC14:42
chandankumarmarios: quiquell weshay|ruck https://www.rdoproject.org/what/trunk-repos/14:42
*** dsneddon has joined #oooq14:44
*** dsneddon has quit IRC14:49
marios"openvswitch " https://www.youtube.com/watch?v=g4ChTL-3c3o14:55
*** chandankumar is now known as chkumar|out15:02
marioso/ folks wdyt http://logs.openstack.org/00/626900/1/check/openstack-tox-docs/af0c893/html/ vs https://docs.openstack.org/tripleo-docs/latest/ simple change at https://review.openstack.org/#/c/626900/ thanks15:04
*** jpena|lunch is now known as jpena15:06
quiquellrlandy|rover: So it worked15:07
quiquellrlandy|rover: shared image from nodepool to my tenant15:07
quiquellrlandy|rover: problem is the naming15:07
rlandy|rovermarios: lol - openvswitch video15:07
rlandy|roverquiquell: nice - good start15:07
*** ykarel is now known as ykarel|afk15:08
rlandy|roverquiquell: we had this naming issue before15:08
rlandy|roverI think maybe downstream15:08
quiquellrlandy|rover: we can search for meta15:08
quiquelland select one of the image15:08
rlandy|roverwe go some tag to specify latest15:08
rlandy|roverI had a ticket out15:08
quiquelland put something at tebroker that share the images15:08
rlandy|roverto rdo admins15:08
rlandy|rovertebroker is gone15:09
quiquellkforde also say something about sharing the same project15:09
rlandy|roverquiquell: I can search for that info15:09
rlandy|roverwe definitely got data to specify latest image15:09
quiquellWell somwwhere wehre we can have the nodepool creds so we can do an "openstack image add project"15:09
rlandy|roverbecause they don't delete the old image15:09
rlandy|roverwhen they add a new one15:09
quiquellThey don't ?15:10
quiquellThen we are good15:10
rlandy|roverthere are a few of them there15:10
rlandy|rovermaybe four15:10
rlandy|rovercentos15:10
rlandy|rovertwo fedora15:10
quiquellDon't  know if they are good enough15:10
quiquellI suppose15:10
quiquellThey will disappear when te broker disappear ?15:11
rlandy|roverwhat's the connection to te-broker15:11
quiquellOr this is not related ?15:11
rlandy|roverthe images are nodepool related15:11
quiquellok ok15:11
rlandy|roverte-broker deals with overcloud nodes15:11
quiquellGive me the uuid of your tenant15:11
rlandy|roverso ...15:11
rlandy|roversec - getting15:11
rlandy|roverquiquell: now in upshift - it's like kforde was mentioning yesterday15:13
rlandy|roverwe have a group tenant15:13
rlandy|roverand everyone in weshay|ruck's org can share it15:13
rlandy|roverso that is the future I think15:13
rlandy|roverthat will make things easier15:13
quiquellmuch easier15:14
quiquellWe cannot have the same at rdocloud ?15:14
*** ykarel|afk has quit IRC15:14
*** dsneddon has joined #oooq15:20
rlandy|roverI think it will be like that soon15:24
rlandy|rover192.168.0.0/24 compute.build_timeout 500 network.build_timeout 500 volume.build_timeout 500 service_available.aodh_plugin False orchestration.stack_owner_role Member15:25
*** dsneddon has quit IRC15:25
rlandy|roverand a rhos-13 error - not so table either15:25
rlandy|roverperiodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset021-rocky - been timing out forever15:35
*** ykarel|afk has joined #oooq15:40
quiquellGo now, to all that are not going to be around next week, have a good christmas and all15:42
*** quiquell is now known as quiquell|off15:43
*** ykarel|afk is now known as ykarel15:43
rlandy|roverweshay|ruck: looking into fs021 timeout in rocky15:48
*** dsneddon has joined #oooq15:51
weshay|ruckrlandy|rover, meh.. it's non-criteria15:52
weshay|ruckrlandy|rover, did fs20 pass?15:52
weshay|ruckrfolco, you ready?15:52
rfolcoweshay|ruck, I am always ready15:53
rlandy|roverweshay|ruck, 20 is still going15:55
weshay|ruckrlandy|rover, those jobs should be the same, however 21 does not use a tempest skip list and pulls in centos CR15:55
rlandy|roverfailed15:55
rlandy|roverweshay|ruck; it consistently times out15:55
rlandy|roverin tempest15:56
rlandy|roverfor ages15:56
weshay|ruckya..15:56
rlandy|rovernot critical - we can discuss later15:56
weshay|ruckso it's failing on a known tempest problem probably15:56
rlandy|roverperiodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky - success15:56
rlandy|roverjust waiting on 1615:57
rlandy|roverweshay|ruck: ^^15:57
rlandy|roverwe may promote something15:58
*** dsneddon has quit IRC15:59
*** kopecmartin is now known as kopecmartin|off16:01
*** rlandy|rover is now known as rlandy|rover|brb16:04
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (1 more message)16:08
*** dsneddon has joined #oooq16:24
mariosttyl folks /me getting ready to go16:25
marioshave a great break see you in few days16:25
*** rlandy|rover|brb is now known as rlandy|rover16:30
weshay|ruckmarios, later man.. take care16:36
weshay|ruckrlandy|rover, rocky is pretty good16:37
weshay|ruckand I don't think they are importing atm16:37
* weshay|ruck just trying to help you prioritize16:37
* weshay|ruck checks16:37
weshay|ruckhttp://rhos-release.virt.bos.redhat.com:3030/rhosp16:37
weshay|ruck2 days old in ooo16:37
rlandy|roverweshay|ruck: here's hpoing16:37
weshay|ruckrlandy|rover, they havent taken a build from upstream in 66 days16:37
rlandy|roverwow16:37
weshay|ruckrlandy|rover, not a huge priority :)16:37
rlandy|roverclearly no urgency16:37
weshay|ruckrlandy|rover, aye16:38
rlandy|roverweshay|ruck: something else that is urgent?16:38
rlandy|roverweshay|ruck: old ovb reproducer looks ok16:38
rlandy|rovertrying libvirt16:38
weshay|ruckok good ;)16:38
weshay|ruckthank you!!16:38
weshay|ruckrlandy|rover, gate looks clean16:38
rlandy|roverso easing rovering with weshay|ruck16:38
* rlandy|rover has little to do16:39
weshay|rucklolz16:39
weshay|ruckrlandy|rover, can't do it now... but ya.. focus is on a master promotion16:39
* weshay|ruck looking at a few things sec16:40
weshay|ruckrlandy|rover, https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset00116:40
weshay|ruckwe'd want to understand what is causing the low success rate here16:40
weshay|ruckrlandy|rover, <hubbot1> FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (116:41
weshay|ruckmore message)16:41
weshay|ruckstill speaks truth16:41
weshay|ruckbut often ignored16:41
weshay|ruckhttps://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset03516:41
weshay|ruckthose are the kinds of things I'd be poking out..16:42
weshay|ruckI can divide and conquer with you16:42
weshay|ruckquiquell|off, did you get the image shared?16:42
weshay|ruckkforde says he showed you how16:42
weshay|ruckrlandy|rover, woot https://review.openstack.org/#/c/626511/ merged16:43
rlandy|rovernice16:46
rlandy|roverweshay|ruck: will start looking through those16:47
weshay|ruckrlandy|rover, we should see gate failures go down to next 2 by EOD http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=116:52
*** chem has quit IRC16:52
rlandy|roverweshay|ruck; hopefully16:54
*** chem has joined #oooq16:54
rlandy|rovernot so bad16:54
*** gkadam has quit IRC16:58
*** gkadam has joined #oooq16:58
*** holser_ has quit IRC17:23
*** jpena is now known as jpena|off17:31
*** derekh has quit IRC17:36
rlandy|rovertripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 still failing17:39
rlandy|roverthat should be passing now17:39
*** trown is now known as trown|lunch17:44
*** ykarel is now known as ykarel|away17:58
*** amoralej is now known as amoralej|off18:05
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-standalone, tripleo-ci-fedora-28-standalone, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb- (1 more message)18:08
*** holser_ has joined #oooq18:10
*** dsneddon has quit IRC18:12
*** dsneddon has joined #oooq18:13
*** amoralej|off has quit IRC18:15
*** trown|lunch is now known as trown18:45
*** holser_ has quit IRC18:50
*** ykarel|away has quit IRC19:09
rlandy|roverugh - back with prep images failures19:30
rlandy|roverweshay|ruck: guess who's baaaaaaccckkk? and I could actually repo this19:31
rlandy|roverhttp://logs.rdoproject.org/90/625890/4/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/6cfbbc1/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz19:31
rlandy|roverintrospection19:32
rlandy|roverpasses - and then the timeout19:32
rlandy|roverprobably hitting 35 an d53 as well19:32
rlandy|rover2018-12-21 11:12:26.508544 | primary | Friday 21 December 2018  11:12:26 +0000 (0:00:03.492)       0:00:26.892 *******19:52
rlandy|roversame deal19:52
rlandy|rover overcloud_prep_images_introspection_debug.pcap.log.txt.gz20:04
rlandy|roverwhat is that?20:04
rlandy|roverweshay|ruck: ^^?20:04
rlandy|roverdoesn;t open20:04
hubbot1FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/ocata: tripleo-ci-centos-7-ovb- (1 more message)20:08
rlandy|roversuspect node cleaning20:26
* weshay|ruck reads20:47
weshay|ruckrlandy|rover, fyi https://review.openstack.org/#/c/626955/20:47
weshay|ruckrlandy|rover, that pcap file is the tcp dump20:47
weshay|ruckyou can import that into wireshark20:47
weshay|ruckrlandy|rover, one way to compare .. well maybe is w/ https://review.openstack.org/#/c/614633/20:48
weshay|ruckhttp://cistatus.tripleo.org/20:49
weshay|ruck■ tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset00120:49
weshay|ruck▼ tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost20:49
weshay|ruckheh.. vexxhost is f'd too https://logs.rdoproject.org/33/614633/11/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master-vexxhost/62ce6e2/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz20:50
weshay|ruckrlandy|rover, ah.. so I suspect rdo-cloud has networking issues20:52
weshay|ruckrlandy|rover, causing 2018-12-21 11:01:34.471912 | primary | TASK [overcloud-prep-images : Prepare the overcloud images for deploy] *********20:52
rlandy|roverweshay|ruck: yep I see that20:52
weshay|ruckwe should see that happen in vexx I think20:52
rlandy|roverweshay|ruck: I think it's node cleaning20:53
weshay|ruckwe need to get these errors defined in sova too20:53
rlandy|roverI am logging a bug20:53
rlandy|roversee ...20:53
rlandy|rover2018-12-21 07:56:02.587 9 INFO ironic.conductor.task_manager [req-fbce19be-f856-48ae-957c-9f914c0c7117 - - - - -] Node 4fb46068-d1e6-4a25-ad0d-3d7c92ca486b moved to provision state "clean failed" from state "clean wait"; target provision state is "available20:53
rlandy|rovervar/log/containers/ironic/ironic-conductor.log.txt.gz shows20:53
rlandy|rover^^20:53
weshay|ruckrlandy|rover, well let's compare w/ vexx20:53
rlandy|roverIn my reproducer I can see the node state20:53
weshay|ruckOH20:53
*** chem` has joined #oooq20:53
* weshay|ruck waves at chem as wishes him a happy new year20:54
rlandy|rover+--------------------------------------+-------------------+---------------+-------------+--------------------+-------------+20:54
rlandy|rover| UUID                                 | Name              | Instance UUID | Power State | Provisioning State | Maintenance |20:54
rlandy|rover+--------------------------------------+-------------------+---------------+-------------+--------------------+-------------+20:54
rlandy|rover| ee506bde-d8be-4701-824b-466b82de56f5 | baremetal-67734-0 | None          | power off   | clean failed       | True        |20:54
rlandy|rover| da989ef3-5553-47f1-b98d-a0f2c12e8965 | baremetal-67734-1 | None          | power off   | clean failed       | True        |20:54
rlandy|rover| 7fcd9095-7f46-45fc-9d2a-c66e0d91af05 | baremetal-67734-2 | None          | power off   | clean failed       | True        |20:54
weshay|ruckah ok20:54
rlandy|rover| 20bc4e02-3776-4f54-aa9f-9a538a361d7a | baremetal-67734-3 | None          | power off   | clean failed       | True        |20:54
rlandy|rover+--------------------------------------+-------------------+---------------+-------------+--------------------+-------------+20:54
rlandy|roverclean failed20:54
rlandy|roveronly in master20:54
rlandy|roverfs00120:54
rlandy|rover03520:54
rlandy|rover05320:54
rlandy|rover35 still has  deploy errors20:54
weshay|ruckya.. what ever is going on it's just master20:54
rlandy|roverwhich shoudl have been fixed :(20:55
weshay|ruckwow.. we may get a merge party in #tripleo20:55
rlandy|roverhappy new year to us20:55
* rlandy|rover will assign the bug to me20:55
rlandy|roverand track there20:55
weshay|ruckk20:56
*** chem has quit IRC20:57
weshay|ruckrlandy|rover, this is the best doc to go by, /me trying again https://docs.google.com/document/d/1i1-_L2mx8_erETAVblchLnymC6-9h7A2x02begy99UQ/edit21:03
rlandy|roverweshay|ruck: yes21:04
weshay|ruckdanka21:04
rlandy|roverI was hoping everyone would update it21:04
rlandy|rovernot sure sure that happened21:04
rlandy|roverI think quique did though21:04
rlandy|roverOVB is still a mystery to me21:04
weshay|ruckhrm.. ok.. well I guess we'll worry about that in January21:05
rlandy|roverweshay|ruck: yeah, need to log off ... hope you have a good holiday21:05
weshay|ruckthanks you too!21:05
*** rlandy|rover has quit IRC21:05
*** trown is now known as trown|outtypewww21:07
*** chem` has quit IRC21:12
*** dsneddon has quit IRC21:19
*** dsneddon has joined #oooq21:34
*** dsneddon has quit IRC21:43
*** dsneddon has joined #oooq21:46
hubbot1FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containers-multinode, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/ocata: tripleo-ci-centos-7-ovb- (1 more message)22:08
-openstackstatus- NOTICE: the gerrit service on review.openstack.org is being restarted to pick up new configuration changes, and will return momentarily22:55
*** dsneddon has quit IRC22:57
*** rascasoft has quit IRC23:09
*** dsneddon has joined #oooq23:32
*** dsneddon has quit IRC23:36
*** dsneddon has joined #oooq23:45
*** dsneddon has quit IRC23:50
« Thursday, 2018-12-20 Index Saturday, 2018-12-22 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!