Friday, 2019-03-08

*** rascasoft has joined #oooq00:02
*** rascasoft has quit IRC00:08
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo- (4 more messages)00:14
*** chem has quit IRC00:30
*** chem has joined #oooq00:34
*** rlandy has quit IRC00:35
*** chem has quit IRC01:32
*** rascasoft has joined #oooq01:34
*** chem has joined #oooq01:36
*** rascasoft has quit IRC01:40
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo- (4 more messages)02:14
*** rascasoft has joined #oooq02:32
*** chem has quit IRC02:35
*** chem has joined #oooq02:40
*** rascasoft has quit IRC02:43
*** apetrich has quit IRC03:16
*** jbadiapa has quit IRC03:24
*** jbadiapa has joined #oooq03:25
*** ykarel|away has joined #oooq03:36
*** ykarel|away is now known as ykarel03:37
*** skramaja has joined #oooq03:49
*** skramaja_ has joined #oooq03:54
*** skramaja has quit IRC03:55
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages)04:14
*** udesale has joined #oooq04:24
*** ratailor has joined #oooq06:09
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages)06:15
*** saneax has joined #oooq06:43
*** jfrancoa has joined #oooq07:05
*** chem has quit IRC07:36
*** chem has joined #oooq07:37
*** ykarel_ has joined #oooq07:45
*** ykarel has quit IRC07:46
*** holser_ has joined #oooq07:49
*** gkadam__ has joined #oooq07:50
*** holser_ has quit IRC08:08
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (3 more messages)08:15
*** holser_ has joined #oooq08:18
*** amoralej|off is now known as amoralej08:24
*** rfolco has joined #oooq08:36
*** rfolco|ruck has quit IRC08:38
*** chem has quit IRC08:41
*** chem has joined #oooq08:46
*** bogdando has joined #oooq09:08
panda|roverphew09:11
*** ccamacho has joined #oooq09:16
*** ykarel_ is now known as ykarel|lunch09:16
panda|rovermarios: need reviews ?09:37
*** ykarel_ has joined #oooq09:43
*** ykarel_ is now known as ykarel09:43
*** ykarel|lunch has quit IRC09:45
*** ykarel is now known as ykarel|lunch09:46
*** rascasoft has joined #oooq10:05
zbr|ssbarneamarios: are you around? I am trying to understand wes CR on distro prefix10:05
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (3 more messages)10:15
*** ykarel|lunch has quit IRC10:15
*** ykarel has joined #oooq10:16
*** ccamacho has quit IRC10:21
marioszbr|ssbarnea: o/ hi10:23
marioszbr|ssbarnea: yeah i had a look earlier too left some questions on it i was hopin you knew more :D10:23
mariospanda|rover: thanks the retag stuff merged so ok for now10:24
zbr|ssbarneamarios: i know that we need to update the prefix, what I do not know is why wes ended up with such a complex process to update that yaml file10:24
zbr|ssbarneai am not saying is wrong, but i wonder if there is no easier way to do it.10:24
panda|roverzbr|ssbarnea: I think he asked alex10:25
panda|roverzbr|ssbarnea: what's the review again ?10:25
marioshttps://review.openstack.org/#/c/641797/510:27
mariospanda|rover: ^10:27
zbr|ssbarneasorry, got a nasty surprise last evening, my NAS died and is out of support, probably the PSU.10:30
*** ccamacho has joined #oooq10:30
panda|roverzbr|ssbarnea: sinology ?10:31
zbr|ssbarneapanda|rover: yeah, I was planning to replace it. so after fixing it, I will probably put on ebay.10:31
zbr|ssbarneapanda|rover: Synology is a nice toy, but remains a toy.10:33
panda|roverzbr|ssbarnea: and you need something serious ? :)10:33
zbr|ssbarneai seen a huge thread on their forum about the PSU issue, seems to be normal.10:33
panda|roverzbr|ssbarnea: I mean , my qnap is probably a toy too, but does everything I need and more, I can even launch vms and containers inside10:34
zbr|ssbarneapanda|rover: sure, I *need* somewhere to keep my TBs of memoirs10:34
panda|roverzbr|ssbarnea: TBs of memoir means taht you're filming youreself 24h a day10:35
zbr|ssbarneapanda|rover: yeah, they are in same category. anyway, I am going fully opensource now, I already have the hardware. I will just move the HDD to it.10:35
panda|rover"to my heirs, these 6 million hours of filming shows everything I did in my life. I give this gift to you now"10:36
zbr|ssbarneai am considering using glusterfs for HA instead of raid as I already have two nodes already running.10:36
panda|roverzbr|ssbarnea: I moved from fully opensource because it's too time consuming and the results will never be as good as something a group of developers wojrk on because it's their job.10:37
zbr|ssbarneapanda|rover: its a bit too much work, but I am seeing it as a training ground. the reality is that I could probably stick an USB disk on one computer.10:37
panda|roverthe only reason I still maintain a separate raspi is because my printer is not compatible with the nas, and prints garbage10:38
panda|roveryes I liked the training for a while, then it became a second job10:39
zbr|ssbarneapanda|rover: not sure about qnap but I am afraid that syno' did cut too many corners in the last years. I would lose to stick with them but is no longer a good bet.10:39
panda|roverespecially when my wife is using it and there are a lot of things to handle to make it user friendly10:39
zbr|ssbarneapanda|rover: yeah... but replacing it every 3years is clearly not cheap...10:39
zbr|ssbarneapanda|rover: my wife is going to kill me if I order a new box.... i would love to get one with 10GBe and a proper CPU.10:41
mariospanda|rover: periodics ok today? trying to see if job_name=periodic-tripleo-centos-7-master-containers-build-push is OK now10:41
mariospanda|rover: i see https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic&job_name=periodic-tripleo-centos-7-master-containers-build-push10:41
mariospanda|rover: BUT10:41
mariospanda|rover: the latest one there is not the new one we merged last night (i.e. https://review.openstack.org/#/c/641348 )10:42
mariospanda|rover: cos it still has 'tripleo-ci-testing' there instead of hash http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/aa09e6d/conf/kolla-build.conf.txt.gz10:42
mariospanda|rover: hoping it will be good on the next run10:42
panda|rovermarios: no OVBs failing on a pacemake problem10:42
mariospanda|rover: but that job hasn't run on 8th yet10:43
panda|rovermarios: mmhh, when did the retag merge ?10:43
panda|rovermarios: it should be showing by now10:43
mariospanda|rover: ah 643 am10:44
mariospanda|rover: like 6 hours ago10:44
mariosi thought it was last night10:44
panda|rovermhh, 6:43 your TZ10:45
mariospanda|rover: right that's why i added the 6 hours ref :D10:45
mariospanda|rover: for you 074310:45
panda|roverwe have a periodic trigger at 9:10 GMT10:45
mariosno10:45
marios044310:45
mariosfor you you are gmt?10:45
panda|roverit should have triggered with the patch ...10:45
marioshm so it should have run10:46
mariospanda|rover: well if that didn't run then nothing else will either10:46
panda|rovermarios: paste me the patch again10:46
marioshttps://review.openstack.org/#/c/641348/10:46
marioshttps://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic&job_name=periodic-tripleo-centos-7-master-containers-build-push10:46
panda|rovermarios: this is the last trigger https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/a033437/10:47
panda|rovermarios: it didn't complete yet10:47
panda|rovermarios: started 1h ago10:47
mariospanda|rover: cool should be done any second now then10:47
mariospanda|rover: and it has the hash good http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-master-containers-build-push/aa09e6d/conf/kolla-build.conf.txt.gz10:48
mariospanda|rover: thanks!10:48
mariospanda|rover: so just delay in reporting then (to https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic&job_name=periodic-tripleo-centos-7-master-containers-build-push10:49
mariospanda|rover: or i shoul d be looking elsewhere?10:49
panda|rovermarios: maybe it reporst whan all the jobs in the pipeline will finish10:53
mariosk thx panda|rover10:55
*** holser_ is now known as holser|lunch10:59
*** udesale has quit IRC11:03
*** udesale has joined #oooq11:04
*** dtantsur|afk is now known as dtantsur11:08
*** ratailor has quit IRC11:17
*** jaosorior has quit IRC11:20
*** jaosorior has joined #oooq11:32
*** panda|rover is now known as panda|rover|lunc11:54
*** gkadam__ has quit IRC12:02
*** gkadam has joined #oooq12:04
*** tosky has joined #oooq12:12
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages)12:15
weshaymarios zbr|ssbarnea join my blue when you have a moment12:18
*** skramaja has joined #oooq12:19
*** skramaja_ has quit IRC12:19
mariosweshay: sure gimme 2 mins12:20
*** holser|lunch is now known as holser_12:21
weshayzbr|ssbarnea you avail?12:22
weshaymarios google hangout?12:24
weshaymaybe12:24
weshaymarios shall we both reboot?12:25
weshayor hang out12:25
weshayI dont see you in hangouts12:25
weshaywhat's ur gmail?12:25
weshayhangout12:26
weshayweshayutin@gmail.com12:26
weshaymarios https://bluejeans.com/11112:46
weshayto test12:46
*** zbr|ssbarnea has quit IRC12:46
*** zbr has joined #oooq12:47
zbrweshay: where to join?12:48
weshayzbr let's chat privately for a minute12:48
mariosweshay: yeah it works fine12:51
mariosi can hear the parrot12:51
marios:D12:51
weshayaye worked for me too :)12:51
weshaytalking to zbr atm12:52
weshaywill ping you in a sec.. there is no scrum today marios12:52
mariosweshay: ack ok12:52
*** holser_ has quit IRC12:57
mariospanda|rover|lunc: isee it reported at https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic&job_name=periodic-tripleo-centos-7-master-containers-build-push13:01
mariospanda|rover|lunc: so does it now mean the periodics are using those new containers?13:01
mariospanda|rover|lunc: i.e. have we seen other hissues yet13:01
marioswith jobs not running/any fails on containers missing etc13:01
rfolcoarxcruz, any clues on this one ? http://logs.openstack.org/95/640895/1/gate/tripleo-ci-centos-7-containers-multinode/f3edfed/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-03-06_19_32_4513:08
rfolcoarxcruz, tempest failed on a gate job13:08
rfolcopanda|rover|lunc, containers-multinode job is failing for multiple reasons on gate. I'm having hard time to open a bug13:09
*** skramaja has quit IRC13:12
weshaymarios k.. ready13:15
zbrweshay: https://logs.rdoproject.org/55/19155/3/check/periodic-tripleo-ci-fedora-28-scenario003-standalone-master/df537e3/job-output.txt.gz#_2019-03-07_17_30_31_98874213:17
mariosweshay: k gimme 2 joining13:18
*** agopi has joined #oooq13:19
*** panda|rover|lunc is now known as panda|rover13:25
marioszbr: irssi13:26
panda|rovermarios: most probably yes13:31
panda|roverrfolco: wha are you finding difficult ?13:31
rfolcopanda|rover, multiple root causes for the same job failing on gate13:32
*** rlandy has joined #oooq13:32
rfolcopanda|rover, one bug for each?13:32
arxcruzweshay: shall I wait a little bit more ?13:32
panda|roverrfolco: yes13:32
marioshttps://github.com/openstack/python-tripleoclient/blob/ea03eb966fa89ad47962115b788be102516c41e2/setup.cfg#L10413:38
*** derekh has joined #oooq13:53
weshaypanda|rover can we merge https://review.rdoproject.org/r/#/c/19151/13:56
weshayrlandy ^13:57
rlandylooking13:58
rlandyweshay: ack - can merge that if ready - harmless13:59
zbrrlandy: panda|rover : please help merging https://review.rdoproject.org/r/#/c/19151/13:59
rlandyzbr; done13:59
zbrthanks!13:59
panda|roverweshay: why are we depending on the centos counterparts ?14:01
*** ccamacho has quit IRC14:01
*** amoralej is now known as amoralej|lunch14:03
*** vinaykns has joined #oooq14:08
*** rfolco is now known as rfolco|ruck14:08
panda|roverzbr: ?14:08
panda|roverzbr: why are we parenting the scenario jobs in fedora to the scenario jobs in centos ?14:09
zbrpanda|rover: to reuse all their configuration, only thing different is the nodeset.14:09
zbrpanda|rover: we want to avoid accidental divergence of config.14:09
rlandypanda|rover: thought the same thing but the overrides are so complex in those jobs, I understand why14:10
zbrpanda|rover: and as you know, zuul has no multiple-inheritance,....14:10
zbrdoing it the other way around would create a huge amount of cfg duplication14:11
rlandypanda|rover: zbr: the bigger question is the duplication of tripleo-ci and rdo jobs definitions in centos14:12
rlandybut since those already exist, adding the fedora ones is less damage imho14:12
panda|roverzbr: can you show me an example of override we need and add the explanation o the commit ?14:14
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (3 more messages)14:15
panda|roveroh wait14:26
panda|roverthose are parents not dependencies14:26
panda|roverI don't know if it's better to avoid duplication or avoid going up a hieradchy of 10 parents to understand what a job is doing :/14:28
jaosoriorby the way, FYI, all ovb jobs are still broken due to https://bugs.launchpad.net/tripleo/+bug/181910514:32
openstackLaunchpad bug 1819105 in tripleo "IPv6 value in httpd's proxy_ips is breaking horizon" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)14:32
panda|roverjaosorior: so the haproxy bus is solved ?14:33
jaosoriorwe should either push to get the fix merged in puppetlabs-apache or pin it14:33
*** irclogbot_0 has joined #oooq14:33
jaosoriorpanda|rover: yeah; the issue was that pacemaker wasn't being configured due to a change in tripleo-common.14:33
* jaosorior talking about master14:33
*** ykarel is now known as ykarel|away14:40
panda|roverykarel: is pinning puppetlabs-apache an acceptanle solution14:41
panda|rover.... and of course my timing14:41
panda|roverwe'll never know.14:41
ykarel|awaypanda|rover, ?14:42
mariosweshay: got a beaker box that's hitting that  domaintype=kvm thing fyi non virtualised14:43
panda|roverykarel|away: you're baaaack14:44
jaosoriorpanda|rover, ykarel|away: For reference, the commit that caused the horizon issue was https://github.com/puppetlabs/puppetlabs-apache/commit/1503035d788f9fdaa2bab9cdaabfb66e858c2c39#diff-c2ea3c67760696a0d67bab9fb81757c614:44
mariosweshay: added a note at https://tree.taiga.io/project/tripleo-ci-board/task/766 under issue 1 and attached a trace issue1trace.txt (box expires in 35 hours)14:44
jaosorioralthough I did mention it in the comments of the launchpad bug14:44
ykarel|awaypanda|rover, if merging puppet-apache patch is taking time to merge, you can propose pinning to rdoinfo to the version which has the fix14:44
mariosweshay: if you like i can add yr keys if you wana poke you said to tell you if i got one14:44
ykarel|awayjaosorior, btw what i checked, horizon was not running earlier14:45
jaosoriorykarel|away: it wasn't. We used to have a defined set of services for ovb14:45
jaosoriorand that was removed14:45
jaosoriorso we started running horizon14:45
jaosoriorsince it's in our defaults14:45
ykarel|awayjaosorior, ya that i was mentioning14:45
ykarel|awayjaosorior, yes good to pin it in rdo if it's taking time to merge upstream14:46
ykarel|awayto the good commit14:46
ykarel|awayi heard emilien was going to ping someone to get that merged14:46
* ykarel|away need to leave, you can ping on #rdo, someone else can take care of any action needed in rdo side14:48
jaosorioryeah, I gotta go pretty soon too14:48
weshaymarios ya.. let me see w/ tmate14:51
weshayplease14:51
mariosweshay: sec installing tmux14:53
weshaypanda|rover rfolco|ruck how are we doing w/ fs001 /35 ovb jobs... I believe Juan has a patch merged.. re: https://bugs.launchpad.net/tripleo/+bug/1818994  are there additional blocking issues?14:53
openstackLaunchpad bug 1818994 in tripleo "ovb jobs broken because pacemaker is unconfigured" [Critical,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)14:53
rlandyugh - hardcoded assumptions ... https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/ci-scripts/tripleo-upstream/dlrnapi_report.sh#L1314:53
panda|roverrlandy: and you probably need an ansible variable there14:55
jaosoriorweshay: read a little bit up14:55
rlandypanda|rover; yep - story of my downstream life14:55
panda|roverweshay: were just talking about that, and after thet there's another one14:55
panda|roverand there we're done until the next bug.14:56
rfolco|ruckweshay, ready for 1-1 or need more time?15:00
weshayrfolco|ruck need 5min15:01
weshaythanks15:01
rfolco|rucknp15:02
*** jfrancoa has quit IRC15:11
*** amoralej|lunch is now known as amoralej15:14
weshaymarios++15:14
hubbot1weshay: marios's karma is now 815:14
weshayrfolco|ruck ready15:14
rfolco|ruckk15:15
rlandypanda|rover: wdyt ...https://review.rdoproject.org/r/#/c/19208/15:16
mariosweshay: i just downloaded untarred and ran a script man :D you found the issue !15:17
mariosweshay++15:17
hubbot1marios: weshay's karma is now 1715:17
*** udesale has quit IRC15:21
*** udesale has joined #oooq15:21
panda|roverrlandy: I think it'll never be soon enough when we remove those bash scripts :( The patch looks fine anyway. I would have put the default in the bash script itself, but doesn't change really much, maybe except for the fact that you have to specify the default only in one place.15:23
rlandypanda|rover: the bash script current don't have any ansible vars so I stuck with that ... will test this out with testproject before I inflict it on your periodic pipeline15:24
rlandyoh it's config15:25
rlandynvm15:25
rlandypanda|rover: so - in your hands - pls merge it when you are ready for the risk on your periodic pipeline15:27
panda|roveroh right, why am I already married ? I could use some of the quique's strategy here.15:29
mariospanda|rover: any fallout from containers-build-push?15:31
marios(yet_15:31
rlandypanda|rover: you could have another child - that gets you a month off15:31
* marios stops harrassing panda|rover goes to grafana sorry 15:31
mariosdamn we don't have containers-build-push in http://cistatus.tripleo.org/promotion/15:32
mariossshnaidm|off: ^15:32
panda|roverrlandy: and two years without sleep.15:33
panda|roverrlandy: merging, lloks ok, checked 3 times from a different angle15:33
panda|roverrlandy: so it will only fail for two days proably.15:33
rlandypanda|rover: well now you have merge rights, you can revert it if I messed up15:33
rlandy2019-03-08 03:10:08,202 13624 DEBUG    promoter periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master at 2019-03-04T12:07:14, logs at https://logs.rdoproject.org/59/164359/1/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/c45c11415:34
rlandywe are reporting to dlrn15:34
rlandyjust with the wrong logs15:34
mariosweshay: fyi wip at https://review.rdoproject.org/r/19201 gonna likely need some work upstream tipleo-ci if we missed something wrt the branching but we don't have taiga for it so not sure where to track15:45
mariospanda|rover: rfolco|ruck is for next sprint right? ^15:45
*** bogdando has quit IRC15:45
mariosweshay: panda|rover rfolco|ruck also i think we can delete the existing ones now ? https://review.rdoproject.org/r/#/c/19192/ wdyt?15:47
panda|rovermarios: so you're not happy having done the task for this sprint, you started the tasks for the next ?15:48
mariospanda|rover: well not done yet cos we dind't see promote jobs all good but yeah i mean what else am gonna do and also i made a promise to quiquell that i would not forget that15:48
mariosN E V A R  F O R G E T15:48
mariospanda|rover: i'll get excited about the containers build working well once we see some good promotion runs15:49
mariosultimately once we get a master promotion...15:50
rlandymarios; once you are done with your work, I still have a review from quique to put in https://review.rdoproject.org/r/#/c/19030/7/zuul.d/jobs.yaml15:50
rlandywe just didn;t want to disturb your work15:50
rlandypanda|rover: ^^ fyi15:50
mariosrlandy: waaa /me looking15:51
rlandywill discuss at monday's meeting15:51
rlandymarios: ^^15:51
mariosrlandy: we adding a new secret for delorean to use?15:51
mariosrlandy: kk i thought you meant 'about to merge this fyi'15:51
rlandywe did that already15:51
rlandyfor downstream15:51
mariosyeah i see it being used there https://review.rdoproject.org/r/#/c/19030/7/zuul.d/jobs.yaml is what i mean15:51
rlandymarios: no - I am not that merge happy15:51
mariosrlandy: k cool lets talk monday and i'll also review it then15:52
mariosoh15:52
marioswell15:52
mariosi'm out monday15:52
*** dtantsur is now known as creepy_owlet15:52
mariosrlandy: :/ i'll catch up on it tuesday anyway once you all discuss15:52
rlandymarios: k - no major rush on this15:52
mariosrlandy: thanks15:52
panda|rovermarios: once again unto the branch, my friend15:57
*** ccamacho has joined #oooq15:58
mariospanda|rover: qe16:00
panda|rovermarios: I can't I'm rover16:00
mariosk16:00
marios:(16:00
panda|rovermarios: commented your patch16:02
mariospanda|rover: thanks which one16:02
mariospanda|rover: see it thanks for checking yeah i mean its wip so not planning on merging it yet :D16:03
zbrpanda|rover: you asked me something a couple of hours ago but i didn't get the full context.16:03
panda|roverzbr: don't worry, all good16:05
panda|roverwhatever it was16:05
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (3 more messages)16:15
rlandyweshay: marios: (other standalone gurus) ... does standalone config allow for setting ntp servers like https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/standalone/templates/standalone_config.yaml.j2#L916:21
rlandyundercloud_undercloud_ntp_servers16:22
mariosrlandy: well we can pass that in16:22
mariosi mean DnsServers16:22
mariosin the environment file?16:22
marioslike https://github.com/openstack/tripleo-heat-templates/blob/5ff45134ac0771b267cbda6be50b9f2190467328/ci/environments/scenario001-standalone.yaml#L4216:22
mariosrlandy: ^16:22
mariosthat what you mean?16:23
rlandymarios; I need to NtpServers16:23
rlandyis that an acceptable param?16:23
mariosrlandy: sec (yeah pretty sure there is a tripleo heat templates param for setting ntp servers sec)16:24
mariosNtpServer:16:24
rlandyah - no 's'16:24
mariosrlandy: eg https://github.com/openstack/tripleo-heat-templates/blob/5ff45134ac0771b267cbda6be50b9f2190467328/environments/standalone/standalone-tripleo.yaml#L4116:24
weshaymarios sorry back.. were you able to console in?16:25
mariosweshay: no don't think i can via beaker and didn't try drac16:26
weshaymarios zbr woot.. MERGED https://review.openstack.org/#/c/641797/16:27
mariosbut the box is going to die in 35 hours like its from a recipe so sure we can extend but it will go away at some point16:27
weshaymarios drac would be the only way16:27
zbrweshay: yep, i know, i already retriggered the periodic ones.16:27
weshaybut you could reserve a new box w/ the filter for those flags16:27
weshaynice16:27
mariosweshay: sec let me try running the extend time script16:27
mariosso we have i ton monday16:27
weshayk16:27
mariosits only a small box we're not keeping some major resource only 4 gb ram if i recall16:28
weshayand let me share a drac console trick16:28
weshayhttps://github.com/DomiStyle/docker-idrac616:28
weshaymarios docker run -d -p 5800:5800 -p 5900:5900 -e IDRAC_HOST=rhev-i8c-02-drac.mpc.lab.eng.bos.redhat.com -e IDRAC_USER=foo -e IDRAC_PASSWORD=bar domistyle/idrac616:29
rlandyrfolco|ruck: marios: fix for periodic reproducer https://review.openstack.org/#/c/641753/16:29
*** vinaykns has quit IRC16:30
weshayzbr so your patch will fail16:31
weshaythe jobs in check will attempt to use docker.io16:31
weshaypanda|rover actually if you have a sec.. I wanted to chat w/ you about that16:31
weshayre:  periodic jobs that trigger in check16:32
weshayand which registry and hash they should use16:32
mariosweshay: thanks noting16:33
*** gkadam has quit IRC16:33
weshaymarios zbr fyi.. docker-registry.default.svc:5000/tripleomaster/fedora-binary-barbican-keystone-listener16:34
mariosweshay: ?16:34
weshayconfirmed fedora containers pushed to rdo registry16:34
weshay$ sudo docker pull trunk.registry.rdoproject.org/tripleomaster/fedora-binary-barbican-keystone-listener:a13151c3c151d051e2bffb848265c93609995d64_9ef9aea416:34
weshaylike Pregu... it's IN THERE16:35
weshayhttps://www.youtube.com/watch?v=2J87QekxQVI16:35
mariosweshay: you're not lying ! https://paste.fedoraproject.org/paste/79OqPt~A0JrSRvGogF6CYA16:36
weshaypanda|rover before your day ends please hand off ovb fs001/35 to myself and rfolco|ruck16:38
mariosrlandy: wantt me to merge https://review.openstack.org/#/c/641753/216:38
rlandymarios: sure16:38
weshayzbr if you can figure out to trigger jobs named periodic to use the rdo-registry.. that would be neat16:38
zbrsure, will look into.16:39
rlandy2019-03-08 16:27:30.957946 | primary |   "url": "https://sf.hosted.upshift.rdu2.redhat.com/logs/81/164881/1/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/5420f57"16:39
rlandyreporting properly now16:39
weshayzbr and your patch is still in the mix some how http://logs.rdoproject.org/55/19155/4/check/periodic-tripleo-ci-fedora-28-scenario001-standalone-master/86f071d/logs/undercloud/home/zuul/standalone_container_prep.log.txt.gz16:39
weshayzbr you should drop that one16:40
zbrweshay: i seen this few minutes ago but i have no idea why the depends is on your merged job.16:41
weshayzbr ya.. kill it16:43
mariosweshay: you doing something on that box ? cant ssh it16:44
mariosweshay: did you reboot it?16:44
mariosweshay: want to get in to run extend time cos i'm on my way out16:44
mariosweshay: if you're on it can you run extend time otherwise it will leave us tomorrow16:44
weshaynope.. and I'll log out if needed16:45
mariosweshay: k16:45
weshaymarios I also lost access16:45
mariosweshay: lemme reboot it via beaker then16:45
weshaymarios you may have lost the res16:45
mariosno it had like 30 hours16:45
panda|roverweshay: nothing much to pass on, they are currently failing for https://bugs.launchpad.net/tripleo/+bug/1818994. But it has since been fixed. Next runs will fail with https://bugs.launchpad.net/tripleo/+bug/1819105 which will be fixed by a pin in puppetlabs-apache or when the fix proposed will merge there16:50
openstackLaunchpad bug 1818994 in tripleo "ovb jobs broken because pacemaker is unconfigured" [Critical,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)16:50
openstackLaunchpad bug 1819105 in tripleo "IPv6 value in httpd's proxy_ips is breaking horizon" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)16:50
weshaymarios to prevent that issue https://review.openstack.org/64207516:50
weshaymaybe we need to add that to the pre.yaml as well16:51
mariosweshay: nice16:51
weshaymarios zbr this is a bug 2019-03-08 15:14:38 | requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://trunk.registry.rdoproject.org/v2/tripleomasterF28/centos-binary-cinder-api/manifests/tripleo-ci-testing16:52
weshayhttps://trunk.registry.rdoproject.org/v2/tripleomasterF28/16:52
* weshay patches16:52
weshaymarios zbr https://review.openstack.org/#/c/642076/1/config/release/tripleo-ci/Fedora-28/promotion-testing-hash-master.yml16:55
zbrweshay: where can I get credentials for rdo registry? getting 401 errors does not help much for debugging.16:56
weshayzbr what do you need to do w/ the registry?16:57
weshayI don't have the access to give them16:57
zbrread access/ browse16:57
marioszbr: shoudln't need can use skopeo like http://pastebin.test.redhat.com/73319016:57
marioszbr: yum install skopeo and then ^16:57
*** ccamacho has quit IRC16:57
* marios on way to door16:58
weshayl8rs16:58
marioshave a good weekend folks16:58
weshayhave a good wkend16:58
weshayrlandy panda|rover https://review.openstack.org/#/c/642076/116:58
zbrso we do not have an api to query, where I can see exactly what namespeces are defined16:58
weshayzbr same as in docker.io16:59
weshaytripleomaster16:59
weshaytripleopike16:59
weshaytripleoqueens16:59
weshaytripleorocky16:59
zbrweshay: yeah, installing skopeo now.16:59
weshayzbr https://nimb.ws/XELjlT16:59
zbrweshay: that was what I was looking for but apparently my github account is not in the right group. I got a "An authentication error occurred. "17:01
rlandypanda|rover: so now I really have to solve the problem of passing vars to toci (that would override those defined on config/testenv )17:02
rlandyie; not using fs override17:02
* rlandy can hack it - but that is probably bad17:03
panda|roverrlandy: for example ?17:04
panda|roverrlandy: we'll probably need to add envs for downstream ..17:04
rlandypanda|rover: ck - to run anything downstream17:04
panda|roverrlandy: and there you parent the job overriding the values ?17:04
zbrofftopic: does anyone use gertty with multiple gerrit servers? i somehow fail to make it work with rdo.17:04
*** udesale has quit IRC17:04
panda|roverzbr: I do17:05
rlandypanda|rover: depends on the situation - with baremetal, I defined new env type17:05
rfolco|ruckpanda|rover, are we pinning puppetlabs-apache until the PR merges ?17:05
rlandypanda|rover: but with standalone, I really just need  new ntp. dns server vars17:05
rlandyI can create a playbook to add those vars on the fly17:05
panda|roverrfolco|ruck: at this point, no. Looks like Emilien knows someone that knows someone taht knows someone that could merge it quickly17:06
zbrpanda|rover: any magic involved? i added rdo as the second server. but i see it only making some garbage queries on it, like asking about projects from openstack one. it did never showed me my reviews from rdo. also i had trouble getting the password from rdo.17:06
rlandy^^ which is what I am going to do but needs to be solved in the long term17:06
zbrpanda|rover: the get http password option on rdo is broken. it never generates a token.17:07
panda|roverzbr: you sure ?17:08
panda|roverzbr: ah yes, you're lookign at the wrong place17:09
zbri wonder if I should but the github api key there instead.17:09
panda|roverzbr: they have a wrapper around the pages, and the password generator is in a dedicated settings section17:09
zbrgithub pwd is clearly not an option as I have 2FA enabled.17:09
zbrpanda|rover: aha! thanks! obvious, not.17:10
*** rlandy is now known as rlandy|brb17:12
ykarel|awayzbr possibly you are looking for https://lists.rdoproject.org/pipermail/users/2019-February/000481.html17:12
zbrykarel|away: yeah, sorted this issue. thanks17:13
panda|roverzbr: also what are you using as url endpoint ?17:13
ykarel|awayack17:13
zbrpanda|rover: https://review.rdoproject.org/17:14
panda|roverzbr: http://paste.openstack.org/show/747460/17:18
weshaypanda|rover can you jump on my blue for a minute17:19
panda|roverweshay: I'm there.17:20
weshaypanda|rover /me sees http://logs.rdoproject.org/09/640709/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/95b0332/logs/bmc-console.log17:22
weshaypanda|rover https://review.openstack.org/#/c/639620/17:23
weshaypanda|rover https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001&branch=master&result=success17:24
rfolco|ruckweshay, can I join you (assume you're talking about fs001)17:25
weshayrfolco|ruck aye jump in17:26
*** creepy_owlet is now known as dtantsur|afk17:29
panda|roverweshay: https://bugs.launchpad.net/tripleo/+bug/181910517:33
openstackLaunchpad bug 1819105 in tripleo "IPv6 value in httpd's proxy_ips is breaking horizon" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles)17:33
panda|roverweshay: https://github.com/puppetlabs/puppetlabs-apache/pull/189117:33
panda|roverweshay: https://logs.rdoproject.org/60/641660/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035/3529e09/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2019-03-08_03_53_0217:36
*** derekh has quit IRC17:44
*** panda|rover is now known as panda|rover|baby17:48
*** agopi is now known as agopi|afk17:50
zbrpanda|rover|baby: thanks for the file,.... i think i am getting insane with gertty config. it seems to be imposible to make it use the 2nd server defined there, I am sure it does load the config because it complains if I put something wrong there but somehow it always connects only to 1st intance. it does not generate any output files for the 2nd instance.17:50
zbrmy git dirs are the same, but this should not matter, i hope.17:51
*** rlandy|brb is now known as rlandy17:51
rfolco|ruckpanda|rover|baby, lol17:56
panda|rover|babyzbr: I'm using a small wrapper taht passed the configuration and then I have one dir for each endpoint, but some configuration remains shared, like the macros17:58
panda|rover|babyzbr: and you have to specify the name of the endpoint to use taht endpoint17:58
panda|rover|babyzbr: so one instance, one endpoint17:58
zbrpanda|rover|baby: i was able to get some progress and I think... its py3 support is flaky17:58
zbrand the fact that it does not really supports multiple servers sucks big time17:59
zbrit was hopping to see all reviews in one place.17:59
panda|rover|babyzbr: I keep two terminals open .. no big deal17:59
zbri have 5 gerrit servers... true that 99% of time i need only two.18:00
weshayrfolco|ruck https://github.com/redhat-openstack/rdoinfo/blob/master/tags/stein.yml#L55318:02
weshayexample of how to pin https://github.com/redhat-openstack/rdoinfo/commit/314a275931cd8395b8663d210d38108d333d9eb418:03
rfolco|ruckweshay, taking notes18:03
rfolco|ruckweshay, I see some ocata jobs in rdo-jobs... is this the right place ?18:04
weshayaye18:04
zbrpanda|rover|baby: it is so, so stupid, i get HTTP 401 from the server but unless I enabled debug mode i was not able to see any msg related to that.18:07
rfolco|ruckweshay, which you prefer? move ocata jobs to a zuul.d/deprecated.yaml w/ branches: none or simply delete them?18:08
rfolco|ruckbranches: none will turn it off18:09
weshaydon't know to be honest18:11
weshayif there is precedent to move to deprecated follow the precedent18:12
rfolco|ruckweshay, we did this upstream18:12
rfolco|ruck(moved to deprecated.yaml)18:12
rfolco|ruckweshay, https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/deprecated-jobs.yaml18:13
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages)18:15
*** saneax has quit IRC18:22
*** ykarel|away has quit IRC18:22
zbrpanda|rover|baby: when you have time, please tell me if this works for you. I get 401 but on browser I get 200. using the API key.18:30
zbrhttps://gist.github.com/ssbarnea/1d911cf012421546b7d55491f9a3829318:31
*** amoralej is now known as amoralej|off18:44
*** vinaykns has joined #oooq18:52
*** rfolco|ruck is now known as rfolco|ruck|brb19:11
*** jbadiapa has quit IRC19:59
*** irclogbot_0 has quit IRC20:13
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010 (4 more messages)20:15
*** rfolco|ruck|brb is now known as rfolco|ruck20:21
*** agopi|afk is now known as agopi21:03
rlandyweshay: hello - update21:17
weshayrlandy ahoy21:18
rlandyweshay: firstly enabled overcloud-prep on rocky and master bm if you need to compare introspection the logs are there21:18
rlandyweshay: secondly ... https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/6a6b486d0134412faffea2da5e5ea403?logfile=console.log - that's standlone21:19
weshayok cool.. does rocky run through successfully21:19
rlandycentod21:19
rlandyyep21:19
* weshay looks21:19
weshaymaster work?21:19
weshayit may21:19
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/94/164894/1/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-rocky/7f67b74/job-output.txt.gz21:19
rlandyrocky: ^^21:19
rlandymaster also works21:19
weshayah.. rocky got all the way through21:19
* rlandy not running overcloud deploy - just overcloud prep21:19
weshayw/ tempest :)21:19
rlandymaster as well21:19
rlandystandalone - yes21:20
rlandyall the way through21:20
rlandyso we are ready to try rhel821:20
*** jfrancoa has joined #oooq21:20
* rlandy wanted something to compare to downstream before we started21:20
weshayoh boy21:20
weshayhrm.. ok.. my understanding it's still very rough21:20
weshayhowever21:20
rlandyweshay; fine - no rush -21:20
weshaydo you have time to chat for a minute?21:21
rlandyjust updating you on where we are21:21
rlandysure21:21
rlandyjoining your bj21:21
*** jfrancoa has quit IRC21:28
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-fedora-28-standalone, (4 more messages)22:15
*** rlandy has quit IRC22:19
*** jtomasek has quit IRC22:34
*** vinaykns has quit IRC23:00
*** irclogbot_0 has joined #oooq23:01
*** rascasoft has quit IRC23:10
*** agopi has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!