Thursday, 2020-01-16

*** ysandeep has joined #oooq00:05
zbrrlandy: weshay|ruck https://review.opendev.org/#/c/702735/200:36
*** skramaja has quit IRC02:04
*** raukadah has quit IRC03:31
*** chandankumar has joined #oooq03:33
*** bhagyashris has joined #oooq03:48
*** udesale has joined #oooq04:05
*** rlandy has quit IRC04:07
*** ysandeep has quit IRC04:14
*** ykarel|away has joined #oooq04:35
*** ykarel|away is now known as ykarel04:37
*** ysandeep has joined #oooq05:00
*** ysandeep has quit IRC05:24
*** surpatil has joined #oooq05:27
*** akahat has joined #oooq05:37
*** ratailor has joined #oooq06:08
*** saneax has joined #oooq06:18
*** jtomasek has joined #oooq06:25
*** marios has joined #oooq06:32
*** jfrancoa has joined #oooq07:02
*** soniya29 has joined #oooq07:19
mariosmatbu: looks like it worked http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-standalone-upgrade-stein&job_name=tripleo-ci-centos-7-standalone-upgrade-train07:21
mariossshnaidm: ^07:21
marios(  it https://review.opendev.org/#/c/702625/ )07:21
sshnaidmmarios, great07:22
sshnaidmmarios, if you want to run it on master of non-branched repos, we'll need a different job though07:22
mariossshnaidm: hmmm not sure if we want it voting on master. definitely it should block train and not let it break upgrads on train but i think for master we never enforce voting upgrade jobs cos upgrades expected to break while new features are developed07:28
mariosmatbu: wdyt ^07:28
mariossshnaidm: we've always had pushback when trying to make upgrades vote on master07:28
sshnaidmmarios, I mean not upgrade to master, but running the same job on tripleo-ci, quickstart or other non-branched repos07:29
sshnaidmmarios, for example to check if we don't break it with some patch on tripleo-ci07:29
sshnaidmmarios, but not sure it's so relevant today07:29
mariossshnaidm: you mean, to run standalone-upgrade-train on tripleo-quickstart, we need a different job? not sure i follow the issue but maybe irc isn't best for that... we can discuss later in scrum perhaps. i se some q/e and quickstart there though http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-standalone-upgrade-stein&job_name=tripleo-ci-centos-7-standalone-upgrade-train07:31
sshnaidmmarios, yeah, we do need a different job, because tripleo-quickstart runs on master and master is not included in job branches. But maybe that's fine and we don't need to run it there07:33
mariossshnaidm: yeah i see what you mean now. like we explicitly set the branch and it doesn't include master for -train or -steinjobs07:43
*** ykarel is now known as ykarel|lunch07:56
matbumarios: hey07:57
matbumarios: yep agree, i should block only train07:57
mariosmatbu: o/ k there is slightly different problem. the -train job won't run on quickstart/extras or tripleo-ci. we need somethign further if we want that07:58
matbuand not be run on non-branched repo07:58
mariosmatbu: i.e. won't run on any of the non-branched repos07:58
mariosright07:58
matbumarios: yep not sure its really need ?07:58
matbusince we have a master job that runs on those non branched repos07:59
mariosmatbu: yeah me too. i mean it would be nice. it means you get what you want WRT not breaking the upgrade process in changes to tripleo project deployment tools07:59
mariosmatbu: but we might break the job with some change in tripleo-ci role for example07:59
matbuanyway thank you sshnaidm and marios for fixing this07:59
mariosmatbu: yeah ack on the master job though... but as you know its non voting thus ignored ;)07:59
matbumarios: yep, i think we probably need it (standalone-upgrade) voting on master and undercloud containerized upgrade as well08:00
mariosmatbu: whenever we tried that we got push back but bring it up on next weekly tripleo irc08:00
mariosor whever else08:00
matbumarios: we lost a lot of time fixing UC upgrade before the shutdown08:00
mariosmatbu: i mean now is acceptable to make it voting. i think as the cycle matures it might break. its a long conversation we've had it many times ;)08:01
mariosmatbu: i mean you me and many others have talked about that08:01
*** jpena|off is now known as jpena08:01
*** bhagyashris is now known as bhagyashris|NHO08:03
matbumarios: hehe yep08:05
*** tesseract has joined #oooq08:10
matbuhm why im unregistered08:15
* matbu reconnect08:15
*** matbu has quit IRC08:15
*** matbu has joined #oooq08:15
*** surpatil has quit IRC08:22
*** udesale_ has joined #oooq08:25
*** amoralej|off is now known as amoralej08:26
*** udesale has quit IRC08:28
*** tosky has joined #oooq08:32
*** surpatil has joined #oooq08:34
*** dmellado has quit IRC08:35
*** dmellado has joined #oooq08:37
*** ykarel|lunch is now known as ykarel08:38
*** bogdando has joined #oooq08:50
*** ykarel is now known as ykarel|afk09:03
*** marios has quit IRC09:04
*** udesale_ has quit IRC09:04
*** ccamacho has joined #oooq09:29
*** apetrich has joined #oooq09:39
*** marios has joined #oooq10:05
*** zbr is now known as zbr|rover10:09
marioszbr|rover: any idea why that rdo-tox-molecule is running rdo-fedora-28 http://logs.rdoproject.org/52/24552/1/check/rdo-tox-molecule/fecc567/job-output.txt did we define it that way and do you know why it fails10:35
mariosERROR:   molecule: could not install deps10:35
*** soniya29 has quit IRC10:36
*** soniya29 has joined #oooq10:36
*** ykarel|afk is now known as ykarel10:36
zbr|rovermarios: i know, docker needs. but it can be upgraded to newer OS.10:37
*** udesale has joined #oooq10:38
zbr|rovermarios: mainly we need docker-ce with molecule, as old one is too buggy. 28 was picked for convenience at that time but is not a hard req10:38
zbr|roverany fedora newer or centos-8 will do the trick too.10:39
zbr|rovernow that we have centos-8 it would worth making the change.10:39
zbr|roverjust don't try centos-7, please.10:39
*** soniya29 has quit IRC10:41
*** soniya29 has joined #oooq10:42
*** soniya29 has joined #oooq10:43
*** d0ugal has quit IRC10:46
*** d0ugal has joined #oooq10:46
mariossshnaidm: we have a problem re https://review.opendev.org/#/c/702625/1/zuul.d/standalone-jobs.yaml10:47
marios12:44 <@jistr> marios: thanks, so on stein we should only have upgrade-stein?10:47
marios12:44 < marios> jistr: checking puppet-tripleo layout sec10:47
marios12:45 < marios> jistr: ah damnit ... i see why now. so yeah jistr we need in stein only the stein job10:47
marios12:45 -!- gfidente [~gfidente@ovpn-116-26.ams2.redhat.com] has quit [Quit: quit]10:47
marios12:46 < marios> jistr: problem is https://review.opendev.org/#/c/702625/1/zuul.d/standalone-jobs.yaml10:47
marios12:46 < marios> matbu: ^^ ;D10:48
marios12:47 < marios> jistr: they now all match so we get master/train/stein. we did that ^^ because before it wasn't matching and we were not running stein or train anywhere10:48
mariosso now we are getting master/stein/train running in https://review.opendev.org/#/c/702696 for stein10:48
sshnaidmmarios, I see10:49
sshnaidmmarios, so tripleo-ci-centos-7-standalone-upgrade should run on master only, right?10:50
mariossshnaidm: yeah like the master/train upgrade is useless for changes to stein10:50
sshnaidmmarios, I'd try to remove "branch" in tripleo-ci-centos-7-standalone-upgrade and see what happens10:51
mariossshnaidm: hmm (default master(10:51
sshnaidmmarios, idk, maybe it will work with implicit..10:51
*** d0ugal has quit IRC10:52
sshnaidmmarios, other option - just to define 3 different jobs, w/o inheritance and tie each one to its branch explicitly10:52
marioszuul config moved oh noe 404 https://zuul-ci.org/docs/zuul/user/config.html10:52
mariosanyone know?10:52
sshnaidmmarios, https://zuul-ci.org/docs/zuul/ ?10:53
mariossshnaidm: thanks :D10:53
*** d0ugal has joined #oooq10:55
sshnaidmmarios, I read branches section there multiple times, but not much helped me :D10:56
*** dsneddon has quit IRC11:03
mariossshnaidm: i find the explanation quite convoluted not very clear to me11:13
mariossshnaidm: seems it matter when trusted or not11:13
mariossshnaidm: and i need to read it  few more times ;)11:13
mariossshnaidm: thanks i will chekc your reviews too.. they ping me from upgrades team is why i was looking there i.e. now all of the jobs are running for stein and unnecessary11:13
sshnaidmmarios, yeah, it's most vague section there11:14
*** udesale has quit IRC11:14
*** surpatil has quit IRC11:31
*** chandankumar has left #oooq11:33
*** dsneddon has joined #oooq11:33
*** chandankumar has joined #oooq11:40
*** panda is now known as panda|drappt11:46
*** epoojad1 has joined #oooq11:57
*** bhagyashris|NHO is now known as bhagyashris12:02
*** jpena is now known as jpena|lunch12:05
kopecmartinweshay_: joining?12:06
*** dtantsur|afk is now known as dtantsur12:07
ykarelweshay_, zbr|rover, is the issue related to upstream logs already known, logs files are not browsable, forced to download those to check contents12:24
ykarelis there fix already ?12:24
ykareli noticed few days back also, but forgot to ask12:24
ykarelfor reference example:- https://1b29e59047137324253b-6f1974401b778e56e9b593a6fc2d8e70.ssl.cf2.rackcdn.com/702430/1/check/tripleo-ci-centos-7-undercloud-containers/b1773df/logs/undercloud/home/zuul/12:26
ykareli know this is related to upstream change http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011875.html12:26
ykarelbut don't know if it's known for tripleo jobs and is being fixed12:26
zbr|roverykarel: on my today radar, getting back to logs in few minutes (i am on meeting)12:28
ykarelzbr|rover, ack take your time12:28
weshay|ruckkopecmartin, sorry man12:40
weshay|ruckkopecmartin, I slept through my alarm12:40
weshay|ruckdid you guys have a mtg?12:40
*** ykarel has quit IRC12:43
*** ykarel has joined #oooq12:43
weshay|ruckmarios, /me was looking at the zuul config for standalone upgrade yesterday12:49
kopecmartinweshay_: no problem, yes, i did a little presentation of what we have and what i'm working on atm12:49
weshay|ruckkopecmartin, ok.. so sorry I screwed up there12:50
weshay|ruckkopecmartin, so who is on board to help you?12:50
kopecmartinweshay_: arie, avi and vadim told me the were going to run the collect logs in their jobs and experiment with it a little12:51
weshay|ruckmarios, I saw two  standalone jobs fail... in gate.. 1 in drln-build and one w/ deploy12:51
mariosweshay|ruck: ack yeah lets talk about it on scrum. the branchs helped to unblock jobs  https://review.opendev.org/702625 but now its running on stein all the jobs sshnaidm posted some fixes for that bit today12:51
kopecmartini mentioned what i need and we kind of agreed on next steps12:51
weshay|ruckkopecmartin, ok..12:51
weshay|ruckkopecmartin, this is all getting started too late this sprint to present anything...12:52
weshay|ruckso kopecmartin let's just call this an early start on next sprint :)12:52
*** ykarel is now known as ykarel|afk12:52
weshay|ruckkopecmartin, can you summarize the next steps in an email? and send to openstack-pcci?12:53
weshay|ruck+ arie12:53
kopecmartinweshay_: sure, also lots of other distractions happened this sprint, like setuptools upgrade dropping python2 ..12:53
kopecmartinweshay_: sure, I'll send it12:53
weshay|ruckkopecmartin++12:54
weshay|ruckthanks man..12:54
*** ykarel|afk has quit IRC12:54
*** ykarel_ has joined #oooq12:54
*** ykarel_ is now known as ykarel12:54
kopecmartinweshay_: can we move this forward? https://review.opendev.org/#/c/682042/12:55
* weshay|ruck looks12:56
weshay|ruckkopecmartin, of course.. looks good https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d2d/682042/18/check/openstack-tox-molecule/d2dc03e/tox/reports.html12:57
*** ykarel is now known as ykarel|afk12:58
*** ratailor has quit IRC12:58
*** rlandy has joined #oooq12:59
rfolcoscrum time rlandy marios sshnaidm arxcruz|off zbr|rover panda|drappt12:59
*** epoojad1 is now known as epoojad1|afk13:02
*** marios is now known as marios|call13:02
*** amoralej is now known as amoralej|lunch13:02
*** d0ugal has quit IRC13:04
*** soniya29 has quit IRC13:05
*** epoojad1|afk has quit IRC13:06
*** d0ugal has joined #oooq13:10
*** jpena|lunch is now known as jpena13:10
zbr|roverput link to broken molecule job inside https://etherpad.openstack.org/p/ruckroversprint20 -- so I can watch it.13:11
*** panda|drappt is now known as panda13:25
marios|callhttps://review.rdoproject.org/r/#/c/24339/13:37
weshay|ruckhttps://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master%0913:42
weshay|ruckmarios|call, ^13:42
weshay|ruckhttps://softwarefactory-project.io/r/#/c/17307/13:48
*** akahat has quit IRC13:49
*** akahat has joined #oooq13:50
*** udesale has joined #oooq13:52
*** akahat has quit IRC13:56
*** dtantsur is now known as dtantsur|brb14:03
*** amoralej|lunch is now known as amoralej14:04
*** epoojad1 has joined #oooq14:06
*** bhagyashris has quit IRC14:18
weshay|ruckhttps://hackmd.io/5uYRmLaOTI2raTbHWsaiSQ14:20
chandankumarzbr|rover, Hello14:30
chandankumarzbr|rover, https://review.rdoproject.org/zuul/builds?job_name=podman-package-centos-814:31
*** derekh has joined #oooq14:31
chandankumarzbr|rover, is showing post failure , needs fix14:31
zbr|roverchandankumar: yeah, needs but i am already over my head14:32
zbr|roverchandankumar: can you propose a patch?14:32
chandankumarzbr|rover, ok looking14:32
*** epoojad1 has quit IRC14:35
weshay|ruckhttps://softwarefactory-project.io/r/#/c/17307/14:36
marios|callhttps://tree.taiga.io/project/tripleo-ci-board/us/1440?milestone=24927014:36
*** marios|call is now known as marios14:37
weshay|ruckzbr|rover, https://opendev.org/openstack/ansible-role-collect-logs/src/branch/master/tasks/publish.yml14:38
zbr|roverweshay|ruck: https://review.rdoproject.org/r/#/c/24580/14:38
chandankumarkopecmartin, please have a look https://review.opendev.org/70280814:42
*** TrevorV has joined #oooq14:42
*** epoojad1 has joined #oooq14:43
* kopecmartin looking 14:52
weshay|ruckzbr|rover,14:52
weshay|ruck "artcl_gzip_only": true,14:52
weshay|ruck    "artcl_publish": true,14:52
weshay|ruck    "artcl_rsync_logs": false,14:52
weshay|ruck    "artcl_txt_rename": true14:52
weshay|ruckhttps://d0b9bb7d6fcef48d96ec-efd5ea2c2502e38b3629faa5d1e6d102.ssl.cf5.rackcdn.com/702862/1/check/tripleo-ci-centos-7-standalone/ae948fa/logs/undercloud/var/log/extra/dump_variables_vars.json.txt.gz14:52
weshay|ruckmarios, I guess we're having some success w/ standalone-upgrade-$branch15:07
weshay|ruckhttp://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-standalone-upgrade-stein15:07
weshay|ruckoof http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-standalone-upgrade-train#15:08
mariosweshay|ruck: 2 issues. 1 is train broken but matbu fixing with https://review.opendev.org/#/c/701970/15:08
marios13:28 < matbu> and the quick and (a bit) dirty one: https://review.opendev.org/70192015:08
mariosweshay|ruck: the other issue is that now e.g. for stein there https://review.opendev.org/#/c/702696 it runs all the variants cos of the way we have branches15:09
rlandychandankumar: hi ... want to review the baremetal and internal slides?15:09
mariosweshay|ruck: sshnaidm trying to fix it with https://review.opendev.org/702843 OR https://review.opendev.org/70284415:09
rlandylet me know if they make sense to you15:09
weshay|ruckrlandy, what's the osp dlrn link?15:11
chandankumarrlandy, email the link, will get reviews up15:11
rlandyweshay|ruck, http://10.0.76.142/rhel8-osp17/15:12
weshay|ruckhtanks15:12
weshay|ruckrlandy, ah.. so15:13
weshay|ruckhttp://10.0.76.142/rhel8-osp17/component/15:13
weshay|ruckbut go back one dir15:14
weshay|ruckrlandy, http://10.0.76.142/rhel8-osp17/15:14
weshay|ruckboth kinds of builds are avail there15:14
weshay|ruckand we wouldn't have to go through this mess15:14
chandankumarweshay|ruck, please have a look at this [InfraRed] Python-waitress - Is your DFG affected? internal thread15:14
weshay|ruckchandankumar, k15:14
rlandyweshay|ruck: yeah downstream is actually easier :)15:18
weshay|ruckya15:18
rlandyif you look at tripleo-ci-testing ...15:18
rlandyhttp://10.0.76.142/rhel8-osp17/tripleo-ci-testing/delorean.repo15:18
rlandyit's the two components we promoted15:19
weshay|ruckchandankumar, so we're good re: waitress?15:19
weshay|ruckya15:19
chandankumarweshay|ruck, yes, it is taken care, all blame goes to neutron-lib or python-pecan which does not comes under our DFG15:26
weshay|ruckchandankumar++15:29
weshay|ruckthanks for responding15:29
*** epoojad1 has quit IRC15:38
*** holser has quit IRC15:40
*** holser__ has joined #oooq15:41
rlandymarios: should I w+ https://review.rdoproject.org/r/#/c/24549/?15:43
rlandythe impact will be jenkins15:43
weshay|ruckrlandy, he said is was a mistake15:48
*** rfolco is now known as rfolco|doctor15:48
rlandy?15:48
mariosrlandy: thinking /checking15:49
marios17:48 < weshay|ruck> rlandy, he said is was a mistake15:49
mariosweshay|ruck: ?15:49
rlandyweshay|ruck: what was a mistake?15:49
mariosrlandy: indeed concerned about the jenkins since its in the same repo ... i could maybe not chang that bit?15:50
mariosrlandy: before we merge it15:50
mariosrlandy: https://review.rdoproject.org/r/#/c/24549/2/jenkins/jobs/tripleo-upstream-builders.yml this15:50
marioswdyt?15:50
rlandymarios: ok - let's play this safe15:50
rlandyjust add the get-hash15:50
rlandywe merge that15:50
rlandychange jenkins last15:50
rlandyimho15:50
mariosrlandy: yeah -2 it just now15:50
rlandyI think it's fine15:50
mariosrlandy: i'll update sec15:50
weshay|ruckrlandy, the osp dlrn server and components15:53
chandankumarrlandy, marios, weshay|ruck zbr|rover https://review.rdoproject.org/r/2458115:54
chandankumarfor molecule job15:54
mariosrlandy: weshay|ruck: thanks done https://review.rdoproject.org/r/2454915:55
rlandyweshay|ruck: can he make the same mistake upstream?15:56
rlandymarios: thanks - will merge that when CI completes15:57
rlandythen you can start testing15:57
weshay|ruckchandankumar, are you done for the day?15:57
zbr|roverchandankumar: be careful about bindep, i have to -W it, see notes.15:58
chandankumarweshay|ruck, nope15:58
mariosrlandy: thanks for bringing it up and keeping us safe ;)15:58
rlandymarios: jenkins is a murky underworld where we proceed with intense caution15:59
weshay|ruckchandankumar, am I not seeing something obvious here? https://bugs.launchpad.net/tripleo/+bug/186000016:00
openstackLaunchpad bug 1860000 in tripleo "ERROR: cp cannot stat docker, no such file dlrn builds of tht" [Critical,Triaged]16:00
* weshay|ruck needs another pair of eyes.. 16:00
weshay|ruckchandankumar, https://review.opendev.org/#/c/702737/ rlandy please vote16:01
*** dtantsur|brb is now known as dtantsur16:02
chandankumarzbr|rover, done16:03
zbr|roverchandankumar: here is a hint, look at https://github.com/ansible/molecule/pull/2486/files as it could prove useful. is not merged yet but I suspect is right.16:05
weshay|ruckchandankumar, ah fixed the last time https://review.rdoproject.org/r/#/c/20218/16:05
zbr|roverchandankumar: ping me when confirmed by zuul.16:05
rlandymarios: k - want to try flip the switch on config?16:06
rlandyhttps://review.rdoproject.org/r/#/c/24552/16:06
*** jpena is now known as jpena|off16:07
rlandymarios: fyi ... https://github.com/rdo-infra/rdo-jobs/search?q=get-hash.sh&unscoped_q=get-hash.sh16:07
rlandylegacy - probably doesn't matter16:07
mariosrlandy: ack thanks16:09
rlandylooking at tox-molecule failure16:09
chandankumarrlandy, https://review.rdoproject.org/r/2458116:10
chandankumarpleas take over this on now looking into podman failure16:10
rlandyzbr|rover: chandankumar: k - adding a dep job to run tox-molecule16:14
zbr|roverrlandy: better enable the job directly in rdo-jobs, even with a placeholder scenario.16:15
weshay|ruckok.. I need eyes16:15
weshay|ruckzbr|rover, can I borrow you16:15
rlandyzbr|rover: yep - that is what I am doing16:15
weshay|ruckthis is fucking crazy16:15
zbr|roverin fact likely it only needs file patterns updated16:15
zbr|roverweshay|ruck: i know, link16:16
weshay|ruckzbr|rover, check out and clone https://review.opendev.org/#/c/702485/16:16
weshay|ruckit's stable/rocky afaict16:16
weshay|ruckzbr|rover, in the base dir.. look for a docker dir16:17
weshay|ruckthen look at https://opendev.org/openstack/tripleo-heat-templates/src/branch/stable/rocky16:17
weshay|ruckHOW IS THE DOCKER DIR MISSING16:18
zbr|roveris not missing forme, refresh page16:20
weshay|ruckit's in git.. on rocky16:20
weshay|ruckmissing from the patch16:20
weshay|ruckBUT HOW16:21
weshay|ruckdo you see what I see? </me sings>16:21
zbr|roverweshay|ruck: lucky guess: a branch override somewhere and zuul cherrypicks this commit on top of.... master?16:22
zbr|roveror something like this16:22
mariosweshay|ruck: every day as warren g?16:22
weshay|ruckmaybe.. this is fucked16:22
weshay|ruckmarios, and nate dogg of course :)16:23
zbr|roverclearly gerrit does clone the right stuff, must be zuul related or how the code is cloned.16:23
weshay|ruckzbr|rover, nothing to do w/ zuul16:23
weshay|ruckwhen I git-review -d Icaba7a58f458d853912d038f24c0cb4f853e0166 the docker dir is gone16:23
weshay|ruckwhen I checkout rocky it's there16:23
weshay|ruckWHAT THE F>... now it's workign?16:24
weshay|ruckzbr|rover, I need u16:26
weshay|ruckzbr|rover, https://meet.google.com/rwm-azpb-doj?authuser=116:26
mariosweshay|ruck: that album is one of my favs16:26
* marios almost out16:26
weshay|ruckzbr|rover, you avail?16:29
zbr|rovernow... multitasking overload16:29
weshay|ruckzbr|rover, aye.. sorry part of the ruck/rover job16:29
weshay|ruckthis is blowing my mind.. I need to show you this16:29
chandankumar marios sshnaidm thank you for looking and working on upgrade jobs upstream :-)16:30
chandankumarmarios++ sshnaidm++16:30
*** bogdando has quit IRC16:31
*** bogdando_ has joined #oooq16:31
rlandymarios: if we can get my and chandankumar's patches though to clear rdotox-molecule, should we try the switch then?16:31
rlandyreally rattle ruck/rovers16:31
*** holser has joined #oooq16:32
*** holser has quit IRC16:32
*** holser__ has quit IRC16:32
*** holser has joined #oooq16:33
*** bogdando_ is now known as bogdando16:33
mariosrlandy: add me to reviews will check tomorrow16:34
mariosrlandy: looking at some of my slides now but will be going in few mins16:34
rlandymarios: ack - understood16:35
weshay|ruckzbr|rover, https://bugs.launchpad.net/tripleo/+bug/186000016:38
openstackLaunchpad bug 1860000 in tripleo "ERROR: cp cannot stat docker, no such file dlrn builds of tht" [Critical,Triaged]16:38
*** chem has quit IRC16:39
*** tesseract has quit IRC16:39
rlandyzbr|rover: ok ... pls review https://review.rdoproject.org/r/#/c/24582/ rebased off  https://review.rdoproject.org/r/#/c/24581)16:43
chandankumarweshay|ruck, sorry not able to look at that bug, will look into that now or tomorrow morning16:43
rlandysee the rdo-tox-molecule test run in the first patch16:43
rlandychandankumar: zbr|rover: ^^16:43
weshay|ruckzbr|rover,     "artg_repos_dir": "/home/zuul/src/opendev.org/openstack",16:44
rlandyzbr|rover:  if ok - pls remove your -2 from https://review.rdoproject.org/r/#/c/24581/16:44
rlandyand we can merge both16:44
*** chem has joined #oooq16:44
*** marios is now known as marios|out16:46
weshay|ruckchandankumar, np.. we figured out.. issue w/ gerrit16:48
chandankumarweshay|ruck, :-)16:49
* chandankumar heads to my cave, see ya tomorrow16:49
*** chandankumar is now known as raukadah16:49
weshay|ruckraukadah, should I merge the centos8 image change?16:49
weshay|rucklooks fine16:49
raukadahSee ya people tomorrow16:50
raukadahweshay|ruck, we need ack from tristinc and zbr|rover16:50
zbr|roverweshay|ruck: apparently is ok to keep the change id on cherry picks16:50
*** marios|out has quit IRC16:54
zbr|roverrlandy: raukadah: this is what I asked: https://review.rdoproject.org/r/#/c/24581/16:59
zbr|roverto assure that when bindep.txt is touched, we also run the job, avoiding breaking it.17:00
rlandyzbr|rover: that is what I did17:00
rlandyin a separate patch17:01
zbr|roverrlandy: why sep patch? i do not want others to mess with this file and break the job by mistake.17:01
rlandyzbr|rover: does it matter?17:01
zbr|roverthese jobs a not heavy like deploymens, we should run them.17:01
rlandyhttps://review.rdoproject.org/r/#/c/24582/17:02
zbr|rovermatters because it makes review harder and does not prevent future regression.17:02
rlandywhatever17:02
*** baha has quit IRC17:13
*** bogdando_ has joined #oooq17:16
*** bogdando has quit IRC17:17
*** bogdando_ is now known as bogdando17:17
rlandyzbr|rover: merging https://review.rdoproject.org/r/#/c/24581/17:19
zbr|roverrlandy: super17:20
*** baha has joined #oooq17:23
*** fmount is now known as fmount|afk17:28
*** jpena|off is now known as jpena17:29
*** udesale has quit IRC17:31
*** amoralej is now known as amoralej|off17:35
*** saneax has quit IRC17:40
*** rfolco|doctor is now known as rfolco17:42
*** bogdando has quit IRC17:44
mjturekyooooo, could you take a ook here? https://review.rdoproject.org/r/#/c/24560/17:48
mjturekwhoops17:48
mjturekrfolco ^17:48
rfolcomjturek, heh I just asked baha on #rdo about it17:50
rfolcomjturek, what did happen to the blacklist approach ?17:50
rfolcomjturek, +w I won't hold you, just was curious on the previous blacklist approach :)17:57
mjturekit's making progress but we're impatient17:57
rfolcomjturek, ok17:58
mjturekwe're willing to revert if/when it happens17:58
*** derekh has quit IRC18:01
weshay|ruckrlandy, rfolco was someone fixing the config molecule job w/ a bindep change? http://logs.rdoproject.org/52/24552/2/check/rdo-tox-molecule/efaaa78/job-output.txt18:25
rlandyweshay|ruck: ack18:25
rlandyjust merged it18:25
rlandyhttps://review.rdoproject.org/r/#/c/24581/18:26
rlandylooking18:26
rlandyweshay|ruck: my recheck should have been after merge18:27
weshay|ruckrlandy, yup.. ur right18:28
rlandythe change was in rdo-jobs18:28
rlandythis one fails config18:28
weshay|ruckzbr|rover, can we change that from a f28 node?18:30
*** dtantsur is now known as dtantsur|afk18:30
zbr|roveryes18:30
zbr|roverbut i was waiting for rdo to finish upating centos-8 image18:31
weshay|ruckzbr|rover, do you think they will be done today?18:31
weshay|ruckis there a review for that?18:32
zbr|roveryes18:32
weshay|ruckzbr|rover, busy day eh?18:32
weshay|ruckzbr|rover, the molecule install deps work fine on f3018:33
weshay|ruckbut running from the base dir of config18:34
weshay|ruckI get ERROR: file not found: molecule18:34
zbr|roveryeah! and I didn't tell you all things I went on.18:34
rlandyhttps://review.rdoproject.org/zuul/builds?job_name=rdo-tox-molecule18:35
rlandyso we have passes on rdo-jobs18:36
rlandybut failures on config18:36
weshay|ruckmissing env dir for just the molecule command18:36
weshay|ruckzbr|rover, where is the review on the centos8 stuff18:37
weshay|ruckzbr|rover, is rhel8 a quick option?18:38
zbr|rovernope18:38
weshay|ruckit's configured in tox.ini18:38
weshay|ruckk18:38
weshay|ruckzbr|rover, so.. if I have a python3 virt env18:39
weshay|ruckand run tox molecule18:39
zbr|rovergive me few minutes18:39
weshay|ruckthat should work18:39
weshay|ruckk18:39
weshay|ruckdeps are working18:39
weshay|ruck\0/ https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fec/702286/2/check/tripleo-ci-centos-7-standalone/fecdf85/logs/README.html18:39
weshay|ruckrlandy, ^18:40
weshay|ruckhttps://review.opendev.org/#/c/702286/18:40
*** jpena is now known as jpena|off18:41
weshay|ruckrlandy, can you hook /me up w/ a +218:41
rlandyweshay|ruck: congrats18:42
rlandyzbr: do you know why the rdo-tox-molecule job passes on rdo-jobs and fails on config?18:48
rlandyonly defined in rdo-jobs18:48
rlandyand called from projects in config18:48
rlandyzbr|rover: ^^?18:48
zbr|roverrlandy: because bindep is taken from builded repo, so you need to keep it in sync18:48
rlandybindep in config ...18:48
zbr|roveralternative is to put these in playbook that prepares molecule to avoid having to copy/paste it everywhere.18:49
rfolcodoes anybody know what happened to centos-release-opstools on centos8 ?18:49
rfolcocentos-release-opstools-1-8.el7.noarch : Config to enable the repository for OpsTools SIG18:49
rfolcoRepo        : extras18:49
rlandyand if there is none??18:49
rfolcoavailable in extras on centos7, it does not exist in centos818:50
rlandyzbr|rover: ^^ ... so we need to get marios job through18:50
rlandya playbook that calls a single bindep?18:51
zbr|roverrlandy: weshay|ruck : I will take care of switching molecule to centos-8 via https://review.rdoproject.org/r/#/c/24584/ but we need bit of patience to allow tristanC to finish updating the image.18:51
weshay|ruckzbr|rover, I have no patience on this one18:51
weshay|ruckif it's not ready.. let's use f3018:51
zbr|roverahh sure, why not.18:51
rlandywe may live to regret that18:52
tristanCzbr|rover: weshay|ruck: the cloud-centos-8 should be ready and usable now18:52
rlandyweshay|ruck: ^^ request to let zbr|rover try that first18:52
tristanCand we can get more capacity for it with https://softwarefactory-project.io/r/1733318:52
weshay|ruckaye18:52
rlandyotherwise we will be debugging f30 next18:52
* rlandy has appt at the DMV to get that new Real ID license thing ... 18:53
weshay|ruckrlandy, you don't have one w/ real id?18:53
rlandywill be back soon-ish18:53
weshay|rucku sure?18:53
rlandyweshay|ruck: pretty sure18:53
zbr|rovertristanC: what is the correct nodeset name to use for it?18:53
weshay|rucklook for a star18:53
rlandyI have no stars18:54
rlandyits' very sad18:54
*** rlandy is now known as rlandy|afk18:54
weshay|ruckrlandy|afk, heh k18:54
*** dsneddon has quit IRC18:54
tristanCzbr|rover: i didn't create a nodeset, just a label18:54
tristanCso, should we merge https://review.rdoproject.org/r/#/c/24583/ ?18:58
rlandy|afktristanC: I will when CI passes19:01
tristanCrlandy|afk: what job/change are you using to validate it?19:01
rlandy|afkit's ruuning tests now19:03
zbr|roversomething is broken in bindep... aparently [compile test platform:centos-7 platform:rpm] evaluates as true on centos-819:13
*** holser has quit IRC19:13
zbr|roverwtf....19:13
*** holser has joined #oooq19:14
*** ccamacho has quit IRC19:18
zbr|roverin fact I thin that this broken bindep comes from another place, but I have no clue from where19:18
zbr|roveror maybe not,.... condition was not working as expected.19:18
zbr|roverpersonally I would ditch bindep tool and replace it with a bindep.yaml playbook.19:19
*** holser has quit IRC19:20
zbr|roverobviously that ensure-tox chokes .... failing to find tox and pip19:22
raukadahzbr|rover, need some help here https://review.rdoproject.org/r/#/c/24579/19:26
*** dsneddon has joined #oooq19:29
zbr|roverand obviously that now i hit fixes I propose 3 months ago but that were not merged https://review.opendev.org/#/c/690057/319:29
zbr|rovertime to chaise ianw as I think his vote was wrong19:29
*** dsneddon has quit IRC19:34
zbr|rovertristanC: by any chance do we use an outdated clone of zuul-jobs on rdo? because I see a bug that was fixed long time ago in ensure-tox19:35
tristanCzbr|rover: perhaps, there is a fetch-zuul-cloner-fork/ in rdo-jobs, not sure it's used19:45
zbr|rovertristanC: i was not able to find any occurence of  fetch-zuul-cloner-fork on eiher sf nor rdo zuul instances, so i guess not.19:47
zbr|roverapparently we no longer have tox in centos-8,... should I dare to do a 'pip install tox' when we build the image...?19:49
*** tosky has quit IRC20:04
*** dsneddon has joined #oooq20:07
sshnaidmzbr|rover, try to use ! conditions in bindep, like !platform:centos-8 if you want to exclude20:11
*** sshnaidm is now known as sshnaidm|afk20:14
*** dsneddon has quit IRC20:15
weshay|rucktristanC, in the dib build for centos20:22
weshay|ruckcan we make sure pip3 and tox are installed20:22
tristanCweshay|ruck: yes sure, probably using an elements or editing an existing one20:22
weshay|ruckso we can avoid issues like https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-tox/tasks/main.yaml20:22
weshay|ruckzbr|rover, that will do it right?20:23
tristanCiirc zbr|rover proposed something already?20:23
zbr|rovertristanC: working on it, it seems that my fixes on ensure-tox where not merged upstream.20:23
*** jfrancoa has quit IRC20:31
*** dsneddon has joined #oooq20:49
*** dsneddon has quit IRC20:56
*** TrevorV has quit IRC20:59
*** rlandy|afk is now known as rlandy21:03
rlandypodman-package-centos-8NODE_FAILURE in 0s (non-voting)21:05
rlandyrebasing21:06
zbr|rovergood progress on upstream, we will have a fix today for ensure-tox21:07
rlandyweshay|ruck: can you explain your email re: downstream rrcockpit?21:07
weshay|ruckrlandy, so I was poking around sshnaidm|afk's patches.. I think he has everything merged he has up21:08
weshay|ruckrlandy, I guess I need to check the internal repo21:08
weshay|ruckto see if we've added the internal zuul config for telegraf21:08
rlandyweshay|ruck: getting internal repo21:09
weshay|ruckhttps://code.engineering.redhat.com/gerrit/#/q/owner:%22Sagi+Shnaidman+%253Csshnaidm%2540redhat.com%253E%22+project:openstack/rrcockpit21:09
rlandyweshay|ruck: it would be good if this was added to the tenant21:10
rlandyand actually ran tests :)21:10
rlandyweshay|ruck: ok - so in downstream we want both upstream and downstream data21:12
weshay|ruckrlandy, aye21:12
weshay|ruckrlandy, so we'll start w/ just a 1 downstream page.. and experiment from there21:12
rlandyweshay|ruck: and right now we only have upstream data21:12
rlandyso we don;t want to intermingle the two21:13
weshay|ruckrlandy, don't know yet :)21:13
rlandyweshay|ruck: this fight again :)21:13
weshay|ruckbut I for sure I want a page that is downstream only so I can check to see if everything we should see.. is visible21:13
rlandypodman-package-centos-8success (non-voting)21:14
rlandywoohoo21:14
rlandyraukadah: ^^21:15
sshnaidm|afkweshay|ruck, I'll add a page to downstream repo and telegraf file21:15
sshnaidm|afkweshay|ruck, we'll see if it works as designed21:15
sshnaidm|afkI hope it is :)21:15
weshay|ruckrock on21:15
*** dsneddon has joined #oooq21:23
rfolcoshutdown21:23
rfolcohttps://hackmd.io/dSagCbocQ4KSVEZR1uf8Tw weshay|ruck 86 containers built so far21:23
rfolcottyl21:23
weshay|ruckrock21:24
rlandyweshay|ruck: for a manual standalone deployment, are these the docs to follow: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deployment/standalone.html?21:28
zbr|roverweshay|ruck: fix https://review.opendev.org/#/c/702978/21:29
weshay|ruckrlandy, yes.. Emilien also has a nice blog w/ a how to21:29
*** rfolco has quit IRC21:29
weshay|ruckrlandy, in the bootcamp notes I think21:29
rlandyhttps://my1.fr/blog/developer-workflow-with-tripleo/21:29
weshay|ruckzbr|rover, looking21:29
weshay|ruckrlandy, aye21:30
rlandyok21:30
zbr|roverthe fact that rdo does not work with depdens-on on zuul-jobs could prove to be a testing problem21:30
weshay|ruckzbr|rover++21:30
zbr|roveri wonder if is possible to address this21:30
weshay|ruckdon't go too deep :)21:31
zbr|roverbecause I sure there is a LOT of work to about making zuul-roles centos-8 ready, a lot.21:31
weshay|ruckzbr|rover, should be baked into the image imho21:31
zbr|roverweshay|ruck: baking stuff in the images is not a panacea, images are build in many places, using many methods,... bug zuul roles are here to stay.21:32
zbr|roverbaking stuff in an image is more of an optimization AFAIK, but should not be a replacement for having these roles ready.21:32
zbr|roverin fact these roles may be used to bake those images.21:33
weshay|ruckzbr|rover, /me just saying.. I think tox sould be installed and ready to go in a ci system21:37
*** brault has joined #oooq22:07
zbr|roverweshay|ruck: rlandy: gz fix: https://review.opendev.org/#/c/702937/122:09
zbr|roverweshay|ruck: true that it does not solve all, but we will get there.22:12
zbr|roverthere is also https://review.opendev.org/#/c/702949/ which is a collection optimization.22:15
zbr|rovermainly it makes no sense to perform a local disk rsync, instead of creating hardlinks.22:15
zbr|rover --link-dest= feature in rsync is not very well known, but can help a lot, especially on filesystems that do not have CoW22:16
zbr|roverand if i remember well, we do not have CoW in VMs22:16
zbr|roverthere is something weird there as I see the zero date (1970) on some files.22:18
zbr|roverlook at https://1a26646eddef9988f4a8-d413fbe3672f31435774ef2983026d41.ssl.cf5.rackcdn.com/702949/1/check/tripleo-build-containers-centos-7/38f0ec6/logs/buildah-builds/kolla-2020-01-16_18-33-39_ub0uZY/22:18
zbr|roverand go inside22:18
weshay|ruckzbr|rover, ya.. that works22:40
weshay|ruckre: the build-contianers22:40
weshay|ruckbut note.. it doesn't use collect-logs22:40
weshay|ruckso it's pretty clear clark fixed what ever he needed to22:40
weshay|ruckI guess22:40
*** chem has quit IRC23:01
*** chem has joined #oooq23:03
rlandyweshay|ruck: when you run standalone, how many nics is the box expected to have?23:17
weshay|ruckrlandy, it's easier w/ two23:18
rlandyso create eth1 ahead of time23:18
sshnaidm|afkrlandy, weshay|ruck, https://review.rdoproject.org/r/#/q/topic:down+(status:open+OR+status:merged)23:51
rlandythanks - leaving comments from testing patch that we abandoned23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!