Monday, 2020-04-06

*** holser has quit IRC00:20
*** jmasud has quit IRC00:55
*** jmasud has joined #oooq00:56
*** jmasud has quit IRC00:56
*** jmasud has joined #oooq00:59
*** rfolco has joined #oooq01:24
*** ysandeep|away is now known as ysandeep|rover01:30
*** ysandeep|rover is now known as ysandeep|away01:31
*** rfolco has quit IRC01:45
*** Goneri has quit IRC02:15
*** ysandeep|away is now known as ysandeep|rover03:33
*** skramaja has joined #oooq04:07
*** ykarel|away is now known as ykarel04:12
ysandeep|roverykarel, chandankumar fyi.. wes opened https://bugs.launchpad.net/tripleo/+bug/1871010 today.. but issue auto cleared in subsequent run..04:23
openstackLaunchpad bug 1871010 in tripleo " Problem: package python3-tripleoclient-13.1.1-0.20200404091914.624a61f.el8.noarch requires validations-common" [Critical,Triaged]04:23
ysandeep|roverhttps://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master/d18dc30/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz04:23
ykarelysandeep|rover, ack looking04:24
ykarelvalidations-common was recently added04:24
ykarelohh04:25
ykarelneed to add validations-common to delorean current04:25
ykarelalso validations libs04:25
ysandeep|roverits currently coming from :_04:25
ysandeep|rover~~~04:25
ysandeep|rover2020-04-04 17:28:19 |  validations-common                      noarch0.0.1-0.20200403122940.99af090.el8      delorean-component-tripleo      99 k04:25
ysandeep|rover~~04:25
ykarelfor quick fix we can revert04:26
ykarelit would also need fixes in tripleo-repos04:26
ykarelit should be hurting upstream jobs too04:26
ykarellet me check04:26
ykarelstrange it's passing in usptream04:31
ykarelsomething else is wrong04:33
ykarelpackage is missing in http://mirror.regionone.rdo-cloud.rdoproject.org:8080/rdo/centos8/component/tripleo/f3/83/f38338fb2ce36e278adcbb636377428b7190e456_abb1ed0f/04:34
ykarelas per current situation of tripleo-ci-testing it should pass now04:49
ykarelbut need to find root cause of last failure so it can be avoided in future04:49
*** udesale has joined #oooq04:51
*** udesale has quit IRC04:52
*** udesale has joined #oooq04:52
ysandeep|roverykarel, its passing now, agree we need to find root cause..04:52
ykarelohhk so failure was from 5th :)04:54
ykarelbut bug filed today, i see last run was passing04:55
ykarelhttps://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-master04:55
ysandeep|roverykarel, yes.. today run passed04:56
ysandeep|roverAs per https://trunk.rdoproject.org/centos8-master/report.html last rpm build for this package was on 2020-04-03 12:28:132020-04-03 06:12:55validations-common99af0906034c8529d25d040ed41c467a02947c52, its weird how this was missing from repos on 5th04:58
*** udesale has quit IRC05:03
*** udesale has joined #oooq05:03
*** ratailor has joined #oooq05:04
*** marios has joined #oooq05:49
*** jfrancoa has joined #oooq06:03
*** bogdando has joined #oooq06:17
*** jbadiapa has joined #oooq06:27
*** dpawlik has joined #oooq06:28
*** ccamacho has joined #oooq06:34
*** ccamacho has quit IRC06:46
*** ccamacho has joined #oooq06:46
*** saneax has joined #oooq06:53
*** yolanda has joined #oooq06:56
*** apetrich has joined #oooq07:11
mariosysandeep|rover: o/ seen that one ? ERROR! the role 'tripleo-bootstrap' was not found in /          * https://logserver.rdoproject.org/95/717495/1/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/42c89d3/logs/undercloud/home/zuul/undercloud_install.log.txt.gz07:17
mariosysandeep|rover: i can't quickly see it in the ruck rover hackmd07:18
ysandeep|rovermarios, no this is not reported yet07:18
ysandeep|rovermarios, checking07:19
ysandeep|rovermarios, will report bz07:19
mariosysandeep|rover: ack thanks i saw it via https://review.opendev.org/#/c/717495/ tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001FAILURE in 39m 55s (rdo third party)07:19
ysandeep|rovermarios, i know what is causing this.. i will fix that.. https://review.opendev.org/#/c/703477/ - tripleo-bootstap was renamed to tripleo_bootstrap07:23
marioszbr: know of any issues with tox-molecule job ?07:23
mariosysandeep|rover: i see another one now molecule job ^^ failing at https://review.opendev.org/#/c/716896/ '/usr/bin/gpg-agent': No such file or directory\ngpg: can't connect to the agent: No such file or directory",07:24
marioshttps://a6b4f3cbb2df652c48f3-8466237625318fdb84c00b23a21e0718.ssl.cf1.rackcdn.com/716896/6/check/openstack-tox-molecule/9728ae6/job-output.txt07:24
mariosysandeep|rover: sorry... :/ just fyi no emergency add it to your list07:24
ysandeep|rovermarios, sure..07:25
marioszbr: ysandeep|rover: hmm looks like that one might be fixed already  the tox molecule i mean07:26
mariosysandeep|rover: http://zuul.openstack.org/builds?job_name=openstack-tox-molecule07:26
mariosysandeep|rover: will recheck for now thanks07:26
mariosysandeep|rover: from the zuul builds looks like that retry_limit hit a few times on friday 03rd and then green07:26
ysandeep|rovermarios, query - where is rdo third party code defined? i don't find tripleo-bootstrap role anywhere in rdo codesearch - https://codesearch.rdoproject.org/?q=tripleo-bootstrap&i=nope&files=&repos=07:27
mariosysandeep|rover: tripleo-ansible07:30
mariosysandeep|rover: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_bootstrap07:30
mariosysandeep|rover: there is no 'rdo third party code' ... mainly job definitions.. .we do hav some pre/post playbooks but they are called as part of a job... not for the deployment07:32
mariosysandeep|rover: does it help?07:32
*** amoralej has joined #oooq07:33
mariosysandeep|rover: fwiw i use this for codesearch you might find it useful http://paste.openstack.org/raw/791648/07:34
mariosysandeep|rover: i have it in my bash profile07:34
ysandeep|rovermarios, yes tripleo_bootstrap in in tripleo-ansible, from the error it looks we are still calling older tripleo-bootstrap(which was removed here https://review.opendev.org/#/c/703477/) somewhere in rdo third party ci. But can't find where in rdo codesearch.. No worries i will keep looking. thanks!07:34
ysandeep|rovermarios, many thanks for sharing :)07:34
mariosysandeep|rover: ack07:35
*** dtantsur|afk is now known as dtantsur07:41
chandankumarysandeep|rover, in third party, all jobs are defined, all the codes are coming from upstream07:46
chandankumar*defined in rdo07:47
ysandeep|roverchandankumar, okay o/ .. weird i am unable to find https://codesearch.rdoproject.org/?q=tripleo-bootstrap&i=nope&files=&repos= anywhere.. but error depicts its getting used somewhere  https://bugs.launchpad.net/tripleo/+bug/187103307:51
openstackLaunchpad bug 1871033 in tripleo "RDO Third Party CI check failing with ERROR! the role 'tripleo-bootstrap' was not found" [Critical,Triaged]07:51
chandankumarysandeep|rover, https://opendev.org/openstack/tripleo-ansible/src/branch/stable/train/tripleo_ansible/roles/tripleo-bootstrap07:56
chandankumarysandeep|rover, check in stable/train branch code07:56
*** tosky has joined #oooq07:58
chandankumartripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 job is meant for master not sure why it is running on train branch07:58
ysandeep|roverchandankumar, :)07:59
chandankumarysandeep|rover, https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo.yaml#L79108:00
chandankumarit needs to be updated with centos-8 based jobs and replace it with train job08:01
ysandeep|roverchandankumar, thank you.. checking08:04
*** sshnaidm|afk is now known as sshnaidm08:07
zbrmorning08:25
*** dpawlik has quit IRC08:29
*** ysandeep|rover is now known as ysandeep|lunch08:36
*** dpawlik has joined #oooq08:38
ykarelysandeep|lunch, fyi i have updated https://bugs.launchpad.net/tripleo/+bug/1871010 with analysis08:44
openstackLaunchpad bug 1871010 in tripleo " Problem: package python3-tripleoclient-13.1.1-0.20200404091914.624a61f.el8.noarch requires validations-common" [Critical,Triaged]08:45
ykarelwill involve jpena for missing things08:45
ykareljpena|off, can u please check once available08:45
*** ykarel is now known as ykarel|lunch08:50
*** jmasud has quit IRC08:53
*** jmasud has joined #oooq08:56
*** holser has joined #oooq08:57
zbrmarios: what mo' issue where you refering to?09:14
marios10:26 < marios> zbr: ysandeep|rover: hmm looks like that one might be fixed already  the tox molecule i mean09:14
marios10:26 < marios> ysandeep|rover: http://zuul.openstack.org/builds?job_name=openstack-tox-molecule09:14
marios10:26 < marios> ysandeep|rover: from the zuul builds looks like that retry_limit hit a few times on friday 03rd and then green09:14
*** holser has quit IRC09:14
marioszbr: i was seeing it there https://review.opendev.org/#/c/71689609:14
zbrmarios: ahh, that is 99.9% not related to the tool :D09:15
*** ysandeep|lunch is now known as ysandeep09:18
ysandeepykarel|lunch, ack thank you!09:18
*** ysandeep is now known as ysandeep|rover09:19
*** holser has joined #oooq09:21
*** ykarel|lunch is now known as ykarel09:36
*** blueDuckRandom has joined #oooq10:21
*** derekh has joined #oooq10:45
amoralejysandeep|rover, i see jobs failing with an issue related to puppet-collectd10:48
amoralejhttps://logserver.rdoproject.org/46/26246/2/check/rdoinfo-tripleo-master-testing-centos-8-scenario001-standalone/50e92fa/logs/undercloud/var/log/containers/stdouts/container-puppet-collectd.log.txt.gz10:48
amoraleji guess you are also seeing it in tripleo-ci10:48
amoralejroot cause seems to be https://github.com/voxpupuli/puppet-collectd/commit/d7b79c502bd10587d7498ddc1d19484eabfd738010:49
amoralejnot sure if you will push fix to puppet-tripleo or prefer to pin puppet-collectd10:49
ysandeep|roveramoralej o/, looking ^^10:50
ysandeep|roveramoralej, yes upstream jobs are also affected by this11:04
ysandeep|roveropening bz for this11:05
amoralejysandeep|rover, we could pin in rdo to previous commit but then we will not be able to push a fix in puppet-tripleo11:06
ysandeep|roverykarel, ^^ thougts11:06
amoraleji'll try to send a PR to make upstream backwards compatible11:06
ysandeep|roverthoughts*11:06
amoralejso i think the quicker is to pin in rdoinfo11:06
ykarelysandeep|rover, amoralej let's go with both in parallel11:06
ykarelpinning + fixes11:07
amoralejyep11:07
amoralejykarel, can you send the pin review?11:07
ykarelamoralej, sure sending11:07
amoralejysandeep|rover, ykarel https://github.com/voxpupuli/puppet-collectd/pull/93311:26
ysandeep|roveramoralej, thank you, fyi.. i will add info on https://bugs.launchpad.net/tripleo/+bug/187108611:28
openstackLaunchpad bug 1871086 in tripleo "tripleo-ci-centos-8-scenario001-standalone jobs failing with Container(s) with bad ExitCode: [''container-puppet-collectd'']" [Critical,Triaged]11:28
*** chem has joined #oooq11:28
amoralejsure11:29
chandankumaramoralej, ysandeep|rover I hitt this bug last week on fs016 while working on tobiko job, let me know when fix is up, I can respin my patches11:32
ysandeep|roverchandankumar, ack will let you once its fixed11:34
ykarelchandankumar, btw did ur deployment moved forward with ens3 removal?11:36
ykarelmay be i missed ur ping on friday as i had left11:36
chandankumarykarel, yes, I am at container tripleo_container_image_prepare : Run tripleo_container_image_prepare logged to11:36
ykarelchandankumar, okk cool11:36
*** ratailor has quit IRC12:11
*** rlandy has joined #oooq12:12
soniya29arxcruz, I didn't got your first comment on this patch https://review.opendev.org/#/c/717705/12:14
arxcruzsoniya29: there's no default_release, use master, ussuri, rocky, queens, etc12:15
arxcruzif there's no releases option, we can assume is the master12:15
soniya29arxcruz: if lp and reason is common for all releases then should I define all releases or just master is good enough?12:16
arxcruzsoniya29: just master is enough12:17
soniya29arxcruz: okay12:17
*** amoralej is now known as amoralej|lunch12:20
mariosrlandy: o/12:25
mariosrlandy: wana sync before scrum?12:25
*** weshay_ has joined #oooq12:27
*** udesale_ has joined #oooq12:28
*** udesale has quit IRC12:31
*** Goneri has joined #oooq12:31
mariosbrb12:33
weshay_ysandeep|rover, is https://bugs.launchpad.net/tripleo/+bug/1871086 in check or periodic?12:35
openstackLaunchpad bug 1871086 in tripleo "tripleo-ci-centos-8-scenario001-standalone jobs failing with Container(s) with bad ExitCode: [''container-puppet-collectd'']" [Critical,Triaged]12:35
*** rfolco has joined #oooq12:35
weshay_sshnaidm, ysandeep|rover is vexhost down?12:36
ysandeep|roverweshay_, https://bugs.launchpad.net/tripleo/+bug/1871086 - both check and periodic12:37
openstackLaunchpad bug 1871086 in tripleo "tripleo-ci-centos-8-scenario001-standalone jobs failing with Container(s) with bad ExitCode: [''container-puppet-collectd'']" [Critical,Triaged]12:37
sshnaidmweshay_, not that I'm aware of12:37
sshnaidmweshay_, why?12:38
chandankumarrlandy, what is the downstream dlrn rhos-17 url?12:44
rlandymarios: hey - we can12:44
rlandychandankumar: sending in pvt12:44
chandankumarrlandy, aye thanks!12:44
chandankumarrlandy, I need tripleo-ci-testing dir there12:47
chandankumarfor image build12:47
sshnaidmbut OVB seems to be broken12:48
mariosrlandy: lets go for after scrum have to deal with something else for a few mins12:49
weshay_sshnaidm, ya.. is it ovb or vex?12:57
weshay_ysandeep|rover, sshnaidm looks like ironic can not power on the nodes...12:57
weshay_perhaps we can hold a job12:57
weshay_ysandeep|rover, can you please create a test project for ovb jobs in train, stein, rocky.. in one review..12:58
rfolcoping scrum (in one min) >>>>>>>>>>>> arxcruz, rfolco, zbr, panda, sshnaidm, rlandy, marios, ysandeep, bhagyashris, soniya29, pojadhav, akahat, weshay, chandankumar12:58
sshnaidmweshay_, ovb12:58
rfolcoscrum here https://meet.google.com/oiv-geho-mai?authuser=112:58
sshnaidmweshay_, there are various problems, cloudnull is looking into one atm12:59
akahatrfolco, joining..12:59
ysandeep|roverweshay_, on it12:59
weshay_ysandeep|rover, before you commit..12:59
weshay_let me show you one thing13:00
ysandeep|roversure13:00
weshay_ysandeep|rover, https://docs.google.com/document/d/1VZhje7ZN9sk4E31fYVrPxpqMJGz5ZhHRfhte_RYMXxg/edit#heading=h.xn5rxl1lodsl13:01
weshay_note the line... remove_ovb_after_job: false13:01
weshay_add that to each13:01
rfolcoweshay_, you join ?13:01
ysandeep|roverweshay_, ack13:01
*** beagles is now known as beagles_mtg13:03
*** radez has joined #oooq13:06
*** Goneri has quit IRC13:08
*** ykarel is now known as ykarel|afk13:11
weshay_ysandeep|rover, sshnaidm let's jump on a call outside of scrum13:11
ysandeep|roverweshay_, ack13:11
weshay_ysandeep|rover, sshnaidm https://meet.google.com/ndn-orby-pvr?authuser=1 I'm in here looking13:12
*** Goneri has joined #oooq13:15
zbrmarios: "disseminate"13:18
marioszbr: which bit?13:18
zbri was trying to use a posh term for what we are doing during the scrum13:19
marioszbr: :) ... well it sounds correct...13:19
*** dtantsur is now known as dtantsur|brb13:21
*** amoralej|lunch is now known as amoralej13:23
zbrfor operator work it would be a good idea to create a placeholder review even when is not implemented13:25
*** ccamacho has quit IRC13:25
zbrjust to let others knowing that you are working on it13:25
zbrlike avoiding having two people trying to work on the same thing13:25
mariosrfolco: perhaps add a card to the retrospective board for the discussion13:29
mariosrfolco: regarding scrum format13:30
mariossshnaidm: ^^13:30
sshnaidmmarios, yeah, agree13:30
zbrrfolco: ++ for aiming to keep scrum under 15min, anything else should be outside, other meetings with smaller audience, even ad-hoc ones.13:30
zbrout time matters, and having >10 people busy for long is draining / unoptimal use of resources.13:31
mariospanda: +2 with love13:33
marios;)13:33
pandamarios: dont' make me blush13:35
*** ccamacho has joined #oooq13:35
mariosweshay_: when you next get a chance can you please see my latest email reply to sacollins might need a confirmation from you thanks13:41
*** blueDuckRandom has quit IRC13:41
mariospanda: quid pro quo  https://review.opendev.org/#/c/715397/ reviews.enqueue()13:42
pandamarios: thans for you your application, your request has been accepted with id #21939405, currently serving #10. Have a good day.13:43
*** TrevorV has joined #oooq13:56
mariosrlandy: sync at half past? work for you?13:57
sshnaidmrfolco, can you please add this us to sprint instead of "role #2, #3" etc? https://tree.taiga.io/project/tripleo-ci-board/us/157813:59
rfolcook after this mtg will do sshnaidm13:59
rlandymarios: yeah14:01
*** ykarel|afk is now known as ykarel14:06
chandankumarrlandy, sshnaidm marios https://review.opendev.org/#/c/717043/ please have a look at image build role refactoring ,just wanted to make sure going in correct direction14:08
marioschandankumar: ack adding for next reviews14:10
rlandyk14:10
mariospanda: thanks please revote when you can https://review.opendev.org/#/c/715397/14:22
*** sshnaidm is now known as sshnaidm|afk14:24
pandamarios: do you have 5 seconds to talk about Ψ ?14:26
mariospanda: have call with rlandy now but maybe after that?14:28
mariospanda: also what about ψ14:28
*** bogdando has quit IRC14:28
pandamarios: credentials, url ....14:28
*** ysandeep|rover is now known as ysandeep|afk14:31
*** marios is now known as marios|call14:32
*** blueDuckRandom has joined #oooq14:37
rfolcomarios|call, sshnaidm|afk arxcruz zbr: created the retro card, feel free to comment/+1/-1 https://trello.com/c/9GjXhQxm/1228-equalize-scrum-between-shallow-and-deep14:40
*** beagles_mtg is now known as beagles14:46
*** sshnaidm|afk is now known as sshnaidm14:51
sshnaidmweshay_, can you join #rdo?14:53
weshay_sshnaidm, aye.. woke up to a irc issues14:54
*** dtantsur|brb is now known as dtantsur14:58
rlandymarios: https://review.opendev.org/#/c/71667015:01
*** ysandeep|afk is now known as ysandeep15:13
*** marios|call is now known as marios15:20
mariosrfolco: thanks15:21
*** ykarel is now known as ykarel|away15:28
*** ysandeep is now known as ysandeep|away15:39
mariosq15:48
* marios shutdown sequence15:51
*** marios has quit IRC16:04
*** dtantsur is now known as dtantsur|afk16:11
*** holser has quit IRC16:18
*** holser has joined #oooq16:20
*** udesale_ has quit IRC16:30
*** skramaja has quit IRC16:52
*** blueDuckRandom has quit IRC16:56
zbrarxcruz: panda https://review.opendev.org/#/c/717723/17:04
*** rlandy is now known as rlandy|lunch17:14
*** amoralej is now known as amoralej|off17:24
*** dpawlik has quit IRC17:51
*** chem has quit IRC17:53
*** chem has joined #oooq17:55
*** saneax has quit IRC17:56
*** rlandy|lunch is now known as rlandy18:03
*** jmasud has quit IRC18:13
*** derekh has quit IRC18:24
weshay_rlandy, you have a few?18:26
rlandyweshay_: yes18:26
weshay_https://meet.google.com/pby-txje-dgu?authuser=118:27
*** beagles is now known as beagles_afk18:31
rlandyweshay_: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/tenant_vars/rdo_common.yml18:32
*** jmasud has joined #oooq18:37
rlandyweshay_: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/tenant_vars/rdo_common.yml18:38
*** holser has quit IRC19:23
*** ccamacho has quit IRC20:15
*** jfrancoa has quit IRC21:22
*** rlandy is now known as rlandy|afk21:50
*** rfolco has quit IRC22:03
*** TrevorV has quit IRC22:24
*** sshnaidm is now known as sshnaidm|afk22:28
*** tosky has quit IRC23:28

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!