Wednesday, 2019-04-10

*** Goneri has quit IRC00:22
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci- (2 more messages)00:35
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci- (2 more messages)02:35
*** dsneddon has quit IRC02:44
*** dsneddon has joined #oooq02:47
*** udesale has joined #oooq03:26
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci- (2 more messages)04:35
*** openstackstatus has quit IRC04:35
*** openstackstatus has joined #oooq04:35
*** ChanServ sets mode: +v openstackstatus04:35
*** ykarel|away has joined #oooq04:39
*** ykarel_ has joined #oooq04:44
*** ykarel|away has quit IRC04:47
*** quiquell|off is now known as quiquell|rover04:48
quiquell|roverzbr: done04:48
*** ykarel_ is now known as ykarel04:56
ykarelquiquell|rover, can you check comment on https://review.rdoproject.org/r/#/c/20051/04:56
ykarelweshay, there was a typo for stein packstack job ^^04:57
*** marios has joined #oooq04:59
*** ratailor has joined #oooq05:01
quiquell|roverykarel: ack, will fix the typo and then reactivate the job05:05
quiquell|roverThanks05:05
ykarelack05:06
*** ykarel_ has joined #oooq05:09
*** ykarel has quit IRC05:11
*** holser_ has joined #oooq05:33
*** holser_ has quit IRC05:34
*** dsneddon has quit IRC05:44
*** dsneddon has joined #oooq05:45
*** ykarel_ is now known as ykarel05:50
*** matbu has quit IRC05:57
*** matbu has joined #oooq05:57
*** fultonj has quit IRC05:58
*** saneax has joined #oooq05:58
*** jfrancoa has joined #oooq06:06
*** fultonj has joined #oooq06:08
*** kopecmartin|off is now known as kopecmartin06:10
*** dsneddon has quit IRC06:16
chandankumarmarios: quiquell|rover \o/06:27
quiquell|roverchandankumar: o/06:27
chandankumarmarios: quiquell|rover please have a look at these patches https://review.rdoproject.org/r/20037 and https://review.rdoproject.org/r/1991606:27
chandankumarthanks!06:27
quiquell|roverchandankumar: have to drop now for a few will look at them (for sure this time)06:28
*** quiquell|rover is now known as quique|rover|brb06:28
chandankumarquique|rover|brb: great thanks!06:29
*** apetrich has joined #oooq06:30
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci- (2 more messages)06:35
*** skramaja has joined #oooq06:41
*** dsneddon has joined #oooq06:41
mariosack chandankumar will check06:53
*** sshnaidm|pto is now known as sshnaidm07:06
*** quique|rover|brb is now known as quiquell|rover07:11
*** holser_ has joined #oooq07:21
*** tosky has joined #oooq07:28
*** panda has joined #oooq07:33
*** gkadam has joined #oooq07:34
*** gkadam has quit IRC07:34
*** amoralej|off is now known as amoralej07:35
*** saneax has quit IRC07:42
*** ccamacho has joined #oooq07:45
quiquell|roverykarel: tripleo-specs is ok now ? https://review.openstack.org/#/c/651440/07:47
*** saneax has joined #oooq07:50
*** jpena|off is now known as jpena07:52
*** quiquell|rover is now known as quique|rover|bbl07:54
*** ykarel is now known as ykarel|lunch07:59
*** bogdando has joined #oooq08:02
pandamarios: 'morning08:14
mariospanda: yes confirm08:15
pandamarios: confirm it's morning ? That's reassuring.08:16
*** saneax has quit IRC08:19
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci- (2 more messages)08:35
*** derekh has joined #oooq08:35
arxcruzchandankumar: ping08:43
arxcruzchandankumar: what's the tempest_service for telemetry ?08:44
chandankumararxcruz: it is aodh08:53
chandankumararxcruz: Here is the list of services https://github.com/openstack/openstack-ansible-os_tempest/blob/master/vars/redhat-7.yml#L2908:55
pandamarios: you available to chat ?08:56
mariospanda: sure what's the topic. not _right_ now though middle of something can we schedule it half hour or so?08:57
*** dtantsur|afk is now known as dtantsur08:58
mariospanda: if its about the google doc i saw your comment i already have it in my list of things to check later08:58
zbrsshnaidm: can you help reviewing the image promoter fixes https://review.rdoproject.org/r/#/c/19966/ ? tx08:59
pandamarios: mainly if you need task offload. I didn't understand your comment before "yes confirm"08:59
chandankumararxcruz: I am taking care of plain standalone job09:00
chandankumararxcruz: rest is 2 and 4 you can take care of. :-)09:00
mariospanda: ack i think am ok with tasks trying to update taiga/stories/dod etc but don't need to offload09:01
mariosbrb09:02
*** ykarel|lunch is now known as ykarel09:04
arxcruzchandankumar: are you sure it is aodh and not ceilometer?09:05
arxcruzor both ?09:05
arxcruzchandankumar: i think it's both09:05
*** saneax has joined #oooq09:08
chandankumararxcruz: you can take any one, it will install the tempest plugins09:09
chandankumararxcruz: feel free to add a test job against rdo jobs https://review.rdoproject.org/r/#/c/20038/ something like this!09:12
arxcruzalready did09:14
chandankumararxcruz: cool, thanks!09:17
*** quique|rover|bbl is now known as quiquell|rover09:50
zbrquiquell|rover: can you access https://review.rdoproject.org/r/#/admin/projects/rdo-infra/ci-config,access without getting an error?10:18
quiquell|roveryep10:19
zbrquiquell|rover: lucky, you I think the ACLs are messes on that group as I cannot access it.10:20
zbrquiquell|rover: please go to General and assure it has "Make group visible to..."10:20
zbri think that is what is causing the issue10:20
quiquell|roverzbr: better as RDO guys10:21
quiquell|roverzbr: We are not suppose to change those ghinghgs10:21
zbralready asked there, they ended up with the same issue.10:22
zbrmainly only one group listed on ACL page is browsable, and there is no reason for not having all groups browsable.10:22
zbrwithout this a user would not know who is core on a project.10:23
zbris about group config, not project condig.10:23
zbrmainly rdo-ci-core works now as a secret society, where we cannot even see the members :P10:24
quiquell|roverzbr: I cannot change anything at General10:25
zbrquiquell|rover: good to know, but you see the option there right?10:26
quiquell|roverNope10:26
zbrquiquell|rover: look at another group that is correctly configured: https://review.rdoproject.org/r/#/admin/groups/2799,info10:26
zbr"Group Options" - checked.10:26
zbrmaybe these groups are messed somehow. I observe that they are not using the prefix like the one that works.10:27
zbri think I found the issue. these groups do not exist! somehow the project uses invalid config. correct groups are config-core -- see https://review.rdoproject.org/r/#/admin/groups/6,members10:28
zbri can access that group without issues10:29
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-scenario001-multinode-oooq-container, tripleo-ci- (2 more messages)10:35
zbrquiquell|rover: can you please me what groups are listed for you at https://review.rdoproject.org/r/#/settings/group-memberships ?10:40
zbrother than the obvious/default ones.10:40
*** saneax has quit IRC10:41
*** saneax has joined #oooq10:41
quiquell|roverzbr: anon, registerd, config-core, rdo-ci-core, rdo-infra/ansible-role-tripleo-ci-reproducer, stream-authorized10:45
zbrquiquell|rover: aha, thanks. so that group (rdo-ci-core) somehow exists somewhere but is not visible to non-member.10:46
zbrquiquell|rover: thanks, sorted with help from jpena. now both groups are browsable.11:07
quiquell|roverzbr: cool cool11:08
zbrrfolco: see my last comment on https://review.rdoproject.org/r/#/c/20050/ - makes sense?11:18
*** udesale has quit IRC11:24
*** panda is now known as panda|lunch11:24
*** udesale has joined #oooq11:24
*** holser_ is now known as holser|lunch11:33
*** jpena is now known as jpena|lunch11:34
weshayquiquell|rover I can cover the program call, ur welcome to join if you are interested11:51
weshayno need to though11:51
quiquell|roverweshay: ack thanks, I am lookging at the f28 containers build issue11:51
weshayk thanks11:51
weshayquiquell|rover btw..  there is some design regarding regarding checking for missing containers that we'll discuss when rlandy is on11:52
quiquell|roverweshay: ack11:54
quiquell|roverweshay: it's not fixed adding criteria to f28 promotions ?11:55
*** rlandy has joined #oooq11:55
weshayquiquell|rover https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/Fedora-28/stein.ini11:55
quiquell|roverweshay: is till happening ?11:56
weshayhttps://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/Fedora-28/master.ini11:56
quiquell|roverok let's wait for rlandy11:56
* weshay looks11:56
rlandyquiquell|rover: here11:56
*** rlandy is now known as rlandy|ruck11:56
weshayquiquell|rover looks to be working .. f28 standalone is green today11:56
quiquell|roverweshay: if containers are not there periodic job is not going to pass so we are not going to push to docker hub11:57
weshayit would be red, if containers would pushed all the time11:57
weshayquiquell|rover understood11:57
quiquell|roverweshay: do we miss something else ?11:57
weshayquiquell|rover no.. it's just a design nuance that we should chat about11:58
quiquell|roverack whenever you want11:58
quiquell|roverrlandy|ruck: is up now too11:59
rlandy|ruckquiquell|rover: ack - program meeting now11:59
*** udesale has quit IRC11:59
rlandy|ruckweshay: ^^ anything you want raised there?11:59
weshayrlandy|ruck you can skip, we're green12:00
weshaythey will blow right past us12:00
rlandy|ruckok12:00
rlandy|ruckwant to meet re: promoter?12:01
weshaykopecmartin chandankumar thanks for the bugzilla work12:01
rlandy|ruckquiquell|rover: ^^?12:01
weshayrlandy|ruck ya. but give me a few12:01
weshaylike 15min12:01
rlandy|rucksure12:01
chandankumarweshay: , yes now down to 3 bugs only12:03
chandankumarweshay: can we get +w on this https://review.openstack.org/#/c/635478/ ?12:05
weshaychandankumar thanks!12:06
*** amoralej is now known as amoralej|lunch12:10
weshayquiquell|rover sshnaidm so re: standalone repro on libvirt.. wondering if you think this is a tripleo bug or ci bug12:22
weshayhttps://nimb.ws/Yj9ITe12:22
weshaythe image is missing lvm rpms12:22
quiquell|roverweshay: its libvirt bug I think sshnaidm has solution for it12:22
weshaymissing lvm212:22
weshaylvm2.rpm12:22
weshayquiquell|rover shouldn't tripleo ensure that is installed?12:23
quiquell|roverweshay: they do but in the images12:23
weshayas a dep on tripleo-client12:23
sshnaidmweshay, seem like it's preinstalled on images12:23
quiquell|roverweshay: here we use vanilla ones at libvirt12:23
sshnaidmquiquell|rover, weshay I have solution for CI only, need to add it to bindep12:23
weshayya.. but so would a customer12:23
weshaysshnaidm but hold up a sec12:23
weshayI think that may be a tripleo booog12:23
zbr.... role testing could prevent it /end-promotion12:24
quiquell|roverweshay: but maybe you are right about ading dep at distgit12:24
weshayif we fix it in ci.. users would hit this afaik12:24
sshnaidmI'm not sure it's tripleo related, because it's for some CI stuff for cinder iirc12:24
sshnaidmif it's for CI only, tripleo don't need it12:24
weshaythat task is a standalone task12:25
weshaynot a ci task that is failing12:26
quiquell|roverweshay: will try to put a patch for that as rr12:26
weshayquiquell|rover hold on it.. /me asking ooo folks too12:28
quiquell|roverweshay: acl12:28
weshay<EmilienM> add host_prep_task in cinder/cinder-common-container-puppet.yaml12:28
weshaylet's fix it in ooo, if we want to patch ci temp.. that is fine by me too12:29
*** jpena|lunch is now known as jpena12:31
weshayquiquell|rover rlandy|ruck I'm in my blue12:33
rlandy|ruckjoining12:33
quiquell|roverme too12:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb- (2 more messages)12:35
*** udesale has joined #oooq12:36
*** panda|lunch is now known as panda12:37
*** zul has joined #oooq12:38
ykarelquiquell|rover, looks like u missed the typo part stein packstack job: https://review.rdoproject.org/r/#/c/20051/12:39
*** Goneri has joined #oooq12:42
*** mjturek has joined #oooq12:53
*** jfrancoa has quit IRC12:54
weshayrlandy|ruck https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/container-push.yml#L5412:55
weshayrlandy|ruck https://github.com/openstack-infra/tripleo-ci/blob/aed534c529e5ba279195b36b520540d5f74a6d6f/playbooks/tripleo-buildcontainers/templates/build.sh.j212:55
weshayrlandy|ruck https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py12:56
*** yolanda_ has joined #oooq12:58
rfolcoweshay, quiquell|rover ping emit_releases quick chat13:00
rfolcoin my blue world https://bluejeans.com/587845809713:00
quiquell|roverrfolco: going13:00
*** udesale has quit IRC13:01
quiquell|roverykarel: I forgot, busy morning, will address that13:01
ykarelquiquell|rover, no hurry but good to add in ruck rover etherpad, so it's not forgotten13:02
quiquell|roverykarel: done13:02
ykarelack thanks13:02
*** irclogbot_3 has joined #oooq13:02
*** udesale has joined #oooq13:03
*** panda has quit IRC13:03
*** altlogbot_0 has joined #oooq13:06
*** jfrancoa has joined #oooq13:06
*** panda has joined #oooq13:08
pandaDoes anyone needs help in any task ?13:11
panda/US13:12
arxcruzweshay: panda quiquell|rover marios rlandy|ruck https://review.rdoproject.org/r/#/c/20059/13:16
arxcruzwhen you guys have some time13:16
quiquell|roverweshay: will try to put a patch for that as rr13:16
quiquell|roverups that random copy paste13:17
quiquell|roverpanda: https://bugs.launchpad.net/tripleo/+bug/182414113:17
openstackLaunchpad bug 1824141 in tripleo "fedora28 kolla container build job missing python-networking-ovn" [Undecided,In progress] - Assigned to Quique Llorente (quiquell)13:17
quiquell|roverpanda: review https://review.openstack.org/65151513:17
rlandy|ruckarxcruz: left a question. looks ok - not sure why only changing scenario00213:21
*** amoralej|lunch is now known as amoralej13:21
*** aakarsh has joined #oooq13:24
*** holser|lunch is now known as holser_13:24
chandankumarrlandy|ruck: panda: https://review.rdoproject.org/r/#/c/20061/ https://review.rdoproject.org/r/#/c/20037/ https://review.rdoproject.org/r/#/c/19916/ feel free to take a look13:30
chandankumarwhen free!13:30
arxcruzrlandy|ruck: we are chanting each scenario separately13:30
pandaarxcruz: interesting, are you using witchcraft ?13:30
rlandy|rucklol - if only13:31
rlandy|ruckarxcruz: understood13:31
rlandy|ruckarxcruz: ready for w+?13:32
rlandy|ruckdone13:34
quiquell|roverrfolco: fixed for your -1 https://review.openstack.org/65149013:37
rfolcoquiquell|rover, looking13:37
*** quiquell|rover is now known as quiquell|lunch13:38
*** ratailor has quit IRC13:43
*** vinaykns has joined #oooq13:43
*** yolanda_ has quit IRC13:49
rlandy|ruckquiquell|lunch: you are right - we would have to get the vars from tripleo-ci to filter tripleo-common13:51
vinayknsweshay: hey, now that dlrn_hash issue is resolved, but during my overcloud deployment I'm hitting http://pastebin.test.redhat.com/75166313:56
*** aakarsh|2 has joined #oooq14:00
*** quiquell|lunch is now known as quiquell|rover14:02
*** aakarsh has quit IRC14:02
weshayvinaykns that looks like a real bug to me, what featureset are you using?14:03
vinayknsweshay: master-tripleo-ci14:04
weshayvinaykns which featureset config file?14:05
weshaynot release14:05
sshnaidmquiquell|rover, https://review.openstack.org/#/c/651185/14:05
*** aakarsh|2 is now known as aakarsh14:07
vinayknsweshay: I'm not sure if I get it..I guess you're asking for this..I'm using 3ctlr_1comp_1supp.yml  and config as ipa.yml14:08
quiquell|rover sshnaidm +214:08
*** ykarel is now known as ykarel|away14:11
weshayvinaykns what's the full command you are using w/ quickstart14:13
vinayknsbash quickstart.sh -R master-tripleo-ci -v --tags all --teardown all --nodes config/nodes/3ctlr_1comp_1supp.yml --config config/general_config/ipa.yml -p quickstart-extras.yml -e undercloud_disk=100 -e undercloud_undercloud_ntp_servers=clock.redhat.com $VIRTHOST14:14
quiquell|roverrfolco, zbr: ternary cannnot be used at https://review.openstack.org/#/c/65149014:14
mariosweshay: standalone-upgrade green at https://review.openstack.org/#/c/651213/ (the depends-on fixed it) but didnt' validate anything yet just fyi14:14
quiquell|roverI am back to adding a small task since this is a promotion blocker we need to merge14:14
weshaymarios \0/14:15
quiquell|roverIf you want I can put a block to reuse condition14:15
*** Vorrtex has joined #oooq14:15
weshayvinaykns can you have sshnaidm review https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/ipa.yml14:15
zbrquiquell|rover: you can use {% if %}14:15
rfolcoquiquell|rover, can't use default value if the var is undefined ?14:16
weshaysshnaidm is that config up to date w/ fs039 and /me wonders if vinaykns should use a diff fs to run w/ tls14:16
zbrquiquell|rover: i found use of ternary to work  in very few cases14:16
quiquell|roverzbr: ok14:16
quiquell|roverrfolco: no14:16
quiquell|roverzbr: I don't like mixing too much if statements but let's see if it's readable14:16
zbrquiquell|rover: luckily is easy to test this in a separated playbook until we make it work14:17
quiquell|roveryep14:17
zbrif can give it a go if you want14:17
quiquell|roversandboxing is easy here14:17
*** Vorrtex has quit IRC14:18
*** Vorrtex has joined #oooq14:19
sshnaidmweshay, ipa.yml should work for oooq14:21
ykarel|awayweshay, vinaykns i see: Ansible timed out at 3597 seconds. so seems just timeout issue14:21
sshnaidmweshay, featureset is only for reproducing ci job14:21
sshnaidmweshay, vinaykns you should use 1ctlr_1comp_1supp.yml for quickstart, not 3ctrl14:22
vinayknssshnaidm: but I would need to have 3 ctlrs for testing the changes14:23
sshnaidmvinaykns, /quickstart.sh --no-clone --teardown all --clean -p quickstart-extras.yml -N config/nodes/1ctlr_1comp_1supp.yml -c config/general_config/ipa.yml -R master-tripleo-ci -w /home/josorior/local_quickstart --tags all  $VIRTHOST14:24
vinayknssshnaidm: is it something like it's recommended ot it doesn't with that node14:24
vinaykns..type config14:24
sshnaidmvinaykns, 3ctrl wasn't tested on libvirt, not sure anybody tried it14:24
weshayvinaykns imho raise a tripleo bug.. sshnaidm vinaykns we exercise ipa / tls in https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset03914:25
sshnaidmweshay, which bug..?14:25
quiquell|roverzbr, rfolco: better now I think https://review.openstack.org/65149014:25
quiquell|roversandbox is working too14:25
weshayhttp://pastebin.test.redhat.com/75166314:25
ykarel|awayweshay, Ansible timed out at 3597 second in ^^, still tripleo bug?14:26
ykarel|awaymay be just need to increase timeout14:27
weshayykarel|away it's not a ci job14:27
ykarel|awayovercloud deploy also has timeout set14:27
weshayvinaykns correct me if I'm wrong14:27
weshayright14:27
ykarel|awayweshay, i meant --timeout at overcloud deploy14:27
ykarel|awayor config download is not using it?14:28
ykarel|awayi think quickstart var is also there14:28
weshayvinaykns what does the next sprint look like for you14:28
weshayvinaykns now that we can run baremetal in tripleo are you prepared to help us bring your perf jobs to proper tripleo ci jobs?14:28
* weshay hopes14:29
vinayknsweshay: yeah sure..!14:31
ykarel|awayquickstart var: deploy_timeout14:31
* ykarel|away leaving14:31
vinayknsbut I'm not sure what would I do that14:31
vinayknss/what/how14:31
weshayvinaykns ya.. we would help.. basically it would get you aligned w/ all the upstream ci14:31
weshayvinaykns if you like I chat w/ you about it in about 35min14:32
weshayvinaykns I think it would make your life easier quite a bit14:32
quiquell|roverweshay: we can  make voting f28 standalone https://review.openstack.org/#/c/651230/14:33
vinayknsweshay: Yeah sure, ping me when are ready..14:33
quiquell|roverweshay: is passing now14:33
*** Vorrtex has quit IRC14:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb- (2 more messages)14:35
vinayknsykarel|away: I don't think increasing the timeout value would be helpful..14:35
*** ykarel|away has quit IRC14:35
arxcruzweshay: panda what do i need to get a +w on https://review.rdoproject.org/r/#/c/20059 ? :D14:37
quiquell|roverarxcruz: there is a comment from chandankumar14:38
arxcruzquiquell|rover: the comment was from patchset 314:38
arxcruzif you see the latest patch is already fixed14:38
arxcruzand he even +2 :)14:39
quiquell|roverok will workflow it14:39
arxcruzquiquell|rover: muchas gracias mi amigo14:39
quiquell|roverarxcruz: np14:39
quiquell|roverrlandy|ruck: so let's just download the yaml like we do with tripleo-common containers file14:40
pandaarxcruz: you have to chant for me14:40
rlandy|ruckquiquell|rover: got that part - working on the filter piece14:40
quiquell|roverrlandy|ruck: ack, you can run it at your laptop if you copy registry_secrets there14:41
quiquell|roverrlandy|ruck: so we don't mess with promoter14:41
arxcruzpanda: what chant means ?14:41
pandaarxcruz: oh so you didn't get my wonderful joke before14:42
rfolcoarxcruz, entoar cântico14:42
quiquell|roverpanda, rlandy|ruck: Can you help with promoter blocker fix ? https://review.openstack.org/#/c/651490/14:42
* panda insert Hot fuzz shame meme here.14:43
arxcruzachso14:43
arxcruzpanda: there's a video somewhere on internet with me singing this https://www.youtube.com/watch?v=16QXu3E3NEU14:43
arxcruz:P14:43
pandaquiquell|rover: how urgent is it ?14:44
* rfolco could not find it14:44
quiquell|roverpanda: fs030 of periodics is failing there14:44
quiquell|roverpanda: let's just see if CI passes and then workflow it14:45
*** mjturek has quit IRC14:47
weshayhi arxcruz!14:50
weshaythat is all14:50
arxcruzweshay: ?14:51
weshayarxcruz don't people say hi in your country?14:52
arxcruzweshay: not in germany lol14:52
mariosarxcruz: avoid eye contact its a trap14:52
*** aakarsh|2 has joined #oooq14:52
arxcruzIch muss zu meine Deutsches kurs gehen14:53
weshaymarios fyi.. Tengu has a patch that will fix the ocatavia health check, /me -2 onhttps://review.openstack.org/#/c/651213/14:53
weshayvinaykns can you chat for 5 min?14:53
arxcruzdmellado: ^ Ich bin besser jetzt14:53
weshayvinaykns https://bluejeans.com/u/whayutin/14:54
mariosweshay: ack cool. but still depends on the intent... it was originally put there (standalone-upgrade) cos it was a non voting job. tengu may still want to move the validation14:54
*** aakarsh has quit IRC14:54
mariosweshay: i'll tidy that stuff tomorrrow- file a bug for the tripleo-common fix etc and chat to Tengu about it14:54
mariosweshay: but we'll still want that one anyway https://review.openstack.org/651510 cos it fixed the job14:55
weshayvinaykns ?14:55
weshaymarios++14:56
hubbot1weshay: marios's karma is now 1114:56
zbrrlandy|ruck: panda jfrancoa : can we now wf https://review.openstack.org/#/c/650921/ ?14:57
vinayknsweshay: yean on my way14:57
weshaypanda rfolco 5 min late to bug triage14:58
* weshay needs to chat w/ vinaykns14:58
rfolcoweshay, ack14:58
rfolcoweshay, its your bj so ping me and panda when you are free14:59
zbrpanda: btw, I replied to your question about manual run output at https://review.rdoproject.org/r/#/c/19966/ -- hopefully is enough. I would personally prefer to remove the set -x from it as now is already informative enough.15:00
quiquell|roverpanda, weshay, rlandy|ruck: typo as a centoci job https://review.rdoproject.org/r/#/c/20119/15:00
pandarfolco: I see the location is set to yours15:00
rfolcooh15:01
rfolcopanda, you are 100% precisely accurate15:01
*** quiquell|rover is now known as quiquell|off15:01
rfolcopanda, joined :)15:02
rfolcopanda, need to talk to you for 2 min, can you please join my blue magic virtual room?15:03
Tenguweshay: well, it MIGHT correct the thing - but we'd better ensure it's the right root cause. The absence of `pgrep' in some containers is an issue. but apparently, Kamil seems to think there's another one. So if it's OK for you to way until tomorrow, fine.15:03
Tengumarios: -^^15:03
*** mjturek has joined #oooq15:04
Tengumarios: also, I think the new validation can stay (once we have sorted our the healthcheck thingy). Might help raising awareness about crashed containers/services15:04
Tenguit's been there for a while now, and apparently it does actually help showing problems in the CI. So for me... it might be activated on (some|all) voting jobs :)15:05
mariosTengu: ack ok lets talk tomorrow. after we skpped the validation i hit a real problem and posted the depends-on to fix it. if the validations are green sure we can keep them there. i wanted to discuss with you though. for the upgrade job, can we make the validations run at the end, rather than after deploy before upgrade like currently?15:05
Tenguof course I'm open to other feedback ;)15:05
Tengumarios: well, it would be nice to ensure the state of services before the upgrade is fine... ?15:06
Tengubut yeah, if it creates more issues than needed, we can move it to the far end of the upgrade run.15:07
mariosTengu: yeah i guess 'both' ... really i was thinking about it in terms of when you upgrade there ight be services down until the end. but this isn't real world this is ci env and we just deployed a fresh standalone so services _should_ be ok there15:07
*** ykarel has joined #oooq15:07
Tengumarios: same thoughts ;)15:07
Tenguanyway, let's find a time tomorrow for some discussions. We're bot CET right?15:08
mariosTengu: no it was just a thought i had earlier, born mainly from the fact that the validation failing meant it stopped after deployment and the upgrade was never executed. which is why we never saw that other problem which is fixed with https://review.openstack.org/65151015:08
mariosTengu: but i guess it was a valid stop since there apparently is a fix incoming for that northd thing?15:08
mariosTengu: ack i am CET+1 but sure lets talk in the morning15:09
Tengumarios: yeah, early failure, catch things that aren't supposed to crash and the like.15:09
Tengumarios: good :).15:09
TenguI'm going off for today ;)15:09
mariosTengu: ack me too soon have a good one15:09
Tengusee you tomorrow folks !15:09
*** aakarsh|2 is now known as aakarsh15:13
zbrweshay: you should watch #rdo channel, discussion related to ansible 2.815:14
pandazbr: sorry, gerrty keeps messing with me15:16
pandazbr: I had 8 comments in you patch15:16
pandazbr: but for a old PS15:16
zbrpanda: is ok, i do appreciate the effort invested much more. looking now at it.15:18
chandankumarweshay: panda sshnaidm can we get this merged https://review.openstack.org/#/c/650613/ collect-logs cookiecutter patch15:20
mariosrlandy|ruck: sanity check - this look ok to you? https://tree.taiga.io/project/tripleo-ci-board/us/952 (story description - esp... rhel8 right?)15:21
chandankumarsshnaidm: for novajoin https://review.openstack.org/#/c/639324/ does the new reproducer works for fs039?15:22
chandankumarit needs local debugging15:22
mariosrlandy|ruck: posted that for now https://code.engineering.redhat.com/gerrit/167540 (parameterized your playbook here for the version https://code.engineering.redhat.com/gerrit/#/c/167540/1/playbooks/copy-rhos-release-tripleo-quickstart.yaml ) ... no rush just spent some time looking at that one today. trying to finalise the design for tomorrow mostly at this point.15:22
sshnaidmchandankumar, yes15:22
sshnaidmchandankumar, it should work now15:23
chandankumarsshnaidm: ok, I will spin up then :-)15:23
weshayrfolco https://bugs.launchpad.net/tripleo/+bug/181888615:23
openstackLaunchpad bug 1818886 in tripleo "Unexpected API error: Server fault - nova.exception.GlanceConnectionFailed" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz)15:23
sshnaidmchandankumar, tell me how it goes, and if need any help15:24
*** mjturek has quit IRC15:26
sshnaidmchandankumar, your zuul.yaml should look like http://paste.openstack.org/show/749118/15:26
sshnaidmchandankumar, just put your user and pass to rdo cloud instead of these variables15:26
rlandy|ruckmarios: looks ok - run with test to be sure15:28
*** ccamacho has quit IRC15:28
zbrpanda: and now that I have to update it, ok to remove the xtrace?15:30
mariosrlandy|ruck: ack thanks ok will revisit15:31
*** mjturek has joined #oooq15:32
zbrpanda: check https://s3.sbarnea.com/ss/190410-ci-scripts_reviewsorin_sbarnea_zbr_fixpromoter3__.png  and let me know if this output is acceptable.15:34
pandazbr: I would like to see some info on what is the script is renaming. just a single log saying "just updated this link to point to this"15:35
pandazbr: I mean, log on error is ok, but log on success is needed too :)15:36
zbrpanda: sure, easy fix. in my imagination the sftp line with ln -s was enough.15:37
pandazbr: how are you outputting that ?15:38
pandazbr: thought it was the result of the -x somewhere15:39
zbrpanda: done. the screenshot? that is my terminal, which colors stderr in orange. last one was without -x.15:39
chandankumarsshnaidm: I have fired it, I will let you the know the results tomorrow15:39
sshnaidmchandankumar, ack15:39
pandazbr: what is outoutting that INFO ?15:40
pandazbr: what line ?15:40
zbrpanda: the trap15:41
zbrpanda: i would prefer to obliterate the output of mkdirs, they are useless. ok to devnull them?15:42
*** marios has quit IRC15:44
pandazbr: what if the error is legit ? :)15:44
pandazbr: still confused, what's the line that prints the sftp command you're running15:45
zbrpanda: if the folder exists it will display error anyway. if it fails for other reason(?) the worst it can happen is to fail at next step anyway which is not silenced.15:46
zbrpanda: sftp itself prints it!15:46
pandazbr: ok, this is ready16:04
pandazbr: ready to be rewritten in paramiko!!16:04
zbrpanda: v1 or v2 ?16:05
zbri still have a wip change to port one very small project to v2 and is still 6mo+ old and not finished...16:06
pandazbr: v3beta16:06
*** saneax has quit IRC16:07
zbrpanda: i am glad to see that in our team people are not scared to use bash. i know few places where people were trying to rewrite 2-3 lines of bash in 10 ansible tasks just for the sake of it.16:10
zbrusually i would say "less is more", with one errata: "unless is perl".16:11
pandazbr: I usually prefer ansible anyway. For most of the things you get free idempotency and error handling. Even if you don't you have at least a block rescue to handle exception.16:11
*** jfrancoa has quit IRC16:12
pandaI'm trying to withdraw from bash as much as I can. Either you need a oneliner, or you should start using a real programming language.16:12
pandabash just doesn't scale16:12
pandayou put a small script, and you have an unmaintainable mammoth in 6 month.16:13
zbranother one needing rvote https://review.openstack.org/#/c/650921/16:14
*** dtantsur is now known as dtantsur|afk16:15
pandazbr: ip r add ?16:15
zbrpanda: ?16:17
pandazbr: nevermind, rvote not rout.16:17
*** panda has quit IRC16:18
*** derekh has quit IRC16:28
*** bogdando has quit IRC16:34
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb- (2 more messages)16:35
*** jdennis has quit IRC16:35
*** jdennis has joined #oooq16:36
rlandy|ruckoh wow16:44
rlandy|ruckchange in logging16:44
*** altlogbot_0 has quit IRC16:44
*** skramaja has quit IRC16:51
*** aakarsh|2 has joined #oooq17:00
*** ykarel is now known as ykarel|away17:03
*** kopecmartin is now known as kopecmartin|off17:03
*** aakarsh has quit IRC17:03
*** aakarsh|2 has quit IRC17:03
*** aakarsh|2 has joined #oooq17:03
*** jpena is now known as jpena|off17:07
*** aakarsh|2 is now known as aakarsh17:08
weshayrlandy|ruck ?17:11
weshaysshnaidm you cool w/ something like this? https://review.openstack.org/#/c/651364/17:14
sshnaidmweshay, mm.. need to improve it17:15
sshnaidmweshay, not sure if guest service is always good detector17:16
sshnaidmweshay, also I personally use my local docker registry and I know people that do the same17:16
weshaysshnaidm it's only for repro libvirt17:16
weshaynot quickstart.sh libvirt17:16
weshayah crud17:16
sshnaidmweshay, using local docker mirror is even faster than docker.op :)17:17
sshnaidms/op/io/17:17
weshaysshnaidm , this should fix standalone libvirt... https://review.openstack.org/#/c/651612/17:17
weshaysshnaidm ya.. you make a good point17:17
sshnaidmweshay, maybe by default worth to turn it off, but need an option to set it to local one17:18
weshayk17:18
sshnaidmweshay, can you vote pls? https://review.openstack.org/#/c/65118517:19
weshaysshnaidm while we're merging https://review.openstack.org/#/c/650613/17:21
weshaymight as well17:21
*** amoralej is now known as amoralej|off17:22
sshnaidmweshay, what are all these docs/static/ovb-openstack-cloud ?17:23
weshaysshnaidm that creates the auto doc17:23
*** udesale has quit IRC17:23
weshaysshnaidm https://github.com/openstack/tripleo-quickstart-extras/tree/master/roles/collect-logs/docs/static17:24
sshnaidmweshay, ok, commented17:26
weshayrock17:26
weshayrlandy|ruck looks like luca has jobs passing https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds17:53
rlandy|rucksec17:53
*** Vorrtex has joined #oooq18:09
aakarshrlandy|ruck++18:12
hubbot1aakarsh: rlandy|ruck's karma is now 118:12
aakarshjust saw the email18:13
aakarshre: SF for baremetal jobs18:13
rlandy|ruckweshay: sorry - really close to getting filtering patch18:14
rlandy|ruckwill look in a sec18:14
*** tosky has quit IRC18:21
*** Goneri has quit IRC18:24
*** zul has quit IRC18:28
*** holser_ has quit IRC18:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039, tripleo-ci-centos-7-ovb- (2 more messages)18:35
rlandy|ruckweshay: can we bj for a few re:filtering containers18:44
weshayrlandy|ruck aye18:48
weshayhttps://bluejeans.com/u/whayutin/18:48
rlandy|ruckjoining18:48
*** ykarel|away has quit IRC19:00
-openstackstatus- NOTICE: Restarting Gerrit on review.openstack.org to pick up new configuration for the replication plugin19:03
weshayrlandy|ruck http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-centos-7-stein-containers-build-push/8612940/logs/build.log.txt.gz19:12
*** Goneri has joined #oooq19:25
weshayrlandy|ruck https://review.openstack.org/#/c/651612/19:33
*** rlandy|ruck is now known as rlandy|ruck|brb19:41
*** mjturek has quit IRC19:44
*** rlandy|ruck|brb is now known as rlandy|ruck20:09
weshayrlandy|ruck ya.. there was a total spike in bad stacks/nodes but looks like it resolved20:33
weshayalso see some left over node failures20:33
weshayetc..20:33
weshaynothing too crazy yet20:33
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb- (2 more messages)20:35
*** Goneri has quit IRC20:36
rlandy|ruckweshay: yeah - looking better20:36
weshayrlandy|ruck http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1&fullscreen&panelId=20920:37
*** Goneri has joined #oooq20:49
rlandy|ruckspreadsheet update complete20:59
rlandy|rucksettings update in progress20:59
*** holser_ has joined #oooq21:02
*** holser_ has quit IRC21:04
*** holser_ has joined #oooq21:04
rlandy|ruckweshay: going to approve https://review.rdoproject.org/r/#/c/19966/ - pls take a look as well21:08
rlandy|ruckzbr: ^^21:17
rlandy|ruckweshay: wrt hw - I have updated the spreadsheet but we don't have provisioning/external networks info yet. Asking Dan Sneddon21:47
weshayk sec21:48
weshayack21:49
*** holser_ has quit IRC21:57
*** aakarsh|2 has joined #oooq21:59
weshayand the fun begins22:00
*** aakarsh has quit IRC22:02
*** vinaykns has quit IRC22:05
*** tosky has joined #oooq22:26
weshayrlandy|ruck either way.. the status quo is ok.. while we figure it out22:28
rlandy|ruckweshay: fine - so leaving my work - probably not the way to go22:29
rlandy|ruckwill leave notes for quiquell|off22:29
*** aakarsh|2 has quit IRC22:35
hubbot1FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-scenario008-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb- (2 more messages)22:35
*** tosky has quit IRC22:47
*** holser_ has joined #oooq22:50
rlandy|ruckweshay: ok if we try https://review.openstack.org/651662 with push set to false?22:52
weshayrlandy|ruck does the container build check job run against tripleo-ci22:53
weshayit should22:53
weshayrlandy|ruck you don't need a test change22:53
weshayright?22:53
rlandy|ruckright22:53
rlandy|ruckbut not sure what that test job does22:53
rlandy|ruckguess we will find out22:54
weshayrlandy|ruck it builds w/o push22:54
weshayrlandy|ruck not sure what you mean22:54
weshayrlandy|ruck a test is running now22:55
weshayhttp://zuul.openstack.org/stream/44c9d9a0ba074554a462665476f105ba?logfile=console.log22:55
rlandy|ruckoh we have a buildah job22:56
rlandy|ruckcool22:56
* rlandy|ruck watches22:56
*** holser_ has quit IRC23:20
*** rlandy|ruck has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!