Tuesday, 2019-06-18

*** Goneri has quit IRC00:31
*** aakarsh has joined #oooq01:14
*** apetrich has quit IRC01:57
*** aakarsh has quit IRC02:42
*** udesale has joined #oooq04:00
*** ykarel|away has joined #oooq04:53
*** dtrainor_ has quit IRC05:02
*** saneax has joined #oooq05:10
*** saneax has quit IRC05:19
*** raukadah is now known as chandankumar05:21
*** marios has joined #oooq05:24
*** ykarel|away has quit IRC05:26
*** ykarel|away has joined #oooq05:39
*** ykarel|away is now known as ykarel05:40
*** ratailor has joined #oooq05:40
*** dtantsur|afk is now known as dtantsur05:42
*** udesale has quit IRC05:51
*** udesale has joined #oooq05:52
*** ykarel_ has joined #oooq05:52
*** ykarel has quit IRC05:55
*** skramaja has joined #oooq05:55
*** jfrancoa has joined #oooq06:03
*** brault has joined #oooq06:15
*** ratailor has quit IRC06:17
*** ratailor has joined #oooq06:19
*** ykarel__ has joined #oooq06:32
*** ykarel_ has quit IRC06:34
chandankumarmarios: Good morning06:46
chandankumarmarios: yesterday I tried to put some code in your container build patch to pull rhos-release patch but failed06:47
chandankumarmarios: rlandry suggested to use debug or condition for the same06:47
chandankumarwhen there is a depends on of rhos-release exists06:47
marioschandankumar: o/06:57
marioschandankumar: maybe we should do some sync today before/after scrum wdyt?06:58
chandankumarmarios: sure06:58
chandankumarmarios: https://code.engineering.redhat.com/gerrit/#/c/173689/06:59
chandankumarplease have a look at this patch06:59
marioschandankumar: ack i think nit?07:04
marioschandankumar: but nice idea07:04
*** dsneddon has quit IRC07:05
chandankumarmarios: thanks to wes for the idea07:05
*** apetrich has joined #oooq07:05
*** udesale has quit IRC07:11
*** udesale has joined #oooq07:11
*** tosky has joined #oooq07:15
*** tesseract has joined #oooq07:24
*** amoralej|off is now known as amoralej07:25
*** dsneddon has joined #oooq07:29
*** saneax has joined #oooq07:33
*** bogdando has joined #oooq07:37
*** ykarel__ is now known as ykarel|lunch07:40
*** jpena|off is now known as jpena07:43
*** kopecmartin|off is now known as kopecmartin07:49
chandankumarmarios: I am updating this patch 172857 for rhos-release stuff07:55
marioschandankumar: ack... i am going to revisit that today for the ceph stuff at least (i think opstools still tbd hopefully via delorean-deps)07:56
marioschandankumar: checking the updates07:56
marioschandankumar: (ack rhos-release from master?)07:57
chandankumarmarios: yes07:57
*** dalvarez has quit IRC08:12
*** amoralej has quit IRC08:12
*** jpena has quit IRC08:15
*** radez has quit IRC08:16
*** ykarel|lunch is now known as ykarel|08:35
*** ykarel| is now known as ykarel08:36
*** amoralej has joined #oooq08:37
*** jpena has joined #oooq08:44
*** derekh has joined #oooq08:47
*** sshnaidm|afk is now known as sshnaidm09:04
*** holser_ has joined #oooq09:04
chandankumarmarios: so09:08
chandankumarmarios: we can now install rhos-release patches in our jobs09:08
chandankumarmarios: https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/15/check/tripleo-buildimage-overcloud-train-rhel-8/7e5505e/job-output.txt.gz#_2019-06-18_09_03_44_22065709:08
chandankumarmarios: but before this, we need to enable proper repo na?09:08
marioschandankumar: python3-tripleoclient09:11
chandankumarmarios: updated09:11
marioschandankumar: see steps at https://tree.taiga.io/project/tripleo-ci-board/task/1114 (scroll down to 'reviews')09:11
marioschandankumar: but how are we consuming that i mean is it depends on09:11
marioschandankumar: or you added into build like you were talking about09:12
* marios brb09:12
marioschandankumar: no depends on didn't work anyway sorry09:12
*** ratailor has quit IRC09:14
chandankumarmarios: depends on pulled there09:14
marioschandankumar: cool but you did something for that right?09:18
marioschandankumar: we should make that conditional https://code.engineering.redhat.com/gerrit/#/c/172857/10..16/playbooks/build-image/pre.yaml09:20
chandankumarmarios: yes, will do that09:23
chandankumargive me few mins, magic will happen09:23
marioschandankumar: ack thanks09:24
*** panda|off is now known as panda09:30
chandankumarmarios: so we reached to https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/17/check/tripleo-buildimage-overcloud-train-rhel-8/fb3ec70/job-output.txt.gz#_2019-06-18_09_27_59_19462309:33
chandankumarmarios: run Image build09:33
chandankumarmarios: https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/17/check/tripleo-buildimage-overcloud-train-rhel-8/fb3ec70/job-output.txt.gz#_2019-06-18_09_20_48_13753609:34
chandankumarmarios: Execute rhos-release to install train/master rdo repo] is this one ok?09:34
marioschandankumar: what do you mean09:35
marioschandankumar: i mean its missing ceph09:35
chandankumarmarios: in this one https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/17/check/tripleo-buildimage-overcloud-train-rhel-8/fb3ec70/job-output.txt.gz#_2019-06-18_09_20_48_13753609:35
chandankumarwe have used rhos-release depends on things are working, make to confirm that one is ok or not?09:36
marioschandankumar: yeah i mean it fails otherwise09:37
marioschandankumar: like it tries to resolve to centos809:37
marioschandankumar: for delorean repos09:37
chandankumarmarios: ok09:43
chandankumarmarios: https://code.engineering.redhat.com/gerrit/#/c/172857/ please have a look one more time so that I will address all comments in a single go09:43
chandankumarbrb09:45
*** zbr has joined #oooq09:45
marioschandankumar: added ceph from 15 trunk https://code.engineering.redhat.com/gerrit/#/c/173155/09:50
chandankumarmarios: tested it here https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/18/check/tripleo-buildimage-overcloud-train-rhel-8/2520ab5/job-output.txt.gz#_2019-06-18_09_40_46_26384209:52
chandankumarmarios: https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/18/check/tripleo-buildimage-overcloud-train-rhel-8/2520ab5/job-output.txt.gz#_2019-06-18_05_43_18_265019 and here from where repos are coming09:52
*** zbr has quit IRC09:53
* chandankumar needs to tweak to use these stuff in rhel-8 standalone jobs now09:54
marioschandankumar: ack i updated the dependency issues i'm seeing at https://tree.taiga.io/project/tripleo-ci-board/task/1114 issue #610:05
*** saneax has quit IRC10:06
*** saneax has joined #oooq10:07
*** d0ugal has joined #oooq10:07
*** dsneddon has quit IRC10:18
*** d0ugal has quit IRC10:18
*** ykarel is now known as ykarel|afk10:27
*** d0ugal has joined #oooq10:33
*** zbr has joined #oooq10:38
*** d0ugal has quit IRC10:39
*** d0ugal has joined #oooq10:39
mariosbiab10:41
*** marios has quit IRC10:41
*** derekh has quit IRC10:42
*** derekh has joined #oooq10:42
*** dsneddon has joined #oooq10:46
arxcruzpanda sshnaidm may i have your blessing in https://review.opendev.org/#/c/664926/ ?10:55
*** udesale has quit IRC10:57
sshnaidmarxcruz, featureset021 job should run on change of featureset021.yml10:57
arxcruzsshnaidm i didn't get it...10:58
sshnaidmarxcruz, I mean on that patch we need to see f021 job executed10:58
arxcruzsshnaidm I see...10:58
sshnaidmarxcruz, need to configure f021 job to run on its fs changes10:58
arxcruzsshnaidm i've create a dummy patch, it's on the comments10:59
arxcruzi'll configure f021 to also trigger the job in a separated patch, sounds fine for you?10:59
sshnaidmarxcruz, I see, but better to do it automatically, w/o dummy patches10:59
arxcruzsshnaidm agree, do you mind if it's a separated patch ?10:59
sshnaidmarxcruz, it can't be in this patch, it should be in config/ repo11:00
arxcruzsshnaidm yes, you're right11:00
arxcruzsshnaidm i'll work on that :)11:00
sshnaidmarxcruz, just configure it now and we'll see it running11:00
arxcruzok11:00
sshnaidmarxcruz, btw, should you fail this job if any tests pass? :)11:01
arxcruzlol11:01
arxcruzsshnaidm the idea is to always fail, and when it doesn't, we need to update the skiplist11:01
sshnaidmarxcruz, but if one or two test pass, it still will fail when others fail11:02
sshnaidmit's enough for one test only to fail for job to fail11:02
*** amoralej is now known as amoralej|lunch11:03
chandankumarsshnaidm: Hello11:05
sshnaidmchandankumar, hi11:05
chandankumarsshnaidm: https://code.engineering.redhat.com/gerrit/#/c/173337/11:06
chandankumarsshnaidm: with above patch I am hitting this https://sf.hosted.upshift.rdu2.redhat.com/logs/37/173337/13/check/tripleo-ci-rhel-8-standalone-train/d1eb8b7/job-output.txt.gz#_2019-06-18_07_01_28_87184311:06
chandankumarsshnaidm: I am passing relase_file to train11:06
chandankumarbut it is not taking11:06
arxcruzsshnaidm yes, it will fail, next step is have a tool that will read these results and update automatically the skip list11:06
chandankumarsshnaidm: please have a look11:07
*** ykarel|afk is now known as ykarel11:19
*** mjturek has quit IRC11:23
sshnaidmchandankumar, we don't have rehdat 8 yet: https://github.com/openstack/tripleo-quickstart/tree/master/config/release/tripleo-ci11:23
*** jjoyce has quit IRC11:23
*** mjturek has joined #oooq11:23
sshnaidmchandankumar, oops, sorry, it's not a problem11:24
*** jjoyce has joined #oooq11:26
sshnaidmchandankumar, why is it looking for master file in train job?11:27
chandankumarsshnaidm: I changed the job definition11:28
chandankumarsshnaidm: https://sf.hosted.upshift.rdu2.redhat.com/logs/37/173337/14/check/tripleo-ci-rhel-8-standalone-train/96f042d/job-output.txt.gz#_2019-06-18_07_24_38_34507911:28
chandankumarrhos_release_args is not defined11:28
*** ratailor has joined #oooq11:28
sshnaidmchandankumar, release_file: train-rhel11:29
sshnaidm      release: master11:29
sshnaidmchandankumar, release should be train too then11:29
*** zbr has quit IRC11:29
chandankumarsshnaidm: updated11:31
*** jpena is now known as jpena|lunch11:32
*** marios has joined #oooq11:36
chandankumarsshnaidm: https://sf.hosted.upshift.rdu2.redhat.com/logs/37/173337/15/check/tripleo-ci-rhel-8-standalone-train/e25bbd4/job-output.txt.gz#_2019-06-18_07_44_21_52390711:48
chandankumarsshnaidm: if we use release as train then it looks for master11:48
chandankumarsshnaidm: if we set release to master11:48
chandankumarthen it works11:49
weshayzbr|ruck how goes it? need anything?11:50
*** aakarsh has joined #oooq11:52
zbr|ruckweshay: morning! ok for now.11:55
weshaycool.. :)11:55
*** holser_ is now known as holser|ice_cream11:58
zbr|ruckweshay: you know what prevents https://review.opendev.org/#/c/624838/50 from merging?11:58
*** dsneddon has quit IRC11:59
zbr|rucki was looking at your refused backport because it cherry picked that one that did not get in11:59
*** rfolco has joined #oooq11:59
zbr|rucki am inclined to remove the wip on it and rephrase it to ease its merge12:00
weshayhrm.. /me looks12:00
weshayzbr|ruck I think that it's fedora12:01
*** dsneddon has joined #oooq12:02
zbr|ruckyeah, what causes the promotion to fail on stein. if we do not fix it bugs will slip in.12:02
zbr|ruckanyway, seems easy to fix.12:02
weshayzbr ya.. I'd be up for removing the stein fedora work all around12:03
weshayleave master until we have rhel in place12:03
*** ykarel is now known as ykarel|afk12:04
zbr|ruckweshay: i was trying to say that unless that one goes in, you have zero chance of merging your cherry pick to stein.12:08
weshayzbr|ruck I suspect if you ask Alex on the TripleO mtg today about f28 stein he'd agree w/ me12:08
zbr|ruckis not that i care much about master, but wanted to address the line #37.12:08
*** dsneddon has quit IRC12:09
weshaymaster would still be useful12:09
weshayas no one else is testing train yet12:09
*** vinaykns has joined #oooq12:10
zbr|ruckweshay: any worries about https://review.opendev.org/#/c/626360/ ?12:12
weshayzbr|ruck no.. that's on the container-registry repo12:15
weshayzbr|ruck I think rlandy fixed stein f28 w/ https://review.opendev.org/#/c/663417/12:16
zbr|ruckweshay: i seen it, correction: attempted to fix it.12:17
zbr|ruckugly approach, but ok considering the situation.12:17
weshayzbr|ruck ya.. what else could one do?12:18
zbr|ruckyou could wf it?12:18
weshayzbr|ruck so I have a minute so I'm trying to understand more clearly where we're running a lot of jobs12:18
weshayzbr|ruck chat w/ me for a hot sec on blue12:19
zbr|ruckjoined.12:19
*** amoralej|lunch is now known as amoralej12:20
*** holser|ice_cream is now known as holser_12:28
*** aakarsh has quit IRC12:31
*** udesale has joined #oooq12:32
chandankumarweshay: rhel8 sync call12:34
*** rlandy has joined #oooq12:40
weshayzbr|ruck https://docs.google.com/document/d/1u2q7E6SceTo5AxJf4vr9mZK08iAGOhQNJQXSYW1nIVM/edit12:40
*** jpena|lunch is now known as jpena12:40
*** dsneddon has joined #oooq12:41
*** aakarsh has joined #oooq12:41
weshaymarios chandankumar rfolco sshnaidm https://docs.google.com/document/d/1MYVNq4IDKambmbn_LfVwwfGG1rvNgaluBEiDYjbi1Bg/edit12:43
*** tcw has joined #oooq12:44
weshayhttps://code.engineering.redhat.com/gerrit/#/c/173724/2/nodepool/elements/virt-customize/rhel8-cloud.yaml12:45
zbr|ruckrlandy: am I wrong to assume that the only thing alex wanted on https://review.opendev.org/#/c/664947/2 -- was to remove -master suffix from the template?12:50
rlandyzbr|ruck; my understand is that  what that review has as the master template, he wants as the default master set of test triggered12:52
rlandy Because this job should be branch specific when used on a stable branch.12:52
rlandy^^ that is his reasoning12:52
rlandyie: that there shouldn't be a case where we run a stein test on the master branch12:53
rlandythe only reason I kept the other one was for our usage12:53
rlandyI think he may be happy if you named the master one  tripleo-build-containers-jobs12:54
rlandythen that one would automatically be picked up12:54
zbr|ruckin this case how about adding tripleo-ci- prefix?12:54
rlandyand then named the top one full or ci or something12:54
zbr|ruckif we add "-ci" it should tag it as ci-related.12:54
rlandyfine12:54
zbr|ruckat least this would relatively be in sync with current usage pattern12:55
rlandythat is what I was planning to do - just ran out of time to take care of it12:55
zbr|rucki will do it.12:55
rlandyzbr|ruck: that template is used in THT12:55
rlandyyou'd would need to modify the branches to use the correct version12:55
zbr|rucki do not understand, i see that this change is adding new templates, i doubt they can be already used.12:56
rlandyzbr|ruck: on THT stein branch12:58
rlandycurrently, iiuc, the whole set runs12:58
rlandyonly stein template should run there12:58
*** ykarel|afk is now known as ykarel12:59
*** ratailor has quit IRC13:00
*** Goneri has joined #oooq13:02
weshayhttps://trello.com/b/pthrrzK5/prod-chain-council13:04
apetrichmarios, we are making a portokalopita for tonight :)13:08
rlandychandankumar: how's it going with the rhos-release job?13:09
chandankumarmarios: https://code.engineering.redhat.com/gerrit/173630 and https://code.engineering.redhat.com/gerrit/17369613:12
chandankumarweshay: https://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/18/check/tripleo-buildimage-overcloud-train-rhel-8/2520ab5/job-output.txt.gz#_2019-06-18_09_40_46_26384213:14
chandankumarweshay: marios rhos-release patchs in action13:14
zbr|ruckrlandy: i updated it, let me know if it looks ok in its new form.13:14
chandankumarhttps://sf.hosted.upshift.rdu2.redhat.com/logs/57/172857/18/check/tripleo-buildimage-overcloud-train-rhel-8/2520ab5/job-output.txt.gz#_2019-06-18_05_43_18_14550213:15
chandankumarlist of enabled repos13:15
chandankumarrlandy: the job is working now13:15
chandankumarrlandy: we need to fix the repo permission to make it voting13:15
rlandyzbr|ruck: iiuc, he wants the naming the other way around13:21
rlandyI added comments - if those don;t make sense, pls ask alex his intention.13:22
rfolcoci community meeting starting now13:31
rfolcohttps://etherpad.openstack.org/p/tripleo-ci-squad-meeting13:31
rfolcohttps://bluejeans.com/587845809713:31
rfolcoping marios, sshnaidm, weshay, panda, rlandy, arxcruz, rfolco, chkumar, zbr, kopecmartin13:31
weshayhttps://docs.google.com/document/d/1u2q7E6SceTo5AxJf4vr9mZK08iAGOhQNJQXSYW1nIVM/edit13:39
*** skramaja has quit IRC13:44
zbr|ruckrlandy: https://review.opendev.org/#/c/664947/4 -- that new version should fix it without breaking anything. i will also update our jobs to use the "-ci" version.13:46
*** udesale has quit IRC13:48
*** udesale has joined #oooq13:49
*** ykarel is now known as ykarel|away13:50
*** dsneddon has quit IRC13:50
rlandyzbr|ruck: that looks right to me13:53
zbr|ruckin this case add a +1, to mark it.13:54
rfolcopanda, you were right. Using the right config release file, standalone tried to get the hash for tripleo-ci-testing....13:54
rfolcohttps://sf.hosted.upshift.rdu2.redhat.com/logs/28/172928/24/check/periodic-tripleo-ci-rhel-7-standalone-rocky/570e480/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-06-18_08_40_1013:54
rfolcopanda, now just running atomically build container job it will work13:55
rfolcoshould work13:55
rfolcopanda, thanks :)13:55
pandarfolco: I had an accident some years ago. I lost the ability to use my left side. I'm always right now.13:56
rfolcopanda, I am so awesome and modest13:56
rfolcoweshay, marios panda sshnaidm chandankumar: is there a story for rhos-release consolidation ? I'll create one14:03
weshayrfolco there is not afaik14:04
pandarfolco: I thnk it's very tied to composability.14:05
rfolcook, we can make it a task np14:06
*** ykarel|away has quit IRC14:06
*** radez has joined #oooq14:07
chandankumarok guys14:10
chandankumarsee ya from blr tomorrow14:10
*** chandankumar is now known as raukadah14:10
marioso/ raukadah14:10
*** dsneddon has joined #oooq14:11
*** dsneddon has quit IRC14:16
*** dsneddon has joined #oooq14:17
*** dsneddon has quit IRC14:22
*** Vorrtex has joined #oooq14:32
weshayzbr|ruck fyi.. i'm going to promote master https://review.rdoproject.org/r/#/c/21178/1/ci-scripts/dlrnapi_promoter/config/CentOS-7/master.ini14:36
*** dsneddon has joined #oooq14:41
*** dsneddon has quit IRC14:46
pandarfolco: 5 minutes sync ?14:47
rfolcopanda, sure let me find a quiet place14:48
pandarfolco: here https://www.blu-ray.com/movies/A-Quiet-Place-Blu-ray/202647/14:49
*** dsneddon has joined #oooq14:49
rfolcopanda, need to be 10 min, have 1-1 in 10 w/ weshay14:50
rfolcopanda, my bj??14:50
pandarfolco: ok14:50
rfolcopanda, I am there14:51
*** blueDuckRandomSt has quit IRC14:57
*** dsneddon has quit IRC14:57
weshaypanda let me know before you leave what you think re: rdo sf jobs15:00
*** blueDuckRandom has joined #oooq15:00
pandaweshay: yep, reviewed already, writing comments on the doc now.15:01
*** holser__ has joined #oooq15:06
*** holser__ has quit IRC15:08
*** holser__ has joined #oooq15:09
*** holser_ has quit IRC15:09
*** dsneddon has joined #oooq15:12
weshaypanda so you are moving forward on refactoring the rdo sf promotion jobs a bit15:16
pandaweshay: yes15:18
*** dsneddon has quit IRC15:19
*** blueDuckRandom has quit IRC15:29
weshaypanda rlandy where is the trigger definition for internal sf?15:31
weshayrfolco ^15:31
*** blueDuckRandom has joined #oooq15:31
pandaweshay: in the pipeline definition15:31
pandaweshay: tripleo-ci-internal-config/zuul.d/pipelines.yaml15:32
weshaypanda which gerrit is that code.eng?15:33
pandaweshay: yes15:33
rlandyweshay: in projects.yaml15:33
rlandyiiuc15:33
*** zbr|ruck is now known as zbr|brb15:34
weshayrfolco https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/tripleo-ci-internal-config.git;a=blob;f=zuul.d/pipelines.yaml;h=0399da48006dc431ef5adcd548c409d79277918d;hb=HEAD#l12315:36
weshaytrigger:15:36
weshay 131       code.engineering.redhat.com:15:36
weshay 132         - event: comment-added15:36
weshay 133           comment: (?i)^(Patch Set [0-9]+:)?( [\w\\+-]*)*(\n\n)?\s*(retrigger)15:36
weshayseems odd to me..15:36
*** hamzy_ has joined #oooq15:38
pandaI think the idea has always been to trigger by polling the rdosf jobs, instead of leave synchronization to time based trigger.15:38
*** vinaykns has quit IRC15:40
*** hamzy has quit IRC15:40
weshaypanda ya.. I just see from the zuul config how that works exactly15:41
weshaypanda is there a review that comments get added to?15:41
*** dtrainor has joined #oooq15:43
*** marios has quit IRC15:44
weshayrfolco $ sudo docker pull trunk.registry.rdoproject.org/tripleorocky/rhel-binary-cinder-api:8befbdbe5af74fcce2f0e5468d9f291e40b65c27_b01817dd15:47
pandaweshay: there are multipl peridic pipelines configured. "periodic', "peridoc-hourly" and "openstack-periodic". Periodic and periodic-hourly are triggered with time, every day at midnight. Openstack-periodic is what you pasted, but I can't see any job using it.15:47
pandaweshay: periodic-hourly is the one used by quique script, and probably what we're goign to use  too15:47
*** hamzy_ is now known as hamzy15:48
*** dsneddon has joined #oooq15:53
weshaypanda one more question15:56
weshaythe jobs that consume containers in the rdo sf periodic jobs15:57
weshaythey must be logging into the rdo registry15:57
weshaybut I don't see it15:57
pandaweshay: you can pull without authentication15:57
*** dsneddon has quit IRC15:57
weshaypanda hrm15:58
pandaweshay: authentication is needed to pull15:58
weshayauth to push15:58
weshayno auth to pull15:58
pandaweshay: yes sory15:58
weshaybut that's not what I'm currently seeing15:58
weshayah...15:58
weshaynow it's working15:58
weshayrfolco that pull is working for me now15:58
weshaypanda I suspect that the infra team may change that...15:59
weshaysuch that auth is required15:59
*** dsneddon has joined #oooq15:59
*** saneax has quit IRC15:59
weshaypanda perhaps we account for that in our refactor?16:00
weshayrfolco rejoin16:01
weshayrfolco ? blue issues?16:06
*** blueDuckRandomSt has joined #oooq16:06
*** dsneddon has quit IRC16:07
rlandyweshay: hi - posting a bug to padman, any issue with adding the + podman info in https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/tripleo-ci-rhel-8-standalone-rhos-15/58bcb4a/logs/undercloud/var/log/extra/podman/podman_allinfo.log.txt.gz to a public external bug?16:07
rlandypodman16:07
rlandy registry.redhat?16:07
*** blueDuckRandom has quit IRC16:09
weshayrlandy sec16:11
rfolcoweshay, still want me to rejoin? I just had lunch16:11
weshayrfolco so panda is going to put in the change to have ALL tripleo jobs in rdo-sf auth to the rdo reg16:11
weshaypanda ^16:11
weshayrlandy /me reads16:12
weshayrlandy that should be fine16:12
rlandyk- thanks16:13
*** Vorrtex has quit IRC16:13
rlandyweshay: we need to think about locking versions ... we are using podman 1.4.116:14
weshayrlandy ?16:19
weshayur pulling osp builds no?16:19
rlandyack16:19
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/tripleo-ci-rhel-8-standalone-rhos-15/58bcb4a/logs/undercloud/var/log/extra/podman/podman_allinfo.log.txt.gz16:20
rlandywe're updating16:20
*** bogdando has quit IRC16:21
weshayrlandy ?16:22
weshayhttps://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/tripleo-ci-rhel-8-standalone-rhos-15/58bcb4a/logs/undercloud/var/log/dnf.log.txt.gz16:22
weshayrlandy what indicates that to you?16:22
weshaythat we're updating16:22
rlandy+ podman version16:22
rlandyVersion:            1.4.116:22
weshaythat looks like the originally installed version I don't see it getting updated16:23
weshayrlandy http://download.devel.redhat.com/nightly/latest-RHEL-8/compose/AppStream/x86_64/os/Packages/16:25
rlandy[   ] podman-1.0.0-2.git921f98f.module+el8.0.0+2956+30df4692.x86_64.rpm                      04-Apr-2019 13:50  9.0M16:29
rlandy[   ] podman-1.4.1-2.module+el8.1.0+3401+5fb5c576.x86_64.rpm16:29
rlandydifference?16:29
weshaywhere do you see that?16:29
rlandythe link you sent :)16:29
* rlandy really doesn't know16:30
weshayrlandy join my blue16:30
rlandyk16:30
rlandyweshay: https://code.engineering.redhat.com/gerrit/#/c/173832/16:32
*** zbr|brb is now known as zbr|ruck16:38
rlandyweshay: trying test https://code.engineering.redhat.com/gerrit/#/c/173832/ - change in ceph parameters16:39
rlandyweshay: https://code.engineering.redhat.com/gerrit/#/c/173835/16:39
rlandyhttp://download.lab.bos.redhat.com/rcm-guest/puddles/OpenStack/15.0-RHEL-8/container_image_prepare.yaml16:40
rlandyweshay: ^^16:40
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/periodic-tripleo-ci-rhel-8-standalone-need-images-rhos-15/dbfdc9f/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz16:40
*** vkapalav has joined #oooq16:46
*** dsneddon has joined #oooq16:49
*** panda has quit IRC16:56
*** altlogbot_1 has quit IRC17:00
*** altlogbot_0 has joined #oooq17:01
*** panda has joined #oooq17:01
*** irclogbot_0 has quit IRC17:02
*** irclogbot_1 has joined #oooq17:02
*** ykarel|away has joined #oooq17:10
*** amoralej is now known as amoralej|off17:10
*** udesale has quit IRC17:11
*** holser__ has quit IRC17:14
*** holser_ has joined #oooq17:14
*** dtantsur is now known as dtantsur|afk17:25
*** kopecmartin is now known as kopecmartin|off17:37
*** jpena is now known as jpena|off17:43
weshayrlandy fyi.. https://review.rdoproject.org/r/#/c/21180/17:45
*** holser__ has joined #oooq17:48
*** holser_ has quit IRC17:52
rlandythanks17:52
*** holser__ has quit IRC17:52
*** holser_ has joined #oooq17:55
*** holser_ has quit IRC17:59
*** ykarel|away has quit IRC17:59
*** holser_ has joined #oooq18:32
*** derekh has quit IRC18:40
*** jfrancoa has quit IRC18:53
*** jfrancoa has joined #oooq19:07
rfolcopanda, still around? do you have a patch for logging into rdo at the parent job in sf ?19:11
*** panda has quit IRC19:33
rlandyweshay: we got somewhere with rhos-15 ...19:41
rlandydeployment passed - failed tempest https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/tripleo-ci-rhel-8-standalone-rhos-15/10aac93/logs/undercloud/home/zuul/19:42
rlandypassed_phase_119:42
rlandyneeds_images not so much19:42
*** panda has joined #oooq19:43
rlandy  start: '2019-06-18 19:01:11.133148'19:48
rlandy  stderr: |-19:48
rlandy    Error: Unknown repo: 'gating-repo'19:48
rlandy    error while running runtime: exit status 119:48
rlandyweshay: ^^ what's the difference here?19:51
rlandypassed_phase_1 vs. needs_images?19:51
weshayhrm.. so https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/tripleo-ci-rhel-8-standalone-rhos-15/10aac93/logs/undercloud/home/zuul/19:52
weshayis passed phase 1 right..19:52
rlandyit never updates19:53
rlandy  stderr_lines: <omitted>19:53
rlandy  stdout: No packages were found for update...19:53
rlandyas opposed to https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/periodic-tripleo-ci-rhel-8-standalone-need-images-rhos-15/70e32c9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz19:53
rlandywhere19:54
rlandy    Error: Unknown repo: 'gating-repo'19:54
rlandy    error while running runtime: exit status 119:54
rlandy  stderr_lines: <omitted>19:54
rlandy  stdout: ''19:54
rlandy  stdout_lines: <omitted>19:54
rlandywe could get rid of gating-repo if it doesn;t exist19:54
rlandybut rdo has no issue with it19:54
weshayrlandy cirros failed here https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/tripleo-ci-rhel-8-standalone-rhos-15/10aac93/logs/undercloud/home/zuul/tempest.log.txt.gz19:54
weshaythat's an image issue19:55
rlandyI hadn't even got to discuss that yet :)19:55
rlandywas going to rerun and see if it repeated19:55
rlandytrying to sort out needs_images now19:55
weshayother one is https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/periodic-tripleo-ci-rhel-8-standalone-need-images-rhos-15/70e32c9/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz19:56
*** holser_ has quit IRC19:56
weshayrlandy https://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/1/check/periodic-tripleo-ci-rhel-8-standalone-need-images-rhos-15/70e32c9/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz19:56
weshayoslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.19:57
rlandymultiple issues there19:57
weshayError: Unknown repo: 'gating-repo'\n    error while running runtime: exit status 1\n  stderr_lines: <omitted>\n  stdout: ''\n  stdout_lines: <omitted>\n\nPLAY RECAP19:57
rlandygating-repo is not set in this job19:58
rlandyor any job where there is not a change that would create such a repo19:58
weshay update_repo: "gating-repo,rhelosp-15.0-trunk,rhelosp-ceph-4.0-mon-pending,rhelosp-ceph-4.0-tools-pending"19:58
weshayright.. so we have to update the params I guess19:59
weshayin the release file19:59
rlandyif gating-repo exists ...19:59
rlandyhow come no other job has an issue with that?19:59
rlandyRDO jobs?19:59
weshaythat I don't know..20:00
rlandywas look how I could set that without adding another ansible task20:00
rlandybecause I don;t want to change rdo20:01
weshayrlandy maybe we add that on the fly20:01
rlandyhttps://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/overcloud-prep-containers/defaults/main.yml20:02
rlandygating_repo_name: gating-repo20:02
rlandyupdate_containers_repo: "{{ gating_repo_name }},delorean-current"20:02
weshay      update_repo: "gating-repo,delorean-current,quickstart-centos-ceph-nautilus"20:02
weshayhttp://logs.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-master/84d1b03/logs/undercloud/home/zuul/containers-prepare-parameters.yaml.txt.gz20:02
weshaydoes not impact rdo jobs20:02
rlandy      update_repo: "gating-repo,delorean-current,quickstart-centos-ceph-nautilus"20:04
rlandyhow so?20:04
rlandycheck standalone20:05
rlandymaybe we just remove gating from needs_images20:05
rlandyalways periodic20:05
rlandyweshay: ^^?20:05
rlandyone way of getting around the problem20:05
* rlandy cheats20:05
weshayya.. that would be my first move20:07
weshayreally we could do that for any osp job20:07
weshaybecause we're not rebuilding the change properly yet anyway20:07
rlandyok - let's do that20:07
weshayrlandy to have to fix tempest cirros as well20:07
* weshay looks at the image build20:07
weshay    cirros_image_url: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img20:08
*** tcw has quit IRC20:09
rlandyhttps://code.engineering.redhat.com/gerrit/#/c/173832/2/config/release/need_images/rhos-15.yml20:09
rlandyupdated20:09
rlandywhat's wrong with that image?20:10
weshaycirros is missing or misconfigured20:11
rlandy  cirros-0.4.0-x86_64-disk.img                       2017-11-19 19:59   12M20:12
rlandythere20:12
weshaywhere?20:13
rlandy: http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img20:13
weshayyes.. that is fine20:14
weshayhttps://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/config/testenv/multinode.yml#L3420:15
weshayrlandy we need to change     cirros_image_cache_file: "/var/cache/nodepool/cirros-0.4.0-x86_64-disk.img"20:15
weshayto /opt/cache/files20:15
weshayI think20:15
* rlandy looks for that setting20:17
rlandyweshay: hmmm ... unless that has changed recently, it should work20:18
rlandymaybe we can run again and see if the error is repeatable20:18
weshaypretty sure it will be20:18
weshayrlandy look at a successful job.. let's look at the tempest.conf log20:19
weshaydon't rerun20:19
rlandywhich successful job would be closest?20:19
rlandyan old rhos-15?20:19
rlandystein?20:19
weshaylooking at https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/17ed20f8e5474b3892f78c5a28d4179a20:19
rlandyimg_file = cirros-0.4.0-x86_64-disk.img20:20
rlandy[image]20:21
rlandyimage_path = http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img20:21
rlandyhttp_image = http://download.cirros-cloud.net/0.4.0/cirros-0.4.0-x86_64-disk.img20:21
weshayhttps://sf.hosted.upshift.rdu2.redhat.com/logs/periodic/code.engineering.redhat.com/openstack/tripleo-ci-internal-jobs/master/tripleo-ci-rhel-8-standalone-rhos-15/17ed20f/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-06-07_01_05_2120:21
weshayya.. so you just hit a infra error20:21
rlandycharming20:21
weshaybut the job usually uses https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/config/testenv/multinode.yml#L3420:22
weshayso ..20:22
weshaywe should update the image20:22
rlandysorry, I am not following, ... image or image path20:23
weshayrlandy let's update the release file actually20:23
rlandyyou get an update, and you get an update, we all get an update20:24
rlandythe image was in the job definition20:24
rlandyhttp://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git/tree/zuul.d/standalone-jobs.yaml#n9320:24
weshayrlandy where is your latest patch? https://code.engineering.redhat.com/gerrit/#/c/173832/ ?20:25
rlandyyep20:25
rlandyfeel free to edit or comment and I'll fix20:25
weshayoh look at that20:25
weshayhttp://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git/tree/zuul.d/standalone-jobs.yaml#n9320:25
weshayya.. so let's change that to20:25
weshay    cirros_image_cache_file: "/var/cache/nodepool/cirros-0.4.0-x86_64-disk.img"20:26
weshaytempest_test_image_path: /var/cache/nodepool/cirros-0.4.0-x86_64-disk.img20:26
weshayand we should be good20:26
weshayrlandy k?20:26
rlandyso the tempest image is on the rhel-8 cloud image now?20:27
weshayafaict20:27
rlandyI think we set it there initially because it was not20:27
rlandyk - will make the change and we will find out20:27
rlandyweshay: running again we will see what falls out20:30
*** dtrainor has quit IRC20:35
*** dtrainor has joined #oooq20:36
*** Goneri has quit IRC20:42
*** jtomasek has quit IRC20:43
rfolcoweshay, pls review https://code.engineering.redhat.com/gerrit/#/c/173872/.... there is no way to test it without merging.20:46
rfolcoweshay, can wait until tomorrow for panda as I am about to leave today20:47
rfolcowill add him to reviewers20:47
weshayrfolco k.. I suppose if we don't parent at all at that level it's fine20:49
rfolcoweshay, base job is always run20:50
weshayrfolco but the change will need to be in for all jobs that run in rdo-cloud and internally that hit rdo registry20:50
rfolcoit parents off all upstreams ones and comes back to the base20:50
rfolcoits the mother of all jobs20:50
rfolcoalways run the local zuul base job20:50
rfolcoinstead of running upstream base one20:50
rfolcoweshay, hmm we can discuss this tomorrow with panda... I can't see how to parent off the same job (standalone, container build, image build, etc)20:52
rfolcostandalone parents to upstream standalone-base...20:52
weshaythey are all periodic20:53
weshayperiodic base20:53
*** aakarsh has quit IRC20:53
weshayrfolco what you have is fine for a single test20:53
rfolcoweshay, no way to test without merging20:54
rfolcoin config20:54
weshayya.. fine20:54
weshayrfolco review the structure of the rdo sf promotion jobs20:57
weshayrfolco and be prepared to discuss the refactor of rdo sf.. and the creation of internal jobs tomorrow :)20:57
weshayrfolco after that current run.. rfolco I want you to stop working on internal jobs until we have rdo sf refactored20:58
weshayrfolco you hit a infra error on your last standalone run..20:58
weshayI was incorrect, auth is not up yet on pulls20:58
weshaybut it will be20:58
weshayrfolco make sense?20:59
*** rfolco has quit IRC21:01
*** jfrancoa has quit IRC21:10
rlandyweshay: rfolco: https://code.engineering.redhat.com/gerrit/#/c/173872/3/zuul.d/jobs.yaml is killing all our jobs21:25
rlandywhy add a containers playbook to the base job???21:25
rlandythat means every job that runs in our tenant would  playbooks/tripleo-ci-periodic-base/containers-push-registry-login.yaml21:27
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/35/173835/3/check/tripleo-ci-rhel-8-standalone-rhos-15/d7551e6/job-output.txt.gz#_2019-06-18_21_16_37_80753321:28
rlandy2019-06-18 21:16:39.308518 | TASK [Install python docker]21:28
rlandy2019-06-18 21:16:40.885852 | primary | ERROR21:28
rlandy2019-06-18 21:16:40.886231 | primary | {21:28
rlandy2019-06-18 21:16:40.886297 | primary |   "failures": [21:28
rlandy2019-06-18 21:16:40.886339 | primary |     "No package python-docker available."21:28
rlandy2019-06-18 21:16:40.886372 | primary |   ],21:28
rlandy2019-06-18 21:16:40.886403 | primary |   "msg": "Failed to install some of the specified packages",21:28
rlandy2019-06-18 21:16:40.886433 | primary |   "rc": 1,21:28
rlandy2019-06-18 21:16:40.886463 | primary |   "results": []21:28
rlandy2019-06-18 21:16:40.886492 | primary | }21:28
rlandyweshay: https://code.engineering.redhat.com/gerrit/#/c/173878/21:30
*** d0ugal has quit IRC21:30
*** d0ugal has joined #oooq21:30
rlandyweshay:^^ merging this21:41
*** tesseract has quit IRC21:47
weshayrlandy /me looks22:01
weshayrlandy ah fair.. it should only be for rdo jobs22:02
rlandyweshay: too late I merged it :(22:02
weshayrlandy that's fine22:02
weshayit's not needed yet22:02
rlandyweshay: it was killing all my jobs22:02
weshaysorry22:02
rlandywill talk with rfolco tomorrow22:02
rlandynot a bug deal22:02
weshayrfolco was begging for it22:02
rlandybug22:02
rlandybig22:02
weshayrlandy so.. rhel rdo containers will be passwd protected22:03
weshayin the registry22:03
rlandythat's fine22:03
weshayrlandy so we'll need a  rdo-base and rhos-base22:03
rlandywe just need another level of parenting22:03
weshayaye22:03
weshaywe need to map this stuff out22:03
rlandyack22:03
rlandyI can help tomorrow22:03
weshaysorry for the trouble22:04
weshaythank you :)22:04
rlandynot a problem22:04
weshaywoot your f28 fix merged :)22:05
rlandywhich one?22:05
weshayhttps://review.opendev.org/#/c/663417/22:05
rlandyha- I forgot about that22:06
rlandyi just kept dropping stuff on my ruck shift22:06
weshaymeh22:07
rlandyweshay: so ... perf jobs - I am still w/o a successful deploy there - will ping you about it tomorrow22:07
weshayk22:07
rlandyperhaps we can re-think22:07
rlandy'since you fix all my problems'22:07
weshaylolz22:08
weshayrlandy psttt... just challenge me with something difficult please ;/22:08
weshayI could not be more unproductive atm, but I'm turning it around I think22:09
rlandyweshay: oh you wish is my command - you can guess at the correct network situation here22:09
rlandybecause I am fresh out of ideas22:09
rlandyweshay: lol 'I could not be more unproductive atm' - now you are  a real manager22:10
weshayya..  something like that..22:10
*** Goneri has joined #oooq22:36
rlandyugh - what now?22:42
rlandyit just passed deploy on the last run22:42
*** Goneri has quit IRC22:43
*** Goneri has joined #oooq22:47
rlandyhere we go again ...22:55
rlandy2019-06-18 22:40:01 |     "Error running ['podman', 'exec', '--user=root', 'keystone', '/usr/bin/bootstrap_host_exec', 'keystone', 'keystone-manage', 'bootstrap', '--bootstrap-password', 'vOjEJoKGsqeN7RxudGXzgT4hT']. [1]",22:55
*** rlandy has quit IRC23:21
weshayoh weird23:41

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!