| *** d0ugal has quit IRC | 00:07 | |
| *** d0ugal has joined #oooq | 00:24 | |
| *** gouthamr has left #oooq | 00:43 | |
| *** ajo has quit IRC | 00:51 | |
| *** ajo has joined #oooq | 00:51 | |
| *** ykarel has joined #oooq | 03:49 | |
| *** udesale has joined #oooq | 04:01 | |
| *** ykarel has quit IRC | 04:23 | |
| *** ykarel has joined #oooq | 04:41 | |
| *** saneax has joined #oooq | 04:43 | |
| *** skramaja has joined #oooq | 05:05 | |
| *** chandankumar has quit IRC | 05:06 | |
| *** chandankumar has joined #oooq | 05:07 | |
| *** jfrancoa has joined #oooq | 05:08 | |
| *** jfrancoa has quit IRC | 05:13 | |
| *** jfrancoa has joined #oooq | 05:26 | |
| *** ratailor has joined #oooq | 05:40 | |
| *** marios has joined #oooq | 05:42 | |
| *** jbadiapa has joined #oooq | 05:48 | |
| *** brault has joined #oooq | 06:02 | |
| *** irclogbot_1 has quit IRC | 06:18 | |
| *** irclogbot_3 has joined #oooq | 06:18 | |
| *** panda has quit IRC | 06:18 | |
| *** ccamacho has joined #oooq | 06:19 | |
| *** panda has joined #oooq | 06:20 | |
| *** matbu has joined #oooq | 06:34 | |
| *** fmount has joined #oooq | 06:51 | |
| chandankumar | marios: panda Good morning :-) | 06:56 |
|---|---|---|
| *** jtomasek has joined #oooq | 06:58 | |
| marios | o/ | 07:00 |
| *** kopecmartin has joined #oooq | 07:03 | |
| *** apetrich has joined #oooq | 07:05 | |
| *** jaosorior has joined #oooq | 07:11 | |
| *** tesseract has joined #oooq | 07:11 | |
| *** tosky has joined #oooq | 07:14 | |
| *** ccamacho has quit IRC | 07:31 | |
| *** ccamacho has joined #oooq | 07:39 | |
| *** ykarel is now known as ykarel|lunch | 07:48 | |
| *** sshnaidm|off has quit IRC | 07:48 | |
| *** sshnaidm has joined #oooq | 07:49 | |
| *** amoralej|off is now known as amoralej | 07:52 | |
| chandankumar | zbr: Hello | 07:56 |
| chandankumar | zbr: need some help here on tht patch | 07:56 |
| chandankumar | zbr: https://review.opendev.org/#/c/664568/ | 07:59 |
| chandankumar | I have moved clouds_yaml.py script as a part of tht | 07:59 |
| chandankumar | now it becomes a module | 07:59 |
| chandankumar | now I need to reuse as a script in tht | 07:59 |
| chandankumar | directly calling module is not possible as a script | 07:59 |
| chandankumar | as the path will change | 07:59 |
| chandankumar | so will I put a new script to call that module and reuse here? | 08:00 |
| *** jaosorior has quit IRC | 08:13 | |
| *** amoralej is now known as amoralej|off | 08:16 | |
| *** jaosorior has joined #oooq | 08:17 | |
| *** chem has joined #oooq | 08:18 | |
| zbr | chandankumar: done. only some minor comments. | 08:23 |
| zbr | and o/ hello | 08:24 |
| chandankumar | zbr: checking | 08:30 |
| zbr | maybe marios or panda could check my comments too. | 08:31 |
| chandankumar | zbr: thanks, will take a lool | 08:33 |
| arxcruz | hey guys, my daughter fell and demands his father, i'll be a little away but i'm around | 08:33 |
| zbr | marios: re bumping comment https://review.opendev.org/#/c/662577/7/tox.ini@89 -- is ok to use that approach or you want me to avoid it in the future? | 08:41 |
| chandankumar | zbr: sure, taking a look | 08:45 |
| *** holser_ has joined #oooq | 08:51 | |
| *** ykarel|lunch is now known as ykarel | 08:52 | |
| *** udesale has quit IRC | 09:02 | |
| *** udesale has joined #oooq | 09:03 | |
| *** udesale has quit IRC | 09:04 | |
| *** udesale has joined #oooq | 09:08 | |
| marios | zbr: ack will revisit | 09:09 |
| chandankumar | marios: panda zbr weshay one place to view outage https://one.redhat.com/outages/list/incidents | 09:13 |
| chandankumar | feel free to bookmark it | 09:13 |
| marios | nice thx chandankumar | 09:13 |
| *** udesale has quit IRC | 09:13 | |
| *** udesale has joined #oooq | 09:14 | |
| zbr | looks cool, but once i pressed the "create outage" i ended up with loop animation. | 09:15 |
| zbr | i may be picky but "create outage" seems bit weird to me, better would "report/record/add"... unless your nick is chaos-monkey | 09:16 |
| *** udesale has quit IRC | 09:51 | |
| *** bogdando has joined #oooq | 09:51 | |
| *** udesale has joined #oooq | 09:52 | |
| *** udesale has quit IRC | 09:56 | |
| *** udesale has joined #oooq | 09:57 | |
| marios | brb | 09:58 |
| *** marios has quit IRC | 09:58 | |
| *** jpena|off is now known as jpena | 10:00 | |
| *** udesale has quit IRC | 10:00 | |
| *** udesale has joined #oooq | 10:00 | |
| *** amoralej|off is now known as amoralej | 10:02 | |
| chandankumar | :x | 10:11 |
| *** marios has joined #oooq | 10:13 | |
| *** ykarel is now known as ykarel|afk | 10:16 | |
| chandankumar | zbr: panda marios https://twitter.com/rhatdan/status/1143098668221943808 | 10:31 |
| *** rnoriega- has joined #oooq | 10:44 | |
| *** rnoriega has quit IRC | 10:45 | |
| *** marios has quit IRC | 11:11 | |
| *** marios has joined #oooq | 11:13 | |
| *** jpena is now known as jpena|lunch | 11:20 | |
| *** jaosorior_ has joined #oooq | 11:22 | |
| *** jaosorior has quit IRC | 11:23 | |
| *** amoralej is now known as amoralej|lunch | 11:23 | |
| *** ykarel|afk is now known as ykarel | 11:28 | |
| chandankumar | zbr: apart from that there are more changes needed | 11:30 |
| *** udesale has quit IRC | 11:39 | |
| *** skramaja has quit IRC | 11:43 | |
| *** rfolco has joined #oooq | 11:46 | |
| *** rlandy has joined #oooq | 11:55 | |
| *** apetrich has quit IRC | 11:56 | |
| *** ratailor has quit IRC | 11:58 | |
| *** apetrich has joined #oooq | 12:03 | |
| *** ykarel is now known as ykarel|afk | 12:09 | |
| *** ykarel|afk has quit IRC | 12:17 | |
| *** d0ugal has quit IRC | 12:23 | |
| weshay | marios are the deps resolved for RHEL8 / train? | 12:33 |
| *** d0ugal has joined #oooq | 12:37 | |
| marios | o/ weshay | 12:39 |
| marios | weshay: no, well that issue is yeah but it fails on another thing | 12:39 |
| marios | weshay: see update at https://tree.taiga.io/project/tripleo-ci-board/task/1114 issue 6.2 and then at "Reviews" see notes on rhui | 12:40 |
| weshay | marios see it.. k.. thank you.. new bug, new cix for each one you can confirm. I suspect we'll hit 4-5 of these | 12:41 |
| weshay | marios plotnetfg if I'm reading it correctly | 12:42 |
| marios | weshay: yeah, and i reran the disk-image-create manually in the hope of getting more info but same trace/log at http://paste.openstack.org/raw/753305 | 12:42 |
| *** ykarel|afk has joined #oooq | 12:43 | |
| marios | weshay: ack will do switching over to containers for a bit but will file bug etc after scrum | 12:43 |
| *** ykarel|afk is now known as ykarel | 12:43 | |
| weshay | marios k.. I don't see a lp bug yet, go ahead and write it up w/ promotion-blocker | 12:45 |
| marios | weshay: ack yeah 15:44 < marios> weshay: ack will do switching over to containers for a bit but will file bug etc after scrum | 12:46 |
| marios | weshay: will file after scrum | 12:46 |
| weshay | k.. thank you sir | 12:46 |
| sshnaidm | rlandy, hi | 12:50 |
| *** zbr is now known as zbr|ruck | 12:53 | |
| *** amoralej|lunch is now known as amoralej | 12:54 | |
| rlandy | sshnaidm: hello | 12:57 |
| chandankumar | rlandy: kopecmartin marios https://code.engineering.redhat.com/gerrit/#/c/174410/ | 12:59 |
| rfolco | ping scrum marios, sshnaidm, weshay, panda, rlandy, arxcruz, rfolco, chkumar, zbr, kopecmartin | 13:00 |
| weshay | zbr|ruck so we got a lot of promotions over the weekend | 13:01 |
| zbr|ruck | weshay: i know. i wish I could tell it was due to me. | 13:01 |
| *** jpena|lunch is now known as jpena | 13:02 | |
| weshay | zbr|ruck imho.. let's focus on the template design and reviews to ease the load on check | 13:06 |
| zbr|ruck | sure | 13:06 |
| rlandy | weshay: http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-config.git/tree/playbooks/configure-mirrors.yaml | 13:16 |
| rlandy | rfolco: ^^ | 13:16 |
| rlandy | mirror_fqdn: "mirror.regionone.rdo-cloud.rdoproject.org" | 13:17 |
| rlandy | weshay: rfolco: ^^ | 13:17 |
| rlandy | sshnaidm: you can use tripleo-ci-centos-7-standalone-internal-rocky | 13:18 |
| sshnaidm | rlandy, ack | 13:19 |
| rlandy | the rest of the standalone jobs are rhos | 13:19 |
| rlandy | sshnaidm: http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git/tree/zuul.d/standalone-jobs.yaml#n2 should deploy master | 13:21 |
| rlandy | if you need it - but it's not running anywhere | 13:21 |
| sshnaidm | rlandy, I don't mind, just need a job that can finish :) | 13:22 |
| rlandy | sshnaidm: rock - it's been running and passing | 13:22 |
| rlandy | rocky | 13:22 |
| sshnaidm | rlandy, cool | 13:22 |
| rlandy | so you get a clear comparison | 13:22 |
| apetrich | arxcruz, empty ping | 13:24 |
| *** dtrainor has joined #oooq | 13:26 | |
| weshay | marios fyi https://trello.com/c/ktzEkg3n/56-rdo-on-rhel-8 | 13:34 |
| *** dtrainor has quit IRC | 13:47 | |
| marios | weshay: ack | 13:48 |
| *** dtrainor has joined #oooq | 13:48 | |
| *** jaosorior_ has quit IRC | 13:48 | |
| weshay | chandankumar https://drive.google.com/drive/u/1/folders/1IDZXphuvWsX5YSEpXbJYb9DHgQ6Q7Zw- | 13:48 |
| *** jaosorior has joined #oooq | 13:49 | |
| *** jaosorior has quit IRC | 13:49 | |
| marios | rfolco: when you get some time can you point me to the review with rhui job thanks | 13:54 |
| sshnaidm | ansible 2.7 https://review.opendev.org/#/c/650169/ | 13:56 |
| sshnaidm | https://review.opendev.org/#/q/topic:ansible-2.7-ooo+(status:open+OR+status:merged) | 13:56 |
| *** udesale has joined #oooq | 13:58 | |
| *** vinaykns has joined #oooq | 14:00 | |
| *** aakarsh has quit IRC | 14:01 | |
| marios | weshay: https://bugs.launchpad.net/tripleo/+bug/1834043 | 14:02 |
| openstack | Launchpad bug 1834043 in tripleo "[rdotrunk][rhel8] dependency errors for overcloud image build No match for argument: plotnetcfg" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 14:02 |
| weshay | marios needs promotion blocker | 14:07 |
| marios | weshay: ack | 14:09 |
| weshay | thanks | 14:09 |
| weshay | see it | 14:09 |
| *** chem has quit IRC | 14:15 | |
| *** chem has joined #oooq | 14:15 | |
| *** aakarsh has joined #oooq | 14:20 | |
| *** ccamacho has quit IRC | 14:25 | |
| *** jaosorior has joined #oooq | 14:33 | |
| weshay | marios check it out https://trunk.rdoproject.org/rhel8-master/delorean-deps.repo | 14:37 |
| marios | weshay: cool thx | 14:38 |
| marios | weshay: yeah chandankumar said on scrum it was being fixed as we spoke | 14:38 |
| sshnaidm | marios, rlandy please vote https://review.opendev.org/#/c/650169/ | 14:47 |
| *** ykarel is now known as ykarel|afk | 14:49 | |
| weshay | marios sshnaidm rlandy panda rfolco chandankumar FYI.. I've given you guys the wrong info slightly w/ rdo containers and auth.. | 14:49 |
| weshay | we'll only need to turn auth for master/train | 14:50 |
| rlandy | sshnaidm: voted | 14:50 |
| rlandy | zbr|ruck: ^^ fyi | 14:50 |
| rlandy | this will hit on your shift | 14:50 |
| weshay | which is a namespace.. and auth can be turned on/off per namespace | 14:50 |
| weshay | so auth would be on for pulls for train+ | 14:51 |
| rlandy | sshnaidm: are there test jobs on rdo? | 14:51 |
| sshnaidm | rlandy, which test jobs? | 14:52 |
| rlandy | https://logs.rdoproject.org/37/664537/7/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/bf234ea/job-output.txt.gz | 14:53 |
| rlandy | looking at these failures | 14:53 |
| rlandy | sshnaidm: all the ovb job failed https://review.rdoproject.org/zuul/status | 14:54 |
| *** kopecmartin has quit IRC | 14:54 | |
| rlandy | for that review | 14:54 |
| sshnaidm | rlandy, it's rdo cloud issue | 14:57 |
| sshnaidm | rlandy, in previous patchset they all passed | 14:57 |
| rlandy | "[2019/06/24 02:19:32 PM] [ERROR] stdout: ERROR : [/etc/sysconfig/network-scripts/ifup-eth] Error, some other host (86:23:BC:13:83:82) already uses address 172.18.0.67.", | 14:57 |
| rlandy | sshnaidm: kicking a rhos job with this change to check | 14:58 |
| sshnaidm | rlandy, ack | 14:58 |
| sshnaidm | rlandy, 3party fails with "/etc/sysconfig/network-scripts/ifup-eth] Error, some other host (86:23:BC:13:83:82) already uses address 172.18.0.67." | 14:58 |
| sshnaidm | seems like cloud issue.. | 14:58 |
| rfolco | marios, did you have issues with sudo on rhel8 ? https://sf.hosted.upshift.rdu2.redhat.com/logs/39/174039/15/check/tripleo-ci-rhel-8-rhui-test/9b53209/job-output.txt.gz#_2019-06-24_14_32_23_246966 | 14:59 |
| marios | rfolco: not yet that i recall looking | 14:59 |
| marios | rfolco: (also waiting to join a call might be slow for a bit) | 14:59 |
| sshnaidm | rlandy, btw, setting mirror_fqdn in job vars didn't help, it still uses mirror rdo cloud | 14:59 |
| rlandy | sshnaidm: where did you set it? | 15:00 |
| weshay | marios few min | 15:00 |
| weshay | 1-1 | 15:00 |
| marios | weshay: ack am there | 15:00 |
| sshnaidm | rlandy, https://code.engineering.redhat.com/gerrit/#/c/174409/2/zuul.d/standalone-jobs.yaml | 15:00 |
| marios | rfolco: so i just got the same like "binary not yet available for rhel' | 15:00 |
| sshnaidm | rlandy, we need to merge it in tripleo-internal-config in base job | 15:00 |
| marios | rfolco: e | 15:00 |
| marios | rfolco: sorry hold on checking your logs again i think its different sorry | 15:01 |
| sshnaidm | rlandy, sorry, I mean in configure-mirrors playbook | 15:01 |
| marios | rfolco: not seen that | 15:01 |
| marios | weshay: k brb then 2 mins thanks | 15:01 |
| rfolco | marios, yeah, its a bash task with become true | 15:01 |
| rlandy | sshnaidm: yeah - I am not sure if the vars overwrite | 15:01 |
| rlandy | sshnaidm: my suggestion ... | 15:01 |
| weshay | ya.. /me trying to get out of my current.. | 15:01 |
| rfolco | weird | 15:01 |
| rlandy | create a new temp role | 15:01 |
| rlandy | edit the var there | 15:01 |
| rlandy | sshnaidm: I hate config | 15:03 |
| sshnaidm | rlandy, we don't run there something important right now, maybe let's just try and revert | 15:03 |
| rlandy | sshnaidm: the only thing I am running now bm perf and a check on ansible upgrade | 15:04 |
| marios | weshay: ack back/there whenever you can | 15:04 |
| rlandy | sshnaidm: yeah - but you can merge it and revert it | 15:04 |
| rlandy | we have no other way | 15:04 |
| sshnaidm | rlandy, if configure-mirrors have already ran there it won't affect them | 15:05 |
| rlandy | actually - we may have another way | 15:05 |
| *** jaosorior has quit IRC | 15:05 | |
| rlandy | but it's a change on the repo itself | 15:05 |
| sshnaidm | rlandy, I wonder why this playbook in trusted repo at all, does it use secrets? | 15:06 |
| sshnaidm | rlandy, let's just move it to jobs | 15:06 |
| rlandy | sshnaidm: check the job definitions | 15:07 |
| rlandy | I think it's in config because it's part of the job definitions there | 15:07 |
| rlandy | http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-config.git/tree/zuul.d/jobs.yaml#n20 | 15:08 |
| rlandy | ^^ that is why it's there | 15:08 |
| *** panda has quit IRC | 15:09 | |
| *** panda has joined #oooq | 15:11 | |
| sshnaidm | rlandy, mm... I think it will handle it if it'll be in jobs repo too, or we can add "required_project", but not 100% | 15:12 |
| sshnaidm | rlandy, let's try with a patch and revert and then will see.. | 15:12 |
| rlandy | sshnaidm: https://code.engineering.redhat.com/gerrit/#/c/174427/ | 15:13 |
| rlandy | ^^ to test ansible upgrade | 15:13 |
| rlandy | two rhos jobs, one baremetal | 15:13 |
| rlandy | sshnaidm:yeah - go ahead | 15:13 |
| rlandy | we can revert | 15:13 |
| *** jaosorior has joined #oooq | 15:18 | |
| sshnaidm | rlandy, https://code.engineering.redhat.com/gerrit/#/c/174429/ | 15:18 |
| rlandy | sshnaidm: go4it - we will see what happens | 15:21 |
| rlandy | revert if necessary | 15:21 |
| *** kopecmartin has joined #oooq | 15:24 | |
| *** kopecmartin is now known as kopecmartin|off | 15:31 | |
| *** jfrancoa has quit IRC | 15:31 | |
| chandankumar | weshay: preplanning meeting time | 15:32 |
| *** tesseract has quit IRC | 15:33 | |
| *** jaosorior has quit IRC | 15:37 | |
| rfolco | chandankumar, dropping, will reschedule | 15:39 |
| chandankumar | rfolco: ok | 15:39 |
| chandankumar | rfolco: may be tomorrow? | 15:40 |
| weshay | chandankumar rfolco be there now | 15:41 |
| weshay | sorry | 15:41 |
| weshay | little late | 15:41 |
| rfolco | ok rejoining | 15:41 |
| *** gouthamr_ has joined #oooq | 15:45 | |
| marios | weshay: can't schedule events for rhos pto/travel calendar fyi | 15:46 |
| marios | weshay: will add to my calendar and cc you :D | 15:47 |
| *** amoralej is now known as amoralej|off | 15:47 | |
| *** dtrainor_ has joined #oooq | 15:47 | |
| * marios almost out | 15:47 | |
| *** jpena is now known as jpena|off | 15:48 | |
| *** dtrainor has quit IRC | 15:50 | |
| rlandy | sshnaidm: https://sf.hosted.upshift.rdu2.redhat.com/logs/27/174427/1/check/tripleo-ci-rhel-8-standalone-rhos-15/1d099ea/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 15:56 |
| rlandy | ^^ not the fault of ansible upgrade | 15:56 |
| rlandy | we see that every now and again from the registry | 15:56 |
| sshnaidm | rlandy, yeah, I saw it today too | 15:58 |
| sshnaidm | rlandy, sa job with this mirror gonna take 40 min.. look fantastic | 15:59 |
| rlandy | sshnaidm: good news | 16:00 |
| sshnaidm | yeah, hard to believe | 16:01 |
| rlandy | also enable upgrade look ok so far | 16:01 |
| *** marios has quit IRC | 16:01 | |
| rlandy | sshnaidm: after all your effort, it should work | 16:01 |
| rlandy | really, I hacked a lot to get downstream stuff goinf | 16:01 |
| rlandy | some of it was not ideal | 16:01 |
| sshnaidm | rlandy, well, ovb stacks fail because "no valid host", do you know whom we need to turn to? pnt devops? kforde? | 16:02 |
| sshnaidm | I assume kforde is in holiday today | 16:02 |
| sshnaidm | rlandy, better to run bm jobs now to see that this mirror is ok for them too | 16:03 |
| sshnaidm | rlandy, when you feel comfortable, just +w https://review.opendev.org/#/c/650169/ | 16:04 |
| chandankumar | weshay: https://review.rdoproject.org/etherpad/p/collect_logs | 16:07 |
| chandankumar | weshay: https://trello.com/c/Cmbvkgbl/2-70253-neha-alhat | 16:14 |
| rlandy | sshnaidm: sorry - lunch - checking now | 16:38 |
| rlandy | no valid host is a network overload issue | 16:38 |
| rlandy | sometimes | 16:38 |
| *** holser__ has joined #oooq | 16:56 | |
| *** holser_ has quit IRC | 16:58 | |
| *** Goneri has quit IRC | 17:02 | |
| *** udesale has quit IRC | 17:07 | |
| weshay | zbr|ruck reminder to check in w/ #rhos-ops and kieran tomorrow re: ovb stacks, servers, and networks that need to be deleted | 17:10 |
| *** chem has quit IRC | 17:10 | |
| *** bogdando has quit IRC | 17:14 | |
| *** panda has quit IRC | 17:18 | |
| *** panda has joined #oooq | 17:20 | |
| *** holser__ has quit IRC | 17:43 | |
| sshnaidm | rlandy, sa jobs took 1 hour with new mirror, great result, let's run BM to see if mirrors help them | 18:17 |
| rlandy | ssh - rerunning now | 18:19 |
| rlandy | sshnaidm: ^^ | 18:19 |
| sshnaidm | ack | 18:19 |
| *** ykarel|afk has quit IRC | 18:30 | |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/b97314f8257d4680a0d9cb3a64073be9?logfile=console.log | 18:41 |
| rlandy | ^^ looking better | 18:41 |
| *** tosky has quit IRC | 18:41 | |
| rlandy | hmmm .... | 19:13 |
| rlandy | wonder of the baremetal machines can reach internal mirror | 19:13 |
| rlandy | sshnaidm: ^^ think there may be an issue there | 19:13 |
| rlandy | the ansible upgrade is probably alright | 19:13 |
| rlandy | two changes together :( | 19:14 |
| *** jtomasek has quit IRC | 19:22 | |
| rfolco | rlandy, mirror is hardcoded in http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-config.git/tree/playbooks/configure-mirrors.yaml#n20 | 19:23 |
| rlandy | rfolco: yep - I know | 19:24 |
| rlandy | sshnaidm just made that change | 19:24 |
| rlandy | it's config so we had to merge it | 19:24 |
| rfolco | oh | 19:24 |
| rlandy | but now I think bm can't access it | 19:24 |
| rfolco | so the new mirror that nhicher configured is that one | 19:24 |
| rfolco | rlandy, ? | 19:24 |
| rlandy | 2019-06-24 18:59:49 | 2019-06-24 18:59:49.090 | curl: (6) Could not resolve host: cloud.centos.org; Unknown error | 19:25 |
| rfolco | :( | 19:25 |
| rlandy | this is a problem | 19:26 |
| rlandy | could reach rdocloud mirror | 19:27 |
| rlandy | public | 19:27 |
| * rlandy needs to think how to fix this | 19:27 | |
| *** saneax has quit IRC | 19:44 | |
| weshay | rfolco rlandy when you have a zuul job like this one.. there is no way to get a tripleo-common change integrated into the ci run afaict.. am I wrong? https://code.engineering.redhat.com/gerrit/#/c/172857/21/zuul.d/build-image.yaml | 19:56 |
| weshay | needs tripleo-common as a required project and the job I guess would need to build the tripleo-common rpm on the fly.. | 19:57 |
| rfolco | weshay, looking... | 19:57 |
| rfolco | weshay, you mean, tripleo-common rpm build as input for the build-image job ? | 19:59 |
| weshay | rfolco a depends-on: to tripleo-common would not be integrated | 20:00 |
| * rlandy looks | 20:00 | |
| rfolco | weshay, I think it would | 20:01 |
| rlandy | weshay: you means depends-on? | 20:01 |
| rfolco | weshay, as long as tripleo-common is a required-projects | 20:01 |
| weshay | rlandy ya.. so if I put a depends-on to tripleo-common.. what builds that? | 20:01 |
| rlandy | weshay: we would need to make sure it's in the build list | 20:02 |
| weshay | rlandy it doesn't use tq | 20:02 |
| rfolco | tripleo-common depends-on change implements the args in extra_build_opts ? | 20:03 |
| rfolco | weshay, ^ | 20:03 |
| rlandy | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/build-test-packages/defaults/main.yml | 20:03 |
| rlandy | ^^ so adding here would not help? | 20:03 |
| weshay | rlandy no.. it doesn't use tq/tqe | 20:04 |
| rfolco | zuul stuff is not in /usr/share | 20:04 |
| * rlandy is still trying to figure out if the internal registry should be accessible by baremetal machines | 20:05 | |
| weshay | https://github.com/openstack/tripleo-ci/blob/93768b46eec9cf3767fef23c186806e660f69395/playbooks/tripleo-buildimage/run.yaml | 20:07 |
| rlandy | File "/usr/lib/python2.7/site-packages/tripleo_common/image/image_uploader.py", line 351, in authenticate | 20:07 |
| rlandy | I think this is wrong | 20:07 |
| weshay | https://github.com/openstack/tripleo-ci/blob/93768b46eec9cf3767fef23c186806e660f69395/playbooks/tripleo-buildimage/run.yaml#L95 | 20:07 |
| rfolco | weshay, create your tripleo-common change, make sure its on required-projects, and add depends-on to your commit. Reference to tripleo-common using zuul src dir... | 20:08 |
| weshay | rfolco what builds the src? | 20:09 |
| rfolco | weshay, run is not zuulv3 compliant isn't ? | 20:09 |
| weshay | https://github.com/openstack/tripleo-ci/blob/93768b46eec9cf3767fef23c186806e660f69395/playbooks/tripleo-buildimage/run-v3.yaml | 20:13 |
| rfolco | bindep? | 20:13 |
| weshay | rfolco bindep installs rpms.. doesn't build them man | 20:13 |
| rfolco | zuul does not build anything afaik | 20:13 |
| rfolco | bindep_dir: "{{ ansible_user_dir }}/{{ zuul.projects['opendev.org/openstack/python-tripleoclient'].src_dir }}" | 20:14 |
| rfolco | this is the src dir for python-tripleo client, it is the git repo cloned by zuul | 20:14 |
| rfolco | it installs from source, not rpms afaik | 20:15 |
| rfolco | weshay, am I wrong? | 20:16 |
| weshay | rfolco where does it install from src? | 20:16 |
| rfolco | https://github.com/openstack/tripleo-ci/blob/93768b46eec9cf3767fef23c186806e660f69395/playbooks/tripleo-buildimage/run-v3.yaml#L5 | 20:17 |
| rfolco | weshay, this is installing python-tripleoclient from source git repo cloned by zuul | 20:17 |
| weshay | rfolco where does that execute for the internal image build? | 20:18 |
| * weshay looking at https://code.engineering.redhat.com/gerrit/#/c/172857/21 | 20:18 | |
| rfolco | weshay, did not run run-v3.yaml, failed before that point | 20:19 |
| weshay | rfolco what are you saying? | 20:20 |
| weshay | where is run-v3 even called | 20:20 |
| rfolco | <weshay> rfolco where does that execute for the internal image build? | 20:20 |
| weshay | are we not just running.. `openstack overcloud image build` | 20:20 |
| rlandy | sshnaidm: rfolco: weshay: pls review https://code.engineering.redhat.com/gerrit/#/c/174457/ | 20:21 |
| rlandy | I think the bm hardware can't access internal mirror - auth | 20:21 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/27/174427/1/check/periodic-tripleo-ci-centos-7-bm_envD-3ctlr_1comp-featureset001-master/b97314f/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 20:22 |
| rfolco | weshay, right I was just saying there is no run-v3 in that run... | 20:22 |
| weshay | rfolco right.. so where would a tripleo-common patch be built in the internal job? | 20:22 |
| weshay | back to the original question | 20:22 |
| weshay | rlandy so .. the internal asf mirror does not work w/ bm? | 20:23 |
| weshay | what happens? it should take two runs to populate the mirror.. btw | 20:23 |
| weshay | if ur first run failed | 20:23 |
| rfolco | weshay, I think we need to include that line to the internal job playbook/workflow... | 20:23 |
| rfolco | roles: | 20:23 |
| rfolco | - role: bindep | 20:23 |
| rfolco | bindep_dir: "{{ ansible_user_dir }}/{{ zuul.projects['opendev.org/openstack/ | 20:23 |
| sshnaidm | rlandy, maybe we can use "{{ mirror_fqdn }}" parameter in this playbook and set it in job variables? Then base baremetal job will have its own mirror defined | 20:23 |
| weshay | rfolco what does bindep do? | 20:24 |
| rfolco | weshay, nevermind. You want to build the project not its deps. | 20:24 |
| rlandy | sshnaidm: I can't confirm that is the problem but there is an auth issue so I suspect | 20:24 |
| rfolco | I got the point. | 20:24 |
| weshay | aye | 20:25 |
| weshay | rfolco so the final form of the image build job needs to change | 20:25 |
| rlandy | sshnaidm: I am fine with the var idea | 20:25 |
| rlandy | default it | 20:25 |
| rlandy | if you think it's better than https://code.engineering.redhat.com/gerrit/#/c/174457/1/playbooks/configure-mirrors.yaml | 20:25 |
| sshnaidm | rlandy, Name or service not known | 20:26 |
| rlandy | we have not had any bm job pass since | 20:26 |
| weshay | rlandy why would an internal asf mirror not work w/ bm? | 20:26 |
| weshay | rlandy help me understand this point | 20:27 |
| sshnaidm | rlandy, seems like dns issue? | 20:27 |
| rlandy | I'm not sure tbh | 20:28 |
| rlandy | the mirror is pingable | 20:29 |
| sshnaidm | rlandy, weshay mirror-rdo.usersys.redhat.com is resolvable only with internal dns | 20:29 |
| rlandy | sshnaidm: the undercloud and overcloud should both be deployed with internal dns | 20:30 |
| rlandy | 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Could not resolve host: cloud.centos.org; Unknown error | 20:30 |
| rlandy | 2019-06-24 18:58:17 | 2019-06-24 18:58:17.000 | * Closing connection 0 | 20:30 |
| sshnaidm | rlandy, ack | 20:30 |
| rlandy | let's try this deploy one more time | 20:31 |
| *** dtrainor_ is now known as dtrainor | 20:32 | |
| rlandy | let's take ansible upgrade out of the picture | 20:32 |
| rlandy | [heat-admin@overcloud-controller-2 ~]$ ping mirror-rdo.usersys.redhat.com | 20:33 |
| rlandy | PING mirror-rdo.usersys.redhat.com (10.0.78.237) 56(84) bytes of data. | 20:33 |
| rlandy | 64 bytes from vm-10-0-78-237.hosted.upshift.rdu2.redhat.com (10.0.78.237): icmp_seq=1 ttl=58 time=0.379 ms | 20:33 |
| rlandy | sshnaidm: ^^ it's pingable | 20:33 |
| rlandy | from the overcloud node | 20:33 |
| rlandy | so not dns | 20:33 |
| rlandy | oops wrong one | 20:33 |
| rlandy | no - that is the right one | 20:34 |
| sshnaidm | rlandy, let's rerun, maybe just bad luck | 20:34 |
| weshay | rlandy sshnaidm what is not internal in the bm jobs? | 20:34 |
| sshnaidm | weshay, no idea | 20:34 |
| weshay | agree w/ sshnaidm.. the first couple of runs on a new mirror will fail | 20:34 |
| weshay | sshnaidm rlandy hey qq... | 20:34 |
| rlandy | sshnaidm: ack - let's try this again - w/o ansible upgrade change to be sure | 20:35 |
| weshay | if I add files: to job def.. that means the jobs will always trigger on those files? | 20:35 |
| rlandy | to the projects.yaml | 20:35 |
| weshay | we're no longer triggering build-image tests on image-yaml changes https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/build-image.yaml | 20:35 |
| * weshay looks at other jobs | 20:36 | |
| weshay | ya.. think I see it | 20:39 |
| weshay | hrm.. I'm doing something stupid.. https://review.opendev.org/#/c/667169/ | 20:44 |
| weshay | anyone have a sec to tell their boss whats up | 20:44 |
| weshay | no one biting at the line | 20:45 |
| weshay | sshnaidm who owns the patches for ansible 2.7 btw | 20:47 |
| sshnaidm | weshay, owns? we have your patch to merge | 20:48 |
| weshay | oh .. maybe me | 20:48 |
| weshay | lolz | 20:48 |
| weshay | k.. was just poking around | 20:48 |
| weshay | sshnaidm looking at the test results for DNM patches | 20:49 |
| sshnaidm | weshay, just changed them to 2.8, but results are still there | 20:49 |
| weshay | oh.. k.. I see those :) | 20:49 |
| weshay | sshnaidm say.. GO BIG OR GO HOME | 20:50 |
| sshnaidm | weshay, I invented this slogan | 20:52 |
| weshay | :) | 20:53 |
| rlandy | 2019-06-24 20:32:39 | time="2019-06-24T20:32:39Z" level=fatal msg="Get https://auth.docker.io/token?scope=repository%3Aceph%2Fdaemon%3Apull&service=registry.docker.io: dial tcp: lookup auth.docker.io on 10.5.30.160:53: server misbehaving" | 20:56 |
| rlandy | 2019-06-24 20:32:39 | | 20:56 |
| rlandy | server misbehaving" | 20:57 |
| rlandy | ^^ that is new | 20:57 |
| rlandy | and judgmental | 20:57 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/31/169131/14/check/tripleo-ci-base-baremetal-microcloud-ha-2/cddcb98/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz | 20:57 |
| *** rfolco has quit IRC | 20:57 | |
| weshay | sshnaidm are we looking to merge https://review.opendev.org/#/c/650169/ prior to ansible 2.8? | 20:58 |
| sshnaidm | weshay, yep | 20:59 |
| sshnaidm | rlandy, let's revert and try? | 20:59 |
| rlandy | sshnaidm: sorry - revert which piece? | 21:00 |
| sshnaidm | rlandy, mirror | 21:00 |
| rlandy | sshnaidm: why revert when it works well with standalone ... how about https://code.engineering.redhat.com/gerrit/#/c/174457/1/playbooks/configure-mirrors.yaml? | 21:00 |
| rlandy | besides the second run is still going | 21:01 |
| rlandy | the misbehaving error is from per tests | 21:01 |
| rlandy | perf | 21:01 |
| sshnaidm | rlandy, ack | 21:01 |
| sshnaidm | rlandy, I'd prefer to use job vars for that, instead of adding logic to playbook | 21:02 |
| weshay | sshnaidm let's try to get some more cores.. big change | 21:03 |
| rlandy | sshnaidm: whatever - but revert seems pointless either we can handle a different mirror per job | 21:03 |
| weshay | rlandy https://review.opendev.org/#/c/650169 | 21:03 |
| weshay | rlandy this is a good way to verify afaict.. https://review.opendev.org/#/c/664546/ | 21:03 |
| rlandy | weshay: in process of testing it downstream | 21:03 |
| weshay | sshnaidm passed w/ 2.7.11 afaict | 21:03 |
| weshay | rlandy rockin | 21:03 |
| rlandy | weshay: the issue is the we merged the mirror change | 21:04 |
| rlandy | which complicated matters ... | 21:04 |
| weshay | rlandy it's always something | 21:04 |
| rlandy | you can see the results in https://code.engineering.redhat.com/gerrit/#/c/174457/ | 21:04 |
| rlandy | https://code.engineering.redhat.com/gerrit/#/c/174427/1..2//COMMIT_MSG | 21:04 |
| rlandy | ^^ that one | 21:04 |
| rlandy | looks ok for rhos standlaone | 21:05 |
| rlandy | bm is caught in mirrors | 21:05 |
| rlandy | which is what I am waiting for | 21:05 |
| sshnaidm | weshay, how many cores?? | 21:05 |
| rlandy | sshnaidm: I will vote by end of day | 21:05 |
| sshnaidm | weshay, and what can they review? One line of ansible version? | 21:05 |
| sshnaidm | rlandy, let's be sure BM jobs run with it and we're fine I think | 21:06 |
| rlandy | sshnaidm: the bm job got pretty far | 21:06 |
| rlandy | into deploy | 21:06 |
| sshnaidm | cool | 21:06 |
| rlandy | that is why I am running the second time w/o the ansible change | 21:07 |
| rlandy | so we are only looking at the mirror | 21:07 |
| weshay | sshnaidm rlandy, panda is pto.. so no... | 21:07 |
| rlandy | sorry but otherwise I land up with never ending debugging | 21:07 |
| weshay | sshnaidm rlandy is probably your last | 21:07 |
| rlandy | weshay: I'll vote today | 21:07 |
| rlandy | just watching this last bm run | 21:08 |
| weshay | rlandy k.. ur bm job used ansible-2.7.11 | 21:10 |
| weshay | sshnaidm last time I really checked in on this.. the tripleo-upgrade role needed patching.. assuming that was done | 21:11 |
| sshnaidm | weshay, I ran jobs on tripleo upgrade | 21:12 |
| sshnaidm | all pass | 21:12 |
| weshay | k.. great | 21:12 |
| rlandy | the last one did | 21:12 |
| rlandy | used regular | 21:13 |
| weshay | rlandy ok.. I would assume bm jobs are pretty safe.. given what I see but it's reasonable to get the mirror out of the way and retry looking for a pass that hits through tempest | 21:13 |
| weshay | just to be very sure | 21:14 |
| rlandy | weshay: yeah - I think it's probably ok | 21:15 |
| weshay | rlandy you have a sec to help me out? | 21:16 |
| *** gouthamr_ is now known as gouthamr | 21:16 | |
| rlandy | weshay: sure | 21:17 |
| weshay | rlandy trying to get the upstream image build check jobs to trigger on https://review.opendev.org/#/c/667169/ | 21:17 |
| weshay | I thought this would do it.. https://review.opendev.org/#/c/667223/1/zuul.d/layout.yaml | 21:17 |
| weshay | I'm missing something | 21:17 |
| rlandy | sec - looking at review | 21:17 |
| rlandy | weshay: ah - two diff repos | 21:19 |
| weshay | ya | 21:19 |
| rlandy | let's bj for a min | 21:19 |
| rlandy | there is a layout.yaml in tripleo-common | 21:19 |
| rlandy | that is the one to edit | 21:19 |
| rlandy | looking at the file trigger there | 21:19 |
| weshay | rlandy I thought the file: param worked across repos | 21:20 |
| sshnaidm | weshay, yeah, tripleo-ci zuul.yaml is responsible for tripleo-ci repo jobs | 21:20 |
| sshnaidm | weshay, it's under default "- project:" and this is tripleo-ci project | 21:20 |
| rlandy | tripleo-build-containers-jobs | 21:21 |
| weshay | rlandy image-build | 21:21 |
| rlandy | template that is not overridden | 21:21 |
| weshay | oh | 21:21 |
| * weshay looks | 21:21 | |
| rlandy | weshay: https://github.com/openstack/tripleo-common/blob/master/zuul.d/layout.yaml | 21:22 |
| rlandy | ^^ not here at all | 21:22 |
| weshay | k | 21:22 |
| rlandy | unless it's in some template inclusion | 21:22 |
| * weshay puts it in | 21:22 | |
| rlandy | my head hurts | 21:22 |
| weshay | como ze? https://review.opendev.org/667230 | 21:24 |
| weshay | quemo ze (maybe) | 21:25 |
| rlandy | weshay: yeah | 21:25 |
| rlandy | is that rebased on the chnage you want to test? | 21:26 |
| rlandy | otherwise it won;t trigger anything | 21:26 |
| weshay | oh geez | 21:26 |
| weshay | I just did a depends on | 21:26 |
| weshay | IDIOT! | 21:26 |
| rlandy | same repo | 21:26 |
| weshay | rlandy thanks.. now it is | 21:27 |
| weshay | rusty | 21:27 |
| * rlandy looks | 21:27 | |
| weshay | now it's working :) | 21:28 |
| weshay | tripleo-buildimage-overcloud-full-centos-7queued | 21:28 |
| weshay | tripleo-buildimage-ironic-python-agent-centos-7queued (non-voting) | 21:28 |
| weshay | tripleo-buildimage-overcloud-hardened-full-centos-7 | 21:28 |
| rlandy | tripleo-buildimage-overcloud-hardened-full-centos-7queued (non-voting) | 21:28 |
| rlandy | yep | 21:28 |
| rlandy | you got it | 21:28 |
| weshay | thanks! | 21:28 |
| rlandy | sure | 21:28 |
| weshay | rlandy so zuul is not smart enough to use the file pattern for multiple repos | 21:28 |
| weshay | that is confusing to me | 21:28 |
| weshay | it must do it somehow | 21:29 |
| rlandy | weshay: it is if you include the template | 21:29 |
| rlandy | in this case, it was just not included altogether | 21:29 |
| weshay | oh.. it's the template that makes the magic | 21:30 |
| weshay | k | 21:30 |
| rlandy | if that was the case, we would run EVERY tripleo-ci defined test on every repo | 21:30 |
| rlandy | which is not what we want | 21:30 |
| rlandy | templates help there | 21:30 |
| weshay | ok.. ya.. thanks.. | 21:30 |
| weshay | no blue required | 21:30 |
| weshay | efficient | 21:30 |
| rlandy | ok | 21:31 |
| *** vinaykns has quit IRC | 21:42 | |
| rlandy | now we have build images failure | 21:55 |
| rlandy | sshnaidm: I'll +2 the ansible change because w/o it we are not getting any further | 21:55 |
| rlandy | the mirrors need to change | 21:56 |
| sshnaidm | rlandy, ack, thanks | 21:56 |
| *** panda has quit IRC | 21:58 | |
| rlandy | sshnaidm: https://review.opendev.org/#/c/650169 - done - you can W+ | 21:59 |
| rlandy | I'll work on a new patch for mirrors | 21:59 |
| rlandy | bm is getting nowhere | 21:59 |
| sshnaidm | \o/ | 21:59 |
| *** panda has joined #oooq | 22:03 | |
| rlandy | sshnaidm: better? https://code.engineering.redhat.com/gerrit/#/c/174457/ | 22:08 |
| sshnaidm | rlandy, cool, let's see if it works | 22:09 |
| rlandy | pls vote so the change has some legitimacy | 22:10 |
| rlandy | k - let's try this | 22:11 |
| rlandy | here we go again | 22:18 |
| *** aakarsh has quit IRC | 22:35 | |
| *** aakarsh has joined #oooq | 23:24 | |
| *** saneax has joined #oooq | 23:41 | |
| *** rlandy has quit IRC | 23:41 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!