| *** sshnaidm is now known as sshnaidm|afk | 00:08 | |
| *** dmellado_ has joined #oooq | 00:16 | |
| *** dtantsur|afk has quit IRC | 00:21 | |
| *** dmellado has quit IRC | 00:21 | |
| *** dmellado_ is now known as dmellado | 00:22 | |
| *** zbr has quit IRC | 00:27 | |
| *** openstackstatus has quit IRC | 00:28 | |
| *** zbr has joined #oooq | 00:30 | |
| *** ysandeep|away is now known as ysandeep | 00:56 | |
| *** skramaja has quit IRC | 01:11 | |
| *** dsneddon has quit IRC | 01:31 | |
| *** jmasud has quit IRC | 01:59 | |
| *** jmasud_ has joined #oooq | 01:59 | |
| *** dsneddon has joined #oooq | 02:07 | |
| *** dsneddon has quit IRC | 02:21 | |
| *** ykarel|away is now known as ykarel | 04:02 | |
| *** udesale has joined #oooq | 04:15 | |
| *** skramaja has joined #oooq | 04:20 | |
| *** soniya29 has joined #oooq | 04:34 | |
| *** raukadah is now known as chkumar|rover | 05:18 | |
| *** rlandy|bbl has quit IRC | 05:25 | |
| *** udesale_ has joined #oooq | 05:39 | |
| *** udesale has quit IRC | 05:42 | |
| *** yolanda has quit IRC | 06:01 | |
| *** soniya29 has quit IRC | 06:05 | |
| *** ratailor has joined #oooq | 06:05 | |
| *** jfrancoa has joined #oooq | 06:15 | |
| *** holser has joined #oooq | 06:16 | |
| *** saneax has joined #oooq | 06:20 | |
| *** ccamacho has quit IRC | 06:26 | |
| *** holser__ has joined #oooq | 06:35 | |
| *** holser has quit IRC | 06:38 | |
| *** holser__ has quit IRC | 06:51 | |
| *** marios has joined #oooq | 06:54 | |
| *** soniya29 has joined #oooq | 07:36 | |
| *** rascasoft has joined #oooq | 07:37 | |
| *** ykarel is now known as ykarel|mtg | 07:38 | |
| *** jpena|off is now known as jpena | 07:54 | |
| *** jtomasek has joined #oooq | 08:05 | |
| *** jmasud_ has quit IRC | 08:11 | |
| *** holser has joined #oooq | 08:12 | |
| *** tesseract has joined #oooq | 08:13 | |
| *** jmasud has joined #oooq | 08:14 | |
| *** holser has quit IRC | 08:17 | |
| *** holser has joined #oooq | 08:19 | |
| *** yolanda has joined #oooq | 08:19 | |
| *** saneax has quit IRC | 08:22 | |
| *** saneax has joined #oooq | 08:23 | |
| *** amoralej|off is now known as amoralej | 08:23 | |
| *** chem has joined #oooq | 08:26 | |
| *** ykarel|mtg is now known as ykarel|lunch | 08:29 | |
| *** soniya29 has quit IRC | 08:31 | |
| *** ccamacho has joined #oooq | 08:33 | |
| *** holser has quit IRC | 08:37 | |
| chkumar|rover | marios, arxcruz please have a look https://review.opendev.org/#/c/706196/1 | 08:39 |
|---|---|---|
| chkumar|rover | we need to unblock the gates | 08:39 |
| chkumar|rover | permanent solution will take time | 08:39 |
| chkumar|rover | on osa side, it is runned from source, there will be no issue | 08:40 |
| *** tosky has joined #oooq | 08:40 | |
| *** holser has joined #oooq | 08:43 | |
| arxcruz | chkumar|rover: I though we were using from osa | 08:43 |
| marios | chkumar|rover: ack | 08:48 |
| chkumar|rover | arxcruz, in osa it is enabled by default | 08:48 |
| chkumar|rover | but for currently we need to disable it | 08:49 |
| chkumar|rover | till we move to centos8 or i fix the packaging part | 08:49 |
| arxcruz | ok | 08:49 |
| arxcruz | fine by me | 08:49 |
| chkumar|rover | arxcruz, you have tripleo channel access? | 08:51 |
| *** sshnaidm|afk is now known as sshnaidm | 08:54 | |
| arxcruz | chkumar|rover: no | 08:54 |
| *** bogdando has joined #oooq | 08:54 | |
| *** ratailor has quit IRC | 09:01 | |
| arxcruz | zbr: chkumar|rover marios sshnaidm https://review.opendev.org/#/c/704762/ | 09:02 |
| marios | arxcruz: ack will do but not immediately added to next reviews run | 09:02 |
| arxcruz | marios: do it NOW! | 09:03 |
| marios | arxcruz: i'm pretty sure that's an HR violation | 09:03 |
| marios | security! | 09:03 |
| arxcruz | :D | 09:04 |
| arxcruz | omg, i got a +2 from zbr without complains | 09:10 |
| arxcruz | achievement complete | 09:10 |
| zbr | :) | 09:12 |
| zbr | chkumar|rover, marios, sshnaidm: can you help me understand why we rsync to instead of directly to local collection dir? on https://review.opendev.org/#/c/705552/7/tasks/collect.yml | 09:13 |
| zbr | I see arie is trying to overcome a bug in the task, but that specific task makes no sense to me. | 09:13 |
| zbr | it uses hardcoded /tmp to collect stuff, instead of collecting directly to final destination. | 09:14 |
| zbr | in fact that is on managed, host.. so his change makes no sense. | 09:16 |
| sshnaidm | zbr, usually we copied all to /tmp/{{ hostname }} | 09:17 |
| sshnaidm | zbr, a long time ago we could gzip this directories and publish them as is: undercloud.tar.gz, overcloud-compute.tar.gz | 09:18 |
| sshnaidm | zbr, so you could download all undercloud logs at once, for example | 09:18 |
| zbr | so arie extra logic makes no sense. | 09:18 |
| zbr | i know, common downstream | 09:18 |
| *** dtantsur has joined #oooq | 09:18 | |
| *** holser__ has joined #oooq | 09:18 | |
| sshnaidm | zbr, but not it's not the case | 09:18 |
| sshnaidm | we've got problems from infra about logs sizes and duplications, and couldn't browse logs, so.. | 09:19 |
| sshnaidm | zbr, in this file he tries to switch "become" because molecule can't run sudo | 09:20 |
| zbr | molecule may be unable to sudo on localhost, but on managed hosts it does work. | 09:20 |
| zbr | if it does not work, there is not much collection to perform at all. | 09:20 |
| sshnaidm | zbr, I think openstack molecule jobs don't allow sudo | 09:21 |
| zbr | even the first task has a become. | 09:21 |
| sshnaidm | zbr, there is special job we did in tripleo_ansible with sudo | 09:21 |
| zbr | is zuul, also i have the same local, good protection | 09:21 |
| zbr | that is why we use docker container with this job, so we collect from container, where we have sudo. but there is no need to have sudo locally. | 09:22 |
| *** holser has quit IRC | 09:22 | |
| sshnaidm | zbr, anyway, seem like it didn't work for him - if you have better solution for sudo and molecule, you can suggest | 09:22 |
| zbr | if it did not work, it was for a reald issue which we should fix. | 09:23 |
| sshnaidm | zbr, if people make patch, most likely they had a problem | 09:23 |
| zbr | i wonder if we can/should quit fast if we cannot sudo at the start of collec.yml | 09:23 |
| sshnaidm | zbr, and no sudo in molecule job is a known problem | 09:23 |
| zbr | i see it as a feature, not a bug. | 09:24 |
| sshnaidm | zbr, for review you need to understand what problem Arie trying to solve, you can't just -1 because you don't think it's needed | 09:25 |
| zbr | build worker does not have sudo in most ci systems. | 09:25 |
| zbr | i did include clear explanation why. | 09:25 |
| sshnaidm | zbr, you didn't, actually | 09:25 |
| sshnaidm | zbr, you didn't explain anything and didn't provide better solution for the problem | 09:26 |
| sshnaidm | zbr, if you're not sure from the patch what is the problem, then talk to submitter to make it clear | 09:27 |
| sshnaidm | zbr, but you wrote actually "I do not understand the logic here" "Adding extra logic here is not needed." | 09:28 |
| sshnaidm | zbr, if you don't understand, how do you know it's not needed?? | 09:28 |
| zbr | please be careful here, i am considering stop engaging with you on irc. that use of "you" starts to look at something else. | 09:30 |
| *** ykarel|lunch is now known as ykarel | 09:34 | |
| *** bogdando has quit IRC | 09:37 | |
| *** panda|bbl is now known as panda | 09:41 | |
| *** bogdando has joined #oooq | 09:41 | |
| *** holser__ has quit IRC | 09:50 | |
| *** holser has joined #oooq | 09:51 | |
| amoralej | chkumar|rover, what's the status of containers builds for centos8? | 10:20 |
| amoralej | ceph is still the blocker? | 10:20 |
| amoralej | i see kolla has unblocked using copr repo created by ktdreyer + ceph-lab-extras | 10:21 |
| amoralej | https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo#L17-L32 | 10:21 |
| amoralej | it could be a good option until packages are published by Storage SIG | 10:22 |
| chkumar|rover | amoralej, let me check the status | 10:23 |
| chkumar|rover | amoralej, https://review.rdoproject.org/r/#/c/24795/ | 10:24 |
| chkumar|rover | amoralej, currently blocked at rhel-base itself, dpawlik provided a patch, https://review.rdoproject.org/r/#/c/24795/ once current run finishes we have proper data | 10:26 |
| chkumar|rover | arxcruz, sshnaidm marios https://review.opendev.org/#/c/706215/ | 10:27 |
| sshnaidm | chkumar|rover, do standalone jobs use skip_list? | 10:28 |
| amoralej | error building at STEP "COPY repos/CentOS-OpsTools.repo /etc/yum.repos.d/": no files found matching "/tmp/base-build/repos/CentOS-OpsTools.repo" | 10:28 |
| chkumar|rover | sshnaidm, yes | 10:28 |
| chkumar|rover | sshnaidm, it is based on validate tempest for stein only | 10:29 |
| *** ratailor has joined #oooq | 10:29 | |
| sshnaidm | ack | 10:29 |
| chkumar|rover | sshnaidm, for standalone skip list https://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/tempest.yml#L73 | 10:29 |
| sshnaidm | chkumar|rover, when: tempest_cloud_name != 'undercloud' ? | 10:30 |
| chkumar|rover | sshnaidm, yes | 10:30 |
| sshnaidm | chkumar|rover, and what's tempest_cloud_name in standalone case? | 10:30 |
| chkumar|rover | sshnaidm, standalone | 10:30 |
| sshnaidm | ah, ok | 10:30 |
| chkumar|rover | for overcloud it is overcloud | 10:30 |
| dpawlik | chkumar|rover, but tests fails, so one moment, fixing... | 10:34 |
| chkumar|rover | dpawlik, feel free to update the patch, thanks :-) | 10:34 |
| *** dsneddon has joined #oooq | 10:59 | |
| *** udesale_ has quit IRC | 11:02 | |
| chkumar|rover | weshay|ruck, we need to start abandoning the patches, gate queue is 13 hours | 11:33 |
| weshay|ruck | chkumar|rover, you doing ok? | 11:54 |
| weshay|ruck | chkumar|rover, wait wait | 11:54 |
| weshay|ruck | let me look | 11:54 |
| chkumar|rover | weshay|ruck, the world is on fire | 11:54 |
| chkumar|rover | all jobs using os_tempest are broken | 11:55 |
| weshay|ruck | yup | 11:55 |
| weshay|ruck | k | 11:55 |
| chkumar|rover | weshay|ruck, stable/stein gates are also broken | 11:55 |
| weshay|ruck | chkumar|rover, https://meet.google.com/zxi-mvyj-rwe?authuser=1 | 11:56 |
| chkumar|rover | weshay|ruck, https://review.opendev.org/#/c/706196/ | 11:58 |
| *** jpena is now known as jpena|lunch | 12:00 | |
| *** amoralej is now known as amoralej|lunch | 12:00 | |
| *** rascasoft has quit IRC | 12:04 | |
| *** rfolco has joined #oooq | 12:05 | |
| *** jmasud has quit IRC | 12:07 | |
| weshay|ruck | bhagyashris, will be chatting today | 12:07 |
| *** jmasud has joined #oooq | 12:07 | |
| weshay|ruck | akahat, will be chatting | 12:08 |
| bhagyashris | weshay|ruck, sure | 12:08 |
| *** rascasoft has joined #oooq | 12:20 | |
| *** rascasoft has quit IRC | 12:20 | |
| chkumar|rover | brb | 12:25 |
| weshay|ruck | bhagyashris, https://docs.openstack.org/openstack-ansible/latest/user/ceph/full-deploy.html | 12:29 |
| weshay|ruck | https://docs.ceph.com/docs/master/rbd/rbd-openstack/ | 12:29 |
| *** dsneddon has quit IRC | 12:29 | |
| weshay|ruck | bhagyashris, https://github.com/ceph/ceph-ansible | 12:29 |
| *** ysandeep is now known as ysandeep|afk | 12:36 | |
| weshay|ruck | akahat, ping | 12:37 |
| *** ysandeep|afk is now known as ysandeep | 12:37 | |
| weshay|ruck | chkumar|rover, is Soniya around? | 12:38 |
| weshay|ruck | ysandeep, hey.. need to speak w/ you today | 12:38 |
| ysandeep | weshay|ruck, sure.. I am free to talk now | 12:38 |
| weshay|ruck | ysandeep, join https://meet.google.com/fju-oamg-mec?authuser=1 | 12:38 |
| chkumar|rover | weshay|ruck, yes | 12:39 |
| ysandeep | weshay|ruck, please give me just 2 mins | 12:39 |
| weshay|ruck | chkumar|rover, thanks.. I'll schedule a 1-1 w/ her tomorrow | 12:39 |
| weshay|ruck | actually | 12:39 |
| ysandeep | to go to a meeting room | 12:39 |
| chkumar|rover | weshay|ruck, ok | 12:39 |
| weshay|ruck | akahat, you there? | 12:47 |
| chkumar|rover | weshay|ruck, he went outside | 12:48 |
| weshay|ruck | ysandeep, https://trunk.rdoproject.org/centos7-queens/current-tripleo/ | 12:51 |
| chkumar|rover | weshay|ruck, stackviz fix worked | 12:57 |
| chkumar|rover | http://zuul.opendev.org/t/openstack/stream/ba64a06c52b1460fb73e2ec6adb1a98b?logfile=console.log | 12:57 |
| chkumar|rover | once tempest run finishes | 12:57 |
| *** b3nt_pin is now known as beagles | 12:59 | |
| weshay|ruck | chkumar|rover++ | 12:59 |
| chkumar|rover | weshay|ruck, the fix is done by pavel | 13:00 |
| *** ratailor has quit IRC | 13:00 | |
| *** rlandy has joined #oooq | 13:01 | |
| rfolco | weshay|ruck, zbr chkumar|rover sshnaidm panda marios arxcruz scrum now at https://meet.google.com/oiv-geho-mai?authuser=1 | 13:01 |
| *** marios is now known as marios|call | 13:02 | |
| *** yolanda has quit IRC | 13:04 | |
| *** soniya29 has joined #oooq | 13:04 | |
| chkumar|rover | I will be out for a bit | 13:04 |
| soniya29 | weshay|ruck, ping. I had certain issues while connecting to freenode. | 13:06 |
| weshay|ruck | soniya29, np.. I setup a 1-1 for tomorrow | 13:07 |
| weshay|ruck | I have a box for you btw.. just trying to make sure it's running properly | 13:08 |
| rlandy | chkumar|rover: I think I know what is wrong with the key on rhel8 downstream | 13:09 |
| rlandy | the key is copied to tripleo-ci so the path is right | 13:09 |
| rlandy | chkumar|rover: I think it's a primary vs localhost issue | 13:10 |
| *** Goneri has quit IRC | 13:12 | |
| soniya29 | weshay|ruck, Okay. Can you pass me the link of the box? | 13:12 |
| weshay|ruck | soniya29, I will tomorrow | 13:14 |
| soniya29 | weshay|ruck, Okay | 13:15 |
| *** skramaja has quit IRC | 13:18 | |
| *** jpena|lunch is now known as jpena | 13:19 | |
| akahat | weshay|ruck, pong | 13:23 |
| akahat | weshay|ruck, sorry. I was away!! | 13:23 |
| weshay|ruck | akahat, will chat tomorrow.. sorry for the scare :) | 13:23 |
| chkumar|rover | soniya29, ok | 13:23 |
| akahat | weshay|ruck, okay. Np | 13:24 |
| chkumar|rover | rlandy, checking | 13:24 |
| chkumar|rover | sshnaidm, rlandy, https://review.opendev.org/#/c/706196/ please +w | 13:26 |
| chkumar|rover | it will unblock gate | 13:26 |
| *** irclogbot_2 has quit IRC | 13:26 | |
| *** irclogbot_1 has joined #oooq | 13:29 | |
| chkumar|rover | arxcruz, weshay|ruck kopecmartin do we want to cancel tomorrow's tempest meeting? | 13:31 |
| chkumar|rover | I have no any topic to discuss | 13:31 |
| arxcruz | chkumar|rover: up to you | 13:31 |
| chkumar|rover | but I have lots of os_tempest related to tasks | 13:32 |
| chkumar|rover | to discuss | 13:32 |
| chkumar|rover | rlandy, do we need to change the .pem file permission also? | 13:33 |
| rlandy | chkumar|rover: I don;t think so ... see https://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/baremetal-prep-virthost.yml#L4 | 13:34 |
| rlandy | we have to ssh from primary to hardware | 13:35 |
| rlandy | will try fix it after meeting | 13:35 |
| kopecmartin | chkumar|rover: up to you .. i'm fine with cancelling, if weshay|ruck wants i'm available at that time and can discuss collect-logs, but i don't have any tempest urgencies to discuss | 13:35 |
| weshay|ruck | kopecmartin, k.. I'll catch w/ you tomorrow | 13:38 |
| chkumar|rover | kopecmartin, ok keeping it for collect0logs | 13:40 |
| chkumar|rover | weshay|ruck, let me know when free, we can finish 1:1 early | 13:41 |
| weshay|ruck | chkumar|rover, aye.. catching up w/ you right after scrum | 13:46 |
| chkumar|rover | ok | 13:46 |
| *** amoralej|lunch is now known as amoralej | 13:54 | |
| *** Goneri has joined #oooq | 13:58 | |
| chkumar|rover | sshnaidm, seen this https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-train/2d5698c/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 14:02 |
| chkumar|rover | Ensure system is NTP time synced is failing with weird error | 14:02 |
| sshnaidm | chkumar|rover, nope | 14:03 |
| chkumar|rover | sshnaidm, got this bug https://bugzilla.redhat.com/show_bug.cgi?id=1698135 | 14:04 |
| openstack | bugzilla.redhat.com bug 1698135 in openstack-tripleo-common "Overcloud deployment ansible failed on TASK [Ensure system is NTP time synced]" [Urgent,Closed: notabug] - Assigned to apetrich | 14:04 |
| chkumar|rover | weshay|ruck, 1:1 time | 14:05 |
| sshnaidm | chkumar|rover, seems similar, is it reproducing or just one time? | 14:06 |
| chkumar|rover | sshnaidm, seeing currently | 14:06 |
| sshnaidm | chkumar|rover, maybe some network glitch with ntp.. | 14:06 |
| chkumar|rover | need to take a look at previous jobs aslo | 14:06 |
| weshay|ruck | chkumar|rover, aye.. 2 secs for bio and then joining | 14:07 |
| chkumar|rover | weshay|ruck, o | 14:09 |
| chkumar|rover | ok | 14:09 |
| chkumar|rover | weshay|ruck, https://review.opendev.org/#/c/706196/ | 14:11 |
| *** marios|call is now known as marios | 14:11 | |
| chkumar|rover | weshay|ruck, https://review.rdoproject.org/r/#/c/24803/ | 14:12 |
| marios | arxcruz: sorry. am just sick of that fs20 it is dragging on for too long | 14:14 |
| marios | arxcruz: i was unnecessarily grumpy | 14:14 |
| marios | arxcruz: please accept my apology | 14:14 |
| chkumar|rover | weshay|ruck, https://review.opendev.org/#/c/706215/ | 14:14 |
| arxcruz | marios: will you come to berlin in the summit? if so, you need to pay me one wheinstephaner so i can apologize you | 14:18 |
| zbr | marios: i hope that the non-random logic explanation made sense. pick any rule you want as long is predictable, sort, pick first, second and last. | 14:18 |
| *** ccamacho has quit IRC | 14:25 | |
| *** chkumar|rover is now known as raukadah | 14:25 | |
| marios | zbr: let me post it and we can discuss specifically on the review | 14:25 |
| marios | zbr: will be there https://review.rdoproject.org/r/24807 | 14:25 |
| zbr | marios: sure. i was worried that random will make people recheck blindly and not even look at it. | 14:26 |
| *** ccamacho has joined #oooq | 14:33 | |
| raukadah | sshnaidm, https://review.rdoproject.org/r/#/c/24802/ and https://review.rdoproject.org/r/#/c/24701/ | 14:51 |
| ykarel | weshay|ruck can u review https://review.opendev.org/#/c/705379/ | 14:57 |
| weshay|ruck | ykarel, please add Depends-On: https://review.opendev.org/#/c/706196/ | 14:59 |
| ykarel | weshay|ruck, ack doing | 14:59 |
| raukadah | weshay|ruck, rfolco https://review.rdoproject.org/r/#/c/24795/ regarding ceph https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo#L17-L32 | 15:00 |
| ykarel | Done | 15:01 |
| raukadah | sshnaidm, what is the different between normal and lite ovb jobs? it is just a difference between controllers na or something else? | 15:05 |
| raukadah | I want to create the same for rhel8 | 15:05 |
| rfolco | raukadah, please elaborate | 15:06 |
| raukadah | rfolco, kolla people used above repo to get ceph related packages | 15:07 |
| raukadah | we can use the same of c8 container build | 15:07 |
| raukadah | *same in | 15:07 |
| sshnaidm | raukadah, 1ctrl and 3ctrl | 15:07 |
| sshnaidm | raukadah, all the rest is the same | 15:08 |
| raukadah | sshnaidm, ok, I will get one up tomorrow | 15:08 |
| sshnaidm | raukadah, weshay|ruck just fyi, this may overload rdo-cloud: https://review.rdoproject.org/r/#/c/24802/ | 15:10 |
| weshay|ruck | raukadah, sshnaidm imho we should replace centos-7 fs001 w/ rhel | 15:11 |
| weshay|ruck | so we avoid overloading rdo | 15:11 |
| raukadah | sshnaidm, weshay|ruck do we want to switch to lighter version then? | 15:11 |
| weshay|ruck | raukadah, no.. just switch to rhel8 when it's stable | 15:12 |
| raukadah | weshay|ruck, ok | 15:12 |
| sshnaidm | raukadah, we can try to run something in lite mode, either c7 or r8 | 15:12 |
| sshnaidm | raukadah, because together it will start overload and fail | 15:13 |
| raukadah | sshnaidm, yes make sense | 15:13 |
| sshnaidm | raukadah, but.. I'm running now 035 job on vexxhost, maybe we can replace it with 001 on vexxhost and on rdo-cloud to run only rhel8 ovb | 15:13 |
| sshnaidm | weshay|ruck, ^ | 15:13 |
| sshnaidm | so we'll cover both | 15:14 |
| zbr | weshay|ruck: so the gates of... are now open to receive rechecks? | 15:14 |
| raukadah | zbr, still nope | 15:15 |
| weshay|ruck | zbr, if you have the right depends on | 15:15 |
| weshay|ruck | fix hasn't merged | 15:15 |
| weshay|ruck | but it's in the gate | 15:15 |
| raukadah | zbr, https://review.opendev.org/#/c/706196/ to land first | 15:15 |
| raukadah | weshay|ruck, can we update the topic to avoid recheck also | 15:15 |
| zbr | let me grab the popcorn and watch zuul dashboard | 15:15 |
| weshay|ruck | raukadah, I did | 15:15 |
| weshay|ruck | in #tripleo | 15:16 |
| weshay|ruck | not that people read it | 15:16 |
| *** ykarel is now known as ykarel|away | 15:17 | |
| raukadah | sshnaidm, in longer run, we will start moving c8 jobs also to vexxhost? since we will start adding them? | 15:19 |
| sshnaidm | raukadah, everything will move from rdo cloud to vexx in some point | 15:19 |
| *** TrevorV has joined #oooq | 15:30 | |
| raukadah | weshay|ruck, arxcruz ykarel|away and I am working to finding permanent solution for stackviz | 15:35 |
| raukadah | all stuff via package | 15:35 |
| weshay|ruck | raukadah, not sure how to reconcile http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario002-standalone&branch=stable%2Fstein w/ the stein promotion | 15:35 |
| raukadah | it is kind of complicated bist | 15:35 |
| *** ccamacho has quit IRC | 15:36 | |
| weshay|ruck | 2020-02-06 12:26:00.151181 | primary | TASK [validate-tempest : Generate XML results file] **************************** | 15:36 |
| weshay|ruck | 2020-02-06 12:26:00.172354 | primary | Thursday 06 February 2020 12:26:00 +0000 (0:00:01.272) 1:40:56.431 ***** | 15:36 |
| weshay|ruck | 2020-02-06 12:26:01.372423 | primary | fatal: [undercloud]: FAILED! => { | 15:36 |
| weshay|ruck | 2020-02-06 12:26:01.372567 | primary | "changed": true, | 15:36 |
| weshay|ruck | 2020-02-06 12:26:01.379881 | primary | "cmd": "set -o pipefail &&subunit2junitxml /home/zuul/tempest/testrepository.subunit --output-to /home/zuul/tempest/tempest.xml 2>&1 >> tempest.log\n", | 15:36 |
| weshay|ruck | 2020-02-06 12:26:01.380889 | primary | "delta": "0:00:00.662401", | 15:36 |
| weshay|ruck | 2020-02-06 12:26:01.381016 | primary | "end": "2020-02-06 12:26:01.348974", | 15:36 |
| weshay|ruck | raukadah, this is the fixed by the patch from you and pavel ^ | 15:36 |
| raukadah | weshay|ruck, nope | 15:36 |
| * weshay|ruck doesn't see tempest logs https://1dc19a8f26b6d8fa6b0f-1ea15234ebc61f9ad6f390769a79bf22.ssl.cf2.rackcdn.com/705076/3/check/tripleo-ci-centos-7-scenario002-standalone/06af554/logs/ | 15:37 | |
| raukadah | weshay|ruck, wait | 15:37 |
| weshay|ruck | got it https://1dc19a8f26b6d8fa6b0f-1ea15234ebc61f9ad6f390769a79bf22.ssl.cf2.rackcdn.com/705076/3/check/tripleo-ci-centos-7-scenario002-standalone/06af554/logs/undercloud/home/zuul/tempest.log | 15:37 |
| weshay|ruck | ah skip list 2020-02-06 12:25:53 | tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_boot_server_from_encrypted_volume_luks[compute,id-cb78919a-e553-4bab-b73b-10cf4d2eb125,volume] | 15:37 |
| raukadah | weshay|ruck, few days back, we got nasty bug related to ansible and openstackclient | 15:38 |
| raukadah | weshay|ruck, pavel and i fixed this https://bugs.launchpad.net/tripleo/+bug/1862134 | 15:38 |
| openstack | Launchpad bug 1862134 in tripleo "upstream tripleo jobs using os_tempest failing with ERROR: No matching distribution found for oslo.db===7.0.0 " [Critical,In progress] - Assigned to chandan kumar (chkumar246) | 15:38 |
| raukadah | weshay|ruck, https://bugs.launchpad.net/tripleo/+bug/1860810 is the stein issue so we switch back to validate tempest | 15:39 |
| openstack | Launchpad bug 1860810 in tripleo "os_ansible_tempest fails w/ create_network() got an unexpected keyword argument 'dns_domain'" [High,Triaged] | 15:39 |
| *** chem has quit IRC | 15:39 | |
| raukadah | weshay|ruck, https://github.com/ansible/ansible/pull/66944 needed this but it will not fix the problem | 15:39 |
| weshay|ruck | raukadah, ah you got it https://bugs.launchpad.net/tripleo/+bug/1862142 | 15:39 |
| openstack | Launchpad bug 1862142 in tripleo "test_boot_server_from_encrypted_volume_luks failing with u'Key manager error on stable/stein scenario 2" [Critical,Confirmed] | 15:39 |
| raukadah | weshay|ruck, we need to request rdo friends to get latest openstacksdk which is dependent on upstream uc which is not gona happen | 15:40 |
| *** chem has joined #oooq | 15:41 | |
| raukadah | we are now finding issues using tripleo in ansible | 15:41 |
| raukadah | released one | 15:41 |
| raukadah | weshay|ruck, I would we can start gating ansible also as a third party gating, what do you say? | 15:42 |
| weshay|ruck | raukadah, I say we have enough on our plate.. you can write it up, but not approved for working on it | 15:44 |
| raukadah | weshay|ruck, yes I will do that | 15:45 |
| *** fmount has quit IRC | 15:57 | |
| *** bogdando has quit IRC | 15:59 | |
| *** fmount has joined #oooq | 15:59 | |
| rlandy | zbr: you worked on scneario000? | 16:00 |
| *** dsneddon has joined #oooq | 16:01 | |
| weshay|ruck | raukadah, thanks! https://review.rdoproject.org/r/#/c/24795/6..8/zuul.d/projects.yaml | 16:03 |
| raukadah | weshay|ruck, i have done nothing thanks to jpena and zbr for the help | 16:05 |
| *** sshnaidm is now known as sshnaidm|afk | 16:05 | |
| *** jpena is now known as jpena|off | 16:06 | |
| *** jmasud has quit IRC | 16:12 | |
| *** jmasud has joined #oooq | 16:13 | |
| *** dtantsur is now known as dtantsur|afk | 16:19 | |
| marios | zbr: panda: sshnaidm|afk: weshay|ruck: (and anyone else with reviews time) updated https://review.rdoproject.org/r/#/c/24807/ ... its the patch we discussed in scrum with the ppc manifests. let me know what you think on patch when you have time thank you | 16:27 |
| * weshay|ruck looks | 16:27 | |
| weshay|ruck | marios, btw... so thanks for doing this | 16:28 |
| weshay|ruck | marios, the promoter appeared to be working, but never promoted master | 16:28 |
| marios | weshay|ruck: some of that is molecule $stuff so it looks like a big review, but mainly: https://review.rdoproject.org/r/#/c/24807/3/ci-scripts/container-push/roles/containers-promote/tasks/manifest-push.yml to add a switch for ppc manifests. and then | 16:28 |
| marios | weshay|ruck: this one is the 'check N random containers and flip the switch' : https://review.rdoproject.org/r/#/c/24807/3/ci-scripts/container-push/roles/containers-promote/tasks/check_set_ppc_manifests.yml | 16:29 |
| marios | weshay|ruck: once ci reports, you can see the molecule-container-push job for evidence of this working | 16:29 |
| marios | thanks | 16:29 |
| marios | reviews needed please https://review.opendev.org/#/c/701016/19 & Depends-On: https://review.rdoproject.org/r/24781 featurset 20 split thanks | 16:36 |
| marios | rfolco: rlandy: weshay|ruck: panda: sshnaidm|afk: zbr: arxcruz: please when you next have time thank you ^^^ | 16:39 |
| rlandy | ack | 16:39 |
| weshay|ruck | marios, rfolco I think the diff between what I'm seeing in ci vs. local is due to already having a built base container | 16:40 |
| * weshay|ruck cleaning up and retrying | 16:40 | |
| marios | weshay|ruck: the ppc stuff took way longer to do (seemingly simple task... etc ;)) so i will revisit that tomorrow | 16:40 |
| weshay|ruck | marios, aye.. story of our lives.. $too_many_things | 16:41 |
| marios | weshay|ruck: hopefully that is now mostly done but i think there might be some molecule to fix plus any discussion around the random but we'll see hopefully i don't have to spend too much more on that | 16:41 |
| weshay|ruck | bikeshed | 16:41 |
| marios | weshay|ruck: well not really in this case i think that is a bit harsh. i get the issue with the 'randomness' but i have dealt with it in that bit https://review.rdoproject.org/r/#/c/24807/3/ci-scripts/infra-setup/roles/promoter/molecule/container-push/playbook.yml@160 ... the tests are now always deterministic via conditional when | 16:43 |
| zbr | marios: that jinja construct made me dizzy :D | 16:44 |
| marios | zbr: which one | 16:44 |
| zbr | https://review.opendev.org/#/c/701016/19/config/general_config/featureset001.yml | 16:44 |
| raukadah | zbr, you can find similar patterns in os_tempest and validate_tempest where used | 16:45 |
| zbr | i have some doubts it would render to what you want | 16:45 |
| marios | zbr: ah, cross topic. yeah but that is used in more places it isn't my invention, {% if ('periodic' in zuul.pipeline and not job.force_non_periodic|default(false)|bool) or (job.force_periodic|default(false)|bool) -%} | 16:45 |
| raukadah | it all started back when we replaced pingtest with tempest arxcruz knows the feelong | 16:45 |
| marios | zbr: it renders fine see the link there | 16:45 |
| raukadah | marios, it is rlandy and sshnaidm|afk creation | 16:45 |
| marios | zbr: https://logserver.rdoproject.org/39/24339/9/check/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/d3d3ec7/logs/undercloud/var/log/tempest/test_list.txt.gz | 16:46 |
| zbr | marios: +W both, YOLO | 16:46 |
| raukadah | zbr, I am just an user | 16:46 |
| marios | zbr: ;) thanks for checking | 16:46 |
| marios | zbr: err, both/ | 16:47 |
| marios | zbr: i'd rather we follow the 2 +2 rule to be honest | 16:47 |
| marios | zbr: for https://review.rdoproject.org/r/#/c/24781/ | 16:47 |
| raukadah | zbr, you +w it, ? it is going to fail in upstream | 16:47 |
| marios | and in general | 16:47 |
| marios | zbr: unless there is an emergency which it is not in this case | 16:47 |
| * rlandy reads back | 16:47 | |
| raukadah | sorry it's rdo | 16:47 |
| zbr | is rdo, it does not block. | 16:48 |
| * marios almost out | 16:48 | |
| raukadah | thanks to our epel hack and experiment | 16:48 |
| raukadah | weshay|ruck, https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ecf/706215/3/check/tripleo-ci-centos-7-scenario001-standalone/ecfbb50/ post faolure on upstream jobs | 16:49 |
| zbr | i need some feedback on readme work, example result https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b5e/705454/8/check/tripleo-ci-centos-7-containers-undercloud-minion/b5ec965/logs/readme.html | 16:50 |
| raukadah | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_ecf/706215/3/check/tripleo-ci-centos-7-scenario001-standalone/ecfbb50/ | 16:50 |
| raukadah | fetch-output : Collect log output | 16:50 |
| raukadah | "msg": "ssh_exchange_identification: read: Connection reset by peer\r\nrsync: connection unexpectedly closed (0 bytes received so far) [Receiver]\nrsync error: unexplained error (code 255) at io.c(226) | 16:50 |
| raukadah | something got broken usptream | 16:50 |
| marios | raukadah: you mean the fs20 patch? 18:47 < raukadah> zbr, you +w it, ? it is going to fail in upstream | 16:51 |
| marios | raukadah: or something else | 16:51 |
| weshay|ruck | raukadah, probably the same issue w/ the containers blocking collect logs | 16:52 |
| raukadah | marios, i thought you are taking about upstream patches | 16:52 |
| weshay|ruck | raukadah, go away | 16:52 |
| *** weshay|ruck is now known as weshay | 16:52 | |
| raukadah | marios, but it was rdo patches, upstream job in bad shape | 16:52 |
| marios | raukadah: k well one of them is upstream https://review.opendev.org/#/c/701016/19 | 16:53 |
| marios | raukadah: so i guess it won't merge | 16:53 |
| raukadah | marios, yes, sorry for that | 16:54 |
| marios | raukadah: ack thanks | 16:54 |
| *** tesseract has quit IRC | 16:54 | |
| raukadah | marios, but you can put depends on https://review.opendev.org/706196 and +w and make life happy | 16:55 |
| marios | raukadah: tomrrow | 16:55 |
| *** marios is now known as marios|out | 16:55 | |
| raukadah | marios|out, aye, see ya :-) | 16:56 |
| rlandy | weshay: so ... running standalone on a multinode nodeset is a go ... https://review.opendev.org/#/c/706288/ | 16:56 |
| rfolco | raukadah, can you please elaborate on this link you sent me ? https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo#L17-L32 | 16:59 |
| rfolco | raukadah, chandan-api --verbose get-explanation https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo#L17-L32 | 16:59 |
| raukadah | rfolco, amoralej was asking about c8 container status, then i looked into centos8 base image failure then we get this kolla hack for ceph 8 and i thought we can add these repo to our conatiner build and go ahead | 17:02 |
| amoralej | rfolco, those are repos with el8 ceph packages that can be used to unblock containers for centos8 until final builds are shipped in centos repos | 17:04 |
| raukadah | rfolco, amoralej has suggested some change on https://review.opendev.org/#/c/702993/ re opstools and may be we can add those repo or resue that | 17:04 |
| weshay | rfolco, | 17:07 |
| weshay | Error: | 17:07 |
| weshay | Problem: package python3-paramiko-2.6.0-5.el8.noarch requires python3-cryptography >= 2.5, but none of the providers can be installed | 17:07 |
| weshay | - package python3-paramiko-2.6.0-5.el8.noarch requires python3.6dist(cryptography) >= 2.5, but none of the providers can be installed | 17:07 |
| weshay | - python3-cryptography-2.8-1.el8.x86_64 does not belong to a distupgrade repository | 17:07 |
| weshay | - problem with installed package python3-paramiko-2.6.0-5.el8.noarch | 17:07 |
| weshay | (try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) | 17:07 |
| weshay | have you seen this? | 17:07 |
| weshay | amoralej, since ur here | 17:07 |
| amoralej | weshay, where is that? | 17:08 |
| amoralej | log? | 17:08 |
| weshay | it's local | 17:08 |
| rfolco | weshay, haven't seen this | 17:08 |
| weshay | k | 17:08 |
| amoralej | nop | 17:08 |
| amoralej | weshay, you used http://trunk.rdoproject.org/centos8-master/delorean-deps.repo? | 17:10 |
| amoralej | may be missing module_hotfixes=1 | 17:10 |
| * weshay looks | 17:10 | |
| rfolco | raukadah, amoralej: I thought this would be enough | 17:11 |
| rfolco | https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo | 17:11 |
| amoralej | rfolco, doesn't we override repos on tripleo-common? | 17:12 |
| amoralej | i mean, if we are reusing that, it'd be fine | 17:12 |
| rfolco | amoralej, k | 17:13 |
| amoralej | rfolco, as per https://logserver.rdoproject.org/95/24795/8/check/tripleo-build-containers-centos-8-master/a77464f/logs/build-rhel-base.log | 17:13 |
| amoralej | current blocker is the OpsTools repo file | 17:13 |
| amoralej | weshay, what command are you using for that error? | 17:14 |
| amoralej | "does not belong to a distupgrade repository" | 17:15 |
| weshay | amoralej, think that fixed it | 17:18 |
| amoralej | ok | 17:18 |
| weshay | thanks man | 17:18 |
| weshay | amoralej, btw.. we can't work around the opstools stuff atm | 17:18 |
| amoralej | weshay, i think opstools has two issues | 17:18 |
| weshay | it's never used.. only disabled in kolla for centos-8 | 17:18 |
| amoralej | one is the package installation | 17:18 |
| amoralej | i think that can be fine | 17:19 |
| *** marios|out has quit IRC | 17:19 | |
| amoralej | but about the CentOS-OpsTools.repo file | 17:19 |
| amoralej | can't that copy just being removed? | 17:19 |
| amoralej | i mean https://review.opendev.org/#/c/702993/21/roles/build-containers/templates/Dockerfile.centos8.j2 | 17:19 |
| amoralej | linke 5 | 17:19 |
| amoralej | i'd say removing that may be just be fine | 17:20 |
| rfolco | kolla expects opstools repo | 17:21 |
| rfolco | and if we are going to have opstools in centos8, there is no point in patching kolla | 17:21 |
| amoralej | but kolla is passing with c8 | 17:21 |
| rfolco | where? | 17:21 |
| rfolco | I mean, what you mean by passing? | 17:22 |
| amoralej | let me look for it | 17:22 |
| *** yolanda has joined #oooq | 17:22 | |
| amoralej | i.e https://review.opendev.org/#/c/705986/ - kolla-build-centos8-binary passed and i see containers there | 17:23 |
| amoralej | are there missing ones there? | 17:23 |
| rfolco | I assume it does not build 129 containers like in overcloud container build job for tripleo | 17:25 |
| amoralej | rfolco, it'd still be useful to know what packages are needed from opstools that are not available yet, we may be able to workaround in rdo deps repo | 17:26 |
| amoralej | as we've done with collectd | 17:26 |
| amoralej | if it's simple enough | 17:26 |
| *** jpena|off is now known as jpena | 17:27 | |
| rfolco | amoralej, what you see here https://review.opendev.org/#/c/702993/21/roles/build-containers/templates/Dockerfile.centos8.j2 is just a fake opstools repo file | 17:27 |
| amoralej | rfolco, maybe i'm checking the wrong review to track | 17:28 |
| amoralej | i'm checking https://review.rdoproject.org/r/#/c/24795/ | 17:28 |
| amoralej | jobs there are failing because repos/CentOS-OpsTools.repo does not exist | 17:29 |
| amoralej | that's why i suggested that | 17:29 |
| amoralej | is there a fake file in other job that i can check? | 17:29 |
| rfolco | amoralej, yeah the non-existing file is weshay fault | 17:32 |
| rfolco | he realized that yesterday and was going to add task to create it | 17:33 |
| weshay | I have | 17:33 |
| amoralej | rfolco, what i'm trying is to track what exact packages are still missing for tripleo jobs so that we can look for workarounds to unblock | 17:34 |
| *** tosky has quit IRC | 17:34 | |
| amoralej | so far, it seems ceph from the copr still needs some additional dep that is taken from epel in kolla | 17:34 |
| amoralej | i'd like to detect those | 17:35 |
| rfolco | amoralej, I am struggling with the same thing. I am spinning wheels trying to find missing requirements for the missing containers. | 17:36 |
| *** saneax has quit IRC | 17:53 | |
| weshay | rfolco, think I got this nailed | 17:53 |
| *** saneax has joined #oooq | 17:54 | |
| rfolco | weshay, God bless you, share with me pls | 17:54 |
| weshay | k.. rfolco submmitting latest.. and kicking ci | 17:54 |
| amoralej | rfolco, so far we have detected liboath package which is required to ceph-common and it's coming from epel, we may temporarily ship it in rdo deps until storage sig provides full ceph repo | 17:54 |
| zbr | sshnaidm|afk: now is safe to look again at https://review.opendev.org/#/c/705623/ | 17:55 |
| *** saneax has quit IRC | 17:55 | |
| amoralej | scratch https://cbs.centos.org/koji/taskinfo?taskID=1145011 | 17:56 |
| *** saneax has joined #oooq | 17:58 | |
| *** derekh has quit IRC | 18:00 | |
| *** holser has quit IRC | 18:04 | |
| weshay | rfolco, ok.. let's sync | 18:04 |
| rfolco | weshay, link | 18:04 |
| weshay | https://meet.google.com/dxg-okuu-pvj?authuser=1 | 18:04 |
| weshay | rfolco, podman rmi --all | 18:08 |
| *** jpena is now known as jpena|off | 18:22 | |
| *** amoralej is now known as amoralej|off | 18:22 | |
| weshay | 2020-02-06 18:18:36 | ============= SUMMARY =============== | 18:24 |
| weshay | 2020-02-06 18:18:36 | Expected................... 125 | 18:24 |
| weshay | 2020-02-06 18:18:36 | Built...................... 114 | 18:24 |
| weshay | 2020-02-06 18:18:36 | ===================================== | 18:24 |
| weshay | rfolco, going to repin dlrn | 18:24 |
| rfolco | k | 18:24 |
| weshay | cat containers-failed-to-build.log | 18:24 |
| weshay | nova-libvirt | 18:24 |
| weshay | cinder-base | 18:24 |
| weshay | cinder-api | 18:24 |
| weshay | cinder-backup | 18:24 |
| weshay | cinder-scheduler | 18:24 |
| weshay | cinder-volume | 18:24 |
| weshay | glance-base | 18:24 |
| weshay | glance-api | 18:24 |
| weshay | manila-share | 18:24 |
| weshay | rabbitmq | 18:24 |
| rfolco | failed ok | 18:25 |
| rfolco | so I am not crazy | 18:25 |
| weshay | rfolco, the issue w/ nova-libvirt is | 18:42 |
| weshay | Error: Unable to find a match: ceph-common | 18:42 |
| ykarel|away | weshay, we have added missing liboath for ceph-common temporary in RDO, rebuilding with fresh rdo repos should pass | 18:45 |
| weshay | ykarel|away, what's the context? | 18:45 |
| weshay | ceph-common | 18:45 |
| ykarel|away | weshay, yes | 18:45 |
| weshay | ykarel|away, which ceph repo should we use? | 18:45 |
| ykarel|away | weshay, the one shipped with kolla | 18:45 |
| ykarel|away | until storage sig gets ready | 18:46 |
| weshay | ykarel|away, which one is shipped w/ kolla | 18:46 |
| ykarel|away | weshay, https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo | 18:46 |
| ykarel|away | you don't need to explicitly enable | 18:46 |
| ykarel|away | those should be auto enabled | 18:46 |
| ykarel|away | in kolla | 18:46 |
| ykarel|away | unless and until u are using some kolla patch to override | 18:47 |
| weshay | I'll have to run one and mount it | 18:47 |
| weshay | to check | 18:47 |
| ykarel|away | ok, u can run container with base image | 18:48 |
| weshay | ykarel|away, whats up w/ https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo#L3 | 18:48 |
| ykarel|away | weshay, CHACRA_URL=https://2.chacra.ceph.com/r/ceph/master/e6630e6a2918f029e17d7b5af8b92ac3a713273f/centos/8/flavors/default/ | 18:49 |
| weshay | ykarel|away, k.. will try this now | 18:49 |
| weshay | ykarel|away, thank you | 18:49 |
| weshay | ykarel|away, /me notes I'm totally clueless where ceph lives..etc | 18:50 |
| *** dpawlik has quit IRC | 18:50 | |
| ykarel|away | so with ceph-common, all should pass, except glance-api,base and rabbitmq | 18:50 |
| ykarel|away | in above list | 18:50 |
| ykarel|away | glance-api, base were passing in my test, so not sure why it's failing at yours | 18:51 |
| ykarel|away | for rabbitmq needs some adjustment | 18:51 |
| weshay | ykarel|away, k.. I had nothing defined for ceph | 18:52 |
| weshay | atm | 18:52 |
| ykarel|away | weshay, how u checked | 18:52 |
| weshay | sorry? | 18:52 |
| weshay | ykarel|away, how am I building? | 18:52 |
| ykarel|away | i mean u said, I had nothing defined for ceph | 18:53 |
| ykarel|away | i didn't got ^^ | 18:53 |
| weshay | ykarel|away, oh.. I didn't have any yum repos defined for ceph.. | 18:53 |
| weshay | and they don't appear to be dropping in automatically but I've seen some kolla patches around it | 18:54 |
| weshay | tbh, I was focused on bootstrapping the job.. | 18:54 |
| weshay | think I'm there | 18:54 |
| ykarel|away | weshay, so if u run container from centos-binary-base image | 18:55 |
| ykarel|away | kolla is running dnf config-manager --enable ceph --enable ceph-noarch --enable ceph-el8 --enable ceph-lab-extras wherever ceph packages are needed | 18:55 |
| ykarel|away | so to test, run container from base image, run ^^ command, and try installing ceph-common | 18:56 |
| ykarel|away | weshay, okk locally i have 126 containers build | 18:57 |
| ykarel|away | i guess that's all | 18:57 |
| *** tosky has joined #oooq | 18:59 | |
| weshay | ykarel|away, that fixed it | 19:09 |
| weshay | thank you | 19:09 |
| weshay | rfolco, ok.. nova-libvirt and others should be working soon | 19:12 |
| rfolco | weshay, what is the change? | 19:13 |
| weshay | zbr, hey | 19:25 |
| weshay | zbr, why would you do this??? https://review.opendev.org/#/c/701016/ | 19:25 |
| weshay | zbr, why would you workflow that w/o the depends-on? | 19:25 |
| zbr | both where +W | 19:26 |
| weshay | zbr, you know how this works man | 19:26 |
| weshay | zbr, you just reset the entire gate | 19:26 |
| zbr | maybe i missed something, meet? | 19:28 |
| weshay | zbr, https://meet.google.com/epc-wapx-xtv?authuser=1 | 19:28 |
| *** fmount has quit IRC | 19:32 | |
| *** fmount has joined #oooq | 19:34 | |
| *** Goneri has quit IRC | 19:54 | |
| *** openstackstatus has joined #oooq | 19:55 | |
| *** ChanServ sets mode: +v openstackstatus | 19:55 | |
| rfolco | weshay, apparently kolla silently skips/ignores containers that fail to install deps, it only fails if repo is missing | 20:10 |
| weshay | rfolco, so are they showing up in the passed list? | 20:10 |
| rfolco | weshay, well, I am not quite sure, but for me its like it was failing on ceph-common, and not giving an error, just moving forward | 20:13 |
| rfolco | weshay, I fixed ceph repo as ykarel|away suggested, now building again. The image-list command reports now 134 containers to build | 20:13 |
| rfolco | let's see | 20:13 |
| rfolco | weshay, perhaps there is a way to make kolla report a more verbose error when something bad inside the container happens | 20:15 |
| weshay | rfolco, k.. /me keeping https://hackmd.io/cv1483MkQlS8UsnteNyXqg?both up2date | 20:15 |
| weshay | rfolco, would be nice eh | 20:15 |
| weshay | lolz | 20:15 |
| rfolco | ok, updating mine too | 20:16 |
| weshay | rfolco, rabbit fails on No match for argument: rabbitmq-server-3.7.* | 20:16 |
| rfolco | weshay, unpin 3.7 and you'll be passing this one | 20:17 |
| rfolco | s/3.7// | 20:17 |
| rfolco | s/3.7//g | 20:17 |
| weshay | rfolco, where? | 20:17 |
| rfolco | let me get the patch | 20:17 |
| rfolco | weshay, https://review.opendev.org/#/c/703640/2/docker/rabbitmq/Dockerfile.j2 | 20:18 |
| weshay | rfolco, let's fix the conflict k/ | 20:21 |
| weshay | ? | 20:21 |
| rfolco | yeah you have to rebase or apply it manually | 20:22 |
| rfolco | lets rebase on master and fix qrouterd then | 20:22 |
| rfolco | doing that | 20:22 |
| rfolco | weshay, let me double check first.. Tried to install now and it has conflicts. | 20:28 |
| rfolco | weshay, but I was pretty sure I built it previously | 20:29 |
| rfolco | weshay, updated, but w/ a WIP to test | 20:34 |
| weshay | sure | 20:34 |
| weshay | thanks | 20:34 |
| *** jfrancoa has quit IRC | 20:52 | |
| *** irclogbot_1 has quit IRC | 20:53 | |
| *** irclogbot_2 has joined #oooq | 20:53 | |
| *** rlandy is now known as rlandy|afk | 21:00 | |
| rlandy|afk | going to get the last of vaccinations set | 21:00 |
| *** Goneri has joined #oooq | 21:11 | |
| weshay | AND "branch" = 'train' | 21:14 |
| weshay | oops | 21:14 |
| weshay | 2020-02-06 21:12:00 | ============= SUMMARY =============== | 21:14 |
| weshay | 2020-02-06 21:12:00 | Expected................... 126 | 21:14 |
| weshay | 2020-02-06 21:12:00 | Built...................... 126 | 21:14 |
| weshay | 2020-02-06 21:12:00 | ===================================== | 21:14 |
| weshay | 2020-02-06 21:12:00 | | 21:14 |
| weshay | 2020-02-06 21:12:00 | | 21:14 |
| weshay | 2020-02-06 21:12:00 | Return code: [ 0 ] | 21:14 |
| weshay | 2020-02-06 21:12:00 | | 21:14 |
| weshay | 2020-02-06 21:12:00 | | 21:14 |
| weshay | 2020-02-06 21:12:00 | SUCCESS: Congratulations! All containers have been built :) | 21:14 |
| *** TrevorV has quit IRC | 21:16 | |
| *** Goneri has quit IRC | 21:18 | |
| rfolco | weshay, SystemError: nova-libvirt raised Unexpected error while running command. | 21:18 |
| rfolco | weshay, how to fix this | 21:18 |
| weshay | https://review.opendev.org/#/c/702993/24/roles/build-containers/tasks/main.yaml | 21:20 |
| weshay | https://github.com/openstack/kolla/blob/d00fa3a6dda3914e66ed414ef758bf2aeda2906c/docker/base/ceph_master.repo | 21:20 |
| rfolco | $ sudo podman images -a | grep tripleomaster | wc -l | 21:20 |
| rfolco | 125 | 21:20 |
| rfolco | [centos@centos8- | 21:20 |
| weshay | rfolco, ur never going to get the real error w/o | 21:20 |
| weshay | running | 21:20 |
| weshay | sudo buildah bud --tls-verify=False --logfile /var/local/tmp/nova-libvirt-build.log -t trunk.registry.rdoproject.org/tripleomaster/centos-binary-nova-libvirt:abcdef | 21:20 |
| weshay | sudo buildah bud --tls-verify=False --logfile /var/local/tmp/nova-libvirt-build.log -t trunk.registry.rdoproject.org/tripleomaster/centos-binary-nova-libvirt:abcdef nova/nova-libvirt | 21:21 |
| weshay | rather | 21:21 |
| rfolco | no cinder :( | 21:21 |
| rfolco | weshay, cinder is in list-images but didn't built it | 21:23 |
| weshay | 2020-02-06 21:12:00 | trunk.registry.rdoproject.org/tripleomaster/centos-binary-cinder-base abcdef 17b632fd1af5 12 minutes ago 987 MB | 21:24 |
| weshay | 2020-02-06 21:12:00 | trunk.registry.rdoproject.org/tripleomaster/centos-binary-cinder-api abcdef ab5bf4192203 9 minutes ago 1.07 GB | 21:24 |
| weshay | 2020-02-06 21:12:00 | trunk.registry.rdoproject.org/tripleomaster/centos-binary-cinder-backup abcdef 6309239d3e99 9 minutes ago 1.12 GB | 21:24 |
| weshay | 2020-02-06 21:12:00 | trunk.registry.rdoproject.org/tripleomaster/centos-binary-cinder-scheduler abcdef 86be1c96e4e7 10 minutes ago 987 MB | 21:24 |
| weshay | 2020-02-06 21:12:00 | trunk.registry.rdoproject.org/tripleomaster/centos-binary-cinder-volume abcdef 09328c4d9949 9 minutes ago 1.13 G | 21:24 |
| weshay | think I got it | 21:24 |
| rfolco | ok | 21:24 |
| rfolco | gotta go | 21:25 |
| rfolco | ttyl | 21:25 |
| *** Goneri has joined #oooq | 21:56 | |
| *** yolanda has quit IRC | 22:24 | |
| *** jmasud has quit IRC | 22:24 | |
| *** Goneri has quit IRC | 22:31 | |
| *** rlandy|afk is now known as rlandy | 22:32 | |
| rlandy | weshay: can we recheck patches now? | 22:35 |
| weshay | rlandy, aye | 22:52 |
| rlandy | weshay: hi - I rechecked | 22:52 |
| rlandy | executive decision | 22:52 |
| weshay | :) | 22:53 |
| weshay | rlandy, good job executive rlandy | 22:53 |
| rlandy | are we short of nodes in rdcloud? | 23:04 |
| rlandy | rdocloud | 23:04 |
| zbr | weshay: what molecule failure? clearly zuul failed to post any link | 23:05 |
| weshay | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_91b/706215/4/gate/openstack-tox-molecule/91b3cfb/ | 23:06 |
| weshay | zbr, did you see inside my mind? | 23:06 |
| weshay | zbr, are you monitoring the ruck/rover etherpad? | 23:06 |
| weshay | my GOD are you GOD | 23:06 |
| zbr | weshay: that is weird, is someone installing molecule with --pre or directly from git? | 23:07 |
| weshay | no idea | 23:07 |
| zbr | that error is caused by newer schema in v3, i made a new pre-release (3.0a6) an hour ago | 23:08 |
| zbr | yep, that is molecule==3.0a6 | 23:09 |
| zbr | anyway, easy fix it remove any lint sections from molecule.yml file. | 23:09 |
| zbr | example https://review.opendev.org/#/c/706403/ | 23:10 |
| zbr | look for https://review.opendev.org/#/q/topic:mov3+(status:open+OR+status:merged) | 23:10 |
| *** Goneri has joined #oooq | 23:12 | |
| weshay | zbr, is this done? https://projects.engineering.redhat.com/browse/RHOSINFRA-2996 | 23:12 |
| zbr | yep | 23:13 |
| zbr | weshay: going to suspend mode, i bet i will see many issues related to that new pre-release. i did not expect to see our jobs picking it without modification. | 23:22 |
| zbr | porting looks like mov3 tag above | 23:23 |
| weshay | hrm.. | 23:23 |
| weshay | give me more context tomorrow | 23:23 |
| weshay | I'm fried | 23:23 |
| weshay | and you should not be working | 23:23 |
| zbr | me too, | 23:23 |
| rlandy | weshay: are you using the instance we created on the downstream tenant? if not, can I re-purpose it for 16-branch conatiner builds? | 23:26 |
| *** tosky has quit IRC | 23:55 | |
| *** holser has joined #oooq | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!