| *** dviroel|afk is now known as dviroel | 00:08 | |
| *** dviroel is now known as dviroel|out | 01:33 | |
| *** pojadhav|out is now known as pojadhav|ruck | 03:00 | |
| *** yadnesh|away is now known as yadnesh | 04:00 | |
| marios | o/ | 06:09 |
|---|---|---|
| ysandeep | \o hello | 06:13 |
| jm1 | o/ | 06:31 |
| akahat | @team please review : https://review.rdoproject.org/r/c/config/+/45901, https://review.rdoproject.org/r/c/rdo-jobs/+/46181 | 06:32 |
| marios | akahat: https://review.rdoproject.org/r/c/rdo-jobs/+/46181/6#message-9f0d758e4c904751bc679a5a8c62015d7760a640 | 07:23 |
| marios | akahat: left some comments please check when you have time | 07:23 |
| marios | akahat: add those patches to the reviews list for more reviews by the way | 07:29 |
| reviewbot | Do you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks. | 07:29 |
| *** amoralej|off is now known as amoralej | 07:29 | |
| marios | frenzy_friday|rover: o/ https://review.opendev.org/c/openstack/tripleo-quickstart/+/866180 so no longer needed? | 07:56 |
| frenzy_friday|rover | marios, looks like so. I'll keep it till today eod and if everything passes I'll abandon | 08:03 |
| marios | ack frenzy_friday|rover thx | 08:07 |
| *** ysandeep is now known as ysandeep|lunch | 08:12 | |
| *** pojadhav|ruck is now known as pojadhav|lunch | 08:26 | |
| *** jpena|off is now known as jpena | 08:31 | |
| marios | bhagyashris: chandankumar: ysandeep|lunch: dviroel|out: rlandy|out: please add to your review list: https://review.rdoproject.org/r/c/config/+/46161 https://review.rdoproject.org/r/c/rdo-jobs/+/46160 | 08:43 |
| reviewbot | I have added your review to the Review list | 08:43 |
| reviewbot | I have added your review to the Review list | 08:43 |
| * pojadhav|lunch stepping out ~1 hour for imp work | 08:59 | |
| *** yadnesh is now known as yadnesh|afk | 09:00 | |
| *** pojadhav|lunch is now known as pojadhav|ruck | 09:00 | |
| *** ysandeep|lunch is now known as ysandeep | 09:12 | |
| *** yadnesh|afk is now known as yadnesh | 09:58 | |
| Tengu | hello there. Did you find the cause of the /bin/sh: error while loading shared libraries: libc.so.6: cannot change memory protections upon container build? | 10:30 |
| Tengu | we're seeing it in molecule testing in tripleo-ansible as well - I want to know if it's safe to recheck, or if there's still something under investigations. | 10:31 |
| Tengu | also, was it linked to the selinux thing? | 10:31 |
| ysandeep | Tengu: yes it was because of container-selinux (new version leaking from RHEL9.1 content) | 10:34 |
| ysandeep | Tengu, https://bugzilla.redhat.com/show_bug.cgi?id=2143562#c8 | 10:35 |
| Tengu | ah. | 10:35 |
| Tengu | "great". | 10:35 |
| ysandeep | In downstream we excluded the repo that was leaking the affected version of container-selinux | 10:36 |
| *** dviroel|out is now known as dviroel | 10:43 | |
| *** rlandy|out is now known as rlandy | 11:20 | |
| rlandy | Tengu: frenzy_friday|rover: so the selinux thing kind of went away? | 11:21 |
| bhagyashris | marios, rlandy frenzy_friday|rover chandankumar ysandeep please add to your review list https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46303 so that i can test because depends-o is not working in the testproject | 11:21 |
| reviewbot | I have added your review to the Review list | 11:21 |
| rlandy | Tengu; wrt libc.so.6 error - happened once and then not again | 11:22 |
| rlandy | in c9 builds yesterday | 11:22 |
| frenzy_friday|rover | rlandy, yep. I'll close the card eod if I dont see it again. | 11:23 |
| rlandy | frenzy_friday|rover: we have 25 gate failures | 11:23 |
| marios | bhagyashris: ack | 11:23 |
| frenzy_friday|rover | checking | 11:23 |
| rlandy | frenzy_friday|rover: and is zed fs064 still an issue? | 11:23 |
| frenzy_friday|rover | yep, pinged ade_lee | 11:23 |
| rlandy | https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001&skip=0 | 11:25 |
| rlandy | we're also falt red on check ovb | 11:25 |
| frenzy_friday|rover | fs001 - is it the tempest? Checking | 11:25 |
| frenzy_friday|rover | rlandy, fs001 most of the failures are https://bugs.launchpad.net/tripleo/+bug/1998227 | 11:29 |
| frenzy_friday|rover | Some are sporadic | 11:29 |
| rlandy | frenzy_friday|rover: marios: pojadhav|ruck: want to rr sync - since marios and I are taking over next shift | 11:30 |
| rlandy | we can do it now if everyone is around | 11:30 |
| pojadhav|ruck | rlandy, ack | 11:30 |
| pojadhav|ruck | i am here | 11:30 |
| frenzy_friday|rover | rlandy, ack | 11:30 |
| rlandy | waiting to see if marios is around to join | 11:32 |
| marios | rlandy: sure on my way sec | 11:32 |
| marios | ah were are we meeting rlandy :D | 11:33 |
| rlandy | marios: frenzy_friday|rover: pojadhav|ruck: https://meet.google.com/zdj-pnek-udc?pli=1&authuser=0 | 11:34 |
| pojadhav|ruck | rlandy, marios : https://review.gerrithub.io/c/redhat-openstack/infrared/+/543117 | 11:37 |
| ysandeep | dviroel++ | 11:50 |
| rlandy | bhagyashris: marios: frenzy_friday|rover: so looking at https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46303, I am wondering if for 18 - we should go with 18.0 ysandeep and I chatted about this a while back | 12:01 |
| rlandy | using the .minor naming scheme | 12:01 |
| rlandy | the patch looks good - just thinking if it will help us going forward to keep things cleaner if there is 18.1, 2 etc. | 12:02 |
| rlandy | bhagyashris: ^^ can you check with jjoyce about the proposed upcoming 18 releases and if this makes sense | 12:03 |
| rlandy | ysandeep: ^^ appreciate your thoughts here | 12:03 |
| marios | rlandy: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/46303/2#message-285c5a313ac9a1fa8979133ff858b72429ea032d | 12:03 |
| marios | rlandy: not sure it makes a difference in the end so either way ^^^ (cant think of somewhere that we can automate that -0 vs -1 part) | 12:04 |
| ysandeep | rlandy: because the dlrn api endpoint don't have .0 too (rhel9-osp18), I think its okay this way. | 12:04 |
| rlandy | ysandeep: iirc we put some code in to default to .0 if it was not specified | 12:05 |
| rlandy | we needed that for something | 12:05 |
| bhagyashris | rlandy, looking | 12:05 |
| ysandeep | rlandy, iirc it was for build-test-package role.. finding code | 12:06 |
| rlandy | bhagyashris: I think it's ok as is ... because we have corrected for it when needed ^^ | 12:07 |
| rlandy | marios: ysandeep: thanks for your input | 12:07 |
| rlandy | bhagyashris: will +2 | 12:07 |
| bhagyashris | rlandy, ack and agree with sandeep's point (because the dlrn api endpoint don't have .0 too) | 12:08 |
| ysandeep | rlandy, https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/799517/4/roles/build-test-packages/templates/projects.ini.j2 | 12:08 |
| ysandeep | release_minor={{ '0' if '.' not in osp_release.split('-')[1] else osp_release.split('-')[1].split('.')[1] }} | 12:08 |
| rlandy | yep - I remember we fell over it at some point | 12:08 |
| rlandy | marios: since we have your +2 - will w+ that | 12:09 |
| ysandeep | we select minor_release as 0 for both 16/16.0 so we can go i.e way | 12:10 |
| * bhagyashris tea brb | 12:26 | |
| rlandy | chandankumar++ dviroel++ | 12:36 |
| rlandy | thanks for the intro to hive work | 12:36 |
| dviroel | ++ | 12:42 |
| marios | rlandy: link to that? | 12:50 |
| rlandy | marios: following hive section https://docs.engineering.redhat.com/display/OSP/Podified+Control+Plane+-+CI+Documents and chandankumar's card https://issues.redhat.com/browse/OSP-20588 - which I am picking up | 12:55 |
| marios | thanks | 12:56 |
| rlandy | need to go to complex office - brb | 13:03 |
| *** rlandy is now known as rlandy|brb | 13:03 | |
| *** ysandeep is now known as ysandeep|afk | 13:12 | |
| *** rlandy|brb is now known as rlandy | 13:18 | |
| rlandy | frenzy_friday|rover: hey - can you see alan's comment on CIX https://trello.com/c/qzPntXcs/2802-cixlp1998227tripleociproa-unexpected-api-error-class-novaexceptionglanceconnectionfailed - about zed failures | 13:25 |
| rlandy | frenzy_friday|rover: I think you mentioned zed failures as well | 13:26 |
| frenzy_friday|rover | Probably all releases, commenting | 13:26 |
| rlandy | frenzy_friday|rover: alan says only master merged | 13:32 |
| rlandy | frenzy_friday|rover: kicked fs064-zed on internal again | 13:32 |
| rlandy | will let you know | 13:32 |
| frenzy_friday|rover | in zed looks like it is passing | 13:32 |
| frenzy_friday|rover | rlandy, thanks | 13:32 |
| frenzy_friday|rover | lemme check and update in some time | 13:33 |
| *** frenzy_friday|rover is now known as frenzy_friday|rover|lunch | 13:33 | |
| dasm|off | o/ | 13:46 |
| *** dasm|off is now known as dasm | 13:46 | |
| dviroel | dasm: o/ | 13:49 |
| dasm | dviroel: \o | 13:49 |
| dviroel | dasm: poland is classified to knockout stage of world cup \o/ | 13:50 |
| dasm | dviroel: i've heard that too :) | 13:50 |
| dviroel | yey | 13:52 |
| *** yadnesh is now known as yadnesh|away | 13:55 | |
| pojadhav|ruck | hagyashris, chandankumar : scrum time | 14:01 |
| *** frenzy_friday|rover|lunch is now known as frenzy_friday|rover | 14:01 | |
| *** amoralej is now known as amoralej|lunch | 14:03 | |
| *** ysandeep|afk is now known as ysandeep | 14:07 | |
| *** pojadhav|ruck is now known as pojadhav|afk | 14:42 | |
| *** amoralej|lunch is now known as amoralej | 14:42 | |
| * jm1 bbl# | 14:43 | |
| ysandeep | rlandy, dviroel fyi.. i am marking this card to done : "CI - Investigate how to "Create dependency between github action's operator image build and Prow e2e job?" since we know how to do it know if we wants to but not going ahead in-favor of prow image workflow. | 15:01 |
| ysandeep | "/s/know/now* | 15:01 |
| rlandy | pojadhav|afk: marios: is this the new 17.1 bug for us? https://trello.com/c/uPVjLFSE/2805-cixbz2149963osp17-1rhel9pipelineintegration-pcci-171dlrn-rhel-90-virthost-3cont2comp3ceph-ipv4-geneve-ceph-failed-consistently-w | 15:02 |
| dviroel | ysandeep: ack, agree | 15:02 |
| rlandy | ysandeep: pls go ahead | 15:02 |
| rlandy | ysandeep: dviroel: I will mark the MVP design done as well if you guys sign off on it tomorrow | 15:02 |
| ysandeep | rlandy: i went over it once, I can recheck in my morning tomorrow. | 15:03 |
| rlandy | ysandeep: thanks | 15:03 |
| rlandy | just to see if we are missing something you expect there | 15:03 |
| dviroel | yeah, we can sync tomorrow about that | 15:03 |
| marios | rlandy: (I guess? looks like it anyway) | 15:05 |
| marios | dasm: did you file a bug for the upgrade issue or should I ? | 15:06 |
| dasm | marios: i did not. can you do that? | 15:08 |
| marios | dasm: ok | 15:09 |
| dasm | thanks | 15:09 |
| * ysandeep out, Have a great rest of your day everyone o/ | 15:10 | |
| dasm | ysandeep: o/ | 15:10 |
| ysandeep | dasm, \o | 15:10 |
| *** ysandeep is now known as ysandeep|out | 15:10 | |
| rlandy | marios: I mapped Ready for review to ON_QA | 15:21 |
| rlandy | can you try move your card | 15:21 |
| rlandy | and see if that works | 15:21 |
| rlandy | I could move my card to Ready for Review | 15:22 |
| rlandy | chandankumar: marios: dviroel: ^^ board fixed | 15:23 |
| dviroel | great | 15:24 |
| marios | rlandy: cool | 15:28 |
| rlandy | I mapped QE to MODIFIED | 15:29 |
| rlandy | so all columns should be usable | 15:29 |
| Tengu | hello there! quick status update request: what's up with the "error while loading shared libraries: libc.so.6: cannot change memory protections" ? it's hitting regularly on tripleo-ansible jobs - is there anything to do in the project, for instance a downgrade, reinstall or anything in the job preparation? | 15:38 |
| rlandy | Tengu: we saw it once on container builds | 15:38 |
| rlandy | frenzy_friday|rover: ^^ any spotting that error since yesterday? | 15:39 |
| Tengu | https://zuul.opendev.org/t/openstack/build/24d07245ac4545afa68750dac3297700 hitting on a regular basis | 15:39 |
| Tengu | for molecule container build. | 15:39 |
| frenzy_friday|rover | I saw this once on quay container push job. But just that once | 15:39 |
| Tengu | probably missing a workaround or the like? | 15:39 |
| marios | dasm: https://bugs.launchpad.net/tripleo/+bug/1998501 ... i think i'll add the cix flags so we can get it onto the cix board for attention of the upgrades team | 15:40 |
| rlandy | Tengu: it just kind f went away on rerun | 15:40 |
| Tengu | o_O | 15:41 |
| marios | dasm: actually lets try first without cix and we can add it if we need, going to ping folks in tripleo now lets see if we get any attention | 15:41 |
| rlandy | Tengu: master only? all c9? | 15:41 |
| Tengu | yep and yep | 15:41 |
| Tengu | in that tripleo-ansible project - for molecule | 15:41 |
| rlandy | so all c9 | 15:41 |
| dasm | marios++ | 15:41 |
| Tengu | it builds containers in order to run the tests. | 15:41 |
| Tengu | so I guess something's wrong with ... err... some package, selinux, podman, whatever. | 15:41 |
| rlandy | hmmm ... what node? | 15:42 |
| rlandy | 2022-12-01 15:26:20.387675 | localhost | Distro: CentOS 9 | 15:43 |
| rlandy | 2022-12-01 15:26:20.387790 | localhost | Provider: rax-iad | 15:43 |
| rlandy | 2022-12-01 15:26:20.387905 | localhost | Label: centos-9-stream | 15:43 |
| rlandy | seems normal | 15:43 |
| rlandy | https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ansible-centos-stream-role-addition&skip=0 | 15:43 |
| rlandy | so 11/30 killed you | 15:43 |
| rlandy | Tengu: do you have a diff of the last passing job and first failing? | 15:44 |
| rlandy | installed pip and rpms? | 15:44 |
| Tengu | unfortunately no.. | 15:44 |
| rlandy | well let's get that | 15:44 |
| Tengu | I was trying to work on that ansible-galaxy proxy thingy, and... | 15:45 |
| Tengu | wel. | 15:45 |
| rlandy | last master success https://zuul.opendev.org/t/openstack/build/ba3ce20baaad4618bcd6be7219208cbd | 15:45 |
| Tengu | hitting that on a fairly regular basis. | 15:45 |
| rlandy | first master failure ... | 15:45 |
| Tengu | note that some of the failures are due to my tests at getting the proxy in. but that last one? nope | 15:46 |
| Tengu | https://zuul.opendev.org/t/openstack/build/f6b184edf2d8477e817ef634744c2321 first one apparently. | 15:47 |
| Tengu | then 90% of the following failures are due to that libc thing. | 15:48 |
| Tengu | checking on https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ansible-centos-stream-role-addition&branch=master&skip=0 | 15:48 |
| rlandy | ansible-compat-2.2.6 | 15:48 |
| * dviroel lunch | 15:48 | |
| Tengu | rlandy: actually, there's no log about the install packages... ? | 15:51 |
| rlandy | Tengu: no - doffing from log output | 15:52 |
| frenzy_friday|rover | rlandy, checked zed and wallaby - nova.exception.GlanceConnectionFailed is master only | 15:54 |
| frenzy_friday|rover | I have updated cix | 15:54 |
| rlandy | you don't even get to print out installed packages | 15:57 |
| Tengu | rlandy: wait, are you saying the issue is due to the ansible-compat thing? | 16:01 |
| Tengu | (nope, same version on working vs non-working) | 16:01 |
| * Tengu doesn't follow | 16:01 | |
| rlandy | it was the first diff picked up | 16:02 |
| Tengu | oh. .6 vs .0 | 16:02 |
| Tengu | pfrt. | 16:02 |
| rlandy | doesn't mean it's the issue | 16:03 |
| Tengu | need better brain cells | 16:03 |
| rlandy | http://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/centos-stream/9-stream/AppStream/x86_64/os/ also got an update pn 11/29 | 16:04 |
| rlandy | could be anything | 16:04 |
| Tengu | so that one was successful: https://zuul.opendev.org/t/openstack/build/59e94966f3444b8685392f5f38d49cee while it's in the same job run as the previous I pointed. | 16:04 |
| Tengu | cf https://zuul.opendev.org/t/openstack/status#866031 | 16:05 |
| * Tengu scratching his head | 16:05 | |
| rlandy | https://github.com/containers/podman/issues/3234 | 16:08 |
| rlandy | similar: https://bugzilla.redhat.com/show_bug.cgi?id=1868590 | 16:08 |
| Tengu | oh. | 16:09 |
| Tengu | ok. | 16:09 |
| Tengu | so this loops back to container-selinux not being properly installed. | 16:09 |
| Tengu | https://github.com/containers/container-selinux/issues/198#issuecomment-1333877546 | 16:09 |
| rlandy | we looked into the two errors together a bit | 16:10 |
| Tengu | let's see. | 16:13 |
| Tengu | https://review.opendev.org/c/openstack/tripleo-ansible/+/866347 stupid thing, we'll see soon enough. | 16:13 |
| Tengu | f... forgot -y | 16:15 |
| *** amoralej is now known as amoralej|off | 16:27 | |
| Tengu | uho - it fails even before hitting the "pre" things ?! or... | 16:35 |
| rlandy | lunch - brb | 16:37 |
| Tengu | EOD here.... that libc thing is pretty annoying -.-. | 16:44 |
| Tengu | but I can't think anymore. | 16:44 |
| *** marios is now known as marios|out | 16:49 | |
| *** jpena is now known as jpena|off | 17:27 | |
| rlandy | dasm: hey - are the ZED upgrades jobs ready for review or still in progress | 18:38 |
| rlandy | reading review list | 18:38 |
| dasm | rlandy: there is a bug report by marios | 18:41 |
| dasm | rlandy: https://bugs.launchpad.net/tripleo/+bug/1998501 | 18:42 |
| rlandy | dasm: so leave the review for the moment? | 18:42 |
| dasm | rlandy: technically review should be good to go. but we need to figure out update/ugrade process | 18:43 |
| rlandy | k - will check in with you at review time | 18:45 |
| rlandy | frenzy_friday|rover: periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-internal-zed success | 19:38 |
| * dviroel out | 21:02 | |
| *** dviroel is now known as dviroel|out | 21:02 | |
| * dasm => offline | 22:07 | |
| dasm | o/ | 22:07 |
| *** dasm is now known as dasm|off | 22:07 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!