Wednesday, 2024-02-07

rdogerritrdo-trunk rdo-trunk proposed rdoinfo master: Promote CBS tags update for caracal-9s-testing  https://review.rdoproject.org/r/c/rdoinfo/+/5156200:24
rdogerritrdo-trunk rdo-trunk proposed rdoinfo master: Promote CBS tags update for bobcat-9s-testing  https://review.rdoproject.org/r/c/rdoinfo/+/5153700:25
apevectkajinam: yeah, we too hit regressions with latest published CS9 libvirt, maintainers said "The fixed version is libvirt-10.0.0-1.el9" 00:29
apevecit is built in CS Koji but still in the gate https://kojihub.stream.centos.org/koji/buildinfo?buildID=4709100:31
apevecyou can grab RPMs from Koji00:31
rdogerritrdo-trunk rdo-trunk proposed rdoinfo master: Promote CBS tags update for antelope-9s-testing  https://review.rdoproject.org/r/c/rdoinfo/+/5142700:38
rdogerritrdo-trunk rdo-trunk proposed rdoinfo master: Promote CBS tags update for wallaby-8s-testing  https://review.rdoproject.org/r/c/rdoinfo/+/5128200:39
rdogerritrdo-trunk rdo-trunk proposed deps/python-httpx c9s-caracal-rdo: Update to 0.26.0  https://review.rdoproject.org/r/c/deps/python-httpx/+/5156301:04
apevecspotz: FYI I tried to export blogs.r.o using WP plugins for exporting to the static site (OT: as a prep, I had to upgrade WP - it was very out of date, not sure if OSPO is supposed to monitor that, main admin was jbrooks?)08:21
apevecboth Staatic and Simply Static failed with 500 while crawling, looks like they where too aggressive... I'll try external crawler which can more gentle08:23
apevecIn any case, I'd propose to not add any new content to RDO WP instance, while I figure out the best way to export content and move it to a new site08:24
jcapitao[m]amoralejkarolinku : can you review https://review.rdoproject.org/r/c/rdoinfo/+/51549 please ? 08:45
jcapitao[m]it's blocking a FTBFS dated from Feb 308:45
jcapitao[m]this one https://review.rdoproject.org/r/c/openstack/magnum-tempest-plugin-distgit/+/5154008:45
rdogerritMerged rdoinfo master: Add python-oauthlib as dependency  https://review.rdoproject.org/r/c/rdoinfo/+/5154909:16
jcapitao[m]amoralej karolinku : so new libvirt is breaking all of our CS9 releases 09:31
jcapitao[m]dunno if you noticed it09:32
jcapitao[m]messages from apevec  earlier : yeah, we too hit regressions with latest published CS9 libvirt, maintainers said "The fixed version is libvirt-10.0.0-1.el9"09:33
jcapitao[m]and - it is built in CS Koji but still in the gate https://kojihub.stream.centos.org/koji/buildinfo?buildID=4709109:33
amoralejdamn09:33
amoralejwhere are logs?09:34
jcapitao[m]https://logserver.rdoproject.org/37/51537/2/check/validate-buildsys-tags-bobcat-testing-puppet-scenario001-centos9/4ec77d1/logs/weirdo-project/logs/nova/nova-compute.txt.gz09:34
jcapitao[m]for instance09:34
jcapitao[m]from https://review.rdoproject.org/r/c/rdoinfo/+/51537?tab=change-view-tab-header-zuul-results-summary09:34
amoralej"2024-02-06 20:32:04.596 99517 ERROR nova.compute.manager UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 0: invalid start byte" ?09:35
amoralejthat's the issue?09:35
jcapitao[m]yes09:35
jcapitao[m]we have same issue for puppet-promotion pipelines09:35
apevecanother CS horror story in the making 🙄09:38
amoralejintegration sig to the rescue :)09:38
apevecdunno why it is taking so long in the gate, I guess there are other blockers, I'll ping in RHEL Jira ticket09:39
amoralejwe may add an exclude to the repo config in worst case09:39
amoralejbut it'd be better to avoid it if possible09:39
apevecamoralej @_oftc_amoralej:matrix.org:  IntSIG would not override RHEL gate09:40
apevecthat's where the blocker is, afaict09:40
apevecah yes, let's try excluding 9.1009:41
amoralejyes, but if there is a way to label and expose only the composes that have passed openstack jobs ...09:41
amoralejwe may consume that09:41
jcapitao[m]but only for promotion pipelines09:42
amoralejwrt exclusion, we may do something like https://review.rdoproject.org/r/c/rdo-infra/weirdo/+/3390909:45
amoraleji'm preparing the patch09:48
amoralejapevec, jcapitao[m] is there any bz or report about the libvirt issue ?09:54
apevecyes, we have CIX with a link to RHEL Jira, sec09:54
apevechttps://issues.redhat.com/browse/RHEL-23925 was filed by OSP then closed as dup of https://issues.redhat.com/browse/RHEL-22314 by the RHEL team09:55
apeveclemme check if those are public09:55
apevecok the latter is public09:56
jcapitao[m]thx09:56
apevecbut title looks unrelated, dunno why RHEL closed previous as a dup, just b/c "it is fixed in same 10.0" ...09:56
apevecyeah description is for a completely different issue :(09:57
amoralejyep09:58
amoralejnot nice09:58
jcapitao[m]maybe a misclick09:58
apevecah yes, actual dup is https://issues.redhat.com/browse/RHEL-2060909:59
apevecso it was caught by RHEL QE early Jan ...10:00
amoralejgood10:01
apevecjcapitao[m]: I actually mis-copied, Jira filed by OSP is marked as dup of 20609 :)10:01
rdogerritAlfredo Moralejo proposed rdo-infra/weirdo master: Exclude libvirt-9.10.0-1 from appstream repo  https://review.rdoproject.org/r/c/rdo-infra/weirdo/+/5156410:03
amoralejapevec, jcapitao[m] ^ 10:04
amoralejtkajinam proposed similar approach https://review.opendev.org/c/openstack/puppet-openstack-integration/+/90821710:04
jcapitao[m]+2 10:13
jcapitao[m]actually, we should also report it as a bug in our RDO JIRA10:13
amoralejyep10:14
jcapitao[m]I'm creating it10:15
rdogerritMarios Andreou proposed rdo-jobs master: Remove redundant var skip_standalone_vm in DPA CI playbooks and job  https://review.rdoproject.org/r/c/rdo-jobs/+/5154710:20
tkajinamamoralej, jcapitao I wonder if you know any timeline about the 10.0.0 release in centos ?10:26
amoraleji don't10:26
tkajinamit also affects a few other projects having c9s jobs. if it takes some time then I'll propose marking these jobs non-voting10:26
tkajinamthat's a serious regression they should fix asap imo, though10:26
amoralejyes, i agree10:27
tkajinamit happened really annoying time. it blocks all the work needed to wipe out puppet-qdr from yoga before transitioning yoga to unmaintained :-(10:29
amoraleji've asked in https://issues.redhat.com/browse/RHEL-2060910:29
amoralejlet's see if we can get any input10:30
tkajinamjust my own complaint10:30
tkajinamamoralej, thanks !10:30
amoralejlast build for cs9 is from 29-Jan but they must be having some issue because it's not in the compose under testing10:32
amoralejhttps://composes.stream.centos.org/production/latest-CentOS-Stream/compose/AppStream/x86_64/os/Packages/10:33
amoralejapevec, ^ fyi current compose in production is shipping 9.10.010:33
apevecyes I know...10:38
jcapitao[m]https://issues.redhat.com/browse/RDO-18310:39
apevecthanks, I piled it on under "blocks" in RHEL-20609 10:41
apevecand that one is blocked by libvirt rebase https://issues.redhat.com/browse/RHEL-15267 - which probably is blocked on some other issues, sigh...10:42
amoralejyep, it's unlucky that the fix comes with a rebase10:43
apevecwhat if we grab libvirt-10.0.0-2.el9  from CS Koji and put it in a side-repo?10:43
apevectemp in RDO deps maybe?10:43
amoralejwe could, at least to test it10:43
apevecyes, that way we could at least check there isn't another regression lurking in a rebase10:44
jcapitao[m]agree with that proposal10:47
rdogerritMerged rdo-infra/weirdo master: Exclude libvirt-9.10.0-1 from appstream repo  https://review.rdoproject.org/r/c/rdo-infra/weirdo/+/5156410:59
jcapitao[m]amoralej apevec  https://review.opendev.org/c/openstack/puppet-openstack-integration/+/908291 let's see how it goes11:29
amoralejnice11:30
apevecah we already had delorean-deps-test.repo or you added it now?11:31
jcapitao[m]we used it to test Ceph Reef with EPEL enabled11:31
apevecah cool, I see https://trunk.rdoproject.org/centos9-master/deps/cs9-fix/ is the temp repo, let's keep that around ;)11:32
apevecI've put click-bait to Integration SIG matrix11:42
amoralej:)11:45
karolinku[m]that's remind me of joining:)11:45
apevecdouble-click bait! I should move to marketing ;)11:50
apevecBTW bookmark this https://docs.fedoraproject.org/en-US/fedora-accounts/user/#pkinit - after adding 2FA to the FAS account (highly recommended, even though might be annoying)11:59
amoralejjcapitao[m], karolinku[m] see my comment in https://review.rdoproject.org/r/c/rdoinfo/+/51549 please12:03
karolinku[m]ack12:04
jcapitao[m]right12:08
jcapitao[m]I'm going to lower the requirement on kubernetes itself12:09
rdogerritJoel Capitao proposed rdoinfo master: Revert "Add python-oauthlib as dependency"  https://review.rdoproject.org/r/c/rdoinfo/+/5157412:10
jcapitao[m]but I remember having checked that 12:12
jcapitao[m]weird12:12
rdogerritMarios Andreou proposed rdo-jobs master: Remove redundant var skip_standalone_vm in DPA CI playbooks and job  https://review.rdoproject.org/r/c/rdo-jobs/+/5154712:18
jcapitao[m]ok I know what happened, in the spec file we have https://review.rdoproject.org/cgit/deps/python-kubernetes/tree/SPECS/python-kubernetes.spec?id=fd6569e6b4cd6127cbb11a6a78f7ca373082fb0d#n47 12:55
jcapitao[m]Requires:  %{py3}-oauthlib12:55
jcapitao[m]our CI failed because nothing was providing python3.9dist(oauthlib) 12:56
jcapitao[m]so I assumed it was not provided at all12:57
jcapitao[m]but actually the package is using https://review.rdoproject.org/cgit/deps/python-kubernetes/tree/SPECS/python-kubernetes.spec?id=fd6569e6b4cd6127cbb11a6a78f7ca373082fb0d#n112:57
jcapitao[m]which adds a strong requirement based on requirements.txt12:58
rdogerritJoel Capitao proposed deps/python-kubernetes c9s-caracal-rdo: Uncap oauthlib version  https://review.rdoproject.org/r/c/deps/python-kubernetes/+/5156513:00
jcapitao[m]amoralej karolinku  should be better with that ^13:00
jcapitao[m]and reverting the rdoinfo patch https://review.rdoproject.org/r/c/rdoinfo/+/5157413:02
amoralejjcapitao[m], lgtm let's see if that passes ci13:04
jcapitao[m]amoralej apevec  CI jobs are successful with libvirt-10.0.0-2.el9 https://review.opendev.org/c/openstack/puppet-openstack-integration/+/908291?tab=change-view-tab-header-zuul-results-summary13:04
apevecgood news!13:05
jcapitao[m]I updated the JIRA ticket https://issues.redhat.com/browse/RDO-183 with that info13:05
amoralejgreat13:06
rdogerritArx Cruz proposed rdo-jobs master: Removing poi scenario000  https://review.rdoproject.org/r/c/rdo-jobs/+/5156613:07
rdogerritArx Cruz proposed rdo-jobs master: Removing poi scenario000  https://review.rdoproject.org/r/c/rdo-jobs/+/5156613:08
rdogerritArx Cruz proposed rdo-jobs master: Delete tripleo-next-os project template  https://review.rdoproject.org/r/c/rdo-jobs/+/5156613:14
apevecjcapitao: cs9-fix is in master only, can we add it to antelope too, that's what o-k-o is using13:14
jcapitao[m]lemme do the same for antelope13:15
rdogerritMerged rdo-jobs master: Delete tripleo-next-os project template  https://review.rdoproject.org/r/c/rdo-jobs/+/5156613:26
jcapitao[m]https://review.opendev.org/c/openstack/puppet-openstack-integration/+/90830913:32
jcapitao[m]^ for antelope13:32
rdogerritJoel Capitao proposed rdoinfo master: Promote CBS tags update for caracal-9s-testing  https://review.rdoproject.org/r/c/rdoinfo/+/5156213:37
rdogerritMerged deps/python-kubernetes c9s-caracal-rdo: Uncap oauthlib version  https://review.rdoproject.org/r/c/deps/python-kubernetes/+/5156513:38
rdogerritRonelle Landy proposed rdo-jobs master: Remove redundant var skip_standalone_vm in DPA CI playbooks and job  https://review.rdoproject.org/r/c/rdo-jobs/+/5154713:41
rdogerritMarios Andreou proposed rdo-jobs master: Remove redundant var skip_standalone_vm in DPA CI playbooks and job  https://review.rdoproject.org/r/c/rdo-jobs/+/5154713:41
amoralejtoday is video meeting, right?14:01
karolinku[m]looks like so!14:01
amoralej#startmeeting RDO meeting - 2024-02-0714:02
opendevmeetMeeting started Wed Feb  7 14:02:59 2024 UTC and is due to finish in 60 minutes.  The chair is amoralej. Information about MeetBot at http://wiki.debian.org/MeetBot.14:02
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:02
opendevmeetThe meeting name has been set to 'rdo_meeting___2024_02_07'14:02
amoralejmeeting is in #startmeeting RDO meeting - 2024-MM-DD14:03
amoralejmeeting is in https://meet.google.com/uzo-tfkt-top14:03
amoralejjcapitao[m] apevec ^14:03
apevecstill at another call, will jump in a min14:04
amoralej#topic Moving RDO from RHBZ to Jira RDO project in issues.redhat.com 14:21
amoralej#info RDO will be moving to jira to report issues and requests14:21
amoralej#info We need to check which documentation we need to change14:21
amoralej#action spotz[m] will check the documentation for report to jira14:22
amoralej#action karolinku[m] jcapitao[m] amoralej to do a rhbz cleanup session on tuesday 13th at 09:00 UTC14:35
amoralej#topic CS9 libvirt 9.10 regression https://issues.redhat.com/browse/RDO-18314:36
amoralej#info latest libvirt build 9.10.0 has broken all the OpenStack deployment jobs on CentOS Stream 914:36
amoralej#info workarounds are being applied by excluding the known bad build14:37
amoralej#topic collaboration with CentOS integration SIG14:42
amoralej#info during the CentOS integration SIG meetup at CentOS connect RDO team shared our expectations and use cases14:43
amoralej#action we will write our use case and proposal for deliverables from integration SIG to make sure there is a common understanding of it14:44
amoralej#topic documentation changes14:49
amoralej#info blogs content will be moved to a new location to be defined14:50
amoralej#info No more content should be added to blogs.rdoproject.org14:51
amoralej#topic chair next week14:51
amoralej#action jcapitao[m] will chair next week meeting14:52
amoralej#endmeeting14:53
opendevmeetMeeting ended Wed Feb  7 14:53:30 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)14:53
opendevmeetMinutes:        https://meetings.opendev.org/meetings/rdo_meeting___2024_02_07/2024/rdo_meeting___2024_02_07.2024-02-07-14.02.html14:53
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/rdo_meeting___2024_02_07/2024/rdo_meeting___2024_02_07.2024-02-07-14.02.txt14:53
opendevmeetLog:            https://meetings.opendev.org/meetings/rdo_meeting___2024_02_07/2024/rdo_meeting___2024_02_07.2024-02-07-14.02.log.html14:53
*** d34dh0r5- is now known as d34dh0r5314:57
jcapitao[m]apevec: CI jobs are ok with libvirt-10.0.0-2.el9 for Antelope as well https://review.opendev.org/c/openstack/puppet-openstack-integration/+/90830915:02
rdogerritMerged rdoinfo master: Revert "Add python-oauthlib as dependency"  https://review.rdoproject.org/r/c/rdoinfo/+/5157416:08
rdogerritMerged rdoinfo master: Promote CBS tags update for caracal-9s-testing  https://review.rdoproject.org/r/c/rdoinfo/+/5156216:10
jcapitao[m]amoralej: FYI I explained the DLRN issue we hit with neutron FTBFS on Bobcat16:16
jcapitao[m]https://issues.redhat.com/browse/PCTOOLING-121216:16
jcapitao[m]filed as a bug16:16
amoralejcool16:16
jcapitao[m]but dunno if we should fix it though16:16
amoraleji doubt there is a way that would not break in other cases16:18
amoralejit'd be good to check git history to see why that --after was included16:18
jcapitao[m]you mean, if we rely on git tree order it may break ?16:23
amoralejwe may, there may be some reason why we do it as we do it16:26
amoraleji remember hitting issues with ordering and timestamps in the past16:26
jcapitao[m]amoralej: I did the archeology work :)16:29
jcapitao[m]the first "--after" that appeard is from initial commit !16:29
jcapitao[m]https://github.com/softwarefactory-project/DLRN/blame/450aad09f7375b453c8d37be68958e46fd07ca2c/delorean/shell.py#L2116:29
amoralejgood16:29
jcapitao[m]it was then move to https://github.com/softwarefactory-project/DLRN/commit/4a47b45b6049ed57ba2afa6fb6fa701cc2c731c7#diff-4159879fab8dcf9947f58c889240eaf3d2d3e7d6d165fb1a4740ffc8e49058f2R54 few commits after16:30
amoralejthen it may be good to check it, but we need to think well about that16:30
jcapitao[m]so we don't have the explicit reason ...16:30
amoralejjcapitao[m], did you get any info about why the neutron-lib repo has those weird timestamps ?16:34
jcapitao[m]yes, I discussed with neutron guys, TL;DR: Github UI shows the commits based on commit date 16:36
jcapitao[m]https://github.com/openstack/neutron-lib/commits/3.8.1/16:37
jcapitao[m]whereas gitea relies on actual git tree https://opendev.org/openstack/neutron-lib/commits/tag/3.8.116:38
jcapitao[m]amoralej: ^16:38
jcapitao[m]and the actual patches was created on Aug 18 2023, not edited since then16:39
jcapitao[m]Gerrit keep the commit date as-is when merging16:39
jcapitao[m]so it's nominal case, I was mislead by Github UI yesterday16:41
jcapitao[m]IMHO DLRN should relies on git tree order which is the source of truth16:42
jcapitao[m]s/relies/rely/16:42
amoralejjcapitao[m], i don't understand anything at this point16:59
amoralejso, the problem was that the author of the review had the commit locally for months before sending it :)17:03
amoralejand yep, gerrit keeps it unchanged17:03
amoralejactually, imo git or gerrit or something should check it17:03
amoralejbut, i.e. https://review.opendev.org/c/openstack/neutron-lib/+/903334 was created on Dec 1117:04
jcapitao[m]exactly it was written in August but merged in December 17:04
amoralejnot only merged17:05
amoralejsent17:05
amoralejPS1 is from Dec 1117:05
jcapitao[m]yes but commit date was kept https://review.opendev.org/c/openstack/neutron-lib/+/903334/2//COMMIT_MSG#517:06
amoralejyes17:06
amoralejactually, i guess commit date should be newer that author date17:07
amoralejhttps://review.opendev.org/c/openstack/neutron-lib/+/903334/2//COMMIT_MSG17:07
amoralejbut it's the oposite17:07
jcapitao[m]yeah agree, I need to check Gerrit doc on that 17:09
amoraleji suspect this is more an issue in the repo side that in the dlrn side17:10
amoralejeven if 2224 was sent before than the other two, when it was merged it had to be rebased and commitdate should be updated when rebased17:12
amoralejhttps://docs.github.com/en/account-and-profile/setting-up-and-managing-your-github-profile/managing-contribution-settings-on-your-profile/troubleshooting-commits-on-your-timeline17:12
jcapitao[m]I see your point, a "normal" rebase operation would have update commit date 17:13
amoralejyes, that's my understanding17:14
amoralejthe infra team may help17:14
amoralejhe may have a better understanding17:14
jcapitao[m]I'll ask infra team tomorrow 17:16
rdogerritrdo-trunk rdo-trunk proposed openstack/horizon-distgit rpm-master: python-django-horizon: failed to build 8a3006756  https://review.rdoproject.org/r/c/openstack/horizon-distgit/+/5156717:32
apevecwhat is the question for the infra team?22:53
apevecBTW I'm questioning if keeping those repos under https://repos.fedorapeople.org/openstack/EOL/ makes sense, they actually won't work, would they?22:56
apevecfor those where SIG repos got moved to vault.c.o22:56
apevecalso few are yum repos using up space with probably non-functional RPMs:22:57
apevec6.5MEOL/openstack-diablo... (full message at <https://matrix.org/_matrix/media/v3/download/matrix.org/lsuttufcuLfGwRUKrxzbEMgV>)22:57
apevecthose are from the old times, pre-CBS, when we were copying from Fedora Koji EPEL builds...22:58
apevecs/repos/rdo-release*.rpm/23:02
apevecaccidentally, in http://mirror.centos.org/centos/8-stream/cloud/x86_64/openstack-train/Packages/o/ is only ovs, ovn - looks like we have this mis-tagged?23:12

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!