Thursday, 2021-12-02

*** rlandy|ruck|biab is now known as rlandy|ruck00:06
*** rlandy|ruck is now known as rlandy|out00:30
*** rcastillo is now known as rcastillo|out01:23
*** ysandeep|out is now known as ysandeep|ruck04:33
*** pojadhav- is now known as pojadhav05:22
*** ysandeep|ruck is now known as ysandeep|afk05:52
*** ysandeep|afk is now known as ysandeep|ruck06:15
raukadah\o06:51
*** raukadah is now known as chandankumar06:51
ysandeep|ruckchandankumar, o/ welcome back!06:55
chandankumarysandeep|ruck: thanks :-)06:55
marioso/ chandankumar :)07:05
*** ykarel__ is now known as ykarel07:08
chandankumarmarios: \o, Thank you for getting the cs9 mirror patch merged and taking over cs9 work. :-)07:11
marioschandankumar: o/ i didn't get the cs9 mirror merged it happened before you left i think :) ... mainly last 2 weeks was trying to pay some tech debt we had for repos in containers/images (https://issues.redhat.com/browse/TRIPLEOCI-717)07:13
*** ysandeep|ruck is now known as ysandeep|lunch07:23
marioschandankumar: o/ sorry got d/c some contractors flipped the electricity switch and internet went for a few misn07:32
marioschandankumar: i may have missed if you typed something earlier07:33
marios09:13 < marios> chandankumar: o/ i didn't get the cs9 mirror merged it happened before you left i think :) ... mainly last 2 weeks was trying to pay some tech debt we had for repos in containers/images (https://issues.redhat.com/browse/TRIPLEOCI-717)07:33
chandankumarmarios: nope, nothing got missed, I am going through the tripleoci-717, :-)07:33
marioschandankumar: just 2 patches now https://review.opendev.org/c/openstack/tripleo-ci/+/818222 https://review.opendev.org/c/openstack/tripleo-quickstart/+/817161 07:35
*** amoralej|off is now known as amoralej07:54
*** ysandeep|lunch is now known as ysandeep08:28
*** ysandeep is now known as ysandeep|ruck08:28
chandankumarysandeep|ruck: \o, I am seeing couple of issues on cs9 pipeline, wanted to cross check for bugs, (I checked already on prod chain cix,no bugs there)08:41
chandankumar1. https://logserver.rdoproject.org/openstack-periodic-integration-main-cs9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-9-buildimage-ironic-python-agent-master/f83180b/build.log - Error: Unable to find a match: genisoimage08:41
chandankumar2. https://logserver.rdoproject.org/openstack-periodic-integration-main-cs9/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-undercloud-containers-master/0252076/job-output.txt -  Cannot install ansible-core with a pre-existing ansible-base==2.10.1508:42
chandankumarmarios: will go over those reviews, Thanks :-)08:42
marioschandankumar: 1 is known issue but don't think we have a launcpad for it08:42
marioschandankumar: we have bz sec08:43
marioshttps://bugzilla.redhat.com/show_bug.cgi?id=1999713 chandankumar 08:43
ysandeep|ruckchandankumar, genisoimage still mentioned in dib: https://github.com/openstack/ironic-python-agent-builder/blob/master/dib/ironic-python-agent-ramdisk/package-installs.yaml#L15 08:43
ysandeep|ruckchandankumar, regarding 2 sshnaidm have some patches up that will fix the issue, let me check status of them08:44
chandankumarmarios: thanks for passing, for ustream genisoimage issue tracking, will I open a launchpad bug and proppse the fix?08:45
marioschandankumar: sure i didn't do anythign there08:45
chandankumarysandeep|ruck: thank you for the links . 08:46
ysandeep|ruckchandankumar, looks like fix was merged last night: https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/818354 , could you please rerun the job for the 2nd issue08:48
ysandeep|ruckchandankumar, sshnaidm highlighted the geniso issue on Monday, fyi.. some discussion , you may find them useful - https://meetings.opendev.org/irclogs/%23oooq/%23oooq.2021-11-29.log.html#t2021-11-29T12:39:58 08:51
ysandeep|ruckchandankumar, fyi.. we already had one previous cix: but that was closed https://trello.com/c/w6cdlyFc/2142-cixlp1946822tripleociproa-standalone-deployment-on-centos9-failing-during-python3-tripleoclient-dependencies-install-nothing-pro 08:53
chandankumarysandeep|ruck: thank you for the cix link, I will open a bug and link it back to previous failures also and then propose the fix.09:03
chandankumars/failues/CIX09:03
ysandeep|ruckack09:04
mariostripleo-ci please add to your reviews https://review.opendev.org/c/openstack/tripleo-ci/+/818222 https://review.opendev.org/c/openstack/tripleo-quickstart/+/817161 09:23
ysandeep|ruckchandankumar, are these duplicate https://bugs.launchpad.net/tripleo/+bug/1953013 https://bugs.launchpad.net/tripleo/+bug/1953012 https://bugs.launchpad.net/tripleo/+bug/1953011 ?09:32
ysandeep|ruckmarios, ack09:33
chandankumarysandeep|ruck: yes, got created trice, marking them as a duplicate09:34
chandankumarsorry for that09:34
chandankumarthanks for pointing it out09:34
ysandeep|rucknw :) 09:35
*** jbadiapa|off is now known as jbadiapa09:46
mariosanyone know why my jobs aren't triggering here https://code.engineering.redhat.com/gerrit/c/testproject/+/295749/ what did i miss :) ... cant see them at https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/status 10:09
mariosysandeep|ruck: ? ^ 10:09
ysandeep|ruckmarios, checking10:10
mariosysandeep|ruck: thanks ... i added the .gitreview (why dont we just merge that btw we should)10:10
mariosysandeep|ruck: if you cant see something obvious dont worry i will keep digging10:11
mariosysandeep|ruck: just wondering if i missed something for d/stream zuul10:12
marioscoffee refill 10:16
ysandeep|ruckmarios, can you name that .zuul.yaml instead of zuul.yaml10:16
mariosysandeep|ruck: sure can try (but have found earlier examples where it was just zuul.yaml e.g. https://code.engineering.redhat.com/gerrit/c/testproject/+/277478/)10:22
ysandeep|rucksomething else then :)10:23
mariosysandeep|ruck: same.. it appears but then disappears at zuul status page it didn't like something 10:23
mariosysandeep|ruck: thanks for checking 10:23
marios12:16 < ysandeep|ruck> marios, can you name that .zuul.yaml instead of zuul.yaml10:25
mariossorry accidental paste 10:25
ysandeep|ruckcrosschecked all those job are defined, may be worth pinging downstream infra.. they can check zuul server side logs.. zuul should report back to patch - what it didn't like.10:28
mariosysandeep|ruck: thanks for sanity check on the job names ;) ack will do if i cant find something soon 10:28
chandankumarmarios: ysandeep|ruck seen this error "msg": "stdout: AttributeError: 'NoneType' object has no attribute 'version_tuple'\n" ?10:29
chandankumarhttps://logserver.rdoproject.org/97/36997/3/check/periodic-tripleo-ci-build-containers-centos-9-push-master/4b0d576/job-output.txt10:30
chandankumarseeing just recent in my testproject10:30
marioschandankumar: not seen that 10:30
ysandeep|ruckchandankumar, not seen that.. fyi.. We are not monitoring c9 lines till we move image build/container build to use release content...10:31
marioschandankumar: yeah so we have put everything else on hold until we fix to use the proper repos... maybe try with 818222 in depends-on? 10:35
marioschandankumar: e.g. for collectd i had to enable opstools repo with  https://review.opendev.org/c/openstack/tripleo-ci/+/818222/47/roles/tripleo-build-jobs-repos/tasks/main.yaml#16 10:35
chandankumarmarios: ysandeep|ruck ok, sure10:40
sshnaidmmarios, hey, I wonder if we should install tripleo-repos into virtualenv of quickstart, like we do now with other collections: https://github.com/openstack/tripleo-quickstart/blob/master/install-deps.sh#L29610:45
sshnaidmmarios, the default path seems to be not in ansible.cfg: https://review.opendev.org/c/openstack/tripleo-quickstart/+/82014310:46
mariossshnaidm: sure .. you mean move from //github.com/openstack/tripleo-quickstart/blob/master/install-deps.sh#L296 to where the others are installed? 10:54
sshnaidmmarios, in second thought, it will require virtualenv to be set up firstly and it's done only in quickstart.sh.. Ok, probably my patch should be enough10:57
*** dviroel|out is now known as dviroel11:01
*** rlandy|out is now known as rlandy|ruck11:12
rlandy|ruckmarios: I triggered a downstream check11:18
dviroelchandankumar: welcome back o/11:18
rlandy|ruckysandeep|ruck: arxcruz|rover: hey - let11:19
rlandy|rucks sync11:20
ysandeep|ruckarxcruz|rover, ack o/11:20
rlandy|ruckhttps://meet.google.com/vzm-nrah-qqf?authuser=011:20
rlandy|ruckysandeep|ruck: arxcruz|rover: ^^11:21
ysandeep|ruckrlandy|ruck, https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/29576911:22
ysandeep|ruckrlandy|ruck, https://review.rdoproject.org/r/c/rdoinfo/+/3697711:22
rlandy|rucksshnaidm: hi - can we recheck the jenkins tests now? ansible path issue11:29
sshnaidmrlandy|ruck, I'm investigating it in https://review.opendev.org/c/openstack/tripleo-quickstart/+/82014611:30
sshnaidmneed more debug msgs11:31
mariosdviroel: o/ replied at https://review.opendev.org/c/opendev/base-jobs/+/820018/3#message-3f2489964b12345c86fea94e05bad37453a5e931 i think we have to stick to ID /VERSION_ID11:52
mariosdviroel: and replied again ;)11:57
dviroelmarios: 11:59
dviroelmarios: yeah, me too11:59
dviroelmarios: now there is a issue, OS_VERSION_ID does no exists12:00
mariosdviroel: i also updated it :)12:00
mariosdviroel: that's what you get for editing in the browser... fixign12:00
dviroellolz12:01
mariosdviroel: k done12:01
*** pojadhav is now known as pojadhav|afk12:01
dviroel++12:01
mariosdviroel: i guess if nothing else the conditional reads nicer now for the humans12:01
mariosdviroel: functionally its the same 12:01
dviroelmarios: yes12:02
dviroelmarios: since VERSION_ID={{ ansible_distribution_major_version }}12:02
dviroelmarios: you will not have issues like 9.x12:02
mariosright so why do they need to do the cut -d thing... anyway 12:02
dviroelmarios: may legacy code, don't know12:03
dviroelmaybe*12:03
ysandeep|ruckrlandy|ruck, fyi.. retriggered bm line here: https://review.rdoproject.org/r/c/testproject/+/32054 12:24
rlandy|ruckysandeep|ruck; great - thanks12:34
chandankumardviroel: thanks :-)12:39
ysandeep|ruckrlandy|ruck, arxcruz|rover have you noticed any similiar issue:-12:43
ysandeep|ruck~~~12:43
ysandeep|ruck2021-12-02 12:27:37.880072 | primary | TASK [repo-setup : Check DLRN hash format] *************************************12:43
ysandeep|ruck2021-12-02 12:27:37.880292 | primary | Thursday 02 December 2021  12:27:37 +0000 (0:00:00.155)       0:00:16.160 *****12:43
ysandeep|ruck2021-12-02 12:27:37.921679 | primary | fatal: [undercloud]: FAILED! => {"changed": false, "msg": "The DLRN hash or tag is not recognized. The hash or tag should not contain path slashes."}12:43
ysandeep|ruck~~~12:43
ysandeep|ruckhttps://logserver.rdoproject.org/46/28446/57/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-ussuri/8625a47/job-output.txt12:43
rlandy|ruckysandeep|ruck: no - 12:44
ysandeep|ruckack, will investigate12:44
sshnaidmrlandy|ruck, does job tripleo-quickstart-gate-master-tripleo-ci-delorean-full-featureset052 run on centos 7?12:51
sshnaidmand "minimal" too?12:52
rlandy|rucksshnaidm: minimal gets kicked from promotions 12:52
rlandy|ruckso there is a c7 promotion12:53
rlandy|ruckfor fs05212:53
rlandy|ruckthat gets kicked from tq/tqe changes12:53
rlandy|ruckso that should be master12:53
sshnaidmI mean the node OS12:54
*** pojadhav|afk is now known as pojadhav12:54
sshnaidmWe run quickstart on centos 7 nodes12:54
sshnaidmThat's why it didn't work.. I can add collections install for c7 too, didn't think we use it somewhere12:54
sshnaidmbut can't we run on centos 8 nodes? c7 support is expensive12:55
ysandeep|rucksshnaidm, we still have train on c712:56
ysandeep|rucki mean a parallel line but still we have train on c712:57
sshnaidmysandeep|ruck, yeah, but tripleo-quickstart-gate-master-tripleo-ci-delorean-full-featureset052 is "master"12:59
sshnaidmat least it's named so13:00
bhagyashrischandankumar, rlandy|ruck soniya29 crum meeting time13:01
sshnaidmbhagyashris, what is the scrum mtg link?13:02
ysandeep|rucksshnaidm, https://meet.google.com/xxq-vevp-vkd?authuser=013:02
sshnaidmysandeep|ruck, thanks13:02
sshnaidmI don't have scrum in my calendar..13:02
ysandeep|rucksshnaidm, sry i misunderstood - I thought we have issue with quickstart with C7(I thought our train c7 jobs will also fail now), +1 to move those jobs to C813:08
sshnaidmysandeep|ruck, tbh I would check train c7 jobs as well if they are fine13:09
ysandeep|rucki just checked.. looks okay(last run yesterday): https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable4-centos7 13:09
bhagyashrishttps://review.opendev.org/c/openstack/tripleo-ci/+/81699113:12
dviroelhttps://review.rdoproject.org/r/c/rdo-infra/ci-config/+/3691113:23
dviroelhttps://hackmd.io/G6b1q7wDSImTMTlKgPP75w?view13:25
sshnaidmtripleo-quickstart-gate-master-tripleo-ci-delorean-ruck-rover-test-featureset-minimal is running on CentOS 7 node as well, should be replaced to 813:26
*** amoralej is now known as amoralej|off13:28
*** rcastillo|out is now known as rcastillo13:32
mariosreviews please https://review.opendev.org/c/openstack/tripleo-ci/+/818222 https://review.opendev.org/c/openstack/tripleo-quickstart/+/817161 https://review.opendev.org/c/opendev/base-jobs/+/82001813:37
mariossanity testproject @ https://code.engineering.redhat.com/gerrit/c/testproject/+/29574913:38
pojadhavreviews please : https://review.rdoproject.org/r/c/rdo-jobs/+/3671813:40
pojadhavhttps://review.opendev.org/c/openstack/tripleo-ci/+/81876513:41
pojadhavhttps://review.rdoproject.org/r/c/rdo-jobs/+/3695913:41
rcastillohttps://review.rdoproject.org/r/c/testproject/+/3698813:41
bhagyashrishttps://review.rdoproject.org/r/c/rdo-jobs/+/3697413:42
rcastillohttps://review.rdoproject.org/r/c/rdo-jobs/+/3697413:42
* pojadhav will be out for doctor's appointment.13:48
*** pojadhav is now known as pojadhav|afk13:48
ysandeep|ruckhttps://review.opendev.org/q/topic:%22featureset064%22+(status:open%20OR%20status:merged)13:50
chandankumarrlandy|ruck: marios sshnaidm https://paste.openstack.org/raw/811403/ ubi9 beta, It has issues with local pulling Error: Source image rejected, We can use once it is publically available.14:03
rlandy|ruckchandankumar: ack14:03
mariosthanks chandankumar 14:11
*** ysandeep|ruck is now known as ysandeep|afk14:14
sshnaidmchandankumar, just tried and it worked for me, maybe vpn..?14:16
ysandeep|afkmarios, pojadhav|afk arx/me will be ending our rr week tomorrow (+ arx is on PTO starting next week)- I will send a rr handover meeting tomorrow so that you can start next week.  rlandy|ruck fyi.. 14:19
ysandeep|afkarxcruz|rover, ^^ 14:20
*** ysandeep|afk is now known as ysandeep14:20
rlandy|ruckysandeep: perfect - thanks14:20
mariosysandeep: ack 14:20
chandankumarsshnaidm: with or without vpn, same issue14:20
chandankumarsshnaidm: you are pulling from this registry.access.redhat.com/ubi9-beta/ubi:latest ?14:20
chandankumarI am on fedora 3514:21
chandankumarpodman 3.4.214:21
sshnaidmchandankumar, https://paste.opendev.org/show/811406/14:21
chandankumarweired14:22
*** amoralej|off is now known as amoralej14:22
ysandeepmarios, pojadhav|afk rlandy|ruck arxcruz|rover invite for rr sync sent.. Please feel free to propose a new time if that doesn't work with your schedule.14:25
*** ysandeep is now known as ysandeep|afk14:27
chandankumarsshnaidm: figured out https://github.com/containers/buildah/issues/3659 and https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9-beta/html-single/9.0_release_notes/index#known-issue_containers14:28
chandankumarnow pull is working14:28
sshnaidmchandankumar, hmm, interesting, I don't have these beta keys..14:39
*** ysandeep|afk is now known as ysandeep15:12
jfrancoaysandeep|afk: hey, one doubt. how could I get access to the environment from a testproject run: https://review.rdoproject.org/r/c/testproject/+/36859 ?15:12
rlandy|ruckjfrancoa, you can recheck that job15:21
rlandy|ruckwe will add your keys once it starts15:21
rlandy|ruckand then hold that node15:21
jfrancoarlandy|ruck: I did recheck it a while ago. could you add them please? https://github.com/jfrancoa.keys15:22
jfrancoarlandy|ruck: dciabrin has provided me with a patch that might give a hint on why my upgrade fails with DISK_FULL, but he would like to jump into the env once if finishes15:23
rlandy|ruckthere is a merged failed15:24
rlandy|rucklet me recheck15:24
rlandy|ruckThis change or one of its cross-repo dependencies was unable to be automatically merged with the current state of its repository. Please rebase the change and upload a new patchset.15:25
rlandy|ruckjfrancoa: it's complaining about rebase15:27
rlandy|ruck^^15:27
rlandy|ruckI rebased the top patch15:27
rlandy|ruckbut it's likely one of the depends-on15:27
jfrancoarlandy|ruck: oh, thanks..I'm going to review that15:28
rlandy|ruckjfrancoa: once you have a running patch - ping and I'll add your keys and hold your node15:28
ysandeeprlandy|ruck, i rekicked https://review.rdoproject.org/r/c/testproject/+/32054 for metalsmith fix, was killed in vexx outage 15:30
* ysandeep out for the day15:34
*** ysandeep is now known as ysandeep|out15:34
mariosrlandy|ruck: i think i'll workflow -1 @ https://review.opendev.org/c/openstack/tripleo-ci/+/818222 until we sort out the d/stream stuff15:38
rlandy|ruckmarios: ack - thanks15:41
rlandy|rucksorry - bit scattered atm15:42
mariosrlandy|ruck: np i did the wf-1 there15:43
jfrancoarlandy|ruck: I think I fixed it: https://review.rdoproject.org/r/c/testproject/+/36859 could you add the keys and hold the node? (also, I'll need some hints on how to jump into the node or at least the credentials)15:50
rlandy|ruckjob is still queued15:50
rlandy|ruckwill add when job starts15:51
jfrancoarlandy|ruck: ack, thank you very much16:09
rlandy|ruckjfrancoa: I will leave you a PM with the login 16:11
jfrancoarlandy|ruck: great, I appreciate a lot (as I'll have to close in a few)16:11
chandankumarsee ya!16:12
*** chandankumar is now known as raukadah16:12
*** tosky_ is now known as tosky16:17
*** marios is now known as marios|out16:35
ade_leedviroel, so it looks like it finally came back as passed :) so now just scenario 416:50
dviroelade_lee: +117:01
*** amoralej is now known as amoralej|off17:32
rlandy|rucklunch - brb18:19
*** dviroel is now known as dviroel|afk19:53
rlandy|ruck2021-12-02 22:05:49.722527 | primary | [WARNING]: Found both group and host with same name: undercloud22:17
rlandy|ruck2021-12-02 22:05:50.306298 | primary | ERROR! couldn't resolve module/action 'authorized_key'. This often indicates a misspelling, missing collection, or incorrect module path.22:17
rlandy|rucksshnaidm: hey ... ansible fall out? https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5f8/817425/1/gate/tripleo-ci-centos-7-content-provider/5f87ba1/job-output.txt22:18
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-7-content-pro22:18
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-7-content-provider22:18
rlandy|ruckfailing since yesterday22:18
rlandy|rucklogging bug22:20
rlandy|rucksshnaidm: https://review.opendev.org/c/openstack/tripleo-quickstart/+/820170/ fixes ^^23:11
rlandy|ruckmerging unless objections23:11

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!