Friday, 2022-06-03

*** dviroel|afk is now known as dviroel00:19
dviroelrlandy|out: no idea what is happening, shouldn't be triggering that molecule job on that patch, it was removed from check. Should run rdo-mol-container-login01:01
* dviroel out01:01
*** dviroel is now known as dviroel|out01:01
*** ysandeep|out is now known as ysandeep05:00
ysandeepchandankumar, good morning o/ 05:03
ysandeepchandankumar, code.eng working for you?05:03
*** rcastillo_ is now known as rcastillo05:34
chandankumarysandeep: morning \o06:23
chandankumarysandeep: yes, it's working06:23
ysandeepchandankumar, thanks its working for me now, there was some issue in morning06:24
*** jpena|off is now known as jpena07:09
marioschandankumar: come on... you know you want to07:30
mariosbhagyashris|rover: o/ join us? 07:31
mariosthen you can change your nick!07:31
marios;)07:31
*** bhagyashris|rover is now known as bhagyashris07:37
chandankumarmarios: bhagyashris thank you :-)07:56
*** chandankumar is now known as chkumar|ruck07:56
jpodivinchkumar|ruck: I'm seeing a failure of the tox-linters job, flake 8 to be specific. "TypeError: parse_options() takes 2 positional arguments but 4 were given" It feels like I've seen this before, but I can't really recall if we had done anything about it.08:48
jpodivinhttps://zuul.opendev.org/t/openstack/build/03062b77ccb148ac8649581b110ac17708:48
chkumar|ruckjpodivin: checking08:48
jpodivinit looks like a wrong version is being pulled somewhere. 08:49
chkumar|ruckValueError: min-python-version (3.8.0): unknown version08:50
jpodivinyeah, 08:50
jpodivinthat's it. 08:50
jpodivinjust before that you have the arg number mismatch. 08:50
jpodivinIt doesn't happen every time however. The same job passes on the that repo/branch in other cases. And it's also on tripleo-validations08:55
chkumar|ruckweired09:00
jpodivinindeed09:00
chkumar|ruckjpodivin: can you recheck the patch?09:00
chkumar|ruckjpodivin: one query https://codesearch.opendev.org/?q=python_requires%20%3D&i=nope&literal=nope&files=&excludeFiles=&repos=09:01
jpodivinchkumar|ruck: already did. 09:01
chkumar|ruckpython_requires is defined under metadata ,09:01
chkumar|ruckwhy we are defining it under options?09:01
jpodivinchkumar|ruck: well, the setuptools puts them there .09:02
jpodivinchkumar|ruck: https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=python_requires#python-requirement09:03
jpodivinchkumar|ruck: and pbr itself follows the convention in it's own metadata09:04
jpodivinIn that query I see another example https://opendev.org/opendev/elastic-recheck/src/branch/master/setup.cfg#L2209:05
jpodivinin any case. I've been going trough the relevant code, and I may have something 09:19
jpodivinchkumar|ruck: so this is the version of the flake8-typing we are pulling https://github.com/asottile/flake8-typing-imports/blob/c62d1970a8b8ef54aeb3ae2639ffccd6206689e3/flake8_typing_imports.py#L42709:20
jpodivinThe last version of python it has stored is ... 3.7.4 which explains our bug. 09:20
jpodivinBumping the dependency to ... well 1.12.0 is the newest so let's go for that, should help09:21
jpodivinor any other which covers our code. 09:21
chkumar|ruckjpodivin: ok, sure09:24
jpodivinchkumar|ruck: LP submitted https://bugs.launchpad.net/tripleo/+bug/1977495 I'm testing version bump now and linters output is green. I'll submit a fix proper after all tests finish 09:29
*** ysandeep is now known as ysandeep|lunch09:39
frenzy_fridayhey folks, need a help. All the periodic wallaby 8 jobs seem to be using registry namespace tripleowallaby. But it is tripleowallabycentos8 in quay. Where do I change this?09:43
chkumar|ruckfrenzy_friday: job link?09:45
frenzy_fridaychkumar|ruck, https://review.rdoproject.org/r/c/testproject/+/4226509:46
frenzy_fridaylog: https://logserver.rdoproject.org/65/42265/10/check/periodic-tripleo-ci-centos-8-standalone-wallaby/603403b/logs/undercloud/home/zuul/containers-prepare-parameters.yaml.txt.gz09:46
frenzy_fridayhere namespace: quay.rdoproject.org/tripleowallaby should be namespace: quay.rdoproject.org/tripleowallabycentos8 (like it is here https://logserver.rdoproject.org/65/42265/10/check/periodic-tripleo-ci-centos-9-ovb-1ctlr_1comp-featureset002-wallaby/ef505a7/logs/undercloud/home/zuul/containers-prepare-parameter.yaml.txt.gz)09:46
frenzy_fridayI see one difference here https://review.opendev.org/c/openstack/tripleo-quickstart/+/843007/2/config/release/centosci/CentOS-9/wallaby-current-tripleo.yml#b9 vs https://review.opendev.org/c/openstack/tripleo-quickstart/+/843007/2/config/release/tripleo-ci/CentOS-8/promotion-testing-hash-wallaby.yml#b809:48
frenzy_fridaychkumar|ruck, can i change https://review.opendev.org/c/openstack/tripleo-quickstart/+/843007/2/config/release/centosci/CentOS-8/wallaby-current-tripleo.yml to tripleowallabycentos8 ?09:49
chkumar|ruckLooking09:50
chkumar|ruckfrenzy_friday: in promotion-test-hash release file, please set the correct namespace exists in quay09:55
chkumar|ruckotherwise it will not work09:55
chkumar|ruckcommented on the patch09:55
frenzy_fridaychkumar|ruck, thanks09:55
chkumar|ruckmarios: https://logserver.rdoproject.org/99/43399/1/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-wallaby/f37b1d6/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt.gz09:57
chkumar|ruckmarios: you have seen this error ansible-macros-2021.1.2-2.el9s.noarch: Cannot download, all mirrors were already tried without success in the past09:57
chkumar|ruckis it a mirror issue?09:57
marioschkumar|ruck: looking10:12
marioschkumar|ruck: yes i saw this once with rlandy yesterday we weren't sure if it  was a new issue... if you see again must be a new bug 10:13
marioschkumar|ruck: i think worth looking at last 2/3 runs and file bug for it10:13
chkumar|ruckmarios: yes, sure, thanks!10:14
marioschkumar|ruck: hmm might be inconsistent let me add some info here with pointers but worth filing it i think 10:15
marioschkumar|ruck: on 31st i saw it 2x there: (from 30th and 27th May): https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/84c377d/logs/supplemental/home/cloud-user/ipa_prep.sh.log.txt 10:16
marioshttps://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset039-wallaby/749615e/logs/undercloud/home/zuul/undercloud_install.log.txt.gz (2nd one is for diffferent package)10:16
marioschkumar|ruck: so because of that i reran it to see if it was some new issue, but it passed there https://review.rdoproject.org/r/c/testproject/+/43284/1#message-510ad80f353d1e8eebc5f60e59b03da07a75c118 (and we promoted with that result)10:17
chkumar|ruckmarios: thanks, I have two instances from yesterday one for periodic run and testproject10:19
marioschkumar|ruck: after it passed in test i didn't give any more attention i thought it was a transient issue. instead it seems to be inconcistnet? or more likely bad mirror possibly10:21
chkumar|ruckmarios: yes, seems to be bad mirror10:21
*** ysandeep|lunch is now known as ysandeep10:22
*** rlandy|out is now known as rlandy10:23
rlandychkumar|ruck: hi - want to sync?10:25
chkumar|ruckrlandy: yes10:27
rlandychkumar|ruck: https://meet.google.com/evs-rrba-kfg?pli=1&authuser=010:27
* bhagyashris brb10:39
ysandeeprlandy, chkumar|ruck want to know status about 17/9?10:45
rlandyysandeep: yes10:45
rlandypls join https://meet.google.com/evs-rrba-kfg?pli=1&authuser=010:45
frenzy_fridaybot add to review list https://review.opendev.org/c/openstack/tripleo-quickstart/+/843007. https://review.opendev.org/c/openstack/tripleo-ci/+/84255111:01
reviewbotI have added your review to the Review list11:01
reviewbotI have added your review to the Review list11:01
*** dviroel|out is now known as dviroel11:22
rlandydviroel: can you join review time?11:23
jpodivindviroel: debugging set to true for the pipeline https://review.rdoproject.org/r/c/rdo-jobs/+/4340711:41
jpodivinit's for the entire check so we should see everything that moves there.11:41
rlandychkumar|ruck: fyi; we w'ed https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/84227311:54
chkumar|ruckrlandy: ok11:54
soniya29rlandy, marios, frenzy_friday, updated all the patches11:57
mariosack soniya29 will check again11:57
soniya29marios, thanks :)11:57
dviroeljpodivin: thanks by the patch btw12:02
dviroelrlandy: marios: https://review.rdoproject.org/r/c/config/+/43408 and then https://review.rdoproject.org/r/c/rdo-jobs/+/43409 (failing because of config def)12:06
marioslunch biab12:06
* dviroel brb 12:25
mariosdviroel: so it *was* because of rdo-tox-molecule (the parent) ?12:27
dviroelmarios: it is a guess, not sure, but the idea is to remove since we are not testing anything with that job12:32
mariosdviroel: ack ok yeah i checked the patch and i got this impression (you're removing the parent since it isn't needed any more mainly the point is to remove the failing one we're still not 100% sure )12:35
dviroelmarios: yes sir12:39
*** ysandeep is now known as ysandeep|afk12:42
rlandychkumar|ruck: hey - did you rekick rhel-9? may it was ysandeep|afk? either way, thanks12:57
chkumar|ruckrlandy: not yet13:01
rlandyk - it's running now13:01
chkumar|ruckrlandy: marios let's see how kvm octavia job behaves with big node https://code.engineering.redhat.com/gerrit/c/testproject/+/41254613:01
rlandycool13:01
marioschkumar|ruck: thx13:02
rlandypojadhav: 1-1?13:04
*** ysandeep|afk is now known as ysandeep13:06
ysandeeprlandy chkumar|ruck, I ran the 17 line 13:07
ysandeepchkumar|ruck, https://sf.hosted.upshift.rdu2.redhat.com/logs/1e/1ee40aec65d02daee1a0064e1b219a8024eb9288/openstack-periodic-integration-rhos-17-rhel9/periodic-tripleo-rhel-9-upload-job-trigger-rhos-17/19aa9ef/job-output.txt13:08
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/df8aa2bb056f4c1bba20632db2f4d926?logfile=console.log13:08
rlandyrekicked to check if that is legit13:08
chkumar|ruckFailed to download packages: initscripts-10.11.4-1.el9.x86_64: Cannot download, all mirrors were already tried without succes13:09
chkumar|rucknot seen13:09
rlandyack13:09
rlandybut the other jobs have it13:09
chkumar|ruckmirror issue13:09
rlandyfailed again13:09
rlandywhat node is that on?13:09
ysandeepmulti-node-bridge : Install openvswitch - good to check from where other jobs got that13:09
rlandywhy is the trigger job different13:09
rlandymulrinode-bridge set up is the same eveywhere13:10
ysandeepmay be we are not laying rhos-release repos here 13:10
ysandeepin pre playbooks13:10
rlandyLabel: upstream-rhel-9-013:10
rlandyright node13:10
rlandyysandeep: likely no rhos-release13:11
rlandywe had no need for it before13:11
ysandeepwe are not capturing etc/yum.repos.d here for package_list so hard to tell from where it was coming 13:12
ysandeephttps://sf.hosted.upshift.rdu2.redhat.com/logs/1e/1ee40aec65d02daee1a0064e1b219a8024eb9288/openstack-periodic-integration-rhos-17-rhel9/periodic-tripleo-rhel-9-upload-job-trigger-rhos-17/19aa9ef/13:12
chkumar|ruckDoes some packages got removed from osp trunk candidate deps repo?13:12
ysandeepchkumar|ruck, yes 13:12
ysandeepbut good to sync our job with rest of jobs13:12
ysandeepi mean install rhos-release in pre before installing openvswitch 13:13
chkumar|ruckysandeep: in pre prepare-rhos-release : Install rhos-release package gets skipped13:13
chkumar|ruckit is conditional for rhos-16.1 i think13:13
rlandyhmmm ... yeah13:14
chkumar|ruckearlier it was working fine13:14
rlandywe needed the bridge install package with 16.113:14
rlandylet's see if we can re-enable that13:14
rlandy2022-06-03 13:08:40.678039 | TASK [prepare-rhos-release : Install rhos-release package]13:15
rlandy2022-06-03 13:08:40.771775 | primary | skipping: Conditional result was False13:15
chkumar|ruck /var/cache/dnf/rhosp-rhel-9.0-baseos-31c9a7304cdab577/packages/initscripts-10.11.4-1.el9.x86_64.rpm13:18
chkumar|ruckit comes from baseos13:18
rlandybut it only fails on this job13:19
* rlandy trying patch13:19
ysandeeprlandy, chkumar|ruck new issue now: https://sf.hosted.upshift.rdu2.redhat.com/logs/1e/1ee40aec65d02daee1a0064e1b219a8024eb9288/openstack-periodic-integration-rhos-17-rhel9/periodic-tripleo-ci-rhel-9-standalone-rhos-17/6c523ea/logs/undercloud/var/log/tripleo-container-image-prepare.log 13:21
dviroelmarios: rlandy: a comment in layuot.yaml fix the issue for that change https://review.rdoproject.org/r/c/rdo-jobs/+/43391 13:22
ysandeepits looking for containernetworking-plugins-1.0.1-3.el9.x86_64 , but ideally it should not - we use netavark in rhel9.0 now 13:22
* ysandeep holding a node to debug that13:24
rlandydviroel: thanks - +2'ed13:25
ysandeeprlandy, chkumar|ruck the previous bug seems to be solved - br-ctlplane bridge is up13:27
ysandeephttps://sf.hosted.upshift.rdu2.redhat.com/logs/1e/1ee40aec65d02daee1a0064e1b219a8024eb9288/openstack-periodic-integration-rhos-17-rhel9/periodic-tripleo-ci-rhel-9-standalone-rhos-17/6c523ea/logs/undercloud/var/log/extra/network.txt13:27
rlandygood - one down13:27
mariosdviroel: k will check in bit13:27
chkumar|ruckysandeep +++ cool13:27
chkumar|ruckysandeep: earlier it was cni?13:27
ysandeepchkumar|ruck, yeah 13:28
chkumar|rucknow it needs netavark13:28
chkumar|ruckit is introduced in podman 413:28
ysandeepwe already have netavark installed there.. need to find why its complaining about missing cni library13:28
chkumar|ruckysandeep: let me push a patch to tripleo_ansible podman role13:30
chkumar|ruckto configure it based on needs13:30
chkumar|ruckysandeep: https://github.com/containers/common/blame/main/docs/containers.conf.5.md#L30113:32
ysandeepchkumar|ruck, added "netavark" in netavark_backend and it works - http://pastebin.test.redhat.com/105606613:37
ysandeepBut Q is why its not taking netavark by default 13:38
ysandeep# The default value is empty which means that it will automatically choose CNI or netavark. If there are13:38
ysandeep# already containers/images or CNI networks preset it will choose CNI.13:38
chkumar|ruckysandeep: https://review.opendev.org/c/openstack/tripleo-ansible/+/84459313:41
chkumar|ruckysandeep: https://www.redhat.com/sysadmin/podman-new-network-stack13:41
chkumar|ruckysandeep: with above change merges, we can remove the workaround later13:43
chkumar|ruckysandeep: thank you, learned about this change in podman13:43
ysandeepchkumar|ruck, yes I am aware of these changes - had to deal with container build job last week - https://bugzilla.redhat.com/show_bug.cgi?id=2091840 13:45
ysandeepwe need to update this workaround also: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/412327 13:45
ysandeepwas waiting for buildah team to tell how to update mtu in netavark case 13:45
chkumar|ruckysandeep: ok, thanks for taking care of that :-)13:46
chkumar|ruckmarios: rlandy I have good news https://review.rdoproject.org/r/c/rdoinfo/+/43405/1#message-54a3b1a755f0685091c5e399f583ad8593b845a514:03
chkumar|ruckwe can clear one test 14:04
rlandynice14:04
rlandychkumar|ruck: ysandeep: adding rhos-release does not help onmultinode-bridge14:06
chkumar|ruckmarios: rlandy master is still broken14:07
marioschkumar|ruck: nice is that with the tempest patches (grabbing newer tempest?)14:07
rlandythink we another repo there14:07
chkumar|ruckmarios: I put all those reviews in spec file14:07
reviewbotDo you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks.14:07
rlandywill look more after calls14:07
chkumar|ruckmarios: and built it there https://review.rdoproject.org/r/c/rdoinfo/+/43405/1#message-54a3b1a755f0685091c5e399f583ad8593b845a514:08
chkumar|ruckmarios: https://review.rdoproject.org/r/c/openstack/tempest-distgit/+/4340414:08
marioschkumar|ruck: k nice so now run it a few times :) to see14:08
marioschkumar|ruck: but anyway those patches should merge like 3/4 left now?14:08
mariosmaybe can push that14:08
chkumar|ruckmarios: 4 are left to merge14:08
mariosnice one on getting them built and tested chkumar|ruck 14:09
marioschkumar|ruck: wow https://review.rdoproject.org/r/c/openstack/tempest-distgit/+/43404 you actually included the patches there :D14:10
rlandychkumar|ruck; can you join for a bit?14:16
*** dviroel is now known as dviroel|lunch14:53
chkumar|ruckrlandy: I will take a look at that one, may be tomorrow15:04
ysandeepchkumar|ruck, I tested your patch locally https://review.opendev.org/c/openstack/tripleo-ansible/+/844593 (editing /etc/containers/containers.conf) it works 15:04
chkumar|ruckrlandy: please keep an eye on RHOS-17 rhel-915:04
chkumar|ruckand train15:04
ysandeepchkumar|ruck, 17/9 is dead without your patch 15:04
chkumar|ruckand leave npotes there https://hackmd.io/HMjD8MouQXmGWMJP5o76JQ15:04
rlandychkumar|ruck: k - will do15:04
ysandeepchkumar|ruck, Let me open a rhbz for 17/9 and cix that15:05
chkumar|ruckysandeep: sure15:05
* ysandeep doing some testing for root cause currently15:06
chkumar|ruckleaving now, see ya people, happy weekend!15:07
marioshave a good weekend o/ 15:40
*** marios is now known as marios|out15:40
*** xek__ is now known as xek15:56
ysandeepchkumar|ruck, rlandy, https://bugzilla.redhat.com/show_bug.cgi?id=2093406 and cixed16:04
rlandyysandeep: thank you16:04
ysandeeprlandy, please create a wallaby cherry-pick once https://review.opendev.org/c/openstack/tripleo-ansible/+/844593 merges 16:04
ysandeeprlandy, for above bug ^^16:05
*** ysandeep is now known as ysandeep|out16:08
ysandeep|outrlandy, I just noticed we both were tagged here https://trello.com/c/p1VGdfbY/2540-cixlp1975671tripleociproa-component-lines-jobs-and-rdo-third-party-check-jobs-are-failing-because-master-containers-are-not-avai , I can reply on my Monday morning if you don't get to it first.16:11
ysandeep|outteam o/ see you next week , happy weekend!16:11
*** jgilaber|errand is now known as jgilaber|away16:17
*** dviroel|lunch is now known as dviroel16:20
* rlandy_ looks16:28
dviroeloh, we still need another vote here https://review.rdoproject.org/r/c/rdo-jobs/+/4339116:32
rlandy_ysandeep|out: answered16:33
*** rlandy_ is now known as rlandy16:33
rlandydviroel: want me to w+ that?16:33
dviroelrlandy: would fix container-login issue16:34
dviroelrlandy: and we can investigate the molecule issue later16:34
rlandydone16:34
dviroelso, yes16:34
rlandychkumar|ruck: ysandeep|out: got it16:44
rlandytrigger job16:44
rlandyperiodic-tripleo-rhel-9-upload-job-trigger-rhos-17testprojectmastercheck398758,3116:46
rlandy2 mins 19 secs2022-06-03 16:43:14SUCCESS16:46
rlandyhttps://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/413414 Add rhos_release_args for rhel-9  trigger job16:49
rlandychkumar|ruck: ysandeep|out: ^^16:50
rlandydviroel: rcastillo: frenzy_friday: ^^ could I get a review to fix trigger job on rhel-916:51
* dviroel looks16:54
dviroeldone16:56
dviroelrlandy: after that, you may want to try container-login role again16:56
dviroelrerun*16:56
rlandydviroel: sure - lunch - will try when back16:57
rlandythanks for reviews16:58
reviewbotDo you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks.16:58
dviroelack16:58
*** jpena is now known as jpena|off17:06
rlandydviroel: ok - trying the content-provider job17:56
dviroelack17:56
rlandycurrent-tripleo/2022-06-03 13:05 16.2 promoted17:57
dviroel~o~17:57
rlandydviroel: ok - job kick in action18:01
rlandylet's hope18:01
rlandydviroel: pls see prvt18:06
rlandychkumar|ruck: consistent failure on ipa18:54
rlandypinged security18:54
* dviroel food brb19:50
* dviroel back20:02
* dviroel going out - have a great weekend team21:20
*** dviroel is now known as dviroel|out21:21
rlandysee you all tuesday21:58

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!