Tuesday, 2022-02-15

*** pojadhav|afk is now known as pojadhav04:41
*** ysandeep|out is now known as ysandeep05:30
*** amoralej|off is now known as amoralej07:00
frenzy_fridayhey folks, pls add to your review list: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/39424/ thanks!07:11
ysandeepfrenzy_friday, looking07:13
frenzy_fridayysandeep, thanks!07:14
ysandeepfrenzy_friday, in the screenshot you shared for master component line promotion - In the metric I see A-series - what does it mean - (data not available for this)?07:18
frenzy_fridaychecking07:20
ysandeepfrenzy_friday, nice work there thanks!,  left one comment, https://snapshots.raintank.io/dashboard/snapshot/SpO9FbFzMBxgbCCpAIAWM1S7gUuVsiSg?orgId=2 have "Wallaby CentOS-8 Integration periodic jobs"  we need similiar for "Wallaby CentOS-9 Integration periodic jobs" as well.07:23
chandankumarfrenzy_friday: also left one more comment regarding cs9 master current-tripleo-rdo promotion07:24
frenzy_fridayysandeep, somehow the screenshot is not loading for me right now. Checking in a while07:24
pojadhavfolks, kindly add in your review list : https://review.opendev.org/c/openstack/tripleo-validations/+/82821107:51
ysandeeppojadhav, voted.. thanks for working on it08:01
*** ysandeep is now known as ysandeep|lunch08:02
*** jpena|off is now known as jpena08:34
arxcruz|ruckysandeep|lunch marios https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/82919508:47
arxcruz|ruckblocking promotion on wallaby 08:47
mariosarxcruz|ruck: ack08:54
chandankumararxcruz|ruck: please +w it https://review.opendev.org/c/openinfra/python-tempestconf/+/82829808:58
chandankumarthanks!08:58
arxcruz|ruckchandankumar kopecmartin did it already 09:10
chandankumarkopecmartin: arxcruz|ruck thanks!09:10
*** ysandeep|lunch is now known as ysandeep09:12
ysandeeparxcruz|ruck, looking09:12
ysandeepfolks o/ please add 829204: Adding test_create_and_update_port_with_dns_name | https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/829204 in your review list09:42
*** pojadhav- is now known as pojadhav10:24
*** pojadhav is now known as pojadhav|afk10:24
*** dasm|off is now known as dasm11:00
dasmo/11:01
ysandeepdasm, early bird :D 11:05
ysandeepo/11:05
dasmysandeep: my brain betrayed me :P11:06
dasmysandeep: sfinucan mentioned about creating a bug. Do you know if he meant launchpad?11:07
ysandeepdasm: not sure.. Because steve also mentioned to raise a cix for dfg compute to take a look at hw_firmware_type=uefi in PSI, so internal bug on bugzilla seems better choice to me. 11:11
ysandeepdasm, may be once steve is in.. we confirm with him about - for what component we really need bug.11:12
dasmyeah. i'm reaching out to sfinucan11:13
ysandeepAt this point i am not sure if its psi environment issue or its nova issue.11:13
dasmtrue11:13
*** rlandy|out is now known as rlandy|ruck11:16
mariosno review time? rlandy|out ysandeep let me know if you want to rejoin11:16
ysandeepmarios, we are on call11:17
mariosysandeep: thanks11:17
*** dviroel|out is now known as dviroel11:19
*** carloss is now known as carloss|afk11:20
* dviroel kicking octavia component line11:28
dasmpoor octavia... is getting kicked11:29
dviroeldasm: hey - isn't still monday for you?  lol11:30
dasmdviroel: nope. i'm in tuesday for 3.5h already 11:32
ysandeeprlandy|ruck, marios chandankumar https://zuul.openstack.org/status#82922211:37
dviroeldasm: if you didn't sleep yet - it is still monday11:37
dviroel:)11:37
dasmdviroel: nah. my brain was like: hey you! yes, you! you had 4.5h of sleep already. you don't need more... "but i disagr..." no! you're gonna be fine11:39
dviroeldasm: yeah, sometimes happens with me - sometimes is the kid crying 3am - and I can't go back to sleep11:42
dasmdviroel: true. the good part is, we can do the work no matter what time ir11:43
dasm*is11:43
soniya29tosky, did you had a chance to go through abhishekh's comments in this patch:- https://review.opendev.org/c/openinfra/python-tempestconf/+/820169?12:00
chandankumarmarios: rlandy|ruck https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/82922112:01
chandankumarhttps://zuul.openstack.org/status#82922112:02
chandankumarhttps://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/82916412:05
chandankumarhttps://zuul.openstack.org/status#82916412:07
arxcruz|rucker12:15
chandankumarmarios: trying it here: https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/82916412:27
soniya29rlandy|ruck, i will be joining in 5 minutes12:29
*** pojadhav|afk is now known as pojadhav12:31
soniya29rlandy|ruck, i am in12:31
soniya29rlandy|ruck, ^^12:35
soniya29rlandy|ruck, can you hear me?12:37
*** ysandeep is now known as ysandeep|brb12:40
*** ykarel_ is now known as ykarel12:49
chandankumarrlandy|ruck: soniya29 tc ua syncup call13:00
ysandeep|brbrlandy|ruck, https://hackmd.io/NbfR-zG3T9GokDO3IZ0FeA#Full-Integration-pipeline-Status13:09
*** ysandeep|brb is now known as ysandeep13:10
*** amoralej is now known as amoralej|lunch13:20
*** carloss|afk is now known as carloss13:25
chandankumararxcruz|ruck: hello13:28
chandankumararxcruz|ruck: I have tested this https://review.opendev.org/c/openstack/diskimage-builder/+/828617 here https://review.rdoproject.org/r/c/testproject/+/39461 will fix https://bugs.launchpad.net/tripleo/+bug/196052913:29
chandankumarthanks!13:29
ysandeeparxcruz, sshnaidm, rlandy, rlandy|ruck, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, chandankumar, frenzy_friday, anbanerj,  dviroel, rcastillo, dasm, jm1, rlandy|ruck community call in 1 min, please add your agenda13:29
ysandeephttps://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both13:29
rlandy|ruckarxcruz|ruck: hello - how is life in ruck/rover land?13:47
arxcruz|ruckrlandy|ruck want to sync? we had some promotions, and i'm working to promote master c9 13:47
rlandy|ruckarxcruz|ruck: ack ... https://meet.google.com/zhq-gvco-cvz?pli=1&authuser=013:48
ysandeeppojadhav: sc010 debugging might take a while, I am checking with infra how can we solve those cert related issue during build-test-package (incase of depends-on), Could you start working on container-multinode/ipa job in the meantime.13:58
rlandy|ruckarxcruz|ruck: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/downstream-scenario010-kvm-trigger.yaml13:59
ysandeeppojadhav: I followed up on sc07, and after a green run - merged that one also.13:59
pojadhavysandeep, yeah sure14:00
rlandy|ruckarxcruz|ruck: https://github.com/rdo-infra/rdo-jobs/blob/5276cf2e423462cdb2c1a1a95e962e548633c04c/zuul.d/nodesets.yaml#L41714:01
pojadhavysandeep, yeah I saw sc07 got merged. :)14:01
ysandeeprlandy|ruck, jfyi.. dlrn move to https also broke  build-test-package role (we just came to know when trying depends-on), I am working with daniel to fix that.14:01
*** amoralej|lunch is now known as amoralej14:02
ysandeeppojadhav: do you have b.w to start working on container-multinode/and multinode-ipa for rhel-9,  bhagyashris is out this week.14:03
ysandeepI need to debug sc010, ovb and baremetal - so currently my plate is full14:04
rcastillo|rovero/14:06
ysandeeppojadhav: sry, I missed your ping above.. thanks! ++14:07
pojadhavysandeep, yep.. I will work14:08
rlandy|ruckysandeep: thank you14:08
chandankumarysandeep: it might be useful for you https://review.opendev.org/c/openstack/diskimage-builder/+/828617/9#message-251aded78ee1bc4a9b96e0cd901ec58024c59c8714:09
rcastillo|roverrlandy|ruck: arxcruz|ruck seems like I missed the sync call by a bit14:10
rlandy|ruckrcastillo|rover: on meeting ... pls sync with arxcruz|ruck14:10
rlandy|ruckarxcruz|ruck is fixing the trigger jobs14:10
rlandy|ruckwe will need to rerun all those after arxcruz|ruck merges that change14:11
ysandeepchandankumar: thanks for sharing, yes I am aware about this one, steve proposed that for bug which i reported for downstream ovb jobs.. https://bugzilla.redhat.com/show_bug.cgi?id=2053527#c5 14:11
arxcruz|ruckrlandy|ruck rcastillo|rover https://review.rdoproject.org/r/c/config/+/3946514:13
rlandy|ruckarxcruz|ruck: thanks - let's see if fbo has an objections14:14
rlandy|ruckotherwise I will merge14:15
ysandeeppojadhav, are you available for a quick meeting to discuss rhel9?14:23
pojadhavysandeep, yep14:25
pojadhavysandeep, https://meet.google.com/mjg-suoa-ydw14:26
ysandeeppojadhav, thanks, joining in a min14:28
pojadhavysandeep, ack14:28
rcastillo|roverarxcruz|ruck: yo around to sync real fast?14:28
arxcruz|ruckrcastillo|rover sure 14:29
rcastillo|roverarxcruz|ruck: https://meet.google.com/zhq-gvco-cvz14:30
rlandy|ruckarxcruz|ruck: rcastillo|rover: https://review.rdoproject.org/r/c/config/+/39465 merged14:30
rlandy|ruckyou can retest trigger jobs - thank you14:30
arxcruz|ruckrlandy|ruck how can we test it? normal testproject ?14:31
pojadhavysandeep, https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/31131414:41
arxcruz|ruckrlandy|ruck ?14:41
rlandy|ruckhttps://jenkins-cloudsig-ci.apps.ocp.ci.centos.org/view/phase-1-pipelines/job/rdo_trunk-promote-wallaby-centos8-current-tripleo/96/14:54
rlandy|ruckwallaby should have promoted there14:54
rlandy|ruckarxcruz|ruck: hey ... sorry meeting14:55
rlandy|ruckarxcruz|ruck: are you still syncing?14:55
rlandy|ruckyou need to rerun the trigger jobs14:55
arxcruz|ruckrlandy|ruck no, but we figure out how to do it, testing now 14:55
rlandy|ruckin testproject14:55
rlandy|ruckrdo zuul14:55
rlandy|ruckwith the right vars14:56
rlandy|ruckakahat: rcastillo|rover: hello14:57
rlandy|ruckakahat: rcastillo|rover: https://jenkins-cloudsig-ci.apps.ocp.ci.centos.org/view/phase-1-pipelines/job/rdo_trunk-promote-wallaby-centos8-current-tripleo/96/14:57
rlandy|ruckwallaby current-tripleo-rdo did not promote14:57
rlandy|ruckit should have14:57
rlandy|ruckcan you guys check into why14:57
rlandy|ruckit passed all jobs yesterday14:58
rlandy|ruckrcastillo|rover: ussuri and victoria also need to be retriggered there14:58
rlandy|ruckthanks14:58
rlandy|ruckrcastillo|rover: pls check the dlrn promotion tags on https://jenkins-cloudsig-ci.apps.ocp.ci.centos.org/job/tripleo-quickstart-promote-wallaby-current-tripleo-delorean-minimal/165/console14:59
*** pojadhav is now known as pojadhav|dinner15:04
chandankumarsee ya!15:05
rcastillo|roverrlandy|ruck: DLRN url isn't getting set right somehow. Looking15:07
rlandy|ruckchandankumar: have a good night15:08
rcastillo|roverrlandy|ruck: now that I'm looking, I can't find where the promote job loads the file from the get-hash job15:18
rcastillo|rovermight need to source it15:18
rlandy|ruckrcastillo|rover: give me 5 - will look at it with you15:21
*** ysandeep is now known as ysandeep|out15:25
marioslol @ https://opendev.org/openstack/tripleo-heat-templates/src/commit/ca4dbba3471defc27345b80f360d0cd20c1dbdd9/ci/environments/scenario012-standalone.yaml#L1 we merged that comment15:26
dviroellol15:28
rlandy|ruckarxcruz|ruck++ kvm job running15:34
arxcruz|ruckdid it work? O.o15:35
rlandy|ruckarxcruz|ruck: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/e4948e5e5dda4ae1a6f98d622507ea2d?logfile=console.log15:35
rlandy|ruckjob started in downstream zuul15:35
arxcruz|ruckfor me showed it fails 15:35
arxcruz|ruckthe trigger job 15:35
rlandy|ruckarxcruz|ruck: I rechecked your review15:36
*** dviroel is now known as dviroel|lunch15:37
rlandy|ruckhttps://review.rdoproject.org/r/c/testproject/+/3946615:39
rlandy|ruckarxcruz|ruck: hmm ... it did fail15:39
rlandy|ruckbut something triggered15:39
rlandy|ruckchecking15:39
rlandy|ruckperiodic-tripleo-ci-centos-8-scenario010-kvm-internal-standalone-train15:39
rlandy|rucktrain triggered15:39
arxcruz|ruckrlandy|ruck yeah, fail on scp, not sure what happened 15:39
rlandy|ruckarxcruz|ruck: weird15:41
rlandy|ruckno idea how the train job started then15:41
arxcruz|ruckrlandy|ruck maybe is the scp thing failing? 15:46
arxcruz|rucki don't know tbh 15:46
rlandy|ruckarxcruz|ruck: watching current run15:46
rlandy|ruckwe will need to ask fbo15:46
rlandy|ruckhe set up this trigger15:46
rlandy|ruckarxcruz|ruck: asking on #rhos-ops15:47
arxcruz|rucksec, my vpn drops 15:48
rcastillo|roverrlandy|ruck: the job that was missing the delorean url was using the hash file from before my patch merged15:51
rcastillo|rovermight be good to go on this run15:51
rlandy|ruckrcastillo|rover: ok - you'll need to rekick the two failing weirdo jobs after the tripleo job runs15:52
rlandy|ruckthen we will see15:52
rlandy|ruckrcastillo|rover: also if you rekick victoria and ussuri we will know15:52
rcastillo|roverrlandy|ruck: sure, resuming those16:05
rlandy|ruckrcastillo|rover++ thank you16:07
rlandy|ruckyep  that works16:07
*** amoralej is now known as amoralej|off16:08
*** pojadhav|dinner is now known as pojadhav|afk16:16
*** pojadhav|afk is now known as pojadhav16:16
*** marios is now known as marios|run16:21
rlandy|ruckarxcruz|ruck: rcastillo|rover: looks like master c9 and trin c8 ovb jobs all passed16:23
rlandy|ruckjust waiting on kcm jobs to finish to promote16:23
rlandy|rucknice work16:23
rcastillo|rovernice16:24
rlandy|ruckrcastillo|rover: need to meet about anything now?16:24
rcastillo|roverlooking at components now16:24
rlandy|ruckgreat16:24
rlandy|ruckthank you16:24
rcastillo|roverdon't think so16:24
rlandy|ruckysandeep|out: need help with 16.2?16:30
*** ysandeep|out is now known as ysandeep16:33
ysandeeprlandy|ruck, for 16.2 i think we can promote the current hash neglecting fs020 - 1 tempest failure(i have proposed fixes which are already merged but blocked due to tripleo component- ftbfs issue)16:35
ysandeepfix is more job config related and not product related - so okay to promote16:37
ysandeeprlandy|ruck, details in https://trello.com/c/Rm90mCHN/2350-cixbz2053378osp162rhel8periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-rhos-162-is-failing-on-tempest-test-testcreat16:37
rlandy|rucklooking16:37
rlandy|ruckysandeep: if it's blocked we can just report it16:38
rlandy|ruckysandeep: how long are we waiting on FTBS?16:39
rlandy|ruckhttps://trello.com/c/M2qeaHWM/2327-cixftbfsosp-162?16:39
rlandy|ruckI'm ok to report we are blocked by that16:40
rlandy|ruckunless you think we should promote for other reasons16:40
rlandy|rucklet me know16:40
rlandy|ruckysandeep; ^^16:40
ysandeepfine by me, but don't see a point in blocking it as its config related - atleast the fs020 failure.16:40
rlandy|ruckysandeep: ok - do you want to propose the change to temp remove the test or should I?16:41
ysandeeprlandy|ruck, proposing, and will revert after promotion 16:41
ysandeeprlandy|ruck, fyi.. this was the only failing test for fs020: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-ovb-1ctlr_2comp-featureset020-internal-rhos-16.2/79069e1/logs/undercloud/var/log/tempest/failing_tests.log.txt.gz 16:43
ysandeepsame issue was in full-tempest-api - which my tqe patch already fixed.16:43
rlandy|ruckysandeep: got it - thanks16:44
rlandy|ruckrcastillo|rover: hey - when you are done with component lines, can you take a look at c9 ipa-multinode job - failing again16:45
ysandeeprlandy|ruck, https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/31140416:48
rlandy|ruckysandeep: periodic-tripleo-ci-rhel-8-bm_envD-3ctlr_1comp-featureset035-rhos-16.2 - should that still be commeted out?16:49
ysandeeprlandy|ruck, that's running stable now a days, i think we can put it back in criteria, updating .. good point, thanks!16:50
*** dviroel|lunch is now known as dviroel16:52
ysandeeprlandy|ruck, updated https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/31140416:52
rlandy|ruckysandeep: thanks - merged16:53
*** marios|run is now known as marios16:54
mariosrlandy|ruck: need something? /me going in a minute16:56
mariosrlandy|ruck: basically, https://review.opendev.org/829236 please16:56
mariosrlandy|ruck: if time permits, it has a followup @ https://review.opendev.org/c/openstack/tripleo-ci/+/82930916:56
marios(missed scen12 as per dviroel comments but didn't want to delay 829236 with an update) ^^ 16:57
ysandeepmarios, ++ thanks16:57
rlandy|ruckmarios: thanks - will merge both16:58
mariosthanks have a good rest of day then o/ 16:58
rlandy|ruckw+ both16:59
rlandy|ruckdviroel++ nice catch16:59
rlandy|ruckthat's why we keep dviroel around :)16:59
dviroelo/16:59
marios;D16:59
ysandeeprlandy|ruck, we got 16.2 promotion, lets revert https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/31093517:09
rlandy|ruckperfect - merged17:11
ysandeepthanks, I am out.. see you tomorrow o/ 17:11
*** ysandeep is now known as ysandeep|out17:11
*** jpena is now known as jpena|off17:38
rlandy|ruckrcastillo|rover: hey18:07
rlandy|ruckrcastillo|rover: tox molecule failure in gate18:07
rlandy|ruck2022-02-15 17:31:01.465095 | primary | [31mFailed to download metadata for repo 'centos-nfv-ovs': Yum repo downloading error: Downloading error(s): repodata/cb6e309142d9fd04dfde37457631070e418ccf7fb08ff6db1cc9e2c5b7302b0a-primary.xml.gz - Cannot download, all mirrors were already tried without success; repodata/3d6c61b9f9470d383665b26f18462170548bcbd1ba84cdad19654d489dea8884-filelists.xml.gz - Cannot download, all mirrors were18:08
rlandy|ruckalready tried without success[0m18:08
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-tox-molecule&skip=018:09
rlandy|ruckwe need to keep an eye on hat18:10
rlandy|ruckthat 18:10
rlandy|ruckwe need that patch to go through18:10
rlandy|ruckhttp://mirror.regionone.vexxhost-nodepool-tripleo.rdoproject.org/centos/8-stream/nfv/x86_64/18:11
rlandy|ruckmight have just updated18:12
rlandy|rucklunch - brb18:12
rlandy|rucktripleomastercentos9 promoting18:23
ade_leedviroel, hey18:51
ade_leedviroel, so I've been investigating the latest issues in the centos-9 fips jobs18:51
ade_leeand figured out that the problem lies in python-cryptography.18:52
ade_leebasically, the python-cryptography in centos 9 has not yet been updated to detect fips mode in openssl 3.x18:53
dviroelade_lee: hi o/18:53
ade_leedviroel, I have a test build from the python-cryptography maintainer to fix this18:53
ade_leedviroel, so we just need to modify your https://review.opendev.org/c/openstack/tripleo-quickstart/+/828911/1/config/release/tripleo-ci/CentOS-9/master.yml review to add the test version18:54
dviroelade_lee: yes, do you have a repo with it?18:54
ade_leehttp://people.redhat.com/~lhh/python-cryptography/18:54
dviroelack18:55
ade_leedviroel, lmk when ready on that - and then I'll modify the centos-9 triple-ci job -- have to remove my nova depends-on18:56
dviroelade_lee: https://review.opendev.org/c/openstack/tripleo-quickstart/+/828911 updated19:04
ade_leedviroel, thanks -- trying it out 19:05
ade_leedviroel, not sure if I actually need https://review.opendev.org/c/zuul/zuul-jobs/+/827366  as a dependency or not, or if its all handled by your patch19:08
dviroelade_lee: we probably don't need. We need to provide those repos to container build functions, which we are doing with quickstart change19:12
ade_leedviroel, yeah -- I'll kill it in the next iteration19:12
dviroelok19:12
rcastillo|roverlunch, brb19:31
rlandy|ruckrcastillo|rover: hey20:57
rcastillo|roverhi20:57
rlandy|ruckrcastillo|rover: wallaby current-triple-rdo is promoting bt stuck in the promoter21:22
rlandy|ruckpinged akahat about it21:22
rlandy|ruckso the dlrn reporting must be ok21:23
rcastillo|roverrlandy|ruck: ack21:23
*** dviroel is now known as dviroel|out21:35
rlandy|ruckrcastillo|rover: can you look into: https://jenkins-cloudsig-ci.apps.ocp.ci.centos.org/job/tripleo-quickstart-promote-master-centos9-current-tripleo-delorean-minimal/18/console22:33
rcastillo|roverrlandy|ruck: yes22:34
rlandy|ruckwhat a day ... need to go for a run 22:46
rlandy|ruckrcastillo|rover: ^^ back later22:46
rlandy|ruckpls leave notes 22:46
rlandy|ruckif you need anything22:46
*** rlandy|ruck is now known as rlandy|ruck|bbl22:46
rcastillo|roverwill do, have fun22:46
*** rlandy|ruck|bbl is now known as rlandy|ruck23:46
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-7-content-provider&skip=023:49
rlandy|ruckissue there23:49
rcastillo|roverrlandy|ruck: I think there's a bug in the repo_cmd_before of the release file: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/centosci/CentOS-9/master-current-tripleo.yml#L4423:58
rcastillo|rovertries to do things with dlrn_hash_tag_newest even if it's not defined23:59
rlandy|rucksec23:59
rlandy|ruckis there a problem with c923:59
rlandy|rucklooks like we have a gate blcoker on c723:59
rlandy|ruck^^23:59
rlandy|ruckhttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-7-content-provider&skip=023:59

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!