Monday, 2021-08-16

« Friday, 2021-08-13 Index Tuesday, 2021-08-17 »
*** pojadhav- is now known as pojadhav06:38
akahatsoniya29|rover|brb, ping07:05
akahatsoniya29|rover|brb, you are familiar about the package conflict issue: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_3a9/periodic/opendev.org/openstack/neutron/master/tripleo-ci-centos-8-content-provider/3a907f7/logs/undercloud/home/zuul/workspace/logs/container-builds/74f8c613-5c57-4f1f-beb9-8847e4f6265e/base/os/mistral-base/mistral-base-build.log07:06
akahathttps://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-content-provider07:06
akahatsoniya29, hey07:06
soniya29akahat, i am looking07:06
akahatsoniya29, okay.. i thought you didn't go the message.. 07:07
akahatsorry07:07
soniya29akahat, i am not aware about this issue07:07
akahatsoniya29, okay.. let me open bz for it.07:07
*** jpena|off is now known as jpena07:42
zbrmarios|ruck: when do you go into vacation?07:49
chandankumarakahat: above issue got fixed last thursday https://review.opendev.org/q/906265c433003886d87ac29d9daf68eccb69a40c07:50
marioszbr: wed07:50
zbrmarios: in that case we should meet to talk about tripleo.repos asap, preferably including chandankumar, so we do not risk delaying the work more. 07:53
zbri do not want to endup merning these one day before i move ;)07:54
marioszbr: sure do you want to setup a meet? 07:54
marioszbr: send an invite for today ?07:54
zbrok in 6min? or should try more lead time07:55
marioszbr: i can do it i don't know about chandankumar 07:55
zbr30min at most, likely 15m to be enough. 07:55
chandankumarmarios: zbr will join and help in getting this merged.07:56
zbrsent07:57
marioszbr: need 2 mins sorting something just came up here joining in 2 08:00
zbrhttps://review.opendev.org/q/topic:%22tripleo-get-hash%22+(status:open%20OR%20status:merged)08:01
zbrhttps://review.opendev.org/c/openstack/tripleo-quickstart/+/801776/1608:02
zbrsshnaidm: do you have a link to job for publishing collections?08:26
pojadhavchandankumar, ykarel : please help to merge this last patch https://review.rdoproject.org/r/c/config/+/3457508:58
akahatchandankumar, thanks09:06
ykarelpojadhav, ack09:53
pojadhavykarel, thanks !!09:58
*** beagles is now known as eagles10:16
chandankumarpojadhav: will look in a while.10:30
*** sshnaidm|pto is now known as sshnaidm10:30
*** sshnaidm is now known as sshnaidm|pto10:31
* marios biab food 10:32
pojadhavchandankumar, thanks!10:51
*** jpena is now known as jpena|lunch11:16
*** dviroel|out is now known as dviroel|ruck11:26
*** rlandy is now known as rlandy|rover11:34
arxcruzfrenzy_friday: i'm getting the foollowing error when i tried to setup the elastic recheck 11:41
arxcruz    ModuleNotFoundError: No module named 'setuptools_rust'11:41
arxcruzfrenzy_friday: is this a known issue? I'm running on centos stream 11:41
bhagyashrispojadhav, hey cockpit tests are failing https://logserver.rdoproject.org/88/34688/4/check/tox-py36-ci-config/d7b5abe/tox/reports.html11:42
frenzy_fridayarxcruz, did you pip install -r requirements.txt?11:44
* frenzy_friday checks11:45
arxcruzfrenzy_friday: yup 11:46
chandankumarbhagyashris: working on a fix11:46
chandankumargot hit in my patch also11:46
bhagyashrischandankumar, ack, pojadhav ^11:47
frenzy_fridaychecking in some time, maybe I missed something11:49
rlandy|roverdviroel|ruck: hey - how is ruck/rover going?11:49
rlandy|roverupdated CIX11:49
dviroel|ruckrlandy|rover: hi, i was looking that issue that you mentioned in the chat11:50
rlandy|roverI mentioned a few :)11:50
rlandy|roverwhich one?11:50
rlandy|roverussuri promoted11:50
rlandy|roverso train is the real issue now11:51
dviroel|ruckhttps://bugs.launchpad.net/tripleo/+bug/194000711:51
rlandy|roverdviroel|ruck: looks like rabi has the solution11:52
rlandy|roverwe will discuss at CIX11:52
dviroel|ruckyeah, was looking at his commens11:52
rlandy|roverdviroel|ruck: probably going to bug something around train11:53
pojadhavbhagyashris, ack11:53
rlandy|roverreran scenario001 yesterday and it failed11:54
rlandy|roverdviroel|ruck: we have a real issue here: https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario001-standalone-train11:55
rlandy|rovercan you create a LP bug11:55
rlandy|roverand mark it promotion blocker11:55
rlandy|roverlet me know if you need help with that11:56
dviroel|ruckyes, let me take a look in the logs and i'll create the LP11:56
rlandy|roverhmmm ... think maybe we saw that python-tempestconf before 11:57
rlandy|roversoniya29: arxcruz: train failing Executing python-tempestconf https://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-train/0de27d7/job-output.txt11:58
rlandy|roverhttps://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-train/0de27d7/logs/undercloud/var/log/tempest/tempestconf.log.txt.gz11:58
rlandy|rover^^ did that come up last sprint11:58
rlandy|roverdviroel|ruck: ^^ error11:58
rlandy|roverare we waiting on something from a component line here?11:59
rlandy|roverdviroel|ruck: ^^ consistent failure since 08/1112:03
rlandy|roverchecking component line12:03
soniya29rlandy|rover, checking12:03
rlandy|roverhttps://review.rdoproject.org/zuul/builds?pipeline=openstack-component-tempest+12:03
rlandy|roverno scenario001 in that line12:04
dviroel|ruckack12:04
rlandy|roverakahat: hi - fyi - re: scenario010 https://trello.com/c/EDjoBcpf/2056-cixlp1940007tripleociproa-scenario10-and-scenario010-ovn-master-jobs-are-failing-standalone-deploy-failed-to-find-some-config-fi12:04
rlandy|rover^^ need that voting12:04
rlandy|roverysandeep: hey - you back today?12:05
akahatrlandy|rover, ack12:06
weshay|ruck0/12:09
weshay|ruckrlandy|rover, tomorrow12:09
rlandy|roverweshay|ruck: oh sandeep12:10
weshay|ruckya12:10
rlandy|roversorry downstream is very broken12:10
rlandy|rovertrying to elicit some help there :)12:10
weshay|ruckneed eyes?12:10
rlandy|roverthink we got the upstream blockers12:11
rlandy|roverdviroel|ruck is bugging the train scneraio001 one12:11
pojadhavchandankumar, thanks for the fix :)12:15
*** jpena|lunch is now known as jpena12:16
pojadhavmarios, https://review.rdoproject.org/r/c/rdo-jobs/+/34927/3#message-7def77578cbb7d18f99ba673839498c8a2ecb737 when you have time !12:17
dviroel|ruckrlandy|rover: weshay|ruck https://bugs.launchpad.net/tripleo/+bug/194007612:17
rlandy|roverdviroel|ruck: perfect - thanks12:17
dviroel|ruck:)12:17
rlandy|roverdviroel|ruck; feel free to ping soniya29 arxcruz to help work through this one12:18
weshay|ruckdviroel|ruck, can you troll through the errors file please12:18
weshay|ruckmatch time stamps.. etc12:18
dviroel|ruckthis one ?12:19
dviroel|ruckhttps://logserver.rdoproject.org/openstack-periodic-integration-stable4/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-train/0de27d7/logs/undercloud/var/log/extra/errors.txt.gz12:19
weshay|ruckdviroel|ruck, aye12:19
weshay|ruckakahat, master promoter is promoter.rdo?12:19
akahatweshay|ruck, yes.. migrated on friday.12:20
rlandy|roverakahat++ saw all logs there12:21
rlandy|roverthanks12:21
rlandy|roverpromotions going over the weekend12:21
rlandy|roverweshay|ruck: we have a legit master blocker12:21
rlandy|roverhad a patch to remove 10 and ovn from master criteria12:21
chandankumarzbr: one small cleanup https://review.opendev.org/c/openstack/tripleo-repos/+/804221/4#message-fdcf369f63222e3cc64ee8d82f9ed33676320513 then we are good to merge this12:22
rlandy|roverbut they were passing for some time12:22
rlandy|roverwill discuss at CIX12:22
weshay|ruckrlandy|rover, is it bugged?12:22
rlandy|roverweshay|ruck: acked and CIX'ed12:22
weshay|ruckrlandy|rover, perfect... thank you.. then for master all we need is fs3512:23
mariosack pojadhav 12:23
weshay|ruckit's in tempest now12:23
rlandy|roverweshay|ruck: ack - also updated that card in CIX12:23
rlandy|roverpatch in recheck for wallaby12:24
ysandeeprlandy|rover, I am back today..12:26
rlandy|roverysandeep: yay!! welcome back12:26
ysandeepchecking downstream12:27
rlandy|roverhow's the new daddy?12:27
rlandy|roverysandeep: yeah - lots of breakages there12:27
rlandy|roverI could use some help if you have the time12:27
rlandy|roverysandeep: downstream got hit by a bunch of things12:27
rlandy|roverad baremetal is also failing12:27
rlandy|rover16.2 and 17 both promoted last week12:28
rlandy|roverbut are down again12:28
weshay|ruckk.. will look in a few..12:28
rlandy|roverwill have to bring downstream back into ruck/rover cycyle12:28
ysandeepI am good now had enough sleep last night :) Will start checking now12:28
rlandy|roverysandeep++ thanks12:29
ysandeeprlandy|rover, should I start with 16.2 or 17, i guess 17 would be priority now.12:30
rlandy|roverysandeep: depends on the next import schedule12:31
rlandy|roverbut yeah if you can start looking at 1712:31
ysandeepack o/ on it12:31
weshay|ruckdviroel|ruck, join the cix call12:31
dviroel|ruckjoining, fixing mic issues12:32
rlandy|roverdviroel|ruck: you can get the honor of reporting on the next call on wednesday :)12:41
* weshay|ruck looks at 16.212:41
rlandy|roverweshay|ruck: ysandeep: thanks ... I'll take on the baremetal failures12:41
dviroel|ruckok. :)12:42
weshay|ruckrlandy|rover, ovb 2 timed out in tempest, 1 infra issue12:44
rlandy|roverugh - need baremetal back on its feet12:44
ysandeepweshay|ruck, rlandy|rover pojadhav fyi.. just created a hackmd to track downstream issues: https://hackmd.io/YW31LNNFQSm7PZKbYJZRzQ?both 12:45
weshay|ruckI think we can test project 001,035,2012:45
weshay|ruckhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2&project=openstack/tripleo-ci12:45
pojadhavysandeep, ack12:46
rlandy|roverysandeep: can we just use the ruck/rover chat?12:46
weshay|ruckhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-ovb-1ctlr_2comp-featureset020-internal-rhos-16.2&project=openstack/tripleo-ci12:46
rlandy|roverand label them downstream?12:46
rlandy|roverthoughts>12:46
weshay|ruckrlandy|rover, ya.. that is not a real public chat12:47
weshay|ruckit's invite only from rh email12:47
weshay|ruckfeel free12:47
rlandy|roverweshay|ruck:  that way ruc/rover can help when they have time as well12:47
rlandy|roverysandeep: pojadhav: ^^12:47
ysandeeprlandy|rover: Sounds good, I will add on ruck/rover chat if I need to pass some info. 12:47
rlandy|roverpls just label or start with DOWNSTREAM12:47
rlandy|roverso we can search or sort or something12:48
rlandy|roverwhatever label or identification works best12:48
rlandy|roverhttps://review.opendev.org/c/openstack/tripleo-common/+/80459112:49
rlandy|roverdviroel|ruck: could I get another vote there?12:49
rlandy|roverreplace upgrades12:49
rlandy|roverhttps://review.opendev.org/c/openstack/tripleo-common/+/80459212:50
rlandy|roverand here12:50
rlandy|roverchandankumar: ^^ thanks for recheck12:50
rlandy|roveradding ussuri and train review there12:50
weshay|ruckdviroel|ruck, added task for you for 16.212:50
weshay|ruckholler for help12:50
weshay|ruckrlandy|rover, 17 might be missing containers?12:51
rlandy|rovermaybe 12:52
rlandy|roverwasn't that red when I looked on friday12:52
rlandy|roverraised IPA issue12:52
rlandy|roverwhich I would have CIX'ed today12:52
rlandy|roverbut the line was so red, better fix that first12:53
weshay|ruckso far I see.. 2 container pull failures, 1 stuck in heat12:53
weshay|ruckrlandy|rover, we could kill the line and restart 1712:53
ysandeepweshay|ruck, 16.2 line trigger 52 mins back.. we may not need testproject12:53
weshay|ruckysandeep, true12:54
rlandy|roverhttps://review.opendev.org/c/openstack/tripleo-common/+/80459212:56
rlandy|roverweshay|ruck: ^^ pls vote12:56
rlandy|roverhttps://review.opendev.org/c/openstack/tripleo-common/+/80459112:56
rlandy|roverw+ pls12:56
dviroel|ruckweshay|ruck: need help with that, can you give a link with an example?13:00
bhagyashrisakahat, dviroel|ruck rlandy|rover pojadhav soniya29 scrum time13:00
pojadhavbhagyashris, joined13:00
bhagyashriszbr, ^13:02
zbri will be few minutes late, i a another meeting.13:06
zbrplease startw w/o me13:06
zbrdid our scrum meeting already finished?13:22
ysandeepzbr, yes we ended early today13:22
zbrok, no problem.13:22
weshay|ruck035 master timed out13:25
rlandy|roverzbr: will touch base with you in a few re: scrum13:28
zbrrlandy|rover: i already had a talk earlier today with marios and chandan about tripleo.repos work.13:35
rlandy|roverzbr: it's unrelated ... https://meet.google.com/pww-pseu-smt?pli=1&authuser=0 - just for 5 minutes14:08
weshay|ruckrlandy|rover, fyi https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34961 https://review.rdoproject.org/r/c/testproject/+/34960 master 14:41
rlandy|roverhttps://review.rdoproject.org/r/c/testproject/+/34960/1/.zuul.yaml14:42
rlandy|roverweshay|ruck: ^^ missing colon14:42
rlandy|roverline 414:42
rlandy|roverweshay|ruck: wrt scenario010 ... you want to promote up w/o fix https://review.opendev.org/c/openstack/tripleo-heat-templates/+/804732?14:44
weshay|ruckrlandy|rover, let's check back later.. we need that to merge or fs035 to pass14:44
rlandy|roverweshay|ruck: I will approve that if /804732 show scneario010 and ovn passing14:45
rlandy|roverwe don;t want to regress14:45
rlandy|roverI don;t think we have a master revert for fs03514:45
rlandy|roverchecking14:45
weshay|ruckrlandy|rover, feel free to create one and and depends-on the test project15:02
rlandy|roverhttps://zuul.openstack.org/stream/39f67e50b2264eeea1949d5c47fd5c5d?logfile=console.log15:09
rlandy|roverweshay|ruck: scenario010 is going to fail15:09
rlandy|rovershoot15:09
weshay|ruckmeh.. may be a tempest hiccup15:12
rlandy|roverwould prefer not to remove fro criteria15:13
rlandy|roverwe are getting quick fixes15:13
weshay|ruckk15:15
ysandeepweshay|ruck, rlandy|rover fyi.. got what was wrong with 17 baremetal fs001 - networkdatav2 overcloud-networks-deployed.yaml and vip_data.yaml env files are generated but not passed in deploy command. 15:32
rlandy|roverysandeep: yep - was following that up last week when other explosions hit15:33
rlandy|roverysandeep: we missed you!!15:33
ysandeeprlandy|rover, me too missed the team, I will try to fix fs001 baremetal 17 tomorrow.. 15:35
rlandy|roverysandeep: sure - that's fine15:36
ysandeeprlandy|rover, and containers failing to pull was temporary for 17.. I ran a testproject and it passed that step https://code.engineering.redhat.com/gerrit/c/testproject/+/21149415:36
rlandy|rovergood15:37
rlandy|roverone less problem15:37
rlandy|roverysandeep: ipa job was failing15:37
rlandy|roverI raised that to the security team15:37
rlandy|roverif it's still an issue we can BZ it and dump it at their dorr15:38
rlandy|roverdoor15:38
rlandy|roverit was our agreement that they own that job15:38
ysandeeprlandy|rover, yes its still failing 15:38
rlandy|roverk15:39
rlandy|roverhand it over15:39
ysandeepSure, I will bz it tomorrow15:40
*** ysandeep is now known as ysandeep|away15:40
rlandy|roverweshay|ruck: when 804732 merges will testproject the missing sc010 and ovn jobs in master15:43
rlandy|roverto promote faster15:43
weshay|ruck++15:43
* rlandy|rover looks at train blocker15:44
rlandy|roverk - Tengu is on it15:45
Tengume?15:50
Tengushould I boot up my work laptop to push some votes?15:50
rlandy|roverTengu: https://bugs.launchpad.net/tripleo/+bug/194007615:52
rlandy|roverlooks like you commented there - thank you15:52
Tenguoh, THAT one. yeah15:52
TenguI got a ping from Jan for the triaging, and took some time to investigate.15:52
Tengunot sure it has any actual value though, but... well. a failed container, and those permission issues are probably not that nice15:53
Tenguthe CIX has been awarded to Storage following those findings.15:53
rlandy|roverk- awesome 15:53
rlandy|roveras long as someone has it15:53
Tengumaybe pinging actual ppl might help15:53
Tengunot sure who's on Glance (failed container) or Cinder (permission denials)15:54
*** jpena is now known as jpena|off15:58
*** marios is now known as marios|out16:01
rlandy|roverweshay|ruck: need to step away now16:41
weshay|ruckk16:41
rlandy|roverwill be in and out16:41
*** rlandy|rover is now known as rlandy|rover|afk16:41
*** dviroel|ruck is now known as dviroel|out17:49
*** rlandy|rover|afk is now known as rlandy|rover18:18
rlandy|roverweshay|ruck: back18:20
weshay|ruckrlandy|rover, k18:22
rlandy|roverrerunning scenario010 when it hits current18:44
« Friday, 2021-08-13 Index Tuesday, 2021-08-17 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!