Tuesday, 2021-06-01

*** yamamoto has joined #oooq00:12
*** jbadiapa has quit IRC00:21
*** yamamoto has quit IRC00:28
*** yamamoto has joined #oooq00:31
*** jbadiapa has joined #oooq00:34
*** yamamoto has quit IRC01:00
*** yamamoto has joined #oooq01:01
*** yamamoto has quit IRC01:01
*** yamamoto has joined #oooq01:16
*** yamamoto has quit IRC01:24
*** yamamoto has joined #oooq01:50
*** yamamoto has quit IRC02:09
*** zbr has quit IRC02:22
*** ysandeep has joined #oooq02:23
*** zbr has joined #oooq02:34
*** ysandeep has quit IRC02:41
*** zbr is now known as Guest41302:49
*** zbr has joined #oooq02:49
*** zbr is now known as Guest41402:53
*** zbr has joined #oooq02:53
*** Guest413 has quit IRC02:53
*** Guest414 has quit IRC02:57
*** zbr has quit IRC03:01
*** zbr has joined #oooq03:02
*** yamamoto has joined #oooq04:11
*** frenzy_friday has quit IRC04:25
*** frenzy_friday has joined #oooq04:25
*** ysandeep has joined #oooq04:46
*** ysandeep is now known as ysandeep|ruck05:01
*** marios has joined #oooq05:14
*** slaweq has joined #oooq06:17
*** bhagyashris has joined #oooq06:34
*** bhagyashris has quit IRC06:35
*** bhagyashris has joined #oooq06:35
*** bhagyashris has quit IRC06:35
*** bhagyashris has joined #oooq06:35
*** bhagyashris_ has joined #oooq06:37
*** bhagyashris has quit IRC06:37
*** bhagyashris_ has quit IRC06:37
*** bhagyashris has joined #oooq06:37
*** jbadiapa has quit IRC07:02
*** jbadiapa has joined #oooq07:02
*** amoralej|off has joined #oooq07:16
*** amoralej|off is now known as amoralej07:19
*** jpena|off is now known as jpena07:26
arxcruznamaste ysandeep|ruck07:32
*** arxcruz is now known as arxcruz|rover07:32
ysandeep|ruckarxcruz|rover, hello very good morning07:34
arxcruz|roverysandeep|ruck: so, my patrch did not work after all07:34
arxcruz|roverjust saw yours07:34
arxcruz|roveri assume yours works right ?07:34
ysandeep|ruckarxcruz|rover, yes it works.. tested in testproject: https://logserver.rdoproject.org/46/28446/41/check/periodic-tripleo-ci-build-containers-ubi-8-push-victoria/171f1d2/etc/yum.repos.d/CentOS-Stream-AppStream.repo07:35
arxcruz|rovergreat, let's merge it asap07:35
ysandeep|ruckWe are hitting bug: https://bugs.launchpad.net/tripleo/+bug/193032707:35
opendevmeetLaunchpad bug 1930327 in tripleo "Container build is failing with error: /usr/share/ovn/scripts/ovndb-servers.ocf from install of ovn-2021-21.03.0-21.el8s.x86_64 conflicts with file from package ovn2.13-20.12.0-85.el8s.x86_64" [Critical,Triaged]07:35
arxcruz|rovermarios: https://review.opendev.org/c/openstack/tripleo-ci/+/793815 i know you already +2 but would it be possible to +w even without a second core? we are in a hurry and it's blocking us07:35
ysandeep|ruckworked that out ykarel.. we are awaiting for upstream mirror sync07:36
ysandeep|ruckarxcruz|rover: we need to wait, we have -1 from zuul because of above bug07:36
arxcruz|roveroh07:36
arxcruz|rover:/07:36
ysandeep|ruckarxcruz|rover, comment#3 of bug have more details07:37
mariosarxcruz|rover: ack but  zuul is -1 on that07:37
ysandeep|ruckmarios, thanks, yes we need to wait for upstream infra mirror sync.07:38
ysandeep|ruckarxcruz|rover, fyi.. i noticed some interesting conversation around msr issue from last thursday's chat http://eavesdrop.openstack.org/irclogs/%23tripleo/%23tripleo.2021-05-27.log.html#t2021-05-27T15:51:1007:41
ysandeep|ruckarxcruz|rover: I am thinking about if we can virt-customize the overcloud-full image and set root password.07:43
ysandeep|ruckAnd when issue appears, we take console access of overcloud node which failed and see why ping/ssh not working.07:43
*** ysandeep|ruck is now known as ysandeep|lunch07:47
* ysandeep|lunch afk for lunch07:47
*** tosky has joined #oooq07:48
*** tosky has quit IRC07:48
*** tosky has joined #oooq08:08
*** derekh has joined #oooq08:14
*** ysandeep|lunch has quit IRC08:16
*** ysandeep|lunch has joined #oooq08:46
*** ysandeep|lunch is now known as ysandeep|ruck08:46
*** pojadhav has joined #oooq09:22
zbrwhat is the official channel of quay?09:27
zbr(network)09:27
*** sshnaidm|off is now known as sshnaidm09:57
*** sshnaidm has joined #oooq09:58
*** yamamoto has quit IRC10:11
*** yamamoto has joined #oooq10:28
*** yamamoto has quit IRC10:52
*** ykarel has joined #oooq11:04
*** yamamoto has joined #oooq11:26
*** bhagyashris_ has joined #oooq11:29
*** jpena is now known as jpena|lunch11:32
*** bhagyashris has quit IRC11:36
*** yamamoto has quit IRC11:42
*** rlandy_ has joined #oooq11:45
*** yamamoto has joined #oooq11:49
*** sshnaidm has quit IRC11:51
*** dviroel has joined #oooq11:57
*** sshnaidm has joined #oooq11:59
*** jbadiapa has quit IRC12:07
*** jbadiapa has joined #oooq12:09
weshay|ruckysandeep|ruck, arxcruz|rover 0/ in a mtg.. but we should catch up before you folks disappear12:11
ysandeep|ruckweshay|ruck, :) sure i am here for couple of hours..12:11
arxcruz|roverysandeep|ruck: weshay|ruck when you guys want :)12:15
weshay|ruckhttps://review.rdoproject.org/r/c/rdoinfo/+/3392412:17
weshay|ruckwe need this now eh?12:17
arxcruz|roverweshay|ruck: yes, and the fix on ubi8 push containers job12:18
ysandeep|ruckweshay|ruck, we have removed ovn-2021 for now, mirror sync will solve that (next mirror sync ~1.30 hr)12:19
weshay|ruck++12:20
*** rlandy_ is now known as rlandy12:22
*** amoralej is now known as amoralej|lunch12:25
*** yamamoto has quit IRC12:26
*** jpena|lunch is now known as jpena12:29
weshay|ruckarxcruz|rover, ysandeep|ruck we'll need to engage Amol this week and promote some branches w/o ovb12:30
weshay|ruckseems like master is alive again.. let's focus guns there12:30
rlandyysandeep|ruck: arxcruz|rover: https://review.opendev.org/c/openstack/tripleo-quickstart/+/792818 ... fyi12:31
rlandypls review12:31
ysandeep|ruckweshay|ruck, ack o/ yesterday finally we solved fs39 issues..12:33
ysandeep|ruckweshay|ruck, Earlier tripleo-repos didn't fix t/u/v container build https://review.opendev.org/c/openstack/tripleo-repos/+/793157 didn't help, I have posted:  https://review.opendev.org/c/openstack/tripleo-ci/+/79381512:33
arxcruz|roverrlandy: is this the reason we still seeing the edk212:33
ysandeep|ruckrlandy: looking12:33
rlandyarxcruz|rover: ^^ should only impact component jobs12:33
*** yamamoto has joined #oooq12:34
ysandeep|ruckrlandy, +112:34
rlandyysandeep|ruck: weshay|ruck: updated environment ... https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/24284412:38
ysandeep|ruckrlandy, ack, added in my review list12:39
rlandyysandeep|ruck: one more .. still cleaning up ... but we have the list of updated containers ...12:39
rlandyhttps://logserver.rdoproject.org/53/18953/64/check/periodic-tripleo-ci-centos-8-standalone-network-master/4d812da/logs/undercloud/home/zuul/container_image_update_output.log.txt.gz12:40
rlandyysandeep|ruck: ^^12:40
rlandyso we can see if no containers get updated12:40
rlandyhttps://review.opendev.org/c/openstack/ansible-role-tripleo-modify-image/+/793234/12:40
rlandycleaning that  up to make sure it's mergeable12:40
ysandeep|ruckrlandy: nice12:41
rlandythat should be all the  reviews for container updates12:41
ysandeep|ruckrlandy++ i really like the buildah mount idea there12:43
*** ysandeep|ruck is now known as ysandeep|brb12:43
rlandyysandeep|ruck: yeah - tried a bunch of things - that was the only one that worked12:43
* ysandeep|brb brb in 10-15 mins12:43
*** tbarron has quit IRC12:52
*** amoralej|lunch is now known as amoralej12:57
*** ysandeep|brb is now known as ysandeep|ruck12:57
ysandeep|ruckrlandy, weshay|ruck fyi.. downstream is in pretty bad shape.. tempest failure.. ssh to instance failing.. slaweq helped and we trying new ovn to see if it fix the issue .. https://code.engineering.redhat.com/gerrit/c/testproject/+/21164312:58
rlandyysandeep|ruck: k- will pick that up12:58
ysandeep|ruckweshay|ruck, 1:1 time if you are out of mtgs13:01
weshay|ruckya.. need 2 min13:02
*** tbarron has joined #oooq13:07
bhagyashris_arxcruz|rover, , zbr, sshnaidm, rlandy, marios, ysandeep|ruck , bhagyashris, svyas, soniya29, pojadhav, akahat, weshay|ruck , chandankumar, frenzy_friday13:20
bhagyashris_TripleO CI Community Meeting13:20
bhagyashris_in 10 mins13:20
bhagyashris_feel free to add topics here https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2021-06-01-Community-call @line no 3013:21
zbrbhagyashris_: my calendar entry for community meeting still has bluejeans url, is this valid?13:22
*** bhagyashris_ is now known as bhagyashris13:22
bhagyashriszbr nop13:23
bhagyashrisnope13:23
bhagyashriszbr see in the invite you declined the community meeting13:24
bhagyashrisi mean it's showing13:24
bhagyashrisAutomatically declined due to event conflict13:24
bhagyashristhis is our gmeet url: https://meet.google.com/bqx-xwht-wky?authuser=013:26
rlandysshnaidm: when did we start running podman/buildah? train 8?13:33
sshnaidmeven in 7 I think13:34
* rlandy wants to check if we need the dnf/yum switch for container updates13:34
sshnaidmyum should be symlink to dnf in 8 I think?13:35
*** jlarriba has quit IRC13:37
rlandyhttps://review.opendev.org/c/openstack/ansible-role-tripleo-modify-image/+/793234/24/tasks/yum_update_buildah.yml#15813:37
rlandysshnaidm: ^^13:37
rlandyusing \$(command -v dnf || command -v yum) isn't working13:37
rlandyfor the bash -c13:37
rlandymaybe using yum is safe13:37
rlandyfor all13:38
*** jlarriba has joined #oooq13:50
weshay|ruckarxcruz|rover, https://opendev.org/openstack/ansible-role-collect-logs/src/branch/master/roles/collect_logs/defaults/main.yml#L41413:58
*** gouthamr_ has joined #oooq14:35
*** gouthamr has joined #oooq14:47
*** dviroel is now known as dviroel|lunch15:04
*** ykarel is now known as ykarel|away15:09
*** gouthamr_ has quit IRC15:12
*** gouthamr has quit IRC15:13
*** gouthamr has joined #oooq15:14
*** gouthamr is now known as identify15:18
*** gouthamr has joined #oooq15:20
*** ykarel|away has quit IRC15:20
*** identify has quit IRC15:20
weshay|ruckarxcruz|rover, got the same thing.. not bootable15:37
weshay|ruckarxcruz|rover, I think you need to add the kernel from ipa image15:40
* weshay|ruck tries15:40
arxcruz|roverweshay|ruck: ok15:40
weshay|ruckarxcruz|rover, that dumped me into emergency mode15:47
weshay|ruckarxcruz|rover, nothing is easy right?15:47
arxcruz|roverweshay|ruck: same here15:47
arxcruz|roverwith the initramfs from overcloud-full got emergency mode15:47
arxcruz|roverwith the ipa one i got kernel panic15:47
weshay|ruckarxcruz|rover, I wonder if this would be easier.. if we uploaded the image to openstack and tried to boot that way15:49
weshay|ruckbut.. meh15:49
weshay|ruckchanging the passwd should be easy enough15:49
weshay|ruckarxcruz|rover, let's short cut this15:50
arxcruz|roverweshay|ruck: i did it!15:50
weshay|ruckarxcruz|rover, you got it?15:50
arxcruz|roverweshay|ruck: need to pass root=/dev/sda115:50
arxcruz|roverin the kernel parameters15:50
* weshay|ruck tries15:50
weshay|ruckI wonder what the default is.. /dev/sdb15:51
* ysandeep|ruck reading email weshay|ruck talked about15:51
weshay|ruckarxcruz|rover, so you specified the kernel path, initd and kernel args?15:56
weshay|ruckand got to a login?15:56
arxcruz|roverweshay|ruck: yes, i got into the login, but isn't working15:56
arxcruz|roverthe password says it's incorrect15:57
weshay|ruckah.. k.. well.. glad the local boot is working to check15:57
ysandeep|rucki think we just need to virt-customize overcloud-full16:00
*** marios has quit IRC16:02
arxcruz|roverweshay|ruck: is the user root ?16:06
arxcruz|roverysandeep|ruck: i did the virt-customize on overcloud-full16:06
ysandeep|ruckarxcruz|rover, ack o/16:06
*** ysandeep|ruck is now known as ysandeep|dinner16:06
*** ykarel|away has joined #oooq16:11
*** dviroel|lunch is now known as dviroel16:11
weshay|ruckarxcruz|rover, ya.. we want root16:13
weshay|ruckarxcruz|rover, can you INTERNALLY paste the virt customize the command http://pastebin.test.redhat.com/16:14
arxcruz|roverweshay|ruck: i'm using virt manager16:14
arxcruz|roveryou want me share my screen with you?16:14
arxcruz|roverit's weird because the virt-cat is showing the root password changed16:14
weshay|rucksure16:17
weshay|ruckarxcruz|rover, meet.google.com/kox-wmmr-axv16:18
weshay|ruckarxcruz|rover, ysandeep|dinner we need to update the zuul config to launch a container build job from tripleo-repos https://review.opendev.org/c/openstack/tripleo-ci/+/79376316:20
* weshay|ruck adds16:20
*** ykarel|away has quit IRC16:26
rlandyarxcruz|rover: sorry to side track you ... question wrt https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/tasks/python_venv_install.yml#L13016:35
rlandyhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c53/792474/3/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/c5353ad/job-output.txt16:35
rlandydebugging that last ipa failure16:35
rlandywould that step use https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c53/792474/3/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/c5353ad/logs/undercloud/etc/pip.conf16:36
rlandynot resolving the source to upgrade16:36
rlandy149.202.160.10 mirror.gra1.ovh.opendev.org is added to hosts16:37
rlandyhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_c53/792474/3/check/tripleo-ci-centos-8-standalone-on-multinode-ipa/c5353ad/logs/undercloud/etc/hosts16:37
rlandyoh nvm16:39
rlandyreleases.openstack.org16:39
rlandyarxcruz|rover: ^^ sorry - figured it out16:40
*** jpena is now known as jpena|off16:47
*** ysandeep|dinner is now known as ysandeep16:59
ysandeepweshay|ruck, https://review.opendev.org/c/openstack/tripleo-heat-templates/+/793968 this fixes sc03 temporarily..17:00
*** derekh has quit IRC17:00
ysandeepweshay|ruck, rlandy looks like new ovn package fixes the issue: https://code.engineering.redhat.com/gerrit/c/testproject/+/21164317:05
rlandyysandeep: nice17:06
ysandeepweshay|ruck, rlandy: wondering if we can merge this as workaround for the meantime(till rel-del bring this via proper channel): https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/244293 .. i can include same workaround in 17 release files as well17:10
rlandyysandeep: weshay|ruck: hmmm ... working around repos is not ideal17:18
rlandywe're covering up a real mismatch17:19
ysandeeprlandy: yeah let's rel-del fix this properly via rhos-release update17:19
rlandyysandeep: will see when jason returns17:20
ysandeepi will fire more jobs in testproject just to make sure i was not lucky and this really fixes the issue.17:20
rlandyok17:22
*** amoralej is now known as amoralej|off17:26
ysandeepi have updated the trello card to reflect current status17:32
weshay|ruckarxcruz|rover, https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-buildimage-overcloud-full-master/b3ab4c0/build_images.sh17:45
ysandeephttps://review.opendev.org/c/openstack/tripleo-heat-templates/+/793968 - rechecked sc03 temp fix.. failed on random tempest.. badhostkey17:53
ysandeepweshay|ruck, would appreciate some eyes on ^^17:54
ysandeepand this one as well https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/79290417:54
weshay|ruckysandeep, k17:54
weshay|ruckarxcruz|rover, https://logserver.rdoproject.org/68/793468/3/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/9808039/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz17:55
* ysandeep going to bed(almost mid night) see you tomorrow folks o/ 17:56
*** ysandeep is now known as ysandeep|away17:56
weshay|ruckysandeep, that hit a tempest issue.. timeout17:56
* weshay|ruck rechecks17:56
ysandeep|awayi have also posted recheck, few mins back.. :)17:57
*** ysandeep|away has quit IRC18:01
rlandymsg": "Failed to download metadata for repo 'RDO-CentOS8-deps': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried",18:11
rlandyugh18:11
rlandyhow are we dong that downstream?18:11
rlandydoing18:11
rlandyweshay|ruck: you all set with the images?18:42
weshay|ruckrlandy, ya.. we thought we were on to something, but we werent18:54
rlandyweshay|ruck: k18:55
weshay|ruckrlandy, ok.. think I'm caught up finally.. that was a morning19:14
rlandyweshay|ruck: wow - sure was19:14
rlandyweshay|ruck: are we desperate to promote 16.2?19:16
weshay|ruckhrm..19:18
* weshay|ruck looks19:18
weshay|ruckrlandy, was jjoyce fixing rhos-release?19:19
rlandyweshay|ruck: we are discussing now on #rhos-delivery19:20
*** jlarriba has quit IRC20:07
*** jlarriba has joined #oooq20:08
*** rlandy has quit IRC21:17
*** rlandy has joined #oooq21:18
*** tosky has quit IRC23:11
*** frenzy_friday has quit IRC23:42
*** frenzy_friday has joined #oooq23:43

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!