Thursday, 2020-12-10

« Wednesday, 2020-12-09 Index Friday, 2020-12-11 »
*** tosky has quit IRC00:02
*** Goneri has joined #oooq00:02
*** rfolco has joined #oooq00:26
*** Goneri has quit IRC00:29
*** rfolco has quit IRC01:17
*** ysandeep|away is now known as ysandeep01:54
*** jmasud has joined #oooq02:31
*** jmasud has quit IRC02:34
*** apetrich has quit IRC03:19
*** ysandeep is now known as ysandeep|session03:19
*** zbr has quit IRC03:57
*** zbr has joined #oooq03:58
*** ratailor has joined #oooq04:54
*** skramaja has joined #oooq05:06
*** skramaja has quit IRC05:11
*** skramaja has joined #oooq05:29
*** udesale has joined #oooq05:30
*** marios has joined #oooq06:02
*** marios is now known as marios|rover06:03
*** saneax has joined #oooq06:17
*** ysandeep|session is now known as ysandeep06:28
*** jfrancoa has joined #oooq06:28
*** marios|rover has quit IRC06:37
*** ysandeep is now known as ysandeep|sick07:03
*** slaweq has joined #oooq07:18
*** marios has joined #oooq07:18
*** marios is now known as marios|rover07:19
slaweqchkumar|ruck: hi07:40
slaweqchkumar|ruck: thx for rechecking my test patch07:41
slaweqbut I think that there is some another issue there https://review.rdoproject.org/zuul/build/ef77430dd96e4c55a565fbf8c14c7ceb07:41
slaweqdid You saw that already?07:41
chkumar|ruckslaweq: yes current, I am still investigating what is wrong there07:41
slaweqchkumar|ruck: ok, thx07:41
slaweqplease let me know if You will find anything07:42
*** jbadiapa has joined #oooq07:48
*** jpena|off is now known as jpena08:08
chkumar|rucksshnaidm: zbr hello, please have a look when free https://bugs.launchpad.net/tripleo/+bug/190754408:09
openstackLaunchpad bug 1907544 in tripleo "Rename compressed text based files to end with txt.gz extension getting skipped on rdo jobs" [Critical,Triaged]08:09
marios|roverchkumar|ruck: thank you! i was wondering about that it is very annoying08:10
chkumar|ruckah you are back, :-)08:10
marios|roverchkumar|ruck: yeah.. this is a bit of a mess https://review.opendev.org/q/topic:%22tripleo-lower%22+(status:open%20OR%20status:merged) gonna take ages to merge all the things (+cherrypicks back to train)08:10
chkumar|ruckmarios|rover: https://review.opendev.org/c/openstack/tripleo-ci/+/766208 need to merge this one to unblock https://review.opendev.org/c/openstack/tripleo-ci/+/76599608:12
marios|roverchkumar|ruck: thanks08:12
*** apetrich has joined #oooq08:21
*** marios|rover has quit IRC08:25
*** marios|rover has joined #oooq08:34
*** tosky has joined #oooq08:47
marios|roverchkumar|ruck: need help with something?08:54
chkumar|ruckmarios|rover: right now nope08:54
marios|roverchkumar|ruck: ack08:55
chkumar|ruckmarios|rover: please have a look https://bugs.launchpad.net/tripleo/+bug/1907544 when free08:55
openstackLaunchpad bug 1907544 in tripleo "Rename compressed text based files to end with txt.gz extension getting skipped on rdo jobs" [Critical,Triaged]08:55
marios|roverchkumar|ruck: ack ok08:55
*** derekh has joined #oooq08:58
*** zbr has quit IRC09:10
*** zbr has joined #oooq09:11
*** zbr has quit IRC09:13
*** zbr has joined #oooq09:13
akahatchkumar|ruck, ping.. i need help with zuul directory..09:25
akahatchkumar|ruck, this is log: https://logserver.rdoproject.org/14/28014/30/check/tripleo-ci-promotion-staging-integration-pipeline-centos-8/b867b8f/job-output.txt09:25
akahatso i want to pull file from target machine to controller.. what exact variable i need to use here: https://review.rdoproject.org/r/#/c/28014/30/ci-scripts/infra-setup/roles/promoter/tasks/promotion_run.yml09:26
chkumar|ruckakahat: can you share the patch?09:35
*** zbr has quit IRC09:35
akahatchkumar|ruck,  https://review.rdoproject.org/r/#/c/28014/30/ci-scripts/infra-setup/roles/promoter/tasks/promotion_run.yml09:35
chkumar|ruckakahat: gmeet?09:37
*** zbr has joined #oooq09:37
*** zbr has quit IRC09:40
*** zbr has joined #oooq09:40
marios|roverchkumar|ruck: any idea why we are seeing openstack/heat patches in the results at http://dashboard-ci.tripleo.org/d/Z4vLSmOGk/cockpit?orgId=1&fullscreen&panelId=6109:48
marios|rover766178  is heat review09:49
chkumar|ruckmarios|rover: no idea, need to investigate09:49
marios|roverchkumar|ruck: ack not huge issue was just momentarily confused and wondering if there was some change... not sure if there are more than that one patch yet09:50
marios|roverchkumar|ruck: well, it could be a huge issue if that is going into the stats though...09:50
*** zbr has quit IRC09:54
*** zbr has joined #oooq09:56
*** udesale_ has joined #oooq10:05
*** udesale has quit IRC10:08
marios|roverchkumar|ruck: review @ https://review.opendev.org/c/openstack/tripleo-heat-templates/+/766404 for ovb when you have time please (https://bugs.launchpad.net/tripleo/+bug/1907272/comments/5 )10:22
openstackLaunchpad bug 1907272 in tripleo "rdo ovb featureset001 fails baremetal provisioning Permission denied: '/var/lib/ironic/images/" [Critical,Triaged]10:22
chkumar|ruckmarios|rover: thanks!10:35
marios|roverchkumar|ruck: new one fyi https://bugs.launchpad.net/tripleo/+bug/190765110:36
openstackLaunchpad bug 1907651 in tripleo "rdo ovb featureset039 fails during freeipa setup - could not open java-1.8.0 " [Critical,Triaged]10:36
*** zbr has quit IRC10:54
*** zbr has joined #oooq10:56
chkumar|ruckmarios|rover: I am taking care of freeipa one, jpena added some points there10:58
marios|roverchkumar|ruck: ack10:58
chkumar|ruckjpena: marios|rover https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76645211:09
chkumar|ruckjpena: we need to carry the workaround till it fixes https://bugzilla.redhat.com/show_bug.cgi?id=189221611:10
openstackbugzilla.redhat.com bug 1892216 in java-1.8.0-openjdk "openjdk z-stream broke freeipa installation" [Urgent,Verified] - Assigned to ahughes11:10
marios|roverchkumar|ruck: ack11:14
marios|roverdanpawlik: o/ hey re https://bugs.launchpad.net/tripleo/+bug/1907544/comments/3 .. for the rdo jobs it is already fixed but that is for internal jobs right?11:15
openstackLaunchpad bug 1907544 in tripleo "Rename compressed text based files to end with txt.gz extension getting skipped on rdo jobs" [Critical,Triaged]11:15
marios|roverdanpawlik: i mean, did you fix it already for the review.rdoproject.org jobs already? or that is the fix for it @ https://softwarefactory-project.io/r/#/c/20299/11:16
danpawlikmarios|rover: it should be for all jobs11:16
marios|roverdanpawlik: ack ok thanks11:16
marios|roverdanpawlik: i wasn't sure if you already did something else11:16
danpawlikmarios|rover: Let me check if some job was ongoing and that config was changed11:17
marios|roverdanpawlik: np its ok thanks11:17
danpawlikmarios|rover: ah, no. Thats all. It should be fine11:17
*** holser has quit IRC11:30
*** holser has joined #oooq11:33
*** dtantsur|afk is now known as dtantsur11:33
*** holser has quit IRC11:35
*** holser has joined #oooq11:35
marios|roverchkumar|ruck: we got a new master gate issue collecting logs & filing now11:47
marios|roverchkumar|ruck: wew ... this week... come on11:47
marios|roverchkumar|ruck: eg https://7a89c7d607922873abcd-a2a9d29289e3c443a1047aebafc6ba1b.ssl.cf2.rackcdn.com/765964/4/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/f02eecd/logs/undercloud/var/log/tripleo-container-image-prepare.log11:47
*** rfolco has joined #oooq11:54
*** zbr has quit IRC11:58
*** zbr has joined #oooq11:59
marios|roverchkumar|ruck: https://bugs.launchpad.net/tripleo/+bug/190765711:59
openstackLaunchpad bug 1907657 in tripleo "various jobs failing in the gate during container image prepare 404 not found" [Critical,Triaged]11:59
marios|roverchkumar|ruck: having fun yet?12:00
marios|roverchkumar|ruck: we'll clear out the gates today for sure !12:00
* chkumar|ruck runs out12:00
* marios|rover runs over chkumar|ruck running out12:00
chkumar|ruckhahaha12:01
*** skramaja has quit IRC12:05
chkumar|ruckmarios|rover: it is not related to the cloud12:05
marios|roverchkumar|ruck: no sorry i should have said that on the bug... i initially thought it was vexx only but no i saw at least 2 different clouds12:06
chkumar|ruckmarios|rover: I checked all the logs to confirm whether it is linked to a cloud with specific reason but it is not12:06
*** ratailor has quit IRC12:08
*** zbr has quit IRC12:10
*** zbr has joined #oooq12:10
*** zbr has quit IRC12:18
*** zbr has joined #oooq12:20
chkumar|ruckmarios|rover: do we capture the registry logs of content provider somewhere?12:26
chkumar|ruckmay be it can tell what is happening there12:26
marios|roverchkumar|ruck: not aware of something like that... i mean we capture the build logs... maybe in the journal worst case12:30
sshnaidmchkumar|ruck, marios|rover commented on bug12:35
*** jpena is now known as jpena|lunch12:35
chkumar|rucksshnaidm: thanks12:36
chkumar|rucksshnaidm: need some help on fixing this bug https://bugs.launchpad.net/tripleo/+bug/190747412:36
openstackLaunchpad bug 1907474 in tripleo "CentOS8-stream: tempest fails to load tests to be skipped: "The error was: KeyError: 'getpwuid(): uid not found: 1000'"" [High,Triaged] - Assigned to chandan kumar (chkumar246)12:36
marios|roverthanks sshnaidm12:36
*** rfolco has quit IRC12:38
*** rfolco has joined #oooq12:42
chkumar|ruckrfolco: weshay|ruck: marios|rover I will be skipping scrum, will back in 2 hr 30 mins12:51
marios|roverchkumar|ruck: ack12:52
*** rlandy has joined #oooq12:59
*** zbr has quit IRC13:01
weshay|ruckchkumar|ruck, marios|rover what's up w/ https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable313:02
weshay|ruckretry_limits I guess :(13:02
*** zbr has joined #oooq13:03
marios|roverrfolco: am rucking so will drop13:06
marios|roverrfolco: let me know if you need i will rejoin13:06
marios|roverweshay|ruck: o/ i haven't checked promotion pipelines yet been focusing upstream cos our % are getting hammered13:07
marios|roverweshay|ruck: master <70 not only cos of the constraints thing but a lot of gate jobs hitting new issue today https://bugs.launchpad.net/tripleo/+bug/190765713:08
openstackLaunchpad bug 1907657 in tripleo "various jobs failing in the gate during container image prepare 404 not found" [Critical,Triaged]13:08
marios|roverweshay|ruck: another new thing there master promotion blocker fs39 @ https://bugs.launchpad.net/tripleo/+bug/1907651 (but chkumar has a patch https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/766452 )13:08
openstackLaunchpad bug 1907651 in tripleo "ovb featureset039 check & periodic fails during freeipa setup - could not open java-1.8.0" [Critical,Triaged]13:08
*** sshnaidm has quit IRC13:09
marios|roverweshay|ruck: so periodic3 is hitting ... sec13:09
*** sshnaidm has joined #oooq13:09
rlandymarios|rover: weshay|ruck: have you seen no logs in rdocloud jobs - or is that just the stream job? https://review.rdoproject.org/r/#/c/31165/13:09
rlandyno logs on undercloud13:09
marios|roverrlandy: not seen that one13:09
marios|roverrlandy: we did have a logs related issue though but not 'no logs' https://bugs.launchpad.net/tripleo/+bug/190754413:10
openstackLaunchpad bug 1907544 in tripleo "Rename compressed text based files to end with txt.gz extension getting skipped on rdo jobs" [Critical,Triaged]13:10
marios|roverweshay|ruck: so https://bugs.launchpad.net/tripleo/+bug/1907193/comments/6 this is what is hitting 15:02 < weshay|ruck> chkumar|ruck, marios|rover what's up w/ https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable313:11
openstackLaunchpad bug 1907193 in tripleo "package podman-1.6.4-23.module_el8.3.0+566+4759265c.x86_64 is filtered out by modular filtering" [High,Fix released] - Assigned to Alex Schultz (alex-schultz)13:11
marios|roverweshay|ruck: eg there https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-ubi-8-push-train/a31a094/job-output.txt13:12
weshay|ruckzbr, any insight into what changed w/ zuul logs .. w/ regards to rendering log files.  I'm seeing just about everything require a download13:21
weshay|ruckmarios|rover, ah nice13:21
*** zbr has quit IRC13:21
weshay|ruckmarios|rover, do we have a boog on ary |   "msg": "Depsolve Error occured: \n Problem: package tripleo-ansible-0.5.1-0.20201204163449.c4641f4.el8.noarch requires python3-paunch, but none of the providers can be installed\n  - package python3-paunch-5.3.3-0.20201114155407.7e49691.el8.noarch requires podman = 1.6.4, but none of the providers can be installed\n  - conflicting requests\n  - package podman-1.6.4-23.module_el8.3.0+566+4759265c.x13:22
weshay|ruck86_64 is filtered out by modular filtering",13:22
weshay|ruckrlandy, we need to unpin podman13:22
weshay|ruckfor train13:22
rlandyweshay|ruck: should only be pinned for redhat 8.213:23
weshay|ruckmarios|rover, we were just discussing yesterday where all the pins for container-tools are located13:23
rlandytrain is running on 8.2 or 8.313:23
rlandyif you want to unpin podman, change ...13:23
weshay|ruckary |   "msg": "Depsolve Error occured: \n Problem: package tripleo-ansible-0.5.1-0.20201204163449.c4641f4.el8.noarch requires python3-paunch, but none of the providers can be installed\n  - package python3-paunch-5.3.3-0.20201114155407.7e49691.el8.noarch requires podman = 1.6.4, but none of the providers can be installed\n  - conflicting requests\n  - package podman-1.6.4-23.module_el8.3.0+566+4759265c.x86_64 is filtered out by modular13:23
weshay|ruckfiltering",13:23
weshay|ruckrlandy, review ^13:23
*** zbr has joined #oooq13:24
rlandyweshay|ruck: sorry there is a review, I must review or I must put in a review?13:25
marios|roverweshay|ruck: rlandy: https://bugs.launchpad.net/tripleo/+bug/1907193/comments/713:25
openstackLaunchpad bug 1907193 in tripleo "package podman-1.6.4-23.module_el8.3.0+566+4759265c.x86_64 is filtered out by modular filtering" [High,Fix released] - Assigned to Alex Schultz (alex-schultz)13:25
marios|roverweshay|ruck: 15:22 < weshay|ruck> marios|rover, do we have a boog on ary13:25
marios|roverweshay|ruck: what did you mean? ^13:25
rlandyah13:25
zbrweshay|ruck: did you read my comment on the ticket?13:26
weshay|ruckzbr, which ticket?13:27
weshay|ruckmarios|rover, chkumar|ruck let's ruck / rover sync when the scrum is done13:27
marios|roverweshay|ruck: chkumar|ruck: is out for a couple hours13:27
weshay|ruckk.. you avail?13:27
marios|roverweshay|ruck: i dropped from scrum13:27
marios|roverweshay|ruck: sure13:28
weshay|ruckhttps://meet.google.com/jaf-wsvy-ovp13:28
*** slaweq has quit IRC13:29
marios|roverweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1907193/comments/613:30
openstackLaunchpad bug 1907193 in tripleo "package podman-1.6.4-23.module_el8.3.0+566+4759265c.x86_64 is filtered out by modular filtering" [High,Fix released] - Assigned to Alex Schultz (alex-schultz)13:30
marios|roverweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1907193/comments/713:30
*** zbr has quit IRC13:30
rlandymarios|rover: weshay|ruck: https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_podman/vars/redhat-8.2.yml13:32
rlandyhttps://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_podman/vars/redhat.yml13:32
*** zbr has joined #oooq13:32
*** jpena|lunch is now known as jpena13:37
marios|roverrlandy: https://github.com/rdo-packages/paunch-distgit/blob/8521a057b919db641ef67b153915acfbaad852c7/python-paunch.spec#L70-L7513:41
marios|roverrlandy: https://logserver.rdoproject.org/openstack-periodic-integration-stable2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-buildimage-overcloud-hardened-full-ussuri/a4aacd7/job-output.txt13:43
marios|roverrlandy: ussuri ^^13:43
weshay|ruckmarios|rover, ya.. same in container build https://logserver.rdoproject.org/openstack-periodic-integration-stable2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-ubi-8-push-ussuri/870b752/job-output.txt13:45
weshay|ruckmarios|rover, https://github.com/redhat-openstack/rdoinfo/blob/master/rdo.yml13:46
marios|roverweshay|ruck: thx13:46
rlandyweshay|ruck: marios|rover: https://softwarefactory-project.io/cgit/config/tree/nodepool/rdo-cloud.yaml13:50
rlandyweshay|ruck: marios|rover: https://softwarefactory-project.io/cgit/config/tree/nodepool/virt_images/cloud-centos-8.yaml13:52
weshay|ruckzbr, what's the status of centos8.3 in nodepool13:53
weshay|ruckupstream13:53
weshay|ruckcan you please check for me13:53
zbrsure, just finished that chat with folco. i can check that.13:54
weshay|ruckzbr, thanks13:54
zbri guess you want to know if we have stream, or how hard it would be to have it, and how often it would be updated, right?13:54
zbri do remember seeing some msgs yesterday about switching from centos-8 to 8-stream.13:55
weshay|ruckmarios|rover, http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/13:56
marios|roverweshay|ruck: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_520/762124/2/gate/tripleo-ci-centos-8-content-provider-ussuri/520e2d8/job-output.txt13:56
weshay|ruckhttp://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/centos-linux-release-8.3-1.2011.el8.noarch.rpm13:56
chkumar|ruckmarios|rover: rlandy regarding podman issue13:57
chkumar|ruckit is blocked due to tripleo component promotion13:57
chkumar|ruckfs01 is failing there13:57
rlandychkumar|ruck: pls join https://meet.google.com/jaf-wsvy-ovp13:57
chkumar|ruckfor ussuri release13:57
rlandywe're chatting there13:57
weshay|ruck| primary |   "failures": [],13:59
weshay|ruck2020-12-10 04:54:16.190900 | primary |   "msg": "Depsolve Error occured: \n Problem: package tripleo-ansible-0.5.1-0.20201204163449.c4641f4.el8.noarch requires python3-paunch, but none of the providers can be installed\n  - package python3-paunch-5.3.3-0.20201114155407.7e49691.el8.noarch requires podman = 1.6.4, but none of the providers can be installed\n  - conflicting requests\n  - package podman-1.6.4-23.module_el8.3.0+566+4759265c.x8613:59
weshay|ruck_64 is filtered out by modular filtering",13:59
weshay|ruck2020-12-10 04:54:16.190976 | primary |   "rc": 1,13:59
weshay|ruckchkumar|ruck, ^13:59
chkumar|ruckweshay|ruck: rlandy marios|rover https://review.rdoproject.org/r/3130714:00
chkumar|ruckhttps://review.rdoproject.org/r/3130714:00
chkumar|ruckrlandy: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/tripleo-ci/CentOS-8/ussuri.yml#L11114:02
rlandyhttps://github.com/openstack/tripleo-quickstart/blob/master/config/release/tripleo-ci/CentOS-8/train.yml#L12114:02
*** zbr has quit IRC14:05
chkumar|ruckrlandy: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/76640414:05
*** zbr has joined #oooq14:08
marios|roverhttps://bugs.launchpad.net/tripleo/+bug/190754414:11
openstackLaunchpad bug 1907544 in tripleo "Rename compressed text based files to end with txt.gz extension getting skipped on rdo jobs" [Critical,Triaged]14:11
marios|roverrlandy: ^^^14:11
rlandychkumar|ruck: https://logserver.rdoproject.org/65/31165/3/check/periodic-tripleo-ci-centos-8-standalone-centos8stream-container-build-master/3e2ddff/logs/14:11
marios|roverrlandy: https://softwarefactory-project.io/r/#/c/20299/14:11
chkumar|ruckrlandy: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/76644814:12
weshay|ruckchkumar|ruck, marios|rover rlandy https://trunk.rdoproject.org/centos8-ussuri/component/tripleo/tripleo-ci-testing/14:13
weshay|ruckhttps://trunk.rdoproject.org/centos8-train/component/tripleo/tripleo-ci-testing/14:13
rlandychkumar|ruck: testproject with your fix https://review.rdoproject.org/r/#/c/31165/14:15
chkumar|ruckrlandy: marios|rover https://review.opendev.org/c/openstack/diskimage-builder/+/76623214:15
rlandyhttps://logserver.rdoproject.org/65/31165/3/check/periodic-tripleo-ci-centos-8-standalone-centos8stream-container-build-master/3e2ddff/logs/quickstart_collect_logs.log14:16
rlandyzbr: weshay|ruck: ^^14:16
rlandycollect logs error14:16
rlandyseen that error?14:17
chkumar|ruckrlandy: https://bugs.launchpad.net/tripleo/+bug/190754414:18
openstackLaunchpad bug 1907544 in tripleo "Rename compressed text based files to end with txt.gz extension getting skipped on rdo jobs" [Critical,Triaged]14:18
chkumar|ruckrlandy: check this one14:18
zbri do not think that DEBUG does help making the logs easier to read.14:18
*** TrevorV has joined #oooq14:19
weshay|rucksshnaidm,  you are on the container pulls ?14:23
sshnaidmweshay|ruck, container pulls..?14:23
chkumar|ruckweshay|ruck: rlandy https://logserver.rdoproject.org/23/31323/5/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-queens/2528849/logs/undercloud/home/zuul/undercloud_install.log.gz14:31
rlandychkumar|ruck: ^^ related to something or you want help debugging here?14:33
chkumar|ruckrlandy: we need a bug for queens for undercloud install14:34
weshay|rucksshnaidm, https://bugs.launchpad.net/tripleo/+bug/190765714:39
openstackLaunchpad bug 1907657 in tripleo "various jobs failing in the gate during container image prepare 404 not found" [Critical,Triaged]14:39
sshnaidmweshay|ruck, yeah, looking into14:39
weshay|rucksshnaidm, thanks14:41
weshay|ruckchkumar|ruck, I don't think train and ussuri are fixed re: podman14:44
weshay|ruckchkumar|ruck, retries in train line https://review.rdoproject.org/zuul/status14:44
* weshay|ruck watching the console now14:45
weshay|ruckchkumar|ruck, marios|rover sorry.. I promoted directly to tripleo-ci-testing... which was overwritten in the pin job of integration.. re-promoted to promoted-components14:53
weshay|ruckok.. this time it worked14:57
weshay|ruckrlandy, I screwed that up :)14:58
rlandyweshay|ruck: promoted components?14:58
weshay|ruckrlandy, ya.. I went from current -> tripleo-ci-testing the first time14:59
rlandyweshay|ruck: to be honest, I lost the discussion at the need to promote if we are pulling python-paunch from current anyways :)14:59
weshay|ruckwhich was overwritten14:59
rlandyconsistent -> component-ci-testing15:00
weshay|ruckrlandy, so we need to patch tripleo-repos to pull current for tripleo-ci-testing as well15:00
rlandyI didn't think dlrn allowed you to mess up15:00
weshay|ruckhrm.. we're not communicating yet15:00
weshay|ruckI'll explain later15:00
rlandy[zuul@standalone ~]$ rpm -qa | grep podman15:18
rlandypodman-2.1.1-3.module_el8.4.0+575+63b40ad7.x86_6415:18
rlandyweshay|ruck: ^^ get with stream and conatiner-tools:rhel815:18
rlandyweshay|ruck: actually we should unpin container tools for rhel as well15:28
rlandywe probably don;t need it15:28
* rlandy gets15:28
rlandyhttps://github.com/openstack/tripleo-common/blob/master/container-images/tcib/base/base.yaml#L6215:30
marios|roverweshay|ruck: they are starting the stream discussion now if you wanted to watch15:32
marios|roverweshay|ruck: apevec is there too15:32
weshay|ruckk15:32
weshay|ruckI'm there15:32
*** Goneri has joined #oooq15:38
sshnaidmweshay|ruck, commented on https://bugs.launchpad.net/tripleo/+bug/190765715:41
openstackLaunchpad bug 1907657 in tripleo "various jobs failing in the gate during container image prepare 404 not found" [Critical,Triaged]15:41
sshnaidmweshay|ruck, zuul bug and there is a patch to solve it, but not merged15:41
weshay|rucksshnaidm, /me looks ++15:42
weshay|ruckthank you15:42
sshnaidmbtw, might be interesting https://storyboard.openstack.org/#!/story/200833115:43
sshnaidmweshay|ruck, ^15:44
rlandychkumar|ruck: no luck on the tempest fix ...  2020-12-10 15:43:51.661483 | primary | TASK [Pause the ssh connection to avoid getpwuid issue] ************************15:53
rlandy2020-12-10 15:43:51.661582 | primary | Thursday 10 December 2020  15:43:51 +0000 (0:00:00.429)       1:18:41.493 *****15:53
rlandy2020-12-10 15:43:51.685864 | primary | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'getpwuid(): uid not found: 1000'15:53
rlandy2020-12-10 15:43:51.685973 | primary | fatal: [undercloud]: FAILED! => {}15:53
rlandyhttps://review.rdoproject.org/zuul/stream/20909e5e8f904e9ab83f7d7a5473c5eb?logfile=console.log15:53
rlandyuh - no logs again15:57
* zbr docker community all hands just started, are they going to announce a docker-stream? :D16:02
zbrthe best part about the slides: they all have a bottom line: "Docker, Inc. Proprietary" --- just to make it clear ;)16:03
*** zbr has quit IRC16:04
weshay|ruckrlandy, so do we need to sync on updated our zuul config to 8.3?16:04
weshay|rucksorry to drag you into this16:05
*** zbr has joined #oooq16:06
rlandyweshay|ruck: I don;t think so16:06
rlandywe should get it by default16:06
* rlandy checks16:06
weshay|ruckk16:06
rlandy2020-12-10 16:01:35.133117 | localhost | ok:16:07
rlandy2020-12-10 16:01:35.133466 | localhost | # Node Information16:07
rlandy2020-12-10 16:01:35.133536 | localhost | Inventory Hostname: primary16:07
rlandy2020-12-10 16:01:35.133579 | localhost | Hostname: node-000071149616:07
rlandy2020-12-10 16:01:35.133617 | localhost | Username: zuul16:07
rlandy2020-12-10 16:01:35.133655 | localhost | Distro: CentOS 8.316:07
rlandy2020-12-10 16:01:35.133691 | localhost | Provider: vexxhost-nodepool-tripleo16:07
rlandy^^ we're centos 8.3 in vexx16:07
rlandyweshay|ruck: running https://review.rdoproject.org/zuul/stream/a92384f6837e4689991b2109cf0377a7?logfile=console.log16:07
*** zbr has quit IRC16:07
*** zbr has joined #oooq16:08
rlandyand no worries - I'm already knee-deep in centos 8 changes16:09
* rlandy wants to update container-tools downstream16:09
weshay|ruckok.. great16:12
weshay|ruckrfolco, got a sec16:16
rfolcoweshay|ruck, yup16:17
weshay|ruckrfolco, need help w/ a container builds16:17
weshay|ruckrfolco, https://meet.google.com/uzr-kpxh-dbd?authuser=116:17
weshay|ruckrfolco, https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-build-containers-ubi-8-push-train/eea5f05/logs/16:17
weshay|ruckrfolco, System has not been booted with systemd as init system (PID 1). Can't operate.16:21
weshay|ruckFailed to connect to bus: Host is down16:21
*** zbr has quit IRC16:24
*** zbr has joined #oooq16:26
*** zbr has quit IRC16:29
*** zbr has joined #oooq16:29
weshay|ruckrfolco, https://review.opendev.org/c/openstack/tripleo-common/+/762659/16:41
weshay|ruckrfolco,  https://review.opendev.org/c/openstack/tripleo-common/+/76266016:44
*** zbr has quit IRC16:46
*** zbr has joined #oooq16:49
*** zbr has quit IRC16:49
rfolcoweshay|ruck, https://review.rdoproject.org/r/3133616:51
*** zbr has joined #oooq16:55
weshay|ruckrfolco, thanks16:59
marios|roverweshay|ruck: sent http://lists.openstack.org/pipermail/openstack-discuss/2020-December/019338.html17:01
marios|roverweshay|ruck: k am gonna go in a sec need something before i do?17:01
weshay|ruckmarios|rover, nope..  finding new shit.. but some stuff is merging and being resolved17:02
weshay|ruckall on the sheet17:02
marios|roverweshay|ruck: i hit recheck @ https://review.opendev.org/c/openstack/python-tripleoclient/+/766046 after * Merged https://review.opendev.org/c/openstack/tripleo-ci/+/76629717:02
marios|roverweshay|ruck: the shit on the sheet gotcha17:02
marios|roverweshay|ruck: i look forward to it for breakfast17:02
marios|roveri may not even sleep17:03
*** Trevor_V has joined #oooq17:05
*** TrevorV has quit IRC17:06
*** marios|rover has quit IRC17:12
*** jpena is now known as jpena|off17:17
*** zbr has quit IRC17:21
*** zbr has joined #oooq17:24
*** udesale_ has quit IRC17:34
*** zbr has quit IRC17:35
rlandyweshay|ruck: train jobs all set now?17:36
weshay|ruckrlandy, no.. container-build failures17:37
weshay|ruckpatches need to land for it to work17:37
*** zbr has joined #oooq17:37
weshay|ruckrfolco, https://logserver.rdoproject.org/36/31336/1/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/a8f34ae/logs/report.html17:38
weshay|ruckrfolco, https://logserver.rdoproject.org/36/31336/1/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/a8f34ae/logs/buildah-builds/container-builds/224d8975-e560-46f0-8461-9e26b1a5f7c2/base/os/cinder-base/cinder-volume/cinder-volume-build.log17:40
rlandyugh17:41
rfolcoyeah but cinder volume is not listed in images here https://logserver.rdoproject.org/36/31336/1/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/a8f34ae/logs/containers-built.log17:41
weshay|ruckrfolco, k.. ya17:41
weshay|ruckso I see writing hte image didn't complete17:41
weshay|ruckrfolco, /me rekicks.. that could be transient17:41
rfolcohttps://logserver.rdoproject.org/36/31336/1/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/a8f34ae/logs/build.log17:42
rfolco Exception information: 'BuildahBuilder' object has no attribute 'log'17:42
weshay|ruckrfolco, k.. added to the bug https://bugs.launchpad.net/tripleo/+bug/190402517:44
openstackLaunchpad bug 1904025 in tripleo "Container build on failures reports Exception information: 'int' object is not callable" [High,In progress] - Assigned to Alex Schultz (alex-schultz)17:44
rfolcoapparently it applied the fix17:45
rfolco2020-12-10 16:55:29.594909 | primary | ZUUL_CHANGES=openstack/tripleo-common:stable/train:refs/changes/92/762892/1^openstack/tripleo-ci:master:refs/changes/24/755124/57^testproject:master:refs/changes/36/31336/117:45
rfolcoaaaah17:46
rfolcoshould we override branch on job definition ?17:46
rfolconevermind this is periodic, override is here https://review.rdoproject.org/r/gitweb?p=config.git;a=blob;f=zuul.d/tripleoci.yaml#l39617:49
*** zbr has quit IRC17:55
*** jbadiapa has quit IRC17:57
*** zbr has joined #oooq17:57
*** zbr5 has joined #oooq17:57
*** zbr5 has quit IRC17:58
*** zbr has quit IRC17:58
*** zbr has joined #oooq17:59
*** derekh has quit IRC18:01
*** dtantsur is now known as dtantsur|afk18:40
weshay|ruckrfolco, gets worse https://logserver.rdoproject.org/36/31336/1/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/be8a017/logs/report.html19:04
rfolcohmm the containers are not listed in the https://logserver.rdoproject.org/36/31336/1/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/be8a017/logs/containers-built.log19:08
rfolcoperhaps that fix is not enough19:09
weshay|ruckya.. rfolco ya.. still a shitty trace for train19:14
* weshay|ruck reraised the bug19:14
rfolcoweshay|ruck, you had pointed another fix when we chatted19:14
rfolcoweshay|ruck, was the train one merged ?19:14
weshay|ruckit's in your depends-on no?19:15
rfolcono, I add one fix and my report patch19:15
weshay|ruckright.. the one train patch is all we have, and it's not complete afaict19:15
rfolcoaaah19:16
rfolcowe are missing this one then https://review.opendev.org/c/openstack/tripleo-common/+/76266019:16
weshay|ruckmarked it promotion-blocker19:16
rfolcoweshay|ruck, ^19:16
weshay|ruckoh19:16
weshay|ruckrfolco, k.. let's add that and rerun.. thanks.. missed that19:16
weshay|ruckrfolco, wait..19:17
rfolcoweshay|ruck, you add or me19:17
weshay|ruckboth are tc19:17
weshay|ruckI'll do it19:17
rfolcoboth are tc19:17
rfolcoyes19:17
rfolcomaybe the other can be rebased19:17
*** matbu has quit IRC19:20
weshay|ruckrfolco, k.. rebased them19:22
rfolcocool19:22
rfolcothx19:22
*** matbu has joined #oooq19:23
rlandygit.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git19:45
rlandy^^ resolving for you?19:45
rfolcoweshay|ruck, can I make changes to the report patch ?19:46
rfolcoweshay|ruck, I am going to test new unstable features19:47
rlandyrfolco: weshay|ruck: ^^ can you reach internal repos?19:47
rfolcoloading..19:47
rfolcoloading19:47
rfolcorlandy, not working for me19:47
rfolcoit nerver responds19:47
rlandyyep - same here19:47
weshay|ruckwhat's wrong w/ this week19:51
rlandyback now19:58
rlandyweshay|ruck: here's the answer to why we didn't see the podman 1.5.4 issue  in rhel - we did ...20:06
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-dependencies-container_tools/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-container_tools-rhos-16.2/7888432/logs/undercloud/home/zuul/install_packages.sh.log20:06
rlandyright there in the dependency pipeline20:06
rlandywe just didn't watch it20:06
rlandy package podman-1.6.4-23.module+el8.3.0+8233+627fbb78.x86_64 is filtered out by modular filtering20:06
weshay|ruckwell.. at least we know we have the pipelines working20:07
rlandylol20:07
rlandywe need that fix downstream20:07
rlandyweshay|ruck: ^^ we're going to have to do the component promote thing downstream20:08
rlandyhttps://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-dependencies-container_tools20:09
rlandyline was failing since 10/1320:09
rlandylol20:09
rlandytracing fix20:11
rlandylocalhost | Distro: RedHat 8.320:12
*** jfrancoa has quit IRC20:42
weshay|ruckrfolco, https://meet.google.com/uyj-jfha-ivs20:59
*** Trevor_V has quit IRC21:16
weshay|ruckrlandy, so I see a couple patches around the tempest skip list thing.. anything you want me to update in the notes?21:21
*** fuzzball81 has quit IRC21:24
weshay|ruckrlandy, need your +2 if avail21:25
weshay|ruckhttps://review.opendev.org/c/openstack/tripleo-common/+/76651721:26
rlandylooking21:27
rlandyweshay|ruck: idk what will fix skiplist yet21:27
rlandyshooting in the dark here21:27
weshay|ruckk21:27
rlandyweshay|ruck: you know marios is going ask why removing lower constaints21:29
rlandyhttps://review.opendev.org/c/openstack/tripleo-common/+/76651721:29
rlandycommit message?21:29
rlandyif it's urgent, I'll +221:29
weshay|ruckjust everything is blocked21:29
rlandyfine21:30
rlandyweshay|ruck: https://review.opendev.org/c/openstack/tripleo-common/+/766517 done21:31
rlandyadded comment in review21:31
rlandyweshay|ruck: https://review.opendev.org/c/openstack/tripleo-quickstart/+/766286/2/config/release/dependency_ci/centos8stream/repo_config.yaml will need to go21:32
rlandyrepo names changed21:32
*** rfolco has quit IRC21:34
*** fuzzball81 has joined #oooq21:36
*** fuzzball81 has quit IRC21:37
*** fuzzball81 has joined #oooq21:37
*** frenzy_friday has quit IRC22:17
*** frenzy_friday has joined #oooq22:18
*** Goneri has quit IRC23:40
*** rfolco has joined #oooq23:41
*** tosky has quit IRC23:53
« Wednesday, 2020-12-09 Index Friday, 2020-12-11 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!