*** tosky has quit IRC | 00:00 | |
*** ysandeep|away is now known as ysandeep | 00:13 | |
-openstackstatus- NOTICE: The Gerrit service on review.opendev.org is being restarted quickly to make heap memory and jgit config adjustments, downtime should be less than 5 minutes | 01:08 | |
*** Goneri has quit IRC | 01:21 | |
*** rfolco has joined #oooq | 02:34 | |
*** ysandeep is now known as ysandeep|session | 02:36 | |
*** apetrich has quit IRC | 03:19 | |
*** rfolco has quit IRC | 03:24 | |
*** saneax has joined #oooq | 05:20 | |
*** udesale has joined #oooq | 05:33 | |
*** pojadhav|afk is now known as pojadhav | 05:35 | |
*** ratailor has joined #oooq | 05:39 | |
*** zbr has quit IRC | 06:06 | |
*** ysandeep|session is now known as ysandeep|afk | 06:09 | |
*** marios has joined #oooq | 06:44 | |
*** marios is now known as marios|rover | 06:45 | |
chkumar|ruck | marios|rover Good morning | 06:45 |
---|---|---|
marios|rover | o/ | 06:55 |
*** ysandeep|afk is now known as ysandeep|session | 07:01 | |
*** jbadiapa has joined #oooq | 07:09 | |
*** dmellado has quit IRC | 07:26 | |
*** dmellado has joined #oooq | 07:26 | |
*** dmellado has quit IRC | 07:34 | |
*** dmellado has joined #oooq | 07:36 | |
chkumar|ruck | marios|rover: in order to fix ussuri periodic jobs, we need a component promotion of tripleo | 07:46 |
chkumar|ruck | to pull paunch fixes | 07:46 |
marios|rover | chkumar|ruck: why do we need it ? it looks pretty green though except one job periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-ussuri | 07:48 |
marios|rover | chkumar|ruck: oh for the paunch stuff sorry missed that | 07:49 |
chkumar|ruck | marios|rover: yes | 07:50 |
chkumar|ruck | marios|rover: for train periodic, it is started hitting the same issue | 07:50 |
chkumar|ruck | we also need the same tripleo component promotion | 07:50 |
chkumar|ruck | marios|rover: please +w https://review.opendev.org/c/openstack/tripleo-ansible/+/764360 it | 07:51 |
chkumar|ruck | and https://review.opendev.org/c/openstack/tripleo-ansible/+/764361 this also | 07:51 |
*** skramaja has joined #oooq | 07:51 | |
marios|rover | chkumar|ruck: ack will check in sec | 07:56 |
*** tosky has joined #oooq | 08:02 | |
*** ksambor has joined #oooq | 08:06 | |
marios|rover | chkumar|ruck: link to paunch issue? launchpad | 08:12 |
marios|rover | https://bugs.launchpad.net/tripleo/+bug/1907193 that one? | 08:13 |
openstack | Launchpad bug 1907193 in tripleo "package podman-1.6.4-23.module_el8.3.0+566+4759265c.x86_64 is filtered out by modular filtering" [High,Fix released] - Assigned to Alex Schultz (alex-schultz) | 08:13 |
chkumar|ruck | marios|rover: yes | 08:13 |
marios|rover | yah k... but weshay|ruck moved the bug fix-released and closed the cix | 08:13 |
marios|rover | chkumar|ruck: so you mean, the ussuri promotion jobs are failing with that so we need the tripleo component promotion to fix those. | 08:14 |
chkumar|ruck | marios|rover: it fixed upstream content provider | 08:14 |
marios|rover | k | 08:14 |
chkumar|ruck | marios|rover: yes correct | 08:14 |
marios|rover | chkumar|ruck: k thanks man with you now ;) | 08:14 |
marios|rover | chkumar|ruck: k but the bug and cix needs to go back to in progress i think until we aren't seeing it any more | 08:18 |
marios|rover | chkumar|ruck: cos may take a while to get it sorted on ussuri & train | 08:19 |
marios|rover | chkumar|ruck: i'll take care of that in a bit | 08:19 |
*** jfrancoa has joined #oooq | 08:30 | |
*** slaweq has joined #oooq | 08:35 | |
slaweq | sshnaidm: hi, I have a question regarding https://review.opendev.org/c/openstack/tripleo-quickstart/+/759832 | 08:35 |
slaweq | can You tell me how I can test that patch with that "real 020 job"? | 08:36 |
*** saneax has quit IRC | 08:45 | |
*** udesale has quit IRC | 08:48 | |
*** udesale has joined #oooq | 08:50 | |
*** apetrich has joined #oooq | 08:56 | |
*** jpena|off is now known as jpena | 08:59 | |
*** rfolco has joined #oooq | 09:00 | |
*** derekh has joined #oooq | 09:02 | |
*** ysandeep|session is now known as ysandeep|lunch | 09:03 | |
sshnaidm | slaweq, for example to define it here with depends on your patch: https://review.rdoproject.org/r/#/c/23626/15/zuul.yaml | 09:10 |
slaweq | sshnaidm: ok, thx a lot | 09:19 |
slaweq | I will propose such patch now | 09:19 |
slaweq | sshnaidm: done https://review.rdoproject.org/r/31323 | 09:42 |
slaweq | lets wait for the results now :) | 09:42 |
slaweq | sshnaidm++ thx for help | 09:42 |
*** holser has quit IRC | 09:48 | |
*** holser has joined #oooq | 09:53 | |
chkumar|ruck | marios|rover: we got a new problem ,https://review.rdoproject.org/zuul/builds?result=RETRY_LIMIT https://logserver.rdoproject.org/23/31323/1/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-queens/cdac3a9/job-output.txt Verify Login for docker failing | 10:01 |
chkumar|ruck | impacting all ovb jobs | 10:01 |
marios|rover | chkumar|ruck: fak | 10:03 |
chkumar|ruck | opening a bug | 10:03 |
*** udesale has quit IRC | 10:06 | |
slaweq | sshnaidm: I don't know if it's my fault but this job failed for me a lot before it event started tests https://review.rdoproject.org/zuul/build/cdac3a9422824619abeecf0d4b8605b9 | 10:13 |
slaweq | is it broken now? Do You know maybe? | 10:13 |
sshnaidm | slaweq, hmm.. this is something new | 10:17 |
sshnaidm | chkumar|ruck, did you see before? https://logserver.rdoproject.org/23/31323/1/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-queens/cdac3a9/job-output.txt | 10:17 |
sshnaidm | chkumar|ruck, it's from https://review.rdoproject.org/r/#/c/31323/ | 10:17 |
chkumar|ruck | sshnaidm: bug coming up | 10:18 |
*** ssbarnea has joined #oooq | 10:18 | |
slaweq | ups :) | 10:21 |
chkumar|ruck | marios|rover: sshnaidm slaweq https://bugs.launchpad.net/tripleo/+bug/1907439 | 10:22 |
openstack | Launchpad bug 1907439 in tripleo "assertion: invalid username/password in registry_login_podman.results.0.stderr" [Critical,Triaged] | 10:22 |
marios|rover | thanks chkumar|ruck | 10:23 |
chkumar|ruck | checking with rhos-ops on registry creds side | 10:23 |
marios|rover | so much for ussuri tripleo component... | 10:23 |
slaweq | chkumar|ruck: thx | 10:23 |
*** zbr has joined #oooq | 10:23 | |
sshnaidm | chkumar|ruck, thanks, trying w/o login.. | 10:23 |
chkumar|ruck | sshnaidm: sure | 10:23 |
zbr | holla! i am kind back, i moved my containers from c8 to f33 which seams to work. | 10:24 |
sshnaidm | slaweq, changed tha patch, let's see now.. | 10:24 |
slaweq | sshnaidm: thx | 10:25 |
*** ssbarnea has quit IRC | 10:36 | |
*** ysandeep|lunch is now known as ysandeep | 10:38 | |
chkumar|ruck | marios|rover: sshnaidm jpena is working on fixing on an issue with ssl certificate issue | 10:41 |
marios|rover | chkumar|ruck: thank you i was looking at the creds just now wondering if we needed to update it... | 10:42 |
chkumar|ruck | marios|rover: not sure, yesterday dpawalik renewed the centos cert, jpena knows what is going on, he is taking care of that | 10:48 |
jpena | marios|rover: creds are ok, what was renewed yesterday is the CBS cert | 10:48 |
jpena | it's the SSL certificate for trunk.registry.rdoproject.org that is failing atm | 10:48 |
marios|rover | thanks jpena | 10:49 |
*** dtantsur|afk is now known as dtantsur | 10:52 | |
chkumar|ruck | marios|rover: do we have patch for lower constraints job failure? | 10:52 |
marios|rover | chkumar|ruck: well... removal of the job | 10:52 |
marios|rover | chkumar|ruck: sec | 10:52 |
marios|rover | chkumar|ruck: https://bugs.launchpad.net/tripleo/+bug/1907236/comments/5 | 10:53 |
openstack | Launchpad bug 1907236 in tripleo "lower-constraints jobs across tripleo projects broken" [Critical,In progress] | 10:53 |
chkumar|ruck | marios|rover: thanks! | 10:57 |
marios|rover | zbr: have you seen that wdyt? you were looking there last night i think? https://bugs.launchpad.net/tripleo/+bug/1907236/comments/5 | 10:59 |
openstack | Launchpad bug 1907236 in tripleo "lower-constraints jobs across tripleo projects broken" [Critical,In progress] | 10:59 |
marios|rover | brb | 10:59 |
*** saneax has joined #oooq | 11:00 | |
* ysandeep afk for sometime | 11:11 | |
zbr | marios|rover: yes, I am aware of the measure, still the incorrect job configuration prevented some changes from merging. | 11:11 |
marios|rover | thanks zbr | 11:13 |
*** zbr has quit IRC | 11:25 | |
*** zbr has joined #oooq | 11:28 | |
chkumar|ruck | marios|rover: brb for an hour | 11:30 |
jpena | marios|rover, chkumar|ruck: the RDO registry should be back in action now, can you check the jobs? | 11:35 |
marios|rover | jpena: not yet http://dashboard-ci.tripleo.org/d/3-DYSmOGk/jobs-exploration?orgId=1&var-influxdb_filter=pipeline%7C%3D%7Copenstack-check but thanks will keep an eye | 11:39 |
jpena | marios|rover: it was only fixed ~5 mins ago | 11:39 |
soniya29 | kopecmartin, chkumar|ruck, please edit or add any item for the today's agenda of tempest meeting - https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?both | 11:41 |
marios|rover | chkumar|ruck: is that yet another issue now... 2020-12-09 11:07:15.146602 | primary | cat: /home/zuul/workspace/ovb/idnum: No such file or directory https://logserver.rdoproject.org/24/755124/54/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001/2491fbf/job-output.txt | 11:44 |
*** zbr has quit IRC | 11:46 | |
*** zbr has joined #oooq | 11:48 | |
marios|rover | chkumar|ruck: ah no its the same error from https://bugs.launchpad.net/tripleo/+bug/1907439 further down... | 11:51 |
openstack | Launchpad bug 1907439 in tripleo "assertion: invalid username/password in registry_login_podman.results.0.stderr" [Critical,Triaged] | 11:51 |
*** ratailor has quit IRC | 11:59 | |
chkumar|ruck | jpena: re-triggering | 12:09 |
chkumar|ruck | jpena: marios|rover https://review.rdoproject.org/zuul/stream/9c5a25eab68a47e39164e3f7fa8cd89d?logfile=console.log jobs started 20 mins back | 12:14 |
chkumar|ruck | are working | 12:14 |
chkumar|ruck | jpena: can you take a look at jobs we are seeing node failurres | 12:16 |
chkumar|ruck | it just started | 12:16 |
chkumar|ruck | jpena: https://review.rdoproject.org/zuul/builds?result=NODE_FAILURE | 12:17 |
chkumar|ruck | 12:12 onwards | 12:17 |
zbr | the current CI jobs setup is a big mess, attempt to lower what we trigger has to wait at least one working day to pass check/gate when it should likely trigger nothing. | 12:21 |
zbr | https://zuul.opendev.org/t/openstack/status/change/766208,2 | 12:21 |
zbr | correcting a past mistake seams to take ages of CI | 12:22 |
jpena | chkumar|ruck: I've seen some issues in nodepool trying to get the IP for a VM, but it seems to have gone away now | 12:22 |
chkumar|ruck | jpena: ok, thanks for checking | 12:22 |
*** skramaja has quit IRC | 12:30 | |
*** jpena is now known as jpena|lunch | 12:35 | |
*** udesale has joined #oooq | 12:49 | |
weshay|ruck | marios|rover, chkumar|ruck just looking for the latest news, is the rdo-registry cert issue fixed or pending | 12:59 |
marios|rover | weshay|ruck: should be fixed | 13:00 |
chkumar|ruck | weshay|ruck: it got fixed | 13:00 |
marios|rover | weshay|ruck: but only just now i.e. | 13:00 |
chkumar|ruck | jobs are looking fine | 13:00 |
marios|rover | weshay|ruck: about 1 hour ago now | 13:00 |
*** rlandy has joined #oooq | 13:01 | |
chkumar|ruck | component network jobs are running fine also | 13:01 |
chkumar|ruck | marios|rover: in last 3 days, we have seen 3 nightmares :-) | 13:01 |
chkumar|ruck | hope for best tomorrow | 13:01 |
soniya29 | chkumar|ruck, kopecmartin, weshay|ruck, pojadhav, akahat, tempest meeting? | 13:01 |
chkumar|ruck | soniya29: I will be skipping | 13:01 |
weshay|ruck | soniya29, I'm taking the program call today, will have to miss | 13:01 |
marios|rover | chkumar|ruck: :/ | 13:03 |
akahat | soniya29, will joining in min.. | 13:03 |
chkumar|ruck | soniya29: if you need any help, let me know i can switch back | 13:04 |
weshay|ruck | dam.. 1 more queens import in march | 13:09 |
chkumar|ruck | weshay|ruck: till when we need to support queens? | 13:10 |
weshay|ruck | we can probably drop it after march.. | 13:10 |
weshay|ruck | akahat, sshnaidm https://review.opendev.org/q/owner:%22amolkahat+%253Camolkahat%2540gmail.com%253E%22+branch:stable/ussuri | 13:10 |
weshay|ruck | these need to be worked through | 13:10 |
weshay|ruck | chkumar|ruck, ^ | 13:10 |
chkumar|ruck | weshay|ruck: looking | 13:11 |
sshnaidm | weshay|ruck, it fails on upgrade job, isn't it that we wanted to fix? | 13:11 |
weshay|ruck | there are quite a few jobs failing | 13:12 |
*** pojadhav is now known as pojadhav|brb | 13:15 | |
soniya29 | chkumar|ruck, thanks :) | 13:15 |
akahat | tripleo-ci-centos-8-undercloud-upgrade-ussuri and tripleo-ci-centos-8-standalone-upgrade-ussuri is failing.. | 13:16 |
weshay|ruck | akahat, I'm in the program call, may be a little late for 1-1 | 13:20 |
akahat | weshay|ruck, okay. | 13:20 |
rfolco | chkumar|ruck, get a min? | 13:22 |
chkumar|ruck | rfolco: yes | 13:22 |
rfolco | chkumar|ruck, why buildah images command needs to be run as root if I built as normal user ? | 13:22 |
*** Goneri has joined #oooq | 13:24 | |
chkumar|ruck | rfolco: we added it as a root because buildah were not able to discover images which exists on the box | 13:24 |
*** zbr has quit IRC | 13:24 | |
chkumar|ruck | rfolco: but I need to check | 13:25 |
rfolco | chkumar|ruck, build as normal user for centos: https://github.com/openstack/tripleo-ci/blob/master/roles/build-containers/tasks/main.yaml#L210-L215 | 13:26 |
rfolco | chkumar|ruck, but when listing images, if you remove become, it fails | 13:26 |
rfolco | https://github.com/openstack/tripleo-ci/blob/master/roles/build-containers/tasks/main.yaml#L246-L251 | 13:26 |
*** zbr has joined #oooq | 13:27 | |
rfolco | chkumar|ruck, however, on my local machine, this is not true, images built are shown as normal user | 13:27 |
chkumar|ruck | rfolco: not sure, but need to check rhel job | 13:27 |
rfolco | chkumar|ruck, in rhel, it builds as root | 13:28 |
chkumar|ruck | rfolco: ah | 13:28 |
rfolco | chkumar|ruck, maybe we are missing a container group config for the ansible user ? | 13:28 |
chkumar|ruck | rfolco: might be | 13:28 |
chkumar|ruck | rfolco: put the patch, we can test on both rhel and centos | 13:29 |
weshay|ruck | akahat, I'm in | 13:29 |
*** jpena|lunch is now known as jpena | 13:30 | |
weshay|ruck | chkumar|ruck, marios|rover 0/ anything you fine folks need help w/? | 13:31 |
marios|rover | weshay|ruck: not currently something i can see | 13:31 |
marios|rover | weshay|ruck: waiting to see if the gates start clearing up | 13:31 |
chkumar|ruck | weshay|ruck: we need to merge lower constriants patch then we are good, zbr is taking care of that | 13:31 |
chkumar|ruck | rest are good | 13:31 |
chkumar|ruck | more fun will come tomorrow | 13:32 |
weshay|ruck | very nice.. I think we're getting over the hump hopefully | 13:32 |
weshay|ruck | shitty week | 13:32 |
weshay|ruck | chkumar|ruck, I could use your help w/ something | 13:32 |
marios|rover | weshay|ruck: and confirming the fixes eg https://bugs.launchpad.net/tripleo/+bug/1907122/comments/4 | 13:32 |
openstack | Launchpad bug 1907122 in tripleo "gating repo improperly enabled across releases, enabling the wrong patches to be integrated into build" [Critical,Fix released] | 13:32 |
weshay|ruck | will ping a few | 13:32 |
chkumar|ruck | weshay|ruck: sure | 13:32 |
zbr | chkumar|ruck: weshay|ruck: re tripleo-lower hashtag, I am not yet sure my patch works, https://review.opendev.org/c/openstack/tripleo-ci/+/766208 | 13:34 |
chkumar|ruck | zbr: what about firing a testproject on rdo side | 13:34 |
chkumar|ruck | for quick feedback | 13:34 |
*** pojadhav|brb is now known as pojadhav | 13:35 | |
zbr | it does not help, the only side effect of this change would be that touching that file should no longer trigger deployments. | 13:35 |
zbr | the the jobs that are running now on this change are huge waste of time, they don't test anything about it, it only slows us down. | 13:38 |
zbr | and the slowdown is at least 8h, only of one minor tune of CI config. | 13:38 |
*** zbr has quit IRC | 13:45 | |
chkumar|ruck | weshay|ruck: free now | 13:46 |
*** zbr has joined #oooq | 13:47 | |
chkumar|ruck | zbr: where is the depends-on patch? | 13:47 |
*** saneax has quit IRC | 13:49 | |
weshay|ruck | soniya29, may be a few min late | 13:54 |
rfolco | weshay|ruck, chkumar|ruck marios|rover just a teaser on build report: https://logserver.rdoproject.org/40/29840/6/check/periodic-tripleo-ci-build-containers-ubi-8-push/11e4fa2/logs/report.html | 13:55 |
marios|rover | rfolco: very nice | 13:55 |
zbr | chkumar|ruck: it does not work, likely because it needs to be merged first. Take a look at https://review.opendev.org/c/openstack/python-tripleoclient/+/766223 | 13:57 |
rfolco | zbr, https://logserver.rdoproject.org/40/29840/6/check/periodic-tripleo-ci-build-containers-ubi-8-push/11e4fa2/logs/report.html | 13:57 |
zbr | if you dig parents of tripleo-ci-centos-8-undercloud-containers you will discover that ^lower-constraints.txt$ is found at its 4th parent. | 13:57 |
zbr | rfolco: looks neat, i wonder how it looks when it has a failure. i hope more than two lines. | 13:58 |
rfolco | zbr, yes, will append log failures | 13:59 |
rfolco | and full log link will be on links column | 13:59 |
rfolco | brb | 13:59 |
*** rfolco is now known as rfolco|brb | 13:59 | |
*** zbr has quit IRC | 14:03 | |
*** zbr has joined #oooq | 14:04 | |
weshay|ruck | soniya29, I'm in meet | 14:11 |
weshay|ruck | chkumar|ruck, marios|rover incoming https://bugs.launchpad.net/tripleo/+bug/1907457 | 14:14 |
openstack | Launchpad bug 1907457 in tripleo "tripleo-buildimage-overcloud-hardened-full-centos-8 fails with dracut module 'network-legacy' cannot be found or installed" [Critical,Triaged] | 14:14 |
chkumar|ruck | weshay|ruck: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/708543 | 14:16 |
chkumar|ruck | not linked | 14:17 |
*** TrevorV has joined #oooq | 14:17 | |
marios|rover | weshay|ruck: thanks | 14:22 |
weshay|ruck | rlandy, let's rerun centos-stream job and see if that validations repro's | 14:39 |
rlandy | weshay|ruck: ack - will do | 14:54 |
weshay|ruck | chkumar|ruck, marios|rover ysandeep looks like upshift is working now | 15:06 |
ysandeep | weshay|ruck, yes :) shrestha is following up with me for 16.2 promotion, if today's run passes.. Could you please promote 16.2 | 15:09 |
marios|rover | weshay|ruck: thanks | 15:09 |
weshay|ruck | ysandeep, note.. we'll wait for PoC Integration 16 https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/pipeline/ | 15:09 |
ysandeep | weshay|ruck, ack o/ | 15:10 |
*** TrevorV has quit IRC | 15:18 | |
*** Goneri has quit IRC | 15:23 | |
*** Goneri has joined #oooq | 15:24 | |
*** zbr has quit IRC | 15:29 | |
*** zbr has joined #oooq | 15:30 | |
*** ysandeep is now known as ysandeep|away | 15:46 | |
*** zbr has quit IRC | 15:48 | |
*** zbr has joined #oooq | 15:51 | |
chkumar|ruck | marios|rover: weshay|ruck I will take a look tomorrow on dib issue | 15:52 |
marios|rover | chkumar|ruck: ack | 15:52 |
chkumar|ruck | not a piece of cake for today | 15:52 |
slaweq | sshnaidm: hi again, job https://review.rdoproject.org/zuul/build/6a9cb16e1738432993bb88c51d638389 passed now but there is no any test from neutron_tempest_plugin.scenario run there | 15:53 |
slaweq | do You know if those tests were blacklisted somehow? | 15:53 |
sshnaidm | slaweq, yeah, it might be blacklisted in tempest | 15:54 |
chkumar|ruck | slaweq: let me edit the patch for you | 15:54 |
slaweq | chkumar|ruck: sshnaidm thx a lot | 15:54 |
sshnaidm | chkumar|ruck, need to exclude from skiplist? | 15:54 |
chkumar|ruck | sshnaidm: yes | 16:01 |
chkumar|ruck | sshnaidm: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/766252 | 16:01 |
sshnaidm | chkumar|ruck++ | 16:02 |
chkumar|ruck | slaweq: here we maintain the skiplist for all releases https://opendev.org/openstack/openstack-tempest-skiplist/src/branch/master/roles/validate-tempest/vars/tempest_skip.yml | 16:02 |
slaweq | chkumar|ruck++ thx | 16:03 |
rlandy | weshay|ruck: https://review.rdoproject.org/zuul/stream/2fa338cc07ca4c43aa626a058b82b92b?logfile=console.log stream tempest failure reproduces | 16:17 |
weshay|ruck | k... rlandy let's get a lp up | 16:17 |
rlandy | chkumar|ruck: ack | 16:18 |
rlandy | ugh | 16:18 |
rlandy | weshay|ruck: ^^ | 16:18 |
rlandy | sorry chkumar|ruck | 16:18 |
*** udesale has quit IRC | 16:18 | |
* rlandy has the rucks confused | 16:21 | |
rlandy | 2020-12-09 16:10:28.318861 | primary | Wednesday 09 December 2020 16:10:28 +0000 (0:00:00.439) 0:48:49.676 **** | 16:22 |
rlandy | 2020-12-09 16:10:28.345916 | primary | An exception occurred during task execution. To see the full traceback, use -vvv. The error was: KeyError: 'getpwuid(): uid not found: 1000' | 16:22 |
rlandy | reported as an ansible issue | 16:22 |
* rlandy checks ansible version | 16:22 | |
rlandy | weshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1907474 filling out details here | 16:29 |
openstack | Launchpad bug 1907474 in tripleo "CentOS8-stream: tempest fails to load tests to be skipped: "The error was: KeyError: 'getpwuid(): uid not found: 1000'"" [Undecided,New] | 16:29 |
weshay|ruck | rlandy, k.. marking it promotion-blocker so we can get it addressed via cix | 16:30 |
rlandy | weshay|ruck: ack - going to add repo diff output there | 16:30 |
weshay|ruck | rlandy, can you update collect-logs.. to ensure we collect /etc/passwd | 16:32 |
rlandy | yes | 16:32 |
weshay|ruck | https://logserver.rdoproject.org/65/31165/2/check/periodic-tripleo-ci-centos-8-standalone-centos8stream-master/2fa338c/logs/undercloud/etc/ not there | 16:32 |
weshay|ruck | group is | 16:32 |
marios|rover | weshay|ruck: chkumar|ruck: i am off in a bit need something before i do | 16:34 |
chkumar|ruck | marios|rover not from my side | 16:39 |
*** marios|rover is now known as marios|out | 16:45 | |
*** marios|out has quit IRC | 16:46 | |
*** rfolco|brb is now known as rfolco | 16:46 | |
beagles | does anyone know if there is a way to restrict tests that have been tagged as smoke tests with os_tempest | 16:51 |
beagles | we're trying to narrow the octavia tests that are run in the CI jobs | 16:52 |
weshay|ruck | there was always talk about those tags going away.. one of thoese things.. anyhoo https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset020.yml#L205-L210 is how we excplicitly set tempest config | 16:55 |
* beagles nods | 16:59 | |
beagles | weshay|ruck: yeah it appears that white|black lists are the only way to control which tests are run https://opendev.org/openstack/openstack-ansible-os_tempest/src/commit/49004fb05fa491275848a07020a36d4264ab6d49/tasks/tempest_run.yml#L17 | 17:00 |
beagles | weshay|ruck: thanks | 17:00 |
chkumar|ruck | beagles: https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/standalone-jobs.yaml#L734 | 17:01 |
chkumar|ruck | beagles: via job var, we can manipulate any thing | 17:01 |
chkumar|ruck | for octavia, we have fs062 | 17:02 |
beagles | chkumar|ruck: right, we were looking for a way to restrict them according to the tags the developers set in teh tempst plugin .. doesn't seem to be an option I think | 17:02 |
beagles | chkumar|ruck: specifying a subset of tests is workable though | 17:03 |
chkumar|ruck | beagles: specifing tags one needs to experiment | 17:04 |
chkumar|ruck | if stestr handles it, it will work with tempest also | 17:04 |
weshay|ruck | chkumar|ruck, is the notion that tags will be removed finally dead? | 17:04 |
beagles | chkumar|ruck: not sure how it would propagate to where tempest is actually called though | 17:04 |
beagles | probs would need a series of patches | 17:05 |
chkumar|ruck | weshay|ruck: it still exists like smoke tests | 17:05 |
weshay|ruck | ya.. but in the community.. the notion of removing those is dead? | 17:06 |
chkumar|ruck | beagles: if you show the what is needed, may be I can take a look | 17:06 |
weshay|ruck | been hearing that since david krantz roamed the earth | 17:06 |
chkumar|ruck | weshay|ruck: you mean git tag? | 17:06 |
weshay|ruck | no.. "smoke" | 17:06 |
chkumar|ruck | weshay|ruck: no no it still exists | 17:06 |
weshay|ruck | ok | 17:06 |
beagles | chkumar|ruck: actually is there a mechanism where regex's are used at all by os_tempest? | 17:07 |
beagles | chkumar|ruck: the reason I ask is fs 062 sets tempest_test_regex and I don't see where that is used | 17:07 |
beagles | seems like it shoudl be tempest_test_whitelist | 17:08 |
chkumar|ruck | beagles: https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/standalone-jobs.yaml#L759 | 17:08 |
beagles | chkumar|ruck: yep know about that... I'm looking the featureset | 17:08 |
beagles | chkumar|ruck: it's weird .. I think there might be some issues with how the fs is defined | 17:09 |
chkumar|ruck | tempest_test_whitelist: | 17:09 |
chkumar|ruck | - api | 17:09 |
chkumar|ruck | - scenario | 17:09 |
chkumar|ruck | it will run all the tests having api and scenario in the name | 17:09 |
beagles | yep | 17:09 |
beagles | k | 17:09 |
chkumar|ruck | it is just a regex matching | 17:10 |
beagles | ack | 17:11 |
*** TrevorV has joined #oooq | 17:19 | |
*** zbr has quit IRC | 17:27 | |
*** zbr has joined #oooq | 17:29 | |
*** ksambor has quit IRC | 17:31 | |
beagles | forgot the whilelist lines were all just regex's themselves doh | 17:40 |
*** zbr has quit IRC | 17:47 | |
*** zbr has joined #oooq | 17:49 | |
*** zbr has quit IRC | 17:56 | |
*** zbr has joined #oooq | 17:58 | |
-openstackstatus- NOTICE: The Gerrit service on review.opendev.org is currently responding slowly or timing out due to resource starvation, investigation is underway | 18:14 | |
*** derekh has quit IRC | 18:15 | |
*** dtantsur is now known as dtantsur|afk | 18:28 | |
*** chem has quit IRC | 18:31 | |
*** chem has joined #oooq | 18:33 | |
rlandy | weshay|ruck: https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_podman/vars/redhat-8.2.yml | 18:38 |
rlandy | weshay|ruck: https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_podman/vars/redhat-8.2.yml#L19 | 18:39 |
rlandy | weshay|ruck: https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/rhel-common/README.rst#L267 | 18:44 |
rlandy | weshay|ruck: https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/rhel-common/pre-install.d/01-module-configuration#L17 | 18:45 |
*** jpena is now known as jpena|off | 18:46 | |
rlandy | weshay|ruck: http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/config/release/dependency_ci/container_tools/repo_config.yaml | 18:53 |
weshay|ruck | rlandy, if the parent is marked non-voting do the jobs that inherit become non-voting? | 20:02 |
rlandy | weshay|ruck: yes - unless it's over-ridden | 20:03 |
rlandy | weshay|ruck: easiest is to create a mid-parent | 20:04 |
rlandy | with that override if you need it | 20:04 |
weshay|ruck | rlandy, https://review.rdoproject.org/r/#/c/30669/25/20203011_zuul.yaml | 20:14 |
weshay|ruck | that doesn't look like the same nodeset we use in ovb | 20:14 |
weshay|ruck | - nodeset: | 20:14 |
weshay|ruck | 265 name: centos-8 | 20:14 |
weshay|ruck | 266 # status: deprecated, use rdo-centos-8 instead | 20:14 |
weshay|ruck | 267 nodes: | 20:14 |
weshay|ruck | 268 - name: cloud-centos-8 | 20:14 |
weshay|ruck | 269 label: cloud-centos-8 | 20:14 |
weshay|ruck | 270 | 20:14 |
weshay|ruck | 271 | 20:14 |
weshay|ruck | 272 - nodeset: | 20:14 |
weshay|ruck | 273 name: upstream-centos-8-vexxhost | 20:14 |
weshay|ruck | 274 nodes: | 20:14 |
weshay|ruck | 275 - name: primary | 20:14 |
weshay|ruck | 276 label: upstream-centos-8-vexxhost | 20:15 |
* rlandy looks | 20:17 | |
rlandy | I think those two should be the same - tracing back | 20:19 |
rlandy | weshay|ruck: wrt https://review.opendev.org/c/openstack/tripleo-ci/+/766297/1/zuul.d/build-image.yaml | 20:20 |
rlandy | so your concern is that the victoria etc, branches will also go non-voting? | 20:21 |
weshay|ruck | rlandy, no.. I want all those jobs to be nv | 20:22 |
rlandy | ok - so then it's fine | 20:22 |
weshay|ruck | rlandy, thanks! /me updated commit msg to call that out | 20:22 |
rlandy | voted | 20:23 |
rlandy | man, new gerrit is super slow | 20:24 |
rlandy | tripleo-ovb-centos-8-vexxhost | 20:26 |
rlandy | hmmm ... looking for label | 20:26 |
weshay|ruck | rlandy, just wondering because his job got further than our 3rd party check jobs | 20:27 |
weshay|ruck | it should have failed in perm error on /var/lib/images | 20:28 |
* weshay|ruck wondering if he | 20:28 | |
weshay|ruck | he's using a more up to date node or older node.. | 20:28 |
rlandy | - nodeset: | 20:28 |
rlandy | name: tripleo-ovb-centos-8-primary-vexxhost | 20:28 |
rlandy | nodes: | 20:28 |
rlandy | - name: primary | 20:28 |
rlandy | label: upstream-centos-8-vexxhost | 20:28 |
weshay|ruck | centos release is the same | 20:30 |
weshay|ruck | CentOS Linux release 8.3.2011 | 20:30 |
rlandy | https://github.com/rdo-infra/rdo-jobs/search?q=tripleo-ovb-centos-8-primary-vexxhost | 20:30 |
rlandy | weshay|ruck: ^^ same as | 20:30 |
weshay|ruck | hrm | 20:31 |
weshay|ruck | k | 20:31 |
rlandy | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleo-rdo-base.yaml#L83 | 20:33 |
rlandy | looks ok | 20:33 |
rlandy | what am I missing? | 20:33 |
* weshay|ruck is just not sure how this maps.. to what we have in vex | 20:34 | |
weshay|ruck | nodeset: upstream-centos-8-psi-public | 20:34 |
weshay|ruck | branches: master | 20:34 |
weshay|ruck | override-checkout: stable/train | 20:34 |
weshay|ruck | attempts: 1 | 20:34 |
weshay|ruck | vars: | 20:35 |
weshay|ruck | registry_login_enabled: false | 20:35 |
weshay|ruck | quickstart_verbosity: -vv | 20:35 |
weshay|ruck | cloud_name: psi_public_nodepool_tripleo | 20:35 |
weshay|ruck | cloud_settings: | 20:35 |
weshay|ruck | psi_public_nodepool_tripleo: | 20:35 |
weshay|ruck | public_ip_net: internet | 20:35 |
weshay|ruck | undercloud_flavor: ci.s.xl | 20:35 |
weshay|ruck | baremetal_flavor: ci.s.xl | 20:35 |
weshay|ruck | bmc_flavor: ci.s.m | 20:35 |
weshay|ruck | extra_node_flavor: ci.s.m | 20:35 |
weshay|ruck | baremetal_image: centos8 | 20:35 |
rlandy | let's compare job on job | 20:36 |
weshay|ruck | rlandy, the centos-release version was the same.. so | 20:37 |
* weshay|ruck probably barking up the wrong tree | 20:37 | |
rlandy | well let's be sure | 20:38 |
rlandy | nodeset | 20:39 |
rlandy | tripleo-ovb-centos-8-primary-vexxhost | 20:39 |
rlandy | https://review.rdoproject.org/zuul/build/bafbc98b77b844e9a341a2cf055ab09d | 20:39 |
rlandy | weshay|ruck: https://meet.google.com/jbg-ghrv-ztv?pli=1&authuser=0 | 20:39 |
rlandy | lets do this live | 20:40 |
*** rfolco has quit IRC | 21:08 | |
*** slaweq has quit IRC | 21:12 | |
*** Goneri has quit IRC | 21:21 | |
*** slaweq has joined #oooq | 21:41 | |
*** jfrancoa has quit IRC | 22:12 | |
rlandy | weshay|ruck: have you seen collect logs just die like this: https://logserver.rdoproject.org/65/31165/3/check/periodic-tripleo-ci-centos-8-standalone-centos8stream-container-build-master/8910f26/ | 22:40 |
*** jbadiapa has quit IRC | 22:54 | |
*** TrevorV has quit IRC | 23:22 | |
*** slaweq has quit IRC | 23:23 | |
*** rlandy has quit IRC | 23:28 | |
*** zbr has quit IRC | 23:46 | |
*** zbr has joined #oooq | 23:48 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!