| *** saneax has quit IRC | 00:57 | |
| *** Goneri has quit IRC | 00:59 | |
| *** jmasud has quit IRC | 01:11 | |
| *** jmasud has joined #oooq | 02:30 | |
| *** saneax has joined #oooq | 02:35 | |
| *** jmasud has quit IRC | 02:46 | |
| *** udesale has joined #oooq | 03:21 | |
| *** jmasud has joined #oooq | 03:29 | |
| *** jmasud has quit IRC | 03:43 | |
| *** jmasud has joined #oooq | 03:56 | |
| *** ratailor has joined #oooq | 03:56 | |
| *** jmasud has quit IRC | 03:59 | |
| *** ykarel|away has joined #oooq | 04:05 | |
| *** skramaja has joined #oooq | 04:11 | |
| *** ykarel|away has quit IRC | 04:14 | |
| *** ykarel|away has joined #oooq | 04:14 | |
| *** ykarel|away is now known as ykarel | 04:22 | |
| *** udesale_ has joined #oooq | 04:49 | |
| *** udesale has quit IRC | 04:52 | |
| *** apetrich has joined #oooq | 04:57 | |
| *** marios has joined #oooq | 04:58 | |
| *** udesale has joined #oooq | 05:05 | |
| *** udesale_ has quit IRC | 05:06 | |
| *** udesale_ has joined #oooq | 05:22 | |
| *** udesale has quit IRC | 05:24 | |
| *** jmasud has joined #oooq | 05:27 | |
| *** skramaja has quit IRC | 05:29 | |
| *** skramaja has joined #oooq | 05:29 | |
| *** udesale__ has joined #oooq | 05:30 | |
| *** udesale_ has quit IRC | 05:33 | |
| *** saneax has quit IRC | 05:51 | |
| *** jtomasek has joined #oooq | 05:58 | |
| *** ysandeep|away is now known as ysandeep | 06:15 | |
| *** saneax has joined #oooq | 06:15 | |
| *** akahat is now known as akahat|PTO | 06:18 | |
| marios | is there some planned outage i missed rdo zuul gives 503 service unavailable | 06:31 |
|---|---|---|
| marios | jpena|off: ykarel: are you aware of something? ^ | 06:31 |
| marios | bhagyashri|rover: fyi ^^ | 06:32 |
| marios | bhagyashri|rover: did you hear/see somethign about that? | 06:32 |
| bhagyashri|rover | marios, same i am facing | 06:32 |
| bhagyashri|rover | marios, nope | 06:32 |
| marios | bhagyashri|rover: ack | 06:32 |
| marios | bhagyashri|rover: worth joining rhos-ops and asking if someone knows what's up | 06:33 |
| bhagyashri|rover | marios, ack | 06:33 |
| ykarel | marios, nope aware of zuul maintainance | 06:35 |
| ykarel | there is a planned for dlrn db today | 06:35 |
| ykarel | but that not affects zuul | 06:35 |
| chandankumar | marios: bhagyashri|rover https://lists.rdoproject.org/pipermail/dev/2020-September/009478.html | 06:36 |
| marios | ykarel: thanks | 06:36 |
| chandankumar | dlrn db migration not zuul one | 06:36 |
| marios | chandankumar: looking | 06:36 |
| marios | chandankumar: thanks | 06:36 |
| bhagyashri|rover | chandankumar, ykarel thanks | 06:37 |
| *** jpena|off is now known as jpena | 06:43 | |
| *** amoralej|off is now known as amoralej | 07:12 | |
| *** pojadhav is now known as pojadhav|brb | 07:27 | |
| *** saneax has quit IRC | 07:28 | |
| *** pojadhav|brb is now known as pojadhav | 07:39 | |
| *** dtantsur|afk is now known as dtantsur | 07:47 | |
| *** apetrich has quit IRC | 07:48 | |
| *** ratailor has quit IRC | 07:50 | |
| bhagyashri|rover | hey folks there is some issue from vexxhost side | 07:56 |
| bhagyashri|rover | so few nodes are down | 07:57 |
| bhagyashri|rover | you will also get Service Unavailable (503) https://review.rdoproject.org/zuul/api/info on rdoproject zuul | 07:58 |
| bhagyashri|rover | Please take a note of this ^ | 07:58 |
| *** ysandeep is now known as ysandeep|lunch | 07:59 | |
| *** jbadiapa has joined #oooq | 08:02 | |
| *** tosky has joined #oooq | 08:04 | |
| *** saneax has joined #oooq | 08:21 | |
| *** yolanda has joined #oooq | 08:22 | |
| marios | bhagyashri|rover: thank you | 08:25 |
| *** ykarel has quit IRC | 08:25 | |
| marios | bhagyashri|rover: do we have a cix/launchpad bug for it yet (probably a good idea to file one if we don't, so we can timestamp our outage) | 08:26 |
| *** ykarel has joined #oooq | 08:26 | |
| marios | bhagyashri|rover: or you may prefer to see if it isn't resolved by end of today then you should file something up to you | 08:26 |
| bhagyashri|rover | marios, i see on #rhos-ops jpena has created some ticket | 08:27 |
| bhagyashri|rover | <jpena> I hvae opened ticket #LXG-072763, and am trying to contact them directly via IRC | 08:28 |
| bhagyashri|rover | not sure which ticket is that | 08:28 |
| jpena | bhagyashri|rover: that's a ticket on vexxhost's ticketing system | 08:28 |
| bhagyashri|rover | jpena, ok thanks for the info | 08:28 |
| marios | bhagyashri|rover: ack thanks jpena | 08:29 |
| holser | morning folks | 08:29 |
| holser | is http://dashboard-ci.tripleo.org working? | 08:29 |
| marios | bhagyashri|rover: we'll likely need our own if it goes on together with a cix on the prod chain board etc | 08:29 |
| bhagyashri|rover | marios, sure i will open LP Bug | 08:29 |
| marios | holser: i wonder if it is relaetd to vexx outages i think it is hosted there now | 08:30 |
| marios | holser: also down for me just now | 08:30 |
| holser | marios gotcha, thanks | 08:30 |
| bhagyashri|rover | holser, yeah marios thanks | 08:30 |
| bhagyashri|rover | marios, is that ok to open LP Bug | 08:31 |
| marios | holser: check private | 08:32 |
| marios | bhagyashri|rover: yeah i think so ? all rdo jobs are down right now it is very dangerous for us | 08:32 |
| bhagyashri|rover | marios, ok | 08:32 |
| ykarel | upstream jobs are also impacted | 08:33 |
| marios | bhagyashri|rover: just one example from today https://review.opendev.org/#/c/748037/ - upstream jobs fine but breaks rdo jobs. if they aren't even running then we can merge things like this | 08:33 |
| marios | bhagyashri|rover: well it makes it even easier to merge things like this at least ^^ (i.e. they can't block as they are 3rd party but at least they should run so we have a chance to notice something breaking) | 08:33 |
| bhagyashri|rover | marios, agree | 08:35 |
| marios | holser: so you were saying about https://review.opendev.org/#/c/753817/ | 08:37 |
| *** saneax has quit IRC | 08:38 | |
| marios | holser: nice :) I was planning on revisiting the bug today \o/ | 08:38 |
| holser | I am saying we need to merge it | 08:38 |
| *** derekh has joined #oooq | 08:38 | |
| marios | holser: yeah it is fighting in the gate now? | 08:38 |
| holser | marios the problem is we enabled ha and used it for installation | 08:38 |
| holser | but not for upgrade | 08:38 |
| holser | marios bogdando retriggered it | 08:39 |
| marios | holser: ack great.. this is the second bug that was caused by the 'switch ha by default' change | 08:39 |
| marios | holser: then we make it voting ASAP https://review.opendev.org/#/c/749487/ i'll rebase that so it's ready | 08:39 |
| holser | marios thanks | 08:40 |
| bhagyashri|rover | marios, https://bugs.launchpad.net/tripleo/+bug/1896917 | 08:46 |
| openstack | Launchpad bug 1896917 in tripleo " problem with some vexxhost hypervisor" [Critical,Triaged] | 08:46 |
| marios | bhagyashri|rover: ack | 08:46 |
| bhagyashri|rover | marios, thanks :) | 08:48 |
| *** jfrancoa has joined #oooq | 09:06 | |
| *** saneax has joined #oooq | 09:10 | |
| marios | holser: so i got excited too quickly about that fix :( | 09:10 |
| holser | marios what fix? | 09:11 |
| marios | holser: https://bugs.launchpad.net/tripleo/+bug/1887159/comments/11 | 09:11 |
| openstack | Launchpad bug 1887159 in tripleo "centos8 standalone-upgrade ussuri fails Error: cluster is not configured " [Critical,Fix released] - Assigned to Marios Andreou (marios-b) | 09:11 |
| marios | holser: the standalone-upgrade-ussuri fix from alex | 09:11 |
| marios | holser: sorry wrong bug :D that's the original | 09:11 |
| marios | holser: i mean * https://bugs.launchpad.net/tripleo/+bug/1895822/comments/11 | 09:12 |
| openstack | Launchpad bug 1895822 in tripleo "centos8 standalone-upgrade-ussuri fails tempest ping router IP" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 09:12 |
| holser | marios we checked the bash script for upgrade | 09:12 |
| marios | holser: the fix you pointed me to | 09:12 |
| holser | and deployment had /usr/share/openstack-tripleo-heat-templates/environments/docker-ha.yaml | 09:12 |
| holser | and upgrade bash script didn't | 09:13 |
| holser | when I added it manually and ran upgrade so it passed | 09:13 |
| holser | so I don't know what https://review.opendev.org/#/c/742418/13/zuul.d/standalone-jobs.yaml is doing but it doesn' | 09:14 |
| marios | holser: ah i see... so maybe that's it... the fix in https://opendev.org/openstack/tripleo-ci/src/commit/019f2fe1844d94215224104e8c6af88faa715753/zuul.d/standalone-jobs.yaml#L150-L153 passes those environment files for the deployment only | 09:14 |
| holser | t pupulate. proper env file in upograde | 09:14 |
| holser | marios exactly | 09:14 |
| marios | holser: ok cool, will add another note on the bug but hopefully we will see green test https://review.opendev.org/749487 | 09:15 |
| holser | yep | 09:15 |
| marios | holser: sorry test there https://review.opendev.org/739457 | 09:15 |
| holser | thus I asked for cockpit to find one job I can reproduce with alex patch | 09:15 |
| holser | to see how it goes | 09:15 |
| marios | holser: https://bugs.launchpad.net/tripleo/+bug/1895822/comments/12 | 09:18 |
| openstack | Launchpad bug 1895822 in tripleo "centos8 standalone-upgrade-ussuri fails tempest ping router IP" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 09:18 |
| *** ysandeep|lunch is now known as ysandeep | 09:21 | |
| *** yolanda has quit IRC | 09:38 | |
| *** yolanda has joined #oooq | 09:39 | |
| *** dtantsur is now known as dtantsur|brb | 09:43 | |
| *** pojadhav is now known as pojadhav|afk | 09:56 | |
| sshnaidm | ykarel, hi, what is the last release you put for ansible-lint? | 10:09 |
| sshnaidm | I see v4.3.5 as last in github | 10:09 |
| ykarel | sshnaidm, u mean in rdo-jobs? | 10:10 |
| sshnaidm | ykarel, yeah | 10:10 |
| ykarel | sshnaidm, i didn't changed that, it's v4.1.1a6 | 10:10 |
| sshnaidm | ack | 10:10 |
| ykarel | although it's not merged | 10:11 |
| ykarel | https://review.rdoproject.org/r/#/c/29567/ | 10:11 |
| sshnaidm | ykarel, done | 10:12 |
| ykarel | sshnaidm, zuul is down, so nothing will run/merge | 10:13 |
| sshnaidm | ykarel, ah, just saw Javiers mail | 10:14 |
| bhagyashri|rover | sshnaidm, sorry i pushed the patch just few min after you push | 10:23 |
| sshnaidm | bhagyashri|rover, np, I'll update | 10:23 |
| bhagyashri|rover | fixed few of the review comments from marios | 10:24 |
| bhagyashri|rover | sshnaidm, but i have added you r changes also in th erecent push | 10:24 |
| bhagyashri|rover | recent | 10:24 |
| bhagyashri|rover | so i have covered it | 10:24 |
| bhagyashri|rover | in patch set 5 | 10:25 |
| bhagyashri|rover | sshnaidm, https://review.opendev.org/#/c/753760/5 here | 10:25 |
| jpena | bhagyashri|rover, marios: things should be back to normal at vexxhost, Zuul is now up | 10:54 |
| bhagyashri|rover | hey folks now you can access this https://review.rdoproject.org/zuul/status | 10:54 |
| bhagyashri|rover | jpena, yeah thanks | 10:54 |
| bhagyashri|rover | http://images.rdoproject.org/ is still down | 10:56 |
| bhagyashri|rover | jpena, ^ | 10:56 |
| jpena | bhagyashri|rover: yes, ykarel mentioned it. It should be up now | 10:57 |
| *** dtantsur|brb is now known as dtantsur | 10:57 | |
| bhagyashri|rover | yes it's working now | 10:57 |
| *** sanjayu_ has joined #oooq | 11:00 | |
| *** saneax has quit IRC | 11:02 | |
| marios | jpena: great thanks | 11:05 |
| bhagyashri|rover | weshay|ruck, good morning ! today we hit with this https://bugs.launchpad.net/tripleo/+bug/1896917 | 11:36 |
| openstack | Launchpad bug 1896917 in tripleo " problem with some vexxhost hypervisor" [Critical,Triaged] | 11:36 |
| bhagyashri|rover | https://trello.com/c/A5gqcY0R/1681-cixlp1896917allmultiple-rdo-infrastructure-instances-are-down-including-zuul | 11:36 |
| *** jpena is now known as jpena|lunch | 11:37 | |
| bhagyashri|rover | now every things is working from last half hr | 11:40 |
| *** ysandeep is now known as ysandeep|brb | 11:40 | |
| weshay|ruck | bhagyashri|rover, looking | 11:41 |
| weshay|ruck | bhagyashri|rover, thanks for properly documenting it in a bug :) | 11:42 |
| weshay|ruck | ++ | 11:42 |
| bhagyashri|rover | :) | 11:43 |
| bhagyashri|rover | more updates on CIX | 11:43 |
| arxcruz | bhagyashri|rover: is the linter failure fixed already ? | 11:49 |
| bhagyashri|rover | arxcruz, patches are up | 11:49 |
| *** apetrich has joined #oooq | 11:50 | |
| bhagyashri|rover | arxcruz, https://review.opendev.org/#/q/topic:linters-fix+(status:open+OR+status:merged) | 11:50 |
| bhagyashri|rover | one patch got merged others are still under review | 11:50 |
| arxcruz | bhagyashri|rover: thanks, I'll wait the patches get merged to recheck my patch :) | 11:50 |
| bhagyashri|rover | sure | 11:50 |
| bhagyashri|rover | or you can make dependent on ^ according to the repo | 11:51 |
| bhagyashri|rover | arxcruz, ^ | 11:51 |
| arxcruz | well, I'll need to wait the patch get merged anyway, plus mine already have some depends-on, i don't want to mess with | 11:52 |
| weshay|ruck | sshnaidm, fyi.. I think our other vexhost infra is down.. /me looking | 11:53 |
| bhagyashri|rover | arxcruz, ack | 11:53 |
| sshnaidm | weshay|ruck, jpena|lunch sent a mail | 11:53 |
| weshay|ruck | k | 11:53 |
| weshay|ruck | promoter in vex is up | 11:54 |
| weshay|ruck | cockpit is hard down | 11:54 |
| weshay|ruck | oh it's back | 11:54 |
| weshay|ruck | every day this week.. it's been something | 11:54 |
| weshay|ruck | ah no.. dashboard is down | 12:02 |
| bhagyashri|rover | yes dashboard is down | 12:04 |
| *** rlandy has joined #oooq | 12:07 | |
| weshay|ruck | marios, did you catch all of Alex's updates to the upgrade bugs on stein/ussri | 12:10 |
| *** amoralej is now known as amoralej|lunch | 12:10 | |
| marios | weshay|ruck: o/ you mean that https://bugs.launchpad.net/tripleo/+bug/1895822/comments/12 https://review.opendev.org/#/c/753817/2/config/general_config/featureset056.yml ? or something else still | 12:11 |
| openstack | Launchpad bug 1895822 in tripleo "centos8 standalone-upgrade-ussuri fails tempest ping router IP" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 12:11 |
| weshay|ruck | marios, and https://bugs.launchpad.net/bugs/1896537 | 12:16 |
| openstack | Launchpad bug 1896537 in tripleo "[stein] standalone-upgrade failing tempest" [Critical,Triaged] | 12:16 |
| marios | weshay|ruck: no missed that one will check it | 12:18 |
| weshay|ruck | thanks | 12:19 |
| *** ysandeep|brb is now known as ysandeep | 12:21 | |
| weshay|ruck | anyone ever see something like | 12:26 |
| weshay|ruck | 2020-09-24 11:15:28,751 56830 DEBUG urllib3.connectionpool [ ] http://192.168.24.1:8787 "GET /v2/tripleomaster/openstack-ironic-pxe/tags/list HTTP/1.1" 404 245 | 12:26 |
| weshay|ruck | 2020-09-24 11:15:28,752 56830 INFO tripleo_common.image.image_uploader [ ] Non-2xx: id 06c2c15025d0b3b66dba74bf68ee7e6fa95091fb, status 404, reason Not Found, text <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> | 12:26 |
| weshay|ruck | <html><head> | 12:26 |
| weshay|ruck | <title>404 Not Found</title> | 12:26 |
| weshay|ruck | </head><body> | 12:26 |
| weshay|ruck | <h1>Not Found</h1> | 12:26 |
| weshay|ruck | <p>The requested URL /v2/tripleomaster/openstack-ironic-pxe/tags/list was not found on this server.</p> | 12:26 |
| weshay|ruck | </body></html> | 12:26 |
| weshay|ruck | https://e04cd551585f8367e58f-97f1bb270b356fde15ebdabc0499edaa.ssl.cf2.rackcdn.com/753362/5/gate/tripleo-ci-centos-8-undercloud-containers/f8c2617/logs/undercloud/var/log/tripleo-container-image-prepare.log | 12:26 |
| *** pojadhav|afk is now known as pojadhav | 12:27 | |
| chandankumar | sshnaidm: Does ubi8 images mirrored somewhere apart from RH registry? | 12:36 |
| sshnaidm | chandankumar, no idea | 12:36 |
| *** jpena|lunch is now known as jpena | 12:38 | |
| weshay|ruck | bhagyashri|rover, are you crazy busy atm? | 12:39 |
| bhagyashri|rover | weshay|ruck, just checking | 12:39 |
| bhagyashri|rover | weshay|ruck, i dont see this is consistent failure | 12:40 |
| bhagyashri|rover | we are hitting this only in the recent run | 12:41 |
| weshay|ruck | bhagyashri|rover, how are you checking btw.. via elastic-recheck/logstash? | 12:41 |
| bhagyashri|rover | weshay|ruck, https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-undercloud-containers | 12:42 |
| weshay|ruck | bhagyashri|rover, hoping you or rfolco can follow up on https://trello.com/c/74UKSE5Y/1644-cixlp1892682tripleociproa-periodic-scenario-010-ovn-provider-job-failing-with-ovn-provider-is-not-enabled | 12:42 |
| bhagyashri|rover | just checked here | 12:42 |
| weshay|ruck | k | 12:42 |
| weshay|ruck | does someone w/ a fedora laptop.. mind pasting their /etc/yum.repos.d/fedora.repo and fedora-updates.repo config for me? | 12:44 |
| bhagyashri|rover | weshay|ruck, yes, regarding "Periodic Scenario 010 ovn provider job failing with ovn provider is not enabled" we are still hitting this issue even after the patches got merge | 12:44 |
| weshay|ruck | bhagyashri|rover, k.. do you know exactly where the patch is w/ regards to promoting through component, integration? | 12:45 |
| bhagyashri|rover | let me check the lates update | 12:45 |
| *** rfolco has joined #oooq | 12:49 | |
| bhagyashri|rover | weshay|ruck, not seen | 12:52 |
| weshay|ruck | bhagyashri|rover, maybe we'll get through scrum quickly.. and I look at it w/ you | 12:53 |
| bhagyashri|rover | ack | 12:53 |
| weshay|ruck | thanks | 12:53 |
| weshay|ruck | rfolco, hang on after scrum real quick re: https://trello.com/c/74UKSE5Y/1644-cixlp1892682tripleociproa-periodic-scenario-010-ovn-provider-job-failing-with-ovn-provider-is-not-enabled | 12:54 |
| rfolco | weshay|ruck, ok | 12:54 |
| *** rfolco is now known as rfolco|ruck | 12:54 | |
| *** ykarel_ has joined #oooq | 12:56 | |
| *** ykarel has quit IRC | 12:58 | |
| *** skramaja has quit IRC | 12:58 | |
| *** skramaja has joined #oooq | 12:59 | |
| ysandeep | weshay|ruck, rlandy got green runs on sc012 - https://code.engineering.redhat.com/gerrit/#/c/212706/ please review when you get time. | 13:00 |
| *** Goneri has joined #oooq | 13:00 | |
| ysandeep | - periodic-tripleo-ci-rhel-8-scenario012-standalone-baremetal-rhos-16.2 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/7581a61daf9242fa9a3ddf0de601481c : SUCCESS in 56m 26s (non-voting) | 13:00 |
| ysandeep | - periodic-tripleo-ci-rhel-8-scenario012-standalone-compute-rhos-16.2 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/12a09d8679564b0ca2003b84c13c21df : SUCCESS in 58m 50s (non-voting) | 13:00 |
| ysandeep | I had a to add a pre to pull python3-virtualbmc from upstream repos | 13:00 |
| rfolco|ruck | arxcruz, rfolco, zbr, sshnaidm, rlandy, marios, ysandeep, bhagyashris, svyas, pojadhav, akahat, weshay, chandankumar | 13:02 |
| rfolco|ruck | ping scrum | 13:02 |
| rlandy | ysandeep: looks good - train deps for both though? | 13:04 |
| rlandy | or just 16.2? | 13:04 |
| ysandeep | rlandy, yeah doesn't make much difference to get latest python3-virtualbmc | 13:05 |
| ysandeep | for 17 | 13:05 |
| *** jpena is now known as jpena|off | 13:15 | |
| weshay|ruck | bhagyashri|rover, rfolco|ruck https://review.rdoproject.org/r/#/c/29167/ | 13:18 |
| *** amoralej|lunch is now known as amoralej | 13:19 | |
| sshnaidm | arxcruz, commented on review | 13:20 |
| *** ykarel_ has quit IRC | 13:20 | |
| *** ykarel has joined #oooq | 13:21 | |
| arxcruz | sshnaidm: this is not an issue | 13:21 |
| arxcruz | once you do a keypair create | 13:21 |
| sshnaidm | arxcruz, look at other cleanup scripts, we need to delete only old keys, like >6 hours ago created | 13:21 |
| arxcruz | sshnaidm: this is not a problem, when you do a keypair create | 13:21 |
| *** jpena|off is now known as jpena | 13:21 | |
| arxcruz | this is the only time you get the private key | 13:22 |
| arxcruz | that is already in the job | 13:22 |
| arxcruz | and the other vm's that uses the public key, already have it | 13:22 |
| arxcruz | we would have this issue only if we would re-use the key later | 13:22 |
| arxcruz | which is not the case | 13:23 |
| sshnaidm | arxcruz, yeah, makes sense | 13:24 |
| arxcruz | sshnaidm: yeah, I discuss this before with weshay|ruck | 13:24 |
| sshnaidm | arxcruz, but I'd rather check why key deletion doesn't work in job before creating a separate script to clean it: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/roles/ovb-manage/tasks/ovb-delete-stack.yml#L103 | 13:24 |
| arxcruz | sshnaidm: sure, but some times it just fails because openstack | 13:25 |
| sshnaidm | arxcruz, what do you mean? | 13:25 |
| *** udesale_ has joined #oooq | 13:26 | |
| arxcruz | lol | 13:26 |
| arxcruz | sshnaidm: it fails because $random and there's ignore_errors there | 13:26 |
| arxcruz | i think the stack doesn't delete the key when it's in error state and we re-run the cleanup stack script | 13:26 |
| sshnaidm | arxcruz, do you have any logs for that? just curios | 13:27 |
| sshnaidm | maybe this part never works? | 13:27 |
| arxcruz | sshnaidm: no, i just said that's what i think, i can try to check the logs | 13:27 |
| arxcruz | just my though | 13:27 |
| *** udesale__ has quit IRC | 13:28 | |
| ysandeep | rlandy, fixing that | 13:28 |
| sshnaidm | arxcruz, I think it worth checking | 13:28 |
| sshnaidm | chandankumar, marios do you want to talk now? Or to wait for weshay|ruck | 13:30 |
| chandankumar | sshnaidm: we can wait for weshay|ruck | 13:31 |
| marios | sshnaidm: don't mind | 13:32 |
| marios | sshnaidm: but its only half hour? maybe better wait so we don't repeat | 13:32 |
| *** sanjayu_ has quit IRC | 13:35 | |
| chandankumar | rfolco|ruck: please have a look https://review.opendev.org/#/c/752643/ when free | 13:42 |
| ysandeep | rlandy, https://code.engineering.redhat.com/gerrit/#/c/212706/5/playbooks/virtualbmc-from-upstream.yaml fixed, thanks for review :) | 13:44 |
| rlandy | ysandeep; thanks +2 | 13:45 |
| ysandeep | :) | 13:45 |
| *** TrevorV has joined #oooq | 13:46 | |
| bhagyashri|rover | weshay|ruck, thanks | 13:46 |
| rfolco|ruck | bhagyashri|rover, I am going to update ovn cards ok ? let me retrace weshay|ruck steps | 13:49 |
| bhagyashri|rover | rfolco|ruck, ok | 13:49 |
| weshay|ruck | ysandeep, 0/ were you planning on introducing the DF to your thoughts today or some other time? | 13:54 |
| ysandeep | weshay|ruck, next week , I want to brainstorm more before i present my idea | 13:54 |
| chandankumar | rfolco|ruck: https://review.opendev.org/#/c/752643/ | 13:55 |
| weshay|ruck | ysandeep, k. perfect thank you | 13:56 |
| rlandy | ysandeep: promoted 16.2 yesterday | 13:59 |
| rlandy | ovb failed - but that seemed to be the case sporadically | 14:00 |
| chandankumar | rlandy: weshay|ruck marios sshnaidm https://meet.google.com/zio-jhas-ykp?authuser=0 | 14:00 |
| ysandeep | rlandy, ack o/ thanks! | 14:01 |
| ysandeep | rlandy, fyi.. imag build in bm jobs is busted, https://review.opendev.org/#/c/753540/ will fix it | 14:01 |
| ysandeep | image* | 14:01 |
| *** ysandeep is now known as ysandeep|mtg | 14:02 | |
| chandankumar | sshnaidm: we are waiting for you | 14:02 |
| *** ysandeep|mtg is now known as ysandeep | 14:25 | |
| rlandy | ysandeep: will vote after tox error is fixed | 14:35 |
| ysandeep | rlandy: thx, yeah that linter job is busted | 14:36 |
| *** sshnaidm is now known as sshnaidm|afk | 14:48 | |
| *** ysandeep is now known as ysandeep|away | 14:58 | |
| ykarel | bhagyashri|rover, rfolco|ruck is https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039&branch=stable%2Ftrain# known? | 15:01 |
| ykarel | noticed in one of my train patch | 15:01 |
| rfolco|ruck | ykarel, previously it was failing due to memory bug, and quota... | 15:04 |
| rfolco|ruck | 2020-09-23 21:05:36.280099 | TASK [ovb-manage : Recover idnum] | 15:04 |
| rfolco|ruck | 2020-09-23 21:05:36.930493 | primary | cat: /home/zuul/workspace/ovb/idnum: No such file or directory | 15:04 |
| ykarel | no ^ is not real issue | 15:05 |
| ykarel | me posting patch | 15:05 |
| rfolco|ruck | ykarel, I can file a bug then | 15:06 |
| rfolco|ruck | so you add to the patch | 15:06 |
| ykarel | rfolco|ruck, https://review.rdoproject.org/r/29617 | 15:06 |
| ykarel | can get ^ merged, and re run job | 15:06 |
| rfolco|ruck | ok | 15:07 |
| *** ykarel is now known as ykarel|away | 15:08 | |
| bhagyashri|rover | ykarel|away, ack | 15:23 |
| *** ykarel|away has quit IRC | 15:28 | |
| rlandy | weshay|ruck: have a few minutes to discuss the diff script logic? | 15:51 |
| rlandy | the string splits are not where I would expect - if I am reading/testing this correctly | 15:52 |
| marios | rlandy: rfolco|ruck: sshnaidm|afk: easy one needs votes please & weshay|ruck fyi you asked about this couple days ago https://review.opendev.org/753034 & https://bugs.launchpad.net/tripleo/+bug/1895138/comments/11 | 15:53 |
| openstack | Launchpad bug 1895138 in tripleo "centos-8 standalone-upgrade-ussuri fails build-test-packages issue creating /root/DLRN" [Critical,Fix released] - Assigned to wes hayutin (weshayutin) | 15:53 |
| *** jfrancoa has quit IRC | 15:54 | |
| marios | weshay|ruck: why rebase? | 15:55 |
| *** jpena is now known as jpena|off | 15:56 | |
| weshay|ruck | marios, it was failing check | 15:57 |
| weshay|ruck | marios, I'll get it merged.. don't worry :) | 15:58 |
| *** udesale_ has quit IRC | 16:06 | |
| *** dtantsur is now known as dtantsur|afk | 16:17 | |
| *** amoralej is now known as amoralej|off | 16:20 | |
| bhagyashri|rover | weshay|ruck, chandankumar please review https://review.rdoproject.org/r/#/c/29619/ | 16:26 |
| weshay|ruck | rfolco|ruck, can you check if that's needed in ussuri/train | 16:31 |
| rfolco|ruck | weshay|ruck, this ? https://review.opendev.org/753034 | 16:32 |
| bhagyashri|rover | weshay|ruck, for train this job is stable | 16:32 |
| weshay|ruck | rfolco|ruck, is there train/ussuri branches in tqe? | 16:32 |
| bhagyashri|rover | and for ussuri we are getting different issue https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-ussuri/4441da7/logs/undercloud/var/log/tempest/stestr_results.html.gz | 16:32 |
| weshay|ruck | no | 16:32 |
| weshay|ruck | rfolco|ruck, https://review.rdoproject.org/r/#/c/29619/ | 16:32 |
| rfolco|ruck | ah ovn | 16:33 |
| rfolco|ruck | train is good, ussuri is bad | 16:34 |
| bhagyashri|rover | for ussuri this fix will not work because we are not hitting with NotImplemented error | 16:35 |
| * bhagyashri|rover out | 16:38 | |
| rfolco|ruck | what happened was that we added it to train skiplist here https://review.opendev.org/#/c/751162/4/roles/validate-tempest/vars/tempest_skip.yml | 16:43 |
| weshay|ruck | sshnaidm|afk, make sure this looks right to you please https://review.opendev.org/#/c/751273/ | 16:44 |
| rfolco|ruck | weshay|ruck, add to skip list like we did for train ? | 16:47 |
| sshnaidm|afk | weshay|ruck, commented | 16:48 |
| sshnaidm|afk | weshay|ruck, https://zuul.opendev.org/t/openstack/build/5753f4317b484beab35d6fe711eac9e5/log/job-output.txt#926 | 16:48 |
| weshay|ruck | rfolco|ruck, https://review.rdoproject.org/r/#/c/29619/1/zuul.d/standalone-jobs.yaml is for the master job only | 16:49 |
| weshay|ruck | rfolco|ruck, I did not say skip | 16:49 |
| weshay|ruck | or skiplist | 16:49 |
| weshay|ruck | I'm wondering if the tempest config change here: https://review.rdoproject.org/r/#/c/29619/1/zuul.d/standalone-jobs.yaml | 16:49 |
| rfolco|ruck | you mixing 2 bugs | 16:49 |
| weshay|ruck | is also required for the ussuri/train jobs... or perhaps it's inherited | 16:49 |
| rfolco|ruck | the ussuri issue has nothing with the master bug | 16:50 |
| weshay|ruck | rfolco|ruck, ok.. so the answer there is ... | 16:50 |
| rfolco|ruck | weshay|ruck, master is hitting https not implemented | 16:50 |
| weshay|ruck | no weshay|ruck that tempest setting is not required for ussuri | 16:50 |
| weshay|ruck | ok.. rfolco thanks for checking | 16:50 |
| rfolco|ruck | weshay|ruck, and train/ussuri is hitting 'not a valid option for a TCP protocol listener....' | 16:51 |
| rfolco|ruck | weshay|ruck, I am adding to ussuri skiplist | 16:52 |
| weshay|ruck | rfolco|ruck, ya.. so re: ussuri https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-ussuri/4441da7/logs/undercloud/var/log/tempest/stestr_results.html.gz | 16:52 |
| weshay|ruck | just the one test is failing.. i think it's fine to bug, and skip | 16:52 |
| rfolco|ruck | bug is https://bugs.launchpad.net/tripleo/+bug/1895705 | 16:53 |
| openstack | Launchpad bug 1895705 in tripleo "[train|ussuri] scen10-ovn-provider-standalone: insert-headers is not a valid option for a TCP protocol listener" [Critical,Triaged] | 16:53 |
| rfolco|ruck | just updated w/ ussuri info | 16:53 |
| weshay|ruck | rfolco|ruck, and periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-train is passing | 16:53 |
| weshay|ruck | cool.. rfolco|ruck thanks | 16:53 |
| rfolco|ruck | because of skiplist | 16:53 |
| rfolco|ruck | :) | 16:53 |
| weshay|ruck | not every little chick is destined to be a swan | 16:54 |
| weshay|ruck | sometimes the tests fail | 16:54 |
| rfolco|ruck | weshay|ruck, sometimes... where? | 16:55 |
| weshay|ruck | sshnaidm|afk, is that not a legal key.. even if i have this? https://review.opendev.org/#/c/751273/2/tasks/sova.yml | 16:56 |
| weshay|ruck | is the audit log captured in the logstash file.. I thought I looked and it was not | 16:56 |
| weshay|ruck | can recheck | 16:57 |
| sshnaidm|afk | weshay|ruck, every pattern should include logstash, id, type, whatever | 16:58 |
| sshnaidm|afk | weshay|ruck, selinux is not a legal key | 16:58 |
| sshnaidm|afk | weshay|ruck, files config is correct in this case | 16:58 |
| rfolco|ruck | weshay|ruck, btw the ovn octavia worked https://review.rdoproject.org/r/#/c/29620/ | 16:58 |
| sshnaidm|afk | weshay|ruck, it's not in logstash, but pattern requires this key to be set | 16:59 |
| weshay|ruck | k.. so https://review.opendev.org/#/c/751273/2/tasks/sova.yml is correct, but this https://review.opendev.org/#/c/751273/2/vars/sova-patterns.yml still needs to be s/selinux/logstash/ | 17:00 |
| weshay|ruck | rfolco|ruck, ya.. thanks I know that's why + wf the patch | 17:01 |
| rfolco|ruck | weshay|ruck, meh I am always behind | 17:01 |
| *** derekh has quit IRC | 17:01 | |
| rfolco|ruck | https://review.opendev.org/754198 Add octavia test to ussuri skiplist | 17:07 |
| *** chem has quit IRC | 17:16 | |
| *** chem has joined #oooq | 17:18 | |
| rfolco|ruck | weshay|ruck, have you seen this ? | 17:19 |
| rfolco|ruck | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_0a9/753030/2/gate/tripleo-ci-centos-8-containers-multinode/0a93c69/logs/undercloud/home/zuul/dlrn.log | 17:19 |
| *** marios is now known as marios|out | 17:25 | |
| *** skramaja has quit IRC | 17:33 | |
| weshay|ruck | one sec | 17:44 |
| weshay|ruck | rfolco|ruck, vexhost was down | 17:53 |
| rlandy | rfolco|ruck: hey is there a fix for the openstack-tox-linters job? | 17:54 |
| weshay|ruck | rlandy, ya.. sagi has 3 of them | 17:54 |
| rlandy | three ok | 17:54 |
| weshay|ruck | rfolco|ruck, ya.. my best guess is that was related to vexx... it's in recheck now.. | 18:00 |
| rfolco|ruck | k | 18:00 |
| *** tosky has quit IRC | 18:22 | |
| holser | hey folks | 18:50 |
| holser | can anyone merge https://review.opendev.org/#/c/753386/ | 18:50 |
| *** marios|out has quit IRC | 19:03 | |
| rlandy | holser: w+'ed | 19:07 |
| * holser nods | 19:07 | |
| weshay|ruck | holser, 0/ looking | 19:19 |
| *** TrevorV has quit IRC | 19:35 | |
| rlandy | weshay|ruck: hey ... | 19:36 |
| weshay|ruck | rlandy, hey urself | 19:37 |
| rlandy | weshay|ruck: pls ping if/when you can chat re: repoquery script design | 19:37 |
| weshay|ruck | I can chat | 19:37 |
| rlandy | weshay|ruck: meet.google.com/nxe-yozk-zmf | 19:39 |
| weshay|ruck | 14 from dnf.subject import Subject | 19:45 |
| weshay|ruck | 15 import hawkey | 19:45 |
| weshay|ruck | rlandy, ^ | 19:45 |
| *** rfolco|ruck is now known as rfolco|ruck|brb | 19:53 | |
| sshnaidm|afk | weshay|ruck, rlandy +w? https://review.opendev.org/#/c/753685/ | 20:33 |
| *** rfolco|ruck|brb has quit IRC | 20:36 | |
| *** rfolco|ruck|brb has joined #oooq | 20:37 | |
| *** rfolco|ruck|brb is now known as rfolco|ruck | 20:37 | |
| *** jtomasek has quit IRC | 21:11 | |
| *** rfolco|ruck has quit IRC | 21:11 | |
| *** apetrich has quit IRC | 21:28 | |
| weshay|ruck | rlandy, fyi https://review.opendev.org/#/c/754221/ | 21:39 |
| rlandy | weshay|ruck: why the move to common ...https://review.opendev.org/#/c/754221/ | 21:40 |
| * rlandy needs to think about this ... | 21:40 | |
| rlandy | we use ipa is no nova_join | 21:41 |
| rlandy | https://github.com/openstack/tripleo-quickstart/blob/master/config/general_config/featureset039.yml#L79 | 21:46 |
| rlandy | ^^ novajoin is for earlier releases? | 21:47 |
| weshay|ruck | rlandy, ya.. if meta is setup correctly.. which we'll find out.. moving it from a tqe role to tqe/extras-common/defaults should be fine | 21:51 |
| weshay|ruck | rlandy, https://logserver.rdoproject.org/55/730355/14/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/da05a77/logs/undercloud/home/zuul/undercloud.conf.txt.gz | 21:51 |
| weshay|ruck | https://logserver.rdoproject.org/55/730355/14/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/da05a77/logs/undercloud/var/log/extra/dump_variables_vars.yaml.txt.gz | 21:51 |
| weshay|ruck | but we're setting it to false | 21:52 |
| weshay|ruck | so.. we're trying to figure it out | 21:52 |
| rlandy | ok - just need to make sure the ipa jobs set that to false as well | 21:52 |
| rlandy | weshay|ruck: did you get http://pastebin.test.redhat.com/905464? | 22:12 |
| rlandy | installed dnf by pip and rpm | 22:12 |
| weshay|ruck | rlandy, your virtenv has to use system-site-packages | 22:14 |
| weshay|ruck | that's the only way you can use the python modules from rpm | 22:14 |
| rlandy | ah | 22:15 |
| rlandy | k- now we are getting somewhere | 22:19 |
| rlandy | brb | 22:25 |
| *** rlandy has quit IRC | 23:44 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!