| *** dviroel|afk is now known as dviroel | 00:00 | |
| dviroel | rlandy|bbl: so yeah, quay.ceph.io is down atm | 00:10 |
|---|---|---|
| dviroel | we can switch to other registries, like in master branch | 00:10 |
| dviroel | rlandy|bbl: maybe something like this: https://review.opendev.org/c/openstack/tripleo-common/+/841512 | 00:22 |
| * dviroel need to go out o/ | 00:24 | |
| *** dviroel is now known as dviroel|out | 00:24 | |
| *** dviroel|out is now known as dviroel | 00:59 | |
| rlandy|bbl | dviroel: yeah - it's an option if it's not back tomorrow | 01:08 |
| rlandy|bbl | giving up for today | 01:08 |
| *** rlandy|bbl is now known as rlandy|out | 01:09 | |
| dviroel | o/ | 01:09 |
| dviroel | good night | 01:09 |
| dviroel | testing scn001 and scn004 here: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/841513 | 01:10 |
| dviroel | also hits stable/train :( | 01:18 |
| *** ysandeep|rover|out is now known as ysandeep|rover | 01:19 | |
| * ysandeep|rover catching up with all the action | 01:19 | |
| dviroel | ysandeep|rover: hello there, hackmd has all the info I think - and Launchpad too | 01:20 |
| ysandeep|rover | dviroel, ++ thanks! | 01:22 |
| dviroel | ysandeep|rover: o/ np | 01:22 |
| ysandeep|rover | I will wait for quay.ceph.io to comeback, if not in few hour I will chat with franceso when he comes online if we can go ahead with your patch. | 01:22 |
| dviroel | ysandeep|rover: yeah, agree. But since other branches are also affected, will need mode tripleo-common patches to move foward | 01:23 |
| dviroel | ysandeep|rover: so, hope that quay.ceph.io comes back | 01:24 |
| dviroel | hahaha | 01:24 |
| dviroel | ysandeep|rover: going out for today o/ see you later | 01:24 |
| ysandeep|rover | o/ | 01:24 |
| *** dviroel is now known as dviroel|out | 01:24 | |
| ysandeep|rover | see you tomorrow o/ | 01:24 |
| ysandeep|rover | rlandy|out, >> https://review.rdoproject.org/r/c/testproject/+/36140 - no success with networking patch on w c8 | 02:08 |
| ysandeep|rover | I think dlrn is not building rpm for ovsdbapp package - https://logserver.rdoproject.org/40/36140/48/check/periodic-tripleo-ci-build-containers-ubi-8-push-wallaby/3f18fce/etc/yum.repos.d/ no gating repo | 02:08 |
| *** ysandeep|rover is now known as ysandeep|afk | 02:26 | |
| *** ysandeep|afk is now known as ysandeep|rover | 04:44 | |
| *** soniya is now known as soniya|ruck | 05:04 | |
| ysandeep|rover | soniya29, good morning, once you settle down - lets sync - we got a lot in our plate today. | 05:07 |
| ysandeep|rover | soniya|ruck, ^^ | 05:12 |
| ysandeep|rover | rlandy|out, soniya|ruck: I am force promoting tripleo (ignoring ovb failure )- tripleo line need new diskimage-builder(which we cannot get without promotion) , and integration line need new tripleo component to pass node provisioning issue. | 05:24 |
| ysandeep|rover | chandankumar, do you have idea who manages quay.ceph.io - https://bugs.launchpad.net/tripleo/+bug/1973115 wallaby and earlier blocked on gate | 05:27 |
| chandankumar | ysandeep|rover: ceph team | 05:27 |
| ysandeep|rover | chandankumar: ack, I have pinged fmount - waiting for him to come online. | 05:28 |
| chandankumar | ok | 05:40 |
| akahat | Good morning folks o/ | 05:45 |
| marios | \o | 05:47 |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 05:57 | |
| soniya|ruck | ysandeep|rover|brb, hello | 06:03 |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 06:15 | |
| bhagyashris | marios, hey can you give me the edit rights on yesterdays retro board | 06:27 |
| bhagyashris | i want to copy the ideas for improvements for todays planning meeting discussion | 06:28 |
| marios | bhagyashris: ack sec | 06:32 |
| bhagyashris | marios, thanks | 06:32 |
| marios | bhagyashris: i think you can edit now? | 06:33 |
| bhagyashris | yup | 06:33 |
| bhagyashris | thanks | 06:33 |
| marios | bhagyashris: ack | 06:33 |
| marios | coffee brb | 06:33 |
| bhagyashris | marios, Done. you can lock .... | 06:36 |
| marios | thanks bhagyashris | 06:38 |
| jm1 | ysandeep|rover: good morning o/ | 07:09 |
| jm1 | ysandeep|rover: your fix with "force_non_periodic: true" has worked, all our tripleo jobs on stable branch are working now :) | 07:10 |
| ysandeep|rover | nice | 07:10 |
| jm1 | ysandeep|rover: ansible-collections-openstack should now be fixed in rdo as well. at least its pulling the latest stable release here: https://trunk.rdoproject.org/centos9-master/component/clients/current/ | 07:13 |
| ysandeep|rover | jm1, lets see how today's run goes | 07:16 |
| ysandeep|rover | jm1++ thanks for fixing that | 07:16 |
| ysandeep|rover | soniya|ruck, https://bugs.launchpad.net/tripleo/+bug/1973043 fixed now, I am working with francesco to test ceph fixes in gate.. Can you take a look at rdo promotion line to see if we have something new/ | 07:39 |
| soniya|ruck | ysandeep|rover, sure | 07:41 |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 07:50 | |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 08:03 | |
| ysandeep|rover | jm1: I have confirmed - clients component is fixed now | 08:11 |
| * ysandeep|rover closing the bug and cix | 08:11 | |
| jm1 | cool! | 08:12 |
| ysandeep|rover | soniya|ruck, if you get time can you investigate why https://review.rdoproject.org/r/c/testproject/+/36140 - no success with networking patch on w c8 | 09:35 |
| ysandeep|rover | please check why depends-on didn't work with container build job | 09:36 |
| soniya|ruck | ysandeep|rover, ack | 09:38 |
| ysandeep|rover | ++ thanks! | 09:38 |
| chandankumar | ysandeep|rover: hello | 09:39 |
| ysandeep|rover | o/ | 09:39 |
| chandankumar | ysandeep|rover: regarding https://review.rdoproject.org/r/c/testproject/+/36140 with depends on ovsdbapp | 09:39 |
| chandankumar | container build does not uses built test packages so there is no gating repo | 09:40 |
| ysandeep|rover | chandankumar, yeah i saw no build-test-package role running, was that expected? | 09:41 |
| chandankumar | I saw this message https://logserver.rdoproject.org/40/36140/48/check/periodic-tripleo-ci-build-containers-ubi-8-push-wallaby/3f18fce/etc/yum.repos.d/ from morning about gating repo | 09:41 |
| chandankumar | ysandeep|rover: yes, it is expected | 09:41 |
| ysandeep|rover | imho.. there should be a way to rebuild containers with depends-on | 09:41 |
| chandankumar | ysandeep|rover: https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers is used in container build | 09:42 |
| chandankumar | if we switch to https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/container-build then | 09:42 |
| chandankumar | depends on will work | 09:42 |
| chandankumar | and then we can discard toci build-containers role also | 09:43 |
| ysandeep|rover | ack.. basically same role which we use in content-provider | 09:43 |
| chandankumar | yes | 09:44 |
| *** jpena|off is now known as jpena | 09:44 | |
| chandankumar | we can do similar thing for image build also | 09:44 |
| ysandeep|rover | sound like this will need some work and difficult to test right away.. Someone will need to spend some time.. | 09:45 |
| chandankumar | ysandeep|rover: yes, may be food to next sprint | 09:45 |
| ysandeep|rover | I think its worth to investigate if ovb works can work with build_container: true - that way we can directly build container in ovb job - we don't have to rely on container_build job | 09:46 |
| chandankumar | ysandeep|rover: there is another way also for current timeing | 09:46 |
| chandankumar | we can build the package in the dlrn side by sending a dnm patch with upstream-id of the depends on patch | 09:47 |
| chandankumar | and once the rpm is available, we can inject it in release file and build the container | 09:47 |
| ysandeep|rover | interesting.. | 09:48 |
| chandankumar | ysandeep|rover: let;s try with build_containers:true in ovb and see | 09:48 |
| ysandeep|rover | interesting | 09:48 |
| ysandeep|rover | chandankumar, I remember ykarel doing some improvement that allow building container in undercloud itself but not sure we have update ovb to consume container from local registry | 09:50 |
| ysandeep|rover | chandankumar, we can give it a try.. /me updating patch | 09:50 |
| soniya|ruck | ysandeep|rover, c8 sc01 victoria is failing for random failures | 09:58 |
| *** ysandeep|rover is now known as ysandeep|rover|lunch | 10:03 | |
| jm1 | chandankumar: this can be abandoned? https://review.opendev.org/c/openstack/ansible-collections-openstack/+/841384 | 10:13 |
| chandankumar | jm1: yes | 10:13 |
| jm1 | chandankumar: ack | 10:13 |
| *** rlandy|out is now known as rlandy | 10:20 | |
| rlandy | ysandeep|rover|lunch: soniya|ruck: hello - let's sync when you are back | 10:21 |
| frenzy_friday | hey rlandy, ysandeep|rover|lunch can we pl merge https://review.opendev.org/c/openstack/tripleo-quickstart/+/839305 and https://review.opendev.org/c/openstack/tripleo-ci/+/839323 ? | 10:26 |
| rlandy | frenzy_friday: ack - master line was blocked yesterday | 10:26 |
| frenzy_friday | rlandy, oh, np | 10:27 |
| jm1 | ysandeep|rover|lunch, rlandy, chandankumar: can we browse and download rhosp packages somewhere like we can with rdo? | 10:30 |
| rlandy | jm1: what do you mean by browse? if you give me the rdo link you use, can send equivalent osp link | 10:31 |
| rlandy | jm1: like https://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel9-osp17/? | 10:32 |
| rlandy | or released content? | 10:32 |
| jm1 | rlandy: want to download ansible-collections-openstack package for different rhos releases to check its content. | 10:32 |
| jm1 | rlandy: ah, great, will have a look. thanks! | 10:32 |
| chandankumar | jm1: let's chat in #openstack-pcci | 10:32 |
| rlandy | jm1: ^^ internal channel | 10:33 |
| rlandy | you will need to join that | 10:33 |
| jm1 | rlandy: yes, just joined | 10:34 |
| *** ysandeep|rover|lunch is now known as ysandeep|rover | 10:34 | |
| ysandeep|rover | rlandy, sry late lunch today due to rr load :) I am back lets sync | 10:34 |
| ysandeep|rover | soniya|ruck, | 10:34 |
| ysandeep|rover | meet.google.com/sqm-mhfg-znk | 10:35 |
| ysandeep|rover | ratailor, https://review.opendev.org/c/openstack/ovsdbapp/+/841238 | 10:48 |
| ysandeep|rover | ratailor, ignore | 10:48 |
| ysandeep|rover | was trying to ping ronelle | 10:49 |
| rlandy | soniya|ruck: can you hear us? | 10:53 |
| *** soniya29 is now known as soniya29|ruck | 11:08 | |
| ysandeep|rover | soniya29|ruck, I have triggered the ovb jobs in component already, could you please keep an eye: https://review.rdoproject.org/r/c/testproject/+/28537 and see how that goes. | 11:09 |
| soniya29|ruck | ysandeep|rover, ack | 11:09 |
| * soniya29|ruck grabs a tea | 11:10 | |
| *** soniya29|ruck is now known as soniya29|ruck|brb | 11:10 | |
| ysandeep|rover | rlandy: thanks to delorean-current - steve node-provisioning fix is already in our current periodic master run... let see how that goes | 11:14 |
| rlandy | ysandeep|rover: great | 11:15 |
| *** dviroel_ is now known as dviroel | 11:33 | |
| rlandy | dasm|off: hello - when you are back in ... the patch you had for rhel-9 17 for uefi boot | 11:38 |
| rlandy | can you revive that | 11:38 |
| rlandy | and test | 11:38 |
| ysandeep|rover | ovb fs001/fs020/64 passed in master run - looks promising so far | 11:42 |
| * ysandeep|rover retriggered jobs which failed for tempest run - only few failures | 11:44 | |
| jm1 | rlandy: will not be able to join tripleo ci planning meeting because of doc appointment today | 11:52 |
| rlandy | jm1: ok | 11:52 |
| rlandy | ysandeep|rover: thanks | 11:52 |
| rlandy | ysandeep|rover: can we merge https://review.opendev.org/c/openstack/tripleo-common/+/841550? | 11:53 |
| ysandeep|rover | rlandy, waiting for https://zuul.openstack.org/status#841560 test project results | 11:53 |
| rlandy | in standalone deploy | 11:53 |
| rlandy | ysandeep|rover: ok - going to testproject train integration line failures with that deps | 11:54 |
| rlandy | hmmm ... only fs001 | 11:56 |
| rlandy | 2022-05-12T09:00:25+0000 DEBUG Installed: python3-testtools-2.3.0-15.el8.noarch | 11:56 |
| rlandy | soniya: your bug on setup call failures ... | 11:57 |
| *** ysandeep|rover is now known as ysandeep|rover|brb | 12:01 | |
| rlandy | chandankumar: do we need the later version of testtools in train? https://github.com/redhat-openstack/rdoinfo/blob/master/buildsys-tags/cloud8s-openstack-train-testing.yml#L1720 | 12:02 |
| rlandy | seeing setup class failures that we were seeing in downstream | 12:02 |
| chandankumar | rlandy: Do the failure similar to RHOS-17 rhel-8? | 12:06 |
| chandankumar | in wallaby we have python-testtools-2.5.0-2.el8 | 12:07 |
| frenzy_friday | pls add to review list https://review.rdoproject.org/r/c/config/+/42771 (quay zuul secret for wallaby ) | 12:07 |
| reviewbot | I have added your review to the Review list | 12:07 |
| rlandy | chandankumar: no - different error | 12:12 |
| rlandy | unittest2.case.SkipTest | 12:12 |
| rlandy | was in 17 | 12:12 |
| chandankumar | rlandy: can you share the logs? | 12:14 |
| rlandy | chandankumar: this is a keystone access error | 12:15 |
| chandankumar | then may be something else is going on there, might not be related to package | 12:17 |
| *** soniya is now known as soniya|ruck | 12:21 | |
| *** ysandeep|rover|brb is now known as ysandeep|rover | 12:28 | |
| ysandeep|rover | soniya|ruck, I am updating all the launchpad/Cix bugs on which I was working last week with the current status - Please try to leave a comment on your bugs too before rr handover on where you left off and next action plan if any. | 12:44 |
| soniya|ruck | ysandeep|rover, sure | 12:47 |
| jm1 | ysandeep|rover: can you help me with creating a bz bug for downstream? 🙈 | 13:08 |
| ysandeep|rover | jm1, gmeet? | 13:09 |
| jm1 | ysandeep|rover: sure | 13:09 |
| jm1 | ysandeep|rover: gimme a secf | 13:09 |
| *** dasm|off is now known as dasm | 13:15 | |
| dasm | o/ | 13:15 |
| *** marios_ is now known as marios | 13:17 | |
| dviroel | rlandy: chandankumar: question, wrt to ceph daemon promotion, which registry should I push the container to? quay.rdoproject.org? | 13:17 |
| rlandy | dviroel: good question | 13:18 |
| dviroel | i was considering quay.io/tripleomastercentos9 but not sure if 'config' has the creds needed | 13:18 |
| rlandy | dviroel: does the ceph team have a preference? | 13:18 |
| chandankumar | dviroel: yes, I would suggest both | 13:18 |
| chandankumar | dviroel: quay.rdoproject.org and quay.io | 13:19 |
| dviroel | rlandy: probably no, but I can ask them | 13:19 |
| chandankumar | dviroel: for quay.io, arxcruz knows where is the creds, toolbox copy-quay script pushes those containers | 13:19 |
| rlandy | dviroel: checking secrets | 13:19 |
| dviroel | chandankumar: yeah, but I think that playbooks/secrets are not ready for that in 'config' - so i would start with quay.rdoproject.org first | 13:20 |
| rlandy | dviroel: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/secrets.yaml | 13:20 |
| chandankumar | dviroel: sournds good | 13:20 |
| rlandy | there is a general quay_secret | 13:21 |
| rlandy | we'd need to track history on that | 13:21 |
| dviroel | rlandy: ack, but container-login will need to be updated to consider that one too, I think | 13:21 |
| rlandy | lol - I added it | 13:21 |
| dviroel | :) | 13:21 |
| rlandy | dviroel: welcome to my world | 13:22 |
| rlandy | two registries to log in to | 13:22 |
| dviroel | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-rdo-base/container-login.yaml | 13:22 |
| dviroel | latest updates deal with trunk.registry.rdoproject.org and quay.rdoproject.org | 13:23 |
| rlandy | dviroel: git blame on that file | 13:23 |
| rlandy | see the last comment | 13:23 |
| rlandy | from sagi | 13:23 |
| rlandy | https://github.com/rdo-infra/review.rdoproject.org-config/commit/6372330e0f51628f73faa30591747d118abab738 | 13:24 |
| dviroel | rlandy: ah, so 'quay_rdoci' seems to be the one | 13:25 |
| rlandy | dviroel: you tried it? | 13:25 |
| rlandy | it works? | 13:25 |
| rlandy | chandankumar: pls pass me anything you start with downstream fs001 | 13:26 |
| chandankumar | rlandy: sure | 13:26 |
| rlandy | would be interested in carrying that on | 13:26 |
| rlandy | at least testproject | 13:26 |
| ysandeep|rover | rlandy, just need these now to promote master - http://pastebin.test.redhat.com/1051406 | 13:26 |
| dviroel | rlandy: no | 13:26 |
| dviroel | rlandy: didn't try that one | 13:27 |
| rlandy | ysandeep|rover: ok - let's make an assessement at your eod if it makes sense to skip any | 13:27 |
| dviroel | rlandy: container-login playbook doesn't supports it yet - it would be good to have all logins within the same playbook imho | 13:28 |
| ysandeep|rover | rlandy, ack.. /me rerunning failing jobs in the meantime | 13:28 |
| dviroel | rlandy: chandankumar: considering going with 'quay.rdoproject.org' first, to validate my promotion playbooks, after that I can focus on different and multiple registries, wdyt? | 13:31 |
| chandankumar | dviroel: sounds good | 13:32 |
| rlandy | dviroel++ good | 13:33 |
| dviroel | ack | 13:33 |
| rlandy | marios++ https://review.opendev.org/q/topic:ooo_c7_teardown all merged here? | 13:46 |
| rlandy | dasm: pls review comments on https://review.rdoproject.org/r/q/topic:ooo_c7_teardown | 13:47 |
| rlandy | so we can merge what is left | 13:47 |
| jm1 | ysandeep|rover: how to set blocker flag on that bug? | 13:48 |
| jm1 | https://bugzilla.redhat.com/show_bug.cgi?id=2084592 | 13:48 |
| jm1 | ysandeep|rover: i think i found it, but not sure | 13:49 |
| ysandeep|rover | jm1, I have never done that but there is a doc - let me ping you on #openstack-pcci | 13:49 |
| ysandeep|rover | jm1: please read the doc to understand what does blocker actually means. | 13:50 |
| bhagyashris | arxcruz, rlandy, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, chandankumar, frenzy_friday, dviroel, rcastillo, jm1, rlandy|ruck, dasm ysandeep|rover soniya|ruck | 13:57 |
| bhagyashris | Tripleo CI planning meeting in 3 mins | 13:57 |
| bhagyashris | https://hackmd.io/P5sTPMSBREe5kBS_4jDhIg#2022-04-21-Unified-Sprint-62-Planning | 13:57 |
| bhagyashris | https://meet.google.com/avd-ijsb-ejt?authuser=0 | 13:57 |
| *** ysandeep|rover is now known as ysandeep|rover|mtg | 14:00 | |
| bhagyashris | rlandy, ^ | 14:01 |
| bhagyashris | chandankumar, ^ | 14:02 |
| bhagyashris | jm1, ^ | 14:02 |
| bhagyashris | we are waiting ... | 14:02 |
| jm1 | bhagyashris: not joining today because of doc appointment. will be away in a minute | 14:05 |
| ykarel | ysandeep|rover|mtg, just saw your message, ovb jobs use container updates, so packages are build and then containers are updated with it | 14:28 |
| ykarel | so build containers is not needed for this use case | 14:29 |
| ykarel | and yes i too recall i did some improvements in past for undercloud container build, but don't recall the overall status | 14:30 |
| ysandeep|rover|mtg | ykarel: >> ovb jobs use container updates - ack thanks! | 14:30 |
| ykarel | btw not just ovb all deployment based j obs | 14:30 |
| ysandeep|rover|mtg | ykarel: yeah make sense | 15:08 |
| ysandeep|rover|mtg | dviroel, rlandy, https://status.sepia.ceph.com/incidents/4053 fyi.. outage where quay.ceph.io is hosted | 15:09 |
| rlandy | ysandeep|rover|mtg: anything blocking w+ on https://review.opendev.org/c/openstack/tripleo-common/+/841549? | 15:09 |
| ysandeep|rover|mtg | rlandy: just waiting for zuul to report back | 15:10 |
| ysandeep|rover|mtg | rlandy, may be sc-10-kvm is legit failure - failed thrice on same issue | 15:24 |
| ysandeep|rover|mtg | https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/807e54f/logs/undercloud/var/log/tempest/tempest_run.log | 15:24 |
| ysandeep|rover|mtg | https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/4fecd69/logs/undercloud/var/log/tempest/tempest_run.log | 15:24 |
| ysandeep|rover|mtg | https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/1505dda/logs/undercloud/var/log/tempest/tempest_run.log | 15:24 |
| ysandeep|rover|mtg | i was just checking the logs | 15:25 |
| ysandeep|rover|mtg | 2022-05-12 14:17:19.252 ERROR /var/log/containers/nova/nova-api.log: 6 ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) | 15:25 |
| ysandeep|rover|mtg | 2022-05-12 14:17:19.252 ERROR /var/log/containers/nova/nova-api.log: 6 ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair octavia-ssh-key not found for user 7527fbfe4d204aafaff987d944a2501 | 15:25 |
| ysandeep|rover|mtg | https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/807e54f/logs/undercloud/var/log/extra/errors.txt | 15:25 |
| ysandeep|rover|mtg | I am going to report a bug | 15:25 |
| chandankumar | rlandy: tracking stuff here: https://code.engineering.redhat.com/gerrit/q/topic:ovb-centos-downstream downstream centos ovb | 15:29 |
| ysandeep|rover|mtg | rlandy, arxcruz fyi.. this kind of issue I am talking about for tempest... octavia_tempest_plugin.tests.scenario.v2.test_traffic_ops.TrafficOperationsScenarioTest run in integration but not in octavia component. | 15:30 |
| ysandeep|rover|mtg | int: https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/1505dda/logs/undercloud/var/log/tempest/tempest_run.log | 15:30 |
| ysandeep|rover|mtg | vs component: https://sf.hosted.upshift.rdu2.redhat.com/logs/81/42781/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-octavia-master/9e8fac0/logs/undercloud/var/log/tempest/tempest_run.log | 15:30 |
| ysandeep|rover|mtg | chandankumar, ^^ fyi.. | 15:31 |
| dasm | fyi, every other release is going to be called "SLURP" :) | 15:40 |
| dasm | Skip Skip Level Upgrade Release Process | 15:41 |
| chandankumar | rlandy: you can take it from here https://code.engineering.redhat.com/gerrit/q/topic:ovb-centos-downstream cert-install is not working | 15:45 |
| chandankumar | that needs to be fixed | 15:45 |
| chandankumar | I tried fixing here https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/409029/1/playbooks/tripleo-rdo-base/cert-install.yaml | 15:45 |
| chandankumar | please have a look | 15:46 |
| chandankumar | seee ya! | 15:46 |
| ysandeep|rover|mtg | rlandy, https://sf.hosted.upshift.rdu2.redhat.com/logs/81/42781/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-octavia-master/9e8fac0/logs/undercloud/var/log/tempest/stestr_results.html | 15:48 |
| ysandeep|rover|mtg | rlandy, http://pastebin.test.redhat.com/1051465 | 15:50 |
| * dviroel lunch | 15:52 | |
| ysandeep|rover|mtg | gthiemonge, we are hitting a new issue in octavia tempest tests https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/807e54f/logs/undercloud/var/log/tempest/tempest_run.log , we will report a bug soon.. This is blocking master promotion.. If you have any pointers about error please let us know. | 15:56 |
| *** ysandeep|rover|mtg is now known as ysandeep|rover | 15:56 | |
| ysandeep|rover | rlandy, for sc010-kvm do we report a bug or launchpad because logs are private links | 15:57 |
| gthiemonge | ysandeep|rover: hi, yeah it looks like octavia cannot communicate with nova | 15:58 |
| ysandeep|rover | gthiemonge, https://sf.hosted.upshift.rdu2.redhat.com/logs/64/42764/1/check-rdo/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/807e54f/logs/undercloud/var/log/extra/errors.txt | 15:58 |
| gthiemonge | octavia-tempest-plugin not octavia | 15:58 |
| rlandy | ysandeep|rover: ack lp bug beause product is upstream | 15:59 |
| ysandeep|rover | gthiemonge, rpm diff between last good job and current one http://pastebin.test.redhat.com/1051465 | 15:59 |
| rlandy | ysandeep|rover: pls go when you have the bug | 15:59 |
| rlandy | late for you | 15:59 |
| ysandeep|rover | rlandy, ack.. let me open a lp.. I was worried if its okay to attach internal logs url in lp | 16:00 |
| rlandy | ysandeep|rover: it's not ideal ... but no choice | 16:02 |
| rlandy | you can sudo link if you want | 16:02 |
| rlandy | like <internal log server > .... | 16:02 |
| ysandeep|rover | ack, and I will attach logs in cix card | 16:03 |
| ysandeep|rover | pinged logs to gthiemonge already, so he knows where to look | 16:03 |
| ysandeep|rover | gthiemonge, we can hold a reproducer node as well if that helps for debugging | 16:04 |
| rlandy | ysandeep|rover: there is one running right now | 16:04 |
| rlandy | you can hold that | 16:04 |
| *** marios is now known as marios|out | 16:05 | |
| johnsom | ysandeep|rover It's a nova issue, so maybe collect the nova RPM diffs or haproxy config changes. | 16:09 |
| gthiemonge | there's a huge backtrace in nova-api.log, it looks like nova cannot fetch a security-group that was created a few seconds before in neutron | 16:10 |
| *** soniya|ruck is now known as soniya|out | 16:11 | |
| * soniya|out leaving for the day | 16:11 | |
| johnsom | May 12 14:24:23 standalone haproxy[7]: 192.168.24.3:51460 [12/May/2022:14:22:53.266] nova_osapi nova_osapi_be/standalone.ctlplane.localdomain 0/0/0/90093/90093 504 397 - - ---- 41/3/2/2/0 0/0 "POST /v2.1/servers HTTP/1.1" | 16:12 |
| ysandeep|rover | johnsom, gthiemonge could you please also add your investigation comments on https://bugs.launchpad.net/tripleo/+bug/1973223 as well, it would help to reach nova team. | 16:14 |
| gthiemonge | ysandeep|rover: ack | 16:14 |
| johnsom | Yeah, nova-api: 2022-05-12 14:24:47.095 6 ERROR nova.api.openstack [req-1b96272e-da6a-4025-b37a-2c073c789cce 7527fbfe4d204aafaff987d944a25019 516d2e21db7043e8aac29216b12572e0 - default default] Caught error: Apache/mod_wsgi request data read error: Partial results are valid but processing is incomplete.: OSError: Apache/mod_wsgi request data read error: Partial results are valid but processing is incomplete. | 16:14 |
| ysandeep|rover | https://paste.openstack.org/show/beA9TQcrWxSqwX9F6Cz0/ - all the packages rpm diff | 16:17 |
| ysandeep|rover | johnsom, gthiemonge thanks for investigation help, Its late for me (2200 here), if you need any help from us please ping rlandy | 16:21 |
| rlandy | thanks ysandeep|rover | 16:21 |
| ysandeep|rover | rlandy: fyi.. https://bugs.launchpad.net/tripleo/+bug/1973223 , we will add logs link in cix card once it appears.. | 16:22 |
| rlandy | ysandeep|rover: ok | 16:22 |
| ysandeep|rover | rlandy, sry one more thing - please merge ceph V patch once zuul reports on same patch, and recheck W patch once train patch merges - and possibly +w that if all passes. | 16:23 |
| * rlandy looks | 16:24 | |
| ysandeep|rover | testproject for all releases are already green | 16:24 |
| rlandy | ysandeep|rover: ah - that was what I was going to check | 16:24 |
| *** ysandeep|rover is now known as ysandeep|rover|out | 16:28 | |
| ysandeep|rover|out | rlandy: see you tomorrow , thanks for all the help o/ | 16:29 |
| rlandy | ysandeep|rover|out: have a good night | 16:30 |
| * dviroel back | 17:08 | |
| chandankumar | rlandy: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/409118 please +w it | 17:08 |
| rlandy | ok | 17:09 |
| rlandy | lunch brb | 17:13 |
| *** jpena is now known as jpena|off | 17:21 | |
| dasm | hmm.. i'm trying to start uefi jobs downtstream. I see jobs starting, and immediately getting canceled. No report, nothing | 17:22 |
| dasm | Not sure yet what's happening there | 17:23 |
| chandankumar | rlandy: we need to hold the node https://sf.hosted.upshift.rdu2.redhat.com/logs/13/409013/3/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-internal-train/d79e68e/job-output.txt | 17:28 |
| chandankumar | cert is not working for cs8 | 17:28 |
| chandankumar | and https://osp-trunk.hosted.upshift.rdu2.redhat.com/ returning Name or service not known> | 17:29 |
| chandankumar | when around, please have a look. | 17:29 |
| rlandy | chandankumar: ok - I'll look | 17:34 |
| rlandy | have a good night | 17:34 |
| dviroel | dasm: my guess is that you need to do: vars: *vars | 17:36 |
| dasm | dviroel: i'll give it a try. just removed it, copied-pasted | 17:40 |
| dasm | dviroel: and now it's starting. | 17:41 |
| dasm | dviroel++ | 17:41 |
| rlandy | dasm: going to be a few minutes late to 1-1 | 18:29 |
| rlandy | on call | 18:29 |
| rlandy | may run over | 18:29 |
| dasm | k, thanks for heads up | 18:31 |
| rlandy | dasm: so sorry - still ongoing - can I move meeting to tomorrow? | 18:43 |
| rlandy | or after review time? | 18:43 |
| dasm | rlandy: sure. missed your ping. sorry | 18:52 |
| rcastillo | lunch brb | 18:53 |
| rlandy | joined review time | 19:01 |
| rlandy | akahat: hey - we were looking at your OVB patches - can you explain at tomorrow's review meeting? | 19:35 |
| rlandy | we left a comment | 19:35 |
| rcastillo | back | 19:46 |
| * dviroel going afk for a bit | 21:09 | |
| *** dviroel is now known as dviroel|afk | 21:09 | |
| * dasm disappears into bushes | 21:10 | |
| *** dasm is now known as dasm|off | 21:11 | |
| *** rlandy is now known as rlandy|bbl | 23:02 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!