| rlandy | chkumar|rover: bhagyashris|ruck: hey - I updated the program call doc - promoted rhos-17 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/buildset/5dda07a05b834788804701926d71bbed | 02:36 |
|---|---|---|
| rlandy | see you tomorrow | 02:38 |
| *** amoralej|off is now known as amoralej | 07:08 | |
| *** jpena|off is now known as jpena | 07:46 | |
| *** ysandeep is now known as ysandeep|lunch | 07:46 | |
| marios | sshnaidm: o/ | 08:30 |
| sshnaidm | marios, joining | 08:30 |
| zbr | low hanging fix: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34406 | 08:31 |
| *** ykarel is now known as ykarel|lunch | 08:32 | |
| * zbr had a big laugh seeing flask-wtf package name. | 08:33 | |
| bhagyashris_ | chkumar|rover, ykarel|lunch https://bugs.launchpad.net/tripleo/+bug/1934875 | 08:37 |
| *** bhagyashris_ is now known as bhagyashris|ruck | 08:37 | |
| soniya29 | akahat, arxcruz, kopecmartin, ysandeep|lunch, rlandy, chkumar|rover please add/edit the today's agenda for tempest meeting :- https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ | 09:02 |
| marios | needs reviews please https://review.opendev.org/c/openstack/tripleo-repos/+/795333 | 09:18 |
| *** ysandeep|lunch is now known as ysandeep | 09:27 | |
| chkumar|rover | marios: hello, please have a look at this bug https://bugs.launchpad.net/tripleo/+bug/1934880 | 09:58 |
| chkumar|rover | when free, | 09:59 |
| marios | chkumar|rover: k will do | 10:00 |
| slaweq | chkumar|rover: hi, I think that https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/799577 is now good to go. Fix in neutron-tempest-plugin is now merged | 10:07 |
| slaweq | thx for testing that | 10:07 |
| chkumar|rover | slaweq: sure, may be we can wait for the neutron-tempest-plugin review to reach network component line | 10:08 |
| chkumar|rover | slaweq: thanks for working on that :-) | 10:09 |
| slaweq | chkumar|rover: sure | 10:09 |
| *** ykarel|lunch is now known as ykarel | 10:14 | |
| sshnaidm | chkumar|rover, marios bhagyashris|ruck lease review periodic jobs for collections: https://review.rdoproject.org/r/c/rdo-jobs/+/34360 | 10:58 |
| bhagyashris|ruck | sshnaidm, ack | 10:59 |
| marios | sshnaidm: ack added to list | 11:18 |
| ysandeep | Review request: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/799517 | 11:28 |
| *** jpena is now known as jpena|lunch | 11:39 | |
| weshay|ruck | bhagyashris|ruck, chkumar|rover we're promoting too often | 12:05 |
| weshay|ruck | when this happens it leads to https://bugs.launchpad.net/tripleo/+bug/1934880 and issues like it | 12:05 |
| bhagyashris|ruck | weshay|ruck, ok. | 12:07 |
| weshay|ruck | bhagyashris|ruck, chkumar|rover let's chat in 5 min | 12:08 |
| weshay|ruck | sorry program call.. | 12:08 |
| bhagyashris|ruck | weshay|ruck, in program call | 12:08 |
| weshay|ruck | bhagyashris|ruck, chkumar|rover hang on after the cix call for a few min | 12:09 |
| bhagyashris|ruck | weshay|ruck, ack | 12:09 |
| weshay|ruck | bhagyashris|ruck, chkumar|rover it also wouldn't be a terrible idea to shut the promoter down.. or only enable ussuri for today | 12:14 |
| bhagyashris|ruck | weshay|ruck, ok will do that after program call is that ok ? | 12:17 |
| bhagyashris|ruck | or i will do that after status | 12:20 |
| weshay|ruck | chkumar|rover, https://bugs.launchpad.net/tripleo/+bug/1931995 | 12:20 |
| weshay|ruck | ah.. wait.. different issue .. nvrmind | 12:21 |
| bhagyashris|ruck | weshay|ruck, should i off the promoter? | 12:26 |
| weshay|ruck | bhagyashris|ruck, ya.. or just enable ussuri | 12:27 |
| bhagyashris|ruck | ok | 12:27 |
| weshay|ruck | promoting multiple times in a day.. can cause problems | 12:28 |
| bhagyashris|ruck | yeah | 12:29 |
| bhagyashris|ruck | Done | 12:29 |
| *** ysandeep is now known as ysandeep|afk | 12:31 | |
| dviroel | the promoter gets stressed | 12:31 |
| weshay|ruck | dviroel, it's the upstream mirrors/proxies | 12:31 |
| marios | zbr: thank you the linters thing looks good mock_modules is now working | 12:32 |
| marios | zbr:++ | 12:32 |
| weshay|ruck | ykarel++ | 12:35 |
| marios | needs reviews please https://review.opendev.org/c/openstack/tripleo-repos/+/795333 | 12:36 |
| *** amoralej is now known as amoralej|lunch | 12:38 | |
| *** jpena|lunch is now known as jpena | 12:43 | |
| weshay|ruck | bhagyashris|ruck, https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/799779 | 12:44 |
| sshnaidm | seems like problem with mirror.gra1.ovh.opendev.org ? | 12:44 |
| sshnaidm | or general problem with mirrors | 12:45 |
| weshay|ruck | ya.. I think it was ovh the last time | 12:46 |
| bhagyashris|ruck | weshay|ruck, i checked your comment on patch ^ basically periodic sc004 job (only victoria, ussuri and c8 train is failing) is started failing from few hours. i checked the upstream status of sc004 on stable/victoria , ussuri and train so looking at the result - sc004 job last ran on 30th june 2021 | 12:51 |
| bhagyashris|ruck | so not sure | 12:51 |
| weshay|ruck | ysandeep|afk, fyi.. few updates.. and we're using ruck_rover.py in upstream and downstream | 12:51 |
| weshay|ruck | bhagyashris|ruck, check the tests upstream runs.. | 12:51 |
| bhagyashris|ruck | may be i can push test patch upstream to test the job result | 12:51 |
| weshay|ruck | if it's the same.. upstream should be skipped | 12:52 |
| weshay|ruck | bhagyashris|ruck, you don't need to run a job | 12:52 |
| weshay|ruck | also the component pipeline... | 12:52 |
| weshay|ruck | scenario004 is in a lot of places | 12:52 |
| bhagyashris|ruck | ok | 12:52 |
| weshay|ruck | bhagyashris|ruck, https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario004-standalone | 12:53 |
| bhagyashris|ruck | weshay|ruck, yes i checked that but the issue is not with the master | 12:54 |
| weshay|ruck | bhagyashris|ruck, https://f426a06be677e445c4fa-4adf110539aed9252479dc6b548d2884.ssl.cf2.rackcdn.com/778915/46/check/tripleo-ci-centos-8-scenario004-standalone/7df91f9/logs/undercloud/var/log/tempest/stestr_results.html | 12:54 |
| weshay|ruck | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario004-standalone&branch=stable%2Fussuri | 12:54 |
| weshay|ruck | bhagyashris|ruck, https://751e8ffd5efe929acb06-2d54b4071a8d065f516b028813c4bd96.ssl.cf1.rackcdn.com/797088/2/check/tripleo-ci-centos-8-scenario004-standalone/f81b157/logs/undercloud/var/log/tempest/stestr_results.html | 12:55 |
| weshay|ruck | hey arxcruz why is keystone api not running here? https://751e8ffd5efe929acb06-2d54b4071a8d065f516b028813c4bd96.ssl.cf1.rackcdn.com/797088/2/check/tripleo-ci-centos-8-scenario004-standalone/f81b157/logs/undercloud/var/log/tempest/stestr_results.html | 12:55 |
| bhagyashris|ruck | weshay|ruck, if you see the date it ran last on 18th june | 12:56 |
| weshay|ruck | I don't understand why that is relevant | 12:56 |
| bhagyashris|ruck | weshay|ruck, i see the test started failing from today | 12:56 |
| weshay|ruck | bhagyashris|ruck, k.. however if we skip the test in periodic.. promote.. it will hit upstream | 12:57 |
| bhagyashris|ruck | weshay|ruck, yes let me check with upstream | 12:58 |
| weshay|ruck | thanks | 12:59 |
| soniya29 | arxcruz, akahat, chkumar|rover, rlandy, kopecmartin, ysandeep|afk, tempest meeting | 13:00 |
| soniya29 | rlandy, ysandeep|afk, ^^ | 13:03 |
| chkumar|rover | tbarron: Hello, please have a look at this bug https://bugs.launchpad.net/tripleo/+bug/1934879 when free, thanks! | 13:03 |
| rlandy | arxcruz: quick vote on https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/799667 pls | 13:04 |
| tbarron | chkumar|rover: ack | 13:06 |
| *** ysandeep|afk is now known as ysandeep | 13:17 | |
| weshay|ruck | chkumar|rover, thanks for digging into that a bit | 13:19 |
| chkumar|rover | you are welcome sir :-) | 13:19 |
| dviroel | tbarron: o/ i was looking at the logs, it seems that is the volume that was stuck at creation was '8f3d4c33-fa11-40db-a341-4d449e18216a' | 13:23 |
| dviroel | tbarron: one thing that doesn't look right is that the driver 'CephFSDriver@hostgroup@cephfs' is initialized 4 times for the failing job | 13:24 |
| tbarron | dviroel: just starting to drop something else and look now. Which of those are you looking at, standalone-victoria? | 13:24 |
| chkumar|rover | oh , we have another manila person in the house, will bug him next time:-) | 13:25 |
| tbarron | chkumar|rover: hah! | 13:25 |
| dviroel | tbarron: yes, victoria one | 13:25 |
| tbarron | i like looking at issues with dviroel | 13:25 |
| tbarron | dviroel: k, going there | 13:25 |
| chkumar|rover | dviroel: tbarron thank you :-) | 13:26 |
| dviroel | chkumar|rover: i don't have the knowledge in ceph backends like tbarron has :/ | 13:26 |
| chkumar|rover | akahat: left a comment https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/32679 | 13:27 |
| tbarron | dviroel: chkumar|rover: This is https://bugzilla.redhat.com/show_bug.cgi?id=1976693 | 13:28 |
| tbarron | dviroel: chkumar|rover: ideally we would have discovered this here, upstream, before we saw it in a downsterm rhosp compose | 13:28 |
| dviroel | tbarron: ack, the errors in the logs are the same | 13:32 |
| tbarron | dviroel: chkumar|rover: we need this fix for ceph https://github.com/ceph/ceph/pull/42162 and we need it in both the ceph daemon containers and in the manila-share container | 13:32 |
| marios | needs reviews please https://review.opendev.org/c/openstack/tripleo-repos/+/795333 when you have time for such things tripleo thank you | 13:33 |
| frenzy_friday | zbr, there is a playbook task which runs elastic-recheck-query <query.yml> for all the queries. It works fine when I run the playbook, but it is failing when I run through tox. Do you know why this might be? | 13:34 |
| *** amoralej|lunch is now known as amoralej | 13:36 | |
| dviroel | thanks tbarron | 13:36 |
| marios | weshay|ruck: if you're gonna propose a patch for project-config add weshay|ruck as op not admin it is easier to work with | 13:39 |
| marios | weshay|ruck: maybe also switch yourself and rlandy to op from admin for same reason, or ask alex how he did the admin claim | 13:40 |
| chkumar|rover | tbarron: thanks :-) | 13:42 |
| tbarron | chkumar|rover: dviroel we're going to need new cbs images, etc. for ceph and then rebase openstack on these too | 13:43 |
| tbarron | dviroel: just noting that the convo is continuing in #tripleo | 13:52 |
| dviroel | tbarron: ;) | 13:53 |
| *** bhagyashris is now known as bhagyashris|ruck | 13:59 | |
| dviroel | tbarron: i was abou to ask if the failure wouldn't happen in another test, since the issue is at ceph. But now i saw that there is only one manila test running :) | 14:04 |
| marios | rlandy: weshay|ruck: please add to your reviews list? i think it has enough votes ready for workflow | 14:04 |
| marios | weshay|ruck: rlandy: that one :) https://review.opendev.org/c/openstack/tripleo-ci/+/793120 | 14:05 |
| rlandy | ack | 14:05 |
| dviroel | s/one test/one test class | 14:05 |
| marios | rlandy: thank you no emergency just whenever you get to it (maybe tomorrow) | 14:05 |
| tbarron | dviroel: these tests have a limited "budget" for tests. I would run full tempest if we could but we were asked to supplya a simple "ping" test | 14:08 |
| tbarron | dviroel: i've considered replacing it with a scenario test to go a little deeper | 14:08 |
| dviroel | tbarron: yeah, i'm aware of the scenario test | 14:08 |
| tbarron | dviroel: in any case the test needs to be very reliable, not yield false negative | 14:09 |
| tbarron | negatives | 14:09 |
| tbarron | dviroel: since the main thing is to test the deployment tools, more than testing manila itself | 14:09 |
| zbr | frenzy_friday: likely it needs and env var that is not passed by tox (see passenv) | 14:09 |
| dviroel | tbarron: yeah, make sense | 14:10 |
| *** bhagyashris_ is now known as bhagyashris|ruck | 14:24 | |
| *** ykarel is now known as ykarel|away | 15:04 | |
| ysandeep | weshay|ruck, rlandy fyi.. https://hackmd.io/mLu17QSyTLqiwukWjgcuMg?view# | 15:04 |
| ysandeep | rlandy, weshay|ruck chkumar|rover review request: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/799517 | 15:07 |
| *** ysandeep is now known as ysandeep|dinner | 15:12 | |
| * dviroel lunch | 15:32 | |
| rlandy | arxcruz: hey - have a moment to chat about keystone api tests? | 15:46 |
| rlandy | adding new scenario | 15:46 |
| rlandy | test | 15:46 |
| weshay|ruck | need help w/ reviews https://review.rdoproject.org/r/q/topic:%22refactor_job_names%22+(status:open%20OR%20status:merged) | 16:00 |
| marios | weshay|ruck: please add to your reviews list? i think it has enough votes ready for workflow https://review.opendev.org/c/openstack/tripleo-ci/+/793120 | 16:05 |
| marios | needs reviews please https://review.opendev.org/c/openstack/tripleo-repos/+/795333 | 16:06 |
| *** marios is now known as marios|out | 16:06 | |
| * dviroel back | 16:10 | |
| *** ysandeep|dinner is now known as ysandeep | 16:22 | |
| ysandeep | rlandy: thanks for second +2 on https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/799517 , Could you please +w if this looks okay | 16:23 |
| rlandy | done | 16:25 |
| ysandeep | rlandy++ thanks! | 16:26 |
| ysandeep | pojadhav, this one need rebase: https://review.rdoproject.org/r/c/rdo-jobs/+/34381 | 16:27 |
| dviroel | i'll appreciate more thoughts on this review too: https://review.opendev.org/c/openstack/tripleo-repos/+/796529 | 16:27 |
| dviroel | ;) | 16:28 |
| *** amoralej is now known as amoralej|off | 16:30 | |
| ysandeep | weshay|ruck, regarding job renaming patches - I am wondering what the correct step - we should merge in criteria file first and then the job def.. or vice versa.. | 16:33 |
| ysandeep | I guess it won't be much a problem if we make sure those patches merge together(irrespective of the order) | 16:35 |
| weshay|ruck | ya.. we may want to merge on a weekend | 16:51 |
| *** jpena is now known as jpena|off | 16:52 | |
| *** ysandeep is now known as ysandeep|away | 16:53 | |
| rlandy | weshay|ruck: https://review.opendev.org/c/openstack/tripleo-ci/+/799859 | 17:20 |
| dviroel | "OpenStack next release name is final- 'OpenStack Yoga'" | 19:05 |
| rlandy | love that | 19:49 |
| weshay|ruck | heh.. | 20:11 |
| weshay|ruck | dviroel, 0/ | 20:48 |
| weshay|ruck | dviroel, please read through https://projects.engineering.redhat.com/browse/TRIPLEOCI-511 you could play a key role in this one https://projects.engineering.redhat.com/browse/TRIPLEOCI-511 | 20:49 |
| weshay|ruck | rlandy, fyi.. https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34341 going to try and clean up some logging.. something to watch, add input | 21:18 |
| rlandy | ok | 21:19 |
| rlandy | weshay|ruck: almost container building | 21:19 |
| rlandy | trying to keep head down there atm | 21:19 |
| rlandy | weshay|ruck: does this look right for containers in scenario000 standalone? https://logserver.rdoproject.org/65/31165/45/check/periodic-tripleo-ci-centos-8-scenario000-standalone-master/5118dd2/logs/undercloud/var/log/extra/ | 21:45 |
| rlandy | weshay|ruck: btw ... taking notes here: https://review.rdoproject.org/etherpad/p/tcib-c9-containers-stream9-development | 21:48 |
| *** rlandy is now known as rlandy|bbl | 22:25 | |
| weshay|ruck | rlandy this is wrong https://review.rdoproject.org/r/c/rdo-jobs/+/34420/1/zuul.d/standalone-jobs.yaml | 22:43 |
| weshay|ruck | oh wait.. this is what I was looking for https://review.opendev.org/c/openstack/tripleo-ci/+/799859/4/zuul.d/standalone-jobs.yaml | 22:44 |
| weshay|ruck | https://logserver.rdoproject.org/65/31165/45/check/periodic-tripleo-ci-centos-8-scenario000-standalone-master/5118dd2/logs/undercloud/var/log/tempest/stestr_results.html.gz | 22:45 |
| weshay|ruck | that looks right | 22:45 |
| weshay|ruck | although.. it doesn't seem to be just deploying keystone | 22:46 |
| weshay|ruck | maybe need to take another look at https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/ci/environments/scenario000-standalone.yaml | 22:47 |
| weshay|ruck | ya.. it pulled in a bunch of containers https://logserver.rdoproject.org/65/31165/45/check/periodic-tripleo-ci-centos-8-scenario000-standalone-master/5118dd2/logs/undercloud/var/log/extra/podman/podman_allinfo.log.txt.gz | 22:48 |
| weshay|ruck | so.. tldr.. the stack deployment is fine.. is keystone only | 22:52 |
| weshay|ruck | https://logserver.rdoproject.org/65/31165/45/check/periodic-tripleo-ci-centos-8-scenario000-standalone-master/5118dd2/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 22:52 |
| weshay|ruck | however we have to limit what the deployment pulls down for containers | 22:53 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!