| *** rfolco has quit IRC | 00:22 | |
| *** rlandy has quit IRC | 01:28 | |
| *** apetrich has quit IRC | 02:14 | |
| *** skramaja has joined #oooq | 04:01 | |
| *** ykarel|away is now known as ykarel | 04:14 | |
| *** udesale has joined #oooq | 04:28 | |
| *** saneax has joined #oooq | 04:35 | |
| *** raukadah is now known as chandankumar | 05:09 | |
| *** marios has joined #oooq | 05:17 | |
| *** ratailor has joined #oooq | 05:26 | |
| ysandeep | Hi guys, need reviews on sc1 c8 component jobs - https://review.rdoproject.org/r/#/c/28226/ .. test run are green | 05:39 |
|---|---|---|
| ysandeep | train ^^ | 05:43 |
| marios | bhagyashris: o/ hi i think we should change the times a bit there https://review.rdoproject.org/r/#/c/28404/2/zuul.d/upstream.yaml@170 | 06:16 |
| marios | i don't know why weshay_ruck was in such a rush to merge | 06:16 |
| marios | bhagyashris: the stable3 and 4/5 are running at exactly same times | 06:16 |
| marios | bhagyashris: also i guess you'll be adding stable2-centos7 as a new change now | 06:17 |
| marios | ack ysandeep added to my list | 06:19 |
| ysandeep | marios, thank you | 06:20 |
| *** ccamacho has joined #oooq | 06:42 | |
| bhagyashris | marios, yes | 06:45 |
| bhagyashris | marios, plz review these two patches https://review.rdoproject.org/r/#/c/28443/ and https://review.rdoproject.org/r/#/c/28444/ | 06:46 |
| bhagyashris | marios, in the second ^ patch i have changed the time | 06:47 |
| marios | bhagyashris: ack thanks adding to list | 06:48 |
| bhagyashris | marios, thanks :) | 06:48 |
| *** amoralej|off is now known as amoralej | 07:12 | |
| *** jbadiapa has joined #oooq | 07:16 | |
| *** tosky has joined #oooq | 07:33 | |
| *** zbr has joined #oooq | 07:35 | |
| *** jfrancoa has joined #oooq | 07:56 | |
| akahat | kopecmartin, https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_285/731501/19/check/tripleo-ci-centos-8-scenario010-standalone/2856b04/logs/undercloud/home/zuul/tempest/etc/ | 07:57 |
| kopecmartin | akahat: thanks, but the tempest.conf contains f.e. load_balancer.member_role = Member although based on the patch, it should be set to 'member' .. so seems there are some overrides set in the job which makes me wonder what other options are overridden | 08:06 |
| *** dtantsur|afk is now known as dtantsur | 08:10 | |
| chandankumar | kopecmartin, akahat you need to send a patch to remove these setting https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset062.yml#L59 and put temepstconf patch as depends on | 08:13 |
| *** sanjayu_ has joined #oooq | 08:17 | |
| *** derekh has joined #oooq | 08:19 | |
| *** saneax has quit IRC | 08:20 | |
| *** sanjayu_ has quit IRC | 08:22 | |
| *** sanjayu_ has joined #oooq | 08:22 | |
| ysandeep | marios, thanks for review https://review.rdoproject.org/r/#/c/28226/ , updated , Need +w if it looks okay to you | 08:23 |
| ysandeep | in your next turn | 08:23 |
| *** sanjayu__ has joined #oooq | 08:25 | |
| *** sanjayu_ has quit IRC | 08:27 | |
| *** zbr7 has joined #oooq | 08:28 | |
| zbr7 | yesterday it was fedora images failing to create swap, this morning irccloud down for 8h and counting.... | 08:35 |
| zbr7 | had two spend two hours reconfiguring my irc, also cancelled irccloud subscription | 08:35 |
| marios | ack y | 08:43 |
| marios | ysandeep: ^ | 08:43 |
| marios | zbr7: sorry i didn't get to your reviews in time i did check them but they were merged | 08:44 |
| marios | i was so dissapointed. | 08:44 |
| zbr7 | do I sense some sarcasm around? | 08:45 |
| marios | zbr7: :) | 08:45 |
| marios | zbr7: i was not sarcastic about saying sorry, maybe a little about the dissapointment... for the record | 08:45 |
| panda | zbr7: what was the problem with molecule here again ? https://review.rdoproject.org/zuul/build/42463622b404480eb8c6b798d1b2c26a was it the mirrors ? | 09:34 |
| panda | zbr7: please upgrade to zbr8 | 09:35 |
| marios | panda: we had to pin to 7 there was some bug in zbr8 | 09:40 |
| marios | panda: until they release 9 | 09:40 |
| marios | sigh | 09:40 |
| panda | zbr7: you have to test yourself more. | 09:41 |
| panda | marios: I think even zbr7 has crashed | 09:44 |
| panda | I'm going to git bisect and see what's the most stable release we have. Hoping we have tests for that revision | 09:45 |
| marios | panda: honestly this will all go away if we just move to containerized zbr | 09:50 |
| *** derekh has quit IRC | 09:57 | |
| *** derekh has joined #oooq | 09:58 | |
| zbr7 | nope, that is reqs conflict, may be even python version | 10:08 |
| zbr7 | likely this job run with py27 | 10:09 |
| chandankumar | sshnaidm|ruck, bhagyashris ykarel https://review.rdoproject.org/r/28450 quick merge | 10:10 |
| ykarel | chandankumar, ack, chandankumar is the new job added in promotion criteria too? | 10:11 |
| chandankumar | ykarel, sshnaidm|ruck https://review.rdoproject.org/r/28451 this one also | 10:15 |
| ykarel | ack +! | 10:16 |
| sshnaidm|ruck | chandankumar, wait with merging this please ^ | 10:16 |
| sshnaidm|ruck | chandankumar, let's wait for a its first run | 10:16 |
| chandankumar | sshnaidm|ruck, sure | 10:16 |
| panda | zbr7: thanks for not referencing me in the answer | 10:17 |
| panda | zbr7: how cna we move the job to py3 ? I thought your patch already covered everything | 10:18 |
| panda | zbr7: don't disappear again ..... | 10:19 |
| bhagyashris | chandankumar, ack | 10:22 |
| *** pojadhav is now known as pojadhav|afk | 10:26 | |
| ysandeep | Hey folks, Need reviews c8 train sc2/3 component jobs https://review.rdoproject.org/r/#/c/28339/ , test runs are green | 10:32 |
| *** rfolco has joined #oooq | 10:38 | |
| panda | zbr7: I can do it but you have to help me .... | 10:38 |
| zbr7 | ok, i can run/hide but cannot escape | 10:39 |
| chandankumar | zbr7, is someone working on https://logserver.rdoproject.org/51/28451/1/check/molecule-container-push-delegated-centos-7/47eff43/job-output.txt | 10:40 |
| chandankumar | ERROR: Could not find a version that satisfies the requirement molecule<3.1,>=3.0.4 | 10:41 |
| panda | zbr7: all jobs use the same install-test-python, which should use py3 ... | 10:41 |
| panda | zbr7: and the same nodes too | 10:42 |
| *** bhagyashris is now known as bhagyashris|brb | 10:42 | |
| zbr7 | give me 10mins, i am overloaded. | 10:42 |
| *** apetrich has joined #oooq | 10:44 | |
| panda | zbr7: make it 1h then, i'm gonna take my lunch | 10:49 |
| *** bhagyashris|brb is now known as bhagyashris | 10:59 | |
| *** udesale_ has joined #oooq | 11:03 | |
| zbr | tbh, it makes no sense to fix this issue, as we can just switch to mol base job instead. | 11:04 |
| zbr | these legacy jobs give me headaches | 11:05 |
| *** udesale has quit IRC | 11:06 | |
| *** ysandeep is now known as ysandeep|brb | 11:11 | |
| zbr | can someone please explain me what is the purpose of rdo-tox-molecule-delegated-centos-8-ci-config job? | 11:12 |
| zbr | is testing what? | 11:12 |
| zbr | which scenarion? | 11:13 |
| *** pojadhav|afk is now known as pojadhav | 11:14 | |
| panda | zbr: all scenarios that are non-delegated | 11:18 |
| panda | zbr: mmhh, nope | 11:19 |
| zbr | panda: slap yourself | 11:20 |
| arxcruz | soniya29|rover: kopecmartin chandankumar https://review.opendev.org/#/c/739948/ | 11:20 |
| panda | zbr: I wasn't far from the truth | 11:21 |
| panda | zbr: all scenarios that are delegated | 11:21 |
| panda | zbr: defined in rdo-jobs | 11:21 |
| panda | zbr: guess who created the parent for that job ? | 11:21 |
| panda | zbr: now slap yourself even harder. | 11:22 |
| *** dsneddon has quit IRC | 11:25 | |
| *** ysandeep|brb is now known as ysandeep | 11:25 | |
| *** dsneddon has joined #oooq | 11:26 | |
| *** sanjayu__ has quit IRC | 11:36 | |
| *** pojadhav is now known as pojadhav|brb | 11:39 | |
| *** rlandy has joined #oooq | 11:41 | |
| chandankumar | rlandy, Hello, https://review.opendev.org/#/c/739909/ good to go | 11:43 |
| rlandy | chandankumar: k - will merge that | 11:44 |
| rlandy | chandankumar: https://review.opendev.org/#/c/739385/ could go - if we want to merge with octavia-api in excludes list | 11:46 |
| chandankumar | rlandy, yes, https://review.opendev.org/#/c/739385/19 good to go, waiting on marios vote | 11:47 |
| chandankumar | dpawlik2, amoralej ykarel https://review.rdoproject.org/r/#/c/28406/ please have a look, these is a change in the pipeline so we also need to change the path in log pruner path | 11:48 |
| marios | checking rlandy chandankumar | 11:55 |
| rlandy | chandankumar: marios: also - pls review https://code.engineering.redhat.com/gerrit/#/c/205378/ | 11:55 |
| rlandy | won't merge w/o https://review.opendev.org/#/c/739909/ | 11:56 |
| dpawlik2 | chandankumar: we don't use that script anymore, but let me check one think | 11:57 |
| marios | rlandy: ack but did you mean 14:56 < rlandy> won't merge w/o https://review.opendev.org/#/c/739909/ | 11:58 |
| marios | rlandy: did you mean https://review.opendev.org/#/c/739385/ | 11:58 |
| *** amoralej is now known as amoralej|lunch | 12:00 | |
| rlandy | marios: yes | 12:00 |
| marios | rlandy: lgtm can't see any nits | 12:03 |
| marios | rlandy: i expect there will be a new patch to wire up the upload-dockerfiles.sh in internal-config? | 12:03 |
| rlandy | marios: so actually that patch is merged | 12:04 |
| rlandy | config so I had to merge to get it to work | 12:04 |
| marios | rlandy: oh ok :D you wired it up before it existed | 12:04 |
| marios | rlandy: yeah ack | 12:04 |
| ykarel | chandankumar, ack | 12:04 |
| rlandy | marios: I thought about moving the .sh file to config | 12:04 |
| marios | rlandy: no as we discussed before easier to work with like that | 12:05 |
| rlandy | but yeah - much easier to make changes here | 12:05 |
| marios | rlandy: unless there is a reason to lets not put anything we don't have to into config | 12:05 |
| * rlandy avoids config like the plague | 12:05 | |
| panda | config-19 | 12:08 |
| marios | lol | 12:08 |
| chandankumar | dpawlik2, ok, let me know where to change then | 12:09 |
| rlandy | panda++ | 12:09 |
| chandankumar | rlandy, marios I think all patches related to new container build merged or in gates | 12:11 |
| rlandy | nice | 12:11 |
| *** pojadhav|brb is now known as pojadhav | 12:12 | |
| weshay_ruck | rlandy, can you start the retro for me | 12:26 |
| rlandy | weshay_ruck: yep | 12:26 |
| *** ysandeep is now known as ysandeep|mtg | 12:29 | |
| panda | akahat: retrosperctive | 12:40 |
| panda | akahat: ? | 12:46 |
| marios | panda: why are you not using your headset ? (I think you're not?) | 12:52 |
| marios | panda: not as clear as other times | 12:52 |
| panda | marios: maybe jsut low volume | 12:54 |
| marios | panda: ack | 12:55 |
| panda | zbr: retrospective ? | 12:56 |
| zbr | ouch | 12:57 |
| *** amoralej|lunch is now known as amoralej | 13:11 | |
| marios | panda: :( | 13:20 |
| *** ccamacho has quit IRC | 13:23 | |
| *** skramaja has quit IRC | 13:26 | |
| *** TrevorV has joined #oooq | 13:28 | |
| *** ysandeep|mtg is now known as ysandeep | 13:30 | |
| *** pojadhav is now known as pojadhav|brb | 13:31 | |
| *** pojadhav|brb is now known as pojadhav | 13:32 | |
| bhagyashris | Hi all plz vote on these two patches https://review.rdoproject.org/r/#/c/28443/ and https://review.rdoproject.org/r/#/c/28444/ | 13:32 |
| bhagyashris | akahat, arxcruz chandankumar sshnaidm|ruck weshay_ruck marios ysandeep pojadhav soniya29|rover rlandy rfolco panda ^^ | 13:33 |
| pojadhav | bhagyashris, ack | 13:33 |
| soniya29|rover | bhagyashris, sure | 13:33 |
| akahat | Not able to attend meeting due to stuck in rain and electricity. :( | 13:34 |
| ysandeep | bhagyashris, both lgtm +1 posted | 13:42 |
| bhagyashris | ysandeep, thanks :) | 13:43 |
| bhagyashris | weshay_ruck, i will also need to take about rename stuff for cockpit https://tree.taiga.io/project/tripleo-ci-board/task/1865?kanban-status=1447276 any suggestion about that ? | 13:46 |
| bhagyashris | take care * | 13:46 |
| weshay_ruck | bhagyashris, ya.. I'll start writing docs on how to use the cockpit and design it | 13:46 |
| bhagyashris | weshay_ruck, ok, Do i rename there as well or hold for some time .. | 13:48 |
| *** Goneri has joined #oooq | 13:48 | |
| bhagyashris | i see you created this hackmd https://hackmd.io/sZTpsWkIQs2GsUjiK7qhCg | 13:49 |
| *** ysandeep is now known as ysandeep|afk | 13:52 | |
| weshay_ruck | bhagyashris, I'll cc you on patches.. so you have an example | 13:58 |
| *** ysandeep|afk is now known as ysandeep | 14:00 | |
| bhagyashris | weshay_ruck, ok np thank you :) | 14:06 |
| *** bhagyashris is now known as bhagyashris|afk | 14:07 | |
| *** bhagyashris|afk is now known as bhagyashris | 14:17 | |
| *** ratailor has quit IRC | 14:18 | |
| *** pojadhav is now known as pojadhav|afk | 14:18 | |
| *** rlandy is now known as rlandy|ruck | 14:19 | |
| weshay_ruck | OH train promoted :) | 14:29 |
| weshay_ruck | rlandy|ruck, https://review.opendev.org/#/c/739445/ | 14:37 |
| rlandy|ruck | ok - cool - that can run again | 14:38 |
| ysandeep | folks, please review c8 train tempest component jobs - https://review.rdoproject.org/r/#/c/28343/ - test run are green https://review.rdoproject.org/r/#/c/26273/ | 14:44 |
| weshay_ruck | arxcruz, https://review.opendev.org/#/c/739445/ | 14:49 |
| weshay_ruck | ah.. you +1'd | 14:49 |
| weshay_ruck | nevermind | 14:49 |
| arxcruz | ;) | 14:49 |
| weshay_ruck | last call for core? | 14:49 |
| weshay_ruck | thanks rlandy|ruck | 14:50 |
| arxcruz | chandankumar: kopecmartin before you leave https://review.opendev.org/#/c/739948/ | 14:50 |
| arxcruz | weshay_ruck: all good | 14:50 |
| rlandy|ruck | migi: hey ... see any issues with the 'nova_wait_for_compute_service' container in rhos-17? ... | 14:52 |
| rlandy|ruck | https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-compute-rhos-17/a26238a/logs/undercloud/home/zuul/standalone_deploy.log | 14:53 |
| weshay_ruck | arxcruz, can we chat for a sec | 14:53 |
| arxcruz | weshay_ruck: sure | 14:53 |
| rlandy|ruck | FATAL | Check containers status | standalone | error={"changed": false, "msg": "Failed container(s): ['nova_wait_for_compute_service'], check logs in /var/log/containers/stdouts/"} | 14:53 |
| weshay_ruck | arxcruz, https://meet.google.com/pst-adyu-vab | 14:53 |
| rlandy|ruck | migi: ^^ catching that in the compute component pipeline | 14:54 |
| *** ysandeep is now known as ysandeep|away | 14:55 | |
| migi | rlandy|ruck: checking | 15:01 |
| rlandy|ruck | migi: the whole compute pipeline is red with the same error - in process of BZ'ing it - if you are not yet aware of the issue | 15:03 |
| migi | rlandy|ruck: ack container is not running properly, do we have logs from inside of container nova services ? | 15:03 |
| kopecmartin | arxcruz: syntax is fine, do you want me to also check the releases and tests based on the bugs? or are those just test data? | 15:03 |
| rlandy|ruck | migi: ack ... https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-compute-rhos-17/f87e85b/logs/undercloud/var/log/containers/stdouts/nova_wait_for_compute_service.log | 15:04 |
| arxcruz | kopecmartin: just test data, I wrote a script to collect everything from the old validate-tempest skip files and run the validate | 15:04 |
| arxcruz | so i'm 99.99% confident it's working :D | 15:04 |
| *** ykarel is now known as ykarel|away | 15:04 | |
| kopecmartin | arxcruz: i see, ok then | 15:04 |
| migi | rlandy|ruck: do you know what version of ansible is used for deployment ? | 15:05 |
| rlandy|ruck | migi: we install 2.9 | 15:05 |
| migi | rlandy|ruck: ok please log a bug and pass it to me. I will check with my team to see what can be the issue | 15:07 |
| rlandy|ruck | migi: ack - will do | 15:07 |
| rlandy|ruck | panda: can we promote the latest tripleo-ci-testing hash in rhos-17? | 15:08 |
| rlandy|ruck | aggregate_hash": "520c86681a3624544f065b72aab1a983" | 15:08 |
| rlandy|ruck | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-periodic-rhos-17%20 | 15:08 |
| rlandy|ruck | panda: we had a clean run | 15:08 |
| rlandy|ruck | there than 10 | 15:09 |
| panda | rlandy|ruck: missing jobs ['periodic-tripleo-ci-rhel-8-scenario010-standalone-rhos-17'] | 15:11 |
| rlandy|ruck | panda: so the promoter is actually running? | 15:11 |
| panda | rlandy|ruck: yes | 15:12 |
| rlandy|ruck | we haven't promoted since 06/25 | 15:12 |
| rlandy|ruck | sec - finishing bug | 15:12 |
| panda | rlandy|ruck: I can only assume criteria need adjustment | 15:14 |
| rlandy|ruck | panad: ok - I'll take care of it | 15:15 |
| rlandy|ruck | migi: https://bugzilla.redhat.com/show_bug.cgi?id=1854993 | 15:16 |
| openstack | bugzilla.redhat.com bug 1854993 in openstack-nova "Compute component pipeline tests are failing standalone deployment with "Failed container(s): ['nova_wait_for_compute_service']"" [Unspecified,New] - Assigned to mpryc | 15:16 |
| *** ccamacho has joined #oooq | 15:20 | |
| *** ccamacho has quit IRC | 15:20 | |
| *** ccamacho has joined #oooq | 15:20 | |
| migi | rlandy|ruck: thanks output seems similar to https://bugzilla.redhat.com/show_bug.cgi?id=1685310#c2, but may be unrelated as it's different service | 15:21 |
| openstack | bugzilla.redhat.com bug 1685310 in python-oslo-cache "undercloud installation fails while running nova_cellv2_discover_hosts container" [Urgent,Closed: currentrelease] - Assigned to michele | 15:21 |
| migi | rlandy|ruck: anyway will check with dfg | 15:22 |
| rlandy|ruck | that was a long time ago | 15:22 |
| ykarel|away | rlandy|ruck, for nova_wait_compute issues, detail can be found in nova-compute log | 15:25 |
| ykarel|away | cotainers/nova/nova-compute.log | 15:26 |
| ykarel|away | s/cotainers/containers | 15:26 |
| rlandy|ruck | k | 15:26 |
| migi | ykarel|away: rlandy|ruck yeah but are they collected ? | 15:27 |
| migi | rlandy|ruck: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-compute/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-standalone-full-tempest-scenario-compute-rhos-17/f87e85b/logs/undercloud/var/log/containers/nova/nova-compute.log | 15:27 |
| migi | yep | 15:27 |
| rlandy|ruck | ok - so I guess there is your error | 15:28 |
| rlandy|ruck | migi: are you adding that to the BZ or should I? | 15:28 |
| migi | rlandy|ruck: so looks like os_traits should be different | 15:29 |
| migi | I am now | 15:29 |
| dpawlik2 | chandankumar: sorry for delay | 15:29 |
| dpawlik2 | chandankumar so since few months we are using this pruner script: https://softwarefactory-project.io/r/gitweb?p=software-factory/sf-infra.git;a=blob;f=roles/logserver/files/purge-logs.py;h=1a8e1827b134d39d74ee096262fa4f56190cbfad;hb=HEAD | 15:30 |
| migi | rlandy|ruck: component pipeline ROCKS ! see https://bugzilla.redhat.com/show_bug.cgi?id=1854993#c1 | 15:33 |
| openstack | bugzilla.redhat.com bug 1854993 in openstack-nova "Compute component pipeline tests are failing standalone deployment with "Failed container(s): ['nova_wait_for_compute_service']"" [Unspecified,New] - Assigned to mpryc | 15:33 |
| migi | rlandy|ruck: quick and easy to discover such issues :) weshay_ruck ^^ | 15:34 |
| weshay_ruck | ah.. I've seen that issue quite a bit upstream | 15:34 |
| weshay_ruck | I wonder though if it is specific to the nova container.. | 15:35 |
| migi | weshay_ruck: yep but it's nice to catch them in the process of importing different parts | 15:35 |
| weshay_ruck | ++ | 15:35 |
| weshay_ruck | yes | 15:35 |
| weshay_ruck | of course | 15:35 |
| migi | weshay_ruck: and so much easier to find it | 15:35 |
| rlandy|ruck | migi: cool - now we can get that debuf off our hands and into the dfg's | 15:37 |
| migi | rlandy|ruck: yep is os_traits separate component ? If yes then nova will pass when next import happens | 15:37 |
| rlandy|ruck | migi: idk where that comes from - will have to dig for it - but if it's not in your component, we will have to promote | 15:39 |
| weshay_ruck | rlandy|ruck, migi ya.. it looks like both upstream / downstream compute are failing.. thanks for watching it rlandy|ruck | 15:39 |
| rlandy|ruck | which I am trying to do - once we sort out the octavia issue | 15:39 |
| rlandy|ruck | talking with carlos | 15:39 |
| migi | rlandy|ruck: yep | 15:39 |
| rlandy|ruck | migi/l I didn't CIX that | 15:54 |
| migi | rlandy|ruck: ok, now who is responsible for getting new os_traits ? | 15:55 |
| migi | rlandy|ruck: will this happen now automagically if it's separate component | 15:55 |
| rlandy|ruck | sec | 15:55 |
| migi | rlandy|ruck: or if it's within same component may require manual | 15:55 |
| migi | ok | 15:55 |
| migi | rlandy|ruck: I will be online in 2h going offline now | 15:55 |
| migi | need to relocate | 15:56 |
| rlandy|ruck | guess there is a whole repo https://opendev.org/openstack/os-traits/ | 15:57 |
| *** dtantsur is now known as dtantsur|afk | 16:01 | |
| rlandy|ruck | migi: I'm rerunning a test to see if we can promote | 16:04 |
| rlandy|ruck | if you still need a newer version, my guess is that it lands with the dfg | 16:04 |
| rlandy|ruck | I'll comment on the bug | 16:04 |
| *** marios is now known as marios|out | 16:06 | |
| *** amoralej is now known as amoralej|off | 16:06 | |
| *** udesale_ has quit IRC | 16:16 | |
| *** gouthamr has quit IRC | 16:26 | |
| *** derekh has quit IRC | 16:41 | |
| *** chandankumar has quit IRC | 16:44 | |
| *** soniya29|rover has quit IRC | 16:46 | |
| *** ykarel|away has quit IRC | 16:46 | |
| *** akahat has quit IRC | 16:46 | |
| *** whoami-rajat has quit IRC | 16:46 | |
| sshnaidm|ruck | rlandy|ruck, weshay_ruck we are going to promote queens soon, and we'll be totally green | 16:46 |
| *** gouthamr has joined #oooq | 16:52 | |
| rfolco | zbr, mol-promoter is getting close to green... a few more interactions and I fix it I believe... for now, we can merge as is if possible the breakdown | 16:57 |
| rfolco | https://review.rdoproject.org/r/#/c/28439/ | 16:57 |
| rfolco | panda, ^ | 16:57 |
| weshay_ruck | sshnaidm|ruck, \0/ | 17:02 |
| weshay_ruck | sshnaidm|ruck+++++ | 17:02 |
| *** chandan_kumar has joined #oooq | 17:06 | |
| *** jmasud has quit IRC | 17:07 | |
| rlandy|ruck | nice | 17:09 |
| *** chandan_kumar is now known as chandankumar | 17:10 | |
| *** marios|out has quit IRC | 17:12 | |
| weshay_ruck | rlandy|ruck, before I dig in.. any known issues w/ https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa | 17:18 |
| rlandy|ruck | weshay_ruck: yes - there are two open bugs | 17:18 |
| rlandy|ruck | note that job moved to non-voting | 17:19 |
| rlandy|ruck | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa&branch=stable%2Ftrain | 17:20 |
| rlandy|ruck | ^^ may be a new issue | 17:20 |
| * rlandy|ruck will discuss at 1-on-1 | 17:20 | |
| *** akahat has joined #oooq | 17:25 | |
| weshay_ruck | rlandy|ruck, https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa&branch=master&branch=stable%2Fussuri | 17:37 |
| weshay_ruck | rlandy|ruck, https://hackmd.io/SeX-0f7DS6a39T-HNKi4kg | 17:40 |
| *** pojadhav|afk is now known as pojadhav | 17:40 | |
| weshay_ruck | rlandy|ruck, https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-octavia/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario010-standalone-octavia-rhos-17/32de2bc/job-output.txt | 17:48 |
| rlandy|ruck | weshay_ruck: https://bugzilla.redhat.com/show_bug.cgi?id=1854993 | 17:55 |
| openstack | bugzilla.redhat.com bug 1854993 in openstack-nova "Compute component pipeline tests are failing standalone deployment with "Failed container(s): ['nova_wait_for_compute_service']"" [Unspecified,New] - Assigned to mpryc | 17:55 |
| weshay_ruck | rlandy|ruck, https://bugzilla.redhat.com/show_bug.cgi?id=1854993 | 18:02 |
| openstack | bugzilla.redhat.com bug 1854993 in openstack-nova "Compute component pipeline tests are failing standalone deployment with "Failed container(s): ['nova_wait_for_compute_service']"" [Unspecified,New] - Assigned to mpryc | 18:02 |
| *** rlandy|ruck is now known as rlandy|ruck2 | 18:10 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 18:25 | |
| rlandy|ruck2 | weshay_ruck: migi: FYI ... https://bugs.launchpad.net/tripleo/+bug/1886862 - associated upstream bug | 18:28 |
| openstack | Launchpad bug 1886862 in tripleo " Compute component pipeline tests are failing standalone deployment with "Failed container(s): ['nova_wait_for_compute_service']" - module 'os_traits' has no attribute 'HW_CPU_X86_AVX512VBMI'" [Critical,Triaged] | 18:28 |
| *** apetrich has quit IRC | 19:31 | |
| *** apetrich has joined #oooq | 19:31 | |
| weshay_ruck | rlandy|ruck2, have to run an errand, back later | 19:44 |
| rlandy|ruck2 | weshay_ruck: ok | 19:44 |
| *** zbr_ has joined #oooq | 19:51 | |
| *** kopecmartin is now known as kopecmartin|pto | 19:52 | |
| *** johnsom_ has joined #oooq | 20:10 | |
| *** jmasud has joined #oooq | 20:11 | |
| *** jfrancoa has quit IRC | 20:21 | |
| rlandy|ruck2 | weshay_ruck: ping when you get back | 21:00 |
| rlandy|ruck2 | gate failures - rax | 21:04 |
| *** gouthamr has quit IRC | 21:09 | |
| *** gouthamr has joined #oooq | 21:11 | |
| *** gouthamr_ has joined #oooq | 21:15 | |
| *** johnsom_ is now known as johnsom | 21:19 | |
| *** johnsom has joined #oooq | 21:20 | |
| rlandy|ruck2 | weshay_ruck: hey | 21:20 |
| weshay_ruck | rlandy|ruck2, 0/ | 21:20 |
| rlandy|ruck2 | weshay_ruck: gate failures /o\ | 21:20 |
| rlandy|ruck2 | asked on openstack-infra abut RAX | 21:20 |
| weshay_ruck | ya.. I've been trying to see if there is a trend | 21:20 |
| weshay_ruck | other than lots of them | 21:21 |
| rlandy|ruck2 | looks like we are racking up a fair amount of timeouts there on the mirrors | 21:21 |
| weshay_ruck | rlandy|ruck2, we may need to slow our promotions down | 21:22 |
| weshay_ruck | rlandy|ruck2, http://rhos-release.virt.bos.redhat.com:3030/rhosp | 21:22 |
| weshay_ruck | I'll shut off the promoters | 21:22 |
| rlandy|ruck2 | weshay_ruck: in other news, switching the container check job to tripleo-ci-testing got by and merged the new conatiner build/push/upload into the 17 integration line | 21:22 |
| rlandy|ruck2 | checking ci.centos | 21:23 |
| rlandy|ruck2 | weshay_ruck: slow down due to capacity? | 21:23 |
| rlandy|ruck2 | we can change the trigger timing | 21:23 |
| rlandy|ruck2 | once a day | 21:23 |
| weshay_ruck | rlandy|ruck2, aye.. so I think the mirrors have trouble keeping up w/ the container changes | 21:24 |
| rlandy|ruck2 | ah | 21:24 |
| weshay_ruck | rlandy|ruck2, turned them both off | 21:24 |
| rlandy|ruck2 | weshay_ruck: why turn them off? | 21:25 |
| rlandy|ruck2 | we can change the trigger times | 21:25 |
| weshay_ruck | instead of rerunning every 10 minutes | 21:25 |
| weshay_ruck | we could set it to every 24 hours | 21:25 |
| rlandy|ruck2 | weshay_ruck: I'd go for 12 hours | 21:26 |
| weshay_ruck | k.. /me looks at it | 21:26 |
| rlandy|ruck2 | weshay_ruck: just fyi ... opened https://bugs.launchpad.net/tripleo/+bug/1886867 | 21:49 |
| openstack | Launchpad bug 1886867 in tripleo "Tripleo upstream jobs are failing undercloud install/standalone deploy with "requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url"" [High,Triaged] | 21:49 |
| rlandy|ruck2 | also chatting with fungi on #openstack-infra | 21:49 |
| weshay_ruck | oh geez | 21:50 |
| *** rfolco has quit IRC | 21:52 | |
| weshay_ruck | rlandy|ruck2, totally spread out http://dashboard-ci.tripleo.org/d/3sUHfh9Wz/jobs-exploration?orgId=1&from=now-24h&to=now&fullscreen&panelId=25 | 21:52 |
| weshay_ruck | rlandy|ruck2, may be tempest | 21:53 |
| *** TrevorV has quit IRC | 21:53 | |
| rlandy|ruck2 | ugh | 21:53 |
| rlandy|ruck2 | weshay_ruck: sometimes you see the same problem in undercloud install | 21:53 |
| weshay_ruck | rlandy|ruck2, seen this a few times | 21:59 |
| weshay_ruck | rlandy|ruck2, probably related to the bug you and migi where looking at | 22:00 |
| weshay_ruck | msg: 'Failed container(s): [''placement_api''], check logs in /var/log/containers/stdouts/' | 22:00 |
| weshay_ruck | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_110/739764/1/gate/tripleo-ci-centos-8-standalone/110e880/logs/undercloud/home/zuul/standalone_deploy.log | 22:00 |
| *** sshnaidm|afk has quit IRC | 22:03 | |
| weshay_ruck | rlandy|ruck2, http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Failed%20container(s)%3A%20%5B''placement_api''%5D%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20voting%3A1 | 22:09 |
| rlandy|ruck2 | weshay_ruck: also added https://review.rdoproject.org/r/#/c/28465/1/ci-scripts/infra-setup/roles/get_hash/tasks/get_hashes_from_commit_yaml.yaml | 22:13 |
| rlandy|ruck2 | testproject didn;t do much to test it as it's called from a config playbook | 22:13 |
| rlandy|ruck2 | weshay_ruck: we can merge it and try if you agree | 22:13 |
| weshay_ruck | ah /me looks | 22:13 |
| weshay_ruck | 2020-07-08 21:59:32 | Ansible execution failed. playbook: deploy_steps_playbook.yaml, Run Status: failed, Return Code: None | 22:24 |
| weshay_ruck | fun | 22:24 |
| weshay_ruck | rlandy|ruck2, meh.. | 22:26 |
| rlandy|ruck2 | weshay_ruck: ok - shift time at the warehouse - back later | 22:26 |
| weshay_ruck | individual gate pass is high enough | 22:26 |
| weshay_ruck | rlandy|ruck2, 0/ | 22:26 |
| * weshay_ruck continues to poke | 22:26 | |
| *** rlandy|ruck2 is now known as rlandy|bbl | 22:27 | |
| rlandy|bbl | weshay_ruck: pls doc up the bug if anything interesting so we can pass it to morning ruck/rovers | 22:27 |
| weshay_ruck | k | 22:27 |
| rlandy|bbl | totally up zbr's path | 22:27 |
| *** tosky has quit IRC | 22:30 | |
| weshay_ruck | rlandy|bbl, this is part of the problem | 22:35 |
| weshay_ruck | http://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_110/739764/1/gate/tripleo-ci-centos-8-standalone/110e880/logs/undercloud/var/log/extra/logstash.txt | 22:35 |
| weshay_ruck | the deployment logs are not getting added to logstash for some reason | 22:35 |
| *** rfolco has joined #oooq | 22:52 | |
| *** rfolco has quit IRC | 22:58 | |
| *** rfolco has joined #oooq | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!