| *** rlandy is now known as rlandy|out | 01:25 | |
| *** dasm is now known as dasm|off | 04:04 | |
| marios | o/ | 05:05 |
|---|---|---|
| *** ysandeep|out is now known as ysandeep | 05:32 | |
| ysandeep | \o | 05:32 |
| marios | arxcruz|rover: jm1[m]: o/ looks like a new gate blocker :/ https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-undercloud-upgrade | 05:39 |
| marios | package conflict | 05:40 |
| marios | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_696/843835/81/check/tripleo-ci-centos-9-undercloud-upgrade/696f732/logs/undercloud/home/zuul/install_packages.sh.log | 05:40 |
| ysandeep | arxcruz|rover, ++ thank you for reporting https://bugs.launchpad.net/tripleo/+bug/1992465 | 05:58 |
| marios | arxcruz|rover: jm1[m]: filed it there (undercloud-upgrade) https://bugs.launchpad.net/tripleo/+bug/1992560 | 05:58 |
| marios | ysandeep: yeah and it is on the cix board (dlrn thing) so we can discuss today | 05:58 |
| ysandeep | yeah, good to have it on cix board. | 05:59 |
| marios | arxcruz|rover: jm1[m]: lets update the program call status at some point this morning | 06:00 |
| marios | we can do quick call to sync if needed | 06:00 |
| ysandeep | hey rlandy|out marios o/ Rabi requested my help to get me some CI stats for Zed for the PTG retrospctive - CI jobs added, removed, pass rate etc. | 06:25 |
| ysandeep | Since you guys have better grasp on how we capture these stats before PTG than me.Also and marios and pojadhav driving the Zed CI branching work, I told him I will discuss with you first. | 06:25 |
| ysandeep | Do you want to meet sometime this week to capture these stats , We can directly add the our stats in https://etherpad.opendev.org/p/tripleo-ptg-retrospective | 06:25 |
| marios | ysandeep: sure lets go tripleo i can help rabi out | 06:29 |
| ysandeep | thanks! | 06:30 |
| marios | ysandeep: sure man we can fill those out by this friday is just like half hour task | 06:35 |
| pojadhav | ysandeep, ack | 06:35 |
| *** pojadhav is now known as pojadhav|afk | 06:35 | |
| ysandeep | marios, sounds good, Is it okay to tell Rabi that we will fill CI stats before Friday, I think it would be easier if we add CI stats than Rabi do that, what do you think? | 06:39 |
| ysandeep | you already replied, thanks! | 06:40 |
| marios | yah whatever man. lets just bang it out maybe after scrum tomorrow? | 06:40 |
| marios | or if you find time before that and want to dig go ahead | 06:41 |
| marios | ysandeep: ^ | 06:41 |
| marios | ysandeep: as i said maybe you can think of some new/other interesting thing we can say there | 06:41 |
| marios | "we had 30 promotions of master, 35 of wallaby/9 etc etc in the last cycle" | 06:41 |
| marios | for example | 06:42 |
| ysandeep | marios, sounds good to me to meet after tomorrow's scrum. | 06:42 |
| marios | cool ... not sure how we'd get that about promotions... we'd have to query deloreadn? but there is no history ... maybe our grafana has it right? yes in the promotions tab | 06:43 |
| ysandeep | I will check how we did the previous ones in the meantime. | 06:43 |
| marios | 'no history' i mean current-tripleo is just overwritten and we only keep previous-current-tripleo | 06:43 |
| marios | anyway | 06:43 |
| marios | back to reveiws | 06:43 |
| marios | ack | 06:43 |
| marios | pojadhav|afk: can you please check comments at https://review.opendev.org/c/openstack/tripleo-ci/+/860940 when you have time? | 06:45 |
| marios | ysandeep: jm1[m]: ok to submit https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/430746 ? | 06:50 |
| marios | holding 10 mins | 06:50 |
| arxcruz|rover | marios again? :( | 06:54 |
| marios | arxcruz|rover: yeah | 06:54 |
| ysandeep | marios, yes that can go in | 06:54 |
| marios | arxcruz|rover: like it was green for a few hours right? | 06:54 |
| jm1 | good morning folks :) | 07:05 |
| ysandeep | good morning jm1 o/ | 07:07 |
| marios | jm1: arxcruz|rover: want to sync in 50 mins update prog call ? | 07:11 |
| marios | jm1: i see train didn't happen - is it close do you think you can make it today or lots missing/blockers? | 07:12 |
| marios | but good we have the bug/cix to justify that anway | 07:12 |
| ysandeep | jm1 fyi.. all the components jobs which had new hash yesterday for train were passed, only network had one job left sc010 internal kvm - I managed to get a green run for that. | 07:13 |
| jm1 | marios: yeah, lets meet at 8utc. thanks :) | 07:14 |
| jm1 | ysandeep++ oh wow, great | 07:14 |
| jm1 | marios, ysandeep: do we have to kick something to get a new train integration line promotion? | 07:15 |
| marios | jm1: we can discuss in the call but what do you mean kick? it is running periodically no? | 07:16 |
| marios | jm1: but we can also kick it if needed today | 07:16 |
| marios | i mean manually run the pipeline | 07:16 |
| ysandeep | jm1, I am waiting for periodic-tripleo-centos-8-train-component-network-promote-to-promoted-components to trigger first.. Its need to run first before intergation line run. | 07:16 |
| ysandeep | periodic-tripleo-centos-8-train-component-network-promote-to-promoted-components scheduled run is in queue | 07:16 |
| marios | ysandeep: what are you waiting for i mean will you run the line? | 07:17 |
| marios | arxcruz|rover: jm1: invite sent - ysandeep adding you feel free to join to discuss ^^ ? | 07:18 |
| Tengu | heya! twice in a row I hit this one on RDO: fatal: unable to access 'https://opendev.org/openstack/ansible-config_template.git/': Failed to connect to opendev.org port 443: No route to host | 07:18 |
| Tengu | and I think I saw it yesterday as well - though it was less visible. | 07:18 |
| ysandeep | I mean i was trying to promote components which had any new content till yesterday in train(so that those component reach downstream), only network is left.. All jobs passed but promote-to-promoted jobs haven't ran yet. | 07:18 |
| marios | ysandeep: great thanks | 07:18 |
| marios | didn't come across that one Tengu | 07:19 |
| Tengu | hmm ok. | 07:19 |
| marios | maybe others did | 07:19 |
| Tengu | sounds weird.. probably some network glitch | 07:19 |
| marios | ysandeep: added you to invite feel free to ignore or join as you can/want | 07:20 |
| ysandeep | ack, thanks! | 07:20 |
| jm1 | Tengu: saw a lot of no route to host errors yesterday. maybe that is related to the dns server issue. the fix which updated the dns servers got merged yesterday and the one for our baremetal env just a couple of minutes ago | 07:20 |
| Tengu | hmmm... I don't think any dns issue may cause a route failure. | 07:21 |
| Tengu | it can resolve apparently. unless python is doing some terrible output. | 07:21 |
| Tengu | .... wouldn't be surprising. | 07:21 |
| jm1 | Tengu: agree, the error points to some arp issues | 07:23 |
| marios | Tengu: 2022-10-12 03:14:16.277430 | primary | fatal: unable to access 'https://opendev.org/openstack/ansible-config_template.git/': Failed to connect to opendev.org port 443: No route to host | 07:24 |
| marios | https://logserver.rdoproject.org/89/860889/4/openstack-check/tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001/4fed359/job-output.txt | 07:24 |
| marios | Tengu: just found one in reviews ;) | 07:24 |
| reviewbot | Do you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks. | 07:24 |
| marios | Tengu: and it is 3rd party rdo/vexx job | 07:24 |
| Tengu | marios: how can you know it? this is my failing job: https://review.rdoproject.org/zuul/build/05057ffcda1544e29b3d548a477bbbb0 | 07:25 |
| Tengu | (so one more time vexx being lost in translation?) | 07:25 |
| marios | jm1: arxcruz|rover: something to keep an eye on if it continues today we should get a bug | 07:25 |
| marios | Tengu: how can i know what? | 07:25 |
| Tengu | marios: where job is running | 07:25 |
| marios | Tengu: https://logserver.rdoproject.org/36/861036/2/openstack-check/tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001/05057ff/zuul-info/inventory.yaml | 07:26 |
| Tengu | oh, ok | 07:26 |
| marios | Tengu: cloud: vexxhost-nodepool-tripleo | 07:26 |
| Tengu | cool! thanks for that info :) | 07:27 |
| bhagyashris | marios, hi, want to chat with you about mixed rhel ... ping me when you have free time thanks :) | 07:27 |
| marios | Tengu: jm1: arxcruz|rover: in fact maybe same issue as https://bugs.launchpad.net/tripleo/+bug/1991523 | 07:27 |
| marios | 'no route to host' | 07:28 |
| marios | bhagyashris: ack i have call in half hour after that ok? | 07:28 |
| bhagyashris | ok going for lunch may be will meet after 1 r | 07:29 |
| Tengu | marios: meh... | 07:29 |
| bhagyashris | 1 hr | 07:29 |
| marios | bhagyashris: k sounds about right | 07:30 |
| arxcruz|rover | marios iirc we have a ticket for vexx already | 07:32 |
| bhagyashris | ack | 07:33 |
| marios | arxcruz|rover: yes 10:27 < marios> Tengu: jm1: arxcruz|rover: in fact maybe same issue as https://bugs.launchpad.net/tripleo/+bug/1991523 | 07:35 |
| arxcruz|rover | marios we might have another issue | 07:41 |
| arxcruz|rover | 2022-10-12 03:27:51.814207 | primary | HttpException: 403: Client Error for url: https://compute.rdo.mtl2.vexxhost.net/v2.1/4b633c451ac74233be3721a3635275e5/os-keypairs, Quota exceeded, too many key pairs. | 07:41 |
| Tengu | "yay" | 07:42 |
| *** pojadhav|afk is now known as pojadhav | 07:44 | |
| pojadhav | marios, yep working on the same. | 07:49 |
| marios | arxcruz|rover: nice... if you have more than one example file it? | 07:51 |
| arxcruz|rover | marios checking with jcaptao | 07:51 |
| ysandeep | arxcruz|rover, jm1 some ovb stack are also stuck on deletion: http://pastebin.test.redhat.com/1077186 | 07:54 |
| marios | arxcruz|rover: o/ please join us? | 08:00 |
| pojadhav | marios, done fixed review comments, please have a look when free https://review.opendev.org/c/openstack/tripleo-ci/+/860940 | 08:03 |
| pojadhav | marios, i am also zuul error related to freeze job graph on patch : https://review.opendev.org/c/openstack/tripleo-ci/+/860940/2#message-4406fb7f0ce97d8de580aba6619497a3784a8dd0 | 08:04 |
| marios | pojadhav: k will try and check that in a bit | 08:31 |
| marios | pojadhav: btw we may have to revert zed/emit release tbd | 08:31 |
| marios | (just fyi) | 08:31 |
| marios | bhagyashris_: talk in 50 mins OK for you? | 08:38 |
| bhagyashris_ | marios, ack | 08:51 |
| marios | bhagyashris_: sent invite | 09:02 |
| marios | jm1: arxcruz|rover: chandankumar: i think i know what the issue is with that undercloud-upgrade posting test/fix in a asec | 09:02 |
| arxcruz|rover | marios ohhh, great! | 09:02 |
| chandankumar | marios: ok, waiting for review, thank you for digging in :-) | 09:03 |
| marios | chandankumar: thanks for pointer. its something we've had each time we do this .. :/ same story ;) \o/ | 09:03 |
| marios | chandankumar: https://bugs.launchpad.net/tripleo/+bug/1992560/comments/3 | 09:07 |
| chandankumar | ah i see. | 09:08 |
| ysandeep | arxcruz|rover, all the older keypairs are purged. | 09:08 |
| arxcruz|rover | ysandeep++ | 09:09 |
| marios | thank you ysandeep | 09:12 |
| marios | bhagyashris_: o/ | 09:44 |
| jm1 | marios: this one has been fixed, hasnt it? https://trello.com/c/Q8OcH2B6/2746-cixlp1992305tripleociproa-tripleo-ci-centos-9-undercloud-upgrade-is-failing-check-gate-depsolve-error-occurred-package-python3-s | 09:58 |
| bhagyashris_ | https://code.engineering.redhat.com/gerrit/plugins/gitiles/openstack/tripleo-ci-internal-config/+/refs/heads/master/zuul.d/dlrn-jobs.yaml#119 | 09:58 |
| arxcruz|rover | marios jm1 wallaby cs9 will promote | 09:58 |
| marios | bhagyashris_: https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/multinode-jobs.yaml#L64 | 10:00 |
| *** marios is now known as marios|call | 10:00 | |
| * jm1 lunch | 10:01 | |
| marios|call | jm1: yes | 10:04 |
| *** marios|call is now known as marios | 10:04 | |
| marios | jm1: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/860824 | 10:04 |
| marios | arxcruz|rover: cool | 10:04 |
| * marios foooooood | 10:08 | |
| *** ysandeep is now known as ysandeep|lunch | 10:12 | |
| *** rlandy|out is now known as rlandy | 10:30 | |
| rlandy | arxcruz|rover: jm1: hello - how are things? | 10:33 |
| rlandy | marios: ysandeep|lunch: reading back ... | 10:33 |
| arxcruz|rover | rlandy except for train, we are good | 10:33 |
| arxcruz|rover | although we have some issues | 10:33 |
| rlandy | marios: ysandeep|lunch: rt PTG stats, marios yeah we discussed earlier | 10:33 |
| rlandy | marios: ysandeep|lunch: are you all set - or do we need to talk about this? | 10:34 |
| rlandy | arxcruz|rover: wanted to run through some cix cards | 10:34 |
| arxcruz|rover | rlandy up to you | 10:34 |
| rlandy | arxcruz|rover: jm1: let me know when you have time to sync | 10:34 |
| arxcruz|rover | i do have | 10:34 |
| rlandy | also so I know what to watch in my time | 10:35 |
| rlandy | arxcruz|rover: ok - let's touch base now ... | 10:35 |
| rlandy | arxcruz|rover: https://meet.google.com/egf-xqkj-yqv?pli=1&authuser=0 | 10:35 |
| rlandy | marios: will ping afterwards - to catch up | 10:36 |
| marios | rlandy: we're good on stats we'll try do it after scrum tomorrow | 10:42 |
| marios | rlandy: should not take long | 10:43 |
| marios | rlandy: ack did a braindump just now on status ping as needed o/ | 10:44 |
| rlandy | chandankumar: hey - what's the latest on IBM cloud and https://bugs.launchpad.net/tripleo/+bug/1991093? | 11:09 |
| chandankumar | rlandy: we hold the node late night for debugging last friday | 11:18 |
| chandankumar | I have not checked back | 11:18 |
| chandankumar | I will take a look again with nicholas | 11:18 |
| *** ysandeep|lunch is now known as ysandeep | 11:21 | |
| marios | https://review.opendev.org/c/openstack/tripleo-ci/+/860940 | 11:29 |
| marios | rlandy: chandankumar: ^ | 11:29 |
| marios | arxcruz|rover: jm1: so did we get some 'organic' promotions this week i mean wallaby/master? | 11:32 |
| marios | arxcruz|rover: jm1: without chase? | 11:32 |
| marios | seems like that but wanted to confirm | 11:32 |
| marios | jm1|ruck: ^^ | 11:32 |
| *** bhagyashris_ is now known as bhagyashris | 11:33 | |
| chandankumar | pojadhav: commented! | 11:35 |
| pojadhav | chandankumar, thanks will take a look. | 11:36 |
| arxcruz|rover | jm1 yup | 11:39 |
| arxcruz|rover | marios yes, i was also surprised | 11:40 |
| marios | arxcruz|rover: nice.. rlandy confirmed ^^^ looks like the internal jobs & alternative criteria is the way | 11:40 |
| marios | :) | 11:40 |
| bhagyashris | rlandy, https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/430534 | 11:40 |
| arxcruz|rover | marios https://res.cloudinary.com/teepublic/image/private/s--SfW6L4p7--/t_Preview/b_rgb:191919,c_lpad,f_jpg,h_630,q_90,w_1200/v1605237588/production/designs/16053181_0.jpg | 11:41 |
| rlandy | marios: hey - can you remove you -1 from https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/430534 | 11:42 |
| rlandy | we can remove that last job when your BZ is solved | 11:42 |
| rlandy | but now we can get rid of the others | 11:43 |
| * pojadhav break | 11:46 | |
| marios | rlandy: but without that in criteria, it means *nothing* is qualifying the 8 content any more | 11:47 |
| marios | rlandy: we just have containers build | 11:47 |
| rlandy | marios: yeah - it's fine for now | 11:49 |
| marios | rlandy: i mean maybe keep a standalone at least then? | 11:49 |
| rlandy | that's all we want to make sure keep working | 11:49 |
| rlandy | we can keep one | 11:49 |
| marios | rlandy: but nothing is keeping it working i mean the content can break any time if we just build containers and promote each time | 11:49 |
| rlandy | but atm we are using resources we don't need | 11:49 |
| marios | rlandy: k happier to at least keep a standalone until the mixed rhel is good to go into criteria | 11:50 |
| rlandy | marios: k - so pls leave that comment and we can get rid of others | 11:50 |
| rlandy | so we are not blocking on jobs we don't care about | 11:50 |
| rlandy | marios: I'm ok with one standalone | 11:50 |
| marios | rlandy: k i replied to my earlier comment | 11:51 |
| rlandy | bhagyashris: ^^ | 11:53 |
| bhagyashris | rlandy, marios Done | 12:00 |
| rlandy | dasm|off: hi - when you are in - we wanted to get clarity on the official rbac support statement for 17.1 and 18 | 12:01 |
| rlandy | do you have any updates/confirmations there? | 12:01 |
| rlandy | we know 17 is out | 12:01 |
| bhagyashris | rlandy, marios hey should i add standalone job inti the criteria | 12:01 |
| rlandy | ysandeep: marios: ^^ fyi | 12:02 |
| rlandy | bhagyashris: marios: ack pls | 12:02 |
| rlandy | until mixed is added | 12:02 |
| rlandy | pojadhav: chandankumar left comments about the freeze job. pls ping if you need more help | 12:04 |
| rlandy | marios: ^^ fyi re: freeze - just looked at it now - chandankumar pointed to a missing job | 12:05 |
| bhagyashris | rlandy, marios https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/430906 Add rhos17.1 on rhel8 standalone job in criteria | 12:05 |
| bhagyashris | pls review when you have time | 12:05 |
| rlandy | bhagyashris: thanks - +2'ed both | 12:06 |
| bhagyashris | thanks | 12:06 |
| chandankumar | rlandy: review.rdoproject.org is accessible now, can you try again | 12:11 |
| chandankumar | ? | 12:11 |
| rlandy | chandankumar'; ack | 12:14 |
| chandankumar | rlandy: fyi https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001&skip=0 on ibm cloud is doing fine | 12:15 |
| rlandy | see it now | 12:15 |
| rlandy | chandankumar: great - fs035? | 12:15 |
| chandankumar | rlandy: https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035&skip=0 it's good | 12:17 |
| chandankumar | but there is not mch run | 12:17 |
| chandankumar | *much | 12:17 |
| rlandy | chandankumar: ok - will update the card | 12:17 |
| chandankumar | rlandy: regarding cix card https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-train&skip=0 is also not good phase, | 12:18 |
| chandankumar | *not in | 12:18 |
| rlandy | ack - so still there | 12:20 |
| rlandy | arxcruz|rover: can you dequeue and enqueue wallaby? | 12:21 |
| rlandy | stable-1 | 12:21 |
| rlandy | as well as manila, network, glance components | 12:22 |
| rlandy | since outage cleared | 12:22 |
| jm1 | rlandy, arxcruz|rover: the only missing job for train promotion passed tempest, but then failed because of the rdoproject outage. we were soooo close 😭 | 12:23 |
| arxcruz|rover | rlandy cs9, cs8 or both ? | 12:29 |
| *** ysandeep is now known as ysandeep|afk | 12:29 | |
| rlandy | arxcruz|rover: just 9 pls | 12:29 |
| rlandy | 8 is starting now | 12:29 |
| rlandy | jm1" ugh - hang in there | 12:30 |
| rlandy | jm1: at least it's better than the 1 million reruns needed on your last shift | 12:30 |
| arxcruz|rover | rlandy how to get the --newrev hash? i forgot :S | 12:31 |
| rlandy | arxcruz|rover: --newrev hash-latest-commit-on-tripleo-ci | 12:32 |
| arxcruz|rover | rlandy done | 12:35 |
| rlandy | thank you | 12:37 |
| rlandy | arxcruz|rover: jm1: http://pastebin.test.redhat.com/1077242 - from vexxhost ticket | 12:39 |
| rlandy | do you have the latest occurrence for this? | 12:39 |
| jm1 | rlandy: when did they fix the host? | 12:43 |
| arxcruz|rover | rlandy getting the info, pastebin isn't working for me now :/ | 12:44 |
| rlandy | no idea - I'm just ready the ticket | 12:47 |
| rlandy | yesterday | 12:47 |
| rlandy | Guilherme Steinmüller | 12:47 |
| rlandy | 1 day ago | 12:47 |
| rlandy | jm1: ^^ | 12:48 |
| marios | jm1|ruck: fyi updated that one https://trello.com/c/e2us0FZu#comment-6346b78a85945700fb0b9788 | 12:49 |
| rlandy | marios: chandankumar; ysandeep|afk: I need to send a next gen CI status update by me EoD - pls PM me any latest stuff to add | 12:55 |
| rlandy | thank you | 12:55 |
| chandankumar | rlandy: seen this issue https://cb72e34560a2cab5d2df-4713cd4e9994b04569e57693ba543dc2.ssl.cf1.rackcdn.com/843836/49/check/tripleo-ci-centos-9-standalone-external-compute-target-host/ff43cf4/job-output.txt | 12:57 |
| chandankumar | while working on 2 node deployment | 12:57 |
| chandankumar | on second node ssh connection got lost | 12:57 |
| chandankumar | [subnode-1]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host 158.69.67.45 port 22: Connection timed out", "unreachable": true} ? | 12:58 |
| rlandy | chandankumar: what nodeset? | 13:00 |
| rlandy | multinode? | 13:00 |
| chandankumar | rlandy: https://review.opendev.org/c/openstack/tripleo-ci/+/843836/49/zuul.d/standalone-jobs.yaml#1093 | 13:01 |
| rlandy | so yeah if bridge failed | 13:01 |
| chandankumar | nodeset: two-centos-9-nodes | 13:01 |
| pojadhav | rlandy, ack thank you | 13:01 |
| rlandy | so that will operate like a multinode job | 13:01 |
| chandankumar | yes | 13:01 |
| rlandy | is that the networking you want? | 13:02 |
| chandankumar | here is the playbook https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/843835/88/playbooks/external-compute-target-host.yml | 13:02 |
| chandankumar | rlandy: quick gmeet? | 13:02 |
| rlandy | ok | 13:02 |
| marios | rlandy: ok will send | 13:03 |
| chandankumar | rlandy: https://meet.google.com/tid-zthy-gzd?authuser=0&hl=en | 13:04 |
| rlandy | arxcruz|rover: jm1: can you join internal openstack-pcci | 13:05 |
| marios | rlandy: jm1|ruck: arxcruz|rover: looking good on gate blocker https://review.opendev.org/c/openstack/tripleo-ci/+/861047/2#message-421a99fce80e6b3ac15011439123269123da4092 just waiting on minor update to finish/report | 13:09 |
| rlandy | marios: nice - pls ping once it reports and we can vote | 13:25 |
| marios | rlandy: minor update is fine on test https://review.opendev.org/c/openstack/tripleo-heat-templates/+/861048/2#message-9509e42d92c77cc60cdcfd913121f6d246b2dcb9 | 13:25 |
| marios | oooci please vote for gate blocker: https://review.opendev.org/c/openstack/tripleo-ci/+/861047 it is about to report for check @ https://zuul.openstack.org/status#861047 | 13:26 |
| marios | arxcruz|rover: jm1|ruck: rlandy: ysandeep|afk: chandankumar: ^^ | 13:26 |
| arxcruz|rover | voted | 13:30 |
| rlandy | marios: we voted - pls w+ when ready | 13:32 |
| jm1 | rlandy: wrt vexxhost ticket and hostid question: i cannot see that ticket but it would be good to know since when this is fixed because then i know which jobs were affected and which not | 13:34 |
| ysandeep|afk | looking | 13:34 |
| rlandy | jm1: pm'ing you | 13:35 |
| ysandeep|afk | marios, +wed | 13:37 |
| * ysandeep|afk out, see everyone tomorrow | 13:38 | |
| *** ysandeep|afk is now known as ysandeep|out | 13:38 | |
| ysandeep|out | jm1, marios fyi.. downstream 16.2 got a FTBFS for tripleo component - so we need to wait there | 13:38 |
| rlandy | will raise at CIX | 13:39 |
| marios | thanks ysandeep|out | 13:40 |
| marios | rlandy: see pvt when you have a sec just fyi one of your beaker boxes got a ping | 13:41 |
| marios | arxcruz|rover: want to update the list to say trending closed with https://review.opendev.org/c/openstack/tripleo-ci/+/861047 (in case you wont be around when that merges later? ) | 14:05 |
| marios | arxcruz|rover: i was going to but maybe you prefer to close your earlier post | 14:05 |
| arxcruz|rover | marios sure doing it now | 14:05 |
| *** dasm|off is now known as dasm | 14:15 | |
| dasm | o/ | 14:15 |
| rlandy | anyone getting timeouts connecting to code.eng? | 14:16 |
| dasm | rlandy: wrt rbac support statement for 17.1 and 18. I have none. I need to sync up with dmenziba and see what's the progress there | 14:16 |
| rlandy | arxcruz|rover: jm1: ^^? | 14:16 |
| arxcruz|rover | rlandy i don't | 14:17 |
| rlandy | dasm: can you check in with them today and report back? | 14:17 |
| rlandy | dasm: we want to keep rbac jobs ONLY on revelant releases | 14:17 |
| dasm | k | 14:17 |
| jm1 | rlandy: code.eng working for me too | 14:17 |
| marios | thanks arxcruz|rover | 14:17 |
| rlandy | fatal: unable to connect to git.host.prod.eng.bos.redhat.com ... hmmm | 14:21 |
| chandankumar | see ya! | 15:03 |
| jm1 | rlandy: wasnt this the patch that ykarel prepared and pushed into 17.1? https://lada.lab4.eng.bos.redhat.com/?change_id_input=I25f75d7bd8864ea132246cea79e073232d664d76 | 15:04 |
| jm1 | rlandy: the one missing from 17.0 | 15:04 |
| * marios starts plotting his escape | 15:05 | |
| marios | rlandy: jm1|ruck: arxcruz|rover: need sthing from me? | 15:05 |
| rlandy | left ykarel a note on the card to ask if he can push the 17.0 fix | 15:05 |
| arxcruz|rover | nope | 15:05 |
| rlandy | marios: was going to you the same thing - just to get the blocker change in | 15:05 |
| rlandy | ie: I just need to get the blocker change in | 15:06 |
| marios | rlandy: yeah please if you could watch that thanks | 15:06 |
| rlandy | sure | 15:06 |
| rlandy | marios++ for keeping this rolling | 15:06 |
| marios | ++team i did just some part of all the stuff | 15:07 |
| rlandy | gthiemonge: hello ... I've been testing out the new nested-virt nodes on PSI ... | 15:07 |
| jm1 | marios: again, thank you very much for helping out with rr so often this week :) | 15:07 |
| rlandy | master shows the following error: https://sf.hosted.upshift.rdu2.redhat.com/logs/82/430582/1/check/periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master/e65121a/logs/undercloud/var/log/tempest/stestr_results.html | 15:08 |
| rlandy | Details: (PoolScenarioTest:test_UDP_LC_pool_with_listener_CRUD) show_pool operating_status failed to update to ONLINE within the required time 300. Current status of show_pool: OFFLINE | 15:08 |
| rlandy | gthiemonge: ^^ looks to me to be different from the original error we were tracking | 15:08 |
| * rlandy will LP bug this | 15:09 | |
| gthiemonge | rlandy: hi | 15:10 |
| gthiemonge | rlandy: interesting, most of the tests have passed | 15:11 |
| gthiemonge | rlandy: I'm surprised to see the list of enabled tests for sc010, do we have many tests in the skiplist? | 15:12 |
| gthiemonge | rlandy: anyways, I will take a look at this new issue | 15:12 |
| jm1 | rlandy, arxcruz|rover: taking a break now but will check for train jobs later (still running) | 15:13 |
| marios | welcome jm1 o/ | 15:13 |
| marios | rlandy: check email for multi-rhel ci status requested | 15:16 |
| rlandy | gthiemonge: I don't think we have much in the skiplist - creating LP for new failures | 15:17 |
| rlandy | marios: thanks for status update | 15:18 |
| marios | o | 15:20 |
| marios | o/ | 15:20 |
| *** marios is now known as marios|out | 15:20 | |
| rlandy | jm1: k - sure - ping when you want me to take over watching those | 15:27 |
| rlandy | gthiemonge: https://bugs.launchpad.net/tripleo/+bug/1992668 - details and logs included | 15:37 |
| gthiemonge | rlandy: thanks | 15:39 |
| rlandy | jm1: are we ready to rekick 16.2? | 16:10 |
| rlandy | 5 days ago | 16:11 |
| rlandy | osp16-2 | 16:11 |
| rlandy | current-tripleo | 16:11 |
| rlandy | rhel-8 | 16:11 |
| rlandy | 14 hours ago | 16:11 |
| rlandy | network and tripleo components are updated | 16:11 |
| rlandy | 16.2 rekicked | 16:17 |
| rlandy | arxcruz|rover" jm1: ^^ | 16:17 |
| rlandy | jm1": hmmm fs020 failed with diff tempest failures | 16:45 |
| rlandy | train | 16:45 |
| rlandy | ugh | 16:45 |
| rlandy | rekicked | 16:45 |
| rlandy | if no pass, we can promote with this skipped | 16:45 |
| rlandy | lunch - brb | 17:09 |
| rlandy | arxcruz|rover' jm1: fyi ... https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/430956 Add job definitions for fs039 and fs064 RDO jobs | 17:09 |
| rlandy | testing these out | 17:10 |
| rlandy | then will hook up with downstream triggers | 17:10 |
| rlandy | arxcruz|rover: jm1: gate blocker fix merged: https://review.opendev.org/c/openstack/tripleo-ci/+/861047 | 17:46 |
| arxcruz|rover | rlandy sending the email | 17:47 |
| arxcruz|rover | done | 17:48 |
| rlandy | arxcruz|rover: jm1: vexxhost is in the process of fixing more hypervisors | 17:50 |
| rlandy | they are waiting for jobs to finish | 17:50 |
| rlandy | I left a note saying that could take the nodes down to fix them | 17:50 |
| rlandy | we can restart jobs | 17:50 |
| arxcruz|rover | ack | 17:50 |
| rlandy | rater let's get it fixed | 17:50 |
| rlandy | we are only waiting for train fs020 | 17:50 |
| rlandy | which we can kick on ibm if needed | 17:51 |
| jm1 | rlandy: we have two hashes for train, the one from yesterday (which you rekicked) which is missing a single job only and the one from today. the latter is missing more than one job but i am rerunning them as well. lets see who wins | 18:29 |
| jm1 | rlandy: btw last 4 failures of fs20 were tempest test failures, all for different tests. so if we cant get it to pass we might want to skip promote? | 18:33 |
| rlandy | jm1: let's see how it goes | 18:33 |
| rlandy | jm1: if we don't get a pass this time, I'll put in the skip patch | 18:34 |
| rlandy | because the content is old | 18:34 |
| rlandy | jm1: I'm rechecking lucca's patches | 18:41 |
| rlandy | to see if we an merge | 18:41 |
| jm1 | rlandy: regarding 16.2, ysandeep was taking care of that and it looks like integration line is running right now | 18:43 |
| rlandy | jm1: I rekicked it | 18:43 |
| jm1 | rlandy: ah ok, great ^^ | 18:43 |
| jm1 | rlandy: luca's patches are really having a hard time.. always failing on unrelated ci issues :/ | 18:45 |
| rlandy | also rechecked the test project | 18:48 |
| rlandy | jm1: also trying to get fs039 and fs064 passing on internal | 18:49 |
| rlandy | for alternative jobs | 18:49 |
| jm1 | rlandy: saw them, thanks! | 18:51 |
| rlandy | adding trigger jobs etc. now | 18:52 |
| jm1 | rlandy: hope your alternative-cloud-patch gets merged soon | 18:52 |
| jm1 | rlandy: btw where can we find that one? | 18:53 |
| rlandy | jm1: the infra one is in | 18:55 |
| rlandy | I still need to add the rr tool | 18:55 |
| rlandy | one to report using alter jobs | 18:55 |
| jm1 | rlandy: i am curious how this infra patch looks like. it was for dlrn i guess? | 18:56 |
| rlandy | jm1: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/45468 | 18:57 |
| jm1 | rlandy: oh nice! | 18:59 |
| jm1 | arxcruz|rover, rlandy: i am out for today | 19:12 |
| rlandy | jm1: thanks | 19:13 |
| rlandy | have a good night | 19:13 |
| rlandy | jm1: anything other than train fs020 to watch? | 19:14 |
| jm1 | rlandy: there are three testprojects for train running atm, one for the old hash, two for the new hash. except for that you already took care of everything :D | 19:14 |
| jm1 | rlandy: btw fs64 internal has a post failure. reporting to dlrn failed https://sf.hosted.upshift.rdu2.redhat.com/logs/51/428951/12/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp_1supp-featureset064-internal-wallaby/47473ab/job-output.txt | 19:16 |
| rlandy | k - was juts adding trigger jobs - will look now | 19:16 |
| * jm1 out for today, have a nice evening folks :) | 19:18 | |
| rlandy | jm1[m]: victory: periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-traintestprojectmastercheck45621,1 | 19:59 |
| rlandy | 3 hrs 4 mins 22 secs2022-10-12 16:47:16SUCCESS | 19:59 |
| rlandy | ugh - wrong hash report | 20:03 |
| rlandy | arxcruz|rover: jm1[m]: fyi - new gate blocker - reporting | 21:52 |
| rlandy | amoralej|off: hi - when you get in, pls see gthiemonge comment on https://bugs.launchpad.net/tripleo/+bug/1992668/comments/2. Who would update this image? | 21:59 |
| *** rlandy is now known as rlandy|bbl | 22:04 | |
| *** dasm is now known as dasm|off | 22:09 | |
| *** rlandy|bbl is now known as rlandy | 23:50 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
