| *** rlandy is now known as rlandy|out | 00:35 | |
| rlandy|out | Tengu|rover: bhagyashris|ruck: https://review.rdoproject.org/r/c/rdo-jobs/+/46691 Fix trigger job nodeset - wallaby c9 | 01:05 |
|---|---|---|
| ysandeep | hello o/ good morning everyone | 03:51 |
| *** yadnesh|away is now known as yadnesh | 04:04 | |
| *** ysandeep is now known as ysandeep|afk | 05:07 | |
| *** pojadhav- is now known as pojadhav | 05:08 | |
| *** ysandeep|afk is now known as ysandeep | 06:18 | |
| ysandeep | reviewbot, please add in reviewlist: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/871223 | 06:27 |
| reviewbot | I have added your review to the Review list | 06:27 |
| marios | o/ | 06:30 |
| *** amoralej|off is now known as amoralej | 07:20 | |
| marios | all upstream lines are hosed Tengu|rover bhagyashris|ruck | 07:48 |
| marios | maybe that https://bugs.launchpad.net/tripleo/+bug/2003658 | 07:48 |
| Tengu|rover | hello there | 07:59 |
| dpawlik | o/ | 08:01 |
| dpawlik | are you still using registry.rdoproject.org ? | 08:02 |
| Tengu|rover | ah, a fine Monday apparently. | 08:03 |
| Tengu|rover | marios: checking. Lemme connect things up. | 08:04 |
| marios | Tengu|rover: ack i am digging a bit and we can sync in call in 25 | 08:05 |
| Tengu|rover | sup | 08:05 |
| Tengu|rover | *yup | 08:06 |
| marios | dpawlik: we should be all done now ... akahat did we have any pending patches still regarding 10:02 < dpawlik> are you still using registry.rdoproject.org ? | 08:06 |
| Tengu|rover | marios: hmmm, stupid question, but I don't see that much red on the zuul "builds" page.... ? | 08:07 |
| Tengu|rover | ah. at least train got its promotion last week. | 08:09 |
| marios | Tengu|rover: which zuul https://review.rdoproject.org/zuul/status | 08:09 |
| Tengu|rover | oh, I was on opendev | 08:09 |
| marios | eg currently running openstack-component-compute is hitting that see the retry/retry_limit jobs | 08:09 |
| Tengu|rover | hmmm /bin/sh: line 1: /home/zuul/workspace/hash_info.sh: No such file or directory | 08:10 |
| Tengu|rover | that's more infra | 08:10 |
| Tengu|rover | what's supposed to create that file? | 08:11 |
| Tengu|rover | marios: what's supposed to create that hash_info.sh? didn't find a match in oooq, oooq-extras nor tripleo-ci :/ | 08:16 |
| Tengu|rover | ah. ci-scripts/tripleo-upstream/get-hash.sh:# and hash_info.sh from ci-config ? | 08:17 |
| Tengu|rover | hm... the script creating that hash_info.sh (get-hash.sh) doesn't seem to be pushed nor called. | 08:19 |
| marios | Tengu|rover: bhagyashris|ruck: joining | 08:29 |
| Tengu|rover | in already | 08:30 |
| bhagyashris|ruck | marios, joining | 08:32 |
| marios | * https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_podman/tasks/login.yml | 08:35 |
| marios | https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-rdo-base/container-login.yaml | 08:36 |
| marios | Tengu|rover: ^^ | 08:36 |
| marios | * https://github.com/rdo-infra/rdo-jobs/commits/master | 08:36 |
| marios | * https://github.com/rdo-infra/review.rdoproject.org-config/commits/master | 08:36 |
| marios | * https://github.com/rdo-infra/ci-config/commits/master | 08:36 |
| marios | * https://opendev.org/openstack/tripleo-ansible/commits/branch/master | 08:36 |
| dpawlik | marios: so it means that we can write an email and decommission the old registry server :D ? | 08:39 |
| Tengu|rover | marios: apparently those don't have any output in the log: https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-rdo-base/container-login.yaml#L20-L30 | 08:45 |
| Tengu|rover | even with the "no_log: true" we should get something like "primary | <status>" | 08:46 |
| Tengu|rover | but we don't. | 08:46 |
| Tengu|rover | would be interesting to hold a node, and see how things are going there. | 08:46 |
| akahat | marios, dpawlik o/ patches are merged for registry migration. https://review.rdoproject.org/r/q/topic:rdo_registry_decomission | 09:01 |
| dpawlik | akahat: thats very good news | 09:03 |
| dpawlik | thanks akahat | 09:03 |
| Tengu|rover | marios: did you pass the projectname for the node hold? Just wanting to create a "hold-the-node" alias with correct params :) | 09:15 |
| Tengu|rover | ah. I think I've seen "tesproject" in your command line. | 09:15 |
| marios | Tengu|rover: pasted in pvt | 09:16 |
| Tengu|rover | ++ | 09:17 |
| dpawlik | just FYI old registry it seems to work now with new cert. Soon I will send an email about decommissioning it | 09:25 |
| marios | Tengu|rover: and now it doesnt hit.. .2023-01-23 09:25:40.019058 | primary | ok: 10 changed: 8 unreachable: 0 failed: 0 skipped: 15 rescued: 0 ignored: 0 | 09:33 |
| marios | 2023-01-23 09:25:40.019107 | secondary | ok: 10 changed: 8 unreachable: 0 failed: 0 skipped: 15 rescued: 0 ignored: 0 | 09:33 |
| marios | Tengu|rover: looks like it is running on 3rd atempt :/ ? | 09:33 |
| marios | https://review.rdoproject.org/zuul/stream/fc075c0f184349fab53e88972ab0cf6b?logfile=console.log | 09:33 |
| marios | 2nd attempt | 09:34 |
| Tengu|rover | yeah. | 09:34 |
| Tengu|rover | bet autohold doesn't kick in before last failed attempt. | 09:34 |
| Tengu|rover | marios: though it now seems to work as expected... it's deploying the UC. | 09:43 |
| Tengu|rover | marios: lemme abandon + restore the job. | 09:43 |
| Tengu|rover | afaik the autohold should stick | 09:43 |
| Tengu|rover | or I can re-recreate it. | 09:43 |
| Tengu|rover | uho. I can't abandon it... probably missing some rights on the repository. | 09:44 |
| Tengu|rover | marios: I've pushed a stupid commit edition so that it will re-kick and, hopefully, fail. | 09:46 |
| Tengu|rover | marios: o_O issue seems solved ?! | 10:16 |
| marios | :/ | 10:18 |
| marios | \o/ | 10:18 |
| marios | pub? | 10:18 |
| Tengu|rover | I mean, current run is deploying the OC | 10:18 |
| marios | Tengu|rover: yeah that's what i saw earlier 11:33 < marios> Tengu|rover: looks like it is running on 3rd atempt :/ ? | 10:18 |
| Tengu|rover | first attempt | 10:19 |
| Tengu|rover | (I re-kicked with a dumb edit) | 10:19 |
| marios | stable4 is running now Tengu|rover lets watch it | 10:19 |
| marios | https://review.rdoproject.org/zuul/status openstack-periodic-integration-stable4 | 10:19 |
| marios | glance has only 1 | 10:19 |
| marios | but *what* was it? like what on earth | 10:19 |
| marios | someone changing credentials under us? | 10:20 |
| Tengu|rover | no idea. | 10:21 |
| Tengu|rover | ok, and the 2 tasks don't actually show anything like "primary | ok" apparently. | 10:22 |
| Tengu|rover | even in a working run | 10:22 |
| Tengu|rover | there are soooo many things moving together in the current CI... | 10:23 |
| Tengu|rover | to be fair, I do hope we'll be able to reduce drastically the amount of inter-connected repositories in the future. | 10:23 |
| Tengu|rover | marios: guess I can remove the auto-hold? | 10:25 |
| ysandeep | marios, hey o/ I have replied to your comment here: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/871223/1#message-364c20c50608602b279e8761e7b25f202d9c64bd | 10:26 |
| ysandeep | please check when time permits | 10:26 |
| Tengu|rover | yay - successfully removed my autohold :] | 10:27 |
| * Tengu|rover is getting to places | 10:27 | |
| marios | Tengu|rover: yes its gone \o/ | 10:28 |
| marios | ysandeep: checking thanks | 10:28 |
| Tengu|rover | marios: I have a mighty "zc_container" alias, that allows me to pass any random command. and I've built 2 aliases onto it in order to list or create authold. | 10:29 |
| Tengu|rover | marios: sooo.... what about the CIX itself? close LP as "auto-solved, next"? | 10:41 |
| marios | Tengu|rover: lets leave it until the cix call but yeah you could update with some comment like 'looks like it is resolved - still unkown root cause' | 10:43 |
| marios | Tengu|rover: probably we'll mark the LP as "invalid" | 10:44 |
| Tengu|rover | marios: if we can't reproduce it, root-cause will stay "unknown" I think :/ | 10:44 |
| marios | yup likely | 10:44 |
| Tengu|rover | dang.... today I get a cheese fondue. It's already smelling in all the apartment XD | 10:54 |
| Tengu|rover | #food. | 11:03 |
| *** rlandy|out is now known as rlandy | 11:12 | |
| rlandy | marios: Tengu|rover: bhagyashris|ruck: hi - checking rr hackmd ... | 11:14 |
| rlandy | looks like login was resolved? | 11:14 |
| marios | rlandy: o/ on reviews time | 11:15 |
| reviewbot | Do you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks. | 11:15 |
| marios | rlandy: yes looking that way | 11:15 |
| *** dviroel|out is now known as dviroel | 11:18 | |
| rlandy | Tengu|rover: hi - pls ping when around - want to show you how to rekick a line | 11:42 |
| rlandy | we need to rekick master | 11:42 |
| * pojadhav tea break | 11:42 | |
| *** jpena|off is now known as jpena | 11:55 | |
| Tengu|rover | rlandy: back :) | 12:17 |
| Tengu|rover | sorry, the cheese thing is killing me | 12:17 |
| rlandy | Tengu|rover: sec - on call | 12:18 |
| Tengu|rover | sure | 12:18 |
| rlandy | Tengu|rover, hey - | 12:27 |
| rlandy | going to send you example | 12:27 |
| Tengu|rover | rlandy: 'k | 12:27 |
| Tengu|rover | is it via testproject again? | 12:27 |
| rlandy | no - like marios showed you with hold | 12:28 |
| Tengu|rover | oh, ok | 12:28 |
| rlandy | Tengu|rover: 5 mins - will explain | 12:29 |
| rlandy | https://meet.google.com/gsh-btmb-bff?pli=1&authuser=0 | 12:29 |
| rlandy | bhagyashris|ruck: need 5 - just need to chat with Tengu|rover for a few | 12:30 |
| bhagyashris|ruck | rlandy, ack | 12:30 |
| rlandy | bhagyashris|ruck: joining | 12:40 |
| Tengu|rover | oook. so rekicked the upstream master promotion. | 12:41 |
| Tengu|rover | let's see. | 12:41 |
| * soniya29 will be back in hr | 12:44 | |
| *** soniya29 is now known as soniya29|afk | 12:44 | |
| rlandy | Tengu|rover: I'll have time in 40 mins appr - will ping you then | 12:50 |
| Tengu|rover | rlandy: sounds good, thanks! | 12:51 |
| rlandy | pojadhav: frenzy_friday: the TC/UA sync clashes with the prodified meeting | 12:51 |
| rlandy | pojadhav: frenzy_friday: can we move it to the same time on wed? | 12:51 |
| pojadhav | rlandy, yes we can | 12:53 |
| pojadhav | will do that | 12:53 |
| frenzy_friday | pojadhav, Could you pls fwd me the invite? I think I dont have it | 12:53 |
| pojadhav | frenzy_friday, oops.. yes I will send :) | 12:54 |
| frenzy_friday | thanks :) | 12:54 |
| Tengu|rover | hmm, marios or rlandy or bhagyashris|ruck care to give a second read on the upstream promotion date I pushed in the rr hackmd? | 12:54 |
| Tengu|rover | I used the dashboard top-left square and subtracted the amount of days from today's date: http://dashboard-ci.tripleo.org/d/mhV51gdVk/upstream-and-rdo-promotions?orgId=1 | 12:54 |
| rlandy | Tengu|rover: you can check dlrn | 12:54 |
| Tengu|rover | is this a correct process, or is there a better source? | 12:54 |
| rlandy | it's easiest | 12:54 |
| rlandy | see the date on current-tripleo | 12:55 |
| pojadhav | frenzy_friday, have you got invite ? | 12:55 |
| rlandy | https://trunk.rdoproject.org/centos9-master/ example | 12:55 |
| Tengu|rover | ah, in the repo directly then | 12:55 |
| rlandy | ack | 12:55 |
| Tengu|rover | yeah, ok | 12:55 |
| frenzy_friday | pojadhav, yep, thanks! | 12:55 |
| rlandy | dlrn is the source of truth | 12:55 |
| Tengu|rover | oh, so we got master promoted today then? | 12:56 |
| Tengu|rover | ah, no. | 12:56 |
| Tengu|rover | current-tripleo | 12:56 |
| Tengu|rover | ok, it matches what I injected in the doc. | 12:57 |
| Tengu|rover | we're "good". | 12:57 |
| Tengu|rover | and if the master rekick passes, that would be nice as well for today. | 12:57 |
| rlandy | current-tripleo/2023-01-20 07:18 | 13:00 |
| rlandy | today is 23 | 13:00 |
| rlandy | yep | 13:00 |
| marios | jgilaber: afuscoar: o/ sync call | 13:00 |
| jgilaber | coming | 13:00 |
| rlandy | weekend infra hitch | 13:00 |
| Tengu|rover | .... why did my brain replace an h by a b now... | 13:01 |
| *** amoralej is now known as amoralej|lunch | 13:23 | |
| *** ysandeep is now known as ysandeep|brb | 13:24 | |
| Tengu|rover | so far, so green, for the re-kick | 13:26 |
| rlandy | Tengu|rover: want to meet now re: nodeset on train? | 13:32 |
| Tengu|rover | rlandy: ok - is it the "alt criteria", or yet another thing? | 13:33 |
| * Tengu|rover lost with all the names | 13:33 | |
| rlandy | ack | 13:33 |
| rlandy | Tengu|rover: https://meet.google.com/aog-hspf-qnu?pli=1&authuser=0 | 13:33 |
| pojadhav | rlandy, scrum time | 14:01 |
| pojadhav | bhagyashris|ruck, ^^ | 14:01 |
| *** dasm}off is now known as dasm | 14:02 | |
| dasm | o/ | 14:02 |
| *** ysandeep|brb is now known as ysandeep | 14:06 | |
| *** amoralej|lunch is now known as amoralej | 14:10 | |
| * pojadhav cooking..afk | 14:30 | |
| Tengu|rover | rlandy: hm, the testproject doesn't start for now... line is blocker with all of the integration lines running I guess. | 14:36 |
| rlandy | Tengu|rover' hmmm ... tetsproject failed | 15:28 |
| rlandy | let me try define the nodeste | 15:29 |
| rlandy | let's see how that goes | 15:29 |
| Tengu|rover | hmmm I see node_failure - i.e. no node was allocated? | 15:30 |
| Tengu|rover | so basically... no node were allocated - is it because it tried to use some infra that actually don't provide any node? | 15:30 |
| rlandy | guessing it tried the psi node | 15:31 |
| Tengu|rover | hmm. no way to know it, of course. | 15:34 |
| *** dviroel is now known as dviroel|lunch | 15:53 | |
| *** ysandeep is now known as ysandeep|out | 15:54 | |
| rlandy | Tengu|rover: periodic-tripleo-ci-centos-9-scenario010-ovn-provider-standalone-zed need a rekick | 16:03 |
| rlandy | then maybe we can promote zed as well | 16:03 |
| rlandy | can try rekick wallaby afterwards | 16:03 |
| Tengu|rover | lemme try to submit a testproject that will make sense for Zed. | 16:04 |
| Tengu|rover | hmmm.... did I do anything wrong with https://review.rdoproject.org/r/c/testproject/+/46694 ? | 16:09 |
| Tengu|rover | oh. | 16:12 |
| Tengu|rover | ... | 16:12 |
| Tengu|rover | dumb me. | 16:12 |
| Tengu|rover | yeah, now it's better. | 16:15 |
| Tengu|rover | rlandy: so for wallaby (periodic-tripleo-ci-centos-9-scenario010-ovn-provider-standalone-wallaby) we re-kick once Zed is over? | 16:22 |
| Tengu|rover | or may I submit the rekick now already.. | 16:22 |
| rlandy | also pending | 16:23 |
| Tengu|rover | pending - the kvm test, right? | 16:23 |
| frenzy_friday | reviewbot, pls add to review list https://code.engineering.redhat.com/gerrit/c/openstack/rrcockpit/+/438919 | 16:26 |
| reviewbot | I have added your review to the Review list | 16:27 |
| * marios off in few | 16:32 | |
| Tengu|rover | oh, I think Master just promoted? | 16:33 |
| rlandy | all jobs passed | 16:34 |
| Tengu|rover | \o/ | 16:34 |
| rlandy | it will promote | 16:34 |
| Tengu|rover | woohoo | 16:34 |
| Tengu|rover | ah, "last build: 31 minutes ago" yeah | 16:34 |
| Tengu|rover | so we should bet a new promotion shortly | 16:35 |
| Tengu|rover | rlandy: testproject for the virt thing is hanging again... | 16:36 |
| rlandy | maybe there is an issue with that node? | 16:38 |
| rlandy | Tengu|rover: it's not a disaster | 16:38 |
| Tengu|rover | heh, well, we get the alt one, but still | 16:39 |
| Tengu|rover | would be nice getting it green, right? | 16:39 |
| rlandy | we have been relying in donwnstream forever | 16:39 |
| rlandy | and that always works | 16:39 |
| Tengu|rover | «until it doesn't» | 16:39 |
| * Tengu|rover jinxing things around | 16:39 | |
| rlandy | I would guess that the virt node in vexx is not properly configure | 16:39 |
| rlandy | we can report it | 16:39 |
| rlandy | or carry on as we are | 16:40 |
| Tengu|rover | maybe we should drop the upstream job then? | 16:40 |
| Tengu|rover | that would avoid deploying a known to fail thing upstream | 16:40 |
| rlandy | Tengu|rover: probably the most honest | 16:40 |
| Tengu|rover | hence free resources | 16:40 |
| *** amoralej is now known as amoralej|off | 16:41 | |
| rlandy | first let's check that the label actually exists | 16:41 |
| rlandy | and if so, we can on #rhos-ops if they can see any obvious error | 16:41 |
| rlandy | if nothing easy to fox, I'd ditch it | 16:42 |
| rlandy | but probably worth some minimal effort | 16:42 |
| Tengu|rover | 'k | 16:42 |
| *** marios is now known as marios|out | 16:42 | |
| Tengu|rover | when you say "check if label exists", it's first ensuring the nodeset is properly defined, and then check on vexx if we have resources matching that label/whatever? | 16:43 |
| rlandy | correct | 16:45 |
| Tengu|rover | sooo label: cloud-centos-9-stream-tripleo-vexxhost seems to be the one you'll look for in vexxhost infra | 16:46 |
| rlandy | single-centos-9-node-nested-virt maps to cloud-centos-9-stream-tripleo-vexxhost | 16:47 |
| Tengu|rover | yep, and so does at least 2 others. | 16:47 |
| rlandy | and single-centos-8-node-nested-virt maps to upstream-centos-8-stream-psi-public which is wrong | 16:47 |
| Tengu|rover | cloud-centos-9-stream-vexxhost + centos-9-stream | 16:47 |
| Tengu|rover | oh, wait. | 16:47 |
| Tengu|rover | err. wait. how can you see that? | 16:48 |
| Tengu|rover | in the nodeset.yaml from rdo-jobs? | 16:48 |
| rlandy | reading https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/nodesets.yaml | 16:49 |
| Tengu|rover | oh. centos-8 | 16:49 |
| Tengu|rover | yeah | 16:49 |
| Tengu|rover | I'm on it now. we're talking train | 16:49 |
| rlandy | so there is no cloud-centos-8-stream-vexxhost | 16:49 |
| * Tengu|rover needs to find his brain cells again | 16:49 | |
| Tengu|rover | yeah | 16:49 |
| rlandy | only single-centos-8-node-vexxhost | 16:49 |
| rlandy | 'which I think I tried | 16:49 |
| Tengu|rover | it lead to a network issue during standalone deploy, yeah | 16:49 |
| rlandy | but to be sure .... let me try that one again | 16:49 |
| Tengu|rover | https://review.rdoproject.org/r/c/testproject/+/36254/211..212/.zuul.yaml | 16:50 |
| rlandy | single-centos-8-node-vexxhost | 16:50 |
| Tengu|rover | you tested upstream-centos-8-vexxhost | 16:50 |
| rlandy | trying that one | 16:50 |
| rlandy | you never know :) | 16:50 |
| Tengu|rover | heh | 16:50 |
| * rlandy guessing | 16:51 | |
| Tengu|rover | there's also an "rdo-centos-8-vexxhost" | 16:51 |
| *** dviroel|lunch is now known as dviroel | 16:51 | |
| arxcruz | kopecmartin: are tempestconf jobs broken? https://zuul.opendev.org/t/openstack/build/cae15491b1f7494e9d110ae752bdee1d | 16:52 |
| kopecmartin | arxcruz: yes | 16:52 |
| kopecmartin | these 2 patches will solve that: https://review.opendev.org/c/openstack/tempest/+/869440/4 | 16:52 |
| rlandy | Tengu|rover: we'll attempt that one next | 16:55 |
| Tengu|rover | rlandy: ok | 16:55 |
| Tengu|rover | it has a "status: recommended" commented out. | 16:55 |
| Tengu|rover | not sure what it means. | 16:55 |
| Tengu|rover | that same comment is also shown in some other definitions. | 16:55 |
| Tengu|rover | rlandy: at least it found an instance, it's starting the job :). | 16:56 |
| Tengu|rover | hmm. interesting. | 17:09 |
| rlandy | lunch - brb | 17:14 |
| Tengu|rover | OK Team - signing off. rlandy care to take over in case of fire? :) I'll be back© tomorrow. Hopefully fresh and ready for new crashes :) | 17:14 |
| *** jpena is now known as jpena|off | 17:37 | |
| rlandy | Tengu|rover: sure | 17:39 |
| rlandy | have a god night | 17:39 |
| rlandy | good | 17:39 |
| rlandy | Tengu|rover" finally ... periodic-tripleo-ci-centos-8-scenario010-kvm-standalone-traintestprojectmastercheck36254,213 | 18:18 |
| rlandy | 1 hr 4 mins 9 secs2023-01-23 16:51:09SUCCESS | 18:18 |
| rlandy | Tengu|rover: putting in reviews | 18:19 |
| reviewbot | Do you want me to add your patch to the Review list? Please type something like add to review list <your_patch> so that I can understand. Thanks. | 18:19 |
| * rlandy needs to pick up from store - brb | 18:47 | |
| *** rlandy is now known as rlandy|brb | 18:47 | |
| *** rlandy|brb is now known as rlandy | 19:19 | |
| *** dviroel is now known as dviroel|out | 21:25 | |
| * dasm => offline | 23:32 | |
| *** dasm is now known as dasm|off | 23:32 | |
| dasm|off | o/ | 23:32 |
| rlandy | bye | 23:43 |
| *** rlandy is now known as rlandy|out | 23:44 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!