| akahat|rover | o/ | 04:28 |
|---|---|---|
| *** pojadhav is now known as pojadhav|ruck | 04:33 | |
| ysandeep | good morning o/ happy new year everyone :D | 06:07 |
| pojadhav|ruck | \o happy new year all ;) | 06:09 |
| marios | o/ | 06:30 |
| ysandeep | hey marios happy new year o/ | 06:31 |
| ysandeep | hah.. looks like https://bugs.launchpad.net/tripleo/+bug/2000226/ is still haunting us | 06:32 |
| ysandeep | chandankumar, thanks for fixing repo: https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/135b569/logs/overcloud-controller-0/etc/yum.repos.d/quickstart-centos-appstreams.repo.txt.gz | 06:33 |
| ysandeep | looks like mirror finally synced and now we are getting affected kernel via vexx mirror as well. | 06:34 |
| ysandeep | pojadhav|ruck, chandankumar kernel exclude patch didn't help? | 06:36 |
| marios | \o | 06:38 |
| marios | happy new year ysandeep | 06:38 |
| marios | akahat|rover: pojadhav|ruck: i just sent an invite for a sync call please check inbox and let me know if i need to switch the time | 06:45 |
| akahat|rover | marios, o/ thanks updated! | 06:46 |
| marios | ysandeep: chandankumar: can we do anything about the storage on rhos-dev-stage i couldn't launch an instance yesterday because storage quota is full | 06:49 |
| pojadhav|ruck | ysandeep, i think nope.. chandankumar was working on this.. may be he is having more idea what is happening there exactly. | 06:49 |
| pojadhav|ruck | marios, ack accepted! | 06:49 |
| ysandeep | marios, could you please invite me in sync as well just for today, I want to discuss https://bugs.launchpad.net/tripleo/+bug/2000226/ | 06:50 |
| marios | ysandeep: yes thank you added you and chandankumar if available | 06:52 |
| ysandeep | marios, ack about rhos-dev-stage, will check soon | 06:52 |
| ysandeep | marios, thanks! | 06:52 |
| akahat|rover | ysandeep, i've run tp https://review.rdoproject.org/zuul/status#45162 only fs35-master is running.. and other jobs failed on same error. | 06:52 |
| akahat|rover | testing with older kernel: https://review.rdoproject.org/r/c/testproject/+/36719/23#message-ed0dd047abc17955cbfd8c5b1f84b3a1a3b73fd7 | 06:54 |
| ysandeep | akahat|rover, wait let me update the tq patch. | 06:58 |
| akahat|rover | ysandeep, are you trying to use regex? It won't work. | 07:01 |
| ysandeep | kernel*-5.14.0-214* will work | 07:01 |
| ysandeep | earlier chandankumar tried kernel-*-5.14.0-214* | 07:02 |
| ysandeep | extra - | 07:02 |
| * pojadhav|ruck afk for half hour | 07:06 | |
| akahat|rover | oh.. | 07:06 |
| ysandeep | akahat|rover, updated https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605 | 07:07 |
| ysandeep | looking at your testproject now | 07:07 |
| ysandeep | lets also include image build jobs and ovb job from periodic line | 07:08 |
| akahat|rover | ysandeep, just rechecked: https://review.rdoproject.org/r/c/testproject/+/36719/23#message-02f982b65086d176a473a163191abb1b351106a7 | 07:08 |
| ysandeep | akahat|rover, ack | 07:09 |
| ysandeep | let me propose a separate patch for image rebuild and ovb job from periodic line | 07:10 |
| akahat|rover | ysandeep, we need to check the nodepool image kernel also | 07:23 |
| ysandeep | akahat|rover, we don't use nodepool image for overcloud nodes | 07:24 |
| akahat|rover | ysandeep, oh.. ok.. | 07:25 |
| ysandeep | we use the uefi image we build in our image build jobs | 07:25 |
| akahat|rover | ysandeep, chandankumar tried with latest image: https://review.opendev.org/c/openstack/tripleo-ci/+/869004/1/roles/oooci-build-images/vars/centos-9.yaml | 07:25 |
| akahat|rover | ok.. thanks for clarification. | 07:25 |
| akahat|rover | https://review.rdoproject.org/r/c/testproject/+/46503 | 07:25 |
| ysandeep | akahat|rover, ^^ didn't work, right? | 07:27 |
| akahat|rover | ysandeep, 1 job passed in tp | 07:27 |
| akahat|rover | 2 failed | 07:28 |
| ysandeep | 2 failed with same error? | 07:28 |
| akahat|rover | yes | 07:28 |
| ysandeep | okay, than didn't help | 07:28 |
| ysandeep | issue is transient | 07:28 |
| * akahat|rover lunch | 07:29 | |
| jbadiapa | gm all and happy new year! | 08:02 |
| jbadiapa | marios, have you got a minute? | 08:02 |
| jbadiapa | to talk about https://review.opendev.org/c/openstack/tripleo-heat-templates/+/867026 | 08:03 |
| jbadiapa | the last patch was a revert of https://github.com/openstack/tripleo-heat-templates/commit/c8fba9b2375d38dea0ca20a6a52e396d30b20472 as rabi suggested but it never worked | 08:04 |
| jbadiapa | IMHO, we should go back to the original fix, and do the modification later on, what do you think? | 08:05 |
| jbadiapa | the original one is the one in master https://review.opendev.org/c/openstack/tripleo-heat-templates/+/866381 | 08:05 |
| marios | jbadiapa: yes agree | 08:11 |
| marios | jbadiapa: as commented yesterday since we have a version that was tested and works to unblock lets merge that | 08:11 |
| marios | jbadiapa: if there is a fix/improvement it should follow next | 08:12 |
| jbadiapa | As long as the gate gives the outcome, so rabi can take a look at it, I'll update the patch with the original one | 08:23 |
| * akahat|rover back | 08:29 | |
| marios | joining | 08:30 |
| akahat|rover | marios, https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605 | 08:42 |
| akahat|rover | marios, https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-content-provider&project=openstack/tripleo-ansible | 09:02 |
| *** ysandeep is now known as ysandeep|lunch | 09:10 | |
| akahat|rover | ysandeep|lunch, recent tp run with exclude kernel, job failed, kernel-5.14.0.210 got installed on overcloud nodes | 09:30 |
| akahat|rover | but undercloud is still using kernel-5.14.0.210 | 09:30 |
| akahat|rover | https://logserver.rdoproject.org/19/36719/23/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset001-baremetal-master/edd50fa/logs/undercloud/var/log/extra/package-list-installed.txt.gz | 09:30 |
| akahat|rover | Again on overcloud controller it is getting installed from quickstart-centos-base and in undercloud it is using appstream and anaconda | 09:32 |
| *** pojadhav|ruck is now known as pojadhav|lunch | 09:37 | |
| *** pojadhav|lunch is now known as pojadhav|ruck | 10:22 | |
| Tengu | hello there! happy new year to everyone that follows "that" calendar :). Quick question: can anyone link the the (in)famous dashboard showing the table with all the jobs? I thought I had it in my bookmarks, but apparently I either missed it, or am just unable to find it back.. | 10:42 |
| Tengu | there's a search field at the top, and it shows job status. (note: it's NOT https://zuul.opendev.org/t/openstack/status ) | 10:42 |
| Tengu | ah! got it. https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-standalone-external-compute-target-host | 10:45 |
| Tengu | 'k, so "builds" tab. | 10:45 |
| Tengu | and this is why I didn't bookmark it :] | 10:45 |
| *** rlandy|out is now known as rlandy | 10:53 | |
| rlandy | hello - happy new year everyone! | 10:54 |
| marios | \o hello happy new year again | 10:56 |
| rlandy | marios: akahat|rover: pojadhav|ruck: should we sync? | 10:58 |
| marios | rlandy: sure, we had a call earlier already but lets see if there are updates | 10:59 |
| rlandy | marios: yeah - quick update would be helpful | 11:00 |
| rlandy | to catch up | 11:00 |
| marios | lets wait few mins for akahat|rover and pojadhav|ruck | 11:01 |
| rlandy | akahat|rover: pojadhav|ruck: pls ping when you are around | 11:01 |
| rlandy | bhagyashris: hello - how is the osp-18 stand up going? | 11:02 |
| rlandy | bhagyashris: no movement since 12/19????? | 11:03 |
| rlandy | frenzy_friday: hello - are we done with the 9.1 work? | 11:07 |
| rlandy | there is one card open | 11:07 |
| rlandy | frenzy_friday: about the components - done, no? | 11:07 |
| frenzy_friday | rlandy, 0/ Yep, just saw the card, components are done. Moving the card to done | 11:08 |
| rlandy | thanks | 11:08 |
| akahat|rover | rlandy, o/ Happy New Year!! | 11:14 |
| pojadhav|ruck | rlandy, i am around | 11:15 |
| pojadhav|ruck | rlandy, akahat|rover : we can sync now | 11:16 |
| rlandy | marios: ^^ still around? | 11:22 |
| marios | yes rlandy | 11:23 |
| rlandy | ok - let's do this ... | 11:23 |
| rlandy | https://meet.google.com/gar-mtva-qsw?pli=1&authuser=0 | 11:24 |
| rlandy | pojadhav|ruck: akahat|rover: marios: ^^ | 11:24 |
| akahat|rover | marios, https://code.engineering.redhat.com/gerrit/c/testproject/+/428940/7/.zuul.yaml#8 | 11:26 |
| *** ysandeep|lunch is now known as ysandeep | 11:29 | |
| bhagyashris | rlandy, container build passed | 11:32 |
| bhagyashris | rlandy, last we i was on ruck/rover shift | 11:32 |
| pojadhav|ruck | rlandy, https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/pipeline/ | 11:32 |
| bhagyashris | Happy New Year Everyone ! | 11:34 |
| *** dviroel_ is now known as dviroel | 11:39 | |
| dviroel | o/ | 11:40 |
| ysandeep | akahat|rover, hi | 11:40 |
| ysandeep | the last green run, i got last week: https://review.rdoproject.org/r/c/testproject/+/28537/69#message-ab5d371a81952bbc0f65beffc2fd40afa1bd6cc2 | 11:41 |
| ysandeep | had kernel version -205 | 11:42 |
| ysandeep | let me check if we have better results with above version | 11:42 |
| ysandeep | marios, fyi.. ^^ adding 210 also in exclude along with 214 | 11:46 |
| pojadhav|ruck | rlandy, https://bugs.launchpad.net/tripleo/+bug/2000897 | 11:56 |
| marios | ysandeep: ah ok | 11:58 |
| marios | ysandeep: we saw the failed job/testproject was using 210 | 11:58 |
| marios | ysandeep: so that would be interesting | 11:59 |
| rlandy | bhagyashris: pls ping when you are around | 12:02 |
| marios | ysandeep: are you doing this or should we ask akahat|rover to do that | 12:02 |
| bhagyashris | rlandy, hey i am around... | 12:03 |
| ysandeep | marios, I just updated my patch, rerunning testproject | 12:03 |
| marios | ysandeep: k thanks | 12:03 |
| rlandy | bhagyashris: pls see question above | 12:03 |
| bhagyashris | <bhagyashris> rlandy, container build passed | 12:03 |
| bhagyashris | <bhagyashris> rlandy, last we i was on ruck/rover shift | 12:03 |
| bhagyashris | last week.. | 12:04 |
| bhagyashris | currently working on image build stuff | 12:04 |
| rlandy | bhagyashris: pls update the hashes | 12:05 |
| bhagyashris | ok will run promote promoted component to tripleo-ci-testing hash | 12:05 |
| rlandy | bhagyashris: and current-tripleo | 12:07 |
| bhagyashris | sure sure | 12:07 |
| rlandy | bhagyashris: how are the container builds? | 12:07 |
| bhagyashris | it GREEN | 12:07 |
| bhagyashris | it's | 12:08 |
| rlandy | bhagyashris: great ... the start putting patchs into the review list to get these lines in and building every day | 12:08 |
| bhagyashris | https://code.engineering.redhat.com/gerrit/c/testproject/+/436651/17#message-601b64d54122d7acf56087f51aa5a46f019c2172 | 12:08 |
| rlandy | bhagyashris: ie: you should have the promote component jobs in | 12:08 |
| bhagyashris | rlandy, ack ^ | 12:08 |
| bhagyashris | rlandy, yes | 12:08 |
| rlandy | bhagyashris: and the criteria with just the component job | 12:09 |
| rlandy | bhagyashris: so you have everything that works running | 12:09 |
| bhagyashris | rlandy, Putting all the ready for review stuff on review hackmd | 12:09 |
| bhagyashris | yes | 12:09 |
| * pojadhav|ruck needs a tea break... back in few mins | 12:16 | |
| akahat|rover | marios, ysandeep installing dlrn from source if we install it using pip will resolve this issue? https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/build-test-packages/tasks/main.yml#L49 | 12:19 |
| akahat|rover | we can add dlrn_pre_installed: true in release file | 12:19 |
| ysandeep | ideally yes i.e installation from source or a new release with the fix | 12:21 |
| marios | akahat|rover: but i think the other way around? we are installing from pip but from source should fix it | 12:21 |
| akahat|rover | marios, yeah.. sorry.. i shuffled words :( | 12:21 |
| akahat|rover | marios, let me try this. | 12:21 |
| marios | akahat|rover: worth a try as workaround until we get the new version | 12:21 |
| rlandy | anyone get a new laptop or install recently? having an issue getting codeeng gerrit to sign in | 12:31 |
| rlandy | marios: ysandeep: frenzy_friday: dviroel: ^^? | 12:31 |
| dviroel | rlandy: hum, no. last time that I had an issue: "kdestroy -A" and then "kinit" solved | 12:33 |
| rlandy | dviroel++ | 12:35 |
| rlandy | genius | 12:35 |
| rlandy | that worked | 12:35 |
| akahat|rover | https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/5/html/deployment_guide/sso-config-firefox | 12:35 |
| dviroel | yey | 12:35 |
| rlandy | akahat|rover: ack did that | 12:35 |
| rlandy | but I guess I needed to destroy the old ticket | 12:36 |
| rlandy | yay - I have my powers back | 12:36 |
| akahat|rover | marios, rlandy : https://review.opendev.org/c/openstack/tripleo-quickstart/+/869079 | 12:37 |
| marios | ack and we have content provider on there akahat|rover | 12:37 |
| marios | https://zuul.openstack.org/status#869079 lets see | 12:38 |
| marios | rlandy: glad you got it resolved | 12:39 |
| rlandy | need to reboot | 12:39 |
| rlandy | back in a few | 12:39 |
| rlandy | akahat|rover: reading back ... https://review.opendev.org/c/openstack/tripleo-quickstart/+/869079 | 12:52 |
| rlandy | is that tested somewhere? | 12:52 |
| akahat|rover | rlandy, it will get tested in patch.. | 12:54 |
| akahat|rover | https://zuul.opendev.org/t/openstack/status#869079 | 12:54 |
| pojadhav|ruck | rlandy, marios, akahat|rover : rhos17.1 on rhel9.1 promoted... \o/ | 12:54 |
| akahat|rover | content provider job are running | 12:54 |
| akahat|rover | \o/ | 12:54 |
| rlandy | great | 12:55 |
| *** ysandeep is now known as ysandeep|brb | 12:55 | |
| rlandy | akahat|rover: ok - so we need to merge this to fix gate blocker? | 12:56 |
| rlandy | marios: ^^? | 12:56 |
| rlandy | if so, pls vote before you leave | 12:56 |
| rlandy | so I can merge | 12:56 |
| akahat|rover | rlandy, yes.. waiting for job results.. | 12:56 |
| rlandy | nice | 12:56 |
| marios | rlandy: not yet though we are waiting for 14:54 < akahat|rover> https://zuul.opendev.org/t/openstack/status#869079 | 12:58 |
| rlandy | yep | 12:58 |
| rlandy | bhagyashris: sorry - had to reboot - thanks - will review all those once they land on review list | 12:59 |
| * akahat|rover brb | 12:59 | |
| bhagyashris | ack | 12:59 |
| rlandy | looks like nothing on the review list except dasm|off patches | 12:59 |
| rlandy | dviroel: ^^ we can skip review time today unless dasm|off has some patches ready | 12:59 |
| rlandy | pojadhav|ruck: marios: akahat|rover: ysandeep|brb: dviroel: frenzy_friday: bhagyashris: Tengu: arxcruz: jm1: anything for the community call today? | 13:02 |
| rlandy | pojadhav|ruck: marios: akahat|rover: ysandeep|brb: dviroel: frenzy_friday: bhagyashris: Tengu: arxcruz: jm1: if not, we can talk about 2023 planning | 13:03 |
| rlandy | high level | 13:03 |
| rlandy | ahead of our sprint planning meeting on thursday | 13:03 |
| arxcruz | rlandy: nothing from my side | 13:03 |
| Tengu | nothing yet I think | 13:03 |
| frenzy_friday | nothing from me as well | 13:04 |
| bhagyashris | nothing me from my side | 13:05 |
| dviroel | start with 2023 planning sounds good | 13:06 |
| marios | rlandy: just wanted to ask about resources for rhos-dev-stage cant get a vm with nested virt going we are out of storage | 13:07 |
| rlandy | marios: can check | 13:09 |
| rlandy | marios: ack volume and RAM storage both seem low | 13:11 |
| marios | thanks also dviroel chandankumar anything on rhos-dev-stage we can remove (already asked ysandeep|brb earlier) we need some storage back if possible | 13:11 |
| rlandy | let's see what we can clean up | 13:11 |
| marios | rlandy: yeah but storage was the blocker | 13:11 |
| marios | rlandy: i could just about squeeze one vm on the ram | 13:11 |
| rlandy | frenzy_friday: akahat|rover: bhagyashris: marios: there are a bunch of volumes there that say promoter | 13:12 |
| rlandy | two of which are not connected | 13:12 |
| rlandy | can we remove them?? | 13:12 |
| rlandy | dasm|off may be working on that as well? | 13:13 |
| rlandy | marios: ^^ let's ask at community call what can be deleted here | 13:13 |
| marios | rlandy: thanks | 13:13 |
| rlandy | marios: also rhos-01 is available | 13:13 |
| akahat|rover | rlandy, yes.. if they are not connected please remove it. | 13:14 |
| marios | rlandy: does it have nestd virt available (we have to request access if not) | 13:14 |
| marios | rlandy: i.e. the some of the flavors should have 'nested.virt' in their name usually | 13:14 |
| marios | rlandy: we don't have it in rhos-dev-pcci for example | 13:14 |
| bhagyashris | looking | 13:14 |
| pojadhav|ruck | folks, please add agenda if any https://hackmd.io/iraYQWGBT4qPCKH0VNG31A#2023-01-03-Community-Call | 13:15 |
| rlandy | marios: as part of planning, let's talk about the plan for rhos-01 | 13:16 |
| rlandy | then we will know what to request | 13:17 |
| marios | rlandy: ok | 13:21 |
| *** ysandeep|brb is now known as ysandeep | 13:37 | |
| rlandy | akahat|rover: ysandeep: hmmm ... https://review.rdoproject.org/zuul/stream/30215e77372747e6b4c1693178fcf198?logfile=console.log | 13:42 |
| ysandeep | rlandy, older kernel also didn't help.. | 13:44 |
| rlandy | ysandeep: akahat|rover: have we tried downgrading the other rpms suggested on the chat> | 13:45 |
| akahat|rover | rlandy, i've tried with devicemapper | 13:45 |
| rlandy | devicemapper, libblockdev or lvm2 are all potential suspects - per harold | 13:46 |
| rlandy | all of them together | 13:46 |
| rlandy | forcing all to the versions on the successful run | 13:46 |
| ysandeep | rlandy, I tried last week didn't help: https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605/3/config/release/tripleo-ci/CentOS-9/master.yml | 13:46 |
| * akahat|rover heading to home.. will back before community call. | 13:47 | |
| ysandeep | kernel seems to helped that time https://review.rdoproject.org/r/c/testproject/+/28537/69#message-ab5d371a81952bbc0f65beffc2fd40afa1bd6cc2 but looks like that was just a lucky pass as this is a transient issue. | 13:47 |
| rlandy | ysandeep: what about your comment on reinstall? | 13:49 |
| ysandeep | rlandy, reinstall passed at that time but it was just hiding the actual issue. | 13:51 |
| rlandy | ysandeep: try anything else than the downgrades, large node? | 13:52 |
| ysandeep | rlandy, I think akahat|rover mentioned they already tried larger node. | 13:53 |
| rlandy | yep | 13:53 |
| rlandy | I was asking if we went down any other roads | 13:53 |
| ysandeep | rlandy, let me try excluding all the rpm from diff, just to be 100% sure it is not because of any rpm. | 13:55 |
| ysandeep | but devicemapper, libblockdev or lvm2, kernel are already tried | 13:55 |
| rlandy | ysandeep: or it's something else entirely | 13:57 |
| rlandy | ysandeep: the revert didn;t work | 13:57 |
| rlandy | I think chandankumar mentioned some other DIM patch that may help | 13:57 |
| ysandeep | yes could be, maybe we should think how to reproduce this outside of tripleo ci jobs | 13:58 |
| ysandeep | I wonder if we take overcloud-full-uefi and boot it directly, do we hit this issue? | 13:59 |
| ysandeep | rlandy, those dib patches revert didn't help | 13:59 |
| * rlandy reads bug details again | 14:05 | |
| *** dasm|off is now known as dasm | 14:06 | |
| dasm | o/ | 14:06 |
| *** ysandeep is now known as ysandeep|brb | 14:22 | |
| dasm | rlandy: i removed 'promoter_storage', 'promoter-storage' and 'promoter-2-storage' volumes. It released about 250G. There are 2 other: "test-promoter" and "downstream-promoter" which seems to be valid. | 14:26 |
| rlandy | dasm: we need to check with the test-promoter owners if those need to stay | 14:28 |
| dasm | ack | 14:29 |
| dasm | rlandy: it's attached to 'promoter_server_all_releases' | 14:29 |
| rlandy | marios: pojadhav|ruck: akahat|rover: program call is canceled tomorrow | 14:31 |
| rlandy | so you don;t have to update any details there | 14:31 |
| marios | rlandy: thanks | 14:32 |
| pojadhav|ruck | rlandy, thanks | 14:32 |
| rlandy | ysandeep|brb: dviroel: nobody had any topic at podified meeting | 14:32 |
| rlandy | you didn't miss anything | 14:32 |
| dviroel | rlandy: no, took 5 min | 14:33 |
| rlandy | marios: ok - you should be freed up to create a vm on stage | 14:39 |
| rlandy | bhagyashris: two comments on https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/436562 | 14:44 |
| rlandy | bhagyashris: w+'d the other review | 14:44 |
| *** ysandeep|brb is now known as ysandeep | 14:47 | |
| rlandy | ysandeep: can you respond to harold on the gchat to point to the patch where you tested downgrading all the rpms he mentions? | 14:52 |
| ysandeep | rlandy, sure, I just updated https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605/13/config/release/tripleo-ci/CentOS-9/master.yml to have fresh results, will respond to him as soon as ci reports... | 14:56 |
| pojadhav|ruck | cimmunity call in 4 mins : arxcruz, rlandy, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat|rover , chandankumar, frenzy_friday, anbanerj, dviroel, dasm | 14:56 |
| rlandy | thanks | 14:56 |
| bhagyashris | ack | 14:57 |
| pojadhav|ruck | https://meet.google.com/igc-nxwj-gws?authuser=0 | 14:57 |
| pojadhav|ruck | https://hackmd.io/iraYQWGBT4qPCKH0VNG31A#2023-01-03-Community-Call | 14:58 |
| pojadhav|ruck | rlandy, community call | 15:00 |
| pojadhav|ruck | akahat|rover, community call | 15:02 |
| *** dviroel is now known as dviroel|lunch | 15:32 | |
| pojadhav|ruck | rlandy, anything to look into at d/stream ?? | 15:36 |
| rlandy | pojadhav|ruck: on 1-1 - pls check if anything else requires attention | 15:36 |
| pojadhav|ruck | rlandy, sure | 15:37 |
| * pojadhav|ruck out | 15:51 | |
| rlandy | akahat|rover: marios: ysandeep: can we w+ https://review.opendev.org/c/openstack/tripleo-quickstart/+/869079? | 16:02 |
| akahat|rover | rlandy, cs9 content provider ran in ci: | 16:03 |
| marios | rlandy: looking | 16:05 |
| rlandy | akahat|rover: I voted | 16:05 |
| rlandy | want another core vote there | 16:05 |
| marios | great nice one akahat|rover o/ | 16:06 |
| rlandy | akahat|rover: marios: pojadhav|ruck: ok - so just one left??? | 16:07 |
| rlandy | the big node provisioning problem | 16:07 |
| akahat|rover | ok. +1 | 16:07 |
| rlandy | akahat|rover: marios: k - so anything left to try there? | 16:08 |
| akahat|rover | rlandy, marios Thank you :) | 16:08 |
| akahat|rover | yes.. still waiting for resluts https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605 | 16:08 |
| rlandy | hmmm ... I thought ysandeep tried that | 16:10 |
| rlandy | dviroel|lunch: how long since we tried out using an older centos-9 compose? | 16:10 |
| rlandy | akahat|rover: ^^ | 16:10 |
| rlandy | maybe we can try a whole older compose | 16:10 |
| akahat|rover | rlandy, trying with older image ? | 16:11 |
| rlandy | akahat|rover: what was the story here: https://review.opendev.org/c/openstack/diskimage-builder/+/827558? | 16:13 |
| rlandy | OVB jobs still failing in tp[1]. | 16:13 |
| rlandy | oh | 16:13 |
| rlandy | akahat|rover: was that tested with an image rebuild? | 16:14 |
| rlandy | akahat|rover: just interested if that patch was fully tested | 16:15 |
| akahat|rover | rlandy, no. | 16:15 |
| akahat|rover | didn't tested. | 16:16 |
| rlandy | akahat|rover: but you -1'ed it | 16:16 |
| rlandy | and the comment shows one test | 16:16 |
| rlandy | https://review.rdoproject.org/zuul/buildset/a86dfa137ae64927b2a8b9679ea57bcd | 16:17 |
| akahat|rover | yeah i test results was failing.. so i -1 it. | 16:17 |
| rlandy | akahat|rover: can we chat for 5 | 16:17 |
| akahat|rover | but later that i didn't tested it. | 16:17 |
| akahat|rover | rlandy, sure | 16:18 |
| rlandy | https://meet.google.com/rmm-aaeb-tec?pli=1&authuser=0 | 16:18 |
| rlandy | akahat|rover: ^^ | 16:18 |
| ysandeep | mind if I join as well | 16:19 |
| ysandeep | rlandy, akahat|rover ^^ | 16:24 |
| rlandy | ysandeep: sorry - just asked akahat|rover to look at https://review.opendev.org/c/openstack/diskimage-builder/+/827558 | 16:24 |
| rlandy | chandankumar left a comment that it may work | 16:24 |
| rlandy | and it was only tested w/o rebuilding the images | 16:24 |
| rlandy | ysandeep: the other thing I would try is an entire older compose | 16:25 |
| rlandy | if we still can | 16:25 |
| rlandy | idk if we still have that option | 16:25 |
| rlandy | dviroel|lunch would know | 16:25 |
| ysandeep | rlandy, actually I spoke with dviroel|lunch about that option last week but than I got false positive with older kernel and got side tracked. | 16:26 |
| rlandy | k | 16:26 |
| ysandeep | rlandy, latest run passed node proivisioning but need more run to confirm: https://review.rdoproject.org/zuul/stream/2f24f08eeb764a359f6fd1378de45254?logfile=console.log | 16:27 |
| rlandy | ysandeep: yeah 0 so I just wanted akahat|rover to follow those two debug options | 16:27 |
| ysandeep | k.. what was story about -1 on https://review.opendev.org/c/openstack/diskimage-builder/+/827558/6#message-6f174eb6e2aedb3d58e7bd323e89d84ad2eed81f | 16:28 |
| ysandeep | I thought they tested that already | 16:28 |
| rlandy | not with an image rebuild | 16:29 |
| * marios off in a few o/ have a good rest of day folks | 16:29 | |
| marios | rlandy: need somthing before i go? | 16:29 |
| rlandy | dasm: do you have any patches to add to the review list before the meeting? | 16:29 |
| rlandy | otherwise already reviewed what is on today's list | 16:29 |
| dasm | rlandy: no. | 16:30 |
| rlandy | marios: no - just pls to follow up with steve if we are still stuck on the provsioning thing tomorrow | 16:30 |
| dasm | rlandy: catching up with messages backlog from last 2 weeks. | 16:30 |
| rlandy | dasm: k - no problem - just wanted to see if there were any infra patches | 16:30 |
| dasm | not today | 16:30 |
| rlandy | but we can take a look at those later in the week | 16:30 |
| ysandeep | rlandy, akahat|rover but the logs in that comment belong to component jobs.. we build images in component job: https://review.rdoproject.org/r/c/testproject/+/46114/8#message-b42c0e2facf8b8da1c2cc2329d622295ef3d8b75 | 16:30 |
| dasm | thanks for checking with me rlandy | 16:30 |
| rlandy | ysandeep: akahat|rover: ack - that is right | 16:31 |
| rlandy | so I guess that's a no show | 16:31 |
| marios | rlandy: k | 16:31 |
| ysandeep | rlandy, yeah :( | 16:32 |
| *** marios is now known as marios|out | 16:32 | |
| ysandeep | I have included some more packages in excude in my last run, lets see if that helps.. | 16:33 |
| ysandeep | waiting here for results: https://review.rdoproject.org/zuul/stream/f67d99cf95684b099f92a98683177707?logfile=console.log | 16:33 |
| *** ysandeep is now known as ysandeep|brb | 16:34 | |
| akahat|rover | ysandeep|brb, rlandy running with zed https://review.rdoproject.org/r/c/testproject/+/46524 | 16:35 |
| akahat|rover | https://review.rdoproject.org/r/c/testproject/+/46524 | 16:35 |
| *** ysandeep|brb is now known as ysandeep | 16:42 | |
| * akahat|rover dinner | 16:44 | |
| *** dviroel|lunch is now known as dviroel | 16:45 | |
| ysandeep | akahat|rover, rlandy second job passed node provision too: https://review.rdoproject.org/zuul/stream/f67d99cf95684b099f92a98683177707?logfile=console.log , let me run more jobs to confirm | 16:50 |
| ysandeep | triggered more ovb component jobs here: https://review.rdoproject.org/r/c/testproject/+/42657 | 16:56 |
| rlandy | ysandeep: sorry - was on meeting | 17:02 |
| ysandeep | rlandy, nw, I triggered some jobs to be sure of results. | 17:02 |
| rlandy | ysandeep: ok - that's with all the rolebacks | 17:02 |
| rlandy | for master component | 17:02 |
| rlandy | then the fun part will be to see how many we need | 17:03 |
| ysandeep | rlandy, yes with more packages in exclude.. (doing gradual increase so that we find out what fixed the issue.) | 17:04 |
| rlandy | k | 17:04 |
| rlandy | dviroel: dasm: going to skip review time - only two patches there - both reviewed | 17:05 |
| dasm | rlandy: ack, thx | 17:06 |
| *** ysandeep is now known as ysandeep|out | 17:19 | |
| akahat|rover | ysandeep|out, thank you! that's great! | 17:44 |
| ysandeep|out | akahat|rover, 5/5 job passed node provision with https://review.opendev.org/c/openstack/tripleo-quickstart/+/868605/13/config/release/tripleo-ci/CentOS-9/master.yml | 18:14 |
| akahat|rover | ysandeep|out, that's great.. | 18:14 |
| akahat|rover | ysandeep|out, also checking out which packages caused error: https://review.opendev.org/c/openstack/tripleo-quickstart/+/869099 | 18:15 |
| ysandeep|out | lets give more recheck to confirm this is working set of rpms.. | 18:15 |
| ysandeep|out | akahat|rover, yeah need to play with diff to confirm the cause | 18:15 |
| * ysandeep|out out, see you tomorrow | 18:15 | |
| akahat|rover | yup. | 18:16 |
| akahat|rover | ysandeep|out, good night. | 18:16 |
| akahat|rover | rlandy, https://review.rdoproject.org/r/c/testproject/+/46525 tests with exclude packages. | 18:23 |
| * akahat|rover out, see you tomorrow | 18:24 | |
| dviroel | rlandy: wrt new flavor, we are ok with CPU and Disk, we just need to increase memory | 20:17 |
| dviroel | NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% | 20:17 |
| dviroel | oooci-04-master-0 5493m 35% 20155Mi 64% | 20:17 |
| dviroel | Filesystem Size Used Avail Use% Mounted on | 20:17 |
| dviroel | /dev/vda4 160G 24G 137G 15% / | 20:18 |
| dviroel | rlandy: "new flavor: c1.memory.large --disk 120 --vcpus 8 --ram 32768" | 20:19 |
| rlandy | dviroel: do you want a whole new flavor or a change to the one you have? | 20:31 |
| * rlandy gets ticket | 20:32 | |
| rlandy | dviroel: https://support.vexxhost.com/hc/en-us/requests/363274 | 20:39 |
| rlandy | akahat|rover: ha - sandeep's downgrade works | 20:47 |
| dviroel | rlandy: both works | 20:49 |
| * dviroel out | 21:17 | |
| *** dviroel is now known as dviroe|out | 21:17 | |
| dasm | dviroe|out: o/ | 21:19 |
| * dasm => offline | 21:33 | |
| dasm | o/ | 21:33 |
| rlandy | have a good night | 21:40 |
| *** rlandy is now known as rlandy|out | 23:38 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!