| *** ysandeep|out is now known as ysandeep | 04:25 | |
| *** soniya29 is now known as soniya29|ruck | 05:50 | |
| chandankumar | dpawlik: hello | 06:29 |
|---|---|---|
| chandankumar | dpawlik: can I update this testproject https://review.rdoproject.org/r/c/testproject/+/44388 | 06:29 |
| chandankumar | with correct config | 06:29 |
| chandankumar | ? | 06:29 |
| dpawlik | sure chandankumar | 06:40 |
| chandankumar | dpawlik: updated, thanks! | 06:52 |
| *** ysandeep is now known as ysandeep|afk | 07:00 | |
| *** jpena|off is now known as jpena | 07:34 | |
| *** soniya29|ruck is now known as soniya29|ruck|lunch | 08:09 | |
| *** soniya29|ruck|lunch is now known as soniya29|ruck | 09:37 | |
| ysandeep|afk | rlandy|out, retro clashes with today's operator mtg.. Should we cancel operator mtg? | 10:15 |
| *** ysandeep|afk is now known as ysandeep|break | 10:16 | |
| *** rlandy|out is now known as rlandy | 10:37 | |
| rlandy | ysandeep|break: neither pojadhav pls move the retro to thursday as we discussed last time | 10:38 |
| rlandy | pojadhav: hi - let's talk re: sched | 10:38 |
| pojadhav | rlandy, sure | 10:39 |
| rlandy | bhagyashri - pls join if you are around | 10:39 |
| rlandy | https://meet.google.com/bae-jvvs-sfb?pli=1&authuser=0 | 10:39 |
| rlandy | pojadhav: ^^ | 10:39 |
| frenzyfriday | 0/ new vm for downstream cockpit - 10.0.111.46 . After setting up the cockpit there I'll update bitwarden | 10:53 |
| *** ysandeep|break is now known as ysandeep | 11:03 | |
| *** vkmc_ is now known as vkmc | 11:06 | |
| ysandeep | rlandy, pojadhav: thanks for sorting out the schedule. | 11:06 |
| *** tweining___ is now known as tweining | 11:08 | |
| *** pojadhav is now known as pojadhav|afk | 11:14 | |
| rlandy | review time anyone? | 11:17 |
| ysandeep | rlandy, joining | 11:18 |
| rlandy | ysandeep: dropped | 11:19 |
| rlandy | there was nothing on today's list | 11:19 |
| * rlandy looking for akahat's review | 11:19 | |
| ysandeep | rlandy, ack | 11:19 |
| rlandy | akahat: do you still have a review that needs to be merged? | 11:20 |
| rlandy | https://review.rdoproject.org/r/c/rdo-jobs/+/44591 | 11:22 |
| rlandy | akahat: thanks - looks good now | 11:22 |
| rlandy | chandankumar: ^^ can you second core review - then we can merge | 11:22 |
| chandankumar | sure | 11:23 |
| rlandy | frenzyfriday: hi - what's the status of internal cockpit | 11:23 |
| frenzyfriday | rlandy, I have the vm, deployment is in progress | 11:23 |
| chandankumar | I thought it got merged | 11:24 |
| rlandy | jm1: do we need to keep the internal cockpit on hardware because of the domain name? | 11:24 |
| rlandy | can we assign that to a vm on PSI? | 11:24 |
| rlandy | frenzyfriday++ | 11:24 |
| frenzyfriday | we might have to send a mail regarding the domain again | 11:24 |
| akahat | rlandy, that's from marios need to merge now. | 11:25 |
| chandankumar | frenzyfriday: please make sure we get a patch out also for moving to podman & podman compose | 11:25 |
| akahat | rlandy, chandankumar thanks! | 11:25 |
| frenzyfriday | yep | 11:25 |
| chandankumar | frenzyfriday: please keep the old vm rafeal wants to upgrade that | 11:25 |
| *** dviroel is now known as dviroel|rover | 11:26 | |
| dviroel|rover | o/ | 11:26 |
| soniya29|ruck | hello | 11:26 |
| rlandy | dviroel|rover: did a sync with soniya29|ruck ... we went over program call doc | 11:31 |
| rlandy | there is a legit new bug on internal | 11:31 |
| rlandy | dviroel|rover: soniya29|ruck: actually, let's sync quickly - will be easier | 11:31 |
| dviroel|rover | ack | 11:32 |
| soniya29|ruck | ack | 11:32 |
| rlandy | https://meet.google.com/giv-cuob-wjs?pli=1&authuser=0 | 11:32 |
| rlandy | https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/prepare-node/tasks/main.yaml#L43 | 11:37 |
| rlandy | ysandeep: hey - let's touch base about the confluence page | 11:46 |
| rlandy | I added some stuff there and then read your wiki | 11:46 |
| rlandy | will ping after I drop from program call | 11:46 |
| ysandeep | rlandy, ack let me know when you want to touch base | 11:47 |
| jm1 | rlandy, frenzyfriday: we cannot assign that internal name to a psi vm but we could create a new domain name. dont know how that is handled on psi though | 11:49 |
| rlandy | jm1: a domain name is really useful but running off really hardware seems dangerous | 11:50 |
| rlandy | jm1: frenzyfriday: I think let's go with the vm for now | 11:50 |
| rlandy | get it set up | 11:50 |
| rlandy | even if we have to use an ip for now | 11:51 |
| rlandy | then we can try upgrade the vm on the hardware | 11:51 |
| rlandy | so we have one working env to refer to if we mess that one up | 11:51 |
| rlandy | sound ok? | 11:51 |
| jm1 | frenzyfriday: do you use scripts in ci-config to provision and configure the internal vm? | 11:51 |
| rlandy | frenzyfriday: then we can look into how to get a domain name on psi | 11:52 |
| jm1 | rlandy, frenzyfriday: i was not in the loop for the psi vm for downstream cockpit. how do you do it? manually? ansible? | 11:52 |
| rlandy | jm1: ansible | 11:53 |
| rlandy | with some internal repos | 11:53 |
| * akahat back in hour. | 11:54 | |
| jm1 | rlandy, frenzyfriday: afaiu sagi provisioned the downstream cockpit as a libvirt vm (in 2019?). the vm provisioning was probably done manually because we dont have code for that. the configuration inside the vm was (and is) done using ansible playbooks in ci-config repo. its actually the same code as for upstream cockpit with small changes. | 11:58 |
| frenzyfriday | jm1, I managed to find till here: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/grafana_playbook.yml | 11:58 |
| frenzyfriday | in out infra deployment workflow what actually runs this script? | 11:58 |
| rlandy | jm1: ack - the vm itself | 11:59 |
| jm1 | rlandy, frenzyfriday: all our vms on vexxhost are using ansible in pull mode. so be careful when changing ci-config because our running vms will pull that code and update their config | 11:59 |
| jm1 | ^ applies to downstream cockpit as well | 12:00 |
| rlandy | jm1: that's a good thing | 12:00 |
| rlandy | we will get the updates | 12:00 |
| rlandy | in upstream and downstream | 12:00 |
| rlandy | it should be the same way | 12:00 |
| rlandy | join program call | 12:00 |
| rlandy | joining | 12:00 |
| rlandy | sorry - not an ask on anyone else (apart of rr to join) | 12:01 |
| jm1 | rlandy, frenzyfriday: what i mean is that we have to be careful with manually doing stuff on vms because ansible is working in the background (in pull mode). we also have to be careful with updating ci-config for the downstream cockpit on psi because our downstream cockpit on our internal server will also pull these changes | 12:02 |
| rlandy | jm1: that should be the right way to operate | 12:02 |
| frenzyfriday | yep | 12:02 |
| rlandy | for downstream to be able to pull any upstream changes | 12:02 |
| frenzyfriday | jm1, I can see https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/tenant_vars/infra-tripleo/servers.yml#L55 - so looks like there should be a script which creates the Vm as well | 12:03 |
| * rlandy will ask if we can get a new domain name on psi | 12:03 | |
| jm1 | frenzyfriday: right, this is for provisioning vms on vexxhost | 12:04 |
| frenzyfriday | jm1, do you have some time for a quick gmeet? | 12:05 |
| jm1 | sure | 12:05 |
| * frenzyfriday is looking for "the origin" of playbooks | 12:05 | |
| frenzyfriday | jm1, https://meet.google.com/zgm-pnqt-zaw | 12:06 |
| rlandy | soniya29|ruck++ nicely done | 12:23 |
| soniya29|ruck | rlandy, thanks | 12:24 |
| rlandy | dasm|off: pojadhav|afk: soniya29|ruck: I will miss the planning meeting today - have a clashing meeting | 12:30 |
| soniya29|ruck | rlandy, okay | 12:30 |
| rlandy | soniya29|ruck: pls will you create that BZ for standalone component so we have it for the CIX call. thanks | 12:31 |
| soniya29|ruck | rlandy, ohh sorry, i left that i middle | 12:31 |
| rlandy | pojadhav|afk: pls ping when you have the planning section created and I'll add my notes | 12:32 |
| rlandy | ysandeep: time now to talk about confluence? | 12:33 |
| ysandeep | rlandy, sure | 12:33 |
| soniya29|ruck | arxcruz, rlandy, kopecmartin_ , chandankumar, ysandeep, mind if i move tempest meeting to next wednesday since clashing with retro? | 12:33 |
| rlandy | ysandeep: https://meet.google.com/?hs=197&pli=1&authuser=0 | 12:33 |
| rlandy | soniya29|ruck: pojadhav|afk is moving the retro | 12:33 |
| rlandy | ysandeep: sorry - https://meet.google.com/wnn-qmda-yhp?pli=1&authuser=0 | 12:34 |
| soniya29|ruck | she is moving? i can see it still on calendar..so i got confused | 12:34 |
| arxcruz | rlandy so, can we keep it? we have a guest today | 12:34 |
| soniya29|ruck | rlandy, ack, no problem | 12:34 |
| *** pojadhav|afk is now known as pojadhav | 12:35 | |
| pojadhav | rlandy, ack | 12:36 |
| pojadhav | rlandy, added new section for new sprint @line 31 - https://hackmd.io/P5sTPMSBREe5kBS_4jDhIg | 12:37 |
| pojadhav | soniya29|ruck, dasm|off, ysandeep, chandankumar : please add new new topics if any @line 31 - https://hackmd.io/P5sTPMSBREe5kBS_4jDhIg | 12:44 |
| soniya29|ruck | rlandy, https://bugzilla.redhat.com/show_bug.cgi?id=2121094 | 12:47 |
| rlandy | ysandeep: dviroel|rover: so we have anything on the agenda for today's podified meeting? | 12:50 |
| rlandy | if now - maybe we can run through the plans for the POC? | 12:51 |
| rlandy | I'll move the meeting minutes in my afternoon - so you can take notes on the hackmd for today | 12:51 |
| rcastillo | o/ | 12:51 |
| rlandy | soniya29|ruck: thanks | 12:51 |
| ysandeep | rlandy, nothing on agenda, sounds good to discuss POC | 12:51 |
| rlandy | rcastillo: hey - just confirming - retro is tomorrow | 12:51 |
| rlandy | ysandeep: we should also mention move of info to confluence | 12:52 |
| rlandy | rcastillo: you are up as the fearless retro leader, right? | 12:52 |
| rcastillo | rlandy: yup | 12:53 |
| dviroel|rover | rlandy: ysandeep: agree | 12:54 |
| rlandy | soniya29|ruck: pls raise a CIX for that BZ | 13:00 |
| soniya29|ruck | rlandy, done | 13:00 |
| arxcruz | kopecmartin_ soniya29|ruck will you join? | 13:01 |
| rlandy | soniya29|ruck: email to rhos-dev? | 13:01 |
| rlandy | soniya29|ruck: to get a CIX we need that from a BZ | 13:02 |
| soniya29|ruck | rlandy, yeah, its done | 13:03 |
| soniya29|ruck | rlandy, i have forwarded you the email, please check | 13:04 |
| rlandy | soniya29|ruck: thank you - idk I didn't see that before | 13:05 |
| soniya29|ruck | rlandy, no problem :) | 13:06 |
| *** kopecmartin_ is now known as kopecmartin | 13:10 | |
| *** dasm|off is now known as dasm | 13:27 | |
| dasm | o/ | 13:27 |
| * soniya29|ruck needs a break..will be back in 10-15 mins | 13:27 | |
| *** soniya29|ruck is now known as soniya29|ruck|afk | 13:27 | |
| dviroel|rover | soniya29|ruck|afk: rlandy: downstream issue is related to https://opendev.org/openstack/tripleo-ci/commit/f80cf14b624f421d407854b1adddd423aa53fa0f | 13:29 |
| dviroel|rover | i will propose a fix on downstream | 13:30 |
| rlandy | dviroel|rover++ thank you | 13:32 |
| rlandy | dasm: hello ... | 13:33 |
| dasm | rlandy: o/ | 13:33 |
| rlandy | left notes on planning meeting | 13:33 |
| dasm | ack | 13:33 |
| rlandy | have a clash | 13:33 |
| *** soniya29|ruck|afk is now known as soniya29|ruck | 13:40 | |
| dviroel|rover | rlandy: there are more jobs affected, fixing | 13:44 |
| rlandy | dviroel|rover: need help there? | 13:45 |
| dviroel|rover | rlandy: no, on review only | 13:47 |
| frenzyfriday | jm1, hey, do you know what server is provisioner here https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/provision-all.yml#L8? localhost? | 13:48 |
| dviroel|rover | rlandy: chandankumar: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/425936 | 14:23 |
| rlandy | dviroel|rover: chandankumar: ^^ looks ok to me - waiting on chandankumar to vote - then will merge | 14:40 |
| chandankumar | done | 14:41 |
| chandankumar | dviroel|rover++ for fixing it | 14:42 |
| jm1 | frenzyfriday: provisioner is defined here, its ansible_connection=local aka executed on the ansible controller. https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/inventories/inventory.ini | 14:45 |
| jm1 | frenzyfriday: you might have to call ansible-playbook with "-i inventories/inventory.ini" to get it working | 14:45 |
| *** ysandeep is now known as ysandeep|dinner | 14:46 | |
| dasm | hmmm, why ovb one time works, but the other doesn't? (rhetorical question)_ | 14:58 |
| pojadhav | chandankumar, ysandeep|dinner soniya29|ruck : UA/TC/PM sync | 15:02 |
| dasm | jm1: some time ago you mentioned about errors spilling out to journalctl for telegraf. I tried to find it in the history but failed. Can you point me where is this instance? | 15:10 |
| dasm | jm1: wrt TRIPLEOCI-1155 | 15:10 |
| chandankumar | see ya! | 15:17 |
| dasm | chandankumar: o/ | 15:17 |
| dviroel|rover | o/ | 15:18 |
| * pojadhav leaving for the day | 15:18 | |
| *** pojadhav is now known as pojadhav|out | 15:18 | |
| jm1 | dasm: frenzyfriday is working on downstream cockpit, not sure what the status is. i dont want to interrupt her work so i cannot check atm. | 15:22 |
| jm1 | dasm: i am not sure if journalctl was spilling out the errors or docker logs. | 15:23 |
| frenzyfriday | jm1, you can work on the centos7 vm if you want. I am working on the new vm | 15:23 |
| dasm | jm1: nah, i wanted to check that, but i don't remenber the instance :) | 15:23 |
| frenzyfriday | in the centos7 vm telegraf should throw errors if you try to rebuild | 15:23 |
| jm1 | frenzyfriday: ack but the old instance is still running? | 15:24 |
| *** dviroel|rover is now known as dviroel|rover|lunch | 15:24 | |
| frenzyfriday | yep | 15:24 |
| frenzyfriday | jm1, we are just not getting new patches | 15:25 |
| frenzyfriday | jm1, ansible-playbook infra-setup/provision-all.yml -i inventories/inventory.ini did not work as well | 15:25 |
| frenzyfriday | somewhere I am missing host entries | 15:25 |
| jm1 | frenzyfriday: we can debug that tomorrow if you want | 15:25 |
| frenzyfriday | jm1, cool, I'll ping you tomorrow morning | 15:26 |
| jm1 | dasm: login to tripleo-cockpit.lab4.eng.bos.redhat.com and get root. then 'docker logs --since "2022-08-01" telegraf_py3 2>&1 | less' shows you some errors and journalctl shows others | 15:26 |
| jm1 | frenzyfriday: sure! | 15:26 |
| dasm | jm1: cool! i was looking for that instance. thx | 15:27 |
| * soniya29|ruck leaving for the day | 15:31 | |
| frenzyfriday | jm1, it works now, I was in the wrong dir XD Also, we were wondering why the directory was called infra-tripleo - it has to be the name of the tenant on openstack | 15:32 |
| *** ysandeep|dinner is now known as ysandeep | 15:36 | |
| dasm | frenzyfriday: you mentioned about issues to build docker? we're still running cs8 with py3.6 on telegraf. is that why? | 15:36 |
| frenzyfriday | dasm, no, docker pull quay.io/centos/centos:stream9 doesnt work on the centos 7 vm | 15:37 |
| frenzyfriday | due to manifest version mismatch | 15:37 |
| dasm | :( | 15:37 |
| dasm | ack | 15:38 |
| frenzyfriday | So when you try to rebuild telegraf it fails at docker pull | 15:38 |
| dasm | so, there are no updates for last few weeks :( | 15:38 |
| dasm | gotcha | 15:38 |
| frenzyfriday | But the already build image should not cause any problems | 15:38 |
| frenzyfriday | yep, exactly :( | 15:38 |
| dasm | hmmm... can we revert back to docker repo for cs9 until we have a new vm up and running? | 15:39 |
| dasm | would it be useful? | 15:39 |
| dasm | hmm, probably not: > Last pushed a year ago | 15:39 |
| * frenzyfriday running the server setup role for incockpit - hope nothing breaks in psi | 15:47 | |
| ysandeep | rlandy, dviroel|rover|lunch fyi.. After you left there were suggestion that we do things in baby steps, and as a first step just build check/gate jobs without promotion pipeline(and use the current job that promote the latest operator image in post merge). I will update the confluence page in my morning. | 15:48 |
| rlandy | ysandeep: dviroel|rover|lunch: let's meet on friday and talk about what makes sense for us | 15:51 |
| rlandy | listen to all suggestions and see what is the right approach | 15:51 |
| rlandy | ysandeep: dviroel|rover|lunch: I'd like to see your take on this - w/o a whole group meeting | 15:52 |
| ysandeep | There are already jobs that push the latest operator image in post merge step(there are some github action jobs that does that) and we are already attempting to first setup check jobs.. So I think its not a bad idea and actually more practical for Nov meetup timeline. but let's chat more on Friday - I wan to listen dviroel|rover|lunch take on this as well. | 15:55 |
| ysandeep | They are not saying no for promotion pipeline - just as the next step | 15:56 |
| * ysandeep out, see you tomorrow | 15:58 | |
| *** ysandeep is now known as ysandeep|out | 15:58 | |
| rlandy | ysandeep|out: dviroel|rover|lunch: ok - book us a slot for friday | 16:02 |
| rlandy | lunch - biab | 16:05 |
| *** dviroel|rover|lunch is now known as dviroel|rover | 16:21 | |
| *** jpena is now known as jpena|off | 16:42 | |
| jm1 | frenzyfriday: @infra-tripleo tenant directory name: very good observation! please please document that in readme :) | 17:33 |
| * dasm is going for a walk. bbiab | 17:49 | |
| * dasm is back | 18:19 | |
| dviroel|rover | rdo zuul seems to be slow again | 18:37 |
| rlandy | slow how? | 18:43 |
| dviroel|rover | rlandy: lots of jobs on queue, but there isn't so many jobs running | 18:43 |
| rlandy | k - see messages on rhos-ops | 18:43 |
| dasm | dviroel|rover: i see some nod failures on my jobs. some of them stiil haven't started. | 18:45 |
| dasm | it's been in that state for last x minutes. | 18:45 |
| dasm | it's kinda bothering | 18:46 |
| dviroel|rover | vexx is working on another fix | 18:47 |
| dviroel|rover | let's see | 18:47 |
| dasm | ack | 18:47 |
| rlandy | dasm: looking at planning meeting notes ... https://hackmd.io/P5sTPMSBREe5kBS_4jDhIg - line 41 | 18:54 |
| rlandy | how is that relevant? to 17.1? | 18:55 |
| dasm | rlandy: i thin it was mis copied. pojadhav|out ^ | 18:56 |
| dasm | rlandy: oh, i think i know why it is there. it was used as a reference why we're in need to enable 17.1 | 18:56 |
| rlandy | oh ok | 18:57 |
| dviroel|rover | rlandy: already validated that BZ 2121094 is fixed, i will add logs and close | 19:13 |
| rlandy | dviroel|rover++ | 19:13 |
| rlandy | dviroel|rover: kicked a few testprojects downstream for some missing jobs | 19:14 |
| dviroel|rover | rlandy: yeah, i saw, thanks | 19:14 |
| rlandy | dviroel|rover: I guess we'll try rdo when zuul gets back from vacation | 19:14 |
| dviroel|rover | yep, a have a list of jobs to rerun alredy | 19:15 |
| dviroel|rover | lets see if remains stable | 19:15 |
| dviroel|rover | it is back already | 19:15 |
| rlandy | ok - starting to read operator testing docs | 19:16 |
| rlandy | dasm: you need anything? | 19:16 |
| dasm | rlandy: working zuul? | 19:17 |
| dasm | that's really the issue atm. jobs either not starting, or taking forever to start. | 19:17 |
| rlandy | right | 19:17 |
| rlandy | sorry - am pretty helpless there | 19:17 |
| dasm | none of my jobs finished today. just trying to do other things -- not requiring zuul atm | 19:18 |
| dasm | rlandy: i know, it's not your fault. just venting out my disappointment :) | 19:18 |
| dasm | in the same time i know vexx people work to fix anything and everything. i've been there too :) | 19:18 |
| rlandy | k | 19:20 |
| * jm1 have a nice evening #oooq 🍻 | 19:22 | |
| * jm1 signing out | 19:23 | |
| dasm | jm1[m]: o/ | 19:25 |
| rlandy | dviroel|rover: rdo still hosed? | 20:53 |
| * rlandy tries testproject | 20:55 | |
| dviroel|rover | rlandy: no, seems to be working now | 20:55 |
| rlandy | dviroel|rover: ok - I want to test https://review.opendev.org/c/openstack/tripleo-ci/+/854426 on wallaby | 20:56 |
| rlandy | c8 and c9 | 20:56 |
| dviroel|rover | rlandy: ok, i will kick master again in a few minutes - last run was a mess | 20:58 |
| dviroel|rover | and I give up my almost promoted hash | 20:58 |
| rlandy | np - my testproject will just be two container builds | 20:58 |
| rlandy | and pushes | 20:58 |
| *** dasm is now known as dasm|off | 21:32 | |
| dasm|off | take care team! i'm going offline! | 21:32 |
| dviroel|rover | oh tripleo - master is all red | 21:33 |
| dviroel|rover | container-puppet-nova_metadata | 21:34 |
| * dviroel|rover dinner | 22:30 | |
| *** dviroel|rover is now known as dviroel|rover|brb | 22:30 | |
| *** dviroel|rover|brb is now known as dviroel|rover | 23:26 | |
| * rcastillo leaves for the day | 23:52 | |
| rcastillo | o/ | 23:52 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!