Wednesday, 2021-11-10

*** mazzy5098811 is now known as mazzy50988100:10
*** mazzy5098812 is now known as mazzy50988100:18
*** mazzy5098814 is now known as mazzy50988100:42
holtgrewejingvar, mgoddard - let me rephrase. Is the following. I boot *all* nodes with PXE and they will be discovered by bifrost/ironic. Some, I might put into etc/kayobe/inventory/hosts, the rest will appear in etc/kayobe/inventory/overcloud.06:54
jingvarAs I understand autodiscovery is magic 06:58
jingvarI use a simple way, bifrost.yaml with ipmi for my servers06:59
holtgreweAnother question, `kayobe baremetal compute rename` - will this work if ipmi_address is set to a DNS name? Or do i have to use IP addresses? The playbook appears to use string equality only...07:10
*** amoralej|off is now known as amoralej07:57
opendevreviewMark Goddard proposed openstack/kayobe master: docs: Improve all-in-one scenario  https://review.opendev.org/c/openstack/kayobe/+/79700309:15
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/xena: docs: Fix python-openstackclient package name and init-runonce path  https://review.opendev.org/c/openstack/kolla-ansible/+/81726409:15
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/wallaby: docs: Fix python-openstackclient package name and init-runonce path  https://review.opendev.org/c/openstack/kolla-ansible/+/81726509:15
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/victoria: docs: Fix python-openstackclient package name and init-runonce path  https://review.opendev.org/c/openstack/kolla-ansible/+/81726609:15
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/xena: Fix octavia doesn't set subnet gateway_ip  https://review.opendev.org/c/openstack/kolla-ansible/+/81726709:16
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/wallaby: Fix octavia doesn't set subnet gateway_ip  https://review.opendev.org/c/openstack/kolla-ansible/+/81726809:16
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/victoria: Fix octavia doesn't set subnet gateway_ip  https://review.opendev.org/c/openstack/kolla-ansible/+/81726909:17
opendevreviewMark Goddard proposed openstack/kayobe master: Ubuntu: add upgrade jobs to gate  https://review.opendev.org/c/openstack/kayobe/+/81736709:19
opendevreviewMark Goddard proposed openstack/kayobe master: Remove chrony cleanup from overcloud host configure  https://review.opendev.org/c/openstack/kayobe/+/81727909:27
kevkohrw: here ? 09:31
hrwkevko: there09:33
kevkohrw: I'm trying to understand how tarball of "openstack code" is downloaded and extracted into image ...can you point me ? 09:34
hrwlet me check09:34
kevkohrw: because I have templates generated and all I can see is only  "ADD keystone-base-archive /keystone-base-source"  09:35
kevkoand I really don't know where it is downloading, extracting ..etc 09:36
hrwkevko: check kolla/image/build.py build_image_list()09:36
hrwkevko: and process_source() handles fetching09:37
kevkoaaa09:37
kevkothank you :D 09:37
hrwyw09:37
kevkois kolla supporting installation from git source ? 09:38
kevkoi mean ..currently ..out of the box 09:38
kevkohrw: because you probably remember that I'm building binary images for our use-case (it's easy to patch a bug in deb package) ... and lastly I've fixed neutron in debian package -> https://salsa.debian.org/openstack-team/services/neutron/-/blob/debian/wallaby/debian/patches/revert-l3-ha-retry-when-setting-ha-router-gw-status.patch09:40
hrwkevko: git clone kolla, cd kolla, pip install -r requirements.txt, ./tools/build.py09:41
kevkohrw: nope, I mean source from git inside docker image 09:41
kevkohrw: for example above mentioned neutron 09:41
hrwkevko: we do that by default iirc09:42
hrw$ pip list|wc -l09:42
hrw201309:42
hrw;d09:42
kevkohrw: well process_source is downloading tarball from https://tarballs.opendev.org 09:43
kevkoso it's archived tagged version ..isn't it ? 09:43
kevkobut actually not git source ..09:43
kevkoor am I missing something ? 09:43
hrwfetching tarball of master branch was probably quicker than cloning git repos09:44
hrwand takes less space09:44
kevkook, makes sense ..09:46
hrwkevko: users are free to edit SOURCES in kolla/common/config.py to point to git repos directly09:46
hrwkevko: fetching tarballs allows us to switch between "HEAD of branch" and versioned tarballs09:47
hrwkevko: we just stopped using versioned tarballs in stable/* branches09:47
kevkoyeah, that's clever ..because versioned tarballs are buggy in most cases ..09:48
kevkothat's the reason why I'm asking ..i had to patch neutron package in debian ...because we are packaging TAGged version in upstream ...09:48
kevkobut bugfixes are released to stable/* branch 09:48
hrwusing versioned tarballs == need to update from time to time, lack of latest bugfixes09:50
hrwso that's why we moved to 'HEAD of stable/*' tarballs09:50
kevkoyeah, clever ..09:51
hrwand that's why at previous PTG we decided to use stable/previous-branch for first weeks of release cycle09:51
kevkoi will start to migrate to source probably 09:51
hrwor no. that was to avoid additional breakage09:51
hrwkevko: you do not have much choice - binary gets deprecated in Yoga09:52
kevkoi know :) 09:52
kevkowe are delivering images to our client ..09:52
kevkoso i have to write some mechanism to strictly report versions of packages per dockerTAG + write some mechanizm which will check built version with potentionally new build version and report bugs fixed etc (from git commits and Closes-Bug)09:53
kevkoif you understand me :/09:53
kevkoso we can say, you should update, because report is saying ..this is fixed in stable/* of project * and we can build new images ..09:54
kevkohrw: so what to change to fetch sources via git ? 09:55
kevkoaaaa - I see, gnocchi is for example downloading via git 09:56
hrwkevko: for openstack components you may not change anything. git tarballs are created after commit lands in repo09:57
hrwkevko: will you parse logs for package infos?09:57
kevkowell, If i will have some storage with  objects - something like TAG -> { package_name: git_url, commit_id } ... then I can write automatic script which will check new commits and will find closes-bug for example 10:00
kevkobecause our clients are quite sensitive to reconfigure,redeploy of containers ...10:01
kevkoso I have to have good reason and nice report to show them they have to :P10:01
kevkois it bad idea ? 10:02
kevkohrw: and why all is not installed from stable branches ...but only main projects ? 10:04
hrwkevko: what you mean by 'all'?10:11
kevkoi mean oslo libraries etc 10:11
hrwthey are version constrained to what stable/* requirements repo says10:11
hrwas they  have pypi releases10:12
hrwkevko: check install_pip() macro in docker/macros.j210:14
kevkohmm, so if something is fixed in stable branch of some openstack library ..and not released as pypi package ..it's just not installed 10:14
hrwyep10:18
kevkoThat's bad, isn't it?10:20
hrwthere has to be a point where we stop10:21
hrwotherwise we start building glibc and kernel10:21
kevko:D - it is still python 10:25
opendevreviewAdrian Andreias proposed openstack/kolla-ansible master: docs: Install openstack-client with upper constraints  https://review.opendev.org/c/openstack/kolla-ansible/+/81607610:45
admin1haproxy_ssl_self_signed_regen: false   and haproxy_user_ssl_cert and _key is set .. but  the playbook still wants to generate the cert .. has something changed/updated in the new version ? 13:16
admin1sorry wrong chat 13:16
*** amoralej is now known as amoralej|lunch13:22
opendevreviewAdrian Andreias proposed openstack/kolla-ansible master: docs: fix venv path "share/share"  https://review.opendev.org/c/openstack/kolla-ansible/+/81743214:02
*** amoralej|lunch is now known as amoralej14:06
holtgrewejingvar, do you use bios or uefi boot mode?14:37
jingvarlegacy14:40
holtgrewethanks, good to know14:40
jingvarthere no difference in my mind14:40
jingvarit just ironic settings14:41
holtgrewejingvar, did you configure legacy boot anywhere?14:48
yoctozeptomeeting in 10 - mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt frickler adrian-a14:50
jingvarholtgrewe: yes14:58
holtgrewejingvar, would you tell me where? ;-)14:58
holtgrewe*sigh* I had installed Rocky linux manually in UEFI mode on my dell servers previously... somehow an entry was created in UEFI to boot "rocky linux" but that was not present any more.14:59
holtgreweAfter switching to legacy boot in Dell BIOS/iDRAC, I now boot into my properly configured centos 8 stream machines15:00
jingvarstop, which boot do you mean15:00
yoctozepto#startmeeting kolla15:01
opendevmeetMeeting started Wed Nov 10 15:01:35 2021 UTC and is due to finish in 60 minutes.  The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
opendevmeetThe meeting name has been set to 'kolla'15:01
yoctozepto#topic Roll-call15:01
yoctozepto\o/15:01
adrian-a\o15:01
mgoddard\o15:02
osmanlicilegio/15:02
priteauo/15:02
ohorecnyo/15:03
hinermaro/15:03
yoctozeptowelcome all :-)15:03
halomivao/15:03
yoctozepto#topic Agenda15:03
yoctozepto* Roll-call15:04
yoctozepto* Agenda15:04
yoctozepto* Announcements15:04
yoctozepto* Review action items from the last meeting15:04
yoctozepto* CI status15:04
yoctozepto* Release tasks15:04
yoctozepto* Yoga cycle planning15:04
yoctozepto* Open discussion15:04
yoctozeptoplus the items from the whiteboard for today15:04
yoctozeptowhich is "Podman"15:04
yoctozepto#topic Announcements15:04
yoctozepto#info Xena RC2 was released15:05
yoctozeptowe should be wrapping up Xena as soon as mnasiadka returns from vacations15:05
yoctozeptomeaning: releasing the final "first version" of Xena15:05
yoctozeptoany other announcements?15:05
priteauThere are small fixes in Kayobe that would require an RC315:06
priteauIs kolla/kolla-ansible going to have an RC3 too?15:06
yoctozeptopriteau: unlikely but it's not an issue15:06
priteauYes, just curious15:07
yoctozeptook, guessing no more announcements15:08
yoctozepto#topic Review action items from the last meeting15:08
yoctozeptolet me see...15:08
yoctozeptoyoctozepto to run the meeting next week15:09
yoctozeptohmm, seems to be happening15:09
yoctozeptomnasiadka to triage security bugs and update them with resolution plan (if needed)15:09
yoctozeptodid not happen so needs to happen later when mnasiadka returns15:09
yoctozepto#action mnasiadka to triage security bugs and update them with resolution plan (if needed)15:09
yoctozeptoyoctozepto hide properly init-runonce15:09
yoctozeptodid not have time15:09
yoctozepto#action yoctozepto hide properly init-runonce15:10
yoctozepto#topic CI status15:10
yoctozeptothe whiteboard says green for all supported branches of all our projects15:10
yoctozeptoany comments on that?15:10
priteauKayobe is good at the moment15:10
yoctozepto:-)15:10
yoctozeptothanks15:11
yoctozeptoI have seen stuff merging to k and k-a as well so betting all well there as well15:11
yoctozeptoI have clenad up the "issue" under k-a15:12
yoctozeptocleaned*15:12
yoctozeptoas it was no longer valid15:12
yoctozepto#topic Release tasks15:14
yoctozeptolet's see the release cycle plan15:14
yoctozeptoxena is wrapping up so let's focus on yoga15:14
yoctozepto#link https://releases.openstack.org/yoga/schedule.html15:15
yoctozepto#link https://docs.openstack.org/kolla/latest/contributor/release-management.html15:15
yoctozeptoit's R-2015:15
yoctozeptono release tasks for this week15:15
mgoddardI jumped the gun a bit and proposed patches to bump previous_release15:15
yoctozeptothough I have seen mgoddard proposing stuff for R-1715:15
yoctozeptoindeed15:15
yoctozeptoI don't mind, they are quite safe15:16
mgoddardwell I think the idea was to try to look like the previous release for a while15:16
mgoddardto avoid breakages15:17
yoctozeptoyeah, but this one part essentially makes us upgrade from xena to xena so it's pretty safe15:17
yoctozeptowell, we could introduce a bug if we made upgrade=deploy15:17
yoctozeptoand did not notice15:17
yoctozeptobut otherwise...15:18
yoctozeptoanyhow15:18
yoctozeptolet's continue15:18
yoctozepto#topic Yoga cycle planning15:18
yoctozeptook, so it's R-20 in Yoga and we had some glorious plans15:18
yoctozeptowe can discuss Podman today as it's in the agenda and the relevant folks are around15:19
yoctozeptoI have already commented on the change15:19
yoctozepto#link https://review.opendev.org/c/openstack/kolla-ansible/+/81672415:19
yoctozeptoso it's about moving to using systemd15:19
ohorecnyyes, thank you for your comments. Guys from my team need to incorporate them.15:20
yoctozeptoindeed :-) if you have comments/questions, please feel welcome to put them on the change15:21
ohorecnySo what do you think, how it looks like. Since this change should we support only docker containers managed by systemd? Or also managed by docker?15:22
Fl1ntHi team!15:22
yoctozeptoohorecny: both first; I suggested to go in even smaller steps with refactoring itself being the first so that we can merge 1:1 functionality in new, more flexible, layout15:22
yoctozeptohi Fl1nt15:22
Fl1ntSorry for the vanishing, I've been a bit busy lately :(15:23
Fl1ntwhat's up yoctozepto ?15:23
ohorecnyyoctozepto: So you think that both approaches should be supported? And user can choose ?15:24
yoctozeptoFl1nt: we are in the middle of the meeting :-)15:24
yoctozeptoohorecny: I think at least code-wise for a moment they should coexist so that we can weed out weird edge cases15:25
Fl1ntaaaah forget about that TZ daytime switch fuu, thought I would have make it on time for once :(15:25
Fl1ntsorry15:25
yoctozeptoFl1nt: no probs15:25
ohorecnyyoctozepto: aha, ok we went by that way that we are moving it under systemd management15:27
mgoddardhttps://docs.podman.io/en/latest/markdown/podman-generate-systemd.1.html15:28
mgoddardlooks like you are using the start/stop approach15:28
mgoddardjust wondering what are the merits of the --new approach 15:28
yoctozeptomgoddard: "please review the generated files carefully before using them in production" eh15:29
yoctozeptoI guess it would be nice to see how they look like15:30
yoctozeptobut we should nonetheless control the unit files ourselves like we discussed during the ptg15:30
mgoddardagreed, I'm just looking at what podman will generate for you15:31
mgoddardit gives you two options15:31
mgoddardby default it just starts & stops, as in the proposed patch15:31
ohorecnyyes and this is how it is now for docker and merit is that in case of podman we will do it in similar way15:31
mgoddardbut if you pass --new, it will create and destroy15:31
mgoddardboth docker and podman could use either way15:32
ohorecnymgodard: genereting by using podman we already had implemented in that older big patchset15:32
mgoddardyes, I'm not suggesting we do that15:32
mgoddardI'm just saying, podman team has probably thought about this, let's see what they do15:33
mgoddardand they offer two options15:33
mgoddardso which should we follow?15:33
yoctozeptois the destruction on stop?15:34
mgoddardyes15:34
mgoddardhttps://docs.podman.io/en/latest/markdown/podman-generate-systemd.1.html#generate-systemd-unit-file-for-a-container-with-new-flag15:34
yoctozeptohmm, sounds like a way to ensure that containers are themselves stateless15:34
mgoddardIIRC ceph-ansible does it that way15:35
Fl1ntyes and it works perfectly tbn15:35
ohorecnyyes, that is good point, but it is in future now, when we introduce podman. As we discussed in PTG now we need to support docker with systemd15:36
yoctozeptothat's correct15:37
yoctozeptoand I've given you hints how to do it best code-structure-wise15:37
Fl1ntohorecny, you mean lifecycle of containers from systemd units or replace dumb-init with systemd ?15:37
mgoddardit's not necessarily in the future - you can use both patterns with docker and podman15:38
ohorecnyFl1nt: first option15:38
yoctozeptoFl1nt: the first15:38
Fl1ntok thanks :D15:38
mgoddardperhaps the proposed approach is most similar to what we have today15:38
yoctozeptomgoddard: let's maybe make it optional as well?15:39
yoctozeptoI want clean code change with the exact same behaviour except for systemd indirection15:39
Fl1ntjust a quick thought about that, as Ansible community.docker collection is able to really well manage containers lifecycle and that podman control containers from systemd units, is that really useful to integrate for docker?15:39
mgoddardFl1nt: yes15:40
mgoddardexact same ansible code for both15:40
Fl1ntok15:40
yoctozeptoand we need to have more control/interaction with containers than just with the barebones15:40
Fl1ntansible module let you control everything15:41
Fl1ntfrom volume to network and vars/etc15:41
ohorecnyexactly like mgoddard said, it is not necessary, but in other case we will have much bigger change and "duplicated" code15:41
yoctozeptoFl1nt: yeah, but at the cost of repeating a ton of stuff that we have customised for ourselves15:41
Fl1ntyep unicity is best.15:41
Fl1ntgotcha ^^15:42
mgoddardohorecny: one thing that would help: split the change into two parts. 1. refactor module to module_utils 2. make changes15:42
Fl1ntcool initiative about podman indeed :D15:42
mgoddardIMO refactors should not change behaviour15:42
ohorecnyFl1nt: in that other way the patchset will look like this: https://review.opendev.org/c/openstack/kolla-ansible/+/79922915:43
ohorecnyit was our first approach, but too big :(15:43
Fl1ntyep, I kept an eye on it while being away of IRC ;-)15:44
ohorecnymgoddard: ok15:44
ohorecnyGuys hinermar and halomiva do you agree?15:45
hinermarYes15:45
ohorecnythey are mostly working on these changes15:45
ohorecnyok, so next step is do some refactoring 15:46
yoctozeptobasically what I've commented about :-)15:46
ohorecnyin case of any questions about review comments please ask directly in change15:46
yoctozeptook, any other Yoga things to discuss at this stage?15:47
ohorecnyyoctozepto: right, thanks again15:47
Fl1ntyoga?15:48
Fl1ntOh release name ^^15:49
Fl1ntsorry15:49
yoctozeptoohorecny: you are welcome :-)15:49
frickleris there anything to discuss regarding "drop admin endpoints"? seems most people agreed to do it15:49
fricklerso I'd just add a reno and it should be fine?15:50
fricklerhttps://review.opendev.org/c/openstack/kolla-ansible/+/816164 for reference15:50
mgoddardlet's do it15:51
yoctozeptoTM15:52
yoctozeptoand I agree wholeheartedly15:52
yoctozeptothough we could somehow help operators clean up15:52
yoctozeptobut it's easy to do via a simple script I gues15:53
yoctozeptoguess*15:53
frickleryeah, I was just thinking whether to add a script for cleanup that could be ran on demand15:53
Fl1ntaaaah finally that blessed day where admin endpoints disapear !!15:53
fricklerI'll try to come up with something15:54
mgoddard+1 for cleaning up15:54
mgoddardI suppose it might cause issues if we do it during the upgrade15:55
Fl1ntyeah, should probably be a post-task15:55
fricklerI wouldn't want to do it automatically, yes. but make it easy for operators as a separate step seems a good compromise15:55
yoctozeptowell, best to keep it as a helper script15:56
yoctozeptook, we are nearing the end of time15:56
yoctozepto#topic Open discussion15:56
yoctozeptoanything really15:56
adrian-aI think this can be closed as Won't Fix, based on last meeting discussion https://bugs.launchpad.net/kolla-ansible/+bug/155404915:56
headphoneJamesin Yoga - trying to move towards being able enabling the new system scope policies.15:57
yoctozeptoadrian-a: agreed from my side15:57
Fl1ntso I've a discussion topic15:57
Fl1ntsince a bit of time now15:57
Fl1ntat work15:57
yoctozeptoheadphoneJames: and how's that going?15:58
headphoneJamesfirst pass15:58
headphoneJameshttps://review.opendev.org/c/openstack/kolla-ansible/+/81557715:58
headphoneJamesits ready for comments 15:58
Fl1ntwe're experimenting with an ansible method that gave us the ability in 1 repo to get both similar mechanisms than kolla/kolla-ansible15:59
Fl1ntwould you be interested in?15:59
headphoneJamesyoctozepto: The transaction only converts the config for the [keystone_authtoken] section, and not all the configurations for authenticating with other services through keystone.15:59
mgoddardFl1nt: would be interested to take a look16:00
yoctozeptoheadphoneJames: ack, sensible tradeoff16:01
yoctozeptook, out of time16:01
yoctozeptothank you all for participating16:01
yoctozepto#endmeeting16:01
opendevmeetMeeting ended Wed Nov 10 16:01:27 2021 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
opendevmeetMinutes:        https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-11-10-15.01.html16:01
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-11-10-15.01.txt16:01
opendevmeetLog:            https://meetings.opendev.org/meetings/kolla/2021/kolla.2021-11-10-15.01.log.html16:01
Fl1ntok, so basically, the idea is to follow the initial docker proposition for containers, meaning build > ship > run and so get thos three main common roles plus a specific role per service that call them.16:01
fricklerheadphoneJames: do you have any plan how to solve the upgrade problem for that?16:02
Fl1ntby doing so we're able to mimic kolla-ansible tools/wrapper but using pure ansible.16:02
Fl1ntI'll try to find a way to publish a skeleton of this idea without leaking intel :D16:02
headphoneJamesfrickler: no - open to ideas :)16:03
Fl1ntbasically it create a "kolla framework" over ansible engine :D16:03
mgoddardFl1nt: yoctozepto proposed something similar a while back, but I don't think he started it16:04
mgoddardFl1nt: any reason why you're doing this vs upstream kolla-ansible?16:04
Fl1ntaaaah nice! I'm glad to hear about that :D16:04
yoctozeptomgoddard: not quite the same but making it more frameworky for sure16:04
Fl1ntmgoddard, currently prototyping it and then proposing it later on to kolla, we won't use it as is in prod.16:05
yoctozeptobut I changed my responsibilities quite much and can't commit to the original plan16:05
yoctozeptowell, more like they were changed for me16:05
yoctozeptobut you get the point16:05
Fl1ntmgoddard, basically, I'm not yet working upstream on this framework as I don't want to commit on something that I won't be able to deliver as I'm currently really busy at work.16:06
Fl1ntso I'm using my job time to do that internally while still being able to upstream it later on by sharing the idea and a blueprint ^^16:07
mgoddardFl1nt: before you spend your time on this thing, I'd suggest working out if people actually want it16:08
Fl1ntYou're right16:09
yoctozeptothen go upstream asap16:10
yoctozeptobe bold16:10
Fl1ntbut I'll still spend time on it as it will be our internal skeleton/framework for everything else from HW deployment (spine/leaf/nodes/compliance) to SW deployment (internal software).16:10
yoctozeptonice16:11
Fl1ntok, I'll try to find time to create the blueprint.16:12
yoctozeptoor just propose a PoC ;-)16:13
Fl1ntah yeah could be easier tbh16:13
mgoddardFl1nt: having thought about this more, I'm going to say what I'm actually thinking and what I expect others are too17:01
mgoddardFl1nt: if you have some time available, rather than building yet another openstack deployment tool, please work with us to help maintain and improve the code we already have17:01
*** amoralej is now known as amoralej|off18:23
Fl1ntmgoddard, I'm in, no worries, just let me know when you need me :D18:45
opendevreviewJames Kirsch proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134019:29
opendevreviewPierre Riteau proposed openstack/kayobe-config-dev master: Synchronise job config with kayobe  https://review.opendev.org/c/openstack/kayobe-config-dev/+/81747821:24
*** hrww is now known as hrw22:34
opendevreviewMerged openstack/kayobe master: Ubuntu: add upgrade jobs to gate  https://review.opendev.org/c/openstack/kayobe/+/81736722:50

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!