Tuesday, 2022-12-20

jrosserthat sed error from moha7 is odd indeed07:01
jrossermoha7: would be very interesting if you can run ‘/opt/ansible-runtime/bin/python -c "from importlib.metadata import version; print(version('openstack-ansible'))")’ on the 22.04 install that fails07:06
jrosseralso very helpful if you tell us which branch you use and what kind of OS install you have (like cloud-image or ISO installer etc)07:07
moha7Hi; I passed the first deployment step, but I get the following error while running`openstack-ansible setup-infrastructure.yml` ---> https://ibb.co/ngqxSKT08:29
moha7https://www.irccloud.com/pastebin/Ngyfbbms/08:31
noonedeadpunkmoha7: I assume you have some connectivity issues between repo containers08:33
noonedeadpunkor some trafic being firewalled08:33
moha7this is the etc/hosts file of the container "infra1-repo-container-6496250d" -->  https://ibb.co/T4TS87b --> But I don't know why it is using the ranges 172.17.245.0 and 172.17.247.0 while I've set 172.17.246.0 as br-mgmt08:35
moha7The 1st section of my openstack_user_config.yml file: https://p.teknik.io/Raw/jL45m where 172.17.246.0 is set08:38
moha7infra1_horizon_container-64c2e3be on 172.17.245.22108:41
moha7but there's no 172.17.245.0/x range in my networks!08:42
jrossermoha7: are you sure that 172.17.246.0/22 is on a proper /22 boundary? you can't just choose these arbitrarily08:45
jrosseralso if you are deploying a multinode lab on esxi you must turn off whatever mac/ip security there is between VMs as otherwise you will run into many problems like you see with glusterfs08:46
jrossereach node has many IP and many mac addresses and the virtualisation environment doesnt know about them and drops the traffic by default08:46
jrossermoha7: also did you solve the problem on ubuntu 22.04? if there is a bug we need to know how to reproduce it in order to fix it08:47
jrossermoha7: you br-mgmt range is actually 172.17.244.0 to 172.17.247.255 with the config you have made08:53
moha7OMG! Unbelievable mistake; It should be /24. Thanks09:16
moha7For the Ubuntu issue, I'll try it again on a fresh Ubuntu09:18
*** akahat|ruck is now known as akahat|ruck|lunch09:42
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-lxc_hosts stable/zed: Ensure tar is installed on LXC host  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/86817610:40
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-lxc_hosts stable/yoga: Ensure tar is installed on LXC host  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/86817710:41
*** dviroel|out is now known as dviroel10:58
*** akahat|ruck|lunch is now known as akahat|ruck11:05
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Define name for all collections in a-r-r  https://review.opendev.org/c/openstack/openstack-ansible/+/86820512:37
mgariepygood morning everyone12:57
damiandabrowskihi!12:57
mgariepyhow are you doing?12:59
damiandabrowskia bit tired of clearing snow from the yard basically every day :D13:00
mgariepyhaha13:00
mgariepywhere are you located ?13:00
mgariepyi'm in quebec. i prefer having snow instead of slush and water/ice everywhere :D13:00
damiandabrowskiPoland, Cracow13:01
damiandabrowskihaha, that's true :D 13:01
mgariepyi guess you do have snow every winter there :)13:02
mgariepyyou are a bit upper north than me :P13:02
damiandabrowskiwe do but this year is special...didn't see that much snow since at least 5 years :D 13:04
mgariepywinter is relaxing :D13:04
mgariepyi do a big vegetable garden in the summer. so during winter i'm way less busy with various tasks  :D 13:05
damiandabrowskihaving your own vegetables sounds awesome, need to think about it one day :D 13:07
mgariepylast summer was aweful tho lot's of rains and not much heat..13:07
mgariepyso all the heat loving crop didn't produce much 13:08
mgariepyi had like 8 or 10kg of tomatoes from the garden, previous year was more like 45.13:08
damiandabrowski:(13:10
mgariepyat some point i'll build a greenhouse :)13:10
* noonedeadpunk has only rains now ;(13:10
noonedeadpunkon the bright side it's oranges and lemons season now 13:11
mgariepythat must be awesome fresh oranges must be so good.13:11
noonedeadpunkyeah, that's true13:12
mgariepyit's day and night for apples when in seasons. so it must be really good for oranges as well :D13:12
noonedeadpunkQuite mindblowing for me tbh13:12
mgariepyare there a lot for different varieties ?13:13
noonedeadpunkUm, have very little idea to be honest. As I used to the climate with proper winters, and it's first time spending winter in lcimate where lowest temperature by now was +413:14
mgariepyhere in the store we do not have much varieties for oranges. like 2 or 3 13:14
noonedeadpunkah, in stores. I don't think it's a lot. Or well. Each store seems to have it's own supplier so they all slightly different. And can also differ by week in the same store13:15
noonedeadpunkAlso you can see how ppl selling their own oranges from cars on the streets13:15
mgariepyfor apples there are quite a few in stores but there are places where they do grow more kind of apples13:15
mgariepynice :D13:15
noonedeadpunkand landlord has couple of lemon and orage trees as well, so he gave couple just from the tree13:16
opendevreviewKirill Tyugaev proposed openstack/openstack-ansible-ops master: Improve venv cleanup regexp discovery  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/86821213:16
noonedeadpunkBut I think all apples here are Polish. At least they're not local for sure13:17
mgariepyon a side note, chatgpt is impressive imo..13:19
mgariepyhttps://paste.openstack.org/show/bXWsCsyJla5uiCaB7pLR/13:22
noonedeadpunkI usually jsut use https://regex101.com/ lol13:25
noonedeadpunkbut yes, that;s quite impressive13:25
mgariepyit's just fun :D haha.13:25
noonedeadpunksoon we all will loose our jobs and AI will just manage k8s 13:25
mgariepymuch better than any other assistant.13:25
noonedeadpunkthat's true13:26
noonedeadpunkyou're using it as chrome extension or smth?13:26
mgariepynop13:27
mgariepyi don't like extension.13:27
mgariepythey can change hands too easily .. lol13:27
noonedeadpunk`OpenAI's services are not available in your country.` ;(13:28
mgariepy** insert vpn pub here **  13:28
noonedeadpunk but it checks by phone number.13:29
noonedeadpunkso VPN not helpful13:29
noonedeadpunkAnd google voice has banned me :D13:29
mgariepyvoip.ms ?13:29
noonedeadpunkthat is interesting....13:31
mgariepynot free but kinda cheap 13:31
noonedeadpunkI bet they'll block me as well once I try to add my UA credit card...13:32
noonedeadpunkworth a shot though13:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Allow to skip clone process for some repos  https://review.opendev.org/c/openstack/openstack-ansible/+/86821713:59
noonedeadpunk868205 is quite painful for me atm... 14:17
jrossernoonedeadpunk: you mean you need the feature or the patch is difficult?14:25
noonedeadpunkneed feature :)14:26
noonedeadpunkhave totally isolated env - no proxies. And realized I can't really do bootstrap14:26
noonedeadpunk(in proper way)14:26
jrosserahha interesting14:26
jrossermy deploy host at least has always had a proxy14:27
jrossereven if through a bastion to the targets was isolated14:27
noonedeadpunkyeah, so we do mirror git repos now14:27
noonedeadpunkand basically need to say that all we need should be taken from them and original content should be ignored14:27
jrosseryeah we do have that for the targets14:28
jrosserpip/pypi must be hard though14:28
noonedeadpunkLuckily it wasn't me who had to clone pypi :D14:29
noonedeadpunkI think it's kind of proxy though for pypi14:29
jrosserthis is from some time ago but does that https://github.com/bbc/rd-ansible-devpi-proxy14:30
noonedeadpunkI think that pulp was leveraged for that14:30
*** cloudnull5 is now known as cloudnull14:31
noonedeadpunkLooks like we just followed https://docs.pulpproject.org/pulp_python/workflows/pypi.html manually14:36
noonedeadpunk#startmeeting openstack_ansible_meeting15:00
opendevmeetMeeting started Tue Dec 20 15:00:48 2022 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'openstack_ansible_meeting'15:00
noonedeadpunk#topic rollcall15:00
noonedeadpunko/15:00
jamesdentono/15:01
damiandabrowskihi!15:01
mgariepyhey o/15:01
noonedeadpunk#topic office hours15:02
noonedeadpunkSo, congrats and thanks for all contributors who put an effort for Zed release15:03
noonedeadpunkI wanted to count contributors but failed in a way time-wise for that :(15:03
jamesdenton\o/15:03
damiandabrowski\o/15:03
jamesdentonbig thanks to the heavy hitters who keep it all together15:04
noonedeadpunkWe obviously have issue with neutron :D15:05
noonedeadpunkThe thing that hits us on Zed with updated SHA for neutron, also hits for master15:05
noonedeadpunkSo we need to find the reason for https://review.opendev.org/c/openstack/openstack-ansible/+/86795415:05
jamesdentonis there a known upstream bug by chance?15:06
jamesdentoni can try to look into that15:06
noonedeadpunkI reached neutron folks but they were not helpful and were saying that we try to do weird things liek create network multiple times. But likely I just need to narrow down to specific commit that brings this regression...15:06
noonedeadpunkThe thing is that only metal deploy does fail15:07
noonedeadpunkand LXC is good15:07
jamesdentonit's the lock wait timeout issue, right?15:07
noonedeadpunkso it's smth when running neutron-server outside of the container15:07
noonedeadpunkwell... I guess15:07
noonedeadpunkat least that's what I see in neutron logs.15:07
noonedeadpunkI'd assume it can be realted to the sqlalchemy changes...15:08
jamesdentonok, i will try one locally today and see what's upo15:08
noonedeadpunkAs there was a workaround during branching/initial release that afterwards should be replaced with proper fix15:08
noonedeadpunkI won't have time for that until end of the week for sure15:09
noonedeadpunkAlso I'm not really sure what we are testing until 867954 lands. So I'd say it's better to sort it out15:11
noonedeadpunkOnce it's done I'm thinking to check ansible-core 2.14 for 2023.115:12
noonedeadpunkI'm not sure what to do with openstack collection though15:13
noonedeadpunkAlso in terms of Zed. They were planning release of 2.0 early 202315:14
noonedeadpunkAnd now we have SHA in the middle of the master that obviously has some bugs and things in progress15:14
NeilHanlono/, here & late (as usual) 15:15
noonedeadpunkBut I guess it would be good to backport usage of 2.0 after it get's tagged to Z15:15
noonedeadpunkother then that we should add upgrade jobs from Yoga which means updating our upgrade script in some way15:16
noonedeadpunkthat will allow selection of destination?15:17
noonedeadpunkor well, ask for the input?15:17
noonedeadpunkI wonder if it should be smth like dialog or just super simple15:18
jamesdentonmeaning, Y->Z or Y->A?15:18
noonedeadpunkYup15:18
noonedeadpunkor well15:18
noonedeadpunkyes, options are correct, but I've jsut realized that for upgrade script we just checkout to destination and then run it15:19
noonedeadpunkso it sounds, like we should detect just where we are based on the current git state and verify if it's valid or not....15:20
damiandabrowski+1 for backporting openstack ansible collection later if 1.x works fine with Zed15:20
damiandabrowskibut as I can see, it does not officially support Zed :/15:20
damiandabrowski"1.x.x releases of Ansible OpenStack collection are compatible with OpenStack SDK 0.x.x prior to 0.99.0 only (OpenStack Yoga and earlier)."15:20
noonedeadpunkWe use >1 but <2.0. So we use SHA from master, and they will create 2.0 from master as well, but in fairly later commit15:21
noonedeadpunk2.0 simply doesn't exist atm15:21
damiandabrowskiah okok15:21
damiandabrowskifine for me 15:22
noonedeadpunkso we're using even not alpha but some "work in progress" version that is stable enough according to our CI15:22
noonedeadpunkthe more I think about upgrsade jobs the more I feel complexity 15:23
noonedeadpunkOh. Also we discussed yestarday that for PKI role it would be great to allow using *_pipe modules from crypto. Which means that module can accept cert or private key from variable rather from file. And variable can be set to any lookup plugin eventually.15:25
noonedeadpunkThat would enable us to store certs/private keys in vault/sops/encrypted with ansible-vault15:25
noonedeadpunkThe only thing is how to save/create/generate certs for the first time. as while reading them is relatively easy, creating is not in fact15:26
noonedeadpunkI think question I have for everyone is - how do you save/store/manage certs? Hashi vault? 15:28
noonedeadpunktrying to narrow down options that we want to implement outside of file as it is today15:28
damiandabrowskiouh, these *_pipe modules look promising15:31
jrosserour deployments just have their certs using the pki role files on the deploy node today15:39
jrosserbut any other certs we have in hashi vault15:40
jrosseri think that we should refactor the PKI role a bit as it stands to make a `file` backend and switch to the *_pipe modules15:40
noonedeadpunkoh, btw, interesting question - how does hashi lookup module perform?15:40
jrosserthat should put in place all the structure needed to make the backend pluggable15:41
jrosserwe make it be in pre_tasks so it only calls it once, using lookup rather than the native module, but that is just history really15:41
noonedeadpunkso while I do understand how to store in Vault, but I don't really understand how to store using sops for example, or ansible-vault15:42
jrosseransible vault is difficult - i did not find an example for using ansible to create a new ansible-vault-encrypted var15:42
*** dviroel is now known as dviroel|lunch15:43
noonedeadpunksops should support that though....15:44
noonedeadpunkwell, I guess creating such var/file can be matter of command, given that you've defined password file in ENV var15:46
jrosseryeah community.sops.sops_encrypt15:46
noonedeadpunkbut it's tricky/nasty indeed15:46
jrosserso what i was thinking is we should make tasks/standalone/{{backend}}_read.yml / _write.yml15:46
jrosserand in the middle of including those is some 'generate' function that is conditional on if the read worked or not15:47
jrosserabstract away how/where the private key comes from, it just needs to end up in some well known var and a status fact be set15:48
noonedeadpunkand feed result of that to pipe... Yeah, that can work, though will be super complex15:48
noonedeadpunkI wonder if we want to resurrect our idea with roles testing before messing up with that?15:49
noonedeadpunkor well, your idea :)15:49
jrosserthat would be a great thing to do15:49
noonedeadpunkthat way we can try to ensure that we won't break current deplooyments very badly15:49
noonedeadpunkas workflow sounds quite complex15:50
noonedeadpunkbtw we also might want to get our vault role in for tests of integration for that backend15:50
noonedeadpunk(or use some 3rd party at worst)15:51
noonedeadpunkwell.. this has now quite solid list of pre-requisitives15:51
noonedeadpunkbut that's for good I think15:53
noonedeadpunkI do like the idea15:56
damiandabrowskibtw. as I mentioned today, we have broken gating for xena, this patch aims to fix it: https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/86810715:57
damiandabrowskiyesterday*15:57
noonedeadpunkdamiandabrowski: well, you can't do that I think15:57
noonedeadpunkmax supported erlang is 24.2 for rabbit 3.9.815:58
noonedeadpunkhttps://www.rabbitmq.com/which-erlang.html15:58
damiandabrowskiawwww, you're right :|16:00
noonedeadpunkwell, time is over, so will wrap this up16:00
damiandabrowskiso i have to bump rabbitmq version as well16:00
noonedeadpunkAlso want to remind, that next 2 meetings are cancelled16:00
noonedeadpunkSo see you all on next meeting in 2023 :)16:01
noonedeadpunk#endmeeting16:01
opendevmeetMeeting ended Tue Dec 20 16:01:17 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:01
opendevmeetMinutes:        https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-12-20-15.00.html16:01
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-12-20-15.00.txt16:01
opendevmeetLog:            https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-12-20-15.00.log.html16:01
noonedeadpunkdamiandabrowski: I'd rather not16:01
damiandabrowskiany other ideas?16:03
noonedeadpunkoh. they've dropped all 24.2? 16:03
noonedeadpunkdamn it16:03
noonedeadpunkHow I'm tired of these rabbit repos thing16:04
damiandabrowskiyup16:04
damiandabrowskihttps://cloudsmith.io/~rabbitmq/repos/rabbitmq-erlang/packages/?q=distribution%3Aubuntu+AND+distribution%3Afocal+AND+version%3A1%3A24.2*16:04
noonedeadpunkpfffff16:08
noonedeadpunkthe only idea I have is a terrible one which is switch back to https://www.erlang-solutions.com/downloads/ for erlang16:09
noonedeadpunkbut yes, likely we need to update both then16:11
opendevreviewDamian Dąbrowski proposed openstack/openstack-ansible-rabbitmq_server stable/xena: Upgrade rabbitmq and erlang version  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/86810716:17
opendevreviewDamian Dąbrowski proposed openstack/openstack-ansible-rabbitmq_server stable/xena: Upgrade rabbitmq and erlang version  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/86810716:20
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not run dstat by default  https://review.opendev.org/c/openstack/openstack-ansible/+/86822416:42
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not run dstat by default  https://review.opendev.org/c/openstack/openstack-ansible/+/86822416:55
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not run dstat by default  https://review.opendev.org/c/openstack/openstack-ansible/+/86822416:58
*** dviroel|lunch is now known as dviroel16:59
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Unset OSA-defined variables for bootstrap  https://review.opendev.org/c/openstack/openstack-ansible/+/86822717:11
noonedeadpunkthat is proper bug ^17:11
noonedeadpunkwell, both of these are proper bugs :(17:14
BobZAnnapolisfolks, any accurate documentation on how to successfully set up and run Rally/Tempest ? The openstack.org and rally read the docs instructions aren't working as smoothly as we'd like, wrong github urls, missing scenarios and tests after pip installs, etc, tia17:38
BobZAnnapolistrying not to bother you unnecessarily, is there an openstack-rally irc ?17:39
jrosserBobZAnnapolis: not exactly Rally/Tempest but we use refstack 17:49
opendevreviewMerged openstack/openstack-ansible-ops master: Improve venv cleanup regexp discovery  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/86821220:52
spateljamesdenton around21:02
*** dviroel is now known as dviroel|out21:42
*** rgunasekaran_ is now known as rgunasekaran21:54
opendevreviewMerged openstack/ansible-role-pki stable/yoga: Ensure CA privatekey permissions  https://review.opendev.org/c/openstack/ansible-role-pki/+/86763222:22
opendevreviewMerged openstack/openstack-ansible master: Unset OSA-defined variables for bootstrap  https://review.opendev.org/c/openstack/openstack-ansible/+/86822722:51
opendevreviewMerged openstack/openstack-ansible master: [doc] Fix document on dymanic_inventory tox usage  https://review.opendev.org/c/openstack/openstack-ansible/+/86797322:51

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!