Tuesday, 2024-01-16

noonedeadpunkmornings o/08:20
jrossergood morning09:19
jrosserinteresting set of TLS failures on https://review.opendev.org/c/openstack/openstack-ansible/+/90562109:24
jrossernoonedeadpunk: about the lxc config stuff from yesterday09:29
jrosserit is likley complicated like that for vars hierarchy reasons, a bit like we had with migrating haproxy09:30
jrosserwhen the containers are created you basically have vars available for the container hosts09:30
jrosserbut the specific config for a particular type of container may be in group_vars/service 09:31
jrosserand i expect that is why there is a seperate round of config done in the service playbooks09:31
noonedeadpunkI don't think that's the case frankly speaking. As when we create container what we do - `name: {{ inventory_hostname }}` and then delegate task to lxc host09:49
noonedeadpunkor well... maybe its exactly as haproxy case :D09:50
noonedeadpunkyeah, TLS failure is very weird... Like /constraints/upper_constraints_cached.txt should be local path? Why it fails SSL verification (or tries to do it at all)09:53
jrossersomething starting /constraints is odd too09:55
jrosserhttps://internal-vip:repo-port/constraints/.... so maybe it doesnt print the scheme and host part in the error09:57
noonedeadpunk`'dest': '/openstack/venvs/utility-28.1.0.dev25/constraints.txt', 'content': '--constraint https://172.29.236.101:8181/constraints/upper_constraints_cached.txt\n'})` so in paste above it looks fine10:01
opendevreviewMerged openstack/openstack-ansible-lxc_container_create master: Allow LXC container auto mounts to be customised  https://review.opendev.org/c/openstack/openstack-ansible-lxc_container_create/+/89322910:14
opendevreviewMerged openstack/openstack-ansible master: Add support for extra Python packages inside Ansible runtime  https://review.opendev.org/c/openstack/openstack-ansible/+/90522110:22
noonedeadpunkanother review needed for Zed bump: https://review.opendev.org/c/openstack/openstack-ansible/+/90534410:26
jrosserdone10:26
opendevreviewMerged openstack/openstack-ansible stable/2023.2: Bump SHAs for 2023.2 (Bobcat)  https://review.opendev.org/c/openstack/openstack-ansible/+/90558410:34
noonedeadpunkjrosser: do you think we can make 28.1.0?11:12
noonedeadpunkas releases team would insist on that due to release notes we made11:13
noonedeadpunkor should I try to motivate why not11:13
jrossernoonedeadpunk: well i'm not sure - we have not tried 28.x in our lab yet as we have to finish getting everything on jammy first11:16
jrosserso i don't have any direct feedback from us doing an actual upgrade11:16
signedHello, I am doing a AiO LXC deployment with Zun added to the base setup. However it seems that the setup-openstack playbook is hung at Execute tempest set, and has been for at least 30min, do I need it to continue or can I safely abort?11:17
signed"Execute tempest tests"*11:17
noonedeadpunksigned: well, you can abort. Most likely that Zun is having issues then if it's hung for too long11:24
noonedeadpunkYou will have some leftovers though created by tempest and not cleaned up due to abort11:24
signedI don't really care about leftovers (it's a VM), but I don't know if Zun gonna be operational11:27
noonedeadpunktempest jsut execute tests. so it's not affecting zun directly (except running tests towards it). So if it's not operation or not has nothing to do with cancelling the job11:32
noonedeadpunkBut usually when tempest stuck on test, it means that it can't spawn a resource and just waits for it until hits timeout11:32
noonedeadpunkso potentially zun is broken for $reason11:33
noonedeadpunkbtw, signed, do you have nested virtualization enabled for the VM?11:33
noonedeadpunkas that could be one of reasons 11:34
jrosserthere will be a log from tempest in the utility container which might show something helpful about why it is stuck11:42
opendevreviewMerged openstack/openstack-ansible stable/zed: Bump SHAs for Zed  https://review.opendev.org/c/openstack/openstack-ansible/+/90534412:51
noonedeadpunk#startmeeting openstack_ansible_meeting15:00
opendevmeetMeeting started Tue Jan 16 15:00:22 2024 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.15:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:00
opendevmeetThe meeting name has been set to 'openstack_ansible_meeting'15:00
noonedeadpunk#topic rollcall 15:00
NeilHanlono/ 15:00
noonedeadpunko/15:00
NeilHanlonmorning15:00
NeilHanlonafternoon, i guess ;) 15:00
noonedeadpunkheya :)15:00
noonedeadpunkit's slightly closer to evening hehe15:00
noonedeadpunk4pm15:01
NeilHanlonah, yeah15:01
NeilHanlonwe've got lightly falling snow outside right now... not looking forward to cleaning it up later15:01
noonedeadpunkwhen I asked locals when they saw snow last time, they answered that it was in 201615:02
noonedeadpunkso I kinda get nostalgic about snow15:03
noonedeadpunk#topic office hours15:03
noonedeadpunkWe had relatively calm week I'd say15:03
noonedeadpunkas a result - no new bugs. And this allowed to merge bumps and propose new point releases15:04
noonedeadpunkexcept 2023.2, which I proposed as bugfix, rest are minor releases15:04
noonedeadpunkAlso I see that reviews started for openstack_resources role: https://review.opendev.org/q/topic:%22osa/openstack_resources%2215:06
noonedeadpunkAlso interestingly, update of ansible version results with quite weird/bad failures: https://review.opendev.org/c/openstack/openstack-ansible/+/90561915:07
noonedeadpunkwith "Could not find a module for {{hostvars['aio1_repo_container-b7922f21']['ansible_facts']['pkg_mgr']}}." during python_venv_build15:08
noonedeadpunkand fails here o_O https://opendev.org/openstack/ansible-role-python_venv_build/src/branch/master/tasks/python_venv_wheel_build.yml#L19-L3215:09
noonedeadpunkhave close to no idea why it fails only for lxc15:10
noonedeadpunkAnother "interesting" topic changes to dynamic_inventory logic: https://review.opendev.org/c/openstack/openstack-ansible/+/905438/215:11
noonedeadpunkbasically there're 2 goals: 1 - name LXC containers in a same way as their hostname as today we have mess with underscores/dashes in name 15:12
NeilHanlonweird...15:13
noonedeadpunkand second to get rid of ANSIBLE_TRANSFORM_INVALID_GROUP_CHARS: https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/openstack-ansible.rc#L6015:14
noonedeadpunkyeah, ansible failure is just crazy....15:14
noonedeadpunkand what's more interesting, it's only for LXC15:14
noonedeadpunkso potentially somehow related to connection plugin changes...15:15
NeilHanlonyeah, could be...15:18
damiandabrowskihey! sorry, I'm late again...15:20
noonedeadpunkno worries15:20
jrossero/ hello15:25
NeilHanlonheya damiandabrowski, jrosser15:25
noonedeadpunkbut eventually I have nothing to add for today.15:31
noonedeadpunkshortly will try to have a look at pki role regarding vault integration15:31
noonedeadpunkbut likely not this week15:31
jrossernothing much from me, i work a lot on the capi patches at the moment to try to make a CI job15:41
noonedeadpunk++15:41
noonedeadpunkyeah, that is really nice thing to have15:41
jrosserneed to get something to pass and merge it quick15:41
noonedeadpunkI'm pretty much looking forward to it as well15:41
jrosseras there are a lot of moving parts and we need CI to give some assurances about bumping versions of collections etc15:42
noonedeadpunkInteresting part, if vexxhost collections do have some "fixed" requirements for dependencies15:43
noonedeadpunkHopefully, they will keep reasonable compatability for these https://github.com/jrosser/ansible-collection-kubernetes/blob/main/galaxy.yml#L11-L1315:45
noonedeadpunkAs recently we had some hassle with bein unable to resolve ansible requirements between osa and another collection15:46
jrosserah well git blame says i fixed those :)15:49
jrosserbecasue of exactly that reason15:49
noonedeadpunkugh, crap, ok, they did excatly what I was afraid of then15:50
jrosseroriginally there was specific versions called for15:50
* noonedeadpunk no idea why one would do that15:50
noonedeadpunkor well.15:50
jrosserand i changed it to be >=15:50
noonedeadpunkI know the reason but no idea how this can be used in a real world15:50
noonedeadpunkLike galaxy.yml should be treated as pyproject.toml (setup.txt)15:51
noonedeadpunkwhere you define minimal requirements, but then you have some requirements.yml with specific ones15:51
noonedeadpunkor dunno...15:54
noonedeadpunk#endmeeting16:00
opendevmeetMeeting ended Tue Jan 16 16:00:54 2024 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:00
opendevmeetMinutes:        https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-01-16-15.00.html16:00
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-01-16-15.00.txt16:00
opendevmeetLog:            https://meetings.opendev.org/meetings/openstack_ansible_meeting/2024/openstack_ansible_meeting.2024-01-16-15.00.log.html16:00
NeilHanlonthanks for running the meeting noonedeadpunk16:01
noonedeadpunkmy pleasure :)16:01
spatelI have a question about how to migrate VMs from openstack cloud to another openstack cloud? 16:51
spatelwhat is the best and less painful way to move VMs or vm's volumes16:52
noonedeadpunkspatel: have you checked https://github.com/os-migrate/os-migrate already?17:02
spatelreading17:05
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Run ceph_client when cinder uses Ceph  https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/90574117:26
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Run ceph_client when cinder uses Ceph  https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/90574117:29
spatelnoonedeadpunk did you use it before?17:45
spatellooks good17:46
admin1spatel, if its linux, rsync is the best option forward :) 17:58
admin1it puts the customer in control, no downtimes ( based on workload ) 17:59
admin1else its a snapshot, download image, upload image, create vm from image17:59
spatelrsync ?17:59
spatelbetween two ceph ?18:00
spatelHow to to volume migration same way, snapshot.. move...upload 18:00
admin1what is the end consumer ? what kind of data is it ? 18:01
admin1mostly its linux, volume mounted  -- something that can rsync with identical setup 18:01
admin1look it as 1 laptop running linux with a usb disk mounted .. u create another latpop with same linux, mount an empty volume and rsync from 1st -> 2nd the data 18:02
admin1this method puts the end-user in control of the situation, how fast./slow he wants to take, what he wants to backup, when he wants the failover to be done18:02
admin1vs if you are a provider and you commit saying "i am going to take care of it " , and then you might face issues or delays and the client is also not happy with downtime, the way its moved/migrated 18:03
admin1mine is not a technical answer .. its more an operational appraoch 18:03
spatelWe don't know what is running inside VM.. end goal is to move entire VM not matter what kind of data running inside. 18:04
spatelI will try with taking snapshot of VM and download snapshot and move it to next new cloud 18:04
noonedeadpunkspatel: really long time ago18:05
admin1you can snapshot and export and import that in diff ceph 18:05
noonedeadpunkwhen it wasn't collection yet, jsut set of roles/modules18:05
noonedeadpunkso it's nice to see it's healthy and evolving18:05
spatelThis doc is written by Redhat and its redhat tool 18:22
spatelwithout doing experiment its not good to do in production :)18:23
spatelnoonedeadpunk very strange.. when I take snapshot its bytes are 0 ?18:24
spatelDid you see this issue?18:24
noonedeadpunkI _think_ it might depend on the backend18:29
noonedeadpunkas for some it counts difference from original volume as size (at least on backend side)18:29

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!