| *** ysandeep|out is now known as ysandeep | 05:01 | |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add experimental Ubuntu 22.04 support https://review.opendev.org/c/openstack/openstack-ansible/+/839483 | 05:14 |
|---|---|---|
| jrosser | morning | 07:45 |
| kleini | morning | 07:45 |
| noonedeadpunk | o/ | 07:53 |
| *** ysandeep is now known as ysandeep|lunch | 08:07 | |
| opendevreview | Merged openstack/openstack-ansible-os_keystone master: Set pki_dir when keystone calls the pki role https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/840445 | 09:22 |
| opendevreview | Merged openstack/openstack-ansible-os_keystone master: Fix certificate installation for keystone https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/840514 | 09:28 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_placement master: Fix incorrect ansible tag https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/840608 | 09:34 |
| *** ysandeep|lunch is now known as ysandeep | 09:36 | |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/xena: Fix extra facts gathering with tags https://review.opendev.org/c/openstack/openstack-ansible/+/840479 | 10:11 |
| kleini | jrosser: I am sorry, as I was not able to follow discussions here for some time. I am on 23.2.0 and still encountering a problem with too many open files in Nova services. It seems to be caused by connections to Rabbit. I applied some upgrade of amqp to 5.0.8 but this did not seem to fix my issues. Was there a solution for that problem somewhere? | 10:27 |
| noonedeadpunk | kleini: there were 2 parts of the issue. First - upgrade amqp. Second, disable heartbeat_in_pthread on computes/neutron/cinder-volumes | 10:45 |
| noonedeadpunk | We have https://review.opendev.org/q/topic:bug%252F1961603 that we didn't backport as this can be solved with regular overrides | 10:45 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Remove legacy vpnaas agent service https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/827251 | 11:02 |
| kleini | noonedeadpunk: many thanks! | 11:03 |
| kleini | i missed this heartbeat_in_pthread part | 11:03 |
| noonedeadpunk | eventually magnum also affected just in case | 11:04 |
| kleini | I am using Magnum but not that much. Will try to add it once I understand the override. Still struggling with the release of our own software and no time left currently to take care about OS deployment. | 11:05 |
| *** dviroel|out is now known as dviroel | 11:21 | |
| *** ysandeep is now known as ysandeep|afk | 11:46 | |
| mgariepy | good morning | 11:47 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Check for requirements file to verify repo health https://review.opendev.org/c/openstack/openstack-ansible/+/840651 | 11:48 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-repo_server master: Remove all code for lsync, rsync and ssh https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837588 | 11:48 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-repo_server master: Clean up legacy lsycnd, rsync and ssh key config https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837859 | 11:49 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-repo_server master: Clean up legacy lsycnd, rsync and ssh key config https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837859 | 11:49 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_aodh master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/824405 | 11:58 |
| mgariepy | can we merge: https://review.opendev.org/q/topic:osa/include_vars+status:open | 11:58 |
| mgariepy | also: https://review.opendev.org/q/topic:osa/common_tasks+status:open (i'll debug aodh once it's checked if it still fails) | 12:00 |
| jrosser | mgariepy: i noticed that we missed placement, and a few others too https://codesearch.opendev.org/?q=-%20import_tasks%3A%20db_setup.yml | 12:02 |
| mgariepy | placement is there ;) | 12:06 |
| mgariepy | i'll check the remaining roles. | 12:08 |
| mgariepy | oops lol | 12:15 |
| mgariepy | sorry didn't read the search correctly hahaha .. brb.. more coffee. | 12:15 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server stable/victoria: Verify if hosts file already managed with OSA https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/836168 | 12:24 |
| opendevreview | Merged openstack/openstack-ansible-tests master: Initialise ssh and ssl certificates https://review.opendev.org/c/openstack/openstack-ansible-tests/+/840428 | 12:28 |
| *** ysandeep|afk is now known as ysandeep | 12:29 | |
| opendevreview | Merged openstack/openstack-ansible master: Update compatibility matrix for Yoga release https://review.opendev.org/c/openstack/openstack-ansible/+/840458 | 12:35 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Add role for creating simple glusterfs servers https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/837582 | 12:38 |
| opendevreview | Merged openstack/openstack-ansible master: Update documented OS support for Ubuntu 22.04 https://review.opendev.org/c/openstack/openstack-ansible/+/840462 | 12:39 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_adjutant master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/840659 | 12:50 |
| jrosser | is anyone interested in looking at skyline? | 12:50 |
| opendevreview | Merged openstack/openstack-ansible-tests master: Do not filter setuptools from upper-constraints https://review.opendev.org/c/openstack/openstack-ansible-tests/+/783747 | 12:52 |
| mgariepy | spatel was looking at it irrc | 12:55 |
| jrosser | yeah, i think he did some not-very-OSA-hack | 12:56 |
| jrosser | i put together the skeleton of an os_skyline ansible role, but it needs someone who wants to invest time in it really | 12:57 |
| mgariepy | how much work does it need ? | 12:57 |
| jrosser | there are things like missing SSO which are a show stopper for me, but if we don't start then we never get anywhere | 12:57 |
| mgariepy | yeah sso i saw the ML post. did the issue got created ? | 12:58 |
| jrosser | well, even though they used some totally unusual approach to the python, it seems that python_venv_build kind of 'just works' to install the skyline-apiserver in a venv | 12:58 |
| mgariepy | lol. when spreaking of the wolf :) | 12:59 |
| jrosser | though it's obvious they are not using any kind of upper-constraints so the different tools they have don't all work at the same time | 12:59 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_gnocchi master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_gnocchi/+/840662 | 13:01 |
| spatel | jrosser noonedeadpunk any idea what is going on here - https://paste.opendev.org/show/b1u5yN0QiYT0md2wOEff/ | 13:03 |
| spatel | building new production cloud using 23.2.2 wallaby | 13:03 |
| spatel | may be rate-limit on opendev? | 13:04 |
| spatel | I am going to re-run playbook and see | 13:04 |
| noonedeadpunk | not really. | 13:04 |
| noonedeadpunk | have you disable building wheels? | 13:05 |
| jrosser | spatel: which playbook? | 13:05 |
| spatel | setup-openstack.yml | 13:05 |
| spatel | deploying brand new cloud | 13:06 |
| noonedeadpunk | like with venv_wheel_build_enable=false? | 13:06 |
| noonedeadpunk | I guess no.... | 13:06 |
| jrosser | spatel: let me try again :) where in the deployment has it failed? rally? | 13:07 |
| noonedeadpunk | oh, rally would explain that :) | 13:08 |
| noonedeadpunk | haven't we fixed that though? | 13:09 |
| jrosser | well i'm not sure | 13:09 |
| noonedeadpunk | ok, worth taking a look | 13:11 |
| jrosser | rally is like a total corner case somehow | 13:12 |
| noonedeadpunk | I wonder if we can just pass venv_wheel_build_skip_check from there at worst | 13:13 |
| spatel | sorry i was on other call, let me check logs and find some clue | 13:13 |
| noonedeadpunk | or add serial there... or target as tempest only against 1 utility container | 13:14 |
| jrosser | or else make this only install to the first host in the utility group https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-rally-install.yml#L23 | 13:14 |
| noonedeadpunk | I bet we even agreed on that one day :) | 13:14 |
| jrosser | i'm sure :) | 13:14 |
| jrosser | lets do it! | 13:14 |
| spatel | Question why are we running rally? assuming to test something but at present i don't have any compute node in cloud. just building infra node only | 13:20 |
| opendevreview | Merged openstack/ansible-role-systemd_mount master: Add support for glusterfs mounts https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/837750 | 13:27 |
| jrosser | spatel: it is not running it, it is installing it.... | 13:27 |
| spatel | let me run - openstack-ansible os-tempest-install.yml | 13:28 |
| spatel | hmm - https://paste.opendev.org/show/bLbx8WYWwTWxoHJLerO0/ | 13:29 |
| spatel | did i miss something here? | 13:29 |
| jrosser | spatel: do some simple debugging, it didnt run against any hosts | 13:32 |
| jrosser | so look at what it should have run against https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-tempest-install.yml#L27 | 13:32 |
| jrosser | you've got --list-hosts and scripts/inventory_manage to help you | 13:32 |
| opendevreview | Merged openstack/openstack-ansible-os_mistral master: Remove legacy policy.json cleanup handler https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/827448 | 13:33 |
| spatel | jrosser i can see all host in inventory including utility | 13:34 |
| spatel | all playbooks ran successfully so far except that utility or tempest | 13:35 |
| spatel | https://paste.opendev.org/show/b79VSN32iwNhrB6FpDmX/ | 13:36 |
| noonedeadpunk | I guess it should be inventory-manage.py -G ? | 13:38 |
| noonedeadpunk | as it runs rally if group for it is defined | 13:38 |
| noonedeadpunk | and not empty ) | 13:39 |
| spatel | https://paste.opendev.org/show/bRLT6BLX6DzhOQ2KEcem/ | 13:39 |
| opendevreview | Merged openstack/openstack-ansible-os_mistral master: Refactor use of include_vars https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/824325 | 13:39 |
| jrosser | utility_all is missing? | 13:40 |
| spatel | uh? | 13:40 |
| jrosser | i do not see the group utility_all in your paste | 13:40 |
| noonedeadpunk | hm | 13:41 |
| jrosser | and that is what the playbook runs against, what i link you to before | 13:41 |
| noonedeadpunk | do we _always_ install rally? | 13:41 |
| jrosser | yes i think we do | 13:41 |
| spatel | there is not utility_all in other production cloud | 13:41 |
| spatel | how did that work? | 13:41 |
| noonedeadpunk | I'd say we should do smth like https://github.com/openstack/openstack-ansible/blob/master/playbooks/os-tempest-install.yml#L35-L38 | 13:41 |
| opendevreview | Merged openstack/openstack-ansible-os_cloudkitty master: Remove legacy policy.json cleanup handler https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/827436 | 13:42 |
| opendevreview | Merged openstack/openstack-ansible-repo_server master: Add facility to store repo contents on a remote mount https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/837587 | 13:42 |
| opendevreview | Merged openstack/openstack-ansible-os_cloudkitty master: Refactor use of include_vars https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/824336 | 13:42 |
| spatel | jrosser i cross check inventory with other cloud and none has utility_all group | 13:43 |
| spatel | noonedeadpunk i like the idea to not run rally playbook default. Let end users decide if they wants or not. | 13:45 |
| noonedeadpunk | yeah | 13:46 |
| noonedeadpunk | I just realized it's not current;y the case:) | 13:46 |
| jrosser | we need a way to turn it on and off | 13:47 |
| jrosser | it doesnt have the same concept of env.d / conf.d as other things | 13:47 |
| noonedeadpunk | I was thinking if it worth to set a separate env.d file for tempest/rally? | 13:47 |
| jrosser | would it be an extenstion of the utility container | 13:48 |
| jrosser | as thats really the only place they can go, i think? | 13:48 |
| noonedeadpunk | I think they can be anywhere? | 13:48 |
| noonedeadpunk | I mean - it could be smth outside of controllers even | 13:48 |
| noonedeadpunk | Nothing stops you from running rally from outside of the cluster | 13:49 |
| jrosser | it makes more sense to do that really | 13:49 |
| jrosser | as it should see the cluster as if it were a user, rather than being inside | 13:49 |
| noonedeadpunk | To check how apis are responsive through public endpoints | 13:49 |
| noonedeadpunk | yeah | 13:49 |
| jrosser | so that suggests that we should expand env.d stuff to cover rally being installed anywhere | 13:50 |
| noonedeadpunk | The only question is if we should share group for rally and tempest... | 13:50 |
| jrosser | and what we want to happen in CI too | 13:51 |
| jrosser | as we always want tempest there | 13:51 |
| noonedeadpunk | In CI it make sense to set utility_all[0] | 13:51 |
| noonedeadpunk | But we can jsut have conf.d for aio for that | 13:51 |
| opendevreview | Merged openstack/openstack-ansible-os_placement master: Fix incorrect ansible tag https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/840608 | 13:51 |
| noonedeadpunk | I think I'd share groups for tempest/rally.... | 13:52 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_placement master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/840668 | 13:53 |
| opendevreview | Merged openstack/openstack-ansible-os_mistral master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/824383 | 13:55 |
| spatel | I like where this conversation going :) Let me not finish my debugging here and not run rally at this point | 13:56 |
| noonedeadpunk | Ok, have idea :) | 13:56 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_rally master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_rally/+/840669 | 13:57 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_sahara master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_sahara/+/840672 | 14:01 |
| mgariepy | jrosser, i think it's all :d | 14:02 |
| opendevreview | Merged openstack/openstack-ansible-os_cloudkitty master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/824399 | 14:03 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_adjutant master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/840659 | 14:19 |
| noonedeadpunk | hm.... | 14:31 |
| mgariepy | grr my typing this morning :/ common-tasks vs common_tasks .. | 14:52 |
| spatel | jrosser noonedeadpunk is this keepalived bug or what? -> https://paste.opendev.org/show/beK4cVJWVBGgQC5V3RFH/ | 14:54 |
| spatel | external_lb_vip_address should get convert into ip address | 14:55 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Create containers for rally/tempest independently https://review.opendev.org/c/openstack/openstack-ansible/+/840685 | 14:55 |
| jrosser | spatel: it does what the code says? https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/haproxy/keepalived.yml#L58 | 14:59 |
| noonedeadpunk | I guess it's L64 though | 15:00 |
| spatel | jrosser problem is if i change external_lb_vip_address: to ip address then all my endpoint set ip address instead FQDN and SSL failed | 15:01 |
| noonedeadpunk | but if `external_lb_vip_address` is FQDN, then haproxy_keepalived_external_vip_cidr should be an IP, and not "{{external_lb_vip_address}}/32" | 15:01 |
| noonedeadpunk | quote from your paste: The below domain name must resolve to an IP address in the CIDR specified in haproxy_keepalived_external_vip_cidr. | 15:01 |
| spatel | I have resolvable name so it should work | 15:01 |
| noonedeadpunk | So haproxy_keepalived_external_vip_cidr must be a cidr regardless of what's set in external_lb_vip_address | 15:02 |
| noonedeadpunk | you can do `haproxy_keepalived_external_vip_cidr: "{{external_lb_vip_address}}/32"` only if external_lb_vip_address is IP | 15:02 |
| spatel | I got your point but just thinking why logic is not taking FQDN name and convert into ipaddr ? | 15:04 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Install rally only when rally_install is set https://review.opendev.org/c/openstack/openstack-ansible/+/840686 | 15:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Create containers for rally/tempest independently https://review.opendev.org/c/openstack/openstack-ansible/+/840685 | 15:10 |
| noonedeadpunk | spatel: well, I'd say it's doable thing, but needs extra logic/complexity for smth that likely not worth it. | 15:12 |
| noonedeadpunk | as you need to identify if it's ipv4/ipv6 and the resolve it instead of just place the variable that has been provided by deployer | 15:13 |
| spatel | +1 | 15:13 |
| noonedeadpunk | and well, it can resolve to smth like cloudflare for instance... | 15:13 |
| noonedeadpunk | resolving is tricky I'd say | 15:14 |
| noonedeadpunk | and then if you pass domain.name/32 I'm not sure it will be detected as fqdn... | 15:15 |
| noonedeadpunk | so you need to remove netmask, and then re-add it | 15:15 |
| spatel | This is what i did to get going and it works - haproxy_keepalived_external_vip_cidr: "10.72.0.5/32" | 15:18 |
| *** dviroel is now known as dviroel|lunch | 15:53 | |
| *** ysandeep is now known as ysandeep|out | 16:28 | |
| mgariepy | aodh : requirement ceilometer (unavailable) https://zuul.opendev.org/t/openstack/build/310c0128a557412bbf3117c6d1060be3/log/logs/host/python_venv_build.log.txt#41089 | 16:38 |
| mgariepy | on both metal-distro jobs | 16:39 |
| mgariepy | wtf.. why ? https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_310/824405/4/check/openstack-ansible-deploy-aio_distro_metal-ubuntu-focal/310c012/logs/ara-report/results/1598.html | 16:51 |
| *** dviroel|lunch is now known as dviroel | 17:01 | |
| jrosser | i wonder if there is no packaging for that in UCA, or somesuch | 17:05 |
| jrosser | oh hello https://paste.opendev.org/show/bVfNSRGKlPIuUMNe9ULd/ | 17:06 |
| opendevreview | Merged openstack/openstack-ansible-os_placement master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_placement/+/840668 | 17:51 |
| mgariepy | hmm gnochi doen't support distro install :/ | 17:52 |
| mgariepy | help ! | 18:49 |
| mgariepy | https://zuul.opendev.org/t/openstack/build/d7d4032ecf384f1c896eac2da168506b/log/job-output.txt#202 | 18:49 |
| mgariepy | the issue is that ceilometer is not cloned on job start when testing distro install.. | 18:50 |
| mgariepy | so we end up with that : https://paste.openstack.org/show/bM8HZiXjcBiPBHVExs81/ | 18:51 |
| mgariepy | i'm not sure where it needs to be adjusted. | 18:56 |
| jrosser | why do we need the repo for a distro install? | 18:57 |
| mgariepy | cause it doesnt support distro install. | 18:57 |
| mgariepy | https://github.com/openstack/openstack-ansible-os_gnocchi/tree/master/vars | 18:58 |
| mgariepy | or we should maybe add distro support to the role ? | 19:02 |
| mgariepy | or remove the distro test for it.. | 19:03 |
| jrosser | I expect we never had coverage everywhere for distro installs | 19:04 |
| jrosser | and then slowly adding things to the central project templates can accidentally add the jobs where they’re not needed | 19:05 |
| mgariepy | hmm it was passing before.. | 19:05 |
| jrosser | we did refactor in openstack ansible jobs definitions | 19:06 |
| jrosser | there is required_projects to look at in the base jobs | 19:06 |
| jrosser | that where the list of needed repos is set | 19:06 |
| mgariepy | https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/834845 | 19:07 |
| mgariepy | hmm i'm getting confused lol | 19:10 |
| mgariepy | gnocchi was installed via venv then. just like now but ceilometer for some reason now do go via source install instead of distro. | 19:11 |
| jrosser | oh - didn’t we just merge a patch tidying up a bunch of vars? | 19:16 |
| jrosser | https://review.opendev.org/c/openstack/openstack-ansible/+/837845 | 19:17 |
| jrosser | I wonder if there is something broken now | 19:17 |
| mgariepy | https://codesearch.opendev.org/?q=service_install_method&i=nope&literal=nope&files=&excludeFiles=&repos= | 19:25 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_ceilometer master: Use global service variables https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/840724 | 19:28 |
| mgariepy | https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/769400 | 19:29 |
| mgariepy | fun,. | 19:29 |
| mgariepy | talk to you tomorrow. | 19:29 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_aodh master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/824405 | 19:30 |
| mgariepy | let's see how it goes. | 19:30 |
| opendevreview | Merged openstack/openstack-ansible-os_adjutant master: Use common service setup tasks from a collection rather than in-role https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/840659 | 19:32 |
| mgariepy | it's hard to have it consistent accross all repos.. | 19:36 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_murano master: use {{ service_install_method }} https://review.opendev.org/c/openstack/openstack-ansible-os_murano/+/840729 | 19:45 |
| opendevreview | Marc Gariépy proposed openstack/openstack-ansible-os_tempest master: use {{ service_install_method }} https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/840750 | 19:46 |
| *** dviroel is now known as dviroel|out | 20:43 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!