| damiandabrowski | hi! | 08:11 |
|---|---|---|
| jrosser | hello | 08:42 |
| jrosser | damiandabrowski: i have a stack of patches here that you should look at - first adding a CI job for LE/certbot, and second introducing haproxy map files https://review.opendev.org/c/openstack/openstack-ansible/+/876851 | 09:24 |
| jrosser | i think that the map files will greatly simplify the separated config work | 09:25 |
| noonedeadpunk | mornings | 09:38 |
| noonedeadpunk | Sorry was offline yestarday | 09:39 |
| noonedeadpunk | yeah. this nova thing is weird.... | 09:49 |
| jrosser | noonedeadpunk: adding a compute node? | 09:50 |
| jrosser | btw i just look at our log collection a bit - we miss anything thats a symlink | 09:51 |
| jrosser | rsync can't deal with that as symlinks in containers point to files on the host from the log collection script POV | 09:52 |
| noonedeadpunk | jrosser: nah, travelling to Sarajevo to visa center for the summit | 09:52 |
| jrosser | oh no the wierd nova thing :) | 09:53 |
| noonedeadpunk | If we drop qemu-system, I assume we might need to install qemu-system-misc as an extra, but not very sure | 09:54 |
| noonedeadpunk | But for the rest I don't think we need qemu-system indeed | 09:55 |
| jrosser | ah that nova thing - sorry | 09:55 |
| noonedeadpunk | Likely it was due to some regression | 09:55 |
| noonedeadpunk | lol | 09:55 |
| jrosser | so i found that extremely strange things happen if you add an arm node | 09:55 |
| noonedeadpunk | You meant _that_ nova thing :D | 09:55 |
| noonedeadpunk | I still didn't dare to touch yet | 09:55 |
| jrosser | like it just disregards architecture totally in nova scheduler | 09:55 |
| noonedeadpunk | yeah, because anyone can provide you arm | 09:56 |
| jrosser | oh well it booted arm vm on an x86 node under emulation which was also a surprise | 09:56 |
| noonedeadpunk | I would expect having smth in compute config to disable some arches from being scheduled there, but likely it's not the case | 09:56 |
| jrosser | then it also booted x86 on arm nodes which ends badly stuck in the uefi shell | 09:57 |
| jrosser | lots of digging around why hw_architecture property on images not working - first that filter is disabled by default..... | 09:58 |
| noonedeadpunk | have you played with this? https://docs.openstack.org/nova/latest/configuration/config.html#filter_scheduler.image_properties_default_architecture | 09:58 |
| jrosser | and then there is nothing adding HW_ARCH_* trait to compute nodes to allow the filter to decide what to do | 09:58 |
| jrosser | well all that relies on the trait being present | 09:58 |
| noonedeadpunk | yeah, true | 09:59 |
| jrosser | i guess i just expected a lot of this to work out of the box | 09:59 |
| jrosser | but really it doesnt and only by reading the nova code can it be worked out whats happening | 09:59 |
| jrosser | the nova docs suggest that the multi architecture support is more present than actually it is | 10:00 |
| noonedeadpunk | well, you can spawn VMs on aarch64 :D | 10:01 |
| noonedeadpunk | Amnd removing qemu-system-arm does stop VMs from spawning, or nova still attempts to spawn there but fails? | 10:02 |
| jrosser | otoh good news is that nothing really broken in OSA for this :) | 10:02 |
| jrosser | unclear - so many moving parts that checking them all takes a long time | 10:02 |
| noonedeadpunk | ++ | 10:02 |
| damiandabrowski | jrosser: thanks, I will have a look at these patches | 10:03 |
| jrosser | damiandabrowski: those patches are all ones that would happen before your changes | 10:08 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Refactor log collection from lxc containers https://review.opendev.org/c/openstack/openstack-ansible/+/877382 | 14:46 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Refactor log collection from lxc containers https://review.opendev.org/c/openstack/openstack-ansible/+/877382 | 14:47 |
| noonedeadpunk | I should fix condition for dstat run again... | 14:59 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:01 |
| opendevmeet | Meeting started Tue Mar 14 15:01:27 2023 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:01 |
| noonedeadpunk | #topic rollcall | 15:01 |
| noonedeadpunk | o/ | 15:01 |
| damiandabrowski | hi! | 15:01 |
| jrosser | o/ hello | 15:02 |
| noonedeadpunk | #topic office hours | 15:03 |
| noonedeadpunk | First I'd love to ask for some reviews of backports to Xena https://review.opendev.org/q/parentproject:openstack/openstack-ansible+branch:%255Estable/xena+status:open | 15:03 |
| noonedeadpunk | unfortunatelly Zed and Yoga bumps didn't passed yet | 15:04 |
| noonedeadpunk | As agreed, I've set upgrade jobs to NV from W to X | 15:05 |
| damiandabrowski | okok, i'll spent some time on reviews today and tomorrow | 15:05 |
| noonedeadpunk | Regarding upcoming PTG - I've just booked the room for us | 15:08 |
| noonedeadpunk | #link https://ptg.opendev.org/ptg.html | 15:08 |
| noonedeadpunk | I will send ML with info right after the meeting | 15:09 |
| noonedeadpunk | Regarding our haproxy topic - I didn't have chance to iterate on that or review map patches thoroughly but they looked quite good to me and made total sense | 15:10 |
| noonedeadpunk | So I overall had not much time last week. | 15:13 |
| jrosser | i made a patch for the mysql reduced memory settings | 15:14 |
| jrosser | no idea if it worked :) | 15:14 |
| noonedeadpunk | yeah.... | 15:14 |
| jrosser | then got down the rabbit hole of why there is no log for my.cnf | 15:15 |
| noonedeadpunk | dstat is ot running due to some my mistake... | 15:15 |
| noonedeadpunk | For some reason each change I made either makes it run every time, even outside of CI or do not run at all... | 15:15 |
| noonedeadpunk | but I don't have pretty much to say to be frank - a lot of work with haproxy that we should land at least... | 15:24 |
| jrosser | so for haproxy it sounds again like we need to revisit the approach to agree what we do next | 15:24 |
| jrosser | becasue if there is some agreement we could start to merge the things which don't change master | 15:25 |
| jrosser | *change the behaviour | 15:25 |
| jrosser | unfortunately rebasing the previous patches on top of my new work is going to be pretty difficult | 15:26 |
| damiandabrowski | "so for haproxy it sounds again like we need to revisit the approach to agree what we do next" - are you talking about "temporary groups" vs. classic loop or something else? | 15:28 |
| damiandabrowski | sorry it's my first day after vacation so I'm still catching up | 15:28 |
| noonedeadpunk | It's about map support? | 15:29 |
| jrosser | i mean that the map support i did should take away a bunch of complexity from the current separated config patches | 15:30 |
| jrosser | but they will need rebasing in order to do that (and ideally splitting further into smaller patches) | 15:30 |
| jrosser | i saw that currently one giant patch combines separating the config and also putting in the hooks to enable https backends | 15:30 |
| noonedeadpunk | Well, splitting might be tought there | 15:30 |
| noonedeadpunk | well, yes, that part can be :) | 15:31 |
| jrosser | and https backend for repo server changes is currently breaking the splitting out part | 15:31 |
| noonedeadpunk | But it still will be gigantic as we need to move things to playbooks | 15:31 |
| damiandabrowski | jrosser: that's right, map support will simplify my patches | 15:31 |
| noonedeadpunk | and from group_var | 15:31 |
| noonedeadpunk | I;d propose to leave repo_server alone with https coverage right now | 15:32 |
| damiandabrowski | not sure if we can split changes even more but at least I will try | 15:32 |
| jrosser | and the LE CI job feels maybe heavyweight way to validate that | 15:32 |
| jrosser | so i was not sure if that was a good idea or not, but we have no test coverage of that at all and pretty good chance to break it | 15:32 |
| damiandabrowski | noonedeadpunk: with or without https coverage? | 15:33 |
| damiandabrowski | regarding repo_server, I've described the issue before my vacation: https://review.opendev.org/c/openstack/openstack-ansible/+/876426/comments/8cde87f7_169011a5 | 15:33 |
| damiandabrowski | i'll dig into this tomorrow | 15:33 |
| jrosser | yes but there is no reason at all to bring new https backends in with the same patch that splits out the config | 15:34 |
| noonedeadpunk | damiandabrowski: without for now, as it's not really required and we don't have repo frontend covered with TLS as of today | 15:34 |
| noonedeadpunk | So I'd leave repo_server after we do all major changes and cover services | 15:34 |
| damiandabrowski | so maybe do it like this: if I won't be able to get it working tomorrow, I'll move this change to the top of relation chain, ok? | 15:35 |
| jrosser | please just move around the existing config blocks for the services without changing them | 15:35 |
| noonedeadpunk | ++ | 15:36 |
| noonedeadpunk | and all changes do in separate pachsets | 15:36 |
| noonedeadpunk | So let's focus on 1. adopting maps 2. splitting configs to respect group_vars and service configurations on playbook level | 15:37 |
| noonedeadpunk | then we will be able to look into tls with more narrow scope | 15:38 |
| damiandabrowski | "splitting configs to respect group_vars and service configurations on playbook level" what config are we going to split? | 15:38 |
| damiandabrowski | configs* | 15:38 |
| noonedeadpunk | service configruation. Meaning - moving them from group_vars/haproxy to group_vars/service as you did | 15:39 |
| noonedeadpunk | But not changing them at the same patch | 15:39 |
| damiandabrowski | ah ok, got it | 15:39 |
| noonedeadpunk | and including common task. basically https://review.opendev.org/c/openstack/openstack-ansible/+/871189/ | 15:40 |
| jrosser | and on top of this https://review.opendev.org/c/openstack/openstack-ansible/+/876851 | 15:40 |
| noonedeadpunk | But do changes to backends as follow-up, to be able to see what we're chaning, as otherwise it's just all green | 15:40 |
| noonedeadpunk | regarding https://review.opendev.org/c/openstack/openstack-ansible/+/876639 - should we add the same job to haproxy as well? Or you did that already? | 15:42 |
| jrosser | i added it to `openstack-ansible-deploy-infra_lxc-jobs` | 15:44 |
| jrosser | ok yes that should run for haproxy too | 15:44 |
| noonedeadpunk | aha, yes, you're right here | 15:46 |
| noonedeadpunk | I was just thinking about different condition here https://review.opendev.org/c/openstack/openstack-ansible/+/876637/6/tests/roles/bootstrap-host/tasks/main.yml | 15:47 |
| noonedeadpunk | like "'stepca' in bootstrap_host_scenarios_expanded or 'haproxy' in bootstrap_host_scenarios_expanded" | 15:47 |
| noonedeadpunk | but yeah, adding to infra_lxc maybe even better | 15:48 |
| noonedeadpunk | as another job | 15:48 |
| noonedeadpunk | it's quite awkward that we need to generate self-signed to install let's encrypt.... | 15:50 |
| noonedeadpunk | anyway | 15:50 |
| noonedeadpunk | That looks quite good to me. Are these stepca series waiting for anything before being ready for review? | 15:51 |
| noonedeadpunk | oh, well. except missing rpm part :) | 15:54 |
| noonedeadpunk | seems that they do have packages | 15:55 |
| jrosser | no i think the stepca patches are reasonable and can be reviewed | 15:58 |
| jrosser | for a real role to install it you'd want more idempotency but i was thinking not to make it too complicated just for AIO purposes | 15:58 |
| jrosser | 'haproxy' in bootstrap_host_scenarios_expanded" would run it for all haproxy jobs so i think thats not what we want | 15:59 |
| noonedeadpunk | I don't think bootstrap-aio is idemptent as of today | 15:59 |
| jrosser | as also need to test the normal code path as well | 15:59 |
| noonedeadpunk | yeah, I"ve realized that after wrote | 16:00 |
| noonedeadpunk | #endmeeting | 16:04 |
| opendevmeet | Meeting ended Tue Mar 14 16:04:23 2023 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:04 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-03-14-15.01.html | 16:04 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-03-14-15.01.txt | 16:04 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-03-14-15.01.log.html | 16:04 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Deploy step-ca when 'stepca' is part of the deployment scenario. https://review.opendev.org/c/openstack/openstack-ansible/+/876637 | 16:51 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add a /etc/hosts entry for the external IP of an AIO https://review.opendev.org/c/openstack/openstack-ansible/+/876638 | 16:51 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Use certbot to generate SSL cert for the external VIP in 'stepca' scenario https://review.opendev.org/c/openstack/openstack-ansible/+/876639 | 16:52 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Use a map file to select haproxy horizon backend from the base frontend https://review.opendev.org/c/openstack/openstack-ansible/+/876851 | 16:52 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server stable/xena: Restore integrated jobs https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/876748 | 16:56 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server stable/xena: Fix tags usage for letsencrypt setup https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/876683 | 17:28 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server stable/xena: Serialise initial issuing of LetsEncrypt certificates https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/876686 | 17:28 |
| opendevreview | Merged openstack/openstack-ansible-os_ironic master: Update ironic documentation https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/867547 | 17:49 |
| opendevreview | Merged openstack/openstack-ansible stable/xena: Fix comment typo in nova install playbook https://review.opendev.org/c/openstack/openstack-ansible/+/876680 | 17:50 |
| jrosser | noonedeadpunk: https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/867547 is also applicable to Zed - we can backport if you think thats useful | 17:52 |
| opendevreview | Merged openstack/openstack-ansible-ops master: Ensure python3-pexpect is installed on Ubuntu Bionic https://review.opendev.org/c/openstack/openstack-ansible-ops/+/876870 | 17:52 |
| noonedeadpunk | sure, see no reason not to backport | 17:52 |
| jrosser | we also merged stuff for ironic after Zed which would simplify the config a bit so i will make a followup soon | 17:53 |
| opendevreview | Merged openstack/openstack-ansible-ops master: Update beat version for latest release of ELK7 https://review.opendev.org/c/openstack/openstack-ansible-ops/+/876855 | 17:54 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic stable/zed: Update ironic documentation https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/877393 | 17:55 |
| opendevreview | Merged openstack/openstack-ansible-ops master: Collect hardware facts for elastic data nodes https://review.opendev.org/c/openstack/openstack-ansible-ops/+/876852 | 17:56 |
| opendevreview | Merged openstack/openstack-ansible-ops master: Add zuul jobs for elk metrics on ubuntu jammy https://review.opendev.org/c/openstack/openstack-ansible-ops/+/877231 | 17:56 |
| jrosser | i moved the whole stack out of WIP here so reviews welcome https://review.opendev.org/c/openstack/openstack-ansible/+/876851 | 18:56 |
| Ultra | Hey folks, can someone help me debug that issue with `os-rally-install.yml` playbook? Error: https://paste.openstack.org/show/bZs4NUcAcNxzZEuJCoJv/ | 19:51 |
| *** Ultra is now known as ThiagoCMC | 19:56 | |
| opendevreview | Merged openstack/openstack-ansible stable/zed: Bump OpenStack-Ansible Zed https://review.opendev.org/c/openstack/openstack-ansible/+/876028 | 19:58 |
| *** dviroel_ is now known as dviroel | 20:21 | |
| spatel | in keystone endpoint can't i add one more extra endpoint like Interface public2 | 20:59 |
| spatel | can't / can I ? | 20:59 |
| spatel | I want to expose openstack on public endpoint. in short assign public IP on one of interface on haproxy and add that ip/fqdn on keystone | 21:00 |
| jrosser | spatel: do you already use the keystone public endpoint for something else? | 21:30 |
| spatel | I have private address on public endpoint (10.x.x.x range) | 21:30 |
| jrosser | trouble is the useful tools like cli and terraform/ whatever read the contents of the service catalog | 21:31 |
| spatel | Planning to add real public IP on endpoint without downtime so only solution is to setup HAProxy but how do i tell keystone about new public2 endpoint ? | 21:31 |
| spatel | You are correct, company planning to use Terrafrom cloud service which is hosted on public cloud and trying to access openstack but our openstack is running on internal address range. | 21:32 |
| spatel | I can setup ngnix etc.. but it required all endpoint to expose to make connection | 21:33 |
| opendevreview | Merged openstack/openstack-ansible master: Switch proxy job from focal to jammy https://review.opendev.org/c/openstack/openstack-ansible/+/877234 | 21:59 |
| opendevreview | Merged openstack/openstack-ansible stable/yoga: Fix comment typo in nova install playbook https://review.opendev.org/c/openstack/openstack-ansible/+/876679 | 22:19 |
| opendevreview | Merged openstack/openstack-ansible master: Add releasenote for ANSIBLE_INJECT_FACT_VARS defaulting to false https://review.opendev.org/c/openstack/openstack-ansible/+/876764 | 22:19 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-repo_server master: Add TLS support to repo_server backends https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/876429 | 22:37 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible-repo_server master: Turn off absolute_redirect for nginx https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/877429 | 22:37 |
| opendevreview | Damian Dąbrowski proposed openstack/openstack-ansible master: Enable TLS frontend for repo_server by default https://review.opendev.org/c/openstack/openstack-ansible/+/876426 | 22:40 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!