| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job https://review.opendev.org/c/openstack/kolla-ansible/+/905322 | 06:11 |
|---|---|---|
| *** mmalchuk_ is now known as mmalchuk | 06:13 | |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job https://review.opendev.org/c/openstack/kolla-ansible/+/905322 | 06:13 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job https://review.opendev.org/c/openstack/kolla-ansible/+/905322 | 07:01 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.1: CI: Test Nova server resize functionality https://review.opendev.org/c/openstack/kolla-ansible/+/904283 | 07:48 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: CI: Test Nova server resize functionality https://review.opendev.org/c/openstack/kolla-ansible/+/904284 | 08:02 |
| opendevreview | Merged openstack/kolla-ansible stable/yoga: CI: Test Nova server resize functionality https://review.opendev.org/c/openstack/kolla-ansible/+/904285 | 08:02 |
| opendevreview | Merged openstack/kolla stable/2023.2: CI: Use newer podman/buildah on Ubuntu Jammy https://review.opendev.org/c/openstack/kolla/+/904731 | 09:07 |
| opendevreview | Jake Hutchinson proposed openstack/kolla-ansible master: Ironic parameter rework and default NTP server https://review.opendev.org/c/openstack/kolla-ansible/+/893031 | 10:26 |
| opendevreview | Merged openstack/kayobe master: Update python classifier in setup.cfg https://review.opendev.org/c/openstack/kayobe/+/905056 | 11:06 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: openvswitch: Stop using intermediate scripts https://review.opendev.org/c/openstack/kolla-ansible/+/905117 | 11:08 |
| kevko | \o | 11:09 |
| SvenKieske | o/ | 11:14 |
| SvenKieske | just ripped out my ipv6 support because review.opendev.org doesn't really seem to like IPv6 currently | 11:14 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: haproxy: Use -f configdir instead of for and xargs https://review.opendev.org/c/openstack/kolla-ansible/+/905121 | 11:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb-clustercheck: Use socat wrapper https://review.opendev.org/c/openstack/kolla-ansible/+/905131 | 11:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: openvswitch: Stop using intermediate scripts https://review.opendev.org/c/openstack/kolla-ansible/+/905117 | 11:18 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: fail when systemd unit fails to stop https://review.opendev.org/c/openstack/kolla-ansible/+/891781 | 11:18 |
| opendevreview | Merged openstack/kolla-ansible master: Fix trove failed to discover swift endpoint https://review.opendev.org/c/openstack/kolla-ansible/+/904968 | 11:41 |
| opendevreview | Merged openstack/kolla stable/2023.2: rabbitmq: Use timeout in healthcheck script https://review.opendev.org/c/openstack/kolla/+/904519 | 12:01 |
| opendevreview | Merged openstack/kolla stable/2023.1: rabbitmq: Use timeout in healthcheck script https://review.opendev.org/c/openstack/kolla/+/904520 | 12:01 |
| opendevreview | Merged openstack/kolla stable/zed: rabbitmq: Use timeout in healthcheck script https://review.opendev.org/c/openstack/kolla/+/904721 | 12:07 |
| opendevreview | Merged openstack/kolla stable/yoga: rabbitmq: Use timeout in healthcheck script https://review.opendev.org/c/openstack/kolla/+/904722 | 12:07 |
| opendevreview | Verification of a change to openstack/kolla master failed: Fix openstack CADF audit maps and installation https://review.opendev.org/c/openstack/kolla/+/904576 | 12:12 |
| wncslln | hi o/, anyone can help me how to apply a patch in a kolla-ansible deployment, please? i need to apply a patch with some Nova changes to test a feature | 13:28 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.2: Enable glance proxying behaviour https://review.opendev.org/c/openstack/kolla-ansible/+/904850 | 13:52 |
| kevko | wncslln: just apply a patch :D | 14:01 |
| kevko | SvenKieske: most of people don't like IPv6 | 14:03 |
| kevko | :D | 14:03 |
| wncslln | kevko: apply to repository before the deploy? if not, since nova is a container, i wont need to rebuild it after apply the patch? | 14:13 |
| opendevreview | Verification of a change to openstack/kolla master failed: Fix openstack CADF audit maps and installation https://review.opendev.org/c/openstack/kolla/+/904576 | 14:15 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.1: Enable glance proxying behaviour https://review.opendev.org/c/openstack/kolla-ansible/+/904851 | 14:15 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: Enable glance proxying behaviour https://review.opendev.org/c/openstack/kolla-ansible/+/904852 | 14:15 |
| opendevreview | Merged openstack/kolla-ansible stable/yoga: Enable glance proxying behaviour https://review.opendev.org/c/openstack/kolla-ansible/+/904853 | 14:15 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.2: Remove nova cell sync comment https://review.opendev.org/c/openstack/kolla-ansible/+/904725 | 14:15 |
| opendevreview | Merged openstack/ansible-collection-kolla master: Use py3 as the default runtime for tox https://review.opendev.org/c/openstack/ansible-collection-kolla/+/890748 | 14:15 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.2: cadvisor: Set housekeeping interval to Prometheus scrape interval https://review.opendev.org/c/openstack/kolla-ansible/+/904842 | 14:15 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: cadvisor: Set housekeeping interval to Prometheus scrape interval https://review.opendev.org/c/openstack/kolla-ansible/+/904844 | 14:15 |
| r3ap3r | I don't see Hudson around so I figured I'd ask in "general", has anyone attempted Kolla-Ansible deployment with Cephadm in a hyperconverged setup? We may be attempting that kind of setup in the near future and was curious if anyone had already attempted it and what their experience was. I feel like there could be some potential conflicts due to both workloads being containerized but with different "management" planes. | 14:17 |
| r3ap3r | Also not sure if that may be a supported setup with Kayobe or if it will be strictly, "you have to have separate gear for your Ceph cluster" type thing? | 14:18 |
| kevko | wncslln: if you need to patch the image ..you need to provide your git or your tarball .. image will be then built from your source - that's it | 14:36 |
| opendevreview | Merged openstack/kolla-ansible master: CI: Rework docker config vars https://review.opendev.org/c/openstack/kolla-ansible/+/904067 | 14:50 |
| SvenKieske | kevko: I actually like IPv6, but it's annoying that everyone throws stones into the way of it's success :) ;) | 15:25 |
| SvenKieske | r3ap3r: do you want to deploy a hyper converged setup, that is, ceph and openstack nodes on the same servers? | 15:26 |
| r3ap3r | SvenKieske: that is the idea, yes. | 15:30 |
| SvenKieske | yeah that might be difficult, because docker, maybe less painful with podman? does cephadm support podman these days? | 15:36 |
| r3ap3r | Our production Ceph cluster is currently using Podman and not Docker. | 15:39 |
| r3ap3r | The only reason we are entertaining the idea is due to some gear we may potentially getting from another department in the near future who used VMware in a hyperconverged setup. We were hoping to be able to take advantage of the additional gear instead of using either Openstack or Ceph on the nodes. Splitting them up essentially. We will give it a shot regardless and see how it works. | 15:42 |
| r3ap3r | I'll report here to let everyone know, if we get the gear, on how it works out. | 15:43 |
| kevko | SvenKieske: why it should be painfull with docker ? | 15:54 |
| dmsimard[m] | btw if you'll be at upcoming FOSDEM: https://www.meetup.com/brussels-openinfra-meetup-group/events/298420649/ | 16:00 |
| SvenKieske | kevko: because both kolla-ansible and cephadm somewhat lazily assume full control over docker, e.g. updating docker. docker is afaik still a monolith so restarting docker e.g. restarts all docker containers | 16:01 |
| SvenKieske | dmsimard[m]: I'll try to join there :) | 16:01 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job https://review.opendev.org/c/openstack/kolla-ansible/+/905322 | 16:02 |
| jovial | you can of course use docker live-restore to prevent a docker upgrade from restarting all containers | 16:23 |
| SvenKieske | that seems to be disabled for kayobe for redhat reasons: https://review.opendev.org/c/openstack/kayobe/+/566101 I would thus assume it's not supported by cephadm either? | 16:31 |
| SvenKieske | Also: "If the daemon is down for a long time, running containers may fill up the FIFO log the daemon normally reads. A full log blocks containers from logging more data. The default buffer size is 64K. If the buffers fill, you must restart the Docker daemon to flush them." | 16:32 |
| kevko | SvenKieske: but docker is installed in bootstrap which normally not running | 16:32 |
| SvenKieske | kevko: I'm not sure what you mean by that. I'm talking about day-2 operations e.g. like upgrading docker to a newer version, which should be rather common | 16:33 |
| kevko | SvenKieske: yes, agree, but you can choose if you are going to install ceph via cephadm - it supports podman | 16:34 |
| jovial | For what it is worth, we quite frequently use live restore with both kolla and cephadm deployments. Seems to be quite stable these days. | 16:35 |
| kevko | SvenKieske: also, there is no problem to upgrade docker if you are running ceph in docker containers, if you have too enough daemons and you have replicas ..it's about restart containers | 16:35 |
| kevko | SvenKieske: if it is main problem - you are just saying that if physical host will go down (because of some HW issue) ..you are done .. | 16:36 |
| kevko | (same as docker daemon restart - or OS upgrade ) | 16:37 |
| kevko | and live-restore is also good point | 16:37 |
| jovial | I've definitely shot myself in the foot before with the kolla bootstrap restarting docker against multiple hosts at the same time | 16:38 |
| jovial | when run against* | 16:39 |
| kevko | I just want to say that your ceph cluster should be designed in way that one node down can't be a problem | 16:39 |
| jovial | easy to forget that it likes to upgrade docker if a newer package exists :D | 16:39 |
| SvenKieske | well HCI environments are typically more constraint and have naturally a higher impact if a single host goes down and also get in more trouble via unintended side effects, e.g. sudden higher IO pressure due to ceph restarting which might impact kolla containers on the same host unless you have io limits for everything, which I dobut :) | 16:39 |
| kevko | SvenKieske: kolla supports dimensions per container .. | 16:40 |
| SvenKieske | kevko: sure, one host should never be a problem, but an unintended downtime of a single service instance is imho still a problem. | 16:40 |
| SvenKieske | kevko: and sets none, for the most part ;) | 16:40 |
| kevko | Well, if you know what are u doing .. you can do whatever you want :) | 16:41 |
| SvenKieske | that is a universal, abstract truth..so it's practically useless :) | 16:41 |
| greatgatsby | kevko: can I ask what you mean by "dimensions per container"? Or can you point me to the docs? | 16:41 |
| SvenKieske | greatgatsby: see https://docs.openstack.org/kolla-ansible/latest/reference/deployment-config/resource-constraints.html | 16:41 |
| greatgatsby | SvenKieske: thanks a lot, reading now | 16:42 |
| kevko | greatgatsby: + docker doc https://docs.docker.com/config/containers/resource_constraints/ | 16:43 |
| SvenKieske | here's a proposal to actually add some constraints, at least for rabbitmq (which currently assumes it can use the complete host memory): https://review.opendev.org/c/openstack/kolla-ansible/+/900528 | 16:43 |
| greatgatsby | SvenKieske: nice, thanks, was just about to ask if this constraints are commonly used, or only in exceptional circumstances | 16:44 |
| SvenKieske | in theory they should be used for every deployed container imho. in practice it is sometimes impossible - or rather a lot of work - to come up with reasonable defaults | 16:45 |
| SvenKieske | configure values too small and larger users will complain that you break them. configure values too large, and they won't have any impact in shielding other services from DOS. | 16:46 |
| SvenKieske | I guess default values should be rather small and larger users usually have the knowledge to tweak stuff for large environments anyway | 16:47 |
| SvenKieske | but here we are, using docker containers merely as convenient wrappers around apt|pip|dnf, instead of using it for it's actual benefit: isolating processes from each other :) | 16:47 |
| kevko | SvenKieske: i think default should be None ... there is an option to set them ..no reason to specify some default values | 16:47 |
| kevko | SvenKieske: btw, just reading your last comment here -> https://bugs.launchpad.net/kolla-ansible/+bug/1919387 | 16:48 |
| SvenKieske | kevko: the reason is simple security and best practice: a single process should not be able to resource starve other processes by accident, running on the same machine | 16:48 |
| kevko | SvenKieske: working on it ^^ | 16:48 |
| SvenKieske | kevko: thank you, almost forgot about that one | 16:49 |
| kevko | SvenKieske: I want to rewrite it a bit, and mainly bring light to the documentation, because designate-sink is not really needed | 16:50 |
| SvenKieske | I feel our docs are really improving, maybe it's also just me getting more familiar with everything over time and knowing where to look :) | 16:51 |
| kevko | I hate docs :D | 16:51 |
| kevko | kolla-ansible is self-documentary | 16:51 |
| kevko | designate_ns_records from kolla-ansible we need to migrate .... | 16:53 |
| kevko | SvenKieske: do you have some link to proper kolla documentation how to deprecate, migrate some value ? :D | 16:53 |
| SvenKieske | mhm, what do we need to migrate there? isn't that needed anymore? I remember some vague discussion around it from the abandoned patchset? | 16:55 |
| kevko | SvenKieske: okay, what do you think designate_ns_records menas ? | 16:55 |
| kevko | means | 16:55 |
| SvenKieske | my usual steps for deprecation are: 1. announce deprecation via release notes, docs, and maybe mailing list if it's something large, then disable by default in the next release, then remove in the next-next-release | 16:55 |
| kevko | SvenKieske: and what do you think, is it string or list ? | 16:56 |
| kevko | SvenKieske: what do you think it's used for ? | 16:56 |
| SvenKieske | I remember it was a string or a list, depending on where it was used? :D was that right? might be worth to clean it up | 16:56 |
| kevko | SvenKieske: yes .. and it configures designate and neutron in same time :D | 16:56 |
| SvenKieske | I would have to look it up, anyway. | 16:56 |
| SvenKieske | might be better to use dedicated variables for each and maybe add some logic for backward compatibility that sets values for these two new settings based on the old one? | 16:57 |
| kevko | SvenKieske: and the name of value is NS_RECORD ..which sounds like one value ...no records ....moreover ...it's configuring dns zone in neutron ..not record :D | 16:57 |
| kevko | SvenKieske: that's my plan | 16:57 |
| SvenKieske | well a "record" is - afaik - strictly speaking nothing that exists, if I remember my DNS RFCs correctly. you have "resource records" not "records" ;) | 16:58 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/2023.1: Support older Octavia var names in Antelope https://review.opendev.org/c/openstack/kolla-ansible/+/905500 | 16:59 |
| kevko | SvenKieske: i have first patch :) | 16:59 |
| SvenKieske | most people really have no clue about dns - I myself have also just a little knowledge from administrating some authorative servers - and then stuff happens :D | 17:00 |
| SvenKieske | I need to reboot this machine and hope that the largest telco in my part of the world finally fixed their network.. | 17:00 |
| kevko | :D | 17:02 |
| SvenKieske | mhm I guess I leave that for another day, it's getting late anyway | 17:03 |
| kevko | SvenKieske: designate-sink | 17:04 |
| kevko | https://docs.openstack.org/designate/latest/contributor/architecture.html | 17:04 |
| opendevreview | Verification of a change to openstack/kolla-ansible master failed: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 17:04 |
| SvenKieske | kevko: sadly I'm familiar with that page :) | 17:06 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Make designate-sink service optional https://review.opendev.org/c/openstack/kolla-ansible/+/905502 | 17:12 |
| kevko | SvenKieske: ^^ it makes sense, don't it ? | 17:13 |
| SvenKieske | looks reasonable :) | 17:20 |
| kevko | SvenKieske: designate-sink should be used when designate is not fully mplemented for tenants - or you want to do some specific ... | 17:23 |
| kevko | SvenKieske: if you want to create records for example in example.org for public addresses , you can use designate-sink, configre handler for network id and turn on notifications ...then designate-sink is responsible to create that records in zone id which you will also specify .... | 17:24 |
| kevko | SvenKieske: but, the same you can do if you create zone in for example service project and create a SHARE for tenant ... then sink is not needed and everything is gathered from network info dns_domain and dns_name ... | 17:25 |
| kevko | i think this should be configurable in kolla-ansible | 17:25 |
| opendevreview | Verification of a change to openstack/kolla master failed: Fix openstack CADF audit maps and installation https://review.opendev.org/c/openstack/kolla/+/904576 | 17:53 |
| opendevreview | Dawud proposed openstack/kolla-ansible master: Remove the `grafana` volume https://review.opendev.org/c/openstack/kolla-ansible/+/899136 | 18:05 |
| opendevreview | Verification of a change to openstack/kolla-ansible master failed: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 18:21 |
| opendevreview | Merged openstack/kolla-ansible master: Test haproxy single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/841239 | 21:06 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!