Friday, 2024-01-12

opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job  https://review.opendev.org/c/openstack/kolla-ansible/+/90532206:11
*** mmalchuk_ is now known as mmalchuk06:13
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job  https://review.opendev.org/c/openstack/kolla-ansible/+/90532206:13
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job  https://review.opendev.org/c/openstack/kolla-ansible/+/90532207:01
opendevreviewMerged openstack/kolla-ansible stable/2023.1: CI: Test Nova server resize functionality  https://review.opendev.org/c/openstack/kolla-ansible/+/90428307:48
opendevreviewMerged openstack/kolla-ansible stable/zed: CI: Test Nova server resize functionality  https://review.opendev.org/c/openstack/kolla-ansible/+/90428408:02
opendevreviewMerged openstack/kolla-ansible stable/yoga: CI: Test Nova server resize functionality  https://review.opendev.org/c/openstack/kolla-ansible/+/90428508:02
opendevreviewMerged openstack/kolla stable/2023.2: CI: Use newer podman/buildah on Ubuntu Jammy  https://review.opendev.org/c/openstack/kolla/+/90473109:07
opendevreviewJake Hutchinson proposed openstack/kolla-ansible master: Ironic parameter rework and default NTP server  https://review.opendev.org/c/openstack/kolla-ansible/+/89303110:26
opendevreviewMerged openstack/kayobe master: Update python classifier in setup.cfg  https://review.opendev.org/c/openstack/kayobe/+/90505611:06
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: openvswitch: Stop using intermediate scripts  https://review.opendev.org/c/openstack/kolla-ansible/+/90511711:08
kevko\o11:09
SvenKieskeo/11:14
SvenKieskejust ripped out my ipv6 support because review.opendev.org doesn't really seem to like IPv6 currently11:14
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: haproxy: Use -f configdir instead of for and xargs  https://review.opendev.org/c/openstack/kolla-ansible/+/90512111:17
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: mariadb-clustercheck: Use socat wrapper  https://review.opendev.org/c/openstack/kolla-ansible/+/90513111:17
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: openvswitch: Stop using intermediate scripts  https://review.opendev.org/c/openstack/kolla-ansible/+/90511711:18
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: WIP: fail when systemd unit fails to stop  https://review.opendev.org/c/openstack/kolla-ansible/+/89178111:18
opendevreviewMerged openstack/kolla-ansible master: Fix trove failed to discover swift endpoint  https://review.opendev.org/c/openstack/kolla-ansible/+/90496811:41
opendevreviewMerged openstack/kolla stable/2023.2: rabbitmq: Use timeout in healthcheck script  https://review.opendev.org/c/openstack/kolla/+/90451912:01
opendevreviewMerged openstack/kolla stable/2023.1: rabbitmq: Use timeout in healthcheck script  https://review.opendev.org/c/openstack/kolla/+/90452012:01
opendevreviewMerged openstack/kolla stable/zed: rabbitmq: Use timeout in healthcheck script  https://review.opendev.org/c/openstack/kolla/+/90472112:07
opendevreviewMerged openstack/kolla stable/yoga: rabbitmq: Use timeout in healthcheck script  https://review.opendev.org/c/openstack/kolla/+/90472212:07
opendevreviewVerification of a change to openstack/kolla master failed: Fix openstack CADF audit maps and installation  https://review.opendev.org/c/openstack/kolla/+/90457612:12
wncsllnhi o/, anyone can help me how to apply a patch in a kolla-ansible deployment, please? i need to apply a patch with some Nova changes to test a feature13:28
opendevreviewMerged openstack/kolla-ansible stable/2023.2: Enable glance proxying behaviour  https://review.opendev.org/c/openstack/kolla-ansible/+/90485013:52
kevkowncslln: just apply a patch :D 14:01
kevkoSvenKieske: most of people don't like IPv614:03
kevko:D14:03
wncsllnkevko: apply to repository before the deploy? if not, since nova is a container, i wont need to rebuild it after apply the patch?14:13
opendevreviewVerification of a change to openstack/kolla master failed: Fix openstack CADF audit maps and installation  https://review.opendev.org/c/openstack/kolla/+/90457614:15
opendevreviewMerged openstack/kolla-ansible stable/2023.1: Enable glance proxying behaviour  https://review.opendev.org/c/openstack/kolla-ansible/+/90485114:15
opendevreviewMerged openstack/kolla-ansible stable/zed: Enable glance proxying behaviour  https://review.opendev.org/c/openstack/kolla-ansible/+/90485214:15
opendevreviewMerged openstack/kolla-ansible stable/yoga: Enable glance proxying behaviour  https://review.opendev.org/c/openstack/kolla-ansible/+/90485314:15
opendevreviewMerged openstack/kolla-ansible stable/2023.2: Remove nova cell sync comment  https://review.opendev.org/c/openstack/kolla-ansible/+/90472514:15
opendevreviewMerged openstack/ansible-collection-kolla master: Use py3 as the default runtime for tox  https://review.opendev.org/c/openstack/ansible-collection-kolla/+/89074814:15
opendevreviewMerged openstack/kolla-ansible stable/2023.2: cadvisor: Set housekeeping interval to Prometheus scrape interval  https://review.opendev.org/c/openstack/kolla-ansible/+/90484214:15
opendevreviewMerged openstack/kolla-ansible stable/zed: cadvisor: Set housekeeping interval to Prometheus scrape interval  https://review.opendev.org/c/openstack/kolla-ansible/+/90484414:15
r3ap3rI don't see Hudson around so I figured I'd ask in "general", has anyone attempted Kolla-Ansible deployment with Cephadm in a hyperconverged setup? We may be attempting that kind of setup in the near future and was curious if anyone had already attempted it and what their experience was. I feel like there could be some potential conflicts due to both workloads being containerized but with different "management" planes.14:17
r3ap3rAlso not sure if that may be a supported setup with Kayobe or if it will be strictly, "you have to have separate gear for your Ceph cluster" type thing?14:18
kevkowncslln: if you need to patch the image ..you need to provide your git  or your tarball .. image will be then built from your source - that's it 14:36
opendevreviewMerged openstack/kolla-ansible master: CI: Rework docker config vars  https://review.opendev.org/c/openstack/kolla-ansible/+/90406714:50
SvenKieskekevko: I actually like IPv6, but it's annoying that everyone throws stones into the way of it's success :) ;)15:25
SvenKiesker3ap3r: do you want to deploy a hyper converged setup, that is, ceph and openstack nodes on the same servers?15:26
r3ap3rSvenKieske: that is the idea, yes.15:30
SvenKieskeyeah that might be difficult, because docker, maybe less painful with podman? does cephadm support podman these days?15:36
r3ap3rOur production Ceph cluster is currently using Podman and not Docker.15:39
r3ap3rThe only reason we are entertaining the idea is due to some gear we may potentially getting from another department in the near future who used VMware in a hyperconverged setup. We were hoping to be able to take advantage of the additional gear instead of using either Openstack or Ceph on the nodes. Splitting them up essentially. We will give it a shot regardless and see how it works.15:42
r3ap3rI'll report here to let everyone know, if we get the gear, on how it works out.15:43
kevkoSvenKieske: why it should be painfull with docker ? 15:54
dmsimard[m]btw if you'll be at upcoming FOSDEM: https://www.meetup.com/brussels-openinfra-meetup-group/events/298420649/16:00
SvenKieskekevko: because both kolla-ansible and cephadm somewhat lazily assume full control over docker, e.g. updating docker. docker is afaik still a monolith so restarting docker e.g. restarts all docker containers16:01
SvenKieskedmsimard[m]: I'll try to join there :)16:01
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: Run SLURP upgrade job  https://review.opendev.org/c/openstack/kolla-ansible/+/90532216:02
jovialyou can of course use docker live-restore to prevent a docker upgrade from restarting all containers16:23
SvenKieskethat seems to be disabled for kayobe for redhat reasons: https://review.opendev.org/c/openstack/kayobe/+/566101 I would thus assume it's not supported by cephadm either?16:31
SvenKieskeAlso: "If the daemon is down for a long time, running containers may fill up the FIFO log the daemon normally reads. A full log blocks containers from logging more data. The default buffer size is 64K. If the buffers fill, you must restart the Docker daemon to flush them."16:32
kevkoSvenKieske: but docker is installed in bootstrap which normally not running 16:32
SvenKieskekevko: I'm not sure what you mean by that. I'm talking about day-2 operations e.g. like upgrading docker to a newer version, which should be rather common16:33
kevkoSvenKieske: yes, agree, but you can choose if you are going to install ceph via cephadm - it supports podman 16:34
jovialFor what it is worth, we quite frequently use live restore with both kolla and cephadm deployments. Seems to be quite stable these days.16:35
kevkoSvenKieske: also, there is no problem to upgrade docker if you are running ceph in docker containers, if you have too enough daemons and you have replicas ..it's about restart containers 16:35
kevkoSvenKieske: if it is main problem - you are just saying that if physical host will go down (because of some HW issue) ..you are done ..16:36
kevko(same as docker daemon restart - or OS upgrade )16:37
kevkoand live-restore is also good point 16:37
jovialI've definitely shot myself in the foot before with the kolla bootstrap restarting docker against multiple hosts at the same time16:38
jovialwhen run against*16:39
kevkoI just want to say that your ceph cluster should be designed in way that one node down can't be a problem16:39
jovialeasy to forget that it likes to upgrade docker if a newer package exists :D16:39
SvenKieskewell HCI environments are typically more constraint and have naturally a higher impact if a single host goes down and also get in more trouble via unintended side effects, e.g. sudden higher IO pressure due to ceph restarting which might impact kolla containers on the same host unless you have io limits for everything, which I dobut :)16:39
kevkoSvenKieske: kolla supports dimensions per container ..16:40
SvenKieskekevko: sure, one host should never be a problem, but an unintended downtime of a single service instance is imho still a problem.16:40
SvenKieskekevko: and sets none, for the most part ;)16:40
kevkoWell, if you know what are u doing .. you can do whatever you want :) 16:41
SvenKieskethat is a universal, abstract truth..so it's practically useless :)16:41
greatgatsbykevko: can I ask what you mean by "dimensions per container"?  Or can you point me to the docs?16:41
SvenKieskegreatgatsby: see https://docs.openstack.org/kolla-ansible/latest/reference/deployment-config/resource-constraints.html16:41
greatgatsbySvenKieske: thanks a lot, reading now16:42
kevkogreatgatsby: + docker doc https://docs.docker.com/config/containers/resource_constraints/ 16:43
SvenKieskehere's a proposal to actually add some constraints, at least for rabbitmq (which currently assumes it can use the complete host memory): https://review.opendev.org/c/openstack/kolla-ansible/+/90052816:43
greatgatsbySvenKieske: nice, thanks, was just about to ask if this constraints are commonly used, or only in exceptional circumstances16:44
SvenKieskein theory they should be used for every deployed container imho. in practice it is sometimes impossible - or rather a lot of work - to come up with reasonable defaults16:45
SvenKieskeconfigure values too small and larger users will complain that you break them. configure values too large, and they won't have any impact in shielding other services from DOS.16:46
SvenKieskeI guess default values should be rather small and larger users usually have the knowledge to tweak stuff for large environments anyway16:47
SvenKieskebut here we are, using docker containers merely as convenient wrappers around apt|pip|dnf, instead of using it for it's actual benefit: isolating processes from each other :)16:47
kevkoSvenKieske: i think default should be None ... there is an option to set them ..no reason to specify some default values 16:47
kevkoSvenKieske: btw, just reading your last comment here -> https://bugs.launchpad.net/kolla-ansible/+bug/191938716:48
SvenKieskekevko: the reason is simple security and best practice: a single process should not be able to resource starve other processes by accident, running on the same machine16:48
kevkoSvenKieske: working on it ^^ 16:48
SvenKieskekevko: thank you, almost forgot about that one16:49
kevkoSvenKieske: I want to rewrite it a bit, and mainly bring light to the documentation, because designate-sink is not really needed16:50
SvenKieskeI feel our docs are really improving, maybe it's also just me getting more familiar with everything over time and knowing where to look :)16:51
kevkoI hate docs :D 16:51
kevkokolla-ansible is self-documentary 16:51
kevkodesignate_ns_records from kolla-ansible we need to migrate ....16:53
kevkoSvenKieske: do you have some link to proper kolla documentation how to deprecate, migrate some value ? :D 16:53
SvenKieskemhm, what do we need to migrate there? isn't that needed anymore? I remember some vague discussion around it from the abandoned patchset?16:55
kevkoSvenKieske: okay, what do you think designate_ns_records menas ? 16:55
kevkomeans 16:55
SvenKieskemy usual steps for deprecation are: 1. announce deprecation via release notes, docs, and maybe mailing list if it's something large, then disable by default in the next release, then remove in the next-next-release16:55
kevkoSvenKieske: and what do you think, is it string or list ? 16:56
kevkoSvenKieske: what do you think it's used for ? 16:56
SvenKieskeI remember it was a string or a list, depending on where it was used? :D was that right? might be worth to clean it up16:56
kevkoSvenKieske: yes .. and it configures designate and neutron in same time :D 16:56
SvenKieskeI would have to look it up, anyway.16:56
SvenKieskemight be better to use dedicated variables for each and maybe add some logic for backward compatibility that sets values for these two new settings based on the old one?16:57
kevkoSvenKieske: and the name of value is NS_RECORD  ..which sounds like one value ...no records ....moreover ...it's configuring dns zone in neutron ..not record :D 16:57
kevkoSvenKieske: that's my plan 16:57
SvenKieskewell a "record" is - afaik - strictly speaking nothing that exists, if I remember my DNS RFCs correctly. you have "resource records" not "records" ;)16:58
opendevreviewMatt Crees proposed openstack/kolla-ansible stable/2023.1: Support older Octavia var names in Antelope  https://review.opendev.org/c/openstack/kolla-ansible/+/90550016:59
kevkoSvenKieske: i have first patch :) 16:59
SvenKieskemost people really have no clue about dns - I myself have also just a little knowledge from administrating some authorative servers - and then stuff happens :D17:00
SvenKieskeI need to reboot this machine and hope that the largest telco in my part of the world finally fixed their network..17:00
kevko:D17:02
SvenKieskemhm I guess I leave that for another day, it's getting late anyway17:03
kevkoSvenKieske: designate-sink 17:04
kevkohttps://docs.openstack.org/designate/latest/contributor/architecture.html17:04
opendevreviewVerification of a change to openstack/kolla-ansible master failed: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123917:04
SvenKieskekevko: sadly I'm familiar with that page :)17:06
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Make designate-sink service optional  https://review.opendev.org/c/openstack/kolla-ansible/+/90550217:12
kevkoSvenKieske: ^^ it makes sense, don't it ? 17:13
SvenKieskelooks reasonable :)17:20
kevkoSvenKieske: designate-sink should be used when designate is not fully mplemented for tenants - or you want to do some specific ...17:23
kevkoSvenKieske: if you want to create records for example in example.org for public addresses , you can use designate-sink, configre handler for network id and turn on notifications ...then designate-sink is responsible to create that records in zone id which you will also specify ....17:24
kevkoSvenKieske: but, the same you can do if you create zone in for example service project and create a SHARE for tenant ... then sink is not needed and everything is gathered from network info dns_domain and dns_name ...17:25
kevkoi think this should be configurable in kolla-ansible 17:25
opendevreviewVerification of a change to openstack/kolla master failed: Fix openstack CADF audit maps and installation  https://review.opendev.org/c/openstack/kolla/+/90457617:53
opendevreviewDawud proposed openstack/kolla-ansible master: Remove the `grafana` volume  https://review.opendev.org/c/openstack/kolla-ansible/+/89913618:05
opendevreviewVerification of a change to openstack/kolla-ansible master failed: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123918:21
opendevreviewMerged openstack/kolla-ansible master: Test haproxy single external frontend  https://review.opendev.org/c/openstack/kolla-ansible/+/84123921:06

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!