| opendevreview | wu.chunyang proposed openstack/kolla-ansible master: Implement automatic deploy of trove https://review.opendev.org/c/openstack/kolla-ansible/+/863321 | 01:22 |
|---|---|---|
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Make designate-sink service optional https://review.opendev.org/c/openstack/kolla-ansible/+/905502 | 06:48 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Test designate https://review.opendev.org/c/openstack/kolla-ansible/+/905644 | 06:48 |
| kevko | SvenKieske: here ? | 07:55 |
| *** darmach4 is now known as darmach | 08:28 | |
| SvenKieske | kevko: yes, but currently in a Meeting | 09:18 |
| opendevreview | Will Szumski proposed openstack/kolla master: Support CAP_DAC_READ_SEARCH capability https://review.opendev.org/c/openstack/kolla/+/905579 | 09:22 |
| kevko | SvenKieske: just check my reply in designate patch ... | 09:55 |
| kevko | SvenKieske: I was also wondering if we shouldn't have a scenario for designate | 09:56 |
| SvenKieske | you mean a dedicated test scenario in zuul ci? I'm always for more testing :) But we should also keep an eye on pipeline times, as these are already somewhat long :) | 10:01 |
| kevko | SvenKieske: yep, but as you can see in my reply in patch for designate ..you don't need sink ...and definitely don't need audit from nova as it is only for ceilometer | 10:09 |
| SvenKieske | yeah, I was confused at first when I saw that, and the original patch didn't make me any wiser, why this was added | 10:09 |
| kevko | SvenKieske: so I wanted to create a dedicated test scenario for designate for neutron dns_publish_fixed_ip plugin we providing in kolla by default | 10:10 |
| SvenKieske | +1 from me, you get for that :) | 10:10 |
| kevko | SvenKieske: and also sink | 10:11 |
| kevko | let me write something | 10:11 |
| SvenKieske | kevko: you answered someone on the ML regarding custom vault haproxy backend | 10:14 |
| SvenKieske | they seem to do exactly what you told them to do? the paste has the code in /etc/kolla/haproxy/services.d/vault.cfg :) | 10:15 |
| kevko | SvenKieske: nope - he said "/etc/kolla/config/<openstack_service>/<openstack_service.conf> works, the same approach doesn't seem to work for haproxy. " | 10:22 |
| kevko | SvenKieske: I replied "Just place it into /etc/kolla/config/haproxy/services.d/vault.cfg" ...which is different ;-) | 10:22 |
| kevko | SvenKieske: did you get it :) ? | 10:23 |
| SvenKieske | kevko: no, reread the mail and especially follow the link to the paste, that's not what they did. | 10:29 |
| SvenKieske | I had to read it twice myself :) | 10:29 |
| kevko | SvenKieske: :D aaaa ... maybe because he used tag haproxy which don't exist ? | 10:35 |
| kevko | SvenKieske: no ..it exist | 10:35 |
| SvenKieske | did he state which release he uses? wasn't this tag in the past something like "loadbalancer"? maybe that's also just a special case in our downstream, but I doubt it. | 10:51 |
| SvenKieske | just recently did use the tag loadbalancer for haproxy, but that was on a yoga based installation | 10:52 |
| SvenKieske | did that change sometime? | 10:52 |
| SvenKieske | https://review.opendev.org/c/openstack/kolla-ansible/+/877507 | 10:54 |
| kevko | SvenKieske: it includes haproxy and loadbalancer also ... i was the one who rewrote haproxy :D :D | 11:00 |
| kevko | SvenKieske: maybe he don't have good inventory | 11:00 |
| SvenKieske | first question I guess would be which version they run, maybe it's really really old | 11:05 |
| kevko | SvenKieske: coud you please check my bugreport https://bugs.launchpad.net/kolla-ansible/+bug/2049503 <<< | 11:17 |
| SvenKieske | I'll check it out :) | 11:19 |
| kevko | SvenKieske: please can u check it now ? I tried to describe it very well ... because then I can start implementation and (unfortunatelly) docs | 11:20 |
| SvenKieske | ok | 11:25 |
| kevko | SvenKieske: thank you | 11:26 |
| SvenKieske | I added a comment, I guess we already discussed that stuff yesterday or last week, I don't remember exactly. the main reason for this mess is to use a single variable where you need two, imho (at least two, I don't know if two suffices) | 11:30 |
| SvenKieske | maybe there are also other solution, but in my eyes this is a bug | 11:31 |
| SvenKieske | *solutions | 11:31 |
| kevko | SvenKieske: replied | 11:37 |
| SvenKieske | kevko: maybe you also have an opinion on this unrelated topic regarding the switch to ovn by default? https://review.opendev.org/c/openstack/kolla-ansible/+/904959/comment/42e2f26f_e196f1a6/ I found it a little weird | 13:31 |
| kevko | SvenKieske: haha, sometimes I really don't understand why we are switching a big stuff by default ..and sometimes there is a problem with backward compatibility (for example fl1nt patch for designate ) :D :D :D | 13:38 |
| kevko | even if it is reasonable and patch is small :D | 13:38 |
| kevko | SvenKieske: do you know why ? ... btw ..i am going to check | 13:38 |
| SvenKieske | well I guess it has something to do with other software having some advantages in some cases. I also wished different softwares would implement the same stuff, feature wise, so migrations would be more painless, but most devs don't seem to care for this | 13:41 |
| SvenKieske | I agree we are not really neutral when judging different changes. imho we should try to judge everything the same way. e.g. many changes don't really explain why they are done because some core maintainer "knows" why it's done, but everyone else is left in the dark. | 13:42 |
| SvenKieske | but when a new contributor comes in, they have to justify everything they do, feels a little sad imho. :/ | 13:42 |
| SvenKieske | to be clear: of course changes should always explain why they are done, but not only for certain, but all contributions imho. | 13:43 |
| SvenKieske | and large complex changes should all be reviewed with high scrutiny, and not be pushed through because someone needs it downstream. | 13:43 |
| kevko | SvenKieske: haha, I've already learned how to contribute (or I hope ...) ..so, bugreport, discussion ..etc etc :D | 13:43 |
| kevko | SvenKieske: and also, don't switch anything ..just prepare it for user switch and leave defaults as before | 13:44 |
| SvenKieske | what I'm observing in the ovn case is, that more and more people are deploying ovn by default instead of plain ovs. I hoped that the ovn gaps would be closed before I have to migrate myself, but well.. | 13:44 |
| SvenKieske | kevko: agreed. but on the other hand, when some time (releases) have gone by, we also need to switch defaults to "new shiny thing(TM)" since users expect it, I guess. | 13:45 |
| SvenKieske | and I can understand it from user perspective. I also like stuff doing what I want "out of the box" without switching manually 100 config knobs. :D | 13:46 |
| SvenKieske | but first and foremost stuff should work I guess, so I welcome the designate change, that was really not thought out well, to "extend" a string to a list and then not checking all locations where a string is expected. | 13:48 |
| SvenKieske | I guess this is a side effect of python not being strongly typed by default. maybe we should add strong typing via mypy or similar tooling. | 13:48 |
| SvenKieske | that would - in theory, when used properly - catch this stuff during static type checking in CI | 13:49 |
| kevko | SvenKieske: or rewrite openstack to something more sexy ? :D GoLang :D | 13:49 |
| kevko | SvenKieske: or, let's just use k8s :D :D | 13:49 |
| kevko | SvenKieske: or drop ansible and start to use something better (just a joke :D ) | 13:50 |
| SvenKieske | kevko: well I'm a secret member of the "rust evangelism strike force" if you heard that term online maybe :P but I think it might be feasible to at least use types in python, I mean it's in the language since some versions now? | 13:50 |
| kevko | SvenKieske: but nowadays i am sometimes confused when trying several combinations and run reconfigure for minutes :( | 13:50 |
| SvenKieske | kevko: jetporch was unfortunately dropped upstream (ansible rewrite in rust by original ansible author michal de haan) due to lack of interest | 13:51 |
| kevko | SvenKieske: if I am correct and remember things well ... i've already seen some new openstack code which is using this feature | 13:51 |
| SvenKieske | kevko: you mean python types? I have seen it in some projects afaik as well, yes. Well most code is fairly old so I can understand the history. | 13:52 |
| kevko | SvenKieske: i was trying to say that sometimes I am reconfiguring again and again and most of the time it's not ansible what I hate ..but implementation of ansible ...every task is just open ssh -> copy everything include python module file -> run it -> disconnect ..and again again | 13:55 |
| SvenKieske | kevko: well yeah, the advantage of that model is, it is very "easy" to do, programming wise. but imho it's not "simple" when you want to debug stuff :D I know what you mean. a "kubectl apply -f my.yml" seems to work more often, at least it does not produce dozens of ssh timeouts on it's way ;) | 13:57 |
| SvenKieske | the advantage of ansible is, you can train everyone to understand the basic operational model "code execution via ssh". that's dead simple. the devil is in the details then. | 13:58 |
| SvenKieske | also we are basically, at least today, writing our own poor mans orchestrator in kolla-ansible: we need to manage container lifecycle of complex and diverse services and even do clustering of containers and high availability. | 13:59 |
| SvenKieske | basically poor mans kubernetes :) | 13:59 |
| kevko | SvenKieske: next task I would like to work on will be magnum in kolla | 14:01 |
| SvenKieske | on the other hand, kubernetes brings many complexities you maybe don't want or need and is, as a technology, understood by not the most people. many people can "use" k8s, but when it doesn't do what they want they are unable to debug, because they don't understand: DNS, containers/namespaces, complex networking, certificates, etcd clustering <- anything from these | 14:01 |
| SvenKieske | magnum seems to gain some traction again. do you mean working on the deployment of magnum via kolla or do you wanna use magnum for kolla itself? :D | 14:02 |
| kevko | SvenKieske: well, I am not fan of k8s :D, and truly said my knowledge of openstack is better than k8s ... but k8s just works when i am testing from time to time | 14:03 |
| kevko | SvenKieske: well, magnum finally just works after years :D ...but actually everything is done in k8s management cluster | 14:04 |
| kevko | i would like to provide some HA solution to just deploy magnum with management cluster at once | 14:05 |
| kevko | don't need to build it separately | 14:05 |
| SvenKieske | kevko: did you have a look at this then? https://github.com/vexxhost/magnum-cluster-api | 14:06 |
| kevko | SvenKieske: it is what i am talking about | 14:06 |
| SvenKieske | yeah, I find this a rather nice solution, my k8s knowledge is also limited (I ran, debugged and deployed them in the past though), but I think cluster api is a rather nice approach. and what I hear is it seems to work quite well. | 14:08 |
| kevko | SvenKieske: yes, it works ..but everything is done in k8s cluster .. | 14:08 |
| kevko | basically you have k8s cluster on same network as openstack services ...and when some tenant want to create k8s ...magnum just proxy request to that management cluster and that cluster will spin tenant's cluster :) ..and of course then it can be updated, autoscalled, destroyed ...etc etc | 14:09 |
| kevko | SvenKieske: but for production - you probably wants your management cluster to be HA, easy updateable etc etc ... | 14:11 |
| opendevreview | Will Szumski proposed openstack/kolla master: Support CAP_DAC_READ_SEARCH capability https://review.opendev.org/c/openstack/kolla/+/905579 | 14:46 |
| SvenKieske | yeah, I agree | 14:48 |
| opendevreview | Piotr Parczewski proposed openstack/kolla-ansible master: Adjust Ceph metrics scrape interval in Prometheus https://review.opendev.org/c/openstack/kolla-ansible/+/902129 | 15:11 |
| *** Continuity__ is now known as Continuity | 15:48 | |
| kevko | mnasiadka: we have probably broken kolla CI build | 16:08 |
| kevko | Get "https://primary:4000/v2/": http: server gave HTTP response to HTTPS client | 16:08 |
| kevko | probably by your patch :D | 16:10 |
| SvenKieske | I'm fairly certain https did work at some point in time? did something else break that maybe? | 16:11 |
| kevko | SvenKieske: i am 90 percent sure that it's broken by https://review.opendev.org/c/openstack/kolla-ansible/+/904067 :) | 16:12 |
| bbezak | kevko: I'll look into that | 16:21 |
| bbezak | kevko: I'm 100% sure that it was caused by it :) | 16:22 |
| kevko | bbezak: https://365c4224c221ec730c2d-019bc8f0795daf4dab730f80e83974fa.ssl.cf1.rackcdn.com/905116/2/check/kolla-ansible-rocky9-upgrade/58f0bf7/primary/logs/system_configs/docker/daemon.json << OLD daemon.json working | 16:23 |
| kevko | bbezak: new one https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_d07/904576/6/check/kolla-ansible-rocky9-upgrade/d0722d6/primary/logs/system_configs/docker/daemon.json | 16:24 |
| kevko | not working | 16:24 |
| bbezak | it is only seen when running k-a upgrade jobs in kolla. | 16:25 |
| bbezak | yeah, previously last if statement was overidden by global docker config : | 16:26 |
| bbezak | {% if need_build_image and is_previous_release %} | 16:26 |
| bbezak | insecure-registries: | 16:26 |
| bbezak | - primary:4000 | 16:26 |
| kevko | bbezak: well, so we have broken all upgrade jobs, don't we ? :D | 16:26 |
| kevko | bbezak: so actually we can't merge anything :D | 16:27 |
| bbezak | and now we're falling into https://github.com/openstack/kolla-ansible/blob/master/tests/templates/globals-default.j2#L76 | 16:27 |
| kevko | bbezak: i can fix it i think ..but i don't know what was the purpose of this patch ...i mean ..why it was changed ? | 16:27 |
| bbezak | apart of fetching more logs and adding debug for podman, it looks like reorg of vars to be more specific and use native vars both for podman and docker insecure registries | 16:31 |
| SvenKieske | maybe the specific vars slightly differ in behaviour? just a guess though | 16:36 |
| kevko | bbezak , SvenKieske: hmm, i've checked the code by an eye and maybe i know where is problem ..let me try it :) | 16:36 |
| bbezak | exactly SvenKieske: insecure-registries: is a native ansible-collection-kolla var which overrides previous settings | 16:36 |
| bbezak | and now we don't have it | 16:37 |
| bbezak | actually docker_custom_config | 16:37 |
| bbezak | which is now gone | 16:37 |
| bbezak | so either we will add it back, or rework the logic | 16:37 |
| bbezak | the things is that we're not running building images job in kolla-ansible, that's why we didn't catch it | 16:38 |
| SvenKieske | mhm, my bad, I actually didn't review this enough, because I had a slight suspicion when reviewing that part that there might be something odd in there, because I reported a bug in the past about the insecure_registry stuff. | 16:39 |
| SvenKieske | that was a unrelated bug, but I new the code from that incident, but I guess it was too long ago and I didn't quite remember why that code looked a little odd to me, so I just went with the majority vote there. | 16:41 |
| bbezak | (mostly afk now, can look into that tomorrow) | 16:47 |
| kevko | okay, there is some mess with tests/templates/globals-default.j2 or mess in version and is_previous_release | 17:39 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!