Thursday, 2021-10-21

eugenmayerGood morgnin :)05:42
eugenmayerWhen you initially deploy with kolla, do you keep the deployment machine arround or is kolla a once-task for each cluster and one does not rerune after changin / enabling service (and try to converge again)?05:42
jingvarmorning05:47
jingvareugenmayer: it depends on type of changes05:48
jingvarfor example - to change dns, you have to restart docker or restart node. because running containers don't get this change05:50
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible stable/xena: Use Xena images  https://review.opendev.org/c/openstack/kolla-ansible/+/81485605:52
opendevreviewMichal Nasiadka proposed openstack/kolla stable/xena: centos: switch to RDO final release of Xena  https://review.opendev.org/c/openstack/kolla/+/81485705:58
hrwmorning05:58
hrwmnasiadka, yoctozepto: tacker does not install anyjson anymore ;D05:58
mnasiadkathat was fast05:58
hrwmnasiadka: no code changes05:59
jingvarDo you have an idea how to oveeride kolla templates06:05
mnasiadkathe docs have - https://docs.openstack.org/kolla/xena/admin/image-building.html#generic-customisation06:08
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible stable/xena: Use Xena images  https://review.opendev.org/c/openstack/kolla-ansible/+/81485606:16
hrwmnasiadka: https://review.opendev.org/c/openstack/kolla/+/81477906:17
hrwhttps://review.opendev.org/c/openstack/kolla/+/814796 is simple too06:18
jingvarmnasiadka: thanks, but it is not expected way06:22
hrwjingvar: what do you mean?06:22
hrwjingvar: and please describe in 2-3 sentences.06:23
hrwI will be back in ~hour06:25
mnasiadkacentos-release-openstack-xena not on mirrors yet, ugh06:30
jingvarI want an customization for a kolla template file via kayobe-config 06:37
jingvarI can customize an service config file via  kayobe-config 06:38
jingvarroles/haproxy-config/defaults/main.yml:haproxy_service_template: "haproxy_single_service_split.cfg.j2"06:39
jingvarroles/haproxy-config/tasks/main.yml:      - "templates/{{ haproxy_service_template }}"06:39
jingvarhow to deliver via kayobe an teplate to kolla templates folder06:40
jingvarfor keepalived I have kayobe-config/etc/kayobe/kolla/config/keepalived/keepalived.conf 06:41
opendevreviewMichal Nasiadka proposed openstack/kolla stable/xena: centos: switch to RDO final release of Xena  https://review.opendev.org/c/openstack/kolla/+/81485706:43
mnasiadkajingvar: you said kolla, not kolla-ansible06:45
mnasiadkajingvar: be specific in questions, then you'll probably get better suited answers06:45
jingvargot it06:46
mnasiadkajingvar: https://github.com/openstack/kolla-ansible/blob/7b8332e1aaac3548a68fc9d750f308fa14d764f1/ansible/roles/loadbalancer/tasks/config.yml#L60 - here's how we copy custom services into services.d06:47
mnasiadkajingvar: just put your custom files in etc/kayobe/kolla/config/haproxy/services.d06:47
mnasiadkakeepalived in a similar fashion (just check path for keepalived.conf in tasks/config.yml)06:49
jingvarit is not cutom config - it is template to build  config files06:49
jingvarkeepalived works06:49
mnasiadkaI don't think there's a way to override whole template06:53
mnasiadkajingvar: why would you want to override it?06:56
jingvarI thought  kayobe can override any kolla variable and any file Ж06:57
jingvar:)06:57
mnasiadkathat magic is unavailable06:57
jingvarI don't like downstream, but ...06:59
mnasiadkajingvar: I think for us it would be nice to understand why do you want to override the template (e.g. what's missing in haproxy options)07:09
yoctozeptohrw, mnasiadka: debian-ovn copy-paste following07:26
yoctozepto22:55:36 <zigo> yoctozepto: frickler: I've built OVN in the osbpo.debian.net repo. To install it in Bullseye:07:26
yoctozepto22:55:36 <zigo> apt-get install extrepo07:26
yoctozepto22:55:36 <zigo> extrepo enable openstack_xena07:26
yoctozepto22:55:36 <zigo> apt-get update07:26
yoctozepto22:55:36 <zigo> apt-get install ovn-central07:26
hrwok, so now I need to talk with some people to get aarch64 builder for zigo or sth07:28
jingvarI want ot turn back basic Openstack feature - admin, internal, external newtorks and APIs07:29
jingvarI don't see in the last Openstack released admin network is depricated or noticed as unnessary07:30
mnasiadkajingvar: admin/internal/public endpoints in keystone?07:30
jingvaryes, I have done it07:32
yoctozeptoI am busy for the time being, so please try ovn even for x86_6407:32
yoctozepto;-)07:32
yoctozeptoyou can test it with https://review.opendev.org/c/openstack/kolla-ansible/+/81461007:32
yoctozeptojust drop some kolla patch and do depends-on, you know the drill :-)07:33
yoctozeptoubuntu-ovn should then start passing07:33
mnasiadkajingvar: I still don't understand07:35
jingvarI need admin network- admin vip, admin fqdn, admin endpoints,07:36
jingvarI have  admin endponts generation for keystone (and i think for other services), keepoalived with admin_vip07:38
jingvarkeystone_admin_url: {% raw %} "{{ admin_protocol }}://{{ keystone_admin_fqdn | put_address_in_context('url') }}:{{ keystone_admin_port }}" {% endraw %}07:39
mnasiadkaSo just add this additional network in kayobe as an extra network for the controllers and run host configure?07:42
mnasiadkayoctozepto: ubuntu-ovn?07:43
opendevreviewMichal Arbet proposed openstack/kolla master: [DNM] Remove debian ovn-base from unbuildable images  https://review.opendev.org/c/openstack/kolla/+/81486007:45
jingvarit to me ? mnasiadka: o just add this additional network in kayo07:46
mnasiadkajingvar: yes, I assume you want admin endpoints to be on a separate network, right?07:49
jingvarof course I have separated internal, admin and public networks07:51
opendevreviewMerged openstack/kolla master: openstack-base: drop anyjson  https://review.opendev.org/c/openstack/kolla/+/81477907:52
jingvari have custom keepalive config  - that brings to me 3  VIPs 07:52
mnasiadkaso just define a new network in Kayobe and have it configured on hosts, override keystone_admin_url/keystone_internal_url (and probably all the other dicts for services in catalog), add a new VIP to keepalived.conf 07:52
mnasiadkaand I guess that should work07:53
jingvarbut now problem is haproxy config 07:53
jingvarI don't like ugly way to place custom haproxy config for each service07:54
mnasiadkaah, you would need to override a whole dict of a service and add another endpoint in haproxy: part, e.g. keystone_services (https://github.com/openstack/kolla-ansible/blob/7b8332e1aaac3548a68fc9d750f308fa14d764f1/ansible/roles/keystone/defaults/main.yml#L13)07:55
dardelean_Hei. Is there a way from kolla-ansible to install a pip package upon deploy time? For example I need to install certain purestorage packets in cinder, can I do it without building my own images?07:55
jingvaryep07:55
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: monasca: enable for Debian/aarch64  https://review.opendev.org/c/openstack/kolla/+/81479607:55
hrwupdated support matrix07:56
kevkomorning \o/07:56
jingvarbut haproxy template hardcoded with 2 front/backends07:58
jingvarvirtual_ipaddress   if haproxy_enable_external_vip if haproxy_enable_external_vip07:59
frickleryoctozepto: mnasiadka: I think that idea with a test patch is good, but I'll do it with devstack first, less moving parts and easier to have neutron folk debug it if needed ;)08:04
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/xena: docs: Improve info about neutron external interface  https://review.opendev.org/c/openstack/kolla-ansible/+/81476608:08
opendevreviewMark Goddard proposed openstack/kolla-ansible stable/wallaby: docs: Improve info about neutron external interface  https://review.opendev.org/c/openstack/kolla-ansible/+/81476708:08
hrwmnasiadka: during Yoga we should check which images are CentOS only (like qdrouterd) and deprecate/remove them.08:17
mnasiadkahrw: thanks for reminding, added to the PTG etherpad08:18
mnasiadkahrw: https://review.opendev.org/c/openstack/kolla/+/808165 - didn't we have an idea at some point to move it to Kolla codebase?08:19
hrwmnasiadka: yes, we had. note that this is backport08:21
mnasiadkakevko: do we have something that relies on it, that we want to backport as well?08:22
hrwmnasiadka: I will abandon 'infra images' patchset. this would need rewrite anyway and we are moving into one type 08:24
kevkomnasiadka: do you mean https://review.opendev.org/c/openstack/kolla/+/808165 where I add you to reviewer ? 08:24
mnasiadkaYes, asked in a change08:24
mnasiadkathe master change did not have Backport-Candidate set and I don't see any reference to another change that would be backported that would rely on this change08:25
mnasiadkaSo it seems we're backporting this for fun.08:25
kevkoanswered (probably there should be needed-by ? )08:27
kevkonothing what i am pushing for review is for fun :D, it is just not merged like my other +/- 25 patches :( 08:29
* hrw out08:30
mnasiadkaI don't see the answer - and backporting something that is not a bugfix in Wallaby probably should be further discussed - previously we've been very picky on backporting features - but if it's healthchecks related - we could probably do it.08:31
mnasiadkamgoddard: opinions?08:31
mgoddardmgoddard: no opinion08:36
mgoddardtbh I was a bit worried about the fernet healthchecks08:39
mgoddardso maybe we should leave it for xena08:39
kevkomnasiadka: sorry, didn't send reply in gerrit :F 08:39
kevko:D 08:39
mnasiadkakevko: well, neither of the patches has been merged in master, so I don't see why we would be doing the Kolla backport to Wallaby08:41
kevkoif no, then I am ok, but it will be fine to merge mariadb healthcheck as it is super usefull08:41
mnasiadkaonce it's merged, then yes - we can think of backporting both08:41
kevkomnasiadka: well, that one is already merged master .. k-a piece not ..08:42
mnasiadkakevko: yes, but I think it doesn't make any sense to backport the Kolla piece, if nothing will use it.08:43
kevkowell, it's only movement from one layer to another layer, so basically it is used ..but nevermind .. so could you check that mariadb healthcheck k-a part ? 08:44
mnasiadkaWill do, it's a bit misleading there are two patches :)08:45
mnasiadkaI mean conflicting.08:45
kevkoyeah, that is also reason why I'm asking.. , that healthcheck for mariadb is super useful to get quick overview of cluster health ..(both reviews) ... but mine patch[ is checking itself while Lin Peiwen's patch is checking clustercheck port , and there is no guarantee that clustercheck container is running08:48
kevkoand also, it's not adding any additional script, not repeadted logical lines of code in two scripts -> just move clustercheck script which is already adopted for a while (merged) and add ENV to container ..that's it 08:50
mnasiadkaSure, once the stable/xena patches will land and we can cut RC2 - we should try to review all of the unreviewed patches, just be a bit more patient.08:54
kevkomnasiadka: yeah, sure, thank you Michal 08:58
opendevreviewMerged openstack/kolla-ansible stable/wallaby: Update Manila deploy steps for Wallaby  https://review.opendev.org/c/openstack/kolla-ansible/+/81466609:13
eugenmayerwhat i'am wondering about though is, why all docker-images of the controller point to :master as a tag in my case. This seems not really production ready - did i use a strange 'for devs' flag during deployment without noticing it? or is that because i use xena (which is yet not released officiall09:15
mnasiadkaeugenmayer: no, you used stable/xena which is release candidate - and we're in progress to switching it to use xena tagged container images09:23
mnasiadkaeugenmayer: https://review.opendev.org/c/openstack/kolla-ansible/+/81485609:24
eugenmayerthank you (once again!)09:29
eugenmayeri have an opendev account, can i somehow subscribe to https://review.opendev.org/c/openstack/kolla-ansible/+/814856?09:30
opendevreviewMerged openstack/kolla-ansible stable/xena: mariadb: Remove wsrep-notify.sh  https://review.opendev.org/c/openstack/kolla-ansible/+/81466709:30
eugenmayer(not yet known to opendev )09:30
eugenmayer'add cc'?09:31
fricklereugenmayer: either that or you just "star" the change (on the top left of the GUI)09:34
eugenmayerfrickler, holy cow..ok .. that is an interesting UX decision09:39
eugenmayerThank you09:39
jingvarcan I separete kayobe/kolla/globals.yaml into separate folders or files ?09:45
opendevreviewPierre Riteau proposed openstack/kayobe master: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81378109:47
opendevreviewPierre Riteau proposed openstack/kayobe master: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81378109:47
opendevreviewPierre Riteau proposed openstack/kayobe master: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81378109:48
eugenmayerAfter setting up my entire openstack stack, i try to start an instance on one my of the computes. Creating the VM works, but when cirros boots, i see there is an early kernel panik10:15
eugenmayeranybody aware what that might be cause by?10:15
eugenmayerThe compute nodes are vanilla debian11 boxes with https://github.com/EugenMayer/openstack-lab/blob/master/compute/1_prepare_os.sh and everything that kolla-ansible did on them (installing qemu).10:16
eugenmayerhttps://gist.github.com/EugenMayer/f8b74f0ca5c470cd10f367ea06b2c5e310:20
eugenmayerrunning 5.10 as a kernel on the hypervisor compute node10:22
opendevreviewMichal Nasiadka proposed openstack/kayobe master: WIP: disable-selinux: Set to permissive  https://review.opendev.org/c/openstack/kayobe/+/81366110:22
opendevreviewMerged openstack/kayobe stable/victoria: CI: enable DNF tests on CentOS Stream 8  https://review.opendev.org/c/openstack/kayobe/+/81475310:26
opendevreviewVerification of a change to openstack/kayobe stable/wallaby failed: CI: enable DNF tests on CentOS Stream 8  https://review.opendev.org/c/openstack/kayobe/+/81475210:26
jingvarwhy services.d/rabbitmq.cfg consists     bind x.x.x.x:15672 -  usaly services use all rabbits ips 10:32
opendevreviewMerged openstack/kayobe stable/wallaby: Make setup module arguments configurable  https://review.opendev.org/c/openstack/kayobe/+/81178211:08
opendevreviewMichal Nasiadka proposed openstack/kayobe-config-dev master: Sync kayobe-config-dev with kayobe changes  https://review.opendev.org/c/openstack/kayobe-config-dev/+/81473711:18
opendevreviewMichal Nasiadka proposed openstack/kayobe-config-dev master: Sync kayobe-config-dev with kayobe changes  https://review.opendev.org/c/openstack/kayobe-config-dev/+/81473711:19
opendevreviewMichal Nasiadka proposed openstack/kayobe-config-dev master: Sync kayobe-config-dev with kayobe changes  https://review.opendev.org/c/openstack/kayobe-config-dev/+/81473711:19
opendevreviewMichal Nasiadka proposed openstack/kayobe-config-dev master: Sync kayobe-config-dev with kayobe changes  https://review.opendev.org/c/openstack/kayobe-config-dev/+/81473711:20
opendevreviewMichal Nasiadka proposed openstack/kolla master: ovn: Add Debian images  https://review.opendev.org/c/openstack/kolla/+/81490011:34
opendevreviewMark Goddard proposed openstack/kayobe master: Support Ansible diff mode  https://review.opendev.org/c/openstack/kayobe/+/80000312:00
DK4im trying a multinode deploy of kolla-ansible and it says on deploy that my nodes are missing docker module "The error was: ModuleNotFoundError: No module named 'docker'" shouldnt it be installed on deploy?12:05
eugenmayerDK4, it is yes - you do not need to install it on the target controller, it will be done by kolla-ansible12:09
eugenmayerwhich OS are you using on your target nodes?12:09
eugenmayerwait, no module, this rather sounds like python and thus rather smells like a wrong installation of kolla-ansible. Did you use virtenv?12:09
DK4no virtenv, im deploying on ubuntu 20 lts, i think i need to bootstrap-servers first in order to have dem configured properly before doing the deploy12:10
eugenmayerDK4, you should read https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html12:11
eugenmayeryour issue is not that a package called docker is missing on your target nodes12:12
eugenmayerbut rather that kolla-ansible on your deploy-host cannot find the python module 'docker' (i'am new here, so i could be totally wrong)12:12
yoctozeptohrw: now that you have a patch merged to tacker, let's propose your candidacy as Tacker PTL for the Z cycle12:13
*** amoralej is now known as amoralej|lunch12:14
eugenmayerwhich qemu/kvm virtualization driver does kolla-ansible deploy on the compute nodes?12:16
hrwyoctozepto: [...aj]13:06
hrw;P13:06
yoctozeptoxD13:07
*** amoralej|lunch is now known as amoralej13:12
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Drop '--infra-rename' argument  https://review.opendev.org/c/openstack/kolla/+/81491513:16
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: sort unbuildable images  https://review.opendev.org/c/openstack/kolla/+/81479813:20
opendevreviewMerged openstack/kayobe stable/victoria: Set proxy option in early dnf invocation  https://review.opendev.org/c/openstack/kayobe/+/81465913:20
opendevreviewVerification of a change to openstack/kayobe stable/wallaby failed: Set proxy option in early dnf invocation  https://review.opendev.org/c/openstack/kayobe/+/81465813:20
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: sort unbuildable images  https://review.opendev.org/c/openstack/kolla/+/81479813:21
opendevreviewPierre Riteau proposed openstack/kayobe master: Disable heat in upgrade jobs to save disk space  https://review.opendev.org/c/openstack/kayobe/+/81492013:51
opendevreviewRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Test Debian all  https://review.opendev.org/c/openstack/kolla-ansible/+/81461013:59
opendevreviewRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Test Debian all  https://review.opendev.org/c/openstack/kolla-ansible/+/81461013:59
opendevreviewMerged openstack/kolla master: monasca: enable for Debian/aarch64  https://review.opendev.org/c/openstack/kolla/+/81479614:00
opendevreviewPierre Riteau proposed openstack/kayobe stable/wallaby: Disable heat in upgrade jobs to save disk space  https://review.opendev.org/c/openstack/kayobe/+/81493214:08
opendevreviewPierre Riteau proposed openstack/kayobe stable/wallaby: Disable heat in upgrade jobs to save disk space  https://review.opendev.org/c/openstack/kayobe/+/81493214:09
opendevreviewMerged openstack/kayobe stable/victoria: Prevent Bifrost from using firewalld  https://review.opendev.org/c/openstack/kayobe/+/81470514:21
opendevreviewMerged openstack/kayobe stable/wallaby: Prevent Bifrost from using firewalld  https://review.opendev.org/c/openstack/kayobe/+/81470414:21
opendevreviewMerged openstack/kayobe master: CI: add host configure tests for firewalld  https://review.opendev.org/c/openstack/kayobe/+/81229914:21
opendevreviewMerged openstack/kayobe master: Remove stale config dump from seed hypervisor host configure  https://review.opendev.org/c/openstack/kayobe/+/81378614:21
opendevreviewMerged openstack/kayobe master: Add missing hook support for environment-create  https://review.opendev.org/c/openstack/kayobe/+/81457614:21
yoctozeptoheads-up no 1: Debian an official testing platform of Yoga :-)15:32
yoctozepto(and ovn works there)15:32
yoctozeptoheads-up no 2: https://review.rdoproject.org/r/c/rdo-website/+/36342 RDO no longer considers TripleO a production-ready deployment mechanism :/15:37
opendevreviewMerged openstack/kayobe stable/ussuri: Make setup module arguments configurable  https://review.opendev.org/c/openstack/kayobe/+/81178415:47
opendevreviewMerged openstack/kayobe master: Infra VMs: follow up  https://review.opendev.org/c/openstack/kayobe/+/81267015:47
opendevreviewMerged openstack/kayobe master: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81378115:47
jingvareugenmayer:  etc/kayobe/kolla/globals.yml look at nova_compute_virt_type:15:51
*** amoralej is now known as amoralej|off15:56
eugenmayerjingvar, thank you. If found out that the actual virsh tools are inside the container, that was the issue i had understanding how the compute node was provisioned. Beside i found, i'am still not sure why nested virtualization is not working - it is enabled and it was working with the training lab.16:01
eugenmayer Still, we decided we have been going far enough with the lab so we move on formalizing the deployer, it's config in chef for production, including the base node OS with tooling for production and the deploy into our DC next, and test all the VM / network options there (and maybe have to reset the nodes and cluster, but well, not a big issue with the automation i guess)16:03
eugenmayerMaybe this is a virsh 6 / xena isue on the current virtualBox 6.1.x or what so ever, probably not worth digging into it at this momemnt16:03
eugenmayeri will spin that thing up with qemu instead of kvm once, using your hint with nova_compute_virt_type - but i doubt that this is going to change anything16:05
jingvarfrom the doc16:07
jingvarUse QEMU as a lowest common16:07
jingvar# denominator.16:07
jingvarnova_compute_virt_type: qemu16:07
jingvarAs I understand this can help 16:09
opendevreviewMerged openstack/kayobe master: Support Ansible diff mode  https://review.opendev.org/c/openstack/kayobe/+/80000316:31
adrian-aHey. I've hit a bump here, let me know if you have some suggestion https://bugs.launchpad.net/kolla-ansible/+bug/1844710/comments/517:07
opendevreviewAdrian Andreias proposed openstack/kolla-ansible master: docs: Parameterize kolla-ansible version and branch  https://review.opendev.org/c/openstack/kolla-ansible/+/81504317:56
mnasiadkayoctozepto: does it mean tripleo goes stream?:)18:07
mnasiadkayoctozepto: RDO will never update packages for Xena - that sounds like fun.18:30
mnasiadkayoctozepto: I mean tripleo packages in Xena18:32
opendevreviewRadosław Piliszek proposed openstack/kolla stable/xena: openstack-base: drop anyjson  https://review.opendev.org/c/openstack/kolla/+/81493618:56
yoctozeptomnasiadka: tripleo yeah, goes no releases19:03
opendevreviewMerged openstack/kolla-ansible stable/xena: Use Xena images  https://review.opendev.org/c/openstack/kolla-ansible/+/81485620:02
opendevreviewMerged openstack/kolla-ansible stable/wallaby: docs: Improve info about neutron external interface  https://review.opendev.org/c/openstack/kolla-ansible/+/81476720:02
opendevreviewMerged openstack/kolla-ansible stable/xena: docs: Improve info about neutron external interface  https://review.opendev.org/c/openstack/kolla-ansible/+/81476620:02
opendevreviewPierre Riteau proposed openstack/kayobe stable/wallaby: Set proxy option in early dnf invocation  https://review.opendev.org/c/openstack/kayobe/+/81465820:53
opendevreviewPierre Riteau proposed openstack/kayobe stable/wallaby: CI: enable DNF tests on CentOS Stream 8  https://review.opendev.org/c/openstack/kayobe/+/81475220:53
opendevreviewMaksim Malchuk proposed openstack/kayobe stable/wallaby: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81507421:08
opendevreviewMaksim Malchuk proposed openstack/kayobe stable/victoria: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81507521:08
opendevreviewMaksim Malchuk proposed openstack/kayobe stable/ussuri: Add the missing documentation for mariadb recovery  https://review.opendev.org/c/openstack/kayobe/+/81507621:09

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!