| *** gmann_afk is now known as gmann | 00:19 | |
| dardelean | does kolla deploy have any issues with controllers and computes are not on the same L2? traffic is routed | 00:21 |
|---|---|---|
| *** maharg101 has joined #openstack-kolla | 01:38 | |
| *** maharg101 has quit IRC | 01:43 | |
| *** bengates has joined #openstack-kolla | 02:05 | |
| *** bengates_ has joined #openstack-kolla | 02:08 | |
| *** bengates has quit IRC | 02:08 | |
| *** bengates_ has quit IRC | 02:12 | |
| *** bengates has joined #openstack-kolla | 02:15 | |
| *** bengates has quit IRC | 02:20 | |
| *** bengates has joined #openstack-kolla | 02:27 | |
| *** hrw has quit IRC | 02:27 | |
| *** bengates has quit IRC | 02:31 | |
| *** skramaja_ has joined #openstack-kolla | 02:35 | |
| *** skramaja has quit IRC | 02:37 | |
| *** bengates has joined #openstack-kolla | 02:37 | |
| *** bengates_ has joined #openstack-kolla | 02:39 | |
| *** bengates has quit IRC | 02:39 | |
| *** bengates_ has quit IRC | 02:43 | |
| *** bengates has joined #openstack-kolla | 02:47 | |
| *** bengates has quit IRC | 02:51 | |
| *** bengates has joined #openstack-kolla | 02:54 | |
| *** bengates_ has joined #openstack-kolla | 02:58 | |
| *** bengates has quit IRC | 02:59 | |
| *** bengates has joined #openstack-kolla | 03:00 | |
| *** bengates_ has quit IRC | 03:03 | |
| *** bengates has quit IRC | 03:05 | |
| *** bengates has joined #openstack-kolla | 03:08 | |
| *** bengates has quit IRC | 03:13 | |
| *** bengates has joined #openstack-kolla | 03:15 | |
| *** bengates has quit IRC | 03:19 | |
| *** bengates has joined #openstack-kolla | 03:22 | |
| *** bengates has quit IRC | 03:27 | |
| *** LinPeiWen has quit IRC | 03:28 | |
| *** bengates has joined #openstack-kolla | 03:29 | |
| *** bengates_ has joined #openstack-kolla | 03:34 | |
| *** bengates has quit IRC | 03:34 | |
| *** bengates_ has quit IRC | 03:38 | |
| *** bengates has joined #openstack-kolla | 03:39 | |
| *** bengates has quit IRC | 03:43 | |
| *** bengates has joined #openstack-kolla | 03:50 | |
| *** bengates has quit IRC | 03:55 | |
| *** zzzeek has quit IRC | 03:56 | |
| *** zzzeek has joined #openstack-kolla | 03:56 | |
| *** bengates has joined #openstack-kolla | 03:58 | |
| *** bengates has quit IRC | 04:02 | |
| *** bengates has joined #openstack-kolla | 04:06 | |
| *** bengates has quit IRC | 04:11 | |
| *** jopdorp has quit IRC | 04:22 | |
| *** stackedsax has quit IRC | 04:26 | |
| *** stackedsax has joined #openstack-kolla | 04:28 | |
| *** eliaswimmer has quit IRC | 04:37 | |
| *** mrunge has quit IRC | 04:37 | |
| *** calbers has quit IRC | 04:37 | |
| *** eliaswimmer has joined #openstack-kolla | 04:42 | |
| *** mrunge has joined #openstack-kolla | 04:42 | |
| *** calbers has joined #openstack-kolla | 04:42 | |
| *** openstackstatus has quit IRC | 04:58 | |
| *** openstack has joined #openstack-kolla | 05:01 | |
| *** ChanServ sets mode: +o openstack | 05:01 | |
| *** cah_link has joined #openstack-kolla | 06:09 | |
| *** vishalmanchanda has joined #openstack-kolla | 06:12 | |
| *** k_mouza has joined #openstack-kolla | 06:22 | |
| *** k_mouza has quit IRC | 06:26 | |
| *** hrw has joined #openstack-kolla | 07:14 | |
| yoctozepto | dardelean: controllers are assumed to share the same L2 due to keepalived | 07:42 |
| yoctozepto | computes can be elsewhere | 07:42 |
| yoctozepto | as long as it's all routable | 07:42 |
| *** rpittau|afk is now known as rpittau | 07:51 | |
| *** maharg101 has joined #openstack-kolla | 08:13 | |
| *** bengates has joined #openstack-kolla | 08:15 | |
| openstackgerrit | Piotr Parczewski proposed openstack/kayobe master: [docs] Improve contributor's automated setup document https://review.opendev.org/c/openstack/kayobe/+/774777 | 08:19 |
| *** andrewbonney has joined #openstack-kolla | 08:21 | |
| *** gfidente has joined #openstack-kolla | 08:21 | |
| *** bengates has quit IRC | 08:22 | |
| *** bengates has joined #openstack-kolla | 08:22 | |
| *** Tony31 has joined #openstack-kolla | 08:27 | |
| Tony31 | Hello. For deploying openstack using kayobe, can I use a host OS of Ubuntu 20 ? The doc lists only CentOS. Is this correct? Kolla documentation lists Ubuntu 20 as supported. Thank you | 08:28 |
| Tony31 | Ref: https://docs.openstack.org/kayobe/victoria/support-matrix.html | 08:30 |
| Tony31 | I will use kolla-ansible if not supported. | 08:30 |
| *** bengates has quit IRC | 08:40 | |
| parallax | Hi Tony31 - Ubuntu support is work in progress: https://storyboard.openstack.org/#!/story/2004960 | 08:40 |
| *** bengates has joined #openstack-kolla | 08:41 | |
| *** e0ne has joined #openstack-kolla | 08:41 | |
| mnasiadka | morning | 08:44 |
| Tony31 | Thank you Parallax | 08:53 |
| *** dougsz has joined #openstack-kolla | 08:53 | |
| *** dougsz has quit IRC | 08:54 | |
| *** e0ne has quit IRC | 09:04 | |
| *** e0ne has joined #openstack-kolla | 09:05 | |
| hrw | moning | 09:49 |
| openstackgerrit | Maksim Malchuk proposed openstack/kolla-ansible master: Correctly configure S3 Token Middleware for Swift https://review.opendev.org/c/openstack/kolla-ansible/+/774791 | 10:06 |
| *** kevko_ has joined #openstack-kolla | 10:06 | |
| *** kevko_ has quit IRC | 10:09 | |
| hrw | so how we are with CI? | 10:32 |
| *** e0ne has quit IRC | 10:53 | |
| hrw | https://review.opendev.org/c/openstack/kolla-ansible/+/774622 blocks all? | 10:55 |
| *** dougsz has joined #openstack-kolla | 10:57 | |
| *** e0ne has joined #openstack-kolla | 11:03 | |
| *** k_mouza has joined #openstack-kolla | 11:27 | |
| *** dougsz has quit IRC | 11:28 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 11:49 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Refactor mariadb to support shards https://review.opendev.org/c/openstack/kolla-ansible/+/766952 | 11:49 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Rename role haproxy to loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/770618 | 11:49 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 11:49 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 11:49 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 11:49 |
| hrw | this will not fix the problem but rather show that approach was bad | 11:51 |
| hrw | 2021-02-10 12:00:13.132707 | TASK [Record the running state of the environment as seen by the setup module] | 12:02 |
| hrw | 2021-02-10 12:00:13.642447 | primary | /bin/sh: 1: ansible: not found | 12:02 |
| hrw | yay | 12:02 |
| hrw | there needs to be virtualenv in use... | 12:04 |
| mgoddard | hrw: or full path | 12:05 |
| mgoddard | hrw: linters is also broken | 12:05 |
| hrw | mgoddard: then some shell script fail | 12:05 |
| *** e0ne has quit IRC | 12:05 | |
| *** dougsz has joined #openstack-kolla | 12:07 | |
| hrw | anyway as we run shell scripts then all of them need to be changed (or their environment) as we go --user or venv | 12:07 |
| mgoddard | hrw: are you working on it or shall I? | 12:08 |
| hrw | go for it | 12:09 |
| hrw | mgoddard: looks like moving most of shell calls into one block would be a good way so there would be one place where environment: PATH will be set | 12:10 |
| *** dougsz has quit IRC | 12:10 | |
| mgoddard | hrw: or set at the playbook level? | 12:11 |
| hrw | or | 12:11 |
| mgoddard | s/playbook/play/ | 12:11 |
| hrw | mgoddard: you have more ansible experience | 12:11 |
| hrw | ~/.local/bin needs to be at start of $PATH | 12:11 |
| *** glanciano has joined #openstack-kolla | 12:14 | |
| mgoddard | hrw: why at the start? | 12:16 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 12:17 |
| *** kevko has joined #openstack-kolla | 12:17 | |
| hrw | mgoddard: pip3 from just installed pip 19+ or pip 9 from /usr/bin/pip3? | 12:19 |
| mgoddard | hrw: ok | 12:20 |
| *** e0ne has joined #openstack-kolla | 12:22 | |
| glanciano | Hi! AFAICS, at the moment it is not possible to deploy Monasca in dev mode. Is there someone already working on this? | 12:27 |
| hrw | mgoddard: add "--exclude releasenotes" to ansible-lint | 12:30 |
| hrw | (venv3) 13:29 (s) hrw@puchatek:kolla-ansible$ ansible-lint -p --exclude tests --exclude roles --exclude etc -vv --exclude releasenotes/ | 12:30 |
| hrw | Finished with 0 failure(s), 7 warning(s) on 1326 files. | 12:30 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Refactor mariadb to support shards https://review.opendev.org/c/openstack/kolla-ansible/+/766952 | 12:32 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Rename role haproxy to loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/770618 | 12:32 |
| *** glanciano has quit IRC | 12:32 | |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 12:32 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 12:32 |
| openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 12:32 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 13:23 |
| *** hcash has joined #openstack-kolla | 14:17 | |
| hcash | Has anyone an idea where kolla-ansible's "Running Nova API bootstrap container" task caches database credentials? I changed the `kolla_internal_vip_address` but it is still using the old address. All files in /etc/kolla contain the correct IP... also in the container. But the nova-manage.log still reports `Can't connect to MySQL server on '[[OLD | 14:21 |
| hcash | IP]]'` | 14:21 |
| hcash | I already nuked the bootstrap container and greped for the old IP. Everything looks perfect | 14:23 |
| *** bengates has quit IRC | 14:25 | |
| *** zeestrat has joined #openstack-kolla | 14:26 | |
| *** alistarle has joined #openstack-kolla | 14:27 | |
| *** alistarle has quit IRC | 14:35 | |
| *** alistarle has joined #openstack-kolla | 14:35 | |
| mnasiadka | hcash: tried checking /etc/kolla/nova* for nova.conf on the host running bootstrap? | 14:35 |
| hcash | mnasiadka, yes `sudo grep -r [old ip] /etc/kolla` returns nothing on deploy or bootstrap host | 14:40 |
| *** bengates has joined #openstack-kolla | 14:42 | |
| *** bengates has quit IRC | 14:42 | |
| *** bengates has joined #openstack-kolla | 14:43 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 14:46 |
| priteau | hcash: Check you cell v2 configuration? Via `docker exec nova_api nova-manage cell_v2 list_cells` if you have such a container, or directly in the nova_api DB, cell_mappings table. | 14:48 |
| hcash | priteau aaaah thank you! `cell_mappings.database_connection` still contained the old vip | 14:57 |
| priteau | You have nova-manage commands to update i | 14:57 |
| priteau | update it | 14:57 |
| mgoddard | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak | 14:57 |
| mgoddard | ^ meeting in 3 | 14:58 |
| mnasiadka | hcash: feel free to raise a bug with your findings, it should be straightforward :) | 14:58 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Make kolla_set_configs open files in binary mode https://review.opendev.org/c/openstack/kolla/+/773257 | 14:58 |
| hrw | thx mgoddard | 14:58 |
| yoctozepto | mgoddard meeting meeting | 15:00 |
| mgoddard | yes yes | 15:01 |
| mgoddard | #startmeeting kolla | 15:01 |
| openstack | Meeting started Wed Feb 10 15:01:22 2021 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
| *** openstack changes topic to " (Meeting topic: kolla)" | 15:01 | |
| openstack | The meeting name has been set to 'kolla' | 15:01 |
| mgoddard | #topic rollcall | 15:01 |
| *** openstack changes topic to "rollcall (Meeting topic: kolla)" | 15:01 | |
| mgoddard | \o | 15:01 |
| yoctozepto | *o* | 15:01 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:02 |
| hrw | Io/ | 15:02 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 15:03 |
| mgoddard | #topic agenda | 15:04 |
| *** openstack changes topic to "agenda (Meeting topic: kolla)" | 15:04 | |
| mgoddard | * Roll-call | 15:04 |
| mgoddard | * Announcements | 15:04 |
| mgoddard | * Review action items from the last meeting | 15:04 |
| mgoddard | * CI status | 15:04 |
| mgoddard | * CI resources & performance | 15:04 |
| mgoddard | ** http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020235.html | 15:04 |
| mgoddard | ** https://docs.google.com/spreadsheets/d/1U_TaLoLTpX7ZC3v9cQsvBfoJtUI7ClAhkx0HYvUaFrM/edit#gid=1217205417 | 15:04 |
| mgoddard | * Review requests | 15:04 |
| mgoddard | * Wallaby release planning | 15:04 |
| mgoddard | #topic announcements | 15:04 |
| *** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:04 | |
| mgoddard | Nothing from me. Anyone else? | 15:04 |
| hrw | no | 15:05 |
| mgoddard | #topic Review action items from the last meeting | 15:05 |
| *** openstack changes topic to "Review action items from the last meeting (Meeting topic: kolla)" | 15:05 | |
| mgoddard | hrw to document allowed to fail & unbuildable images in contributor docs | 15:05 |
| mgoddard | hrw to build libvirt 7.0.0 for Debian | 15:05 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/774618 | 15:06 |
| mgoddard | both proposed | 15:06 |
| hrw | https://review.opendev.org/c/openstack/kolla/+/774351 | 15:06 |
| hrw | allowed to fail are not documented yet - have to find a place where to put it | 15:06 |
| mgoddard | true | 15:07 |
| mgoddard | #topic CI status | 15:07 |
| *** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:07 | |
| mgoddard | RED | 15:07 |
| mgoddard | https://review.opendev.org/c/openstack/kolla-ansible/+/774622 is the WIP fix | 15:07 |
| hrw | train is RED too. https://review.opendev.org/c/openstack/kolla/+/774602 fixes sensu/ubuntu but bifrost is broken for centos 7/8 | 15:08 |
| hrw | ussuri is RED on aarch64 - https://review.opendev.org/c/openstack/kolla/+/774493 fixes it | 15:09 |
| mgoddard | #action mgoddard fix bifrost on Train | 15:10 |
| mgoddard | anything else for CI? | 15:12 |
| hrw | probably that's all | 15:13 |
| mgoddard | #topic CI resources & performance | 15:13 |
| *** openstack changes topic to "CI resources & performance (Meeting topic: kolla)" | 15:13 | |
| mgoddard | There is a thread on openstack-discuss | 15:13 |
| mgoddard | #link http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020235.html | 15:13 |
| mgoddard | about CI resource usage | 15:13 |
| mgoddard | kolla is in the top 5 | 15:13 |
| mgoddard | although way behind tripleo | 15:14 |
| mgoddard | Project % of total Node Hours Nodes | 15:14 |
| mgoddard | ------------------------------------------ | 15:14 |
| mgoddard | 1. TripleO 38% 31 hours 20 | 15:14 |
| mgoddard | 2. Neutron 13% 38 hours 32 | 15:14 |
| mgoddard | 3. Nova 9% 21 hours 25 | 15:14 |
| mgoddard | 4. Kolla 5% 12 hours 18 | 15:14 |
| mgoddard | 5. OSA 5% 22 hours 17 | 15:14 |
| mnasiadka | but we have less node hours than OSA | 15:14 |
| hrw | were we asked to lower usage? | 15:14 |
| mnasiadka | whatever that means | 15:14 |
| mgoddard | node hours is per changeset | 15:15 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:15 |
| hrw | mnasiadka: node hours == how much time CI run takes | 15:15 |
| hrw | that's how I understood it | 15:15 |
| yoctozepto | yeah and I think this was max | 15:15 |
| yoctozepto | so it's very biased | 15:15 |
| mnasiadka | True, I don't understand what Nodes mean, like at the same time? | 15:16 |
| mgoddard | how many nodes to run jobs for one changeset | 15:16 |
| yoctozepto | again, I think it's max for changeset | 15:16 |
| yoctozepto | mhm | 15:16 |
| mgoddard | I would expect average | 15:16 |
| mgoddard | max for kolla could be huge | 15:16 |
| yoctozepto | true that | 15:17 |
| yoctozepto | though then nodehours | 15:17 |
| yoctozepto | can't be avg | 15:17 |
| mgoddard | why not? | 15:17 |
| yoctozepto | you think we average 12 hours? | 15:17 |
| mgoddard | sure | 15:17 |
| mgoddard | most jobs are ~1 hour | 15:17 |
| mgoddard | some more | 15:17 |
| yoctozepto | ah, you mean like consumed node time | 15:18 |
| mgoddard | multinode upgrade jobs might be 4-5 node hours | 15:18 |
| yoctozepto | true | 15:18 |
| yoctozepto | makes sense | 15:18 |
| mgoddard | nodes * time = node hours | 15:18 |
| yoctozepto | yeah, yeah | 15:18 |
| mgoddard | i.e. what you would pay for on a cloud | 15:18 |
| yoctozepto | now it makes sense | 15:18 |
| mgoddard | anyways | 15:18 |
| mgoddard | I promised I would raise it at this meeting | 15:19 |
| mgoddard | now let's move on | 15:19 |
| mgoddard | :p | 15:19 |
| yoctozepto | you are my hero | 15:19 |
| mgoddard | #link https://docs.google.com/spreadsheets/d/1U_TaLoLTpX7ZC3v9cQsvBfoJtUI7ClAhkx0HYvUaFrM/edit#gid=1217205417 | 15:19 |
| mgoddard | I did some playing with zuul API | 15:19 |
| mgoddard | last 1000 jobs across all branches & kolla projects | 15:20 |
| mgoddard | pivot table is most interesting | 15:20 |
| mgoddard | I expect we could slice the data more | 15:21 |
| yoctozepto | top offenders - ceph jobs | 15:22 |
| mgoddard | yes | 15:22 |
| yoctozepto | though I wonder if it's simply because of using ceph | 15:22 |
| yoctozepto | or because we have to bring up and test cinder | 15:23 |
| mgoddard | interestingly the scenario jobs are generally very low | 15:23 |
| yoctozepto | they run much less often | 15:23 |
| mgoddard | right | 15:23 |
| mgoddard | but that does make a good argument against moving them to experimental | 15:23 |
| mnasiadka | Well, all ceph jobs use ceph-ansible to deploy, it takes reasonable amount of time to deploy even a tiny ceph cluster | 15:23 |
| mgoddard | assuming 1000 jobs is enough data | 15:23 |
| mgoddard | mnasiadka: multinode is the main killer - immediate 3x | 15:24 |
| yoctozepto | yeah, but | 15:24 |
| yoctozepto | compare with multinode ipv6 | 15:24 |
| yoctozepto | about 75% penalty on ceph jobs | 15:25 |
| *** glanciano has joined #openstack-kolla | 15:25 | |
| yoctozepto | but in them we do ceph and cinder | 15:25 |
| mgoddard | yes, but significant extra coverage | 15:25 |
| yoctozepto | yeah, but I do wonder if we switch to cephadm | 15:25 |
| yoctozepto | or perhaps to a custom way | 15:25 |
| yoctozepto | (we don't really do a production cluster anyway) | 15:26 |
| mnasiadka | then we can compare once we switch, I have time to work on that now - so we'll know next week I guess. | 15:26 |
| yoctozepto | great | 15:26 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:26 |
| mnasiadka | we probably could have even a single node ceph | 15:26 |
| yoctozepto | and then only continue with optimisations we do for k-a ansible | 15:26 |
| mnasiadka | and it wouldn't hurt | 15:26 |
| mgoddard | should we consider two nodes in multinode jobs? | 15:27 |
| mnasiadka | (if there's a big time difference between deploying on 1 and 3 nodes) :) | 15:27 |
| yoctozepto | mnasiadka: true that | 15:27 |
| yoctozepto | mgoddard: we can evaluate that as well | 15:27 |
| mnasiadka | two nodes? I don't think it's our recommendation in HA - the docs recommend 3 nodes? | 15:27 |
| mgoddard | mnasiadka: from a resource usage perspective it's 3x | 15:27 |
| yoctozepto | mgoddard, mnasiadka: we can still test clustering correctness | 15:27 |
| yoctozepto | in jobs like mariadb | 15:27 |
| yoctozepto | which are much faster | 15:27 |
| yoctozepto | we could create a simple job that test mariadb+rabbitmq+anything_else_that_we_need_to_ha_cluster | 15:28 |
| yoctozepto | and avoid multinode in biggur scenarios | 15:28 |
| yoctozepto | we could use some unit testing | 15:28 |
| yoctozepto | if we loop correctly in templates | 15:28 |
| yoctozepto | and things like that | 15:28 |
| yoctozepto | but it is much less likely to break it | 15:29 |
| yoctozepto | than simply break the process of deploying/upgrading clustered resources | 15:29 |
| yoctozepto | we can also avoid multinode jobs duplicates per distro | 15:30 |
| yoctozepto | as it's less likely that clustering logic breaks between distros | 15:30 |
| mgoddard | it is good to test 3 nodes. I guess we just need to be careful about when we use it | 15:30 |
| yoctozepto | it's more likely that we break altogether | 15:30 |
| yoctozepto | let's do it in small steps | 15:31 |
| mgoddard | we could do centos multinode ceph, ubuntu single node ceph | 15:31 |
| yoctozepto | perhaps ditching ceph-ansible will give us a sane boost to performance and reliability | 15:31 |
| mgoddard | then the reverse for the ceph upgrade jobs | 15:31 |
| yoctozepto | yeah, that works too | 15:31 |
| mnasiadka | we could run ceph upgrade jobs as periodicals, I think they are the main killer :) | 15:31 |
| hrw | if weeknumber is odd then centos. if even then ubuntu | 15:32 |
| mnasiadka | they/those | 15:32 |
| yoctozepto | yup, periodics are another path | 15:32 |
| yoctozepto | hrw: oh, I like it | 15:32 |
| hrw | such check would not require changing CI scripts | 15:32 |
| yoctozepto | random CI | 15:32 |
| yoctozepto | and then | 15:32 |
| yoctozepto | let's merge it on an odd week | 15:32 |
| yoctozepto | because it fails on even ones | 15:32 |
| mgoddard | let's try to keep it predictable :) | 15:34 |
| mgoddard | ok, I think we have some things to try | 15:35 |
| mgoddard | I'm going to propose restricting the cells job to nova changes | 15:36 |
| yoctozepto | and soon mariadb ones | 15:37 |
| yoctozepto | (for shards) | 15:37 |
| mgoddard | right | 15:37 |
| mgoddard | #topic Review requests | 15:37 |
| *** jamesbenson has joined #openstack-kolla | 15:37 | |
| *** openstack changes topic to "Review requests (Meeting topic: kolla)" | 15:37 | |
| mgoddard | Same as last time | 15:37 |
| mgoddard | Does anyone have a patch they would like to be reviewed? | 15:37 |
| hrw | one patch: https://review.opendev.org/c/openstack/kolla/+/772507 | 15:38 |
| hrw | passed zuul, yoctozepto gave +2 | 15:38 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Lint and fix renos https://review.opendev.org/c/openstack/kolla-ansible/+/759370 | 15:39 |
| yoctozepto | well, you can w+1 ^ | 15:39 |
| yoctozepto | I am a bit overloaded and don't seem to have anything ready proposed at the moment | 15:40 |
| hrw | lgtm | 15:41 |
| mgoddard | don't +A | 15:41 |
| mgoddard | linters job is broken | 15:41 |
| yoctozepto | oh | 15:41 |
| yoctozepto | sad | 15:41 |
| mgoddard | yeah | 15:41 |
| mgoddard | ansible-lint | 15:41 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:41 |
| yoctozepto | argh | 15:41 |
| mgoddard | worst linter ever | 15:41 |
| mnasiadka | ansible-lint broken again? | 15:41 |
| yoctozepto | just drop it | 15:42 |
| hrw | mgoddard: add --exclude releasenotes and be done | 15:42 |
| yoctozepto | or make it non-voting | 15:42 |
| mgoddard | hrw: not required | 15:42 |
| mgoddard | it's fixed in my patch | 15:42 |
| hrw | mgoddard: ok | 15:42 |
| yoctozepto | let it help us, not hinder us | 15:42 |
| mgoddard | we put in the effort to get it passing, may as well keep it | 15:42 |
| yoctozepto | we did | 15:43 |
| mgoddard | I'm going to make https://review.opendev.org/c/openstack/kolla-ansible/+/695432/ my review request this week | 15:43 |
| yoctozepto | that's a review bomb for sure | 15:43 |
| mgoddard | boom | 15:44 |
| yoctozepto | it conflicts with half the proposed changes as well I see :-) | 15:44 |
| yoctozepto | tough nut | 15:44 |
| mgoddard | indeed | 15:44 |
| hrw | can we add 'must pass zuul' to this request queue? | 15:44 |
| mnasiadka | mgoddard: eek, balance source in haproxy? | 15:45 |
| hrw | or 'must pass zuul if core wrote' ;D | 15:45 |
| hrw | ok, I have to go. thanks for meeting | 15:45 |
| mgoddard | mnasiadka: it that's exactly the kind of comment that should be in the review :) | 15:45 |
| mgoddard | thanks headphoneJames | 15:45 |
| mgoddard | sorry, hrw | 15:45 |
| mgoddard | back to sleep headphoneJames | 15:46 |
| mnasiadka | mgoddard: I'll force myself to review that tomorrow ;) | 15:46 |
| mgoddard | it has been around for a long time and the authors have been quite responsive, helpful, and receptive to change | 15:46 |
| mgoddard | I'd like to at least merge that patch and adapt it if we need to | 15:46 |
| mgoddard | mnasiadka: just don't mention SAML ;) | 15:47 |
| mnasiadka | Of course I will, OIDC is not 90% of usage, I would rather claim otherwise :) | 15:47 |
| yoctozepto | I have added my first comment | 15:48 |
| mnasiadka | Anyway, let's not incite religious wars. | 15:48 |
| yoctozepto | XML4life | 15:48 |
| mnasiadka | I never had the ability to read XML files. | 15:48 |
| yoctozepto | they are the best test for patience | 15:48 |
| mnasiadka | It's like obfuscated poetry from an oppressed country. | 15:49 |
| yoctozepto | hahahaha | 15:49 |
| mgoddard | it stands for XXXX my life | 15:49 |
| yoctozepto | the water I was drinking left me through my nose | 15:49 |
| yoctozepto | thank you very much | 15:49 |
| mgoddard | hope your computer is dry | 15:49 |
| yoctozepto | drying it | 15:49 |
| yoctozepto | don't drink and kollameet | 15:50 |
| mgoddard | #topic Wallaby release planning | 15:51 |
| *** openstack changes topic to "Wallaby release planning (Meeting topic: kolla)" | 15:51 | |
| mnasiadka | Around reviews - can somebody look at this: https://review.opendev.org/c/openstack/kolla-ansible/+/774222 ? It solves cases when Neutron does not support OVN native DHCP yet. | 15:51 |
| yoctozepto | wallaby is the cycle we spend on fixing our ci fires | 15:51 |
| mgoddard | Mar 29 - Apr 02 | 15:51 |
| mgoddard | yoctozepto: every cycle is the cycle we spend on fixing our ci fires | 15:52 |
| mnasiadka | and not only our | 15:52 |
| yoctozepto | this one feels sadder | 15:52 |
| mgoddard | I'll take a look | 15:53 |
| mgoddard | mnasiadka: what are those cases? bare metal? | 15:53 |
| mnasiadka | mgoddard: yes, bare metal | 15:53 |
| mnasiadka | OVN has functionality that would fit, but Neutron seems to be lacking here and there... | 15:54 |
| mgoddard | hmm, surprised lucasgomes is not on top of it | 15:55 |
| mnasiadka | well, he's the one proposing to use neutron-dhcp-agent ) | 15:55 |
| mnasiadka | :) | 15:55 |
| mgoddard | ah | 15:55 |
| mgoddard | well there is no better intersection of ironic & ovn knowledge | 15:55 |
| *** skramaja_ has quit IRC | 15:56 | |
| mgoddard | #topic open discussion | 15:56 |
| *** openstack changes topic to "open discussion (Meeting topic: kolla)" | 15:56 | |
| mgoddard | anyone have anything this week? | 15:56 |
| *** alistarle has quit IRC | 15:58 | |
| mnasiadka | I was thinking about extending kayobe's seed-custom-containers to overcloud-custom-containers - is that something we want to do? | 15:58 |
| mgoddard | mnasiadka: I guess I see kolla-ansible as the thing that deploys containers | 15:58 |
| mgoddard | is there a particular use case in mind? | 15:59 |
| yoctozepto | nothing new from me | 15:59 |
| mnasiadka | well, there are some components, that are not covered by kolla-ansible - I can write an external role to do it, or just do ^^ | 15:59 |
| mnasiadka | like I don't know... consul and hashicorp vault | 16:00 |
| mgoddard | I see | 16:00 |
| mgoddard | it probably needs some thought | 16:01 |
| mnasiadka | for sure, we need to make sure we don't have a naming collision for containers and volumes | 16:01 |
| mnasiadka | which might be not so straightforward to write | 16:02 |
| mgoddard | we did discuss making some plugin/extension mechanism for kolla-ansible at the PTG | 16:02 |
| mgoddard | prometheus exporters being the initial use case | 16:02 |
| mgoddard | but potentiall this fits too | 16:02 |
| mnasiadka | yeah, while adding roles from a different path (or using ansible galaxy) is not problematic, we would need to find a way to resolve dependencies/order of running | 16:02 |
| *** jamesbenson has quit IRC | 16:03 | |
| mgoddard | I think there is a question of when you would want to deploy those things - are they coupled to openstack, or not? | 16:03 |
| mgoddard | probably you would not want to touch vault too often | 16:03 |
| mnasiadka | well - touching vault means unsealing - so no :) | 16:04 |
| mgoddard | anyway, meeting over folks | 16:04 |
| mgoddard | thanks | 16:04 |
| mgoddard | #endmeeting | 16:04 |
| *** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:04 | |
| openstack | Meeting ended Wed Feb 10 16:04:16 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:04 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-10-15.01.html | 16:04 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-10-15.01.txt | 16:04 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-10-15.01.log.html | 16:04 |
| *** jamesbenson has joined #openstack-kolla | 16:08 | |
| *** jamesbenson has quit IRC | 16:09 | |
| *** jamesbenson has joined #openstack-kolla | 16:09 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 16:12 |
| *** jamesbenson has quit IRC | 16:20 | |
| *** jamesbenson has joined #openstack-kolla | 16:20 | |
| *** jamesbenson has quit IRC | 16:30 | |
| openstackgerrit | Pedro Henrique Pereira Martins proposed openstack/kolla-ansible master: Add support to OpenID Connect Authentication flow https://review.opendev.org/c/openstack/kolla-ansible/+/695432 | 16:32 |
| *** numans has quit IRC | 16:38 | |
| *** ianw has quit IRC | 16:39 | |
| *** ianw has joined #openstack-kolla | 16:39 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 16:39 |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: only run cells job on changes to nova role https://review.opendev.org/c/openstack/kolla-ansible/+/774894 | 16:39 |
| *** bengates has quit IRC | 16:40 | |
| *** glanciano_ has joined #openstack-kolla | 16:48 | |
| *** glanciano has quit IRC | 16:51 | |
| *** e0ne has quit IRC | 17:16 | |
| *** jamesbenson has joined #openstack-kolla | 17:20 | |
| *** e0ne has joined #openstack-kolla | 17:22 | |
| *** dougsz has joined #openstack-kolla | 17:23 | |
| openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 17:27 |
| *** dougsz has quit IRC | 17:28 | |
| *** glanciano_ has quit IRC | 17:33 | |
| *** hcash has quit IRC | 17:37 | |
| *** maharg101 has quit IRC | 17:46 | |
| *** jamesbenson has quit IRC | 17:47 | |
| *** jbadiapa has quit IRC | 17:51 | |
| *** jonaspaulo has joined #openstack-kolla | 17:52 | |
| *** jamesbenson has joined #openstack-kolla | 17:58 | |
| hrw | 16:52 < mgoddard> yoctozepto: every cycle is the cycle we spend on fixing our ci fires | 17:58 |
| hrw | I try to leave ci fires you everyone else and then some sparks land in my basket... | 17:59 |
| *** jamesbenson has quit IRC | 18:01 | |
| *** jamesbenson has joined #openstack-kolla | 18:01 | |
| *** e0ne has quit IRC | 18:19 | |
| *** jamesbenson has quit IRC | 18:21 | |
| *** jamesbenson has joined #openstack-kolla | 18:21 | |
| *** rpittau is now known as rpittau|afk | 18:23 | |
| *** gfidente has quit IRC | 18:28 | |
| *** jamesbenson has quit IRC | 18:28 | |
| *** e0ne has joined #openstack-kolla | 18:29 | |
| *** e0ne has quit IRC | 18:29 | |
| *** jamesbenson has joined #openstack-kolla | 18:33 | |
| *** k_mouza has quit IRC | 18:33 | |
| *** jamesbenson has quit IRC | 18:49 | |
| *** jamesbenson has joined #openstack-kolla | 18:49 | |
| *** jonaspaulo has quit IRC | 18:56 | |
| openstackgerrit | Pedro Henrique Pereira Martins proposed openstack/kolla-ansible master: Add support to OpenID Connect Authentication flow https://review.opendev.org/c/openstack/kolla-ansible/+/695432 | 18:56 |
| *** jamesbenson has quit IRC | 19:04 | |
| *** jamesbenson has joined #openstack-kolla | 19:06 | |
| *** jamesbenson has quit IRC | 19:12 | |
| *** jamesbenson has joined #openstack-kolla | 19:14 | |
| *** jamesbenson has quit IRC | 19:17 | |
| *** jamesbenson has joined #openstack-kolla | 19:18 | |
| *** jamesbenson has quit IRC | 19:22 | |
| *** jamesbenson has joined #openstack-kolla | 19:23 | |
| *** zzzeek has quit IRC | 19:26 | |
| hrw | mgoddard: WARNING: The script alembic is installed in '/home/zuul/.local/bin' which is not on PATH. | 19:26 |
| hrw | mgoddard: looks like your trick with PATH failed ;( | 19:26 |
| *** zzzeek has joined #openstack-kolla | 19:27 | |
| yoctozepto | mgoddard, hrw: let's just do the venv way now | 19:29 |
| yoctozepto | no more happy hacks | 19:29 |
| hrw | yoctozepto: the problem is that each script call has then to initialize virtualenv | 19:35 |
| hrw | argh | 19:35 |
| hrw | each shell, each command call | 19:36 |
| hrw | or am I wrong? | 19:36 |
| *** jamesbenson has quit IRC | 19:38 | |
| *** jamesbenson has joined #openstack-kolla | 19:38 | |
| hrw | just added some debug output | 19:38 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 19:38 |
| hrw | it will fail | 19:38 |
| *** andrewbonney has quit IRC | 19:39 | |
| *** jamesbenson has quit IRC | 19:42 | |
| hrw | It involved too much s-f ideas. | 19:43 |
| *** maharg101 has joined #openstack-kolla | 19:43 | |
| hrw | | 19:43 |
| hrw | ops | 19:43 |
| hrw | 2021-02-10 19:43:13.128758 | TASK [show us PATH value] | 19:43 |
| hrw | 2021-02-10 19:43:13.854894 | primary | /home/zuul.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games | 19:43 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 19:44 |
| *** jamesbenson has joined #openstack-kolla | 19:45 | |
| *** maharg101 has quit IRC | 19:47 | |
| *** zzzeek has quit IRC | 19:53 | |
| openstackgerrit | James Kirsch proposed openstack/kolla master: Add LetsEncrypt images for cert request/renewal https://review.opendev.org/c/openstack/kolla/+/741339 | 19:53 |
| *** kevko has quit IRC | 19:55 | |
| yoctozepto | well, that was a bad-bad typo | 19:56 |
| *** zzzeek has joined #openstack-kolla | 19:57 | |
| *** maharg101 has joined #openstack-kolla | 20:03 | |
| *** jamesbenson has quit IRC | 20:03 | |
| *** cah_link has quit IRC | 20:09 | |
| *** jamesbenson has joined #openstack-kolla | 20:10 | |
| hrw | 2021-02-10 20:11:46.799144 | primary | "msg": "non-zero return code", | 20:14 |
| hrw | 2021-02-10 20:11:46.799238 | primary | "rc": 1 | 20:14 |
| hrw | so still wrong | 20:14 |
| hrw | should work as it was debian where it failed on deploy so probably lack of libvirt 6+ | 20:18 |
| *** zzzeek has quit IRC | 20:21 | |
| *** zzzeek has joined #openstack-kolla | 20:22 | |
| hrw | post_failure ;( | 20:35 |
| *** maharg101 has quit IRC | 20:37 | |
| *** jamesbenson has quit IRC | 20:54 | |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 21:02 |
| hrw | this version should pass | 21:02 |
| *** jamesbenson has joined #openstack-kolla | 21:19 | |
| *** jamesbenson has quit IRC | 21:21 | |
| *** jamesbenson has joined #openstack-kolla | 21:53 | |
| hrw | works! | 22:07 |
| *** zzzeek has quit IRC | 22:37 | |
| *** zzzeek has joined #openstack-kolla | 22:37 | |
| openstackgerrit | James Kirsch proposed openstack/kolla master: Add LetsEncrypt images for cert request/renewal https://review.opendev.org/c/openstack/kolla/+/741339 | 22:38 |
| *** zzzeek has quit IRC | 22:50 | |
| *** vishalmanchanda has quit IRC | 22:51 | |
| *** zzzeek has joined #openstack-kolla | 22:52 | |
| *** zzzeek has quit IRC | 23:02 | |
| *** zzzeek has joined #openstack-kolla | 23:04 | |
| *** cah_link has joined #openstack-kolla | 23:16 | |
| *** zzzeek has quit IRC | 23:27 | |
| *** zzzeek has joined #openstack-kolla | 23:29 | |
| *** zzzeek has quit IRC | 23:34 | |
| *** zzzeek has joined #openstack-kolla | 23:36 | |
| *** jamesbenson has quit IRC | 23:46 | |
| *** cah_link has quit IRC | 23:53 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!