*** gmann_afk is now known as gmann | 00:19 | |
dardelean | does kolla deploy have any issues with controllers and computes are not on the same L2? traffic is routed | 00:21 |
---|---|---|
*** maharg101 has joined #openstack-kolla | 01:38 | |
*** maharg101 has quit IRC | 01:43 | |
*** bengates has joined #openstack-kolla | 02:05 | |
*** bengates_ has joined #openstack-kolla | 02:08 | |
*** bengates has quit IRC | 02:08 | |
*** bengates_ has quit IRC | 02:12 | |
*** bengates has joined #openstack-kolla | 02:15 | |
*** bengates has quit IRC | 02:20 | |
*** bengates has joined #openstack-kolla | 02:27 | |
*** hrw has quit IRC | 02:27 | |
*** bengates has quit IRC | 02:31 | |
*** skramaja_ has joined #openstack-kolla | 02:35 | |
*** skramaja has quit IRC | 02:37 | |
*** bengates has joined #openstack-kolla | 02:37 | |
*** bengates_ has joined #openstack-kolla | 02:39 | |
*** bengates has quit IRC | 02:39 | |
*** bengates_ has quit IRC | 02:43 | |
*** bengates has joined #openstack-kolla | 02:47 | |
*** bengates has quit IRC | 02:51 | |
*** bengates has joined #openstack-kolla | 02:54 | |
*** bengates_ has joined #openstack-kolla | 02:58 | |
*** bengates has quit IRC | 02:59 | |
*** bengates has joined #openstack-kolla | 03:00 | |
*** bengates_ has quit IRC | 03:03 | |
*** bengates has quit IRC | 03:05 | |
*** bengates has joined #openstack-kolla | 03:08 | |
*** bengates has quit IRC | 03:13 | |
*** bengates has joined #openstack-kolla | 03:15 | |
*** bengates has quit IRC | 03:19 | |
*** bengates has joined #openstack-kolla | 03:22 | |
*** bengates has quit IRC | 03:27 | |
*** LinPeiWen has quit IRC | 03:28 | |
*** bengates has joined #openstack-kolla | 03:29 | |
*** bengates_ has joined #openstack-kolla | 03:34 | |
*** bengates has quit IRC | 03:34 | |
*** bengates_ has quit IRC | 03:38 | |
*** bengates has joined #openstack-kolla | 03:39 | |
*** bengates has quit IRC | 03:43 | |
*** bengates has joined #openstack-kolla | 03:50 | |
*** bengates has quit IRC | 03:55 | |
*** zzzeek has quit IRC | 03:56 | |
*** zzzeek has joined #openstack-kolla | 03:56 | |
*** bengates has joined #openstack-kolla | 03:58 | |
*** bengates has quit IRC | 04:02 | |
*** bengates has joined #openstack-kolla | 04:06 | |
*** bengates has quit IRC | 04:11 | |
*** jopdorp has quit IRC | 04:22 | |
*** stackedsax has quit IRC | 04:26 | |
*** stackedsax has joined #openstack-kolla | 04:28 | |
*** eliaswimmer has quit IRC | 04:37 | |
*** mrunge has quit IRC | 04:37 | |
*** calbers has quit IRC | 04:37 | |
*** eliaswimmer has joined #openstack-kolla | 04:42 | |
*** mrunge has joined #openstack-kolla | 04:42 | |
*** calbers has joined #openstack-kolla | 04:42 | |
*** openstackstatus has quit IRC | 04:58 | |
*** openstack has joined #openstack-kolla | 05:01 | |
*** ChanServ sets mode: +o openstack | 05:01 | |
*** cah_link has joined #openstack-kolla | 06:09 | |
*** vishalmanchanda has joined #openstack-kolla | 06:12 | |
*** k_mouza has joined #openstack-kolla | 06:22 | |
*** k_mouza has quit IRC | 06:26 | |
*** hrw has joined #openstack-kolla | 07:14 | |
yoctozepto | dardelean: controllers are assumed to share the same L2 due to keepalived | 07:42 |
yoctozepto | computes can be elsewhere | 07:42 |
yoctozepto | as long as it's all routable | 07:42 |
*** rpittau|afk is now known as rpittau | 07:51 | |
*** maharg101 has joined #openstack-kolla | 08:13 | |
*** bengates has joined #openstack-kolla | 08:15 | |
openstackgerrit | Piotr Parczewski proposed openstack/kayobe master: [docs] Improve contributor's automated setup document https://review.opendev.org/c/openstack/kayobe/+/774777 | 08:19 |
*** andrewbonney has joined #openstack-kolla | 08:21 | |
*** gfidente has joined #openstack-kolla | 08:21 | |
*** bengates has quit IRC | 08:22 | |
*** bengates has joined #openstack-kolla | 08:22 | |
*** Tony31 has joined #openstack-kolla | 08:27 | |
Tony31 | Hello. For deploying openstack using kayobe, can I use a host OS of Ubuntu 20 ? The doc lists only CentOS. Is this correct? Kolla documentation lists Ubuntu 20 as supported. Thank you | 08:28 |
Tony31 | Ref: https://docs.openstack.org/kayobe/victoria/support-matrix.html | 08:30 |
Tony31 | I will use kolla-ansible if not supported. | 08:30 |
*** bengates has quit IRC | 08:40 | |
parallax | Hi Tony31 - Ubuntu support is work in progress: https://storyboard.openstack.org/#!/story/2004960 | 08:40 |
*** bengates has joined #openstack-kolla | 08:41 | |
*** e0ne has joined #openstack-kolla | 08:41 | |
mnasiadka | morning | 08:44 |
Tony31 | Thank you Parallax | 08:53 |
*** dougsz has joined #openstack-kolla | 08:53 | |
*** dougsz has quit IRC | 08:54 | |
*** e0ne has quit IRC | 09:04 | |
*** e0ne has joined #openstack-kolla | 09:05 | |
hrw | moning | 09:49 |
openstackgerrit | Maksim Malchuk proposed openstack/kolla-ansible master: Correctly configure S3 Token Middleware for Swift https://review.opendev.org/c/openstack/kolla-ansible/+/774791 | 10:06 |
*** kevko_ has joined #openstack-kolla | 10:06 | |
*** kevko_ has quit IRC | 10:09 | |
hrw | so how we are with CI? | 10:32 |
*** e0ne has quit IRC | 10:53 | |
hrw | https://review.opendev.org/c/openstack/kolla-ansible/+/774622 blocks all? | 10:55 |
*** dougsz has joined #openstack-kolla | 10:57 | |
*** e0ne has joined #openstack-kolla | 11:03 | |
*** k_mouza has joined #openstack-kolla | 11:27 | |
*** dougsz has quit IRC | 11:28 | |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 11:49 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Refactor mariadb to support shards https://review.opendev.org/c/openstack/kolla-ansible/+/766952 | 11:49 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Rename role haproxy to loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/770618 | 11:49 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 11:49 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 11:49 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 11:49 |
hrw | this will not fix the problem but rather show that approach was bad | 11:51 |
hrw | 2021-02-10 12:00:13.132707 | TASK [Record the running state of the environment as seen by the setup module] | 12:02 |
hrw | 2021-02-10 12:00:13.642447 | primary | /bin/sh: 1: ansible: not found | 12:02 |
hrw | yay | 12:02 |
hrw | there needs to be virtualenv in use... | 12:04 |
mgoddard | hrw: or full path | 12:05 |
mgoddard | hrw: linters is also broken | 12:05 |
hrw | mgoddard: then some shell script fail | 12:05 |
*** e0ne has quit IRC | 12:05 | |
*** dougsz has joined #openstack-kolla | 12:07 | |
hrw | anyway as we run shell scripts then all of them need to be changed (or their environment) as we go --user or venv | 12:07 |
mgoddard | hrw: are you working on it or shall I? | 12:08 |
hrw | go for it | 12:09 |
hrw | mgoddard: looks like moving most of shell calls into one block would be a good way so there would be one place where environment: PATH will be set | 12:10 |
*** dougsz has quit IRC | 12:10 | |
mgoddard | hrw: or set at the playbook level? | 12:11 |
hrw | or | 12:11 |
mgoddard | s/playbook/play/ | 12:11 |
hrw | mgoddard: you have more ansible experience | 12:11 |
hrw | ~/.local/bin needs to be at start of $PATH | 12:11 |
*** glanciano has joined #openstack-kolla | 12:14 | |
mgoddard | hrw: why at the start? | 12:16 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 12:17 |
*** kevko has joined #openstack-kolla | 12:17 | |
hrw | mgoddard: pip3 from just installed pip 19+ or pip 9 from /usr/bin/pip3? | 12:19 |
mgoddard | hrw: ok | 12:20 |
*** e0ne has joined #openstack-kolla | 12:22 | |
glanciano | Hi! AFAICS, at the moment it is not possible to deploy Monasca in dev mode. Is there someone already working on this? | 12:27 |
hrw | mgoddard: add "--exclude releasenotes" to ansible-lint | 12:30 |
hrw | (venv3) 13:29 (s) hrw@puchatek:kolla-ansible$ ansible-lint -p --exclude tests --exclude roles --exclude etc -vv --exclude releasenotes/ | 12:30 |
hrw | Finished with 0 failure(s), 7 warning(s) on 1326 files. | 12:30 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Refactor mariadb to support shards https://review.opendev.org/c/openstack/kolla-ansible/+/766952 | 12:32 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Rename role haproxy to loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/770618 | 12:32 |
*** glanciano has quit IRC | 12:32 | |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Add proxysql support for database https://review.opendev.org/c/openstack/kolla-ansible/+/770215 | 12:32 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: Edit services roles to support database sharding https://review.opendev.org/c/openstack/kolla-ansible/+/770216 | 12:32 |
openstackgerrit | Michal Arbet proposed openstack/kolla-ansible master: [CI] Test ProxySQL with shards in the nova cells scenario https://review.opendev.org/c/openstack/kolla-ansible/+/770621 | 12:32 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 13:23 |
*** hcash has joined #openstack-kolla | 14:17 | |
hcash | Has anyone an idea where kolla-ansible's "Running Nova API bootstrap container" task caches database credentials? I changed the `kolla_internal_vip_address` but it is still using the old address. All files in /etc/kolla contain the correct IP... also in the container. But the nova-manage.log still reports `Can't connect to MySQL server on '[[OLD | 14:21 |
hcash | IP]]'` | 14:21 |
hcash | I already nuked the bootstrap container and greped for the old IP. Everything looks perfect | 14:23 |
*** bengates has quit IRC | 14:25 | |
*** zeestrat has joined #openstack-kolla | 14:26 | |
*** alistarle has joined #openstack-kolla | 14:27 | |
*** alistarle has quit IRC | 14:35 | |
*** alistarle has joined #openstack-kolla | 14:35 | |
mnasiadka | hcash: tried checking /etc/kolla/nova* for nova.conf on the host running bootstrap? | 14:35 |
hcash | mnasiadka, yes `sudo grep -r [old ip] /etc/kolla` returns nothing on deploy or bootstrap host | 14:40 |
*** bengates has joined #openstack-kolla | 14:42 | |
*** bengates has quit IRC | 14:42 | |
*** bengates has joined #openstack-kolla | 14:43 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 14:46 |
priteau | hcash: Check you cell v2 configuration? Via `docker exec nova_api nova-manage cell_v2 list_cells` if you have such a container, or directly in the nova_api DB, cell_mappings table. | 14:48 |
hcash | priteau aaaah thank you! `cell_mappings.database_connection` still contained the old vip | 14:57 |
priteau | You have nova-manage commands to update i | 14:57 |
priteau | update it | 14:57 |
mgoddard | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak | 14:57 |
mgoddard | ^ meeting in 3 | 14:58 |
mnasiadka | hcash: feel free to raise a bug with your findings, it should be straightforward :) | 14:58 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla master: Make kolla_set_configs open files in binary mode https://review.opendev.org/c/openstack/kolla/+/773257 | 14:58 |
hrw | thx mgoddard | 14:58 |
yoctozepto | mgoddard meeting meeting | 15:00 |
mgoddard | yes yes | 15:01 |
mgoddard | #startmeeting kolla | 15:01 |
openstack | Meeting started Wed Feb 10 15:01:22 2021 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:01 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:01 |
*** openstack changes topic to " (Meeting topic: kolla)" | 15:01 | |
openstack | The meeting name has been set to 'kolla' | 15:01 |
mgoddard | #topic rollcall | 15:01 |
*** openstack changes topic to "rollcall (Meeting topic: kolla)" | 15:01 | |
mgoddard | \o | 15:01 |
yoctozepto | *o* | 15:01 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:02 |
hrw | Io/ | 15:02 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 15:03 |
mgoddard | #topic agenda | 15:04 |
*** openstack changes topic to "agenda (Meeting topic: kolla)" | 15:04 | |
mgoddard | * Roll-call | 15:04 |
mgoddard | * Announcements | 15:04 |
mgoddard | * Review action items from the last meeting | 15:04 |
mgoddard | * CI status | 15:04 |
mgoddard | * CI resources & performance | 15:04 |
mgoddard | ** http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020235.html | 15:04 |
mgoddard | ** https://docs.google.com/spreadsheets/d/1U_TaLoLTpX7ZC3v9cQsvBfoJtUI7ClAhkx0HYvUaFrM/edit#gid=1217205417 | 15:04 |
mgoddard | * Review requests | 15:04 |
mgoddard | * Wallaby release planning | 15:04 |
mgoddard | #topic announcements | 15:04 |
*** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:04 | |
mgoddard | Nothing from me. Anyone else? | 15:04 |
hrw | no | 15:05 |
mgoddard | #topic Review action items from the last meeting | 15:05 |
*** openstack changes topic to "Review action items from the last meeting (Meeting topic: kolla)" | 15:05 | |
mgoddard | hrw to document allowed to fail & unbuildable images in contributor docs | 15:05 |
mgoddard | hrw to build libvirt 7.0.0 for Debian | 15:05 |
hrw | https://review.opendev.org/c/openstack/kolla/+/774618 | 15:06 |
mgoddard | both proposed | 15:06 |
hrw | https://review.opendev.org/c/openstack/kolla/+/774351 | 15:06 |
hrw | allowed to fail are not documented yet - have to find a place where to put it | 15:06 |
mgoddard | true | 15:07 |
mgoddard | #topic CI status | 15:07 |
*** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:07 | |
mgoddard | RED | 15:07 |
mgoddard | https://review.opendev.org/c/openstack/kolla-ansible/+/774622 is the WIP fix | 15:07 |
hrw | train is RED too. https://review.opendev.org/c/openstack/kolla/+/774602 fixes sensu/ubuntu but bifrost is broken for centos 7/8 | 15:08 |
hrw | ussuri is RED on aarch64 - https://review.opendev.org/c/openstack/kolla/+/774493 fixes it | 15:09 |
mgoddard | #action mgoddard fix bifrost on Train | 15:10 |
mgoddard | anything else for CI? | 15:12 |
hrw | probably that's all | 15:13 |
mgoddard | #topic CI resources & performance | 15:13 |
*** openstack changes topic to "CI resources & performance (Meeting topic: kolla)" | 15:13 | |
mgoddard | There is a thread on openstack-discuss | 15:13 |
mgoddard | #link http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020235.html | 15:13 |
mgoddard | about CI resource usage | 15:13 |
mgoddard | kolla is in the top 5 | 15:13 |
mgoddard | although way behind tripleo | 15:14 |
mgoddard | Project % of total Node Hours Nodes | 15:14 |
mgoddard | ------------------------------------------ | 15:14 |
mgoddard | 1. TripleO 38% 31 hours 20 | 15:14 |
mgoddard | 2. Neutron 13% 38 hours 32 | 15:14 |
mgoddard | 3. Nova 9% 21 hours 25 | 15:14 |
mgoddard | 4. Kolla 5% 12 hours 18 | 15:14 |
mgoddard | 5. OSA 5% 22 hours 17 | 15:14 |
mnasiadka | but we have less node hours than OSA | 15:14 |
hrw | were we asked to lower usage? | 15:14 |
mnasiadka | whatever that means | 15:14 |
mgoddard | node hours is per changeset | 15:15 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:15 |
hrw | mnasiadka: node hours == how much time CI run takes | 15:15 |
hrw | that's how I understood it | 15:15 |
yoctozepto | yeah and I think this was max | 15:15 |
yoctozepto | so it's very biased | 15:15 |
mnasiadka | True, I don't understand what Nodes mean, like at the same time? | 15:16 |
mgoddard | how many nodes to run jobs for one changeset | 15:16 |
yoctozepto | again, I think it's max for changeset | 15:16 |
yoctozepto | mhm | 15:16 |
mgoddard | I would expect average | 15:16 |
mgoddard | max for kolla could be huge | 15:16 |
yoctozepto | true that | 15:17 |
yoctozepto | though then nodehours | 15:17 |
yoctozepto | can't be avg | 15:17 |
mgoddard | why not? | 15:17 |
yoctozepto | you think we average 12 hours? | 15:17 |
mgoddard | sure | 15:17 |
mgoddard | most jobs are ~1 hour | 15:17 |
mgoddard | some more | 15:17 |
yoctozepto | ah, you mean like consumed node time | 15:18 |
mgoddard | multinode upgrade jobs might be 4-5 node hours | 15:18 |
yoctozepto | true | 15:18 |
yoctozepto | makes sense | 15:18 |
mgoddard | nodes * time = node hours | 15:18 |
yoctozepto | yeah, yeah | 15:18 |
mgoddard | i.e. what you would pay for on a cloud | 15:18 |
yoctozepto | now it makes sense | 15:18 |
mgoddard | anyways | 15:18 |
mgoddard | I promised I would raise it at this meeting | 15:19 |
mgoddard | now let's move on | 15:19 |
mgoddard | :p | 15:19 |
yoctozepto | you are my hero | 15:19 |
mgoddard | #link https://docs.google.com/spreadsheets/d/1U_TaLoLTpX7ZC3v9cQsvBfoJtUI7ClAhkx0HYvUaFrM/edit#gid=1217205417 | 15:19 |
mgoddard | I did some playing with zuul API | 15:19 |
mgoddard | last 1000 jobs across all branches & kolla projects | 15:20 |
mgoddard | pivot table is most interesting | 15:20 |
mgoddard | I expect we could slice the data more | 15:21 |
yoctozepto | top offenders - ceph jobs | 15:22 |
mgoddard | yes | 15:22 |
yoctozepto | though I wonder if it's simply because of using ceph | 15:22 |
yoctozepto | or because we have to bring up and test cinder | 15:23 |
mgoddard | interestingly the scenario jobs are generally very low | 15:23 |
yoctozepto | they run much less often | 15:23 |
mgoddard | right | 15:23 |
mgoddard | but that does make a good argument against moving them to experimental | 15:23 |
mnasiadka | Well, all ceph jobs use ceph-ansible to deploy, it takes reasonable amount of time to deploy even a tiny ceph cluster | 15:23 |
mgoddard | assuming 1000 jobs is enough data | 15:23 |
mgoddard | mnasiadka: multinode is the main killer - immediate 3x | 15:24 |
yoctozepto | yeah, but | 15:24 |
yoctozepto | compare with multinode ipv6 | 15:24 |
yoctozepto | about 75% penalty on ceph jobs | 15:25 |
*** glanciano has joined #openstack-kolla | 15:25 | |
yoctozepto | but in them we do ceph and cinder | 15:25 |
mgoddard | yes, but significant extra coverage | 15:25 |
yoctozepto | yeah, but I do wonder if we switch to cephadm | 15:25 |
yoctozepto | or perhaps to a custom way | 15:25 |
yoctozepto | (we don't really do a production cluster anyway) | 15:26 |
mnasiadka | then we can compare once we switch, I have time to work on that now - so we'll know next week I guess. | 15:26 |
yoctozepto | great | 15:26 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:26 |
mnasiadka | we probably could have even a single node ceph | 15:26 |
yoctozepto | and then only continue with optimisations we do for k-a ansible | 15:26 |
mnasiadka | and it wouldn't hurt | 15:26 |
mgoddard | should we consider two nodes in multinode jobs? | 15:27 |
mnasiadka | (if there's a big time difference between deploying on 1 and 3 nodes) :) | 15:27 |
yoctozepto | mnasiadka: true that | 15:27 |
yoctozepto | mgoddard: we can evaluate that as well | 15:27 |
mnasiadka | two nodes? I don't think it's our recommendation in HA - the docs recommend 3 nodes? | 15:27 |
mgoddard | mnasiadka: from a resource usage perspective it's 3x | 15:27 |
yoctozepto | mgoddard, mnasiadka: we can still test clustering correctness | 15:27 |
yoctozepto | in jobs like mariadb | 15:27 |
yoctozepto | which are much faster | 15:27 |
yoctozepto | we could create a simple job that test mariadb+rabbitmq+anything_else_that_we_need_to_ha_cluster | 15:28 |
yoctozepto | and avoid multinode in biggur scenarios | 15:28 |
yoctozepto | we could use some unit testing | 15:28 |
yoctozepto | if we loop correctly in templates | 15:28 |
yoctozepto | and things like that | 15:28 |
yoctozepto | but it is much less likely to break it | 15:29 |
yoctozepto | than simply break the process of deploying/upgrading clustered resources | 15:29 |
yoctozepto | we can also avoid multinode jobs duplicates per distro | 15:30 |
yoctozepto | as it's less likely that clustering logic breaks between distros | 15:30 |
mgoddard | it is good to test 3 nodes. I guess we just need to be careful about when we use it | 15:30 |
yoctozepto | it's more likely that we break altogether | 15:30 |
yoctozepto | let's do it in small steps | 15:31 |
mgoddard | we could do centos multinode ceph, ubuntu single node ceph | 15:31 |
yoctozepto | perhaps ditching ceph-ansible will give us a sane boost to performance and reliability | 15:31 |
mgoddard | then the reverse for the ceph upgrade jobs | 15:31 |
yoctozepto | yeah, that works too | 15:31 |
mnasiadka | we could run ceph upgrade jobs as periodicals, I think they are the main killer :) | 15:31 |
hrw | if weeknumber is odd then centos. if even then ubuntu | 15:32 |
mnasiadka | they/those | 15:32 |
yoctozepto | yup, periodics are another path | 15:32 |
yoctozepto | hrw: oh, I like it | 15:32 |
hrw | such check would not require changing CI scripts | 15:32 |
yoctozepto | random CI | 15:32 |
yoctozepto | and then | 15:32 |
yoctozepto | let's merge it on an odd week | 15:32 |
yoctozepto | because it fails on even ones | 15:32 |
mgoddard | let's try to keep it predictable :) | 15:34 |
mgoddard | ok, I think we have some things to try | 15:35 |
mgoddard | I'm going to propose restricting the cells job to nova changes | 15:36 |
yoctozepto | and soon mariadb ones | 15:37 |
yoctozepto | (for shards) | 15:37 |
mgoddard | right | 15:37 |
mgoddard | #topic Review requests | 15:37 |
*** jamesbenson has joined #openstack-kolla | 15:37 | |
*** openstack changes topic to "Review requests (Meeting topic: kolla)" | 15:37 | |
mgoddard | Same as last time | 15:37 |
mgoddard | Does anyone have a patch they would like to be reviewed? | 15:37 |
hrw | one patch: https://review.opendev.org/c/openstack/kolla/+/772507 | 15:38 |
hrw | passed zuul, yoctozepto gave +2 | 15:38 |
openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible master: Lint and fix renos https://review.opendev.org/c/openstack/kolla-ansible/+/759370 | 15:39 |
yoctozepto | well, you can w+1 ^ | 15:39 |
yoctozepto | I am a bit overloaded and don't seem to have anything ready proposed at the moment | 15:40 |
hrw | lgtm | 15:41 |
mgoddard | don't +A | 15:41 |
mgoddard | linters job is broken | 15:41 |
yoctozepto | oh | 15:41 |
yoctozepto | sad | 15:41 |
mgoddard | yeah | 15:41 |
mgoddard | ansible-lint | 15:41 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 15:41 |
yoctozepto | argh | 15:41 |
mgoddard | worst linter ever | 15:41 |
mnasiadka | ansible-lint broken again? | 15:41 |
yoctozepto | just drop it | 15:42 |
hrw | mgoddard: add --exclude releasenotes and be done | 15:42 |
yoctozepto | or make it non-voting | 15:42 |
mgoddard | hrw: not required | 15:42 |
mgoddard | it's fixed in my patch | 15:42 |
hrw | mgoddard: ok | 15:42 |
yoctozepto | let it help us, not hinder us | 15:42 |
mgoddard | we put in the effort to get it passing, may as well keep it | 15:42 |
yoctozepto | we did | 15:43 |
mgoddard | I'm going to make https://review.opendev.org/c/openstack/kolla-ansible/+/695432/ my review request this week | 15:43 |
yoctozepto | that's a review bomb for sure | 15:43 |
mgoddard | boom | 15:44 |
yoctozepto | it conflicts with half the proposed changes as well I see :-) | 15:44 |
yoctozepto | tough nut | 15:44 |
mgoddard | indeed | 15:44 |
hrw | can we add 'must pass zuul' to this request queue? | 15:44 |
mnasiadka | mgoddard: eek, balance source in haproxy? | 15:45 |
hrw | or 'must pass zuul if core wrote' ;D | 15:45 |
hrw | ok, I have to go. thanks for meeting | 15:45 |
mgoddard | mnasiadka: it that's exactly the kind of comment that should be in the review :) | 15:45 |
mgoddard | thanks headphoneJames | 15:45 |
mgoddard | sorry, hrw | 15:45 |
mgoddard | back to sleep headphoneJames | 15:46 |
mnasiadka | mgoddard: I'll force myself to review that tomorrow ;) | 15:46 |
mgoddard | it has been around for a long time and the authors have been quite responsive, helpful, and receptive to change | 15:46 |
mgoddard | I'd like to at least merge that patch and adapt it if we need to | 15:46 |
mgoddard | mnasiadka: just don't mention SAML ;) | 15:47 |
mnasiadka | Of course I will, OIDC is not 90% of usage, I would rather claim otherwise :) | 15:47 |
yoctozepto | I have added my first comment | 15:48 |
mnasiadka | Anyway, let's not incite religious wars. | 15:48 |
yoctozepto | XML4life | 15:48 |
mnasiadka | I never had the ability to read XML files. | 15:48 |
yoctozepto | they are the best test for patience | 15:48 |
mnasiadka | It's like obfuscated poetry from an oppressed country. | 15:49 |
yoctozepto | hahahaha | 15:49 |
mgoddard | it stands for XXXX my life | 15:49 |
yoctozepto | the water I was drinking left me through my nose | 15:49 |
yoctozepto | thank you very much | 15:49 |
mgoddard | hope your computer is dry | 15:49 |
yoctozepto | drying it | 15:49 |
yoctozepto | don't drink and kollameet | 15:50 |
mgoddard | #topic Wallaby release planning | 15:51 |
*** openstack changes topic to "Wallaby release planning (Meeting topic: kolla)" | 15:51 | |
mnasiadka | Around reviews - can somebody look at this: https://review.opendev.org/c/openstack/kolla-ansible/+/774222 ? It solves cases when Neutron does not support OVN native DHCP yet. | 15:51 |
yoctozepto | wallaby is the cycle we spend on fixing our ci fires | 15:51 |
mgoddard | Mar 29 - Apr 02 | 15:51 |
mgoddard | yoctozepto: every cycle is the cycle we spend on fixing our ci fires | 15:52 |
mnasiadka | and not only our | 15:52 |
yoctozepto | this one feels sadder | 15:52 |
mgoddard | I'll take a look | 15:53 |
mgoddard | mnasiadka: what are those cases? bare metal? | 15:53 |
mnasiadka | mgoddard: yes, bare metal | 15:53 |
mnasiadka | OVN has functionality that would fit, but Neutron seems to be lacking here and there... | 15:54 |
mgoddard | hmm, surprised lucasgomes is not on top of it | 15:55 |
mnasiadka | well, he's the one proposing to use neutron-dhcp-agent ) | 15:55 |
mnasiadka | :) | 15:55 |
mgoddard | ah | 15:55 |
mgoddard | well there is no better intersection of ironic & ovn knowledge | 15:55 |
*** skramaja_ has quit IRC | 15:56 | |
mgoddard | #topic open discussion | 15:56 |
*** openstack changes topic to "open discussion (Meeting topic: kolla)" | 15:56 | |
mgoddard | anyone have anything this week? | 15:56 |
*** alistarle has quit IRC | 15:58 | |
mnasiadka | I was thinking about extending kayobe's seed-custom-containers to overcloud-custom-containers - is that something we want to do? | 15:58 |
mgoddard | mnasiadka: I guess I see kolla-ansible as the thing that deploys containers | 15:58 |
mgoddard | is there a particular use case in mind? | 15:59 |
yoctozepto | nothing new from me | 15:59 |
mnasiadka | well, there are some components, that are not covered by kolla-ansible - I can write an external role to do it, or just do ^^ | 15:59 |
mnasiadka | like I don't know... consul and hashicorp vault | 16:00 |
mgoddard | I see | 16:00 |
mgoddard | it probably needs some thought | 16:01 |
mnasiadka | for sure, we need to make sure we don't have a naming collision for containers and volumes | 16:01 |
mnasiadka | which might be not so straightforward to write | 16:02 |
mgoddard | we did discuss making some plugin/extension mechanism for kolla-ansible at the PTG | 16:02 |
mgoddard | prometheus exporters being the initial use case | 16:02 |
mgoddard | but potentiall this fits too | 16:02 |
mnasiadka | yeah, while adding roles from a different path (or using ansible galaxy) is not problematic, we would need to find a way to resolve dependencies/order of running | 16:02 |
*** jamesbenson has quit IRC | 16:03 | |
mgoddard | I think there is a question of when you would want to deploy those things - are they coupled to openstack, or not? | 16:03 |
mgoddard | probably you would not want to touch vault too often | 16:03 |
mnasiadka | well - touching vault means unsealing - so no :) | 16:04 |
mgoddard | anyway, meeting over folks | 16:04 |
mgoddard | thanks | 16:04 |
mgoddard | #endmeeting | 16:04 |
*** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:04 | |
openstack | Meeting ended Wed Feb 10 16:04:16 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:04 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-10-15.01.html | 16:04 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-10-15.01.txt | 16:04 |
openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-10-15.01.log.html | 16:04 |
*** jamesbenson has joined #openstack-kolla | 16:08 | |
*** jamesbenson has quit IRC | 16:09 | |
*** jamesbenson has joined #openstack-kolla | 16:09 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 16:12 |
*** jamesbenson has quit IRC | 16:20 | |
*** jamesbenson has joined #openstack-kolla | 16:20 | |
*** jamesbenson has quit IRC | 16:30 | |
openstackgerrit | Pedro Henrique Pereira Martins proposed openstack/kolla-ansible master: Add support to OpenID Connect Authentication flow https://review.opendev.org/c/openstack/kolla-ansible/+/695432 | 16:32 |
*** numans has quit IRC | 16:38 | |
*** ianw has quit IRC | 16:39 | |
*** ianw has joined #openstack-kolla | 16:39 | |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 16:39 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: only run cells job on changes to nova role https://review.opendev.org/c/openstack/kolla-ansible/+/774894 | 16:39 |
*** bengates has quit IRC | 16:40 | |
*** glanciano_ has joined #openstack-kolla | 16:48 | |
*** glanciano has quit IRC | 16:51 | |
*** e0ne has quit IRC | 17:16 | |
*** jamesbenson has joined #openstack-kolla | 17:20 | |
*** e0ne has joined #openstack-kolla | 17:22 | |
*** dougsz has joined #openstack-kolla | 17:23 | |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 17:27 |
*** dougsz has quit IRC | 17:28 | |
*** glanciano_ has quit IRC | 17:33 | |
*** hcash has quit IRC | 17:37 | |
*** maharg101 has quit IRC | 17:46 | |
*** jamesbenson has quit IRC | 17:47 | |
*** jbadiapa has quit IRC | 17:51 | |
*** jonaspaulo has joined #openstack-kolla | 17:52 | |
*** jamesbenson has joined #openstack-kolla | 17:58 | |
hrw | 16:52 < mgoddard> yoctozepto: every cycle is the cycle we spend on fixing our ci fires | 17:58 |
hrw | I try to leave ci fires you everyone else and then some sparks land in my basket... | 17:59 |
*** jamesbenson has quit IRC | 18:01 | |
*** jamesbenson has joined #openstack-kolla | 18:01 | |
*** e0ne has quit IRC | 18:19 | |
*** jamesbenson has quit IRC | 18:21 | |
*** jamesbenson has joined #openstack-kolla | 18:21 | |
*** rpittau is now known as rpittau|afk | 18:23 | |
*** gfidente has quit IRC | 18:28 | |
*** jamesbenson has quit IRC | 18:28 | |
*** e0ne has joined #openstack-kolla | 18:29 | |
*** e0ne has quit IRC | 18:29 | |
*** jamesbenson has joined #openstack-kolla | 18:33 | |
*** k_mouza has quit IRC | 18:33 | |
*** jamesbenson has quit IRC | 18:49 | |
*** jamesbenson has joined #openstack-kolla | 18:49 | |
*** jonaspaulo has quit IRC | 18:56 | |
openstackgerrit | Pedro Henrique Pereira Martins proposed openstack/kolla-ansible master: Add support to OpenID Connect Authentication flow https://review.opendev.org/c/openstack/kolla-ansible/+/695432 | 18:56 |
*** jamesbenson has quit IRC | 19:04 | |
*** jamesbenson has joined #openstack-kolla | 19:06 | |
*** jamesbenson has quit IRC | 19:12 | |
*** jamesbenson has joined #openstack-kolla | 19:14 | |
*** jamesbenson has quit IRC | 19:17 | |
*** jamesbenson has joined #openstack-kolla | 19:18 | |
*** jamesbenson has quit IRC | 19:22 | |
*** jamesbenson has joined #openstack-kolla | 19:23 | |
*** zzzeek has quit IRC | 19:26 | |
hrw | mgoddard: WARNING: The script alembic is installed in '/home/zuul/.local/bin' which is not on PATH. | 19:26 |
hrw | mgoddard: looks like your trick with PATH failed ;( | 19:26 |
*** zzzeek has joined #openstack-kolla | 19:27 | |
yoctozepto | mgoddard, hrw: let's just do the venv way now | 19:29 |
yoctozepto | no more happy hacks | 19:29 |
hrw | yoctozepto: the problem is that each script call has then to initialize virtualenv | 19:35 |
hrw | argh | 19:35 |
hrw | each shell, each command call | 19:36 |
hrw | or am I wrong? | 19:36 |
*** jamesbenson has quit IRC | 19:38 | |
*** jamesbenson has joined #openstack-kolla | 19:38 | |
hrw | just added some debug output | 19:38 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 19:38 |
hrw | it will fail | 19:38 |
*** andrewbonney has quit IRC | 19:39 | |
*** jamesbenson has quit IRC | 19:42 | |
hrw | It involved too much s-f ideas. | 19:43 |
*** maharg101 has joined #openstack-kolla | 19:43 | |
hrw | | 19:43 |
hrw | ops | 19:43 |
hrw | 2021-02-10 19:43:13.128758 | TASK [show us PATH value] | 19:43 |
hrw | 2021-02-10 19:43:13.854894 | primary | /home/zuul.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games | 19:43 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 19:44 |
*** jamesbenson has joined #openstack-kolla | 19:45 | |
*** maharg101 has quit IRC | 19:47 | |
*** zzzeek has quit IRC | 19:53 | |
openstackgerrit | James Kirsch proposed openstack/kolla master: Add LetsEncrypt images for cert request/renewal https://review.opendev.org/c/openstack/kolla/+/741339 | 19:53 |
*** kevko has quit IRC | 19:55 | |
yoctozepto | well, that was a bad-bad typo | 19:56 |
*** zzzeek has joined #openstack-kolla | 19:57 | |
*** maharg101 has joined #openstack-kolla | 20:03 | |
*** jamesbenson has quit IRC | 20:03 | |
*** cah_link has quit IRC | 20:09 | |
*** jamesbenson has joined #openstack-kolla | 20:10 | |
hrw | 2021-02-10 20:11:46.799144 | primary | "msg": "non-zero return code", | 20:14 |
hrw | 2021-02-10 20:11:46.799238 | primary | "rc": 1 | 20:14 |
hrw | so still wrong | 20:14 |
hrw | should work as it was debian where it failed on deploy so probably lack of libvirt 6+ | 20:18 |
*** zzzeek has quit IRC | 20:21 | |
*** zzzeek has joined #openstack-kolla | 20:22 | |
hrw | post_failure ;( | 20:35 |
*** maharg101 has quit IRC | 20:37 | |
*** jamesbenson has quit IRC | 20:54 | |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla-ansible master: CI: fix kolla-ansible installation after cryptography 3.4 release https://review.opendev.org/c/openstack/kolla-ansible/+/774622 | 21:02 |
hrw | this version should pass | 21:02 |
*** jamesbenson has joined #openstack-kolla | 21:19 | |
*** jamesbenson has quit IRC | 21:21 | |
*** jamesbenson has joined #openstack-kolla | 21:53 | |
hrw | works! | 22:07 |
*** zzzeek has quit IRC | 22:37 | |
*** zzzeek has joined #openstack-kolla | 22:37 | |
openstackgerrit | James Kirsch proposed openstack/kolla master: Add LetsEncrypt images for cert request/renewal https://review.opendev.org/c/openstack/kolla/+/741339 | 22:38 |
*** zzzeek has quit IRC | 22:50 | |
*** vishalmanchanda has quit IRC | 22:51 | |
*** zzzeek has joined #openstack-kolla | 22:52 | |
*** zzzeek has quit IRC | 23:02 | |
*** zzzeek has joined #openstack-kolla | 23:04 | |
*** cah_link has joined #openstack-kolla | 23:16 | |
*** zzzeek has quit IRC | 23:27 | |
*** zzzeek has joined #openstack-kolla | 23:29 | |
*** zzzeek has quit IRC | 23:34 | |
*** zzzeek has joined #openstack-kolla | 23:36 | |
*** jamesbenson has quit IRC | 23:46 | |
*** cah_link has quit IRC | 23:53 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!