*** e0ne has quit IRC | 00:01 | |
*** jamesbenson has joined #openstack-kolla | 00:03 | |
*** jamesbenson has quit IRC | 00:08 | |
*** jamesbenson has joined #openstack-kolla | 00:21 | |
*** stingrayza has quit IRC | 00:29 | |
*** jamesbenson has quit IRC | 00:39 | |
*** brinzhang has joined #openstack-kolla | 00:40 | |
*** LinPeiWen has joined #openstack-kolla | 00:40 | |
*** stingrayza has joined #openstack-kolla | 00:41 | |
*** jamesbenson has joined #openstack-kolla | 00:49 | |
*** mchlumsky1 has joined #openstack-kolla | 00:49 | |
*** dmsimard0 has joined #openstack-kolla | 00:50 | |
*** dmsimard has quit IRC | 00:51 | |
*** dmsimard0 is now known as dmsimard | 00:51 | |
*** mchlumsky has quit IRC | 00:51 | |
*** mchlumsky1 is now known as mchlumsky | 00:51 | |
*** jamesbenson has quit IRC | 00:54 | |
*** xinliang has joined #openstack-kolla | 01:11 | |
*** mchlumsky has quit IRC | 01:11 | |
*** mchlumsky has joined #openstack-kolla | 01:12 | |
*** brinzhang has quit IRC | 01:13 | |
*** brinzhang has joined #openstack-kolla | 01:15 | |
*** zzzeek has quit IRC | 01:15 | |
*** brinzhang has quit IRC | 01:16 | |
*** zzzeek has joined #openstack-kolla | 01:17 | |
*** ricolin_ has joined #openstack-kolla | 01:21 | |
*** maharg101 has joined #openstack-kolla | 01:22 | |
*** maharg101 has quit IRC | 01:26 | |
*** ivan_lin has joined #openstack-kolla | 01:27 | |
*** cz3 is now known as Halabala | 01:34 | |
*** Halabala is now known as cz3 | 01:34 | |
*** jamesbenson has joined #openstack-kolla | 02:00 | |
*** jamesbenson has quit IRC | 02:04 | |
*** zul has quit IRC | 02:08 | |
*** ricolin_ has quit IRC | 02:11 | |
*** ricolin_ has joined #openstack-kolla | 02:11 | |
*** jamesbenson has joined #openstack-kolla | 02:15 | |
*** jamesbenson has quit IRC | 02:20 | |
*** calbers has quit IRC | 02:37 | |
*** calbers has joined #openstack-kolla | 02:37 | |
*** jamesbenson has joined #openstack-kolla | 02:59 | |
*** jamesbenson has quit IRC | 03:04 | |
*** brinzhang has joined #openstack-kolla | 03:17 | |
*** maharg101 has joined #openstack-kolla | 03:22 | |
*** maharg101 has quit IRC | 03:27 | |
*** jamesbenson has joined #openstack-kolla | 03:51 | |
*** portdirect has joined #openstack-kolla | 03:52 | |
*** jamesbenson has quit IRC | 03:56 | |
*** jamesbenson has joined #openstack-kolla | 03:59 | |
*** xinliang has quit IRC | 04:01 | |
*** jamesbenson has quit IRC | 04:04 | |
*** jamesbenson has joined #openstack-kolla | 04:53 | |
*** jamesbenson has quit IRC | 04:57 | |
*** cah_link has joined #openstack-kolla | 05:03 | |
*** cah_link has quit IRC | 05:12 | |
oklhost | morning | 05:20 |
---|---|---|
*** maharg101 has joined #openstack-kolla | 05:23 | |
*** maharg101 has quit IRC | 05:28 | |
*** jamesbenson has joined #openstack-kolla | 05:54 | |
*** jamesbenson has quit IRC | 05:58 | |
*** jamesbenson has joined #openstack-kolla | 05:59 | |
*** ricolin has quit IRC | 06:00 | |
*** jamesbenson has quit IRC | 06:03 | |
*** LinPeiWen has quit IRC | 06:16 | |
*** LinPeiWen has joined #openstack-kolla | 06:17 | |
*** LinPeiWen79 has joined #openstack-kolla | 06:24 | |
*** LinPeiWen has quit IRC | 06:25 | |
*** ricolin_ has quit IRC | 06:27 | |
*** ricolin has joined #openstack-kolla | 06:28 | |
*** jamesbenson has joined #openstack-kolla | 06:33 | |
*** cah_link has joined #openstack-kolla | 06:43 | |
*** jbadiapa has quit IRC | 06:45 | |
*** jamesbenson has quit IRC | 06:51 | |
*** LinPeiWen79 has quit IRC | 06:58 | |
*** jamesbenson has joined #openstack-kolla | 06:59 | |
*** jamesbenson has quit IRC | 07:04 | |
*** LinPeiWen has joined #openstack-kolla | 07:05 | |
yoctozepto | moorning | 07:14 |
*** e0ne has joined #openstack-kolla | 07:38 | |
*** e0ne has quit IRC | 07:39 | |
*** jamesbenson has joined #openstack-kolla | 07:46 | |
*** jamesbenson has quit IRC | 07:51 | |
*** luksky has joined #openstack-kolla | 07:54 | |
*** jamesbenson has joined #openstack-kolla | 07:59 | |
*** amoralej|off is now known as amoralej | 08:00 | |
*** jamesbenson has quit IRC | 08:04 | |
*** rpittau|afk is now known as rpittau | 08:11 | |
*** LinPeiWen has quit IRC | 08:13 | |
*** maharg101 has joined #openstack-kolla | 08:20 | |
*** bengates has joined #openstack-kolla | 08:23 | |
*** andrewbonney has joined #openstack-kolla | 08:27 | |
*** jamesbenson has joined #openstack-kolla | 08:34 | |
*** jamesbenson has quit IRC | 08:39 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe-config-dev stable/victoria: Reduce neutron RPC workers to 1 https://review.opendev.org/c/openstack/kayobe-config-dev/+/777130 | 08:41 |
openstackgerrit | Mark Goddard proposed openstack/kayobe-config-dev stable/ussuri: Reduce neutron RPC workers to 1 https://review.opendev.org/c/openstack/kayobe-config-dev/+/777131 | 08:41 |
openstackgerrit | Mark Goddard proposed openstack/kayobe-config-dev stable/train: Reduce neutron RPC workers to 1 https://review.opendev.org/c/openstack/kayobe-config-dev/+/777132 | 08:41 |
mgoddard | morning | 08:47 |
mnasiadka | morning | 08:56 |
*** jamesbenson has joined #openstack-kolla | 08:59 | |
*** jamesbenson has quit IRC | 09:04 | |
*** dougsz has joined #openstack-kolla | 09:17 | |
mnasiadka | I have a feeling that ubuntu nodes in the CI are cursed | 09:18 |
mnasiadka | first fail - timedatectl error (connection timed out), second try - mariadb is unreachable | 09:18 |
openstackgerrit | Ana Peric proposed openstack/kolla-ansible master: Avoid errors in apparmor chrony and libvirt profile removal in Debian https://review.opendev.org/c/openstack/kolla-ansible/+/776252 | 09:20 |
*** LinPeiWen has joined #openstack-kolla | 09:21 | |
*** e0ne has joined #openstack-kolla | 09:21 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Ubuntu: document overcloud development environment https://review.opendev.org/c/openstack/kayobe/+/769791 | 09:21 |
openstackgerrit | Michal Nasiadka proposed openstack/kayobe master: Add output printing to host-command-run https://review.opendev.org/c/openstack/kayobe/+/776471 | 09:22 |
*** dougsz has quit IRC | 09:23 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Fail on fluentd log parsing errors https://review.opendev.org/c/openstack/kolla-ansible/+/742399 | 09:24 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add missing octavia-driver-agent https://review.opendev.org/c/openstack/kolla-ansible/+/761872 | 09:26 |
*** bengates has quit IRC | 09:28 | |
*** bengates has joined #openstack-kolla | 09:29 | |
*** gfidente has joined #openstack-kolla | 09:30 | |
*** bengates has quit IRC | 09:32 | |
*** bengates has joined #openstack-kolla | 09:32 | |
*** gfidente is now known as gfidente|afk | 09:39 | |
*** wuchunyang has joined #openstack-kolla | 09:44 | |
*** jamesbenson has joined #openstack-kolla | 09:45 | |
*** jamesbenson has quit IRC | 09:50 | |
openstackgerrit | Merged openstack/kolla-ansible master: Improve trove releasenote https://review.opendev.org/c/openstack/kolla-ansible/+/777113 | 09:57 |
*** LinPeiWen has quit IRC | 09:57 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Ubuntu: container image build https://review.opendev.org/c/openstack/kayobe/+/775129 | 09:58 |
*** jamesbenson has joined #openstack-kolla | 09:59 | |
*** jamesbenson has quit IRC | 10:04 | |
*** brinzhang has quit IRC | 10:04 | |
*** brinzhang has joined #openstack-kolla | 10:05 | |
*** dougsz has joined #openstack-kolla | 10:15 | |
*** wuchunyang has quit IRC | 10:16 | |
*** Fl1nt has joined #openstack-kolla | 10:26 | |
Fl1nt | Good morning everyone! | 10:27 |
oklhost | morning Fl1nt | 10:29 |
Fl1nt | o/ oklhost Did you fixed your problem with cURL? | 10:30 |
oklhost | Fl1nt: Yes, with -L the build went well | 10:30 |
*** zijlboot has quit IRC | 10:34 | |
*** zijlboot has joined #openstack-kolla | 10:34 | |
*** jamesbenson has joined #openstack-kolla | 10:35 | |
oklhost | Fl1nt: you mentioned the project needs more debian/ubuntu reviewers, maybe I can do that? is there some guidance for reviewers? | 10:36 |
oklhost | we're running ubuntu in production | 10:37 |
Fl1nt | ah sweeet! | 10:38 |
*** jamesbenson has quit IRC | 10:39 | |
Fl1nt | I would say the contribut guide is a first good read and then, ask complementary questions in here :D | 10:39 |
Fl1nt | https://docs.openstack.org/kolla-ansible/latest/contributor/index.html | 10:39 |
*** dougsz has quit IRC | 10:39 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Ubuntu: update Apt cache before package installation https://review.opendev.org/c/openstack/kayobe/+/775130 | 10:40 |
mgoddard | oklhost: reviews always appreciated. There's some info here: https://docs.openstack.org/kolla/latest/contributor/code-reviews.html | 10:41 |
oklhost | Thanks! Well, I will try to understand everything... which I didn't a few month ago. Thats the reason why I never submitted any blueprints of my code to deploy adjutant w/ kolla/kolla-ansible ;) gerrit etc. is pretty new to me and sounds a bit complicated at the first view, for a newbie like me. I do not want to annoy anyone with stupid questions ;) | 10:47 |
mgoddard | oklhost: no question too stupid :) | 10:48 |
*** k_mouza has joined #openstack-kolla | 10:48 | |
oklhost | mgoddard: nice to hear. I'll give it a try than :) It would be an honor to help. | 10:49 |
hrw | morning | 10:52 |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: docs: add code reviews to contributor guide https://review.opendev.org/c/openstack/kolla-ansible/+/777356 | 10:52 |
Fl1nt | oklhost, regarding gerrit, feel free to ask, it's a bit different from what we're all used to with classic github/git process, but once you get the minimal amount of knowledge it's pretty streamlined. | 10:54 |
*** LinPeiWen has joined #openstack-kolla | 10:54 | |
oklhost | Fl1nt: thanks! :) | 10:55 |
*** k_mouza has quit IRC | 10:55 | |
*** k_mouza has joined #openstack-kolla | 10:56 | |
*** jamesbenson has joined #openstack-kolla | 10:59 | |
*** eliaswimmer has joined #openstack-kolla | 11:01 | |
*** k_mouza_ has joined #openstack-kolla | 11:03 | |
*** jamesbenson has quit IRC | 11:04 | |
*** k_mouza has quit IRC | 11:06 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Report available entropy https://review.opendev.org/c/openstack/kayobe/+/777365 | 11:15 |
openstackgerrit | Mark Goddard proposed openstack/kayobe stable/train: Fix copying Swift ring files https://review.opendev.org/c/openstack/kayobe/+/773900 | 11:17 |
openstackgerrit | Mark Goddard proposed openstack/kayobe stable/train: DNM: Test Swift https://review.opendev.org/c/openstack/kayobe/+/773903 | 11:18 |
*** e0ne has quit IRC | 11:20 | |
*** e0ne has joined #openstack-kolla | 11:22 | |
*** bengates has quit IRC | 11:23 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: Add missing octavia-driver-agent https://review.opendev.org/c/openstack/kolla-ansible/+/761872 | 11:23 |
*** bengates has joined #openstack-kolla | 11:23 | |
*** kevko has joined #openstack-kolla | 11:24 | |
*** kevko_ has joined #openstack-kolla | 11:26 | |
*** kevko_ has quit IRC | 11:27 | |
*** kevko_ has joined #openstack-kolla | 11:28 | |
*** kevko has quit IRC | 11:28 | |
openstackgerrit | Mark Goddard proposed openstack/kolla-ansible master: docs: add code reviews to contributor guide https://review.opendev.org/c/openstack/kolla-ansible/+/777356 | 11:32 |
*** k_mouza_ has quit IRC | 11:45 | |
*** k_mouza has joined #openstack-kolla | 11:46 | |
*** jamesbenson has joined #openstack-kolla | 11:50 | |
*** jamesbenson has quit IRC | 11:54 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: CI: display VM console log on ping or SSH failure https://review.opendev.org/c/openstack/kayobe/+/777372 | 11:55 |
*** jamesbenson has joined #openstack-kolla | 11:59 | |
*** jamesbenson has quit IRC | 12:04 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe stable/train: CI: use overlay2 storage driver for overcloud deploy https://review.opendev.org/c/openstack/kayobe/+/777373 | 12:07 |
*** k_mouza has quit IRC | 12:08 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe stable/train: Fix copying Swift ring files https://review.opendev.org/c/openstack/kayobe/+/773900 | 12:09 |
openstackgerrit | Mark Goddard proposed openstack/kayobe stable/train: DNM: Test Swift https://review.opendev.org/c/openstack/kayobe/+/773903 | 12:10 |
*** jamesbenson has joined #openstack-kolla | 12:18 | |
*** e0ne has quit IRC | 12:19 | |
*** k_mouza has joined #openstack-kolla | 12:21 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Refactor network filter plugins to a Python module https://review.opendev.org/c/openstack/kayobe/+/777175 | 12:31 |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Refactor veth configuration role variable generation https://review.opendev.org/c/openstack/kayobe/+/777176 | 12:31 |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: WIP: Ubuntu: support systemd-networkd https://review.opendev.org/c/openstack/kayobe/+/776500 | 12:31 |
openstackgerrit | Mark Goddard proposed openstack/kayobe stable/train: CI: use overlay2 storage driver for overcloud deploy https://review.opendev.org/c/openstack/kayobe/+/777373 | 12:31 |
*** gfidente|afk is now known as gfidente | 12:37 | |
*** vishalmanchanda has joined #openstack-kolla | 12:46 | |
openstackgerrit | cyrille babon proposed openstack/kolla master: add lsscsi due to change in os-brick https://review.opendev.org/c/openstack/kolla/+/777142 | 12:57 |
*** amoralej is now known as amoralej|lunch | 12:59 | |
*** jamesbenson1 has joined #openstack-kolla | 12:59 | |
*** jamesbenson has quit IRC | 13:02 | |
openstackgerrit | cyrille babon proposed openstack/kolla master: add lsscsi due to change in os-brick https://review.opendev.org/c/openstack/kolla/+/777142 | 13:02 |
*** dougsz has joined #openstack-kolla | 13:15 | |
*** LinPeiWen has quit IRC | 13:17 | |
*** jamesbenson1 has quit IRC | 13:26 | |
openstackgerrit | Doug Szumski proposed openstack/kolla-ansible master: Fix Cinder log parsing https://review.opendev.org/c/openstack/kolla-ansible/+/777381 | 13:27 |
*** dougsz has quit IRC | 13:31 | |
*** k_mouza has quit IRC | 13:31 | |
*** k_mouza has joined #openstack-kolla | 13:32 | |
*** dougsz has joined #openstack-kolla | 13:38 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: WIP: Ubuntu: support systemd-networkd https://review.opendev.org/c/openstack/kayobe/+/776500 | 13:39 |
*** zul has joined #openstack-kolla | 13:41 | |
*** dougsz has quit IRC | 13:43 | |
*** Luzi has joined #openstack-kolla | 13:47 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Move from ceph-ansible to cephadm https://review.opendev.org/c/openstack/kolla-ansible/+/771418 | 13:48 |
*** e0ne has joined #openstack-kolla | 13:50 | |
*** jamesbenson has joined #openstack-kolla | 13:52 | |
*** jamesbenson has quit IRC | 13:57 | |
*** dciabrin has quit IRC | 13:58 | |
*** amoralej|lunch is now known as amoralej | 13:58 | |
*** dciabrin has joined #openstack-kolla | 13:58 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Ubuntu: update Apt cache before package installation https://review.opendev.org/c/openstack/kayobe/+/775130 | 13:59 |
*** dciabrin has quit IRC | 14:03 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe master: Report available entropy https://review.opendev.org/c/openstack/kayobe/+/777365 | 14:04 |
*** rpittau is now known as rpittau|afk | 14:04 | |
*** dciabrin has joined #openstack-kolla | 14:09 | |
*** LinPeiWen has joined #openstack-kolla | 14:15 | |
*** ivan_lin has quit IRC | 14:18 | |
*** Luzi has quit IRC | 14:28 | |
*** luksky has quit IRC | 14:36 | |
*** jamesbenson has joined #openstack-kolla | 14:37 | |
*** Luzi has joined #openstack-kolla | 14:43 | |
*** dciabrin has quit IRC | 14:46 | |
mgoddard | mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt | 14:51 |
mgoddard | ^ meeting in 10 | 14:51 |
*** jamesbenson has quit IRC | 14:56 | |
*** jamesbenson has joined #openstack-kolla | 14:59 | |
*** wuchunyang has joined #openstack-kolla | 14:59 | |
mgoddard | #startmeeting kolla | 15:00 |
openstack | Meeting started Wed Feb 24 15:00:12 2021 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
*** openstack changes topic to " (Meeting topic: kolla)" | 15:00 | |
openstack | The meeting name has been set to 'kolla' | 15:00 |
mgoddard | #topic rollcall | 15:00 |
*** openstack changes topic to "rollcall (Meeting topic: kolla)" | 15:00 | |
mgoddard | \o | 15:00 |
headphoneJames | o/ | 15:00 |
wuchunyang | \o | 15:00 |
yoctozepto | \o/ | 15:03 |
hrw | ]o[ | 15:03 |
mgoddard | #topic agenda | 15:03 |
*** openstack changes topic to "agenda (Meeting topic: kolla)" | 15:03 | |
kevko_ | hi :P | 15:03 |
mgoddard | * Roll-call | 15:04 |
mgoddard | * Announcements | 15:04 |
mgoddard | * Review action items from the last meeting | 15:04 |
mgoddard | * CI status | 15:04 |
mgoddard | * Review requests | 15:04 |
mgoddard | * yoctozepto: Stop caring about NTP/chrony | 15:04 |
mgoddard | * Wallaby release planning | 15:04 |
mgoddard | #topic announcements | 15:04 |
*** openstack changes topic to "announcements (Meeting topic: kolla)" | 15:04 | |
mgoddard | None from me | 15:04 |
mgoddard | Anyone else? | 15:04 |
wuchunyang | no | 15:04 |
mgoddard | #topic Review action items from the last meeting | 15:05 |
*** openstack changes topic to "Review action items from the last meeting (Meeting topic: kolla)" | 15:05 | |
mgoddard | There were none | 15:05 |
mgoddard | #topic CI status | 15:05 |
*** openstack changes topic to "CI status (Meeting topic: kolla)" | 15:05 | |
mgoddard | Kolla Train still broken | 15:07 |
mgoddard | just rechecked https://review.opendev.org/c/openstack/kolla/+/774602 | 15:07 |
hrw | uf. it was just pull limit | 15:08 |
*** ohorecny2 has joined #openstack-kolla | 15:08 | |
mgoddard | yeah | 15:09 |
mnasiadka | I've seen multiple occurences of jobs where one of the hosts fail on "Failed to connect to the host via ssh", but that's independent from us... | 15:09 |
mgoddard | I started this to track mounting kayobe CI failures: https://etherpad.opendev.org/p/kayobe-ci-failures | 15:10 |
mnasiadka | do we have ssh retries configured in Ansible? | 15:10 |
*** jamesbenson has quit IRC | 15:10 | |
mnasiadka | Seems not really, I'll submit a patch | 15:10 |
mgoddard | mnasiadka: is it zuul's ansible or ours? | 15:11 |
ohorecny2 | Hi all, I would like to ask if there are some plans to support Podman instead of Docker in case of kolla and kolla-ansible. Do community want to support it in future? | 15:12 |
*** dciabrin has joined #openstack-kolla | 15:12 | |
mgoddard | ohorecny2: hi, we are currently in a meeting | 15:14 |
mnasiadka | mgoddard: it's ours, during deploy | 15:14 |
mgoddard | ohorecny2: there is interest in the community for podman, but no one has started work on it yet. You might want to talk to Fl1nt about it | 15:15 |
mgoddard | ohorecny2: I also made a small PoC: https://github.com/stackhpc/kolla-ansible/commit/e44d4b028e3aa24955dd12271783287ae43a5603 | 15:15 |
mgoddard | mnasiadka: we could try adding retries | 15:15 |
mnasiadka | mgoddard: that's what I said - will raise a patch :) | 15:15 |
mgoddard | mnasiadka: I was agreeing | 15:16 |
mnasiadka | mgoddard: In darkest corners of my brain I couldn't see a situation somebody would disagree :) | 15:16 |
mnasiadka | But thank you for agreeing :) | 15:16 |
mgoddard | it will only help if the network issues are temporary | 15:17 |
mgoddard | #topic Review requests | 15:19 |
*** openstack changes topic to "Review requests (Meeting topic: kolla)" | 15:19 | |
mgoddard | Does anyone have a patch they would like to be reviewer | 15:19 |
mgoddard | *reviewed | 15:19 |
*** wuchunyang has quit IRC | 15:19 | |
headphoneJames | https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 15:19 |
*** wuchunyang has joined #openstack-kolla | 15:19 | |
yoctozepto | all WIP | 15:19 |
headphoneJames | The Lets Encrypt patch is ready, still putting the test together | 15:20 |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Add ssh retries and disable group names validation https://review.opendev.org/c/openstack/kolla-ansible/+/777402 | 15:20 |
wuchunyang | what 's the plan of the octavia management network patch ? i have raised it to email . but no help for me. | 15:21 |
ohorecny2 | @mgoddard: thank you for reply, I will contact @Fl1nt | 15:22 |
mgoddard | wuchunyang: bbezak has been investigating | 15:22 |
mgoddard | wuchunyang: will see if he's available to discuss | 15:22 |
mgoddard | headphoneJames: I'll add RP+1 | 15:22 |
*** jamesbenson has joined #openstack-kolla | 15:22 | |
mgoddard | seems like bbezak is busy | 15:23 |
kevko_ | mgoddard: yes, i want to review , but you know :) ..just joking | 15:23 |
wuchunyang | ok, BTW, who is bbezak . sorry i don't know him. | 15:24 |
mgoddard | wuchunyang: bbezak found that qemu-kvm package is the cause. | 15:24 |
mnasiadka | I've been investigating with him | 15:24 |
mgoddard | ok, mnasiadka can update | 15:24 |
wuchunyang | nice. that's enough to me.. haha | 15:24 |
mgoddard | wuchunyang: bbezak is my colleague, Bartosz Bezak | 15:24 |
wuchunyang | thanks very much. | 15:24 |
mnasiadka | qemu-kvm 5.1 brings the issue, but it seems it's only problematic if the VM is Ubuntu and the flavor is like 1vcpu 2-4G RAM | 15:24 |
mnasiadka | on octavia CentOS 8 test image it works properly | 15:25 |
mnasiadka | on bigger flavors it also works with Ubuntu | 15:25 |
mnasiadka | and the issue is crashing the VM while adding second port (for vrrp) | 15:25 |
mnasiadka | bbezak will raise a bug in Nova/qemu, and do a followup on the mailing list | 15:26 |
mnasiadka | in the meantime, we could pin qemu-kvm to 4.2 in CentOS images | 15:26 |
mnasiadka | wuchunyang/mgoddard: do we have the same problem on Ubuntu hypervisors? | 15:26 |
wuchunyang | no | 15:27 |
wuchunyang | just centos 8 | 15:27 |
bbezak | Sorry I'afk. Will update this change and reply to openstack discuss group | 15:27 |
bbezak | I'm afk | 15:27 |
mnasiadka | bbezak: that's what I wanted to write, that you'll also update the change :) | 15:27 |
wuchunyang | you said if i use centos image. it should work around this problem. right ? | 15:28 |
mgoddard | yes | 15:28 |
wuchunyang | nice. Let's work around for CI first. | 15:29 |
*** muhaha has joined #openstack-kolla | 15:29 | |
mgoddard | ok | 15:30 |
wuchunyang | thanks for mgoddard mgoddard mgoddard | 15:30 |
mgoddard | mgoddard^3 | 15:30 |
mgoddard | #topic yoctozepto: Stop caring about NTP/chrony | 15:30 |
*** openstack changes topic to "yoctozepto: Stop caring about NTP/chrony (Meeting topic: kolla)" | 15:30 | |
mgoddard | yoctozepto stopped caring about time | 15:31 |
yoctozepto | yup | 15:31 |
yoctozepto | that's the plan | 15:31 |
yoctozepto | 8-) | 15:31 |
mnasiadka | you mean finally deprecate chrony container? :) | 15:31 |
yoctozepto | joking aside, this is what we have planned | 15:31 |
yoctozepto | yes, mnasiadka :P | 15:32 |
yoctozepto | we are moving this cycle-cycle | 15:32 |
yoctozepto | time to enact :D | 15:32 |
mnasiadka | thank to gods of total time chaos | 15:32 |
yoctozepto | it is time to drop time | 15:33 |
mgoddard | seriously though, what is the plan? | 15:33 |
*** jbadiapa has joined #openstack-kolla | 15:35 | |
mgoddard | yoctozepto? | 15:38 |
mgoddard | anyone? | 15:38 |
wuchunyang | ? | 15:39 |
mgoddard | seems like we have lost the leader of this discussion. Let's come back to it later | 15:40 |
mgoddard | #topic Wallaby release planning | 15:40 |
*** openstack changes topic to "Wallaby release planning (Meeting topic: kolla)" | 15:40 | |
mgoddard | Friendly reminder: Kolla feature freeze: Mar 29 - Apr 02 | 15:40 |
yoctozepto | mgoddard: sorry, yes, funky connection | 15:41 |
yoctozepto | you can't see me gone because of my bouncer | 15:41 |
yoctozepto | anyhow | 15:41 |
mgoddard | ok | 15:41 |
mgoddard | #undo | 15:41 |
openstack | Removing item from minutes: #topic Wallaby release planning | 15:41 |
yoctozepto | thanks | 15:42 |
yoctozepto | so | 15:42 |
yoctozepto | afair, kayobe handles ntp on the host side | 15:42 |
yoctozepto | and any default install configures ntp nowadays | 15:42 |
yoctozepto | for a myriad of reasons | 15:42 |
yoctozepto | poor man's option is to recommend an ansible module to manage ntp | 15:43 |
yoctozepto | (I guess kayobe uses one already, can go with it) | 15:43 |
yoctozepto | luxurious option would be to integrate it with kolla-ansible | 15:43 |
yoctozepto | but it is blurry | 15:43 |
yoctozepto | and I think it belongs to the level of tools like kayobe | 15:44 |
mgoddard | kayobe actually removed support for NTP config since CentOS 8 | 15:45 |
yoctozepto | we have timesync prechecks in already | 15:45 |
mgoddard | and relies on Kolla Ansible :) | 15:45 |
yoctozepto | mgoddard: oh gosh | 15:45 |
mgoddard | however, we'll probably change change approach | 15:45 |
mgoddard | and configure chrony on CentOS, possibly systemd-resolved or chrony on Ubuntu | 15:46 |
openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: switch to CentOS 8 Stream https://review.opendev.org/c/openstack/kolla/+/772841 | 15:46 |
hrw | why not systemd-resolved on all distros? | 15:46 |
mgoddard | so I don't think it affects this discussion | 15:46 |
mgoddard | hrw: chrony is standard out of the box on centos | 15:46 |
mgoddard | hold on | 15:47 |
mgoddard | sorry, systemd-timesyncd | 15:47 |
*** admin0 has joined #openstack-kolla | 15:47 | |
Fl1nt | I'm in sorry ^^ | 15:47 |
mgoddard | too many systemd-*d in ubuntu :) | 15:47 |
yoctozepto | yup | 15:47 |
yoctozepto | all right then | 15:48 |
yoctozepto | so we have the timesync prechecks since victoria | 15:48 |
yoctozepto | https://docs.openstack.org/releasenotes/kolla-ansible/victoria.html | 15:48 |
yoctozepto | as I mentioned | 15:48 |
yoctozepto | so the sanity checks are there in case users want to go rogue 8-) | 15:48 |
mgoddard | I suppose my only concern is for non-kayobe users who want more than the out of the box NTP config | 15:48 |
yoctozepto | we can adapt docs | 15:48 |
yoctozepto | slap renos | 15:48 |
yoctozepto | go ml | 15:48 |
yoctozepto | yeah, we would point them to ansible roles for that | 15:49 |
yoctozepto | galaxy should have something nice | 15:49 |
yoctozepto | fwiw, I have this part in puppet | 15:49 |
yoctozepto | <no judging> | 15:49 |
Fl1nt | systemd-*d are not too many as it is a way to address concerns to people that can't stop complaining that systemd is a monolithic monster that isn't respecting unix principles :p | 15:50 |
yoctozepto | we have *not* officially deprecated chrony | 15:50 |
yoctozepto | so we should do a deprec this cycle | 15:50 |
Fl1nt | which alternative would you like to use? | 15:50 |
yoctozepto | deprec on both kolla and k-a | 15:51 |
yoctozepto | and with recommendations | 15:51 |
yoctozepto | Fl1nt: give recommendations and leave it to the users | 15:51 |
yoctozepto | ntp is nothing extraordinary nowadays | 15:52 |
Fl1nt | you mean, we let it free for user as we're now relying on hosts right? | 15:52 |
Fl1nt | mgoddard, I synced with ohorecny2 about podman btw | 15:53 |
mgoddard | Fl1nt: ok, we can discuss in a min if you want | 15:54 |
mgoddard | does anyone have opinions on the proposed approach to NTP? | 15:54 |
Fl1nt | TBN: as we talk about wallaby, in order for me to work on SAML2 keystone SSO, I need the OIDC patch to be merged on master if everything is fine as I'll base my patch on it. | 15:54 |
yoctozepto | Fl1nt: what hosts? | 15:54 |
yoctozepto | Fl1nt: we are talking time synchronization | 15:54 |
Fl1nt | yes | 15:54 |
mgoddard | Fl1nt: it's merged | 15:54 |
Fl1nt | mgoddard, oooh sweet, perfect! | 15:55 |
yoctozepto | mgoddard: do you have any opinions? | 15:55 |
Fl1nt | yoctozepto, HW host that provide time to containers services? | 15:55 |
mgoddard | yoctozepto: I gave one | 15:56 |
mgoddard | yoctozepto: I think we should solicit feedback from the community | 15:56 |
Fl1nt | personally I can use chrony or systemd-timesyncd no issue as long as it works ^^ | 15:56 |
yoctozepto | mgoddard: ack | 15:56 |
yoctozepto | Fl1nt: leaving ntp to users | 15:57 |
Fl1nt | yes | 15:57 |
mgoddard | an NTP client isn't difficult to setup, but it might be a bit annoying for users currently relying on k-a for it | 15:57 |
kevko_ | can I just ask why you want to remove chrony/ntp from kolla ? | 15:58 |
mgoddard | but if there are no strong objections from the community then I'm fine with it | 15:58 |
mgoddard | good question kevko_ :) | 15:58 |
kevko_ | :D | 15:58 |
kevko_ | so, what is the answer ? :) | 15:59 |
Fl1nt | honestly, I don't have strong opinion on this one, leave it or remove it I don't really care as long as services are able to get time, personnally all my hosts are ansible bootstraped using chrony for now but don't really care. | 15:59 |
Fl1nt | to switch to something else | 16:00 |
yoctozepto | kevko_: mostly because we are forced to handle whatever the host throws at us | 16:00 |
yoctozepto | and it's extra code to maintain | 16:00 |
yoctozepto | for no real benefit | 16:00 |
yoctozepto | it's not like kolla-ansible brings new ntp features to the table :-) | 16:01 |
Fl1nt | good reason, if it doesn't add any feature or advantage, just remove it. | 16:01 |
mgoddard | well hold on | 16:01 |
mgoddard | kolla-ansible is about automation, not augmenting network time protocols | 16:01 |
Fl1nt | yes | 16:02 |
kevko_ | well, I don't thing that especially some small chrony image (and k-a part) is difficult to maintain | 16:02 |
rockey | not to throw a log on the fire, but doesn't k-a still need to do routine checks on ntpd's on system before relying on the hosts either way? k-a is dependant on synced time, laying this out on the enduser could backfire, right? | 16:02 |
Fl1nt | but the point is, time is already automated pretty much everywhere in enterprise by either the provisioning service or another bootstrap/base ansible internal playbook or puppet or whatever | 16:02 |
mgoddard | so if installing and configuring an NTP client makes users lives easier then that is arguably beneficial | 16:02 |
*** ivan_lin has joined #openstack-kolla | 16:03 | |
mgoddard | rockey: we did add a check that time is synced | 16:03 |
Fl1nt | are you talking about the chrony container or the host bootstrap configuration task ? | 16:03 |
mgoddard | ansible/roles/prechecks/tasks/timesync_checks.yml | 16:03 |
*** dougsz has joined #openstack-kolla | 16:03 | |
mgoddard | Fl1nt: probably both | 16:04 |
yoctozepto | yeah, the point is that most hosts come with some ntp daemon running | 16:04 |
mgoddard | I think for various reasons, we can probably agree that running chrony in a container isn't a great idea | 16:04 |
yoctozepto | and we spend time fixing bugs on detecting those or adapting security profiles | 16:04 |
mgoddard | but mainly what because yoctozepto just said | 16:04 |
yoctozepto | and it's a very basic service nowadays | 16:04 |
yoctozepto | yeah, mgoddard | 16:04 |
kevko_ | agree, I also added some patch .. | 16:05 |
mgoddard | so I'm definitely +1 to removing the container | 16:05 |
kevko_ | so, remember | 16:05 |
Fl1nt | container is probably too much, automate the ntp configuration at task level can still be let as is using chrony or up to the user as long as the appropriate binding is made within services containers. | 16:05 |
*** LinPeiWen has quit IRC | 16:05 | |
mgoddard | the question for me is whether it is useful to manage a service on the host | 16:05 |
mgoddard | the main use case being to use a custom NTP server | 16:05 |
yoctozepto | mgoddard: yup, that's what the recommendations would be for | 16:05 |
yoctozepto | do note k-a never configured a server | 16:06 |
yoctozepto | so users had to do it elsewhere | 16:06 |
mgoddard | I mean, configure the client to use a custom NTP server | 16:06 |
mgoddard | that was possible with the chrony container | 16:07 |
Fl1nt | that only make sense as you pointed out, if kolla/k-a is providing with a ntp server for the cluster, but it's highly unlikely that anyone on enterprise side of things will base their cluster on it as most of ITs or new installation do it on auxilliary/commander/bastion/whatever its named server. | 16:07 |
yoctozepto | yes, my point is that such users are already knowledgeable enough to do it with any ntp role :-) | 16:07 |
*** LinPeiWen has joined #openstack-kolla | 16:07 | |
rockey | mgoddard: well then, as a happy user, i'd +1 the removal too, mostly due to "that most hosts come with some ntp daemon running" statement from yoctozepto | 16:08 |
yoctozepto | ayeee, we are past time | 16:08 |
mgoddard | indeed | 16:08 |
yoctozepto | action this on me | 16:08 |
yoctozepto | and let's close | 16:08 |
yoctozepto | sorry for taking so much time | 16:08 |
mgoddard | #action yoctozepto to ask openstack-discuss about NTP | 16:08 |
mgoddard | Thanks all | 16:08 |
mgoddard | #endmeeting | 16:08 |
*** openstack changes topic to "IRC meetings on Wednesdays @ 15:00 UTC - agenda @ https://goo.gl/OXB0DL | Whiteboard: https://bit.ly/2MM7mWF | IRC channel is *LOGGED* @ http://goo.gl/3mzZ7b" | 16:08 | |
openstack | Meeting ended Wed Feb 24 16:08:45 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:08 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-24-15.00.html | 16:08 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-24-15.00.txt | 16:08 |
openstack | Log: http://eavesdrop.openstack.org/meetings/kolla/2021/kolla.2021-02-24-15.00.log.html | 16:08 |
Fl1nt | thanks ^^ | 16:09 |
*** ivan_lin has quit IRC | 16:10 | |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Refactor veth configuration role variable generation https://review.opendev.org/c/openstack/kayobe/+/777176 | 16:10 |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: WIP: Ubuntu: support systemd-networkd https://review.opendev.org/c/openstack/kayobe/+/776500 | 16:11 |
*** LinPeiWen has quit IRC | 16:11 | |
*** LinPeiWen has joined #openstack-kolla | 16:11 | |
*** k_mouza_ has joined #openstack-kolla | 16:14 | |
openstackgerrit | wu.chunyang proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/c/openstack/kolla-ansible/+/754285 | 16:17 |
*** k_mouza has quit IRC | 16:17 | |
openstackgerrit | wu.chunyang proposed openstack/kolla-ansible master: CI: octavia: create and test a load balancer https://review.opendev.org/c/openstack/kolla-ansible/+/754285 | 16:18 |
*** wuchunyang has quit IRC | 16:20 | |
*** dougsz has quit IRC | 16:25 | |
kevko_ | if noone is going to talk .. | 16:26 |
kevko_ | can I discuss my findings with keystone weird approach (was discussing these days here ..) | 16:26 |
*** vishalmanchanda has quit IRC | 16:36 | |
openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: CI: Add ssh retries https://review.opendev.org/c/openstack/kolla-ansible/+/777402 | 16:42 |
openstackgerrit | Michal Nasiadka proposed openstack/kayobe master: Add output printing to host-command-run https://review.opendev.org/c/openstack/kayobe/+/776471 | 16:53 |
Fl1nt | kevko_, go ahead | 16:55 |
*** jamesbenson has quit IRC | 17:01 | |
mnasiadka | yoctozepto: where did we end up with Cinder Active/Active? Some rabbit hole? :) | 17:01 |
*** luksky has joined #openstack-kolla | 17:02 | |
*** jamesbenson has joined #openstack-kolla | 17:02 | |
kevko_ | Well, I found that if I restart keepalived, and VIP is switched to another controller WSGI Workers are busy and trying to connect to DB .. | 17:07 |
kevko_ | BUT keepalived sent GARP so ConnectionPool on host where keystone is invalid and I can see in strace for wsgi process that it is trying to 'select 1' | 17:08 |
kevko_ | if i kill wsgi process or restart apache ..everything is OK ..does anyone now how to fix it ? | 17:09 |
kevko_ | maybe it can be done by some kernel tuning or wsgidaemonprocess setting in apache ? | 17:09 |
kevko_ | I think this is problem not only for keystone ..but for OS services globally | 17:09 |
kevko_ | strace -> http://paste.openstack.org/show/802969/ | 17:10 |
kevko_ | Fl1nt: ^^^ | 17:12 |
*** k_mouza_ has quit IRC | 17:14 | |
*** k_mouza has joined #openstack-kolla | 17:14 | |
*** amoralej is now known as amoralej|off | 17:14 | |
Fl1nt | kevko_, interesting, how many clients and nodes on your cluster? | 17:14 |
kevko_ | Fl1nt: well, now I can reproduce it with bash which is sending token issue to keystone :D | 17:15 |
kevko_ | Fl1nt: did you see my debug video I post here few days ago ? | 17:15 |
Fl1nt | nope ^^ | 17:16 |
kevko_ | Fl1nt: let me send you | 17:16 |
kevko_ | Fl1nt: https://download.kevko.ultimum.cloud/video_debug.mp4 | 17:17 |
kevko_ | Fl1nt: on left I have controller0 on right controller1 ..in center windows i am running ip a (254 is vip) ..and when I switch VIP ..you can see keystone where VIP was before is stucked | 17:18 |
kevko_ | Did you catch the point ? | 17:19 |
mnasiadka | kevko_: you mean that the connection to MariaDB through the VIP is not closed on keepalived switch? | 17:19 |
Fl1nt | kevko_, I think your situation will (is?) fixed by the patch that lower the connection pool of services to 0 | 17:20 |
kevko_ | mnasiadka: well, yes ..but it's not that easy (i think) | 17:21 |
kevko_ | Fl1nt: nope, I've lowered pool to 0 and it is same | 17:22 |
Fl1nt | hum... I'll have a look on tomorrow ;-) | 17:23 |
Fl1nt | as it's pretty late in here ^^ | 17:23 |
kevko_ | mnasiadka: point is that if worker start to process request and VIP is switched in the middle..it is broken | 17:23 |
kevko_ | late ? | 17:24 |
kevko_ | where are u from :P | 17:24 |
mnasiadka | kevko_: that we know for a long time :) | 17:24 |
kevko_ | mnasiadka: well, but I think it should be fixed :D ... I know that if i restart haproxy ..it will be ok ..but it is just workaround | 17:25 |
kevko_ | (or keystone itself) | 17:25 |
Fl1nt | kevko_, late for work stuff :p | 17:26 |
kevko_ | but, i think that there has to be some solution | 17:26 |
Fl1nt | I need to relax. | 17:26 |
kevko_ | Fl1nt: ah, ok .. This is my relax :) | 17:26 |
kevko_ | I was just wondering there should be some kernel tuning or maybe wsgi option to fix this behaviour | 17:26 |
openstackgerrit | Mark Goddard proposed openstack/kayobe master: Ubuntu: update Apt cache before package installation https://review.opendev.org/c/openstack/kayobe/+/775130 | 17:33 |
*** bengates has quit IRC | 17:34 | |
*** gfidente is now known as gfidente|afk | 17:40 | |
*** jonaspaulo has joined #openstack-kolla | 17:42 | |
*** jamesbenson has quit IRC | 17:47 | |
*** jamesbenson has joined #openstack-kolla | 17:50 | |
*** jamesbenson has quit IRC | 17:52 | |
*** jamesbenson has joined #openstack-kolla | 17:52 | |
*** Luzi has quit IRC | 17:56 | |
*** bengates has joined #openstack-kolla | 17:57 | |
*** jamesbenson has quit IRC | 17:58 | |
*** maharg101 has quit IRC | 18:00 | |
*** bengates has quit IRC | 18:01 | |
*** bkslash has joined #openstack-kolla | 18:03 | |
*** bkslash has quit IRC | 18:04 | |
*** bkslash has joined #openstack-kolla | 18:13 | |
*** jonaspaulo has quit IRC | 18:15 | |
*** bkslash has quit IRC | 18:15 | |
*** muhaha has quit IRC | 18:21 | |
*** Fl1nt has quit IRC | 18:30 | |
*** andrewbonney has quit IRC | 18:42 | |
*** maharg101 has joined #openstack-kolla | 18:53 | |
*** k_mouza has quit IRC | 19:00 | |
*** bengates has joined #openstack-kolla | 19:13 | |
*** cah_link has quit IRC | 19:17 | |
*** eliaswimmer has quit IRC | 19:18 | |
*** bengates has quit IRC | 19:20 | |
*** eliaswimmer has joined #openstack-kolla | 19:51 | |
*** bengates has joined #openstack-kolla | 19:57 | |
*** bengates has quit IRC | 20:01 | |
*** k_mouza has joined #openstack-kolla | 20:14 | |
*** k_mouza has quit IRC | 20:19 | |
*** zzzeek has quit IRC | 20:34 | |
*** zzzeek has joined #openstack-kolla | 20:34 | |
openstackgerrit | Pierre Riteau proposed openstack/kayobe stable/train: CI: use overlay2 storage driver for overcloud deploy https://review.opendev.org/c/openstack/kayobe/+/777373 | 20:43 |
*** maharg101 has quit IRC | 20:54 | |
*** eliaswimmer has quit IRC | 20:57 | |
*** eliaswimmer has joined #openstack-kolla | 21:08 | |
*** LaurentDumont has quit IRC | 21:17 | |
*** LaurentDumont has joined #openstack-kolla | 21:19 | |
*** kevko_ has quit IRC | 21:24 | |
*** zzzeek has quit IRC | 21:26 | |
*** zzzeek has joined #openstack-kolla | 21:28 | |
*** zzzeek has quit IRC | 21:49 | |
*** zzzeek has joined #openstack-kolla | 21:53 | |
*** eliaswimmer has quit IRC | 21:58 | |
*** ohorecny2 has quit IRC | 22:12 | |
*** bengates has joined #openstack-kolla | 22:38 | |
*** luksky has quit IRC | 22:39 | |
*** bengates has quit IRC | 22:49 | |
*** maharg101 has joined #openstack-kolla | 22:51 | |
*** maharg101 has quit IRC | 22:56 | |
*** zzzeek has quit IRC | 23:14 | |
*** zzzeek has joined #openstack-kolla | 23:15 | |
*** e0ne has quit IRC | 23:17 | |
*** ivan_lin has joined #openstack-kolla | 23:22 | |
*** LinPeiWen has quit IRC | 23:25 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!