| *** brinzhang0 has quit IRC | 00:37 | |
| *** brinzhang has joined #openstack-kolla | 00:37 | |
| *** wuchunyang has joined #openstack-kolla | 00:56 | |
| *** dave-mccowan has quit IRC | 01:59 | |
| *** LinPeiWen has joined #openstack-kolla | 02:07 | |
| *** hongbin has joined #openstack-kolla | 02:12 | |
| *** oncall-pokemon has joined #openstack-kolla | 02:37 | |
| *** cah_link has joined #openstack-kolla | 03:01 | |
| *** skramaja has joined #openstack-kolla | 03:03 | |
| *** wuchunyang has quit IRC | 03:05 | |
| *** hongbin has quit IRC | 03:16 | |
| *** wuchunyang has joined #openstack-kolla | 03:55 | |
| *** wuchunyang has quit IRC | 04:00 | |
| *** k_mouza has joined #openstack-kolla | 04:16 | |
| *** k_mouza has quit IRC | 04:21 | |
| *** cah_link has quit IRC | 04:47 | |
| *** oncall-pokemon has quit IRC | 05:07 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-kolla | 05:33 | |
| *** wuchunyang has joined #openstack-kolla | 05:47 | |
| *** vishalmanchanda has joined #openstack-kolla | 05:48 | |
| *** hjensas has quit IRC | 05:57 | |
| *** hjensas has joined #openstack-kolla | 05:59 | |
| *** suff has joined #openstack-kolla | 06:17 | |
| *** cah_link has joined #openstack-kolla | 07:05 | |
| *** brinzhang_ has joined #openstack-kolla | 07:21 | |
| *** brinzhang has quit IRC | 07:24 | |
| *** wuchunyang has quit IRC | 07:58 | |
| *** wuchunyang has joined #openstack-kolla | 08:05 | |
| *** xinliang has joined #openstack-kolla | 08:08 | |
| *** rpittau|afk is now known as rpittau | 08:19 | |
| mnasiadka | morning | 08:21 |
|---|---|---|
| *** bengates has joined #openstack-kolla | 08:24 | |
| *** jbadiapa has joined #openstack-kolla | 08:37 | |
| *** dougsz has joined #openstack-kolla | 08:49 | |
| *** bengates has quit IRC | 08:54 | |
| *** bengates has joined #openstack-kolla | 09:02 | |
| *** e0ne has joined #openstack-kolla | 09:04 | |
| *** bengates has quit IRC | 09:11 | |
| *** ivan_lin has quit IRC | 09:16 | |
| *** ivan_lin has joined #openstack-kolla | 09:16 | |
| *** hrw has joined #openstack-kolla | 09:16 | |
| *** bengates has joined #openstack-kolla | 09:17 | |
| *** gfidente has joined #openstack-kolla | 09:21 | |
| hrw | https://review.opendev.org/#/c/762107/2 - vote! | 09:37 |
| patchbot | patch 762107 - kolla (stable/victoria) - Use Victoria RDO - 2 patch sets | 09:37 |
| mnasiadka | yeah, centos8-binary passed on this run | 09:40 |
| mnasiadka | (both kolla and k-a) | 09:40 |
| *** sean-k-mooney1 has joined #openstack-kolla | 09:42 | |
| *** sean-k-mooney has quit IRC | 09:43 | |
| *** zzzeek has quit IRC | 09:45 | |
| *** zzzeek has joined #openstack-kolla | 09:46 | |
| *** hongbin has joined #openstack-kolla | 09:46 | |
| *** xinliang has quit IRC | 09:49 | |
| *** k_mouza has joined #openstack-kolla | 09:55 | |
| *** jaicaa has quit IRC | 09:57 | |
| *** jaicaa has joined #openstack-kolla | 10:00 | |
| *** LinPeiWen has quit IRC | 10:04 | |
| *** zzzeek has quit IRC | 10:22 | |
| *** zzzeek has joined #openstack-kolla | 10:25 | |
| *** muhaha has joined #openstack-kolla | 10:30 | |
| *** hongbin has quit IRC | 10:36 | |
| *** hongbin has joined #openstack-kolla | 10:37 | |
| *** LinPeiWen has joined #openstack-kolla | 10:38 | |
| *** ivan_lin has quit IRC | 10:38 | |
| *** k_mouza has quit IRC | 10:39 | |
| *** k_mouza has joined #openstack-kolla | 10:48 | |
| *** kukacz has quit IRC | 10:51 | |
| *** kukacz has joined #openstack-kolla | 10:56 | |
| *** sean-k-mooney1 is now known as sean-k-mooney | 11:36 | |
| *** jan00 has joined #openstack-kolla | 11:42 | |
| *** hongbin has quit IRC | 11:48 | |
| *** k_mouza has quit IRC | 11:51 | |
| *** bengates has quit IRC | 12:01 | |
| jan00 | We are seeing failures building an "ubuntu-source-openstack-base" image with kolla from stable/ussuri branch: https://pastebin.com/EP9TRxg8 . The last successful CI run was from "stable/ussuri HEAD~2". I am just trying to confirm whether this is a known issue or something specific to our build job before digging into it | 12:03 |
| mnasiadka | jan00: it's probably something on your side - this is a link to todays periodic build&publish jobhttps://zuul.openstack.org/build/5ab26769e4fe44a99e405ce0db57b4b2 | 12:10 |
| *** k_mouza has joined #openstack-kolla | 12:14 | |
| *** wuchunyang has joined #openstack-kolla | 12:31 | |
| *** wuchunyang has quit IRC | 12:37 | |
| jan00 | mnasiadka: Thanks, I guess I'll start digging then :) | 12:45 |
| openstackgerrit | Merged openstack/kolla stable/victoria: Use Victoria RDO https://review.opendev.org/762107 | 12:51 |
| *** bengates has joined #openstack-kolla | 12:54 | |
| mnasiadka | yay, centos-binary victoria images tomorrow | 12:56 |
| *** devfaz has quit IRC | 13:32 | |
| *** devfaz has joined #openstack-kolla | 13:37 | |
| *** Luzi has joined #openstack-kolla | 13:44 | |
| *** devfaz has quit IRC | 13:48 | |
| kplant | we're currently trying to pull the "network" role out of the "control" hosts and onto dedicated nodes, in our lab testing we're seeing the routers having to be deleted and recreated | 13:51 |
| kplant | is there something we're missing that would make this step of deleting vrouters unnecessary? | 13:51 |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: horizon: Support custom themes https://review.opendev.org/761364 | 13:52 |
| *** devfaz has joined #openstack-kolla | 13:54 | |
| *** TrevorV has joined #openstack-kolla | 14:01 | |
| mnasiadka | kplant: well, do you remove the agents from control hosts in that testing? | 14:06 |
| kplant | yeah | 14:06 |
| kplant | and all of the neutron_* containers, besides neutron_server | 14:07 |
| kplant | as well as removing openvswitch_* | 14:07 |
| mnasiadka | well, neutron should reschedule the routers to alive agents | 14:07 |
| mnasiadka | maybe you need to disable them via neutron API (or remove them) | 14:07 |
| kplant | it doesn't seem to be, that's what i'm currently testing | 14:08 |
| kplant | i killed network-02 (where the current vrouter is) and waited for neutron to change the state to 'XXX' | 14:08 |
| kplant | and it's not moving the vrouter | 14:08 |
| kplant | well alive is 'XXX' state is still 'UP' | 14:09 |
| kplant | which is strange | 14:09 |
| mnasiadka | well, then try to change the state to disabled :) | 14:10 |
| kplant | yeah, wondering why it isn't happening automagically | 14:11 |
| *** wuchunyang has joined #openstack-kolla | 14:11 | |
| mnasiadka | kplant: that's probably a question you should ask to Neutrinos ;) | 14:15 |
| kplant | yeah :-( | 14:15 |
| kplant | i just caught that the state was staying UP | 14:15 |
| kplant | not really a kolla thing | 14:15 |
| kplant | ugh... even if i disable the agents it doesn't move | 14:25 |
| kplant | ffffffffffffffffffff | 14:25 |
| *** kwazar has joined #openstack-kolla | 14:30 | |
| *** Luzi has quit IRC | 14:32 | |
| *** wuchunyang has quit IRC | 14:43 | |
| priteau | kplant: have you enabled L3 HA? | 14:45 |
| kplant | no, not here. thankfully yes in production | 14:46 |
| *** vishalmanchanda has quit IRC | 14:47 | |
| *** wuchunyang has joined #openstack-kolla | 14:49 | |
| mnasiadka | kplant: so then it might not reschedule ;) | 14:50 |
| kplant | yeah, was hoping for some kind of resiliency | 14:51 |
| priteau | So have you run `network agent remove router` and `openstack network agent add router` commands to manually migrate the L3? | 14:51 |
| kplant | yeah that works fine, just miffed there's no rescheduling at all | 14:51 |
| priteau | Check allow_automatic_l3agent_failover | 14:52 |
| priteau | It's false by default | 14:53 |
| kplant | ooo | 14:53 |
| kplant | is that in neutron.conf ? | 14:53 |
| kplant | yeah looks it, thanks! | 14:53 |
| priteau | https://docs.openstack.org/ocata/config-reference/networking/networking_options_reference.html | 14:53 |
| kplant | yeah this is exactly what i was expecting | 14:54 |
| kplant | perfect :-) | 14:54 |
| *** wuchunyang has quit IRC | 15:01 | |
| JamesBenson | Q: I've security scanned a bunch of the docker images, I was wondering if there has been any push to mitigate some of the vulnerabilities there? Many of them are micro-version upgrades, like upgrading libvirt from version 6.0.0-25.2 to 6.0.0-28 | 15:09 |
| JamesBenson | yoctozepto, I thought I remember kolla 4.x was able to configure swift to use ceph as a storage backend. | 15:12 |
| yoctozepto | JamesBenson: internal ceph | 15:13 |
| yoctozepto | kolla-ceph | 15:13 |
| yoctozepto | or whatever to call it | 15:13 |
| yoctozepto | JamesBenson: hmm, we always install the latest from repos | 15:13 |
| yoctozepto | JamesBenson: so it should not have any already-fixed vulnerabilities | 15:14 |
| *** born2bake has joined #openstack-kolla | 15:14 | |
| yoctozepto | JamesBenson: if you have a specific case (distro, version, etc.), we can discuss it on launchpad in a bug report | 15:14 |
| yoctozepto | we might discover that distros are simply lagging in patching | 15:15 |
| JamesBenson | I don't have any specific case, just figured it would help the cause to increase security baseline. ;-) | 15:15 |
| mnasiadka | yoctozepto: Centos victoria package in extras just landed, will push a change ;) | 15:20 |
| *** muhaha has quit IRC | 15:21 | |
| *** e0ne has quit IRC | 15:22 | |
| *** e0ne has joined #openstack-kolla | 15:23 | |
| yoctozepto | mnasiadka: eh :D | 15:35 |
| yoctozepto | JamesBenson: but it's a valid point! | 15:35 |
| yoctozepto | JamesBenson: please report | 15:36 |
| mnasiadka | yoctozepto: it’s always like this, you wait a week, nothing happens, and just when you implement a workaround... | 15:36 |
| yoctozepto | :D | 15:36 |
| JamesBenson | thanks and sorry for going back and forth between security and ceph/swift. | 15:36 |
| JamesBenson | one last thing about swift/ceph: Is there any interest in bringing back that feature for external ceph/swift integration? | 15:37 |
| mnasiadka | JamesBenson: just submit a review if you need it and we’ll review :) | 15:37 |
| *** skramaja has quit IRC | 15:38 | |
| *** bengates has quit IRC | 15:48 | |
| *** bengates has joined #openstack-kolla | 15:48 | |
| *** cah_link has quit IRC | 15:52 | |
| yoctozepto | JamesBenson: I remember others asking for it too | 15:57 |
| yoctozepto | it could have already been proposed too | 15:57 |
| yoctozepto | or at least as a report in launchpad | 15:57 |
| JamesBenson | okay, it looks like something that shouldn't be too hard to incorporate. | 15:57 |
| JamesBenson | but granted, that's coming from someone who hasn't done it yet ;-) | 15:58 |
| yoctozepto | :D | 16:05 |
| *** k_mouza has quit IRC | 16:06 | |
| openstackgerrit | Michal Nasiadka proposed openstack/kolla stable/victoria: Switch to centos-release-openstack-victoria package https://review.opendev.org/762546 | 16:16 |
| *** e0ne has quit IRC | 16:23 | |
| *** e0ne has joined #openstack-kolla | 16:24 | |
| *** e0ne has quit IRC | 16:24 | |
| mnasiadka | uhh, new package not yet on mirrors, let's wait until tomorrow | 16:28 |
| *** bengates_ has joined #openstack-kolla | 16:33 | |
| *** bengates has quit IRC | 16:36 | |
| *** k_mouza has joined #openstack-kolla | 16:40 | |
| *** bengates_ has quit IRC | 16:45 | |
| *** bengates has joined #openstack-kolla | 16:46 | |
| *** bengates has quit IRC | 16:50 | |
| *** jan00 has quit IRC | 17:05 | |
| *** jan00 has joined #openstack-kolla | 17:08 | |
| *** k_mouza has quit IRC | 17:21 | |
| *** k_mouza has joined #openstack-kolla | 17:21 | |
| *** dougsz has quit IRC | 17:31 | |
| *** k_mouza has quit IRC | 17:45 | |
| *** k_mouza has joined #openstack-kolla | 17:46 | |
| *** dking has quit IRC | 17:47 | |
| *** rpittau is now known as rpittau|afk | 17:51 | |
| *** k_mouza has quit IRC | 17:51 | |
| *** k_mouza has joined #openstack-kolla | 17:52 | |
| *** k_mouza has quit IRC | 17:52 | |
| *** gfidente is now known as gfidente|afk | 18:16 | |
| *** numans_ is now known as numans | 18:16 | |
| *** jan00 has quit IRC | 18:20 | |
| *** gfidente|afk has quit IRC | 18:27 | |
| *** jonaspaulo has joined #openstack-kolla | 18:54 | |
| *** rouk has joined #openstack-kolla | 18:56 | |
| *** jonaspaulo has quit IRC | 18:56 | |
| rouk | cinder-volume no longer is configured as rbd:volumes@rbd-1 in ussuri, is this expected? it breaks all our volumes... | 18:56 |
| rouk | now we get a cinder-volume agent per control node, which... what happens when one node goes down? | 18:57 |
| openstackgerrit | Merged openstack/kolla master: Revert "Switch to Victoria stable branches" https://review.opendev.org/761743 | 19:25 |
| rouk | seems "backend_host=rbd:volumes" cant be set by kolla's external ceph, and will brick anyone who doesnt manually patch it in. | 20:24 |
| *** TrevorV has quit IRC | 20:34 | |
| yoctozepto | rouk: report on launchpad | 20:46 |
| yoctozepto | we could miss something (as always) | 20:47 |
| rouk | yeah, rbd-1 needs backend_host=rbd:volumes added else all old volumes from before ussuri will brick (as in the before times, external ceph was manual, and had this set) | 20:47 |
| rouk | and also it will result in 3 cinder-volume services, 1 for each control node, all pointing to the same storage, which means one control node dying you lose all volumes that are "on" that node, even though theyre in ceph. | 20:48 |
| rouk | will open report for it. | 20:48 |
| rouk | yoctozepto: for ref https://bugs.launchpad.net/kolla-ansible/+bug/1904062 | 21:20 |
| openstack | Launchpad bug 1904062 in kolla-ansible "external_ceph cinder-volume config break volumes on ussuri upgrade" [Undecided,New] | 21:20 |
| *** jbadiapa has quit IRC | 21:30 | |
| *** k_mouza has joined #openstack-kolla | 21:53 | |
| *** k_mouza has quit IRC | 21:57 | |
| *** zzzeek has quit IRC | 22:34 | |
| *** zzzeek has joined #openstack-kolla | 22:34 | |
| *** ysirndjuro has quit IRC | 23:24 | |
| *** suff has quit IRC | 23:48 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!