Thursday, 2020-11-12

*** brinzhang0 has quit IRC00:37
*** brinzhang has joined #openstack-kolla00:37
*** wuchunyang has joined #openstack-kolla00:56
*** dave-mccowan has quit IRC01:59
*** LinPeiWen has joined #openstack-kolla02:07
*** hongbin has joined #openstack-kolla02:12
*** oncall-pokemon has joined #openstack-kolla02:37
*** cah_link has joined #openstack-kolla03:01
*** skramaja has joined #openstack-kolla03:03
*** wuchunyang has quit IRC03:05
*** hongbin has quit IRC03:16
*** wuchunyang has joined #openstack-kolla03:55
*** wuchunyang has quit IRC04:00
*** k_mouza has joined #openstack-kolla04:16
*** k_mouza has quit IRC04:21
*** cah_link has quit IRC04:47
*** oncall-pokemon has quit IRC05:07
*** evrardjp has quit IRC05:33
*** evrardjp has joined #openstack-kolla05:33
*** wuchunyang has joined #openstack-kolla05:47
*** vishalmanchanda has joined #openstack-kolla05:48
*** hjensas has quit IRC05:57
*** hjensas has joined #openstack-kolla05:59
*** suff has joined #openstack-kolla06:17
*** cah_link has joined #openstack-kolla07:05
*** brinzhang_ has joined #openstack-kolla07:21
*** brinzhang has quit IRC07:24
*** wuchunyang has quit IRC07:58
*** wuchunyang has joined #openstack-kolla08:05
*** xinliang has joined #openstack-kolla08:08
*** rpittau|afk is now known as rpittau08:19
mnasiadkamorning08:21
*** bengates has joined #openstack-kolla08:24
*** jbadiapa has joined #openstack-kolla08:37
*** dougsz has joined #openstack-kolla08:49
*** bengates has quit IRC08:54
*** bengates has joined #openstack-kolla09:02
*** e0ne has joined #openstack-kolla09:04
*** bengates has quit IRC09:11
*** ivan_lin has quit IRC09:16
*** ivan_lin has joined #openstack-kolla09:16
*** hrw has joined #openstack-kolla09:16
*** bengates has joined #openstack-kolla09:17
*** gfidente has joined #openstack-kolla09:21
hrwhttps://review.opendev.org/#/c/762107/2 - vote!09:37
patchbotpatch 762107 - kolla (stable/victoria) - Use Victoria RDO - 2 patch sets09:37
mnasiadkayeah, centos8-binary passed on this run09:40
mnasiadka(both kolla and k-a)09:40
*** sean-k-mooney1 has joined #openstack-kolla09:42
*** sean-k-mooney has quit IRC09:43
*** zzzeek has quit IRC09:45
*** zzzeek has joined #openstack-kolla09:46
*** hongbin has joined #openstack-kolla09:46
*** xinliang has quit IRC09:49
*** k_mouza has joined #openstack-kolla09:55
*** jaicaa has quit IRC09:57
*** jaicaa has joined #openstack-kolla10:00
*** LinPeiWen has quit IRC10:04
*** zzzeek has quit IRC10:22
*** zzzeek has joined #openstack-kolla10:25
*** muhaha has joined #openstack-kolla10:30
*** hongbin has quit IRC10:36
*** hongbin has joined #openstack-kolla10:37
*** LinPeiWen has joined #openstack-kolla10:38
*** ivan_lin has quit IRC10:38
*** k_mouza has quit IRC10:39
*** k_mouza has joined #openstack-kolla10:48
*** kukacz has quit IRC10:51
*** kukacz has joined #openstack-kolla10:56
*** sean-k-mooney1 is now known as sean-k-mooney11:36
*** jan00 has joined #openstack-kolla11:42
*** hongbin has quit IRC11:48
*** k_mouza has quit IRC11:51
*** bengates has quit IRC12:01
jan00We are seeing failures building an "ubuntu-source-openstack-base" image with kolla from stable/ussuri branch: https://pastebin.com/EP9TRxg8 . The last successful CI run was from "stable/ussuri HEAD~2". I am just trying to confirm whether this is a known issue or something specific to our build job before digging into it12:03
mnasiadkajan00: it's probably something on your side - this is a link to todays periodic build&publish jobhttps://zuul.openstack.org/build/5ab26769e4fe44a99e405ce0db57b4b212:10
*** k_mouza has joined #openstack-kolla12:14
*** wuchunyang has joined #openstack-kolla12:31
*** wuchunyang has quit IRC12:37
jan00mnasiadka: Thanks, I guess I'll start digging then :)12:45
openstackgerritMerged openstack/kolla stable/victoria: Use Victoria RDO  https://review.opendev.org/76210712:51
*** bengates has joined #openstack-kolla12:54
mnasiadkayay, centos-binary victoria images tomorrow12:56
*** devfaz has quit IRC13:32
*** devfaz has joined #openstack-kolla13:37
*** Luzi has joined #openstack-kolla13:44
*** devfaz has quit IRC13:48
kplantwe're currently trying to pull the "network" role out of the "control" hosts and onto dedicated nodes, in our lab testing we're seeing the routers having to be deleted and recreated13:51
kplantis there something we're missing that would make this step of deleting vrouters unnecessary?13:51
openstackgerritMichal Nasiadka proposed openstack/kolla-ansible master: WIP: horizon: Support custom themes  https://review.opendev.org/76136413:52
*** devfaz has joined #openstack-kolla13:54
*** TrevorV has joined #openstack-kolla14:01
mnasiadkakplant: well, do you remove the agents from control hosts in that testing?14:06
kplantyeah14:06
kplantand all of the neutron_* containers, besides neutron_server14:07
kplantas well as removing openvswitch_*14:07
mnasiadkawell, neutron should reschedule the routers to alive agents14:07
mnasiadkamaybe you need to disable them via neutron API (or remove them)14:07
kplantit doesn't seem to be, that's what i'm currently testing14:08
kplanti killed network-02 (where the current vrouter is) and waited for neutron to change the state to 'XXX'14:08
kplantand it's not moving the vrouter14:08
kplantwell alive is 'XXX' state is still 'UP'14:09
kplantwhich is strange14:09
mnasiadkawell, then try to change the state to disabled :)14:10
kplantyeah, wondering why it isn't happening automagically14:11
*** wuchunyang has joined #openstack-kolla14:11
mnasiadkakplant: that's probably a question you should ask to Neutrinos ;)14:15
kplantyeah :-(14:15
kplanti just caught that the state was staying UP14:15
kplantnot really a kolla thing14:15
kplantugh... even if i disable the agents it doesn't move14:25
kplantffffffffffffffffffff14:25
*** kwazar has joined #openstack-kolla14:30
*** Luzi has quit IRC14:32
*** wuchunyang has quit IRC14:43
priteaukplant: have you enabled L3 HA?14:45
kplantno, not here. thankfully yes in production14:46
*** vishalmanchanda has quit IRC14:47
*** wuchunyang has joined #openstack-kolla14:49
mnasiadkakplant: so then it might not reschedule ;)14:50
kplantyeah, was hoping for some kind of resiliency14:51
priteauSo have you run `network agent remove router` and `openstack network agent add router` commands to manually migrate the L3?14:51
kplantyeah that works fine, just miffed there's no rescheduling at all14:51
priteauCheck allow_automatic_l3agent_failover14:52
priteauIt's false by default14:53
kplantooo14:53
kplantis that in neutron.conf ?14:53
kplantyeah looks it, thanks!14:53
priteauhttps://docs.openstack.org/ocata/config-reference/networking/networking_options_reference.html14:53
kplantyeah this is exactly what i was expecting14:54
kplantperfect :-)14:54
*** wuchunyang has quit IRC15:01
JamesBensonQ: I've security scanned a bunch of the docker images, I was wondering if there has been any push to mitigate some of the vulnerabilities there?  Many of them are micro-version upgrades, like upgrading libvirt from version 6.0.0-25.2 to 6.0.0-2815:09
JamesBensonyoctozepto, I thought I remember kolla 4.x was able to configure swift to use ceph as a storage backend.15:12
yoctozeptoJamesBenson: internal ceph15:13
yoctozeptokolla-ceph15:13
yoctozeptoor whatever to call it15:13
yoctozeptoJamesBenson: hmm, we always install the latest from repos15:13
yoctozeptoJamesBenson: so it should not have any already-fixed vulnerabilities15:14
*** born2bake has joined #openstack-kolla15:14
yoctozeptoJamesBenson: if you have a specific case (distro, version, etc.), we can discuss it on launchpad in a bug report15:14
yoctozeptowe might discover that distros are simply lagging in patching15:15
JamesBensonI don't have any specific case, just figured it would help the cause to increase security baseline. ;-)15:15
mnasiadkayoctozepto: Centos victoria package in extras just landed, will push a change ;)15:20
*** muhaha has quit IRC15:21
*** e0ne has quit IRC15:22
*** e0ne has joined #openstack-kolla15:23
yoctozeptomnasiadka: eh :D15:35
yoctozeptoJamesBenson: but it's a valid point!15:35
yoctozeptoJamesBenson: please report15:36
mnasiadkayoctozepto: it’s always like this, you wait a week, nothing happens, and just when you implement a workaround...15:36
yoctozepto:D15:36
JamesBensonthanks and sorry for going back and forth between security and ceph/swift.15:36
JamesBensonone last thing about swift/ceph:  Is there any interest in bringing back that feature for external ceph/swift integration?15:37
mnasiadkaJamesBenson: just submit a review if you need it and we’ll review :)15:37
*** skramaja has quit IRC15:38
*** bengates has quit IRC15:48
*** bengates has joined #openstack-kolla15:48
*** cah_link has quit IRC15:52
yoctozeptoJamesBenson: I remember others asking for it too15:57
yoctozeptoit could have already been proposed too15:57
yoctozeptoor at least as a report in launchpad15:57
JamesBensonokay, it looks like something that shouldn't be too hard to incorporate.15:57
JamesBensonbut granted, that's coming from someone who hasn't done it yet ;-)15:58
yoctozepto:D16:05
*** k_mouza has quit IRC16:06
openstackgerritMichal Nasiadka proposed openstack/kolla stable/victoria: Switch to centos-release-openstack-victoria package  https://review.opendev.org/76254616:16
*** e0ne has quit IRC16:23
*** e0ne has joined #openstack-kolla16:24
*** e0ne has quit IRC16:24
mnasiadkauhh, new package not yet on mirrors, let's wait until tomorrow16:28
*** bengates_ has joined #openstack-kolla16:33
*** bengates has quit IRC16:36
*** k_mouza has joined #openstack-kolla16:40
*** bengates_ has quit IRC16:45
*** bengates has joined #openstack-kolla16:46
*** bengates has quit IRC16:50
*** jan00 has quit IRC17:05
*** jan00 has joined #openstack-kolla17:08
*** k_mouza has quit IRC17:21
*** k_mouza has joined #openstack-kolla17:21
*** dougsz has quit IRC17:31
*** k_mouza has quit IRC17:45
*** k_mouza has joined #openstack-kolla17:46
*** dking has quit IRC17:47
*** rpittau is now known as rpittau|afk17:51
*** k_mouza has quit IRC17:51
*** k_mouza has joined #openstack-kolla17:52
*** k_mouza has quit IRC17:52
*** gfidente is now known as gfidente|afk18:16
*** numans_ is now known as numans18:16
*** jan00 has quit IRC18:20
*** gfidente|afk has quit IRC18:27
*** jonaspaulo has joined #openstack-kolla18:54
*** rouk has joined #openstack-kolla18:56
*** jonaspaulo has quit IRC18:56
roukcinder-volume no longer is configured as rbd:volumes@rbd-1 in ussuri, is this expected? it breaks all our volumes...18:56
rouknow we get a cinder-volume agent per control node, which... what happens when one node goes down?18:57
openstackgerritMerged openstack/kolla master: Revert "Switch to Victoria stable branches"  https://review.opendev.org/76174319:25
roukseems "backend_host=rbd:volumes" cant be set by kolla's external ceph,  and will brick anyone who doesnt manually patch it in.20:24
*** TrevorV has quit IRC20:34
yoctozeptorouk: report on launchpad20:46
yoctozeptowe could miss something (as always)20:47
roukyeah, rbd-1 needs backend_host=rbd:volumes added else all old volumes from before ussuri will brick (as in the before times, external ceph was manual, and had this set)20:47
roukand also it will result in 3 cinder-volume services, 1 for each control node, all pointing to the same storage, which means one control node dying you lose all volumes that are "on" that node, even though theyre in ceph.20:48
roukwill open report for it.20:48
roukyoctozepto: for ref https://bugs.launchpad.net/kolla-ansible/+bug/190406221:20
openstackLaunchpad bug 1904062 in kolla-ansible "external_ceph cinder-volume config break volumes on ussuri upgrade" [Undecided,New]21:20
*** jbadiapa has quit IRC21:30
*** k_mouza has joined #openstack-kolla21:53
*** k_mouza has quit IRC21:57
*** zzzeek has quit IRC22:34
*** zzzeek has joined #openstack-kolla22:34
*** ysirndjuro has quit IRC23:24
*** suff has quit IRC23:48

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!