Friday, 2020-05-08

JamesBen_@sean-k-mooney, I've never been able to get ironic working.  Any help explaining that mess would be awesome.  I'm hoping in the next few months to redo our prod from ocata to latest :-). Would love to enable ironic with it00:31
*** suff has quit IRC00:31
*** suff has joined #openstack-kolla00:39
sean-k-mooneythe main things i change were 1 dont use ipxe due to the networking issue i hit (in the past i always did) 2 put your api network and ipmi network, ironic inspector network all on the same vlan. 3 create a neuton provider netork on the same vlan as the api/ipmi/inspection network and use that as the cleaning/provisioning network.00:48
sean-k-mooneyi am using the ironic flat networking mode which is what kolla enables by default00:49
sean-k-mooneyJamesBen_: im not strictly sure that putting the api network on the same vlan as ipmi is required however ipxe does not support static routing apprently00:51
sean-k-mooneythe requireemnt is really that the ironic python agent neets to be able to connect to the api so it needs to be routeable and the server needs to be able to pxe boot the ipa image00:53
*** xinliang has joined #openstack-kolla01:03
*** suff has quit IRC01:09
*** seco has quit IRC01:15
*** suff has joined #openstack-kolla01:17
*** seco has joined #openstack-kolla01:46
*** seco has quit IRC01:51
*** weshay|ruck is now known as weshay_pto01:52
*** xinliang has quit IRC02:04
*** seco has joined #openstack-kolla02:25
*** seco has quit IRC02:29
*** seco has joined #openstack-kolla02:42
*** ysirndjuro has quit IRC02:44
*** ysirndjuro has joined #openstack-kolla02:45
*** seco has quit IRC02:47
openstackgerritJames Kirsch proposed openstack/kolla-ansible master: Add support for encrypting Barbican API  https://review.opendev.org/72625802:52
*** xclan has quit IRC03:05
*** wuchunyang has joined #openstack-kolla03:08
*** wuchunyang has quit IRC03:22
*** ysirndjuro has quit IRC03:26
*** ricolin has quit IRC03:41
*** ykarel|away is now known as ykare04:18
*** schwicht has quit IRC04:21
*** schwicht has joined #openstack-kolla04:24
*** schwicht has quit IRC04:29
*** evrardjp has quit IRC04:36
*** evrardjp has joined #openstack-kolla04:36
*** hamzy__ has joined #openstack-kolla04:38
*** hamzy_ has quit IRC04:41
*** seco has joined #openstack-kolla04:43
*** seco has quit IRC04:48
*** skramaja has joined #openstack-kolla05:06
*** wuchunyang has joined #openstack-kolla05:06
*** schwicht has joined #openstack-kolla05:17
*** ricolin has joined #openstack-kolla05:39
*** srin has quit IRC05:41
*** cah_link has joined #openstack-kolla05:48
*** srin has joined #openstack-kolla05:54
*** srinn has joined #openstack-kolla05:55
*** srin has quit IRC05:59
*** wuchunyang has quit IRC06:15
*** schwicht has quit IRC06:18
openstackgerritFabian Zimmermann proposed openstack/kolla-ansible master: multipath requires udev-rules in host  https://review.opendev.org/72628306:20
openstackgerritFabian Zimmermann proposed openstack/kolla-ansible master: multipath requires udev-rules in host  https://review.opendev.org/72628306:33
*** seco has joined #openstack-kolla06:44
openstackgerritXiaojueGuan proposed openstack/kolla-ansible master: Fix failure of change default index pattern  https://review.opendev.org/72628906:44
*** seco has quit IRC06:48
*** suff has quit IRC06:57
*** e0ne has joined #openstack-kolla06:57
*** e0ne has quit IRC06:58
yoctozeptosean-k-mooney, JamesBen_: I think there is only the requirement of routability towards the image source07:01
openstackgerritXiaojueGuan proposed openstack/kolla-ansible master: Fix failure of change default index pattern  https://review.opendev.org/72628907:06
*** suff has joined #openstack-kolla07:08
yoctozeptoheadphoneJames: sleepless nights? :-) are you running in wishbox mode? you are quick :D07:13
*** suff has quit IRC07:15
*** seco has joined #openstack-kolla07:16
*** suff has joined #openstack-kolla07:17
*** ysirndjuro has joined #openstack-kolla07:18
*** cah_link has quit IRC07:29
mnasiadkamorning07:34
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Barbican scenario  https://review.opendev.org/71903707:38
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Barbican scenario  https://review.opendev.org/71903707:39
hrwmorning07:42
*** eliaswimmer__ has left #openstack-kolla07:55
*** eliaswimmer has joined #openstack-kolla07:55
*** nathharp_ has joined #openstack-kolla08:02
*** nathharp has quit IRC08:04
*** JangwonLee_ has joined #openstack-kolla08:06
*** cah_link has joined #openstack-kolla08:07
*** JangwonLee__ has quit IRC08:10
eliaswimmermorning08:11
eliaswimmerCan I choose the vm image for the seed vm in kayobe?08:11
hrweliaswimmer: which one is used now?08:14
eliaswimmerhrw: centos708:14
eliaswimmerhrw: I think it's not really important as it must not really be persistent as far as I understand08:16
*** e0ne has joined #openstack-kolla08:16
*** e0ne has quit IRC08:17
*** e0ne has joined #openstack-kolla08:40
*** livelace has joined #openstack-kolla08:50
*** seco has quit IRC09:00
*** suff has quit IRC09:01
*** suff has joined #openstack-kolla09:02
*** ykare is now known as ykarel09:03
*** livelace has quit IRC09:11
*** kevinz has joined #openstack-kolla09:11
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Barbican scenario  https://review.opendev.org/71903709:30
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Barbican scenario  https://review.opendev.org/71903709:33
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Barbican scenario  https://review.opendev.org/71903709:34
hrweliaswimmer: and you need something to test does it work09:36
*** threestrands has quit IRC09:41
*** ricolin_ has joined #openstack-kolla09:44
openstackgerritXiaojueGuan proposed openstack/kolla-ansible master: Fix: make mariadb could be deployed  https://review.opendev.org/72632409:47
*** seco has joined #openstack-kolla09:55
*** seco has quit IRC09:59
*** wuchunyang has joined #openstack-kolla10:04
*** Fl1nt has joined #openstack-kolla10:05
Fl1ntHi everybody!10:05
*** Fl1nt has quit IRC10:11
*** wuchunyang has quit IRC10:12
mgoddardmorning10:19
mgoddardIt's a public holiday today in the UK, so I'm out10:19
mgoddardjust catching up on IRC logs10:19
mgoddardif there's anything urgent, ping me10:19
mgoddardit's been a bit of a busy week...10:20
*** ricolin_ has quit IRC10:21
*** ricolin_ has joined #openstack-kolla10:22
*** ykarel is now known as ykarel|lunch10:23
*** ricolin_ has quit IRC10:31
*** seco has joined #openstack-kolla10:31
*** wuchunyang has joined #openstack-kolla10:41
mgoddardsean-k-mooney: just catching up on IRC10:42
mgoddardsean-k-mooney: octavia issue is known10:43
mgoddardI recently went through it, was quite painful to work out10:43
mgoddardI think a simple checklist in docs of requirements would be a good start10:43
*** wuchunyang has quit IRC10:43
mgoddardend goal is full automation10:43
*** wuchunyang has joined #openstack-kolla10:44
mgoddardwe only just merged the support for dual CA, so it was broken in train & master10:44
mgoddardhaven't hit the ipxe issue yet, although I have used ipxe on bifrost + centos 8 so it's not fundamentally broken10:46
mgoddardeliaswimmer: check out seed-vm.yml for the VM image URL10:51
*** ykarel|lunch is now known as ykarel11:08
*** gfidente|afk is now known as gfidente11:09
*** wuchunyang has quit IRC11:19
*** wuchunyang has joined #openstack-kolla11:19
*** livelace has joined #openstack-kolla11:32
*** k_mouza has joined #openstack-kolla11:39
*** k_mouza has quit IRC12:13
*** ysirndjuro has quit IRC12:17
*** jonaspaulo has joined #openstack-kolla12:18
sean-k-mooneymgoddard: well in my case at least the ipxe image was not able to transmit packets regradelsss of if the destination was in the same subnet or not so it seams like its pretty broken but i dont know. it could be hardware related too. e.g. the specific driver form my nic12:19
*** wuchunyang has quit IRC12:22
*** k_mouza has joined #openstack-kolla12:22
*** skramaja has quit IRC12:27
*** livelace has quit IRC12:55
*** livelace has joined #openstack-kolla13:06
*** TrevorV has joined #openstack-kolla13:21
*** ykarel is now known as ykarel|afk13:23
*** ysirndjuro has joined #openstack-kolla13:41
openstackgerritRadosław Piliszek proposed openstack/kolla-ansible master: [DNM] Barbican scenario  https://review.opendev.org/71903714:04
*** k_mouza has quit IRC14:09
*** k_mouza has joined #openstack-kolla14:10
*** k_mouza has quit IRC14:24
noxoidim going through the ironic (not bifrost/kayobe) setup for the first time and i hit the same quirk that sean-k-mooney mentioned, where the nodes require access to the api network. this seems reasonable but as of stein it looks like that network by default is also the one that libvirt listens to. and since the libvirt api isnt authenticated that14:31
noxoidseems like a security problem. was there a conscious decision to use api_interface specifically in ironic templates? for example dnsmasq: https://opendev.org/openstack/kolla-ansible/src/branch/stable/stein/ansible/roles/ironic/templates/ironic-dnsmasq.conf.j2#L714:31
noxoidit seems to go against a lot of other roles in k-a that allow you to customize the interface of every service, something like ironic_provisioning_interface14:33
noxoidjust want to make sure im not missing something obvious from a design point of view14:33
*** ykarel|afk is now known as ykarel14:33
sean-k-mooneynoxoid: the api network and your loadblancer dont need to be the same14:40
sean-k-mooneyi have my public vip address on a differnt netwrok to my api netwrok14:40
sean-k-mooneybut ya in ooo they have locked it down more14:40
sean-k-mooneywe dont really need tcp access to libvirt14:40
sean-k-mooneyat least beyond localhost. nova default to a unix socket for connecting14:41
noxoidyea but i thought libvirt has to listen on the network for migration requests14:41
noxoidand yea i have my api network separate, i have an internet-facing api network and an internal lan-only one14:42
sean-k-mooneyi dont belive so14:42
noxoidheavily implies it does https://opendev.org/openstack/kolla-ansible/src/branch/stable/stein/ansible/roles/nova/templates/libvirtd.conf.j2#L714:42
sean-k-mooney ooo run a seperate migration container with ssh for that usecase14:43
sean-k-mooneynoxoid: so it  looke liek we can use different networks now https://opendev.org/openstack/kolla-ansible/commit/864e589803827830bcf4afe30b3fc789c2c5846f14:44
sean-k-mooneynoxoid: but ya if your using ipxe https://github.com/ipxe/ipxe/pull/104 prevents static host routes form working14:46
noxoidfun, but yea im sticking with regular PXE for now to simplify things14:46
sean-k-mooneyyep it seams to just work14:46
sean-k-mooneythat said i know some older hardware had buggy pxe implemantion which was my main reason for using ipxe14:47
noxoidguess i need to figure out how to safely give bare metal nodes access to the api network14:49
noxoidmaybe ill just trunk a new vlan and call that my migration interface14:51
noxoidinstead of re-using api_interface14:51
noxoidthanks for being a sounding board sean-k-mooney. if anyone else has thoughts please let me know14:53
*** k_mouza has joined #openstack-kolla15:05
sean-k-mooneynoxoid: for what it worth since my case is just my home lab i use for upstream dev i dont really have to worry about it but yes i would likely trnk another vlan and use that as my migration interface if i did not trust the workloads15:09
sean-k-mooneythat is what im already doing for the api_interface15:10
sean-k-mooneyits on vlan 3. where as my public netwrok is the flat network and  i have configured my tor to add each of the bmc/pxe interface on to vlan 3 as there native vlan15:11
sean-k-mooneyso if i wanted to make it more secure i woudl jsut add vlan 4 and use that for the migration netwrok to prevent my ironic nodes form acessing it15:11
noxoidfair. yea i prefer to have a different vlan for everything to avoid having to craft picky firewall rules that allow certain controller/compute nodes but not others depending on which service is running15:13
*** seco has quit IRC15:13
noxoidbut maybe a lot of my security concerns will go away if i can get ironic/neutron integrated with our arista ToR switches15:14
noxoidso it would only be attached to the api_network vlan during cleaning/deploy and then neutron will change the port to the internet-facing vlan for workloads15:15
noxoidor whatever vlan the workload requires15:15
*** seco has joined #openstack-kolla15:22
*** seco has quit IRC15:26
*** mcnugit has joined #openstack-kolla15:35
*** wuchunyang has joined #openstack-kolla15:37
*** k_mouza has quit IRC15:46
*** k_mouza has joined #openstack-kolla15:50
*** k_mouza has quit IRC15:51
*** ykarel is now known as ykarel|away15:57
*** hjensas is now known as hjensas|afk15:59
*** jbadiapa has joined #openstack-kolla16:04
*** suff has quit IRC16:16
*** suff has joined #openstack-kolla16:17
*** TrevorV has quit IRC16:32
*** evrardjp has quit IRC16:36
*** evrardjp has joined #openstack-kolla16:36
*** wuchunyang has quit IRC16:41
*** hamzy__ has quit IRC16:43
*** hamzy has joined #openstack-kolla16:47
*** cah_link has quit IRC16:54
*** cah_link has joined #openstack-kolla17:04
*** mcnugit has quit IRC17:22
yoctozeptosean-k-mooney: hi, I know it's a bit off-topic but I'm interested into yours insight about nova-lxd and why it did not succeed, I am not entirely persuaded zun is a 1:1 replacement for it (or close to that)17:27
*** mcnugit has joined #openstack-kolla17:27
*** mcnugit has quit IRC17:29
sean-k-mooneyyoctozepto: nova-lxd did not succeed partly because it was out of tree17:39
sean-k-mooneyif it was merged into the main nova tree and propely maintianed it would have been a very nice addtion17:40
sean-k-mooneyi like libivrt/lxc too but its also not really wroking at the moment17:40
sean-k-mooneyin terms of zun they are more focused on docker/cri compatiable contaienr then system containers17:41
sean-k-mooneyim not sure if zun could be used as a replacemnet as a result. i think zun were planning to mainatin a nova compatiabel driver. basicaly a port of the nova-docker diriver17:41
sean-k-mooneybut i dont know what teh current status is.17:42
sean-k-mooneyi would very much like there to be a nova system contaienr driver that worked and was used17:42
sean-k-mooneythe libvirt/lxc one is proably the closes that is still there today but it has bugs17:43
sean-k-mooneyyoctozepto: im honestly not familar enough with zun to say if ti can be used as a replamcent but given its focous on app conatiner instead o fsystem contaienr proably not but if you find out more let me know17:50
*** k_mouza has joined #openstack-kolla17:51
*** k_mouza has quit IRC17:56
yoctozeptosean-k-mooney: yeah, my issue is mostly about focus on app containers while I am looking for system containers; also it lacks in terms of configurability levels of nova, the scheduler cannot act on PCI devices etc.17:59
sean-k-mooneyyoctozepto: they have a pci manager as far as i can see18:00
sean-k-mooneyso i think they have some pci support18:00
sean-k-mooneyyou should talk to them and see18:00
sean-k-mooneythat or we should fix the nova libvirt/lxc backend18:01
yoctozeptosean-k-mooney: I would preferably want lxc in nova but yeah, I'm investigating what zun can do for me :-)18:01
sean-k-mooneyyoctozepto: it kind of works today in limited usecases18:07
*** seco has joined #openstack-kolla18:08
sean-k-mooneynova does not resize the continer filesystem to match the flavor and i dont think some thing like cinder volumes work properly18:08
sean-k-mooneywe got cloud init fixed a few months ago https://review.opendev.org/#/c/667976/18:09
patchbotpatch 667976 - nova - Add support for cloud-init on LXC instances (MERGED) - 4 patch sets18:09
*** suff has quit IRC18:10
sean-k-mooneyand fixed the python 3 issues https://review.opendev.org/#/c/676263/ and mount issue https://review.opendev.org/#/c/659780/18:11
patchbotpatch 676263 - nova - lxc: make use of filter python3 compatible (MERGED) - 4 patch sets18:11
patchbotpatch 659780 - nova - Fix type error on call to mount device (MERGED) - 4 patch sets18:11
sean-k-mooneybut its still not tested in the ci https://review.opendev.org/#/c/676024/ and its not really being maintained altohg that is partly due to lack of people working on it18:12
patchbotpatch 676024 - nova - Add nova-lxc job to the experimental queue (ABANDONED) - 5 patch sets18:12
*** seco has quit IRC18:12
sean-k-mooneyyoctozepto: as far as i am aware nova libvirt lxc should work but unless your willing to be the one maintaineing it im not suer peopel should use it.18:13
yoctozeptosean-k-mooney: thanks, I was fixed at nova-lxd and saw it discontinued, and did not recognize lxc was a separate one :D18:23
yoctozeptosean-k-mooney: I'll have a look at nova lxc then18:24
sean-k-mooneyyes its seperate and predates it18:24
sean-k-mooneycanonical belived that adding a lxd deamon instead of using libvirt had some advantages and created nova-lxd to expoloer those18:25
*** gfidente is now known as gfidente|afk18:25
sean-k-mooneythe main one was the could integrete lxd with juju as a provide and orcestrate lxd contiaenr as via the same charmes that used baremetal18:25
sean-k-mooney*manged18:25
sean-k-mooneybut libvirt has had the ablity to mange lxc contaiern for many many years and it was used in proxmox for a long time as there contaienr solution18:26
sean-k-mooneyproxmox used to use openvz before the change to lxc in proxmox5 i belive18:27
sean-k-mooneyin anycase the libvirt/lxc support is still in nova so you should be able to use it with kolla today18:28
yoctozeptoI had the (mis)fortune to use openvz at some point in time as a customer, boy was it misconfigured ;o18:28
sean-k-mooneyall yo need to do is set virt_type=lxc in the libvirt section of the nova.conf18:28
yoctozeptosean-k-mooney: yeah, I think I'll tinker with it, presumably it will fail at some part18:28
sean-k-mooneyyou might need to also install the lxc tools into the nova libvirt contianer not sure about that18:29
yoctozeptoto check18:29
sean-k-mooneyit should be easy enough to do with a template override18:29
sean-k-mooneyi have only used nova with libvirt/lxc with devstack so not sure what else is needed in kolla18:30
sean-k-mooneyi would proably try it with devstack first and if you like it then port support to kolla18:30
sean-k-mooneyubuntu 16.04 and 18.04 i belvie both have the lxc tools preinstalled18:31
sean-k-mooneyso with devstack you just the config option in nova and it basically worked without much else to do18:31
yoctozeptosean-k-mooney: thanks, I'll do it this way then, makes sense as I never used it (nor libvirt/lxc, only lxd)18:49
*** k_mouza has joined #openstack-kolla18:52
*** k_mouza has quit IRC18:57
sean-k-mooneyyoctozepto: by the way im sure you know this already but tre are prebuild lxc images for mulple operating systems here https://us.images.linuxcontainers.org/images/18:59
*** suff has joined #openstack-kolla19:03
openstackgerritMerged openstack/kolla master: Use Python 3 for mod_wsgi in cyborg and monasca on Debian/Ubuntu  https://review.opendev.org/72522119:07
openstackgerritRadosław Piliszek proposed openstack/kolla stable/train: Use Python 3 for mod_wsgi in cyborg and monasca on Debian/Ubuntu  https://review.opendev.org/72646019:13
*** suff has quit IRC19:14
*** suff has joined #openstack-kolla19:15
*** ysirndjuro has quit IRC19:36
adebergDo you think there's any way to use kolla-ansible to deploy 2 mariadb on the same machines, 1 dedicated to keystone and 1 for the rest ? I feel it's not thought for that, but i'd like to leverage the kolla-ansible tooling.... :( So far i'm looking at having a separate repo for the keystone DB stuff and patching on the fly to avoid container names19:41
adebergclash but i'd love to avoid doing that....19:41
noxoidadeberg: im not aware of a way to do that with k-a but ive also thought that would be useful. specifically a multi-region galera for keystone (AT&T did it in the past)19:45
adebergthis is exactly my use case19:45
yoctozeptoadeberg, noxoid: the work is pending but halted atm https://review.opendev.org/61975619:52
patchbotpatch 619756 - kolla-ansible - Minimal support for deploying multiple instances o... - 19 patch sets19:52
noxoidah yea i think ive seen that before, thanks19:52
adebergI totally missed it !19:53
*** benj_ has quit IRC19:55
*** benj_ has joined #openstack-kolla19:56
*** _Cyclone_ has quit IRC19:56
*** _Cyclone_ has joined #openstack-kolla20:07
*** e0ne has quit IRC20:34
*** suff has quit IRC20:34
*** _Cyclone_ has quit IRC20:43
*** _Cyclone_ has joined #openstack-kolla20:53
*** KeithMnemonic has quit IRC21:01
*** mcnugit has joined #openstack-kolla21:18
*** dking has quit IRC21:19
adebergmgoddard that patch yoctozepto mentioned ^-- I'm no mariadb role expert but, was it missing lot of thing back in times ? Now that patch would had to be reviewed with latest changes in master but i'm trying to have an idea of the effort21:23
adebergI'll probably have a look next week as I badly need that21:24
*** eliaswimmer has quit IRC21:25
*** jcmdln has quit IRC21:47
*** jcmdln has joined #openstack-kolla21:49
*** riuzen has joined #openstack-kolla21:57
*** bornie2bake has joined #openstack-kolla22:03
*** cah_link1 has joined #openstack-kolla22:06
*** cah_link has quit IRC22:08
*** cah_link1 is now known as cah_link22:08
*** seco has joined #openstack-kolla22:08
*** seco has quit IRC22:13
*** riuzen has quit IRC22:20
*** dking has joined #openstack-kolla22:25
*** bornie2bake has quit IRC22:37
*** livelace has quit IRC22:38
*** k_mouza has joined #openstack-kolla22:53
*** k_mouza has quit IRC22:57
*** jonaspaulo has quit IRC22:58
*** seco has joined #openstack-kolla23:17
*** seco has quit IRC23:21
*** sorin-mihai has joined #openstack-kolla23:33

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!