| *** vishalmanchanda has quit IRC | 00:18 | |
| *** xinliang has joined #openstack-kolla | 00:57 | |
| *** wuchunyang has joined #openstack-kolla | 01:24 | |
| *** wuchunyang has quit IRC | 01:29 | |
| *** wuchunyang has joined #openstack-kolla | 01:47 | |
| *** wuchunyang has quit IRC | 01:54 | |
| *** wuchunyang has joined #openstack-kolla | 02:02 | |
| *** wuchunyang has quit IRC | 02:02 | |
| *** wuchunyang has joined #openstack-kolla | 02:33 | |
| *** xinliang has quit IRC | 03:41 | |
| *** wuchunyang has quit IRC | 04:04 | |
| *** skramaja has joined #openstack-kolla | 04:05 | |
| *** wuchunyang has joined #openstack-kolla | 04:07 | |
| *** k_mouza has joined #openstack-kolla | 04:12 | |
| *** wuchunyang has quit IRC | 04:14 | |
| *** k_mouza has quit IRC | 04:16 | |
| *** ykarel|away is now known as ykarel | 04:21 | |
| *** evrardjp has quit IRC | 04:36 | |
| *** evrardjp has joined #openstack-kolla | 04:36 | |
| *** wuchunyang has joined #openstack-kolla | 04:52 | |
| *** wuchunyang has quit IRC | 04:54 | |
| *** mcnugit has joined #openstack-kolla | 05:00 | |
| openstackgerrit | XiaojueGuan proposed openstack/kolla-ansible master: Fix bug 1875812 https://review.opendev.org/724460 | 05:08 |
|---|---|---|
| openstack | bug 1875812 in kolla-ansible ussuri "invalid_index_name_exception" [Medium,In progress] https://launchpad.net/bugs/1875812 - Assigned to XiaojueGuan (xiaojuegaun) | 05:08 |
| *** vishalmanchanda has joined #openstack-kolla | 05:45 | |
| *** seco_ has joined #openstack-kolla | 05:47 | |
| *** JangwonLee_ has joined #openstack-kolla | 05:50 | |
| *** JangwonLee__ has joined #openstack-kolla | 05:53 | |
| *** JangwonLee has quit IRC | 05:53 | |
| *** JangwonLee_ has quit IRC | 05:56 | |
| *** abdysn has joined #openstack-kolla | 05:56 | |
| *** cah_link has joined #openstack-kolla | 06:10 | |
| *** nikparasyr has joined #openstack-kolla | 06:14 | |
| openstackgerrit | XiaojueGuan proposed openstack/kolla-ansible master: Fix bug 1875812 https://review.opendev.org/724460 | 07:09 |
| openstack | bug 1875812 in kolla-ansible ussuri "invalid_index_name_exception" [Medium,In progress] https://launchpad.net/bugs/1875812 - Assigned to XiaojueGuan (xiaojuegaun) | 07:09 |
| hrw | morning | 07:10 |
| *** nathharp_ has quit IRC | 07:11 | |
| *** nathharp has joined #openstack-kolla | 07:12 | |
| openstackgerrit | XiaojueGuan proposed openstack/kolla-ansible master: Fix Self-Signed Certificates failed https://review.opendev.org/724217 | 07:16 |
| *** bengates has joined #openstack-kolla | 07:19 | |
| openstackgerrit | XiaojueGuan proposed openstack/kolla-ansible master: Fix bug 1875812 https://review.opendev.org/724460 | 07:21 |
| openstack | bug 1875812 in kolla-ansible ussuri "invalid_index_name_exception" [Medium,In progress] https://launchpad.net/bugs/1875812 - Assigned to XiaojueGuan (xiaojuegaun) | 07:21 |
| openstackgerrit | XiaojueGuan proposed openstack/kolla-ansible master: Fix bug 1875812 https://review.opendev.org/724460 | 07:24 |
| openstack | bug 1875812 in kolla-ansible ussuri "invalid_index_name_exception" [Medium,In progress] https://launchpad.net/bugs/1875812 - Assigned to XiaojueGuan (xiaojuegaun) | 07:24 |
| *** jaicaa has quit IRC | 07:31 | |
| *** jaicaa has joined #openstack-kolla | 07:33 | |
| *** xinliang has joined #openstack-kolla | 07:35 | |
| *** born2bake has joined #openstack-kolla | 07:44 | |
| *** rpittau|afk is now known as rpittau | 07:47 | |
| *** mcnugit has quit IRC | 07:54 | |
| *** born2bake has quit IRC | 07:54 | |
| *** TheJulia has quit IRC | 07:56 | |
| *** mnasiadka has quit IRC | 07:56 | |
| *** TheJulia has joined #openstack-kolla | 08:01 | |
| *** mnasiadka has joined #openstack-kolla | 08:01 | |
| kevko | Hi guys, we encounter some problems in openvswitch due link saturation, we consider the possibility of throwing it from the container directly into the host system, which could bring better use of performance. It would be a non-default option (by default OVS would still run in the container). Communication between OVS and OpenStack would take place either via a tunneled socket or via tcp (if I'm not mistaken, tcp is used | 08:11 |
| kevko | now). What do u think, is this acceptable for you? Can I write some blueprint for this ? | 08:11 |
| yoctozepto | kevko: I doubt using container for ovs is impacting the performance in any observable way but interested about the results | 08:26 |
| *** maliblatt has joined #openstack-kolla | 08:26 | |
| maliblatt | hi folks! is there anyone with experience in using bgp routing in kolla openstack? since days we try to get our setup running but always stuck at the same point. bgp session to the provider core routers are stablished and floating ip host routes are getting announced. but the floating ips are not reachable from outside. we are using a transfer network between the upstream routers and our openstack env. the transfer | 08:27 |
| maliblatt | network is created as the openstack routers external network so that the next-hop ist getting announced from within that network. for the bgp session one adress from that tarnsfer network is configured on br-ex so that the neutron-bgp-dragent can each the upstream routers with an source address from the transfer network. | 08:27 |
| maliblatt | packets are arriving on the external interface on the host but not getting forwarded to the qrouter namespace | 08:29 |
| kevko | yoctozepto: i just wanted to know if patch can be merged from your side if this option will be non-default configurable , and if I can create blueprint and implement solution | 08:36 |
| yoctozepto | kevko: I would like to avoid that but I am eager to see if it really impacted the performance ;-) | 08:37 |
| *** suff has joined #openstack-kolla | 08:37 | |
| yoctozepto | maliblatt: nothing in l3 agent logs? maybe it failed to wire the addresses up, are they assigned in the namespaces? did floating work previously? is the reverse routing having the same path? | 08:39 |
| suff | Hi guys! I cant join to Kolla SIG today, got a lot of working meeting. Sorry. I try to explain ops story next time. | 08:41 |
| maliblatt | yoctozepto: in router namespace i see the routers own address from transfer network and on the same interface the floating ip. router's ip in transfer network is reachable from outside | 08:42 |
| maliblatt | yoctozepto: but packets to the floating address are not reaching the router namespace | 08:42 |
| maliblatt | yoctozepto: l3 logs are not showing anything special | 08:43 |
| maliblatt | befor implementing the bgp stuff, floating ips were working :) | 08:43 |
| *** eliaswimmer has joined #openstack-kolla | 08:50 | |
| yoctozepto | suff: sorry to hear that, but feel welcome to join us at any time :-) | 08:52 |
| yoctozepto | maliblatt: how are you sure they are not reaching the floating ip? do you inspect some counters or just claim based on no traffic to vm? | 08:53 |
| *** livelace has joined #openstack-kolla | 08:53 | |
| maliblatt | yoctozepto: tcpdump inside the namespace does not show the packets | 08:58 |
| *** ykarel is now known as ykarel|lunch | 09:01 | |
| mnasiadka | good morning | 09:03 |
| hrw | morning | 09:03 |
| *** k_mouza has joined #openstack-kolla | 09:07 | |
| yoctozepto | maliblatt: weird, is that interface properly wired? I guess it is since you said the other ip address (router's) works | 09:09 |
| *** livelace has quit IRC | 09:16 | |
| *** born2bake has joined #openstack-kolla | 09:28 | |
| *** irclogbot_3 has quit IRC | 09:41 | |
| *** irclogbot_0 has joined #openstack-kolla | 09:42 | |
| *** ykarel|lunch is now known as ykarel | 09:44 | |
| maliblatt | yoctozepto: may be we found an issue. the upstream juniper routers seems to use the wrong MAC for the next-hop. using the mac from the dragent not from the virtual router interface | 09:50 |
| maliblatt | yoctozepto: we investigate and i let you know | 09:50 |
| *** xinliang has quit IRC | 10:00 | |
| maliblatt | yoctozepto: okay, we've got it. the wrong policy was used on juniper router side... thanks for your efford! | 10:04 |
| maliblatt | yoctozepto: fyi: set policy-options policy-statement policy-customer-in term nexthop then next-hop peer-address | 10:05 |
| yoctozepto | maliblatt: you are welcome and thanks for following up :-) | 10:07 |
| yoctozepto | mgoddard: maybe some mail about klub? | 10:08 |
| *** rpittau is now known as rpittau|bbl | 10:10 | |
| mgoddard | yoctozepto: yeah will do | 10:10 |
| yoctozepto | mgoddard: thanks, I was afraid you want to cede the chairing of klub too :p | 10:11 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Add extend_lists option to merge_yaml https://review.opendev.org/685671 | 10:11 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Support customizing prometheus.cfg files https://review.opendev.org/686205 | 10:11 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Add extras directory to prometheus config https://review.opendev.org/688374 | 10:11 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Stop mocking ansible modules globally https://review.opendev.org/726081 | 10:11 |
| *** gfidente|afk is now known as gfidente | 10:12 | |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Stop mocking ansible modules globally https://review.opendev.org/726081 | 10:15 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Add extend_lists option to merge_yaml https://review.opendev.org/685671 | 10:15 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Support customizing prometheus.cfg files https://review.opendev.org/686205 | 10:15 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Add extras directory to prometheus config https://review.opendev.org/688374 | 10:15 |
| *** skramaja has quit IRC | 10:16 | |
| *** skramaja has joined #openstack-kolla | 10:16 | |
| *** maliblatt has quit IRC | 10:18 | |
| *** born2bake has quit IRC | 10:19 | |
| *** bersace has joined #openstack-kolla | 10:31 | |
| *** k_mouza has quit IRC | 10:35 | |
| *** cah_link has quit IRC | 10:38 | |
| *** k_mouza has joined #openstack-kolla | 10:47 | |
| yoctozepto | mgoddard: actually Dmitry (suff) cancelled its presentation today :-( | 10:58 |
| *** k_mouza has quit IRC | 10:58 | |
| mgoddard | yoctozepto: yeah I saw | 10:58 |
| yoctozepto | his* oopsie | 10:58 |
| yoctozepto | mgoddard: ml post did not see it though | 10:59 |
| mgoddard | thankfully Fl1nt has 3 | 10:59 |
| yoctozepto | that true | 10:59 |
| *** k_mouza has joined #openstack-kolla | 11:04 | |
| *** k_mouza has quit IRC | 11:05 | |
| *** k_mouza has joined #openstack-kolla | 11:05 | |
| stingrayza | anybody got a minute to eyeball https://review.opendev.org/#/c/721241/ (and hopefully +1 workflow ;) )? | 11:10 |
| patchbot | patch 721241 - kolla-ansible - Update Advanced Config guide to clarify paths - 1 patch set | 11:10 |
| *** cah_link has joined #openstack-kolla | 11:24 | |
| *** skramaja has quit IRC | 11:27 | |
| *** skramaja has joined #openstack-kolla | 11:27 | |
| mnasiadka | rolling | 11:29 |
| *** livelace has joined #openstack-kolla | 11:38 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Update Advanced Config guide to clarify paths https://review.opendev.org/721241 | 11:41 |
| hrw | does anyone here has any time for kolla? | 11:46 |
| openstackgerrit | Marcin Juszkiewicz proposed openstack/kolla master: CentOS: disable more repos https://review.opendev.org/725032 | 11:47 |
| mnasiadka | hrw: you? :) | 12:04 |
| hrw | it's just rebase | 12:06 |
| hrw | faster then typing recheck and also makes sure that patch is uptodate | 12:06 |
| *** rpittau|bbl is now known as rpittau | 12:11 | |
| *** abdysn has quit IRC | 12:16 | |
| mnasiadka | hrw: I was answering to your question - not asking about the bot notification :) | 12:28 |
| *** seco_ has quit IRC | 12:28 | |
| hrw | mnasiadka: it was rethorical question | 12:29 |
| *** seco has joined #openstack-kolla | 12:30 | |
| *** srin has joined #openstack-kolla | 12:42 | |
| *** srinn has quit IRC | 12:43 | |
| *** seco has quit IRC | 12:45 | |
| *** seco has joined #openstack-kolla | 12:46 | |
| *** cah_link has quit IRC | 12:49 | |
| stingrayza | thanks mnasiadka ! :) | 12:49 |
| *** skramaja has quit IRC | 12:50 | |
| *** cah_link has joined #openstack-kolla | 12:56 | |
| *** cah_link has quit IRC | 13:00 | |
| *** ykarel is now known as ykarel|afk | 13:05 | |
| *** Fl1nt has joined #openstack-kolla | 13:12 | |
| Fl1nt | Hi everyone! | 13:12 |
| seco | hi o/ | 13:16 |
| *** TrevorV has joined #openstack-kolla | 13:32 | |
| *** ysirndjuro has joined #openstack-kolla | 14:05 | |
| *** ykarel|afk is now known as ykarel | 14:07 | |
| *** xclan has joined #openstack-kolla | 14:34 | |
| *** wuchunyang has joined #openstack-kolla | 14:36 | |
| osmanlicilegi | unfortunately I will skip today's klub meeting. timezone is sometimes a big issue for me... anyways, Fl1nt: your presentation looks great. good luck :] | 14:40 |
| yoctozepto | osmanlicilegi: man, you are just one hour off of us, don't blame the timezone :D | 14:41 |
| yoctozepto | hrw: nope, look in a different channel :-) | 14:42 |
| osmanlicilegi | yoctozepto: the problem is it will be 18 pm here 20 minutes later :) I wish it was 17 so I could be wife-single :] | 14:43 |
| osmanlicilegi | when the work is over, I have to help housework today. | 14:44 |
| yoctozepto | osmanlicilegi: ok, understood (or not quite, wife-free here) | 14:44 |
| yoctozepto | (but wifi-full :-) ) | 14:44 |
| osmanlicilegi | :] I hope one day Turkey will return back to gmt+2... | 14:48 |
| yoctozepto | osmanlicilegi: yeah, that would make sense for the western parts at least | 14:50 |
| *** vishalmanchanda has quit IRC | 14:54 | |
| *** dking has joined #openstack-kolla | 14:55 | |
| *** wuchunyang has quit IRC | 14:58 | |
| yoctozepto | kolla klub meeting in 2 minutes https://meet.google.com/hph-pynx-vsy | 14:59 |
| Fl1nt | \o/ | 14:59 |
| yoctozepto | mgoddard: kolla klub in 1 minute, link above | 14:59 |
| mgoddard | +1 | 14:59 |
| Fl1nt | @osmanlicilegi, thanks a lot, sorry for the news. | 15:00 |
| *** wuchunyang has joined #openstack-kolla | 15:00 | |
| *** mcnugit has joined #openstack-kolla | 15:11 | |
| *** nikparasyr has quit IRC | 15:15 | |
| *** livelace has quit IRC | 15:26 | |
| *** livelace has joined #openstack-kolla | 15:34 | |
| *** wuchunyang has quit IRC | 15:35 | |
| *** priteau has joined #openstack-kolla | 15:45 | |
| *** eliaswimmer_ has joined #openstack-kolla | 15:49 | |
| *** eliaswimmer has quit IRC | 15:52 | |
| yoctozepto | another successful meeting, thanks Fl1nt for the presentation | 16:03 |
| yoctozepto | btw, that last voice was me | 16:04 |
| Fl1nt | Thanks to you guys, and feel free to gives feedback about the slides, so I could improve them and make a better emphase around maybe more specific or technical questions. | 16:04 |
| Fl1nt | Ah yeah nice of you for the hard work on ceph so ^^ | 16:04 |
| priteau | Thanks Fl1nt for the presentation. I still don't really understand your Ceph client issues, would be good to have some details in a bug report | 16:06 |
| Fl1nt | @priteau, it's not a CEPH issue, it's just that nova at some point decided to switch from libvirt doing the volume job directly to actually use cinder's call ^^ | 16:07 |
| dking | Fl1nt: Thank you for the presentation. | 16:07 |
| *** ykarel is now known as ykarel|away | 16:07 | |
| Fl1nt | So if I should make a quick resume of why companies that I worked with were looking for reproducability, life cycle, easy to use, easy to understand, easy to deploy ^^ | 16:07 |
| *** seco has quit IRC | 16:08 | |
| Fl1nt | + were looking for kolla -^ | 16:08 |
| Fl1nt | @dking, thanks to you | 16:08 |
| priteau | Fl1nt: I am just wondering when the change happened and why I haven't seen the issues in some of the deployments we support. I think in a previous discussion you mentioned using different caps or different auth key for nova than the one I would expect. | 16:11 |
| openstackgerrit | Radosław Piliszek proposed openstack/kolla-ansible stable/train: Update Advanced Config guide to clarify paths https://review.opendev.org/726189 | 16:12 |
| *** diurnalist has joined #openstack-kolla | 16:12 | |
| Fl1nt | Yep, our issue is a mix of "troubles" | 16:13 |
| Fl1nt | first | 16:13 |
| Fl1nt | we don't allow nova to access all pools | 16:13 |
| Fl1nt | same things for cinder | 16:13 |
| Fl1nt | then | 16:13 |
| Fl1nt | the problem is that cinder requesting the volumes/vms using the nova key is actually throwing a CEPH warning/error as you come with cinder user and give him nova keyring | 16:14 |
| Fl1nt | so to resolve that issue | 16:14 |
| Fl1nt | we just use cinder user and keyring within nova config | 16:14 |
| Fl1nt | because having to specify a different keypath to the cinder user within the ceph config would be a bit uggly. | 16:15 |
| noxoid | ^ can confirm, i do similar to make cinder-backup work with an external ceph cluster | 16:17 |
| noxoid | its not pretty | 16:17 |
| yoctozepto | the issue could be that nova has this "ephemeral" disks via ceph which use nova's config, bypassing cinder entirely | 16:18 |
| Fl1nt | yep | 16:18 |
| yoctozepto | for the normal "volumes" via cinder, it uses cinder's credentials for all the operations with connection provided via os-brick module | 16:18 |
| Fl1nt | and definitely the issue is with ephemeral, that changed the way they're bring to the VM from libvirt direct call to using cinder call or vice-versa, I can't exactly remember the cas but noticed it for later fix/patch | 16:19 |
| yoctozepto | you might also like to read http://lists.openstack.org/pipermail/openstack-discuss/2020-April/014280.html - I was curious about why ceph is in this odd position of having two independent interfaces as far as vms are concerned | 16:19 |
| yoctozepto | well, nova also switched something regarding the rbd calls in the meantime | 16:21 |
| yoctozepto | lemme check | 16:21 |
| Fl1nt | Actually pretty good question that I asked myself too, and the only thing that I can think of is for performance, like, having libvirt calling CEPH directly is way more quicker than having to use cinder | 16:21 |
| yoctozepto | since Train -> https://review.opendev.org/702198 | 16:23 |
| patchbot | patch 702198 - kolla-ansible - External Ceph: copy also cinder keyring to nova se... (MERGED) - 4 patch sets | 16:23 |
| yoctozepto | Fl1nt: nope, it still uses libvirt directly to orchestrate rbd mount in vm | 16:24 |
| Fl1nt | yes, just checked the sourcecode | 16:24 |
| yoctozepto | Fl1nt: it is just that people wanted ceph for ephemeral disks, while there is no general support for cinder ephemeral disks | 16:24 |
| yoctozepto | and so it sticked | 16:24 |
| yoctozepto | oh well | 16:24 |
| Fl1nt | so both cinder and nova use rbd python lib | 16:25 |
| Fl1nt | but | 16:25 |
| Fl1nt | before that | 16:25 |
| Fl1nt | libvirt actually handled those directly | 16:25 |
| yoctozepto | yeah, cleanup | 16:25 |
| Fl1nt | yep, hence why that doesn't work with rocky anymore | 16:25 |
| *** Limech has joined #openstack-kolla | 16:25 | |
| yoctozepto | well, it does not affect rocky | 16:25 |
| yoctozepto | rocky uses libvirt to do the cleanup still | 16:26 |
| Fl1nt | it's, I'm running rocky and get that issue | 16:26 |
| yoctozepto | :S | 16:26 |
| Fl1nt | ^^ | 16:26 |
| yoctozepto | I'm lost then ;p | 16:26 |
| Fl1nt | hance why I noted that issue and how to solve it within kolla ^^ | 16:26 |
| Fl1nt | -a+e | 16:27 |
| priteau | Fl1nt: It would be nice to see your rbd config for nova.conf and cinder.conf, as well as what calls are exactly failing. I've configured external ceph on various releases, from queens to train, and haven't noticed it | 16:29 |
| Fl1nt | I'll do deploy on tomorrow, I'll capture that | 16:30 |
| priteau | Great thanks | 16:30 |
| Fl1nt | BTW, having both nova and cinder keyring make it pretty much useless as we would only have one key used and both access the same pools, so even if you want to put volumes on spinning and vms on NVMe you just have to push a differnt ruleset for both pools | 16:31 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Improve fernet_token_expiry precheck https://review.opendev.org/698441 | 16:31 |
| *** rpittau is now known as rpittau|afk | 16:34 | |
| *** diurnalist has quit IRC | 16:35 | |
| *** evrardjp has quit IRC | 16:36 | |
| *** evrardjp has joined #openstack-kolla | 16:36 | |
| priteau | Fl1nt: Ah, you use images_rbd_pool? | 16:37 |
| priteau | nova_backend == "rbd" | 16:37 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Stop mocking ansible modules globally https://review.opendev.org/726081 | 16:38 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Add extend_lists option to merge_yaml https://review.opendev.org/685671 | 16:38 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Support customizing prometheus.cfg files https://review.opendev.org/686205 | 16:38 |
| openstackgerrit | Will Szumski proposed openstack/kolla-ansible master: Add extras directory to prometheus config https://review.opendev.org/688374 | 16:38 |
| openstackgerrit | Merged openstack/kolla-ansible stable/train: Update Advanced Config guide to clarify paths https://review.opendev.org/726189 | 16:42 |
| Fl1nt | yes @priteau everything is on CEPH | 16:42 |
| yoctozepto | and we got rid of nova/nova.conf mentions, yay | 16:43 |
| *** bengates has quit IRC | 16:45 | |
| Fl1nt | @yoctozepto, what do you mean? We deprecated another thing? :D | 16:45 |
| priteau | OK. The one deployment I work with that uses this actually has the same key for nova and cinder, so I can't say that there is no bug... | 16:45 |
| Fl1nt | ^^ | 16:46 |
| Fl1nt | we'll narrow things down the rabbit hole and fix that :D | 16:46 |
| priteau | And this is a new issue in Rocky, not in Queens? | 16:46 |
| *** seco has joined #openstack-kolla | 16:46 | |
| Fl1nt | can't say for Queens as I don't get HCI nodes outside of rocky version | 16:47 |
| Fl1nt | @priteau, you so still use local disk and just ceph as volumes tho? | 16:47 |
| priteau | no, ceph for everything, no local disk used | 16:47 |
| priteau | (in that one deployment) | 16:48 |
| Fl1nt | so your nova_backend is rbd actually isn't ? | 16:48 |
| priteau | yes | 16:48 |
| Fl1nt | aaah yes, it's just you only use one user/key for both services ^^ | 16:49 |
| Fl1nt | ok got it | 16:49 |
| Fl1nt | sorry, my brain is melting at the moment ^^ | 16:49 |
| priteau | On other deployments we generally use local disks for nova images, and ceph for cinder volumes. Nova has its own ceph keys that can access the volume pool. That works well. | 16:50 |
| Fl1nt | Yes, when splitting things it work fine ^^ | 16:50 |
| *** seco has quit IRC | 16:51 | |
| sean-k-mooney | fyi i just filed https://bugs.launchpad.net/kolla-ansible/+bug/1877417 | 16:51 |
| openstack | Launchpad bug 1877417 in kolla-ansible "there are no docs for using octavia" [Undecided,New] | 16:51 |
| sean-k-mooney | kolla currently does not deploy octavia in a usable state | 16:52 |
| sean-k-mooney | and there are no docs | 16:52 |
| Fl1nt | so all in all, for those interested in HCI, our tech demo is currently running smoothly, 100 instances per nodes, all both CEPH/Nova 20% CPUs cap for CEPH, 70% for openstack 10% for OS. 4Gb memory per OSD set at ceph.conf level and instances available memory constrained at nova level. | 16:52 |
| Fl1nt | @sean-k-mooney, which release are you using? | 16:52 |
| sean-k-mooney | so i was able to get it too work but it took alot of time and i had to work around a few bugs | 16:52 |
| sean-k-mooney | Fl1nt: master | 16:52 |
| sean-k-mooney | but there are no docs on any release | 16:53 |
| Fl1nt | I definitely get a working octavia with Rocky and Stein. | 16:53 |
| Fl1nt | ok, let's add that to the todo list ^^ | 16:53 |
| Fl1nt | CA/TLS and region for neutrons and services related being the main issues isn't it @sean-k-mooney ? | 16:54 |
| sean-k-mooney | Fl1nt: given it does not create the network required to create loadblancer vms or create the ssh key or upload an amopara image to glance im not sure what you get is what i waould call working | 16:54 |
| sean-k-mooney | Fl1nt: the ca/certs are one issue | 16:54 |
| sean-k-mooney | that aslo is not done | 16:54 |
| sean-k-mooney | but there are a lot of issue with how its deployed today | 16:54 |
| sean-k-mooney | no certs, no image, no network, missing keypair, wrong project in service_auth config, no defualt flavor and i think one other issue | 16:55 |
| Fl1nt | @sean-k-mooney, oh right, yeah, need a little bit of love, CA\Certs works tho, or at least worked ^^ | 16:56 |
| sean-k-mooney | kolla ansible does deploy a running octavia service if you create the keypars but everything else is missing | 16:56 |
| Fl1nt | default flavor and project works too | 16:56 |
| Fl1nt | BUT | 16:56 |
| *** ysirndjuro has quit IRC | 16:56 | |
| *** ysirndjuro has joined #openstack-kolla | 16:57 | |
| Fl1nt | once again, that was for rocky and stein, may have broke later on. | 16:57 |
| Fl1nt | ok, that's nive of you @sean-k-mooney for the issue | 16:57 |
| sean-k-mooney | yep i think it did because kolla-ansible was not or is not running tempest to validate the service that are deploy actully work | 16:57 |
| Fl1nt | Interesing as I'm actually working on that exact part for now ^^ | 16:58 |
| sean-k-mooney | the problem is that form a docker perspceitve if you have the ca/tls certs | 16:58 |
| sean-k-mooney | then all the service will be running | 16:58 |
| sean-k-mooney | but if you try to create a loadbalancer then it will fail | 16:59 |
| *** mcnugit has quit IRC | 16:59 | |
| Fl1nt | hence why I'm bulding scenarios for my current customer, just to test the whole process, not just units | 16:59 |
| sean-k-mooney | so if you just check that the contianer are ther or that its registered in keystone then sure it will look ok until you go to use it | 16:59 |
| sean-k-mooney | yep | 17:00 |
| sean-k-mooney | there was an idea of haveing post checks at one point | 17:00 |
| sean-k-mooney | but i dont know if that was ever completed | 17:00 |
| sean-k-mooney | anyway with enough effort you can make it work but its needs a lot of reading of there docs and then figuring out how to make it woke with kolla-ansbile | 17:01 |
| *** seco has joined #openstack-kolla | 17:09 | |
| openstackgerrit | Fabian Zimmermann proposed openstack/kolla-ansible master: multipath.conf should be configureable https://review.opendev.org/726211 | 17:12 |
| *** nathharp_ has joined #openstack-kolla | 17:12 | |
| *** nathharp has quit IRC | 17:15 | |
| *** seco has quit IRC | 17:20 | |
| yoctozepto | Fl1nt: no, we fixed docs :-) | 17:20 |
| yoctozepto | sean-k-mooney: kolla-ansible has no scenario with octavia in CI :-) | 17:21 |
| yoctozepto | sean-k-mooney: we are also not running full tempest but applying simple osc-based usage scenarios for the implemented deployment scenarios | 17:22 |
| yoctozepto | but *nothing* for octavia | 17:22 |
| yoctozepto | sean-k-mooney: https://docs.openstack.org/kolla/latest/support_matrix.html | 17:22 |
| *** gmann is now known as gmann_afk | 17:25 | |
| *** priteau has quit IRC | 17:25 | |
| sean-k-mooney | yoctozepto: yep which is also why i was suggesting marking octavia support as tech preview or expeimental as there is no test, no docs and it does not work out of the box without modifying configs and doing several other manual steps | 17:28 |
| yoctozepto | sean-k-mooney: well, it applies to more than octavia | 17:28 |
| yoctozepto | sean-k-mooney: we aren't using the term 'tech preview' as we are not selling the product :-) | 17:29 |
| sean-k-mooney | it does but when i used to work on kolla-ansible there uses to be a requirement to include docs before merging any new service | 17:29 |
| yoctozepto | per the support matrix, octavia is community supported | 17:29 |
| sean-k-mooney | it seams like that was droped at somepoint | 17:29 |
| *** gfidente is now known as gfidente|afk | 17:29 | |
| yoctozepto | sean-k-mooney: likely I'm afraid :-( | 17:29 |
| yoctozepto | sean-k-mooney: though we are caring about docs for the last 3 cycles at least | 17:30 |
| yoctozepto | that is, including ussuri in progress | 17:30 |
| yoctozepto | obviously we aren't removing code for octaiva just because it has no docs | 17:30 |
| yoctozepto | :-) | 17:30 |
| sean-k-mooney | it used to be a hard requirement around newtone-queens ish | 17:31 |
| sean-k-mooney | e.g. core reviews would not +2 any feature/service unless it had docs that explained how to deploy it | 17:31 |
| sean-k-mooney | if that is not a current requirement i would suggest reintoducing it | 17:31 |
| sean-k-mooney | yoctozepto: an ya no need to remove the octavia code just letting you know its broken in as is | 17:32 |
| sean-k-mooney | but can be correccted via config overrides and some manual steps | 17:32 |
| yoctozepto | sean-k-mooney: yeah, we kinda know but nobody from core team was able to handle this, thanks for summarizing the issues though | 17:33 |
| yoctozepto | and the "not-ableness" stems stems from both lack of time and knowledge | 17:33 |
| sean-k-mooney | ya i have not used octavia before | 17:34 |
| sean-k-mooney | most of the time it took was learning how octavia worked and figuing out what was not done | 17:34 |
| sean-k-mooney | as another aside the ipxe image in centos8 appears to be broken | 17:34 |
| sean-k-mooney | i was able to eventually get ironic working | 17:35 |
| yoctozepto | sean-k-mooney: https://review.opendev.org/404648 | 17:35 |
| patchbot | patch 404648 - kolla-ansible - Implement octavia ansible role (MERGED) - 5 patch sets | 17:35 |
| sean-k-mooney | but i had to disable ipxe and call back to pxe | 17:35 |
| yoctozepto | it got in in ocata | 17:35 |
| sean-k-mooney | yoctozepto: yep i went back and checked | 17:35 |
| sean-k-mooney | i was hoping to fine docs or info in the commit messagne ro reslese notes but no | 17:36 |
| yoctozepto | sean-k-mooney: I think mgoddard would like to know that ipxe is b0rken with centos8 | 17:36 |
| yoctozepto | sean-k-mooney: yeah, we are left what it is :-) | 17:36 |
| yoctozepto | sean-k-mooney: if you can spare any cycles for upstreaming workarounds/fixes, then that would be great | 17:36 |
| sean-k-mooney | ya ill have to see if i can | 17:37 |
| sean-k-mooney | there are also no docs avaiable on how the neutron dynamic router plugin works in kolla but i also set that up yesterday | 17:38 |
| sean-k-mooney | i proably should have taken notes | 17:38 |
| sean-k-mooney | but it did work flawlessly | 17:38 |
| sean-k-mooney | my home openstack is now advertising routes to dynamically created tenant netwrok automatically to my home router via bgp | 17:39 |
| sean-k-mooney | its so nice | 17:39 |
| yoctozepto | sean-k-mooney: home routers normally don't speak bgp :D | 17:41 |
| sean-k-mooney | mgoddard: regarding the ipxe issue. it wouls apper that ipxe in legacy boot with ipv4 on centos8 is unable to open a network conenction to download the deploy ramdisk | 17:41 |
| sean-k-mooney | yoctozepto: im using a ubiquit edgerouter x as my home router | 17:41 |
| sean-k-mooney | et0: 34:97:f6:5a:fe:e8 using undionly on UNI-PCI04:00.0 (open) | 17:41 |
| sean-k-mooney | [Link:up, TX:0 TXE:1 RX:0 RXE:0] | 17:41 |
| sean-k-mooney | [TXE: 1 x "Network unreachable (http://ipxe.org/28086011"] | 17:41 |
| sean-k-mooney | that is more or less the error i get | 17:41 |
| yoctozepto | sean-k-mooney: crazy | 17:41 |
| yoctozepto | sean-k-mooney: as for ipxe -> issue is welcome :-) | 17:42 |
| yoctozepto | I mean lp bug report that is | 17:42 |
| sean-k-mooney | yoctozepto: oh its not a kolla issue | 17:42 |
| *** jbadiapa has quit IRC | 17:42 | |
| sean-k-mooney | yoctozepto: i think the binary that is packaged in centos is buggy | 17:42 |
| yoctozepto | but we are affected, aren't we? | 17:42 |
| sean-k-mooney | yes | 17:42 |
| yoctozepto | aye, then it's to be upstreamed to rdo | 17:43 |
| sean-k-mooney | but its optinal | 17:43 |
| sean-k-mooney | and ipxe is disabel by default | 17:43 |
| sean-k-mooney | so if you just fall back to pxe its fine | 17:43 |
| sean-k-mooney | yoctozepto: this is what i use for my home router https://www.ui.com/edgemax/edgerouter-x/ | 17:43 |
| sean-k-mooney | yoctozepto: i have fibe to the home so i plug the fiber terminater i get form my isp into eth0 directly and dont use any of ther equiment beyond that | 17:44 |
| sean-k-mooney | for like 30-60 dollar its really an amazing router for what i can do | 17:44 |
| sean-k-mooney | anyway i better go do some nova work. ill be deploying a few other service using master + centos-8 source | 17:46 |
| yoctozepto | sean-k-mooney: I see, well, in Poland you usually get ISP-specific router for your home network | 17:46 |
| sean-k-mooney | if i encounter any other issue or have time to fix them ill let ye know and file bugs or patches as i can | 17:46 |
| yoctozepto | sean-k-mooney: like, the fiber has no standard sfp interface | 17:46 |
| sean-k-mooney | yoctozepto: oh they sent me one im just not suing it | 17:46 |
| yoctozepto | yeah, but did you have to tweak the fiber to support sfp? | 17:47 |
| yoctozepto | that would violate my warranty ;p | 17:47 |
| sean-k-mooney | i have a optical terminator box that converts there fiber to ethernet over cat7 | 17:47 |
| yoctozepto | ah, so you did a tweak there, a'ight | 17:47 |
| sean-k-mooney | i dont have any sfp | 17:47 |
| yoctozepto | ok | 17:47 |
| yoctozepto | makes sense | 17:47 |
| sean-k-mooney | they provide a box made by hauwih to terminate the fiber. its basicaly a modem and then they just plug a router/gateway into it | 17:48 |
| sean-k-mooney | so i kept that but replated ther router with my own | 17:48 |
| yoctozepto | I see | 17:49 |
| yoctozepto | well, I'd have to first get that termination box | 17:49 |
| yoctozepto | but so far I'm happy with my ISP's router | 17:49 |
| yoctozepto | also its builtin AP | 17:50 |
| *** seco has joined #openstack-kolla | 17:53 | |
| Fl1nt | @sean-k-mooney, thanks a lot for bringing that doc issue, I've put it on my to fix list as I'll have to deploy it again later on and operators that will work on it will just hang me if there is no doc. | 17:54 |
| yoctozepto | Fl1nt: so you are running octavia? | 17:55 |
| Fl1nt | yep | 17:55 |
| yoctozepto | I guess we really need to run that questionnaire @ mgoddard | 17:56 |
| Fl1nt | Services that I actually run NOW: | 17:56 |
| Fl1nt | Designate / Octavia / Trove / Nova / Cinder / Swift on CEPH / Swift native / Glance / Keystone / Horizon / Sahara / Magnum | 17:57 |
| *** seco has quit IRC | 17:58 | |
| Fl1nt | forgot about Neutron in DVR Mode | 17:58 |
| yoctozepto | oh, designate, octavia, trove, sahara and magnum all need some love | 17:58 |
| Fl1nt | yeah ^^ | 17:58 |
| yoctozepto | I mean, in kolla-ansible | 17:58 |
| Fl1nt | hence why I've a to fix list that start to be pretty long :D | 17:59 |
| Fl1nt | yes | 17:59 |
| Fl1nt | I also forgot about Rally and Tempest | 17:59 |
| Fl1nt | which IMHO need the most love :p | 17:59 |
| Fl1nt | Planned services (during the year): Qinling / OVN / Masakari / Ironic | 18:00 |
| yoctozepto | osmanlicilegi: as for those survey platforms, we can actually run a hosted one it seems (noprofit research fits my institution), so if you (or anyone else) knows of good software for running surveys then it's also an option (and we support Chinese folks afaik) | 18:00 |
| Fl1nt | ironic is actually already running for our GPUs but we need to bring it back to the new cluster | 18:00 |
| yoctozepto | Fl1nt: well, yeah, they need some real support | 18:00 |
| yoctozepto | Fl1nt: ironic is pretty fine, ovn seemingly too | 18:01 |
| Fl1nt | yep, working on that topic currently | 18:01 |
| Fl1nt | ok cool ^^ | 18:01 |
| yoctozepto | masakari got in recently, not sure what it can do but it does something | 18:01 |
| yoctozepto | similarly qinling | 18:01 |
| yoctozepto | but I guess it needs an external kubernetes | 18:01 |
| yoctozepto | or reconfig after using magnum | 18:01 |
| Fl1nt | Masakari is for VMs HA | 18:01 |
| yoctozepto | (after each use of magnum that is, which kinda sucks if you ask me) | 18:01 |
| yoctozepto | Fl1nt: yeah, I know | 18:02 |
| yoctozepto | just not how much it works in kolla-ansible atm | 18:02 |
| Fl1nt | ah yes sorry, I didn't read that you were answering about qinling | 18:02 |
| Fl1nt | nevermind ^^ | 18:03 |
| yoctozepto | Fl1nt: got ya | 18:03 |
| yoctozepto | anyways, going afk for now, thanks for nice presentation, buh-bye | 18:03 |
| Fl1nt | for now, I'll stick with reviews,documentation and small patchs for kolla | 18:03 |
| Fl1nt | yeah me too, thanks everyone who assisted ^^ | 18:04 |
| *** Limech has quit IRC | 18:04 | |
| sean-k-mooney | Fl1nt: Desgignate is next on my list to deploy. im debating if i will add magnum. i dont really need it but i am plannign to run k8s on the openstack cloud at some point | 18:17 |
| sean-k-mooney | excpet for heat the service i have set to no are the service im thinking of enabling http://paste.openstack.org/show/793294/ | 18:22 |
| sean-k-mooney | the networking for ironic is not really explained currently and that took a lot of effort to get working | 18:23 |
| sean-k-mooney | i had to move my api network on to the ipmi network | 18:24 |
| sean-k-mooney | althoguh that might not be required if you dont try to use ipxe | 18:24 |
| sean-k-mooney | we hard code the address the ipxe https server listents on to be the api_interface | 18:25 |
| sean-k-mooney | so we realy shoudl have a seperate configuratbl for that | 18:25 |
| sean-k-mooney | https://github.com/ipxe/ipxe/pull/104 | 18:25 |
| sean-k-mooney | ipex dose not supprot static routes so it unclear if it would have worked or not | 18:26 |
| sean-k-mooney | in either case i was blocked form testing due to the other ipxe netwrok issue becaue | 18:26 |
| sean-k-mooney | it failed to conenct even when they were all on the same subnet | 18:26 |
| *** schwicht has joined #openstack-kolla | 18:28 | |
| *** e0ne has joined #openstack-kolla | 18:40 | |
| *** e0ne has quit IRC | 18:56 | |
| *** gmann_afk is now known as gmann | 19:06 | |
| *** k_mouza has quit IRC | 19:16 | |
| *** mcnugit has joined #openstack-kolla | 19:19 | |
| *** jonaspaulo has joined #openstack-kolla | 19:39 | |
| *** born2bake has joined #openstack-kolla | 19:46 | |
| *** livelace has quit IRC | 19:48 | |
| *** born2bake has joined #openstack-kolla | 19:48 | |
| *** k_mouza has joined #openstack-kolla | 20:05 | |
| *** mcnugit has quit IRC | 20:28 | |
| *** livelace has joined #openstack-kolla | 20:39 | |
| *** TrevorV has quit IRC | 20:44 | |
| *** born2bake has quit IRC | 20:54 | |
| *** e0ne has joined #openstack-kolla | 20:57 | |
| *** eliaswimmer__ has joined #openstack-kolla | 20:57 | |
| *** e0ne has quit IRC | 20:59 | |
| *** eliaswimmer_ has quit IRC | 21:00 | |
| *** livelace has quit IRC | 21:14 | |
| *** born2bake has joined #openstack-kolla | 21:30 | |
| *** mcnugit has joined #openstack-kolla | 21:47 | |
| *** nathharp_ has quit IRC | 22:01 | |
| *** nathharp has joined #openstack-kolla | 22:01 | |
| *** born2bake has quit IRC | 22:03 | |
| *** Fl1nt has quit IRC | 22:08 | |
| *** jonaspaulo has quit IRC | 22:27 | |
| *** born2bake has joined #openstack-kolla | 22:34 | |
| *** born2bake has quit IRC | 22:35 | |
| *** seco has joined #openstack-kolla | 22:37 | |
| openstackgerrit | James Kirsch proposed openstack/kolla-ansible master: Add support for encrypting Barbican API https://review.opendev.org/726258 | 22:50 |
| *** mcnugit has quit IRC | 22:53 | |
| *** spotz has quit IRC | 23:16 | |
| *** k_mouza has quit IRC | 23:18 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!