| *** tosky has quit IRC | 00:26 | |
| *** cshen has joined #openstack-ansible | 00:35 | |
| *** macz_ has quit IRC | 00:39 | |
| *** cshen has quit IRC | 00:40 | |
| *** mmethot_ has quit IRC | 00:51 | |
| *** gshippey has quit IRC | 02:25 | |
| *** spatel has joined #openstack-ansible | 02:29 | |
| *** cshen has joined #openstack-ansible | 02:36 | |
| *** cshen has quit IRC | 02:40 | |
| *** ianychoi has quit IRC | 03:00 | |
| *** ianychoi has joined #openstack-ansible | 03:21 | |
| *** ThiagoCMC has quit IRC | 04:09 | |
| *** NewJorg has quit IRC | 04:26 | |
| *** rh-jelabarre has quit IRC | 04:27 | |
| *** NewJorg has joined #openstack-ansible | 04:28 | |
| *** cshen has joined #openstack-ansible | 04:36 | |
| *** cshen has quit IRC | 04:40 | |
| *** spatel has quit IRC | 05:16 | |
| *** yasemind96 has joined #openstack-ansible | 05:28 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-ansible | 05:33 | |
| *** cshen has joined #openstack-ansible | 05:40 | |
| *** cshen has quit IRC | 05:44 | |
| *** djhankb has quit IRC | 06:13 | |
| *** djhankb has joined #openstack-ansible | 06:14 | |
| *** cshen has joined #openstack-ansible | 07:00 | |
| *** cshen has quit IRC | 07:04 | |
| *** miloa has joined #openstack-ansible | 07:07 | |
| *** cshen has joined #openstack-ansible | 07:23 | |
| *** pcaruana has joined #openstack-ansible | 07:25 | |
| *** jbadiapa has joined #openstack-ansible | 07:41 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova stable/ussuri: Do not symlink ceph libraries for distro path https://review.opendev.org/761072 | 08:07 |
|---|---|---|
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova stable/train: Do not symlink ceph libraries for distro path https://review.opendev.org/761073 | 08:07 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova stable/stein: Do not symlink ceph libraries for distro path https://review.opendev.org/761074 | 08:07 |
| *** andrewbonney has joined #openstack-ansible | 08:14 | |
| *** macz_ has joined #openstack-ansible | 08:18 | |
| *** rpittau|afk is now known as rpittau | 08:19 | |
| *** _kukacz has joined #openstack-ansible | 08:23 | |
| *** macz_ has quit IRC | 08:23 | |
| *** tosky has joined #openstack-ansible | 08:40 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/stein: Return bionic jobs to voting https://review.opendev.org/759676 | 08:59 |
| *** kukacz_ has joined #openstack-ansible | 09:04 | |
| *** yasemind96 has quit IRC | 09:19 | |
| *** kukacz_ has left #openstack-ansible | 09:33 | |
| *** _kukacz has quit IRC | 09:40 | |
| *** _kukacz has joined #openstack-ansible | 09:42 | |
| *** frenzy_friday has joined #openstack-ansible | 09:44 | |
| *** _kukacz is now known as kukacz | 09:48 | |
| *** sshnaidm|afk is now known as sshnaidm|rover | 09:56 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance stable/ussuri: Do not symlink ceph libraries for distro path https://review.opendev.org/760816 | 10:02 |
| openstackgerrit | Merged openstack/ansible-role-systemd_networkd master: Add GPG Key for EPEL8 Repo https://review.opendev.org/759145 | 10:06 |
| *** macz_ has joined #openstack-ansible | 10:06 | |
| *** macz_ has quit IRC | 10:11 | |
| *** spatel has joined #openstack-ansible | 10:18 | |
| *** spatel has quit IRC | 10:23 | |
| *** fanfi has quit IRC | 10:29 | |
| noonedeadpunk | another vote for https://review.opendev.org/#/c/759472/ would be great:) | 10:37 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance stable/train: Do not symlink ceph libraries for distro path https://review.opendev.org/760820 | 10:37 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/ussuri: Do not symlink ceph libraries for distro path https://review.opendev.org/760814 | 10:38 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/stein: Do not symlink ceph libraries for distro path https://review.opendev.org/760815 | 10:38 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/stein: Actually use ANSIBLE_FORKS in openstack-ansible.rc https://review.opendev.org/760540 | 10:41 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/stein: Actually use ANSIBLE_FORKS in openstack-ansible.rc https://review.opendev.org/760540 | 10:42 |
| *** d34dh0r53 has quit IRC | 10:50 | |
| *** kukacz has quit IRC | 10:51 | |
| *** kukacz has joined #openstack-ansible | 10:51 | |
| *** macz_ has joined #openstack-ansible | 10:57 | |
| *** macz_ has quit IRC | 11:02 | |
| *** gshippey has joined #openstack-ansible | 11:09 | |
| *** pcaruana has quit IRC | 11:20 | |
| *** pcaruana has joined #openstack-ansible | 11:33 | |
| *** cloudxtiny has joined #openstack-ansible | 11:46 | |
| cloudxtiny | hello everyone. sorry if this is the wrong place for this question, but I am having trouble with lxc containers in openstack ansible. containers on 2 of my infra nodes can't ping each other but they can all ping the 3rd infra node | 11:46 |
| cloudxtiny | any one had a similar issue? | 11:47 |
| noonedeadpunk | place is correct, but not sure how to help here | 11:49 |
| noonedeadpunk | I was experiencing such issues but it was networking issue on the lower layer | 11:50 |
| cloudxtiny | noonedeadpunk thanks. Its really strange. I can see the ARP request hit the br-mgmt bride on both servers but it never gets to the container | 11:51 |
| noonedeadpunk | and all lxc hosts can ping each other right? | 11:51 |
| cloudxtiny | yes they can | 11:51 |
| cloudxtiny | really strange. I have added an IP address to the br-mgmt bridge on each host and they can ping each others containers | 11:52 |
| noonedeadpunk | I can recall seing exact same thing but don't remeber how I fixed that... | 11:52 |
| cloudxtiny | its just the containers on two of the hosts that can't ping each other on the br-mgmt network | 11:52 |
| cloudxtiny | I was wondering if it's a CentOS 8 issue? | 11:53 |
| noonedeadpunk | and they have net.ipv4.ip_forward enabled in sysctl right? | 11:53 |
| cloudxtiny | yes | 11:53 |
| noonedeadpunk | spatel had everything working so I don't think it's centos 8 specific thing | 11:55 |
| cloudxtiny | okay. So it must be my setup then. | 11:57 |
| cloudxtiny | I am using tlb bonding on the physical interface for all nodes so it's strange that this is affecting 2 them | 11:57 |
| noonedeadpunk | I'm wondering if I had to configurre something on switches... | 11:57 |
| noonedeadpunk | but really can't recall, it was years ago from now:( | 11:58 |
| cloudxtiny | thanks anyway :-). | 11:58 |
| noonedeadpunk | but it was nothing wrong with containers themselves 100% | 11:59 |
| cloudxtiny | thanks. I can rule that out :-). It strange that the APR reply gets to the bridge but not the bridged interface of the target container. | 12:00 |
| *** rfolco has joined #openstack-ansible | 12:00 | |
| cloudxtiny | I worry it's something do do with MTU but it's set the same on all hosts | 12:01 |
| *** mike44333 has quit IRC | 12:04 | |
| noonedeadpunk | oh, well, containers mtu is 1500, so yeah, it might be it | 12:11 |
| noonedeadpunk | in case you have vlans with 1450 mtu | 12:12 |
| cloudxtiny | I set it 9000 on all interfaces except eth0 (lxcbr0) | 12:21 |
| cloudxtiny | *I set it to 9000 on all interfaces except eth0 (lxcbr0) | 12:23 |
| *** kukacz has quit IRC | 12:30 | |
| *** kukacz has joined #openstack-ansible | 12:31 | |
| *** shyamb has joined #openstack-ansible | 12:34 | |
| cloudxtiny | hummm. problem seems to be related to the subnet. | 12:37 |
| cloudxtiny | I am using 172.16.8.0/22 but I can't ping any ip address in the 172.16.9.xx from the problem containers even though they all have the same subnet mask | 12:39 |
| *** shyamb has quit IRC | 12:48 | |
| *** rh-jelabarre has joined #openstack-ansible | 13:07 | |
| mgariepy | default ping will get through even in cas of mtu issue unless you pass it -S <some big number> to force the packet size. | 13:10 |
| *** rh-jelabarre has quit IRC | 13:17 | |
| *** rh-jelabarre has joined #openstack-ansible | 13:17 | |
| mgariepy | cloudxtiny, is the netmask/ route configured correclty ? | 13:21 |
| mgariepy | cloudxtiny, are your server connected in the same switch ? or you have a fancy network configuration ? | 13:24 |
| *** kukacz is now known as kukacz_ | 13:27 | |
| *** mmethot has joined #openstack-ansible | 13:31 | |
| *** yann-kaelig has joined #openstack-ansible | 13:37 | |
| *** kukacz_ has quit IRC | 13:52 | |
| *** kukacz has joined #openstack-ansible | 13:52 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/ussuri: Do not symlink ceph libraries for distro path https://review.opendev.org/761072 | 14:06 |
| *** kukacz_ has joined #openstack-ansible | 14:08 | |
| *** kukacz has left #openstack-ansible | 14:08 | |
| *** d34dh0r53 has joined #openstack-ansible | 14:10 | |
| *** spatel has joined #openstack-ansible | 14:11 | |
| *** kukacz_ has quit IRC | 14:16 | |
| *** kukacz_ has joined #openstack-ansible | 14:16 | |
| *** jamesdenton has joined #openstack-ansible | 14:17 | |
| spatel | jamesdenton: morning | 14:20 |
| jamesdenton | morning | 14:20 |
| *** kukacz_ is now known as kukacz | 14:20 | |
| spatel | finally i did some hacks here and there to make OVS and Linuxbridge compute working in mix environment | 14:21 |
| spatel | one compute running in OVS and other compute running in LinuxBridge | 14:21 |
| spatel | but its messy configuration which i am trying to avoid (not a good idea) | 14:21 |
| spatel | get back to dpdk (I don't think centos8 version of opensvswitch support native dpdk) | 14:22 |
| spatel | i did my best to look around on internet and all i found compile/compile/compile but not a single RPM | 14:23 |
| jamesdenton | hmm, i would be highly surprised if it didn't, but you never know | 14:23 |
| cloudxtiny | mgariepy servers on the same switch and I have checked the subnet masks, they are all /22 :-( | 14:29 |
| cloudxtiny | thanks | 14:29 |
| spatel | jamesdenton: i tried so many thing but none of them work | 14:30 |
| spatel | lets see what you come up with | 14:30 |
| cloudxtiny | spatel so are you sugguestion not o use openvswitch with centos8? I was planing to do that once I get through my container network problem | 14:31 |
| spatel | cloudxtiny: no, I am going to use OVS :) all i am saying centos8 doesn't have support of openvswitch with DPDK with native RPMS. | 14:32 |
| cloudxtiny | if its too much of a headache, then maybe I'll just change the config back to use linuxbridge | 14:32 |
| cloudxtiny | spatel ahh. I see, thanks :-) | 14:32 |
| spatel | cloudxtiny: i would say go with OVS (in future you can utilize many features ) | 14:32 |
| spatel | OVS is much stable now days compare to previous years and getting better. | 14:33 |
| jamesdenton | spatel spinning up a server now, but it may take a while to get things worked out | 14:36 |
| spatel | jamesdenton: i am thinking to deploy my new 200 node cloud using OVS (good by to LinuxBridge), and later i can add dpdk whenever its ready | 14:37 |
| spatel | if i go with LinuxBridge then i will get stuck with it and never able to get out without hacking my codes. | 14:37 |
| jamesdenton | Well, if you are to believe this: https://docs.openstack.org/kolla/train/admin/template-override/ovs-dpdk.html, then you're right, there's no rpm | 14:38 |
| spatel | oh boy.. so only option is build RPM right | 14:42 |
| spatel | jamesdenton: what do you think? | 14:44 |
| jamesdenton | not sure yet | 14:44 |
| spatel | i can compile RPM with same version and put it in OSA repo to get it out | 14:44 |
| spatel | let me ask this question to CentOS8 community and see what they offer | 14:45 |
| mgariepy | jamesdenton, i'll go back to lxb for my deployment and try to improve the ovn support in osa before i switch to it. | 14:45 |
| *** cloudxtiny has quit IRC | 14:45 | |
| jamesdenton | mgariepy having issues? | 14:45 |
| mgariepy | i need to figure out the containerization of the gateway nodes, and the ha stuff for ovn that needs to be fixed also. | 14:46 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/train: Do not symlink ceph libraries for distro path https://review.opendev.org/760813 | 14:46 |
| mgariepy | for the ha part i think we could just run the ovndb northd and southd (ovsdb) process in cluster mode which should be simple enough | 14:47 |
| jamesdenton | have you checked to see if it's not already in that mode? | 14:52 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ussuri: Actually use ANSIBLE_FORKS in openstack-ansible.rc https://review.opendev.org/760366 | 14:53 |
| *** csmart has quit IRC | 14:54 | |
| *** fridtjof[m] has quit IRC | 14:54 | |
| *** csmart has joined #openstack-ansible | 14:55 | |
| *** fridtjof[m] has joined #openstack-ansible | 14:55 | |
| *** masterpe has quit IRC | 14:57 | |
| *** ioni has quit IRC | 14:57 | |
| *** csmart has quit IRC | 14:59 | |
| *** fridtjof[m] has quit IRC | 14:59 | |
| mgariepy | jamesdenton, it's not, | 15:04 |
| openstackgerrit | Merged openstack/openstack-ansible stable/train: Actually use ANSIBLE_FORKS in openstack-ansible.rc https://review.opendev.org/760367 | 15:05 |
| jamesdenton | cool | 15:05 |
| mgariepy | but i'm still interested to it. just don't have quite enough time to tests/fix all the little things. | 15:05 |
| jamesdenton | understood | 15:06 |
| mgariepy | i'll build a smaller testbed with less complex network to test. | 15:06 |
| jamesdenton | if you want to make a list of the little things you saw, i can try to work on it as time allows | 15:06 |
| mgariepy | ok cool. | 15:06 |
| mgariepy | i'll ping you when i can take the time for that. | 15:07 |
| mgariepy | also, with lxb i kinda know where it fails. and how to fix differents stuff if needed. | 15:07 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/train: Do not symlink ceph libraries for distro path https://review.opendev.org/761073 | 15:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/stein: Do not symlink ceph libraries for distro path https://review.opendev.org/761074 | 15:14 |
| mgariepy | jamesdenton, not so sure anymore lol. | 15:20 |
| jamesdenton | :) | 15:21 |
| *** ioni has joined #openstack-ansible | 15:31 | |
| mgariepy | i need to tests it more ;d haha | 15:33 |
| *** fridtjof[m] has joined #openstack-ansible | 15:38 | |
| *** csmart has joined #openstack-ansible | 15:39 | |
| mgariepy | jamesdenton, https://bugs.launchpad.net/openstack-ansible/+bug/1790933 | 15:44 |
| openstack | Launchpad bug 1790933 in openstack-ansible "Implement OVSDB clustering for OVN" [Medium,In progress] - Assigned to James Denton (james-denton) | 15:44 |
| mgariepy | lol | 15:44 |
| jamesdenton | :| | 15:44 |
| jamesdenton | see i thought it sounded familiar | 15:44 |
| jamesdenton | lol | 15:44 |
| mgariepy | haha indeed. | 15:45 |
| mgariepy | but yep it needs some testing. | 15:45 |
| jamesdenton | no doubt | 15:45 |
| *** ioni has quit IRC | 15:47 | |
| *** fridtjof[m] has quit IRC | 15:47 | |
| *** csmart has quit IRC | 15:47 | |
| mgariepy | jamesdenton, do you run all on metal with ovs + dvr ? or you also have centralized gateways somewhere | 15:49 |
| mgariepy | ? | 15:49 |
| jamesdenton | with Stein+ we have run on metal w/ DVR, yes. the infra nodes are the dvr_snat nodes | 15:50 |
| mgariepy | ok | 15:52 |
| mgariepy | ovs works well ? | 15:52 |
| jamesdenton | there are some issues with DVR for certain use cases, but overall OVS itself is fine | 15:54 |
| mgariepy | k | 15:58 |
| *** fridtjof[m] has joined #openstack-ansible | 15:59 | |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 16:00 |
| openstack | Meeting started Tue Nov 3 16:00:24 2020 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:00 | |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:00 |
| noonedeadpunk | #topic rollcall | 16:00 |
| *** openstack changes topic to "rollcall (Meeting topic: openstack_ansible_meeting)" | 16:00 | |
| noonedeadpunk | o/ | 16:00 |
| mgariepy | hey ! | 16:00 |
| noonedeadpunk | seems we won't have much more attandees :) | 16:04 |
| noonedeadpunk | s/:)/:(/ | 16:04 |
| noonedeadpunk | #topic bug triage | 16:04 |
| *** openstack changes topic to "bug triage (Meeting topic: openstack_ansible_meeting)" | 16:04 | |
| mgariepy | for once i'm here. | 16:05 |
| noonedeadpunk | yeah:) | 16:06 |
| jamesdenton | o/ | 16:06 |
| gshippey | o/ | 16:06 |
| noonedeadpunk | so we have one new bug and it jamesdenton one | 16:06 |
| noonedeadpunk | https://bugs.launchpad.net/openstack-ansible/+bug/1902585 | 16:06 |
| openstack | Launchpad bug 1902585 in openstack-ansible "uwsgi socket backlog exceeded" [Undecided,New] | 16:06 |
| jamesdenton | ahh yes | 16:07 |
| noonedeadpunk | well I think we need to bump limits here indeed | 16:08 |
| noonedeadpunk | not sure about reasonable defaults part) | 16:08 |
| noonedeadpunk | as 1024 sounds as enough... | 16:09 |
| jamesdenton | i think there is a memory penalty.. maybe 8kB per, or something like that. nothing huge | 16:09 |
| noonedeadpunk | jamesdenton: `some buffer` means you've bumped smth else except mentioned things? | 16:10 |
| jamesdenton | well yes, we did 1024 and it worked for a bit, but then that limit was hit. and so on | 16:10 |
| jamesdenton | i'm not really sure why the queue wasn't emptied fast enough | 16:10 |
| mgariepy | how many hosts ? | 16:10 |
| noonedeadpunk | I think it was about how many connections it was serving | 16:11 |
| *** shyamb has joined #openstack-ansible | 16:11 | |
| jamesdenton | good question, i am not sure. looking | 16:12 |
| *** shyamb has quit IRC | 16:12 | |
| jamesdenton | 137 computes | 16:14 |
| *** gyee has joined #openstack-ansible | 16:14 | |
| *** fanfi has joined #openstack-ansible | 16:17 | |
| *** macz_ has joined #openstack-ansible | 16:18 | |
| *** dwilde has joined #openstack-ansible | 16:22 | |
| noonedeadpunk | sorry got distracted | 16:23 |
| noonedeadpunk | so, um | 16:23 |
| noonedeadpunk | eventually it's pretty easy to override uwsgi listen | 16:24 |
| *** d34dh0r53 has quit IRC | 16:24 | |
| *** dwilde has quit IRC | 16:24 | |
| *** d34dh0r53 has joined #openstack-ansible | 16:24 | |
| *** masterpe has joined #openstack-ansible | 16:24 | |
| *** ioni has joined #openstack-ansible | 16:24 | |
| *** csmart has joined #openstack-ansible | 16:25 | |
| jamesdenton | true. | 16:25 |
| noonedeadpunk | but not globally.... | 16:25 |
| noonedeadpunk | the one thing I was facing rcently is need to bump prlimit | 16:26 |
| noonedeadpunk | * LimitNOFILE | 16:26 |
| noonedeadpunk | but it feels it's not the case here | 16:26 |
| noonedeadpunk | So what I'm trying to understand if we should change defaults and how exactly | 16:27 |
| jamesdenton | right, i am not sure. it might help to have an override available and maybe a directive on how to use it. but keep the defaults as-is? They seem to work OK except in the case of a severely downsized control plane | 16:27 |
| jamesdenton | in out case we went from 3->1 | 16:28 |
| jamesdenton | *our | 16:28 |
| jamesdenton | and i don't have data at this time to show normal load vs connections growing at that time | 16:28 |
| *** gyee has quit IRC | 16:31 | |
| noonedeadpunk | well, sysctl can be overwritten with openstack_user_kernel_options | 16:33 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-openstack_hosts/src/branch/master/defaults/main.yml#L129 | 16:33 |
| noonedeadpunk | and uwsgi can also be overwritten... | 16:33 |
| noonedeadpunk | the only thing missing is global override for all uwsgi services | 16:33 |
| noonedeadpunk | will this cover the usecase? | 16:34 |
| *** dave-mccowan has joined #openstack-ansible | 16:36 | |
| noonedeadpunk | however this downsize is kind of example of the upgrade scenario | 16:37 |
| jamesdenton | yeah, the kernel param is no issue. but would the listen override need to go here? https://opendev.org/openstack/ansible-role-uwsgi/src/branch/master/templates/uwsgi.ini.j2 | 16:39 |
| jamesdenton | and that template is used for individual uwsgi apps, right? | 16:39 |
| noonedeadpunk | yep | 16:39 |
| noonedeadpunk | and I think each service has overrides | 16:39 |
| noonedeadpunk | like nova_api_metadata_uwsgi_ini_overrides | 16:40 |
| jamesdenton | ahh ok | 16:40 |
| jamesdenton | yes i think you're right | 16:40 |
| *** gyee has joined #openstack-ansible | 16:41 | |
| spatel | jamesdenton: I have now DPDK supported OVS | 16:41 |
| -spatel- [root@compute-2 ~]# ovs-vswitchd --version | 16:41 | |
| -spatel- ovs-vswitchd (Open vSwitch) 2.12.0 | 16:41 | |
| -spatel- DPDK 19.11.0 | 16:41 | |
| jamesdenton | nova_api_os_compute_uwsgi_ini_overrides in this case. ok, so maybe just a docs change? | 16:41 |
| jamesdenton | looks like somaxconn is bumped to 4096 in latest 5.x kernel | 16:41 |
| noonedeadpunk | yeah, I guess we can just document things | 16:42 |
| jamesdenton | of course, folks have to READ :D | 16:42 |
| noonedeadpunk | and I will add patch to uwsgi to be able to have overwrite for all services at once rather than do it for each separately | 16:42 |
| jamesdenton | cool | 16:43 |
| *** gyee has quit IRC | 16:45 | |
| noonedeadpunk | btw, where do you see docs? | 16:45 |
| noonedeadpunk | should we do it for each role or just some small section in the integrated repo? | 16:46 |
| *** rpittau is now known as rpittau|afk | 16:46 | |
| jamesdenton | hmm | 16:46 |
| jamesdenton | https://docs.openstack.org/project-deploy-guide/openstack-ansible/draft/app-advanced-config-override.html | 16:47 |
| jamesdenton | ? | 16:47 |
| jamesdenton | seems old | 16:47 |
| noonedeadpunk | what is draft.... | 16:48 |
| jamesdenton | no idea | 16:48 |
| noonedeadpunk | lol what is it:) | 16:48 |
| noonedeadpunk | well, I'm not sure we can maintain all type of overrides... | 16:49 |
| *** cloudxtiny has joined #openstack-ansible | 16:49 | |
| noonedeadpunk | I'd rather drop list of `Currently available overrides` and did describe general approach or naming convention we're using and asking to double check naming for each role | 16:49 |
| jamesdenton | agreed | 16:50 |
| noonedeadpunk | well, we already have such note.... | 16:50 |
| noonedeadpunk | but yeah. we need to fix this page | 16:50 |
| noonedeadpunk | #agreed document way of overrides usage (including uwsgi ones). use https://docs.openstack.org/project-deploy-guide/openstack-ansible/draft/app-advanced-config-override.html as base | 16:51 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/ansible-role-uwsgi master: Allow to globaly override uwsgi params https://review.opendev.org/761198 | 16:55 |
| noonedeadpunk | #topic office hours | 16:55 |
| *** openstack changes topic to "office hours (Meeting topic: openstack_ansible_meeting)" | 16:55 | |
| noonedeadpunk | eventually what I wanted to say is that next wednesday stein goes to extended maintenance, so I'm going to do last release this week | 16:56 |
| noonedeadpunk | and afterwards I will switch branches to stable/stein for services and roles, to tag it with EM | 16:56 |
| noonedeadpunk | so that there was no need in bumps (as they won't have much sense anyway) | 16:57 |
| noonedeadpunk | and thanks everyone for attending! | 16:57 |
| noonedeadpunk | #endmeeting | 16:57 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: http://bit.ly/osa-review-board-v2" | 16:57 | |
| openstack | Meeting ended Tue Nov 3 16:57:42 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:57 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2020/openstack_ansible_meeting.2020-11-03-16.00.html | 16:57 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2020/openstack_ansible_meeting.2020-11-03-16.00.txt | 16:57 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2020/openstack_ansible_meeting.2020-11-03-16.00.log.html | 16:57 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/ansible-role-uwsgi master: Allow to globaly override uwsgi params https://review.opendev.org/761198 | 16:59 |
| noonedeadpunk | jamesdenton: I think that is relevant doc https://docs.openstack.org/openstack-ansible/latest/reference/configuration/using-overrides.html | 17:01 |
| noonedeadpunk | also missing uwsgi ones though | 17:01 |
| jamesdenton | ahh yeah | 17:02 |
| spatel | jamesdenton: i got this reply from centOS-8 community developer regrading DPDK support - http://paste.openstack.org/show/799662/ | 17:05 |
| jamesdenton | sounds promising | 17:06 |
| spatel | does that means i have to ditch ussuri and start looking at victoria | 17:07 |
| *** dave-mccowan has quit IRC | 17:12 | |
| cloudxtiny | anyone know where I get the urls and ports for the services when I am using an external LB? | 17:14 |
| cloudxtiny | can't see anything in the docs | 17:14 |
| *** dave-mccowan has joined #openstack-ansible | 17:17 | |
| noonedeadpunk | cloudxtiny: we have haproxy config defined here https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/haproxy/haproxy.yml#L38 | 17:18 |
| noonedeadpunk | eventually it's set of vars with which it will be defined, but you may get list of ports | 17:19 |
| cloudxtiny | noonedeadpunk thanks | 17:19 |
| *** fresta has quit IRC | 17:22 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 17:22 | |
| spatel | jamesdenton: what this block trying to do - https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/tasks/providers/setup_ovs_dpdk.yml#L25 | 17:32 |
| spatel | my playbook failed around it | 17:32 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [doc] Mention uwsgi overrides in doc https://review.opendev.org/761206 | 17:35 |
| *** suryasingh has quit IRC | 17:36 | |
| spatel | may need to fix that code and create mkdir -p /etc/dpdk/ | 17:37 |
| noonedeadpunk | yeah, seems it is | 17:39 |
| noonedeadpunk | worth creating here https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/tasks/neutron_pre_install.yml#L53-L88 conditionally | 17:39 |
| spatel | Yes, i think ubuntu use /etc/dpdk but i am not seeing that is the case in CentOS-8 but let me verify otherwise i will submit patch | 17:41 |
| *** tosky has quit IRC | 17:57 | |
| *** dwilde has joined #openstack-ansible | 18:05 | |
| *** d34dh0r53 has quit IRC | 18:05 | |
| spatel | jamesdenton: i am getting this error on my OVS - http://paste.openstack.org/show/799664/ | 18:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum stable/ussuri: Drop magnum distro CI jobs https://review.opendev.org/760527 | 18:15 |
| *** miloa has quit IRC | 18:17 | |
| dmsimard | jamesdenton: o/ did you figure out how to run the ansible ovs tests locally ? | 18:20 |
| jamesdenton | hi dmsimard - i haven't had a chance to circle back to that | 18:21 |
| dmsimard | I suppose you know about https://github.com/ansible-collections/openvswitch.openvswitch already ? | 18:21 |
| jamesdenton | indeed! this is my PR: https://github.com/ansible-collections/openvswitch.openvswitch/pull/58 | 18:22 |
| jamesdenton | but i was hoping to be able to test locally before pushing changes, just cant quite get the right local env | 18:22 |
| dmsimard | jamesdenton: if it's integration tests (aka playbooks/tasks) I've used https://docs.ansible.com/ansible/latest/dev_guide/testing_integration.html#running-integration-tests in a past life | 18:23 |
| dmsimard | but I haven't tried since the split out to collections | 18:23 |
| dmsimard | the way it'd work is that you would have "targets" here https://github.com/ansible/ansible/tree/devel/test/integration/targets and then you could run tests for a particular target | 18:24 |
| jamesdenton | ok, i can give that a shot. i was trying just unit tests yesterday and running into dependency issues. | 18:24 |
| jamesdenton | https://docs.ansible.com/ansible/latest/dev_guide/developing_collections.html#unit-tests | 18:24 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum stable/ussuri: Add deployment of keystone_auth_default_policy https://review.opendev.org/759471 | 18:24 |
| jamesdenton | specifically, http://paste.openstack.org/show/799665/ | 18:25 |
| dmsimard | yeah I'm not familiar with the unit tests but if you do a good amount of integration tests that might be good enough | 18:25 |
| jamesdenton | ok, sure, i can give that a go. seems like the existing example is fairly basic: https://github.com/ansible-collections/openvswitch.openvswitch/blob/main/tests/integration/targets/openvswitch_bridge/tests/basic.yaml | 18:26 |
| jamesdenton | but i may have to create some kind of dummy port, since a port would be required for a bond | 18:26 |
| jamesdenton | s/port/interface | 18:26 |
| dmsimard | the tasks run inside the container so as long as you can do that there it should be fine | 18:27 |
| jamesdenton | cool. we'll see :D | 18:27 |
| dmsimard | wish I could help more but I at least wanted to point you in the right direction, I'm far from an authority in #ansible-network :p | 18:28 |
| *** arxcruz has quit IRC | 18:28 | |
| jamesdenton | well, i appreciate the direction. i remember losing interest in this last year due to the... hurdles... | 18:28 |
| jamesdenton | but the collection breakout helps there | 18:28 |
| dmsimard | feel free to poke me about hurdles, I'm actually starting friday in the community engineering team :) | 18:29 |
| dmsimard | noonedeadpunk, jrosser: ^ that goes for you too <3 | 18:30 |
| jamesdenton | oh sweet! well, the hurdles these days are self-imposed. i'm more familiar with the openstack contribution workflow and everything else is foreign. i think last year, just trying to squeeze a PR in the devel branch was tough. | 18:30 |
| *** gyee has joined #openstack-ansible | 18:32 | |
| dmsimard | oh yeah, I prefer the openstack contribution workflow to github too :p | 18:33 |
| noonedeadpunk | dmsimard: what dissapointed me in openvswitch collection was that it's impossible to install it from git because of galaxy file but pabelanger aid it's "ok"?) | 18:52 |
| dmsimard | how long ago was that ? I know that galaxy didn't support installing collections from git at first but I think it should be good now | 18:53 |
| *** jamesdenton has quit IRC | 18:54 | |
| noonedeadpunk | It as during ansiblefest | 18:54 |
| dmsimard | like the one from a couple weeks ago ? | 18:54 |
| noonedeadpunk | yep | 18:54 |
| dmsimard | hmmm, should work IMO but let me hunt that down | 18:55 |
| dmsimard | noonedeadpunk: https://docs.ansible.com/ansible/devel/user_guide/collections_using.html#installing-a-collection-from-a-git-repository | 18:56 |
| noonedeadpunk | dmsimard: it was specificly openvswitch collection | 18:56 |
| dmsimard | oh yeah ? | 18:57 |
| noonedeadpunk | and it was because of missing version info in galaxy.yml | 18:57 |
| noonedeadpunk | and everyone was aware and was going to do nothing as far as I got... | 18:57 |
| dmsimard | :/ | 18:57 |
| noonedeadpunk | it needs like https://github.com/openstack/ansible-collections-openstack/blob/master/galaxy.yml#L36 | 18:58 |
| dmsimard | yeah that doesn't sound right | 18:58 |
| noonedeadpunk | well, let me rephrase, folks pointed that ansible-galaxy is just stupid as it should get version out from git tags | 18:59 |
| dmsimard | https://galaxy.ansible.com/openvswitch/openvswitch is parsed as 1.0.6 but I'm not sure where it comes from | 18:59 |
| dmsimard | btw I set up a hound instance like the one we have for openstack https://codesearch.recordsansible.org/ | 19:00 |
| noonedeadpunk | iirc it does zuul or smth like that, so you can specifily push version when publishing | 19:00 |
| noonedeadpunk | but it makes impossible to install from git.... | 19:00 |
| noonedeadpunk | oh! and where does it search? only ara? | 19:01 |
| *** nurdie has joined #openstack-ansible | 19:01 | |
| noonedeadpunk | oh, collections! | 19:01 |
| dmsimard | no, across all the repos in the ansible, ansible-community and ansible-collections github organizations | 19:01 |
| noonedeadpunk | that is neat | 19:01 |
| spotz | dmsimard! | 19:01 |
| dmsimard | spotz: o/ | 19:01 |
| noonedeadpunk | that is awesome! | 19:01 |
| dmsimard | noonedeadpunk: I created https://github.com/ansible-collections/openvswitch.openvswitch/issues/59 | 19:06 |
| noonedeadpunk | oh, thanks) | 19:07 |
| noonedeadpunk | I guess pabelanger is part of the team as well?:) | 19:07 |
| dmsimard | i don't know for sure which team he is in but he's part of ansible, yes | 19:08 |
| *** arxcruz has joined #openstack-ansible | 19:28 | |
| *** andrewbonney has quit IRC | 19:37 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Bump SHAs for master https://review.opendev.org/755973 | 19:38 |
| *** ThiagoCMC has joined #openstack-ansible | 19:50 | |
| *** cloudxtiny has quit IRC | 19:50 | |
| ThiagoCMC | Hey guys, the following TASK is failing: "lxc_hosts : Wait for base image download", aria2c is unable to download the ubuntu-base-20.04-base-amd64.tar.gz image, but the machine can "ping google.com" and "apt update" works as well, any idea? | 19:51 |
| ThiagoCMC | aria2c logs shows only: "(ERR):error occurred." | 19:51 |
| ThiagoCMC | I tried to run: "aria2c --max-connection-per-server=4 --allow-overwrite=true --dir=/tmp --out=ubuntu-base-20.04-base-amd64.tar.gz --check-certificate=true --follow-metalink=mem --on-download-complete=/usr/local/bin/aria2c-download-helper.sh http://cdimage.ubuntu.com/ubuntu-base/releases/20.04/release/ubuntu-base-20.04-base-amd64.tar.gz" manually... It says: "errorCode=3 Resource not found" | 19:52 |
| ThiagoCMC | checking more lol | 19:52 |
| ThiagoCMC | http://cdimage.ubuntu.com/ubuntu-base/releases/20.04/release/ubuntu-base-20.04-base-amd64.tar.gz is 404! O_O | 19:52 |
| ThiagoCMC | Whaaat | 19:53 |
| ThiagoCMC | Oh, the file is "20.04.1" now... Not just "20.04"... :-/ | 19:53 |
| ThiagoCMC | I'm on branch "stable/ussuri", BTW. | 19:55 |
| *** cshen has quit IRC | 19:56 | |
| ThiagoCMC | Ok, cool... I've updated: roles/lxc_hosts/vars/ubuntu-20.04.yml:_lxc_hosts_container_image_url with the correct file. Sorry about the buzz! | 19:58 |
| noonedeadpunk | ThiagoCMC: yeah well known thing | 20:07 |
| noonedeadpunk | already fixed in stable but not tagged yet | 20:07 |
| noonedeadpunk | pretty sure we will have fixed version during this week | 20:08 |
| ThiagoCMC | Okdok! | 20:08 |
| noonedeadpunk | ThiagoCMC: http://lists.openstack.org/pipermail/openstack-discuss/2020-October/018280.html | 20:08 |
| noonedeadpunk | you can find workaround in ML | 20:08 |
| noonedeadpunk | but yeah you did right:) | 20:09 |
| ThiagoCMC | Hmmm... But something might be missing... Now, the TASK: "lxc_container_create : Create container (machinectl)" is the one failing... :-( | 20:10 |
| ThiagoCMC | Not sure but I didn't do the step 2 from that e-mail thread. | 20:10 |
| ThiagoCMC | I have "lxc_container_backing_store: machinectl" - my /var/lib/machines is under a btrfs filesystem (I did this back with Train or Stein and Ubuntu 18.04) and it worked. | 20:11 |
| *** cshen has joined #openstack-ansible | 20:13 | |
| ThiagoCMC | I'm seriously thinking about deploying OSA on top of LXD containers! Using "is_metal" everywhere! This way, OSA will "think" that it's bare-metal but, it's not. LOL | 20:15 |
| ThiagoCMC | Then, I'll avoid letting OSA to manage containers, downloads and etc... | 20:15 |
| ThiagoCMC | LXD is so cool! | 20:15 |
| ThiagoCMC | I wonder why OSA doesn't support it...? | 20:15 |
| *** jamesdenton has joined #openstack-ansible | 20:17 | |
| ThiagoCMC | Can someone help me understand why the TASK "lxc_container_create : Create container (machinectl)" is failing? Logs: http://paste.openstack.org/show/799669/ | 20:17 |
| ThiagoCMC | OMG! https://docs.openstack.org/releasenotes/openstack-ansible/ussuri.html: "To provide compatibility with Centos-8 the LXC cache preparation has been greatly simplified to remove the requirement for machinectl and btrfs, which is a combination not available on Centos-8. This has the side effect of machinectl no longer being a supported backing store for LXC." | 20:19 |
| ThiagoCMC | Noooo!!! | 20:19 |
| ThiagoCMC | Using machinectl was the fastest way of deploying OSA! | 20:20 |
| ThiagoCMC | This is bad, btrfs works just fine on Ubuntu... :-@ | 20:21 |
| *** tosky has joined #openstack-ansible | 20:27 | |
| ThiagoCMC | I'm very disappointed that OSA dropped machineclt only because CentOS 8 doesn't have it! I would rather drop CentOS support entirely. LOL :-( | 20:30 |
| *** gshippey has quit IRC | 20:31 | |
| *** frenzy_friday has quit IRC | 20:46 | |
| spatel | jamesdenton: hey, i moved on from that permission error | 20:54 |
| jamesdenton | oh? what was it? | 20:54 |
| spatel | i found it was reading user_id permission from somewhere else, (this is centos8) | 20:55 |
| spatel | as soon as i gave it root permission, it fixed that issue and service started successfully | 20:55 |
| spatel | now i am in last stage, attaching dpdk to br-provider bridge but giving me this error | 20:55 |
| spatel | http://paste.openstack.org/show/799671/ | 20:56 |
| spatel | I am totally stuck here | 20:56 |
| spatel | some folks saying hardware issue or some saying firmware version | 20:57 |
| spatel | did you ever see that issue? | 20:57 |
| spatel | doing some firmware upgrade to see if it go away but if this is dead end then i can't use dpdk :( | 21:00 |
| jamesdenton | what chassis? | 21:09 |
| spatel | HP C7000 blade center with ProLiant BL460c Gen9 | 21:18 |
| spatel | jamesdenton: ^^ | 21:18 |
| jamesdenton | can you check dmesg and see if there's anything relevant? | 21:19 |
| -spatel- [root@compute-2 ~]# dmesg | grep fail | 21:20 | |
| -spatel- [ 1.320675] pci 0000:06:00.1: BAR 6: failed to assign [mem size 0x00080000 pref] | 21:20 | |
| spatel | let me look for more details | 21:21 |
| spatel | my ovs-vsctl show showing datapath_type: netdev | 21:21 |
| spatel | does that means anything or its normal | 21:22 |
| jamesdenton | thats expected | 21:23 |
| *** rfolco has quit IRC | 21:24 | |
| *** rfolco has joined #openstack-ansible | 21:24 | |
| spatel | k | 21:24 |
| *** ChanServ has quit IRC | 21:26 | |
| *** rfolco has quit IRC | 21:29 | |
| *** ChanServ has joined #openstack-ansible | 21:32 | |
| *** tepper.freenode.net sets mode: +o ChanServ | 21:32 | |
| spatel | jamesdenton: read this, its interesting - http://mails.dpdk.org/archives/users/2017-April/001811.html | 21:39 |
| *** cshen has quit IRC | 21:42 | |
| *** cshen has joined #openstack-ansible | 21:48 | |
| *** elduderino80 has joined #openstack-ansible | 21:48 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum stable/train: Add deployment of keystone_auth_default_policy https://review.opendev.org/759472 | 22:01 |
| *** aedc has joined #openstack-ansible | 22:03 | |
| *** aedc has quit IRC | 22:04 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_neutron stable/ussuri: Enable notifications when Designate is enabled https://review.opendev.org/761240 | 22:31 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_neutron stable/train: Enable notifications when Designate is enabled https://review.opendev.org/761241 | 22:31 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_neutron stable/stein: Enable notifications when Designate is enabled https://review.opendev.org/761242 | 22:32 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_neutron master: Enable notifications when Designate is enabled https://review.opendev.org/757937 | 22:33 |
| jrosser | ThiagoCMC: i started patches to deploy with LXD https://review.opendev.org/738599 and https://review.opendev.org/738657 but it need someone to commit effort if those are of interest | 22:33 |
| jrosser | ThiagoCMC: you can switch to ZFS from btrfs with the current code and get the same copy-on-write performance during container creation | 22:35 |
| openstackgerrit | Merged openstack/openstack-ansible-os_manila master: Do not symlink ceph libraries for distro path https://review.opendev.org/760585 | 22:35 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: Move nspawn to unmaintained state https://review.opendev.org/760954 | 22:39 |
| *** spatel has quit IRC | 22:41 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/ussuri: Bump SHAs for stable/ussuri https://review.opendev.org/760731 | 22:43 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-tests stable/stein: Add requirements_git_install_branch to test-vars https://review.opendev.org/732522 | 22:47 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/train: Bump SHAs for stable/train https://review.opendev.org/760728 | 22:59 |
| *** tosky has quit IRC | 23:01 | |
| ThiagoCMC | jrosser, that, is, awesome!!! You can count on me to test it! | 23:08 |
| ThiagoCMC | Is there docs to use ZFS? | 23:08 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible stable/stein: Bump SHAs for stable/stein https://review.opendev.org/760730 | 23:09 |
| ThiagoCMC | So, to benefit from ZFS, I can just set 'lxc_container_zfs_root_name: "pool/lxc"' and prepare it in advance (before run the OSA playbooks)? | 23:15 |
| *** mmethot has quit IRC | 23:18 | |
| *** yann-kaelig has quit IRC | 23:27 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: [doc] Mention uwsgi overrides in doc https://review.opendev.org/761206 | 23:32 |
| *** mmethot has joined #openstack-ansible | 23:34 | |
| *** cloudxtiny has joined #openstack-ansible | 23:36 | |
| cloudxtiny | hello. for my bonded interface I am using tlb mode. Is it better to use active backup? | 23:37 |
| cloudxtiny | even though all my infra servers are configured this way, only containers on 1 of the infra servers is "pingable" on br-mgmt | 23:38 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!