openstackgerrit | Ghanshyam Mann proposed openstack/python-octaviaclient master: Fix hacking min version to 3.0.1 https://review.opendev.org/727554 | 01:03 |
---|---|---|
openstackgerrit | Ghanshyam Mann proposed openstack/octavia-lib master: Fix hacking min version to 3.0.1 https://review.opendev.org/727555 | 01:03 |
openstackgerrit | Ghanshyam Mann proposed openstack/octavia-tempest-plugin master: Fix hacking min version to 3.0.1 https://review.opendev.org/727556 | 01:03 |
*** livelace has quit IRC | 01:18 | |
*** wuchunyang has joined #openstack-lbaas | 01:21 | |
*** sapd1 has joined #openstack-lbaas | 01:31 | |
rm_work | damn, gates are really backed up | 01:42 |
johnsom | I'm sure that flake8 change caused carnage | 01:51 |
*** sapd1_x has joined #openstack-lbaas | 02:04 | |
*** archiephan has joined #openstack-lbaas | 02:23 | |
rm_work | yep | 02:24 |
*** kevinz has joined #openstack-lbaas | 02:35 | |
*** wuchunyang has quit IRC | 02:40 | |
*** psachin has joined #openstack-lbaas | 03:38 | |
*** wuchunyang has joined #openstack-lbaas | 03:50 | |
*** wuchunyang has quit IRC | 04:02 | |
openstackgerrit | Jie Li proposed openstack/octavia master: add the verify for the session https://review.opendev.org/726567 | 04:08 |
*** gcheresh_ has joined #openstack-lbaas | 04:10 | |
openstackgerrit | Jie Li proposed openstack/octavia master: add the verify for the session https://review.opendev.org/726567 | 04:10 |
*** dmellado has quit IRC | 04:45 | |
*** psachin has quit IRC | 04:56 | |
*** wuchunyang has joined #openstack-lbaas | 05:00 | |
*** gcheresh_ has quit IRC | 06:49 | |
*** vishalmanchanda has joined #openstack-lbaas | 07:02 | |
*** gcheresh_ has joined #openstack-lbaas | 07:24 | |
*** rpittau|afk is now known as rpittau | 07:29 | |
*** gcheresh has joined #openstack-lbaas | 07:29 | |
*** gcheresh_ has quit IRC | 07:29 | |
*** dmellado has joined #openstack-lbaas | 07:37 | |
*** maciejjozefczyk has joined #openstack-lbaas | 07:38 | |
*** gcheresh_ has joined #openstack-lbaas | 07:39 | |
*** gcheresh has quit IRC | 07:39 | |
*** gcheresh_ has quit IRC | 07:41 | |
*** gcheresh has joined #openstack-lbaas | 07:41 | |
*** livelace has joined #openstack-lbaas | 08:12 | |
*** salmankhan has joined #openstack-lbaas | 08:30 | |
*** wuchunyang has quit IRC | 08:35 | |
*** livelace has quit IRC | 08:50 | |
*** riuzen has joined #openstack-lbaas | 08:56 | |
*** salmankhan1 has joined #openstack-lbaas | 08:57 | |
*** livelace has joined #openstack-lbaas | 08:57 | |
*** salmankhan has quit IRC | 08:59 | |
*** salmankhan1 is now known as salmankhan | 08:59 | |
*** livelace has quit IRC | 09:02 | |
*** livelace has joined #openstack-lbaas | 09:08 | |
*** livelace has quit IRC | 09:13 | |
*** livelace has joined #openstack-lbaas | 09:34 | |
*** wuchunyang has joined #openstack-lbaas | 10:06 | |
*** wuchunyang has quit IRC | 10:10 | |
*** wuchunyang has joined #openstack-lbaas | 10:39 | |
*** wuchunyang has quit IRC | 11:00 | |
*** rpittau is now known as rpittau|bbl | 11:04 | |
*** salmankhan1 has joined #openstack-lbaas | 11:28 | |
*** salmankhan has quit IRC | 11:28 | |
*** salmankhan1 is now known as salmankhan | 11:28 | |
*** ccamposr has quit IRC | 11:37 | |
*** ccamposr has joined #openstack-lbaas | 11:37 | |
*** yamamoto has joined #openstack-lbaas | 11:43 | |
*** yamamoto has quit IRC | 11:43 | |
*** ccamposr__ has joined #openstack-lbaas | 11:44 | |
*** ccamposr has quit IRC | 11:47 | |
*** servagem has joined #openstack-lbaas | 11:53 | |
*** rpittau|bbl is now known as rpittau | 12:19 | |
*** livelace has quit IRC | 12:23 | |
*** tkajinam has quit IRC | 12:24 | |
*** ccamposr has joined #openstack-lbaas | 12:28 | |
*** livelace has joined #openstack-lbaas | 12:29 | |
*** ccamposr__ has quit IRC | 12:32 | |
*** wuchunyang has joined #openstack-lbaas | 12:52 | |
*** ianychoi has quit IRC | 12:55 | |
*** riuzen has quit IRC | 13:11 | |
*** sapd1_x has quit IRC | 13:31 | |
*** TrevorV has joined #openstack-lbaas | 13:42 | |
*** archiephan has quit IRC | 13:44 | |
*** ataraday_ has joined #openstack-lbaas | 13:53 | |
*** jamesdenton has quit IRC | 14:03 | |
*** jamesdenton has joined #openstack-lbaas | 14:11 | |
*** wuchunyang has quit IRC | 14:41 | |
*** livelace has quit IRC | 14:45 | |
*** gcheresh has quit IRC | 15:09 | |
*** jamesdenton has quit IRC | 15:42 | |
*** jamesdenton has joined #openstack-lbaas | 15:46 | |
*** sapd1_x has joined #openstack-lbaas | 15:47 | |
*** ccamposr has quit IRC | 15:54 | |
*** ccamposr has joined #openstack-lbaas | 15:54 | |
johnsom | #startmeeting Octavia | 16:00 |
openstack | Meeting started Wed May 13 16:00:01 2020 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
*** openstack changes topic to " (Meeting topic: Octavia)" | 16:00 | |
openstack | The meeting name has been set to 'octavia' | 16:00 |
johnsom | #chair rm_work | 16:00 |
openstack | Current chairs: johnsom rm_work | 16:00 |
cgoncalves | hi | 16:00 |
ataraday_ | hi | 16:00 |
johnsom | Hi everyone! | 16:00 |
haleyb | hi | 16:00 |
johnsom | #topic Announcements | 16:01 |
*** openstack changes topic to "Announcements (Meeting topic: Octavia)" | 16:01 | |
johnsom | Ussuri is released! | 16:01 |
johnsom | #link https://www.openstack.org/software/ussuri/ | 16:01 |
johnsom | If you didn't attend the highlights meeting this morning, the release is out. | 16:01 |
johnsom | Thank you all for your contributions! | 16:02 |
johnsom | Octavia got some attention this cycle in the release announcements, etc. so good stuff. | 16:02 |
johnsom | Both AZs and TLS ciphers are in the screenshots section | 16:02 |
cgoncalves | well done, team! | 16:03 |
johnsom | Along those lines, thank you to rm_work for his leadership during Ussuri! | 16:03 |
johnsom | This is his last week as PTL and then you are stuck with me. grin | 16:03 |
cgoncalves | ... again | 16:04 |
johnsom | lol | 16:04 |
cgoncalves | thank you, rm_work! | 16:04 |
*** maciejjozefczyk has quit IRC | 16:04 | |
johnsom | As the release is out, we are officially open for Victoria patch merge. (Though I think we have been for a bit anyway) | 16:04 |
johnsom | Also a friendly reminder, PTG registration is open: | 16:05 |
johnsom | #link https://www.openstack.org/ptg/ | 16:05 |
johnsom | and we have an etherpad: | 16:05 |
johnsom | #link https://etherpad.opendev.org/p/octavia-virtual-V-ptg | 16:05 |
johnsom | Please include any topics you would like to discuss during our PTG sessions. | 16:05 |
johnsom | Any other announcements this week? | 16:06 |
johnsom | #topic Brief progress reports / bugs needing review | 16:07 |
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)" | 16:07 | |
johnsom | Ok, other than some reviews and working on the content the foundation requested, I have been focusing on an internal project this week. | 16:07 |
ataraday_ | I pushed small patch for an issue I discovered on our internal cloud | 16:08 |
ataraday_ | #link https://review.opendev.org/#/c/726084/ | 16:08 |
ataraday_ | some reviews appreciated :) | 16:09 |
johnsom | I should also mention I am giving a short talk at the OpenStack meetings today covering our Ussuri highlights. So, shout outs for those features. | 16:09 |
cgoncalves | same content as meeting ~2 hours ago, right? | 16:10 |
johnsom | Hmm, interesting. Even an API get is failing... | 16:10 |
johnsom | Yes, same exact content, just a separate time (zone) | 16:10 |
johnsom | ataraday_ Looks good. My only comment would be to maybe tie the retry counts to configuration file settings. | 16:11 |
johnsom | We have got feedback in the past that some people want things to retry forever. | 16:11 |
johnsom | Personally, I'm getting a bit concerned about the number of these "retry" values we have defined in the configuration file. | 16:12 |
ataraday_ | johnsom, sure, will add it | 16:12 |
ataraday_ | yeah, to many reties... | 16:12 |
johnsom | I would be interested to hear if people have ideas/thoughts on how to improve that situation or if it is ok as it is. | 16:12 |
johnsom | Maybe we can consolidate them somehow. | 16:12 |
johnsom | I know I have added a bunch of them, and I'm just not sure if it's useful for tuning, or just too much. | 16:13 |
ataraday_ | ideally we want to have all retries configurable? | 16:13 |
johnsom | ataraday_ Historically that has been our position. | 16:14 |
johnsom | Mostly because of the very wide performance difference between the nested virtualization the zuul gates use vs. the production situations. | 16:14 |
johnsom | But I would like to have a discussion about it. Is it too much, is it good, etc. | 16:15 |
*** jamesdenton has quit IRC | 16:16 | |
johnsom | No other comments? | 16:18 |
haleyb | I have been trying to get the gates functional, https://review.opendev.org/#/c/726972/ and https://review.opendev.org/#/c/727280/ (ussuri, needs further work) | 16:18 |
ataraday_ | The question with retries would also be - should be a big number by default or not, I guess a lot can be disscussed.. | 16:18 |
johnsom | Yes, I agree. I think it would be good for the team to have that discussion. Maybe this is a good PTG topic | 16:19 |
johnsom | haleyb Thank you. | 16:19 |
johnsom | I did have a question I haven't posted for one of your patches yet. If we fix those issues, why do we still need to pin it? | 16:20 |
haleyb | johnsom: well, for stable we're only fixing until the next release of flake8/pycodestyle/etc since they're unbounded | 16:21 |
johnsom | Ugh, looks like we are caught in the "zuul is really busy, so jobs randomly timeout" cycle there | 16:21 |
*** rpittau is now known as rpittau|afk | 16:21 | |
haleyb | yeah, pile-up in zuul | 16:22 |
johnsom | This is a cap for Ussuri right? | 16:22 |
johnsom | #link https://review.opendev.org/#/c/727280/ | 16:23 |
haleyb | yes, but i'm assuming other branches are broken too, at least they are in neutron back to rocky | 16:23 |
*** ramishra has quit IRC | 16:23 | |
johnsom | Ok, so your plan was to fix master, cap the stable branches? | 16:24 |
haleyb | johnsom: yes, just based on how large the patch was | 16:24 |
johnsom | I think requirements has talked about capping the linters for stable branches before, but it tends to get messy | 16:24 |
johnsom | Yeah, though it's pretty low risk assuming it's just the single letter variable name issue | 16:25 |
haleyb | who doesn't like the letter 'l' ? :) | 16:25 |
johnsom | I don't | 16:26 |
johnsom | lol | 16:26 |
johnsom | Funny story actually of being a wise guy in college and getting stuck in a 101 class due to requirements. I crashed the compiler because I variables were too descriptive. | 16:26 |
johnsom | Ah the good old days. | 16:27 |
haleyb | i might need to cherry-pick if that cap doesn't work | 16:27 |
johnsom | Ok, I am not opposed to that | 16:27 |
* cgoncalves challenges johnsom to not use letter 'l' until end of the meeting | 16:27 | |
johnsom | Thanks for working on it! | 16:27 |
haleyb | we all learned in school to use i, j, k for looping, guess that's all we're allowed | 16:27 |
johnsom | Ok, any other topics today? | 16:28 |
* johnsom notes, none of that character in the sentence.... | 16:28 | |
johnsom | #topic Open Discussion | 16:29 |
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)" | 16:29 | |
johnsom | opps, forgot the topic change | 16:29 |
johnsom | Ok, thank you everyone! Happy Ussuri re1ease day! | 16:30 |
cgoncalves | gotcha! re*l*ease! | 16:31 |
cgoncalves | thanks! | 16:31 |
haleyb | he used a 1 to be snarky | 16:31 |
johnsom | That was #1 | 16:31 |
johnsom | grin | 16:31 |
johnsom | #endmeeting | 16:31 |
*** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews" | 16:31 | |
openstack | Meeting ended Wed May 13 16:31:25 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:31 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-05-13-16.00.html | 16:31 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-05-13-16.00.txt | 16:31 |
openstack | Log: http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-05-13-16.00.log.html | 16:31 |
cgoncalves | d'oh!! | 16:31 |
cgoncalves | well done sir | 16:31 |
johnsom | cgoncalves I think you just proved the point of that flake8 change.... | 16:31 |
cgoncalves | hah! | 16:32 |
haleyb | johnsom: seems stable/train is passing pep8 locally, but stein is a "bloody mess", hundreds of new failures with the new flake8 | 16:36 |
johnsom | lol, joy. Do you have a plan or do more folks need to get involved? | 16:37 |
haleyb | johnsom: i'll see if capping works, i'm already in this dep | 16:38 |
*** jamesdenton has joined #openstack-lbaas | 16:38 | |
johnsom | Thank you | 16:39 |
*** sapd1_x has quit IRC | 16:40 | |
*** sapd__x has joined #openstack-lbaas | 16:40 | |
openstackgerrit | Brian Haley proposed openstack/octavia stable/stein: DNM - pep8 test for stable/stein https://review.opendev.org/727833 | 16:51 |
openstackgerrit | Brian Haley proposed openstack/octavia stable/train: DNM - pep8 test for stable/train https://review.opendev.org/727834 | 16:52 |
*** ataraday_ has quit IRC | 17:12 | |
*** jamesdenton has quit IRC | 17:15 | |
*** salmankhan has quit IRC | 17:22 | |
*** gcheresh has joined #openstack-lbaas | 17:48 | |
*** jamesdenton has joined #openstack-lbaas | 17:57 | |
*** jamesdenton has quit IRC | 18:04 | |
*** jamesdenton has joined #openstack-lbaas | 18:09 | |
*** sapd__x has quit IRC | 18:31 | |
*** jamesdenton has quit IRC | 18:53 | |
*** jamesdenton has joined #openstack-lbaas | 18:57 | |
*** maciejjozefczyk has joined #openstack-lbaas | 19:08 | |
*** livelace has joined #openstack-lbaas | 19:09 | |
*** gcheresh has quit IRC | 19:09 | |
*** maciejjozefczyk has quit IRC | 19:25 | |
rm_work | Yeah I think capping would be fine for stable branches | 20:12 |
rm_work | Also, thanks johnsom for taking back over, I am so glad to not have that PTL responsibility anymore (especially given how much I've been forced downstream recently) | 20:13 |
*** jamesdenton has quit IRC | 20:16 | |
openstackgerrit | Brian Haley proposed openstack/octavia master: Remove deprecated data model classes https://review.opendev.org/727866 | 20:17 |
openstackgerrit | Brian Haley proposed openstack/octavia master: Remove deprecated data model classes https://review.opendev.org/727866 | 20:20 |
openstackgerrit | Brian Haley proposed openstack/octavia master: Remove deprecated data model classes https://review.opendev.org/727866 | 20:28 |
*** jamesdenton has joined #openstack-lbaas | 20:30 | |
*** vishalmanchanda has quit IRC | 20:31 | |
openstackgerrit | Brian Haley proposed openstack/octavia master: Remove deprecated DriverLibrary class https://review.opendev.org/727871 | 20:43 |
openstackgerrit | Brian Haley proposed openstack/octavia master: Remove deprecated ProviderDriver class https://review.opendev.org/727872 | 20:43 |
*** servagem has quit IRC | 20:46 | |
*** livelace has quit IRC | 21:11 | |
*** livelace has joined #openstack-lbaas | 21:17 | |
*** livelace has quit IRC | 21:18 | |
*** jamesdenton has quit IRC | 21:30 | |
KeithMnemonic | johnsom: do you have a fwe minutes to help me understand an error i am seeing with plug_network ? | 21:48 |
KeithMnemonic | i can manually do all of the tests in the octavia-tempest-plugin and those seem to be fine. i do this for example in a demo tenant | 21:48 |
KeithMnemonic | when i run the test though i get an odd error when it tries to add the members about "cannot find eth2" | 21:49 |
KeithMnemonic | i am not quite finding where it is getting this from | 21:49 |
johnsom | KeithMnemonic Sure, I can use a break from spreadsheet engineering | 21:49 |
KeithMnemonic | Amphora agent returned unexpected result code 500 with response {u'message': u'Error plugging network', u'details': u'eth2 no-device\n'} | 21:50 |
KeithMnemonic | so i am looking here https://github.com/openstack/octavia/tree/stable/rocky/octavia/amphorae/backends/agent/api_server | 21:50 |
KeithMnemonic | so i think this is the 500 https://github.com/openstack/octavia/blob/stable/rocky/octavia/amphorae/backends/agent/api_server/server.py#L35 | 21:50 |
johnsom | Yeah, that is the catch-all exception wrapper. So, do you know if nova was able to actually attach the neutron interface to the VM? | 21:51 |
KeithMnemonic | and i can create an lb/listener/pool/member fine from the cli | 21:51 |
KeithMnemonic | i do not think it did, but for my manual tests that have active members, if i ssh to the amphora instance i only see eth1 | 21:52 |
johnsom | Yeah, you should only see eth1, which is the lb-mgmt-net. The tenant traffic interfaces are inside the network namespace. | 21:53 |
johnsom | "sudo ip netns exec amphora-haproxy ip a" would show you the tenant interfaces | 21:53 |
johnsom | But, I don't think the problem is in the amphora. It's likely a mis-configuration in tempest/nova/neutron somewhere. I would start with the worker log file, find where tempest is making the call to create the member, and step through the log to see why nova wasn't able to attach the member network port. | 21:55 |
KeithMnemonic | so i have active/standby. i check both instances and the netns has the correct info but only eth1 | 21:55 |
johnsom | Also, don't get confused by "eth1, eth2, eth3" etc. We tightly control those names, and rename interfaces as part of the normal process. | 21:55 |
johnsom | Ok, so you didn't specify a subnet when you created the member or it resides on the same subnet as the VIP | 21:56 |
johnsom | In which case there is only one network port needed. | 21:56 |
KeithMnemonic | so for my manual test, i did like tempest , 3 networks | 21:56 |
KeithMnemonic | one for inst1, one for inst2, and a third for lb1 | 21:57 |
johnsom | And you added "--subnet <uuid>" to the member create command line? | 21:57 |
johnsom | sorry, it is "--subnet-id <uuid>" | 21:58 |
KeithMnemonic | i used --vip-subnet-id | 21:58 |
KeithMnemonic | so maybe my manual test is wrong | 21:58 |
johnsom | No, that is only valid on load balancer create, not member create | 21:58 |
johnsom | You should get an error if you used --vip-subnet-id on a member create call | 21:59 |
KeithMnemonic | of sorry | 21:59 |
KeithMnemonic | for member i gave the address of the instance | 21:59 |
KeithMnemonic | openstack loadbalancer member create --protocol-port 80 --name demo-member1 --address 172.22.0.130 | 21:59 |
johnsom | If that was all, you told Octavia to only use the VIP network. So, yeah, you didn't configure those to be on the tenant networks. | 22:00 |
KeithMnemonic | ah ok so i need to also add --vip-subnet-id | 22:00 |
johnsom | It created a load balancer that expects the 172.22.0.130 IP address to be reachable from the VIP network. | 22:00 |
johnsom | Yes, if it is on a network different from the VIP. | 22:00 |
KeithMnemonic | ok so that is how my manual test and the tempest differ, let me kill the member and recreate it | 22:01 |
johnsom | openstack loadbalancer member create --subnet-id tenant-2-subnet --address $webserver2ip --protocol-port 80 pool1 | 22:01 |
johnsom | That is an example from my test script | 22:01 |
KeithMnemonic | ah ok so i need to also add --vip-subnet-id | 22:02 |
KeithMnemonic | ok let me try that now | 22:02 |
KeithMnemonic | ops | 22:03 |
KeithMnemonic | that was a typo | 22:03 |
KeithMnemonic | i know what i need to do | 22:03 |
KeithMnemonic | ok interesting | 22:06 |
KeithMnemonic | i get the same error | 22:06 |
KeithMnemonic | but i did the eth2 for a split second | 22:06 |
KeithMnemonic | amphora-8cf40858-3e6d-4ab8-9a12-f16b1b547564:~ # sudo ip netns exec amphora-haproxy ip a | 22:07 |
KeithMnemonic | ... | 22:07 |
KeithMnemonic | 4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> | 22:07 |
johnsom | Did it get an address? | 22:07 |
*** spatel has joined #openstack-lbaas | 22:08 | |
KeithMnemonic | nope, it disappeared almost immediately | 22:08 |
KeithMnemonic | so it did not seem like it waited long if it was trying to get one | 22:08 |
johnsom | Yeah, if nova fails, we go straight into cleanup mode and start undoing stuff | 22:09 |
KeithMnemonic | ok let me boot a cirros and try and attach all of these nets and see what happens | 22:10 |
KeithMnemonic | is that a sufficient test, to have the cirros sit on the instance and loadbalancer nets, as well as octavia mgmt? | 22:10 |
johnsom | If you can get into the amphora, check the syslog for amphora agent records, it will probably just say the same thing as the controller worker captured. At that point you really need to go back the worker log where the original error was probably caught | 22:10 |
johnsom | No, probably not, much different animal | 22:11 |
johnsom | cirros doesn't really support more than one interface anyway | 22:11 |
KeithMnemonic | i have octavia in debug but am not really seeing much as the first error is right after the plug | 22:11 |
johnsom | You have to do a bunch of manual stuff to get them working | 22:12 |
*** spatel has quit IRC | 22:12 | |
KeithMnemonic | i could boot an amphora instance manually and test | 22:12 |
johnsom | Do you want to a good hangout to share the log with me? | 22:12 |
KeithMnemonic | sure | 22:13 |
KeithMnemonic | or zoom or .... | 22:13 |
*** spatel has joined #openstack-lbaas | 22:20 | |
*** TrevorV has quit IRC | 22:42 | |
*** spatel has quit IRC | 22:45 | |
*** tkajinam has joined #openstack-lbaas | 22:46 | |
*** ccamposr__ has joined #openstack-lbaas | 22:52 | |
*** ccamposr has quit IRC | 22:55 | |
*** hongbin has joined #openstack-lbaas | 23:05 | |
*** threestrands has joined #openstack-lbaas | 23:29 | |
*** rcernin has quit IRC | 23:31 | |
*** rcernin has joined #openstack-lbaas | 23:32 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!