| *** armax has quit IRC | 01:37 | |
| *** rcernin has quit IRC | 01:54 | |
| *** sapd__x has joined #openstack-lbaas | 02:13 | |
| *** armax has joined #openstack-lbaas | 02:32 | |
| *** armax has quit IRC | 02:35 | |
| *** rcernin has joined #openstack-lbaas | 02:38 | |
| *** rcernin has quit IRC | 02:43 | |
| *** rcernin has joined #openstack-lbaas | 03:05 | |
| *** rcernin has quit IRC | 03:12 | |
| *** rcernin has joined #openstack-lbaas | 03:12 | |
| *** psachin has joined #openstack-lbaas | 03:37 | |
| *** jamesden_ has quit IRC | 04:08 | |
| *** logan- has quit IRC | 04:08 | |
| *** f0o has quit IRC | 04:08 | |
| *** andy_ has quit IRC | 04:08 | |
| *** zigo has quit IRC | 04:08 | |
| *** sorrison has quit IRC | 04:08 | |
| *** f0o has joined #openstack-lbaas | 04:14 | |
| *** andy_ has joined #openstack-lbaas | 04:14 | |
| *** zigo has joined #openstack-lbaas | 04:14 | |
| *** sorrison has joined #openstack-lbaas | 04:14 | |
| *** wolsen has quit IRC | 04:20 | |
| *** sapd__x has quit IRC | 04:28 | |
| *** jamesden_ has joined #openstack-lbaas | 04:33 | |
| *** logan- has joined #openstack-lbaas | 04:33 | |
| *** jamesden_ has quit IRC | 04:36 | |
| *** logan- has quit IRC | 04:36 | |
| *** jamesden_ has joined #openstack-lbaas | 04:38 | |
| *** logan- has joined #openstack-lbaas | 04:38 | |
| *** vishalmanchanda has joined #openstack-lbaas | 04:41 | |
| *** jamesden_ has quit IRC | 04:54 | |
| *** logan- has quit IRC | 04:54 | |
| *** jamesden_ has joined #openstack-lbaas | 04:57 | |
| *** logan- has joined #openstack-lbaas | 04:57 | |
| *** sapd__x has joined #openstack-lbaas | 05:19 | |
| *** spatel has joined #openstack-lbaas | 05:29 | |
| *** sapd__x has quit IRC | 05:32 | |
| *** spatel has quit IRC | 05:33 | |
| *** wolsen has joined #openstack-lbaas | 05:39 | |
| *** gcheresh has joined #openstack-lbaas | 05:53 | |
| rm_work | johnsom: did we ever merge that delete function for amphora? | 06:30 |
|---|---|---|
| johnsom | I don’t know | 06:30 |
| johnsom | It was done | 06:31 |
| rm_work | digging through commits | 06:31 |
| rm_work | i don't have it in my client at least | 06:31 |
| *** psachin has quit IRC | 06:45 | |
| *** psachin has joined #openstack-lbaas | 07:01 | |
| *** ccamposr__ has joined #openstack-lbaas | 07:19 | |
| *** ccamposr has quit IRC | 07:21 | |
| gthiemonge | Hi, some functional tests have failed since we merged "Add alpn_protocols to the pool data model" in octavia-lib: https://9f9aee74b45263b2a9d8-795792c1f104e79962e44448ab55e3f1.ssl.cf1.rackcdn.com/767086/5/check/octavia-tox-functional-py37-tips/b34d9d1/testr_results.html | 07:35 |
| gthiemonge | we probably need https://review.opendev.org/c/openstack/octavia/+/752095 | 07:35 |
| *** psachin has quit IRC | 07:36 | |
| *** psachin has joined #openstack-lbaas | 07:38 | |
| *** sapd__x has joined #openstack-lbaas | 07:45 | |
| *** xgerman has quit IRC | 07:55 | |
| cgoncalves | Correct. And to merge that patch we need a new octavia-lib release -- https://review.opendev.org/c/openstack/releases/+/772380 | 08:09 |
| gthiemonge | cgoncalves: do you think we could propose a simple fix for the tests? (basically spliting your commit in 2 parts) it would be easier to get CR+2s | 08:13 |
| cgoncalves | gthiemonge, in the past where we faced similar situations as this one we did not split patches. how would you suggest splitting it in 2 here? | 08:16 |
| gthiemonge | it would depend on the fix (https://review.opendev.org/c/openstack/octavia/+/752095), we have to update it, it got V-1 | 08:19 |
| cgoncalves | weird, I can't see past CR votes in the change log section (comments) anymore. I'm pretty sure johnsom had +2 it. I see johnsom's comment entry but no vote/text. | 08:19 |
| gthiemonge | oh it needs the octavia-lib release | 08:19 |
| cgoncalves | pinged the Release team | 08:23 |
| cgoncalves | boom, approved. I'll update the ALPN patch shortly after octavia-lib 2.3.0 is released | 08:25 |
| *** rpittau|afk is now known as rpittau | 08:30 | |
| *** yamamoto has quit IRC | 08:33 | |
| gthiemonge | cgoncalves: thanks ;-) | 08:34 |
| *** sapd__x has quit IRC | 08:49 | |
| *** rcernin has quit IRC | 08:53 | |
| *** vishalmanchanda has quit IRC | 09:00 | |
| *** spatel has joined #openstack-lbaas | 09:01 | |
| openstackgerrit | Carlos Gonçalves proposed openstack/octavia master: Add ALPN support for TLS-enabled pools https://review.opendev.org/c/openstack/octavia/+/752095 | 09:02 |
| *** rcernin has joined #openstack-lbaas | 09:04 | |
| *** sapd__x has joined #openstack-lbaas | 09:05 | |
| *** spatel has quit IRC | 09:05 | |
| *** sapd__x has quit IRC | 09:09 | |
| *** eandersson has quit IRC | 09:11 | |
| *** eandersson has joined #openstack-lbaas | 09:12 | |
| *** yamamoto has joined #openstack-lbaas | 09:17 | |
| *** tkajinam has quit IRC | 09:22 | |
| *** yamamoto has quit IRC | 09:26 | |
| rm_work | johnsom: ah yes, so we did merge everything, but the client needs a new release :/ | 09:58 |
| rm_work | I don't know if I'll be able to wake up for the meeting in the morning... but if I can, I wanted to bring up possibly focusing quickly on cleaning up any pending client patches that are close to ready and cutting a release (seems there's only a couple, and I am reviewing some now) | 10:01 |
| rm_work | 2 in the pipe to merge, one needs a rebase I guess but isn't REALLY important | 10:08 |
| rm_work | oh, maybe no rebase | 10:09 |
| rm_work | this one is trivial but makes sense as we move to py3: https://review.opendev.org/c/openstack/python-octaviaclient/+/769042 | 10:09 |
| rm_work | needs one review | 10:09 |
| rm_work | tag support is a much bigger review, but will try to dive in | 10:10 |
| cgoncalves | got it | 10:10 |
| rm_work | have you tested the tags one? | 10:10 |
| rm_work | I guess I can pull it down and poke around | 10:10 |
| rm_work | crap I have been kinda out of it so long that I don't know where everything is in the new gerrit layout yet XD | 10:11 |
| cgoncalves | briefly weeks ago | 10:11 |
| rm_work | tested, seems good. added, showed, removed, filtered by tags on LB, will check the code to make sure all objects implemented it the same and then +W :) | 10:14 |
| rm_work | interesting that we can't remove a single tag tho, only clear all? | 10:15 |
| rm_work | hmm actually --no-tag isn't working | 10:16 |
| rm_work | gthiemonge: see comment on https://review.opendev.org/c/openstack/python-octaviaclient/+/762668 | 10:18 |
| gthiemonge | hmmmm | 10:19 |
| rm_work | oh nevermind | 10:20 |
| rm_work | I was using it wrong | 10:20 |
| rm_work | but I don't understand what --no-tag is for I guess | 10:20 |
| gthiemonge | well I don't even remember how to use it | 10:20 |
| rm_work | ah nm | 10:20 |
| rm_work | it still isn't working | 10:20 |
| rm_work | openstack loadbalancer unset --tag testing d5721c94-3042-47da-8601-3154287dc256 | 10:20 |
| rm_work | tag "testing" still present | 10:20 |
| rm_work | openstack loadbalancer unset --all-tag d5721c94-3042-47da-8601-3154287dc256 | 10:21 |
| rm_work | tag "testing" still present | 10:21 |
| gthiemonge | ok, I'm looking at it | 10:21 |
| rm_work | openstack loadbalancer set --no-tag d5721c94-3042-47da-8601-3154287dc256 | 10:21 |
| rm_work | tag "testing" still present (is this supposed to be the same as "unset --all-tag"? | 10:21 |
| rm_work | also, I'd prefer "all-tags" / "no-tags" :D | 10:22 |
| rm_work | plural makes more sense IMO? | 10:22 |
| rm_work | but i really don't understand the "no-tag" thing anyway | 10:22 |
| rm_work | i mean it's possible the bug is on the service side, let me debug to see what the client is doing | 10:23 |
| rm_work | hmm, can't tell, debug doesn't show me the body of the PUT it sends | 10:24 |
| openstackgerrit | Merged openstack/python-octaviaclient master: remove unicode from code https://review.opendev.org/c/openstack/python-octaviaclient/+/769042 | 10:24 |
| *** rcernin has quit IRC | 10:28 | |
| *** vishalmanchanda has joined #openstack-lbaas | 10:28 | |
| gthiemonge | set/unset --tag <tag> works here, but set/unset --all-tag/--no-tag doesn't | 10:31 |
| rm_work | hmmm | 10:33 |
| rm_work | i just ran unset on a tag three times in a row, did a show of the LB, and it's still there | 10:34 |
| gthiemonge | what are the params to unset the tags? {..., "tags": []} ? | 10:34 |
| rm_work | not sure | 10:34 |
| *** spatel has joined #openstack-lbaas | 10:49 | |
| *** spatel has quit IRC | 10:54 | |
| *** ianychoi_ has joined #openstack-lbaas | 11:09 | |
| gthiemonge | rm_work: there's probably something wrong there: https://opendev.org/openstack/octavia/src/branch/master/octavia/db/repositories.py#L108-L111 | 11:10 |
| *** yamamoto has joined #openstack-lbaas | 11:11 | |
| *** ianychoi__ has quit IRC | 11:12 | |
| gthiemonge | I'll propose a patch to fix it | 11:15 |
| *** rcernin has joined #openstack-lbaas | 11:17 | |
| *** rcernin has quit IRC | 11:31 | |
| *** yamamoto has quit IRC | 12:24 | |
| *** yamamoto has joined #openstack-lbaas | 13:13 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: Fix removing tags from Octavia resources https://review.opendev.org/c/openstack/octavia/+/772692 | 13:17 |
| *** yamamoto has quit IRC | 13:25 | |
| *** spatel has joined #openstack-lbaas | 13:28 | |
| *** spatel has quit IRC | 13:33 | |
| *** sapd__x has joined #openstack-lbaas | 13:55 | |
| *** dayou_ has quit IRC | 13:55 | |
| *** spatel has joined #openstack-lbaas | 13:59 | |
| *** spatel has quit IRC | 14:01 | |
| *** spatel has joined #openstack-lbaas | 14:01 | |
| openstackgerrit | Arieh Maron proposed openstack/octavia-tempest-plugin master: Modified test_traffic_ops for failover testing https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/771251 | 14:16 |
| openstackgerrit | Arieh Maron proposed openstack/octavia-tempest-plugin master: Updating _test_pool_CRUD to enable testing of updates to the load balancer algorithm: https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/770163 | 14:31 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-dashboard master: Add TLS enabled support for pools https://review.opendev.org/c/openstack/octavia-dashboard/+/763352 | 14:38 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-dashboard master: Allows creation of resources without forcing the creation of children https://review.opendev.org/c/openstack/octavia-dashboard/+/763353 | 14:38 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-dashboard master: Add TLS versions support for listeners and pools https://review.opendev.org/c/openstack/octavia-dashboard/+/763569 | 14:38 |
| *** irclogbot_1 has quit IRC | 14:55 | |
| *** irclogbot_2 has joined #openstack-lbaas | 14:58 | |
| *** sapd__x has quit IRC | 15:07 | |
| *** spatel has quit IRC | 15:24 | |
| *** yamamoto has joined #openstack-lbaas | 15:26 | |
| *** yamamoto has quit IRC | 15:33 | |
| *** armax has joined #openstack-lbaas | 15:35 | |
| *** rcernin has joined #openstack-lbaas | 15:36 | |
| *** rcernin has quit IRC | 15:41 | |
| johnsom | #startmeeting Octavia | 16:01 |
| openstack | Meeting started Wed Jan 27 16:01:00 2021 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| *** openstack changes topic to " (Meeting topic: Octavia)" | 16:01 | |
| openstack | The meeting name has been set to 'octavia' | 16:01 |
| cgoncalves | o/ | 16:01 |
| gthiemonge | hi | 16:01 |
| johnsom | I think rm_work said he might not make this meeting today | 16:02 |
| johnsom | #topic Announcements | 16:02 |
| *** openstack changes topic to "Announcements (Meeting topic: Octavia)" | 16:02 | |
| johnsom | Let's get started | 16:02 |
| johnsom | Final client release is first week in March | 16:03 |
| johnsom | Feature freeze for everything else is the second week in March | 16:03 |
| johnsom | Just a reminder, we are more than half way through the Wallaby release. | 16:03 |
| johnsom | Please plan to have things wrapped up as soon as possible if you want them to make Wallaby. | 16:03 |
| johnsom | Any other announcements this week? | 16:04 |
| cgoncalves | We released octavia-lib 2.3.0, first release part of the Wallaby cycle | 16:05 |
| johnsom | Yeah, that is good. | 16:05 |
| johnsom | I feel like we are way behind on releases, stable branch and master | 16:05 |
| johnsom | #topic Brief progress reports / bugs needing review | 16:06 |
| *** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)" | 16:06 | |
| johnsom | Ok, updates from folks | 16:06 |
| johnsom | I have not done much beyond a few reviews on Octavia. My focus is on tripleo for designate at the moment | 16:06 |
| cgoncalves | I feel the same too, and it's mostly my bad as stable liaison. if anyone would like to take on the role, please contact me | 16:07 |
| haleyb | cgoncalves: stable liaison proposes/approves stable release tags, correct? | 16:08 |
| gthiemonge | I proposed #link https://review.opendev.org/c/openstack/octavia/+/772692 this morning, it's a fix for an open review in octaviaclient (tag support) | 16:08 |
| haleyb | Also, checking calendar, day ends in a 'y' so the gate is broken, at least for tempest | 16:09 |
| *** ht_a10 has joined #openstack-lbaas | 16:09 | |
| gthiemonge | I also have #link https://review.opendev.org/c/openstack/octavia/+/753247 (SCTP support in the amphora), it's huge but it refactors/renames a lot of thing for keepalived management | 16:09 |
| haleyb | gthiemonge: i'll update your changes on that | 16:10 |
| johnsom | Is anyone managing a priority review list? I think Ann was going to update/start it for Wallaby. | 16:11 |
| gthiemonge | I cleaned up the priority review list a few weeks ago | 16:11 |
| gthiemonge | johnsom: I'll update it | 16:11 |
| johnsom | Ok, thank you. I will check after the meeting if the channel topic still has the link | 16:12 |
| johnsom | #topic Prioritize client patch reviews (rm_work) | 16:13 |
| *** openstack changes topic to "Prioritize client patch reviews (rm_work) (Meeting topic: Octavia)" | 16:13 | |
| johnsom | Our MIA PTL asked if folks can prioritize reviewing Octavia client patches. | 16:13 |
| johnsom | I think he is looking for a release that includes a previously landed feature. | 16:14 |
| johnsom | #topic Open Discussion | 16:14 |
| *** openstack changes topic to "Open Discussion (Meeting topic: Octavia)" | 16:14 | |
| johnsom | Any other topics today? | 16:14 |
| johnsom | Ok then.... Thanks everyone for joining! | 16:15 |
| johnsom | #endmeeting | 16:16 |
| *** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews" | 16:16 | |
| openstack | Meeting ended Wed Jan 27 16:16:00 2021 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:16 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/octavia/2021/octavia.2021-01-27-16.01.html | 16:16 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/octavia/2021/octavia.2021-01-27-16.01.txt | 16:16 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/octavia/2021/octavia.2021-01-27-16.01.log.html | 16:16 |
| gthiemonge | thanks johnsom | 16:16 |
| *** livelace has joined #openstack-lbaas | 16:21 | |
| *** rpittau is now known as rpittau|afk | 16:24 | |
| *** yamamoto has joined #openstack-lbaas | 16:32 | |
| *** stevenglasford has quit IRC | 16:37 | |
| *** NobodyCam has quit IRC | 16:37 | |
| *** mnaser has quit IRC | 16:37 | |
| *** jamespage has quit IRC | 16:37 | |
| *** vishalmanchanda has quit IRC | 16:37 | |
| *** johnsom has quit IRC | 16:38 | |
| *** bbezak has quit IRC | 16:38 | |
| *** gmann has quit IRC | 16:38 | |
| *** headphoneJames has quit IRC | 16:38 | |
| *** dougwig has quit IRC | 16:38 | |
| *** emccormick has quit IRC | 16:38 | |
| *** rpittau|afk has quit IRC | 16:38 | |
| *** fyx has quit IRC | 16:38 | |
| *** guilhermesp has quit IRC | 16:38 | |
| *** andrein has quit IRC | 16:38 | |
| *** mnasiadka has quit IRC | 16:38 | |
| *** nicolasbock has quit IRC | 16:38 | |
| *** parallax has quit IRC | 16:38 | |
| *** jamespage has joined #openstack-lbaas | 16:39 | |
| *** johnsom has joined #openstack-lbaas | 16:39 | |
| *** NobodyCam has joined #openstack-lbaas | 16:39 | |
| *** stevenglasford has joined #openstack-lbaas | 16:40 | |
| *** jrosser has quit IRC | 16:40 | |
| *** vishalmanchanda has joined #openstack-lbaas | 16:40 | |
| *** bbezak has joined #openstack-lbaas | 16:40 | |
| *** dougwig has joined #openstack-lbaas | 16:40 | |
| *** emccormick has joined #openstack-lbaas | 16:40 | |
| *** rm_work has quit IRC | 16:40 | |
| *** rpittau|afk has joined #openstack-lbaas | 16:40 | |
| *** yamamoto has quit IRC | 16:42 | |
| *** guilhermesp has joined #openstack-lbaas | 16:42 | |
| *** andrein has joined #openstack-lbaas | 16:44 | |
| *** rouk has joined #openstack-lbaas | 16:45 | |
| *** headphoneJames has joined #openstack-lbaas | 16:45 | |
| *** gmann has joined #openstack-lbaas | 16:45 | |
| *** mnaser has joined #openstack-lbaas | 16:45 | |
| *** fyx has joined #openstack-lbaas | 16:50 | |
| *** parallax has joined #openstack-lbaas | 16:50 | |
| *** nicolasbock has joined #openstack-lbaas | 16:50 | |
| *** mnasiadka has joined #openstack-lbaas | 16:50 | |
| *** ht_a10 has quit IRC | 16:55 | |
| *** jrosser has joined #openstack-lbaas | 16:55 | |
| *** nicolasbock has quit IRC | 16:55 | |
| *** mnasiadka has quit IRC | 16:55 | |
| *** rm_work has joined #openstack-lbaas | 16:55 | |
| *** rm_work has quit IRC | 16:55 | |
| *** parallax has quit IRC | 16:55 | |
| *** rm_work has joined #openstack-lbaas | 16:56 | |
| *** rm_work has quit IRC | 16:57 | |
| *** parallax has joined #openstack-lbaas | 16:57 | |
| *** mnasiadka has joined #openstack-lbaas | 16:57 | |
| *** nicolasbock has joined #openstack-lbaas | 16:57 | |
| *** rm_work has joined #openstack-lbaas | 16:57 | |
| *** rm_work has quit IRC | 16:57 | |
| *** rm_work has joined #openstack-lbaas | 16:58 | |
| *** rm_work has quit IRC | 16:58 | |
| *** rm_work has joined #openstack-lbaas | 17:09 | |
| *** rm_work has quit IRC | 17:09 | |
| *** ccamposr has joined #openstack-lbaas | 17:12 | |
| *** rm_work has joined #openstack-lbaas | 17:13 | |
| *** rm_work has quit IRC | 17:13 | |
| *** psachin has quit IRC | 17:14 | |
| *** ccamposr__ has quit IRC | 17:15 | |
| *** nicolasbock has quit IRC | 17:18 | |
| *** mnasiadka has quit IRC | 17:18 | |
| *** parallax has quit IRC | 17:18 | |
| *** fyx has quit IRC | 17:18 | |
| *** fyx has joined #openstack-lbaas | 17:19 | |
| *** nicolasbock has joined #openstack-lbaas | 17:19 | |
| *** parallax has joined #openstack-lbaas | 17:19 | |
| *** mnasiadka has joined #openstack-lbaas | 17:20 | |
| *** stand has joined #openstack-lbaas | 17:20 | |
| *** gthiemonge has quit IRC | 17:28 | |
| *** rouk has quit IRC | 17:28 | |
| *** gthiemonge has joined #openstack-lbaas | 17:30 | |
| *** rm_work has joined #openstack-lbaas | 17:30 | |
| *** rm_work has quit IRC | 17:30 | |
| *** livelace has quit IRC | 17:34 | |
| *** rm_work has joined #openstack-lbaas | 17:34 | |
| *** rm_work has quit IRC | 17:34 | |
| *** livelace has joined #openstack-lbaas | 17:37 | |
| *** rouk has joined #openstack-lbaas | 17:41 | |
| *** rm_work has joined #openstack-lbaas | 17:41 | |
| *** rm_work has quit IRC | 17:41 | |
| rouk | is testing octavia with amphoras from the previous release a thing thats done? | 17:41 |
| rouk | ive... got some deadlocked lbs which are on train images, with ussuri octavia | 17:42 |
| johnsom | rouk Yes, the grenade job does that. | 17:42 |
| rouk | ussuri images, no problem, but failing over a train amphora on ussuri deadlocks it. | 17:42 |
| rouk | destroys both heads, and throws whole thing into error. | 17:42 |
| *** gthiemonge has quit IRC | 17:43 | |
| rouk | now that ive updated the image, when telling these older lbs to failover, i see the failover event in the logs, then The amphora is unavailable. Reason: HTTPSConnectionPool(host='172.31.1.119', port=9443), then nothing happens, never boots a replacement. | 17:43 |
| johnsom | rouk Can you provide logs? | 17:43 |
| johnsom | paste.openstack.org | 17:44 |
| johnsom | Well, the connection warning can be normal as it waits for nova | 17:44 |
| rouk | sure, ive got the pre-image update logs, which are of the failover failing then reverting then crashing the lb, and then i also have the logs after updating image/restarting octavia, where the LB is thrown into pending, then back to error, and nothing happens. | 17:44 |
| rouk | give me a bit to collect them all | 17:45 |
| johnsom | Sure, NP | 17:45 |
| johnsom | What version of ussuri are you running? | 17:45 |
| rouk | as it stands, i have 128 LBs to update, and im hoping they dont all crash on failover :p | 17:45 |
| johnsom | Sigh, it looks like a stable/ussuri hasn't been released in a long time. It probably includes a failover fix you will want. | 17:47 |
| rouk | octavia==6.1.1.dev5 | 17:47 |
| johnsom | Ah, so you have ussuri master. Ok, then you should have the fix | 17:47 |
| rouk | this is just a build from upstream kolla, not currently running patched... so that... surprises me? | 17:48 |
| johnsom | https://docs.openstack.org/releasenotes/octavia/ussuri.html | 17:48 |
| johnsom | 6.1.0-19 is a un-released master version of ussuri | 17:49 |
| rouk | funky, alright. | 17:49 |
| johnsom | It will likely become 6.1.1 on a release | 17:49 |
| rouk | i usually build my own octavia images, but this is just what kolla sent out for ussuri a few months ago | 17:49 |
| johnsom | I don't know anything about kolla | 17:50 |
| rouk | usually they build from release packages, so im a bit confused is all. | 17:50 |
| cgoncalves | johnsom, we can cut an ussuri release, was just hoping these could also get merged before: https://review.opendev.org/q/project:openstack/octavia+branch:stable/ussuri+status:open | 17:52 |
| *** rm_work has joined #openstack-lbaas | 17:52 | |
| *** rm_work has quit IRC | 17:52 | |
| johnsom | cgoncalves Let me help with that... | 17:53 |
| *** gthiemonge has joined #openstack-lbaas | 17:55 | |
| *** ccamposr has quit IRC | 17:57 | |
| *** ccamposr has joined #openstack-lbaas | 17:57 | |
| *** vishalmanchanda has quit IRC | 18:00 | |
| *** mnasiadka has quit IRC | 18:00 | |
| cgoncalves | thank you | 18:00 |
| *** vishalmanchanda has joined #openstack-lbaas | 18:00 | |
| *** mnasiadka has joined #openstack-lbaas | 18:00 | |
| *** emccormick has quit IRC | 18:04 | |
| *** emccormick has joined #openstack-lbaas | 18:05 | |
| *** gthiemonge has quit IRC | 18:05 | |
| *** vishalmanchanda has quit IRC | 18:08 | |
| *** mnasiadka has quit IRC | 18:13 | |
| *** emccormick has quit IRC | 18:14 | |
| *** emccormick has joined #openstack-lbaas | 18:15 | |
| *** bbezak has quit IRC | 18:17 | |
| *** fyx has quit IRC | 18:17 | |
| *** dougwig has quit IRC | 18:17 | |
| *** guilhermesp has quit IRC | 18:17 | |
| *** nicolasbock has quit IRC | 18:17 | |
| *** mnaser has quit IRC | 18:17 | |
| *** bbezak has joined #openstack-lbaas | 18:18 | |
| *** mnaser has joined #openstack-lbaas | 18:18 | |
| *** nicolasbock has joined #openstack-lbaas | 18:18 | |
| *** mnasiadka has joined #openstack-lbaas | 18:18 | |
| *** fyx has joined #openstack-lbaas | 18:18 | |
| *** guilhermesp has joined #openstack-lbaas | 18:18 | |
| *** dougwig has joined #openstack-lbaas | 18:19 | |
| *** bbezak has quit IRC | 18:24 | |
| *** dougwig has quit IRC | 18:25 | |
| *** mnasiadka has quit IRC | 18:25 | |
| *** guilhermesp has quit IRC | 18:25 | |
| *** fyx has quit IRC | 18:25 | |
| *** mnaser has quit IRC | 18:27 | |
| *** nicolasbock has quit IRC | 18:27 | |
| *** guilhermesp has joined #openstack-lbaas | 18:31 | |
| *** fyx has joined #openstack-lbaas | 18:31 | |
| *** bbezak has joined #openstack-lbaas | 18:32 | |
| *** mnaser has joined #openstack-lbaas | 18:32 | |
| *** dougwig has joined #openstack-lbaas | 18:33 | |
| *** mnasiadka has joined #openstack-lbaas | 18:33 | |
| *** nicolasbock has joined #openstack-lbaas | 18:33 | |
| *** yamamoto has joined #openstack-lbaas | 18:42 | |
| rouk | johnsom: http://paste.openstack.org/show/yqTBPdhyh6xcdFiLjIqm/ | 18:46 |
| rouk | heres what i pieced together for the original issue. | 18:46 |
| johnsom | rouk FYI, the true error is above the tree in the logs. However I can probably still see what I need | 18:47 |
| rouk | those logs are newest first. | 18:47 |
| rouk | (sorry, its how my collection works) | 18:48 |
| rouk | cloud has too many lbs, and people are constantly generating octavia ERROR tier garbage, so i have to wade through a mess to get you the useful messages :p | 18:50 |
| johnsom | rouk, yeah it looks like "Fix load balancers with failed amphora failover" patch might be the issue here. | 18:52 |
| rouk | hmm? | 18:52 |
| johnsom | unable to find local peer 'D1haLflY3NHEW1zpOW0laoi1afw' in peers section is the telling line from the log | 18:52 |
| rouk | yeah, is that the -x bug? | 18:52 |
| rouk | so the curious bit, is after restarting all octavia services, i have a different problem. | 18:52 |
| johnsom | No a different one where sqlalchemy had bad cache data in the session | 18:52 |
| johnsom | Can you check for this change in your controller? https://review.opendev.org/c/openstack/octavia/+/756903/2/octavia/controller/worker/v1/tasks/amphora_driver_tasks.py | 18:53 |
| *** yamamoto has quit IRC | 18:53 | |
| *** mnasiadka has quit IRC | 18:54 | |
| *** guilhermesp has quit IRC | 18:54 | |
| *** fyx has quit IRC | 18:55 | |
| *** nicolasbock has quit IRC | 18:55 | |
| *** dougwig has quit IRC | 18:55 | |
| *** mnaser has quit IRC | 18:55 | |
| rouk | moment, just digging through logs for the current error, where nothing happens on failover. | 18:55 |
| *** ccamposr__ has joined #openstack-lbaas | 18:55 | |
| *** ccamposr has quit IRC | 18:58 | |
| *** gcheresh has quit IRC | 18:58 | |
| rouk | ah, so the current error is the same issue, just never even gets to the BOOTING stage. | 18:59 |
| rouk | let me check the code. | 18:59 |
| rouk | for the sake of information, heres the slightly different trace after restarting octavia http://paste.openstack.org/show/DQDmkE6hxxzRPpM5b8yc/ | 19:00 |
| johnsom | Yeah, that seems like that same sqlalchemy cache issue. It's writing out a bad haproxy config with the wrong peer IDs in it. That includes it's own local peer ID, which means haproxy will abort and not start, so then the LB looks like it is missing | 19:02 |
| rouk | i do not have that patch. | 19:03 |
| rouk | so ill build stable/ussuri and roll that out? | 19:04 |
| johnsom | Well, it's two lines. Fixing that, restarting worker and health manager will resolve your issue | 19:04 |
| johnsom | Yeah, you could pull master and roll it out. It's merged on Ussuri | 19:04 |
| rouk | sure, but in my funny business process the size doesnt matter, if im rolling it, i may as well get the fresh code. | 19:04 |
| johnsom | Up to you | 19:04 |
| johnsom | Just don't pull the release package, this patch isn't in a release yet sadly | 19:05 |
| johnsom | Though I think cgoncalves is working on that now | 19:05 |
| rouk | so next question, why on a ussuri latest amphora image, can i failover fine? | 19:06 |
| rouk | does it not hit this? v2? | 19:07 |
| johnsom | No, v2 is controller side only. It's likely the others didn't have a dual amp in ERROR scenario. | 19:07 |
| rouk | ah, that would be correct, yes. | 19:07 |
| rouk | the previous image failed even when failing over a lb with 2 happy amps | 19:08 |
| rouk | both amps in ALLOCATED, hit failover, both amps get destroyed and error. | 19:09 |
| *** rm_work has joined #openstack-lbaas | 19:18 | |
| *** rm_work has quit IRC | 19:18 | |
| *** mnaser has joined #openstack-lbaas | 19:19 | |
| *** dougwig has joined #openstack-lbaas | 19:20 | |
| *** guilhermesp has joined #openstack-lbaas | 19:20 | |
| *** fyx has joined #openstack-lbaas | 19:20 | |
| *** mnasiadka has joined #openstack-lbaas | 19:20 | |
| *** nicolasbock has joined #openstack-lbaas | 19:20 | |
| *** dougwig has quit IRC | 19:40 | |
| *** mnaser has quit IRC | 19:40 | |
| *** fyx has quit IRC | 19:40 | |
| *** guilhermesp has quit IRC | 19:40 | |
| *** mnasiadka has quit IRC | 19:40 | |
| *** nicolasbock has quit IRC | 19:40 | |
| *** dougwig has joined #openstack-lbaas | 19:41 | |
| *** mnasiadka has joined #openstack-lbaas | 19:42 | |
| *** fyx has joined #openstack-lbaas | 19:42 | |
| *** guilhermesp has joined #openstack-lbaas | 19:42 | |
| *** mnaser has joined #openstack-lbaas | 19:42 | |
| *** nicolasbock has joined #openstack-lbaas | 19:45 | |
| *** ht_a10 has joined #openstack-lbaas | 19:49 | |
| *** rcernin has joined #openstack-lbaas | 20:01 | |
| *** ht_a10 has quit IRC | 20:09 | |
| *** rcernin has quit IRC | 20:41 | |
| *** stand has quit IRC | 20:42 | |
| *** stand has joined #openstack-lbaas | 20:46 | |
| *** yamamoto has joined #openstack-lbaas | 20:52 | |
| *** livelace has quit IRC | 21:09 | |
| *** yamamoto has quit IRC | 21:16 | |
| *** rm_work has joined #openstack-lbaas | 21:19 | |
| *** rm_work has quit IRC | 21:19 | |
| *** wolsen has quit IRC | 21:50 | |
| *** ccamposr has joined #openstack-lbaas | 21:52 | |
| *** rcernin has joined #openstack-lbaas | 21:52 | |
| *** ccamposr__ has quit IRC | 21:54 | |
| *** rm_work has joined #openstack-lbaas | 21:55 | |
| *** rm_work has quit IRC | 21:55 | |
| *** rcernin has quit IRC | 21:55 | |
| *** rcernin has joined #openstack-lbaas | 21:56 | |
| *** rcernin has quit IRC | 22:05 | |
| *** rcernin has joined #openstack-lbaas | 22:05 | |
| *** lxkong has quit IRC | 22:09 | |
| *** lxkong has joined #openstack-lbaas | 22:10 | |
| *** yamamoto has joined #openstack-lbaas | 22:20 | |
| *** f0o has quit IRC | 22:29 | |
| *** andy_ has quit IRC | 22:29 | |
| *** zigo has quit IRC | 22:29 | |
| *** sorrison has quit IRC | 22:29 | |
| *** f0o has joined #openstack-lbaas | 22:30 | |
| *** andy_ has joined #openstack-lbaas | 22:30 | |
| *** zigo has joined #openstack-lbaas | 22:30 | |
| *** sorrison has joined #openstack-lbaas | 22:30 | |
| *** rm_work has joined #openstack-lbaas | 22:55 | |
| *** rm_work has quit IRC | 22:55 | |
| *** tkajinam has joined #openstack-lbaas | 22:59 | |
| *** rm_work has joined #openstack-lbaas | 23:06 | |
| rm_work | have been dealing with IRC connection issues today with my bouncer... I think I was in too many channels >_< | 23:09 |
| rm_work | what's the status of our gates? someone working on them already (or fixed already)? | 23:09 |
| rm_work | johnsom makes a good point -- I have been mostly absent -- going to try to fix that shortly | 23:10 |
| rm_work | (though morning meetings are still likely a no-go) | 23:10 |
| rm_work | ugh new gerrit doesn't seem to show me the zuul job statuses? have to go look at comments? :/ | 23:11 |
| johnsom | Your ZNC isn't alone, the IRC network has had some issues over the last week or so. | 23:11 |
| *** rouk has quit IRC | 23:31 | |
| *** wolsen has joined #openstack-lbaas | 23:35 | |
| *** yamamoto has quit IRC | 23:51 | |
| *** yamamoto has joined #openstack-lbaas | 23:58 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!