| johnsom | aannuusshhkkaa I have not, sorry, lost track of that one. Will look at it tomorrow | 00:03 |
|---|---|---|
| openstackgerrit | Yang JianFeng proposed openstack/octavia-lib master: Define the protocols supported by listener and pool respectively https://review.opendev.org/642252 | 00:30 |
| *** yamamoto has joined #openstack-lbaas | 00:46 | |
| *** yamamoto has quit IRC | 00:49 | |
| *** yamamoto has joined #openstack-lbaas | 00:58 | |
| *** also_stingrayza has joined #openstack-lbaas | 02:03 | |
| *** stingrayza has quit IRC | 02:07 | |
| *** ramishra has quit IRC | 02:30 | |
| *** ramishra has joined #openstack-lbaas | 02:31 | |
| *** wuchunyang has joined #openstack-lbaas | 02:53 | |
| *** wuchunyang has quit IRC | 02:58 | |
| aannuusshhkkaa | johnsom, no worries! Thank you.. :) | 03:15 |
| *** armax has quit IRC | 03:19 | |
| *** psachin has joined #openstack-lbaas | 03:33 | |
| *** wuchunyang has joined #openstack-lbaas | 04:37 | |
| *** gcheresh has joined #openstack-lbaas | 04:46 | |
| *** mugsie has quit IRC | 04:53 | |
| *** wuchunyang has quit IRC | 04:55 | |
| *** wuchunyang has joined #openstack-lbaas | 04:56 | |
| *** gcheresh has quit IRC | 04:56 | |
| *** wuchunyang has quit IRC | 04:57 | |
| *** mugsie has joined #openstack-lbaas | 04:57 | |
| *** strigazi has quit IRC | 05:04 | |
| *** gcheresh has joined #openstack-lbaas | 05:05 | |
| *** strigazi has joined #openstack-lbaas | 05:06 | |
| *** gcheresh has quit IRC | 05:12 | |
| *** gcheresh_ has joined #openstack-lbaas | 05:12 | |
| *** vishalmanchanda has joined #openstack-lbaas | 05:23 | |
| *** wuchunyang has joined #openstack-lbaas | 05:28 | |
| *** wuchunyang has quit IRC | 05:35 | |
| *** gcheresh_ has quit IRC | 05:44 | |
| *** strigazi has quit IRC | 05:46 | |
| *** gcheresh_ has joined #openstack-lbaas | 05:50 | |
| *** ccamposr__ has joined #openstack-lbaas | 06:14 | |
| *** ccamposr has quit IRC | 06:17 | |
| *** njohnston has quit IRC | 06:18 | |
| *** ccamposr has joined #openstack-lbaas | 06:31 | |
| *** michchap has joined #openstack-lbaas | 06:32 | |
| *** ccamposr__ has quit IRC | 06:34 | |
| *** also_stingrayza is now known as stingrayza | 06:49 | |
| *** luksky has joined #openstack-lbaas | 07:02 | |
| *** rcernin has quit IRC | 07:02 | |
| *** rcernin has joined #openstack-lbaas | 07:04 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 07:15 | |
| *** rcernin has quit IRC | 07:30 | |
| openstackgerrit | Ann Taraday proposed openstack/octavia master: [Amphorav2] Healthmonitor operation minor fixes https://review.opendev.org/738609 | 07:43 |
| *** rcernin has joined #openstack-lbaas | 07:50 | |
| *** born2bake has joined #openstack-lbaas | 07:57 | |
| *** rcernin has quit IRC | 08:06 | |
| *** ramishra has quit IRC | 08:09 | |
| *** sapd1 has joined #openstack-lbaas | 08:18 | |
| *** ataraday_ has quit IRC | 09:15 | |
| *** yamamoto has quit IRC | 09:37 | |
| *** yamamoto has joined #openstack-lbaas | 09:55 | |
| *** tkajinam has quit IRC | 09:59 | |
| *** yamamoto has quit IRC | 10:06 | |
| *** yamamoto has joined #openstack-lbaas | 10:07 | |
| *** gcheresh_ has quit IRC | 10:10 | |
| *** gcheresh_ has joined #openstack-lbaas | 10:34 | |
| *** ramishra has joined #openstack-lbaas | 10:45 | |
| lxkong | johnsom, rm_work: When the amphora status is set to ERROR (e.g. because of failed communication), how the ERROR status will be restored back to ALLOCATED? | 10:48 |
| rm_work | failover | 10:48 |
| lxkong | rm_work: yeah, that's what i thought. Why octavia didn't reset amphora status when updating health status? | 10:49 |
| lxkong | rm_work: sometimes because of the a temporary network issue, amphora status becomes ERROR, but after several minutes, the networking is back to normal, shouldn't octavia automatically reset the status to normal, given it is stilll receiving data from amphora? | 10:51 |
| rm_work | hmm | 10:52 |
| rm_work | well, maybe? but we don't record exactly why it went to error | 10:53 |
| rm_work | it could be in ERROR because of bad certificates, or failed service, or not heartbeating | 10:53 |
| rm_work | so if heartbeats reset it, i guess that'd handle most of that, but not sure it'd strictly cover like ... our ability to reach out (cert / network) | 10:53 |
| rm_work | like UDP packets might not be blocked, but TCP into 9443 is for the agent HTTPS server? or the certs on that are bad? | 10:54 |
| rm_work | so we would have to also do a test connection to that | 10:54 |
| rm_work | it's possible you could do some code to handle that... maybe housekeeping could periodically check connection to ERROR amps, and if they both recently received a health packet AND an info-get works to the agent API, maybe we could mark them up again? | 10:55 |
| rm_work | would have to think about what else could be the reason for ERROR | 10:55 |
| lxkong | hmm, make sense. | 10:56 |
| lxkong | so maybe failover is the solution for now, although sometimes the amphorae are still functioning | 10:59 |
| *** njohnston has joined #openstack-lbaas | 11:01 | |
| rm_work | yeah, but it's safest to always just failover | 11:03 |
| lxkong | rm_work: another issue we have met is, the lb operating status is ONLINE, but the amphorae are actually disappeared | 11:06 |
| lxkong | so the operating status is only updated when there is data received by health manager, right? | 11:06 |
| lxkong | i am not asking why the amphorae are missing, just want to know in this case, ONLINE is expected? | 11:07 |
| *** kevinz has quit IRC | 11:11 | |
| rm_work | yeah, only updated in that case, yes | 11:11 |
| rm_work | if UDP packets NEVER happen (like, ACLs blocking them forever) for example, it will always be ONLINE | 11:11 |
| lxkong | rm_work: hmm, ok, sounds weird but yeah, that's the mechanism. | 11:12 |
| rm_work | there is probably some fun additional stuff we can do with like, housekeeping or something | 11:13 |
| rm_work | but that's all extra | 11:13 |
| lxkong | right, thanks for the answers | 11:14 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia stable/stein: Remove scenario bionic job from check https://review.opendev.org/738797 | 11:38 |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-tempest-plugin master: WIP SCTP traffic scenario tests https://review.opendev.org/738643 | 11:53 |
| *** sapd1 has quit IRC | 12:05 | |
| *** sapd1 has joined #openstack-lbaas | 12:05 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia-lib master: Add SCTP protocol and health-monitor constants https://review.opendev.org/738834 | 12:07 |
| *** irclogbot_0 has quit IRC | 12:14 | |
| *** irclogbot_1 has joined #openstack-lbaas | 12:17 | |
| *** trident has quit IRC | 12:46 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: WIP Add SCTP support in API and Amphora https://review.opendev.org/738381 | 12:47 |
| *** trident has joined #openstack-lbaas | 12:49 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia master: WIP Add SCTP support in API and Amphora https://review.opendev.org/738381 | 13:08 |
| *** psachin has quit IRC | 13:31 | |
| *** TrevorV has joined #openstack-lbaas | 13:31 | |
| *** yamamoto has quit IRC | 13:50 | |
| *** yamamoto has joined #openstack-lbaas | 14:11 | |
| *** yamamoto has quit IRC | 14:11 | |
| *** yamamoto has joined #openstack-lbaas | 14:12 | |
| *** yamamoto has quit IRC | 14:16 | |
| *** gcheresh_ has quit IRC | 14:28 | |
| *** gcheresh_ has joined #openstack-lbaas | 14:39 | |
| *** armax has joined #openstack-lbaas | 14:47 | |
| *** gcheresh_ has quit IRC | 15:33 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Update IPv6 job for mirrors and log offloading https://review.opendev.org/738715 | 15:38 |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Update IPv6 job for mirrors and log offloading https://review.opendev.org/738715 | 15:39 |
| cgoncalves | have we considered either renaming the amphorav2 provider to amphora or alias amphora->amphorav2 once we switch the default to amphorav2? | 15:56 |
| *** gcheresh_ has joined #openstack-lbaas | 15:58 | |
| *** ataraday_ has joined #openstack-lbaas | 15:58 | |
| *** shtepanie has joined #openstack-lbaas | 15:59 | |
| johnsom | I think it is a logical choice to make that change at some point, but given the extra infrastructure I'm not sure aliasing amphorav2 to amphora is a good idea. | 16:00 |
| johnsom | #startmeeting Octavia | 16:00 |
| openstack | Meeting started Wed Jul 1 16:00:14 2020 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: Octavia)" | 16:00 | |
| openstack | The meeting name has been set to 'octavia' | 16:00 |
| rm_work | o/ | 16:00 |
| johnsom | Hi everyone | 16:00 |
| ataraday_ | hi | 16:00 |
| cgoncalves | hi | 16:00 |
| johnsom | #topic Announcements | 16:00 |
| *** openstack changes topic to "Announcements (Meeting topic: Octavia)" | 16:00 | |
| johnsom | I don't really have any announcements this week. | 16:01 |
| johnsom | We finally got a stable/train release out the door. I think it had been nine months since the last release. | 16:01 |
| * johnsom shames himself | 16:01 | |
| johnsom | Any other announcements this week? | 16:01 |
| johnsom | #topic Brief progress reports / bugs needing review | 16:02 |
| *** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)" | 16:02 | |
| ataraday_ | I create patch with experimental job with amphorav2 https://review.opendev.org/#/c/737993/ and find out that several things is/got broken. | 16:03 |
| ataraday_ | #link https://review.opendev.org/#/c/738609/ | 16:03 |
| johnsom | Aside from reviews, rebases, and the occasional small bug fix, I have been focusing on failover for amphorav2. It's a bit slow going, but making progress. | 16:04 |
| ataraday_ | there is also issue with barbican tls, still looking into it.. | 16:04 |
| johnsom | I think after we have that landed there are going to be some patches to simplify and clean up stuff. I'm trying to not go crazy doing that now as I don't want another monster patch and want to move faster on this patch. | 16:04 |
| johnsom | Nice, thank you. | 16:05 |
| ataraday_ | johnsom, Thanks for proposing failover for amphorav2! | 16:05 |
| johnsom | I'm also looking into why the IPv6 job is timing out. sigh | 16:05 |
| johnsom | ataraday_ Still a lot to be done, but work in progress | 16:05 |
| *** armax has quit IRC | 16:06 | |
| cgoncalves | I extended the amphora flavor capabilities to add amp_image_tag. this is useful for multi-architecture clouds and testing amphora images in staging environments, for example. as part of that work I also removed some deprecated options and create an image driver interface (noop and glance drivers) | 16:06 |
| johnsom | Yeah, nice! I think I have reviewed about half of that now | 16:07 |
| *** gcheresh_ has quit IRC | 16:07 | |
| rm_work | I'm revisiting the failover threshold thingy | 16:07 |
| rm_work | #link https://review.opendev.org/#/c/656811/ | 16:07 |
| johnsom | Nice, also helpful | 16:07 |
| cgoncalves | I successfully deployed via devstack Octavia in an arm64/aarch64 system, although the amphora agent is not coming up in Zuul CI. this is a side project, thus low priority for me | 16:07 |
| rm_work | isn't TrevorV working on that stuff? aarch64 | 16:08 |
| johnsom | Trevor is working on power | 16:08 |
| rm_work | ahh he's on ppc, ok | 16:08 |
| johnsom | There was a mailing list post about the ppc work and Octavia I mentioned last week. I think I was the only one to reply | 16:09 |
| cgoncalves | OpenDev has aarch64 nodepool nodes, thanks to Linaro! | 16:09 |
| rm_work | what's a "mailing list" | 16:09 |
| johnsom | lol | 16:09 |
| rm_work | I forgot what email was after Google killed Inbox | 16:09 |
| rm_work | RIP Inbox | 16:09 |
| johnsom | You sign up for coupons with one | 16:09 |
| rm_work | RIP email | 16:09 |
| cgoncalves | IT'S A TRAP! | 16:09 |
| rm_work | ooo nice, aarch64 testing resources is good :) do we also have ppc? | 16:10 |
| cgoncalves | I found a clever way to run our CI jobs in nest-virt enabled nodes, cutting job time from close to 2 hours down to as low as 38 minutes | 16:10 |
| cgoncalves | #link https://review.opendev.org/#/c/738246/ | 16:10 |
| johnsom | I don't think so, not in zuul at least. Red Hat has some ppc stuffs | 16:10 |
| rm_work | yeah I just +2'd that, seems workable | 16:11 |
| cgoncalves | yeah, we have also ppc systems internally but, as you said, TrevorV has been working on that | 16:11 |
| *** armax has joined #openstack-lbaas | 16:11 | |
| johnsom | Yeah, I don't have cycles to put into that at the moment | 16:12 |
| * rm_work is just enjoying causing TrevorV's IRC client to possibly ping | 16:12 | |
| TrevorV | Woah woah woah! I haven't touched arm | 16:12 |
| rm_work | haha there we go | 16:12 |
| TrevorV | What'd I miss now? | 16:12 |
| rm_work | yes, johnsom pointed out it was ppc :D | 16:12 |
| johnsom | Arm either, though I have run our code on a raspberry pi 4 with very good results. | 16:12 |
| rm_work | I just remembered you were on some alternate arch | 16:12 |
| cgoncalves | TrevorV, we were saying you've been working on ppc, not arm | 16:12 |
| rm_work | ok so moving on | 16:13 |
| johnsom | Yep | 16:13 |
| johnsom | #topic Open Discussion | 16:14 |
| *** openstack changes topic to "Open Discussion (Meeting topic: Octavia)" | 16:14 | |
| johnsom | Someone seems anxious.... | 16:14 |
| johnsom | Any other topics this week? | 16:14 |
| cgoncalves | I asked just before the meeting started a question about the amphorav2. would now be a good time to discuss it? | 16:15 |
| johnsom | Sure | 16:16 |
| cgoncalves | ok, I'll paste the question again: | 16:16 |
| cgoncalves | have we considered either renaming the amphorav2 provider to amphora or alias amphora->amphorav2 once we switch the default to amphorav2? | 16:16 |
| johnsom | I think it is a logical choice to make that change at some point, but given the extra infrastructure I'm not sure aliasing amphorav2 to amphora is a good idea. | 16:16 |
| * johnsom pastes his answer again | 16:16 | |
| cgoncalves | ok, so you're not for aliasing. are you then for renaming or? | 16:16 |
| *** etp has quit IRC | 16:16 | |
| johnsom | That said, I might look at if we can have the v2 path jobboard/extra requirements optional. It seems like it should be do-able | 16:17 |
| johnsom | Interested in what people think.... | 16:18 |
| rm_work | Yeah I had some concerns here | 16:18 |
| rm_work | If we deprecate/remove the amphora (v1) driver, we are going to be explicitly choosing to make our default deployment more complex than previously (introducing a second required service) | 16:18 |
| rm_work | if we DON'T, we will forever have two code-paths, and that sucks, so I don't think it's really an option T_T | 16:19 |
| *** etp has joined #openstack-lbaas | 16:19 | |
| johnsom | Yeah, we have to deprecate the v1 path. It's just not workable long term IMO | 16:19 |
| rm_work | SO going from that: if we deprecate the old version, and force people over to the new version, given the new complexity/requirements, they MUST explicitly switch over for it to work | 16:19 |
| rm_work | so for people who do upgrades without reading release-notes, if they upgrade from say X->Y release and v1 becomes v2, it's just going to explode but not "cleanly"/"clearly" | 16:20 |
| rm_work | rather than "the provider does not exist" it'll be some error nestled down inside the worker where it tries and fails to connect to unconfigured redis | 16:21 |
| johnsom | We can put some tooling in to warn people or make it obvious early. I think Ann already has updated the upgrade check tool. | 16:22 |
| * johnsom notes, upgrade check tool nobody runs.... | 16:22 | |
| cgoncalves | ataraday_ has a pre-upgrade check for the amphorav2 provider that may be handy | 16:22 |
| cgoncalves | #link https://review.opendev.org/#/c/735556/ | 16:22 |
| rm_work | yeah | 16:22 |
| johnsom | We can set it up to check on startup and fail to start if it's not present as well. | 16:23 |
| rm_work | fail to start if a flavor is using a provider that doesn't exist? | 16:23 |
| johnsom | Or the default is v2, yeah, basically. It's ugly, but possible | 16:24 |
| *** etp has quit IRC | 16:24 | |
| rm_work | anyway, yeah, if we rename/alias automatically then it really hides stuff from the operator that I don't think we should be hiding | 16:25 |
| johnsom | So I think the original question was about the name. I assume for upgrade reasons. Is there a need to rename it from amphorav2? | 16:26 |
| cgoncalves | amphorav2 should be able manage amphorav1-created resources, correct? when we remove the amphorav1 code, we could either do a db migration to change the provider driver of existing LBs to amphorav2 or rename amphorav2 to amphora (with the bonus that we don't potentially break things for the user as the provider name would not change) | 16:26 |
| rm_work | i mean, it does look kinda weird | 16:26 |
| rm_work | I guess so, with the caveat that we prevent the service from starting if everything for amphorav2 isn't configured properly? | 16:27 |
| rm_work | which kinda handles that in a more up-front way | 16:28 |
| cgoncalves | my question comes from a deployment side as I started work to support the amphorav2 in tripleo | 16:28 |
| rm_work | yeah ok i'm coming around, maybe we do alias it, and deal with my concerns via the service startup checking its config is valid | 16:28 |
| rm_work | same as "can i connect to SQL" and "can I connect to RMQ", add "can I connect to Redis/Zookeeper" | 16:29 |
| johnsom | I think we should also look at making the "jobboard" part of the v2 driver optional. | 16:29 |
| rm_work | hmm that is the other posibility | 16:29 |
| rm_work | I actually would prefer that but I didn't know if THAT was feasible, as it's pretty baked in | 16:29 |
| johnsom | We should be able to run the new flows just like we have, without the need for redis, and have a bit-flip for jobboard | 16:29 |
| rm_work | yeah ok I think that is my #1 preferred option -- and then yes, alias amphora to amphorav2 | 16:30 |
| johnsom | Really it's about how we launch the flows. | 16:30 |
| rm_work | and keep the default to False for "use_jobboard" | 16:30 |
| ataraday_ | without jobboard we won't be able to resume jobs | 16:31 |
| rm_work | right | 16:31 |
| ataraday_ | so why we will may need that? | 16:31 |
| johnsom | Right, that would be the trade off of that setting | 16:31 |
| rm_work | so Octavia is still possible to run without Redis | 16:31 |
| ataraday_ | in this case we may leave amphora | 16:31 |
| rm_work | and using no additional upgrade requirements | 16:31 |
| openstackgerrit | Pierre Riteau proposed openstack/octavia master: Add debootstrap installation instructions for CentOS https://review.opendev.org/738885 | 16:32 |
| rm_work | basically that allows for what I wanted as far as "keeping v1 and v2 around" except we don't ACTUALLY have to keep v1 around, and consolidate code paths | 16:32 |
| rm_work | because the ONLY advantage of v1 was not requiring Redis | 16:32 |
| johnsom | Yeah, I really don't want to keep the v1 code around. That is asking for mistakes to be made and doubles work effort (I'm feeling the pain now, lol) | 16:34 |
| rm_work | yep | 16:35 |
| rm_work | i absolutely do not want to keep it around, for the record | 16:35 |
| cgoncalves | +1. amphorav2++ | 16:35 |
| rm_work | I just hated the idea of having no way to run without jobboard (if you want a simpler install, with the possibility of stuff stuck in PENDING) | 16:35 |
| johnsom | So, maybe in parallel someone can look at making that config setting? Or I could look at it after failover is done. | 16:36 |
| johnsom | I think it's just making a method that runs the flows like the v1 driver does or like the v2 driver does, depending on the config setting. | 16:37 |
| cgoncalves | I have little to none (more like none TBH, lol) understanding on the amphorav2/jobboard but I could maybe take a look | 16:38 |
| ataraday_ | this may make code really twisted | 16:38 |
| johnsom | cgoncalves Ok cool. I can give you a few pointers to what I'm thinking. | 16:38 |
| * cgoncalves l | 16:38 | |
| cgoncalves | oops! | 16:38 |
| *** ccamposr__ has joined #openstack-lbaas | 16:39 | |
| rm_work | yeah, my concern (and why I didn't ask for this originally) was that it might not even be possible or would make things incredibly more complex within v2 | 16:39 |
| johnsom | ataraday_ Yeah, I may be overlooking something, but I think we should give it a shot. | 16:39 |
| aannuusshhkkaa | we needed feedback on metric selection for the amphoras.. can we take that up next? | 16:40 |
| rm_work | ok, so decision was: yes to alias, and try to make v2 have a "jobboardless" option? | 16:40 |
| cgoncalves | if feasible, I'd advocate for use_jobboard=true as default. devstack can set redis up and is our recommendation for production environments, right? | 16:40 |
| johnsom | Or was it "alias if we can make the jobboard part optional"? | 16:40 |
| rm_work | hmm maybe that | 16:40 |
| rm_work | yes, let's plan to take that topic up next aannuusshhkkaa! | 16:41 |
| johnsom | Yeah, we should push for it as the default | 16:41 |
| rm_work | I am unsure I agree | 16:41 |
| *** ccamposr has quit IRC | 16:41 | |
| rm_work | but I guess it is as simple as "oh, my service won't start due to an error that says I don't have redis configured and might need to turn off use_jobboard" and then they do that | 16:42 |
| johnsom | I think it is a question of timing. | 16:42 |
| cgoncalves | rm_work, before this conversation, amphorav2 was already set to become the default in Victoria or later release so either way Redis would be required | 16:42 |
| rm_work | yeah which I never liked | 16:43 |
| johnsom | We need to make a call by the end of ms2 because we need to send an e-mail out about the need for Redis to the deployment tools have time to add it. | 16:43 |
| rm_work | ok | 16:43 |
| cgoncalves | you get the bonus now that there may be a chance jobboard/redis not to be mandatory :) | 16:43 |
| rm_work | lets do some research on this and see if it's even possible to make it optional? | 16:43 |
| rm_work | then revisit before ms2 | 16:44 |
| johnsom | Ok, yeah, I agree | 16:44 |
| *** etp has joined #openstack-lbaas | 16:44 | |
| johnsom | MS2 is coming up fast though | 16:44 |
| rm_work | kk | 16:44 |
| johnsom | Week of July 27th | 16:44 |
| rm_work | can we move on to metrics then? | 16:44 |
| aannuusshhkkaa | yes! | 16:44 |
| johnsom | #link https://releases.openstack.org/victoria/schedule.html | 16:44 |
| aannuusshhkkaa | :D | 16:44 |
| cgoncalves | metrics, that's why you were anxious earlier :D | 16:45 |
| johnsom | I think so. What is up with metrics? I know there is a patch that needs some reviews | 16:45 |
| rm_work | heh | 16:45 |
| aannuusshhkkaa | here is the list of metrics we are thinking of implementing: | 16:45 |
| aannuusshhkkaa | Must Haves: | 16:45 |
| aannuusshhkkaa | CPU Usage (Current %) | 16:45 |
| aannuusshhkkaa | Load Averages | 16:45 |
| aannuusshhkkaa | RAM Usage (some combo of: total / free / available / cached / etc) | 16:45 |
| aannuusshhkkaa | Nice to Haves: | 16:45 |
| aannuusshhkkaa | Disk usage (used/free? or used%?) | 16:45 |
| aannuusshhkkaa | Random extra HAProxy fields (taking suggestions) | 16:45 |
| aannuusshhkkaa | are we good on the ones we have selected? are we missing something? have we included something that isn’t plausible? | 16:45 |
| rm_work | yes, we have one patch up that changes the interfaces around slightly | 16:45 |
| rm_work | ack, in the future you should paste multi-line stuff into something like http://paste.openstack.org/ | 16:45 |
| johnsom | Do we need load averages? Personally I would like to keep the data minimal, so just %'s | 16:46 |
| rm_work | I feel like it's generally more useful than "point in time" CPU usage | 16:47 |
| johnsom | Yeah, you can get booted off the server to too much multi-line pastes. | 16:47 |
| aannuusshhkkaa | rm_work gotcha | 16:47 |
| aannuusshhkkaa | johnsom, haha okay! thanks for the correction.. | 16:48 |
| johnsom | Ok, if you have a use for it. | 16:48 |
| rm_work | Well, it's super easy for a single point in time CPU % to be totally off | 16:48 |
| johnsom | aannuusshhkkaa The server can consider you a spam bot basically. | 16:48 |
| aannuusshhkkaa | yeap that makes sense.. will keep that in mind.. | 16:49 |
| johnsom | Yeah, but we will get samples every 10 seconds or so. | 16:49 |
| rm_work | whereas load averages are very nice locally generated averages that we can keep collecting at a much slower interval and still have them be useful | 16:49 |
| *** ataraday_ has quit IRC | 16:49 | |
| rm_work | yeah if we were collecting every second, doing our own averages might make more sense, but... | 16:50 |
| johnsom | Yeah, maybe. You would have to then get the number of cores to calculate a percent | 16:50 |
| rm_work | hmm yes that is true | 16:51 |
| rm_work | possibly do THAT locally? and return load average %s | 16:51 |
| rm_work | rather than have to ship that info up and calculate later | 16:51 |
| johnsom | That could work. | 16:52 |
| rm_work | anyway, we will look into options for that -- any comments about the others? RAM numbers that are actually useful? | 16:52 |
| rm_work | Linux "free memory" is basically a useless metric | 16:52 |
| rm_work | AFAICT | 16:52 |
| rm_work | but I don't know exactly which combination of RAM metrics is *actually* most useful | 16:52 |
| johnsom | Yeah, really I'm looking for % of available memory | 16:53 |
| johnsom | available/total | 16:53 |
| johnsom | cache and free, not so useful to me | 16:53 |
| aannuusshhkkaa | okay.. what about disk usage? used %s? | 16:55 |
| rm_work | I would assume used% is better possibly? though I also wonder if without context that could be not so useful if it says 50% and then you wonder "at what rate will that fill" | 16:56 |
| johnsom | Rate is why you have time series | 16:56 |
| johnsom | IMO | 16:57 |
| johnsom | timeseries/deltas | 16:57 |
| johnsom | A simple scaling driver could have just thresholds and ignore rate. A fancy driver could build a rate and make decisions. | 16:58 |
| johnsom | Then at some point someone can add AI and ML, then build a model of what works and doesn't. Then we retire | 16:58 |
| aannuusshhkkaa | lol | 16:59 |
| rm_work | so that means... percentage? or used/total? lol | 16:59 |
| johnsom | Oh, just about out of time for the meeting. We can continue after if you still have questions. | 16:59 |
| johnsom | Or defer to next week. | 16:59 |
| aannuusshhkkaa | yes we do.. | 16:59 |
| aannuusshhkkaa | next week would probably be a little too late.. | 17:00 |
| johnsom | #endmeeting | 17:00 |
| *** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews" | 17:00 | |
| openstack | Meeting ended Wed Jul 1 17:00:10 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-07-01-16.00.html | 17:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-07-01-16.00.txt | 17:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-07-01-16.00.log.html | 17:00 |
| johnsom | Ok, I can hang around a bit | 17:00 |
| aannuusshhkkaa | thanks! | 17:00 |
| aannuusshhkkaa | so for load averages - % or used/total? | 17:01 |
| rm_work | oh, the reason we were bringing it up during the meeting and not just whenever, was we wanted the largest possible audience for input on anything we missed | 17:02 |
| johnsom | I would lean towards % for each over used/total, but what do you think rm_work? | 17:02 |
| rm_work | did anyone have any comments before they run off about things they'd want to see but we didn't list? | 17:02 |
| rm_work | yeah I think generally standardizing on percentage makes sense | 17:02 |
| johnsom | I don't have any other metrics I need from harpoxy right now. | 17:03 |
| johnsom | I think ram, cpu, disk are the high priority items for me. | 17:03 |
| johnsom | Looking at: http://cbonte.github.io/haproxy-dconv/1.8/management.html#9 | 17:04 |
| aannuusshhkkaa | alright.. | 17:05 |
| rm_work | yeah we talked about some specific haproxy metrics last week but i don't think we decided on any that were REALLY that important | 17:05 |
| rm_work | there's a bunch that are very specific to HTTP or whatever | 17:05 |
| rm_work | or things that just... don't really matter | 17:05 |
| johnsom | Yeah, or could be collected via the log offloading | 17:06 |
| aannuusshhkkaa | my list of metrics was much longer, but rm_work seemed to think the same thing.. | 17:07 |
| aannuusshhkkaa | so in a nutshell, johnsom, % of used and total cpu/ram/disk is what's most important for you right? | 17:09 |
| johnsom | I lean towards keeping these metrics "operational" in that things we need to operate the service effectively. For "analysis" type metrics I would much rather analyze the log files. Partly because I know we need to keep the message reasonably small so that the heartbeat packet can handle a lot of listeners. | 17:09 |
| johnsom | Yes | 17:09 |
| aannuusshhkkaa | okay understood. | 17:10 |
| johnsom | Any other metrics topics you need to discuss before next week? | 17:12 |
| aannuusshhkkaa | johnsom, no.. i think we have everything we need to get started | 17:13 |
| johnsom | +1 great! | 17:14 |
| aannuusshhkkaa | thank you so much for sticking around! | 17:14 |
| *** jamespage has quit IRC | 17:15 | |
| *** jmccrory has quit IRC | 17:15 | |
| *** jamespage has joined #openstack-lbaas | 17:18 | |
| *** jmccrory has joined #openstack-lbaas | 17:18 | |
| *** ccamposr__ has quit IRC | 17:26 | |
| *** born2bake has quit IRC | 17:26 | |
| *** michchap has quit IRC | 17:26 | |
| *** TMM has quit IRC | 17:26 | |
| *** zzzeek has quit IRC | 17:26 | |
| *** devfaz has quit IRC | 17:26 | |
| *** numans has quit IRC | 17:26 | |
| *** hongbin has joined #openstack-lbaas | 17:28 | |
| *** irclogbot_1 has quit IRC | 17:28 | |
| *** ccamposr__ has joined #openstack-lbaas | 17:29 | |
| *** born2bake has joined #openstack-lbaas | 17:29 | |
| *** michchap has joined #openstack-lbaas | 17:29 | |
| *** TMM has joined #openstack-lbaas | 17:29 | |
| *** zzzeek has joined #openstack-lbaas | 17:29 | |
| *** devfaz has joined #openstack-lbaas | 17:29 | |
| *** numans has joined #openstack-lbaas | 17:29 | |
| *** irclogbot_3 has joined #openstack-lbaas | 17:30 | |
| *** njohnston_ has joined #openstack-lbaas | 17:43 | |
| *** njohnston has quit IRC | 17:43 | |
| *** yamamoto has joined #openstack-lbaas | 17:44 | |
| *** njohnston_ is now known as njohnston | 17:45 | |
| *** yamamoto has quit IRC | 17:53 | |
| *** ramishra has quit IRC | 18:01 | |
| *** dasp_ has quit IRC | 18:39 | |
| *** dasp has joined #openstack-lbaas | 18:41 | |
| *** hongbin has quit IRC | 18:44 | |
| *** hongbin has joined #openstack-lbaas | 19:18 | |
| *** vishalmanchanda has quit IRC | 19:47 | |
| *** spatel has joined #openstack-lbaas | 19:55 | |
| *** hongbin has quit IRC | 20:06 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Update IPv6 job for mirrors and log offloading https://review.opendev.org/738715 | 20:22 |
| *** spatel has quit IRC | 20:29 | |
| *** spatel has joined #openstack-lbaas | 20:30 | |
| *** spatel has quit IRC | 20:30 | |
| *** also_stingrayza has joined #openstack-lbaas | 20:33 | |
| *** maciejjozefczyk has quit IRC | 20:36 | |
| *** stingrayza has quit IRC | 20:37 | |
| *** gcheresh_ has joined #openstack-lbaas | 20:42 | |
| *** hongbin has joined #openstack-lbaas | 20:54 | |
| *** gcheresh_ has quit IRC | 21:06 | |
| *** haleyb has quit IRC | 21:40 | |
| *** haleyb has joined #openstack-lbaas | 21:52 | |
| *** born2bake has quit IRC | 22:08 | |
| *** gregwork has joined #openstack-lbaas | 22:09 | |
| *** luksky has quit IRC | 22:29 | |
| openstackgerrit | Merged openstack/octavia-tempest-plugin master: Apply Octavia hacking checks to the tempest plugin https://review.opendev.org/738678 | 22:33 |
| *** rcernin has joined #openstack-lbaas | 22:36 | |
| *** tkajinam has joined #openstack-lbaas | 22:46 | |
| *** rcernin has quit IRC | 22:47 | |
| *** rcernin has joined #openstack-lbaas | 22:47 | |
| *** TrevorV has quit IRC | 22:54 | |
| *** spatel has joined #openstack-lbaas | 22:58 | |
| *** spatel has quit IRC | 23:07 | |
| *** hongbin has quit IRC | 23:11 | |
| *** hongbin has joined #openstack-lbaas | 23:16 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Update IPv6 job for mirrors and log offloading https://review.opendev.org/738715 | 23:28 |
| *** yamamoto has joined #openstack-lbaas | 23:52 | |
| *** yamamoto has quit IRC | 23:57 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!