| *** ramishra has quit IRC | 00:01 | |
| *** longkb has joined #openstack-lbaas | 00:37 | |
| *** yamamoto has joined #openstack-lbaas | 00:47 | |
| *** yamamoto has quit IRC | 00:51 | |
| *** hongbin has joined #openstack-lbaas | 01:05 | |
| *** fnaval has joined #openstack-lbaas | 01:42 | |
| *** yamamoto has joined #openstack-lbaas | 01:47 | |
| *** yamamoto has quit IRC | 01:53 | |
| *** huangshan has joined #openstack-lbaas | 02:05 | |
| *** ramishra has joined #openstack-lbaas | 02:12 | |
| sapd | hi johnsom , does this change related my issue https://github.com/openstack/octavia/commit/48e85569f7e43e3dd5f09fd1ef4cb165526a92cd#diff-bca5cd1020cb0a80eee6beb9f45c8d55? | 02:21 |
|---|---|---|
| sapd | my previous issue: http://paste.openstack.org/show/723774/ | 02:22 |
| johnsom | sapd: It might be if you are running master/Rocky code | 02:39 |
| johnsom | There was a race condition introduced with the provider driver work. | 02:39 |
| johnsom | There are two patches to address those, one for create and one for updates | 02:40 |
| sapd | yep. I'm running on this commit https://github.com/openstack/octavia/commit/1a807242bac4095ee2eebd6dda0520195fe5ab64 | 02:41 |
| sapd | I will update source code and test again | 02:42 |
| *** huangshan has quit IRC | 02:42 | |
| *** huangshan has joined #openstack-lbaas | 02:43 | |
| johnsom | Ok, the update patch has not yet merged, it needs review | 02:44 |
| *** sapd_ has joined #openstack-lbaas | 02:48 | |
| *** sapd has quit IRC | 02:48 | |
| *** yamamoto has joined #openstack-lbaas | 02:49 | |
| *** yamamoto has quit IRC | 02:55 | |
| *** fnaval has quit IRC | 02:57 | |
| *** blake has joined #openstack-lbaas | 02:57 | |
| *** jiteka- has quit IRC | 03:09 | |
| *** hongbin has quit IRC | 03:10 | |
| *** jiteka- has joined #openstack-lbaas | 03:10 | |
| openstackgerrit | Merged openstack/octavia-dashboard master: Set a minimum coverage threshold for javascript https://review.openstack.org/570451 | 03:15 |
| *** ramishra has quit IRC | 03:49 | |
| *** yamamoto has joined #openstack-lbaas | 03:51 | |
| *** ramishra has joined #openstack-lbaas | 03:51 | |
| *** yamamoto has quit IRC | 03:57 | |
| *** yamamoto has joined #openstack-lbaas | 04:01 | |
| *** blake has quit IRC | 04:03 | |
| *** blake has joined #openstack-lbaas | 04:04 | |
| *** hongbin has joined #openstack-lbaas | 04:05 | |
| *** hongbin has quit IRC | 04:14 | |
| openstackgerrit | Merged openstack/octavia master: Stop logging API not found as ERROR https://review.openstack.org/575999 | 04:24 |
| *** strigazi has quit IRC | 04:46 | |
| *** strigazi has joined #openstack-lbaas | 04:48 | |
| *** gans has joined #openstack-lbaas | 04:53 | |
| *** strigazi has quit IRC | 04:55 | |
| *** strigazi has joined #openstack-lbaas | 04:57 | |
| *** AlexeyAbashkin has joined #openstack-lbaas | 05:01 | |
| *** gans has quit IRC | 05:16 | |
| *** links has joined #openstack-lbaas | 05:33 | |
| *** AlexeyAbashkin has quit IRC | 05:34 | |
| *** AlexeyAbashkin has joined #openstack-lbaas | 05:39 | |
| *** blake has quit IRC | 06:02 | |
| *** AlexeyAbashkin has quit IRC | 06:07 | |
| *** ispp has joined #openstack-lbaas | 06:39 | |
| openstackgerrit | huangshan proposed openstack/octavia master: Add exception handling for housekeeping service https://review.openstack.org/576388 | 06:53 |
| *** tesseract has joined #openstack-lbaas | 07:02 | |
| *** rcernin has quit IRC | 07:05 | |
| *** blake has joined #openstack-lbaas | 07:28 | |
| *** shananigans has quit IRC | 07:33 | |
| *** logan- has quit IRC | 07:35 | |
| *** shananigans has joined #openstack-lbaas | 07:38 | |
| *** kobis has joined #openstack-lbaas | 07:39 | |
| *** logan- has joined #openstack-lbaas | 07:39 | |
| *** pcaruana has joined #openstack-lbaas | 07:50 | |
| openstackgerrit | Rafal Pietrzak proposed openstack/octavia master: Adding support for the octavia listener X-Forwarded-Proto header insertion. https://review.openstack.org/573227 | 07:56 |
| *** blake_ has joined #openstack-lbaas | 08:05 | |
| *** blake has quit IRC | 08:08 | |
| *** blake_ has quit IRC | 08:09 | |
| *** ktibi has joined #openstack-lbaas | 08:15 | |
| *** yboaron has joined #openstack-lbaas | 08:17 | |
| devfaz | johnsom: jet again, sorry! | 08:19 |
| devfaz | rm_work: Im currently using pike, octavia 1.0.2 | 08:21 |
| devfaz | What I already found is that the number of threads octavia-health is using is increasing. Currently each process is using 1800 threads and I restarted the process yesterday. Im able to reproduce this by starting some lb and wait. The amounts of threads is increasing over time. | 08:23 |
| *** AlexeyAbashkin has joined #openstack-lbaas | 08:25 | |
| *** salmankhan has joined #openstack-lbaas | 09:32 | |
| *** Eran_Kuris has joined #openstack-lbaas | 09:37 | |
| *** huangshan has quit IRC | 09:50 | |
| *** annp has quit IRC | 10:36 | |
| *** hvhaugwitz has quit IRC | 10:38 | |
| *** hvhaugwitz has joined #openstack-lbaas | 10:38 | |
| *** yamamoto has quit IRC | 10:50 | |
| *** kobis has quit IRC | 10:52 | |
| *** nmanos has joined #openstack-lbaas | 10:55 | |
| *** yamamoto has joined #openstack-lbaas | 10:58 | |
| *** yamamoto has quit IRC | 10:58 | |
| *** salmankhan has quit IRC | 10:59 | |
| devfaz | just updated my octavia to latest git, nothing changed. Still increasing threads/cpu-load. | 10:59 |
| *** salmankhan has joined #openstack-lbaas | 10:59 | |
| *** links has quit IRC | 11:06 | |
| *** salmankhan has quit IRC | 11:06 | |
| *** kobis has joined #openstack-lbaas | 11:06 | |
| *** salmankhan has joined #openstack-lbaas | 11:08 | |
| *** links has joined #openstack-lbaas | 11:22 | |
| *** ww2 has joined #openstack-lbaas | 11:33 | |
| *** ww2 has quit IRC | 11:35 | |
| *** links has quit IRC | 11:39 | |
| *** yamamoto has joined #openstack-lbaas | 11:48 | |
| *** links has joined #openstack-lbaas | 12:05 | |
| *** yamamoto has quit IRC | 12:15 | |
| *** longkb has quit IRC | 12:28 | |
| *** fnaval has joined #openstack-lbaas | 12:31 | |
| devfaz | another update: looks like removing "event_streamer_driver"-line from config fixes the "increasing threads"-problem. Any known bugs with this part? | 12:32 |
| *** yamamoto has joined #openstack-lbaas | 12:46 | |
| *** links has quit IRC | 13:22 | |
| *** yamamoto has quit IRC | 13:34 | |
| *** atoth has joined #openstack-lbaas | 13:35 | |
| *** LutzB has quit IRC | 13:37 | |
| *** LutzB has joined #openstack-lbaas | 13:37 | |
| johnsom | devfaz: event streamer is only used if you are using neutron-lbaas. It will backup threads if it is enabled and the rabbit queue for it is not configured. Disabling it is the best answer. | 13:48 |
| *** Eran_Kuris has quit IRC | 13:51 | |
| *** yamamoto has joined #openstack-lbaas | 13:55 | |
| *** yamamoto has quit IRC | 13:55 | |
| *** yamamoto has joined #openstack-lbaas | 13:57 | |
| *** salmankhan has quit IRC | 13:57 | |
| *** blake has joined #openstack-lbaas | 13:59 | |
| *** salmankhan has joined #openstack-lbaas | 14:00 | |
| *** nmanos has quit IRC | 14:02 | |
| *** yamamoto has quit IRC | 14:16 | |
| openstackgerrit | huangshan proposed openstack/python-octaviaclient master: Support backup members https://review.openstack.org/576530 | 14:20 |
| *** yamamoto has joined #openstack-lbaas | 14:30 | |
| *** yamamoto has quit IRC | 14:35 | |
| *** kobis has quit IRC | 14:41 | |
| *** yamamoto has joined #openstack-lbaas | 14:46 | |
| *** yboaron has quit IRC | 14:50 | |
| *** yamamoto has quit IRC | 14:50 | |
| rm_work | devfaz: yeah, both of those issues are known -- you upgraded to master though and still see some issue? I thought we fixed both | 15:00 |
| *** yamamoto has joined #openstack-lbaas | 15:01 | |
| rm_work | latest pike release would possibly still show some of those problems tho | 15:01 |
| *** yamamoto has quit IRC | 15:05 | |
| *** kobis has joined #openstack-lbaas | 15:09 | |
| *** yamamoto has joined #openstack-lbaas | 15:16 | |
| *** yamamoto has quit IRC | 15:20 | |
| *** kobis has quit IRC | 15:28 | |
| *** pcaruana has quit IRC | 15:29 | |
| *** yamamoto has joined #openstack-lbaas | 15:31 | |
| *** yamamoto has quit IRC | 15:35 | |
| xgerman_ | I am running Pike and it is fine (if the eventstreamer is disabled and you don’t run noisy neigbors) | 15:37 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: UDP jinja template https://review.openstack.org/525420 | 15:38 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: UDP for [2] https://review.openstack.org/529651 | 15:38 |
| openstackgerrit | ZhaoBo proposed openstack/octavia master: UDP for [3][5][6] https://review.openstack.org/539391 | 15:38 |
| *** kobis has joined #openstack-lbaas | 15:39 | |
| *** ispp has quit IRC | 15:41 | |
| *** yamamoto has joined #openstack-lbaas | 15:46 | |
| *** yamamoto has quit IRC | 15:50 | |
| *** kobis has quit IRC | 15:52 | |
| *** kobis has joined #openstack-lbaas | 15:53 | |
| *** kobis has quit IRC | 15:53 | |
| *** kobis has joined #openstack-lbaas | 15:58 | |
| *** ktibi has quit IRC | 16:03 | |
| *** yamamoto has joined #openstack-lbaas | 16:15 | |
| *** yamamoto has quit IRC | 16:20 | |
| *** yamamoto has joined #openstack-lbaas | 16:30 | |
| *** yamamoto has quit IRC | 16:35 | |
| *** blake has quit IRC | 16:43 | |
| *** blake has joined #openstack-lbaas | 16:43 | |
| *** phuoc has quit IRC | 16:45 | |
| *** yamamoto has joined #openstack-lbaas | 16:45 | |
| *** blake has quit IRC | 16:48 | |
| *** yamamoto has quit IRC | 16:50 | |
| *** AlexeyAbashkin has quit IRC | 16:52 | |
| *** blake has joined #openstack-lbaas | 16:54 | |
| *** blake has quit IRC | 16:55 | |
| *** blake has joined #openstack-lbaas | 16:55 | |
| *** kobis has quit IRC | 16:56 | |
| *** blake has quit IRC | 17:00 | |
| *** yamamoto has joined #openstack-lbaas | 17:01 | |
| *** Eran_Kuris has joined #openstack-lbaas | 17:01 | |
| *** pcaruana has joined #openstack-lbaas | 17:05 | |
| *** yamamoto has quit IRC | 17:05 | |
| *** blake has joined #openstack-lbaas | 17:06 | |
| *** tesseract has quit IRC | 17:08 | |
| *** ramishra has quit IRC | 17:09 | |
| *** salmankhan has quit IRC | 17:10 | |
| *** kobis has joined #openstack-lbaas | 17:12 | |
| *** yamamoto has joined #openstack-lbaas | 17:15 | |
| *** kobis has quit IRC | 17:17 | |
| *** yamamoto has quit IRC | 17:20 | |
| *** yamamoto has joined #openstack-lbaas | 17:31 | |
| *** yamamoto has quit IRC | 17:35 | |
| *** yamamoto has joined #openstack-lbaas | 17:45 | |
| *** kobis has joined #openstack-lbaas | 17:48 | |
| *** yamamoto has quit IRC | 17:50 | |
| *** kobis has quit IRC | 18:00 | |
| *** yamamoto has joined #openstack-lbaas | 18:00 | |
| *** yamamoto has quit IRC | 18:05 | |
| *** yamamoto has joined #openstack-lbaas | 18:15 | |
| *** yamamoto has quit IRC | 18:20 | |
| *** yamamoto has joined #openstack-lbaas | 18:30 | |
| *** yamamoto has quit IRC | 18:35 | |
| *** yamamoto has joined #openstack-lbaas | 18:36 | |
| *** yamamoto has quit IRC | 18:36 | |
| *** atoth has quit IRC | 18:51 | |
| *** AlexeyAbashkin has joined #openstack-lbaas | 19:26 | |
| devfaz | johnsom: hmm, i still need neutron-lbaas, because heat is in pike not able to use octavia directly isnt it? So i have to check my configuration. | 19:32 |
| johnsom | I know heat supports octavia, but I'm not sure which release they added it in. | 19:32 |
| devfaz | rm_work: yes, the threads are increasing issue is still in master, if that what you wanted to know? | 19:33 |
| rm_work | hmmm ok | 19:33 |
| rm_work | devfaz: if you are running octavia master, you can do an L7 redirect and skip neutron-lbaas entirely and it will "just work" | 19:33 |
| johnsom | Yes, good point | 19:34 |
| devfaz | Good idea! I will try it out | 19:34 |
| rm_work | on the LB in front of your neutron API, you can redirect /lbaas/ to octavia's endpoint | 19:34 |
| rm_work | see: https://review.openstack.org/#/c/561049/ | 19:34 |
| rm_work | https://review.openstack.org/#/c/561049/38/devstack/etc/neutron/haproxy.cfg | 19:34 |
| rm_work | this is proven to work :) | 19:35 |
| rm_work | BTW can we review/merge that? | 19:35 |
| devfaz | :) | 19:35 |
| *** yamamoto has joined #openstack-lbaas | 19:36 | |
| *** AlexeyAbashkin has quit IRC | 19:37 | |
| devfaz | Maybe someone is able to answer an other question. I lost some Amphore - LB mappings in the past and the database already got cleaned. Now im looking for a way to recreate the missing amphora. Most of the remaining are masters, but even an lb-failover will just recreate the master without doing any Backup-amphora recreation. Any ideas? | 19:38 |
| rm_work | yeah | 19:39 |
| rm_work | so what I do, is copy the master record, and just make up random uuids for the compute_id and all of the port id fields | 19:39 |
| rm_work | and let it assign a new uuid for the uniqueid | 19:39 |
| devfaz | Perfekt thx! | 19:39 |
| rm_work | and mark that as BACKUP | 19:39 |
| rm_work | and then do a failover on it | 19:39 |
| rm_work | the newest code makes that much less likely to happen, so that is good | 19:40 |
| rm_work | (it doesn't clean up the records so quickly) | 19:40 |
| johnsom | rm_work do we have a bug on that? Do we know why? | 19:40 |
| rm_work | johnsom: it's fixed | 19:40 |
| rm_work | when failovers fail ... the records get wiped like instantly in the old code | 19:40 |
| rm_work | because it doesn't have an expiry for the amps | 19:40 |
| johnsom | Ok. I know the dual amp failure ends in something like this, which is what I am working on right now | 19:41 |
| rm_work | it's a bit different | 19:41 |
| rm_work | it's just the record being deleted out from under you like, instantly | 19:41 |
| rm_work | happened to me a lot before | 19:41 |
| *** blake has quit IRC | 19:41 | |
| johnsom | Ok, yeah, I remember that now | 19:41 |
| rm_work | which is why i made sure to do https://review.openstack.org/#/c/548989/ | 19:41 |
| *** blake has joined #openstack-lbaas | 19:42 | |
| *** yamamoto has quit IRC | 19:42 | |
| xgerman_ | yeah, we also have that neutron proxy plugin which is now more than a year in development… | 19:45 |
| *** blake has quit IRC | 19:47 | |
| *** aojea has joined #openstack-lbaas | 19:48 | |
| rm_work | xgerman_: and yet still strikes me as only necessary if you're stuck on an older version of octavia that wasn't fully API compatible <_< | 19:52 |
| rm_work | for anyone who can run master / Rocky, it is *significantly* simpler and less moving parts to just do the L7 redirect | 19:52 |
| rm_work | they have distinctly different uses IMO | 19:52 |
| *** salmankhan has joined #openstack-lbaas | 19:53 | |
| xgerman_ | well, I just did what we agreed to in Atlanta and then things got slow walked | 19:59 |
| rm_work | i think we revised our thinking :P | 20:03 |
| rm_work | that said, I don't think the work was wasted | 20:03 |
| rm_work | for anyone stuck on pike/queens, you probably need that method | 20:04 |
| rm_work | but i don't think it's something we should be advising people use going forward, if they are able to run rocky+ | 20:04 |
| rm_work | basically, if Octavia<=Queens: use proxy plugin, if Octavia>=Rocky: use L7 redirect if possible | 20:05 |
| rm_work | it's also possible there are cases where people can't do L7 stuff in front of neutron | 20:05 |
| rm_work | in which case, they'd also need the plugin | 20:05 |
| devfaz | rm_work: if I remember correctly, it should be fine to run octavia master in an pike cloud, isn't it? | 20:05 |
| johnsom | yes | 20:05 |
| rm_work | yep | 20:06 |
| rm_work | I run octavia master on a Liberty cloud <_< | 20:06 |
| johnsom | Like a crazy man | 20:06 |
| rm_work | (I couldn't find a Juno or Kilo cloud to use instead) :P | 20:06 |
| rm_work | I *think* it'd ork | 20:07 |
| rm_work | *work | 20:07 |
| devfaz | Ok, then I have enough to do tomorrow. Thanks a lot for your work and help! | 20:07 |
| *** salmankhan has quit IRC | 20:22 | |
| *** pcaruana has quit IRC | 20:29 | |
| rm_work | no worries, good luck! | 20:36 |
| *** yamamoto has joined #openstack-lbaas | 20:38 | |
| *** yamamoto has quit IRC | 20:47 | |
| *** Swami has joined #openstack-lbaas | 21:24 | |
| *** aojea has quit IRC | 21:41 | |
| *** blake has joined #openstack-lbaas | 21:42 | |
| *** yamamoto has joined #openstack-lbaas | 21:43 | |
| *** yamamoto has quit IRC | 21:48 | |
| *** rcernin has joined #openstack-lbaas | 22:12 | |
| *** blake has quit IRC | 22:19 | |
| *** blake has joined #openstack-lbaas | 22:20 | |
| *** blake has quit IRC | 22:24 | |
| *** fnaval has quit IRC | 22:28 | |
| *** yamamoto has joined #openstack-lbaas | 22:44 | |
| *** yamamoto has quit IRC | 22:49 | |
| *** blake has joined #openstack-lbaas | 22:52 | |
| *** yamamoto has joined #openstack-lbaas | 23:46 | |
| *** yamamoto has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!