| *** goldyfruit has joined #openstack-lbaas | 02:16 | |
| *** yamamoto has joined #openstack-lbaas | 02:18 | |
| *** goldyfruit has quit IRC | 02:24 | |
| *** rcernin has quit IRC | 02:45 | |
| *** rcernin has joined #openstack-lbaas | 02:47 | |
| *** gcheresh has joined #openstack-lbaas | 03:04 | |
| *** ricolin_ has joined #openstack-lbaas | 03:06 | |
| openstackgerrit | Noboru Iwamatsu proposed openstack/octavia master: Add failover logging to show the amphora details. https://review.opendev.org/667316 | 03:15 |
|---|---|---|
| *** yamamoto has quit IRC | 03:19 | |
| *** gcheresh has quit IRC | 03:23 | |
| *** gcheresh has joined #openstack-lbaas | 03:28 | |
| *** yamamoto has joined #openstack-lbaas | 03:29 | |
| *** psachin has joined #openstack-lbaas | 03:31 | |
| *** psachin has quit IRC | 03:34 | |
| *** psachin has joined #openstack-lbaas | 03:37 | |
| *** psachin has quit IRC | 03:55 | |
| *** gcheresh has quit IRC | 04:15 | |
| *** yamamoto has quit IRC | 04:29 | |
| *** ramishra has joined #openstack-lbaas | 04:38 | |
| *** yamamoto has joined #openstack-lbaas | 04:41 | |
| *** gcheresh has joined #openstack-lbaas | 05:14 | |
| *** ccamposr has joined #openstack-lbaas | 06:11 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 06:42 | |
| *** tesseract has joined #openstack-lbaas | 07:08 | |
| *** rcernin has quit IRC | 07:11 | |
| *** rpittau|afk is now known as rpittau | 07:11 | |
| *** pcaruana has joined #openstack-lbaas | 07:31 | |
| *** ivve has joined #openstack-lbaas | 08:15 | |
| *** tkajinam has quit IRC | 08:30 | |
| *** lemko has joined #openstack-lbaas | 08:42 | |
| lemko | Hi, After some control plane issues, I've some problem with amphoraes status. The loadbalancer work fine but the amphora are in error state even if everything is working fine. Also octavia housepeeking is doing some "Could not connect to instance. Retrying.: ConnectTimeout: HTTPSConnectionPool(host='10.22.0.45', port=9443): Max retries exceeded with url: /0.5/info (Caused by | 08:49 |
| lemko | ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f2afe363e90>, 'Connection to 10.22.0.45 timed out. (connect timeout=10.0)'))" but the problem is that there is no amphora with this IP. Also my 2 amphora spare pool is not respected, there's no spare pool anymore. Any idea what might be wrong ? | 08:49 |
| *** tesseract has quit IRC | 08:58 | |
| *** trident has quit IRC | 09:16 | |
| *** trident has joined #openstack-lbaas | 09:17 | |
| *** tesseract has joined #openstack-lbaas | 09:20 | |
| *** baffle has quit IRC | 09:22 | |
| *** yamamoto has quit IRC | 09:23 | |
| *** altlogbot_3 has quit IRC | 09:24 | |
| *** yamamoto has joined #openstack-lbaas | 09:24 | |
| *** altlogbot_1 has joined #openstack-lbaas | 09:25 | |
| *** yamamoto has quit IRC | 09:27 | |
| *** ricolin_ has quit IRC | 09:29 | |
| *** yamamoto has joined #openstack-lbaas | 09:29 | |
| *** baffle has joined #openstack-lbaas | 09:33 | |
| *** yamamoto has quit IRC | 09:41 | |
| *** yamamoto has joined #openstack-lbaas | 09:49 | |
| openstackgerrit | Merged openstack/octavia-dashboard master: Imported Translations from Zanata https://review.opendev.org/672891 | 10:21 |
| openstackgerrit | Maciej Józefczyk proposed openstack/octavia-tempest-plugin master: Specify used algorithm for tests https://review.opendev.org/672264 | 11:11 |
| openstackgerrit | Maciej Józefczyk proposed openstack/octavia-tempest-plugin master: Add an option to reuse connections https://review.opendev.org/672976 | 11:11 |
| cgoncalves | lemko, hey there. could you check the worker and health manager logs to see if there's an error? may you share what was the control plane issue you had? | 11:14 |
| cgoncalves | your housekeeping service might not be connected properly to the lb-mgmt-net, which might explain why you don't have 2 amphorae in the spare pool | 11:16 |
| cgoncalves | as well as the ConnectTimeout | 11:16 |
| *** boden has joined #openstack-lbaas | 12:10 | |
| *** ricolin_ has joined #openstack-lbaas | 12:30 | |
| *** ivve has quit IRC | 12:34 | |
| *** yamamoto has quit IRC | 12:44 | |
| *** yamamoto has joined #openstack-lbaas | 12:45 | |
| *** ramishra has quit IRC | 13:04 | |
| *** ivve has joined #openstack-lbaas | 13:11 | |
| *** ivve has quit IRC | 13:16 | |
| *** Vorrtex has joined #openstack-lbaas | 13:38 | |
| lemko | Hi cgoncalves, rabbitmq and the database storage got slower. I restarted rabbitmq and openstack started working fine again. The amphorae of the spare pool are (magically back). http://paste.openstack.org/show/755069/ is a log from healthmanager. Sometimes my db cannot repply to the requests and therefore it provokes octavia to not like this very much. http://paste.openstack.org/show/755070/ is the output of my | 13:43 |
| lemko | loadbalancer cli. Old amphorae are in error state. But the loadbalancer are fine. Any idea how to fix it? | 13:43 |
| *** yamamoto has quit IRC | 13:57 | |
| *** ramishra has joined #openstack-lbaas | 14:09 | |
| cgoncalves | lemko, we'd need more log information, but it is likely that the health manager tried to fail over the amps and failed. it then set the status to ERROR. try to fail over the load balancer | 14:15 |
| cgoncalves | I'd also encourage you to have a look at your DB and fix it, else you'll run into this problem rather often | 14:17 |
| lemko | Thanks for your answer cgoncalves. Yep I'm impatiently waiting for my new ssd storage to come ;) | 14:17 |
| lemko | Well I encountered something like "2019-07-29 07:52:44,326.326 17 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [req-63bc02c9-58ad-4cda-8eb2-a83da07cba6c - 1990a23dde4144359d56b7c7d12b9bf6 - - -] Could not | 14:17 |
| lemko | connect to instance. Retrying.: SSLError: HTTPSConnectionPool(host='10.22.0.12', port=9443): Max retries exceeded with url: /0.5/info (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'ssl3_get_record', 'wrong version number')],)",),))" when trying to fail them over | 14:17 |
| lemko | All the amporae remaining are production loadbalancer, so I'm a bit afraid to failover them, since they workd :( | 14:18 |
| lemko | failing over the amphorae, not the loadbalancer, sorry | 14:18 |
| johnsom | Are those amphora even still associated with a load balancer, or were the amphora on the load balancer replaced with new amphora? Those could just be old failed amphora that were replaced with others. | 14:20 |
| lemko | they're still associated with the loadbalancer | 14:20 |
| lemko | and working really fine | 14:20 |
| *** yamamoto has joined #openstack-lbaas | 14:27 | |
| *** yamamoto has quit IRC | 14:38 | |
| *** yamamoto has joined #openstack-lbaas | 14:39 | |
| *** happyhemant has joined #openstack-lbaas | 14:42 | |
| *** yamamoto has quit IRC | 14:44 | |
| *** ivve has joined #openstack-lbaas | 14:55 | |
| *** gcheresh has quit IRC | 15:00 | |
| openstackgerrit | Merged openstack/octavia stable/pike: Add bindep.txt and ignore sha1 warning https://review.opendev.org/667924 | 15:59 |
| *** rpittau is now known as rpittau|afk | 16:04 | |
| colin- | is there a client command for individual amphora failover? openstack loadbalancer amphora failover <ID>? | 16:05 |
| johnsom | https://docs.openstack.org/python-octaviaclient/latest/cli/index.html#loadbalancer-amphora-failover | 16:12 |
| *** goldyfruit has joined #openstack-lbaas | 16:13 | |
| colin- | thanks, is that in rocky? or was it a stein feature | 16:15 |
| johnsom | The API was in Queens, the client added it later, Stein I think. https://docs.openstack.org/releasenotes/octavia/queens.html#new-features | 16:18 |
| *** ivve has quit IRC | 16:21 | |
| *** tesseract has quit IRC | 16:28 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Lookup interfaces by MAC directly https://review.opendev.org/673337 | 16:51 |
| *** ivve has joined #openstack-lbaas | 16:52 | |
| *** ccamposr has quit IRC | 16:58 | |
| *** ivve has quit IRC | 16:59 | |
| *** goldyfruit has quit IRC | 17:02 | |
| *** ramishra has quit IRC | 17:03 | |
| *** tomtom001 has quit IRC | 17:05 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Lookup interfaces by MAC directly https://review.opendev.org/673337 | 17:06 |
| *** mithilarun has joined #openstack-lbaas | 17:09 | |
| *** happyhemant has quit IRC | 17:12 | |
| *** ricolin_ has quit IRC | 17:24 | |
| *** mithilarun has quit IRC | 17:25 | |
| openstackgerrit | Sandeep Yadav proposed openstack/octavia master: Task: 36020 Story: 2006304 https://review.opendev.org/673344 | 17:37 |
| *** trident has quit IRC | 17:47 | |
| openstackgerrit | Sandeep Yadav proposed openstack/octavia master: Correcting typo in healthmonitors-list-response.json - http_vesion to http_version. https://review.opendev.org/673344 | 17:48 |
| *** trident has joined #openstack-lbaas | 17:51 | |
| *** goldyfruit has joined #openstack-lbaas | 17:59 | |
| openstackgerrit | Sandeep Yadav proposed openstack/octavia master: Correcting typo in healthmonitors-list-response.json - http_vesion to http_version. https://review.opendev.org/673344 | 18:05 |
| *** ysandeep has joined #openstack-lbaas | 18:12 | |
| *** lemko has quit IRC | 18:14 | |
| ysandeep | johnsom, cgoncalves, Thank you so much for the review on https://review.opendev.org/#/c/673344/.. | 18:16 |
| johnsom | Thank you for the patch! | 18:16 |
| ysandeep | johnsom, My pleasure, it was my second patch :) | 18:19 |
| ysandeep | cgoncalves i will keep in mind about commit message from future. thanks for the useful link! | 18:19 |
| johnsom | Nice. Feel free to ask questions here about the process. We are happy to help folks get started. | 18:19 |
| ysandeep | Thank you :) | 18:21 |
| *** ysandeep has quit IRC | 18:29 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-lib master: Add get methods to the driver-lib https://review.opendev.org/665027 | 18:30 |
| *** maciejjozefczyk has quit IRC | 19:14 | |
| openstackgerrit | Merged openstack/octavia master: Remove octavia-v2-dsvm-scenario-ubuntu-bionic job https://review.opendev.org/652671 | 19:17 |
| openstackgerrit | Merged openstack/octavia master: Correcting typo in healthmonitors-list-response.json - http_vesion to http_version. https://review.opendev.org/673344 | 19:17 |
| *** yamamoto has joined #openstack-lbaas | 19:25 | |
| *** abaindur has joined #openstack-lbaas | 19:28 | |
| *** yamamoto has quit IRC | 19:32 | |
| *** goldyfruit has quit IRC | 19:37 | |
| openstackgerrit | Austin Russell proposed openstack/octavia master: loadbalancer vip-network-id IP availability check https://review.opendev.org/673358 | 19:42 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia master: Add missing lib dependencies to requirements.txt https://review.opendev.org/673359 | 19:54 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Lookup interfaces by MAC directly https://review.opendev.org/673337 | 20:05 |
| *** goldyfruit has joined #openstack-lbaas | 20:16 | |
| *** lemko has joined #openstack-lbaas | 20:30 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix L7 repository create methods https://review.opendev.org/673154 | 20:45 |
| *** bcafarel has joined #openstack-lbaas | 20:47 | |
| *** trident has quit IRC | 20:49 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix provider driver utils https://review.opendev.org/673155 | 20:51 |
| *** trident has joined #openstack-lbaas | 20:52 | |
| rm_work | Ok more reviews should happen today ;) | 20:52 |
| rm_work | Hold me to that, lol | 20:53 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Add get method support to the driver-agent https://review.opendev.org/665029 | 20:53 |
| johnsom | My e-mail is so full of reviews/merge/etc.... | 20:53 |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add a test job for cinder volume backed amps https://review.opendev.org/638293 | 21:00 |
| cgoncalves | don't forget storyboard notifications too | 21:03 |
| johnsom | rm_work Let me help: https://review.opendev.org/#/c/668700/ and https://review.opendev.org/668697 | 21:06 |
| *** goldyfruit has quit IRC | 21:07 | |
| openstackgerrit | Austin Russell proposed openstack/octavia master: loadbalancer vip-network-id IP availability check https://review.opendev.org/673358 | 21:24 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia-tempest-plugin master: Gate on octavia-v2-dsvm-tls-barbican jobs https://review.opendev.org/666986 | 21:29 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix l7rule API handling of None updates https://review.opendev.org/662569 | 21:30 |
| johnsom | rm_work What do you propose for this? https://review.opendev.org/#/c/631113/10/octavia_tempest_plugin/config.py | 21:33 |
| johnsom | Scrap this check? https://review.opendev.org/#/c/631113/10/octavia_tempest_plugin/tests/api/v2/test_flavor_capabilities.py@59 | 21:34 |
| *** boden has quit IRC | 21:39 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix flavor profile API handling of None updates https://review.opendev.org/662344 | 21:49 |
| openstackgerrit | Merged openstack/octavia master: Fix multi-listener load balancers https://review.opendev.org/668068 | 22:14 |
| *** tkajinam has joined #openstack-lbaas | 22:54 | |
| *** rcernin has joined #openstack-lbaas | 23:02 | |
| colin- | \o/ | 23:06 |
| johnsom | I know right? | 23:07 |
| *** Vorrtex has quit IRC | 23:07 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add an active/standby scenario test https://review.opendev.org/584681 | 23:09 |
| *** lemko has quit IRC | 23:13 | |
| *** abaindur has quit IRC | 23:27 | |
| *** abaindur has joined #openstack-lbaas | 23:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!