| *** Swami has quit IRC | 00:02 | |
| *** hyang has joined #openstack-lbaas | 00:42 | |
| *** hyang has quit IRC | 00:47 | |
| *** abaindur has quit IRC | 01:22 | |
| *** Emine has quit IRC | 01:22 | |
| *** Emine has joined #openstack-lbaas | 01:23 | |
| *** Emine has quit IRC | 02:23 | |
| *** hongbin has joined #openstack-lbaas | 02:29 | |
| *** rcernin has quit IRC | 03:02 | |
| *** jarodwl has joined #openstack-lbaas | 03:21 | |
| *** jarodwl is now known as jarodwl_ | 03:23 | |
| *** jarodwl_ is now known as jarodwl__ | 03:23 | |
| *** jarodwl__ is now known as jarodwl | 03:24 | |
| *** rcernin has joined #openstack-lbaas | 03:28 | |
| *** ramishra has joined #openstack-lbaas | 03:35 | |
| *** hongbin has quit IRC | 03:57 | |
| *** yamamoto has quit IRC | 04:34 | |
| *** yamamoto has joined #openstack-lbaas | 04:34 | |
| *** yboaron_ has quit IRC | 04:42 | |
| *** Emine has joined #openstack-lbaas | 06:37 | |
| *** yboaron_ has joined #openstack-lbaas | 06:39 | |
| *** Emine has quit IRC | 06:42 | |
| *** rcernin has quit IRC | 07:07 | |
| *** celebdor has joined #openstack-lbaas | 07:14 | |
| *** abaindur has joined #openstack-lbaas | 07:18 | |
| cgoncalves | FYI: issue observed often in octavia-v2-dsvm-scenario-centos-7 job should be fixed now thanks to https://review.openstack.org/#/c/609169/ | 07:30 |
|---|---|---|
| *** openstackgerrit has quit IRC | 07:35 | |
| *** abaindur has quit IRC | 07:37 | |
| *** abaindur has joined #openstack-lbaas | 07:37 | |
| *** abaindur has quit IRC | 08:09 | |
| *** salmankhan has joined #openstack-lbaas | 08:54 | |
| *** salmankhan has quit IRC | 08:56 | |
| *** salmankhan has joined #openstack-lbaas | 08:56 | |
| *** yamamoto has quit IRC | 10:11 | |
| *** yamamoto has joined #openstack-lbaas | 10:12 | |
| *** yamamoto has quit IRC | 10:16 | |
| *** yamamoto has joined #openstack-lbaas | 10:16 | |
| *** yamamoto has quit IRC | 11:20 | |
| *** yamamoto has joined #openstack-lbaas | 11:21 | |
| *** yamamoto has quit IRC | 11:25 | |
| *** yamamoto has joined #openstack-lbaas | 12:03 | |
| *** salmankhan has quit IRC | 12:11 | |
| *** hvhaugwitz has quit IRC | 12:19 | |
| *** hvhaugwitz has joined #openstack-lbaas | 12:19 | |
| *** yamamoto has quit IRC | 12:20 | |
| *** yamamoto has joined #openstack-lbaas | 12:20 | |
| *** yboaron_ has quit IRC | 12:29 | |
| *** yboaron_ has joined #openstack-lbaas | 12:29 | |
| *** dims has quit IRC | 12:30 | |
| *** dims has joined #openstack-lbaas | 12:33 | |
| *** salmankhan has joined #openstack-lbaas | 12:49 | |
| *** sri_ has joined #openstack-lbaas | 13:00 | |
| *** yboaron_ has quit IRC | 13:47 | |
| *** yboaron_ has joined #openstack-lbaas | 14:09 | |
| johnsom | Nice! The bionic fix also merged, so we should be in better shape | 14:49 |
| cgoncalves | even better once https://review.openstack.org/#/c/606741/ merges | 14:51 |
| cgoncalves | johnsom, re: centos amps not handling multiple subnets, shouldn't centos job fail in that case in https://review.openstack.org/#/c/611405/ ? | 14:54 |
| *** openstackgerrit has joined #openstack-lbaas | 14:54 | |
| openstackgerrit | Kamil Sambor proposed openstack/octavia stable/queens: Add config option https://review.openstack.org/611642 | 14:54 |
| johnsom | It did... | 14:54 |
| cgoncalves | johnsom, on the last CI run, but not in the previous (same patch set) | 14:55 |
| cgoncalves | http://logs.openstack.org/05/611405/3/check/octavia-v2-dsvm-scenario-centos-7/3084583/testr_results.html.gz | 14:55 |
| johnsom | cgoncalves It also might be indeterminate, as the IPv6 address might get plugged on one pass, IPv4 on another. | 14:56 |
| *** Swami has joined #openstack-lbaas | 14:56 | |
| *** yamamoto has quit IRC | 15:21 | |
| *** yamamoto has joined #openstack-lbaas | 15:22 | |
| *** yamamoto has quit IRC | 15:26 | |
| openstackgerrit | Merged openstack/octavia-tempest-plugin master: Raise build_timeout from 60 to 300 https://review.openstack.org/606741 | 15:34 |
| *** yboaron_ has quit IRC | 15:39 | |
| *** yamamoto has joined #openstack-lbaas | 16:11 | |
| *** Swami has quit IRC | 16:21 | |
| xgerman_ | cgoncalves: where are you with our slides? Usually I dod them on the flight over but it looks like OpenStack wants to style them? | 16:25 |
| *** yamamoto has quit IRC | 16:56 | |
| *** yamamoto has joined #openstack-lbaas | 16:56 | |
| *** celebdor has quit IRC | 17:09 | |
| cgoncalves | xgerman_, I haven't started yet. we should do it a bit in advance if we want to record a demo | 17:11 |
| *** ramishra has quit IRC | 17:11 | |
| xgerman_ | yeah + there is some pressure form the foundation… | 17:12 |
| *** hyang has joined #openstack-lbaas | 17:26 | |
| *** yamamoto has quit IRC | 17:33 | |
| *** salmankhan has quit IRC | 17:47 | |
| *** fnaval has joined #openstack-lbaas | 18:11 | |
| *** yamamoto has joined #openstack-lbaas | 18:14 | |
| *** salmankhan has joined #openstack-lbaas | 18:17 | |
| *** salmankhan has quit IRC | 18:23 | |
| *** salmankhan has joined #openstack-lbaas | 18:31 | |
| *** salmankhan has quit IRC | 18:40 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Bring up secondary IPs on member networks https://review.openstack.org/611460 | 19:06 |
| *** hyang has quit IRC | 19:23 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add an active/standby scenario test https://review.openstack.org/584681 | 19:41 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Remove deprecated API settings https://review.openstack.org/600819 | 19:42 |
| cgoncalves | is there any task other than the controller.worker.tasks.network_tasks.AllocateVIP that creates the VIP port? | 20:01 |
| cgoncalves | for some reason, on a composable role deployment (api running in one node, hm, cw and hk on another), the worker is telling that the VIP was already created so it then tries to GET it from neutron | 20:03 |
| cgoncalves | which then fails because it indeed doesn't exist. looking deeper, I see also in worker log that a 198.51.100.1 VIP port was presumably created | 20:04 |
| xgerman_ | it’s the only one as I know… but user can post their own port so in that case we would get it from newton | 20:04 |
| cgoncalves | 198.51.100.1 is defined in noop_driver. I checked and all services are set to load the AAP driver | 20:04 |
| cgoncalves | xgerman_, I am passing --vip-subnet-id | 20:04 |
| xgerman_ | k, then logs… | 20:05 |
| cgoncalves | http://cgoncalves.pt/trash/openstack/octavia/worker-fd6c1976-8294-43f3-8133-2ca948fbbaa8.log | 20:15 |
| cgoncalves | Allocate_vip port_id 0d662ae9-3e2c-493a-b153-24d2bf3b6b89, subnet_id 900a07fe-90f4-49d7-9185-2699696c5b7b,ip_address 198.51.100.1 execute | 20:15 |
| cgoncalves | subnet_id is correct one, ip address is not. subnet is 192.168.1.0/24 | 20:16 |
| johnsom | The API creates the VIP, so we can return the assigned IP in the API response | 20:17 |
| cgoncalves | oh! in that case... :D | 20:19 |
| cgoncalves | all 3 controllers running o-api do not have network_driver set | 20:19 |
| johnsom | There you go.... You are getting noop networking | 20:20 |
| cgoncalves | mixed network drivers, yay! | 20:20 |
| cgoncalves | johnsom, confirmed, works now. remind me to pay you whatever is your favorite drink next time ;) | 20:30 |
| johnsom | lol, no worries | 20:30 |
| cgoncalves | see?! another tripleo assignment for me :/ | 20:32 |
| johnsom | Just say no man.... You have more important things to be working on... Grin | 20:32 |
| johnsom | There, you have fulfilled your tripleo time for the year, now you can go back to Octavia.... | 20:33 |
| johnsom | grin | 20:33 |
| cgoncalves | I wish it would work like that | 20:34 |
| *** salmankhan has joined #openstack-lbaas | 20:35 | |
| *** salmankhan has quit IRC | 20:35 | |
| *** openstackgerrit has quit IRC | 20:36 | |
| *** salmankhan has joined #openstack-lbaas | 20:36 | |
| xgerman_ | cgoncalves: just make triple o call out to OSA | 20:39 |
| *** openstackgerrit has joined #openstack-lbaas | 20:41 | |
| openstackgerrit | German Eichberger proposed openstack/octavia master: Refactor the pluggin of the VIP https://review.openstack.org/604479 | 20:41 |
| *** pcaruana has quit IRC | 20:44 | |
| cgoncalves | xgerman_, not a bad idea. I could then re-assign all bugs to you | 20:53 |
| xgerman_ | or johnsom | 20:54 |
| cgoncalves | actually, in this case, it is an issue in puppet-octavia | 20:54 |
| cgoncalves | xgerman_, you're my favorite for OSA | 20:54 |
| xgerman_ | puppet… the Chief Product Officer there used to be our VP at HP… so if you need me to @him on one of my tweets to compalin | 20:57 |
| *** abaindur has joined #openstack-lbaas | 21:01 | |
| johnsom | I could just drive up the road and pay them a visit.... | 21:07 |
| xgerman_ | yeah, make them buy you beer :-) | 21:07 |
| *** abaindur_ has joined #openstack-lbaas | 22:15 | |
| *** abaindur has quit IRC | 22:18 | |
| lxkong | hi guys, if the first amp failover failed during the lb failover proces, will octavia continue to do handle the second amp? | 22:21 |
| lxkong | for stable/queens | 22:21 |
| *** abaindur has joined #openstack-lbaas | 22:22 | |
| johnsom | Yes, the secondary amphora will continue to operate and would be failed over should it also fail. | 22:22 |
| *** abaindur_ has quit IRC | 22:22 | |
| johnsom | Note, you may need a current version of Octavia queens for that. I think we back ported the dual-down amp failover fix. | 22:24 |
| lxkong | yeah, i saw that and we met with the issue | 22:24 |
| lxkong | both amp are gone | 22:24 |
| lxkong | due to db connection issue | 22:24 |
| johnsom | We backported a fix for that too | 22:24 |
| lxkong | johnsom: in that case, it can not be recovered, right? | 22:25 |
| lxkong | there is no records in amp db table | 22:25 |
| johnsom | If the amp records got purged, yeah, you have to recreate it | 22:26 |
| johnsom | This was the db patch: https://review.openstack.org/#/c/602431/ | 22:27 |
| lxkong | hmm.... | 22:27 |
| lxkong | we are doomed | 22:28 |
| johnsom | You have a lot down like that? | 22:28 |
| lxkong | we have some customers using lb for their production business :-( | 22:28 |
| johnsom | If you are highly motivated I think there is a way to fake up amp records to get a amp failover to work. | 22:29 |
| johnsom | The person you want to talk with is not online for a few weeks though. | 22:30 |
| johnsom | Let me think about it for a minute. | 22:30 |
| lxkong | i see we need the old amp information to restore the instance, but it's gone | 22:32 |
| johnsom | Yeah, I would try faking up amphora records if you don't have a backup to use | 22:32 |
| *** rcernin has joined #openstack-lbaas | 22:33 | |
| johnsom | So, do this for the fields: id = new uuid, compute id = new uuid, status = ERROR, lb_id = the load balancer ID you are recovering, lb_network_ip = random IP, vrrp_ip = random IP, ha_ip = the LB VIP IP, vrrp_port_id = random UUID, ha_port_id = try random uuid, role = STANDALONE or one MASTER and one BACKUP, cert_expiration = 2020 date, cert_busy = 0, vrrp_interfance NULL, vrrp_id 1, vrrp_priority Null or 100, | 22:36 |
| johnsom | cached_zone nova | 22:36 |
| johnsom | Then fire a failover | 22:37 |
| lxkong | i'm wondering if it failed for the first amp, why not we stopping the whole failover process for the lb? | 22:38 |
| johnsom | Well, it only fails over an amp if it detects that the amp is failed, so if only one was down, it would have only tried the first and stopped marking it ERROR. However, if the other is detected as failed, it will try it as well. | 22:39 |
| lxkong | well, when something happened for the db connection, i think it's highly likely that both are marked down | 22:41 |
| johnsom | Yeah, it was likely that the DB was unreachable for over 60 seconds (or your failover interval), and when you brought the DB back online, the next health check saw amps that had not reported in for over 60 seconds and started attempting to failover the amps. | 22:42 |
| johnsom | Or your DB went read only for a while | 22:44 |
| *** abaindur_ has joined #openstack-lbaas | 23:04 | |
| *** abaindur has quit IRC | 23:05 | |
| *** abaindur_ has quit IRC | 23:07 | |
| *** abaindur has joined #openstack-lbaas | 23:08 | |
| *** abaindur_ has joined #openstack-lbaas | 23:11 | |
| *** abaindur has quit IRC | 23:12 | |
| *** fnaval has quit IRC | 23:14 | |
| *** abaindur has joined #openstack-lbaas | 23:16 | |
| *** hyang has joined #openstack-lbaas | 23:18 | |
| *** abaindur_ has quit IRC | 23:19 | |
| *** salmankhan has quit IRC | 23:22 | |
| *** hyang has quit IRC | 23:22 | |
| *** abaindur_ has joined #openstack-lbaas | 23:22 | |
| *** abaindur has quit IRC | 23:23 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!