| bogdan | amphora_agent.agent_server_network_dir = /etc/network/interfaces.d/ log_opt_values /usr/lib/python2.7/site-packages/oslo_config/cfg.py:2525 | 00:01 |
|---|---|---|
| bogdan | git log in my octavia host says: | 00:03 |
| bogdan | commit 5d5dcf9951676ee45b0575a8c48c57ce3a32adf6 Merge: cf97283 66b1f2b Author: Jenkins <jenkins@review.openstack.org> Date: Thu Apr 14 19:11:44 2016 +0000 Merge "Replace mock open with namespace builtins with OpenFixture" | 00:03 |
| bogdan | could it be that my octavia host code is older than the image ?:) | 00:04 |
| johnsom | Yes, that is the case | 00:05 |
| bogdan | :) finally | 00:05 |
| bogdan | will update now | 00:05 |
| *** cody-somerville has quit IRC | 00:07 | |
| *** diogogmt has joined #openstack-lbaas | 00:28 | |
| *** bana_k has quit IRC | 00:29 | |
| *** kevo has quit IRC | 00:39 | |
| *** rcernin has quit IRC | 00:54 | |
| *** mixos has joined #openstack-lbaas | 01:00 | |
| *** crc32 has quit IRC | 01:14 | |
| openstackgerrit | Lingxian Kong proposed openstack/octavia: Add timestamp to octavia resources https://review.openstack.org/309253 | 01:20 |
| *** shakamunyi has joined #openstack-lbaas | 01:23 | |
| *** bogdan has quit IRC | 01:30 | |
| *** BjoernT has joined #openstack-lbaas | 01:31 | |
| *** kobis has quit IRC | 01:32 | |
| *** mixos has quit IRC | 01:36 | |
| *** amotoki has joined #openstack-lbaas | 01:46 | |
| *** ducttape_ has joined #openstack-lbaas | 01:46 | |
| *** Purandar has quit IRC | 01:48 | |
| *** bharathm has quit IRC | 01:51 | |
| *** yuanying has quit IRC | 01:51 | |
| *** yuanying has joined #openstack-lbaas | 01:52 | |
| *** yuanying has quit IRC | 01:52 | |
| *** yuanying has joined #openstack-lbaas | 01:53 | |
| *** AnothrLundquist has quit IRC | 01:54 | |
| *** jaff_cheng has joined #openstack-lbaas | 01:59 | |
| *** BjoernT has quit IRC | 02:13 | |
| *** Purandar has joined #openstack-lbaas | 02:18 | |
| *** ducttape_ has quit IRC | 02:27 | |
| *** ducttape_ has joined #openstack-lbaas | 02:46 | |
| *** bana_k has joined #openstack-lbaas | 02:49 | |
| *** mixos has joined #openstack-lbaas | 02:50 | |
| *** ducttape_ has quit IRC | 02:51 | |
| *** TrevorV|Home has joined #openstack-lbaas | 02:53 | |
| *** TrevorV|Home has joined #openstack-lbaas | 02:54 | |
| *** ducttape_ has joined #openstack-lbaas | 02:54 | |
| *** TrevorV|Home has quit IRC | 03:00 | |
| *** TrevorV has joined #openstack-lbaas | 03:00 | |
| *** yuanying has quit IRC | 03:00 | |
| *** ducttape_ has quit IRC | 03:09 | |
| *** ducttape_ has joined #openstack-lbaas | 03:15 | |
| *** diogogmt has quit IRC | 03:18 | |
| *** yuanying has joined #openstack-lbaas | 03:24 | |
| *** prabampm has joined #openstack-lbaas | 03:30 | |
| *** fnaval_ has joined #openstack-lbaas | 03:31 | |
| *** fnaval has quit IRC | 03:33 | |
| *** links has joined #openstack-lbaas | 03:34 | |
| *** ducttape_ has quit IRC | 03:39 | |
| *** yuanying has quit IRC | 03:42 | |
| *** yuanying has joined #openstack-lbaas | 03:44 | |
| rm_work | AHHH you got bit by that | 03:57 |
| rm_work | my devstack script level-sets the image to the version of the host code <_< | 03:57 |
| rm_work | oh, well... the mitaka branch does | 03:57 |
| rm_work | hmm i should do that same thing in the main one prolly | 03:57 |
| *** bana_k has quit IRC | 03:58 | |
| *** yuanying has quit IRC | 04:01 | |
| *** yuanying has joined #openstack-lbaas | 04:07 | |
| *** AnothrLundquist has joined #openstack-lbaas | 04:10 | |
| *** pcaruana has joined #openstack-lbaas | 04:25 | |
| *** AnothrLundquist has quit IRC | 04:26 | |
| *** yuanying has quit IRC | 04:27 | |
| *** prabampm has quit IRC | 04:28 | |
| *** pcaruana has quit IRC | 04:32 | |
| *** TrevorV has quit IRC | 04:32 | |
| *** fnaval_ has quit IRC | 04:32 | |
| *** yuanying has joined #openstack-lbaas | 04:34 | |
| *** yuanying has quit IRC | 04:36 | |
| *** yuanying has joined #openstack-lbaas | 04:37 | |
| *** reedip__ has joined #openstack-lbaas | 04:37 | |
| *** yuanying has quit IRC | 04:39 | |
| *** fnaval has joined #openstack-lbaas | 04:41 | |
| *** yuanying has joined #openstack-lbaas | 04:41 | |
| *** yuanying has quit IRC | 04:42 | |
| *** yuanying has joined #openstack-lbaas | 04:43 | |
| *** yuanying has quit IRC | 04:48 | |
| *** yuanying has joined #openstack-lbaas | 04:51 | |
| *** yuanying has quit IRC | 04:53 | |
| *** yuanying has joined #openstack-lbaas | 04:54 | |
| *** yuanying has quit IRC | 05:00 | |
| *** armax has quit IRC | 05:13 | |
| *** yuanying has joined #openstack-lbaas | 05:18 | |
| *** woodster_ has quit IRC | 05:18 | |
| *** yuanying has quit IRC | 05:23 | |
| *** prabampm has joined #openstack-lbaas | 05:25 | |
| *** yuanying has joined #openstack-lbaas | 05:25 | |
| *** yuanying has quit IRC | 05:31 | |
| *** yuanying has joined #openstack-lbaas | 05:32 | |
| *** yuanying has quit IRC | 05:33 | |
| *** yuanying has joined #openstack-lbaas | 05:35 | |
| *** amotoki_ has joined #openstack-lbaas | 05:39 | |
| *** amotoki has quit IRC | 05:43 | |
| *** rcernin has joined #openstack-lbaas | 05:54 | |
| *** mixos has quit IRC | 05:54 | |
| *** yuanying has quit IRC | 05:55 | |
| *** prabampm has quit IRC | 05:55 | |
| *** yuanying has joined #openstack-lbaas | 05:57 | |
| *** Purandar has quit IRC | 06:00 | |
| *** yuanying has quit IRC | 06:03 | |
| *** yuanying has joined #openstack-lbaas | 06:04 | |
| *** bana_k has joined #openstack-lbaas | 06:08 | |
| *** pcaruana has joined #openstack-lbaas | 06:12 | |
| *** bana_k has quit IRC | 06:17 | |
| *** prabampm has joined #openstack-lbaas | 06:18 | |
| *** evgenyf has joined #openstack-lbaas | 06:30 | |
| *** anilvenkata has joined #openstack-lbaas | 06:39 | |
| *** yamamoto has quit IRC | 06:49 | |
| *** yamamoto has joined #openstack-lbaas | 06:50 | |
| *** evgenyf has quit IRC | 06:55 | |
| *** kfox1111 has quit IRC | 06:59 | |
| *** kfox1111 has joined #openstack-lbaas | 07:00 | |
| *** tesseract has joined #openstack-lbaas | 07:05 | |
| *** tesseract has quit IRC | 07:09 | |
| *** dmk0202 has joined #openstack-lbaas | 07:40 | |
| *** penchal has joined #openstack-lbaas | 07:51 | |
| *** bogdan has joined #openstack-lbaas | 07:56 | |
| bogdan | I did a pull of Octavia yesterday and now if I create a new LB I get 2 amphoras created with only one of them logged in the worker.log (yes I see 2 different amphora vms but the second is not mentioned in the worker.log) with each of the two amphora VMs taking 3 IPs! in amphora/mgmt network, is this by design? | 07:58 |
| bogdan | soon after LB creation the second amphora vm gets deleted (I also see it in Nova logs) but I do not see a word about it in the worker.log.... | 07:59 |
| *** bharathm has joined #openstack-lbaas | 08:25 | |
| *** mjblack has quit IRC | 08:34 | |
| *** mjblack has joined #openstack-lbaas | 08:34 | |
| penchal | Hi, when i trying to create loadbalancer getting resource could not be found error what could be the reason | 08:35 |
| penchal | stack@devstack:~/devstack$ neutron lbaas-loadbalancer-create --name lb1 private-subnet | 08:35 |
| penchal | The resource could not be found.<br /><br /> | 08:35 |
| penchal | below is the local.conf i used for devstack | 08:36 |
| penchal | ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta | 08:36 |
| penchal | ENABLED_SERVICES+=,q-lbaasv2 | 08:36 |
| penchal | ENABLED_SERVICES+=,octavia,o-cw,o-hk,o-hm,o-api | 08:36 |
| kong | penchal: you use neutron with octavia or lbaas-agent? | 08:51 |
| kong | ooh, i see, octavia | 08:52 |
| kong | do you have any log? | 08:52 |
| penchal | stack@ubuntu2box:~/devstack$ neutron lbaas-loadbalancer-create --name lb1 private-subnet --debug | 08:53 |
| penchal | DEBUG: keystoneclient.session REQ: curl -g -i -X GET http://135.75.169.98:5000/v2.0 -H "Accept: application/json" -H "User-Agent: python-keystoneclient" | 08:53 |
| penchal | DEBUG: urllib3.util.retry Converted retries value: 0 -> Retry(total=0, connect=None, read=None, redirect=0) | 08:53 |
| penchal | DEBUG: keystoneclient.session RESP: [200] content-length: 339 vary: X-Auth-Token server: Apache/2.4.7 (Ubuntu) date: Thu, 12 May 2016 14:30:31 GMT content-type: application/json x-openstack-request-id: req-4fae756a-bc39-4ed4-8f6e-71cc32e114cc | 08:53 |
| penchal | RESP BODY: {"version": {"status": "stable", "updated": "2014-04-17T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "http://135.75.169.98:5000/v2.0/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}]}} | 08:53 |
| penchal | DEBUG: stevedore.extension found extension EntryPoint.parse('yaml = clifftablib.formatters:YamlFormatter') | 08:53 |
| penchal | DEBUG: stevedore.extension found extension EntryPoint.parse('json = clifftablib.formatters:JsonFormatter') | 08:53 |
| *** penchal has quit IRC | 08:53 | |
| kong | then? | 09:00 |
| *** pcaruana is now known as pcaruana|afk| | 09:01 | |
| *** openstackgerrit has quit IRC | 09:04 | |
| *** openstackgerrit has joined #openstack-lbaas | 09:05 | |
| *** bogdan has quit IRC | 09:17 | |
| *** evgenyf has joined #openstack-lbaas | 09:29 | |
| *** prabampm1 has joined #openstack-lbaas | 09:32 | |
| *** prabampm has quit IRC | 09:34 | |
| *** penchal has joined #openstack-lbaas | 09:41 | |
| penchal | kong: any output | 09:42 |
| *** penchal has quit IRC | 09:42 | |
| *** prabampm1 has quit IRC | 09:43 | |
| *** amotoki has joined #openstack-lbaas | 09:52 | |
| *** evgenyf has quit IRC | 09:52 | |
| *** penchal has joined #openstack-lbaas | 09:52 | |
| penchal | http://paste.openstack.org/show/496858/ | 09:53 |
| penchal | kong: any output | 09:54 |
| *** amotoki_ has quit IRC | 09:54 | |
| kong | penchal: you didn't install neutron-lbaas correctly? | 10:04 |
| *** yamamoto has quit IRC | 10:05 | |
| penchal | kong: i installed devstack along with neutron lbaas with below conf | 10:05 |
| penchal | ENABLED_SERVICES+=,neutron,q-svc,q-agt,q-dhcp,q-l3,q-meta ENABLED_SERVICES+=,q-lbaasv2 ENABLED_SERVICES+=,octavia,o-cw,o-hk,o-hm,o-api | 10:06 |
| penchal | do we need to install anything after devstack successfull? | 10:07 |
| kong | penchal: what's the 'service_plugins' config item in your /etc/neutron/neutron.conf | 10:08 |
| kong | is the neutron service running with no errors? | 10:09 |
| penchal | service_plugins = neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPlugini | 10:10 |
| kong | penchal: neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2 | 10:12 |
| penchal | ohh ok | 10:13 |
| penchal | i can add comma seperated ? | 10:14 |
| kong | penchal: service_plugins = neutron.services.l3_router.l3_router_plugin.L3RouterPlugin,neutron_lbaas.services.loadbalancer.plugin.LoadBalancerPluginv2 | 10:14 |
| kong | this is mine | 10:14 |
| penchal | ok | 10:16 |
| *** bogdan has joined #openstack-lbaas | 10:17 | |
| bogdan | any idea why I am getting the 3 amphora network ports per amphora and 2 amphoras per LB? | 10:18 |
| kong | bogdan: number of amphora ports has nothing to do with number of amphorae | 10:19 |
| penchal | before creating oadbalancer do we need to spin up the VMs ? | 10:22 |
| bogdan | penchal, octavia will create the VM automatically upon LB creation | 10:25 |
| bogdan | you need to have the amphora image build and configured in octavia.conf for it to work | 10:25 |
| *** jaff_cheng has quit IRC | 10:28 | |
| *** reedip__ has quit IRC | 10:33 | |
| *** yamamoto_ has joined #openstack-lbaas | 10:34 | |
| penchal | do we need to add amphora image in devstack local.conf | 10:43 |
| *** bogdan has quit IRC | 10:49 | |
| *** rtheis has joined #openstack-lbaas | 10:59 | |
| *** madhu_ak has quit IRC | 11:01 | |
| *** amotoki has quit IRC | 11:06 | |
| *** amotoki has joined #openstack-lbaas | 11:10 | |
| *** amotoki has quit IRC | 11:14 | |
| *** diogogmt has joined #openstack-lbaas | 11:33 | |
| *** ducttape_ has joined #openstack-lbaas | 11:41 | |
| *** prabampm has joined #openstack-lbaas | 11:49 | |
| penchal | stable/kilo devstack should support for neutron-lbaasv2 right ? | 11:55 |
| *** prabampm has quit IRC | 11:56 | |
| *** prabampm has joined #openstack-lbaas | 12:00 | |
| *** ducttape_ has quit IRC | 12:02 | |
| *** jaff_cheng has joined #openstack-lbaas | 12:10 | |
| *** bana_k has joined #openstack-lbaas | 12:36 | |
| *** matt-borland has joined #openstack-lbaas | 12:39 | |
| *** bana_k has quit IRC | 12:50 | |
| *** prabampm1 has joined #openstack-lbaas | 12:54 | |
| *** reedip__ has joined #openstack-lbaas | 12:55 | |
| *** prabampm has quit IRC | 12:56 | |
| *** pcaruana|afk| is now known as pcaruana | 13:01 | |
| *** anilvenkata has quit IRC | 13:14 | |
| *** links has quit IRC | 13:14 | |
| *** amotoki has joined #openstack-lbaas | 13:21 | |
| *** yamamoto_ has quit IRC | 13:33 | |
| *** reedip__ has quit IRC | 13:35 | |
| *** BjoernT has joined #openstack-lbaas | 13:43 | |
| *** reedip__ has joined #openstack-lbaas | 13:48 | |
| *** BjoernT has quit IRC | 13:49 | |
| *** ducttape_ has joined #openstack-lbaas | 14:03 | |
| *** _ducttape_ has joined #openstack-lbaas | 14:07 | |
| *** ducttape_ has quit IRC | 14:09 | |
| *** doug-fish has joined #openstack-lbaas | 14:11 | |
| *** bogdan has joined #openstack-lbaas | 14:17 | |
| *** prabampm1 has quit IRC | 14:25 | |
| *** prabampm has joined #openstack-lbaas | 14:29 | |
| bogdan | after I updated octavia code yesterday I can no longer create lbs - it goes like this "Could not connect to instance. Retrying." and after I added some detailed print I can see the reasons and they are: [Errno 113] EHOSTUNREACH (while vm not booted), then [Errno 111] ECONNREFUSED (until the agent is up) and finally - 3 times ReadTimeout: HTTPSConnectionPool(host='10.1.0.37', port=9443): Read timed out. (read timeout=60.0) - | 14:31 |
| bogdan | right after that I get Task 'octavia.controller.worker.tasks.amphora_driver_tasks.AmphoraPostVIPPlug' (...) transitioned into state 'FAILURE' from state 'RUNNING' | 14:32 |
| *** reedip__ has quit IRC | 14:32 | |
| *** yamamoto_ has joined #openstack-lbaas | 14:33 | |
| *** links has joined #openstack-lbaas | 14:36 | |
| *** yamamoto_ has quit IRC | 14:43 | |
| *** TrevorV has joined #openstack-lbaas | 14:55 | |
| *** fnaval has quit IRC | 15:02 | |
| *** mixos has joined #openstack-lbaas | 15:09 | |
| *** mixos has quit IRC | 15:14 | |
| *** kobis has joined #openstack-lbaas | 15:15 | |
| *** fnaval has joined #openstack-lbaas | 15:16 | |
| *** shakamunyi has quit IRC | 15:19 | |
| TrevorV | johnsom you around? | 15:21 |
| johnsom | TrevorV yeah man | 15:22 |
| TrevorV | Soooo I'm unable to make servers in nova on devstack | 15:22 |
| TrevorV | Maybe you can point me in the right direction | 15:22 |
| TrevorV | lemme gist it. | 15:22 |
| johnsom | Sure | 15:23 |
| TrevorV | https://gist.github.com/TrevorV/dba8970ae61f9432def3c64e9360ee8e | 15:23 |
| TrevorV | johnsom ^^ | 15:24 |
| Frito | TrevorV: Did you check the n-sch logs? | 15:24 |
| johnsom | Yeah, the answer to your problem is in the n-sch log | 15:25 |
| TrevorV | I'll edit the gist and put what is in the logs... | 15:26 |
| *** catintheroof has joined #openstack-lbaas | 15:26 | |
| *** catintheroof has quit IRC | 15:27 | |
| *** catintheroof has joined #openstack-lbaas | 15:29 | |
| TrevorV | johnsom doesn't look like a clear answer to me, updated the gist | 15:29 |
| *** links has quit IRC | 15:32 | |
| johnsom | TrevorV typically there is a line higher in the n-sch log that says which filter decided there are no valid hosts left. | 15:32 |
| *** Purandar has joined #openstack-lbaas | 15:34 | |
| johnsom | On a good pass there should be a list of "Filter ComputeFilter returned 1 host(s)" | 15:34 |
| *** armax has joined #openstack-lbaas | 15:34 | |
| *** fnaval_ has joined #openstack-lbaas | 15:34 | |
| johnsom | Though on your run there should be a line saying returned 0 | 15:35 |
| TrevorV | johnsom I think I threw in the section that we're seeing the difference, but no clear reason why. lemme check or update again | 15:35 |
| *** dmk0202 has quit IRC | 15:35 | |
| TrevorV | johnsom starting with line 68 in the gist | 15:36 |
| *** dmk0202 has joined #openstack-lbaas | 15:36 | |
| TrevorV | It says "starting with 1 host" in line 67, but then line 68 says "host fails" but doesn't say why or anything | 15:36 |
| johnsom | Yeah, it's at retry by then. Nova typically isn't as good at logging the real issue during the retry | 15:37 |
| TrevorV | I'm just not sure how this fails, I clean and stack every morning | 15:37 |
| *** fnaval has quit IRC | 15:37 | |
| johnsom | Out of disk space? | 15:37 |
| TrevorV | johnsom I can't see it saying that, but I also would't know how to verify that. | 15:38 |
| TrevorV | johnsom I added more | 15:38 |
| TrevorV | to the gist I mean | 15:39 |
| johnsom | Haha, you need more coffee. df -h | 15:39 |
| TrevorV | output: /dev/sda3 79G 15G 60G 20% / | 15:40 |
| johnsom | Well, that isn't it | 15:40 |
| TrevorV | All the filters in the list (now in the gist above) show "1 host returned" | 15:41 |
| TrevorV | but then it shows none at the retry below | 15:41 |
| johnsom | Ok, that is odd, I see all of the normal filters returned 1 host | 15:41 |
| TrevorV | nova debug is enabled too, so I don't know where I'd get MORE logs, ya know? | 15:41 |
| *** madhu_ak has joined #openstack-lbaas | 15:43 | |
| TrevorV | blogan had suggested something yesterday that I can't remember, so I was going to bother him when I saw him next, but I haven't seen him yet today | 15:44 |
| johnsom | Yeah, there just isn't a description of why I won't build. Maybe try manually booting an instance? | 15:47 |
| johnsom | I suspect it will give the same "no hosts" error though | 15:47 |
| blogan | if its a scheduler issue i dont know | 15:47 |
| TrevorV | johnsom that IS me manually booting an instance :) | 15:48 |
| johnsom | Yeah, usually there is an entry in the n-sch that tells you which filter failed. I just don't see it in Trevor's log | 15:49 |
| Frito | TrevorV: I can give you the hashes of different project versions in git if you want to try that. It may help track down if it's a bug or not at least. | 15:49 |
| johnsom | TrevorV and nova show on the failed instance, the error reported is just "no hosts"? | 15:49 |
| TrevorV | johnsom that's the top of the gist I sent | 15:50 |
| TrevorV | Frito I just nuked all the repos again and will rebuild from scratch once more locally. | 15:51 |
| TrevorV | This is getting frustrating though. | 15:51 |
| TrevorV | Doing THAT every morning will sink too much time | 15:51 |
| Frito | Right on. If that doesn't work the offer still stands | 15:51 |
| TrevorV | thanks Frito | 15:51 |
| Frito | Of course if it does work the offer would still stand. I'm just not sure you would choose to try it ;-) | 15:51 |
| johnsom | Yeah, other project breakage is a pain. If it happens again I recommend jumping in the nova IRC channel and seeing if you can get help there.... | 15:52 |
| TrevorV | Yeah, I'll do that. | 15:52 |
| johnsom | I'm glad I just restacked and didn't pull down the latest repos this morning. | 15:52 |
| TrevorV | I normally assume that I've done something foolish that you guys already know the answer, so I ask here, but you're right, I should direct questions that way | 15:52 |
| TrevorV | See that's the thing johnsom , I did this yesterday morning... Nuked all the repos (except neutron lbaas, cuz my changes are there) and restacked, and it worked fine. | 15:53 |
| johnsom | Yeah, usually that is a good idea as we all typically hit the same things. | 15:53 |
| TrevorV | Start it up today, and fails. | 15:53 |
| TrevorV | idk | 15:53 |
| johnsom | Welcome to OpenStack! | 15:53 |
| johnsom | grin | 15:53 |
| * TrevorV holds back a swift punch | 15:53 | |
| TrevorV | :P | 15:53 |
| *** madhu_ak has quit IRC | 15:54 | |
| TrevorV | johnsom so you'll be our way next week, right? | 15:54 |
| TrevorV | For your HP conferency thingy? | 15:54 |
| johnsom | Yeah, but locked away at the conference. I won't have a car | 15:54 |
| johnsom | At least my presentation is over early on Monday | 15:55 |
| TrevorV | Ah, right you guys said something about not being "allowed" to venture out | 15:55 |
| TrevorV | or something like that | 15:55 |
| johnsom | Yeah, originally I had hoped to extend the trip to visit you guys, but that is not an option at the moment | 15:55 |
| TrevorV | All good man. | 15:56 |
| Frito | That doesn't rule out volentary kidnappings... | 15:56 |
| Frito | adultnappings? | 15:57 |
| TrevorV | I think the definition of kidnapping rules out "voluntary kidnappings" | 15:57 |
| Frito | *shrugs* | 15:57 |
| johnsom | Frito I'm liking you more and more | 15:57 |
| Frito | lol | 15:57 |
| johnsom | Maybe I will send out a bat signal | 15:57 |
| TrevorV | You just carry something like that around in your luggage? | 15:58 |
| johnsom | There is an app for that.... | 15:58 |
| TrevorV | Not sure its that effective...but hey | 15:59 |
| *** doug-fis_ has joined #openstack-lbaas | 16:01 | |
| johnsom | http://www.thingiverse.com/thing:657146 | 16:02 |
| TrevorV | but can it reach the HEAVENS | 16:03 |
| *** doug-fi__ has joined #openstack-lbaas | 16:03 | |
| *** doug-fish has quit IRC | 16:05 | |
| *** doug-fis_ has quit IRC | 16:06 | |
| *** doug-fi__ has quit IRC | 16:08 | |
| *** ri0 has joined #openstack-lbaas | 16:13 | |
| *** diogogmt has quit IRC | 16:15 | |
| *** dmk0202 has quit IRC | 16:15 | |
| *** penchal has quit IRC | 16:35 | |
| *** doug-fish has joined #openstack-lbaas | 16:43 | |
| *** AnothrLundquist has joined #openstack-lbaas | 16:46 | |
| *** doug-fish has quit IRC | 16:48 | |
| bogdan | johnsom, after I did git pull of octavia and reinstalled I am getting 404 Not Found after amphora is up and service is serving ... see log at http://paste.openstack.org/show/496947/ | 16:51 |
| *** mageshgv has joined #openstack-lbaas | 16:51 | |
| *** mageshgv has quit IRC | 16:51 | |
| *** woodster_ has joined #openstack-lbaas | 16:52 | |
| *** madhu_ak has joined #openstack-lbaas | 16:53 | |
| *** madhu_ak has quit IRC | 16:54 | |
| *** madhu_ak has joined #openstack-lbaas | 16:54 | |
| *** matt-borland has quit IRC | 16:55 | |
| bogdan | what in the latest changes could lead to issues with neutron communication? | 16:55 |
| *** AnothrLundquist has quit IRC | 16:55 | |
| *** kevo has joined #openstack-lbaas | 16:59 | |
| *** fnaval_ is now known as fnaval | 17:00 | |
| johnsom | That is an installation issue and not a bug | 17:01 |
| *** diogogmt has joined #openstack-lbaas | 17:01 | |
| bogdan | what do you mean by installation? | 17:03 |
| bogdan | all i did was python setup.py install | 17:04 |
| *** AnothrLundquist has joined #openstack-lbaas | 17:07 | |
| bogdan | johnson, I forgot to rebuild the amphora image... let me try and will get back | 17:18 |
| *** doug-fish has joined #openstack-lbaas | 17:22 | |
| *** AnothrLundquist has quit IRC | 17:23 | |
| *** doug-fish has quit IRC | 17:23 | |
| *** doug-fish has joined #openstack-lbaas | 17:24 | |
| *** AnothrLundquist has joined #openstack-lbaas | 17:24 | |
| *** doug-fish has quit IRC | 17:28 | |
| *** bana_k has joined #openstack-lbaas | 17:31 | |
| *** SumitNaiksatam has joined #openstack-lbaas | 17:42 | |
| *** _ducttape_ has quit IRC | 17:46 | |
| *** doug-fis_ has joined #openstack-lbaas | 17:48 | |
| *** mjblack has quit IRC | 17:49 | |
| *** reedip__ has joined #openstack-lbaas | 17:53 | |
| *** doug-fis_ has quit IRC | 17:54 | |
| *** pcaruana has quit IRC | 17:58 | |
| *** kobis has quit IRC | 18:00 | |
| bogdan | johnsom, any idea what this is http://paste.openstack.org/show/496955/ ? | 18:05 |
| *** AnothrLundquist has quit IRC | 18:05 | |
| *** doug-fish has joined #openstack-lbaas | 18:05 | |
| *** AnothrLundquist has joined #openstack-lbaas | 18:07 | |
| *** doug-fis_ has joined #openstack-lbaas | 18:08 | |
| *** rcernin has quit IRC | 18:09 | |
| *** doug-fish has quit IRC | 18:10 | |
| *** TrevorV has quit IRC | 18:11 | |
| *** doug-fis_ has quit IRC | 18:13 | |
| *** openstackgerrit has quit IRC | 18:18 | |
| johnsom | bogdan Yes, this means that the compute host is slow and the amphora isn't able to respond fast enough to the controller. Just a minute and I will get you the configuration settings to tune. | 18:18 |
| *** openstackgerrit has joined #openstack-lbaas | 18:18 | |
| johnsom | bogdan | 18:19 |
| johnsom | # rest_request_conn_timeout = 10 | 18:19 |
| johnsom | and # rest_request_read_timeout = 60 | 18:19 |
| *** TrevorV has joined #openstack-lbaas | 18:20 | |
| TrevorV | rm_work you near your computer right now? | 18:23 |
| *** ri0 has quit IRC | 18:27 | |
| *** ducttape_ has joined #openstack-lbaas | 18:28 | |
| *** ri0 has joined #openstack-lbaas | 18:29 | |
| rm_work | yes | 18:30 |
| TrevorV | rm_work I can't seem to get into Rackspace IRC right now. | 18:32 |
| TrevorV | Keeps timing me out. | 18:32 |
| TrevorV | You in the channels internally? | 18:32 |
| rm_work | ah, external IRC went down | 18:32 |
| rm_work | netsplit | 18:32 |
| rm_work | it's still not back up | 18:32 |
| TrevorV | I'm not talking about external :) | 18:32 |
| rm_work | internal is working fine | 18:32 |
| TrevorV | Mine isn't. | 18:33 |
| TrevorV | I'll pm | 18:33 |
| rm_work | yeah | 18:33 |
| *** ri0 has quit IRC | 18:33 | |
| *** BjoernT has joined #openstack-lbaas | 18:33 | |
| *** mjblack has joined #openstack-lbaas | 18:36 | |
| ptoohill | oo | 18:42 |
| *** BjoernT has quit IRC | 18:42 | |
| *** reedip__ has quit IRC | 18:44 | |
| *** dmk0202 has joined #openstack-lbaas | 18:47 | |
| TrevorV | alrighty ptoohill | 18:49 |
| ptoohill | alrighty | 18:49 |
| ptoohill | ? | 18:49 |
| TrevorV | I can't message on internal IRC right now, but I finally got the bug fixed for the l7rule | 18:49 |
| TrevorV | I think | 18:49 |
| TrevorV | Ha ha ha | 18:50 |
| ptoohill | nice | 18:50 |
| *** crc32 has joined #openstack-lbaas | 18:50 | |
| TrevorV | I verified the haproxy config got it, and octavia got it, so I'm 99% sure it's a solid fix. | 18:50 |
| TrevorV | You gots time to test it sometime soon? | 18:50 |
| ptoohill | Have you verified loadbalancing to make sure it got set up right? | 18:50 |
| TrevorV | That was my next sweep. | 18:51 |
| TrevorV | I was verying haproxy config first and foremost | 18:51 |
| ptoohill | I can spin up and test, though if im going to tear down what im working on to do that i want to know that youve verified it this time | 18:51 |
| TrevorV | ptoohill lemme run that first then, and I'll ping you one more time later | 18:51 |
| TrevorV | The biggest issue thus far has been getting running member nodes. | 18:51 |
| ptoohill | how so? | 18:52 |
| *** Alex_Stef has quit IRC | 18:52 | |
| TrevorV | My local devstack has been borked for 2 days... members nodes have been unreliable locally to say the least. | 18:52 |
| ptoohill | you can do one node with two nc webservers on different ports to achieve same thing | 18:52 |
| ptoohill | ah | 18:52 |
| TrevorV | I've got it in a cloud instance currently, so I'll push up the change I have currently to my review, and then I'll pull it down as part of the fresh stack process locally once more, and then test it all. | 18:53 |
| TrevorV | I'll let you know, kk? | 18:53 |
| ptoohill | That works | 18:53 |
| rm_work | if you're using my devstack script, literally just run | 18:55 |
| rm_work | gen_backend | 18:55 |
| TrevorV | uh, what? | 18:55 |
| rm_work | it spins up two member nodes and starts a web service on them | 18:55 |
| rm_work | it's a bash function in .profile | 18:55 |
| rm_work | sourced in when you su to stack | 18:55 |
| TrevorV | Oh. | 18:55 |
| TrevorV | I don't su to stack | 18:55 |
| TrevorV | o_0 | 18:55 |
| rm_work | wat | 18:56 |
| TrevorV | I copy my key and then ssh as stack | 18:56 |
| TrevorV | :) | 18:56 |
| rm_work | oh well whatev | 18:56 |
| TrevorV | :P | 18:56 |
| rm_work | same thing | 18:56 |
| TrevorV | you're a same thing | 18:56 |
| rm_work | 同つ | 18:56 |
| rm_work | あなたは同ですよ | 18:56 |
| ptoohill | coffecup? | 18:57 |
| ptoohill | half empty, just like life? | 18:57 |
| ptoohill | >< | 18:58 |
| TrevorV | 大丈夫ですか | 18:58 |
| ptoohill | Those look like symbols on the floor in the chicken coop | 18:59 |
| rm_work | はい、大丈夫です | 18:59 |
| ptoohill | So.. does that mean..... | 18:59 |
| ptoohill | O.O | 18:59 |
| TrevorV | rm_work did you ask me what that is? | 19:00 |
| TrevorV | lulz | 19:00 |
| rm_work | he asked "Daijoubu desu ka?" (Are you OK?) | 19:00 |
| TrevorV | ah, you respodned | 19:00 |
| rm_work | and I answered yes, I am ok | 19:00 |
| TrevorV | "yes, I'm okay" | 19:00 |
| TrevorV | yeah lulz | 19:00 |
| TrevorV | got it | 19:00 |
| TrevorV | literally JUST tehn | 19:00 |
| TrevorV | then**** | 19:00 |
| TrevorV | ha ha ha | 19:00 |
| TrevorV | http://jisho.org/ | 19:00 |
| TrevorV | I like this website | 19:00 |
| rm_work | yeah it's good | 19:00 |
| rm_work | though simple stuff like that I can actually just ... read now | 19:01 |
| rm_work | ... this is what happens when our internal IRC isn't working :P | 19:01 |
| ptoohill | >< | 19:02 |
| *** AnothrLundquist has quit IRC | 19:03 | |
| *** kevo has quit IRC | 19:03 | |
| *** Alex_Stef has joined #openstack-lbaas | 19:04 | |
| *** mjblack has quit IRC | 19:04 | |
| TrevorV | Oh jeez... I forgot we weren't in the internal chat room..... | 19:04 |
| TrevorV | Spam | 19:04 |
| *** kevo has joined #openstack-lbaas | 19:04 | |
| *** AnothrLundquist has joined #openstack-lbaas | 19:05 | |
| rm_work | haha | 19:06 |
| *** armax has quit IRC | 19:07 | |
| *** AnothrLundquist has quit IRC | 19:07 | |
| *** AnothrLundquist has joined #openstack-lbaas | 19:08 | |
| *** matt-borland has joined #openstack-lbaas | 19:09 | |
| openstackgerrit | Trevor Vardeman proposed openstack/neutron-lbaas: Get Me A LB https://review.openstack.org/257201 | 19:13 |
| ptoohill | Dang everything in the gate is busted now :( | 19:27 |
| ptoohill | http://logs.openstack.org/53/314853/1/check/gate-octavia-python27/86f1ffd/console.html | 19:28 |
| ptoohill | mock brokeh? | 19:29 |
| TrevorV | I didn't do it... | 19:36 |
| TrevorV | OMG OMG OMG I got a fresh stack | 19:36 |
| johnsom | What is up with this? RAX just taking over the openstack-lbaas channel because your internal on is down??? grin | 19:37 |
| TrevorV | Yep. Definitely | 19:38 |
| TrevorV | I don't see the problem | 19:38 |
| * TrevorV actually doesn't like doing this | 19:38 | |
| johnsom | Could be fun, we might hear what you really think of us | 19:38 |
| Frito | The internal one isn't down per-say | 19:38 |
| Frito | lol | 19:38 |
| TrevorV | I'm unable to connect to the internal one Frito | 19:38 |
| TrevorV | and I'm hardlined | 19:38 |
| Frito | ptoohill, carlos, adam and I have no problems :-P | 19:39 |
| TrevorV | Yeah. You're all on bouncers and/or VPNd... Idk why I can't connect. | 19:39 |
| johnsom | Ah, I see where the real problem lies... | 19:39 |
| ptoohill | you dont vpn | 19:39 |
| ptoohill | lol | 19:39 |
| Frito | haha, I'm on bouncer | 19:39 |
| ptoohill | johnsom: huzzh! | 19:40 |
| TrevorV | Why should I VPN when I'm literally hard-wired into the damn network? 100% don't agree with that concept | 19:40 |
| Frito | How else would you know that your bits are truely secure *queues ominous music* | 19:40 |
| TrevorV | I don't even like bits | 19:40 |
| TrevorV | I only take bytes | 19:40 |
| ptoohill | bites? | 19:40 |
| TrevorV | nom sayin? | 19:40 |
| ptoohill | :) | 19:41 |
| TrevorV | NOM SAYIN?!??! | 19:41 |
| Frito | ptoohill is a pro man. He used VPN to VPN to that VPN he's using for the VPN... to VPN | 19:41 |
| Frito | my apologies. I couldn't figure out how to make that any worse in short notice. I'll work harder next time. | 19:42 |
| johnsom | With TLS on top... | 19:42 |
| Frito | haha, all about the TLS cherry on top | 19:42 |
| *** AnothrLundquist has quit IRC | 19:43 | |
| TrevorV | Don't forget his Keystone integration for his auth | 19:43 |
| *** AnothrLundquist has joined #openstack-lbaas | 19:44 | |
| * Frito is trying to figure out how to work in https://en.wikipedia.org/wiki/IP_over_Avian_Carriers but failing | 19:44 | |
| Frito | only 55% packet loss! | 19:44 |
| johnsom | There is an actual RFC for that. A couple of enhancements too if I remember right. | 19:45 |
| ptoohill | I think crc32 has a full fledged plan for that too | 19:45 |
| ptoohill | :) | 19:45 |
| crc32 | I read that. Its really just barcodes wrapped around the birds feet. | 19:48 |
| Frito | TrevorV: try hitting internal now | 19:51 |
| TrevorV | Works now :D | 19:52 |
| *** rcernin has joined #openstack-lbaas | 20:24 | |
| * ptoohill hoping the gate magically fixes itself o.o | 20:25 | |
| * TrevorV always hopes things magically fix themselves | 20:25 | |
| bogdan | I could finally create LB and it works - I am curling the Floating IP of the VIP port and am getting the responses from the two members :) | 20:26 |
| johnsom | Wahooo. | 20:26 |
| bogdan | wondering why curl -s -H "X-Auth-Token: $TOK" http://localhost:9696/v2.0/lbaas/loadbalancers/$lb_id/statuses still reports DEGRADED :) | 20:26 |
| johnsom | That is odd. Do you have a health monitor defined on the pool? | 20:27 |
| bogdan | nope :) | 20:27 |
| bogdan | completely forgot about it :) | 20:27 |
| bogdan | will add it tomorrow, do not want to make another working day till 4:00 am :) | 20:28 |
| johnsom | Ugh, no. Have a good evening | 20:29 |
| bogdan | 10x ;) | 20:30 |
| rm_work | weird that it gets DEGRADED though, would expect NO_MONITOR | 20:45 |
| rm_work | or is that an Octavia only thing that doesn't go up to n-lbaas? | 20:45 |
| johnsom | rm_work No, I added NO_MONITOR to n-lbaas as well | 20:49 |
| rm_work | hmm | 20:49 |
| rm_work | that's weird then | 20:49 |
| rm_work | if no monitor EVER existed for it, how the heck would DEGRADED be possible | 20:50 |
| *** armax has joined #openstack-lbaas | 20:50 | |
| johnsom | It could be one of the upper statuses is DEGRADED but the members are NO_MONITOR | 20:50 |
| rm_work | I'd expect like ... NO_MONITOR or even some other bad status like OFFLINE or ERROR or something possibly as a bug | 20:50 |
| rm_work | but *DEGRADED*!? | 20:50 |
| rm_work | ah, what else can even cause a degraded state? | 20:50 |
| rm_work | i thought it was just member HM failures | 20:50 |
| rm_work | active_standby topology with one LB down? is that DEGRADED? | 20:50 |
| rm_work | err, one amp down | 20:51 |
| bogdan | I've ust added a health monitor, the LB status went to DEGRADED/PENDING_UPDATE and few seconds later again DEGRADED/ACTIVE | 20:53 |
| bogdan | all subcomponents are ACTIVE/ONLINE | 20:53 |
| bogdan | http://paste.openstack.org/show/496980/ | 20:53 |
| johnsom | No, act/stndby only reports the lb health from the user perspective. We don't expose the amp level detail at all | 20:53 |
| rm_work | hmm then i'm unaware of anything that COULD cause degraded besides members | 20:54 |
| rm_work | nothing else has a "partial" provisioning status, I didn't think | 20:54 |
| bogdan | where can I set a break point to debug and see why DEGRADED is calculated? | 20:55 |
| rm_work | I'm not sure offhand where the status tree calculation code is | 20:57 |
| rm_work | I can look into it later, if there's a probable bug | 20:57 |
| blogan | https://github.com/openstack/neutron-lbaas/blob/master/neutron_lbaas/services/loadbalancer/plugin.py#L1273 | 20:57 |
| blogan | that monstrosity | 20:58 |
| *** TrevorV has quit IRC | 20:59 | |
| bogdan | btw, if I call the status of LB it is OFFLINE - see the paste above - i've updated it - so calculated tree says DEGRADED but status of LB alone is OFFLINE | 20:59 |
| bogdan | but this is Neutron right? I am using Liberty there | 21:00 |
| rm_work | yeah probably there's a bug | 21:00 |
| rm_work | ah | 21:00 |
| rm_work | maybe it is an OLD bug then >_> | 21:00 |
| rm_work | hard to say | 21:00 |
| rm_work | I don't THINK i've seen that happening recently | 21:00 |
| rm_work | Liberty is barely usable IMO <_< but maybe I shouldn't say that | 21:00 |
| bogdan | so, you recommend moving to Mitaka? | 21:01 |
| johnsom | Yeah, somewhere along here I made some major changes to status calc. | 21:01 |
| bogdan | why is liberty not usable :)? | 21:01 |
| rm_work | yes, Mitaka is the minimum I would recommend deploying | 21:01 |
| rm_work | many bugfixes | 21:01 |
| rm_work | much more stable | 21:01 |
| bogdan | ok, will consider it | 21:01 |
| rm_work | that's just my opinion though :) | 21:02 |
| rm_work | as a dev, I lean towards bleeding edge | 21:02 |
| rm_work | as a deployer, that's not always the best strategy ^_^ | 21:02 |
| bogdan | yup :) | 21:04 |
| bogdan | Thanks for the support so far! Without it I would not be able to make it run! Thanks johnsom, rm_work, blogan, bharathm! | 21:06 |
| *** ducttape_ has quit IRC | 21:09 | |
| blogan | i just provided a link :) very helpful im sure | 21:10 |
| rm_work | heh np | 21:10 |
| rm_work | I will be on more reliably tomorrow | 21:10 |
| * rm_work fades into the shadows | 21:11 | |
| *** ducttape_ has joined #openstack-lbaas | 21:14 | |
| *** bogdan has quit IRC | 21:17 | |
| *** mixos has joined #openstack-lbaas | 21:18 | |
| *** matt-borland has quit IRC | 21:35 | |
| *** arjun has joined #openstack-lbaas | 21:38 | |
| *** arjun is now known as Guest41990 | 21:39 | |
| Guest41990 | Hello, I have a question regarding using Octavia. Have a running setup w/ controller node, network, hypervisors. We got LBAASv2 working with haproxy, installing lbaasv2 on the controller and lbaas-agent and haproxy on network host. | 21:41 |
| Guest41990 | with Octavia, where exactly is the module installed? On controller, network, or on whichever hypervisor we want to deploy our load balancing VM? | 21:42 |
| Guest41990 | also, do we need a VM image for octavia to deploy, and if so, how to prep it/what all needs to be preconfigured on the image? | 21:43 |
| *** rtheis has quit IRC | 21:43 | |
| *** doug-fish has joined #openstack-lbaas | 21:43 | |
| johnsom | Octavia boots service VMs to do the load balancing. The Octavia controller processes live on your OpenStack controller nodes. | 21:44 |
| johnsom | As for the service VM image, we provide a script that will build an image in /diskimage-create under the Octavia directory | 21:44 |
| Guest41990 | so we do not need to install any additional on our network hosts or hypervisors? | 21:45 |
| johnsom | Our test gates use "diskimage-create.sh -s 2" to create a 2GB image | 21:45 |
| Guest41990 | (as we did with haproxy) | 21:45 |
| johnsom | Octavia does not require to be installed on specific nodes | 21:45 |
| johnsom | Right, the haproxy drive has to be on the network node | 21:45 |
| johnsom | Maybe I mis-understood. You do need to install Octavia somewhere in your OpenStack | 21:47 |
| bharathm | Guest41990: To use Octavia, instead of installing lbaasv2 agent and haproxy, you should install Octavia control processes (API, worker, health manager and housekeeping) | 21:47 |
| *** doug-fish has quit IRC | 21:48 | |
| bharathm | You can install them anywhere you wanted as long as Neutron can reach the octavia API | 21:48 |
| Guest41990 | Yes, I was just confused as to whether we needed a separate octavia module running somewhere on our hosts | 21:48 |
| Guest41990 | in addition to controller node / node where neutron-server lies | 21:48 |
| Guest41990 | does octavia deploy the loadbalancing VM automatically when we create a load-balancer/listener (similar to how haproxy and agent creates an qlbaas Linux namespace) | 21:49 |
| johnsom | Yes, Octavia handles booting the VM | 21:50 |
| Guest41990 | sorry 1 more question :) How does it know to use the image created via the above script? We have many images in our image library | 21:52 |
| Guest41990 | is the image name/path specified somewhere in the lbaas CLI or API? | 21:52 |
| bharathm | You provide image id or tag in octavia.conf file | 21:52 |
| Guest41990 | thanks | 21:53 |
| Guest41990 | will try that out.. | 21:54 |
| bharathm | Sure Np | 21:54 |
| *** dmk0202 has quit IRC | 22:00 | |
| *** daneyon has joined #openstack-lbaas | 22:12 | |
| *** prabampm has quit IRC | 22:12 | |
| *** mixos has quit IRC | 22:13 | |
| *** daneyon_ has quit IRC | 22:16 | |
| *** ducttape_ has quit IRC | 22:23 | |
| *** Guest41990 has quit IRC | 22:37 | |
| *** daneyon has quit IRC | 22:43 | |
| *** intr1nsic has quit IRC | 23:13 | |
| *** mjblack has joined #openstack-lbaas | 23:13 | |
| *** intr1nsic has joined #openstack-lbaas | 23:15 | |
| *** AnothrLundquist has quit IRC | 23:18 | |
| *** AnothrLundquist has joined #openstack-lbaas | 23:19 | |
| *** yuanying_ has joined #openstack-lbaas | 23:22 | |
| *** yuanying has quit IRC | 23:23 | |
| *** daneyon has joined #openstack-lbaas | 23:26 | |
| *** AnothrLundquist has quit IRC | 23:26 | |
| *** chlong has quit IRC | 23:35 | |
| *** daneyon_ has joined #openstack-lbaas | 23:36 | |
| *** daneyon has quit IRC | 23:36 | |
| *** AnothrLundquist has joined #openstack-lbaas | 23:42 | |
| *** SumitNaiksatam has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!