| *** yamamoto has joined #openstack-lbaas | 00:05 | |
| *** doug-fish has joined #openstack-lbaas | 00:23 | |
| *** doug-fish has quit IRC | 00:23 | |
| *** armax_ has joined #openstack-lbaas | 00:23 | |
| *** armax has quit IRC | 00:23 | |
| *** armax_ is now known as armax | 00:23 | |
| sbalukoff | xgerman: Great! I'll take a look, eh! | 00:24 |
|---|---|---|
| xgerman | yeah, I ran it 5 times in pycharm and always green | 00:27 |
| sbalukoff | xgerman: Are you able to get an error in pycharm in 5 or less runs without this patch? | 00:30 |
| xgerman | yep | 00:30 |
| xgerman | actually in 1/3 | 00:30 |
| sbalukoff | Ok. Cool. For me it seemed less frequent: 1/10. But I should know in an hour or two with this loop running. | 00:31 |
| sbalukoff | If we can get the gate un-stuck, I'd be for merging it now, eh. | 00:31 |
| openstackgerrit | min wang proposed openstack/octavia: Implements: blueprint anti-affinity server group https://review.openstack.org/272344 | 00:42 |
| *** minwang2 has quit IRC | 00:45 | |
| *** madhu_ak has quit IRC | 00:45 | |
| *** piet has quit IRC | 00:47 | |
| *** yamamoto has quit IRC | 00:52 | |
| *** drjones has joined #openstack-lbaas | 00:55 | |
| *** drjones has quit IRC | 00:57 | |
| *** ramishra__ has joined #openstack-lbaas | 00:57 | |
| *** ctracey_ has joined #openstack-lbaas | 00:57 | |
| *** piet has joined #openstack-lbaas | 00:57 | |
| *** Aish_ has joined #openstack-lbaas | 00:58 | |
| *** clduser has joined #openstack-lbaas | 00:59 | |
| *** HenryG_ has joined #openstack-lbaas | 01:00 | |
| *** crc32_znc has joined #openstack-lbaas | 01:02 | |
| *** ptoohill has joined #openstack-lbaas | 01:03 | |
| *** rcernin has quit IRC | 01:03 | |
| *** Aish has quit IRC | 01:04 | |
| *** ajmiller has quit IRC | 01:04 | |
| *** _cjones_ has quit IRC | 01:04 | |
| *** a2hill has quit IRC | 01:04 | |
| *** ctracey has quit IRC | 01:04 | |
| *** clduser_ has quit IRC | 01:04 | |
| *** crc32|znc has quit IRC | 01:04 | |
| *** HenryG has quit IRC | 01:04 | |
| *** sballe has quit IRC | 01:04 | |
| *** ramishra_ has quit IRC | 01:04 | |
| *** Aish_ is now known as Aish | 01:04 | |
| *** ctracey_ is now known as ctracey | 01:05 | |
| *** Aish has quit IRC | 01:07 | |
| *** sballe has joined #openstack-lbaas | 01:07 | |
| *** ramishra__ is now known as ramishra_ | 01:08 | |
| *** piet has quit IRC | 01:11 | |
| *** diogogmt has joined #openstack-lbaas | 01:13 | |
| *** armax has quit IRC | 01:14 | |
| *** armax has joined #openstack-lbaas | 01:16 | |
| *** armax has quit IRC | 01:37 | |
| *** barra204 has quit IRC | 01:53 | |
| openstackgerrit | Doug Wiegley proposed openstack/neutron-lbaas: Janky hack to get gate passing https://review.openstack.org/274272 | 02:04 |
| *** bana_k has quit IRC | 02:13 | |
| *** harlowja has quit IRC | 02:30 | |
| *** harlowja has joined #openstack-lbaas | 02:30 | |
| *** davidlenwell has quit IRC | 02:36 | |
| *** davidlenwell has joined #openstack-lbaas | 02:39 | |
| *** minwang2 has joined #openstack-lbaas | 02:47 | |
| *** fnaval has quit IRC | 02:48 | |
| *** liamji has joined #openstack-lbaas | 02:52 | |
| *** fnaval has joined #openstack-lbaas | 02:55 | |
| *** piet has joined #openstack-lbaas | 02:59 | |
| *** woodster_ has quit IRC | 03:06 | |
| *** SumitNaiksatam has joined #openstack-lbaas | 03:23 | |
| *** johnsom_ has joined #openstack-lbaas | 03:37 | |
| *** minwang2 has quit IRC | 03:47 | |
| *** minwang2 has joined #openstack-lbaas | 04:05 | |
| *** Aish has joined #openstack-lbaas | 04:14 | |
| *** Aish has quit IRC | 04:14 | |
| *** piet has quit IRC | 04:14 | |
| johnsom_ | FYI, going to take a crack at that next layer of the gate onion | 04:16 |
| *** ducttape_ has joined #openstack-lbaas | 04:23 | |
| sbalukoff | johnsom! | 04:28 |
| sbalukoff | johnsom_: I appreciate that! | 04:28 |
| openstackgerrit | Michael Johnson proposed openstack/neutron-lbaas: Janky hack to get gate passing https://review.openstack.org/274272 | 04:29 |
| sbalukoff | Also, I'm trying to test the active_standby stuff... I just spun up a totally new devstack with a fresh environment... and I'm still getting the same behavior. This tells me I must be doing something wrong. | 04:29 |
| johnsom_ | Might be an vertical up hill battle. keystone made the tenant->project switch and our in-tree tempest isn't up for it | 04:29 |
| johnsom_ | sbalukoff Oh? What is happening? | 04:29 |
| sbalukoff | What all do I need to do to try out active-standby in a devstack that differs from stand-alone.. | 04:30 |
| sbalukoff | OK! | 04:30 |
| sbalukoff | So, I try to deploy a load balancer with neutron-lbaas. | 04:30 |
| sbalukoff | I see two amphorae get spawned and I'm eventually able to log into both. | 04:30 |
| johnsom_ | Should just be changing the octavia.conf setting | 04:30 |
| sbalukoff | I see the octavia database gets up dated with their roles, and the master gets the VIP address on eth1 | 04:30 |
| johnsom_ | ok, cool. That sounds good so far. | 04:31 |
| sbalukoff | But, no keepalive configuration gets pushed out, and the load_balancer object stays in PENDING_CREATE | 04:31 |
| sbalukoff | eventually a timeout occurs and the load balancer goes into ERROR state. | 04:31 |
| sbalukoff | that's as far as I've gotten, so I've been assuming I must be doing something screwey in my environment, so I figured I should ask y'all. | 04:32 |
| johnsom_ | Hmmm, ok. I suspect the image is bad. After you do a clean.sh, delete the amphora qcow2 in /opt/stack/octavia/diskimage-create | 04:32 |
| sbalukoff | Again, stand-alone works fine. | 04:32 |
| johnsom_ | There is a strange issue with DIB that it won't overwrite the old image | 04:32 |
| sbalukoff | Oh-- this was on a totally fresh devstack. | 04:33 |
| sbalukoff | So the image was defnitely created when the stack was built. | 04:33 |
| johnsom_ | Ok. Hmm, on the amp, check the /var/log/upstart/amphora-agent.log | 04:33 |
| johnsom_ | Is it clean? | 04:33 |
| sbalukoff | And again, I can run stand-alone just fine... so the image works for that at least. | 04:33 |
| sbalukoff | Let me look. | 04:33 |
| johnsom_ | Yeah, it just sounds like the agent is not working right | 04:34 |
| sbalukoff | 2016-01-30 04:18:56.549 496 INFO werkzeug [-] 192.168.0.3 - - [30/Jan/2016 04:18 | 04:35 |
| sbalukoff | :56] "POST /0.5/plug/vip/172.24.4.3 HTTP/1.1" 202 - | 04:35 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.ser | 04:35 |
| sbalukoff | ver [-] Exception on /0.5/interface/172.24.4.4 [GET] | 04:35 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.ser | 04:35 |
| sbalukoff | ver Traceback (most recent call last): | 04:35 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.ser | 04:35 |
| sbalukoff | ver File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in | 04:35 |
| sbalukoff | wsgi_app | 04:35 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.ser | 04:35 |
| sbalukoff | ver response = self.full_dispatch_request() | 04:35 |
| sbalukoff | So... that's fun. | 04:36 |
| sbalukoff | There's more, but I figure the rest of this channel doesn't want to see it. | 04:36 |
| sbalukoff | Ok! | 04:37 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.server for i in netifaces.ifaddresses(interface)[address_format]: | 04:37 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.server KeyError: 2 | 04:37 |
| sbalukoff | 2016-01-30 04:24:38.342 496 ERROR octavia.amphorae.backends.agent.api_server.server | 04:37 |
| sbalukoff | D'oh! | 04:37 |
| johnsom_ | Yeah, hmmm, definitely a bad version of the agent. You can check the version by going to /opt/<something> I forgot, there is a octavia git checkout in there, check the versino | 04:37 |
| sbalukoff | So yeah... but in the agent? | 04:37 |
| sbalukoff | bug | 04:37 |
| sbalukoff | Ok... | 04:37 |
| sbalukoff | It's running off the master | 04:38 |
| *** ducttape_ has quit IRC | 04:38 | |
| johnsom_ | Well, that is neat | 04:38 |
| sbalukoff | Latest commit is: commit 79179de31094782c03e21ca51fb179fd862f6c58 | 04:38 |
| sbalukoff | Author: OpenStack Proposal Bot <openstack-infra@lists.openstack.org> | 04:38 |
| sbalukoff | Date: Wed Jan 27 14:23:51 2016 +0000 | 04:38 |
| sbalukoff | Which I think really was the last merge. | 04:38 |
| sbalukoff | So... I've stumbled across a bug in the amphora agent, eh? | 04:39 |
| sbalukoff | Ok, well at least *that* gives me something to look into. | 04:39 |
| johnsom_ | Yeah, so ok, we have a good image. This is definitely a bug. Something unique in your environment. | 04:39 |
| johnsom_ | gist your ifconfig? | 04:39 |
| sbalukoff | Sure... standby. | 04:39 |
| johnsom_ | and ip addr | 04:39 |
| sbalukoff | Oh! This is definitely different than before. | 04:40 |
| johnsom_ | Inside the amp BTW | 04:40 |
| sbalukoff | So whatever revert tasks were attempted killed the eth1 information. | 04:41 |
| johnsom_ | Sigh, yeah, they do... | 04:41 |
| sbalukoff | I can re-run this and catch it while it's live. | 04:41 |
| sbalukoff | But that'll probably take me a while. | 04:41 |
| sbalukoff | (It's sooooo annoying how long these amphora take to spin up, even on bare metal.) | 04:41 |
| johnsom_ | If you have vt-x it's like 30 seconds | 04:42 |
| johnsom_ | Without, it's 2-8 minutes | 04:42 |
| johnsom_ | Fun with qemu | 04:42 |
| sbalukoff | Eh... it's an older machine. Does have vt-x. But then... also dealing with DNS timeouts until my fix for that merges. | 04:43 |
| johnsom_ | I'm poking at the gate issues.... At least zuul is faster on a friday night. | 04:43 |
| sbalukoff | Yep. | 04:44 |
| sbalukoff | Ok, well, actually, I'm going to build an image with DNS disabled so that I can iterate on troubleshooting this faster. | 04:44 |
| sbalukoff | I do hope you're able to clear those gate issues. :P | 04:44 |
| sbalukoff | I don't suppose there's a simple way to troubleshoot that locally, eh? | 04:45 |
| sbalukoff | (Because iterating on gate jobs is a serious PITA... as I'm sure you're well aware. XD ) | 04:45 |
| johnsom_ | Yeah, I could run those tests local, it's just I'm not setup to run those at the moment. | 04:46 |
| sbalukoff | Well, I hope you're doing something fun while you wait for the gate... | 04:47 |
| johnsom_ | Well, I'm watching TV with the wife. That's why I say poking at it | 04:48 |
| xgerman | johnsom_ appreciated | 04:49 |
| johnsom_ | Check your host for /proc/cpuinfo to have vmx or svm , that is what qemu decides on and really impacts the boot times | 04:49 |
| *** yamamoto has joined #openstack-lbaas | 04:50 | |
| *** yamamoto has quit IRC | 04:51 | |
| sbalukoff | It's in there. | 04:51 |
| *** kobis has joined #openstack-lbaas | 04:53 | |
| *** kobis has quit IRC | 04:59 | |
| openstackgerrit | Michael Johnson proposed openstack/neutron-lbaas: Janky hack to get gate passing https://review.openstack.org/274272 | 05:01 |
| johnsom_ | Well, that was annoying, I missed two line updates in the last patch. round 2 | 05:01 |
| sbalukoff | Heh! | 05:03 |
| sbalukoff | Well, again, at least the gate is quick on Fridays. | 05:03 |
| johnsom_ | And it is slightly better than nothing tried | 05:03 |
| sbalukoff | I'm completing IBM's business conduct guideline training (which mostly involved watching videos)... gotta do it yearly before mid-February, apparently. :P | 05:04 |
| johnsom_ | Yeah, we have a similar video yearly | 05:05 |
| johnsom_ | I love the joke in our gates: Running devstack this takes 10 - 15 minutes... | 05:07 |
| johnsom_ | That never happens | 05:07 |
| sbalukoff | Heh! | 05:10 |
| sbalukoff | Cutting down on the image build time should help a lot. | 05:10 |
| *** kobis has joined #openstack-lbaas | 05:12 | |
| johnsom_ | This is a v1 gate I'm watching, so don't have the image penalty. Still doubt it will finish in 10-15 | 05:13 |
| sbalukoff | Heh! | 05:13 |
| *** piet has joined #openstack-lbaas | 05:15 | |
| openstackgerrit | Michael Johnson proposed openstack/neutron-lbaas: Optimize API gate devstack build time https://review.openstack.org/274319 | 05:25 |
| johnsom_ | Well, that should take another chunk of time out of the API based gates | 05:25 |
| *** armax has joined #openstack-lbaas | 05:30 | |
| sbalukoff | Nice. | 05:31 |
| *** piet has quit IRC | 05:35 | |
| *** piet has joined #openstack-lbaas | 05:35 | |
| openstackgerrit | Michael Johnson proposed openstack/neutron-lbaas: Janky hack to get gate passing https://review.openstack.org/274272 | 05:41 |
| *** kobis has quit IRC | 05:41 | |
| johnsom_ | Ok, we have a chance with this one | 05:42 |
| *** piet has quit IRC | 05:47 | |
| *** piet has joined #openstack-lbaas | 05:50 | |
| sbalukoff | I'll keep my fingers crossed! | 05:53 |
| *** bana_k has joined #openstack-lbaas | 05:58 | |
| johnsom_ | Looking good. It's a horrible hack, but Cedric has a patch coming to fix this mess, so I don't feel too bad. | 06:07 |
| sbalukoff | Heh! | 06:15 |
| sbalukoff | Ok, I submitted a bug report for the active-standby booting bug I encountered with logs and stuff. | 06:15 |
| sbalukoff | (I'm a bit fried after a long day today to look much further into that one.) | 06:16 |
| johnsom_ | Ok, I will take a look | 06:18 |
| sbalukoff | Good work on the gate-fixing hack. Looks like it might work this time! | 06:21 |
| *** piet has quit IRC | 06:35 | |
| *** minwang2 has quit IRC | 06:45 | |
| dougwig | johnsom: your update did the trick. | 06:48 |
| johnsom_ | Just added another layer of hack | 06:49 |
| johnsom_ | Since it is your patch I will give it a +2 | 06:50 |
| johnsom_ | It looks like your A10 gate is checking out the wrong code though... | 06:50 |
| johnsom_ | Ha, or not... | 06:52 |
| sbalukoff | Yay! | 06:53 |
| * sbalukoff watches it slooowly getting merged.... | 06:53 | |
| dougwig | johnsom: that's an old build. | 06:55 |
| dougwig | johnsom: oh yes, it's pulling master, i think. there's a reason it's not voting yet. it just got rebuilt (the CI). | 06:56 |
| johnsom_ | Ah, got it. | 06:56 |
| dougwig | let me go fix *that* right now. | 06:56 |
| *** johnsom_ has quit IRC | 07:01 | |
| *** johnsom_ has joined #openstack-lbaas | 07:02 | |
| *** rcernin has joined #openstack-lbaas | 07:28 | |
| openstackgerrit | Merged openstack/neutron-lbaas: Janky hack to get gate passing https://review.openstack.org/274272 | 07:30 |
| sbalukoff | Yay, merged! | 07:31 |
| openstackgerrit | Michael Johnson proposed openstack/neutron-lbaas: Optimize API gate devstack build time https://review.openstack.org/274319 | 07:32 |
| johnsom_ | Rebase that. | 07:32 |
| johnsom_ | Go team. No, really, go to bed. At least I am. Catch you all next week. | 07:33 |
| johnsom_ | rm_work FYI, this needs updated: https://wiki.openstack.org/wiki/ThirdPartySystems/HP_Octavia_Sonar_CI | 07:36 |
| *** johnsom_ has quit IRC | 07:42 | |
| *** rcernin has quit IRC | 08:18 | |
| *** sbalukoff has quit IRC | 08:35 | |
| *** sbalukoff has joined #openstack-lbaas | 08:35 | |
| *** armax has quit IRC | 09:35 | |
| *** eranra_ has joined #openstack-lbaas | 09:42 | |
| *** eranra has quit IRC | 09:46 | |
| *** _laco has quit IRC | 09:50 | |
| *** _laco has joined #openstack-lbaas | 10:05 | |
| openstackgerrit | Merged openstack/neutron-lbaas: Update translation setup https://review.openstack.org/273783 | 10:32 |
| *** drajcan has joined #openstack-lbaas | 10:54 | |
| drajcan | Hi all | 10:54 |
| drajcan | please could anyone help me ? | 10:55 |
| *** eranra_ has quit IRC | 11:21 | |
| *** eranra has joined #openstack-lbaas | 11:23 | |
| openstackgerrit | Cedric Brandily proposed openstack/neutron-lbaas: Delete tempest fork https://review.openstack.org/273817 | 11:49 |
| *** piet has joined #openstack-lbaas | 12:22 | |
| *** piet has quit IRC | 12:38 | |
| *** drajcan has left #openstack-lbaas | 12:50 | |
| *** yamamoto has joined #openstack-lbaas | 12:52 | |
| *** paco20151113 has joined #openstack-lbaas | 13:01 | |
| *** alejandrito has joined #openstack-lbaas | 13:01 | |
| paco20151113 | hi, Is there anyone have information for how to set up LBv2 Octavia on multi-node ? | 13:02 |
| paco20151113 | I have successfully run it with devstack. | 13:03 |
| paco20151113 | but can not manaully set up with mulit node . | 13:03 |
| *** yamamoto has quit IRC | 13:05 | |
| *** yamamoto has joined #openstack-lbaas | 13:05 | |
| *** yamamoto has quit IRC | 13:09 | |
| *** alejandrito has quit IRC | 13:09 | |
| *** yamamoto has joined #openstack-lbaas | 13:12 | |
| *** yamamoto has quit IRC | 13:15 | |
| *** yamamoto has joined #openstack-lbaas | 13:16 | |
| *** yamamoto has quit IRC | 13:21 | |
| *** _ducttape_ has joined #openstack-lbaas | 13:35 | |
| *** _ducttape_ has quit IRC | 13:41 | |
| *** _ducttape_ has joined #openstack-lbaas | 14:02 | |
| *** _ducttape_ has quit IRC | 14:10 | |
| *** _ducttape_ has joined #openstack-lbaas | 14:44 | |
| openstackgerrit | Cedric Brandily proposed openstack/neutron-lbaas: Delete tempest fork https://review.openstack.org/273817 | 15:01 |
| *** longstaff has joined #openstack-lbaas | 15:14 | |
| *** _ducttape_ has quit IRC | 15:18 | |
| *** piet has joined #openstack-lbaas | 15:56 | |
| openstackgerrit | Cedric Brandily proposed openstack/neutron-lbaas: Delete tempest fork https://review.openstack.org/273817 | 16:05 |
| *** longstaff has quit IRC | 16:28 | |
| *** piet has quit IRC | 16:28 | |
| openstackgerrit | Cedric Brandily proposed openstack/neutron-lbaas: Delete tempest fork https://review.openstack.org/273817 | 16:35 |
| *** paco20151113 has quit IRC | 16:37 | |
| *** liamji has quit IRC | 17:11 | |
| *** piet has joined #openstack-lbaas | 17:14 | |
| *** longstaff has joined #openstack-lbaas | 17:19 | |
| *** longstaff has quit IRC | 17:26 | |
| *** kobis has joined #openstack-lbaas | 17:30 | |
| *** longstaff has joined #openstack-lbaas | 17:32 | |
| openstackgerrit | Cedric Brandily proposed openstack/neutron-lbaas: Delete tempest fork https://review.openstack.org/273817 | 17:41 |
| *** piet has quit IRC | 17:53 | |
| *** longstaff has quit IRC | 17:53 | |
| *** kobis has quit IRC | 17:53 | |
| *** bdrich has joined #openstack-lbaas | 18:00 | |
| *** piet has joined #openstack-lbaas | 18:00 | |
| *** longstaff has joined #openstack-lbaas | 18:03 | |
| *** eranra_ has joined #openstack-lbaas | 18:07 | |
| *** longstaff has quit IRC | 18:08 | |
| *** _ducttape_ has joined #openstack-lbaas | 18:09 | |
| *** eranra has quit IRC | 18:10 | |
| *** _ducttape_ has quit IRC | 18:20 | |
| *** minwang2 has joined #openstack-lbaas | 18:25 | |
| *** piet has quit IRC | 18:26 | |
| *** piet has joined #openstack-lbaas | 18:27 | |
| *** kobis has joined #openstack-lbaas | 18:39 | |
| *** piet has quit IRC | 18:43 | |
| *** bdrich has quit IRC | 19:05 | |
| *** longstaff has joined #openstack-lbaas | 19:08 | |
| *** longstaff has quit IRC | 19:13 | |
| openstackgerrit | Cedric Brandily proposed openstack/neutron-lbaas: Delete tempest fork https://review.openstack.org/273817 | 19:13 |
| *** longstaff has joined #openstack-lbaas | 19:59 | |
| *** kobis has quit IRC | 20:05 | |
| *** kobis has joined #openstack-lbaas | 20:42 | |
| *** minwang2 has quit IRC | 21:21 | |
| *** crc32 has joined #openstack-lbaas | 21:27 | |
| *** longstaff has quit IRC | 21:30 | |
| *** longstaff has joined #openstack-lbaas | 21:38 | |
| *** kobis has quit IRC | 21:40 | |
| *** kobis has joined #openstack-lbaas | 21:40 | |
| *** longstaff has quit IRC | 21:42 | |
| *** kobis has quit IRC | 21:45 | |
| *** minwang2 has joined #openstack-lbaas | 21:45 | |
| *** crc32 has quit IRC | 22:04 | |
| *** longstaff has joined #openstack-lbaas | 22:11 | |
| *** piet_ has joined #openstack-lbaas | 22:12 | |
| *** longstaff has quit IRC | 22:15 | |
| *** longstaff has joined #openstack-lbaas | 22:19 | |
| *** piet_ has quit IRC | 22:32 | |
| *** kobis has joined #openstack-lbaas | 22:41 | |
| *** kobis has quit IRC | 22:45 | |
| *** SumitNaiksatam_ has joined #openstack-lbaas | 23:09 | |
| *** SumitNaiksatam has quit IRC | 23:10 | |
| *** SumitNaiksatam_ is now known as SumitNaiksatam | 23:10 | |
| *** minwang2 has quit IRC | 23:14 | |
| *** minwang2 has joined #openstack-lbaas | 23:48 | |
| *** minwang2 has quit IRC | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!