| *** diogogmt has quit IRC | 00:01 | |
| *** vivek-ebay has joined #openstack-lbaas | 00:01 | |
| *** diogogmt has joined #openstack-lbaas | 00:02 | |
| *** jongchoi has quit IRC | 00:02 | |
| *** vivek-ebay has quit IRC | 00:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/octavia: Updated from global requirements https://review.openstack.org/239028 | 00:26 |
|---|---|---|
| openstackgerrit | Carlos Garza proposed openstack/octavia: Implementing EventStreamer https://review.openstack.org/218735 | 01:02 |
| crc32 | rebase | 01:02 |
| *** bharathm has quit IRC | 01:03 | |
| *** KunalGandhi has quit IRC | 01:13 | |
| *** crc32 has quit IRC | 01:29 | |
| *** blogan_ has quit IRC | 01:32 | |
| *** bank has quit IRC | 01:41 | |
| *** alejandrito has quit IRC | 01:46 | |
| *** diogogmt_ has joined #openstack-lbaas | 01:53 | |
| *** diogogmt has quit IRC | 01:54 | |
| *** diogogmt_ is now known as diogogmt | 01:54 | |
| *** vivek-ebay has joined #openstack-lbaas | 02:04 | |
| *** vivek-ebay has quit IRC | 02:08 | |
| rm_work | blogan: you there? | 02:28 |
| rm_work | ffff us texans are f'ed | 02:29 |
| rm_work | flooding due to hurricane | 02:29 |
| rm_work | wrecking a ton of flights tomorrow | 02:29 |
| sbalukoff | Oh geez! that sucks! | 02:40 |
| rm_work | woking through options with airline rep now <_< | 02:41 |
| sbalukoff | Good luck! | 02:41 |
| rm_work | i think blogan's flight may be affected too but not sure | 02:41 |
| sbalukoff | johnsom: What are the login credentials for that virtual machine? | 03:07 |
| *** sbalukoff has quit IRC | 03:21 | |
| *** minwang2 has joined #openstack-lbaas | 03:29 | |
| *** sbalukoff has joined #openstack-lbaas | 03:31 | |
| *** bharathm has joined #openstack-lbaas | 03:39 | |
| *** woodster_ has quit IRC | 03:39 | |
| *** bharathm has quit IRC | 03:49 | |
| *** armax has quit IRC | 03:54 | |
| *** minwang2 has quit IRC | 04:13 | |
| *** minwang2 has joined #openstack-lbaas | 04:19 | |
| rm_work | it was ubuntu:ubuntu | 04:19 |
| rm_work | sbalukoff1: | 04:19 |
| rm_work | sbalukoff: | 04:19 |
| *** diogogmt has quit IRC | 04:21 | |
| rm_work | johnsom: so it resumes, but there's no nova instances -- did you mean to set up the two lb backend nodes like were setup before? | 04:46 |
| rm_work | oh nm i see them | 04:46 |
| rm_work | they're on admin now, instead of demo | 04:46 |
| rm_work | not sure if that's what you meant to do | 04:46 |
| rm_work | cool, well, did the whole demo as "admin" instead of the actual demo user | 04:51 |
| rm_work | but it works | 04:52 |
| rm_work | so that's cool :P | 04:52 |
| rm_work | not sure if we want to try to fix that | 04:52 |
| rm_work | johnsom: is the address in the "simulate a failover section actually right? 172.24.4.1:5555 | 04:57 |
| rm_work | I feel like that's ... not related anymore | 04:58 |
| *** yamamoto has quit IRC | 05:09 | |
| *** vivek-ebay has joined #openstack-lbaas | 05:13 | |
| *** minwang2 has quit IRC | 05:20 | |
| *** minwang2 has joined #openstack-lbaas | 05:24 | |
| *** jongchoi has joined #openstack-lbaas | 05:33 | |
| sbalukoff | rm_work: Hmmm... the failover didn't work for me. | 05:43 |
| sbalukoff | I'm not sure what I'm doing wrong there... | 05:43 |
| sbalukoff | And yes, I ran the whole demo as the admin user as well. | 05:44 |
| sbalukoff | Also, is it a known thing that a devstack doesn't survive a reboot? (ie. I'm guessing you're supposed to restack after every restart of the machine?) | 05:45 |
| sbalukoff | I've noticed I have to do this every time on other devstacks I've used... I'm just wondering whether I'm doing something wrong. :/ | 05:45 |
| sbalukoff | In any case, it's nice to have this saved VMware image-- it comes back to a "restored" state pretty quickly, and you've got everything already running, ready to do the demo. | 05:46 |
| sbalukoff | Other than the failover which didn't work for me... I'm feeling pretty confident about our lab. Good work there, johnsom, eh! | 05:46 |
| *** minwang2 has quit IRC | 05:46 | |
| *** alejandrito has joined #openstack-lbaas | 05:48 | |
| sbalukoff | Also learned today: Do not attempt to run 12+ GB of virtual machines on a laptop with 16GB ram while also running a browser with 50 tabs open. And stuff. | 05:48 |
| sbalukoff | ;) | 05:48 |
| openstackgerrit | Merged openstack/neutron-lbaas: Adds a Barbican option to local.conf https://review.openstack.org/237125 | 05:51 |
| johnsom | Sorry, took a break. I didn't realize xgerman did demo account. I did everything as admin like I usually do. | 06:08 |
| johnsom | 172.24.4.1:5555 is the "hack" work around for the controller IP. It isn't the right answer, but works for now | 06:08 |
| johnsom | Reboots always break devstack for me. I always have to restack. So, no answers there | 06:09 |
| johnsom | Surprised failover doesn't work. Did you make sure to set the controller IP setting and restart cw and hm? | 06:10 |
| johnsom | Come to think about it. 172.24.4.1 might not be the right answer with this image... | 06:13 |
| johnsom | Will have to look at that tomorrow | 06:14 |
| johnsom | The static IP may have changed that. | 06:14 |
| *** vivek-ebay has quit IRC | 06:21 | |
| *** yamamoto has joined #openstack-lbaas | 06:22 | |
| sbalukoff | Aah-- I only restarted cw, not hm. | 06:39 |
| sbalukoff | That would probably be why it didn't work for me. | 06:39 |
| sbalukoff | So, I think the instructions only say to restart cw. Should probably add hm in there as well. | 06:39 |
| *** yamamoto has quit IRC | 06:49 | |
| *** bharathm has joined #openstack-lbaas | 06:50 | |
| *** yamamoto has joined #openstack-lbaas | 06:51 | |
| *** bharathm has quit IRC | 06:54 | |
| *** yamamoto has quit IRC | 07:12 | |
| *** yamamoto has joined #openstack-lbaas | 07:13 | |
| *** yamamoto has quit IRC | 07:24 | |
| *** alejandrito has quit IRC | 07:30 | |
| *** jongchoi has quit IRC | 07:39 | |
| *** jongchoi has joined #openstack-lbaas | 07:43 | |
| *** denghuiH has joined #openstack-lbaas | 08:06 | |
| *** alejandrito has joined #openstack-lbaas | 09:06 | |
| *** jongchoi has quit IRC | 09:35 | |
| *** yamamoto has joined #openstack-lbaas | 09:53 | |
| *** jongchoi has joined #openstack-lbaas | 10:43 | |
| *** alejandrito has quit IRC | 10:52 | |
| *** yamamoto has quit IRC | 11:26 | |
| *** jongchoi has quit IRC | 12:05 | |
| *** jongchoi has joined #openstack-lbaas | 12:13 | |
| *** orionx has quit IRC | 12:27 | |
| *** jongchoi has quit IRC | 12:40 | |
| *** denghuiH has quit IRC | 12:44 | |
| *** diogogmt has joined #openstack-lbaas | 12:58 | |
| *** jongchoi has joined #openstack-lbaas | 13:03 | |
| *** yamamoto has joined #openstack-lbaas | 13:08 | |
| *** yamamoto has quit IRC | 13:13 | |
| *** chlong has joined #openstack-lbaas | 13:29 | |
| *** diogogmt has quit IRC | 13:34 | |
| *** jongchoi has quit IRC | 13:46 | |
| *** yamamoto has joined #openstack-lbaas | 13:49 | |
| *** jongchoi has joined #openstack-lbaas | 13:49 | |
| *** diogogmt has joined #openstack-lbaas | 13:52 | |
| *** yamamoto has quit IRC | 13:54 | |
| *** jongchoi has quit IRC | 14:03 | |
| *** yamamoto has joined #openstack-lbaas | 14:04 | |
| *** alejandrito has joined #openstack-lbaas | 14:09 | |
| *** apuimedo has joined #openstack-lbaas | 15:05 | |
| *** alejandrito has quit IRC | 15:08 | |
| *** orionx has joined #openstack-lbaas | 15:29 | |
| *** orionx has quit IRC | 15:33 | |
| *** orionx has joined #openstack-lbaas | 15:34 | |
| *** yamamoto has quit IRC | 15:48 | |
| *** armax has joined #openstack-lbaas | 16:01 | |
| rm_work | sbalukoff / sbalukoff1 / johnsom: yeah not sure if that address is right since the image IPs all changed, there's no 172.* on that box at all | 16:02 |
| rm_work | also, on a reboot you should be able to just use rejoin-stack.sh | 16:03 |
| rm_work | not actually restack | 16:03 |
| rm_work | it just starts everything again quickly | 16:03 |
| rm_work | has "worked" for me | 16:03 |
| rm_work | you have to manually start any nova VMs tho | 16:03 |
| *** vivek-ebay has joined #openstack-lbaas | 16:15 | |
| *** diogogmt has quit IRC | 16:20 | |
| *** zigo has quit IRC | 16:38 | |
| *** zigo has joined #openstack-lbaas | 16:38 | |
| *** armax has quit IRC | 16:43 | |
| *** minwang2 has joined #openstack-lbaas | 16:44 | |
| *** yamamoto has joined #openstack-lbaas | 16:49 | |
| *** zigo has quit IRC | 16:49 | |
| *** zigo has joined #openstack-lbaas | 16:51 | |
| *** yamamoto has quit IRC | 16:54 | |
| johnsom | I never get rejoin to actually work. But maybe that is fixed now. The processes are going but the bridge interfaces are scrambled | 17:19 |
| johnsom | The address should be the gateway on the public net I think | 17:20 |
| johnsom | I will give it a go later and find out | 17:20 |
| *** diogogmt has joined #openstack-lbaas | 17:29 | |
| *** diogogmt has quit IRC | 17:48 | |
| *** diogogmt has joined #openstack-lbaas | 17:50 | |
| *** minwang2 has quit IRC | 18:10 | |
| *** vivek-ebay has quit IRC | 18:11 | |
| *** vivek-ebay has joined #openstack-lbaas | 18:12 | |
| *** vivek-ebay has quit IRC | 18:17 | |
| *** johnsom_ has joined #openstack-lbaas | 18:27 | |
| *** johnsom has quit IRC | 18:29 | |
| rm_work | johnsom_: if you used HOST_IP=127.0.0.1 nothing gets scrambled because the addresses can't change :P | 18:44 |
| rm_work | johnsom_: that is the primary benefit of doing it that way, I believe | 18:44 |
| rm_work | the bridge interfaces for nova and neutron SHOULDN'T break, but it's possible I guess (might explain some of the wonky stuff we were seeing on German's image actually) | 18:45 |
| *** apuimedo has quit IRC | 18:48 | |
| johnsom_ | It's the neutron interfaces that broke for me. It's not that the host IP changes, it's static, it's that the interfaces don't all get put into the files for startup | 18:53 |
| *** johnsom_ is now known as johnsom | 18:55 | |
| johnsom | rm_work The public net is still 172.24.4.0/24 so 172.24.4.1 should still be correct | 19:02 |
| johnsom | It is on br-ex | 19:03 |
| rm_work | hmm k | 19:03 |
| rm_work | i don't see it | 19:04 |
| rm_work | weird | 19:04 |
| johnsom | neutron net-list | 19:04 |
| rm_work | i did: | 19:04 |
| johnsom | What is your public lan | 19:04 |
| johnsom | ? | 19:04 |
| rm_work | ifconfig | grep 172 | 19:04 |
| rm_work | and nothing | 19:04 |
| rm_work | does it not show up as a bound interface? | 19:04 |
| johnsom | I have it on mine | 19:05 |
| johnsom | sudo ifconfig br-ex gives me the 172.24.4.1 address | 19:05 |
| johnsom | If you rebooted you may have lost that interface | 19:06 |
| *** bank has joined #openstack-lbaas | 19:09 | |
| rm_work | i did not... | 19:16 |
| rm_work | oh, no nevermind, it's showing up now wtf | 19:17 |
| rm_work | k, well | 19:17 |
| rm_work | maybe i was just hallucinating last night | 19:17 |
| rm_work | or typo'd somethinf | 19:17 |
| rm_work | johnsom: oh you will need to have them do | 19:18 |
| johnsom | Or forgot the sudo | 19:18 |
| rm_work | sudo chmod go+rw `tty` | 19:18 |
| rm_work | nah the interfaces don't change whether you sudo or not | 19:18 |
| johnsom | Yeah, xgerman put the "script /dev/null" trick in the slides | 19:18 |
| rm_work | just have to run it from /sbin/ifconfig explicitly | 19:18 |
| rm_work | oh | 19:18 |
| rm_work | ehhh | 19:18 |
| rm_work | that is... less optimal | 19:18 |
| johnsom | Yeah, but if you don't sudo ifconfig gives an error so the grep will come up empty | 19:18 |
| rm_work | than just fixing the tty perms | 19:18 |
| rm_work | johnsom: it ... doesn't? | 19:19 |
| rm_work | ifconfig runs fine as any user | 19:19 |
| rm_work | it's just a debianism that it's in sbin | 19:19 |
| johnsom | It's not in the path for stack user I think | 19:19 |
| rm_work | no actual permissions issue | 19:19 |
| rm_work | right | 19:19 |
| rm_work | you just have to run it from /sbin explicitly | 19:19 |
| johnsom | I usually use the chmod trick, this was the first I had heard of the script trick | 19:19 |
| rm_work | yeah the script trick has weird side effects tho | 19:19 |
| johnsom | So, failover works. If you don't nova delete the instance. | 19:20 |
| johnsom | It looks like the failover flow reverts if the instance was deleted. It tries to delete it and fails | 19:20 |
| rm_work | hmm | 19:20 |
| rm_work | about to test it | 19:20 |
| rm_work | just restarted o-cw and o-hm | 19:20 |
| johnsom | The steps are: update octavia.conf with controller IP/port, restart cm and hm, create a new load balancer | 19:21 |
| rm_work | yeah | 19:21 |
| rm_work | doing that | 19:21 |
| johnsom | Then stop the haproxy on the amp, wait 10 or 30 seconds, whatever is in the conf | 19:21 |
| johnsom | I just opened https://bugs.launchpad.net/octavia/+bug/1509706 for that instance delete issue | 19:22 |
| openstack | Launchpad bug 1509706 in octavia "Failover fails when instance is deleted" [Critical,New] | 19:22 |
| xgerman | in airport | 19:23 |
| rm_work | xgerman: >_> | 19:24 |
| rm_work | xgerman: at home waiting for rescheduled flight tomorrow at 6am >_> | 19:24 |
| xgerman | :-( | 19:24 |
| xgerman | charging all my devices again... | 19:25 |
| johnsom | so, since we can't just delete the instance, we should stop the amphora-agent | 19:25 |
| rm_work | k | 19:26 |
| johnsom | Since haproxy isn't running until a listener is configured | 19:26 |
| rm_work | how long does it take to fail over | 19:26 |
| johnsom | Then you can stop the haproxy instance | 19:26 |
| *** woodster_ has joined #openstack-lbaas | 19:26 | |
| rm_work | ok there it went | 19:26 |
| rm_work | it took a sec | 19:26 |
| johnsom | Yeah, the default config as a long interval | 19:27 |
| rm_work | but it worked | 19:27 |
| rm_work | cool k | 19:27 |
| rm_work | this is using the UDP heartbeat stuff? :P | 19:27 |
| rm_work | right? | 19:27 |
| johnsom | Yep | 19:27 |
| rm_work | cool | 19:27 |
| xgerman | yeah | 19:27 |
| rm_work | nice to see it actually *working* | 19:27 |
| rm_work | awesome | 19:27 |
| rm_work | ok, i'll start imaging this tonight | 19:27 |
| rm_work | or maybe right now | 19:27 |
| johnsom | Cool! Thanks Adam! | 19:28 |
| rm_work | so many drives to do, and they are soooo slow >_> | 19:28 |
| xgerman | now you have more time... | 19:28 |
| rm_work | i still haven't looked at the troubleshooting section, hopefully someone has filled in something T_T | 19:28 |
| rm_work | heh | 19:28 |
| xgerman | I did | 19:28 |
| rm_work | well, i WAS planning to do it on the plane :P | 19:28 |
| xgerman | we are good | 19:28 |
| rm_work | kk cool | 19:28 |
| rm_work | honestly I feel like it'd be more useful just to have a couple of us wandering around | 19:29 |
| johnsom | xgerman fyi, I updated the slide to include restarting the o-hm | 19:29 |
| rm_work | helping people do the commands to unpack everything, get stuff up and running, etc | 19:29 |
| rm_work | so the speaker can just keep doing the speaker thing and not be interrupted | 19:29 |
| xgerman | +1 | 19:29 |
| johnsom | Yeah, I think we will have as many of us as people | 19:29 |
| xgerman | johnsom thanks — worked for me without but who knows... | 19:29 |
| rm_work | heh yeah | 19:30 |
| rm_work | seems that way | 19:30 |
| johnsom | the o-hm might only be needed to failover the failover instance | 19:32 |
| johnsom | Ok, I am going to stop monkeying with the image, it seems to be in good shape. | 19:33 |
| johnsom | I wrote some slick scripts for the demo last night, so I need to make the recordings | 19:34 |
| *** bank has quit IRC | 19:35 | |
| *** bank has joined #openstack-lbaas | 19:35 | |
| rm_work | heh k | 19:59 |
| rm_work | imaging the first set | 19:59 |
| rm_work | then will test one of them | 19:59 |
| rm_work | looks like ~60m each to write | 19:59 |
| rm_work | doing 8 at a time <_< | 19:59 |
| rm_work | i love tmux pane-sync | 20:00 |
| *** ducttape_ has joined #openstack-lbaas | 20:38 | |
| *** ducttape_ has quit IRC | 20:42 | |
| *** armax has joined #openstack-lbaas | 20:47 | |
| johnsom | Ok, I have my demo videos and I have cut the presentation to clean up on the plane | 21:02 |
| *** vivek-ebay has joined #openstack-lbaas | 21:14 | |
| *** armax has quit IRC | 21:49 | |
| *** vivek-ebay has quit IRC | 21:52 | |
| *** openstackgerrit has quit IRC | 22:16 | |
| *** openstackgerrit has joined #openstack-lbaas | 22:17 | |
| johnsom | Interesting, NTT has a talk where they say they need UDP in LBaaS | 22:19 |
| johnsom | http://sched.co/4A8V | 22:19 |
| *** diogogmt has quit IRC | 22:21 | |
| *** ducttape_ has joined #openstack-lbaas | 22:29 | |
| *** ducttape_ has quit IRC | 22:48 | |
| *** bank has quit IRC | 22:50 | |
| *** bank has joined #openstack-lbaas | 22:54 | |
| *** bank has quit IRC | 23:14 | |
| *** harlowja has joined #openstack-lbaas | 23:26 | |
| *** harlowja has quit IRC | 23:26 | |
| *** openstack has joined #openstack-lbaas | 23:33 | |
| *** bank has joined #openstack-lbaas | 23:42 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!