Monday, 2017-05-01

JudeCjohnsom: In regard to your comment about searching for the subnet using its name or UUID. Is the best way to do that through the neutron API?00:01
*** aojea has quit IRC00:02
johnsomJudeC Yes, there should be other examples.  Let me see if I can find one00:02
JudeCsweet, just making sure I am barking up the right tree.00:03
johnsomThis: https://github.com/openstack/python-neutronclient/blob/master/neutronclient/osc/v2/trunk/network_trunk.py#L387 and https://github.com/openstack/python-neutronclient/blob/master/neutronclient/osc/v2/trunk/network_trunk.py#L32900:06
johnsomThough those might not be the best examples.  Let me keep looking00:06
johnsomHere is another way: https://github.com/openstack/python-novaclient/blob/master/novaclient/v2/networks.py#L3500:08
johnsomStill not sure that is great eitehr00:08
johnsomThere is this too in osc-lib: https://github.com/openstack/osc-lib/blob/master/osc_lib/api/api.py#L31400:11
*** bzhao has quit IRC00:11
JudeCok I think I can get the osc_lib way working pretty easily.00:12
JudeCthanks a bunch :)00:12
johnsomNP00:13
*** JudeC has quit IRC00:18
openstackgerritAdam Harwell proposed openstack/octavia master: Change auth_strategy default to keystone  https://review.openstack.org/45965400:35
rm_workjohnsom: this is pissing me off00:35
rm_worksomething about the wsgi change is broken00:35
rm_worksee #infra, but00:35
rm_work[09:13:17]  <rm_work>drwxr-xr-x 2 stack root 4096 Apr 30 23:48 /var/cache/octavia00:36
rm_work[09:13:25]  <rm_work>stack 7627 0.0 0.0 13340 1100 '?' S 23:48 0:00 grep apache2 root 27414 0.0 0.0 89992 5976 '?' Ss 23:39 0:00 /usr/sbin/apache2 -k start www-data 27417 0.0 0.1 448592 15116 '?' Sl 23:39 0:00 /usr/sbin/apache2 -k start www-data 27418 0.0 0.1 514144 13852 '?' Sl 23:39 0:00 /usr/sbin/apache2 -k start00:36
*** amotoki has joined #openstack-lbaas00:38
johnsomWell, we can specify the user to run it under...00:40
rm_workuser seems right00:40
rm_workbut can't access00:40
rm_work2017-05-01 00:05:27.362008 keystonemiddleware.exceptions.ConfigurationError: unable to access signing_dir /var/cache/octavia00:40
rm_workno idea why00:40
rm_workthis is the exact config glance has00:40
rm_worksame directory00:40
rm_worksame owners and such00:41
rm_worktrying to track down our wsgi script to see wtf it's trying to do00:41
johnsomWell, apache chowns the process.  Where id you get that log?   A previous run?00:41
rm_workso in this run00:42
rm_workhttp://logs.openstack.org/54/459654/9/check/gate-octavia-v1-dsvm-py3x-scenario-ubuntu-xenial/7f937da/logs/apache/octavia_error.txt.gz#_2017-05-01_00_05_27_36200800:42
*** amotoki has quit IRC00:42
rm_worklet me find it again00:42
rm_workhttp://logs.openstack.org/54/459654/9/check/gate-octavia-v1-dsvm-py3x-scenario-ubuntu-xenial/7f937da/logs/devstacklog.txt.gz#_2017-04-30_23_48_45_95600:43
rm_workI put some echos00:43
rm_workfor debugging00:43
rm_work    echo -e `ls -ld $OCTAVIA_AUTH_CACHE_DIR`00:43
rm_work    echo -e `ps aux | grep $APACHE_NAME`00:43
rm_workthe job that's running  now I just put chmod 777 to see if it's a permissions issue or like a chroot issue00:44
johnsomNo, it's most likely running under the wrong owner00:45
rm_workwell that grep shows `stack`00:45
johnsomIt looks like most teams are running under a service account, "octavia" for example.00:47
rm_workok00:48
rm_workyou want to fix it inside that patch?00:48
johnsomHmm.  My suspicion is we just need to add "user=" to the WSGIDaemonProcess  in the template.00:48
rm_workah k00:48
rm_workgive me the location and i can do it00:48
rm_workor you can do it00:48
johnsomWell, I'm not sure "octavia" is the right answer, as I'm not sure we set that up00:48
rm_workremove the 777 then00:48
rm_workk00:48
johnsomIt lives here: https://github.com/openstack/octavia/blob/master/devstack/files/wsgi/octavia-api.template#L2100:49
rm_workk00:50
johnsomOk, I think two things might need to happen.00:51
johnsom1. change the template to have user octavia, probably should be a sed from the settings file00:51
johnsom2. change the create_cache_dir in the devstack plugin to create the directory with octavia as the owenr00:52
johnsomWant me to do it?00:52
rm_workyeah IMO00:54
rm_worki mean00:54
rm_worki'm THERE00:54
rm_workbut kinda distracted00:54
rm_workif you want to just do it00:54
johnsomOk00:54
johnsomJust not sure what else I will break doing this, but I guess we will see00:55
johnsomWhy did you take this out?00:55
johnsominiset $OCTAVIA_CONF service_auth signing_dir $signing_dir00:55
rm_workit's deprecated in our config00:56
rm_workand doing the other keystone middleware line DOES IT00:56
rm_workinternally00:56
rm_workso ours is pointless00:56
rm_workconfigure_auth_token_middleware $OCTAVIA_CONF octavia $OCTAVIA_AUTH_CACHE_DIR00:57
openstackgerritMichael Johnson proposed openstack/octavia master: Change auth_strategy default to keystone  https://review.openstack.org/45965401:03
johnsomLet's try that01:03
rm_work$OCTAVIA is a username?01:08
rm_workhmm01:16
rm_worksomething didn't work...01:16
rm_workah yeah that doesn't create a SYSTEM user01:43
rm_workjust the keystone user01:43
rm_workjohnsom: ^^01:45
rm_workdon't know if we want to do that01:45
rm_workor just...01:45
rm_workactually use the STACK user01:45
rm_workit seems others just do that maybe...01:45
johnsomAh, ok, my quick glance at devstack scripts directed me wrong.01:46
johnsomIt just seems like everyone is using their own username for the apache wsgi01:47
johnsomhmmm01:47
*** JudeC has joined #openstack-lbaas01:47
rm_workhold on01:48
rm_workgonna put it back to stack user01:48
rm_worki think it's fine01:48
rm_workwith the other changes you made01:48
rm_work1 sec01:48
rm_workthere IS an $APACHE_USER01:49
rm_workbut01:49
openstackgerritAdam Harwell proposed openstack/octavia master: Change auth_strategy default to keystone  https://review.openstack.org/45965401:49
rm_workit defaults to same as $STACK_USER01:49
rm_workbut probably safest to use it anyway01:49
rm_workI believe with your user thing set it might work01:49
rm_worksee that change01:49
rm_work(also moved the service account create back to the old spot)01:49
rm_work(and put my debugs in again temporarily)01:50
rm_workjohnsom: ^^01:52
*** aojea has joined #openstack-lbaas01:58
*** aojea has quit IRC02:04
rm_workjohnsom: did you happen to catch the logs from my `chmod 777` attempt?02:19
rm_worki never actually resolved 100% that the issue was permissions02:19
rm_workbut hoping to see some results from this maybe... if we're lucky02:19
rm_workoh02:21
rm_workit looks like it worked02:21
rm_worki see one ubuntu dsvm success02:21
rm_workand two octavia dsvm02:22
rm_workwoooo02:22
rm_workok should i remove those debugs?02:22
rm_workpart of me just wants to +A this rofl02:22
*** yamamoto has joined #openstack-lbaas02:24
rm_workah damn there's one failure, wtf02:30
rm_workah, nova servers aren't building02:31
rm_workk02:31
rm_workwell then i'll submit the patch to remove those debugs02:31
rm_workright, the OVH problem is back02:32
rm_workfff02:32
rm_workwith the number of gates we have voting now that require VMs, this is kinda shitty, we have like a 50% or higher chance to get one voting failure, i think02:32
rm_workbased on rough head-maths02:32
rm_workoh right nm the neutron-lbaas ones aren't voting yet, yay me02:33
rm_workjohnsom: i'm assuming you're in and out02:33
rm_worki'll post this once the last test finishes there (I want it to actually record that run)02:34
rm_workand we can +2/+A :)02:34
*** amotoki has joined #openstack-lbaas02:39
*** amotoki has quit IRC02:43
johnsomYeah, on and off02:44
johnsomYeah, I'm not sure what is up with OVH, some pass, some don't02:44
*** JudeC has quit IRC02:44
rm_workah two fails02:46
rm_work>_>02:46
rm_work.... want this to finish so i can kick the next02:51
rm_workbut it's taking forever02:51
rm_workand i need to go to lunch now... so maybe I will just give up waiting even though it's close02:51
rm_worklog for a fail: http://logs.openstack.org/54/459654/12/check/gate-octavia-v1-dsvm-scenario-ubuntu-xenial/dff38c1/02:52
rm_worklog for a success: http://logs.openstack.org/54/459654/12/check/gate-octavia-v1-dsvm-py3x-scenario-ubuntu-xenial/302dbf8/02:52
rm_workhmm02:52
rm_workinterestingly it was both py2 that failed and both py3 that succeeded >_>02:52
rm_workprobably coincidence02:52
rm_work... and py3 on n-lbaas that are succeeded02:52
rm_workand py2 that are still going...02:52
rm_workumm02:52
rm_workmaybe not coincidence?02:53
rm_work>_>02:53
rm_workah ok nope, there we go02:53
rm_worksuccess on a py202:53
rm_workso it is just coincidence <_< weird02:54
rm_workwish this last one would finish02:54
rm_workk don't care02:55
openstackgerritAdam Harwell proposed openstack/octavia master: Change auth_strategy default to keystone  https://review.openstack.org/45965402:55
*** armax has joined #openstack-lbaas03:06
*** fnaval has quit IRC03:10
*** fnaval has joined #openstack-lbaas03:13
*** yamamoto has quit IRC03:27
*** aojea has joined #openstack-lbaas03:35
*** aojea has quit IRC03:40
*** armax has quit IRC03:41
*** aojea has joined #openstack-lbaas03:45
*** aojea has quit IRC03:46
*** aojea has joined #openstack-lbaas03:46
*** aojea has quit IRC03:51
*** armax has joined #openstack-lbaas04:00
*** armax has quit IRC04:09
*** yamamoto has joined #openstack-lbaas04:09
*** links has joined #openstack-lbaas04:12
*** armax has joined #openstack-lbaas04:14
*** gcheresh has joined #openstack-lbaas04:22
*** armax has quit IRC04:46
*** gcheresh has quit IRC04:46
*** armax has joined #openstack-lbaas05:00
rm_workjohnsom: feel free to +A :P05:01
*** armax has quit IRC05:04
*** gcheresh has joined #openstack-lbaas05:05
*** krypto has joined #openstack-lbaas05:15
openstackgerritAdam Harwell proposed openstack/octavia master: Fix a bug where ports may not be deleted  https://review.openstack.org/43132405:24
rm_workjohnsom: got bored waiting for you to fix this one:05:24
rm_workso I fixed it :P05:24
rm_workbrb05:24
*** JudeC has joined #openstack-lbaas05:43
*** aojea has joined #openstack-lbaas05:47
*** aojea has quit IRC05:52
*** aojea has joined #openstack-lbaas06:19
*** krypto has quit IRC06:22
rm_workAUGH I MISSED A PEP806:23
rm_workI am a shame to my family06:24
rm_workoh didn't mean to leave that in anyway06:26
openstackgerritAdam Harwell proposed openstack/octavia master: Fix a bug where ports may not be deleted  https://review.openstack.org/43132406:26
nmagnezirm_work, LOL06:37
rm_worko/06:37
nmagnezio/06:37
nmagnezirm_work, thanks for helping with the noauth/keystone patch. I planned to continue with it next week but i guess the sooner the better :)06:39
rm_workjohnsom helped as well :)06:40
rm_workyeah this particular one was annoying me06:40
rm_workand I had started on it before06:40
*** amotoki has joined #openstack-lbaas06:40
rm_workand was about to pick it back up when you submitted one06:40
nmagnezioh :D06:40
rm_workit would have been a lot easier if it'd merged BEFORE the wsgi-server patch06:40
* rm_work sighs06:40
rm_workmostly it was an issue with that thing's config06:41
nmagneziand yeah I saw Michael helped but he is prolly sleeping now so i wouldn't  ping him :D06:41
rm_worklol06:41
rm_workmy theory is, if people are available to be pinged, they'll be pinged, and if not, it's their own fault if they don't mute their phones <_<06:41
rm_workbut i leave the decision about availability in their hands06:41
rm_workno assumptions on my part :P06:41
*** amotoki has quit IRC06:45
openstackgerritAdam Harwell proposed openstack/octavia master: Vote: Should project_id remain in the response for all objects?  https://review.openstack.org/46134406:46
rm_worknmagnezi: ^^ what do you think06:46
rm_workJudeC: ^^06:46
* nmagnezi looks06:47
rm_workjust throwing it out there, easier to just do the code and put it up for review, than to try to explain it06:47
JudeClooking06:48
JudeCLGTM lol but thats not saying a whole lot :)06:50
rm_workit's kinda an opinion piece :P06:53
*** tesseract has joined #openstack-lbaas07:06
*** rcernin has joined #openstack-lbaas07:12
nmagnezirm_work, I think it's a good idea overall, but i'd like to check something in the v2 api ref07:13
nmagnezirm_work, but looks like https://developer.openstack.org/api-ref/load-balancing/v2/index.html is broken07:13
nmagnezirm_work, or did I get the wrong url?07:13
nmagnezirm_work, ah, it's https://developer.openstack.org/api-ref/load-balancer/v2/index.html07:13
*** rcernin has quit IRC07:32
*** JudeC has quit IRC07:48
*** JudeC has joined #openstack-lbaas08:17
*** aojea has quit IRC08:50
*** aojea has joined #openstack-lbaas08:50
*** voelzmo has joined #openstack-lbaas08:53
*** aojea has quit IRC08:54
*** JudeC has quit IRC09:11
*** voelzmo has quit IRC09:30
*** voelzmo has joined #openstack-lbaas09:31
*** aojea has joined #openstack-lbaas09:32
*** voelzmo has quit IRC09:45
*** chlong has joined #openstack-lbaas11:07
*** gcheresh has quit IRC11:36
*** gcheresh has joined #openstack-lbaas12:11
*** amotoki has joined #openstack-lbaas12:27
*** links has quit IRC13:29
*** cpuga has joined #openstack-lbaas13:39
*** cpuga_ has joined #openstack-lbaas13:41
rm_workxgerman: when you get on, this please: https://review.openstack.org/#/c/459654/ :)13:43
*** cpuga_ has quit IRC13:43
*** cpuga_ has joined #openstack-lbaas13:44
*** cpuga has quit IRC13:44
xgermank13:45
rm_workah you are already on :P13:46
nmagneziThree +2, must be my lucky day :D13:46
rm_workand I think https://review.openstack.org/#/c/431324/ is ready as well13:49
rm_workI added the test for johnsom13:49
rm_workxgerman: also I want to know what you think on a couple patches13:49
xgermansure13:49
rm_work1) cascade delete, implemented slightly differently: https://review.openstack.org/#/c/461182/13:49
*** gcheresh has quit IRC13:50
rm_work2) project_id on responses: https://review.openstack.org/#/c/461344/13:50
rm_work(I realized I THINK it is a little weird to remove it even from responses, though definitely we should not store it on every object)13:50
rm_workah, also single create: https://review.openstack.org/45903213:51
rm_workI will try to poke at status-tree today13:51
rm_workbut right now I need to sleep for a few hours >_>13:51
johnsomThanks Adam14:01
xgerman+114:03
xgermanjohnsom you have a decent single create set-up?14:04
xgermanor should I do some in-depth testing?14:04
johnsomThere is an example in the patch docs.  But I had problems with the patch last week.14:05
xgermanok, I guess then I need to stack it —14:07
*** gcheresh has joined #openstack-lbaas14:12
*** krypto has joined #openstack-lbaas14:25
*** krypto has joined #openstack-lbaas14:25
*** other_krypt0 has joined #openstack-lbaas14:28
*** krypto has quit IRC14:29
*** catintheroof has joined #openstack-lbaas14:30
*** krypto has joined #openstack-lbaas14:35
*** other_krypt0 has quit IRC14:35
*** catintheroof has quit IRC14:36
*** catintheroof has joined #openstack-lbaas14:40
*** other_Krypt0 has joined #openstack-lbaas14:40
*** krypto has quit IRC14:41
*** catintheroof has quit IRC14:42
*** other_Krypt0 is now known as krypto14:43
*** krypto is now known as Guest6075314:43
*** fnaval has quit IRC14:46
openstackgerritMerged openstack/octavia master: Change auth_strategy default to keystone  https://review.openstack.org/45965414:48
*** gcheresh has quit IRC14:57
*** fnaval has joined #openstack-lbaas15:00
xgermanmmh, our single create example is bogus15:04
openstackgerritGerman Eichberger proposed openstack/octavia master: Fixes small issues  https://review.openstack.org/46142515:08
*** aojea has quit IRC15:11
*** links has joined #openstack-lbaas15:13
*** links has quit IRC15:17
xgermanok, and when I send it against Adam’s patch it causes an error (might be me but we really should in that case review our validation logic)15:17
*** gcheresh has joined #openstack-lbaas15:18
*** gcheresh has quit IRC15:19
*** voelzmo has joined #openstack-lbaas15:33
*** amotoki has quit IRC15:55
*** aojea has joined #openstack-lbaas16:11
*** aojea has quit IRC16:16
*** tesseract has quit IRC16:29
*** openstack has joined #openstack-lbaas17:02
*** adams.freenode.net sets mode: +ns 17:02
*** openstack has joined #openstack-lbaas17:25
*** bzhao has joined #openstack-lbaas17:41
*** voelzmo has joined #openstack-lbaas18:00
*** voelzmo has quit IRC18:01
*** voelzmo has joined #openstack-lbaas18:02
*** Guest60753 has quit IRC18:08
xgermanafk for a bit18:10
*** voelzmo has quit IRC18:12
*** aojea has joined #openstack-lbaas18:13
*** openstackstatus has joined #openstack-lbaas18:14
*** ChanServ sets mode: +v openstackstatus18:14
*** aojea has quit IRC18:18
*** blogan_ has joined #openstack-lbaas18:37
*** blogan has quit IRC18:39
*** JudeC has joined #openstack-lbaas18:45
johnsomOh!  found my problem with devstack.  We introduced a bug recently with the new threading stuffs.18:54
johnsom./clean.sh isn't shutting down the octavia-worker18:55
johnsomhttps://www.irccloud.com/pastebin/GjKrW27s/18:55
johnsomSounds like an afternoon project18:55
johnsomI'm going to get lunch, back in a few18:56
*** voelzmo has joined #openstack-lbaas19:01
*** aojea has joined #openstack-lbaas19:04
*** dougwig has joined #openstack-lbaas19:15
xgermanoh, yeah, noticed that as well - thought it was my un-even devstack19:16
*** blogan_ is now known as blogan19:26
*** gcheresh has joined #openstack-lbaas19:41
*** KeithMnemonic has joined #openstack-lbaas19:41
johnsomNope, we broke something...19:54
*** JudeC has quit IRC20:00
*** KeithMnemonic has quit IRC20:09
*** kishore448 has joined #openstack-lbaas20:12
johnsomYeah, neat.  kill -15 has no effect....20:18
*** blogan has quit IRC20:22
xgermankill -9 worked for me20:24
kishore448hi all, need assistance in installing octavia on ubunut16.0420:25
johnsomxgerman Yes, but....20:25
johnsomkishore448 Hi, what issue are you seeing?20:26
*** voelzmo has quit IRC20:27
kishore448I installed octavia using "pip install octavia" .. all the octavia-*.py files in /usr/local/bin ..  i also performed all the pre-req steps like creating lb-mgmt-net, aphora image, nova flavor etc...20:27
johnsomOk, sounds good so far20:28
kishore448after creating octavia.conf file with appropriate parameters, I am not sure how to start the octavia services20:28
*** cpuga_ has quit IRC20:28
*** blogan has joined #openstack-lbaas20:28
johnsomkishore448 We do this in dev environments:20:30
johnsom /usr/local/bin/octavia-api  --config-file /etc/octavia/octavia.conf & echo $! >/opt/stack/status/stack/o-api.pid; fg20:30
johnsom /usr/local/bin/octavia-worker  --config-file /etc/octavia/octavia.conf & echo $! >/opt/stack/status/stack/o-cw.pid; fg20:30
johnsom...20:30
johnsomAll four processes work the same20:31
kishore448Are there any written start up scripts for these services or I need to do the same...BTW we are trying setup in production ( replace existing haproxy plugin and use octavia)20:33
johnsomAh, sorry no.  We haven't developed any startup scripts for the processes yet.  So much to do, not enough time in a day20:34
xgermanyou can always look at waht we did for openstack ansible20:36
kishore448Ah ok..I will try use your commands...Thanks a lot Johnsom ...  Thanks xgerman20:38
*** gcheresh has quit IRC20:54
*** cpuga has joined #openstack-lbaas20:57
*** JudeC has joined #openstack-lbaas20:58
*** JudeC has quit IRC21:00
*** sanfern has joined #openstack-lbaas21:16
*** aojea has quit IRC21:18
*** sanfern has quit IRC21:29
*** sanfern has joined #openstack-lbaas21:30
*** catintheroof has joined #openstack-lbaas21:50
*** KeithMnemonic has joined #openstack-lbaas21:52
*** JudeC has joined #openstack-lbaas21:59
rm_workxgerman: i fixed the api-ref *in* that patch :)22:02
rm_workhmmm interesting error22:03
xgermanyep, noticed22:04
JudeCjohnsom: I got the searching for subnet by name figured out, should I do the same for network?22:06
*** kishore448 has quit IRC22:09
*** KeithMnemonic has quit IRC22:10
rm_worktaking a look at your example now22:10
rm_worktrying to reproduce22:10
rm_workbut gotta restack i think :/22:10
openstackgerritAdam Harwell proposed openstack/octavia master: Add support for single-create for APIv2  https://review.openstack.org/45903222:11
rm_work^^rebaswe22:11
johnsomJudeC Yes22:12
JudeCfigured :)22:12
JudeCty22:12
johnsomAs for stacking, if you reclone nova dumped n-crt (nova-cert) so pull it from localrc or stack bombs....22:12
rm_workjohnsom: does harlowja's patch fix your thing? https://review.openstack.org/#/c/460823/22:13
johnsomI have been feeling the devstack love this afternoon...  Sigh22:13
rm_worki've never tried restacking22:13
rm_workso22:13
rm_workor rather22:13
rm_worki never use clean.sh22:13
rm_worki just blow the VM away every time22:13
johnsomDarn, forgot about that patch22:14
johnsomIt might22:14
johnsomJudeC the bugs should say "name or ID" on the params that need to be looked up22:27
JudeCkk22:35
*** catintheroof has quit IRC22:39
*** JudeC has quit IRC22:42
*** JudeC has joined #openstack-lbaas23:07
*** fnaval has quit IRC23:09
openstackgerritJude Cross proposed openstack/python-octaviaclient master: Add loadbalancer commands to client  https://review.openstack.org/45451623:25
rm_workaugh23:30
rm_worksomething else in the gate broke now23:30
* rm_work sighs23:30
rm_workhttp://logs.openstack.org/24/431324/4/check/gate-octavia-v1-dsvm-scenario-ubuntu-xenial/c365b5d/logs/devstacklog.txt.gz#_2017-05-01_22_48_05_59323:30
rm_workBarbican not starting23:30
johnsomHmmmm systemd....23:31
johnsomI bet they switched over to systemd default23:35
johnsomI'm not sure if those "newfangled" systemd logs are even captured anywhere....23:36
johnsomIt is this line: http://logs.openstack.org/24/431324/4/check/gate-octavia-v1-dsvm-scenario-ubuntu-xenial/c365b5d/logs/devstacklog.txt.gz#_2017-05-01_22_48_05_27123:37
openstackgerritJude Cross proposed openstack/python-octaviaclient master: Add loadbalancer commands to client  https://review.openstack.org/45451623:37
johnsomsystemd requires the full path, no idea why barbican is different than the others though23:37
openstackgerritJoshua Harlow proposed openstack/octavia master: Shutdown a endpoint workers executor on termination  https://review.openstack.org/46082323:39
openstackgerritJude Cross proposed openstack/python-octaviaclient master: Add loadbalancer commands to client  https://review.openstack.org/45451623:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!