| *** banix has quit IRC | 00:05 | |
| *** mark-casey has quit IRC | 00:09 | |
| *** mark-casey has joined #openstack-kolla | 00:10 | |
| *** mwheckmann has quit IRC | 00:14 | |
| *** ccesario_ has joined #openstack-kolla | 00:19 | |
| ccesario_ | night! | 00:19 |
|---|---|---|
| *** jtriley has joined #openstack-kolla | 00:26 | |
| *** jtriley has quit IRC | 00:40 | |
| *** daneyon_ has joined #openstack-kolla | 00:41 | |
| *** daneyon__ has joined #openstack-kolla | 00:42 | |
| *** rhallisey has quit IRC | 00:44 | |
| *** daneyon has quit IRC | 00:45 | |
| *** daneyon_ has quit IRC | 00:46 | |
| *** jtriley has joined #openstack-kolla | 00:47 | |
| *** mark-casey1 has joined #openstack-kolla | 00:48 | |
| *** mark-casey has quit IRC | 00:50 | |
| *** ccesario has left #openstack-kolla | 00:54 | |
| *** ccesario___ has quit IRC | 00:59 | |
| *** daneyon has joined #openstack-kolla | 01:00 | |
| *** sdake_ has joined #openstack-kolla | 01:02 | |
| *** sdake has quit IRC | 01:02 | |
| *** daneyon__ has quit IRC | 01:04 | |
| *** jtriley has quit IRC | 01:08 | |
| *** sdake_ is now known as sdake | 01:08 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the wrong package name in Ubuntu binary build https://review.openstack.org/303707 | 01:18 |
| *** mark-casey1 has quit IRC | 01:21 | |
| *** Jeffrey4l__ has joined #openstack-kolla | 01:24 | |
| *** jtriley has joined #openstack-kolla | 01:25 | |
| *** stvnoyes has joined #openstack-kolla | 01:26 | |
| *** ccesario_ has quit IRC | 01:31 | |
| *** ccesario___ has joined #openstack-kolla | 01:33 | |
| sdake | damn good thing we brought a towel | 01:35 |
| *** jtriley has quit IRC | 01:44 | |
| sdake | Jeffrey4l__ are you getting sleep dude ;) | 01:50 |
| Jeffrey4l__ | sdake, on a train. :D | 01:51 |
| Jeffrey4l__ | need to another place. | 01:51 |
| sdake | take care to get atleast 6 hours a night :) | 01:52 |
| sdake | Jeffrey4l__ openstack-infra said their default firwall rules would prevent leaking of multicat packets | 01:55 |
| *** ccesario___ has quit IRC | 01:55 | |
| Jeffrey4l__ | sdake, so bluebox do not stop this? and keepalived connect to each other | 01:56 |
| sdake | who knows such things | 01:57 |
| *** salv-orlando has joined #openstack-kolla | 01:58 | |
| Jeffrey4l__ | sdake, could we make sure with openstack-infra guys? may be they made some mistake in bluebox case. | 01:58 |
| sdake | feel free to ping them in openstack-infra | 01:59 |
| sdake | they know who yyou are ;) | 01:59 |
| sdake | they were pretty clear on it though | 01:59 |
| sdake | but i'd think we have to drop all our firewall rules in our vm to deploy properly | 01:59 |
| Jeffrey4l__ | ok , I will ping them. | 02:00 |
| sdake | note to all, I've reprioritized kolla 2.0.0 bugs | 02:01 |
| sdake | feel free to change as you see fit | 02:01 |
| sdake | https://launchpad.net/kolla/+milestone/2.0.0 | 02:01 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix the neutron reconfigure issue when enable linuxbridge-agent https://review.openstack.org/304320 | 02:01 |
| Jeffrey4l__ | no, drop all in our firewall is not good. when we have multi node test env in the gate, drop all will crash everything. So i do not recommend to do that. | 02:02 |
| Jeffrey4l__ | cool. i will check the bug list. | 02:02 |
| sdake | i left the implemented ones as is | 02:02 |
| sdake | since they are done | 02:02 |
| sdake | and hae no impact on the next week of dev | 02:02 |
| Jeffrey4l__ | https://review.openstack.org/303707 please check this . This block the ubuntu-binary in the mitaka branch. | 02:02 |
| sdake | sbezverk after those copy commnds you cn add a trailing backslash and not have to specify the name | 02:05 |
| sdake | Jeffrey4l__ acked | 02:05 |
| sdake | and rechecked | 02:06 |
| Jeffrey4l__ | sdake, wait. when you saying `drop all our firewall rule` you mean drop all the packets? because, in the current image, the iptable should be disabled. | 02:06 |
| Jeffrey4l__ | sdake, thanks. the centos-source failed due to the keystone issue. | 02:06 |
| sdake | openstack-infra is the right place to ask | 02:06 |
| sdake | which keystone issue | 02:06 |
| *** salv-orlando has quit IRC | 02:07 | |
| sdake | you mean this http://logs.openstack.org/07/303707/2/check/gate-kolla-dsvm-deploy-centos-source/c3a17e9/console.html#_2016-04-12_01_33_10_618 | 02:07 |
| sdake | hard to knwo if we are chasing gate bugs or kolla bugs | 02:08 |
| sdake | this bug always happened when rabbitmq crashed previously | 02:08 |
| sdake | vhosakot ^^ | 02:08 |
| Jeffrey4l__ | sdake, yea. this issue is wired too. But i do not think it is related to the rabbitmq issue. Will make some debug later. | 02:10 |
| *** unicell1 has quit IRC | 02:10 | |
| vhosakot | sdake: that too, I think is a gate issue.. like the keepalived VIP bug in gate.... I happened everytime before rabbitmq crashed... but, now, since rabbitmq crash is fixed, that keystone JSON error is still there.. I pritned the output in debug in gate, and the output will be emtpy and cannot be converted into JSSON and hecne fails | 02:11 |
| *** yuanying has quit IRC | 02:12 | |
| *** yuanying has joined #openstack-kolla | 02:12 | |
| sdake | why is the output empty | 02:12 |
| *** kbyrne has quit IRC | 02:13 | |
| vhosakot | sdake: 1 sec.. let me send the link to the logs | 02:13 |
| *** weiyu_ has joined #openstack-kolla | 02:14 | |
| Jeffrey4l__ | I think there is something wrong exist either in the `keystone-manage bootstrap` or in the our `keystone_bootstrap.sh` scripts. | 02:15 |
| vhosakot | Jeffrey4l__: sdake: here is my debug message that shows empty output in gate http://logs.openstack.org/86/303686/4/check/gate-kolla-dsvm-deploy-centos-source/17631e5/console.html#_2016-04-09_14_16_26_140 | 02:16 |
| sdake | vhosakot i get its empty | 02:16 |
| sdake | why is it empty is what I asked ;) | 02:16 |
| *** pbourke has quit IRC | 02:16 | |
| sdake | so Jeffrey4l__ is spot on | 02:17 |
| sdake | one of those two things is busted | 02:17 |
| *** pbourke has joined #openstack-kolla | 02:17 | |
| sdake | lets figure out which one | 02:17 |
| sdake | and fix it :) | 02:17 |
| vhosakot | sdake: yep, as Jeffrey4l__ said, keystone-manage is buggy in gate https://github.com/openstack/kolla/blob/master/docker/keystone/keystone_bootstrap.sh#L27 | 02:17 |
| sdake | orthe parsing script is buggy | 02:19 |
| sdake | which one ? | 02:19 |
| *** Jeffrey4l__ has quit IRC | 02:20 | |
| sdake | so here is what we know | 02:20 |
| sdake | keystone-manage output int othe parsing script returns nothing | 02:20 |
| sdake | next step is determining which of those is busted | 02:21 |
| sdake | looks more like a legitimate bug rather then a gate problem | 02:23 |
| sdake | but who knows for sure | 02:23 |
| sdake | we need debug output at both input and in the script | 02:23 |
| sdake | e.g. all variables passed to the bootstrap script | 02:23 |
| vhosakot | yep, since output is empty, the exit_json function is not hit.. so, somthing in these lines is breaking in gate https://github.com/openstack/kolla/blob/master/docker/keystone/keystone_bootstrap.sh#L27-L46 | 02:24 |
| *** kbyrne has joined #openstack-kolla | 02:24 | |
| sdake | logging the output of keystone_bootstrap duringa failureand during success would help | 02:26 |
| sdake | at line 31 | 02:26 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MASTER BRANCH https://review.openstack.org/294906 | 02:27 |
| sdake | this is also not executing https://github.com/openstack/kolla/blob/master/docker/keystone/keystone_bootstrap.sh#L29 | 02:27 |
| vhosakot | sdake: if we add an echo line in that script, will it be seen in gate logs as it will be run _inside_ the cotainers' shell | 02:27 |
| sdake | echo > /var/log/kolla/personal_debug | 02:28 |
| sdake | ad modify the copy logs script | 02:28 |
| vhosakot | sdake: like that... | 02:28 |
| vhosakot | sdake: ah, that should work, cool | 02:28 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MASTER BRANCH https://review.openstack.org/294906 | 02:28 |
| vhosakot | sdake: generally, will echo in a container's starup script be seen in "docker logs" of that cotnainer ? | 02:29 |
| sdake | good question | 02:29 |
| sdake | depends on if the heka volume is mounted or not | 02:29 |
| *** daneyon_ has joined #openstack-kolla | 02:29 | |
| vhosakot | sdake: ah ok | 02:29 |
| sdake | time to feed myself back in 30 mins or so | 02:30 |
| vhosakot | then, if "docker logs" can see it, we could just add "set -x" and "set -o" at the top of the script | 02:30 |
| sdake | i dont knwo if docker logs ar ecpied out or not | 02:31 |
| sdake | i need to eat, or i'd tell you :) | 02:31 |
| vhosakot | sdake: good to add "set -o pipefail" as well.... | 02:31 |
| vhosakot | sdake: cool, eat :) | 02:31 |
| *** Jeffrey4l__ has joined #openstack-kolla | 02:33 | |
| *** daneyon_ has quit IRC | 02:34 | |
| *** coolsvap has joined #openstack-kolla | 02:35 | |
| *** jtriley has joined #openstack-kolla | 02:37 | |
| *** weiyu_ has quit IRC | 02:41 | |
| *** weiyu_ has joined #openstack-kolla | 02:43 | |
| *** jtriley has quit IRC | 02:44 | |
| *** Jeffrey4l__ has quit IRC | 02:45 | |
| *** yuanying has quit IRC | 02:52 | |
| *** weiyu_ has quit IRC | 02:52 | |
| sdake | ok back | 02:56 |
| openstackgerrit | Merged openstack/kolla: Fix the neutron reconfigure issue when enable linuxbridge-agent https://review.openstack.org/304320 | 02:57 |
| *** weiyu_ has joined #openstack-kolla | 02:57 | |
| *** klint has joined #openstack-kolla | 03:00 | |
| *** banix has joined #openstack-kolla | 03:03 | |
| *** weiyu_ has quit IRC | 03:09 | |
| coolsvap | vhosakot, i think the ubuntu-binary is only for stable branch | 03:11 |
| vhosakot | coolsvap: ah ok, thanks for the info! | 03:12 |
| *** jtriley has joined #openstack-kolla | 03:12 | |
| vhosakot | coolsvap: so, just for mitaka ? | 03:12 |
| *** sdake_ has joined #openstack-kolla | 03:12 | |
| coolsvap | yes, due to the package | 03:12 |
| coolsvap | i dont recall the exact reason, need to search on mailing list i think | 03:13 |
| *** Marga_ has quit IRC | 03:14 | |
| vhosakot | coolsvap: ah ok... thanks for the info! | 03:14 |
| *** sdake has quit IRC | 03:15 | |
| coolsvap | vhosakot, https://openstack.nimeyo.com/69943/openstack-dev-kolla-adding-ubuntu-liberty-to-kolla-mitaka | 03:16 |
| *** sdake has joined #openstack-kolla | 03:18 | |
| *** jtriley has quit IRC | 03:18 | |
| vhosakot | coolsvap: thanks for the link! | 03:20 |
| *** sdake_ has quit IRC | 03:20 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: TESTING on Mitaka https://review.openstack.org/304373 | 03:23 |
| coolsvap | vhosakot, lets see the results on stable | 03:23 |
| sdake | dave-mccowan around? | 03:25 |
| kjejlly_ | Hello everyone, Does anyone know what's wrong with me? http://paste.openstack.org/show/493709/ | 03:28 |
| sdake | nothing is wrong with you | 03:28 |
| sdake | 127.0.1.1 is in your /etc/hosts file | 03:28 |
| sdake | with your hostname used in your deployment | 03:28 |
| vhosakot | coolsvap: cool | 03:29 |
| sdake | it hsould be the machines phsyical ip | 03:29 |
| dave-mccowan | sdake pong | 03:29 |
| sdake | (in th /etc/hosts file) in place of 127.0.1.1 | 03:29 |
| sdake | dave-mccowan kibana is supposed to have an haproxxy password | 03:29 |
| sdake | it does not | 03:29 |
| sdake | ring any bells? | 03:29 |
| dave-mccowan | i used a password to log into kibana a little while ago. | 03:30 |
| kjejlly_ | sdake: Thanks. It works ! | 03:31 |
| dave-mccowan | i remember talking about using basicauth, instead of requiring shield. | 03:31 |
| sdake | [sdake@minime-03 kolla]$ docker images | grep kibana | 03:31 |
| sdake | 192.168.1.103:4000/kollaglue/centos-source-kibana 2.0.0 7681b7187eae 2 hours ago 490.8 MB | 03:31 |
| sdake | i am not presented with a login prompt | 03:31 |
| sdake | we can't use shield | 03:31 |
| sdake | it has a 30 day expiry license | 03:31 |
| dave-mccowan | i know. | 03:32 |
| sdake | yes but you said you were presented with a login prompt | 03:32 |
| sdake | i was not | 03:32 |
| sdake | why? :) | 03:32 |
| dave-mccowan | i was presented with a login prompt. what port did you use in your url? | 03:32 |
| sdake | 5601 on the box itself | 03:32 |
| sdake | aoiding haproxy i guess | 03:32 |
| sdake | nm ;) | 03:32 |
| dave-mccowan | so there's a bug | 03:33 |
| dave-mccowan | want me to fix it? | 03:34 |
| sdake | no bug | 03:35 |
| sdake | what is the kibana username | 03:35 |
| vhosakot | sdake: https://github.com/openstack/kolla/blame/master/doc/quickstart.rst#L649-L667 | 03:36 |
| sdake | cool it works | 03:36 |
| sdake | dave-mccowan i was bypasing haproxy to directly acess my ndoes hence no auth | 03:36 |
| sdake | myw ife wants to make a dash board for kolla i guess ;) | 03:36 |
| *** banix has quit IRC | 03:36 | |
| dave-mccowan | you should not be allowed to bypass the auth | 03:37 |
| sdake | if you have access to the management network? | 03:37 |
| sdake | not sure how we could possibly stop that | 03:38 |
| *** salv-orlando has joined #openstack-kolla | 03:38 | |
| sdake | kibana runs on 192.168.1.103:5601 | 03:38 |
| sdake | if i connect directly to that, can bypass auth | 03:38 |
| sdake | if i use broked.selfip.net:5601 pressented with an auth screen | 03:39 |
| sdake | from 1990s | 03:39 |
| sdake | kibana 4 does not run in apache | 03:40 |
| dave-mccowan | right. ya. i was thinking kibana could listen on 127.0.0.1. but, if it's listening only on management interface, that's good enough. | 03:40 |
| sdake | it runs in nodejs or some other hpster thing | 03:40 |
| sdake | i dont know if it wild card binds | 03:41 |
| sdake | if i t does that is something we need to fix | 03:41 |
| sdake | [sdake@minime-03 kolla]$ netstat -nl | grep 5601 | 03:41 |
| sdake | tcp 0 0 192.168.1.103:5601 0.0.0.0:* LISTEN | 03:41 |
| sdake | tcp 0 0 192.168.1.149:5601 0.0.0.0:* LISTEN | 03:42 |
| sdake | tcp 0 0 192.168.1.148:5601 0.0.0.0:* LISTEN | 03:42 |
| sdake | not good | 03:42 |
| dave-mccowan | i think that's ok. hard to tell for sure with all three addrs on the same subnet. | 03:43 |
| sdake | 0.0.0.0 is bind to all intefaces | 03:44 |
| sdake | that is the default | 03:44 |
| dave-mccowan | that column is the remote addr | 03:44 |
| dave-mccowan | for local addr, it's only listening to host addr, internal vip, and external vip. | 03:44 |
| *** jtriley has joined #openstack-kolla | 03:44 | |
| sdake | oh you checked it out? | 03:45 |
| dave-mccowan | yep. same time as you did. | 03:45 |
| sdake | oh right | 03:46 |
| sdake | somehow i competely missed the host ip binds | 03:46 |
| * sdake running low on fuel | 03:46 | |
| dave-mccowan | looks like you're running with TLS. or just two vips? | 03:47 |
| sdake | moemnt | 03:47 |
| sdake | yes tls enabled | 03:47 |
| dave-mccowan | cool. glad it's getting extra testing... but looks like a kibana bug. that session is not over TLS. :-( | 03:49 |
| *** achanda has joined #openstack-kolla | 03:49 | |
| dave-mccowan | i thought I brought that up in a code review. need to add TLS for kibana. | 03:49 |
| sdake | there is a enable tls bug ffor kibana atm | 03:49 |
| *** yuanying has joined #openstack-kolla | 03:50 | |
| sdake | server.host is not being set | 03:50 |
| sdake | which is wrong | 03:50 |
| sdake | the default is 0.0.0.0 | 03:50 |
| sdake | so it is binding to all interfaces on my system | 03:50 |
| *** salv-orlando has quit IRC | 03:51 | |
| sdake | https://www.elastic.co/guide/en/kibana/current/kibana-server-properties.html | 03:51 |
| *** jtriley has quit IRC | 03:51 | |
| *** jasonsb has joined #openstack-kolla | 03:52 | |
| dave-mccowan | server.host is set on my system | 03:53 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1569148 | 03:54 |
| openstack | Launchpad bug 1569148 in kolla mitaka "kibana not setting server.host resulting in wildcard bind" [Critical,Confirmed] | 03:54 |
| sdake | its not in the .j2 file in the repo | 03:54 |
| dave-mccowan | the netstat you sent did not show binding to all interfaces. | 03:54 |
| sdake | master doesn't define server.host | 03:54 |
| sdake | the docs ssay that is a 0.0.0.0 bind | 03:54 |
| dave-mccowan | what do you have in /etc/kolla/kibana/kibana.yaml? | 03:55 |
| sdake | ya its set there | 03:56 |
| dave-mccowan | in the .j2 file, line 3, there is server.host | 03:56 |
| sdake | ok i guess im a dummy | 03:56 |
| sdake | paranoid of security problems ;) | 03:56 |
| dave-mccowan | missing TLS is a problem. should be a one line fix. | 03:56 |
| sdake | my wie is developing on my only box atm | 03:57 |
| dave-mccowan | bug is assigned to jeff. | 03:58 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1566117 | 03:59 |
| openstack | Launchpad bug 1566117 in kolla mitaka "Add TLS to the kibana service" [Critical,Confirmed] - Assigned to Jeffrey Zhang (jeffrey4l) | 03:59 |
| sdake | tbh i dont htink he would mind giving it up | 04:00 |
| sdake | he has like 15 bugs on his plat atm | 04:00 |
| sdake | he self assigned | 04:00 |
| sdake | i just brought yet another bug into 2.0.0 :( | 04:00 |
| dave-mccowan | i think i brought it up in a code review and he opened the bug instead of fixing it with original commit. | 04:00 |
| sdake | that is a commoon thing in this project ;) | 04:01 |
| sdake | dave-mccowan if you want to take it over wfm ;) | 04:02 |
| sdake | if not i'll fix when my wife gets done doing her thiing | 04:02 |
| coolsvap | excellent ubuntu binary build is green with https://review.openstack.org/#/c/303707 | 04:02 |
| coolsvap | no deploy is green sorry | 04:03 |
| *** Marga_ has joined #openstack-kolla | 04:04 | |
| sdake | that is ubuntu source | 04:05 |
| sdake | we dont have ubuntu binary in master | 04:05 |
| *** achanda has quit IRC | 04:05 | |
| sdake | i guess we should rethink that decision | 04:05 |
| sdake | rather in the master gates | 04:06 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Fix the wrong package name in Ubuntu binary build https://review.openstack.org/304373 | 04:07 |
| sdake | coolsvap you mean https://review.openstack.org/#/c/304373/ | 04:07 |
| patchbot | sdake: patch 304373 - kolla (stable/mitaka) - Fix the wrong package name in Ubuntu binary build | 04:07 |
| sdake | ubuntu binary gate failing for whatever reason | 04:08 |
| sdake | but deploy is green!! | 04:08 |
| coolsvap | yes | 04:09 |
| coolsvap | we were just trying out the master fix on the ubuntu-binary gates | 04:09 |
| openstackgerrit | Merged openstack/kolla: Fix the wrong package name in Ubuntu binary build https://review.openstack.org/303707 | 04:09 |
| sdake | yp apolgoies for the double cherrypick | 04:10 |
| sdake | didn't realize it had already been done | 04:10 |
| coolsvap | np | 04:11 |
| coolsvap | it was just a test | 04:11 |
| dave-mccowan | sdake testing fix now | 04:11 |
| sdake | 70 billion dollars of pure destruction | 04:11 |
| sdake | the md500 | 04:11 |
| *** Marga__ has joined #openstack-kolla | 04:12 | |
| *** banix has joined #openstack-kolla | 04:14 | |
| *** Marga_ has quit IRC | 04:15 | |
| sdake | boy i really like ender's game | 04:15 |
| sdake | i've watched it every day for a week | 04:15 |
| sdake | needs ack https://review.openstack.org/#/c/304373/2 | 04:19 |
| patchbot | sdake: patch 304373 - kolla (stable/mitaka) - Fix the wrong package name in Ubuntu binary build | 04:19 |
| sdake | this is an epic change: https://github.com/docker/docker/pull/20662 | 04:36 |
| sdake | runc | 04:36 |
| sdake | this allows containers to be run and docker to be restarted without disrupting the containers or stoppign them | 04:37 |
| sdake | in 1.11 | 04:37 |
| Marga__ | Hi | 04:38 |
| *** Marga__ is now known as fragatina | 04:39 | |
| fragatina | Hi guys/gals! | 04:39 |
| fragatina | I'm having some questions with installing kolla in ubuntu | 04:39 |
| sdake | sup fragatina | 04:40 |
| fragatina | since I haven't successfully installed this yet, this questions will be obvious to many | 04:40 |
| fragatina | or that I'm doing something wrong | 04:40 |
| sdake | if its not covered in the QSG its not obvious :) | 04:40 |
| fragatina | so, background: I have ubuntu 14.04 with 4.2.0-35-generic kernel | 04:41 |
| fragatina | and I'm running docker as: /usr/bin/docker daemon --insecure-registry 10.0.2.15:4000 --storage-driver=overlay | 04:42 |
| vhosakot | fragatina: yep, 4.2 kernel is good | 04:42 |
| fragatina | I've removed /var/lib/docker directory, so it doesn't exists | 04:43 |
| fragatina | and I run: docker run -d -p 4000:5000 --restart=always --name registry registry:2 | 04:43 |
| fragatina | (as I see it in the instructions) | 04:44 |
| sdake | docker will create /var/lib/docker | 04:44 |
| fragatina | also, I'm running everything as a root since I found that solves me some issues | 04:44 |
| openstackgerrit | Merged openstack/kolla: Fix the wrong package name in Ubuntu binary build https://review.openstack.org/304373 | 04:44 |
| sdake | but to do that it has to be started | 04:44 |
| sdake | yay 1 less bug to fix :) | 04:44 |
| fragatina | # ps -ef | grep docker | 04:45 |
| fragatina | root 950 1 0 21:28 ? 00:00:00 /usr/bin/docker daemon --insecure-registry 10.0.2.15:4000 --storage-driver=overlay | 04:45 |
| fragatina | so, docker is running but it didn't create the /var/lib/docker | 04:45 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Add quotes for RBMQ nodes definition https://review.openstack.org/303919 | 04:46 |
| fragatina | so when I try to do the registry it fails with: "docker: open /var/lib/docker/tmp/GetImageBlob590488645: no such file or directory. " | 04:46 |
| vhosakot | fragatina: you see that error when you run "docker run -d -p 4000:5000 --restart=always --name registry registry:2" ? | 04:46 |
| fragatina | yes | 04:46 |
| fragatina | completely output: | 04:47 |
| vhosakot | fragatina: can you resatrt the daemon and retry the docker run command | 04:47 |
| vhosakot | sudo service docker restart | 04:47 |
| fragatina | # docker run -d -p 4000:5000 --restart=always --name registry registry:2 | 04:47 |
| fragatina | Unable to find image 'registry:2' locally | 04:47 |
| fragatina | 2: Pulling from library/registry | 04:47 |
| fragatina | fdd5d7827f33: Downloading [==================================================>] 51.37 MB/51.37 MB | 04:47 |
| fragatina | a3ed95caeb02: Downloading [==================================================>] 32 B/32 B | 04:47 |
| fragatina | a79b4a92697e: Downloading [==================================================>] 7.572 MB/7.572 MB | 04:47 |
| fragatina | 6cbb75c7cc30: Waiting | 04:47 |
| fragatina | 4831699594bc: Downloading [==================================================>] 379 B/379 B | 04:47 |
| fragatina | docker: open /var/lib/docker/tmp/GetImageBlob590488645: no such file or directory. | 04:47 |
| fragatina | See 'docker run --help'. | 04:47 |
| vhosakot | sudo service docker restart | 04:47 |
| vhosakot | this should recreate /var/lib/docker | 04:48 |
| fragatina | weird, why it didn't the first time | 04:48 |
| fragatina | np, thx! | 04:48 |
| vhosakot | well, why did you delete /var/lib/docker ? :) | 04:49 |
| sdake | becaue she switched to overlayfs from aufs of course ;) | 04:50 |
| *** salv-orlando has joined #openstack-kolla | 04:51 | |
| openstackgerrit | Merged openstack/kolla: Add LVM2/iSCSI documentation https://review.openstack.org/300824 | 04:51 |
| *** rstarmer has joined #openstack-kolla | 04:52 | |
| fragatina | of course sdake! :-) | 04:52 |
| *** banix has quit IRC | 04:56 | |
| *** weiyu has joined #openstack-kolla | 04:57 | |
| fragatina | thx vhosakot, it seems at least it started building the images now | 05:02 |
| fragatina | thx! | 05:02 |
| vhosakot | fragatina: cool :) | 05:02 |
| *** salv-orlando has quit IRC | 05:03 | |
| gmmaha | vhosakot: is this patch just pending someone testing the linuxbridge on a deployment? https://review.openstack.org/#/c/302721/ | 05:03 |
| patchbot | gmmaha: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 05:03 |
| fragatina | @vhosakot, will let you know in a couple of hours how it went :-) | 05:04 |
| vhosakot | fragatina: cool | 05:04 |
| vhosakot | gmmaha: yeah, I was gonna test it sometime | 05:05 |
| gmmaha | vhosakot: aah ok.. i am building it on my setup | 05:05 |
| gmmaha | the latest master as of 10 mins ago with this patch | 05:05 |
| gmmaha | will update the patch if i get it working fine | 05:05 |
| vhosakot | gmmaha: cool, are you planning to test it ? | 05:05 |
| gmmaha | vhosakot: yeah | 05:06 |
| gmmaha | running it for some benchmark testing i am doing | 05:06 |
| vhosakot | sdake: can we have a mechanism to run linuxbridge on-demand in gate... ? | 05:06 |
| gmmaha | 1000 VMs | 05:06 |
| vhosakot | gmmaha: nice | 05:06 |
| sdake | vhosakot feel free to add linuxbridge gating :) | 05:07 |
| sdake | at ummit i am going to trian everyone how to add gates | 05:07 |
| sdake | and how our gatingworks | 05:07 |
| vhosakot | sdake: then, how we gate OVS ? I was thinking like an ondemand way to run linuxbrdige inn gate just to test linuxbrodge patches | 05:07 |
| sdake | dave-mccowan ping | 05:09 |
| sdake | dave-mccowan can you try this link http://bit.ly/1VkE8hD | 05:09 |
| sdake | except change my ip address with yours | 05:09 |
| sdake | linuxbridge should be tested all the time | 05:10 |
| sdake | in a sepraet set of gates | 05:10 |
| sdake | as well as tls | 05:10 |
| vhosakot | sdake: like centos-source-ovs and centos-source-linuxbridge, ...,...,,.. like that ? | 05:11 |
| sdake | yup | 05:11 |
| sdake | deploy-ovs | 05:11 |
| sdake | but yup | 05:11 |
| sdake | right now our deploy is deploy_ovs | 05:11 |
| vhosakot | sdake: it will double jenkins run time | 05:12 |
| sdake | or deploy-ovs | 05:12 |
| sdake | no jenkisn jobs run in prallel | 05:12 |
| *** daneyon_ has joined #openstack-kolla | 05:12 | |
| vhosakot | sdake: ah in parallel.. nice! | 05:12 |
| dave-mccowan | sdake i got could not locate dashboard | 05:12 |
| vhosakot | sdake: I will looking into adding linuxbridge gate then... many kolla users use it | 05:12 |
| *** rbergeron has joined #openstack-kolla | 05:15 | |
| sdake | dave-mccowan import this http://paste.fedoraproject.org/354486/4381091/ -> http://paste.fedoraproject.org/354486/04381091 | 05:15 |
| sdake | settings->objects->import | 05:15 |
| rbergeron | oh, you're already there. | 05:15 |
| rbergeron | i was going to say -- https://gist.github.com/robynbergeron/d1fd3634790464064d77f06508c62740 | 05:15 |
| rbergeron | (save it as a whateverfilename.json) | 05:16 |
| sdake | make sur eto make your index first of log-* | 05:16 |
| *** daneyon_ has quit IRC | 05:17 | |
| sdake | vhosakot typically yo uwill wnt for all distro variants | 05:17 |
| vhosakot | sdake: ok.. | 05:17 |
| sdake | the first step is to add a tox target | 05:17 |
| vhosakot | right | 05:17 |
| sdake | the second step is to modify deploy_aio so it takes an argument to do the job you want | 05:17 |
| sdake | i think it may already hae this | 05:18 |
| sdake | or setup_gate | 05:18 |
| sdake | not sure which | 05:18 |
| sdake | training - friday of summit ;) | 05:18 |
| dave-mccowan | i'm out of steam for tonight. i'm looking forward to trying it tomorrow. | 05:18 |
| *** dave-mccowan has quit IRC | 05:19 | |
| sdake | vhosakot do you have elasticsearch enabled | 05:19 |
| gmmaha | vhosakot: for that bug, i am not saying if the file /etc/kolla/passwords.yml does not get updated if a key is there and there are no passwords. | 05:20 |
| *** weiyu has quit IRC | 05:20 | |
| gmmaha | just saying that if the file is an old one the script does not add the new items into the file. | 05:20 |
| vhosakot | sdake: no... i have not enabled central_logging.. | 05:20 |
| sdake | gmmaha yes it is imperfect | 05:20 |
| gmmaha | let me push up a patch and maybe that can explain what i am tinking | 05:20 |
| sdake | vhosakot do so so you can check out the dashboard my wife made for us ;) | 05:20 |
| sdake | rbergeron can stuff link from one dashboard to another? | 05:21 |
| vhosakot | sdake: cool, so, should I enable central_logging and rebuilf ? | 05:21 |
| vhosakot | rebuild* | 05:21 |
| sdake | not rebuild | 05:21 |
| sdake | redeploy | 05:21 |
| sdake | connect to ip:5601 | 05:21 |
| sdake | index log-* | 05:21 |
| vhosakot | right right re-deploy | 05:21 |
| sdake | instead of logstash-* | 05:21 |
| vhosakot | gmmaha: what "new items" are you talkinga bout in passworks.yml ? | 05:22 |
| vhosakot | gmmaha: kolla-genpwd creates passwords to all fields that are emppty in passwords.yml | 05:22 |
| sdake | vhosakot he is talking about when config files change | 05:23 |
| sdake | and we hve no way to handle that upgrade atm | 05:23 |
| sdake | gmmaha in newton | 05:23 |
| sdake | we will deal with that | 05:23 |
| sdake | not now | 05:23 |
| gmmaha | vhosakot: correct.. but lets say today you add a new key neutron-XXX: in the source tree, if i run generate_passwords.py or genpwd.py tjat new keu wil not be adde to my /etc/kolla/passwords.yml | 05:23 |
| gmmaha | unless i add the neutron-XXX: to the file myself and then it will generate a password | 05:23 |
| gmmaha | sdake: for sure.. | 05:24 |
| gmmaha | no rush on it. a very very low priority bug for sure | 05:24 |
| vhosakot | gmmaha: as part of installation, we need to copy etc/kolla in repo to /etc/kolla | 05:24 |
| sdake | its a fetaure not a bug ;) | 05:24 |
| sdake | vhosakot he is takling about upgrading kolla from mitaka to newton | 05:24 |
| vhosakot | gmmaha: https://github.com/openstack/kolla/blame/master/doc/quickstart.rst#L169-L174 | 05:24 |
| sdake | or master to master | 05:24 |
| *** coolsvap has quit IRC | 05:25 | |
| vhosakot | ah upgrade | 05:25 |
| sdake | vhosakot focus - dashboards :) | 05:25 |
| vhosakot | yeah... doing it.. 1 sec | 05:25 |
| gmmaha | sdake: vhosakot: thanks and sorry for de-railing | 05:25 |
| sdake | rbergeron it would be cool i the daashboard clicked through | 05:25 |
| sdake | does kibana do that? | 05:26 |
| vhosakot | gmmaha: haha.. no need to for any sorry :) | 05:26 |
| sdake | like click on neutron an dsee a neutron specific dashboard | 05:26 |
| vhosakot | gmmaha: push the PS... i'll see in review | 05:26 |
| gmmaha | vhosakot: aye | 05:26 |
| sdake | rbergeron whats with that vertical thing on the dashboard | 05:28 |
| rbergeron | sdake: if you click on openstack.neutron in the total count by logger box | 05:29 |
| rbergeron | you'll notice that eerything gets filtered by that on the dashboard | 05:29 |
| rbergeron | and you'll see the little green box up towards the top right under the dashboard name | 05:29 |
| rbergeron | that says "logger openstack.neutron" | 05:29 |
| sdake | how do you see actual logs :) | 05:29 |
| rbergeron | and then you can click on that to remove the filter, save it, pin it, whatever | 05:30 |
| openstackgerrit | Merged openstack/kolla: Add quotes for RBMQ nodes definition https://review.openstack.org/303919 | 05:30 |
| rbergeron | sdake: looking but running out of steam :) | 05:33 |
| rbergeron | it's in the _source field but that isn't indexed but ... not sure if that can be fixed offhand | 05:33 |
| sdake | ya i've been out of steam for months | 05:33 |
| *** vincent_vdk has quit IRC | 05:33 | |
| sdake | we an reindex | 05:33 |
| sdake | rbergeron i'mg oing to spin up a vm | 05:35 |
| sdake | see what happens :) | 05:36 |
| rbergeron | uhh | 05:43 |
| rbergeron | okay | 05:43 |
| *** vincent_vdk has joined #openstack-kolla | 05:44 | |
| *** tfukushima has joined #openstack-kolla | 05:45 | |
| sdake | rbergeron i got you some more log to work with | 05:45 |
| sdake | i spun up 3 vms via heat | 05:45 |
| *** ydeshmukh has joined #openstack-kolla | 05:46 | |
| sdake | getting hang of kibana | 05:46 |
| sdake | you create filters in it | 05:46 |
| sdake | vhosakot you ready to roll? | 05:48 |
| vhosakot | de-reployed fine... openining ip:5601.. moment | 05:49 |
| elemoine_ | sdake looking at creating default Kibana dashboards? We may look into that pretty soon for our kolla central logging talk. Today maybe. | 05:49 |
| sdake | elemoine_ check out http://paste.fedoraproject.org/354486/4381091/ -> http://paste.fedoraproject.org/354486/04381091 | 05:50 |
| *** rstarmer has quit IRC | 05:50 | |
| sdake | vhosakot when it asks for an index type in log-* | 05:50 |
| sdake | elemoine_ i think my wife replicated the fuel dashboard in about 45 minutes | 05:51 |
| vhosakot | sdake: ah ok... let me type it | 05:51 |
| sdake | she rocks like that :) | 05:51 |
| elemoine_ | sdake I believe you! | 05:51 |
| elemoine_ | sdake more on that later... | 05:52 |
| sdake | more on which | 05:52 |
| rbergeron | i don't know about replicated | 05:53 |
| rbergeron | but poked certainly | 05:53 |
| sdake | made up her own | 05:53 |
| sdake | whatever the case wtb actual log messages ;) | 05:53 |
| * sdake excied :) | 05:53 | |
| rbergeron | sdake: patience | 05:53 |
| sdake | padowan | 05:54 |
| rbergeron | maybe tomorrow, though tomorrow is supposed to be my day for googlebigquery fun with githubarchive and metrics | 05:54 |
| sdake | do you hear nothing I say? :) | 05:54 |
| rbergeron | but | 05:54 |
| rbergeron | maybe | 05:54 |
| rbergeron | sdake: i think you should be able to curl that json into the kibana index in ES | 05:55 |
| sdake | yup | 05:55 |
| rbergeron | because that is apparently where the dashboards get saved | 05:55 |
| sdake | pretty sure i said that earlier | 05:55 |
| rbergeron | and then kibana is basically looking in that index when you want to open a saved one | 05:55 |
| rbergeron | sdake: it's not entirely supported yet is the thing | 05:56 |
| rbergeron | it used to be the norm | 05:56 |
| rbergeron | then went away | 05:56 |
| rbergeron | then they added "import / export" as a button | 05:56 |
| *** salv-orlando has joined #openstack-kolla | 05:56 | |
| rbergeron | this is all kibana 3 to kibana 4 stuff | 05:56 |
| sdake | how do i get a box that prints out jus the log field | 05:56 |
| vhosakot | sdake: after typing in log-* as index pattern, I don't see submit button and see "Unable to fetch mapping" faded out | 05:57 |
| vhosakot | sdake: http://picpaste.com/pics/kibana-zfl8CdoO.1460440537.png | 05:57 |
| sdake | now click dashboard | 05:57 |
| vhosakot | ok | 05:58 |
| sdake | you should see a horizontal graph | 05:58 |
| sdake | click settings->object->import | 05:58 |
| sdake | and import that json in the paste above | 05:59 |
| rbergeron | sdake: he has to be able to find the index | 05:59 |
| rbergeron | for that part to matter :) | 05:59 |
| vhosakot | sdake: should I copy paste the json into kibana ui ? | 05:59 |
| sdake | vhosakot no use import button dude ;) | 06:00 |
| sdake | settings->object->import button | 06:00 |
| rbergeron | vhosakot: were you able to find the index pattern? | 06:00 |
| vhosakot | sdake: should I save the contents in pastebin as a .json file on disk and then import it ? | 06:01 |
| sdake | right | 06:01 |
| sdake | hey elemoine_ why didn't you mention logstash-forwarder was deprecated upstream during heka dev | 06:02 |
| vhosakot | sdake: ok I see "Dashboards are Awesomesauce" | 06:03 |
| sdake | there ya go | 06:03 |
| vhosakot | sdake: in kibana ui.. what next ? | 06:03 |
| sdake | click it | 06:03 |
| openstackgerrit | Michal Rostecki proposed openstack/kolla: Create and chown /etc/heka directory https://review.openstack.org/302742 | 06:06 |
| sdake | do ou see a dashboard in black? | 06:09 |
| sdake | with what appears to be log information? | 06:09 |
| *** weiyu has joined #openstack-kolla | 06:10 | |
| *** ydeshmukh has quit IRC | 06:12 | |
| sdake | vhosakot did you get it rolling | 06:13 |
| *** unicell has joined #openstack-kolla | 06:20 | |
| *** achanda has joined #openstack-kolla | 06:23 | |
| *** vhosakot has quit IRC | 06:31 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: Add Heka support https://review.openstack.org/286301 | 06:34 |
| *** unicell has quit IRC | 06:40 | |
| *** unicell has joined #openstack-kolla | 06:40 | |
| *** coolsvap has joined #openstack-kolla | 06:44 | |
| openstackgerrit | Michal Rostecki proposed openstack/kolla: Create /var/cache/swift in source installation https://review.openstack.org/303849 | 06:46 |
| sdake | latest iteration of dashboard: http://paste.fedoraproject.org/354511/4437821/ -> http://paste.fedoraproject.org/354511/04437821 | 06:49 |
| sdake | elemoine_ you can rpobably start with this | 06:50 |
| *** salv-orlando has quit IRC | 06:54 | |
| *** weiyu has quit IRC | 06:57 | |
| *** weiyu has joined #openstack-kolla | 07:00 | |
| *** athomas has joined #openstack-kolla | 07:08 | |
| sdake | akwasnie you around | 07:12 |
| *** mikelk has joined #openstack-kolla | 07:13 | |
| *** salv-orlando has joined #openstack-kolla | 07:15 | |
| *** weiyu has quit IRC | 07:18 | |
| *** sdake_ has joined #openstack-kolla | 07:24 | |
| *** sdake has quit IRC | 07:26 | |
| *** sdake has joined #openstack-kolla | 07:27 | |
| *** sdake_ has quit IRC | 07:30 | |
| sdake | elemoine_ any problems with severity level 4 in the lua code your aware of relating to nova? | 07:31 |
| sdake | elemoine_ i've got sev level 4 errors but no payload | 07:32 |
| elemoine_ | sdake: no | 07:32 |
| elemoine_ | could you be more specific? | 07:32 |
| sdake | elasticsearch when filtering on payloa dlevel 4 doesnot show the payload | 07:33 |
| sdake | it has no results with level 4 but the graph i've produced does | 07:33 |
| sdake | in nova-api and nova-compute | 07:33 |
| *** salv-orl_ has joined #openstack-kolla | 07:34 | |
| elemoine_ | sdake: maybe create a bug report with screenshots demonstrating the problem | 07:35 |
| sdake | possible | 07:35 |
| sdake | i think that prototype my wife and i produced is a good place for you to start | 07:35 |
| sdake | it doesnt allow anding of priorities | 07:36 |
| sdake | which I'd like to see | 07:36 |
| sdake | not quite sure how to fix that | 07:36 |
| sdake | still jerking around with learning how kibana works | 07:36 |
| *** jmccarthy has joined #openstack-kolla | 07:37 | |
| *** salv-orlando has quit IRC | 07:37 | |
| *** nihilifer has quit IRC | 07:38 | |
| *** unicell has quit IRC | 07:39 | |
| *** unicell has joined #openstack-kolla | 07:41 | |
| *** nihilifer has joined #openstack-kolla | 07:42 | |
| *** tfukushima has quit IRC | 07:43 | |
| *** mbound has joined #openstack-kolla | 07:47 | |
| sdake | cool i just found filters | 07:47 |
| sdake | winning | 07:47 |
| *** shardy has joined #openstack-kolla | 07:48 | |
| *** tfukushima has joined #openstack-kolla | 07:50 | |
| *** ccesario___ has joined #openstack-kolla | 07:51 | |
| *** ydeshmukh has joined #openstack-kolla | 07:55 | |
| *** ccesario___ has quit IRC | 08:03 | |
| *** ccesario___ has joined #openstack-kolla | 08:03 | |
| ydeshmukh | hey sdake | 08:08 |
| ydeshmukh | I still stuck at 'Creating the Glance service and endpoint' when deploying Kolla on 3 Node setup (each control, compute and network). | 08:08 |
| ydeshmukh | Now I'm running with haproxy. I've a question here, which node hosts haproxy and keepalive containers? In my setup they are showing on network node. | 08:09 |
| sdake | that may be a bug | 08:09 |
| sdake | they should be hosted on the control nodes | 08:10 |
| *** fragatina has quit IRC | 08:11 | |
| ydeshmukh | sdake ok, I changed 'haproxy:children' network->control in inventory file. now haproxy and keepalive are running on control node. | 08:13 |
| sdake | did that fix deployment? | 08:14 |
| ydeshmukh | But still facing issue with 'Creating the Glance service and endpoint'. | 08:14 |
| ydeshmukh | I tried kolla-ansible prechecks earlier and that worked fine. | 08:15 |
| ydeshmukh | Could you guide me how to debug for 'Creating the Glance service and endpoint' issue? | 08:15 |
| *** weiyu has joined #openstack-kolla | 08:16 | |
| *** weiyu has quit IRC | 08:17 | |
| kjejlly_ | ydeshmukh: post the error message. Maybe other people will help you | 08:21 |
| kjejlly_ | post error message in here. http://paste.openstack.org/ | 08:22 |
| sdake | i'd recommend a reboot o all 3 nodes, keepalived still has vip running on wrong network host | 08:24 |
| sdake | your vip is an unused ip on your management network, correct? | 08:25 |
| ydeshmukh | kjejlly_ thanks, pasted on http://paste.openstack.org/show/493725/ | 08:25 |
| sdake | elemoine_ what is severity=5? | 08:26 |
| sdake | i think it houd be ok to run haproxy and keepalived on network nodes | 08:27 |
| elemoine_ | sdake: https://github.com/openstack/kolla/blob/master/docker/heka/plugins/modules/os_utils.lua#L28-L37 | 08:27 |
| sdake | but it seems odd | 08:27 |
| ydeshmukh | sdake ok, will reboot and check. yeah vip has unused ip and its pingable from all node. | 08:28 |
| *** Serlex has joined #openstack-kolla | 08:36 | |
| *** nihilifer has quit IRC | 08:44 | |
| sdake | something is generating alot of femergeny and critical errors | 08:46 |
| sdake | it hsouldn't respond to ping | 08:47 |
| sdake | if you reboot every ndoe | 08:47 |
| *** gfidente has joined #openstack-kolla | 08:48 | |
| *** gfidente has quit IRC | 08:48 | |
| *** gfidente has joined #openstack-kolla | 08:48 | |
| *** weiyu_ has joined #openstack-kolla | 08:51 | |
| *** weiyu_ has quit IRC | 08:52 | |
| *** weiyu has joined #openstack-kolla | 08:54 | |
| ydeshmukh | sdake, no luck still same issue. | 08:55 |
| sdake | after rebooting the vip responds to ping? | 08:55 |
| sdake | cleanup-containers | 08:56 |
| sdake | reboot all nodes | 08:56 |
| sdake | see if ip responds to ping | 08:56 |
| ydeshmukh | yeah | 08:56 |
| sdake | it should not | 08:56 |
| sdake | if it does, that means your vip is in use | 08:56 |
| ydeshmukh | do you mean after cleanup-containers? | 08:56 |
| *** nihilifer has joined #openstack-kolla | 08:57 | |
| *** nihilifer has quit IRC | 08:57 | |
| *** weiyu has quit IRC | 08:57 | |
| *** nihilifer has joined #openstack-kolla | 08:58 | |
| ydeshmukh | no vip is not pinable after cleanup-containers | 08:58 |
| *** nihilifer has joined #openstack-kolla | 08:58 | |
| *** nihilifer has joined #openstack-kolla | 08:59 | |
| ydeshmukh | if vip is not pinable or used somewhere else this ' Waiting for MariaDB service to be ready through VIP' step would have failed. | 09:00 |
| ydeshmukh | sdake, right? | 09:02 |
| sdake | ok | 09:03 |
| sdake | which distro re you using | 09:03 |
| sdake | centos binary centos source ubuntu binary ubuntu source? | 09:03 |
| *** dims has quit IRC | 09:04 | |
| *** weiyu_ has joined #openstack-kolla | 09:04 | |
| ydeshmukh | centos-binary | 09:04 |
| *** dims has joined #openstack-kolla | 09:05 | |
| sdake | ok centos binary is probably bust atm | 09:08 |
| sdake | i'd try centos source | 09:08 |
| sdake | we are waiting on rdo to release mitaka final | 09:08 |
| sdake | they are not done yet | 09:08 |
| sdake | most devs use centos binary or centos source | 09:09 |
| sdake | sorry cenots source or ubuntu source | 09:09 |
| sdake | thats not to say we dont want centos binry and ubuntu binary to work | 09:09 |
| sdake | they just may be less well tetted atm | 09:09 |
| sdake | elemoine_ is programname a multifield? | 09:09 |
| ydeshmukh | sdake, ok, can I give try with cenots source? | 09:11 |
| sdake | yes please do | 09:11 |
| ydeshmukh | ok thanks, will update you. | 09:13 |
| *** weiyu_ has quit IRC | 09:14 | |
| *** openstackgerrit has quit IRC | 09:17 | |
| *** openstackgerrit has joined #openstack-kolla | 09:17 | |
| *** coolsvap has quit IRC | 09:18 | |
| openstackgerrit | Merged openstack/kolla: Add prechecks to validate kolla_internal_vip_address and kolla_external_vip_address https://review.openstack.org/303938 | 09:18 |
| openstackgerrit | Merged openstack/kolla: Identify container through PS1 shell variable https://review.openstack.org/297984 | 09:19 |
| *** weiyu_ has joined #openstack-kolla | 09:23 | |
| *** coolsvap has joined #openstack-kolla | 09:24 | |
| openstackgerrit | Merged openstack/kolla: Remove /etc/kolla configuration folders https://review.openstack.org/303055 | 09:28 |
| *** jmccarthy has left #openstack-kolla | 09:29 | |
| *** andrewvwebber has joined #openstack-kolla | 09:30 | |
| openstackgerrit | Merged openstack/kolla: Add note about heat being busted for development https://review.openstack.org/290276 | 09:32 |
| *** weiyu_ has quit IRC | 09:35 | |
| *** weiyu_ has joined #openstack-kolla | 09:36 | |
| *** daneyon_ has joined #openstack-kolla | 09:43 | |
| *** daneyon_ has quit IRC | 09:48 | |
| elemoine_ | sdake: what makes you think it's a multifield? | 09:49 |
| sdake | nothign just would be handy if it were | 09:49 |
| *** shardy has quit IRC | 09:52 | |
| *** weiyu_ has quit IRC | 09:54 | |
| *** weiyu_ has joined #openstack-kolla | 09:55 | |
| *** openstackstatus has quit IRC | 09:57 | |
| *** openstack has joined #openstack-kolla | 09:58 | |
| *** openstackstatus has joined #openstack-kolla | 09:59 | |
| *** ChanServ sets mode: +v openstackstatus | 09:59 | |
| *** shardy has joined #openstack-kolla | 10:02 | |
| *** salv-orl_ has quit IRC | 10:06 | |
| sdake | elemoine_ akwasnie https://drive.google.com/open?id=0B8q6xDPETSkHc1NJV2VxOTZJU1E | 10:07 |
| *** achanda has quit IRC | 10:08 | |
| sdake | elemoine_ akwasnie time for bed, but heres a start at a dashboard http://paste.fedoraproject.org/354583/14604558/ -> http://paste.fedoraproject.org/354583/14604558 | 10:10 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add missing base os values to kolla-build.conf https://review.openstack.org/304499 | 10:11 |
| *** sdake has quit IRC | 10:11 | |
| *** weiyu_ has quit IRC | 10:12 | |
| *** salv-orlando has joined #openstack-kolla | 10:15 | |
| *** ccesario___ has quit IRC | 10:15 | |
| *** weiyu_ has joined #openstack-kolla | 10:22 | |
| *** ccesario___ has joined #openstack-kolla | 10:23 | |
| *** ccesario___ has quit IRC | 10:32 | |
| *** weiyu_ has quit IRC | 10:33 | |
| *** ccesario___ has joined #openstack-kolla | 10:34 | |
| *** achanda has joined #openstack-kolla | 10:38 | |
| *** ccesario___ has quit IRC | 10:39 | |
| *** ccesario___ has joined #openstack-kolla | 10:39 | |
| *** ccesario___ has quit IRC | 10:42 | |
| *** tfukushima has quit IRC | 10:43 | |
| *** achanda has quit IRC | 10:47 | |
| *** ccesario___ has joined #openstack-kolla | 10:47 | |
| *** salv-orlando has quit IRC | 10:48 | |
| *** salv-orlando has joined #openstack-kolla | 10:48 | |
| *** weiyu has joined #openstack-kolla | 10:52 | |
| *** salv-orlando has quit IRC | 10:53 | |
| coolsvap | new cleanup-host should not delete the kolla-build.conf :/ | 10:53 |
| *** ccesario has joined #openstack-kolla | 10:59 | |
| ccesario | morning :) | 10:59 |
| *** ccesario___ has quit IRC | 11:00 | |
| *** dave-mccowan has joined #openstack-kolla | 11:01 | |
| *** banix has joined #openstack-kolla | 11:02 | |
| *** weiyu has quit IRC | 11:03 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: cleanup-host should not remove kolla-build.conf https://review.openstack.org/304518 | 11:03 |
| *** weiyu has joined #openstack-kolla | 11:05 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add Mistral Binary Containers for CentOS https://review.openstack.org/304519 | 11:06 |
| *** achanda has joined #openstack-kolla | 11:08 | |
| *** banix has quit IRC | 11:10 | |
| *** achanda has quit IRC | 11:12 | |
| *** weiyu has quit IRC | 11:16 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add Mistral Binary Containers for CentOS https://review.openstack.org/304519 | 11:17 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add Mistral Binary Containers https://review.openstack.org/304519 | 11:21 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add missing base os values to kolla-build.conf https://review.openstack.org/304499 | 11:32 |
| mlima | coolsvap, sorry i don't use kolla-build.conf and therefore I did not notice the failure | 11:36 |
| *** banix has joined #openstack-kolla | 11:36 | |
| coolsvap | mlima: np | 11:37 |
| coolsvap | I found out and submitted for review | 11:38 |
| mlima | coolsvap, i did it https://review.openstack.org/303575 :) | 11:39 |
| *** rhallisey has joined #openstack-kolla | 11:39 | |
| coolsvap | mlima: i am not sure about keeping the scripts in etc/tools | 11:41 |
| coolsvap | but I will check, im setting up my dev environment again today, will review | 11:42 |
| mlima | coolsvap, ok :) | 11:42 |
| mlima | but this role is very usefull | 11:42 |
| mlima | ^^ | 11:42 |
| *** banix has quit IRC | 11:44 | |
| coolsvap | mlima: agree | 11:44 |
| *** openstack has quit IRC | 12:04 | |
| *** openstack has joined #openstack-kolla | 12:05 | |
| *** coolsvap is now known as coolsvap|away | 12:25 | |
| *** salv-orlando has joined #openstack-kolla | 12:39 | |
| *** jtriley has joined #openstack-kolla | 12:47 | |
| *** fragatina has joined #openstack-kolla | 12:51 | |
| *** stvnoyes has left #openstack-kolla | 12:52 | |
| *** mwheckmann has joined #openstack-kolla | 12:54 | |
| *** andrewvwebber has quit IRC | 12:55 | |
| *** klint has quit IRC | 13:01 | |
| *** ayoung has quit IRC | 13:12 | |
| nihilifer | can someone ack this commit? https://review.openstack.org/#/c/301636/ | 13:18 |
| patchbot | nihilifer: patch 301636 - kolla-mesos - RBMQ cluster cant use ip adrr as names, so we reve... | 13:18 |
| *** daneyon_ has joined #openstack-kolla | 13:19 | |
| *** daneyon_ has quit IRC | 13:24 | |
| openstackgerrit | Merged openstack/kolla-mesos: RBMQ cluster cant use ip adrr as names, so we reveting this back to hostnames https://review.openstack.org/301636 | 13:29 |
| *** mgoddard_ has joined #openstack-kolla | 13:45 | |
| *** tfukushima has joined #openstack-kolla | 13:46 | |
| *** mgoddard has quit IRC | 13:49 | |
| *** rstarmer has joined #openstack-kolla | 13:51 | |
| *** allen_gao has quit IRC | 13:52 | |
| *** banix has joined #openstack-kolla | 13:54 | |
| *** jhesketh has left #openstack-kolla | 13:55 | |
| *** andrewvwebber has joined #openstack-kolla | 13:56 | |
| pmisiak | hi | 14:01 |
| pmisiak | I have a problem with cinder-volume on centos:source | 14:01 |
| pmisiak | I can't create volume on a LVM | 14:01 |
| pmisiak | lvcreate is waiting on semaphore forever | 14:02 |
| pmisiak | anyone had the same issue here? | 14:03 |
| *** rstarmer has quit IRC | 14:03 | |
| pmisiak | hmm, anyway i see LVM backend will not work | 14:12 |
| pmisiak | because i dont see any mechanism for LVM volume attaching to VMs | 14:12 |
| pmisiak | there is no iscsi target configured or something similar | 14:13 |
| sbezverk | pmisiak the patch for lvm2 support and iscsi has not been yet merged | 14:13 |
| pmisiak | so i should use ceph? | 14:14 |
| sbezverk | if everything goes well, it should be merged today, I just need to convience core to ack last part of PS | 14:14 |
| sbezverk | pmisiak you can cherry pick my patch and try | 14:15 |
| sbezverk | pmisiak or wait for it hitting the master or of course use ceph ;-) | 14:15 |
| pmisiak | i think it wont resolve my issue because cinder-volume can not create LVM volume :/ | 14:22 |
| pmisiak | so I have no volumes to attach | 14:22 |
| pmisiak | this command hangs when run by cinder-volume inside container: /sbin/lvcreate -n cf76107a-183c-474e-aba0-a04d85ea7749 cinder-volumes -L 1g | 14:23 |
| pmisiak | when I run this commant directly on host volume is created | 14:24 |
| pmisiak | when I run this command directly on host, volume is created | 14:25 |
| pmisiak | maybe this is because I have centos containers on Ubuntu host? | 14:26 |
| pmisiak | some udev/kernel mismatches | 14:26 |
| sbezverk | pmisiak believe me I have it working lvm2 as a backend | 14:27 |
| sbezverk | for cinder and iscsi :-) | 14:28 |
| pmisiak | do you have the same OS on host and containers? | 14:28 |
| sbezverk | pmisiak your current issue is because by default container runs in a differnet ipc name space than your hos | 14:28 |
| sbezverk | so lvcreate cannot get ack from udev on creation of new device | 14:29 |
| *** inc0 has joined #openstack-kolla | 14:29 | |
| sbezverk | we explicitely tells containers which needs to create lvm to run in the same ipc name space oas the host | 14:29 |
| pmisiak | in yours patch ? | 14:30 |
| sbezverk | yes | 14:30 |
| pmisiak | ok, so I will try | 14:30 |
| sbezverk | it creats all infra you need for lvm2 to be backend of cinder | 14:30 |
| sbezverk | do not forget to check the doc which is already merged into master | 14:31 |
| sbezverk | it is funny doc is ready but patch not ;-) usually it is in reverse | 14:31 |
| *** sdake has joined #openstack-kolla | 14:32 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: DO NOT MERGE: TEST MASTER BRANCH https://review.openstack.org/294906 | 14:34 |
| inc0 | good morning | 14:35 |
| *** allen_gao has joined #openstack-kolla | 14:36 | |
| pmisiak | so I have to recreate all nova-compute and cinder-volume containers? | 14:38 |
| sdake | morning | 14:38 |
| sdake | inc0 i wrote a dashboard dude | 14:39 |
| sdake | did you see it | 14:39 |
| sdake | akscram | 14:39 |
| openstackgerrit | Merged openstack/kolla: Rabbitmq Upgrade restarts container only when image was changed. https://review.openstack.org/303858 | 14:39 |
| sdake | akwasnie ^^ | 14:39 |
| inc0 | sdake, so far I'm trying my kibana to work | 14:40 |
| inc0 | but I didn't have much time to debug what's wrong | 14:40 |
| sdake | did you see my screenshot? | 14:40 |
| inc0 | no sir I did not | 14:41 |
| openstackgerrit | Merged openstack/kolla: Create and chown /etc/heka directory https://review.openstack.org/302742 | 14:41 |
| sdake | https://drive.google.com/open?id=0B8q6xDPETSkHc1NJV2VxOTZJU1E | 14:41 |
| inc0 | this is cool dashboard indeed;) | 14:42 |
| sdake | my wife made the first one in 45 minutes | 14:42 |
| sdake | it took me about 4 hours to figure out how kibana worke well enough to do that | 14:42 |
| sdake | at hte bottom is a scroller of the filtered log mesages | 14:42 |
| *** haplo37 has joined #openstack-kolla | 14:43 | |
| sdake | we have some work to do with heka i think to make the filtered output look a little more usable in kibana | 14:43 |
| *** mgoddard_ has quit IRC | 14:43 | |
| inc0 | will you upstream it? | 14:44 |
| *** mgoddard has joined #openstack-kolla | 14:44 | |
| sdake | upstream is not a verb! | 14:44 |
| sdake | I was hoping someone would take it over | 14:44 |
| sdake | i have a super full plate atm | 14:44 |
| sdake | http://paste.fedoraproject.org/354700/72295146/ -> http://paste.fedoraproject.org/354700/72295146 | 14:45 |
| inc0 | did you become grammar nazi after ML thread or always was hidden one just waiting for an excuse? | 14:45 |
| *** cbits has joined #openstack-kolla | 14:45 | |
| sdake | sorry upstream and verb are a pet peeve of mine :) | 14:45 |
| sdake | anywy you can take that paste | 14:46 |
| sdake | and import it | 14:46 |
| sdake | if you want ot take over the work | 14:46 |
| ccesario | sdake, cool dashboard!! | 14:46 |
| sdake | if not I guess I will finish the job after release | 14:46 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the gate lying https://review.openstack.org/304205 | 14:46 |
| sdake | once imported it can be edited live | 14:47 |
| inc0 | after release is good, I still need to figure out how to backport | 14:47 |
| sdake | right | 14:48 |
| sdake | i will actually beocme free after release more so then i am now | 14:48 |
| sdake | but i had planned pto 20-23rd | 14:48 |
| sdake | dave-mccowan did you want to try that dashboard and see if an import works? :) | 14:48 |
| dave-mccowan | sdake sure. do i save the json on my laptop or on the controller node? | 14:49 |
| sdake | on laptop | 14:50 |
| sdake | click settings->objects->import | 14:50 |
| sdake | that shoul do the trick | 14:50 |
| sdake | i want to make sure it works for others before i blow away my containers | 14:50 |
| sdake | which i need to do to prep for this webinar i have tomorrow at 10am pt | 14:50 |
| *** sdake_ has joined #openstack-kolla | 14:51 | |
| dave-mccowan | it'll take a few minutes. redeploying now. | 14:53 |
| dave-mccowan | did you see the note i added to the kibana tls bug? | 14:54 |
| ccesario | sdake, talking about kibana, what to do when Unable create a index pattern ?! http://puu.sh/ofC8I/bdfaea1534.png like describe in kolla quickstart doc | 14:54 |
| *** sdake has quit IRC | 14:55 | |
| *** blahRus has joined #openstack-kolla | 14:58 | |
| ccesario | guys does kibana config should not point to IP address, since enabled http://paste.openstack.org/show/493819/ | 14:58 |
| ccesario | ? | 14:59 |
| ccesario | s/IP address/VIP address/ | 14:59 |
| *** inc0 has quit IRC | 15:01 | |
| *** dougs1 has joined #openstack-kolla | 15:02 | |
| *** coolsvap|away is now known as coolsvap | 15:05 | |
| *** dougs has quit IRC | 15:05 | |
| dave-mccowan | sdake_ i imported the dashboard | 15:07 |
| sdake_ | now click dashboard | 15:08 |
| sdake_ | it should jut work | 15:08 |
| sdake_ | or yu may have to load it first | 15:08 |
| sdake_ | i'd like to know which on a fresh install | 15:08 |
| coolsvap | mlima: can you tell me what you expect in the review? | 15:08 |
| coolsvap | https://review.openstack.org/#/c/304519 | 15:09 |
| sdake_ | coolsvap if your takingabut mistral | 15:10 |
| sdake_ | I just added a review to it | 15:10 |
| sdake_ | to clarify mlima's comments | 15:10 |
| sdake_ | feel free to read :) | 15:10 |
| coolsvap | sdake_: thanks | 15:10 |
| dave-mccowan | i had to setup a default index. i did that. then opened the dashboard. i got a lot of errors "could not locate that visualization" | 15:11 |
| sdake_ | dave-mccowan do you have things in your object list? | 15:11 |
| sdake_ | let me make sure I sent you the right file | 15:11 |
| dave-mccowan | settings->objects has one dashboard, zero searches, zero visualizations. | 15:12 |
| *** sdake has joined #openstack-kolla | 15:12 | |
| sdake | had to reconnect | 15:12 |
| sdake | coolsvap ncie work on mistral bp btw :) | 15:12 |
| sdake | dave-mccowan can you tell me what visualization it says is missing and verify your json file is 25k | 15:13 |
| sdake | http://paste.fedoraproject.org/354713/47405214/ -> http://paste.fedoraproject.org/354713/47405214 | 15:14 |
| dave-mccowan | my json file is tiny. 22 lines. | 15:14 |
| sdake | oh i guess i gave yu wrong one initially | 15:14 |
| sdake | my bad ;( | 15:14 |
| sdake | just import that one instead | 15:14 |
| sdake | it wasn't intentional I have a bunch of files son my hd | 15:14 |
| sdake | or rather ssd I guess :) | 15:14 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add Mistral Binary Containers https://review.openstack.org/304519 | 15:15 |
| *** sdake_ has quit IRC | 15:16 | |
| dave-mccowan | a little better. i have one widget and 3 errors now. missing Total-Logs-Count-for-Today, Log-Messages-Past-Hour, and Total-Count-by-Severity. | 15:16 |
| coolsvap | sdake: just found the packages on delorean so added the patchset | 15:16 |
| mlima | coolsvap, sorry i should have explained better :) | 15:17 |
| coolsvap | mlima: np | 15:17 |
| sdake | dave-mccowan i need to export the latest moment | 15:17 |
| coolsvap | actually i found it different from what you said so just wanted to confirm | 15:17 |
| sdake | coolsvap look at other containers | 15:17 |
| coolsvap | sdake: i did | 15:17 |
| dave-mccowan | that json has 6 dashboards. probably i have selected the wrong one. (and there are 5 that should be removed) | 15:18 |
| *** openstackgerrit has quit IRC | 15:18 | |
| sdake | the idea is to break after the -y \ | 15:18 |
| sdake | oh | 15:18 |
| sdake | right | 15:18 |
| sdake | kolla diagnotics | 15:18 |
| *** openstackgerrit has joined #openstack-kolla | 15:18 | |
| sdake | ya it was 3am when i finished | 15:18 |
| mlima | sdake, i think that add a CONTRIBUTING.rst can be interesting | 15:18 |
| sdake | or semi-finished | 15:18 |
| sdake | we have a contirbuting guide dont 'we? | 15:18 |
| mlima | no, we haven't | 15:19 |
| *** ayoung has joined #openstack-kolla | 15:20 | |
| dave-mccowan | sdake yep. looks nice. | 15:20 |
| *** vhosakot has joined #openstack-kolla | 15:20 | |
| sdake | mlima feel free to make one :) | 15:20 |
| mlima | ok | 15:20 |
| sdake | that will be some serious pain ;) | 15:20 |
| sdake | dave-mccowan thee box in the lower left is wrong | 15:21 |
| sdake | its supposed to wor kas a filter | 15:21 |
| sdake | so you can select two seveities at once | 15:21 |
| sdake | and ssee all the logs with those severities | 15:21 |
| sdake | i had it working at one point | 15:21 |
| sdake | and then changed something | 15:21 |
| sdake | and it stopped working | 15:21 |
| sdake | upper left box also not reporting in a style I like | 15:21 |
| openstackgerrit | Merged openstack/kolla: Create /var/cache/swift in source installation https://review.openstack.org/303849 | 15:22 |
| sdake | the real quetion is - is it usable | 15:22 |
| *** salv-orlando has quit IRC | 15:22 | |
| sdake | each of those things per box acts as a filter | 15:22 |
| sdake | for the log text | 15:23 |
| sdake | you cancreate warnings by going over quota | 15:23 |
| dave-mccowan | i clicked around. i like it. easy to say show me "warnings" + "neutron". | 15:23 |
| sdake | teh way I do that is create a flavor that requires 1000gb disk | 15:23 |
| sdake | i think we need changes in something | 15:23 |
| sdake | because the programname field isn't always the right data to use | 15:24 |
| sdake | it would be nice if the dashboard loading could be automated | 15:24 |
| sdake | my wife siad it can be curled into es | 15:24 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Aodh dockerfiles for formatting https://review.openstack.org/304699 | 15:24 |
| sdake | she wrote the first dashboard which took her 45 minutes | 15:24 |
| *** achanda has joined #openstack-kolla | 15:25 | |
| sdake | dave-mccowan i'd like to get to soething more like this | 15:29 |
| sdake | http://fuel-plugin-elasticsearch-kibana.readthedocs.org/en/stable/_images/kibana_logs_dash.png | 15:29 |
| vhosakot | ooh, cool UI | 15:30 |
| sdake | vhosakot did you see my go at it | 15:31 |
| sdake | https://drive.google.com/open?id=0B8q6xDPETSkHc1NJV2VxOTZJU1E | 15:31 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Ceilometer dockerfiles for formatting https://review.openstack.org/304704 | 15:32 |
| vhosakot | sdake: nice! | 15:32 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Aodh dockerfiles for formatting https://review.openstack.org/304699 | 15:34 |
| *** logan- has quit IRC | 15:34 | |
| *** achanda has quit IRC | 15:38 | |
| *** prithiv has joined #openstack-kolla | 15:39 | |
| ccesario | guys, could someone please, confirm if is it right!? http://paste.openstack.org/show/493819/ | 15:40 |
| *** logan- has joined #openstack-kolla | 15:40 | |
| *** sdake_ has joined #openstack-kolla | 15:41 | |
| *** sdake has quit IRC | 15:43 | |
| *** mikelk has quit IRC | 15:43 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update dockerfiles for formatting https://review.openstack.org/304716 | 15:46 |
| sdake_ | coolsvap can you do those one docker set at a time please | 15:46 |
| sdake_ | as in break apart that review | 15:46 |
| coolsvap | sdake_: its only one merged since gnocchi had only one file but i can split | 15:47 |
| *** salv-orlando has joined #openstack-kolla | 15:48 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Cinder dockerfiles for formatting https://review.openstack.org/304716 | 15:50 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Gnocchi dockerfiles for formatting https://review.openstack.org/304717 | 15:52 |
| *** tfukushima has quit IRC | 15:54 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Heat dockerfiles for formatting https://review.openstack.org/304720 | 15:55 |
| *** sdake_ has quit IRC | 15:55 | |
| *** athomas has quit IRC | 15:55 | |
| *** sdake has joined #openstack-kolla | 15:55 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Heka dockerfile for formatting https://review.openstack.org/304725 | 15:58 |
| *** prithiv has quit IRC | 16:01 | |
| *** daneyon_ has joined #openstack-kolla | 16:01 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Ironic dockerfile for formatting https://review.openstack.org/304727 | 16:01 |
| sdake | something isn't quite right with how heka reports programname | 16:03 |
| sdake | or how i'm using the fields in kibana one of the two ;) | 16:03 |
| gmmaha | vhosakot: i tried getting linuxbridge running withthe patch and the contaner stil kept crashing | 16:03 |
| gmmaha | :( | 16:03 |
| gmmaha | have a feeling my setup is busted.. | 16:03 |
| gmmaha | wont have time to debug that no unfortunately.. | 16:04 |
| gmmaha | sorry couldnt be of help | 16:04 |
| *** jmccarthy1 has joined #openstack-kolla | 16:04 | |
| *** jmccarthy1 has left #openstack-kolla | 16:04 | |
| *** prithiv has joined #openstack-kolla | 16:04 | |
| sdake | gmmaha its ok dude, thanks for all the help so far! | 16:05 |
| gmmaha | sdake: :) | 16:05 |
| sdake | gmmaha feelings good about kolla or badly? | 16:05 |
| gmmaha | ohh definitely good | 16:05 |
| sdake | I recognze this last minute bustage with linuxbridge is subptimal | 16:05 |
| gmmaha | and ntpttr will vouch for that :) | 16:05 |
| *** daneyon_ has quit IRC | 16:06 | |
| *** prithiv has quit IRC | 16:06 | |
| ntpttr | yes, certainly a better experience than OSA for us I'd say | 16:06 |
| gmmaha | sdake: i think the linuxbridge breakage is a small thing.. | 16:07 |
| sdake | well not if we ship our artifact as such :) | 16:07 |
| gmmaha | i am told that manjeet was able to get it working but didnt see huge improvements necessarily | 16:07 |
| gmmaha | sdake: :) | 16:07 |
| sdake | i'm glad it was able to be caught | 16:07 |
| vhosakot | gmmaha: linuxbridge container is not starting, or vm is not racing active state ? | 16:07 |
| vhosakot | reaching* | 16:07 |
| gmmaha | vhosakot: linuxbridge_agent container is constantly restarting | 16:07 |
| sdake | got any log of it? | 16:08 |
| sdake | docke logs linuxbridge_agent | 16:08 |
| vhosakot | gmmaha: are you testing this ? https://review.openstack.org/#/c/302721/ | 16:08 |
| patchbot | vhosakot: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 16:08 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Magnum dockerfiles for formatting https://review.openstack.org/304732 | 16:08 |
| gmmaha | vhosakot: yeah | 16:09 |
| gmmaha | thats the patch | 16:09 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Manila dockerfiles for formatting https://review.openstack.org/304736 | 16:10 |
| *** dwalsh has joined #openstack-kolla | 16:13 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Marathon dockerfile for formatting https://review.openstack.org/304741 | 16:13 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Manila dockerfile for formatting https://review.openstack.org/304736 | 16:14 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Mariadb dockerfile for formatting https://review.openstack.org/304743 | 16:15 |
| vhosakot | gmmaha: yeah,, docker logs will help | 16:16 |
| gmmaha | vhosakot: thre was nothing useful in them | 16:17 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Memcached dockerfile for formatting https://review.openstack.org/304744 | 16:17 |
| gmmaha | vhosakot: it said that its sarting the linuxbridge_agent with the conf file and ini file | 16:18 |
| gmmaha | and then crashed out | 16:18 |
| *** jasonsb has quit IRC | 16:18 | |
| ccesario | gmmaha, are you using multinode ? | 16:19 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Mesos dockerfiles for formatting https://review.openstack.org/304747 | 16:19 |
| gmmaha | ccesario: yes i am | 16:19 |
| sbezverk | gmmaha you do have a fix for missing ebtable binary right? | 16:19 |
| gmmaha | sbezverk: yes i pulled the patch | 16:19 |
| gmmaha | was tesing it out | 16:19 |
| ccesario | gmmaha, after apply this ps https://review.openstack.org/#/c/302721/ did you rebuild the images ? | 16:20 |
| patchbot | ccesario: patch 302721 - kolla - Fix launch vm failed in neutron-linuxbridge-agent env | 16:20 |
| gmmaha | i have a feeling it has to do with my env | 16:20 |
| gmmaha | ccesario: yes, removed all images, rebuilt them | 16:20 |
| ccesario | hmmm rigth.... only to check ... :) | 16:20 |
| gmmaha | vhosakot: maybe you can test it when uou get a chance and we will know if its the patch or my setup | 16:20 |
| ccesario | let me try deploy with this patch | 16:20 |
| gmmaha | ccesario: awesome.. thanks | 16:20 |
| ccesario | gmmaha, could you pls share your globals ? | 16:21 |
| vhosakot | ccesario: thanks! please cleanup first and then re-deploy | 16:21 |
| vhosakot | ccesario: also, remove OVS bridges in existing deployment | 16:21 |
| ccesario | vhosakot, yes... and rebuild the images :) | 16:21 |
| vhosakot | gmmaha: did you have ovs bridges from before ? | 16:21 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Mongodb dockerfile for formatting https://review.openstack.org/304748 | 16:22 |
| gmmaha | vhosakot: yeah | 16:22 |
| gmmaha | but i made sure none of those existed in the system, had the rm rebooted | 16:23 |
| ccesario | vhosakot, I dont 't have ovs brigdes :) | 16:23 |
| gmmaha | ccesario: aah changed it to get back to OVS and stuff | 16:23 |
| gmmaha | but yes i can share the globals | 16:23 |
| sbezverk | gmmaha sorry if it is duplicate request, could you share docker logs from linux bridge container | 16:25 |
| *** achanda has joined #openstack-kolla | 16:25 | |
| ccesario | nice :) | 16:25 |
| gmmaha | sbezverk: the docker logs didnt show anything | 16:25 |
| ccesario | I 'm rebuilding the images now | 16:25 |
| gmmaha | it jsut said its starting linuxbridge_agent with options and then the same logs over and over again | 16:25 |
| gmmaha | nothing to show errors | 16:25 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Neutron dockerfiles for formatting https://review.openstack.org/304755 | 16:25 |
| vhosakot | no interface except docker0 should be there.... al those ovs stale bridges, ovs-system, blah blah must be deldted | 16:27 |
| *** achanda has quit IRC | 16:30 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Nova dockerfiles for formatting https://review.openstack.org/304763 | 16:30 |
| openstackgerrit | Merged openstack/kolla: Put the variables in alphabetical order in all.yml https://review.openstack.org/303796 | 16:30 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update OpenVSwitch dockerfile for formatting https://review.openstack.org/304765 | 16:32 |
| pbourke | bug ids are still required for all changes hitting master atm right? | 16:33 |
| *** unicell has quit IRC | 16:34 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Swift dockerfiles for formatting https://review.openstack.org/304775 | 16:35 |
| *** ccesario___ has joined #openstack-kolla | 16:36 | |
| pbourke | coolsvap: killing me with the multiple patches :p | 16:36 |
| coolsvap | pbourke: sorry i am just putting for each service | 16:36 |
| coolsvap | almost done | 16:36 |
| vhosakot | coolsvap: you could have merged into a single PS | 16:38 |
| vhosakot | :) | 16:38 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Trove dockerfiles for formatting https://review.openstack.org/304777 | 16:38 |
| openstackgerrit | Merged openstack/kolla: Configure Neutron to use internalURL for contacting Nova https://review.openstack.org/302948 | 16:38 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Zaqar dockerfile for formatting https://review.openstack.org/304779 | 16:40 |
| andrewvwebber | ubuntu/source 'Failed sending gratuitous ARP to 10.0.0.1 on qr-eb5f9327-f4 in namespace qrouter-923e9b8c-6d8c-440c-bb2d-2da1c9168af2' - anything to be concerned about? | 16:41 |
| ccesario | pbourke, The bugs fixed on master will be backported/merged on mitaka branch? | 16:42 |
| *** mark-casey has joined #openstack-kolla | 16:44 | |
| *** ccesario___ has quit IRC | 16:44 | |
| vhosakot | andrewvwebber: wow, when do you see this ? when booting a VM ? | 16:44 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add iscsid/tgtd specific parameters to Kolla infrastructure https://review.openstack.org/300004 | 16:46 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Adds ansible code for iscsid and tgtd containers https://review.openstack.org/299995 | 16:46 |
| andrewvwebber | vhosakot: yes | 16:47 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update base dockerfile for formatting https://review.openstack.org/304783 | 16:48 |
| andrewvwebber | vhosakot: well technically when creating a router | 16:49 |
| vhosakot | andrewvwebber: is there an IP conflict somewhere ? | 16:50 |
| andrewvwebber | vhosakot: you mean on the external network? | 16:51 |
| vhosakot | andrewvwebber: yep, and also in the qrouter namespace | 16:52 |
| vhosakot | andrewvwebber: also, was mac address of any interface changed manuaaly ? | 16:53 |
| andrewvwebber | vhosakot: is is a fresh machine, wiped. i have not changed any mac address. i will check the external network | 16:54 |
| *** ydeshmukh has quit IRC | 16:54 | |
| vhosakot | andrewvwebber: i'd check neutron l3 agent log | 16:55 |
| vhosakot | sdake: should we link kolla design summit link here ? https://wiki.openstack.org/wiki/Design_Summit/Newton/Etherpads#Kolla | 16:56 |
| ccesario | gmmaha, globals .. pls | 16:58 |
| andrewvwebber | vhosakot: http://pastebin.com/raw/41kUBG8S | 16:58 |
| gmmaha | ccesario: aah apologies.. | 16:58 |
| *** salv-orlando has quit IRC | 16:58 | |
| mlima | vhosakot, can you remove -1? https://review.openstack.org/#/c/303575/ | 16:58 |
| patchbot | mlima: patch 303575 - kolla - Add a role to cleanup containers after deployment | 16:58 |
| gmmaha | gtting itnow | 16:58 |
| sdake | vhosakot when i have the sessions schedueld yes | 16:59 |
| vhosakot | sdake: cool | 16:59 |
| gmmaha | ccesario: http://paste.openstack.org/show/493838/ | 16:59 |
| vhosakot | mlima: did you test it without sudo ? | 17:00 |
| ccesario | gmmaha, | 17:00 |
| mlima | vhosakot, yes | 17:00 |
| ccesario | neutron_plugin_agent: "openvswitch" ? | 17:00 |
| gmmaha | ccesario: i had to switch it back | 17:00 |
| gmmaha | :) | 17:00 |
| vhosakot | mlima: cool! so, it works without sudo right ? | 17:00 |
| ccesario | ahhh ok | 17:00 |
| gmmaha | so replace that with linuxbridge | 17:00 |
| *** cbits has quit IRC | 17:00 | |
| gmmaha | i had to get the system up running asap.. tehre was soem testing that needed to happen | 17:00 |
| mlima | vhosakot, test you also to confirm. :P | 17:00 |
| *** ravig_ has joined #openstack-kolla | 17:00 | |
| vhosakot | mlima: yep! :) | 17:00 |
| ccesario | rigth ... give some time until my byuild finish... | 17:01 |
| vhosakot | mlima: did you test it in multinode ? | 17:02 |
| mlima | yes | 17:03 |
| vhosakot | mlima: cool, will the tools directory, validate-docker-execute.sh and all the cleanup scirips be copied to each node (control, network, compute and storage) ? | 17:04 |
| *** Serlex has quit IRC | 17:04 | |
| mlima | yes | 17:04 |
| mlima | just cleanup-containers and cleanup-host | 17:05 |
| vhosakot | right.. | 17:05 |
| *** rajathagasthya has joined #openstack-kolla | 17:10 | |
| *** mgoddard has quit IRC | 17:11 | |
| *** mgoddard has joined #openstack-kolla | 17:12 | |
| *** dougs has joined #openstack-kolla | 17:12 | |
| *** ravig has joined #openstack-kolla | 17:14 | |
| *** gfidente has quit IRC | 17:14 | |
| sdake | do we have some new bug with linuxbridge? | 17:16 |
| *** dougs1 has quit IRC | 17:16 | |
| sdake | ccesario anything in 2.0.0 that is fixed in master will also be fixed in stable/mitaka | 17:16 |
| sdake | pbourke yes, and nyting that will require a backport requires a bug id for management | 17:17 |
| andrewvwebber | vhosakot: systemd-udevd[12804]: Could not generate persistent MAC address for qg-954a3d7e-22: No such file or directory | 17:17 |
| sdake | pbourke i think we cn relax our rules around bug ids after the 15th (or whenver we tag 2.0.0) | 17:17 |
| sdake | but anything that needs a backport needs a bug id not ot get lost | 17:18 |
| sdake | andrewvwebber distro type and base? | 17:18 |
| andrewvwebber | ubuntu/source | 17:19 |
| sdake | 4.2 kernel? | 17:20 |
| *** unicell has joined #openstack-kolla | 17:20 | |
| sdake | andrewvwebber 14.04 or some othr version? | 17:20 |
| andrewvwebber | 4.5.0 | 17:20 |
| sdake | andrewvwebber off the top of my head my gues is the base distro doessn't match the continer image distro | 17:21 |
| *** ravig has quit IRC | 17:23 | |
| andrewvwebber | sdake: does this mean my container build server is too old? | 17:23 |
| sdake | which ubuntu distor are you running on your host | 17:23 |
| sdake | https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L62 | 17:24 |
| *** salv-orlando has joined #openstack-kolla | 17:24 | |
| andrewvwebber | im running coreos, i was successful last week however i decided to rebuild the containers and now external SNATing no longer works. SSH between instances works | 17:24 |
| sdake | you see there our ubuntu runtime is ubuntu:latest | 17:24 |
| sdake | nobody tests that case | 17:25 |
| sdake | your probably teh first :) | 17:25 |
| mlima | sdake, can you review this patchet https://review.openstack.org/#/c/303575/ ? | 17:25 |
| patchbot | mlima: patch 303575 - kolla - Add a role to cleanup containers after deployment | 17:25 |
| andrewvwebber | im happy to be your coreos tester | 17:25 |
| sdake | i'd try centos source in the short term | 17:25 |
| andrewvwebber | thanks | 17:26 |
| sdake | the problem is the systemd in the containers is incompatible with the systemd on the system | 17:26 |
| sdake | ccentos is more up to date wrt systemd then the ubuntu 14.04 image ;) | 17:26 |
| *** ravig_ has quit IRC | 17:27 | |
| andrewvwebber | i moved to ubuntu due to a bug that is probably now fixed :) | 17:27 |
| sdake | the userpsace and kernel need to match for containers to really wor properly | 17:27 |
| sdake | but coreos does not provide a userspace for contaner runtimes | 17:27 |
| sdake | we do some compelx things like communicating with the host systemd directly | 17:27 |
| sdake | that abi must match also | 17:27 |
| sdake | which is why we have multidistor support in the first place ;) | 17:28 |
| sdake | you were running centos source when it worked? | 17:28 |
| sdake | or ubuntu source? | 17:28 |
| andrewvwebber | ubuntu | 17:28 |
| sdake | we being libvirt | 17:28 |
| sdake | libvirtuses the dbus libraries to communicate with systemd i think ;) | 17:29 |
| sdake | they are supposed to be versioned | 17:29 |
| sdake | but ymmv | 17:29 |
| mark-casey | sdake: for my understanding is this a requirement that the machine that builds the images is similar to the one that runs them, or is this based on things that change based on the value of --base | 17:29 |
| sdake | mark-casey not builds but runs | 17:29 |
| sdake | the container image contains things like /usr/lib/include | 17:30 |
| sdake | and other libraries | 17:30 |
| andrewvwebber | so i have to be careful of coreos-updates... | 17:30 |
| sdake | all of which have abis of some type or another | 17:30 |
| sdake | if they are incompatible, kolla just wont work | 17:30 |
| *** cbits has joined #openstack-kolla | 17:30 | |
| andrewvwebber | understood | 17:30 |
| sdake | this is a problem docker likes to pretend doessn't exist | 17:30 |
| sdake | it even existss if you use all isolation features of kernel | 17:31 |
| andrewvwebber | this is probably a case for being able to build the images within a container...? | 17:31 |
| mark-casey | andrewvwebber: I have done that successfully | 17:31 |
| sdake | of kernel/of docker | 17:31 |
| mark-casey | well... Horizon is up. I haven't launched a VM on said images | 17:31 |
| andrewvwebber | mark-casey: legendary... | 17:31 |
| *** daneyon_ has joined #openstack-kolla | 17:33 | |
| andrewvwebber | sdake: do you now recommend centos/rdo(binary) again? | 17:33 |
| sdake | centos source | 17:33 |
| andrewvwebber | ok | 17:33 |
| sdake | rdo binary waiting on rdo package maintainers to finish their job | 17:33 |
| andrewvwebber | :) | 17:34 |
| *** achanda has joined #openstack-kolla | 17:34 | |
| mlima | sdake, mitaka milestone is wrong here? https://bugs.launchpad.net/kolla/mitaka/+bug/1563972 | 17:36 |
| openstack | Launchpad bug 1563972 in kolla mitaka "Manila keystone v3 issues for all clients" [Low,Fix committed] - Assigned to Mauricio Lima (mlima) (mauricio-lima) | 17:36 |
| *** andrewvwebber has quit IRC | 17:36 | |
| *** salv-orl_ has joined #openstack-kolla | 17:37 | |
| *** andrewvwebber has joined #openstack-kolla | 17:37 | |
| sdake | fixed | 17:38 |
| *** salv-orlando has quit IRC | 17:39 | |
| mlima | :) | 17:39 |
| mark-casey | andrewvwebber: I'm running all of my Kolla infra from Vagrant to facilitate tearing down and rebuilding PXE and docker registry. Should it be of use this is the box for image building/the deployer https://github.com/ropsoft/kolla_from_vagrant/blob/master/Vagrantfile#L107-L115 (which just builds https://github.com/ropsoft/kolla_from_vagrant/blob/master/deployer_dockerfile/Dockerfile) | 17:40 |
| *** sdake_ has joined #openstack-kolla | 17:40 | |
| *** ravig has joined #openstack-kolla | 17:40 | |
| mark-casey | MAAS's PXE would probably work ok for coreos if you weren't planning on using CoreOS's own PXE solution, which IIRC is still in beta | 17:40 |
| *** daneyon_ has quit IRC | 17:41 | |
| andrewvwebber | mark-casey: it sounds like the key factor is the vagrant VM being the same kernel version as the target hosts. so i guess i need a coreos vagrant host.. | 17:42 |
| *** ccesario___ has joined #openstack-kolla | 17:42 | |
| *** sdake has quit IRC | 17:43 | |
| andrewvwebber | mark-casey: i am planning on upgrading an existing coreos bare-metal cluster - with no down time :) | 17:43 |
| mark-casey | andrewvwebber: re: coreos vagrant. ah, yea that could be complex. To do the image build in a container I assume you'd have to tell vagrant to use coreos's box image as the VM to run docker inside | 17:45 |
| mark-casey | and awesome on the upgrade | 17:45 |
| *** shardy has quit IRC | 17:47 | |
| mark-casey | I'd love to see some sort of how-to! I'm headed that way myself but probably going with Ubuntu Wily for the time being | 17:47 |
| *** ccesario___ has quit IRC | 17:47 | |
| andrewvwebber | mark-casey: i cant physically live without systemd | 17:47 |
| *** ccesario___ has joined #openstack-kolla | 17:47 | |
| mark-casey | andrewvwebber: Wily has it. I have to run the CentOS steps for /run | 17:48 |
| mark-casey | but I still want to move later | 17:48 |
| andrewvwebber | mark-casey: minimal OSs like Atomic and CoreOS are fun and actually work. I would expect to see a Kolla/Atomic blog sometime soon | 17:50 |
| *** sdake has joined #openstack-kolla | 17:54 | |
| andrewvwebber | mark-casey: your Dockerfile is very useful. I will give it a try on CoreOS using a docker.sock volume. thanks | 17:54 |
| mark-casey | sure. the docker run command I used for the kolla-build is in readme | 17:55 |
| *** andrewvwebber has quit IRC | 17:57 | |
| *** sdake_ has quit IRC | 17:57 | |
| *** ccesario___ has quit IRC | 17:57 | |
| *** ccesario___ has joined #openstack-kolla | 17:57 | |
| *** banix has quit IRC | 17:58 | |
| *** daneyon_ has joined #openstack-kolla | 18:01 | |
| *** ccesario___ has quit IRC | 18:02 | |
| *** ravig_ has joined #openstack-kolla | 18:05 | |
| * coolsvap want to give try out the kibana dashboard | 18:09 | |
| coolsvap | but leaving now, first thing tomorrow morning | 18:10 |
| *** coolsvap has quit IRC | 18:10 | |
| *** achanda has quit IRC | 18:23 | |
| *** salv-orl_ has quit IRC | 18:26 | |
| *** salv-orlando has joined #openstack-kolla | 18:27 | |
| sbezverk | is it s known issue kolla_toolbox is failing to be built on the latest master? | 18:29 |
| ccesario | sbezverk, not for me | 18:30 |
| *** salv-orlando has quit IRC | 18:31 | |
| sbezverk | hm, I will try to rebuild, maybe it was intemittent glitch. | 18:32 |
| *** daneyon_ has quit IRC | 18:33 | |
| gmmaha | ccesario: sorry if you had alrady mentioned, were you able to get linuxbridge working on your setup | 18:36 |
| gmmaha | if you are still building things, then i will wait.. :) | 18:36 |
| *** daneyon_ has joined #openstack-kolla | 18:37 | |
| ccesario | gmmaha, finishing the deploy | 18:40 |
| ccesario | sorry for delay | 18:40 |
| gmmaha | ccesario: no worries.. | 18:40 |
| gmmaha | nothing of a rush.. just more curious | 18:40 |
| ccesario | PLAY RECAP ******************************************************************** | 18:43 |
| ccesario | 192.168.201.2 : ok=291 changed=61 unreachable=0 failed=0 | 18:43 |
| ccesario | gmmaha, any specific test ? the deploy it working and the process is up/// | 18:44 |
| ccesario | let me check endpoints | 18:44 |
| gmmaha | ccesario: is the linuxbridge_agent container running properly? | 18:45 |
| sdake | ok cats | 18:45 |
| sdake | mitaka is out | 18:45 |
| sdake | i am try8ing to get it to build | 18:45 |
| sdake | rather mitaka rdo | 18:45 |
| ccesario | gmmaha, yes ...is running | 18:46 |
| gmmaha | ccesario: on my tests the deploy was successful, but the linuxbridge_agent container was on a constant reboot pat | 18:46 |
| gmmaha | path* | 18:46 |
| gmmaha | ccesario: great.. thanks.. then i guess the patch is fine, but something funky with my setup | 18:46 |
| ccesario | gmmaha, http://paste.openstack.org/show/493862/ | 18:50 |
| gmmaha | ccesario: awesome.. atleast now we know neutron linuxbrdige is working with the patch | 18:51 |
| ccesario | e8958739e714 operator.local:4000/kolla/ubuntu-source-neutron-linuxbridge-agent:2.0.0 "kolla_start" 9 minutes ago Up 9 minutes neutron_linuxbridge_agent | 18:51 |
| gmmaha | nice | 18:51 |
| ccesario | gmmaha, any more spcific test ? | 18:51 |
| gmmaha | not sure why its on a constant reboot path on my setup.. but its a debug for another day | 18:51 |
| ccesario | because I'll destry my env to test other thing :P | 18:51 |
| gmmaha | ccesario: i dont have any.. unless vhosakot or sdake have any | 18:51 |
| ccesario | fell free to ask :) | 18:52 |
| gmmaha | ccesario: i couldnt get it to this far. :) | 18:52 |
| *** banix has joined #openstack-kolla | 18:54 | |
| *** daneyon_ has quit IRC | 18:59 | |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Add iscsid/tgtd specific parameters to Kolla infrastructure https://review.openstack.org/300004 | 19:00 |
| openstackgerrit | Serguei Bezverkhi proposed openstack/kolla: Adds ansible code for iscsid and tgtd containers https://review.openstack.org/299995 | 19:00 |
| *** daneyon_ has joined #openstack-kolla | 19:01 | |
| vhosakot | ccesario: excellent! thanks a lot for testing! are you able to boot a VM fine witt that linuxbridge path | 19:01 |
| *** rajathagasthya has quit IRC | 19:02 | |
| *** yuanying has quit IRC | 19:04 | |
| *** achanda has joined #openstack-kolla | 19:05 | |
| *** achanda has quit IRC | 19:05 | |
| ccesario | vhosakot, doing it now | 19:05 |
| vhosakot | ccesario: cool! | 19:05 |
| ccesario | give me a momment to complete it.... | 19:06 |
| ccesario | vhosakot, $ ip netns exec qrouter-010b0ce0-1dcd-46d6-9105-5907029da365 ifconfig | 19:16 |
| ccesario | Cannot open network namespace "qrouter-010b0ce0-1dcd-46d6-9105-5907029da365": Permission denied | 19:16 |
| ccesario | denied ? | 19:16 |
| *** inc0 has joined #openstack-kolla | 19:26 | |
| inc0 | hey I'm back | 19:27 |
| sbezverk | guys I consistently see kolla_toolbox is failing to built in centos/source | 19:28 |
| sbezverk | on the latest master | 19:28 |
| *** allen_gao has quit IRC | 19:29 | |
| sbezverk | http://paste.openstack.org/show/493868/ | 19:30 |
| *** ravig__ has joined #openstack-kolla | 19:30 | |
| *** ravig has quit IRC | 19:30 | |
| *** allen_gao has joined #openstack-kolla | 19:31 | |
| inc0 | sbezverk, merged your patches:) | 19:35 |
| inc0 | if we'll run into problems, we'll fix it as bugs | 19:35 |
| sbezverk | inc0 for sure, I am willing to gets assigned automatically for iscsi related bugs ;-) | 19:36 |
| mlima | inc0, can you review it https://review.openstack.org/#/c/303575/ ? | 19:36 |
| patchbot | mlima: patch 303575 - kolla - Add a role to cleanup containers after deployment | 19:36 |
| *** rajathagasthya has joined #openstack-kolla | 19:37 | |
| openstackgerrit | Merged openstack/kolla: Adds iscsid and tgtd docker containers https://review.openstack.org/291285 | 19:38 |
| inc0 | mlima, done | 19:44 |
| mlima | I disagree inc0 "/etc/ is not a place for code, it's place for configs. Also I'd rather see ansible do cleanup than copy shell and then run ansible. A bit more work but much more reliable, less error prone and error handling will be well..there | 19:45 |
| mlima | " | 19:45 |
| inc0 | ok, let's talk then | 19:47 |
| inc0 | what aren't you agreeing with specifically? | 19:47 |
| mlima | i left a comment there | 19:50 |
| mlima | :) | 19:50 |
| fragatina | hi! | 19:50 |
| fragatina | question about kolla-build | 19:50 |
| fragatina | I'm trying to deploy kolla and I get an error because the nova images where not build (I'm using ubuntu/source) | 19:51 |
| fragatina | I get the following error when I do: kolla-build --base ubuntu --type source nova | 19:52 |
| fragatina | in the following error: | 19:52 |
| fragatina | INFO:kolla.cmd.build:nova-base:Fetched 3930 kB in 5s (697 kB/s) | 19:52 |
| fragatina | INFO:kolla.cmd.build:nova-base:E: Failed to fetch http://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/libv/libvirt/libvirt0_1.3.1-1ubuntu8~cloud0_amd64.deb 404 Not Found [IP: 91.189.92.152 80] | 19:52 |
| fragatina | INFO:kolla.cmd.build:nova-base:E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? | 19:52 |
| fragatina | INFO:kolla.cmd.build:nova-base: | 19:52 |
| fragatina | INFO:kolla.cmd.build:nova-base:Removing intermediate container c166f0ea7519 | 19:53 |
| fragatina | I see that if I can succesfully get a package if I manually do a wget of that link but instead of ubuntu8 I use ubuntu9 | 19:54 |
| fragatina | but of course, I cannot change what nova-base pulls down, or I can? | 19:54 |
| mark-casey | mlima inc0 would script module help? http://docs.ansible.com/ansible/script_module.html | 19:54 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: add curlrc file to base image https://review.openstack.org/301911 | 19:55 |
| *** salv-orlando has joined #openstack-kolla | 19:55 | |
| fragatina | any suggestion of how to proceed with this? I don't have libvirt running on this host | 19:55 |
| ccesario | $ ip netns exec qrouter-010b0ce0-1dcd-46d6-9105-5907029da365 ifconfig is there any permission needed ? | 19:55 |
| ccesario | Cannot open network namespace "qrouter-010b0ce0-1dcd-46d6-9105-5907029da365": Permission denied | 19:55 |
| mark-casey | ansible -i /usr/local/share/kolla/ansible/inventory/multinode -m script -a '/kolla/tools/cleanup-containers' all | 19:55 |
| inc0 | fragatina, is your resolv.conf doing well? | 19:55 |
| *** vhosakot has quit IRC | 19:55 | |
| inc0 | I mean it looks like dns issue | 19:55 |
| inc0 | what you can do is | 19:55 |
| inc0 | docker run -it ubuntu bash | 19:55 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add a role to cleanup containers after deployment https://review.openstack.org/303575 | 19:56 |
| inc0 | and check resolv.conf and such | 19:56 |
| *** mlima has quit IRC | 19:57 | |
| inc0 | mark-casey, good idea! | 19:57 |
| inc0 | I'd rather have ansible doing logic but for now it's ok | 19:57 |
| mark-casey | cool :) | 19:58 |
| fragatina | thx @inc0, let me see | 19:59 |
| *** salv-orlando has quit IRC | 19:59 | |
| sbezverk | inc0 could you check why two other parts of PS are not merging? especially this one: https://review.openstack.org/#/c/300004/20 | 19:59 |
| patchbot | sbezverk: patch 300004 - kolla - Add iscsid/tgtd specific parameters to Kolla infr... | 19:59 |
| sbezverk | it is all green :-) | 20:00 |
| inc0 | sbezverk, https://review.openstack.org/#/c/299995/11 this one has only my +2;) | 20:01 |
| patchbot | inc0: patch 299995 - kolla - Adds ansible code for iscsid and tgtd containers | 20:01 |
| inc0 | and another is dependant on this one | 20:01 |
| *** mark-casey has left #openstack-kolla | 20:02 | |
| sbezverk | inc0 got it.. thanks for clarification.. | 20:02 |
| *** vhosakot has joined #openstack-kolla | 20:03 | |
| *** pabelanger has left #openstack-kolla | 20:04 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Use debian repos for debian base docker image https://review.openstack.org/288936 | 20:04 |
| *** cbits has quit IRC | 20:06 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make container dind build with debian https://review.openstack.org/297434 | 20:20 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make heka container build with debian https://review.openstack.org/297737 | 20:23 |
| *** banix has quit IRC | 20:24 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make kibana container build with debian https://review.openstack.org/297206 | 20:24 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make marathon container build with debian https://review.openstack.org/297378 | 20:25 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make mesos-slave container build with debian https://review.openstack.org/297385 | 20:27 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make mesosphere-base build with debian https://review.openstack.org/297358 | 20:28 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make nova-libvirt container build with debian https://review.openstack.org/297169 | 20:29 |
| ccesario | gmmaha, vhosakot http://puu.sh/ofWXJ/342743d2e3.png http://puu.sh/ofX0W/650d5b3699.png | 20:30 |
| openstackgerrit | Dave McCowan proposed openstack/kolla: Remove Disable HAProxy option from Quick Start Guide https://review.openstack.org/304848 | 20:32 |
| *** jtriley has quit IRC | 20:32 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: make rabbitmq container build with debian https://review.openstack.org/297728 | 20:34 |
| fragatina | inc0, it doesn't seem a dns issue since I manually pick up the package in: http://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/libv/libvirt/libvirt0_1.3.1-1ubuntu8~cloud0_amd64.deb | 20:34 |
| fragatina | but I cannot get the package in: http://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/libv/libvirt/libvirt0_1.3.1-1ubuntu8~cloud0_amd64.deb | 20:35 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make swift-base container build with debian https://review.openstack.org/297121 | 20:35 |
| fragatina | sorry, typo, the one that I can do succesfully is this one: http://ubuntu-cloud.archive.canonical.com/ubuntu/pool/main/libv/libvirt/libvirt0_1.3.1-1ubuntu9~cloud0_amd64.deb | 20:35 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make container kolla-toolbox build with debian https://review.openstack.org/297104 | 20:36 |
| fragatina | so, kolla-build for nova tries to get libvirt*ubuntu8*.deb package which is not there in ubuntu-cloud but the file name libvirt*ubuntu9*.deb is there in ubuntu-cloud | 20:36 |
| *** daneyon_ has quit IRC | 20:36 | |
| fragatina | so, is there a way to make kolla pick up the right package from ubuntu cloud? | 20:37 |
| sbezverk | Anybody on centos, please ping back, need sanity check.. | 20:41 |
| *** salv-orlando has joined #openstack-kolla | 20:41 | |
| gmmaha | ccesario: awesome.. *thumbs_up* | 20:42 |
| ccesario | gmmaha, ;) | 20:46 |
| *** ccesario___ has joined #openstack-kolla | 20:50 | |
| *** inc0 has quit IRC | 20:51 | |
| *** inc0 has joined #openstack-kolla | 20:51 | |
| *** salv-orlando has quit IRC | 20:51 | |
| *** salv-orlando has joined #openstack-kolla | 20:52 | |
| *** vhosakot has quit IRC | 21:00 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make container dind unpin old docker relase https://review.openstack.org/297434 | 21:01 |
| *** sdake has quit IRC | 21:05 | |
| *** daneyon_ has joined #openstack-kolla | 21:06 | |
| *** sdake has joined #openstack-kolla | 21:08 | |
| *** ccesario___ has quit IRC | 21:09 | |
| *** jtriley has joined #openstack-kolla | 21:10 | |
| sdake | http://paste.fedoraproject.org/354817/14604955/ -> http://paste.fedoraproject.org/354817/14604955 | 21:13 |
| sdake | have a look at that fpaste devs... | 21:13 |
| sdake | this is how much docker rocks | 21:13 |
| *** ccesario___ has joined #openstack-kolla | 21:18 | |
| sbezverk | sdake I am experiencing consisten failure on building kolla_toolbox, the issue is module pytz, the only solution I found to get around it is to install it before it is needed. | 21:18 |
| *** mark-casey has joined #openstack-kolla | 21:19 | |
| sdake | using overlay graph driver? | 21:19 |
| sbezverk | sdake I saw some notes about removing it and there is rpm -e for this module, notes mentioned your name, do you recall why you wanted to remove pytz? | 21:19 |
| sdake | yes becaues it doesn't work with source builds | 21:20 |
| sdake | we have a mix of source and dbinary in the system | 21:20 |
| sdake | which is wrong | 21:20 |
| sdake | for python modules | 21:20 |
| sdake | this was introduced by the mesos work | 21:20 |
| sbezverk | sdake I added it as pip install pytz | 21:20 |
| sbezverk | to Docker's file for kolla_toolbox | 21:21 |
| sdake | ya the above only happens with overlayfs | 21:22 |
| sdake | the defult graph driver seems to work | 21:22 |
| sbezverk | sdake should I file a bug? | 21:24 |
| sdake | aobut which | 21:25 |
| sdake | which graph driver are you using | 21:25 |
| *** ccesario___ has quit IRC | 21:25 | |
| *** ccesario___ has joined #openstack-kolla | 21:26 | |
| *** ccesario___ has quit IRC | 21:29 | |
| sdake | sbezverk ^^^ | 21:31 |
| *** ccesario___ has joined #openstack-kolla | 21:31 | |
| sbezverk | sdake I am sorry, not sure where graph driver is coming from | 21:32 |
| sdake | --storage-driver= what? | 21:32 |
| sbezverk | sdake I am building kolla_toolbox image using build.py, I relly doubt --storage-driver has anything to do here. | 21:34 |
| *** ccesario___ has quit IRC | 21:34 | |
| sdake | the above paste doesn't happen with default storage driver | 21:34 |
| *** inc0 has quit IRC | 21:34 | |
| sdake | so yes storage driver has someting to do with it | 21:35 |
| *** ccesario___ has joined #openstack-kolla | 21:35 | |
| sbezverk | sdake when you clone master, go into tools and start build.py top build images, where is the storage driver is defined?? Unless it is defined somewhere by default, so I am not touching any default values.. | 21:36 |
| sdake | ps -ef | grep docker on your system pleae | 21:36 |
| sbezverk | sdake http://paste.openstack.org/show/493877/ | 21:37 |
| sdake | you are not using the overlayfs graph driver as I was | 21:38 |
| sdake | how that rpm -e behaves is diferent between he two | 21:38 |
| sbezverk | bb in 30 min | 21:39 |
| *** daneyon_ has quit IRC | 21:39 | |
| *** ccesario__ has joined #openstack-kolla | 21:41 | |
| *** ccesario___ has quit IRC | 21:42 | |
| *** haplo37 has quit IRC | 21:42 | |
| *** ccesario___ has joined #openstack-kolla | 21:44 | |
| *** ayoung has quit IRC | 21:46 | |
| sdake | nice rdo mitaka works | 21:48 |
| *** ccesario__ has quit IRC | 21:48 | |
| *** ccesario___ has quit IRC | 21:53 | |
| *** sdake_ has joined #openstack-kolla | 21:53 | |
| *** fragatina has quit IRC | 21:54 | |
| *** sdake has quit IRC | 21:54 | |
| *** banix has joined #openstack-kolla | 21:57 | |
| *** salv-orlando has quit IRC | 22:00 | |
| *** salv-orlando has joined #openstack-kolla | 22:01 | |
| openstackgerrit | Steven Dake proposed openstack/kolla: Port stable/mitaka to use RDO for binary https://review.openstack.org/304884 | 22:03 |
| *** daneyon_ has joined #openstack-kolla | 22:04 | |
| *** sdake has joined #openstack-kolla | 22:04 | |
| sbezverk | sdake what is more strange is that yesterday the same operation was working just fine, my building server has not changed, kolla_toolbox's files has not been changed | 22:05 |
| *** sdake_ has quit IRC | 22:06 | |
| sbezverk | the only difference is the latest master | 22:06 |
| sdake | and all the bits that make u pteh containers | 22:06 |
| sdake | including the base image and rpms ;) | 22:06 |
| sdake | lots of stuff changes | 22:06 |
| sbezverk | sdake I was trying to find somebody with centos/source to test master but could not, would you be able to test it later today? | 22:13 |
| sdake | sbezverk i amgetting a haircut then putting my office together | 22:14 |
| sdake | so unlikely but the lat time i tested centos source which was yessterday it worked fine | 22:14 |
| sdake | at some point tonight probably after your asleep i'll be testing my demo and whatnot | 22:15 |
| sdake | multinode | 22:15 |
| *** ccesario_ has joined #openstack-kolla | 22:15 | |
| ccesario_ | night! | 22:15 |
| *** dwalsh has quit IRC | 22:16 | |
| sbezverk | sdake ok, I have my workaround so I am ok with my stuff | 22:16 |
| ccesario_ | guys, does someone have idea about exec it ? | 22:17 |
| ccesario_ | $ ip netns exec qrouter-010b0ce0-1dcd-46d6-9105-5907029da365 ifconfig Cannot open network namespace "qrouter-010b0ce0-1dcd-46d6-9105-5907029da365": Permission denied | 22:17 |
| ccesario_ | I 'm running a set of tests and checks, and I remember that ip netns exec it was allowed time ago (maybe neutron conaner it was running as root) | 22:19 |
| ccesario_ | another point is that... http://paste.openstack.org/show/493819/ | 22:22 |
| ccesario_ | since haproxy is enabled, why kibana config does not point to VIP address ? | 22:22 |
| ccesario_ | sdake: ^^ any info about these points ? | 22:23 |
| *** fragatina has joined #openstack-kolla | 22:24 | |
| *** sdake has quit IRC | 22:25 | |
| *** fragatina has quit IRC | 22:30 | |
| *** ayoung has joined #openstack-kolla | 22:31 | |
| *** Marga_ has joined #openstack-kolla | 22:32 | |
| *** daneyon_ has quit IRC | 22:33 | |
| *** Marga_ has quit IRC | 22:35 | |
| *** Marga_ has joined #openstack-kolla | 22:36 | |
| *** asalkeld has quit IRC | 22:38 | |
| *** asalkeld has joined #openstack-kolla | 22:41 | |
| *** jtriley has quit IRC | 22:49 | |
| *** mwheckmann has quit IRC | 22:52 | |
| *** blahRus has quit IRC | 22:59 | |
| *** daneyon_ has joined #openstack-kolla | 23:03 | |
| *** daneyon_ has quit IRC | 23:07 | |
| *** salv-orlando has quit IRC | 23:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/304895 | 23:09 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-mesos: Updated from global requirements https://review.openstack.org/302970 | 23:09 |
| *** yuanying has joined #openstack-kolla | 23:18 | |
| *** sdake has joined #openstack-kolla | 23:18 | |
| *** allen_gao has quit IRC | 23:22 | |
| *** daneyon_ has joined #openstack-kolla | 23:33 | |
| *** yuanying has quit IRC | 23:36 | |
| *** allen_gao has joined #openstack-kolla | 23:37 | |
| *** vhosakot has joined #openstack-kolla | 23:47 | |
| *** sdake has quit IRC | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
