Friday, 2016-07-22

*** chopmann has quit IRC00:03
*** ayoung has quit IRC00:06
*** diogogmt has quit IRC00:08
openstackgerritqinchunhua proposed openstack/kolla: Fix dict.values() PY3 compatible  https://review.openstack.org/34570600:13
*** jascott1_tmp has quit IRC00:15
*** banix has quit IRC00:24
*** ravig has joined #openstack-kolla00:26
*** brad[] has quit IRC00:28
Mech422v1k0d3n: ubuntu source builds seem to be having issues today00:33
Mech422v1k0d3n: but overall work pretty well for multi-node deployment - I rebuild multiple times a day...00:33
*** banix has joined #openstack-kolla00:35
*** jascott1_tmp has joined #openstack-kolla00:38
*** ravig has quit IRC00:42
*** sdake has quit IRC00:47
*** aernhart_ has joined #openstack-kolla00:47
*** jascott1_tmp has quit IRC00:53
*** ravig has joined #openstack-kolla00:54
*** aernhart__ has joined #openstack-kolla00:58
*** aernhart_ has quit IRC01:01
*** zhurong has joined #openstack-kolla01:05
*** daneyon has joined #openstack-kolla01:07
*** banix has quit IRC01:08
*** daneyon has quit IRC01:11
*** dave-mccowan has joined #openstack-kolla01:16
*** neophy has joined #openstack-kolla01:18
*** Liuqing has joined #openstack-kolla01:21
*** derekjhyang has joined #openstack-kolla01:24
derekjhyanghttps://www.irccloud.com/pastebin/zkXlW8ki/nova-libvirt%20setup%20cgmanager%20error01:26
derekjhyanggood morning, guys01:27
*** salv-orlando has joined #openstack-kolla01:29
*** jtriley has joined #openstack-kolla01:44
*** salv-orlando has quit IRC01:46
*** david-lyle has joined #openstack-kolla01:46
*** michauds has quit IRC01:48
*** jtriley_ has joined #openstack-kolla01:49
*** jtriley has quit IRC01:49
*** david-lyle has quit IRC01:56
*** Jeffrey4l_ has joined #openstack-kolla01:56
*** zhurong has quit IRC01:58
*** zhurong has joined #openstack-kolla01:59
*** ravig has quit IRC02:04
*** JianqingJiang has joined #openstack-kolla02:06
*** JianqingJiang has quit IRC02:11
*** daneyon has joined #openstack-kolla02:26
*** daneyon has quit IRC02:28
*** jascott1_tmp has joined #openstack-kolla02:30
*** daneyon_ has quit IRC02:30
*** jtriley_ has quit IRC02:35
*** ravig has joined #openstack-kolla02:37
*** ravig has quit IRC02:38
*** jascott1_tmp has quit IRC02:40
*** wmiller has quit IRC02:50
*** neophy has quit IRC02:51
*** wmiller has joined #openstack-kolla02:51
*** ravig has joined #openstack-kolla02:51
*** daneyon has joined #openstack-kolla02:55
*** unicell1 has joined #openstack-kolla02:56
*** Liuqing has quit IRC02:57
*** zhurong has quit IRC02:57
*** unicell has quit IRC02:57
*** Liuqing has joined #openstack-kolla02:57
*** zhurong has joined #openstack-kolla02:58
*** daneyon has quit IRC03:00
openstackgerritMerged openstack/kolla: Added ceilometer decision in neutron.conf for event notifications  https://review.openstack.org/34478203:00
openstackgerritqinchunhua proposed openstack/kolla: Fix dict.values() PY3 compatible  https://review.openstack.org/34570603:01
openstackgerritqinchunhua proposed openstack/kolla: Fix dict.values() PY3 compatible  https://review.openstack.org/34570603:02
openstackgerritMerged openstack/kolla: Bind the ceilometer-api process to the api_interface only  https://review.openstack.org/34545103:02
openstackgerritqinchunhua proposed openstack/kolla: Fix dict.values() PY3 compatible  https://review.openstack.org/34570603:06
*** rhallisey has quit IRC03:08
*** JianqingJiang has joined #openstack-kolla03:08
*** JianqingJiang has quit IRC03:08
*** JianqingJiang has joined #openstack-kolla03:09
*** yuanying has quit IRC03:21
*** JianqingJiang has quit IRC03:21
*** salv-orlando has joined #openstack-kolla03:22
*** brad[] has joined #openstack-kolla03:30
*** salv-orlando has quit IRC03:40
openstackgerritJeffrey Zhang proposed openstack/kolla: Implement mongodb replication set cluster  https://review.openstack.org/34575103:41
*** mark-casey has left #openstack-kolla03:43
*** dave-mccowan has quit IRC03:48
*** yuanying has joined #openstack-kolla03:50
*** ljjjustin has quit IRC04:09
openstackgerritJeffrey Zhang proposed openstack/kolla: Use lbaas alias instead of the class name  https://review.openstack.org/34577704:10
*** ravig has quit IRC04:11
*** ravig has joined #openstack-kolla04:11
*** ljjjustin has joined #openstack-kolla04:11
*** ravig has quit IRC04:19
*** ravig has joined #openstack-kolla04:20
*** ravig has quit IRC04:23
*** ravig has joined #openstack-kolla04:24
*** bdaca has joined #openstack-kolla04:31
*** salv-orl_ has joined #openstack-kolla04:42
*** daneyon has joined #openstack-kolla04:43
*** daneyon has quit IRC04:48
*** harlowja has joined #openstack-kolla04:57
*** sdake has joined #openstack-kolla04:57
*** smemon has joined #openstack-kolla05:06
*** ravig has quit IRC05:09
*** ravig has joined #openstack-kolla05:10
*** ravig has quit IRC05:12
*** ravig_ has joined #openstack-kolla05:12
*** ravig_ has quit IRC05:13
smemonHi05:18
smemonI'm trying to deploy Kolla all-in-one in a VM05:20
smemonwhen I run kolla-ansible pull I get the following error05:20
smemonfatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Unknown error message: Tag 3.0.0 not found in repository docker.io/kolla/centos-binary-kolla-toolbox"}05:20
smemonwhere should I look to resolve the error?05:21
smemonany suggestions?05:25
openstackgerritMathias proposed openstack/kolla: External Ceph - Implementation Nova  https://review.openstack.org/34470505:25
*** mewald has joined #openstack-kolla05:26
mewaldgood morning05:26
bjolomorning05:26
*** gbraad has quit IRC05:26
mewaldanyone care to talk about monitoring stack?05:26
*** jascott1_tmp has joined #openstack-kolla05:26
smemonMorning!05:27
bjoloyes05:27
bjolofirst one q for you mewald05:27
bjoloTLS... it is all terminated at haproxy right?05:27
mewaldbjolo: that's the typical setup, yes, and I believe Kolla does it the same way (haven't tried)05:28
bjoloi deployed a small swift with kolla yesterday to get object store right, but normal users get access denied05:29
bjologot to tired to dig down on it05:29
bjolook monitoring... whats your thinking?05:30
smemonCould someone please give me any hint about this error?05:31
smemonfatal: [localhost]: FAILED! => {"changed": false, "failed": true, "msg": "Unknown error message: Tag 3.0.0 not found in repository docker.io/kolla/centos-binary-kolla-toolbox"}05:31
bjolosmemon, you dont have any images with tag 3.0.005:31
mewaldsmemon: you are trying to pull a docker image from docker.io with a tag that doesnt exist05:31
mewaldset up your own registry and build your images05:32
mewaldOk, so I started implementing collectd + influxdb this week and had a look at influxdb clustering with influxdb-relay. I also looked at telegraf which seems to have a couple of advantages: 1) It uses http rest to talk to influxdb which can be load balanced 2) It allows the configuration of multiple influxdb databases as output targets. Collectd on the other hand typically uses UDP to write to influxdb which has its advantages but lacks05:32
smemonI tried with Tag 2.0.0 but also got the same error05:32
bjoloprivate docker registry is kinda horrible though05:32
smemonfor all-in-one deploye do we really need private registry?05:33
bjolonope, the build artifacts ends up as local images05:33
mewald2nd: I could set up an entire influxdb + influxdb-relay + haproxy stack but I feel the need to separate it out to a different set of hosts in order to keep that load away from controllers05:33
mewaldsmemon: in that case look at your global.yaml05:34
bjoloagree on that. controllers need to be clear from as much as possible05:34
mewaldyou might have set the tag wrong so docker will first try to find the image locally and fails, then goes to docker.io05:35
bjolosmemon, if you build locally for an all in one, openstack_version needs to match in kolla-build.conf and globals.conf05:35
mewaldbjolo: yeah we already have ELK on the controllers which sucks imho05:35
bjoloyes, i made some comments about that as well for rados_gw and swift proxy. default placement is on the controller nodes.05:36
bjolobad idea IMO05:36
bjolobut configurable :)05:36
bjolowhen you run fuel, you dont have that option05:37
bjolocontroller is controller + network node combined05:37
*** harlowja has quit IRC05:38
mewaldgenerally, the placement of influxdb + relay is configurable too, but if we really need an haproxy Kolla has to deploy a second keepalived + haproxy cluster05:38
mewaldand I don't know if that can be done easily05:38
bjolomewald, so how critical is you time metrics? my influxdb is currently wiped, but i have that running as vm on vmware05:38
bjolovmware cluster05:38
bjolowith seperate disks for data. so if the phys node goes down, restart on other node05:39
*** ravig has joined #openstack-kolla05:39
bjoloif influx node gets corrupted, i can deploy a new one and attach the data drive.05:39
bjolothe cool think about collectd is that it caches locally05:40
bjoloso if influx is down, it will backfill ones it gets up05:40
*** sdake_ has joined #openstack-kolla05:41
mewaldhmm - difficult ...05:41
bjoloone way of doing it...05:41
bjoloin the target env, what options do you have available?05:41
mewaldyou know I am thinking of extending this one day to have trending and prediction that stuff available. But maybe I am thinking to far ahead05:42
mewaldand a single instance influxdb would be sufficient05:42
mewaldwhich invalidates my telegraf argument :D05:43
smemonbjolo, I think kolla-build.conf is not generated yet05:43
*** sdake has quit IRC05:43
mewaldsmemon: I dont have to build yourself. If you node has access to docker.io just put a tag that docker.io has into global.yml05:43
bjoloi dont think you have to worry about performance on one influxdb. when testing it out i had about 100 nodes runing collectd with quite a few plugins all forwarding to one quad core virtual influx on vmware05:44
mewaldok, so I will start with a patch that includes single instance influxdb05:46
mewaldremains the question of collectd vs telegraf05:46
mewaldI somehow like the idea to have a consistent stack (TICK stack)05:46
derekjhyanghttps://www.irccloud.com/pastebin/mrLcMlVE/05:46
mewaldand it might allow for better extensibility later on05:46
smemonmewald, thanks! will look into it05:47
derekjhyanghttps://www.irccloud.com/pastebin/zkXlW8ki/nova-libvirt%20setup%20cgmanager%20error05:47
bjolomewald, have you looked at beats from elastic?05:47
mewaldderekjhyang: seriously? I post your conversational text in a pastebin? :D05:47
mewaldbjolo: not really no, I have read about it some time back. why?05:48
bjolohttps://www.elastic.co/products/beats05:48
bjolothis is for kolla right?05:48
bjolosince we already runing elastic for logs05:49
bjolowith beats we can add timeseries also05:49
bjologet everything in the same DB, using the same kibana frontend05:49
derekjhyangmewald: where? I've not found it @@05:50
mewaldoh boy, thats horrible. too many options :D05:50
bjoloi know!!!05:50
mewaldderekjhyang: https://www.irccloud.com/pastebin/mrLcMlVE/ "Hey folks, I have a question:" :D no worries, was just kidding05:50
mewaldbjolo: but I think that option is too far from being well tested and proven to work05:51
bjoloi did kinda a technolog mapping a couple of months back, and it is just insane how many different options there is for data today05:51
mewaldyeah, only a few days back someone here in the channel said prometheus.io :D never heard of that05:51
mewaldI stumbled across kafka and other stuff though05:51
bjoloyes, beats is way new, but it does have some synergies05:51
derekjhyangmewald: Oh, I see. But I have no ideas to resolve it. Any suggestions?05:52
mewaldderekjhyang: which version of ubuntu are you building?05:52
derekjhyangnewald: Ubutnu 14.0405:53
derekjhyangnewald: Ubuntu 14.0405:53
mewaldhmm, let me try something05:53
mewaldderekjhyang: that's weird, I just installed cgmanager on trusty without problems https://gist.github.com/mewald1/96227c96ccaaa7bfd0c7f7b44e98e7a805:55
derekjhyangmewald: As far as I know, libvirt-bin depends on the package cgmanager (a kinda cgroup mgmt package), but it responded me that it couldn't setup /etc/init.d/cgmanger correctly.05:57
derekjhyangmemwald: That's so werid, right? At first, I can build the image, but a few days ago, it can't.05:58
derekjhyanghttps://www.irccloud.com/pastebin/GF52M3vL/05:59
bjolothere was some talk on the channel last night about issues with ubuntu builds05:59
bjoloi build centos, so i was not paying that much attention, but look at the logs from later yesterday, early this morning06:00
*** kklimonda_ has joined #openstack-kolla06:02
*** kklimonda_ has quit IRC06:02
derekjhyanghttps://www.irccloud.com/pastebin/LVXHmygY/06:03
*** aernhart__ has quit IRC06:07
mewaldhmm, I seriously have not clue … could it be that your base images is faulty? like something wasn't downloaded correctly and now causes problems?06:12
bjoloit seems most people favor ubuntu containers. any particular reason for that?06:14
*** sogabe has quit IRC06:16
mewaldbjolo: probably just personal preference.06:17
*** smemon has quit IRC06:17
derekjhyangbjolo: yup, I agree with mewald. I've also used the centos7 image in the devstack initially, but I met some issues that I couldn't resolve on OpenStack. After that, I tried to use another distro (Ubuntu), then anything has seemed to be okay.06:22
mewaldderekjhyang: have you tried resetting everything to start including deletion of all images?06:23
bjoloubuntu is the primary dev platform for openstack so it makes sense from that end06:23
bjolopersonally i prefer to run centos on the host. in my experience, centos/RHEL is little more stable when it comes to hardware06:24
*** dmueller-evoila has joined #openstack-kolla06:25
bjoloredhat also owns ansible, and ceph. One would think those will work *best* on rhel/centos06:25
bjoloso i just picked centos for containers since i run centos on the hosts06:26
derekjhyangmewald: Definitely! Only thing I never tried is killing the Ubuntu-based image, because I've built a docker private registry repo for kolla.06:26
mewaldtry killing it all just to make sure06:26
mewaldyou can build stuff with "tools/build.py regex nova-libvirt" just to be faster :)06:27
bjolomewald, aha thats how you do it :)06:28
derekjhyangmewald: Is there the fully detailed example ?06:29
mewald:D have you always waited 60min for the entire build? :D06:29
mewaldderekjhyang: of what?06:29
derekjhyangmewald: for this, "tools/build.py regex nova-libvirt" just to be faster06:29
bjolono i meant building in git repo06:29
mewaldbjolo: ahh06:29
bjoloi have always done pip install, kolla-build06:30
mewaldderekjhyang: tools/build.py references a script in the kolla git repository06:30
mewaldthe command is already complete06:30
mewaldI was just using the script from the repo instead what "pip install ./kolla" installs on your system06:30
*** daneyon has joined #openstack-kolla06:31
derekjhyangmewald: I know that, I can the help command of tools/build.py.06:32
Mech422mewald: yeah - don't forget reiman, prometheus, influxdb/Tick stack, etc etc - its a huge list06:33
derekjhyangBut I don't know how to use the [regex]06:33
derekjhyanghttps://www.irccloud.com/pastebin/kYRvQ9lQ/06:33
mewaldjust type exactly as I posted :D06:33
mewaldnova-libvirt IS the regex06:34
mewaldMech422: I decided to go with telegraf + single instance influxdb for now06:34
derekjhyangmewald: Oh, I know what you meant. :)06:34
Mech422derekjhyang: there is a problem with ubuntu source build as of 12 hours ago..06:35
mewaldderekjhyang: are you building source?06:35
Mech422derekjhyang: the base container appears to need apt-transport-https installed - I'm getting ready to patch06:35
Mech422mewald: I'm thinking about TICK for my home lab06:35
mewaldI just doing the TI of TICK now ^^06:36
*** daneyon has quit IRC06:36
mewaldactually it's TIG :D I add Grafana optionally06:36
Mech422mewald: a full os install + ubuntu source build/deploy is about an hour for me now06:36
derekjhyangmewald: Yes, I cloned it from github with liberty version.06:36
Mech422mewald: yeah - gonna add grafana to mine too06:36
openstackgerritLi Yingjun proposed openstack/kolla: Use neutron-server image for lbaas bootstrap  https://review.openstack.org/34584306:37
Mech422mewald: I was thinking of prometheus with influxdb and telegraf/grafana too06:37
mewaldyeah I am sure there is a lot to think about but probably very few that actually works ^^06:37
derekjhyangMech422: Oh, really?! Any news about that Ubuntu problem ?06:38
*** ravig has quit IRC06:38
Mech422derekjhyang: give me an hour or so - I'm gonna rebuild now and see if I can fix it - just need a 1 liner in the docker/base/Dockerfile to install apt-transport-https06:38
*** kklimonda_ has joined #openstack-kolla06:39
derekjhyangMech422: No problem, thank you for your help06:39
*** nick-ma has joined #openstack-kolla06:41
derekjhyangMech422: Which version of the Ubuntu image do you build ?06:45
Mech422master06:47
Mech422I build ubuntu master from source with ansible 2.1.0.0 multi-node with ceph enabled using fresh installs of Ubuntu 14.04 for all hosts06:48
Mech422or at least I do when network-manager doesn't hose the DNS on my desktop06:49
* Mech422 screams06:49
Mech422ok - got the Ubuntu installs started...we're cooking with gas now!06:52
derekjhyanggogo06:53
Mech422derekjhyang: here's the patch I applied: http://paste.openstack.org/show/539666/06:53
Mech422derekjhyang: if I works, I'll submit it upstream06:54
*** jascott1_tmp has quit IRC06:54
bjoloMech422, get rid of NetworkManager06:54
derekjhyangWish it will not be the greenhouse gases lol06:54
Mech422bjolo: yeah - it has become a pain with my work vpn...06:55
Mech422bjolo: its prolly not network managers fault - the vpn prolly specifies new nameservers...06:55
Mech422bjolo: but I could have sworn I told it to get 'routes only' and ignore nameservers06:55
*** sbezverk_ has joined #openstack-kolla06:56
*** b_bezak has joined #openstack-kolla06:57
*** sbezverk has quit IRC06:58
*** jascott1_tmp has joined #openstack-kolla07:00
bjologee i have no idea why swift is giving access denied07:01
*** chopmann has joined #openstack-kolla07:02
nick-mahi all, i'm trying to deploy kolla, but it failed due to 'tag not found for heka'. how can I fix this problem? https://bugs.launchpad.net/kolla/+bug/160549207:02
openstackLaunchpad bug 1605492 in kolla "tag 3.0.0 doesn't exist for heka" [Undecided,New]07:02
Mech422nick-ma: I haven't seen that one...07:04
Mech422nick-ma: I'm rebuilding my images/cluster now, give me an hour or so, and I'll see what I get07:04
bjolodoes kolla-build generate any log files? i.e. i mean detailed manifest files so one can backtrack exactly what was installed in each container?07:06
*** sdake has joined #openstack-kolla07:07
*** sdake_ has quit IRC07:07
derekjhyangbjolo: maybe you can try to modify kolla-build.conf with the option 'debug=True', then using the linux pipe command like ' 2>&1 | tee kolla-build.log'07:09
*** JianqingJiang has joined #openstack-kolla07:09
bjoloif i had time when i come back from vacation end of august, i want to create a docker image named kolla-buildserver or something like that07:11
bjolohave a docker registry in it and a simple web ui where one can specify exactly what to build and keep an artifact repository07:12
openstackgerritZhijiang Hu proposed openstack/kolla: Prevent incomplete intermediate or incomplete final image when building rabbitmq  https://review.openstack.org/34587207:12
derekjhyangbjolo: haha, I've used the Jenkins to trigger Kolla build task. Hence, I have the full log when I build it every time.07:12
bjoloderekjhyang, nice07:13
nick-maok, thanks. i will try to find out the log files.07:13
derekjhyangbjolo: maybe you can try to do it.07:13
bjoloor put jenkins in the kolla-buildserver :)07:13
dmueller-evoilatwpython dict07:13
bjoloi just feel that kolla needs to be alittle bit easier for operators07:14
nick-ma@Mech422: thanks.07:14
derekjhyangIn fact, I imitate the progress of OpenStack CI to build my own DevOps service.07:15
*** salv-orl_ has quit IRC07:16
*** tyrola has joined #openstack-kolla07:17
Mech422bjolo: I actually drive kolla from a separate set of ansible playbooks07:17
Mech422bjolo: they setup the hosts, apply patches, and invoke the kolla stuff for me07:17
bjoloMech422, i know, you told me :)07:17
bjoloand its very cool07:17
Mech422bjolo: hopefully a lot of that will be added to kolla soon07:17
Mech422bjolo: oh ...yeah, I talk so much I forgot what I've said :-P07:18
bjolobut lets think kolla running rackspace and a large team of ops that need to work together07:18
Mech422bjolo: a big part of that is 'plugability' - which is high on the list of stuff to do07:18
bjololarge scale ops, large clouds07:18
Mech422bjolo: but first, I think they want to solidify the customization stuff07:19
bjolothe toolset and processes needed for that07:19
bjoloMech422, agree, kolla is not there yet, but...07:19
bjoloi have some opinions on how the code is structured today from an operators point of view07:20
bjoloit is not that it is wrong or doesnt work07:20
bjoloit is just in the large scale ops scenario, it is probably not the way you want it07:20
Mech422bjolo: I think plugability and docs will go a long way...07:21
bjoloyes definetly07:21
Mech422bjolo: kolla can do stuff like add/remove a node, upgrade the cluster, etc - its just not easy to find the docs/procedures07:21
bjoloagree07:22
bjoloim using kolla 2 right now, so upgrades/reconfig does not always work thou07:22
Mech422bjolo: I'm going to start doing rolling-upgrades of our QA cluster with Kolla end of next week...07:22
Mech422bjolo: hopefully, that will give me an excuse to help with docs07:23
bjoloMech422, you running the QA cluster?07:24
Mech422bjolo: err...for my employer - not the kolla qa stuff07:25
bjolook07:25
*** chopmann has quit IRC07:25
bjoloaha ok now i read your line again07:25
Mech422bjolo: its basically POC, so we're planning on doing weekly rolling upgrades till newton is cut, just to get some experience on upgrade procedure07:26
*** chopmann has joined #openstack-kolla07:26
mewaldGuys, I am having this problem again where neutron_dhcp_agent starts and the entire system starts to run mad. OOM kills and complete unavailability of the system07:26
mewalddid anyone make any progress with this?07:26
bjolomewald, kolla 307:26
bjolo?07:26
mewaldyes07:26
mewaldmaster07:26
bjoloi see this on kolla 2. no matter what image i run07:28
bjolo[root@eselde02u37 tools]# ps -elf | grep defunct07:28
bjolo1 Z root     21618 19473  0  80   0 -     0 exit   07:46 ?        00:00:00 [dnsmasq] <defunct>07:28
bjolo1 Z root     21978 19534  0  80   0 -     0 exit   07:46 ?        00:00:00 [neutron-ns-meta] <defunct>07:28
bjolo0 S root     24145  9935  0  80   0 - 28162 pipe_w 09:27 pts/1    00:00:00 grep --color=auto defunct07:28
*** salv-orlando has joined #openstack-kolla07:28
mewaldawesome07:28
Mech422huh - I haven't had any issues with that...07:28
bjoloi just want to sort out this swift access issue, then im done done done done and can go on vacation :)07:29
openstackgerritZhijiang Hu proposed openstack/kolla: Prevent incomplete intermediate or incomplete final image when building rabbitmq  https://review.openstack.org/34587207:29
*** lulei has quit IRC07:29
bjolodnsmasq still works thou, it just seems like the container fails on some07:31
bjolo[root@eselde02u37 tools]# docker exec neutron_dhcp_agent ps -el | grep dnsmasq07:31
bjolo1 Z     0    88     1  0  80   0 -     0 exit   ?        00:00:00 dnsmasq <defunct>07:31
bjolo5 S    99    89     1  0  80   0 -  3889 poll_s ?        00:00:00 dnsmasq07:31
*** chopmann has quit IRC07:31
bjoloso i want to move my swift proxies from control to network nodes. no new images or anything, just simply move them. what would be the "correct" way of doing that?07:33
mewalda reset of the box solves it07:33
mewaldvery strange bug07:33
bjolokolla-ansible deploy?07:33
bjolokolla-ansible reconfigure; kolla-ansible deploy?07:33
mewaldjust a deplyo07:33
mewalddeploy07:33
bjoloi have update the inventory file accordingly of course :)07:33
bjolomewald, and then stop the old proxies manually and remove image07:34
bjolo?07:34
mewaldalso, after the reset of the box "docker ps -a" hangs for a while07:34
*** jascott1_tmp has quit IRC07:44
*** aernhart__ has joined #openstack-kolla07:45
openstackgerritLi Yingjun proposed openstack/kolla: Add note for enable_mongodb option  https://review.openstack.org/34588107:46
*** jascott1_tmp has joined #openstack-kolla07:46
*** tyrola has quit IRC07:47
*** tyrola has joined #openstack-kolla07:47
b_bezakbjolo:, I have the same case on 2.0.2 with zombie processes - dnsmasq, metadata proxy, and some time IP (when removing networks)07:50
b_bezakbjolo: I am using multinode baremetal (basically 3x alloinone), with dvr+l3ha on 16.04.107:53
b_bezakand I am not sure whether it is a problem with docker/kernel version, or kolla itself07:54
bjolob_bezak, ok bug confirmed then :)07:54
b_bezakbut it looks like - when process is dropping root then this old root one is going zombie07:55
bjoloi have 3 control, 3 network, 10 compute, 6 storage. no customization anywhere really07:55
b_bezakso after days of testing - removing networks, creating etc. I had max ~700 zombies :P07:55
bjoloouch07:55
b_bezakI saw that next week there will be new release of docker 1.12, which supposedly is fixing some defunct problems07:56
b_bezaklet's see :)07:56
bjolocentos 7 hosts07:56
b_bezakheh, so probably not kernel problem07:56
b_bezakdocker version?07:56
bjolob_bezak, 1.11.207:57
b_bezaklatest then07:57
bjoloyup07:57
b_bezaklet's hope 1.12 will fix it :)07:57
bjolois it a docker bug though?07:57
b_bezaknot sure07:57
bjolohave you done any investigation07:57
b_bezaknot really. just some testing with regular l3agent settings. and it was the same of course07:58
bjoloi have not digged into it, but i have defunct and working dnsmasq in the same container so i assumed it is either openstack07:58
bjoloor the way that kolla starts up the container. i.e. how it spawns a new dnsmasq in the container07:59
b_bezakyeah, it is not functional bug - processes are working07:59
*** jascott1_tmp has quit IRC07:59
b_bezakbjolo: are you using kolla 3?07:59
bjolo[root@eselde02u35 haproxy]# docker exec -it neutron_dhcp_agent bash08:00
bjolo(neutron-dhcp-agent)[neutron@eselde02u35 /]$ ps -elf08:00
bjoloF S UID        PID  PPID  C PRI  NI ADDR SZ WCHAN  STIME TTY          TIME CMD08:00
bjolo4 S neutron      1     0  1  80   0 - 51991 ep_pol 07:41 ?        00:02:38 /var/lib/kolla/venv/bin/python /var/lib/08:00
bjolo1 Z root        87     1  0  80   0 -     0 exit   07:46 ?        00:00:00 [dnsmasq] <defunct>08:00
bjolo5 S nobody      88     1  0  80   0 -  3889 poll_s 07:46 ?        00:00:00 dnsmasq --no-hosts --no-resolv --strict-08:00
bjolo4 S neutron    113     0  0  80   0 -  3344 wait   09:59 ?        00:00:00 bash08:00
bjolo0 R neutron    141   113  0  80   0 - 12256 -      09:59 ?        00:00:00 ps -elf08:00
bjolokolla 2 latest08:00
b_bezakok08:00
b_bezakroot@infra2:~# ps -ef | grep -i defunc | wc -l08:00
b_bezak1908:00
bjoloim new to kolla so im just trying to learn the ropes right now and get something up so my users can try it out.08:01
bjolowhen im back from vacation end of august, i will jump on kolla 308:01
Mech422hmm - not much info on heka vs hindsight08:10
*** openstackgerrit has quit IRC08:11
*** manjeets- has quit IRC08:11
*** flaper87 has quit IRC08:12
*** tyrola has quit IRC08:12
*** flaper87 has joined #openstack-kolla08:12
*** flaper87 has quit IRC08:12
*** flaper87 has joined #openstack-kolla08:12
*** jrist has quit IRC08:12
*** elemoine- has quit IRC08:12
*** tyrola has joined #openstack-kolla08:12
derekjhyangWhat's the tag 'heka'?08:13
*** shardy has joined #openstack-kolla08:13
*** jrist has joined #openstack-kolla08:13
*** openstackgerrit has joined #openstack-kolla08:14
*** elemoine has joined #openstack-kolla08:14
*** JianqingJiang has quit IRC08:15
*** JianqingJiang has joined #openstack-kolla08:16
*** sdake_ has joined #openstack-kolla08:17
Mech422derekjhyang: dam - my ubuntu source build crapped out with that https transport error08:18
Mech422derekjhyang: so my one liner isn't cutting it08:18
*** sdake has quit IRC08:19
*** daneyon has joined #openstack-kolla08:20
*** sdake_ has quit IRC08:21
bjolook anyone on the channel that knows swift and can help me TS?08:24
bjoloi have gone through the docs, etc and it all looks correct08:24
*** daneyon has quit IRC08:24
*** aernhart_ has joined #openstack-kolla08:24
*** tyrola has quit IRC08:25
bjoloswift works, but only admin in admin tenant can list,create,delete08:25
*** tyrola has joined #openstack-kolla08:25
bjoloall others, including admin in demo tenant, get access denied08:25
*** aernhart__ has quit IRC08:28
*** lulei has joined #openstack-kolla08:32
*** bjolo is now known as bjolo-afk08:38
*** sogabe has joined #openstack-kolla08:42
*** bjolo-afk has quit IRC08:43
openstackgerritLi Yingjun proposed openstack/kolla: Install python-bson for Ceilometer for ubuntu  https://review.openstack.org/34591208:45
derekjhyangwhat's the meaning of "TS" ?08:47
*** lulei has quit IRC08:54
*** lulei has joined #openstack-kolla08:55
mewaldI have a setup with internal and external VIP (haproxy) but a load balancer that translates from internet to external vip and does SSL termination. Is there a way to tell Kolla to deploy https endpoints into the keystone service catalog?08:55
*** gfidente has joined #openstack-kolla08:55
*** gfidente has joined #openstack-kolla08:55
*** JianqingJiang has quit IRC08:55
*** JianqingJiang has joined #openstack-kolla08:56
Mech422derekjhyang: ok - I got past the elastic.co https transport error...now I'm getting an error on the percona repos08:56
derekjhyang@Mech422: thank you08:58
Mech422derekjhyang: this patch fixes the https thing, but not sure if it broke percona: http://paste.openstack.org/show/539672/08:59
Mech422derekjhyang: not sure why it would break percona repo and no other repo though08:59
*** aernhart__ has joined #openstack-kolla09:00
*** aernhart__ has quit IRC09:01
*** aernhart__ has joined #openstack-kolla09:01
*** manjeets_ has joined #openstack-kolla09:03
*** aernhart_ has quit IRC09:04
*** JianqingJiang has quit IRC09:07
*** Jianqing_ has joined #openstack-kolla09:08
*** mbound_ has joined #openstack-kolla09:13
*** mbound has quit IRC09:17
*** dwalsh has joined #openstack-kolla09:17
*** dwalsh_ has joined #openstack-kolla09:19
*** b_bezak_ has joined #openstack-kolla09:23
*** aernhart__ has quit IRC09:24
*** b_bezak has quit IRC09:27
*** Jianqing_ has quit IRC09:27
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593809:28
*** dolpher has joined #openstack-kolla09:29
openstackgerritMerged openstack/kolla: implement Ansible Tempest role  https://review.openstack.org/32152209:34
*** athomas has quit IRC09:36
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593809:42
mewaldGuys please take a look at https://review.openstack.org/#/c/345938/ It's a first step towards a performance monitoring stack in Kolla.09:43
patchbotmewald: patch 345938 - kolla - Implement Basic Performance Monitoring09:43
*** athomas has joined #openstack-kolla09:43
*** salv-orlando has quit IRC09:49
*** salv-orlando has joined #openstack-kolla09:49
*** egonzalez has joined #openstack-kolla09:49
*** salv-orlando has quit IRC09:54
Mech422mewald: I think there are some customization blocks that go in the Dockerfile ?09:54
mewaldMech422: what exactly?09:55
Mech422mewald: look at the 'base' Dockerfile.j209:55
*** dolpher has left #openstack-kolla09:55
Mech422mewald: block base_header09:55
Mech422mewald: not sure if each template is supposed to have block like that09:55
Mech422mewald: inc0 did a whole bunch of stuff on customization recently - you could check with him09:56
mewaldMech422: I used keystone/Dockerfile.j2 as a template09:56
Mech422mewald: dunno - like I said, inc0 knows more about that... ask him, and if its good, its good :-)09:57
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593810:00
Mech422mewald: btw - nice work :-)10:01
openstackgerritEduardo Gonzalez proposed openstack/kolla: Add Senlin Ansible role  https://review.openstack.org/33972510:01
mewaldwell thanks ^^ There is a lot of room for improvement10:01
Mech422mewald: there always is :-)  But thats a nice chunk of functionality we didn't have before !! :-D10:03
*** zhurong has quit IRC10:06
*** ntpttr___ has joined #openstack-kolla10:07
*** daneyon has joined #openstack-kolla10:08
*** Liuqing has quit IRC10:08
*** ntpttr has quit IRC10:09
mewaldMech422: What I really want to add is that the datasource is automatically created in Grafana10:12
mewaldand having some prepared dashboards would be awesome, too10:13
Mech422mewald: flashey lights always impress mgmt - be a nice 'selling' point for kolla10:13
*** daneyon has quit IRC10:13
mewaldyeah ^^10:13
mewaldand creating dashboards is really time consuming10:13
mewaldI have spent weeks with that shit10:14
mewaldwhat will be interesting is to integrate all services properly10:15
mewaldlike haproxy stats, ceph, etc10:15
mewaldthat's the next step10:15
Mech422mewald: well, for my small contribution for the night - I think I got the ubuntu source builds working again10:16
mewaldnice!10:16
Mech422mewald: just needed to add apt-transport-https and ca-certificates to the base image10:16
mewaldtell derekjhyang :)10:16
Mech422its a 2 liner, so hopefully it can be pushed out tommorrow as a 'trivialfix'10:16
Mech422I think he's asleep ?10:16
Mech422derekjhyang: you still around ?10:17
*** godleon has joined #openstack-kolla10:29
*** egonzalez has quit IRC10:39
openstackgerritBartłomiej Daca proposed openstack/kolla: Fixing kolla's problem with ansible var precedence  https://review.openstack.org/34597210:39
*** sogabe has quit IRC10:41
openstackgerritDave Walker proposed openstack/kolla: Add Watcher ansible roles and templates  https://review.openstack.org/33755610:45
derekjhyangMech422: Okay, where should I get the latest Ubuntu image?10:55
*** neophy has joined #openstack-kolla11:07
*** salv-orlando has joined #openstack-kolla11:07
*** zhurong has joined #openstack-kolla11:12
*** bjolo has joined #openstack-kolla11:14
*** zhurong has quit IRC11:16
*** zhurong has joined #openstack-kolla11:17
*** ccesario has joined #openstack-kolla11:18
*** mliima has joined #openstack-kolla11:19
Mech422derekjhyang: eh?11:20
Mech422derekjhyang: I didn't make an image for upload - I was just running a local build from source11:20
Mech422derekjhyang: here's the patch I used: http://paste.openstack.org/show/539735/11:22
*** rhallisey has joined #openstack-kolla11:35
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593811:55
mewaldhow to people make sure they get trailing whitespaces and all that stuff right before they push into gerrit? It takes to much time reacting to comments and providing new patchsets :D11:57
Mech422mewald: for python, there's pep8.. dunno tools for yaml or Dockerfiles11:58
*** zeus has quit IRC12:00
mewaldok12:10
bjolomewald, sed -i 's/[ \t]*$//' "$1"12:27
mewaldbjolo: ha, didnt even think of that :D12:27
bjololinux to the rescue! :)12:28
bjolomewald, what editor are you using?12:29
bjolovim?12:29
mewaldyes12:29
bjolocan do in vim as well12:29
bjolohttp://vim.wikia.com/wiki/Remove_unwanted_spaces12:29
bjolovim +1 :)12:30
bjolois ubuntu builds still having issues?12:30
bjolocant get my swift access issue sorted out. it is like the requests just disappear Figure i try deploying with ubuntu12:31
bjolonot really hoping much that will fix it, but wanted to try ubuntu anyway12:33
mewaldI like vim but not enough to go through this scripting hell :D12:33
bjoloheheh12:33
bjoloagree vim hacking is weird12:33
bjolosearch replace is enough for me12:33
*** bdaca has quit IRC12:40
*** dwalsh_ has quit IRC12:40
*** dwalsh has quit IRC12:41
*** dave-mccowan has joined #openstack-kolla12:51
openstackgerritNenad Radojevic proposed openstack/kolla: Keystone "user" role added by default.  https://review.openstack.org/34600812:51
*** salv-orlando has quit IRC12:53
*** inc0 has joined #openstack-kolla12:53
*** Serlex has joined #openstack-kolla12:56
bjolohmm if i do a git review -s, it should come up here on irc right?12:58
*** inc0 has quit IRC12:59
mewaldno12:59
mewaldonly "git review"12:59
bjolook12:59
bjoloerror missing change id?13:00
bjoloi just corrected a small typo in the swift documentation13:00
mewaldgit review didnt work, did it?13:00
mewaldgit review -s13:00
*** shardy is now known as shardy_mtg13:01
bjoloqbjolof@eseldulx1040:~/tmp/kolla/doc:15:01:19> git commit -s13:01
bjoloOn branch master13:01
bjoloYour branch is ahead of 'origin/master' by 1 commit.13:01
bjolo  (use "git push" to publish your local commits)13:01
bjolonothing to commit, working directory clean13:01
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593813:02
openstackgerritbjorn lofdahl proposed openstack/kolla: Added weight when creating drives in swift  https://review.openstack.org/34602413:02
bjoloouch13:03
bjolowait damn13:04
mewaldQuestion for whoever is deep in the code of Kolla: I want to add code to automatically create a "datasource" in Grafana connceting it to the influxdb instance. Grafana takes a while when starting before it is actually up accepting API requests. The only way to programmatically add a datasource I see is via API request though. What is the proper way to do this? One way I see is: Create bootstrap script that waits until API is ready, the13:05
bjolothe weigt is there, its just bad format in my browser. need to scroll horizontal to see it13:06
bjolomewald, can i delete my own commit?13:07
bjolo-113:07
rhalliseymewald, I think what you want is to do some sort of bootstrap step which starts the service. Then ansible will wait until it's ready to create a datasource13:08
mewaldrhallisey: hmm currently the runcommand is executed by https://github.com/openstack/kolla/blob/master/docker/base/start.sh So one way would be to add e.g. /run.sh wrapping the grafana service start command and use /run.sh as the commend to execute in config.json13:11
mewaldIs that what you mean?13:11
*** inc0 has joined #openstack-kolla13:11
rhalliseymewald, I don't think we want to automatically create the datastore in the container13:12
inc0good morning13:12
bjolomorning inc013:12
rhalliseywe want it as a separate action done by ansible13:12
*** lulei has quit IRC13:12
mewaldrhallisey: to the same bootstrap mechanism used by Keystone could be used13:13
mewaldinject bootstrap script via Docker file, execute it via Ansible (docker exec)13:13
inc0mewald, what's the issue13:14
inc0?13:14
rhalliseymewald, is this something that needs to be in the contianer?13:14
rhalliseyor something that can get created remotely like a user/password13:14
mewaldhmm, no actually not13:14
*** jtriley has joined #openstack-kolla13:15
mewaldyou're right13:15
mewaldcould just be run remotely13:15
rhalliseyok then look at using kolla-toolbox13:15
*** ravig has joined #openstack-kolla13:15
mewaldthey way I understand kolla-toolbox is as a set of arbitrary scripts, right? Why are they not just executed from Ansible directly?13:16
inc0mewald, reason we created it was because of shade13:17
rhalliseymewald, you can add a custom module in the ansible dir13:17
rhalliseythat works too13:17
inc0we had this small part of ansible 2.0 when we were still ansible 1.913:17
*** mbound has joined #openstack-kolla13:18
rhalliseyshade has like 1000 deps13:18
bjoloshade as in libshade?13:18
mewaldwhat is "shade"?13:18
inc0now technically we can get rid of it as shade is already in ansible 2 and we use it, however kolla-toolbox still is helpful to other consumers like k8s13:18
inc0bjolo shade as openstack module for ansible13:18
inc0or ansible module for openstack;)13:19
*** mbound_ has quit IRC13:21
bjolohttp://docs.openstack.org/infra/shade/13:22
bjolohttps://pypi.python.org/pypi/shade-ansible13:22
bjoloright?13:22
inc0"now it's all in ansible?13:24
inc0"13:24
inc0but right, that was the thing13:24
bjolook13:26
bjoloive been playing with kolla 2 now for a few weeks, and it is great! little rough on some spots, but overall im sold :)13:27
inc0glad to hear13:27
bjolokudos to everyone who has made this happen13:27
rhalliseybjolo, let the community know about any pain points :).  We'd love to help13:27
inc0we work on these edges, but if we would do it all, we'd wouldn't get to keep ourselves to commit to kolla, so we take our time with this;)13:28
bjolowell, my back is kinda sore. thats one painpoint13:28
bjolo;)13:28
rhalliseyha13:28
rhalliseyya we can't help you there :)13:29
bjolooff the top of my head, documentation is a little hard to follow. one thing is it says git clone.... right, but you actually have to checkout the branch you wanna deploy13:29
rhalliseybjolo, agreed, there is a massive re work in progress13:29
inc0bjolo docs are worst part of kolla right now imho13:29
bjolomost operators i know unfortunately dont have a lot of git knowledge13:30
inc0so if you want to commit something, that would be great place to start;)13:30
bjoloespecially branching13:30
bjoloand as an operator i dont really need the git repo either (besides for the tools folder)13:30
bjolowould it be possible to setup a real pip package for kolla?13:31
bjoloso as an operator, i would just have to do a: pip install kolla version=2.0.213:31
inc0yes13:31
inc0should be;)13:32
inc0let me try13:32
bjolorhallisey, i go on vacation today and be gone for a month, but when im back i plan to help out if i can be of any use13:33
rhalliseyawesome13:33
bjoloreading the "new" docs and have ops kollegues try them out13:33
bjolois a good place to start13:34
*** shardy_mtg has quit IRC13:34
*** shardy_mtg has joined #openstack-kolla13:36
*** shardy_mtg is now known as shardy13:36
*** diogogmt has joined #openstack-kolla13:38
bjoloinc0, ansible is also in pip right?13:38
bjoloinc0, so if, pip install kolla kan select the right ansible version as well?13:39
inc0bjolo in our docs we do have pip isntall ansible==2.1.013:44
bjoloinc0, where?13:44
inc0btw bjolo -  Bartek?:)13:44
bjolo?13:45
*** mgoddard_ has joined #openstack-kolla13:45
inc0nvm, I was seeing another Polish name in reviews and I was wondering if that's you13:45
bjoloim swedish!13:45
bjolo:)13:45
bjolobjorn lofdahl13:45
inc0close enough13:45
inc0;)13:45
bjolohaha to you maybe13:45
inc0well, I'm Polish born and bred so I know significance of Baltic sea between our countries13:46
inc0;)13:46
bjoloah ok13:46
inc0but I used to live in Gdansk and there is ferry twice a day to Sweden13:46
bjolopoland is very nice13:47
inc0so it felt close;)13:47
bjoloyes13:47
bjolothere is a ferry from karlskrona to gdynia i have take a few times13:48
bjolosopol area is GREAT13:48
*** mgoddard has quit IRC13:48
inc0yeah, I really like that part of Poland13:48
mewaldI just wrote a script for kolla-toolbox to create a grafana datasource. What is the best way to deal with the situation where grafana is not up yet? Wait infinitely in the script until it is available? Can we handle this in ansible somehow? Like retry 10 times in 20s intervals or something?13:48
inc0mewald, there is wait_for clausule in ansible13:49
*** ravig has quit IRC13:49
inc0http://docs.ansible.com/ansible/wait_for_module.html13:49
mewaldnice will look at that!13:49
inc0mewald, grafana? you're adding monitoring to kolla?:)13:50
mewaldyup13:50
inc0cool!13:50
inc0any chances to have it upstream?13:50
mewaldpart 1 is for review already :)13:50
inc0sensu one?13:50
mewaldno sensu yet13:50
mewaldtelegraf + influxdb13:50
mewald+ grafana13:50
inc0ok, I've seen sensu in reviews13:50
inc0hold on13:50
inc0let me find it13:51
inc0also brb, changing PCs13:51
*** inc0 has quit IRC13:51
*** inc0 has joined #openstack-kolla13:54
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593814:00
*** diogogmt has quit IRC14:00
*** salv-orlando has joined #openstack-kolla14:01
*** zhurong has quit IRC14:04
*** zhurong has joined #openstack-kolla14:05
*** wmiller has quit IRC14:07
*** zhurong has quit IRC14:07
*** wmiller has joined #openstack-kolla14:07
bjoloquestion about pluggability and customization. From what i understand, that is the "big focus" for kolla 3. is there any docs, BP, design principles i can read up on?14:12
mewaldpush14:12
bjolo?14:13
rhalliseybjolo, https://etherpad.openstack.org/p/kolla-newton-midcycle14:17
*** mgoddard has joined #openstack-kolla14:17
*** mgoddard_ has quit IRC14:18
*** dmueller-evoila has quit IRC14:21
bjolorhallisey, tnx exactly what i was looking for14:22
inc0bjolo, I don't think we have it gathered in any single place14:24
inc0but I would be person to ask, so just ask;)14:25
bjolorhallisey sent me a link to etherpad from latest midcycle14:25
bjoloshould get me started14:25
inc0yeah, but that14:25
*** diogogmt has joined #openstack-kolla14:26
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593814:27
openstackgerritMathias proposed openstack/kolla: Implement Basic Performance Monitoring  https://review.openstack.org/34593814:29
*** ravig has joined #openstack-kolla14:29
mewaldCan I have some feedback on https://review.openstack.org/#/c/344705/ please? :)14:30
patchbotmewald: patch 344705 - kolla - External Ceph - Implementation Nova14:30
*** ravig has quit IRC14:31
*** ravig has joined #openstack-kolla14:31
*** haplo37_ has joined #openstack-kolla14:33
bjoloi give up on this swift issue14:35
*** michauds has joined #openstack-kolla14:43
*** b_bezak_ has quit IRC14:46
*** jruano has joined #openstack-kolla14:53
*** neouf has quit IRC14:53
bjolook folks14:54
bjoloim outtee14:54
bjoloyihah14:54
bjolovacation14:54
mewaldc ya !14:55
bjolomight drop in once in a while, but i tend to unplugg completely when im off14:55
bjoloand you know read a book, talk to humans, etc etc :)14:55
bjolohave a great one14:56
bjolociao for now14:56
bjolo~14:56
*** bjolo is now known as bjolo-has-left-t14:56
mewalddont like that, I stay here :D14:56
bjolo-has-left-thaha14:57
*** mliima has quit IRC14:57
*** inc0 has quit IRC14:59
*** inc0 has joined #openstack-kolla14:59
*** bjolo-has-left-t has quit IRC15:02
*** zeus has joined #openstack-kolla15:11
*** zeus is now known as Guest5826015:11
*** mewald has quit IRC15:12
*** jogam has quit IRC15:12
*** Guest58260 is now known as zeus`15:13
*** zeus` is now known as zeus15:14
*** zeus has quit IRC15:14
*** zeus has joined #openstack-kolla15:14
*** mliima has joined #openstack-kolla15:17
*** daneyon has joined #openstack-kolla15:25
*** vhosakot has joined #openstack-kolla15:35
*** mgoddard has quit IRC15:37
*** inc0 has quit IRC15:38
*** inc0 has joined #openstack-kolla15:38
*** ravig has quit IRC15:39
*** godleon has quit IRC15:39
*** flaper87 has quit IRC15:39
*** brad[] has quit IRC15:39
*** wznoinsk_ has quit IRC15:39
*** PyroMani has quit IRC15:39
*** dasm has quit IRC15:39
*** mark-casey has joined #openstack-kolla15:39
*** david-lyle has joined #openstack-kolla15:46
*** flaper87 has joined #openstack-kolla15:47
*** wznoinsk has joined #openstack-kolla15:47
*** brad[] has joined #openstack-kolla15:47
*** dasm has joined #openstack-kolla15:47
*** ravig has joined #openstack-kolla15:47
*** flaper87 is now known as Guest3290615:47
*** PyroMani has joined #openstack-kolla15:48
*** mgoddard has joined #openstack-kolla15:49
*** godleon has joined #openstack-kolla15:52
*** unicell1 has quit IRC16:07
*** wirehead_ has quit IRC16:11
*** wirehead_ has joined #openstack-kolla16:13
*** JianqingJiang has joined #openstack-kolla16:16
*** david-lyle_ has joined #openstack-kolla16:17
v1k0d3nMech422: yeah...been having so many issues with kolla and i've tried a few different package builds :(16:21
v1k0d3nkind of frustrated...i can't seem to get things working. docs are all updated too? does working off of master work correctly at this time?16:22
v1k0d3nor do i need to work off stable/mitaka?16:23
*** david-lyle_ has quit IRC16:23
v1k0d3ni pulled back even to a single AIO node, followed the exact same instructions that i used before...and building my images still don't work. i noticed that kolla isn't really pinning the base container, but it's pulling ubuntu:latest....is that right? could that be causing some issues?16:24
v1k0d3ni even got the build to deploy all the way, but horizon never came up because it couldn't find "developer" something-or-other...i'm getting really confused because of multiple issues at this point.16:24
v1k0d3njust want known stable deployment, multi-node for a demo that i have coming up on aug 3rd. starting to run out of time.16:25
*** daneyon_ has joined #openstack-kolla16:27
v1k0d3nbrb...i need to leave chan for a few minutes to sign in on a different dev box.16:28
*** v1k0d3n has quit IRC16:30
*** daneyon_ has quit IRC16:32
*** v1k0d3n has joined #openstack-kolla16:35
*** severion has joined #openstack-kolla16:35
*** ayoung has joined #openstack-kolla16:36
*** godleon has quit IRC16:39
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Fixing labels for glance's service and pod resources  https://review.openstack.org/34612316:42
*** ravig has quit IRC16:43
*** ravig has joined #openstack-kolla16:44
*** ravig has quit IRC16:46
*** ravig has joined #openstack-kolla16:46
*** jruano has quit IRC16:48
*** ravig has quit IRC16:50
*** mewald has joined #openstack-kolla16:51
*** gfidente has quit IRC16:55
*** jruano has joined #openstack-kolla16:59
*** jruano has quit IRC16:59
*** inc0 has quit IRC17:09
*** inc0 has joined #openstack-kolla17:10
*** salv-orlando has quit IRC17:23
*** sdake has joined #openstack-kolla17:26
*** inc0_ has joined #openstack-kolla17:36
*** inc0__ has joined #openstack-kolla17:38
*** inc0 has quit IRC17:40
*** ravig has joined #openstack-kolla17:40
*** harlowja_at_home has joined #openstack-kolla17:41
*** inc0_ has quit IRC17:42
openstackgerritMerged openstack/kolla-kubernetes: Fixing labels for glance's service and pod resources  https://review.openstack.org/34612317:48
*** JianqingJiang has quit IRC17:54
*** Jeffrey4l_ has quit IRC17:57
*** haplo37_ has quit IRC18:00
Mech422v1k0d3n: ubuntu source builds are broken atm18:12
Mech422v1k0d3n: I have a patch you can apply to fix it, and I'll submit it upstream18:12
Mech422v1k0d3n: the patch is: http://paste.openstack.org/show/539735/18:12
*** daneyon_ has joined #openstack-kolla18:15
sdakehey folks18:16
sdakenice work mech!18:17
sdakeMech422 if you have trouble submitting that patch or need a walkthroug the workflow, i am here to help :)18:18
sdakeMech422 or would you like me to submit it?18:18
*** daneyon_ has quit IRC18:20
*** david-lyle_ has joined #openstack-kolla18:21
Mech422sdake: eh - I need to learn this gerrit stuff18:23
Mech422sdake: I have another trivialfix for the ceph partition stuff18:24
*** mgoddard has quit IRC18:25
Mech422sdake: http://paste.openstack.org/show/539832/18:25
v1k0d3nah! i was wondering.18:25
*** david-lyle_ has quit IRC18:25
*** bootsha has joined #openstack-kolla18:26
v1k0d3nreally just need to know what is actually working and not, as in versions and which to build.18:26
v1k0d3nwant to get to a place where i can get it up, then peel back and get more involved. i've been tied up on so many other things for a few months. sdake knows this.18:26
v1k0d3nhence my long absence.18:26
Mech422vikI build ubuntu master from source with ansible 2.1.0.0 multi-node with ceph enabled using fresh installs of Ubuntu 14.04 for all hosts18:27
v1k0d3nbut, my role is changing soon...soon will be 100% community focused with a dev team to help out.18:27
Mech422v1k0d3n: :  I built ubuntu master from source with ansible 2.1.0.0 multi-node with ceph enabled using fresh installs of Ubuntu 14.04 for all hosts last night with these patches18:28
v1k0d3ndoes the kernel still need to be upgraded to wily?18:29
Mech422v1k0d3n: I don't think it _needs_ to, but I upgrade18:30
sdakeMech422 the reason I asked about your os is because I typically use a tool called webex to train people18:30
sdakeMech422 it o nly runs on windows and macosx18:30
Mech422sdake: oh - I can skype...18:30
sdakeso if you can get to your linux machines frmo your windows machine, then we are all set18:30
sdakei need screen sharing18:31
Mech422yeah - thats not a problem18:31
sdakeso i can see what yor typing18:31
v1k0d3nMech422: sdake was awesome at helping me...awesome dude, and super helpful!18:31
Mech422sdake: I already installed the gerrit git plugin18:31
sdakeare you opposed to webex or have some it policy against it?18:31
sdakethanks v1k0d3n  :)18:31
v1k0d3nyou guys want to try AT&T connect? i can let you guys use my bridge if you don't mind me tagging along....18:31
sdakev1k0d3n now to finish the job on yoru patch :(18:31
sdakev1k0d3n you can tag along anyway but i like webex :)18:32
v1k0d3nsdake: still out there?18:32
sdakebut not doing it now, have to run to the post office18:32
sdakeMech422 i'll be back in 30 minutes18:32
Mech422sdake: nah - not against it...I just mean I have that available18:32
Mech422oh - anyone can review code ?  I thought only core devs could +118:34
v1k0d3nMech422: anyone can review18:37
v1k0d3nand +1 i think, but only devs can accept/merge.18:37
*** athomas has quit IRC18:38
v1k0d3nso Mech422 i have a cluster now...fresh install...i would like multi-node, can you help me get through this?18:38
Mech422v1k0d3n: possibly ?18:39
Mech422v1k0d3n: I've only done ubuntu source builds - haven't touched RH boxes in 10 years18:39
v1k0d3ni'm doing all ubuntu18:40
v1k0d3nso that works perfectly from a baseline perspective. :)18:40
Mech422v1k0d3n: lets give it a shot then :-)18:40
v1k0d3ndid the current docs work 100% for you other than the patch?18:40
Mech422v1k0d3n: I haven't looked at the install docs in like 3 months...18:42
Mech422v1k0d3n: I have a set of ansible playbooks that 'drive' kolla for me18:42
*** unicell has joined #openstack-kolla18:42
Mech422v1k0d3n: hmm - actually, I could probaby just upload the playbooks somewhere for you...18:43
Mech422v1k0d3n: then you could see exactly how I build18:43
v1k0d3nMech422 !! that's awesome...that's exactly what i was trying to do myself...!!18:43
Mech422v1k0d3n: is there an imgur like sharing service for raw files?  be a nuisance to paste them all into pastes18:44
v1k0d3ni have a private repo setup at the moment because i have some public key stuff in there...(not that big a deal), but it's not 100% complete. if you have something, i would definitely contribute to that effort...big tim.18:44
v1k0d3njust git them up to github?18:44
Mech422v1k0d3n: oh - I could probably just tar them and dcc them to you18:45
Mech422v1k0d3n: nah - I have keys and such I need to clean up before I can make them public18:45
v1k0d3nhaha, pub or private?18:45
v1k0d3nhere...let's do this...18:47
*** sdake has quit IRC18:47
v1k0d3ni can create a github repo, add you as a member...then you can copy the repo and only add the files you want with git add commands. that work Mech422?18:48
v1k0d3nwhat is your github id?18:48
v1k0d3ni will make it a private repo18:48
Mech422v1k0d3n: I can upload a tar file for ya - I just tared it all18:48
v1k0d3noh that works too.18:48
Mech422v1k0d3n: DCC send request sent18:49
Mech42218.1 KB file18:49
*** salv-orlando has joined #openstack-kolla18:49
*** salv-orl_ has joined #openstack-kolla18:50
*** salv-orlando has quit IRC18:51
*** ravig has quit IRC18:52
*** salv-orlando has joined #openstack-kolla18:54
*** salv-orl_ has quit IRC18:55
*** Mech422 has quit IRC18:55
*** ravig has joined #openstack-kolla18:55
*** Mech422 has joined #openstack-kolla18:55
Mech422blah18:56
Mech422v1k0d3n: holler if you have any questions with that stuff18:56
*** bootsha has quit IRC18:56
Mech422v1k0d3n: the playbook order is: prereqs, checkout, build, configure, deploy18:56
*** mdnadeem_ has joined #openstack-kolla18:56
*** ravig has quit IRC18:57
v1k0d3nok, awesome. thanks Mech422 let me start playing around with this. are mech22 on github as well?18:58
*** ravig has joined #openstack-kolla18:58
*** mdnadeem has quit IRC19:00
Mech422yeah19:00
Mech422I'm pretty much mech422 everywhere...19:00
Mech422for some reason, that nick is always available :-P19:01
Mech422v1k0d3n: btw, I'm in an out of the livingroom watching the cnn coverage of berlin shootings...19:01
Mech422v1k0d3n: so I might be a lil slow to respond19:01
v1k0d3n:( yeah...crazy times. i understand man.19:02
v1k0d3nthanks for the help. i'll ping if i start running into issues.19:02
openstackgerritSteve Hindle proposed openstack/kolla: Ensure apt-transport-https is available.  https://review.openstack.org/34617819:10
*** diogogmt has quit IRC19:13
*** diogogmt has joined #openstack-kolla19:14
*** v1k0d3n has quit IRC19:20
*** mewald has quit IRC19:20
*** banix has joined #openstack-kolla19:20
*** ravig has quit IRC19:21
*** ravig has joined #openstack-kolla19:21
*** rbergeron has quit IRC19:22
*** rbergeron has joined #openstack-kolla19:22
*** huikang has joined #openstack-kolla19:23
*** diogogmt has quit IRC19:24
*** v1k0d3n has joined #openstack-kolla19:24
v1k0d3nMech422: hey man...looks like there may be a global_vars file?19:25
v1k0d3ncluster_action and some others...19:25
v1k0d3ngoing to dig in and see what it's looking fro.19:25
v1k0d3nfor19:25
*** dave-mccowan has quit IRC19:25
Mech422v1k0d3n: hmm - don't recall global_vars - just globals.yml19:26
v1k0d3nah, yeah...do you have what's being used in globals?19:26
Mech422v1k0d3n: oh - I should prolly paste my group_vars/kolla19:26
*** diogogmt has joined #openstack-kolla19:27
Mech422v1k0d3n: here's group_vars/kolla.yml: http://paste.openstack.org/show/539837/19:28
*** salv-orlando has quit IRC19:28
*** tyrola has quit IRC19:31
*** ravig has quit IRC19:31
*** ravig has joined #openstack-kolla19:32
v1k0d3nhmmm Mech422 can't find cluster_action definition19:32
openstackgerritSteve Hindle proposed openstack/kolla: Use $PATH to find sgdisk. Update udev after partition changes.  https://review.openstack.org/34621419:33
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: WIP Glance bootstrap to create service/project/role  https://review.openstack.org/34621519:33
Mech422v1k0d3n: that gets passed in from the driver script19:33
Mech422v1k0d3n: its just 'prereqs', 'checkout', 'build', etc19:33
Mech422v1k0d3n: the cluster actions match the names of the yaml files in .../tasks19:34
Mech422v1k0d3n: you'll probably want to create your own playbooks, so I'd just look at what gets done in each playbook19:34
Mech422v1k0d3n: also, I run these from a seperate host, that builds the kolla depoloyment host...19:35
v1k0d3nyeah i do that as well...19:35
*** ravig has quit IRC19:35
Mech422v1k0d3n: ahh...ok19:36
v1k0d3njust a bit different.19:36
Mech422v1k0d3n: LOL - I can imagine :-)19:36
v1k0d3nit's an undefined variable...so it's not being declared somewhere...19:36
*** ravig has joined #openstack-kolla19:36
Mech422v1k0d3n: cluster_action is passed in via the driver script as an 'extra var'19:36
v1k0d3ni included ansible_os_family: debian, because i saw that's required somewhere.19:36
v1k0d3ni dont see a driver script...19:37
Mech422v1k0d3n: ansible sets that auto-magicaly19:37
Mech422v1k0d3n: I didn't send the driver - just the role19:37
*** banix has quit IRC19:37
v1k0d3noh!19:37
v1k0d3nok...was thinking i could run against the role19:37
Mech422nah - even with the driver, it'll probably blow up on you...19:37
Mech422I just sent it so you could see what steps I perform to build multi-node19:38
DavieyHey, cany chance i can get https://review.openstack.org/#/c/337556/ merged... i really don't want to rebase it for a 4th time. Ta :)19:38
patchbotDaviey: patch 337556 - kolla - Add Watcher ansible roles and templates19:38
Mech422v1k0d3n: if you really want it, here's the driver: http://paste.openstack.org/show/539839/19:39
Mech422v1k0d3n: and heres the (REALLY crufty) inventory file: http://paste.openstack.org/show/539841/19:41
Mech422v1k0d3n: the inventory stuff for zookeeper and cassandra aren't needed...prolly other cruft in there too19:41
*** mliima has quit IRC19:44
*** vhosakot has quit IRC19:53
*** sdake has joined #openstack-kolla19:53
sdakehello folks back20:01
sdakeMech422 if you want to run through the workflow now would be a good time20:01
sdakeMech422 your set of ansible playboooks that drive kolla for you - would you mind expanding on what they do?20:02
Mech422sdake: Nothing big/good :-P20:02
sdakestill curious pls ;)20:03
sdakeunless its secret sauce20:03
Mech422sdake: its embarassingly crap20:03
sdakeok well yo udont have to show just tell what it doe ss :)20:04
Mech422sdake: but I can email you the tarball of the role if ya like20:04
*** daneyon_ has joined #openstack-kolla20:04
sdakeif you could djust explain it real quick20:04
sdakecurious if its something we need in kolla proper20:04
sdake(new ideas = good :)20:04
Mech422sdake: oh - the 'kolla' part just 'drives' kolla checkout, build, configure, and deploy20:04
*** huikang has quit IRC20:04
sdakei see - we expect that to be handled by a ci/cd pipeline20:05
Mech422it installs the prereqs, applies patches, generates the multi-node inventory file, etc20:05
sdakesounds pretty cool ;)20:05
sdakewhich patches do you apply20:05
*** huikang has joined #openstack-kolla20:05
sdakesomething specific for your environment?20:05
sdakei think generation of inventory files will have to wait until sean gets further down teh road with bifrost20:06
Mech422sdake: yeah - very site specific20:06
Mech422sdake: its really just a way for me to keep track of all the steps in running kolla and make sure I don't forget anything20:06
sdakeroger that20:07
Mech422sdake: I noticed last night though, I have to use 'kolla/.tox/bin/kolla-build' rather then the path listed in the docs20:07
sdakedocs going throug hchanges - not even sure what they say atm20:07
sdakeshould be tools/build.py20:07
sdakeor /usr/local/bin/kolla-build20:08
sdakeor /usr/bin/kolla-build20:08
Mech422sdake: yeah - I have to use one from the .tox dir for some reason20:08
*** daneyon_ has quit IRC20:08
*** huikang has quit IRC20:09
*** huikang has joined #openstack-kolla20:09
*** salv-orlando has joined #openstack-kolla20:09
sdakeMech422 no idea dude20:10
sdakeits not supposed to work like that20:10
sdakeif you cd kolla20:10
sdakepip install .20:10
sdakeit should install in system directories20:10
Mech422sdake: I'll have to review my playbooks - maybe I missed a step20:12
sdakerhallisey coolsvap or inc0__ can one ofy ou look at https://review.openstack.org/#/c/346178/120:15
patchbotsdake: patch 346178 - kolla - Ensure apt-transport-https is available.20:15
sdakeubuntu is broken apparently without it20:15
Mech422yeah - thats the one that gets the build working again20:15
rhalliseyreally?20:15
rhalliseynice20:16
Mech422rhallisey: its only a 1 liner :-P20:17
*** mewald has joined #openstack-kolla20:21
sdakeoh no shit mech?20:22
sdakenice job dude!!20:22
Mech422sdake: thanks20:22
*** david-lyle_ has joined #openstack-kolla20:22
sdakerhallisey well lets wait for gate then20:23
Mech422sdake: the 'hard' part was waiting for container rebuilds :-P20:23
sdakeso we can declare victory :)20:23
sdakeya we need someone to make a mirroring tool that ubilds faster20:23
sdakefor folks without fast internet an ddisks20:23
sdakei mean the mirroring tools exist20:23
sdakewhat we need is a ansible playbook to run them properly20:24
Mech422btw - did you see mewald dropped a monitoring patch last night...20:24
*** ccesario has quit IRC20:24
Mech422so we get live system metrics in grafana :-)20:24
sdakeno - watched RNC last night20:24
sdakecool I dont know what grafana is20:24
Mech422also...20:24
Mech422<Daviey> Hey, cany chance i can get https://review.openstack.org/#/c/337556/ merged... i really don't want to rebase it for a 4th time. Ta :)20:24
patchbotMech422: patch 337556 - kolla - Add Watcher ansible roles and templates20:24
sdakeso could folkks tell me how grafana and collectd relate, if at all?20:26
*** david-lyle_ has quit IRC20:28
Mech422grafan is the 'dashboard'/graphing component AFAIK20:28
Mech422lots of pretty charts, and real-time display of metrics such as cpu usage20:28
Mech422I guess grafana is to metrics as kibana is to logs ?20:29
sdakeright - what produces the metrics for grafana?20:29
sdakewhat is watcher20:30
Mech422dunno watcher20:31
Mech422grafana can get metrics from various TSDBs that are fed via collectd, diamond, telegraf, etc.20:31
*** huikang has quit IRC20:32
*** huikang has joined #openstack-kolla20:33
*** huikang has quit IRC20:37
sdakecollectd is the one i've heard of20:39
sdakenot that means anything20:39
sdakebut it seems pretty popular20:39
sdakehow did mewald handle metric collection?20:39
*** dave-mccowan has joined #openstack-kolla20:41
*** huikang has joined #openstack-kolla20:41
rhalliseysdake, watches is an additional metric on top of ceilometer20:41
rhalliseyI believe20:41
sdakecool so big tent?20:42
rhalliseythink so20:42
rhalliseythe patch looks good20:42
sdakelet me check gov repo20:42
sdakewe dont have an anti-bigtent requirement actually20:43
sdakeya i guess we can take non-big tent projects on a case by case basis20:43
rhalliseywell this one is all set20:45
rhalliseyso why not20:45
sdakesure20:47
*** huikang has quit IRC20:50
openstackgerritMerged openstack/kolla: Ensure apt-transport-https is available.  https://review.openstack.org/34617820:52
sdakerhallisey i htink this is good for merge20:52
sdakehttps://review.openstack.org/#/c/329651/420:52
patchbotsdake: patch 329651 - kolla - Customizations for heat20:52
*** britthouser has quit IRC20:57
rahulssdake: I have few queries21:03
sdakerahuls shooot21:03
rahulscurrent builds of stable/mitaka gets a tag of 2.0.321:03
sdakeyes we need to fix that in stable/mitaka21:04
rahulsI opened this bug: https://review.openstack.org/#/c/345166/21:04
patchbotrahuls: patch 345166 - kolla (stable/mitaka) - updates default registry tag to latest version21:04
rahulsbut this seems to be wrong way21:04
rahulsJeffrey pointed out that there needs to be something else21:04
sdakereading review21:04
sdakemoment21:04
Mech422sdake: oh! are we going to update the ceph version for newton? I notice we're still on hammer...21:06
rahulswhen I check the python code, pbr points to version 2.0.321:06
*** Serlex has quit IRC21:06
rahuls>>> import pbr.version21:06
rahuls>>> print pbr.version.VersionInfo('kolla')21:06
rahuls2.0.321:06
rahuls>>>21:06
sdakerahuls moment sidetracked - reading review :)21:06
openstackgerritDave Walker proposed openstack/kolla: Add initial Sensu Docker support  https://review.openstack.org/34186121:07
*** kbyrne has quit IRC21:07
mewaldWhat's wrong with my while loop? Have this complaint: http://logs.openstack.org/38/345938/8/check/gate-kolla-pep8/3db7e68/console.html21:08
*** ntpttr___ is now known as ntpttr21:08
mewald[E] E010: The "do" should be on same line as while: 'while true; do sleep 1000; done'21:08
mewaldwell, the "do" _is_ on the same line ...21:09
*** kbyrne has joined #openstack-kolla21:11
sdakerhallisey i consider this an emergency fix21:12
sdakecan you ack even though ejffrey is in disagreement21:12
sdakehttps://review.openstack.org/#/c/34516621:12
patchbotsdake: patch 345166 - kolla (stable/mitaka) - updates default registry tag to latest version21:12
rahulssdake: git tag contains only uptil 2.0.2 and then we have 3.0.0 and others21:13
rahulsI found one other way21:13
*** ravig has quit IRC21:13
sdakerahuls I am intimately familiar with how the versioning works :)21:13
sdakeand tagging21:13
sdakeand pbr21:13
rahulsyeah... same here21:13
sdakejust not sure how to solve the problem to jeffrey's liking as in today21:13
rahulsI added a new tag 2.0.2.0rc121:13
rahulsand then did pip install --upgrade kolla/21:14
rahulsand now it has proper 2.0.2 tag21:14
sdakeyes but we want it to work from our pushed registry images21:14
rahuls:X21:14
rahulsafter the new 2.0.2.0rc1 git tag21:14
rahuls>>> import pbr.version21:14
rahuls>>> print pbr.version.VersionInfo('kolla')21:14
sdake2.0.2 is not the proper tag if you do a git checkout21:14
rahuls2.0.221:14
*** daneyon has quit IRC21:14
sdakeyup i get it21:14
sdakeits 2.0.2 + some delta21:14
rahulsyeah21:14
sdakeso 2.0.2 is not correct21:14
sdake2.0.2 should only be used in the tarball/pip uploaded to pypi21:15
rahulseither we can add a new tag, or change the documentation to 2.0.321:15
sdakei can't add tags21:15
rahulsohk, I had no idea, so I wanted to first check with you21:15
sdakethat permissio nwas removed from all ptls about 3 mo ago21:15
*** michauds has quit IRC21:15
sdakewhich is good, tagging makes me nervous :)21:15
rahuls:)21:16
*** bootsha has joined #openstack-kolla21:16
Mech422sdake: oh - now we got all these new dockerfiles coming in - were there any changes needed for inc0's customization stuff21:17
Mech422?21:17
sdakewe will sort out the new dockerfiles later21:17
sdakewe have a list21:17
*** jtriley has quit IRC21:18
*** ravig has joined #openstack-kolla21:18
Davieysdake: I hoped that branch using pbr was going to respect the devX suffix for kolla build / deploy if using latest21:20
Davieypbr does expose this.. I started working on a change that did somethin similar, before i noticed someone else did21:23
DavieyI was getting too much version clash when constantly building off Master, and not getting consistent deployments21:24
*** bootsha has quit IRC21:24
*** dave-mccowan has quit IRC21:26
Mech422sdake: ping?21:29
sdakeMech422 shoot21:30
Mech422just wanted to get you that code21:31
sdakethanks21:33
sdakeDaviey the problem with using dev001 or whtaever is that each git change requires a change to the globals.yml21:37
sdakefor developers thats fine21:37
sdakefor non-develoeprs sounds like a couple trainwrecks head on21:37
sdakewe really want non-developers to either install from pip or distro packaging21:38
*** ayoung has quit IRC21:39
*** zhurong has joined #openstack-kolla21:41
*** zhurong has quit IRC21:44
*** mark-casey has quit IRC21:47
*** ravig has quit IRC21:49
*** shardy has quit IRC21:50
*** ravig has joined #openstack-kolla21:51
*** daneyon has joined #openstack-kolla21:52
Davieysdake: No.. What i was thinking is that globals.yml uses openstack_release='latest'.. and then on each run, that var is checked.  If latest, then pbr version is used.. If a hard coded version, then that is used21:52
*** mbound has quit IRC21:53
Davieysdake: I would imagine many local deployments would choose to hard-code to their own version, to have better control of image build and release.. but people that don't want that level of care, can leave it as default latest.21:54
Daviey(and it doesn't need to be updated on each tagged release)21:54
Daviey+ updating configs for distros is a total PITA.21:55
sdakeDaviey sounds reasonable at first glance - strugglign to see how you capture the pbr version inside the ansible codebase21:56
sdakeDaviey would you mind starting a mailing list thread - clearly what we are doing now is suboptimal21:57
*** daneyon has quit IRC21:57
sdakebut i'd like wider audience on the conversation if ipossible21:57
Davieysdake: let me find my branch..21:58
Davieysdake: regarding the split.. I did just this for watcher.. 3 different commits... Docker, Ansible and then config .. https://blueprints.launchpad.net/kolla/+spec/watcher21:59
Davieyso it makes sense to me21:59
Daviey(but i'm not sure i agree that trival commits makes sense to split, until after the trees split)21:59
sdakecool so watcher21:59
sdakecan you tell me about it22:00
Davieysdake: I'm not involved with the upstream project.. but it basically uses ceilometer data to see if compute hosts are not balanced.. either due to number of vm's.. or noisy vm's co-located22:00
Davieyand suggests rebalancing via live migration22:00
DavieyEither as a daemon or on demand22:01
sdakeis it an openstack project?22:01
Davieyyeah22:01
sdakebig tent?22:01
Davieyhttps://github.com/openstack/watcher22:01
Davieyyeah, i checked that first.22:01
sdakewe dont have a big tent requirement22:01
sdakei would like to integrate vmtp at some point22:01
sdakewhich is a pretty sweet network testing tool22:01
sdakeas part of our sanity testing22:02
Davieyfirst time i've looked at that... looks interesting22:02
Davieypretty graph aswell22:02
Davieysdake: Sorry to jump on you.. but could you also have a look at https://blueprints.launchpad.net/kolla/+spec/sensu22:03
DavieyI'm currently blocked on getting that accepted before i push forward with it22:03
sdakeDaviey so we decided at midcycle that monitoring was off the table for newton22:04
sdakethat said, if you think you can handle it on your own, more power to ya :)22:04
DavieyOh right.. thanks for sharing :/22:04
sdakeand if it merges, it merges22:04
sdakewhen we decided that22:04
Davieywell.. i didn't know it was off the table :)22:04
sdakewe lacked context that someone else other the nthe cr team was going to do the work22:05
sdakewhat i mean by off the table22:05
sdakewrong term22:05
sdakewe dont have capacity to deal with it in the cr team as an implementation item22:05
sdakereview item, sure22:05
DavieyI pushed up a sensu docker change, but it was -2 - pending discussion... So i've pretty much shelved the work on it22:05
sdakeimplementing it - not so much22:05
sdakelink to -2 change22:05
Davieyhttps://review.openstack.org/#/c/341861/22:05
patchbotDaviey: patch 341861 - kolla - Add initial Sensu Docker support22:05
DavieyI've basically got it working locally.. but I probably need to spend a day cleaning it up for submission22:06
DavieyOnly with host up / down monitoring at current tho.. but that might be enough for release, with the ability to add custom json plugins by the deployer.. Newton + 1 could probably do per-service / container checking22:07
openstackgerritIldar proposed openstack/kolla: Customizations for kibana  https://review.openstack.org/34625222:10
*** ravig has quit IRC22:14
*** ravig has joined #openstack-kolla22:15
*** harlowja_at_home has quit IRC22:15
sdakeDaviey your wish is my command ;)22:15
Davieysdake: I'm ok to commit to getting just host up/down monitoring done in the next 2 weeks.22:17
sdakewell whatevery ou get done i'd be open to merging22:17
sdakeour deadline is sept222:17
DavieyOh, that is ages.22:18
sdakeafter that, no new features22:18
sdakebut where we are at moment is you are blocked waiting on a discussion of a monitoring solution choice22:18
sdakeso i kicked that off on the mailing list22:18
sdakefolks, if your ever blocked by something please either let the mailing list know or let me know or let anothe rcore reviewer know22:19
sdakei have trouble keeping on top of the entire review queue22:19
sdakealong with all the other stuff i got going on22:19
sdakedefinately dont give up ;)22:19
Davieysdake: well, there isn't exactly the level of dialogue on the list that helps.. :/22:20
sdakeits my job to break logjams22:20
Davieysdake: This thread didn't get anywhere, http://lists.openstack.org/pipermail/openstack-dev/2016-July/098707.html22:20
sdakeya dude sorry i was probably on pto ;)22:20
Davieysdake: Not blaming you. :)22:21
sdakeoh that discussion22:21
*** mewald has quit IRC22:21
sdakeso we are going to handle that with customizations22:21
sdake(I think)22:21
Davieysdake: but that thread is probably blocking https://review.openstack.org/#/c/336943/ , and other stuff now22:21
patchbotDaviey: patch 336943 - kolla - Install neutron-lbaas-dashboard from source in hor...22:21
sdakewe want to do all plugins in customizations of kolla j2 files22:22
DavieyI also have a conditional horizon plugin i'd like to add22:22
sdakei think we can get you a condtional horizon plugin or a lbaas horizon plugin without much effort once the 40 containers customization work is finished22:23
Davieysdake: So.. does that mean that not all docker images will contain the world?22:23
Davieyas in, conditional if they get included?22:23
sdakecomplex quesiton22:23
sdakecould you simplif yfor me please22:23
DavieyI'm sorry if this was all at the midcycle.. i couldn't really follow22:23
sdakeya following a midcycle disucsision in person is hard :)22:24
Davieysdake: Sorry, i mean... for p in $(possible horizon plugins) ; install p ; done VS for p in $(globals.yml enabled horizon plugins) ; install p ; done22:24
*** david-lyle_ has joined #openstack-kolla22:25
sdakeyou will be able to select which plugisn yo uwant22:25
sdakeit doesn't key off globals.yml22:25
Davieywell.. but i mean, adding a plugin will need rebuilding of the images?22:26
sdakeit keys off a customization file which tells which files to load into the container that are extra22:26
sdakeyes rebuild required22:26
openstackgerritMerged openstack/kolla: Customizations for heat  https://review.openstack.org/32965122:26
Davieyhow does that work with versioning?22:26
sdakeversioining is totally up to the operator22:26
Daviey2.0.0 without plugin == 2.0.0 with plugin22:26
sdakei described how to handle this in the docs - moment22:27
DavieyOh, i should have read it.. sorry22:27
sdakehttp://docs.openstack.org/developer/kolla/operating-kolla.html22:27
sdakeits ok - it will leave you wanting22:27
Davieyno, i got that..22:27
sdakebut its the best i had at the time - hadn't considered variance like plugins22:27
Davieybut what i mean is.. at the moment.. we have deterministic versioning.. if i install kolla version FOO.. and build, i get known docker images, independent of how many times i change the config and rebuild22:28
sdakeyup22:28
sdakeand with this new method you dont22:28
sdake(always)22:28
Davieywith non-deterministic builds based on configs, i get a messy difference22:28
Davieywhere version 3.0.0 could be anything22:29
Davieyand updating the config, rebuilding.. i can't be sure what is really running on the nodes22:29
sdakebuild world might be better for plugins22:29
DavieyUnless i use lcoal tags22:29
openstackgerritIldar proposed openstack/kolla: Customizations for kibana  https://review.openstack.org/34625222:29
sdakebut if we do build world with plugins, i would prefer it to tie into the customizations22:29
*** david-lyle_ has quit IRC22:29
*** ravig has quit IRC22:31
*** neophy has quit IRC22:32
Davieysdake: agree22:32
*** ravig has joined #openstack-kolla22:35
openstackgerritqinchunhua proposed openstack/kolla: Fix dict.values() PY3 compatible  https://review.openstack.org/34570622:47
Mech422sdake, Daviey: there is also mewalds monitoring stuff that was sent for review last night... it appears we now have 2 monitoring options for Kolla :-)22:47
Mech422Daviey: Does sensu play nice with Grafana ?22:48
sdakeya grafana22:48
sdakewell it would be an either or thing22:48
sdakenot a both together thing i htink22:48
Mech422sdake: yeah - I'd imagine you'd pick one via globals ?22:48
sdakei dont like picking winners22:49
sdakethe only winner we have picked thus far is docker22:49
sdakeand we haven't even really picked that22:49
Mech422I agree - its nice to have options and let the operator pick22:49
sdakeMech422 wfm :)22:49
sdakenice to let the operators choose22:49
Mech422hmm - grafana can be used by a lot of stuff (sensu maybe?) - might make sense to break that out to its own 'component' that can be used by various monitoring solutions22:53
sdakewell here is the deal22:53
sdakei dont know anything at all about monitoring22:53
*** mbound has joined #openstack-kolla22:53
sdakebut mewald and Daviey appear to - so lets let them sort it out :)22:53
Mech422sounds good22:54
*** mbound has quit IRC22:58
sdake10:45 pm showing of star trek ftw22:59
*** inc0__ has quit IRC22:59
* sdake can't wait22:59
sdakehate to admit it, but i'm more of a star trek fan then star wars fan :)23:00
jascott1resisting urge to engage...23:03
jascott1:)23:03
jascott1heheh "engage"23:05
sdakeMech422 fwiw your patch did not fix ubuntu from source here https://review.openstack.org/#/c/346178/23:06
patchbotsdake: patch 346178 - kolla - Ensure apt-transport-https is available. (MERGED)23:06
sdakeit may have fixed a different problem23:06
sdakebut not the one we have at present23:06
sdakeor the one that is blocking the gate23:06
Mech422sdake: odd - it built here this morning23:13
sdakeit seems to build out  of the gate23:13
sdakebut not in the gate23:13
sdakemay be a different problem like i said23:14
Mech422sdake: looking at the console log, it looks like a nova issue ?23:15
sdakeno its a rabbitmq crash i think23:16
Mech422ahh23:16
Mech422looks like it choked when it tried to start a vm ?23:22
Mech422"fault": {"message": "No valid host was found. There are not enough hosts available.", "code": 500, "details": "  File \"/var/lib/kolla/venv/local/lib/python2.7/site-packages/nova/conductor/manager.py\", line 471, in build_instances\n23:23
*** harlowja_at_home has joined #openstack-kolla23:23
*** ravig has quit IRC23:23
*** ravig has joined #openstack-kolla23:24
*** ravig has quit IRC23:27
*** ravig has joined #openstack-kolla23:30
openstackgerritChristian Berendt proposed openstack/kolla: Allow custom configurations for ceilometer  https://review.openstack.org/34627323:30
*** daneyon has joined #openstack-kolla23:30
sdakeMech422 yes because rabbitmq crashed23:37
*** daneyon_ has joined #openstack-kolla23:41
Mech422sdake: hopefully, the next run will be ok :-)23:42
Mech422sdake: thinking about 'pluggability' and operator options...it would be kind of cool if kolla had a suite of components designed to work together, and operators could do a 'docker compose' type thing to figure out which ones to deploy ?23:43
Mech422sdake: just a thought...but it might get us out of the 'MxN' support matrix as our options for plugins grow23:43
*** daneyon_ has quit IRC23:45
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: WIP Glance bootstrap to create service/project/role  https://review.openstack.org/34621523:46
*** daneyon_ has joined #openstack-kolla23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!