Friday, 2017-08-11

*** thingee_ has quit IRC00:00
*** awiddersheim has quit IRC00:03
*** awiddersheim has joined #openstack-kolla00:03
*** itlinux has quit IRC00:06
*** zhangfei has joined #openstack-kolla00:07
*** ducttape_ has quit IRC00:10
*** tonanhngo has joined #openstack-kolla00:12
*** eaguilar has joined #openstack-kolla00:12
*** k2so is now known as sdake00:16
*** yangyapeng has quit IRC00:18
*** sbezverk has quit IRC00:21
*** jamesbenson has joined #openstack-kolla00:23
*** ducttape_ has joined #openstack-kolla00:25
*** jamesbenson has quit IRC00:27
*** tonanhngo has quit IRC00:34
*** tonanhngo has joined #openstack-kolla00:35
*** ducttape_ has quit IRC00:36
*** ducttape_ has joined #openstack-kolla00:37
*** tonanhngo has quit IRC00:40
*** ducttape_ has quit IRC00:42
*** tonanhngo has joined #openstack-kolla00:43
*** tonanhngo has quit IRC00:43
*** manheim_ has joined #openstack-kolla00:44
*** tovin07_ has joined #openstack-kolla00:48
*** manheim_ has quit IRC00:49
*** zhurong has joined #openstack-kolla00:51
*** openstackgerrit has joined #openstack-kolla00:58
openstackgerritchao liu proposed openstack/kolla-ansible master: Fix a typo in all.yml: replace Vaid with Valid  https://review.openstack.org/49273900:58
*** HyperJohnGraham has quit IRC01:01
*** itlinux has joined #openstack-kolla01:04
*** kevinz has joined #openstack-kolla01:06
*** StephenWang1991 has joined #openstack-kolla01:15
*** StephenWang1991 has quit IRC01:16
*** goldyfruit has joined #openstack-kolla01:17
*** StephenWang1991 has joined #openstack-kolla01:17
*** stefan_bo has joined #openstack-kolla01:20
*** yangyapeng has joined #openstack-kolla01:21
*** stefan_bo has quit IRC01:26
*** caowei has joined #openstack-kolla01:47
openstackgerritTaeha Kim proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49116101:48
*** stefan_bo has joined #openstack-kolla01:49
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Remove service_providers in vpnaas_agent.ini  https://review.openstack.org/49276401:51
openstackgerritTaeha Kim proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49116101:59
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add vpnaas support to vsphere by vmware-nsx plugin  https://review.openstack.org/48429302:01
*** zhangfei has quit IRC02:02
openstackgerritTaeha Kim proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49116102:02
*** goldyfruit has quit IRC02:04
*** HyperJohnGraham has joined #openstack-kolla02:05
openstackgerritTaeha Kim proposed openstack/kolla stable/ocata: Add RHEL subscription registration  https://review.openstack.org/49135502:07
*** stefan_bo has quit IRC02:10
*** yangyape_ has joined #openstack-kolla02:11
*** StephenW_ has joined #openstack-kolla02:11
*** stefan_bo has joined #openstack-kolla02:12
*** cinerama` has joined #openstack-kolla02:13
*** cinerama has quit IRC02:18
*** yangyapeng has quit IRC02:18
*** StephenWang1991 has quit IRC02:18
*** jistr has quit IRC02:18
*** dciabrin has quit IRC02:18
*** tovin07_ has quit IRC02:18
*** gaowen has quit IRC02:18
*** clayton has quit IRC02:18
*** duritong has quit IRC02:18
*** eaguilar has quit IRC02:19
*** zhangfei has joined #openstack-kolla02:19
*** eaguilar_ has joined #openstack-kolla02:19
*** clayton has joined #openstack-kolla02:19
*** openstackgerrit has quit IRC02:21
*** jistr has joined #openstack-kolla02:23
*** iniazi has joined #openstack-kolla02:25
*** duritong has joined #openstack-kolla02:27
*** bmace has quit IRC02:28
*** bmace has joined #openstack-kolla02:29
*** dciabrin has joined #openstack-kolla02:29
*** goldyfruit has joined #openstack-kolla02:29
*** gaowen has joined #openstack-kolla02:29
*** goldyfruit has quit IRC02:30
*** zhangfei has quit IRC02:32
*** kristian__ has joined #openstack-kolla02:33
*** unicell has quit IRC02:38
*** ducttape_ has joined #openstack-kolla02:38
*** stefan_bo has quit IRC02:38
*** ducttape_ has quit IRC02:43
mw_I'm getting "manifest for 192.168.50.110:5000/kolla/centos-binary-kolla-toolbox:4.0.2 not found" while running `kolla-ansible pull`02:46
mw_Has anyone seen that before?02:46
mw_4.0.2 doesn't seem to exist in docker hub, so I tried a version that exists, and that one fails too.02:47
mw_SamYaple: Any ideas?02:47
*** kkxue has joined #openstack-kolla02:49
SamYaplethat is pointing to a private registry02:49
*** zhangfei has joined #openstack-kolla02:49
mw_Yup.02:52
mw_The instructions are pretty vague.. How does one pull the necessary images to the local registry...02:52
*** openstackgerrit has joined #openstack-kolla02:59
openstackgerritzhangfei gao proposed openstack/kolla master: nova-libvirt: fix no /usr/sbin/libvirtd issue  https://review.openstack.org/49277802:59
openstackgerritzhangfei gao proposed openstack/kolla master: nova-libvirt: add package qemu-efi to support uefi iamge  https://review.openstack.org/49277902:59
openstackgerritzhangfei gao proposed openstack/kolla master: nova-libvirt: fix kvm permission issue  https://review.openstack.org/49278002:59
openstackgerritzhangfei gao proposed openstack/kolla master: nova-compute: add package qemu-efi to support uefi iamge  https://review.openstack.org/49278102:59
openstackgerritzhangfei gao proposed openstack/kolla master: kolla-toolbox: fix version dependency of pbr & os-client-config  https://review.openstack.org/49278202:59
*** stefan_bo has joined #openstack-kolla03:01
*** dave-mccowan has quit IRC03:06
*** HyperJohnGraham has quit IRC03:07
*** unicell has joined #openstack-kolla03:12
*** HyperJohnGraham_ has joined #openstack-kolla03:13
*** sbezverk has joined #openstack-kolla03:24
kkxueHi,guys,how to test odl patch?03:27
kkxuehttps://docs.openstack.org/kolla/latest/template-override/opendaylight-source.html03:27
*** kkxue has quit IRC03:27
*** kkxue has joined #openstack-kolla03:27
*** duonghq has joined #openstack-kolla03:28
duonghqinc0, ansbile-become ps is ready for review03:28
kkxueIn addition to the above document, do you have a more detailed description?03:29
*** stefan_bo has quit IRC03:31
*** zhangfei has quit IRC03:33
*** HyperJohnGraham_ has quit IRC03:33
*** unicell has quit IRC03:37
mw_SamYaple: Is there a process to go through to pull/push the images to my registry?03:37
mw_kolla-ansible pull doesn't work, and that's what seems to be documented from what I can tell...03:38
mw_SamYaple: Think I figured it out... Nevermind :)03:39
*** itlinux_ has joined #openstack-kolla03:44
*** manheim_ has joined #openstack-kolla03:45
*** manheim_ has quit IRC03:49
*** zhangfei has joined #openstack-kolla03:50
mw_SamYaple: Is there a trick to running kolla-ansible pull?03:51
mw_Shouldn't be difficult, but it's failing on me.03:51
*** yangyape_ has quit IRC03:51
*** SamYaple has quit IRC03:57
*** jamesbenson has joined #openstack-kolla03:59
sdakemw_ mind being more specific about what is failing04:01
*** tonanhngo has joined #openstack-kolla04:02
*** itlinux_ has quit IRC04:03
mw_sdake: Sorry.. I think I had set the docker registry in globals.yml before running kolla-ansible pull, so it was using the registry instead of docker hub. I fixed that. Now I'm getting "manifest for kolla/c entos-binary-fluentd:3.0.2 not found"04:04
*** jamesbenson has quit IRC04:04
mw_That error makes sense... that actually doesn't exist.04:04
mw_My question is why is it trying to pull that in the first place?04:04
openstackgerritwanghongxu proposed openstack/kolla-ansible master: Set haproxy deploy on control nodes  https://review.openstack.org/49279504:04
sdakefluentd didn't exist in mitaka04:05
mw_Do I need to pip install specific versions of kolla and kolla ansible to match the version of the packages I want to pull?04:05
mw_3.0.2 is newton, no?04:05
sdakerather newton04:05
sdake3.0.2 uses heka04:05
sdakemw_ pip show kolla -> paste04:06
mw_Version: 4.0.204:06
mw_That what you're lookin for?04:06
sdakepip show kolla-ansible -> paste04:06
mw_4.0.204:06
mw_do I need to ==3.0.2?04:06
sdakepaste globals.yml04:06
mw_openstack_release: "3.0.2"04:07
mw_(globals is huge..)04:07
sdakeright04:07
*** sbezverk has quit IRC04:07
*** kkxue has quit IRC04:07
*** kristian__ has quit IRC04:07
mw_I'm assuming I need to ==3.0.2 for both of those pip installs?04:07
sdakewhy deploying 3.0.2 images with 4.0.2 of kolla-ansible?04:07
*** kkxue has joined #openstack-kolla04:08
mw_I didn't know I couldn't, basically.04:08
mw_I didn't realized the kolla-ansible releases were specifically tied to OS releases04:08
mw_sdake: To install a different version of OS do I need to just use a different version of the kolla* packages?04:10
sdakeyes although there is no "LTS" of OpenStack04:10
mw_manifest for kolla/centos-binary-kolla-toolbox:4.0.2 not found04:10
mw_sdake: Right.. I knew that :)04:10
mw_So.. I can't run the 4.0.2 defaults?04:11
*** unicell has joined #openstack-kolla04:11
mw_Because the required packages don't exist?04:11
*** SamYaple has joined #openstack-kolla04:11
sdakenewton is still a maintained branch04:11
sdaketo deploy newton, you need newton version of the ansible playbooks04:12
sdakein newton, the ansible playbooks are part of the kolla pip package04:12
mw_sdake: Ok. But if I just pip install kolla and kolla-ansible without a specific version (defaults to latest which is 4.0.2) I can't run kolla-ansible pull.04:13
mw_Because: manifest for kolla/centos-binary-kolla-toolbox:4.0.2 not found04:13
mw_Is that correct?04:13
sdakesudo pip install -U kolla==3.0.204:13
sdakesudo pip uninstall kolla-ansible prior to that04:13
mw_I'm not trying to install newton anymore04:14
mw_I'll take the latest04:14
sdakewhy is 3.0.2 in your globals.yaml?04:14
mw_Because the deault wasn't working04:14
mw_so i tried the newton version04:14
sdakei see04:14
mw_the default version gave me that error04:14
mw_Sorry for the confusion :S04:14
sdakewell remove that openstack_version override and lets work on getting ocata operatoinal for you04:14
mw_I did04:15
mw_That's why I'm getting this error now: manifest for kolla/centos-binary-kolla-toolbox:4.0.2 not found04:15
sdakewhat is your registry setting in globals.yml04:15
mw_It's commented out04:15
sdakegot it04:15
openstackgerritJeffrey Zhang proposed openstack/kolla master: Use immutable journal path rather than device name  https://review.openstack.org/49279604:15
*** itlinux_ has joined #openstack-kolla04:16
mw_Since it's commented out, it's using 4.0.2, which doesn't appear to have all the required packages in docker hub, is that right? There is definitely no 4.0.2 for that image.04:16
sdake4.0.1 and 4.0.2 are not in the dockerhub04:17
sdakeif you want those versions you have to build them in a local registry04:17
sdakeare you simply trying to eval?04:17
mw_I'm trying to deploy several versions using kolla04:17
sdakeseveral versions of which?04:18
sdakeopenstack?04:18
mw_I've been using other deployment methods, but wanted to give kolla a go. Several versions of OS.04:18
mw_Yeah04:18
sdakei see04:18
sdakeset openstack_version to 4.0.004:18
sdakethat will get you a 4.0.0 deployment04:18
sdakethere are no 4.0.1 images in docker hub, nor are there 4.0.2 images in dockerhub04:18
mw_And pip uninstall/install 4.0.0 for kolla* as well?04:18
sdakeif you want 4.0.1 or 4.0.2 images, you will need to build your own docker images04:19
sdakeat present, we are working with openstack infra to sort out a proper artifact release mechanism to dockerhub04:19
*** itlinux_ has quit IRC04:19
*** itlinux_ has joined #openstack-kolla04:19
mw_Ok, sounds good. Thanks for your help!!04:20
mw_Sorry about all the confusion :S04:20
mw_That's entirely my fault.04:20
*** unicell1 has joined #openstack-kolla04:21
sdakewe typically tag every 45 days, and for a long time released docker images on tag, but we prefer a model where what we test is what becomes tagged04:21
*** unicell has quit IRC04:21
sdakeso we have put a hold on dockerhub image publishing while inc0 sorts that out04:21
mw_sdake: Good to know. That's great. Thanks a lot  :)04:22
*** itlinux_ has quit IRC04:23
*** kristian__ has joined #openstack-kolla04:28
mw_sdake: This was another reason I tried 4.0.2...04:29
mw_"Hostname has to resolve to IP address of api_interface"04:29
mw_for rabbitmq04:29
mw_That precheck is failing. My 2 nodes can resolve eachother by hostname.04:29
*** kristian__ has quit IRC04:32
mw_sdake: That's with 4.0.004:34
*** zhubingbing has joined #openstack-kolla04:35
mw_The error was: Invalid conditional detected: EOL while scanning string literal (<unknown>, line 1)\n\nThe error appears to have been in '/usr/share/kolla-ansible/ansible/roles/rabbitmq/tasks/precheck.yml': line 54, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- fail: msg=\"Hostname has to resolve to IP address of api_interface\"\n  ^ here\n"}04:36
mw_sdake: bad yaml file?04:37
mw_sdake: Any idea?04:46
*** vhosakot has joined #openstack-kolla04:47
*** vhosakot has quit IRC04:52
*** vhosakot has joined #openstack-kolla04:52
*** caowei has quit IRC04:56
*** awiddersheim has quit IRC05:05
sdakemw_ was watcing the tube05:10
sdakemw_ not really sure although ocata was 4.0.0 was not DOA05:11
sdakemw_ that for certain worked when tagged, and tags in openstack are immutable05:11
sdakeJeffrey4l around?05:12
*** awidders_ has joined #openstack-kolla05:13
sdakemw_ would lov eto help more - but need to hit the rack05:13
sdakemw_ my recommendation at this point would be to build 4.0.2 images05:14
sdakemw_ very simple to setup your own docker registry and build your own images05:14
sdakemw_ after building 4.0.2 images use kolla-ansible 4.0.205:15
*** vhosakot has quit IRC05:16
sdakehttps://docs.openstack.org/kolla/latest/image-building.html05:17
Jeffrey4lsdake, yep.05:17
sdakemw_ after installing a registry (read docker docs - its a one liner command) run kolla-build --type source --registry 172.22.2.81:5000 --push05:18
sdakewait 30 minutes or so05:18
sdakewhile it biulds05:18
sdakethen point your globals.yml registry at it05:19
sdakeJeffrey4l does that error mw_ pointed out above make any sense to you05:19
* Jeffrey4l is reading scroll.05:23
*** ducttape_ has joined #openstack-kolla05:30
*** caowei has joined #openstack-kolla05:33
*** kristian__ has joined #openstack-kolla05:34
*** ducttape_ has quit IRC05:34
*** kristian__ has quit IRC05:38
Jeffrey4lmw_, has u solved you issue?05:45
*** lpetrut has joined #openstack-kolla05:52
*** zhubingbing has quit IRC05:54
*** itlinux has quit IRC05:55
*** zhubingbing has joined #openstack-kolla05:56
*** itlinux has joined #openstack-kolla05:57
*** zhubingbing has quit IRC05:58
*** stefan_bo has joined #openstack-kolla06:02
*** gfidente has joined #openstack-kolla06:05
*** lvdombrkr has joined #openstack-kolla06:06
iniaziafter i do kolla-ansible pull, i get "manifest for 10.0.0.32:5000/kolla/centos-binary-kolla-toolbox:4.0.2 not fou06:12
iniazind"06:12
*** janki has joined #openstack-kolla06:16
openstackgerritMerged openstack/kolla master: Fix title spell and rst  https://review.openstack.org/48996506:16
*** skramaja has joined #openstack-kolla06:20
openstackgerritMerged openstack/kolla-ansible master: Fix a typo in globals.yml: replace mathcing with matching  https://review.openstack.org/49240806:21
*** itlinux has quit IRC06:28
*** ducttape_ has joined #openstack-kolla06:31
openstackgerritMerged openstack/kolla master: Add RHEL distro available for building images  https://review.openstack.org/49123806:32
*** kristian__ has joined #openstack-kolla06:33
*** kristian__ has quit IRC06:34
*** ducttape_ has quit IRC06:35
*** stefan_bo has quit IRC06:36
lvdombrkrhello guys, is any one has deploy VPNaaS in production?06:40
*** ashishb has joined #openstack-kolla06:41
*** zhubingbing has joined #openstack-kolla06:42
*** kkxue has quit IRC06:47
*** manheim_ has joined #openstack-kolla06:50
*** lpetrut has quit IRC06:52
*** manheim_ has quit IRC06:53
*** kristian__ has joined #openstack-kolla07:00
*** zhongshengping has joined #openstack-kolla07:00
*** zhongshengping has quit IRC07:01
*** jaosorior has quit IRC07:03
openstackgerritzhangfei gao proposed openstack/kolla master: nova-libvirt: add package qemu-efi to support uefi image  https://review.openstack.org/49277907:03
openstackgerritzhangfei gao proposed openstack/kolla master: nova-libvirt: fix kvm permission issue  https://review.openstack.org/49278007:03
openstackgerritzhangfei gao proposed openstack/kolla master: nova-compute: add package qemu-efi to support uefi image  https://review.openstack.org/49278107:03
openstackgerritzhangfei gao proposed openstack/kolla master: kolla-toolbox: fix version dependency of pbr & os-client-config  https://review.openstack.org/49278207:03
*** magicboiz has joined #openstack-kolla07:08
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Standardize Fluentd grammar usage  https://review.openstack.org/49283207:12
*** jaosorior has joined #openstack-kolla07:15
openstackgerritTaeha Kim proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49116107:15
openstackgerritTaeha Kim proposed openstack/kolla-ansible master: Fix nova-libvirt and nova-compute fails to deploy  https://review.openstack.org/49203207:16
openstackgerritTaeha Kim proposed openstack/kolla master: Fix ceilometer-compute fails to build in RHEL  https://review.openstack.org/49168507:18
openstackgerritTaeha Kim proposed openstack/kolla master: Fix nova-compute fails to build in RHEL  https://review.openstack.org/49171007:18
*** kristian__ has quit IRC07:23
*** stefan_bo has joined #openstack-kolla07:33
*** jamesbenson has joined #openstack-kolla07:35
lvdombrkr hello guys, is any one has deploy VPNaaS in production?07:37
*** jamesbenson has quit IRC07:40
*** rwsu has joined #openstack-kolla07:40
*** kristian__ has joined #openstack-kolla07:41
*** magicboiz has quit IRC07:47
*** magicboiz has joined #openstack-kolla07:47
*** kristia__ has joined #openstack-kolla07:51
*** kristian__ has quit IRC07:53
*** manheim_ has joined #openstack-kolla07:55
*** manheim_ has quit IRC07:56
*** manheim_ has joined #openstack-kolla07:56
*** daidv has joined #openstack-kolla07:57
*** rwsu has quit IRC07:58
*** rwsu has joined #openstack-kolla08:00
*** openstackgerrit has quit IRC08:02
*** stefan_bo has quit IRC08:05
*** magicboiz has quit IRC08:06
*** tovin07_ has joined #openstack-kolla08:14
*** mgoddard has joined #openstack-kolla08:18
*** stefan_bo has joined #openstack-kolla08:19
*** serlex has joined #openstack-kolla08:20
*** magicboiz has joined #openstack-kolla08:21
*** shardy has joined #openstack-kolla08:21
*** yangyapeng has joined #openstack-kolla08:30
*** stefan_bo has quit IRC08:32
*** mgoddard has quit IRC08:35
*** rwsu has quit IRC08:37
*** mgoddard has joined #openstack-kolla08:43
pbourkelvdombrkr: I have a feeling that project is not being maintained, may be wrong08:44
lvdombrkrpbourke: thank you. does anyone know something else/more?08:45
*** jmccarthy has joined #openstack-kolla08:48
*** rwsu has joined #openstack-kolla08:49
jmccarthyHmm I'm running into what seem to be uuid formatting errors with designate08:55
jmccarthyI have a zone_id like zone_id = e30bc143-ab2d-4c18-a157-c71fed3e984e, but the zone_id I see in the record in the db is like: e30bc143ab2d4c18a157c71fed3e984e08:56
jmccarthyand designate central for example complains: ValueError: badly formed hexadecimal UUID string08:56
jmccarthyShouldn't the record in the db have the same format as I put in the designate-sink.conf file ?08:56
*** stefan_bo has joined #openstack-kolla09:00
*** sambetts|afk is now known as sambetts09:02
*** yangyapeng has quit IRC09:03
jmccarthyWhere is egonzalez when you need him ;)09:04
*** stefan_bo has quit IRC09:05
pbourkejmccarthy: sounds maybe more like a question for designate folks?09:13
jmccarthypbourke: Yea, I'm asking there too :)09:13
pbourkelvdombrkr: I may be wrong on vpnaas, there seems to be active commits going in lately. here's some discussion on it in prod if it's helpful http://lists.openstack.org/pipermail/openstack-operators/2016-May/010295.html09:15
*** openstackgerrit has joined #openstack-kolla09:21
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add inventory file path in swift-guide.rst  https://review.openstack.org/49292009:21
*** stefan_bo has joined #openstack-kolla09:41
*** rwsu has quit IRC09:44
*** stefan_bo has quit IRC09:47
*** kevinz has quit IRC09:54
*** stefan_bo has joined #openstack-kolla09:55
*** rwsu has joined #openstack-kolla09:56
*** daidv has quit IRC10:00
*** StephenW_ has quit IRC10:02
*** egonzalez has joined #openstack-kolla10:03
*** duonghq has quit IRC10:03
*** tovin07_ has quit IRC10:03
*** yangyapeng has joined #openstack-kolla10:04
lvdombrkrpbourke: thank you, its aprox. 2 years old )) i will test myself, maybe i will be second in production10:06
lvdombrkr))10:06
egonzalezjmccarthy whats the issue with designate? mind sharing designate-sink config file and output of designate zones/domain? what openstack release?10:08
*** stefan_bo has quit IRC10:08
*** yangyapeng has quit IRC10:09
jmccarthyegonalez: I'm not sure quite where the disconnect is, but after I reconfigure kolla with the zone_id it never goes to 'active'  - I raised this: https://bugs.launchpad.net/designate/+bug/171012310:11
openstackLaunchpad bug 1710123 in Designate "Designate central complains ValueError: badly formed hexadecimal UUID string" [Undecided,New]10:11
jmccarthyThis is with images I built off master10:11
openstackgerritMerged openstack/kolla-ansible master: Fix a typo in all.yml: replace Vaid with Valid  https://review.openstack.org/49273910:12
openstackgerritMerged openstack/kolla-ansible master: Skydive: enable logging in file  https://review.openstack.org/49056110:13
egonzalezjmccarthy maybe something changed in designate that need to be modified for pike, will try a deploy10:15
egonzalezjmccarthy mind sharing your /etc/kolla/designate-sink/designate.conf?10:16
jmccarthyegonzalez: Sure, http://paste.openstack.org/show/618153/10:19
*** eaguilar_ has quit IRC10:22
openstackgerritMerged openstack/kolla master: Fix code indentation formate on octavia dockerfile  https://review.openstack.org/49137510:33
*** ducttape_ has joined #openstack-kolla10:36
*** ducttape_ has quit IRC10:41
serlexAnyone seen "caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:89: jailing process inside rootfs caused" on a container (neutron_l3_agent in this case)?10:49
serlex"rror mkdir /var/lib/docker/devicemapper/mnt/...no space left on device" I don't have a space issue10:50
egonzalezjmccarthy just deployed and worked fine for me, centos source, images built 46 hours ago10:54
*** rwsu has quit IRC10:56
jmccarthyegonzalez: :/ ok that suggests it's something in my env then10:57
*** shardy is now known as shardy_lunch10:59
jmccarthyegonzalez: Thanks for trying it !10:59
*** zhurong has quit IRC11:00
*** magicboiz has quit IRC11:05
*** yangyapeng has joined #openstack-kolla11:05
*** magicboiz has joined #openstack-kolla11:05
*** hieulq has quit IRC11:07
*** rwsu has joined #openstack-kolla11:08
*** yangyapeng has quit IRC11:10
*** zhubingbing has quit IRC11:13
*** magicboiz has quit IRC11:13
*** magicboiz has joined #openstack-kolla11:14
*** caowei has quit IRC11:15
openstackgerritMerged openstack/kolla-ansible master: Clear all l3 related namespace before starting neutron-l3-agent  https://review.openstack.org/48196911:16
*** iniazi has quit IRC11:19
openstackgerritMerged openstack/kolla-ansible master: Add fwaas support to vsphere by vmware-nsx plugin  https://review.openstack.org/48410511:21
*** magicboiz has quit IRC11:24
*** egonzalez has quit IRC11:24
*** shardy_lunch has quit IRC11:24
*** kristia__ has quit IRC11:24
*** zhangfei has quit IRC11:25
*** shardy_lunch has joined #openstack-kolla11:30
*** magicboiz has joined #openstack-kolla11:32
s-deanhello, could some one remind me what the kolla_internal_vip_address is for again, i keep binding this to the management network but pre-checks complains its pingable.11:32
*** shardy_lunch is now known as shardy11:33
*** ducttape_ has joined #openstack-kolla11:37
*** ducttape_ has quit IRC11:40
*** ducttape_ has joined #openstack-kolla11:40
*** sbezverk has joined #openstack-kolla11:42
*** ducttape_ has quit IRC11:42
*** ducttape_ has joined #openstack-kolla11:42
*** rwsu has quit IRC11:42
*** magicboiz has quit IRC11:46
*** ducttape_ has quit IRC11:46
*** magicboiz has joined #openstack-kolla11:46
*** sbezverk has quit IRC11:50
*** kristian__ has joined #openstack-kolla11:51
*** magicboiz has quit IRC11:53
*** magicboiz has joined #openstack-kolla11:54
*** egonzalez has joined #openstack-kolla12:05
*** rhallisey has joined #openstack-kolla12:10
serlexs-dean; it should be an unused IP address in the internal management network12:13
serlexIt is used by keepalived as VIP12:13
sdakemorning folks12:18
*** mwhahaha has quit IRC12:19
*** zioproto has quit IRC12:19
*** nanliu_ has quit IRC12:19
*** classicsnail has quit IRC12:19
*** berendt has quit IRC12:19
*** rwellum has quit IRC12:19
sdakes-dean to answer the second part of your question, the reason there is a complaint about being pingable is because it shouldn't be on the same network as the network_interface12:19
*** classicsnail has joined #openstack-kolla12:19
*** zioproto has joined #openstack-kolla12:19
*** berendt has joined #openstack-kolla12:19
*** nanliu_ has joined #openstack-kolla12:19
*** rwellum has joined #openstack-kolla12:20
*** mwhahaha has joined #openstack-kolla12:20
*** awidders_ has quit IRC12:20
*** awiddersheim has joined #openstack-kolla12:20
*** clayton has quit IRC12:21
*** schwicht has joined #openstack-kolla12:22
*** mandre_mtg has quit IRC12:22
*** lpetrut has joined #openstack-kolla12:22
*** clayton has joined #openstack-kolla12:24
*** mandre has joined #openstack-kolla12:24
*** dave-mccowan has joined #openstack-kolla12:25
*** schwicht has quit IRC12:28
*** schwicht has joined #openstack-kolla12:29
rwellumMorning sdake  - looking over your review comments - might have a qn or two :)12:37
sdakerwellum shoot12:39
rwellumOh I'll reply in the review - just wanted to give you a heads up.12:40
rwellumAnd ty for the review.12:40
sdakesure happy to help12:40
*** mnasiadka has joined #openstack-kolla12:41
*** ducttape_ has joined #openstack-kolla12:43
*** eaguilar has joined #openstack-kolla12:45
s-deanis there anyway to turn off HA Proxy12:45
*** eaguilar_ has joined #openstack-kolla12:47
*** ducttape_ has quit IRC12:47
s-deansdake: so basically its can be a random IP doesn't have to even align with the management network IP's. Is it internal only ?12:48
s-deane.g. between containers12:48
*** eaguilar has quit IRC12:50
*** schwicht has quit IRC12:50
*** ducttape_ has joined #openstack-kolla12:50
sdakes-dean you can turn off haproxy for single node deploys - however for multlinode the answer is no12:53
s-deanahhh12:53
sdakes-dean (if you want things to work) - unless you have a hardware load balancer in which caes - your on your own :)12:53
sdakes-dean your assessment of the basically is correct - it is internal only - all traffic is sent to th einternal VIP - the internal VIP is then routed to the various endpoints based upon HAProxy standard behaviors12:54
*** ducttape_ has quit IRC12:54
*** janki has quit IRC12:58
*** jtriley has joined #openstack-kolla13:02
*** yangyapeng has joined #openstack-kolla13:07
rwellumHey sdake - I have a new toy - it's a 'mini' server -basically small high powered PC - 20G RAM, 1TB disk. It currently has wifi and one NIC. What the chances that I can run kolla-k8s on it? I'm currently ignoring the WIFi connection, have created a bridge off the eth for a second interface and k8s comes up just fine. But after I label the node as compute and control - I lose connectivity to k8's and it's13:08
rwellumbasically dead. All bare-metal btw - OS is Ubuntu 16.04. Is this feasible you think?13:08
*** sbezverk has joined #openstack-kolla13:09
sdakekolla-k8s has a 8gb memory footprint requirement and 40-80gb of disk space requirement13:10
sdakerwellum so i'd say the gear is good13:10
sdakewifi is a problem typically13:11
rwellumsdake: yeah I started with 2G and it was unhappy - but added 16G yesterday and got a lot further.13:11
*** yangyapeng has quit IRC13:11
sdakerwellum brain still booting13:11
sdakerwellum give me a few minutes to think about your question plese13:11
rwellumsdake: no worries13:12
*** jtriley has quit IRC13:12
rwellumI'm sure the issue will be networking :)13:12
serlexBIOS or UEFI sdake?13:12
serlexif that's a terrible joke then I'm not sorry13:12
*** skramaja has quit IRC13:13
rwellumMine's DOS serlex13:13
*** egonzalez has quit IRC13:13
serlex:D13:13
rwellumThere's no multi-threading at all.13:14
*** egonzalez has joined #openstack-kolla13:16
*** jtriley has joined #openstack-kolla13:16
*** zhubingbing_ has joined #openstack-kolla13:17
*** goldyfruit has joined #openstack-kolla13:17
*** oanson has quit IRC13:19
*** oanson has joined #openstack-kolla13:19
*** schwicht has joined #openstack-kolla13:22
openstackgerritTaeha Kim proposed openstack/kolla-ansible master: Fix nova-libvirt and nova-compute fails to deploy  https://review.openstack.org/49203213:22
*** ismc has quit IRC13:31
sdakeserlex fwiw I really like UEFO13:33
sdakeit only took 15 years to make a reality - but hey - who is counting :)13:34
sdakerwellum loss of connectvity to k8s and its deadness indicates neutron is imploding your management network13:34
sdakerwellum if you can not use a bridge, and instead use a second nic, that would be the best way to proceed13:35
rwellumsdake: sadly only one nic is this thing. But they have plans for 4.13:36
rwellumBut sdake - it's not connected to a switch - just like a PC, so I only have 1 ethernet connection anyway. Hence the bridge.13:40
*** hongbin has joined #openstack-kolla13:44
*** StephenWang1991 has joined #openstack-kolla13:44
*** aagate| has joined #openstack-kolla13:44
*** zhangfei has joined #openstack-kolla13:46
spsuryarwellum: just saw your comment13:52
spsuryaLGTM13:53
*** lucasxu has joined #openstack-kolla13:54
*** StephenWang1991 has quit IRC13:55
*** schwicht has quit IRC13:56
rwellumty spsurya13:58
*** schwicht has joined #openstack-kolla13:58
sdakerwellum USB nic might work if you can sort out the switch thing13:58
sdakerwellum do you have pci slots open?13:58
rwellumThere's a ton of those I think sdake - have to experiment to see if they are active/open.13:59
sdakerwellum 100mbit nic is 29$ at amazon.com14:00
sdakeuse the 100mbit nic for neutron14:00
sdakeuse whatever your normal network is for the management/kubernetes network14:00
*** manheim_ has quit IRC14:01
*** jmccarthy has quit IRC14:04
*** jmccarthy has joined #openstack-kolla14:04
*** aagate| has quit IRC14:06
*** yangyapeng has joined #openstack-kolla14:08
*** yangyapeng has quit IRC14:12
*** mnasiadka has quit IRC14:16
*** mnasiadka has joined #openstack-kolla14:16
*** zhangfei has quit IRC14:17
*** jamesbenson has joined #openstack-kolla14:19
*** unicell1 has quit IRC14:20
*** sambetts is now known as sambetts|afk14:21
*** StephenWang1991 has joined #openstack-kolla14:21
*** mnasiadka has quit IRC14:21
*** mnasiadka has joined #openstack-kolla14:27
*** manheim_ has joined #openstack-kolla14:28
*** zhangfei has joined #openstack-kolla14:29
*** schwicht has quit IRC14:30
*** mnasiadka has quit IRC14:31
*** manheim_ has quit IRC14:32
*** itlinux has joined #openstack-kolla14:34
*** SamYaple has quit IRC14:35
*** SamYaple has joined #openstack-kolla14:35
*** rwsu has joined #openstack-kolla14:36
*** mnasiadka has joined #openstack-kolla14:36
*** StephenW_ has joined #openstack-kolla14:37
*** StephenWang1991 has quit IRC14:39
*** mnasiadka has quit IRC14:41
*** yangyapeng has joined #openstack-kolla14:44
*** kristian__ has quit IRC14:46
*** zhangfei has quit IRC14:47
*** mnasiadka has joined #openstack-kolla14:47
*** mnasiadka has quit IRC14:52
*** sbezverk has quit IRC14:54
*** sbezverk has joined #openstack-kolla14:55
*** mnasiadka has joined #openstack-kolla14:56
*** thingee_ has joined #openstack-kolla14:58
*** schwicht has joined #openstack-kolla14:59
*** mnasiadka has quit IRC15:01
*** eaguilar_ has quit IRC15:05
*** mnasiadka has joined #openstack-kolla15:07
*** bswartz has joined #openstack-kolla15:09
*** eaguilar has joined #openstack-kolla15:11
*** mnasiadka has quit IRC15:11
*** dklyle has joined #openstack-kolla15:15
*** david-lyle has quit IRC15:15
*** StephenW_ has quit IRC15:16
*** mnasiadka has joined #openstack-kolla15:16
*** ducttape_ has joined #openstack-kolla15:17
*** mnasiadka has quit IRC15:21
*** StephenWang1991 has joined #openstack-kolla15:21
*** shardy has quit IRC15:24
*** mnasiadka has joined #openstack-kolla15:26
*** jaosorior has quit IRC15:27
*** StephenWang1991 has quit IRC15:30
*** mnasiadka has quit IRC15:31
*** Manheim has quit IRC15:31
inc0good morning15:33
*** StephenWang1991 has joined #openstack-kolla15:35
*** aagate has quit IRC15:35
*** mnasiadka has joined #openstack-kolla15:36
*** StephenWang1991 has quit IRC15:39
*** mnasiadka has quit IRC15:41
*** jmccarthy has left #openstack-kolla15:41
*** aagate has joined #openstack-kolla15:42
*** matrohon has quit IRC15:45
*** itlinux has quit IRC15:46
*** mnasiadka has joined #openstack-kolla15:46
*** iniazi has joined #openstack-kolla15:47
sdakesup inc99915:48
*** mgoddard has quit IRC15:49
*** thingee_ has left #openstack-kolla15:51
*** mnasiadka has quit IRC15:51
*** eaguilar has quit IRC15:53
*** zhubingbing_ has quit IRC15:55
*** zhubingbing has joined #openstack-kolla15:56
*** mnasiadka has joined #openstack-kolla15:56
*** eaguilar has joined #openstack-kolla15:57
*** ntpttr_laptop has joined #openstack-kolla16:01
*** mnasiadka has quit IRC16:01
masbergood morning, multi-attach volume fails during creation process https://pastebin.com/raw/RQ4fv8iG any idea why?16:01
*** zhubingbing has quit IRC16:01
*** sbezverk has quit IRC16:05
*** stefan_bo has joined #openstack-kolla16:06
*** mnasiadka has joined #openstack-kolla16:06
*** lucasxu has quit IRC16:07
inc0masber: I'd ask on cinder channel16:08
*** mnasiadka has quit IRC16:11
*** stefan_bo has quit IRC16:11
*** egonzalez has quit IRC16:12
*** mnasiadka has joined #openstack-kolla16:16
*** vhosakot has joined #openstack-kolla16:17
*** lpetrut has quit IRC16:18
*** mnasiadka has quit IRC16:22
*** serlex has left #openstack-kolla16:23
vhosakotawesome rwellum nominated for core for kolla-k8s ... :)16:25
*** mnasiadka has joined #openstack-kolla16:26
*** mnasiadka has quit IRC16:31
inc0:)16:32
*** itlinux has joined #openstack-kolla16:34
*** mnasiadka has joined #openstack-kolla16:36
iniazihey guys, so I finally got the prechecks to go all green (kolla-ansible).  but having a problem with pulling images.  it fails on kolla/centos-binary-kolla-toolbox:4.0.2 .16:38
iniaziwell the other nodes are using local registry and on registry, I try to search and don't see that.. i think i found 4.0.0.  so then I tried to do kolla-build, but that givs an error on percona repo.  basically it can't download the file.  i can do it myself, so maybe docker doesn't have the right network connect info?  (i'm assuming kolla-build looks builds stuff in docker )16:39
*** mnasiadka has quit IRC16:41
mw_inc0: Is there documentation somewhere regarding changing configs for services inside the docker containers?16:43
mw_I need to make some small changes... what's the best way to do that?16:43
*** dklyle has quit IRC16:44
*** dklyle has joined #openstack-kolla16:45
*** egonzalez has joined #openstack-kolla16:46
*** mnasiadka has joined #openstack-kolla16:46
*** spiette has quit IRC16:49
*** mnasiadka has quit IRC16:51
sdakeinc0 why dont we just push 4.0.2 images to dockerhub to quell the constant questions that folks like iniazi are struggling with16:53
iniazior am I not supposed to use "docker-ansible pull"?16:55
iniazior is it an issue with using a local registry?16:55
mw_sdake: Sound familiar?16:56
mw_It's last night for me all over again :)16:56
sdakemw_ pretty much16:56
mw_iniazi: You need to change the openstack version in globals.yml16:56
mw_before kolla-ansible pull16:56
*** mgoddard has joined #openstack-kolla16:56
*** mnasiadka has joined #openstack-kolla16:56
mw_sdake: my memcached container has been rebooting constantly since my stack came up so I can't actually do anything...16:57
mw_Seen that before?16:57
inc0sdake: TC said images have to be refreshed16:57
iniazioh i guess I missed that.  let me check16:57
iniazimw_ thanks16:57
inc0in resolution we worked on16:57
sdakeinc0 i dont follow16:58
inc0so I'm kinda holding off pushing images until we have system for updating these16:58
inc0check resolution regarding binary publishing16:58
sdakeok - i understand images have to be refreshed16:58
sdakethe system we have used forever for updating is manual16:58
inc0right16:59
sdakeis that some sort of violation of the tc resolution?16:59
inc0I think it might16:59
inc0hence I didn't push since resolution merged16:59
iniaziIs this what i need to change: #openstack_release: "auto" ?16:59
inc0iniazi: question - is this prod?16:59
inc0if it is, please build your own images17:00
iniaziyes, trying to do multinode... new install17:00
inc0then openstack_release: auto works17:00
iniaziand trying to do Ocata, i guess... or is it trying to do pike?17:00
inc0I have to go to appointment now, so I'll be back in 2hrs or so, but generally what you should do is build your own images17:00
inc0if you did pip install kolla17:01
inc0then it's ocaa17:01
inc0Pike is not released yet17:01
iniazihow do I build images... using kolla-build?17:01
*** mnasiadka has quit IRC17:01
inc0right17:01
iniaziyeah, i did pip install17:01
iniaziok, that one had an issue with downloading percona yum repo17:01
inc0https://docs.openstack.org/kolla/latest/image-building.html this is pretty comprehensive17:01
sdakehttps://docs.openstack.org/kolla/latest/image-building.html17:01
sdakeiniazi 4.0.0 does not build as a result of upstream changes in the percona repository17:02
sdakeiniazi it did when it was released, and the tagged images work properly17:02
inc0latest pip install is 4.0.217:02
iniaziok, cool, I'll look through.  or atleast thats the only red i saw in the output of kolla-build.  first time I jsut ran "kolla-build" but second time I ran it with my local registry, but didn't see any images in my local registry17:02
inc0I think17:02
inc0iniazi: you need to run registry first17:02
inc0then do kolla-build --registry="address:port" --push17:03
iniazirun registry?  i mean my registry is running, oh maybe I left --push17:03
sdakeiniazi - i'd go with -t source as well :)17:03
iniazithis one of the errors i see with build:17:03
iniaziINFO:kolla.image.build.base:Cannot open: http://repo.percona.com/release/7/RPMS/x86_64/percona-release-0.1-4.noarch.rpm. Skipping.17:03
iniaziINFO:kolla.image.build.base:Error: Nothing to do17:03
inc0I'll be back in a bit17:03
iniaziinc0: ok, cool.  see ya.  I'll see if the docs help me with the percona thing17:04
sdakeiniazi run pip show kolla17:04
iniazipip show kolla: Version: 4.0.217:04
iniazicentos 7.317:04
sdakeiniazi let me do a test build here - takes about 30 minutes17:04
sdakeiniazi can you provide the exact command line you used please17:05
mw_sdake: Quick one for you: "failed to set rlimit for open files. Try starting as root or requesting smaller maxconns value."17:05
iniazito build, I only used "kolla-build" first, and then I realized I need to specify registry, and then I ran with --registry, but didn't do --push.  maybe percona isn't needed?17:05
mw_That's why memcached container is constantly rebooting. I decreased the size of -c and it comes up.17:05
mw_Should that be getting ran as root? Or what's the deal there?17:06
iniazibut on both of the kolla-build commands, I got that percona error.  actually i get a few of those percona errors.17:06
sdakeone at atime fellas17:06
iniazimemcache shouldn't matter if its root or not17:06
*** mnasiadka has joined #openstack-kolla17:07
rwellumvhosakot: ty :)17:07
vhosakotrwellum: :)17:07
mw_iniazi: That error seems odd then17:07
mw_it's too large for a non root user to set17:07
mw_but that's the selected dfefault?17:07
mw_So how does that work for anyone?17:07
vhosakotiniazi: are you pushing twice with same tag by any chance?17:07
iniazioh didn't see the other thing.. yeha you need to up the open files...17:07
mw_iniazi: How?17:08
iniazivhosakot, i don't think so.  unless I ran some other command to set a tag, but I ran "kolla-build" first, and then "kolla-build --registry=local.ip.addy:5000"17:09
iniazibut let me run it with --push also, but in the first build i ran, and subsequent, all output is white, but except for the percona messages.  first one is "Cannot open: http://repo.percona.com/release/7/RPMS/x86_64/percona-release-0.1-4.noarch.rpm. Skipping"  so it can't connect I guess.  I can get to that url fine, from the same host17:09
*** mnasiadka has quit IRC17:11
iniazioh i didn't do the tox initially17:12
iniazii ran tox later on, but not sure where I'm supposed to run tox -e genconfig from?17:12
vhosakotiniazi: well, well push is useless if build is failing :)   I'd check build logs.17:13
iniaziwhere are the build logs?17:13
sdakemw_ as soon as i get a build kicked off - i'll lend you hand - hang tightplz17:13
vhosakotiniazi: on the stdout screen or you can pass the arg --build-logs17:13
mw_sdake: Will do. Thanks!!17:13
iniazioh ok, yeah I'm looking and there is only that error I pasted above...17:13
iniaziit can't open the percona repo rpm17:14
sdakeiniazi - ok so just to be specific17:14
sdakeiniazi you can "kolla-build"17:14
sdakeiniazi the nyou can "kolla-build --registery host:port"17:14
sdakethen you can kolla-build --push?17:14
iniaziactually after percona error, it fails to build everything (well the message it prints before exiting the build... packages that failed to build is pretty long).17:16
sdakeiniazi - tell me commands you typed17:16
*** mnasiadka has joined #openstack-kolla17:16
sdakei am going to type them in on my gear17:17
sdakeand see if it is reproducible17:17
iniaziyeah I did "kolla-build" first.  saw those percona errors.  than I noticed on the docs that I was supposed to specify --registry so I ran it "kolla-build --registry 10.0.0.32:5000"17:18
iniaziand right now, I just ran "kolla-build --registry 10.0.0.32:5000 --push", but i cancelled it after I saw percona errors17:18
sdakeok - next question - run pip show docker-py17:18
iniazidocker-py: Version: 1.10.617:18
sdakedocker --version17:18
sdake^^17:18
iniaziDocker version 17.06.0-ce, build 02c1d8717:19
iniazioh yeah, that was my other question... docker version, if its supported.17:19
sdakeok - we have a different docker version17:19
sdakewill take me a bit to determine - let me get started on the build17:19
iniazidocs aren't clear on version of docker.  they recommend to use docker's official way of installing, which gets you the latest.17:20
*** mnasiadka has quit IRC17:21
iniazisdake: and thanks for trying to reproducce it.  appreciate it17:21
*** ntpttr_laptop has quit IRC17:23
sdakeiniazi building17:26
sdakemw_ - ok could you explain the problem yo uhave plz17:26
sdakeI have 25 minutes17:26
sdakeand then gotta roll17:26
*** mnasiadka has joined #openstack-kolla17:26
sdakeiniazi are you willing to change around some dependency versions?17:28
*** mnasiadka has quit IRC17:31
*** mnasiadka has joined #openstack-kolla17:36
*** tosika has quit IRC17:37
*** ducttape_ has quit IRC17:39
*** mnasiadka has quit IRC17:41
*** ducttape_ has joined #openstack-kolla17:42
mw_sdake: memcached -c 5000 causes the container to reboot constantly17:42
mw_sdake: "failed to set rlimit for open files. Try starting as root or requesting smaller maxconns value."17:42
mw_5000 being the default value17:42
mw_Since it doesn't run as root it can't increase that limit17:43
mw_Also, is there a sudo password for each of the users? memcached/neutron/etc...? The PWs in the passwords file don't work.17:43
*** ntpttr_laptop has joined #openstack-kolla17:44
*** ducttape_ has quit IRC17:46
*** mnasiadka has joined #openstack-kolla17:46
sdakemw_ sudo is totally locked down for security purposes17:47
sdakewhere does the -c 5000 option come from?17:47
mw_sdake: /etc/kolla/memcached/config.json17:48
sdakemw_  thanks - version of kolla?17:48
mw_4.0.017:48
sdakethanks - let me take a look17:48
mw_sdake: thanks :)17:49
*** ducttape_ has joined #openstack-kolla17:50
sdakemw_ which version of kolla-ansible?17:50
mw_4.0.017:50
*** mnasiadka has quit IRC17:51
*** schwicht has quit IRC17:53
*** jbrooks has quit IRC17:54
*** egonzalez has quit IRC17:56
*** mnasiadka has joined #openstack-kolla17:56
*** robellison has joined #openstack-kolla17:57
*** jbrooks has joined #openstack-kolla17:58
sdakemw_ here is the bug where the change was made which is producing the issue you see: https://bugs.launchpad.net/kolla/+bug/163316917:59
openstackLaunchpad bug 1633169 in kolla "Memcache connection limit too low" [Medium,Fix released] - Assigned to Mathias Ewald (mewald)17:59
*** schwicht has joined #openstack-kolla18:00
sdakemw_ here is the review: https://review.openstack.org/#/c/386105/18:00
sdakemw_ i'll be back in a bit - have meeting18:00
mw_sdake: How did that pass tests if it didn't work. Ok. ttys.18:01
sdakemw_ it did - it isn't anymore18:01
sdakedocker --version please?18:01
*** mnasiadka has quit IRC18:01
mw_Ah18:01
mw_sdake: Docker version 17.06.0-ce, build 02c1d8718:01
sdakemw_ would you try running 1.12.6 of docker18:02
mw_I can't right now, but I will get to it. Yeah.18:02
*** egonzalez has joined #openstack-kolla18:04
sdakeegonzalez can you please take a look at the scrollback re mw_'s bug18:05
*** mnasiadka has joined #openstack-kolla18:06
*** gfidente has quit IRC18:07
*** eaguilar has quit IRC18:08
*** egonzalez has quit IRC18:08
*** mnasiadka has quit IRC18:11
*** eaguilar has joined #openstack-kolla18:12
*** kiennt has joined #openstack-kolla18:13
*** mnasiadka has joined #openstack-kolla18:15
sdakemw_ you will need docker-py 1.8.1 as well18:15
vhosakotinc0: is there a plan for any kolla workshops/demos for ops at ptg?  Don't know if ptg has the time for workshops to users/ops.18:16
mw_sdake: I have 1.10.6 now. Too new?18:16
kolla-slack<inc0> It doesn't, also not really an audience18:16
*** tosika has joined #openstack-kolla18:22
*** retr0h has quit IRC18:23
*** vcn[m] has quit IRC18:23
*** bachp has quit IRC18:23
*** nhlfr has quit IRC18:23
*** mnasiadka has quit IRC18:25
*** vcn[m] has joined #openstack-kolla18:29
*** nhlfr has joined #openstack-kolla18:34
*** bachp has joined #openstack-kolla18:34
*** retr0h has joined #openstack-kolla18:34
*** kiennt has quit IRC18:36
mw_inc0: If I'd like to make a change to a deployment (like enabling DVR in globals.yml), how does one go about changing and deploying that?18:38
mw_Is there a recommended method?18:38
iniazisdake, sorry, stepped away for brunch18:42
iniazisdake: yeah i can change the dependencies etc.  not sure what the effect would be on my deployment...18:43
*** klindgren__ has joined #openstack-kolla18:45
*** klindgren_ has quit IRC18:48
inc0I'm back18:58
inc0mw_: soo, not sure about dvr (as it might require some steps, I don't know)18:58
inc0but in general running kolla-ansible deploy should be enough18:59
inc0if it's jsut config change, run kolla-ansible reconfigure18:59
sdakei need some brunch18:59
sdakehere comes the champagne and orange juice!18:59
mw_inc0: You mean make the change in globals and run kolla-ansible reconfigure?18:59
mw_That's it?18:59
sdakemw_ 1.10.6 is good, and your running docker-py 1.8.1 - and everything works for you?19:00
inc0mw_: again, depends on change. Enabling DVR would change network architecture19:00
inc0not sure if just reconfigure would be sufficient19:00
inc0reconfigure is meant for config file changes19:00
sdakereconfigure is 100% guranteed to work for config values in /etc/kolla/config - that is all its guranteed to work for19:01
sdakeanything else - may or may not work19:01
mw_Enabling DVR means I'd need new containers on each compute node.19:01
mw_Which means I'd need a deploy19:01
mw_I'm assuming....19:01
inc0right19:01
sdakeinc0 sadly reconfigure seemed like such a good name19:01
inc0what I'm worried about is existing routers and stuff19:01
sdakeinc0 cache invalidation and naming problems...19:01
inc0I don't know how they'll handle move to DVR19:02
inc0and off by one errors19:02
sdakethose are easy to solve inc0 :)19:02
SamYapleinc0: routers have keys. legacy, l3ha, dvr19:02
SamYaplea router doesnt convert19:02
sdakemw_ can you confirm or deny the change in docker versions worked please19:02
inc0SamYaple: what about l3 agents?19:03
mw_I can't change docker on all my hosts right now19:03
mw_sdake: Sorry...19:03
SamYapleinc0: l3 agents do not have specific keys. they can handle them all19:03
kolla-slack<mnasiadka> routers will stay fine, but still mw_ is right - enabling dvr will deploy metadata and l3 agents on computes, so he needs a deploy19:03
SamYaplemw_: update your inventory to put l3 router on all of your compute nodes19:03
*** ducttape_ has quit IRC19:03
SamYaplemw_: once it is deployed you can create DVR routers19:03
sdakemw_ ok - so how did you get past the fact that memcached doesn't work?19:03
SamYapleim pretty sure you dont need any neutron.conf changes for this to work19:03
mw_sdake: I changed the value from 5000 to the default. That's a bug imo though.19:04
mw_SamYaple: I should be able to change the dvr value in globals and be all set after a deployy?19:04
kolla-slack<mnasiadka> Well, enabling dvr is a change to ml2.conf19:05
inc0mw_: (reading scrollback) there are no sure users for users in containers19:05
*** manheim has joined #openstack-kolla19:05
inc0to exec into container use docker exec -it -u root << cont name >> bash19:05
SamYaplemnasiadka right. i forgot. you have to update the 'agent_mode' to dvr19:06
inc0enable_dvr in globals will do it for you19:06
*** manheim has quit IRC19:06
kolla-slack<mnasiadka> yup, it will - so change it, fire off deploy (luckily with neutron tag) and should work19:07
mw_I don't need to change my inventory as well, do I?19:07
mw_Just globals?19:07
sdakeinc0 it sounds like 4.0.z does not work with docker 1719:07
SamYaplemw_: no you need the inventory change and the globals.yaml value update19:08
mw_SamYaple: What's the inventory change?19:08
* inc0 looking for dvr docs19:08
mw_add the compute node to the [network] section?19:08
* inc0 thinks we really need to clean up our docs19:08
SamYapleinc0: twas wrong. agent_mode controls what it can do. if its dvr_snat it can do all the routers, just dvr is compute only19:08
sdakeinc0 when did you come to that conclusion?  5 summits ago? :)19:08
SamYaplemw_: you need to deploy neutron-l3-agent to the compute nodes19:09
*** ducttape_ has joined #openstack-kolla19:09
mw_SamYaple: Riight. Metadata too.19:09
SamYapleyup19:09
inc0fun, it seems we don't have dedicated dvr docs -.-19:10
mw_So...19:10
SamYaplenot dhcp, unless you want it to chew through all your ip addresses19:10
inc0damn19:10
*** eaguilar has quit IRC19:10
mw_No, not DHCP haha19:10
mw_SamYaple: [neutron:children] network19:10
mw_That stays the same?19:10
mw_[neutron-l3-agent:children] neutron19:10
mw_This is what changes?19:10
SamYaplemw_: yea that look right19:10
inc0SamYaple: you don't need to hange inventory19:10
SamYaplejust put compute in that group19:10
SamYapleinc0: you sure?19:10
inc0https://github.com/openstack/kolla-ansible/blob/d6db2569bb521f7295f03afe279b407161cfacbc/ansible/roles/neutron/defaults/main.yml#L8819:10
inc0you do need to specify enable_neutron_dvr: "yes" in globals.yml19:11
kolla-slack<mnasiadka> enable_neutron_dvr deploys metadata and l3 agents on hosts in compute group (in addition to network nodes when dvr is not set)19:11
SamYaplecool19:11
mw_Ok. So. One last check here haha19:11
kolla-slack<mnasiadka> Yeah, inc0 pasted file I’m just looking at19:11
mw_enable_neutron_dvr = yes19:12
inc0and fire off deploy19:12
mw_kolla-ansible deploy -i MYFILE neutron19:12
mw_(Can I specify neutron only like that????)19:12
inc0with a bit of lucj it'll just work (tm)19:12
inc0-t neutron if you want tags19:12
*** ddyer has joined #openstack-kolla19:12
inc0kolla-ansible deploy -i MYFILE -t neutron19:13
mw_Ah. Thanks :)19:13
*** ducttap__ has joined #openstack-kolla19:15
*** ducttape_ has quit IRC19:16
kolla-slack<mnasiadka> I think he needs more than bit of luck :slightly_smiling_face:19:16
mw_haha this doesn't help my confidence here guys19:17
iniazisdake: so what did you want me try?19:18
*** ntpttr_laptop has quit IRC19:18
inc0mw_: truth be told I never ran non-DVR -> DVR19:20
inc0reconfigure19:20
inc0or know anyone who did19:20
*** ducttap__ has quit IRC19:20
inc0so if you run into problems I'll do my best to help, but can't guarantee how it works19:21
mw_inc0: Sounds good. Thanks :)19:23
inc0np, let us know how it went19:23
mw_Will do19:25
*** lpetrut has joined #openstack-kolla19:25
sdakeiniazi well - i tried buidling 4.0.0 imags19:27
sdakeand ceph is broken19:27
sdakewhich raises the question, how did mw_ get 4.0.0 images built19:27
sdakemw_ would you mind sharing?19:27
*** ducttape_ has joined #openstack-kolla19:29
inc0latest centos source for ocata build in gates is from 7-2619:30
inc0http://tarballs.openstack.org/kolla/images/centos-source-registry-ocata.tar.gz19:30
mw_sdake: easy. I didn't build ceph.19:30
sdakemw_ you built toolbox?19:31
mw_yyup19:31
*** vhosakot has quit IRC19:31
*** vhosakot has joined #openstack-kolla19:32
iniazisdake: well I'm also using external ceph19:32
mw_actually... how can I check what got built?19:32
mw_Once I got the dependency hell sorted out I just ran kolla-ansible pull then deployed19:33
iniazimw_: oh missed your message; i'm using centos 7.319:34
*** jtriley has quit IRC19:34
sdakeinc0 does that build all images or just the gate profile images?19:34
inc0gate profile, but it's pretty complete list19:35
sdakefar from complete19:35
kolla-slack<egonzalez> Build jobs all images, gate profile is only for ansible deploy jobs19:35
inc0ah right egonzalez19:36
*** eaguilar has joined #openstack-kolla19:36
*** vhosakot has quit IRC19:43
*** vhosakot has joined #openstack-kolla19:44
*** ntpttr_laptop has joined #openstack-kolla19:45
jamesbensoninc0, FYI: I think I figured out what's wrong with deploying ceph locally.  It appears I have some disks that have major latency issues with issuing parted/sgdisk commands.  I discovered this when doing it manually.  So at least when I try to deploy a single node ceph cluster on that machines, it fails, if I switch to a machine that doesn't have the latency issues, it works.19:49
*** eaguilar has quit IRC19:49
jamesbensonI haven't scaled up yet to do all drives that don't have latency issues, but it seems that was the issue.  The servers detect the drives as healthy as well, so maybe early detection of failure....19:50
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Test of kolla-kubernetes gate  https://review.openstack.org/49321419:50
*** egonzalez has joined #openstack-kolla19:51
inc0jamesbenson: so that's strange tho, for me mons are proble19:51
inc0m19:51
inc0maybe we had 2 separate issues?19:51
jamesbensonperhaps, and they just looked similar?  I have a script for deploying ceph manually if you want me to share it with you...19:52
jamesbensonit requires a little bit of manual input, but makes it easier, I can get external ceph up in about 30 minutes.19:52
inc0nah I'll play along kolla19:52
jamesbensonat least for me, deploying ceph manually gave me some more insight into ceph itself, so that's nice :-)19:53
kfox1111afternoon.19:53
sdakeinc0 - it looks to me like 4.0.2 does not build at all for centos - can you verify19:53
inc0hey Kevin19:53
kfox1111finally tracked down the hep problem.19:53
inc0sdake: I don't have centos handy, but I'll try on buntu19:54
jamesbensonI built 4.0.2 with centos as a base not too long ago, a few weeks without issue19:54
kfox1111was seeing massive performance issues from vm's. some were getting 200kbits/s transfer out of the cloud.19:54
sdakeubuntu isn't what doesn't build19:54
jamesbensonhost was ubuntu19:54
kfox1111but getting gigabit between vm's in the tenant.19:54
kfox1111darndest thing.19:54
inc0sdake: host ubuntu but build centos19:54
kfox1111hypervisor-hypervisor and hypervisor-network node traffic was 10+gigabit.19:55
sdakeinc0 sounds good19:55
kfox1111turns out it was a bad kernel. :/19:55
jamesbensonkfox1111, ubuntu's latest kernel dropped support for our 10G... :-(19:55
kfox1111bummer.19:56
jamesbensonkfox1111, have you seen this? https://www.cyberciti.biz/cloud-computing/increase-your-linux-server-internet-speed-with-tcp-bbr-congestion-control/19:56
egonzalezsdake what images are failing to build?19:56
sdakeegonzalez most of them19:57
sdakeegonzalez pip install kolla==4.0.219:57
jamesbensonnot that it would fix your issue, but interesting nonetheless19:57
*** rwsu has quit IRC19:57
kfox1111jamesbenson: no, I hadn't. thanks for the link. :)19:58
jamesbensonI was going to try it on our dev cluster to see if it helps at all... :-)19:58
sdakeegonzalez https://da.gd/S3lYe -> https://paste.fedoraproject.org/paste/RgjRHu2nnfL-KRMH-V4WBw/19:58
*** nhlfr has quit IRC20:01
*** vcn[m] has quit IRC20:01
*** bachp has quit IRC20:01
*** retr0h has quit IRC20:01
*** lpetrut has quit IRC20:01
*** lpetrut has joined #openstack-kolla20:02
*** vcn[m] has joined #openstack-kolla20:04
*** pc_m has quit IRC20:04
*** pc_m has joined #openstack-kolla20:04
*** bachp has joined #openstack-kolla20:08
*** nhlfr has joined #openstack-kolla20:08
*** retr0h has joined #openstack-kolla20:08
*** goldyfruit has quit IRC20:12
sdakekfox1111 which kernel?20:12
*** slagle has quit IRC20:15
sdakegetting a whole lot of this: INFO:kolla.image.build.ceilometer-notification:https://mirrors.kernel.org/fedora-epel/7/x86_64/repodata/repomd.xml: [Errno -1] repomd.xml does not match metalink for epel20:15
sdaketalkingto centos folks - say its how mirroring works (or doesn't) in epel ;)20:18
*** magicboiz has quit IRC20:19
kfox11113.10.0-514.26.2.el7.x86_64 fixed our issues.20:20
kfox11113.10.0-514.2.2.el7.x86_64 really really bad.20:21
*** unicell has joined #openstack-kolla20:34
inc0sdake: images built, fluentd had issues20:41
*** itlinux has quit IRC20:46
*** rhallisey has quit IRC20:54
*** tosika has quit IRC20:55
*** jistr is now known as jistr|off21:02
mw_inc0: which version?21:08
inc0mw_: 4.0.221:09
mw_inc0: What's the error?21:09
mw_I saw an error with that one last night21:09
*** egonzalez has quit IRC21:11
*** dave-mccowan has quit IRC21:12
*** vhosakot has quit IRC21:20
*** vhosakot has joined #openstack-kolla21:20
*** mgoddard has quit IRC21:29
*** jamesbenson_ has quit IRC21:32
mw_inc0: ?21:34
*** vhosakot_ has joined #openstack-kolla21:36
*** vhosakot has quit IRC21:38
*** ntpttr_laptop has quit IRC21:40
*** jamesbenson_ has joined #openstack-kolla21:42
sdakeinc0 cool - i guess  the mirrors were doing their sync thing when my build was going21:50
sdakeso fleuntd is busted then21:51
*** jamesbenson has quit IRC21:51
*** vhosakot_ has quit IRC22:02
*** vhosakot has joined #openstack-kolla22:02
*** stefan_bo has joined #openstack-kolla22:09
*** stefan_bo has quit IRC22:14
*** unicell has quit IRC22:15
*** ntpttr_laptop has joined #openstack-kolla22:16
*** ducttape_ has quit IRC22:18
*** unicell has joined #openstack-kolla22:20
*** ducttape_ has joined #openstack-kolla22:21
*** lpetrut has quit IRC22:23
*** vhosakot has quit IRC22:32
*** schwicht has quit IRC22:35
*** kbyrne has joined #openstack-kolla22:44
*** schwicht has joined #openstack-kolla22:58
*** goldyfruit has joined #openstack-kolla23:07
*** unicell has quit IRC23:09
*** eaguilar has joined #openstack-kolla23:10
*** hongbin has quit IRC23:12
*** pbourke has quit IRC23:13
*** pbourke has joined #openstack-kolla23:15
*** clayton has quit IRC23:18
*** clayton has joined #openstack-kolla23:18
*** schwicht has quit IRC23:21
*** krtaylor has quit IRC23:21
*** goldyfruit has quit IRC23:29
*** vhosakot has joined #openstack-kolla23:33
*** manheim has joined #openstack-kolla23:39
*** schwicht has joined #openstack-kolla23:50
*** goldyfruit has joined #openstack-kolla23:52
*** goldyfruit has quit IRC23:54
*** goldyfruit has joined #openstack-kolla23:57
*** goldyfruit has quit IRC23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!