Monday, 2017-06-12

*** manheim has quit IRC00:01
*** randomhack has joined #openstack-kolla00:05
*** pradk has quit IRC00:09
openstackgerritMerged openstack/kolla-ansible stable/ocata: Add makestep in chrony.conf  https://review.openstack.org/47213400:31
*** randomhack has quit IRC00:39
*** randomhack has joined #openstack-kolla00:40
*** randomhack has quit IRC00:44
*** randomhack has joined #openstack-kolla00:58
*** salv-orlando has joined #openstack-kolla01:01
*** tovin07_ has joined #openstack-kolla01:02
*** cuongnv has joined #openstack-kolla01:02
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Remove keystone-paste.ini file in kolla  https://review.openstack.org/46253501:03
*** salv-orlando has quit IRC01:05
*** salv-orlando has joined #openstack-kolla01:13
*** salv-orlando has quit IRC01:15
*** salv-orlando has joined #openstack-kolla01:15
*** sbezverk has joined #openstack-kolla01:17
*** salv-orlando has quit IRC01:19
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Use unified vars in task  https://review.openstack.org/47316601:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Flatten neutron_* variables  https://review.openstack.org/47316801:26
sbezverkJeffrey4l: ping01:27
Jeffrey4lsup sbezverk01:27
sbezverkJeffrey4l are you on mac?01:27
Jeffrey4lsbezverk, on. linux ( archlinux )01:27
Jeffrey4lon*01:27
Jeffrey4lsbezverk, is kolla-k8s still using kolla-ansible's genconfig feature?01:28
sbezverkJeffrey4l thanks, I do not have "git review" on mac01:28
Jeffrey4lsbezverk, found this https://www.mediawiki.org/wiki/Gerrit/git-review#Mac_OS_X  :)01:29
sbezverkJeffrey4l : nope, inc0 build parallel version of configuration01:29
sbezverkJeffrey4l thanks for this link01:29
Jeffrey4lsbezverk, could u show me the codes?01:29
*** ssurana has joined #openstack-kolla01:30
sbezverkJeffrey4l : sure,  https://github.com/openstack/kolla-kubernetes/tree/master/ansible01:30
Jeffrey4lthen kolla-ansible could remove all thing related to k8s?01:31
Jeffrey4llike this "  host_in_groups: "{{ True if orchestration_engine == 'KUBERNETES' else inventory_hostname in groups['neutron-dhcp-agent'] }}"01:31
Jeffrey4l"01:31
Jeffrey4lsbezverk, right?01:33
sbezverkJeffrey4l : yes, I think so01:34
Jeffrey4lcool. thanks.01:34
Jeffrey4lduonghq, around?01:36
duonghqJeffrey4l, I'm here01:36
Jeffrey4lanything wanna to talk about https://review.openstack.org/#/c/452067/ duonghq01:37
Jeffrey4li saw you added it into next meeting agenda.01:37
duonghqyes, just want to know if we bump the Ansbile version in P01:37
duonghqif in this case, we should have plan to upgrade our kolla-ansible logic01:38
Jeffrey4lshould i -1 for my patch, in case it is merged before meeting.01:38
duonghqit's depend on you :P01:38
Jeffrey4lgot, thanks.01:39
duonghqnp, thanks01:39
Jeffrey4lbtw, i think we have to fix all WARNING in current kolla-ansible once the bump ansible version patch is merged.  saw lots of warning in CI.01:41
*** ssurana has left #openstack-kolla01:44
*** zhubingbing has joined #openstack-kolla01:50
*** ducttape_ has quit IRC01:50
*** ducttape_ has joined #openstack-kolla01:51
*** xinliang has quit IRC01:51
*** ducttap__ has joined #openstack-kolla01:52
*** ducttape_ has quit IRC01:55
*** zhurong has joined #openstack-kolla01:56
*** manheim has joined #openstack-kolla02:01
*** xinliang has joined #openstack-kolla02:04
*** xinliang has quit IRC02:04
*** xinliang has joined #openstack-kolla02:04
*** manheim has quit IRC02:06
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Use unified vars in task  https://review.openstack.org/47316602:06
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: Adding fallback to default keystone port  https://review.openstack.org/47317902:07
*** ducttap__ has quit IRC02:08
*** ducttape_ has joined #openstack-kolla02:09
*** ducttape_ has quit IRC02:13
openstackgerritjimmygc proposed openstack/kolla master: Support VMware DVS  https://review.openstack.org/45927302:19
*** zhuzeyu has joined #openstack-kolla02:21
*** itlinux_ has joined #openstack-kolla02:25
*** itlinux has joined #openstack-kolla02:25
*** ducttape_ has joined #openstack-kolla02:26
*** kiennt26_ has quit IRC02:26
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible  https://review.openstack.org/45927002:27
openstackgerritjimmygc proposed openstack/kolla master: Support VMware DVS  https://review.openstack.org/45927302:28
openstackgerritYotaro Konishi proposed openstack/kolla-ansible master: Fix service_type of Octavia  https://review.openstack.org/47318302:29
*** zhubingbing has quit IRC02:34
*** randomhack has quit IRC02:37
*** ducttape_ has quit IRC02:37
*** itlinux_ has quit IRC02:37
*** itlinux has quit IRC02:37
*** ducttape_ has joined #openstack-kolla02:38
*** ducttape_ has quit IRC02:42
*** yingjun_1 has joined #openstack-kolla02:44
openstackgerritYotaro Konishi proposed openstack/kolla-ansible master: Fix service_type of Octavia  https://review.openstack.org/47318502:44
*** jascott1 has joined #openstack-kolla02:45
*** jascott1 has quit IRC02:50
*** zhubingbing has joined #openstack-kolla03:02
*** tonanhngo has joined #openstack-kolla03:05
*** tonanhngo has quit IRC03:06
*** salv-orlando has joined #openstack-kolla03:16
*** salv-orlando has quit IRC03:20
*** caowei has joined #openstack-kolla03:27
*** randomha1k has joined #openstack-kolla03:33
*** dave-mccowan has quit IRC03:56
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Add timeout parameter for precheck tasks  https://review.openstack.org/47320503:58
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Rewrite haproxy precheck conditions to not use string interpolation  https://review.openstack.org/47320604:00
*** trozet has joined #openstack-kolla04:02
*** zhubingbing has quit IRC04:05
*** rwsu has joined #openstack-kolla04:06
*** yingjun_1 has quit IRC04:23
*** zhurong has quit IRC04:26
*** trozet has quit IRC04:26
*** eaguilar has quit IRC04:31
*** zhurong has joined #openstack-kolla04:35
openstackgerritTuan Luong-Anh proposed openstack/kolla master: Remove unused function set_status  https://review.openstack.org/47321104:38
*** skramaja has joined #openstack-kolla04:38
*** yingjun_1 has joined #openstack-kolla04:44
*** spsurya has joined #openstack-kolla05:01
spsuryamorning all05:01
spsuryasup duonghq05:01
*** salv-orlando has joined #openstack-kolla05:05
*** zhurong has quit IRC05:06
*** zhurong has joined #openstack-kolla05:07
*** rstarmer has joined #openstack-kolla05:12
*** trozet has joined #openstack-kolla05:38
*** omenv has joined #openstack-kolla05:39
*** janki has joined #openstack-kolla05:56
*** denaitre has quit IRC05:58
*** jascott1 has joined #openstack-kolla06:05
*** zhuzeyu has quit IRC06:09
*** mrunge has quit IRC06:16
*** mrunge has joined #openstack-kolla06:19
*** salv-orlando has quit IRC06:19
*** salv-orlando has joined #openstack-kolla06:20
*** sbezverk has quit IRC06:21
*** salv-orlando has quit IRC06:24
*** sbezverk has joined #openstack-kolla06:32
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: Adding fallback to default keystone port  https://review.openstack.org/47317906:34
*** yingjun_1 has quit IRC06:34
*** yingjun has joined #openstack-kolla06:43
*** sbezverk has quit IRC06:49
*** pcaruana has joined #openstack-kolla06:52
openstackgerritMerged openstack/kolla-ansible master: Fix service_type of Octavia  https://review.openstack.org/47318506:53
*** skramaja is now known as skramaja_afk07:02
*** shardy has joined #openstack-kolla07:03
openstackgerritMerged openstack/kolla-ansible master: Nova oslo_messaging_notifications config error  https://review.openstack.org/44450407:07
openstackgerritMerged openstack/kolla-ansible master: Neutron oslo_messaging_notifications config error  https://review.openstack.org/44450607:07
*** magicboiz has joined #openstack-kolla07:09
*** salv-orlando has joined #openstack-kolla07:11
*** magicboiz has quit IRC07:14
*** magicboiz has joined #openstack-kolla07:14
*** omenv has quit IRC07:17
*** manheim has joined #openstack-kolla07:17
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware DVS support to kolla-ansible  https://review.openstack.org/45927007:21
*** lpetrut has joined #openstack-kolla07:23
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vSphere support to nova_compute  https://review.openstack.org/44952907:24
*** karimb has joined #openstack-kolla07:30
*** trozet has quit IRC07:30
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add VMware NSXV support to neutron  https://review.openstack.org/45009207:30
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Change Solume keystone v2 to v3  https://review.openstack.org/47293107:33
*** zhubingbing_ has joined #openstack-kolla07:33
zhubingbing_ping Bertrand Lallau07:33
openstackgerritJuan J. Martínez proposed openstack/kolla-ansible master: Update Vagrant dev environment  https://review.openstack.org/47028007:33
reidracmorning; +w on https://review.openstack.org/#/c/470270/ anyone? :)07:34
*** unicell1 has joined #openstack-kolla07:38
*** ducttape_ has joined #openstack-kolla07:38
*** unicell has quit IRC07:39
*** mrunge_ has joined #openstack-kolla07:39
*** mrunge has quit IRC07:40
*** ducttape_ has quit IRC07:43
*** numans_ has joined #openstack-kolla07:45
*** salv-orlando has quit IRC07:47
*** numans has quit IRC07:48
*** salv-orlando has joined #openstack-kolla07:48
*** egonzalez has joined #openstack-kolla08:02
*** daidv has quit IRC08:03
*** sbezverk has joined #openstack-kolla08:04
*** karimb has quit IRC08:12
*** blallau has joined #openstack-kolla08:13
*** jaosorior has joined #openstack-kolla08:16
*** daidv has joined #openstack-kolla08:19
*** rstarmer has quit IRC08:27
*** mgoddard has joined #openstack-kolla08:28
*** trozet has joined #openstack-kolla08:37
manheimhi08:43
*** pbourke has quit IRC08:44
*** pbourke has joined #openstack-kolla08:45
manheimi have an issue deploying the keystone chart for kolla-k8s08:47
manheimRunning command: '/usr/sbin/httpd -DFOREGROUND'08:47
manheim/usr/local/bin/kolla_start: line 24: /usr/sbin/httpd: No such file or directory08:47
manheimthey other keystone pods are running08:48
manheimthis is keystone-1902127961-2gldb08:48
sbezverkmanheim kubectl get pod -n kolla | grep keystone08:50
manheimhttps://paste.ubuntu.com/24840040/08:51
manheimubuntu 16.0408:51
manheimalso, at the forst deploy time none of the keystone pods worked, they started working after a reboot, same for mariadb08:52
*** karimb has joined #openstack-kolla08:52
*** athomas has joined #openstack-kolla08:52
sbezverkmanheim how did you start it? iit does not look right at all. you have keystone bootstrap things running08:52
manheimi followed the docs08:53
manheimhelm install --debug kolla-kubernetes/helm/service/keystone --namespace kolla --name keystone --values ./cloud.yaml08:53
sbezverkmanheim keystone-manage-db-vt7q4 means it tries to prepare keystone db08:54
manheimok, so it should go away after it finishes, right?08:54
sbezverkit should not happen, it should be running for a couple of minutes and then go away08:54
manheimaha08:54
manheimwell, it throws the httpd command not existing08:55
manheimcould it be the docker image?08:55
manheimit's the first time working with kolla-k8s so I'm kinda in the dark08:56
sbezverkmanheim kubectl get pods -n kolla08:56
manheimhttps://paste.ubuntu.com/24840055/08:56
sbezverkmanheim : helm delete keystone --purge08:58
sbezverkhelm install helm/service/keystone-cleanup. -- you need to add here your cloud08:59
sbezverkand then reinstall keystone09:00
openstackgerritMerged openstack/kolla-ansible master: Fix sed replacement command in Ubuntu's install_docker  https://review.openstack.org/47027009:00
reidracthanks sdake! \o/09:01
manheimwill do09:01
manheimthanks09:01
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Remove port-security in gates  https://review.openstack.org/47332109:03
openstackgerritMerged openstack/kolla-kubernetes master: Allow overriding images for ironic pods with multiple containers  https://review.openstack.org/46948309:04
sdakesbezverk did you see https://review.openstack.org/#/c/472349/109:06
manheimlol, even keystone delete enters crashloop state09:07
sbezverksdake: not yet, looking at it..09:08
*** zhubingbing_ has quit IRC09:08
*** zhubingbing_ has joined #openstack-kolla09:09
*** rmart04 has joined #openstack-kolla09:17
manheimhmm, I get connection refused from the DB09:18
*** jascott1 has quit IRC09:19
*** jascott1 has joined #openstack-kolla09:19
sbezverkmanheim : probably your mariadb does not function properly09:20
sbezverktry to test it by using mysql tool09:20
openstackgerritMerged openstack/kolla master: Add "Vitrage" to "Openstack services" in README.rst  https://review.openstack.org/46322809:21
*** karimb has quit IRC09:21
*** karimb has joined #openstack-kolla09:21
*** jascott1 has quit IRC09:24
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix kolla-toolbox build  https://review.openstack.org/47333209:26
sdakenothing like waking up to $1300 tube blown09:29
* sdake growls09:29
openstackgerritMerged openstack/kolla-ansible master: Add the ceph keyring files related task condition  https://review.openstack.org/45841709:32
openstackgerritMerged openstack/kolla-ansible master: Remove symlink from fluentd global conf  https://review.openstack.org/45879109:32
*** trozet has quit IRC09:35
*** wagiel has joined #openstack-kolla09:36
openstackgerritjimmygc proposed openstack/kolla-ansible master: Make glance filesystem_store_datadir configurable.  https://review.openstack.org/45209409:36
*** skramaja_afk is now known as skramaja09:37
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Implement blazar Docker images  https://review.openstack.org/45474509:38
openstackgerritjimmygc proposed openstack/kolla-ansible master: Make glance filesystem_store_datadir configurable.  https://review.openstack.org/45209409:41
openstackgerritjimmygc proposed openstack/kolla-ansible master: Make glance filesystem_store_datadir configurable.  https://review.openstack.org/45209409:42
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Change zun rewrite rule number in fluentd  https://review.openstack.org/47334109:44
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement blazar ansible role  https://review.openstack.org/45475209:46
*** tvignaud has quit IRC09:46
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add VMware NSXV support to neutron  https://review.openstack.org/45009209:51
openstackgerritMerged openstack/kolla-ansible master: Add default_boot_option configuration  https://review.openstack.org/47211209:55
openstackgerritMerged openstack/kolla-ansible master: Remove deprecated options designate  https://review.openstack.org/45071509:57
*** magicboiz has quit IRC09:58
*** david-lyle has quit IRC10:02
*** tvignaud has joined #openstack-kolla10:04
*** daidv has quit IRC10:12
*** zhurong has quit IRC10:12
*** magicboiz has joined #openstack-kolla10:14
*** duonghq has quit IRC10:16
*** magicboiz has quit IRC10:18
*** magicboiz has joined #openstack-kolla10:18
*** wagiel has quit IRC10:19
*** tovin07_ has quit IRC10:20
*** david-lyle has joined #openstack-kolla10:24
*** wagiel has joined #openstack-kolla10:25
egonzalezcan I have reviews https://review.openstack.org/#/c/473332/ , kolla-build is broken in master and stable branchs due missing make package for rpm distros10:26
egonzalez*kolla-toolbox10:26
*** daidv has joined #openstack-kolla10:26
reidracegonzalez: just fixed this in my SUSE environment; PyNaCl now requires make for libsodium10:30
reidracI can confirm that adding GNU make to the installed packages fixes the problem10:30
egonzalezreidrac, yep10:30
reidracah, oh; that's the actual text of the commit hah10:31
egonzalezreidrac, the change add make to rpm distros, .deb distros already have make as part of build-essential package10:33
reidracyes, exacly what happens with SUSE10:35
egonzalezreidrac, are you working with suse in kolla?10:36
reidracthis sounds a bit like the issue with setuptools; perhaps pip installed dependencies would be best pinned to a specific version10:36
reidracegonzalez: yes, I'm woring on it; got the dev environment with vagrant + keystone and dependencies up an running; for now10:37
egonzalezreidrac, cool10:37
reidracall running openSUSE 42.2 but fluentd, because td-agent doesn't support SUSE10:37
reidracbut that container can be run with CentOS and is fine; until I get to fix that10:38
*** MarginHu has joined #openstack-kolla10:38
*** sbhojpur has joined #openstack-kolla10:39
*** zhurong has joined #openstack-kolla10:39
openstackgerritjimmygc proposed openstack/kolla-ansible master: Fix fwaas options  https://review.openstack.org/46388210:40
reidracwhen this is merged: https://review.openstack.org/#/c/470280/ ; I'll start pushing some of the SUSE bits, at least the dev environment that is self contained10:40
*** Yumeng has joined #openstack-kolla10:41
sbhojpurNeed help with Kolla Magnum installation10:43
sbhojpurSomehow my setup is stuck in cluster creation till it times out10:44
sbhojpurHeat engine logs says 'creating HeatWaitCondition "master_wait_condition" Stack "test_ashish-kmdzvrbp5byu-kube_masters-dbjvn7oxpd7q-0-aagbote25v4t" [747d1952-bd43-4dab-b06d-e915e903d9d7] '10:46
*** caowei has quit IRC10:49
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Change ubuntu ODL repo to ppa's  https://review.openstack.org/47335910:51
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Change ubuntu ODL repo to ppa's  https://review.openstack.org/47335910:52
*** yingjun has quit IRC10:54
egonzalezreidrac, there you go :)10:56
reidracegonzalez: cheers! :)10:57
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix "Enable influxdb datasource" random error  https://review.openstack.org/47297710:57
openstackgerritMerged openstack/kolla-ansible master: Update Vagrant dev environment  https://review.openstack.org/47028010:59
*** wagiel has quit IRC11:00
egonzalezreidrac, btw, dont need to wait for dependant changes to merge, if adds Depends-On: <CHANGE_ID> in commit message, dependant change will be used in gates11:00
reidracegonzalez: OK, thanks for the tip11:00
*** sbezverk has quit IRC11:05
*** cuongnv has quit IRC11:08
*** sbezverk has joined #openstack-kolla11:16
*** jascott1 has joined #openstack-kolla11:20
*** jascott1 has quit IRC11:24
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: Do not merge. Review test  https://review.openstack.org/47337811:25
manheimhow can I delete keystone-cleanup?11:31
*** pradk has joined #openstack-kolla11:31
manheimdoes not work w helm delete11:33
*** randomha1k has quit IRC11:36
*** ducttape_ has joined #openstack-kolla11:39
sbezverkmanheim  you should be able to delete any helm deployment with  helm delete --purge11:40
sbezverkhelm ls11:40
manheimif i do helm list I don;t see it11:41
manheimnote: it's in crashloop state11:41
sbezverkmanheim it should not matter11:42
sbezverkmanheim kubectl get job -n kolla | grep keystone11:42
manheimError: Unable to lock release keystone-delete-db: release not found11:43
*** ducttape_ has quit IRC11:43
sbezverkmanheim kubectl get job -n kolla | grep keystone | awk '{print "kubectl delete job -n kolla "$1}' | sh -l11:44
sbezverkthat should clean up all the jobs11:44
sbezverkyou also must make sure mariadb is working11:45
sbezverkand if the name resolution working too, if keystone cannot locate mariadb service it will be very upset and will start crash loop11:46
manheimresolution should woek11:56
manheimwork11:57
*** ducttape_ has joined #openstack-kolla11:59
manheimif I reinstall the DB pod, the main container from mariadb-0 does not start12:00
manheimError from server (BadRequest): container "main" in pod "mariadb-0" is waiting to start: PodInitializing12:01
egonzalezanyone using kuryr with kolla?12:02
egonzalezas far I can see, only kuryr api is implemented, no backend at all(libnetwork or k8s)12:03
sbezverkmanheim that is the clear indication that there is problem with your setup12:03
manheimhmm, I see, makes sense12:04
manheimtho i followed the docs12:04
sbezverkare you using aio?12:04
manheimyes12:06
manheimin a ubuntu VM12:06
manheimthing is after a reboot the pods start correctly12:07
*** ducttape_ has quit IRC12:11
*** ducttape_ has joined #openstack-kolla12:11
openstackgerritMerged openstack/kolla-ansible master: Change zun rewrite rule number in fluentd  https://review.openstack.org/47334112:12
*** ducttape_ has quit IRC12:16
*** zhurong has quit IRC12:16
*** mrunge_ is now known as mrunge12:17
openstackgerritBertrand Lallau proposed openstack/kolla-ansible stable/ocata: Fix Telegraf container restarting issue  https://review.openstack.org/46403912:18
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Change Solum keystone v2 to v3  https://review.openstack.org/47293112:19
*** sbhojpur has quit IRC12:22
*** pomac_ is now known as pomac12:22
*** hrw has quit IRC12:24
*** karimb has quit IRC12:26
*** schwicht has joined #openstack-kolla12:28
*** hrw has joined #openstack-kolla12:29
openstackgerritAndy Smith proposed openstack/kolla-ansible master: Add support for hybrid messaging backends  https://review.openstack.org/46896612:30
sdakemanheim if you delete the pod related to mariadb you ahve to remove all backoding data the pod created12:32
*** janki has quit IRC12:33
*** schwicht has quit IRC12:34
*** numans has joined #openstack-kolla12:36
*** numans_ has quit IRC12:39
*** dave-mccowan has joined #openstack-kolla12:39
*** omenv has joined #openstack-kolla12:40
manheimand that is done with helm delete --purge, right?12:41
sdakeno12:41
*** omenv has quit IRC12:43
*** omenv has joined #openstack-kolla12:43
bmaltaisHi, trying to deploy to a new node this morning and I get "The requested image does not exist: registry.ssclab.com:5000/kolla/centos-binary-fluentd:5.0.0"12:48
bmaltaisI did a git pull this morning and notices a lot of changes12:49
bmaltaisWas something introduced that would break the deploy based on a non existand fluentd 5.0.0 pre-built container?12:49
*** schwicht has joined #openstack-kolla12:49
*** karimb has joined #openstack-kolla12:50
*** schwicht has quit IRC12:52
*** ducttape_ has joined #openstack-kolla12:52
bmaltaisSetting "openstack_release: "4.0.0"" has fixed the issue12:54
egonzalezbmaltais, kolla 5.0.0 (master) was tagged on friday12:54
bmaltaisegonzalez: Are pre-build containers for it available? Is this why it fail?12:55
bmaltaisegonzalez: Or do I have to run an upgrade on my setup?12:55
egonzalezbmaltais, nope, containers are master images12:56
egonzalezbut if you not set openstack_release at globals it will try to use 5.0.0 due pbr versioning12:56
bmaltaisegonzalez: I never built my containers... so forcing 4.0.0 is probably safer for now?12:56
egonzalezbmaltais, yep12:57
*** ducttape_ has quit IRC12:57
bmaltaisegonzalez: OK. I am in the process of converting docker to btrfs... I have some strange issues withlvm I think12:57
MarginHu I have used btrfs for 2 weeks   as docker backend , no issue happened13:00
*** rwallner has joined #openstack-kolla13:00
manheimhow can I remove all the data mariadb pod created?13:01
sbezverkmanheim normally volumes are created13:01
sbezverkat /var/lib/kolla/volumes13:02
manheimsudo rm -rf /var/lib/kolla/volumes/*13:02
manheimgot it13:02
sbezverkyep13:02
*** randomhack has joined #openstack-kolla13:06
*** ducttape_ has joined #openstack-kolla13:08
*** ducttap__ has joined #openstack-kolla13:10
*** schwicht has joined #openstack-kolla13:11
*** rwallner has quit IRC13:12
*** rwallner has joined #openstack-kolla13:12
*** ducttape_ has quit IRC13:12
reidracI'm having a weird issue deploying fluentd; ansible_eth0 variable not defined replacing the template for 01.syslog.conf; this is after api_interface_address generalisation13:14
*** eanylin has quit IRC13:15
*** ducttap__ has quit IRC13:16
reidracis like is not reading all.yml13:16
*** skramaja has quit IRC13:21
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Fix bootstrap-servers with invalid api_interface  https://review.openstack.org/43494013:25
*** ducttape_ has joined #openstack-kolla13:25
*** krtaylor has joined #openstack-kolla13:26
*** lucasxu has joined #openstack-kolla13:29
*** ducttape_ has quit IRC13:31
*** ducttape_ has joined #openstack-kolla13:33
*** tumble has joined #openstack-kolla13:35
*** randomhack has quit IRC13:35
*** eanylin has joined #openstack-kolla13:36
*** emccormick has joined #openstack-kolla13:42
*** karimb has quit IRC13:48
*** ducttape_ has quit IRC13:49
blallau@reidrac this is due to a Facts loading issue13:54
blallauit should be fix by https://review.openstack.org/#/c/472660/13:55
*** ducttape_ has joined #openstack-kolla13:57
*** ducttape_ has quit IRC13:58
*** ducttape_ has joined #openstack-kolla13:59
sdakesbezverk yup the macs rock14:02
sdakesbezverk compared to windows for dev atleast - mac blows pc away14:02
*** karimb has joined #openstack-kolla14:02
*** schwicht has quit IRC14:04
*** schwicht has joined #openstack-kolla14:05
*** ducttape_ has quit IRC14:05
reidracblallau: thanks for the pointer14:16
reidracblallau: yes! that was it, thanks!14:18
blallau@reidac you are welcome ;)14:18
*** ducttape_ has joined #openstack-kolla14:21
*** bmace has quit IRC14:25
*** jtriley has joined #openstack-kolla14:25
*** bmace has joined #openstack-kolla14:25
hrwsdake: mac blows pc or is it rather macos blows mswindows?14:26
sdakesomething blows tahts for sure14:27
sdakelike my 1300$ elrog 845 tube in my tube amp this mroning14:27
* sdake FURIOUS14:27
*** ducttape_ has quit IRC14:28
*** jamesbenson has joined #openstack-kolla14:34
*** sgrasley has joined #openstack-kolla14:37
*** sgrasley has left #openstack-kolla14:37
*** jtriley has quit IRC14:38
*** jtriley has joined #openstack-kolla14:38
*** eaguilar has joined #openstack-kolla14:39
openstackgerritChristian Berendt proposed openstack/kolla-ansible master: Use templates for keystone domain specific configurations  https://review.openstack.org/47348314:42
*** MarginHu has quit IRC14:43
*** ducttape_ has joined #openstack-kolla14:45
openstackgerritMathias Ewald proposed openstack/kolla-ansible master: Fix Horizon Static Files Path for Ubuntu  https://review.openstack.org/47348514:46
*** mewald has joined #openstack-kolla14:46
*** jtriley has quit IRC14:46
*** jtriley has joined #openstack-kolla14:50
*** srwilkers has joined #openstack-kolla14:50
reidracquestion: how much disk needs all-in-one deployment?14:52
reidracI wild guess would be OK; say... around 40GB?14:52
reidrac"official" openSUSE image is 20GB and I run out disk deploying keystone and deps14:53
*** jtriley has quit IRC14:57
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Add openvswitch and bridge-utils in kuryr  https://review.openstack.org/47349014:58
*** jtriley has joined #openstack-kolla14:58
*** hongbin has joined #openstack-kolla14:59
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Add openvswitch and bridge-utils in kuryr  https://review.openstack.org/47349015:01
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Add openvswitch and bridge-utils in kuryr  https://review.openstack.org/47349015:01
*** ducttape_ has quit IRC15:03
openstackgerritWayne Chan proposed openstack/kolla master: Optimize the link address  https://review.openstack.org/47349315:04
*** ducttape_ has joined #openstack-kolla15:04
*** jtriley has quit IRC15:05
*** dklyle has joined #openstack-kolla15:07
openstackgerritWayne Chan proposed openstack/kolla master: Use https links for documentation  https://review.openstack.org/47349315:07
*** jtriley has joined #openstack-kolla15:07
*** omenv has quit IRC15:07
*** omenv has joined #openstack-kolla15:07
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: WIP: fix kuryr  https://review.openstack.org/47349515:08
*** fooliouno has joined #openstack-kolla15:08
*** david-lyle has quit IRC15:10
openstackgerritAndy Smith proposed openstack/kolla-ansible master: Add qdrouterd role for messaging infrastructure component  https://review.openstack.org/46946215:16
egonzalezplease, review. kolla-toolbox fail to build in all rpm distros for stable and master branchs https://review.openstack.org/#/c/473332/15:16
*** blallau has quit IRC15:17
*** jtriley has quit IRC15:20
*** lpetrut has quit IRC15:21
*** srwilkers has quit IRC15:21
*** srwilkers has joined #openstack-kolla15:25
*** rmart04 has quit IRC15:29
sdakeegonzalez its still fialing many gates?15:30
egonzalezsdake, failed in ubuntu gates which are unrelated to the change. make package is a dependency of build-essential15:30
*** pcaruana has quit IRC15:31
*** eaguilar has quit IRC15:36
*** hieulq_ has joined #openstack-kolla15:37
inc0good morning15:37
*** vhosakot has joined #openstack-kolla15:38
*** eaguilar has joined #openstack-kolla15:38
*** goldyfruit has joined #openstack-kolla15:40
*** manheim has quit IRC15:40
kfox1111morning.15:41
sbezverkgood morning15:42
*** salv-orlando has quit IRC15:46
*** salv-orlando has joined #openstack-kolla15:47
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Update cinder sanity check to use built in shade  https://review.openstack.org/43792015:47
*** jtriley has joined #openstack-kolla15:47
*** iniazi has quit IRC15:48
*** krtaylor has quit IRC15:48
*** dklyle has quit IRC15:49
*** itlinux has joined #openstack-kolla15:49
*** itlinux_ has joined #openstack-kolla15:49
*** david-lyle has joined #openstack-kolla15:49
spsuryainc0: morning15:51
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement blazar ansible role  https://review.openstack.org/45475215:53
*** rhallisey has quit IRC16:00
*** unicell1 has quit IRC16:05
*** egonzalez has quit IRC16:06
*** krtaylor has joined #openstack-kolla16:07
*** itlinux_ has quit IRC16:07
*** itlinux has quit IRC16:07
inc0kfox1111: http://logs.openstack.org/15/466815/10/experimental/gate-kolla-ansible-dsvm-deploy-ceph-centos-source-centos-7-2-node-nv/6d7e75e/logs/docker_logs/bootstrap_osd_1 have you seen anythin like this fron your ceph in gates thing?16:07
*** itlinux has joined #openstack-kolla16:08
*** itlinux_ has joined #openstack-kolla16:08
kfox1111inc0: looking....16:10
inc0thanks16:10
kfox1111hmm... probably not relevent, but it looks odd to see only quorum members. you trying to launch a third there?16:11
kfox1111have not seen that problem before....16:11
inc0it's gonna be hard on 2 node gate16:11
*** daidv_ has joined #openstack-kolla16:11
kfox1111maybe zap the partition table before running?16:11
kfox1111yeah. but it won't hbe happy with an even number.16:11
kfox1111a single node ceph will be ok for testing.16:11
inc0meh16:12
kfox1111there's nothing really different between a single node and multi as far as openstack's concerned.16:12
inc0well, I can fix that16:12
sbezverkkfox1111: in your comment, you do not like me removing port from the workflow common config or falling back to 5000?16:12
inc0error doesn't seem to be related tho16:12
kfox1111sbezverk: oh. thought I was looking at all_values.yaml. which it should be in there?16:13
kfox1111yeah. removing it from common_workflow_config.sh is good.16:13
inc0ok, off to ceph debug I go16:13
kfox1111defaulting it I think shouldn't be needed as the values.yaml is the defaults.16:14
sbezverkkfox1111: unfortunately it does not default16:14
kfox1111inc0: I don't think they ever test 2 node ceph as its not a supported config.16:14
kfox1111sbezverk: is the search path not picking it up then?16:14
inc0I'll do 1 node16:14
sbezverkkfox1111: right, by some reason it does not find it..16:15
*** manheim has joined #openstack-kolla16:16
kfox1111sbezverk: oh. I see....16:18
kfox1111sbezverk: search for 5000 on all_values.yaml16:18
inc0I think this is sgdisk rather than ceph16:18
kfox1111its set on each package explicitly.16:18
kfox1111inc0: quite possible.16:18
inc0https://github.com/openstack/kolla/blob/master/docker/ceph/ceph-osd/extend_start.sh#L20-L2816:18
kfox1111inc0: hmm.... the journal dev is not configurable there...16:19
kfox1111maybe its using too much space for your volumes?16:19
inc0you can configure journal differently16:19
*** rwallner has quit IRC16:20
*** manheim has quit IRC16:20
*** karimb has quit IRC16:21
inc0kfox1111: plan to watch E3?16:21
kfox1111not sure.16:21
inc0new star wars looks interesting16:22
*** pbourke has quit IRC16:24
*** jascott1 has joined #openstack-kolla16:35
*** jascott1 has quit IRC16:36
*** eaguilar has quit IRC16:37
*** jascott1 has joined #openstack-kolla16:48
*** rwallner has joined #openstack-kolla16:50
vhosakotegonzalez: good work in https://review.openstack.org/#/c/473332/, I've reviewed it.  I saw the same issue related to pynacl when building.16:51
*** mewald has quit IRC16:53
sdakeits a good thing my tubes have a 1 year warranty16:53
sdakei am less  ragiing now16:53
sdakethe vendor grilled me "any power problems" "any fuse problems" "what type of wiring do you have" "blah blah blah"16:54
sdaketubes failing after 200 hours of use (for a 10k hr rated tube) is completely unacceptable16:54
*** hieulq_ has quit IRC16:55
*** iniazi has joined #openstack-kolla16:55
*** mgoddard has quit IRC16:57
*** rhallisey has joined #openstack-kolla16:59
*** salv-orl_ has joined #openstack-kolla17:02
vhosakotinc0: does stable/oacat also have multinode gates now? --> https://review.openstack.org/#/c/471401/17:04
*** salv-orlando has quit IRC17:04
inc0vhosakot: yes, but you need to run check experimental for that17:05
inc0https://review.openstack.org/#/c/471428/ until this merges17:06
*** rhallisey_ has joined #openstack-kolla17:10
*** rhallisey has quit IRC17:10
vhosakotinc0: cool, thought so..  I want to get started with upgrade gates... will sync with duonghq later today.17:10
inc0this needs to merge;) https://review.openstack.org/#/c/471428/17:11
inc0for upgrade gates too17:11
inc0it includes new gate - upgrade17:11
inc0just centos source for now17:11
vhosakotinc0: infra pre-work.. got it.. ncie work17:11
*** mgoddard has joined #openstack-kolla17:11
vhosakotsdake: sorry to hear about your tubes.. why do you use those vacuum tubes for?17:12
sdakelistening to toons17:12
openstackgerritMarcus Williams proposed openstack/kolla-ansible master: Add OpenDaylight role  https://review.openstack.org/41636717:13
vhosakotsdake: I see17:13
*** jtriley has quit IRC17:15
*** unicell has joined #openstack-kolla17:15
*** jtriley has joined #openstack-kolla17:22
*** lucasxu has quit IRC17:24
*** egonzalez has joined #openstack-kolla17:25
*** sambetts is now known as sambetts|afk17:27
*** itlinux_ has quit IRC17:33
*** itlinux has quit IRC17:33
*** itlinux has joined #openstack-kolla17:38
*** itlinux_ has joined #openstack-kolla17:38
inc0hey guys, I need sencond +2 on https://review.openstack.org/#/c/471428/ <- needed for gate17:40
*** jaosorior is now known as jaosorior_away17:41
*** eaguilar has joined #openstack-kolla17:42
inc0btw kfox1111 I figured out this GPU passthrough thing17:46
kfox1111yeah? sweet. :)17:47
inc0it turns out 1. intel skylake put my amd gpu and nvidia gpu in same iommu group17:47
inc0so I bought PCI extention riser and switched pci ports17:47
inc0for nvidia17:47
kfox1111wow. all 3 types of gpu's in one box? :)17:47
*** kolla-slack2 has quit IRC17:47
*** kolla-slack has joined #openstack-kolla17:47
inc0that split iommu groups but intell still puts it's audio driver on nvidia audio device17:47
kfox1111ah.17:47
inc0so I need to unbind it before I start vm17:48
inc0yeah I have all 3 types of them;)17:48
inc0next stop, make them work together on same tensorflow computation17:48
inc0that's a strech tho:P17:48
kfox1111helm install tensorflow. :)17:48
kfox1111https://github.com/kubernetes/charts/tree/master/incubator/tensorflow-inception17:49
inc0right now I'm still using vagrant because I actually need pci passthrough17:49
inc0..because of all 3 gpus in my box17:49
*** daidv_ has quit IRC17:49
inc0I need that to make a vm with only one type;)17:49
kfox1111hmm.... ok.17:50
inc0installing tensorflow is easy17:50
inc0making it work with gpu is harder17:50
kfox1111yeah. I could believe that.17:50
inc0it was fun ride, but got it working and yesterday made my first gpu based computatuon17:51
inc0I was damn happy;)17:51
kfox1111yeah. the gpu passthrough stuff can be really tricky. :/17:52
kfox1111figured with all the virutlaization push over the years, they would have cracked that problem a lot sooner.17:52
inc0well problem is quite limited to few use cases17:53
kfox1111there's a whole indestry around virtual desktops.17:54
*** eaguilar_ has joined #openstack-kolla18:01
inc0kfox1111: well fwiw virtualbox was super easy to pci passthrough18:02
inc0reason I dropped virtualbox is because nvidia drivers hates pci passthrough18:02
*** eaguilar has quit IRC18:04
*** tonanhngo has joined #openstack-kolla18:05
*** tonanhngo has quit IRC18:05
*** tonanhngo has joined #openstack-kolla18:06
*** lucasxu has joined #openstack-kolla18:08
*** Manheim has joined #openstack-kolla18:09
*** rhallisey has joined #openstack-kolla18:10
*** manheim_ has joined #openstack-kolla18:12
*** rhallisey_ has quit IRC18:13
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multi-scenerio gates  https://review.openstack.org/46681518:19
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable deploying ceph on loopback devices  https://review.openstack.org/47277118:19
*** cinerama` is now known as cinerama18:24
*** manheim_ has quit IRC18:32
*** erlon has joined #openstack-kolla18:38
*** mgoddard has quit IRC18:40
*** eaguilar_ has quit IRC18:43
*** bjolo has quit IRC18:52
*** bjolo has joined #openstack-kolla18:53
Manheiminc0, did you have a change to take a look on my latest patchset?18:53
inc0https://review.openstack.org/#/c/455684/ gates are red18:53
inc0;)18:53
inc0let me do recheck18:54
Manheimok18:55
*** vhosakot has quit IRC19:04
*** itlinux_ has quit IRC19:09
*** itlinux has quit IRC19:09
*** ducttape_ has quit IRC19:21
*** itlinux_ has joined #openstack-kolla19:21
*** ducttape_ has joined #openstack-kolla19:21
*** itlinux has joined #openstack-kolla19:21
*** itlinux has quit IRC19:22
*** itlinux_ has quit IRC19:22
*** mgoddard has joined #openstack-kolla19:29
Manheimthey failed again :(19:30
*** eaguilar has joined #openstack-kolla19:33
*** mgoddard has quit IRC19:35
*** itlinux has joined #openstack-kolla19:37
*** itlinux_ has joined #openstack-kolla19:37
kfox1111inc0: yeah, its not very hard to do the passthrough in kvm either. but its making it actually work, thats the ugly. ;)19:39
dave-mccowankfox1111 i'm looking at the haproxy support for openstack services in kolla-kubernetes.  i see it is there for nova/cinder/glance, but not horizon and keystone.  do you know if is there any reason why?19:46
kfox1111yeah.19:46
kfox1111horizon and keystone both embed nicely in apache in kolla <= 4.19:47
kfox1111while the rest are more often done python stand alone.19:47
kfox1111its a goal of pike I think to support all of them in apache.19:47
*** salv-orl_ has quit IRC19:55
*** salv-orlando has joined #openstack-kolla19:55
*** salv-orlando has quit IRC19:59
*** shardy has quit IRC20:07
inc0kfox1111: I made progress with ceph thing, so, ever seen this? http://logs.openstack.org/15/466815/11/experimental/gate-kolla-ansible-dsvm-deploy-ceph-centos-source-centos-7-2-node-nv/92345a1/logs/kolla/ceph/ceph-osd.0.log20:12
sbezverkkfox1111: when do we plan to use parent grouping? the reason I am asking is if it is soon, then no reason to merge this ps, but if it is in "near" future, maybe it is worth to merge it., thought?20:16
*** nea1 has joined #openstack-kolla20:17
nea1FAILED! => {"failed": true, "msg": "The conditional check ''{{ hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] }}' not in '{{ item.stdout }}'' failed. The error was: Invalid conditional detected: EOL while scanning string literal (<unknown>, line 1)\n\nThe error appears to have been in20:17
nea1'/home/openstack/.venv/share/kolla-ansible/ansible/roles/rabbitmq/tasks/precheck.yml': line 69, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- fail: msg=\"Hostname has to resolve to IP address of api_interface\"\n  ^ here\n"}20:17
nea1anyone with an idea what could be wrong?20:18
inc0nea1: which task?20:18
inc0ahh I see20:18
nea1i did a debug output of both variables, and they seam fine20:18
inc0Hostname has to resolve to IP address of api_interface20:18
nea1yep, it does20:18
*** rwallner has quit IRC20:19
inc0all controllers has to resolve each other20:19
nea1debug of item.stdout ""msg": "172.31.15.1     STREAM rz15n1\n172.31.15.1     DGRAM  \n172.31.15.1     RAW    ""20:19
inc0using api_interface address20:19
nea1debug of hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] "msg": "172.31.15.1"20:19
inc0all of control nodes can resolve all other using api_interface?20:20
inc0hostname -> api interface address20:20
inc0all control nodes to all other20:21
nea1its only one node20:21
*** MarginHu has joined #openstack-kolla20:21
*** MarginHu has quit IRC20:21
inc0so it's aio?20:21
*** MarginHu has joined #openstack-kolla20:21
inc0all in one20:21
nea1yep, but i'm using a modified hosts file20:22
nea1as i'm not runing it as root20:22
inc0modified how? mind pasting it?20:22
nea1not runing kolla-ansible as root20:22
nea1ansible_user=root and with the ip20:23
nea1[network]20:23
nea1172.31.15.1 ansible_user=root20:23
inc0/etc/hosts20:23
inc0plz20:23
nea1# BEGIN ANSIBLE GENERATED HOSTS20:24
nea1172.31.15.1 rz15n120:24
nea1# END ANSIBLE GENERATED HOSTS20:24
inc0aaand rz15n1 doesn't resolve to anything else?20:25
nea1in the lines above only localhost and some ipv6 hosts like ip6-allhosts20:25
inc0as in, ping rz15n1 will show 172...?20:25
nea1yep20:25
nea1thats what the test above does20:25
inc0and there is only one node in [control] ?20:25
nea1and the test is working but the fail: when: part is failing20:26
inc0yeah, and fails if there is something wrong with hostnames20:26
nea1all the same20:26
nea1well then the test should fail not the "The conditional check [..."20:26
inc0also, you're deploying on node you run ansible from?20:26
nea1yep, but ansible is installed in a venv under the user openstack, not as root20:27
inc0that's fine20:27
inc0plz add debug with {{ hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] }}'20:28
nea122:19:52            nea1 | debug of hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] "msg": "172.31.15.1"20:29
inc0hmm, which ansible version?20:29
*** emccormick has quit IRC20:30
nea1ansible==2.3.1.020:30
inc0try rollinb back to 2.2.*20:30
inc0if that helps, I guess we have bug20:30
nea1yep, that fixed it20:31
*** manheim_ has joined #openstack-kolla20:32
*** jtriley has quit IRC20:33
*** salv-orlando has joined #openstack-kolla20:36
*** manheim_ has quit IRC20:37
egonzalezbackport was merged for that issue, will be available in the next ocata release20:38
inc0right20:39
inc0speaking of which20:39
inc0Jeffrey4l: when you wake up :) ^20:39
egonzalezinc0 can you +w, will  need backport befora tagging  https://review.openstack.org/#/c/473332/20:39
inc0done egonzalez20:40
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/ocata: Fix kolla-toolbox build  https://review.openstack.org/47358020:44
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/ocata: Fix kolla-toolbox build  https://review.openstack.org/47358020:45
*** emccormick has joined #openstack-kolla20:47
*** eaguilar has quit IRC20:51
jascott1kfox1111 ping20:53
*** iniazi has quit IRC21:00
*** eaguilar has joined #openstack-kolla21:00
*** harlowja has quit IRC21:02
openstackgerritBradley Jones proposed openstack/kolla-kubernetes master: (WIP) Deploy playbook using microservices  https://review.openstack.org/47358821:02
*** rwsu has quit IRC21:03
*** rhallisey has quit IRC21:07
*** itlinux_ has quit IRC21:08
*** itlinux has quit IRC21:08
*** iniazi has joined #openstack-kolla21:08
*** lucasxu has quit IRC21:12
*** eaguilar has quit IRC21:18
sdakeinc0 kfox1111 https://review.openstack.org/#/c/473588/121:22
inc0sdake: soooo....that needs a lot of work21:22
sdakeagreed21:22
inc0biggest issue we have (and reason we need entrypoint) is that we need to wait till things finish21:23
sdakethat is why gerrit dudes invented the comment button21:23
inc0and that's biggest and most important thing to build here21:23
inc0we need design discussion21:23
sdaketell brad where the sync points on21:23
inc0not just review21:23
sdakeon/are21:24
inc0bradjones, sdake can we have webex/zoom/hangouts session about that?21:24
sdakeinc0 bradjones is in uk timezone21:24
sdakeinc0 i htink morning time shoudl work21:24
inc0wed before meeting?21:25
inc0or tomorrow around meeting time?21:25
inc016UTC that is21:25
sdakemight as well schedule it i guess21:25
sdakeif you could leave comments in the meantime - would appreciate it21:25
inc0sdake: would you do me a favor and schedule it? since you have your cisco webex thingy21:25
sdakeyup21:26
inc0thank you good sir21:26
inc0there are few points to consider21:26
sdakecan ou leave comments in the meantime21:26
sdakebradjones doesn't understand why entrypoint is there today (the synchronization points)21:26
sdakebradjones comments will help him understand that21:26
inc01. how to put it into kolla-toolbox (or rather sub container)21:26
inc02. Create wait for tasks in every required spot21:27
inc0I'd focus on mariadb+keystone role first as it has to be first and it is quite complex in terms of orchiestration21:28
inc0we could drop few microservices too, things like registering users21:28
inc0since we'd have kolla-toolbox, we might as well use it21:29
sdakewe need the users registered21:29
inc0yes, with ansible21:29
inc0we have all the plumbing21:29
sdakemicrocharts read cloud.yaml21:29
sdakeour plumbing does not21:29
inc0we'll run orch within container21:30
inc0we can modify tasks, but general idea is still sound21:30
sdakeright21:30
inc0depends how we want to do it21:30
inc0that's why we need design discussion21:30
sdakesee how smple that is - add in some sync points - wrap in a container - wrap container in a helm chart - bam HELM NATIVE helm install openstack21:32
inc0devil's in the details tho21:32
inc0I like idea of ansible driving k8s, I said that before21:33
inc0and I was proponent for ansible-based operators, which this kinda is21:33
inc0or however you call it21:33
sdakek8s-controller is the nerd-term21:33
sdakean operator combines several controllers into one21:34
inc0yeah, controller21:34
openstackgerritMerged openstack/kolla master: Fix kolla-toolbox build  https://review.openstack.org/47333221:38
sdakekfox1111 if your alive - please review https://review.openstack.org/#/c/473588/121:39
*** jamesbenson has quit IRC21:39
*** itlinux_ has joined #openstack-kolla21:44
*** itlinux has joined #openstack-kolla21:45
sbezverksdake: do you have PoC of this solution somewhere?21:46
hawii do have question regarding designate: is it safe to deploy it to existing environment? i mean, it needs to restart some other containers as well?21:46
sdakesbezverk no - i asked brad to write the code about a week ago21:46
sdakethat is what has written21:47
sdakehe says it works for him21:47
sbezverksdake: I would think before that massive coding, there should be PoC or it might be a waste of time21:47
sdakealthough as is, it looks racy21:47
sdakethere is no massive coding needed21:47
sbezverksdake: I have hard time to see it working unless some specialized  kubernetes aware library is used.21:48
sdakethe one thing that patch is missing is sync points21:48
sbezverkI mean ansible specialized library, just curling will not cut it..21:48
sdakeyou mean for the sync points?21:49
sdakekubectl can tell you when a job is finished, no?21:49
openstackgerritDan Ardelean proposed openstack/kolla-ansible master: Add Hyper-V role  https://review.openstack.org/45568421:49
Manheimthink i know where the issue was21:50
Manheimi'll see if the passes this time21:50
sbezverksdake: of course not21:50
egonzalezhawi what you mean by restart other containers?21:50
sbezverkyou just start it21:50
sdakeand check for its exit code21:50
sdakein a loop21:51
sbezverkand then you get control back right away21:51
sdakeyou use kubectl to get the exit code of the job in a loop - when its done its done21:51
sbezverkjob might be hanging or crashing21:51
sbezverketc21:51
egonzalezhawi yeah, will need to restart neutron containers21:51
sdakeif a job crashes, the deployment is cratered21:52
sdakeif a job hangs, we can set timeouts on the loop21:52
egonzalezhawi designate will enable port_security extension in neutron, saw issues enabling it in existing networks21:52
hawiegonzalez: i see, thanks. will try to find some safe time for enabling it.21:52
sdakein either case, the deployment is cratered - as with the entyrpoint solution21:53
egonzalezhawi what I saw is that existing networks not get dhcp, just a--disable-dhcp and--enable-dhcp on each subnet fixes the issue21:53
*** srwilkers has quit IRC21:54
hawii do have already live setup, had some issues finding proper solution for dns and outside recursers pointing to designate, it's almost the last chance to apply it :(21:54
sbezverksdake: you are starting to build infra which is already exist in entrypoint, why reinvent the wheel?21:54
sdakeentrypoint is a dead upstream21:54
sdakebecause of its governance structure, even if i wanted ot, as a cisco employee I cannot contribute to it21:55
sbezverkit does what it supposed to do and does it well21:55
sdakejust like I can't contribute to ansible itself21:55
sdakedoes it well is debateable21:55
sdakewe cannot base our solution on a dead upstream21:55
sbezverkif you are not happy with macos thing, you do not throu yourself in re-writing it right?21:55
sdakecoreos has abandoned stackanetes21:56
sbezverksdake: other than baseless comments you cannot show any problem with entrypoint.. I am sorry but at least me I have not seen any issue after services piece got fixed.21:56
sdakewit hthis model, we can do upgrades easily21:56
sdakehow is "DEAD UPSTREAM" baseless?21:56
openstackgerritEduardo Gonzalez proposed openstack/kolla stable/ocata: Fix kolla deploy gates  https://review.openstack.org/47360521:57
sbezverksdake:  where does it day that?21:57
sbezverksorry I meant say that21:57
sbezverksdake: I heard it so far only from you21:58
sdakeinc0 is entrypoint upstream dead?21:58
inc0sdake: dead upstream can be fixed by simple fork21:58
sdakeinc0 i cannot contribute ot that because of the attorneys.....21:58
*** jamesbenson has joined #openstack-kolla21:58
inc0ok, but let's assume we get code entrypoint like that will not be dead21:59
sdakeright - so rewrite entrypoint21:59
sdakeor altenately, ust use this21:59
inc0yeah21:59
inc0both are hard tbh21:59
sdakewriting sync points in this ansible code is trivially easy21:59
inc0don't underestimate this22:00
inc0especially if we get to a world of dying ansible containers that needs to be restarted22:00
sdakewe dont have to write a genreal purpose tool22:00
sdake(which entyrpoint is)22:00
inc0it's good idea, but by no means easy22:00
sdakewe have to write speicfic sycn points for specific things22:00
sdakewhich is easy22:00
inc0but we already wrote sync points for entrypoint22:00
inc0we just need entrypoitn that is alive22:01
sbezverksdake: if it is that easy as you say how come you have not showed that in all its beauty ;)22:01
sdakethe sync points dont manage upgrades22:01
sdakesbezverk its patchset #122:01
sdakewritten in 2 hours22:01
sdakei'm not really sure what you expect - miracles? :)22:01
inc0upgrades can be orchiestrated by ansible22:01
inc0we can do all of that22:01
sbezverksdake: but it is just lines without any logic in them ;)22:02
inc0sdake: right, hard part are still to come22:02
inc0right now it's equivalent of shellscript22:02
sbezverkinc0: in shell we have waiting subroutines22:02
sdakegranted using the mysql module is a little ocmpicated in ansible22:02
sbezverkhere there are none so far..22:02
inc0in ansible we can have them too22:02
sdakegranted writing  a loop in ansible is hard waiting on a kubectl output22:02
inc0sbezverk: so idea in general is good22:03
*** jamesbenson has quit IRC22:03
inc0I like this, I have really good experiences with ansible driving k8s22:03
sdakethose are the two things we ahve t odo22:03
*** salv-orlando has quit IRC22:03
sdakenot write a genreal purpose orchestration replacement for the dead upstream entyrpoint22:03
inc0sdake: no it's not22:03
inc0we can actually write ansible module that will wait for k8s output22:03
*** salv-orlando has joined #openstack-kolla22:04
sdakethat could be done - seems like overkill -but wfm :)22:04
sbezverkinc0: maybe but when it comes for complex logic, I am not sure if ansible will be better than entrypoint calling k8s api functions to get status of k8s objects22:04
inc0sbezverk: it's easy22:04
inc0hard part is what's not in k8s22:05
sbezverkinc0: listening to you guys, everything is easy ;) until it gets hard :)22:05
sdakesbezverk w hae a team of people that are ansile experts22:05
*** jamesbenson has joined #openstack-kolla22:05
sdakeeverything is easy when you can ask an expert :)22:05
inc0sbezverk: it's not bad, entrypount is really really simple thing22:06
*** harlowja has joined #openstack-kolla22:06
inc0ansible won't be any more difficult22:06
sbezverkinc0: I see it as advatage22:06
sdakesbezverk try upgrading with entyrpoint.22:06
inc0sbezverk: limitations of entrypoint are issye tho22:06
inc0sdake is right about upgrades22:06
inc0they're non-trivial22:06
inc0possible, but non-trivial22:07
sbezverkinc0: that part I have not seen it yet..22:07
sdakethis really isn't about deployment - any money could write that - this is about upgrades22:07
*** schwicht has quit IRC22:07
*** salv-orlando has quit IRC22:08
sdakeand with helm containing a container containint hte ansible, helm native baby- all the way22:08
inc0sdake: how about we focus with portdirect and srwilkers on writing brave new entrypoint22:08
sdakethis microchart ansible code is 2 weeks out max22:09
inc0and make ansible upgrade mechanism22:09
sdakenew entrypoint - who knows - several months22:09
inc0sdake: uhh...that's unlikely.22:09
sdakewith no real advantage22:09
*** dave-mccowan has quit IRC22:09
inc0let me rephase it - ansible *will not be easy*22:09
*** jamesbenson has quit IRC22:09
inc0it looks easy on paper, but kolla-ansible took us a year to make22:10
sdakeeasier than an entrypoint that is upgradable22:10
sdakeupgrades mut happen in a certain order22:10
sdakeentrypoint has no ordering22:10
inc0ofc it has22:10
*** spsurya has quit IRC22:10
inc0dependencies determine order22:10
inc0ordering is one and only reason entrypoint exists22:10
sdakeentrypoint is an orchestration tool22:11
sdakewhich is more powerful, entrypoint or ansible?22:11
inc0orchiestration is glorified word for dependnecy management22:11
*** Manheim has quit IRC22:11
kolla-slack<srwilkers> Oh hello22:13
sdakegive it 2 weeks22:13
sdakewill be done working and gating22:13
*** rwsu has joined #openstack-kolla22:13
sdakewhat more could you ask for?22:13
inc0sdake: right, I'll be happy to see that;)22:13
inc0srwilkers, we need to move spaceship at full speed ahead22:14
sbezverksdake: as a generic tool ansible is more powerfull, but we do not need all its generic beauty, we need lean and mean dependency thing22:14
sdakesbezverk this stuff needs to be upgradeable22:14
sdakeentyrpoitn will never be lean and mean with ugprades in mind22:14
sbezverkswilkers: if you have something to share in term of development, please let me know..22:15
*** fooliouno has quit IRC22:15
SamYaplejgriffith: youre all good. im going to add you to the core group for loci22:16
kolla-slack<srwilkers> Yeah. Driving currently so hard to respond. Will be in front of a keyboard in 3022:16
*** manheim has joined #openstack-kolla22:19
sdakewe have atleast 3 workflows to handle - reconfigure, ugprade, deploy22:19
sdakeentrypoint could probably be made to do these things22:19
sdakeansible can do them faster22:20
sdakei have tech support duty for my parents, gotta jet - review the patch - be constructive - keep an open mind22:21
sdakesbezverk also keep in mind ansible -> container -> helm -> HELM NATIVE22:22
sbezverksdake: a bit strange sequence, in your case ansible drives helm and then helm launches releases.22:23
sdakesbezverk nah - ansible goes into a container - container goes into helm - helm native22:23
sdakei htink you misunderstood me22:23
portdirecto/22:23
inc0sbezverk: helm creates job based of kolla-toolbox, kolla-toolbox runs ansible play that builds rest of resrouces22:23
inc0hey portdirect22:23
* portdirect reading scrollback :)22:23
inc0we're having orchiestration discussion and 2 competing ideas22:24
portdirecti pick the 3rd one22:24
* portdirect argues point bravely22:24
sbezverkportdirect: why am I not surprised ;)22:24
portdirectwhat are they?22:24
inc0portdirect: you're in luck, it exists22:24
* portdirect regrets joke again22:25
inc01. entrypoint rewritten in spaceship22:25
inc02. ansible in container driving rest of helm22:25
sdakesadly 2 only works with microcharts inc022:25
inc03. https://github.com/helm/rudder-appcontroller this thing22:25
sdakewhich is good, because we aren't shipping spaceship we are shipping kolla-kubernetes ;)22:25
inc0yeah, dependencies within same chart still have to be addressed22:26
portdirects/spaceship/spaceport :P22:26
inc0well22:26
inc0yeah22:27
inc0anyway22:27
inc0competing ideas22:27
portdirectok - so I'll need some more context, but (and I'm obviously biased) but my gut says 1, and 3 combined have most utility?22:27
manheimheh, it passed22:27
*** emccormick has quit IRC22:27
inc03 isn't something I'd put my money on22:28
inc0but that's my biases22:28
sbezverkinc0: that app controller is what Mirantis was pushing?22:28
inc0ansible is good in stuff like quorum building, ugprades22:28
inc0these won't be trivial without external (or semi external) orch tool22:28
inc0sbezverk: yuo22:28
inc0yup22:28
sbezverkinc0: I see, curious if anybody played with that thing..22:29
sdakepath of least resistance is to implement orchestration in an orchestration engine with a rockin upstream - which is ansible22:29
inc0sdake: again, it won't be as easy as you think, but feel free to prove me wrong22:30
inc0to me 1 and 2 are equally appealing22:30
*** egonzalez has quit IRC22:30
portdirectwhats missing from k8s entrypoint thats needed here22:31
inc0portdirect: stuff like making mariadb quorum22:31
portdirect(I have a list of things i want - but not sure if they are the same pain points.)22:31
inc0would be much easier22:31
portdirectah - so having the ability to work as a sidecar doing things like leader election?22:32
inc0yeah22:32
inc0upgrades too22:32
*** goldyfruit has quit IRC22:32
inc0that kind of fun22:32
inc0also ansible has a lot of utility built into it already22:33
inc0waiting for stuff and such22:33
portdirect++22:33
inc0I think some hybrid solution might actually be best22:33
portdirect(oh -- on the waiting, but ++ on the utility :)22:33
portdirect^^ yup, I think theres not gonna be a single tool that does (r perhaps should) fit the job here22:33
portdirectI'm a big fan of a lot of very simple tiny tools over a multi-tool22:34
portdirectand ansible def has a space there22:34
portdirecthows the ansible integration  with k8s these days?22:35
inc0not horrible22:35
inc0but definetly could be better22:35
inc0ansible is really good templating engine22:35
inc0I think my favorite solution would be: use dependency-manager (let's stop using word entrypoint plz)22:39
inc0for 90% of stuff when it's easy22:40
inc0and use ansible sidecar for 10% that is hard22:40
kolla-slack<srwilkers> Yes. Entry points been used generally in this discussion but it really comes down to dependency checking that makes sense22:40
inc0or ansible-driven jobs rather than sidecar22:40
inc0it's glorified sleep until deps are met22:40
inc0that's all it is22:40
kolla-slack<srwilkers> We need to distinguish between dependency checking and orchestration here22:40
*** n0isyn0i1e has joined #openstack-kolla22:42
inc0hey, I need to step out to do some grocery shopping to avoid anger of hungry and tired wife, can we continue discussion in like 30?22:42
inc0or tomorrow22:42
inc0bbiaf22:43
kolla-slack<srwilkers> Hey me too. I'm chatting while grocery shopping now22:43
*** n0isyn0ise has quit IRC22:45
*** rhallisey has joined #openstack-kolla22:49
*** itlinux has quit IRC22:50
*** itlinux_ has quit IRC22:50
inc0back23:10
*** manheim has quit IRC23:10
jascott1in black?23:12
inc0I was trying to reply with another rhyme, but only thing I could come up with was crack, and that wasn't greatest word to reply23:13
inc0I'd be horrible rapper23:13
jascott1most of them are23:13
jascott1:D23:13
inc0yeah, I guess, I'll keep it as backup career23:14
inc0guys need critical +2s on these two changes https://review.openstack.org/#/c/473580/ https://review.openstack.org/#/c/473605/123:16
inc0in this order23:16
*** schwicht has joined #openstack-kolla23:18
*** ducttape_ has quit IRC23:31
Daviey6.1006.8023:37
*** schwicht has quit IRC23:43
*** eaguilar has joined #openstack-kolla23:43
*** schwicht has joined #openstack-kolla23:45
*** schwicht has quit IRC23:47

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!