Friday, 2017-03-17

*** eaguilar has joined #openstack-kolla00:04
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155000:05
*** eaguilar_ has quit IRC00:06
*** yingjun has joined #openstack-kolla00:11
*** eaguilar has quit IRC00:13
*** sbezverk has quit IRC00:18
*** sbezverk has joined #openstack-kolla00:20
*** sbezverk has quit IRC00:24
*** Jeffrey4l has quit IRC00:25
*** gema has quit IRC00:27
*** gema has joined #openstack-kolla00:27
*** jrobinson has quit IRC00:27
*** jaesang has joined #openstack-kolla00:39
*** duonghq has joined #openstack-kolla00:47
duonghqmorning guys00:47
*** deadnull has quit IRC00:48
*** sbezverk has joined #openstack-kolla00:53
kfox1111duonghq: hi. :)00:53
duonghqevening kfox111100:54
*** cuongnv has joined #openstack-kolla01:00
*** MasterOfBugs has quit IRC01:01
hawihi. i do have question. ubuntu 16.04.2, Images that failed to build list is really long. what's wrong?01:01
*** jrobinson has joined #openstack-kolla01:03
*** tonanhngo has quit IRC01:04
*** sayantani01 has quit IRC01:07
*** daidv has joined #openstack-kolla01:09
duonghqhawi, can you post the log?01:11
duonghqnormally, one of base image failed01:11
duonghqlead to all other fail01:11
*** sbezverk has quit IRC01:12
*** sbezverk has joined #openstack-kolla01:12
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155001:13
*** tonanhngo has joined #openstack-kolla01:15
hawiduonghq: http://pastebin.com/eSkxAsSx01:16
duonghqhawi, your registry is not proper setup01:19
*** unicell has quit IRC01:19
*** tonanhngo has quit IRC01:19
hawiduonghq: thanks, will try to find out what's wrong with it.01:21
duonghqhawi, how do you setup this?01:21
*** unicell has joined #openstack-kolla01:21
*** schwicht has joined #openstack-kolla01:21
hawiduonghq: docker run -d -p 4000:5000 --restart=always --name registry registry:2 and i modified /etc/default/docker as well01:25
hawiduonghq: DOCKER_OPTS="--insecure-registry http://10.79.3.254:4000"01:26
duonghqhow about kolla-build.conf?01:27
*** vhosakot has joined #openstack-kolla01:28
hawiduonghq: it seems that i dont have it where i expected it to be01:28
*** zhubingbing has joined #openstack-kolla01:29
duonghqhawi, sorry, do not understand01:29
duonghqI see that your settings is applied01:29
duonghqbut it cannot connect to the registry01:29
duonghqyou can try push some images to the registry manually01:30
vhosakotsbezverk: thanks for your reply, checking IRC logs now. so, did you run the steps in these scripts manually to install kolla-k8s? https://github.com/openstack/kolla-kubernetes/blob/master/tools/build_dev_image.sh  and  https://github.com/openstack/kolla-kubernetes/blob/master/tools/run_dev_image.sh ?01:31
*** unicell has quit IRC01:31
hawiduonghq: thanks, will try to sort it out01:33
vhosakotsbezverk: again, I'll read your reply later in the IRC logs if I get disconnected when I'm eating dinner now :)01:38
*** schwicht has quit IRC01:44
*** schwicht_at_work has joined #openstack-kolla01:48
sbezverkvhosakot: yes but I modified them to match my environment01:48
vhosakotsbezverk: cool, thanks for the info, will run them now.01:49
*** Jeffrey4l has joined #openstack-kolla01:53
*** zioproto has quit IRC02:01
*** sayantani01 has joined #openstack-kolla02:06
*** schwicht_at_work has quit IRC02:12
*** Jeffrey4l has quit IRC02:14
*** Jeffrey4l has joined #openstack-kolla02:14
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155002:15
sbezverkkfox1111: another pain :( nova scheduler failing to contact placement.. but curling placement service returns correct response02:19
*** masber has joined #openstack-kolla02:26
*** cuongnv has quit IRC02:29
*** cuongnv has joined #openstack-kolla02:32
*** vhosakot has quit IRC02:32
*** vhosakot has joined #openstack-kolla02:32
sdakehey peeps02:34
duonghqhi sdake02:35
sdakesup peeps02:38
sdakesupdu02:38
sdakesup duonghq02:39
*** Jeffrey4l has quit IRC02:41
*** Jeffrey4l has joined #openstack-kolla02:41
*** yuanying has quit IRC03:03
*** jrobinson has quit IRC03:08
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix can't find /usr/lib/libCryptoki2_64.so in barbican  https://review.openstack.org/44456103:14
zhubingbinghttps://review.openstack.org/#/c/444561/03:14
zhubingbingwho please help me review it03:14
zhubingbing;)03:14
zhubingbingduonghq03:14
vhosakothi sdake.. since I ran into many vagrant issues, I reverse-engineered the Vagrfile's contents, setup the networking myself, booted four bare-metal VMs, and deployed a four-node (1 master, 3 workers) k8s cluster manually using kubeadm using https://kubernetes.io/docs/getting-started-guides/kubeadm/.   Here is my cluster -->  http://paste.openstack.org/show/603032/.03:16
vhosakotI'll now refer https://github.com/openstack/kolla-kubernetes/blob/master/tools/build_dev_image.sh  and  https://github.com/openstack/kolla-kubernetes/blob/master/tools/run_dev_image.sh, and deploy kolla-k8s on my k8s cluster.03:16
vhosakotthen, I'll help sbezverk and kfox1111 improve the docs so docs have all the steps for new contributors.03:17
*** rwsu has joined #openstack-kolla03:23
*** Margin has joined #openstack-kolla03:27
Marginhi, I met an issue ,https://bugs.launchpad.net/kolla/+bug/167365703:27
openstackLaunchpad bug 1673657 in kolla "port 6000 which is used by swift object-server isn't listening after deploy openstack with multi-node " [Undecided,New]03:27
*** lazyPwr has quit IRC03:29
*** zhugaoxiao has quit IRC03:29
*** zhugaoxiao has joined #openstack-kolla03:30
*** lazyPower has joined #openstack-kolla03:30
duonghqvhosakot, thank for you doc, does the ps is merged?03:37
vhosakotduonghq: no, I'll add the docs soon, I just got my cluster up :)03:38
duonghqvhosakot, roger03:38
sdakesbezverk i pushed 4.0.0 for centos source03:41
sdakevhosakot there is a doc available to shows how to deploy kolla-kubernetes using kubeadm03:44
*** vhosakot_ has joined #openstack-kolla03:44
sdakevhosakot vhosakot_ https://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP03:44
*** vhosakot has quit IRC03:46
vhosakot_sdake: I started with that, a lot of the steps have strikethrough. hence, did not know the exact steps/flow.  I was able to use the upstream docs for kubeadm and deploy 4-node cluster successfully - https://kubernetes.io/docs/getting-started-guides/kubeadm/.03:47
sdakevhosakot_ the non-strikethrough are the correct docs more or less03:47
sdakethey have been tested by 5 people+03:47
vhosakot_sdake: can I refer them to deploy kolla-k8s on my empty cluster?03:48
sdakei'd recommend testing them03:49
sdakeare you doing it today or tomorrow?03:49
sdakeif today - i have some time to do with you03:49
vhosakot_sdake: yes, I have a working empty cluster on which I'm deploying my test apps to learn k8s... I now need OpenStack (kolla-k8s) on my empty cluster.03:50
vhosakot_sdake: here is my empty cluster on which sbezverk says I can now start installing OpenStack (kolla-k8s) http://paste.openstack.org/show/603032/03:51
sdakevhosakot_ if your oding it now - i can help you via webex03:52
vhosakot_sdake: how long does it take?  does building take a lot of time.. I can spend few hours now, sure.. I wast kolla-k8s up to be able to contribute more to kolla-k8s project :)03:53
vhosakot_wast ==want*03:53
vhosakot_sdake: do you think my empty cluster at http://paste.openstack.org/show/603032/ is a good one to start with, or should I deploy k8s form scratch?03:54
sdakethats what i'd use03:54
sdakethe one you ahve deployed03:54
sdakefew hours max03:54
sdakehttp://cisco.webex.com/join/stdake03:55
vhosakot_cool, let me jump in then :) 2 mins :)03:55
vhosakot_I'm in :)  can you hear me03:58
duonghqsdake, I'm not sure why we softlink kolla-ansible to kolla04:01
duonghqIt's confuse from very beginning04:01
sdakeduonghq we shouldn't04:04
*** Margin has quit IRC04:04
sdakeits nanoying04:04
sdakevhosakot_ i can't hear you04:04
sdakevhosakot_ i think my network imploded - let me reocnnect04:04
vhosakot_I'm back04:04
*** goldyfruit has quit IRC04:08
*** skramaja has joined #openstack-kolla04:20
duonghqsdake, I'm not sure why we clone into kolla-ansible, then cd kolla/ansible?04:33
duonghqthere is not setup.py here04:33
*** lamt has joined #openstack-kolla04:36
*** unicell has joined #openstack-kolla04:41
*** sayantani01 has quit IRC04:52
*** sayantani01 has joined #openstack-kolla04:54
*** yuanying has joined #openstack-kolla04:56
*** unicell1 has joined #openstack-kolla05:11
*** unicell has quit IRC05:11
*** caowei has joined #openstack-kolla05:11
*** Margin has joined #openstack-kolla05:29
*** erlon has quit IRC05:35
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155005:42
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla master: Fix oslo_debug_helper not running  https://review.openstack.org/44439805:52
masberhi, I am testing kolla and i got this error --> http://paste.openstack.org/show/603070/06:02
masberwas wondering if someone could help me out finding out the issue06:02
*** Jeffrey4l has quit IRC06:02
masberthis paste here has more information --> http://paste.openstack.org/show/603071/06:04
*** jascott1- is now known as jascott106:06
*** sayantan_ has joined #openstack-kolla06:11
*** sayantani01 has quit IRC06:14
Marginmasber, I met your issue06:15
masberhow did you fixed it, I am checking on stackoverflow and looks like configuration parameters are missing?06:16
Marginhttps://bugs.launchpad.net/kolla/+bug/162633906:16
openstackLaunchpad bug 1626339 in kolla-ansible "Kolla mongodb error: can't find self in the replset config" [High,In progress] - Assigned to Jeffrey Zhang (jeffrey4l)06:16
Marginplease refer https://review.openstack.org/#/c/422377/06:16
masberok, will do06:16
*** Jeffrey4l has joined #openstack-kolla06:17
*** zhurong has joined #openstack-kolla06:21
sbezverkJeffrey4l: ping06:21
Jeffrey4lsbezverk, pong06:21
masberthis patch has been commited right?06:22
sbezverkJeffrey4l: to be able to start an instance with 4.0.0 images is it mandatory to have cell confiugred?06:22
Jeffrey4lsbezverk, yep.06:22
sbezverkJeffrey4l: :(06:22
Jeffrey4lotherwise, sync db will complain, iirc.06:22
masbersbezverk, ocata required cell configuration as far as I am aware of06:23
sbezverkJeffrey4l: what do I need for it?06:23
sbezverkJeffrey4l: is it just a sequence of commands? or it requires more?06:23
Jeffrey4lsbezverk, check https://docs.openstack.org/developer/nova/cells.html#fresh-install06:23
sbezverkJeffrey4l: thank you..06:24
Jeffrey4lsbezverk, simple_cell_setup is easier.06:24
sbezverkJeffrey4l: man, it is like anohter bootstrap process :(06:25
Jeffrey4lkolla-ansible is using simple_cell_setup command.06:26
Jeffrey4lduring deploy06:26
Jeffrey4l1. create nova_cell0 database and proper credential.  2. sync db as normal   3. run nova-manage cell_v2 simple_cell_setup06:26
sbezverkJeffrey4l: which image is used to run this command nova-api?06:26
Jeffrey4lyep.06:26
Jeffrey4l nova_api with nova.conf file.06:26
masberMargin, has this patch been applied yet?06:27
*** mpansky has joined #openstack-kolla06:27
*** saneax-_-|AFK is now known as saneax06:27
sbezverkJeffrey4l: I use nova-api for bootstrapping nova-api DB06:27
Marginoh, I don't know06:28
sbezverkcurious why it is not executeds as a part of normal bootstrap process?06:28
Jeffrey4lsbezverk, which?06:28
sbezverkJeffrey4l: we run 2 bootstrap jobs, 1 for db create/manage of nova db adn one for create/manage nova-api db06:29
Jeffrey4lwe need third database: nova_cell006:29
sbezverkJeffrey4l: so extended start script does not automatically execute simple_cell_setup06:30
sbezverkJeffrey4l: OMG another db :(06:30
Jeffrey4lsimple cell setup should run after all nova compute is up.06:30
*** satyar has joined #openstack-kolla06:31
sbezverkJeffrey4l: how do you tell nova-api image to run simple_cell_setup06:31
sbezverkif the same image is used for everything06:31
sbezverk?06:31
Jeffrey4ljust "docker exec nova_api nova-manage cell_v2 simple_cell_setup"06:31
sbezverkJeffrey4l: ok got it, thank you, will build a chart for it tomorrow..06:32
Jeffrey4lnp.06:32
*** mnasiadka has joined #openstack-kolla06:41
*** unicell1 has quit IRC07:04
*** unicell has joined #openstack-kolla07:04
*** sayantan_ has quit IRC07:15
*** mrhillsman has quit IRC07:20
*** caowei has quit IRC07:23
*** caowei has joined #openstack-kolla07:23
*** vhosakot_ has quit IRC07:25
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Update infra components list in README.rst  https://review.openstack.org/44687307:33
*** mrhillsman has joined #openstack-kolla07:34
*** manheim has joined #openstack-kolla07:35
*** manheim has quit IRC07:37
*** manheim has joined #openstack-kolla07:38
*** manheim has quit IRC07:38
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix can't find /usr/lib/libCryptoki2_64.so in barbican  https://review.openstack.org/44456107:38
*** l4yerffeJ_ has joined #openstack-kolla07:38
*** manheim has joined #openstack-kolla07:38
*** pcaruana has joined #openstack-kolla07:43
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Install kolla-ansible in develop mode in gate  https://review.openstack.org/44687707:49
*** matrohon has joined #openstack-kolla07:54
*** Jezekus has joined #openstack-kolla07:57
*** mpansky has quit IRC08:09
*** gfidente has joined #openstack-kolla08:11
*** zioproto has joined #openstack-kolla08:18
*** mgoddard has joined #openstack-kolla08:27
*** zhurong has quit IRC08:27
masberdo I need to run kolla-ansible deploy after kolla-ansible reconfigure?08:29
mgoddardthomas_oneill: I've got some patches for Bifrost that I've not yet pushed upstream including sorting out the logging. Let me know if you would like any.08:29
openstackgerritDou Rui Yuan proposed openstack/kolla-ansible master: update expressions for more clear and unified  https://review.openstack.org/44688208:31
openstackgerritpangliye proposed openstack/kolla-ansible master: Fixes Bug1673691 There is no files in directory "/etc/kolla/telegraf/config/",so the operation of copy should be deleted Fixes Bug1673691  https://review.openstack.org/44688808:36
*** duke__ has joined #openstack-kolla08:36
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Register Cinder v3 endpoint in keystone  https://review.openstack.org/44688908:37
*** skramaja has quit IRC08:37
*** mgoddard has quit IRC08:43
Jeffrey4lpbourke, could u review https://review.openstack.org/446877 which fix the gate ( pbr version issue )08:48
*** l4yerffeJ_ has quit IRC08:51
*** mpansky has joined #openstack-kolla08:53
*** shardy has joined #openstack-kolla09:01
*** Serlex has joined #openstack-kolla09:06
*** blallau has joined #openstack-kolla09:12
*** skramaja has joined #openstack-kolla09:14
*** david-lyle_ has joined #openstack-kolla09:18
*** david-lyle has quit IRC09:18
*** cmceniry has quit IRC09:20
*** mgoddard has joined #openstack-kolla09:20
*** cmceniry has joined #openstack-kolla09:20
openstackgerritJuan Antonio Osorio Robles proposed openstack/kolla master: RHEL: Add mod_ssl for services running over httpd  https://review.openstack.org/44691109:27
thomas_oneillmgoddard: morning!! yes please!! :)09:27
thomas_oneillI was really zonked at the end of yesterday so I could hardly think, but once I'd gotten bifrost actually deploying in a container and the ironic service (at least the API) running, it seemed that the ironic conductor was not running and no drivers were being listed as available09:28
thomas_oneillneed to take another look again now, just going to run through it manually and try to see what might be going wrong09:28
thomas_oneillbut yes will be useful to see at any patches that might make things easier :D09:29
thomas_oneillmgoddard: could I also ask your opinion on the python mysqldb dep screw-up issue? There are two sets of bugfixes for this problem, sayantani posted this at the end of january https://bugs.launchpad.net/kolla-ansible/+bug/1658840 https://review.openstack.org/#/c/424391/ and then I posted this just yesterday I think https://bugs.launchpad.net/kolla/+bug/1673420 and09:32
openstackLaunchpad bug 1658840 in kolla-ansible "Bootstrapping bifrost services in Bifrost container fails with error in creating MySQL DB" [Undecided,In progress] - Assigned to sayantani (sayantani.goswami)09:32
thomas_oneillhttps://review.openstack.org/#/c/446609/09:32
openstackLaunchpad bug 1673420 in kolla "Bifrost image's mysqldb module is broken" [Undecided,Confirmed]09:32
*** qiliang27 has quit IRC09:33
*** athomas has joined #openstack-kolla09:38
*** zhubingbing has quit IRC09:43
mgoddardthomas_oneill: I'm running centos so haven't hit the mysql issue09:46
mgoddardthomas_oneill: here's bifrost: https://github.com/stackhpc/bifrost/commits/stackhpc-3.0.009:49
thomas_oneillcheers :)09:50
mgoddardthomas_oneill: this one pins bifrost dependencies in the kolla image: https://github.com/stackhpc/kolla/commit/4bf7d181f60a2195a7de14060dea4f6d2215fe7209:50
mgoddardthomas_oneill: and some for kolla-ansible: https://github.com/stackhpc/kolla-ansible/commits/stackhpc-4.0.0.0rc109:51
mgoddardI think they're all rebased onto a recent stable/ocata, so stop looking when the commits are no longer by me or 'stack'09:52
*** egonzalez has joined #openstack-kolla09:52
thomas_oneilldamn there's quite a lot here eh! are you going to submit some PS's soon? :p09:53
mgoddardthomas_oneill: some are already submitted, the rest I will submit soon09:54
mgoddardthomas_oneill: tried to prioritise stuff that was actually broken over improvements09:55
openstackgerritcaoyuan proposed openstack/kolla master: Update the Fluent link format in README.rst  https://review.openstack.org/44692809:56
*** pbourke has quit IRC10:05
*** pbourke has joined #openstack-kolla10:05
*** Margin has quit IRC10:06
*** Jezekus has quit IRC10:06
*** jaesang has quit IRC10:08
*** daidv has quit IRC10:08
*** mgoddard has quit IRC10:09
*** cuongnv has quit IRC10:15
*** yingjun has quit IRC10:15
*** satyar has quit IRC10:20
openstackgerritMerged openstack/kolla master: Update the Fluent link format in README.rst  https://review.openstack.org/44692810:25
*** duonghq has quit IRC10:26
*** Jezekus has joined #openstack-kolla10:28
*** deadnull has joined #openstack-kolla10:30
*** deadnull has quit IRC10:30
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Register Cinder v3 endpoint in keystone  https://review.openstack.org/44688910:40
*** egonzalez has quit IRC10:43
*** yuanying has quit IRC10:45
thomas_oneillhttps://review.openstack.org/#/c/443747/ I think this needs more reviews for workflow? Or does a core have to tick the workflow separately?10:46
*** athomas has quit IRC10:54
*** jaosorior has joined #openstack-kolla10:55
*** jaosorior has joined #openstack-kolla10:56
*** mgoddard has joined #openstack-kolla10:57
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Update infra components list in README.rst  https://review.openstack.org/44687310:59
*** caowei has quit IRC11:00
*** rwallner has joined #openstack-kolla11:04
*** zhurong has joined #openstack-kolla11:14
*** athomas has joined #openstack-kolla11:17
*** papacz has joined #openstack-kolla11:21
jaosoriorhey folks, any help debugging this failure? https://review.openstack.org/#/c/446911/ not sure if it's related to the patch.11:24
thomas_oneilljaosorior: one or more of the images failed to build :p11:31
thomas_oneillubuntu source11:31
thomas_oneillbifrost-base failed, dunno if that's excluded11:31
thomas_oneillhttp://logs.openstack.org/11/446911/1/check/gate-kolla-dsvm-build-ubuntu-source-ubuntu-xenial/294a687/console.html find which image(s) failed and aren't excluded, try to build manually locally to verify that they can be built or not, I guess?11:32
jaosoriorthomas_oneill: I only changed stuff for RHEL O_o11:32
thomas_oneillI don't know I've not debugged failed gates before11:32
thomas_oneilloh11:32
*** duke__ has quit IRC11:32
*** athomas has quit IRC11:36
thomas_oneillit tests against the stable/ocata releases of the openstack projects codes right?11:40
*** yuanying has joined #openstack-kolla11:40
*** mpansky has quit IRC11:40
openstackgerritOpenStack Proposal Bot proposed openstack/kolla master: Updated from global requirements  https://review.openstack.org/44677111:41
mgoddardthomas_oneill, jaosorior: bifrost is an odd one though - kolla installs ocata release of bifrost, but bifrost by default pulls down latest versions of its dependencies11:42
mgoddardhence https://github.com/stackhpc/kolla/commit/4bf7d181f60a2195a7de14060dea4f6d2215fe7211:43
jaosoriormgoddard: so... what do I do there? do I just recheck the gate?11:44
mgoddardjaosorior: let me check the logs :)11:45
mgoddardjaosorior: hmm. looks like bifrost is failing to install shade11:47
jaosoriormgoddard: seems quite unrelated to my change11:47
mgoddardjaosorior: error message is TypeError: __init__() got an unexpected keyword argument 'encoding'11:48
openstackgerritMohammed Naser proposed openstack/kolla master: Remove global privilege escalation for deploy_aio.sh  https://review.openstack.org/44664511:49
thomas_oneillmgoddard: oh yeah that's why it always ends up being 3.0.1 which is newton right?11:49
jaosoriormgoddard: so, all I did was add mod_ssl to RHEL based systems https://review.openstack.org/#/c/446911/11:49
mgoddardjaosorior: yeh I don't think it's caused by your change11:50
mgoddardjaosorior: try a recheck, just to see, but I doubt it will work :)11:53
hawihi. i have question regarding prechecks. When checking for docker-py, it gives me failed, invalid syntax. so, my question is - what version of python and python-py it expects actually? on ubuntu 16.04.2 i do have only python3 right now12:00
mnasermorning everyone12:01
*** jaosorior is now known as jaosorior_brb12:01
mnaserjaosorior_brb im checking your fails12:02
*** mpansky has joined #openstack-kolla12:02
mnaserjaosorior_brb found your problem12:03
*** rhallisey has joined #openstack-kolla12:03
mnaser(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:44312:03
mnaserthere is another service taking 44312:03
mnasermost likely its haproxy12:03
*** rhallisey has quit IRC12:04
jaosorior_brbmnaser: interesting, it shouldn't even be trying to bind to 443; it should just add the mod_ssl package :/12:04
mnaserjaosorior_brb im pretty sure installing mod_ssl drops a file in conf.d with a Listen 443 in rhel-based systems12:04
*** shardy is now known as shardy_lunch12:04
mnaseris there a reason why you dont want haproxy to do ssl instead?12:05
mnaser(just out of curiosity)12:05
jaosorior_brbmnaser: cause we're trying to do SSL everywhere12:06
mnaserok so even ssl between haproxy and horizon12:06
jaosorior_brbmnaser: so haproxy is doing TLS termination, and then it will go to the services, also encrypted.12:06
jaosorior_brbmnaser: that's correct.12:06
*** satyar has joined #openstack-kolla12:07
mnaseri see12:07
mnaserbtw, i got this setup which can maybe help us gauge our overall stability - http://grafana.openstack.org/dashboard/db/kolla-failure-rate12:08
*** caowei has joined #openstack-kolla12:09
mnaserlooks like ubuntu xenial source builds are a bit of am ess12:09
*** zhugaoxiao has quit IRC12:13
*** zhugaoxiao has joined #openstack-kolla12:14
*** caowei has quit IRC12:16
*** Serlex has quit IRC12:17
satyarHi everyone12:18
satyarone question is freezer implemented fully with kolla?12:18
*** athomas has joined #openstack-kolla12:23
*** rwellum has joined #openstack-kolla12:27
hawiinstalled python-minimal (2.7.11-1) and python-docker (1.8.0-0ubuntu1), now i can move forward with installation, but with python3 docker-py fails on ubuntu 16.04.2, when doing prechecks12:27
*** saneax is now known as saneax-_-|AFK12:27
hawiand got another weird thing from, for some hosts precheck rabbitmq says that Hostname has to resolve to IP address of api_interface12:28
*** goldyfruit has joined #openstack-kolla12:29
*** lamt has quit IRC12:30
dimsSamYaple : could use your input https://etherpad.openstack.org/p/single-approach-to-containers-for-projects12:31
dimsinc0 : anyone else interested ^12:31
*** erlon has joined #openstack-kolla12:33
hawirabbitmq precheck fail "Hostname has to resolve to IP address of api_interface" doesnt appear for other machines, only for three machines (all controllers)12:44
hawii see12:44
mnasercan every node resolve the hostname of other controllers hawi ?12:46
hawiyes. i think this precheck (havent looked inside yet) assumes that interfaces from API network for these controller hosts resolv to hostname12:47
mnaserit has to hawi12:48
mnaseror the rabbitmq cluster wont form quorum because it uses dns resolution12:48
hawimnaser: http://pastebin.com/vNWbKjGE12:50
hawisomething like this12:50
mnaserdo they resolve?12:51
hawiyes, they do12:53
hawifrom each node12:53
hawibut yes, API ip is different, hostname is related to PXE network IP, what i have12:53
hawiAPI interface ip12:53
mnaserthe hostname has to resolve to the api interface ip12:53
mnaserit doesnt sound like it is12:54
*** schwicht has joined #openstack-kolla12:54
hawii used MAAS for deploying machines, so PXE network has hostname. aghhh. anyway, i can fix it12:55
*** manheim has quit IRC12:56
*** manheim has joined #openstack-kolla12:56
*** mpansky has quit IRC12:58
*** mpansky has joined #openstack-kolla12:58
*** rhallisey has joined #openstack-kolla13:01
*** schwicht has quit IRC13:03
nea1how long will it take until ocata is officially released?13:03
*** schwicht has joined #openstack-kolla13:05
mnasernea1 ocata is out i believe13:05
mnaserstable/ocata is tagged already13:05
*** Jezekus has quit IRC13:06
mnaserstuff tagged 4.0.0 i think is stable/ocata13:06
nea1mnaser: ok then other question when is it uploaded to https://hub.docker.com/r/kolla/centos-binary-kolla-toolbox/tags/ or a other docker_registry?13:18
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155013:19
*** blallau_ has joined #openstack-kolla13:21
*** schwicht has quit IRC13:21
*** Jezekus has joined #openstack-kolla13:22
*** duonghq has joined #openstack-kolla13:23
duonghqevening guys13:23
nea1TASK [haproxy : Waiting for virtual IP to appear] ******************************13:24
nea1fatal: [localhost]: FAILED! => {"changed": false, "elapsed": 301, "failed": true, "msg": "Timeout when waiting for vip.kolla.[...]:3306"}13:24
nea1seams like keepalived is failing to add the IP to the interface13:25
sdakemorning peeps13:27
sdakeyou off duonghq ?13:27
sdakeduonghq you had asked a quesiton earlier before i PTFO'ed for the night13:28
sdake21:33:03]  <duonghq>sdake, I'm not sure why we clone into kolla-ansible, then cd kolla/ansible?13:28
sdake[21:33:13]  <duonghq>there is not setup.py here13:28
sdakeduonghq would you mind expanding on the question - I'm not sure what your referencing there13:28
duonghqsdake, minutes, I need finding the link to kolla-k8s deploy bp etherpad13:29
duonghqbookmark at work and home are not synced13:29
sdakemoment i've got it13:29
*** dave-mccowan has joined #openstack-kolla13:30
sdakehttps://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP13:31
sdakesup dave-mccowan13:31
sdakeduonghq ^^13:31
duonghqthanks13:31
*** mpansky has quit IRC13:31
dave-mccowanhi sdake13:31
*** srwilkers has joined #openstack-kolla13:31
sdakesup srwilkers13:31
duonghqin line 78, we clone kolla-ansible repository, but in like 80, we cd kolla/ansible13:32
duonghqhi srwilkers13:32
*** jtriley has joined #openstack-kolla13:32
Jeffrey4lsrwilkers, my irc client said: " srwilkers aka. srwilkers_away aka. srwilkers_aaway aka. srwilkers_coffee aka. srwilkers_brb aka. srwilkers_lunch "13:33
Jeffrey4lhi sdake13:34
sdakesup Jeffrey4l13:34
Jeffrey4lcould u review https://review.openstack.org/446877 , which related to the pbr issue.13:34
Jeffrey4lneed install kolla-ansible in env to make pbr work as expected.13:34
sdakewhy -e?13:35
sdakeand not just pip install kolla-ansible?13:35
Jeffrey4ljust like "usedevelop=True" in tox.ini13:35
Jeffrey4li can not explain why.13:36
*** zhugaoxiao has quit IRC13:36
Jeffrey4lbut by using "pip install kolla-ansible" it return "4.0.0" on ocata branch13:36
Jeffrey4lwhen using "pip install -e kolla-ansible" it return "4.0.1" on ocata branch.13:36
Jeffrey4lthe latter one is expected.13:36
Jeffrey4land tox -e works like this.13:36
*** zhugaoxiao has joined #openstack-kolla13:37
* sdake groans13:37
Jeffrey4lso currently, kolla stable/ocata generation 4.0.1 image, but kolla-ansible try to use 4.0.0 image.13:37
sdakeok - are we expecting operators to pip install -e then?13:37
Jeffrey4loperator should use "pip install kolla-ansible" ( from pypi )13:37
sdakecool13:37
Jeffrey4lwe are using " pip install -e kolla-ansible ( from git repo ) "13:38
Jeffrey4lin  gate.13:38
sdakehttps://etherpad.openstack.org/p/single-approach-to-containers-for-projects13:43
*** schwicht has joined #openstack-kolla13:45
duonghqsdake, can you take a look at my question above?,13:47
duonghqin kolla-kubernetes deploy guide in line 78, we clone kolla-ansible repository, but in like 80, we cd kolla/ansible13:47
sdakeduonghq yes - I fixed it in the etherpad13:47
sdakecan you reload?13:47
duonghqthanks,13:48
sdakeI answered that question on line 8713:48
duonghqnetwork laggy, I expected it update in realtime13:48
sdakeya etherpad a little flakey sometimes13:48
duonghqsorry13:48
sdakeno reason to apologize13:48
sdakeits a toosl problem not a person problem :)13:48
duonghq:)13:48
sdakeno reason to apologize for tools problems13:49
duonghqroger13:49
sdakeapologize when you actually hurt someone - or alternatively ask for forgiveness ;)13:49
*** mpansky has joined #openstack-kolla13:50
*** jaosorior_brb is now known as jaosorior13:50
*** blallau_ has quit IRC13:51
*** Jezekus has quit IRC13:52
*** yingjun has joined #openstack-kolla13:52
*** shardy_lunch is now known as shardy13:56
*** zhurong has quit IRC13:56
*** mpansky has quit IRC14:02
*** mpansky has joined #openstack-kolla14:03
*** lamt has joined #openstack-kolla14:06
sbezverkJeffrey4l: tried to create cell, it is failing.. but it does not produce much debugging info. I was wondering if you might have log generated by simple_cell create command so I could use it as a baseline14:06
Jeffrey4lsbezverk, when running simple_cell_setup, the nova-compute must up and shown in 'nova service-list'. did you di this?14:07
sdakesbezverk morning fine sir14:08
Jeffrey4lsimple_cell_setup generate few lines of logs.14:08
sbezverksdake: good morning14:08
sdakesbezverk quick q, are you back in the office?14:08
Jeffrey4li do not think it is useful.14:08
sbezverksdake: nope still in SJ14:08
sdakesbezverk cool - you are back monday?14:08
sbezverkJeffrey4l: :( I will make sure all services are up before running it..14:09
sdakesbezverk reason I ask is - I want to start cracking on external config maps and ejecting the kolla-ansible dependency14:09
sbezverksdake: yeah should be back on monday14:09
sdakeI think this along with docs are our number one and one priorities14:09
Jeffrey4lnot up. need keep "nova-compute" is shown in "nova service-list"14:09
sdakeI am not quite sure where to start14:09
sbezverksdake: makes two of us ;)14:09
sdakeok - well the way to answer that question is to get started :)14:10
sdakelet me look at what hcoas my calendar entails14:10
sbezverksdake: I need to get vm running with ocata images and it is not yet happening, so it is a priority for me.. I will join you after14:11
sdakesbezverk perhaps we can have a livve brainstorming session for those folks interested in the community14:11
sdakesbezverk is ocata in the gating atm?14:11
sdakesbezverk (not sure if it was added yet)14:11
sdake(is that what your working on?)14:11
sbezverksdake: my ironic jobs are using 4.0.0 images14:11
sdakenice14:11
sdakei pushed centos src yesterday14:11
sbezverksdake: but it is not wokring14:11
sdakeuse_neutron = true14:12
sdakecould be one thing14:12
sdakeok - so your working on gating 4.0.0 images then?14:12
sbezverksdake: no, it is cell14:12
sbezverksdake: no cell, no vm :)14:12
sdakeright14:13
* sdake groans14:13
sdakewell gating 4.0.0 images is probably t he top of the stack of things we need to do :)14:13
sdakenext on the stack is docs + external config maps14:13
sbezverksdake: that is what I think too14:13
sbezverkJeffrey4l: could you take a look I got from a container creating cell to see if it makes sense to you, since I have never seen any working scenario, it is hard for m to tell if it is ok or not, I do not see any obvious errors14:16
Jeffrey4lok. logs or?14:16
sbezverkJeffrey4l: nvm log size is 0 bytes :(14:18
Jeffrey4lcheck the return code.14:18
sbezverkJeffrey4l: cool thanks for the idea14:19
Jeffrey4lsbezverk, check the nova code here https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L108314:19
Jeffrey4li guess you may exit in here https://github.com/openstack/nova/blob/master/nova/cmd/manage.py#L111414:19
Jeffrey4lnova-compute isn't up and shown in "nova service-list"14:19
sbezverkJeffrey4l: I have a function in the gate code that can wait for nova to be fully up before creating cell.. will try it and also get ret code..14:22
Jeffrey4lgot.14:22
*** Serlex has joined #openstack-kolla14:25
*** mnasiadka has quit IRC14:27
*** sayantani01 has joined #openstack-kolla14:31
thomas_oneillany of you cores fancy final review/workflow for this? https://review.openstack.org/#/c/443747/14:33
*** lamt has quit IRC14:34
matrohonhi kolla experts14:39
matrohonI just registered a new BP in kolla14:39
matrohondo I need the corresponding one in kolla-ansible?14:39
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155014:40
*** rmart04 has joined #openstack-kolla14:42
*** zhubingbing_ has joined #openstack-kolla14:44
*** manheim has quit IRC14:57
mnaserurgh14:59
mnasermore gate breakge14:59
mnaserlooks like master source seems broken?15:00
mnaserbtw i pasted this earlier but i figured ill repost, this will hopefully help us track down the state of things and see where we can improve things - http://grafana.openstack.org/dashboard/db/kolla-failure-rate15:00
*** mpansky has quit IRC15:01
*** sbezverk has quit IRC15:03
thomas_oneillsayantani01: did you see updates on my bug/fix that's the same as yours but also completely different? :P15:04
satyarHi all15:04
satyarany one tried freezer?15:04
thomas_oneillI'm pretty new to contributing so my ability is limited, but a decision needs to be made on how best to resolve the issue, I don't think either of our PS's are great, both pretty hackish/workaroundish15:04
sayantani01thomas_oneill I didn't get a chance to take a look yesterday but I will definitely get on it today15:05
thomas_oneillcool15:05
*** mpansky has joined #openstack-kolla15:05
*** Serlex has quit IRC15:06
sayantani01Can you also look at the comments on my patch thomas_oneill ?15:06
thomas_oneillyeah, sec15:07
dmsimardmnaser: I wonder if you can drill down those graphs further, (with distros)15:09
dmsimardmnaser: oh wait they're already there15:09
mnaser:D15:09
mnaserubuntu source seems to have been pooping out so im working on that15:10
mnaserim trying to replicate right now15:10
*** mpansky has quit IRC15:10
mnaserunfortunately source build failures => things cant go through :(15:10
dmsimardmnaser: if kolla would run tempest, you could also get data from openstack-health15:12
mnaserdmsimard thats the goal eventually15:13
mnaserim trying to figure out these ubuntu failures.. random pip install problem about a library15:13
thomas_oneillhow do I actually make a comment on a PS? I've added a comment and it's saying it's a draft15:14
dmsimardthomas_oneill: gotta click "post" after clicking on the "reply" button at the top15:14
*** manheim has joined #openstack-kolla15:15
*** dave-mccowan has quit IRC15:15
thomas_oneilldmsimard: I don't see this, it's not a reply it's an original comment. I've done replies before with no problem.15:16
mnaserthomas_oneill once you have drafts in, going back to the main gerrit review page and click on "reply"15:16
mnaserit'll show all the comments and click post15:16
thomas_oneilloh I see15:16
goldyfruitDo you guys had this issue with Keystone:  ContextualVersionConflict: (pbr 2.0.0 (/var/lib/kolla/venv/lib/python2.7/site-packages), Requirement.parse('pbr<2.0,>=1.3'), set(['sqlalchemy-migrate']))15:21
goldyfruitI'm trying to redeploy Mitaka but Keystone complains every time that I try to reach his endpoint15:22
*** manheim has quit IRC15:22
sdakeportdirect pingola15:23
goldyfruitIf I downgrade pbr to 1.10 I have complains about Olso: ContextualVersionConflict: (pbr 1.10.0 (/var/lib/kolla/venv/lib/python2.7/site-packages), Requirement.parse('pbr>=2.0.0'), set(['oslo.i18n', 'oslo.utils', 'oslo.middleware', 'oslo.context', 'oslo.log', 'stevedore', 'oslo.messaging', 'debtcollector', 'oslo.concurrency']))15:26
*** ipsecguy_ is now known as ipsecguy15:27
*** dave-mccowan has joined #openstack-kolla15:29
*** dciabrin is now known as dciabrin|away15:29
portdirecthey sdake o/15:30
portdirectjust heading into a meeting but will be back in 1:30 (I hope :/ )15:30
sdakeportdirect enjoy15:31
*** yingjun has quit IRC15:31
sdakeportdirect ping me when you return15:31
*** hrw has quit IRC15:35
*** hrw has joined #openstack-kolla15:36
*** Serlex has joined #openstack-kolla15:40
*** Jezekus has joined #openstack-kolla15:50
*** rmart04 has quit IRC15:52
*** sbezverk has joined #openstack-kolla15:52
*** rmart04 has joined #openstack-kolla15:54
*** dciabrin|away is now known as dciabrin15:57
*** kbaegis has joined #openstack-kolla15:58
*** rmart04 has quit IRC16:00
kfox1111morning.16:03
goldyfruitMorning16:03
mnasero/16:03
*** duonghq has quit IRC16:05
kfox1111heh; https://etherpad.openstack.org/p/single-approach-to-containers-for-projects16:12
kfox1111getting called out for multiple docker container implementations. :/16:12
kfox1111ah. I see others have already found it. :)16:13
*** kbaegis has quit IRC16:13
*** MasterOfBugs has joined #openstack-kolla16:17
*** kbaegis has joined #openstack-kolla16:18
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155016:19
openstackgerritMohammed Naser proposed openstack/kolla master: Switch to Kolla virtual environment for Bifrost  https://review.openstack.org/44705316:20
mnaseranother gate fix ^16:20
mnaserthis should clear out ubuntu-source build fails16:20
mnaserwhich has been at around 50%16:21
mnaserat at the moment 35% failure rate compared to centos/oracle's 6%16:21
*** shananigans has quit IRC16:23
mnasermgoddard thanks for the catch!16:38
mgoddardmnaster: no problem. odd how often these 'ships in the night' moments happen :)16:39
mnaserglad we caught the other one, better off to just keep working with our venv rather and avoid getting bit by weird system python issues16:40
*** goldyfruit has quit IRC16:42
*** kbaegis has quit IRC16:44
mgoddardmnaster: agreed16:46
*** bmace has quit IRC16:48
*** bmace has joined #openstack-kolla16:48
*** eaguilar has joined #openstack-kolla16:49
*** zhubingbing_ has quit IRC16:50
*** kbaegis has joined #openstack-kolla16:52
*** rabel has joined #openstack-kolla16:55
*** rabel has left #openstack-kolla16:55
*** jaosorior has quit IRC16:55
*** skramaja has quit IRC16:57
*** goldyfruit has joined #openstack-kolla16:58
sdakekfox1111 any progress on external config maps?17:04
*** athomas has quit IRC17:07
sdakejascott1 you alive17:09
jascott1idk maybe17:10
jascott1feeling like crap today17:10
jascott1hows things sdake?17:10
sdakejascott1 i have really tried to take your advice and not drop knowledge or wisdom17:10
sdakejascott1 yet everyone wants it :)17:11
jascott1LOL17:11
sdakejascott1 you got a plan B:)17:11
sdakejascott1 here was my plan A: https://etherpad.openstack.org/p/single-approach-to-containers-for-projects17:12
*** matrohon has quit IRC17:13
*** haplo37 has quit IRC17:15
*** kbaegis has quit IRC17:16
jascott1gee17:18
*** pcaruana has quit IRC17:18
jascott1lotsa stuff in there oh ya17:18
goldyfruitAbout my previous issue: (11:21:22 AM) goldyfruit: Do you guys had this issue with Keystone:  ContextualVersionConflict: (pbr 2.0.0 (/var/lib/kolla/venv/lib/python2.7/site-packages), Requirement.parse('pbr<2.0,>=1.3'), set(['sqlalchemy-migrate']))17:18
goldyfruitSolution: In keystone container:  sed -i 's/Requires-Dist: pbr (>=1.3,<2.0)/Requires-Dist: pbr (>=1.3,<=2.0)/g' /var/lib/kolla/venv/lib/python2.7/site-packages/sqlalchemy_migrate-0.10.0.dist-info/METADATA17:18
*** Jezekus has quit IRC17:24
*** haplo37 has joined #openstack-kolla17:24
jascott1sdake are you going for blueprint outta that doc?17:28
sdakejascott1 nope17:30
sdakejascott1 this is dims rodeo :)17:30
sdakejascott1 dims is interested in seeking a collaboration so OpenStack doesn't end up with two dueling container implementations when one would suffice17:30
sdakeatleast that is my speculation, I actually dont know what he is after ;-)17:31
jascott1i see17:31
dims"We have openstack-deb* openstack-ansible* openstack-charm* openstack-puppet* in our OpenStack git repo, can we have openstack-docker-* for folks to collaborate together?"17:31
dimsright on top of the etherpad17:31
dimsjascott1 : ^17:35
mnaserisnt kolla what openstack-docker-* is?17:38
jascott1yeah but community differs on opinions about what should be in images17:39
sdakemnaser kolla pioneered containers - in openstack we don't stop people from competing on the periphery (such as deployment tools)17:40
mnasersdake correct, but kolla right now is a project with dockerfiles only17:40
mnaserkolla-ansible is the deployment tool that uses kolla container17:40
sdakemnaser as a result, there are a couple of different teams that have tried different approaches17:40
mnaseri mean in all terms, we can rename kolla to openstack-docker and keep kolla-ansible to keep using it17:41
SamYaplemnaser: kolla is a project with a bunch of custom tooling and is very non-standard docker project17:41
SamYaplemnaser: so youd get alot of pushback there17:41
SamYaplerightfully so17:41
sdakemnaser kolla could do that - and take that namespace easily17:41
mnaseri think the problem is there is so many ways of deploying with docker17:41
sdakemnaser we can rename our repos however we like17:42
mnaserand that's a very hard thing to solve17:42
mnaserex: how do you configure services in containers, kolla took the volume approach17:42
mnaser*maybe* some people might want to go and run it through something like etcd for example17:42
sdakemnaser however it is confusing to have one global namespace such as openstack-docker or openstack-helm for a project that should be neutral to all parties17:42
SamYaplemnaser: no thats deploy tooling problems17:42
mnaseryou're right, at the end of the day $service expects a config in /etc/$service/$service.conf17:43
SamYaplemnaser: if you look at the official docker projects youll find the kolla build process is the issue here. a process that i at one point pushed hard (stuff like adding openstack-base)17:43
SamYaplebut docker as a whole has gone another direction17:43
mnaserwhat do you mean by the build process being the issue17:43
SamYaplewe produce shared common layers in kolla.... that have no other purpose17:43
SamYapleno one does that anywhere else17:44
sdakeSamYaple docker has gone the same direction it was headed when the kolla images were created17:44
SamYapleand it kills CI/CD17:44
SamYaplesdake: this is incorrect info17:44
*** blallau has quit IRC17:44
mnaseri think having a Dockerfile that sits inside say, openstack/nova would be very nice17:44
sdakecommon shared layers *were* best practice according to you17:44
sdakehowever they are *NOT* best practices today17:44
mnaserbut there would be a lot of duplicating layers17:44
SamYaplesdake: this is correct17:45
sdakemnaser yup indeed there is duplication17:45
lazyPowerduplicated layers only matter if you *really* care about image size being transferrred.17:45
mnaserbut i think the duplication is very minimal and a common base shouldnt be enforced17:45
SamYaplelazyPower: actually funny you say that, images can be prouduced a third of the size of kolla.... and not share layers17:45
mnaserhonestly the kolla images are so layer heavy right now17:45
lazyPowerSamYaple: sure, bu ti said transferred, meaning fetching that shared common base once and not again for each subsidiary image17:46
lazyPowerSamYaple: the savings are marginal at best imho.17:46
SamYaplelazyPower: no im saying how ever you want to slice it, total size transfered and stored _can_ be less than kolla right now17:46
sdakeSamYaple most people don't really care about image size17:46
mnaseralso what sdake said. i cant imagine it being that much heavier than it is now17:46
sdakeSamYaple most peple care about two thigns -> CURL all over the internet17:47
SamYaplemnaser: right. and thats what is being discussed. Kolla being layer heavy. what that adds and what it prevents (like good CICD)17:47
sdakeand CI/CD17:47
lazyPower^17:47
SamYaplesdake: kolla sucks at cicd and we all know it17:47
lazyPowerthat17:47
mnaserive been doing an effort to fix the ci/cd stuff :(17:47
lazyPowercurl + ci process.17:47
lazyPoweroh but i wasn't trying to jump on a bash wagon though17:47
sdakeSamYaple i agree curl and ci/cd are a problem17:47
mnaserthere is just so many things going everywhere and the fact that its not voting seems to have kinda let it stale a bit17:47
sdakei never said they weren't17:47
sdakein fact i wrote down in the etherpad curl shouldn't be used in the image at all17:48
SamYaplesdake: but you are fighting for a structure that inhibits CI/CD17:48
mnasersdake but then you have another interesting issue17:48
mnaserwhere do you get deps from17:48
lazyPowerso, dumb question17:48
sdakeSamYaple i have not fought for any structure that inhibits ci/cd17:48
lazyPowerwhy aren't we running a mirror or a minio bucket that warehouses all the deps17:48
lazyPoweror a swift bucket because #openstack17:48
sdakeSamYaple i haev fought for a structure that preserves the  API17:48
sdakeSamYaple i dont care if the images are one or many17:48
sdakerather have one layer or many17:49
mnaserhow would you handle source/binary types in the Dockerfile17:49
lazyPowerthats typically how this is resolved in enterprise scenarios, they setup a mirror and the mirror becomes teh canonicalized source to prevent random curls.17:49
sdakelazyPower good question - we can't get it done17:49
SamYaplesdake: which is great. preserve that api. you should. but thats not what me (or others) are looking for these days17:49
sdakeSamYaple not sure what your argument has to do with layering at all?17:50
lazyPoweroh ok.17:50
mnaseron another point, while i love the enthusiasm behind reducing layers, is that really the reason behind the CI/CD problems? :x17:50
sdakemnaser the answer is yes17:50
mnaserhow so? i would feel things would become so much scarier if things were all over the place (in-repo Dockerfile's for example)17:50
sdakemnaser here is how today's CI/CD systems work17:51
sdaketake ZUUl as an example17:51
sdakeZUUL checks out a clean fresh repo on a clean fresh OS17:51
sdakeit builds a nova-novncproxy17:51
sdakeits done17:51
jascott1does new squash make the layering conversation moot as user can just run that at end of their process?17:51
SamYaplesdake: the sharedlayering is whats preventing ci/cd17:51
sdaketo do that, it has to build atleast 3 layers17:51
sdakeSamYaple yes i agreed above - "The answer is yes"17:52
mnaseropenstack-base, nova-base and nova-novncproxy -- so what's the problem with it having to build all 3 layers?17:52
sdakethe 3 layers have to be rebuilt every time each container is built17:52
sdakethis takes 4-5 hours in a real datacenter17:52
lazyPoweri think its just the level of complexity being introduced to have those 3 common layers.17:52
sdakewith a fresh checkout of each container17:52
SamYaplemnaser: with cicd, if those layers are shared, they have to build all the layers at the same time17:52
lazyPoweror am i off base here?17:52
sdakelazyPower no the layers reduce complexity17:53
SamYaplelazyPower: youre right, but there is more to the picuter as well17:53
sdakelazyPower compressing the layers into one dockerfile increeases complexity17:53
mnaserbut at the end of the day, the series of commands which are done to build nova-novncproxy will still be the same in a single Dockerfile17:53
*** m1dev has joined #openstack-kolla17:53
SamYaplelazyPower: they completely increase complexity, despite what others say.17:53
mnaseryou'll still have to add the repos, you'll still have to install base packages, you'll still have to install nova and then install nova-novncproxy17:53
lazyPowerwell this is fun, 2 differing opinions being voiced quite loudly.17:53
SamYaplelazyPower: asking anyone who works with docker and big projects like this, and they will agree17:53
mnasernow if i have to make a change to the base, 7 dockerfiles have to be edited17:54
lazyPoweri work with docker every day, and i avoid shared layers like the plague if i can. I only use the base layer as my shared layer, and from there each component gets its own dockerfile and goes.17:54
sdakelazyPower if you want to really break out the salt and butter check out https://etherpad.openstack.org/p/single-approach-to-containers-for-projects17:54
SamYaplelazyPower: to be fair, most people who agree with me have left this project. im just here to try to sway the project for the future into a better path17:54
lazyPowerand when i say base layer, i mean FROM: ubuntu or FROM: alpine17:55
sdakeSamYaple i disagree with your reasoning on why people have left the project - i think you are making a blanket assertion without stating facts17:55
sdakeSamYaple and poisining relationships in the process17:55
sdakeenjoy :)17:55
lazyPowerbut at the end of the day, this is the complexity problem with the golden image paradigm yes?17:55
lazyPoweryou make a change in a base layer and it domino's all teh way out to each edge component17:55
SamYaplelazyPower: yea. no image can meet 100%17:55
SamYaplethats just how it works17:55
mnaserlazyPower while id love to have a very small thin layer, but editing something in nova base could result in 7 changes17:56
mnaserinstead of one change in a base layer17:56
SamYapleso if you notice some patchsets in the kolla queue, youll see where we have discussed making these images flatter17:56
SamYaplethe issue comes down to sort of a lock-in affect17:56
*** mgoddard has quit IRC17:56
mnasermy question goes back to whats the difference between 3 images to get a final one17:56
SamYaplewe have to maintain compatibility with the apis that have been put im place. that unfortunately means arbitrary jinja2 injects like, everywhere17:56
mnaseror 1 single thick one17:57
SamYaplemnaser: so the issue comes down to building17:57
mnaseri mean if you want to template the dockerfiles to include a base17:57
SamYaplemnaser: if you want keystone you build base->openstack-base->keystone-*17:57
SamYaplemnaser: if you want nova you build base->openstack-base->nova-*17:57
SamYapleyouve build base and openstack-base twice17:57
SamYapleand they are bit-for-bit different images17:57
SamYaplethey do not share layers17:57
mnaseri thought kolla builds in dependency order?17:58
SamYapleright, but we are talking cicd17:58
sdakemnaser indeed images are built in order17:58
SamYaplenow do w emake every commit build _all_ kolla images?17:58
mnaserwhy not17:58
SamYaplethats the only way to do it in the current system17:58
sdakemnaser what SamYaple is speaking of is that docker's CI/Cd system doesn't provide ordering17:58
SamYaplesdake: dont speak for me. thats not what im saying at all17:58
openstackgerritAlicja Kwasniewska proposed openstack/kolla master: Added snap-telemetry container  https://review.openstack.org/44616417:58
mnaserthe same reason we run devstack/tempest for a nova change17:58
sdakeSamYaple you speak for me why shouldn't i speak for you?17:59
mnaserthe same reason tripleo deploys an entire cloud17:59
lazyPowerbut you're running this through jenkins right? CloudBee's has released an addon to do this for you, where you mark jobs as dependent, and it will notify downstream jobs to rebuild when a base-layer image change is merged....17:59
mnaseras part of their CI17:59
SamYaplesdake: you are an openstack board member. stop please.17:59
sdakeSamYaple i always represent kolla in this channel17:59
lazyPowernot that i'm necessarily + or - on this convo, i just know that this conversation is getting quite heated and it doesn't seem like all the tech is being represented.17:59
sdakeand nobody else17:59
mnaserlazyPower i agree :x17:59
sdakeSamYaple DONT FORGET IT17:59
lazyPowerad-hominim?18:00
mnaserlazyPower imho, i think building $world is the way to continue going18:00
mnaserto say that we should test a single component alone if that single component changes is risky18:00
lazyPowerhave you guys looked at how kubernetes does its CI pipeline?18:00
lazyPowerhonest question ^18:00
sdakelazyPower indeed kuberntes uses jenkins at present18:01
sdakeand in the future they plan to use zuul theoretically18:01
mnaseri havent, any interesting docs lazyPower ?18:01
lazyPowermnaser: so this is to build off the "single component change single component check" statement18:01
*** zioproto has quit IRC18:01
mnaserbut then how do you make sure that these components didnt break something else18:01
lazyPowergoogle has like proably 18 different jobs, just realted to how it verifies a k8s change. And its layered. and it always ENDS with an e2e test with that changeset18:01
mnaseri dont know if we can do depenencies in zuul?18:02
mnaserex: how i imagine ci ideally: build all images, deploy cloud with kolla, run tempest against it18:02
lazyPowermeaning it can stgart by testing 1 component in a quick test, but before its fully accepted, its making it into an e2e test suite, that flexes the full stack of components (images w/ components in this case) in an integration test suite.18:02
lazyPowertake a look at the gubernator dashboard. and for docs - http://github.com/kubernetes/test-infra/18:02
mnaserright, i guess that helps in detecting early failures18:02
lazyPowerthats the only way they can feesibly do it, and we joined up to contribue results daily from the CDK stack of things. So we can pull in the prior days commits and both a) determine if we are broken because of a change, b) indicate to the project they've broken the build in a configuration they didn't account for in their own e2e suite.18:03
lazyPowernot that we break very often, but the 1.6 cycle has been in no-go state since freeze, which is a telling indicator of gaps in e2e.18:03
lazyPowerbut just because there are issues there, doesn't mean the approach is inferior.18:04
mnaserlazyPower i see what you're onto, but i think that's an optimization of the current state of CI18:04
lazyPowermnaser: maybe thats where we start instead of making changes to the baseline plubming18:04
lazyPower*plumbing18:04
mnaserand i agree with you lazyPower.  that's why i've been pushing patches to fix CI failures18:05
mnaseri also added this http://grafana.openstack.org/dashboard/db/kolla-failure-rate18:05
lazyPowerthats an excellent direction to be in, in my opinion18:05
mnaserso that we can drive that % down as much as possible18:05
mnaser(the ubuntu source failures i managed to find here - https://review.openstack.org/#/c/447053/)18:05
lazyPowerinc0: i provided some modicum of value ;) ^18:05
lazyPowermnaser: oh nice, even better if its fixing builds on ubuntu18:06
lazyPoweri'm clearly biased here :O18:06
mnasermy ideal world: get the fail % as low as possible18:06
mnaseradd tempest runs to the check/gate18:06
lazyPowerright, once you have confidence in your built artifacts you can start mucking about in the plubming and making bigger changes18:06
mnaserand get the deploy jobs to become voting18:06
mnaser100% agreed18:06
sdakemnaser kolla is almost there wrt voting gates18:07
sdakein fact the images vote18:07
mnaseryep ive noticed which is a great step18:07
sdakethe deploy jobs depend on infra to become voting - and that is in progress18:07
mnaserif we cant deploy everything in the simplest and cleanest way ... then trying to fix the plumbing is looking the wrong place (imho)18:07
sdakethe deploy jobs for kolla-kubernetes could be made voting today - I believe but can't confirm that is the case kfox1111 would have more detail18:07
mnaserthat's great to hear so we can have both ansible and k8s deploy jobs18:08
mnaseranyways, i've also personally mentioned that if anyone finds themself "recheck"-ing too often, feel free to reach out and ill do my best to help troubleshoot and solve18:09
* inc0 reading back18:12
inc0sorry I had cf of meetings this morning18:12
*** eglute has joined #openstack-kolla18:12
inc0mnaser: sdake so ad voting gates, I plan to rework our gates when zuul v3 lands18:13
mnaserinc0 do you know when that is because it's been a while i think :p18:14
inc01. multinode gates are going to be much easier and clean so multinode will be first class citizen18:14
sdakezuul v3 lands in june18:14
mnaseroh cool thanks sdake18:14
sdakenot sure when inc0 will land his rework of the gates for zuulv3 however18:14
mnaseri was hoping to start doing some work to getting tempest to run now (that can hopefully be reused when zuulv3 comes)18:14
mnaserinstead of the simple sanity check18:15
sdakemnaser i dont think the project wants to block on voting gates for zuulv3 to release18:15
inc0sdake: it's going to be ansible based so I'm hopeful we can write cool ansible code to turon one ansible to another18:15
sdakemnaser Jeffrey4l is doing gating work today18:15
sdakemnaser along with the rest of our emea crew18:15
inc0not only him18:15
sdakeinc0 right18:15
mnaserokay cool, i can help with some of that progress (if need be)18:15
inc0tempest voting is one thing we need, true18:15
sdakemnaser contributions much appreciated :)18:16
inc0I also want full upgrade gates18:16
inc0but that really matters when we can do multinode gating18:16
inc0and that depends on zuulv318:16
mnaserin the meantime, clearing out all the small issues that keep coming up is what i've been trying to do18:16
inc0sure18:17
inc0with zuul v3 gates will be built with ansible rather than bash18:17
*** satyar has quit IRC18:17
mnaserhint hint https://review.openstack.org/#/c/447053/ fixes ubuntu source builds which are blocking gate18:17
inc0so I'd be reluctant to spend too much time hacking bashscripts when we're only few weeks before we could do something much better18:17
sdakeif by few weeks you mean 8-12 weeks :)18:18
inc0sdake: when did you hear about june tho?18:18
inc0last time I checked they didn't have final date yet18:18
sdakeinc0 moment18:18
sdakeinc0 can't find the reference - saw it on twitter iirc from monty - can't find it now - who knows why :)18:24
mnasersdake i remember in the ML they mentioned that there is something using zuulv3 right now18:24
sdakemnaser possible - I am not on the zuulv3 team so I woulnd't know18:25
mnaserhttp://lists.openstack.org/pipermail/openstack-dev/2017-February/113065.html18:25
mnaserthis18:25
mnaserbuuut i dont think there's a date18:25
sdakemnaser right - perhaps i inferred one from this:18:26
sdake"There are far too many new and exciting things in Zuul v3 to cover in a18:26
sdakesingle post, and many of the topics (such as Ansible Jobs, or Job18:26
sdakeInheritance and Reuse) are deep topics we can dive in to over time. The18:26
sdakelong and short of it is that Zuul v3 is coming soon to an OpenStack18:26
sdakeInfra near you, so expect more and more communication about what that18:26
sdakemeans over the next few months."18:26
sdaketwo months + middle of march = middle of june18:26
mnaseri guess that's a fair way of putting it18:26
sdakehmm off by 1 error, may at the earliest :)18:26
sdakein any regard, few months of communication + some finalizatino time - inc0 still dont think we want to b lock18:28
inc0depends on what you mean by block18:28
inc0I'd block all major refactors on it, yes18:28
inc0not fixes of what's broken18:28
inc0also if someone actually wants to make big refactor before that - I'll be happy to review and help18:29
inc0and won't -218:29
inc0maybe I phrased it incorrectly - I'll, myself, will work on multinode gates, upgade gates and all-out gate refactoring18:30
inc0but after zuulv318:30
sdakeinc0 right sounds good18:30
inc0any major gate changes disrupts development of other features18:31
inc0especially if we're talking voting18:31
inc0I want to limit these disruptions by ensuring that we'll work efficiently18:31
inc0and zuul v3 will allow us to be much more efficient18:31
portdirectword up - back o/18:33
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155018:35
*** gfidente has quit IRC18:37
*** mgoddard has joined #openstack-kolla18:37
*** rhallisey has quit IRC18:37
*** rhallisey has joined #openstack-kolla18:38
*** rwellum has quit IRC18:40
v1k0d3nhey all good afternoon18:43
*** david-lyle_ is now known as david-lyle18:43
*** alop has joined #openstack-kolla18:48
alophas anyone seen this error deploying compute-kit:18:48
alopl3-agent-daemonset.yaml" at <eq $restype "compute...>: error calling eq: invalid type for comparison18:49
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155018:51
*** unicell has joined #openstack-kolla18:56
*** unicell has quit IRC18:57
v1k0d3nsdake, i'm just reading back...and i really wish i didn't have to comment on this (trust me)18:59
v1k0d3nbut size actually came up from several folks at the fishbowl in barcelona. inc0 was there. i know he know this to be true.18:59
v1k0d3nportdirect: was actually one of those as an individual contributor.19:00
v1k0d3nSAP pleaded for a roadmap to reduce size.19:00
v1k0d3ni know it has to be done the way that kolla is today, but to say that it's not a concern isn't the whole picture/story.19:00
v1k0d3nand that's my only .02 out of that whole coversation earlier with SamYaple19:01
*** unicell has joined #openstack-kolla19:01
*** Jezekus has joined #openstack-kolla19:01
sdakethe most common arguments i have heard for size reduction are tied up in CI/CD, not the actual fact that the images take 30gig on disk19:02
sdakeimage size reduction is about increasing performance of CI/CD mostly - not deployment performance19:03
sdakesolving CI/CD performance naturally alleviates people's size concerns19:04
sdakedeployment performance of kolla is 20 minutes on 123 node cluster19:05
v1k0d3nbut if there is a vulnerability common to base, that's not exactly ideal.19:05
openstackgerritMohammed Naser proposed openstack/kolla master: Add mechanism to store keys locally to avoid going out to network  https://review.openstack.org/44710119:05
sdakebuilding 170 images individually in CI/CD takes hours/days19:05
sdakebuild wold works perfectly well and is very performant - many people want a faster CI/CD pipeline that doesn't "build world" but instead "builds indivdiually"19:06
jascott1doesnt `squash` help with size? or at least when its not beta feature anymore19:06
sdakejascott1 it does help with size19:06
sdakejascott1 it doesn't help with ci/cd build performance19:06
sdakejascott1 as most ppls that are deploying kolla want to put a ci/cd pipeline in front of the docker image creation19:06
sdakeand they do that by using jenkins or zuul to build each container indepndently19:07
sdake(or the DBS)19:07
kfox1111hi19:07
jascott1hey kfox111119:07
sdakehey kfox111119:07
jascott1whats up on this fine Friday?19:07
sdakealop had a question above - which I am unable to answer19:07
sdakejascott1 ITS FRIDAY19:07
jascott1\o/19:07
alopsdake: I think something is wrong with my test setup19:08
sdakealop i see - helm version work for you?19:08
jascott1kfox1111 was that issue with child->parent imports limitation I just posted a problem?19:09
kfox1111jascott1: I responded just now.19:09
kfox1111I don't think so?19:09
jascott1oh ok great19:09
kfox1111but maybe you have a use case in mind that I don't?19:09
alopyeah, I think my installer picked a much newer version than what is expected19:09
jascott1im not on VPN atm so not getting the mails19:09
sdakealop what is the output of helm version?19:10
alop2.2.219:10
alopI think that's my problem19:10
sdakelet me run 2.2.2 throug hthe gate19:10
alopI notice that the Dockerfile in kolla-kubernetes specifies 2.1.319:10
kfox1111should be 2.2.2 everywhere, unless we missed a place.19:11
kfox1111we should be requiring 2.2.x now.19:11
alopI can update the dockerfile19:11
*** rwallner has quit IRC19:12
kfox1111ah. yeah. its wrong in the docker file. :/19:12
kfox1111we really need to gate the dev docs. :/19:12
kfox1111so easy to break them. :/19:12
mnasersdake just so i can keep this on my mind and give it thought over the next few days.  so the issue here is that a change in dockerfile for nova-novncproxy will result in $world being rebuilt, when in reality, we only need to rebuild nova-novncproxy because the rest did not change19:12
sdakemnaser if you run in jenkins, that is the case as jenkins doesn't keep the existing docker cache in place19:13
sdakemnaser you can build each image indiviudallly (with layers)19:13
mnaserso ideally CI would just rebuild nova-novncproxy, tack it into the rest of the prebuilt and run CI against it19:13
sdakemnaser and then each image takes less time to build (rather then  buidl world)19:13
sdakemnaser yup19:14
sdakejenkins provides a  fresh environment each time19:14
sdakedocker cache is removed per ci/cd run19:14
kfox1111yeah. with some kind of container fingerprinting,19:14
kfox1111ci/cd can then deterimine which doesn't need rebuilding all the time.19:14
sdakekfox1111 ya like that docker hash that containers have ;)19:14
kfox1111sdake: no, thats not sufficient.19:15
sayantani01Can someone please point me to the document which talks about various interfaces? network interfaces, api interfaces etc?19:15
sdakekfox1111 i see19:15
kfox1111sdake: cause the hash changes all the time for things that don't matter.19:15
mnaserand to build the hash, you'd have to build the images19:15
mnaserwhich brings you back to square one19:15
mnaserso how long can a docker image tag be19:15
mnaseraka what about tagging images in a common registry with service:<md5ofdockerfile>19:16
kfox1111mnaser: true. I guess I'm thinking, you don't have to ci/cd the artifact if it hashes to an already tested/deployed thing.19:16
kfox1111so no expensive deployment testing, or notifying admins they need to roll it into production, etc.19:16
mnaserso you host an internal registry where instead you tag things based on the md5 of the dockerfile19:16
sdakekfox1111 - quick question - https://github.com/openstack/kolla-kubernetes/blob/master/tools/test.sh#L10-L1119:16
sdakei see 2.1.3 templates used19:17
kfox1111mnaser: doesn't work either. repo deps change.19:17
sdakeand 2.2.0 is deployed in gate19:17
sdakewe ideally want to be using 2.2.219:17
sdakewhich templates should be used?19:17
mnaserthis issue seems so straightforward yet so complicated, heh.19:17
kfox1111sdake: yeah, probablhy should update that one too. I think only 2.2.0 was out at the time.19:17
sdakekfox1111 i can update 2.2.0 to 2.2.2 what should I update the templates to?19:18
kfox1111sdake: we probably should try and figure out how to make ONE and only one variable that defines which version to use, to simplify tweaking it.19:18
kfox1111sdake: not sure if there is a newer version of templates.19:18
sdakekfox1111 for the moment i only want to run a test, alop indicates he ran 2.2.2 and got a helm error19:18
alopFWIW - I'm not doing *exactly* the same thing19:18
kfox1111mnaser: yeah. if it was easy, it would be fixed in the docker community a long time ago. :/19:19
kfox1111mnaser: I think its something like, (rpm -qa | sort; pip list ...| sort ) | md5sum19:19
mnaseri mean if you look at how devstack gate runs19:19
mnaserhow would that differ between what kolla does19:20
mnaserbuild and install everything then run tests19:20
mnaserand that's a model that has worked for quite sometime19:20
kfox1111devstack isn't for production though.19:20
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Update to Helm 2.2.2  https://review.openstack.org/44710219:20
mnaserright19:20
kfox1111its for building dev stuff.19:20
mnaseri may be biased because i'm thinking this in the scope of openstack ci only19:20
kfox1111I want a much more clear pipeline for prod.19:20
mnaserand not some on site instance at $company that is doing CI before deploying19:20
kfox11111. artefacts are produced.19:21
kfox11112. artefacts are tested.19:21
kfox11113. artefacts are used to stand up a production system.19:21
mnaserTIL you can spell it as artefact19:21
kfox11114. step 1 is repeated and checked to see if actually different.19:21
sdakemnaser ;-)19:21
kfox11115. step 4 things tested.19:22
kfox11116. prodcution system updated using stuff from step 419:22
sdakealop right - are you installing helm templates per chance?19:22
sdakehttps://github.com/technosophos/helm-template/releases/download/2.2.2%2B1/helm-template-linux-2.2.2.1.tgz19:22
sdakealop if so, you may need this helm template to go along with helm 2.2.219:22
sdakealop i just submitted a review to test 2.2.2 against the gate19:22
sdakethe gate was previously on 2.2.019:23
alopI'll try that out, thanks19:23
sdakealop rather currently on 2.2.0 of helm and 2.1.3 of the templtews19:23
*** kbaegis has joined #openstack-kolla19:23
kfox1111I guess those two versions are compatible and only used for testing the template rendering, not talking with tiller, so never broke the gate.19:24
sdakekfox1111 the templates do what then?19:28
openstackgerritMohammed Naser proposed openstack/kolla master: Add mechanism to store keys locally to avoid going out to network  https://review.openstack.org/44710119:28
sdakealop rather - the gate is currently on 2.2.0 - not previously - if your using a recent git checkout - it will default to 2.2.019:29
sdakealop although if your not using the dev_env.sh script it may do somethign different19:29
hogepodgeinc0: fwiw, I can't get the kolla-ansible bifrost container to work. One problem is the binaries in the virtualenv aren't being put into the exec path, so ironic can't find them. When I fix that iscsi falls over when ironic tries to provision machines with it19:29
sdakewe are documeing a non-dev_env deployment guide in etherpad19:29
alopright, I think I'm on the right path now, thanks sdake19:29
sdakealop all godo thanks :)19:30
kfox1111sdake: theres a py27 check that does some sanity checks on the rendered templates.19:30
*** Pavo has joined #openstack-kolla19:30
kfox1111the helm template plugin allows you to render templates without having to install k8s/tiller just to spit out some yaml. :)19:30
sdakekfox1111 i see - so linting more or less19:30
kfox1111yeah.19:30
sdakekfox1111 i attempted to customize the image_name_full btw19:31
sdakekfox1111 and found that doesn't work for the nova bootstrap init job19:31
sdakekfox1111 i looked at the helm chart - it looks like it reference a toolbox container19:31
sdakekfox1111 any idea how to decipher that?19:32
kfox1111yeah, we need to come up with a document that describes how each package can be configured. or a tool or something.19:32
sdakekfox1111 well i did it with cloud.yaml19:32
sdakekfox1111 i was able to laod the custom image name for nova19:32
sdakehowever nova's db create job didn't seem to use the same image19:32
kfox1111sdake: see: helm/kolla-common/templates/_common_create_db_job.yaml19:33
sdakei saw that - it looks like it pulls in toolbox19:33
kfox1111it does use kolla_toolbox to do the db creation.19:33
sdakekolla_toolbox contains every db bootstrap then?19:34
sdakethe container was in a crash loopbackoff19:34
sdake(the db create job)19:34
sdakenova itself was in an init state iirc19:34
kfox1111I think the create db is basically just 'mysql create db; mysql create user; mysql perm user x to db y'19:34
kfox1111the manage-db is service specific, which loads the schema.19:35
sdakei see19:35
*** tonanhngo has joined #openstack-kolla19:35
*** tonanhngo has quit IRC19:35
kfox1111firefox can't get a feature to allow you to figure out what tab is eating up 100% cpu fast enough IMO. :/19:36
sdakekfox1111 how does one go about customizing the image name in kolla-kubernetes?19:41
sdakekfox1111 have image tag customization sorted out19:41
*** manheim has joined #openstack-kolla19:41
kfox1111we support either tweaking just the tag, or setting image_full.19:41
sdakeOH19:41
sdakeimage_full = centos-source-nova-api:4.0.0.A?19:42
kfox1111if you don't follow the naming convention for image name, then you might not follow it for tags too.19:42
kfox1111yeah.19:42
sdakei didn't have the tag in there19:42
kfox1111and namespace too.19:42
kfox1111and hub if you got one.19:42
sdakeya i did have docker.io/kolla/centos-source-nova-api19:42
sdakeand that didn't work19:42
openstackgerritMohammed Naser proposed openstack/kolla master: Speed up builds by increasing threads for builds  https://review.openstack.org/44670119:42
*** Pavo has quit IRC19:42
kfox1111the issue is, image name is built out of components normally.19:43
sdakei would think if the tag overrides it overrides always19:43
kfox1111$distro-$[binary/source]-nova-api19:43
openstackgerritMohammed Naser proposed openstack/kolla master: Remove global privilege escalation for deploy_aio.sh  https://review.openstack.org/44664519:43
kfox1111well,19:43
*** shardy has quit IRC19:44
kfox1111$reposerver/$repo/$distro-$[binary/source]-nova-api19:44
kfox1111if you follow the pattern, you can override any of those things.19:44
kfox1111if you deviate from the pattern, you must specify it all as part of _full19:45
sdakeand _full must include a tag?19:45
kfox1111yeah.19:45
kfox1111_full is handed directly to image:19:45
sdakeimage_short should be a thing ;-)19:45
kfox1111people will be shocked to hear it from me, but, I think thats too much configurability. ;)19:45
sdakeor image_name19:45
*** tonanhngo has joined #openstack-kolla19:46
sdakeperhaps image_full_and_tag would help clarify it19:46
kfox1111is image_name = nova-api, and the centos-source override stuff tacked on?19:46
sdakeimage_name = nova-api19:46
kfox1111image_full is litterally the full thingy.19:46
*** vhosakot has joined #openstack-kolla19:46
sdakekfox1111 right got it19:47
kfox1111thats why its full. :)19:47
inc0kfox1111: can you make it to next weeks meeting?19:47
kfox1111otherwise, we get image_full_with_type_and_distro_and_repo_and.... for clarity. :)19:47
inc0we have one thing to talk about which needs your presence19:47
kfox1111inc0: I'm going to try19:47
kfox1111k. Ill try hard. :)19:47
inc0thank you good sir;)19:47
kfox1111np. :)19:47
inc0we can move it around19:47
inc0just let me know19:48
inc0but quick context - canonical has it's own "proper" way to deploy k8s19:48
inc0and offered help to work with us on CI and stuff to streamline it19:48
kfox1111cool.19:49
kfox1111which 'it' are we talking about?19:49
kfox1111kolla-kubernetes?19:49
inc0yup19:49
kfox1111cool. :)19:50
inc0gates and/or dev guide19:50
* srwilkers sees charms in the future19:51
sdakealop looks like 2.2.2 with the templates works19:51
*** Jezekus has quit IRC19:51
sdakealop https://review.openstack.org/#/c/447102/19:52
kfox1111srwilkers: thats some of that alternative orchestration options I've been saying someone will want to code at some point or another. :)19:52
kfox1111sdake: cool.19:52
srwilkerskfox1111, :)19:52
sdakekfox1111 it appears atlaeest to pass the gates - which are extensively annoying :)19:53
sdakekfox1111 are we waiting on something to make deploy gates voting?19:53
*** kbaegis has quit IRC19:53
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155019:54
kfox1111sdake: no, I think we can make them voting at any time?19:54
sdakekfox1111 i will make it so19:54
kfox1111sdake: we've been treating them as voting for a long time.19:54
kfox1111k.19:54
sdakekfox1111 after i get back from a tour of a gaming office today ;)19:54
sbezverkkfox1111: I am not sure it is a good idea to make voting gates version 2.0.0 jobs19:56
sbezverkkfox1111: it seems you can run just bare basic things with this version and it should not be our target19:57
kfox1111sbezverk: they test a lot of stuff 3.x currently doesnt.19:57
sbezverkkfox1111: 2.0.0 cannot test lots of things too19:57
sdakeare the 4.0.0 helm charts going to be compatible with 2.0.0 charts?19:57
kfox1111yeah. but that doens't meen they shouldn't be voting?19:57
sdakegiven the placement api thing19:57
kfox1111sdake: yeah. placement is purely additive.19:58
sdakethere was something else to 4.0.0 -forget which19:58
sbezverksdake: kfox1111: running old images adds complexity19:58
sbezverkas you need to check which features are supported which not19:58
kfox1111not testing old images doesn't allow easy migration to kolla-kubernetes.19:58
sdakesbezverk yup we had this discussion in kolla-ansible a long long time ago19:58
sdakeand pretty much decided to only implement master plus latest release19:58
kfox1111once we're closer to a 1.0, I'm totally for dropping all older versions and only doing one version at a time per release.19:59
sbezverksdake: right19:59
kfox1111with the quick turnover in functionality in the repo though, cutting off a version from trunk basically seals its fate as not an option for a migration path.19:59
kfox1111if it was a large burdon to carry, I'd say we drop it. but I don't think it currently is.20:00
kfox1111and a migration path for folks with older clouds has some big benifits.20:00
*** dave-mccowan has quit IRC20:07
sbezverkso many version makes life more and more difficult, now this damn cells :(20:08
kfox1111new versions are making things difficult.20:09
kfox1111not sure the old ones are.20:09
kfox1111but ops have known that for a long time. :/20:10
kfox1111devs like to break stuff all the time moving forward. :/20:10
sbezverkkfox1111: but it is very hard without moving forward20:11
*** rwallner has joined #openstack-kolla20:12
*** rwallner has quit IRC20:17
kfox1111what is?20:17
kfox1111not breaking things?20:17
sbezverkkfox1111: no :) developmnt..20:18
kfox1111you can move forward and develop stuff without breaking things. :)20:18
kfox1111or you can break things and pass the buck to the ops to deal with.20:18
kfox1111sometimes breaking things really is the right solution.20:18
kfox1111but often its just becuase of lack of developer attention.20:18
openstackgerritMohammed Naser proposed openstack/kolla master: Add mechanism to store keys locally to avoid going out to network  https://review.openstack.org/44710120:22
openstackgerritMohammed Naser proposed openstack/kolla master: Speed up builds by increasing threads for builds  https://review.openstack.org/44670120:23
sbezverkkfox1111: could you check this, http://paste.openstack.org/show/603184/ ny head is about to explode from training :( by some reason it does not run all caoomnds, do you see something wrong?20:24
*** manheim has quit IRC20:26
mnaseri think your pipe is breaking things sbezverk20:26
mnaser&& (yes | cp -aLf {{ $containerConfigDirectory }}/..data/nova.conf /etc/nova/nova.conf) \20:27
sbezverkmnaser: have noi choice since nova.conf already exists20:27
sbezverkneeds to overwrtie it wihtout prompt20:27
mnaseryes i know20:27
mnaserbut if you add the parantheses(spelling?) around it20:28
mnaserit will run that command as a unit20:28
mnaserinstead it's piping the whole thing and creating a mess20:28
sbezverkmnaser: ah, true, let me try it, thanks for the idea20:28
mnaseralso sbezverk its probably the bash alias breaking things20:28
mnaserinstead of that20:28
mnaserjust do /bin/cp20:28
mnasercp is aliased to cp -i usually20:28
sbezverkmnaser: cool will do both ;) just in case20:29
*** jmontleon has quit IRC20:30
kfox1111sbezverk: looking20:32
*** jmontleon has joined #openstack-kolla20:32
*** jtriley has quit IRC20:32
kfox1111sbezverk: looks ok to me.20:33
kfox1111the pipe only shoudl go from the yes to the cp, but not sure the follow on &&'s error behavior.20:34
kfox1111parans would help there I think.20:34
sbezverkkfox1111" thanks20:35
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155020:35
*** srwilkers has quit IRC20:40
*** kbaegis has joined #openstack-kolla20:41
*** Serlex has quit IRC20:50
*** eaguilar has quit IRC20:52
*** eaguilar has joined #openstack-kolla20:52
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155021:10
*** ipsecguy has quit IRC21:21
*** ipsecguy has joined #openstack-kolla21:22
*** sbezverk has quit IRC21:23
*** Jeffrey4l has quit IRC21:35
*** schwicht has quit IRC21:36
*** Jeffrey4l has joined #openstack-kolla21:36
*** kbaegis has quit IRC21:37
*** srwilkers has joined #openstack-kolla21:40
*** lamt has joined #openstack-kolla21:43
*** srwilkers has quit IRC21:53
*** lamt has quit IRC21:53
*** lamt has joined #openstack-kolla22:01
*** eaguilar has quit IRC22:10
*** sbezverk has joined #openstack-kolla22:12
*** m1dev has quit IRC22:12
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155022:16
*** saneax-_-|AFK is now known as saneax22:19
*** kbaegis has joined #openstack-kolla22:19
*** schwicht has joined #openstack-kolla22:22
*** kbaegis has quit IRC22:27
*** kbaegis has joined #openstack-kolla22:27
openstackgerritKevin Fox proposed openstack/kolla-ansible master: WIP: Fix for Kolla-Kubernetes Fernet Tokens  https://review.openstack.org/44715922:31
*** alop has quit IRC22:31
*** kbaegis has quit IRC22:32
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support  https://review.openstack.org/44627422:32
sdakeportdirect kfox1111 sbezverk how about some reviews on https://review.openstack.org/#/c/447102/22:34
sdakeor inc0 if your about22:34
kfox1111looking22:34
kfox1111+222:35
sdaketx22:35
kfox1111we need to split that those values out somewhere though, so we have a shared shell script with those versions in it.22:35
kfox1111so we don't keep forgetting where all the places are. :/22:35
sdakeyou mean like a versions.sh or something?22:35
kfox1111yeah.22:35
sdakei'll see if i can sort that out22:35
kfox1111cool. thanks. :)22:36
*** rwallner has joined #openstack-kolla22:37
*** rwallner has quit IRC22:42
openstackgerritMerged openstack/kolla-kubernetes master: Update to Helm 2.2.2  https://review.openstack.org/44710222:47
*** kbaegis has joined #openstack-kolla22:49
*** kbaegis has quit IRC22:50
*** schwicht has quit IRC22:51
*** kbaegis has joined #openstack-kolla22:52
*** kbaegis has quit IRC22:56
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support  https://review.openstack.org/44627422:57
*** lamt has quit IRC22:59
*** jbadiapa has quit IRC23:01
openstackgerritSteven Dake proposed openstack/kolla-kubernetes master: Remove EXPORT and use search and replace instead  https://review.openstack.org/44716423:06
sdakeportdirect kfox1111 ^^23:06
*** mgoddard has quit IRC23:10
openstackgerritSayantani Goswami proposed openstack/kolla-ansible master: Added playbooks for snap deployment  https://review.openstack.org/44616523:11
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Update Searchlight resource_plugin configuration  https://review.openstack.org/44262823:11
*** erlon has quit IRC23:15
*** schwicht has joined #openstack-kolla23:17
openstackgerritSayantani Goswami proposed openstack/kolla-ansible master: [WIP] Added playbooks for snap deployment  https://review.openstack.org/44616523:18
*** eaguilar has joined #openstack-kolla23:21
*** kbaegis has joined #openstack-kolla23:24
*** kbaegis has quit IRC23:24
*** eaguilar has quit IRC23:27
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support  https://review.openstack.org/44627423:29
*** unicell has quit IRC23:31
sdakekfox1111 around?23:51
sdakekfox1111 next week is mostly clear for me23:51
sdakekfox1111 plan to make external config helm charts23:51
sdakekfox1111 do we need one per microchart - or one per service?23:51
sdakesbezverk did the ironic service chart land?23:53
*** kbaegis has joined #openstack-kolla23:54
spsuryamorning all23:58
openstackgerritKevin Fox proposed openstack/kolla-kubernetes master: WIP: Fernet Token Support  https://review.openstack.org/44627423:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!