Thursday, 2016-08-18

*** thumpba_ has joined #openstack-kolla00:03
*** bmace has quit IRC00:03
*** zhurong has quit IRC00:04
*** thumpba has quit IRC00:05
*** jtriley has joined #openstack-kolla00:09
*** severion has quit IRC00:13
*** v1k0d3n has joined #openstack-kolla00:13
*** Jeffrey4l has joined #openstack-kolla00:16
*** bmace has joined #openstack-kolla00:20
openstackgerritJeffrey Zhang proposed openstack/kolla: Bump the ansible version to 2.1.1.0 in kolla-toolbox  https://review.openstack.org/35631100:28
*** duonghq has joined #openstack-kolla00:34
*** sdake has joined #openstack-kolla00:35
sdakepavo back alive00:36
duonghqmorning sdake00:37
sdakeduonghq afternon actuallyy00:38
sdakepowernap + powerdinner00:38
sdakeready to rck00:38
sdakeany9one need any help00:38
duonghqwhat is current temperature sdake?00:39
sdakehot00:39
duonghqdose the winter is better?00:39
sdake5pm00:39
sdakewinter is much better00:39
sdakeprorbably 105-1110 atm00:39
sdake110 that is00:40
duonghqgood weather, huh? does it rain?00:41
duonghqI mean winter00:41
sdakeduring summer yes00:42
sdakeduring winter no00:42
sdakeduring summer we have monsoons00:42
duonghqmonsoons festival?00:43
sdakemonsoons are like a big rain storm00:43
duonghqah00:43
sdakewhich cools off the heat00:43
sdakepeopel in arizona are angry during summer time00:43
sdakeand happy during winter00:43
sdakeits kind of wierd00:43
sdakei think its opposite to the rest of the us ;)00:44
duonghqthere is storm come here in about 1-2 days :(, long time no storm go here00:44
sdakeangry might be the wrong word00:44
sdakedispleased might be better00:44
duonghqI like the winter, too00:44
sdakei've lost 10 punds in the lat 1 month because of the heat affecting my appetite00:44
sdakecontinuous problem in ariona00:44
sdakearizona isn't problem free00:45
sdakebut all the datacentres in the us are here00:45
sdakebecuase nothing ever goes wrong00:45
sdakeexcept its hotter then hell ;)00:45
duonghqyay00:45
sdakeno tornadoes no earthquakes no nothing00:46
duonghqjust hot?00:46
sdakeyup just hot00:46
sdakeand 280% murder rate above avereage00:46
sdakealthough i don't live in that part of town00:46
sdaketown/city00:47
sdakemurder rate increases in summer time - presumably because of resource constraints00:47
duonghqbad "habbit"00:47
*** jtriley has quit IRC00:48
sdakei feel safe all the time00:48
sdakei dont even lock my doors00:48
sdakeothers - in other parts of town - have 5 deadbolts on their door00:49
duonghqso good for you00:49
sdakelocation location location00:49
wirehead_Most of the year, it’s the valley of the sun.  Part of the year, it’s the surface of the sun.00:49
sdakewirehead_ wow - never picked up on that00:50
duonghqthe chant spirit from list meeting still around,00:50
sdakewirehead_ surpried- and i'm 42 years old :)00:50
wirehead_One of my grand-bosses at Rackspace lived in Phoenix and said that.00:50
sdakehe must have been grand00:50
sdakeand more experienced then I00:51
sdakewirehead_ do you use jabber00:51
wirehead_I think I’m supposed to, but I don't.00:51
sdakei see your not on00:51
sdakewould you mind indulging me for a moment00:51
sdakebattery at 2% brb if power dies out00:54
wirehead_K, well the Jabber client is forcing me to download a 65 meg update.  But I think I’m on.00:55
sbezverkwirehead_ I see some issue with neutron boostrap, wondering if you see something similar00:56
sbezverkwirehead_ when it tries to run db upgrade on neutron db and is it was already upgraded, I see continuous restart of bootstrap process00:57
wirehead_hmmmm.00:58
sbezverkwirehead_ wondering if there is way to check if db was already upgraded and then ignore upgrade00:58
*** sdake has quit IRC00:59
Pavosdake cool, I am setting up nodes agan00:59
wirehead_That seems like a bug in the update process.00:59
Pavolol like for the 6th time00:59
duonghqsdake: how about the SAP case?01:00
sbezverkwirehead_ most likely, they should handle it more gracefully01:00
*** jtriley has joined #openstack-kolla01:00
sbezverkwill open a bug tomorrow against neutron01:01
Pavoso whats the most stable release 2.0.3?01:01
*** sdake has joined #openstack-kolla01:02
duonghq sdake: how about the SAP case?01:03
sdakewirehead_ around on jabber?01:03
sdakei lost power01:03
sdakeduonghq huh?01:03
duonghqSAP use our work without license include01:03
sdakeduonghq someone told me that had been fixed01:04
wirehead_sbezverk: When I test, I usually nuke everything and start from scratch.01:04
sdakehave a link to the offending repo?01:04
*** zhurong has joined #openstack-kolla01:10
*** jtriley has quit IRC01:11
*** v1k0d3n has quit IRC01:11
*** v1k0d3n has joined #openstack-kolla01:11
*** sdake has quit IRC01:11
*** v1k0d3n has quit IRC01:14
*** v1k0d3n has joined #openstack-kolla01:15
duonghqsdake: still there?01:15
kbaegissdake I still need help :)01:17
*** chrichip has joined #openstack-kolla01:19
*** v1k0d3n has quit IRC01:23
*** dave-mccowan has quit IRC01:25
kbaegisI guess another way to go would be to build my own l2 agent01:26
duonghqkbaegis: from scratch?01:26
kbaegisNo alternative :)01:29
kbaegisduonghq so I guess so.  Centos works for EVERYTHING but ovs or linux bridging01:29
kbaegisthere's vikraman/gentoo01:29
duonghqhow about the ubuntu ones?01:29
*** huikang has joined #openstack-kolla01:30
kbaegisMight do.  I didn't test that.01:35
kbaegisI'll hardcode and reset.  I'm pretty sure it's a kernel version thing though01:38
kbaegiswho's responsible for the centos one ooc?  Could give me pointers?01:38
*** unicell has quit IRC01:39
sbezverksdake ping01:42
duonghqsbezverk: he is go out01:43
sbezverkduonghq thank you will try to catch him later01:43
duonghq(try stop openstack meeting bot from logging from last meeting, insane bug)01:45
duonghq#endmeeting01:45
openstackMeeting ended Thu Aug 18 01:45:50 2016 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)01:45
openstackMinutes:        http://eavesdrop.openstack.org/meetings/kolla/2016/kolla.2016-08-17-16.00.html01:45
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/kolla/2016/kolla.2016-08-17-16.00.txt01:45
openstackLog:            http://eavesdrop.openstack.org/meetings/kolla/2016/kolla.2016-08-17-16.00.log.html01:45
kbaegislol01:46
duonghqinsane bot01:46
duonghqhttp://eavesdrop.openstack.org/meetings/kolla/2016/kolla.2016-08-17-16.00.log.txt01:46
kbaegisI'm the most talkative!01:46
*** daneyon has joined #openstack-kolla01:47
duonghqJeffrey4l: can you review my patchset? https://review.openstack.org/#/c/356183/01:48
patchbotduonghq: patch 356183 - kolla - Port with_ loop variable syntax to Ansbile 2 (finish)01:48
duonghqkbaegis: really bad case, seem that you lost our meeting log01:49
duonghq*you -> we01:49
*** haplo37__ has joined #openstack-kolla01:50
*** daneyon has quit IRC01:52
*** thumpba_ has quit IRC01:52
kbaegisYeah, hope I didn't do anything :)01:56
duonghqkbaegis: somebody call startmeeting on this channel,01:59
Pavodoes this look normal during image build process02:02
Pavohttp://pastebin.com/S5v35xZ602:02
duonghqPavo: centos-based?02:02
Pavoyes02:03
duonghqI saw this sometime when yum try to find best mirror02:03
Pavoso... its normal then02:04
duonghqguess that, as long as this can install needed package02:04
Pavobecause looks like I am getting ti alot02:04
openstackgerritSerguei Bezverkhi proposed openstack/kolla: In kubernetes neutron-ovs-agent needs to run as root  https://review.openstack.org/35682702:05
duonghqdoes it get package eventually02:05
Pavolooks like it02:05
duonghqso, I'll ok, just wait for summary report, but it seem that the mirror is not in sync02:06
openstackgerritSerguei Bezverkhi proposed openstack/kolla: In kubernetes neutron-ovs-agent needs to run as root  https://review.openstack.org/35682702:07
openstackgerritSerguei Bezverkhi proposed openstack/kolla: In kubernetes neutron-ovs-agent needs to run as root  https://review.openstack.org/35682702:12
*** neophy has joined #openstack-kolla02:13
*** chrichip has quit IRC02:13
*** chrichip has joined #openstack-kolla02:15
*** diogogmt has joined #openstack-kolla02:17
*** diogogmt has quit IRC02:18
openstackgerritLeon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution  https://review.openstack.org/35618102:19
*** haplo37__ has quit IRC02:20
*** v1k0d3n has joined #openstack-kolla02:22
*** thumpba has joined #openstack-kolla02:23
*** mewald has joined #openstack-kolla02:24
*** mewald1 has quit IRC02:26
*** v1k0d3n has quit IRC02:28
*** caowei has joined #openstack-kolla02:32
*** unicell has joined #openstack-kolla02:37
*** inc0 has quit IRC02:38
*** coolsvap has joined #openstack-kolla02:45
*** chrichip has quit IRC02:45
*** yuanying_ has quit IRC02:46
*** chrichip has joined #openstack-kolla02:47
*** huikang has quit IRC02:47
*** huikang has joined #openstack-kolla02:48
*** bjolo has quit IRC02:51
*** huikang has quit IRC02:52
*** yuanying has joined #openstack-kolla02:52
openstackgerritJosh Lothian proposed openstack/kolla: Customizations for heka  https://review.openstack.org/34686103:00
openstackgerritLeon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution  https://review.openstack.org/35618103:03
openstackgerritMerged openstack/kolla: Fix tar command path for magnum conductor using ubuntu  https://review.openstack.org/35600803:07
openstackgerritMerged openstack/kolla: Skips a debug log during image build  https://review.openstack.org/35110803:08
openstackgerritLu lei proposed openstack/kolla: Fix Kolla home-page URL  https://review.openstack.org/35527503:10
*** huikang has joined #openstack-kolla03:12
openstackgerritMerged openstack/kolla: Minor doc fix  https://review.openstack.org/35606803:12
duonghqcoolsvap: are you there?03:13
coolsvapduonghq: yup03:13
duonghqas described in http://docs.openstack.org/developer/manila/devref/commit_message_tags.html03:14
duonghqthe TrivialFix tag is when the commit does not require the creation of a separate bug or blueprint in Launchpad03:14
openstackgerritMerged openstack/kolla: Move the keystone_*_url variable from common role to group vars  https://review.openstack.org/35631903:14
duonghqbut my Port .... commit is belong to bp ansbile203:14
duonghqso, it is still should have TrivialFix tag?03:15
coolsvapduonghq: http://docs.openstack.org/developer/kolla/CONTRIBUTING.html #403:15
*** thumpba has quit IRC03:15
duonghqah, thanks03:15
openstackgerritMerged openstack/kolla: Add a note about the sandbox environment to the CONTRIBUTING doc  https://review.openstack.org/35601703:15
coolsvapduonghq: wc :)03:16
coolsvapsorry i should have added it in review03:16
kbaegisSo since I'm trying to reverse engineer a specific container (openvswitchd), is there a doc/description on how kolla is actually using kolla_start?03:17
kbaegisIt would be really handy to know the actual command that's getting run on a functional container03:17
kbaegisI'm assuming that I can't simply run openvswitch on the host03:18
*** yuanying has quit IRC03:18
*** yuanying has joined #openstack-kolla03:19
openstackgerritDuong Ha-Quang proposed openstack/kolla: Port with_ loop variable syntax to Ansbile 2 (finish)  https://review.openstack.org/35618303:19
Pavodoes ansible have to be installed on all nodes, including target nodes?03:19
duonghqPavo: no03:19
duonghqansible using push model03:19
Pavohmmm prechecks are failing because no version is being reported from target nodes03:20
duonghqweird03:20
Pavoyeah03:20
coolsvapduonghq: we dont need both bp and TrivialFix03:20
Pavoalso I should only use ansible 1.9.4 right03:20
Pavonot 2.1.103:20
duonghqPavo: 2.1.x has trouble with OpenSSH 6.6,03:21
duonghqsad03:21
openstackgerritMerged openstack/kolla: Fix the Ubuntu Xenial build gate  https://review.openstack.org/35527403:21
Pavoso should use 1.9.403:21
*** coolsvap is now known as _coolsvap_03:21
duonghq_coolsvap_: thanks, next time I'll pay more attention03:22
kbaegisOh, I see what's going on here :)03:22
duonghqPavo: I'm looking for a way to upgrade may OpenSSH03:22
*** v1k0d3n has joined #openstack-kolla03:25
openstackgerritMd Nadeem proposed openstack/kolla: Added Ansible playbook for congress deploy  https://review.openstack.org/35062903:31
*** daneyon has joined #openstack-kolla03:36
duonghqJeffrey4l: so in this case, I should use TrivialFix or bp link?03:40
duonghqin Port ... case03:40
*** daneyon has quit IRC03:40
Jeffrey4lduonghq, bp is OK.03:40
Jeffrey4lPort?03:40
Jeffrey4lduonghq, you mean backport?03:40
duonghqah, the patchset Port with_ loop03:40
duonghqI don't know with tag is better03:41
duonghqfor this case03:41
Jeffrey4lduonghq, one of them both OK. choose one.03:42
*** yuanying has quit IRC03:42
*** inc0 has joined #openstack-kolla03:42
duonghqah, ok03:42
*** yuanying has joined #openstack-kolla03:42
openstackgerritDuong Ha-Quang proposed openstack/kolla: Port with_ loop variable syntax to Ansbile 2 (finish)  https://review.openstack.org/35618303:43
kbaegisgot it! :)03:43
duonghqJeffrey4l: fixed, thank you03:43
*** yuanying has quit IRC03:43
duonghqkbaegis: anything news?03:43
Jeffrey4lthanks.03:43
kbaegisI hacked a container03:44
duonghqwhich ones?03:45
kbaegisPretty simple actually.  Built ovs-vswitchd from source mapping it to my current kernel on a stage3, mapped through run, and found the command actually run by kolla03:45
*** huikang has quit IRC03:45
kbaegisAgent was working fine03:45
kbaegisSo I just had to hack the ovs03:45
*** huikang has joined #openstack-kolla03:46
duonghqah, nice03:46
*** ad_rien_ has joined #openstack-kolla03:47
*** ad_rien_ has quit IRC03:47
*** yuanying has joined #openstack-kolla03:48
openstackgerritMerged openstack/kolla: Fix neutron-server section in AIO sample inventory  https://review.openstack.org/35605603:50
*** huikang_ has joined #openstack-kolla03:50
*** huikang has quit IRC03:50
openstackgerritWei Cao proposed openstack/kolla: Remove duplicate items  https://review.openstack.org/35537703:56
*** v1k0d3n has quit IRC04:01
openstackgerritHui Kang proposed openstack/kolla: Add performance monitoring container to build config file  https://review.openstack.org/35685404:09
*** neophy has quit IRC04:19
openstackgerritWei Cao proposed openstack/kolla: Update instructions for ubuntu with systemd  https://review.openstack.org/34690004:22
*** neophy has joined #openstack-kolla04:26
*** inc0 has quit IRC04:30
*** v1k0d3n has joined #openstack-kolla04:31
*** sdake___ has joined #openstack-kolla04:38
sdake___hey folks04:39
sdake___my laptop is broken04:39
sdake___so I will be impaired a bit for 1-2 weeks04:39
sdake___but i have full access to my entire lab04:39
sdake___which i got operational today04:39
sdake___just an fyi04:40
duonghqsdake___, why?04:40
Pavothat sucks sdake___04:42
sdake___shit happens04:42
Pavothats why I have a mac they never break04:43
Pavolol04:43
sdake___ya thats what i got too04:43
Pavoouch how did you break it04:44
sdake___the power popped off as I was closing it, and it jammed between the screen and the keyboard as i was closing it04:44
Pavoah04:44
sdake___this busted the keyboard04:44
duonghqsdake___: do we have cleanup-container for multinode deploy?04:44
duonghqsdake___: unimageable case04:44
sdake___duonghq yes - you do04:44
Pavoduonghq kolla-ansible destroy -i ansible/inventory/multinode04:44
duonghqthank Pavo, sdake___04:45
sdake___duonghq i know pretty odd huh04:45
sdake___sorry this busted the screen not keyboard04:45
sdake___so ive been running it like that for about 3 weeks04:45
Pavoand you can scp the tool folder in kolla over to nodes and run the script separate04:45
sdake___and now it wont turn on04:45
sdake___i think the bad display probably burned out the power supply but who knows04:45
Pavoyeah those screen are expensive04:46
sdake___ya - may be chepaer t obuy a new laptop then get mine repaired04:47
sdake___screens are 800 bucks04:47
sdake___the power supply rebuild is about 1600 total04:47
sdake___i dont nkow what the total bil lwould be04:47
sdake___my mac is top of the line tho - so not sure04:47
Pavoyeah just buy a new one, they have skylake cpus now04:48
Pavoso on board graphics are pretty good04:48
Pavogrrr getting stuck at TASK: [neutron | Starting neutron-server container] during deployment04:52
duonghqfor how many min?04:52
Pavofor about 3 minutes now04:52
Pavoseems like its not finishing the bridging on the node04:53
*** chrichip has quit IRC04:53
*** chrichip has joined #openstack-kolla04:53
*** ajpower has joined #openstack-kolla04:53
Pavobr-ex and ovs-system interfaces are down04:55
Pavocould this be a ansible version issue?04:56
Pavonow docker0 interface is down also04:57
*** sdake___ has quit IRC05:00
*** fragatina has quit IRC05:01
duonghqno way05:01
Pavohow about kolla version05:02
Pavowas using stable/mitaka version05:02
Pavogonna try with latest now05:02
*** v1k0d3n has quit IRC05:08
Pavowell now a different issue, /kolla/centos-binary-openvswitch-db-server not starting05:13
Pavologs show /usr/local/bin/kolla_start: line 24: exec: start-ovsdb-server: not found05:13
duonghqah,05:20
duonghqmisalign docker and ansible version05:20
duonghq:)05:20
duonghqPavo,05:20
Pavorunning latest docker and latest ansible05:21
duonghqdo you clean image and rebuild?05:22
duonghquninstall kolla and install05:22
Pavojust rebuild that image05:22
Pavonow pulling it to nodes05:22
*** daneyon has joined #openstack-kolla05:24
duonghqPavo: did you get  'module' object has no attribute 'EEXISTS'", "path": "/etc/kolla//memcached", "state": "absent"} error05:27
Pavodon'tt think so05:27
Pavodamnit stuck back at the starting neutron server part again05:28
duonghqansible cannot create /etc/kolla/memcached folder05:28
duonghqPavo: what is the logs tell?05:28
*** daneyon has quit IRC05:28
Pavoits not generating logs in /var/lib/docker/volumes/kolla_logs/neutron05:29
Pavoand the container isn't running05:30
duonghqsound familiar05:31
Pavoguess I'll rebuild images to latest version05:34
Pavothat was using 2.0.3 images build05:34
duonghq2.0.3 image with latest ansible playbook?05:34
Pavoyeah05:34
Pavoguess thats bad huh lol05:35
duonghqthe ovsdb start command just change recently05:35
Pavowell it was getting stuck at the same spot using 1.9.4 ansible05:35
Pavoand stable kolla05:35
Pavowell its 1:30am here gonna start this build images cmd and head to bed05:37
Pavowill try again tomorrow05:37
duonghqbye now Pavo05:37
Pavosee ya laters05:37
*** mntdevops has quit IRC05:42
*** bootsha has joined #openstack-kolla05:42
*** bootsha has quit IRC05:47
*** bootsha has joined #openstack-kolla05:48
*** bootsha has quit IRC05:53
*** rstarmer has joined #openstack-kolla05:59
*** huikang_ has quit IRC06:06
duonghqdo I need root access for deploying Kolla multinode?06:06
*** neophy has quit IRC06:08
*** Mr_Broken has quit IRC06:09
*** Mr_Broken has joined #openstack-kolla06:09
*** rstarmer has quit IRC06:10
duonghqkbaegis: do you try multinode deploy?06:12
*** Mr_Broken has quit IRC06:14
duonghqmewald: do you deploy mulitnode with Kolla?06:27
duonghqmdnadeem: I see you join in osic testing? so you already do multinode deploy?06:29
openstackgerritJeffrey Zhang proposed openstack/kolla: Add warning to kolla-ansible destroy action  https://review.openstack.org/35695706:30
mdnadeemduonghq, yes, i have try it  once06:30
duonghqDo you need any special settings for deploy node?06:31
mdnadeemduonghq, I dnt think so06:32
duonghqjust normal user?06:32
mdnadeemduonghq, running kolla-ansible with sudo06:32
duonghqbut what for other nodes?06:33
duonghq(not deploy node)06:33
duonghqwhich ansible_user do you use?06:33
mdnadeemduonghq, What issue you gets06:33
duonghqah, what does control node meaning?06:33
duonghqI got error the ansible cannot create /etc/kolla/memcached06:34
mdnadeemduonghq, can you paste full log06:34
duonghqI'll try again with sudo and copy log06:35
mdnadeemduonghq, it seems permission issue, sudo will resolve it06:35
duonghqbut sudo just for deploy node06:35
mdnadeemduonghq, In case of multinode, you need to execute sudo kolla-ansible deploy from operator node only06:36
duonghqokay06:36
duonghqI don't know why I already ssh-copy-id to target node but still cannot access this node use public key06:37
duonghqtrying06:38
*** ad_rien_ has joined #openstack-kolla06:44
duonghqmdnadeem: I got this http://paste.openstack.org/show/560725/06:46
*** ad_rien_ has joined #openstack-kolla06:46
mdnadeemduonghq, what command you run?06:47
duonghqssudo -E kolla-ansible -i multinode deploy06:47
*** unicell1 has joined #openstack-kolla06:48
*** unicell has quit IRC06:48
*** Mr_Broken has joined #openstack-kolla06:48
duonghqthe inventory file: http://paste.openstack.org/show/560727/06:48
duonghq(just head of inventory)06:49
duonghqmdnadeem06:50
mdnadeemduonghq, it may be due to ansible_ssh_user=stack06:50
duonghqif I removed it, the user to ssh is None06:51
mdnadeemduonghq, remove it and try with sudo06:51
duonghqmdnadeem: It cannot ssh to target node, of cousre06:52
duonghqfatal: [k8s]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh.", "unreachable": true}06:52
mdnadeemhave u resolve the ip in etc/host?06:53
duonghqyes06:53
duonghqI can ping using hostname06:53
duonghqwith ansible_ssh_user, the prechecks command run success06:54
mdnadeemduonghq, i can deploy using http://paste.openstack.org/show/560729/06:56
duonghqfrom normal user?06:56
duonghqmdnadeem: do you have any ssh setting?06:57
duonghqlike ansible_ssh_key_file06:57
duonghqthe target nodes has any special settingS/06:57
mdnadeemduonghq, no06:57
duonghqmdnadeem: you set just control node?06:58
duonghqonly control node?06:58
duonghqfor test06:58
*** b_bezak has joined #openstack-kolla06:58
mdnadeemduonghq, just resolve host name in /etc/hosts06:59
duonghqmdnadeem: the ip and hostname is already in place06:59
mdnadeemduonghq, let me try once, will let u know if i missed something07:00
duonghqthanks mdnadeem07:00
openstackgerritJeffrey Zhang proposed openstack/kolla: Create the _member_ role in the horizon role  https://review.openstack.org/35608207:04
*** imcsk8 has quit IRC07:05
*** imcsk8 has joined #openstack-kolla07:06
*** dmueller-evoila has joined #openstack-kolla07:07
*** imcsk8_ has joined #openstack-kolla07:11
*** imcsk8 has quit IRC07:11
*** daneyon has joined #openstack-kolla07:12
*** dmueller-evoila has quit IRC07:15
*** daneyon has quit IRC07:17
*** dmueller-evoila has joined #openstack-kolla07:21
*** matrohon has joined #openstack-kolla07:26
*** dmueller-evoila has quit IRC07:31
duonghqmdnadeem: do user need nopasswd sudoer07:33
mdnadeemduonghq, you can do password less ssh login by adding id_rsa.pub entry of operator node to authorized key of deployed node07:35
duonghqalready do this07:35
duonghqmdnadeem: I mean do you make users in sudoers groups with nopasswd option?07:36
mdnadeemduonghq,  can you post output of ansible control -i mutinode -m ping -vvv07:37
*** fragatina has joined #openstack-kolla07:37
duonghqmdnadeem: without ansible_ssh_user option?07:38
mdnadeemduonghq, with user07:38
mdnadeemwith ansible_ssh_user07:39
duonghqwith ansible_ssh_user option, the ping is successful07:39
mdnadeemhmm07:39
mdnadeemcan you change ansible_ssh_user=root and try07:40
duonghqubuntu don't allow remote root login07:40
duonghqthe root is disable indeed07:40
*** fragatina has quit IRC07:42
*** matrohon has quit IRC07:43
mdnadeemduonghq, i will try to deploy and check it.07:47
duonghqthanks07:47
duonghqmdnadeem I don't understand why the config playbook can run without any escalate permission07:52
duonghqit copy file to system dir07:53
duonghqwirehead_, are you free for some minutes?07:55
*** berendt has joined #openstack-kolla08:17
berendtCan sombody please have a look at https://review.openstack.org/#/c/346444/08:26
patchbotberendt: patch 346444 - kolla - Optimize horizon cache and compress for http response08:26
*** mnasiadka has joined #openstack-kolla08:36
*** athomas has joined #openstack-kolla08:41
*** papacz has joined #openstack-kolla08:41
duonghqberendt: do you have any experience in multinode deployment?08:43
berendtduonghq i think so08:43
duonghqdo you need root user for deploy?08:43
openstackgerritJeffrey Zhang proposed openstack/kolla: Use dumb-init to manage the pid 1 process  https://review.openstack.org/35701508:44
duonghqI don't understand why ansible can copy file to /etc/kolla... without escalate priviledge08:44
berendtduonghq at the moment i use root for deployment, not using root is on my todo list, i think setting become in the ansible configuration file will work for kolla-ansible08:45
duonghqhmm, you don't use Debian-based as host?08:45
duonghqI grep in ansible dir and cannot find any become08:46
*** bootsha has joined #openstack-kolla08:50
*** Serlex has joined #openstack-kolla08:52
openstackgerritJeffrey Zhang proposed openstack/kolla: Sync the local_settings.j2 with horizon  https://review.openstack.org/35647108:53
*** daneyon has joined #openstack-kolla09:00
berendtduonghq i use ubuntu hosts09:04
duonghqberendt: So you enable root and deploy using this?09:05
berendtduonghq but i have not yet played with non-root user deployment09:05
berendtduonghq for the moment, yes09:05
*** daneyon has quit IRC09:05
duonghqberendt: okay, thanks :D09:05
berendtduonghq but this should be tested, i think it makes sense to work on this during the next cycle09:06
duonghqagree09:06
*** _coolsvap_ is now known as coolsvap09:06
duonghqbut I wonder if nobody test multinode deploy with non-root yet?09:07
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Vagrant doc updates  https://review.openstack.org/35324009:09
*** bootsha has quit IRC09:11
openstackgerritMerged openstack/kolla: Fix Kolla home-page URL  https://review.openstack.org/35527509:15
*** papacz has quit IRC09:18
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for aodh  https://review.openstack.org/34629909:20
duonghqJeffrey41: it seems that my patchset at https://review.openstack.org/#/c/356183/ pass almost test, expect ubuntu-related ones09:29
patchbotduonghq: patch 356183 - kolla - Port with_ loop variable syntax to Ansbile 2 (finish)09:29
duonghqbut afaik the ubuntu gate has trouble?09:29
duonghqJeffrey4l: it seems that my patchset at https://review.openstack.org/#/c/356183/ pass almost test, expect ubuntu-related ones09:29
patchbotduonghq: patch 356183 - kolla - Port with_ loop variable syntax to Ansbile 2 (finish)09:30
duonghq but afaik the ubuntu gate has trouble?09:30
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for murano  https://review.openstack.org/34634009:31
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for zaqar  https://review.openstack.org/34635709:38
*** berendt has quit IRC09:40
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for sahara  https://review.openstack.org/34659009:41
duonghqcoolsvap: https://review.openstack.org/#/c/356183/ I fixed my commit message, can you review my work?09:41
patchbotduonghq: patch 356183 - kolla - Port with_ loop variable syntax to Ansbile 2 (finish)09:41
coolsvapduonghq: i think i already +2ed it09:41
duonghqcoolsvap: but Jeffrey4l want me remove bp or trivialfix09:42
coolsvapalright09:42
coolsvapchecking09:42
*** zhubingbing has joined #openstack-kolla09:42
duonghqSeems that we have not fixed ubuntu gates09:43
*** papacz has joined #openstack-kolla09:44
*** berendt has joined #openstack-kolla09:44
duonghqand the commit: https://review.openstack.org/#/c/356235/ unblock doc generation at my pc09:46
patchbotduonghq: patch 356235 - kolla - Fix guide formating errors09:46
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for trove  https://review.openstack.org/34634209:48
openstackgerritOpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements  https://review.openstack.org/35693009:51
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for trove  https://review.openstack.org/34634209:52
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for sahara  https://review.openstack.org/34659009:52
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for murano  https://review.openstack.org/34634009:53
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for aodh  https://review.openstack.org/34629909:53
coolsvapJeffrey4l: ping09:56
*** zhurong has quit IRC09:58
openstackgerritMathias proposed openstack/kolla: Added influxdb role  https://review.openstack.org/34644909:58
mewaldany ideas what the comment means? https://review.openstack.org/#/c/346450/2009:59
patchbotmewald: patch 346450 - kolla - Added grafana role09:59
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update cirros image visibility in init-runonce  https://review.openstack.org/35393310:03
openstackgerritDuong Ha-Quang proposed openstack/kolla: Fix rabbitmq-clusterer template format  https://review.openstack.org/35706510:07
duonghqcoolsvap: can you review this commit, too https://review.openstack.org/#/c/356235/10:10
patchbotduonghq: patch 356235 - kolla - Fix guide formating errors10:10
*** bootsha has joined #openstack-kolla10:11
duonghqthis unblock doc generation at my machine10:12
*** ad_rien_ has joined #openstack-kolla10:13
openstackgerritWei Cao proposed openstack/kolla: Update instructions for ubuntu with systemd  https://review.openstack.org/34690010:14
*** berendt has quit IRC10:14
*** jrich523 has quit IRC10:15
*** berendt has joined #openstack-kolla10:15
*** egonzalez90 has joined #openstack-kolla10:15
coolsvapduonghq: I would recommend you sync with rhallisey regarding doc updates10:26
*** bootsha has quit IRC10:26
duonghqthank coolsvap for the contact point10:27
coolsvapsince he's reorganizing the docs10:27
duonghqthanks10:27
*** berendt has quit IRC10:31
openstackgerritDuong Ha-Quang proposed openstack/kolla: Improve rabbitmq-clusterer template code format  https://review.openstack.org/35706510:31
*** duonghq has quit IRC10:34
Jeffrey4lcoolsvap, pong10:34
coolsvapJeffrey4l: I am currently confused with the footer part in the base images with customization work10:34
coolsvapi feel we dont have it and we dont need it10:35
coolsvapbut I see mixed reviews open and merged10:35
coolsvapwhat do you think about it?10:36
Jeffrey4lcoolsvap, customization base can help us like: add custom repo?10:36
Jeffrey4lor change the default repo.10:36
Jeffrey4lcoolsvap, is there any difficulty to implement the customization base?10:37
coolsvapJeffrey4l: no10:37
coolsvapbut just a thought10:37
Jeffrey4lcoolsvap, like how about i want to use a higher ceph repo.10:37
Jeffrey4li mush change something in the base image.10:38
Jeffrey4lmust*10:38
coolsvapJeffrey4l: i get what you are trying to do here, but ceph is not the correct example for this imho10:40
coolsvapadding repo to the footer will make no sense in base images10:40
coolsvapJeffrey4l: to be clear i am talking about <<service>>-base images here10:41
coolsvapnot the base/openstack-base image10:41
Jeffrey4loh. I misunderstand.10:42
*** mnasiadka has quit IRC10:42
Jeffrey4lit is really useless to add the footer in the <<service>>-base image, i think.10:43
Jeffrey4lcoolsvap, ^10:43
Jeffrey4li can not image a use case for that.10:43
*** berendt has joined #openstack-kolla10:44
coolsvapJeffrey4l: yes10:44
openstackgerritEduardo Gonzalez proposed openstack/kolla: Update kubectl version at magnum-conductor container  https://review.openstack.org/35298610:44
Jeffrey4lBut 1. it is easy to add.   2. maybe some one need this: add something in all the nova-* images10:46
Jeffrey4lso i am OK to add this. coolsvap10:46
coolsvapfine with me as well10:46
coolsvapbut I was not too keen on adding this and having -1 comments for it10:46
coolsvapthats why I asked10:46
Jeffrey4lwe should have a agreement, whether we should add this feature. i -1 for this, just because other image do not have this.10:47
egonzalez90agree with Jeffrey4l, but it need to be documented so all people know that footers in base images. Actually there are a lot with footers and others without them10:47
Jeffrey4lwe have a agreement now. So let's add this in all base images.10:48
coolsvapyes10:48
coolsvapi already updated all my change sets for adding in base images10:49
Jeffrey4lcool10:49
*** daneyon has joined #openstack-kolla10:49
pbourkecoolsvap: Jeffrey4l: we're making use of the template customisations heavily in our internal fork10:49
pbourkecoolsvap: Jeffrey4l: and the base footers are needed imo10:50
coolsvappbourke: ack10:50
pbourkewe also potentially need headers10:50
Jeffrey4lyes. headers is also need.10:50
kbaegisMorning everyone10:52
kbaegisJeffrey4l: I hear that you're the one to thank for all the timely patches :) ty10:52
*** daneyon has quit IRC10:53
Jeffrey4lkbaegis, :)10:55
Jeffrey4lbrb10:55
*** coolsvap is now known as _coolsvap_10:55
*** egonzalez90 has quit IRC10:55
kbaegisAlright, testing my proposed hack11:00
*** egonzalez90 has joined #openstack-kolla11:01
openstackgerritMerged openstack/kolla: Updated from global requirements  https://review.openstack.org/35693011:04
openstackgerritWei Cao proposed openstack/kolla: Update instructions for ubuntu with systemd  https://review.openstack.org/34690011:06
*** caowei has quit IRC11:07
openstackgerritMerged openstack/kolla: Customizations for heka  https://review.openstack.org/34686111:08
*** neophy has joined #openstack-kolla11:08
*** JoseMello has joined #openstack-kolla11:09
*** neophy has quit IRC11:09
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Customizations for zaqar  https://review.openstack.org/34635711:11
openstackgerritJeffrey Zhang proposed openstack/kolla: Create the _member_ role in the horizon role  https://review.openstack.org/35608211:15
openstackgerritMerged openstack/kolla: Port with_ loop variable syntax to Ansbile 2 (finish)  https://review.openstack.org/35618311:16
openstackgerritMerged openstack/kolla: Customizations for Cinder  https://review.openstack.org/35058711:23
openstackgerritVladislav Belogrudov proposed openstack/kolla: Fix murano notification driver  https://review.openstack.org/35597911:23
openstackgerritSerguei Bezverkhi proposed openstack/kolla: In kubernetes ovs and l3 agents need to run as root  https://review.openstack.org/35682711:24
openstackgerritMerged openstack/kolla: Remove duplicate items  https://review.openstack.org/35537711:26
*** bootsha has joined #openstack-kolla11:27
openstackgerritMerged openstack/kolla: Build a more organized doc structure  https://review.openstack.org/34802911:27
sbezverkJeffrey4l: ping11:28
*** berendt has quit IRC11:29
*** berendt has joined #openstack-kolla11:30
*** papacz has quit IRC11:31
*** bootsha has quit IRC11:32
openstackgerritEduardo Gonzalez proposed openstack/kolla: Missing hosts at haproxy in site.yml  https://review.openstack.org/35586111:32
Jeffrey4lkbaegis, yo, what's it?11:33
Jeffrey4lsbezverk, pong11:33
kbaegisJeffrey4l:  I built a gentoo based container with openvswitch and mapped the kernel source into it11:34
kbaegisThat was having permission issues though.  I'm verifying this right now: ovs-vswitchd unix:/run/openvswitch/db.sock -vconsole:emer -vsyslog:err -vfile:info --mlockall --log-file=/var/log/openstack/openvswitch/ovs-vswitchd.log11:34
Jeffrey4lkbaegis, do u want the kolla to support gentoo?11:34
kbaegisJeffrey4l: I'd like you guys to be distro agnostic, yes :)11:35
kbaegisThat may or may not be feasible, and I understand either way.11:35
Jeffrey4lkbaegis, hmm that will be hard for the current kolla.11:35
sbezverkJeffrey4l: there was a concern about ovs. I am now testing it on kube11:36
sbezverkwould like to understand better11:36
*** Mr_Broken has quit IRC11:36
Jeffrey4lnow we have centos/ubuntu/debian/oraclelinux, we almost can not maintain them.11:36
kbaegisJeffrey4l: I've already been using ovs on my standard systems.  Trivial modification for me :)11:36
*** Mr_Broken has joined #openstack-kolla11:36
Jeffrey4lkbaegis, cool11:36
Jeffrey4lsbezverk, OK11:37
kbaegisJeffrey4l: Agreed.  You need either direct support from the distros or to abstract away and become agnostic11:37
kbaegisEither change is terribly difficult actually11:37
openstackgerritLeon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution  https://review.openstack.org/35618111:37
Jeffrey4lkbaegis, if we can reach that, that will be ideal11:38
kbaegissbezverk: There's fuel-ccp too11:38
kbaegissbezverk: Fuel microservices :) I'm avoiding using that for my lower layer11:38
kbaegisI really like the design behind kolla, plus gentoo is not an easy host for k8s11:39
*** Mr_Broken has quit IRC11:40
openstackgerritJeffrey Zhang proposed openstack/kolla: Sync the local_settings.j2 with horizon  https://review.openstack.org/35647111:42
*** haplo37__ has joined #openstack-kolla11:42
*** papacz has joined #openstack-kolla11:48
*** dwalsh has joined #openstack-kolla11:49
*** ayoung has quit IRC11:50
kbaegisDang11:56
kbaegisiptables are still breaking my deployment11:56
*** papacz has quit IRC11:56
kbaegistraceback, for any and all interested http://hastebin.com/qikeqiboza.py11:57
*** Mr_Broken has joined #openstack-kolla11:59
*** dave-mccowan has joined #openstack-kolla11:59
kbaegisWell, this looks like it's entirely neutron's fault12:05
kbaegisand the iptables_manager code12:05
*** kbaegis has quit IRC12:16
*** kbaegis has joined #openstack-kolla12:16
*** haplo37__ has quit IRC12:21
*** berendt has quit IRC12:22
*** papacz has joined #openstack-kolla12:24
*** bootsha has joined #openstack-kolla12:28
*** papacz has quit IRC12:29
*** bootsha has quit IRC12:34
*** athomas has quit IRC12:34
*** Mr_Broken has quit IRC12:34
*** Mr_Broken has joined #openstack-kolla12:35
openstackgerritJeffrey Zhang proposed openstack/kolla: Use dumb-init to manage the pid 1 process  https://review.openstack.org/35701512:39
*** athomas has joined #openstack-kolla12:39
*** egonzalez90 has quit IRC12:40
*** berendt has joined #openstack-kolla12:41
*** zhurong has joined #openstack-kolla12:45
*** imcsk8_ is now known as imcsk812:52
*** mnasiadka has joined #openstack-kolla12:54
openstackgerritJeffrey Zhang proposed openstack/kolla: Add pull.yml file for tempest  https://review.openstack.org/35718512:54
*** b_bezak has quit IRC13:00
openstackgerritJeffrey Zhang proposed openstack/kolla: DO_NOT_MERGE: Use better URL for China  https://review.openstack.org/32906013:01
*** b_bezak has joined #openstack-kolla13:01
*** v1k0d3n has joined #openstack-kolla13:01
*** papacz has joined #openstack-kolla13:03
*** papacz has quit IRC13:06
kbaegisWhere is kolla pulling the /etc/ directory from?  I don't see it mapped as a volume13:10
kbaegisBut kolla start is taking those from a file, correct?13:11
kbaegiscopying into /etc?13:11
kbaegisnvm.  also in /etc :)13:11
*** inc0 has joined #openstack-kolla13:12
inc0good morning13:13
kbaegisHAHA!13:13
kbaegisGot it.  :)13:13
kbaegisinc0: Good morning13:13
kbaegisOkay, so I need to write a post of my adventures getting kolla-on-gentoo.  Created my first instance, and required me to run the 'noop' firewall driver13:14
kbaegisBut it's up and running with ovs.  Created instances, volumes, networks, etc13:14
openstackgerritRoss Krumbeck proposed openstack/kolla: Ansible-ize OpenStack Designate  https://review.openstack.org/35326113:15
*** thumpba has joined #openstack-kolla13:17
*** berendt has quit IRC13:18
*** sdake has joined #openstack-kolla13:21
sdakepbourke around?13:21
sdakere https://review.openstack.org/#/c/356181/713:21
patchbotsdake: patch 356181 - kolla - Simplify install process for faster quickstart exe...13:21
sdakei really want to get this merged13:21
pbourkesdake: hey13:21
kbaegissdake: Sorry to hear about your laptop.  Also, works! Just spun up 32 networked instances on kolla/gentoo13:21
sdakeyou said "Clean up the first note about branches"13:23
sdakedo you mean the pip pin?13:23
pbourkesdake: i meant the initial note of master vs stable13:23
pbourkesdake: but yeah the pin needs to be removed also13:23
sdakei think that should say13:23
sdakestay13:23
*** mntdevops has joined #openstack-kolla13:23
sdakethe first part13:23
sdakewe can remove the pin tho - i just tested that works correctly13:23
sdakei had thought we were pubhlsing milestone releases to pypyi13:24
pbourkeok13:24
sdakeyou can see my rationale in the review13:24
sdakeplease can you respond to it13:25
sdakeso we can ge tthtis merged this morning13:25
sdakeif you disagree strongly and it will block the commit then i'll just accept what you want :)13:25
sdakethis causes me serious time sink13:26
sdakeand leon hs done a fantastic job writing up the solution13:26
sdakealthough i sort of f igured out all the stuff along with your help and dhellman and jrolls' help :)13:26
sdakethe part that cuses e time sink is the support on this point13:26
sdakenot reviewing...13:27
sdakejeffrey4l need your email address13:28
sdake_coolsvap_ need your email address13:28
pbourkesdake: as soon as the pin is removed +213:28
sdakemy outlook hasn't cached it yet13:28
Jeffrey4lsdake, zhang.lei.fly@gmail.com13:28
sdakepbourke great thanks13:28
*** egonzalez90 has joined #openstack-kolla13:28
*** ad_rien_ has quit IRC13:30
*** bootsha has joined #openstack-kolla13:31
*** daneyon has joined #openstack-kolla13:31
sdakebtw foolks my laptop is busted13:31
sdakei am working on a 3rd laptop i have available to me13:31
sdakei am a little bit impaired as a result13:32
sdakeno idea how long my other 2 laptops will take to get repaired13:32
sdakeprobably after 8/3113:32
*** ad_rien_ has joined #openstack-kolla13:33
*** ad_rien_ has quit IRC13:34
*** jtriley has joined #openstack-kolla13:34
*** bootsha has quit IRC13:35
sdakedamn missed chuck on cnbc13:35
*** daneyon has quit IRC13:36
_coolsvap_sdake: me@coolsvap.net13:36
openstackgerritMerged openstack/kolla: Customizations for trove  https://review.openstack.org/34634213:38
sdakebrb13:39
openstackgerritMerged openstack/kolla: Update kubectl version at magnum-conductor container  https://review.openstack.org/35298613:39
*** caowei has joined #openstack-kolla13:40
*** _coolsvap_ is now known as coolsvap13:42
*** sdake has quit IRC13:43
openstackgerritMerged openstack/kolla: Customizations for aodh  https://review.openstack.org/34629913:47
sbezverkJeffrey4l: ping13:48
Jeffrey4lsbezverk, pong13:48
sbezverkJeffrey4l: in kube there is no way to use socket13:49
Jeffrey4lsbezverk, socket? or ip:port?13:49
sbezverkip:port13:49
sbezverksorry13:49
sbezverkbecause there is no way to know in advance which ip will be used13:49
Jeffrey4lsbezverk, why? just treat it as a normal api service?13:50
sbezverkand ovs does not support 0.0.0.0:port13:50
Jeffrey4lsbezverk, hmm. isn't it?13:50
*** kbaegis has quit IRC13:50
Jeffrey4llet me test the 0.0.0.0:port format.13:50
Jeffrey4lor do you have any confirm link?13:50
sbezverkalso this option is supported: ovsdb_connection = unix:/var/run/openvswitch/db.sock13:51
*** sdake has joined #openstack-kolla13:51
sbezverkat least some folks from neutron team think so13:51
*** magnumbonum has joined #openstack-kolla13:51
Jeffrey4lsbezverk, socket is slow and the ip:port format is fast.13:51
openstackgerritMerged openstack/kolla: Fix murano notification driver  https://review.openstack.org/35597913:51
inc0coolsvap, pbourke any progress on rally front?13:51
Jeffrey4lso the ip:port became the default configuration in neutron.13:52
pbourkeinc0: none from me, did you get anywhere yesterday?13:52
Jeffrey4lsbezverk, yep. it support the both format: socket and ip:port13:52
Jeffrey4ltwo different drivers.13:52
sbezverkJeffrey4l: well, but for kube environment we need to use socket13:52
inc0coolsvap was playing with it, there were some errors13:52
sdakei hyink we were waiting on some undefined misconfigurations that coolsvap was going to fix13:52
coolsvapinc0: not much was bit occupied today we need to split and work on it individually13:52
sbezverkso I am confused by your -2 when it has no impact on kolla??13:52
inc0ok13:52
coolsvapsdake: yes13:52
Jeffrey4lsbezverk, my question is: why the ip:port not work?13:53
coolsvapi did the validation yesterday13:53
sdakecoolsvap write down a list ofissues13:53
sdakeso others know whats wrong13:53
coolsvapmany default scenarios cannot be validated13:53
inc0Jeffrey4l, sbezverk can you plz link review in question?13:53
magnumbonumI am trying out creating nested templates with python-heatclient. When I try to reference a local file, it fails with ERROR: Resource could not be loaded13:53
*** kbaegis has joined #openstack-kolla13:53
magnumbonumI am not running on the control host, but from an admin workstation... should this work?13:54
sdakemagnumbonum did you try #openstack-heat?13:54
sbezverkJeffrey4l: because when I tried to use it it was crashing generating tracebacks for invalid ip address13:54
Jeffrey4lsbezverk, there is no orchestration_engine variable in the build stage.13:54
sdakeor rather #heat13:54
Jeffrey4linc0, https://review.openstack.org/35682713:54
magnumbonumsdake yup no answer13:54
*** papacz has joined #openstack-kolla13:54
Jeffrey4lso yes. it break the ansible now.13:54
sbezverkovsdb server fails to start when configured to listen on a wildcard ip13:54
openstackgerritEduardo Gonzalez proposed openstack/kolla: Missing hosts at haproxy in site.yml  https://review.openstack.org/35586113:54
sdakemagnumbonum well we dont support heat here ;)13:54
inc0sbezverk, Jeffrey is correct, you can't put this if into build13:54
magnumbonumI am thinking along the lines that it is related to kolla / docker etc13:55
Jeffrey4lsbezverk, OK. let me check this first. `ovsdb server fails to start when configured to listen on a wildcard ip`13:55
sdakemagnumbonum but yes, I would expect that to work - perhaps you have an environmental problem13:55
sdakemagnumbonum i use heat in kolla all the time13:55
sdakeworks for me13:55
sdakei use resource groups though, not nested stacks13:55
sbezverkinc0: why?13:55
Jeffrey4lsbezverk, this variable is defined in ansible-playbooks13:55
inc0sbezverk, build should be totally indifferent to orchiestration13:55
Jeffrey4lno docker build stage.13:56
inc0if you do this, you're effectively make impossible for us to push images to dockerhub13:56
Jeffrey4lthese two are totally difference.13:56
sdakemagnumbonum try using the heat example templates13:56
*** Mr_Broken has quit IRC13:56
sdakemagnumbonum if those dont work - we can talk more :)13:56
inc0as images in dockerhub have no idea what orchiestration tools will be used, nor they should have13:56
magnumbonumsdake, the referenced template works if created directly. but not when nesting...13:56
magnumbonumsdake: yeah I am typing from examples. :-o13:57
*** Mr_Broken has joined #openstack-kolla13:57
sdakemagnumbonum not sure - we need ot wait for the heat experts to wake up13:57
inc0can you please explain how this is relevant only to k8s?13:57
inc0maybe we can work around it13:57
sbezverkinc0: then selection of a user under which this process run must be done by orchestration and NOT hardcoded in docker image13:57
*** ayoung has joined #openstack-kolla13:57
inc0sbezverk, that is different discussion, having underprivileged user is one of biggest values in kolla security model13:58
coolsvapNow we have the list of scenarios in a file and shell script to iterate through it.13:58
coolsvapI have made the image public and changed its name to match in the default scenarios.13:58
coolsvapNow we have two options13:58
coolsvap1] Update the shell script to modify values in the scenarios dynamically13:58
coolsvap2] Duplicate the scenario files for different values to benchmark against13:58
coolsvapI personally feel #2 is fast and everyone can validate the scenarios as soon as they update it13:58
coolsvapOnce we create this list, its just running the shell script ove rand over again13:58
sdakemagnumbonum please join #heat so we can see if i can get anyone's attention13:58
magnumbonumsdake: i am in there...13:58
inc0if we make k8s much less secure than ansible, we need to know why13:58
sbezverkinc0: have you read PS notes? I think it explain pretty clear why13:58
*** ad_rien_ has joined #openstack-kolla13:58
sbezverkI mean commit message13:59
inc0yes, I've read this, but why is this happening on k8s and not ansible?13:59
coolsvapinc0: sdake pbourke ^^13:59
sbezverkinc0: because with ansible you can use ip:port to start ovsdb process14:00
coolsvapif we take 1-2 services each its a 1-2 hr job to get all scenarios upto date and validated14:00
sbezverkinc0: in kube I have to use named socket for ovsdb as there is no way to know in advance when config gets built which ip address will be used by the compute node14:00
sdakedave-mccowan do you have any tricks for getting irc working insdide the vpn?14:00
inc0sbezverk, and if you run ovsdb with net=host, will that help?14:00
inc0I see14:01
sbezverkinc0: we already do it14:01
inc0hmm, let me think about this for a moment14:01
*** Mr_Broken has quit IRC14:01
sbezverkbut since we want to leverage kube dynamic nature and scale of/off14:01
*** papacz has quit IRC14:01
sbezverkwithout any intervention from orcestration14:02
inc0so Jeffrey4l is totally right, build phase shouldn't have any notion of orchiestration engine14:02
Jeffrey4lsbezverk, is it possible to regenerate the configure file before the openvswitch is deployed or moved?14:02
*** b_bezak has quit IRC14:02
sbezverkguys, we want to use kube because it is DYNAMIC!!!14:02
sbezverkwe do not want to do anything when new compute node gets added/removed14:03
inc0sbezverk, computes can't be that dynamic anyway14:03
sbezverkyes they can14:03
sdakesbezverk got it -but docker file is not place to put orchestration variable imo14:03
sbezverkwe use daemon sets14:03
kbaegisThanks kolla team! :) http://pasteboard.co/a96c8uI6N.png14:03
inc0if k8s removes a compute but VMs are still there, you're in a world of pain14:03
Jeffrey4lsbezverk, i means when the openvswitch container get deployed or moved. not the nodes.14:03
sdakekbaegis thats gentoo?14:04
sbezverkok for removal you are right, but for adding it can totally work14:04
sdakekbaegis what was the problem14:04
*** duonghq has joined #openstack-kolla14:04
*** caowei has quit IRC14:04
inc0sbezverk, adding will be no different than deployment14:05
inc0in any case, you need ovs on control nodes too14:05
inc0network nodes rather14:05
sbezverkinc0: you think in terms of ansible14:06
kbaegissdake Yes :D14:06
inc0so let me ask you this sbezverk, ovsdb starts to listen on ip:port, but vswitchd seems to use sock14:06
*** huikang has joined #openstack-kolla14:06
kbaegisWell, I had to set firewall_driver (ml2) to noop14:06
sbezverkbut in kube all compute nodes are netowrk nodes and compute nodes14:06
kbaegisHuge issue.  You guys already fixed the mount --make-shared /run14:06
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update heat base Dockerfile to include footers  https://review.openstack.org/35724014:06
inc0sbezverk, that won't work like you think14:06
*** caowei has joined #openstack-kolla14:06
kbaegisAnd make sure the correct packages/use-flags and kernel parameters were configured14:06
inc0router still has one IP14:07
sbezverkinc0: no ovsdb start with named socket because ip:port cannot be used14:07
inc0routes will still be A-P14:07
duonghqkbaegis: did you try deploy multinode with non-root user?14:07
kbaegisAlso, openvswitch has to be run on the host, not in a container.14:07
kbaegisduonghq: Not yet14:07
mntdevopsduonghq: is it possible to deploy certain containers as non-root?14:08
*** Mr_Broken has joined #openstack-kolla14:08
inc0sbezverk, it cannot be used because of net=host?14:08
mntdevopslike14:08
Jeffrey4lsbezverk, why u get the permission issue when using socket?14:08
sbezverkinc0: no because we do not know which ip address to use at the time config gets generated14:08
mntdevopscan you run nova-compute as a regular user?14:08
*** ad_rien_ has quit IRC14:08
Jeffrey4lsbezverk, ansible never hit such issue.14:08
duonghqmntdevops: ansible has become option, so the ansible_ssh_user do not need to be root14:08
duonghqI mean the user use to connect to nodes14:09
mntdevopsah, right14:09
sbezverkJeffrey4l: why you keep refering to ansible?!?! we are talking about different operating environment.14:09
inc0sbezverk, and shared socker has perms issues? can you change perms of this socket?14:09
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update heat base Dockerfile to include footers  https://review.openstack.org/35724014:09
Jeffrey4lsbezverk, ya. So what permission issue you get?14:09
inc0sbezverk, ansible is relevant14:09
duonghqmntdevops: did you try this?14:10
mntdevopsduonghq: are you having an issue with the playbooks running on a non-local system during deploy?14:10
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update openvswitch base Dockerfile to include footers  https://review.openstack.org/35724314:10
Jeffrey4lbefore one month ago, ansible use the socket for communication only. I just want to the details of the issue.14:10
Jeffrey4li admit i am not familiar with the kube.14:10
mntdevopsduonghq: I look forward to my current work project being done this week to have time to go multi-node next week14:11
sbezverkinc0: I though about changing permissions for the socket, but that would require extra step in bootstrap process or modification of extended_start14:11
Jeffrey4lBut any change in the image may break the ansible deploy. So we should be careful with that.14:11
inc0sbezverk, so modification of extended start is totally ok14:11
mntdevopskolla is a hobby project of mine right now14:11
inc0you can inject orchiestration engine info there14:11
inc0via ENV14:11
duonghqmntdevops: yeah, it cannot create /etc/kolla/... without enough priviledge, but allow root remote access is not so wise14:11
inc0on runtime14:11
duonghqansible has become for these purpose14:12
duonghq*become options14:12
inc0so just put this perm change in if $K8S14:12
duonghqbut in current playbook, there is no become yet14:12
sbezverkinc0: ok I thought you guys will not accept it14:12
pbourkecoolsvap: I can try updating the scripts now14:12
duonghqmntdevops: am I wrong?14:12
inc0nah, I think it's cool14:12
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update ceilometer base Dockerfile to include footers  https://review.openstack.org/35724514:13
pbourkecoolsvap: i just had one question though, how do we decide the values to update14:13
*** ad_rien_ has joined #openstack-kolla14:13
Jeffrey4lsbezverk, i really want to make clear what the issue is. could u explain it more?14:13
inc0just make sure to get privileges right, not 777 unless it's totally required14:13
mntdevopsduonghq: I'm very familiar with ansible, so I know what those options do.  You're probably not wrong.  I just haven't run into those issues yet14:13
coolsvappbourke: have you seen the scenario files?14:13
pbourkecoolsvap: ya14:13
Jeffrey4lwhy u get permission issue when using socket?14:13
coolsvaplets jump to tmux14:13
coolsvappbourke: do you have time?14:13
pbourkecoolsvap: yes14:14
sbezverkinc0: it should be 75514:14
pbourkejust getting connected14:14
sbezverkas process need to read/write14:14
duonghqmntdevops: yeah, I need some more people confirm my observe,14:14
mntdevopsduonghq: in my experience with nested roles and includes, sometimes handlers and includes don't automatically "become: yes" "become_user: root" even though the parent does14:14
inc0sbezverk, but also...I vaguely remember that ovs did indeed use socket14:14
inc0as Jeffrey4l pointed ot14:14
inc0so what is different?14:15
pbourkecoolsvap: why are the default values not ok14:15
duonghqmntdevops: can you grep through the kolla source, I cannot find any "become"14:15
coolsvappbourke: it will not test the cluster we got14:15
mntdevopsduonghq: I've had plays to hit ec2 resources that call roles.  Inside those roles, the tasks run as root if needed but the handlers needed "help"14:15
sbezverkinc0: when I start ovsdb with punix:/blah/blah socket it becomes compute node config agnostic14:15
pbourkecoolsvap: why not14:15
mntdevopsduonghq: stand by14:15
Jeffrey4lsorry need to out, brb.14:16
duonghqmntdevops: ok, thanks14:16
sbezverkinc0 you follow me?14:16
inc0sbezverk, wich is cool14:16
openstackgerritzhubingbing proposed openstack/kolla: Add gnocchi ansible role  https://review.openstack.org/34935114:16
coolsvapit benchmarks on the values 10-20 instances/volumes and concurrency level of single digit14:16
openstackgerritzhubingbing proposed openstack/kolla: repair aodh image dockerfile  https://review.openstack.org/35071514:16
coolsvapwe need it to be in 100s/10000s imo14:16
inc0https://github.com/openstack/kolla/blob/stable/mitaka/ansible/roles/neutron/templates/openvswitch-db-server.json.j2 so look, there is no notion of IP14:16
sbezverkinc0: for kube env this level of lfexibility is requirement, otherwise you just kill it14:16
inc0no, you make a pod with ovsdb and vswitchd14:17
coolsvapso my idea was to create 2-3 set of template copies with these different values14:17
inc0they will have to be on same host but trust me, that's how you want it14:17
inc0you don't want ovsdb to be somewhere else14:17
inc0it's only relevant to local ovs14:17
coolsvapand just run those everytime so that we are testing on a constant values and see the results14:18
sbezverkinc0: correct each node will be running local copy of ovsdb14:18
sdakekbaegis sounds like a whole lot of caveats14:18
inc0ok, so what's the issue with sock?14:18
inc0it used to work14:18
pbourkecoolsvap: ok lets do that14:18
kbaegissdake Yeah, big ones too14:19
sbezverkinc0: when it gets created by ovsdb process sock has permissions 70014:19
mntdevopsduonghq: you're right: I don't see it in the source.  However, you can edit your inventory file to add per-host ansible options: http://docs.ansible.com/ansible/become.html14:19
sbezverkinc0 ovsdb process runs as root14:19
mntdevopsthat *should* work14:20
sdakesbezverk extensions to run scripts are fine14:20
coolsvappbourke: okay I will start with nova then14:20
inc0ok, in stable/mitaka we didn't drop-root ovs14:20
duonghqmntdevops: I'm not really comfort with this workaround14:20
mntdevopsduonghq: I'd add ansible_become=yes ansible_become_user=root ansible_ssh_user=mysshuser14:21
duonghqmntdevops: because many groups need write to /etc/kolla, it require root14:21
mntdevopswhat makes you uncomfortable with editing your inventory file?14:21
sbezverkinc0: neutron-openvswitch-agent runs as neutron hence fails accessing this socket14:21
mntdevopsso that when ansible runs on that host, it knows how to sudo when needed?14:22
duonghqmntdevops: it will give the become for whole host, but I prefer just give to neccesary task14:22
inc0sbezverk, ...no...stable/mitaka works...14:22
inc0https://github.com/openstack/kolla/blob/stable/mitaka/docker/neutron/neutron-l3-agent/Dockerfile.j214:22
inc0look14:22
inc0in stable/mitaka there is nothing about IP14:23
inc0it's all over sockets14:23
mntdevopsduonghq: makes sense.  I thought it was just a matter of getting it to work first.  then getting it locked down14:23
duonghqmntdevops: agree with you14:23
mntdevopsduonghq: I figured it was a roadblock for you14:23
duonghqsomething like that14:24
mntdevopsduonghq: it would mean editing each role that needs it14:24
mntdevopsat the start of each role14:24
sbezverkinc0: I am not here to explain why stable/mitaka works, I here to show that permissions for db.sock do not allow neutron-openvswitch to access it14:24
mntdevopsit gets messy really quickly adding it to specific tasks only14:24
inc0sbezverk, yet mitaka works...so we need to figure out what changed14:24
inc0because if you can get it to work same way as mitaka14:24
inc0you're good14:24
duonghqthank you mntdevops14:24
sbezverkinc0: have you tried to run stable/mitaka on kube cluster?14:25
inc0sbezverk, kube doesn't matter - this is low level container runtime14:25
duonghqsdake, can you review this bp: https://blueprints.launchpad.net/kolla/+spec/ansible-become-support I think it is quite importance bp14:25
inc0and it has nothing to do with IP kube will give it14:25
mntdevopsduonghq: I hope I've helped14:26
mntdevopsif not, no worries14:26
*** kangh_ has joined #openstack-kolla14:26
duonghqmntdevops: your help is  valuable to me14:27
*** haplo37__ has joined #openstack-kolla14:28
mntdevopsduonghq: well, I hope it helps me next week, too :-)14:28
duonghq:)14:28
sbezverkinc0: btw here is the line used to start ovsdb in master: /usr/sbin/ovsdb-server /etc/openvswitch/conf.db -vconsole:emer -vsyslog:err -vfile:info --remote=punix:/run/openvswitch/db.sock --remote=ptcp:6640:$ovsdb_ip --log-file=/var/log/kolla/openvswitch/ovsdb-server.log14:28
openstackgerritLeon Zachery proposed openstack/kolla: Simplify install process for faster quickstart execution  https://review.openstack.org/35618114:28
inc0ok, I know that it uses remote, you can use different line in k8s14:29
inc0this is config of runtime14:29
*** ad_rien_ has quit IRC14:29
inc0what I mean is: -remote=punix:/run/openvswitch/db.sock does work in stable/mitaka yet it doesn't work in your case14:30
*** ad_rien_ has joined #openstack-kolla14:30
inc0if you can get it to work, k8s will be fine14:30
*** huikang has quit IRC14:30
sbezverkinc0: if you have kolla running handly could you check the actual permission14:30
sbezverkon the socket file?14:30
inc0yeah, let me log in to OSIC14:31
pbourkeinc0: is the osic deploy broken14:31
pbourkeinc0: to your knowledge14:31
coolsvapit was working yesterday when I configured rally and validated scenarios14:32
inc0sbezverk, root@729579-comp-s3700-027:~# docker exec -it openvswitch_db ls -la /run/openvswitch/db.sock14:34
inc0srwxr-x--- 1 root root 0 Aug 16 10:06 /run/openvswitch/db.sock14:34
*** sdake has quit IRC14:34
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update openvswitch base Dockerfile to include footers  https://review.openstack.org/35724314:36
*** ad_rien_ has quit IRC14:36
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/kolla: Update ceilometer base Dockerfile to include footers  https://review.openstack.org/35724514:36
pbourkecoolsvap: disk seems full14:36
sbezverkinc0: same thing I have14:37
sbezverkinc0: docker exec neutron-openvswitch-agent whoami14:37
inc0sbezverk, but we need to check stable mitaka for that14:37
*** ad_rien_ has joined #openstack-kolla14:38
egonzalez90coolsvap: i'll prefer to push all base footers in a single commit, that will be easier to review14:38
sbezverkinc0: would you agree that under normal circumstances neutron use will not have access to this sock file?14:38
sbezverkI meant neutron user.14:39
openstackgerritzhubingbing proposed openstack/kolla: Add aodh role  https://review.openstack.org/35102714:39
*** kbaegis has quit IRC14:39
inc0so not exactly, this is how ovs-agent talks to ovs14:39
inc0via db14:39
inc0and socket is exactly right way to do it14:39
inc0what I would agree is that normally neutron and ovs are run by same used14:40
inc0user14:40
sbezverkinc0: but you need permissions to open it and neutron user has none14:40
inc0sbezverk, yeah...yet it used to work right?14:40
*** kbaegis has joined #openstack-kolla14:42
sbezverkinc0: I guess but it does not help much..14:44
inc0it should, because all we need to do in k8s is to mirror this exact behaviour14:45
kbaegisIs there a convenient way to change the hostname for novncproxy?14:45
Jeffrey4lsbezverk, i tried: `/usr/sbin/ovsdb-server /etc/openvswitch/conf.db --remote=ptcp:6641:0.0.0.0`  it works14:46
kbaegisI can just change novncproxy_base_url = right?14:47
Jeffrey4lthe 0.0.0.0 can be omitted.14:47
*** ajpower has quit IRC14:49
sbezverkJeffrey4l: I will re-try but still neutron-openvswitch-agent needs to communicate with ovsdb via named socket14:50
kbaegisI think it's vncserver_proxyclient_address, novncproxy_base_url if I want to hide the vncproxy behind an apache proxy, right?14:50
sbezverkas ml2_conf.ini will not have actual kube compute node ip address in advance14:50
Jeffrey4lsbezverk, if you change the ovsdb_connection and ovsdb_interface=native, it will communicate though tcp port.14:51
Jeffrey4land i still confused the permission issue. it shouldn't be happen. :(14:51
Jeffrey4lsbezverk, here is the related option in neutron. https://github.com/openstack/neutron/blob/master/neutron/agent/ovsdb/api.py#L3514:52
coolsvappbourke: rally task list has the option to filter by deployment uuid14:52
pbourkecoolsvap: im thinking seen as there's one deployment all these tasks should go into it by default14:52
pbourkecoolsvap: maybe not though14:52
coolsvapcurrently they will14:53
coolsvapbut if we have multiple deployments then14:53
sbezverkJeffrey4l: neutron-openvswitch-agent at least at this point cannot communicate via tcp port because see my explanation above14:53
coolsvappbourke: http://rally.readthedocs.io/en/latest/cli/cli_reference.html#rally-task-list14:53
openstackgerritPaul Bourke proposed openstack/kolla: Add OSIC Scale Testing Documentation  https://review.openstack.org/35210114:54
inc0pbourke, I'll put bug about this elastic14:55
Jeffrey4lsbezverk, sorry i can not find the point. would u mind to copy them again?14:55
pbourkeinc0: thanks14:55
inc0we need to document that /var/lib/docker needs a lot of space14:55
*** mnasiadka has quit IRC14:57
sbezverkJeffrey4l: ml2_conf.ini gets generated offline and it points to vip address. we do not know in advance which ip actually will be used on kube compute node14:57
sbezverkso we need to use something generic which will work in every situation, which is named socket14:58
egonzalez90kbaegis: proxyclient_address is the api_interface IP address, base_url is the URL nova will give you to access from a browser, in kolla points to kolla_external_fqdn14:58
pbourkecoolsvap: we should generate json + html incase html can't be generated from the json after the fact14:58
inc0https://bugs.launchpad.net/kolla/+bug/161457214:58
openstackLaunchpad bug 1614572 in kolla "Elastic search eats up space quickly" [Medium,Triaged]14:58
egonzalez90to hide the address you only need to change base_url at compute nodes14:59
*** michauds has joined #openstack-kolla15:00
Jeffrey4lsbezverk, ok.15:00
*** mewald has quit IRC15:00
*** salv-orlando has joined #openstack-kolla15:00
openstackgerritzhubingbing proposed openstack/kolla: Add sahara ansible role  https://review.openstack.org/35129415:01
coolsvappbourke: yes I always do both15:03
inc0pbourke, coolsvap lots of "unauthorized" Traces15:03
inc0seems like rally issue?15:03
coolsvapinc0: default scenarios have hardcoded values15:04
pbourkecan't see whats wrong with that neutron scenario15:07
*** Mr_Broken has quit IRC15:08
*** Guest52040 has joined #openstack-kolla15:08
*** Mr_Broken has joined #openstack-kolla15:08
*** sdake has joined #openstack-kolla15:11
sdakeyo15:11
openstackgerritJeffrey Zhang proposed openstack/kolla: Customizations for Multipathd  https://review.openstack.org/35442115:12
duonghqsdake: can you review my bp: https://blueprints.launchpad.net/kolla/+spec/ansible-specific-task-become15:12
*** Mr_Broken has quit IRC15:13
*** magnumbonum has quit IRC15:13
duonghqstill do not get used to different timezone at IRC :(15:14
duonghqsdake: how about your mac?15:15
* coolsvap needs to step out for dinner15:16
*** coolsvap is now known as _coolsvap_15:16
duonghq_coolsvap_ quite late for dinner?15:16
*** zhubingbing has quit IRC15:17
*** ajpower has joined #openstack-kolla15:17
*** sdake has quit IRC15:17
*** michauds has quit IRC15:19
*** sdake has joined #openstack-kolla15:19
duonghqsdake: how about your mac?15:19
*** vbel has quit IRC15:20
*** vbel has joined #openstack-kolla15:20
sdakewhat about it15:21
sdakei have like 6 macs at home15:21
sdakei keep spar egear around in case of emergency15:21
duonghqreally?15:22
sdakeyup - wife in high tech too15:23
sdakewe are pretty tortourous on our gear15:23
duonghqoh15:23
*** dave-mccowan has quit IRC15:24
*** dave-mccowan has joined #openstack-kolla15:24
duonghqyour amp arrived?15:24
*** sdake has quit IRC15:24
*** sdake has joined #openstack-kolla15:25
duonghqsdake: you're blinking?15:25
sdakei'm going back to bed15:25
duonghqhuh?15:26
sdakeduonghq busy atm trying to setup my new laptop15:26
duonghqah, sorry15:26
egonzalez90were you able to vagrant up kolla in mac with virtualbox?15:27
egonzalez90always getting error with ssh key15:28
*** vhosakot has joined #openstack-kolla15:28
Pavoegonzalez90 is this your site http://egonzalez.org/openstack-kolla-deployment-from-rdo-packages/15:28
*** sdake_ has joined #openstack-kolla15:29
egonzalez90pavo: yes, probably there are a lot of errors in there15:29
Pavoyeah I found that out15:30
*** hieulq_ has joined #openstack-kolla15:31
sdake_wirehead_ about?15:33
*** zhurong has quit IRC15:34
*** papacz has joined #openstack-kolla15:37
*** sdake has quit IRC15:41
*** HyperJohnGraham_ has joined #openstack-kolla15:42
*** duonghq has left #openstack-kolla15:43
*** HyperJohnGraham has quit IRC15:44
*** HyperJohnGraham_ is now known as HyperJohnGraham15:44
*** jrich523 has joined #openstack-kolla15:44
openstackgerritMerged openstack/kolla: Simplify install process for faster quickstart execution  https://review.openstack.org/35618115:50
openstackgerritJeffrey Zhang proposed openstack/kolla: Implement mongodb replication set cluster  https://review.openstack.org/34575115:51
*** michauds has joined #openstack-kolla15:52
openstackgerritEduardo Gonzalez proposed openstack/kolla: Remove unnecessary host at haproxy in site.yml  https://review.openstack.org/35586115:52
Pavokeep getting ERROR:kolla.image.build:neutron-networking-sfc-agent Failed with status: error15:55
Pavoguessing its because of ERROR:kolla.image.build.neutron-networking-sfc-agent:The command '/bin/sh -c echo 'binary not yet available for centos'     && /bin/false' returned a non-zero code: 115:56
egonzalez90Pavo: review link?15:58
*** dwalsh has quit IRC15:58
Pavothink thats normal error since I am building centos images15:59
sdake_that is normal pavo15:59
Pavojust kind of weird that it would even try to build images that are not centos based15:59
sdake_that is not an error its a warning15:59
sdake_pavo we don't have a mapping of those things15:59
sdake_since we dont have a mapping, it builds it all16:00
Pavoroger that16:00
sdake_we could easily create a mapping using the profiles feature16:00
Pavothink I am gonna try and build a repo container and have all nodes poit to it to get their packages16:01
wirehead_sdake_: now I am16:01
sdake_wirehead_ up for jabber16:01
openstackgerritMerged openstack/kolla: Add pull.yml file for tempest  https://review.openstack.org/35718516:01
Pavoinstead of the internet16:01
sdake_my laptop cratered last night16:01
jrich523i need to drop my laptop16:02
jrich523a few times...16:02
jrich523work wont give me a new one until it breaks16:03
sdake_jrich523 once is all it takes :)16:07
jrich523lol.. im getting pretty close to doing it16:07
jrich523figure a few smashes on the couch should ruin the drive but not show any external damage16:08
jrich523"i dont know, just started to throw errors"16:08
*** Serlex has quit IRC16:08
*** daneyon has joined #openstack-kolla16:14
*** dwalsh has joined #openstack-kolla16:14
jrich523not that you guys probably care, but you can do this now: yum install powershell   :)16:15
jrich523or apt-get, whatever16:15
sdake_off to breafast brb16:16
*** sdake_ has quit IRC16:16
*** egonzalez90 has quit IRC16:16
*** daneyon has quit IRC16:18
sbezverkinc0: ping16:19
inc0I'm here16:19
sbezverkinc0: I want to use the same approach I used in ovsdb, replacing command in ovsdb template by the name of the script, in this case I should be able to get IP address I need and not use socket16:21
sbezverkwould it be acceptable solution from kolla perspective?16:21
inc0you mean to change command for running ovsdb and vswitchd depending on orch engine?16:22
inc0yeah, thats acceptable16:23
sbezverkinc0: ok I will test it and if it resolves socket problem I will abandon previous PS16:24
*** michauds has quit IRC16:25
sbezverkinc0: this time I will use script for starting neutron-openvswitch-agent and neutron-l3-agent, both need access to ovsdb16:25
inc0ok, that's fair16:25
openstackgerritMerged openstack/kolla: Add support for building vmtp container  https://review.openstack.org/35436216:26
*** mark-casey has joined #openstack-kolla16:28
*** caowei has quit IRC16:32
*** hieulq_ has quit IRC16:35
kbaegisSo I'm trying to get novnc behind a reverse proxy for ease-of-use16:36
kbaegisAnyone have an apache vhost for this already?16:36
kbaegisIdeally I'd like it to bind to something on https://<frontend.domain>:443/vnc/<openstackurl> with the actual novnc configuration just being <ip>:608016:37
*** thumpba_ has joined #openstack-kolla16:37
kbaegisI keep my lab on a different network, and it's handy to have a web proxy :)16:37
kbaegisI'm editing cd /etc/kolla/nova-novncproxy/nova.conf directly w/ COPY_ALWAYS16:38
kbaegisRecommendations?  I asked on #openstack-nova and #openstack already too16:39
*** thumpba has quit IRC16:39
*** dwalsh has quit IRC16:40
*** unicell has joined #openstack-kolla16:41
*** unicell1 has quit IRC16:43
*** sdake has joined #openstack-kolla16:43
sdakeinc0 ping - can you forward that email to ryan16:45
sdakeit bounced on my end16:45
sdakehe isn't on16:45
sdakeand i dont know what his email address it16:45
sdakeis16:45
pbourkebritthou_: inc0: _coolsvap_: im headed off soon, I think I fixed up the auth error in rally so all scenarios are being run through again. Next steps: 1) fix up any remaining failed scenarios 2) tweak the values in each scenario to better excercise the cluster 3) re-do the first deploy scenario on the etherpad and obtain full set of rally results (json+html)16:48
sdakei thin kwe need to not burn up alot of time on part #216:49
sdakei could see that taking weeks16:50
sdake_coolsvap_ or if you could forward the mail that would work too16:50
sdakepretty epic: http://www.opensym.org/os2016/proceedings-files/p101-schneider.pdf16:54
*** stingray_ has joined #openstack-kolla16:57
*** fragatina has joined #openstack-kolla16:59
*** fragatina has quit IRC17:00
*** fragatina has joined #openstack-kolla17:00
*** kangh_ has quit IRC17:02
*** rstarmer has joined #openstack-kolla17:10
*** rstarmer has quit IRC17:11
*** fragatina has quit IRC17:16
*** athomas has quit IRC17:25
mark-caseysdake: that pdf is amazing17:26
*** unicell has quit IRC17:33
*** rstarmer has joined #openstack-kolla17:35
*** Guest52040 has quit IRC17:40
*** gfidente has quit IRC17:43
*** dwalsh has joined #openstack-kolla17:46
*** HyperJohnGraham has quit IRC17:46
*** dmueller-evoila has joined #openstack-kolla17:53
*** unicell has joined #openstack-kolla17:55
*** dmueller-evoila has quit IRC17:58
*** rstarmer has quit IRC18:01
*** daneyon has joined #openstack-kolla18:02
*** inc0 has quit IRC18:02
*** daneyon has quit IRC18:06
*** dmueller-evoila has joined #openstack-kolla18:08
*** fragatina has joined #openstack-kolla18:11
*** HyperJohnGraham has joined #openstack-kolla18:13
*** dwalsh has quit IRC18:18
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Deployment, Readiness, Safe Shutdown, & Scaling for glance-api  https://review.openstack.org/35489518:21
*** inc0 has joined #openstack-kolla18:22
*** rstarmer has joined #openstack-kolla18:22
*** rstarmer has quit IRC18:24
wirehead_Hm.  The thanks vs. actually table.18:26
*** ayoung has quit IRC18:37
*** michauds has joined #openstack-kolla18:38
*** HyperJohnGraham has quit IRC18:43
kbaegislol18:46
kbaegisI always remember the f*#! you NVIDIA speech18:48
kfox1111sbezverk: have you gotten a chance to test https://review.openstack.org/#/c/356114 ?18:57
patchbotkfox1111: patch 356114 - kolla-kubernetes - WIP- openvswitch ip18:57
kfox1111I think its probably ready to take off the WIP.18:57
kfox1111or it could wait until crudini merges.18:58
*** HyperJohnGraham has joined #openstack-kolla19:02
*** stingray_ has quit IRC19:02
*** mewald has joined #openstack-kolla19:03
*** mewald has left #openstack-kolla19:07
*** HyperJohnGraham has quit IRC19:08
*** HyperJohnGraham has joined #openstack-kolla19:09
*** salv-orl_ has joined #openstack-kolla19:15
*** salv-orlando has quit IRC19:18
*** vhosakot has quit IRC19:18
*** vhosakot has joined #openstack-kolla19:18
Davieysdake: hey... sorry for the slow response... but RE: Master images.  They will probably never be correct on docker's hub, due to each commit rightly needing a new image.  Any developer would surely be building their own imaages locally?19:20
kfox1111sbezverk: Just looked at ingress a bit to get a handle on what its doing.19:21
Davieysdake: If we are saying that Master doesn't need a new image per-commit, i strongly disagree.19:21
kfox1111it looks like its mostly just a l7 lb.19:21
kfox1111I'm not sure it buys us anything outside of when running within GCE/GKE.19:22
kfox1111we still need the plubming to get network access into the ingress pod.19:22
*** dwalsh has joined #openstack-kolla19:23
*** vhosakot has quit IRC19:23
*** vhosakot has joined #openstack-kolla19:24
*** thumpba_ has quit IRC19:29
sdakeDaviey - right master will be rebuilt each time19:30
sdakeby the devs19:30
sdakeDaviey - perhaps i don't totally understand your idea, but it applies to stable branches?19:31
sdakei didn't think it worked well with stable branches when i looked at it - but willing to look again :)19:31
*** egonzalez90 has joined #openstack-kolla19:32
*** dgonzalez has quit IRC19:34
openstackgerritMerged openstack/kolla: Add crudini  https://review.openstack.org/35615519:38
*** thumpba has joined #openstack-kolla19:39
*** dgonzalez has joined #openstack-kolla19:45
inc0sdake, crudini...?:P19:50
inc0deja vu?19:50
sdakewhatever moves things along19:50
sdakecan clean up the mess later19:50
sdakek8s wil take along time to mature just as kolla-ansible did19:50
*** daneyon has joined #openstack-kolla19:50
*** HyperJohnGraham has quit IRC19:51
inc0yeah, I know19:51
inc0just I've seen this before19:51
inc0all I'm saying;)19:51
sdakei've seen the whole patterns couple times19:51
sdakebut things are slightly different this time environmentally19:51
*** egonzalez90 has quit IRC19:53
*** huikang has joined #openstack-kolla19:55
*** daneyon has quit IRC19:55
*** egonzalez90 has joined #openstack-kolla19:56
*** HyperJohnGraham has joined #openstack-kolla19:58
*** dwalsh has quit IRC19:59
egonzalez90ntpttr: ping20:01
*** thumpba has quit IRC20:04
Pavoso.... why would my networks go down and not come back up when it gets to the starting Neutron server during deployment?20:05
*** ayoung has joined #openstack-kolla20:09
*** JoseMello has quit IRC20:10
ntpttr___egonzalez90: hi20:11
ntpttr___egonzalez90: sorry my IRC bouncer is experiencing some technical difficulties at the moment so I missed your ping20:12
ntpttr___hence the double underscore there20:12
egonzalez90can you try changing privsep_entrypoint section to privsep_osbrick at nova/cinder.conf in compute and backup containers?20:14
ntpttr___egonzalez90: sure, can I just manually change those and restart the services in those containers for the change to take effect?20:15
egonzalez90I tried to find the root cause of your issue, I'm thinking on oslo.privsep issue20:16
ntpttr___egonzalez90: I don't see privsep_osbrick in the cinder configuration reference, is that something that gets used? http://docs.openstack.org/mitaka/config-reference/block-storage/block-storage-sample-configuration-files.html#cinder-conf20:20
ntpttr___I don't see privsep_entrypoint in there either actually20:20
egonzalez90new configuration at master, is not documented yet. devstack people use privsep_osbrick section20:21
*** _coolsvap_ has quit IRC20:21
Davieysdake: Why wouldn't it work well with stable?20:25
Davieysdake: If they are building from git, they need to build their local images.  If they are installing via "pip install kolla" then they have the option of using docker hob images20:26
sdakeDaviey i don't recall specifics now, perhaps we can start over ;)20:26
DavieyThis seems reasonable to me :)20:26
sdakeDaviey our docs now recommend not to use git for stable eval/deployments20:26
sdakethis was after about 20+ hours of research on how to best handle the versioining problems pbr creates on tagged images20:26
Davieysdake: yeah, but people have the option to install from git - but they need to build their own images20:26
Pavodoc right now say just git https://git.openstack.org/openstack/kolla20:27
sdakeif they do that, they are on their own20:27
Davieysdake: 20 hours?!20:27
sdakeDaviey estimate on how much time i spent jering around on the problem20:27
Davieysdake: Geez.  You must be burned out.20:27
*** diogogmt has joined #openstack-kolla20:27
sdakereference: https://review.openstack.org/35618120:28
sdakeDaviey far from it20:28
sdakeDaviey but on this particular problem yes :)20:28
Pavoshouldn't there really only 2 taggings20:28
Pavostable and latest?20:28
Pavoand docs should say what versions of kolla and ansible for each20:29
sdakepavo we support liberty and mitaka - so we have two stables20:29
sdakepavo look at the above review20:29
Pavook for each openstack version then20:29
sdakeDaviey read that review, see if it makes sense to you20:29
sdakeit started off wrong, but i think we have the right model going forward baed upon conversations with dhellmann and jroll20:30
*** HyperJohnGraham_ has joined #openstack-kolla20:30
Davieysdake: sorry, i meant this problem.. not generally20:30
*** HyperJohnGraham has quit IRC20:30
*** HyperJohnGraham_ is now known as HyperJohnGraham20:31
sdakeDaviey yes - honetly I don't recall exactly what your proposal was20:31
sdakeI remember reading it20:31
sdakeand thinking 3 things20:31
sdake1. does it have impact on upgrades20:31
sdake2. does it have impact on reocnfigure20:31
sdake3. does it work with two stable branches20:31
sdakei dug into 3 a bit, so we can get the correct install model going forward20:31
sdake(thats the 20 hours part)20:31
sdakeso given the above docs, any chancey ou could rfresh me on your solution20:32
sdake(the ones in the review)20:32
Pavook sounds like an issue with he two different versions ie pip installed or git installed20:33
sdakepavo - right there is no precedence in pbr20:34
sdakeits one or the other20:34
Pavoyeah I can see how thats confusing20:34
Davieysdake: wait, has someone landed the pbr version tagging in the building process now???20:34
sdakeif pip installed - pip version is used - if pip not installed, git is used20:34
sdakeDaviey who knows - this project movesfaster then i can keep u p:)20:34
Pavoso wait guess I have been doing wrong the entire time then20:34
*** Jeffrey4l has quit IRC20:35
Davieysdake: that review suggests it is.. but i thought i was on the hook to do that20:35
PavoI git clone kolla from git and then use pip install /kolla20:35
*** Jeffrey4l has joined #openstack-kolla20:35
sdakeDaviey that doc review was a short term put out the  fire house problem20:35
Pavothats the wrong way?20:35
sdakeDaviey i expected you would come up with something more long term20:35
sdakepavo yup your doing it wrong20:35
sdakebecause our docs were wrong20:35
ntpttr___egonzalez90: huh not sure what's wrong but now creating a volume doesn't even work - it's hanging on 'creating'20:35
Pavowell no wonder I am not getting anywhere lol20:36
Pavocan the docs alos be changed to have separation for different distros also, ie Centos only shows Centos cmds and Ubuntu only shows Ubuntu cmds?20:37
*** jtriley has quit IRC20:37
Pavosdake so if I git clone kolla I should use what to install its dependiecies?20:39
Davieysdake: Odd, but i don't seem to have voting rights for that review20:41
*** vhosakot has quit IRC20:41
sdakeDaviey the review is merged20:41
DavieyOh ffs20:41
sdakehence no voting rights20:41
egonzalez90ntpttr: ups, it should work. in theory just need add the section and the command to make it work https://openstack.nimeyo.com/75137/openstack-brick-cinder-brick-privsep-change-awaiting-review20:41
Davieysdake: well, i just added a bunch of comments20:41
Pavoso how do you install kolla if you git clone it?20:43
DavieyPavo... git clone http://etc/kolla ; cd kolla ; sudo pip install .20:45
PavoI just asked if that way was wrong and sdake said yup20:45
PavoI have been doing this: git clone https://git.openstack.org/openstack/kolla then pip install kolla/20:46
PavoI'm so.... confused lol20:47
sdakeDaviey i responded on your review20:48
sdakepavo - read the docs in that review20:48
sdakepavo you dont install kolla as a developer20:48
sdakeyou run from git20:49
PavoI am and they say to git clone the pip install kolla/20:49
Pavooh ok20:49
sdakepavo read this review: https://review.openstack.org/35618120:49
sdakepavo - docs fix20:49
sdakethe docs job hasn't finished publishing the update yet so docs.oo doesn't have the latest stuff20:50
Pavoso last commit has latest corrected docs?20:50
sdakepavo that patch is mergedd20:51
Pavook20:51
Pavosorry if I am sounding newbish20:51
sdakeread the changes by clicking the quickstart.rst button20:51
sdakeit will show you a "diff' of the changess20:51
sdakewhich should be easy enough to understand what we haned in the docs20:51
sdakevs you determining them rom a visual inspection :)20:52
*** vhosakot has joined #openstack-kolla20:52
Pavook gotta redo my deployer completely then20:53
sdakeshouldn't need to20:53
sdakebut how to get rom old docs to new docs is a manual process20:54
Pavowell atleast delete everything from pip and re clone master20:54
sdakei can walk you throug hit20:54
kfox1111inc0: have you seen it cause problems with single entry changes?20:54
sdakepavo you want to work from git trees as a developer then?20:54
kfox1111crudini that is20:54
sdakepavo if I repeat myself with questions its only because my brian has limited capacity for remembering everything ;)20:54
Pavoright now I just want to get up a working system to see it work20:54
sdakefollow the "Installing Kolla for eval/deployment"20:55
kfox1111I've used it to make single entry changes in my cobbler kickstarts to good effect.20:55
sdakeand not the other branch which is for development20:55
inc0kfox1111, not really, just super early kolla used crudini to configure *EVERYTHING*20:55
kfox1111inc0: ah. yeah. I would believe that would cause problems. :)20:56
inc0and when I saw this review I felt nostalgic20:56
Pavook so I yum -y remove ansible and then pip install -U ansible==1.9.420:57
*** rstarmer has joined #openstack-kolla20:58
kfox1111sbezverk: alive?20:59
sdakeyup you want that20:59
PavoI alos done a pip uninstall kolla and reinstalled it using pip install kolla21:00
Pavowhich installed kolla 2.0.221:00
Pavosdake  ok in the review it says install using pip install kolla then it says Kolla holds configurations files in ``etc/kolla``. Copy the configuration files to ``/etc``:21:01
Pavothere is no kolla folder now in my root directory21:02
Pavoso can not cd kolla21:02
sdakesweet21:02
sdakeyou found a bug in the docs ;)21:02
Pavonor can I cp -r etc/kolla /etc/21:02
sdakehow about in /usr/share/kolla/etc?21:02
DavieyPavo: "pip install kolla" will install the latest release, which is Mitaka.  git clone .. ; cd kolla ; pip install , will install the dev release21:02
sdakepavo we sort of made up the docs in the review via iteration21:03
Pavosdake /usr/share/kolla/etc_examples/kolla/21:03
sdakepavo i didn't specifically tet tthem21:03
sdakePavo cool - will get that into the docs right now21:03
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Deployment, Readiness, Safe Shutdown, & Scaling for glance-api  https://review.openstack.org/35489521:03
Pavoso... cp /usr/share/kolla/etc_examples/kolla/ /etc/21:04
sdakepavo cp -aR I think21:04
Pavosdake cp -r  /usr/share/kolla/etc_examples/kolla/ /etc/21:05
Pavoworked21:05
*** HyperJohnGraham has quit IRC21:06
wznoinskPavo: tox -e genconfig should work too21:09
sdaketox is not available in a pip install21:09
sdakewznoinsk - docs have changed21:09
wznoinskah, ok21:09
sdakeI'd encourage everyone to read this review: https://review.openstack.org/35618121:09
sdakeits a pretty significant change in thinking of how things should be done21:10
sdakebut its the only sanitary path forward i know of atm21:10
sdakemaybe Daviey has better ideas on how to handle it going forward21:10
Pavois it possible to add the tools directory from git into the pip install?21:10
*** dmueller-evoila has quit IRC21:11
sdakepavo if that needs to be done i think we should do so selectively21:11
sdakepavo which tools do you need specifically?21:11
sdakei thought we already did that though, in /usr/share/kolla21:11
PavoI deffenantly think the docs should be sectioned out for different distros, ie.. CentOS cmds only shows for CentOS and Unbuntu cmds only shows Ubuntu21:12
sdakepavo the answer for current stable releaes is no21:12
Pavook sdake no problem21:12
sdakepavo yup we got that on our plates21:12
Pavothe clean scripts are a god sin to me21:12
sdakepavo we plan to do that is what i mean21:12
wznoinsksdake: thanks, was thinking of ansible way of verifying deps, +121:12
sdakevalidating deps would be great21:13
wznoinsk(and was hit badly by not having everything on the right version a couple of times too)21:13
sdakeI think we need to also tell what deps should be there up front21:13
Pavoagreed21:14
*** jtriley has joined #openstack-kolla21:16
*** egonzalez90 has quit IRC21:16
sbezverkinc0: ping21:23
inc0here21:24
kfox1111sbezverk: ingress has some interesting limitations.21:25
sbezverkinc0: so the workaround with script worked21:25
inc0cool, just put it in if and we're good21:25
sbezverkkfox1111: I am not fun of it either, but other folks suggested it is a way to go21:26
kfox1111we still need to do a vip somehow. it doesn't really cover how to do that.21:26
sbezverkinc0: yep it is already with if, I will push it for review soon21:26
kfox1111it also seems to want to bind to port 80/443 and own it.21:26
inc0cool, thanks, I'm glad we got this figured out21:27
kfox1111so everything shows up there, instead of the regular ports. which means we'd need dns entries for everything and need to redo all the public url's in the keystone bootstraps.21:27
kfox1111I'm guessing thats a show stopper?21:28
sbezverkinc0: there is still mistery, how it could work in stable mitaka ;-)21:28
sbezverkkfox1111: we already introduced kube public kind of vip which is used for public urls21:29
kfox1111where's that?21:29
kfox1111kolla_kubernetes_external_vip21:30
sbezverkkfox1111, please check kubernetes.yml21:30
sbezverkyes21:30
kfox1111yeah. I see it.21:30
sbezverkso it is just a hook, you can then plumb it to whatever you want21:30
kfox1111but with ingress, you'd use it as glance-api.cloud.pnnl.gov:80 that resolves to kolla_kubernetes_external_vip21:30
sbezverkI use it with external haproxy21:30
kfox1111we can't register kolla_kubernetes_external_vip as the address for public as the port has to be port 80 or 443.21:31
kfox1111ingress wants to either use host based naming, or path based naming.21:32
kfox1111so kolla_kubernetes_external_vip:443/glance-api/... might work.21:32
sbezverkkfox1111: as I said I am not ingress advocate, but I am not in a position to decide anything, hence my review has 021:32
kfox1111who's deciding that one?21:33
sbezverkI would defer it to folks who suggested ingress21:33
kfox1111who's that?21:33
sbezverkwirehead and David were suggesting to use it21:33
kfox1111k. I'll try and talk with them. thx.21:34
wirehead_I don’t think I have a super-strong opinion about ingress versus otherwise.21:34
sbezverkinc0: one more question.. do we still have requirement for rabbitmq to have host name running rabbitmq resolvable by dns and pingable by its name?21:34
wirehead_Ingress is probably the right long-term direction, but that also complicates install.21:35
wirehead_Ergo, I’m fairly OK with doing something and then ripping it out and replacing it with something else later.21:38
*** dave-mccowan has quit IRC21:38
wirehead_(Actually, I’m OK with that even in shipping production stable code, as long as the users are prevented from unpleasant consequences)21:39
kfox1111k.21:39
*** daneyon has joined #openstack-kolla21:39
*** rstarmer has quit IRC21:42
*** daneyon has quit IRC21:43
sdakefor those looking to buy gold - this is a monster good price: http://www.ebay.com/itm/12155631147121:44
inc0sbezverk, not for single node21:45
inc0it's requirement for clusters21:46
sbezverkinc0: thank you that is whatI thought too21:46
*** jtriley has quit IRC21:47
kfox1111I'm going to try the ingress thing real quick with path based prefixes. that would be a simple solution if it works.21:51
kfox1111and see if multiple ingress rules in different yamls will merge. if not, then I think its pretty much a show stopper.21:53
sbezverkwirehead_: ping21:53
wirehead_k21:54
*** vhosakot has quit IRC21:54
sbezverkwirehead_: I see some instability in bootstraping, it seems there is some sort of racing between differnet container in a job. I suspect some containers try to access DB before it got created and they fail and then retry21:55
sbezverkeventually it goes through but sometimes I failures especially in db upgrade process21:56
sbezverkI think it will be much safer to split db create and db upgrade into two different job and start them after 1st succeed21:57
sbezverkwhat do you think?21:57
kfox1111sbezverk: +121:58
wirehead_ok21:58
sbezverkI have been fighting whole day today trying to see the pattern of this failures21:58
wirehead_Failing for poorly-explained reasons on a regular basis is for other OpenStack installation and orchestrating tooling sets. :D21:59
kfox1111wirehead_: also +1. :)21:59
wirehead_Well, you might try merging all of the steps into one bash script.21:59
wirehead_Potentially with a sleep, instead of 2 jobs.21:59
sbezverkwirehead_: cool then I document this behavior as a sophisticated feature21:59
kfox1111if split, then you can use the db upgrade job seperate anyway for major version upgrades.22:00
wirehead_I did that on some of my later bootstrap jobs because david pointed out how many discrete container jobs there were.22:00
wirehead_Hm.  Also true.22:00
kfox1111the tasks really shoudl be seperate. then ansible workflow can order them.22:00
sbezverkwirehead_ they use differnet images, not sure if putting everything in one script is good solution22:00
*** dave-mccowan has joined #openstack-kolla22:01
wirehead_yeah, kfox1111 is right about that22:01
sbezverkI can repro this instability in my cluster22:02
sbezverkso I will break neutron bootstrap into small chunks and test22:03
*** huikang has quit IRC22:06
*** huikang has joined #openstack-kolla22:06
sbezverkthe only problem I see, is kolla-kubernetes will not render these chunks22:08
kfox1111what do you mean?22:08
sbezverkkfox1111: I mean there is bootstrap resource and it is associated with 1 template22:09
kfox1111and we can't have more then one currently?22:10
sbezverkif I have 4 templates for bootstrap now, I am not sure if it can be associated with multiple templates22:10
kfox1111we need a way to be able to name each one individually and launch/status it.22:10
*** egonzalez has joined #openstack-kolla22:11
*** huikang has quit IRC22:11
kfox1111hmm... made a second ingress type object, loaded it, and its not working....22:12
*** huikang has joined #openstack-kolla22:13
kfox1111gona nuke the ingress and see if it comes back healthier.22:13
kfox1111maybe it is not auto refreshing config right.22:14
kfox1111yup. it came back better. it merged both ingress rules...22:14
kfox1111gotta try path based next.22:14
*** dave-mccowan has quit IRC22:18
sdakesup egonzalez22:23
sdakesup huikang22:23
*** dave-mccowan has joined #openstack-kolla22:24
egonzalezHi sdake22:24
huikangHi Sdake22:25
kfox1111ingress just crashed.. :/22:26
openstackgerritAbishek Subramanian proposed openstack/kolla: Fix the location of Kolla configurations in the evaluation workflow  https://review.openstack.org/35752022:30
*** ntpttr has quit IRC22:31
kfox1111arg...22:33
kfox1111unless we do an ingress reverse proxy of some kind, path based will be expensive.22:33
kfox1111no, let me reword that. if we do it path based, we must use a reverse proxy.22:34
kfox1111which is expensive.22:34
kfox1111k. my vote is -1 on ingress for now.22:34
*** ntpttr has joined #openstack-kolla22:36
sbezverkkfox1111: maybe we should put aside all fancy stuff until we get basics? like running vm and pinging from that VM to upstream router :-)22:37
kfox1111I'm tangengially working on kolla-kubernetes right now. was looking at ingress for a different project and finding out limitations that affect kolla-kubernetes. so relaying that info as I find things.22:38
Pavook I have to have something wrong either in globals.yml or something because its getting stuck at Starting Neutron-server even on stable/mitaka deployment22:39
*** inc0 has quit IRC22:42
Pavosomeone mind taking at look at this and tell wtf I am doing wrong22:42
*** inc0 has joined #openstack-kolla22:42
Pavohttps://www.dropbox.com/s/2694rq4mkojt9ef/globals.yml?dl=022:42
Pavothis is my network diagram https://www.lucidchart.com/invitations/accept/edf94421-2e30-4f04-a035-0c02676cf40022:43
PavoI have no clue why it stops networking on target nodes22:44
*** daneyon has joined #openstack-kolla22:45
*** huikang has quit IRC22:48
*** huikang has joined #openstack-kolla22:48
*** daneyon has quit IRC22:50
*** huikang has quit IRC22:53
openstackgerritSerguei Bezverkhi proposed openstack/kolla: Kube OVS DB to listen on tcp:port and neutron-ovs-agent to use script  https://review.openstack.org/35753322:59
*** Pavo has quit IRC23:00
*** haplo37__ has quit IRC23:03
*** Pavo has joined #openstack-kolla23:06
kfox1111wirehead_: are you different then wirehead:?23:07
*** michauds has quit IRC23:07
wirehead_yah23:08
kfox1111ah. ok.23:08
*** severion has joined #openstack-kolla23:11
*** sdake_ has joined #openstack-kolla23:12
*** v1k0d3n has quit IRC23:13
*** sdake has quit IRC23:13
*** egonzalez has quit IRC23:14
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: openvswitch ip  https://review.openstack.org/35611423:14
Mech422wirehead_: hey - do you happen to know if us randoms are eligble to vote for maintainers?23:20
Mech422wirehead_: I just saw sdace's nomination come thru23:21
wirehead_i forget23:21
Mech422wirehead_: thanks - I'll ask sdace later23:23
Mech422err..sdake :-P23:23
*** inc0 has quit IRC23:41
*** ajpower has quit IRC23:41
*** eaguilar has joined #openstack-kolla23:50
*** zhurong has joined #openstack-kolla23:53
*** ajpower has joined #openstack-kolla23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!