Friday, 2016-08-26

wally__hi sbeverk, there00:00
wally__how can I just ping someone here00:00
sdakei am off to get dinner00:00
sdakesbezverk_ is probaby asleep by noow - on eas tcoast00:00
sdakebbiaf00:00
jrich523enjoy sdake00:00
wally__ok,00:00
wally__hi jrich523, there00:01
jrich523that pings, but its based on my client00:01
wally__did u create a ovs bridge when u do the deploy?00:01
jrich523nah00:01
jrich523im still struggling with other parts00:01
jrich523mostly the docker/ansible multinode part00:01
jrich523but, im close00:01
jrich523real close00:01
wally__did u do the kolla-ansible deploy?00:01
jrich523if this populates my registry correctly, then i'll be able to deploy00:01
*** chas has quit IRC00:02
jrich523not yet00:02
wally__ok i c00:02
jrich523just finished building the docker registry00:02
jrich523now im building the images and pushing them in to the registry00:02
jrich523then i'll do the deploy00:02
aNupoiscjrich523: you need to run docker, build images and then run deploy00:03
jrich523yeah docker is setup no the controller, its compiling now and populating registry00:04
*** zhurong has quit IRC00:04
wally__anyone know do i need to create ovs bridge be-ex when do the deploy?00:04
jrich523im configuring the registry on the next node now00:04
jrich523ok, node01 is configured for registry00:06
jrich523just waiting on the image creation00:06
aNupoiscwally__ what are you trying to deploy? Openstack right?00:06
aNupoiscwally__ or is it ceph00:06
aNupoiscor something00:06
jrich523hey he has a all in one kolla setup00:06
wally__yeah, openstack00:06
wally__use kolla, AIO00:07
aNupoiscwally__ yeah then you don't need create ovs bridge00:07
wally__I know it use ovs, but i check the openvswithc.config file, it will create a bridge "br-ex"00:08
wally__right00:08
wally__bridge_mappings = phynset1-ex00:08
wally__bridge_mappings = phynset1: br-ex00:08
aNupoiscwally__ this was my globals http://pastebin.com/BM1W6LC500:08
wally__but i do not see the br-ex is create in my server when i do ifconfig00:09
aNupoiscwally__ also there is a option # Valid options neutron_plugin_agent: "openvswitch"00:09
aNupoiscwhere you can specify linuxbridge if you want00:09
aNupoiscwally__ ahh00:09
wally__ok i c00:10
aNupoiscwally__ i never faced that. Interesting00:10
wally__so u have br-ex create?00:10
wally__can u output your ifconfig for me please00:10
aNupoiscwally__ i am actually right now trying to deploy ceph so don't have setup for openstack00:10
*** bmace has quit IRC00:11
aNupoiscwally__ but yeah when i tried couple of weeks back that time the globals. yml above worked00:11
*** bmace has joined #openstack-kolla00:11
wally__on your globla,ayml file00:11
wally__neutron_external_interface: "p787p1"00:11
wally__the interface is a bridge?00:11
aNupoiscthat's my bridge00:11
wally__or a phycial net working interface?00:12
Pavowally__ kolla should build the br-ex interface based off the interface you use as neutron_external_interface setting in globals00:12
Pavoand neutron_external_interface should be named as the what the host sees the physical interface00:12
aNupoiscwally__ its a physical one00:13
aNupoiscwally__ that i had00:13
wally__oh u mean, kolla will create a br-ex and add the br to the port:neutron_external_interface?00:13
Pavoeth, en, etc....00:13
Pavono what ever interface you set in your globals.yml file kolla will use to to make the br-ex interface off of00:13
wally__like #ovs-vsctl add-port br-ex neutron_external_interface?00:14
Pavono, what interface did you set in globals.yml as your neutron_external_interface?00:14
wally__eth100:14
wally__then what is the bridge,00:15
Pavothen on the network node you should be able to do a ifconfig or ip addr and see br-ex which is bridged to eth100:15
wally__i use allinone, do i need to long to the neutron docker container?00:16
Pavophysical to logical it should go eth1 -> br-ex -> ovs -> br-int00:16
Pavoso for allinone all interfaces inside globals.yml should be the same interface00:17
wally__so eth1 is my pyhsical network interface, and i should br-ex showing when i check ifconfig, right00:18
Pavoyes00:18
Pavoif there isn't a br-ex or ovs-bridge then something went wrong00:18
wally__i have 4 nics, eth0, eth1. eth2, eth3, i use eth0 for management. so I can use eth1 or eth2,, eth3 for the bridge?00:19
wally__correct?00:19
Pavonot sure for allinone deployment00:19
PavoI think allinone deployment uses the same interface for all00:19
Pavoie.... as the title says allinone00:20
wally__u mean, use eth0 as well00:20
Pavoyes00:20
wally__so but if this is the case, then during the deploying, it will disconnect when it create bridge,00:21
*** david-lyle has quit IRC00:21
Pavofor a mill sec yeah, but when its creating this section all the cmds its needs is already there and should reconnect afrer00:22
Pavoafter00:22
PavoI could be completely wrong here but thats how I see it00:24
PavoI haven't rried allinone deployment yet00:24
Pavotried*00:24
jrich523ok, starting the deploy!00:25
jrich523ugh00:25
jrich523rewind!00:26
*** fguillot has joined #openstack-kolla00:37
openstackgerritLi Yingjun proposed openstack/kolla: Add option ceilometer_database_type for ceilometer  https://review.openstack.org/34588100:40
Pavowally__ maybe this article will help you00:45
Pavohttp://mntdevops.com/2016/08/08/iaas-2/00:45
*** salv-orlando has joined #openstack-kolla00:46
openstackgerritLi Yingjun proposed openstack/kolla: Fix toolbox permission issue when enabling ceph  https://review.openstack.org/35897300:46
*** salv-orlando has quit IRC00:51
*** unicell has quit IRC00:58
*** unicell has joined #openstack-kolla01:00
*** unicell1 has joined #openstack-kolla01:01
*** unicell has quit IRC01:01
*** zhubingbing has joined #openstack-kolla01:02
*** Jeffrey4l__ has joined #openstack-kolla01:04
*** dave-mccowan has quit IRC01:06
*** ssurana has quit IRC01:16
*** zhurong has joined #openstack-kolla01:17
*** huikang has joined #openstack-kolla01:19
*** aNupoisc has quit IRC01:26
*** dave-mccowan has joined #openstack-kolla01:29
*** daneyon has joined #openstack-kolla01:31
*** zhiwei has joined #openstack-kolla01:33
*** daneyon has quit IRC01:36
*** unicell1 has quit IRC01:52
*** kbaegis has joined #openstack-kolla01:53
*** salv-orlando has joined #openstack-kolla01:53
*** wally__ has quit IRC01:55
*** salv-orlando has quit IRC02:04
*** fguillot has quit IRC02:06
*** sdake_ has joined #openstack-kolla02:08
*** fguillot has joined #openstack-kolla02:09
zhubingbinghello02:11
*** sdake has quit IRC02:12
*** chas has joined #openstack-kolla02:14
kbaegiszhubingbing: Hello02:14
*** zhubingbing has quit IRC02:16
*** zhubingbing has joined #openstack-kolla02:16
*** sdake_ has quit IRC02:17
*** chas has quit IRC02:18
*** sdake has joined #openstack-kolla02:19
zhubingbingsdakeļ¼Œ morning02:19
*** sdake has quit IRC02:23
*** sdake has joined #openstack-kolla02:29
*** lamt has joined #openstack-kolla02:32
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Doc for multi node development  https://review.openstack.org/36085302:34
openstackgerritJeffrey Zhang proposed openstack/kolla: repair gnocchi  dockerfile  https://review.openstack.org/34838802:39
sdakeJeffrey4l__ did you do any work on the oosic cluster?02:42
Jeffrey4l__sdake, no.02:42
sdakeJeffrey4l__ - ok we are on scenario #2 atm02:42
sdakeit is almost finished02:42
Jeffrey4l__cool.02:43
sdakemay need hep setting up next scenario02:43
Jeffrey4l__OK.02:43
Jeffrey4l__i can help today.02:43
sdakei dont know what is required to make ceph run with caching02:43
sdakewhich is scneario 302:43
sdakei have been thinking we may just cut this scenario out02:43
sdakeand go straight to linuxbridge02:44
Jeffrey4l__I can try ( but never tried)02:44
Jeffrey4l__we all all ssd, the ceph cache should not improve the performance very much.02:44
Jeffrey4l__The disks we use are all ssd..02:45
sdakeyup02:45
sdakewe are also running different scenarios now02:45
sdake128x96, 128x48, 128x2402:45
sdakeand tempest02:46
sdakethere is one script that runs these02:46
sdakei wrote a script to get the data out but it doesnt seem to work with deployments02:46
sdakeso i will ahve to sort that out aftre this rally finishes02:46
Jeffrey4l__we are running tempest now?02:47
sdakerally02:47
sdaketempest is part of the script to run all 4 cases02:47
sdakelook at the script02:47
sdakeso you understand whatit does02:47
sdakei think we need a rally deployment use operation in the reporting02:48
sdakeit takes about 3 hours to run rally now02:48
sdakemaybe 402:48
sdakeall 4 scenarios02:48
sdakeall 4 cases we test02:48
*** britthouser has quit IRC02:48
*** britthouser has joined #openstack-kolla02:49
Jeffrey4l__lost the context.02:49
Jeffrey4l__:(02:49
sdakeneed me to do something on my side?02:50
Jeffrey4l__not clear what should do next.02:50
sdakelook at scripts in ~/rally.git02:50
sdakeattach to tmux02:50
*** ntpttr has quit IRC02:53
*** fguillot has quit IRC02:53
*** brad[] has quit IRC02:55
*** ntpttr has joined #openstack-kolla03:00
*** salv-orlando has joined #openstack-kolla03:03
openstackgerritzhubingbing proposed openstack/kolla: Add gnocchi ansible role  https://review.openstack.org/34935103:05
*** salv-orlando has quit IRC03:06
sdakeJeffrey4l__ just lot access to the clussteer for some reaosn03:11
Jeffrey4l__sdake, do not get the point..03:12
*** sdake has quit IRC03:12
*** sdake has joined #openstack-kolla03:13
Jeffrey4l__sdake, do not get the point. :(03:13
Jeffrey4l__OK lot-> lost.03:13
sdakepoint of which03:13
*** brad[] has joined #openstack-kolla03:13
sdakeoh i'm back in03:14
sdakenm03:14
*** inc0 has joined #openstack-kolla03:19
*** daneyon has joined #openstack-kolla03:20
openstackgerritTin Lam proposed openstack/kolla: Add ansible role for vmtp container  https://review.openstack.org/35830503:23
*** daneyon has quit IRC03:24
kbaegisAnyone played around with novnc baseurl?03:32
kbaegisI'm actually hiding horizon behind an apache reverse proxy. UI works perfectly, but vnc is still broken03:32
kbaegisI'm noticing that clients are still redirected to the old url03:34
*** Jeffrey4l__ has left #openstack-kolla03:34
*** Jeffrey4l__ has joined #openstack-kolla03:34
Jeffrey4l__kbaegis, change it to the url u want.03:34
kbaegisI did, restarted the container for novnc03:35
kbaegisDo I need to do that in nova-api instead?03:35
kbaegisCause clients are still getting redirected to the old url03:35
Jeffrey4l__kbaegis, how did u changed it?03:35
Jeffrey4l__kbaegis, you should change the nova_compute03:36
kbaegisJeffrey4l__: Thanks.  I have copy_always, and was just editing /etc/kolla/nova-novncproxy/nova.conf03:36
kbaegisrestart the container03:36
mdnadeemJeffrey4l__, sdake , coolsvap_ , can you please review https://review.openstack.org/#/c/350629/03:37
Jeffrey4l__kbaegis, check this http://docs.openstack.org/developer/kolla/advanced-configuration.html#openstack-service-configuration-in-kolla03:37
Jeffrey4l__mdnadeem, ok03:38
*** gfidente has quit IRC03:40
*** duonghq has joined #openstack-kolla03:42
kbaegisSo that's the correct way to do it, but mine works too, no :)03:42
kbaegisIt's ultimately restarting the container and copying it from a shared directory03:42
kbaegisnova compute it is :)03:43
kbaegisOkay, did that in nova-compute03:45
kbaegisHorizon is still redirecting to the wrong host03:45
*** zhubingbing has quit IRC03:53
kbaegisJeffrey4l__: So on that link- is it based on fqdn or hostname?03:54
kbaegis'myhost'03:55
*** sdake has quit IRC03:56
Jeffrey4l__use fqdn, or use the custom configuration file.03:59
kbaegisthat's what I was asking.  Tied to the path :)03:59
Jeffrey4l__change nova_base url in  the /etc/kolla/config/nova.conf  and run kolla-ansible reconfigure04:00
*** huikang has quit IRC04:06
*** salv-orlando has joined #openstack-kolla04:07
*** dave-mccowan has quit IRC04:09
kbaegisJeffrey4l__: Thanks.  Just did that04:10
*** mark-casey has quit IRC04:10
*** senk has joined #openstack-kolla04:14
kbaegishow about cpu_allocation_ratio?  Anyone play with that?04:14
*** salv-orlando has quit IRC04:14
*** chas has joined #openstack-kolla04:15
kbaegisDidn't work.  Old url is still being used04:17
kbaegisFile was '/etc/kolla/config/crucible.lab/nova/nova.conf', aio04:19
*** chas has quit IRC04:20
*** sdake has joined #openstack-kolla04:22
kbaegisWell, copied nova.conf to the root config directory and symlinked the fqdn to the hostname04:24
kbaegisretrying04:24
duonghqkbaegis: what are you doing?04:27
kbaegisduonghq: trying to alter the baseurl for novnc proxy.  I'm hiding it behind a reverse httpd proxy04:28
duonghqkbaegis: roger04:28
duonghqhow many node you have for now04:29
kbaegisduonghq: aio still04:29
duonghqokay04:30
sdakeJeffrey4l__ we need the json output04:30
Jeffrey4l__sdake, yep.04:30
sdakeJeffrey4l__ so we can parse it and produce rsts with it04:30
Jeffrey4l__just test the html group report.04:30
sdakealso would like html output04:30
Jeffrey4l__result seem have no such parameter.04:31
kbaegisYay!  That worked04:33
kbaegisSo either the path is not accurately linked to fqdn (maybe just hostname)04:33
kbaegisOr it's only reading out of /etc/kolla/config/04:33
sdakeJeffrey4l__ i'm all done, i wantedd to see if switching deployments got us at the other data04:33
Jeffrey4l__sdake, ok04:33
*** kbaegis has quit IRC04:35
sdakeervice-level agreement04:41
sdakeCriterionDetailSuccess04:41
sdakesomething_went_wrongUnexpected error: Invalid scenario argument: 'Image with pattern '^cirros.*uec$' not found'04:41
*** chas has joined #openstack-kolla04:46
*** chas has quit IRC04:50
sdakeJeffrey4l__ aftere this finishes i want an hour with oprofile04:52
sdakethen on to next scenario04:52
Jeffrey4l__sdake, the script seems good now. will check later. need be out. brb04:52
sdakesound good?04:52
Jeffrey4l__sdake, good. no idea about the oprofile :(04:53
Jeffrey4l__never used it.04:53
sdakei got it04:53
Jeffrey4l__brb04:53
Jeffrey4l__need check the collected report. should OK. but double check is batter :)04:54
*** v1k0d3n has quit IRC04:56
*** duonghq has quit IRC04:56
*** unicell has joined #openstack-kolla05:01
*** unicell has quit IRC05:02
*** unicell has joined #openstack-kolla05:03
*** unicell has quit IRC05:03
*** unicell has joined #openstack-kolla05:04
*** unicell has quit IRC05:04
*** inc0 has quit IRC05:09
*** unicell has joined #openstack-kolla05:10
*** senk has quit IRC05:11
*** salv-orlando has joined #openstack-kolla05:17
*** unicell1 has joined #openstack-kolla05:19
*** unicell has quit IRC05:19
*** salv-orlando has quit IRC05:20
*** salv-orlando has joined #openstack-kolla05:25
*** salv-orlando has joined #openstack-kolla05:26
sdakeJeffrey4l__ to get any good data out of oprofile we will need dbg info files in them05:27
sdakeJeffrey4l__ so for hte moment going to just run a quck test05:27
sdakeand then if we have time i'll build images with dbginfo in them05:28
sdakei dont want to break the images we have ;-)05:28
*** zhubingbing has joined #openstack-kolla05:32
*** senk has joined #openstack-kolla05:49
*** duonghq has joined #openstack-kolla06:05
*** unicell has joined #openstack-kolla06:11
*** unicell1 has quit IRC06:11
*** chas has joined #openstack-kolla06:12
*** fragatin_ has joined #openstack-kolla06:12
*** fragatina has quit IRC06:15
*** fragatin_ has quit IRC06:17
*** shardy has joined #openstack-kolla06:23
*** egonzalez90 has joined #openstack-kolla06:24
openstackgerritMd Nadeem proposed openstack/kolla: Added Ansible playbook for congress deploy  https://review.openstack.org/35062906:28
*** bjolo has quit IRC06:30
*** bjolo has joined #openstack-kolla06:31
*** salv-orlando has quit IRC06:31
*** caowei has joined #openstack-kolla06:33
sdakeJeffrey4l__ - something is busted on the test cluster - complaining about domian not found in all client calls06:33
sdakeJeffrey4l__ inc0 said he fixed this by adding a user domain06:34
sdakebut we already have a default domain06:34
sdakeand i read the relevant bugs, our admin-openrc.shlooks ok06:34
*** fragatina has joined #openstack-kolla06:36
*** Serlex has joined #openstack-kolla06:39
*** fragatina has quit IRC06:41
*** mewald has joined #openstack-kolla06:44
sdakepost deploy is broken06:45
*** salv-orlando has joined #openstack-kolla06:45
sdakeinstead of ID shoudd be NAME for the domains06:45
Jeffrey4l__sdake, back06:47
sdakeJeffrey4l__ i am running an oprofile test atm06:47
Jeffrey4l__sdake, still failed?06:47
sdakeas soon sa rally wraps up06:47
sdaketime to move on to scenario 306:47
sdakecan you get that kicked off, getting pretty late here06:47
sdakei think this rally job will take about 45 mins or so06:47
Jeffrey4l__cool. is the domain not found solved?06:47
Jeffrey4l__sdake, i think i can fix it.06:47
sdakei solved it06:48
sdake%s/ID/NAME with domain in the lines06:48
Jeffrey4l__yep06:48
sdakei'll get it fixed06:48
Jeffrey4l__or change the `default` domain name to `Default`06:49
*** aNupoisc has joined #openstack-kolla06:49
sdakeJeffrey4l__ jump in tmux - I want ot show you oprofile06:51
sdakethis is something you need to know how to use06:52
openstackgerritMathias proposed openstack/kolla: Added influxdb role  https://review.openstack.org/34644906:53
*** Mr_Broken has joined #openstack-kolla06:55
*** daneyon has joined #openstack-kolla06:56
sdakewhat is #3 in cpu cycles06:59
sdakeJeffrey4l__ ^06:59
sdakethats right, #1 is java06:59
sdake30% of control node consumed by java06:59
sdake25% python07:00
sdake13% by hekad07:00
Jeffrey4l__OK. clearly about his.\07:00
Jeffrey4l__this07:00
sdakethere was one above hekad07:00
Jeffrey4l__rabbitmq07:01
sdakebeam.smp07:01
sdakeya rabbitmq07:01
sdakepretty useful huh?07:01
Jeffrey4l__yes.07:01
sdaketells me we need to eject java from our system if it were possibel :(07:01
*** daneyon has quit IRC07:01
sdakewhen you get done with that simple view i'll show you a more comlex one07:01
sdakelet me know07:01
sdakein channel07:02
Jeffrey4l__sdake, ok. go on07:02
sdakepretty flat across symbols07:04
*** unicell has quit IRC07:04
sdakethat is what you like to see when optimizing07:04
Jeffrey4l__yep07:04
sdakeit typically means those symbols are highly optimized already07:04
sdake6687      0.2704  vmlinux-4.4.0-34-generic sudo                     /vmlinux-4.4.0-34-generic07:05
sdakebusy users of sudo :)07:05
Jeffrey4l__hmm. Cool07:05
Jeffrey4l__sdake, but few we can do in Kolla, right07:07
sdakenot a whole lot unless it involves ejecting elasticsearch07:08
*** zhubingbing has quit IRC07:08
Jeffrey4l__ok.07:08
sdakehekad cpu hog07:09
sdakeuses the kernel way too much07:09
Jeffrey4l__yes.07:09
sdake7% of the total sample time of the systme is spent processing hekad systemcalls07:09
Jeffrey4l__can we get more info about what kernel call he used?07:10
sdaketry --callgraph07:10
sdakeon centos the system call names are printed out07:11
sdakebut on 14.04 looks like that is no bueno07:11
sdakenope it is treating the kernel as one blob07:11
Jeffrey4l__ok07:11
sdakeannoying07:11
Jeffrey4l__but very useful tools :)07:12
*** unicell has joined #openstack-kolla07:12
sdakelets go back to the run of rally07:12
*** salv-orlando has quit IRC07:16
mewaldgit noob needs help :D https://review.openstack.org/#/c/346450/ How can I solve this "Patch in Merge Conflict" thing?07:16
Jeffrey4l__mewald, make a local rebase origin/master and solve the merge conflict, then `git review`07:17
*** jrich523 has quit IRC07:23
*** jrich523 has joined #openstack-kolla07:24
bjolomorning07:24
*** zhubingbing has joined #openstack-kolla07:27
*** egonzalez90 has quit IRC07:38
*** matrohon has joined #openstack-kolla07:39
*** zhiwei has quit IRC07:44
*** caowei has quit IRC07:44
zhubingbingsnake ping07:48
zhubingbingsnake ping07:49
zhubingbingsnake ping07:49
zhubingbingsdake  ping07:49
sdakesnake here07:49
sdakesup07:49
zhubingbinghi,07:50
sdakemewald - git pull --rebase07:50
zhubingbinghttps://review.openstack.org/#/c/348388/07:50
duonghqsnake, where is snake07:50
zhubingbingSdake, you look at this branch can not hurry review merger, because I role gnocchi rely on this branch07:51
zhubingbing- -duonqhq sorry, I'm spelling mistakes.07:52
duonghqjust kiding, don't mind07:52
zhubingbing --ļ¼‰07:52
duonghqzhubingbing: As I understand: you have other ps depend on this ps?07:54
duonghqor you just want to make this feature running?07:54
*** salv-orlando has joined #openstack-kolla07:54
zhubingbingrole gnocchi branch before the dockerfile gnocchi branch was carried out with the rebase operation07:57
*** aNupoisc has quit IRC07:58
zhubingbingNow I submit the role gnocchi branch, which will cause the 2 branch to submit,07:58
zhubingbingping Hui Kong07:59
zhubingbingping Hui Kang07:59
duonghqzhubingbing: seem that he don't online right now08:00
duonghqand I think you should ping using his irc nick than his name08:00
zhubingbingokļ¼Œ thanks08:01
zhubingbingI need to talk to him about the issue of etcd.08:01
*** sdake has quit IRC08:05
*** sdake has joined #openstack-kolla08:05
*** berendt has joined #openstack-kolla08:18
*** openstackgerrit has quit IRC08:18
*** openstackgerrit has joined #openstack-kolla08:18
*** zhubingbing has quit IRC08:20
*** egonzalez90 has joined #openstack-kolla08:21
*** gfidente has joined #openstack-kolla08:21
*** gfidente has joined #openstack-kolla08:21
*** coolsvap_ has quit IRC08:21
*** zhubingbing has joined #openstack-kolla08:23
openstackgerritChristian Berendt proposed openstack/kolla: Add cloudkitty containers for type source  https://review.openstack.org/34929508:26
openstackgerritzhubingbing proposed openstack/kolla: Add gnocchi ansible role  https://review.openstack.org/34935108:26
openstackgerritLi Yingjun proposed openstack/kolla: Add option ceilometer_database_type for ceilometer  https://review.openstack.org/34588108:28
*** zhubingbing has quit IRC08:34
*** zhubingbing has joined #openstack-kolla08:35
openstackgerritLi Yingjun proposed openstack/kolla: Add option to support dhcp/l3 agent ha  https://review.openstack.org/35218008:36
*** zhubingbing has quit IRC08:40
duonghqwhat happen if a host is in compute group but network group?08:41
*** daneyon has joined #openstack-kolla08:44
openstackgerritAndreas Jaeger proposed openstack/kolla-kubernetes: Properly use constraints  https://review.openstack.org/36098408:46
Jeffrey4l__pbourke, ping08:47
pbourkeJeffrey4l__: hey08:47
Jeffrey4l__do u have any idea the default gw 172.21.112.1 can not reach on isic pbourke08:48
*** sdake has quit IRC08:48
Jeffrey4l__osic08:48
pbourkewhat do you mean it can not reach osic?08:49
*** daneyon has quit IRC08:49
Jeffrey4l__pbourke, i can not ping the 172.21.112.1 ip08:49
Jeffrey4l__it should be the default gw.08:49
Jeffrey4l__for p4p1 interface08:49
Jeffrey4l__with vlan 73308:50
openstackgerritAndreas Jaeger proposed openstack/kolla-kubernetes: Cleanup tox.ini constraints  https://review.openstack.org/36098408:52
pbourkeJeffrey4l__: on now08:52
pbourkeim not sure but can have a look08:52
pbourkeJeffrey4l__: whats the latest in the testing08:52
Jeffrey4l__OK. thanks.08:52
Jeffrey4l__pbourke, i need setup a external network now for some rally tests. They need connect the vms through ssh from the deploy node.08:54
pbourkeJeffrey4l__: there is a script to do that08:57
Jeffrey4l__pbourke, this init-runonce-osic ?  if so, it set  up a wrong external network.08:58
*** athomas has joined #openstack-kolla08:59
Jeffrey4l__pbourke, found some thing wrong. fixing..08:59
pbourkeok09:06
*** salv-orl_ has joined #openstack-kolla09:17
*** salv-orlando has quit IRC09:20
*** dwalsh has joined #openstack-kolla09:20
*** zhubingbing has joined #openstack-kolla09:36
*** duonghq has left #openstack-kolla09:57
*** zhubingbing has quit IRC09:59
*** zhurong has quit IRC10:01
*** rmart04 has joined #openstack-kolla10:09
*** zhubingbing_ has joined #openstack-kolla10:10
*** Mr_Broken has quit IRC10:24
*** Mr_Broken has joined #openstack-kolla10:25
*** kbaegis has joined #openstack-kolla10:28
kbaegisGood morning everyone10:28
egonzalez90morning kbaegis10:29
kbaegisegonzalez90: We're up early :)10:30
*** Mr_Broken has quit IRC10:30
egonzalez90not really, EMEA time ;)10:30
kbaegisOh, gotcha10:31
kbaegis'round noon there?10:31
*** daneyon has joined #openstack-kolla10:33
egonzalez90midday10:33
*** zhubingbing_ has quit IRC10:35
*** zhubingbing has joined #openstack-kolla10:36
openstackgerritzhubingbing proposed openstack/kolla: repair aodh image dockerfile  https://review.openstack.org/35071510:37
*** daneyon has quit IRC10:38
*** chas_ has joined #openstack-kolla10:40
*** zhiwei has joined #openstack-kolla10:41
*** chas has quit IRC10:42
*** rhallisey has joined #openstack-kolla10:43
*** zhiwei has quit IRC10:46
*** JoseMello has joined #openstack-kolla11:01
openstackgerritzhubingbing proposed openstack/kolla: repair aodh image dockerfile  https://review.openstack.org/35071511:08
*** Mr_Broken has joined #openstack-kolla11:11
*** shardy is now known as shardy_lunch11:13
*** zhubingbing has quit IRC11:21
*** salv-orl_ has quit IRC11:27
*** bootsha has joined #openstack-kolla11:34
*** bootsha has quit IRC11:44
*** senk has quit IRC11:47
*** chas_ has quit IRC11:51
*** jamesdenton has quit IRC11:57
*** shardy_lunch is now known as shardy12:11
*** senk has joined #openstack-kolla12:16
*** pece has joined #openstack-kolla12:16
*** jtriley has joined #openstack-kolla12:17
*** daneyon has joined #openstack-kolla12:21
bjolois it possible to overide network names for particular hosts? i.e. in globals i have  network_interface: "bond0"12:23
bjolobut a slew of my machines will not have a bonb012:23
bjolobond012:23
bjolothey will just have eth0 and eth0.<vlan ID>12:24
bjoloetc12:24
*** daneyon has quit IRC12:26
*** jtriley has quit IRC12:27
*** msimonin has joined #openstack-kolla12:34
*** jtriley has joined #openstack-kolla12:36
*** celo_r has joined #openstack-kolla12:37
*** senk has quit IRC12:41
*** v1k0d3n has joined #openstack-kolla12:43
*** jtriley has quit IRC12:44
Mr_Brokenhello12:46
Mr_BrokenI believe you can do it in ansible Inventory file12:46
Mr_Brokenlook at this :12:46
Mr_Brokenhttps://github.com/openstack/kolla/blob/master/ansible/inventory/multinode12:46
Mr_Broken# When compute nodes and control nodes use different interfaces,12:47
Mr_Broken# you can specify "api_interface" and another interfaces like below:12:47
Mr_Broken#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em112:47
Mr_Brokenyou can also, change the names of your interfaces on the machines, and standarize then across the infrastructure ... but thats more hustle ;]12:47
vbelMr_Broken: https://review.openstack.org/#/c/354173/2/etc/kolla/globals.yml12:54
vbelbjolo, ^12:54
Mr_Brokenuuu new documentation changes :D12:55
*** egonzalez90 has quit IRC12:55
Mr_Brokendid it get to stable/mitaka too ?12:56
Mr_Brokenor just master branch ?12:56
kbaegisAnyone have issues with scheduling boot from images?12:56
kbaegisI can boot from volumes just fine12:57
Mr_Brokensounds like glance issues12:57
Mr_Brokenhad different issues depending on glance backend and OS release12:58
Mr_Brokenfinally in stable/mitaka on ceph - everything seems to be working fine12:58
Mr_Brokenare you trying to use multinode topology and glance with file backend ?12:59
kbaegisMr_Broken: glance logs look clean12:59
kbaegisMr_Broken: I just have a little aio install12:59
Mr_Brokenso what are the issues exactly ?13:00
Mr_Brokenin what state is the Instance after trying to boot ?13:00
Mr_Brokenwhat image are you using ?13:00
kbaegisMr_Broken: Checking now.  I'm guessing either a cinder-volume timeout or something else I just fixed13:01
kbaegisI'm running kolla on gentoo, so I have to configure firewall_driver to noop13:01
kbaegisBut I just persisted it in /etc/kolla/conf/ml2_conf.ini13:02
kbaegisForgot to do that last time I reconfigured :)13:02
Mr_Brokenyae, might also be a network issue :]13:03
kbaegisMr_Broken: ovs runs on the host uncontainerized.  Works just fine.  iptables is the issue13:03
*** salv-orlando has joined #openstack-kolla13:03
*** zhurong has joined #openstack-kolla13:04
kbaegissomething about the syntax w/ iptables_manager in neutron doesn't work well with the 4.8 kernel or utilities13:04
Mr_Brokenand neutron_server (with agents) still on container ?13:04
kbaegisYeah13:04
Mr_Brokenand it works ?! :P13:04
kbaegisEverything except ovs is managed by kolla13:04
kbaegisincluding ovs-db13:04
kbaegisMr_Broken: I made it work, yes :)13:04
Mr_Brokenyou managed to get ovs uncontenerized with contenerized ovs-db ?!13:04
kbaegisI'm just that good ;)13:05
Mr_Brokenmy hero !13:05
Mr_Brokenhow ?! :P13:05
Mr_Brokenshare !13:05
Mr_BrokenI was strugling with the same problem weeks ago13:05
Mr_Broken<--- gave up and uncontenerized ovs-db and some neutron_agents ;]13:05
kbaegisI have ovs running in foreground in screen.  Command: http://hastebin.com/ebijijevex.vbs13:06
msimoninHi All13:06
msimoninI've some trouble with the reconfigure action, should I expose here or in launchpad ?13:06
Mr_Broken@kbaegis: heh, good idea :]13:08
Mr_Broken@kbaegis: did you run into any issues across reboots ?13:08
kbaegisLet's see13:09
Mr_Brokenor with ovs-ports not being removed when instance is being removed ?13:09
Mr_Brokenetc13:09
Mr_Brokenguess my multinode topology also adds a little network complexity13:10
kbaegis"mount --make-shared /run"13:10
Mr_Brokensince my l3 agents and dhcp agents actually have some work to do, and are "trying" to work in HA :P13:10
Mr_Brokenclassic :}13:10
kbaegis, firewall noop13:10
kbaegisThose were the only 3 issues13:10
kbaegisovs, noop firewall, and mount ā€”make-shared /run13:11
kbaegisAlso disabled haproxy/keepalived13:11
Mr_Brokenyeah ;]13:11
kbaegisI don't need them, since my other nodes have <4GB ram13:11
Mr_Brokenand master branch ?13:11
kbaegisYes13:11
Mr_Brokenyeah, a lot is fixed there13:11
kbaegisOnly serious drawback I'm running into: no ui support for designate, magnum, etc13:11
kbaegisAlso, w/ my setup I was having issues w/ my reverse proxy and novnc13:12
Mr_Brokendo those even fully work in master branch already ? :P13:12
kbaegisYes13:12
Mr_Brokenmagnum / designate ?13:12
kbaegisYou can make bays from cli13:12
*** dave-mccowan has joined #openstack-kolla13:12
kbaegisThe BIND backend I haven't tested w/ designate13:12
Mr_Brokeni think there are some ansible roles still missing for designate13:13
kbaegisMakes sense13:13
*** jtriley has joined #openstack-kolla13:13
Mr_Brokeni am kind of forced to use stable/mitaka branch, so I might be wrong though13:13
kbaegisipv6 would be nice with neutron too.  Also dvr13:13
Mr_Brokendvr support is being worked on currently13:14
*** sdake has joined #openstack-kolla13:14
Mr_Brokenshould be merged soon13:14
Mr_Brokenseen some blueprints13:14
*** v1k0d3n has quit IRC13:15
*** sdake_ has joined #openstack-kolla13:15
Mr_Brokenhttps://blueprints.launchpad.net/kolla/+spec/support-network-ha13:15
kbaegisThat'll be nice.  That's when I'll really move from AIO13:16
Mr_Brokenwe actually have internal stable/mitaka branch that support dvr's in HA13:16
kbaegisI need about 10k worth of equipment before I'll even get much use out of dvr13:16
Mr_Brokenbut dont have approvals from my company yet , to publish into community13:16
sdake_morning13:16
Mr_Brokenmorning13:17
kbaegissdake: morning13:17
sdake_Jeffrey4l__ - hows testing going13:17
kbaegissdake_: Wait, are there two of you?13:17
kbaegissdake_: Underscore the difference?13:17
sdake_connecte to vpn13:17
sdake_disconnects my original connection - it willl ping timeout13:17
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Getting NOVA pieces together to launch VM  https://review.openstack.org/35891113:17
*** jtriley has quit IRC13:17
Jeffrey4l__sdake_, fixed some network issue. Just try to start the test13:17
sdake_start linuxbridge test13:18
sdake_or start fio test13:18
sbezverk_sdake: vm in kube is running, how is this for good morning ;-)13:18
sdake_pbourke i gathered oprofile data13:18
sdake_pbourke interested in seeing?13:18
sdake_sbezverk_ fantastic13:18
sdake_sbezverk_ the talk on kube was accepted13:18
sdake_so I guess we should have something working by summit then ;)13:19
*** sdake has quit IRC13:19
openstackgerritMerged openstack/kolla: Fix toolbox permission issue when enabling ceph  https://review.openstack.org/35897313:19
sdake_kbaegis the way to tell its me is my cloak will be @fedora i.e. sdake@fedora/dake13:19
sdake_i do this so people dont gather my home ip address and harass my gear13:20
sdake_it is infinately more feasible to hack soemone you know13:20
Jeffrey4l__sdake_, test started13:20
sdake_then someone you don't ;)13:20
sdake_jefwhich test, fio or the scenario 3?13:20
sdake_or rather scenario 413:21
Jeffrey4l__made some small fio test.13:21
Jeffrey4l__no is scenario 413:21
Jeffrey4l__now is13:21
sdake_fio worked on scenario 213:21
sdake_?13:21
Jeffrey4l__yep13:21
Jeffrey4l__network issue.13:21
Jeffrey4l__fixed.13:22
sdake_we might as well be capturing thse fio tets13:22
msimonin(sorry for interrupting) I've some issue with the reconfigure action when using fake driver and neutron_openvswitch_agent : can anyone help me here or should I go to launchpad ?13:22
sdake_the network issue- was it codified in init-runonce-osic?13:22
Jeffrey4l__sdake_, yes.13:22
sdake_msimonin does it just fail?13:22
sdake_Jeffrey4l__ i mean did you codify the fix13:22
Jeffrey4l__sdake_, a fixed version is here /root/osic_playbooks/init-runonce-osic13:23
msimoninsdake_: yes, just like this (stable/mitaka) :13:23
msimoninTASK: [neutron | Restart the neutron_openvswitch_agent container] *************13:23
msimoninskipping: [parasilo-28-kavlan-4.rennes.grid5000.fr]13:23
msimoninskipping: [parasilo-8-kavlan-4.rennes.grid5000.fr]13:23
msimoninfatal: [parasilo-5-kavlan-4.rennes.grid5000.fr] => error while evaluating conditional: openvswitch_agent_envs['KOLLA_CONFIG_STRATEGY'] != 'COPY_ONCE'13:23
sdake_msimonin file a bug pleae13:23
sdake_msimonin may I suggest you use stable?13:24
Jeffrey4l__check the fio test html report on the deployment node:/tmp/a.html13:24
sdake_msimonin are are you aprt of the project to test openstack at 5k node scale?13:24
Jeffrey4l__sdake_, ^13:24
msimoninsdake_: yes13:24
msimoninI think the test here https://github.com/openstack/kolla/blob/stable/mitaka/ansible/roles/neutron/tasks/do_reconfigure.yml#L237 is wrong13:25
sdake_msimonin i'd still recommend stable unless you want to test all of openstack master13:25
msimoninI can fill a bug13:25
msimoninI'm using stable/mitaka13:25
sdake_there should be no problem with stable/mitaka with that code13:25
msimoninhm13:25
sdake_but pleawe filea bug13:25
msimoninActually I've some more traces I can attach them in the bug13:26
sdake_different issues?13:26
msimoninno same on with more debug to help identifying13:26
msimoninsame one*13:26
sdake_msimonin - attaching your globals.yml would be helpfu a well13:26
sdake_or linking to it13:26
sdake_;)13:26
msimoninsure13:26
sdake_what scale are you at currently with kolla?13:27
msimonin1000 compute fake driver13:27
sdake_interesting13:27
sdake_and that works?13:27
msimoninactually I had also an issue with the mariadb configuration13:27
*** aswadr_ has joined #openstack-kolla13:28
sdake_Jeffrey4l__ did you get fio output in json?13:28
msimoninthat works after removing all wrep* configuration from galera.cnf13:28
msimonin(I was using only one mariadb node)13:28
sdake_so without ha, mariadb scales just fine :)13:28
sdake_how many control nodes?13:28
Jeffrey4l__sdake_, just small test, to test my script work.13:28
Jeffrey4l__i will get the detailed test result later.13:29
msimoninsdake_: only one control13:29
sdake_1 control node for 1000 computes?13:29
msimoninyes13:29
msimoninI don't say the rally benchmarks were smooth :)13:29
sdake_we can help theree13:30
msimoninok cool !13:30
msimoninI'll be pleased13:30
sdake_we hae just been jering aroudn with getting rallly running for over 2 weeks13:30
sdake_on a 130 node physical cluster13:30
kbaegisso when I created /etc/kolla/config/ml2_config.ini, it's options didn't overwrite openvswitch-agent13:30
sdake_without fake driver13:30
kbaegisWhat did I do wrong there?13:30
sdake_maybe m2_config.ini isn't customizable13:31
sdake_but i'm not certain13:31
msimoninsdake_: how things are going with this deployment ?13:31
sdake_msimonin we can do a 123 node  deployment in roughly 20 minutes13:31
sdake_w ehae 6 scale scenarios we are testing across two different oses13:31
sdake_i think we are eliminating one of the scale test13:31
sdake_so we will haea total of 10 tests13:31
sdake_we are on test #2 now13:32
sdake_it takes about 5 hours to run rally scenarios e have defined13:32
msimoninsdake_: What strategy do you have to speed up the deployment (prefetched images ? )13:32
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Getting NOVA pieces together to launch VM  https://review.openstack.org/35891113:33
sdake_the tactical plan we executed on was to automate testing, and automate information gathering13:33
sdake_the automated testing takes about 4-5 hours13:33
sdake_sbezverk_ does that nova vm have ovs connectivity?13:33
sbezverk_sdake_ I do VM interface plugged into ovs bridge, but I need to bring up vnc pod to actually test it13:34
kbaegisNice.  boot from image is working13:35
sbezverk_sdake_: I meant I do see VM interface ..13:35
sdake_msimonin wee can share our automatation and test selection that works13:35
kbaegisThink I'm still having issues with root volumes from flavor though.  Can those timeout?13:35
pbourkeJeffrey4l__: you still on osic? need any help?13:36
sdake_msimonin i am happy with teh peed of  deployment - 20 minutes for 123 nodess13:36
sdake_pbourke if your on osic check out the file in there ending in .txt startin with oprofile13:36
*** jtriley has joined #openstack-kolla13:36
msimoninsdake_: That would be great if you can share13:36
sdake_msimonin ya give me a moment to get on the test cluster13:37
*** senk has joined #openstack-kolla13:37
*** sdake_ has quit IRC13:37
msimoninsdake_: we don't have yet a real strategy to deploy at scale just a local prefecthed registry13:37
Jeffrey4l__pbourke, no right now. the test is running.13:38
Jeffrey4l__we are testing scenario 4, now.13:38
*** rcherrueau has joined #openstack-kolla13:39
*** sdake has joined #openstack-kolla13:40
kbaegismsimonin: Use bittorrent!  ;)13:40
*** salv-orlando has quit IRC13:40
kbaegismsimonin: Docker containers can be spread by torrent, right13:40
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Document some Conventions  https://review.openstack.org/36118613:40
kbaegisUse riffle just to make it complicated13:41
*** jtriley has quit IRC13:41
msimoninkbaegis: wow right !13:42
sdakemsimonin : http://paste.fedoraproject.org/414343/21896914/ -> http://paste.fedoraproject.org/414343/2189691413:43
*** mgoddard_ has joined #openstack-kolla13:44
msimoninsdake: bookmarked :)13:44
*** berendt has quit IRC13:44
msimoninsdake: what is the diff between the two ?13:44
sdakemsimonin i'd download it - fpaste doesnt' keep data around for a long time13:46
sdakesame link13:46
sdakemsimonin fpaste is supposed to url shorten things13:46
sdakebut that doesn't seem tobe working lately13:46
msimoninok13:46
msimonin:)13:46
*** mgoddard has quit IRC13:47
sdakeJeffrey4l__ can you update nextsteps file pls13:48
Jeffrey4l__sdake, hmm, what's the detail?13:49
Jeffrey4l__next scenario?13:49
openstackgerritPaul Bourke proposed openstack/kolla: Architecture guide doc  https://review.openstack.org/36079613:53
sdakeexact steps13:55
sdakeclearly the next scenario isn't 2 its 3 i think13:55
sdakecheck etherpad13:55
sdakeits about setting up tls13:55
*** pece has quit IRC13:56
sdakehttps://etherpad.openstack.org/p/kolla-N-midcycle-osic13:56
sdakemsimonin there are several scripts in there as well as our"ignore" file13:57
*** fguillot has joined #openstack-kolla14:00
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Document some Conventions  https://review.openstack.org/36118614:01
msimoninsdake:  I filled a bug report -> https://bugs.launchpad.net/kolla/+bug/161733414:03
openstackLaunchpad bug 1617334 in kolla "reconfigure action fails on [neutron | Restart the neutron_openvswitch_agent container] " [Undecided,New]14:03
openstackgerritLi Yingjun proposed openstack/kolla: Add option ceilometer_database_type for ceilometer  https://review.openstack.org/34588114:04
*** bootsha has joined #openstack-kolla14:05
*** jtriley has joined #openstack-kolla14:07
*** senk has quit IRC14:09
*** mgoddard_ has quit IRC14:09
*** mgoddard has joined #openstack-kolla14:10
*** daneyon has joined #openstack-kolla14:10
msimoninkbaegis: to distribute over torrent do you use a modified registry ?14:13
kbaegismsimonin: The only place I've seen that model is with cisco phones14:14
*** daneyon has quit IRC14:14
kbaegisthey set up p2p connections to redistribute firmware.  Never seen it w/ docker14:14
msimoninMy point is how kolla team achiev 20min deployment for 123 nodes ?14:15
kbaegis10gb/s connections14:15
kbaegisAnd network copy14:15
kbaegisI don't think ansible hyperscales14:17
kbaegisI haven't looked into tower's architecture14:18
*** michauds has joined #openstack-kolla14:18
sdakeirtermite are we on 40gig network in the osic lab, or 20gig or 10gig?14:20
*** bootsha has quit IRC14:21
*** dwalsh has quit IRC14:23
*** inc0 has joined #openstack-kolla14:26
inc0good morning14:27
*** Mr_Broken has quit IRC14:31
*** Mr_Broken has joined #openstack-kolla14:32
*** dwalsh has joined #openstack-kolla14:34
sdakesup inc014:35
sdakewe finally have one scenario captured14:35
sdakescenartio #3 in progress now14:35
inc0yay14:35
inc0pbourke, ping14:35
pbourkeinc0: yo14:35
sdakeso i need some shell help14:35
inc0pbourke, why do we want to move haproxy to control?14:35
sdakeI've got a document that has as its first column a list of numbers14:35
sdakeinc0 i asked the same hting :)14:36
sdakepbourke said it was ineffecient14:36
inc0it'll make upgrades harder, security weaker...and I can't see much value in it14:36
pbourkeinc0: it's standard convention to run it on the control nodes14:36
*** Mr_Broken has quit IRC14:36
pbourkeit should never have been placed on the network14:36
inc0but why?14:36
inc0if you want to have public endpoints, you don't really want to expose your controllers to external net14:37
pbourkerequests on control nodes have to go to another node14:37
inc0and you always need to do this for network node14:37
inc0well, 2/3 of controllers will go on a different node anyway14:37
inc0actually more14:38
inc08/9 of requests will cross between nodes with 3 controllers14:38
pbourkeok im happy enough to leave it if you dont think it makes sense14:39
inc0well, changing arch is hell of a migration14:39
inc0control downtime will be required14:39
inc0maybe even long as ARP tables would need to reload14:39
inc0also it might break if somebody would make their arch with external network only connected to network node14:40
*** jheroux has joined #openstack-kolla14:40
inc0and just ran upgrade with pre-defined inventory14:40
inc0actually no, if they have pre-defined inventory then it's ok14:40
inc0but moment they rebase their inventory and miss this change14:41
inc0things might break14:41
inc0granted, it's not super trivial to understand why haproxy+keepalived is on network node, but it does make a lot of sense14:41
inc0from security standpoint14:41
inc0in fact, I wanted to suggest people in this arch guide14:42
inc0to connect external facing network just to network node14:43
inc0s14:43
inc0and hide control plane from the world alltogether14:43
sbezverk_kfox1111: ping14:43
sdakeinc0 use my diagram pls14:45
sdakeinc0 it explains the network architecture clearly14:45
inc0sdake, mind linking it?14:45
sdakeinc0 moment14:45
sdakeso folks, here are oprofile results of one kolla controller node under rally 128x96 test case14:50
sdakehttps://drive.google.com/file/d/0B8q6xDPETSkHY3RFWE51U1J3WlE/view?usp=sharing14:50
sdakeinc0: https://www.gliffy.com/go/publish/1048675514:52
inc0that's incorrect sdake14:53
sdakepbourke just looking at that diagram, it appears the controller nodes will rquire *4* IPS14:53
*** coolsvap_ has joined #openstack-kolla14:53
sdakeinc0 feel free to fix it14:53
sdakeinc0 i want it to be correct - its part off our security  analysis14:53
sdakeit took about 6 hours to make14:54
inc0so my point is, you don't need to connect to external network from controllers at all14:54
sdakeso i'd suggest just finetuningthat one14:54
inc0also we're missing few networks14:54
sdakethe  api network shoudl connect to routed internet for CSPs14:54
sdakemaybe we need anothe rline "externally routed API network link"14:55
sdakeso it runs well in lab enironments as well14:55
sdakei can make you a collaborator on this document14:55
sdaketha tlittle ebox on the api network line is a router with firewlal ;)14:56
sdakenotice the title of the document "Deployment Architecture"14:57
sdakethat is probably what you should call your doc in the queue14:58
sdakeinc0 do you know if osic is 40gig, 20gig, or 10gig?14:58
sdakethe nic in use is 10gig14:58
sdakerather 40gig14:59
sdakebutnot sure about the actual gear behind it14:59
*** kbaegis1 has joined #openstack-kolla15:00
*** kbaegis has quit IRC15:01
*** ajpower has quit IRC15:02
*** senk has joined #openstack-kolla15:02
*** zhurong has quit IRC15:03
*** Serlex has quit IRC15:04
*** kbaegis has joined #openstack-kolla15:04
*** kbaegis1 has quit IRC15:04
cloudnullsdake: the OSIC uses 10GBE devices. The nics are intel x710, the switches are cisco nexus and are setup to allow for LACP if you configure it15:05
sdakecloudnull thnks bro15:05
cloudnullall of the nics are the same using: X710 for 10GbE SFP+ (rev 01)15:06
cloudnullanytime15:06
*** signed8bit has joined #openstack-kolla15:08
sdakecloudnull thanks :)15:09
sdakebrb need to relog networks15:09
inc0sdake, 10gig all the way15:11
inc0sdake, API network should *never* connect to external15:11
*** sdake_ has joined #openstack-kolla15:11
inc0public endpoints sits on HAProxy which sits on network node15:11
sdake_hey folks15:12
sdake_so from my oprofile analysis, and using top, we are only using about 10-30% of the control node's capacity15:12
sdake_under full load15:12
sdake_for folks that wnat to see oprofile summary report: https://drive.google.com/open?id=0B8q6xDPETSkHY3RFWE51U1J3WlE15:14
*** sdake has quit IRC15:15
*** berendt has joined #openstack-kolla15:16
*** rmart04 has quit IRC15:18
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: Document some Conventions  https://review.openstack.org/36118615:18
britthousersdake - I'm not gonna be able to pitch in on OSIC today.  Gotta take my kid to doc appt in 30min.15:19
britthousersdake_ - I'm not gonna be able to pitch in on OSIC today.  Gotta take my kid to doc appt in 30min.15:19
openstackgerritPaul Bourke proposed openstack/kolla: Move rabbitmq precheck from start  https://review.openstack.org/36129815:20
sdake_britthouser I htink we have it under control15:20
sdake_britthouser I do hae a huge favor to ask15:20
*** mewald has quit IRC15:20
britthousersure15:20
sdake_britthouser assuming we get ubuntu finished this weekend, I'm going to need centos7 deployed15:20
sdake_(this weekend)15:20
britthouserI started on that yesterday, lemme see if I can wrap it up.15:21
sdake_i'll also needsoemone to do the actual deploy15:21
sdake_as I honestly dont know how :)15:21
sdake_what i need is 2-4 hours of your time this wekeend to sort this out15:21
sdake_i'll get inc0 lined up on the timing as well so we can all3 bang on it15:22
britthouserThis weekend I'm not gonna be home. =/  Heading to Atlanta15:22
sdake_oh bummer15:22
*** bootsha has joined #openstack-kolla15:22
sdake_ok - well then if you can finish anything you have around centos import15:22
sdake_and provide some instructions somewhere15:22
sdake_on how to get rolling15:22
*** zhubingbing has joined #openstack-kolla15:22
sdake_that would be super appreciated then :)15:22
sdake_the reason i asked for weekend time - we only he cluster until tuesday15:23
*** vbel has quit IRC15:23
*** rcherrueau has quit IRC15:23
*** vbel has joined #openstack-kolla15:23
sdake_so we need to make sure its fullly utilized the next 4 days15:23
britthouserYeah I understand.  If I was gonna be at home I would.15:23
sdake_np15:23
sdake_ifyou could get it staged and ready - that owuld be a huge help too :)15:23
sdake_thats something you may be able to do today?15:23
sdake_after retrieving munchkins from dr?15:24
sdake_inc0 pbourke open up this document: https://drive.google.com/open?id=0B8q6xDPETSkHY3RFWE51U1J3WlE15:26
*** bootsha has quit IRC15:27
pbourkewhat is this showing15:27
*** michauds has quit IRC15:27
sdake_total cpu cycle count on controller node15:27
sdake_divided by cateegory of service15:28
*** ajpower has joined #openstack-kolla15:28
pbourkewow ehkstack taking almost 25%15:29
sdake_load avg was 9-2015:30
sdake_so we have plenty of cpu cycles to burn15:30
sdake_load avg measures running processes sor processes waiting to be schedueld15:30
sdake_but ya, heka piggy15:30
sdake_heka must have implementation problems, it spends alot of time in kernel space15:31
sdake_elasticsearch -  well thats java for you15:32
sdake_pehraps we can do some jre optimziation15:32
*** bootsha has joined #openstack-kolla15:36
*** zhubingbing has quit IRC15:38
openstackgerritKevin Fox proposed openstack/kolla-kubernetes: l3/metadata DaemonSets + DVR, & labels  https://review.openstack.org/35755715:52
*** senk has quit IRC15:52
sdake_sean mooney available anywhere?15:56
*** michauds has joined #openstack-kolla15:58
sdake_what would be nice is an elasticseach log forwarder written in c specifically for kolla's needs16:03
*** matrohon has quit IRC16:03
sdake_we are not running a log forwarder in java16:03
sdake_and the heka implementation seems pokey16:04
sdake_possibly because of the lua parsing16:04
sdake_hard to tell, I couldn't get a  symbol list of heka because it is stripped...16:05
openstackgerritMerged openstack/kolla: Add etcd container  https://review.openstack.org/35515616:10
jrich523ok, so everything got installed on multiple nodes, now horizon is giving me a 503 'no server availble to handle this request'16:13
openstackgerritTin Lam proposed openstack/kolla: Add ansible role for vmtp container  https://review.openstack.org/35830516:15
jrich523which based on coversations from other folks a few days ago, its a known issue?16:15
*** senk has joined #openstack-kolla16:17
inc0back - was pulled out to meeting16:17
sdake_inc0 https://drive.google.com/open?id=0B8q6xDPETSkHY3RFWE51U1J3WlE16:18
sdake_look16:18
inc0what's that?16:18
sdake_click and you will see16:18
*** salv-orlando has joined #openstack-kolla16:18
inc0yeah, I still don't know what these numbers mean16:19
*** Mr_Broken has joined #openstack-kolla16:19
inc0a pie chart, nice colors16:19
sdake_percentages of that components systemwide cpu count16:19
sdake_cpu cycle count16:19
sdake_the components include their kernel access16:19
sdake_with a 128x96 rally run16:20
inc0it waits on IO...16:20
inc0which is kinda expected16:20
sdake_waiting on io count doesn't generate cycle count16:21
sdake_this is pure execution - no waiting included16:21
inc0ok, we do share controllers and network nodes16:21
sdake_this isn't runtime, but clock cycles used by binary16:21
inc0we might want to separate these16:21
inc0and re-bench16:21
sdake_its not a benchmark its a performance profile16:22
sdake_(just to be accurate:)16:22
*** aNupoisc has joined #openstack-kolla16:22
inc0ok, rerun benchmark and get new performance profile16:22
inc0brb16:22
sdake_anyway i thiink we wont have time to do that, but if we do i can do it16:22
sdake_inc0 i am going to need your help this weekend to redeploy the nodees to centos most likely16:22
*** bootsha has quit IRC16:25
sdake_inc0 also Jeffrey4l__ fixed a network issue in the osic setup script16:25
sdake_so now vms can access the interneets16:25
sdake_and wrote a specific io benchmark test for rally16:25
sdake_using fio16:25
sdake_its really a nice peice of work16:25
sdake_hope he submits to rally16:25
*** wirehead_ has quit IRC16:26
*** ssurana has joined #openstack-kolla16:26
*** vhosakot has joined #openstack-kolla16:29
*** wirehead_ has joined #openstack-kolla16:29
*** fragatina has joined #openstack-kolla16:31
*** fragatina has quit IRC16:33
*** fragatina has joined #openstack-kolla16:33
*** Mr_Broken has quit IRC16:34
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Getting NOVA pieces together to launch VM  https://review.openstack.org/35891116:34
*** Mr_Broken has joined #openstack-kolla16:35
*** msimonin has quit IRC16:35
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: Architecture guide doc  https://review.openstack.org/36079616:36
*** david-lyle has joined #openstack-kolla16:36
*** Mr_Broken has quit IRC16:40
jrich523when i do a pip install, where do the tools end up? specifically the admin-openrc.sh16:44
*** unicell has quit IRC16:46
*** unicell has joined #openstack-kolla16:46
*** gfidente has quit IRC16:50
*** gfidente has joined #openstack-kolla16:51
ntpttr___jrich523: You have to run the post-install script to generate that one pretty sure16:52
jrich523you sir are correct16:52
jrich523jsut did that16:53
ntpttr___jrich523: noice16:53
jrich523indeed!16:53
jrich523and im in!16:53
jrich523lets see if both nodes show up16:53
jrich523nice :)16:54
ntpttr___jrich523: happy stacking!16:54
jrich523lol oh i dont think we're there yet :)16:54
jrich523i've made it this far before16:54
jrich523hmm so i noticed in the globals, that heat was an options (just like horizon, which i turned on)16:56
jrich523i left heat off for now, cuz.. why add complexity... but i noticed it built the docker images16:56
jrich523which is cool cuz i want heat, but... shouldnt it have not done that?16:56
sdake_jrich523 all images are built unles you specify the --profile otpion16:57
jrich523oh cool16:57
jrich523hmm built, deployed and configured?16:58
ntpttr___jrich523: also I could be wrong, but it looks like unless you explicitly uncomment the 'enable_heat: no' in globals it's enabled by default16:58
ntpttr___jrich523: https://github.com/openstack/kolla/blob/master/ansible/group_vars/all.yml#L22216:58
jrich523oh cool16:59
jrich523thanks16:59
jrich523yeah, this is all the defaults, good to look at, thanks16:59
*** gfidente has quit IRC16:59
ntpttr___np17:00
*** unicell has quit IRC17:00
*** unicell has joined #openstack-kolla17:02
*** bootsha has joined #openstack-kolla17:02
*** srwilkers_ has joined #openstack-kolla17:12
openstackgerritChristine Hicks proposed openstack/kolla: Changed admin-openrc from DOMAIN_ID to DOMAIN_NAME  https://review.openstack.org/36139017:13
*** pbourke has quit IRC17:14
*** pbourke has joined #openstack-kolla17:14
sdake_pbourke mind reviewing the above change ^^17:16
sdake_dont workflow  a +2 would do ;)17:16
sdake_want to wait for gate on workflow17:16
sdake_inc0 ^^17:18
*** fragatina has quit IRC17:23
*** aswadr_ has quit IRC17:31
*** lyncos has joined #openstack-kolla17:33
sdake_rhallisey ping17:39
rhalliseysdake_, yo17:41
sdake_can you review https://review.openstack.org/36139017:41
sdake_a +2 will do17:41
sdake_and i'll workflow after gate shows success17:41
*** wally__ has joined #openstack-kolla17:42
*** bootsha has quit IRC17:42
wally__hi aNuposic, there17:42
*** lyncos has quit IRC17:42
rhalliseylook fine17:43
wally__I use the global.yaml file you send to me yesterday, and tried to deploy again, but I still not see the bridage was created..:(17:44
*** daneyon has joined #openstack-kolla17:46
*** berendt has quit IRC17:47
*** v1k0d3n has joined #openstack-kolla17:47
*** jheroux has quit IRC17:49
*** daneyon has quit IRC17:50
*** mewald has joined #openstack-kolla17:51
*** rhallisey has quit IRC17:53
*** bootsha has joined #openstack-kolla17:58
*** fragatina has joined #openstack-kolla17:58
*** bootsha has quit IRC17:59
wally__hi , i just check the debug log ,there is no bridge was create, http://pastebin.com/qz13FwGm17:59
wally__can anyone please take a look for me18:00
wally__this is my global.yaml18:01
wally__http://pastebin.com/RLU1zEZh18:01
*** fragatin_ has joined #openstack-kolla18:02
*** sdake_ has quit IRC18:02
*** fragatina has quit IRC18:06
*** senk has quit IRC18:08
*** msimonin has joined #openstack-kolla18:11
*** msimonin has quit IRC18:13
*** matrohon has joined #openstack-kolla18:16
*** bootsha has joined #openstack-kolla18:20
*** ntpttr___ is now known as ntpttr__18:21
*** bootsha has quit IRC18:22
*** ntpttr__ has quit IRC18:25
*** ntpttr___ has joined #openstack-kolla18:25
*** ntpttr___ is now known as ntpttr__18:25
*** dwalsh has quit IRC18:27
*** ntpttr has quit IRC18:27
*** ntpttr has joined #openstack-kolla18:28
*** ntpttr has quit IRC18:28
*** ntpttr__ is now known as ntpttr18:28
*** ntpttr is now known as ntpttr__18:28
*** ntpttr__ is now known as ntpttr18:28
*** ntpttr- has joined #openstack-kolla18:28
*** shardy has quit IRC18:33
*** msimonin has joined #openstack-kolla18:40
wally__hi guys, I am trying to use linux bridge to deploy, but I got some error when i did the kolla-ansible deploy18:42
wally__changed: [localhost]  TASK: [haproxy | Waiting for virtual IP to appear] ****************************  failed: [localhost] => {"elapsed": 301, "failed": true} msg: Timeout when waiting for 10.145.213.62:3306  FATAL: all hosts have already failed -- aborting18:42
openstackgerritMerged openstack/kolla: Changed admin-openrc from DOMAIN_ID to DOMAIN_NAME  https://review.openstack.org/36139018:42
wally__the precheck is fine, is anyone know what happen18:42
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: Customization for base  https://review.openstack.org/35804218:42
inc0wally__, you check if keepalived managed to setup floating ip18:43
inc0go to first controller node and do ip a18:43
inc0see if ip lands on network_interface18:43
wally__hi inc0 can you take alook at this for me http://pastebin.com/87zzLEta18:44
wally__i paste the ip a output18:45
inc0can I see your globals.yml?18:45
wally__ok one sec18:45
wally__http://pastebin.com/Pas5XYPP18:46
wally__this is one, I use linux bridge,18:46
wally__before I use ovsbridg,but i dontknow why it did not create ovs bridge for me18:46
inc0how many controllers do you have?18:47
wally__i am install AIO18:48
wally__just one18:48
inc0please paste output of docker ps -a18:48
wally__ok18:48
*** wirehead_ has quit IRC18:49
wally__http://pastebin.com/0X1SSvET18:49
*** wirehead_ has joined #openstack-kolla18:49
wally__in the middle of deploy18:49
inc0ok, docker logs keepalived18:49
*** msimonin has quit IRC18:50
wally__http://pastebin.com/Q8fdXnGR18:50
wally__somthing wrong18:50
inc0hmm...yeah18:51
wally__how do deploy kolla with linux bridge18:51
inc0just change driver to linuxbridge18:51
*** dwalsh has joined #openstack-kolla18:51
inc0but that's not this issue18:51
inc0keepalived is the same regardless18:52
wally__do u have your global.yaml18:52
wally__can i see18:52
inc0do you have another kolla running in same network?18:53
wally__yeah,18:53
inc0or rather, another keepalived running?18:53
inc0there ya goi18:53
wally__yeah, how do u know18:53
inc0#keepalived_virtual_router_id: "51"18:53
wally__yeah, that is another one18:53
inc0uncomment this line and change a number18:53
inc0to something else18:53
inc0then cleanup and redeploy18:53
*** wally__ has quit IRC18:54
*** wally__ has joined #openstack-kolla18:55
wally__hi inc)18:55
wally__hi inc018:55
inc0you were out for a minute...or less;)18:55
inc0but hello;)18:56
wally__yeahn18:56
wally__so hwat do u need to do18:56
wally__i did not see it18:56
wally__waht do i need to do18:56
inc0<inc0> #keepalived_virtual_router_id: "51"18:56
inc0[13:53]  <inc0> uncomment this line and change a number18:56
wally__this line is #keepalived_virtual_router_id: "51"18:56
wally__alrady18:56
inc0after that cleanup and redeploy18:56
wally__uncoment18:56
wally__okok18:56
inc0change number to anything else18:56
wally__what is this for18:57
wally__keepalived_virtual_router_id: "61"18:57
inc0keepalived communicates with each other using broadcasts18:57
wally__is thati ok?18:57
inc0yeah18:57
wally__ok let me try again18:58
inc0so if old keepalived, from another kolla, will broadcast its shit, new one will get confused18:58
inc0this identifies mutliple keepalived instances18:58
wally__ok, i c ,18:58
inc0so number itself doesn't really mean anything, it just has to be unique18:58
wally__so if i have one than one kolla in the same subnet, I need to uncommend this18:58
inc0and change number, yeah18:59
wally__ok18:59
inc0let me know if it fixes your issue18:59
wally__ok i am doing the cleaning18:59
inc0good luck:)18:59
*** rhallisey has joined #openstack-kolla19:00
wally__another quesiotn, i use ovs-bridge before, but why there is no br-ex create when i do the ifcofig19:00
wally__so confused19:00
wally__so I now use linux bridge19:00
inc0wally__, it's inside container19:02
inc0try docker exec -it openvswitch vsctl show19:02
inc0it's there19:02
wally__oh,,19:02
wally__but after that why my VM has not internet accesss, cannot reach outside,19:03
*** fragatin_ has quit IRC19:03
inc0then we should debug this:)19:03
wally__ok, let me try again,19:04
*** fragatina has joined #openstack-kolla19:04
wally__do u have a globla.yaml file , can you send to me19:04
openstackgerritMerged openstack/kolla: Move rabbitmq precheck from start  https://review.openstack.org/36129819:04
*** jemcevoy has joined #openstack-kolla19:06
*** vhosakot has quit IRC19:07
wally__hi inc019:08
inc0sorry I wasnt looking at irc19:09
inc0I don't have any linuxbridge conf atm19:09
wally__http://pastebin.com/t43YNKzM19:09
*** salv-orlando has quit IRC19:10
wally__I just check the ovs bridge , seems it there. can you plese check19:10
wally__but seems something wrong19:10
inc0wally__, inside docker19:10
wally__yeah, i login the docker19:10
wally__(openvswitch-vswitchd)19:10
inc0a yeah I see that19:10
wally__http://pastebin.com/t43YNKzM19:11
wally__can you cehck this19:11
inc0seems like something broke during deployment19:11
wally__it is down19:11
wally__ovs-vsctl     ovs-vswitchd   (openvswitch-vswitchd)[root@kolla /]# ovs-vsctl show 2016-08-26T19:08:26Z|00001|ovsdb_idl|WARN|Interface table in Open_vSwitch database lacks error column (database needs upgrade?) f554bb5b-4dba-42d1-ae59-3352997afe21     ovs_version: "2.0.2"19:12
wally__somthing wrong?19:12
inc0yeah19:12
inc0can you do docker logs openvswitch_vswitchd19:12
inc0and openvswitch_db?19:12
wally__once sec19:12
wally__http://pastebin.com/AHXgMXi519:13
wally__http://pastebin.com/yppcfXdN19:13
wally__thses all19:13
wally__somehting wrong with db?19:14
inc0can you look at /var/log/openvswitch...?19:14
inc0inside containers19:14
wally__one sec19:14
wally__http://pastebin.com/1L8k6XtV19:15
wally__this is the one19:15
*** bootsha has joined #openstack-kolla19:16
*** fbarilla has joined #openstack-kolla19:17
wally__all the br-ex, br-int are down,,, it created the bridge,,19:17
jemcevoyinc0, I almost have my cluster working... But have a couple of Neutron questions.  Does neutron/openvswitch need to own a NIC on all six server or only the three controller running neutron-network?19:18
inc0hmm logs looks good19:18
*** michauds has quit IRC19:18
*** michauds has joined #openstack-kolla19:18
inc0jemcevoy, nic has to be owned really on compute19:18
inc0and network19:19
wally__:( then waht is the problem, the error19:19
inc0pretty much wherever openvswitch is running19:19
jemcevoyI need to know If I need 3 cables run from eth1 to the switch or six19:19
jemcevoyso it is six cables19:19
inc0yeah19:19
inc0computes needs it19:19
wally__is my globla.yaml ok?19:20
inc0well you've shown me one for linuxbridge, but looks correct19:20
inc0problem maybe somewhere else, hard to say now19:20
jemcevoyCool... And how do I force the uplink to use VLAN 10.  the only way to the outside of my dev rack is on VLAN 1019:20
inc0jemcevoy, there 2 ways19:21
*** bootsha has quit IRC19:21
inc0either you make it native vlan on neutron_external_interface19:21
inc0and use flat network19:21
inc0or you use tagged vlan networking, but for that you need to change neutron config slightly19:21
wally__where?19:21
*** senk has joined #openstack-kolla19:22
wally__neutron_external_interface: "eth1"19:22
wally__this is for my ovs deploy19:22
inc0wally__, so, logs of ovs looks good19:22
wally__eth1 is another interface with no ip assign19:22
*** lamt has quit IRC19:22
inc0can you show me logs of ovs_db too?19:22
*** michauds has quit IRC19:22
wally__ok19:23
*** michauds has joined #openstack-kolla19:23
wally__(openvswitch-db)[root@kolla openvswitch]# vi ovsdb-server.log   2016-08-26T17:36:23.651Z|00001|vlog|INFO|opened log file /var/log/openvswitch/ovsdb-server.log 2016-08-26T17:36:23.657Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.5.0 2016-08-26T17:36:33.660Z|00003|memory|INFO|8388 kB peak resident set size after 10.0 seconds 2016-08-26T17:36:33.660Z|00004|memory|INFO|cells:232 monitors:1 sessions:119:23
wally__only this19:23
wally__seem good19:23
wally__but when i do  ovs-vsctl list-br, in the container ,nothing19:24
kfox1111Update cadvisor to v0.23.9 to fix a problem where attempting to gather container filesystem usage statistics could result in corrupted devicemapper thin pool storage for Docker. (#30307, @sjenning)19:26
*** bootsha has joined #openstack-kolla19:27
kfox1111thats probably the issue that broke my box before... :/19:27
inc0yay19:27
inc0well, you're using devicemapper19:27
inc0you're kinda asking for it19:27
kfox1111heh. I switched to btrfs this last time...19:27
kfox1111not sure which one I trust more though...19:28
inc0ubuntu seems to trust btrfs enough to put it's os on top of it by default19:28
inc0overlay is there by default in xenial too19:28
*** kbyrne has quit IRC19:29
inc0wally__, tbh I don't have idea now19:30
inc0you need to debug, check perms of ovs db19:30
inc0although we had some issues with perms before...just didn't show always19:31
inc0which is strange19:31
wally__.....19:31
wally__so .did u try deploy with ovs before?19:31
*** bootsha has quit IRC19:31
inc0lest time yesterday on 130 nodes...19:31
wally__oh, really, it work?19:32
inc0ofc19:32
inc0anyway, you might have hit some bug19:32
wally__can u see your globla.yaml file19:32
inc0which we haven't seen before19:32
wally__please19:32
Pavoevening everyone and happy friday19:32
*** kbyrne has joined #openstack-kolla19:32
inc0nothing too fancy there, let me see if it's published in our review:)19:33
inc0anyway, could you try to deploy master please?19:33
inc0I might have theory19:33
wally__yeah, I am goinng to do it again19:33
inc0try master+ovs19:33
inc0and let's see19:33
wally__waht do u mena master + ovs19:34
inc0kolla from master19:34
wally__waht do u mean, mastger +ovs19:34
wally__how many nodes,19:34
inc0https://github.com/openstack/kolla19:34
wally__one for kolla19:34
inc0similar setup as one broken19:34
wally__i am using allin one19:35
inc0that's ok, try it19:35
wally__ok, can you please send me your globla file19:36
kfox1111nice... k8s 1.4 alpha 3 has scheduled jobs.19:36
kfox1111we can do fernet token roling with that. :)19:36
inc0wally__, I really don't want to send this config, I have several of them with all sorts of different stuff in it19:37
wally__ok, i c19:37
inc0also we need to figure out what's wrong with yours if anything19:37
inc0I couldn't find anythign wrong wit it19:37
inc0with it19:37
wally__before I do it agin, i just want to make sue my network setup is good19:38
wally__how is your netowrk setup in your master19:38
inc0for aio it doesn't mean that much anyway19:38
inc0same as mitaka, also again, for aio it's not a big issue19:38
*** senk has quit IRC19:38
Pavowally__ did you read the article I posted for you lastnight?19:39
Pavoit was about aio deployments19:39
*** bootsha has joined #openstack-kolla19:39
wally__Yeah, i following the same19:39
Pavook cool19:39
wally__but there iss some probloem  about the bridge19:39
Pavohave you tried with stable branch?19:40
Pavoor master?19:40
wally__http://pastebin.com/t43YNKzM19:40
wally__can you pleas cehfck this19:40
wally__@pavo19:40
Pavolooks right to me19:40
wally__the bridge is down19:40
jemcevoyinc0,  Do all [compute] nodes as well as [network] nodes need to have "neutron_external_interface="enp1s0f1" defined?19:41
Pavoand you should use the openstack clients and sourcing your keystone rc to check ovs cmds19:41
inc0jemcevoy, yeah19:41
Pavobridge status might say down but thats might because of how the bridge works19:41
*** zhiwei has joined #openstack-kolla19:41
inc0jemcevoy, https://review.openstack.org/#/c/360796/4/doc/architecture-guide.rst19:41
wally__(openvswitch-vswitchd)[root@kolla /]# ovs-vsctl show    2016-08-26T19:24:01Z|00001|ovsdb_idl|WARN|Interface table in Open_vSwitch database lacks error column (database needs upgrade?) f554bb5b-4dba-42d1-ae59-3352997afe21     ovs_version: "2.0.2"19:42
inc0take a look at this19:42
wally__and i did not see any bridge19:42
wally__so, u tried it and it working for you right?19:42
Pavowally__ install the openstck-clients and do a kolla-ansible post-deploy on master node19:42
jemcevoyinc0, Thanks that doc looks very helpful...19:43
Pavoand then source keystone rc file and then from master node use openstack ovs blah blah19:43
inc0hehe19:43
wally__I did the post deploy19:43
inc0just wrote it yesterday;) I figured it'll gonna be useful19:43
Pavook source /etc/kolla/rc file19:43
Pavothen do openstack ovs-vsctl list19:44
wally__openstack ovs blah?19:44
wally__root@kolla:/etc/kolla# openstack ovs-vsctl list openstack: 'ovs-vsctl' is not an openstack command. See 'openstack --help'. Did you mean one of these?19:44
wally__??19:44
PavoI don't have my environment up right now so don't know the rc file name or ovs cmds right off bat sorry19:44
Pavook try ovs-vsctl list19:45
Pavowithout openstack19:45
PavoI can't remember19:45
*** zhiwei has quit IRC19:45
wally__root@kolla:/etc/kolla# ovs-vsctl list-br root@kolla:/etc/kolla#19:46
wally__show nothing,19:46
wally__but this is only for the master node, right19:46
PavoI'll throw up a aio VM later tonight and see what I can do19:47
wally__when you insall the kolla, did u use stable branch or the master?19:47
PavoI do stable19:47
Pavounless you plan on doing development and testing I would use stable19:47
wally__how did u insall kolla19:47
wally__pip install kolla?19:47
Pavoyeah19:48
wally__ then I use the same19:48
Pavoand pip install ansible==1.9.419:48
wally__yeah, the same19:48
wally__ok, man, can you plase try aio again?19:48
wally__just take 15 mins to do19:48
Pavoyeah I will once I get done with dinner19:48
inc0wally__, also, which distro you have on host and which one in container?19:49
PavoI use centos for host and centos for containers19:49
wally__ubunt 140419:49
wally__all ubuntu19:49
inc0you did mount --make-shared /run?19:49
PavoI never use ubuntu19:49
inc0right?19:49
wally__yeah19:49
wally__i use mount --make-shared /run?19:49
inc0ok, it would break otherwise19:49
wally__yeah i run mount --make-shared /run19:50
inc0well without ? but yeah you need this in ubu1419:50
wally__yeah i did19:50
*** berendt has joined #openstack-kolla19:50
wally__can you pleaes try with ubuntn19:50
aNupoiscHi all, is it always required to have 2 or more nics on the machine where kolla is deployed? http://docs.openstack.org/developer/kolla/quickstart.html19:50
inc0yesterday - that was all ubuntu19:50
inc0but master19:50
openstackgerritMerged openstack/kolla: Add full support for fernet  https://review.openstack.org/34936619:50
inc0aNupoisc, yes, if you're running neutron that is19:51
rhalliseyinc0, nice!19:51
rhalliseyfernet got in19:51
*** bootsha has quit IRC19:51
rhalliseyinc0, https://review.openstack.org/#/c/351139/2519:51
inc0yeah with one caveat19:51
rhalliseywhat's the caveat19:51
inc0I don't really like keystone_ssh19:51
inc0we have nova_ssh19:51
inc0which kinda became a standard19:52
inc0but I guess we need to figure out this stuff19:52
rhalliseyI don't know what you mean19:52
inc0fernet requires token distribution19:52
inc0it uses ssh here19:52
inc0it should use ansible19:52
inc0but I don't want to stop whole feature on this19:52
aNupoiscinc0: ahh good to know. I am doing all-in-one setup with just ceph19:52
wally__and also, pavo, do u have your init-runonce file19:52
wally__can I see it19:53
*** srwilkers_ has quit IRC19:53
wally__to create the network19:53
aNupoiscinc0: so i will be enabling ceph and ceph_rgw from globals.yml19:53
inc0rhallisey,  can you take a look at it? https://review.openstack.org/#/c/360796/19:53
aNupoiscinc0: so I assume that requirement is not meant for me.19:53
rhalliseyI get your point, but how would that look with anisble19:53
rhalliseyI'm not seeing it19:53
rhalliseyinc0, was going to ask you to rebase on my doc struacture19:53
inc0aNupoisc, no, you should be ok without this setting19:54
rhalliseylet me fix my doc structure though real quick19:54
inc0sure19:54
jemcevoyinc0,  I have a few things that i have stumbled over that may be useful in a doc like yours.  Can I email then to you next week?  IRC seems too ephemeral19:54
aNupoiscinc0: cool.19:54
inc0jemcevoy, either that or write a patch with it19:54
inc0I'd prefere latter ;) you should take credit for this stuff too19:55
inc0but if you don't want to submit it for whatever reason, sure mail would be ok19:55
jemcevoyThat would mean that I would need to signup with OpenStack?19:55
inc0yeah, it only cost a soul19:55
inc0but jokes apart, it's really cool when you see your patch merged19:56
inc0I really like this feeling;)19:56
jemcevoyI'll ask my buddy from Marantis if he can help me through that hurdle19:57
*** dwalsh has quit IRC19:57
inc0we can help you if you need help19:57
inc0there is guide for it actually, hold on19:57
inc0http://docs.openstack.org/infra/manual/developers.html19:58
jemcevoyFirst question... Should I use my work or personal email?19:58
inc0depends, is your company contributing to openstack?19:58
jemcevoyThey may have done a little... Penguin Computing not sure19:59
inc0well, let me put it this way, if it's just you, I'd go with personal mail and individual contributor agreement20:00
inc0there is paragraph about this in guide I've linked20:00
jemcevoyI will get stared on that over the weekend...20:01
inc0thanks!20:01
inc0it's not as bad as it seems, and you'll be listed as one of authors of kolla20:01
inc0also contributors get free access to summit20:01
inc0full pass20:02
jemcevoyCool..20:02
*** NachoDuck has quit IRC20:03
wally__hi, pavo, can I give you a server, and you can try on it?20:03
wally__i will give you a public ip to loing20:03
wally__login20:03
PavoI have never used ubuntu before so not sure if that would be good idea20:03
inc0wally__, I can do it if you want20:04
wally__ok20:04
wally__good, I will set up a ubuntnu 1404 server and give you public ip later20:04
*** lamt has joined #openstack-kolla20:04
wally__give me 2 hours20:04
*** serverascode has quit IRC20:04
*** NobodyCam has quit IRC20:04
wally__ubuntu 1404 server, right?20:04
inc0ahh 2 hrs...Friday evening:(20:04
wally__ok, ik c20:04
wally__next monday20:05
inc0I'll play over weekend if that's ok20:05
inc0or Monday20:05
wally__is it ok for your?20:05
inc0sure, thanks20:05
*** bootsha has joined #openstack-kolla20:05
wally__i will set it up fist , then email to you? can u tell me your email?20:05
wally__i will send you all the detail include network subnet range, then you can assign for the floating ip20:08
openstackgerritMerged openstack/kolla: Add dockerfiles for keystone fernet  https://review.openstack.org/35113920:09
*** elemoine__ has joined #openstack-kolla20:17
openstackgerritMerged openstack/kolla: Fix bug in Elasticsearch role  https://review.openstack.org/35959620:18
*** elemoine_ has quit IRC20:20
*** elemoine__ is now known as elemoine_20:20
*** mewald has quit IRC20:21
jemcevoyinc0,  One more question... Can neutron/openvswitch share a NIC with haproxy, horizon, etc?  Or will I need two NICs on VLAN 10?20:26
*** lyncos has joined #openstack-kolla20:26
inc0you want 2 nics20:26
inc0neutron will bind bridge to the port20:26
inc0which will render port unreachable20:26
inc0there are ways to make them share network, but not supported here20:26
inc0however20:26
inc0what you can do20:27
inc0is to make 2 nivs20:27
inc0nics20:27
jemcevoyOK... I am going to need another 1Gig switch20:27
inc0one for vlan 10 another in whatever you want20:27
inc0nah20:27
inc0can you trunk your netowork therE?20:27
wally__hi inc0, can you gvie me you email, then I send you the server info later20:28
inc0wally__, sent over in pm20:28
inc0elemoine_, hey Eric, long time no see:)20:28
wally__in pm?20:28
inc0private message, you should have it in irc20:28
inc0somewhere20:28
*** lyncos has quit IRC20:28
wally__how to send in pm?20:29
jemcevoyI will need eth0.10 for haproxy horizon, etc and eth1 native VLAN10 for neutron and the rest of the traffic on the 10Gig internal network20:29
inc0hmm, sooo20:31
inc0let me think20:31
inc0you want flat network to be in same net as haproxy20:31
inc0ok, so what we can do20:31
inc0is change your inventory in a way to put haproxy and keepalived on control nodes20:32
inc0they don't need connection to public network at all20:32
inc0so you can just create port.10 there and bind haproxy to irt20:32
*** NachoDuck has joined #openstack-kolla20:33
inc0either that or you just create virtual interface in network node and manually add it to br-ex20:34
inc0and use it for haproxy20:34
*** Jeffrey4l_ has joined #openstack-kolla20:34
*** Jeffrey4l__ has quit IRC20:35
*** bootsha has quit IRC20:36
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla: Architecture guide doc  https://review.openstack.org/36079620:38
inc0jemcevoy, ^ added note about kolla external interface20:38
inc0you might want to laverage that20:38
*** aNupoisc has quit IRC20:39
*** fguillot has quit IRC20:39
*** serverascode has joined #openstack-kolla20:42
*** aNupoisc has joined #openstack-kolla20:55
*** ayoung has joined #openstack-kolla20:59
*** sdake has joined #openstack-kolla21:02
openstackgerritRyan Hallisey proposed openstack/kolla: Add docs for host setup  https://review.openstack.org/34322321:03
openstackgerritRyan Hallisey proposed openstack/kolla: Shrink the quickstart massively and break it into 2  https://review.openstack.org/34322421:03
*** salv-orlando has joined #openstack-kolla21:03
*** inc0 has quit IRC21:06
*** NobodyCam has joined #openstack-kolla21:08
*** slagle has quit IRC21:12
*** matrohon has quit IRC21:12
*** salv-orl_ has joined #openstack-kolla21:18
openstackgerritRyan Hallisey proposed openstack/kolla: Add docs for host setup  https://review.openstack.org/34322321:20
openstackgerritRyan Hallisey proposed openstack/kolla: Shrink the quickstart massively and break it into 2  https://review.openstack.org/34322421:20
*** salv-orlando has quit IRC21:20
rhalliseydown to 277 for devs21:20
rhalliseywould like under 150..21:20
*** daneyon has joined #openstack-kolla21:23
*** daneyon has quit IRC21:27
*** jtriley has quit IRC21:33
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes: Getting NOVA pieces together to launch VM  https://review.openstack.org/35891121:33
sdakerhallisey 27 comments in the quickstart guide change21:38
sdakelook at comment #721:38
sdakerhallisey like under r15 0what rhallisey ?21:38
rhallisey150 lines21:39
*** rhallisey has quit IRC21:48
*** lamt has quit IRC21:54
sdake150 line of what?21:55
*** kbaegis has quit IRC21:57
*** kbaegis has joined #openstack-kolla21:57
*** JoseMello has quit IRC22:00
*** sdake_ has joined #openstack-kolla22:01
haplo37Hi guys, i am not able to take a snapshot of my instance. I have this issue : http://paste.openstack.org/show/564088/ . Do you have an idea what happen ?22:03
*** sdake has quit IRC22:03
*** lamt has joined #openstack-kolla22:05
*** lamt has quit IRC22:05
*** fbarilla has quit IRC22:06
*** dmsimard is now known as dmsimard|pto22:12
*** fguillot has joined #openstack-kolla22:18
*** unicell has quit IRC22:22
*** unicell has joined #openstack-kolla22:23
*** michauds has quit IRC22:25
*** berendt has quit IRC22:30
*** berendt has joined #openstack-kolla22:36
*** berendt has quit IRC22:36
*** kbaegis has quit IRC22:37
*** kbaegis has joined #openstack-kolla22:45
*** signed8bit is now known as signed8bit_Zzz22:54
*** signed8bit_Zzz is now known as signed8bit22:56
*** salv-orl_ has quit IRC23:00
*** serverascode has quit IRC23:01
*** NobodyCam has quit IRC23:01
*** NachoDuck has quit IRC23:02
*** daneyon has joined #openstack-kolla23:11
*** NobodyCam has joined #openstack-kolla23:15
*** daneyon has quit IRC23:15
*** vhosakot has joined #openstack-kolla23:16
*** vhosakot has quit IRC23:19
*** serverascode has joined #openstack-kolla23:20
jemcevoyinc0,  I got those eth1 ports plugged in to a switch... How do I restart to get neutron running properly...23:23
*** unicell1 has joined #openstack-kolla23:25
*** unicell has quit IRC23:26
*** NachoDuck has joined #openstack-kolla23:29
*** fguillot is now known as fguillot_afk23:29
*** fguillot_afk has quit IRC23:30
*** unicell1 has quit IRC23:40
*** unicell has joined #openstack-kolla23:42
*** fguillot has joined #openstack-kolla23:43
sdake_jemcevoy best hting to do is probably redeploy but i don't know what state your in23:46
*** NachoDuck has quit IRC23:46
*** NobodyCam has quit IRC23:46
*** serverascode has quit IRC23:46
sdake_what version are you running?23:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!