Friday, 2017-03-10

kfox1111inc0: there are three subnets involved. the one from the underlying system,00:00
kfox1111the one for svc's.00:00
kfox1111and the one for pods.00:00
inc0kubeadm init --token {{ init_token }} --use-kubernetes-version {{ kube_version }} --api-advertise-addresses {{ kube_api_endpoint.stdout }} --pod-network-cidr={{ kube_cluster_ip_cidr }}00:01
inc0which is which?00:01
kbaegis:(00:01
inc0I have interface with 192.168.100.0/2400:01
inc0already on host00:01
inc0kbaegis: /var/lib/docker/volumes/kolla_log/_data00:02
sbezverkinc0: kubeadm init --pod-network-cidr 10.57.0.0/19  --service-cidr 10.57.32.0/19 --service-dns-domain openstack.kolla.00:02
inc0check what's there00:02
sbezverkyou can tell from this output which is which I think ;)00:02
kbaegisinc0: There's no file for nova-compute00:02
inc0docker ps -a plz00:03
kbaegishuh. iscsid is stopped00:03
kbaegishttps://hastebin.com/gahoreyipa.nginx00:03
sbezverkgot to run, bbiab00:03
*** yingjun has joined #openstack-kolla00:07
*** lamt has quit IRC00:19
*** gfhellma_ has quit IRC00:22
*** mattmceuen has quit IRC00:23
inc0ok, I'll go back to it tomorrow00:28
inc0need to learn more about networking in k8s really00:28
inc0cya guys00:29
*** salv-orlando has quit IRC00:30
*** gema has quit IRC00:32
*** MarginHu has joined #openstack-kolla00:34
*** gema has joined #openstack-kolla00:34
*** gema has joined #openstack-kolla00:34
*** duonghq has joined #openstack-kolla00:36
*** tovin07_ has joined #openstack-kolla00:38
*** adrian_otto has joined #openstack-kolla00:46
kfox1111anyone play with fluentd-bit at all?00:53
*** saneax is now known as saneax-_-|AFK00:55
*** zhurong has joined #openstack-kolla00:56
*** tovin07 has joined #openstack-kolla00:58
kbaegiskfox1111: Not I unfortunately01:01
kfox1111this looks interesting: http://fluentbit.org/kubernetes/01:02
*** yuanying_ has quit IRC01:05
*** jtriley has joined #openstack-kolla01:05
*** cuongnv has joined #openstack-kolla01:05
*** daidv has joined #openstack-kolla01:06
jascott1hmmm is that an alternative or has something in particular?01:06
*** jrobinson has joined #openstack-kolla01:08
kfox1111looking at logshippers again. and bumped into that.01:14
kbaegisSo I'm not getting libvirt up01:14
kfox1111looks like fluentd and logstash are fairly comparable,01:14
kfox1111and fluent-bit is comparable to filebeat.01:14
*** Mannheim has quit IRC01:15
*** zhubingbing_ has joined #openstack-kolla01:15
kbaegis:/ I'll rebuild nova-libvirt01:15
kfox1111though fluent-bit is c and filebeat is go.01:15
*** manheim has quit IRC01:17
*** manheim has joined #openstack-kolla01:18
*** mattmceuen has joined #openstack-kolla01:19
kfox1111though: http://fluentbit.io/documentation/0.11/about/fluentd_and_fluentbit.html01:19
*** sayantan_ has joined #openstack-kolla01:19
*** yuanying has joined #openstack-kolla01:19
kfox1111I would assume go memory consumption could not be made that small.01:19
duonghqmorning guys01:24
kfox1111morning01:25
*** MasterOfBugs has quit IRC01:31
*** adrian_otto has quit IRC01:44
*** zhubingbing__ has joined #openstack-kolla01:46
*** zhubingbing_ has quit IRC01:47
*** deadnull has quit IRC01:48
kbaegishuh02:05
kbaegismorning guys02:05
kbaegisHey duonghq, have you ever had nova-libvirt break on you?02:05
kbaegishttps://hastebin.com/loliqatike.sql02:05
*** zhubingbing_ has joined #openstack-kolla02:05
kbaegisOh, it's the apparmor02:06
kbaegisMaybe?02:06
*** zhubingbing__ has quit IRC02:06
*** manheim has quit IRC02:07
openstackgerritMerged openstack/kolla-ansible master: Fix typo in nova external ceph  https://review.openstack.org/44375302:09
*** eaguilar has joined #openstack-kolla02:10
*** adrian_otto has joined #openstack-kolla02:19
*** yuanying has quit IRC02:20
*** yuanying has joined #openstack-kolla02:21
kbaegisapparmor fixed the issue with nova-libvirt crashing02:21
kbaegisnova-compute is still crashing02:21
*** unicell has quit IRC02:24
sdakekfox1111 i think in a perfect world our log forwarding used in kolla-kubernetes would reuse the techniques and technology and support models used in kolla-ansible02:28
sdakefluentd was chosen specifically because it is supported by an upstream02:28
sdake(that upstream backed by the linux foundation and CNCF)02:29
sdakein my opinion that was a major factor in the choosing of fluentd as a log forwarder02:29
*** adrian_otto has quit IRC02:29
sdakeFluentd’s performance has been put to the test at many large services; in fact, a regular PC box can handle 18,000 messages/second with a single process.02:31
sdakefrom fluentd's website02:31
sdakeindependent of language choice, that is their claim02:31
sdakewhich should be more then enough for all openstack services on a node02:31
sdakezhubingbing_ should know the details02:32
zhubingbing_yeah02:32
sdakezhubingbing_ how goes fluentd integration with kolla-kubernetes?02:32
sdakezhubingbing_ also did i read correctly you ran through the etherpad QSG?02:32
zhubingbing_We mainly have to consider the future replacement costs02:33
sdakeright - future replacement cost is very very high02:33
sdakethat is the main consideraiton02:33
zhubingbing_i will do fluentd integration with kolla-kubernetes02:33
sdakewe have replaced logstash with heka now with fluentd02:33
zhubingbing_yes02:33
sdakemainly to stay ahead of the fact that the upstreams are not reliable :(02:33
sdakeor don't have reliable lifetimes02:33
zhubingbing_so we can not obviously rely on a component02:33
zhubingbing_agree02:34
sdakezhubingbing_ i think kfox1111 was proposing earlier to use fluentd-bit02:34
sdakewhich performs better and has a smaller footprint (as it is written in C)02:34
zhubingbing_Fluentd we mainly used to make a forwarding effect, not obviously dependent on, if the back to replace the cost is very low02:34
sdakehowever trades that off for lack of a backed communiity02:34
*** sayantan_ has quit IRC02:35
sdakebrb walking downstairs to get some food and hit the bar :)02:35
zhubingbing_;)02:35
*** daidv has quit IRC02:39
*** sudipto has joined #openstack-kolla02:41
sdakezhangqiankun back ;)02:41
sdakezhubingbing_ back that is :)02:41
zhubingbing_;)02:42
zhubingbing_kfox1111 why use flunent-io02:43
zhubingbing_fluentdbit02:43
*** yuanying has quit IRC02:48
sdakezhubingbing_ i think its midnight where kfox1111 may be -as its 10pm on the east coast atm02:49
zhubingbing_;)02:50
zhubingbing_We support elastaicsearch in Kolla-kubernetes?02:50
*** m1dev has quit IRC02:51
*** yingjun has quit IRC02:54
sdakeHUNGREE02:55
sdaketwo waitresses in this whole slammed bad02:56
sdakebar02:56
sdakeand a bunch of racket02:56
sdakecaptive audience ftl02:56
zhubingbing_;)02:56
zhubingbing_Where are you now ?02:56
sdakeat the aloft in boston02:57
zhubingbing_wow02:58
sdakethis place is rockin atm02:58
zhubingbing_It sounds wonderful02:58
sdakenot really02:58
sdakeWTB FOOD02:58
spsuryasdake: if this time I got the VISA approved then I will get a chance to meet you.02:59
spsuryain Boston02:59
spsurya:)02:59
sdakespsurya cool :)02:59
sdakeyou assume i dont drop dead between now and May ;_)02:59
spsuryayeh03:00
spsurya:P03:00
*** sudipto has quit IRC03:03
*** goldyfruit has joined #openstack-kolla03:10
*** adrian_otto has joined #openstack-kolla03:10
*** lucasxu has joined #openstack-kolla03:11
*** kbaegis has quit IRC03:11
*** kbaegis1 has joined #openstack-kolla03:11
*** kbaegis has joined #openstack-kolla03:22
kbaegissdake: Hey, so are some projects just broken under kolla?03:22
sdakekbaegis not that i am aware of?03:22
kbaegisBy my count I've had to disable mistral, murano, vpnaas and mongo03:22
sdakekbaegis oh you mean projects in kolla-ansible deliverable03:22
sdakekbaegis i really don't know - i haven't personally tested those services03:22
sdakekbaegis file bugs if you think they are broken -the final release has been cut03:23
sdakekb atleast for ocata03:23
kbaegissdake: cut? So there won't be a 4.x?03:23
sdakeyes 4.0.0 = ocata03:23
sdake3.0.0 = newton03:23
sdakeinstead of cut i should have used the word tagged03:23
kbaegisAhh03:23
kbaegisYeah, I'm trying to get up on an AIO w/ ubuntu xenial03:24
kbaegisI enable all the services I really want to play with, and then I have to stand back the broken ones03:25
*** daidv has joined #openstack-kolla03:25
*** yingjun has joined #openstack-kolla03:26
*** lucasxu has quit IRC03:31
kbaegissdake: I keep getting " (container_state.Running == false): 'dict object' has no attribute 'Running'"}" with these services I'm talking about03:32
kbaegiskolla-ansible03:32
openstackgerritzhubingbing proposed openstack/kolla-kubernetes master: test patch for kolla-K8s  https://review.openstack.org/44407803:33
*** lucasxu has joined #openstack-kolla03:40
*** adrian_otto has quit IRC03:40
*** adrian_otto has joined #openstack-kolla03:42
*** adrian_otto has quit IRC03:43
sdakekbaegis no idea - bugs ftw :)03:51
sdakekbaegis at 2% power and sitting in a bar with a bunch of people - gott ajet ;-)03:51
*** ssurana has quit IRC03:52
masberhi, I have this issue while running "kolla-ansible -i all-in-one bootstrap-servers" http://pastebin.com/raw/GyhpnRJw03:52
masberit says can't restart docker03:52
masber?03:53
masberim using newtown (kolla 3.0.2)03:53
kbaegissdake: Thanks, ttyl sdake03:54
kbaegismasber: run docker info?03:54
masberhttp://paste.openstack.org/raw/602150/03:55
masberI changed the all-in-one inventory pointing from localhost to another server, is that the right way to do this? I wanted to have kolla deployed on a separate VM03:57
masbersdake, ^^^03:59
masberkbaegis, ^^^04:00
kbaegismasber: I think that should work. You have to be sure you have a shared insecure/secure docker registry04:03
masberExecStart=04:03
masberExecStart=/usr/bin/docker daemon \04:03
masber -H fd://04:03
masberkbaegis, ^^^ this makes my docker to stop working04:03
masbernot sure why04:03
masberI am using a fresh server with centos 704:04
masberand the way I installed docker is curl -sSL https://get.docker.io | bash and then yum downgrade docker-engine-1.12.6 docker-engine-selinux-1.12.6 -y04:04
masberso it is pretty standard installation04:04
*** salv-orlando has joined #openstack-kolla04:05
masbersomehow kolla-ansible -i all-in-one bootstrap-servers add those lines to my docker-service/kolla.conf file breaking docker running on the host04:05
masberkbaegis, how can I edit the playbook so I can keep going?04:07
masberI can't find the playbook file04:08
*** salv-orlando has quit IRC04:09
*** saneax-_-|AFK is now known as saneax04:09
*** jrobinson has quit IRC04:09
*** yuanying has joined #openstack-kolla04:10
*** adrian_otto has joined #openstack-kolla04:12
*** eaguilar has quit IRC04:13
kbaegismasber: I don't know enough unfortunately04:14
masberoh ok04:14
masberthank you for your help anyways04:14
kbaegisDocker is a pita04:14
*** sudipto has joined #openstack-kolla04:14
masberagree but it is better for deployment than bare-metal04:15
*** david-lyle has quit IRC04:16
kbaegismasber: Docker is now CE and enterprise too, remmeber04:16
kbaegisThey forked it themselves04:16
*** lucasxu has quit IRC04:24
*** ssurana has joined #openstack-kolla04:26
*** jrobinson has joined #openstack-kolla04:33
*** mdnadeem has joined #openstack-kolla04:36
*** adrian_otto1 has joined #openstack-kolla04:37
*** adrian_otto has quit IRC04:40
*** goldyfruit has quit IRC04:42
*** schwicht has quit IRC04:46
*** jrobinson has quit IRC04:52
*** daidv has quit IRC04:52
*** adrian_otto1 has quit IRC04:57
masberok05:03
masberI got it working05:03
*** salv-orlando has joined #openstack-kolla05:05
kbaegisNice05:06
kbaegisHey, anyone here set up a provider network on kolla?05:07
kbaegisWhat's the practice?05:07
kbaegis Do I use /etc/kolla/config?05:07
*** skramaja has joined #openstack-kolla05:09
*** salv-orlando has quit IRC05:10
*** daidv has joined #openstack-kolla05:29
kbaegisMy actual problem is that the network topology screen is showing an OOPS05:32
*** yingjun has quit IRC05:33
*** daidv has quit IRC05:36
*** jrobinson has joined #openstack-kolla05:44
kbaegishuh05:49
kbaegisRecoverable error: Project ID <ADMIN> is not a valid project05:49
kbaegisso if the admin project isn't valid on load, wth is?05:49
*** saneax is now known as saneax-_-|AFK05:51
masberhi again05:51
masberso kolla_internal_vip_address canot be pingable by any other host in the cluster, is that right?05:52
*** jtriley has quit IRC05:53
*** saneax-_-|AFK is now known as saneax05:59
kbaegismasber: I don't know. I only ever do all-in-one06:01
kbaegisI only have one server myself06:02
*** daidv has joined #openstack-kolla06:02
masberkbaegis, 1 server for both control and computing?06:02
kbaegismasber: I mean yeah, it's got a 16 core proc in it06:05
masberok06:06
masbernice06:06
*** salv-orlando has joined #openstack-kolla06:06
*** unicell has joined #openstack-kolla06:08
SamYaplemasber: kolla_internal_vip_address should be an _unused_ ip in the cluster06:11
SamYapleunpingable, yes. but specifically unused06:11
*** salv-orlando has quit IRC06:11
SamYapleit can be unpingable and still used06:11
masberSamYaple, prechecks fails if that ip is pingable06:11
SamYaplemasber: if its pingable, its in use06:12
SamYaplejust becuase it _isnt_ pingable, doesnt mean it isn't in use06:12
SamYaplethe prechecks can only do a basic check06:12
*** unicell has quit IRC06:13
*** unicell1 has joined #openstack-kolla06:13
masberok, so is kolla going to setup that ip on a nic?06:15
masberas a second IP?06:15
SamYaplemasber: if you use keepalived and haproxy, yes06:16
masberok, ok I understand now06:16
masberkeepalived is for multinode setup is that right?06:17
*** saneax is now known as saneax-_-|AFK06:27
*** jtriley has joined #openstack-kolla06:30
*** caowei has joined #openstack-kolla06:33
openstackgerritMerged openstack/kolla stable/newton: Move Kuryr to start after neutron has started  https://review.openstack.org/43351606:34
*** Jezekus has joined #openstack-kolla06:36
*** jtriley has quit IRC06:40
*** satyar has joined #openstack-kolla06:44
zhubingbing_https://review.openstack.org/#/c/443660/06:47
zhubingbing_inc0 can u look it ?06:47
*** bmace has quit IRC06:53
*** bmace has joined #openstack-kolla06:53
*** manheim has joined #openstack-kolla06:56
*** jbadiapa_ has joined #openstack-kolla06:56
*** yingjun has joined #openstack-kolla06:58
*** rabel has joined #openstack-kolla06:58
*** manheim has quit IRC07:00
*** salv-orlando has joined #openstack-kolla07:07
*** salv-orlando has quit IRC07:12
*** salv-orlando has joined #openstack-kolla07:12
duonghqJeffrey4l_, is there anything block this ps: https://review.openstack.org/#/c/425766/07:19
Jeffrey4l_duonghq, LGTM. but i am not good at docs, so wanna another core to review and +w ;)07:20
*** saneax-_-|AFK is now known as saneax07:20
duonghqJeffrey4l_, roger07:22
*** rabel has quit IRC07:22
openstackgerritJavier Castillo proposed openstack/kolla master: Sahara needs to run ssh-keygen  https://review.openstack.org/43686907:22
*** salv-orl_ has joined #openstack-kolla07:23
*** salv-orlando has quit IRC07:26
*** jtriley has joined #openstack-kolla07:29
*** jtriley has quit IRC07:34
*** jtriley has joined #openstack-kolla07:34
*** jtriley has quit IRC07:39
*** Serlex has joined #openstack-kolla07:43
openstackgerritJavier Castillo proposed openstack/kolla-ansible master: Fix permissions for ceilometer log files  https://review.openstack.org/44413107:45
*** rabel has joined #openstack-kolla07:52
*** shardy has joined #openstack-kolla07:58
*** sudipto has quit IRC08:00
*** matrohon has joined #openstack-kolla08:00
*** jrobinson has quit IRC08:00
*** ssurana has quit IRC08:01
*** l4yerffeJ has quit IRC08:04
*** l4yerffeJ_ has joined #openstack-kolla08:04
*** magicboiz has joined #openstack-kolla08:09
*** dmsimard has quit IRC08:12
*** Mannheim has joined #openstack-kolla08:17
*** salv-orl_ has quit IRC08:20
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set neutron rpc_workers using openstack_service_workers  https://review.openstack.org/44415108:23
*** g3ek has quit IRC08:27
*** haplo37 has quit IRC08:27
*** jtriley has joined #openstack-kolla08:28
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set searchlight-api workers using openstack_service_workers  https://review.openstack.org/44415408:32
*** jtriley has quit IRC08:33
*** yingjun has quit IRC08:33
*** jtriley has joined #openstack-kolla08:33
*** gfidente has joined #openstack-kolla08:34
*** gfidente has quit IRC08:34
*** gfidente has joined #openstack-kolla08:34
*** dmsimard has joined #openstack-kolla08:35
*** nathharp has joined #openstack-kolla08:36
*** haplo37 has joined #openstack-kolla08:36
*** salv-orlando has joined #openstack-kolla08:36
*** g3ek has joined #openstack-kolla08:37
*** jtriley has quit IRC08:38
*** egonzalez has joined #openstack-kolla08:48
*** zioproto has joined #openstack-kolla08:52
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set senlin-api workers using openstack_service_workers  https://review.openstack.org/44416108:52
*** yuanying has quit IRC08:58
JezekusHello everybody, I have followed kolla-ansible quickstart and everything went quite well, bud there are problems with css/pictures in horizon. It is a known issue?08:58
egonzalezJezekus, what kolla version? afaik there is not issue atm09:00
Jezekusegonzalez: kolla-ansible (4.0.0.0rc2)09:03
egonzalezubuntu or centos/oracle?09:03
Jezekusubuntu09:04
Jezekusubuntu base,ubuntu binary in docker09:04
egonzalezhave any other service enabled outside defaults(ironic,sahara, etc)09:05
Jezekusno it's basic clean deployment done this morning09:06
Jezekusonly edited things are NICs, IP, ubuntu a binary install and openstack version tag set to master09:06
egonzalezis there any error at horizon logs?09:08
Jezekusso far only missing favicon.ico information09:11
Jezekuscould you please point me where the images and css should be located ?09:15
Jezekusit looks that horizon is working ok but missing some files09:16
*** mgoddard has joined #openstack-kolla09:18
*** manheim has joined #openstack-kolla09:18
*** manheim has quit IRC09:18
egonzalezJezekus, dunno where are located, i'm deploying to test09:21
des_consoladois caoyuan here under a differnet name?09:21
Jezekusegonzalez: thanks! overall status is pretty fine, deployed demo and it's up and running09:22
openstackgerritThomas O'Neill proposed openstack/kolla master: Add neutron-bgp-dragent service.  https://review.openstack.org/44373209:22
*** manheim has joined #openstack-kolla09:24
*** pbourke has joined #openstack-kolla09:26
*** jtriley has joined #openstack-kolla09:28
openstackgerritThomas O'Neill proposed openstack/kolla-ansible master: Add neutron-bgp-dragent playbooks and sensible defaults.  https://review.openstack.org/44374709:28
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix vpnaas custom config dir  https://review.openstack.org/44417809:30
*** nathharp has quit IRC09:32
*** jtriley has quit IRC09:33
*** haplo37 has quit IRC09:33
*** jtriley has joined #openstack-kolla09:33
sdakemorning09:35
des_consoladohi09:36
duonghqmorning sdake09:36
sdakesup duonghq09:36
sdakeduonghq wtb reviews in kolla-kubernetes ;-)09:36
duonghqsdake, what is wtb?09:37
sdakewant to buy09:37
sdakeits an oldschool video gaming term09:37
sdakein chat when you needed to trade stuff around you would say "WTB"09:38
sdake"WTB XYZ food"09:38
sdakeetc09:38
*** jtriley has quit IRC09:38
duonghqya, I'm revising k8s and helm so I can understand kolla-kubernetes better, it's way different than ansible one, so I realized that I need to renew my knowledge09:39
*** hrw has quit IRC09:40
sdakeyes it is different09:40
duonghqinteresting stuff09:41
*** haplo37 has joined #openstack-kolla09:41
*** yuanying has joined #openstack-kolla09:41
*** hrw has joined #openstack-kolla09:45
duonghqsdake, you have only one talk at Boston?09:46
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set mistral-server workers using openstack_service_workers  https://review.openstack.org/44418009:46
*** sudipto has joined #openstack-kolla09:47
egonzalezJezekus, yep, horizon not load images09:47
Jezekusegonzalez: glad not screwed on my side :-D09:48
*** MarginHu has quit IRC09:49
*** MarginHu has joined #openstack-kolla09:49
des_consoladooh the vpnaas config.yml block I copied had a bug, I see, haha10:00
egonzalezdes_consolado, yep hehe10:02
*** cuongnv has quit IRC10:02
des_consoladoegonzalez: I'm not sure it's sensible to define a default bgp router ID for people, seems odd?10:03
egonzalezdes_consolado, at least define a default id, user will have to change if the need10:03
egonzalezis the same as vrrp id, by default is 51 but you can use other id10:04
des_consoladoah yeah I guess10:04
egonzalezif the var is not set, user don't know that need to use their own10:04
des_consoladomaybe I'll use 1.1.1.1 then10:05
egonzalezthat sounds like a gbp docs ;)10:05
*** MarginHu has quit IRC10:05
des_consoladogbp docs?10:06
*** MarginHu has joined #openstack-kolla10:06
*** MarginHu has quit IRC10:09
*** zhurong has quit IRC10:10
openstackgerritThomas O'Neill proposed openstack/kolla-ansible master: Add neutron-bgp-dragent playbooks and sensible defaults.  https://review.openstack.org/44374710:14
*** egonzalez has quit IRC10:15
*** Serlex has quit IRC10:18
*** gfidente has quit IRC10:20
sdakeduonghq only one? :)10:21
sdakeduonghq i didn't think I had any10:21
openstackgerritThomas O'Neill proposed openstack/kolla master: Add neutron-bgp-dragent service.  https://review.openstack.org/44373210:21
duonghqsdake, yeah, Project Update - Kolla10:23
sdakeduonghq ya i got the email10:23
sdakeduonghq although I had not submitted myself or that talk10:24
sdakeor/for10:24
sdakenow I guess i'm roped into giving it :)10:24
sdakewill probablybe some other last minute stuff i get jammed into talking about10:24
duonghqyep10:25
*** qiliang27 has quit IRC10:25
*** sambetts|afk is now known as sambetts10:25
sdakemorning sambetts10:26
des_consoladoonce you've contributed to an openstack project does that mean you get to go to these conferences for free or on a discount or something?? :D10:26
sdakedes_consolado no10:27
*** jtriley has joined #openstack-kolla10:27
*** tovin07_ has quit IRC10:27
sdakedes_consolado to attend the summit - the cost is $300 (IIRC) for early bird and 600 for late registration10:27
*** tovin07 has quit IRC10:27
sdakedes_consolado then you have to pay for yout travel and whatnot10:27
*** deadnull has joined #openstack-kolla10:27
sdakedes_consolado if you pay the $100 (it was at the last PTG) and slug your way there, you get a free pass to summit10:28
*** caowei has quit IRC10:28
des_consoladowait, the $100 for what? you said $300 or $60010:28
des_consoladosounds like a case of lobbying my employer to send me to something in any case :p10:29
sdake100 for the ptg = free pass to summit10:29
sdakeyou missed the ptg10:29
des_consoladooh right10:29
sdakethat was in february10:29
sdakeso you have to pay (iirc 600 bucks for a ticket)10:29
sdakethe summit/forum is essential for devs to attend10:29
*** manheim has quit IRC10:30
sdakeit may be going  to 1200 soon - so I'd get on it :)10:30
*** egonzalez has joined #openstack-kolla10:31
*** satyar has quit IRC10:31
des_consoladoi see, thanks for the info!10:31
des_consoladostill early days for me, I'll harass my manager tho :)10:32
*** magicboiz has quit IRC10:32
*** openstackgerrit has quit IRC10:33
*** openstackgerrit has joined #openstack-kolla10:33
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set magnum-api workers using openstack_service_workers  https://review.openstack.org/44420210:33
*** nathharp has joined #openstack-kolla10:36
*** magicboiz has joined #openstack-kolla10:36
*** manheim has joined #openstack-kolla10:37
*** daidv has quit IRC10:40
*** matrohon has quit IRC10:41
*** matrohon has joined #openstack-kolla10:42
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set karbor-api workers using openstack_service_workers  https://review.openstack.org/44420910:42
*** jtriley has quit IRC10:51
*** mnasiadka has joined #openstack-kolla10:53
zhubingbing_sup haqduong11:00
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add cli options to cleanup-images  https://review.openstack.org/43150311:01
*** salv-orlando has quit IRC11:02
*** salv-orlando has joined #openstack-kolla11:02
*** salv-orlando has quit IRC11:02
zhubingbing_sup duonghq11:02
duonghqzhubingbing_, sup11:06
zhubingbing_https://review.openstack.org/#/c/430137/11:06
zhubingbing_http://paste.openstack.org/show/602231/11:07
zhubingbing_i don‘t find /usr/lib/libCryptoki2_64.so11:07
zhubingbing_can u help me look it ?11:07
duonghqzhubingbing_, which distro?11:09
zhubingbing_centos711:09
zhubingbing_centos11:09
zhubingbing_centos source11:09
*** salv-orlando has joined #openstack-kolla11:11
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix heat-engine and heat-api-cfn workers configuration  https://review.openstack.org/44422311:14
*** manheim has quit IRC11:14
masberhi11:15
masbermu kolla-ansible deploy finish but the br-ex is not setup properly11:15
masberI can see that the IP gateway configuration is still on the nic and hasn't been transfered to br-ex11:16
duonghqzhubingbing_, sorry, currently I have no idea, it was run when I pushed the ps11:16
zhubingbing_np11:17
masberalso the nic type is ethernet and not ovswitch11:17
zhubingbing_i go to openstack-barbican ask11:17
zhubingbing_;)11:17
masberalso the only way I can access the server after deployment is through the console11:18
masberis anyone aware of this?11:18
masberalso the file /usr/share/kolla/ansible/roles/baremetal/tasks/post-install.yml has a problem11:19
duonghqzhubingbing_, it's kind of software HSM module11:19
masberdocker stops working after executing "configure docker service"11:19
masberim using newton 3.0.211:20
masbernot sure if this version is still supported?11:20
*** manheim has joined #openstack-kolla11:20
duonghqzhubingbing_, one option is install softhsm package and use it's lib. But I cannot recall why I was able to get the libCryp.... file in the system11:21
*** manheim has quit IRC11:21
duonghqbye now11:21
*** duonghq has quit IRC11:21
*** manheim has joined #openstack-kolla11:21
*** jtriley has joined #openstack-kolla11:26
egonzalezmasber, can you share your globals.yml and output from ip a11:28
*** saneax is now known as saneax-_-|AFK11:28
masberegonzalez, I can put screenshots as right now I can only access through the console11:29
masber\but yeah give 1 sec11:29
egonzalezmasber, ip a in the host not the instance11:29
masberthe host is a vm11:30
masberhttp://pasteboard.co/HIgKFBh7m.png11:31
*** jtriley has quit IRC11:35
masberhttp://pasteboard.co/HIkQbsj7w.png11:35
masberhttp://pasteboard.co/HIl7l664w.png11:35
masberhttp://pasteboard.co/HIlrOK8Q0.png11:36
masber https://ibin.co/3F4mNjJcrSV0.png11:37
masberhttps://ibin.co/3F4mX62bvQM6.png11:37
masberhope that helps11:39
egonzalezmasber, what is neutron_external_interface value?11:39
egonzalezmasber, also, saw that external_fqdn is modified but not kolla_enable_tls_external11:40
masberneutron_external_interface is my public interface11:41
*** ccesario__ has joined #openstack-kolla11:41
masberdo I need kolla_enable_tls_external? description says it for encription and authentication11:43
egonzaleztls_external is used when an SSL endpoint is needed. If is public endpoint is not ssl public endpoint = internal_vip11:46
egonzalezfor neutron external network, should be an unused interface in the host, as it will be managed by ovs11:46
masberegonzalez, my host has 2 nics11:47
*** rwallner has joined #openstack-kolla11:47
masberone nice has 129.94.72.X and the other 10.1.0.X11:47
*** rmart04 has joined #openstack-kolla11:47
masbersorry what do you mean by "unused interface"11:48
egonzalezie: if you connect to the host at eth0, eth0 cannot be used for neutron_external_interface, otherwise will loose access to the host11:49
*** skramaja has quit IRC11:51
masberah11:52
masberI got you know11:52
masberthat is what is happenning to me11:52
masberso basically the nic cant migrate the settings to the br-ex because it is used by my connection11:53
egonzalezyep, but usually the behaviour is different. During deployment you lost access once the NIC is included in br-ex11:54
masberok11:55
masbernot sure why in my case my host loose connectivity but the settings are not migrated11:56
des_consoladogah what does PS stand for again?11:56
egonzalezpatch set :D11:56
des_consolado:)) so if you submit a PS that Depends-On another PS, does the dependency PS need to actually get merged before the other PS can be verified?11:56
egonzalezdes_consolado, yep, if the parent PS have merge conflic, the children will also fail11:58
masberegonzalez, so what should I do?11:59
masberdo the deployment connected to the internal nic?11:59
egonzalezif only have two nics, use one for network _interface (the one on which you connect) and the other nic left unused as neutron_external_interface12:00
des_consoladoegonzalez: oh I have merge conflicts... but not conflicts with master, conflicts with other PS's, it looks like?12:00
des_consolado(I did try re-pulling and rebasing on master and there wasn't anything to do, this is on the kolla branch change https://review.openstack.org/#/c/443732/ )12:01
des_consoladoso why is it saying that this change cannot be merged due to conflicts...12:01
masberright, what about kolla_enable_tls_external? do I still need to set it up?12:02
des_consoladomasber: what are you trying to do, just establish basic flat connectivity?12:02
egonzalezmasber, if you don't need ssl termination on public endpoints not. anyway need another interface for external_tls12:02
des_consoladoif you don't want to use TLS you don't need to set that, it's optional12:02
masberkk12:03
egonzalezdes_consolado, im seeing what happened, when pushed the change to kolla PS, kolla-ansible PS entered in conflict12:03
*** manheim has quit IRC12:05
des_consoladoso is there something I need to do?12:06
egonzalezdes_consolado, after rebasing locally don't let you git review?12:11
*** satyar has joined #openstack-kolla12:18
des_consoladoegonzalez: http://paste.openstack.org/show/602248/12:21
*** jtriley has joined #openstack-kolla12:24
*** shardy is now known as shardy_lunch12:26
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Set tacker-server api_workers using openstack_service_workers  https://review.openstack.org/44426512:26
*** jtriley has quit IRC12:34
*** jtriley has joined #openstack-kolla12:35
*** thomas_oneill has joined #openstack-kolla12:39
egonzalezdes_consolado, i cannot rebase from gerrit :/ can you?12:39
*** jtriley has quit IRC12:40
des_consoladoegonzalez: how? by pressing the rebase button in the web interface?12:40
egonzalezyep12:41
des_consoladoand then tick the box and put 'master' in the box and press rebase?12:41
des_consoladoor maybe just leave it empty...12:41
des_consoladoit says the change is already up to date12:42
*** rhallisey has joined #openstack-kolla12:45
*** manheim has joined #openstack-kolla12:45
*** salv-orlando has quit IRC12:48
*** spsurya__ has joined #openstack-kolla12:52
openstackgerritThomas O'Neill proposed openstack/kolla master: Add neutron-bgp-dragent service.  https://review.openstack.org/44373212:58
*** salv-orlando has joined #openstack-kolla13:00
*** magicboiz has quit IRC13:05
*** manheim has quit IRC13:11
*** thomas_oneill has quit IRC13:13
*** thomas_oneill has joined #openstack-kolla13:13
thomas_oneillgoing to just chat from my real name from now on... goodbye forever des_consolado13:14
*** des_consolado has quit IRC13:14
egonzalezthomas_oneill, hehe, des_consolado have a weird meaning in spanish13:15
thomas_oneilllol i don't even remember how i came up with that, just wanted something really anonymous and random because i was scared of irc i think ;p13:16
*** sudipto has quit IRC13:16
*** salv-orl_ has joined #openstack-kolla13:23
*** rwellum has joined #openstack-kolla13:24
*** salv-orl_ has quit IRC13:24
*** salv-orl_ has joined #openstack-kolla13:25
openstackgerritRonan-Alexandre Cherrueau proposed openstack/kolla-ansible master: Multi-regions Support  https://review.openstack.org/43165813:25
*** schwicht has joined #openstack-kolla13:26
*** salv-orlando has quit IRC13:26
*** mattmceuen has quit IRC13:27
*** manheim has joined #openstack-kolla13:32
*** schwicht has quit IRC13:32
*** dave-mccowan has joined #openstack-kolla13:32
*** Jezekus has quit IRC13:33
*** shardy_lunch is now known as shardy13:33
*** Serlex has joined #openstack-kolla13:37
*** jtriley has joined #openstack-kolla13:38
*** schwicht has joined #openstack-kolla13:39
nathharpHi all - do you have a process for accepting pull requests?13:41
nathharpthink I have found (and fixed) a bug13:41
Serlexnathharp is this any help - https://docs.openstack.org/infra/manual/developers.html13:44
egonzaleznathharp, if the issue need to be backported, open a bug in launchpad and then push the change to gerrit13:45
nathharpok - defintely found a bug - thought I had fixed it, but not quite sure yet13:46
*** mliima has joined #openstack-kolla13:46
*** rhallisey has quit IRC13:46
egonzaleznathharp, need help?13:46
egonzalezlet me know if need support setting all accounts hell13:47
*** lamt has joined #openstack-kolla13:47
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding virtualbmc to the gate  https://review.openstack.org/44155013:47
nathharpegonzalez you might be able to help me track down the issue13:47
*** Jezekus has joined #openstack-kolla13:47
*** spsurya__ has quit IRC13:48
egonzaleznathharp, whats the issue?13:50
nathharpegonzalez: first thing is related to the external_ceph.yml that I raised a bug for.   That bit definitely works, but further down the file there are some problems with ‘Pushing secrets key for libvirt”13:50
egonzaleznathharp, the "state" bug is already fixed13:50
nathharpegonzalez: yep, it is - but something breaks further down13:50
egonzalezhrm, something like "secrets is a directory error"?13:51
nathharpnathharp: line 79 -       content: cinder_cephx_raw_key - writes just ‘cinder_cephx_raw_key’ to the .base64 file, rather than the key value13:51
*** schwicht has quit IRC13:52
nathharpegonzalez:       content:  "{{ cinder_cephx_raw_key.stdout }}” grabs the right value13:52
*** deadnull has quit IRC13:52
egonzalezgot it, seems right adding stdout, otherwise is putting all content13:53
nathharpegonzalez: yep - currently it just prints the string, as without brackets it’s not referencing the variable.  Deployment works and libvirtd doesn’t complain any more13:54
egonzalezopen a bug report and assign to you13:55
nathharpegonzalez: I am still getting ceph auth issues in nova-compute, but I can’t work out if the RBD user is hardcoded (as the keys are now)13:55
nathharpegonzalez: ok will do - just a heads up, I’m not actually a developer, so bear with me if I submit incorrectly!13:56
egonzalezwhat auth issues? any error log?13:56
egonzaleznathharp, follow the developer guide Serlex posted before13:57
*** fguillot has joined #openstack-kolla13:57
nathharpegonzalez: http://paste.openstack.org/show/602261/13:59
*** rmart04 has quit IRC13:59
nathharpegonzalez: I’m not using either ‘cinder’ or ‘nova’ as my ceph users, so I suspect this is causing problems with the ceph.client.nova.key file14:00
*** mliima has quit IRC14:01
egonzaleznathharp, yep, users are hardcoded, maybe we have to move into customizable vars14:02
nathharpegonzalez:  Glance and Cinder both seem to be happy with whatever ceph client names I give them, as the keys are pulled in using a wildcard14:03
nathharpegonzalez: Nova is hardcoded now - shouldn’t take much to fix though, maybe I’ll propose that as a change too14:04
*** mattmceuen has joined #openstack-kolla14:05
*** schwicht has joined #openstack-kolla14:07
*** salv-orl_ has quit IRC14:08
manheimwhat is the role of kolla-toolbox container?14:08
*** spsurya__ has joined #openstack-kolla14:09
*** goldyfruit has joined #openstack-kolla14:11
*** manheim has quit IRC14:13
*** salv-orlando has joined #openstack-kolla14:18
*** salv-orlando has quit IRC14:20
*** sudipto has joined #openstack-kolla14:21
*** manheim has joined #openstack-kolla14:21
openstackgerritMartin André proposed openstack/kolla master: Add python-redis to aodh-evaluator image  https://review.openstack.org/44432014:22
*** mdnadeem has quit IRC14:27
openstackgerritMerged openstack/kolla-ansible master: Set senlin-api workers using openstack_service_workers  https://review.openstack.org/44416114:36
openstackgerritMerged openstack/kolla-ansible master: Set mistral-server workers using openstack_service_workers  https://review.openstack.org/44418014:36
egonzaleznathharp, the difference between cinder/glance with nova is that nova have to insert the secret into libvirt14:39
egonzalezthat's the reason why is not using wildcard14:39
*** Serlex has quit IRC14:40
*** rhallisey has joined #openstack-kolla14:40
egonzalezJeffrey4l_, what's your opinion on this https://review.openstack.org/#/c/439024/14:40
Jeffrey4l_egonzalez, why not use kolla_service_module?14:41
egonzalezi didn't used the module because service description is mandatory and don't want to change that behaviour14:42
egonzalezI can do it if prefer kolla_service_module14:42
Jeffrey4l_hrm, what's mean by mandatory. I can not understand that word very well.14:43
egonzalezrequired https://github.com/openstack/kolla/blob/master/docker/kolla-toolbox/kolla_keystone_service.py#L3214:44
Jeffrey4l_yep. i think it is OK. it is not big deal.14:45
egonzalezother services have description, but keystone not14:45
Jeffrey4l_this is a keystone-bootstrap issue. he do not add description.14:45
egonzalezOK, i'll rework to use the module14:46
Jeffrey4l_add desc is OK. if using kolla_keystone_service.14:46
Jeffrey4l_cool.14:46
Jeffrey4l_iirc, keystone-bootstrap seem can not update url.14:46
Jeffrey4l_did u reproduce this?14:46
egonzalezAdd a description or change to not required description?14:46
Jeffrey4l_add a description.14:47
egonzaleztried yesterday with kolla_keystone_service and adding a description updated the endpoint right14:47
Jeffrey4l_if keystone bootstrap update works, we do not need use kolla_keystone_service module.14:47
Jeffrey4l_so i guess keystone bootsrap can not update urls.14:47
Jeffrey4l_egonzalez, right? ^^14:48
egonzalezhrm , let me check that14:48
Jeffrey4l_tbh, if bootstrap update works. just change the url should works.14:48
kbaegisMorning gents. Has anyone had issues with "Project is not a valid project" or nova v2.1 api?14:49
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix heat-engine and heat-api-cfn workers configuration  https://review.openstack.org/44422314:50
kbaegisSeems like there's an issue on my deployment with quotas14:50
egonzalezkbaegis, yep, keystone endpoints must be versionless14:50
kbaegisegonzalez: Ahh, it's the keystone ones?14:50
egonzalezkbaegis, https://review.openstack.org/#/c/439024/14:50
kbaegistyvm14:50
egonzalezseem is not supported update endpoints in bootstrap https://github.com/openstack/keystone/blob/master/keystone/cmd/cli.py#L37014:56
zhubingbing_hey guys14:57
sbezverkJeffrey4l_: hi, qq when do you plan to publish ocata images?14:59
*** rwellum has quit IRC14:59
*** duonghq has joined #openstack-kolla15:00
Jeffrey4l_sbezverk, ocata is not released actually. even though  4.0.0.0rc2 is almost the final release.15:00
Jeffrey4l_i will push the images when it is really released.15:00
Jeffrey4l_guess 2-3 days later.15:00
*** lamt has quit IRC15:01
kbaegisegonzalez: Do you know if that will fix the weird issues I was having with the horizon neutron topology tab?15:02
kbaegisAnd quota creation?15:02
egonzalezkbaegis, what issues with neutron topology?15:03
egonzaleznova quota creation is fixed removing v3 from keystone endpoints15:03
kbaegisegonzalez: Horizon kept giving tracebacks on loading the topology section, referencing nova api15:04
kbaegisSo all you see is a big "Oops, something went wrong"15:04
*** david-lyle has joined #openstack-kolla15:05
egonzalezhrm, i've not seen that issue. may be an horizon bug, recently opened a bug because heat topology tab did't appear15:05
kbaegisYeah, I might have nuked gentoo- but I'm still kolla-build ubuntu source15:05
kbaegisHeat would be handy too15:06
kbaegisI was hoping to demo on Kolla today and again on Monday, but I might have to use my old fuel cluster15:06
kbaegis:/15:06
kbaegisSimply for stability15:06
sdakeJeffrey4l_ if you can push as soon as the tagging is done, that owuld be helpful - if you need help with that for centos or ubuntu let me know15:07
egonzalezquota issue is only on Pike(master) ocata should be fine15:07
egonzalezkbaegis, ^^15:07
Jeffrey4l_roger. i got some way to build the images faster in china.15:07
sdakeJeffrey4l_ not sure when the release team will tag - so we have to wait on them :)15:07
kbaegisegonzalez: Really? I'm using the stable/ocata and I still saw it15:07
Jeffrey4l_so i guess i can push them. if not, will ask help from guys.15:08
egonzalezkbaegis, actually there are two issues with nova quotas:15:08
*** lamt has joined #openstack-kolla15:08
sdakeJeffrey4l_ you can always use my box to push - i have gige internet15:08
Jeffrey4l_sdake, the final release is created by release team right?15:08
egonzalezkbaegis, if cannot do quota update is only in master15:08
sdakeJeffrey4l_ i am out of the office next week15:08
sdakeJeffrey4l_ right - the release team +2 +w's your release15:08
Jeffrey4l_ok. that will be cool15:08
*** david-lyle_ has joined #openstack-kolla15:08
*** david-lyle_ has quit IRC15:08
sdakeand then it tags as usual via automation15:08
*** david-lyle has quit IRC15:08
egonzalezkbaegis, if quotas are not released in DB, affect ocata too15:09
kbaegisegonzalez: I used your patch15:09
*** lucasxu has joined #openstack-kolla15:09
Jeffrey4l_sdake, seems i can not reach you node now.15:09
sdakeip may ahve changed15:09
Jeffrey4l_got.15:10
Jeffrey4l_i will sync with him.15:10
*** lamt has quit IRC15:10
kbaegisegonzalez: You just removed the /v3 from the endpoint correct?15:10
egonzalezkbaegis, yep15:11
kbaegisegonzalez: Awesome. Did a destroy;deploy, so hopefully it fixes that. I'll follow up :)15:11
*** shardy has quit IRC15:17
sbezverksdake: morning.. I just realized that atm we cannot use ocata, due to nova placement thing.. :(15:17
*** lucasxu has quit IRC15:20
sbezverkJeffrey4l_: I know you are busy now, but I was wondering if you could help me. Could you re-enable nova-placement in PS but keep it not merged until I bring nova-placement to kolla-kubernetes?15:20
sdakesbezverk patches need to merge - we are way past the release deadline15:21
Jeffrey4l_sbezverk, re-enable means? remove the if-condition in nova.conf?15:21
sbezverkJeffrey4l_: yep15:22
sbezverkor whatever other changes were done to disable it..15:22
Jeffrey4l_it is easy. np.15:22
Jeffrey4l_moment.15:22
sbezverkJeffrey4l_: let's make sure it is not merged until it is fully operational on kube side15:23
*** duonghq has quit IRC15:23
Jeffrey4l_roger.15:23
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Revert "Fixes problem with kolla-kubernetes 3.0.2 images"  https://review.openstack.org/44435115:25
Jeffrey4l_sbezverk, check ^^15:25
*** Jezekus has quit IRC15:25
*** Jezekus has joined #openstack-kolla15:26
sbezverkJeffrey4l_: thanks a lot15:28
Jeffrey4l_np :)15:28
*** lucasxu has joined #openstack-kolla15:32
*** manheim has quit IRC15:32
*** Jezekus has quit IRC15:33
*** sbezverk has quit IRC15:35
kbaegisAnnnd we're back to 'no space left on device'15:37
kbaegisDespite having no saturated drives whatsoever15:37
kbaegisI'm beginning to loathe docker logging15:37
*** manheim has joined #openstack-kolla15:39
srwilkersmorning o.15:42
srwilkerso/15:42
kbaegismorning!15:43
thomas_oneillhey, what actually is this kind of stuff? http://logs.openstack.org/32/443732/8/check/gate-kolla-dsvm-build-ubuntu-binary-ubuntu-xenial-nv/7a49e2a/ - like where do these files come from?15:44
thomas_oneillsetup_gate.sh maybe..15:45
*** salv-orlando has joined #openstack-kolla15:50
*** sbezverk has joined #openstack-kolla15:54
*** sbezverk is now known as Guest6254015:54
*** salv-orlando has quit IRC15:55
*** eaguilar has joined #openstack-kolla15:57
mgoddardcongr16:02
kbaegisthomas_oneill: Haven't played around with gating personally16:05
*** vhosakot has joined #openstack-kolla16:05
egonzalezthomas_oneill, yep, setup_gate, setup_<distro> and deploy_aio.sh16:06
*** rwellum has joined #openstack-kolla16:07
kbaegisDoes anyone want some updates to the ubuntu xenial guide I've been working on? Some of the steps are inaccurate16:07
*** rabel has quit IRC16:07
kbaegisOne of the big ones: apparmor_parser -R /etc/apparmor.d/usr.sbin.libvirtd really needs to be in rc.local16:08
kbaegisalong with configfs being loaded in modules16:08
kbaegisI think there is actually a note there for the later16:08
kbaegisBut it's under storage16:08
kbaegisand libvirt-bin needs to be systemctl disabled rather than service stopped16:09
*** bpetit_ has quit IRC16:11
*** unicell has joined #openstack-kolla16:11
openstackgerritNathan Harper proposed openstack/kolla-ansible master: Closes-Bug:1671862 Nova external-ceph.yml writes keyname into libvirt .base64 instead of value  https://review.openstack.org/44438916:11
*** unicell1 has quit IRC16:14
*** Jezekus has joined #openstack-kolla16:14
*** mnasiadka has quit IRC16:14
rwellumHi bmace16:16
*** sayantan_ has joined #openstack-kolla16:17
*** bpetit has joined #openstack-kolla16:20
*** egonzalez has quit IRC16:23
*** sayantan_ has quit IRC16:24
*** vhosakot_ has joined #openstack-kolla16:26
inc0good morning16:28
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla master: Fix oslo_debug_helper not running  https://review.openstack.org/44439816:29
*** vhosakot has quit IRC16:30
kbaegismorning!16:31
*** manheim has quit IRC16:31
vhosakot_morning all, still in the process of getting my cluster up for kolla-k8s dev env.16:31
spsuryaGd Mrg inc016:31
rwellumvhosakot_, me too. It got as far as ceph then it's died.16:31
vhosakot_rwellum: ah ok16:32
spsuryaBt I would like to gd Nt, here 1:30 AM in Japan16:32
vhosakot_inc0: were you able to use the inventory and deploy https://github.com/att-comdev/halcyon-kubernetes?16:32
*** sayantan_ has joined #openstack-kolla16:32
rwellumvhosakot_, http://paste.openstack.org/show/602288/16:33
vhosakot_rwellum: mounting failed.. ok16:33
srwilkersmorning inc016:33
vhosakot_rwellum: may be you could wait few mins for k8s to self-heal :)16:33
rwellumvhosakot_, http://paste.openstack.org/show/602289/16:34
rwellumBeen 20h now16:34
rwellumvhosakot_, what I am not sure about as well, is how to back out of this - clean up and retry. I think the dev guide needs some basic cleanup and debugging steps.16:39
vhosakot_rwellum: agreed, we need steps to tear down.16:40
*** spsurya__ has quit IRC16:41
kbaegisHey guys, what's the appropriate way to set up a provider network under kolla?16:42
thomas_oneillsame way as with any openstack16:42
thomas_oneillsee tools/init-runonce in the repo16:42
kbaegisthanks16:42
kbaegisJust don't know what it's called by default :)16:43
thomas_oneillthat init-runonce script should do the basics of setting up a couple of networks, a router, and a cirros instance, as a proof of the deployment working16:43
thomas_oneillshould be some calls like: openstack network create16:43
*** dims_ has quit IRC16:45
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Fix the "tox -e docs" command failed  https://review.openstack.org/44440416:45
*** zhubingbing_ has quit IRC16:45
kbaegisthomas_oneill: Just altered it to match my network16:46
kbaegisthomas_oneill: thanks16:46
kbaegisphysnet1 was what I was looking for :)16:46
kbaegisI created all my flavors manually from https://aws.amazon.com/ec2/instance-types/16:46
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Remove oslo_messaging_notifications section config  https://review.openstack.org/44440516:48
inc0vhosakot_: yeah, still figuring out networking16:48
thomas_oneillkbaegis: also note that https://github.com/openstack/kolla-ansible/blob/master/ansible/library/kolla_docker.py#L531 - the host's network namespace is shared by every container, so networking is exactly as it would be without using docker/containerisation16:49
vhosakot_inc0: I have kinda figured out the networking by reverse-engineering the vagrant part.16:49
inc0with canal?16:49
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Cloudkitty: remove oslo_messaging_notifications config  https://review.openstack.org/44440516:50
kbaegisthomas_oneill: That's a pretty cool fact16:50
kbaegisDoes anyone here work with opendaylight ooc?16:51
rwelluminc0, are you in-line with https://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP - or is this completely sep. setup?16:51
inc0rwellum: separate16:52
inc0my goal is to create good and repeatable way to deploy k8s on multinode16:53
inc0after that I'll make notes and put them in this etherpad16:53
rwelluminc0, you're not tying it in with kolla? Because to me k8s was fairly easy to set up - albeit I did single node.16:54
inc0rwellum: that's the thing16:54
*** rabel has joined #openstack-kolla16:55
inc0so way I work with kolla-ansible16:55
inc0is I spawn 5 or so vms, fresh16:55
openstackgerritNathan Harper proposed openstack/kolla-ansible master: Closes-Bug:1671862 external-ceph.yml writes keyname into libvirt  https://review.openstack.org/44440916:55
inc0and deploy on them16:55
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Freezer: remove oslo_messaging_notifications config  https://review.openstack.org/44441016:55
inc0that requires automation:)16:55
*** pbourke has quit IRC16:55
vhosakot_inc0: not with canal yet, I did weave, I'll post my networking setup soon..16:55
inc0kk16:55
inc0so it's a bit more...complex scenerio16:56
vhosakot_inc0: so, the idea is to remove all and any vagrant bits (that have trouble me a lot :)), and use just the Ansible playbooks of halcyon with the right inventory16:56
*** lucasxu has quit IRC16:56
*** logan_ has joined #openstack-kolla16:56
*** gfhellma has joined #openstack-kolla16:56
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Congress: remove oslo_messaging_notifications config  https://review.openstack.org/44441116:56
*** logan_ is now known as Guest2958916:57
inc0after that I'll start working on etherpad and also using few features that we merged recently to lower amount of steps in this etherpad16:57
inc0I should have good idea today16:57
inc0just let me kick start my brain;)16:57
*** jtriley_ has joined #openstack-kolla16:57
openstackgerritNathan Harper proposed openstack/kolla-ansible master: Nova external-ceph.yml writes keyname into libvirt.base64 Change-Id: I6b77773238cd65a3cc009d8d9a38bdbea0035212  https://review.openstack.org/44438916:57
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Solum: remove oslo_messaging_notifications config  https://review.openstack.org/44441216:58
*** lucasxu has joined #openstack-kolla16:58
*** logan- has quit IRC16:59
rwelluminc0, that'll be great.17:00
*** jtriley has quit IRC17:00
rwelluminc0, vhosakot_ one thing I like about vagrant... You want to cleanly kill k8s - 'vagrant destroy' - bring it back up 'vagrant up'.17:00
vhosakot_rwellum: yes, destroy is easy in vagrant..17:01
vhosakot_rwellum: vagrant has also destroyed my brain a bit too ;)17:01
rwellumvhosakot_, haha!17:01
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Cloudkitty: remove oslo_messaging_notifications config  https://review.openstack.org/44440517:01
*** lucasxu has quit IRC17:02
*** Guest29589 is now known as logan-17:03
inc0issue for me is - if we use vagrant too much17:06
inc0aio one17:06
inc0we will miss out lot's of cases which appears on multinode17:06
inc0that's why I have no interest whatsoever in aio17:07
vhosakot_rwellum: one way to tear down is, delete pods, services, replication controllers, kills k8s control plane (controller-manager, api-server, shceduler, cni, dashboard), and restart the nodes, to get back to day-0 state.17:08
*** zhubingbing has joined #openstack-kolla17:08
inc0in my case it's nova rebuild <<all of my vms>>;)17:08
vhosakot_inc0: yes, I also hate AIOs.. not a big fan of vagrant either.17:08
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Sahara: add oslo_messaging_notifications config  https://review.openstack.org/44442017:09
vhosakot_brb 517:10
rwelluminc0, don't fully understand - vagrant can handle multi-nodes - in fact does it very well?17:11
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Trove: add oslo_messaging_notifications config  https://review.openstack.org/44442217:13
*** nathharp has quit IRC17:15
*** rabel has quit IRC17:17
*** sudipto has quit IRC17:17
zhubingbing;)17:18
*** matrohon has quit IRC17:19
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Senlin: enabled oslo_messaging_notifications config  https://review.openstack.org/44442317:21
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Trove: add oslo_messaging_notifications config  https://review.openstack.org/44442217:23
kbaegisOh. yikes17:24
kbaegisDid a kolla-ansible reconfigure to enable murano17:24
kbaegisNow horizon logs this: https://hastebin.com/lepivusuzo.sql17:24
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442617:24
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Sahara: add oslo_messaging_notifications config  https://review.openstack.org/44442017:25
*** salv-orlando has joined #openstack-kolla17:26
kbaegisalso murano fails to deploy17:26
*** vhosakot has joined #openstack-kolla17:26
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442617:26
*** salv-orlando has quit IRC17:27
*** salv-orlando has joined #openstack-kolla17:27
kbaegisGuess murano stays disabled17:28
kbaegisHere's the deploy failure log on -vvv https://hastebin.com/hutamefuqo.md17:28
*** david-lyle has joined #openstack-kolla17:28
*** vhosakot_ has quit IRC17:28
*** lucasxu has joined #openstack-kolla17:29
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Mistral: add oslo_messaging_notifications config  https://review.openstack.org/44442817:30
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Mistral: enabled oslo_messaging_notifications config  https://review.openstack.org/44442817:30
*** JoseMello has joined #openstack-kolla17:31
kbaegisAnyone seen that before? No such table: django_session?17:32
*** lamt has joined #openstack-kolla17:37
*** mattmceuen has quit IRC17:39
*** ssurana has joined #openstack-kolla17:40
inc0rwellum: sorry, was moving to office17:42
inc0well, vagrant doesn't work for me;)17:42
vhosakotinc0: what issue do you see with vagrant?  high CPU usage like >200%.  that's the main reason I'm deploying halcyon minus the vagrant bits using just the Ansible playbooks.17:44
*** unicell has quit IRC17:44
Guest62540kfox1111: when you have a chance please ping me17:45
inc0vhosakot: issue of me having 8gig of ram in my macbook air and access to metric ton of resources in openstack external17:45
inc0that and fact that vagrant-openstack is no bueno17:45
*** Guest62540 is now known as sbezverk17:46
vhosakotinc0: ah yeah, I think 8 G is less.17:46
inc0also I like to deploy my own stuff so I can 1. understand it better and 2. can deploy it for prod when needed17:46
vhosakotinc0: how? using the playbooks in https://github.com/att-comdev/halcyon-kubernetes?17:46
inc0vhosakot: well, when I learn this stuff, I'll decide17:47
inc0now it's more about excersise in learning17:47
*** kbaegis1 has joined #openstack-kolla17:48
vhosakotinc0: cool, other option is to use k8s offcial docs to standup kubeadm needed for kolla-k8s - https://kubernetes.io/docs/getting-started-guides/kubeadm/17:48
vhosakotit is non-vagrant and close to prod IMO ^^^17:48
inc0vhosakot: halcyon wraps that17:48
inc0and kubeadm is manual to a point17:48
*** kbaegis has quit IRC17:49
inc0I mean you need to log into node and run it17:49
inc0and I don't want to log into nodes:(17:49
zhubingbingyes17:49
inc0I'd rather have ansible log into all of them17:49
vhosakotinc0: yes, you need to do kubeadmin join/init on each nodes logging in.. true17:49
*** kbaegis1 has quit IRC17:49
*** kbaegis has joined #openstack-kolla17:50
inc0it's easy to ansiblize tho17:50
zhubingbingvhosakot  K8S use kubeadmin install it17:50
zhubingbing;)17:50
*** zhubingbing has quit IRC17:50
rwellumI concur - that guide is great for bringing up k8s. There's some networking gotchas to be aware of17:50
inc0yeah that's what I'm working on now17:51
inc0I'll try just flannel without calico17:51
*** zhubingbing has joined #openstack-kolla17:51
vhosakotzhubingbing: well, not all.. kubeadm is one new/easy/beta way... kargo (https://github.com/kubernetes-incubator/kargo) does not use it and still working on implementing it (https://github.com/kubernetes-incubator/kargo/issues/553).17:52
vhosakotzhubingbing: the k8s official docs to deploy multi-node CentOS also does not use kubeadm - https://kubernetes.io/docs/getting-started-guides/centos/centos_manual_config/17:53
inc0kargo doesn't work:(17:53
inc0it never really did unfortunately, but before that I could hack around some issues17:53
inc0now I don't want to, I just don't trust them any more;)17:54
vhosakotinc0: well, kargo works great for me.. I have deployed it on 4-node with calico, play with it everyday,17:54
inc0I had tons of strange issues with it17:54
vhosakotinc0: ah ok, the kargo slack channel is really helpful.17:54
inc0I'll wait till kubeadm is prod-ready and write extremally simple ansible playbook17:54
inc0well, ok not so simple17:54
inc0but kubeadm has great potential to be properly ansiblized17:55
zhubingbingi will finish it ;kolla-k8s preinstall17:55
zhubingbingpre-install17:55
vhosakotsbezverk: to standup kolla-k8s dev env, any reason why we _must_ deploy using kubeadmn and not using non-kubeadm ways like https://github.com/kubernetes-incubator/kargo or https://kubernetes.io/docs/getting-started-guides/centos/centos_manual_config/?17:55
vhosakotinc0: well, kubeadm will drastically reduce/minimize the use of ansible IMO17:55
inc0that's what I'm hoping for17:56
inc0you just template initial manifest17:56
rwellumzhubingbing, what will you finish?17:56
inc0and run couple of commands17:56
vhosakotinc0: yep, "kubeadmin init" and "kubeadmn join" do the work of like 2-5 playbooks or something :)17:56
vhosakotsbezverk: I'd love to see your reply about the requirement to deploy kubeadm to standup kolla-k8s dev.17:57
rwellumSo one thing that confuses me (well another of many) - when I started to play with kubeadm - it quickly became a 'deployment on bare metal' thing. But I just want currently a development environment as simple as possible.17:57
zhubingbingrwellum  deploy kolla-k8S pre-install something17:57
zhubingbinglike there project https://github.com/kubernetes-incubator/kargo17:58
vhosakotrwellum: did you deploy using kubeadm?17:58
rwellumvhosakot, yes17:58
zhubingbingme too17:58
vhosakotrwellum: cool, thanks for the info.17:58
inc0I'm trying pure flannel17:59
zhubingbingi usually  use kubedam17:59
zhubingbinginc017:59
vhosakotzhubingbing: cool... I might just try https://kubernetes.io/docs/getting-started-guides/kubeadm/ then..17:59
inc0flannel - networking overlay17:59
zhubingbinginc0 me too i use flannel17:59
inc0I tried canal but first things first;)17:59
zhubingbingkubeadm it‘s very nice17:59
zhubingbing;)17:59
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442617:59
zhubingbingtwo command it is samples18:00
rwellumvhosakot, http://paste.openstack.org/show/602296/ - it's been up for many days on my VM18:00
vhosakotinc0: yep, I've used flannel too a bit.. it is pretty cool18:00
zhubingbingcalico i use it have some probelmes18:01
rwellumvhosakot, now I need someone to help me get kolla running on this... :p18:01
*** mgoddard has quit IRC18:01
zhubingbingi usually use flannel18:02
inc0I want to move to calico as it's more fun18:02
inc0but later18:02
zhubingbingkolla?or kolla-Kubernetes?18:02
inc0k8s18:02
zhubingbinghttps://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP18:03
kbaegison AIO, the API interface is used for br-ex as well correct?18:03
zhubingbingi hope it will hole you18:03
kbaegisI can create VMs, but they can't ping in/out18:03
zhubingbingsdake write it,i have test passed18:03
zhubingbingrwellum https://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP18:04
kbaegisnvm18:04
inc0kbaegis: negative, neutron external interface18:04
kbaegisIt just took a second for the configs to sync18:04
kbaegisinc0 Ah, got it. Thought that was for overlay only18:05
rwellumzhubingbing, I am writing that guide too18:05
inc0kbaegis: https://docs.openstack.org/developer/kolla-ansible/production-architecture-guide.html#network-configuration18:05
zhubingbingkbaegis you have to know the physical network environment18:05
zhubingbingrwellum ;)18:06
kbaegisinc0: Thanks!18:07
kbaegisdnat is working wonderfully, snat not so much18:07
kbaegisJust troubleshooting now18:07
*** lucasxu has quit IRC18:08
kbaegisSo the float is allocated, but status is down. I see the arp come in from br-ex, but it's not natted from there18:09
kbaegisSecurity groups should allow icmp/trcp18:09
kbaegis*tcp18:09
kbaegishmm18:09
inc0kbaegis: yeah...that's neutron for ya18:09
kbaegisinc0: :)18:09
zhubingbingguys good night18:09
zhubingbing;)bye18:09
inc0bye zhubingbing18:09
kbaegisinc0: I've had more issues with dnat than snat for sure18:10
kbaegiszhangqiankun: night!18:10
*** zhubingbing has quit IRC18:10
kbaegisinc0: Well, magically working now18:10
inc0lol18:11
inc0rrrriigggghtt....18:11
inc0well, it's working so don't touch it18:11
*** zioproto has quit IRC18:11
kbaegisinc0: lol Yeah-18:11
inc0I'd refrain from breathing anywhere near these machines too18:11
kbaegisIf I could just get murano and designate working on my cluster I'd be a happy happy man. Only have 8 desired services that aren't deploying under kolla currently18:12
inc0designate should be there - at least in ocata18:13
rwelluminc0, educate me - why is openstack-vagrant 'no bueno'? No interest? Not appropriate?18:13
inc0murano should be there even before18:13
inc0rwellum: broken18:13
inc0;)18:13
inc0couldn't get it to work, had strangest ruby error traces18:14
rwelluminc0, IC.18:15
jascott1ruby... pfffft18:15
*** lucasxu has joined #openstack-kolla18:15
inc0anyway, I'm this operator type with unhealthy affection to basements18:17
inc0so vagrant isn't something that moves me18:18
*** hieulq has quit IRC18:21
*** zhubingbing_ has joined #openstack-kolla18:22
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442618:25
vhosakotbrb 1018:27
rwellumvagrant's power is that you can run vm's programatically on laptops imo. If you look at some network providers like Cisco, Juniper etc - they provide VM's of their OS's/Stacks in vagrant form. Then their customers can learn and play with a cheap VM instead of buying expensive n/w gear.18:27
*** unicell has joined #openstack-kolla18:28
vhosakotrwellum: cool, since you have setup up, have you considered the remaining steps to deploy kolla-k8s on your cluster by running build_dev_image.sh and run_dev_image.sh at the end of https://docs.openstack.org/developer/kolla-kubernetes/development-environment.html ?18:28
*** Jezekus has quit IRC18:29
sbezverkinc0: qq I need to run a specific version of kolla-ansible at the gate for debugging purposes.  do you recall the syntax to apply a not merged review on top of cloned repo?18:30
rwellumvhosakot, that's what sbezverk suggested. I tried and had issues. It really should be plug-and-play.18:31
rwellumvhosakot, mainly had certification issues.18:32
vhosakotrwellum: I thought build_dev_image.sh and run_dev_image.sh do not assume the underlying k8s cluster has been deployed using vagrnat from the same doc.18:32
vhosakotrwellum: ah ok, I will try it today, and let you know.18:33
rwellumvhosakot, I'll try again. I'll pastebin the output if you're interested.18:33
vhosakotsure, please.18:33
vhosakotrwellum: i'll be offline now for an hr.. I can surely check back18:33
rwellumok ttyl18:33
vhosakotrwellum: I'll check the IRC logs... ttyl18:34
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442618:36
*** hieulq has joined #openstack-kolla18:36
*** g3ek has quit IRC18:37
*** vhosakot has quit IRC18:38
*** sambetts is now known as sambetts|afk18:41
*** zhubingbing_ has quit IRC18:42
*** zhubingbing has joined #openstack-kolla18:44
*** g3ek has joined #openstack-kolla18:46
*** zhubingbing has quit IRC18:46
*** rhallisey has quit IRC18:47
*** satyar has quit IRC18:48
inc0sbezverk: when you go to review.openstakc.org18:48
inc0to your review18:48
inc0on top right corner there is "download" button18:48
*** rhallisey has joined #openstack-kolla18:48
inc0copy contents:)18:48
openstackgerritQin Wang (qwang) proposed openstack/kolla-kubernetes master: [WIP] Condition and tags in requirements.yaml for keystone service  https://review.openstack.org/44446118:51
*** sayantan_ has quit IRC18:51
sbezverkinc0: that part I know :)18:51
sbezverkI need to pull that PS at the gate18:51
sbezverkin the script18:52
inc0sbezverk: ehh...sounds...bad18:52
sbezverkinc0: on top of the master which I cloned18:52
inc0well, use this command18:52
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442618:56
*** mgoddard has joined #openstack-kolla19:00
sbezverkinc0: which command?19:02
inc0git checkout ...19:02
sbezverkinc0: but if PS is not merged, can I still check it out??19:03
inc0yes, this is what this command is about19:04
inc0I mean I really don't like it to be done in gate19:04
inc0and by don19:04
inc0don't like I mean it's really bad19:04
sbezverkinc0: it is only temporarely since placement is not supported by other images19:04
sbezverkI need to enable placement to test charts for it19:05
sbezverkif I get placement fix merge on ansible side, then all out gates will be red..19:06
*** MasterOfBugs has joined #openstack-kolla19:06
*** rabel has joined #openstack-kolla19:08
*** rabel_b1 has joined #openstack-kolla19:13
*** rabel has quit IRC19:17
*** kelv has joined #openstack-kolla19:17
inc0is there a call in k8s "show me all the resources and types they have"?19:19
inc0like, I want to remove tiller and Ihave to guess if it's replicaset,pod or whatever?19:19
jascott1kubectl get all ? there is something like that19:21
openstackgerritAlicja Kwasniewska proposed openstack/kolla-ansible master: Improved documentation for central logging.  https://review.openstack.org/44447719:21
SamYapleinc0: you dont have to guess19:21
SamYaplewhat do you mean?19:21
inc0get all works19:22
inc0I'm trying to debug why my tiller is not working19:22
*** rabel_b1 has quit IRC19:23
jascott1prob in crash loop?19:23
*** salv-orl_ has joined #openstack-kolla19:23
inc0jascott1: but why19:24
inc0that's the question19:24
inc0logs shows nothing19:24
inc0describe shows "connection refused" on port19:24
rwellumkubectl describe po <pod-name> --namespace='<something>19:24
inc0and k8s is supposed to help with debug :(19:24
rwelluminc0, http://paste.openstack.org/show/602306/19:26
rwellumworks for me even on a busted pod19:26
*** salv-orlando has quit IRC19:26
inc0yeah I know19:26
inc0it's not k8s issue, it might be tiller issue19:26
inc0all I'm seing is "container is unhealthy, killing"19:26
inc0which is completely not helpful19:26
inc0pinging container ip works19:27
*** sayantan_ has joined #openstack-kolla19:28
inc0it just doesn't start for reasons unknown19:28
inc0and liveness probe kills it19:28
rwellumI've seen this too I'm sure - permissions maybe?19:28
inc0rwellum: no idea...I'm trying to find logs to work with19:28
jascott1is that from automation or are you starting it manually?19:28
*** salv-orl_ has quit IRC19:28
inc0jascott1: I tried both19:29
jascott1did you kubectl describe pod <tillerpod>19:29
inc0yes I did;)19:29
rwellumSomething like: k logs tiller-deploy-610593871-xq3bk --previous  --namespace=kube-system19:29
inc0it's utterly unhelpful19:29
jascott1ask in #helm for troubleshooting tips maybe19:31
inc0yeah, I'm about to19:31
jascott1ive had similar with vagrant but I just nuked the whole thing, my issues are usually around the proxy but thats prob not your cae19:31
SamYapleinc0: its always dns!19:31
rwellumI had to kill it a few times looking at my history: "kubectl --namespace kube-system delete deployment tiller-deploy"19:31
rwellumnm help sorry19:31
inc0http://paste.openstack.org/show/602307/19:32
inc0ehh...kube dns is new neutron I see19:32
SamYapleinc0: its always dns!19:32
rwellumI saw lots of this using canal - less with weave - eventually k8s came up happy. Don't know if 7m is waiting long enough19:33
inc0but liveness probes trues ip :(19:33
inc0this is flannel and ping to this ip works19:34
SamYapleyea it segfaults with the wrong dns address or something19:34
SamYapleor rather without proper resolution19:34
inc0eeee19:34
inc0well...19:34
inc0duh19:35
*** mgoddard has quit IRC19:36
rwellumwow19:37
*** rhallisey has quit IRC19:49
*** dims has joined #openstack-kolla19:49
*** rhallisey has joined #openstack-kolla19:56
*** JoseMello has quit IRC20:05
*** sambetts|afk has quit IRC20:06
*** sambetts_ has joined #openstack-kolla20:08
*** manheim has joined #openstack-kolla20:11
*** bpattewar has joined #openstack-kolla20:12
bpattewarHi20:12
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Mistral: enabled oslo_messaging_notifications config  https://review.openstack.org/44442820:13
bpattewarDo we any docs/ presentations for kolla and kolla-ansible architecture?  Most docs in repo just speak about how to create image , how to install..20:13
inc0bpattewar: https://docs.openstack.org/developer/kolla-ansible/production-architecture-guide.html#network-configuration a little here20:15
*** goldyfruit has quit IRC20:15
inc0but what is your question?20:15
*** manheim has quit IRC20:16
*** goldyfruit has joined #openstack-kolla20:16
bpattewarinc0 I am trying to understand how docker images and ansible is glued together..20:17
inc0bpattewar: https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/handlers/main.yml#L2-L26 with tasks like that20:19
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Senlin: enabled oslo_messaging_notifications config  https://review.openstack.org/44442320:19
inc0ansible in low lever will ssh to node and do equivalent to docker run -d ubuntu-source-keystone:4.0.0 with volumes and configs in plays20:19
inc0https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/defaults/main.yml#L4-L14 these are configs for running a container20:20
inc0this pattern is replicated across the project20:20
*** nathharp has joined #openstack-kolla20:21
*** vhosakot has joined #openstack-kolla20:23
*** salv-orlando has joined #openstack-kolla20:29
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Sahara: add oslo_messaging_notifications config  https://review.openstack.org/44442020:29
bpattewarinc0 thanks for the help. I am working on heat service. I am getting issue with heat-manage-db microservice. All other microservices run some commands, but I am not able ot understand purpose of manage-db microservices20:29
inc0bpattewar: ahh...you talk about kolla-kubernetes?20:30
inc0or kolla-ansible?20:30
bpattewarinc0 , I am talking about kolla-kubernetes . It also don't executes any commands( I don't see anything in it) ,20:30
bpattewardo you have any idea20:30
vhosakotrwellum: back, sorry was offline longer than an hour... so, I opened this bug to propose the new command "vagrant top" https://github.com/mitchellh/vagrant/issues/8358.  "vagrant top" may be helpful to debug issues I’m seeing when vagrant VMs consume a lot of CPU of the host.  feel free to thumbs up for it on GitHub if you think this feature is useful so it will be implemented quickly :)20:32
*** ssurana has quit IRC20:32
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Trove: add oslo_messaging_notifications config  https://review.openstack.org/44442220:32
*** ssurana has joined #openstack-kolla20:33
*** salv-orlando has quit IRC20:34
rwellumvhosakot, done20:34
vhosakotthanks a lot rwellum!20:35
inc0bpattewar: good question... sbezverk kfox1111 can you help?20:35
rwellumbpattewar, the kolla-k8s guide is in early stages: https://etherpad.openstack.org/p/kolla-kubernetes-deploy-guide-BP20:36
rwellumbpattewar, And there's a dev guide too: https://docs.openstack.org/developer/kolla-kubernetes/development-environment.html20:36
rwellumvhosakot, why don't you write the command? :) I can help...20:37
openstackgerritCharlie Kang proposed openstack/kolla master: cinder-volume ubuntu source build  https://review.openstack.org/44449820:42
*** manheim has joined #openstack-kolla20:53
*** manheim has quit IRC20:53
*** lucasxu has quit IRC20:57
*** sayantan_ has quit IRC20:57
*** lucasxu has joined #openstack-kolla20:57
vhosakotrwellum: haha, really? vagrant is all Ruby, I'm no Ruby expert :)20:59
*** sayantan_ has joined #openstack-kolla21:00
openstackgerritNathan Harper proposed openstack/kolla-ansible master: Nova external-ceph.yml writes keyname into libvirt.base64 Change-Id: I6b77773238cd65a3cc009d8d9a38bdbea0035212  https://review.openstack.org/44438921:01
*** sayantan_ has quit IRC21:01
*** sayantan_ has joined #openstack-kolla21:02
*** sayantan_ has quit IRC21:05
*** eaguilar has quit IRC21:07
*** fguillot has quit IRC21:15
*** sayantan_ has joined #openstack-kolla21:20
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Nova oslo_messaging_notifications configuration error  https://review.openstack.org/44450421:26
*** manheim has joined #openstack-kolla21:27
*** manheim has quit IRC21:28
*** jtriley_ has quit IRC21:28
*** salv-orlando has joined #openstack-kolla21:30
*** Jeffrey4l__ has joined #openstack-kolla21:35
*** l4yerffeJ_ has quit IRC21:35
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Neutron oslo_messaging_notifications configuration error  https://review.openstack.org/44450621:35
*** Jeffrey4l_ has quit IRC21:35
*** srwilkers has quit IRC21:36
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Neutron oslo_messaging_notifications config error  https://review.openstack.org/44450621:37
*** deadnull has joined #openstack-kolla21:38
kbaegisAnyone doing something like this? /etc/kolla/config/cinder.conf https://hastebin.com/apazoderus.ini21:39
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Nova oslo_messaging_notifications configuration error  https://review.openstack.org/44450421:39
openstackgerritQin Wang (qwang) proposed openstack/kolla-kubernetes master: [WIP] Condition and tags in requirements.yaml for keystone service  https://review.openstack.org/44446121:40
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442621:41
*** vhosakot has quit IRC21:46
jemcevoyinc0 are you going to the Docker meetup at Intel this time?  https://www.meetup.com/Docker-Portland-OR/events/238043125/?fromEmail=238043125&rv=me121:49
*** lucasxu has quit IRC21:52
*** lucasxu has joined #openstack-kolla21:53
*** rwellum has quit IRC21:53
*** rwellum has joined #openstack-kolla21:54
*** dave-mccowan has quit IRC21:57
*** jtriley has joined #openstack-kolla21:58
*** nathharp has quit IRC22:00
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Heat: add oslo_messaging_notifications config  https://review.openstack.org/44451522:05
*** rwellum has quit IRC22:06
openstackgerritBalaji G Pattewar proposed openstack/kolla-kubernetes master: Heat service deployment  https://review.openstack.org/43725222:07
*** g3ek has quit IRC22:07
*** jtriley has quit IRC22:09
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442622:13
*** goldyfruit has quit IRC22:14
*** mgoddard has joined #openstack-kolla22:15
*** g3ek has joined #openstack-kolla22:16
inc0anyone goes to ops midcycle?22:18
inc0jemcevoy: I'll be there22:23
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Make Ironic inspector honour inventory  https://review.openstack.org/43692922:26
*** lucasxu has quit IRC22:35
*** rwallner has quit IRC22:36
sdakeinc0 milan sounds like fun22:37
sdakeinc0 not in my budget sadly22:37
inc0yeah22:38
inc0sdake: but you'll be happy to hear that we got our kolla hands on labs22:38
inc0in summit22:38
sdakeinc0 yes i saw all the summit talks related to kolla yesterday when they were published22:39
sdakealthough i didn't catch who was presenting the alb22:39
inc0labs are made by my team22:39
sdakeoh right it was intel peeps22:39
sdakekolla had a pretty good haul at summit this go around22:40
inc0agree22:40
sdakesomehow i got roped into a 4 person update talk on kolla ;-)22:40
inc0it's gonna be glorious22:40
inc0blame me22:40
sdakeexhuasting more like it ;)22:40
inc0;)22:40
inc0but I can do it all by myself22:40
sdakeinc0 my name was attached to the talk, i am expected to be there ;)22:43
inc0I was meant to provide names22:43
inc0so I provided names22:43
sdakeasked you mean?22:43
inc0and I think we talked about that;)22:43
inc0yeah22:43
sdakewhatever works22:44
inc0that was late last year - kolla update22:44
sdakei dont recall - doesn't mean it didn't happen22:44
sdakeatleast its not a ninja panel :)22:44
inc0LD22:44
inc0:D22:44
sbezverksdake: need your help22:46
sdakesbezverk sup dude22:47
sdakei am on a plane22:47
sdakenot sure i can hlep much22:47
sbezverksdake: I got stuck on placement22:47
sbezverksdake: I see22:47
sdakestuck where22:47
*** mgoddard has quit IRC22:47
sbezverksdake: remember once you added to the gate applying of not merged PS22:47
sbezverkafter kolla-ansible got clonned22:48
sdakesbezverk yes22:48
sbezverkto move forward with placement I need to do the same trick22:48
sdakeok22:48
sbezverkcannot recall how you did it22:48
sdakethats easy22:48
sdakeneed me to do it for ya - or want me to teach you22:48
sdakei can't use webex on this plane fwiw22:48
sbezverkI would prefer to learn it22:49
sbezverkas it might be useful in future22:49
sdakeok22:49
sdakestep 1. modify kolla-ansible master and git commit -a the change you want22:49
sdakethen git format-patch -122:49
sdakethis will make a patch called 0001-whatever in your CWD22:49
sdakeget that far22:49
sbezverksdake: it is not my change, jeffrey4l did it for me22:50
sbezverkso I do not have this change in my repo, do I need to cherrypick it?22:50
openstackgerritQin Wang (qwang) proposed openstack/kolla-kubernetes master: [WIP] Condition and tags in requirements.yaml for keystone service  https://review.openstack.org/44446122:50
sdakeyup hcerrypick it22:51
sdakethen git format-patch -122:51
sbezverksdake: ok then I just add it to my ps and apply as patch?22:52
sdakegit am path-to-patch22:52
sdakeafter checking out hte kolla-ansible repo22:52
*** vhosakot has joined #openstack-kolla22:52
sbezverksdake: hm never seen git am command22:52
sdakeit applies a formatted patch22:52
sdaketo an existing repository22:53
sdakeits how you pass around patches "in the old days prior to gerrit ;-)22:53
sdakeon mailing lists and whatnot22:53
sbezverksdake: cool thanks. I will try to finish placement today tomorrow, when you see it is readdy please ack it22:54
*** spsurya__ has joined #openstack-kolla23:00
vhosakotsbezverk: ping23:03
*** schwicht has quit IRC23:03
kfox1111jascott1: ping23:03
vhosakotsbezverk: you saw the error "Package ed1cdb328f2ee2cd2c9a200f3a3ae549801cd51216ef82e37d2b92acdbd72cf5-kubectl-1.5.4-0.x86_64.rpm is not signed" in gate today?23:03
vhosakotwhen doing "yum -y install kubectl"?23:04
vhosakotsbezverk: Is kubectl yum package broken?  I see it too.23:04
sbezverkvhosakot: yeah, we need to disable gpgcheck on that repo23:05
sbezverkthey forgot to sign it23:05
sbezverkbut ubuntu works fine23:05
vhosakotsbezverk: can you send the link for the gate please, I just need to compare versions.23:05
sbezverkkfox1111: addressing placement-api to be able to use ocata images23:06
vhosakotsbezverk: I was not able to find the gate link in IRC logs, sorry.23:06
kfox1111sbezverk: cool. thanks.23:06
sbezverkvhosakot: not sure what you mean23:06
kfox1111sbezverk: is there an ocata set of containers on the hub yet?23:06
vhosakotsbezverk: you sent a link to the gate logs I remember.23:06
sbezverkkfox1111: not yet, I was told in 2-3 days23:06
sbezverkvhosakot: ah, I am not sure I have it handy23:07
vhosakotsbezverk: cool, np, thanks for the info.23:07
sbezverkkfox1111: I am planning to apply jeffrey4l patch right after kolla-ansible gets cloned to re-active placement, but only for version 3 images for now23:08
kfox1111k.23:10
*** salv-orlando has quit IRC23:18
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442623:23
spsurya__morning all23:29
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442623:30
*** sayantan_ has quit IRC23:30
*** saneax-_-|AFK is now known as saneax23:30
*** schwicht has joined #openstack-kolla23:31
*** MasterOfBugs has quit IRC23:34
vhosakotv1k0d3n: portdirect: I think we need to change https://github.com/att-comdev/halcyon-kubernetes/blob/master/kube-deploy/roles/deploy-kube/tasks/centos.yml#L23 to "gpgcheck: no" to resolve the broken gpg error "Package ed1cdb328f2ee2cd2c9a200f3a3ae549801cd51216ef82e37d2b92acdbd72cf5-kubectl-1.5.4-0.x86_64.rpm is not signed" we see when the "install kubeadm and required prereqs" is run.23:34
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442623:36
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442623:43
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Fix oslo_debug_helper not running  https://review.openstack.org/44454223:44
*** rwallner has joined #openstack-kolla23:44
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Fix oslo_debug_helper not running  https://review.openstack.org/44454223:45
*** kelv has quit IRC23:46
openstackgerritSerguei Bezverkhi proposed openstack/kolla-kubernetes master: WIP Adding nova placement api to kolla kubernetes  https://review.openstack.org/44442623:47
*** rwallner has quit IRC23:48
*** ssurana has quit IRC23:53
*** ssurana has joined #openstack-kolla23:53
*** vhosakot has quit IRC23:57
*** schwicht has quit IRC23:57
*** ssurana has quit IRC23:58
*** schwicht has joined #openstack-kolla23:58
*** schwicht has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!