Wednesday, 2017-08-16

*** yingjun has joined #openstack-kolla00:03
*** lucasxu has joined #openstack-kolla00:24
*** schwicht has quit IRC00:26
*** magicboiz has quit IRC00:32
*** dave-mccowan has quit IRC00:38
*** stefan_bo has joined #openstack-kolla00:38
*** awidders_ has joined #openstack-kolla00:42
*** awiddersheim has quit IRC00:42
*** duonghq has joined #openstack-kolla00:49
*** vhosakot has quit IRC00:50
*** aagate has joined #openstack-kolla00:50
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' to necessary tasks (general roles)  https://review.openstack.org/39868200:54
*** lucasxu has quit IRC00:54
*** zhurong has joined #openstack-kolla00:58
*** dixiaoli has joined #openstack-kolla01:01
*** dixiaoli_ has joined #openstack-kolla01:02
*** dixiaoli has quit IRC01:02
*** tovin07_ has joined #openstack-kolla01:03
*** aagate has quit IRC01:06
openstackgerritjimmygc proposed openstack/kolla-ansible master: Add vmware support to ceilometer  https://review.openstack.org/46284201:07
*** stefan_bo has quit IRC01:09
*** lucasxu has joined #openstack-kolla01:11
*** lucasxu has quit IRC01:13
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only necessary tasks (default roles)  https://review.openstack.org/39868401:13
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Specify 'become' for only neccesary tasks (all other roles)  https://review.openstack.org/39868501:13
*** cocakolla has joined #openstack-kolla01:14
*** Pavo has quit IRC01:16
*** StephenWang1991 has joined #openstack-kolla01:24
*** stefan_bo has joined #openstack-kolla01:25
*** StephenW_ has joined #openstack-kolla01:28
*** StephenWang1991 has quit IRC01:28
*** dave-mccowan has joined #openstack-kolla01:28
*** iniazi has joined #openstack-kolla01:34
*** StephenW_ has quit IRC01:36
*** iniazi_ has quit IRC01:36
*** manheim has joined #openstack-kolla01:37
*** dave-mcc_ has joined #openstack-kolla01:37
*** dave-mccowan has quit IRC01:39
*** StephenWang1991 has joined #openstack-kolla01:40
*** manheim has quit IRC01:41
*** StephenW_ has joined #openstack-kolla01:46
*** StephenWang1991 has quit IRC01:46
duonghqsdake, how much does Elrog cost you?02:06
sdakethose were warranty02:08
sdakeso free02:08
sdakethe first pair i bought failed and the company was restructured and wiped out their liability02:08
sdake1600 poofed in about 500 hours of listening02:08
sdakethe second pair i bought poofed in 200 hours of listening with a 1 year warranty02:08
*** trozet has quit IRC02:08
sdakeirritating - wont be buying this brand again02:09
sdakeeven though they are very impressive02:09
sdake845 tubes should last 40k hours or more02:09
duonghqyeah, <1k hr is short lifespan02:09
duonghqtoo short02:09
sdakecartridges for tonearms last 1.5k hours - i can tolerate that02:10
sdakea tube that should last 10-40k hours should last 10-40k hours02:10
sdakenot poof in 200 hours02:10
*** trozet has joined #openstack-kolla02:10
sdakeatleast this time the factory redesigned them02:10
sdakelove beta testing other peoples products02:10
duonghqdoes it sound different?02:11
sdakenext time its http://psvanetube.com/wordpress/store/products/psvane-we845-western-electric-replica-tubes-pair/02:11
sdakeyes - every tube sounds different02:11
sdakebetter is subjective02:11
sdakethese 845s ahve a ton of power02:12
sdakethe psvane - had a pair of those - and back to back failures on those as well02:12
sdakeatleast its a  western electric copy02:12
sdakeWE invented tubes and knew how to make em02:12
sdakethese psvane cats took them apart and completley replicated them02:12
duonghqhmm, interesting02:13
sdakethe psvane 845s dont have as smuch power02:13
sdakebut theoretically since they are a copy of we, they should have 40k hour lifespans02:13
sdakehave to totally rebias my amp now though - which is annoing02:14
sdakethat takes hours02:14
sdakemostly to find my tools :)02:14
duonghqnice02:14
duonghqyour amp is kind of DIY or from factory?02:14
*** trozet has quit IRC02:17
*** unicell has quit IRC02:19
*** trozet has joined #openstack-kolla02:20
openstackgerritshaofeng cheng proposed openstack/kolla-ansible master: Add pull reconfigure and upgrade commands  https://review.openstack.org/49406002:35
*** trozet has quit IRC02:38
sdakefrom factory02:39
sdakeduonghq http://www.toneimports.com/lmaudio/219ia.html02:42
*** unicell has joined #openstack-kolla02:42
*** krtaylor has joined #openstack-kolla02:43
inc0I need new headphones02:43
inc0but I don't want to spend more than 100$02:43
inc0so that's gonna be fun02:43
sdakegrado possibly02:45
*** unicell has quit IRC02:46
inc0interesting I never had one of these02:48
*** zhangfei has joined #openstack-kolla02:53
jamesbenson_[James Benson, jamesbenson] I had good success with Amazon basics actually....02:59
*** eaguilar has quit IRC03:00
duonghqI always trust Sony STH-30 for entry level audio stuff,03:03
duonghqsdake, your amp uses 4 pairs of tubes?03:03
sdakeright03:03
spsuryamorning all03:04
duonghqand only power tubes poofed?03:04
sdakeonly 1 845 poofed03:04
sdakedude said it was a microcrack in teh glass03:04
spsuryasdake: duonghq inc0 morning03:04
duonghqspsurya, morning03:04
sdakethe tubes had a design defect that they were running 50c hotter than desired03:04
sdakeand this caused the glass to break03:04
sdakeand that of course is doom for a tube :)03:05
sdakesup spsurya03:05
duonghqnot so good QA and design03:05
spsuryasdake: how are you days ?03:05
spsuryahave you recovered ?03:06
sdakerecovered from what?03:06
inc0I always wanted these.. https://www.amazon.com/Sennheiser-HD-598-Over-Ear-Headphones/dp/B0042A8CW203:06
spsuryasome shit happened to few weeks back03:06
spsuryayou told03:06
sdakeya - fell - fully recovered03:06
spsuryanice03:06
duonghqinc0, once it was my dream when I younger :P03:06
sdakejust damaged my arm a bit - which heals thankfully03:06
spsuryasdake: good to hear that03:07
*** cocakolla has quit IRC03:07
duonghqback in time, sennheiser is most famous brand in Vietnam03:07
inc0good headphones, really, I just think it's gonna be a waste as work phones go03:07
inc0really? interesting03:07
inc0I mean they're great03:07
sdakei had some sennheiser cans03:08
sdakeand lost them in an airport03:08
sdakethey were pretty solid - although b&w better for travel imo03:09
sdakethe winning thing about b&w is replacceable cable and they fold up03:09
inc0I'm looking for office phones03:09
duonghqthere was some myth those headphone (HD5xx) need some kind of amps for driver that well? not sure. But after had HD6xx on my ear is headphone festival, I forgot HD5xx03:09
inc0hehe03:09
inc0I might just go to store somewhere and try few of them03:09
*** stefan_bo has quit IRC03:10
duonghqthe most comfortable phones I have had is STH-30, very unique design (except Apple one) but should good in its price range (for my ear, of course)03:10
duonghqI have 1-2 pair for backup in my home :P03:11
duonghq*at home03:11
sdakemy office cans are these: https://www.audeze.com/products/lcd-collection/lcd-403:11
duonghqdo you plug it in any kind of amp?03:12
sdakethey dont travel well sadly03:12
*** dklyle has quit IRC03:12
sdakeya my headphone amp is : https://oppodigital.com/headphone-amplifier-ha-1/03:12
inc04k$ fuck03:12
duonghqAudeze makes very exciting headphone, but should be paired with good amps, this is sadly03:12
sdakeya audeze does really well with that oppo amp03:13
duonghqinteresting03:13
sdakelcd4 is an amazing experience03:14
sdakei wish my stereo sounded as good as the lcd4s do :)03:14
duonghqbut it is always make my ear sweat03:14
inc0at home I have Audiotechnica ATH M50X and I'm quite happy with them03:14
duonghqinc0, so I think Grado will make you surprise or you will totally dislike it (if you try)03:15
inc0they're bright usually from what I read?03:15
sdakegrado?03:16
inc0I might actually like it, I listen to lot of violin and such sounds03:16
sdakemy wife has a pair - not impresse03:16
sdaked03:16
sdakebut they are in the under 100 range :)03:16
duonghqya, this is kind of general signature of Grado, bright and good speed03:16
inc0right03:16
sdakeyes very fast03:16
sdakegood for digital listening03:17
inc0Sennheiser are quite bassy03:17
duonghqbut the decay is too fast if you like piano or even viola,...03:17
sdakeduonghq sort of depends on your source i think03:17
sdakeeverything does better with a headphone amp (vs the junk in your phone)03:17
duonghqah, sure,03:17
inc0and a proper DAC03:18
inc0I was also looking at buying a DAC for home03:18
sdakeI  *highly* recommend this for traveling: https://oppodigital.com/headphone-amplifier-ha-2SE/03:18
sdakeactually I have lasts years model with a different dac, but that model should be better03:18
duonghqmy friend has RS-1i (quite old one), and its sound is very good, easy for pairing with amp. She told me that newer RS1i do not good as her.03:18
sdakeinc0 oppo puts dac and amp in one package03:18
*** cocakolla has joined #openstack-kolla03:19
duonghqiirc, oppo is quite new player in the field03:19
*** goldyfruit has quit IRC03:20
sdakemy audio hobby is sort of ludicrous :)03:20
sdakebut whatever03:20
sdakei really like oppo's designs03:20
duonghqgot it03:20
sdaketheir headphone standalone amp could really use dual mono power supplies03:20
sdakeother than that, its almost perfect03:20
*** dave-mcc_ has quit IRC03:21
duonghqaudio-gd is also good manufactor from China, (for desktop setup)03:22
inc0dac for android phones? interesting03:22
*** iniazi has quit IRC03:22
sdakeduonghq nice layout03:24
sdakeduonghq wonder how it sounds :)03:24
*** sbezverk has joined #openstack-kolla03:24
sdakewow only 33003:25
sdakepretty affordable03:25
sdakegood feature set - good sabre dac03:25
inc0hmm03:25
inc0yeah this audio-gd looks interesting03:25
duonghqsdake, nice at its price range, quite detail, powerful but not overwhelming amp (when pair with LCD-2). In general, I don't expect more from the price03:26
sdakechinese hi-fi is making a comeback ;)03:26
inc0http://www.audio-gd.com/Pro/Headphoneamp/NFB1AMP/NFB1AMPEN.htm but their website looks 90's ;)03:27
duonghqyah,03:28
inc0sdake: did you find US distributor for them?03:28
*** yangyapeng has quit IRC03:28
duonghqinc0,  can you do some review?03:29
inc0shoot duonghq03:29
duonghqinc0, https://review.openstack.org/#/c/425446/ -> this keystone upgrade one, and the ansible-become one: https://review.openstack.org/#/c/398682/03:30
sdakeinc0 - here is the dac for you bro: https://youtu.be/CdQcLqvKu-I03:31
*** david-lyle has joined #openstack-kolla03:31
*** yangyapeng has joined #openstack-kolla03:32
inc0hmm, I'm worried about amount of red gates lately:/03:32
duonghqyeah, I'm testing a few more time, but egonzalez told me it's ok from his box,03:33
duonghqIn my box, even normal upgrade fails a few times03:33
inc0I think ceph gates runs out of space03:34
inc0:S03:34
inc0plus binary distros fail03:35
inc0damn03:35
duonghqI tried read back the log03:43
duonghqbut don't have any idea03:43
duonghqhttp://logs.openstack.org/46/425446/26/check/gate-kolla-ansible-dsvm-deploy-multinode-ubuntu-binary-ubuntu-xenial-2-node-nv/f2b4b28/logs/kolla/nova/nova-api.log03:45
openstackgerritMerged openstack/kolla-ansible master: Support customizing grafana home dashboard  https://review.openstack.org/48593103:52
*** david-lyle has quit IRC04:06
*** david-lyle has joined #openstack-kolla04:08
*** iniazi has joined #openstack-kolla04:11
*** yangyapeng has quit IRC04:15
*** dklyle has joined #openstack-kolla04:15
*** david-lyle has quit IRC04:18
sdakeinc0 i don't see a us distributor for audio-gd - they had price on their website in usd - they probably ship direct04:57
*** janki has joined #openstack-kolla05:03
openstackgerritMerged openstack/kolla-ansible master: Standardize Fluentd grammar usage  https://review.openstack.org/49283205:26
*** mdnadeem has joined #openstack-kolla05:27
*** bmace has quit IRC05:30
*** bmace has joined #openstack-kolla05:31
*** zhangfei has quit IRC05:31
*** stefan_bo has joined #openstack-kolla05:32
*** mdnadeem has quit IRC05:37
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Clear all l3 related namespace before starting neutron-l3-agent  https://review.openstack.org/49407805:43
*** pcaruana has joined #openstack-kolla05:45
*** zhangfei has joined #openstack-kolla05:45
*** mdnadeem has joined #openstack-kolla05:49
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: Bump OpenStack service version  https://review.openstack.org/49407906:00
*** jbadiapa has joined #openstack-kolla06:04
*** stefan_bo has quit IRC06:07
*** stefan_bo has joined #openstack-kolla06:09
*** magicboiz has joined #openstack-kolla06:10
*** skramaja has joined #openstack-kolla06:12
*** magicboiz has quit IRC06:15
*** magicboiz has joined #openstack-kolla06:15
*** zhurong has quit IRC06:19
*** zhurong has joined #openstack-kolla06:22
*** stefan_bo has quit IRC06:22
*** dciabrin has joined #openstack-kolla06:25
*** stefan_bo has joined #openstack-kolla06:27
openstackgerritJavier Castillo proposed openstack/kolla-ansible master: Fix designate interface selection  https://review.openstack.org/48832906:28
*** magicboiz has quit IRC06:31
*** manheim has joined #openstack-kolla06:34
*** manheim has quit IRC06:38
*** jamesbenson has joined #openstack-kolla06:47
*** magicboiz has joined #openstack-kolla06:48
*** jamesbenson has quit IRC06:51
*** magicboiz has quit IRC06:52
*** magicboiz has joined #openstack-kolla06:53
*** zhurong has quit IRC06:53
*** coolsvap has joined #openstack-kolla06:54
*** shardy_afk is now known as shardy07:10
*** unicell has joined #openstack-kolla07:12
*** stefan_bo has quit IRC07:12
*** masuberu has joined #openstack-kolla07:14
*** masber has quit IRC07:16
*** mgoddard has joined #openstack-kolla07:17
*** masuberu has quit IRC07:18
*** unicell has quit IRC07:18
*** stefan_bo has joined #openstack-kolla07:18
*** masuberu has joined #openstack-kolla07:28
*** manheim has joined #openstack-kolla07:30
*** masuberu has quit IRC07:31
*** shardy is now known as shardy_afk07:32
*** reidrac has joined #openstack-kolla07:35
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Add ceph log in fluentd  https://review.openstack.org/49409907:37
*** matrohon has joined #openstack-kolla07:37
*** egonzalez has joined #openstack-kolla07:41
*** stefan_bo has quit IRC07:42
openstackgerritMerged openstack/kolla master: libvirt should be installed from OS rather than pip  https://review.openstack.org/48944207:43
*** stefan_bo has joined #openstack-kolla07:46
*** guoshan has joined #openstack-kolla07:47
*** guoshan_ has joined #openstack-kolla07:50
*** guoshan has quit IRC07:53
*** ducttap__ has quit IRC08:00
openstackgerritjimmygc proposed openstack/kolla-ansible master: Disable grafana Getting Started panel  https://review.openstack.org/48593208:06
openstackgerritjimmygc proposed openstack/kolla-ansible master: Disable grafana Getting Started panel  https://review.openstack.org/48593208:07
*** bjolo has joined #openstack-kolla08:12
bjolomorning08:13
bjoloback from vacation :)08:13
manheimhello08:13
*** ducttape_ has joined #openstack-kolla08:14
*** mgoddard has quit IRC08:17
*** shardy_afk is now known as shardy08:17
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Use kolla_internal_vip_address for glance_api servers  https://review.openstack.org/49411008:21
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Add panko log in fluentd  https://review.openstack.org/48984508:22
*** gfidente has joined #openstack-kolla08:23
*** gfidente has quit IRC08:23
*** gfidente has joined #openstack-kolla08:23
*** stefan_bo has quit IRC08:26
openstackgerritMerged openstack/kolla-ansible master: Fix wrong condition for iscsid container  https://review.openstack.org/49197908:29
*** dixiaoli has joined #openstack-kolla08:34
*** stefan_bo has joined #openstack-kolla08:35
*** dixiaoli has quit IRC08:36
*** dixiaoli_ has quit IRC08:38
*** jbadiapa_ has joined #openstack-kolla08:50
*** jbadiapa has quit IRC08:51
*** serlex has joined #openstack-kolla08:55
duonghqmorning bjolo08:56
*** manheim has quit IRC08:57
*** sambetts|afk is now known as sambetts08:58
*** manheim has joined #openstack-kolla08:58
openstackgerritJeffrey Zhang proposed openstack/kolla stable/ocata: libvirt should be installed from OS rather than pip  https://review.openstack.org/49413809:02
*** stefan_bo has quit IRC09:12
*** dixiaoli has joined #openstack-kolla09:14
*** zhangfei has quit IRC09:22
*** zhangfei has joined #openstack-kolla09:35
*** yingjun has quit IRC09:38
openstackgerritMerged openstack/kolla-ansible stable/ocata: Magnum: update clients config groups  https://review.openstack.org/48693709:40
*** dixiaoli has quit IRC09:45
*** dixiaoli has joined #openstack-kolla09:48
openstackgerritMerged openstack/kolla-ansible master: Change cleaning_network_uuid to cleaning_network  https://review.openstack.org/49247909:53
*** tovin07_ has quit IRC09:57
openstackgerritjimmygc proposed openstack/kolla-ansible master: Disable grafana Getting Started panel  https://review.openstack.org/48593209:57
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement tacker-conductor service  https://review.openstack.org/47751709:59
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Implement tacker-conductor image, rework all tacker  https://review.openstack.org/47749409:59
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Add ceph log in fluentd  https://review.openstack.org/49409909:59
*** duonghq has quit IRC10:00
*** jamesbenson has joined #openstack-kolla10:01
*** dixiaoli has quit IRC10:03
*** jamesbenson has quit IRC10:06
*** StephenW_ has quit IRC10:12
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix mariadb log not match  https://review.openstack.org/49071210:16
*** lpetrut has joined #openstack-kolla10:17
*** guoshan_ has left #openstack-kolla10:20
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Fix mistral coordination  https://review.openstack.org/47760010:30
*** jbadiapa_ has quit IRC10:30
*** jbadiapa_ has joined #openstack-kolla10:31
*** kevinz has quit IRC10:32
*** mgoddard has joined #openstack-kolla10:38
openstackgerritMerged openstack/kolla master: Remove networking-sfc in n-server-opendaylight  https://review.openstack.org/49379610:38
*** zhangfei has quit IRC10:52
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add tacker sfc contrib demo  https://review.openstack.org/49385611:03
*** reidrac has quit IRC11:08
*** schwicht has joined #openstack-kolla11:21
*** jbadiapa_ has quit IRC11:27
*** iniazi_ has joined #openstack-kolla11:30
*** iniazi has quit IRC11:33
*** jbadiapa_ has joined #openstack-kolla11:42
*** rhallisey has joined #openstack-kolla11:43
*** schwicht has quit IRC11:45
openstackgerritMerged openstack/kolla-ansible master: Add ironic log in fluentd  https://review.openstack.org/49151411:48
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement tacker-conductor service  https://review.openstack.org/47751711:50
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement tacker-conductor service  https://review.openstack.org/47751711:52
*** StephenWang1991 has joined #openstack-kolla12:01
*** markhargrove28 has joined #openstack-kolla12:03
*** StephenWang1991 has quit IRC12:05
*** awiddersheim has joined #openstack-kolla12:15
*** eaguilar has joined #openstack-kolla12:17
*** awidders_ has quit IRC12:19
*** kbaegis has joined #openstack-kolla12:21
*** StephenWang1991 has joined #openstack-kolla12:23
*** reidrac has joined #openstack-kolla12:26
sdakemorning folks12:27
pbourkehey sdake12:29
*** skramaja has quit IRC12:33
rwellumMorning sdake pbourke12:36
rwellumI have an odd issue with my kolla-k8s. I am running 4.0.0. When I try to create a vm through openstack CLI, it seems to be kicked off but nova list doesn't show anything. If I got to Horizon, and do it through the UI - it works just fine.12:37
*** calbers has joined #openstack-kolla12:38
sdakehttps://www.theverge.com/2017/6/28/15887698/eero-second-generation-mesh-router-review12:40
sdakeyou would think for a 500 dollar mesh network they would do something a little more compelling then a 3rd radio as far as upgrades go12:40
sdakehey pbourke12:40
sdakehe rwellum12:40
sdakerwellum 5am here - let me process your question for a bit12:41
sdakerather 5:4112:41
sdakemy immediate give up answer is I don't know12:41
sdakebut I suspect we could find the correct answer once my brain has booted12:41
sdakealso, for those with wireless netowrks, eero is the absolute way to go12:42
sdakealthough i am not keen on the subscription model12:43
sdakei have so many "9.99/mo subscriptions" I could scream12:43
sdakethe sad thing is all of them but a couple are essential :(12:44
sdakemy porsche car connect is like 1k a year12:44
sdake(that tells me where my porsche is, adds 24/7 monitored security, etc)12:45
*** kiennt has joined #openstack-kolla12:45
sdakesadly softwre subscriptoins are the way the world has headed12:45
sdakemore or less pioneered by red hat12:46
sdakerwellum ok - now that i'm angry12:46
sdakebrain working better now :)12:46
sdake(channel the anger:)12:46
sdakekicked off how?12:46
*** StephenWang1991 has quit IRC12:47
sdakewhen i was working on kolla-k8s in the past, nova boot always worked with 4.0.0 images12:47
*** s-dean has quit IRC12:47
egonzalezrwellum, may be the case the instance is in a diff project? may try with nova list --all-tenants12:48
sdakeegonzalez good idea dude-  thanks :)12:48
sdakeegonzalez typically our operation of kolla-k8s runs with teh admin tenant12:49
rwellumegonzalez - no luck - just shows the original VM I created that subsequently failed. https://www.irccloud.com/pastebin/9oZ6SChQ/12:52
rwellumI wonder if this is blocking my other 'CLI' VM's. But I've dived through nova-conductor and nova-scheduler logs and I see nothing suspicious.12:52
rwellumsdake: I considered eero for a while, but seem to be getting by with a high end router and a 1G service - at least no kids complaining.12:55
egonzalezfunny thing for tacker, in ocata was just tacker+heat, in pike is tacker+heat+mistral+barbican and dependencies of all those services12:55
*** kbaegis has quit IRC12:56
rwellumegonzalez: I have a different issue in Pike :). There I get the dreaded 'no Nova hosts available'. I think my cell setup is still wrong.12:56
egonzalezrwellum, are nova-compute binaries in nova service-list?12:57
*** kevzha01 has joined #openstack-kolla12:58
rwellumegonzalez - looks good to me https://www.irccloud.com/pastebin/Oy3ktNwS/12:58
rwellumThis is bare-metal btw12:58
egonzalezrwellum, in pike if services appear there means cell setup is OK, service list polls cells looking for services13:00
rwellumegonzalez - this is an example of how I create the vm and how nova list doesn't see it. https://www.irccloud.com/pastebin/pINtmtwN/13:00
rwellumSorry egonzalez - was still talking about the Ocata issue13:00
rwellumGotta fix one thing at a time..13:01
*** trozet has joined #openstack-kolla13:02
rwellumAnything obviously wrong in how I create the VM? This works sometimes, just not consistently.13:02
egonzalezcommand looks good, if was not correctly will raise an error, weird13:03
rwellumSee how there is no Network?13:03
rwellumFor the failed one.13:03
egonzalezno error in logs?13:04
*** goldyfruit has joined #openstack-kolla13:04
egonzalezserver create is just an rest api 200, does not means resources are actually created13:05
rwellumI couldn't see anything obvious. I even did a tail -f on nova-scheduler and nova-conductor logs on the Containers and executed the command.13:06
sdakerwellum can you paste the nova-api log13:06
egonzalezduring building steps will gather images, create a port in neutron (same 200 API) in this step will show an IP address and then bind a port to the instance13:06
*** schwicht has joined #openstack-kolla13:06
rwellumsdake: sure - will dig them out13:07
egonzalezmost of the times fails in latest steps, while nova-compute and ovs try to really bind the port into br-int13:07
rwellumegonzalez: bear in mind it works through the UI tho13:07
sdakerwellum with these things we need to trace throug hthe entire call chain of nova13:07
sdakewhich - is painful13:07
rwellumsdake: learning that yeah.13:07
sdakerwellum also can you run kubectl get pods -n kolla -> paste13:08
sdakein a worst case scenario, nova should keep a record of a failure to schedule the vm13:08
sdakeI guess  Ineed to find two KNC miner cubes now13:10
* sdake groans13:10
egonzalezrwellum, weird UI works and CLI not, may be the case the LB balance to one failing API server in CLI and in UI balance you to a working one?13:14
*** eaguilar_ has joined #openstack-kolla13:15
*** eaguilar has quit IRC13:17
rwellumWell I see plenty of errors in nova-api... Can you guys see this file: https://docs.google.com/document/d/1oAQtgjrTgYvDhMdFLcL_BrMGy1tV65KL_wVjb0Nm-A4/edit?usp=sharing13:17
rwellumsdake:  https://www.irccloud.com/pastebin/5sL13pQy/13:18
*** stefan_bo has joined #openstack-kolla13:19
egonzalezrwellum, database errors13:20
egonzalezrwellum, columns is complaining about should be created in db sync13:20
rwellumAnd I see this: "2017-08-15 22:25:31.279 22 ERROR nova.api.openstack.extensions ProgrammingError: (pymysql.err.ProgrammingError) (1146, u"Table 'nova_cell0.instance_extra' doesn't exist")"13:21
rwellumwth is going on? :)13:21
sdakeok -well that is a problm13:22
sdakeare you using the microchart approach?13:22
sdakeI never could get the 4.0.0 images to work well without a patch like this: https://review.openstack.org/#/c/473588/13:24
*** stefan_bo has quit IRC13:24
sdakeegonzalez thats a pretty good insight - although the way kolla-kubernetes is structured, IIRC everything is always loadbalanced13:25
sdakealthough horizon may be configured to use source routing13:25
*** janki has quit IRC13:28
*** janki has joined #openstack-kolla13:28
sdakerwellum the key problem is the synchronization of the cell databsae creation and the nova vm creation13:29
sdakeentrypoint doesnt get the job done13:29
sdakethe gate doesn't "launch everything all at once" - instead it synchronizes the launch13:29
sdakeso it really doesn't test entrypoint13:29
sdakeor minimally tests it13:29
*** dave-mccowan has joined #openstack-kolla13:30
*** eaguilar_ has quit IRC13:34
*** jtriley has joined #openstack-kolla13:34
*** StephenWang1991 has joined #openstack-kolla13:37
*** schwicht has quit IRC13:40
rwellumsdake: yes I am following the deployment guide still - so nothing funky13:43
rwellumsdake: the above isn't a patch - it's a whole playbook.13:44
rwellum:)13:44
*** eaguilar has joined #openstack-kolla13:45
sdakerwellum the deployment guide operates with a human - so human runs a command, then runs another command etc13:45
sdakethere are sync points int he human running the command13:45
sdake(atleast  there was in teh past) around the creation of the nova cells service13:46
sdakenot sure where it is at presently13:46
sdaketime to get kids to school bbl13:46
*** s-dean has joined #openstack-kolla13:48
*** s-dean has quit IRC13:49
*** s-dean has joined #openstack-kolla13:49
rwellumegonzalez: regarding your comment, 'columns is complaining about should be created in db sync' - - anything obvious I can try/look at?13:53
egonzalezrwellum, yep, check what nova databases have, check database connection url in nova.conf and paste please. maybe cell db sync was in the wrong database13:55
*** eaguilar_ has joined #openstack-kolla13:56
egonzalezrwellum, also can try just with a simple db sync in nova-api container/ nova-manage db sync , nova-manage api_db sync and nova-manage db online_data_migrations13:56
*** aagate has joined #openstack-kolla13:56
*** eaguilar has quit IRC13:56
egonzalezrwellum, if cell0 is created, that will populate schema into nova_cell0 db13:57
*** schwicht has joined #openstack-kolla13:57
rwellumThanks egonzalez  - will brb after meeting.13:58
egonzalezrwellum, check first  nova-manage cells_v2 list-cells --verbose, there should be 2 cells (cell0 and None)13:58
rwellumegonzalez - my nova.conf fyi https://www.irccloud.com/pastebin/EpVKyZ3A/13:59
*** kbaegis has joined #openstack-kolla13:59
*** bmace has quit IRC13:59
rwellumhttps://www.irccloud.com/pastebin/wsnLqOhR/13:59
egonzalezrwellum, cells are created right, something went wrong during db sync14:02
*** kevzha01 has quit IRC14:03
*** yangyapeng has joined #openstack-kolla14:03
*** eaguilar_ has quit IRC14:04
*** eaguilar has joined #openstack-kolla14:05
*** kiennt has quit IRC14:06
*** schwicht has quit IRC14:08
sdakeinc0 when your alive ping me plz14:08
sdakeegonzalez the dbsync is out of order imo14:09
sdakeegonzalez the only way i could ever get it to work properly with 4.0.0 was to orderize it with ansible14:09
*** schwicht has joined #openstack-kolla14:10
*** jamesbenson has joined #openstack-kolla14:18
*** janki has quit IRC14:22
*** ntpttr_laptop has joined #openstack-kolla14:24
*** zhubingbing_ has quit IRC14:25
*** lucasxu has joined #openstack-kolla14:25
*** mdnadeem has quit IRC14:26
*** schwicht has quit IRC14:30
*** schwicht has joined #openstack-kolla14:35
*** kiennt has joined #openstack-kolla14:39
*** StephenWang1991 has quit IRC14:43
*** iniazi_ is now known as iniazi14:43
*** StephenWang1991 has joined #openstack-kolla14:44
*** iniazi is now known as iniaziaway14:46
*** iniaziaway is now known as iniazi14:47
*** tovin07_ has joined #openstack-kolla14:52
*** Jezekus has joined #openstack-kolla14:56
JezekusHello we have run into a trouble with kolla-toolbox container, it keeps restarting and complains about missing uid 42401. Does anyone have a clue?14:57
*** jamesbenson has quit IRC14:57
rwellumsdake: I can get it working 99% of the time on Ubuntu VM, 100% on Centos. But on Bare-Metal seeing this everytime.14:59
*** jamesbenson has joined #openstack-kolla14:59
*** gkadam has joined #openstack-kolla15:01
egonzalezfirst time deployed skydive, awesome service to have15:01
sean-k-mooneyegonzalez: is that the network visulisation service15:02
egonzalezsean-k-mooney, yep, can see everything, taps, namespaces, bridges, packet flows, etc15:03
*** jamesbenson has quit IRC15:03
sean-k-mooneyegonzalez: cool ill have to give it a try sometime. i wonder if i can make it explode with dpdk lol15:03
rwellumegonzalez: did the database connection url in nova.conf that I pasted above seem ok?15:04
egonzalezsean-k-mooney, sample http://imgur.com/JixcIJv15:05
rwellumegonzalez: Also not sure what you mean by 'also can try just with a simple db sync in nova-api container/ nova-manage db sync , nova-manage api_db sync and nova-manage db online_data_migrations'15:05
*** jamesbenson has joined #openstack-kolla15:05
sean-k-mooneyegonzalez: thats pretty cool. it would be awsome if that was intergrated in horizon as an admin network topology view akin to the tenant one15:06
cocakollaHello everyone. I've been following the kolla-kubernetes (https://docs.openstack.org/developer/kolla-kubernetes/deployment-guide.html). So far I've gotten past all of the issues. But I'm stumped now. I'm trying to spin up the mariadb containers with step "helm install --debug kolla-kubernetes/helm/service/mariadb --namespace kolla --name mariadb --values ./cloud.yaml"15:06
cocakollamy container is stuck in "Entrypoint WARNING: 2017/08/16 15:04:03 entrypoint.go:81: Resolving dependency for mariadb-init-element failed: Job mariadb-init-element is not completed yet"15:07
egonzalezcocakolla, awesome nick lol15:07
cocakollathanks! :)15:08
egonzalezrwellum, execute those commands in nova_api container to make nova populate tables again15:08
rwellumegonzalez: ah ok 1s15:09
*** duonghq has joined #openstack-kolla15:09
sdakerwellum bare metal is much faster then vm15:09
duonghqevening guys15:09
egonzalezrwellum, yep, nova.conf looks good, what are the command used to create cells? cannot find how is done in kolla-kubernetes15:09
sdakelater duonghq15:09
rwellumsdake: but before I create I make sure kolla is fully up and happy.15:10
rwellumegonzalez: just configured in the cloud.yaml15:10
sdakecocakolla - after every deploy - you need to delete the PV contents on disk - mariadb cannot init over itself15:10
sean-k-mooneyegonzalez: regarding the networking-guide when you have time. im ok with spliting it into sub guides but do you want to do that now or can it wait till early queens?15:10
sdakecocakolla - and grats dude, you know how to say kolla properly :)15:11
sdakeeveryone always says "KAWLA"15:11
sdakeits "COLA" :)15:11
krtaylor+100015:11
sean-k-mooneysdake: really i have never heared anyone say "kawla"15:12
sdakei hear it all hte time15:12
rwellumegonzalez https://www.irccloud.com/pastebin/No83EJvb/15:12
sean-k-mooneyi guess the log does not help but again i refer to that as the kolla kawla15:12
sdakeegonzalez its a little complicated - how many hours do you have to understand cell creation in kolla-kubernetes :)15:13
*** awiddersheim has quit IRC15:13
*** awiddersheim has joined #openstack-kolla15:13
cocakollasdake - This is a new deploy. Would "helm delete mariadb --purge" not delete the PV contents on disk?15:13
sdakecocakolla it does not purge the contents of the pv - no15:14
sdakecocakolla ls -l /var/lib/kolla/volumes15:14
sdake-> paste15:14
cocakolladrwxr-xr-x. 5 42434 42434 4096 Aug 16 15:11 mariadb15:15
sdakelist out mariadb15:15
sdakewhich platform btw?15:15
cocakollacentos 7.315:16
sdakerun getenforce15:16
cocakollamariadb has a lot of files15:16
cocakollaselinux is set to permissive15:16
sdakepurge all helm charts - rm -rf /var/lib/kolla/volumes/*15:16
sdakeand try again plz - only mariadb15:16
cocakollashould I run a "helm delete mariadb --purge" first?15:17
sdakeyup15:17
cocakollaok15:17
sdakerun kubectl get pods -n kolla -> paste plz15:17
egonzalezsean-k-mooney, I dont mind doing it now or later. But we need to do it in some day. networking-guide is becoming in a big mash of services15:17
sean-k-mooneysdake: im not sure rm -rf /var/lib/kolla/volumes/* is a good idea15:17
sean-k-mooneyit would be better to do that via docker15:18
sdakesean-k-mooney this is kubernetes - i dont think kubernetes uses docker volumes15:18
cocakollasdake - I just ran mariadb --purge15:18
sdakeit has a pv constract15:18
sdakecocakolla cool15:18
cocakollakubectl get pods -n kolla15:18
cocakollaNo resources found.15:18
sdakenice15:18
*** schwicht has quit IRC15:18
sdakecocakolla systemctl status firewalld15:18
sean-k-mooneysdake: oh soory yes i missread the path i assumed that was /var/lib/docker/volumes not /var/lib/kolla/volumes15:19
cocakollafirewalld: Active: inactive (dead)15:19
rwellumegonzalez:  https://www.irccloud.com/pastebin/y0u1rJXc/15:19
inc0good morning15:20
*** Jezekus has quit IRC15:20
sean-k-mooneyegonzalez: i dont mind takeing a go at spliting it out i would just prefer to get the inital dpdk docs megable this week and then im happy to start spliting it out after that15:20
sdakecocakolla cool15:20
*** awidders_ has joined #openstack-kolla15:20
sdakecocakolla can you paste your globals.yml plz15:21
sean-k-mooneyegonzalez: are there examples of where we have nested the docs already15:21
sdakerather cloud.yaml15:21
sdakesup inc015:21
sean-k-mooneyinc0: o/15:21
rwelluminc0: o/15:22
egonzalezrwellum, worked? will need to do ``nova-manage cell_v2 discover_hosts`` too15:22
cocakollaglobals.yml15:22
sdakecocakolla sudo yum install fpaste15:22
sdakefpaste globals.yml15:22
egonzalezrwellum, this will gather compute nodes and register them in None cell15:22
egonzalezrwellum, after that all should be working (from cells side)15:22
*** awidders_ has quit IRC15:22
rwellumegonzalez: done but still nova list is empty15:23
*** awidders_ has joined #openstack-kolla15:23
*** awiddersheim has quit IRC15:23
*** gkadam is now known as gkadam-afk15:24
egonzalezrwellum, check nova-api logs and see if database errors still or have gone15:25
openstackgerritMerged openstack/kolla-ansible master: Add vmware support to ceilometer  https://review.openstack.org/46284215:26
rwellumegonzalez: should I create a new vm, then watch lohgs?15:26
egonzalezrwellum, yep15:26
*** coolsvap has quit IRC15:27
*** shardy is now known as shardy_afk15:27
*** gkadam-afk has quit IRC15:31
sean-k-mooneyegonzalez: did we recently change setting on the docs job?15:31
sean-k-mooneyegonzalez: the odl docs exceed the linelenght check.15:32
*** gkadam has joined #openstack-kolla15:32
sean-k-mooneyim going to submit an addtional patch to fix that and then rebase my dpdk docs ontop15:32
rwellumegonzalez: so I think I got no errors this time. But also no VM15:32
egonzalezsean-k-mooney, a couple weeks ago added a path to make docs gates fail if does not cover standards15:33
sean-k-mooneyegonzalez: i see well ill fix the errors in any case there are only a cople of lines that need to be wrapped15:34
egonzalezrwellum, weird :/15:34
*** schwicht has joined #openstack-kolla15:34
*** duonghq has quit IRC15:35
rwellumegonzalez: ok I booked some time with sbezverk to help me go over my cell creation for both 4.x and 5.x as there is something wrong with each. I can rule out user-error hopefully.15:36
*** jbrooks has quit IRC15:36
*** mdnadeem has joined #openstack-kolla15:37
*** klindgren__ has joined #openstack-kolla15:37
*** jbrooks has joined #openstack-kolla15:39
*** klindgren has quit IRC15:39
*** klindgren_ has quit IRC15:40
*** klindgren has joined #openstack-kolla15:40
*** iniaziaway has joined #openstack-kolla15:40
*** Eswar has joined #openstack-kolla15:41
EswarHi All, i am new to kolla-k8s , i am following https://docs.openstack.org/kolla-kubernetes/latest/deployment-guide.html15:42
Eswarbut getting below error15:42
Eswarkolla         mariadb-0                        0/1       Init:0/1           0          5m kolla         mariadb-init-element-jn0j8       0/1       CrashLoopBackOff   3          5m15:42
Eswarkolla         mariadb-0                        0/1       Init:0/1           0          5m15:42
cocakollasdake - got pulled away, but here is my globals.yml: https://pastebin.com/Y03VUL9b15:43
Eswarkolla         mariadb-init-element-jn0j8       0/1       CrashLoopBackOff   3          5m15:43
*** iniazi has quit IRC15:43
Eswarcan someone please help me in resolving it ??15:44
cocakollaEswar - I'm stuck on the same thing15:44
manheimhi Eswar15:44
EswarHi Manheim15:44
manheimcan you please paste the output in something like https://paste.ubuntu.com/15:44
manheimand give us the link15:45
*** dave-mcc_ has joined #openstack-kolla15:45
manheimmuch easier to see what is the issue15:45
Eswarok will do ... meanwhile is it related to https://bugs.launchpad.net/kolla/+bug/170467115:45
openstackLaunchpad bug 1704671 in kolla "mariadb fails to bootstrap in kolla-kubernetes" [Undecided,Fix released] - Assigned to Serguei Bezverkhi (sbezverk)15:45
Eswarok will do now15:45
manheimty15:45
*** duonghq has joined #openstack-kolla15:45
*** awiddersheim has joined #openstack-kolla15:46
cocakollaEswar - run "docker ps -a | grep k8s_dependencies_mariadb", get the container id for the active container and run "docker attach THE_CONTAINER_ID"15:46
inc0meeting in 1515:47
sbezverkEswar make sure you do not have leftovers in /var/lib/kolla/volumes/mariadb from your previous attaempts15:47
sdakereading that bug report15:47
*** dave-mccowan has quit IRC15:47
sdakeyou need new images15:47
cocakollasdake - no luck after I cleaned out that /var/lib/kolla/volumes/15:47
egonzalezmatrohon, ping15:47
matrohonegonzalez: hi15:48
Eswar@cocakolla Entrypoint WARNING: 2017/08/16 15:47:08 entrypoint.go:81: Resolving dependency for mariadb-init-element failed: Job mariadb-init-element is not completed yet15:48
sbezverkthat bug fix went into master only if I remember right15:48
rwelluminc0: where is the meeting schedule hosted? So I can add an item please?15:48
cocakollaEswar - that's the exact same issue I'm having15:48
egonzalezmatrohon, just noticed skydive logs goes to /var/log/kolla/skydive and also to stdout15:48
Eswar@sbezverk ... shall i clean and re-run ??15:48
inc0rwellum: agenda is empty15:48
inc0so yeah, be my guest15:48
matrohonegonzalez: yep15:48
sbezverkEswar yes, please15:48
sdakerwellum https://wiki.openstack.org/wiki/Meetings/Kolla15:49
sbezverkEswar what do you use to deploy?15:49
*** awidders_ has quit IRC15:49
sdakerwellum anyone can edit the agenda as they  like15:49
Eswarok sir .. i will do and re-run ....@sbezverk .... is it ok to take kolla-k8s as master or 0.6.015:49
egonzalezmatrohon, is that expected? will consume a big amount of disk space, many logs in analyzer container15:49
sdakesbezverk https://bugs.launchpad.net/kolla/+bug/170467115:49
openstackLaunchpad bug 1704671 in kolla "mariadb fails to bootstrap in kolla-kubernetes" [Undecided,Fix released] - Assigned to Serguei Bezverkhi (sbezverk)15:49
*** ntpttr_laptop has quit IRC15:49
sbezverkEswar please use the latest kolla-k8s15:50
sdakesbezverk that means you need the latest 5.0.0 images for kolla-kubernetes to work properly15:50
*** StephenWang1991 has quit IRC15:50
sdakeand 5.0.0 is not yet released15:50
*** dave-mccowan has joined #openstack-kolla15:50
Eswarok @sbezverk i will do re-run and let u know15:50
sbezverksdake if you use private registry chart15:50
matrohonegonzalez: there is  logrotate, but it might work only for the log file15:50
sbezverkyou can deploy latest 5.0.0 images15:51
sdakesbezverk - ok that is the problem these folks are having15:51
cocakollasbezverk - how can I deploy the latest 5.0.0 images?15:51
rwellumsbezverk: oh I was there but didn't realize that was editable. Even though I've done it a couple of times before. ;-/15:51
sbezverkthat is how we test master at the gate15:51
matrohonegonzalez: I don't know where the stdout goes at the end?15:51
Eswaru mean in image_tag as 5.0.0 instead of 4.0.015:51
sbezverkone sec guys. 1. sdake is right no official 5.0.0 images15:51
matrohonegonzalez: but removing the stdout is fine for me15:51
sbezverk2. you can still deploy using a tar ball 5.0.0 images15:52
sbezverkbut to do that you need to deploy a new private registry chart15:52
egonzalezmatrohon, yep, topology tab works fine, the other two I dont know because I have them empty hehe15:52
*** dave-mcc_ has quit IRC15:52
matrohonegonzalez: oh, me either, I never use them :)15:53
sdakesbezverk to be more accurate - no POC/EVAL images for 5.0.015:53
sbezverkplease this link: https://github.com/openstack/kolla-kubernetes/tree/master/helm/microservice/registry-deployment15:53
sbezverkthis chart will run a private registry inside of your kube cluster15:54
matrohonegonzalez: the most interesting features to me are the ability to go back in the past and the flow capture15:54
sbezverkand download automativally latest tarball with 5.0.0 images15:54
cocakollasbezverk - thanks, I will try that.15:55
cocakollaEswar - since we're having the same exact issue let let me know if deploying the 5.0 containers works for you. I'll do  the same.15:55
sbezverkfor reference you can you gate code, see what is done for BRANCH == 't'15:55
Eswarcocakolla ... i am looking at it ... will let u know15:56
*** Ally77 has joined #openstack-kolla15:56
*** ntpttr_laptop has joined #openstack-kolla15:56
egonzalezmatrohon, yeah, allowing to follow traffic from all devices is really useful15:57
*** pcaruana has quit IRC15:57
Ally77Hi all, wondering if anyone has deployed ceph replication with kolla? Looking to configure this for DR purpose15:57
inc0it's meeeeeeeting time!16:00
*** Teck7 is now known as rfxn16:00
*** eze has joined #openstack-kolla16:00
*** tovin07_ has left #openstack-kolla16:02
*** eaguilar has quit IRC16:03
*** lpetrut has quit IRC16:04
*** jascott1 has joined #openstack-kolla16:05
Eswarcocakolla .... Still i am trying to figure out how to deploy 5.0 containers ....don't know exactly how to do .... if possible can you please do and let me know ...Thanks16:06
cocakollaEswar - I'm trying to figure out the same16:07
*** kiennt has quit IRC16:11
cocakollasbezverk - I think I'm on the right track. I've replaced the contents of "kolla-bringup/kolla-kubernetes/helm/microservice/registry-deployment" with the repo you linked16:15
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla master: Enable xtrace on start scripts  https://review.openstack.org/49425816:15
cocakollasbezverk - I don't want to run the wrong thing and break everything I've gotten working so far. Any ideas on what I should run now that I have the private registry config in registry-deployment?16:16
sbezverkcocakolla when you have your cluster up but before you start deploying openstack you will need to deploy this registry and then change your cloud yaml file that it points to your private registry and not docker hub16:18
*** tonanhngo has joined #openstack-kolla16:19
openstackgerritsean mooney proposed openstack/kolla-ansible master: This change allow setting kernel commandline args  https://review.openstack.org/40887616:19
openstackgerritsean mooney proposed openstack/kolla-ansible master: add ovs-dpdk docs  https://review.openstack.org/49391316:19
openstackgerritsean mooney proposed openstack/kolla-ansible master: automate ovs datapath configuration  https://review.openstack.org/49391616:19
openstackgerritsean mooney proposed openstack/kolla-ansible master: make dpdk tunnel ip assignment configurable  https://review.openstack.org/49393916:19
openstackgerritsean mooney proposed openstack/kolla-ansible master: fix line lenght issues in networking-guide  https://review.openstack.org/49425916:19
*** ismc has joined #openstack-kolla16:20
sbezverkcocakolla here is command I used to deploy it on my cluster: helm install helm/microservice/registry-deployment --namespace kolla --name registry-ubuntu --set distro=ubuntu --set node_port=30402 --set initial_load=true16:21
*** pbourke has quit IRC16:21
cocakollaand what setting do you use for your "docker_registry" in your cloud.yml?16:21
sbezverk      docker_namespace: lokolla16:21
sbezverk      docker_registry: 127.0.0.1:3040216:21
cocakollaahh ok16:21
cocakollathanks16:21
sbezverkcocakolla it takes about 10 minutes to get registry into a running state so be patient16:22
*** itlinux has quit IRC16:22
*** pbourke has joined #openstack-kolla16:23
cocakollasbezverk - I don't currently have a  docker_namespace: lokolla entry in my cloud.yml, should I add it?16:25
sbezverkcocakolla yes, that tarball is actually a snapshot of a registry and name lokolla is hardcoded16:26
sbezverkso you have to use the same and configure it in cloud.yaml16:26
sean-k-mooneyby the way is the kolla meeting on now16:31
*** TxGirlGeek has joined #openstack-kolla16:32
*** Ally77 has quit IRC16:35
*** reddream has joined #openstack-kolla16:35
Eswar@cocakolla ... i face kolla         kube-registry-2362051601-9krpk   0/1       Init:CrashLoopBackOff   4          3m16:35
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Implement Keystone zero-downtime upgrade  https://review.openstack.org/42544616:36
cocakollaEswar - I'm still trying  to figure out how to deploy that private registry16:36
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Implement Keystone zero-downtime upgrade  https://review.openstack.org/42544616:36
reddreamhello, is it possible to make kolla Ocata to work with ceph luminous?16:37
*** hoaile has joined #openstack-kolla16:37
Eswar@cocakolla .. as sbezverk mentioned ... i ran helm install helm/microservice/registry-deployment --namespace kolla --name registry-ubuntu --set distro=ubuntu --set node_port=30402 --set initial_load=true16:37
*** serlex has quit IRC16:37
Eswar@sbezverk  .... plz correct me .... as u  mentioned i ran #helm install helm/microservice/registry-deployment --namespace kolla --name registry-ubuntu --set distro=ubuntu --set node_port=30402 --set initial_load=true16:39
Eswarbut its in  # kolla         kube-registry-2362051601-9krpk   0/1       Init:CrashLoopBackOff   4          3m16:39
sean-k-mooneyreddream: it has not been tested. you may be able to use it but it is not planned to be supported until queens16:39
sean-k-mooneyreddream: if you are using external ceph then it will likely work just fine. kolla deployed ceph is a different story16:41
reddreamsean-k-mooney: thank you, you also answered the question I'm about to ask16:42
cocakollamy port 30402 is already being used by tcp6       0      0 :::30402                :::*                    LISTEN      29519/kube-proxy16:42
*** hoaile has quit IRC16:42
*** vhosakot has joined #openstack-kolla16:43
jamesbensonreddream: I thought liminous isn't out yet...16:43
*** yangyapeng has quit IRC16:44
jamesbensons/liminous/luminous/16:45
reddreamjamesbenson: no, just RC516:45
sbezverkcocakolla it is normal that kube-proxy uses that port16:45
*** lpetrut has joined #openstack-kolla16:45
Eswar@sbezverk ....... kolla         kube-registry-2362051601-9krpk   0/1       Init:CrashLoopBackOff   7          13m16:46
egonzalezrfxn, looks like your know a lot about galera and DBs, could you open a blueprint with a feature request, I've no idea what garbd is?16:46
sbezverkEswar please try to capture docker log from that crashing container16:46
sbezverkto see what is going on16:46
jamesbensonsoon I imagine then :-). I'm waiting for it too16:46
rwellumkubectl describe pod kube-registry-2362051601-9krpk - start with that.16:46
duonghqinc0,  can you help me on this: https://review.openstack.org/#/c/491412/?16:46
duonghqI don't cleary understand the project-config16:47
*** yangyapeng has joined #openstack-kolla16:47
sean-k-mooneyjamesbenson: liminous is not no it wont be relased until october/november i think16:47
rfxnegonzalez, certainly can16:48
Eswarubuntu@ubuntu:~/kolla-bringup/kolla-kubernetes$ kubectl describe pod kube-registry-2362051601-9krpk -n kolla16:49
rfxnwould be my first such submission so if i dont conform to some best practice way of doing things, go easy on me16:49
Eswar15m           10s             93      kubelet, ubuntu                                                 Warning         FailedSync              Error syncing pod   15m           10s             86      kubelet, ubuntu         spec.initContainers{init-myservice}     Warning         BackOff                 Back-off restarting failed container16:49
jamesbensonoh man, thanks for the info sean-k-mooney16:49
egonzalezrfxn, dont worry, we have a whole release :)16:50
rfxnkolla or kolla-ansible the right spot to submit it?16:50
sean-k-mooneyduonghq: looking at https://review.openstack.org/#/c/491412/3 im not sure why Andreas Jaeger left those comments16:51
egonzalezrfxn, likely will need both, on blueprint for image addition and other in kolla-ansible dependent of kolla's bp for ansible changes16:51
rfxnack16:51
sean-k-mooneyduonghq: well  gerrit/acls/kolla-ansible-plugins.config is not needed because you have  gerrit/acls/openstack/kolla-ansible-plugins.config16:52
sean-k-mooneybut not sure why you would remove the python jobs16:52
*** yangyapeng has quit IRC16:52
duonghqsean-k-mooney, thank you, I have not had tests, but due to Ansible plugins is written in Python, I think it should have?16:53
cocakollasbezverk - do you have any docs anywhere on setting up a private registry step by step?16:53
*** egonzalez has quit IRC16:53
jamesbensoncocakolla I have a script, pretty easy16:54
sean-k-mooneyduonghq: ya you should have the python-jobs for when pep8 and python unit tests are added to the plugin repo16:54
jamesbensonhttps://github.com/JamesOBenson/Kolla-misc/blob/master/setup_kolla.sh16:54
cocakollajamesbenson - I'll take a look. Thanks.16:54
duonghqsean-k-mooney, so I should remove it and put back after I add the test?16:55
sean-k-mooneycocakolla: its basically 1 command you just run the registry in a docker container16:55
sbezverkcocakolla unfortunately no, you know there is always higher priority thing than writing the docs..16:55
sbezverkwhich is wrong but it always happens16:56
*** rhallisey has quit IRC16:57
sean-k-mooneysbezverk: well it used to be in the multinode quickstart at lest if you are using the docker container16:58
sbezverksean-k-mooney no it is new solution which runs inside of k8s cluster16:58
sean-k-mooneysbezverk: oh i was just going to suggest using https://github.com/openstack/kolla/blob/master/tools/start-registry16:59
sbezverkcocakolla actually since I pushed it to helm upstream and it got merged, you can find doc in helm incubation repo..16:59
*** strigazi_OFF is now known as strigazi17:00
*** reidrac has quit IRC17:00
*** awiddersheim has quit IRC17:00
*** awiddersheim has joined #openstack-kolla17:00
*** matrohon has quit IRC17:01
sbezverksean-k-mooney I was not aware of this sorry17:01
*** rhallisey has joined #openstack-kolla17:02
*** rhallisey has quit IRC17:02
sean-k-mooneysbezverk: well it might make sense to add a similar tool to kolla-k8s if you want to have the registry hosted by k8s17:02
*** rhallisey has joined #openstack-kolla17:02
*** mgoddard has quit IRC17:02
sean-k-mooneysbezverk: or even to kolla i guess17:03
*** manheim has quit IRC17:03
*** jtriley has quit IRC17:03
*** shardy_afk is now known as shardy17:03
*** lpetrut has quit IRC17:07
*** Eswar has quit IRC17:10
*** gfidente is now known as gfidente|afk17:12
*** jtriley has joined #openstack-kolla17:16
*** duonghq has quit IRC17:19
*** jascott1 has quit IRC17:19
*** jascott1 has joined #openstack-kolla17:20
*** sambetts is now known as sambetts|afk17:23
*** shardy has quit IRC17:23
*** jascott1 has quit IRC17:24
*** jascott1 has joined #openstack-kolla17:25
*** reddream has quit IRC17:25
*** gkadam has quit IRC17:25
*** jtriley has quit IRC17:25
*** jtriley has joined #openstack-kolla17:28
*** jascott1 has quit IRC17:29
sdakecocakolla fwiw - docs are the 2nd priority in my book right behind building a sustainable community - but not everyone always agrees :)17:33
*** manheim has joined #openstack-kolla17:35
cocakollasdake / sbezverk - yeah that's totally fine.17:38
sdakecocakolla if you build something that doesn't have documentation - nobody can use it but the developers who wrote it - whats the point? :)17:38
*** manheim has quit IRC17:39
*** itlinux has joined #openstack-kolla17:40
cocakollasdake - job security? heh :)17:40
sbezverksean-k-mooney that is what registry deployment chart does in kolla-k8s17:41
sdakecocakolla job security comes through 3 things 1) 1 year emergency fund 2) cutting edge skill set 3) large network of peers17:41
sdakewith 1 alone your invincible ;)17:41
sdakewith 1+2+3 your Lucy17:42
inc0https://www.thc.org/root/phun/unmaintain.html17:42
cocakollaI'm still trying to get this private registry going. I've modified my "cloud.yaml" with the private registry updates. I ran this modified helm command "helm install --debug kolla-kubernetes/helm/service/mariadb --namespace lokolla --name mariadb --values ./cloud.yaml"17:43
cocakollanow it can't find the config maps17:43
cocakolla Couldn't get configMap lokolla/mariadb: configmaps "mariadb" not found17:43
sdakeinc0 my browser tells me that site is unsafe :)17:44
sdakei'll take its word for it17:44
inc0yeah, it's just plain html17:44
inc0it's still good read;)17:45
sdakecurl ftw :)17:46
sdakeinc0 yup pretty funny :)17:48
sdakereminds me of how the russian jet mechanics in the cold war used to protect their families17:48
sean-k-mooneysdake: job security also comes form being the only perosn who understands how the thing you maintain works but companies frown on that for some reason :)17:49
cocakollaok so I think I'm understanding this a bit better, I'm supposed to spin up a private registry container listening on some port.17:49
*** lpetrut has joined #openstack-kolla17:50
sean-k-mooneyinc0: hehe i think i have the Creative Miss-spelling part down17:51
sdakesean-k-mooney ya in the cold war each jet was totally customized to the jet mechanic in russia17:52
cocakollawhen I try to run "helm install --debug kolla-kubernetes/helm/microservice/registry-deployment --namespace kolla --name registry-centos --set distro=centos --set node_port=30402 --set initial_load=true" I get this message "Error: release registry-centos failed: Service "kube-registry" is invalid: spec.ports[0].nodePort: Invalid value: 30402: provided port is already allocated" but the port is not in use before I issue the17:52
sdakeonly the mechanic knew how to work on a specific mig17:52
cocakolla command, for some reason it thinks it's already in use17:52
sdakethe migs were expensive - so as long as the mechanic didn't cause any trouble, he lived relatively well and in safety17:53
sean-k-mooneysdake: the one problem with that approch is if its broke you have to fix it cause no one else can17:53
sdakeyup17:54
sdakebeen there done that17:54
sdakethats why I adopt the *other* philosophy 1/2/3 above ;)17:54
*** strigazi is now known as strigazi_OFF17:55
cocakollagot a good laugh out of that article17:56
*** manheim has joined #openstack-kolla17:57
*** eaguilar has joined #openstack-kolla18:00
*** mgoddard has joined #openstack-kolla18:01
*** eze has quit IRC18:03
*** eaguilar has quit IRC18:05
*** dave-mccowan has quit IRC18:05
inc0it's a good one;)18:07
inc0worth risking your safety;)18:07
*** eaguilar has joined #openstack-kolla18:08
*** dave-mcc_ has joined #openstack-kolla18:09
*** pcaruana has joined #openstack-kolla18:09
*** manheim has quit IRC18:10
*** ismc has quit IRC18:10
jamesbenson:-(. Our IT blocked that domain.18:11
*** jascott1 has joined #openstack-kolla18:11
jamesbensonlook ma, i'm in Miami now... and I get the website. :-)18:12
*** rhallisey has quit IRC18:13
sean-k-mooneyinc0: qq have you got approval for ptg yet?18:15
inc0negative18:15
inc0but I'm coming18:15
sean-k-mooneysame there was ment to be a review yesterday or monday but i havent heard if it went ahead.18:16
jamesbensonnice read inc018:17
sean-k-mooneyi might go myself if the travel is not approve but the prices are starting to go up now so i would perfer to know one way or the other sooner rather then later18:17
*** rhallisey has joined #openstack-kolla18:17
inc0right18:17
cocakollainc0 - I curled it also, no risk there. But, I am risking my patience with this private registry...18:21
rwellumcocakolla: funny thing is, once you get the registry working, you'll have to figure out the new cell0 functionality... But cross that bridge when it comes...18:22
inc0cocakolla: I haven't seen full convo, might summing it up for me?18:22
srwilkersinc0 in Denver? success!18:22
inc0if you try to run registry for kolla-k8s18:25
cocakollainc0 - Well I ran into a bug with the mysql container it seems. Some else, Eswar, also had the same exact issue. It seems that it might be this bug: https://bugs.launchpad.net/kolla/+bug/170467118:25
openstackLaunchpad bug 1704671 in kolla "mariadb fails to bootstrap in kolla-kubernetes" [Undecided,Fix released] - Assigned to Serguei Bezverkhi (sbezverk)18:25
cocakollasbezverk and others suggested switching to using a private registry with the 5.0.0 containers, but I can't seem to get that working18:26
sbezverkcocakolla man I am positive, it is something with the way you start it as we have it running at the gate. can you get kubectl get svc/pod/namespaces before you start18:28
inc0cocakolla: did you try to use registry helm package?18:28
inc0https://github.com/openstack/kolla-kubernetes/tree/master/helm/microservice/registry-deployment this one18:29
*** markhargrove281 has joined #openstack-kolla18:29
inc0try helm install -v . --values initial_load=true18:29
*** markhargrove28 has quit IRC18:31
*** unicell has joined #openstack-kolla18:32
*** rhallisey has quit IRC18:34
rwellumsbezverk: how do I see the gate logs for 5.0.0 - in particular I'd like to see the cloud.yaml?18:34
rwellumsbezverk: we run 4.0.0 in the gates all the time too - so not sure why moving to 5.0.0 which is very experimental is a solution to 4.0.0 woes...18:35
sbezverkrwellum there is no coud.yaml, we build pieces configuration inside of scripts18:35
sbezverkdepending on the branch against which a job is executed18:36
rwellumwhen you do helm install  - what is passed to --values then?18:36
cocakollainc0 - yeah I downloaded that registry helm package and replaced the kolla-kubernetes/helm/microservice/registry-deployment... but let me try the helm install18:36
sbezverkrwellum dynamically built file18:36
rwellumyeah - how do I see that - in logs? :)18:37
inc0cocakolla: init container in this package downlaods actual images18:37
sbezverkrwellum you need to open unsuccessful job log and you will see it there search for 'echo'18:38
sbezverkrwellum we do not generate much of logs for a successful job18:38
rwellumsbezverk: eek!18:38
rwellumLike it's hard to say 'it works in the gates' - but we can't prove it other than the gate passes.18:39
rwellumAnyway...18:39
cocakollahmm where do I run that "helm install -v . --values initial_load=true" command? I get "Error: unknown shorthand flag: 'v' in -v"18:40
cocakollaI tried modifying it but it's not working.18:41
*** mgoddard has quit IRC18:41
inc0sbezverk: what was flag to specify value override?18:42
sbezverkinc0: which flag you want to override?18:43
inc0initial_load18:43
sbezverk--values inital_load=true18:43
sbezverkcocakolla there is no '-v' keyword in helm install command18:44
inc0right, my bad18:44
sbezverkrwellum our checks at the gate a way more severe than normal user would ever do. I can guarantee you that18:45
sbezverklike switching volumes between two instances and end to end ping between them18:45
rwellumcocakolla: If you use port 30402 then you need the following in your globals.yaml18:45
rwellumhttps://www.irccloud.com/pastebin/hDk1evPl/18:46
rwellumsbezverk: no argument. I just have a hard time looking through the gate code and logs and translating to something I can manually reproduce.18:46
jascott1cocakolla also `--values` is for a file, i think you want `--set`18:47
cocakollaI keep getting an error saying that that port is already in use.18:47
cocakolla[root@kolla registry-deployment]# helm install . --namespace kolla --name registry-deployment --set distro=centos --set node_port=30402 --set initial_load=true18:47
cocakollaError: release registry-deployment failed: Service "kube-registry" is invalid: spec.ports[0].nodePort: Invalid value: 30402: provided port is already allocated18:47
inc0did you try to run it before?18:47
cocakollabut port 30402 is not in use until I run the command...18:47
rwellumcocakolla: you'll have to completely clean up, reboot vm, sudo kubeadm reset etc18:48
inc0so what this will do is it'll use 30402 on *every* node in k8s cluster18:48
cocakollaahh ok18:48
cocakollaI didn't know I had to reset and reboot.18:48
cocakollalet me try that18:48
inc0then all the images will be localhost:30402/...18:49
rwellumcocakolla: fairly sure - at least I never recover from partially running OpenStack configurations. If it's a VM and cheap easier to clean up.18:50
inc0rwellum: removing all k8s resources related to kolla is no good?18:51
rwellumI have tried - I mean it can work, depends on how you are using Cinder etc.18:52
rwellumI thought sdake had a cleanup playbook working though.18:52
*** egonzalez has joined #openstack-kolla18:55
*** mdnadeem has quit IRC19:00
*** manheim has joined #openstack-kolla19:16
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Fixing ceph gates  https://review.openstack.org/49430119:16
*** eaguilar has quit IRC19:20
*** robbbe has joined #openstack-kolla19:21
*** stefan_bo has joined #openstack-kolla19:22
*** kbaegis1 has joined #openstack-kolla19:22
*** kbaegis has quit IRC19:23
*** markhargrove28 has joined #openstack-kolla19:24
*** eaguilar has joined #openstack-kolla19:26
*** ducttap__ has joined #openstack-kolla19:26
*** stefan_bo has quit IRC19:27
openstackgerritMerged openstack/kolla-ansible master: Enable ovsdpdk w/ ODL Neutron_Plugin_Agent  https://review.openstack.org/49400919:27
*** markhargrove281 has quit IRC19:27
*** ducttape_ has quit IRC19:29
*** rhallisey has joined #openstack-kolla19:30
*** matrohon has joined #openstack-kolla19:30
*** manheim has quit IRC19:34
*** ducttape_ has joined #openstack-kolla19:36
rwellumsbezverk: so I am looking at a failed gate job - where do I find the log of the 'helm inits' - see what was passed in etc?19:36
egonzalezinc0 iirc ceph gates was caused because servers used by infra does not mount a second volume19:36
inc0egonzalez: right so we have only root disk right?19:37
*** ducttap__ has quit IRC19:40
*** yangyapeng has joined #openstack-kolla19:44
*** yangyapeng has quit IRC19:49
*** iniaziaway is now known as iniazi19:55
iniazistill trying at the build stage.  last week was talking about my builds failing for 4.0.2.  after little investigating, for me at least, its initially failing because docker doesn't have internet connectivity19:56
iniaziI'm new to docker and following the kolla-ansible guide... did I miss a step on setting up network for docker?  not too sure what to do for that part19:56
sdakeiniazi behind a firewall?19:57
sdakeproxy?19:57
sdakehave a meeting - if your behind a proxy - you need to setup docker to work with the proxy19:57
sdakealso kolla needs to be setup in this way19:57
sdakegtg19:57
iniazisdake: behind a firewall.  the physical (where i'm building the image as well as where openstack is to be installed) is on 10.0.0.0/22 subnet (behind a firewall)19:57
iniazisetup a private registry but both registry & the build host that kolla-build launches can't connect out.  on one i think its a 172.17.x.x network.  not sure if docker is supposed to nat?  or I need to configure it so it uses same subnet/ip as my host network?19:58
*** ntpttr_laptop has quit IRC19:59
*** ntpttr_laptop has joined #openstack-kolla20:00
inc0iniazi: building doesn't use net=host20:05
inc0try running docker run -it busybox bash20:07
inc0that will get you bash inside container20:08
inc0(busybox is image name, you can use any of images available on dockerhub there)20:08
inc0and try to debug networking from within20:08
*** awidders_ has joined #openstack-kolla20:09
*** sean-k-mooney has quit IRC20:11
*** trozet has quit IRC20:11
*** awiddersheim has quit IRC20:13
iniaziyeah, busybox gets an ip on a 172.17.0.0/16 subnet.  which looks like docker default.  so I guess I'm supposed to configure that?  or is it that i need to enable natting on docker?  i think this is what I found earlier, just not sure if I missed a step in kolla docs or i'm generally suppsoed to configure docker before starting with kolla-ansible?20:15
inc0docker is supposed to take care about nat20:20
inc0it should work ootb, so issue is somwehere else20:21
inc0ping 8.8.8.8 works from inside container?20:21
*** trozet has joined #openstack-kolla20:26
*** jtriley has quit IRC20:38
*** trozet has quit IRC20:40
*** pcaruana has quit IRC20:41
*** jamesbenson has quit IRC20:41
rwellumHey inc0 - did you find your slides?20:46
inc0trying,I see just template, not finished one:/20:48
*** jamesben_ has joined #openstack-kolla20:48
*** jamesben_ has quit IRC20:51
*** jtriley has joined #openstack-kolla20:52
*** dciabrin has quit IRC20:52
*** rhallisey has quit IRC20:54
*** trozet has joined #openstack-kolla20:54
*** jamesbenson has joined #openstack-kolla20:55
*** lpetrut has quit IRC20:56
*** lucasxu has quit IRC20:57
*** markhargrove28 has quit IRC20:58
*** lpetrut has joined #openstack-kolla20:58
*** awiddersheim has joined #openstack-kolla21:03
*** awidders_ has quit IRC21:03
*** dciabrin has joined #openstack-kolla21:05
*** jtriley has quit IRC21:06
inc0can't find it rwellum :/21:08
*** Pavo has joined #openstack-kolla21:14
*** itlinux has quit IRC21:21
*** jamesbenson has quit IRC21:22
*** jamesbenson has joined #openstack-kolla21:22
*** jamesbenson has quit IRC21:27
*** lpetrut has quit IRC21:34
*** manheim has joined #openstack-kolla21:34
*** manheim has quit IRC21:39
*** kbaegis has joined #openstack-kolla21:44
*** kbaegis1 has quit IRC21:47
*** gfidente|afk is now known as gfidente21:50
*** Manheim has joined #openstack-kolla21:51
Manheimfun fact, did you know mesosphere is a platinum sponsor for this years kubecon? lol21:51
*** schwicht has quit IRC22:00
*** jascott1 has quit IRC22:00
*** jascott1 has joined #openstack-kolla22:01
*** jascott1 has quit IRC22:05
*** jascott1 has joined #openstack-kolla22:06
*** jascott1 has quit IRC22:07
*** jascott1 has joined #openstack-kolla22:07
*** jiriprox has quit IRC22:08
*** jascott1 has quit IRC22:12
*** egonzalez has quit IRC22:13
*** matrohon has quit IRC22:14
*** kbaegis1 has joined #openstack-kolla22:18
*** kbaegis has quit IRC22:21
*** tonanhngo has quit IRC22:22
*** jascott1 has joined #openstack-kolla22:24
*** jbrooks has quit IRC22:26
*** unicell has quit IRC22:26
*** StephenWang1991 has joined #openstack-kolla22:27
*** unicell has joined #openstack-kolla22:27
*** schwicht has joined #openstack-kolla22:29
*** StephenWang1991 has quit IRC22:32
*** manheim_ has joined #openstack-kolla22:36
*** goldyfruit has quit IRC22:46
*** ducttap__ has joined #openstack-kolla22:52
*** ducttape_ has quit IRC22:55
*** schwicht has quit IRC22:58
*** vhosakot has quit IRC23:14
*** schwicht has joined #openstack-kolla23:18
iniaziinc0: hey sorry.  there is no network connectivity.  verified already.  can't telnet to port 80, fore example, on the physical (where the container is running)23:43
inc0:/23:43
inc0I hate networking23:43
inc0so normally it would be bridged to your regular iface23:44
iniazidocker's network docs are not clear also23:44
inc0and NATted23:44
iniaziyeah if its bridged only different subnet won't work.  but they don't say if I need to add that subnet to my firewall... but if its NAT'd (which i'd assume it would do since there is no specific mention otherwise)23:45
*** yangyapeng has joined #openstack-kolla23:45
iniazimeans something is broke when I installed it.  or their install method is broken.23:45
*** ntpttr_laptop has quit IRC23:47
*** manheim_ has quit IRC23:50
iniazii can ping other containers on the same box, but can't connect out.  i'm trying to google how to reset the networking/bridge (or recreate it).  i think thats probably whats needed23:52
*** yangyapeng has quit IRC23:54
sdakegroan complete tire replacement on the porsche23:56
* sdake GROANS23:56

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!