Thursday, 2017-11-09

*** signed8bit has joined #openstack-kolla00:00
*** jascott1 has joined #openstack-kolla00:00
*** jascott1 has quit IRC00:05
*** salv-orlando has quit IRC00:05
*** litmaj0r has quit IRC00:05
*** salv-orlando has joined #openstack-kolla00:06
*** salv-orlando has quit IRC00:10
*** zshi has joined #openstack-kolla00:14
*** dmsimard is now known as dmsimard|off00:22
*** signed8bit is now known as signed8bit_Zzz00:23
*** signed8bit_Zzz has quit IRC00:23
*** huanxie has joined #openstack-kolla00:36
*** signed8bit has joined #openstack-kolla00:38
*** jascott1 has joined #openstack-kolla00:45
*** huanxie has quit IRC00:52
openstackgerritWei Zheng proposed openstack/kolla-kubernetes master: writing convention: do not use “-y” for package install  https://review.openstack.org/50702600:57
*** zshi has quit IRC01:11
*** caowei has joined #openstack-kolla01:14
*** signed8bit is now known as signed8bit_Zzz01:16
*** zhangfei has joined #openstack-kolla01:16
*** signed8bit_Zzz has quit IRC01:16
*** jtriley has joined #openstack-kolla01:22
*** Sean-md has joined #openstack-kolla01:24
*** zshi has joined #openstack-kolla01:25
*** kiennt26 has joined #openstack-kolla01:34
*** sgordon has quit IRC01:42
*** sgordon has joined #openstack-kolla01:51
*** Pavo has joined #openstack-kolla01:59
*** zhangfei has quit IRC02:05
*** dan_ardelean has joined #openstack-kolla02:07
*** salv-orlando has joined #openstack-kolla02:07
*** dan_ardelean has quit IRC02:11
*** salv-orlando has quit IRC02:12
*** signed8bit has joined #openstack-kolla02:13
*** zhangfei has joined #openstack-kolla02:18
*** zshi has quit IRC02:19
*** signed8bit is now known as signed8bit_Zzz02:30
*** signed8bit_Zzz has quit IRC02:31
*** bswartz has quit IRC02:49
*** jascott1_ has joined #openstack-kolla02:49
*** pc_m has quit IRC02:50
*** Jeffrey4l has quit IRC02:50
*** pc_m has joined #openstack-kolla02:50
*** inc0 has quit IRC02:51
*** jaosorior has quit IRC02:51
*** daidv has quit IRC02:51
*** Daviey_ has quit IRC02:51
*** dosaboy has quit IRC02:51
*** Daviey has joined #openstack-kolla02:51
*** jascott1 has quit IRC02:51
*** ansmith has quit IRC02:51
*** zhubingbing__ has quit IRC02:51
*** dmellado has quit IRC02:51
*** leifmadsen has quit IRC02:51
*** zhubingbing__ has joined #openstack-kolla02:52
*** oanson has quit IRC02:53
*** ArminderSingh has quit IRC02:54
*** leifmadsen has joined #openstack-kolla02:57
*** dmellado has joined #openstack-kolla02:57
*** daidv has joined #openstack-kolla02:57
*** ArminderSingh has joined #openstack-kolla02:59
*** oanson has joined #openstack-kolla03:00
*** signed8bit has joined #openstack-kolla03:00
*** jtriley has quit IRC03:00
spsuryamorning guys03:03
*** ansmith has joined #openstack-kolla03:04
*** dosaboy has joined #openstack-kolla03:04
*** jaosorior has joined #openstack-kolla03:04
*** inc0 has joined #openstack-kolla03:05
*** Jeffrey4l has joined #openstack-kolla03:05
*** salv-orlando has joined #openstack-kolla03:08
*** jtriley has joined #openstack-kolla03:11
*** salv-orlando has quit IRC03:13
*** zhangfei has quit IRC03:13
*** retr0h has quit IRC03:14
*** nhlfr has quit IRC03:15
*** bachp has quit IRC03:16
*** Jeffrey4l has quit IRC03:17
*** unicell has joined #openstack-kolla03:18
*** unicell has quit IRC03:23
*** coolsvap has joined #openstack-kolla03:23
*** Jeffrey4l has joined #openstack-kolla03:24
spsuryainc0: https://review.openstack.org/#/c/518630/103:25
spsuryadone03:25
*** zhangfei has joined #openstack-kolla03:26
openstackgerritWei Zheng proposed openstack/kolla-kubernetes master: writing convention: do not use “-y” for package install  https://review.openstack.org/50702603:27
*** dgedia has joined #openstack-kolla03:27
kolla-slack<inc0> Thanks Surya03:28
*** lukepatrick has joined #openstack-kolla03:29
*** dgedia has quit IRC03:32
*** dgedia has joined #openstack-kolla03:37
dgediaHi, I deployed openstack through kolla-ansible. However, when I am trying to create a cirros instance I am facing an error "No valid host was found. There are not enough hosts available." I am not able to figure the cause of this error. Any help would be really appreciated. Am I missing any configuration?03:38
dgediaplease help03:41
*** zhangfei has quit IRC03:44
*** mdnadeem has joined #openstack-kolla03:54
inc0duonghq: when you're around, can you check this out? https://review.openstack.org/#/c/518630/103:58
inc0dgedia: check logs of nova-compute03:58
inc0they are in /var/lib/docker/volumes/kolla_logs03:59
*** mdnadeem is now known as mdnadeem|afk03:59
*** zhangfei has joined #openstack-kolla04:00
dgediathanks inc0, on build-server, it says no file "kolla_logs"04:05
dgediainside /var/lib/docker/volumes/04:05
dgediais it named "kolla_logs"?04:07
dgediaor should I check on compute node?04:07
*** Pavo has quit IRC04:07
duonghqinc0, done04:07
*** salv-orlando has joined #openstack-kolla04:09
*** skramaja has joined #openstack-kolla04:13
*** salv-orlando has quit IRC04:14
dgediainc0, I've pasted logs here:https://pastebin.com/w80dFdGY . I am not not sure how should I make Sql connection. Any hints ?04:18
dgediaThese are nova-schedular logs04:18
*** hyakuhei has quit IRC04:19
*** lukepatrick has quit IRC04:20
*** jtriley has quit IRC04:21
*** jtriley has joined #openstack-kolla04:30
*** zshi has joined #openstack-kolla04:36
*** jtriley has quit IRC04:39
*** jtriley has joined #openstack-kolla04:43
dgediainc0, how can I enable sql service in compute nodes? It seems that is the issue04:44
inc0sql service pn compute node?05:03
inc0that doesn't sound right05:03
inc0but looking at these logs seems like your mariadb have issues05:04
openstackgerritMerged openstack/kolla master: Fixup small bug with dockerhub publisher  https://review.openstack.org/51863005:05
*** salv-orlando has joined #openstack-kolla05:10
*** salv-orlando has quit IRC05:14
*** dgedia has quit IRC05:29
*** signed8b_ has joined #openstack-kolla05:32
*** signed8b_ has quit IRC05:32
*** signed8bit has quit IRC05:34
*** nhlfr has joined #openstack-kolla05:38
*** retr0h has joined #openstack-kolla05:46
*** bachp has joined #openstack-kolla05:46
gouthamHi all05:51
gouthamI deployed openstack with kolla-5.0.1 but i couldnt ping the vm's05:55
gouthamin the console i see that ip is not assigned to interface of VM05:56
*** xinliang has quit IRC05:58
*** janki has joined #openstack-kolla06:03
*** yee379 has joined #openstack-kolla06:08
*** tone_z has joined #openstack-kolla06:09
*** iniazi_ has joined #openstack-kolla06:09
*** xinliang has joined #openstack-kolla06:11
*** salv-orlando has joined #openstack-kolla06:12
*** salv-orl_ has joined #openstack-kolla06:13
*** hemna has quit IRC06:15
*** thomasem has quit IRC06:15
*** tone_zrt has quit IRC06:15
*** calebb has quit IRC06:15
*** pomac has quit IRC06:15
*** yee37928 has quit IRC06:15
*** iniazi has quit IRC06:15
*** hemna has joined #openstack-kolla06:15
*** tone_zrt has joined #openstack-kolla06:15
*** hachi_ has joined #openstack-kolla06:15
*** skramaja_ has joined #openstack-kolla06:16
*** thomasem has joined #openstack-kolla06:16
*** calebb has joined #openstack-kolla06:17
*** mandre has quit IRC06:17
openstackgerritzhubingbing proposed openstack/kolla master: wip Implement onos Docker images  https://review.openstack.org/51827006:22
*** salv-orlando has quit IRC06:22
*** tone_z has quit IRC06:22
*** skramaja has quit IRC06:22
*** hachi_ has quit IRC06:22
*** lukl1 has quit IRC06:23
*** jtriley has quit IRC06:26
*** imcsk8 has quit IRC06:41
*** imcsk8 has joined #openstack-kolla06:41
*** rwsu has joined #openstack-kolla06:43
*** zhurong has joined #openstack-kolla06:46
*** lpetrut has joined #openstack-kolla06:49
*** dklyle has joined #openstack-kolla06:55
*** david-lyle has quit IRC06:58
*** tone_zrt has quit IRC06:58
*** tone_zrt has joined #openstack-kolla07:00
*** dklyle has quit IRC07:08
*** dklyle has joined #openstack-kolla07:08
*** obre has quit IRC07:09
*** EmilienM has quit IRC07:09
spsuryazhubingbing__can you https://review.openstack.org/#/c/518300/07:13
spsurya?07:13
spsuryapbourke: https://review.openstack.org/#/c/425446/07:14
*** jascott1_ has quit IRC07:20
*** jascott1 has joined #openstack-kolla07:20
*** hachi_ has joined #openstack-kolla07:22
*** jascott1 has quit IRC07:25
inc0everyone, just wanted to say that I'm looking at pushes going on to dockerhub07:27
inc0images are being pushed as we speak:)07:27
chasoninc0 good good! Cheers \o/07:32
masberinc0, is this 5.0.1?07:32
inc0masber: master07:33
inc0but we'll backport this to ocata and pike07:33
*** mandre has joined #openstack-kolla07:33
inc0mandre: https://hub.docker.com/r/kolla/ubuntu-source-base/tags/07:34
inc0well, everyone really;)07:34
inc0https://hub.docker.com/r/kolla/ubuntu-source-base/tags/ <- last update, 2min ago07:34
inc0next update - tomorrow07:35
hrwmorning07:35
inc0hey:)07:35
hrwinc0: are they 6.0.0?07:35
inc0well, master07:35
inc0but yeah07:35
masberinc0, 6.0.0 is queens?07:35
inc0autopushed images are tagged with queens07:35
inc0yes07:35
masberim confused :(07:35
masberwow how come so early?07:35
mandrehey! nice07:36
inc0it's master code07:36
masberok, is it stable?07:36
hrwmandre: consider it development07:36
inc0no, it's master;)07:36
inc0but we'll push stable too07:36
hrwmasber: ^^07:36
inc0now that it's proven07:36
masberok, I understand now07:36
masbergood07:36
inc0so brace for daily portion of fresh images :)07:36
hrwinc0: next step: multi-arch images ;D07:36
masberso how come you are doing that? I thought people were owkring on the outo-build feature?07:36
inc0we'll need your nodepool for that07:36
masber*working07:36
*** salv-orl_ has quit IRC07:37
masberanother question, does it comes with ceph illuminus07:37
masber?07:37
*** salv-orlando has joined #openstack-kolla07:37
inc0no, it's Jewel still, we'll upgrade to L during this release07:37
inc0I think ubuntu might have luminous in master tho07:38
hrwinc0: probably not before we get our devops guy to get gema into normal state (instead of overloaded firefighter one)07:38
inc0yeah well, such is life07:38
inc0in any case, when we have arm-nodepool07:38
inc0we'd need to work with infra folk to enable jobs focusing given nodepool07:39
inc0and then build arm images:)07:39
hrwmasber: if you use ubuntu then it may just do not work as we want Jewel and they have Luminous for which we still lack ceph-mgr07:39
inc0anyway, on this happy note, I'm going to sleep07:39
inc0see you all tomorrow07:40
hrwmasber: hint, hint: make ceph-mgr image and submit it for review07:40
hrwinc0: miłych snów07:40
inc0give a newly pushed images a run please07:40
inc0dzieki:)07:40
*** lpetrut has quit IRC07:40
* inc0 off07:40
mandrehow come the version on https://docs.openstack.org/kolla/latest/ still says 5.0.0.0rc2.dev112?07:40
*** salv-orlando has quit IRC07:42
*** iniazi_ has quit IRC07:43
*** kbyrne has quit IRC07:44
*** kbyrne has joined #openstack-kolla07:47
*** jbadiapa_ has joined #openstack-kolla07:54
*** jbadiapa has quit IRC07:57
*** dan_ardelean has joined #openstack-kolla08:01
*** helpee has joined #openstack-kolla08:03
helpeehello08:03
*** helpee has quit IRC08:04
*** liblib has joined #openstack-kolla08:05
*** zhangfei has quit IRC08:07
*** mgoddard has joined #openstack-kolla08:11
*** dan_ardelean has quit IRC08:16
*** dan_ardelean has joined #openstack-kolla08:16
*** zhangfei has joined #openstack-kolla08:20
*** dan_ardelean has quit IRC08:21
*** adac has joined #openstack-kolla08:24
masberhi, what the database_recovery does?08:24
masberhow does it knows the right state of the database?08:25
*** rmart04 has joined #openstack-kolla08:27
*** salv-orlando has joined #openstack-kolla08:36
*** zhurong has quit IRC08:42
*** jaosorior has quit IRC08:44
openstackgerritChason Chan proposed openstack/kolla-ansible master: User more accurate links  https://review.openstack.org/51565608:45
*** dougsz has quit IRC08:45
*** jistr is now known as jistr|mtg08:48
*** stefan_bo has joined #openstack-kolla08:55
*** zshi has quit IRC08:59
*** zshi has joined #openstack-kolla09:00
*** dan_ardelean has joined #openstack-kolla09:02
*** gfidente has joined #openstack-kolla09:07
*** liblib has quit IRC09:07
gouthamHi all09:08
gouthamwhen i want to upgrade my setup09:08
gouthami mean kolla upgrade it is failing with this error https://hastebin.com/wefezoniku.md any help ??? thanks in advance09:09
*** Guest70505 has joined #openstack-kolla09:15
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible stable/pike: Fix the incorrect container_facts  https://review.openstack.org/51869309:19
*** dougsz has joined #openstack-kolla09:22
openstackgerritMerged openstack/kolla-ansible master: Fix the incorrect container_facts  https://review.openstack.org/51723709:23
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Optimise the "openstack_auth" for freezer  https://review.openstack.org/51527609:24
openstackgerritHeetae Ahn proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49514809:26
*** pomac has joined #openstack-kolla09:27
*** dan_arde_ has joined #openstack-kolla09:29
*** dan_ardelean has quit IRC09:31
*** gfidente has quit IRC09:31
*** gfidente has joined #openstack-kolla09:32
*** gfidente has quit IRC09:32
*** gfidente has joined #openstack-kolla09:32
*** EmilienM has joined #openstack-kolla09:32
*** Guest70505 has quit IRC09:35
openstackgerritChason Chan proposed openstack/kolla-ansible stable/pike: Fix the incorrect container_facts  https://review.openstack.org/51869309:37
*** obre has joined #openstack-kolla09:38
gouthamAny help on how to upgrade my setup ? i.e i want to add a new node to my setup09:39
gouthami used   ``kolla-ansible -i multinode upgrade`` by making changes to inventory file09:40
openstackgerritDuong Ha-Quang proposed openstack/kolla master: Implement Almanach Docker images  https://review.openstack.org/49893809:40
*** zhangfei has quit IRC09:42
*** skramaja_ has quit IRC09:51
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix Ansible WARNING logs on reserved name: action  https://review.openstack.org/51147309:54
*** skramaja has joined #openstack-kolla09:58
*** eduardo_ has joined #openstack-kolla09:59
eduardo_bhaiyo namaskar09:59
eduardo_kripya madad kijiye09:59
eduardo_hey dosaboy09:59
eduardo_you seem indian09:59
eduardo_are u??09:59
gouthameduardo_: yes im10:03
*** eduardo_ has quit IRC10:04
openstackgerritPierre Hanselmann proposed openstack/kolla master: Add missing vmware libs  https://review.openstack.org/51828110:08
*** egonzalez has joined #openstack-kolla10:12
egonzalezmorning10:13
*** jistr|mtg is now known as jistr10:15
chasonHi egonzalez!10:22
chasonhttps://review.openstack.org/#/c/518490/10:23
chasonegonzalez Please see my comment, ^^^^10:23
openstackgerritMerged openstack/kolla-ansible master: Murano application should contact rabbitmq by external FQDN  https://review.openstack.org/51732110:24
*** Sean-md has quit IRC10:31
egonzalezchason, until image builds are fixed for rpm images, will still be using old fluentd plugins10:32
chasonegonzalez OK, thanks!10:33
chasonegonzalez Can I do something to push forward this fix? :)10:34
berendtegonzalez: duonghq pleas also review https://review.openstack.org/#/c/517418/10:36
berendtthis is a requirement for the chrony integration in telegraf10:36
openstackgerritMerged openstack/kolla-ansible stable/pike: Fix the incorrect container_facts  https://review.openstack.org/51869310:36
openstackgerritMerged openstack/kolla-ansible master: Optimise the "openstack_auth" for freezer  https://review.openstack.org/51527610:36
openstackgerritMerged openstack/kolla-ansible master: Add the full name of PBR in its first usage  https://review.openstack.org/51721310:36
egonzalezchason, check change 516691 which removes ceilometer-api images10:40
egonzalezAnd 518610 to fix designate builds10:40
chasonegonzalez OK, very appreciated! :D10:41
*** karmabytes has quit IRC10:43
*** s-dean has quit IRC10:43
*** ktibi has joined #openstack-kolla10:48
*** jascott1 has joined #openstack-kolla10:54
*** kevzha01 has joined #openstack-kolla10:54
*** kevinz has quit IRC10:55
*** hachi_ has quit IRC10:57
*** hachi_ has joined #openstack-kolla10:57
*** jascott1 has quit IRC10:58
*** hachi__ has joined #openstack-kolla10:59
*** hachi_ has quit IRC11:01
duonghqberendt, done11:02
openstackgerritMerged openstack/kolla master: Fixes the Deploy Registry link  https://review.openstack.org/51830011:07
berendtduonghq: thanks11:07
berendtduonghq: could you please review https://review.openstack.org/#/c/518529/ and https://review.openstack.org/#/c/515532/11:08
berendtduonghq: no-brainer waiting for approval11:08
duonghqberendt, done11:09
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Implement blazar ansible role  https://review.openstack.org/45475211:10
berendtduonghq: thanks11:10
*** pbourke has quit IRC11:10
duonghqberendt, I need going out now, see you later11:11
*** duonghq has quit IRC11:12
*** pbourke has joined #openstack-kolla11:12
openstackgerritMerged openstack/kolla-ansible master: Remove trailing whitespaces  https://review.openstack.org/51553211:14
*** ansmith has quit IRC11:16
*** ansmith_ has quit IRC11:17
*** zshi has quit IRC11:21
openstackgerritMerged openstack/kolla-ansible stable/pike: Remove check flat network type task in ironic pre-check.yml  https://review.openstack.org/51619411:23
openstackgerritzenghui.shi proposed openstack/kolla master: Add ptp service docker image  https://review.openstack.org/51370311:41
*** jaosorior has joined #openstack-kolla11:47
berendtanother no-brainer waiting for approval: https://review.openstack.org/#/c/508470/11:52
berendtas well as https://review.openstack.org/#/c/50847111:53
gouthamHi all when i m trying to deploy using kolla-ansible it is failing with the error https://pasteboard.co/GSPUw7G.png any help11:54
goutham??11:54
*** salv-orlando has quit IRC11:58
*** egonzalez90 has joined #openstack-kolla12:07
openstackgerritChristian Berendt proposed openstack/kolla master: Add squash parameter to kolla-build  https://review.openstack.org/51871712:08
*** zhubingbing_ has joined #openstack-kolla12:09
*** egonzalez has quit IRC12:10
*** zhubingbing__ has quit IRC12:11
kolla-slack<aslancimbom100> goutham is docker running on 192.168.122.82 node?12:13
*** salv-orlando has joined #openstack-kolla12:14
gouthamyes kolla-slack12:14
gouthamand docker info points to the insecure registry12:14
gouthamshows that it points to the insecure registry correctly **12:15
openstackgerritMerged openstack/kolla-ansible master: Add missing NOTE marker to site.yml  https://review.openstack.org/50847012:15
openstackgerritMerged openstack/kolla-ansible master: Fix name of nova-hyperv play in site.yml  https://review.openstack.org/50847112:15
gouthamkolla-slack: BTW i m trying to add a node to the existing setup12:16
gouthamegonzalez suggested to use kolla-ansible deploy to add a node to the existing setup12:17
egonzalez90goutham, try running docker pull<image> manually12:20
*** mandre_ has joined #openstack-kolla12:22
gouthamok i wil try that12:22
goutham docker pull ubuntu Using default tag: latest Error response from daemon: Get https://registry-1.docker.io/v2/: dial tcp 52.20.146.203:443: getsockopt: connection refused12:24
*** zhubingbing_ has quit IRC12:24
gouthamMy node doesnt have internet connectivity12:24
egonzalez90Try pulling one of the kolla images in your private registry12:25
egonzalez90To pull a public image need internet12:25
gouthamyes12:25
gouthamthere is an image with name ubuntu in private registry12:26
gouthamdocker pull 192.168.122.192:5443/kolla/ubuntu-source-base Using default tag: latest Error response from daemon: manifest for 192.168.122.192:5443/kolla/ubuntu-source-base:latest not found12:26
*** salv-orlando has quit IRC12:27
egonzalez90Add the tag :5.0.0 or the version you have12:27
*** salv-orlando has joined #openstack-kolla12:27
gouthamit is working12:27
gouthamyes yes i have added the tag and it started working12:28
egonzalez90Btw, what docker or docker-py have installed in the host failing?12:28
egonzalez90Should be docker, not docker-py12:29
gouthamthere is a version change in the previous version machines and the new one in docker.__version__ ==2.6.1 in the new one where is in the already present ones it 2.5.112:31
*** salv-orlando has quit IRC12:32
goutham>>> import docker >>>>> print docker.__version__ 2.5.112:32
gouthamin the old machines and in the new machine >>> import docker >>> print docker.__version__ 2.6.112:32
gouthamwill this be an issue??12:33
*** dougsz has quit IRC12:34
berendtJeffrey4l: egonzalez90: congratulations to the contributor award!12:35
gouthamcongrats egonzalez9012:36
openstackgerritMerged openstack/kolla-ansible stable/ocata: Remove warning during kolla_docker execution  https://review.openstack.org/51568012:36
gouthamegonzalez90: https://hastebin.com/vifetojiwi.py some different error this time12:46
*** ansmith has joined #openstack-kolla12:55
*** ansmith_ has joined #openstack-kolla12:55
gouthamegonzalez90:12:55
gouthamthat error is gone12:56
gouthamdeployment is sucessful13:05
gouthamissue with docker py13:06
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/50944313:15
*** coolsvap has quit IRC13:21
*** egonzalez90 has quit IRC13:28
*** salv-orlando has joined #openstack-kolla13:28
*** salv-orlando has quit IRC13:32
openstackgerritMerged openstack/kolla master: Install chrony in telegraf image  https://review.openstack.org/51741813:39
*** mwall has joined #openstack-kolla13:47
*** jaosorior has quit IRC13:51
*** skramaja has quit IRC13:53
*** jaosorior has joined #openstack-kolla13:53
mwallhi all, i couldn't find anything inside the documentation/configuration about ceph using filestore or bluestore. i run into trouble with glance client expecting my pool to have a feature based on bluestore, but my ceph runs on filestore. Can i change to bluestore easily or does one of you had a similar issue?13:55
*** jascott1 has joined #openstack-kolla13:56
*** jascott1 has quit IRC14:00
*** salv-orlando has joined #openstack-kolla14:09
*** lukepatrick has joined #openstack-kolla14:10
*** janki has quit IRC14:17
*** cah_link has joined #openstack-kolla14:32
*** hachi_ has joined #openstack-kolla14:35
*** hachi__ has quit IRC14:37
*** salv-orlando has quit IRC14:41
*** salv-orlando has joined #openstack-kolla14:42
*** lukepatrick has quit IRC14:43
*** cah_link has quit IRC14:45
*** cah_link has joined #openstack-kolla14:45
*** salv-orlando has quit IRC14:47
*** salv-orlando has joined #openstack-kolla14:49
cah_linkHi together. how can i asure mod_ssl is enabled vor the containers? kolla_enable_tls_external: "yes" doesn't do that14:53
*** hachi__ has joined #openstack-kolla15:00
cah_linki am using 5.0.015:00
*** ansiwen[q] has quit IRC15:02
*** ansiwen[q] has joined #openstack-kolla15:02
*** hachi_ has quit IRC15:02
*** ansiwen[q] has quit IRC15:05
inc0good morning15:11
inc0behold! images on dockerhub https://hub.docker.com/r/kolla/ubuntu-source-keystone/tags/15:11
openstackgerritMerged openstack/kolla master: Implement Almanach Docker images  https://review.openstack.org/49893815:12
dmsimard|offinc0: re: parallel push.. seeing push errors when I put push_threads >115:15
dmsimard|off¯\_(ツ)_/¯15:15
inc0:(15:15
dmsimard|offinc0: can you point me to the logs for one of the jobs that pushes to dockerhub ? curious15:19
inc0dmsimard|off: http://logs.openstack.org/periodic/git.openstack.org/openstack/kolla/master/15:21
dmsimard|offinc0: looks like ~2 hrs15:22
dmsimard|offwhich is on par with what we're getting for building and pushing a set of images15:22
dmsimard|offwhich sucks btw15:23
*** zhubingbing_ has joined #openstack-kolla15:23
inc0well yeah15:24
dmsimard|offbuilding is fast, it's the pushing that is awful15:24
dmsimard|offyou're hitting the same bottleneck as us15:24
dmsimard|offyou start pushing at 06:45 and the job ends at 08:0015:25
inc0its both disk and network io15:25
dmsimard|offnot really15:25
dmsimard|offour machine is almost idle when it's pushing15:26
inc0well, dockerhub is also fairly slow15:26
openstackgerritMerged openstack/kolla master: Add Apache packages to mistral-api container  https://review.openstack.org/51308915:26
inc0I'll need to try this sequential push on local registry15:26
dmsimard|offWe're getting the same performance on a private registry that is on the same network, same cloud and probably even the same rack15:26
inc0ok15:26
dmsimard|offFrom looking a little bit at the logs on the registry side15:27
inc0and you're using openshift registry15:27
inc0?15:27
dan_arde_is there a doc explaining how to deploy openstack in a HA manner with kolla?15:27
inc0dan_arde_: HA is default15:27
inc0if you put more than 3 nodes, it's going to be HA15:28
inc0well 2 nodes would work too15:28
dan_arde_even for the the database or amqp?15:28
inc03 nodes is healthier15:28
inc0yes15:28
dan_arde_oh15:28
dmsimard|offYou can see the docker client check if each layer is already there before pushing it, it's I/O bound on the registry side -- the registry does a lookup for the image sha1 and returns 200 or 404, if it's 404, the client pushes. I believe the fact that there is many layers to each image, it exponentially increases the push time due to the lookup of each layer15:29
dan_arde_galera for the DB I assume15:29
inc0dan_arde_: yes15:29
inc0dmsimard|off: yeah that makes sense15:29
dan_arde_still there should be a statement somewhere in the docs stating this15:29
inc0sooo one option would be to write local cache of pushed layers15:29
dan_arde_oh, I think I found it https://github.com/openstack/kolla-ansible/blob/7bb88390d88aebfc257e09e7e055a09fe4121782/specs/high-availability.rst15:29
inc0but that'd be interesting rewrite of push mechanism15:30
inc0that's super old dan_arde_ :)15:30
dmsimard|offinc0: The pro of having many layers is shared utilization (less storage, less bandwidth) but I guess there's cons too15:30
dan_arde_i  see :(15:30
inc0right, that's why loci exists dmsimard|off15:30
inc0the thing is they're single layer15:31
*** jtriley has joined #openstack-kolla15:31
dan_arde_any better documentation on this for the moment?15:31
dmsimard|offinc0: local cache of pushed layers? the docker client will push all the layers of the image anyway15:31
inc0dan_arde_: well...architecturaly no15:31
inc0deployment - as I said, it's default15:31
inc0dmsimard|off: how about pushing through http proxy?15:31
inc0well it would have to be clever cache15:32
inc0as in, if you check for layer existance, cache response15:32
dmsimard|offinc0: I don't know... if you search for "docker push slow" on Google, it seems like it's a fact of life that it's slow15:32
inc0if it's existing15:32
dmsimard|offThere's things ranging from compression algorhitm issue, to sha1 layer lookups, etc.15:33
inc0our case is different than most tho15:33
inc0we push a lot of images15:33
inc0not one or two15:33
dan_arde_inc0 outside the box HA? this sounds to good to be true :D but it is true15:33
inc0with massive amount of sharing15:33
dmsimard|offinc0: yeah, I get that, and that's fine15:33
inc0but I think we can write small cache to speed it up massively15:34
dmsimard|offinc0: but we're building those like 12 times a day and taking 1hr to push things is really awful15:34
inc0when client checks if image layer exist -> if it does, cache response15:34
inc0then 200 images asking for same layers will speed up15:35
*** mwall has quit IRC15:36
dmsimard|offdunno15:40
dmsimard|offwe need to figure something out though15:40
inc0yeah I agree15:41
dmsimard|offjust nova-api is like 30 layers15:41
dmsimard|offno lie15:41
inc0right, that makes sense15:41
inc0it has 4 levels of images15:41
inc0there is this experimental --squash15:42
inc0but that will make build longer15:42
dmsimard|offinc0: I forgot to answer your question but yes we're using openshift registry15:42
inc0sooo...maybe it's good idea to try to optimize it?15:43
dmsimard|offIt's been working fairly well for us and we've been working with upstream to make our use case better15:43
dmsimard|offinc0: optimize what? squash ?15:43
*** jamesbenson has joined #openstack-kolla15:43
inc0no, openshift registry15:43
dmsimard|offopenshift registry is nothing magic, it's a (vendored) docker-registry container running in a pod -- openshift, however, patches a lot of holes/missing features from docker-registry15:44
inc0dmsimard|off: anyway...try to cover it with squid15:45
dmsimard|offeasy acl/rbac (we use github oauth), docker-hub like interface, easy image pruning/deletion15:45
inc0that'd give you quick speedup of checking for layers15:45
dmsimard|offmaybe15:46
inc0nova-api has 40 layers and 35 of them are shared15:46
inc0so no need for checking for their existance more than twice (once for 404 -> push -> 200)15:46
inc0docker registry is very dump in way it stores stuff15:47
inc0it's just dirs15:47
inc0so every push is 40 * stat /var/lib/docker/registry.../blobs/12312312315:48
dmsimard|offsomething like that, yeah15:48
dmsimard|offtimes 125+ images15:49
inc0if I'm right, cache would help tremendously15:49
inc0well with this part of push anyway15:49
*** jamesbenson has quit IRC15:50
dmsimard|offWorth considering, dunno15:52
inc0right, more research required15:52
*** jamesbenson has joined #openstack-kolla15:53
dmsimard|offAlso, the bigger the registry, the slower it becomes because of all those lookups..15:54
inc0yeah, I wonder what they did in dockerhub15:54
inc0it's still slow15:54
inc0but not 100s of thousands of images slwo15:55
inc0some sharding of layers?15:55
inc0(which wouldn't be bad idea for scalability of it)15:55
inc0anyway, if I'd want to make $$$ of cloud startup - scalable, managable registry for k8s would be my bet15:56
dmsimard|offopenshift registry is exactly that btw :p15:56
inc0right;) I'd fail my startup:(15:57
dmsimard|offBasically it's the system registry of openshift/k8s, that is used when you, like, click I want a Jenkins app15:57
dmsimard|offIn a typical deployment that registry stays private, a standalone registry exposes it publicly15:57
inc0I think openshift could really use tight integration with helm15:57
inc0politics aside15:58
inc0helm install jenkins is supposed to do exactly what you described15:58
dmsimard|offNever used helm before and can't pretend I'm an openshift expert either so dunno15:58
inc0just make dedicated charts/images15:58
inc0it's nothing special - hosted templates for k8s resources15:58
inc0so helm install jenkins chart will pull template for jenkins-on-k8s, render it and deploy it15:59
inc0https://github.com/kubernetes/charts/tree/master/stable15:59
*** hachi__ has quit IRC16:01
*** jaosorior has quit IRC16:02
*** itlinux has joined #openstack-kolla16:03
*** zhubingbing has joined #openstack-kolla16:07
*** zhubingbing_ has quit IRC16:07
*** lukepatrick has joined #openstack-kolla16:14
ktibiHi, the bind server of designate listen on private network. For me it's not good because the internal can't join by other dns or anythin ouside of private network. What do you think about that ?16:19
ktibihttps://github.com/openstack/kolla-ansible/blob/master/ansible/roles/designate/templates/named.conf.j2#L316:20
inc0ktibi: hmm...should we add haproxy endpoint over it?16:21
ktibiit's an UDP protocol16:22
ktibihaproxy doesn't support UDP16:23
inc0ah damn16:23
inc0yeah I guess we need to add external port to named too16:23
*** itlinux has quit IRC16:23
*** itlinux has joined #openstack-kolla16:25
*** zhubingbing__ has joined #openstack-kolla16:26
*** zhubingbing has quit IRC16:27
ktibiinc0 easy to add second IP on listen-on, public network by example16:30
inc0ktibi: sounds like great patch to propse;)16:31
inc0if you want to, I'll be happy to help you thorugh process and provide any help I have to offer16:32
*** mdnadeem|afk has quit IRC16:32
inc0I didn't mean to say this twice...lack of caffeine :(16:32
ktibiYeah I already tried to use gerrit without success.....16:32
inc0ok, then let's try again, I'll help you16:33
inc01. did you create gerrit account?16:33
*** rmart04 has left #openstack-kolla16:33
*** lukepatrick has quit IRC16:37
*** adac has quit IRC16:45
*** litmaj0r has joined #openstack-kolla16:46
*** lukepatrick has joined #openstack-kolla16:46
*** ntpttr_laptop has joined #openstack-kolla16:53
*** dan_arde_ has quit IRC16:55
*** dan_ardelean has joined #openstack-kolla16:55
*** dan_ardelean has quit IRC17:00
inc0ktibi: so what I'm thinking is to add new variable called named_listen_host or sth like this17:02
*** jbadiapa_ has quit IRC17:02
inc0https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/haproxy/templates/haproxy.cfg.j2#L82-L8917:02
inc0and if it's overriden, to say 0.0.0.017:02
inc0it will not use regular ip17:03
ktibiinc0 for bind it's any17:03
ktibiI don't know if 0.0.0.0 works17:03
inc0well17:03
inc0you know what I mean17:03
inc0maybe have variable "named_listen_on_all"17:04
inc0well hold on17:04
ktibia bool variable ?17:04
inc0yeah17:04
inc0well17:04
inc0we only have external interface17:05
ktibiok I see, bool variable for force any listen, and a variable listen_host with default value ==> interne and external IP17:05
inc0yeah, that would make sense17:05
inc0but we'll allow other reviewers to provide feedback too17:06
ktibiI don't know how work kolla with all-in-one17:06
inc0we don't have extenral ip really17:06
inc0that's the issue17:06
*** cah_link has quit IRC17:06
ktibihum ip for public endpoint ?17:06
inc0kolla_external_vip_interface we have this (potentially)17:07
inc0problem is, ti might not have IP17:07
inc0because for now it's only for keepalived17:07
*** litmaj0r has quit IRC17:07
inc0and keepalived adds external vip which is what's important17:08
inc0well17:08
inc0I guess we can use it17:08
inc0that would provide HA17:08
ktibihum ok I use kayobe, and each node in my cluster have an IP17:08
ktibiin any network17:08
inc0but I don't know how named would work if started on host without IP configured17:08
inc0if it's ok17:08
ktibiit fail17:08
inc0damn17:08
*** litmaj0r has joined #openstack-kolla17:09
inc0hmm...so we'll have no HA on this?:(17:10
ktibino17:10
inc0anyway17:10
ktibinamed is in cluster mode17:10
inc0let's check if IP exist17:11
inc0and use variable kolla_external_vip_interface17:11
inc0but also we need to make sure it's not default17:11
inc0because there will be some requirements for it17:11
inc0like...well...designate landing on network nodes17:12
inc0(potentially, because that's where we require external interface)17:12
inc0(if you want it at all)17:12
inc0aaaand it doesn't17:13
inc0so bottom line17:13
inc0let's add variable "bind_listen_on_external"17:13
inc0(or name like this)17:13
inc0default to false17:13
inc0and if it's set to true, it will add another listening host which on external_vip_interface17:14
inc0and wait for other reviewers to check it out17:14
ktibi{{ hostvars[inventory_hostname]['ansible_' + hostvars[inventory_hostname]['neutron_external_interface']]['ipv4']['address'] }} can works ?17:14
inc0no it won't work17:15
inc0neutron_external_interface is taken over by ovs17:15
inc0there is no L3 termination on network node17:15
inc0you want kolla_external_vip_interface17:15
inc0https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/haproxy/templates/keepalived.conf.j2#L1817:16
inc0this is only place this interface is used that I can think of17:16
ktibiif not we use variable for bind with default value = {{ hostvars[inventory_hostname]['ansible_' + hostvars[inventory_hostname]['api_interface']]['ipv4']['address'] }} and if people want to override we can :/17:16
inc0I think we'll need both external and internal tho17:16
*** rmart04 has joined #openstack-kolla17:22
ktibiinc0 end of work in France ;) I'll work on that tomorrow :)17:25
inc0np17:25
inc0reach out to egonzalez17:25
inc0he's in Spain17:25
inc0he is better at designate than me:)17:26
ktibiyes I know him ;)17:26
inc0just before you go17:26
*** zhubingbing__ has quit IRC17:26
inc0wanna do quick test of pushing code to gerrit?17:26
*** ntpttr_laptop has quit IRC17:26
ktibiyep17:26
inc0some fake change, just to make sure it's working17:26
inc0try it17:27
inc0do any commit, add that it's fake or sth in commit message17:27
inc0and run git review17:27
ktibion fake branch ?17:27
ktibior on master ?17:27
inc0branch doesn't matter in gerrit that much17:27
ktibik17:27
ktibicommitter email address .... does not match your user account.17:28
ktibiso need to change email ;)17:28
inc0I think you can have more than one in gerrit17:31
inc0https://review.openstack.org/#/settings/contact17:31
ktibican I add email just for .gitreview ?17:31
inc0hmm17:31
inc0not sure17:31
*** jascott1 has joined #openstack-kolla17:32
openstackgerritKevin Tibi proposed openstack/kolla-ansible master: modify bind on named conf  https://review.openstack.org/51876217:32
ktibiO_o17:32
ktibimy first gerrit commit T_O17:33
inc0yay17:34
inc0cool17:34
inc0tomorrow I'll explain you how to address reviews and stuff17:34
inc0and few useful tricks17:34
ktibiomg now I can push all my local modif for kolla mouhaha17:34
ktibiyeah thx ;) bye17:35
inc0have a great night ktibi17:35
*** dan_ardelean has joined #openstack-kolla17:36
*** dardelean has joined #openstack-kolla17:37
*** dan_ardelean has quit IRC17:40
dardeleanrwellum you around?17:43
*** litmaj0r has quit IRC18:00
*** litmaj0r has joined #openstack-kolla18:01
*** jascott1 has quit IRC18:04
*** itlinux has quit IRC18:04
*** rmart04 has quit IRC18:05
*** itlinux has joined #openstack-kolla18:05
*** jascott1 has joined #openstack-kolla18:10
*** jascott1 has quit IRC18:10
*** jascott1 has joined #openstack-kolla18:16
*** hachi_ has joined #openstack-kolla18:33
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode scenarios in zuulv3  https://review.openstack.org/51277918:38
*** ansmith has quit IRC19:05
*** ansmith_ has quit IRC19:05
*** dave-mcc_ has joined #openstack-kolla19:17
*** ansmith_ has joined #openstack-kolla19:20
*** ansmith has joined #openstack-kolla19:20
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode scenarios in zuulv3  https://review.openstack.org/51277919:20
*** mwall has joined #openstack-kolla19:27
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla-ansible master: Enable multinode scenarios in zuulv3  https://review.openstack.org/51277919:29
inc0dmsimard|off: still around?19:35
*** adac has joined #openstack-kolla19:47
*** hachi_ has quit IRC19:47
*** ansmith_ has quit IRC19:58
*** ansmith has quit IRC19:58
*** ansmith_ has joined #openstack-kolla19:58
*** ansmith has joined #openstack-kolla19:58
*** dgedia has joined #openstack-kolla20:00
*** litmaj0r has quit IRC20:01
*** dave-mcc_ has quit IRC20:06
*** jbrooks has quit IRC20:11
*** salv-orlando has quit IRC20:12
*** mgoddard has quit IRC20:12
*** salv-orlando has joined #openstack-kolla20:12
*** itlinux has quit IRC20:16
*** salv-orlando has quit IRC20:17
*** jbrooks has joined #openstack-kolla20:18
*** salv-orlando has joined #openstack-kolla20:20
dmsimard|offinc0: not in any official capacity :)20:22
inc0ahh20:22
inc0I'll leave you to your leave then;)20:22
*** mgoddard has joined #openstack-kolla20:31
*** jtriley has quit IRC20:34
inc0jascott1: just saw on twitter - terraform + ansible = terrible20:34
jascott1terribly awesome20:35
*** jtriley has joined #openstack-kolla20:36
*** mgoddard has quit IRC20:41
*** dgedia has quit IRC20:54
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla stable/pike: Moving jobs to kolla repo  https://review.openstack.org/51360720:57
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla stable/pike: Make binary gates non-voting  https://review.openstack.org/51879520:57
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla stable/pike: Zuul: add file extension to playbook path  https://review.openstack.org/51879620:57
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla stable/pike: Dockerhub publisher job  https://review.openstack.org/51879720:57
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla stable/pike: Fixup small bug with dockerhub publisher  https://review.openstack.org/51879820:57
*** jtriley has quit IRC20:59
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla stable/pike: Moving jobs to kolla repo  https://review.openstack.org/51360721:05
*** itlinux has joined #openstack-kolla21:12
*** ansmith_ has quit IRC21:23
*** ansmith has quit IRC21:23
*** itlinux has quit IRC21:29
*** ktibi has quit IRC21:38
*** mwall has quit IRC21:44
*** hrw has quit IRC21:49
*** adac has quit IRC21:54
*** hrw has joined #openstack-kolla21:58
*** itlinux has joined #openstack-kolla22:07
*** ansmith_ has joined #openstack-kolla22:13
*** ansmith has joined #openstack-kolla22:13
*** dave-mccowan has joined #openstack-kolla22:23
*** jascott1 has quit IRC22:27
*** dave-mccowan has quit IRC22:29
*** dave-mccowan has joined #openstack-kolla22:36
*** jascott1 has joined #openstack-kolla22:37
*** mgoddard has joined #openstack-kolla22:41
*** mgoddard has quit IRC22:48
*** lukepatrick has quit IRC22:51
*** Pavo has joined #openstack-kolla22:55
sdakeheads up - worth a read imo :) https://blogs.cisco.com/cloud/openstack-solving-for-integration-in-open-source-adjacent-communities22:59
sdakespsurya i just saw you were a core reviewer - I must have missed the vote during travel - nice job!23:00
*** salv-orlando has quit IRC23:00
*** jamesbenson has quit IRC23:01
*** salv-orlando has joined #openstack-kolla23:01
*** kbyrne has quit IRC23:02
*** kbyrne has joined #openstack-kolla23:02
*** gfidente has quit IRC23:05
*** salv-orlando has quit IRC23:06
*** dklyle has quit IRC23:20
*** dave-mccowan has quit IRC23:26
*** itlinux has quit IRC23:26
masbermorning all, how can i check nova logs from a compute node?23:31
*** newmember has quit IRC23:32
*** newmember has joined #openstack-kolla23:33
masberhello?23:39
*** dardelean has quit IRC23:42
*** dan_ardelean has joined #openstack-kolla23:47
*** jbrooks has quit IRC23:47
*** jbrooks has joined #openstack-kolla23:49
*** gfidente has joined #openstack-kolla23:50
*** Pavo has quit IRC23:50
*** jbrooks has quit IRC23:54
*** jbrooks has joined #openstack-kolla23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!