Monday, 2017-08-28

*** jascott1 has joined #openstack-kolla00:09
*** jascott1 has quit IRC00:13
*** hieulq has joined #openstack-kolla00:33
*** duonghq has joined #openstack-kolla00:49
duonghqmorning guys00:49
*** yangyapeng has joined #openstack-kolla00:57
*** StephenWang1991 has quit IRC00:58
*** yangyapeng has quit IRC00:58
*** yangyape_ has joined #openstack-kolla01:00
*** tovin07_ has joined #openstack-kolla01:00
*** StephenWang1991 has joined #openstack-kolla01:00
*** yangyapeng has joined #openstack-kolla01:01
*** yangyape_ has quit IRC01:04
*** chas has joined #openstack-kolla01:17
openstackgerritJeffrey Zhang proposed openstack/kolla master: Move to pike release for OpenStack services  https://review.openstack.org/49555501:18
*** daidv has joined #openstack-kolla01:20
*** chas has quit IRC01:21
openstackgerritchao liu proposed openstack/kolla-ansible master: writing convention set to use "." to source script files  https://review.openstack.org/49822001:22
*** yingjun has joined #openstack-kolla01:28
*** zhurong has quit IRC01:28
openstackgerritoctopuszhang proposed openstack/kolla-ansible master: Add a note to help user avoid docker start failure  https://review.openstack.org/49777501:30
*** caowei has joined #openstack-kolla01:41
openstackgerritchao liu proposed openstack/kolla-ansible master: Correct a grammar mistake  https://review.openstack.org/49818901:44
*** zhangfei has joined #openstack-kolla01:52
*** yangyapeng has quit IRC02:02
*** StephenWang1991 has quit IRC02:03
*** yangyapeng has joined #openstack-kolla02:04
*** StephenWang1991 has joined #openstack-kolla02:05
*** StephenW_ has joined #openstack-kolla02:11
*** StephenWang1991 has quit IRC02:14
*** glycerine has quit IRC02:17
*** zhurong has joined #openstack-kolla02:38
*** itlinux has quit IRC02:45
openstackgerritDuong Ha-Quang proposed openstack/kolla-ansible master: Implement Keystone zero-downtime upgrade  https://review.openstack.org/42544602:50
*** chas has joined #openstack-kolla03:18
*** chas has quit IRC03:22
spsuryamorning all03:23
spsuryaduonghq: morning03:23
duonghqmorning spsurya03:26
*** gkadam has joined #openstack-kolla03:33
*** yangyapeng has quit IRC03:45
*** ysheng has quit IRC04:00
masbergood morning all, I am using ceph as a data plane, any idea how can  I do to have ephemeral disks as local instead of using ceph?04:16
*** jascott1 has joined #openstack-kolla04:19
*** yangyapeng has joined #openstack-kolla04:22
*** jascott1 has quit IRC04:23
*** yangyapeng has quit IRC04:27
*** yangyapeng has joined #openstack-kolla04:28
duonghqanybody get this error? "/usr/local/bin/kolla_start: line 17: /usr/sbin/td-agent: No such file or directory"04:40
duonghqI get it quite frequently04:40
*** StephenW_ has quit IRC04:42
*** StephenWang1991 has joined #openstack-kolla04:50
*** StephenWang1991 has quit IRC04:52
*** StephenW_ has joined #openstack-kolla04:52
*** StephenWang1991 has joined #openstack-kolla04:54
*** StephenW_ has quit IRC04:54
*** jascott1 has joined #openstack-kolla04:55
*** StephenWang1991 has quit IRC04:59
*** StephenWang1991 has joined #openstack-kolla04:59
*** jascott1 has quit IRC04:59
*** StephenWang1991 has quit IRC05:03
*** StephenWang1991 has joined #openstack-kolla05:03
*** pbourke has quit IRC05:08
*** caowei has quit IRC05:09
*** pbourke has joined #openstack-kolla05:10
*** yangyapeng has quit IRC05:12
*** yangyapeng has joined #openstack-kolla05:12
*** jaosorior has joined #openstack-kolla05:18
*** chas has joined #openstack-kolla05:19
*** chas has quit IRC05:23
*** jascott1 has joined #openstack-kolla05:33
*** mdnadeem has joined #openstack-kolla05:39
*** mdnadeem has quit IRC05:40
*** mdnadeem has joined #openstack-kolla05:40
*** caowei has joined #openstack-kolla05:43
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla master: Changed as per the standard convention set to use "."  https://review.openstack.org/49831605:49
*** jbrooks has quit IRC05:51
*** yangyapeng has quit IRC06:02
*** yangyapeng has joined #openstack-kolla06:05
*** magicboiz has joined #openstack-kolla06:08
*** yangyapeng has quit IRC06:10
*** yangyapeng has joined #openstack-kolla06:11
*** yangyapeng has quit IRC06:22
*** yangyapeng has joined #openstack-kolla06:23
*** unicell has joined #openstack-kolla06:28
*** StephenWang1991 has quit IRC06:34
*** StephenWang1991 has joined #openstack-kolla06:35
*** unicell has quit IRC06:35
*** pcaruana has joined #openstack-kolla06:36
*** ntpttr_laptop has joined #openstack-kolla06:42
*** gfidente has joined #openstack-kolla06:44
*** gkadam_ has joined #openstack-kolla06:45
*** ntpttr_laptop has quit IRC06:46
*** gkadam has quit IRC06:47
*** dmsimard has quit IRC06:50
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd/Kibana can't sort logs by log level  https://review.openstack.org/49833106:58
kolla-slack2<egonzalez> duonghq, use latest images and rebase master, was a change in rpm distros06:59
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd/Kibana can't sort logs by log level  https://review.openstack.org/49833107:01
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Changed as per the standard convention set to use "."  https://review.openstack.org/49833207:01
*** janki has joined #openstack-kolla07:04
*** janki has quit IRC07:05
*** janki has joined #openstack-kolla07:05
*** seanmurphy has joined #openstack-kolla07:06
duonghqegonzalez: I use images built myself, will try again07:13
*** yingjun has quit IRC07:14
openstackgerritjiangpch proposed openstack/kolla master: fix image build error of fluentd  https://review.openstack.org/49833907:15
*** dmsimard has joined #openstack-kolla07:20
*** noVNC has joined #openstack-kolla07:20
noVNChello07:21
noVNCis posible to specify MTU in kolla ?07:21
noVNCto support jumbo frames07:22
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd "progname" field not always generated  https://review.openstack.org/49834307:22
*** manheim has joined #openstack-kolla07:24
*** chas has joined #openstack-kolla07:25
*** egonzalez has joined #openstack-kolla07:42
*** coolsvap has joined #openstack-kolla07:52
openstackgerritMerged openstack/kolla-ansible master: writing convention set to use "." to source script files  https://review.openstack.org/49822007:54
*** zhubingbing has joined #openstack-kolla08:03
egonzalezcan we have reviews into https://review.openstack.org/#/c/498339/ , is the fix for current image building failures08:03
*** glycerine has joined #openstack-kolla08:14
openstackgerritJeffrey Zhang proposed openstack/kolla master: Restrict ruby public_suffix package version to 2.0.5  https://review.openstack.org/49836008:19
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd "progname" field not always generated  https://review.openstack.org/49834308:23
*** jaosorior has quit IRC08:39
*** zhubingbing has quit IRC08:43
*** zhubingbing has joined #openstack-kolla08:47
*** jaosorior has joined #openstack-kolla08:50
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: Libvirt logs are missing  https://review.openstack.org/49837308:52
openstackgerritJeffrey Zhang proposed openstack/kolla master: Move to pike release for OpenStack services  https://review.openstack.org/49555508:57
spsuryamandre: regarding https://review.openstack.org/#/c/498316/109:02
spsuryaping09:02
*** lpetrut has joined #openstack-kolla09:03
spsuryathen i will change for whole codebase09:04
*** mgoddard has joined #openstack-kolla09:05
mandrespsurya: ack, I found a few occurrences in the docker dir09:06
spsuryamandre: yes i also found few09:07
spsuryachanging the same09:07
*** noVNC has quit IRC09:08
*** mgoddard has quit IRC09:14
hrwmorning09:16
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla master: Changed as per the standard convention set to use "."  https://review.openstack.org/49831609:22
openstackgerritEduardo Gonzalez proposed openstack/kolla master: Fix gates out of disk space  https://review.openstack.org/49740809:23
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla master: Changed as per the standard convention set to use "."  https://review.openstack.org/49831609:28
spsuryamandre: done :)09:28
mandrespsurya: yep, thanks, already +2'ed09:29
spsuryamandre: check again09:29
mandrespsurya: just did :)09:30
spsurya👍09:31
spsuryathanks09:31
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: releasenotes: expand note about aarch64/ppc64le support  https://review.openstack.org/49839709:32
hrwcan you guys look at ^^?09:32
hrwI hope that I got prelude part right09:33
*** chas has quit IRC09:33
spsuryahrw: done09:37
hrwthanks09:37
hrwwriting release notes for Debian now09:37
spsuryaplease09:40
openstackgerrityang wang proposed openstack/kolla-ansible master: Add a note to help user avoid kolla-ansible installation error  https://review.openstack.org/49840109:40
*** ipsecguy has quit IRC09:42
*** ipsecguy has joined #openstack-kolla09:43
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: releasenotes: note Debian related changes  https://review.openstack.org/49840209:44
hrwdone09:45
egonzalezhrw, i dont think prelude is for single changes notes09:48
egonzalezshould be a main commit including a description of all kolla release with main features description09:48
hrwegonzalez: imho going from 'Debian support does not work' to 'You can use Debian based images' is kind of big change09:50
egonzalezs/should/IMO should/09:50
hrwegonzalez: probably needs rewording, just do not have idea for it09:50
egonzalezhrw, yeah i know is a big change, what i says is prelude release notes be a commit for all kolla release notes, with a release description09:50
hrwbtw - is there a way to make reno to generate html instead of markdown/rst?09:51
egonzaleza summary of the release, check nova as example https://review.openstack.org/#/c/491424/09:52
egonzalezreno is rst only iirc09:52
hrwthx09:52
hrwdropped prelude part09:53
openstackgerritMarcin Juszkiewicz proposed openstack/kolla master: releasenotes: note Debian related changes  https://review.openstack.org/49840209:53
openstackgerritLingyong Xu proposed openstack/kolla-ansible master: Fix to use . to source script files  https://review.openstack.org/49840809:54
egonzalezhrw, thanks, maybe in this week meeting we can create a summary of important features09:55
hrwegonzalez: I will be not present at meeting due to other duties09:56
hrwegonzalez: from my side important features to mention are: aarch64/ppc64le support, Debian09:56
egonzalezhrw, roger, no problem09:56
openstackgerritDan Ardelean proposed openstack/kolla master: Change networking-hyperv version for Pike  https://review.openstack.org/49840909:59
*** yangyapeng has quit IRC10:19
*** StephenWang1991 has quit IRC10:19
*** StephenWang1991 has joined #openstack-kolla10:20
*** StephenWang1991 has quit IRC10:24
*** tovin07_ has quit IRC10:28
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-kubernetes master: Changed as per the standard convention set to use "."  https://review.openstack.org/49833210:42
*** Pavo has joined #openstack-kolla10:45
openstackgerritSurya Prakash (spsurya) proposed openstack/kolla-ansible master: Changed as per the standard convention set to use "."  https://review.openstack.org/49841710:45
*** Pavo has quit IRC10:49
*** caowei has quit IRC10:55
*** zhangfei has quit IRC10:58
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Only copy nfs_shares to cinder-volume groups  https://review.openstack.org/49779411:01
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Fix typo in cinder guide for starting nfs  https://review.openstack.org/49842211:01
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: process only required log files  https://review.openstack.org/49460411:06
manheimhi, is there a problem with the gates?11:09
manheimoracle and centos mainly11:09
egonzalezmanheim, yep, fix is being merged11:10
*** chas has joined #openstack-kolla11:12
manheimty ezonzalez :D11:14
*** iniaziaway has joined #openstack-kolla11:20
*** iniazi has quit IRC11:23
*** wahnsinn has joined #openstack-kolla11:31
*** dave-mccowan has joined #openstack-kolla11:33
*** chas has quit IRC11:43
*** ducttap__ has quit IRC12:08
*** ducttape_ has joined #openstack-kolla12:16
hrwegonzalez: thx12:18
*** zhurong has quit IRC12:23
*** japestinho has joined #openstack-kolla12:25
*** awiddersheim has quit IRC12:27
*** awiddersheim has joined #openstack-kolla12:27
*** dave-mccowan has quit IRC12:38
*** glycerine has quit IRC12:39
openstackgerritMartin André proposed openstack/kolla-ansible master: [WIP] Use local cirros image if available  https://review.openstack.org/49844912:41
*** ducttape_ has quit IRC12:48
*** rhallisey has joined #openstack-kolla12:48
*** dave-mccowan has joined #openstack-kolla12:49
openstackgerritMerged openstack/kolla master: fix image build error of fluentd  https://review.openstack.org/49833912:57
*** gkadam_ has quit IRC13:04
*** lucasxu has joined #openstack-kolla13:05
*** jmccarthy has joined #openstack-kolla13:09
openstackgerritMerged openstack/kolla-ansible master: Add missing ifaces and groups in prod guide  https://review.openstack.org/49723813:16
*** zhangfei has joined #openstack-kolla13:19
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add warning interface names with dashes  https://review.openstack.org/49577013:22
*** jtriley has joined #openstack-kolla13:31
*** ducttape_ has joined #openstack-kolla13:33
sdaketextual 7 won't upgrade13:37
* sdake growls13:37
egonzalezsdake, they stop the support?13:37
sdakenah - looks like a bug or something13:37
sdakeit locks up when i do teh "restore purchase" button13:38
sdakeand i paid 4.99 for the upgrade13:38
sdakewhich is doubly annoying13:38
*** janki has quit IRC13:38
egonzalezhrm, i'm not sure which version I have :/13:39
egonzaleztextual is the only decent IRC client for mac os, which is pity13:40
*** ducttape_ has quit IRC13:48
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Update Nova to use Cinder v3  https://review.openstack.org/49846913:51
openstackgerritPaul Bourke (pbourke) proposed openstack/kolla-ansible master: Update Nova to use Cinder v3  https://review.openstack.org/49846913:52
*** ducttape_ has joined #openstack-kolla13:55
sdakeok got textual 7 running13:55
hrwflickering tests...13:55
hrwegonzalez: irssi in iConsole2 should be fine13:56
*** jtriley_ has joined #openstack-kolla13:59
*** hongbin has joined #openstack-kolla14:00
*** kiennt has joined #openstack-kolla14:01
*** jtriley has quit IRC14:01
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: rework and fix tail source config  https://review.openstack.org/49460414:05
*** noVNC has joined #openstack-kolla14:08
*** david-lyle has quit IRC14:10
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd "progname" field not always generated  https://review.openstack.org/49834314:14
*** jtriley_ has quit IRC14:17
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: Libvirt logs are missing  https://review.openstack.org/49837314:19
*** jtriley has joined #openstack-kolla14:20
*** zhangfei has quit IRC14:24
*** jtriley has quit IRC14:27
egonzalezis anyone testing upgrades? ocata-pike ?14:27
wahnsinnhey guys, I'm trying to run the kolla-ansible bootstrap-servers playbook and limiting the hosts with --limit flag, but I get this error: "The error was: 'dict object' has no attribute 'ansible_enp2s0'\n\"14:27
*** livelace has joined #openstack-kolla14:27
wahnsinnanybody knows this error?14:28
egonzalezwahnsinn, what release?14:28
wahnsinn4.0.214:28
wahnsinnstable/ocata branch14:28
wahnsinnI've found this, but downgrading of ansible didn't work: https://ask.openstack.org/en/question/102309/kolla-ansible-cant-add-compute-node-using-ansible-playbook-limit/14:30
*** signed8bit has joined #openstack-kolla14:31
egonzalezwahnsinn, when executing bootstrap servers, are both tasks executed:14:32
egonzalezGather facts for all hosts14:32
egonzalezand Gather facts for all hosts (if using --limit) ?14:32
egonzalezwahnsinn, changer or OK, not skipped14:33
wahnsinnegonzalez, only the first one is executed on the host I limit the playbook to, and then it fails in the "Generate /etc/hosts for all of the nodes" task14:34
manheimwhen are you guys planning to release the 5.0.0?14:35
egonzalezpbourke, any idea about this ^^14:35
egonzalezpbourke, the --limit thing not release :)14:36
pbourkedid I see a patch related to this recently?14:36
pbourkewill check again14:36
*** jtriley has joined #openstack-kolla14:36
egonzalezpbourke, yep, but is adding tag: always, so is executing even not using --limit https://review.openstack.org/#/c/497715/14:37
*** zhangfei has joined #openstack-kolla14:37
*** zhangfei has quit IRC14:37
pbourkemaybe its a bootstrap thing14:38
egonzalezmaybe this change https://review.openstack.org/#/c/434940/14:38
egonzalezbut wasnt backported, so 4.0.2 should be fine14:39
egonzalezwahnsinn, can you paste the exact error? along with a `ip a` in deployment node?14:40
egonzalezwahnsinn, is deployment node different from the rest of hosts?14:40
*** jamesbenson has joined #openstack-kolla14:42
wahnsinnsure, one sec14:43
wahnsinnthe deployment node is my laptop, so yes it is different from the rest of the nodes14:44
egonzalezwahnsinn, i know whats your issue14:44
wahnsinn?14:45
egonzalezhttps://bugs.launchpad.net/kolla-ansible/+bug/166536414:45
openstackLaunchpad bug 1665364 in kolla-ansible "kolla-ansible bootstrap-servers fails when api_interface is invalid" [Medium,Fix released] - Assigned to Mark Goddard (mgoddard)14:45
*** hrw has quit IRC14:47
wahnsinnegonzalez, so it is failing because my deployment node doesn't have the api_interface?14:48
egonzalezwahnsinn, 3 options: use a node with the interface to deploy (a controller ie), cherry-pick this fix https://review.openstack.org/#/c/434940/, or change ``all`` to baremetal in https://github.com/openstack/kolla-ansible/blob/4.0.2/ansible/roles/baremetal/tasks/pre-install.yml#L2414:48
*** hrw has joined #openstack-kolla14:48
egonzalezwahnsinn, exactly14:48
egonzalezwahnsinn, the safer one is use one controller as deployer node14:49
egonzalezwahnsinn, i'd discourage to use option 2 or 3 unless using a dev branch14:49
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: manage multiline log pattern  https://review.openstack.org/49463514:50
wahnsinnegonzalez, ok thx, so this issue is fixed in 5.0.0? Because I would prefer to not use a controller node14:51
egonzalezwahnsinn, yep, master(pike) fixed the issue14:51
jamesbensonhey, has anyone deployed lvm = yes?14:52
wahnsinnnice, but I guess the 5.0.0b3 tag isn't considered stable for production yet, is it?14:52
jamesbensonlvm seems to be deploying iscsi even though all of my iscsi is set to no.14:53
*** awiddersheim has quit IRC14:53
egonzalezwahnsinn, nope, 5.0.* was not release yet, in less than a month14:53
pbourkejamesbenson: im using it yes14:54
pbourkejamesbenson: perhaps you have something else enabled that requires iscsi?14:54
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: manage multiline log pattern  https://review.openstack.org/49463514:54
jamesbenson:-/ I don't think so, everything else is standard.... or so I think.  I've only switched ceph to lvm14:55
*** awiddersheim has joined #openstack-kolla14:56
*** signed8bit has quit IRC14:56
*** zhubingbing_ has joined #openstack-kolla14:57
*** zhubingbing has quit IRC15:00
jamesbensonIt looks like this process runs anytime there is a device in compute/storage?  iscsid container15:00
egonzalezwahnsinn, add following in inventory15:02
jamesbensonhttp://paste.openstack.org/show/619646/15:02
egonzalez[deployment]15:03
egonzalezlocalhost       ansible_connection=local15:03
egonzalez network_interface=<existing iface>15:03
noVNChello15:03
egonzaleznoVNC, sup15:04
noVNCfine egonzalez15:07
jaosoriorkfox1111: hey, you going to the PTG?15:07
noVNCegonzalez, is posible to specify MTU in kolla ?15:07
egonzaleznoVNC, MTU for instances, docker or?15:08
noVNCyes docker and neutron15:08
noVNCeverything for jumbo frames15:08
egonzaleznoVNC, docker https://docs.openstack.org/kolla-ansible/latest/quickstart.html#build-container-images15:08
egonzaleznoVNC, you should be able to override values on each config file to match this https://docs.openstack.org/ocata/networking-guide/config-mtu.html15:10
egonzaleznoVNC, /etc/kolla/config/neutron.conf ie, add section and option/values on that file and will override values in neutron.conf15:11
noVNCegonzalez, but the .conf needs to have all the parameters or just add mtu configuration ?15:13
jamesbensonping sdake, jeffrey4l15:14
sdakesup jamesbenson15:14
jamesbensonhey, remember this bug: https://bugs.launchpad.net/kolla/+bug/163107215:15
openstackLaunchpad bug 1631072 in kolla "iscsid container: mkdir /sys/kernel/config: operation not permitted" [High,Confirmed]15:15
jamesbenson?15:15
sdakeno15:15
egonzaleznoVNC, only section and options to override, configs will be merged with main file https://docs.openstack.org/kolla-ansible/latest/advanced-configuration.html#openstack-service-configuration-in-kolla15:15
*** livelace has quit IRC15:16
sdakeboy litecoin up 35% n a week15:16
jamesbensonlol, okay... I was going to inquire about it.  I seem to have it as well...15:16
sdakecrazy growth15:16
*** david-lyle has joined #openstack-kolla15:16
noVNCegonzalez, super nice15:17
noVNCthanks15:17
wahnsinnegonzalez, I've tried your addition to the inventory file -> same error15:19
egonzalezwahnsinn, can you share your inventory and globals?15:19
wahnsinnegonzalez, furthermore I've tried to run it on the controller node and still got the same error15:19
wahnsinnsure give me a sec15:20
*** cduchesne has quit IRC15:20
*** ntpttr_laptop has joined #openstack-kolla15:20
*** cduchesne has joined #openstack-kolla15:22
wahnsinnegonzalez: my inventory: https://pastebin.com/ENg5HfuT15:23
*** kiennt has quit IRC15:24
wahnsinnegonzalez, my globals: https://pastebin.com/1mTaXrxW15:24
*** jamesbenson has quit IRC15:25
*** japestinho has quit IRC15:25
egonzalezwahnsinn, try this inventory and comment network_interface in globals http://paste.openstack.org/show/619648/15:25
wahnsinnthis is what I'm trying to run: https://pastebin.com/M2QHsfHS15:25
*** tvignaud has quit IRC15:26
egonzalezwahnsinn, try the inventory i pasted and let me know15:27
*** vhosakot has joined #openstack-kolla15:28
wahnsinnegonzalez, still the same error - now it is complaining about the network_interface I've set for localhost15:32
egonzalezwahnsinn, please, share error15:32
*** jistr is now known as jistr|off|mtg15:33
*** jistr|off|mtg is now known as jistr15:33
*** chas has joined #openstack-kolla15:33
wahnsinnegonzalez, https://pastebin.com/USQhkbyn15:34
*** jamesbenson has joined #openstack-kolla15:35
*** chas has quit IRC15:35
*** jamesbenson has quit IRC15:35
kfox1111morning.15:38
kfox1111jaosorior: yeah, I'll be there. you?15:38
jaosoriorkfox1111: same15:38
kfox1111cool. will see you there then. :)15:39
*** jamesbenson has joined #openstack-kolla15:44
*** pcaruana has quit IRC15:47
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: manage multiline log pattern  https://review.openstack.org/49463515:48
wahnsinnegonzalez, I will call it a day now15:48
wahnsinnegonzalez, thanks for your help - you guys are doing an amazing job with kolla ;)15:49
*** jascott1 has quit IRC15:53
*** wahnsinn has quit IRC15:58
*** jamesben_ has joined #openstack-kolla15:59
inc0good morning16:00
*** jamesbenson has quit IRC16:00
inc0kfox1111: which storage driver you use on centos? for docker that is16:03
ulf`Hi inc016:03
*** egonzalez has quit IRC16:04
inc0hey ulf` ^ just making inquiries for you;)16:04
ulf`inc0: *G*16:04
ulf`inc0: I need to head to the Farm and poke at the loadbalancer16:04
ulf`inc0: that thing is offline again16:04
inc0:/16:05
inc0that would explain etcd issues16:05
ulf`inc0: nah Theresa has a test environment16:05
ulf`inc0: So she has been testing using it16:05
inc0ok16:05
Reepicheepafter doing a kolla-ansible upgrade to new kolla images I seem to have broken my manila :(16:05
ReepicheepI have it tracked to the fact that the manila_share service on the network nodes can not communicate to the manile instanecs on their  subnets16:06
*** seanmurphy has quit IRC16:06
*** livelace has joined #openstack-kolla16:06
inc0damn, as in can't connect to them over network Reepicheep16:07
inc0?16:07
ReepicheepI see the routes in the routing table on the network nodes point to a tapedl3dd29-3c interface but it does not seem to work16:07
Reepicheepso I'm wondering where to start, neutron, openvswich? I'm using neutron_agent_ha?16:08
Reepicheepinc0: yeah.. no connection from the manila_share service to the nova VMs16:09
Reepicheepthe existing shares that existed pre upgrade are working from the client standpoint but not from the manila_share service16:10
ReepicheepNFS using the generic driver16:10
Reepicheepso these subnets are connected to the share network subnets just not to the network nodes themselves16:11
*** zhubingbing has joined #openstack-kolla16:12
inc0first, do you have connection to vms at all?16:12
ReepicheepI have connection to all the vms, the existing manila vms I can connect to the shares from existing allowed instances16:14
inc0so it's just manila nodes that can't connect to manila workers?16:14
Reepicheepinc0: yes that is what it appears, that is as long as manila workers are the manila_share containers on the network nodes ;)16:16
*** jmccarthy has left #openstack-kolla16:16
* inc0 doesn't know first thing about manila sorry:(16:16
inc0which IP it uses?16:17
ReepicheepI believe that that is the only control service that need access to the manila-service nodes16:17
Reepicheepso when creating a new node my manila_share node errors with this exception:16:18
ReepicheepServiceInstanceException: SSH connection has not been established to 10.254.0.70 in 300s. Giving up.16:18
Reepicheepthat 10.254.0.70 is the IP of the newly created manila-service VM16:19
Reepicheepon the network noded I see the route for that network16:19
Reepicheep10.254.0.64/28 dev taped13dd29-3c  proto kernel  scope link  src 10.254.0.7216:19
Reepicheepbut there is no connectivity16:20
inc0can you connect host-to-host over this IP?16:20
Reepicheepbetween my network nodes?  let me try16:21
Reepicheepeach manila subnet usually only has one manila-service instance16:21
Reepicheepthat subnet is then connected to the route that the share is connected to, plus the manila_share service needs connectivity (my network nodes)16:22
ReepicheepIt only has the routes / interface from the active network node16:24
Reepicheepso to answer your question if host-to-host is active network node to manila-service VM, no I can not connect16:25
inc0tcpdump hell out of it?:)16:27
*** jamesben_ has quit IRC16:27
ReepicheepI can connect from an VM running in openstack that was given access and was working prior to the upgrade, but that is a different IP because it is a different manila-service instance an a differnt /28 subnet16:27
ReepicheepI will do that .. it's the network nodes not having access to openstack subnets they need to have access to16:28
inc0well truth be told usually network nodes doesn't have direct access to subnets, not outside of net namespaces anyway16:29
*** lpetrut has quit IRC16:30
*** jamesbenson has joined #openstack-kolla16:30
*** wahnsinn has joined #openstack-kolla16:31
*** manheim has quit IRC16:31
Reepicheepinc0: yeah I know.. I just realized this is how manila works through my trouble shooting this issue16:32
*** mgoddard has joined #openstack-kolla16:32
*** jamesbenson has quit IRC16:34
Reepicheepinc0: I'm thinking that is why manila_share got moved to the network nodes though,  I was confused by this issue when I initially started deploying manila16:35
Reepicheephttps://bugs.launchpad.net/kolla/+bug/163902816:35
openstackLaunchpad bug 1639028 in kolla newton "Move manila-share service to network node" [High,Fix committed] - Assigned to Mauricio Lima (mliima) (mauricio-lima)16:35
*** jamesbenson has joined #openstack-kolla16:37
inc0hmm makes sense16:38
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: Fix gates out of disk space  https://review.openstack.org/49740816:44
Reepicheepso am I correct in understanding that this must be the manila_share service that creates these ports / routes on the network nodes to the manila subnets?16:47
Reepicheepit can't be kolla-ansible because it after the fact and I don't see anything really custom in the kolla manila_share image16:48
*** manheim has joined #openstack-kolla16:51
*** manheim has quit IRC16:55
*** zhubingbing has quit IRC16:59
*** mgoddard has quit IRC17:08
*** mgoddard has joined #openstack-kolla17:09
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fix Fluentd/Kibana log level issue  https://review.openstack.org/49833117:11
*** noVNC has quit IRC17:12
*** lpetrut has joined #openstack-kolla17:30
ulf`inc0: r u wfh today?17:33
ulf`inc0: I want to take a stab at updating the kernel on all hosts17:33
ulf`inc0: The issue I saw on the loadbalancer was device mapper looping over XFS nodes that had crashewd and it was stuck there17:33
ulf`inc0: had to hard reset the box17:33
ulf`inc0: I don't think using the ancient CentOS 7 kernel with Docker is such a good idea17:33
inc0probably not17:34
inc0also I have some kolla work to do today17:34
inc0our gates are in bad shape:/17:34
ulf`So is the demo :)17:34
inc0fun week ahead of us;)17:35
inc0when is the demo?17:35
ulf`I travel Sep 917:35
ulf`So there is some time17:35
inc0I'm out from Friday on tho17:35
ulf`inc0: YIKES17:37
ulf`inc0: So I'm putting kernel 4.12.9 on the load balancer as well17:38
inc0yeah do that17:38
ulf`definitely17:38
*** noVNC has joined #openstack-kolla17:44
*** livelace has quit IRC17:47
*** unicell has joined #openstack-kolla17:47
*** jascott1 has joined #openstack-kolla17:52
*** manheim has joined #openstack-kolla17:53
*** manheim has quit IRC17:54
*** egonzalez has joined #openstack-kolla17:54
*** tvignaud has joined #openstack-kolla17:55
*** unicell has quit IRC17:57
*** jtriley has quit IRC18:00
*** mdnadeem has quit IRC18:05
*** manheim has joined #openstack-kolla18:07
*** manheim has quit IRC18:08
*** aagate has quit IRC18:18
*** itlinux has joined #openstack-kolla18:23
*** noVNC has quit IRC18:34
*** manheim has joined #openstack-kolla18:42
*** manheim has quit IRC18:46
*** mwynne has quit IRC18:51
*** mwynne has joined #openstack-kolla18:52
*** lpetrut has quit IRC18:56
kfox1111ulf`: redhat's kernel version number's don't reflect reality.18:56
kfox1111never use them as a direct comparison to upstream.18:56
inc0Schedule for PTG: https://etherpad.openstack.org/p/kolla-queens-ptg-schedule18:58
kfox1111looks good.18:59
inc0since we have 3 days I wanted it to be more free-form19:00
inc0we'll also have bot where we can broadcast current topic19:00
*** ducttape_ has quit IRC19:02
sbezverkinc0 any arrangements for remote access, webex for example?19:04
inc0last time it didn't really work out too well19:05
inc0you're not coming?:(19:05
sbezverkinc0: nope I have a very limited travel budget :(19:06
kfox1111bummer. :/19:07
kfox1111travel support?19:07
sbezverkkfox1111 nah, it is not even finance related, it more time19:07
kfox1111ah. k.19:07
kfox1111well, still a bummer.19:07
inc0sdake: can you take mic you had last time please?19:08
sbezverkkfox1111 right, but I am sure you will hold the fort ;)19:08
kfox1111not sure how much there is to hold up. but sure.19:09
*** ducttape_ has joined #openstack-kolla19:10
*** ducttape_ has quit IRC19:15
jascott1hold the... door?19:17
*** iniazi has joined #openstack-kolla19:18
*** iniaziaway has quit IRC19:20
sdakeinc0 sure i love lugging around 15 pounds :)19:22
*** mgoddard has quit IRC19:22
sdakemy daugther still hasn't got it out of the box19:22
*** ducttap__ has joined #openstack-kolla19:27
*** ducttap__ has quit IRC19:32
*** ducttape_ has joined #openstack-kolla19:33
*** mgoddard has joined #openstack-kolla19:38
rwellumTriple-o falls on Friday inc0 ?19:39
inc0yes19:39
inc0but I want to talk to them before putting hour on it19:39
rwellumThey know this?19:39
inc0EmilienM: ^19:39
inc0just so we're on same page:) https://etherpad.openstack.org/p/kolla-queens-ptg-schedule19:39
EmilienMhello19:42
EmilienMI'll leave early friday afternoon19:42
EmilienMlet's find time on wed/thu19:42
inc0we can do it first thing in the morning19:42
inc0thu after lunch then? in Kolla-k8s block19:43
EmilienMdoing what? we have different topics19:43
inc0I'm thinking of kolla-k8s + tripleo19:43
EmilienMok19:43
inc0cross-deployment-tool stuff I think would be better mon-tue19:43
EmilienMbut related19:44
EmilienMlet's meet at the bar instead19:45
EmilienMso no schedule19:45
inc0sure19:45
hogepodgeinc0: what are the latest kolla builds I can access?19:46
inc0images you mean?19:46
hogepodgeinc0: do you have anything from today available, or do I need to build it myself?19:46
hogepodgeinc0: yeah, latest builds.19:46
inc0http://tarballs.openstack.org/kolla/images/19:47
inc0so master is today19:47
inc0you can download this tarball19:47
inc0untar it19:47
inc0and run registry -v /var/lib/registry:/path/to/untar19:47
EmilienMinc0: so we have the Packaging WG (#packaging) room19:47
EmilienMsounds like we meet in this one19:48
inc0hogepodge: https://github.com/openstack/kolla-ansible/blob/master/tools/setup_gate.sh#L174-L17819:48
EmilienMand discuss about https://etherpad.openstack.org/p/deployment-queens19:48
inc0that works EmilienM, that's what I was thinking about for cross deployment stuff19:48
EmilienMyea19:49
inc0mon-tue is option there?19:49
EmilienMinc0: so I'm going to draft an agenda for Mon - Tue and send an update on the ML, open for feedback so we can try to finalize the agenda before the PTG19:52
inc0wonderful, thank you Emilien19:52
hogepodgeinc0: I forget, if I do a kolla build from source does it pull from master, or lastest stable?19:59
inc0stable will pull stable, master will pull master20:00
hogepodgeinc0: how so I specify master on the command line?20:00
inc0kolla stabe/ocata pulls sources stable/ocata20:00
hogepodgeinc0: ok, I see, depends on the kolla installation...20:00
inc0just use kolla from master, but in general it's these confs https://github.com/openstack/kolla/blob/master/kolla/common/config.py#L27920:01
inc0so you can always modify it yourself20:01
rwelluminc0: we still doing the image work for Pike this week?20:04
inc0image work for pike?20:06
*** awiddersheim has quit IRC20:08
*** awiddersheim has joined #openstack-kolla20:09
rwellumYeah- was it rc -1 I think? I volunteered to help.20:13
inc0ahh20:17
inc0well, when Jeffrey4l creates change to tag rc-1 I'll link you, but this has to merge first https://review.openstack.org/#/c/495555/720:18
inc0problem is, gates are overloaded it seems and we have issues merging it20:18
*** ducttape_ has quit IRC20:19
egonzalezissues were fixed, cirros site is up again20:20
egonzalezfluentd building issue also fixed20:21
*** bmace has quit IRC20:22
*** bmace has joined #openstack-kolla20:23
egonzalezsome jobs are stuck in jenkins queue for more than 4 hours :/20:24
*** ducttape_ has joined #openstack-kolla20:24
*** mgoddard has quit IRC20:25
*** ducttape_ has quit IRC20:25
*** ducttape_ has joined #openstack-kolla20:26
*** ducttap__ has joined #openstack-kolla20:29
*** ducttape_ has quit IRC20:33
*** ducttap__ has quit IRC20:35
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: Make Oraclelinux use infra mirrors  https://review.openstack.org/49859220:36
rwellumBit confused inc0 - what does this mean? 'revert this when kolla pike release is cut.'20:36
inc0rwellum: it goes like this20:37
inc0master kolla -> we pin sources for all services to pike20:37
inc0branch out stable pike20:37
inc0rewert in master20:37
inc0so it will point to master sources again20:38
rwellumAh!20:38
rwellumThen on the stable pike branch - generate images and publish to dockerhub?20:40
egonzalezups,  missing binary repos to pike20:40
inc0egonzalez: https://review.openstack.org/#/c/498592/120:41
inc0keep eye on this20:41
rwellumegonzalez: 'ups'?20:41
rwelluminc0: 'oraclelineux'?20:41
rwellumI'm just full of questions.... :-/20:42
inc0rwellum: what about it?20:42
rwellumJust never heard of this before.20:42
inc0well, guess which company made their own distro...;)20:43
kfox1111inc0: "made"? :)20:43
rwellumReading up now... :)20:43
inc0no need to be sarcastic kfox1111 :P20:44
kfox1111not being sarcastic. just an interesting choice of words. :)20:45
kfox1111its about the same thing as saying centos "made" a distro.20:45
kfox1111kind of true. from a certain point of view.20:45
inc0right20:46
inc0it's all about repos20:46
*** lucasxu has quit IRC20:46
jamesbensoninc0 have a second?20:53
inc0sure20:53
jamesbensonI was deploying a cluster with LVM and ran into a bug, https://bugs.launchpad.net/kolla/+bug/1631072, the work around that jeffery mentioned worked for me.20:54
openstackLaunchpad bug 1631072 in kolla "iscsid container: mkdir /sys/kernel/config: operation not permitted" [High,Confirmed]20:54
jamesbensonShould I work that into the prechecks?20:54
inc0I wouldn't put too much effort to lvm tbh, and for sure not right now20:54
inc0let's talk after ptg when we're in queens plz20:55
jamesbensonok20:55
*** manheim has joined #openstack-kolla21:01
*** gfidente has quit IRC21:02
openstackgerritJames Benson proposed openstack/kolla-ansible master: Add veth networking sections  https://review.openstack.org/48400621:03
*** tonyb has quit IRC21:04
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: manage multiline log pattern  https://review.openstack.org/49463521:06
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: manage multiline log pattern  https://review.openstack.org/49463521:07
*** tonyb has joined #openstack-kolla21:11
inc0new fulla dac arrived21:16
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Fluentd: rework and fix tail source config  https://review.openstack.org/49460421:19
*** vkmc has joined #openstack-kolla21:27
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: Make Oraclelinux use infra mirrors  https://review.openstack.org/49859221:32
*** mrunge has quit IRC21:36
*** mrunge has joined #openstack-kolla21:38
*** unicell has joined #openstack-kolla21:49
*** jascott1 has quit IRC21:57
inc0dac is awesome21:58
*** jamesbenson has quit IRC22:01
*** noVNC has joined #openstack-kolla22:02
EmilienMinc0: I was working on the schedule for wednesday and then I realized it would be great that we meet in the afternoon, so both teams can meet first, get some warm up and hopefully ideas for the afternoon common session22:02
EmilienMfor ex, we want to discuss about CI jobs between kolla & tripleo22:03
EmilienMbut first we need to discuss in tripleo team what we need / want22:03
EmilienMbefore coming to you asking collaboration22:03
EmilienMdoes it makes sense?22:03
inc0yeah, althoug afternoon would need to mean after our sessions22:03
*** egonzalez has quit IRC22:03
EmilienMcan't you do your sessions on the morning?22:03
noVNChello22:04
noVNCthe only way to create a docker bridge with MTU=9000 is by creating it on top of a dummy interface, with mtu 9000 already configured??22:04
inc0we'll have enough for 2 days at least22:04
inc0morning too22:04
inc0we have room wed-fri22:04
EmilienMyou guys busy22:04
inc03 subprojects can do it to you;)22:04
EmilienMok, maybe we can say, end of the morning then22:04
inc0or we can have lunch together22:05
EmilienMlike 11.00 am22:05
*** Pavo has joined #openstack-kolla22:05
EmilienMno no, it's noisy22:05
EmilienMlet's stay in a room22:05
*** Pavo has quit IRC22:05
EmilienMlet's try 11am and see how it works22:05
inc0well can't say now, 11 am might be possible22:05
EmilienMok good22:05
EmilienMlet's "try"22:05
inc0I'm sure we'll figure something out22:05
inc0after all we all stay in same location and days are long:)22:06
inc0I'll happily share scotch with you all22:06
EmilienMhehe22:06
*** Pavo has joined #openstack-kolla22:07
*** jascott1 has joined #openstack-kolla22:17
*** jascott1 has quit IRC22:17
*** jascott1 has joined #openstack-kolla22:18
*** jascott1 has quit IRC22:22
*** jascott1 has joined #openstack-kolla22:23
*** itlinux has quit IRC22:28
ulf`inc0: got some more time now? ;)22:30
inc0yeah22:30
inc0what's up?22:30
ulf`inc0: need help to make K8 work with kernel 4.1222:31
inc0what's wrong with it?22:31
ulf`inc0: I don't know?22:38
ulf`inc0: I don't even know how to start it let's begin with that :)22:38
inc0ulf`: how about Theresa make a clean redeploy with fresh kernels?22:39
inc0instead of trying to figure it out;)22:39
inc0and we'll set it up tomorrow in office?22:39
ulf`inc0: OK22:40
*** vhosakot has quit IRC22:42
*** vhosakot has joined #openstack-kolla22:43
sdakerwellum - i just noticed a +1 to your core review nomination in my "drafts" folder22:58
sdakeI was pretty sure I responded on the mailing list, but if I didn't, rest assured I was definately +9000 on your cr nomination :)22:59
sdakeoutlook is such a pita :(22:59
noVNChowdo I remove br-int and br-tun ?23:00
noVNCip link del br-tun type bridge23:00
noVNCRTNETLINK answers: Operation not supported23:00
sdake                                         what is wrong with the default kernel?23:03
*** hongbin has quit IRC23:07
*** vhosakot_ has joined #openstack-kolla23:08
*** vhosakot has quit IRC23:09
*** iniaziaway has joined #openstack-kolla23:19
*** iniazi has quit IRC23:22
ulf`inc0: another question is: How to reproduce this demo. I would think the quickest way would be to export all the pod docker images and yaml config files and import them in a new environment. Correct?23:22
*** itlinux has joined #openstack-kolla23:28
*** itlinux has quit IRC23:28
*** manheim has quit IRC23:40
*** unicell has quit IRC23:42
*** unicell has joined #openstack-kolla23:43
*** glycerine has joined #openstack-kolla23:46
*** signed8bit has joined #openstack-kolla23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!