Tuesday, 2018-09-04

*** hamzy has quit IRC00:01
*** hamzy has joined #openstack-kolla00:14
*** tone_zrt has joined #openstack-kolla00:29
*** donghm has joined #openstack-kolla00:43
*** hoangcx has joined #openstack-kolla00:45
*** hongbin has joined #openstack-kolla00:58
*** ducnv has joined #openstack-kolla01:01
*** schwicht has quit IRC01:02
openstackgerritcaoyuan proposed openstack/kolla-ansible master: Remove the unused service_credentials section for sahara  https://review.openstack.org/57432101:36
*** skramaja has joined #openstack-kolla03:21
*** jrist has joined #openstack-kolla03:31
openstackgerritHa Manh Dong proposed openstack/kolla-ansible master: Apply Neutron rolling upgrade logic  https://review.openstack.org/40792203:45
*** skramaja_ has joined #openstack-kolla03:49
*** skramaja has quit IRC03:52
*** zhangfei has joined #openstack-kolla04:14
*** Bhujay has joined #openstack-kolla04:33
*** janki has joined #openstack-kolla04:42
*** zhangfei has quit IRC04:51
*** hongbin has quit IRC04:58
*** shyamb has joined #openstack-kolla04:59
*** shyamb has quit IRC05:08
*** shyamb has joined #openstack-kolla05:08
*** k_mouza has joined #openstack-kolla05:10
*** k_mouza has quit IRC05:14
*** shyamb has quit IRC05:23
*** cah_link has joined #openstack-kolla05:37
openstackgerritHa Manh Dong proposed openstack/kolla-ansible master: Apply Nova rolling upgrade logic  https://review.openstack.org/55877005:39
*** shyamb has joined #openstack-kolla05:42
*** zigo has quit IRC05:49
shyambHi06:06
shyambGood morning06:06
shyambI am stuck with this issue from last week06:07
shyambhttp://paste.openstack.org/show/729324/06:07
shyambNot getting clear answer from anybody06:07
shyambIs this still an issue in latest kolla code?06:08
shyambplease confirm06:08
*** zhangfei has joined #openstack-kolla06:10
*** Bhujay has quit IRC06:16
shyambLogged bug for this: https://bugs.launchpad.net/kolla/+bug/179057906:20
openstackLaunchpad bug 1790579 in kolla "kolla-build tool failes to create rhel based containers - TypeError: argument of type 'NoneType' is not iterable" [Undecided,New]06:20
*** shyamb has quit IRC06:25
*** shyamb has joined #openstack-kolla06:25
*** shyam89 has joined #openstack-kolla06:30
*** shyamb has quit IRC06:34
*** Bhujay has joined #openstack-kolla06:38
*** rgogunskiy has joined #openstack-kolla06:53
donghmshardy, I have just tested, this bug was fixed in Rocky at https://review.openstack.org/#/c/588518/06:55
donghmyou can use kolla in git repo for the master version06:56
donghmcause the latest kolla package when you install by pip is 6.1.006:56
*** luksky has joined #openstack-kolla06:58
donghmsr, I want tag shyamb07:01
*** nathharp has joined #openstack-kolla07:33
shyam89Anybody seen this issue: http://paste.openstack.org/show/729383/07:42
*** hamzaachi has joined #openstack-kolla07:42
shyam89I am getting this issue while building horizon container07:42
mandreshyam89: not sure about the issue, could be that it didn't take into account your subscription?07:45
shyam89possible07:45
shyam89that;s why I tried with centos base now07:46
mandreshyam89: any reason why you're using rhel based images instead of centos? you would have better community support with centos07:46
shyam89with dockerhub registry07:46
shyam89mandre: yes, we need to go with centos07:46
shyam89We are doing a official qualification of redhat integration07:47
shyam89so..07:47
shyam89mandre: I want to copy my yum repo file07:47
shyam89I kept that file in current directry from which I am executing my command07:48
shyam89but it's not working07:48
shyam89http://paste.openstack.org/show/729384/07:48
mandreshyam89: FYI, building rhel base image with kolla is not supported by Red Hat, see https://bugs.launchpad.net/kolla/+bug/164874507:49
openstackLaunchpad bug 1648745 in kolla "redhat containers uses old repositories" [Undecided,Expired]07:49
shyam89ok07:49
shyam89I need to check it07:49
shyam89will test on centos at-least07:50
mandreshyam89: if you need to modify an existing image (vs building from scratch), that's a different story... in tripleo we suggest you add new layer to the image, see https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/3rd_party.html#adding-layers-to-existing-containers07:52
*** ktibi has joined #openstack-kolla07:53
*** shyam89 has quit IRC07:55
*** tonythomas has joined #openstack-kolla07:56
*** mattgo has joined #openstack-kolla08:03
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Apply Neutron rolling upgrade logic  https://review.openstack.org/40792208:04
*** dougsz_ has joined #openstack-kolla08:10
*** k_mouza has joined #openstack-kolla08:19
*** rabel has joined #openstack-kolla08:35
*** shyam89 has joined #openstack-kolla08:37
*** niedbalski has joined #openstack-kolla08:38
rabelhi there. are there any efforts to use kolla on debian?08:46
rabeland to use debian base-images ?08:47
ktibirabel, you can build debian image.08:48
*** tonyb has quit IRC08:50
*** lvdombrkr has joined #openstack-kolla08:50
shyam89mandre: Hi08:51
shyam89The information you have provided is imp08:51
shyam89I need some more details there08:51
shyam89My container use case is, it's a plugin in horizon, so we can not run in a independent container08:52
rabelktibi: so everything works fine with that?08:52
ktibirabel Ask to hrw I guess he work on debian image.08:53
hrwelo08:53
hrwrabel: on arm64/debian kolla based setup we reached 100% of refstack tests08:53
hrwrabel: we do not test x86-64/debian08:54
lvdombrkrhello folks08:56
shyam89mandre:  Hi08:58
shyam89I think redhat uses a different method to build their containers08:58
rabelhrw: are you using the binary packages from official debian repos in the docker images?08:58
shyam89if I use centos as base and go ahead, they might not accept it08:59
shyam89till I commit my changes to upstream repo08:59
hrwrabel: source images08:59
shyam89If I want to make it work till I commit my code to upstream, I think I need to use rhel horzion as base image and build docker container with my dockerfile09:00
lvdombrkrwho use kolla in production? can you describe your network, nodes, storages setup etc? (how many nodes, HA, physical network connections, external network (floating ip or provider) etc.09:00
shyam89That is the way I consider09:00
lvdombrkrlvdombrkr: im preparing to deploy my first production env, so want to hear "best practices" from folks09:02
rabelhrw: just curious: why not binary packages?09:03
hrwrabel: when we started using kolla there were no up-to-date binary packages. We (Linaro) also had to make some upstream changes in openstack projects so source based builds were warranty that we have our fixes right after they got merged09:04
mandreshyam89: depends what you want to do, if you want to contribute your code upstream then you may want to work on a centos base and contribute it to kolla, then it will automatically be picked up when building container images for tripleo09:04
hrwrabel: price for being different architecture ;d09:05
*** mgoddard has joined #openstack-kolla09:10
rabelhrw: thanks for the infos :) and thanks for your work on debian-based kolla images09:11
hrwrabel: yw09:12
shyam89mandre: yeah, that's the path we want t follow09:13
shyam89but we are not able to chase the timelines of upstream09:14
shyam89we are working on queens now09:14
shyam89and upstream is already moved ahead09:14
hrwshyam89: get it working on queens for your work and then get it working in stein?09:15
shyam89yeah, that's the way I am thinking now09:15
*** gfidente has joined #openstack-kolla09:16
shyam89at-least I need to get it working for queens09:16
shyam89and commit my code in master09:16
shyam89mandre: hrw: I am new to upstream, how I can raise my first PR?09:17
hrwshyam89: 'git review'09:17
shyam89ok09:17
lvdombrkrfolks?09:17
hrwshyam89: remember to keep your changes small and well described09:17
shyam89it will be a new project in openstack09:17
hrwlvdombrkr: ask niedbalski - maybe he will say something about our developer cloud (or force you to sign pile of nda docs first) :D09:18
shyam89We also need to modify horizon, I think I should start there09:18
hrwshyam89: ok, then no idea. never had to start new project09:18
shyam89hrw: I can start with horizon changes required for our plugin to work09:19
lvdombrkrhrw: "(or force you to sign pile of nda docs first)" sorry??09:19
shyam89hrw: We need changes in kolla dockerfile template09:20
hrwshyam89: then it is set of changes for kolla too09:20
shyam89Should I make changes and raise it for review directly?09:20
hrwlvdombrkr: joking (we have nda hw in lab)09:21
shyam89hrw: First of all, I need to fork kolla project from git hub: https://github.com/openstack/kolla09:22
shyam89then I need to make changes in dockerfile09:22
shyam89and raise pull request09:22
hrwshyam89: we do not use github for reviews09:22
shyam89ok09:22
shyam89gerrit?09:22
hrwshyam89: fetch kolla code, do changes, commit them. 'git review'09:22
hrwyes09:22
shyam89once I commit my changes to my fork09:23
shyam89how to intiate git review?09:23
shyam89any command?09:23
shyam89or ui?09:23
donghmshyam89, may you need this https://docs.openstack.org/infra/manual/developers.html09:24
shyam89ok09:24
shyam89thanks09:24
*** rabel has left #openstack-kolla09:25
*** k_mouza has quit IRC09:27
*** k_mouza has joined #openstack-kolla09:28
lvdombrkrhrw: ohhh understood ))) you didnt deploy any production yourself ?09:28
lvdombrkr niedbalski : ping09:29
hrwlvdombrkr: I only do all-in-one+compute for tests09:29
*** shyam89 has quit IRC09:29
hrwlvdombrkr: we have function split in our team. I did bringup, some builds. now builds are done by our CI09:31
nathharphey guys - any thoughts on a process for replacing controller nodes?09:37
lvdombrkrnathharp: im also interesting on it09:40
lvdombrkrhrw: understood, thanks )09:41
*** hamzaachi_ has joined #openstack-kolla09:42
*** hamzaachi has quit IRC09:44
mnasiadkanathharp: feel free to test out and add some documentation around it :)09:45
nathharpmnasladka: I suspected I would have to, but was hoping not to have to do so from scratch!09:46
*** shyam89 has joined #openstack-kolla09:47
*** jaosorior has joined #openstack-kolla09:47
*** k_mouza has quit IRC09:50
*** tonyb has joined #openstack-kolla09:53
*** shyam89 has quit IRC09:55
*** k_mouza has joined #openstack-kolla10:11
*** k_mouza has quit IRC10:20
*** k_mouza has joined #openstack-kolla10:20
*** dave-mccowan has joined #openstack-kolla10:41
*** hamzaachi_ has quit IRC10:52
*** pbourke has quit IRC10:55
*** pbourke has joined #openstack-kolla10:57
*** hamzaachi has joined #openstack-kolla11:00
*** shyam89 has joined #openstack-kolla11:04
*** luksky has quit IRC11:05
*** zhangfei has quit IRC11:25
*** beagles has joined #openstack-kolla11:31
*** luksky has joined #openstack-kolla11:40
*** shyam89 has quit IRC11:41
*** sambetts_ has quit IRC11:55
*** dims has joined #openstack-kolla11:56
*** sambetts_ has joined #openstack-kolla11:57
mnasiadkanathharp: I did some tests in the past by just removing one node, running reconfigure to update the templates and restart relevant services, and then adding another host to the controllers group12:03
*** shyam89 has joined #openstack-kolla12:04
*** trozet has joined #openstack-kolla12:08
*** schwicht has joined #openstack-kolla12:20
lvdombrkrmnasiadka: meet any issues in process?12:28
mnasiadkalvdombrkr: probably some, but now I don't remember - most likely rabbitmq clustering stuff, but it should be fixed in Rocky - since we changed the clustering mechanism12:28
*** trozet has quit IRC12:29
*** jrist has quit IRC12:39
*** jrist has joined #openstack-kolla12:40
*** schwicht has quit IRC12:47
*** shyam89 has quit IRC12:58
*** goldyfruit has joined #openstack-kolla12:59
*** skramaja_ has quit IRC13:01
*** tonythomas has quit IRC13:12
*** schwicht has joined #openstack-kolla13:14
*** mgoddard has quit IRC13:21
*** k_mouza has quit IRC13:32
*** k_mouza has joined #openstack-kolla13:34
niedbalskilvdombrkr, what's the question you have? I may be able to help you, otherwise you should use the openstack-operators mailing list.13:41
*** mgoddard has joined #openstack-kolla13:48
openstackgerritJeffrey Zhang proposed openstack/kolla master: add .travis support  https://review.openstack.org/59960914:03
*** itlinux has quit IRC14:15
*** Bhujay has quit IRC14:17
*** mjturek has joined #openstack-kolla14:19
*** dougsz_ has quit IRC14:23
*** dougsz_ has joined #openstack-kolla14:26
*** cah_link has quit IRC14:29
*** aagate has joined #openstack-kolla14:31
*** k_mouza has quit IRC14:36
*** mchlumsky has joined #openstack-kolla14:38
*** gfidente has quit IRC14:40
*** k_mouza has joined #openstack-kolla14:47
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Add check.yml for prometheus and vitrage  https://review.openstack.org/59962514:49
*** iclon has joined #openstack-kolla14:51
*** k_mouza has quit IRC14:51
*** k_mouza has joined #openstack-kolla14:53
*** goldyfruit has quit IRC15:03
*** gfidente has joined #openstack-kolla15:06
*** itlinux has joined #openstack-kolla15:09
*** rgogunskiy has quit IRC15:09
*** goldyfruit has joined #openstack-kolla15:19
*** openstackgerrit has quit IRC15:20
*** mjturek has quit IRC15:28
sdakeegonzalez i saw some terse warning about branching15:36
sdakeegonzalez typically the PTL or their designate creates the branch in the releases repo - are you aware of that process?15:36
*** sshnaidm is now known as sshnaidm|afk15:37
egonzalezsdake: is being proposed a week ago, just asked for reviews15:39
sdakeroger15:39
sdakeok - ping if you need any coaching :)15:39
sdakesounds like all is well15:39
*** ktibi has quit IRC15:40
egonzalezIf you could check this change https://review.openstack.org/#/c/597536/ let me know if everything is ok15:40
sdakethe format of that file has been modified from 2 years ago when I did the work15:43
sdakeI am not sure about 6-815:43
sdakethe cycle trailing gives Kolla 4 weeks to release after the release deadline15:43
sdakeiirc.15:43
sdakethat is what we negotiated15:43
sdakeThe release deadline for projects using the release:cycle-trailing model that follow the main release cycle is set to November 28, 2018.15:45
sdakenot sure why the onerous terse message on the mailing list - plenty of time to release prior to nov 2815:45
egonzalezsdake what message, in openstack-dev ML or a different one15:46
sdakeya the ml15:46
*** mjturek has joined #openstack-kolla15:48
*** mattgo has quit IRC15:48
*** lvdombrkr has quit IRC15:58
*** jlvilla-viva is now known as jlvillal15:58
*** openstackgerrit has joined #openstack-kolla16:03
openstackgerritSandhya Dasu proposed openstack/kolla master: Changes required to support networking-cisco  https://review.openstack.org/55211916:03
*** k_mouza has quit IRC16:13
*** k_mouza has joined #openstack-kolla16:15
*** Bhujay has joined #openstack-kolla16:17
*** hamzaachi has quit IRC16:19
openstackgerritOpenStack Release Bot proposed openstack/kolla-ansible stable/rocky: Update .gitreview for stable/rocky  https://review.openstack.org/59967416:24
openstackgerritOpenStack Release Bot proposed openstack/kolla-ansible stable/rocky: Update UPPER_CONSTRAINTS_FILE for stable/rocky  https://review.openstack.org/59967516:24
openstackgerritOpenStack Release Bot proposed openstack/kolla-ansible master: Update reno for stable/rocky  https://review.openstack.org/59967616:24
*** janki has quit IRC16:26
*** janki has joined #openstack-kolla16:27
*** nathharp has quit IRC16:30
*** goldyfruit has quit IRC16:37
*** goldyfruit has joined #openstack-kolla16:37
*** janki has quit IRC16:38
*** k_mouza has quit IRC16:40
*** mjturek has quit IRC16:41
*** mjturek has joined #openstack-kolla16:42
egonzalezsdake ^^ k-ansible branched, missing +2+w on kolla's patch16:47
*** luksky has quit IRC16:53
*** lvdombrkr has joined #openstack-kolla16:54
lvdombrkrhello folks16:54
lvdombrkrtrying to upload large image to glance16:54
lvdombrkrbut after some time get error: ERROR: openstack 401 Unauthorized: Authentication required (HTTP 401)16:55
lvdombrkrand image stuck on : saving16:55
lvdombrkrany ideas how to resolve it?16:55
lvdombrkrincrease keystone token experation time?16:56
lvdombrkrimage is about 300gb16:56
*** dougsz_ has quit IRC16:59
egonzalezlvdombrkr: how many time was uploading until failed? For largr imaged maybe need to increase  https://github.com/openstack/kolla-ansible/blob/stable/rocky/ansible/roles/haproxy/defaults/main.yml#L4717:04
*** rgogunskiy has joined #openstack-kolla17:04
egonzalezBut yep, it looks like keystone token get expired17:05
*** goldyfruit has quit IRC17:05
*** rgogunskiy has quit IRC17:05
mgoddardlvdombrkr: we use service tokens to avoid token timeout issues17:11
*** cah_link has joined #openstack-kolla17:12
*** goldyfruit has joined #openstack-kolla17:21
*** openstackgerrit has quit IRC17:22
lvdombrkregonzalez: probbably after aprox 5-6 hours upload ends17:30
lvdombrkrdont know exact time17:30
lvdombrkrmgoddard: what do you mean by service tokens?17:30
*** Bhujay has quit IRC17:31
*** lvdombrkr has quit IRC17:35
*** cah_link has quit IRC17:35
mgoddardlvdombrkr: https://specs.openstack.org/openstack/keystone-specs/specs/keystonemiddleware/implemented/service-tokens.html17:35
mgoddardlvdombrkr: https://specs.openstack.org/openstack/nova-specs/specs/ocata/implemented/use-service-tokens.html17:36
*** lvdombrkr has joined #openstack-kolla17:36
mgoddardhttps://github.com/stackhpc/eod-kayobe-config/blob/master/etc/kayobe/kolla/config/glance.conf17:36
mgoddardlvdombrkr: did you see my messages?17:37
*** mjturek has quit IRC17:53
lvdombrkrmgoddard: yes18:07
lvdombrkrmgoddard : what excatly this config changes?18:07
mgoddardhmm, perhaps it doesn't work in this case18:08
mgoddardnormally you use it with heat or nova18:08
mgoddardif I make a long running request e.g. to build many servers, and my token times out, the operation will still succeed18:09
mgoddardif nova talks to e.g. glance, it sends my token and a service token. If my token is invalid glance will check the service token18:10
*** openstackgerrit has joined #openstack-kolla18:17
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [DNM] Test Zun job  https://review.openstack.org/59347418:17
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: Add Zun scenario job for ubuntu  https://review.openstack.org/56524018:19
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [DNM] Test Zun job  https://review.openstack.org/59347418:19
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Check containers during kolla-ansible check  https://review.openstack.org/59973518:25
openstackgerritMark Goddard proposed openstack/kolla-ansible master: Remove compute hosts from etcd group  https://review.openstack.org/59973618:27
*** luksky has joined #openstack-kolla18:28
*** spiette has quit IRC18:39
*** lvdombrkr has quit IRC18:40
*** spiette has joined #openstack-kolla18:42
*** gfidente is now known as gfidente|afk18:45
*** hamzaachi has joined #openstack-kolla18:48
*** mjturek has joined #openstack-kolla18:49
*** mgoddard has quit IRC18:52
*** itlinux has quit IRC18:58
*** mjturek has quit IRC19:11
*** mjturek has joined #openstack-kolla19:30
*** hamzaachi has quit IRC19:48
*** schwicht has quit IRC19:53
*** schwicht has joined #openstack-kolla20:03
*** hamzaachi has joined #openstack-kolla20:14
*** hamzaachi has quit IRC20:25
*** hamzaachi has joined #openstack-kolla20:29
*** itlinux has joined #openstack-kolla20:41
rm_workegonzalez: i responded to your comments on https://review.openstack.org/#/c/576361/ if you have a minute :)20:44
rm_workwho else do I need reviews from? mgoddard?20:52
*** schwicht has quit IRC20:55
egonzalezrm_work: sure, will check out later, now I'm on pto this week20:58
rm_workah ok :)20:58
rm_workno worries20:58
egonzalezAnyone can review it20:58
rm_workhoping we can knock this out during the PTG20:58
egonzalezYep, is a good starting point for no DRY20:59
rm_workyes, and some have i think, but it's been up for months and i think we really just need cores to decide to get it in20:59
rm_workpossibly the complexity is a bit intimidating for a lot of folks too (though really it's just a ton of lines, not THAT complex other than some macro stuff)20:59
rm_workwe've been running it for a while21:00
rm_workand we deploy daily without seeing any significant hiccups (I remember one concern was on the restart timing of the container, and i have yet to see it be an issue in our env)21:00
egonzalezGood, i just didn't knew haproxy supported conf.d sub configs21:00
rm_workyeah so in 1.7+ it does natively and we could just say "use config directory conf/"21:01
rm_workbut since we support centos on 1.6 i had to make a script to enumerate them21:01
rm_workhttps://review.openstack.org/#/c/576361/29/ansible/roles/haproxy/templates/haproxy_run.sh.j221:02
egonzalezrm_work: have you tried upgrades from old behavior to the new one?21:04
egonzalezWill probably screw up all services until their role id reached21:05
egonzalezs/id/is21:06
rm_workwe did it once per env21:08
rm_workdidn't see any issues that I recall21:08
rm_workbut yeah21:09
rm_workwe usually maint the APIs during major upgrades, so it's possible21:09
egonzalezSome service call its APIs during upgrade, elasticsearch as example, cannot recall if is usuin VIP or direct IP tho21:11
rm_workyeah but that'd be inside its own service right?21:11
rm_workand the haproxy stuff always waits until the VIP is back up before proceeding21:12
rm_workso for any given service, it should start the upgrade, do the haproxy config for that service, wait for the vip to be back, then do the rest of the upgrade21:13
rm_workhmm though at what point is that handler run...21:13
egonzalezWill need to review the change a bit deeper21:14
egonzalezMakes sense for most services21:14
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: Add Zun scenario job for ubuntu  https://review.openstack.org/56524021:18
openstackgerritHongbin Lu proposed openstack/kolla-ansible master: [DNM] Test Zun job  https://review.openstack.org/59347421:18
openstackgerritFeng Pan proposed openstack/kolla master: Add image for networking-vpp ML2 driver and agent  https://review.openstack.org/59985421:25
*** luksky has quit IRC21:28
*** itlinux is now known as itlinux-away21:30
*** hamzaachi has quit IRC21:30
*** itlinux-away is now known as itlinux21:30
*** hamzaachi_ has joined #openstack-kolla21:30
*** itlinux is now known as itlinux-away21:30
*** itlinux-away is now known as itlinux21:34
*** itlinux is now known as itlinux-away21:35
*** hamzaachi_ has quit IRC21:35
*** itlinux-away is now known as itlinux21:35
*** itlinux is now known as itlinux-away21:35
*** hamzaachi_ has joined #openstack-kolla21:45
*** itlinux-away is now known as itlinux22:01
*** itlinux has quit IRC22:02
*** hamzaachi_ has quit IRC22:16
*** mchlumsky has quit IRC22:22
*** aagate has quit IRC22:27
*** goldyfruit has quit IRC22:29
*** spartakos has joined #openstack-kolla22:48
*** shardy has quit IRC22:52
*** shardy has joined #openstack-kolla22:52
*** itlinux has joined #openstack-kolla23:04
*** gfidente|afk has quit IRC23:12
*** aagate has joined #openstack-kolla23:31
*** itlinux is now known as itlinux-away23:34
*** itlinux-away is now known as itlinux23:50
*** mjturek has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!