Wednesday, 2017-08-30

*** classicsnail has quit IRC00:00
*** classicsnail has joined #openstack-kolla00:00
*** Pavo has quit IRC00:01
*** kbaegis has joined #openstack-kolla00:03
*** yangyapeng has joined #openstack-kolla00:05
*** yangyapeng has quit IRC00:10
sdakebkero welcome to kolla :)00:16
*** awidders_ has quit IRC00:16
*** awiddersheim has joined #openstack-kolla00:17
*** hongbin has quit IRC00:23
*** zhangfei has joined #openstack-kolla00:24
duonghqmorning guys00:38
*** tovin07_ has joined #openstack-kolla00:41
duonghqwelcome bkero00:41
bkeroThanks!00:41
*** zhurong has joined #openstack-kolla00:45
*** manheim has joined #openstack-kolla00:48
*** jbrooks has quit IRC00:50
*** manheim has quit IRC00:52
inc0Jeffrey4l: around?01:00
openstackgerritzhongshengping proposed openstack/kolla-ansible master: Add WSGI support for cloudkitty-api  https://review.openstack.org/49699601:00
*** jamesbenson has joined #openstack-kolla01:02
*** jamesbenson has quit IRC01:07
*** zhangfei has quit IRC01:08
openstackgerritzhongshengping proposed openstack/kolla master: Add WSGI support for cloudkitty-api  https://review.openstack.org/49699301:10
*** aagate has quit IRC01:11
Jeffrey4linc0, yep01:13
inc0hey, what's the plan for rc101:13
inc0?01:13
Jeffrey4ltoday or tomorrow.01:14
inc0cool01:14
inc0add me to review01:14
Jeffrey4lthis is the last week to release rc.01:14
Jeffrey4lsure01:14
inc0yeah, I won't have access to net from Friday on I think01:15
Jeffrey4lgo to the mountain? :P01:15
inc0yup01:15
inc0paragliding trip \o/01:15
Jeffrey4lcool01:15
* sdake buys life insurance on inc0 :)01:16
inc0lol01:16
sdakeis rc1 in good shape?01:16
Jeffrey4llol01:16
Jeffrey4lcore service: yes. others may not.01:16
inc0testing becomes increasingly hard01:17
sdakeok - so some ancillary services may have problems - I think we can handle that with z streams01:17
inc0you know what? I'll make it a PTG session01:17
sdakewelcome to the world of unbounded mission statements :)01:17
inc0I think we could make collaborative doc to describe scenarios which needs testing/was tested and such01:17
sdakeIt was permanently burned into my brain that Doug had indicated the TC wanted broad mission statements..01:17
inc0we need to put some process around testing RC01:18
sdakewhen we applied for big tent years ago01:18
sdakewe gave a broad one for sure :)01:18
*** yangyapeng has joined #openstack-kolla01:20
*** yangyapeng has quit IRC01:20
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [MASTER][KOLLA-ANSIBLE] TEST HEAT RELATED SERVICES  https://review.openstack.org/41227901:20
*** yangyapeng has joined #openstack-kolla01:20
*** zhangfei has joined #openstack-kolla01:21
*** yangyapeng has quit IRC01:24
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: [KOLLA-ANSIBLE][MASTER]TEST CEILOMETER RELATED SERVICE  https://review.openstack.org/41227801:28
*** itlinux has joined #openstack-kolla01:34
*** caowei has joined #openstack-kolla01:38
*** kbaegis has quit IRC01:51
*** kbaegis has joined #openstack-kolla01:51
*** eaguilar has joined #openstack-kolla02:11
*** awidders_ has joined #openstack-kolla02:13
openstackgerritHeetae Ahn proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49514802:15
openstackgerritHeetae Ahn proposed openstack/kolla master: Fix the order of the RHEL 7 server optional repository  https://review.openstack.org/49515402:15
*** awiddersheim has quit IRC02:16
openstackgerritWei Cao proposed openstack/kolla-ansible master: Update designate-sink custom configuration folder  https://review.openstack.org/49900902:19
*** salv-orlando has joined #openstack-kolla02:26
*** yangyapeng has joined #openstack-kolla02:29
*** dgedia has joined #openstack-kolla02:30
dgediaHello I am trying to install Openstack Kolla using Kolla-ansible. However I am facing following horizon time out error when I am doing precheck: https://pastebin.com/X2cYA6E1 . Any help would be really appreciate02:32
dgediaI tried changing the timeout value too in /usr/local/share/kolla-ansible/ansible/roles/horizon/tasks/precheck.yml, but it still timesout02:42
dgediaPlease help!02:42
*** wanghongxu has joined #openstack-kolla02:43
dgediaHello I am trying to install Openstack Kolla using Kolla-ansible. However I am facing following horizon time out error when I am doing precheck: https://pastebin.com/X2cYA6E1 . Any help would be really appreciated02:46
*** dgedia_ has joined #openstack-kolla02:49
wanghongxuyou need check ip and port :44.128.25.28:8002:49
dgediaThe ip is local interface where I am deploying aio02:50
dgediaI tried changing the timeout value too in /usr/local/share/kolla-ansible/ansible/roles/horizon/tasks/precheck.yml, but it still timesout02:51
dgedia_out put of nmap:02:56
dgedia_Starting Nmap 7.01 ( https://nmap.org ) at 2017-08-30 02:54 UTC02:56
dgedia_Nmap scan report for  (44.128.25.28)02:56
dgedia_Host is up (0.0000090s latency).02:56
dgedia_Not shown: 998 closed ports02:56
dgedia_PORT   STATE SERVICE02:56
dgedia_22/tcp open  ssh02:56
dgedia_80/tcp open  http02:56
wanghongxuYour 80 port has been occupied.02:57
wanghongxuPlease stop the web service first02:58
spsuryamorning03:09
spsuryamandre: https://review.openstack.org/#/c/498417/03:09
spsuryacan you review this one too03:09
spsurya?03:09
spsuryaduonghq03:13
duonghqspsurya, hmm, something make gates red03:17
duonghqmany red03:17
spsuryanon voting one03:18
spsuryaduonghq03:18
spsuryamake the changes throughout the k-ansible code03:18
duonghqI saw03:19
duonghqbut it's release week03:19
duonghqdoes it matter if it's merged or not?03:19
spsuryaIt won't, IIUC03:20
duonghqI think it can wait for a couple of week until we release P03:20
spsuryanp03:20
duonghqor we can make at least half of gate green03:20
dgedia_thank you wanghongxu. That fixed. :)03:20
duonghqnot sure why gate is failing03:21
duonghqI'm checking the logs03:21
*** kbaegis has quit IRC03:22
*** kbaegis has joined #openstack-kolla03:22
*** kbaegis has quit IRC03:23
*** yangyapeng has quit IRC03:23
*** yangyapeng has joined #openstack-kolla03:24
wanghongxu:)03:29
*** gkadam_ has joined #openstack-kolla03:33
dgediaAny Idea about "manifest for kolla/ubuntu-source-fluentd:4.0.2 not found" error. I am using the deploy command through kolla-ansible. Detailed error info: https://pastebin.com/Byzb2w4E03:38
*** dave-mccowan has quit IRC03:40
dgediawanghongxu any idea?03:40
*** eaguilar has quit IRC03:42
dgediaAny Idea about "manifest for kolla/ubuntu-source-fluentd:4.0.2 not found" error. I am using the deploy command through kolla-ansible. Detailed error info: https://pastebin.com/Byzb2w4E03:43
bkeroHm trying to figure out why this failure is happening. Maybe kolla-ansible deploy doesn't like ansible_connection=local03:44
bkerofatal: [localhost]: FAILED! => {"failed": true, "msg": "The conditional check ''{{ hostvars[item['item']]['ansible_' + hostvars[item['item']]['api_interface']]['ipv4']['address'] }}' not in '{{ item.stdout }}'' failed. The error was: Invalid conditional detected: EOL while scanning string literal (<unknown>, line 1)\n\nThe error appears to have been in03:44
bkero'/usr/local/share/kolla/ansible/roles/rabbitmq/tasks/bootstrap.yml': line 8, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- fail: msg=\"Hostname has to resolve to IP address of api_interface\"\n  ^ here\n"}03:44
bkeroOh. https://bugs.launchpad.net/kolla-ansible/+bug/169511103:46
openstackLaunchpad bug 1695111 in kolla-ansible pike "Ansible 2.3.1 broke preckecks" [Critical,Fix released] - Assigned to Duong Ha-Quang (duonghq)03:47
*** Arminder- has joined #openstack-kolla03:49
duonghqbkero,  which version are you using?03:51
bkeroduonghq: A mitaka-era fork. >_>03:55
duonghqbkero,  ya, nobody backported the fix to M yet03:56
*** eaguilar has joined #openstack-kolla03:57
*** mdnadeem has joined #openstack-kolla04:01
rstarmerdoes anyone have an example of using the docker rally deployment to run rally tests?  I'm not sure I understand how we're supposed to leverage it?04:04
openstackgerritMerged openstack/kolla master: releasenotes: expand note about aarch64/ppc64le support  https://review.openstack.org/49839704:09
*** eaguilar has quit IRC04:15
wanghongxudgedia , Build these containers by yourself04:27
*** zhubingbing_ has quit IRC04:37
*** zhubingbing has joined #openstack-kolla04:37
*** rstarmer has quit IRC04:43
*** rstarmer has joined #openstack-kolla04:43
*** livelace has joined #openstack-kolla04:48
*** zhubingbing has quit IRC04:54
*** caowei has quit IRC04:54
sdakerstarmer which docker rally deployment04:57
rstarmerI'm using kolla 4.0.0, said "rally: yes" in globals.yml04:57
sdakerstarmer - I can tell you in mitaka we ran rally on 123 nodes - although we ran rally manually - not via kolla itself04:58
rstarmerso I have a rally container, and yes, it's got rally installed, talks to mysql, all that is great.04:58
sdakerstarmer git blame the line - who is the author?04:58
sdakerstarmer i can provide you our tarball from the 123 node testing04:58
rstarmerOh, it's not that there's anything wrong, it's just, I'm trying to understand if someone had a model for using it,04:58
*** salv-orlando has quit IRC04:58
rstarmerbecause man it's a pain to docker exec everything04:58
sdakerstarmer i'd really recommend just running rally manually04:59
rstarmerbut perhaps that was the idea?04:59
sdakerstarmer and we had to turn off a bunch of stuff to get rally to work properly04:59
rstarmeryeah, I'm basically there. Was just trying to figure out if there was a model for using this that I'm missing04:59
rstarmerI.e., why include it if it's just there...04:59
sdakehttps://drive.google.com/open?id=0B8q6xDPETSkHd0hnRndzMld1dnM05:00
*** jamesbenson has joined #openstack-kolla05:00
rstarmerI assume someone wanted to run some automated testing, but what was the model.05:00
sdakehttps://drive.google.com/open?id=0B8q6xDPETSkHajZQbVp5U1I0djQ05:00
*** zhubingbing has joined #openstack-kolla05:01
sdaked3d347232 (zhubingbing                  2016-08-31 07:46:41 +0000 201) #enable_rally: "no"05:01
sdakerstarmer speak of the evil - zhubingbing is your guy :)05:01
zhubingbingHi05:01
sdakedevil that is05:01
sdakesorry zhubingbing - that was a legit typo05:01
rstarmer:)05:01
sdakerstarmer pull those google drive links05:02
zhubingbingRally ?05:02
rstarmerzhubingbinb: I was trying to leverage the rally container, just trying to understand the usecase/useage model expected.05:02
sdakerstarmer https://review.openstack.org/#/c/352101/05:02
rstarmerdownloading05:03
sdakecommit 3afe149e517230cf0511c692bc15b14ac9b69da005:04
sdakeAuthor: Jeffrey Zhang <jeffrey.zhang@99cloud.net>05:04
sdakeDate:   Tue Jun 7 20:14:24 2016 +080005:04
sdake    Implement Rally Dockerfile05:04
sdake05:04
sdake    Partially Implements: blueprint rally-container05:04
sdake05:04
sdake    Change-Id: If8c4fdb51b8083078dddc68598d48c6e139a615a05:04
sdakethat is the first commit05:04
sdakeJeffrey4l is probably the best person to speak to regarding the functionality05:04
*** jamesbenson has quit IRC05:04
sdakecommit d3d3472326598b89a2d589e4446ac167d989521a05:06
sdakeAuthor: zhubingbing <1392607554@qq.com>05:06
sdakeDate:   Wed Aug 31 07:46:41 2016 +000005:06
sdake    Implement Ansible rally role05:06
sdake05:06
sdake    Change-Id: I647c38adbfd00c70874cf51c0bfcb68d243e26cc05:06
sdake    Partially-Implements: blueprint rally-role05:06
sdakehttps://blueprints.launchpad.net/kolla/+spec/rally-role05:07
sdakehttps://blueprints.launchpad.net/kolla/+spec/rally-container05:07
sdakeagian, I'd just run rally manually05:07
sdakethat tarball has our setup for rally as of newton or mitaka (forget which:)05:08
zhubingbing;) forget05:08
zhubingbinghttps://docs.openstack.org/rally/latest/05:09
rstarmercool, will read through the OSIC docs05:09
sdakerally was a huge pain in the ass to use during the OSIC scale testing05:09
sdakei'd imagine its even harder to use in a container :)05:10
sdakerstarmer check out that review as well05:10
sdakeit has a ton of performance data in it05:10
rstarmerand I'm trying to work through the readthedocs rally stuff, yeah, like I said, i was curious as to _why_ the container.05:10
rstarmerok, will dig as deeply as I can.05:10
zhubingbing- -)05:10
rstarmer@zhubingbing I got the doc, but things like https_insecure: true as a deployment json parameter don't seem to work, but that's not kolla, that's clearly rally05:11
zhubingbingrstarmer if you have any questions you can ask me05:11
sdakeSTDAKE-M-26CJ:notes sdake$ more add-rally-c6*05:11
sdake---05:11
sdakefeatures:05:11
sdake  - Implement rally ansible role05:11
zhubingbingKolla rally  working well05:11
sdakerstarmer not alot of detail in the releasenotes05:11
sdakezhubingbing thanks for helping rstarmer out05:11
sdakeok - time to P T F O05:12
rstarmerso in 4.0.0, I have a functioning (except for the lack of an insecure https acess) service05:12
sdakenight all05:12
rstarmersdake: gnite!05:12
zhubingbingsdake night05:12
zhubingbingdo you have running rally in kolla ?05:14
zhubingbingrstarmer05:14
rstarmeryes05:14
zhubingbinggood, have you some problems for rally or  rally on kolla ?05:15
rstarmerI while I can create a deployment (e.g. pass in the appropriate 'openrc' info), I can't get 'rally deployment check' to work, it compalins about https access errors05:15
rstarmerI think in this case, it may just be rally, not specifically kolla05:16
zhubingbing - -05:16
zhubingbingyeah05:16
rstarmerthe thread started with me trying to understand if using the container had a model for _how_ one was expected ot use it05:16
zhubingbingopenrc maybe have error05:16
zhubingbing;)05:16
rstarmerdocker exec rally bash doesn't seem to be the simplest case05:16
zhubingbingsource openrc, you can05:17
rstarmerwell, I have to pass --insecure, or I have to figure out how to install the cert in the container05:17
zhubingbingRun command to test "open stack images list"05:17
rstarmeryeah, that works05:17
rstarmerin the rally container05:17
rstarmeralthough it's 'openstack --insecure images list'05:17
zhubingbingI can write a document to you tomorrow,05:17
zhubingbingrally in kolla05:17
rstarmerwould be great!05:18
*** lpetrut has joined #openstack-kolla05:22
*** salv-orlando has joined #openstack-kolla05:23
*** daidv has quit IRC05:26
*** duonghq has quit IRC05:26
*** daidv has joined #openstack-kolla05:26
*** duonghq has joined #openstack-kolla05:27
*** mdnadeem has quit IRC05:31
*** caowei has joined #openstack-kolla05:37
*** manheim has joined #openstack-kolla05:39
*** reddream has joined #openstack-kolla05:48
*** manheim has quit IRC05:49
reddreamHello, is it possible for kolla to enable both lvm and ceph backend?05:50
*** zhongshengping has joined #openstack-kolla05:51
*** janki has joined #openstack-kolla06:05
*** jascott1 has quit IRC06:10
*** jascott1 has joined #openstack-kolla06:11
*** jascott1 has quit IRC06:15
*** skramaja has joined #openstack-kolla06:18
*** zhongshengping has quit IRC06:22
*** mdnadeem has joined #openstack-kolla06:27
*** vharsh has joined #openstack-kolla06:30
*** aagate has joined #openstack-kolla06:32
*** lpetrut has quit IRC06:36
*** pcaruana has joined #openstack-kolla06:43
*** itlinux has quit IRC06:51
*** daidv has quit IRC06:54
*** daidv has joined #openstack-kolla06:54
*** lpetrut has joined #openstack-kolla06:56
*** serlex has joined #openstack-kolla07:00
*** saneax has quit IRC07:07
*** lpetrut has quit IRC07:07
*** david-lyle has quit IRC07:08
*** david-lyle has joined #openstack-kolla07:08
*** saneax has joined #openstack-kolla07:12
*** ddyer has quit IRC07:14
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: Revert "Fix modify external ceph.conf not restart related containers"  https://review.openstack.org/49905507:19
*** jascott1 has joined #openstack-kolla07:25
*** robbbe has quit IRC07:25
*** matrohon has joined #openstack-kolla07:29
*** egonzalez has joined #openstack-kolla07:41
*** seanmurphy has joined #openstack-kolla07:43
*** vharsh has quit IRC07:50
*** rmart04 has joined #openstack-kolla07:51
openstackgerritMerged openstack/kolla-ansible master: Update designate-sink custom configuration folder  https://review.openstack.org/49900907:53
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: WIP: Add Almanach deployment  https://review.openstack.org/49894407:59
*** vharsh has joined #openstack-kolla08:07
*** lpetrut has joined #openstack-kolla08:09
*** yuanying_ has joined #openstack-kolla08:10
egonzalezmandre, around for a rhel subscription question?08:11
egonzalezmandre, do rhel images need subscription on each image during build or in base is enough? https://review.openstack.org/#/c/495148/08:12
*** yuanying has quit IRC08:13
*** mgoddard has joined #openstack-kolla08:20
mandreegonzalez: I wonder if nowadays the subscription doesn't need to be on the host system while building rhel containers08:35
mandreegonzalez: anyway, I'm not really in a favor of https://review.openstack.org/#/c/495148/, I don't see in which use case it's useful08:36
mandreegonzalez: the RHEL containers built like this will never be supported by Red Hat08:37
mandreegonzalez: I've already shared some thoughts in https://bugs.launchpad.net/kolla/+bug/1648745 (now I see Jeffrey4l added a comment :P)08:38
openstackLaunchpad bug 1648745 in kolla "redhat containers uses old repositories" [High,Confirmed] - Assigned to Martin André (mandre)08:38
mandreegonzalez: maybe it's a good discussion to have at the PTG08:38
egonzalezmandre, yeah, maybe we can find a free slot on friday08:45
egonzalezshould we maintain rhel without being supported by red hat or drop rhel support as is not supported08:46
egonzalezwell, this can be extent to SUSE or any other distro (there was people working on it too)08:47
*** salv-orlando has quit IRC08:55
*** jaosorior has quit IRC08:55
*** salv-orlando has joined #openstack-kolla08:57
sean-k-mooneyegonzalez: we dont currently support suse but if we had opensuse tumbelweed that would be kindof awsome08:59
*** vharsh has quit IRC08:59
sean-k-mooneycurrently the rhel subscription can be enabled with a template override08:59
mandreegonzalez, Jeffrey4l: I added more info to https://bugs.launchpad.net/kolla/+bug/164874508:59
openstackLaunchpad bug 1648745 in kolla "redhat containers uses old repositories" [High,Confirmed] - Assigned to Martin André (mandre)08:59
*** jaosorior has joined #openstack-kolla09:00
egonzalezsean-k-mooney, there is people working on it, but are out of kolla for now :/09:00
sean-k-mooneyi think registing in the base image is fine the only usecase i can think of for haveing a per image registation would be if you had different subsciption key for say ceph vs base os vs osp repos instead of a singel key09:01
sean-k-mooneyegonzalez: on tumbelweed or just opensuse?09:01
sean-k-mooneye.g. leap09:01
mandreegonzalez: we want to keep the 'rhel' base distro, but I personally don't see a case where it's useful to build your own rhel-based images as they will never be supported09:02
egonzalezsean-k-mooney,  opensuse only afik09:02
mandrein that case just roll with centos09:02
sean-k-mooneytumbelweed would give us basically nightly builds of binary openstack09:02
mandreegonzalez: I'll try to involve more people in the discussion and we can talk about it at the ptg09:03
sean-k-mooneymandre: do you work at redhat out of interest or is that just an assumtion.09:03
mandresean-k-mooney: I do, but I assume it's the same for opensuse if they require a subscription... they're probably not going to support something they didn't build and distributed themselves09:05
egonzalezmandre, i see a case where the user use a non default driver, ex: cisco aci or netapp (supported by RH) and need to add those packages09:06
sean-k-mooneywell opensuese is like centos09:06
mandresean-k-mooney: right09:06
sean-k-mooneyits not the same as sles09:06
sean-k-mooneyegonzalez: the same would be true for odl/ovn09:07
mandreegonzalez: but in that case they'll build upon certified images distributed though the normal channels, not from a fresh kolla checkout09:08
sean-k-mooneywe do now have an odl neutron server container but the principal is the same for any neutron sdn controller supported by osp that does not have a dedicated neutron server image09:08
sean-k-mooneymandre: they may or the may build for the stable branch of kolla with rhel binary images09:09
sean-k-mooneymandre: the result should be the same09:09
mandresean-k-mooney: except it won't be supported :-/ and if you choose rhel it's for the support09:10
sean-k-mooneymandre: why would it not be supported.09:10
sean-k-mooneythe only reason the image could be different assuming no temlate overrides are used is if rhel modified kolla in which case it may be suppoted by redhat09:11
egonzaleznot always, maybe for stable and tested packages for 2-4 years09:12
sean-k-mooneybut i would not expect it to be supported by the kolla stable team09:12
sean-k-mooneyegonzalez: are you assuming at the rhel images would have old pinned packages?09:12
sean-k-mooneyif so then building on top of them man potentia security vulnerbilities09:13
egonzalezsean-k-mooney, assuming openstack release packages are maintained for 2years by redhat rather than 1 community year09:13
mandresean-k-mooney: because it can't be certified... I'll try to involve more people in the conversation so we can have a definitive answer09:13
sean-k-mooneymandre: yes but isnt that true also if it take you certified rhel image and do a dnf install netapp driver on top09:14
*** sambetts|afk is now known as sambetts09:15
mandresean-k-mooney: there are two different things, installing a certified netapp driver package in rhel is not the same thing as building a brand new container image from a fresh kolla-checkout09:17
*** manheim has joined #openstack-kolla09:20
egonzalezin any case, from what i'm seeing in tripleo overrides, it cannot work with kolla-ansible at all now, changing mariadb version which is not correctly configured in ansible09:21
*** blallau has joined #openstack-kolla09:22
*** vharsh has joined #openstack-kolla09:22
mandreegonzalez: that's why we keep this modifications in tripleo09:23
egonzalezand those images assume are used by tripleo only, not custom deployments or other, HEALTHCHECKs as example09:23
mandreegonzalez: yes, these is the override file the tripleo flavored kolla images09:24
mandreegonzalez: everything that is not tripleo specific goes in kolla directly09:24
egonzalezmandre, but if the images supported by RH are only based for tripleo, clients consuming those images without tripleo will need to make many changes09:25
*** dims has quit IRC09:28
mandreegonzalez: the support can only goes to the limit of the products, so if the product is tripleo there is no support for rhel-based container images outside of tripleo09:30
sean-k-mooneymandre: the product is also osp09:30
sean-k-mooneyusing osp does not require ment to use trippleo09:30
sean-k-mooneyi can do a manual install too and i belive in the past packstack was also supported09:31
mandresean-k-mooney: that would be a question for a product manager :) I can't speak for them09:31
sean-k-mooneythat fair09:31
egonzalezyeah, in my prev company, we used only osp packages, with custom ansible deployment tool, and was fully supported by RH after a complete validation of the environment09:31
manheimhi guys09:32
sean-k-mooneyis kolla included as a rpm package in osp out of interest.09:32
egonzalezsean-k-mooney, it is in rdo since ocata09:32
egonzalezmanheim, sup09:32
mandresean-k-mooney: it's in rdo but not in osp AFAIK09:33
mandresean-k-mooney: will need to check09:33
sean-k-mooneyegonzalez: im wondering if instead of a random kolla checkout would you have support if you used the version of kolla provied in the osp repo if its there to build rhel binary images.09:33
egonzalezanyway, interesting discussion to have in PTG, mandre ensure a product manager is comming with you :)09:33
mandreegonzalez: yes!09:34
sean-k-mooneymandre: i assume steve gordon will be at the ptg?09:34
sean-k-mooneynot sure if he is the right person but he is usally who i would reach out too with this type of question.09:35
mandresean-k-mooney: I was about to send sgordon an email about this (since he's probably afk now), not sure if he's going to the PTG09:36
sean-k-mooneywell if he is not im sure he can point you to the right person who is09:36
*** kbaegis has joined #openstack-kolla09:37
openstackgerritzhongshengping proposed openstack/kolla master: Add WSGI support for cloudkitty-api  https://review.openstack.org/49699309:37
openstackgerritHeetae Ahn proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49514809:43
*** zhangfei has quit IRC09:47
*** kbaegis has quit IRC09:49
*** kbaegis has joined #openstack-kolla09:49
openstackgerritHeetae Ahn proposed openstack/kolla master: Fix the order of the RHEL 7 server optional repository  https://review.openstack.org/49515409:51
*** dims has joined #openstack-kolla09:58
*** jamesbenson has joined #openstack-kolla09:59
*** jamesbenson has quit IRC10:03
*** zhangfei has joined #openstack-kolla10:04
*** yangyapeng has quit IRC10:05
*** manheim has quit IRC10:06
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add doc note for glance-api container  https://review.openstack.org/49910010:06
*** manheim has joined #openstack-kolla10:09
*** manheim has joined #openstack-kolla10:09
*** tovin07_ has quit IRC10:10
*** manheim has quit IRC10:12
*** kbaegis1 has joined #openstack-kolla10:14
*** kbaegis has quit IRC10:17
*** salv-orlando has quit IRC10:17
*** salv-orlando has joined #openstack-kolla10:18
*** pbourke_ has joined #openstack-kolla10:20
egonzalezsean-k-mooney, by chance know if adding --pidfile to an existing ovs deployment during upgrade will cause any issue? https://bugs.launchpad.net/kolla/+bug/169757010:20
openstackLaunchpad bug 1697570 in kolla "failed to run "ovs-appctl or ovs-dpctl" command in openvswitch container" [Undecided,New]10:20
*** zhangfei has quit IRC10:24
spsuryaegonzalez: https://review.openstack.org/#/c/498417/10:25
spsuryawanted your review on this10:25
egonzalezspsurya, gates are red due an issue in cirros download site10:26
spsuryaIIUC not related to changes10:28
*** dave-mccowan has joined #openstack-kolla10:29
egonzalezspsurya, yep, after recheck gates should be green10:30
spsuryaegonzalez: thanks10:32
*** manheim has joined #openstack-kolla10:33
*** kbaegis1 has quit IRC10:33
*** zhangfei has joined #openstack-kolla10:37
*** manheim has quit IRC10:37
*** seanmurphy has quit IRC10:41
sean-k-mooneyegonzalez: well since the container is restarted as part of the upgrade it should not have any negitive effect10:46
sean-k-mooneyegonzalez: but since your not using pid=host on the container the pid in the pid file will be wrong for all other continers10:47
sean-k-mooneyegonzalez: for examlple the pidfile will not work for the neutron-openvswitch container10:47
*** Pavo has joined #openstack-kolla10:47
sean-k-mooneyegonzalez: so ingeneral its proably not a good idea unless we move the nova-compute agent, and neuton l2,l3 and dhcp agent to un in the pid namespace of the ovs-vswitchd10:48
*** reddream has quit IRC10:48
*** zhangfei has quit IRC10:49
*** zhurong has quit IRC10:52
rstarmeranyone using trove in kolla 4.0.0?  trove-taskmanager looks like it's geting the right config, but launching an instance fails with the service trying to contact keyston on 0.0.0.010:52
pbourke_rstarmer: haven't used it yet but sounds like a configuration bug10:53
pbourke_maybe compare whats been generated with a reference conf10:54
rstarmeryeah, was doing that and the "trove_auth" parameter looks right, just haven't found anything else to blame yet :)10:54
rstarmerthe thing that's odd is that the debug output appears to have the right parameters, and I'm not seeing any 0.0.0.0 stanzas10:55
sean-k-mooneyso i have been thinking about two items for the ptg that could be of interest for a while.10:57
sean-k-mooneyone a way to make smaller images and second a different way of running our containers with docker10:58
sean-k-mooneyhow would people feel if we used systemd to contol the starting of our docker container so that we could correctly express the dependcies between containers?10:58
sean-k-mooneyfor example the ovs-vswithd should not be started untill the ovsdb has started10:59
sean-k-mooneykeystone should not start untill mysql and rabbitmq have started10:59
sean-k-mooneyif we only used before to refer to other containers we can hard code the list of all container it should start before regardless of if that container is deployed or not11:00
sean-k-mooneytaughts?11:00
*** caowei_ has joined #openstack-kolla11:01
*** caowei has quit IRC11:01
*** caowei_ is now known as caowei11:01
sean-k-mooneythis is for kolla ansible by the way11:01
sean-k-mooneythe other taught was relating to image build. i think it would be relitvly trivial to create a run macro that took a defered option that would allow us to keep declaring commands as seperate lines but patch them  up into a singel line when we render the template.11:04
sean-k-mooneyby makeing defered a kolla-build option we can contiue to do the multi layer build for debuging but in production enabled defered=true and make smaller images11:04
sean-k-mooneyinc0: egonzalez pbourke_ SamYaple ^^ any taughts on this11:05
pbourke_the systemd sounds interesting, but is this a problem we have? the order of the containers is handled by the playbooks11:07
sean-k-mooneypbourke_: the issue is on reboot11:08
pbourke_aha11:08
sean-k-mooneydocker does not have a native way to haneld that today11:08
pbourke_and is systemd generally accepted as the best way to this today?11:08
sean-k-mooneyovs-dpdk used to have a hard dependcy  on the ovsdb running before the vsitchd. it currently does not but it might come back with hardware offload to smartnics in the future11:09
sean-k-mooneypbourke_: am i have seen systemd advocated by some11:09
pbourke_I think we can definitely look at it11:10
sean-k-mooneyits the only thing that is present already on all the os we currently support11:10
pbourke_Im not sure I understand the deferred thing...11:10
pbourke_I think our image size issues are more down to how dependencies/requirements are managed11:10
pbourke_we install too much in openstack-base11:11
sean-k-mooneysimple if we replace all the run ... line with a macro that macro can accumalate the run commands in a arrya and if we add a flush macros at the end of the file it can concatanate all the lines in teh arry and preferom all the run command in a singel layer11:11
pbourke_yeah but how much space will that really gain11:11
sean-k-mooneyfor ovs-dpdk souce build about 700mb for other not sure but i would guess a resonably amount as it mean any time we curl a package  we can install and delete it in one go11:12
pbourke_ok that's pretty significant11:13
sean-k-mooneypbourke_: i was thinking of prototypeing it in the config override for ovs-dpdk and then when that was working move to trying to convert the base image if people taught it was useful11:14
sean-k-mooneyfor the ovs-dpdk source build template i can also manually ajust that image without the macro to get teh same effect but at the cost of makeing debuing harder. but with the macro we get both11:15
pbourke_how does it make debugging harder?11:15
sean-k-mooneybassically the run macros with deffered=false becomes just RUN ... in the docker file11:15
rstarmerRKT uses systemd instead of dockerd to manage container lifecycle, and the linux distributions have in general gone to systemd as their process management mechanism of choice, so I think it's worth further investigation11:16
sean-k-mooneypbourke_: to get teh same effect without the macro i need to do everythin in the temlate in one RUN command inclding downloading the source,compileing and cleaning up11:16
pbourke_if its possible to have things as one RUN from the get go I dont see an issue with having that as default. That's the way it should be11:17
pbourke_good to know that about rkt rstarmer11:17
*** caowei has quit IRC11:17
sean-k-mooneypbourke_: ok well i have an idea of how to do this with jinja 2 ill try and prototype someting by the ptg and we can get more input there.11:18
pbourke_yeah sounds good sean-k-mooney11:18
pbourke_tbh I still really want to see someone do kolla-ansible with loci images11:18
sean-k-mooneyalso on the systemd thing we can likely do that via the kolla-docker module too so im hoping that will not be invasive11:19
sean-k-mooneypbourke_: ya im hoping deffered commands would get us to similar sized immages.11:19
sean-k-mooneythose would be cool to use though also11:19
pbourke_Im not sure it will but I look forward to been proven wrong ;)11:20
pbourke_*being11:20
*** pbourke has quit IRC11:20
*** pbourke has joined #openstack-kolla11:21
sean-k-mooneywell the the ovs-dpdk loci image adds about 50mb to the base image. when compiling from source. my kolla one adds almost 700mb when compiling from source because i currently can get rid of the git trees or compilation dependcies because of the layering but i a singel line version it should come out to be about the same 50mb delta over base image11:22
sean-k-mooneythey have smaller base images so mine will still be bigger but you get the idea11:22
*** keico has joined #openstack-kolla11:29
*** Pavo has quit IRC11:32
*** wanghongxu has quit IRC11:32
*** bmace has quit IRC11:38
*** bmace has joined #openstack-kolla11:38
*** rstarmer has quit IRC11:41
*** rstarmer has joined #openstack-kolla11:42
sdakezomg sore11:45
sdaketime for some advil11:45
*** seanmurphy has joined #openstack-kolla11:46
keicoHello. I am trying to stand up a multinode openstack cluster (occata) with kolla-kubernetes, using this document as my installation guide: https://docs.openstack.org/kolla-kubernetes/latest/deployment-guide.html. When generating the kolla config maps, I am seing this exact error: "Unable to locate mariadb directory in search_paths=[/etc/kolla, /etc/kolla-kubernetes, /usr/lib/python2.7/kolla/etc/kolla, /usr/lib/python2.7/site-11:49
keicoDoes anyone know what causes it and how to resolve it?11:49
*** coolsvap has joined #openstack-kolla11:50
*** gkadam_ has quit IRC11:54
*** kbaegis has joined #openstack-kolla12:02
kbaegisHi all. Having issues with kolla/ceph: ImageNotFound: error opening image volume-e08695f0-7683-4e03-8070-a2b21eb49cb1 at snapshot None when creating a volume from image12:03
kbaegisTraceback: https://hastebin.com/usijupevuh.sql12:03
*** iniaziaway has joined #openstack-kolla12:06
*** egonzalez has quit IRC12:07
*** serlex has quit IRC12:07
*** iniazi has quit IRC12:07
*** brad[] has quit IRC12:07
*** Jeffrey4l has quit IRC12:07
*** logan- has quit IRC12:07
*** ulf` has quit IRC12:07
*** hoaile has quit IRC12:07
*** fuegoel has quit IRC12:07
*** calbers has quit IRC12:07
*** nanliu_ has quit IRC12:07
*** SaMnCo has quit IRC12:07
*** jbryce has quit IRC12:07
*** Jeffrey4l has joined #openstack-kolla12:07
*** brad[]` has joined #openstack-kolla12:07
*** egonzalez has joined #openstack-kolla12:07
*** goldyfruit has joined #openstack-kolla12:09
pbourke_kbaegis: im looking into a similar issue at the moment12:09
pbourke_kbaegis: will let you know once I find something12:09
egonzalezsean-k-mooney, thanks for good ovs pidfile explanation12:09
pbourke_sdake: sore from too much mining?12:10
*** kbyrne has quit IRC12:10
sdakepbourke_ https://twitter.com/steven_dake/status/90253232550996787212:11
egonzalezre systemd is a good discussion/design topic12:11
*** calbers has joined #openstack-kolla12:11
pbourke_sdake: ah yeah saw that12:11
sdakepbourke_ - 0.67 tons loaded and unloadedd - took 4+ hours in 108F wheather12:11
pbourke_sdake: what was the job?12:11
sdakeits not like i'm a young strapping gun like you :)12:11
sdake(with hollow legs:)12:12
*** kbyrne has joined #openstack-kolla12:12
*** seanmurphy has quit IRC12:12
sdakepbourke_ will you be at ptg or summit?12:12
pbourke_sdake: ptg :)12:12
pbourke_sdake: unsure about summit, Id say its unlikely I'll make that trip12:12
kbaegispbourke: cool, ty12:13
sdakepbourke_ cool you can drink me under the table for the nth time12:14
pbourke_look forward to it :)12:14
*** rhallisey has joined #openstack-kolla12:16
kbaegispbourke: It's funny- I'm not seeing any failures from the rgw end; only from cinder-volume12:16
*** ulf` has joined #openstack-kolla12:17
sdakewhy is the only 1500 watt power supply i can buy 449 bucks12:17
* sdake growls12:17
sean-k-mooneyegonzalez: glad i could help. adding the pidfile is not a bad idea but as is said if you want it to work outside of the vsiwchd contaienr also then all the container you want it to work in need to share the same pid namespace12:18
egonzalezsean-k-mooney, no outside ovs containers, just inside, the bug was not able to do apctl inside containers, i think is not a good idea share same pid outside containers12:19
sean-k-mooneywell if you wanted to use ovs-appctl from the neutron l2/l3/dhcp agent you could not but then again you could just say go to vswitchd container instead12:21
*** seanmurphy has joined #openstack-kolla12:21
*** logan- has joined #openstack-kolla12:22
*** hoaile has joined #openstack-kolla12:22
*** fuegoel has joined #openstack-kolla12:22
*** nanliu_ has joined #openstack-kolla12:22
*** SaMnCo has joined #openstack-kolla12:22
*** jbryce has joined #openstack-kolla12:22
sean-k-mooneythe pid in the pid file is used to construct the name of the unix contol socket for the vswitchd wich is localted in /run/openvswitch12:22
egonzalezsean-k-mooney, problem is the time i verified the bug (long time ago) appctl was not working inside vswitchd without using targets or pidfile12:23
sean-k-mooneyegonzalez: that is still the case12:23
sean-k-mooneyif you dont have the pidfile it wont know how to create teh name12:23
sean-k-mooneyhence you need to use -t to specify the target socket or the pid12:23
*** eaguilar has joined #openstack-kolla12:24
sdakepbourke_ do you have your visa and all that?12:25
*** eaguilar has quit IRC12:25
*** salv-orlando has quit IRC12:25
*** eaguilar has joined #openstack-kolla12:26
egonzalezhrm, understood, is that somewhere documented in ovs? they lack of good documentation about ovs internals12:26
*** manheim has joined #openstack-kolla12:27
*** seanmurphy has quit IRC12:28
*** eze has joined #openstack-kolla12:28
*** eaguilar has quit IRC12:30
*** manheim has quit IRC12:31
kbaegispbourke: Sooo- I can create vol from image for ceph using a <1G centos image, but it's failing for the one I showed earlier12:32
pbourke_sdake: yup, should be good to go12:35
pbourke_kbaegis: yeah not sure yet Im still trying to get my osds to bootstrap12:35
pbourke_always seems to be hassle with it12:35
kbaegisMy ceph cluster is up/healthy, but the cinder-volume keeps telling me that there's no such image12:39
kbaegisDo you happen to know if I can audit that in rados?12:40
sdakei still need to obtain a visa12:40
* sdake groans12:40
sdakegood thing they are only 50 bucks12:40
*** hachi has joined #openstack-kolla12:42
*** seanmurphy has joined #openstack-kolla12:43
*** jascott1 has quit IRC12:44
pbourke_kbaegis: it could be a cinder problem12:46
pbourke_kbaegis: ceph cluster is also healthy in the bug report I received12:46
kbaegisCluster is healthy and rados ls -p images shows: rbd_id.92dd3c36-170f-4407-952a-a25d9cab72e8 which is the corresponding uuid for the image I'm attempting to create from12:47
egonzalezcould be a container size issue, not being able to copy the image locally into cache?12:48
kbaegisSo glance image store looks good, but yeah. Cinder-vol returns a nasty error/traceback when creating large image volumes for me12:48
kbaegisWorks for <1G12:48
kbaegisHmm. How could we validate that?12:49
kbaegisWhere is nova keeping the libvirt cache?12:51
kbaegisAnd wouldn't that apply generically to all images of a certain size?12:51
kbaegisNo, I'm not understanding this. Cinder-vol would be the culprit yes?12:52
kbaegisYou're suggesting the container doesn't have enough space to create a volume from the image?12:52
*** japestinho has joined #openstack-kolla12:53
*** eze has quit IRC12:53
*** serlex has joined #openstack-kolla12:53
kbaegisMan, that's a pain to track down on a ceph cluster because cinder-vol is probably running across a large number of nodes. Just 6 in my case.12:53
*** eaguilar has joined #openstack-kolla13:01
*** magicboiz has quit IRC13:01
sean-k-mooneykbaegis: well if your using ceph then cinder should not be creating any vloumes locally13:12
sean-k-mooneywhen uploading an image to glance though i dont know if the glance api server need to cache it locally before coping it to ceph or if it will somehow proxy the image upload directly to ceph13:13
*** magicboiz has joined #openstack-kolla13:14
egonzalezit is for copying image conversion, dont know if creating volume includes image conversion  https://github.com/openstack/cinder/blob/df6a8b65dbd5cd80cc0a50003f4d21338f177e99/releasenotes/notes/remove_volume_tmp_dir_option-c83c5341e5a42378.yaml13:15
egonzalezbut, out of space is tested and should raise an error due this. not missing image13:15
sean-k-mooneyegonzalez: regarding how ovs-appctl works with the pidfile im not sure. i have just been dealing with ovs and ovsdpdk for so long that i have had to learn how some of the internals work including this point13:16
egonzalezhaha, i took a look in ovs site and not much info other than man pages for commands13:17
*** matrohon has quit IRC13:17
egonzalezexperience is all :)13:17
sean-k-mooneykbaegis: so when you upload into glance with ceph as a backend it adds metadata to the image specifing with a ceph rbd path e.g. rbd://669991ed-c92d-420c-861a-b065466f7270/images/6d69ee33-821b-4930-9a8d-8c075f378a0b/snap13:18
sean-k-mooneywith rados can you find that image?13:18
sean-k-mooneykbaegis: in my case i can do rbd info  -p images 6d69ee33-821b-4930-9a8d-8c075f378a0b to verify the image is actull in ceph in the images pool13:23
*** k_mouza has joined #openstack-kolla13:24
sean-k-mooneykbaegis: so basically you should be able to do the following to validate the image is in ceph correctly http://paste.openstack.org/show/619891/13:26
sean-k-mooneyif that all checks out then its on the nova side in teh image cache13:27
*** jtriley has joined #openstack-kolla13:31
sean-k-mooneygot to run to a meeting brb13:33
*** zhangfei has joined #openstack-kolla13:36
serlex"OpenStack Pike is officially released!"13:42
serlexwoop...woop?13:42
*** ducttape_ has joined #openstack-kolla13:47
*** lucasxu has joined #openstack-kolla13:48
*** k_mouza has quit IRC13:54
openstackgerritzhubingbing proposed openstack/kolla-ansible master: fix error docker service path  https://review.openstack.org/49916014:01
*** salv-orlando has joined #openstack-kolla14:01
dmsimardegonzalez, sdake, inc0, Jeffrey4l: heads up, the -testing repos for RDO will be fully populated in preparation for imminent release today14:03
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Fix openstack_python log not match multiple case  https://review.openstack.org/49443314:04
egonzalezdmsimard, thanks for advise :)14:04
*** pcaruana has quit IRC14:06
*** aagate has quit IRC14:07
*** rstarmer has quit IRC14:07
*** aagate has joined #openstack-kolla14:07
*** skramaja has quit IRC14:11
*** livelace has quit IRC14:11
*** hongbin has joined #openstack-kolla14:13
*** awidders_ has quit IRC14:13
*** awiddersheim has joined #openstack-kolla14:13
*** awidders_ has joined #openstack-kolla14:15
*** jamesbenson has joined #openstack-kolla14:16
openstackgerritMerged openstack/kolla-ansible master: Changed as per the standard convention set to use "."  https://review.openstack.org/49841714:18
*** awiddersheim has quit IRC14:18
*** jamesbenson has quit IRC14:21
*** blallau has quit IRC14:23
*** jmccarthy has joined #openstack-kolla14:23
*** bmace has quit IRC14:24
*** cocakolla has joined #openstack-kolla14:26
*** jamesbenson has joined #openstack-kolla14:26
*** ducttap__ has joined #openstack-kolla14:28
*** ducttape_ has quit IRC14:28
*** ntpttr_laptop has joined #openstack-kolla14:30
*** ddyer has joined #openstack-kolla14:32
*** hachi has quit IRC14:33
*** hachi has joined #openstack-kolla14:33
*** robbbe has joined #openstack-kolla14:34
*** matrohon has joined #openstack-kolla14:37
*** lpetrut has quit IRC14:38
*** zhangfei has quit IRC14:42
*** mdnadeem has quit IRC14:42
*** ducttape_ has joined #openstack-kolla14:42
*** hachi has quit IRC14:43
*** ducttap__ has quit IRC14:45
*** robbbe has quit IRC14:46
*** hrw has quit IRC14:47
*** robbbe has joined #openstack-kolla14:47
*** hrw has joined #openstack-kolla14:48
*** ducttap__ has joined #openstack-kolla14:50
*** ducttape_ has quit IRC14:53
*** hachi has joined #openstack-kolla14:54
kbaegisback15:02
*** janki has quit IRC15:07
*** thegreenhundred has joined #openstack-kolla15:10
*** kbaegis has quit IRC15:14
*** sri_ has joined #openstack-kolla15:14
*** eze has joined #openstack-kolla15:17
sri_Hello everybody! : quick question in kolla-ansible docker continer don't have a ip address is it sharing the host network ?15:17
*** vharsh has quit IRC15:17
*** eaguilar has quit IRC15:17
egonzalezsri_, yep, all containers use --net host15:17
*** hachi has quit IRC15:19
sri_egonzalez: thanks , so we are using container for service and dependency isolation ?15:19
*** hachi has joined #openstack-kolla15:19
*** duonghq has quit IRC15:19
*** duonghq has joined #openstack-kolla15:19
egonzalezsri_, yes, but containers share IP with host, and some pid (libvirt)15:20
*** duonghq_ has joined #openstack-kolla15:20
*** awidders_ has quit IRC15:21
sri_egonzalez: got it15:21
*** awiddersheim has joined #openstack-kolla15:22
*** duonghq has quit IRC15:24
*** duonghq has joined #openstack-kolla15:24
duonghqevening guys15:24
*** jamesbenson has quit IRC15:31
*** livelace has joined #openstack-kolla15:33
*** kbaegis has joined #openstack-kolla15:33
*** iniaziaway is now known as iniazi15:34
kfox1111morning.15:36
kfox1111sri_: yes.15:36
*** kbaegis has quit IRC15:38
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add container state check in kolla_docker  https://review.openstack.org/49920515:40
*** kbaegis has joined #openstack-kolla15:40
*** salv-orlando has quit IRC15:41
*** robbbe has quit IRC15:41
*** robbbe has joined #openstack-kolla15:42
*** salv-orlando has joined #openstack-kolla15:42
*** robbbe has quit IRC15:43
*** robbbe has joined #openstack-kolla15:44
*** srnbckr has joined #openstack-kolla15:45
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Add container state check in kolla_docker  https://review.openstack.org/49920515:46
inc0good morning15:46
spsuryainc0: morning15:46
duonghqmorning inc015:46
*** itlinux has joined #openstack-kolla15:47
*** itlinux has quit IRC15:48
*** kolla-slack has quit IRC15:48
*** kolla-slack has joined #openstack-kolla15:48
*** thegreenhundred has quit IRC15:48
*** manjaroi3 has joined #openstack-kolla15:48
*** strigazi is now known as strigazi_AFK15:50
inc010min till meeting15:50
duonghqroger inc015:50
*** hachi has quit IRC15:52
*** awiddersheim has quit IRC15:53
*** pcaruana has joined #openstack-kolla15:54
*** awiddersheim has joined #openstack-kolla15:55
*** rmart04 has quit IRC15:56
*** manjaroi3 has quit IRC15:56
*** thegreenhundred has joined #openstack-kolla15:56
*** thegreenhundred has quit IRC15:58
*** thegreenhundred has joined #openstack-kolla15:58
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Enable chrony by default  https://review.openstack.org/49921115:59
*** vhosakot has joined #openstack-kolla16:00
*** jascott1 has joined #openstack-kolla16:00
*** seanmurphy has quit IRC16:02
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Run registering keystone ceph_rgw user task only on one node  https://review.openstack.org/49921416:02
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/ocata: Mount host /tmp folder to horizon container  https://review.openstack.org/49921516:04
*** manjeets has joined #openstack-kolla16:05
manjeetsquestion regarding docker Python for kolla is the pip package docker-py or just docker ?16:06
openstackgerritJeffrey Zhang proposed openstack/kolla master: Prepare Pike branch for kolla  https://review.openstack.org/49921716:07
manjeetsI added just docker via pip seems like it fails for not having docker.Client16:07
egonzalezmanjeets, master/pike docker, ocata both (not really sure) and previous only docker-py16:07
*** salv-orlando has quit IRC16:08
manjeetsegonzalez, I have master but i get above ^^^ failure16:10
egonzalezmanjeets, docker version?16:11
*** japestinho has quit IRC16:11
manjeetsI had 2.5.0 first then tried 2.0.0 both failed16:11
egonzalezmanjeets, sorry, docker-engine version rather the python client16:20
manjeets17.06.1-ce16:22
*** matrohon has quit IRC16:24
*** jmccarthy has left #openstack-kolla16:24
sri_kfox1111: got it thanks16:25
*** ntpttr_laptop has quit IRC16:25
*** klindgren_ has joined #openstack-kolla16:28
robbbequick one.. did the version of docker-py change between 4.0.2 and the later versions?16:29
*** itlinux has joined #openstack-kolla16:31
*** klindgren has quit IRC16:32
robbbei guess docker-py rather than docker?16:33
*** pbourke_ has quit IRC16:34
robbbewhen is 5.0.0 stable due?16:40
manjeetsrobbe if I install docker-py then prechecks passed16:41
egonzalezrobbbe, around 15 sep for kolla16:41
robbbemanjeets : yeah i've sorted it now... docker-py in 4.0.0, docker in 5.0.0. bit of a pain16:41
robbbeegonzalez - same for kolla-ansible?16:42
egonzalezmanjeets, ensure docker-py and docker are not installed in the same env16:42
egonzalezrobbbe, yep16:42
robbbecool16:42
manjeetsohk I have both prechecks passes but i'll remove docker before i fire install16:42
*** ddyer has quit IRC16:43
robbbemanjeets - i had exactly the same issue as you16:43
robbbebest to check your containers and kolla-ansible are from the same branch16:43
*** egonzalez has quit IRC16:45
robbbedoes anyone else use centos as the hostOS and the container base ?16:45
robbbeand btrfs storage driver / fstype16:46
*** aagate has quit IRC16:51
*** jamesbenson has joined #openstack-kolla16:53
manjeetsrobbbe, i use ubuntu16:56
manjeetsanyone have any idea about this http://paste.openstack.org/show/619920/ error ? I made all hostname resolvable from deployment host by adding /etc/hosts file16:57
*** jamesbenson has quit IRC16:57
*** gkadam_ has joined #openstack-kolla16:57
manjeetsi can ping hostname16:57
*** duonghq has quit IRC16:57
robbbemanjeets you might have 127.0.0.1 in there as well16:58
*** ddyer has joined #openstack-kolla16:59
kolla-slack<egonzalez> manjeets, also all nodes should resolve each other17:00
manjeets127.0.0.1 is localhost17:00
manjeetsI have all resolvable from each other17:01
robbbeyeah, but if the line for the host also has 127.0.0.1 then it will be wrong17:02
robbbeor maybe you are looking at the wrong interface17:02
*** strigazi_AFK is now known as strigazi17:03
*** mgoddard has quit IRC17:03
ulf`inc0: you in? need to come by and ask you something real quick17:05
inc0no, what's up?17:05
*** salv-orlando has joined #openstack-kolla17:09
manjeetshmm, i don't 127.0.0.1 on hostname its just localhost but still having the same issue17:12
vhosakotmanjeets: can you do  "nslookup <IP address set for api_interface>"  ?17:13
*** krtaylor has quit IRC17:13
*** strigazi has quit IRC17:14
vhosakotthe hostname of the IP address of api_interface must be resolvable.17:14
*** strigazi has joined #openstack-kolla17:14
*** dims has quit IRC17:14
*** salv-orlando has quit IRC17:14
manjeetsping hostname works vhosakot17:15
*** ntpttr_laptop has joined #openstack-kolla17:17
vhosakotmanjeets: can you ping that hostname from a different box, not from the same box... local resolution is not the issue.17:18
*** dims has joined #openstack-kolla17:18
manjeetsyes vhosakot i can ping hostname from all others and deployment node17:18
vhosakotmanjeets: then, I'd print "hostname -f" and "hostnamectl status" as debug tasks before the failure.17:20
*** robbbe has quit IRC17:21
vhosakotprobably some task in the playbook is messing the hostname temporarily which you don't see after the playbook ends.17:21
manjeetsi doubt i hit this bug for multinode https://review.openstack.org/#/c/300152/17:22
manjeetsvhosakot, hostame -f ans hostnamectl status are ok http://paste.openstack.org/show/619923/17:23
vhosakotmanjeets: it's ok now :)  I'd print it as debug statements in the playbook before the failure.  May be ".localdomain" is also needed in the hostname?17:24
kolla-slack<egonzalez> manjeets check , getent ahostsv4 hostname17:25
kolla-slack<egonzalez> Is what the task is executing17:25
manjeetshttp://paste.openstack.org/show/619924/17:27
kolla-slack<egonzalez> Is 172.16.3.6 the IP?17:29
SamYaple/win/win 5217:30
manjeetsyes17:30
*** eze has quit IRC17:35
*** kbaegis has quit IRC17:35
*** kbaegis has joined #openstack-kolla17:36
*** serlex has quit IRC17:38
*** sambetts is now known as sambetts|afk17:40
*** ducttap__ has quit IRC17:49
*** eaguilar has joined #openstack-kolla17:53
*** krtaylor has joined #openstack-kolla17:57
*** gkadam_ has quit IRC18:01
*** ducttape_ has joined #openstack-kolla18:05
ulf`if I use nodeport and I have an external service port set, and I have 3 pods in my cluster that can fullfil this service, how can I tell which pod processed the request?!?!18:08
*** robbbe has joined #openstack-kolla18:09
*** robbbe has quit IRC18:09
*** robbbe has joined #openstack-kolla18:10
*** salv-orlando has joined #openstack-kolla18:10
*** eaguilar has quit IRC18:12
jascott1ulf` how do you need the info? You need the pod to log it and return a some pod identifying string in the respose?18:12
*** salv-orlando has quit IRC18:15
*** Manheim has joined #openstack-kolla18:16
ulf`jascott1: for debugging purposes18:18
ulf`jascott1: each pod provides a REST API service and I need to see the response18:18
jascott1is whatever running in the pod logging to stdout?18:19
ulf`jascott1: yes18:20
jascott1that should be captured in logs18:21
ulf`jascott1: I mean kub logs <podname>18:21
ulf`jascott1: but then I have to try all three to see which one picked i tup18:21
ulf`it up18:21
jascott1right so will need an aggregated log solution maybe18:21
*** jamesbenson has joined #openstack-kolla18:22
jascott1ulf` you do have a unique id in the request so you can trace it back?18:22
jascott1if you had rather examine the response for the pod id, you could front the pod with an nginx container and add that as a header18:23
ulf`jascott1: not a bad or idea easier, not run 3 pods at the same time, but only one while I debug this18:24
jascott1hey thats a good way :)18:24
ulf`It's not like there is a load requiring there to be three for things to not fall apart18:25
jascott1makes sense depending on what you are trying to debug18:25
jascott1ultimately tho I would look into logging options for the cluster18:25
jascott1i havent spent any time implementing logging in k8s but im sure someone in here or k8s users slack can give advice on that later18:26
*** eaguilar has joined #openstack-kolla18:29
*** livelace has quit IRC18:30
*** eaguilar has quit IRC18:33
*** ntpttr_laptop has quit IRC18:33
*** eaguilar has joined #openstack-kolla18:34
*** jascott1 has quit IRC18:36
*** egonzalez has joined #openstack-kolla18:47
*** rmart04 has joined #openstack-kolla18:48
*** egonzalez has quit IRC18:54
*** egonzalez has joined #openstack-kolla18:55
*** egonzalez has quit IRC18:58
*** tvignaud has quit IRC19:00
*** egonzalez has joined #openstack-kolla19:01
*** jascott1 has joined #openstack-kolla19:08
*** eaguilar has quit IRC19:10
*** salv-orlando has joined #openstack-kolla19:11
*** rmart04 has quit IRC19:12
*** jamesben_ has joined #openstack-kolla19:14
*** jamesbenson has quit IRC19:15
*** salv-orlando has quit IRC19:17
*** keico has quit IRC19:24
*** dims has quit IRC19:28
*** ducttape_ has quit IRC19:28
*** ducttape_ has joined #openstack-kolla19:28
*** dims has joined #openstack-kolla19:31
openstackgerritsean mooney proposed openstack/kolla-ansible master: This change allow setting kernel commandline args  https://review.openstack.org/40887619:33
openstackgerritsean mooney proposed openstack/kolla-ansible master: add ovs-dpdk docs  https://review.openstack.org/49391319:33
openstackgerritsean mooney proposed openstack/kolla-ansible master: automate ovs datapath configuration  https://review.openstack.org/49391619:33
openstackgerritsean mooney proposed openstack/kolla-ansible master: make dpdk tunnel ip assignment configurable  https://review.openstack.org/49393919:33
*** coolsvap has quit IRC19:39
*** jamesben_ has quit IRC19:45
*** jamesbenson has joined #openstack-kolla19:45
*** manheim_ has joined #openstack-kolla19:58
*** manheim_ has quit IRC20:00
*** manheim_ has joined #openstack-kolla20:00
*** manheim_ has quit IRC20:04
*** lpetrut has joined #openstack-kolla20:10
*** ys__ has joined #openstack-kolla20:11
ys__hello, I am testing kolla-ansible 4.0.2 with ansible version 2.0.0.0 in centOS.  I got error for task [nova : include].20:13
*** salv-orlando has joined #openstack-kolla20:13
ys__fatal: [con]: FAILED! => {"failed": true, "reason": "ERROR! The field 'until' is supposed to be a string type, however the incoming data structure is a <class 'ansible.parsing.yaml.objects.AnsibleSequence'>\n\nThe error appears to have been in '/usr/share/kolla-ansible/ansible/roles/nova/tasks/simple_cell_setup.yml': line 2, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending lin20:14
ys__i am pulling images from docker hub directly20:14
ys__is this a version issue?20:15
*** awiddersheim has quit IRC20:18
*** salv-orlando has quit IRC20:18
*** awiddersheim has joined #openstack-kolla20:19
*** kbaegis has quit IRC20:22
ys__seems complaining: https://github.com/openstack/kolla-ansible/blob/129ad93c5244764c890f8277efedcbd665920bcd/ansible/roles/nova/tasks/simple_cell_setup.yml#L2020:25
Manheimhi, ansible should be <2.0.0, https://docs.openstack.org/kolla-ansible/latest/quickstart.html20:26
Manheimsorry, that was mitaka20:26
Manheimocata should be ok with ansible 2.0.020:27
Manheimyou have 4.0.0 images?20:27
Manheimif you have 4.0.2 kolla installed, try to build 4.0.2 images then, I had issues in the past due to version differences20:28
*** jamesbenson has quit IRC20:28
*** jamesbenson has joined #openstack-kolla20:37
*** pcaruana has quit IRC20:40
*** ntpttr_laptop has joined #openstack-kolla20:41
*** manheim_ has joined #openstack-kolla20:41
*** fuegoel has quit IRC20:44
egonzalezys__ it is a bug in ansible not allowing lists for some conditionals, could you try with 2.0.2.0 (minor update) https://github.com/ansible/ansible/pull/1483320:44
ys__egonzalez: thanks!20:48
*** lucasxu has quit IRC20:49
*** ducttape_ has quit IRC21:02
*** jtriley has quit IRC21:03
*** ducttape_ has joined #openstack-kolla21:04
*** Pavo has joined #openstack-kolla21:04
openstackgerritBertrand Lallau proposed openstack/kolla master: WIP: add Almanach Docker images  https://review.openstack.org/49893821:05
*** salv-orlando has joined #openstack-kolla21:14
*** salv-orlando has quit IRC21:19
*** krtaylor has quit IRC21:20
openstackgerritBertrand Lallau proposed openstack/kolla-ansible master: WIP: Add Almanach deployment  https://review.openstack.org/49894421:22
*** tvignaud has joined #openstack-kolla21:24
*** jamesbenson has quit IRC21:27
*** matrohon has joined #openstack-kolla21:28
*** Pavo has quit IRC21:30
*** jamesbenson has joined #openstack-kolla21:30
*** egonzalez has quit IRC21:30
*** jamesbenson has quit IRC21:34
*** manheim_ has quit IRC21:35
*** vhosakot has quit IRC21:37
*** ducttape_ has quit IRC21:38
*** lpetrut has quit IRC21:45
*** ducttape_ has joined #openstack-kolla21:46
*** matrohon has quit IRC21:55
*** thegreenhundred has quit IRC21:56
*** tvignaud has quit IRC21:59
*** krtaylor has joined #openstack-kolla22:06
openstackgerritManjeet Singh Bhatia proposed openstack/kolla-ansible master: Update document for Host Ubuntu  https://review.openstack.org/49937022:09
*** dave-mccowan has quit IRC22:12
*** salv-orlando has joined #openstack-kolla22:15
*** thegreenhundred has joined #openstack-kolla22:18
*** salv-orlando has quit IRC22:20
*** eaguilar has joined #openstack-kolla22:20
*** itlinux has quit IRC22:23
openstackgerritManjeet Singh Bhatia proposed openstack/kolla-ansible master: Update document to add note  https://review.openstack.org/49937022:33
*** rstarmer has joined #openstack-kolla22:35
openstackgerritManjeet Singh Bhatia proposed openstack/kolla-ansible master: Update document to add note  https://review.openstack.org/49937022:36
*** thegreenhundred has quit IRC22:39
*** kbyrne has quit IRC22:40
*** kbyrne has joined #openstack-kolla22:41
*** ducttap__ has joined #openstack-kolla22:49
sdakekfox1111 see PM plz22:50
*** ducttape_ has quit IRC22:52
*** imcsk8 has quit IRC22:53
manjeetsegonzalez I guess dockerd was added over an year ago so wouldn't it be better docs uses dockerd instead of docker daemon and leave a note for docker daemon ?22:55
manjeetsinc0, are you in office today ?22:56
kolla-slack<egonzalez> manjeets, our role to deploy baremetal installs engine 12 https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/baremetal/defaults/main.yml#L2422:59
inc0manjeets: negative22:59
kolla-slack<egonzalez> Guess is too late for upgrading to newer versions23:00
inc0what's up? docker 1.12 no worky on ubuntu?23:00
manjeetsinc0, docker is now working I have another issue which I can't get over right now, kolla-build is building 51 images and failing here is trace http://paste.openstack.org/show/619954/23:03
inc0manjeets: I'll come over tomorrow and look at it ok?23:06
manjeetsinc0, ohk thanks23:06
*** hongbin has quit IRC23:10
*** ducttap__ has quit IRC23:12
*** salv-orlando has joined #openstack-kolla23:16
manjeetsinc0, I think it just worked, I removed kolla-build.conf just used the tags along with command seems like working reahced upto 117 images still going23:16
*** salv-orlando has quit IRC23:21
openstackgerritOpenStack Proposal Bot proposed openstack/kolla master: Updated from global requirements  https://review.openstack.org/49482223:21
*** ducttape_ has joined #openstack-kolla23:21
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements  https://review.openstack.org/49482323:21
*** itlinux has joined #openstack-kolla23:22
*** ducttape_ has quit IRC23:26
*** ddyer has quit IRC23:34
*** manheim_ has joined #openstack-kolla23:36
*** rstarmer has quit IRC23:38
kfox11111.12 is the latest docker most distro's will support.23:39
kfox1111it was the last version to not include swarm.23:39
kfox1111its fairly likely it will stay that way until containerd 1.0.23:40
*** manheim_ has quit IRC23:40
*** rhallisey has quit IRC23:42
*** zhangfei has joined #openstack-kolla23:50
*** kbyrne has quit IRC23:51
*** eaguilar has quit IRC23:52
*** eaguilar has joined #openstack-kolla23:53
*** kbyrne has joined #openstack-kolla23:54
*** zhangfei has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!