Tuesday, 2017-10-24

SamYaplebut yea anyway if you run `ceph-authtool -p -C /dev/null -g`00:00
SamYapleit will pump out a key thats note tied to the cluster00:00
SamYaplejust the raw key00:00
SamYapleits a struct with a timestamp and then the random data00:01
SamYapleyou can actually make it by hand without issue00:01
*** goldyfruit has joined #openstack-kolla00:05
inc0SamYaple: I'm thinking we could do it next release00:10
SamYaplei think to deprecate you need a migration path in place. so if you get taht work done by queens, then yea00:11
SamYaplebasically in queens people would need to be able to migrate to the new method00:12
*** yangyapeng has quit IRC00:12
inc0well if we deprecate in queens, then by rocky we need to have migration path00:12
inc0by deprecate in queens I mean queens still have fully functional L00:13
*** rwsu has joined #openstack-kolla00:13
inc0but rocky have migration path00:13
*** yangyapeng has joined #openstack-kolla00:13
inc0and requirement for upgrade Q->R would be to externalize ceph00:13
SamYapleinc0: you need the migration path for a full cycle though00:13
SamYaplenah you cant do that00:13
inc0why tho?00:14
SamYaplepolicy?00:14
SamYaplei mean theres no *technical* reason00:14
inc0well in any case we can still have ceph in Rocky00:14
inc0I mean if we upgrade to L it's just gonna be bugfix00:14
SamYaplebut you need a time period (a cycle) of old way depcreated with new way working00:14
SamYapleso if thats not there by Q, you cant remove til S00:15
inc0so keeping ceph code for till S isn't issue00:15
inc0I mean we can do it00:15
SamYapleas long as you have a migration path for a cycle before you remove the code you can call it whatever you want00:16
inc0that was the plan00:16
inc0if we get migration path in Q, all the better00:16
SamYaple00:13:50 < inc0> and requirement for upgrade Q->R would be to externalize ceph00:16
SamYaplethats whats confusing me there00:16
inc0well00:16
inc0you have Q00:16
inc0you want to upgrade to R00:16
inc0since R is already released00:17
inc0part of R is this magical playbook to move ceph from kolla to external00:17
SamYapleyea thats not proper deprecation00:17
inc0run this magical playbook before you call kolla-ansible upgrade00:17
*** jtriley has joined #openstack-kolla00:17
*** yangyapeng has quit IRC00:17
SamYaplein that case you couldnt call it deprecated until R00:17
SamYaplewith an S removal00:17
inc0we can do it00:17
inc0although you could argue that external ceph is already working00:18
inc0because it is00:18
SamYapleok let me rephrase, policy says you cant do that00:18
SamYaplemigration path00:18
inc0you need release of both working, external and kolla-supported00:18
inc0and that's reality today00:18
SamYaplewith a migration path00:18
SamYapleall in one release00:18
inc0yeah I guess00:18
inc0anyway, we can keep ceph inR00:18
SamYaplelisten you do what you want, but its policy because otherwise you break people00:18
inc0L is lts anyway00:18
inc0I don't think you'll break in our case - it's not like you need to cause downtime for your cloud00:19
inc0but we'll do what will be best for operators00:19
inc0if that means keeping ceph code for one more release, we'll keep code for one more release00:20
openstackgerritHeetae Ahn proposed openstack/kolla master: Add RHEL subscription registration  https://review.openstack.org/49514800:40
*** huanxie has joined #openstack-kolla00:44
*** tovin07_ has joined #openstack-kolla00:47
*** kolla-slack has quit IRC01:00
*** kolla-slack has joined #openstack-kolla01:00
*** huanxie has quit IRC01:01
openstackgerrityanpeifei proposed openstack/kolla master: update "copy_links:true" ARG.  https://review.openstack.org/51449301:02
*** kiennt26 has joined #openstack-kolla01:02
*** dave-mccowan has joined #openstack-kolla01:02
*** sbezverk has quit IRC01:06
*** sbezverk has joined #openstack-kolla01:07
*** yangyapeng has joined #openstack-kolla01:12
*** yangyapeng has quit IRC01:12
*** yangyapeng has joined #openstack-kolla01:13
*** daidv has joined #openstack-kolla01:16
*** jinke_ has joined #openstack-kolla01:19
*** hrw has joined #openstack-kolla01:19
*** Mratrend has joined #openstack-kolla01:24
*** zhubingbing has joined #openstack-kolla01:26
*** pramodrj07 has quit IRC01:34
*** MasterOfBugs has quit IRC01:34
*** salv-orlando has joined #openstack-kolla01:36
*** salv-orlando has quit IRC01:40
*** jtriley has quit IRC01:44
MratrendHi guys! I have a question, How to use kolla-ansible to deploy neutron-metering-agent  containers?  globals.yml havent enable_neutron_metering_agent option01:53
*** zhangfei has joined #openstack-kolla01:58
*** yingjun has joined #openstack-kolla02:00
Jeffrey4linc0, around?02:04
*** yangyape_ has joined #openstack-kolla02:07
*** yangyapeng has quit IRC02:10
*** jtriley has joined #openstack-kolla02:12
*** zshi has quit IRC02:14
*** zshi has joined #openstack-kolla02:16
openstackgerritMerged openstack/kolla-ansible master: Remove check flat network type task in ironic pre-check.yml  https://review.openstack.org/51365102:24
inc0Jeffrey4l: now I am02:37
Jeffrey4linc0, what kinds of issue did u see?02:37
inc0so masber notices something that I tried to debug in gates02:37
inc0mariadb failed to bootstrap02:37
inc0but for him (and gates as it turns out) it was issue that keepalived didn't setup floating ip02:38
Jeffrey4ldid u get any error message?02:38
inc0I don't have env to reproduce02:38
Jeffrey4lok.02:38
inc0timeouts on mariadb bootsrap02:38
inc0http://logs.openstack.org/79/512779/13/check/kolla-ansible-ubuntu-source-ceph/d609bd6/ <- look here02:39
Jeffrey4lso this happens in master?02:39
inc0yeah, and possibly 5.002:40
Jeffrey4lhrm, allow address pair issue?02:41
Jeffrey4lvip can not be used in openstack directly. need configure through --all-address-pair02:41
*** calebb has quit IRC02:41
inc0I know, in gates I used vxlan tunnel02:42
Jeffrey4lbtw, deploy jobs is working now. https://review.openstack.org/51397102:42
inc0nice02:43
Jeffrey4lseems both keepalived run into master state.02:43
inc0right02:43
Jeffrey4lso i still doubt the underlay openstack issue ( address pair (02:44
inc0well, no, I dont think so02:44
Jeffrey4lok. i will check this today.02:45
inc0I've setup vxlan tunnel spanning across nodes02:45
inc0and used it for api_interface02:45
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361202:45
inc0that wou;dn't be affected by address pairs02:45
Jeffrey4lyes.02:45
inc0double check it, and I'll try it tomorrow too02:45
Jeffrey4li will.02:46
inc0https://review.openstack.org/#/c/512779/19 also, if you have time, see if you can figure out what's wrong here02:48
inc0I disabled keepalived, but mariadb still fails to bootstrap02:49
inc0so it might not be keepalived after all...02:49
*** heigao has joined #openstack-kolla02:50
Jeffrey4lroger.02:51
inc0thanks02:51
*** calebb has joined #openstack-kolla02:51
inc0I'll look at that tomorrow, have a great day!02:51
*** heigao has quit IRC02:52
*** heigao has joined #openstack-kolla02:52
Jeffrey4lthanks. good night.02:54
masberI am going to deploy ceph using ceph-ansible and then I would like to deploy kolla and connect it to my ceph cluster as described here https://docs.openstack.org/kolla-ansible/latest/reference/external-ceph-guide.html02:58
masbermy question is, who is going to deploy ceph clients? ceph-ansible or kolla?02:58
masber*who should02:59
SamYaplemasber: the clients are baked into the images03:02
SamYaplemasber: so you need to build your images with the clients03:03
SamYapleif you just want the clients on hosts outside of kolla, ceph-ansible can handle that for you03:03
masberSamYaple, ok, I guess prefer to deploy ceph client through kolla as I have no idea how to connect cinder/swift with the ceph clients...03:04
masberSamYaple, just curious, what is your preferred way of deploying ceph? though kolla or ceph-ansible?03:05
masberSamYaple, and have you noticed performance degradation by running ceph and compute on same nodes?03:05
*** jinke_ has quit IRC03:07
*** jinke_ has joined #openstack-kolla03:08
*** yangyape_ has quit IRC03:08
*** yangyapeng has joined #openstack-kolla03:09
*** yangyape_ has joined #openstack-kolla03:10
spsuryamorning all03:12
*** yangyapeng has quit IRC03:16
SamYaplemasber: hyperconveraged can impact performance yes. if you have enough nodes, seperating everything out is, im sure, prefered.03:21
*** mdnadeem has joined #openstack-kolla03:27
*** lukepatrick has joined #openstack-kolla03:37
*** salv-orlando has joined #openstack-kolla03:37
openstackgerritHanxi Liu proposed openstack/kolla-kubernetes master: Remove extra characters  https://review.openstack.org/51451103:39
*** salv-orlando has quit IRC03:42
*** dave-mccowan has quit IRC03:47
*** goldyfruit has quit IRC03:53
*** heigao has quit IRC03:54
*** gkadam_ has joined #openstack-kolla03:56
*** newmember has joined #openstack-kolla04:07
*** yangyape_ has quit IRC04:24
*** yangyapeng has joined #openstack-kolla04:25
*** gnupyx has quit IRC04:45
*** skramaja has joined #openstack-kolla04:47
*** kiennt26 has quit IRC04:51
*** lukepatrick has quit IRC04:53
*** jtriley has quit IRC04:54
*** janki has joined #openstack-kolla04:55
*** salv-orlando has joined #openstack-kolla05:05
*** jinke_ has quit IRC05:12
*** dmellado has quit IRC05:21
*** dmellado has joined #openstack-kolla05:22
*** janki has quit IRC05:23
*** janki has joined #openstack-kolla05:23
*** janki has quit IRC05:28
kolla-slack<egonzalez> @masber reading scroll back to your keepaalived issue, can you check keepalived logs, also ensure no other vrrp vid is not used in any other place in the same network https://github.com/openstack/kolla-ansible/blob/175c65e4900c9cc68917242fd7c9677fc32a2087/ansible/group_vars/all.yml#L11805:29
*** janki has joined #openstack-kolla05:30
kolla-slack<masber> @egonzalez what's vrrp?05:31
kolla-slack<egonzalez> Thw protocol used by keepalived05:32
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/pike: Optimize zuul v3 jobs  https://review.openstack.org/51451705:32
*** lpetrut has joined #openstack-kolla05:33
kolla-slack<masber> @egonzalez ok, I checked and the vid assigned is free in the network05:34
pratapagouthamkolla-slack: I could clear all the prechecks tasks but i have an issue in pulling the images present in docker-registry05:35
pratapagouthamcommand i used is : kolla-ansible pull -i  <multi_node_file> and error it is throwing up is  https://hastebin.com/dugimabaha.py05:35
pratapagouthamkolla-slack: any help ?? please05:35
*** jtriley has joined #openstack-kolla05:37
*** dmellado has quit IRC05:37
SamYaplepratapagoutham: the 5.0.0 images dont exist in dockerhub05:38
SamYapleyoull need to build them yourself05:38
*** egonzalez has joined #openstack-kolla05:39
pratapagouthamSamYaple:  yes  i have built it with this "kolla-build --base ubuntu --type source --registry 172.16.69.71:5443 --push "05:40
*** dmellado has joined #openstack-kolla05:40
SamYapleah i see that now05:41
SamYapledoes `docker pull  172.16.69.71:5443/kolla/ubuntu-binary-kolla-toolbox:5.0.0` work for you?05:42
SamYapleoh wait, there is the issue05:42
SamYaplebinary vs source05:42
SamYapleyou built from source images (good) but you didnt set the type in globals.yaml for deploying with kolla-ansible05:42
SamYaplepratapagoutham: ^05:42
pratapagouthamhaan yes i have that to source in /etc/kolla/globals.yml05:44
pratapagouthamSamYaple: ^05:44
SamYapleits definetely trying to pull binary images, so perhaps your value is getting overridden somewhere?05:45
pratapagouthamSamYaple: yes seems this solved the issue05:45
SamYaplepratapagoutham: awesome! good luck05:45
SamYaplenighttime for me05:46
pratapagouthamSamYaple: https://hastebin.com/exoweqakon.py but then this??05:46
egonzalezpratapagaoutham, check docker py version installed. No docker-py should be present in the deploy host05:49
egonzalezOnly docker and docker-pycreds05:49
pratapagouthamegonzalez: only docker is there05:51
egonzalezHrm, maybe is insecure registey not set in docker systemd service05:51
pratapagouthami installed it with pip install docker05:51
*** vharsh has joined #openstack-kolla05:59
*** kevinz has quit IRC06:00
*** kevinz has joined #openstack-kolla06:00
*** tovin07_ has quit IRC06:01
spsuryapbourke: can you please review https://review.openstack.org/#/c/425446/06:06
*** coolsvap has joined #openstack-kolla06:06
pratapagouthamany one who can help me out with this error while pulling images "https://hastebin.com/exoweqakon.py "06:07
*** zhangfei has quit IRC06:09
*** jtriley has quit IRC06:14
*** brad[] has quit IRC06:18
*** brad[] has joined #openstack-kolla06:19
*** Mratrend has quit IRC06:19
*** lpetrut has quit IRC06:21
*** egonzalez has quit IRC06:21
*** zhangfei has joined #openstack-kolla06:22
*** egonzalez has joined #openstack-kolla06:25
*** zhubingbing_ has joined #openstack-kolla06:26
korean101ERROR 2003 (HY000): Can't connect to MySQL server on '172.16.0.99' (113 \"No route to host\")06:27
*** zhubingbing has quit IRC06:27
korean101hi. why my multinode deployment is wrong?06:29
korean101got an ERRORs06:29
*** tvignaud has joined #openstack-kolla06:30
masberjust double checking, if I decide to install ceph by myself, should I deploy jewel or kraken?06:31
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361206:33
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/pike: Optimize zuul v3 jobs  https://review.openstack.org/51451706:37
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/pike: Move to zuul v3 in project jobs  https://review.openstack.org/51360806:41
*** jbadiapa_ has joined #openstack-kolla06:41
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible stable/pike: Optimize zuul v3 jobs  https://review.openstack.org/51451706:41
*** jbadiapa has quit IRC06:44
*** danardelean has joined #openstack-kolla06:45
*** zhangfei has quit IRC06:46
*** danardelean has quit IRC06:50
*** dardelean has joined #openstack-kolla06:53
*** gfidente has joined #openstack-kolla06:55
*** gfidente has quit IRC06:55
*** gfidente has joined #openstack-kolla06:55
*** kevinz has quit IRC06:59
*** kevinz has joined #openstack-kolla07:00
*** cah_link has joined #openstack-kolla07:00
korean101kolla_internal_vip_address, kolla_external_vip_address not on arp table07:01
korean101(incomplete)07:01
korean101someone knows this?07:01
*** pcaruana has joined #openstack-kolla07:02
*** zhangfei has joined #openstack-kolla07:03
*** jtriley has joined #openstack-kolla07:04
*** cah_link has quit IRC07:04
*** kiennt26 has joined #openstack-kolla07:12
*** cah_link has joined #openstack-kolla07:14
*** yangyape_ has joined #openstack-kolla07:14
*** yangyapeng has quit IRC07:16
openstackgerritChason Chan proposed openstack/kolla-ansible master: Add content about cluster_network interface  https://review.openstack.org/51454407:18
*** cah_link has quit IRC07:18
*** danardelean has joined #openstack-kolla07:19
*** vharsh has quit IRC07:22
*** rmart04 has joined #openstack-kolla07:25
*** zhangfei has quit IRC07:25
*** vharsh has joined #openstack-kolla07:26
*** yangyape_ has quit IRC07:27
*** yangyapeng has joined #openstack-kolla07:27
*** egonzalez_afk has joined #openstack-kolla07:30
*** yangyape_ has joined #openstack-kolla07:30
*** egonzalez_afk has quit IRC07:31
*** egonzalez has quit IRC07:31
*** yangyapeng has quit IRC07:32
*** jaosorior has quit IRC07:33
*** egonzalez has joined #openstack-kolla07:37
*** zhangfei has joined #openstack-kolla07:38
korean101http://paste.openstack.org/show/624426/ ERROR about multinode deploy07:44
*** dasTor has quit IRC07:44
egonzalezkorean101, check if mariadb is running in the hosts, also ensure the VIP is bind into correct interface in network nodes07:45
korean101egonzalez: mariadb in node-00107:46
korean101egonzalez: haproxy in node-00207:46
korean101egonzalez: node-001 --> haproxy ping fails07:46
korean101[root@node-001 ~]# docker exec mariadb mysql -h 10.0.0.99 -P 3306 -u haproxy -e show databases;07:48
korean101ERROR 2003 (HY000): Can't connect to MySQL server on '10.0.0.99' (113 "No route to host")07:48
openstackgerritChason Chan proposed openstack/kolla master: Upgrade the rst convention of the Admin Guide  https://review.openstack.org/51165907:48
korean101can't reach haproxy (10.0.0.99)07:49
korean101OMG... so hard07:49
egonzalezkorean101, is 10.0.0.99 the IP of the haproxy host or the VIP?07:49
korean101egonzalez: yes07:50
korean101egonzalez: kolla_internal_vip_address: "10.0.0.99"07:50
egonzalezcheck if the vip is bind in the interface07:50
korean101egonzalez: arp -a node-002 (10.0.0.99) at <incomplete> on eth007:50
korean101egonzalez:     inet 10.0.0.99/32 scope global eth007:50
korean101bind is ok07:50
korean101egonzalez: but can't ping07:51
korean101other nodes...07:51
egonzalezkorean101, from the eth0 IP can ping to mariadb hosts?07:51
korean101egonzalez: yes!07:51
korean10110.0.0.101 (mariadb) 10.0.0.102 (haproxy)07:51
korean10110.0.0.99 (haproxy VIP)07:52
korean101.101 <--> .102 ping each other07:52
korean101egonzalez: but can't ping 10.0.0.9907:52
korean101egonzalez: but ip addr command show 10.0.0.99 (eth0)07:52
korean101egonzalez: but arp incomplete07:53
korean101i use 3 openstack VMs07:53
egonzalezhrm, is port security disabled or allowed-addresses in the VIP?07:53
egonzalezneutron will block any conn not matching mac addr07:53
bjoloegonzalez: was just about to ask the same :)07:54
korean101egonzalez: http://paste.openstack.org/show/624427/07:55
korean101bjolo: same error?07:55
egonzalezkorean101, if not have access to enable allowed addresses and only have one haproxy node, just disable haproxy and set vip to the host IP07:55
korean101egonzalez: all secgroup is allowed07:56
korean101| tcp         | 1         | 65535   | 0.0.0.0/0 |              |07:57
korean101| udp         | 1         | 65535   | 0.0.0.0/0 |              |07:57
korean101| icmp        | -1        | -1      | 0.0.0.0/0 |              |07:57
*** jaosorior has joined #openstack-kolla07:57
korean101egonzalez: prohibit use openstack VMs for target hosts?07:57
korean101only use baremetal?07:57
*** dougsz has joined #openstack-kolla08:00
egonzalezkorean101, neutron will block the mac if does not match with the port to avoid arp spoofing, will need to disable or port-security or add allowed-addresses-pair to the eth0 port08:02
*** vharsh has quit IRC08:10
*** ktibi has joined #openstack-kolla08:24
*** danardelean has quit IRC08:26
*** danardelean has joined #openstack-kolla08:27
*** danardelean has quit IRC08:31
*** dougsz1 has joined #openstack-kolla08:37
openstackgerritzhubingbing proposed openstack/kolla-ansible master: Use internal_protocol variable to replace http  https://review.openstack.org/51456108:38
*** dougsz has quit IRC08:40
*** cah_link has joined #openstack-kolla08:40
*** danardelean has joined #openstack-kolla08:41
korean101egonzalez: OK. got it08:43
*** cah_link has quit IRC08:47
*** danardel_ has joined #openstack-kolla08:49
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: [WIP] Add support policy.yaml file - Aodh  https://review.openstack.org/51456508:52
*** danardelean has quit IRC08:53
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: [WIP] Add support policy.yaml file - Aodh  https://review.openstack.org/51456508:54
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: [WIP] Add support policy.yaml file - Aodh  https://review.openstack.org/51456508:55
korean101jesus08:55
korean101egonzalez: you're genius08:56
korean101egonzalez: ping's OK!08:56
korean101egonzalez: oh god... thanks08:56
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/pike: cinder-api: handle Debian too  https://review.openstack.org/51456708:56
hrwmorning08:57
*** yangyapeng has joined #openstack-kolla08:57
hrwegonzalez: can you approve https://review.openstack.org/#/c/514567/ one? It is backport for stable/pike to make cinder-api working with Debian. bug was reported as it should08:57
*** yangyape_ has quit IRC09:01
egonzalezhrw, please, next time do a cherry-pick for backports09:03
*** daidv has quit IRC09:03
hrwegonzalez: to keep same changeid?09:03
egonzalezyep09:03
hrwok, will do09:03
*** daidv has joined #openstack-kolla09:03
hrwwill push cherrypick one then09:03
*** daidv_ has joined #openstack-kolla09:04
*** daidv_ has quit IRC09:04
*** daidv has quit IRC09:04
*** daidv has joined #openstack-kolla09:04
*** daidv has quit IRC09:05
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/pike: cinder-api: handle Debian too  https://review.openstack.org/51457309:06
*** daidv has joined #openstack-kolla09:06
ZopharHello... when enabling Designate the task is hanging at "Update DNS pools" (designate-manage pool update).... in designate_manage.log i got "2017-10-24 10:56:38.501 18 ERROR oslo.messaging._drivers.impl_rabbit [designate-manage - - - - -] [31d094af-b00d-42b3-ba2b-6c128daf652b] AMQP server on 127.0.0.1:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 32 seconds. Client port: None: error: [Errno 111] ECONNREFUSED"09:07
hrwegonzalez: resent09:07
ZopharBut the transport_url in designate point on the right ip.. not on localhost09:07
Zopharany id?09:07
*** lxdong has quit IRC09:07
ktibiZophar send all you designate conf file09:08
Zopharok give me 1 min09:08
Zopharhttp://paste.openstack.org/show/624443/09:10
*** gkadam_ has quit IRC09:11
Zopharthis is designate.conf of designate_worker (where the pool update happens)... don't know if other conf files are relevant here09:12
*** gkadam has joined #openstack-kolla09:13
Zopharrabbitmq is listening on 10.56.18.192:567209:13
Zophartested with telnet and i see messages comming from other service09:13
*** dasTor has joined #openstack-kolla09:14
*** dasTor_ has joined #openstack-kolla09:14
*** dasTor_ has quit IRC09:14
openstackgerritChristian Berendt proposed openstack/kolla master: Make env variables accessible in template files  https://review.openstack.org/50845909:15
hrwdoes someone know how to manage automatic push to hub.docker.com from CI without adding username/pass into job config?09:15
openstackgerritMerged openstack/kolla-ansible master: Unify global syntax in baremetal role  https://review.openstack.org/50513209:17
*** athomas has quit IRC09:32
*** dasTor has quit IRC09:33
*** dasTor has joined #openstack-kolla09:33
*** danardel_ has quit IRC09:38
*** danardelean has joined #openstack-kolla09:39
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Add support policy.yaml file - Aodh  https://review.openstack.org/51456509:40
*** yangyape_ has joined #openstack-kolla09:40
*** danardelean has quit IRC09:40
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Add support policy.yaml file - Aodh  https://review.openstack.org/51456509:41
*** dasTor has quit IRC09:41
*** zhubingbing_ has quit IRC09:43
*** sambetts|afk is now known as sambetts09:43
*** yangyapeng has quit IRC09:44
*** yingjun has quit IRC09:45
Zopharanyone has an idea on designate rabbitmq issue?09:48
Zopharwhy the hell is trying to connect on 127.0.0.1?09:49
masberis it documented somewhere how tell nova to use local storage for ephemeral while using ceph for volumes?09:49
*** danardelean has joined #openstack-kolla09:50
*** danardelean has quit IRC09:50
*** danardelean has joined #openstack-kolla09:50
*** huanxie has joined #openstack-kolla09:51
*** hachi_ has joined #openstack-kolla09:52
kolla-slack<aslancimbom100> there should be an option in globals09:52
kolla-slack<aslancimbom100> masber...something like #nova_backend_ceph: "{{ enable_ceph }}"09:52
ZopharOk strange i found what going wrong on designate manage09:56
Zopharin designate.conf the rabbitmq is set on transport_url (which should be fine) as documented09:57
Zophari try to set it in oslo_messaging_rabbit section09:57
Zopharand it works09:57
ZopharWhy!?!?09:57
Zopharthe log says it's depracated... but now the designate-manage pool update process run (and doesn't hang)09:58
egonzalezZophar, is a known bug in designate10:00
openstackgerritEduardo Gonzalez proposed openstack/kolla-ansible master: Revert "Remove deprecated oslo_messaging_rabbit options"  https://review.openstack.org/51458510:00
egonzalezis not fixed yet afik10:01
*** yangyapeng has joined #openstack-kolla10:02
ZopharEdouardo you're my hero ;)10:03
*** yangyapeng has quit IRC10:03
*** yangyapeng has joined #openstack-kolla10:03
*** yangyape_ has quit IRC10:06
*** egonzalez has quit IRC10:06
*** yangyapeng has quit IRC10:08
Zopharbtw it has been fixed here: https://bugs.launchpad.net/designate/+bug/165157610:09
openstackLaunchpad bug 1651576 in Designate "designate-manage does not properly support DEFAULT/transport_url" [Medium,Fix released] - Assigned to Dr. Jens Harbott (j-harbott)10:09
Zophar16th september10:09
Zopharwill do a review to push it in stable/pike10:11
Zopharon designate repo... thank for your help10:11
masber #nova_backend_ceph will tell nova to use ceph for ephemeral, I want tell nova to use local storage as ephemeral10:14
*** daidv has quit IRC10:14
berendtmasber: use overlay files to override the nova compute configuration10:27
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Add hostname to all log messages  https://review.openstack.org/51459310:30
*** jinke_ has joined #openstack-kolla10:37
*** pbourke has quit IRC10:38
*** pbourke has joined #openstack-kolla10:40
*** dasTor has joined #openstack-kolla10:42
*** lpetrut has joined #openstack-kolla10:42
kolla-slack<aslancimbom100> what happens if you set #nova_backend_ceph to no? doesn't it fallback?10:44
spsuryakrtaylor: around ?10:52
*** egonzalez has joined #openstack-kolla10:54
kevinzkfox1111: ping10:59
*** zhangfei has quit IRC10:59
*** kevinz has quit IRC11:01
*** kevinz has joined #openstack-kolla11:02
kevinzHi kolla, I have a problem when deploying with Kolla-kubernetes.  I've followed the "development guide", run the command: kollakube res create configmap \11:06
kevinz    mariadb11:06
kevinzbut when I run helm to enable mariadb: helm install --debug kolla-kubernetes/helm/service/mariadb --namespace kolla --name mariadb --values ./cloud.yaml11:07
*** serlex has joined #openstack-kolla11:07
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Add support policy.yaml file - Aodh  https://review.openstack.org/51456511:08
openstackgerritDai Dang Van proposed openstack/kolla-ansible master: Add support policy.yaml file - Aodh  https://review.openstack.org/51456511:08
kevinzGot issue: http://paste.openstack.org/show/624452/11:09
*** newmember has quit IRC11:10
openstackgerritChangBo Guo(gcb) proposed openstack/kolla master: Remove redundant statement in help string of config option 'base'  https://review.openstack.org/51460811:22
*** huanxie has quit IRC11:29
*** yangyapeng has joined #openstack-kolla11:32
*** yangyapeng has quit IRC11:33
*** yangyapeng has joined #openstack-kolla11:33
openstackgerritMarcin Juszkiewicz proposed openstack/kolla stable/pike: base: use new Linaro repo  https://review.openstack.org/51461411:38
*** jinke_ has quit IRC11:38
*** lhx__ has joined #openstack-kolla11:42
openstackgerritChangBo Guo(gcb) proposed openstack/kolla master: Remove duplicated statement of parameter choices of config options  https://review.openstack.org/51460811:44
openstackgerritChangBo Guo(gcb) proposed openstack/kolla master: Fix docstring of method run_build  https://review.openstack.org/51461811:53
*** ansmith has quit IRC11:56
hrwany update on 6.0.0 tag?11:57
*** lhx__ has quit IRC12:18
*** hachi_ has quit IRC12:24
openstackgerritJuan Antonio Osorio Robles proposed openstack/kolla master: Add rsyslog image  https://review.openstack.org/51463212:25
*** tvignaud has quit IRC12:25
openstackgerritJuan Antonio Osorio Robles proposed openstack/kolla master: Add rsyslog image  https://review.openstack.org/51463212:25
*** dave-mccowan has joined #openstack-kolla12:31
*** tvignaud has joined #openstack-kolla12:34
*** tvignaud has quit IRC12:34
*** tvignaud has joined #openstack-kolla12:35
*** goldyfruit has joined #openstack-kolla12:41
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Extract log level from syslog  https://review.openstack.org/51463712:45
*** sbezverk has quit IRC12:45
openstackgerritVladislav Belogrudov proposed openstack/kolla-ansible master: Cinder fails some operations with secure NAS features  https://review.openstack.org/51463912:46
*** tvignaud has quit IRC12:52
*** tvignaud_ has joined #openstack-kolla12:52
*** tvignaud_ is now known as tvignaud12:54
rwellumhi kevinz : did you get past your mariadb issue?12:55
*** Sean-md has joined #openstack-kolla13:00
*** ansmith has joined #openstack-kolla13:02
*** jtriley has quit IRC13:03
krtaylormorning kolla13:07
krtaylorspsurya, you rang?13:08
*** athomas has joined #openstack-kolla13:12
Sean-mdHi all. I get "error" messages as below when trying to connect cinder-api using "curl" or "haproxy readiness"13:16
Sean-md"mod_wsgi (pid=74): Target WSGI script '/var/www/cgi-bin/cinder/cinder-wsgi' cannot be loaded as Python module."13:16
Sean-mdAll messages are https://paste.ubuntu.com/25809148/13:16
Sean-mdenv = kolla:5.0.1, centos713:16
Sean-mdDose anyone know a solution?13:16
egonzalezSean-md, IOError: [Errno 13] Permission denied: '/var/log/kolla/cinder/cinder-wsgi.log'13:18
*** goldyfruit has quit IRC13:20
*** skramaja has quit IRC13:21
*** goldyfruit has joined #openstack-kolla13:24
*** jtriley has joined #openstack-kolla13:27
*** Sean-md_ has joined #openstack-kolla13:30
openstackgerritPierre Hanselmann proposed openstack/kolla-ansible master: Add horizon domain dropdown  https://review.openstack.org/51467613:40
*** Sean-md_ has quit IRC13:41
openstackgerritPierre Hanselmann proposed openstack/kolla-ansible master: Add horizon domain dropdown  https://review.openstack.org/51467613:41
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361213:46
*** rmart04 has quit IRC13:49
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361213:51
*** coolsvap has quit IRC13:53
*** sbezverk has joined #openstack-kolla13:54
*** sbezverk has quit IRC13:59
*** lukepatrick has joined #openstack-kolla14:01
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361214:03
*** lukepatrick has quit IRC14:04
*** Sean-md_ has joined #openstack-kolla14:04
Sean-md_egonzalez, The log file dose not exist, I thought I could overlook, I will try after creating the the file. Thanks14:05
*** mdnadeem has quit IRC14:07
*** Sean-md_ has quit IRC14:07
masbergood afternoon memcached container fails, this is the error from container logs http://paste.openstack.org/raw/624483/14:09
masberI am using 5.0.1 centos binary14:10
masberalso, I am testing ceph integrated in kolla, uploading an image freezes14:12
*** kiennt26 has quit IRC14:14
*** shardy has joined #openstack-kolla14:15
*** hongbin has joined #openstack-kolla14:18
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Add missing hostnames to fluentd log messages  https://review.openstack.org/51469314:24
openstackgerritDoug Szumski proposed openstack/kolla-ansible master: Add missing hostnames to fluentd log messages  https://review.openstack.org/51459314:26
*** jamesbenson has joined #openstack-kolla14:39
jamesbensonmorning all :-)14:41
*** xion- has joined #openstack-kolla14:53
*** egonzalez has quit IRC15:03
*** salv-orlando has quit IRC15:03
*** salv-orlando has joined #openstack-kolla15:04
*** salv-orlando has quit IRC15:08
*** lrensing has joined #openstack-kolla15:09
*** danardelean has quit IRC15:12
*** danardelean has joined #openstack-kolla15:13
*** mummer has joined #openstack-kolla15:17
*** danardelean has quit IRC15:17
*** magicboiz has quit IRC15:22
*** magicboiz has joined #openstack-kolla15:22
*** TxGirlGeek has joined #openstack-kolla15:23
*** lpetrut has quit IRC15:23
*** lpetrut has joined #openstack-kolla15:23
*** athomas has quit IRC15:25
*** serlex has quit IRC15:25
*** magicboiz has quit IRC15:31
kfox1111morning.15:34
xion-evening :)15:37
kfox1111evening. :)15:39
jamesbensonevening ;-)15:42
ktibihi inc015:45
ktibiI resolve my issue with vxlan port, missing l2population ;)15:45
*** newmember has joined #openstack-kolla15:46
inc0ktibi: cool15:50
rwellumkfox1111: do you have a little time to look at my libvirt issue with Pike? Serguei said he could later this week but I'm impatient. :)15:50
kfox1111sure.15:51
kfox1111I can look a bit now, or later today.15:51
rwellumkfox1111: btw - I don't get this issue with ocata - which makes me very suspicious about the environment being the issue.15:51
kfox1111yeah.15:51
kfox1111we don't test pike much yet. :(15:51
kfox1111can you dump/upload libvirt's configmap?15:52
rwellumyes 1s15:52
*** jmccarthy has joined #openstack-kolla15:54
rwellumhttps://www.irccloud.com/pastebin/CKP4mxx5/15:54
rwellumThere's a big TODO for pike in that comment15:55
*** magicboiz has joined #openstack-kolla15:55
kfox1111can you edit the nova-libvirt daemonset and do the bin/bash sleep thing to it so we can dig in?15:55
rwellumyes15:56
ktibiI want to use vlan for my external network. So I need to conf network_vlan_ranges in my ml2 conf file. with kolla, I can modify this value with "{% if enable_ironic | bool %}"15:57
*** klindgren has quit IRC15:57
*** klindgren has joined #openstack-kolla15:57
ktibipossible to override this value ?15:58
inc0rwellum: hmm, where did you get your pike images?15:59
rwellumkfox1111: so I edit the config map for nova-libvirt and there's no kolla_start - correct config map?15:59
rwelluminc0: I'm using the registry method to download from tarball16:00
kfox1111rwellum: I think it is probably ok, but we'll have to debug.16:00
rwellumkfox1111: I recall the kolla_start before so not sure.16:00
kfox1111the kolla_start thing is in the daemonset16:00
kfox1111kubectl edit daemonset nova-libvirt -n kolla16:01
rwellumkfox1111: ah it's edit daemonset not edit configmap :)16:01
* kfox1111 nods16:01
ktibiok I found https://review.openstack.org/#/c/464675/16:02
rwellumkfox1111: ok edited - now restart the container?16:02
*** magicboiz has quit IRC16:02
kfox1111yeah. just nuke it and it hsould come back.16:02
*** salv-orlando has joined #openstack-kolla16:05
inc0rwellum: so I remember how this user (todo) thing would show - you couldn't access console log16:05
inc0whcih is not the case for you right? libvirt doesn't even start16:05
rwellumok done, it's up and I am exec'ed in16:05
rwelluminc0: yes correct - container crashing16:05
inc0any logs?16:05
kfox1111do a kolla_start. see if it dies.16:05
kfox1111if it does, pastebin /var/log/kolla/libvirt/*16:06
rwelluminc0: I'm on the container now - debugging with kfox111116:06
inc0yeah do what kfox1111 said:)16:06
rwellum:)16:06
* rwellum doing16:06
kfox1111:)16:06
inc0kfox1111: you pass envs with k8s right?16:06
kfox1111inc0: environment variables?16:07
rwellumkfox1111 ^^^ https://www.irccloud.com/pastebin/L2NjBzdS/16:07
inc0ktibi: yeah you can override16:07
inc0kfox1111: right16:07
kfox1111inc0: as few as I can get away with. :)16:07
kfox1111inc0: generally configmaps are better.16:08
kfox1111rwellum: ok, thats interesting.16:08
inc0what I'm saying kolla_start needs it16:08
inc0ktibi: https://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration.html#openstack-service-configuration-in-kolla16:08
kfox1111inc0: we pass the stuff kolla_start needs.16:08
ktibiinc0 thx16:08
inc0rwellum: well..we have error16:09
inc0but I've never seen it16:09
kfox1111https://github.com/openstack/kolla-kubernetes/blob/master/helm/kolla-common/templates/_common_lib.yaml#L2616:09
*** salv-orlando has quit IRC16:09
kfox1111inc0: I think it may have to do with sbezverk enabling tcp on libvirt?16:09
inc0maybe16:10
kfox1111looks like it may try and use tls by default.16:10
*** magicboiz has joined #openstack-kolla16:10
kfox1111I preffer using the socket anyway.16:10
kfox1111no reason to open up libvirt to the world.16:10
rwellumI see a conversation with other people seeing the same here: http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-kolla.2017-09-18.log.html16:10
jamesbensonstill trying to figure out where the breakdown is in our network....  everything looks good. :-(16:11
kfox1111so kolla-ansible's seeing it too probably.16:11
inc0maybe something new in 2.4?16:11
rwelluminc0: kfox1111 but our gates work for pike?16:12
kfox1111could be.16:12
kfox1111rwellum: do we have pike gates?16:12
rwellumkfox1111: we have master - I think Serguei set that up - but uses label 5.0.0 - which grabs the master tarball16:13
kfox1111we don't have voting pike I guess.16:13
kfox1111only experimental.16:13
inc0try to rollback tcp patch?16:13
kfox1111I think there is a way to disable tcp.... lets see...16:14
inc0it's config option16:14
kfox1111enable_libvirt_tcp = false16:14
inc0rwellum: try to edit listen_tcp in libvirtd.conf16:16
inc0and restart container16:16
inc0listen_tcp = 016:16
kfox1111ugg. its hardcoded in ansible roles. :(16:16
*** jascott1 has joined #openstack-kolla16:17
jmccarthyAnyone run into an issue where with cirros, console logging works in horizon, but with something else, it doesn't ?16:17
rwelluminc0: just to make sure - on the libvirt daemonset container, edit the libvirt.conf there?16:18
inc0rwellum: edit configmap16:18
kfox1111rwellum: inside the container, vi /etc/libvirt/libvirt.conf16:18
kfox1111then kolla_start quick.16:18
kfox1111shoudl be ok.16:18
inc0I guess that's fastest16:18
rwellumkfox1111: done but same message/error in the log16:19
kfox1111hmm....16:20
inc0libvirtd --version16:20
kfox1111maybe get rid of auth_tcp and ca_file too?16:20
rwellumok well there's nothing in this .conf file16:20
rwellumhttps://www.irccloud.com/pastebin/wXmtOEVn/16:21
kfox1111does it start?16:21
inc0ahh so probably it's wrong conf file16:21
kfox1111add:16:21
kfox1111log_level = 316:21
kfox1111log_outputs = "3:file:/var/log/kolla/libvirt/libvirtd.log"16:21
kfox1111oh..16:21
kfox1111hmm.16:21
rwellumoh it does not - missed that...16:21
rwellumhttps://www.irccloud.com/pastebin/0khyyerf/16:21
kfox1111OH.16:22
kfox1111libvirtd.conf, not libvirt.conf16:22
rwellumurgh!16:22
rwellumstill getting the same start error though - I did remove the line from libvirt.conf16:23
inc0set it explicitly to 016:23
inc0I don't know what's default16:23
inc0and try to remove all other notions of listening on network16:24
kfox1111+1. remove ca_file and auth_tcp, and listen_addr.16:24
kfox1111so shold only be 3 lines. listen_tcp = 0, log_level = 3,  log_outputs = "3:file:/var/log/kolla/libvirt/libvirtd.log"16:25
*** pcaruana has quit IRC16:25
rwellumhttps://www.irccloud.com/pastebin/Y8nDkoES/16:26
inc0so libvirt runs right?16:27
rwellumNo I don't think so - see that ERROR?16:27
inc0that shouldn't matter16:27
inc0check /var/log/kolla...16:27
rwellumhttps://www.irccloud.com/pastebin/dUFgjIQi/16:28
inc0interesting16:28
inc0seems like your host has broken certificates16:29
kfox1111inc0: they come from the container though.16:29
inc0hmm, you're using ubuntu container on top of centos?16:29
kfox1111inc0: we don't pass host certs to the container, so that shouldn't matter?16:29
inc0I think ca certs are taken from host16:29
inc0by docker16:29
kfox1111gotta go in 5.16:29
kfox1111uhhh... what?16:30
rwellumMy host VM is ubuntu, but I see the same issues with centos host vm16:30
kfox1111I don't think I've seen docker do that. unless its in a much newer version.16:30
inc0rwellum: do you mix container distro and host distro?16:30
inc0because one line shows something about centos, second shows ubuntu16:31
rwellumYeah I think so in this case16:31
inc0I've seen libvirt misbehave when that happens16:31
inc0(in ansible tho)16:31
rwellumbase_distro: "centos"16:32
rwellumSo yeah this is centos on ubuntu - but tbh never seen any issues.16:32
rwellumbefore16:32
*** shardy has quit IRC16:33
rwellumI can respin centos-centos - and make that config change16:33
kfox1111generally its ok. but I too have seen some issues with libvirt in a mixed env.16:33
inc0yeah it was just libvirt that had issues as I remember, but there were those16:34
rwellumWhat is the change I need? Because kfox1111 you said hard coded somewhere?16:34
*** jascott1 has quit IRC16:35
inc0https://github.com/openstack/kolla-kubernetes/blob/master/ansible/roles/nova/templates/libvirtd.conf.j2 <- try this file16:36
*** janki has quit IRC16:37
hrwinc0, rwellum: any idea when 6.0.0 tag will be done? iirc schedule mentioned 19th ;(16:37
rwellumOk and remove those three lines16:37
rwellumhrw: I saw your question earlier but not sure if it's wrapped up in inc0 infra changes16:37
inc0hrw: yeah, I guess zuulv3 chaos made us miss it;)16:38
hrwinc0: ;)16:38
inc0check with Jeffrey4l tomorrow yours morning16:38
inc0he does releases16:38
hrwinc0: will do. thanks16:38
inc0thanks for keeping an eye for it16:38
hrwjust built 5.0.1 today instead. and this one is not tagged too16:38
hrwatleast not at http://git.openstack.org/cgit/openstack/kolla/refs/tags16:39
inc0no, we didn't tag yet16:39
hrwand (while we are at pike) I have two backports in queue: https://review.openstack.org/#/c/514573/ (make cinder-api work in Debian) https://review.openstack.org/#/c/514614/ (add Linaro kolla:pike repo for Debian to have some extra backports required for aarch64 (x86-64 binaries also provided))16:41
*** lukepatrick has joined #openstack-kolla16:45
inc0sooo keepalived works SamYaple -.-16:52
inc0for me16:52
*** jascott1 has joined #openstack-kolla16:53
*** unicell has joined #openstack-kolla16:55
*** jascott1 has quit IRC16:57
*** jascott1 has joined #openstack-kolla16:59
*** sambetts is now known as sambetts|afk17:00
inc0masber: any progress on keepalived issue?17:00
*** jascott1 has quit IRC17:02
*** itlinux has joined #openstack-kolla17:02
*** jascott1 has joined #openstack-kolla17:03
*** jmccarthy has left #openstack-kolla17:07
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385517:07
*** lpetrut_ has joined #openstack-kolla17:11
kfox1111fasinating....17:12
kfox1111https://github.com/sapcc/kubernetes-operators/tree/master/openstack-operator17:12
*** lpetrut has quit IRC17:14
*** lpetrut_ has quit IRC17:21
*** dougsz1 has quit IRC17:22
*** magicboiz has quit IRC17:22
jascott1kfox1111 yeah saw that... early days i think17:25
jascott1this was news to me: go codegen required for operators in 1.8? https://blog.openshift.com/kubernetes-deep-dive-code-generation-customresources/17:25
*** salv-orlando has joined #openstack-kolla17:27
*** gkadam has quit IRC17:30
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385517:32
openstackgerritchandu proposed openstack/kolla-ansible stable/ocata: Removing groups scope from all to baremetal in baremetal pre-install task under bootstrap-server  https://review.openstack.org/51477417:32
*** lpetrut has joined #openstack-kolla17:35
*** jinke_ has joined #openstack-kolla17:35
*** dougsz has joined #openstack-kolla17:37
*** jinke_ has quit IRC17:39
*** jascott1 has quit IRC17:45
*** jascott1 has joined #openstack-kolla17:46
*** lrensing has quit IRC17:48
*** jascott1 has quit IRC17:49
*** tvignaud has quit IRC17:53
*** jascott1 has joined #openstack-kolla17:54
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385517:58
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385518:00
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385518:09
*** danardelean has joined #openstack-kolla18:13
*** danardelean has quit IRC18:17
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385518:21
rwellumkfox1111: when a pod is in init state, and it's downloading - is there are more granular way of tracking progress?18:21
kfox1111like, you wondering how much left it has to download, etc?18:22
rwellumyes exactly18:22
kfox1111I don't know of a way to do that currently.18:22
rwellumThe registry can vary from 20m to 2 hours annoyingly18:23
kfox1111would like there to be a feature like that myself. :)18:23
rwellumAh :)18:23
kfox1111wow.18:23
kfox1111need to cache that locally somehow I guess.18:23
*** salv-orlando has quit IRC18:25
*** salv-orlando has joined #openstack-kolla18:25
rwellumI notice that the registry uses a 'curl' - wondering if an scp or better rsync would work better18:27
kfox1111which registry?18:28
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385518:29
*** salv-orlando has quit IRC18:30
inc0I wrote quick playbook that will setup local registry for you if you want18:32
rwellumSorry - Serguie's registry for deploying Pike/Master that he's using in the gate.18:34
*** lpetrut has quit IRC18:34
kfox1111oh. docker registry18:34
rwellumyup18:34
*** lpetrut has joined #openstack-kolla18:34
jascott1sbezverks registry chart works for me18:34
rwellumkfox1111: that's a work-around correct? Until we sort the publishing18:35
*** TxGirlGeek has quit IRC18:35
rwellumjascott1: yeah I'm just complaining that it's slow and unpredictable :)18:35
inc0https://github.com/inc0/kolla-registry18:35
jascott1ah18:35
inc0don't overuse it please tho18:35
kfox1111rwellum: probably should tweak that and add an option to use a pv.18:35
kfox1111rwellum: then you could set it to use a host path and point it at a previously downloaded tar.18:35
jamesbensonwhere is the qemu instance being hosted in kolla?  /etc/libvirt/qemu/instance....xml file?18:36
jascott1man i need a ceph survival guide18:36
kfox1111jascott1: whats up?18:36
rwellumkfox1111: tweak inc0 's playbook?18:36
inc0jascott1: /var/lib/docker/volumes/nova_compute ...18:36
kfox1111rwellum: sbezverk's chart18:37
rwellumOh right.18:37
kfox1111kube, not ansible. ;)18:37
jascott1inc0 ?18:37
inc0jamesbenson: * ;)18:37
rwellumjascott1: ceph took me 3 months to get working bare-metal.18:37
jamesbensonyep caught that, thanks inc018:37
jascott1so I have everything installed from the etherpad but something is missing18:37
jamesbensonrwellum, wow!  with ceph-deploy or something else?18:38
jascott1rwellum woah!18:38
kfox1111that seems like a long time. wall clock or real effort?18:38
*** danardelean has joined #openstack-kolla18:38
rwellumYeah.. Well a lot of networking issues.18:38
rwellumwall clock - I was doing other things.18:38
kfox1111jascott1: I ran into a weird issue where the instructions I wrote layed down the wrong version of ceph.18:39
rwellumBut tweaking it for performance took all the time.18:39
kfox1111jascott1: did you end up with a luminious or jewel version?18:39
jascott1ceph version 10.2.718:39
kfox1111yeah. you can tweak it for a long time.18:39
rwelluminc0: that pb looks cool - will try it out18:39
inc0tweak dockerfile to use ubuntu or centos18:39
kfox1111weird. 10.2.7 is jewel too.18:40
kfox1111ok.18:40
kfox1111jascott1: so, what seems to be the problem?18:40
jascott1so after everyting is installed in including your provisioner chart, I can create a PV and its seems ok but pvc stays pending18:41
inc0jascott1: describe da pvc!18:41
jascott1well before i create PV I created a ceph colyme18:41
jascott1volume18:41
jascott1thats not in the instructions18:42
kfox1111jascott1: ah. so probably a problem with the  provisioner not being fully working, not ceph itself.18:42
jascott1idk18:42
jamesbensonah yeah, I didn't really tweak it at all.. just got it running and go with it ;-) Do you have a github/post that shares the configs?18:42
kfox1111yeah. that part of the meeting was a little rough, and I don't think I got all of the instructions.18:42
kfox1111there could very well be a step or two missing still.18:42
jascott1but i should be creating a rbd volune?18:42
*** salv-orlando has joined #openstack-kolla18:42
*** danardelean has quit IRC18:42
kfox1111the provisioner shoudl create the rbd volume.18:42
kfox1111but we need to ensure the ceph user and rbd pool are created.18:43
kfox1111got a link to the etherpad handy?18:43
jascott1https://etherpad.openstack.org/p/kolla-k8s-braindump18:43
jascott1oh ok that helps, didnt know the provisioner should create the rbd vol18:43
inc0what it needs tho is rbd pool18:44
inc0that you need to create yourself18:44
inc0also call "ceph -s" please18:44
jamesbensonmissing this too in the etherpad: --release luminous18:45
jamesbensonotherwise the default is 10.2.718:45
jamesbensonceph-deploy install --release luminous $ALL_NODES18:45
rwellumkfox1111: inc0 unfortunately libvirt still crashing even with TCP listen turned off. Centos host - centos containers.18:45
jascott1i have a kollavolumes pool18:45
inc0rwellum: crap...18:45
inc0ok, back to debugging18:45
inc0same error as before?18:46
rwellumYes - I'll do the same debug as before and see what's going on. Have to pop home but will be back on later.18:46
jascott1ceph -s says HEALTH_OK18:46
inc0how about describe of pvc?18:46
inc0also ceph_provisioner pod is healthy?18:47
jascott1yup18:47
inc0describe da pvc!18:47
inc0kubectl describe pvc ...18:47
jascott1to be clear, dont make a rbd vol but do make a k8s pv first right?18:48
jascott1or no?18:48
inc0you don't make vol, you make pvc first and then pv18:48
kfox1111rwellum: yeah, I'm thinking there is something that has to be explicitly turned off.18:49
jascott1where is example for pv? I have one but could be wrong18:49
inc0jascott1: it's pod thing18:49
jascott1what?18:50
kfox1111jascott1: yeah, looking at the docs, we still need the bit to create the user,18:50
kfox1111to create the pool,18:50
inc0jascott1: I used these http://paste.openstack.org/show/624512/18:51
jascott1i have the kolla user and the kollavolumes pool18:51
kfox1111and to pass the ceph admin key into the provisioner.18:51
kfox1111inc0: do you still have notes for those actions, or should I try to dig up the right commands again?18:51
inc0kfox1111: I don't think so18:53
inc0I have machine tho18:53
jascott1inc0 describing the pvc just says its waitin on external provisioner to create volume18:54
inc0then kubectl logs ceph_provision18:55
jascott1kfox1111 where is $key in following? helm install stable/ceph-provisioner --name ceph-provisioner --namespace kube-system --set "adminSecret.key=$key,monitors[0]=172.17.0.1"  --values ...18:55
inc0ahh18:55
inc0right, that's probably what you're missing18:56
inc0when you created kolla used you should create key file too18:56
jascott1yeah just realized that I templated the values heredoc but neglected the --set art18:56
jascott1part18:56
inc0make sure to point monitors to actual monitor18:57
kfox1111jascott1: needs to be set to the admin key18:57
inc0s18:57
jascott1admin in  key created in ~/mycluster?18:57
kfox1111yeah. but just the base64 part. so something like:18:57
kfox1111key=$(grep key | awk '{print $3}')18:57
kfox1111grep key <filename>18:58
jascott1cool thanks.. got someone calling me brb18:58
kfox1111should be able to create the pool with something like:18:59
kfox1111ceph osd pool create kubevolumes 16 1619:00
kfox1111ceph auth get-or-create client.kube mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=kubevolumes'19:01
kfox1111or something like that for the user.19:01
kfox1111then need to put that secret into a kube secret.19:01
jascott1yeah i think I got all that from the doc but has missed the --set line and defining the $key19:08
jascott1going to grab some grub and then will try it19:08
kfox1111k.19:10
sdaketoys: https://twitter.com/steven_dake/status/92289697838359347219:13
sdakejascott1 ^^19:13
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385519:17
*** lrensing has joined #openstack-kolla19:20
*** jascott1 has quit IRC19:20
*** jascott1 has joined #openstack-kolla19:21
*** TxGirlGeek has joined #openstack-kolla19:23
*** jascott1 has quit IRC19:26
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385519:27
*** TxGirlGeek has quit IRC19:30
*** TxGirlGeek has joined #openstack-kolla19:30
*** jascott1 has joined #openstack-kolla19:33
*** TxGirlGeek has quit IRC19:33
*** TxGirlGeek has joined #openstack-kolla19:34
*** gfidente has quit IRC19:35
*** jascott1 has quit IRC19:35
*** TxGirlGeek has quit IRC19:38
*** hrw has quit IRC19:39
openstackgerritChristian Berendt proposed openstack/kolla-ansible master: Add prometheus output to telegraf  https://review.openstack.org/51480619:39
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385519:40
*** egonzalez has joined #openstack-kolla19:40
*** hrw has joined #openstack-kolla19:40
*** jascott1 has joined #openstack-kolla19:45
*** jascott1 has quit IRC19:48
*** TxGirlGeek has joined #openstack-kolla19:51
openstackgerritchandu proposed openstack/kolla-ansible stable/ocata: Removing groups scope from all to baremetal in baremetal pre-install task under bootstrap-server  https://review.openstack.org/51477419:52
*** TxGirlGeek has quit IRC19:56
*** TxGirlGeek has joined #openstack-kolla19:57
*** jascott1 has joined #openstack-kolla19:58
*** jascott1 has quit IRC19:59
*** jascott1 has joined #openstack-kolla20:00
rwellumsdake: you try-ode them out?20:00
jascott1sdake neato! what are they for?20:00
*** rmart04 has joined #openstack-kolla20:03
*** rmart04 has quit IRC20:03
*** TxGirlGeek has quit IRC20:04
*** bmace has quit IRC20:06
*** bmace has joined #openstack-kolla20:06
rwelluminc0: kfox1111 same error centos on centos:20:14
rwellum2017-10-24 20:14:01.941+0000: 20735: error : virNetTLSContextCheckCertFile:120 : Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory20:14
*** ktibi has quit IRC20:16
jamesbensonsdake you've got some old servers there ;-)20:18
jamesbensonhttp://1.bp.blogspot.com/-wtVCNAAt3E0/Ve98ynS4kbI/AAAAAAAABhI/i5RaddQel-M/s1600/eniac_vacuum_tubes.png20:19
*** TxGirlGeek has joined #openstack-kolla20:20
*** itlinux has quit IRC20:22
*** TxGirlGeek has quit IRC20:29
*** itlinux has joined #openstack-kolla20:30
*** jascott1 has quit IRC20:31
*** jascott1 has joined #openstack-kolla20:31
*** TxGirlGeek has joined #openstack-kolla20:32
*** jascott1 has quit IRC20:32
*** jascott1 has joined #openstack-kolla20:33
*** jascott1 has quit IRC20:35
*** ansmith has quit IRC20:35
*** lukepatrick has quit IRC20:42
*** danardelean has joined #openstack-kolla20:43
*** paramite has quit IRC20:46
openstackgerritMerged openstack/kolla master: Create murano applications package for import  https://review.openstack.org/51363220:47
*** TxGirlGeek has quit IRC20:47
*** danardelean has quit IRC20:47
*** TxGirlGeek has joined #openstack-kolla20:47
inc0rwellum: do you have /etc/pki/CA in your container?:)20:50
*** egonzalez has quit IRC21:03
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385521:06
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385521:07
*** TxGirlGeek has quit IRC21:11
*** TxGirlGeek has joined #openstack-kolla21:12
*** jtriley has quit IRC21:12
*** TxGirlGeek has quit IRC21:16
*** jascott1 has joined #openstack-kolla21:17
*** itlinux has quit IRC21:18
rwelluminc0: I do but not the cacert.pem21:18
jascott1hmm my pvc is still pending...21:19
jascott1just to triple check, I dont have to create a vol with ceoh AND i dont have to create a pv?21:19
jascott1i get21:20
jascott1rbd-test   Pending                                      rbd            2m21:20
rwelluminc0: so possibly an issue with my host vm? It's an openstack vm - so maybe there's some setup missing.21:20
rwellum3 months jascott1 ...21:20
jascott1haha21:22
jascott1maybe I can shave some time off that... only looking for dev quality21:22
*** danardelean has joined #openstack-kolla21:23
jascott1rwellum ah so you are expert now :)21:23
jascott1how do I troubleshoot?21:23
*** tvignaud has joined #openstack-kolla21:24
*** TxGirlGeek has joined #openstack-kolla21:24
rwellumI went through the redhat ceph webpage many times..... But this is with added complexity - containers.21:25
*** danardelean has quit IRC21:26
*** itlinux has joined #openstack-kolla21:26
*** ansmith has joined #openstack-kolla21:26
inc0rwellum: can't say, it's strange tho21:27
inc0I normally deploy without any additional conf needed21:27
inc0on top of openstack vm21:27
rwellumAlso weird that same VM, same infra - no problem with libvirt and ocata21:27
inc0then it's not vm21:27
inc0how about you override just libvirt container21:28
inc0and use it with ocata?21:28
inc0to confirm it's pike-libvirt issue21:28
inc0also, ocata from tarballs?21:28
rwellumAh good call - to debug21:28
rwellumNo ocata is from dockerhub21:28
inc0right, so it will be older version of libvirt21:28
inc0try ocata from tarballs plz21:28
inc0dockerhub images weren't updated21:29
rwellumOk will do21:29
openstackgerritMichal Jastrzebski (inc0) proposed openstack/kolla master: WIP Dockerhub publisher job  https://review.openstack.org/51385521:30
rwellumSo to use ocata libvirt I'll need to think this out, because I'm using the docker registry21:30
inc0rwellum: just setup second registry on different port21:31
inc0or deploy all ocata but from tarballs21:31
inc0seems like new libvirt have some strange issues with certs21:31
jamesbensoninc0: I deployed but for some reason some didn't ssh and failed on the node, can I just hit deploy again to finish up that node or two that it failed on?21:32
jamesbensonor does it need to destroy/deploy again?21:32
inc0depends, what task failed?21:33
inc0which*21:33
jamesbensonglance : Ensuring config directories exist21:33
inc0just hit deploy21:33
inc0if it's after rabbitmq/mariadb you should be fine21:33
jamesbensonand the setup failed for one node: fatal: [r2-710-07]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: mux_client_hello_exchange: write packet: Broken pipe\r\n", "unreachable": true}21:34
inc0if it's mariadb, you still should be fine with rerunning deploy, but since it has all the clustering gimmics, I'd suggest clean redeploy21:34
jamesbensonyeah, the "dirty" deploy is still going.... hasn't errored out21:35
jascott1hmm found this21:36
jascott1librados: client.admin authentication error (110) Connection timed out rbd: couldn't connect to the cluster!21:37
*** TxGirlGeek has quit IRC21:37
*** goldyfruit has quit IRC21:39
inc0jascott1: check security groups in cluster21:42
jascott1will do21:42
*** jamesbenson has quit IRC21:43
*** TxGirlGeek has joined #openstack-kolla21:45
rwelluminc0: you mentioned remove ca_file="" - perhaps I should keep this line?21:46
rwellumactually maybe you didn't mentioned removing that line...21:47
inc0or...21:48
inc0let me check somethign21:48
inc0hmf21:49
rwellumkk21:49
inc0maybe play with this conf option?21:49
rwellumjascott1 https://www.irccloud.com/pastebin/lREw6TKt/21:49
*** TxGirlGeek has quit IRC21:49
*** TxGirlGeek has joined #openstack-kolla21:49
jascott1rwellum thanks. all that looks good21:51
jascott1(afaik)21:51
rwellumThat's cool then21:51
jascott1seems like its the linkage from the provisioner to ceph21:52
*** TxGirlGeek has quit IRC21:53
*** TxGirlGeek has joined #openstack-kolla21:54
rwellumYeah - I know the OSD's have their own private n/w to communicate - wonder if this is a problem. Have you/can you try running the rdb command manually?21:55
rwelluminc0: ? what conf option?21:55
jascott1rwellum I can manually create a volume with `rbd create myvol --size 1G --pool kollavolumes`21:58
kfox1111back.21:58
rwellumHuh that's odd jascott1  - seems to be very healthy21:59
jascott1kfox1111 when you get a sec can you sanity check this?21:59
jascott1http://paste.openstack.org/show/624523/21:59
kfox1111jascott1: looks prety sane. is your mon's ip really 172.17.0.1?22:00
jascott1thats docker022:01
jascott1was in the etherpad22:01
kfox1111hmm... I think in the etherpad I used $(hostname)22:01
kfox1111hmm... I think in the etherpad I used $(hostname -s)22:01
kfox1111I wonder what the mon is listenign on though.22:02
*** danardelean has joined #openstack-kolla22:02
jascott1let me see22:02
*** danardelean has quit IRC22:03
jascott1this etherpad had that 172.x https://etherpad.openstack.org/p/kolla-k8s-braindump22:03
inc0jascott1: use regular mon22:03
*** danardelean has joined #openstack-kolla22:03
jascott1probably the 192.x from ens3?22:03
kfox1111jascott1: yeah, but not sure that exact arg ever got tested.22:03
jascott1gotcha22:03
kfox1111netstat -pnl | grep mon22:03
*** TxGirlGeek has quit IRC22:04
jascott1ya its the 192.x let me fix that22:04
jascott1thanks22:04
kfox1111np.22:04
*** lpetrut has quit IRC22:05
rwellumGood catch22:05
*** TxGirlGeek has joined #openstack-kolla22:05
*** kolla-slack has quit IRC22:06
*** kolla-slack has joined #openstack-kolla22:06
*** danardelean has quit IRC22:07
openstackgerritChristian Berendt proposed openstack/kolla-ansible master: Telegraf should only output to influxdb when influxdb is enabled  https://review.openstack.org/51485122:09
jascott1inc0 i wrapped the terraform and ansible in a little bash script so I have a one liner to destroy and rebuild everything from scratch. its uh very nice-uh!22:09
SamYaplejascott1: now alias it to `JUSTWORKALREADY`22:09
jascott1haha22:10
jascott1aliases for the win \o/22:10
*** goldyfruit has joined #openstack-kolla22:12
rwellumjascott1: shouldn't you have an ansible playbook to wrap the ansinble playbook. :)22:12
SamYaplerwellum: soon youll be asking for a kolla api22:12
rwellumOr like some code I was looking at ruby->calling bash->calling python22:12
inc0rwellum: that's what we're doing in gates22:12
jascott1yass!22:12
rwellumSamYaple: +1 :)22:13
rwelluminc0:  - you said above 'try this config:22:13
jascott1SamYapple ive got a `export all="crd,endpoints,all --all-namespaces"` so I can do `k get $all` and acutally see everyting22:15
jascott1instead of the lies k8s tells by default22:15
kfox1111hehe.22:15
jascott1well its not everyting, cause some like sc,pv wont work with namspace22:15
jascott1this is interesting but i think k8s bash completions are equally useful https://medium.com/@Mikulas/kubectl-repl-e6f86cac862622:17
SamYaplejascott1: that just means youll be screwed when youre not on that box22:18
jascott1haha exactly22:18
openstackgerritChristian Berendt proposed openstack/kolla-ansible master: Add prometheus output to telegraf  https://review.openstack.org/51480622:19
inc0ok, I'm done with dockerhub publisher it seems22:19
jascott1woot sir22:19
jascott1maximum woot.22:20
inc0well now I need to merge it and hope for the best22:20
inc0because I can't *really* test it22:20
SamYapleinc0: it wont work22:20
inc0y tho22:20
SamYaplenot all comments in first PS were addressed22:21
rwellumEven worse when you have it aliased in your bashrc and then you forget and spend weeks wondering why grep doesn't work... 🙋‍♂️22:21
inc0yeah I still need to fix .zuul22:21
inc0but what I'm saying is logic is as close to ready as I can get it22:22
jascott1ah the sweet taste of success22:22
jascott1it worked22:22
inc0and I can't do it in post because rather than build->push I'll just use --push22:22
rwellumjascott1: +1!22:22
jascott1thanks everyone22:22
inc0jascott1: multinode worky?22:22
jascott1haha no22:22
jascott1BUT22:22
inc0tomorrow22:22
jascott1should be a piece of cake22:23
jascott1what? the night is young and i am not22:23
rwellumjascott1: ceph happy with just one node?22:23
jascott1ya22:23
inc0yeah, we just need to figure out this cluster networking issue22:23
SamYapleinc0: im a bit worried about https://review.openstack.org/#/c/513855/18/tests/templates/kolla-build.conf.j222:23
SamYapleinc0: that was implemented for a reason22:23
SamYaplebecause the gates were pulling in upstream images22:23
inc0lokolla thing?22:23
SamYaplewhy not just do a post job and retag it?22:23
rwellumPretty sure that is not supposed to work22:23
SamYapleyes22:23
SamYaplerwellum: its not but here we are22:23
jascott1dont suppose we have ceph multinode example in gates or somewher?22:23
*** goldyfruit has quit IRC22:23
inc0well, plan is that gates *will* pull upstream images22:24
inc0with cache proxy22:24
rwellumWell is ceph single node works why the heck am I messing with LVM...22:24
SamYapleright but the way you have the patch written that only does it for "publisher"22:24
rwellum(if)22:24
SamYaplewhen it explicitly needs to *not* do that for publisher22:25
inc0so for first stab, I'm ok with publisher pushing images to dockerhub and ansible gates pulling images from dockerhub+cache22:25
inc0after that we'll deal with better testing pipeline22:25
inc0but as it stands today, we need some sort of publishing images and tarballs won't work22:26
SamYaplei dont think we are tlaking about the same thing exactly22:26
inc0maybe, what are you talking about?22:26
*** salv-orlando has quit IRC22:26
*** salv-orlando has joined #openstack-kolla22:27
SamYaplewhen building your images, you should *keep* the lokolla name and retag right before you push the images. otherwise you run the risk of not building and only pulling images or pulling then pushing the same image (the reason "lokolla" was added in the first place to prevent)22:27
SamYapleright now you are doing the opposite of that22:27
inc0ahh22:27
inc0I see what you're saying22:27
kfox1111more years go by.... diagraming stuff still sucks....22:28
jascott1no doubt22:28
jascott1i like plantUML22:28
jascott1definitely rules for seq diagrams22:29
jascott1kinda hit or miss for other ones22:29
inc0let me think about it for a moment, I don't want to push in post job because it's going to be nightmarish long22:29
kfox1111yeah. diagraming uml these days isn't too bad.22:29
inc0since bash is not multithreaded22:29
kfox1111diagraming other things, not as much. :(22:29
jascott1they added deployment diagrams if thats what you are doing22:30
jascott1its ok22:30
kfox1111nodes, racks, switches, networks, services, etc.22:30
SamYapleinc0: where do you think you are going to push from?22:31
SamYapleinc0: you cant push from the gate22:31
jascott1god their website is hideous http://plantuml.com/deployment-diagram22:31
inc0SamYaple: why?22:31
inc0well22:31
SamYaplehas to be post (or other protected) pipelines22:31
inc0periodic22:31
*** salv-orlando has quit IRC22:31
SamYaplehmmm let me find the list22:31
inc0it's going to be periodic job22:31
inc0periodic is fine I think22:31
rwellumjascott1: looks like my kids drawings22:31
SamYaplebtw, you keep misunderstanding what i mean by post22:32
*** TxGirlGeek has quit IRC22:32
SamYaplei mean post-run not post job22:32
inc0I know22:32
inc0post playbook22:32
SamYapleyea, so you can move everythign there22:32
SamYaplethat runs on the job22:32
SamYaplenot just the post pipeline22:32
inc0tagging and pushing, yeah, but if we figure out how to use --push safely it's going to be just fine22:33
jascott1rwellum ya you can mess with the style. a lot of IDEs support it so it makes it easy to work with. I just like I can keep it in git22:33
rwellumYeah that's cool. I have played with https://www.draw.io/ - and that's pretty good too.22:33
SamYapleah yea inc0 even gate is good22:34
jascott1saw a project the other day that was outputing some network tracing to automatically create plantuml seq diagrams22:34
jascott1pretty neat22:34
rwellumI've used it with customers before for an on-the-fly demo and then used gdrive - they can interact and get a copy etc22:34
*** TxGirlGeek has joined #openstack-kolla22:34
SamYapleinc0: im of the opinion that you should change kolla-build a bit to allow a second push-only call22:35
SamYapletrying to get them to work toeghther in the gate is going to be a problem22:35
inc0that might be option22:35
SamYaplei forget, do you only push *after* all images build successfully?22:36
SamYapleor do you push as you go?22:36
inc0I think you build as they go22:36
inc0so it's quite optimal22:36
inc0also multithreaded22:36
SamYapleyea thats bad. what if an image fails to build, then youll have pushed a bunch of images out of sync22:36
*** mummer has quit IRC22:37
inc0I'll need to think about it22:38
kfox1111draw.io is quite interesting....22:38
inc0if we build->push we probably gonna hit timeout22:38
inc0unless it's multithreaded22:38
kfox1111the source is even open it seems.22:38
SamYapleif build AND push and it fails you break your images22:38
inc0maybe it's worth to just write some python to retag and push multithreaded22:39
*** TxGirlGeek has quit IRC22:39
inc0yeha I hear you22:39
rwellumyes kfox1111  - you can suggest changes etc22:40
rwellumAnd you can export in html etc22:40
rwellumXML22:41
*** TxGirlGeek has joined #openstack-kolla22:42
*** TxGirlGeek has quit IRC22:49
*** TxGirlGeek has joined #openstack-kolla22:51
*** itlinux has quit IRC22:59
sdakejascott1 for my tube amp :)22:59
*** TxGirlGe_ has joined #openstack-kolla23:00
*** TxGirlGeek has quit IRC23:01
*** TxGirlGe_ has quit IRC23:11
*** hongbin has quit IRC23:14
*** thomasem has quit IRC23:21
openstackgerritJeffrey Zhang proposed openstack/kolla-ansible master: Optimize zuul v3 jobs  https://review.openstack.org/51361223:22
*** salv-orlando has joined #openstack-kolla23:28
*** lrensing has quit IRC23:28
*** thomasem has joined #openstack-kolla23:28
*** TxGirlGeek has joined #openstack-kolla23:31
*** TxGirlGeek has quit IRC23:31
*** salv-orlando has quit IRC23:32
jascott1sdake what kind?23:32
sdake@jascott1 one of these http://www.stevehuffphoto.com/line-magnetic-219ia-integrated-tube-amp-review-300b-and-845-tube-magic/23:33
jascott1sweet23:35
* jascott1 updates festivus wishlist23:36
jascott1kfox1111 can you tell me purpose of this in etherpad: swapoff /dev/dm-123:39
jascott1which component is that for?23:40
kfox1111jascott1: k8s 1.8+ doesn't like swap on. it fails a check in kubelet startup.23:40
jascott1ah thanks23:40
kfox1111could probably change that to 'swapoff -a' I think.23:40
jascott1thats before kubeadm init i think>23:41
jascott1?23:41
kfox1111yeah.23:41
kfox1111should be one of the first things I think.23:41
jascott1k i will stick it with the firewall off and selinux etc23:41
kfox1111yeah. that works.23:41
jascott1all nodes?23:42
kfox1111yeah.23:44
jascott1kfox1111 just for install or persist after reboot in fstab?23:47
kfox1111probably need to remove it out of fstab too.23:47
jascott1got it. thanks! youve save me much googlin'23:47
kfox1111np. glad to help. :)23:48
kfox1111rarrg.23:51
kfox1111I really like graphviz, right up until I dont. :(23:52
openstackgerritOpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements  https://review.openstack.org/50944323:52
kfox1111Its super helpful most of the time. but then when you get to a certain point in the graph, it just falls all apart.23:52
jascott1graphviz: when it just has to be ugly23:52
kfox1111Its super easy to do a lot of things with it though. far too few graphing tools just let you put in relations and have it auto build your layout.23:53
kfox1111"here's some raw data. graph it."23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!