| *** manheim has quit IRC | 00:04 | |
| *** yangyapeng has joined #openstack-kolla | 00:19 | |
| *** caoyuan has joined #openstack-kolla | 00:20 | |
| *** yangyapeng has quit IRC | 00:30 | |
| *** yangyapeng has joined #openstack-kolla | 00:30 | |
| *** caoyuan has quit IRC | 00:30 | |
| *** yangyapeng has quit IRC | 00:34 | |
| *** tonanhngo has joined #openstack-kolla | 00:52 | |
| *** tonanhngo has quit IRC | 00:56 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-kubernetes master: Updated from global requirements https://review.openstack.org/451018 | 01:04 |
|---|---|---|
| *** g3ek has quit IRC | 01:08 | |
| *** yangyapeng has joined #openstack-kolla | 01:15 | |
| *** g3ek has joined #openstack-kolla | 01:17 | |
| *** papacz has quit IRC | 01:51 | |
| *** shashank_t_ has joined #openstack-kolla | 01:54 | |
| *** ccesario has quit IRC | 02:03 | |
| *** ccesario has joined #openstack-kolla | 02:04 | |
| *** tonanhngo has joined #openstack-kolla | 02:11 | |
| *** tonanhngo has quit IRC | 02:13 | |
| *** shashank_t_ has quit IRC | 02:15 | |
| *** shashank_t_ has joined #openstack-kolla | 02:16 | |
| sdake | sbezverk nice job on kubernetes 1.6 dude! | 02:17 |
| klindgren | mnaser, thanks for the response here is what I have mapped in: | 02:19 |
| klindgren | volumes: | 02:19 |
| klindgren | - /etc/localtime:/etc/localtime:ro | 02:19 |
| klindgren | - /etc/kolla/nova-compute:/var/lib/kolla/config_files:ro | 02:19 |
| klindgren | - /var/log/nova:/var/log/nova | 02:19 |
| klindgren | - /var/lib/nova:/var/lib/nova | 02:19 |
| klindgren | - /tmp/nova-compute:/tmp/nova-compute | 02:19 |
| klindgren | - /run:/run:shared | 02:19 |
| klindgren | - /dev:/dev | 02:19 |
| klindgren | - /lib/modules:/lib/modules | 02:19 |
| klindgren | - /var/lib/libvirt:/var/lib/libvirt | 02:19 |
| klindgren | privileged: true | 02:19 |
| *** tonanhngo has joined #openstack-kolla | 02:21 | |
| *** tonanhngo has quit IRC | 02:32 | |
| *** caoyuan has joined #openstack-kolla | 02:48 | |
| *** tonanhngo has joined #openstack-kolla | 02:59 | |
| *** shashank_t_ has quit IRC | 03:02 | |
| *** shashank_t_ has joined #openstack-kolla | 03:02 | |
| *** hrw has quit IRC | 03:02 | |
| *** tonanhngo has quit IRC | 03:03 | |
| *** shashank_t_ has quit IRC | 03:07 | |
| *** tonanhngo has joined #openstack-kolla | 03:09 | |
| *** tonanhngo has quit IRC | 03:14 | |
| *** adrian_otto has joined #openstack-kolla | 03:23 | |
| *** adrian_otto has quit IRC | 03:25 | |
| *** lamt has quit IRC | 03:28 | |
| openstackgerrit | caoyuan proposed openstack/kolla-ansible master: Add neutron sanity check to use built in shade https://review.openstack.org/452506 | 03:31 |
| *** madgoat has joined #openstack-kolla | 03:38 | |
| *** madgoat has left #openstack-kolla | 03:38 | |
| *** caoyuan has quit IRC | 03:39 | |
| *** tonanhngo has joined #openstack-kolla | 03:40 | |
| *** tonanhngo has quit IRC | 03:57 | |
| *** caoyuan has joined #openstack-kolla | 04:15 | |
| *** unicell has joined #openstack-kolla | 04:20 | |
| *** unicell is now known as Guest73800 | 04:20 | |
| *** igordcard has quit IRC | 04:24 | |
| *** igordcard has joined #openstack-kolla | 04:26 | |
| *** haplo37 has quit IRC | 04:29 | |
| *** leseb has quit IRC | 04:30 | |
| *** leseb has joined #openstack-kolla | 04:31 | |
| *** dciabrin has quit IRC | 04:32 | |
| *** dciabrin has joined #openstack-kolla | 04:32 | |
| *** haplo37 has joined #openstack-kolla | 04:32 | |
| *** dasTor has quit IRC | 04:33 | |
| *** dasTor has joined #openstack-kolla | 04:33 | |
| *** sgordon has quit IRC | 04:35 | |
| *** sgordon has joined #openstack-kolla | 04:35 | |
| *** bmace has quit IRC | 04:44 | |
| *** bmace has joined #openstack-kolla | 04:45 | |
| *** tonanhngo has joined #openstack-kolla | 04:54 | |
| *** tonanhngo has quit IRC | 05:03 | |
| eanylin | sdake: Still there? | 05:07 |
| *** tonanhngo has joined #openstack-kolla | 05:08 | |
| *** unicell has joined #openstack-kolla | 05:12 | |
| *** tonanhngo has quit IRC | 05:44 | |
| *** haplo37_ has quit IRC | 05:46 | |
| *** Guest73800 has quit IRC | 05:47 | |
| *** haplo37_ has joined #openstack-kolla | 05:49 | |
| *** unicell has quit IRC | 05:49 | |
| *** jascott1_ has joined #openstack-kolla | 05:56 | |
| *** jascott1_ has quit IRC | 06:02 | |
| *** unicell has joined #openstack-kolla | 06:02 | |
| *** unicell1 has joined #openstack-kolla | 06:06 | |
| *** unicell1 has quit IRC | 06:10 | |
| *** unicell has quit IRC | 06:14 | |
| *** unicell has joined #openstack-kolla | 06:20 | |
| *** tonanhngo has joined #openstack-kolla | 06:25 | |
| *** caoyuan has quit IRC | 06:34 | |
| openstackgerrit | Bertrand Lallau proposed openstack/kolla-ansible master: Fix Fluentd warn on dnsmasq.log file parsing https://review.openstack.org/452498 | 06:59 |
| *** tonanhngo has quit IRC | 07:01 | |
| *** iceyao has joined #openstack-kolla | 07:01 | |
| *** iceyao has quit IRC | 07:18 | |
| *** caoyuan has joined #openstack-kolla | 07:34 | |
| *** caoyuan has quit IRC | 07:39 | |
| *** jascott1_ has joined #openstack-kolla | 08:07 | |
| *** iceyao has joined #openstack-kolla | 08:09 | |
| *** manheim has joined #openstack-kolla | 08:30 | |
| *** manheim has quit IRC | 08:34 | |
| *** caoyuan has joined #openstack-kolla | 08:36 | |
| *** caoyuan has quit IRC | 08:40 | |
| *** zhubingbing has joined #openstack-kolla | 08:56 | |
| *** caoyuan has joined #openstack-kolla | 08:58 | |
| *** manheim has joined #openstack-kolla | 09:15 | |
| *** shashank_t_ has joined #openstack-kolla | 09:16 | |
| *** shashank_t_ has quit IRC | 09:21 | |
| *** manheim has quit IRC | 09:31 | |
| openstackgerrit | caoyuan proposed openstack/kolla master: Update min Ansible version to 2.2.0 https://review.openstack.org/452529 | 09:33 |
| *** jascott1_ has quit IRC | 09:34 | |
| *** jascott1_ has joined #openstack-kolla | 09:35 | |
| *** jascott1_ has quit IRC | 09:35 | |
| *** iceyao has quit IRC | 09:36 | |
| *** jascott1_ has joined #openstack-kolla | 09:36 | |
| *** iceyao has joined #openstack-kolla | 09:47 | |
| *** iceyao has quit IRC | 09:50 | |
| *** n0isyn0i1e is now known as n0isyn0ise | 10:23 | |
| *** oanson has quit IRC | 10:25 | |
| *** lamt has joined #openstack-kolla | 10:31 | |
| *** pbourke has quit IRC | 10:34 | |
| *** pbourke has joined #openstack-kolla | 10:35 | |
| *** hachi has quit IRC | 10:40 | |
| *** oanson has joined #openstack-kolla | 10:44 | |
| *** hachi has joined #openstack-kolla | 10:55 | |
| *** oanson has quit IRC | 10:59 | |
| *** oanson has joined #openstack-kolla | 10:59 | |
| *** oanson has quit IRC | 11:23 | |
| *** oanson has joined #openstack-kolla | 11:27 | |
| *** oanson has quit IRC | 11:27 | |
| *** oanson has joined #openstack-kolla | 11:29 | |
| *** oanson has quit IRC | 11:37 | |
| *** pbandark has joined #openstack-kolla | 11:50 | |
| *** pbandark has quit IRC | 11:50 | |
| *** yangyapeng has quit IRC | 11:52 | |
| *** yangyapeng has joined #openstack-kolla | 12:01 | |
| *** oanson has joined #openstack-kolla | 12:22 | |
| *** oanson has quit IRC | 12:34 | |
| *** oanson has joined #openstack-kolla | 12:34 | |
| *** n0isyn0ise has quit IRC | 12:38 | |
| *** iceyao has joined #openstack-kolla | 12:45 | |
| *** oanson has quit IRC | 12:46 | |
| *** oanson has joined #openstack-kolla | 12:47 | |
| *** manheim has joined #openstack-kolla | 12:47 | |
| *** iceyao has quit IRC | 12:49 | |
| *** caoyuan has quit IRC | 13:01 | |
| *** iceyao has joined #openstack-kolla | 13:04 | |
| *** manheim has quit IRC | 13:05 | |
| *** yangyapeng has quit IRC | 13:05 | |
| sdake | eanylin morning | 13:06 |
| nea1 | does the neutron_external_interface have to be available on all servers or only on the network nodes? | 13:16 |
| *** erlon has joined #openstack-kolla | 13:35 | |
| eanylin | sdake: Morning | 13:38 |
| eanylin | sdake: FYI, I was able to reach the same stage as before by applying all the workarounds that sbezverk applied with the gate for 1.6.0 | 13:42 |
| sdake | eanylin thats good news | 13:44 |
| sdake | eanylin i guess we should just document how to work with 1.6 then - i'm not really sure | 13:45 |
| eanylin | sdake: Yeah, maybe we should | 13:45 |
| eanylin | sdake: Since the gate is based on 1.6 now | 13:45 |
| eanylin | sdake: I went further actually, I was able to ping 8.8.8.8 from my VM now | 13:45 |
| eanylin | sdake: And I can ping across 2 VMs | 13:46 |
| eanylin | sdake: And I got NoVNC working | 13:46 |
| *** specialguy has joined #openstack-kolla | 13:47 | |
| eanylin | sdake: However, I still cannot ssh to the Cirros VM from my host machine but the Cirros VM can reach my host machine, am guessing firewall... | 13:48 |
| specialguy | Hey :) Where can I see 3.0.3 release note and upgrading to 4.0.0 requirements | 13:49 |
| specialguy | Danke | 13:49 |
| *** specialguy has quit IRC | 14:00 | |
| nea1 | is it possible to have multiple neutron_external_interfaces? | 14:05 |
| eanylin | nea1: Have not tested multiple interfaces before, what would be the use-case for this though? | 14:07 |
| *** yangyapeng has joined #openstack-kolla | 14:08 | |
| *** yangyape_ has joined #openstack-kolla | 14:09 | |
| *** caoyuan has joined #openstack-kolla | 14:10 | |
| *** yangyapeng has quit IRC | 14:13 | |
| *** aolwas has quit IRC | 14:19 | |
| sdake | eanylin wow thats good! | 14:23 |
| sdake | eanylin sorry jerking aroudn with my blueray 4k UHD player | 14:23 |
| sdake | boy this ultraviolet thing is a racket | 14:24 |
| *** caoyuan_ has joined #openstack-kolla | 14:28 | |
| *** caoyuan has quit IRC | 14:31 | |
| *** jtriley has joined #openstack-kolla | 14:33 | |
| nea1 | I'm still thinking about how to deploy my nodes | 14:34 |
| nea1 | eanylin: I think my actual idea is not really realizable, the point is I have two different external networks which are connected to different nodes | 14:36 |
| eanylin | sdake: Yeah, I think went further this time round and I think we missed something for LVM Cinder | 14:36 |
| eanylin | sdake: Nice, 4K UHD player | 14:37 |
| sdake | eanylin ya got the panasonic | 14:37 |
| eanylin | sdake: I see | 14:37 |
| sdake | eanylin it was a return - so got 20% off | 14:37 |
| sdake | the cat didn't realize I think that a microsd card is needed in order for it to operate | 14:37 |
| eanylin | sdake: Ah, ok. Good deal then :) | 14:37 |
| eanylin | nea1: Multiple clouds, VIM zone? | 14:38 |
| eanylin | sdake: You seen the issue with not being able to SSH/ping the Cirros VM before? I think this test passed in the gate, as I can see that its one of the tests. Wasnt sure where I need to look at... | 14:40 |
| *** zhangqiankun has joined #openstack-kolla | 14:40 | |
| nea1 | eanylin: no more route the IPs from the one datacenter to the other, compute / storage apparently won't be setup in the old datacenter | 14:40 |
| sdake | eanylin i have not seen this issue you describe although I ahve seen it all the time with kolla-ansible | 14:40 |
| sdake | eanylin since you are running in a vm your vm interface may need some help | 14:41 |
| sdake | eanylin are you using kvm or virutalbox? | 14:41 |
| eanylin | nea1: I see. Hmm, I know tacker can handle multiple VIMs etc. And we can connect different datacenters together, not sure if that will be useful for you. | 14:42 |
| *** jtriley has quit IRC | 14:42 | |
| eanylin | sdake: I see this problem in kolla-ansible too | 14:42 |
| sdake | no idea what tacker is :) | 14:42 |
| sdake | eanylin right - its an envrionmental setup problem - i suspect if you wer erunning on bare metal yo uwould be good to go | 14:42 |
| eanylin | sdake: Tacker is quite interesting. It is a big tent project as well. | 14:43 |
| eanylin | sdake: Driven mainly by the Brocade guys (Not sure if its still the case now) | 14:43 |
| eanylin | sdake: I see. I am on KVM, QEMU | 14:43 |
| eanylin | sdake: I dont have bare metal lab unfortunately :( | 14:43 |
| eanylin | sdake: I got it to work before in AIO Kolla-Ansible, but I had to use veth interface | 14:44 |
| nea1 | eanylin: well my first Idea was for testing simply to have the network nodes in the other dc and use tinc for the main network_interface to connect them, but then there would be the problem that I can't route anything out over the servers there | 14:45 |
| nea1 | other Idea was use tinc to connect the network to the new DC, but that way I'd have to route the traffic | 14:46 |
| eanylin | nea1: Hmm, I havent use tinc before | 14:48 |
| sdake | eanylin the gate uses a veth interface - pretty sure | 14:50 |
| nea1 | like openvpn only in simple and with multi server which then uses peer to peer to route the packages (so that not everything has to go over the server) | 14:50 |
| sdake | eanylin could be wrong on that - it may be a bridge | 14:50 |
| eanylin | eanylin: So thats how they got it to work then | 14:50 |
| eanylin | sorry, sdake* | 14:50 |
| eanylin | sdake: I think we need a bridge too, for it to work | 14:51 |
| sdake | eanylin i dont know if its a veth or a bridge | 14:51 |
| *** krtaylor has quit IRC | 14:51 | |
| eanylin | sdake: I did it on my MacBook the other day, cannot remember what I did though. That was Kolla Ansible AIO, with 3.0.2 | 14:51 |
| eanylin | sdake: Ok, probably I need to look at how I set it up | 14:51 |
| sdake | eanylin what are your thoughts of kolla ansible? | 14:52 |
| eanylin | sdake: I think it is stable | 14:52 |
| sdake | anything else? | 14:52 |
| eanylin | sdake: I worked on it in Jan and was able to get things up pretty quickly | 14:52 |
| eanylin | sdake: Its good too, similar to Fuel in terms of concepts, just that it is using ansible instead of puppet | 14:52 |
| sdake | intereseting - never looked at fuel | 14:53 |
| eanylin | I got the multi-node working on my laptop too | 14:53 |
| eanylin | sdake: I used Fuel quite a lot | 14:53 |
| eanylin | sdake: The only problem I had with multi-node was my Cirros VM hung while booting up, probably related to the fact that I am using QEMU, it would have worked if I have bare metal | 14:54 |
| eanylin | sdake: I showed to my colleagues before that I can tear down multi-node Kolla Ansible and then bring it up again in about 20 to 30 mins on a powerful laptop, using the Operator node and Private registry built with Kolla-Build | 14:55 |
| eanylin | sdake: Thought it was pretty neat :) | 14:55 |
| *** zhangqiankun has quit IRC | 14:55 | |
| eanylin | nea1: Ok. Seems like its open-source, self-routing, mesh networking protocol, used for compressed, encrypted, virtual private networks (based on wikipedia) :) | 14:56 |
| eanylin | nea1: Your neutron would be the old datacenter though? | 15:00 |
| nea1 | well that is the big question, if i put it in the old DC then how would the routing be if I want to go out over the new one? | 15:04 |
| *** krtaylor has joined #openstack-kolla | 15:05 | |
| eanylin | nea1: So you are actually doing migration? | 15:05 |
| nea1 | I'd need multiple network nodes in both DCs which only take care of the network of the DC they are in | 15:06 |
| nea1 | not really, currently the Idea is keeping two nodes in the old DC as that has some advantages in some points | 15:07 |
| sdake | eanylin good to hear - lots o people like kolla-ansible | 15:07 |
| sdake | eanylin i am hopeful we can make kolla-kubernetes as good as kolla-ansible | 15:07 |
| sdake | eanylin as you can tell its a work in progress - getting it to work properly is the primary concern :) | 15:07 |
| nea1 | for e.g. IPs are cheap there | 15:07 |
| eanylin | nea1: Ok. I would think that it should be treated as 2 different clouds/VIMs | 15:08 |
| sdake | eanylin i am not entirely sure what to add to the current document to document sbezverk's workarounds | 15:08 |
| sdake | eanylin almost all of the workarounds were kubernetes related right? | 15:08 |
| eanylin | sdake: Yes, I think Kolla-Ansible is pretty good. The advantage that Kolla Kubernetes have is container cluster management (in my opinion) | 15:09 |
| eanylin | sdake: Yes, its mostly related to 1.6.0 workarounds in the gate | 15:09 |
| sdake | those are dictated by the fact that kubernetes rpm packaging wasn't gated and isn't gated | 15:09 |
| eanylin | sdake: I think sbezverk spent quite a lot of time fixing it | 15:09 |
| sdake | kbueadm also wasn't running e2e tests for the week in the runup to kubecon | 15:09 |
| eanylin | sdake: He did a great job fixing it :) | 15:10 |
| eanylin | sdake: Oh, ok. Didnt know that | 15:10 |
| sdake | kubeadm 1.6.0 = doa | 15:10 |
| sdake | kubelet RPM packaging for 1.6 = doa | 15:11 |
| sdake | his workarounds deal with the doa'ness | 15:11 |
| eanylin | sdake: What is doa? | 15:11 |
| sdake | dead on arrival | 15:11 |
| eanylin | sdake: Haha, yeah. I see a lot of people cursing it online ;) | 15:11 |
| sdake | thats a term used to describe a release that is dead on delivery :) | 15:11 |
| eanylin | at least for the past few days :) | 15:11 |
| sdake | eanylin what in the doc aroudn kolla-kubernets specifically is missing? | 15:12 |
| eanylin | sdake: I wonder if 1.6.1 would be out soon | 15:12 |
| sdake | eanylin who can predict such things | 15:12 |
| eanylin | sdake: I think Rich put some comments there but I am not sure if its all correct | 15:12 |
| eanylin | sdake: And I think he has problems bringing his environment up | 15:13 |
| sdake | indeed he seems to | 15:13 |
| sdake | i struggle to keep up with the comments as the editor of the document | 15:13 |
| eanylin | sdake: Yeah, its getting large | 15:13 |
| sdake | also its sunday :) | 15:13 |
| eanylin | the document I mean | 15:13 |
| eanylin | sdake: Yeah :) | 15:14 |
| sdake | i think we should have a "workaround kubadm guide" | 15:14 |
| sdake | and a "kolla-kubernetes quickstart" | 15:14 |
| eanylin | sdake: We can do that too | 15:14 |
| sdake | or deployment guide or whatever we are calling this thing | 15:14 |
| eanylin | sdake: To split it out from the main one | 15:14 |
| eanylin | sdake: Just to address the kubeadm part | 15:14 |
| sdake | id rather kubeadm document kubeadm | 15:14 |
| sbezverk | sdake: the issue was not with kubeadm, it was in kubelet | 15:15 |
| sdake | however the fact that kubeadm is broken is a blocker | 15:15 |
| eanylin | sdake: Ok | 15:15 |
| sdake | sbezverk ok - fiar enough kubelet then :) | 15:15 |
| eanylin | sbezverk: Ok | 15:15 |
| sdake | sbezverk the gate still isn't consisntetly green so i think kubeadm has problems | 15:15 |
| sdake | seems kubeadm join (multi node) has problems | 15:16 |
| sdake | AIO seems solid | 15:16 |
| sbezverk | sdake: there are two issues, one with version 3.0.0 and placement, which seems to be nova bug | 15:16 |
| eanylin | Most changes were made in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf | 15:17 |
| sbezverk | sdake: second there is a race condition some time hits on multi node | 15:17 |
| sbezverk | the rest should be solid | 15:17 |
| sdake | sbezverk right - that race condition on multinode is with kubeadm right? | 15:17 |
| sbezverk | sdake: if you carefully check the log, you will see that sometimes on multinode BOTH nodes report as not ready, which is again NOT related to kubeadm but kubelet.. | 15:18 |
| sdake | cool thanks | 15:18 |
| eanylin | sbezverk: I was discussing with sdake on why I cannot ping/ssh to Cirros VM. Any tips? | 15:19 |
| sdake | sbezverk we had debated whether the gate used a veth or bridge to setup multiinterface on one interface | 15:19 |
| eanylin | My cirros VM is able to ping 8.8.8.8 and the host and able to ping across 2 Cirros VMs | 15:19 |
| sbezverk | eanylin: I guess you have eliminated a possibility of security group misconfig? | 15:19 |
| eanylin | the init-runonce would add security policy for port 22 and ICMP | 15:20 |
| eanylin | think that one should be ok | 15:20 |
| eanylin | I think ping across 2 Cirros VMs would not work if security groups are not set up properly (if I remembered correctly) | 15:20 |
| sbezverk | eanylin: what I usually is hop by hop approach, which is tcpdump starting from tap interface on the instance and then until I find the place where packets get dropped.. | 15:21 |
| eanylin | ah, ok | 15:21 |
| eanylin | I was looking at the ovs container | 15:21 |
| sbezverk | once you know where the drop happens you will get some extra clues where to look next | 15:21 |
| eanylin | is that the right place to look at? | 15:21 |
| eanylin | ok | 15:22 |
| sbezverk | eanylin: well, you need to start from the instance | 15:22 |
| eanylin | ok | 15:22 |
| sdake | eanylin as the instructions stand now, are the kolla ones correct (step 3?) | 15:22 |
| sbezverk | virsh dumpxml instance name | grep tap | 15:22 |
| sbezverk | you need to do it in libvirt container | 15:22 |
| eanylin | sdake: /kolla-kubernetes/tools/patches/0001* is no longer needed, the latest code has been patched | 15:23 |
| sdake | eanylin beyond that? | 15:24 |
| eanylin | sdake: cat <<EOF > nova.conf | 15:24 |
| eanylin | [libvirt] | 15:24 |
| eanylin | virt_type=qemu | 15:24 |
| eanylin | cpu_mode=none | 15:24 |
| eanylin | sdake: Should come after genconfig | 15:24 |
| eanylin | else genconfig will over-write it | 15:24 |
| sdake | eanylin how do you figure | 15:24 |
| sdake | eanylin its cat /etc/kolla/config/nova.conf | 15:24 |
| sdake | config/nova.conf overrides defautls in /etc/kolla/nova.conf | 15:25 |
| eanylin | sdake: I used crunidi | 15:25 |
| eanylin | to update the j2 template | 15:25 |
| sdake | ya - thats the wrong way to do it for now :) | 15:25 |
| sdake | the /etc/kolla/config/nova.conf is the correct method | 15:25 |
| eanylin | sbezverk: Good point, I will take a look at the libvirt container | 15:25 |
| sdake | i added that in reaction to your crudini recommendation | 15:26 |
| eanylin | sdake: Ok. Are they different? | 15:26 |
| sdake | eanylin /etc/kolla/nova.conf is set with defaults from nova.conf.j2 | 15:26 |
| eanylin | sdake: Ultimately we need to make sure the config persist even with cluster reboot | 15:26 |
| eanylin | sdake: Ok | 15:26 |
| sdake | /etc/kolla/config/nova.conf _overrides_ any settings in /etc/kolla/nova.conf | 15:26 |
| sdake | genconfig operation does the overriding | 15:26 |
| eanylin | sdake: Ok | 15:27 |
| sdake | eanylin this is how kolla can be customized anyway you like | 15:27 |
| sdake | eanylin without us having to maintain 100's ofvariables | 15:27 |
| eanylin | sdake: Hmm, ok. I did it in that sequence and it was working ok. | 15:28 |
| sdake | ya - crudini will work | 15:28 |
| sdake | its a hack tho :) | 15:28 |
| eanylin | sdake: Okie. Cause I think thats how the gate did it too | 15:28 |
| eanylin | sdake: I see | 15:28 |
| sdake | the gate is doing it wrong | 15:29 |
| eanylin | sdake: lol :) | 15:29 |
| eanylin | sdake: Lets keep it your way then | 15:29 |
| sdake | i gotta say ultra hd on bluray is way better than 4k streaming | 15:29 |
| sdake | technology is maddenign :) | 15:30 |
| eanylin | sdake: Sounds like you are having fun :) | 15:30 |
| sdake | got a samsung tv at end of 2016 (closeout model) that was top of the line for 2k (around 1k savings) | 15:30 |
| sdake | 75" | 15:30 |
| eanylin | woot | 15:31 |
| sdake | the new samsung 75" QLED crap is like 5k | 15:31 |
| sdake | rediculous | 15:31 |
| sdake | i thought 2k was rediculous | 15:31 |
| eanylin | wah, thats expensive | 15:32 |
| sdake | ya my wife was like "you will just want a new tv at the end of 2017" | 15:32 |
| sdake | qled vs my tv is not worth 3k | 15:32 |
| eanylin | :) | 15:32 |
| sdake | the picture quality is slightly better | 15:32 |
| sdake | the picture quality of my 2016 HDR 4k tv is 3x better then my 2015 4k tv :) | 15:33 |
| sdake | i just rotate tvs through the house | 15:33 |
| sdake | my kids get hte leftovers :) | 15:33 |
| eanylin | nice ;) | 15:33 |
| sdake | vudu has something called hdx which is close to blueray 1k | 15:34 |
| sdake | amazon has hd which is 480p | 15:34 |
| sdake | all my stuff is on amazon | 15:34 |
| * sdake groans | 15:34 | |
| eanylin | ok | 15:35 |
| sdake | buying blueray movies with ultraviolet is the way to go imo | 15:35 |
| sdake | then you get a digital copy and a 4k copy | 15:35 |
| sdake | i guess i'll scan all my blueray movies into my vudu account - 2$/crack = racket | 15:36 |
| sdake | really tired of physical media, however, streamign 4k UHD = 25mbit network requirement plus 2mbit for audio track | 15:37 |
| sdake | i've got gigabit, but most people rocking 20mbit or so ;) | 15:37 |
| eanylin | yeah | 15:37 |
| sdake | and my gigabit only applies to the peering in Arizona | 15:37 |
| sdake | (which is where most peering takes place as there are no natural disasters) | 15:38 |
| sdake | there are endless datacenters here with network peering equipment here that ou would never know existed | 15:38 |
| sdake | as a result my end to end internet is usually pretty snappy :) | 15:39 |
| sdake | everyone in every other city complains about their pokey internets | 15:39 |
| sdake | if i could just get a /28 allocation from my isp, i'd be godlen :) | 15:40 |
| *** yangyape_ has quit IRC | 15:42 | |
| sdake | sbezverk i ordered that server | 15:42 |
| sdake | sbezverk and a rack for it to go in | 15:43 |
| sdake | sbezverk rack requires assembly | 15:43 |
| * sdake UGHS | 15:43 | |
| sdake | pretty beefy server, 32 hyperthreads - only 32gig ram | 15:43 |
| sdake | lol | 15:43 |
| sdake | need more ram | 15:43 |
| sdake | 128gb ram = 1.4k | 15:43 |
| sdake | i did not get an ilom vnic | 15:43 |
| sdake | for the moment going to roll with my CNA intel 10gig nic | 15:44 |
| sbezverk | sdake: cool, more cores would be a bit better but it is a good start and you can always replace CPU later | 15:44 |
| sdake | i've always operated under the "replace cpu later" theory and never eplaced the cpu | 15:45 |
| sdake | although have always augmented ram | 15:45 |
| sdake | my z820 has 128gb | 15:45 |
| sbezverk | sdake: I do, need more vms need more cpu | 15:45 |
| sdake | although my son has taken it over with overwatch | 15:45 |
| sdake | sbezverk my end game is to have 9 of these servers - so will have 288 cores | 15:46 |
| sdake | sbezverk wife wont let me buy em all at once ;) | 15:46 |
| sdake | my minidells have about had it | 15:47 |
| sbezverk | I suggest instead of investing in number of servers, invest in higher end cpu | 15:47 |
| sdake | i want bare metal not virutalization | 15:47 |
| sbezverk | 9 servers will be too noisy and suck tons of electricity | 15:47 |
| sbezverk | sdake: if you find a way for either alternative electricity source ;) or make cisco pay for it. let me know. I would be very interested | 15:48 |
| eanylin | you guys really invest heavily in servers ;) | 15:48 |
| sdake | sbezverk i can't get cisco to pay for anything | 15:49 |
| sdake | eanylin well i have 3 minidells, and 2 are broken I think | 15:50 |
| sdake | eanylin although i'm not really sure if the 2 are actually broken | 15:50 |
| eanylin | sdake: I only have 1 laptop with Xeon processor and 64GB RAM | 15:50 |
| sdake | eanylin sadly I need them to make forward progress in my work | 15:50 |
| eanylin | you guys are much better than me in that regard ;) | 15:50 |
| sdake | the switches are 35k list - but i can buy them for 12k from provantage | 15:51 |
| sdake | redonkulus price | 15:51 |
| sdake | instead I buy netgear - 100$/port | 15:51 |
| sdake | nto very feature rich but does the job | 15:51 |
| eanylin | nice | 15:52 |
| *** caoyuan_ has quit IRC | 15:53 | |
| sdake | star trek is amazing in 4k UHD | 15:55 |
| sdake | so much more detail | 15:55 |
| eanylin | I can imagine | 15:56 |
| eanylin | comes with a price tag though :D | 15:56 |
| sdake | can't take it with me ;) | 15:56 |
| *** manheim has joined #openstack-kolla | 15:58 | |
| *** manheim has quit IRC | 16:07 | |
| *** markmcclain has quit IRC | 16:09 | |
| *** markmcclain has joined #openstack-kolla | 16:10 | |
| klindgren | any ideas on why nova-compute can't create bridges that are seen by the docker host with the following config: | 16:13 |
| klindgren | https://gist.github.com/krislindgren/ad0815ec8308d0d407c3cbcd0e37b87a | 16:13 |
| *** shashank_t_ has joined #openstack-kolla | 16:15 | |
| klindgren | I believe that nothing special is done when running nova-libvirt as a container. IE nova-libvirt and nova-compute are not setup to share a network name space. I thought that with privileged enabled it would be able to create real dev devices. but it seems like the dev devices are limited to the network namespace that nova-compute runs in. | 16:15 |
| sdake | klindgren net=host does is used on nova-compute and (all other containers) in kolla - which uses the host networking and doesn't set it up in a specific namespace | 16:18 |
| klindgren | are you guys also doing net=host, ipc=host, pid_mode=host as well? | 16:22 |
| sdake | klindgren pid=host is used in nova-libvirt container | 16:22 |
| sdake | ipc=host is used in various containers but not many | 16:22 |
| sdake | klindgren like me to show you what to search for? | 16:22 |
| klindgren | yes please - I was going through the template stuff, but it must be in the kolla-Container stuff that I am not seeing as vars. | 16:23 |
| sdake | klindgren https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/iscsi/tasks/start.yml#L8 | 16:24 |
| klindgren | kk - also what is "group" used for? | 16:26 |
| sdake | klindgren note that ipc mode and pid mode default to off: https://github.com/openstack/kolla-ansible/blob/master/ansible/library/kolla_docker.py#L720-L723 | 16:26 |
| sdake | klindgren need more context for "group" | 16:27 |
| klindgren | https://github.com/openstack/kolla-ansible/blob/700089f5b3e2c06546c7745c57beda55d77e1cb9/ansible/roles/telegraf/defaults/main.yml#L10 | 16:27 |
| mnaser | klindgren thats related to the ansible inventory group afaik | 16:32 |
| mnaser | klindgren aka here https://github.com/openstack/kolla-ansible/blob/700089f5b3e2c06546c7745c57beda55d77e1cb9/ansible/inventory/multinode | 16:33 |
| klindgren | kk - which is then used on the docker-pull images for group task? Like glance where you have multiple container images to pull down | 16:33 |
| *** tonanhngo has joined #openstack-kolla | 16:49 | |
| klindgren | sdake, thanks that fixed up my issue. | 16:51 |
| sdake | klindgren which fixed your issue? :) | 16:52 |
| klindgren | setting nova-compute to use host for network_mode | 16:52 |
| sdake | klindgren could you expand on what you did? | 16:52 |
| sdake | i'm surprised that isn't default | 16:53 |
| sdake | morning mnaser | 16:54 |
| mnaser | o/ sdake | 16:54 |
| sdake | disney movie anywhere and ultraviolet plus vudu and amazon video all work together in a cross fucntional sort of way | 16:54 |
| sdake | annoying these movie co's can't work together | 16:54 |
| klindgren | I am doing a different way of deploy containers. API containers doing a blue/green approach, with managing the active container in haproxy as which is active. Complete with draining/wait support. | 16:55 |
| sdake | klindgren sounds cool - ought to implement upstream :) | 16:55 |
| sdake | klindgren in your copious spare time im sure :) | 16:55 |
| klindgren | For RPC containers, for those that support it I am starting a script up that creates a socket, and telling those services to use the systemd_notify flag. So we will wait/ assure that that the process inside actually started | 16:56 |
| klindgren | will also test that it has EStablished RPC connections to rabbitmq | 16:56 |
| klindgren | which is about as good as we can get these days for external testing ofr RPC | 16:56 |
| klindgren | for API's all services brought up we wait until the ports they are suppose to be listening on work, then execute a per API specific healthcheck to confirm its actuall working | 16:57 |
| klindgren | before moving the new container into the LB and stopping hte old container | 16:57 |
| klindgren | IF tests fail - RPC containers the old container is automatically restarted. and the deployment fails. On API containers the new container is stopped and destroyed (configurable) | 16:58 |
| klindgren | basically just trying to really nail the deploy a container and make sure its "working" | 16:59 |
| klindgren | Our current infra using puppet to blindly make config change and restart services - and pray that everything was working afterwards. | 16:59 |
| klindgren | So part of moving to containers and revamping that deploy process was to try to make each change (config or new software) as completely tested as possible | 17:00 |
| spsurya | night all | 17:01 |
| klindgren | We also save the last working container. To allow generic "rollbacks" | 17:01 |
| *** tonanhngo has quit IRC | 17:02 | |
| sbezverk | sdake: so which samsung model you would recommend which is not qled? | 17:08 |
| *** shashank_t_ has quit IRC | 17:11 | |
| mnaser | klindgren sounds really nice how are you testing the containers that they work | 17:13 |
| klindgren | API containers - with a customer script, per service that I execute against it after I wait for the ports to become available. | 17:15 |
| klindgren | RPC - if it notifies for systemd, it means that it started, but may not have connected to rabbitmq, but alteast the main loop is up. | 17:16 |
| klindgren | ALso have other protections like. If connectivity to rabbitmq is down. Refuse to do anything with neutron-openvswitch agent. | 17:17 |
| klindgren | except stop. | 17:17 |
| klindgren | Dont allow a deployment, dont allow a start, dont allow a restart. Until connectivity to RMQ is confirmed. | 17:17 |
| klindgren | As an example, glance I may call into the API to list all the images. Nova-api I may list services | 17:22 |
| klindgren | We also run cells V1 so depending on what you call into may test a deeper communication path than expected. like a nova host-aggregate-list actually makes calls to all the child cells. | 17:23 |
| klindgren | So the key is to get something that tests that just the thing that you started is working. | 17:24 |
| klindgren | Some of them we have healtcheck api's enabled, so we will make a call against that. | 17:24 |
| klindgren | sdake, is their a SPEC to start/run all the API services the same way? | 17:24 |
| klindgren | vs's some as eventlet and some behind apache. | 17:25 |
| klindgren | Internally we were running everything but neutron behind uwsgi, with that being fronted by haproxy. | 17:25 |
| *** iceyao has quit IRC | 17:32 | |
| zhubingbing | hi guys | 17:35 |
| SamYaple | klindgren: glance doesnt support uwsgi | 17:43 |
| klindgren | touche - except for neutron and glance. Neutron only because we are running an old enough release that we get screwed by neutron not having the https link support stuff. | 17:45 |
| SamYaple | neutron doesnt support uwsgi | 17:46 |
| klindgren | we had it running fine under uwsgi | 17:47 |
| klindgren | except for the ssl being offloaded. and neutron not supporting fixing up the links | 17:47 |
| SamYaple | klindgren: i mean, it doesnt support it though. its a goal for Pike, but officially the stance is they dont support it | 17:48 |
| SamYaple | are you thinking of pecan? | 17:48 |
| SamYaple | they tried doing that | 17:49 |
| *** iceyao has joined #openstack-kolla | 17:52 | |
| *** iceyao has quit IRC | 17:57 | |
| *** zhubingbing has quit IRC | 17:58 | |
| *** rhallisey has joined #openstack-kolla | 18:04 | |
| *** dave-mccowan has joined #openstack-kolla | 18:51 | |
| sdake | klindgren not that i am aware of | 18:53 |
| sdake | sbezverk i really like my 2016 tv - let me see if i can find he model number | 18:54 |
| sdake | sbezverk this is what i have - in 75" I think http://www.bestbuy.com/site/samsung-55-class-54-6-diag--led-2160p-smart-4k-ultra-hd-tv-with-high-dynamic-range-silver/5034800.p?skuId=5034800 | 18:55 |
| sdake | sbezverk i could possibly have a 65" and upgraded from a 55" | 18:55 |
| sdake | tbh not certain | 18:55 |
| sdake | http://www.bestbuy.com/site/samsung-65-class-64-5-diag--led-2160p-smart-4k-ultra-hd-tv-with-high-dynamic-range-silver/5065600.p?skuId=5065600 | 18:56 |
| sdake | 4 months ago they were 2499 | 18:56 |
| klindgren | sdake, I have the same tv - I like it as well. but in 60" - they had some pretty wicked deals on black friday. | 19:22 |
| klindgren | iirc it was $1100 - for the 60" | 19:22 |
| *** dave-mccowan has quit IRC | 19:23 | |
| sdake | wow thats a good deal | 19:23 |
| sdake | although I did want maximum screen space | 19:23 |
| sdake | i bought mine dec 24 iirc | 19:23 |
| sdake | or dec 28th iirc | 19:23 |
| sdake | something like that | 19:23 |
| klindgren | yea - the space where my tv goes a 60" is the largest that can fit | 19:23 |
| sdake | right after christmas = good deals too | 19:23 |
| sdake | i think my largest space is 65" | 19:24 |
| sdake | i got the right exact tv for the space :) | 19:24 |
| sdake | klindgren the panasonic blueray goes well with it | 19:24 |
| sdake | although its 600 bucks which is pretty expensive | 19:24 |
| sdake | https://www.amazon.com/Panasonic-Blu-ray-playback-support-DMP-UB900-K/dp/B01GRLCM60/ref=sr_1_11?ie=UTF8&qid=1491161149&sr=8-11&keywords=panasonic+bluray | 19:26 |
| sdake | i got mine for 600 bucks open box at frys | 19:26 |
| klindgren | I got the Samsung 4k blu-ray player. The remote is kinda shitty. But I couldn't justify that much on a blueray player | 19:26 |
| sdake | ya i had trouble justifying it | 19:27 |
| klindgren | http://www.samsung.com/us/televisions-home-theater/home-theater/blu-ray-dvd-players/4k-ultra-hd-blu-ray-player-ubd-k8500-za/ - was on sale for like 200/250 | 19:27 |
| klindgren | replaced an old PS3 that I use to use for movie watching | 19:27 |
| klindgren | much happier with the new player | 19:28 |
| klindgren | much quiter, has hdmi cnc | 19:28 |
| *** rhallisey has quit IRC | 19:42 | |
| *** kong has quit IRC | 19:49 | |
| openstackgerrit | Merged openstack/kolla stable/newton: Remove signing_dir option https://review.openstack.org/450203 | 20:03 |
| *** shashank_t_ has joined #openstack-kolla | 20:21 | |
| *** dave-mccowan has joined #openstack-kolla | 21:11 | |
| *** shashank_t_ has quit IRC | 21:25 | |
| *** dave-mcc_ has joined #openstack-kolla | 21:36 | |
| *** dave-mccowan has quit IRC | 21:39 | |
| *** kong has joined #openstack-kolla | 21:39 | |
| *** manheim has joined #openstack-kolla | 21:39 | |
| *** manheim has quit IRC | 21:48 | |
| *** manheim has joined #openstack-kolla | 21:48 | |
| *** jrist has quit IRC | 22:39 | |
| *** manheim has quit IRC | 22:51 | |
| *** jrist has joined #openstack-kolla | 22:57 | |
| *** jrist has quit IRC | 22:57 | |
| *** jrist has joined #openstack-kolla | 22:57 | |
| *** jrobinson has joined #openstack-kolla | 23:08 | |
| *** shashank_t_ has joined #openstack-kolla | 23:11 | |
| *** shashank_t_ has quit IRC | 23:14 | |
| *** shashank_t_ has joined #openstack-kolla | 23:14 | |
| *** shashank_t_ has quit IRC | 23:19 | |
| *** yangyapeng has joined #openstack-kolla | 23:40 | |
| *** yangyapeng has quit IRC | 23:45 | |
| *** eaguilar has joined #openstack-kolla | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!