Friday, 2019-03-29

*** flwang has quit IRC00:16
*** ianychoi has quit IRC00:17
*** ianychoi has joined #openstack-containers00:18
*** goldyfruit has left #openstack-containers00:27
*** hongbin has joined #openstack-containers00:40
*** itlinux has joined #openstack-containers01:05
*** yolanda has quit IRC01:07
*** yolanda has joined #openstack-containers01:15
openstackgerritOpenStack Release Bot proposed openstack/magnum-ui stable/stein: Update .gitreview for stable/stein  https://review.openstack.org/64857101:16
openstackgerritOpenStack Release Bot proposed openstack/magnum-ui stable/stein: Update UPPER_CONSTRAINTS_FILE for stable/stein  https://review.openstack.org/64857201:16
openstackgerritOpenStack Release Bot proposed openstack/magnum-ui master: Update master for stable/stein  https://review.openstack.org/64857301:16
*** flwang has joined #openstack-containers01:20
*** ricolin has joined #openstack-containers01:21
openstackgerritMerged openstack/magnum-ui stable/stein: Update .gitreview for stable/stein  https://review.openstack.org/64857101:52
openstackgerritMerged openstack/magnum-ui master: Update master for stable/stein  https://review.openstack.org/64857301:52
openstackgerritMerged openstack/magnum-ui stable/stein: Update UPPER_CONSTRAINTS_FILE for stable/stein  https://review.openstack.org/64857202:07
*** ramishra has joined #openstack-containers02:43
openstackgerritFeilong Wang proposed openstack/magnum master: Allow admin update cluster/template in any project  https://review.openstack.org/64859003:04
*** goldyfruit has joined #openstack-containers03:09
*** ykarel has joined #openstack-containers03:22
*** hongbin has quit IRC03:52
*** udesale has joined #openstack-containers03:56
*** ykarel has quit IRC04:47
*** ykarel has joined #openstack-containers05:02
*** sidx64 has joined #openstack-containers05:43
*** janki has joined #openstack-containers06:10
*** rcernin has quit IRC06:38
*** udesale has quit IRC06:41
*** udesale has joined #openstack-containers06:41
*** sidx64 has quit IRC07:09
*** sidx64 has joined #openstack-containers07:14
*** sidx64 has quit IRC07:39
*** ykarel is now known as ykarel|lunch07:42
*** rcernin has joined #openstack-containers08:23
*** sidx64 has joined #openstack-containers08:25
*** sidx64 has quit IRC08:26
*** ttsiouts has joined #openstack-containers08:28
*** sidx64 has joined #openstack-containers08:28
*** ttsiouts has quit IRC08:40
*** ykarel|lunch is now known as ykarel08:49
*** rcernin has quit IRC08:57
*** udesale has quit IRC08:57
dioguerra:goldyfruit what flag are you using?08:58
*** udesale has joined #openstack-containers08:58
*** udesale has quit IRC09:07
*** udesale has joined #openstack-containers09:10
openstackgerritRicardo Rocha proposed openstack/magnum master: Set a fixed cipher suite set for Traefik  https://review.openstack.org/64864909:12
*** pcaruana has joined #openstack-containers09:15
*** udesale has quit IRC09:15
*** sapd1_x has joined #openstack-containers09:17
*** ttsiouts has joined #openstack-containers09:20
*** alisanhaji has joined #openstack-containers09:21
*** itlinux_ has joined #openstack-containers09:29
*** itlinux has quit IRC09:31
*** sapd1_x has quit IRC09:32
openstackgerritRicardo Rocha proposed openstack/magnum master: [k8s] Add nginx based ingress controller  https://review.openstack.org/64865509:48
*** sapd1_x has joined #openstack-containers09:55
*** colby_ has quit IRC10:07
*** colby_ has joined #openstack-containers10:10
*** sapd1_x has quit IRC10:13
*** ut2k3 has joined #openstack-containers10:22
*** sapd1_x has joined #openstack-containers10:23
*** ut2k3 has left #openstack-containers10:30
*** ricolin has quit IRC10:33
*** colby_ has quit IRC10:34
*** ttsiouts has quit IRC11:12
*** ttsiouts has joined #openstack-containers11:12
*** ttsiouts has quit IRC11:17
*** colby_ has joined #openstack-containers11:18
*** benohara has joined #openstack-containers11:40
*** sapd1_x has quit IRC11:40
benoharahey, anyone able to help trouble shoot an issue...when the kubernetes master boots, it runs cloudinit and fails when trying to retrieve the certificate11:41
benoharamagnum throws an exception...Remote error: Forbidden You are not authorized to perform the requested action: identity:get_user11:41
benoharausing barbican as the cert backend11:42
benohararoles all look correct according to the queens install guide11:43
benoharai also see when deleting a cluster, it tries to delete certificates, but uses http://localhost:9311??11:44
*** ttsiouts has joined #openstack-containers11:48
*** ttsiouts has quit IRC11:53
*** benohara has quit IRC12:08
*** benohara has joined #openstack-containers12:09
*** janki has quit IRC12:20
*** ttsiouts has joined #openstack-containers12:24
*** goldyfruit has quit IRC12:25
*** ttsiouts has quit IRC12:46
*** ttsiouts has joined #openstack-containers12:47
*** lpetrut has joined #openstack-containers13:15
dioguerraflwang: Managed to test everythin up until CA but not healing (broke my devstack)13:27
openstackgerritDiogo Guerra proposed openstack/magnum master:  [k8s] Add missing container_prefix parameter  https://review.openstack.org/63793913:42
openstackgerritDiogo Guerra proposed openstack/magnum stable/stein:  [k8s] Add missing container_prefix parameter  https://review.openstack.org/64869013:46
*** munimeha1 has joined #openstack-containers14:18
NobodyCamGood Morning Container folks, Happy Friday14:53
NobodyCamI wanted to thank everyone here for all the help14:54
NobodyCamI was able to solve my cluster stability issue after adding networks to the minion nodes14:56
*** alisanhaji has quit IRC14:57
NobodyCamsolution involved some hacking in configure-kubernetes-master.sh14:57
NobodyCamedited `--kubelet-preferred-address-types` the is set in kube_api_args14:58
NobodyCamand added a startup script to build /etc/hosts file with all the internal addresses15:00
*** hongbin has joined #openstack-containers15:26
*** gmann is now known as gmann_afk15:32
*** ykarel has quit IRC15:40
*** ricolin has joined #openstack-containers15:43
*** lpetrut has quit IRC15:47
*** sidx64 has quit IRC15:52
*** ricolin has quit IRC16:07
*** hongbin has quit IRC16:18
*** benohara has quit IRC16:32
brtknrNobodyCam: what stability issue were youfacing?16:36
brtknrNobodyCam: what did you edit --kubelet-preferred-address-types to?16:37
NobodyCamGood Morning brtknr : My kube cluster would become unstable if I attached additional networks to the minion nodes (private vlan's are workload specific)16:39
brtknrNobodyCam: We were having the same problems16:40
brtknrI have 3 NIC per instance16:40
brtknrnodes were randomly getting assigned IP from one of the 3 interfaces16:41
NobodyCamI've got it stable and working, but it's a awful hack16:41
brtknrNobodyCam: we fixed the problem by setting --kubelet-preferred-address-types=Hostname,InternalIP,ExternalIP16:42
*** ttsiouts has quit IRC16:42
brtknrBut we have a DNS server which resolves Hostname to node IP16:42
NobodyCamsame here, but I needed to do it with out dns16:43
NobodyCamso I changed the setting in the same way, I removed externalIP16:44
*** jmlowe has quit IRC16:46
NobodyCamthen I create a .sh file in usr/local/bin that uses kubectl to get the ip off the minion nodes and creates a /etc/hosts file in the containers for the kube-* services, Really the only one that needed it kube-apiserver but I wanted to make sure they all had the same file16:46
NobodyCamI also add a kube-register systemd service that registers the nodes to failure domain nova and kicks off the .sh script16:47
brtknrI think the problem lies with openstack cloud provider16:55
brtknrIf you remove --cloud-provider=openstack, the problem never occurs16:57
brtknrIf you remove --cloud-provider=openstack from kubelet args, the problem never occurs16:57
*** flwang1 has joined #openstack-containers16:57
*** ykarel has joined #openstack-containers16:59
NobodyCamyea the the cloud-provider seems to always return all networks attached to the node, then just picks one:  at least that what I've seen in the load balancer code I've been looking into: https://github.com/kubernetes/cloud-provider-openstack/blob/b100f24bb0e4b70c71587e24cac6ae84ddfa77d4/pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go#L79817:01
NobodyCam(note above may be from correct tag)17:02
NobodyCam(note above may NOT be from correct tag)17:02
NobodyCam:p17:02
* NobodyCam needs more coffee17:02
*** ykarel has quit IRC17:03
*** jmlowe has joined #openstack-containers17:07
brtknrNobodyCam: have you seen this PR: https://github.com/kubernetes/cloud-provider-openstack/pull/44417:45
NobodyCamOh I had not seen that, but I like what it does!17:48
*** yolanda has quit IRC17:53
*** ramishra has quit IRC18:01
*** jmlowe has quit IRC18:07
*** jmlowe has joined #openstack-containers18:16
NobodyCamis there a listing of roles required to create a cluster, I am attempting to setup a project / user but am hitting : Forbidden: resources[0].resources.kube_master_eth0: ((rule: create_port and rule: create_port: allowed_address_pairs) and rule: create_port: fixed_ips) is disallowed by policy Neutron server18:18
NobodyCamI added _member_ and heat_stack_owner roles18:18
openstackgerritFeilong Wang proposed openstack/magnum master: Add cluster upgrade to the API  https://review.openstack.org/51495918:21
*** pcaruana has quit IRC18:38
*** gmann_afk is now known as gmann19:19
*** yolanda has joined #openstack-containers19:58
*** jmlowe has quit IRC20:21
*** munimeha1 has quit IRC20:59
*** jmlowe has joined #openstack-containers21:14

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!