Tuesday, 2018-04-24

*** pcaruana has quit IRC00:02
*** sfilatov has joined #openstack-containers00:17
*** sfilatov has quit IRC00:21
*** sfilatov has joined #openstack-containers00:35
*** sfilatov has quit IRC00:39
*** sfilatov has joined #openstack-containers00:52
*** sfilatov has quit IRC00:57
*** sfilatov has joined #openstack-containers01:17
*** sfilatov has quit IRC01:21
*** hongbin_ has joined #openstack-containers01:32
*** sfilatov has joined #openstack-containers01:38
*** sfilatov has quit IRC01:43
*** zhubingbing__ has joined #openstack-containers01:47
*** zhubingbing_ has quit IRC01:49
*** ramishra has joined #openstack-containers01:52
*** sfilatov has joined #openstack-containers02:01
*** fragatina has quit IRC02:02
*** sfilatov has quit IRC02:05
*** ricolin has joined #openstack-containers02:25
*** sfilatov has joined #openstack-containers02:26
*** sfilatov has quit IRC02:31
*** fragatina has joined #openstack-containers02:39
*** sfilatov has joined #openstack-containers02:42
*** sfilatov has quit IRC02:47
*** yamamoto has quit IRC02:53
*** yamamoto has joined #openstack-containers02:54
*** yamamoto has quit IRC02:59
*** sfilatov has joined #openstack-containers03:05
*** sfilatov has quit IRC03:09
*** sfilatov has joined #openstack-containers03:34
*** sfilatov has quit IRC03:39
*** lpetrut has joined #openstack-containers03:39
*** janki has joined #openstack-containers03:44
openstackgerritcaoyuan proposed openstack/magnum master: Fix the magnum description  https://review.openstack.org/56380803:53
*** shu-mutow-AWAY is now known as shu-mutow03:57
*** hongbin_ has quit IRC03:58
*** sfilatov has joined #openstack-containers03:59
zhubingbing__Hi guys04:02
zhubingbing__Hello is anyone familiar with this error? ERROR oslo.service.loopingcall ValueError: Field `master_addresses[0]' cannot be None04:02
*** ianychoi has quit IRC04:02
*** sfilatov has quit IRC04:03
*** ianychoi has joined #openstack-containers04:04
*** sfilatov has joined #openstack-containers04:09
*** mvpnitesh has joined #openstack-containers04:09
*** lpetrut has quit IRC04:11
*** sfilatov has quit IRC04:13
*** ianychoi has quit IRC04:28
*** sfilatov has joined #openstack-containers04:29
*** ianychoi has joined #openstack-containers04:31
*** sfilatov has quit IRC04:33
*** janki has quit IRC04:39
*** flwang1 has quit IRC04:45
*** sfilatov has joined #openstack-containers04:47
*** sfilatov has quit IRC04:52
*** lpetrut has joined #openstack-containers04:58
*** sfilatov has joined #openstack-containers05:03
openstackgerritShu Muto proposed openstack/magnum-ui master: [DNM] INVESTIGATING CI JOBS AND TOX  https://review.openstack.org/56348305:05
*** sfilatov has quit IRC05:08
openstackgerritShu Muto proposed openstack/magnum-ui master: [DNM] INVESTIGATING CI JOBS AND TOX  https://review.openstack.org/56348305:08
*** lpetrut has quit IRC05:27
*** yolanda_ has joined #openstack-containers05:27
*** yolanda has quit IRC05:28
*** sfilatov has joined #openstack-containers05:30
*** sfilatov has quit IRC05:35
*** sfilatov has joined #openstack-containers05:47
*** janki has joined #openstack-containers05:49
*** shu-mutow is now known as shu-mutow-AWAY05:59
*** iranzo has joined #openstack-containers06:00
*** iranzo has joined #openstack-containers06:00
*** armaan has joined #openstack-containers06:24
*** armaan has quit IRC06:24
*** trinaths has joined #openstack-containers06:24
*** lpetrut has joined #openstack-containers06:29
*** threestrands has quit IRC06:32
*** mago has joined #openstack-containers06:47
*** threestrands has joined #openstack-containers06:50
*** threestrands has quit IRC06:50
*** threestrands has joined #openstack-containers06:50
*** mago has quit IRC06:57
openstackgerritOpenStack Proposal Bot proposed openstack/magnum master: Imported Translations from Zanata  https://review.openstack.org/54875306:57
*** pcaruana has joined #openstack-containers06:58
openstackgerritShu Muto proposed openstack/magnum-ui master: [DNM] INVESTIGATING CI JOBS AND TOX  https://review.openstack.org/56348306:58
*** ianychoi has quit IRC07:00
*** gsimondon has joined #openstack-containers07:06
*** armaan has joined #openstack-containers07:06
openstackgerritShu Muto proposed openstack/magnum-ui master: [DNM] INVESTIGATING CI JOBS AND TOX  https://review.openstack.org/56348307:08
openstackgerritKirsten G. proposed openstack/magnum master: Add and improve tests for certificate manager  https://review.openstack.org/55224407:09
*** threestrands has quit IRC07:10
*** ianychoi has joined #openstack-containers07:11
*** mago has joined #openstack-containers07:13
*** rcernin has quit IRC07:16
*** jchhatbar has joined #openstack-containers07:21
*** oikiki has joined #openstack-containers07:23
*** janki has quit IRC07:23
*** armaan has quit IRC07:28
oikikihi strigazi!  this small patch was approved, but failed to merge bc it needed a manual rebase (touched same file as a prior patch that was merged immediately before). it's up to date now (zuul finishing up tests) so could you reapprove when you get a chance? https://review.openstack.org/#/c/552244/07:31
*** derekjhyang has quit IRC07:44
*** yolanda_ has quit IRC07:49
*** mgoddard has joined #openstack-containers07:57
*** sfilatov has quit IRC08:00
openstackgerritShu Muto proposed openstack/magnum-ui master: [DNM] INVESTIGATING CI JOBS AND TOX  https://review.openstack.org/56348308:00
*** yolanda_ has joined #openstack-containers08:02
*** yolanda_ is now known as yolanda08:02
*** sfilatov has joined #openstack-containers08:03
*** sfilatov has joined #openstack-containers08:03
*** oikiki has quit IRC08:03
*** AlexeyAbashkin has joined #openstack-containers08:05
*** armaan has joined #openstack-containers08:08
*** ktibi has joined #openstack-containers08:10
*** gsimondo1 has joined #openstack-containers08:12
*** gsimondon has quit IRC08:15
*** dsariel has joined #openstack-containers08:20
*** mvpnitesh has quit IRC08:23
*** ArchiFleKs has joined #openstack-containers08:25
*** sfilatov has quit IRC08:35
*** sfilatov has joined #openstack-containers08:37
*** mdnadeem has joined #openstack-containers08:39
*** zhubingbing_ has joined #openstack-containers08:43
*** zhubingbing__ has quit IRC08:46
*** mvpnitesh has joined #openstack-containers08:47
*** yolanda has quit IRC08:48
*** armaan has quit IRC08:58
*** armaan has joined #openstack-containers08:58
*** yolanda has joined #openstack-containers09:01
*** ricolin_ has joined #openstack-containers09:01
*** sfilatov has quit IRC09:02
*** armaan has quit IRC09:03
*** sfilatov has joined #openstack-containers09:03
*** sfilatov has quit IRC09:03
*** ricolin has quit IRC09:04
*** sfilatov has joined #openstack-containers09:04
*** flwang1 has joined #openstack-containers09:04
*** sfilatov has joined #openstack-containers09:05
*** ricolin_ has quit IRC09:25
*** sfilatov has quit IRC09:30
*** sfilatov has joined #openstack-containers09:32
flwang1zhubingbing_: where did you see the error?09:37
openstackgerritShu Muto proposed openstack/magnum-ui master: [DNM] INVESTIGATING CI JOBS AND TOX  https://review.openstack.org/56348309:40
zhubingbing_I don't create volume type09:43
zhubingbing_I create volume types can solved this error09:43
zhubingbing_https://ask.openstack.org/en/question/110729/magnum-cluster-create-k8s-cluster-error-resourcefailure/09:43
flwang1cool09:47
flwang1zhubingbing_: btw, good to see you're joining this channel09:47
zhubingbing_- -09:48
flwang1strigazi: meetings in 10 mins?09:51
strigaziyes09:52
strigazi8'09:52
flwang1cool09:53
flwang1strigazi: btw, i have verified your patch, https://review.openstack.org/#/c/563676/  all good09:54
strigazithanks, I wanted to do some final tests with calico today, but then devstack hit me with pip fun09:55
strigazi error in setup command: Error parsing /opt/stack/cinder/setup.cfg: SyntaxError: '<' operator not allowed in environment markers09:56
strigaziI'm just creating a new one09:56
flwang1strigazi: ok, cool, thanks09:57
flwang1strigazi: mind approving this https://review.openstack.org/#/c/555223/ ?09:57
flwang1and i have a question about atomic install for you after the meeting09:58
strigazidone09:58
strigazi#startmeeting containers09:59
openstackMeeting started Tue Apr 24 09:59:46 2018 UTC and is due to finish in 60 minutes.  The chair is strigazi. Information about MeetBot at http://wiki.debian.org/MeetBot.09:59
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.09:59
*** openstack changes topic to " (Meeting topic: containers)"09:59
openstackThe meeting name has been set to 'containers'09:59
strigazi#topic Roll Call09:59
*** openstack changes topic to "Roll Call (Meeting topic: containers)"10:00
flwang1o/10:00
rochapor1oo/10:00
flwang1rochapor1o: waves from Wellington10:01
rochapor1oflwang1: waves back from gva :)10:02
rochapor1osay hi to everyone10:02
*** armaan has joined #openstack-containers10:02
strigazi#topic Blueprints/Bugs/Ideas10:02
*** openstack changes topic to "Blueprints/Bugs/Ideas (Meeting topic: containers)"10:02
strigaziFrom my side, I discovered this one that I10:03
strigaziI'm also resposible for:10:03
strigazi#link https://bugs.launchpad.net/magnum/+bug/176628410:03
openstackLaunchpad bug 1766284 in Magnum rocky "k8s_fedora: Kubernetes dashboard service account must not have any permissions" [Critical,In progress] - Assigned to Spyros Trigazis (strigazi)10:03
flwang1strigazi: yep, that's a good catch10:03
strigaziThe fix is easy, just create an admin user10:04
strigaziand use each token10:04
strigaziand use its token10:04
strigaziApart from that, I don't update from last week, eventlet is still stuck and I haven't updated my other patches, will do this week10:05
flwang1strigazi: finished?10:06
strigaziJust some input, for flwang1. 548139 looks good, I'm testing it today10:07
strigaziflwang1: yes, not much this week from me10:07
flwang1strigazi: ok, from my side10:07
flwang1I finally make the calico on master patch works, i think it's ready to go10:07
flwang1and still working on the keystone integration work, need some help about the atomic install10:08
strigaziflwang1: are we sure that we don't need kube-proxy on master?10:08
flwang1besides, start to work on the health monitoring10:08
*** mvpnitesh has quit IRC10:08
flwang1strigazi: i tested it, so far i haven't seen any issue10:09
flwang1dashboard, dns, heapster, all work10:09
flwang1btw, i will take vacation from Wed to next Mon, back next Tue, just FYI10:10
strigazido you have any input for the health monitor? Are you working on the status?10:10
sfilatovHi! I just created a bug: https://bugs.launchpad.net/magnum/+bug/176654610:11
openstackLaunchpad bug 1766546 in Magnum "Multi-Master deployments for k8s driver use different service account keys" [Undecided,New]10:11
flwang1strigazi: yep, i'm working on adding 2 new fields for the schema10:11
sfilatovI would like to discuss the way to resolve this10:11
flwang1sfilatov: we're in meeting, can we discuss offline?10:11
strigazisfilatov give us few mins, rochapor1o fyi 176654610:11
strigaziflwang1: I pinged him to discuss it in open discussion10:12
flwang1strigazi: cool10:12
flwang1that's all on my side, thanks10:12
sfilatovWe can workaround it using ca.key so each server has the same artifact10:12
strigaziAre the two fields in the spec?10:12
flwang1strigazi: yes10:12
strigaziflwang1: perfect10:13
flwang1as we discussed last week10:13
strigazigood10:13
sfilatovMy concern is that it's not safe to expose it and we could generate this pair separately10:13
flwang1strigazi: i'm very excited to see magnum  is going to have auto-healing10:13
sfilatovAnd not use ca.key for serviceaccount tokens10:13
strigaziflwang1: :)10:15
strigaziflwang1: that's all from you, right?10:15
flwang1yes10:15
flwang1i'm clear10:15
strigazisfilatov:10:15
rochapor1olooking10:16
strigaziyou have a concern on using the ca.key in general or just for the serviceaccount key?10:16
sfilatovstrigazi: serviceaccount key. My concern is we defintely dont need it for serviceaccount key10:17
sfilatovstrigazi: So if we could not ewxpose it, we better not to10:17
strigazisfilatov: you mean, not set it at all?10:18
strigazisfilatov: and let kubernetes generate one? Have you tried it?10:18
sfilatovstrigazi: Not pass it to users vm10:19
rochapor1oflwang1: i've start looking at auto healing10:19
rochapor1oyou'll be at the summit? we can discuss there10:19
strigazisfilatov: I'm confused, the problem is that we need the same key. And you suggest to use what as a key for service account tokens?10:20
rochapor1osfilatov: the goal is to have a separate cert/key pair for the kube services, is it? we can store it in the cert store (barbican or the default db) and load from there10:20
*** mvpnitesh has joined #openstack-containers10:21
flwang1rochapor1o: no, i won't be there because limited budget ;(10:21
sfilatovstrigazi: Yes we need the same key. I suggest generate it. Since ca.key is a safety issue - whoever has it could authenticate in our k8s cluster10:21
sfilatovrochapor1o: yes, though I'm not sure we need to store it at all. We could generate it and pass through heat variables10:22
rochapor1owe already use barbican to store the cluster secrets10:23
rochapor1oyou would need them if you decide to update the number of masters later10:23
rochapor1oor for upgrades10:23
sfilatovrochapor1o: yes you are right10:24
strigazirochapor1o: the key will be available in heat as a paramater10:24
strigaziit will work when scaling as well10:24
flwang1any impact if there is no barbican?10:24
rochapor1osecrets go to the magnum db if there's no barbican (the default)10:25
strigazisfilatov: What I don't get is why it is a security concern since the key will be only on master nodes, and the master ndoes already have the api server certs.10:26
sfilatovstrigazi: I suppose api keys are not that important as ca key10:26
sfilatovstrigazi: Though I might be wrong10:27
flwang1rochapor1o: yep, i know. but if saving it in magnum db, magnum can only save 1 ca file10:27
sfilatovstrigazi: I see that anyone who has the ca.key can authenticate as any user in magnum10:27
strigazisfilatov: we can double check then, if the current certs on the master node(s) are similar security  risks10:27
strigazisfilatov: yes, the ca.key can give access to everything10:28
*** armaan has quit IRC10:28
rochapor1oflwang1: i think db uses the same secret naming convention. if we change to have a special cert/keypair for the api server which is shared among all masters, a change for barbican will also work for the db case10:29
*** armaan has joined #openstack-containers10:29
strigazisfilatov: let's take this offline in the bug then10:29
flwang1strigazi: +110:29
flwang1let's discuss this offline10:30
strigazisfilatov: can you have a look if the other certs are a security risk, I hope they are not.10:30
sfilatovstrigazi: Can I try to implement this fix If we decide to generate a key it? :)10:31
sfilatovstrigazi: Yes, I will check other certs10:31
strigazisfilatov: and if so, we can generate a key just for this purpose10:31
strigazisfilatov: maybe it can also be used for the cert mananger?10:31
strigazisfilatov: if we decide to go this way, go for it10:32
sfilatovstrigazi: do you mean controller manager?10:32
openstackgerritMerged openstack/magnum master: Add and improve tests for certificate manager  https://review.openstack.org/55224410:32
strigazisfilatov: no, this on https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/10:32
strigazisfilatov: no, this one https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/10:32
*** armaan has quit IRC10:33
flwang1strigazi: should we move on?10:33
strigaziyes10:33
*** armaan has joined #openstack-containers10:33
*** lxkong has quit IRC10:35
strigaziflwang1: Do you have anything else for the meeting?10:35
flwang1strigazi: there are only 2 milestones for Rocky10:36
flwang1we should review our current pipeline to see where we should pay more attention10:37
flwang1to make sure we can meet our goal for R10:37
strigaziWe can try to aim for rocky-210:38
strigaziflwang1:  Do you have standing review apart from calico?10:39
flwang1i'm blocked by the atomic command for keystone10:39
strigaziflwang1: ok10:39
flwang1after finished  keystone, i can focus on the monitoring stuff10:40
flwang1which i think is the pre-condition for auto-healing10:40
flwang1strigazi: and i'm also keen to review the upgrade PoC patch10:40
strigaziok, let's aim for Thursday to test10:41
flwang1strigazi: and i have this tiny patch https://review.openstack.org/56245410:41
sfilatovflwang1: can you DM upgrade PoC patch to me? I'm also interested10:41
flwang1rename some scritps10:41
flwang1sfilatov: haha, i'm waiting for the magic link from strigazi ;)10:42
strigaziI'll put you both in the loop10:42
sfilatovflwang1: looking forward :)10:42
strigazithanks folks10:43
flwang1sfilatov: let's push this guy ;)10:43
strigazipush is good10:44
flwang1strigazi: and it would be nice if you can update the spec to reflect the api changes (show available versions)10:45
strigaziRenaming things is not exciting, but I see the point10:45
flwang1strigazi: i know it's not exciting, so no rush10:45
flwang1strigazi: btw, another thing i mentioned before is the fluentd support10:46
flwang1i think it's a low-hanging-fruit, but it's not my first priority10:46
flwang1in Rocky, i'd like to help get the auto-healing and upgrade done10:47
strigaziflwang1: yes, did you create a bp or bug?10:47
flwang1strigazi: not yet10:47
flwang1i'm happy to create one in case if there is new contributor interested in10:47
strigaziCreate a bp then10:48
strigaziI hope we can find someone10:48
flwang1strigazi: no problem10:48
strigaziflwang1: do you want to discuss the atomic issue? Is there anything else?10:50
flwang1strigazi: yep, i prefer to discuss the atomic issue offline10:50
flwang1i don't want to waste others time10:50
*** jianhua has joined #openstack-containers10:51
strigaziok10:51
strigazilet's wrap this then10:52
flwang1strigazi: here you go https://blueprints.launchpad.net/magnum/+spec/fluentd10:52
*** mdnadeem has quit IRC10:53
*** mdnadeem has joined #openstack-containers10:53
strigaziflwang1: thanks, can you add some more info? Will this be self contained in the cluster?10:53
strigaziWill it accept endpoint to push to different places?10:54
strigaziWill it accept an endpoint as a parameter to push to different places?10:54
flwang1strigazi: sure, i will add more info later10:54
flwang1based on my understanding with fluentd, it's most like a logging agent, so we will start it with dameonset10:55
flwang1to make sure it can run on each node10:55
strigaziWhere the data will be stored?10:56
flwang1Elasticsearch10:57
strigaziIn a provided endpoint? or inside the cluster?10:57
flwang1strigazi: https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch10:58
flwang1strigazi: in cluster10:58
strigaziok self-contained then10:58
flwang1yes10:58
strigaziWe can also look in the option to give an external es endpoint10:58
flwang1i'm also referring this https://kubernetes.io/docs/concepts/cluster-administration/logging/10:58
flwang1strigazi: absolutely10:59
flwang1i can imagine cern has already got a elasticsearch setup10:59
strigazitime is up, we can discuss in the channel10:59
flwang1yes10:59
flwang1sure10:59
strigaziending the meeting then, thanks :)10:59
strigazi#endmeeting11:00
*** openstack changes topic to "OpenStack Containers Team"11:00
openstackMeeting ended Tue Apr 24 11:00:04 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)11:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/containers/2018/containers.2018-04-24-09.59.html11:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/containers/2018/containers.2018-04-24-09.59.txt11:00
openstackLog:            http://eavesdrop.openstack.org/meetings/containers/2018/containers.2018-04-24-09.59.log.html11:00
flwang1thank you11:00
openstackgerritMerged openstack/magnum master: Make DNS pod autoscale  https://review.openstack.org/55522311:00
strigaziLet's have a quick one for atomic11:00
mnasiadkawell, just give option to specify output plugin in fluentd, then you can send to ES or Influx, or anything else11:00
strigazimnasiadka: makes sense11:01
flwang1mnasiadka: interested in to take it? ;)11:01
strigaziflwang1: you can base the keystone server on this one https://github.com/projectatomic/atomic-system-containers/tree/master/etcd11:01
flwang1strigazi: as for atomic, see http://paste.openstack.org/show/719805/11:01
mnasiadkaWhy not, I could learn something :)11:01
strigazimnasiadka: excellent11:02
flwang1i'd like to understand to how translate it into an atomic install command11:02
flwang1mnasiadka: fantastic, thanks11:02
strigaziflwang1: with atomic you can set variable like : --set var1='value1'11:02
strigazihttps://github.com/openstack/magnum/blob/master/magnum/drivers/common/image/heat-container-agent/config.json.template#L35311:03
strigaziin this REQUESTS_CA_BUNDLE is a parameter for atomic install11:03
strigaziand defaults can go here https://github.com/openstack/magnum/blob/master/magnum/drivers/common/image/heat-container-agent/manifest.json11:04
strigaziflwang1:  like this https://github.com/projectatomic/atomic-system-containers/blob/master/etcd/manifest.json11:04
strigaziflwang1:  makes sense?11:04
flwang1strigazi: cool, i will give it a try11:05
flwang1strigazi: btw, we can host the k8s-keystone-auth image under openstackmagnum?11:05
flwang1or you prefer to host it somewhere?11:06
strigaziflwang1: I need to go, we can continue later or tmr I guess wuth more time? Of course we can11:06
flwang1else11:06
flwang1strigazi: ok, cool11:06
flwang1ttyl11:06
flwang1thanks a lot for all the support11:06
strigazigood night11:06
flwang1have a nice day11:06
strigaziAnytime!11:06
strigaziThanks11:06
*** sfilatov has quit IRC11:06
*** sfilatov has joined #openstack-containers11:08
*** sfilatov has quit IRC11:09
*** sfilatov has joined #openstack-containers11:09
*** jianhua has quit IRC11:13
*** ktibi has quit IRC11:13
*** jianhua has joined #openstack-containers11:13
*** trinaths has quit IRC11:14
*** armaan has quit IRC11:26
*** armaan has joined #openstack-containers11:26
*** armaan has quit IRC11:33
*** armaan has joined #openstack-containers11:33
*** ktibi has joined #openstack-containers11:43
*** jianhua has quit IRC11:51
*** yamamoto has joined #openstack-containers12:14
magoHi team, do you know if Queens/Magnum can be working with Pike/Nova, Pike/Neutron ... Pike/Heat ? Any limitations, or "fancy" features that could be not working ?12:24
strigazimago it works12:24
magothanks strigazi12:24
*** portdirect has quit IRC12:36
*** portdirect has joined #openstack-containers12:36
zhubingbing_strigazi Queens/Magnum don't work12:56
zhubingbing_It is  timeout create heat source12:57
strigazizhubingbing_: openstack coe cluster show <your cluster> | grep stack_id12:59
strigazizhubingbing_: openstack stack resource list -n 2 <the stack_id>13:00
zhubingbing_root@zbb:/var/lib/docker/volumes/kolla_logs/_data# openstack stack13:04
zhubingbing_openstack: 'stack' is not an openstack command. See 'openstack --help'.13:04
zhubingbing_Did you mean one of these?13:04
zhubingbing_  backup create13:04
zhubingbing_  backup delete13:04
zhubingbing_  backup list13:04
zhubingbing_  backup restore13:04
zhubingbing_  backup show13:04
zhubingbing_I don't have openstack stack command13:04
*** mvpnitesh has quit IRC13:05
zhubingbing_@strigazi http://paste.openstack.org/show/719819/13:08
zhubingbing_Please look it13:08
zhubingbing_;)13:08
*** sfilatov has quit IRC13:09
*** armaan has quit IRC13:12
strigazizhubingbing_: have you set openstack_ca_file if you use tls protected openstack endpoints?13:13
*** armaan has joined #openstack-containers13:13
strigazizhubingbing_: it looks like the agent can't access the heat api, maybe other scripts failed too.13:13
zhubingbing_ i don't set openstack_ca_file13:13
strigazizhubingbing_: check if the openstack api are reachable from the vms13:14
strigazizhubingbing_: and if the endpoints are https13:14
strigazizhubingbing_: you need to pass openstack_ca_file in magnum.conf13:14
zhubingbing_root@zbb:/var/lib/docker/volumes/kolla_logs/_data# openstack endpoint show 968eec6c71f74217a94624910152024c13:14
zhubingbing_+--------------+----------------------------------+13:14
zhubingbing_| Field        | Value                            |13:14
zhubingbing_+--------------+----------------------------------+13:14
zhubingbing_| enabled      | True                             |13:14
zhubingbing_| id           | 968eec6c71f74217a94624910152024c |13:14
zhubingbing_| interface    | internal                         |13:15
zhubingbing_| region       | RegionOne                        |13:15
zhubingbing_| region_id    | RegionOne                        |13:15
zhubingbing_| service_id   | f7eb674da47c47749746babf57a9e002 |13:15
zhubingbing_| service_name | magnum                           |13:15
zhubingbing_| service_type | container-infra                  |13:15
zhubingbing_| url          | http://172.16.130.150:9511/v113:15
zhubingbing_I use kolla to deploy magnum13:16
zhubingbing_;(13:16
*** dave-mccowan has joined #openstack-containers13:16
*** sfilatov has joined #openstack-containers13:19
*** armaan has quit IRC13:20
*** armaan has joined #openstack-containers13:21
*** dave-mccowan has quit IRC13:21
*** dave-mccowan has joined #openstack-containers13:22
*** sfilatov has joined #openstack-containers13:25
*** jmlowe has quit IRC13:27
*** yamamoto has quit IRC13:28
*** yamamoto has joined #openstack-containers13:28
*** armaan has quit IRC13:32
*** armaan has joined #openstack-containers13:32
strigazizhubingbing_: can yu login in a cluster node and do curl http://172.16.130.150:9511/v1 ?13:37
zhubingbing_yeah13:37
zhubingbing_root@zbb:/var/lib/docker/volumes/kolla_logs/_data# curl http://172.16.130.150:9511/v113:38
zhubingbing_{"media_types": [{"base": "application/json", "type": "application/vnd.openstack.magnum.v1+json"}], "stats": [{"href": "http://172.16.130.150:9511/v1/stats/", "rel": "self"}, {"href": "http://172.16.130.150:9511/stats/", "rel": "bookmark"}], "links": [{"href": "http://172.16.130.150:9511/v1/", "rel": "self"}, {"href": "http://docs.openstack.org/developer/magnum/dev/api-spec-v1.html", "type": "text/html", "rel": "describedby"}], "quotas13:38
zhubingbing_": [{"href": "http://172.16.130.150:9511/v1/quotas/", "rel": "self"}, {"href": "http://172.16.130.150:9511/quotas/", "rel": "bookmark"}], "mservices": [{"href": "http://172.16.130.150:9511/v1/mservices/", "rel": "self"}, {"href": "http://172.16.130.150:9511/mservices/", "rel": "bookmark"}], "bays": [{"href": "http://172.16.130.150:9511/v1/bays/", "rel": "self"}, {"href": "http://172.16.130.150:9511/bays/", "rel": "bookmark"}], "federat13:38
zhubingbing_ions": [{"href": "http://172.16.130.150:9511/v1/federations/", "rel": "self"}, {"href": "http://172.16.130.150:9511/federations/", "rel": "bookmark"}], "clustertemplates": [{"href": "http://172.16.130.150:9511/v1/clustertemplates/", "rel": "self"}, {"href": "http://172.16.130.150:9511/clustertemplates/", "rel": "bookmark"}], "certificates": [{"href": "http://172.16.130.150:9511/v1/certificates/", "rel": "self"}, {"href": "http://172.1613:38
zhubingbing_.130.150:9511/certificates/", "rel": "bookmark"}], "clusters": [{"href": "http://172.16.130.150:9511/v1/clusters/", "rel": "self"}, {"href": "http://172.16.130.150:9511/clusters/", "rel": "bookmark"}], "baymodels": [{"href": "http://172.16.130.150:9511/v1/baymodels/", "rel": "self"}, {"href": "http://172.16.130.150:9511/baymodels/", "rel": "bookmark"}], "id": "v1"}13:38
zhubingbing_http://paste.openstack.org/show/719821/13:41
zhubingbing_@strigazi  I set to pass openstack_ca_file in magnum.conf but it still timeout13:41
mnasiadkazhubingbing_: login to the k8s master in your deployment and check the cloud init output13:42
zhubingbing_ok13:42
zhubingbing_check the cloud init output  I should how to run which command13:43
zhubingbing_;)13:43
zhubingbing_Cloud-init v. 0.7.9 running 'modules:config' at Tue, 24 Apr 2018 13:30:47 +0000. Up 23.27 seconds.13:46
zhubingbing_+ CA_FILE=/etc/pki/ca-trust/source/anchors/openstack-ca.pem13:46
zhubingbing_+ '[' -n '' ']'13:46
zhubingbing_Getting image source signatures13:46
zhubingbing_Copying blob sha256:7ecc8a985be6848a7ee66b80217c168de6e79dcbda3c01ee36a45367c3a3a6bc13:46
zhubingbing_ 86.23 MB / 86.23 MB  6m21s13:46
zhubingbing_Copying blob sha256:a6091bfb179ad957897301750adebbb3dc821e0aff7e2695e1e87128f91d705c13:46
zhubingbing_ 14.82 MB / 14.82 MB  1m2s13:46
zhubingbing_Copying blob sha256:a6220ca8d39b93c71aa73bc81d83160f4c414f1ca6688b79a4e343a0f1ec597313:46
zhubingbing_ 372 B / 372 B  0s13:46
zhubingbing_Copying blob sha256:76cd47524e799b1197a4a641ec9fe2c102265d5f188cacadf07bfda2bee5da0d13:47
zhubingbing_time="2018-04-24T13:39:35Z" level=fatal msg="Error reading blob sha256:76cd47524e799b1197a4a641ec9fe2c102265d5f188cacadf07bfda2bee5da0d: Get https://registry-1.docker.io/v2/openstackmagnum/etcd/blobs/sha256:76cd47524e799b1197a4a641ec9fe2c102265d5f188cacadf07bfda2bee5da0d: net/http: TLS handshake timeout"13:47
zhubingbing_@mnasiadka13:47
zhubingbing_;)13:47
zhubingbing_I set not user openstack_ca_file , but it is still use it13:47
mnasiadkaare you sure your internet access is working properly? :D13:48
*** zul has quit IRC13:48
*** zul has joined #openstack-containers13:48
*** armaan has quit IRC13:51
*** armaan has joined #openstack-containers13:51
strigazizhubingbing_: you can pull all the containers locally and use the container_infra_prefix label13:53
zhubingbing_I try do it13:53
*** sfilatov has quit IRC13:55
*** sfilatov_ has joined #openstack-containers13:58
*** hongbin_ has joined #openstack-containers13:59
zhubingbing_[root@ping-ynnpbv5uixj3-master-0 docker]# docker pull calico/kube-controllers:v1.0.314:00
zhubingbing_Trying to pull repository docker.io/calico/kube-controllers ...14:00
zhubingbing_sha256:0f22f00965e6406373c45175a3f130065b693c09cb4681c3d255acb51c6263f3: Pulling from docker.io/calico/kube-controllers14:00
zhubingbing_550fe1bea624: Pull complete14:00
zhubingbing_15ea80db3836: Pulling fs layer14:00
zhubingbing_It is look good14:01
*** sfilatov_ has quit IRC14:01
*** sfilatov has joined #openstack-containers14:01
*** AlexeyAbashkin has quit IRC14:25
*** yolanda_ has joined #openstack-containers14:55
*** yolanda has quit IRC14:58
*** yamamoto has quit IRC14:58
*** sfilatov has quit IRC15:00
*** sfilatov has joined #openstack-containers15:01
*** gsimondo1 has quit IRC15:06
*** AlexeyAbashkin has joined #openstack-containers15:08
*** armaan has quit IRC15:14
*** armaan has joined #openstack-containers15:14
*** spsurya has quit IRC15:31
*** spsurya has joined #openstack-containers15:32
*** pcaruana has quit IRC15:51
*** trinaths has joined #openstack-containers15:52
*** jmlowe has joined #openstack-containers15:55
*** jmlowe has quit IRC15:59
*** yamamoto has joined #openstack-containers15:59
*** gsimondon has joined #openstack-containers16:01
*** iranzo has quit IRC16:02
*** armaan has quit IRC16:04
*** yamamoto has quit IRC16:04
*** armaan has joined #openstack-containers16:05
*** ramishra has quit IRC16:05
*** armaan_ has joined #openstack-containers16:07
*** armaan has quit IRC16:08
*** armaan_ has quit IRC16:09
*** armaan has joined #openstack-containers16:12
*** armaan has quit IRC16:13
*** mgoddard has quit IRC16:30
*** jchhatbar has quit IRC16:32
*** sfilatov has quit IRC16:37
*** sfilatov has joined #openstack-containers16:38
*** sfilatov has quit IRC16:43
*** pcaruana has joined #openstack-containers16:43
*** armaan has joined #openstack-containers16:46
*** trinaths has quit IRC16:54
*** itlinux has joined #openstack-containers16:57
*** fragatina has quit IRC16:59
*** yamamoto has joined #openstack-containers17:00
*** mgoddard has joined #openstack-containers17:01
*** AlexeyAbashkin has quit IRC17:01
*** fragatina has joined #openstack-containers17:03
*** yamamoto has quit IRC17:06
*** fragatina has quit IRC17:08
*** fragatina has joined #openstack-containers17:11
*** dsariel has quit IRC17:13
*** udesale has joined #openstack-containers17:15
*** fragatina has quit IRC17:17
*** mdnadeem has quit IRC17:26
*** mago has quit IRC17:28
*** armaan has quit IRC17:33
*** lpetrut_ has joined #openstack-containers17:34
*** gsimondon has quit IRC17:36
*** fragatina has joined #openstack-containers17:37
*** lpetrut has quit IRC17:38
*** gsimondon has joined #openstack-containers17:52
*** udesale has quit IRC17:57
*** gsimondon has quit IRC17:57
*** yamamoto has joined #openstack-containers18:03
*** yamamoto has quit IRC18:08
*** lpetrut_ has quit IRC18:17
*** ianychoi has quit IRC18:26
*** ianychoi has joined #openstack-containers18:26
*** pcaruana has quit IRC18:32
*** mgoddard has quit IRC18:33
*** iranzo has joined #openstack-containers18:39
*** iranzo has joined #openstack-containers18:39
*** dims has quit IRC19:02
*** yamamoto has joined #openstack-containers19:04
*** lpetrut_ has joined #openstack-containers19:04
*** dims has joined #openstack-containers19:09
*** gsimondon has joined #openstack-containers19:09
*** yamamoto has quit IRC19:10
*** ktibi has quit IRC19:14
*** yolanda__ has joined #openstack-containers19:15
*** yolanda_ has quit IRC19:18
*** ianychoi has quit IRC19:39
*** iranzo has quit IRC19:39
*** ianychoi has joined #openstack-containers19:41
*** fragatina has quit IRC20:01
*** fragatina has joined #openstack-containers20:02
*** ianychoi has quit IRC20:05
*** yamamoto has joined #openstack-containers20:06
*** lpetrut_ has quit IRC20:07
*** ianychoi has joined #openstack-containers20:09
*** yamamoto has quit IRC20:11
*** itlinux has quit IRC20:13
*** itlinux has joined #openstack-containers20:19
*** fragatina has quit IRC20:22
*** fragatina has joined #openstack-containers20:22
*** ianychoi has quit IRC20:39
*** ianychoi has joined #openstack-containers20:40
*** jmlowe has joined #openstack-containers20:50
*** jmlowe has quit IRC20:52
*** yamamoto has joined #openstack-containers21:08
*** lxkong has joined #openstack-containers21:09
*** itlinux has quit IRC21:12
*** yamamoto has quit IRC21:14
*** flwang1 has quit IRC21:16
*** gsimondo1 has joined #openstack-containers21:20
*** gsimondon has quit IRC21:21
*** ianychoi has quit IRC21:24
*** ianychoi has joined #openstack-containers21:25
*** gsimondon has joined #openstack-containers21:25
*** gsimondo1 has quit IRC21:26
*** ianychoi has quit IRC21:35
*** ianychoi has joined #openstack-containers21:36
*** ianychoi has quit IRC22:02
*** ianychoi has joined #openstack-containers22:04
*** yamamoto has joined #openstack-containers22:09
*** flwang1 has joined #openstack-containers22:12
*** yamamoto has quit IRC22:14
*** dave-mccowan has quit IRC22:15
*** gsimondon has quit IRC22:26
*** gsimondon has joined #openstack-containers22:34
*** gsimondon has quit IRC22:39
*** yamamoto has joined #openstack-containers23:10
*** yamamoto has quit IRC23:16
*** hongbin_ has quit IRC23:33
*** yamamoto has joined #openstack-containers23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!