| *** mrodriguez has quit IRC | 00:03 | |
| *** itlinux has joined #openstack-containers | 00:21 | |
| *** Florian has joined #openstack-containers | 00:33 | |
| *** sdake has joined #openstack-containers | 00:58 | |
| *** hongbin has joined #openstack-containers | 01:07 | |
| *** dave-mccowan has joined #openstack-containers | 01:07 | |
| *** _fragatina has quit IRC | 01:21 | |
| *** itlinux has quit IRC | 01:28 | |
| *** itlinux has joined #openstack-containers | 02:09 | |
| *** sdake has quit IRC | 02:36 | |
| *** openstackgerrit has joined #openstack-containers | 02:49 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: Mount /usr/libexec/kubernetes/ for FlexVolumes https://review.openstack.org/638064 | 02:49 |
|---|---|---|
| *** ricolin has joined #openstack-containers | 03:01 | |
| openstackgerrit | Lingxian Kong proposed openstack/magnum master: [k8s-fedora-atomic] Security group definition for worker nodes https://review.openstack.org/638069 | 03:33 |
| *** itlinux has quit IRC | 03:45 | |
| *** udesale has joined #openstack-containers | 03:56 | |
| *** itlinux has joined #openstack-containers | 04:03 | |
| *** ykarel|away has joined #openstack-containers | 04:08 | |
| *** ykarel|away is now known as ykarel | 04:09 | |
| *** itlinux has quit IRC | 04:10 | |
| *** itlinux has joined #openstack-containers | 04:12 | |
| *** itlinux has quit IRC | 04:18 | |
| *** ricolin has quit IRC | 04:18 | |
| *** janki has joined #openstack-containers | 04:22 | |
| *** itlinux has joined #openstack-containers | 04:23 | |
| *** ramishra has joined #openstack-containers | 04:23 | |
| *** itlinux has quit IRC | 04:28 | |
| *** ricolin has joined #openstack-containers | 04:30 | |
| *** dave-mccowan has quit IRC | 04:31 | |
| *** itlinux has joined #openstack-containers | 04:32 | |
| openstackgerrit | Jake Yip proposed openstack/python-magnumclient master: python3 fixes https://review.openstack.org/638077 | 04:40 |
| *** ykarel has quit IRC | 04:48 | |
| eandersson | jakeyip, actually not sure if my comment is right | 04:48 |
| eandersson | but I think it is | 04:49 |
| jakeyip | you're fast @eandersson :P | 04:53 |
| jakeyip | I'm still testing it out actually | 04:53 |
| eandersson | I tested it locally, and with encode_as_bytes it looks good | 04:54 |
| jakeyip | no text is correct | 04:54 |
| eandersson | *meant text hah :D | 04:54 |
| eandersson | because bytes generated invalid text | 04:54 |
| eandersson | My VM is being stupid and not allowing me to copy and paste | 04:55 |
| jakeyip | the client-certificate-data and client-key-data will be different each time I run `openstack coe cluster config <cluster>` ? Cos I'm seeing some difference | 04:56 |
| eandersson | I am gonna go play some games. It's 9PM over here. :D | 04:56 |
| eandersson | but let me know jakeyip when it's ready for review | 04:56 |
| jakeyip | sure | 04:56 |
| jakeyip | some... blizzard games? : | 04:57 |
| eandersson | Of course =] | 04:57 |
| *** hongbin has quit IRC | 05:04 | |
| *** ykarel has joined #openstack-containers | 05:05 | |
| openstackgerrit | Jake Yip proposed openstack/python-magnumclient master: python3 fixes https://review.openstack.org/638077 | 05:08 |
| *** itlinux_ has joined #openstack-containers | 05:24 | |
| *** itlinux has quit IRC | 05:27 | |
| *** ArchiFleKs has quit IRC | 05:28 | |
| *** ArchiFleKs has joined #openstack-containers | 05:35 | |
| *** itlinux has joined #openstack-containers | 05:40 | |
| *** itlinux_ has quit IRC | 05:43 | |
| *** _fragatina has joined #openstack-containers | 05:44 | |
| *** _fragatina has quit IRC | 05:45 | |
| *** _fragatina has joined #openstack-containers | 05:46 | |
| *** itlinux has quit IRC | 05:56 | |
| *** itlinux has joined #openstack-containers | 06:23 | |
| *** sdake has joined #openstack-containers | 06:25 | |
| *** sdake has quit IRC | 06:27 | |
| *** itlinux has quit IRC | 06:28 | |
| *** itlinux has joined #openstack-containers | 06:32 | |
| *** itlinux has quit IRC | 06:50 | |
| *** _fragatina has quit IRC | 06:53 | |
| *** _fragatina has joined #openstack-containers | 06:54 | |
| *** _fragatina has quit IRC | 07:00 | |
| *** itlinux has joined #openstack-containers | 07:03 | |
| *** itlinux has quit IRC | 07:07 | |
| *** itlinux has joined #openstack-containers | 07:26 | |
| *** udesale has quit IRC | 07:34 | |
| *** udesale has joined #openstack-containers | 07:35 | |
| *** itlinux has quit IRC | 07:36 | |
| *** ykarel is now known as ykarel|lunch | 07:49 | |
| *** itlinux has joined #openstack-containers | 07:56 | |
| *** pcaruana has joined #openstack-containers | 07:59 | |
| *** itlinux has quit IRC | 08:01 | |
| *** jaewook_oh has joined #openstack-containers | 08:02 | |
| *** itlinux has joined #openstack-containers | 08:03 | |
| *** jchhatbar has joined #openstack-containers | 08:09 | |
| *** jchhatbar has quit IRC | 08:10 | |
| *** jchhatbar has joined #openstack-containers | 08:11 | |
| *** janki has quit IRC | 08:11 | |
| *** itlinux has quit IRC | 08:16 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum stable/rocky: [k8s] Update cluster health status by native API https://review.openstack.org/638102 | 08:17 |
| *** flwang1 has joined #openstack-containers | 08:20 | |
| *** ttsiouts has joined #openstack-containers | 08:22 | |
| *** mgoerens has joined #openstack-containers | 08:25 | |
| *** rcernin has quit IRC | 08:29 | |
| openstackgerrit | Ricardo Rocha proposed openstack/magnum master: [k8s] Add trustee as a secret in kube-system https://review.openstack.org/636725 | 08:45 |
| *** ykarel|lunch is now known as ykarel | 08:51 | |
| *** ricolin has quit IRC | 08:55 | |
| *** ykarel is now known as ykarel|afk | 09:13 | |
| *** ykarel|afk is now known as ykarel | 09:17 | |
| openstackgerrit | Feilong Wang proposed openstack/magnum-specs master: Add auto healing spec https://review.openstack.org/638123 | 10:14 |
| openstackgerrit | Feilong Wang proposed openstack/magnum-specs master: Add auto healing spec https://review.openstack.org/638123 | 10:18 |
| openstackgerrit | Lingxian Kong proposed openstack/magnum master: [k8s-fedora-atomic] Remove NodePort type service https://review.openstack.org/638125 | 10:26 |
| *** Florian has quit IRC | 10:39 | |
| *** ianychoi has quit IRC | 10:59 | |
| openstackgerrit | Merged openstack/magnum master: [k8s] Add trustee as a secret in kube-system https://review.openstack.org/636725 | 11:08 |
| *** ttsiouts has quit IRC | 11:13 | |
| *** ttsiouts has joined #openstack-containers | 11:13 | |
| *** udesale has quit IRC | 11:14 | |
| *** ttsiouts has quit IRC | 11:18 | |
| *** sapd1 has quit IRC | 11:38 | |
| *** sapd1 has joined #openstack-containers | 11:55 | |
| *** ttsiouts has joined #openstack-containers | 11:59 | |
| Horrorcat | lots of delay, but: strigazi your cluster_user_trust suggestion worked! I have a working F27 cluster on top of Magnum Rocky. Thank you very much. | 12:09 |
| *** sapd1 has quit IRC | 12:10 | |
| strigazi | Horrorcat cheers! | 12:11 |
| *** janki has joined #openstack-containers | 12:14 | |
| *** jchhatbar has quit IRC | 12:15 | |
| *** sapd1 has joined #openstack-containers | 12:22 | |
| Horrorcat | is there a way to configure which version of kubernetes will be deployed? I see that it’s shown in `openstack coe cluster show`, but AFAICT it’s not configurable in `openstack coe cluster template set`, nor did I find anything in the configuration (by searching for "k8s", "version" or "coe" in <https://docs.openstack.org/magnum/latest/configuration/sample-config.html> at least) | 12:29 |
| Horrorcat | aha! --labels kube_tag=... | 12:35 |
| *** mgoerens has quit IRC | 12:36 | |
| *** mgoerens has joined #openstack-containers | 12:36 | |
| *** ArchiFleKs has quit IRC | 12:40 | |
| *** ArchiFleKs has joined #openstack-containers | 12:45 | |
| *** ivve has joined #openstack-containers | 13:10 | |
| *** jaewook_oh has quit IRC | 13:12 | |
| *** ianychoi has joined #openstack-containers | 13:22 | |
| *** spsurya has joined #openstack-containers | 13:44 | |
| *** jmlowe has quit IRC | 13:47 | |
| *** mgoerens has quit IRC | 13:48 | |
| *** mgoerens has joined #openstack-containers | 13:49 | |
| *** jchhatbar has joined #openstack-containers | 13:52 | |
| *** jchhatbar has quit IRC | 13:53 | |
| *** jchhatbar has joined #openstack-containers | 13:54 | |
| *** janki has quit IRC | 13:54 | |
| *** ykarel is now known as ykarel|pto | 14:02 | |
| *** alisanhaji has joined #openstack-containers | 14:09 | |
| *** mrodriguez has joined #openstack-containers | 14:10 | |
| *** sdake has joined #openstack-containers | 14:10 | |
| alisanhaji | exit | 14:12 |
| alisanhaji | oups wrong window... | 14:12 |
| *** alisanhaji has quit IRC | 14:13 | |
| *** _fragatina has joined #openstack-containers | 14:13 | |
| *** ttsiouts has quit IRC | 14:15 | |
| *** dave-mccowan has joined #openstack-containers | 14:15 | |
| *** ttsiouts has joined #openstack-containers | 14:15 | |
| *** sdake has quit IRC | 14:18 | |
| *** alisanhaji has joined #openstack-containers | 14:18 | |
| *** mgoerens has quit IRC | 14:28 | |
| *** ykarel|pto has quit IRC | 14:29 | |
| *** jmlowe has joined #openstack-containers | 14:30 | |
| henriqueof_ | Have anyone seem this erro on the nodes: runc[2760]: publicURL endpoint for orchestration service in null region not found? | 14:32 |
| *** sdake has joined #openstack-containers | 14:33 | |
| strigazi | flwang1: flwang ^^ | 14:34 |
| *** mgoerens has joined #openstack-containers | 14:34 | |
| henriqueof_ | strigazi: Thanks! Sorry, I didn't read the messages before | 14:50 |
| *** ykarel|pto has joined #openstack-containers | 14:50 | |
| *** sdake has quit IRC | 14:58 | |
| henriqueof_ | I think I just solved by adding 'region_name_for_services = RegionOne' on DEFAULT section of heat config. | 14:58 |
| strigazi | jakeyip: can you have a look: https://storyboard.openstack.org/#!/story/2005057 | 15:04 |
| strigazi | brtknr: would this help? adding 'region_name_for_services = RegionOne' on DEFAULT | 15:16 |
| *** jchhatbar has quit IRC | 15:19 | |
| *** sapd1_x has joined #openstack-containers | 15:23 | |
| *** sapd1 has quit IRC | 15:23 | |
| *** ivve has quit IRC | 15:52 | |
| *** ttsiouts has quit IRC | 15:58 | |
| *** ttsiouts has joined #openstack-containers | 15:59 | |
| *** ttsiouts has quit IRC | 16:03 | |
| *** ttsiouts has joined #openstack-containers | 16:05 | |
| *** ttsiouts has quit IRC | 16:33 | |
| *** ttsiouts has joined #openstack-containers | 16:33 | |
| *** ttsiouts has quit IRC | 16:38 | |
| *** hongbin has joined #openstack-containers | 16:38 | |
| *** mgoerens has quit IRC | 16:46 | |
| *** Florian has joined #openstack-containers | 16:56 | |
| *** ramishra has quit IRC | 17:12 | |
| *** pcaruana has quit IRC | 17:13 | |
| *** _fragatina has quit IRC | 17:14 | |
| *** mgoerens has joined #openstack-containers | 17:19 | |
| openstackgerrit | Diogo Guerra proposed openstack/magnum master: Install prometheus monitoring with helm https://review.openstack.org/629577 | 17:22 |
| *** mgoerens has quit IRC | 17:28 | |
| henriqueof_ | The publicURL problem is fixed, now magum can't create worked nodes, it just says 'No valid host was found' can't find more details in the logs, I am using a kolla based deployment btw, does anyone have any clues. | 17:31 |
| strigazi | henriqueof_: nova compute down? | 17:31 |
| flwang1 | strigazi: can you revisit https://review.openstack.org/#/c/623092/ ? | 17:33 |
| flwang1 | henriqueof_: for no valid host error, better check your nova scheduler log | 17:33 |
| henriqueof_ | strigazi: I tested and successfully created a instance with same image and flavor, so probably not a nova error. The message I got from heat engine logs, you can see it here: https://pastebin.com/aQLDBfdf | 17:36 |
| henriqueof_ | nova-scheduler says: Got no allocation candidates from the Placement API. This could be due to insufficient resources or a temporary occurrence as compute nodes start up. | 17:37 |
| *** flwang1 has quit IRC | 17:48 | |
| *** henriqueof has joined #openstack-containers | 17:53 | |
| *** henriqueof_ has quit IRC | 17:54 | |
| *** sdake has joined #openstack-containers | 18:04 | |
| *** _fragatina has joined #openstack-containers | 18:12 | |
| *** ykarel|pto has quit IRC | 18:22 | |
| strigazi | https://review.openstack.org/#/c/623092/ does it really make sense to have both contexts in the file? A big plus of keystone is to not leak the admin certificate | 18:22 |
| strigazi | flwang: ^^ | 18:22 |
| *** hongbin has quit IRC | 18:33 | |
| henriqueof | Does magnum require swift? The only difference between master and worked node stack is that worker have swift params. I don't have swift installed and the worker nodes fail to create. :/ | 18:33 |
| *** ArchiFleKs has quit IRC | 18:54 | |
| *** sdake has quit IRC | 18:58 | |
| *** ArchiFleKs has joined #openstack-containers | 19:04 | |
| *** sdake has joined #openstack-containers | 19:13 | |
| *** jmlowe has quit IRC | 19:19 | |
| *** hongbin has joined #openstack-containers | 19:23 | |
| flwang | strigazi: so you mean, we should avoid generate the admin cert if user just want to use keystone? | 19:27 |
| flwang | henriqueof: are you enabling container registry in your cluster template? | 19:27 |
| flwang | otherwise, magnum doesn't depend on swift | 19:27 |
| *** jmlowe has joined #openstack-containers | 19:30 | |
| henriqueof | flwang: No, registry is disabled on cluster template. | 19:33 |
| flwang | can you share the log? | 19:33 |
| flwang | without log we can't help much | 19:33 |
| henriqueof | Sure, one moment, I'll recreate the cluster and capture logs. | 19:36 |
| henriqueof | I see errors on nova-scheduler and heat-engine logs. | 19:38 |
| henriqueof | nova-scheduler: https://pastebin.com/y30LuDUz | 19:39 |
| henriqueof | heat-engine: https://pastebin.com/AWuMc0mU | 19:40 |
| henriqueof | I see neutron interfaces are ok, volumes are allocated too so no problems with neutron nor cinder. | 19:41 |
| flwang | firstly, you probably need to check if your nova is working OK | 19:45 |
| flwang | i can't see obvious error from the scheduler log, so firstly, check if all your compute hosts are up | 19:46 |
| flwang | and then try to create an instance to see if it works | 19:46 |
| flwang | to narrow the problem scope | 19:46 |
| flwang | at this stage, i don't think it's related to Magnum | 19:46 |
| henriqueof | I tested nova too, as soon I the error occurs I launched an instance with same image and flavor, it started normally, that's why I think there is some extra resource the kubeminion templateis trying to allocate. | 19:50 |
| *** sdake has quit IRC | 19:50 | |
| flwang | did you check quota? | 19:53 |
| flwang | after you launched your master nodes, are you sure you have enough resources on the host for the minion nodes? | 19:53 |
| henriqueof | Sure, this is a new OS deployment, there are no other nova instances, also if it was quota I wouldn't be able to launch the test instance after the minion failed. | 19:55 |
| henriqueof | I've noticed this message on nova-scheduler log: The instance sync for host 'compute-00' did not match. Re-created its InstanceList. | 20:01 |
| flwang | henriqueof: then i would recommend enable debug for nova scheduler | 20:01 |
| henriqueof | Can it be causing problems? | 20:01 |
| flwang | based on the log you provide, i can't see why it failed to schedule | 20:01 |
| henriqueof | Okay, will retry with debug enabled. | 20:02 |
| flwang | as i mentioned above, first, run openstack hypervisor list to get all the compute host status | 20:02 |
| flwang | then, enable debug for nova scheduler | 20:02 |
| flwang | then monitor the log to catch the root cause | 20:03 |
| *** jmlowe has quit IRC | 20:06 | |
| *** alisanhaji has quit IRC | 20:07 | |
| *** jmlowe has joined #openstack-containers | 20:08 | |
| henriqueof | nova-schedular still show no relevant info about the problem after enabling debug. :/ | 20:39 |
| *** spsurya has quit IRC | 20:42 | |
| henriqueof | I think I found the problem but still don't understand it. | 20:43 |
| openstackgerrit | Lingxian Kong proposed openstack/magnum master: [k8s-fedora-atomic] Remove NodePort type service https://review.openstack.org/638125 | 20:44 |
| flwang | what's the problem | 20:50 |
| henriqueof | I am using LVM on my storage node that happens to be the compute node too, the system have a reserved LV with 40Gb and everything else is a pool for volume LVs, when creating the k8s nodes there is a use of ~20Gb on the system disk, when another machine is being spawned there is no space left and creating is cancelled. | 20:50 |
| henriqueof | The instance volumes shouldn't be using system volume space, I think maybe it is being used by magnum assets maybe? | 20:51 |
| henriqueof | I just confirmed that I can allocate volumes of up to 100Gb without any system volume usage. | 20:52 |
| *** jmlowe has quit IRC | 20:53 | |
| openstackgerrit | Lingxian Kong proposed openstack/magnum master: [k8s-fedora-atomic] Remove NodePort type service https://review.openstack.org/638125 | 20:57 |
| *** ttsiouts has joined #openstack-containers | 21:01 | |
| *** sdake has joined #openstack-containers | 21:11 | |
| *** sdake has quit IRC | 21:38 | |
| *** sdake has joined #openstack-containers | 21:40 | |
| *** sdake has quit IRC | 21:48 | |
| *** sdake has joined #openstack-containers | 21:50 | |
| *** sdake has quit IRC | 21:59 | |
| *** dave-mccowan has quit IRC | 22:01 | |
| *** sdake has joined #openstack-containers | 22:03 | |
| *** rcernin has joined #openstack-containers | 22:27 | |
| *** jmlowe has joined #openstack-containers | 22:47 | |
| *** hongbin has quit IRC | 22:51 | |
| *** ttsiouts has quit IRC | 22:52 | |
| *** rcernin has quit IRC | 23:06 | |
| *** rcernin has joined #openstack-containers | 23:08 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!