| openstackgerrit | Feilong Wang proposed openstack/magnum master: [fedora_atomic] Support auto healing for k8s https://review.openstack.org/631378 | 00:47 |
|---|---|---|
| flwang | guimaluf: for existing cluster? no | 00:56 |
| *** cbrumm_ has joined #openstack-containers | 01:10 | |
| *** cbrumm has quit IRC | 01:12 | |
| *** spotz has quit IRC | 01:12 | |
| *** hongbin has joined #openstack-containers | 01:41 | |
| *** ykarel|away has joined #openstack-containers | 02:06 | |
| *** spsurya has joined #openstack-containers | 02:12 | |
| *** hongbin has quit IRC | 03:00 | |
| *** udesale has joined #openstack-containers | 04:00 | |
| *** flwang1 has quit IRC | 04:02 | |
| *** ramishra has joined #openstack-containers | 04:17 | |
| *** ykarel|away has quit IRC | 04:26 | |
| *** ykarel|away has joined #openstack-containers | 04:42 | |
| *** ykarel|away has quit IRC | 05:18 | |
| *** ykarel|away has joined #openstack-containers | 05:24 | |
| *** janki has joined #openstack-containers | 06:17 | |
| *** pcaruana has joined #openstack-containers | 06:36 | |
| *** pcaruana has quit IRC | 06:38 | |
| *** pcaruana has joined #openstack-containers | 06:38 | |
| *** gsimondon has joined #openstack-containers | 07:27 | |
| *** gsimondo1 has joined #openstack-containers | 07:27 | |
| *** gsimondo2 has joined #openstack-containers | 07:32 | |
| *** gsimondo1 has quit IRC | 07:32 | |
| *** gsimondon has quit IRC | 07:32 | |
| *** yolanda has joined #openstack-containers | 07:33 | |
| *** gsimondon has joined #openstack-containers | 07:37 | |
| *** gsimondo2 has quit IRC | 07:50 | |
| *** gsimondon has quit IRC | 07:52 | |
| *** gsimondo1 has joined #openstack-containers | 07:52 | |
| *** ramishra has quit IRC | 08:07 | |
| *** ttsiouts has joined #openstack-containers | 08:18 | |
| *** ramishra has joined #openstack-containers | 08:19 | |
| *** ykarel|away is now known as ykarel|lunch | 08:30 | |
| *** ykarel|lunch has quit IRC | 08:35 | |
| *** flwang1 has joined #openstack-containers | 08:51 | |
| *** ykarel|lunch has joined #openstack-containers | 09:00 | |
| flwang1 | strigazi: around? | 09:17 |
| strigazi | flwang1: hello | 09:19 |
| flwang1 | strigazi: thanks for showing up ;) | 09:20 |
| flwang1 | did you see my comments on your patch? | 09:20 |
| strigazi | I saw your comments yesterday | 09:20 |
| strigazi | I mean 12 hours ago | 09:21 |
| flwang1 | i tested base os image update later and i got an etcd issue | 09:21 |
| strigazi | did you have a volume for etcd? | 09:21 |
| flwang1 | the master node will be recreated with same IP address and because the ip has been registered, so etcd service can't start | 09:21 |
| flwang1 | ah, i see | 09:22 |
| flwang1 | i don't, i will test it again with a etcd volume | 09:22 |
| flwang1 | cool | 09:22 |
| strigazi | I was testing 1.14.0 | 09:22 |
| strigazi | something changed for cni | 09:22 |
| strigazi | the nodes are not ready | 09:23 |
| strigazi | with the usual error | 09:23 |
| flwang1 | strigazi: so we probably need to upgrade calico? | 09:23 |
| strigazi | can't find networks config | 09:23 |
| strigazi | not sure | 09:23 |
| flwang1 | ok, i will test it later | 09:23 |
| strigazi | flannel AND calico behaved the same | 09:24 |
| flwang1 | did you try to use a newer cni version? | 09:25 |
| strigazi | no | 09:26 |
| flwang1 | thanks for the headsup, i will try a newer cni version. and i'm going to upgrade calico as well | 09:26 |
| strigazi | probably is smth in the kubelet config | 09:27 |
| strigazi | I checking now | 09:27 |
| strigazi | I'll update my patch right after | 09:27 |
| strigazi | did review the NG patches? | 09:28 |
| strigazi | did you review the NG patches? | 09:28 |
| flwang1 | strigazi: i haven't got time today, we had a production pvc issue today | 09:29 |
| flwang1 | i can see you commented https://review.openstack.org/#/c/514960/4/magnum/drivers/k8s_fedora_atomic_v1/templates/kubeminion.yaml@369 | 09:29 |
| flwang1 | the docker registry, is it temporary? or you're going to remove it permanently | 09:29 |
| strigazi | we should | 09:30 |
| strigazi | I don't know if this feature works | 09:30 |
| flwang1 | strigazi: we(catalystcloud) are not using it, but we would like to enable it in the future | 09:31 |
| strigazi | we can keep it | 09:32 |
| strigazi | I don't know if it works | 09:32 |
| strigazi | not sure if it makes sense | 09:32 |
| flwang1 | are you going to do the master upgrade implement in same patch? | 09:33 |
| strigazi | yes | 09:33 |
| flwang1 | btw, i'd suggest when posting new patchset, better make sure the cluster create works firstly, otherwise, we may introduce regression issues | 09:34 |
| flwang1 | i understand this is one of the most tricky features we have to do asap | 09:35 |
| strigazi | ok | 09:35 |
| flwang1 | i will test the k8s version upgrade again | 09:35 |
| flwang1 | and then os image upgrade with etcd volume size | 09:36 |
| flwang1 | is ng5 the last one we have to get it in stein? | 09:37 |
| flwang1 | for NG feature i mean | 09:37 |
| strigazi | it depens | 09:37 |
| strigazi | if we want to add drivers with nodegroups | 09:37 |
| flwang1 | strigazi: probably not, it's too risky for us | 09:38 |
| flwang1 | we're on stein now as i mentioned | 09:38 |
| flwang1 | i'm not very confident, TBH | 09:38 |
| strigazi | where are you basing this? ^^ | 09:39 |
| flwang1 | will the new driver change a lot existing logic? | 09:40 |
| strigazi | the new driver will have a different logic | 09:41 |
| flwang1 | it's not a really isolated feature AFAICS | 09:41 |
| flwang1 | yep, i know | 09:41 |
| strigazi | existing drivers will have the current logic and won't change | 09:41 |
| flwang1 | ok, let's see | 09:41 |
| strigazi | it's not a really isolated feature, can you explain this a bit more? | 09:42 |
| flwang1 | at this moment, i just want to focus on rolling upgrade and auto scaling, as long as we get them in, i will do more testing for NG | 09:42 |
| strigazi | it is an isolated feature and can be done in parallel | 09:42 |
| flwang1 | it's not a really isolated feature, can you explain this a bit more? it's not a feature like adding a label to enable something, the NG feature has changed quite a lot logic | 09:43 |
| flwang1 | and i has already seen some impacts | 09:43 |
| flwang1 | e.g. the very slow cluster deletion | 09:43 |
| strigazi | the 20s delay? | 09:43 |
| flwang1 | much more than 20s | 09:43 |
| strigazi | so based on that you are not confident | 09:44 |
| strigazi | I'm pretty sure is the bloody cloud-provider | 09:44 |
| flwang1 | http://paste.openstack.org/show/748777/ | 09:44 |
| strigazi | I'm pretty sure it is the bloody cloud-provider (the 1.14 issue) | 09:44 |
| flwang1 | strigazi: yep, and i'm not using cloud provider 1.14 | 09:45 |
| strigazi | Is the slow deletion reproducible? | 09:45 |
| flwang1 | for me, yes | 09:46 |
| strigazi | ttsiouts: ^^ | 09:46 |
| flwang1 | strigazi: let's back to the rolling upgrade feature, is there anything i can help now? | 09:49 |
| strigazi | flwang1: we will check the slow deletion | 09:49 |
| strigazi | yes, test the minion upgrade | 09:49 |
| strigazi | not the master | 09:50 |
| flwang1 | yes, that's my job for this whole week | 09:50 |
| flwang1 | strigazi: oh, btw, let's merge the api part for easier testing | 09:50 |
| flwang1 | https://review.openstack.org/#/c/514959/16/magnum/drivers/k8s_fedora_atomic_v1/driver.py@87 | 09:51 |
| strigazi | I'll leave a comment | 09:51 |
| flwang1 | are you happy with the param name? | 09:51 |
| strigazi | yes | 09:51 |
| flwang1 | update_max_batch_size or just max_batch_size? | 09:51 |
| strigazi | update_max_batch_size | 09:52 |
| flwang1 | ok, cool | 09:52 |
| strigazi | for ngs, is there another concern apart from the slow deletion sync? | 09:52 |
| flwang1 | it changed some logic of the api and conductor, overall it looks good for me. but i don't have much time to do a full testing. i'm happy to merge them into master. but when we do cherry pick, i will pay more attention and testing atm | 09:55 |
| flwang1 | i don't want to block this and i really appreciate the great work you guys have done. but give we're public cloud, i just want to be careful since it's such a big feature and a lot of code change | 09:56 |
| flwang1 | s/give/given | 09:56 |
| ttsiouts | flwang1: when you have time I wanted to ask you about the slow deletion | 09:58 |
| flwang1 | ttsiouts: sure | 09:58 |
| ttsiouts | flwang1: I am trying to reproduce but with no luck till now.. do you happen to have the conductor logs maybe? | 10:00 |
| *** sidx64_ has joined #openstack-containers | 10:02 | |
| flwang1 | ttsiouts: i have | 10:06 |
| flwang1 | but i can't do it for you now | 10:06 |
| flwang1 | i'm testing rolling upgrade | 10:06 |
| flwang1 | i will collect the log for you tomorrow | 10:06 |
| ttsiouts | flwang1: great!! thanks a lot!!! | 10:07 |
| flwang1 | ttsiouts: thank you | 10:07 |
| flwang1 | ttsiouts: i run into it again, and i can't see anything wrong for condunctor log | 10:14 |
| *** sidx64_ has quit IRC | 10:18 | |
| *** sidx64 has joined #openstack-containers | 10:20 | |
| *** ykarel|lunch is now known as ykarel | 10:20 | |
| *** ttsiouts has quit IRC | 10:22 | |
| *** ttsiouts has joined #openstack-containers | 10:23 | |
| *** ttsiouts has quit IRC | 10:27 | |
| *** ramishra has quit IRC | 10:56 | |
| *** sidx64 has quit IRC | 11:01 | |
| *** sidx64_ has joined #openstack-containers | 11:02 | |
| *** ramishra has joined #openstack-containers | 11:04 | |
| *** ttsiouts has joined #openstack-containers | 11:08 | |
| *** udesale has quit IRC | 11:26 | |
| *** sidx64_ has quit IRC | 11:32 | |
| *** rcernin has quit IRC | 11:38 | |
| *** ramishra has quit IRC | 12:10 | |
| *** ramishra has joined #openstack-containers | 12:12 | |
| *** janki has quit IRC | 12:38 | |
| *** ricolin_phone has joined #openstack-containers | 12:53 | |
| *** jmlowe has quit IRC | 12:57 | |
| openstackgerrit | Spyros Trigazis proposed openstack/magnum master: WIP: k8s_fedora: Update to kubernetes v1.14.0 https://review.openstack.org/649609 | 13:21 |
| *** ricolin_phone has quit IRC | 13:25 | |
| *** ricolin has joined #openstack-containers | 13:30 | |
| *** ykarel has quit IRC | 13:41 | |
| *** ttsiouts has quit IRC | 13:43 | |
| *** ttsiouts has joined #openstack-containers | 13:45 | |
| *** jmlowe has joined #openstack-containers | 13:49 | |
| *** yolanda has quit IRC | 13:55 | |
| *** udesale has joined #openstack-containers | 13:56 | |
| *** ricolin_phone has joined #openstack-containers | 14:02 | |
| *** ricolin_phone has quit IRC | 14:05 | |
| *** yolanda has joined #openstack-containers | 14:12 | |
| *** ramishra has quit IRC | 14:16 | |
| *** ykarel has joined #openstack-containers | 14:28 | |
| *** ramishra has joined #openstack-containers | 14:32 | |
| *** yolanda has quit IRC | 14:50 | |
| *** ramishra has quit IRC | 14:57 | |
| *** yolanda has joined #openstack-containers | 14:57 | |
| *** goldyfruit has joined #openstack-containers | 15:37 | |
| *** gsimondo1 has quit IRC | 15:37 | |
| *** dioguerra has quit IRC | 15:39 | |
| *** Dobroslaw is now known as Dobroslaw_afk | 15:40 | |
| *** ykarel is now known as ykarel|away | 16:02 | |
| *** udesale has quit IRC | 16:19 | |
| *** cbrumm_ has quit IRC | 16:29 | |
| *** ykarel|away has quit IRC | 16:30 | |
| *** ttsiouts has quit IRC | 16:32 | |
| *** ttsiouts has joined #openstack-containers | 16:33 | |
| *** ttsiouts has quit IRC | 16:37 | |
| *** spsurya has quit IRC | 16:46 | |
| *** cbrumm_ has joined #openstack-containers | 16:50 | |
| *** cbrumm_ has quit IRC | 16:52 | |
| *** cbrumm has joined #openstack-containers | 16:59 | |
| *** spsurya has joined #openstack-containers | 17:27 | |
| NobodyCam | Good Morning Magnum folks, What roles are required for a new user to be able to deploy clusters, I can get things working if I add admin role, but would prefer not to give users that access. can someone point me in the right direction? | 17:45 |
| *** gsimondon has joined #openstack-containers | 17:47 | |
| *** jmlowe has quit IRC | 17:52 | |
| *** jmlowe has joined #openstack-containers | 17:55 | |
| *** ricolin has quit IRC | 18:38 | |
| *** flwang1 has quit IRC | 18:39 | |
| goldyfruit | NobodyCam: just the member role | 18:51 |
| goldyfruit | NobodyCam: https://docs.openstack.org/magnum/latest/install/install-ubuntu.html | 18:52 |
| *** jmlowe has quit IRC | 18:57 | |
| *** sidx64 has joined #openstack-containers | 19:18 | |
| *** jmlowe has joined #openstack-containers | 19:48 | |
| *** sidx64 has quit IRC | 20:01 | |
| NobodyCam | goldyfruit: Thank you! | 20:09 |
| NobodyCam | seems I have a neutron policy issue | 20:10 |
| NobodyCam | http://paste.openstack.org/show/BMst551iZBEhwI3bTTHG/ | 20:10 |
| openstackgerrit | Feilong Wang proposed openstack/magnum master: Add API reference for cluster upgrade https://review.openstack.org/649722 | 20:42 |
| *** rcernin has joined #openstack-containers | 20:45 | |
| *** spsurya has quit IRC | 20:59 | |
| *** pcaruana has quit IRC | 21:06 | |
| *** gsimondon has quit IRC | 21:24 | |
| brtknr | (venv-openstack) ➜ ~ sonobuoy e2e 201904022156_sonobuoy_1b34ade6-8764-448d-81d4-5d8f9c5a89ce.tar.gz | 21:29 |
| brtknr | failed tests: 0 | 21:29 |
| brtknr | flwang: I reran the sonobuoy test as you asked and it passed this time :D | 21:29 |
| *** rcernin has quit IRC | 21:42 | |
| *** goldyfruit has quit IRC | 21:46 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!