*** eandersson has quit IRC | 00:16 | |
*** eandersson_ has joined #openstack-containers | 00:17 | |
*** hongbin has quit IRC | 00:19 | |
*** mrodriguez has quit IRC | 00:36 | |
*** sapd1 has joined #openstack-containers | 00:45 | |
*** hongbin has joined #openstack-containers | 01:55 | |
*** _fragatina has quit IRC | 02:05 | |
openstackgerrit | Feilong Wang proposed openstack/python-magnumclient master: Keystone auth support https://review.openstack.org/623092 | 02:10 |
---|---|---|
*** nwonknu has quit IRC | 02:11 | |
*** nwonknu has joined #openstack-containers | 02:17 | |
flwang | jakeyip: ping | 02:20 |
openstackgerrit | Feilong Wang proposed openstack/python-magnumclient master: Keystone auth support https://review.openstack.org/623092 | 02:47 |
*** mgariepy has quit IRC | 02:53 | |
*** mgariepy has joined #openstack-containers | 02:56 | |
*** sapd1 has quit IRC | 02:57 | |
*** sapd1 has joined #openstack-containers | 03:04 | |
*** sapd1 has quit IRC | 03:14 | |
*** ykarel has joined #openstack-containers | 03:19 | |
*** hongbin has quit IRC | 03:33 | |
*** ramishra has joined #openstack-containers | 03:36 | |
*** udesale has joined #openstack-containers | 03:54 | |
*** sdake has joined #openstack-containers | 04:07 | |
*** sdake has quit IRC | 04:10 | |
*** sdake has joined #openstack-containers | 04:12 | |
*** sdake has quit IRC | 04:13 | |
*** sdake has joined #openstack-containers | 04:14 | |
*** sdake has quit IRC | 04:23 | |
jakeyip | sorry was away, what's up flwang ? | 04:25 |
*** dave-mccowan has quit IRC | 04:44 | |
openstackgerrit | Jake Yip proposed openstack/magnum master: python3 fix: encode content to UTF-8 https://review.openstack.org/638336 | 04:45 |
openstackgerrit | Jake Yip proposed openstack/magnum master: python3 fix: decode binary cert data if encountered https://review.openstack.org/638336 | 04:48 |
*** _fragatina has joined #openstack-containers | 05:12 | |
*** janki has joined #openstack-containers | 05:26 | |
*** ramishra has quit IRC | 05:32 | |
*** ramishra has joined #openstack-containers | 05:38 | |
*** _fragatina has quit IRC | 05:47 | |
*** ivve has joined #openstack-containers | 06:23 | |
*** jchhatbar has joined #openstack-containers | 06:43 | |
*** janki has quit IRC | 06:45 | |
*** ramishra has quit IRC | 07:31 | |
*** ramishra has joined #openstack-containers | 07:32 | |
*** logan- has quit IRC | 07:35 | |
*** logan- has joined #openstack-containers | 07:37 | |
*** _fragatina has joined #openstack-containers | 07:41 | |
*** ykarel is now known as ykarel|lunch | 07:56 | |
*** rcernin has quit IRC | 07:56 | |
*** alisanhaji has joined #openstack-containers | 08:19 | |
*** _fragatina has quit IRC | 08:23 | |
*** serhatd has joined #openstack-containers | 08:31 | |
*** pcaruana has joined #openstack-containers | 08:35 | |
*** ttsiouts has joined #openstack-containers | 08:49 | |
*** _fragatina has joined #openstack-containers | 08:54 | |
*** ign0tus has joined #openstack-containers | 08:59 | |
*** ttsiouts has quit IRC | 09:07 | |
*** ttsiouts has joined #openstack-containers | 09:08 | |
*** ykarel|lunch is now known as ykarel | 09:10 | |
*** ttsiouts has quit IRC | 09:13 | |
*** ttsiouts has joined #openstack-containers | 09:14 | |
*** jchhatbar has quit IRC | 09:16 | |
*** jchhatbar has joined #openstack-containers | 09:16 | |
*** jchhatbar has quit IRC | 09:22 | |
*** jchhatbar has joined #openstack-containers | 09:23 | |
*** FlorianFa has quit IRC | 09:30 | |
*** tbarron has quit IRC | 09:30 | |
*** spiette has quit IRC | 09:35 | |
*** FlorianFa has joined #openstack-containers | 09:35 | |
*** spiette has joined #openstack-containers | 09:37 | |
*** _fragatina has quit IRC | 09:39 | |
*** sapd1 has joined #openstack-containers | 09:49 | |
*** ricolin has joined #openstack-containers | 10:10 | |
*** _fragatina has joined #openstack-containers | 10:39 | |
*** ttsiouts has quit IRC | 11:07 | |
*** ttsiouts has joined #openstack-containers | 11:08 | |
*** ttsiouts has quit IRC | 11:12 | |
*** udesale has quit IRC | 11:29 | |
*** ttsiouts has joined #openstack-containers | 12:08 | |
*** janki has joined #openstack-containers | 12:14 | |
*** jchhatbar has quit IRC | 12:15 | |
*** janki has quit IRC | 12:24 | |
*** janki has joined #openstack-containers | 12:24 | |
*** logan_ has joined #openstack-containers | 12:27 | |
*** logan_ is now known as Guest11647 | 12:28 | |
*** janki has quit IRC | 12:29 | |
*** kaiokmo has quit IRC | 12:30 | |
*** jento has quit IRC | 12:30 | |
*** ArchiFleKs has quit IRC | 12:30 | |
*** logan- has quit IRC | 12:30 | |
*** ramishra has quit IRC | 12:30 | |
*** mgariepy has quit IRC | 12:30 | |
*** nwonknu has quit IRC | 12:30 | |
*** openstackgerrit has quit IRC | 12:30 | |
*** ArchiFleKs has joined #openstack-containers | 12:31 | |
*** Guest11647 is now known as logan- | 12:31 | |
*** nwonknu has joined #openstack-containers | 12:37 | |
*** ramishra_ has joined #openstack-containers | 12:38 | |
*** _fragatina has quit IRC | 12:44 | |
*** henriqueof has joined #openstack-containers | 12:47 | |
*** ttsiouts has quit IRC | 12:51 | |
*** ttsiouts has joined #openstack-containers | 12:51 | |
*** ttsiouts has quit IRC | 12:56 | |
brtknr | strigazi: is there a problem using '\' in passwords inside cloud-config? | 12:58 |
*** ttsiouts has joined #openstack-containers | 13:03 | |
*** sdake has joined #openstack-containers | 13:15 | |
*** mgariepy has joined #openstack-containers | 13:18 | |
brtknr | Dont worry, I figured it out... the password needs to be inside double quotes with escaped \ | 13:23 |
*** dave-mccowan has joined #openstack-containers | 13:54 | |
*** serhatd_ has joined #openstack-containers | 13:58 | |
*** serhatd has quit IRC | 14:00 | |
*** kaiokmo has joined #openstack-containers | 14:12 | |
*** ykarel is now known as ykarel|away | 14:13 | |
*** alisanhaji has quit IRC | 14:17 | |
*** alisanhaji has joined #openstack-containers | 14:19 | |
*** sdake has quit IRC | 14:21 | |
*** ivve has quit IRC | 14:27 | |
*** sdake has joined #openstack-containers | 14:33 | |
*** dave-mccowan has quit IRC | 14:34 | |
*** ykarel|away is now known as ykarel | 14:46 | |
*** sdake has quit IRC | 14:55 | |
*** jento has joined #openstack-containers | 14:58 | |
*** ykarel_ has joined #openstack-containers | 14:59 | |
*** sdake has joined #openstack-containers | 15:00 | |
*** ykarel has quit IRC | 15:00 | |
*** hongbin has joined #openstack-containers | 15:03 | |
*** ttsiouts has quit IRC | 15:13 | |
*** ttsiouts has joined #openstack-containers | 15:13 | |
*** ttsiouts has quit IRC | 15:18 | |
*** ykarel_ is now known as ykarel|away | 15:32 | |
*** rado has joined #openstack-containers | 15:32 | |
*** ttsiouts has joined #openstack-containers | 15:34 | |
*** sapd1 has quit IRC | 15:35 | |
*** rado has quit IRC | 15:38 | |
*** rado has joined #openstack-containers | 15:40 | |
*** rado has quit IRC | 15:45 | |
*** hongbin has quit IRC | 15:48 | |
*** ricolin has quit IRC | 15:52 | |
*** sdake has quit IRC | 15:59 | |
*** sdake has joined #openstack-containers | 16:01 | |
*** sdake has quit IRC | 16:02 | |
*** ign0tus has quit IRC | 16:15 | |
*** sdake has joined #openstack-containers | 16:18 | |
*** belmoreira has quit IRC | 16:20 | |
*** itlinux has joined #openstack-containers | 16:25 | |
*** sdake has quit IRC | 16:25 | |
*** sdake has joined #openstack-containers | 16:27 | |
*** dave-mccowan has joined #openstack-containers | 16:28 | |
*** jmlowe has quit IRC | 16:32 | |
*** pcaruana has quit IRC | 16:32 | |
*** sdake has quit IRC | 16:35 | |
*** sdake has joined #openstack-containers | 16:36 | |
*** sdake has quit IRC | 16:40 | |
*** sdake_ has joined #openstack-containers | 16:41 | |
*** ykarel|away has quit IRC | 16:41 | |
*** ykarel has joined #openstack-containers | 16:42 | |
*** mrodriguez has joined #openstack-containers | 16:45 | |
*** sdake_ has quit IRC | 16:45 | |
*** sdake has joined #openstack-containers | 16:47 | |
*** sdake has quit IRC | 16:50 | |
*** _fragatina has joined #openstack-containers | 16:52 | |
*** sdake has joined #openstack-containers | 16:53 | |
*** ttsiouts has quit IRC | 16:59 | |
*** ttsiouts has joined #openstack-containers | 17:00 | |
*** sdake has quit IRC | 17:00 | |
*** sdake_ has joined #openstack-containers | 17:02 | |
*** ttsiouts has quit IRC | 17:05 | |
*** _fragatina has quit IRC | 17:08 | |
*** itlinux_ has joined #openstack-containers | 17:09 | |
*** _fragatina has joined #openstack-containers | 17:10 | |
*** itlinux has quit IRC | 17:13 | |
*** serhatd_ has quit IRC | 17:28 | |
*** jmlowe has joined #openstack-containers | 17:40 | |
*** flwang1 has joined #openstack-containers | 17:43 | |
flwang1 | brtknr: what's the node-ip for? just the fixed ip of the worker node? | 17:44 |
flwang1 | strigazi: do we have weekly meeting today? | 17:44 |
brtknr | flwang1: yes, to make sure that the kubelet is serving on a fixed ip but it doesnt seem to be working for me | 17:52 |
strigazi | flwang1: We have a meeting today. See you then | 17:54 |
*** kaiokmo has quit IRC | 17:58 | |
flwang1 | brtknr: ok, i have no objection for that argument, but we do need some tests | 18:01 |
flwang1 | strigazi: cool | 18:01 |
*** flwang1 has quit IRC | 18:02 | |
*** itlinux_ has quit IRC | 18:13 | |
*** itlinux has joined #openstack-containers | 18:17 | |
*** sdake_ has quit IRC | 18:30 | |
*** sdake has joined #openstack-containers | 18:31 | |
*** sdake has quit IRC | 18:35 | |
*** sdake has joined #openstack-containers | 18:37 | |
*** sdake has quit IRC | 18:46 | |
*** _fragatina has quit IRC | 18:50 | |
*** sdake has joined #openstack-containers | 18:51 | |
*** sdake has quit IRC | 18:55 | |
*** sdake_ has joined #openstack-containers | 18:56 | |
*** sdake_ has quit IRC | 18:56 | |
*** sdake has joined #openstack-containers | 18:59 | |
*** sdake has quit IRC | 19:00 | |
*** sdake_ has joined #openstack-containers | 19:00 | |
henriqueof | flwang1: When I add the 'ingress_controller=octavia' label to cluster, shouldn't it install and configure the Octavia ingress controller? | 19:01 |
henriqueof | Does anyone know? | 19:03 |
*** henriqueof has quit IRC | 19:04 | |
*** sdake_ has quit IRC | 19:06 | |
*** sdake has joined #openstack-containers | 19:06 | |
*** ramishra_ has quit IRC | 19:09 | |
*** sdake has quit IRC | 19:10 | |
*** sdake has joined #openstack-containers | 19:12 | |
*** itlinux_ has joined #openstack-containers | 19:19 | |
*** nwonknu has quit IRC | 19:20 | |
*** sdake has quit IRC | 19:20 | |
*** itlinux has quit IRC | 19:21 | |
*** sdake_ has joined #openstack-containers | 19:21 | |
*** sdake_ has quit IRC | 19:25 | |
*** sdake has joined #openstack-containers | 19:26 | |
*** nwonknu has joined #openstack-containers | 19:27 | |
*** ykarel has quit IRC | 19:33 | |
*** sdake has quit IRC | 19:35 | |
*** sdake_ has joined #openstack-containers | 19:36 | |
*** henriqueof has joined #openstack-containers | 19:43 | |
*** sdake_ has quit IRC | 19:45 | |
*** sdake has joined #openstack-containers | 19:47 | |
*** itlinux_ has quit IRC | 19:52 | |
*** sdake has quit IRC | 19:55 | |
*** itlinux has joined #openstack-containers | 19:56 | |
*** sdake has joined #openstack-containers | 19:56 | |
*** sdake has quit IRC | 20:03 | |
*** itlinux has quit IRC | 20:16 | |
*** itlinux has joined #openstack-containers | 20:20 | |
*** alisanhaji has quit IRC | 20:22 | |
brtknr | flwang: if i can explain what i mean by it not working for me, i can see that the node-ip is labelled correctly but its not respected | 20:22 |
brtknr | this is what I'm talking about: http://paste.openstack.org/show/746081/ | 20:26 |
brtknr | Notice the discrepancy between Internal IP and 'alpha.kubernetes.io/provided-node-ip' | 20:27 |
*** colby_ has joined #openstack-containers | 20:30 | |
flwang | brtknr: yep, that's interesting | 20:34 |
brtknr | flwang: mind you, this is only an issue when there is more than 1 interface | 20:35 |
flwang | brtknr: i would suggest file a bug/story for tracking | 20:37 |
brtknr | flwang: i am not sure where... i am not certain whether this is a fedora atomic issue or kubernetes issue | 20:38 |
flwang | brtknr: i understand, but a bug will be easy for tracking and get help from others | 20:39 |
flwang | strigazi: seems the /resize api works well, i'm so excited | 20:39 |
brtknr | flwang: what was the silver bullet with resize api? what was the missing piece? | 20:40 |
flwang | brtknr: we need a way to delete particular nodes from the cluster | 20:40 |
flwang | though we can do it with heat api, but we want to do it in magnum for autoscaling/auto healting stuff | 20:41 |
brtknr | is this a precedent to node groups? | 20:41 |
flwang | not really a precedent, but it will support node groups when it's ready | 20:42 |
flwang | i have already put a nodegroup_id as a parameter | 20:42 |
flwang | brtknr: here is the patch https://review.openstack.org/#/c/638572/ | 20:43 |
brtknr | im looking at it now | 20:43 |
flwang | brb | 20:44 |
brtknr | is it only interactable through curl or also magnumclient? | 20:44 |
jakeyip | ui2zx9 | 20:45 |
jakeyip | sorry ignore me | 20:48 |
flwang | jakeyip: that's a good password | 20:48 |
flwang | brtknr: both | 20:48 |
jakeyip | argh | 20:48 |
flwang | but i haven't started the client work | 20:49 |
flwang | jakeyip: are you working on the py3 issue of heath check? | 20:49 |
jakeyip | yeah I've submitted another changeset yesterday. needs to discuss about it | 20:49 |
henriqueof | Does adding 'ingress_controller=octavia' annotation to cluster configures octavia ingress or should I do the steps described here https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-octavia-ingress-controller.md? | 20:49 |
*** jmlowe has quit IRC | 20:49 | |
jakeyip | the issue is I am not sure what format the certs come in, it depends on the backend. so I need to just make them text. does that make sense? | 20:51 |
strigazi | brtknr: let's discuss the ip issue in the meeting? | 20:52 |
jakeyip | wondering if @eandersson_ is around? | 20:53 |
flwang | jakeyip: can you pls share the link? | 20:53 |
schaney | flwang: is there still ongoing discussion on that autoscaler thread about using instance IP vs instance ID as the nodes_to_remove parameter? | 20:53 |
jakeyip | https://review.openstack.org/#/c/638336/3 | 20:53 |
flwang | schaney: i think we have decided to go for instance ID since it's more reliable than IP | 20:54 |
flwang | with the new resize api, we should be able to do auto healing easily | 20:55 |
*** itlinux has quit IRC | 20:59 | |
strigazi | #startmeeting containers | 20:59 |
openstack | Meeting started Tue Feb 26 20:59:26 2019 UTC and is due to finish in 60 minutes. The chair is strigazi. Information about MeetBot at http://wiki.debian.org/MeetBot. | 20:59 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 20:59 |
*** openstack changes topic to " (Meeting topic: containers)" | 20:59 | |
openstack | The meeting name has been set to 'containers' | 20:59 |
strigazi | #topic Roll Call | 20:59 |
*** openstack changes topic to "Roll Call (Meeting topic: containers)" | 20:59 | |
strigazi | o/ | 20:59 |
schaney | o/ | 20:59 |
jakeyip | o/ | 20:59 |
brtknr | o/ | 20:59 |
flwang | o/ | 21:00 |
strigazi | #topic Stories/Tasks | 21:00 |
*** openstack changes topic to "Stories/Tasks (Meeting topic: containers)" | 21:00 | |
strigazi | 1. openstack autoscaler | 21:00 |
strigazi | #link https://github.com/kubernetes/autoscaler/pull/1690 | 21:01 |
strigazi | by far the highest number of comments in the repo | 21:01 |
schaney | was glad to get a chance to review, any thoughts on how far off a merge is? | 21:01 |
flwang | schaney: are you Scott? | 21:02 |
schaney | will we want to wait on the magnum resize API? | 21:02 |
schaney | flwang: yes thats me | 21:02 |
strigazi | I think it is close if we show aggrement to them | 21:02 |
flwang | schaney: welcome to join us | 21:02 |
strigazi | from the openstack and magnum side | 21:02 |
schaney | :) | 21:02 |
strigazi | Form openstack PoV should be ok, | 21:02 |
flwang | schaney: we also need work in gophercloud for the new api, so i'm not sure if we can wait | 21:03 |
*** spsurya has quit IRC | 21:03 | |
flwang | strigazi: ^ | 21:03 |
strigazi | since they (CA maitainers) are ok to have two | 21:03 |
strigazi | What difference does it make to us? | 21:03 |
flwang | strigazi: yep, we can get current one in, and propose the new magnum_manager | 21:03 |
schaney | any thoughts on moving away from the API polling once the magnum implementation is complete? | 21:03 |
flwang | no difference for Magnum | 21:04 |
strigazi | if we agree on the design, implementation and direction | 21:04 |
strigazi | schaney: we can do that too | 21:04 |
schaney | awesome | 21:04 |
strigazi | we can leave that as a third step? | 21:05 |
flwang | strigazi: can we just rename the current pr to openstack_magnum_manager.go | 21:05 |
strigazi | First merge like it is now | 21:05 |
flwang | and refactor it as long as we have the new api | 21:05 |
strigazi | 2nd add resixe api | 21:05 |
strigazi | and then remove polling | 21:05 |
strigazi | schaney: makes sense? | 21:06 |
*** jmlowe has joined #openstack-containers | 21:06 | |
strigazi | all this would happen in this cycle | 21:06 |
schaney | sounds good, it will be easier to start tackling specific areas once it's out there | 21:06 |
flwang | FWIW, i don't mind get current PR in with current status, and as long as the new /resize api ready, we can decide how to do in CA | 21:06 |
strigazi | if there are no more objections with the current implementation, we can push the CA team to merge | 21:07 |
*** itlinux has joined #openstack-containers | 21:07 | |
brtknr | its not so much an objection but how will the cluster API stuff affect this? | 21:08 |
strigazi | Is the ip vs id vs uuid thing clear? | 21:08 |
strigazi | brtknr: not at all | 21:08 |
strigazi | the cluster api will be very different | 21:08 |
schaney | I am not actually clear on how your templates create the IP mapping | 21:08 |
strigazi | like google has two implementations, one for gce and one for gke | 21:08 |
flwang | schaney: are you talking about this https://review.openstack.org/639053 ? | 21:09 |
strigazi | schaney: flwang bare with me for the explanation, also this change needs more comments in the commit message ^^ | 21:09 |
strigazi | in heat a resource group creates a stack with depth two | 21:10 |
strigazi | the first nested stack kube_minions has a ref_map output | 21:10 |
strigazi | which goes like this: | 21:10 |
strigazi | 0: <smth> | 21:10 |
strigazi | 1: <smth> | 21:10 |
strigazi | and so on | 21:10 |
strigazi | These indices are the minion-INDEX numbers | 21:11 |
strigazi | and the indices in the ResourceGroup | 21:11 |
strigazi | A RG supports removal_poliies | 21:12 |
strigazi | A RG supports removal_policies | 21:12 |
strigazi | which means you can pass a list of indices as a param, and heat will remove these resources from the RG | 21:12 |
brtknr | I am not clear on what is using the change made in https://review.openstack.org/639053 atm | 21:13 |
strigazi | additionally, heat will track which indices have removed and won't create them again | 21:13 |
strigazi | brtknr: bare with me | 21:13 |
strigazi | so, | 21:13 |
strigazi | in the first imlementation of removal policies in the k8s templates | 21:14 |
strigazi | the IP was used as an id in this list: | 21:14 |
strigazi | 0: private-ip-1 | 21:14 |
strigazi | 1: private-ip-2 | 21:14 |
strigazi | (or zero based :)) | 21:14 |
strigazi | then it was changes with this commit: | 21:15 |
strigazi | https://github.com/openstack/magnum/commit/3ca2eb30369a00240a92c254c95bea6c7a60fee1 | 21:15 |
strigazi | and the ref_map became like this: | 21:16 |
strigazi | 0: stack_id_of_nested_stack_0_depth_2 | 21:16 |
strigazi | 1: stack_id_of_nested_stack_1_depth_2 | 21:16 |
strigazi | and the above patch broke the removal policy of the resouce group | 21:16 |
strigazi | meaning, if you passed a list of ips to the removal policy after the above patch | 21:17 |
strigazi | heat wouldn't understand to whoch index in the RG that ip belonged too | 21:17 |
strigazi | that is why for flwang and schaney didn't work | 21:18 |
schaney | gotcha | 21:18 |
strigazi | flwang: now proposes a change | 21:18 |
strigazi | to make the ref_map: | 21:18 |
strigazi | 0: nova_server_uuid_0 | 21:18 |
strigazi | 1: nova_server_uuid_1 | 21:18 |
strigazi | you can inspect this map in current cluster like this: | 21:19 |
*** imdigitaljim has joined #openstack-containers | 21:19 | |
colin- | sorry i'm late | 21:19 |
strigazi | openstack stack list --nested | grep <parent_stack_name> | grep kube_minions | 21:19 |
strigazi | and then show the nested stack of depth 1 | 21:20 |
strigazi | you will see the list | 21:20 |
strigazi | you will see the ref_map | 21:20 |
strigazi | eg: | 21:20 |
brtknr | `openstack stack list --nested` is a nice trick! | 21:21 |
brtknr | til | 21:21 |
strigazi | http://paste.openstack.org/show/746304/ | 21:22 |
strigazi | this is with the IP ^^ | 21:22 |
brtknr | ive always done `openstack stack resource list k8s-stack --nested-depth=4 | 21:22 |
eandersson_ | o/ | 21:23 |
strigazi | http://paste.openstack.org/show/746305/ | 21:23 |
strigazi | this is with the stack_id | 21:23 |
imdigitaljim | \o | 21:23 |
strigazi | check uuid b4e8a1ec-0b76-48cb-b486-2a5459ea45d4 | 21:23 |
strigazi | in the ref_map and in the list of stacks | 21:24 |
imdigitaljim | i like the new change to uuid =) | 21:24 |
flwang | imdigitaljim: yep, uuid is more reliable than ip for some cases | 21:24 |
strigazi | after said change, we will see the nova uuid there | 21:24 |
strigazi | so, in heat we can pass either the server uuid or the index | 21:25 |
strigazi | then heat will store the removed ids here: | 21:25 |
strigazi | http://paste.openstack.org/show/746306/ | 21:26 |
strigazi | makes sense? | 21:26 |
jakeyip | sounds good to me | 21:27 |
schaney | yep! the confusion on my end was the "output" relationship to removal policy member | 21:27 |
schaney | and the nested stack vs the resource representing the stack | 21:28 |
schaney | makes sense now though | 21:28 |
strigazi | I spent a full moring with thomas on this | 21:29 |
brtknr | do you need https://review.openstack.org/639053 for resize to work? | 21:29 |
strigazi | brtknr: yes | 21:29 |
strigazi | brtknr: to work by giving nova uuids | 21:30 |
brtknr | as it doesnt seem linked on gerrit as a dependency | 21:30 |
jakeyip | in https://github.com/openstack/magnum/commit/3ca2eb30369a00240a92c254c95bea6c7a60fee1 the name for key is OS::stack_id does that need to change or will that be confusing if we use it for soething else | 21:30 |
strigazi | jakeyip: i don't think we have an option there | 21:31 |
strigazi | it needs to be explained well | 21:31 |
brtknr | yes, probably better to call it nova_uuid? | 21:31 |
flwang | brtknr: because i assume https://review.openstack.org/639053 will be merged very soon | 21:31 |
brtknr | or OS::nova_uuid? | 21:31 |
flwang | but the resize patch may take a bit long, sorry for the cofustion | 21:31 |
strigazi | brtknr: doesn't work nova_uuid or other name | 21:31 |
strigazi | not sure if OS::nova_uuid makes sense | 21:31 |
strigazi | not sure if OS::nova_uuid makes sense to heat | 21:32 |
strigazi | (to me it does) | 21:32 |
brtknr | oh okay! i didnt realise it was a component of heat | 21:32 |
strigazi | brtknr: needs to be double checked | 21:33 |
strigazi | the important part is that the ref_map I mentioned before has the correct content | 21:33 |
brtknr | https://docs.openstack.org/heat/rocky/template_guide/composition.html | 21:33 |
brtknr | sounds like we're stuck with OS::stack_id | 21:33 |
strigazi | yeap | 21:34 |
flwang | we should move and discuss details on the patch | 21:34 |
strigazi | with comments in code, should be ok | 21:34 |
flwang | strigazi: i will update the patch based on above discussion | 21:35 |
strigazi | schaney and colleagues, flwang brtknr we have agreement right? | 21:35 |
brtknr | +1 | 21:35 |
strigazi | imdigitaljim: colin- eandersson_ ^^ | 21:35 |
colin- | on the UID portion? | 21:36 |
schaney | yeah UUID will work well for us | 21:36 |
strigazi | yes | 21:36 |
colin- | lgtm | 21:36 |
imdigitaljim | thanks for the clarity, uuid looks good and works for us | 21:36 |
strigazi | \o/ | 21:36 |
jakeyip | I don't have objections but I would like to read the patch first, I am a bit confused whether stack_id is same as nova_uuid or can you get one from the other | 21:37 |
strigazi | jakeyip: they are different | 21:37 |
strigazi | but that stack_id logically corresponds the a nova_uuid | 21:37 |
strigazi | but that stack_id logically corresponds to that nova_uuid | 21:38 |
jakeyip | if we can dervie nova_uuid from stack_id should we do that instead? | 21:38 |
eandersson_ | sounds good | 21:38 |
strigazi | jakeyip: well it is the other way round | 21:39 |
strigazi | jakeyip: derive the stack_id from the nova_uuid | 21:39 |
imdigitaljim | well the stack contains the nova server | 21:39 |
imdigitaljim | so it makes sense to use stack anywasy | 21:39 |
strigazi | jakeyip: the CA or the user will know which server wants to remove | 21:39 |
jakeyip | I thought the stack will have a nova_server with the uuid | 21:40 |
*** eandersson_ is now known as eandersson | 21:40 | |
strigazi | imdigitaljim: that is correct but the user or the CA will know the uuid | 21:40 |
imdigitaljim | you mean the nova uuid? | 21:40 |
imdigitaljim | strigazi:^ | 21:40 |
strigazi | yes | 21:40 |
strigazi | eg when you do kubectl descibe node <foo> you see the nova_uuid of the server | 21:41 |
imdigitaljim | yeah but its in autoscaler, im missing why the user's knowledge even matters | 21:42 |
strigazi | jakeyip: also to be clear, the nova_uuid won't replace the stack uuid, the stack will still have its uuid | 21:42 |
imdigitaljim | either way, im happy with the approach | 21:42 |
jakeyip | yes. but can whichever code just looks for stackid and the OS::Nova::Server uuid of that stack ? | 21:42 |
imdigitaljim | good choices | 21:42 |
strigazi | imdigitaljim: for example for the resize API, there are cases that a user won't to get rid of a node | 21:43 |
brtknr | oh yeah you're right, under `ProviderID: openstack:///231ba791-91ec-4540-a580-3ef493e36055` | 21:43 |
imdigitaljim | ah fair point | 21:43 |
imdigitaljim | good call | 21:43 |
strigazi | jakeyip: can you image the user frustration? additionally, in the autoscaler | 21:44 |
strigazi | the CA wants to remove nova_server with uuid A. | 21:44 |
strigazi | then the CA needs to call heat and show all nested stacks to find which stack this server belongs too | 21:45 |
schaney | it saves a gnarly reverse lookup =) | 21:46 |
colin- | certificate authority? | 21:46 |
jakeyip | that's just code? I am more worried about hijacking a variable that used to mean one thing and making it mean another | 21:46 |
colin- | sorry not following the thread of convo | 21:46 |
brtknr | colin-: cluster autoscaler? | 21:46 |
strigazi | maybe can be done with a single list? or with an extra map we maintain as stack output | 21:46 |
colin- | oh | 21:46 |
colin- | that might get tricky :) | 21:46 |
imdigitaljim | CAS maybe haha | 21:46 |
strigazi | colin-: I got used to it xD | 21:46 |
flwang | should we move to next topoic? | 21:47 |
flwang | strigazi: i'd like to know the rolling upgrade work and the design of resize/upgrade api | 21:47 |
imdigitaljim | could you include both server_id and stack_id in the output and use as a reference point? | 21:48 |
imdigitaljim | is that a thing? | 21:48 |
strigazi | imdigitaljim: I don't think so | 21:48 |
jakeyip | agree with flwang maybe we discuss this straight after meeting | 21:48 |
flwang | i'm thinking if we should use POST instead of PATCH for actions and if we should follow the actions api design like nova/cinder/etc | 21:48 |
imdigitaljim | would be interesting to test | 21:48 |
imdigitaljim | if the heat update can target either or | 21:48 |
imdigitaljim | just like if ip AND stackid are present | 21:48 |
imdigitaljim | because then it would be trivial problem | 21:49 |
strigazi | it is a map, so I don't think so | 21:49 |
strigazi | flwang: what do you mean? | 21:50 |
jakeyip | flwang: is there a review with this topic ? | 21:50 |
imdigitaljim | strigazi: is the key it looks for OS::stack_id? | 21:50 |
strigazi | resize != upgrade | 21:51 |
imdigitaljim | (im new to some of this convo) | 21:51 |
brtknr | https://storyboard.openstack.org/#!/story/2005054 | 21:51 |
strigazi | imdigitaljim: the key is stack_id | 21:51 |
imdigitaljim | kk | 21:51 |
imdigitaljim | thanks | 21:51 |
strigazi | flwang: can you explain more the PATCH vs POST | 21:52 |
flwang | strigazi: nova/cinder is using api like <id>/action and including action name in the post body | 21:52 |
strigazi | flwang: we can do that | 21:52 |
flwang | so two points, 1. should we use POST instead of PATCH 2. should we follow the post body like nova/cinder, etc | 21:52 |
strigazi | flwang: in practice, we won't see any difference | 21:52 |
*** itlinux has quit IRC | 21:53 | |
strigazi | pointer on 2.? | 21:53 |
flwang | strigazi: i know, but i think we should make openstack like a building, instead of building blocks with different design | 21:53 |
imdigitaljim | well | 21:53 |
imdigitaljim | following a more restful paradigm | 21:53 |
imdigitaljim | patch is practically the only appropriate option | 21:53 |
imdigitaljim | https://fullstack-developer.academy/restful-api-design-post-vs-put-vs-patch/ | 21:53 |
flwang | strigazi: https://developer.openstack.org/api-ref/compute/?expanded=rebuild-server-rebuild-action-detail,resume-suspended-server-resume-action-detail | 21:53 |
imdigitaljim | somethign liket his | 21:53 |
flwang | imdigitaljim: openstack do have some guidelines about api design | 21:54 |
flwang | but the thing i'm discusing is a bit different from the method | 21:54 |
jakeyip | flwang: pardon my ignorance what is the difference between this and PATCH at https://developer.openstack.org/api-ref/container-infrastructure-management/?expanded=update-information-of-cluster-detail#update-information-of-cluster | 21:55 |
flwang | jakeyip: we're going to add new api <cluster id>/actions for upgrade and resize | 21:55 |
strigazi | imdigitaljim: flwang I agree with flwang , we can follow a similar pattern than other projects. | 21:56 |
imdigitaljim | i also agree following similar patterns as other projects | 21:56 |
imdigitaljim | just making sure we understand them =) | 21:56 |
*** ttsiouts has joined #openstack-containers | 21:56 | |
flwang | imdigitaljim: thanks, and yes, we're aware of the http method differences | 21:56 |
jakeyip | flwang: for resize it will be something in addition to original PATCH function ? | 21:57 |
flwang | and here, upgrade/resize are really not normal update for the resource(cluster here) | 21:57 |
strigazi | personally I prefer patch, but for the data model we have, there is no real difference, at least IMO | 21:57 |
brtknr | flwang: although nova seems to use PUT for update rather than PATCH or POST | 21:57 |
flwang | both resize and upgrade cases, we're doing node replacement, delete, add new, etc | 21:57 |
flwang | brtknr: yep, but that's history issue i think | 21:58 |
strigazi | brtknr: also put is user for properties/metadata only | 21:58 |
strigazi | brtknr: also put is used for properties/metadata only | 21:58 |
flwang | when we say PATCH, it's most like a normal partial update for the resource | 21:58 |
flwang | but those actions are really beyond that | 21:59 |
*** henriqueof has quit IRC | 21:59 | |
strigazi | I might add that they are "to infinity and beyond" | 21:59 |
flwang | strigazi: haha, buzz lightyear fans here | 21:59 |
brtknr | hmm i'd vote for PATCH but there is not much precedence in other openstack projects... i wonder why | 22:00 |
jakeyip | I feel POST is good. PUT/PATCH is more restrictive. It's much easier to refactor POST into PATCH/PUT later if it makes sense later, but not the other way round | 22:00 |
jakeyip | since we don't have a concrete idea of how it is going to look like POST let us get on with it for now | 22:01 |
flwang | yep, we can discuss on patch | 22:01 |
flwang | we're running out time | 22:01 |
flwang | strigazi: ^ | 22:01 |
strigazi | yes, | 22:02 |
strigazi | just vert quickly, brtknr can you mention the kubelet/node-ip thing? | 22:02 |
imdigitaljim | post makes sense for these scaling operations | 22:02 |
imdigitaljim | but maybe patch if versions are updated or anything? | 22:02 |
brtknr | strigazi: yes, its been bugging me for weeks, my minion InternalIP keeps flipping between the ip addresses it has been assigned on 3 different interfaces... | 22:03 |
strigazi | I think we can drop the node-ip, since the certs has only one ip | 22:03 |
brtknr | I have a special setup where each node has 3 interfaces, 1 for provider network and 1 high throughput and 1 high latency | 22:03 |
strigazi | I think we can drop the node-ip, since the certificate has only one ip | 22:03 |
brtknr | however assigning node-ip is not working | 22:04 |
colin- | whose poor app gets the high latency card XD? | 22:04 |
brtknr | colin-: low latency :P | 22:04 |
colin- | sorry- understand we're short on time | 22:04 |
*** itlinux has joined #openstack-containers | 22:04 | |
brtknr | i have applied --node-ip arg to kubelet and the ip doesnt stick, the ips still keep changing | 22:05 |
*** rcernin has joined #openstack-containers | 22:05 | |
brtknr | the consequence of this is that pods running on those minions become unavailable for the duration that the ip is on a different interface | 22:05 |
brtknr | my temporary workaround is that the order that kube-apiserver resolves host is Hostname,InternalIP,ExternalIP | 22:06 |
strigazi | brtknr: I thought it might be simpler :) we can discuss it tmr or in storyboard/mailing list? | 22:06 |
imdigitaljim | random question | 22:06 |
brtknr | It was InternalIP,Hostname,ExternalIP | 22:06 |
imdigitaljim | https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ | 22:06 |
imdigitaljim | --address 0.0.0.0 | 22:06 |
imdigitaljim | do you bind a specific address? | 22:06 |
brtknr | imdigitaljim: yes, i bound it to node IP | 22:07 |
imdigitaljim | gotcha | 22:07 |
brtknr | it was already bound to node ip | 22:07 |
brtknr | by default | 22:07 |
imdigitaljim | just curious of how that is all done with multi-interface | 22:07 |
colin- | personally curious how the kube-proxy or similar would handle such a setup and rule/translation enforcement etc | 22:07 |
brtknr | is there any reason why we cant do Hostname,InternalIP,ExternalIP ordering by default | 22:07 |
imdigitaljim | https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ | 22:07 |
imdigitaljim | same with kube-proxy? | 22:08 |
imdigitaljim | do you do stuff here different? | 22:08 |
*** itlinux has quit IRC | 22:08 | |
brtknr | I havent touched kube-proxy settings because I couldnt find it | 22:09 |
*** openstackgerrit has joined #openstack-containers | 22:09 | |
openstackgerrit | Feilong Wang proposed openstack/magnum master: [WIP] Support <cluster>/actions/resize API https://review.openstack.org/638572 | 22:09 |
imdigitaljim | --bind-address 0.0.0.0 Default: 0.0.0.0 | 22:09 |
imdigitaljim | maaybe? | 22:09 |
strigazi | brtknr: /etc/kubernetes/proxy | 22:09 |
imdigitaljim | check this out? | 22:09 |
strigazi | in magnum it has the default | 22:09 |
imdigitaljim | which is all interfaces? | 22:10 |
strigazi | yes | 22:10 |
imdigitaljim | shouldnt it be node only here/ | 22:10 |
imdigitaljim | ? | 22:10 |
strigazi | for proxy? | 22:10 |
imdigitaljim | i guess what hes doing with his interfaces | 22:10 |
brtknr | oh okay, i'll try adding --bind-address=NODE_IP | 22:10 |
brtknr | to /etc/kubernetes/proxy | 22:10 |
imdigitaljim | im just curious i dont have a solution | 22:11 |
colin- | failing that i'd try imdigitaljim suggestion of wildcarding it | 22:11 |
imdigitaljim | but maybe worth a shot | 22:11 |
colin- | just for troubleshooting | 22:11 |
brtknr | wildcarding? | 22:11 |
colin- | oh, that may be the default my mistake | 22:11 |
colin- | 0.0.0.0/0 | 22:11 |
brtknr | colin-: how would that help? | 22:12 |
brtknr | according to the docs, 0.0.0.0 is already the default | 22:12 |
brtknr | https://kubernetes.io/docs/reference/command-line-tools-reference/kube-proxy/ | 22:12 |
brtknr | for --bind-address | 22:12 |
strigazi | brtknr: colin- imdigitaljim let's end the meeting and just continue? | 22:12 |
brtknr | sure, this is not going to be resolved very easily :) | 22:13 |
strigazi | thanks | 22:13 |
imdigitaljim | yeah im just throwing out ideas | 22:13 |
imdigitaljim | maybe a few things to think/try | 22:13 |
*** gyee has joined #openstack-containers | 22:13 | |
imdigitaljim | maybe to get brtknr unstuck | 22:14 |
strigazi | flwang: brtknr jakeyip schaney colin- eandersson imdigitaljim thanks for joining and for the discussion on autoscaler. | 22:14 |
imdigitaljim | yeah thanks for clearing up some stuff =) | 22:14 |
imdigitaljim | looking forward to the merge | 22:14 |
strigazi | :) | 22:15 |
strigazi | #endmeeting | 22:15 |
*** openstack changes topic to "OpenStack Containers Team" | 22:15 | |
openstack | Meeting ended Tue Feb 26 22:15:19 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 22:15 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/containers/2019/containers.2019-02-26-20.59.html | 22:15 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/containers/2019/containers.2019-02-26-20.59.txt | 22:15 |
openstack | Log: http://eavesdrop.openstack.org/meetings/containers/2019/containers.2019-02-26-20.59.log.html | 22:15 |
brtknr | nice, i'm excited about all the developments! | 22:15 |
eandersson | same! | 22:15 |
jakeyip | thanks all! | 22:15 |
brtknr | i am keen to get more closely involved in development of nodegroups | 22:16 |
*** ttsiouts has quit IRC | 22:16 | |
flwang | brtknr: pls go for it | 22:16 |
brtknr | i remember there is a blueprint for that somewhere | 22:16 |
flwang | strigazi: do you still have a moment ? | 22:16 |
flwang | brtknr: there is already a patch somewhere | 22:16 |
*** itlinux has joined #openstack-containers | 22:16 | |
*** ttsiouts has joined #openstack-containers | 22:16 | |
brtknr | flwang: do you know why its stuck? | 22:17 |
strigazi | flwang: let's do it quickly, I still need to review a paper :) | 22:17 |
flwang | strigazi: i'd like to understand the rolling upgrade progress and design | 22:18 |
strigazi | brtknr: for NGs a colleague of mine is working on it, I push him to push :) | 22:18 |
strigazi | flwang: do you have a question? | 22:18 |
flwang | if we want to do the base os image upgrade and k8s upgrade | 22:18 |
flwang | strigazi: is the functionality patch ready for review? | 22:19 |
strigazi | I think we discussed the design before. The upgrade per API, is controller by CTs | 22:20 |
strigazi | if you change the image OS, the server will be rebuilt | 22:21 |
flwang | where to control the cordon/drain order? | 22:21 |
*** ttsiouts has quit IRC | 22:21 | |
strigazi | let's take it one by one? | 22:21 |
strigazi | start from the API | 22:21 |
strigazi | the api is has the CT and nodegroup as payload | 22:22 |
strigazi | is that clear enough? | 22:22 |
flwang | yes | 22:22 |
flwang | and conductor will finally get the call | 22:22 |
strigazi | correct | 22:22 |
flwang | does that mean conductor will control the order and do the orchestration | 22:23 |
strigazi | let's start with he easy scenario which is to upgrade containers in the nodes | 22:23 |
flwang | you mean k8s components here, right? | 22:23 |
strigazi | initially yes | 22:24 |
strigazi | but the same applies to ther containers too | 22:24 |
strigazi | like flannel | 22:24 |
strigazi | or coredns | 22:24 |
flwang | yep | 22:24 |
*** itlinux has quit IRC | 22:24 | |
flwang | those are the container running on top of the k8s | 22:24 |
strigazi | the conductor as in any other operation will just contact heat and a software deployment will be triggered. | 22:25 |
*** itlinux has joined #openstack-containers | 22:26 | |
strigazi | the script of the deployment will upgrade what ever it contains, do kubectl apply, atomic update or other commans | 22:26 |
flwang | so for the k8s rolling upgrade case, after calling heat stack update, heat-container-agent will call the upgrade.sh to do the magic? | 22:27 |
strigazi | so if you pass a CT that has only version changes it will change only what I mentioned | 22:27 |
strigazi | yes, as it does with cluster creation now | 22:28 |
*** ttsiouts has joined #openstack-containers | 22:28 | |
*** sdake has joined #openstack-containers | 22:28 | |
strigazi | this what I'm doing at the moment | 22:30 |
flwang | cool, will you post a new patch set this week for the functionality patch? | 22:30 |
strigazi | tmr moring, like in ~11 hours | 22:30 |
flwang | fantastic | 22:30 |
flwang | now i'm polishing the resize api | 22:30 |
strigazi | I was doing this all day | 22:30 |
*** sdake has quit IRC | 22:30 | |
flwang | so i'm keen to get agreement with the api design | 22:31 |
brtknr | strigazi: is your colleague actively working on NG btw? | 22:31 |
flwang | combine with the upgrade api | 22:31 |
strigazi | brtknr: yes | 22:31 |
*** itlinux has quit IRC | 22:31 | |
strigazi | you can ping him in the gerrit | 22:31 |
*** sdake has joined #openstack-containers | 22:31 | |
strigazi | brtknr: https://review.openstack.org/#/q/owner:theodoros.tsioutsias%2540cern.ch+status:open | 22:31 |
brtknr | cool thanks! | 22:32 |
*** dave-mccowan has quit IRC | 22:33 | |
flwang | strigazi: if we go for the way like clusterID/action {'upgrade': {}} | 22:33 |
strigazi | brtknr: start from: https://review.openstack.org/#/c/604823/ https://review.openstack.org/#/c/604824/ | 22:33 |
flwang | then we just need one API patch for both resize and upgrade | 22:33 |
strigazi | flwang: interesting | 22:34 |
strigazi | I'll take a look in the moring | 22:35 |
flwang | strigazi: that's the design in nova/cinder/manila/senlin, etc | 22:35 |
strigazi | I'll take a look in the morning | 22:35 |
flwang | cool, let me know your thoughts later, cheers | 22:35 |
strigazi | cheers | 22:35 |
flwang | i'm so excited for all those good work we're doing | 22:36 |
strigazi | :) | 22:36 |
* strigazi has signed off | 22:36 | |
brtknr | flwang: :) | 22:37 |
brtknr | I am currently managing the upgrades using an ansible playbook... a native upgrade mechanism would be pretty cool | 22:38 |
*** _fragatina has joined #openstack-containers | 22:40 | |
*** sdake has quit IRC | 22:40 | |
*** sdake_ has joined #openstack-containers | 22:41 | |
flwang | brtknr: for our case, we can't call our k8s service as beta without rolling upgrade support | 22:42 |
imdigitaljim | im currently working on a simple kickoff mechanism for kubernetes to manage the in-place upgrade | 22:43 |
imdigitaljim | it is our intention to share it too when we're satisfied with the work | 22:43 |
imdigitaljim | we're looking forward to putting out the whole driver with everything ready | 22:43 |
brtknr | whats the diff between upgrade and rolling upgrade? | 22:43 |
imdigitaljim | well im not sure what context youre refering but when the upgrades are done what they do is roll through each node | 22:44 |
imdigitaljim | and swap it out with a newer version | 22:44 |
flwang | rolling generally means one by one and no down time for services | 22:44 |
imdigitaljim | the roll meaning old one goes offline, new one comes online | 22:44 |
imdigitaljim | yeah to maintain no downtime | 22:45 |
brtknr | oh ok | 22:45 |
imdigitaljim | instead of all offline swap out, all back online new versions | 22:45 |
imdigitaljim | or a full k8s rebuild | 22:45 |
brtknr | gotcha | 22:45 |
*** sdake_ has quit IRC | 22:45 | |
brtknr | i think i am doing a non-rolling upgrade atm :P | 22:45 |
brtknr | in that case | 22:45 |
imdigitaljim | do you basically just rebuild the clusters? | 22:45 |
brtknr | no, i run the ansible-playbook in parallel across all the nodes | 22:46 |
brtknr | not one-by-one | 22:46 |
brtknr | and upgrade the kubernetes related containers | 22:46 |
imdigitaljim | what do you do with them? | 22:46 |
imdigitaljim | just change their versions/download new containers? | 22:46 |
brtknr | this: https://github.com/stackhpc/p3-appliances/blob/master/ansible/container-infra-upgrade.yml | 22:47 |
brtknr | yep basically | 22:47 |
imdigitaljim | so if there are any config changes that probably wont without manual changes from version to version no? | 22:47 |
imdigitaljim | like new required ones that you dont already have or removed old ones that are no longer present in new version | 22:48 |
imdigitaljim | not just reconfigu | 22:48 |
*** sdake has joined #openstack-containers | 22:48 | |
brtknr | yeah thats true, but so far ive not had any problems | 22:48 |
brtknr | api is pretty backwards compatible | 22:48 |
brtknr | k8s api* | 22:49 |
brtknr | im running queens | 22:49 |
*** henriqueof has joined #openstack-containers | 22:49 | |
brtknr | so started at 1.9.3 and now im running 1.13.2 | 22:49 |
brtknr | i can downgrade to earlier versions at any point | 22:49 |
*** itlinux has joined #openstack-containers | 22:50 | |
brtknr | i miss out on things like occm for particular version of k8s | 22:52 |
brtknr | imdigitaljim: --bind-address=NODE_IP didnt work | 22:54 |
brtknr | i dont think --bind-address=0.0.0.0/0 is valid | 22:54 |
*** sdake has quit IRC | 22:56 | |
*** sdake has joined #openstack-containers | 22:57 | |
*** itlinux has quit IRC | 22:57 | |
*** sdake has quit IRC | 23:00 | |
*** sdake has joined #openstack-containers | 23:01 | |
*** sdake has quit IRC | 23:06 | |
*** sdake_ has joined #openstack-containers | 23:06 | |
*** sdake_ has quit IRC | 23:10 | |
*** sdake has joined #openstack-containers | 23:11 | |
*** dave-mccowan has joined #openstack-containers | 23:18 | |
*** sdake has quit IRC | 23:20 | |
*** sdake has joined #openstack-containers | 23:22 | |
*** sdake has quit IRC | 23:25 | |
*** sdake has joined #openstack-containers | 23:27 | |
*** henriqueof has quit IRC | 23:28 | |
*** sdake has quit IRC | 23:30 | |
*** ttsiouts has quit IRC | 23:30 | |
*** ttsiouts has joined #openstack-containers | 23:31 | |
*** sdake has joined #openstack-containers | 23:33 | |
*** sapd1 has joined #openstack-containers | 23:33 | |
*** sdake has quit IRC | 23:35 | |
*** ttsiouts has quit IRC | 23:35 | |
*** sdake has joined #openstack-containers | 23:36 | |
*** sdake has quit IRC | 23:37 | |
*** sdake has joined #openstack-containers | 23:38 | |
*** sdake has quit IRC | 23:40 | |
*** sdake has joined #openstack-containers | 23:42 | |
*** henriqueof has joined #openstack-containers | 23:43 | |
*** sdake has quit IRC | 23:50 | |
*** sdake has joined #openstack-containers | 23:53 | |
*** sdake has quit IRC | 23:55 | |
*** sdake has joined #openstack-containers | 23:56 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!