*** garyloug_ has quit IRC | 00:01 | |
*** salv-orlando has joined #openstack-kuryr | 00:02 | |
*** salv-orlando has quit IRC | 00:08 | |
*** gouthamr has quit IRC | 00:48 | |
*** salv-orlando has joined #openstack-kuryr | 01:03 | |
*** salv-orlando has quit IRC | 01:07 | |
*** gouthamr has joined #openstack-kuryr | 01:50 | |
*** caowei has joined #openstack-kuryr | 01:57 | |
*** salv-orlando has joined #openstack-kuryr | 02:04 | |
*** salv-orlando has quit IRC | 02:09 | |
*** yamamoto has joined #openstack-kuryr | 02:20 | |
*** atoth has quit IRC | 02:31 | |
*** kiennt26 has joined #openstack-kuryr | 02:33 | |
*** gouthamr has quit IRC | 02:40 | |
*** gouthamr has joined #openstack-kuryr | 02:45 | |
*** salv-orlando has joined #openstack-kuryr | 03:04 | |
*** salv-orlando has quit IRC | 03:10 | |
*** gouthamr has quit IRC | 03:34 | |
*** gouthamr has joined #openstack-kuryr | 03:39 | |
*** aojea has joined #openstack-kuryr | 03:53 | |
*** aojea has quit IRC | 03:57 | |
*** gouthamr has quit IRC | 03:59 | |
*** salv-orlando has joined #openstack-kuryr | 04:05 | |
*** salv-orlando has quit IRC | 04:10 | |
*** yamamoto has quit IRC | 04:38 | |
*** yamamoto has joined #openstack-kuryr | 04:41 | |
*** caowei has quit IRC | 04:47 | |
*** caowei has joined #openstack-kuryr | 05:01 | |
*** salv-orlando has joined #openstack-kuryr | 05:06 | |
*** salv-orlando has quit IRC | 05:10 | |
*** janki has joined #openstack-kuryr | 05:21 | |
*** dims has quit IRC | 06:05 | |
*** pc_m has quit IRC | 06:06 | |
*** dims has joined #openstack-kuryr | 06:12 | |
*** salv-orlando has joined #openstack-kuryr | 06:13 | |
*** pc_m has joined #openstack-kuryr | 06:14 | |
*** janki has quit IRC | 06:26 | |
*** janki has joined #openstack-kuryr | 06:27 | |
*** janki has quit IRC | 06:28 | |
*** janki has joined #openstack-kuryr | 06:28 | |
*** aojea has joined #openstack-kuryr | 06:58 | |
*** yboaron has joined #openstack-kuryr | 07:40 | |
*** aojea has quit IRC | 08:02 | |
openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Add VIF-Handler And Drivers Design approach https://review.openstack.org/513715 | 08:10 |
---|---|---|
*** salv-orlando has quit IRC | 08:18 | |
*** salv-orlando has joined #openstack-kuryr | 08:19 | |
*** salv-orlando has quit IRC | 08:23 | |
openstackgerrit | Danil Golov proposed openstack/kuryr-kubernetes master: Add VIF-Handler And Drivers Design approach https://review.openstack.org/513715 | 08:25 |
*** aojea has joined #openstack-kuryr | 08:31 | |
ltomasbo | hi irenab, don't forget about: https://review.openstack.org/#/c/510157/ :D | 08:35 |
*** salv-orlando has joined #openstack-kuryr | 08:58 | |
ltomasbo | hi dulek! I just tested your patch (containerized+daemon) with the other fixed | 09:03 |
ltomasbo | and now it works like a charm! | 09:03 |
dulek | ltomasbo: Yay! | 09:04 |
ltomasbo | no processes left behind, and all containers are starting without any problems! | 09:04 |
dulek | ltomasbo: I'm glad to hear that. | 09:05 |
ltomasbo | ohh, I was too fast... | 09:05 |
ltomasbo | I tried to create 30 pods now at once | 09:06 |
ltomasbo | and I get: http://paste.openstack.org/show/625803/ | 09:06 |
dulek | ltomasbo: Hm, I've did 40 without problems yesterday. What happened? | 09:06 |
ltomasbo | dulek, ^^ | 09:06 |
ltomasbo | ok, but at the end, they become active anyway | 09:07 |
ltomasbo | so, it killed the sandbox and recreated it | 09:07 |
dulek | ltomasbo: Do: `for ip in $(kubectl get pods -o wide | tr -s " " | cut -f 6 -d " " | grep 10); do ping -c 1 $ip; done` to see if all of them are pingable | 09:07 |
dulek | (command should just end immediately) | 09:08 |
dulek | (ah, grep 10 works if your pod network starts with 10 :P) | 09:08 |
dulek | I've hit an exception yesterday, but retry done by kubelet helped. | 09:08 |
ltomasbo | umm | 09:09 |
ltomasbo | not all are pingable | 09:09 |
dulek | ltomasbo: http://paste.openstack.org/show/625804/ - you can look for that in daemon logs. | 09:09 |
dulek | Eh, so there must be another mistake. I'll try to create a bunch of pods a few times, maybe I'll get it. | 09:10 |
ltomasbo | actually, the onces that are not working, as the ones with that CNI error I pasted | 09:11 |
dulek | ltomasbo: Any chance to get the cni-daemon logs? | 09:12 |
ltomasbo | sure | 09:12 |
dulek | ltomasbo: It would be best if you could send me the logs along with output of "openstack port list" | 09:12 |
dulek | That's how I debugged yesterday's problems. | 09:12 |
ltomasbo | dulek, http://paste.openstack.org/show/625805/ | 09:13 |
dulek | ltomasbo: I guess "fa:16:3e:ad:50:69" matches the address of non-active port in openstack port list? | 09:14 |
ltomasbo | dulek, do you prefer access to the env or the logs? | 09:14 |
ltomasbo | let me check that | 09:15 |
ltomasbo | yep, they are down | 09:15 |
dulek | ltomasbo: Okay, so I think I'm getting the same on my env. | 09:16 |
dulek | Okay, I'll look into it. At least I know how to debug this stuff. :) | 09:16 |
ltomasbo | xD | 09:17 |
ltomasbo | that helps! | 09:17 |
ltomasbo | any initial idea? | 09:17 |
ltomasbo | some kind of race getting some info from the namespaces? | 09:17 |
*** garyloug has joined #openstack-kuryr | 09:30 | |
*** garyloug has quit IRC | 09:34 | |
openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Add readiness probe to kuryr-controller pod https://review.openstack.org/518502 | 09:42 |
*** irenab has quit IRC | 09:43 | |
*** leyal has quit IRC | 09:43 | |
*** jerms has quit IRC | 09:43 | |
*** nhlfr has quit IRC | 09:46 | |
*** enick_354 has quit IRC | 09:47 | |
*** garyloug has joined #openstack-kuryr | 09:48 | |
*** irenab has joined #openstack-kuryr | 09:49 | |
*** leyal has joined #openstack-kuryr | 09:49 | |
*** jerms has joined #openstack-kuryr | 09:49 | |
*** deepika has joined #openstack-kuryr | 09:58 | |
*** deepika is now known as Guest49950 | 09:58 | |
dulek | ltomasbo: Oh, sorry, I've dived too much into the code. For now I know this is an exception raised during exception handling. :D | 09:58 |
dulek | ltomasbo: I'm modifying pyroute2 to print initial exception. | 09:59 |
*** dougbtv has joined #openstack-kuryr | 09:59 | |
*** kiennt26 has quit IRC | 10:00 | |
*** dougbtv_ has quit IRC | 10:00 | |
*** dougbtv_ has joined #openstack-kuryr | 10:01 | |
ltomasbo | dulek, no worries! I was just curious! | 10:02 |
ltomasbo | dulek, I added initial support for the readiness probe when containerized and ports pool enabled: https://review.openstack.org/518502 | 10:02 |
ltomasbo | dulek, I'll adapt the openshift-ansible too and it seems to be working: http://paste.openstack.org/show/625810/ | 10:03 |
irenab | ltomasbo, reviewed | 10:04 |
ltomasbo | irenab, thanks! I'll take a look | 10:04 |
dulek | ltomasbo: I've added myself as reviewer. I'm slacking off with reviews, but fighting those transient problems is super time consuming. | 10:04 |
*** dougbtv has quit IRC | 10:04 | |
ltomasbo | dulek, I can imagine... those are difficult to find though usually fixes are just a couple of lines... | 10:05 |
*** nhlfr has joined #openstack-kuryr | 10:06 | |
*** caowei has quit IRC | 10:10 | |
dulek | ltomasbo: Aww, looks like it's a timeout issue - kernel works slower than pyroute.ipdb expects. | 10:30 |
dulek | ltomasbo: I'm checking that now. | 10:30 |
ltomasbo | umm | 10:32 |
dulek | ltomasbo: I'm still getting tracebacks but surprisingly this time all the pods are pingable… Awww. | 10:38 |
ltomasbo | with the ports in down status?? | 10:38 |
ltomasbo | dulek, ^^ | 10:38 |
dulek | ltomasbo: No, they're ACTIVE. | 10:38 |
dulek | ltomasbo: Can you explain what's responsible for turning ports from DOWN to ACTIVE? | 10:39 |
dulek | l3 agent? | 10:39 |
ltomasbo | yep | 10:40 |
ltomasbo | I think the agent triggers a notification when the flows are there | 10:40 |
ltomasbo | and the server update the status in the DB | 10:40 |
dulek | ltomasbo: So it sees that the port gets connected by kuryr and reports that? | 10:40 |
dulek | Ok. | 10:40 |
openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Avoid neutron calls at recovering precreated ports https://review.openstack.org/510157 | 10:42 |
dulek | ltomasbo: I fear that there may be another race condition. Because why sometimes when kubelet retries, ports are getting into DOWN state? | 10:44 |
dulek | ltomasbo: On failure kubelet always tries to do DEL first. | 10:44 |
dulek | ltomasbo: I wonder if there isn't a race condition between DEL and ADD. VIF stays the same between retries. | 10:45 |
dulek | ltomasbo: If ADD somehow gets processed faster, and then delayed DEL comes - pod ends up without network. | 10:45 |
dulek | Seems a bit bold, but I think it's possible. | 10:46 |
ltomasbo | dulek, so, you think it fails and the delete comes after the next retry has taken place? | 10:46 |
dulek | ltomasbo: IMO kubelet should wait for DEL to complete before sending another ADD. | 10:47 |
dulek | ltomasbo: So I'm not sure here… | 10:47 |
ltomasbo | dulek, it should, but maybe not. Didn't you discuss that case with apuimedo at some point? | 10:48 |
ltomasbo | was due to a related problem? | 10:48 |
dulek | ltomasbo: No, I thought of this race condition just yesterday. | 10:49 |
dulek | ltomasbo: Now I've got 12/30 pods not connected… | 10:49 |
ltomasbo | dulek, that is pretty much what I was gettting | 10:50 |
*** aojea has quit IRC | 10:50 | |
ltomasbo | like a few containers not getting to Running that fast, and then not having connectivity | 10:51 |
dulek | ltomasbo: Oh, okay, I've committed timeout increase to wrong branch! Let me recheck that… | 10:56 |
dulek | (too less coffee today) | 10:56 |
ltomasbo | xD | 10:56 |
dulek | too little? | 10:56 |
dulek | :P | 10:56 |
ltomasbo | solution: more coffee!! jejejeje | 10:57 |
*** yamamoto has quit IRC | 11:03 | |
*** yboaron_ has joined #openstack-kuryr | 11:04 | |
*** yboaron has quit IRC | 11:07 | |
*** aojea has joined #openstack-kuryr | 11:12 | |
*** yboaron_ has quit IRC | 11:23 | |
*** Guest96098 has joined #openstack-kuryr | 11:36 | |
*** yamamoto has joined #openstack-kuryr | 11:37 | |
dulek | ltomasbo: I've increased VIF annotation timeout and this kernel timeout and it looks promising. | 11:39 |
dulek | ltomasbo: I'll scale the deployment few more times. Fingers crossed. | 11:40 |
*** yboaron_ has joined #openstack-kuryr | 11:43 | |
ltomasbo | dulek, great! | 11:43 |
ltomasbo | dulek, fingers crossed! let me know the outcome! | 11:44 |
*** Guest96098 has quit IRC | 11:45 | |
dulek | ltomasbo: Eh, still same tracebacks, one pod haven't got the networks… | 11:47 |
ltomasbo | umm | 11:47 |
dulek | ltomasbo: Waaaait… | 11:48 |
dulek | No, it has normal IPv4, I wondered if it isn't IPv6 fault. | 11:48 |
ltomasbo | dulek, ohh, but the port is active? | 11:49 |
dulek | No. | 11:49 |
ltomasbo | and it has ipv4 connectivity?? | 11:49 |
dulek | No, no, it's normal port, just must have been disconnected. Maybe I can track how it was processed through logs. | 11:50 |
ltomasbo | but if the port is down, the agent should remove the flows, and it should not have connectivity, even if the pod is up and running and connected to the br-int, right? | 11:51 |
dulek | No, no, it has *no* connectivity. | 11:52 |
dulek | Sorry for confusing you. | 11:52 |
ltomasbo | ahh, ok | 11:52 |
dulek | Yeah, last operation on the vif was Successfully unplugged vif | 11:52 |
dulek | Now why… | 11:52 |
dulek | Ah, because that's how request came. Now why the strange order… | 11:53 |
ltomasbo | so, the add (retry) was performed before the delete (of the failed previous attempt) | 11:55 |
dulek | I think so. Which is rather strange. | 11:55 |
dulek | It was ADD, DEL, ADD, DEL for this pod. | 11:56 |
ltomasbo | umm, then the order is good | 11:56 |
ltomasbo | the bad thing is that there is an extra del, right? | 11:56 |
dulek | Yup. | 11:56 |
ltomasbo | umm, why 2 dels.... | 11:57 |
dulek | How about - ADD errors out, DEL is issued, DEL finishes with failure, so ADD is issued, but failed DEL is retried. | 11:57 |
dulek | That would make sense. | 11:57 |
dulek | And that would make our VIF allocation algorithm not compatible with what kubelet is doing. | 11:57 |
ltomasbo | umm | 11:58 |
ltomasbo | that could be | 11:58 |
ltomasbo | but not sure why it happens with many containers | 11:58 |
ltomasbo | why del finishes with failure? | 11:58 |
dulek | Timeouts I guess? But good question, I'll track that down… | 11:59 |
ltomasbo | yep, timeouts could be a good explanation | 12:01 |
ltomasbo | but I wonder why this was not happening before | 12:02 |
ltomasbo | dulek, due to having the ipdb cache? | 12:02 |
dulek | ltomasbo: We've had a single process for each request before. They are parallelizing much better in Python. | 12:03 |
dulek | Actually if I've eliminated orphaned threads issue, I should be able to spawn more processes. I'll try that later. | 12:04 |
*** aojea has quit IRC | 12:05 | |
dulek | Okay, I'll need to add more logs to track requests correctly. | 12:08 |
dulek | ltomasbo: Uhm, and by the way first DEL seems to return success, so I'm puzzled why there's a second issued. | 12:08 |
ltomasbo | umm | 12:12 |
ltomasbo | there should be a second del if the second add fails, right? | 12:14 |
dulek | ltomasbo: But I see second ADD to complete correctly in kubelet logs. | 12:15 |
ltomasbo | ummm | 12:15 |
dulek | ltomasbo: CNI Daemon returned "202 ACCEPTED", followed by correct info about IP. | 12:15 |
dulek | Then I have: | 12:15 |
ltomasbo | how is that possible then | 12:15 |
dulek | NetworkPlugin cni failed on the status hook for pod "test-3202410914-qlcbc_default": Unexpected command output nsenter: cannot open : No such file or directory | 12:15 |
ltomasbo | could it be the message is lost? | 12:15 |
dulek | And then I see DEL being issued. | 12:16 |
dulek | Now this nsenter error is something I've seen a lot of time in the logs. | 12:16 |
*** atoth has joined #openstack-kuryr | 12:17 | |
*** aojea has joined #openstack-kuryr | 12:28 | |
*** salv-orlando has quit IRC | 13:18 | |
*** salv-orlando has joined #openstack-kuryr | 13:19 | |
*** salv-orlando has quit IRC | 13:23 | |
irenab | dulek, I wonder what level of testing can be done for the https://review.openstack.org/#/c/517406/3 | 13:24 |
irenab | I wonder if some level unit testing can be added | 13:24 |
dulek | irenab: I think we're missing unit tests for whole 'cni' module. | 13:31 |
irenab | you are correct | 13:31 |
dulek | irenab: I've added some in CNI daemon patch, but I've omitted cni.binding because, well… I haven't understood what's going on in those functions. | 13:32 |
irenab | :-), I thunk now you are the expert | 13:32 |
irenab | think | 13:32 |
dulek | So I can add them in a separate patch or include it in the bugfix, whatever you like. | 13:33 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr master: Updated from global requirements https://review.openstack.org/516936 | 13:35 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr-libnetwork master: Updated from global requirements https://review.openstack.org/516937 | 13:35 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr-kubernetes master: Updated from global requirements https://review.openstack.org/509781 | 13:35 |
irenab | better to have it as part of the patch, at least some basic tests | 13:37 |
dulek | irenab: Okay, I'll work on that. | 13:40 |
*** aojea has quit IRC | 13:45 | |
*** aojea has joined #openstack-kuryr | 13:45 | |
openstackgerrit | Luis Tomas Bolivar proposed openstack/kuryr-kubernetes master: Add readiness probe to kuryr-controller pod https://review.openstack.org/518502 | 13:54 |
*** yamamoto has quit IRC | 13:55 | |
*** yamamoto has joined #openstack-kuryr | 13:56 | |
*** yamamoto has quit IRC | 14:01 | |
*** aojea has quit IRC | 14:05 | |
openstackgerrit | Merged openstack/kuryr-kubernetes master: Avoid neutron calls at recovering precreated ports https://review.openstack.org/510157 | 14:06 |
*** aojea has joined #openstack-kuryr | 14:06 | |
dulek | ltomasbo: Ghhaaaw! I should turn off port_debug to test in scale, shouldn't I? | 14:11 |
ltomasbo | yep | 14:12 |
ltomasbo | dulek, that will skip neutron calls (changing names) | 14:12 |
ltomasbo | but that will affect mostly when using pools | 14:12 |
ltomasbo | dulek, otherwise it may not skip that many calls, possibly none | 14:13 |
dulek | ltomasbo: Ah, right… So that's why I'm seeing kuryr-kubernetes to annotate pods with VIFs really slowly? | 14:15 |
*** aojea has quit IRC | 14:17 | |
ltomasbo | xD | 14:18 |
ltomasbo | could be | 14:19 |
ltomasbo | dulek, there will be a lot of calls to neutron to create the ports, pooling to know when the port becomes active, change the name and device-id, ... | 14:19 |
*** yamamoto has joined #openstack-kuryr | 14:26 | |
*** yamamoto has quit IRC | 14:31 | |
*** salv-orlando has joined #openstack-kuryr | 14:31 | |
huats_ | Hi guys | 14:35 |
huats_ | I am quite surprised by the documentation | 14:35 |
huats_ | I I have followed the libkuryr-network installation doc and at the end I don't have /usr/local/libexec/kuryr/ created | 14:36 |
huats_ | thus I don't have the various script for bindings | 14:36 |
huats_ | (like the ovs in my case) | 14:36 |
huats_ | any idea ? | 14:36 |
huats_ | ok | 14:42 |
huats_ | I was missing kuryr installation | 14:42 |
*** aojea has joined #openstack-kuryr | 14:43 | |
*** premsankar has joined #openstack-kuryr | 15:05 | |
*** aojea has quit IRC | 15:07 | |
*** yboaron_ has quit IRC | 15:08 | |
*** janki has quit IRC | 15:10 | |
*** aojea has joined #openstack-kuryr | 15:11 | |
*** yamamoto has joined #openstack-kuryr | 15:12 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr master: Updated from global requirements https://review.openstack.org/516936 | 15:15 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr-libnetwork master: Updated from global requirements https://review.openstack.org/516937 | 15:15 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr-kubernetes master: Updated from global requirements https://review.openstack.org/509781 | 15:15 |
*** yamamoto has quit IRC | 15:16 | |
*** aojea has quit IRC | 15:22 | |
*** aojea has joined #openstack-kuryr | 15:24 | |
*** aojea has quit IRC | 15:34 | |
*** aojea has joined #openstack-kuryr | 15:41 | |
*** salv-orlando has quit IRC | 15:42 | |
*** salv-orlando has joined #openstack-kuryr | 15:43 | |
*** aojea has quit IRC | 15:45 | |
*** salv-orlando has quit IRC | 15:47 | |
*** aojea has joined #openstack-kuryr | 15:51 | |
*** yamamoto has joined #openstack-kuryr | 15:54 | |
*** yamamoto has quit IRC | 16:01 | |
*** yamamoto has joined #openstack-kuryr | 16:03 | |
*** aojea has quit IRC | 16:04 | |
*** aojea has joined #openstack-kuryr | 16:07 | |
dulek | Rather lame question but… how do I run a single unit test? | 16:14 |
*** yamamoto has quit IRC | 16:16 | |
ltomasbo | dulek, enable the virtalenv | 16:19 |
ltomasbo | source .tox/py27/bin/activate | 16:19 |
ltomasbo | and then run this: | 16:19 |
dulek | Oh, no `tox -e py27 -- --regex path.to.module`? | 16:19 |
*** aojea has quit IRC | 16:19 | |
ltomasbo | python -m testtools.run [test module path] | 16:20 |
ltomasbo | that is the way I do it, pretty sure there are other ways... | 16:20 |
dulek | ltomasbo: Thanks, that's much better than nothing. :) | 16:22 |
ltomasbo | xD | 16:22 |
*** yamamoto has joined #openstack-kuryr | 16:26 | |
*** yamamoto has quit IRC | 16:31 | |
*** salv-orlando has joined #openstack-kuryr | 16:43 | |
*** openstackgerrit has quit IRC | 16:48 | |
*** salv-orlando has quit IRC | 16:50 | |
*** salv-orlando has joined #openstack-kuryr | 17:27 | |
*** yamamoto has joined #openstack-kuryr | 17:35 | |
*** garyloug_ has joined #openstack-kuryr | 17:40 | |
*** garyloug_ has quit IRC | 17:43 | |
*** yamamoto has quit IRC | 17:43 | |
*** garyloug has quit IRC | 17:43 | |
*** aojea has joined #openstack-kuryr | 17:55 | |
*** aojea has quit IRC | 18:21 | |
*** salv-orlando has quit IRC | 18:26 | |
*** salv-orlando has joined #openstack-kuryr | 18:34 | |
*** aojea has joined #openstack-kuryr | 18:37 | |
*** aojea has quit IRC | 18:54 | |
*** aojea has joined #openstack-kuryr | 19:00 | |
*** aojea has quit IRC | 19:18 | |
*** openstackgerrit has joined #openstack-kuryr | 19:19 | |
openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Prevent pyroute2.IPDB threads leaking https://review.openstack.org/517406 | 19:19 |
openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: CNI split - introducing CNI daemon https://review.openstack.org/515186 | 19:19 |
openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: Support kuryr-daemon when running containerized https://review.openstack.org/518024 | 19:19 |
dulek | irenab: It wasn't too easy, but I've added unit tests for cni.binding modules. | 19:22 |
openstackgerrit | Michał Dulko proposed openstack/kuryr-kubernetes master: DNM: Trying to run gate tests with CNI Daemon https://review.openstack.org/509765 | 19:30 |
*** aojea has joined #openstack-kuryr | 19:31 | |
*** aojea has quit IRC | 19:41 | |
*** aojea has joined #openstack-kuryr | 19:41 | |
*** aojea has quit IRC | 19:53 | |
*** aojea has joined #openstack-kuryr | 20:04 | |
*** atoth has quit IRC | 20:11 | |
*** vikasc has quit IRC | 20:28 | |
*** aojea has quit IRC | 20:44 | |
*** dougbtv__ has joined #openstack-kuryr | 22:08 | |
*** aojea has joined #openstack-kuryr | 22:44 | |
*** aojea has quit IRC | 22:49 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr master: Updated from global requirements https://review.openstack.org/516936 | 23:55 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr-libnetwork master: Updated from global requirements https://review.openstack.org/516937 | 23:55 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/kuryr-kubernetes master: Updated from global requirements https://review.openstack.org/509781 | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!