| *** jamesmcarthur has quit IRC | 00:01 | |
| *** macza has quit IRC | 00:06 | |
| *** markvoelker has joined #openstack-meeting-3 | 00:19 | |
| *** mjturek has joined #openstack-meeting-3 | 00:45 | |
| *** mjturek has quit IRC | 00:51 | |
| *** slaweq has quit IRC | 00:52 | |
| *** logan- has quit IRC | 01:22 | |
| *** logan- has joined #openstack-meeting-3 | 01:23 | |
| *** logan- has quit IRC | 01:31 | |
| *** logan- has joined #openstack-meeting-3 | 01:31 | |
| *** sdake has quit IRC | 01:44 | |
| *** sdake has joined #openstack-meeting-3 | 01:46 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 02:10 | |
| *** jamesmcarthur has quit IRC | 02:37 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 02:42 | |
| *** purplerbot has quit IRC | 02:43 | |
| *** purplerbot has joined #openstack-meeting-3 | 02:44 | |
| *** hongbin has joined #openstack-meeting-3 | 02:48 | |
| *** sdake has quit IRC | 02:48 | |
| *** sdake has joined #openstack-meeting-3 | 02:49 | |
| *** psachin has joined #openstack-meeting-3 | 02:52 | |
| *** sdake has quit IRC | 02:57 | |
| *** sdake has joined #openstack-meeting-3 | 03:00 | |
| *** dklyle has quit IRC | 03:03 | |
| *** jamesmcarthur has quit IRC | 03:08 | |
| *** apetrich has quit IRC | 03:14 | |
| *** dklyle has joined #openstack-meeting-3 | 03:17 | |
| *** markvoelker has quit IRC | 03:20 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 03:32 | |
| *** spsurya has joined #openstack-meeting-3 | 03:34 | |
| *** jamesmcarthur has quit IRC | 04:00 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 04:01 | |
| *** jamesmcarthur has quit IRC | 04:05 | |
| *** markvoelker has joined #openstack-meeting-3 | 04:21 | |
| *** ianychoi has quit IRC | 04:31 | |
| *** sdake has quit IRC | 04:50 | |
| *** markvoelker has quit IRC | 04:55 | |
| *** sdake has joined #openstack-meeting-3 | 04:55 | |
| *** hongbin has quit IRC | 05:04 | |
| *** diablo_rojo has quit IRC | 05:05 | |
| *** sdake has quit IRC | 05:14 | |
| *** sdake has joined #openstack-meeting-3 | 05:16 | |
| *** yamamoto has quit IRC | 05:44 | |
| *** yamamoto has joined #openstack-meeting-3 | 05:48 | |
| *** sdake has quit IRC | 05:48 | |
| *** sdake has joined #openstack-meeting-3 | 05:50 | |
| *** markvoelker has joined #openstack-meeting-3 | 05:52 | |
| *** sdake has quit IRC | 06:03 | |
| *** sdake has joined #openstack-meeting-3 | 06:05 | |
| *** markvoelker has quit IRC | 06:25 | |
| *** spsurya has quit IRC | 06:25 | |
| *** sdake has quit IRC | 06:37 | |
| *** Luzi has joined #openstack-meeting-3 | 06:40 | |
| *** igordc has quit IRC | 06:52 | |
| *** yamamoto has quit IRC | 06:59 | |
| *** slaweq has joined #openstack-meeting-3 | 07:08 | |
| *** psachin has quit IRC | 07:10 | |
| *** spsurya has joined #openstack-meeting-3 | 07:17 | |
| *** pcaruana has joined #openstack-meeting-3 | 07:19 | |
| *** markvoelker has joined #openstack-meeting-3 | 07:22 | |
| *** sambetts_ has quit IRC | 07:29 | |
| *** sambetts_ has joined #openstack-meeting-3 | 07:30 | |
| *** psachin has joined #openstack-meeting-3 | 07:32 | |
| *** apetrich has joined #openstack-meeting-3 | 07:33 | |
| *** yamamoto has joined #openstack-meeting-3 | 07:38 | |
| *** markvoelker has quit IRC | 07:55 | |
| *** _pewp_ has quit IRC | 07:58 | |
| *** _pewp_ has joined #openstack-meeting-3 | 07:59 | |
| *** tssurya has joined #openstack-meeting-3 | 07:59 | |
| *** ralonsoh has joined #openstack-meeting-3 | 08:51 | |
| *** markvoelker has joined #openstack-meeting-3 | 08:52 | |
| *** yamamoto has quit IRC | 08:56 | |
| *** yamamoto has joined #openstack-meeting-3 | 09:04 | |
| *** e0ne has joined #openstack-meeting-3 | 09:14 | |
| *** markvoelker has quit IRC | 09:25 | |
| *** moguimar has joined #openstack-meeting-3 | 09:33 | |
| *** tssurya has quit IRC | 09:43 | |
| *** tssurya has joined #openstack-meeting-3 | 09:57 | |
| *** yamamoto has quit IRC | 09:58 | |
| *** markvoelker has joined #openstack-meeting-3 | 10:17 | |
| *** spsurya has quit IRC | 10:22 | |
| *** yamamoto has joined #openstack-meeting-3 | 10:34 | |
| *** psachin has quit IRC | 10:38 | |
| *** yamamoto has quit IRC | 10:39 | |
| *** psachin has joined #openstack-meeting-3 | 10:42 | |
| *** lpetrut has joined #openstack-meeting-3 | 11:03 | |
| *** tssurya has quit IRC | 11:05 | |
| *** psachin has quit IRC | 11:22 | |
| *** yamamoto has joined #openstack-meeting-3 | 11:29 | |
| *** raildo has joined #openstack-meeting-3 | 11:33 | |
| *** yamamoto has quit IRC | 11:34 | |
| *** sdake has joined #openstack-meeting-3 | 12:02 | |
| *** tssurya has joined #openstack-meeting-3 | 12:18 | |
| *** e0ne has quit IRC | 12:24 | |
| *** yamamoto has joined #openstack-meeting-3 | 12:28 | |
| *** edleafe has joined #openstack-meeting-3 | 12:30 | |
| *** e0ne has joined #openstack-meeting-3 | 12:31 | |
| *** yamamoto has quit IRC | 12:34 | |
| *** yamamoto has joined #openstack-meeting-3 | 13:03 | |
| *** yamamoto has quit IRC | 13:08 | |
| *** markvoelker has quit IRC | 13:17 | |
| *** markvoelker has joined #openstack-meeting-3 | 13:18 | |
| *** markvoelker has quit IRC | 13:22 | |
| *** sdake has quit IRC | 13:23 | |
| *** tosky has joined #openstack-meeting-3 | 13:24 | |
| *** sdake has joined #openstack-meeting-3 | 13:25 | |
| *** yamamoto has joined #openstack-meeting-3 | 13:36 | |
| *** sdake has quit IRC | 13:36 | |
| *** yamamoto has quit IRC | 13:36 | |
| *** sdake has joined #openstack-meeting-3 | 13:38 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 13:43 | |
| *** yamamoto has joined #openstack-meeting-3 | 13:47 | |
| *** tellesnobrega has joined #openstack-meeting-3 | 13:57 | |
| *** jamesmcarthur has quit IRC | 13:58 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 13:58 | |
| tellesnobrega | #startmeeting sahara | 14:00 |
|---|---|---|
| openstack | Meeting started Thu Feb 21 14:00:22 2019 UTC and is due to finish in 60 minutes. The chair is tellesnobrega. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: sahara)" | 14:00 | |
| openstack | The meeting name has been set to 'sahara' | 14:00 |
| tosky | o/ | 14:00 |
| tellesnobrega | waiting to see if jeremyfreudberg joins | 14:01 |
| tellesnobrega | apparently not | 14:02 |
| tellesnobrega | #topic News/Updates | 14:02 |
| *** openstack changes topic to "News/Updates (Meeting topic: sahara)" | 14:02 | |
| *** njohnston has joined #openstack-meeting-3 | 14:02 | |
| *** njohnston has left #openstack-meeting-3 | 14:02 | |
| tellesnobrega | I'm stuck on python3 issue related to pickle | 14:03 |
| tellesnobrega | I've reached out for some help but nothing so far has worked | 14:03 |
| *** jamesmcarthur has quit IRC | 14:03 | |
| tosky | pickles from pickle | 14:03 |
| tellesnobrega | I'm now trying to set up a python3 only image so I can test if the error is only related to python2->python3 or even with both environments running on python3 if it still raises the same issue | 14:04 |
| tosky | I've helped with a tons of backports to the older branches | 14:04 |
| tosky | we have a blocker for merging stuff to pike, the issue is a bit complex and this bug contains all the details: https://bugs.launchpad.net/tempest/+bug/1816022 | 14:05 |
| openstack | Launchpad bug 1816022 in tempest "Incompatible requirements break tempest (at least) on stable/pike" [Undecided,New] | 14:05 |
| tosky | I still need to test the mapr fixes | 14:07 |
| tosky | but we are in a better shape, also the number of open reviews lowered (thanks to a Friday-night review by Jeremy :) | 14:07 |
| tellesnobrega | yes, I guess python 3 is the only critical stuff right now | 14:08 |
| tellesnobrega | should I explain a bit of the python 3 issue here | 14:11 |
| tellesnobrega | so we have it on records? | 14:11 |
| tosky | yep, I'd say so | 14:11 |
| tellesnobrega | #topic python3 | 14:11 |
| *** openstack changes topic to "python3 (Meeting topic: sahara)" | 14:11 | |
| tosky | so we can point to these logs in emails | 14:11 |
| tellesnobrega | So, the issue we are facing with python3 now is related to pickle | 14:11 |
| tellesnobrega | Sahara uses Pickle to serialize functions (path of a function) to send them to be executed in a Subprocess.Popen | 14:12 |
| tellesnobrega | either locally or over ssh (depending on the command) | 14:12 |
| tellesnobrega | At first, the error being returned was a Run of input, which we fixed by adding buffer on the process and dumping from pickle into the subprocess buffer | 14:13 |
| *** yamamoto has quit IRC | 14:13 | |
| tellesnobrega | right now, the error being returned is a invalid load key '<', which we thought to be encoding but that didn't pay off | 14:14 |
| *** sdake has quit IRC | 14:14 | |
| *** yamamoto has joined #openstack-meeting-3 | 14:14 | |
| tosky | unless we encoded it in the wrong way, but it's difficult to say | 14:14 |
| *** yamamoto has quit IRC | 14:14 | |
| *** yamamoto has joined #openstack-meeting-3 | 14:15 | |
| tellesnobrega | the most consistent suggestion from python experts was to replace pickle with json, for safety, but I'm not sure how to do the proper serialization with json | 14:15 |
| tellesnobrega | did I miss anything tosky ? | 14:16 |
| tosky | I don't think so | 14:16 |
| tosky | I mean, nothing else to add | 14:17 |
| tellesnobrega | cool | 14:17 |
| tellesnobrega | maybe jeremy will read and come up with something to help | 14:17 |
| *** jeremyfreudberg has joined #openstack-meeting-3 | 14:18 | |
| jeremyfreudberg | sorry, forgot about the meeting... | 14:18 |
| jeremyfreudberg | anything for me to weigh in on? | 14:18 |
| fungi | replacing pickle with structured data serialization | 14:18 |
| *** markvoelker has joined #openstack-meeting-3 | 14:18 | |
| * fungi is lurking | 14:18 | |
| tellesnobrega | fungi, we need help on that | 14:19 |
| tellesnobrega | anyone who is willing to do so is more than welcome | 14:19 |
| *** yamamoto has quit IRC | 14:19 | |
| fungi | oh, i was answering jeremyfreudberg's question | 14:19 |
| tellesnobrega | oh, ok | 14:20 |
| tellesnobrega | jeremyfreudberg, we are kind of blocked on python3 pickle issue | 14:20 |
| fungi | though i am interested in it from the vmt perspective simply because there are a number of potential risks with pickle use, i doubt i have time to implement anything there | 14:20 |
| fungi | seeing more occurrences of pickle get squashed in favor of safer serialization would be nice | 14:21 |
| tosky | fungi: do you know of examples of such replacements that already happened in openstack (or anywhere else)? | 14:21 |
| tellesnobrega | fungi, we understand that, and I'm willing to replace it, if I can get some help understanding how to do it | 14:22 |
| fungi | tosky: i don't have any specific examples off the top of my head though i think there have been at least a few over the years | 14:22 |
| fungi | do you pass complex objects and/or functions over pickle, or just basic data types? | 14:23 |
| tellesnobrega | functions | 14:23 |
| tellesnobrega | but iiuc the serialization is just a path of the function | 14:24 |
| fungi | in that case it may not be any safer security-wise if you want to be able to send arbitrary functions to a subprocess for execution (avoiding doing that "accidentally" is the main reason most people warn against using pickle, after all) | 14:25 |
| tellesnobrega | fungi, here is an example of what happens in sahara | 14:25 |
| tellesnobrega | >>> ser = pickle.dumps(identity) | 14:25 |
| tellesnobrega | >>> ser | 14:25 |
| tellesnobrega | ‘c__main__\nidentity\np0\n.’ | 14:25 |
| tellesnobrega | but we do it on subprocess stdin | 14:26 |
| fungi | yeah, i mean in this case the risk is low anyway because you're (hopefully) never going to have untrusted pickle data passed into these subprocesses anyway | 14:28 |
| fungi | i expect you're fighting the use of binary sequences for file descriptors in python3 | 14:28 |
| tellesnobrega | there are only a few cases of that happening and none are controlled by the user | 14:28 |
| tosky | our case is complicated by the fact that we trasmit the python code from python3 to python2 | 14:29 |
| tosky | as most of the hadoop vendors still run (and will run for a while) on older distributions | 14:29 |
| *** jamesmcarthur has joined #openstack-meeting-3 | 14:29 | |
| fungi | oh, in that case the pickle is likely just not valid at all. i don't know that python2 can be expected to even make use of a pickle payload from python3 | 14:30 |
| fungi | has anyone looked into whether that's supported? | 14:30 |
| tellesnobrega | even we specify the protocol? | 14:30 |
| jeremyfreudberg | tosky, are we sure that's what's happening? i thought the pickling was just between sahara-engine and _sahara-subprocess , no depickling done on the instances | 14:31 |
| jeremyfreudberg | but i might have missed something | 14:31 |
| *** sdake has joined #openstack-meeting-3 | 14:31 | |
| tosky | oh | 14:31 |
| tosky | uhm, maybe | 14:31 |
| tosky | better if I just follow the discussion :) | 14:31 |
| tellesnobrega | jeremyfreudberg, when we call _execute_command("ls .ssh/authorized_keys") doesn't it run on the instances? | 14:32 |
| fungi | ahh, yeah looks like perhaps if you write the pickle as data stream format #2 in python3 then python2.3 and later should be able to read it | 14:32 |
| tellesnobrega | fungi, I have set pickle to use protocol=2, it solved some issues, but the ones we are seeing now came up | 14:33 |
| *** jamesmcarthur has quit IRC | 14:34 | |
| jeremyfreudberg | tellesnobrega: with _execute_command i think that is after the depickling has already happened (we open an ssh session with paramiko and the command is just a string) | 14:34 |
| fungi | yeah, pickle is pretty opaque without digging into pep 307 and earlier relevant protocol specifications it builds on | 14:35 |
| fungi | so hard to say | 14:35 |
| jeremyfreudberg | tellesnobrega, i will look a little closer, there may be some info that i am missing | 14:36 |
| tellesnobrega | jeremyfreudberg, thanks, please do, I might be mistaken as well | 14:36 |
| fungi | as far as the "invalid load key '<'" exceptions | 14:36 |
| tellesnobrega | what I saw is the same pickling path for bot _connect (when we open the ssh connection) and the _execute_command | 14:37 |
| jeremyfreudberg | tellesnobrega: what's "path"? | 14:37 |
| tellesnobrega | jeremyfreudberg, I mean, it goes through the same logic, pickling the function and sending to subprocess stdin and same for args and kwargs | 14:39 |
| jeremyfreudberg | oh, got it | 14:39 |
| *** hongbin has joined #openstack-meeting-3 | 14:42 | |
| *** hongbin has quit IRC | 14:42 | |
| jeremyfreudberg | tosky, tellesnobrega, has anyone actually looked at the full traceback with the "Ran out of input" error? | 14:44 |
| jeremyfreudberg | (the traceback is suppressed in the logs) | 14:44 |
| tosky | I did not | 14:44 |
| tellesnobrega | me neither | 14:44 |
| tosky | uhm, if it's suppressed, where should it be visible? | 14:44 |
| tellesnobrega | I think I saw it once, while running with debug, but not much was useful, but you can try as well | 14:45 |
| jeremyfreudberg | https://github.com/openstack/sahara/blob/8659169c84c9a2198d5aee9e94c6d145a1f8d93c/sahara/service/engine.py#L125 | 14:45 |
| tosky | ah, removing that exception, or printing more details | 14:47 |
| jeremyfreudberg | yup | 14:48 |
| jeremyfreudberg | instead of the repr of the exception, the entire trace would be helpful, at least to me | 14:49 |
| tellesnobrega | jeremyfreudberg, I will run again and get that exception | 14:50 |
| tellesnobrega | and send by email | 14:50 |
| jeremyfreudberg | tellesnobrega: great, thanks! | 14:50 |
| jeremyfreudberg | (no need for it in the gate, it's just the one spot) | 14:50 |
| *** mjturek has joined #openstack-meeting-3 | 14:51 | |
| tellesnobrega | ok | 14:51 |
| *** sdake has quit IRC | 14:51 | |
| *** markvoelker has quit IRC | 14:53 | |
| jeremyfreudberg | any other topics for these last few minutes? | 14:53 |
| tosky | there was an interesting discussion yesterday evening on #openstack-sahara about the dashboard, please check the logs | 14:54 |
| * jeremyfreudberg looking now | 14:54 | |
| tosky | I fixed the native integration tests again following your suggestions (now experimental), ready to be merged and backported to remove the legacy job | 14:54 |
| *** yamamoto has joined #openstack-meeting-3 | 14:57 | |
| jeremyfreudberg | right now every test fails, right? or are there a very small number which pass | 14:57 |
| jeremyfreudberg | in any case, ivan is in enough agreement that we can merge the patch | 14:57 |
| tosky | all of them fails, for different reasons :) | 14:58 |
| tosky | but they are executed | 14:58 |
| jeremyfreudberg | yup, i see. 8 tests run and 8 tests failed :) | 14:58 |
| tellesnobrega | cool | 14:58 |
| jeremyfreudberg | so let's merge that | 14:58 |
| tellesnobrega | yes | 14:58 |
| jeremyfreudberg | we're at the hour | 14:59 |
| tellesnobrega | yes | 14:59 |
| tellesnobrega | thanks jeremyfreudberg and tosky | 14:59 |
| tellesnobrega | and thanks fungi as well | 14:59 |
| tellesnobrega | lets see if we can figure out the python3 issue this week | 15:00 |
| tellesnobrega | see you all next week | 15:00 |
| tellesnobrega | #endmeeting | 15:00 |
| tosky | see you! | 15:00 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 15:00 | |
| openstack | Meeting ended Thu Feb 21 15:00:11 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:00 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/sahara/2019/sahara.2019-02-21-14.00.html | 15:00 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/sahara/2019/sahara.2019-02-21-14.00.txt | 15:00 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/sahara/2019/sahara.2019-02-21-14.00.log.html | 15:00 |
| *** tosky has left #openstack-meeting-3 | 15:00 | |
| *** jeremyfreudberg has left #openstack-meeting-3 | 15:00 | |
| *** tellesnobrega has left #openstack-meeting-3 | 15:00 | |
| *** hongbin has joined #openstack-meeting-3 | 15:00 | |
| *** yamamoto has quit IRC | 15:01 | |
| *** sdake has joined #openstack-meeting-3 | 15:02 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 15:10 | |
| *** Luzi has quit IRC | 15:12 | |
| *** jamesmcarthur has quit IRC | 15:17 | |
| *** aagate has joined #openstack-meeting-3 | 15:17 | |
| *** xubozhang has joined #openstack-meeting-3 | 15:24 | |
| *** xubozhang has quit IRC | 15:31 | |
| *** apetrich has quit IRC | 15:36 | |
| *** xubozhang has joined #openstack-meeting-3 | 15:37 | |
| *** apetrich has joined #openstack-meeting-3 | 15:41 | |
| *** liuyulong has quit IRC | 15:43 | |
| *** liuyulong has joined #openstack-meeting-3 | 15:43 | |
| *** sdake has quit IRC | 15:48 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 15:48 | |
| *** markvoelker has joined #openstack-meeting-3 | 15:50 | |
| *** sdake has joined #openstack-meeting-3 | 15:50 | |
| *** jamesmcarthur has quit IRC | 15:53 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 15:58 | |
| *** jamesmcarthur has quit IRC | 15:58 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 15:58 | |
| *** moguimar has quit IRC | 15:59 | |
| *** moguimar has joined #openstack-meeting-3 | 15:59 | |
| *** diablo_rojo has joined #openstack-meeting-3 | 16:00 | |
| *** moguimar has quit IRC | 16:01 | |
| *** moguimar has joined #openstack-meeting-3 | 16:02 | |
| *** Gaasmann has joined #openstack-meeting-3 | 16:06 | |
| *** Gaasmann has left #openstack-meeting-3 | 16:06 | |
| *** jamesmcarthur has quit IRC | 16:13 | |
| *** e0ne has quit IRC | 16:23 | |
| *** markvoelker has quit IRC | 16:23 | |
| *** macza has joined #openstack-meeting-3 | 16:24 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 16:27 | |
| *** yamamoto has joined #openstack-meeting-3 | 16:45 | |
| *** lpetrut has quit IRC | 16:46 | |
| *** lpetrut has joined #openstack-meeting-3 | 16:49 | |
| *** yamamoto has quit IRC | 16:50 | |
| *** pcaruana has quit IRC | 17:07 | |
| *** tssurya has quit IRC | 17:08 | |
| *** moguimar has quit IRC | 17:17 | |
| *** yamamoto has joined #openstack-meeting-3 | 17:20 | |
| *** markvoelker has joined #openstack-meeting-3 | 17:20 | |
| *** jamesmcarthur_ has joined #openstack-meeting-3 | 17:24 | |
| *** yamamoto has quit IRC | 17:25 | |
| *** jamesmcarthur has quit IRC | 17:27 | |
| *** yamamoto has joined #openstack-meeting-3 | 17:47 | |
| *** markvoelker has quit IRC | 17:53 | |
| *** lpetrut has quit IRC | 17:54 | |
| *** sdake has quit IRC | 18:05 | |
| *** sdake has joined #openstack-meeting-3 | 18:05 | |
| *** igordc has joined #openstack-meeting-3 | 18:09 | |
| *** markvoelker has joined #openstack-meeting-3 | 18:50 | |
| *** ralonsoh has quit IRC | 18:56 | |
| *** liuyulong has quit IRC | 19:13 | |
| *** yamahata__ has quit IRC | 19:14 | |
| *** manjeets has quit IRC | 19:14 | |
| *** yamahata__ has joined #openstack-meeting-3 | 19:14 | |
| *** manjeets has joined #openstack-meeting-3 | 19:14 | |
| *** markvoelker has quit IRC | 19:23 | |
| *** jamesmcarthur_ has quit IRC | 19:56 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 19:57 | |
| *** jamesmcarthur has quit IRC | 20:01 | |
| *** e0ne has joined #openstack-meeting-3 | 20:14 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 20:18 | |
| *** markvoelker has joined #openstack-meeting-3 | 20:21 | |
| *** e0ne has quit IRC | 20:37 | |
| *** jamesmcarthur has quit IRC | 20:40 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 20:40 | |
| *** jamesmcarthur has quit IRC | 20:45 | |
| *** macza has quit IRC | 20:45 | |
| *** macza has joined #openstack-meeting-3 | 20:45 | |
| *** _pewp_ has quit IRC | 20:48 | |
| *** _pewp_ has joined #openstack-meeting-3 | 20:48 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 20:53 | |
| *** markvoelker has quit IRC | 20:55 | |
| *** macza has quit IRC | 21:19 | |
| *** slaweq has quit IRC | 21:35 | |
| *** slaweq has joined #openstack-meeting-3 | 21:48 | |
| *** markvoelker has joined #openstack-meeting-3 | 21:53 | |
| *** macza has joined #openstack-meeting-3 | 21:53 | |
| *** macza has quit IRC | 21:55 | |
| *** macza has joined #openstack-meeting-3 | 21:55 | |
| *** jamesmcarthur has quit IRC | 21:57 | |
| *** jamesmcarthur has joined #openstack-meeting-3 | 22:05 | |
| *** jamesmcarthur has quit IRC | 22:07 | |
| *** markvoelker has quit IRC | 22:25 | |
| *** sdake has quit IRC | 22:27 | |
| *** slaweq has quit IRC | 22:43 | |
| *** slaweq has joined #openstack-meeting-3 | 22:55 | |
| *** slaweq has quit IRC | 23:00 | |
| *** slaweq has joined #openstack-meeting-3 | 23:11 | |
| *** sdake has joined #openstack-meeting-3 | 23:12 | |
| *** slaweq has quit IRC | 23:16 | |
| *** markvoelker has joined #openstack-meeting-3 | 23:22 | |
| *** igordc has quit IRC | 23:30 | |
| *** sdake has quit IRC | 23:33 | |
| *** sdake has joined #openstack-meeting-3 | 23:38 | |
| *** raildo has quit IRC | 23:39 | |
| *** sdake has quit IRC | 23:46 | |
| *** sdake has joined #openstack-meeting-3 | 23:48 | |
| *** sdake has quit IRC | 23:51 | |
| *** sdake_ has joined #openstack-meeting-3 | 23:51 | |
| *** markvoelker has quit IRC | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!