| *** Qiming has quit IRC | 00:02 | |
| *** Qiming has joined #senlin | 01:11 | |
| *** Yanyanhu has joined #senlin | 01:15 | |
| Qiming | Hi, Yanyanhu | 01:24 |
|---|---|---|
| Yanyanhu | hi | 01:24 |
| Qiming | https://review.openstack.org/#/c/209736/2/jenkins/jobs/senlin.yaml | 01:24 |
| Qiming | line 21 | 01:25 |
| Qiming | is the $ sign necessary? | 01:25 |
| Qiming | it is a weird syntax to me | 01:25 |
| Yanyanhu | em, seems so, since without it, we can't get correct DEVSTACK_LOCAL_CONFIG | 01:26 |
| Yanyanhu | also found other projects used the similar way to add more than one line into DEVSTACK_LOCAL_CONFIG | 01:26 |
| Qiming | okay, just want to make usre it is not an ignorance | 01:26 |
| Yanyanhu | yes | 01:27 |
| openstackgerrit | Merged stackforge/senlin: Do not update other properties if exception happens https://review.openstack.org/209026 | 01:28 |
| *** Tennyson has joined #senlin | 01:37 | |
| *** ChrisSen has joined #senlin | 01:38 | |
| Yanyanhu | hi, Qiming, about the functional test, have made some investigations but still didn't find a good way to fake the interaction between Senlin and other services... | 01:45 |
| Qiming | okay, keep digging then, :) | 01:46 |
| *** mathspanda has joined #senlin | 01:46 | |
| Qiming | it's not that easy a task I know | 01:46 |
| Yanyanhu | hmm, I'm thinking maybe we just put those cases where Senlin need to talk with other services into integration test? | 01:46 |
| *** Qiming has quit IRC | 01:48 | |
| *** Qiming has joined #senlin | 01:48 | |
| Qiming | well .... then there would be many workflows not tested in the 'functional' tests | 01:49 |
| Yanyanhu | that's true | 01:49 |
| Yanyanhu | and about functional test in other services, I found nova/cinder are using fixture to build their functional tests, but using that way, they fake the entire nova/cinder services I think... and may need to reorg the design of our service to support this way | 01:50 |
| Yanyanhu | actually I think nova/cinder's functional test is actually an enhanced version of unit test :) | 01:51 |
| Qiming | that sounds the right thing to do though | 01:51 |
| Qiming | maybe we can use fixtures just for sdk? | 01:52 |
| Yanyanhu | em, but that's not the functional test we discussed I think, they don't run nova/cinder in real environment actually | 01:52 |
| Qiming | .... | 01:52 |
| Qiming | we fake sdk's calls as if compute is there, orchestration is there | 01:52 |
| Qiming | is this possible? | 01:53 |
| Yanyanhu | I'm afriad we can't since they need fake the entire nova/cinder services to achieve this goal... | 01:53 |
| Yanyanhu | then they can control every methods called in services | 01:53 |
| Qiming | what do you mean by fake the nova service? | 01:53 |
| Qiming | given that your goal is to test the nova service? | 01:54 |
| Yanyanhu | will go downstairs, back soon | 01:54 |
| Yanyanhu | hi, I'm back | 01:59 |
| Yanyanhu | I'm not quite sure how to use this way, but it seems that they run a 'fake' nova/cinder service using fixture support | 02:00 |
| Yanyanhu | then mock methods when necessary | 02:01 |
| Yanyanhu | actually, they mainly use this to support their unit test | 02:01 |
| Qiming | that is where we need to dig ? | 02:01 |
| Yanyanhu | em, yes | 02:01 |
| Qiming | we borrow but we don't just copy, right? | 02:01 |
| Yanyanhu | but I think maybe it's not a proper way for us | 02:02 |
| Qiming | maybe | 02:02 |
| Qiming | just don't know | 02:02 |
| Yanyanhu | since for nova/cinder, they only have little requirement to talk with other services | 02:02 |
| Yanyanhu | this is different from us | 02:02 |
| Qiming | right, what we need is to make a fake sdk | 02:03 |
| Yanyanhu | so the scale of their functional test is small | 02:03 |
| Qiming | just like you have made a "TestProfile" | 02:03 |
| Yanyanhu | yes, that's what we want | 02:03 |
| Qiming | we borrow, but we don't just copy | 02:03 |
| Yanyanhu | just not sure we can do that without mock the entire services, since sdk is a lib actually | 02:03 |
| Qiming | how many interfaces are we actually using? | 02:04 |
| Yanyanhu | if so, maybe fake the driver is a better way? | 02:04 |
| Yanyanhu | em, if we want to test all policies and profile types, I think most of the methods in driver may need to be faked | 02:05 |
| Yanyanhu | anyway, will keep investigating this to see whether we can just fake part of the service using this way | 02:05 |
| Yanyanhu | if so, that would be the best I think | 02:06 |
| Qiming | we just fake/mock what is called | 02:06 |
| Yanyanhu | yes | 02:06 |
| Qiming | e.g. server_interfaces? keypairs? images? .... | 02:06 |
| Qiming | there are so many things we are not using today | 02:06 |
| Yanyanhu | if our driver is implemented as plugin, that's much better :) | 02:06 |
| *** jqiu has joined #senlin | 02:07 | |
| *** ChrisSen has joined #senlin | 02:07 | |
| *** jqiu has left #senlin | 02:07 | |
| Qiming | when we are using them someday, we fake them then | 02:07 |
| Yanyanhu | yes | 02:08 |
| Yanyanhu | just not sure whether we can fake a method inside a service that running in the real environment | 02:08 |
| Qiming | google it? | 02:09 |
| Yanyanhu | let me make more deep digging | 02:09 |
| Qiming | if there are evidences that is impractical | 02:09 |
| Yanyanhu | can't access it... sigh, so just used Bing | 02:09 |
| Qiming | .... I mean "google" ... | 02:09 |
| Qiming | it would be great if we can switch modules without restarting a service | 02:10 |
| Yanyanhu | yes, definitely | 02:11 |
| Qiming | if that is proved not possible, we switch between modules and restart the service | 02:11 |
| Yanyanhu | ok | 02:11 |
| Qiming | for some projects, they even provide a different config file, even a different service wrapper for this purpose | 02:11 |
| Yanyanhu | service wrapper is? | 02:12 |
| Qiming | even that has to be done, okay, we'd better make the changes less intrusive | 02:12 |
| Yanyanhu | yes | 02:13 |
| Qiming | the so called 'fake service' | 02:13 |
| Yanyanhu | ok | 02:13 |
| Qiming | you got to control which part you will run the real code, which part you will run the mocks | 02:13 |
| Yanyanhu | understand | 02:15 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Tune scaling_in policy unit tests https://review.openstack.org/210185 | 02:47 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Tune scaling_in policy unit test https://review.openstack.org/210186 | 02:55 |
| ChrisSen | I tried to create a cluster with a specified capacity of 2 nodes. One node was created successfully, but the creation of the second one failed with an error. But the error does not seem to be reproduceable, since the creation of the cluster succeeded entirely when I tried it again: | 03:15 |
| ChrisSen | http://paste.openstack.org/show/412146/ | 03:15 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Fix version middleware and test case https://review.openstack.org/210188 | 03:17 |
| Qiming | ChrisSen, looks like a intermittent or sporadic event? | 03:18 |
| ChrisSen | I think so. Yesterday the test was successful, today the error occurred only one time. | 03:20 |
| ChrisSen | I can keep an eye on it and will tell you, if it happens again. | 03:23 |
| *** ChrisSen has quit IRC | 04:37 | |
| *** jruano has joined #senlin | 04:39 | |
| *** jdandrea has quit IRC | 04:39 | |
| *** ChrisSen has joined #senlin | 04:53 | |
| *** jruano has quit IRC | 04:58 | |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Improve trigger base test case coverage https://review.openstack.org/210199 | 05:05 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Improve event unit test coverage https://review.openstack.org/210201 | 05:11 |
| Yanyanhu | hi, Qiming, the commit msg and title of this patch is incorrect https://review.openstack.org/#/c/210186/ | 05:24 |
| Qiming | ok, fixed | 05:31 |
| openstackgerrit | Merged stackforge/senlin: Fix coverage test https://review.openstack.org/209936 | 05:33 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Tune scaling_out policy unit test https://review.openstack.org/210186 | 05:33 |
| Yanyanhu | hi, Qiming, about the reload support in wsgi, the reload operation is triggered by a SIGHUPInterrupt? | 05:48 |
| Qiming | yes | 05:49 |
| Yanyanhu | ok | 05:49 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Fix profiles API and test cases https://review.openstack.org/210205 | 05:50 |
| ChrisSen | In the following scenario I tried to resize a cluster, but the cluster_resize action waits for one node_create action even though all nodes were created successfully (infinite loop). When you try to delete one node of the cluster a deadlock is caused, because the cluster is already locked by the never ending cluster_resize action. | 06:09 |
| ChrisSen | http://paste.openstack.org/show/412150/ | 06:09 |
| Qiming | ChrisSen, thanks for the analysis | 06:12 |
| Qiming | it seems that all the later errors were caused by the "unsuccessful" CLUSTER_RESIZE operation | 06:13 |
| Qiming | can we nail down the problem with a shorter story? | 06:14 |
| Qiming | e.g. create an empty cluster, resize it to 2 nodes | 06:15 |
| Qiming | then check if the CLUSTER_RESIZE operation was completed, i.e. with lock released | 06:15 |
| ChrisSen | I can test that. | 06:15 |
| Yanyanhu | hi, Qiming, made a test and found oslo_service has supported restart service and reload configure. | 06:16 |
| Yanyanhu | just need to run service with deamon mode and send SIGHUP to it :) | 06:16 |
| Qiming | smells like a concurrency control problem | 06:16 |
| Yanyanhu | e.g. kill -1 | 06:16 |
| Qiming | \o/ | 06:16 |
| Qiming | wanna dance for that | 06:16 |
| Yanyanhu | yep :) | 06:17 |
| Qiming | that sounds beautiful, isn't it | 06:17 |
| Yanyanhu | so will try to rework driver using plugin mode | 06:17 |
| Qiming | cool | 06:17 |
| Yanyanhu | yes, much better than the current way | 06:17 |
| Yanyanhu | will file a bp for this work first | 06:17 |
| Qiming | yes, we borrow from others but we always try work the right way, :) | 06:18 |
| Qiming | cool | 06:18 |
| Yanyanhu | yes :) | 06:18 |
| *** wizardYVE has joined #senlin | 06:29 | |
| openstackgerrit | Merged stackforge/senlin: Revise server delete-wait logic https://review.openstack.org/209899 | 06:31 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Fix nodes API and test cases https://review.openstack.org/210221 | 06:49 |
| *** xuhaiwei has quit IRC | 07:05 | |
| *** xuhaiwei has joined #senlin | 07:12 | |
| ChrisSen | Qiming, the creation of an empty cluster and resizing it to 2 nodes was successful (with lock released). But after that I was not able to create any node any more (neither with node-create, nor with cluster-create , nor with cluster-resize), it always fails with "KeyError: 'error'". | 07:16 |
| Qiming | umm | 07:16 |
| ChrisSen | I can restart my VM to run further tests for creating and resizing clusters and will try to summarize the findings. | 07:17 |
| Qiming | thx, ChrisSen | 07:17 |
| openstackgerrit | Qiming Teng proposed stackforge/senlin: Fix errors in policies API and test cases https://review.openstack.org/210229 | 07:27 |
| Yanyanhu | hi, Qiming, found my test environment break. Seems some werid tmp directories wee added to load_list of subunit.run | 07:34 |
| Yanyanhu | error like this: | 07:34 |
| Yanyanhu | ${PYTHON} -m subunit.run discover -s . --load-list /tmp/tmpzAu3rk | 07:34 |
| Yanyanhu | did this happen in your env? | 07:34 |
| Qiming | no | 07:34 |
| Qiming | check if your .testrconf is correct | 07:35 |
| Qiming | maybe do a rm -fr .testrepository | 07:35 |
| Yanyanhu | ok, let me have a try | 07:35 |
| Yanyanhu | it worked will yesterday | 07:35 |
| Yanyanhu | it works | 07:37 |
| Yanyanhu | why clean .testrepository can solve the problem? | 07:37 |
| Yanyanhu | the number of unit test cases in my env is 1111 now :) | 07:39 |
| openstackgerrit | Yanyan Hu proposed stackforge/senlin: Add environment support for driver plugin https://review.openstack.org/210232 | 07:39 |
| Qiming | hi, yanyan | 07:42 |
| Yanyanhu | hi | 07:42 |
| Qiming | looking at your patch | 07:42 |
| Yanyanhu | ok | 07:42 |
| Qiming | commenting here instead of on gerrit | 07:42 |
| Qiming | custom_drivers seems not necessary | 07:43 |
| Yanyanhu | ok | 07:43 |
| Qiming | the likelyhood of having users to contribute new cloud drivers is low | 07:43 |
| Qiming | so we may don't need that | 07:44 |
| Qiming | remove CUSTOM_DRIVERS section then | 07:44 |
| Yanyanhu | ok, understand | 07:44 |
| Qiming | it is not about 'drivers' I think | 07:44 |
| Qiming | it is more about a 'cloud_backends' | 07:45 |
| Qiming | for example, we have 'openstack' as one option under /drivers/openstack | 07:45 |
| Yanyanhu | yes | 07:46 |
| Qiming | so ... maybe we can learn from sdk's design | 07:46 |
| Yanyanhu | but I think we can only manage each individual plugin? | 07:46 |
| Qiming | we get a backend, and having each backend provide an attribute 'compute', an attribute 'network' ... | 07:47 |
| Qiming | just like sdk.conn.compute.create_server | 07:47 |
| *** ChrisSen has quit IRC | 07:47 | |
| Qiming | in that case, we don't need many drivers for each service | 07:48 |
| Yanyanhu | ok, make sense. but maybe we can't ensure the second level module can have the same method? | 07:48 |
| Qiming | by default, we raise not supported | 07:48 |
| Qiming | not implemented ... | 07:49 |
| Yanyanhu | em, so this will be a real driver | 07:49 |
| Yanyanhu | not like the current one, which only make simple interface mapping | 07:50 |
| Yanyanhu | understand | 07:50 |
| Qiming | so we treat all modules from under openstack/ as a single driver | 07:50 |
| Qiming | making them each look like a different driver would not buy us anything, just problems | 07:51 |
| Qiming | one driver has create_server, the other driver has create_alarm | 07:51 |
| Yanyanhu | right | 07:51 |
| Qiming | you will need "driver type" .... a nightmare | 07:52 |
| Yanyanhu | so, for server creation, we always invoke cloud_backend_name.compute.create_server | 07:52 |
| Qiming | right | 07:52 |
| Yanyanhu | with the same parameters | 07:52 |
| Qiming | in an ideal world | 07:52 |
| Yanyanhu | em, looks nice | 07:52 |
| Qiming | we can eliminate the senlin/drivers/openstack dir | 07:52 |
| Qiming | we call sdk directly | 07:53 |
| Qiming | we still maintain that dir for reasons of stability (exception catching), independence (we don't rely on its success) ... | 07:53 |
| Yanyanhu | actually, in my mind I thought the drivers/openstack will contains several folders including 'compute', 'network', 'identity'... | 07:55 |
| Qiming | when the 'cloud_backend' is set to 'openstack', we rely on openstacksdk, when it is set to something else, hopefully, there will be a sdk for use ... | 07:55 |
| Yanyanhu | and also the same for drivers/aws or drivers/softlayer | 07:55 |
| Qiming | when there are more than one compute, more than one network service for openstack, that will make sense | 07:56 |
| Yanyanhu | you mean there is only one file under drivers/openstack/ path? | 07:56 |
| Yanyanhu | like drivers/openstack.py | 07:56 |
| Yanyanhu | or drivers/softlayer.py | 07:56 |
| Qiming | ... | 07:56 |
| Qiming | drivers/openstack/ will be left there | 07:56 |
| Qiming | why are we combining the existing drivers into a single file? | 07:57 |
| Yanyanhu | oh, you mean we just create a new directory to place these new drivers | 07:57 |
| Qiming | just the drivers/ directory would be fine, right? | 07:58 |
| Yanyanhu | yes | 07:58 |
| Qiming | senlin | 08:02 |
| Qiming | `--drivers | 08:02 |
| Qiming | +--- generic intefaces | 08:02 |
| Qiming | +--- and driver modules | 08:02 |
| Qiming | +--- openstack | 08:02 |
| Qiming | | +-- nova_v2 | 08:02 |
| Qiming | | +-- keystone_v3 | 08:02 |
| Qiming | | `-- neutron_v2 | 08:02 |
| Qiming | `--- softlayer | 08:02 |
| Qiming | +-- compute_v1 | 08:02 |
| Qiming | `-- metric_v5 | 08:02 |
| *** ChrisSen has joined #senlin | 08:06 | |
| openstackgerrit | Yanyan Hu proposed stackforge/senlin: Add environment support for driver plugin https://review.openstack.org/210232 | 08:20 |
| openstackgerrit | xu-haiwei proposed stackforge/senlin: Revise senlin senlin exception handling from SDK side https://review.openstack.org/210254 | 08:42 |
| openstackgerrit | xu-haiwei proposed stackforge/senlin: Revise senlin exception handling from SDK side https://review.openstack.org/210254 | 08:46 |
| openstackgerrit | Merged stackforge/senlin: Add environment support for driver plugin https://review.openstack.org/210232 | 08:46 |
| xuhaiwei | Qiming, about the patch I just submitted, just a try to do it, no comment about it? | 08:49 |
| Qiming | looks tood | 08:50 |
| Qiming | good | 08:50 |
| xuhaiwei | ok, since it is passed, will do more jobs about it in the next week | 08:51 |
| xuhaiwei | I will leave now, have a good weekend guys | 08:52 |
| xuhaiwei | see you next week | 08:52 |
| Qiming | you too | 08:52 |
| *** xuhaiwei has quit IRC | 08:52 | |
| Yanyanhu | this guys left too soon, even didn't get chance to say byebye to him :) | 08:54 |
| openstackgerrit | Merged stackforge/senlin: Revise senlin exception handling from SDK side https://review.openstack.org/210254 | 08:57 |
| openstackgerrit | Yanyan Hu proposed stackforge/senlin: Add cloud_backend_name option into Senlin config https://review.openstack.org/210257 | 08:59 |
| Yanyanhu | hi, Qiming, about this patch which tries to add the cloud_backend_name option https://review.openstack.org/210257 | 09:00 |
| Qiming | Yanyanhu, seems you are "putting the horses bind the carts" | 09:00 |
| Yanyanhu | ah? | 09:01 |
| Qiming | 210257 is supposed to be te last patch, right? | 09:01 |
| Yanyanhu | oh, just start of the blueprint I think | 09:02 |
| Yanyanhu | will also added generic driver implementation | 09:02 |
| Qiming | right | 09:02 |
| Qiming | once you have a generic driver implementation | 09:02 |
| Qiming | this option value will be used | 09:03 |
| Yanyanhu | yes | 09:03 |
| Yanyanhu | you mean I shouldn't use the same branch to propose multiple patches? | 09:05 |
| *** mathspanda has quit IRC | 09:12 | |
| Qiming | no | 09:14 |
| Qiming | maybe I was wrong | 09:14 |
| Qiming | this option should be there in the first place, then the driver interface would determin which backend to use | 09:14 |
| Yanyanhu | yes, that's right I think | 09:15 |
| Yanyanhu | and the option should define the default cloud_backend senlin will talk with when invoking driver methods | 09:16 |
| Qiming | yes | 09:16 |
| Yanyanhu | but if cloud_bakend_name is provided in e.g. profile, we will use the specified one to get the correct driver module | 09:16 |
| Yanyanhu | so we can talk to multiple clouds at the same time | 09:17 |
| Qiming | cloud_backend_name provided in profile? | 09:17 |
| Yanyanhu | maybe not this name | 09:17 |
| Yanyanhu | since I think a profile should be able to decide which cloud it will talk with | 09:17 |
| Yanyanhu | in that case, we need to load correct driver based on profile definition | 09:17 |
| Qiming | that is called context | 09:18 |
| Yanyanhu | yes | 09:18 |
| Yanyanhu | the context | 09:18 |
| Qiming | we don't know what other clouds would look like yet | 09:19 |
| Yanyanhu | yes, but we can know its name and if we have a driver plugin for that cloud, we should be able to get it based on the name? | 09:20 |
| *** Tennyson has quit IRC | 09:20 | |
| Qiming | backend is about the cloud type | 09:21 |
| Yanyanhu | maybe not the name, could be a unique symbol we can identify it | 09:21 |
| Yanyanhu | yes | 09:21 |
| Qiming | you are talking about a cloud instance | 09:21 |
| Qiming | that will be indicated via some auth_url kind of thing | 09:21 |
| Yanyanhu | right | 09:21 |
| Yanyanhu | for serveral cloud instances with the same type, we just use the same drivers | 09:22 |
| Qiming | link to stackforge namespace patch? | 09:22 |
| Qiming | yes | 09:22 |
| Qiming | driver is a implementation thing | 09:22 |
| Qiming | we will figure out the interface for users when there is a need | 09:22 |
| Qiming | when we really have multiple implementations | 09:22 |
| Qiming | link to stackforge namespace patch? | 09:23 |
| Yanyanhu | stackforge namespace patch is? | 09:23 |
| Qiming | ... | 09:24 |
| Qiming | the patch about deprecating stackforge namespace | 09:24 |
| Qiming | I cannot find the linke | 09:24 |
| Yanyanhu | oh | 09:24 |
| Yanyanhu | sigh... | 09:24 |
| Yanyanhu | https://review.openstack.org/192016 | 09:24 |
| Yanyanhu | I thought you said we should do something about 'link to stackforge namespace patch' when new driver implementation is added... | 09:25 |
| Yanyanhu | -_-! | 09:25 |
| Qiming | ... | 09:29 |
| ChrisSen | Qiming, I retestet the cluster create and resize operations and met the same problems as before: 1. Creating an empty cluster and resizing it to 2 nodes succeeds. 2. Creating a cluster with minimum, maximum and default capacity and resizing it causes an infinite loop - either the cluster_create or the cluster_resize action waits for the node_create action to be finished even though this operation already succeeded. 3. After a while it is not possible t | 09:34 |
| ChrisSen | o create any nodes any more because node_create causes a "KeyError: 'error'". I could observe this behavior several times, but it can not be forecasted 100%. | 09:34 |
| *** wizardYVE has quit IRC | 09:37 | |
| Qiming | okay, please help file a bug? | 09:38 |
| Qiming | need to check it when cycles allow | 09:39 |
| Qiming | the worst case would be some concurrency problem on locks | 09:39 |
| *** Qiming has quit IRC | 09:45 | |
| *** Yanyanhu has quit IRC | 09:45 | |
| *** Yanyanhu has joined #senlin | 09:45 | |
| *** Yanyanhu has quit IRC | 09:50 | |
| *** ChrisSen has quit IRC | 09:56 | |
| *** openstackgerrit has quit IRC | 10:45 | |
| *** openstackgerrit has joined #senlin | 10:49 | |
| *** mathspanda has joined #senlin | 11:19 | |
| *** mathspanda has quit IRC | 12:33 | |
| *** mathspanda has joined #senlin | 12:36 | |
| *** jruano has joined #senlin | 12:41 | |
| *** lixinhui has quit IRC | 13:01 | |
| *** lixinhui has joined #senlin | 13:02 | |
| *** jhroyal has joined #senlin | 13:03 | |
| *** jdandrea has joined #senlin | 13:21 | |
| *** jhroyal1 has joined #senlin | 14:03 | |
| *** jhroyal has quit IRC | 14:04 | |
| *** lkarm has joined #senlin | 14:21 | |
| *** lkarm has quit IRC | 14:28 | |
| openstackgerrit | LinPeiyu proposed stackforge/senlin: B64encode user_data content before handling it to nova https://review.openstack.org/209754 | 14:52 |
| *** mathspanda has quit IRC | 15:39 | |
| *** Qiming has joined #senlin | 15:48 | |
| *** mathspanda has joined #senlin | 15:56 | |
| *** mathspanda has quit IRC | 16:08 | |
| *** mathspanda has joined #senlin | 16:09 | |
| *** Qiming has quit IRC | 16:12 | |
| *** mathspan_ has joined #senlin | 16:12 | |
| *** mathspanda has quit IRC | 16:15 | |
| *** mathspan_ has quit IRC | 16:20 | |
| *** jdandrea has left #senlin | 18:10 | |
| *** jhroyal1 has quit IRC | 18:17 | |
| *** jhroyal has joined #senlin | 18:33 | |
| *** jhroyal has quit IRC | 20:15 | |
| *** jruano has quit IRC | 20:45 | |
| *** jruano has joined #senlin | 23:30 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!