| *** ganso has quit IRC | 01:30 | |
| *** tinwood has quit IRC | 01:50 | |
| *** tinwood has joined #openstack-manila | 01:51 | |
| *** markstur_ has quit IRC | 02:17 | |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Do not merge https://review.openstack.org/512193 | 02:31 |
|---|---|---|
| zhongjun | vponomaryov: Hi, There are some docs about share group features. Since you are familiar with this feature. Could you take a look it if you have free time. Thanks in advance https://review.openstack.org/#/q/status:open+project:openstack/manila+branch:master+topic:bug/1724716 | 03:00 |
| *** junboli has joined #openstack-manila | 03:06 | |
| *** Manor has joined #openstack-manila | 03:16 | |
| *** Manor has quit IRC | 03:20 | |
| *** wes_dillingham has quit IRC | 03:21 | |
| masber | good afternoon, I have a manila share | 03:46 |
| masber | I created an NFS manila share | 03:46 |
| masber | then I mounted on a couple of machines which works fine, however I am hitting a problem where I can't connect more than 4 devices | 03:50 |
| masber | I checked manila quota and I could not find a limit in the number of mounts a share can have | 03:51 |
| masber | any thoughts? | 03:51 |
| *** maysamacedos has joined #openstack-manila | 03:53 | |
| *** AlexeyAbashkin has joined #openstack-manila | 04:18 | |
| openstackgerrit | Merged openstack/puppet-manila master: Switch to Zuul v3 testing https://review.openstack.org/514992 | 04:18 |
| *** AlexeyAbashkin has quit IRC | 04:22 | |
| gouthamr | masber: manila is out of the data path, could there be an issue with the access-rules? | 04:40 |
| masber | gouthamr, dont think so. It looks like is related with the number of instances with the share mounted. | 04:41 |
| masber | the client hangs while trying to mount the partition | 04:42 |
| masber | but if I unmount the share from a different node, then the first attempt is successful | 04:43 |
| masber | so lets say I have 3 nodes A, B and C with the same share mounted | 04:43 |
| masber | if I try to mount a fourth node D it will hang | 04:43 |
| masber | but then I can unmount the share from C for instance and the D will mount | 04:44 |
| gouthamr | oh, what backend are you using? | 04:45 |
| *** markstur has joined #openstack-manila | 04:59 | |
| *** markstur_ has joined #openstack-manila | 05:01 | |
| *** markstur has quit IRC | 05:04 | |
| masber | gouthamr, ceph | 05:10 |
| gouthamr | paging tbarron / arnewiebalck ^ | 05:14 |
| *** arnewiebalck_ has joined #openstack-manila | 05:47 | |
| *** arnewiebalck_ has quit IRC | 05:51 | |
| *** pcaruana has joined #openstack-manila | 05:58 | |
| *** markstur_ has quit IRC | 06:01 | |
| *** markstur has joined #openstack-manila | 06:21 | |
| *** markstur has quit IRC | 06:26 | |
| *** dsariel__ has joined #openstack-manila | 06:52 | |
| *** Manor has joined #openstack-manila | 07:03 | |
| *** hoonetorg has quit IRC | 07:05 | |
| *** lpetrut has joined #openstack-manila | 07:17 | |
| *** jprovazn has joined #openstack-manila | 07:19 | |
| *** hoonetorg has joined #openstack-manila | 07:21 | |
| *** YuYangWang has joined #openstack-manila | 07:23 | |
| *** YuYangWang has quit IRC | 07:26 | |
| *** Manor_ has joined #openstack-manila | 07:30 | |
| *** Manor has quit IRC | 07:34 | |
| *** Manor has joined #openstack-manila | 07:53 | |
| *** Manor_ has quit IRC | 07:55 | |
| arnewiebalck | masber: Clients are using NFS to access a CephFS backend? | 07:56 |
| masber | arnewiebalck, mmm I created the share as NFS | 07:57 |
| masber | and just mount it | 07:57 |
| masber | but the backend is ceph | 07:57 |
| masber | I am not using cephfs | 07:57 |
| arnewiebalck | so, you’re on Pike | 07:57 |
| masber | I built the system using kolla-ansible | 07:58 |
| masber | arnewiebalck, yes Im on pike | 07:58 |
| *** AlexeyAbashkin has joined #openstack-manila | 07:58 | |
| arnewiebalck | We’re still on Ocata and haven’t tried the NFS re-export yet. Our clients are using cephfs directly. And they mount the same share from ~80 clients simultaneously. | 07:59 |
| arnewiebalck | I’d guess tbarron or rraja may be able to help. | 08:01 |
| arnewiebalck | Does the ganesha server say anything in its logs? | 08:01 |
| masber | ganesha? | 08:02 |
| arnewiebalck | From what I understand, using NFS as the protocol with a CephFS backend will launch an instance with Ganesha (a user-mode NFS server). | 08:03 |
| arnewiebalck | This server will talk to CephFS at the back and NFS to the clients. | 08:04 |
| arnewiebalck | This is how clients can talk NFS to a CephFS cluster. | 08:04 |
| arnewiebalck | If my understanding is correct, you should a have an instance in your tenant runnign the Ganesha server. | 08:05 |
| masber | arnewiebalck, we don't use cephfs, we don't have mds deployed | 08:07 |
| masber | only mgr and rgw | 08:08 |
| arnewiebalck | What’s the share_driver you’re using for your backend definition in your manila.conf ? | 08:11 |
| openstackgerrit | Tom Barron proposed openstack/puppet-manila master: Add support for address family parameters https://review.openstack.org/516449 | 08:17 |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Do not merge https://review.openstack.org/512193 | 08:24 |
| tbarron | masber: are you using the manila generic driver with NFS protocol for mounts from compute instances with a cinder storage back end where | 08:24 |
| tbarron | the cinder storage back end is using cinder rbd for storage? | 08:24 |
| *** lpetrut has quit IRC | 08:25 | |
| tbarron | that wouldn't use cephfs, or cephfs with nfs-ganesha gateway | 08:25 |
| masber | tbarron, I followed this tutorial https://docs.openstack.org/kolla-ansible/latest/reference/manila-guide.html | 08:26 |
| masber | I don't have cephfs for sure | 08:26 |
| masber | so I guess the storage is provided by rgw | 08:26 |
| tbarron | yes, you are using the generic driver | 08:27 |
| tbarron | that tutorial doesn't say anything about ceph | 08:27 |
| tbarron | but the generic driver gets its storage from cinder | 08:28 |
| tbarron | and cinder itself has different possibilities for setting up block storage back ends | 08:28 |
| tbarron | are you using ceph for the block storage back end for cinder? | 08:28 |
| tbarron | I'm just trying to figure out gouthamr:"what backend are you using?" masber:"ceph" | 08:30 |
| tbarron | From manila standpoint the back end you are using is "openstack nova and cinder" i.e. generic | 08:30 |
| tbarron | then cinder will have its own back end | 08:30 |
| *** lpetrut has joined #openstack-manila | 08:31 | |
| masber | tbarron, yes I am using ceph for cinder | 08:31 |
| tbarron | so I don't know *why* the fourth mount of a single share to A, B, C, D fails but the first three work. | 08:31 |
| tbarron | It seems that it's not because D is on some different network path than A, B, C because if you unmount A you can then get the mount to D to succeed. | 08:32 |
| masber | tbarron, correct | 08:33 |
| masber | plus all servers are in same network and project | 08:33 |
| tbarron | My guess is that it has to do with some limitation or error from the nova compute instance serving as "share server" and that you need to look at the logs on that share server to see why it refuses the fourth mount | 08:33 |
| masber | is there any logs I can collect for you? | 08:33 |
| tbarron | masber: well just show you know you got pointed my way b/c I work with the ceph and ceph/nfs drivers and this issue doesn't pertain to them. | 08:34 |
| tbarron | masber: not saying I won't look or help, just that I'm not an expert in this area. | 08:34 |
| openstackgerrit | zhongjun proposed openstack/manila master: Add count info in /shares and /shares/detail response https://review.openstack.org/517208 | 08:35 |
| tbarron | Can you get use the same credentials (user/password or ssh) that the manila service user uses to interact with the service VM, log on to it, and run dmesg, journalctl, look at /var/log/messages, etc. while attempting the fourth mount? | 08:36 |
| tbarron | That's the way I would debug this. | 08:36 |
| arnewiebalck | I’d start there, too. (That’s why I suggested to have a look at the non-existing Ganesha server :) ) | 08:37 |
| tbarron | masber: the issue as you have described it won't show up in manila logs | 08:37 |
| masber | ok | 08:38 |
| tbarron | masber: presumably you are just using the same service image that we use for CI gate, the one posted at tarballs.openstack.org .... | 08:38 |
| masber | tbarron, correct | 08:39 |
| masber | how can I login to that server? | 08:39 |
| tbarron | masber: your /etc/manila.conf has a configuration in it with a back end definition\ | 08:40 |
| tbarron | masber: probably called '[generic] | 08:40 |
| tbarron | '[generic]' at the end | 08:40 |
| tbarron | with an earlier 'enabled_share_back_ends = generic' | 08:41 |
| tbarron | I'm just guessing, you *could* have set this up in many ways | 08:41 |
| tbarron | But the relevant back end section will have the credentials for accessing the service VM | 08:41 |
| tbarron | If you've just been following a guide and using that reference image then you *may* be able to ssh to that service VM with user manila and password manila | 08:42 |
| tbarron | Note that IMO you should *only* be using that supplied image with those credentials, etc. for reference/learning purposes. | 08:43 |
| tbarron | Think about what was just said, in public, a few seconds ago :) | 08:43 |
| masber | ok | 08:44 |
| tbarron | masber: also, no one to my knowledge has tested use of that service image for production purposes. These reference guides on how to use generic driver (which derive before my time) are I think intended as instructional, get you started stuff. | 08:45 |
| tbarron | masber: Are you planning to use the generic driver in this way on a production system? | 08:45 |
| masber | tbarron, production | 08:45 |
| tbarron | masber: well I personally would not do that because you have a single point of failure in the data path. If you can live with that, OK. | 08:46 |
| tbarron | But you'll definitely want to change the credentials on your service VM image. | 08:47 |
| tbarron | And test running at some scale after you solve this problem (beyond D). Check the "flavor" of the VMs that are being used for service VMs. | 08:47 |
| tbarron | We by default use small as we can service VMs because we have limited resources when running tests in CI gate. | 08:48 |
| tbarron | In a real production system an NFS server should be a robust beast. | 08:48 |
| tbarron | masber: Are you or your team involved with the kolla community? If so it might make sense to find out whether anyone there is working on support for the cephfs/nfs driver. | 08:50 |
| tbarron | masber: I work for red hat so we use TripleO (with kolla images in the latest though) rather than kolla orchestration. | 08:51 |
| masber | yep | 08:52 |
| masber | ok | 08:52 |
| masber | I will ask | 08:52 |
| tbarron | masber: we're working on robust deployment of the new cephfs-nfs back end for manila, production quality, for folks like you who have a ceph cluster already being used for cinder and who want to use the same cluster as backing store for manila shares but present the shares via NFS | 08:52 |
| zhongjun | masber: how can I login to that server -- Maybe you could refer to this: https://www.server-world.info/en/note?os=CentOS_7&p=openstack_mitaka2&f=18 | 08:53 |
| zhongjun | tbarron: It looks like manila client is broken now :) | 08:53 |
| tbarron | zhongjun: that looks like a good reference, thanks | 08:54 |
| tbarron | zhongjun: yuck, do you know when it broke? | 08:54 |
| zhongjun | tbarron: I tested it, it looks like the environment (admin_user_domain_name) variable has no value | 08:54 |
| tbarron | zhongjun: do you think this is a result of a requirements change, or of a change in the python-manilaclient code itself? | 08:56 |
| zhongjun | tbarron: In recent day, Maybe yesterday | 08:56 |
| tbarron | zhongjun: maybe support for keystone prior-to-v3 being dropped? | 08:57 |
| tbarron | zhongjun: we should be able to use an older client with the new requirements to debug where the issue came in. | 08:57 |
| zhongjun | tbarron: It doesn't like something changed in the python-manilaclient code itself, because we didn't merge anything for a long time | 08:58 |
| * tbarron is just waking up, needs coffee | 08:58 | |
| tbarron | zhongjun: ack | 08:58 |
| zhongjun | tbarron : Do you know raissa will migration the leagcy job in python-manilaclient | 08:58 |
| tbarron | zhongjun: in a few minutes (after a coffee) I'll look at cinder and see if they also hit the issue | 08:58 |
| zhongjun | tbarron : they are hit another issue | 08:59 |
| zhongjun | tbarron : Because of a cinder patch has been merged. Maybe it will meet the same issue after the bug has been finished, I just guessing :) | 09:00 |
| tbarron | zhongjun: raissa will probably answer your question herself in channel later but I think her plan is to get the split of the tempest tests into their own repo done and then anyone who wants to help can migrate jobs over | 09:00 |
| tbarron | I'll help with that, others can too | 09:00 |
| tbarron | also we need to fix up the stable branches | 09:01 |
| tbarron | maybe we should have this as a meeting topic today | 09:01 |
| tbarron | ^^^ raissa ^^^ | 09:01 |
| zhongjun | tbarron: " manila list --os-user-domain-name" if you run this command, you can reproduce the failure that we meet it in our client function test gate. | 09:04 |
| zhongjun | tbarron: cool | 09:06 |
| tbarron | masber: zhongjun I looked at the link zhongun gave earlier and noticed that it's really showing you logging in to the nova compute intance that is doing the mount (a centos machine), not the nova compute instance that is acting as the nfs server (if you use the tarballs image it's a ubuntu machine) | 09:09 |
| tbarron | masber: using that article you would instead log in to 10.254.0.3 (the server) and debug what's going on when mounts are attempted | 09:10 |
| *** lpetrut has quit IRC | 09:15 | |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Do not merge https://review.openstack.org/512193 | 09:16 |
| openstackgerrit | Merged openstack/puppet-manila master: Deprecate service_instance_network_helper_type https://review.openstack.org/516430 | 09:24 |
| *** gouthamr has quit IRC | 09:26 | |
| *** lpetrut has joined #openstack-manila | 10:06 | |
| openstackgerrit | Tom Barron proposed openstack/puppet-manila master: Deprecate standalone_network_plugin_ip_version https://review.openstack.org/516437 | 10:06 |
| openstackgerrit | Tom Barron proposed openstack/puppet-manila master: Add support for address family parameters https://review.openstack.org/516449 | 10:06 |
| *** lpetrut has quit IRC | 10:12 | |
| *** vivsoni has quit IRC | 10:25 | |
| *** Manor has quit IRC | 10:29 | |
| *** markstur has joined #openstack-manila | 10:29 | |
| *** Manor has joined #openstack-manila | 10:30 | |
| raissa | zhongjun: which legacy job? I migrated part of them - the tempest related ones - to manila repo | 10:31 |
| raissa | the ones that still have legacy in the name can be migrated to each repository. like the ones concerning manilaclient can live there and so on | 10:31 |
| *** markstur has quit IRC | 10:34 | |
| *** arnewiebalck has quit IRC | 10:35 | |
| *** arnewiebalck has joined #openstack-manila | 10:37 | |
| *** Manor has quit IRC | 10:37 | |
| *** Manor has joined #openstack-manila | 10:54 | |
| zhongjun | raissa: The job is: legacy-manilaclient-dsvm-neutron-functional . I just want to check whether you are working on migrate the other legacy jobs to each repository | 11:34 |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Migrating legacy jobs https://review.openstack.org/517276 | 11:35 |
| raissa | zhongjun: not currently :) | 11:35 |
| zhongjun | raissa: so I submit a simple patch for that. I am not sure is it correct. Could you take a look? https://review.openstack.org/517276 | 11:37 |
| raissa | zhongjun: are you going to leave rally-dsvm-manila-multibackend on the project-config? I think you'd have to put it there also and just leave the ones that should live there (https://docs.openstack.org/infra/manual/zuulv3.html#what-not-to-convert) | 11:44 |
| raissa | only in the project part of the zuul.yaml, the definition for those (playbooks) are in the manila repo | 11:45 |
| zhongjun | raissa: yeah, It should follow your patches : https://review.openstack.org/#/c/513075/ and https://review.openstack.org/#/c/513076/ | 11:46 |
| zhongjun | raissa : Add rally-dsvm-manila-multibackend in https://review.openstack.org/#/c/517276/1/.zuul.yaml ? | 11:48 |
| raissa | and the other one | 11:48 |
| raissa | zhongjun: like so https://github.com/openstack-infra/project-config/blob/master/zuul.d/projects.yaml#L21493 | 11:49 |
| zhongjun | raissa: got it | 11:50 |
| openstackgerrit | Tom Barron proposed openstack/puppet-manila master: Add support for address family parameters https://review.openstack.org/516449 | 11:50 |
| *** dsariel__ has quit IRC | 11:51 | |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Migrating legacy jobs https://review.openstack.org/517276 | 11:52 |
| zhongjun | raissa: thanks | 11:52 |
| *** lpetrut has joined #openstack-manila | 11:53 | |
| *** lpetrut has quit IRC | 12:09 | |
| *** dsariel__ has joined #openstack-manila | 12:26 | |
| *** lpetrut has joined #openstack-manila | 12:27 | |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Migrating legacy jobs https://review.openstack.org/517276 | 12:32 |
| *** catintheroof has joined #openstack-manila | 12:34 | |
| openstackgerrit | Merged openstack/puppet-manila master: Remove nova network plugins https://review.openstack.org/516446 | 12:37 |
| openstackgerrit | zhongjun proposed openstack/python-manilaclient master: Do not merge https://review.openstack.org/512193 | 12:48 |
| *** markstur has joined #openstack-manila | 12:48 | |
| *** markstur has quit IRC | 12:53 | |
| *** markstur has joined #openstack-manila | 13:00 | |
| *** junboli has quit IRC | 13:01 | |
| *** lpetrut has quit IRC | 13:03 | |
| *** lpetrut has joined #openstack-manila | 13:04 | |
| *** markstur has quit IRC | 13:05 | |
| *** dustins has joined #openstack-manila | 13:16 | |
| *** AlexeyAbashkin has quit IRC | 13:23 | |
| *** wes_dillingham has joined #openstack-manila | 13:26 | |
| *** jprovazn has quit IRC | 13:34 | |
| *** jmlowe_ has quit IRC | 13:36 | |
| *** AlexeyAbashkin has joined #openstack-manila | 13:45 | |
| *** eharney has joined #openstack-manila | 13:56 | |
| *** jmlowe has joined #openstack-manila | 14:05 | |
| *** jmlowe has quit IRC | 14:11 | |
| *** jmlowe has joined #openstack-manila | 14:27 | |
| *** jprovazn has joined #openstack-manila | 14:35 | |
| *** maysamacedos has quit IRC | 14:38 | |
| *** rraja has joined #openstack-manila | 14:40 | |
| *** markstur has joined #openstack-manila | 14:54 | |
| *** ganso has joined #openstack-manila | 14:56 | |
| *** arnewiebalck_ has joined #openstack-manila | 15:00 | |
| openstackgerrit | Merged openstack/puppet-manila master: Deprecate standalone_network_plugin_ip_version https://review.openstack.org/516437 | 15:32 |
| *** AlexeyAbashkin has quit IRC | 15:53 | |
| bswartz | zhongjun: whitespace issue https://review.openstack.org/#/c/516659/ | 15:56 |
| openstackgerrit | Tom Barron proposed openstack/manila master: Remove glusterfs-native job from check queue https://review.openstack.org/517357 | 15:57 |
| openstackgerrit | zhongjun proposed openstack/manila master: Add API document for share group [3/3] https://review.openstack.org/516659 | 15:58 |
| zhongjun | bswartz: Oh, sorry for that, done | 15:58 |
| *** Manor has quit IRC | 16:00 | |
| *** rraja has quit IRC | 16:02 | |
| openstackgerrit | Tom Barron proposed openstack/manila master: Remove hdfs job from check queue https://review.openstack.org/517362 | 16:02 |
| openstackgerrit | zhongjun proposed openstack/manila master: Add API document for share group [3/3] https://review.openstack.org/516659 | 16:02 |
| openstackgerrit | Merged openstack/manila master: Add API document for share group [1/3] https://review.openstack.org/490716 | 16:03 |
| *** arnewiebalck_ has quit IRC | 16:06 | |
| *** lpetrut has quit IRC | 16:12 | |
| *** dsariel__ has quit IRC | 16:19 | |
| openstackgerrit | Amit Oren proposed openstack/manila master: INFINIDAT add Manila driver. https://review.openstack.org/515432 | 16:23 |
| openstackgerrit | Tom Barron proposed openstack/manila master: Remove glusterfs-native job from check queue https://review.openstack.org/517357 | 16:24 |
| *** catintheroof has quit IRC | 16:26 | |
| openstackgerrit | Tom Barron proposed openstack/manila master: Remove hdfs job from check queue https://review.openstack.org/517362 | 16:27 |
| *** catintheroof has joined #openstack-manila | 16:30 | |
| *** lpetrut has joined #openstack-manila | 16:31 | |
| openstackgerrit | zhongjun proposed openstack/manila master: Add API document for share group [2/3] https://review.openstack.org/516632 | 16:35 |
| openstackgerrit | zhongjun proposed openstack/manila master: Add API document for share group [3/3] https://review.openstack.org/516659 | 16:38 |
| *** maysamacedos has joined #openstack-manila | 17:09 | |
| *** Manor has joined #openstack-manila | 17:18 | |
| *** Manor has quit IRC | 17:22 | |
| *** AlexeyAbashkin has joined #openstack-manila | 17:31 | |
| *** AlexeyAbashkin has quit IRC | 17:35 | |
| *** a-pugachev has joined #openstack-manila | 17:37 | |
| *** lpetrut has quit IRC | 17:58 | |
| *** lpetrut has joined #openstack-manila | 17:59 | |
| *** a-pugachev has quit IRC | 18:00 | |
| *** gouthamr has joined #openstack-manila | 18:00 | |
| *** maysamacedos has quit IRC | 18:05 | |
| openstackgerrit | Merged openstack/manila master: Add API document for share group [2/3] https://review.openstack.org/516632 | 18:06 |
| *** arnewiebalck_ has joined #openstack-manila | 18:08 | |
| *** AlexeyAbashkin has joined #openstack-manila | 18:09 | |
| openstackgerrit | Merged openstack/manila master: Add API document for share group [3/3] https://review.openstack.org/516659 | 18:10 |
| *** pcaruana has quit IRC | 18:10 | |
| *** AlexeyAbashkin has quit IRC | 18:14 | |
| *** abhi has quit IRC | 18:20 | |
| *** eharney has quit IRC | 18:52 | |
| *** LobsterRoll has joined #openstack-manila | 19:20 | |
| *** wes_dillingham has quit IRC | 19:23 | |
| *** LobsterRoll is now known as wes_dillingham | 19:23 | |
| *** arnewiebalck_ has quit IRC | 19:57 | |
| *** wes_dillingham has quit IRC | 20:00 | |
| *** wes_dillingham has joined #openstack-manila | 20:01 | |
| *** eharney has joined #openstack-manila | 20:02 | |
| *** wes_dillingham has quit IRC | 20:09 | |
| *** arnewiebalck_ has joined #openstack-manila | 20:30 | |
| *** jprovazn has quit IRC | 20:46 | |
| *** jmlowe has quit IRC | 20:48 | |
| *** lpetrut has quit IRC | 20:56 | |
| *** arnewiebalck_ has quit IRC | 20:58 | |
| *** eharney has quit IRC | 21:08 | |
| *** AlexeyAbashkin has joined #openstack-manila | 21:08 | |
| *** markstur_ has joined #openstack-manila | 21:10 | |
| *** markstur has quit IRC | 21:12 | |
| *** AlexeyAbashkin has quit IRC | 21:13 | |
| *** jmlowe has joined #openstack-manila | 21:41 | |
| *** catintheroof has quit IRC | 21:53 | |
| *** AlexeyAbashkin has joined #openstack-manila | 22:08 | |
| *** AlexeyAbashkin has quit IRC | 22:13 | |
| *** wes_dillingham has joined #openstack-manila | 22:22 | |
| *** dustins has quit IRC | 22:28 | |
| *** Manor has joined #openstack-manila | 23:01 | |
| *** wes_dillingham has quit IRC | 23:04 | |
| *** Manor has quit IRC | 23:05 | |
| *** wes_dillingham has joined #openstack-manila | 23:11 | |
| *** markstur has joined #openstack-manila | 23:46 | |
| *** markstur_ has quit IRC | 23:47 | |
| *** cyberde has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!