Thursday, 2016-04-07

*** zenoway has joined #openstack-dragonflow00:12
*** zenoway has quit IRC00:16
*** zenoway has joined #openstack-dragonflow00:48
*** zenoway has quit IRC00:54
*** zenoway has joined #openstack-dragonflow01:03
*** zenoway has quit IRC01:08
openstackgerritheshan proposed openstack/dragonflow: code for local controller reliability  https://review.openstack.org/29206001:57
*** gongysh has quit IRC02:22
*** gongysh has joined #openstack-dragonflow02:22
*** gongysh has quit IRC02:32
*** gongysh has joined #openstack-dragonflow02:32
*** yamamoto_ has joined #openstack-dragonflow02:37
*** gongysh has quit IRC03:01
*** gongysh has joined #openstack-dragonflow03:08
*** yamamoto_ has quit IRC03:20
*** gongysh has quit IRC03:30
*** gongysh has joined #openstack-dragonflow03:32
*** yamamoto_ has joined #openstack-dragonflow04:10
*** irenab has quit IRC04:37
*** oanson has joined #openstack-dragonflow05:03
openstackgerritOmer Anson proposed openstack/dragonflow: use IDLs built-in notify for ovsdb monitor  https://review.openstack.org/30169105:11
openstackgerritOmer Anson proposed openstack/dragonflow: Allow filtering of tables and columns in OVS DB monitor  https://review.openstack.org/30222805:11
*** gongysh has quit IRC05:17
*** gongysh has joined #openstack-dragonflow05:19
*** gampel1 has joined #openstack-dragonflow05:35
*** irenab has joined #openstack-dragonflow05:36
gampel1 oanson: ) hujie:  whats the status with the idl sync_start syn_finish05:49
gampel1Frank_Duan: ping05:50
gampel1dingboopt: ping05:52
openstackgerritMerged openstack/dragonflow: Add control plane test doc  https://review.openstack.org/28287306:05
gampel1 hujie: ping06:07
dingbooptgampel1: pong06:09
hujieI'm back now, we are noon rest just now:)06:09
gampel1Hi how are you ? hi whats the status off the  SG patch06:10
*** gongysh has quit IRC06:13
dingbooptI am busy with other things these days. And I will pick up SG patch soon. Sorry for the delay06:13
gampel1ok talk with me after the rest06:13
gampel1sorry06:13
hujiewe have waked up:)06:13
*** gongysh has joined #openstack-dragonflow06:14
hujiewe can work now, it's 14:15 in Beijing06:14
gampel1Ok we discussed the sync finish/start06:14
gampel1and gal have an idea that i think could work06:15
gampel1If we read all the port form the idl before running the monitor06:16
gampel1update them all into the application and then send syen_finishe and then start the monitor06:17
hujieI think current patch has to monitor all tables, lack sync_start and sync_finish mechanism and filter patch and tunnel ports, maybe we need more work:)06:17
oansonpatch and tunnel ports are filtered.06:18
gampel1I am not sure i understand , @06:18
gampel1omer are you here06:18
oansonFilter for specific tables is done06:18
gampel1yes it is done in a patch on top06:18
*** gongysh has quit IRC06:18
oansonPatch: https://review.openstack.org/#/c/302228/06:18
hujieok I see there are other patched to resolve the problem?06:18
oansonI am working on the sync_start and sync_finish now, as per Gal's suggestion.06:19
gampel1https://review.openstack.org/#/c/302228/06:19
oansonI'll have it done today.06:19
gampel1 hujie: I think that if we resolve the sync start/finish we should use the new  patches  introduce by omer06:21
gampel1hujie: what do you think ?06:21
gampel1Frank_Duan: are you here06:21
hujieI have thought about the ideal that Gal suggest before I wrote ovsdb monitor code06:22
gampel1@Omer will do it, we need to focus on the  other features we want to merge before code freeze06:23
gampel1is Frank_Duan there i do not know whats  the status of the   SG patch06:23
hujieI think it is a bit complicate, but if we could not find better method we can use it, maybe I could think about the sequence of this method06:24
gampel1I am not sure i understand  we will not loose any messages , the worst case is that we get each  port twice06:26
hujieduankebo will be back soon06:26
gampel1ok do you know the status of teh SG06:26
gampel1is there someone taking over  dingboopt  patch we need to merge it today06:26
hujieok wait a moment please06:26
hujieI'll ask yuanwei06:27
Frank_DuanHi06:27
gampel1hi how are you?06:27
gampel1we have 3 item  to discuss06:27
hshangampel: reliability feature is ready to merge06:28
Frank_DuanFine, thank you06:28
gampel11) the Ovs Monitor sync start/finishe06:28
gampel12) SG status06:28
Frank_DuanYuanwei has updated the sg patch yestoday06:28
gampel12) reliability feature06:28
gampel13)06:28
gampel1https://review.openstack.org/#/c/280538/06:29
gampel1it was not updated since06:29
gampel1Uploaded patch set 15.Mar 29 3:42 PM06:29
*** gongysh has joined #openstack-dragonflow06:30
Frank_DuanOr, my mistake, he is updating.06:30
gampel1 hshan: did you modify the l3 app using the Cookie06:31
hshanyes06:31
gampel1Ok lets all reveiw it today06:32
hshanok06:32
gampel1gsagie, oanson : please review the reliability patches06:32
oansonWill do06:32
gsagieok06:33
gampel1 Frank_Duan:  we discuss with hujie , we think we have a solution for sync start end andmonitoring only what we need06:33
hujieI have a question, when do you read all the existing port info from IDL cache?06:33
gampel1when do you think we will have a new patch of SG we need to test it a bit before meraging06:34
oansonhujie, I think the DF controller does it.06:34
hujiehow do you make sure all the existing ovs port have been in the IDL cache when you read it06:34
oansonThere is a call to vswitch_api.sync, which calls idl.run, which reads all the information from the OVS DB06:35
hujieI'm worried when you read IDL cache the data is not the same to ovsdb itself, maybe we will have conclusion after we review your new patch:)06:36
gampel1we will not loose those event they will be in the queue06:36
gampel1we read after calling sync on the idl06:37
hujiewhen you invoke idl.run() you have begined to receive message by your overrided notify() method06:37
hujieI mean the IDL cache update and your receive message by notify() method is working at the same time06:39
gsagiehujie: we are talking about a process that happens before we replace the notify or register for events06:40
gsagielets say i dont have your code, you are saying that after i run idl.run() the idl is not synced?06:40
gsagiei think its syncing it in a synchronouse matter06:41
gsagieright?06:41
oansonOnce the notify function is overriden, no.06:42
hujieok  Omer got it:)06:42
oansonThere is a different thread that continuously calls idl.run06:42
gsagieok, so if you override it, and you call idl.run() dont we get all events? or one event at a time?06:42
oansonhujie, It is possible that we will see the same event twice.06:43
hujieOmer when will you commit your new patch?06:43
oansongsagie, we get only the differences06:43
hujietwice event is not problem before we could resolve the problem:)06:44
oansonhujie, I will have the patch ready soon.06:44
oansonCould you please explain what you mean?06:45
openstackgerritNguyen Hung Phuong proposed openstack/dragonflow: Fix typos in Dragonflow files  https://review.openstack.org/30253806:45
hujie@@06:45
oansonhujie, in any case, once we see that the OVSDB monitor works well, we can remove the relevant parts from the sync loop in DF controller, and rely only on the OVSDB monitor.06:46
hujiejust now you say There is a different thread that continuously calls idl.run06:47
hujieI'll think about it:)06:47
hujiewhat I worried is  the IDL cache update and your receive message by notify() method is working at the same time, It is no longer synchronouse mode after the notify function is overriden06:49
hujieif you have consider this and resolve it in new patch, that 's ok:)06:49
gsagiehujie: all events finally reach the df-controller, so if we are worried about same events we can "filter" them there06:50
gsagieyou just need to know if port up or not06:50
oansonhujie: The handling of events is all done in the same thread. That's synchronous.06:50
oansoneven if the events come in from different ports.06:50
oansonI can verify the DF controller poll-sync loop to verify it behaves correctly,06:51
oansonbut 1) We are working with greenthreads. Things don't happen in parallel.06:51
gsagieI personally think that we can register the OVSDB monitor after we do the sync, we will get new events (regarding new ports that were added) when we finally run the ovsdb monitor thread and call idl.run() again06:52
oansonThis is doubly true in python code, due to the GIL.06:52
openstackgerritOmer Anson proposed openstack/dragonflow: Add sync_started and sync_finished notification events  https://review.openstack.org/30254006:52
oanson^^^^ This is the patch for sync_started and sync_finished06:52
hujiegsagie: I'm worried about read data from IDL cache is not complete:)06:56
hujiebecause it is not  synchronouse mode when you invoke idl.run06:57
hujielet's review Omer's new patch to see what has happened:)06:58
gsagieokie :)06:58
gampel1Yuanwei: hi06:59
openstackgerrityuan wei proposed openstack/dragonflow: This patch implements security group app  https://review.openstack.org/28053806:59
openstackgerritOmer Anson proposed openstack/dragonflow: Add publisher's table  https://review.openstack.org/29716707:00
gampel1Yuanwei: will you update this patch as well  patch  https://review.openstack.org/#/c/299105/  thx07:01
gampel1 hshan: hi can you please see why the  CI test are failing07:13
hshanyes, I'm checking that now07:16
openstackgerritNguyen Hung Phuong proposed openstack/dragonflow: Fix typos in Dragonflow files  https://review.openstack.org/30253807:19
hujieOk, I see it, currently you try to read data from IDL cache, I think the data is complete, but is the notify overrided late? or maybe we will lose new online port notify message07:31
oansonThe notify is overriden before the read of the IDL cache.07:31
oansonIf an interface is added before - it will be in the cache.07:32
oansonIf an interface is added after - we will be notified.07:32
oansonIf an interface is added in between - we will get the event twice: Once from the cache, and one from the notification.07:32
oansonThe third case is very unlikely07:32
hujieok I see it07:33
hujiegood job:)07:34
hujiemonitor just one table, you need to modify Neutron code?07:34
hujiewhy filter patch and tunnel port?07:35
hujieI think we need them07:35
yuanweiOK.I will update patch https://review.openstack.org/#/c/299105/ as soon as I can07:36
oansonI thought we only wanted events on VM ports.07:37
hujieok please let me explain it:)07:38
todinmorning, I am testing dragonflow with dist. DNAT, how would be the br-ex interface of the compute node connected to the physical network interstuctur?07:38
hujieyesterday I said we need to cache ofport info of vm, patch, tunnel port into our own cache, not IDL cache, so we can improve our speed of search, do you agree?07:39
hujieand vlan feature designed by Liuhaixia need the patch port07:40
hujiemany of our search operations are the tunnel port search because of remote vm online/offline07:42
yuanweiPlease review & test locally patch https://review.openstack.org/280538, thx. We wish we could merge this patch to master this weekend:)07:42
gsagieyuanwei: testing now07:45
gampel1oanson: ping07:47
yuanweigsagie: thx:)07:49
*** zenoway has joined #openstack-dragonflow07:50
nick-maovsdb local address needs root privilege for access. anyone encounter this problem? https://bugs.launchpad.net/dragonflow/+bug/156724807:50
openstackLaunchpad bug 1567248 in DragonFlow "No access to ovsdb local address" [High,New]07:50
gampel1nick-ma: it did not happend to me07:52
oansonpong07:53
oansonnick-ma, on what distribution did this happen?07:54
gampel1is it new, maybe it is related to the OVS service install07:54
nick-maoanson: Yes, ubuntu server 14.04 with OVS service patch.07:55
nick-maoanson: I don't know if it is related with the OVS service patch. I'm testing.07:56
oansonAnd you say changing it to /var/run/openvswitch/db.sock solved it?07:56
oansonI think  it is07:56
hujieoanson: what's your opinion about my reply?07:56
oansonI think the solution is to try and find the 'default' value, as ovsdb-client --help reports it.07:56
nick-manop. the ovsdb local address by default is /usr/local/var/. I have to change it to /var/.07:56
oansonnick-ma, I think installing it via the OS as done in the OVS service install patch changes that07:57
oansonhujie, I missed it :). Yes, you are right.07:58
oansonI forgot we need ofport info for patch and tunnel.07:58
nick-maWhen I run df-local-controller, it cannot connect to OVSDB because the local address is wrong by default. And when I changed it to /var/, which is reported by ovsdb-client, it shows 'No Access'. Then I rerun controller by sudo, the problem is solved.07:59
oansonCould you please run 'ls -l /var/run/openvswitch/db.sock'?07:59
oansonWe may need to update the permissions on that file.08:00
nick-masrwxr-x--- 1 root root 0 Apr  7 12:22 /var/run/openvswitch/db.sock.08:00
nick-madefinitely, root permission.08:00
oansonYes.08:00
hujieok:)08:01
oansonIn https://review.openstack.org/#/c/301691/ we go back to use the same connection as the DF controller, which still uses IP.08:01
oansonBut it would be better if that went via the unix socket as well.08:01
oansonnick-ma, Could you also run 'id', when logged in as stack?08:05
nick-maok. uid=1001(stack) gid=1001(stack) groups=1001(stack),117(libvirtd)08:05
gampel1hujie: unless there are any objection i am going to merger the IDL patches08:09
hujieok08:11
gampel1can you all please review https://review.openstack.org/#/c/302228/208:12
gampel1and https://review.openstack.org/#/c/302540/108:12
gampel1hujie: please review 302228 and 30354008:23
openstackgerritMerged openstack/dragonflow: use IDLs built-in notify for ovsdb monitor  https://review.openstack.org/30169108:25
gampel1gsagie, nick-ma please look at 302228 and 303540  we want to merge them as soon as possible08:26
nick-magampel1: ok08:28
gampel1thx08:29
openstackgerritEran Gampel proposed openstack/dragonflow: Enable selective distribution with Pub/Sub  https://review.openstack.org/30259508:29
*** yuli_s has joined #openstack-dragonflow08:31
yuli_shello all08:31
gampel1hi08:31
gsagieyuanwei: i dont know if its related to your version but i cant get L3 to work08:31
gsagiei added 2 networks connected them to router and add a VM at each of them and try to ping (default security group with added rules for ping)08:32
gsagiei will try to see if this also happens on master Dragonflow08:32
hujieok08:34
yuanweiok, i will also try this locally08:35
openstackgerritMerged openstack/dragonflow: Fix typos in Dragonflow files  https://review.openstack.org/30253808:36
gsagiebtw yuanwei i am counting on your vote for the TC ;)08:43
gampel1hi please review  https://review.openstack.org/#/c/297167/708:43
yuli_si want to talk about this patch: https://review.openstack.org/#/c/298131/308:44
yuli_sespecially this file: https://review.openstack.org/#/c/298131/3/dragonflow/tests/common/utils.py08:44
yuli_si added wait_until_true() funtion that executes the neutron.agent.linux.utils.wait_until_true() function08:45
yuli_sI change the default timeout value08:45
yuli_sthis was the point when I say that this fn make the code more maintanable08:45
gampel1you just wrapped  there function with yours i do not see the added value08:46
oansonYou can give the timeout explicitly08:47
oansonIt would make the code of the calling function more clear08:47
oansonI don't think changing a default value warrants writing a function that does nothing but wrap an identical other function.08:47
openstackgerritEran Gampel proposed openstack/dragonflow: Send Chassis update on SEND_ALL topic  https://review.openstack.org/30260108:47
yuli_sok,08:47
yuanweigsagie: OK, no problem:)08:48
yuanweibesides, in your test, those two VMs are both in the default security group? I tried locally, and found L3 works in my environments... using the default security group(not added rules for ping)08:49
gsagieyes08:50
gsagiesec, checking now with master08:50
*** gampel1 has quit IRC08:51
gsagieyou use 'demo' as tenant or 'admin' ?08:51
yuanweiadmin08:51
gsagieyeah getting in master the problem as well08:55
gsagiei get "ping: sendto: Network is unreachable"08:56
gsagieyou use q-dhcp or the DHCP app?08:56
yuli_su created a network08:56
yuli_sor u placed vm on public network ?08:56
gsagiei created a network08:57
yuli_si have just run stack with fresh version08:58
yuli_si am also getting an error of other kind08:58
yuli_sbr-int is not a bridge or a socket08:58
yuli_sbr-int is not a bridge or a socket08:58
openstackgerritYuli proposed openstack/dragonflow: Fix DHCP fullstack tests  https://review.openstack.org/29813108:59
yuli_si will reboot and run stack again08:59
yuanweiI used DHCP app09:02
*** yuli_s has quit IRC09:05
*** yuli_s has joined #openstack-dragonflow09:18
*** yamamoto_ has quit IRC09:40
*** gongysh has quit IRC09:49
*** gongysh has joined #openstack-dragonflow09:50
gampel yuanwei: ping10:09
gampeli get allot of error messages when testing the SG patch10:10
gampelDEBUG RyuDFAdapter [-] EventOFPErrorMsg received.10:10
gampelversion=0x4, msg_type=0x1, msg_len=0x4c, xid=0xd1afc97110:10
gampel `-- msg_type: OFPT_ERROR(1)10:10
gampelOFPErrorMsg(type=0x2, code=0x0, data=b'\x04\x0e\x00\x68\xd1\xaf\xc9\x71\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x64\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x10:10
gampel01\x00\x12\x80\x00\x00\x04\x00\x00\x00\x19\x80\x00\x0a\x02')10:10
gampel |-- type: OFPET_BAD_ACTION(2)10:10
gampel |-- code: OFPBAC_BAD_TYPE(0) from (pid=14905) error_msg_handler /usr/lib/python2.7/site-packages/ryu/controller/ofp_handler.py:26410:10
gampel10:10
gampel yuanwei: ping10:16
*** gongysh has quit IRC10:32
gampelFrank_Duan:ping10:33
yuanwei.10:38
yuanweihi gampel10:38
yuanweiwhat is your SG configuration?10:38
oansonHi. Using dragonflow master + sg apps, nova reports that there are no flavors. Has anyone else ran into this? Or is this an issue on my setup?10:39
gampelhi10:41
oansonThe following tests fail because of that: dragonflow.tests.fullstack.test_l2_multicast.TestL2Multicast.test_vm_multicast dragonflow.tests.fullstack.test_l2_responder.ArpResponderTest.test_arp_responder dragonflow.tests.fullstack.test_topology.TestTopology.test_topology_create_vm dragonflow.tests.fullstack.test_topology.TestTopology.test_topology_create_vm210:41
gampelI am running sg and i am getting Ryu errors when installing the flows10:41
*** yamamoto has joined #openstack-dragonflow10:42
gampelDEBUG RyuDFAdapter [-] EventOFPErrorMsg received.10:42
gampelversion=0x4, msg_type=0x1, msg_len=0x4c, xid=0xd1afc99910:42
gampel `-- msg_type: OFPT_ERROR(1)10:42
gampelOFPErrorMsg(type=0x4, code=0x8, data=b'\x04\x0e\x00\x60\xd1\xaf\xc9\x99\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x4d\x00\x00\x00\x00\x00\x00\x02\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00\x00\x00\x00\x00\x10:42
gampel01\x00\x18\x00\x01\x0e\x04\x00\x00\x00\x3b\x00\x01\xd3\x08')                                                                                                                                                                                  10:42
gampel |-- type: OFPET_BAD_MATCH(4)10:42
gampel |-- code: OFPBMC_BAD_MASK(8) from (pid=14905) error_msg_handler /usr/lib/python2.7/site-packages/ryu/controller/ofp_handler.py:26410:42
gampel2016-04-07 13:18:59.481 DEBUG RyuDFAdapter [-]  `-- data: version=0x4, msg_type=0xe, msg_len=0x60, xid=0xd1afc99910:42
gampel     `-- msg_type: OFPT_FLOW_MOD(14) from (pid=14905) error_msg_handler /usr/lib/python2.7/site-packages/ryu/controller/ofp_handler.py:27110:42
gampel2016-04-07 13:18:59.482 DEBUG RyuDFAdapter [-] EventOFPErrorMsg received.10:42
gampelversion=0x4, msg_type=0x1, msg_len=0x4c, xid=0xd1afc99a10:42
gampel `-- msg_type: OFPT_ERROR(1)10:42
openstackgerritOmer Anson proposed openstack/dragonflow: Allow filtering of tables and columns in OVS DB monitor  https://review.openstack.org/30222810:42
*** gongysh has joined #openstack-dragonflow10:44
gampelyuanwei: Do you get the same error10:44
yuanweino...Do you create some security group rules?10:46
yuanweiwhat rules do you create? I will try that in my environment10:47
yuanweisorry..I just check logs in my environment, and i find this error10:48
*** gongysh has quit IRC11:02
*** yuanwei has quit IRC11:02
*** yamamoto has quit IRC11:04
*** yamamoto has joined #openstack-dragonflow11:06
*** yuanwei has joined #openstack-dragonflow11:09
gampelyuanwei: will you take care of this11:17
yuanweiyes, I'm working on that11:18
openstackgerritMerged openstack/dragonflow: Fix DHCP fullstack tests  https://review.openstack.org/29813111:22
gampelwhats the status of https://review.openstack.org/#/c/292060/11:37
gampelwhat about https://review.openstack.org/#/c/299105/11:38
gampelyuanwei: You could  continue to work on this two tomorrow , we can try and merge it tomorrow11:39
gampelIt is late your time11:39
yuanweithx:), not too late, it's OK to me11:41
openstackgerritOmer Anson proposed openstack/dragonflow: Allow filtering of tables and columns in OVS DB monitor  https://review.openstack.org/30222811:49
openstackgerritMerged openstack/dragonflow: Enable selective distribution with Pub/Sub  https://review.openstack.org/30259512:19
*** yuli_s has quit IRC12:21
oansonHi, could you please review https://review.openstack.org/#/c/302228/ ?12:41
*** yuanwei has quit IRC12:41
*** yuanwei has joined #openstack-dragonflow12:46
*** Mic22 has quit IRC12:55
*** Mic22 has joined #openstack-dragonflow12:59
openstackgerrityuan wei proposed openstack/dragonflow: This patch implements security group app  https://review.openstack.org/28053813:04
yuanweigampel13:10
yuanweigampel: I found the cause of that problem. It's because of the lack of thought about IPv6 rules..and I update the patch https://review.openstack.org/280538 to fix this bug13:10
*** Mic22 has quit IRC13:14
*** yuanwei has quit IRC13:22
*** yuanwei has joined #openstack-dragonflow13:22
*** oanson has quit IRC13:22
*** yamamoto has quit IRC13:40
openstackgerritMerged openstack/dragonflow: Allow filtering of tables and columns in OVS DB monitor  https://review.openstack.org/30222813:44
*** irenab has quit IRC13:44
*** yamamoto has joined #openstack-dragonflow13:57
*** yamamoto has quit IRC14:03
*** yamamoto has joined #openstack-dragonflow14:29
openstackgerritEran Gampel proposed openstack/dragonflow: Add sync_started and sync_finished notification events  https://review.openstack.org/30254014:58
openstackgerritOpenStack Proposal Bot proposed openstack/dragonflow: Updated from global requirements  https://review.openstack.org/30284515:20
*** yamamoto has quit IRC16:06
*** zenoway has quit IRC16:09
*** zenoway has joined #openstack-dragonflow16:10
*** zenoway has quit IRC16:11
*** zenoway has joined #openstack-dragonflow16:11
*** zenoway has quit IRC16:15
*** zenoway has joined #openstack-dragonflow16:16
*** zenoway has quit IRC16:20
*** oanson has joined #openstack-dragonflow17:01
*** oanson has quit IRC17:06
*** yamamoto has joined #openstack-dragonflow17:07
*** yuanwei has quit IRC17:10
*** yuanwei has joined #openstack-dragonflow17:11
*** oanson has joined #openstack-dragonflow17:14
*** yamamoto has quit IRC17:16
*** hujie has quit IRC17:52
*** hujie has joined #openstack-dragonflow17:52
*** yuanwei has quit IRC18:07
*** yuanwei has joined #openstack-dragonflow18:07
*** zenoway has joined #openstack-dragonflow18:12
*** zenoway has quit IRC18:16
todinhmm, is rule wants to jump to table 30, but there is no table 30 priority=100,ip,metadata=0x3,nw_src=192.168.42.101 actions=resubmit(,30)18:19
*** oanson has quit IRC18:24
*** zenoway has joined #openstack-dragonflow18:30
*** gampel has quit IRC18:39
*** CrayZee has quit IRC18:39
*** saggi has quit IRC18:40
*** zenoway has quit IRC23:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!