Tuesday, 2017-02-07

*** spzala has joined #openstack-oslo00:01
*** salv-orl_ has quit IRC00:05
*** spzala has quit IRC00:06
mtreinishharlowja, dims: http://logs.openstack.org/44/429844/3/check/gate-tempest-dsvm-neutron-src-oslo.service-ubuntu-xenial/60ed0fc/logs/ 47 snapshots to play with00:19
harlowjawoah nice00:19
mtreinishit also caught a py35 bug in nova/tempest with a new test :)00:19
harlowjalol00:19
*** spzala has joined #openstack-oslo00:21
mtreinishhmm I don't think it's collecting any of the children. Just the parent process00:23
mtreinishmost of the ones I'm looking at are the same00:23
mtreinishwell more or less00:23
harlowjahmmm, u farther along then me, ha00:23
openstackgerritMatthew Treinish proposed openstack/oslo.service master: WIP: Add tracemalloc support  https://review.openstack.org/42984400:24
harlowjaya, weird not seeing like any huge memory usage (according to these stats)00:29
harlowjamtreinish https://gist.github.com/harlowja/0d35e8887126a6b5e78ef96452b7bfe5 crappy little script if its useful, ha00:31
*** ansmith has quit IRC00:33
harlowjathough just from one of those snapshots i get Size = 32992b/32.21875kb00:34
harlowjawhich seems low, lol00:34
harlowjai wonder if we need to hook earlier in the service startup00:34
harlowjalike before it even import stuff00:35
mtreinishharlowja: yeah according to the ps output the rss for process 23816 is 266256 kb: http://logs.openstack.org/44/429844/3/check/gate-tempest-dsvm-neutron-src-oslo.service-ubuntu-xenial/60ed0fc/logs/ps.txt.gz00:37
mtreinishso we're definitely missing a big chunk :)00:38
harlowjadef, ha00:38
harlowjamtreinish prob try increasing the tracemalloc.start frame number00:40
harlowjahttps://docs.python.org/3/library/tracemalloc.html#tracemalloc.start looks like it may need more ?00:40
*** zz_dimtruck is now known as dimtruck00:40
harlowjai see the pants folks picked 12 @ https://github.com/pantsbuild/pants/wiki/Debugging-Tips:-tracemalloc-for-python-2.700:41
harlowjaseems like a good enough number, lol00:41
mtreinishI pushed a new version of https://review.openstack.org/#/c/429846/2 a few min ago that used 12000:42
mtreinishwe can change it to be whatever00:42
harlowjakk00:42
harlowjalet's see how that goes, ha00:42
mtreinishI also need to update the docstring in https://review.openstack.org/#/c/429844/4/oslo_service/tracemalloc.py because I clearly didn't read the docs :)00:43
harlowja:-p00:43
*** electrichead is now known as rerobot00:45
openstackgerritgecong proposed openstack/taskflow master: Use https instead of http for git.openstack.org  https://review.openstack.org/42994800:48
*** spzala has quit IRC00:57
*** david-lyle has quit IRC01:06
*** spzala has joined #openstack-oslo01:09
openstackgerritMerged openstack/oslo.vmware master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42981201:10
dimsharlowja : a bunch of quick reviews please - https://review.openstack.org/#/q/status:open+branch:master+topic:bug/166226601:20
harlowjawoah01:20
harlowjaha01:20
mtreinishharlowja: round 2: http://logs.openstack.org/44/429844/4/check/gate-tempest-dsvm-neutron-src-oslo.service-ubuntu-xenial/8608cd2/logs/01:20
harlowjaoh man, which is more interesting :-P01:21
harlowjadescisions descisions01:21
harlowjahmmm, mtreinish picked one, it still seems weirdly small, lol01:21
mtreinishheh, yeah clearly we're doing something wrong01:23
mtreinishI'll investigate more tomorrow01:23
harlowjaya, we need the haypo power01:23
harlowjajd__ where's haypo :-P01:23
*** xgerman has joined #openstack-oslo01:24
*** spzala has quit IRC01:41
*** spzala has joined #openstack-oslo01:42
*** spzala has quit IRC01:46
*** spzala has joined #openstack-oslo01:53
*** sdague_ has quit IRC01:57
openstackgerritgengchc2 proposed openstack/oslosphinx master: Use https instead of http for git.openstack.org  https://review.openstack.org/42996702:07
*** gcb has joined #openstack-oslo02:12
openstackgerritMerged openstack/oslo.service master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42981102:17
openstackgerritMerged openstack/oslo.rootwrap master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42981002:23
openstackgerritMerged openstack/oslo.middleware master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980702:27
openstackgerritMerged openstack/oslo.privsep master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980902:29
openstackgerritMerged openstack/oslo.log master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980402:30
openstackgerritMerged openstack/oslo.cache master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980002:32
openstackgerritchen.xing proposed openstack/oslo.messaging master: Fix the typo  https://review.openstack.org/42999402:39
*** spzala has quit IRC02:41
openstackgerritMerged openstack/oslo.messaging master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980502:45
openstackgerritMerged openstack/oslo.context master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980302:47
*** sdague has joined #openstack-oslo02:55
*** spzala has joined #openstack-oslo02:55
openstackgerritZhaoBo proposed openstack/oslo.db master: Make paginate_query work with association proxies.  https://review.openstack.org/42969503:00
*** spzala has quit IRC03:00
openstackgerritMerged openstack/oslo.concurrency master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980103:00
openstackgerritMerged openstack/oslo.reports master: Remove references to Python 3.4  https://review.openstack.org/42142703:08
openstackgerritMerged openstack/oslo.service master: Remove duplicated register_opts call  https://review.openstack.org/42650703:10
mtreinishharlowja: so I made something: http://blog.kortar.org/wp-content/uploads/2017/02/flame.svg03:13
*** david-lyle has joined #openstack-oslo03:13
*** sdague has quit IRC03:13
mtreinishyour browser is going to hate me though03:13
openstackgerritMerged openstack/oslo.messaging master: [zmq] Dummy add value aging mechanism  https://review.openstack.org/42866303:13
mtreinishharlowja: it was generated with: http://paste.openstack.org/show/597889/03:17
openstackgerritMerged openstack/oslo.log master: Remove 'verbose' option (again)  https://review.openstack.org/31754703:17
*** yamahata has quit IRC03:22
*** spzala has joined #openstack-oslo03:36
*** ihrachys has joined #openstack-oslo03:38
*** ihrachys has quit IRC03:39
*** links has joined #openstack-oslo03:41
*** spzala has quit IRC03:41
*** yamahata has joined #openstack-oslo03:42
openstackgerritMerged openstack/oslo.versionedobjects master: fields: fix stringifying of objects when subclasses=True  https://review.openstack.org/37158303:45
openstackgerritMerged openstack/oslo.versionedobjects master: Change Object schema type to be array instead of string.  https://review.openstack.org/41182203:45
*** spzala has joined #openstack-oslo03:53
*** sdague has joined #openstack-oslo03:54
*** spzala has quit IRC03:58
*** nicolasbock has quit IRC04:01
openstackgerritAnh Tran proposed openstack/futurist master: Typo fix: arbitary => arbitrary  https://review.openstack.org/43005004:14
*** Nakato_ is now known as Nakato04:18
openstackgerritMerged openstack/oslo.config master: pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)  https://review.openstack.org/42980205:13
*** spzala has joined #openstack-oslo05:15
openstackgerritJoshua Harlow proposed openstack/taskflow master: WIP of a stack/register machine for de/re/compiling a workflow  https://review.openstack.org/43006505:17
*** spzala has quit IRC05:20
*** eck` is now known as jeckersb_gone05:21
*** salv-orlando has joined #openstack-oslo05:39
*** david-lyle has quit IRC05:40
*** salv-orlando has quit IRC05:44
*** links has quit IRC05:51
*** lpetrut has joined #openstack-oslo05:54
*** AJaeger has left #openstack-oslo05:54
*** rcernin has joined #openstack-oslo06:00
*** rcernin has quit IRC06:03
*** rcernin has joined #openstack-oslo06:04
*** links has joined #openstack-oslo06:07
openstackgerritMerged openstack/oslo.privsep master: Remove references to Python 3.4  https://review.openstack.org/42142506:18
openstackgerritMerged openstack/oslo.db master: Prepare for using standard python tests  https://review.openstack.org/42985706:20
*** links has quit IRC06:22
*** nkrinner_afk is now known as nkrinner06:22
*** Syed__ has quit IRC06:35
*** links has joined #openstack-oslo06:38
*** salv-orlando has joined #openstack-oslo06:40
*** salv-orlando has quit IRC06:44
*** lpetrut has quit IRC06:46
*** tovin07_ has joined #openstack-oslo06:57
*** dimtruck is now known as zz_dimtruck07:06
*** links has quit IRC07:07
*** nkrinner is now known as nkrinner_afk07:08
*** eezhova has joined #openstack-oslo07:11
*** salv-orlando has joined #openstack-oslo07:11
*** links has joined #openstack-oslo07:19
*** rasca has joined #openstack-oslo07:20
*** nkrinner_afk is now known as nkrinner07:30
*** links has quit IRC07:30
*** tesseract has joined #openstack-oslo07:31
*** yamamoto has quit IRC07:38
*** links has joined #openstack-oslo07:42
*** pcaruana has joined #openstack-oslo07:46
*** pcaruana has quit IRC07:48
*** salv-orl_ has joined #openstack-oslo07:49
*** salv-orlando has quit IRC07:52
*** pcaruana has joined #openstack-oslo07:55
*** jamielennox is now known as jamielennox|away07:59
*** jamielennox|away is now known as jamielennox08:06
openstackgerritMerged openstack/oslo.db master: Explain paginate_query doesn't provide parameter offset  https://review.openstack.org/42944908:13
*** salv-orlando has joined #openstack-oslo08:15
*** salv-orl_ has quit IRC08:21
*** yamamoto has joined #openstack-oslo08:32
*** lpetrut has joined #openstack-oslo08:32
gcbsileht:  https://review.openstack.org/#/c/428693/ is ready to merge. please help look at it.08:51
*** zzzeek has quit IRC09:00
*** zzzeek has joined #openstack-oslo09:00
openstackgerritTovin Seven proposed openstack/osprofiler master: Change some bindings to one-time bindings  https://review.openstack.org/42447409:01
*** salv-orlando has quit IRC09:04
Dinesh_BhorHi all, is there a way in oslo_concurrency to achieve non-blocking lock?09:05
*** salv-orlando has joined #openstack-oslo09:07
*** yamahata has quit IRC09:17
*** lucas-afk is now known as lucasagomes09:29
*** yamamoto has quit IRC09:30
openstackgerritMerged openstack/oslo.vmware master: Add in support for removeKey  https://review.openstack.org/42940409:32
*** yamamoto has joined #openstack-oslo09:33
*** yassine has joined #openstack-oslo09:37
*** yassine is now known as Guest5868909:37
*** yamamoto has quit IRC09:39
jd__harlowja: not sure he hangs a lot in OpenStack nowadays, so feel free to mail him09:49
*** tovin07_ has quit IRC09:55
*** tovin07 has quit IRC09:56
*** jschwarz has quit IRC10:01
*** jschwarz has joined #openstack-oslo10:01
*** mhickey has joined #openstack-oslo10:09
*** mhickey has quit IRC10:11
*** mhickey has joined #openstack-oslo10:11
*** salv-orl_ has joined #openstack-oslo10:19
*** salv-orlando has quit IRC10:21
*** sambetts_ is now known as sambetts10:43
*** szaher has joined #openstack-oslo10:43
jpenahi all, would it be possible to get a new oslo.log release? We need https://review.openstack.org/425400 in a tagged release to get a working build in master and stable/ocata10:46
*** cdent has joined #openstack-oslo10:51
*** Guest58689 has quit IRC10:51
*** Guest58689 has joined #openstack-oslo10:55
*** nicolasbock has joined #openstack-oslo11:01
*** boden has joined #openstack-oslo11:11
openstackgerritMerged openstack/mox3 master: Remove vim header from source files  https://review.openstack.org/42049211:14
gcbjpena, branch stable/ocata includes the fix,  I think you need update requirements project11:20
jpenagcb: true, but there's no tagged release including it, iiuic11:21
gcbhttps://review.openstack.org/#/c/422060/4/deliverables/ocata/oslo.log.yaml11:22
gcbjpena, you mean you don't know which release of oslo.log include the commit, right?11:23
jpenagcb: no, I mean release 3.20.0 doesn't include the commit11:24
jpena3.20.0 dates back to Dec 22, but this patch was merged on Jan 2611:25
gcbgot it , I think you need a new release 3.20.1 for stable/ocata , right ?11:26
jpenagcb: that's correct, yes11:26
gcbjpena, submit a patch like https://review.openstack.org/#/c/429856/, and add me as reviewer11:27
gcbI think we need wait  until requirements project cut off stable/ocata branch11:27
jpenagcb: ack, I'll do so. Thanks!11:28
gcbjpena: np11:29
*** salv-orl_ has quit IRC11:47
*** salv-orlando has joined #openstack-oslo11:48
*** ansmith has joined #openstack-oslo11:52
*** jschwarz has quit IRC12:00
*** ansmith has quit IRC12:08
*** purplerbot has quit IRC12:15
*** purplerbot has joined #openstack-oslo12:15
*** lucasagomes is now known as lucas-hungry12:18
*** cdent has quit IRC12:30
openstackgerritMerged openstack/oslo.policy master: Add optional exception for check_rules  https://review.openstack.org/37425112:31
*** sdague_ has joined #openstack-oslo12:32
*** gcb has quit IRC12:36
*** gcb has joined #openstack-oslo12:37
*** gcb has quit IRC12:45
*** links has quit IRC12:48
*** sdague has quit IRC12:52
*** gcb has joined #openstack-oslo12:53
*** gcb has quit IRC13:09
*** salv-orlando has quit IRC13:09
*** lucas-hungry is now known as lucasagomes13:20
*** kgiusti has joined #openstack-oslo13:29
*** gordc has joined #openstack-oslo13:29
*** jpena is now known as jpena|lunch13:31
*** jeckersb_gone is now known as eck`13:42
*** ansmith has joined #openstack-oslo13:42
*** amotoki has joined #openstack-oslo13:44
*** trown|brb has joined #openstack-oslo13:45
*** trown|brb is now known as trown|mtg13:45
*** trown|mtg is now known as trown13:45
*** mtanino has quit IRC13:47
*** zz_dimtruck is now known as dimtruck13:47
*** salv-orlando has joined #openstack-oslo13:50
*** kgiusti1 has joined #openstack-oslo14:00
*** kgiusti has quit IRC14:00
*** kgiusti1 has left #openstack-oslo14:01
*** kgiusti has joined #openstack-oslo14:01
*** kgiusti has quit IRC14:06
*** kgiusti has joined #openstack-oslo14:07
*** ihrachys has joined #openstack-oslo14:11
*** dtardivel has joined #openstack-oslo14:21
*** mriedem has joined #openstack-oslo14:27
openstackgerritMerged openstack/oslo.vmware master: Improve unit test coverage  https://review.openstack.org/43017114:29
*** jpena|lunch is now known as jpena14:38
*** szaher has quit IRC14:50
*** dimtruck is now known as zz_dimtruck14:51
*** mtanino has joined #openstack-oslo14:54
*** david-lyle has joined #openstack-oslo15:08
*** david-lyle has quit IRC15:10
*** david-lyle has joined #openstack-oslo15:11
*** cdent has joined #openstack-oslo15:14
*** david-lyle has quit IRC15:16
*** sdague_ is now known as sdague15:17
*** eck` is now known as jeckersb_gone15:18
*** bauzas has quit IRC15:19
*** zz_dimtruck is now known as dimtruck15:19
*** trown is now known as trown|brb15:21
*** bauzas has joined #openstack-oslo15:22
*** trown|brb is now known as trown15:23
*** jeckersb_gone is now known as eck`15:23
*** MVenesio has joined #openstack-oslo15:32
*** tovin07 has joined #openstack-oslo15:42
*** eck` is now known as jeckersb_gone15:42
*** yamahata has joined #openstack-oslo15:43
*** salv-orlando has quit IRC15:46
*** david-lyle has joined #openstack-oslo15:53
*** lpetrut has quit IRC15:56
*** salv-orlando has joined #openstack-oslo16:04
*** tovin07 has left #openstack-oslo16:11
*** rcernin has quit IRC16:16
*** pcaruana has quit IRC16:17
*** jeckersb_gone is now known as eck`16:17
*** szaher has joined #openstack-oslo16:25
*** nkrinner is now known as nkrinner_afk16:33
*** lpetrut has joined #openstack-oslo16:35
*** openstackgerrit has quit IRC16:35
*** eezhova has quit IRC16:50
*** lpetrut has quit IRC16:56
*** eezhova has joined #openstack-oslo16:59
harlowjamtreinish jd__ thx, will do17:04
*** lpetrut has joined #openstack-oslo17:08
*** trown is now known as trown|lunch17:09
harlowjamtreinish yup my browser hates u, lol17:10
harlowjaoopening chrome instead (so that it messes up it instead, lol)17:10
mtreinishheh17:10
mtreinishif you can get your browser to load it, it's pretty cool and interactive17:11
harlowjawhat does it mean :-P17:11
mtreinishyou can dive into a specific section and see more details17:11
mtreinishthe x axis is memory consumption (so the wider a bar is the more memory it is consuming)17:11
mtreinishand the height is stack depth17:11
harlowjakk17:11
mtreinishhttp://www.brendangregg.com/flamegraphs.html17:12
harlowjaneats17:12
harlowjaother idea i had was that the thread we are running, do u know if that is also connected into the child processes that say neutron spins off?17:13
harlowjaor is it just the 'master/root/parent'  process17:13
harlowjamaybe need to ensure its also started in each child?17:13
mtreinishI think it's just running on the parent process17:16
*** yamahata has quit IRC17:16
harlowjak17:16
harlowjaother thing that would be nice is to figure out what objects are in memory17:16
harlowjamaybe in that thread take some of https://github.com/mgedmin/dozer/blob/master/dozer/leak.py#L134-L15817:17
harlowjaand also write that out?17:17
harlowjaor some variation of that tick() method there17:17
harlowjamainly https://github.com/mgedmin/dozer/blob/master/dozer/leak.py#L135-L14917:18
harlowjathen say just write that out everytime tracemalloc is triggered17:18
mtreinishharlowja: sure, it's worth a try17:19
mtreinishfeel free to push up revs of any of the patches in the stack if you want to play around with stuff17:19
harlowjak17:20
mtreinishheh, it's not like it's super useful in it's current form :)17:20
harlowja:-P17:20
*** eezhova has quit IRC17:23
*** mhickey has quit IRC17:36
*** eezhova has joined #openstack-oslo17:40
*** eezhova has quit IRC17:42
*** eezhova has joined #openstack-oslo17:49
*** Syed__ has joined #openstack-oslo17:55
*** browne has joined #openstack-oslo17:57
*** tesseract has quit IRC18:01
*** lucasagomes is now known as lucas-afk18:02
*** yamahata has joined #openstack-oslo18:02
*** trown|lunch is now known as trown18:15
*** jpena is now known as jpena|away18:33
*** jpena|away is now known as jpena18:33
*** rasca has quit IRC18:35
*** eck` is now known as jeckersb_gone18:37
*** jpena is now known as jpena|lunch18:37
*** jpena|lunch is now known as jpena|away18:37
*** sambetts is now known as sambetts|afk18:38
*** eezhova has quit IRC18:40
*** jeckersb_gone is now known as eck`19:09
*** Rockyg has joined #openstack-oslo19:18
*** eezhova has joined #openstack-oslo19:22
*** salv-orl_ has joined #openstack-oslo19:50
*** salv-orlando has quit IRC19:52
*** ihrachys_ has joined #openstack-oslo20:01
*** ansmith has quit IRC20:03
*** ihrachys has quit IRC20:03
*** salv-orl_ has quit IRC20:04
*** ansmith has joined #openstack-oslo20:13
*** kgiusti has left #openstack-oslo20:19
*** mhickey has joined #openstack-oslo20:25
*** ansmith has quit IRC20:29
*** ansmith has joined #openstack-oslo20:44
*** openstackgerrit has joined #openstack-oslo20:45
openstackgerritJoshua Harlow proposed openstack/oslo.service master: WIP: Add tracemalloc support  https://review.openstack.org/42984420:45
harlowjamtreinish  ^20:45
harlowjashould now also write objcounts20:45
harlowjaand i think handle forking20:46
*** dimtruck is now known as zz_dimtruck20:47
openstackgerritJoshua Harlow proposed openstack/oslo.service master: WIP: Add tracemalloc support  https://review.openstack.org/42984420:49
openstackgerritJoshua Harlow proposed openstack/oslo.service master: WIP: Add tracemalloc support  https://review.openstack.org/42984420:49
*** ansmith has quit IRC20:51
*** amotoki has quit IRC20:51
*** rtheis has joined #openstack-oslo20:53
*** rtheis has quit IRC20:53
harlowjamtreinish what job did u poke to run that?20:54
harlowja(maybe nothing special needed)20:54
mtreinishharlowja: the depends on should pull in all the setup to "just work"20:56
mtreinishthe tempest job should run on py35 and neutron should call the helper20:56
harlowjak20:56
mtreinishif you changed the helper function you'll have to update the neutron patch and recheck20:56
harlowjanope, should be same20:56
*** ansmith has joined #openstack-oslo21:03
*** nicolasbock has quit IRC21:04
*** harlowja has quit IRC21:09
*** nicolasbock has joined #openstack-oslo21:11
*** zz_dimtruck is now known as dimtruck21:14
*** ansmith has quit IRC21:34
*** eezhova has quit IRC21:35
*** harlowja has joined #openstack-oslo21:35
*** cdent has quit IRC21:36
*** MVenesio has quit IRC21:37
*** salv-orlando has joined #openstack-oslo21:37
*** jamielennox is now known as jamielennox|away21:42
*** salv-orlando has quit IRC21:42
*** yamamoto has joined #openstack-oslo21:43
*** salv-orlando has joined #openstack-oslo21:50
*** haypo has joined #openstack-oslo21:53
*** haypo has quit IRC21:53
*** haypo has joined #openstack-oslo21:53
*** tesseract has joined #openstack-oslo21:59
*** tesseract- has joined #openstack-oslo22:00
*** yamamoto has quit IRC22:07
*** tesseract has quit IRC22:07
*** tesseract- has quit IRC22:07
*** tesseract has joined #openstack-oslo22:08
*** ihrachys_ has quit IRC22:09
*** yamamoto has joined #openstack-oslo22:12
*** trown is now known as trown|outtypewww22:24
*** tesseract is now known as tesseract-RH22:26
*** tesseract-RH is now known as tesseract22:31
mtreinishharlowja: hmm, only 1 pickle this time: http://logs.openstack.org/44/429844/7/check/gate-tempest-dsvm-neutron-src-oslo.service-ubuntu-xenial/be4ead8/logs/22:36
*** jamielennox|away is now known as jamielennox22:42
*** mhickey has quit IRC22:49
*** mhickey has joined #openstack-oslo22:49
*** mriedem has quit IRC22:51
*** tesseract has quit IRC23:00
*** tesseract has joined #openstack-oslo23:00
*** tesseract-RH has joined #openstack-oslo23:01
*** tesseract has quit IRC23:01
*** tesseract-RH has quit IRC23:01
*** boden has quit IRC23:06
*** lpetrut has quit IRC23:07
harlowjahmmm23:10
harlowjathat's odd, checking, lol23:11
*** sdague has quit IRC23:12
harlowjaah, json issues23:14
harlowjaand unicode23:14
harlowjalol23:14
openstackgerritJoshua Harlow proposed openstack/oslo.service master: WIP: Add tracemalloc support  https://review.openstack.org/42984423:17
harlowjaok mtreinish should be fixed, stupid utf823:18
harlowjalol23:18
*** mhickey has quit IRC23:30
*** dimtruck is now known as zz_dimtruck23:33
*** gordc has quit IRC23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!