Tuesday, 2019-09-10

« Monday, 2019-09-09 Index Wednesday, 2019-09-11 »
*** cdearborn has quit IRC00:06
*** ociuhandu has joined #openstack-ironic00:10
*** ociuhandu has quit IRC00:15
*** goldyfruit_ has joined #openstack-ironic00:20
*** gyee has quit IRC00:42
*** kiseok7 has joined #openstack-ironic01:09
*** rloo has quit IRC01:27
*** rloo has joined #openstack-ironic01:28
*** kaifeng has joined #openstack-ironic01:36
*** rloo has quit IRC02:31
*** hamzy_ has quit IRC03:06
kaifenghi stendulker, when you have time, could you take a look on the https://review.openstack.org/680652 ? it seems fixing the broken gate03:26
patchbothttps://review.opendev.org/#/c/680652/ - ironic - CI: reduce the sizes of disks in the standalone job - 1 patch set03:26
*** igordc has quit IRC03:31
*** rh-jelabarre has quit IRC03:34
*** mkrai has joined #openstack-ironic03:59
*** gkadam has joined #openstack-ironic03:59
*** ociuhandu has joined #openstack-ironic04:30
*** pradip_k has joined #openstack-ironic04:31
pradip_kGood Morning, Ironic!04:31
*** ociuhandu has quit IRC04:34
*** jtomasek has joined #openstack-ironic04:38
*** mkrai has quit IRC04:43
*** igordc has joined #openstack-ironic04:44
*** ricolin has joined #openstack-ironic05:00
*** pradip_k has quit IRC05:01
*** tzumainn has quit IRC05:18
*** jtomasek has quit IRC05:29
*** diga has joined #openstack-ironic05:33
*** rachit7 has joined #openstack-ironic05:34
*** stendulker has joined #openstack-ironic05:44
*** dtantsur|afk is now known as dtantsur06:15
dtantsurclarkb: https://review.opendev.org/#/c/680652/ may be related06:16
patchbotpatch 680652 - ironic - CI: reduce the sizes of disks in the standalone job - 1 patch set06:16
kaifenghey dtantsur, clarkb left some comment in the patch06:17
dtantsuryep, I'd still land it06:19
kaifengsure, as long as it works06:20
arne_wiebalckgood morning kaifeng dtantsur and ironic!06:25
*** pradip_k has joined #openstack-ironic06:26
dtantsurmorning arne_wiebalck! could you check the patch above?06:27
* arne_wiebalck checking06:29
*** jawad_axd has joined #openstack-ironic06:29
openstackgerritDirk Mueller proposed openstack/bifrost master: Revert "Make the opensuse-15 jobs non-voting"  https://review.opendev.org/68116306:44
kaifengmorning arne_wiebalck :)06:46
openstackgerritMerged openstack/ironic master: Follow-up to power sync reno  https://review.opendev.org/67640006:47
kaifengdtantsur: maybe we could use different specs for the VMs in the future06:48
dtantsurkaifeng: yep, we should probably get smarter with how we allocate these VMs. but currently it's one gross hack.06:48
dtantsurmaybe we need to split away the RAID job, dunno06:49
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Skip the manual cleaning test if software RAID testing is enabled  https://review.opendev.org/68116606:52
*** igordc has quit IRC07:00
openstackgerritRachit Kapadia proposed openstack/ironic master: DRAC: Fix a bug for delete_configuration with multiple controllers  https://review.opendev.org/68066007:04
*** tesseract has joined #openstack-ironic07:05
*** rcernin has quit IRC07:09
*** ociuhandu has joined #openstack-ironic07:14
openstackgerritRachit Kapadia proposed openstack/ironic master: DRAC: Fix a bug for delete_configuration with multiple controllers  https://review.opendev.org/68066007:19
*** threestrands has quit IRC07:20
*** ociuhandu has quit IRC07:21
*** rpittau|afk is now known as rpittau07:28
rpittaugood morning ironic! o/07:28
*** tssurya has joined #openstack-ironic07:31
*** gkadam has quit IRC07:34
dtantsurhey rpittau07:43
*** pradip_k has quit IRC07:43
rpittauhi dtantsur :)07:43
*** trident has quit IRC07:50
*** rachit7 has quit IRC07:59
*** trident has joined #openstack-ironic08:01
openstackgerritDmitry Tantsur proposed openstack/ironic master: CI: move the fasttrack job to the experimental pipeline  https://review.opendev.org/68117808:01
openstackgerritDmitry Tantsur proposed openstack/ironic master: CI: move the fast-track job to the experimental pipeline  https://review.opendev.org/68117808:03
openstackgerritDmitry Tantsur proposed openstack/ironic master: Revert "CI: move the fast-track job to the experimental pipeline"  https://review.opendev.org/68118008:05
*** e0ne has joined #openstack-ironic08:06
*** priteau has joined #openstack-ironic08:07
*** jtomasek has joined #openstack-ironic08:07
openstackgerritDmitry Tantsur proposed openstack/ironic-tempest-plugin master: Add fast-track testing capability  https://review.opendev.org/63703908:10
*** bobmel has quit IRC08:10
*** dougsz has joined #openstack-ironic08:12
openstackgerritDmitry Tantsur proposed openstack/ironic-python-agent master: Adds CI job that uses DIB build images  https://review.opendev.org/67623908:17
*** rachit7 has joined #openstack-ironic08:18
*** ociuhandu has joined #openstack-ironic08:20
openstackgerritRachit Kapadia proposed openstack/ironic master: DRAC: Fix a bug for delete_configuration with multiple controllers  https://review.opendev.org/68066008:22
openstackgerritMerged openstack/networking-baremetal master: Build pdf doc  https://review.opendev.org/66678808:22
*** ociuhandu has quit IRC08:25
*** ociuhandu has joined #openstack-ironic08:26
*** ociuhandu has quit IRC08:30
*** ociuhandu has joined #openstack-ironic08:30
*** alexmcleod has joined #openstack-ironic08:31
*** derekh has joined #openstack-ironic08:33
*** ociuhandu has quit IRC08:40
*** ociuhandu has joined #openstack-ironic08:41
*** ociuhandu has quit IRC08:44
* dtantsur has finished cleaning up the ironic gate, please do not recheck anything except for the gate fix08:48
*** yaawang_ has joined #openstack-ironic08:49
*** yaawang has quit IRC08:49
* kaifeng is surprised to know that how a patch gets removed from the queue09:03
openstackgerritDmitry Tantsur proposed openstack/ironic master: Release notes cleanup for 13.0.0  https://review.opendev.org/67998509:04
dtantsurkaifeng: I've learnt this trick from tripleo people.09:05
dtantsurthey have to do it more often than we :)09:06
kaifengheh, skill++09:06
*** rachit7 has quit IRC09:12
*** yaawang_ has quit IRC09:33
*** yaawang has joined #openstack-ironic09:33
*** yaawang has quit IRC09:37
*** yaawang has joined #openstack-ironic09:37
*** rcernin has joined #openstack-ironic09:38
*** yaawang has quit IRC09:41
*** yaawang has joined #openstack-ironic09:43
openstackgerritMerged openstack/ironic-prometheus-exporter master: Move timestamp registry to a sharable 'header' module  https://review.opendev.org/68043209:54
openstackgerritMerged openstack/bifrost master: Revert "Make the opensuse-15 jobs non-voting"  https://review.opendev.org/68116310:19
*** rachit7 has joined #openstack-ironic10:27
openstackgerritIlya Etingof proposed openstack/ironic-prometheus-exporter master: Add Redfish metrics support  https://review.opendev.org/68073110:30
openstackgerritIlya Etingof proposed openstack/ironic-prometheus-exporter master: Add metrics descriptions  https://review.opendev.org/68100710:30
openstackgerritIlya Etingof proposed openstack/ironic-prometheus-exporter master: Add metrics descriptions  https://review.opendev.org/68100710:32
*** jawad_axd has quit IRC10:40
dtantsurhow is it called when a patch you need fails in the CI the most?10:43
dtantsurwe may need to land https://review.opendev.org/#/c/679949/ pretty soon as well10:47
patchbotpatch 679949 - ironic - Correctly set OS_TEST_TIMEOUT in the multinode CI job - 1 patch set10:47
rpittaudtantsur: Murphy's Law10:50
dtantsurindeed10:52
kaifengurh10:55
*** ociuhandu has joined #openstack-ironic11:00
*** ociuhandu has quit IRC11:01
*** ociuhandu has joined #openstack-ironic11:07
*** stendulker has quit IRC11:20
*** jawad_axd has joined #openstack-ironic11:20
*** rh-jelabarre has joined #openstack-ironic11:29
*** rh-jelabarre has quit IRC11:30
*** rh-jelabarre has joined #openstack-ironic11:30
*** priteau has quit IRC11:38
*** bfournie has quit IRC11:42
*** goldyfruit_ has quit IRC12:18
*** Lucas_Gray has joined #openstack-ironic12:25
*** Lucas_Gray has quit IRC12:32
*** Wryhder has joined #openstack-ironic12:32
*** Wryhder is now known as Lucas_Gray12:33
*** bfournie has joined #openstack-ironic12:36
*** derekh has quit IRC12:43
*** derekh has joined #openstack-ironic12:43
dtantsurwhile the ironic CI is not feeling well, could someone please check the sushy-tools reviews: https://review.opendev.org/#/q/project:openstack/sushy-tools+status:open12:46
dtantsuris there anything we have to land in ironic-inspector in train? kaifeng?12:54
*** Goneri has joined #openstack-ironic12:57
*** hamzy_ has joined #openstack-ironic12:59
*** mbeierl has joined #openstack-ironic13:04
*** tzumainn has joined #openstack-ironic13:08
*** rloo has joined #openstack-ironic13:11
openstackgerritMerged openstack/ironic-prometheus-exporter master: Add Redfish metrics support  https://review.opendev.org/68073113:16
openstackgerritMerged openstack/ironic-prometheus-exporter master: Add metrics descriptions  https://review.opendev.org/68100713:16
*** mbeierl has quit IRC13:16
*** sthussey has joined #openstack-ironic13:18
*** goldyfruit_ has joined #openstack-ironic13:23
openstackgerritMerged openstack/bifrost master: Add quotes around ansible ipv4_gateway variable  https://review.opendev.org/68103513:36
*** jawad_axd has quit IRC13:43
openstackgerritMerged openstack/sushy-tools master: Fix non-functional driver configuration on Py2  https://review.opendev.org/67945613:44
* etingof holds his breath for Zuul to digest the avalanche of merges...13:44
*** rcernin has quit IRC13:45
*** jawad_axd has joined #openstack-ironic13:45
*** jawad_axd has quit IRC13:45
openstackgerritMerged openstack/ironic master: Release notes cleanup for 13.0.0  https://review.opendev.org/67998513:45
openstackgerritMerged openstack/sushy-tools master: Remove obsolete limitation from the doc  https://review.opendev.org/68043613:47
*** jawad_axd has joined #openstack-ironic13:49
openstackgerritIlya Etingof proposed openstack/sushy-tools master: Store virtual media device changes persistently  https://review.opendev.org/67744913:49
openstackgerritDmitry Tantsur proposed openstack/ironic master: [WIP] Foundation for boot/network management for in-band inspection  https://review.opendev.org/68126113:52
*** jawad_axd has quit IRC13:53
openstackgerritMerged openstack/sushy-tools master: Add Volume resource support  https://review.opendev.org/67711913:55
*** cdearborn has joined #openstack-ironic13:56
*** ociuhandu has quit IRC13:57
*** ociuhandu has joined #openstack-ironic13:58
*** jtomasek has quit IRC14:00
*** jtomasek has joined #openstack-ironic14:00
*** tkajinam has joined #openstack-ironic14:02
*** ociuhandu has quit IRC14:03
*** rachit7 has quit IRC14:08
dtantsuroh, we seem to sometimes run out of space in multinode as well Oo14:19
openstackgerritMark Goddard proposed openstack/ironic master: Add iDRAC RAID deploy steps  https://review.opendev.org/64173114:21
openstackgerritMark Goddard proposed openstack/ironic master: Don't resume deployment on heartbeat when polling  https://review.opendev.org/68127214:21
mgoddard^ finally, idrac raid deploy step is no longer WIP14:22
mgoddardhad to add another patch14:23
openstackgerritDmitry Tantsur proposed openstack/ironic master: Correctly set OS_TEST_TIMEOUT in the multinode CI job  https://review.opendev.org/67994914:24
dtantsurrpittau: could you re-review please ^^^ also reduced the disk size here14:24
dtantsur(we may need to reduce them globally)14:24
dtantsurmgoddard: \o/14:24
*** Lucas_Gray has quit IRC14:24
dtantsurfolks, I had an early start today, so going a bit earlier as well14:25
dtantsurplease keep an eye on the CI fixing patches14:25
donnydI am hoping someone can give me a push in the right direction with BM and floating ips14:25
*** Lucas_Gray has joined #openstack-ironic14:25
*** dtantsur is now known as dtantsur|afk14:25
mgoddardrpioso, cdearborn: I would love a review of https://review.opendev.org/#/c/64173114:25
patchbotpatch 641731 - ironic - Add iDRAC RAID deploy steps - 16 patch sets14:25
clarkbdid you see my note that you only have ~36gb total on / for /usr and logs and your images? so 5*7 amd 5*12 isnt really small enough. Are those upper bounds?14:27
donnydThe issue I am having is they don't work14:27
cdearbornmgoddard: will try to get to it today14:27
mgoddardcdearborn: thanks :)14:27
dtantsur|afkclarkb: it does pass so far, I'm assuming qemu is smart enough not to allocate too much at once14:27
clarkbdtantsur|afk: well it depends on the cloud. Did you verify it passeson rackspace?14:28
dtantsur|afkclarkb: I had a feeling it did, the feeling might be wrong though.14:28
dtantsur|afkclarkb: is /opt always present and big?14:29
*** Lucas_Gray has quit IRC14:30
clarkbdtantsur|afk: on the clouds with smaller root disks /opt is where we put the extra disk for devstack since that is where devstack runs out of. I dont know if we do it for all jobs but definitely do on devstack jobs14:31
clarkbon other clouds /opt is same fs as / so will be shared with bigger disk14:31
dtantsur|afkaha, so we can just unconditionally move the libvirt pool to /opt? I can look into that tomorrow.14:31
dtantsur|afkI still think reducing the disk size is not a wrong thing to do14:31
clarkbare you running cirros too?14:32
dtantsur|afkin most jobs14:32
clarkbif so those diaks can be like 512MB14:32
clarkbmaybe even smaller14:32
dtantsur|afkironic requires special handling with such small disks. it assumes the disks are more real life sized (4+ Gi)14:32
*** ociuhandu has joined #openstack-ironic14:32
*** ociuhandu has quit IRC14:33
dtantsur|afknothing too difficult, but a bit further away from a production configuration14:33
clarkbassumptions like that make testing hard (vinder does this too as you cantprovision volumes smaller than 1GB)14:33
clarkbpersonally I see those asbugs since testing is important14:33
*** ociuhandu has joined #openstack-ironic14:33
dtantsur|afksupport for disks of 0.5G is cirros-specific. I don't think a real OS would even fit there.14:34
tzumainndtantsur|afk, hi! thanks for the comments on the spec; just for clarification, your last comment means that the 'owner' was never meant as a node admin; it's meant as someone who can - use? - the node however they'd like?14:35
dtantsur|afktzumainn: hmm, we may be using a different terminology. in our world, admin is a role that owns the *cluster*.14:35
dtantsur|afk*owner* owns only some nodes, but not the whole cluster14:35
tzumainnsorry for what may be an obvious question, but - is a cluster the entire ironic deployment, or something else?14:36
dtantsur|afkyep, all the nodes14:36
*** ianychoi_ is now known as ianychoi14:37
dtantsur|afkcurrently ironic API is usually deployed as admin-only, i.e. only users with an admin role can do anything with bare metal nodes14:37
*** Lucas_Gray has joined #openstack-ironic14:37
tzumainnso *could* an owner be simply defined as a single-node-only administrator?14:37
dtantsur|afkif you wish :)14:37
*** ociuhandu has quit IRC14:37
clarkbdtantsur|afk: network appliance images tend to be tiny too and it would be totally valid to deploy them with ironic14:37
tzumainnokay, I'll add that clarification to the spec and let the wisdom of the masses tell me if it's a bad clarification : )14:37
tzumainnthanks!14:37
clarkbbut ignoring that testing is important14:38
clarkband scaling down workloads makes testing the control plane much easier14:38
clarkbsee also novas fake drover14:38
dtantsur|afkon a related note, we may need to move away from cirros since it doesn't have grub214:38
dtantsur|afkwe may reconsider the 4G limitation, but it requires code changes, which are hard to do when the CI is mostly red14:39
*** ociuhandu has joined #openstack-ironic14:39
dtantsur|afktzumainn: you're welcome, thanks for picking this work up14:39
* dtantsur|afk really has to run now, sorry all!14:39
*** tzumainn has quit IRC14:43
*** e0ne has quit IRC14:44
*** e0ne has joined #openstack-ironic14:44
openstackgerritMark Goddard proposed openstack/ironic master: Add iDRAC RAID deploy steps  https://review.opendev.org/64173114:45
openstackgerritIlya Etingof proposed openstack/ironic master: [WIP] Add iDRAC boot interface  https://review.opendev.org/67249814:47
*** tzumainn has joined #openstack-ironic14:48
fungian alternative, if the image sizes/counts can't be reduced, is to stop running those jobs altogether since they currently waste significant amounts of job resources14:49
*** Lucas_Gray has quit IRC14:52
*** michchap has joined #openstack-ironic14:54
donnydnm, i figured it out14:56
clarkbfwiw I was able to track down some jobs that did run on rax14:56
rpittaufungi: I'm not sure that's something we could actually do :/14:56
clarkbthey used 30GB of / so were ~6GB under14:56
clarkbI think that means qcow2s there are thinly provisioned14:56
rpiosomgoddard: Reviewing it is on my short list.14:56
clarkband we are under the limit now, so getting that change in today will help14:56
mgoddardrpioso: excellent14:57
clarkbbut longer term you may want mitigation that avoids the limit entirely14:57
fungirpittau: there are two ways to stop running the jobs: the ironic team can disable them in their job configuration, or the opendev sysadmins can disable all testing for ironic in the global configuration14:57
clarkb(because theoretically you can still go over the limit with the limit on those qcow2s)14:57
fungirpittau: but yes, alternatives to those options would be preferable14:57
clarkbrpioso: out of curiousity what prevents smaller disks?14:58
clarkber sorry rpioso that was for rpittau (tab complete fail)14:59
rpittaufungi: I'm also pretty sure there are better ways to deal with that :)14:59
rpiosoclarkb: No worries :)15:00
clarkbrpittau: where does that lower bound come from in ironic?15:03
clarkb(In cinders case from my example above the issue is they take whole integer volume sizes without units and the unit is assumed to be gigabyte so 1GB is the smallest volume size)15:04
*** jawad_axd has joined #openstack-ironic15:06
rpittauclarkb: mainly from caching images, it can take a lot of space on disk depending on various options15:10
clarkbyes I know its the images, but I'm told we can't make those images smaller (which seems like the natural fix especially if testing with cirros)15:12
*** tkajinam has quit IRC15:13
*** tssurya has quit IRC15:13
openstackgerritMark Goddard proposed openstack/python-ironicclient master: Fix AttributeError in negotiate_version  https://review.opendev.org/67594015:13
clarkbhttp://zuul.openstack.org/build/1a507f48907b469bbdb22e184e72cbe0/log/controller/logs/df.txt.gz that one was actually much closer to the limit than the othe 315:17
clarkbof 6 ironic-standalone jobs on that fix change 4 ran on rax. 3 had 6GB of headroom and one had 40MB of headroom15:17
clarkbso this is unlikely to be a longterm fix. (maybe we can go all the way down to the lower bound size (4GB I think?)15:18
*** ociuhandu has quit IRC15:19
rpittauclarkb: I think one longterm fix that we might consider would be moving some stuff to /opt instead of using /15:19
*** ociuhandu has joined #openstack-ironic15:19
rpittaualso, there might be some margin of size optimization but not that much15:20
clarkbthat will help but the upper bound size is still greater than the size of the ephemeral disk iirc15:20
clarkb12 * 5 = 60GB. We have ~57GB on those rax nodes after the git repos and devstack stuff looks like15:21
clarkb(will be close)15:21
clarkbdefinitely more room than / though15:21
*** ociuhandu has quit IRC15:22
*** ociuhandu has joined #openstack-ironic15:23
*** ociuhandu has quit IRC15:28
*** ociuhandu has joined #openstack-ironic15:31
*** gyee has joined #openstack-ironic15:35
*** ociuhandu has quit IRC15:38
*** ociuhandu has joined #openstack-ironic15:47
*** ociuhandu has quit IRC15:58
*** ociuhandu has joined #openstack-ironic15:58
*** ociuhandu has quit IRC16:02
*** ociuhandu has joined #openstack-ironic16:03
*** tesseract has quit IRC16:06
openstackgerritMark Goddard proposed openstack/ironic master: Don't resume deployment on heartbeat when polling  https://review.opendev.org/68127216:06
openstackgerritMark Goddard proposed openstack/ironic master: Add iDRAC RAID deploy steps  https://review.opendev.org/64173116:06
*** rpittau is now known as rpittau|afk16:06
rpittau|afkgoodnight o/16:06
openstackgerritTzu-Mainn Chen proposed openstack/ironic-specs master: Expose node owner information to oslo.policy checks  https://review.opendev.org/68106616:13
openstackgerritVladyslav Drok proposed openstack/ironic master: Read in non-blocking fashion when starting console  https://review.opendev.org/68108716:16
*** e0ne has quit IRC16:18
openstackgerritVladyslav Drok proposed openstack/ironic master: Read in non-blocking fashion when starting console  https://review.opendev.org/68108716:22
*** igordc has joined #openstack-ironic16:39
*** jawad_axd has quit IRC16:41
*** dougsz has quit IRC16:41
*** igordc has quit IRC16:44
*** derekh has quit IRC16:44
*** bobmel has joined #openstack-ironic16:44
*** tzumainn has quit IRC16:57
*** alexmcleod has quit IRC16:57
*** ociuhandu_ has joined #openstack-ironic17:00
*** ociuhandu has quit IRC17:03
*** ociuhandu_ has quit IRC17:05
openstackgerritOpenStack Release Bot proposed openstack/python-ironicclient stable/train: Update .gitreview for stable/train  https://review.opendev.org/68130617:06
openstackgerritOpenStack Release Bot proposed openstack/python-ironicclient stable/train: Update TOX/UPPER_CONSTRAINTS_FILE for stable/train  https://review.opendev.org/68130717:06
openstackgerritOpenStack Release Bot proposed openstack/python-ironicclient master: Update master for stable/train  https://review.opendev.org/68130817:06
*** bobmel has quit IRC17:10
openstackgerritMerged openstack/networking-generic-switch master: Build pdf doc  https://review.opendev.org/66678617:20
*** hjensas is now known as hjensas|afk17:29
*** jtomasek has quit IRC17:37
*** igordc has joined #openstack-ironic17:44
*** trident has quit IRC17:46
*** e0ne has joined #openstack-ironic17:46
*** igordc has quit IRC17:50
*** igordc has joined #openstack-ironic17:57
*** trident has joined #openstack-ironic17:59
*** igordc has quit IRC18:06
*** igordc has joined #openstack-ironic18:06
*** igordc has quit IRC18:15
*** ociuhandu has joined #openstack-ironic18:39
*** ociuhandu has quit IRC18:44
*** goldyfruit___ has joined #openstack-ironic18:49
*** gyee has quit IRC18:49
*** gyee has joined #openstack-ironic18:50
*** goldyfruit_ has quit IRC18:52
*** ricolin has quit IRC18:58
*** pcaruana has quit IRC19:20
*** e0ne has quit IRC20:19
*** ociuhandu has joined #openstack-ironic20:37
*** ociuhandu has quit IRC20:52
*** ociuhandu has joined #openstack-ironic20:53
*** ociuhandu has quit IRC20:57
*** Goneri has quit IRC21:00
*** henriqueof1 has joined #openstack-ironic21:30
*** henriqueof has quit IRC21:31
*** bfournie has quit IRC21:36
*** rh-jelabarre has quit IRC21:37
*** rh-jelabarre has joined #openstack-ironic21:43
*** bfournie has joined #openstack-ironic22:29
*** ociuhandu has joined #openstack-ironic22:30
*** ociuhandu has quit IRC22:35
*** fbouliane has joined #openstack-ironic22:50
*** rcernin has joined #openstack-ironic22:59
*** henriqueof1 has quit IRC23:02
openstackgerritRichard G. Pioso proposed openstack/ironic master: Add first idrac HW type Redfish interface support  https://review.opendev.org/68104623:14
*** goldyfruit___ has quit IRC23:17
*** sthussey has quit IRC23:46
« Monday, 2019-09-09 Index Wednesday, 2019-09-11 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!