Tuesday, 2017-11-21

*** thework1 has joined #rdo00:05
rdogerritrdo-trunk created rdoinfo master: Bump rdoinfo queens-uc tags to latest upper-constraints  https://review.rdoproject.org/r/1059400:06
rdogerritrdo-trunk created rdoinfo master: Bump rdoinfo pike tags to latest upper-constraints  https://review.rdoproject.org/r/1059500:06
rdogerritrdo-trunk created rdoinfo master: Promote CBS tags update for pike-testing  https://review.rdoproject.org/r/1059600:07
*** jrist-afk is now known as jrist00:12
*** tosky has quit IRC00:18
*** kellyp has quit IRC00:24
miscdmsimard: why raise it ? we usually keep it low so we can migrate fast between VM00:27
EmilienMdmsimard: ack00:36
*** dmacpher has joined #rdo00:40
*** Goneri has quit IRC00:42
*** Sandy619 has joined #rdo00:43
*** Sandy619 has quit IRC00:45
*** kellyp has joined #rdo01:00
rdogerritMerged rdoinfo master: Bump rdoinfo queens-uc tags to latest upper-constraints  https://review.rdoproject.org/r/1059401:01
*** kellyp has quit IRC01:05
*** suresh12 has quit IRC01:06
*** jhershbe has joined #rdo01:07
*** suresh12 has joined #rdo01:18
*** daidv has quit IRC01:18
*** daidv has joined #rdo01:19
*** thrash is now known as thrash|g0ne01:31
*** kellyp has joined #rdo01:34
*** kellyp has quit IRC01:39
*** liujiong has joined #rdo01:39
*** ihrachys has quit IRC01:42
*** gvrangan has joined #rdo01:43
*** Son_Goku has quit IRC01:43
*** gvrangan has quit IRC01:47
*** athomas has quit IRC01:49
*** Son_Goku has joined #rdo01:53
dmsimardmisc: in practice is still takes >4hrs to propagate regardless because resolvers can take TTLs as a suggestion :)01:57
dmsimardmisc: i.e, powerdns/bind/unbound have configurations to enforce a minimum TTL01:57
miscdmsimard: well, we didn't wait 4h each time we did sitch VM01:59
miscswitch01:59
misc(but yes, I keep telling to people that we can't rely on the TTL to be magically propagated everywhere)01:59
dmsimardmisc: We'll lower the TTL back eventually, this is an experiment to confirm a theory we have -- we're seeing DNS lookup failures through OpenDNS and Google DNS for records under rdoproject.org just like we do for some other specific domains (fedoraproject, github, some others) and they all have a low TTL.02:00
miscbut as long as it propagate fast enough for them to see it got changed, i think that's ok02:00
miscdmsimard: so if that's temporary, I would be much ok to say "go for it"02:00
miscdmsimard: yu can also set the TTL just for the record that fail if you want to verify the theory :)02:01
dmsimardOur current local recursive DNS (unbound) is configured to respect the TTL so if the TTL is short, it means there are more forwarded queries... and OpenStack runs like 20k jobs per day02:01
dmsimardmisc: eh there was one TTL for the whole zone, I wasn't going to modify the entire zone to re-add the same TTL everywhere except for one record02:01
miscdmsimard: no, i was more thinking of having the TTL increased for 1 specific record02:02
dmsimardmisc: anyway, if the theory proves true (we'll give it a week), we'll start enforcing a minimum TTL02:02
dmsimardmisc: I understood what you meant, but I didn't see TTLs for each individual record, only one for the zone02:03
dmsimardso inherited02:03
miscdmsimard: you can just add one later in the zone, no ?02:04
misc(iirc)02:04
dmsimardhmmmm I'm not sure02:05
dmsimardand actually now that you made me look again I think I screwed up02:05
*** rwsu has joined #rdo02:06
dmsimardI updated the TTL on the SOA02:06
dmsimardbut I see a "$TTL 300" at the top of the zone02:06
dmsimardyeah, oops02:07
dmsimardI'll change that02:07
miscso, looking at my own zone, I do have a $TTL 15m02:07
misclater in the file, so the syntax is valid, and I think it would work like we do for $origin02:07
miscbut well, the whole zone is ok, no reason to make complex stuff :)02:08
*** scorcoran_afk has joined #rdo02:09
dmsimardpushed a new commit to change around SOA TTL and actual TTL02:09
*** jlabarre has quit IRC02:12
Duckquack02:14
Duckdmsimard: quack02:16
dmsimardmeow02:16
DuckI think people don't want to be annoyed when they do migration, because if you need to decrease for a migration then you have to wait first for the decrese to be propagated02:16
Duckbut that's clearly not a good use of the cache02:17
Duckso I think that's a good idea02:17
dmsimardDuck: yeah, you can lower ahead of time if you know you have a migration coming (that's what I usually do)02:18
dmsimardI guess it doesn't work so well if the migration isn't planned (i.e, disaster recovery)02:18
dmsimardI'm used to 14400 as a default TTL (4 hours)02:18
dmsimardbut you're still at the mercy of whatever resolver people end up using anyway02:19
dmsimardsome records on I add on rdoproject end up taking like a whole day02:19
Duckyep, this needs planning, which is not what we seem to do best :-/02:20
dmsimardThe experiment will run for a week, I'll lower the TTL back afterwards :P02:21
*** Son_Goku has quit IRC02:27
*** Amita has joined #rdo02:37
*** suresh12 has quit IRC02:38
*** Son_Goku has joined #rdo02:45
*** fzdarsky_ has joined #rdo02:47
*** athomas has joined #rdo02:47
*** fzdarsky has quit IRC02:51
*** athomas has quit IRC02:52
*** absubram has quit IRC02:58
*** jhershbe has quit IRC03:01
*** Son_Goku has quit IRC03:02
*** Son_Goku has joined #rdo03:04
*** Son_Goku has quit IRC03:04
*** suresh12 has joined #rdo03:07
*** suresh12 has quit IRC03:11
*** gvrangan has joined #rdo03:17
*** links has joined #rdo03:19
*** Son_Goku has joined #rdo03:24
*** Son_Goku has quit IRC03:24
*** jhershbe has joined #rdo03:26
*** rlandy|rover has quit IRC03:27
*** jhershbe has quit IRC03:31
*** athomas has joined #rdo03:39
*** scorcoran_afk has quit IRC03:49
*** gszasz has quit IRC03:53
*** gildub has quit IRC03:53
*** shreshtha has joined #rdo03:56
*** Son_Goku has joined #rdo03:57
*** imcleod has quit IRC04:07
*** ykarel|away has joined #rdo04:09
*** gildub has joined #rdo04:12
*** ykarel|away is now known as ykarel04:13
*** gvrangan has quit IRC04:14
*** Son_Goku has quit IRC04:18
*** dbecker has quit IRC04:24
*** aditya_r has joined #rdo04:29
*** jatanmalde has joined #rdo04:30
*** dbecker has joined #rdo04:37
*** anilvenkata has joined #rdo04:40
*** gkadam has joined #rdo04:46
*** ratailor has joined #rdo04:59
*** aditya_ra has joined #rdo05:12
*** janki has joined #rdo05:13
*** aditya_r has quit IRC05:15
*** aditya_ra has quit IRC05:19
*** suresh12 has joined #rdo05:20
*** dpawar has joined #rdo05:27
*** udesale has joined #rdo05:32
*** kellyp has joined #rdo05:34
*** pgadiya has joined #rdo05:37
*** pgadiya has quit IRC05:37
*** pgadiya has joined #rdo05:38
*** pgadiya has quit IRC05:38
*** kellyp has quit IRC05:38
*** pcaruana has joined #rdo05:39
*** lkuchlan_ has joined #rdo05:44
*** lkuchlan_ has quit IRC05:44
*** lkuchlan has joined #rdo05:44
*** dneary has quit IRC05:47
*** lkuchlan is now known as lkuchlan_wfh05:47
*** akshaysh has joined #rdo05:49
*** akshay has joined #rdo05:57
*** dneary has joined #rdo06:00
*** pcaruana has quit IRC06:13
*** akshay has quit IRC06:14
*** sasha2 has quit IRC06:15
*** dciabrin has quit IRC06:18
*** dciabrin has joined #rdo06:19
*** sbulage|afk has joined #rdo06:21
*** dciabrin has quit IRC06:22
*** janki has quit IRC06:23
*** dyasny_ has quit IRC06:23
*** janki has joined #rdo06:23
*** dyasny has joined #rdo06:23
*** dciabrin has joined #rdo06:25
*** bandini has quit IRC06:25
*** bandini has joined #rdo06:26
*** dmacpher has quit IRC06:28
*** suresh12 has quit IRC06:31
*** suresh12 has joined #rdo06:33
*** suresh12 has quit IRC06:38
*** paragan has joined #rdo06:42
*** jtomasek has joined #rdo06:50
*** dpawar has quit IRC06:53
*** dsneddon has quit IRC06:57
*** rcernin has quit IRC06:59
*** dtantsur|afk is now known as dtantsur07:01
*** gildub has quit IRC07:01
*** iranzo has joined #rdo07:03
*** josecastroleon has quit IRC07:04
*** josecastroleon1 has joined #rdo07:04
*** spectr has joined #rdo07:11
*** sbulage|afk is now known as sbulage|lunch07:12
*** spectr has quit IRC07:12
*** jprovazn has joined #rdo07:12
*** oidgar has joined #rdo07:12
*** egonzalez has joined #rdo07:18
*** sbonazzo has joined #rdo07:18
*** gkadam has quit IRC07:20
*** gildub has joined #rdo07:26
*** pcaruana has joined #rdo07:30
*** udesale has quit IRC07:32
*** smeyer has joined #rdo07:34
*** emalikov has joined #rdo07:35
*** josecastroleon1 has quit IRC07:40
*** agurenko has joined #rdo07:41
*** tshefi has joined #rdo07:42
*** hseipp has joined #rdo07:43
*** ykarel is now known as ykarel|lunch07:44
*** shardy has joined #rdo07:49
*** florianf has joined #rdo07:50
*** nyechiel has joined #rdo08:04
*** dpawar has joined #rdo08:06
*** karthiks has quit IRC08:06
*** d0ugal has joined #rdo08:08
*** josecastroleon has joined #rdo08:12
*** ushkalim has joined #rdo08:12
*** sbulage|lunch is now known as sbulage08:13
*** ccamacho has joined #rdo08:14
*** sbulage has quit IRC08:15
*** lkuchlan_wfh has quit IRC08:16
*** lkuchlan_wfh has joined #rdo08:16
*** jparrill has joined #rdo08:18
*** aufi has joined #rdo08:19
*** udesale has joined #rdo08:19
*** rbartal has joined #rdo08:19
*** hseipp has quit IRC08:20
*** tesseract has joined #rdo08:23
*** karthiks has joined #rdo08:23
*** hseipp has joined #rdo08:25
*** shreshtha_ has joined #rdo08:25
*** shreshtha has quit IRC08:26
*** sbulage has joined #rdo08:27
*** ykarel|lunch is now known as ykarel08:31
*** lkuchlan_wfh has quit IRC08:31
*** lkuchlan_wfh has joined #rdo08:31
*** karthiks has quit IRC08:33
*** karthiks has joined #rdo08:33
*** mcornea has joined #rdo08:34
*** shreshtha_ has quit IRC08:37
*** shreshtha has joined #rdo08:37
*** karimb has joined #rdo08:38
*** ragiman has joined #rdo08:42
*** jpena|off is now known as jpena08:43
jpenagood morning08:45
*** aditya_r has joined #rdo08:46
*** cylopez has joined #rdo08:46
*** tumble has quit IRC08:49
*** sshnaidm|afk is now known as sshnaidm08:50
*** bsurfer has joined #rdo08:50
*** tumble has joined #rdo08:51
*** jlibosva has joined #rdo08:51
*** jprovazn has quit IRC08:53
*** gkadam has joined #rdo09:01
*** amoralej|off is now known as amoralej09:01
amoralejgood morning09:01
*** shreshtha has quit IRC09:02
rdogerritAlfredo Moralejo proposed rdoinfo master: Bump rdoinfo pike tags to latest upper-constraints  https://review.rdoproject.org/r/1059509:02
*** shreshtha has joined #rdo09:03
rdogerritJavier Peña proposed config master: Create project for rsdclient  https://review.rdoproject.org/r/1057909:04
rdogerritJavier Peña proposed config master: Create project for rsd-lib  https://review.rdoproject.org/r/1057809:04
*** MrAbaddon has quit IRC09:04
*** Sandy619 has joined #rdo09:05
rdogerritMerged openstack/oslo-utils-distgit rpm-master: Use explicit Python version in "python2-oslo-utils-tests" package name  https://review.rdoproject.org/r/1059309:07
*** jpich has joined #rdo09:07
*** tumble has quit IRC09:08
*** tdasilva has quit IRC09:09
*** honza has quit IRC09:09
*** gszasz has joined #rdo09:10
*** tumble has joined #rdo09:10
*** oidgar has quit IRC09:13
*** honza has joined #rdo09:14
*** honza is now known as Guest1908509:14
*** tdasilva has joined #rdo09:15
*** dtantsur_ has joined #rdo09:21
*** dtantsur has quit IRC09:22
*** dtantsur_ is now known as dtantsur09:22
*** gildub has quit IRC09:22
*** jprovazn has joined #rdo09:23
*** abregman has joined #rdo09:27
rdogerritMerged config master: Create project for rsdclient  https://review.rdoproject.org/r/1057909:28
rdogerrithguemar proposed rdoinfo master: Promote CBS tags update for pike-testing  https://review.rdoproject.org/r/1059609:28
*** rdogerrit has quit IRC09:29
*** Sandy619 has quit IRC09:30
*** mvk has quit IRC09:30
*** mvk has joined #rdo09:30
*** ddmitriev has left #rdo09:31
*** lucas-afk is now known as lucasagomes09:32
*** rdogerrit has joined #rdo09:35
rdogerritMerged openstack/neutron-distgit rpm-master: Destroy patch ports only if canary flow is not present  https://review.rdoproject.org/r/1048009:35
*** derekh has joined #rdo09:41
rdogerritMerged rdoinfo master: Bump OVS from 2.7.2 to 2.7.3 for Pike testing  https://review.rdoproject.org/r/1058609:44
rdogerritJavier Peña proposed config master: Create project for rsd-lib  https://review.rdoproject.org/r/1057809:45
*** apevec has joined #rdo09:47
*** gfidente has joined #rdo09:56
*** gfidente has quit IRC09:56
*** gfidente has joined #rdo09:56
rdogerritMartin Mágr proposed config master: Adds centos-opstools/rubygem-sensu-extensions-check-dependencies  https://review.rdoproject.org/r/1015109:57
*** sbulage is now known as sbulage|brb09:57
rdogerritMerged config master: Create project for rsd-lib  https://review.rdoproject.org/r/1057810:00
rdogerrithguemar proposed config master: Move the DLRN-rpmbuild-fedora job to the check pipeline  https://review.rdoproject.org/r/1056210:00
*** rdogerrit has quit IRC10:00
*** yselkowitz has quit IRC10:01
*** aditya_r has quit IRC10:01
*** rcernin has joined #rdo10:01
*** rdogerrit has joined #rdo10:01
rdogerritMerged config master: Move the DLRN-rpmbuild-fedora job to the check pipeline  https://review.rdoproject.org/r/1056210:01
*** janki has quit IRC10:02
*** janki has joined #rdo10:02
rdogerrithguemar proposed config master: Convert DLRN-rpmbuild job into a builder  https://review.rdoproject.org/r/1058310:02
*** cylopez has quit IRC10:02
*** paramite has joined #rdo10:04
*** daidv has quit IRC10:05
*** hewbrocca_afk is now known as hewbrocca10:07
rdogerritJavier Peña created rdo_gating_scripts master: Add stable/pike branch clone to rdo-populate-project.py  https://review.rdoproject.org/r/1059810:10
*** liujiong has quit IRC10:10
*** tosky has joined #rdo10:12
*** gszasz has quit IRC10:21
*** MrAbaddon has joined #rdo10:21
openstackgerritJavier Peña proposed openstack/packstack master: Add multinode CI job  https://review.openstack.org/51764410:24
*** ratailor has quit IRC10:30
rdogerritJakub Libosvar proposed openstack/neutron-distgit pike-rdo: Destroy patch ports only if canary flow is not present  https://review.rdoproject.org/r/1048110:39
rdogerritJakub Libosvar proposed openstack/neutron-distgit pike-rdo: Destroy patch ports only if canary flow is not present  https://review.rdoproject.org/r/1048110:42
*** ratailor has joined #rdo10:47
rdogerritAlfredo Moralejo proposed gating_scripts master: Separate build existence check and list packages  https://review.rdoproject.org/r/1059010:50
*** ratailor_ has joined #rdo10:53
*** jparrill has quit IRC10:53
rdogerritAlfredo Moralejo proposed gating_scripts master: Separate build existence check and list packages  https://review.rdoproject.org/r/1059010:54
rdogerritMerged gating_scripts master: Separate build existence check and list packages  https://review.rdoproject.org/r/1059010:56
*** ratailor has quit IRC10:56
*** rfolco has joined #rdo10:58
rdogerritMartin Mágr proposed config master: Adds centos-opstools/rubygem-sensu-extensions-check-dependencies  https://review.rdoproject.org/r/1015111:00
*** sbulage|brb is now known as sbulage11:02
*** rfolco is now known as rfolco_brb11:02
*** wolverineav has joined #rdo11:02
*** gszasz has joined #rdo11:06
*** Son_Goku has joined #rdo11:11
rdogerritrdo-trunk created puppet/puppet-horizon-distgit newton-rdo: Update to 9.7.0  https://review.rdoproject.org/r/1059911:11
rdogerritrdo-trunk created puppet/puppet-horizon-distgit ocata-rdo: Update to 10.5.0  https://review.rdoproject.org/r/1060011:12
rdogerritMerged rdoinfo master: Promote CBS tags update for pike-testing  https://review.rdoproject.org/r/1059611:13
rdogerritrdo-trunk created puppet/puppet-horizon-distgit pike-rdo: Update to 11.4.0  https://review.rdoproject.org/r/1060111:13
*** wolverineav has quit IRC11:15
*** apetrich has quit IRC11:16
rdogerritAlfredo Moralejo created gating_scripts master: Fix sourcing rpm-koji-gating-lib.common from other directories  https://review.rdoproject.org/r/1060211:17
*** thrash|g0ne is now known as thrash11:19
*** Son_Goku has quit IRC11:24
*** mvk has quit IRC11:27
*** abregman has quit IRC11:29
*** udesale has quit IRC11:30
*** Son_Goku has joined #rdo11:31
*** abregman has joined #rdo11:33
rdogerritMerged config master: Convert DLRN-rpmbuild job into a builder  https://review.rdoproject.org/r/1058311:33
*** Son_Goku has quit IRC11:38
*** shreshtha has quit IRC11:39
*** sbonazzo has quit IRC11:40
rdogerritMerged config master: Add default branch for k8sclient-distgit  https://review.rdoproject.org/r/986411:43
rdogerritMerged puppet/puppet-horizon-distgit newton-rdo: Update to 9.7.0  https://review.rdoproject.org/r/1059911:48
rdogerritMerged puppet/puppet-horizon-distgit ocata-rdo: Update to 10.5.0  https://review.rdoproject.org/r/1060011:48
*** oidgar has joined #rdo11:48
rdogerritMerged puppet/puppet-horizon-distgit pike-rdo: Update to 11.4.0  https://review.rdoproject.org/r/1060111:48
*** abregman is now known as abregman|mtg11:49
*** raissa has joined #rdo11:50
*** mvk has joined #rdo11:57
*** martinkopec has joined #rdo12:04
*** abregman|mtg has quit IRC12:04
*** jkilpatr has joined #rdo12:04
*** sbulage is now known as sbulage|mtg12:06
rdogerritAlfredo Moralejo proposed rdoinfo master: Bump rdoinfo pike tags to latest upper-constraints  https://review.rdoproject.org/r/1059512:14
*** rhallisey has joined #rdo12:15
*** Son_Goku has joined #rdo12:17
*** paragan has quit IRC12:27
*** Guest19085 is now known as honza12:27
*** julim has quit IRC12:28
*** akshaysh has quit IRC12:32
*** abregman has joined #rdo12:38
*** sbulage|mtg is now known as sbulage12:39
*** morazi has joined #rdo12:43
*** martinkopec has quit IRC12:44
*** MrAbaddon has quit IRC12:44
*** haleyb has quit IRC12:46
*** lucasagomes is now known as lucas-hungry12:49
*** jayg|g0n3 is now known as jayg12:49
*** scorcoran_afk has joined #rdo12:54
*** jprovazn has quit IRC12:54
*** rfolco_brb is now known as rfolco12:56
*** jprovazn has joined #rdo12:57
*** jpena is now known as jpena|lunch12:58
*** amoralej is now known as amoralej|lunch13:00
*** pkovar has joined #rdo13:00
*** jlabarre has joined #rdo13:01
*** lkuchlan_wfh has quit IRC13:05
*** dneary has quit IRC13:07
*** sbulage is now known as sbulage|afk13:08
*** dprince has joined #rdo13:15
*** MrAbaddon has joined #rdo13:16
*** sbulage|afk has quit IRC13:16
*** pchavva has joined #rdo13:17
*** flepied_ has joined #rdo13:17
*** flepied__ has quit IRC13:19
*** dneary has joined #rdo13:22
*** shardy is now known as shardy_lunch13:22
rdogerritDaniel Alvarez created rdoinfo master: Bump OVS from 2.7.2 to 2.7.3 for Pike release  https://review.rdoproject.org/r/1060313:27
*** rcernin has quit IRC13:30
*** aditya_r has joined #rdo13:31
*** ratailor_ is now known as ratailor13:31
*** rlandy has joined #rdo13:32
*** suresh12 has joined #rdo13:34
*** apetrich has joined #rdo13:35
*** zenpwner has joined #rdo13:35
*** rlandy is now known as rlandy|rover13:39
*** suresh12 has quit IRC13:39
dmsimardamoralej|lunch, jpena|lunch: fyi there was an interesting discussion about DNS lookup failures upstream yesterday. Let me know when you're back from lunch :P13:44
*** amoralej|lunch is now known as amoralej13:44
amoralejdmsimard, i'm back, just forgot to change the nick13:44
*** imcleod has joined #rdo13:44
amoralej:)13:44
*** trown|outtypewww is now known as trown13:44
*** udesale has joined #rdo13:45
*** shardy_lunch is now known as shardy13:45
*** jprovazn has quit IRC13:46
dmsimardamoralej: so after digging a bit yesterday, we realized that the DNS lookup failures tend to occur on hostnames with low TTLs such as trunk.rdoproject.org (300), github.com (60), etc13:47
amoralejthat's using local unbound?13:47
*** jprovazn has joined #rdo13:47
dmsimardThe current unbound configuration is set to respect whatever TTLs is configured for the records so we end up querying for these domains more often13:47
dmsimardyeah, I'm talking strictly upstream here13:48
amoralejok13:48
*** Son_Goku has quit IRC13:48
dmsimardso we're doing a small scale experiment to see if that theory is true13:48
dmsimardwe increase the TTL for rdoproejct.org from 300 to 3600 and we'll let it run for a week to see if there is a reduction in lookup failures13:48
amoralejok, makes sense13:49
dmsimardif this has an impact, we'll enforce a minimum TTL in unbound -- so that even if github has a TTL of 60 seconds, we'll make it.. 10 minutes or something13:49
dmsimardThere's a configuration to avoid caching bogus/not found records that we'll also have to tweak13:50
apevec60s ?! are they doing round-robin balancing via DNS ?13:50
dmsimardapevec: they have two IPs for A github.com13:50
*** jpena|lunch is now known as jpena13:50
dmsimardmaybe they rotate servers in and out, who knows13:50
*** lucas-hungry is now known as lucasagomes13:51
rdogerritAlfredo Moralejo proposed gating_scripts master: Add tools to automate building dependencies from fedora builds  https://review.rdoproject.org/r/1056913:51
*** jhershbe has joined #rdo13:52
jpenadmsimard: aha, interesting. So by reducing the number of upstream queries in unbound we expect to reduce the frequency of lookup failures13:52
dmsimardjpena: yes and no, by caching successful lookups for longer, we won't be as likely to 1) get an unlucky lookup failure 2) get rate limited13:53
dmsimardwe used to have DNS lookup failures even for review.rdo right ? when cloning things ?13:53
jpenadmsimard: yes, we got those even for config repo updates, which run on review.rdo itself13:54
dmsimardso we should have those in kibana then, let me see13:54
dmsimarder, kibana looks broken13:55
*** sreichar has joined #rdo13:55
*** akshay has joined #rdo13:55
dmsimardkibana's saying elasticsearch is not available13:57
amoralejdmsimard, in review.r.o ?13:57
amoralejwe had similar issue last week13:57
dmsimardout of memory on elasticsearch: http://paste.openstack.org/raw/626950/13:58
dmsimardI just restarted it13:58
dmsimardThe ELK node was a test deployment I had done with tristanC, backed by ephemeral storage13:59
*** pradk has joined #rdo13:59
rdogerritAlfredo Moralejo proposed gating_scripts master: Add tools to automate building dependencies from fedora builds  https://review.rdoproject.org/r/1056913:59
dmsimardI'll add a BZ to redeploy it properly13:59
jpenadmsimard: we have 8GB in the VM, and the elasticsearch heap is just 512m. Maybe we can increase it13:59
dmsimardWe had originally planned to deploy elk/influx/grafana on logs.rdo so logs.rdo was sized accordingly13:59
*** gszasz has quit IRC13:59
dmsimardjpena: right, I saw that too, need to figure out where that is through sfconfig14:00
*** jkilpatr_ has joined #rdo14:00
*** sasha2 has joined #rdo14:00
dmsimardhttps://github.com/softwarefactory-project/sf-config/blob/master/ansible/roles/sf-elasticsearch/defaults/main.yml#L1314:00
*** gszasz has joined #rdo14:02
*** _bsurfer_ has joined #rdo14:02
*** jkilpatr has quit IRC14:03
jpenadmsimard: we have this in sfconfig.yaml14:04
jpenaelasticsearch:14:04
jpena  heap_size: 512m14:04
*** jmelvin has joined #rdo14:04
*** mgariepy has quit IRC14:05
*** abregman has quit IRC14:05
*** bsurfer has quit IRC14:05
dmsimardCan start with 102414:05
dmsimardand see what happens14:05
*** haleyb has joined #rdo14:05
rdogerritMerged config master: Add backup group to hosts file in rdo-infra-playbooks-integration  https://review.rdoproject.org/r/1056514:07
jpenadmsimard: ok, let's give it a try14:07
dmsimardjpena: are you doing it ?14:07
jpenayep14:07
dmsimardk14:07
rdogerritDavid Moreau Simard created config master: Print the Sensu configuration if there is one  https://review.rdoproject.org/r/1060414:11
*** not_rdobot has joined #rdo14:12
dmsimardgetting HTTP 503's on review.rdo.. what is going on..14:14
*** moung is now known as myoung14:14
dmsimardoh, surely sfconfig restarting things14:14
*** mgariepy has joined #rdo14:14
jpenadmsimard: yes, that was it14:14
*** thework1 has joined #rdo14:15
dmsimardprobably a bug in there, restarting something that didn't need to14:15
*** haleyb_ has joined #rdo14:15
dmsimardwe only changed elk config, it shouldn't touch gerrit or apache14:15
*** Goneri has joined #rdo14:17
rdogerritMerged config master: Print the Sensu configuration if there is one  https://review.rdoproject.org/r/1060414:17
dmsimardsshnaidm, adarazs|ruck: that's a LOT of jobs in experimental14:17
dmsimardjust sayin14:17
dmsimardyou have more jobs in experimental than in check14:17
dmsimardhttps://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul/upstream.yaml#L327 is particularly bad14:18
dmsimarddoing even just one "check experimental" must be super expensive14:18
sshnaidmdmsimard, that only seems so, most of them are branch related14:19
*** toure_biab is now known as toure14:19
*** akshayg96 has joined #rdo14:19
sshnaidmdmsimard, usually it will run only 1/3 of them14:19
dmsimardsshnaidm: ah okay14:19
dmsimardsshnaidm: btw in case you missed it http://lists.openstack.org/pipermail/openstack-dev/2017-November/124733.html14:19
dmsimardbrb14:19
*** abregman has joined #rdo14:20
*** ratailor has quit IRC14:21
*** not_rdobot has quit IRC14:21
sshnaidmdmsimard, yeah, I read this.. need to discuss the scope of what we want to do (and when)14:22
*** zaneb has joined #rdo14:25
rdogerritMerged gating_scripts master: Fix sourcing rpm-koji-gating-lib.common from other directories  https://review.rdoproject.org/r/1060214:25
*** jhershbe has quit IRC14:26
*** emalikov has quit IRC14:26
*** suresh12 has joined #rdo14:26
rdogerrithguemar proposed openstack/osops-tools-monitoring-oschecks-distgit pike-rdo: Initial Pike release (62160d10683023c8c9d96f616223d8def88b870d)  https://review.rdoproject.org/r/1019614:31
*** suresh12 has quit IRC14:31
*** akshay has quit IRC14:31
*** gszasz has quit IRC14:32
*** lkuchlan has joined #rdo14:32
*** MrAbaddon has quit IRC14:34
*** Son_Goku has joined #rdo14:34
*** MrAbaddon has joined #rdo14:35
*** anilvenkata has quit IRC14:35
*** nyechiel has quit IRC14:35
*** wolverineav has joined #rdo14:39
*** artom has joined #rdo14:39
*** iranzo has quit IRC14:40
*** iranzo has joined #rdo14:41
*** iranzo has quit IRC14:41
*** iranzo has joined #rdo14:41
rdogerritMerged rdoinfo master: Bump rdoinfo pike tags to latest upper-constraints  https://review.rdoproject.org/r/1059514:42
openstackgerritMerged openstack/packstack stable/pike: Zuul: add file extension to playbook path  https://review.openstack.org/51608814:42
*** egonzalez has quit IRC14:43
rdogerritrdo-trunk created openstack/oslotest-distgit pike-rdo: Update to 2.17.1  https://review.rdoproject.org/r/1060514:43
rdogerritrdo-trunk created openstack/oslo-cache-distgit pike-rdo: Update to 1.25.1  https://review.rdoproject.org/r/1060614:43
rdogerritrdo-trunk created openstack/oslo-serialization-distgit pike-rdo: Update to 2.20.1  https://review.rdoproject.org/r/1060714:43
rdogerritrdo-trunk created openstack/oslo-db-distgit pike-rdo: Update to 4.25.1  https://review.rdoproject.org/r/1060814:43
rdogerritrdo-trunk created openstack/oslo-context-distgit pike-rdo: Update to 2.17.1  https://review.rdoproject.org/r/1060914:43
rdogerritrdo-trunk created openstack/oslo-log-distgit pike-rdo: Update to 3.30.1  https://review.rdoproject.org/r/1061014:44
rdogerritrdo-trunk created openstack/oslo-policy-distgit pike-rdo: Update to 1.25.2  https://review.rdoproject.org/r/1061114:44
*** gszasz has joined #rdo14:44
rdogerritrdo-trunk created openstack/oslo-versionedobjects-distgit pike-rdo: Update to 1.26.1  https://review.rdoproject.org/r/1061214:44
rdogerrithguemar proposed openstack/osops-tools-monitoring-oschecks-distgit pike-rdo: Initial Pike release (62160d10683023c8c9d96f616223d8def88b870d)  https://review.rdoproject.org/r/1019614:44
rdogerritrdo-trunk created openstack/oslo-service-distgit pike-rdo: Update to 1.25.1  https://review.rdoproject.org/r/1061314:44
rdogerritrdo-trunk created openstack/oslo-reports-distgit pike-rdo: Update to 1.22.1  https://review.rdoproject.org/r/1061414:45
rdogerritrdo-trunk created openstack/oslo-privsep-distgit pike-rdo: Update to 1.22.1  https://review.rdoproject.org/r/1061514:45
rdogerritrdo-trunk created openstack/stevedore-distgit pike-rdo: Update to 1.25.1  https://review.rdoproject.org/r/1061614:45
rdogerritrdo-trunk created openstack/taskflow-distgit pike-rdo: Update to 2.14.1  https://review.rdoproject.org/r/1061714:46
*** egonzalez has joined #rdo14:46
*** aditya_ra has joined #rdo14:46
*** aditya_r has quit IRC14:47
*** links has quit IRC14:47
rdogerritAlfredo Moralejo proposed gating_scripts master: Add tools to automate building dependencies from fedora builds  https://review.rdoproject.org/r/1056914:48
*** julim has joined #rdo14:53
*** sreichar has quit IRC14:54
*** ohamada has joined #rdo14:54
*** Son_Goku has quit IRC14:55
*** weshay is now known as weshay_pto14:55
*** ohamada has quit IRC14:56
*** ohamada has joined #rdo14:56
*** ohamada has quit IRC14:56
*** thework1 has quit IRC14:57
*** sreichar has joined #rdo14:57
*** aditya_ra has quit IRC14:57
*** cylopez has joined #rdo14:58
*** thework1 has joined #rdo15:00
*** cylopez has quit IRC15:00
*** ihrachys has joined #rdo15:01
*** Son_Goku has joined #rdo15:02
*** oidgar has quit IRC15:03
*** mvk has quit IRC15:03
*** scorcoran_afk is now known as scorcoran_food15:03
*** eharney has joined #rdo15:03
*** janki has quit IRC15:07
*** udesale has quit IRC15:07
*** agurenko has quit IRC15:08
*** agurenko has joined #rdo15:09
*** ohamada has joined #rdo15:09
*** dciabrin has quit IRC15:09
*** dciabrin has joined #rdo15:09
*** Son_Goku has quit IRC15:10
*** jprovazn has quit IRC15:10
*** rlandy|rover is now known as rlandy|rover|mtg15:15
*** ohamada has quit IRC15:16
*** agurenko has quit IRC15:17
*** agurenko has joined #rdo15:18
*** mvk has joined #rdo15:19
*** ykarel is now known as ykarel|away15:19
*** rbartal has quit IRC15:19
*** jhershbe has joined #rdo15:21
*** ykarel|away has quit IRC15:24
rdogerritMerged openstack/oslo-privsep-distgit pike-rdo: Update to 1.22.1  https://review.rdoproject.org/r/1061515:26
rdogerritMerged openstack/taskflow-distgit pike-rdo: Update to 2.14.1  https://review.rdoproject.org/r/1061715:26
rdogerritMerged openstack/oslo-reports-distgit pike-rdo: Update to 1.22.1  https://review.rdoproject.org/r/1061415:29
rdogerritMerged openstack/stevedore-distgit pike-rdo: Update to 1.25.1  https://review.rdoproject.org/r/1061615:29
rdogerritMerged openstack/oslo-service-distgit pike-rdo: Update to 1.25.1  https://review.rdoproject.org/r/1061315:29
rdogerritMerged openstack/oslo-versionedobjects-distgit pike-rdo: Update to 1.26.1  https://review.rdoproject.org/r/1061215:29
rdogerritMerged openstack/oslo-policy-distgit pike-rdo: Update to 1.25.2  https://review.rdoproject.org/r/1061115:29
rdogerritMerged openstack/oslo-context-distgit pike-rdo: Update to 2.17.1  https://review.rdoproject.org/r/1060915:29
rdogerritMerged openstack/oslo-log-distgit pike-rdo: Update to 3.30.1  https://review.rdoproject.org/r/1061015:29
rdogerritMerged openstack/oslo-db-distgit pike-rdo: Update to 4.25.1  https://review.rdoproject.org/r/1060815:29
rdogerritMerged openstack/oslo-cache-distgit pike-rdo: Update to 1.25.1  https://review.rdoproject.org/r/1060615:29
*** alphacc has quit IRC15:30
rdogerritMerged openstack/oslo-serialization-distgit pike-rdo: Update to 2.20.1  https://review.rdoproject.org/r/1060715:31
*** dciabrin has quit IRC15:32
*** tvignaud has quit IRC15:32
rdogerritMerged openstack/oslotest-distgit pike-rdo: Update to 2.17.1  https://review.rdoproject.org/r/1060515:34
*** jatanmalde has quit IRC15:37
*** nyechiel has joined #rdo15:41
*** scorcoran_food is now known as scorcoran15:44
*** haleyb has quit IRC15:47
*** haleyb_ is now known as haleyb15:47
*** rlandy|rover|mtg is now known as rlandy|rover15:48
*** tvignaud has joined #rdo15:49
*** mvk has quit IRC15:54
*** dpawar has quit IRC15:55
number80btw, tomorrow I intend to raise test days topic during our weekly meeting15:57
number80https://etherpad.openstack.org/p/RDO-Meeting15:57
number80We're following a bad trend in participation, so I'd like us to act and improve the situation by the next test day15:58
number80so if you have any ideas, please contribute to the discussion ^15:58
number80@all ^15:58
*** nyechiel has quit IRC15:59
*** julim_ has joined #rdo15:59
*** jistr is now known as jistr|mtg15:59
*** julim has quit IRC16:01
*** julim_ has quit IRC16:02
*** egonzalez has quit IRC16:02
*** julim has joined #rdo16:03
*** links has joined #rdo16:03
*** aufi has quit IRC16:03
*** kgiusti has joined #rdo16:06
*** janki has joined #rdo16:06
*** BarBQ has joined #rdo16:07
*** mvk has joined #rdo16:07
*** links has quit IRC16:12
*** oidgar has joined #rdo16:12
*** dsneddon has joined #rdo16:15
*** gvrangan has joined #rdo16:16
*** jistr|mtg is now known as jistr16:16
*** clecomte has quit IRC16:16
*** toure is now known as toure_biab16:16
*** apetrich has quit IRC16:18
*** rhallisey has quit IRC16:18
*** aufi has joined #rdo16:19
rdogerritAlfredo Moralejo proposed gating_scripts master: Add tools to automate building dependencies from fedora builds  https://review.rdoproject.org/r/1056916:22
amoralejnumber80, apevec i'd appreciate youre reviews on ^16:22
dmsimardamoralej: I know elasticsearch/kibana has been out for the past couple days but let's try to keep an eye on slave disconnections16:22
dmsimardwe reduced the amount of max nodes yesterday16:23
amoralejdmsimard, ok16:23
amoraleji saved a query and posted url in trello card16:23
amoralejdmsimard, so root cause is jenkins overloaded?16:23
dmsimardamoralej: it's just an intuition16:24
amoralejo16:24
amoralejok16:24
dmsimardamoralej: we're asking jenkins to keep a connection opened on 100 slaves and stream the console16:24
dmsimardwith the I/O performance that we have16:24
dmsimardWe also did not have slave disconnections before raising tripleo's max servers from 4016:24
dmsimardI don't know the exact root cause but it's surely related to concurrency16:25
amoralejmmm, that's right16:25
*** josecastroleon has quit IRC16:29
*** abregman has quit IRC16:34
*** toure_biab is now known as toure16:36
*** jhershbe has quit IRC16:38
*** ccamacho has quit IRC16:42
*** rhallisey has joined #rdo16:42
*** anilvenkata has joined #rdo16:42
*** aufi has quit IRC16:42
*** fragatina has quit IRC16:43
*** paragan has joined #rdo16:43
*** iranzo has quit IRC16:44
*** fragatina has joined #rdo16:44
*** fragatina has quit IRC16:46
*** fragatina has joined #rdo16:46
*** yselkowitz has joined #rdo16:47
number80amoralej: I was looking at it, I need more time though but so far, positive16:47
amoralejok16:47
amoralejthx16:47
* number80 find it hard to test gating, not sure how to do it properly without staging16:48
number80*plaftorm16:48
amoralejnumber80, i'm testing those scripts easily in a centos7 server16:48
number80Yeah, I do that into VMs but it's not in the exact same conditions16:49
amoralejjust needed to export WORKSPACE=$HOME16:49
amoralejyeah16:49
amoralejbut yes, it's hard to test the full workflow16:49
number80That's why I'm slow on such reviews (unless they're trivial enough not to worry about side effects) :)16:49
amoralejyeah, no problem, take your time16:50
amoraleji just wanted to bring some attention, as i really want folks to validate my approach16:50
*** fragatina has quit IRC16:50
amoralejand i'm reusing some of your functions16:51
*** fragatina has joined #rdo16:51
number80Well, my biggest issue is not related to the change itself, it's hard to handle deps rebuild, bootstrapping, centos/fedora differences16:51
*** zenpwner has quit IRC16:51
number80AFAIK, that's the most sensible path you're taking16:51
amoralejyes16:52
amoralejnumber80, share your concernts in the review anyway, i may be missing something of the big picturee16:52
number80ack16:52
*** lkuchlan has quit IRC16:53
amoraleji want to start automating what is pure fedora rebuilds16:53
amoralejthen move to rdo-common16:53
amoralejwhich is harder16:53
amoralejand report status in RDO of new dependencies added to upstream requirements projects16:53
amoralejto raise visibility of issues when adding new ones16:54
*** agurenko has quit IRC16:54
*** dtantsur is now known as dtantsur|afk16:55
*** ragiman has quit IRC16:55
*** fragatina has quit IRC16:55
number80*nods*16:57
*** kellyp has joined #rdo16:57
*** openstackstatus has quit IRC16:58
*** openstackstatus has joined #rdo17:00
*** ChanServ sets mode: +v openstackstatus17:00
rdogerrithguemar proposed openstack/osops-tools-monitoring-oschecks-distgit pike-rdo: Initial Pike release (62160d10683023c8c9d96f616223d8def88b870d)  https://review.rdoproject.org/r/1019617:02
number80one more component in that package name and steam will come out of my ears17:03
*** ratailor has joined #rdo17:04
*** tumble has quit IRC17:07
*** artom_ has joined #rdo17:08
*** tesseract has quit IRC17:09
*** josecastroleon has joined #rdo17:10
*** ratailor has quit IRC17:10
sfbenderMatthieu Huin created rdopkg master: Include feature scenarios in documentation  https://softwarefactory-project.io/r/1030817:12
*** artom has quit IRC17:13
*** lucasagomes is now known as lucas-afk17:15
rdogerrithguemar proposed openstack/osops-tools-monitoring-oschecks-distgit pike-rdo: Initial Pike release (62160d10683023c8c9d96f616223d8def88b870d)  https://review.rdoproject.org/r/1019617:17
*** jackNemrod has joined #rdo17:21
*** paragan has quit IRC17:21
*** janki has quit IRC17:22
*** hseipp has quit IRC17:23
rdogerritMerged openstack/neutron-distgit pike-rdo: Destroy patch ports only if canary flow is not present  https://review.rdoproject.org/r/1048117:24
rdogerrithguemar proposed openstack/osops-tools-monitoring-oschecks-distgit pike-rdo: Initial Pike release (1a3ca01844799d2126e17783fa8bee37ad1b86cb)  https://review.rdoproject.org/r/1019617:25
*** gkadam has quit IRC17:26
*** d0ugal has quit IRC17:30
*** fragatina has joined #rdo17:32
*** absubram has joined #rdo17:34
*** smeyer has quit IRC17:34
*** artom_ is now known as artom17:36
*** flepied__ has joined #rdo17:37
*** flepied_ has quit IRC17:39
*** gvrangan has quit IRC17:39
*** not_rdobot has joined #rdo17:43
*** jhershbe has joined #rdo17:45
rdogerritMerged openstack/osops-tools-monitoring-oschecks-distgit pike-rdo: Initial Pike release (1a3ca01844799d2126e17783fa8bee37ad1b86cb)  https://review.rdoproject.org/r/1019617:47
*** pcaruana has quit IRC17:48
*** d0ugal has joined #rdo17:50
*** rhallisey has quit IRC17:53
*** not_rdobot has quit IRC17:55
*** derekh has quit IRC17:58
*** agurenko has joined #rdo17:58
*** as0bu has quit IRC17:59
*** flepied_ has joined #rdo18:00
*** flepied__ has quit IRC18:02
*** trown is now known as trown|lunch18:03
*** jistr is now known as jistr|off|trng18:04
*** jlibosva has quit IRC18:05
*** mvk has quit IRC18:09
*** MrAbaddon has quit IRC18:09
*** dciabrin has joined #rdo18:14
*** amoralej is now known as amoralej|off18:16
*** jpena is now known as jpena|off18:20
*** rhallisey has joined #rdo18:21
*** hewbrocca is now known as hewbrocca_afk18:22
*** jpich has quit IRC18:23
*** ccamacho has joined #rdo18:24
*** shardy has quit IRC18:26
*** tosky has quit IRC18:30
*** ccamacho has quit IRC18:33
*** pkovar has quit IRC18:33
*** ccamacho has joined #rdo18:36
*** jhershbe has quit IRC18:38
*** sbulage|afk has joined #rdo18:42
*** imacdonn has quit IRC18:45
*** imacdonn has joined #rdo18:45
*** pchavva has quit IRC18:49
*** mcornea has quit IRC18:51
*** mvk has joined #rdo18:54
*** paramite has quit IRC18:59
*** MrAbaddon has joined #rdo19:00
*** oidgar has quit IRC19:02
*** racedo has quit IRC19:03
*** trown|lunch is now known as trown19:03
*** racedo has joined #rdo19:04
*** racedo has quit IRC19:08
*** apetrich has joined #rdo19:08
*** tosky has joined #rdo19:11
*** sbulage|afk has quit IRC19:19
*** gfidente is now known as gfidente|afk19:20
*** julim_ has joined #rdo19:23
*** julim has quit IRC19:25
*** ccamacho has quit IRC19:28
*** gszasz has quit IRC19:29
*** pchavva has joined #rdo19:29
*** josecastroleon has quit IRC19:48
*** jtomasek has quit IRC19:51
dmsimardsshnaidm: is there a launchpad/trello for dns issues in tripleo ?19:56
*** suresh12 has joined #rdo20:00
*** suresh12 has quit IRC20:05
*** pchavva has quit IRC20:07
*** morazi has quit IRC20:10
*** Goneri has quit IRC20:10
sshnaidmdmsimard, https://bugs.launchpad.net/tripleo/+bug/173093120:24
openstackLaunchpad bug 1730931 in tripleo "Github.com is used in TripleO CI for cloning distgit repos" [Critical,Triaged]20:24
*** apetrich has quit IRC20:30
*** kellyp has quit IRC20:36
*** apetrich has joined #rdo20:38
*** suresh12 has joined #rdo20:38
*** suresh12 has quit IRC20:40
*** rumors_ has quit IRC20:42
*** kellyp has joined #rdo20:57
*** jackNemrod has quit IRC20:57
*** suresh12 has joined #rdo21:00
*** Goneri has joined #rdo21:03
*** kellyp has quit IRC21:05
*** gfidente|afk has quit IRC21:07
*** kellyp has joined #rdo21:10
*** kgiusti has left #rdo21:15
*** jbrooks has quit IRC21:26
*** julim_ has quit IRC21:30
*** cylopez has joined #rdo21:35
*** eharney has quit IRC21:37
*** dprince has quit IRC21:37
*** thework1 is now known as theaway121:46
*** not_rdobot has joined #rdo21:46
*** jkilpatr_ has quit IRC21:47
*** threestrands has joined #rdo21:48
*** rcernin has joined #rdo21:50
rdogerritrdo-trunk created openstack/horizon-distgit pike-rdo: Update to 12.0.1  https://review.rdoproject.org/r/1061821:51
*** rbowen has quit IRC21:52
*** not_rdobot has quit IRC21:55
*** trown is now known as trown|outtypewww21:58
*** mvk has quit IRC21:59
*** rlandy|rover is now known as rlandy|rover|bbl22:01
*** jkilpatr has joined #rdo22:05
*** scorcoran is now known as scorcoran_biab22:09
*** mvk has joined #rdo22:15
*** raissa has quit IRC22:21
*** thrash is now known as thrash|bbl22:22
*** ntpttr_laptop has joined #rdo22:27
ntpttr_laptophi all, I've just had changes to rdo.yml merged here https://review.rdoproject.org/r/#/c/10490/, is my next step to wait for a new distgit project repo to be able to push the spec file up to?22:28
*** florianf has quit IRC22:36
*** suresh12 has quit IRC22:37
*** suresh12 has joined #rdo22:42
apevecntpttr_laptop, project was created, see https://review.rdoproject.org/r/#/q/topic:add-rsd-lib22:46
apevecconfig change is created automatically in post pipeline when rdoinfo addition is merged22:46
apevechttps://review.rdoproject.org/r/#/admin/projects/?filter=rsd22:47
ntpttr_laptopthank you apevec22:47
apevecrsd-lib-distgit and rsdclient-distgit are there, you can send initial spec review there22:47
*** wolverineav has quit IRC22:49
*** dneary has quit IRC22:50
*** cylopez has quit IRC22:51
*** scorcoran_biab is now known as scorcoran22:51
*** eharney has joined #rdo22:52
*** apevec has quit IRC22:54
*** scorcoran is now known as scorcoran_afk23:02
*** jmelvin has quit IRC23:05
*** tshefi has quit IRC23:08
*** bnemec has quit IRC23:09
*** bnemec has joined #rdo23:12
*** Sandy619 has joined #rdo23:28
*** Sandy619 has quit IRC23:30
rdogerritUser ntpttr created openstack/rsd-lib-distgit rpm-master: Initial import of spec file  https://review.rdoproject.org/r/1061923:31
rdogerritUser ntpttr created openstack/rsdclient-distgit rpm-master: Initial import of spec file  https://review.rdoproject.org/r/1062023:33
*** gildub has joined #rdo23:33
*** gildub has quit IRC23:34
*** sshnaidm is now known as sshnaidm|afk23:38
*** rhallisey has quit IRC23:40
*** ihrachys has quit IRC23:44
*** ihrachys has joined #rdo23:44
*** jayg is now known as jayg|g0n323:48
*** ntpttr_laptop has quit IRC23:56
*** d0ugal has quit IRC23:58
*** d0ugal has joined #rdo23:59

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!