| *** shardy_afk has quit IRC | 00:01 | |
| *** dyasny has quit IRC | 00:01 | |
| dmsimard | rbowen, number80, EmilienM, pabelanger: things have started recovering | 00:03 |
|---|---|---|
| dmsimard | looking at the damage | 00:03 |
| pabelanger | likely git fsck on git repos | 00:08 |
| *** shardy_afk has joined #rdo | 00:09 | |
| *** gildub has joined #rdo | 00:15 | |
| dmsimard | pabelanger: vms seems to be okay | 00:16 |
| dmsimard | I think most of the hit was on ceph and control nodes | 00:16 |
| dmsimard | uptime wasn't affect | 00:16 |
| dmsimard | affected* | 00:16 |
| dmsimard | filesystems aren't even read only | 00:16 |
| dmsimard | still seeing some ugly kernel traces and I'm safely rebooting nodes | 00:16 |
| *** rdobot has joined #rdo | 00:25 | |
| *** ChanServ sets mode: +v rdobot | 00:25 | |
| tristanC | number80: fbo_: gerrit version in review.rdo didn't change recently... | 00:27 |
| tristanC | the tricky part of doing an upgrade (so far) is that they changed the build system and I couldn't make it work through rpmbuild yet | 00:28 |
| *** dmellado_ has joined #rdo | 00:32 | |
| *** shardy_afk has quit IRC | 00:33 | |
| *** stevebaker has joined #rdo | 00:33 | |
| *** derekh has quit IRC | 00:35 | |
| *** limao has joined #rdo | 00:35 | |
| *** eaguilar has quit IRC | 00:38 | |
| *** eaguilar has joined #rdo | 00:40 | |
| *** shardy_afk has joined #rdo | 00:41 | |
| *** dtrainor has quit IRC | 00:44 | |
| *** liujiong has joined #rdo | 00:45 | |
| *** dmacpher has joined #rdo | 01:03 | |
| *** rlandy|PTO is now known as rlandy | 01:07 | |
| *** hamzy has joined #rdo | 01:08 | |
| *** suresh12 has quit IRC | 01:15 | |
| *** suresh12 has joined #rdo | 01:15 | |
| *** suresh12 has quit IRC | 01:17 | |
| *** rlandy has quit IRC | 01:19 | |
| *** rdobot has quit IRC | 01:23 | |
| *** suresh12 has joined #rdo | 01:26 | |
| *** suresh12 has quit IRC | 01:30 | |
| *** rdobot has joined #rdo | 01:31 | |
| *** ChanServ sets mode: +v rdobot | 01:31 | |
| *** paragan has quit IRC | 01:35 | |
| *** fzdarsky_ has joined #rdo | 01:50 | |
| *** fzdarsky__ has quit IRC | 01:52 | |
| *** thrash is now known as thrash|g0ne | 01:58 | |
| *** eaguilar has quit IRC | 02:07 | |
| *** eaguilar has joined #rdo | 02:09 | |
| *** jlabarre has joined #rdo | 02:14 | |
| *** eaguilar has quit IRC | 02:17 | |
| *** imcleod has quit IRC | 02:23 | |
| *** Son_Goku has joined #rdo | 02:25 | |
| *** imcleod has joined #rdo | 02:29 | |
| *** eaguilar has joined #rdo | 02:32 | |
| *** jlabarre has quit IRC | 02:42 | |
| *** rdogerrit has joined #rdo | 02:43 | |
| rdogerrit | David Moreau Simard created config master: Bump min-ready of upstream-centos-7-2-node from 1 to 5 https://review.rdoproject.org/r/9384 | 02:43 |
| dmsimard | rbowen: signing off for the night, we're all green | 02:51 |
| *** eaguilar has quit IRC | 02:51 | |
| *** imcleod has quit IRC | 02:52 | |
| *** jkilpatr has quit IRC | 02:59 | |
| *** paragan has joined #rdo | 03:01 | |
| *** myoung is now known as myoung|afk | 03:04 | |
| *** nmagnezi has quit IRC | 03:04 | |
| *** Guest24_ has joined #rdo | 03:28 | |
| *** Guest24_ has quit IRC | 03:29 | |
| *** Son_Goku has quit IRC | 03:35 | |
| *** udesale has joined #rdo | 03:41 | |
| *** dmacpher has quit IRC | 03:46 | |
| *** hrw has quit IRC | 03:46 | |
| *** hrw has joined #rdo | 03:47 | |
| *** gkadam__ has joined #rdo | 03:50 | |
| *** dbecker has quit IRC | 03:54 | |
| *** suresh12 has joined #rdo | 03:56 | |
| *** owalsh_ has joined #rdo | 03:57 | |
| *** jhershbe_ has joined #rdo | 03:58 | |
| *** dmacpher has joined #rdo | 03:59 | |
| *** owalsh has quit IRC | 04:00 | |
| *** liujiong_lj has joined #rdo | 04:00 | |
| *** liujiong has quit IRC | 04:00 | |
| *** cshastri has joined #rdo | 04:04 | |
| *** shreshtha has joined #rdo | 04:06 | |
| *** jhershbe_ has quit IRC | 04:06 | |
| *** sasha2 has quit IRC | 04:06 | |
| *** dbecker has joined #rdo | 04:07 | |
| *** maeca has quit IRC | 04:08 | |
| rdogerrit | rdo-trunk created openstack/ironic-python-agent-distgit rpm-master: openstack-ironic-python-agent: failed to build c90b150 https://review.rdoproject.org/r/9385 | 04:13 |
| *** links has joined #rdo | 04:19 | |
| *** rovyas has joined #rdo | 04:22 | |
| *** anilvenkata has joined #rdo | 04:29 | |
| *** sfbender has joined #rdo | 04:37 | |
| sfbender | Jon Schlueter created rdopkg master: Add newversion.feature scenario using --bug https://softwarefactory-project.io/r/9618 | 04:37 |
| *** coolsvap has joined #rdo | 04:37 | |
| sfbender | Jon Schlueter created rdopkg master: make spec file Then assert more descriptive https://softwarefactory-project.io/r/9619 | 04:38 |
| sfbender | Jon Schlueter created rdopkg master: distgitmagic run guard for default log_run https://softwarefactory-project.io/r/9620 | 04:39 |
| *** paragan has quit IRC | 04:43 | |
| rdogerrit | rdo-trunk created openstack/patrole-distgit rpm-master: python-patrole: failed to build 6f4bc67 https://review.rdoproject.org/r/9386 | 04:50 |
| *** sbulage has joined #rdo | 04:50 | |
| *** ratailor has joined #rdo | 04:58 | |
| *** sshnaidm|afk is now known as sshnaidm|off | 05:02 | |
| *** paragan has joined #rdo | 05:05 | |
| *** rovyas has quit IRC | 05:06 | |
| *** rejy has joined #rdo | 05:09 | |
| rdogerrit | Steve Baker proposed rdo-infra/ansible-role-rdo-kolla-build master: Generate tripleo_default_images with 'prepare' https://review.rdoproject.org/r/9328 | 05:09 |
| rdogerrit | Steve Baker proposed rdo-infra/ansible-role-rdo-kolla-build master: Use kolla-build.conf to specify profile and push https://review.rdoproject.org/r/9327 | 05:09 |
| *** paragan has quit IRC | 05:15 | |
| *** rovyas has joined #rdo | 05:19 | |
| *** chlong__ has quit IRC | 05:20 | |
| *** lebauce has joined #rdo | 05:22 | |
| *** chlong__ has joined #rdo | 05:27 | |
| *** dpawar has joined #rdo | 05:33 | |
| *** janki has joined #rdo | 05:33 | |
| *** jackNemrod has joined #rdo | 05:34 | |
| *** paramite has joined #rdo | 05:39 | |
| *** jackNemrod has quit IRC | 05:41 | |
| *** ykarel has joined #rdo | 05:46 | |
| *** suresh12 has quit IRC | 05:46 | |
| *** suresh12 has joined #rdo | 05:47 | |
| *** sbonazzo has joined #rdo | 05:48 | |
| *** iranzo has joined #rdo | 05:53 | |
| *** Amita has joined #rdo | 05:55 | |
| *** Amita has quit IRC | 05:57 | |
| *** Amita has joined #rdo | 05:58 | |
| *** shreshtha has quit IRC | 06:09 | |
| *** jackNemrod has joined #rdo | 06:10 | |
| *** egallen has joined #rdo | 06:12 | |
| *** egallen has left #rdo | 06:12 | |
| *** suresh12 has quit IRC | 06:15 | |
| *** gfidente has joined #rdo | 06:17 | |
| *** DV has quit IRC | 06:20 | |
| *** shreshtha has joined #rdo | 06:20 | |
| *** jackNemrod has quit IRC | 06:21 | |
| *** jprovazn has joined #rdo | 06:21 | |
| *** paragan has joined #rdo | 06:28 | |
| *** emalikov has joined #rdo | 06:31 | |
| *** rcernin has joined #rdo | 06:32 | |
| *** sbulage has quit IRC | 06:33 | |
| *** jackNemrod has joined #rdo | 06:36 | |
| *** pgadiya has joined #rdo | 06:42 | |
| *** gszasz has joined #rdo | 06:47 | |
| *** larizmen has joined #rdo | 06:48 | |
| *** fbo_ is now known as fbo | 06:48 | |
| *** nyechiel has joined #rdo | 06:50 | |
| *** pgadiya has quit IRC | 07:00 | |
| *** limao has quit IRC | 07:03 | |
| *** noslzzp has quit IRC | 07:03 | |
| *** limao has joined #rdo | 07:04 | |
| *** noslzzp has joined #rdo | 07:07 | |
| *** florianf has joined #rdo | 07:08 | |
| *** aufi has joined #rdo | 07:09 | |
| *** janki is now known as janki|lunch | 07:10 | |
| *** chlong__ has quit IRC | 07:16 | |
| *** jpich has joined #rdo | 07:20 | |
| *** sbulage has joined #rdo | 07:21 | |
| *** gildub has quit IRC | 07:24 | |
| *** tesseract has joined #rdo | 07:31 | |
| *** milan has joined #rdo | 07:33 | |
| *** jpena|off is now known as jpena | 07:36 | |
| *** chlong has joined #rdo | 07:36 | |
| jpena | good morning | 07:37 |
| *** emalikov has quit IRC | 07:41 | |
| *** chlong has quit IRC | 07:45 | |
| *** cylopez has joined #rdo | 07:49 | |
| *** jackNemrod_ has joined #rdo | 07:49 | |
| *** jackNemrod has quit IRC | 07:50 | |
| *** jackNemrod_ is now known as jackNemrod | 07:50 | |
| *** cylopez1 has joined #rdo | 07:50 | |
| *** egonzalez has joined #rdo | 07:50 | |
| *** emalikov has joined #rdo | 07:50 | |
| *** racedo has joined #rdo | 07:53 | |
| *** cylopez has quit IRC | 07:54 | |
| *** jlibosva has joined #rdo | 07:57 | |
| *** karimb has joined #rdo | 07:57 | |
| *** shardy_afk is now known as shardy | 08:07 | |
| *** ykarel is now known as ykarel|lunch | 08:08 | |
| *** anshul has joined #rdo | 08:12 | |
| *** vnogin has joined #rdo | 08:18 | |
| *** owalsh_ is now known as owalsh | 08:19 | |
| *** mcornea has joined #rdo | 08:22 | |
| *** chlong has joined #rdo | 08:25 | |
| jpena | number80: we need python-stestr in the dependency repo (I see it failed to build in CBS) for os-testr | 08:29 |
| number80 | yeah, yesterday we had infra down so I couldn't merge anything | 08:32 |
| number80 | I can tag it manually as a workaround though | 08:33 |
| *** snarwade has joined #rdo | 08:33 | |
| rdogerrit | hguemar proposed rdoinfo master: Added python-stestr and python-subunit2sql to common-testing https://review.rdoproject.org/r/8031 | 08:34 |
| *** dtantsur|afk is now known as dtantsur | 08:34 | |
| *** chlong has quit IRC | 08:35 | |
| *** MrAbaddon has quit IRC | 08:36 | |
| *** chlong has joined #rdo | 08:51 | |
| *** nyechiel has quit IRC | 08:52 | |
| *** chlong_ has joined #rdo | 08:54 | |
| *** chlong has quit IRC | 08:58 | |
| *** dmacpher has quit IRC | 08:58 | |
| *** derekh has joined #rdo | 08:58 | |
| *** hewbrocca_afk is now known as hewbrocca | 09:02 | |
| *** chlong_ has quit IRC | 09:04 | |
| *** dmacpher has joined #rdo | 09:10 | |
| *** MrAbaddon has joined #rdo | 09:11 | |
| *** vnogin has quit IRC | 09:13 | |
| *** dmellado_ has quit IRC | 09:18 | |
| *** dmellado_ has joined #rdo | 09:20 | |
| *** chlong has joined #rdo | 09:21 | |
| *** Caterpillar has joined #rdo | 09:23 | |
| *** vnogin has joined #rdo | 09:23 | |
| *** paragan has quit IRC | 09:23 | |
| *** gcerami has joined #rdo | 09:23 | |
| *** ykarel|lunch is now known as ykarel | 09:23 | |
| *** chlong has quit IRC | 09:24 | |
| *** chlong has joined #rdo | 09:25 | |
| *** limao has quit IRC | 09:26 | |
| *** limao has joined #rdo | 09:26 | |
| *** iranzo1 has joined #rdo | 09:27 | |
| *** iranzo has quit IRC | 09:27 | |
| *** dmellado_ has quit IRC | 09:29 | |
| *** shardy is now known as shardy_mtg | 09:29 | |
| *** dmellado_ has joined #rdo | 09:30 | |
| *** limao has quit IRC | 09:31 | |
| *** dmellado_ has quit IRC | 09:33 | |
| *** _lebauce_ has joined #rdo | 09:34 | |
| *** iranzo1 has quit IRC | 09:34 | |
| *** iranzo has joined #rdo | 09:34 | |
| *** sbulage has quit IRC | 09:36 | |
| *** pkovar has joined #rdo | 09:37 | |
| *** iranzo has quit IRC | 09:37 | |
| *** iranzo has joined #rdo | 09:37 | |
| *** iranzo has quit IRC | 09:37 | |
| *** iranzo has joined #rdo | 09:37 | |
| *** dmellado_ has joined #rdo | 09:39 | |
| *** _lebauce_ has quit IRC | 09:39 | |
| *** sbulage has joined #rdo | 09:39 | |
| *** dmellado_ has quit IRC | 09:39 | |
| *** _lebauce_ has joined #rdo | 09:39 | |
| *** dmellado_ has joined #rdo | 09:41 | |
| *** dmellado_ has quit IRC | 09:42 | |
| rdogerrit | hguemar proposed rdoinfo master: Added python-stestr and python-subunit2sql to common-testing https://review.rdoproject.org/r/8031 | 09:42 |
| *** dmellado_ has joined #rdo | 09:43 | |
| *** jprovazn has quit IRC | 09:44 | |
| *** abregman has quit IRC | 09:46 | |
| *** dmellado_ has quit IRC | 09:47 | |
| *** dmellado_ has joined #rdo | 09:47 | |
| gcerami | adarazs: how's going ? | 09:48 |
| adarazs | gcerami: good, getting close to a working promotion script, the pipelines seem to be working otherwise and reporting \o/ :) | 09:49 |
| *** cylopez1 has quit IRC | 09:50 | |
| hewbrocca | rdo-cloud is back up and running, yes? | 09:50 |
| *** dmellado_ is now known as dmellado | 09:51 | |
| rdogerrit | rdo-trunk created openstack/glare-distgit rpm-master: openstack-glare: failed to build df81a6d https://review.rdoproject.org/r/9387 | 09:51 |
| adarazs | hewbrocca, gcerami: oh yeah, about that... there's something with heat (probably) on rdocloud. we're failing a lot of OVB jobs that try to bring up stacks for our jobs. rlandy is tracking it and looking at the OVB jobs they are still in the red. | 09:52 |
| *** gfidente is now known as gfidente|afk | 09:52 | |
| *** scorcoran has joined #rdo | 09:53 | |
| *** chlong has quit IRC | 09:55 | |
| *** rovyas has quit IRC | 09:56 | |
| *** sbonazzo has quit IRC | 09:57 | |
| hewbrocca | You know the entire storage cluster was inadvertently powered off yesterday, right? | 09:57 |
| adarazs | huh? no. | 09:58 |
| hewbrocca | yeah | 09:58 |
| hewbrocca | It has been powered back on, but if you're having problems with jobs, that is probably why | 09:58 |
| adarazs | they should tape down those powerlines so that the cleaning lady won't trip. :) | 09:58 |
| *** dbrandon has quit IRC | 10:00 | |
| *** scorcoran is now known as scorcoran_biab | 10:01 | |
| *** dbrandon has joined #rdo | 10:02 | |
| *** vnogin has quit IRC | 10:03 | |
| *** vnogin has joined #rdo | 10:05 | |
| *** milan has quit IRC | 10:06 | |
| adarazs | shadower: ^ FYI | 10:06 |
| adarazs | looks like something is still not right with it. | 10:06 |
| *** thrash|g0ne is now known as thrash | 10:06 | |
| *** brault_ has joined #rdo | 10:09 | |
| *** liujiong_lj has quit IRC | 10:11 | |
| *** Caterpillar2 has joined #rdo | 10:11 | |
| *** brault has quit IRC | 10:12 | |
| *** Caterpillar has quit IRC | 10:12 | |
| *** noslzzp_ has joined #rdo | 10:12 | |
| *** vnogin has quit IRC | 10:12 | |
| *** rovyas has joined #rdo | 10:12 | |
| *** vnogin has joined #rdo | 10:13 | |
| *** noslzzp has quit IRC | 10:14 | |
| *** cylopez has joined #rdo | 10:14 | |
| *** karthiks has quit IRC | 10:15 | |
| *** rovyas has quit IRC | 10:19 | |
| *** wolverineav has joined #rdo | 10:23 | |
| *** Son_Goku has joined #rdo | 10:24 | |
| *** Son_Goku has quit IRC | 10:25 | |
| *** cylopez has quit IRC | 10:29 | |
| *** tosky has joined #rdo | 10:29 | |
| *** dpawar has quit IRC | 10:33 | |
| *** scorcoran_biab is now known as scorcoran | 10:34 | |
| *** athomas has quit IRC | 10:35 | |
| *** udesale has quit IRC | 10:35 | |
| *** paragan has joined #rdo | 10:37 | |
| number80 | adarazs: well, it was not the cleaning lady ... | 10:37 |
| *** apevec has joined #rdo | 10:38 | |
| *** rovyas has joined #rdo | 10:39 | |
| *** eaguilar has joined #rdo | 10:39 | |
| adarazs | number80: is the poor system back in some way? I see our "openstack-periodic-4hr" pipeline being stuck for more than 6 hours now. | 10:41 |
| *** dtantsur is now known as dtantsur|bbl | 10:43 | |
| *** limao has joined #rdo | 10:44 | |
| apevec | adarazs, dmsimard mentioned there were stuck OVB Heat stacks, have you cleaned them up? | 10:45 |
| adarazs | I sure didn't. let me check. | 10:46 |
| apevec | if you need help with that, open ops ticket so dmanchad can have a look | 10:46 |
| apevec | adarazs, CCed you | 10:47 |
| -openstackstatus- NOTICE: Our CI systems experience a hickup, no new jobs are started. Please stay tuned and wait untils this resolved. | 10:48 | |
| *** sbulage has quit IRC | 10:48 | |
| *** limao_ has joined #rdo | 10:48 | |
| apevec | hewbrocca, adarazs, jpena - good that dc ops didn't start scrubbing disks ... | 10:49 |
| adarazs | hehe, yeah :) | 10:49 |
| *** limao has quit IRC | 10:49 | |
| adarazs | apevec: thanks for the CC | 10:49 |
| jpena | apevec: that would've been bad | 10:49 |
| apevec | jpena, or perfect opportunity to test disaster restore procedure! | 10:50 |
| adarazs | disaster... what? :) | 10:51 |
| *** gildub has joined #rdo | 10:51 | |
| apevec | jpena, ykarel - to fix poi, let's pull https://github.com/voxpupuli/puppet-rabbitmq/pull/621/commits/0b372609c79923d993bdbfa880849c0048b72e6c as a temp patch in puppet-rabbitmq RPM ? | 10:57 |
| *** jkilpatr has joined #rdo | 10:57 | |
| apevec | it's rdo promotion blocker on master and pike | 10:57 |
| jpena | that's the proposed fix for the idempotency issue, right? | 10:57 |
| ykarel | apevec, Ok | 10:58 |
| ykarel | jpena, yes https://github.com/voxpupuli/puppet-rabbitmq/issues/618 | 10:58 |
| adarazs | apevec: seems like the creds to our rdocloud accounts have changed, I will need to wait for rlandy. | 10:58 |
| apevec | jpena, dmanchad ^ was there restore of rdocloud keystone db from a backup? | 10:59 |
| jpena | ykarel: are you proposing the patch? If not, I can do it | 10:59 |
| ykarel | jpena, do it please | 10:59 |
| jpena | ack | 11:00 |
| dmanchad | apevec , jpena : no | 11:00 |
| adarazs | apevec: I think I just haven't logged in in a while and don't know the password, it's not a recent thing I can't access it. I never really did. | 11:00 |
| *** daidv has quit IRC | 11:00 | |
| dmanchad | galera was able to restore to the latest known status | 11:00 |
| apevec | adarazs, ah, then you need to ask for password reset | 11:00 |
| *** snarwade_ has joined #rdo | 11:00 | |
| adarazs | apevec: rlandy should be here in an hour and I ask her :) | 11:00 |
| apevec | ok | 11:01 |
| *** sbulage has joined #rdo | 11:01 | |
| *** nyechiel has joined #rdo | 11:01 | |
| *** snarwade has quit IRC | 11:03 | |
| *** ykarel is now known as ykarel|afk | 11:06 | |
| *** vnogin has quit IRC | 11:06 | |
| *** cylopez has joined #rdo | 11:07 | |
| *** dpawar has joined #rdo | 11:08 | |
| *** dtantsur|bbl has quit IRC | 11:09 | |
| *** dtantsur has joined #rdo | 11:09 | |
| *** dtantsur has quit IRC | 11:09 | |
| *** dtantsur has joined #rdo | 11:09 | |
| *** jbadiapa has quit IRC | 11:09 | |
| *** limao_ has quit IRC | 11:09 | |
| *** rovyas has quit IRC | 11:09 | |
| *** rcernin has quit IRC | 11:09 | |
| *** jbadiapa has joined #rdo | 11:09 | |
| *** rovyas has joined #rdo | 11:09 | |
| *** vnogin has joined #rdo | 11:10 | |
| *** eaguilar has quit IRC | 11:11 | |
| *** s1061123 has quit IRC | 11:13 | |
| *** s1061123 has joined #rdo | 11:13 | |
| *** Son_Goku has joined #rdo | 11:16 | |
| *** snarwade_ has quit IRC | 11:16 | |
| *** rhallisey has joined #rdo | 11:17 | |
| rdogerrit | Javier Peña created puppet/puppet-rabbitmq-distgit rpm-master: Add temporary patch to restore idempotency https://review.rdoproject.org/r/9388 | 11:18 |
| *** jpena is now known as jpena|lunch | 11:19 | |
| *** social has quit IRC | 11:22 | |
| *** pkovar has quit IRC | 11:22 | |
| *** nyechiel has quit IRC | 11:22 | |
| *** tesseract has quit IRC | 11:23 | |
| *** tesseract has joined #rdo | 11:26 | |
| *** jlabarre has joined #rdo | 11:26 | |
| number80 | I think we should drop patches branches, they are not used | 11:26 |
| * number80 will submit that topic to the next RDO meeting | 11:27 | |
| *** limao has joined #rdo | 11:28 | |
| apevec | number80, where we have them? | 11:28 |
| number80 | apevec: all disgit repo in review.rdoproject.org have mirrors of the sources, but nobody uses the open-reviews thing to keep track of patches | 11:29 |
| number80 | so we're just cluttering the server with unused repos | 11:30 |
| *** vnogin has quit IRC | 11:30 | |
| number80 | s/branches/repo | 11:30 |
| *** mvk has quit IRC | 11:31 | |
| number80 | And we have tooling around patches management, so next step would be removing those jobs and rdopkg support code | 11:33 |
| *** vnogin has joined #rdo | 11:33 | |
| apevec | number80, ah those, I thought there is some old *-patches branch | 11:34 |
| *** jprovazn has joined #rdo | 11:34 | |
| number80 | no, it | 11:34 |
| *** lebauce has quit IRC | 11:34 | |
| number80 | 's just old habit of calling them branches | 11:34 |
| apevec | number80, sometimes we do need a temp patch like now in puppet-rabbitmq | 11:34 |
| number80 | Yeah, that's not the issue | 11:34 |
| apevec | maybe we can create them on demand for such cases? | 11:34 |
| number80 | but we don't uses the repo at all | 11:35 |
| apevec | but yeah, keeping them in sync w/ upstream when we normally do not have patches is just waste of resources | 11:35 |
| number80 | like since newton | 11:35 |
| *** snarwade_ has joined #rdo | 11:35 | |
| apevec | I just asked jpena to push the review with the patch | 11:35 |
| apevec | it's easier to maintain as gerrit review | 11:36 |
| number80 | Well, there are also the EOL releases patches | 11:36 |
| apevec | anyway, let's have it as a topic | 11:36 |
| apevec | next Wed | 11:36 |
| * number80 has a full page in gerrit of them | 11:36 | |
| number80 | yep | 11:36 |
| number80 | Just want to raise the attention so people don't discover the topic next week | 11:37 |
| *** rcernin has joined #rdo | 11:37 | |
| *** lebauce has joined #rdo | 11:40 | |
| *** Amita has quit IRC | 11:41 | |
| *** flepied has joined #rdo | 11:45 | |
| *** shreshtha has quit IRC | 11:47 | |
| dmsimard | apevec, jpena: everything under control ? | 11:48 |
| apevec | dmsimard, adarazs had some trouble w/ ovb | 11:48 |
| apevec | ...as you suspected | 11:48 |
| apevec | since there are leftover heat stacks | 11:48 |
| *** eck` is now known as eck`gone | 11:49 | |
| adarazs | dmsimard: well, problem #1 is that the "openstack-periodic-4hr" pipeline is stuck for the last ~8 hours: https://review.rdoproject.org/zuul/ | 11:49 |
| dmsimard | There are a LOT of stacks that aren't deleted properly | 11:49 |
| dmsimard | There was one or two that were active | 11:49 |
| adarazs | not sure if that's a problem or a symptom of another problem. | 11:49 |
| dmsimard | The rest were there, but 'deleted', not sure what's up with that | 11:49 |
| dmsimard | adarazs: did someone check the 'infra' servers ? Broker etc ? | 11:50 |
| dmsimard | Ideally these would all be restarted | 11:50 |
| adarazs | dmsimard: I realized I don't have the correct password for our rdocloud users, so I couldn't check on it yet. waiting for rlandy or somebody who has them to look around. | 11:51 |
| dmsimard | Ceph and controllers were knocked out, it's a good idea to get VMs to fire on a new kvm process | 11:51 |
| dmsimard | adarazs: do you know if someone other than Sagi has access to the infra servers ? | 11:51 |
| *** jayg|g0n3 is now known as jayg | 11:52 | |
| adarazs | maybe trown|outtypewww? | 11:52 |
| dmsimard | I mean, I can hard reboot the servers but I'd rather not | 11:52 |
| adarazs | he should be in soon | 11:52 |
| dmsimard | I shutdown'd servers and started them manually last night. | 11:53 |
| dmsimard | Scary things in dmesg :) | 11:53 |
| dmsimard | Well, I did that for the RDO servers, not the tripleo ones. | 11:54 |
| dmsimard | afk for a bit | 11:55 |
| *** pkovar has joined #rdo | 11:55 | |
| *** dneary has joined #rdo | 11:55 | |
| *** limao has quit IRC | 11:55 | |
| *** jprovazn has quit IRC | 11:55 | |
| *** jprovazn has joined #rdo | 11:58 | |
| *** athomas has joined #rdo | 11:59 | |
| number80 | stestr is coming if you're looking at trunk builds | 12:01 |
| * number80 has to fix deps and redo the whole rdoinfo review | 12:01 | |
| *** rovyas has quit IRC | 12:03 | |
| *** rovyas has joined #rdo | 12:03 | |
| *** trown|outtypewww is now known as trown | 12:04 | |
| *** snarwade__ has joined #rdo | 12:07 | |
| *** gfidente|afk is now known as gfidente | 12:07 | |
| *** aditya_r has joined #rdo | 12:07 | |
| trown | adarazs: dmsimard, I dont think I have creds for anything but the mirror-server | 12:09 |
| trown | whicha appears fine | 12:09 |
| *** limao has joined #rdo | 12:09 | |
| *** snarwade_ has quit IRC | 12:09 | |
| apevec | jpena|lunch, dmsimard. trown - let's disable https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-pike-current-tripleo/ until we have https://review.rdoproject.org/r/9388 in current-tripleo | 12:11 |
| *** limao_ has joined #rdo | 12:12 | |
| *** limao has quit IRC | 12:13 | |
| *** tesseract has quit IRC | 12:14 | |
| *** tesseract has joined #rdo | 12:14 | |
| *** aditya_ra has joined #rdo | 12:14 | |
| *** rlandy has joined #rdo | 12:14 | |
| *** dprince has joined #rdo | 12:15 | |
| *** shardy_mtg is now known as shardy | 12:16 | |
| *** aditya_r has quit IRC | 12:16 | |
| *** maeca has joined #rdo | 12:18 | |
| *** maeca has quit IRC | 12:18 | |
| *** tonyb has quit IRC | 12:19 | |
| *** tonyb has joined #rdo | 12:20 | |
| *** jpena|lunch is now known as jpena | 12:20 | |
| jpena | apevec: cak for me | 12:21 |
| jpena | ack | 12:21 |
| *** scorcoran is now known as scorcoran_biab | 12:25 | |
| *** emalikov is now known as eliska-mtg | 12:25 | |
| *** imcleod has joined #rdo | 12:30 | |
| *** gkadam__ has quit IRC | 12:31 | |
| *** alee has quit IRC | 12:32 | |
| *** _bsurfer_ has joined #rdo | 12:35 | |
| *** jlabarre has quit IRC | 12:36 | |
| *** toure_biab is now known as toure | 12:36 | |
| *** dpawar has quit IRC | 12:38 | |
| *** dpawar has joined #rdo | 12:38 | |
| *** dpawar has quit IRC | 12:38 | |
| *** bsurfer has quit IRC | 12:38 | |
| *** dpawar has joined #rdo | 12:39 | |
| *** sasha2 has joined #rdo | 12:41 | |
| *** richm has joined #rdo | 12:42 | |
| *** ajo has joined #rdo | 12:42 | |
| *** dpawar has quit IRC | 12:43 | |
| *** aditya_ra has quit IRC | 12:44 | |
| *** aditya_r has joined #rdo | 12:44 | |
| *** jlabarre has joined #rdo | 12:45 | |
| *** snarwade__ has quit IRC | 12:46 | |
| *** eharney has quit IRC | 12:46 | |
| *** verdurin has quit IRC | 12:47 | |
| *** eck`gone is now known as eck` | 12:48 | |
| *** amuller has joined #rdo | 12:49 | |
| *** apevec has quit IRC | 12:50 | |
| *** imcleod has quit IRC | 12:50 | |
| *** dyasny has joined #rdo | 12:51 | |
| *** jmelvin has joined #rdo | 12:51 | |
| dmsimard | ok | 12:51 |
| *** larizmen has quit IRC | 12:52 | |
| *** pchavva has joined #rdo | 12:53 | |
| jpena | apevec dmsimard: I see nobody has disabled the project, I'll do it | 12:54 |
| *** flepied has quit IRC | 12:54 | |
| dmsimard | jpena: ok | 12:55 |
| *** ratailor has quit IRC | 12:59 | |
| *** scorcoran_biab is now known as scorcoran | 12:59 | |
| *** scorcoran is now known as scorcoran_mtg | 12:59 | |
| *** aditya_r has quit IRC | 12:59 | |
| *** cshastri has quit IRC | 13:00 | |
| *** abregman has joined #rdo | 13:00 | |
| *** jackNemrod_ has joined #rdo | 13:01 | |
| *** jackNemrod has quit IRC | 13:02 | |
| *** jackNemrod_ is now known as jackNemrod | 13:02 | |
| *** kgiusti has joined #rdo | 13:05 | |
| *** milan has joined #rdo | 13:07 | |
| *** flepied has joined #rdo | 13:08 | |
| *** vnogin has quit IRC | 13:10 | |
| *** Amita has joined #rdo | 13:11 | |
| *** imcleod has joined #rdo | 13:13 | |
| *** Goneri has joined #rdo | 13:17 | |
| *** vnogin has joined #rdo | 13:18 | |
| adarazs | number80, jpena: there's something wrong with https://softwarefactory-project.io/r/9300 -- which is needed for the promotion get change. | 13:24 |
| adarazs | can you try to merge it? | 13:24 |
| adarazs | looks like number80's vote is still -1 despite seeing +2 in the history. | 13:25 |
| *** gildub has quit IRC | 13:26 | |
| *** ajo has quit IRC | 13:27 | |
| number80 | adarazs: I don't remember why -1 here | 13:28 |
| adarazs | number80: you +2'd later, but it doesn't show up on the top. | 13:29 |
| number80 | Yeah | 13:29 |
| *** suresh12 has joined #rdo | 13:30 | |
| *** nyechiel has joined #rdo | 13:31 | |
| *** myoung|afk is now known as myoung | 13:31 | |
| *** Son_Goku has quit IRC | 13:31 | |
| *** links has quit IRC | 13:31 | |
| *** eharney has joined #rdo | 13:34 | |
| *** rovyas has quit IRC | 13:34 | |
| *** julim has quit IRC | 13:36 | |
| *** jprovazn has quit IRC | 13:36 | |
| *** alee has joined #rdo | 13:37 | |
| *** cylopez has quit IRC | 13:37 | |
| *** julim has joined #rdo | 13:38 | |
| *** vnogin has quit IRC | 13:41 | |
| *** _bsurfer_ has quit IRC | 13:42 | |
| *** jbadiapa has quit IRC | 13:43 | |
| *** jprovazn has joined #rdo | 13:45 | |
| *** limao_ has quit IRC | 13:45 | |
| rdogerrit | User vakwetu created rdoinfo master: Add queens tag to novajoin-tempest-plugin https://review.rdoproject.org/r/9389 | 13:45 |
| *** abregman is now known as abregman|afk | 13:47 | |
| -openstackstatus- NOTICE: nodepool issue related to bad images has been resolved, builds should be coming back online soon. Restarted gerrit due to reasons. Happy Friday. | 13:47 | |
| *** Son_Goku has joined #rdo | 13:48 | |
| *** vnogin has joined #rdo | 13:49 | |
| honza | dmsimard: is rdo-cloud still having issues? can't seem to create a new stack right now | 13:50 |
| *** rovyas has joined #rdo | 13:51 | |
| number80 | damn | 13:52 |
| *** shreshtha has joined #rdo | 13:52 | |
| *** jackNemrod has quit IRC | 13:53 | |
| dmsimard | number80, jschlueter, jpena: got a ~15-20 minutes for an informal chat about rpm-packaging/altarch ? | 13:56 |
| jpena | dmsimard: yep | 13:56 |
| number80 | dmsimard: gimme 5min | 13:56 |
| * jschlueter nods | 13:56 | |
| dmsimard | looks like there might be a reldel meeting though | 13:56 |
| dmsimard | jschlueter, number80: ping us when the meeting is done ? | 13:56 |
| * number80 is in the middle of debugging a CBS build | 13:57 | |
| *** paramite has quit IRC | 13:58 | |
| jschlueter | dmsimard: the people who would be on that call are interested in this topic ... so when you all are ready | 13:58 |
| dmsimard | ok let me fire something up | 13:59 |
| *** richm has quit IRC | 14:00 | |
| dmsimard | jschlueter, number80, jpena: 3800541668 | 14:00 |
| *** richm has joined #rdo | 14:00 | |
| jschlueter | dmsimard: ok joining then | 14:01 |
| dmsimard | number80: that was not a yubikey number, waiting on you to join :P | 14:02 |
| number80 | dmsimard: yeah | 14:02 |
| *** coolsvap has quit IRC | 14:03 | |
| *** bnemec is now known as beekneemech | 14:03 | |
| *** gregwork has joined #rdo | 14:04 | |
| *** jackNemrod has joined #rdo | 14:04 | |
| *** richm has quit IRC | 14:06 | |
| *** anshul has quit IRC | 14:08 | |
| *** richm has joined #rdo | 14:08 | |
| *** suresh12 has quit IRC | 14:08 | |
| sfbender | Merged DLRN master: Store promotion history in database https://softwarefactory-project.io/r/9300 | 14:09 |
| *** _bsurfer_ has joined #rdo | 14:09 | |
| *** scorcoran_mtg is now known as scorcoran | 14:13 | |
| sfbender | Merged DLRN master: Expose promotion info in DLRN API https://softwarefactory-project.io/r/9312 | 14:15 |
| *** ratailor has joined #rdo | 14:15 | |
| mnaser | fyi puppet-ceph 2.4.1 is released (which i believe is something folks were looking for) | 14:17 |
| *** eaguilar has joined #rdo | 14:21 | |
| *** eaguilar has quit IRC | 14:22 | |
| *** mcornea has quit IRC | 14:23 | |
| *** Son_Goku has quit IRC | 14:23 | |
| rlandy | therve: thank you for working on the newton backport https://review.openstack.org/#/c/501592/ ,,, related questiom | 14:24 |
| rdogerrit | Attila Darazs created rdo-infra/ci-config master: Add DLRN API based promoter script https://review.rdoproject.org/r/9390 | 14:24 |
| adarazs | trown, myoung: ^ FYI, here's my first patchset for the new DLRN API promoter script | 14:24 |
| rlandy | therve: we now have some stacks in a stuck state because the resource (which did not exist) delete failed | 14:25 |
| rlandy | is there any way to recover from that? | 14:25 |
| rlandy | stack delete keeps failing | 14:25 |
| *** vnogin has quit IRC | 14:26 | |
| *** suresh12 has joined #rdo | 14:27 | |
| *** adarazs is now known as adarazs_brb | 14:28 | |
| *** vnogin has joined #rdo | 14:28 | |
| *** adarazs_brb is now known as adarazs | 14:31 | |
| *** ykarel|afk has quit IRC | 14:33 | |
| *** jprovazn has quit IRC | 14:35 | |
| *** jcoufal has joined #rdo | 14:36 | |
| *** suresh12 has quit IRC | 14:38 | |
| *** _lebauce_ has quit IRC | 14:39 | |
| dmsimard | jruzicka: you still around ? | 14:39 |
| *** cylopez has joined #rdo | 14:40 | |
| *** suresh12 has joined #rdo | 14:41 | |
| dmsimard | jruzicka: I was told this might be relevant to your interests around automatically rendering example specs | 14:41 |
| dmsimard | https://github.com/openstack/puppet-openstack-cookiecutter | 14:41 |
| dmsimard | https://github.com/openstack-dev/cookiecutter | 14:42 |
| number80 | reconnecting | 14:42 |
| dmsimard | https://cookiecutter.readthedocs.io/en/latest/ | 14:42 |
| rdogerrit | Attila Darazs proposed rdo-infra/ci-config master: Add DLRN API based promoter script https://review.rdoproject.org/r/9390 | 14:44 |
| *** suresh12 has quit IRC | 14:46 | |
| *** hrw has quit IRC | 14:47 | |
| *** Amita has quit IRC | 14:48 | |
| *** jlabarre has quit IRC | 14:48 | |
| *** hrw has joined #rdo | 14:49 | |
| *** cylopez has quit IRC | 14:51 | |
| *** jlabarre has joined #rdo | 14:51 | |
| jschlueter | dmsimard: anything for cookiecutter dist-git generation? | 14:51 |
| dmsimard | jschlueter: they're just templates that we have to create, but earlier I was mentioning that instead of https://github.com/rdo-infra/example-distgit we could have a cookiecutter template. | 14:52 |
| *** suresh12 has joined #rdo | 14:52 | |
| dmsimard | jschlueter: I don't think we have a cookiecutter template for dist-git. | 14:52 |
| jpena | apevec: github to review.rdo sync is not working atm. For puppet-rabbitmq, I'm going to do that by hand and propose the patch, ok? | 14:52 |
| jschlueter | dmsimard: ack | 14:52 |
| jpena | btw number80, do you know how we were running rdo_gating_scripts/rdo-upstream-sync/upstream-sync.py in review.rdo? | 14:52 |
| jschlueter | dmsimard: that would be a good thing to try to generate and see if it works or if it fails hard for other issues | 14:52 |
| jschlueter | dmsimard: so that one repo has about 6 different possible templates? hmmm | 14:53 |
| *** ratailor has quit IRC | 14:55 | |
| dmsimard | jschlueter: because the spec file depends on the type of thing you're packaging | 14:56 |
| dmsimard | jschlueter: a spec for a client will not be the same thing as the spec for a puppet module, or a lib | 14:56 |
| *** jlabarre has quit IRC | 14:56 | |
| *** gszasz has quit IRC | 14:56 | |
| number80 | jpena: nope, I think it's a cron tasks but that may have change since | 14:57 |
| *** suresh12 has quit IRC | 14:57 | |
| number80 | Like last time, the rdoinfo update jobs were all run at wednesday after midnight | 14:58 |
| *** suresh12 has joined #rdo | 14:58 | |
| weshay | dmsimard, how would you and the infra team like to receive help requests or bugs? | 14:58 |
| dmsimard | weshay: it's something I've discussed with someone recently | 14:59 |
| dmsimard | who was it | 14:59 |
| dmsimard | It was with Duck | 14:59 |
| *** alee has quit IRC | 14:59 | |
| *** shardy is now known as shardy_mtg | 15:00 | |
| weshay | a what, a duck .. a what? | 15:00 |
| dmsimard | weshay: Duck is a person :) | 15:00 |
| weshay | :) | 15:00 |
| *** jlabarre has joined #rdo | 15:00 | |
| dmsimard | weshay: he works with OSAS and they stood up the ticketing system for rhos ops | 15:00 |
| weshay | cool | 15:00 |
| dmsimard | weshay: truth is I don't really know. When we're asked things and we don't do them *now* we tend to forget and get sidetracked by other fires -- so in that sense, a ticketing system works best but I don't really want to deal with tickets on top of trello and storyboard things | 15:01 |
| trown | dmsimard: looks like we have a bunch of stuff just hung in zuul... do you know how to clear that out? https://review.rdoproject.org/zuul/ | 15:01 |
| *** egonzalez has quit IRC | 15:01 | |
| *** anilvenkata has quit IRC | 15:02 | |
| dmsimard | weshay: and sending emails to one person is not very effective.. maybe something inbetween a ticket system and emails... a mailing list for infra specific things ? | 15:02 |
| rdogerrit | Javier Peña created puppet/puppet-rabbitmq master: Fix for #618, adds config_ranch parameter (default: true) to suppress certain config lines https://review.rdoproject.org/r/9391 | 15:02 |
| *** _milan_ has joined #rdo | 15:02 | |
| dmsimard | trown: looking | 15:02 |
| *** suresh12 has quit IRC | 15:02 | |
| dmsimard | trown: ah shit.. I think we got hit by the same problem as upstream since we build the same images | 15:03 |
| dmsimard | let me look | 15:03 |
| dmsimard | weshay, trown: which reminds me we need to talk about images in review.rdo during the ptg /me adds topic to etherpad | 15:04 |
| dmsimard | because upstream and review.rdo will diverge due to zuul v3 transition | 15:04 |
| *** eliska-mtg has quit IRC | 15:05 | |
| *** artom has quit IRC | 15:05 | |
| *** Amita has joined #rdo | 15:05 | |
| *** anilvenkata has joined #rdo | 15:06 | |
| dmsimard | weshay, trown: I have to drain all the tripleo nodepool VMs, they're using a bad image -- and then rope a good image back in, it'll take a bit. | 15:07 |
| *** rovyas has quit IRC | 15:07 | |
| trown | dmsimard++ thanks! | 15:08 |
| dmsimard | weshay, trown: It's the same problem that made the upstream graphs go flat this morning: http://grafana.openstack.org/dashboard/db/nodepool | 15:08 |
| jschlueter | chandankumar: another one for easyfixes is %license entries in .spec files | 15:11 |
| *** abregman|afk has quit IRC | 15:11 | |
| rdogerrit | Jon Schlueter created openstack/nova-distgit rpm-master: normalize usage of %license https://review.rdoproject.org/r/9392 | 15:11 |
| *** alee has joined #rdo | 15:14 | |
| *** Amita has quit IRC | 15:15 | |
| Duck | quack | 15:15 |
| *** suresh12 has joined #rdo | 15:16 | |
| Duck | dmsimard: you're already using trello? | 15:17 |
| Duck | I thought there was nothing | 15:17 |
| Duck | I'm ok using whatever not-too-crappy system you're already using | 15:17 |
| number80 | Duck: we have a dozen of trello around | 15:17 |
| number80 | Duck: don't worry, we are not using JIRA :) | 15:18 |
| misc | (not yet) | 15:18 |
| Duck | also a ticketing system is not for trivial tasks, it's a waste of time, but for real tasks, projects, todolist | 15:18 |
| dmsimard | Duck: not everyone uses trello, and that's part of the problem | 15:18 |
| Duck | I'm already having "fun" with the oVirt JIRA | 15:18 |
| Duck | the problem I see now is that rbowen wanted to use a tracker noone is using | 15:19 |
| Duck | even *he* is not using it it seems | 15:19 |
| Duck | I'm still waiting for replies | 15:19 |
| *** links has joined #rdo | 15:20 | |
| * number80 wants to use a spreadsheet for bugs like all corporate old timers | 15:20 | |
| Duck | also there seem to be a lack of "team" decisions | 15:20 |
| dmsimard | weshay, trown: cleanup still in progress | 15:20 |
| Duck | let's chose one and stick to it | 15:20 |
| weshay | thank you! | 15:20 |
| Duck | weshay: I'm a real duck, but that's OK :-) | 15:21 |
| weshay | heh :) | 15:21 |
| weshay | Duck, dmsimard reminded me of http://www.egadideas.com/1990/01/do-you-want-to-buy-a-duck-variation/ | 15:22 |
| rbowen | Duck: Sorry, it's far from clear to me what replies you're looking for. | 15:22 |
| rbowen | I don't have any opinion about Fedora auth, and the UI is what the UI is. | 15:22 |
| Duck | rbowen: well, I posted questions, you might have your own ideas, like if Fedora auth is useful to your community | 15:23 |
| *** gfidente has quit IRC | 15:23 | |
| *** apevec has joined #rdo | 15:23 | |
| rbowen | I'll go with "no" since we don't support Fedora at this time. | 15:23 |
| Duck | anyway if the other infra peopel are not looking at these tickets, there is no collaboration ongoing | 15:23 |
| rbowen | I'll reply to the ticket | 15:23 |
| rbowen | And this is why I asked where I should open the ticket. | 15:23 |
| rbowen | If this isn't the right place, then where is? | 15:24 |
| rbowen | That was the question we started with. | 15:24 |
| Duck | rbowen: do you realize I'm on a dev on the project and so I'm not the one going to moderate the lists? so if the tool needs tuning, it's NOW | 15:24 |
| Duck | not | 15:24 |
| Duck | that's a good question | 15:24 |
| Duck | I'm fine withe the tool, but as long as the _entire_ team is not using it it's just a waste of time for me | 15:25 |
| rbowen | Getting a 500 trying to log in to lists.rdoproject.org | 15:25 |
| rbowen | It's not a waste of time for me, since I need a place that I can point people when they ask "what's the status" | 15:25 |
| rbowen | I'm sorry this is a waste of time for you. | 15:25 |
| misc | shouldn't the first question be "what is the team" ? | 15:25 |
| Duck | rbowen: you cannot juste point any people to it, it might get very technical because there's bug to solve, so unreadable | 15:26 |
| Duck | misc: YES | 15:26 |
| *** links has quit IRC | 15:26 | |
| Duck | misc: could you try login-in too? | 15:27 |
| misc | cause there is different needs, intra team communication, extra team communication, and I think duck look for the first, while rbowen look for the 2nd | 15:27 |
| sfbender | Javier Peña proposed DLRN master: Refactor DB usage https://softwarefactory-project.io/r/9244 | 15:27 |
| *** abregman has joined #rdo | 15:27 | |
| misc | Duck: same 500 as rbowen | 15:27 |
| Duck | ack | 15:27 |
| misc | I will mail root@localhost | 15:27 |
| Duck | frack | 15:27 |
| misc | and then forward you the email :p | 15:28 |
| Duck | yeah, I'm on we now, so on my other computer | 15:28 |
| *** jlabarre has quit IRC | 15:28 | |
| misc | but so, i tried the github auth | 15:28 |
| Duck | testing a a good things | 15:28 |
| Duck | I had no problem simce recently but the extra repos are causing mayhem and yum-cron might have draggued UI-breaking stuff again | 15:29 |
| dmsimard | Duck: I think misc said the other day I probably broke stuff due to the extra repos the monitoring setup pulls in | 15:29 |
| misc | yeah, but I did remove the packages using my m4d h4ck3r skills with yum-shell | 15:30 |
| dmsimard | the repositories are not disabled after installation, we can improve the roles to disable them to prevent them from interfering | 15:30 |
| *** jlabarre has joined #rdo | 15:30 | |
| Duck | dmsimard: that's not *you*, that's just a provlem we should face together | 15:31 |
| *** hewbrocca is now known as hewbrocca_afk | 15:31 | |
| Duck | dmsimard: but why are openstack stuff installed on this kind of machines? | 15:31 |
| Duck | except epel and sensu I see nothing related | 15:32 |
| dmsimard | Duck: the sensu implementation is provided by opstools, which is a CentOS SIG, and opstools relies on the Cloud SIG repositories (RDO) for some of the packaging that we provide | 15:32 |
| dmsimard | it's common for CentOS SIGs to rely on each other for packages, just like we use the Ceph SIG when we install Ceph, we don't re-package Ceph | 15:32 |
| Duck | what a nightmare | 15:32 |
| dmsimard | Duck: Not everything can be in EPEL, SIGs have to exist | 15:33 |
| Duck | disabling might help but how can you be sure to not drag breaking-deps from the start ? | 15:33 |
| misc | we should put everything in containers | 15:33 |
| *** wolverineav has quit IRC | 15:33 | |
| *** wolverineav has joined #rdo | 15:33 | |
| misc | (now, everybody wonder if that's sarcasm or not) | 15:34 |
| dmsimard | Duck: I'm busy fixing production stuff right now, can discuss later | 15:34 |
| Duck | I discovered certbot is broken too | 15:34 |
| Duck | and this time this is a problem in the repo, as the required dep around python crypto is not right | 15:34 |
| Duck | (see details in the ticket) | 15:34 |
| Duck | misc: you're a troll :-) | 15:35 |
| misc | Duck: well, not so much, cause if sensu was using a separate FS, this wouldn't interfere | 15:35 |
| misc | maybe having a lxc for mailman would work | 15:35 |
| misc | (or one for sensu, for that matter, but I think sensu need to have more access to the system than mailman need) | 15:36 |
| Duck | misc: no, apache and capabilities, remember my testing of LXD | 15:36 |
| *** eaguilar has joined #rdo | 15:36 | |
| Duck | or we should use very unsecure containers | 15:36 |
| *** shardy_mtg is now known as shardy | 15:37 | |
| misc | that's ok if they are "insecure" in the sense that we want more a chroot than real isolation | 15:37 |
| *** rlandy is now known as rlandy|brb | 15:37 | |
| misc | right now, there is no separation between sensu and apache/mailman 3 | 15:37 |
| *** rlandy|brb is now known as rlandy | 15:37 | |
| *** wolverineav has quit IRC | 15:38 | |
| misc | so if using lxc mean we have no separation, situation is not worst than now, the goal is just to solve that each want a different view of the same package | 15:38 |
| misc | another solution is to use a different LE client, like one self sufficient | 15:38 |
| Duck | LE ? | 15:38 |
| misc | Lets encrypt | 15:39 |
| dmsimard | Duck, misc: fwiw I've asked about putting sensu-client in a container which would make things far easier | 15:39 |
| dmsimard | but monitoring a host from inside a container also brings it's share of challenges | 15:39 |
| *** dprince has quit IRC | 15:39 | |
| misc | (I do not like this, because that's adding complexity, but I am sure we can find a perl one liner that do the whole negociation :p) | 15:39 |
| Duck | misc: is the auth is broken it's probably again because of deps | 15:39 |
| Duck | so that's not just certbot | 15:39 |
| dmsimard | weshay, trown: ok I *think* things are ok, I'm re-enabling nodepool for tripleo and will monitoring things | 15:39 |
| Duck | dmsimard: thanks | 15:40 |
| Duck | so once we have a solution we should trash the VM and start over with the new solution | 15:40 |
| *** mkovacik__ has joined #rdo | 15:41 | |
| dmsimard | weshay, trown: note that I am disabling the project-config upstream nodepool elements and scripts sync because zuul v3 divergence will break us | 15:41 |
| dmsimard | it's a topic for next week CI session | 15:41 |
| misc | Duck: seems python-django come from centos-openstack-ocata , and hyperkitty provides a different version, albeit compatible | 15:41 |
| misc | 1.8.14 from centos ocata, 1.8.8 from hyperkitty | 15:41 |
| Duck | yep, I checked a few components and the changelogs and it should not be it | 15:42 |
| Duck | but the sig is going to evolve and I don't want to fix the world every two days | 15:42 |
| *** _milan_ has quit IRC | 15:43 | |
| *** nyechiel has quit IRC | 15:43 | |
| misc | never gonna fear to be out of work, that's the perfect post modern capitalist dream | 15:43 |
| *** janki|lunch has quit IRC | 15:45 | |
| *** limao has joined #rdo | 15:46 | |
| misc | Duck: [Fri Sep 08 15:46:12.583572 2017] [core:notice] [pid 6655] AH00052: child pid 6656 exit signal Segmentation fault (11) | 15:46 |
| *** aufi has quit IRC | 15:47 | |
| *** jackNemrod has quit IRC | 15:47 | |
| Duck | misc: that's going to be fun again | 15:47 |
| Duck | I'd rather the isolation solution | 15:47 |
| *** iranzo has quit IRC | 15:48 | |
| misc | Duck: so, looking with gdb, that's linked to request using urllib3, using openssl | 15:50 |
| misc | and #4 Frame 0x7fba92c00fa0, for file /usr/lib/python2.7/site-packages/OpenSSL/SSL.py, line 226, in __init__ (self=<_VerifyHelper(_problems=[]) at remote 0x7fba92bb36d0>, callback=<function at remote 0x7fba929bd8c0>, wrapper=<function at remote 0x7fba92bb5410>) "int (*)(int, X509_STORE_CTX *)", wrapper) | 15:50 |
| misc | python2-pyOpenSSL come from centos-openstack-ocata | 15:51 |
| dmsimard | sorry guys :( | 15:51 |
| *** vnogin has quit IRC | 15:51 | |
| Duck | the crtbot problem is pyOpenSSL with incompatible version of python-crypto | 15:52 |
| Duck | IIRC | 15:52 |
| misc | yep, I downgraded the package to the one in base | 15:52 |
| Duck | dmsimard: this should be reported to whoever is in change of the repo providing it | 15:52 |
| *** shreshtha_ has joined #rdo | 15:53 | |
| misc | Duck: and now the 500 is gone | 15:53 |
| Duck | because whatever we do for this host, the repo is providing a broken combination | 15:53 |
| Duck | thanks misc | 15:53 |
| rbowen | Ok, I'm able to log in to lists.rdo now | 15:53 |
| Duck | so you can resume your li^W testing the wonderful new Ml UI | 15:53 |
| dmsimard | Duck: please open an issue in https://github.com/rdo-infra/rdo-infra-playbooks | 15:53 |
| misc | python2-pyOpenSSL did replace pyOpenSSL (base), so I wasn't able to detect that it was upgraded | 15:54 |
| dmsimard | brb getting kids from school | 15:54 |
| Duck | dmsimard: I can't now, I do not have proper creds here | 15:54 |
| Duck | it's 00:54 on Saturday here | 15:54 |
| rdogerrit | Attila Darazs proposed rdo-infra/ci-config master: Add DLRN API based promoter script https://review.rdoproject.org/r/9390 | 15:54 |
| *** shreshtha has quit IRC | 15:55 | |
| *** dprince has joined #rdo | 15:56 | |
| Duck | thanks misc | 15:56 |
| *** myoung is now known as myoung|biab | 15:57 | |
| misc | and I am a bit too knocked out by antibio to figure how to use yum and repoquery to find what I am looking for | 15:57 |
| Duck | misc: ill again? | 15:58 |
| misc | Duck: wosdom teeth infection | 15:59 |
| misc | wosdom | 15:59 |
| misc | rah | 15:59 |
| misc | wisdom | 15:59 |
| Duck | ouch :-/ | 15:59 |
| Duck | removal ? | 15:59 |
| *** karimb has quit IRC | 16:00 | |
| misc | Duck: next week | 16:00 |
| misc | (or rather in 10 days) | 16:00 |
| Duck | still have mine so I don't know how it feels, but… gambatte ne! | 16:01 |
| misc | well, I also still have mine | 16:01 |
| misc | (for now) | 16:01 |
| Duck | seems it is not totally necessary | 16:01 |
| *** eaguilar has quit IRC | 16:01 | |
| Duck | a lot of people have them removed much earlier no? | 16:02 |
| misc | dunno | 16:03 |
| *** jbrooks has joined #rdo | 16:05 | |
| *** flepied has quit IRC | 16:05 | |
| *** tesseract has quit IRC | 16:08 | |
| *** wolverineav has joined #rdo | 16:08 | |
| dmsimard | weshay, trown: tripleo jobs have successfully dequeued as far as I can tell so considering that fixed, let me know if there's anything else | 16:09 |
| mnaser | if folks need any help fixing up rdocloud i'd be more than happy to assist btw :> | 16:09 |
| mnaser | i've ran a cloud or two with devstack before... | 16:09 |
| mnaser | :-p | 16:09 |
| dmsimard | mnaser: it's operated by a different team, we're end users | 16:09 |
| mnaser | ah okay :> | 16:10 |
| *** dtantsur is now known as dtantsur|afk | 16:10 | |
| dmsimard | mnaser: the idea is to have different people operate it on purpose to have a better feedback loop between dev/engineering/ops | 16:12 |
| *** jmelvin has quit IRC | 16:12 | |
| dmsimard | they even moved to a different Red Hat organization now :) | 16:13 |
| mnaser | sounds like the right way of doing things, but i guess with the scary part of spof | 16:13 |
| dmsimard | Duck: you don't have an account on github? at all? | 16:13 |
| mnaser | where dev/engineering/ops all stop :p | 16:13 |
| *** dbecker has quit IRC | 16:13 | |
| *** limao has quit IRC | 16:14 | |
| *** abregman is now known as abregman|afk | 16:16 | |
| dmsimard | number80, apevec, trown, weshay: https://lists.centos.org/pipermail/centos-devel/2017-September/016159.html | 16:16 |
| number80 | Good | 16:17 |
| apevec | which version is it in Extras? | 16:18 |
| apevec | 2.3.1 | 16:18 |
| *** jlabarre has quit IRC | 16:18 | |
| *** jlabarre has joined #rdo | 16:18 | |
| dmsimard | same as RHEL is my understanding | 16:18 |
| dmsimard | so it might be 2.3.1 forever? I dunno. | 16:18 |
| apevec | we have 2.3.0 in common-release | 16:18 |
| apevec | so how do we test it? | 16:19 |
| dmsimard | 2.3.2 is already out and 2.3.3 will be likely be tagged not long after 2.4 | 16:19 |
| apevec | extras is not built in CBS so we cannot crosstag | 16:19 |
| *** mvk has joined #rdo | 16:19 | |
| dmsimard | apevec: it's in extras so we don't need to? | 16:19 |
| dmsimard | apevec: you just install it | 16:19 |
| honza | dmsimard: thanks for the email; the issue persists so i guess i'll need to open a ticket | 16:19 |
| apevec | dmsimard, that's unreleased 7.4 extras isn't it? | 16:19 |
| apevec | or was it released ? | 16:20 |
| dmsimard | apevec: doesn't seem necessarily tied to 7.4, but he does mention a few days | 16:20 |
| apevec | dmsimard, re. 2.3.2 yeah - paas sig and ovirt are testing with 2.3.2 http://cbs.centos.org/koji/packageinfo?packageID=1947 | 16:20 |
| *** flepied has joined #rdo | 16:20 | |
| apevec | Extras is tied to 7.4 | 16:20 |
| dmsimard | so we might be stuck with 2.3.1 forever ? :D | 16:20 |
| dmsimard | or can extras be rebuilt without dot releases | 16:21 |
| apevec | centos extras = rhel extras + few centos things, like centos cloud-init fork | 16:21 |
| *** _bsurfer_ has quit IRC | 16:21 | |
| apevec | dmsimard, extras has more relaxed updated policy | 16:21 |
| dmsimard | ok | 16:21 |
| apevec | and SLA | 16:21 |
| apevec | so rebases are happening | 16:21 |
| dmsimard | good | 16:22 |
| dmsimard | apevec: I replied and asked how we could test it prior to release | 16:23 |
| dmsimard | honza: you know the email address ? | 16:23 |
| Duck | dmsimard: personnal one? no. I host my own stuff, I don't trust third parties | 16:23 |
| dmsimard | ok.. | 16:23 |
| honza | dmsimard: i think so; same place that sent me credentials? | 16:24 |
| *** rejy has quit IRC | 16:25 | |
| *** jbrooks has quit IRC | 16:30 | |
| *** jbrooks has joined #rdo | 16:32 | |
| misc | Duck: so the server should now be reasonably cleared of external packages | 16:33 |
| misc | the renaming of the rpm didn't help, and the reason why we had to rename them (because original name are incosnsitent) didn't also help to script the cleanup :/ | 16:33 |
| *** jpich has quit IRC | 16:35 | |
| *** dustins has joined #rdo | 16:35 | |
| *** jlabarre has quit IRC | 16:36 | |
| honza | dmsimard: actually, i just found the right info in the manual; thanks | 16:38 |
| *** jmelvin has joined #rdo | 16:38 | |
| *** artom has joined #rdo | 16:38 | |
| *** jlabarre has joined #rdo | 16:39 | |
| *** rcernin has quit IRC | 16:40 | |
| *** derekh has quit IRC | 16:42 | |
| *** vnogin has joined #rdo | 16:43 | |
| *** noslzzp_ has quit IRC | 16:47 | |
| *** shardy has quit IRC | 16:47 | |
| *** noslzzp has joined #rdo | 16:47 | |
| *** vnogin has quit IRC | 16:47 | |
| *** suresh12 has quit IRC | 16:51 | |
| *** suresh12 has joined #rdo | 16:51 | |
| *** jcoufal has quit IRC | 16:53 | |
| *** milan has quit IRC | 16:54 | |
| *** mkovacik__ has quit IRC | 16:55 | |
| *** suresh12 has quit IRC | 16:57 | |
| *** jlibosva has quit IRC | 16:59 | |
| *** MrAbaddon has quit IRC | 17:00 | |
| jpena | have a nice weekend everyone, and a safe trip those going to the PTG! | 17:01 |
| dmsimard | jpena: see ya | 17:01 |
| *** gcerami has quit IRC | 17:02 | |
| *** jpena is now known as jpena|off | 17:02 | |
| *** thrash is now known as thrash|biab | 17:09 | |
| *** jmlowe has quit IRC | 17:09 | |
| *** jmlowe has joined #rdo | 17:10 | |
| *** athomas has quit IRC | 17:10 | |
| *** aditya_r has joined #rdo | 17:12 | |
| *** suresh12 has joined #rdo | 17:12 | |
| *** suresh12 has quit IRC | 17:16 | |
| *** tosky has quit IRC | 17:17 | |
| *** trown is now known as trown|lunch | 17:25 | |
| *** apevec has quit IRC | 17:30 | |
| *** DV has joined #rdo | 17:32 | |
| *** myoung|biab is now known as myoung | 17:33 | |
| *** jcoufal has joined #rdo | 17:35 | |
| *** jcoufal has quit IRC | 17:35 | |
| *** jcoufal has joined #rdo | 17:36 | |
| *** ykarel has joined #rdo | 17:37 | |
| *** vnogin has joined #rdo | 17:41 | |
| *** MrAbaddon has joined #rdo | 17:41 | |
| *** tosky has joined #rdo | 17:41 | |
| *** jcoufal has quit IRC | 17:42 | |
| *** jcoufal has joined #rdo | 17:42 | |
| *** vnogin has quit IRC | 17:45 | |
| *** jcoufal has quit IRC | 17:45 | |
| *** jcoufal has joined #rdo | 17:45 | |
| *** anilvenkata has quit IRC | 17:46 | |
| *** paragan has quit IRC | 17:46 | |
| *** jlabarre has quit IRC | 17:48 | |
| *** jcoufal has quit IRC | 17:48 | |
| *** jlabarre has joined #rdo | 17:48 | |
| *** jcoufal has joined #rdo | 17:48 | |
| *** eaguilar has joined #rdo | 17:51 | |
| *** jackNemrod has joined #rdo | 17:51 | |
| *** Goneri has quit IRC | 17:52 | |
| *** abregman|afk has quit IRC | 17:53 | |
| *** jackNemrod has quit IRC | 17:55 | |
| *** jackNemrod has joined #rdo | 17:56 | |
| *** eaguilar has quit IRC | 17:58 | |
| *** ykarel has quit IRC | 18:01 | |
| *** Son_Goku has joined #rdo | 18:03 | |
| sfbender | Jon Schlueter created rdopkg master: Extend newversion.feature for coverage of -H https://softwarefactory-project.io/r/9634 | 18:04 |
| *** dtrainor has joined #rdo | 18:04 | |
| *** sbulage has quit IRC | 18:07 | |
| *** jlabarre has quit IRC | 18:10 | |
| *** jlabarre has joined #rdo | 18:10 | |
| *** jackNemrod_ has joined #rdo | 18:16 | |
| *** jackNemrod has quit IRC | 18:17 | |
| *** jackNemrod_ is now known as jackNemrod | 18:17 | |
| *** egonzalez has joined #rdo | 18:20 | |
| *** thrash|biab is now known as thrash | 18:22 | |
| *** jackNemrod has quit IRC | 18:24 | |
| *** florianf has quit IRC | 18:29 | |
| *** Son_Goku has quit IRC | 18:30 | |
| *** eharney has quit IRC | 18:31 | |
| *** aditya_r has quit IRC | 18:32 | |
| *** aditya_r has joined #rdo | 18:33 | |
| rdogerrit | Aditya Ramteke created openstack/oslo-cache-distgit rpm-master: Moved the description to global variables. https://review.rdoproject.org/r/9393 | 18:36 |
| *** jlabarre has quit IRC | 18:38 | |
| *** jlabarre has joined #rdo | 18:38 | |
| *** Son_Goku has joined #rdo | 18:39 | |
| *** aditya_r has quit IRC | 18:41 | |
| *** scorcoran is now known as scorcoran_biab | 18:43 | |
| *** Son_Goku has quit IRC | 18:43 | |
| *** eharney has joined #rdo | 18:44 | |
| *** jbrooks has quit IRC | 18:44 | |
| sfbender | Jon Schlueter created rdopkg master: add scenario for Reproducing fix bump error https://softwarefactory-project.io/r/9636 | 18:47 |
| *** jlabarre has quit IRC | 18:48 | |
| jschlueter | jjoyce: https://softwarefactory-project.io/r/9634 - building up coverage in behave to try to get a reproducer for https://github.com/softwarefactory-project/rdopkg/issues/121 | 18:48 |
| *** jlabarre has joined #rdo | 18:48 | |
| jschlueter | jruzicka: some more rdopkg behave scenario patches incoming | 18:49 |
| sfbender | Jon Schlueter proposed rdopkg master: add scenario for Reproducing fix bump error https://softwarefactory-project.io/r/9636 | 18:51 |
| *** pkovar has quit IRC | 18:52 | |
| *** jcoufal has quit IRC | 18:54 | |
| *** scorcoran_biab is now known as scorcoran | 19:06 | |
| *** trown|lunch is now known as trown | 19:10 | |
| *** jlabarre has quit IRC | 19:12 | |
| number80 | o/ | 19:12 |
| *** jlabarre has joined #rdo | 19:14 | |
| *** jackNemrod has joined #rdo | 19:20 | |
| *** eck` is now known as eck`gone | 19:24 | |
| *** Goneri has joined #rdo | 19:32 | |
| *** smcmurray has joined #rdo | 19:32 | |
| *** smcmurray has left #rdo | 19:32 | |
| *** pkovar has joined #rdo | 19:33 | |
| *** esem has joined #rdo | 19:33 | |
| esem | The ooo docs say it requires at least three baremetal hosts. 1. undercloud 2. overcloud manager 3. compute node | 19:37 |
| esem | Can't 1 and 2 be combined? | 19:37 |
| *** alee has quit IRC | 19:42 | |
| *** egonzalez has quit IRC | 19:47 | |
| *** pchavva has quit IRC | 19:49 | |
| *** tosky has quit IRC | 19:49 | |
| *** kgiusti has left #rdo | 19:53 | |
| *** alee has joined #rdo | 19:55 | |
| *** tosky has joined #rdo | 19:56 | |
| dmsimard | esem: you can do virtual deployments for test purposes | 19:57 |
| dmsimard | esem: see https://github.com/openstack/tripleo-quickstart and https://www.rdoproject.org/tripleo/ | 19:57 |
| *** dtrainor has quit IRC | 20:00 | |
| *** eck`gone is now known as eck` | 20:02 | |
| esem | dmsimard: I've been there. I'm trying to digest it all | 20:10 |
| *** jayg is now known as jayg|g0n3 | 20:17 | |
| *** artom has quit IRC | 20:20 | |
| *** karimb_ has joined #rdo | 20:21 | |
| *** pkovar has quit IRC | 20:21 | |
| *** rbowen has quit IRC | 20:31 | |
| *** beekneemech has quit IRC | 20:35 | |
| *** trown is now known as trown|outtypewww | 20:35 | |
| *** jbrooks has joined #rdo | 20:38 | |
| *** scorcoran is now known as scorcoran_afk | 20:40 | |
| *** eharney has quit IRC | 20:49 | |
| *** dprince has quit IRC | 20:49 | |
| *** funzo_ has quit IRC | 20:53 | |
| *** egallen_ has joined #rdo | 20:57 | |
| *** egallen_ has quit IRC | 20:57 | |
| *** aludwar has quit IRC | 20:59 | |
| *** aludwar has joined #rdo | 20:59 | |
| *** aludwar has joined #rdo | 21:00 | |
| *** rcernin has joined #rdo | 21:06 | |
| *** julim has quit IRC | 21:07 | |
| *** jackNemrod has quit IRC | 21:11 | |
| *** vnogin has joined #rdo | 21:19 | |
| *** dustins has quit IRC | 21:22 | |
| *** jlabarre has quit IRC | 21:25 | |
| *** myoung is now known as myoung|afk | 21:26 | |
| sfbender | Jon Schlueter proposed rdopkg master: make spec file Then assert more descriptive https://softwarefactory-project.io/r/9619 | 21:34 |
| sfbender | Jon Schlueter proposed rdopkg master: add scenario for Reproducing fix bump error https://softwarefactory-project.io/r/9636 | 21:34 |
| sfbender | Jon Schlueter proposed rdopkg master: Add newversion.feature scenario using --bug https://softwarefactory-project.io/r/9618 | 21:36 |
| sfbender | Jon Schlueter proposed rdopkg master: Extend newversion.feature for coverage of -H https://softwarefactory-project.io/r/9634 | 21:36 |
| *** imcleod has quit IRC | 21:44 | |
| *** mvk has quit IRC | 21:48 | |
| *** alee has quit IRC | 21:48 | |
| stevebaker | dmsimard: hey, it looks like zm01.review.rdoproject.org is still down since the outage https://review.rdoproject.org/jenkins/job/gate-tox-ansible-lint/128/console | 21:52 |
| *** funzo has joined #rdo | 21:53 | |
| *** scorcoran_afk has quit IRC | 21:53 | |
| esem | why does the root user for undercloud have to be passwordless to run the ansible scripts? | 21:56 |
| *** funzo has quit IRC | 21:58 | |
| *** jmelvin has quit IRC | 22:01 | |
| *** mvk has joined #rdo | 22:04 | |
| *** vnogin has quit IRC | 22:13 | |
| *** artom has joined #rdo | 22:13 | |
| *** rcernin has quit IRC | 22:19 | |
| esem | ooo needs 60GB free diskspace on undercloud. On which part of the filesystem does that space need to be? /var? | 22:24 |
| *** wolverineav has quit IRC | 22:27 | |
| *** wolverineav has joined #rdo | 22:27 | |
| dmsimard | stevebaker: that's odd, unless mistaken we ran fine today. I'll have a look. | 22:32 |
| *** karimb_ has quit IRC | 22:36 | |
| *** vnogin has joined #rdo | 22:36 | |
| *** alee has joined #rdo | 22:40 | |
| *** wolverineav has quit IRC | 22:41 | |
| rdogerrit | Merged puppet/puppet-rabbitmq-distgit rpm-master: Add temporary patch to restore idempotency https://review.rdoproject.org/r/9388 | 22:47 |
| *** thrash is now known as thrash|g0ne | 22:51 | |
| *** rlandy has quit IRC | 22:52 | |
| *** amuller has quit IRC | 22:53 | |
| *** Caterpillar2 has quit IRC | 22:54 | |
| *** funzo has joined #rdo | 22:54 | |
| *** funzo has quit IRC | 22:58 | |
| *** pppingme has quit IRC | 23:19 | |
| *** tosky has quit IRC | 23:19 | |
| *** pppingme has joined #rdo | 23:32 | |
| *** vnogin_ has joined #rdo | 23:42 | |
| *** julim has joined #rdo | 23:44 | |
| *** vnogin has quit IRC | 23:44 | |
| *** shrjoshi has joined #rdo | 23:54 | |
| *** funzo has joined #rdo | 23:55 | |
| *** shreshtha_ has quit IRC | 23:56 | |
| *** funzo has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!