*** rlandy has quit IRC | 00:07 | |
*** Goneri has joined #oooq | 00:29 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291 | 00:48 |
---|---|---|
*** Goneri has quit IRC | 01:09 | |
*** username_ has joined #oooq | 01:20 | |
*** username_ is now known as username__ | 01:23 | |
*** saneax is now known as saneax-_-|AFK | 02:02 | |
*** username__ has quit IRC | 02:35 | |
*** myoung|ruck|bbl is now known as myoung|ruck | 02:38 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 02:48 |
*** udesale has joined #oooq | 03:36 | |
*** links has joined #oooq | 03:43 | |
*** udesale has quit IRC | 04:00 | |
*** udesale has joined #oooq | 04:01 | |
*** ykarel|away has joined #oooq | 04:18 | |
*** udesale_ has joined #oooq | 04:36 | |
*** udesale has quit IRC | 04:39 | |
*** ratailor has joined #oooq | 04:42 | |
*** marios has joined #oooq | 04:46 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 04:48 |
*** ykarel|away is now known as ykarel | 05:07 | |
ykarel | failed: [localhost] (item=[u'current-tripleo-rdo', u'swift-proxy-server']) => {"changed": false, "item": ["current-tripleo-rdo", "swift-proxy-server"], "msg": "Error pushing image docker.io/tripleomaster/centos-binary-swift-proxy-server: Error: Status 403 trying to push repository tripleomaster/centos-binary-swift-proxy-server: \"Push failed due to a network error. Please try again. If the problem persists, it may be due to a slow connection.\""} | 05:07 |
*** udesale_ has quit IRC | 05:22 | |
bandini | a nice and simple one, folks. great way to start the morning ;) https://review.openstack.org/#/c/490590/ (ignore the old august jenkins failures) | 05:35 |
*** jfrancoa has joined #oooq | 05:36 | |
*** pgadiya has joined #oooq | 05:37 | |
*** pgadiya has quit IRC | 05:37 | |
*** jtomasek has joined #oooq | 05:41 | |
*** jtomasek has quit IRC | 05:41 | |
*** ratailor has quit IRC | 05:41 | |
*** jtomasek has joined #oooq | 05:42 | |
ykarel | bandini, strange | 05:42 |
ykarel | bandini, 3 months back only commit message merged for this fix: https://review.openstack.org/#/c/509084/ | 05:43 |
ykarel | 4th PS had the change command --> shell | 05:43 |
ykarel | not sure how it happened | 05:44 |
bandini | ykarel: oh wow. that is so weird | 05:44 |
bandini | oh well | 05:45 |
*** quiquell|off has quit IRC | 05:45 | |
ykarel | whats going on i can see your patch also is merged 4 months ago: https://review.openstack.org/#/c/534232/1 | 05:48 |
ykarel | ohh this patch is too old | 05:48 |
ykarel | can be ignored https://review.openstack.org/#/c/490590/ | 05:48 |
*** quiquell has joined #oooq | 05:49 | |
*** udesale_ has joined #oooq | 05:51 | |
ykarel | bandini, commented | 05:53 |
*** udesale_ has quit IRC | 05:53 | |
*** udesale has joined #oooq | 05:53 | |
*** kopecmartin has joined #oooq | 05:54 | |
bandini | oh I see, ack | 05:55 |
*** ratailor has joined #oooq | 05:55 | |
*** ratailor has quit IRC | 06:05 | |
*** ratailor_ has joined #oooq | 06:05 | |
*** saneax-_-|AFK is now known as saneax | 06:10 | |
*** holser__ has joined #oooq | 06:14 | |
*** quiquell has quit IRC | 06:26 | |
*** quiquell has joined #oooq | 06:27 | |
*** saneax is now known as saneax-_-|AFK | 06:30 | |
*** saneax-_-|AFK is now known as saneax | 06:33 | |
*** jaosorior has joined #oooq | 06:34 | |
*** ccamacho has quit IRC | 06:38 | |
*** brault has joined #oooq | 06:38 | |
*** ccamacho has joined #oooq | 06:39 | |
*** holser___ has joined #oooq | 06:43 | |
*** holser__ has quit IRC | 06:44 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 06:48 |
*** quiquell is now known as quiquell|ack | 06:58 | |
*** quiquell|ack is now known as quiquell|afk | 06:58 | |
*** ratailor__ has joined #oooq | 07:08 | |
*** ratailor_ has quit IRC | 07:10 | |
*** ratailor_ has joined #oooq | 07:15 | |
*** florianf has joined #oooq | 07:16 | |
*** ratailor__ has quit IRC | 07:17 | |
*** tesseract has joined #oooq | 07:22 | |
*** amoralej|off is now known as amoralej | 07:36 | |
sshnaidm|rover | rasca, hi | 07:37 |
*** tosky has joined #oooq | 07:37 | |
sshnaidm|rover | rasca, is it still relevant? https://review.openstack.org/#/c/553465/ | 07:37 |
*** bogdando has joined #oooq | 07:46 | |
*** dtantsur|afk is now known as dtantsur | 07:53 | |
*** ykarel is now known as ykarel|lunch | 07:58 | |
sshnaidm|rover | arxcruz, what's about https://review.openstack.org/#/c/565716/ ? why is it DNM? | 08:02 |
arxcruz | sshnaidm|rover: checking | 08:03 |
arxcruz | sshnaidm|rover: i was testing first if the ironic tests would pass | 08:04 |
arxcruz | then i need to figure out which tests to run | 08:04 |
arxcruz | that's why is still a dnm | 08:04 |
sshnaidm|rover | arxcruz, ok, better to have DNM in first line, so that people will see it without opening a patch | 08:05 |
arxcruz | sshnaidm|rover: gotcha | 08:05 |
*** apetrich has quit IRC | 08:07 | |
quiquell|afk | sshnaidm|rover: Reproducers + libvirt is usable ? | 08:07 |
*** quiquell|afk is now known as quiquell | 08:08 | |
sshnaidm|rover | quiquell|afk, for some yes, not for me though | 08:08 |
sshnaidm|rover | quiquell, if you will have dns problem like me, you can run with https://review.openstack.org/#/c/565839/ | 08:10 |
quiquell | sshnaidm|rover: Using it reduce deploy time ? | 08:16 |
sshnaidm|rover | quiquell, using what? | 08:16 |
quiquell | reproducer + Ãlibvirt | 08:16 |
sshnaidm|rover | quiquell, well, depends, rdo cloud I think would be faster, but it's sometimes not available | 08:18 |
quiquell | Ahhh, so reproducer + libvirt can be used as a fallback | 08:18 |
quiquell | okok | 08:18 |
quiquell | Thanks | 08:18 |
sshnaidm|rover | quiquell, although when I use docker and http proxy with libvirt, I think it could be faster than rdo cloud | 08:18 |
quiquell | What http proxy ? | 08:19 |
sshnaidm|rover | quiquell, just squid, it's not supported yet, but I use it with my old POC: https://review.openstack.org/#/c/543429/ | 08:20 |
sshnaidm|rover | quiquell, I hope we could use it with current libvirt reproducer too | 08:20 |
sshnaidm|rover | sometime | 08:22 |
quiquell | sshnaidm|rover: Would be great | 08:22 |
*** skramaja has joined #oooq | 08:26 | |
*** udesale_ has joined #oooq | 08:28 | |
*** udesale has quit IRC | 08:28 | |
*** lucas-bbl is now known as lucasagomes | 08:31 | |
rasca | sshnaidm|rover, yes, I will take care of this very soon | 08:32 |
rasca | sshnaidm|rover, this is something we really need | 08:32 |
rasca | from a baremetal perspective | 08:32 |
*** jaosorior has quit IRC | 08:36 | |
*** jaosorior has joined #oooq | 08:41 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 08:48 |
*** udesale_ has quit IRC | 08:59 | |
*** udesale_ has joined #oooq | 08:59 | |
*** udesale_ has quit IRC | 09:00 | |
*** udesale has joined #oooq | 09:00 | |
*** panda|off is now known as panda | 09:01 | |
*** gkadam__ has joined #oooq | 09:06 | |
quiquell | sshnaidm|rover: To check toci_type in reproducers at runtime | 09:11 |
quiquell | https://review.openstack.org/#/c/567162 | 09:11 |
quiquell | With this you can change from multinode to ovb without changing the reproducer | 09:11 |
*** ykarel|lunch is now known as ykarel | 09:28 | |
*** zoli is now known as zoli|lunch | 09:31 | |
*** apetrich has joined #oooq | 09:37 | |
*** jaganathan has quit IRC | 09:39 | |
*** ratailor__ has joined #oooq | 09:44 | |
*** ratailor_ has quit IRC | 09:47 | |
quiquell | panda: Do we have a card to poke at undercloud upgrade ? | 10:11 |
sshnaidm|rover | quiquell, you need to configure your editor to strip spaces automatically from end on line | 10:17 |
quiquell | sshnaidm|rover: Did I do it again ? damn... | 10:18 |
quiquell | I have to... | 10:18 |
panda | quiquell: there are currently no cards | 10:31 |
quiquell | panda: I am testing it changing the overcloud-upgrade role, maybe we need a card to state the status of it | 10:33 |
panda | quiquell: what are you testing exactly ? | 10:34 |
quiquell | Using trowns patch I pass the correct release file to undercloud-upgrade | 10:34 |
quiquell | And changing the role so it doesn't read from UPGRADE_RELEASE | 10:34 |
quiquell | instead use {{ release }}Ã | 10:34 |
panda | quiquell: can we chat a sec ? | 10:36 |
quiquell | https://bluejeans.com/7891065232 | 10:37 |
*** saneax is now known as saneax-_-|AFK | 10:38 | |
*** zoli|lunch is now known as zoli | 10:46 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 10:48 |
*** ykarel_ has joined #oooq | 10:59 | |
*** gkadam_ has joined #oooq | 10:59 | |
*** gkadam__ has quit IRC | 10:59 | |
*** pliu has quit IRC | 10:59 | |
*** ykarel has quit IRC | 11:00 | |
*** tcw has quit IRC | 11:01 | |
*** pliu has joined #oooq | 11:02 | |
*** lucasagomes is now known as lucas-hungry | 11:06 | |
*** tcw has joined #oooq | 11:09 | |
*** atoth has joined #oooq | 11:25 | |
*** ratailor__ has quit IRC | 11:47 | |
*** apetrich has quit IRC | 11:53 | |
*** quiquell is now known as quiquell|lunch | 11:54 | |
*** apetrich has joined #oooq | 11:55 | |
weshay | myoung|ruck, updating the status a bit | 11:59 |
weshay | this meeting is about osp-13 status, osp-14 planning | 12:00 |
weshay | myoung|ruck, keep it tight on queens and on promotions :) | 12:01 |
weshay | panda, howdy | 12:07 |
*** lucas-hungry is now known as lucasagomes | 12:07 | |
weshay | sshnaidm|rover, what change did you have to make to get the patch expiration to work? | 12:08 |
sshnaidm|rover | weshay, what do you mean? | 12:11 |
weshay | sshnaidm|rover, so I see patches are being abandoned was that done via a policy, script, how? | 12:11 |
sshnaidm|rover | weshay, using my hands and eyes :) | 12:12 |
weshay | oh no | 12:12 |
weshay | k | 12:12 |
sshnaidm|rover | weshay, better to look at every case separately | 12:13 |
sshnaidm|rover | weshay, please see a mail | 12:13 |
weshay | ah thanks | 12:13 |
weshay | sshnaidm|rover, thanks for cleaning it up! | 12:14 |
weshay | sshnaidm++ | 12:14 |
weshay | would like to see it done automatically | 12:15 |
weshay | that was quite a bit of work | 12:15 |
*** tcw has quit IRC | 12:15 | |
*** tcw has joined #oooq | 12:17 | |
*** tcw has quit IRC | 12:18 | |
sshnaidm|rover | weshay, yeah, but it's for 2 years | 12:18 |
*** tcw has joined #oooq | 12:18 | |
sshnaidm|rover | weshay, next time it will be much easier | 12:18 |
weshay | panda, you around? | 12:20 |
weshay | sshnaidm|rover, panda need a review for the ui guys https://review.openstack.org/#/c/518043/ | 12:21 |
sshnaidm|rover | weshay, it's not tested in ci, right? | 12:23 |
weshay | sshnaidm|rover, https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-extras-gate-newton-delorean-full-minimal-5726/console.txt.gz | 12:23 |
weshay | 2018-05-08 22:17:37.531 | TASK [validate-ui : Execute the simple tripleo-ui validation] ****************** | 12:24 |
weshay | is the only test we have for htat | 12:24 |
weshay | that | 12:24 |
sshnaidm|rover | weshay, and it's for newton | 12:24 |
weshay | ya :( | 12:24 |
weshay | sshnaidm|rover, I wonder if it's retrying a more recent release | 12:25 |
weshay | in ci.centos | 12:25 |
weshay | panda, where you at | 12:26 |
sshnaidm|rover | weshay, I have problem with expressions like "release not in ['master', ....]" | 12:26 |
sshnaidm|rover | weshay, every time when master moves it's broken | 12:26 |
sshnaidm|rover | weshay, better to use from older releases, like "release in [newton, ocata] release not in [newton, ocata]" | 12:27 |
weshay | sshnaidm|rover, ya.. I'd be more comfortable doing that if there was more support for leaving if statements on branches that are not executed upstream | 12:28 |
panda | weshay: hey, sorry, lunch | 12:28 |
weshay | panda, ok.. I need 2 min.. and then we should sync | 12:28 |
sshnaidm|rover | weshay, we have support from newton afaik | 12:29 |
sshnaidm|rover | so it could be as oldest branch | 12:29 |
sshnaidm|rover | weshay, maybe we need an ansible lint rule for that.. | 12:29 |
*** rlandy has joined #oooq | 12:32 | |
weshay | sshnaidm|rover, ya.. but we have a history of removing the older branches from if statements | 12:33 |
weshay | panda, k.. ready | 12:33 |
weshay | panda, https://bluejeans.com/u/whayutin/ | 12:33 |
*** trown|outtypewww is now known as trown | 12:34 | |
weshay | sshnaidm|rover, so I can update the patch to flip it.. we just need to consider our policy on removing old branches from if statements | 12:34 |
sshnaidm|rover | weshay, I don't understand - what is the issue with removing old branches? | 12:35 |
sshnaidm|rover | weshay, if we remove "newton" everywhere it still works | 12:35 |
weshay | sshnaidm|rover, except when I run on osp 10 | 12:35 |
sshnaidm|rover | weshay, not sure I understand | 12:36 |
weshay | ok.. so if I say.. when: release in [ kilo, liberty, newton, ocata ] vs.. when release not in [ pike, queens, master ] the logic of the when statement is likely to change if we rip out old releases from if statements | 12:37 |
weshay | so I've been leaning on not doing the former | 12:38 |
weshay | I can change the when's so there is less maint for us as you suggested.. just looking for that to be considered when we from time to time remove the older branches from when statements | 12:39 |
weshay | that is all :) | 12:39 |
sshnaidm|rover | weshay, I don't understand why logic of when statement should change? If no old releases anymore, you just remove "when" | 12:40 |
sshnaidm|rover | weshay, removing old branches support and using releases in "when" are different things, btw | 12:41 |
sshnaidm|rover | weshay, if ocata and newton are deprecated for oooq - we just remove this "when" | 12:42 |
weshay | sshnaidm|rover, so do we only consider what is deprecated upstream, or do we take into account osp? | 12:42 |
sshnaidm|rover | weshay, I don't understand how it's related in this case | 12:43 |
weshay | well.. I'm speaking to how I generally craft the when statements | 12:43 |
weshay | to be defensive with regards to older releases | 12:44 |
sshnaidm|rover | weshay, what's problem with osp when you use older branches in "when" statements? | 12:44 |
weshay | sshnaidm|rover, we tend to remove the older branches from when statements from time to time | 12:45 |
sshnaidm|rover | weshay, yeah | 12:46 |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 12:48 |
*** skramaja_ has joined #oooq | 12:49 | |
*** skramaja has quit IRC | 12:50 | |
weshay | sshnaidm|rover, did you see https://bugs.launchpad.net/tripleo/+bug/1770021 | 12:50 |
openstack | Launchpad bug 1770021 in tripleo "hubbot: TQE check jobs have not been kicked by hubbot since 4/25" [Critical,Triaged] | 12:50 |
sshnaidm|rover | weshay, yeah, didn't find any problem, waiting for myoung|ruck to explain | 12:51 |
weshay | k | 12:51 |
quiquell|lunch | sshnaidm|rover, myoung|ruck, weshay: I have put together a dashboard for ruck/rover | 12:51 |
quiquell|lunch | http://38.145.34.131:7272/ | 12:51 |
quiquell|lunch | From stuff I needed in my previous ruck/rovering | 12:52 |
*** quiquell|lunch is now known as quiquell | 12:52 | |
weshay | sshnaidm|rover, so look at https://review.openstack.org/#/c/560452 | 12:52 |
sshnaidm|rover | quiquell++ | 12:52 |
hubbot | sshnaidm|rover: quiquell's karma is now 1 | 12:52 |
trown | nice | 12:52 |
sshnaidm|rover | quiquell, you read my thoughts, just started to do it with grafana :D | 12:53 |
quiquell | sshnaidm|rover: Let me know if it's of any use or we can change something | 12:53 |
*** skramaja has joined #oooq | 12:53 | |
quiquell | Is still some WIP | 12:53 |
*** skramaja_ has quit IRC | 12:54 | |
sshnaidm|rover | quiquell, what do you use for dashboard? | 12:54 |
quiquell | tipboard | 12:54 |
quiquell | Didn't customize it too much | 12:54 |
quiquell | But you can | 12:54 |
quiquell | Just want to show the info | 12:54 |
sshnaidm|rover | quiquell, wanna switch to telegraph+grafana? | 12:54 |
quiquell | Let me check, didn't want graphs I want green/red boxes :-) | 12:55 |
quiquell | And links pointing to logs | 12:55 |
quiquell | That's pending tough | 12:55 |
sshnaidm|rover | quiquell, grafana has boxes too | 12:56 |
quiquell | Will check | 12:56 |
sshnaidm|rover | quiquell, with alerts, etc | 12:56 |
quiquell | Then we decide | 12:56 |
quiquell | I choose the simplest to just show it to you | 12:56 |
sshnaidm|rover | quiquell, looks great, I think we can do it more official | 12:57 |
quiquell | Let's play a little with it first | 12:57 |
quiquell | I have to move it to config-ci and do a prper ansible role to setup things | 12:57 |
sshnaidm|rover | quiquell, does it take data from your scripts? | 12:57 |
quiquell | I push the data from the scripts to tipbaord | 12:58 |
quiquell | every 5 minutes | 12:58 |
sshnaidm|rover | quiquell, great | 12:58 |
quiquell | I can update each box in parallel | 12:58 |
quiquell | Not done yet | 12:58 |
*** amoralej is now known as amoralej|lunch | 12:58 | |
sshnaidm|rover | quiquell, so take a look at grafana, because we use it in Ci already, need to create a dashboard, it would be nice to reuse it instead of new tool | 12:59 |
sshnaidm|rover | quiquell, all your scripts will be integrated without a problem | 12:59 |
quiquell | sshnaidm|rover: They just render data into pandas, it's just a matter of using it | 13:00 |
myoung|ruck | weshay: I saw your changes, makes sense. I had put things to Yellow overall due to the CIX issues, but from a focused osp13 status, agree with your edits to team status. It certainly made tripleo-ci go quickly on the call :) | 13:01 |
sshnaidm|rover | quiquell, I was thinking about Telegraf which sends to influxdb adn then rendered by grafana: https://docs.influxdata.com/telegraf/v1.6/concepts/data_formats_input/ | 13:01 |
*** Goneri has joined #oooq | 13:02 | |
weshay | myoung|ruck, the osp board determines the status | 13:03 |
weshay | color | 13:03 |
quiquell | myoung|ruck: Would be possible to do team meetings at 1 PM UTC instead of 2 PM UTC in general ? | 13:04 |
myoung|ruck | quiquell: the only problem with 1pm UTC is that for folks in mountain time (wes) that's a 7am meeting | 13:07 |
*** rasca has quit IRC | 13:08 | |
*** rasca has joined #oooq | 13:08 | |
myoung|ruck | weshay: re: boards and color, that's why I had indicated yellow, we have 2 rdo2 tiles that are yellow...including master. when I wrote the status it was broadly, and I was going to caveat and say "yellow due to rdo2 and CIX, green for beta" | 13:09 |
quiquell | Ok, I just mean one hour earlier | 13:09 |
weshay | myoung|ruck, which queens card on the board is yellow? | 13:09 |
myoung|ruck | ...hence..."green for beta, yellow for reasons above" When I wrote the status earlier I hadn't realized the meeting was narrowly focused on queens/13 beta | 13:10 |
myoung|ruck | I had assumed it was "beta/13 status, 14 planning" - so master mattered. Not trying to be pedantic here...I noticed the edits from my phone in the car. Thanks for streamlining the status :) | 13:11 |
*** ykarel_ is now known as ykarel | 13:13 | |
myoung|ruck | quiquell: re: UTC 1400 meetings not working, please bring it up in retrospective, or we can discuss at next standup (if it doesn't take too much time). AFAIK 1300 UTC is 7am for some team members, and the 1300-1400 block has been problematic for other team members as well | 13:13 |
quiquell | myoung|ruck: 7am is a bitch, no need for discussions, I totally understand | 13:14 |
*** yolanda has joined #oooq | 13:16 | |
quiquell | trown: Do you have brain cycle for me over bj ? | 13:16 |
myoung|ruck | sshnaidm|rover: have a moment to sync? | 13:17 |
sshnaidm|rover | myoung|ruck, yep | 13:17 |
myoung|ruck | cool...my place or yours? | 13:17 |
sshnaidm|rover | myoung|ruck, paste a link | 13:17 |
*** yolanda_ has quit IRC | 13:18 | |
myoung|ruck | sshnaidm|rover: https://bluejeans.com/7050859455 | 13:18 |
trown | quiquell: sure | 13:19 |
quiquell | cool, https://bluejeans.com/7891065232 | 13:19 |
*** saneax-_-|AFK is now known as saneax | 13:21 | |
rlandy | trown: weshay: we could use more design around the dry-run option. how we will trigger that. what it will spew out exactly | 13:22 |
rlandy | https://review.openstack.org/#/c/567060/ | 13:23 |
rlandy | started here | 13:23 |
*** amoralej|lunch is now known as amoralej | 13:31 | |
sshnaidm|rover | myoung|ruck, supybot.plugins.GateStatus.changeIDs | 13:33 |
*** myoung|ruck is now known as myoung | 13:36 | |
sshnaidm|rover | myoung, you don't hear me | 13:38 |
*** jaosorior has quit IRC | 13:43 | |
myoung | sshnaidm|rover: https://review.rdoproject.org/etherpad/p/ruckrover-sprint13 | 13:47 |
myoung | %gatestatus | 13:50 |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 13:50 |
*** jaosorior has joined #oooq | 13:50 | |
weshay | quiquell, myoung trown rlandy panda fyi.. panda and I have reworded and reordered the cards | 13:50 |
weshay | probably needs some discussion after your review | 13:51 |
*** ykarel is now known as ykarel|away | 13:53 | |
myoung | %config plugins.GateStatus.changeIDs | 13:53 |
hubbot | myoung: I9af1028314859b63cab9f55d2c906b1344ca2300 I214272a6f25feb75496e44eb0a16269c6ee4cfe2 I4c5bdf00ce8cf7eabf669b248b99cb8443e82fab If12c8fe9bd0bea98a4842f279399285344f22246 | 13:53 |
myoung | %config plugins.GateStatus.changeIDs | 13:55 |
hubbot | myoung: I9af1028314859b63cab9f55d2c906b1344ca2300 I214272a6f25feb75496e44eb0a16269c6ee4cfe2 I4c5bdf00ce8cf7eabf669b248b99cb8443e82fab If12c8fe9bd0bea98a4842f279399285344f22246 | 13:55 |
rlandy | updating review | 13:56 |
*** hubbot has quit IRC | 13:57 | |
*** hubbot has joined #oooq | 13:57 | |
myoung | %config plugins.GateStatus.changeIDs | 13:58 |
hubbot | myoung: I0cbf9ffb8552411e4dd891c38702ff8d1f6db5b1 I214272a6f25feb75496e44eb0a16269c6ee4cfe2 I4c5bdf00ce8cf7eabf669b248b99cb8443e82fab If12c8fe9bd0bea98a4842f279399285344f22246 | 13:58 |
myoung | sshnaidm|rover: \o/ ^^ | 13:58 |
*** myoung is now known as myoung|ruck | 13:58 | |
myoung|ruck | %gatestatus | 13:58 |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 13:58 |
weshay | myoung|ruck, ya.. fyi.. the program call is release specific so we need to target our status and summary to the release | 13:59 |
sshnaidm|rover | hrybacki, ping | 14:13 |
hrybacki | sshnaidm|rover: pong | 14:13 |
sshnaidm|rover | hrybacki, did you create a ovb stack on rdo cloud recently? | 14:13 |
hrybacki | sshnaidm|rover: not this week, no | 14:14 |
sshnaidm|rover | hrybacki, can you try it? because I can't.. | 14:14 |
hrybacki | sshnaidm|rover: sure, gimmige a few | 14:14 |
sshnaidm|rover | hrybacki, ops told me to use smaller flavors, but I'm not sure how it works with ovb case | 14:15 |
hrybacki | sshnaidm|rover: yeah capacity is an issue but this again raises my point of -- how are we supposed to test out true CI? | 14:15 |
quiquell | 00.. | 14:19 |
quiquell | ups | 14:20 |
weshay | myoung|ruck, sshnaidm|rover so is this fixed? https://bugs.launchpad.net/tripleo/+bug/1770021 | 14:22 |
openstack | Launchpad bug 1770021 in tripleo "hubbot: TQE check jobs have not been kicked by hubbot since 4/25" [Critical,Triaged] | 14:22 |
*** skramaja has quit IRC | 14:22 | |
weshay | asking because I see \0/ | 14:23 |
sshnaidm|rover | weshay, yep | 14:23 |
weshay | cool, thanks | 14:23 |
myoung|ruck | weshay: aye | 14:25 |
hrybacki | sshnaidm|rover: nope, I can't re-deploy either now | 14:25 |
hrybacki | are medium flavors large enough? | 14:25 |
sshnaidm|rover | hrybacki, maybe, need to check.. | 14:26 |
sshnaidm|rover | hrybacki, actually we need only stack created, no need to install there undercloud | 14:26 |
*** quiquell is now known as quiquell|off | 14:27 | |
myoung|ruck | sshnaidm|rover, weshay, I just updated the LP with the details of what we did, and what the bot is monitoring now. https://bugs.launchpad.net/tripleo/+bug/1770021/comments/1 | 14:29 |
openstack | Launchpad bug 1770021 in tripleo "hubbot: TQE check jobs have not been kicked by hubbot since 4/25" [Critical,Fix released] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 14:29 |
myoung|ruck | sshnaidm|rover: please update ^^ if I missed anything | 14:29 |
sshnaidm|rover | myoung|ruck, no, that's fine | 14:30 |
hrybacki | sshnaidm|rover: that's true (at this point anyways) | 14:34 |
hrybacki | medium sized flavors would at least get you far enough to figure out the key issue? | 14:34 |
sshnaidm|rover | hrybacki, I think so | 14:36 |
rlandy | weshay: panda: myoung|ruck: trown: could you pls review the description, DoD and see if this captires what we intend? | 14:37 |
rlandy | https://trello.com/c/Qnd1IJY1/753-create-a-new-upstream-zuul-job-that-executes-toci-dry-run-with-linter-and-unit-tests | 14:37 |
* myoung|ruck is CIX call but looks anyway | 14:38 | |
myoung|ruck | panda: do you care if I drop the "Maturity: " from the labels? All of your new labels start with "Needs", and the ":" character in label borks trello's filtering links on the URI (trello bug) | 14:39 |
trown | rlandy: card looks good, I dont see any reason to not run the job on openstack infra | 14:40 |
trown | rlandy: should be <10min job | 14:41 |
*** jaosorior has quit IRC | 14:41 | |
trown | rlandy: the patch is not exactly what I was thinking, but I will comment there | 14:42 |
*** apetrich has quit IRC | 14:43 | |
*** chandankumar is now known as chkumar|off | 14:45 | |
*** udesale has quit IRC | 14:46 | |
rlandy | trown: thanks - will review your comments | 14:49 |
*** apetrich has joined #oooq | 14:50 | |
*** moguimar has quit IRC | 14:50 | |
*** kopecmartin has quit IRC | 14:52 | |
myoung|ruck | rlandy: the card looks great to me. IMHO a minor addition to DoD would be "a README.md exists (somewhere) that describes how to run the linter/dry run tests locally" - but that's just 'imho' | 15:06 |
rlandy | myoung|ruck: sure - can add that | 15:06 |
panda | rlandy: you should skip collectlogs for the dry run | 15:07 |
rlandy | trown: thanks for your comments - wrt what the option kicking the dry run is named - let's bike shed a little here. I am fine changing the name but I am not sure about unit | 15:08 |
panda | myoung|ruck: Is it ok if we just change the : to - ? | 15:08 |
rlandy | skipping the playbook run does not essentially mean we run unit tests | 15:08 |
rlandy | panda: I was going to move the collected vars to the $LOGS fir to collect logs would then collect something | 15:09 |
myoung|ruck | panda: sure (caveat: for me personally visually parsing "(Maturity * 5)" for each and every card does not add value) | 15:09 |
rlandy | I'll updated the patch and pls comment again there if you want it still removed | 15:09 |
trown | rlandy: right but the toci_jobtype is what we are parsing there... it is not a cli option | 15:09 |
myoung|ruck | panda: "Needs $something" is more clear...but I'm not stuck on it. The broken links are the actual issue. | 15:09 |
trown | rlandy: so if we will run dryrun in a unit test job makes more sense to have the jobtype be named something not "dryrun" | 15:10 |
*** dtrainor has quit IRC | 15:10 | |
trown | rlandy: "dryrun" in a jobtype would make me think the job does nothing | 15:10 |
*** moguimar has joined #oooq | 15:10 | |
rlandy | trown: I don't know a better way to invoke this option w/o adding something to jobtype | 15:10 |
myoung|ruck | panda: if you update the labels to whatever-you-want-not-including-: in place they all autoupdate | 15:11 |
rlandy | also I am not sure how we plan to kick the unit tests | 15:11 |
panda | can we add a parameter to toci_quickstart.sh --dry-run ? | 15:11 |
trown | rlandy: ya I think going off jobtype makes sense, but just pointing out what jobtype will want that option is a unittest job | 15:12 |
panda | also the unit test don't necesarl=ily need to be invoked by any existing scripts | 15:12 |
rlandy | ^^ exactly | 15:12 |
rlandy | I am not sure linking dry run and unit tests is correct | 15:13 |
rlandy | question above | 15:13 |
rlandy | how do we plan on kicking the unit tests? | 15:13 |
rlandy | I can think of a ansible role to do it | 15:13 |
rlandy | then we wouls have a fs | 15:13 |
panda | with a call to tox ? maybe look at the other openstack-tox jobs ? | 15:14 |
rlandy | and I would understand the unit test naming | 15:14 |
rlandy | if we kick it indep, the two are unrelated | 15:14 |
trown | panda: we cant skip collectlogs if we want logs collected :P | 15:18 |
panda | what do we collect from dry runs ? | 15:19 |
rlandy | yep | 15:20 |
trown | panda: the log of the dry run | 15:20 |
rlandy | I don't have the dry run data all in yet | 15:20 |
trown | panda: if we dont collect anything, what is the point of running the job | 15:20 |
trown | panda: oh nevermind... if we put stuff in $LOGS_DIR we are good without running collect-logs | 15:22 |
panda | trown: how much data is the current collect will be used in a dry run | 15:22 |
panda | yep ok | 15:25 |
*** links has quit IRC | 15:25 | |
*** saneax is now known as saneax-_-|AFK | 15:26 | |
*** ykarel|away has quit IRC | 15:26 | |
rlandy | so we are still not agreed on the option name? | 15:26 |
*** saneax-_-|AFK is now known as saneax | 15:26 | |
* rlandy fixes other comments | 15:26 | |
trown | rlandy: I am fine with the option name, just not the jobtype (aka name) that uses that option | 15:27 |
*** saneax is now known as saneax-_-|AFK | 15:28 | |
rlandy | ok | 15:29 |
panda | trown: did you find a card to work on ? | 15:32 |
trown | panda: im working on python script with quiquell | 15:32 |
myoung|ruck | sshnaidm|rover: just saw this gate failure in the queue, and it seems odd to me...any idea what this might be? https://review.rdoproject.org/etherpad/p/ruckrover-sprint13 @ L56 | 15:32 |
trown | panda: do you have a card to work on? | 15:33 |
panda | trown: did you find a card to work on ?? | 15:33 |
panda | oops | 15:33 |
panda | trown: you're working on this one https://trello.com/c/oSW5OQer/732-implement-the-script-that-creates-releases-file ? | 15:34 |
trown | yes | 15:34 |
panda | trown: ok, I can work on the unit tests card | 15:34 |
panda | trown: do we have any agreement on the output ? | 15:35 |
rlandy | why is https://trello.com/c/nkEay4We/752-create-a-linter-script-for-the-tool in progress? | 15:35 |
trown | panda: ugh... there is now a seperate unit tests card? I thought we were doing tdd so the unit tests are what I am working on... as is quiquell | 15:35 |
panda | rlandy: bevcause we started from the in progress column, and added cards there, I'll move to backlog again | 15:36 |
panda | trown: ok | 15:36 |
rlandy | panda: ack - thanks | 15:36 |
rlandy | we need to see what is really in flight here | 15:36 |
panda | rlandy: the cards are also sorted by dependency, so how are you working on a job taht calls the linter if you don't know how to call it ? | 15:37 |
panda | maybe we created the cards after that one | 15:37 |
panda | we need to regroup | 15:37 |
panda | trown: rlandy 15 minutes meeting ? | 15:38 |
panda | no more, promise | 15:38 |
rlandy | ok | 15:38 |
rlandy | your bj? | 15:39 |
panda | yep | 15:39 |
trown | im a hard stop at 15min | 15:40 |
trown | meetings are getting way out of hand | 15:40 |
*** tosky has quit IRC | 15:48 | |
*** tosky has joined #oooq | 15:48 | |
*** ykarel|away has joined #oooq | 15:55 | |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades, tripleo-ci-centos-7-undercloud-oooq @ https://review.openstack.org/564291 | 15:57 |
myoung|ruck | sshnaidm|rover: fyi - adding rhos-13 jobs https://code.engineering.redhat.com/gerrit/#/c/138269 | 16:00 |
*** zoli is now known as zoli|gone | 16:02 | |
*** zoli|gone is now known as zoli | 16:02 | |
*** bogdando has quit IRC | 16:06 | |
*** dtrainor has joined #oooq | 16:13 | |
*** ykarel|away has quit IRC | 16:16 | |
*** panda is now known as panda|off | 16:18 | |
*** dtrainor has quit IRC | 16:21 | |
*** trown is now known as trown|lunch | 16:21 | |
*** dtrainor has joined #oooq | 16:24 | |
*** marios has quit IRC | 16:25 | |
*** dtrainor has quit IRC | 16:26 | |
*** hamzy has quit IRC | 16:28 | |
*** jfrancoa has quit IRC | 16:31 | |
*** udesale has joined #oooq | 16:32 | |
myoung|ruck | sshnaidm|rover: FYI first rhos-13 gate job is in flight: https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/tq-gate-rhos-13-ci-rhos-ovb-minimal-pacemaker-public-bond/1/console | 16:33 |
*** florianf has quit IRC | 16:34 | |
*** sshnaidm|rover is now known as sshnaidm|off | 16:36 | |
*** lucasagomes is now known as lucas-afk | 16:39 | |
*** jfrancoa has joined #oooq | 16:45 | |
weshay | rlandy, ok.. had time to review https://wiki.openstack.org/wiki/Tripleo-upgrades-fs-variables | 16:49 |
weshay | that does indeed look good | 16:49 |
weshay | panda|off, rlandy given the scope in the card, I agree that can be closed | 16:50 |
weshay | quiquell++ | 16:50 |
panda|off | we are missing updates | 16:52 |
weshay | true, | 16:52 |
*** atoth_ has joined #oooq | 16:55 | |
*** atoth has quit IRC | 16:57 | |
*** tosky has quit IRC | 16:57 | |
weshay | k.. panda|off thanks, I also updated where the job runs | 16:59 |
*** tosky has joined #oooq | 16:59 | |
rlandy | cool | 17:08 |
*** tesseract has quit IRC | 17:15 | |
weshay | myoung|ruck, https://thirdparty.logs.rdoproject.org/ appears to be down can please raise the issue w/ rdo-infra | 17:27 |
*** hamzy has joined #oooq | 17:27 | |
*** hubbot has quit IRC | 17:28 | |
myoung|ruck | weshay: was about to hit enter in #rdo, then saw your ping | 17:29 |
*** trown|lunch is now known as trown | 17:29 | |
*** dmellado has quit IRC | 17:29 | |
*** atoth has joined #oooq | 17:34 | |
*** atoth_ has quit IRC | 17:37 | |
*** gkadam_ has quit IRC | 17:43 | |
*** jtomasek has quit IRC | 17:44 | |
*** udesale has quit IRC | 17:50 | |
*** holser___ has quit IRC | 18:12 | |
*** jtomasek has joined #oooq | 18:22 | |
*** jfrancoa has quit IRC | 18:24 | |
weshay | rlandy, trown myoung|ruck fyi.. rdo services are down atm.. it will impact local reproducers etc | 18:32 |
weshay | myoung|ruck, sshnaidm|off expect it to impact promotion jobs as well | 18:33 |
weshay | conversation in #rhos-ops | 18:33 |
rlandy | thanks - looking | 18:37 |
*** d0ugal has quit IRC | 18:37 | |
*** d0ugal_ has joined #oooq | 18:37 | |
weshay | rlandy, re: the dry run.. here's a snip from a run from your review http://paste.openstack.org/show/720698/ | 18:43 |
weshay | is that roughly what you are getting as well? | 18:43 |
rlandy | looking | 18:46 |
rlandy | weshay: pretty much - you should see files in /tmp/logs | 18:47 |
rlandy | I have more playbook output | 18:48 |
rlandy | getting | 18:48 |
weshay | cool.. nice start :) | 18:49 |
*** dtantsur is now known as dtantsur|afk | 18:49 | |
rlandy | weshay: https://paste.fedoraproject.org/paste/av9xRieSq4H2DrQh1AF4hA | 18:49 |
rlandy | weshay: essentially people should not read the console output | 18:50 |
rlandy | they get the relevant info in the log files | 18:50 |
rlandy | that are put dir into the logs dir | 18:50 |
rlandy | so no collect logs anymore | 18:50 |
rlandy | if there are no playbook executed | 18:50 |
weshay | rlandy, what do you think about putting some thought into breaking up " $QUICKSTART_INSTALL_CMD" so that it's very readable | 18:50 |
weshay | like every param on each own line as a possibility | 18:51 |
weshay | not sure if you do, but I find it very difficult to parse our playbook commands | 18:51 |
weshay | ansible-playbook | 18:51 |
weshay | --e foo | 18:51 |
rlandy | weshay: any playbook command | 18:51 |
weshay | -e bar | 18:51 |
weshay | etc | 18:51 |
weshay | rlandy, ya | 18:51 |
* rlandy tests | 18:51 | |
rlandy | let's see what we can do | 18:51 |
weshay | awesome thanks | 18:52 |
rlandy | weshay: I also started a doc section | 18:52 |
rlandy | very prelim | 18:52 |
rlandy | but somewhere to start collecting info | 18:52 |
*** dtrainor has joined #oooq | 18:54 | |
*** dtrainor has quit IRC | 18:56 | |
*** dtrainor has joined #oooq | 18:56 | |
rlandy | trown: wrt https://review.openstack.org/#/c/567060/4/toci_quickstart.sh - if I indent that text going into the collect_logs.sh file will be indented. we ok with that? | 18:58 |
*** dmellado has joined #oooq | 18:58 | |
*** dtrainor has quit IRC | 18:59 | |
*** dtrainor has joined #oooq | 18:59 | |
trown | rlandy: hmm ... maybe it doesnt matter... I had trouble reading it at first because I thought you were missing "fi" ... but bash doesnt care about indentation | 18:59 |
rlandy | fine - ok | 18:59 |
trown | maybe we should move all that huge block of collect-logs stuff to a collect-logs function ... | 19:01 |
trown | probably not important for your patch though... | 19:01 |
*** hamzy has quit IRC | 19:01 | |
rlandy | I added the indents for now | 19:01 |
rlandy | let's see how we go | 19:02 |
rlandy | actually function is much cleaner - implementing | 19:04 |
*** hubbot has joined #oooq | 19:16 | |
*** hamzy has joined #oooq | 19:18 | |
*** agopi is now known as agopi|brb | 19:20 | |
*** links has joined #oooq | 19:23 | |
*** links has quit IRC | 19:45 | |
*** atoth has quit IRC | 19:46 | |
weshay | myoung|ruck, if you can reschedule the scrum tomorrow back 1/2 hour I can attend | 19:52 |
weshay | responded w/ no for now | 19:52 |
hubbot | FAILING CHECK JOBS on stable/ocata: tripleo-ci-centos-7-undercloud-upgrades @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-3nodes-multinode, tripleo-quickstart-extras-gate-newton-delorean-full-minimal @ https://review.openstack.org/560445 | 19:58 |
myoung|ruck | weshay: ack, moved back 30. a few folks have requested it as well, will raise at retro as well. I'll send out a doodle to the squads to see where folks would like to normally meet as well. | 20:07 |
*** matbu has quit IRC | 20:28 | |
rlandy | weshay: hello ... | 20:34 |
rlandy | weshay: question about where to add the unittest job | 20:35 |
weshay | rlandy, k | 20:35 |
weshay | rlandy, so you guys are free to do what you think is best | 20:35 |
weshay | but imho | 20:35 |
rlandy | weshay: so looking at https://github.com/openstack-infra/tripleo-ci/tree/master/zuul.d | 20:35 |
rlandy | we could run a multinode job | 20:35 |
weshay | the dry-run is for humans | 20:35 |
rlandy | named with unittest | 20:36 |
rlandy | maybe that is overkill | 20:36 |
weshay | the unit and lint run the tests | 20:36 |
weshay | seems like they could be one job | 20:36 |
myoung|ruck | weshay, rlandy, sshnaidm|off, so results from very first TQ gate job using rhos-13 are....*drumroll* | 20:36 |
myoung|ruck | ^^ Looks like we're hitting a bug weshay opened last year: https://bugs.launchpad.net/tripleo/+bug/1737356, a patch was posted (https://review.openstack.org/#/c/529548) which was -1 by slagel, and subsequently abandoned. | 20:37 |
openstack | Launchpad bug 1737356 in tripleo "Hostname validation for undercloud is too strict" [Medium,In progress] - Assigned to Martin André (mandre) | 20:37 |
weshay | undercloud install fail :) | 20:37 |
rlandy | hmmm | 20:37 |
rlandy | haven't looked at that in a while | 20:37 |
* myoung|ruck is attempting to understand why a non-FQDN is such a huge deal...not to mention why "subnode-0" works if that's the case | 20:38 | |
weshay | rlandy, so we generate some fake subunit data | 20:39 |
weshay | and we also generate some subunit data for the ping test and a few other things | 20:40 |
rlandy | looking at what we have for other units tests | 20:40 |
rlandy | adding a unittest-jobs.yaml | 20:40 |
weshay | the linting could probably just be tacked onto openstack-tox-linters | 20:40 |
weshay | rlandy, ya.. so if you think we should run the unit tests separately that is fine by me | 20:41 |
weshay | rlandy, one could also just run https://review.openstack.org/#/c/567320/1/scripts/emit-releases-file.py w/ pytest | 20:42 |
weshay | on a dryrun job that always runs too.. based on which files are changed etc. | 20:42 |
weshay | we could key off playbooks, toci.. | 20:42 |
weshay | pytest does exit 1, when it fails | 20:43 |
rlandy | I would kick the dry run job is anything toci* changed | 20:43 |
weshay | and if we generated subunit results they could also be added to elastic recheck.. sshnaidm|off could help there | 20:44 |
rlandy | yeah and maybe fs? | 20:44 |
weshay | fs000 ? | 20:44 |
weshay | lolz | 20:44 |
rlandy | fs-1? | 20:44 |
weshay | ya.. maybe we use a featureset for a dryrun | 20:44 |
weshay | we have a featureset001 | 20:44 |
rlandy | fs {-1} | 20:44 |
weshay | lolz | 20:45 |
rlandy | but that isn't actually what I meant | 20:45 |
rlandy | whatever, I'll put in something just to get comments | 20:45 |
rlandy | then I'll change it | 20:45 |
weshay | rlandy, would be interesting to depends on https://review.openstack.org/#/c/567320/1/scripts/emit-releases-file.py | 20:45 |
weshay | and execute w/ pytest | 20:45 |
weshay | or go w/ it's own job.. get input from others | 20:46 |
weshay | tbh I'm not sure what is "right" | 20:46 |
weshay | I'll be back on later tonight | 20:47 |
rlandy | ok - no rush | 20:47 |
rlandy | just very little design on it so I am thinking out aloud on irc | 20:48 |
*** holser__ has joined #oooq | 20:48 | |
weshay | myoung|ruck, this may also be related https://review.openstack.org/#/c/566911/ | 20:49 |
weshay | myoung|ruck, may want to ff after that lands | 20:49 |
*** Goneri has quit IRC | 20:50 | |
myoung|ruck | ack, i've made a note, 12 branches are 6 weeks old anyway, and 13 branches are 13 days old. will FF both after this lands | 20:55 |
*** trown is now known as trown|outtypewww | 21:00 | |
*** hamzy has quit IRC | 21:01 | |
*** holser__ has quit IRC | 21:11 | |
*** hamzy has joined #oooq | 21:11 | |
*** hubbot has quit IRC | 21:13 | |
*** dmellado has quit IRC | 21:15 | |
*** hamzy has quit IRC | 21:15 | |
myoung|ruck | fyi - we still appear to have ongoing intermittent networking issues with dlrn, ops is on it | 21:16 |
*** dmellado has joined #oooq | 21:23 | |
*** dmellado has quit IRC | 21:27 | |
myoung|ruck | %gatestatus | 21:29 |
*** hamzy_ has joined #oooq | 21:29 | |
*** hamzy_ is now known as hamzy | 21:30 | |
*** matbu has joined #oooq | 21:38 | |
*** hubbot has joined #oooq | 21:49 | |
*** myoung|ruck is now known as myoung|ruck|bbl | 21:59 | |
*** hubbot has quit IRC | 22:01 | |
*** hubbot has joined #oooq | 22:06 | |
*** jtomasek has quit IRC | 22:07 | |
*** rlandy has quit IRC | 22:09 | |
sshnaidm|off | myoung|ruck|bbl, weshay it's related to undercloud install: https://review.openstack.org/#/c/566911/ | 22:17 |
*** hamzy has quit IRC | 22:39 | |
*** myoung|ruck|bbl is now known as myoung|ruck|off | 22:55 | |
*** myoung|ruck|off is now known as myoung|ruck | 22:57 | |
*** tosky has quit IRC | 23:38 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!