| *** d0ugal has quit IRC | 00:08 | |
| *** d0ugal has joined #oooq | 00:12 | |
| weshay_ptg | 0/ | 00:42 |
|---|---|---|
| raukadah|ptg | \o | 01:52 |
| *** Goneri has quit IRC | 02:16 | |
| *** rfolco has quit IRC | 02:35 | |
| *** apetrich has quit IRC | 03:10 | |
| *** jpena|off has quit IRC | 03:27 | |
| *** jpena|off has joined #oooq | 03:28 | |
| *** bhagyashris has joined #oooq | 03:33 | |
| *** udesale has joined #oooq | 03:33 | |
| *** dsneddon has quit IRC | 03:44 | |
| *** rlandy|ruck|bbl has quit IRC | 03:56 | |
| *** rlandy|ruck|bbl has joined #oooq | 03:57 | |
| *** ykarel has joined #oooq | 03:59 | |
| *** rlandy|ruck|bbl is now known as rlandy|ruck | 04:00 | |
| *** dsneddon has joined #oooq | 04:08 | |
| *** udesale has quit IRC | 04:11 | |
| *** rlandy|ruck has quit IRC | 04:12 | |
| *** udesale has joined #oooq | 04:12 | |
| *** udesale has quit IRC | 04:17 | |
| *** udesale has joined #oooq | 04:18 | |
| *** udesale has quit IRC | 04:21 | |
| *** udesale has joined #oooq | 04:22 | |
| *** udesale has quit IRC | 04:26 | |
| *** dsneddon has quit IRC | 04:49 | |
| *** dsneddon has joined #oooq | 04:49 | |
| *** dsneddon has quit IRC | 05:02 | |
| *** dsneddon has joined #oooq | 05:05 | |
| *** dsneddon has quit IRC | 05:14 | |
| *** soniya29 has joined #oooq | 05:15 | |
| *** ykarel is now known as ykarel|afk | 05:19 | |
| *** surpatil has joined #oooq | 05:19 | |
| *** bhagyashris has quit IRC | 05:32 | |
| *** epoojad1 has joined #oooq | 05:33 | |
| *** ykarel|afk is now known as ykarel | 05:38 | |
| *** dsneddon has joined #oooq | 05:46 | |
| *** ratailor has joined #oooq | 05:55 | |
| *** marios has joined #oooq | 06:09 | |
| *** akahat has joined #oooq | 06:22 | |
| *** dsneddon has quit IRC | 06:40 | |
| *** bhagyashris has joined #oooq | 06:47 | |
| *** saneax has joined #oooq | 06:53 | |
| marios | anyone know whats up with linters there? https://review.opendev.org/#/c/655423/69 https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_721/655423/69/check/openstack-tox-linters/721492d/job-output.txt (but locally is fine http://paste.openstack.org/show/785873/ ) | 07:03 |
| *** dsneddon has joined #oooq | 07:04 | |
| *** dsneddon has quit IRC | 07:09 | |
| *** jfrancoa has joined #oooq | 07:11 | |
| *** dsneddon has joined #oooq | 07:11 | |
| *** dsneddon has quit IRC | 07:15 | |
| *** dsneddon has joined #oooq | 07:16 | |
| *** dsneddon has quit IRC | 07:21 | |
| *** dsneddon has joined #oooq | 07:28 | |
| *** dsneddon has quit IRC | 07:33 | |
| *** brault has joined #oooq | 07:33 | |
| *** soniya29 has quit IRC | 07:43 | |
| *** surpatil is now known as surpatil|lunch | 07:45 | |
| *** apetrich has joined #oooq | 07:48 | |
| *** tesseract has joined #oooq | 07:52 | |
| *** bhagyashris has quit IRC | 08:09 | |
| *** dsneddon has joined #oooq | 08:13 | |
| *** tosky has joined #oooq | 08:19 | |
| *** dsneddon has quit IRC | 08:21 | |
| ykarel | marios, ^^ failing for me locally | 08:27 |
| ykarel | after rebasing over master | 08:27 |
| marios | ykarel: ah ok thanks.. leme try | 08:29 |
| marios | ykarel: so looks like valid linting issue then adding comment thanks | 08:33 |
| marios | sshnaidm|afk: o/ remove +A https://review.opendev.org/#/c/655423/69 fyi ssbarnea did you merge somethign to tqe linting? see my last comments at https://review.opendev.org/#/c/655423/69 | 08:34 |
| *** jpena|off is now known as jpena | 08:38 | |
| *** amoralej|off is now known as amoralej | 08:39 | |
| *** bhagyashris has joined #oooq | 08:47 | |
| *** chem|eod is now known as chem | 08:51 | |
| *** dsneddon has joined #oooq | 08:52 | |
| *** bhagyashris_ has joined #oooq | 08:52 | |
| arxcruz | ykarel: around? | 08:52 |
| arxcruz | ykarel: need your help to test ceph-ansible | 08:52 |
| arxcruz | ykarel: https://review.rdoproject.org/r/#/c/23619/ what am i doing wrong here? | 08:53 |
| arxcruz | ssbarnea: for god sake, is zbr or ssbarnea now? | 08:53 |
| ykarel | arxcruz, hi | 08:54 |
| *** bhagyashris has quit IRC | 08:54 | |
| ykarel | arxcruz, in depends-on job that job has condition for branches | 08:54 |
| ykarel | branches: stable-4.0 | 08:54 |
| ykarel | and u running in master branch | 08:55 |
| ykarel | u can try overriding branches in ur test patch | 08:55 |
| ykarel | and it should attempt to run | 08:55 |
| arxcruz | ykarel: with override-branches: master ? | 08:55 |
| ykarel | arxcruz, nope | 08:56 |
| ykarel | branches: | 08:56 |
| ykarel | branches: stable-4.0 | 08:56 |
| *** tosky_ has joined #oooq | 08:56 | |
| ykarel | arxcruz, branches: master | 08:56 |
| *** dsneddon has quit IRC | 08:57 | |
| arxcruz | ykarel: didn't work.. | 08:58 |
| ykarel | arxcruz, /me look ur patch | 08:58 |
| arxcruz | ykarel: ok, thanks | 08:59 |
| *** tosky is now known as Guest25059 | 08:59 | |
| *** tosky_ is now known as tosky | 08:59 | |
| *** bhagyashris_ is now known as bhagyashris | 08:59 | |
| *** Guest25059 has quit IRC | 08:59 | |
| ykarel | arxcruz, hmm branches can not be overridden like this https://zuul-ci.org/docs/zuul/user/config.html#attr-job.branches | 09:09 |
| ykarel | jpena, would it be possible to add stable-4.0 to https://review.rdoproject.org/r/#/admin/projects/testproject,branches | 09:09 |
| ykarel | for testing new jobs with branches: stable-4.0 | 09:10 |
| *** zbr has joined #oooq | 09:11 | |
| zbr | hi! | 09:11 |
| jpena | ykarel: let me try, I think it's doable | 09:12 |
| ykarel | jpena, Thanks | 09:12 |
| *** sshnaidm|afk is now known as sshnaidm|rover | 09:12 | |
| arxcruz | jpena: ykarel thanks! | 09:13 |
| ykarel | arxcruz, there are other ways we can try like adding new jobs in test project with branches: master | 09:13 |
| ykarel | but if stable-4.0 branch is available it would be more easier | 09:13 |
| jpena | https://review.rdoproject.org/r/gitweb?p=testproject.git;a=shortlog;h=refs%2Fheads%2Fstable-4.0 | 09:13 |
| arxcruz | ykarel: the problem with this is ceph-ansible rpm build script, uses git describe --tags which is wrong | 09:13 |
| arxcruz | ykarel: it's pointing to 4.0.0 instead of whatever the highest version is | 09:14 |
| ykarel | arxcruz, okk, stable-4.0 is created, send similar patch to that branch | 09:14 |
| *** arxcruz has left #oooq | 09:14 | |
| sshnaidm|rover | marios, done, I think linters just started to check all files | 09:19 |
| marios | sshnaidm|rover: ack i vaguely remember zbr merged something to do with that check (space around {{ }} ) | 09:19 |
| *** arxcruz has joined #oooq | 09:22 | |
| *** ChanServ sets mode: +o arxcruz | 09:22 | |
| *** surpatil|lunch is now known as surpatil | 09:28 | |
| arxcruz | ykarel: didn't work https://review.rdoproject.org/r/#/c/23624/ | 09:31 |
| ykarel | :( | 09:31 |
| ykarel | looking | 09:31 |
| arxcruz | ykarel: nevermind, my bad | 09:32 |
| arxcruz | ykarel: hmm, the rpm is building, but the standalone job no | 09:34 |
| ykarel | arxcruz, okk standalone should run next | 09:34 |
| ykarel | i see u have parent child dep | 09:34 |
| ykarel | mmm it's not in queue so there is some issue | 09:35 |
| arxcruz | ykarel: exactly, it's not in the queue | 09:35 |
| *** epoojad1 has quit IRC | 09:37 | |
| ykarel | hmm looking | 09:37 |
| *** dtantsur|afk is now known as dtantsur | 09:37 | |
| *** derekh has joined #oooq | 09:38 | |
| arxcruz | ykarel: at least the tag is correct, 4.0.5 :) | 09:41 |
| marios | jpena: o/ was it you that was/is working on "components" support for dlrn? /me working there https://tree.taiga.io/project/tripleo-ci-board/task/1378 ... at some point we'll have to report to dlrn about that job and i think it must be a new components url to report to? (or something ;) )... do you have anything to point at to get me started? taiga? topic branch etc? | 09:52 |
| marios | jpena: (if it isn't you sorry do you know who it is?) | 09:52 |
| ykarel | arxcruz, ack, from job side nothing looks wrong to me, it should have executed | 09:52 |
| *** dsneddon has joined #oooq | 09:52 | |
| arxcruz | ykarel: maybe because of https://review.rdoproject.org/r/#/c/23386/25/zuul.d/ceph-ansible.yaml@63 ? | 09:53 |
| ykarel | arxcruz, same was with other job, but that ran fine | 09:53 |
| arxcruz | jpena: can you help me here? | 09:53 |
| jpena | marios: yes, it's me. All component-based changes are grouped in https://softwarefactory-project.io/r/#/q/branch:master+topic:component-based-dlrn | 09:53 |
| marios | jpena: thanks looking | 09:54 |
| arxcruz | jpena: https://review.rdoproject.org/r/#/c/23624/ the first job ran, but the dependency one doesn't | 09:54 |
| jpena | marios: TL;DR: there is no separate API endpoint, the API deduces which component is associated during promotions from the commit hash | 09:54 |
| ykarel | arxcruz, looks like dependenices do not work well with branches | 09:54 |
| marios | jpena: ok nice thanks... mostly lookign to confirm who the contact is... we aren't far enough yet but will likely need to talk about some point :D | 09:55 |
| jpena | marios: but when querying, you can specify the component to filter results out | 09:55 |
| jpena | marios: sure :) | 09:55 |
| jpena | arxcruz: reading back | 09:55 |
| *** epoojad1 has joined #oooq | 09:57 | |
| *** dsneddon has quit IRC | 09:57 | |
| jpena | I'm going to check the zuul logs, to see if they shed some light | 09:57 |
| ykarel | ack | 09:58 |
| arxcruz | jpena: ack, thanks! | 09:58 |
| jpena | arxcruz: hm, I found this in the logs: "No matching parents for job tripleo-ci-rhel-8-standalone-rdo and change <Change 0x7f98ddd7d710 testproject 23624,2>" | 10:07 |
| jpena | I see the tripleo-ci-rhel-8-standalone-rdo job definition has "branches: ^master$". I would expect the child definition to override that, but based on the log it looks like it doesn't | 10:08 |
| jpena | for the tests, could you comment out that line from https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/standalone-jobs.yaml#L480 ? I see the child jobs already specify the branch | 10:09 |
| ykarel | hmm if parent has branches: then ^^ is expected | 10:11 |
| ykarel | i have seen it earlier too | 10:11 |
| *** dsneddon has joined #oooq | 10:29 | |
| *** ykarel is now known as ykarel|afk | 10:33 | |
| arxcruz | jpena: ack | 10:33 |
| *** dsneddon has quit IRC | 10:35 | |
| arxcruz | jpena: didn't work commenting the branches on tripleo-ci-rhel-8-standalone-rdo | 10:36 |
| arxcruz | the rpm build job is running, but not the standalone job | 10:37 |
| ykarel|afk | arxcruz, u commented wrong place | 10:37 |
| arxcruz | ykarel|afk: of course I did | 10:38 |
| arxcruz | xD | 10:38 |
| raukadah|ptg | arxcruz: jpena does https://review.rdoproject.org/r/#/c/23386/ the tag got fixed? | 10:38 |
| jpena | raukadah|ptg: when switching the branch to use for the tests to stable-4.0, yes | 10:38 |
| arxcruz | raukadah|ptg: you mean the master pointing to a different than 4.0.0? nope | 10:39 |
| jpena | we're still discussing the tagging in master | 10:39 |
| raukadah|ptg | jpena: arxcruz I was talking with gfindente he told it got fixed, if not then he will do a followup | 10:39 |
| arxcruz | raukadah|ptg: i did not check recently | 10:39 |
| raukadah|ptg | jpena: arxcruz ack | 10:40 |
| arxcruz | raukadah|ptg: last email was just, yeah, we should change it basically | 10:40 |
| arxcruz | but I see no action | 10:40 |
| jpena | arxcruz: aha, now the standalone job is properly queued | 10:40 |
| ykarel|afk | [o/ | 10:40 |
| arxcruz | jpena: okay, but how can we fix it? I don't think we should remove the branches from the parent job... | 10:41 |
| raukadah|ptg | \o] | 10:41 |
| jpena | arxcruz: I think a parent job should not restrict branches, child jobs should (and this is what they do already) | 10:41 |
| arxcruz | jpena: indeed, so i believe it won't hurt... | 10:42 |
| arxcruz | let's wait the results first :) | 10:42 |
| jpena | yep, let's fix issues one by one | 10:42 |
| raukadah|ptg | zbr: panda if free, please take a look at zuul job behaviour on podman prs as it got merged https://github.com/openshift/release/pull/5668 | 10:43 |
| *** jaosorior has joined #oooq | 10:56 | |
| *** dsneddon has joined #oooq | 11:02 | |
| *** ykarel|afk is now known as ykarel | 11:03 | |
| *** rfolco has joined #oooq | 11:07 | |
| amoralej | i've cleaned a bunch of old jobs in ci.centos.org today. I'd like you to take a look to the ones in https://ci.centos.org/view/rdo/view/tripleo-periodic/ | 11:17 |
| amoralej | and https://ci.centos.org/view/rdo/view/tripleo-gate/ | 11:17 |
| amoralej | those still have recent runs | 11:18 |
| amoralej | although i think they are just leftovers | 11:18 |
| amoralej | also i think we can remove the view https://ci.centos.org/view/rdo/view/POC/ | 11:18 |
| amoralej | unless there is any job to add there | 11:18 |
| rfolco | panda, do you have some doc or drawing on how promotion works today vs component ? If not I can try something. | 11:23 |
| *** saneax has quit IRC | 11:31 | |
| *** ratailor has quit IRC | 11:36 | |
| panda | rfolco: it would be nice as an excercise | 11:38 |
| panda | rfolco: for you to demo your understanding, but I can help | 11:38 |
| rfolco | k cool | 11:38 |
| panda | rfolco: just swrote the description on the card | 11:39 |
| panda | rfolco: no drawing | 11:39 |
| rfolco | panda, excellent | 11:39 |
| *** rfolco is now known as rfolco|doctor | 11:57 | |
| *** dsneddon has quit IRC | 11:58 | |
| *** jpena is now known as jpena|lunch | 12:08 | |
| *** amoralej is now known as amoralej|lunch | 12:09 | |
| *** ykarel is now known as ykarel|afk | 12:18 | |
| *** ykarel|afk has quit IRC | 12:28 | |
| *** ykarel has joined #oooq | 12:29 | |
| *** dsneddon has joined #oooq | 12:30 | |
| *** saneax has joined #oooq | 12:31 | |
| *** bhagyashris_ has joined #oooq | 12:34 | |
| *** bhagyashris has quit IRC | 12:36 | |
| *** ykarel_ has joined #oooq | 12:37 | |
| sshnaidm|rover | marios, rfolco|doctor did you hit "no space left" issue on promoter? | 12:38 |
| *** ykarel__ has joined #oooq | 12:39 | |
| *** ykarel has quit IRC | 12:39 | |
| *** rfolco|doctor is now known as rfolco | 12:41 | |
| rfolco | sshnaidm|rover, I didn't notice it | 12:41 |
| *** ykarel_ has quit IRC | 12:42 | |
| *** epoojad1 has quit IRC | 12:52 | |
| *** rlandy has joined #oooq | 12:53 | |
| *** rlandy is now known as rlandy|ruck | 12:54 | |
| rlandy|ruck | sshnaidm|rover: hey | 12:54 |
| *** ykarel__ is now known as ykarel|afk | 12:54 | |
| sshnaidm|rover | rlandy|ruck, hi | 12:54 |
| rlandy|ruck | sshnaidm|rover: did you do anything to help rocky/train to promote? | 12:54 |
| rlandy|ruck | or did that just work on its own last night? | 12:55 |
| * rlandy|ruck looks at stein | 12:55 | |
| sshnaidm|rover | rlandy|ruck, just worked I think | 12:55 |
| sshnaidm|rover | rlandy|ruck, stein jobs are good, but promoter for stein crashed | 12:55 |
| rlandy|ruck | sshnaidm|rover: yay - I was thinking the image server might be broken but must be ok then | 12:55 |
| ykarel|afk | rocky should have promoted by the testproject run done by rlandy|ruck | 12:55 |
| sshnaidm|rover | rlandy|ruck, http://38.145.34.55/centos7_stein.log | 12:55 |
| rlandy|ruck | sshnaidm|rover: ^^ ah - the same thing happened with rocky | 12:55 |
| ykarel|afk | i commented today morning in the test project about the potential issue | 12:55 |
| rlandy|ruck | different issue | 12:56 |
| rlandy|ruck | rocky said image did not exist | 12:56 |
| rlandy|ruck | stein is a container crash | 12:56 |
| * rlandy|ruck watches that | 12:57 | |
| rlandy|ruck | to see if we have an issue on the image server | 12:57 |
| rlandy|ruck | stein should have promoted yesterday as well | 12:57 |
| rlandy|ruck | the container issue I showed weshay_ptg last week - he didn't seem too concerned | 12:58 |
| rlandy|ruck | the image issues as well actually :) | 12:59 |
| * rlandy|ruck looks for master fix | 12:59 | |
| marios | sshnaidm|rover: no didn't see that | 12:59 |
| panda | rlandy|ruck: zbr sshnaidm|rover marios arxcruz kopecmartin|sick raukadah|ptg weshay_ptg ping scrum meeting | 13:00 |
| sshnaidm|rover | rlandy|ruck, stein script failure shows "no space left on device", but there is plenty of place on promoter server | 13:01 |
| rlandy|ruck | sshnaidm|rover: the image server | 13:01 |
| sshnaidm|rover | rlandy|ruck, ? | 13:01 |
| rlandy|ruck | we need to check that one ^^ | 13:01 |
| * rlandy|ruck gets | 13:02 | |
| sshnaidm|rover | rlandy|ruck, why image server? | 13:02 |
| sshnaidm|rover | it's containers | 13:02 |
| rlandy|ruck | checking where that shows up | 13:02 |
| *** akahat has quit IRC | 13:03 | |
| rlandy|ruck | oh there | 13:03 |
| rlandy|ruck | wonder if that showed up yesterday | 13:05 |
| rlandy|ruck | sshnaidm|rover: it's running again now | 13:06 |
| rlandy|ruck | maybe a temp hitch? | 13:06 |
| sshnaidm|rover | rlandy|ruck, we'll see | 13:06 |
| *** ykarel_ has joined #oooq | 13:06 | |
| rlandy|ruck | sshnaidm|rover: I saw that error before with image server | 13:06 |
| rlandy|ruck | first time with containers | 13:07 |
| rlandy|ruck | clean up should be immediate | 13:07 |
| sshnaidm|rover | /dev/vdb1 80G 76G 4.4G 95% /var/lib/docker | 13:08 |
| sshnaidm|rover | seems like really no space.. | 13:08 |
| *** ykarel|afk has quit IRC | 13:09 | |
| *** ykarel has joined #oooq | 13:09 | |
| sshnaidm|rover | shouldn't it clear containers after promotion? | 13:10 |
| *** jpena|lunch is now known as jpena | 13:11 | |
| *** ykarel_ has quit IRC | 13:13 | |
| jpena | arxcruz: I see the job passed for ceph-ansible, but I can't see the new yum repo in http://logs.rdoproject.org/24/23624/2/check/tripleo-ceph-integration-rhel-8-standalone-featureset016/899c82c/ | 13:15 |
| arxcruz | jpena: yeah | 13:15 |
| arxcruz | i'm checking why | 13:15 |
| jpena | ack | 13:15 |
| *** derekh has quit IRC | 13:16 | |
| sshnaidm|rover | rlandy|ruck, ok, removing all that a few weeks ago there | 13:17 |
| rlandy|ruck | sshnaidm|rover: thanks | 13:17 |
| rlandy|ruck | sshnaidm|rover: I think containers should clear - unless the promoter dies in the middle | 13:18 |
| sshnaidm|rover | rlandy|ruck, there're containers that are 4-6 weeks old | 13:18 |
| sshnaidm|rover | rlandy|ruck, no idea why.. | 13:18 |
| rlandy|ruck | old/new promoter? | 13:18 |
| rlandy|ruck | are those a once off | 13:18 |
| rlandy|ruck | or we have them building up? | 13:19 |
| sshnaidm|rover | rlandy|ruck, idk | 13:19 |
| sshnaidm|rover | and removing all I don't know | 13:19 |
| sshnaidm|rover | if I don't understand something, I delete it | 13:20 |
| rlandy|ruck | lol | 13:20 |
| sshnaidm|rover | rlandy|ruck, but it may affect current stein promoter run, so maybe it'll pass in next one | 13:21 |
| rlandy|ruck | sshnaidm|rover: iow - if there were left over containers for a periodic of time - and then no longer, possible new promoter/old promoter issue - if we have them building up from yseteday also, we have a bigger problem | 13:22 |
| rlandy|ruck | sshnaidm|rover: not a problem - stein will run again soon | 13:22 |
| rlandy|ruck | nothing else is up for promotion | 13:22 |
| sshnaidm|rover | rlandy|ruck, amoralej|lunch fixed rhel8 containers also | 13:22 |
| rlandy|ruck | amoralej|lunch++ | 13:22 |
| rlandy|ruck | great | 13:23 |
| sshnaidm|rover | rlandy|ruck, so only this occasional build image failure left | 13:23 |
| rlandy|ruck | sshnaidm|rover: this promoter is a whole lot of black magic | 13:23 |
| sshnaidm|rover | rlandy|ruck, yeah | 13:23 |
| panda | Sorry my connection dropped | 13:24 |
| *** jaosorior has quit IRC | 13:24 | |
| rlandy|ruck | sshnaidm|rover: I'm closing https://bugs.launchpad.net/tripleo/+bug/1850188 out | 13:28 |
| openstack | Launchpad bug 1850188 in tripleo "periodic container build (train) timing out" [Critical,Triaged] | 13:28 |
| *** bhagyashris__ has joined #oooq | 13:28 | |
| sshnaidm|rover | rlandy|ruck, great | 13:28 |
| *** amoralej|lunch is now known as amoralej | 13:29 | |
| *** bhagyashris_ has quit IRC | 13:30 | |
| marios | https://review.opendev.org/#/c/675313/ i will +A that | 13:36 |
| marios | any objections | 13:36 |
| rlandy|ruck | ha - master didn't catch the heat change yet | 13:36 |
| sshnaidm|rover | rlandy|ruck, is it heat change or nova? | 13:37 |
| rlandy|ruck | heat | 13:38 |
| rlandy|ruck | it merged at 10 pm last night | 13:38 |
| sshnaidm|rover | rlandy|ruck, let's see if it's in consistent repo | 13:38 |
| rlandy|ruck | I testproject tested it yesterday with depends on | 13:38 |
| rlandy|ruck | and it works | 13:38 |
| rlandy|ruck | so we can see if we catch it next time | 13:38 |
| rlandy|ruck | ok ... | 13:38 |
| rlandy|ruck | so we still have a rhel scenario4 failure | 13:39 |
| sshnaidm|rover | https://trunk.rdoproject.org/centos7-train/report.html | 13:39 |
| sshnaidm|rover | sorry, https://trunk.rdoproject.org/centos7-master/report.html | 13:40 |
| rlandy|ruck | 2019-11-07 12:48:06 | "Error: Failed to apply catalog: Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe SELECT CONCAT(User, '@',Host) AS User FROM mysql.user' returned 1: ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)", | 13:40 |
| *** bhagyashris__ has quit IRC | 13:40 | |
| *** surpatil has quit IRC | 13:40 | |
| sshnaidm|rover | rlandy|ruck, it can be different issues | 13:41 |
| rlandy|ruck | looking more | 13:41 |
| sshnaidm|rover | rlandy|ruck, which commit id of this heat change? | 13:41 |
| rlandy|ruck | https://review.opendev.org/#/c/693165/ | 13:42 |
| sshnaidm|rover | rlandy|ruck, it's there: https://trunk.rdoproject.org/centos7-master/report.html | 13:42 |
| sshnaidm|rover | rlandy|ruck, it should have been picked up | 13:43 |
| rlandy|ruck | it should work - we can see again on next run | 13:43 |
| rlandy|ruck | it does work | 13:43 |
| rlandy|ruck | I tested it before | 13:43 |
| * rlandy|ruck looks at scenario4 errors atm | 13:43 | |
| sshnaidm|rover | rlandy|ruck, ok, I see jobs didn't start yet | 13:45 |
| *** saneax has quit IRC | 13:46 | |
| *** saneax has joined #oooq | 13:47 | |
| *** saneax has quit IRC | 13:48 | |
| *** Goneri has joined #oooq | 13:52 | |
| *** dsneddon has quit IRC | 13:56 | |
| rlandy|ruck | 2019-11-07 08:23:04.014247 | primary | 2019-11-07 08:23:03 | | 13:57 |
| rlandy|ruck | 2019-11-07 08:23:04.014251 | primary | 2019-11-07 08:23:03 | The downloaded packages were saved in cache until the next successful transaction. | 13:57 |
| rlandy|ruck | 2019-11-07 08:23:04.014256 | primary | 2019-11-07 08:23:03 | You can remove cached packages by executing 'dnf clean packages'. | 13:57 |
| rlandy|ruck | 2019-11-07 08:23:04.014260 | primary | 2019-11-07 08:23:03 | Error: Error downloading packages: | 13:57 |
| rlandy|ruck | 2019-11-07 08:23:04.014265 | primary | 2019-11-07 08:23:03 | Cannot download createrepo_c-libs-0.11.0-3.el8.x86_64.rpm: All mirrors were tried | 13:57 |
| rlandy|ruck | 2019-11-07 08:23:04.014269 | primary | | 13:57 |
| panda | jpena: do yuo have any plans to make delorean-deps a component ? (lightninh thought, not sure if it would make sense) | 13:58 |
| jpena | panda: not for now | 13:58 |
| panda | jpena: ok | 13:59 |
| jpena | it's not built by DLRN, so it wouldn't fit easily | 13:59 |
| panda | jpena: yeah I susppected that would be the reason. | 13:59 |
| *** saneax has joined #oooq | 14:00 | |
| *** dmellado has quit IRC | 14:01 | |
| *** gouthamr has quit IRC | 14:02 | |
| *** gouthamr has joined #oooq | 14:04 | |
| *** dmellado has joined #oooq | 14:06 | |
| *** derekh has joined #oooq | 14:23 | |
| *** dsneddon has joined #oooq | 14:30 | |
| rlandy|ruck | marios: hi - have you worked with the overcloud image build at all? | 14:31 |
| marios | rlandy|ruck: not really at least not for a very long time ... | 14:32 |
| marios | rlandy|ruck: whats the problem maybe i can help tho | 14:32 |
| rlandy|ruck | marios: we've been looking at https://bugs.launchpad.net/tripleo/+bug/1851274 | 14:34 |
| openstack | Launchpad bug 1851274 in tripleo "periodic-tripleo-rhel-8-buildimage-overcloud-full-master is failing - failed to glob pattern /etc/rc0.d/[SK][0-9][0-9]network: No such file or directory" [Critical,In progress] - Assigned to Luke Short (ekultails) | 14:34 |
| rlandy|ruck | ipa image build never fails | 14:34 |
| rlandy|ruck | train overcloud build never fails | 14:34 |
| rlandy|ruck | just master | 14:34 |
| rlandy|ruck | just overcloud | 14:34 |
| rlandy|ruck | just sometimes | 14:34 |
| rlandy|ruck | something is hitting master and not train | 14:35 |
| marios | :/ rlandy|ruck oh :D (reading the bug ... but i mean the last bit was the killer 'just sometimes' | 14:35 |
| rlandy|ruck | marios: isn't it always | 14:35 |
| rlandy|ruck | so I think we have a race condition | 14:35 |
| rlandy|ruck | but it doesn't show up in train | 14:36 |
| rlandy|ruck | if you haven't worked here recently - no worries | 14:36 |
| marios | rlandy|ruck: ack sorry can't think of something... i guess the added debug statement didn't help or we wouldn't be having this conversation... | 14:36 |
| rlandy|ruck | not really | 14:36 |
| rlandy|ruck | will ping ianw again | 14:37 |
| marios | rlandy|ruck: only rhel or also centos? i just see rhel there | 14:37 |
| marios | (in the bug) | 14:37 |
| rlandy|ruck | only rhel | 14:37 |
| *** derekh has quit IRC | 14:47 | |
| *** ykarel is now known as ykarel|afk | 14:49 | |
| amoralej | sshnaidm|rover, in my environment with rhel8 and train repos, nova-libvirt packages install fine | 14:52 |
| amoralej | let's see what ci says with those changes | 14:52 |
| sshnaidm|rover | amoralej, ack | 14:54 |
| *** bogdando has quit IRC | 14:59 | |
| *** bogdando has joined #oooq | 15:00 | |
| *** TrevorV has joined #oooq | 15:09 | |
| *** ykarel|afk is now known as ykarel | 15:14 | |
| sshnaidm|rover | rlandy|ruck, stein crashed with 500 Server Error: Internal Server Error in https://trunk.registry.rdoproject.org | 15:25 |
| sshnaidm|rover | rlandy|ruck, hope next time will be ok | 15:26 |
| rlandy|ruck | sshnaidm|rover: we should probably tell #sf-ops | 15:26 |
| rlandy|ruck | if next run doen't work, we have a real issue here | 15:27 |
| rlandy|ruck | sshnaidm|rover: the image build job has a new error | 15:28 |
| rlandy|ruck | in your test run ... | 15:28 |
| rlandy|ruck | 2019-11-07 14:47:42.093 | + /tmp/dib_build.uACCfuII/hooks/extra-data.d/99-yum-repo-conf:main:18 : echo '/etc/yum.repos.d/rhel8-master-deps.repo is not a valid yum repo configuration file.' | 15:28 |
| rlandy|ruck | 2019-11-07 14:47:42.093 | /etc/yum.repos.d/rhel8-master-deps.repo is not a valid yum repo configuration file. | 15:28 |
| rlandy|ruck | 2019-11-07 14:47:42.095 | + /tmp/dib_build.uACCfuII/hooks/extra-data.d/99-yum-repo-conf:main:19 : echo 'You should assign a list of proper yum repo configuration' | 15:28 |
| sshnaidm|rover | rlandy|ruck, hmm | 15:28 |
| rlandy|ruck | expecting that^^? | 15:29 |
| * rlandy|ruck needs to kick another test run | 15:29 | |
| rlandy|ruck | anyways | 15:29 |
| rlandy|ruck | http://logs.rdoproject.org/42/23542/7/check/periodic-tripleo-rhel-8-buildimage-overcloud-full-master/7b0ba27/overcloud-full.log | 15:30 |
| rlandy|ruck | same in my test run | 15:30 |
| sshnaidm|rover | rlandy|ruck, we just renamed this repo file http://logs.rdoproject.org/42/23542/7/check/periodic-tripleo-rhel-8-buildimage-overcloud-full-master/7b0ba27/yum.repos.d/delorean-rhel8-master-deps.repo | 15:31 |
| sshnaidm|rover | rlandy|ruck, is rhel8-master-deps.repo name hardcoded somewhere? | 15:31 |
| sshnaidm|rover | rlandy|ruck, seems like we solve techdebts and remove hacks for a whole release cycle this week.. | 15:33 |
| rlandy|ruck | sshnaidm|rover: ack | 15:33 |
| rlandy|ruck | maybe release file | 15:33 |
| rlandy|ruck | looking | 15:33 |
| rlandy|ruck | sshnaidm|rover: not in release file ... | 15:36 |
| rlandy|ruck | checking | 15:36 |
| sshnaidm|rover | rlandy|ruck, it's DIB_YUM_REPO_CONF set somewhere | 15:36 |
| rlandy|ruck | extras | 15:36 |
| *** jpena is now known as jpena|off | 15:37 | |
| sshnaidm|rover | rlandy|ruck, found: https://github.com/openstack/tripleo-ci/blob/master/roles/oooci-build-images/tasks/pre.yaml#L13:L13 | 15:38 |
| zbr | guys, i need to consult with you regarding an issue related to pre-commit project, who can join me in a quick call at https://meet.google.com/qpw-fsaw-yru ? | 15:45 |
| zbr | marios: rfolco i think you are ones aware about the recent failures. | 15:48 |
| sshnaidm|rover | rlandy|ruck, would be nice to have also train buildimage jobs running on tripleo-ci | 15:54 |
| sshnaidm|rover | rlandy|ruck, so we cover all | 15:55 |
| rlandy|ruck | sshnaidm|rover: yep | 15:56 |
| rlandy|ruck | no problem with that | 15:56 |
| marios | zbr: can we setup a call tomorrow? | 15:56 |
| zbr | marios: we could but i wanted to start addressing the issues while it was hot, read https://github.com/pre-commit/pre-commit/issues/1213 | 15:57 |
| zbr | mainly we have being shown the finger, and that has implications as we already use it in lots of repos. i kinda find myself guilty now as I was like no1 promoter. | 15:59 |
| zbr | i hope that with some resonable feedback we would reconsider his position and accept to support centos-7 as a platform. | 15:59 |
| *** dsneddon has quit IRC | 16:01 | |
| marios | zbr: which bit (reconsider) about the ci job for centos pre-commit? | 16:01 |
| rlandy|ruck | oh dear our gate queue | 16:02 |
| rlandy|ruck | growing | 16:02 |
| zbr | marios: i know that I can reconfigure tox-linters to run on centos-8, but i cannot force people to drop using c7 locally. | 16:02 |
| panda | we should all use c4 | 16:03 |
| *** Trevor_V has joined #oooq | 16:03 | |
| zbr | panda: c8 = c4*2 | 16:04 |
| marios | panda: aaand you're now on *(yet another) list | 16:05 |
| marios | zbr: so seriously lets have a call but would prefer tomorrow/planned especially if you want more than just me there ;) | 16:06 |
| *** ykarel_ has joined #oooq | 16:06 | |
| *** ykarel_ is now known as ykarel|away | 16:07 | |
| marios | zbr: just pick a time for tomorrow and send an invite... fwiw me/panda and folco will likely sync tomorrow during "scrum" time if that works for you? | 16:07 |
| *** TrevorV has quit IRC | 16:07 | |
| panda | marios: oh, thank you. | 16:07 |
| marios | panda: rfolco scrum time tomorrow OK ? sending invite | 16:08 |
| marios | panda: rfolco just decline if it doesn't work we work it out | 16:09 |
| *** ykarel has quit IRC | 16:09 | |
| marios | docs review please when anyone has some time thanks https://review.opendev.org/#/c/689530/ | 16:14 |
| marios | https://storage.bhs1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_29d/689530/7/check/openstack-tox-docs/29d4bb3/docs/ci/chasing_promotions.html | 16:14 |
| *** jaosorior has joined #oooq | 16:18 | |
| *** marios is now known as marios|out | 16:25 | |
| rfolco | marios|out, ack | 16:26 |
| sshnaidm|rover | amoralej, trains is better, but still fails: http://logs.rdoproject.org/63/22163/10/check/periodic-tripleo-rhel-8-train-containers-build-push/fdad2f0/logs/buildah-builds/kolla-rbpiuifx/docker/keystone/keystone-base/keystone-base-build.log.txt.gz | 16:28 |
| sshnaidm|rover | amoralej, only keystone | 16:28 |
| sshnaidm|rover | can't find mod_auth_openidc | 16:29 |
| *** ykarel|away has quit IRC | 16:31 | |
| *** marios|out has quit IRC | 16:32 | |
| amoralej | looks like missing dep | 16:32 |
| amoralej | lemme check | 16:32 |
| rlandy|ruck | looks like stein is promoting again | 16:37 |
| *** dsneddon_ has joined #oooq | 16:40 | |
| amoralej | sshnaidm|rover, i think that job is not applying rhel8 patch in kolla? | 16:41 |
| sshnaidm|rover | amoralej, hmm.. should be | 16:41 |
| amoralej | the problem in keystone is managed in https://review.opendev.org/#/c/671183/ | 16:42 |
| rlandy|ruck | hmmm .... fs039 | 16:43 |
| amoralej | sshnaidm|rover, you need to propose cherry-pick https://review.opendev.org/#/c/671183/ | 16:43 |
| rlandy|ruck | Data could not be sent to remote host "supplemental". Make sure this host can be reached over ssh: Warning: Permanently added '192.168.102.93' (ECDSA) to the list of known hosts. | 16:43 |
| rlandy|ruck | 2019-11-07 15:12:02.614100 | primary | Permission denied (publickey,gssapi-keyex,gssapi-with-mic). | 16:43 |
| amoralej | to train | 16:43 |
| amoralej | and rebase https://review.opendev.org/#/c/693159/ | 16:44 |
| sshnaidm|rover | amoralej, I apply this: https://review.opendev.org/#/c/693159/ | 16:44 |
| amoralej | note this one is on top of 671183 in master | 16:44 |
| sshnaidm|rover | oh, I see.. damn | 16:44 |
| amoralej | i mean https://review.opendev.org/#/c/669471/12 | 16:45 |
| sshnaidm|rover | all these hacks | 16:45 |
| amoralej | yep, hard to follow | 16:45 |
| amoralej | good point is that it seems we are closer | 16:45 |
| sshnaidm|rover | amoralej, https://review.opendev.org/#/c/693390/ | 16:46 |
| amoralej | now you need to rebase 693159 on 693390 | 16:48 |
| sshnaidm|rover | yep | 16:50 |
| sshnaidm|rover | rfolco, please make sure we include https://tree.taiga.io/project/tripleo-ci-board/task/1384?kanban-status=1447274 in the next sprint | 16:52 |
| sshnaidm|rover | raukadah|ptg, ^^ | 16:52 |
| sshnaidm|rover | weshay_ptg, ^^ | 16:52 |
| rfolco | sshnaidm|rover, ack | 16:53 |
| sshnaidm|rover | rfolco, these awful hacks shouldn't have been accepted as completed work at all | 16:53 |
| rfolco | sshnaidm|rover, agreed | 16:54 |
| *** jfrancoa has quit IRC | 16:55 | |
| *** tesseract has quit IRC | 17:03 | |
| sshnaidm|rover | amoralej, should be ok now | 17:04 |
| amoralej | ok | 17:05 |
| *** bogdando has quit IRC | 17:09 | |
| *** dtantsur is now known as dtantsur|afk | 17:15 | |
| sshnaidm|rover | rlandy|ruck, did you see this error? https://0c0090163291e905c3e7-fb3a89cd73503eeaa3ff317ecc3301da.ssl.cf1.rackcdn.com/693379/2/check/tripleo-buildimage-ironic-python-agent-centos-7/b9dd099/build.log | 17:17 |
| sshnaidm|rover | Element 'ironic-python-agent-ramdisk' not found | 17:17 |
| rlandy|ruck | The resource could not be found. | 17:17 |
| panda | niente letto, niente riscaldamenti | 17:18 |
| panda | oops wrong window. | 17:18 |
| rlandy|ruck | sshnaidm|rover: that repeatable> | 17:18 |
| sshnaidm|rover | rlandy|ruck, idk, rechecking | 17:19 |
| rlandy|ruck | sorry ben talking with sirov people | 17:19 |
| rlandy|ruck | ugh - we have ironic errors now | 17:19 |
| sshnaidm|rover | rlandy|ruck, and patch doesn't touch centos code at all | 17:21 |
| rlandy|ruck | probably node issue | 17:21 |
| rlandy|ruck | the state is "enroll", error: Failed to get power state for node dc5da850-d3b5-4225-9610-dcd8f7904255. Error: IPMI call failed: power status.\'}, {u\'result\': u\'Node 5f0f4109-08df-466b-a6d3-664a6a14a6b4 did not reach state "manageable", the state is "enroll", error: Failed to get power state for node 5f0f4109-08df-466b-a6d3-664a6a14a6b4. Error: IPMI call failed: power status.\'})\n'} | 17:22 |
| rlandy|ruck | ugh - haven't seen that in a while | 17:22 |
| sshnaidm|rover | rlandy|ruck, ovb? | 17:23 |
| rlandy|ruck | yes | 17:23 |
| sshnaidm|rover | rlandy|ruck, happens.. | 17:23 |
| rlandy|ruck | sshnaidm|rover: on every job? | 17:24 |
| sshnaidm|rover | rlandy|ruck, nope | 17:24 |
| rlandy|ruck | sshnaidm|rover: check the master promotion - every ovb has the same failure | 17:24 |
| * rlandy|ruck sees if we need clean up | 17:25 | |
| *** jpena|off is now known as jpena | 17:26 | |
| sshnaidm|rover | rlandy|ruck, well.. it's a problem | 17:28 |
| sshnaidm|rover | rlandy|ruck, and check jobs? | 17:28 |
| rlandy|ruck | the periodic ones, yes | 17:29 |
| rlandy|ruck | stack look ok though | 17:29 |
| rlandy|ruck | let's try rerun just fs001 | 17:29 |
| sshnaidm|rover | rlandy|ruck, I see this in check jobs too: http://logs.rdoproject.org/85/692985/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/6bd0e9f/logs/undercloud/home/zuul/overcloud_prep_images.log.txt.gz | 17:30 |
| rlandy|ruck | https://review.rdoproject.org/zuul/stream/698195df49ca4bd59dc959a96f30b9f1?logfile=console.log in progess | 17:31 |
| rlandy|ruck | may go as well | 17:31 |
| rlandy|ruck | master and other releases? | 17:31 |
| rlandy|ruck | just master? | 17:31 |
| *** TrevorV has joined #oooq | 17:32 | |
| sshnaidm|rover | https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 | 17:33 |
| sshnaidm|rover | rlandy|ruck, all branches | 17:33 |
| sshnaidm|rover | rlandy|ruck, I'd blame rdo cloud | 17:33 |
| sshnaidm|rover | rlandy|ruck, and this is a time to try a job comparison tool \o/ | 17:34 |
| rlandy|ruck | sshnaidm|rover: vexxhost is chugging along | 17:34 |
| rlandy|ruck | yay vexxhost | 17:34 |
| sshnaidm|rover | rlandy|ruck, but it has its own problems.. | 17:34 |
| rlandy|ruck | oh introspection errors, - how I love those | 17:34 |
| *** Trevor_V has quit IRC | 17:35 | |
| rlandy|ruck | Stderr: u'Get Auth Capabilities error\nError issuing Get Channel Authentication Capabilities request\nError: Unable to establish IPMI v2 / RMCP+ session\n'.: ProcessExecutionError: Unexpected error while running command. | 17:36 |
| rlandy|ruck | sshnaidm|rover: trying https://review.rdoproject.org/r/#/c/21683/ by itself | 17:37 |
| rlandy|ruck | fun. fun | 17:40 |
| rlandy|ruck | sshnaidm|rover: stein must have completed promotion | 17:42 |
| sshnaidm|rover | rlandy|ruck, oh, finally | 17:42 |
| rlandy|ruck | sshnaidm|rover: one win at a time | 17:43 |
| sshnaidm|rover | rlandy|ruck, merging this: https://review.opendev.org/#/c/693365 - it passed in rhel build containers jobs, train and master | 17:46 |
| sshnaidm|rover | amoralej, ^ | 17:47 |
| rlandy|ruck | sshnaidm|rover: k - voted so you have more legitimacy | 17:47 |
| *** dsneddon_ has quit IRC | 17:49 | |
| sshnaidm|rover | rlandy|ruck, I don't see answers from rhos-ops.. | 17:51 |
| sshnaidm|rover | rlandy|ruck, try to ping them later maybe | 17:51 |
| rlandy|ruck | not a busy channel these days | 17:52 |
| *** dsneddon_ has joined #oooq | 17:52 | |
| rlandy|ruck | we have a ton of active nodes but not many in error | 17:53 |
| sshnaidm|rover | rlandy|ruck, maybe overload, we'll see later if it pass tonight | 17:54 |
| rlandy|ruck | sshnaidm|rover: yep - thinking the same | 17:55 |
| rlandy|ruck | may calm down when europe starts logging off | 17:55 |
| sshnaidm|rover | rlandy|ruck, I'll help with that | 17:55 |
| sshnaidm|rover | although not really europe | 17:56 |
| sshnaidm|rover | rlandy|ruck, logging out for a few hours | 17:56 |
| sshnaidm|rover | rlandy|ruck, do you have someone to help tomorrow? | 17:56 |
| rlandy|ruck | sshnaidm|rover: nah - will be ok | 17:56 |
| rlandy|ruck | will ask the team to jump in if I am really struggling | 17:57 |
| rlandy|ruck | sshnaidm|rover: just let me know where you are leaving off | 17:57 |
| sshnaidm|rover | rlandy|ruck, cool, then leave me messages in etherpad | 17:57 |
| rlandy|ruck | sshnaidm|rover: ack - for sunday | 17:57 |
| rlandy|ruck | I'll log in sunday afternoon as well | 17:57 |
| sshnaidm|rover | rlandy|ruck, why? | 17:58 |
| rlandy|ruck | sshnaidm|rover: so I know how big a disaster monday will be :) | 17:58 |
| sshnaidm|rover | rlandy|ruck, isn't better to be a surprise? :) | 17:59 |
| rlandy|ruck | sshnaidm|rover: I don't like surprises | 17:59 |
| sshnaidm|rover | rlandy|ruck, oh, I | 17:59 |
| rlandy|ruck | I'm boring that way | 17:59 |
| sshnaidm|rover | rlandy|ruck, I'm in pto in Monday | 17:59 |
| rlandy|ruck | ok - enjoy | 18:00 |
| sshnaidm|rover | rlandy|ruck, so will leave you messages in Sunday | 18:00 |
| rlandy|ruck | perfect | 18:00 |
| rlandy|ruck | weshay_ptg, will be back | 18:00 |
| rlandy|ruck | so he can join the fun on monday | 18:00 |
| sshnaidm|rover | rlandy|ruck, great | 18:00 |
| sshnaidm|rover | rlandy|ruck, ok, logging out for now | 18:01 |
| rlandy|ruck | sshnaidm|rover: are there changes to merge? | 18:01 |
| sshnaidm|rover | rlandy|ruck, nope, I think I handled all | 18:01 |
| rlandy|ruck | ok - great - will watch ofr next run | 18:01 |
| *** sshnaidm|rover is now known as sshnaidm|afk | 18:03 | |
| rlandy|ruck | lunch brb | 18:08 |
| *** rascasoft has quit IRC | 18:22 | |
| *** jfrancoa has joined #oooq | 18:53 | |
| *** amoralej is now known as amoralej|off | 19:00 | |
| *** jfrancoa has quit IRC | 19:06 | |
| *** jaosorior has quit IRC | 19:15 | |
| rlandy|ruck | we're still failing out on introspection | 19:17 |
| *** jpena is now known as jpena|off | 19:42 | |
| *** Goneri has quit IRC | 19:43 | |
| arxcruz | jpena|off: I think we should move to master, since they fix the branch, the distro isn't working properly, and I don't know why the vars are not being passed to the standalone job | 19:50 |
| arxcruz | maybe yatin have some idea tomorrow | 19:51 |
| arxcruz | i'm off now | 19:51 |
| *** brault has quit IRC | 20:01 | |
| *** brault has joined #oooq | 20:02 | |
| *** dsneddon_ has quit IRC | 20:15 | |
| *** dsneddon_ has joined #oooq | 20:18 | |
| *** dsneddon_ has quit IRC | 20:52 | |
| *** dsneddon_ has joined #oooq | 20:56 | |
| *** dsneddon_ has quit IRC | 20:59 | |
| *** Goneri has joined #oooq | 21:01 | |
| *** dsneddon_ has joined #oooq | 21:08 | |
| *** dsneddon_ has quit IRC | 21:24 | |
| *** TrevorV has quit IRC | 21:28 | |
| *** brault has quit IRC | 21:44 | |
| *** jfrancoa has joined #oooq | 21:59 | |
| *** jfrancoa has quit IRC | 22:03 | |
| *** dsneddon_ has joined #oooq | 22:09 | |
| *** brault has joined #oooq | 22:10 | |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 22:30 | |
| *** dsneddon_ has quit IRC | 22:54 | |
| *** tosky has quit IRC | 23:24 | |
| *** dsneddon_ has joined #oooq | 23:32 | |
| *** jfrancoa has joined #oooq | 23:34 | |
| *** rfolco has quit IRC | 23:37 | |
| *** rfolco has joined #oooq | 23:53 | |
| *** jfrancoa has quit IRC | 23:57 | |
| *** rfolco has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!