Wednesday, 2021-06-23

*** ykarel|away is now known as ykarel04:37
*** marios is now known as marios|ruck05:23
*** ysandeep|out is now known as ysandeep05:52
*** amoralej|off is now known as amoralej06:38
*** jpena|off is now known as jpena06:56
anbanerj|rovermarios|ruck, O/07:18
anbanerj|rovermarios|ruck, tripleo-buildimage-overcloud-hardened-uefi-full-centos-8 has failed 4-5 times with No config file ...overcloud-hardened-images-uefi-centos8 I am creating a bug for it07:22
marios|ruckanbanerj|rover: ack thank you it is very new so if it persists we can just move to non voting and ping sbaker07:24
*** bhagyashris_ is now known as bhagyashris08:27
*** ykarel is now known as ykarel|lunch08:52
marios|ruckanbanerj|rover: biab getting some food09:17
* zbr had a 'good' start today, uncovering a python bug https://bugs.python.org/issue4449709:23
*** ykarel|lunch is now known as ykarel09:56
marios|ruckthanks anbanerj|rover 10:11
marios|ruckanbanerj|rover: o/ are you ok for the program call today? 10:29
marios|ruckanbanerj|rover: i mean are you ok to give our status?10:29
marios|ruckanbanerj|rover: (email from weshay|ruck )10:29
anbanerj|rovermarios|ruck, I was checking the promotions. Master and train look ok - 2 days ago. Do we need to promote ussuri today?10:30
marios|ruckanbanerj|rover: yeah ussuri is quite old 10:31
anbanerj|roverdo we need to get a promotion before the program call?10:31
marios|ruckanbanerj|rover: but the ones we care about for the program call are train and master/wallaby so no need to mention anything else10:31
marios|ruckanbanerj|rover: no10:32
ysandeepanbanerj|rover: fyi.. weshay|ruck usually add status what we need to report.. we don't report ussuri in program call 10:32
marios|ruckanbanerj|rover: the status is already updated there10:32
anbanerj|roverok, got it10:32
marios|ruckysandeep: well he didn't always ;) normally the ruck|rover prepares that status but yeah weshay|ruck is helping us by doing it 10:32
ysandeepmarios|ruck: o/ 10:35
ysandeepi just checked he updated the doc already , weshay++10:35
anbanerj|roverysandeep, marios|ruck yes the doc is updated. I know abt the vexhost and the octavia issues. And the outage on thursday. What is happening for osp 17? 10:37
anbanerj|roverIs this the s=dashboard http://tripleo-cockpit.usersys.redhat.com/d/v8gltz4Mz/rhos-17-full-component-pipeline?orgId=1 ?10:37
ysandeepanbanerj|rover: there are some ongoing cix card for 17 currently worked upon, apart from that we have promoted component this week to refresh builds.10:39
*** pojadhav is now known as pojadhav|afk10:41
marios|ruckysandeep: don't really need to report on that osp17 i mean 10:41
anbanerj|roverthanks10:41
marios|rucksorry anbanerj|rover 10:41
marios|ruckanbanerj|rover: you should only report for train & osp1610:41
marios|ruckanbanerj|rover: 17 is blocked but it is blocked for everyeone still so not a big deal 10:42
marios|ruckanbanerj|rover: not a big deal for us i mean, yet10:42
marios|ruckanbanerj|rover: for the "outage on thursday" let me update actually we haven't hear back from vexx yet 10:42
marios|ruckjpena: did we hear back from vex re the upgrade? I don't see something in the cix card10:43
marios|ruckjpena: for the "potential disruption" was meant to be last week 10:43
anbanerj|rovermarios|ruck, ok. For train and master we are good - we had promotions on monday10:43
jpenamarios|ruck: no, this is getting weird :-/. I asked about it on Monday, they said ". We have now forwarded this once more to our team so they can confirm the possibility of the upgrade this coming Friday, June 25. We'll let you know once the team confirmed the details."10:52
marios|ruckjpena: ack thanks 10:53
marios|ruckjpena: so its fine i was just wondering as we are reporting in the program call in case we had specific news 10:54
marios|ruckjpena: thank you for dealing with that ticket10:54
anbanerj|roverjpena, marios|ruck thanks10:55
*** amoralej is now known as amoralej|lunch11:28
*** jpena is now known as jpena|lunch11:30
*** dviroel|out is now known as dviroel11:37
cgoncalvesmarios|ruck, 11:51
cgoncalvesoops, sorry :)11:51
marios|rucko/ cgoncalves 11:53
cgoncalvesmarios|ruck, weshay|ruck: what do you think about making tripleo-ci-centos-8-standalone-on-multinode-ipa voting? it caught a regression but it was overlooked. see https://bugs.launchpad.net/tripleo/+bug/193333011:53
weshay|ruckcgoncalves, we were just talking about it yesterday11:54
weshay|ruckrlandy, was proposing making it voting yesterday ^ :)11:54
marios|ruckcgoncalves: so yes but we should make it stable first? 11:54
marios|ruckcgoncalves: weshay|ruck: doesn't look very stable here https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa 11:54
rlandyweshay|ruck: marios|ruck: yep - might be in a place where it's worth making it voting on certain repos11:55
cgoncalvesmarios|ruck, I think it is stable, see https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa&branch=master11:55
cgoncalvesthe last builds failed most likely because of the regression I mentioned ^11:55
marios|ruckcgoncalves: so you want to make it stable only for master?11:56
rlandyade_lee was investigating another real issue the test caught11:56
weshay|ruckrlandy, blood sweat and twears rlandy++11:56
marios|ruckcgoncalves: ack anyway no push back from me if the job is stable yes but probably we should fix that bug first then cos 14:54 < marios|ruck> cgoncalves: weshay|ruck: doesn't look very stable here https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa 11:56
rlandytwears?11:56
rlandywear and tear + tears?11:56
rlandyexactly11:56
cgoncalvesmarios|ruck, no, it would be best if voting in all branches11:57
marios|ruckcgoncalves: k just checking cos you pointed to master11:57
weshay|ruckmarios|ruck, 0/ any luck w/ a molecule playbook/role to fix tox?11:57
rlandymarios|ruck: weshay|ruck: so the job is down to 68% pass but I think the errors are legit11:57
weshay|ruckI need to dig into molecule11:58
marios|ruckweshay|ruck: btw https://review.rdoproject.org/r/c/testproject/+/34256 did use the right hash (i mean 'container_build_id is the one we passed' but th eproblem is with get_hash role and reporting to delorean, am digging abit there)11:58
weshay|ruckOH11:58
weshay|ruckwell that's cool11:58
marios|ruckweshay|ruck: so it just ignores the thing you passed and reports to delorean with the current value of ci-testing11:58
weshay|ruckmarios|ruck, if the yum repos are correct and the containers are correct.. we would be safe to waive the tests on the promoter11:59
marios|ruckweshay|ruck: well the containers are i think the yum repos are but we don't have the md5 there just he actual delorean.repo so i am not 1000% about that11:59
weshay|ruckmarios|ruck, if you want to do that... just tell anbanerj|rover or akahat 12:00
marios|ruckweshay|ruck: but for sure the reporting to delorean is wrong/ignores what you pass12:00
*** marios|ruck is now known as marios|ruck|call12:00
rlandymarios|ruck|call: cgoncalves: weshay|ruck: here is the patch to make the ipa job voting again ... https://review.opendev.org/c/openstack/tripleo-ci/+/762766 - I removed my w-112:00
cgoncalvesrlandy, thanks!12:01
marios|ruck|callrlandy: ack adding to queue12:03
rlandymarios|ruck|call: lol - you voted ages ago :) but will leave it to ruck/rovers to w+ since they will have to deal with the fallout12:03
zbrweshay|ruck: fix delayed by bug i found in molecule...12:10
chandankumarweshay|ruck: kopecmartin arxcruz rlandy tempest meeting is conflicting with Paul open forum webcast? Do we want to cancel or keep it?12:23
weshay|ruckzbr, thank you!12:23
rlandychandankumar: either way - fine by me12:23
weshay|ruckzbr, https://review.opendev.org/c/openstack/requirements/+/797620 ?12:23
weshay|ruckis that the fix12:23
kopecmartinchandankumar: i don't have anything specific, I'm fine with cancelling it12:25
*** ysandeep is now known as ysandeep|afk12:26
*** amoralej|lunch is now known as amoralej12:30
chandankumarrlandy: kopecmartin ok then keeping it12:31
*** pojadhav|afk is now known as pojadhav12:32
*** jpena|lunch is now known as jpena12:33
rlandyweshay|ruck: commenting on CIX card related to https://bugzilla.redhat.com/show_bug.cgi?id=1954108 ...12:33
rlandythe logs in the BZ are no longer available12:33
rlandylooks like if we get deploy to pass we're good12:34
rlandyleaving that note12:34
zbrweshay|ruck: note, that is adding our test reqs to upper constraints, it may be an alternative to pip-compile (if it gets approved)12:34
zbrweshay|ruck: were did you copy the bit with notify: yum-clean-metadata from?12:39
zbrnevermind, i found it on google.12:42
anbanerj|rovermarios|ruck|call, yes, I can remove the tests from the promotion server criteria and run the promotion for ussuri12:44
marios|ruck|callanbanerj|rover: ?12:46
marios|ruck|callanbanerj|rover: which tests did i miss a conversation here ... me scrolls up12:46
*** marios|ruck|call is now known as marios|ruck12:47
anbanerj|rovermarios|ruck|call, I mean I can work with you on the ussuri promotion I thought there are some failing jobs which we should remove from criteria and run promotions12:48
marios|ruckanbanerj|rover: couple of jobs were failing but we got them through with testproject on https://review.rdoproject.org/r/c/testproject/+/34256 . but the problem is with our ci-config tooling because even though those jobs are using the right hash, they are not reporting it, instead reporting the current value of tripleo-ci-testing not the one we are overriding with12:49
marios|ruckthat needs to be fixed 12:49
marios|ruckanbanerj|rover: so we could consider promoting aggregate: 4a47487922712b220c9e80ee0ca16ed9 i.e. waive missing jobs {'periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-ussuri', 'periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-ussuri'}12:49
marios|ruck(the ones that passed in https://review.rdoproject.org/r/c/testproject/+/34256 )12:49
anbanerj|rovermarios|ruck, oh ok. I still need to find out a lot more about how promotions work. /me going for lunch. Will read through the promotion hackmd after that. I have a 1-1 today. If it is ok with you and rlandy I can try the ussuri promotion then12:57
rlandyanbanerj|rover: ok by me12:57
marios|ruckanbanerj|rover: sure np i want to dig some more into how to fixup the dlrn reporting for now12:58
anbanerj|rovermarios|ruck, rlandy thanks12:58
marios|ruckanbanerj|rover: we have some recently updated docs in tripleo-docs too you should check 12:58
marios|ruckanbanerj|rover: that one https://docs.openstack.org/tripleo-docs/latest/ci/stages-overview.html12:58
marios|rucka lot there worth reading again if you already have12:58
anbanerj|rovermarios|ruck, thanks. I'll check12:59
chandankumararxcruz, akhat, chandankumar, mkopec, soniya29, weshay, rlandy, ysandeep tempest meeting: https://meet.google.com/zvw-hnds-ncu?authuser=2 and agenda: https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view13:01
*** ysandeep|afk is now known as ysandeep13:04
* ysandeep facing issues with sso13:04
chandankumarrlandy: https://review.opendev.org/c/openstack/openstack-tempest-skiplist/+/78962513:19
weshay|ruckysandeep, your patch merged.. vote on https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/3426913:24
ysandeepweshay|ruck, yes and its on box already, i just rebased your patch ^^ it was in merge conflict13:25
weshay|ruckk13:26
ysandeeprlandy: when you have time, this one need another vote: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34269 13:45
rlandyysandeep: looking13:49
rlandyysandeep: we're also having a bad time with OVB downstream13:49
rlandygoing to look at that more today13:50
akahatmarios|ruck, o/ rdo jobs are getting node failure issues. is it known? https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34273, https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/3419613:50
marios|ruckakahat: yes https://bugs.launchpad.net/tripleo/+bug/1931226 we have a ticket with vex about that and cix but no update so far they are working on it13:50
marios|ruckakahat: it is not consistent so try recheck for now13:51
rlandyysandeep: ok to w+ that?13:51
ysandeeprlandy, yes13:51
rlandydone13:51
akahatmarios|ruck, okay. thanks for update !!13:51
arxcruzweshay|ruck: rlandy http://38.102.83.131/quay/master-report.html let me know if it's missing something 13:57
rlandyarxcruz++ nice13:58
ysandeeprlandy: Shrestha commented on cix card that new ovn is released, we can remove the workaround, i tested locally.. looks okay: http://pastebin.test.redhat.com/97421314:01
ysandeeprlandy, I have created reverts: https://code.engineering.redhat.com/gerrit/q/topic:%22revert_ovn_pinning%22+(status:open%20OR%20status:merged)14:02
rlandyysandeep: great - thanks14:02
ysandeeprlandy: ack for ovb issues in downstream, please let me know if you need a hand14:03
ysandeepweshay|ruck: fyi.. your patch is on box as well14:12
*** ysandeep is now known as ysandeep|out14:14
*** marios|ruck is now known as marios|ruck|call14:31
*** marios|ruck|call is now known as marios|ruck14:51
weshay|ruckmarios|ruck, anbanerj|rover you folks chatting?14:58
anbanerj|roverweshay|ruck, not yet14:59
marios|ruckweshay|ruck: about what? 15:00
weshay|ruckzbr, anything I can help review / merge re: molecule15:00
weshay|ruckmarios|ruck, anbanerj|rover you mentioned you might chat after the cix call..15:00
marios|ruckweshay|ruck: don't recall... what was the subject/issue15:00
weshay|ruckjust trying to catch up and help if I can15:00
marios|ruckweshay|ruck: or rather is there something you need to discuss? 15:00
marios|ruckweshay|ruck: sure we can catchup 15:00
anbanerj|rovermarios|ruck, shall I make the job non voting?15:00
marios|ruckanbanerj|rover: yeah that would be great15:01
chandankumarrlandy: bhagyashris_ waiting for you to join15:01
marios|ruckanbanerj|rover: for https://bugs.launchpad.net/tripleo/+bug/1933309 you mean 15:01
marios|ruckanbanerj|rover: but can you please please update the bug when you do that so we keep the info on there when sbaker checks it out in his morning15:01
marios|ruckanbanerj|rover: thank you !15:01
weshay|ruckthe only thing I don't have clarity on is ussuri promotion atm15:01
anbanerj|rovermarios|ruck, yep 1933309. I'll update the bug15:02
anbanerj|roverweshay|ruck, I'll work with rlandy in todays 1-1 for ussuri promotion15:02
marios|ruckweshay|ruck: well, i am trying to do it with https://review.rdoproject.org/r/c/testproject/+/34256 assuming https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34275 works15:02
weshay|ruckah k.. thank you both.. /me looks15:02
marios|ruckweshay|ruck: otherwise as before, i.e. the jobs ran and passed in testproject/+/34256 but the dlrn reporting was off. i confirmed the containers are also good i.e. TASK | Pull 192.168.24.1:8787/tripleoussuri/centos-binary-mariadb:4a47487922712b220c9e80ee0ca16ed9-updated-20210622161906 image15:02
marios|ruckweshay|ruck: i didn't confirm the repos15:03
weshay|ruckthat looks like the right idea to me :)15:03
marios|ruckweshay|ruck: the point is, we could consider promoting that hash 4a47487922712b220c9e80ee0ca16ed9 based on those jobs passing 15:03
marios|ruckweshay|ruck: or wait to see if my testproject works then it should report it OK for the promoter to find15:03
weshay|ruckyes.. 15:04
weshay|ruckmarios|ruck, anbanerj|rover any questions or concernts on the tripleo-ci tasks?15:04
marios|ruckweshay|ruck: so anyway i'm around for the next hour ish if you want to discuss further about that ussuri promote stuff15:04
weshay|ruckmarios|ruck, your patches could do the trick.. and we should try to land https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34275/1/ci-scripts/infra-setup/roles/get_hash/tasks/get_hash.yaml15:05
weshay|ruckif it doesn't we'll take the exit ramp w/ anbanerj|rover later today w/ ronelle for ussuri15:06
weshay|ruckso we remove the time pressure15:06
weshay|ruckanbanerj|rover, ur on the hook for victoria15:06
rlandychandankumar: periodic-tripleo-ci-rhel-8-bm_envD-3ctlr_1comp-featureset035-rhos-16.215:09
marios|ruckweshay|ruck: well ok lets see it took a lot of staring to post those 11 lines so might need tweaking but yeah hopefully it is something simple like that15:09
rlandychandankumar: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/buildset/40d90d0062d544a688224ba981e4a89115:11
weshay|ruckysandeep|out, the latest patches have been automatically pulled into the tool box15:12
weshay|ruckmarios|ruck++15:12
weshay|ruckmarios|ruck, anbanerj|rover fyi.. toolbox scripts that clean stacks from vexx appears to be working properly again15:13
marios|ruckweshay|ruck: thanks15:15
zbrweshay|ruck: i am getting closer,...15:20
*** ykarel is now known as ykarel|away15:20
weshay|ruckzbr, k.. thank you15:21
anbanerj|roverHey rlandy, if you have some time now we can chat about the ussuri promotion15:33
rlandyanbanerj|rover: sure15:37
anbanerj|roverrlandy, thanks, joining the 1-115:37
rlandyweshay|ruck: ^^ just fyi15:37
rlandyworking on ussuri promotion15:37
marios|ruckanbanerj|rover: weshay|ruck: me close to end of file ping me gchat for anything to look at in the morning as usual ;)15:49
*** marios|ruck is now known as marios|out15:55
chandankumarrlandy: weshay|ruck we made some changes https://sf.hosted.upshift.rdu2.redhat.com/images/redhat8/rhos-16.2/wes-current-tripleo/15:57
chandankumarqcow2 client is working now15:57
weshay|ruckwoot15:58
chandankumarakahat: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/24954215:58
*** jpena is now known as jpena|off16:00
rlandychandankumar: nice thanks16:17
*** frenzy_friday is now known as anbanerj|rover16:19
*** amoralej is now known as amoralej|off16:25
rlandychandankumar: rekicked 16.2 jobs that hit node failures: https://code.engineering.redhat.com/gerrit/c/testproject/+/18944016:28
*** sshnaidm is now known as sshnaidm|afk16:35
weshay|ruckarxcruz, woot.. tempest.keystone.api in the house :)16:37
weshay|ruckzbr++ well done.. ran it locally :)16:39
weshay|ruckzbr, did molecule version need a bump for prepare?16:40
arxcruzweshay|ruck: yup16:42
arxcruzweshay|ruck: and here's your report http://38.102.83.131/quay/master-report.html and http://38.102.83.131/quay/tag/wallaby-report.html16:42
arxcruzcheck if you liked or should i change something 16:42
weshay|ruckAH... look at that16:43
weshay|ruckI love it16:43
weshay|ruckarxcruz, that looks great.. so how do I know if I need to respond to openstack-aodh-evaluator?16:45
weshay|ruckneed reviews on https://review.opendev.org/c/openstack/tripleo-ci/+/79752816:46
arxcruzmanually try to copy or check what's wrong with that one 16:46
arxcruzweshay|ruck: that html will always be the latest run, it will not keep old ones 16:46
weshay|ruckk16:46
arxcruzthe other txt log file will be available though 16:46
weshay|ruckyou add a README.html w/ your tips/tricks16:46
weshay|ruckcontext16:46
arxcruzok16:47
weshay|ruckthank you!! you are knocking these down16:47
zbrweshay|ruck: TBH, i am not sure if molecule needed a bump for our fix, but i did not want to have to deal with an unsupported version16:48
weshay|ruck++16:48
zbrprobably not, the changed_when bit was required, otherwise idempotence would have failed (was disabled before)16:49
zbrwe did not run prepare and idempotence before this change.16:49
rlandyanbanerj|rover: https://trunk.rdoproject.org/centos8-ussuri/current-tripleo/delorean.repo.md517:01
rlandy^^ that what you expected?17:01
rlandydlrn_hash_tag: 4a47487922712b220c9e80ee0ca16ed917:02
rlandyin job17:02
rlandy4b498feadc27f8999f24a1acf91f3b7f17:02
rlandyweshay|ruck: anbanerj|rover: ^^?17:02
rlandyrefresh ok17:03
rlandy4a47487922712b220c9e80ee0ca16ed917:03
rlandysorry - nvm 17:03
anbanerj|roverrlandy,  Yep! 4a47487922712b220c9e80ee0ca16ed917:31
* anbanerj|rover leaving for the day17:33
weshay|ruckrlandy, https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/3411517:46
weshay|ruckrlandy, https://review.opendev.org/c/openstack/tripleo-ci/+/79752818:00
rlandyweshay|ruck: and then lo and behold ... ovb returns 18:32
rlandyperiodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2success18:32
rlandyperiodic-tripleo-ci-rhel-8-ovb-1ctlr_2comp-featureset020-internal-rhos-16.218:32
rlandychandankumar: ^^ well we promised you promotable hash18:33
rlandywaiting on two jobs to rerun18:33
weshay|ruckah nice18:36
rlandyweshay|ruck: hmmm ... https://review.opendev.org/c/openstack/tripleo-quickstart/+/797541 didn;t work18:47
weshay|ruckya.. I commented18:48
weshay|ruckrlandy, we could exclude ansible from that repo18:48
rlandypriority=20 is there18:48
weshay|ruckaye18:48
rlandyweshay|ruck: ack18:48
rlandybetter way to go18:49
rlandyweshay|ruck: updating that patch18:49
weshay|ruckdviroel, ^ tooling.. in play eventually18:49
dviroelweshay|ruck: +1, soon18:51
rlandychandankumar: weshay|ruck: akahat: http://osp-trunk.hosted.upshift.rdu2.redhat.com/api-rhel8-osp16-2/api/civotes_agg_detail.html?ref_hash=93bf835ed8672dcec32e052836842dd719:33
rlandy^^ good hash to promote 19:33
rlandyall passed except BM19:33
rlandyeven two ovb jobs19:34
rlandychecking BM box19:34
rlandymay be an issue there19:34
weshay|ruckhrm19:34
* weshay|ruck looks19:35
weshay|ruckrlandy, ya.. ran for 10 hours in gather facts19:36
weshay|ruck2021-06-23 09:54:48.803026 | primary | TASK [Gathering Facts] *********************************************************19:36
weshay|ruck2021-06-23 09:54:48.803033 | primary | Wednesday 23 June 2021  09:54:48 -0400 (0:00:00.080)       0:00:00.177 ********19:36
weshay|ruck2021-06-23 19:16:59.998426 | RUN END RESULT_TIMED_OUT: [untrusted : opendev.org/openstack/tripleo-ci/playbooks/tripleo-ci/run-v3.yaml@master]19:36
weshay|ruck2021-06-23 19:16:59.998787 | POST-RUN START: [trusted : code.engineering.redhat.com/openstack/tripleo-ci-internal-config/playbooks/dlrn/post-dlrn.yaml@master]19:36
weshay|ruckodd19:36
weshay|ruckrlandy, ack to promote19:36
weshay|ruckrlandy, can you set the ussuri criteria back?19:36
rlandyweshay|ruck: yeah = something up there19:37
rlandyweshay|ruck: checking - I didn't anbanerj|rover may have19:37
weshay|ruckthanks19:38
rlandyweshay|ruck: k - criteria are back19:39
rlandyand hash count19:39
weshay|ruck:)19:39
rlandyweshay|ruck: can you try  enqueue-ref 17? when I do it never starts properly19:40
rlandyjust want to see if it's juts me19:40
rlandyjust 19:40
weshay|ruckrlandy, sure19:41
weshay|rucksec19:41
rlandythanks19:41
weshay|rucktriggered.. not started19:43
weshay|ruckrlandy, it started.. then died19:54
rlandyweshay|ruck: same with me19:55
rlandythe minute you start it, you need to say kaddish for the line19:56
rlandynever gets going19:56
rlandywill wait for the next kick19:56
weshay|ruckrlandy, I'll email mathieu19:56
rlandyk - thanks19:57
*** dviroel is now known as dviroel|out21:13
anbanerj|roverrlandy, weshay|ruck omg! I forgot to reset the criteria!21:33
rlandyanbanerj|rover: np - I did it21:33
anbanerj|roverrlandy, thanks21:34
*** rlandy is now known as rlandy|bbl22:15

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!