Friday, 2020-12-18

« Thursday, 2020-12-17 Index Saturday, 2020-12-19 »
*** chem has quit IRC00:43
*** chem has joined #oooq00:45
*** jmasud has joined #oooq00:48
*** jmasud has quit IRC00:53
*** jmasud has joined #oooq01:05
*** jmasud has quit IRC01:14
*** jmasud has joined #oooq01:16
*** jmasud has quit IRC01:20
*** jmasud has joined #oooq01:25
*** jmasud has quit IRC01:32
*** jmasud has joined #oooq01:33
*** jmasud has quit IRC01:53
*** jmasud has joined #oooq02:09
*** jmasud has quit IRC02:15
*** jmasud has joined #oooq02:18
*** jmasud has quit IRC02:20
*** jmasud has joined #oooq02:25
*** jmasud has quit IRC02:30
*** apetrich has quit IRC03:19
*** udesale has joined #oooq04:45
*** jmasud has joined #oooq04:54
*** ykarel has joined #oooq04:57
*** ratailor has joined #oooq05:31
*** skramaja has joined #oooq05:49
*** jmasud has quit IRC06:08
*** ysandeep|away is now known as ysandeep06:11
*** marios has joined #oooq06:20
* ysandeep wondering if we can restore someone else patch? 06:24
bhagyashrisysandeep, i think owner and cores having those rights to restore the any ones patch07:17
ysandeepbhagyashris, thanks!07:17
chandankumarysandeep: which one needs to be restored?07:17
ysandeepchandankumar, thanks!.. i will request owner to restore..07:18
*** slaweq has joined #oooq07:43
*** jmasud has joined #oooq07:47
*** amoralej|off is now known as amoralej08:01
*** jpena|off is now known as jpena08:17
*** ysandeep is now known as ysandeep|lunch08:18
*** jfrancoa has joined #oooq08:26
zbr|roverbhagyashris: hi!09:02
zbr|roverthat issue with timeout is very weird, i am looking at build history on https://review.rdoproject.org/zuul/builds?job_name=tox-py36-ci-config&project=rdo-infra/ci-config09:02
zbr|roveri see previous execution times of ~4-5minutes09:03
zbr|roverwith your patch, it passed in ~20mins, but i see failures at 30min and 60min, what is the real timeout 30 or 06?09:03
bhagyashriszbr|rover, which patch you are talking09:04
bhagyashriszbr|rover, on this patch it never passed09:05
zbr|roverhttps://review.rdoproject.org/r/#/c/31399/09:05
bhagyashriszbr|rover, but with patch it's passing https://review.rdoproject.org/r/#/c/31399/09:05
bhagyashrison this https://review.rdoproject.org/r/#/c/28014/ patch it never passed09:06
bhagyashriszbr|rover, on this patch it's passing https://review.rdoproject.org/r/#/c/31399/ and i haven't changed any timeout value09:06
zbr|roveryep, at least once :D, but i will workflow it and we can investigate more on the follow-up i created09:07
zbr|rovermy main concern was that using old resolver is hack, and that it may have a boomerang effect later.09:07
bhagyashriszbr|rover, yeah09:09
zbr|roverbit of mind blowing09:09
zbr|roveri need to identify which version of pip it used when it failed. i am unable to make it fail locally.09:09
bhagyashrissame zbr|rover even i am unable to make it fail locally09:10
bhagyashrisnot sure why it's hitting in zuul09:10
bhagyashrisUsing pip 20.3.1 from /home/zuul/src/review.rdoproject.org/rdo-infra/ci-config/.tox/py36/lib/python3.6/site-packages/pip (python 3.6)09:13
bhagyashrishttps://logserver.rdoproject.org/58/31358/9/check/tox-py36-ci-config/cd11408/tox/py36-1.log09:13
bhagyashrishere is more details https://bugs.launchpad.net/tripleo/+bug/1908274/comments/409:14
openstackLaunchpad bug 1908274 in tripleo "tox-py36-ci-config job is broken " [Critical,Triaged]09:14
*** slaweq has quit IRC09:16
chandankumarzbr|rover: bhagyashris I was thinking to merge the old deps resolver and then make the job passing and then start fixing the stuf based on new resolver to unblock the patches09:16
*** derekh has joined #oooq09:17
zbr|roverchandankumar: i already +W it, it will merge soon.09:17
chandankumarzbr|rover: thanks :-)09:17
chandankumarbhagyashris: I will help on splitting the jobs based on different projects09:18
bhagyashrischandankumar, zbr|rover thanks ! :)09:18
bhagyashrischandankumar, ack09:18
*** slaweq has joined #oooq09:18
bhagyashrismarios, fyi, work around patch https://review.rdoproject.org/r/#/c/31399/ got merge. you can rebase your patch :)09:20
*** slaweq has quit IRC09:20
zbr|roveri am considering something I already use on other projects: a lock file and use of pip-tools for updating dependencies.09:20
zbr|roverthis means no longer depending on pip to install whatever is latest and always using what passed last time09:21
zbr|roverthis will avoid any endless attempts to find working dependencies, because they are pinned.09:21
*** ysandeep|lunch is now known as ysandeep09:22
zbr|roverbtw, why we are using upper constraints from openstack? i bet this is the cause of the conflicts09:23
zbr|roverthese constraints may conflicts with what we need, and can be changes outside our power.09:23
zbr|roverespecially for ci-config which tests only our internal code, it makes no sense on depending on them.09:24
zbr|roverinteresting fact, my pip is also stuck....09:24
*** slaweq has joined #oooq09:25
zbr|roveryes: i commented the -c... line and works fine.09:26
zbr|roverit is ok to use constraint files, but only if we are the ones producing them.09:26
mariosthanks bhagyashris09:37
marioschandankumar: bhagyashris: so in the end we went with legacy resolver not updating pip/setuptool with download true09:38
chandankumarmarios: yes correct09:39
zbr|rovermarios: wait few minutes, i will have a long term fix09:39
marioschandankumar: ack thanks either way good to be unblocked :)09:39
marioszbr|rover: ack09:39
marioszbr|rover: please add some info or track whatever fix you poste there https://bugs.launchpad.net/tripleo/+bug/1908274/comments/809:42
openstackLaunchpad bug 1908274 in tripleo "tox-py36-ci-config job is broken " [Critical,Fix released]09:42
*** jmasud has quit IRC10:00
*** jmasud has joined #oooq10:04
*** jmasud has quit IRC10:31
*** jmasud has joined #oooq10:38
*** slaweq has quit IRC10:51
*** slaweq has joined #oooq10:56
*** ysandeep is now known as ysandeep|afk11:24
zbr|rovermarios: bhagyashris chandankumar: please check the fix for deps install: https://review.rdoproject.org/r/#/c/31400/11:26
zbr|roverlet me know if further explanation is needed, i am happy to even explain on meet if needed11:26
chandankumarzbr|rover: just one thing, we are working on breaking the jobs in 3 parts11:30
chandankumarso i think we can also move the pip compile generated in 3 parts11:31
chandankumaras your patch looks good apart from that pip compiled test requirements file11:31
zbr|rovernope: rebuilding deps will be done manually whenever we want11:32
zbr|roverthey would change too often to include them in automation.11:32
*** udesale_ has joined #oooq11:32
chandankumarzbr|rover: ok11:33
marioszbr|rover: ack will do11:33
zbr|roveri do have an example with far less deps that has an automated pipeline, look at https://github.com/ansible-community/toolset/pulls?q=is%3Apr+is%3Aclosed11:33
zbr|roverbased on how many we have, we would have like 5-10 changes per day to keep it fresh.11:34
zbr|rovercan we cope with that?...11:34
zbr|roverbasically we control when we bump them, basically only when we need.11:35
*** udesale has quit IRC11:35
*** jmasud has quit IRC11:43
chandankumarstepping out for a bit, back in 30 mins11:52
*** ykarel_ has joined #oooq12:14
*** ykarel has quit IRC12:17
*** ykarel_ is now known as ykarel12:18
*** ratailor has quit IRC12:26
chandankumarzbr|rover: does the collect-logs issue got fixed?12:35
zbr|roverchandankumar: not be me for sure12:36
zbr|rovergoing out for a few mins for lunch12:36
*** rfolco has joined #oooq12:37
chandankumarok12:39
*** tosky has joined #oooq12:58
*** jpena is now known as jpena|lunch13:00
*** ysandeep|afk is now known as ysandeep13:00
weshay|ruckakahat|ruck, zbr|rover as you have time.. I'll need an update so I know what to push/watch before you go13:19
chandankumarweshay|ruck: do you have the link of collect-logs bug link handy?13:22
weshay|ruckI should.. /me looks at ss13:22
weshay|ruckchandankumar, https://bugs.launchpad.net/tripleo/+bug/190805413:24
openstackLaunchpad bug 1908054 in tripleo "Collect logs failing to collect logs" [Critical,Triaged]13:24
chandankumarweshay|ruck: thanks!13:24
weshay|ruckchandankumar, I think sshnaidm fixed it yesterday.. fix = has patch13:24
weshay|ruckchandankumar, https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/76758013:25
chandankumarhttps://review.opendev.org/c/openstack/ansible-role-collect-logs/+/76758013:25
chandankumarweshay|ruck: https://review.opendev.org/c/openstack/tripleo-ci/+/76745613:26
chandankumarweshay|ruck: https://a455f3929f61c821afa0-d5237dd035b9f3c7532839ca52b23c88.ssl.cf2.rackcdn.com/767456/2/gate/tripleo-ci-centos-8-content-provider-ussuri/2a933c6/logs/index.html13:26
chandankumarhttps://322149cf6bb5d1d418ae-1e96e5dddd851f74afd1442b0a5110b5.ssl.cf1.rackcdn.com/767456/2/gate/tripleo-ci-centos-8-content-provider-train/a125d83/logs/index.html13:26
chandankumarno logs here13:27
chandankumarcollect logs merged recently13:28
chandankumarso might be that patch is not used there13:28
*** sanjayu_ has quit IRC13:29
weshay|ruckchandankumar,13:29
weshay|ruck2020-12-18 05:19:34.584224 | primary -> localhost | To git+ssh://199.204.45.243:22//home/zuul/src/opendev.org/openstack/ansible-role-collect-logs13:29
weshay|ruck2020-12-18 05:19:34.584320 | primary -> localhost |    09a8861..61e3cc3  master -> master13:29
weshay|ruck2020-12-18 05:19:34.584429 | primary -> localhost |  * [new branch]      origin/master -> origin/master13:29
weshay|ruck2020-12-18 05:19:34.584522 | primary -> localhost |  * [new branch]      refs/zuul/4f26aeafdb2367620a393c973eddbe8f8b846ebd -> refs/zuul/4f26aeafdb2367620a393c973eddbe8f8b846ebd13:29
weshay|ruckhrm..13:29
weshay|ruckdont see that commit at all in the repo13:30
*** skramaja has quit IRC13:32
weshay|ruckchandankumar, what hash is checked out?13:33
chandankumaropendev.org/openstack/ansible-role-collect-logs checked out to:13:34
chandankumar2020-12-18 05:19:52.009491 | primary | 61e3cc33bdd9b9082299a377dc916b16782dd827 Remove --size from container command13:34
chandankumarweshay|ruck: sagi fix is there13:35
weshay|rucku sure?13:35
* weshay|ruck looks13:35
chandankumarhttps://a455f3929f61c821afa0-d5237dd035b9f3c7532839ca52b23c88.ssl.cf2.rackcdn.com/767456/2/gate/tripleo-ci-centos-8-content-provider-ussuri/2a933c6/job-output.txt13:35
chandankumar2020-12-18 05:19:52.009491 | primary | 61e3cc33bdd9b9082299a377dc916b16782dd827 Remove --size from container command13:35
*** amoralej is now known as amoralej|lunch13:36
chandankumarweshay|ruck: ^^13:37
chandankumarlet me check other passed jobs13:37
weshay|ruckchandankumar, you are debugging https://review.opendev.org/c/openstack/tripleo-ci/+/76745613:37
weshay|ruck?13:37
chandankumarweshay|ruck: nope I was rechecking the failed one13:38
chandankumarthen I was looking for collect-logs issue13:38
weshay|rucktripleo-ci-centos-8-content-provider-train13:38
weshay|ruckah and ussuri13:38
weshay|ruckya13:38
weshay|ruckchandankumar, ya.. they worked this time13:39
chandankumarnot sure what is going on there13:39
weshay|ruckchandankumar, ya.. we need to increase the pass rates on content-providers13:39
weshay|ruckchandankumar, although.. I see it's getting better.. we still need to be mindful13:40
weshay|rucklast 24 hours.. /me notes latest zuul results not included13:40
weshay|ruckcentos-8-provider 100% w/ 19 jobs13:40
weshay|ruckin gate13:40
weshay|ruckussuri / train are at 79% and 77%13:41
chandankumarweshay|ruck: kopecmartin soniya29 akahat|ruck fyi https://review.opendev.org/c/openstack/project-config/+/767665 glance tempest plugin coming13:41
weshay|ruckchandankumar, I suspect the lack of promotion13:41
weshay|ruckchandankumar, nice13:41
weshay|ruckzbr|rover, akahat|ruck let's chat13:41
akahat|ruckweshay|ruck, okay13:41
akahat|ruckchandankumar, ack13:41
rfolcoysandeep, can you tell me what is the equivalent job in downstream (code eng) to the container build ubi8 upstream ?13:41
weshay|ruckhttps://meet.google.com/gpw-bvet-zzr?authuser=113:42
rfolcoperiodic-tripleo-build-containers-rhel-8-master ?13:42
chandankumarweshay|ruck: train and ussuri is not promoted from last 14 days13:42
weshay|ruckchandankumar, ya.. I know13:42
weshay|ruckthey haven't built13:42
rfolcoysandeep, or this ? tripleo-build-containers-ubi-8-internal13:42
chandankumarweshay|ruck: container build is still stuck?13:43
rfolcoysandeep, maybe this one ? tripleo-build-containers-ubi-8-internal-rhel-8-rhos-1713:43
weshay|ruckchandankumar, yes sir13:43
chandankumaroh no13:43
ysandeeprfolco, yes this one for 1713:43
weshay|ruckchandankumar, buildah13:43
rfolcoysandeep, tripleo-build-containers-ubi-8-internal-rhel-8-rhos-17 ?13:43
ysandeeprfolco, yes13:43
rfolcoysandeep, cool, thanks13:44
ysandeeprfolco, tripleo-build-containers-ubi-8-internal-rhel-8-rhos-16.2 for 1613:44
rfolcoysandeep, ok will run both, and what about one for older branches that still use kolla ?13:44
ysandeeprfolco, afaik.. we only have 16.2 and 17 pipelines in downstream.. sshnaidm setup 13 line for operator work.. but i am not sure they have container build job there or not13:46
rfolcoysandeep, ok I think these 2 are enough13:46
rfolcoysandeep, thanks :)13:46
weshay|ruckakahat|ruck, zbr|rover https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_8f4/767679/1/check/tripleo-ci-centos-8-containers-multinode/8f43a97/logs/undercloud/etc/yum.repos.d/index.html13:50
*** jpena|lunch is now known as jpena14:00
weshay|ruckakahat|ruck, marios, on monday can we chat about http://dashboard-ci.tripleo.org/d/Z4vLSmOGk/cockpit?orgId=1&from=now-90d&to=now&var-launchpad_tags=alert&var-releases=master&var-promotion_names=current-tripleo&var-promotion_names=current-tripleo-rdo&var-influxdb_filter=branch%7C%3D%7Cstable%2Ftrain&var-influxdb_filter=job_name%7C%3D~%7C%2F.*upgrade.*%2F14:07
mariosweshay|ruck: sure... but it is centos7... i think we should ask the upgrade team. if they don't want/care, let's nuke14:13
mariosweshay|ruck: i don't think they know/want/care14:13
weshay|ruckmarios, rock.. good plan14:14
soniya29chandankumar, that's great14:14
*** amoralej|lunch is now known as amoralej14:18
rfolcochandankumar, my 'job.provider_job' when clause is skipping...  https://review.opendev.org/c/openstack/tripleo-ci/+/767591/1/playbooks/tripleo-ci/post.yaml, any thoughts ?14:21
rfolco    provider_job: true      in https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_36a/767591/1/check/tripleo-ci-centos-8-content-provider/36a02a0/zuul-info/inventory.yaml14:21
*** ysandeep is now known as ysandeep|away14:22
rfolco2020-12-17 21:22:56.530040 | TASK [Build report]14:22
rfolco2020-12-17 21:22:56.602400 | localhost | skipping: Conditional result was False14:22
chandankumarrfolco: may be the provider_job value is not passed in the post playbook14:25
chandankumarrfolco: can be make cacheable is to true somewhere?14:25
rfolcochandankumar, I thought provider_job var was a job definition in the zuul files14:26
chandankumarrfolco: yes it is14:26
rfolcochandankumar, why post can't see it then14:27
ykarelrfolco, try provider_job instead of job.provider_job, as that's how it visible to ci playbooks, for quickstart u can use job.provider_job14:27
rfolcoykarel, oh cool, thanks :)14:28
chandankumarykarel: yes, thanks :-)14:28
rfolcochandankumar, thx bro14:30
*** udesale_ has quit IRC14:38
weshay|ruckzbr|rover, did you by chance fix tripleo-tox-molecule15:05
zbr|roverweshay|ruck: which repo my memory started to become bit fuzzy15:07
weshay|ruckzbr|rover, V15:07
weshay|ruck76745615:07
zbr|roveryeah, change is fixing it but failed to merge due to unrelated reason15:08
zbr|roverit is bit tricky as we still need to fix tripleo-repos15:09
zbr|roverthat one only fixes the job and switching testing of tripleo-repos from c7 to c815:09
zbr|roverwhen ideally we should test it on both15:09
zbr|roverat this moment tripleo-repo changes are not tested against centos-7, so is easy to break it.15:12
zbr|roverin fact this is what happened15:12
zbr|roverweshay|ruck: take a look at https://review.opendev.org/c/openstack/tripleo-repos/+/674571 -- two important bits: comment from alex and the age of the change.15:13
zbr|roverone year ago alex argued that we don't need molecule testing on it, now we have failures as changes that broke it were introduced.15:14
weshay|ruckzbr|rover, raise it to the ptl15:15
weshay|ruckmarios, ^15:15
zbr|roverability to test that we can configure repos should not take more than 10 mins to run15:16
mariosk15:16
marioszbr|rover: consider adding a note in https://etherpad.opendev.org/p/tripleo-meeting-items and if folks don't show up in the meeting tuesday we can reach out for a discussion with DF15:19
zbr|roveradded a note15:23
mariosthanks15:23
marioshttp://lists.openstack.org/pipermail/openstack-discuss/2020-December/019522.html [tripleo] next meeting Tuesday Dec 22 @ 1400 UTC in #tripleo15:26
mariosPlease consider adding your news/blockers/plans in https://etherpad.opendev.org/p/tripleo-meeting-items even if you cannot attend.15:26
mariosneeds votes please https://review.rdoproject.org/r/#/c/31358/ but weshay|ruck it will have to be manually changed on the promoter anyway right?15:35
weshay|ruckmarios, no.. components don't use the promoter15:37
mariosweshay|ruck: yes i know they don't (we have job to do that) but the criteria file lives there or not at all?15:38
weshay|ruckmarios, please check.. afaik.. it's done from src in the component proomtion jobs.. rfolco would know off the top of his head15:39
mariosweshay|ruck: i mean that http://38.145.34.55/config/CentOS-8/component/ ... i guess it isn't used at all in the promoter or referenced15:39
mariosweshay|ruck: yeah i am just confused because all the criteria is cloned for the promoter there ^^^15:39
mariosweshay|ruck: even if it isn't used15:39
mariosweshay|ruck: k thanks15:39
weshay|rucknp15:40
rfolcoweshay|ruck, marios my memory chip is from 80's, don't know, need to look. I assume these criteria files under component/ are not used... not sure15:43
weshay|ruckrfolco, why would you think they are not used?15:43
weshay|rucklolz15:43
weshay|ruckjebus15:43
weshay|ruckmaybe you mean on the promoter..15:44
rfolcoyes15:44
weshay|ruckaye15:44
rfolcothe promoter15:44
weshay|ruck k15:44
weshay|ruckrfolco, the jobs you wrote iirc get criteria from src15:44
rfolcoweshay|ruck, don't remember even that I wrote those... I'm getting old15:45
weshay|ruckrfolco, take some ginseng and look :)15:45
rfolcoI don't know what had for dinner yesterday if you ask me15:45
weshay|rucklolz15:45
weshay|ruckVHAT YOU SAY YOUNG MAN15:45
weshay|ruckback in my day sunny... and other old'ism15:46
rfolco:)15:46
weshay|ruckOK.. taking bets on tripleo c8 train pass rate after being down for 2 weeks15:48
mariosall in15:48
weshay|ruckmarios, what % pass/fail15:49
mariosweshay|ruck: i read it too quickly on first pass i thought it was 'for promotion' but w/e15:50
chandankumarsee ya all on thursday, Have a nice weekend & holidays ahead :-)16:53
*** chandankumar is now known as raukadah16:53
*** rlandy has joined #oooq16:58
weshay|ruckChanServ, 0/16:59
weshay|ruckraukadah, 0/16:59
*** jfrancoa has quit IRC17:01
marioshave a good one tripleo-ci see you monday17:01
*** marios is now known as marios|out17:01
*** jpena is now known as jpena|off17:16
rfolcoweshay|ruck, pls review https://review.opendev.org/c/openstack/tripleo-ci/+/755124 --> testproject and codeeng test works, so not breaking anything. Will ask for reviews next week, hopefully we can merge by Tue/Wed.17:20
rfolcoweshay|ruck, content provider report is another story, working in a separate patch17:20
weshay|ruckah k17:20
weshay|ruckthanks for clarifying17:20
rfolcoweshay|ruck, do a first pass and if any respin I can get it ready for Monday. Thanks !!17:21
rfolcocodeeng >> https://code.engineering.redhat.com/gerrit/#/c/22175217:22
rfolcotestproject >> https://review.rdoproject.org/r/#/c/29840/17:22
*** marios|out has quit IRC17:29
*** jmasud has joined #oooq17:30
*** derekh has quit IRC17:31
*** amoralej is now known as amoralej|off17:52
*** ykarel has quit IRC18:28
weshay|ruckrfolco, I forget if the skiplist will work w/o full tempest test names19:31
weshay|ruckI think so right.. but it was just harder to remove19:32
rfolcoweshay|ruck, I always use full name, but I think it works w/ partial names too like classes of tests19:32
weshay|ruckya.. it should.. cool thanks19:33
*** jmasud has quit IRC19:43
weshay|ruckthanks fixed19:45
*** jmasud has joined #oooq19:46
*** rlandy has quit IRC20:14
*** rfolco has quit IRC20:40
*** rfolco has joined #oooq20:40
*** jmasud has quit IRC20:53
*** slaweq has quit IRC20:54
*** jmasud has joined #oooq21:11
« Thursday, 2020-12-17 Index Saturday, 2020-12-19 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!