Friday, 2020-07-31

« Thursday, 2020-07-30 Index Saturday, 2020-08-01 »
*** artom has quit IRC00:15
*** rlandy|biab is now known as rlandy00:26
rlandyarxcruz: hey - ci.centos will haunt you forever00:27
*** rfolco has quit IRC00:29
*** rlandy has quit IRC00:56
*** owalsh has joined #oooq02:29
*** owalsh_ has quit IRC02:33
*** Goneri has quit IRC03:10
*** bhagyashris|away is now known as bhagyashris03:52
*** ratailor has joined #oooq04:30
*** ykarel|away has joined #oooq04:33
*** ykarel|away is now known as ykarel04:34
*** ykarel has joined #oooq04:36
*** ratailor_ has joined #oooq04:39
*** ratailor has quit IRC04:39
*** ykarel has quit IRC04:39
*** ykarel has joined #oooq04:41
*** ykarel has quit IRC04:42
*** ykarel has joined #oooq04:43
*** ykarel has quit IRC04:47
*** ykarel has joined #oooq04:47
bhagyashrischkumar|rover, hi good morning !04:48
bhagyashrischkumar|rover, do we really need these jobs https://review.rdoproject.org/zuul/builds?job_name=tripleo-rhel-8-buildimage-overcloud-full-train&job_name=tripleo-rhel-8-buildimage-ironic-python-agent-train05:05
bhagyashristhey never passed and those are rhel based05:06
chkumar|roverbhagyashris: we need to remove those05:06
bhagyashrischkumar|rover, ack thanks05:06
bhagyashrishttps://review.rdoproject.org/r/28793 Remove rhel-8 buildimage train jobs05:25
*** udesale has joined #oooq05:27
*** jfrancoa has joined #oooq05:37
*** udesale has quit IRC05:50
*** sanjayu__ has joined #oooq05:55
chkumar|roverysandeep|away: please have a look https://review.opendev.org/#/c/743998/106:00
*** udesale has joined #oooq06:04
*** ysandeep|away is now known as ysandeep06:48
*** pojadhav|lunch is now known as pojadhav06:49
ysandeepchkumar|rover, ack06:49
*** amoralej|off is now known as amoralej07:08
*** tosky has joined #oooq07:19
bhagyashrischkumar|rover, should remove all the rhel-8 jobs in https://review.rdoproject.org/r/2879307:25
bhagyashrisshould i remove ...07:26
chkumar|roveryes07:26
bhagyashrisack07:27
*** bogdando has joined #oooq07:56
*** jpena|off is now known as jpena07:56
ysandeepchkumar|rover, I have replied on review https://review.opendev.org/#/c/743998/.. Just want to crosscheck before adding job you requested there- we have another c8 multinode job there but with a different fs - if that is enough for coverage or not..08:02
*** jbadiapa has joined #oooq08:28
zbrplease merge https://review.opendev.org/#/c/742696/408:28
*** chem has joined #oooq08:29
*** chem has quit IRC08:29
zbrchkumar|rover: weshay|ruck ^ safe to +W, and something ruck/rovers would want to have.08:29
*** chem has joined #oooq08:29
chkumar|roverzbr: +w it08:30
*** derekh has joined #oooq08:34
chkumar|roverarxcruz: kopecmartin svyas added agenda for tempest meeting https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view#2020-07-31 please add or remove08:55
svyaschkumar|rover, ack08:55
*** yolanda has quit IRC09:06
*** ysandeep is now known as ysandeep|afk09:07
*** yolanda has joined #oooq09:16
*** dtantsur|afk is now known as dtantsur09:19
*** ysandeep|afk is now known as ysandeep10:37
*** jpena is now known as jpena|afk10:56
*** zbr is now known as zbr|pto10:56
*** jmasud has quit IRC11:04
*** jpena|afk is now known as jpena11:20
ysandeepchkumar|rover, could you please review my comment https://review.opendev.org/#/c/743998/ when you have time.11:48
*** jpena is now known as jpena|lunch11:48
chkumar|roverysandeep: thanks, make sense11:51
ysandeepchkumar|rover, thank you :)11:51
*** rlandy has joined #oooq11:58
*** amoralej is now known as amoralej|lunch12:00
*** ysandeep is now known as ysandeep|brb12:03
rlandyysandeep|brb: hey - sent you an email re: 16.2 work - let me know what works for you12:09
rlandychkumar|rover: hey is marios out today?12:14
chkumar|roverrlandy: yup12:14
rlandychkumar|rover: any idea if we expect panda to be around at all today?12:15
chkumar|roverrlandy: no idea12:15
chkumar|roverI have not seen today on irc12:15
rlandychkumar|rover: you ok  with upstream? need help with something?12:15
chkumar|roverrlandy: upstream doing fine, need to land this patch https://review.opendev.org/744022 first12:16
*** ysandeep|brb is now known as ysandeep12:16
chkumar|roverbut one job failed on different patch12:16
chkumar|roverstill few jobs are hitting docker.io rate limit issue12:17
weshay|ruckzbr|pto, thanks12:17
rlandychkumar|rover: well it will be at the top next12:17
chkumar|roverrlandy: yuo12:17
chkumar|roveryup12:17
weshay|ruckchkumar|rover, we're looking good my man12:18
ysandeeprlandy, thanks! looking at image build failure now.. But will probably get more chance on monday to debug this properly.12:18
chkumar|roverrlandy: we are hitting introspection failure on fs01 component pipeline12:18
rlandyysandeep: it may be something we need from the components that have not promoted ...12:18
rlandywill work with weshay|ruck today to manually promote12:18
chkumar|roverweshay|ruck: want to sync quickly?12:18
weshay|ruckchkumar|rover, sure12:19
ysandeeprlandy, ohh okay12:19
rlandychkumar|rover: in rdo component pipeline? baremetal or tripleo?12:19
weshay|ruckhttps://meet.google.com/svf-agce-ysf12:19
chkumar|roverweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/188976412:20
openstackLaunchpad bug 1889764 in tripleo "/sbin/pcs cluster setup tripleo_cluster standalone addr=192.168.24.1 --token 10000 --encryption 1' returned 1 instead of one of [0]" [High,Triaged]12:20
weshay|ruckchkumar|rover, https://trello.com/c/PgoVYID4/1602-cixlp1887176tripleociproa-centos8-train-minor-update-failing-error-configuring-rabbitinittasks12:24
*** rfolco has joined #oooq12:24
weshay|ruckchkumar|rover, https://review.rdoproject.org/r/#/c/28704/12:25
rlandyysandeep: https://code.engineering.redhat.com/gerrit/#/c/189440/ - just fyi ... testproject where testing container builds12:25
ysandeeprlandy, ack, thanks for sharing12:26
rlandyysandeep: just waiting to see where this falls down :)12:26
weshay|ruckchkumar|rover, https://review.rdoproject.org/r/#/c/28790/12:26
chkumar|roverweshay|ruck: https://review.rdoproject.org/r/2879712:29
chkumar|roverweshay|ruck: https://review.rdoproject.org/zuul/status12:29
chkumar|roverweshay|ruck: https://opendev.org/openstack/tempest/src/branch/master/tox.ini#L4112:49
chkumar|roverweshay|ruck: https://zuul.opendev.org/t/openstack/build/2130c6c258e84abe80b69630c161759512:50
*** amoralej|lunch is now known as amoralej12:52
*** ratailor_ has quit IRC12:53
chkumar|roverweshay|ruck: arxcruz https://discuss.python.org/t/announcement-pip-20-2-release/486312:57
chkumar|roverhttps://pip.pypa.io/en/stable/news/#id112:58
*** jpena|lunch is now known as jpena12:59
weshay|ruckchkumar|rover, https://review.opendev.org/#/c/744007/212:59
chkumar|roverweshay|ruck: thanks!13:00
chkumar|roverweshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1889764 please have a look when time13:04
openstackLaunchpad bug 1889764 in tripleo "/sbin/pcs cluster setup tripleo_cluster standalone addr=192.168.24.1 --token 10000 --encryption 1' returned 1 instead of one of [0]" [High,Triaged]13:04
*** ykarel_ has joined #oooq13:12
*** ykarel has quit IRC13:15
*** jbadiapa has quit IRC13:17
rlandychkumar|rover: following up on introspection failures13:22
rlandyhttps://review.rdoproject.org/zuul/builds?pipeline=openstack-component-baremetal13:23
rlandychkumar|rover: looks not bad13:23
chkumar|roverrlandy: please have a look at tripleo componet13:24
chkumar|roverhttps://review.rdoproject.org/zuul/builds?pipeline=openstack-component-tripleo13:24
chkumar|roveroh sorry that one is not bad13:25
rlandychkumar|rover: ok - checking that one13:26
rlandychkumar|rover: ok - going to rerun to check13:27
sshnaidm|afkzbr|pto, weshay|ruck collect logs job is broken: https://review.opendev.org/#/c/744124/  https://zuul.opendev.org/t/openstack/build/75825349e145433b8fb82098d7964bb3/log/job-output.txt#613-61713:29
weshay|rucksshnaidm|afk, what's this guy here for?13:31
weshay|ruckshit13:32
*** TrevorV has joined #oooq13:33
*** pojadhav is now known as pojadhav|off13:34
weshay|rucksshnaidm|afk, thanks.. will pursue it13:34
weshay|ruckchkumar|rover, fyi.. /me writes a boog13:34
*** sanjayu__ has quit IRC13:36
chkumar|roverweshay|ruck: sure13:38
rlandyweshay|ruck: why is the branch in https://github.com/openstack/tripleo-ci/blob/master/zuul.d/build-containers.yaml#L215 master and not train?13:58
rlandybranchful13:58
rlandynvm13:59
weshay|ruckrlandy, https://zuul.openstack.org/builds?job_name=tripleo-build-containers-ubi-8-train14:00
* weshay|ruck wondering why we haven't seen that execute14:00
rlandyussuri is the same14:00
weshay|ruckthis is good.. though https://zuul.openstack.org/builds?job_name=tripleo-build-containers-ubi-8-train&job_name=tripleo-build-containers-ubi-8&branch=stable%2Ftrain14:01
rlandyweshay|ruck: so it's probably fine14:02
* rlandy just trying to get right checkout for 16.214:02
rlandyto match train where there is no downstream repo14:03
*** udesale_ has joined #oooq14:05
weshay|rucksshnaidm|afk, /me wonders how often we're seeing that collect logs failure... no way to use any tools to determine that :(14:06
*** udesale has quit IRC14:08
weshay|ruckchkumar|rover, ++ fyi https://review.opendev.org/#/c/744178/14:11
*** Goneri has joined #oooq14:11
*** ykarel_ is now known as ykarel14:11
chkumar|roverweshay|ruck: cool14:12
chkumar|roverweshay|ruck: but why stable/train is pulling stuff from master14:13
weshay|ruckchkumar|rover, we'll need cherry picks I assume14:15
*** bogdando has quit IRC14:19
weshay|ruckchkumar|rover, https://meet.google.com/ncp-wzpk-yeq14:25
chkumar|roverweshay|ruck: https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/release/tripleo-ci/CentOS-8/promotion-testing-hash-train.yml#L6614:29
weshay|ruckrlandy, chkumar|rover https://review.rdoproject.org/r/2880314:42
*** jbadiapa has joined #oooq14:45
*** rascasoft has joined #oooq14:48
weshay|ruckrlandy, chkumar|rover https://review.rdoproject.org/r/2880414:50
rlandyweshay|ruck: ^^ +2 on adding the job definitions14:51
rlandyiirc, these jobs were failing14:51
rlandywe asked ade to look in to it14:51
weshay|ruckrlandy, re: fs039?14:52
rlandyno ipa-multinode14:52
weshay|ruckoh.. https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone-on-multinode-ipa&branch=stable%2Ftrain14:52
weshay|ruckrlandy, we have a boog?14:53
rlandyweshay|ruck: possible a missing backport in THT14:53
rlandypojadhav|off was working on it14:53
weshay|ruckk14:54
*** chem has quit IRC14:57
chkumar|roverweshay|ruck: regarding security jobs https://review.rdoproject.org/r/#/c/28804/ can we add https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/project-templates-components.yaml there also so that they can get triggered14:59
chkumar|roverin the same patch?15:00
*** rlandy is now known as rlandy|brb15:00
weshay|ruckchkumar|rover, oh ya15:07
weshay|rucksorry15:07
weshay|ruckslipped my mind15:07
chkumar|roverweshay|ruck: we need this patch first https://review.opendev.org/#/c/743629/ from any other patch15:07
weshay|ruckchkumar|rover, updated.. thank you https://review.rdoproject.org/r/2880415:11
weshay|ruckchkumar|rover, we should be able to remove tripleo-build-containers-centos-815:11
chkumar|roverweshay|ruck: you sent the patch or I should send it?15:12
weshay|ruckchkumar|rover, it's late for you.. I'll do it15:12
chkumar|roverweshay|ruck: aye thanks!15:12
chkumar|roversee ya on monday then15:12
*** chkumar|rover is now known as raukadah15:12
ykarelweshay|ruck, hey15:17
ykarelweshay|ruck, i tried to run standalone on ci.centos with couple of patches with results in https://review.opendev.org/#/c/743919/15:19
ykarelto catch issues early15:19
ykarelwdyt about formalizing it?15:20
ykarelarxcruz, fyi too^15:26
*** amoralej is now known as amoralej|off15:26
*** amoralej|off is now known as amoralej15:29
*** amoralej has quit IRC15:30
*** rlandy|brb is now known as rlandy15:30
weshay|ruckykarel, AHHH look at that.. it's like ykarel  is in my head :)15:33
weshay|ruckykarel++15:33
ykarelweshay|ruck, ohkk will work next week to clear it, i did some hacks to run it like that :)15:37
*** ykarel is now known as ykarel|away15:37
weshay|ruckya.. I'll spend some time w/ it as well15:38
weshay|ruckI had it working in a very hacky way a LONG time ago.. and my patches were not mergable15:38
weshay|ruckeveryone said YUCH!15:39
ykarel|awaymay be this time will get it!15:41
*** jmasud has joined #oooq15:42
*** udesale_ has quit IRC15:47
*** irclogbot_0 has quit IRC15:56
*** irclogbot_1 has joined #oooq15:57
*** irclogbot_1 has quit IRC16:15
*** irclogbot_2 has joined #oooq16:15
*** jfrancoa has quit IRC16:25
*** rascasoft has quit IRC16:39
ysandeepweshay|ruck, sry late for review here- https://review.rdoproject.org/r/#/c/28804/ but we need override-checkout: "stable/train" , otherwise you will hit - https://bugs.launchpad.net/tripleo/+bug/188561716:47
openstackLaunchpad bug 1885617 in tripleo "CentOS 8 train jobs fail with "tripleo-podman not found" error" [High,Fix released] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm)16:47
ysandeepsending a patch to correct it16:48
weshay|ruckaye... easy to forget.. I patched the same thing in other jobs.. NICE CATCH16:49
*** jpena is now known as jpena|off17:01
ysandeepweshay|ruck, https://review.rdoproject.org/r/28805 fyi..17:01
ysandeepweshay|ruck, also i think that job will still hit - https://bugs.launchpad.net/tripleo/+bug/188763317:03
openstackLaunchpad bug 1887633 in tripleo "periodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-train job fails with pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'overcloud.ctlplane.ooo.test' ([Errno 113] EHOSTUNREACH)")" [Undecided,Triaged]17:03
ysandeepweshay|ruck, rlandy informed adee_lee about this.. i think he said he will raise it will pidone17:04
ysandeepwith* pidone17:04
rlandyyeah - will follow up17:05
ysandeeprlandy++ thanks!17:06
*** ykarel|away has quit IRC17:08
ysandeeprlandy, looking at container build now - ERROR! the role 'tripleo-container-image-build' was not found -- checking what's wrong with this role.. probably someone renamed and its pulling from master17:12
ysandeepI can only find tripleo_container_image_build17:31
*** dtantsur is now known as dtantsur|afk17:33
raukadahysandeep: do you have the logs for above issue?17:33
ysandeepand this seems added recently here https://review.opendev.org/#/c/729408/17:34
ysandeepraukadah, https://sf.hosted.upshift.rdu2.redhat.com/logs/40/189440/77/check/tripleo-build-containers-ubi-8-internal-rhel-8-rhos-16.2/ce9197f/logs/containers-build-errors.log17:34
ysandeepraukadah, and testpatch - https://code.engineering.redhat.com/gerrit/#/c/189440/17:35
raukadahlet me take a look at the job17:38
ysandeepi wonder if we should use global override-checkout: as rhos-16.2-trunk-patches instead of 'stable/train' here at https://code.engineering.redhat.com/gerrit/#/c/207802/3/zuul.d/tripleo-build-containers.yaml@37117:43
raukadahysandeep: can we remove this https://review.opendev.org/#/c/729408/17:44
raukadahit is not needed17:44
raukadahas tripleo-ci/build-containers role is involved in build the containers17:44
raukadahsorry17:45
raukadahwrong patch17:45
raukadahand ignore above message17:45
ysandeepack17:45
ysandeepraukadah, rlandy i wonder if we should use global override-checkout: as rhos-16.2-trunk-patches instead of 'stable/train' here at https://code.engineering.redhat.com/gerrit/#/c/207802/3/zuul.d/tripleo-build-containers.yaml@37117:46
rlandyysandeep: sorry - in mtg17:47
raukadahrlandy: yes, correct if required projects coming from downstream17:47
raukadahysandeep: ^^17:48
ysandeepraukadah, let me try that..17:48
weshay|ruckrlandy, got access to the internal image server17:52
ysandeepronelle i am trying your test patch with https://code.engineering.redhat.com/gerrit/#/c/207812/ , if it don't work - will revert to your original depends-on17:54
rlandyweshay|ruck: woohoo - freedom17:55
ysandeeprlandy, lets see how this goes this time - https://code.engineering.redhat.com/gerrit/#/c/189440/17:55
ysandeepsry if you are still in mtg, ignore me then17:56
rlandyysandeep: sure - sorry - was on meeting with weshay|ruck trying to get internal promoter working17:56
ysandeepno worries :)17:56
rlandyysandeep: there are some repos where there is no downstream equivalent17:57
rlandylets see what happens there17:57
ysandeep there are some repos where there is no downstream equivalent --- oh, i wasn't aware17:57
ysandeepNot sure where tripleo-container-image-build is coming from? I couldn't trace it on github search/codesearch17:59
*** ysandeep is now known as ysandeep|brb18:00
arxcruzrlandy: weshay|ruck +w https://review.opendev.org/#/c/744097/ please? :)18:00
rlandy tripleo-container-image-build triplao-ansible18:00
weshay|ruckrlandy, for arx ^18:01
rlandyweshay|ruck; reviewed - will e+18:01
rlandyw+18:01
*** ysandeep|brb is now known as ysandeep18:07
ysandeeprlandy, meh.. it failed with same error :(18:07
rlandyysandeep,: no worries  - will work on it this afternoon18:08
rlandyany luck with image build error?18:08
ysandeeprlandy, I was waiting for you guys to promote things first..18:10
ysandeepi didn't get understand your msg then properly earlier:-18:11
ysandeep~~~18:11
ysandeep<rlandy> ysandeep: it may be something we need from the components that have not promoted ...18:11
ysandeep~~~18:11
ysandeepnot sure it was in context with image jobs18:12
rlandysure18:12
weshay|ruckrlandy, neutron-mlnx?18:22
weshay|ruckit failed18:22
weshay|rucksure we're not supposed to skip that?18:22
rlandyweshay|ruck: weird18:24
rlandyshould be excluded18:24
rlandynot picking up excludes list18:24
rlandyweshay|ruck: logs still says it's running18:25
rlandychecking push log18:26
rlandylooks like it's still going18:27
weshay|ruckk18:33
weshay|ruckrlandy.. it18:39
weshay|ruckit's on swift.. so maybe we got past sahara18:39
rlandyweshay|ruck++18:39
rlandyyour git skills didn't let us down18:39
weshay|ruckonly a few hacks.. lolz18:39
weshay|ruckpst...18:39
weshay|ruckthat was ugly.. been only using gerrit18:39
weshay|ruckfailed18:42
weshay|ruckrlandy, this is correct now http://osp-trunk.hosted.upshift.rdu2.redhat.com/rhel8-osp17/current-tripleo/delorean.repo.md518:46
weshay|ruckprobably just need to update the images18:46
weshay|ruckrlandy, I've updated the images link.. where are the dockerfiles?18:49
weshay|ruckfound them.. updated18:50
weshay|ruckrlandy, rhos-17 should be good to go18:50
weshay|ruckf... the promoter18:50
* rlandy looks18:52
rlandyweshay|ruck: no18:53
weshay|ruck?18:53
rlandywrong hash18:53
rlandyoh now it's right18:53
weshay|ruckk.. fun18:53
weshay|ruck:)18:53
rlandyeed587246e6375165eeed73cc7c03dcd18:53
weshay|ruckya18:53
weshay|ruckcheck the images and docker files18:53
rlandybreathe, breathe18:53
* rlandy checks links18:54
weshay|ruckif we have that right.. we should be good18:54
rlandyimages are good18:54
rlandy "stderr_lines": ["no such manifest: docker-registry.upshift.redhat.com/tripleorhos-17/rhel-binary-neutron-mlnx-agent:eed587246e6375165eeed73cc7c03dcd"],18:55
rlandy^^ we should exclude that18:56
rlandyweshay|ruck: looks better though18:56
rlandythank you18:56
rlandyysandeep: rechecked build-images patch now that we promoted18:58
ysandeeprlandy, thanks!18:58
rlandyweshay|ruck: so in the end you just manually promoted?18:58
weshay|ruckrlandy, no.. I suspect dlrn just took time to update current-tripleo18:58
weshay|ruckit has to go hash by hash18:59
weshay|ruckit may be on a slow server internally?18:59
ysandeeprlandy, will check on monday if it fails.. Regarding container build my intital thoughts are maybe - we never tested override-checkout works with downstream branches and it don't works well with downstream branches.. I found a todo which make me suspect more about it - https://code.engineering.redhat.com/gerrit/gitweb?p=openstack/osp-jobs.git;a=blob;f=zuul.d/project-templates.yaml#l6219:02
rlandyysandeep: yeah - first experience with overrides in downstream19:03
ysandeeprlandy, okay, I am going to log off now19:04
ysandeeprlandy, weshay|ruck happy weekend o/19:04
*** ysandeep is now known as ysandeep|away19:05
rlandybye19:06
weshay|ruckysandeep|away, thanks for the patch!19:09
rlandyraukadah: weshay|ruck: https://review.rdoproject.org/r/#/c/21072/ two tripleo OVB pipeline jobs - rerun passed19:16
rlandyso I think it's ok19:16
weshay|ruckrlandy, is that w/ cr?19:17
weshay|ruckwhat's the context?19:17
rlandyweshay|ruck: "<chkumar|rover> rlandy: please have a look at tripleo componet"19:18
weshay|ruckoh19:18
weshay|ruckmaster19:18
rlandyweshay|ruck: chandan asked me to look at the OVB failure on the tripleo component19:18
rlandyboth look ok now19:18
weshay|ruckk19:18
rlandyysandeep|away: https://code.engineering.redhat.com/gerrit/#/q/topic:image-build-fix1+(status:open+OR+status:merged) will fix image build errors19:49
*** TrevorV has quit IRC20:20
*** owalsh has quit IRC20:24
*** owalsh has joined #oooq20:45
*** rfolco has quit IRC21:01
*** owalsh has quit IRC21:17
*** owalsh has joined #oooq21:32
*** tosky has quit IRC22:10
*** rlandy has quit IRC23:02
« Thursday, 2020-07-30 Index Saturday, 2020-08-01 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!