Wednesday, 2020-03-04

*** jmasud has joined #oooq00:42
*** jmasud has quit IRC00:53
*** jmasud has joined #oooq00:55
*** owalsh_ has quit IRC02:09
*** owalsh has joined #oooq02:29
*** Goneri has quit IRC02:41
*** dsneddon has quit IRC03:06
*** apetrich has quit IRC03:18
*** apetrich has joined #oooq03:56
*** rlandy has quit IRC04:03
*** owalsh_ has joined #oooq04:52
*** owalsh has quit IRC04:53
*** jmasud has quit IRC04:53
*** owalsh_ has quit IRC04:53
*** jmasud has joined #oooq04:55
*** owalsh has joined #oooq04:59
*** udesale has joined #oooq05:07
*** raukadah is now known as chandankumar05:09
*** skramaja has joined #oooq05:12
*** ykarel|away is now known as ykarel05:25
*** udesale_ has joined #oooq05:38
*** udesale has quit IRC05:40
*** dpawlik has joined #oooq06:33
*** udesale has joined #oooq06:34
*** udesale_ has quit IRC06:36
*** marios|ruck has joined #oooq06:37
*** jbadiapa has joined #oooq06:38
*** udesale_ has joined #oooq06:42
*** udesale has quit IRC06:45
*** udesale has joined #oooq06:46
*** udesale_ has quit IRC06:46
*** jfrancoa has joined #oooq06:52
*** saneax has joined #oooq06:53
*** hamzy has quit IRC07:04
*** hamzy has joined #oooq07:05
*** udesale_ has joined #oooq07:12
*** udesale has quit IRC07:15
*** saneax has quit IRC07:20
*** saneax has joined #oooq07:27
*** ykarel is now known as ykarel|lunch07:45
*** tesseract has joined #oooq08:09
*** hamzy has quit IRC08:12
*** hamzy has joined #oooq08:13
*** jmasud has quit IRC08:13
*** jmasud has joined #oooq08:20
*** amoralej|off is now known as amoralej08:22
*** tosky has joined #oooq08:32
*** bogdando has joined #oooq08:42
*** jpena|off is now known as jpena08:48
*** ykarel|lunch is now known as ykarel08:50
*** jmasud has quit IRC08:53
*** jmasud has joined #oooq08:56
*** udesale_ has quit IRC09:08
*** hamzy has quit IRC09:34
marios|ruckneeds reviews please https://review.opendev.org/#/c/710836/ https://bugs.launchpad.net/tripleo/+bug/186495309:34
openstackLaunchpad bug 1864953 in tripleo "(intermittent/race?) Image prepare failed: [Errno 17] File exists" [Medium,In progress] - Assigned to Bogdan Dobrelya (bogdando)09:34
*** hamzy has joined #oooq09:34
*** derekh has joined #oooq09:40
*** apetrich has quit IRC09:52
*** ChanServ sets mode: +o panda09:53
*** panda changes topic to "Docs: https://docs.openstack.org/tripleo-quickstart/latest/ || 4th Mar promoter patch to review https://review.rdoproject.org/r/24716 || next promoter patch in the chain https://review.rdoproject.org/r/24721""09:54
*** panda sets mode: -o panda09:54
*** ccamacho has quit IRC10:02
*** hamzy has quit IRC10:03
*** hamzy has joined #oooq10:04
*** hamzy has quit IRC10:06
*** hamzy_ has joined #oooq10:06
*** dtantsur|afk is now known as dtantsur10:09
*** hamzy has joined #oooq10:20
*** hamzy_ has quit IRC10:21
chandankumarsshnaidm, we are testing undercloud hang fix on ovb10:21
sshnaidmchandankumar, ack10:21
*** hamzy has quit IRC10:26
*** hamzy has joined #oooq10:28
*** holser has joined #oooq10:28
*** holser has quit IRC10:29
*** holser has joined #oooq10:29
*** hamzy has quit IRC10:33
*** jbadiapa has quit IRC10:39
*** ccamacho has joined #oooq10:58
sshnaidmchandankumar, do you know what is going on with c8 ovb images?11:03
sshnaidmchandankumar, where should we have them?11:03
chandankumarsshnaidm, you mean overcloud images na? https://github.com/rdo-infra/review.rdoproject.org-config/commit/127bacb1fc722af0c5bfdfd70e49b7b2dd6ac5b0 rlandy fixed it last night, build image jobs depend on this https://review.opendev.org/#/c/708599/ or may be I missed something11:09
sshnaidmchandankumar, ack, and when are we going to push them in periodics? I don't see a centos8 specific folder on images server yet11:11
chandankumarsshnaidm, we have periodic jobs defined https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/zuul.d/tripleoci.yaml#L37011:21
chandankumarwe need to add it to promotion pipeline but depends on abive patch11:21
chandankumarsshnaidm, may be we can run test project changes and push it there?11:22
sshnaidmchandankumar, do you remember where is uploading script?11:23
chandankumarsshnaidm, https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/ci-scripts/tripleo-upstream/upload-cloud-images.sh11:23
sshnaidmchandankumar, if we run these jobs we'll override c7 images by c811:24
chandankumarsshnaidm, I think I have pushed images there11:25
sshnaidmchandankumar, when?11:25
chandankumarsshnaidm, https://review.rdoproject.org/r/#/c/24997/11:25
sshnaidmchandankumar, in which patchset exactly?11:26
chandankumarnope I think11:26
chandankumarjust built it11:26
chandankumarsshnaidm, https://review.rdoproject.org/r/#/c/24997/1411:26
chandankumarcheck patch 1411:27
sshnaidmchandankumar, I don't see it's being pushed11:29
chandankumarsshnaidm, yes, sorry11:30
sshnaidmchandankumar, ok, so if we are gonna push them, we could overwrite c7 images11:30
chandankumarlet me add a testproject and see11:30
sshnaidmchandankumar, but don't push them please, it will be a mess..11:30
sshnaidmchandankumar, actually it will be fine when we change all c7 ovb jobs to be c8, then it will depend on release, master images will be c8, all others c711:31
chandankumarsshnaidm, yes11:32
*** openstackstatus has joined #oooq11:50
*** ChanServ sets mode: +v openstackstatus11:50
*** apetrich has joined #oooq12:01
chandankumarsshnaidm, https://logserver.rdoproject.org/68/25668/1/check/periodic-tripleo-centos-8-buildimage-ironic-python-agent-master/7f0479b/job-output.txt12:03
chandankumarsshnaidm, can you check this image is there or not in image server 423cb54e84da6ddf5c163d1a5c1d512b12:04
sshnaidmchandankumar, https://images.rdoproject.org/master/rdo_trunk/12:05
sshnaidmchandankumar, I don't see it12:05
chandankumardpawlik, ^^12:06
sshnaidmchandankumar, dpawlik should it be here? https://images.rdoproject.org/centos8/12:06
sshnaidmdpawlik, can you please check - what is base URL for cento8 images?12:07
*** dsneddon has joined #oooq12:09
chandankumarsshnaidm, regarding this https://review.opendev.org/711081 for periodic jobs, do we want to build the image or just download it and reuse that?12:25
chandankumarjust like rhel-812:25
sshnaidmchandankumar, in most of regular jobs - yes, but there are jobs that build image during the run, like in repos tripleo-common, tripleoclient, etc12:27
sshnaidmchandankumar, when we test actually building image code for example12:27
sshnaidmchandankumar, although we have these image build jobs for that12:28
sshnaidmchandankumar, but anyway, they only build the image, but not test it like OVB job does12:28
sshnaidmchandankumar, marios|ruck do you know what is the reason of fs001 ovb failures from 02 Mar?12:31
*** jpena is now known as jpena|lunch12:32
chandankumarsshnaidm, earlier undercloud deploy was hanging12:32
chandankumaron all places12:32
sshnaidmchandankumar, it's overcloud deploy that fails there12:32
chandankumarsorry not seen that error12:33
chandankumarsshnaidm, I think we need to sync this one https://review.opendev.org/#/c/711081/4/roles/build-images/templates/overcloud-image-build.sh.j2 with https://review.opendev.org/#/c/708599/42/roles/oooci-build-images/templates/build-images.sh.j212:35
chandankumarhttps://logserver.rdoproject.org/32/25332/31/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master/242141b/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz12:35
chandankumarit is still taking c7 stuff12:35
marios|rucksshnaidm: no i do not... i noticed the fs2 upload is failing for centos7 but didn't check further (generally not been looking since those were removed from criteria)12:36
sshnaidmchandankumar, yeah, I think the plan was just to use oooci-build-images role instead of current build-images on ovb12:36
* chandankumar will connect from home back in an hour12:37
*** saneax has quit IRC12:39
marios|rucksshnaidm: looks like same root cause12:40
marios|rucksshnaidm: https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/83794af/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz12:40
marios|rucksshnaidm: 2020-03-04 03:44:08 |     Error: /Stage[main]/Tripleo::Profile::Pacemaker::Ovn_dbs_bundle/Pacemaker::Constraint::Order[ip-192.168.24.18-with-ovndb_servers]/Pcmk_constraint[order-ovn-dbs-bundle-ip-192.168.24.18]/ensure: change from 'absent' to 'present' failed: pcs -f /var/lib/pacemaker/cib/puppet-cib-backup20200304-9-df720q create constraint failed: Error: Resource 'ip-192.168.24.18' does not12:40
marios|ruckexist12:40
marios|rucksshnaidm: https://bugs.launchpad.net/tripleo/+bug/186603112:40
openstackLaunchpad bug 1866031 in tripleo "periodic centos7 fs2 master upload fails overcloud deploy pcs "create constraint failed"" [Critical,Triaged]12:40
sshnaidmmarios|ruck, see #tripleo12:40
*** dsneddon has quit IRC12:41
*** ykarel is now known as ykarel|afk12:42
*** jmasud has quit IRC12:54
*** jmasud has joined #oooq12:55
*** rlandy has joined #oooq12:56
rlandychandankumar: weshay|ruck: the platform meeting is in a few so I didn't book a sync meeting today - do you want to meet?12:57
rlandywe can for 1/2 hour before retro12:58
*** marios|ruck is now known as marios|ruck|call13:01
sshnaidmEmilienM, and seems like linters woke up..13:02
*** udesale has joined #oooq13:03
rlandyweshay|ruck: hey - gm13:04
chandankumarrlandy, weshay|ruck we cleared undercloud hang issue13:05
chandankumarit is gone now13:05
rlandychandankumar: amazing!!!13:05
rlandychandankumar: zuul is killing us now though13:06
chandankumarweshay|ruck, rlandy thanks to amoralej jpena|lunch ykarel|afk apevec for fixing that13:06
rlandycan't get anything through :(13:06
rlandychandankumar: want to sync?13:07
rlandyI didn't book it due to platform meeting but we can13:07
chandankumarrlandy, Not much updates today except undercloud13:08
weshay|ruckchandankumar, kopecmartin https://review.opendev.org/#/c/711185/13:08
rlandychandankumar: put in the component pipeline review for what we have ... can you review https://review.opendev.org/#/c/71115613:08
weshay|rucksoniya29, ^13:08
rlandyhttps://review.rdoproject.org/r/#/c/25649/13:09
rlandyit's somewhata  duplicate of what we have in standalone but needed13:09
sshnaidmchandankumar, https://review.opendev.org/#/c/710179 is working, good to go13:09
sshnaidmchandankumar, sa 010 failed there, worth to rekick ^13:10
rlandyperiodic-tripleo-ci-centos-8-scenario001-standalone-master - also passed13:11
rlandyperiodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-master13:11
rlandypassed13:11
chandankumarsshnaidm, rlandy tripleo-ci-centos-7-scenario010-standalone can we move it to c8?13:12
chandankumarit is working fine13:12
chandankumarjust needs one patch13:12
chandankumarto add it in gates13:12
sshnaidmchandankumar, probably13:12
rlandychandankumar: we have that patch ... getting13:13
rlandychandankumar: https://review.opendev.org/#/c/710856/13:13
marios|ruck|callweshay|ruck: what went wrong with the reporting after green build at https://ci.centos.org/view/rdo/view/promotion-pipeline/job/rdo_trunk-promote-train-current-tripleo/130/13:14
rlandyoh wait wrong job13:14
chandankumarrlandy, https://review.rdoproject.org/r/#/c/25667/13:14
weshay|ruckmarios|ruck|call, check weirdo jobs13:14
rlandychandankumar: we don't - will add that13:14
weshay|ruckrlandy++13:14
rlandy?13:15
weshay|ruckmarios|ruck|call,13:15
weshay|ruck2020-03-04 04:19:56.830437 | primary | CONFLICT (content): Merge conflict in docker/keystone/keystone-base/Dockerfile.j213:15
weshay|ruck2020-03-04 04:19:56.830445 | primary | error: Failed to merge in the changes.13:15
weshay|ruck2020-03-04 04:19:56.830451 | primary | Patch failed at 0001 DNM: Add dnf modules for rhel8/py3 containers13:15
weshay|ruck2020-03-04 04:19:56.830456 | primary | Use 'git am --show-current-patch' to see the failed patch13:15
weshay|ruck2020-03-04 04:19:56.830462 | primary |13:15
weshay|ruck2020-03-04 04:19:56.830468 | primary | Resolve all conflicts manually, mark them as resolved with13:15
weshay|ruck2020-03-04 04:19:56.830473 | primary | "git add/rm <conflicted_files>", then run "git rebase --continue".13:16
weshay|ruck2020-03-04 04:19:56.830480 | primary | You can instead skip this commit: run "git rebase --skip".13:16
weshay|ruck2020-03-04 04:19:56.830485 | primary | To abort and get back to the state before "git rebase", run "git rebase --abort".13:16
weshay|ruckhttps://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-rhel-8-train-containers-build-push/3945c21/job-output.txt13:16
weshay|ruckmarios|ruck|call, looks like we need to rebase13:16
*** Goneri has joined #oooq13:16
marios|ruck|callweshay|ruck: yeah i know its on the etherpad (rhel8 kolla patch thing)13:16
marios|ruck|callweshay|ruck: i tried w/out the patch but we need it... and centos fails cos missing dlrn13:17
marios|ruck|callweshay|ruck: testproject https://review.rdoproject.org/r/2566313:17
weshay|ruckk.. thanks13:19
rlandychandankumar: opinion ... https://review.opendev.org/#/c/708599/  tripleo-ci-centos-7-scenario010-standalone failed this patch twice ....13:19
rlandywhat do you think about marking that job non-voting in this patch13:20
rlandyand then we can replace it with c8 shortly13:20
weshay|ruckrlandy, chandankumar did we get the updated rpm? https://review.rdoproject.org/r/#/c/25647/13:20
chandankumartempest failure, make it nv and move to c8 directly13:20
rlandychandankumar: k - will just put up the c8 patch13:20
chandankumarweshay|ruck, https://review.rdoproject.org/r/#/c/25332/ -> undercloud twice success13:21
rlandyweshay|ruck: yeah13:21
rlandyweshay|ruck: scen10 on centos-7 is killing us now in zuul13:21
rlandygetting that moved to 813:21
*** jbadiapa has joined #oooq13:23
*** amoralej is now known as amoralej|lunch13:24
marios|ruck|callweshay|ruck: :)13:24
weshay|ruckstupid call13:24
marios|ruck|callweshay|ruck: less enthusiastically next time, if possible13:24
marios|ruck|call:D13:24
marios|ruck|callweshay|ruck: it should be 'only come if you're not green' really though13:25
weshay|ruckpoor kermit, always left out13:26
weshay|ruckhttps://trello.com/b/0VFswmht/pcci-retrospective?menu=filter&filter=UniSprint2213:27
chandankumarrlandy, weshay|ruck status added https://hackmd.io/HrQd03c9SxOMtFPFrq50tg?view#Unpinning-Ussuri13:27
weshay|ruckFAK13:28
weshay|ruckwhy are there more than one label again13:28
rlandywasn't me13:29
chandankumarsorry I am bad at editing13:29
weshay|ruckhttps://trello.com/b/0VFswmht/pcci-retrospective?menu=filter&filter=label:UniSprint2213:30
chandankumarweshay|ruck, rlandy check https://review.rdoproject.org/zuul/status and add review number 25612 and see13:30
*** skramaja has quit IRC13:31
rlandychandankumar++13:31
rlandythat's cool13:31
sshnaidmweshay|ruck, is there a mtg?13:31
sshnaidmand bluejeans13:31
*** ykarel|afk is now known as ykarel13:32
weshay|ruckarxcruz, kopecmartin retro13:33
arxcruzweshay|ruck:  joining13:33
weshay|ruckpanda, retro13:33
kopecmartinweshay|ruck: joined13:33
*** jpena|lunch is now known as jpena13:33
weshay|ruckhttps://trello.com/b/0VFswmht/pcci-retrospective?menu=filter&filter=label:UniSprint2213:34
*** amoralej|lunch is now known as amoralej13:53
sshnaidmchandankumar, can you maybe to copy these changes to build-images role? Seems like we are gonna still use it for a while13:56
chandankumarsshnaidm, yes go ahead13:56
sshnaidm:)13:56
chandankumarsshnaidm, may be taking a llok at https://logserver.rdoproject.org/68/25668/1/check/periodic-tripleo-centos-8-buildimage-ironic-python-agent-master/7f0479b/build_images.sh it has clear stuff14:03
*** chem has quit IRC14:03
*** chem has joined #oooq14:04
*** arxcruz is now known as arxcruz|rover14:13
arxcruz|rovermarios|ruck|call: u want to sync today or tomorrow morning?14:15
sshnaidm"Failed to download metadata for repo 'RDO-CentOS8-deps'" <- https://71d6e15cf166f312dff0-9dc687b1ca3b864db38ca15564d88e40.ssl.cf2.rackcdn.com/711201/3/check/tripleo-ci-centos-8-standalone/c1733cf/job-output.txt14:17
sshnaidmykarel, jpena ^^14:17
*** ccamacho has quit IRC14:18
*** ccamacho has joined #oooq14:20
*** ccamacho has quit IRC14:20
*** ccamacho has joined #oooq14:23
chandankumarjpena, How can I see all the jobs running against patch in a single view?14:24
chandankumarjpena, https://review.rdoproject.org/r/#/c/25612/ for this review14:24
*** marios|ruck|call is now known as marios|ruck14:25
marios|ruckarxcruz|rover: fyi that is updated https://etherpad.openstack.org/p/ruckroversprint2214:25
chandankumardoes some one sent patch to make scenario 10 c7 nv?14:25
marios|ruckarxcruz|rover: sure let's do a quick sync now if you lik? or in 10 mins?14:25
weshay|ruckchandankumar, ok.. so what review is that screenshot from?14:26
ykarelsshnaidm, happening multiple places?14:26
ykarelmay be a mirror issues14:26
chandankumarweshay|ruck, https://review.rdoproject.org/r/#/c/25612/14:26
chandankumarhttps://review.rdoproject.org/zuul/ we can enter the review number we can find that14:26
sshnaidmykarel, just saw it14:26
chandankumarit was so big14:26
sshnaidmykarel, let's keep an eye on it14:26
ykarelsshnaidm, ack14:26
weshay|ruckchandankumar++ nice14:26
arxcruz|rovermarios|ruck: up to you man, i'm free now, just need to grab some water14:27
weshay|ruckchandankumar, k.. and triggered w/ https://review.rdoproject.org/r/#/c/25659/14:27
chandankumarweshay|ruck, yes need some love14:28
chandankumarbut waiting on upstream job coverage14:28
chandankumarsshnaidm, https://review.opendev.org/#/c/711238/1/toci-quickstart/config/testenv/ovb-rdocloud.yml any reason to hardcoding image hash?14:29
marios|ruckarxcruz|rover: k grabbing coffee and lets do quick call14:29
sshnaidmchandankumar, just for testing14:30
chandankumarsshnaidm, aye then14:30
chandankumarweshay|ruck, rlandy We got C8 standalone and panda promoter patch queue clearing, what a great sprint :-)14:30
sshnaidmchandankumar, actually periodic jobs should use triple-ci-testing.. so no need to use hash actually14:31
sshnaidmchandankumar, will change next patch14:31
chandankumarsshnaidm, ok14:31
arxcruz|rovermarios|ruck: just waiting for you :)14:33
marios|ruckarxcruz|rover: yes14:34
marios|ruckarxcruz|rover: lets14:34
marios|ruckarxcruz|rover: https://meet.google.com/jzr-bxcy-urq14:34
*** marios|ruck is now known as marios|ruck|call14:35
rlandychandankumar: is the promoter using the promote criteria?14:36
rlandywe need to update that to include all the jobs we've added14:37
chandankumarrlandy, not looked at that part14:37
chandankumarrlandy, I am pushing patch  to make s10 nv14:37
rlandychandankumar: ok - I'll do that update today14:37
chandankumarweshay|ruck, do we want to talk about package manifest list at prod chain call?14:38
rlandychandankumar: maybe we should make https://review.opendev.org/#/c/708599/ depends-on your patch to remove tripleo-ci-centos-7-scenario010-ovn-provider-standalone14:38
rlandykilling us - and we need that in14:38
weshay|ruckchandankumar, more detail please.. I think so14:39
rlandyand the multinode patch14:39
chandankumarweshay|ruck, apevec knows that14:39
chandankumarweshay|ruck, ansible 2.8 issue example14:39
weshay|ruckright.. what change are you looking for?14:39
weshay|ruckor what info are you trying to expose14:39
chandankumarweshay|ruck, keeping the same deps package version at both places14:40
weshay|rucksorry.. I know we briefly chatted about this14:40
chandankumarupstream and downstream14:40
weshay|ruckheh14:40
weshay|ruckyes..  fine to raise it again here.. constant complaint of mine14:40
chandankumarpublishing  a package list from each puddle and from promoted component from upstream side14:40
weshay|ruckhold for my prompt14:41
weshay|ruckchandankumar, agenda is here: https://docs.engineering.redhat.com/pages/resumedraft.action?draftId=126301776&draftShareId=66532c0f-ce0b-4056-9328-d6fa2e54f6de14:41
*** marios|ruck|call is now known as marios|ruck14:41
chandankumarrlandy, looking14:42
chandankumarweshay|ruck, please add the same, low network connectivity14:45
chandankumarrlandy, https://review.opendev.org/711242 please take over14:45
*** ykarel is now known as ykarel|away14:46
rlandychandankumar: ack - will do14:46
*** dtantsur is now known as dtantsur|brb14:46
chandankumarweshay|ruck, another example for package manifest python3-secretstorage undercloud hang14:46
marios|ruckarxcruz|rover: fyi cix call in 45 mins i'll join but might be good idea to track it for update14:50
EmilienMchandankumar: FYI https://gitlab.com/takluyver/jeepney/-/merge_requests/1314:50
EmilienMalex is testing it today14:50
arxcruz|rovermarios|ruck: ack14:50
EmilienMchandankumar: FYI https://gitlab.com/takluyver/jeepney/-/merge_requests/12 was his PR but maintainers are proposing another fix14:50
chandankumarEmilienM, we fixed the hang by downdraging , looking forward with this pr14:51
EmilienMok14:51
chandankumarEmilienM, Alex thanks :-)14:51
chandankumarweshay|ruck, now 314:54
chandankumarweshay|ruck, c7 https://review.opendev.org/#/c/710913/ and https://review.opendev.org/#/c/688958/14:56
pandaarxcruz|rover: thanks I was really missing the sound of the '90s boy bands' pop14:57
arxcruz|roverpanda: anytime ;)14:58
*** udesale_ has joined #oooq14:59
*** udesale has quit IRC15:02
sshnaidmweshay|ruck, https://review.rdoproject.org/r/#/c/25362/15:03
weshay|ruck+215:03
weshay|ruckchandankumar++15:05
weshay|ruckchandankumar, I've never seen this much acceptance of moving forward on this topic15:05
chandankumarweshay|ruck, I have done nothing15:13
weshay|ruckwell.. I've been complaining about it for years..15:14
chandankumarweshay|ruck, rlandy sorry need to leave early, tomorrow need to attend coc us pycon workshop so early15:19
chandankumarsee ya tomorrow15:19
*** chandankumar is now known as raukadah15:20
rlandyraukadah: sure - chat tomorrow15:20
*** jfrancoa has quit IRC15:25
arxcruz|roverweshay|ruck: marios|ruck https://hackmd.io/7MBqFHurTA2e5H8kYRwgag i'll update the cockpit soon15:29
*** marios|ruck is now known as marios|ruck|call15:30
marios|ruck|callack arxcruz|rover15:30
raukadahweshay|ruck, I was a good meeting, I would try to join more?15:35
rlandyraukadah: -1'ed the non-voting - testing the revert while putting up c815:36
rlandybetter way to go15:36
raukadahrlandy, aye thanks!15:36
weshay|ruckarxcruz|rover, soniya29  https://review.opendev.org/#/c/711185/1/tempest/scenario/test_volume_boot_pattern.py15:40
weshay|ruckplease review ^15:40
*** jfrancoa has joined #oooq15:40
soniya29weshay|ruck, sure15:40
arxcruz|roverweshay|ruck:  it's on my list15:40
raukadahrlandy, weshay|ruck sshnaidm as per Alex we are good with this patch https://review.opendev.org/#/c/710913/ it will clear c715:41
raukadahcentos8 scenario 715:41
rlandyraukadah: -1 Carlos Goncalves15:46
raukadahrlandy, sorry another task, it is going to be debatable15:46
raukadahrlandy, we need c7 jobs definition also15:46
*** TrevorV has joined #oooq15:46
rlandyraukadah: fine - I'll add that this afternoon15:46
raukadahrlandy, cool, thanks mate :-)15:47
*** marios|ruck|call is now known as marios15:47
arxcruz|roverweshay|ruck: marios|ruck|call i'm leaving for today, i'll be half hour late tomorrow, due the office change15:47
arxcruz|roverbut still earlier15:47
mariosarxcruz|rover: ack have a good one15:47
mariosarxcruz|rover: just let me know if you need further sync15:47
weshay|ruckarxcruz|rover, k15:48
raukadahjpena, sshnaidm soemthing got broken on zuul side https://logserver.rdoproject.org/32/25332/33/check/periodic-tripleo-ci-centos-8-singlenode-featureset027-master/33b3609/job-output.txt15:49
raukadah "msg": "Destination directory /home/zuul/workspace/logs does not exist"15:49
sshnaidmraukadah, it's not zuul side..15:49
sshnaidmraukadah, "msg": "Failed to download metadata for repo 'RDO-CentOS8-deps'",15:49
sshnaidmykarel|away, jpena happens again ^^15:49
raukadahah I see openvswitch15:50
sshnaidmand it happened a few times today15:50
raukadahthere is some deps change happened15:50
raukadahmay be amoralej ^^ knows15:50
sshnaidmand we have this job in gate, seems like current it blocks gates..15:50
amoralejsshnaidm, i think that should be fixed now15:50
sshnaidmweshay|ruck, ^^15:50
amoralejis it still failing?15:50
raukadahamoralej, https://logserver.rdoproject.org/32/25332/33/check/periodic-tripleo-ci-centos-8-singlenode-featureset027-master/33b3609/job-output.txt15:51
amoralejsshnaidm, could  you retry?15:51
raukadahrecent failure15:51
sshnaidmraukadah, let's rerun it? &15:51
raukadahsshnaidm, aye15:51
weshay|rucksshnaidm, which job is blocking gates?15:51
sshnaidmweshay|ruck, centos815:51
sshnaidmstandalone15:52
weshay|rucksshnaidm, that's on retry limit15:52
sshnaidmweshay|ruck, raukadah is checking if it's still an issue ^^15:52
weshay|ruckoh we have a log though15:52
* weshay|ruck looks15:52
amoralejweshay|ruck, it's related to issue in deos affecting multi-node roles15:52
amoralejit should be fixed now15:52
amoralejwas my fault15:52
weshay|ruckamoralej, dep change?15:52
amoralejand the fact that multi-node role do not use delorean-deps.repo15:53
raukadahsshnaidm, amoralej rekicked the patch15:53
amoralejbut hardcoded baseurl15:53
weshay|ruckamoralej, yes we do15:53
weshay|ruckoh.. on the rdo-info test jobs15:53
weshay|ruckraukadah, fix amoralej's jobs please15:53
weshay|ruckamoralej, fs16-19 have not been in play since rocky I think15:54
raukadahsshnaidm, amoralej https://review.rdoproject.org/zuul/stream/4c4933252281491b9fde90e560c51fcd?logfile=console.log15:54
amoralejweshay|ruck, http://eavesdrop.openstack.org/irclogs/%23openstack-infra/latest.log.html#t2020-03-04T15:29:5315:54
amoralejshould be fixed since some time ago15:54
raukadahweshay|ruck, we fix that15:54
sshnaidmamoralej++15:55
raukadahrlandy, weshay|ruck https://review.opendev.org/#/c/710913/ c7 cleared15:55
raukadah*s715:55
amoralejraukadah, install openvswitch task passed15:56
raukadahbut gate will -2 sorry s10 issue15:56
raukadahamoralej, thanks!15:56
weshay|rucknice16:00
*** dtantsur|brb is now known as dtantsur16:00
raukadahweshay|ruck, can we update channel topic related to scenario10 issue16:02
raukadahit is going to block all gate patches16:02
raukadahrlandy, ^^16:02
*** Trevor_V has joined #oooq16:02
rlandyraukadah: https://review.opendev.org/#/c/71125016:02
rlandyweshay|ruck: ^^16:03
rlandyhttps://review.opendev.org/#/c/711252/16:03
rlandytesting there16:03
raukadahwant to go for force merge?16:03
raukadahrlandy, so big revert16:03
rlandyraukadah: it was the patch ykarel|away  pointed out16:03
rlandythe timing of the failure looks to match16:04
rlandyI -1'ed the revert to test it16:04
rlandybut up to you and weshay|ruck if we want to force that16:04
raukadahrlandy, /me is ok to force if weshay|ruck agrees16:05
rlandyraukadah: also started seeing standalone-c8 failures :(16:05
raukadahotherwise we need to go for abandon and restore game16:05
rlandyjust killed my patch in gates16:05
rlandynot consistent yet but watching it16:05
*** TrevorV has quit IRC16:06
weshay|ruckrlandy, raukadah what's up?16:08
weshay|ruckforce merge what?16:09
weshay|ruckhttps://review.opendev.org/#/c/711250 ?16:09
rlandyweshay|ruck: the revert16:10
rlandyyeah16:10
weshay|ruckwhy force merge?16:11
rlandybecause it's killing gate16:11
weshay|ruckhttps://meet.google.com/kcx-buvr-ptx?authuser=116:11
raukadahweshay|ruck, all patches are going with -2 so in gate16:11
rlandyweshay|ruck: it's passed zuul16:11
weshay|ruckhappens16:11
weshay|ruckshow me16:12
weshay|ruckit's a tall order to force merge16:12
rlandyjoining16:12
raukadahhttps://review.opendev.org/#/c/711156/16:12
raukadahhttps://review.opendev.org/#/c/710910/16:12
raukadahhttps://review.opendev.org/#/c/710179/16:14
raukadahrlandy, weshay|ruck ^^ above examples more to come soon16:14
weshay|ruckraukadah, let's propose the centos-8-standalone job on puppet-octavia16:14
raukadahrlandy, ^^16:15
rlandyyeah - I'm on it16:15
raukadahweshay|ruck, can I join the call?16:15
raukadahrlandy, aye16:16
weshay|ruckraukadah, is a mirror issue https://review.opendev.org/#/c/711156/16:16
weshay|ruckraukadah, we have a bug?16:17
raukadahweshay|ruck, yes retry limit yes16:17
raukadahweshay|ruck, nope16:17
raukadahno bug filed16:17
raukadahit started poping up npw16:17
raukadahsee ya16:24
weshay|ruckraukadah, thanks16:25
rlandybug for scen10?16:27
*** marios is now known as marios|out16:37
*** udesale_ has quit IRC16:41
*** marios|out has quit IRC16:42
sshnaidmraukadah, another fix for ovb, arghh16:43
weshay|ruckrlandy, raukadah if there is no bug.. I'll try to write one now16:45
rlandythought raukadah made one?16:48
* weshay|ruck looks16:48
weshay|ruckhttp://dashboard-ci.tripleo.org/d/jobs/jobs-exploration?orgId=1&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-8-standalone&var-influxdb_filter=pipeline%7C%3D%7Cgate16:50
weshay|ruck20-03-04 12:57:38.697153 | TASK [multi-node-bridge : Install openvswitch]16:51
weshay|ruck2020-03-04 12:57:41.133293 | primary | ERROR16:51
weshay|ruck2020-03-04 12:57:41.136788 | primary | {16:51
weshay|ruck2020-03-04 12:57:41.136910 | primary |   "msg": "Failed to download metadata for repo 'RDO-CentOS8-deps'",16:51
weshay|ruck2020-03-04 12:57:41.137021 | primary |   "rc": 1,16:51
weshay|ruck2020-03-04 12:57:41.137129 | primary |   "results": []16:51
weshay|ruck2020-03-04 12:57:41.137236 | primary | }16:51
weshay|ruckis that the issue? ^16:51
rlandylooking16:52
weshay|ruckI only see two gate failures on it http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone&pipeline=gate16:52
rlandyyes - only two today16:52
rlandyalso no consistently16:52
rlandyhttp://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-standalone16:52
rlandy2020-03-04 13:38:26.148939 | primary | error: Curl error (7): Couldn't connect to server for http://mirror.regionone.limestone.openstack.org/centos/8/AppStream/x86_64/os/repodata/repomd.xml [Failed to connect to mirror.regionone.limestone.openstack.org port 80: No route to host] (http://mirror.regionone.limestone.openstack.org/centos/8/AppStream/x86_64/os/repodata/repomd.xml).16:53
rlandy^^ see as the error16:53
*** bogdando has quit IRC16:54
rlandyweshay|ruck: can we w+ https://review.opendev.org/#/c/710980/?16:55
rlandyon the off chance it passes :)16:55
weshay|ruckrlandy, centos-8 is in tempest16:58
weshay|ruckso ya16:58
weshay|ruckrlandy, raukadah so.. I don't see the need atm for a revert or a force merge.. unless I'm mistaken16:59
rlandyweshay|ruck: not yet imho17:04
rlandyraukadah: https://review.rdoproject.org/r/#/c/25618/3/zuul.d/standalone-jobs.yaml17:04
rlandy^^ scenario10 definition is different from tripleo-ci c717:05
weshay|ruckrlandy, one comment https://review.opendev.org/#/c/711287/117:17
rlandyweshay|ruck: so that is interesting as I saw that https://review.rdoproject.org/r/#/c/25618/3/zuul.d/standalone-jobs.yaml is defined differently17:18
rlandyshould I use 052 with the additional env/octavia?17:19
weshay|ruckrlandy, not now.. but later17:19
weshay|ruckunless you have time to dig into it17:19
weshay|ruckwhich I don't think you do17:19
rlandyok  - because I think that job missed stuff17:19
weshay|ruckrlandy, you have 1-1 stuff or skip17:20
rlandyweshay|ruck: I don;t think there is anything to discuss ....17:20
weshay|ruckk17:20
rlandylet's just meet this afternoon re: centos-817:20
weshay|ruckaye17:21
*** Trevor_V has quit IRC17:21
*** TrevorV has joined #oooq17:22
*** dtantsur is now known as dtantsur|afk17:41
*** jpena is now known as jpena|off17:46
*** amoralej is now known as amoralej|off17:51
*** jfrancoa has quit IRC17:58
*** sshnaidm is now known as sshnaidm|afk17:59
*** derekh has quit IRC18:01
*** jfrancoa has joined #oooq18:14
*** dsneddon has joined #oooq18:19
rlandyweshay|ruck: when you have time, let's review the patches and what order we can get them through18:39
weshay|ruckrlandy, k.. now?18:39
weshay|ruckhave some time18:39
rlandyweshay|ruck: sure18:39
weshay|ruckhttps://meet.google.com/for-kekm-mps?authuser=118:40
rlandyweshay|ruck: https://review.opendev.org/#/c/711287/18:43
*** tosky has quit IRC18:44
weshay|ruckrlandy, 186188618:44
*** tesseract has quit IRC18:55
weshay|ruckrlandy, https://review.opendev.org/#/q/topic:centos-8-jobs+(status:open+OR+status:merged)19:03
rlandyweshay|ruck: https://review.opendev.org/#/c/711287/19:10
rlandyweshay|ruck: https://review.opendev.org/#/c/710179/ multinode-containers19:12
rlandyweshay|ruck: https://review.opendev.org/#/c/710876/19:12
rlandyupdates19:12
weshay|ruckrlandy, https://review.opendev.org/#/c/710230/19:15
*** holser has quit IRC19:19
*** dsneddon has quit IRC19:20
*** dmellado has quit IRC19:24
*** dmellado has joined #oooq19:25
*** jbadiapa has quit IRC19:29
*** dsneddon has joined #oooq19:35
weshay|ruckrlandy, ha.. your revert is about to merge19:56
weshay|ruckso quick!!19:56
weshay|ruckit's like molecule19:56
rlandyawesome19:56
rlandyat least something merges19:56
rlandyweshay|ruck: when we add new jobs - should they all be non-voting by default or only of they fail?20:10
weshay|ruckrlandy, only if we're not 80% sure they pass20:11
rlandyweshay|ruck: ok - so scenario 7 and 10 should go in as non-voting and not in gate until they have a passing record?20:12
weshay|ruckrlandy, so we've seen them pass in rdo check20:13
rlandyweshay|ruck: but I am not sure if that's a direct correlation20:13
weshay|ruckbut non-voting is safer20:13
weshay|ruckas long as we quickly follow up20:13
weshay|rucklike 1 day of results to move to voting + gating20:14
weshay|rucknot two weeks20:14
rlandyhttps://review.opendev.org/#/c/711287/ has scenario010 voting and in gate20:14
rlandywill update20:14
rlandyhere's hoping we get through gates on this round of rechecks20:37
weshay|ruckheh20:38
rlandyweshay|ruck: closing https://bugs.launchpad.net/tripleo/+bug/186610720:40
openstackLaunchpad bug 1866107 in tripleo "tripleo-ci-centos-7-scenario010-standalone jobs are failing tesmpest tests "ServiceError: Request on service 'load-balancer' with url 'http://192.168.24.1:9876' failed with code 500"" [Critical,Triaged]20:40
weshay|ruckrlandy++20:45
rlandyweshay|ruck: reviews for scenario 007 and 010 should be ok now if I got the depends-on right20:48
rlandyboth are set to non-voting20:48
rlandybo gates20:48
rlandyno gates20:48
rlandygoing back to rdocloud component work20:49
*** jfrancoa has quit IRC21:11
rlandyweshay|ruck: is scenario012 still relevant for master? https://github.com/openstack/tripleo-heat-templates/blob/master/ci/environments/scenario012-standalone.yaml21:12
rlandynote OS::TripleO::Services::NovaIronic21:12
weshay|ruckrlandy, don't think so21:20
rlandyweshay|ruck: ok - so I'm leaving it  for component bm21:20
rlandyweshay|ruck: one more ... why is periodic and tripleo-ci check different for scenario010?21:21
weshay|ruck?21:21
rlandyhttps://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/standalone-jobs.yaml#L15421:22
rlandyvs21:22
rlandyhttps://github.com/openstack/tripleo-ci/blob/master/zuul.d/standalone-jobs.yaml#L57221:22
rlandyweshay|ruck: ^^ see the diff in the definitions21:23
rlandyfs is diff etc.21:23
weshay|ruckrlandy, hrm.. brent added both21:24
weshay|ruckhe fucked it up?21:24
weshay|ruck:)21:24
weshay|rucklolz21:24
rlandyweshay|ruck: I assume check is right?21:24
rlandycarlos seemed to comment as such21:24
weshay|ruck52 is right.. the question atm.. is what work21:25
weshay|ruckleave it as is21:25
rlandyweshay|ruck: I thought you said earlier 62 is right21:25
rlandylol21:25
weshay|ruckno21:25
rlandyleave check as 6221:25
weshay|ruckbreaks the standalone pattern21:25
weshay|ruckjust convert both to centos-821:25
rlandyand periodic as 52?21:25
weshay|ruckwe'll fix 62->52 later21:25
rlandyno, no ...21:25
rlandyso it's like this21:26
rlandyit's not a c7, c8 split21:26
rlandyit's a check, periodic split21:26
weshay|ruckunderstood21:26
rlandyso I must ....21:26
weshay|rucktripleo-ci has 62, and rdo has 5221:26
rlandyleave all the c7 jobs as they are???21:26
rlandymake both c8 jobs use 52?21:27
weshay|ruckfor now.. yes21:27
weshay|ruckno21:27
rlandyleave periodic on 5221:27
rlandyand check on 6221:27
weshay|ruckaye21:27
rlandylol21:27
weshay|ruckyes21:27
rlandylogic is>?>>21:27
weshay|rucklet's not mix the pot atm?21:27
rlandyalso yatin - the knower of all things says periodic misses stuff21:28
weshay|ruckmoving the wounded victim may do more harm21:28
*** jmasud has quit IRC21:28
rlandyso I think 62 is more correct21:28
weshay|ruckrlandy, ya.. thanks for the specific feedback21:28
weshay|ruck:)21:28
weshay|rucklolz21:28
rlandy<<brain fired >>21:28
* weshay|ruck diffs the two21:28
weshay|rucksame21:28
weshay|ruckrlandy, btw. .and testing fs16-19 on master does what21:29
weshay|ruck:)21:29
rlandyI have no idea21:29
weshay|rucknothing21:29
rlandymeaning?21:30
weshay|ruck####################################################################################################21:30
weshay|ruck# FEATURESET062 - TRIPLEO STANDALONE FOR OCTAVIA - TRACKS SCENARIO 1021:30
weshay|ruck# This is a modified version of featureset 052 (standalone) so changes made21:30
weshay|ruck# to that featureset should be examined to see if they are relevant here.21:30
weshay|ruck####################################################################################################21:30
weshay|ruckrlandy, 62 has added things like amphora_url: 'https://images.rdoproject.org/octavia/stein/amphora-x64-haproxy-centos.qcow2'21:31
rlandystein21:31
weshay|ruckrlandy, so in this case.. I think 62 is correct and 52 is wrong..21:31
weshay|ruckrlandy, so .. ok21:31
rlandyclearly 62 could use some work as well21:32
weshay|ruckmake rdo 06221:32
rlandyjust for c8?21:32
weshay|ruckand we'll clean up the mess later21:32
weshay|ruckwe should be removing c7 master21:32
rlandyok - let's test that21:32
rlandyand see if it explodes21:32
rlandyBOOM21:32
*** Tengu has quit IRC22:04
*** Tengu has joined #oooq22:06
*** tosky has joined #oooq22:11
rlandyweshay|ruck: wrt https://review.rdoproject.org/r/#/c/25649/ - I ma going to remove the depends-on and merge the jobs - they are just job definitions22:14
rlandyI'll put the depends-on on the review that adds to the pipeline22:14
weshay|ruckrlandy, fyi https://review.rdoproject.org/r/#/c/25655/22:18
rlandylooking22:19
*** TrevorV has quit IRC22:55
*** rlandy is now known as rlandy|bbl23:15
*** Goneri has quit IRC23:36
*** tosky has quit IRC23:40
*** Goneri has joined #oooq23:42

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!