Thursday, 2020-06-25

« Wednesday, 2020-06-24 Index Friday, 2020-06-26 »
*** rfolco has quit IRC00:53
*** rlandy|ruck|bbl is now known as rlandy|ruck02:01
*** rlandy|ruck has quit IRC02:12
*** Tengu has quit IRC02:25
*** Tengu has joined #oooq02:27
*** jmasud has joined #oooq03:14
*** udesale has joined #oooq03:52
*** ratailor has joined #oooq04:00
*** ykarel|away is now known as ykarel04:21
*** ysandeep|away is now known as ysandeep04:43
*** ccamacho has quit IRC04:53
*** skramaja has joined #oooq05:11
*** Tengu has quit IRC05:47
*** ysandeep is now known as ysandeep|afk05:48
*** marios has joined #oooq05:49
*** Tengu has joined #oooq05:54
*** saneax has joined #oooq05:55
chandankumarysandeep|afk, Hello06:04
chandankumarysandeep|afk, you are looking at reposetup patch work?06:04
chandankumarysandeep|afk, https://logserver.rdoproject.org/04/28204/6/check/periodic-tripleo-ci-centos-8-standalone-train/04a8978/logs/undercloud/home/zuul/repo_setup.log.txt.gz we are back to same issue which you were facing in check, let me know when back06:06
*** jbadiapa has quit IRC06:18
chandankumarmarios, ykarel added two new tasks https://tree.taiga.io/project/tripleo-ci-board/task/1836?kanban-status=1447274 and https://tree.taiga.io/project/tripleo-ci-board/task/1837?kanban-status=144727406:34
*** skramaja has quit IRC06:34
*** skramaja has joined #oooq06:35
marioschandankumar: thanks06:35
*** jbadiapa has joined #oooq06:36
ykarelchandankumar, ack06:39
ykarelwith components this needs to be fixed06:40
*** ysandeep|afk is now known as ysandeep06:44
ysandeepchandankumar, hey o/ yes I will work on it today06:47
ysandeepchandankumar, Yesterday patches didn't help?06:48
chandankumarysandeep, I am about to test that06:53
chandankumarysandeep, we need to fix two issues for c8 train component pipeline for check and periodic jobs06:53
ysandeepchandankumar, yes couple of issue need care, i was working on a new periodic component it failed with https://logserver.rdoproject.org/26/28226/1/check/periodic-tripleo-ci-centos-8-scenario001-standalone-cinder-train/77103f6/job-output.txt -  failed with - /home/zuul/workspace/hash_info.sh: No such file or directory06:55
*** ccamacho has joined #oooq07:05
*** dtantsur|afk has quit IRC07:10
*** jtomasek has quit IRC07:12
*** jtomasek has joined #oooq07:16
*** amoralej|off is now known as amoralej07:30
*** jfrancoa has joined #oooq07:31
*** dtantsur has joined #oooq07:39
*** tosky has joined #oooq07:42
chandankumarmarios, I am merging this https://review.rdoproject.org/r/#/c/28214/ one if works then we are good, if not will revert back07:50
marioschandankumar: k07:51
*** jpena|off is now known as jpena07:58
bhagyashrishey pojadhav fyi you can update your patch https://review.rdoproject.org/r/#/c/28198/ as my patch https://review.rdoproject.org/r/#/c/28216/ got merged ...08:01
ysandeepmarios, o/ Hey I am looking at the repo setup role. I came across a review where - You raised come good points with some objections keeping C7/8 train in mind when we were changing previous condition - https://review.opendev.org/#/c/720776/14/roles/repo-setup/tasks/get-dlrn-hash.yml , But I don't see an answers to those queries in the review.. Just wondering if this was discussed offline?08:02
mariosysandeep: no we didn't08:05
mariosysandeep: centos8 train wasn't aproblem at the time :)08:06
mariosysandeep: :*08:06
marios:(08:06
ysandeepmarios, ack thank you, ack I will try to add a condition to segregate based on c8/c7 for train then08:07
mariosysandeep: but didn't we already fix get-hash08:07
mariosysandeep: oh this is repo setup08:07
mariosysandeep: ack08:07
mariosysandeep: yeah you'll basically have to re-introduce the os version in the conditionals i think08:08
ysandeepmarios, yes08:10
chandankumarysandeep, marios https://logserver.rdoproject.org/33/28233/1/check/periodic-tripleo-centos-8-train-component-common-promote-to-promoted-components/c025c52/job-output.txt adding dummy component dummy config file worked08:21
ysandeepchandankumar, good news08:22
ysandeepmarios, Just wondering if we need release condition at all then.. thinking b/w revert of the older patch vs Addition of OS conditional keeping release conditional untouched08:22
mariosysandeep: post it and we can discuss on the review is easier to point to things and be specific08:24
ysandeepmarios, ack posting08:24
*** ykarel is now known as ykarel|lunch08:39
pojadhavbhagyashris, ack08:48
*** derekh has joined #oooq08:58
ysandeepchandankumar, c8 consistent for some component have reference to old centos8-train-components - https://trunk.rdoproject.org/centos8-train/consistent/delorean.repo09:07
chandankumarysandeep, will check09:09
ysandeepchandankumar, thanks!09:10
chandankumarysandeep, current is fine https://trunk.rdoproject.org/centos8-train/current/delorean.repo09:13
ysandeepchandankumar, okay09:13
marioszbr: fyi re zuul native & collections - i added notes under "zuul native ansible" in https://tree.taiga.io/project/tripleo-ci-board/task/1791 ; editing ansible.cfg was OK but it still didn't help. The changes made in pre.yaml were still not effected on run.yaml09:15
marioszbr: so basically we are blocked on zuul native tripleo-operator-ansible until zuul provides collections support09:15
marioszbr: i need to reach out to someone over there about that perhaps they are already working on it09:16
zbrzuul-discuss is your friend09:16
zbrirc is good for small/easy subject, but if we are to speedup things, ML is the way09:17
marioszbr: ack i'll send something out today09:17
zbri will back you09:18
marioszbr: https://duckduckgo.com/?t=ffab&q=collections+site%3Ahttp%3A%2F%2Flists.zuul-ci.org%2Fpipermail%2Fzuul-discuss%2F&ia=web09:19
marioszbr: doesn't find someting in archives09:19
marioszbr: ack thanks09:19
*** ysandeep is now known as ysandeep|afk09:39
*** bhagyashris is now known as bhagyashris|afk09:55
*** ykarel|lunch is now known as ykarel09:58
*** ysandeep|afk is now known as ysandeep10:05
*** jmasud has quit IRC10:44
*** jmasud has joined #oooq10:46
*** bhagyashris|afk is now known as bhagyashris11:00
soniya29|roverrlandy|ruck, looking at https://bugs.launchpad.net/tripleo/+bug/188162411:07
openstackLaunchpad bug 1881624 in tripleo "full-tempest-scenario-master failing on test_ipv6_hotplug" [Critical,Triaged]11:07
*** jschlueter has joined #oooq11:17
*** chem has quit IRC11:17
*** ysandeep is now known as ysandeep|break11:17
*** holser has quit IRC11:22
*** chem has joined #oooq11:22
*** jpena is now known as jpena|lunch11:37
*** ysandeep|break is now known as ysandeep11:47
*** udesale_ has joined #oooq11:52
*** soniya29|rover is now known as soniya29|rover|a11:53
*** soniya29|rover|a is now known as soniya29|afk11:53
*** dpawlik6 has quit IRC11:54
*** udesale has quit IRC11:54
*** amoralej is now known as amoralej|lunch12:02
*** skramaja has quit IRC12:09
*** rlandy has joined #oooq12:13
*** rlandy is now known as rlandy|ruck12:13
*** dpawlik6 has joined #oooq12:19
rlandy|rucksoniya29|afk: hey12:21
rlandy|ruckhow are things?12:21
rlandy|ruckwe gave train failures12:21
*** rfolco has joined #oooq12:21
rlandy|ruckchandankumar: hey - need to borrow your brain ....12:22
rlandy|ruckhttps://logserver.rdoproject.org/openstack-periodic-24hr/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-train/7f1d063/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz12:23
rlandy|ruckError: Unable to find a match: python3-tripleoclient12:23
rlandy|ruckpackaging issue?12:23
rlandy|ruckoh nvm - that centos8 train12:25
ysandeeprlandy|ruck, yes its c8 train - trying to reposetup role for ^^ with https://review.opendev.org/#/c/737923/12:27
ysandeeptrying to fix* reposetup12:27
chandankumarysandeep, is fixing that12:28
chandankumarrlandy|ruck, ^^12:29
rlandy|ruckk - thanks12:29
*** derekh has quit IRC12:30
*** jpena|lunch is now known as jpena12:42
*** derekh has joined #oooq12:54
soniya29|afkrlandy|ruck, hello12:54
rlandy|rucksoniya29|afk: hey12:56
rlandy|rucksoniya29|afk: there are some cards on the escalation board that I am not sure are still needed12:59
*** marios is now known as marios|call12:59
*** holser has joined #oooq12:59
rlandy|ruckwe need to review if those failures are still a problem13:00
rfolcoarxcruz, rfolco, zbr, panda, sshnaidm, rlandy, marios, ysandeep, bhagyashris, soniya29, pojadhav, akahat, weshay, chandankumar13:00
rfolcoping scrum13:00
rfolcohttps://tree.taiga.io/project/tripleo-ci-board/taskboard/unified-sprint-29?milestone=25771713:00
rfolcohttps://meet.google.com/oiv-geho-mai?pli=1&authuser=113:01
rlandy|rucksoniya29|afk: also watching the scenario tests on THT13:01
rlandy|ruckin gates13:01
bhagyashrismarios|call, plz add this in your review list https://review.rdoproject.org/r/#/c/28217/13:02
soniya29|afkrlandy|ruck, like which cards?13:06
rlandy|rucksoniya29|afk: getting13:09
rlandy|ruckchandankumar: https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario001-standalone-master/a4bcfd5/logs/undercloud/var/log/tempest/stestr_results.html.gz - is that as a result of the new cirros image?13:12
chandankumarrlandy|ruck, looking13:15
chandankumarrlandy|ruck, [    6.002692] ---[ end Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ]---13:15
rlandy|ruckhttps://review.opendev.org/#/c/733676/3/playbooks/tempest.yml13:16
*** holser has quit IRC13:20
*** soniya29|afk is now known as soniya29|rover13:21
*** holser has joined #oooq13:21
arxcruzrfolco: https://00639ef95b734e189ce1-4e4c7f9e4e66be3b004244bfc6fd74b6.ssl.cf1.rackcdn.com/726773/6/check/tripleo-ci-centos-8-containers-multinode/6766a15/job-output.txt13:21
rlandy|ruckchandankumar: hmmm .. all scenario tests failed tempest on master ... https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-master%20 - but idfferent tests13:23
rlandy|ruckwill rerun those13:23
soniya29|roverrlandy|ruck, we are not facing the rdocloud stack deleting failure anymore, shall I close it13:23
rlandy|rucksoniya29|rover, I'm going to watch that until the end of the week13:23
rlandy|ruckon friday, you can close it13:23
soniya29|roverrlandy|ruck, tommorrow you mean?13:24
rlandy|rucksoniya29|rover: ack13:24
rlandy|rucksoniya29|rover: also - we need to rerun the tests that failed on the master pipeline ...13:24
rlandy|ruckperiodic-tripleo-ci-centos-8-scenario004-standalone-masteropenstack/tripleo-cimasteropenstack-periodic-mastermaster42342020-06-25T08:54:25FAILURE13:24
rlandy|ruckperiodic-tripleo-ci-centos-8-scenario003-standalone-masteropenstack/tripleo-cimasteropenstack-periodic-mastermaster37942020-06-25T08:54:30FAILURE13:24
rlandy|ruckperiodic-tripleo-ci-centos-8-scenario002-standalone-masteropenstack/tripleo-cimasteropenstack-periodic-mastermaster63832020-06-25T08:54:29FAILURE13:24
rlandy|ruckperiodic-tripleo-ci-centos-8-scenario001-standalone-master13:24
rlandy|ruckperiodic-tripleo-ci-centos-8-standalone-on-multinode-ipa-master13:24
rlandy|ruckperiodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master13:25
rlandy|ruck^^ all tempest related13:25
*** ratailor has quit IRC13:25
soniya29|roverrlandy|ruck, let me check, give me a moment13:25
rlandy|rucksoniya29|rover: so you want to build that testproject job?13:25
rlandy|ruckif not, I'll do it13:25
soniya29|roverrlandy|ruck, How to build that testproject job?13:26
chandankumarrlandy|ruck, just specific test or all test?13:26
rlandy|ruckchandankumar: all networking13:26
rlandy|ruckbut infra increased the number of nodes we used13:26
rlandy|ruckI could ask them ti drop the node numbers13:27
chandankumargive me some time will take a look13:27
rlandy|ruckchandankumar: ok - I need to catch dpawlik6 before his EOD if we want to drop the node numbers13:27
soniya29|roverrlandy|ruck, never built a testproject job earlier13:27
chandankumarrlandy|ruck, https://review.rdoproject.org/r/28236 quick merge needed for container build13:32
rlandy|ruckchandankumar: dine13:32
rlandy|ruckdone13:32
chandankumarrlandy|ruck, thanks!13:34
chandankumarrlandy|ruck, sorry https://review.opendev.org/#/c/733676/3/playbooks/tempest.yml you mean this one reverting13:40
chandankumarkernel panic is coming there13:40
rlandy|rucksoniya29|rover: no worries - I'll do it13:40
rlandy|ruckchandankumar: juts a guess13:40
rlandy|ruckI'm rerunning13:41
chandankumarrlandy|ruck, might be due to image13:41
rlandy|ruckworth a revert?13:41
rlandy|ruckchandankumar: we can test a revert on testproject13:41
chandankumarrlandy|ruck, yes13:42
rlandy|ruckchandankumar: let me do that13:42
rlandy|ruckhttps://review.opendev.org/#/c/738025/113:42
rlandy|rucklet's see how that goes13:42
ysandeepperiodic-tripleo-ci-rhel-8-multinode-1ctlr-featureset010-clients-rhos-17 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/c23178e41c9b4c35b7bfbe386fb6ae2a : POST_FAILURE in 2h 05m 10s (non-voting)13:43
ysandeepmarios|call, ^^13:43
marios|callysandeep: thanks looking13:44
*** TrevorV has joined #oooq13:47
*** dpawlik6 is now known as dpawlik-213:48
*** dpawlik-2 is now known as danpawlik13:48
*** amoralej|lunch is now known as amoralej13:51
*** marios|call is now known as marios13:53
rlandy|ruckchandankumar: soniya29|rover: https://review.rdoproject.org/r/#/c/21853/ - let's see how that goes13:55
bhagyashrismarios, could you please merge this https://review.rdoproject.org/r/#/c/28217/ ?13:56
soniya29|roverrlandy|ruck, periodic-tripleo-ci-centos-8-scenario002-standalone-master has ssh time out issue13:58
rlandy|ruckhttps://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario002-standalone-master - only in the last job13:59
mariosbhagyashris: no can't needs rebase13:59
rlandy|ruckthe rerun will tell us if it's reproducible13:59
mariosbhagyashris: " Cannot Merge " on the review13:59
ysandeeprfolco, hey were you talking about this one https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.post-timeout ?13:59
bhagyashrismarios, ahh sorry my bad i haven't refresh tab so cant see it's in merged conflict14:01
bhagyashrismarios,  i will do that in sec14:01
mariosneed reviews please https://review.rdoproject.org/r/28207 & depends-on https://review.rdoproject.org/r/#/c/28206 wires up promoter periodics pipeline14:01
bhagyashrismarios, done.14:02
rfolcoysandeep, yes post-timeout14:04
mariosbhagyashris: ping me to +A once zul reports should be quick14:04
bhagyashrismarios, sure14:04
ysandeeprfolco, thanks for confirming, will give it a try14:05
soniya29|roverrlandy|ruck, yeah, I checked latest run which had failure because of ssh time our14:05
bhagyashrisrfolco, chandankumar marios  https://review.rdoproject.org/r/#/c/28211/ plz add in your review list14:06
soniya29|roverrlandy|ruck, same with periodic-tripleo-ci-centos-8-scenario001-standalone-master14:06
bhagyashrismarios, zuul +1 https://review.rdoproject.org/r/#/c/28217/14:12
mariosbhagyashris: make me14:13
mariosbhagyashris: (done)14:13
bhagyashrismarios, thanks! :)14:13
pojadhavmarios, rfolco : pls help to merge this https://review.rdoproject.org/r/#/c/28220/14:26
*** jfrancoa has quit IRC14:28
soniya29|roverrlandy|ruck, okay14:28
*** jfrancoa has joined #oooq14:28
*** ysandeep is now known as ysandeep|away14:29
*** ykarel is now known as ykarel|away14:36
marioszbr: http://lists.zuul-ci.org/pipermail/zuul-discuss/2020-June/001282.html14:37
*** holser has quit IRC14:44
*** dsneddon has quit IRC14:48
danpawlikrlandy|ruck, chandankumar: is possible to get how the stack looks like: https://review.rdoproject.org/zuul/build/97bc507a797f470a9a52da995eb54f7c/log/job-output.txt#854-927 ?14:50
rlandy|ruckit could die in the middle for a few reasons ...14:51
danpawlikrlandy|ruck, chandankumar: Vexxhost reply with a question "(...) Could you kindly provide us with heat stack template you're using, as we don't see any resource defined in the job stdout? (...)"14:51
rlandy|ruckgetting14:53
rlandy|ruckhttps://opendev.org/openstack/openstack-virtual-baremetal/src/branch/master/templates14:57
rlandy|rucktemplates come from there - OVB args14:58
danpawlikrlandy|ruck: thank you++14:59
danpawlikrlandy|ruck++14:59
rlandy|ruckand env templates ...14:59
rlandy|ruckhttps://opendev.org/openstack/openstack-virtual-baremetal/src/branch/master/environments14:59
rlandy|ruckwhen we create a stack ...14:59
rlandy|ruckthe command is:14:59
rlandy|ruckpicks up15:00
rlandy|ruck--name baremetal_735930_2_85357 --quintupleo --id 735930_2_85357 -e /home/zuul/workspace/ovb/env-735930_2_85357-base.yaml -e /home/zuul/workspace/ovb/openstack-virtual-baremetal/environments/bmc-use-cache.yaml -e /home/zuul/workspace/ovb/openstack-virtual-baremetal/environments/undercloud-floating-none.yaml -e /home/zuul/workspace/ovb/openstack-virtual-baremetal/environments/quintupleo-no-undercloud.yaml -e15:00
rlandy|ruck/home/zuul/workspace/ovb/openstack-virtual-baremetal/environments/all-networks-port-security.yaml15:00
rlandy|ruckthose are in the links above15:00
danpawlikPerfect. One more time thank you15:00
rlandy|ruck{{ ovb_working_dir }}/.venv/bin/python {{ ovb_repo_directory }}/bin/deploy.py {{ ovb_args }}15:08
*** pojadhav is now known as pojadhav|dinner15:17
*** bhagyashris is now known as bhagyashris|afk15:19
rlandy|rucksoniya29|rover: sorry - back to escalation cards ...15:25
chandankumarmarios, can i make the suggested change https://review.opendev.org/#/c/727200/ in a new review for build test packages15:26
chandankumarit is a very big one15:26
rlandy|ruckhttps://bugs.launchpad.net/tripleo/+bug/188162415:26
openstackLaunchpad bug 1881624 in tripleo "full-tempest-scenario-master failing on test_ipv6_hotplug" [Critical,Triaged]15:26
marioschandankumar: sure i mean as i wrote there "you've already done it now so don't change it just saying that might have been easier"15:27
rlandy|ruckhttps://review.opendev.org/#/c/732420/ added ipv6 hotplug tests to skip list15:27
marioschandankumar: might be cleaner with ini_file but if you do decide its worth it for sure in a followup i wouldn't change that now15:27
chandankumarmarios, thanks15:28
*** udesale_ has quit IRC15:28
*** udesale has joined #oooq15:34
chandankumarrlandy|ruck, https://review.opendev.org/#/c/727200/ is good to go :-)15:48
rlandy|ruckcool - will look in afternoon15:50
rlandy|ruckon PSI public setup atm15:50
marios /nick marios|out16:06
*** marios is now known as marios|out16:06
rlandy|ruckClient Error: Too Many Requests for url: http://mirror.regionone.rdo-cloud-tripleo.rdoproject.org:8082/v2/ceph/daemon/blobs/sha256:ea7eb661de71977e5d57c2d6935d5bca6496780665cc7537595b19f8d431b676"], "stdout": "", "stdout_lines": []}16:20
rlandy|ruckugh16:20
*** amoralej is now known as amoralej|off16:30
soniya29|roverrlandy|ruck, periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master is passing now, can we close that card?16:35
rlandy|rucksoniya29|rover: hey - which card was that one?16:37
soniya29|roverrlandy|ruck, https://trello.com/c/JUO07FZa/1529-cixlp1881624tripleociproa-full-tempest-scenario-master-failing-on-testipv6hotplug16:38
rlandy|rucksoniya29|rover: periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-master  is passing because that because the test is in the skip list16:39
rlandy|ruckI was just checking fif we could revert the patch that added the test to the skip list - maybe not yet16:40
rlandy|ruckbut thanks for looking at that16:40
*** dsneddon has joined #oooq16:41
*** aakarsh has joined #oooq16:42
*** pojadhav|dinner is now known as pojadhav16:44
soniya29|roverrlandy|ruck, ack16:44
*** dtantsur is now known as dtantsur|afk16:48
*** udesale has quit IRC16:53
*** holser has joined #oooq16:54
soniya29|roverrlandy|ruck, the patch addressing this card - https://trello.com/c/mJvj47I8/1528-cixlp1881592tripleociproa-ussuri-tempestapicomputeimagestestimagesoneservernegativeimagesoneservernegativetestjson is abandoned16:54
*** derekh has quit IRC17:00
*** holser has quit IRC17:02
chandankumarrlandy|ruck, I think containers got pushed https://logserver.rdoproject.org/94/28194/9/check/periodic-tripleo-build-containers-ubi-8/9cdb451/job-output.txt but job got post_faiure17:05
*** holser has joined #oooq17:06
soniya29|roverrfolco, how is the patch https://review.opendev.org/#/c/735659/ helping the ^^ above card?17:09
soniya29|roverrfolco, I didn't get the context17:10
rfolcosoniya29|rover, we were getting inconsistent results in periodic jobs. Now its time to check if these tests are working and if still failing, add to skip list.17:11
rfolcosoniya29|rover, if not failing the same tests, ignore. Restore the patch otherwise.17:13
soniya29|roverrfolco, but how will adding TestNetworkAdvancedServerOps to skiplist will help tempest.api.compute.images.test_images_oneserver_negative.ImagesOneServerNegativeTestJSON?17:14
rlandy|ruck| Error: Error copying image to the remote destination: Error trying to reuse blob sha256:f9327c1608ad0ae79b5f4a4c47eda907a980ea9640c70e6036811ea76d641249 at destination: error pinging docker registry\17:14
rlandy|ruckchandankumar: that's cool17:14
chandankumarrlandy|ruck, sorry failed at Retag and push image localhost/testingmaster/openstack-cinder-volume:latest w/ _arch tag suffix17:15
rfolcosoniya29|rover, ignore the patch and add the failing one to skip list. I don't remember the history behind this.17:16
rlandy|rucksoniya29|rover: is that test still running?17:16
rlandy|ruckrfolco: soniya29|rover: wes has a patch in cannot merge state to add to skip list17:17
rlandy|ruckdo we still need to?17:17
soniya29|roverrlandy|ruck, gmann has added a fix, but the build has failed17:17
rlandy|ruckwe should confirm with current results17:17
*** marios|out has quit IRC17:19
rlandy|rucksoniya29|rover: pls patch patch - will look at it17:20
soniya29|roverrlandy|ruck, ack17:21
rlandy|rucksoniya29|rover: I mean the patch you mentioned above17:22
rlandy|ruck gmann has added a fix, but the build has failed17:22
soniya29|roverrlandy|ruck, yeah, got it17:23
rlandy|ruckThe only patck i see is the one from wes17:24
rlandy|rucklet's check ussuri results17:25
soniya29|roverrlandy|ruck, the test is passing in latest run17:26
rlandy|ruckhttps://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/11407a4/logs/undercloud/var/log/tempest/stestr_results.html.gz17:27
rlandy|ruck^^ master has the problem17:27
rlandy|rucktest_server_connectivity_cold_migration[compute,id-a4858f6c-401e-4155-9a49-d5cd053d1a2f,network,slow]17:27
rlandy|ruckfail17:27
rlandy|rucktest_server_connectivity_resize[compute,id-719eb59d-2f42-4b66-b8b1-bb1254473967,network,slow]17:27
*** jpena is now known as jpena|off17:28
soniya29|roverrlandy|ruck, those two tests are different from the test in the card17:28
rlandy|rucktest_server_connectivity_cold_migration[compute,id-a4858f6c-401e-4155-9a49-d5cd053d1a2f,network,slow]17:28
rlandy|ruckfail17:28
rlandy|rucktest_server_connectivity_resize[compute,id-719eb59d-2f42-4b66-b8b1-bb1254473967,network,slow]17:28
rlandy|ruckfail17:28
rlandy|ruckhttps://review.opendev.org/#/c/735659/1/roles/validate-tempest/vars/tempest_skip_ussuri.yml17:28
rlandy|ruck  - test: 'tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_cold_migration'17:29
rlandy|ruck    reason: 'System is booting up. Unprivileged users are not permitted to log in yet. Please come back later.'17:29
rlandy|ruck    lp: 'https://bugs.launchpad.net/tripleo/+bug/1881642'17:29
rlandy|ruck  - test: 'tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_resize'17:29
openstackLaunchpad bug 1881642 in tripleo "ussuri | master fs020 failing on TestNetworkAdvancedServerOps" [Critical,Triaged]17:29
rlandy|ruck    reason: 'System is booting up. Unprivileged users are not permitted to log in yet. Please come back later.'17:29
rlandy|ruck    lp: 'https://bugs.launchpad.net/tripleo/+bug/1881642'17:29
rlandy|rucksoniya29|rover: how are they different?17:29
rlandy|ruckI am looking at wes's patch17:29
soniya29|roverrlandy|ruck, the card is about tempest.api.compute.images.test_images_oneserver_negative.ImagesOneServerNegativeTestJSON17:29
rlandy|ruckPatch reviewed https://review.opendev.org/#/c/735659/, recheck for Zuul failure - posted the wrong patch17:30
soniya29|roverI guess patch will not address the test failure the card is talking about17:31
rlandy|rucktempest.api.compute.images.test_images_oneserver_negative.ImagesOneServerNegativeTestJSON 5 5 0 0 0 Detail17:32
rlandy|rucksoniya29|rover: so you are right there17:32
soniya29|roverrlandy|ruck, have a look at this:- https://logserver.rdoproject.org/openstack-periodic-latest-released/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-ussuri/3ec4f27/logs/undercloud/var/log/tempest/stestr_results.html.gz17:32
rlandy|ruckthat one is passing17:32
rlandy|rucksoniya29|rover: ok - go ahead and comment on eth card ... https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-standalone-full-tempest-api-ussuri17:33
rlandy|ruckshow the results that the tests look to be passing17:33
soniya29|roverrlandy|ruck, thanks, I will do that right now17:34
rlandy|rucksoniya29|rover: also comment that https://review.opendev.org/#/c/735659/1/roles/validate-tempest/vars/tempest_skip_ussuri.yml is the unrelated17:34
rlandy|rucksoniya29|rover: leave comments on https://bugs.launchpad.net/tripleo/+bug/1881592 as well17:34
openstackLaunchpad bug 1881592 in tripleo "ussuri: tempest.api.compute.images.test_images_oneserver_negative.ImagesOneServerNegativeTestJSON " [Critical,Triaged]17:34
soniya29|roverrlandy|ruck, yeah, I will update both of them17:35
soniya29|roverrlandy|ruck, done17:46
rlandy|ruckpanda: ping - need your help with promoter17:48
rlandy|ruckthe one on rdocloud17:48
rlandy|ruckhttp://38.145.34.55/17:48
rlandy|ruckthere are no logs from today17:49
rlandy|ruckchandankumar: should centos8 train be promoting?18:05
chandankumarrlandy|ruck, not today18:05
chandankumarrlandy|ruck, https://review.rdoproject.org/r/#/c/28204/ three patches needed for that18:05
rlandy|ruckchandankumar: I am talking in generla18:07
rlandy|ruckgeneral18:07
rlandy|ruckit looks like rdocloud promoter go shut down18:07
rlandy|ruckpossible because needed to reboot nova in rdocloud18:07
rlandy|rucknow all release are back18:07
chandankumarrlandy|ruck, leave it for today18:08
rlandy|ruckchandankumar: ok - so centos8 train is not included anywhere for the moment.18:08
chandankumaryes18:08
chandankumarrlandy|ruck, is it possible to promote the current train c8 tripleo ci testing18:09
chandankumarit is working, tempest testing going on18:09
chandankumarso that we can run check jobs tomorrow18:10
rlandy|ruckthere is something very wrong with promoter18:12
chandankumarhttps://review.rdoproject.org/zuul/stream/6d85e4fe784e4d6ca84cf3fbaae24e68?logfile=console.log standone running here18:12
* rlandy|ruck needs panda input18:12
* rlandy|ruck stops promoter again18:13
rlandy|ruckwaiting on panda18:13
rlandy|ruckgreat - no panda, no weshay_pto and no promoter18:14
rlandy|rucksoniya29|rover: ^^ fyi18:21
rlandy|ruckpromoter issues on rdocloud18:21
rlandy|ruckimpacts master, ussuri18:21
soniya29|roverrlandy|ruck, okay18:23
rlandy|ruckrfolco: ping - where did the ansible pull go for the promoter? you moved it to the service?18:33
rfolcorlandy|ruck, I think panda disabled the continuous delivery18:33
rlandy|ruckrfolco: he did but the promoter server was restarted18:34
rlandy|ruckand I need to shut that down again18:34
rlandy|ruckhttps://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/promoter/templates/dlrn-promoter-service.sh.j2#L1718:34
rfolcorlandy|ruck, yeah this line18:41
rfolco    cd ~/ci-config;18:41
rlandy|ruckrfolco: well I need to find where that template goes18:41
rfolco /home/centos/ci-config/ci-scripts/dlrnapi_promoter/dlrn-promoter-service.sh18:41
rlandy|ruckrfolco: thanks - I'm just leaving the promoter disabled until panda or weshay_pto can put back what we lost18:44
rfolcorlandy|ruck, ack18:44
rlandy|ruckI have no idea what they were doing here18:45
rlandy|ruckmarios or zbr might18:45
rfolcorlandy|ruck, ussuri and master on vexhost promoter18:50
rfolcorlandy|ruck, older branches on old promoter18:50
rlandy|ruckrfolco: are you sure about that??18:50
rlandy|ruckhttp://38.145.34.55/18:50
rfolcoyes, I did some promotions with wes18:50
rlandy|ruckhttp://38.102.83.109/18:51
rlandy|ruckrfolco: check the logs on the two18:51
rlandy|ruckhttp://38.102.83.109/ - there are no master logs here18:51
rlandy|ruckit's the other way around afaict18:52
rfolcohmmm weird let me double check18:52
rlandy|ruckrfolco: that's right per was weshay_pto told me18:53
rlandy|ruckwhatever18:53
rlandy|rucknobody died18:53
rlandy|ruckwhen panda gets back we'll sort it out18:53
rfolcosomething happened18:53
rfolcoI lost access to regular promoter18:54
rfolcoand vexxhost promoter has old branches on the dlrn-promoter script18:54
rfolcoso no clues what is going on there18:54
rlandy|ruckrfolco: weshay_pto locked everyone except the rucks out of the server18:56
rlandy|ruckno worries18:56
rlandy|ruckI emailed panda18:57
rlandy|ruckwe will just wait it out18:57
rfolcok18:57
rlandy|ruckchandankumar: hey ...19:38
rlandy|ruckhttps://review.opendev.org/#/c/73802519:38
rlandy|ruck'^^ think we should +2 this19:39
rlandy|rucksee the gate failures19:39
rlandy|rucka bunch of scenraio failures19:39
chandankumarrfolco, yes done19:56
chandankumarah rlandy|ruck19:57
rlandy|ruckchandankumar: thanks - I think the gate failures and the master scenario failures are related19:57
chandankumarrlandy|ruck, might be, I will take a look at cirros mess tomorrow19:58
rlandy|ruckchandankumar: thanks as usual!20:00
*** TrevorV has quit IRC20:04
*** jtomasek has quit IRC20:34
*** saneax has quit IRC21:02
*** aakarsh has quit IRC21:06
*** jfrancoa has quit IRC22:21
*** rlandy|ruck is now known as rlandy|ruck|bbl22:25
*** tosky has quit IRC22:40
*** dsneddon has quit IRC23:22
« Wednesday, 2020-06-24 Index Friday, 2020-06-26 »

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!