Thursday, 2022-07-07

ysandeep|outrlandy|out, :) rcastillo|rover already created revert of fs035 skip patch, looks like you created revert of revert03:13
ysandeep|outrlandy|out, rcastillo|rover https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4368203:13
ysandeep|outor was that intentional?03:13
*** ysandeep|out is now known as ysandeep|ruck04:32
ysandeep|ruckbhagyashris, fyi.. psi is unstable currently04:39
ysandeep|ruckincase you hit node_failure in downstream04:39
bhagyashrisysandeep|ruck, ack04:40
bhagyashristhanks 04:40
*** ysandeep|ruck is now known as ysandeep|ruck|afk04:47
*** ysandeep|ruck|afk is now known as ysandeep|ruck05:35
ysandeep|ruckhah, this is interesting:-05:50
ysandeep|ruckhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b16/848733/3/check/tripleo-ci-centos-9-content-provider/b16b062/logs/delorean_logs/component/tripleo/c8/07/c807e163361f3d100df4933d8e951574efcbb2c0_dev/rpmbuild.log.txt.gz05:50
ysandeep|ruck- Curl error (28): Timeout was reached for http://mirror.stream.centos.org/SIGs/9-stream/messaging/x86_64/rabbitmq-38/repodata/repomd.xml [Failed to connect to mirror.stream.centos.org port 80: Connection timed out]05:50
ysandeep|ruckbut the repo which we were using:05:50
ysandeep|ruckhttps://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b16/848733/3/check/tripleo-ci-centos-9-content-provider/b16b062/logs/undercloud/etc/yum.repos.d/delorean-wallaby-deps.repo05:50
ysandeep|ruckbaseurl=http://mirror.mtl01.iweb.opendev.org/centos-stream/SIGs/9-stream/messaging/$basearch/rabbitmq-38/05:50
ysandeep|ruckwhy its still using mirror.stream.centos.org as baseurl05:51
ysandeep|ruckmay be cache is not cleared  05:52
pojadhavhey folks, need reviews and approval on this https://review.opendev.org/c/openstack/tripleo-quickstart/+/84384005:59
chandankumarpojadhav: commented06:15
chandankumarcan you do the same for standalone upgrade fs also06:15
pojadhavchandankumar, checking06:21
jm1moin :)06:41
pojadhavchandankumar, marios : kindly have a look when free - updated patch https://review.opendev.org/q/topic:tempest_allow_list+owner:pojadhav%2540redhat.com+status:open06:49
ysandeep|ruckvexxhost low performance/network issues are back - random issues all over - going crazy as ruck/rovers :D and PSI is completely dead - time to take a break 06:56
mariospojadhav: ack adding to list (just finished a round of reviews ) 06:58
pojadhavmarios, np thank you :)07:01
*** amoralej|off is now known as amoralej07:04
*** ysandeep|ruck is now known as ysandeep|ruck|afk07:05
*** ysandeep|ruck|afk is now known as ysandeep|ruck08:37
*** AndroUser is now known as Pooja_Jadhav09:36
ysandeep|ruckwohoo finally psi is back09:55
*** ysandeep|ruck is now known as ysandeep|ruck|brb10:06
*** ysandeep|ruck|brb is now known as ysandeep|ruck10:27
*** rlandy|out is now known as rlandy10:28
rlandyysandeep|ruck: hi - want to sync?10:29
ysandeep|ruckgood morning rlandy o/10:29
ysandeep|rucksure 10:29
rlandyysandeep|ruck: https://meet.google.com/xcp-gcsw-kvj10:30
rlandyysandeep|ruck: can y make that meeting half an hour? 10:56
rlandycuts into scrum10:56
ysandeep|rucksry I intend to mark that as half an hout10:57
ysandeep|ruckrlandy, okay its half an hour mtg now, start time works for you?10:58
rlandyysandeep|ruck: ack11:00
ysandeep|ruckgreat11:00
*** ysandeep|ruck is now known as ysandeep|ruck|afk11:01
*** pojadhav is now known as pojadhav|afk11:16
*** dviroel|out is now known as dviroel11:27
ysandeep|ruck|afkrlandy, network promoted.. .me triggering 17/9 line as well11:48
*** ysandeep|ruck|afk is now known as ysandeep|ruck11:50
rlandyysandeep|ruck: ack - I added a component promote job11:58
ysandeep|ruckoh thanks for doing that - I thought it promoted via scheduled run11:59
ysandeep|ruckrlandy++11:59
rlandychandankumar: 1-1?12:01
*** amoralej is now known as amoralej|lunch12:13
ysandeep|ruckrcastillo|rover, hey o/ you around?12:29
*** pojadhav|afk is now known as pojadhav12:30
ysandeep|ruckrlandy, dviroel https://sf.hosted.upshift.rdu2.redhat.com/logs/aa/aa2924a3fca31b90e24dd1623f549538da2910f5/openstack-periodic-integration-rhos-17/periodic-tripleo-ci-rhel-8-scenario001-standalone-rhos-17/0d657cc/logs/undercloud/home/zuul/tripleo-deploy/standalone-ansible-wbn_or3c/cephadm/cephadm_command.log12:56
rcastillo|roverysandeep|ruck: just got in12:57
*** dasm|off is now known as dasm12:58
dasmo/12:58
bhagyashrisfolks scrum time13:00
dasmmarios: thanks for reviow on: https://review.rdoproject.org/r/c/config/+/43904 I was informed you can't run testproject against "config". Can I do it somehow?13:04
*** amoralej|lunch is now known as amoralej13:14
ysandeep|ruckmarios, also saw same in upstream today: https://review.opendev.org/c/openstack/puppet-tripleo/+/848733/3#message-b8a44d79982074120be1fdc0652de3217f755c9013:23
bhagyashrishttps://review.rdoproject.org/r/q/topic:ooo_c7_teardown+owner:pojadhav%2540redhat.com13:26
mariosthanks ysandeep|ruck 13:26
bhagyashrishttps://review.opendev.org/q/topic:tempest_allow_list+owner:pojadhav%2540redhat.com+status:open13:26
bhagyashrishttps://review.opendev.org/c/openstack/tripleo-ci/+/84826013:29
bhagyashrishttps://review.rdoproject.org/r/c/rdo-jobs/+/4389713:30
bhagyashrishttps://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/41145813:30
rcastillo|roverrlandy: we're getting rid of the virthost jobs? did I get this right?13:36
rlandyrcastillo|rover: ? where?13:37
rcastillo|roveralan's latest email in that chain13:37
rlandywe need to add virthost jobs in vexx13:37
rlandywe are getting rid of them in ci,centos13:37
rcastillo|roveroh, ok, must've misread13:38
rlandyresponded13:39
dasmarxcruz: o/ did you have an opportunity to review https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/43815 again?13:47
ysandeep|ruckdviroel, rlandy rcastillo|rover fyi.. 16.2 full tempest seems legit: https://bugzilla.redhat.com/show_bug.cgi?id=2104931 and cixed.13:50
dviroelysandeep|ruck: ack, thanks13:51
rlandyysandeep|ruck: ack - thanks13:54
ysandeep|ruckspoke with neutron ralonsoh already - we need newer ovsdbapp version to fix this13:54
ysandeep|rucklet me ping delivery13:54
mariosysandeep|ruck: o/ will you join us 14:33
mariosysandeep|ruck: (operator call)14:33
ysandeep|ruckmarios: thanks for reminder, I lost track of time14:33
mariosnp14:48
rlandyysandeep|ruck: you EoD?15:41
ysandeep|ruckyeah but trying to figure out sc01/rhel-8 before leaving 15:42
rlandyPSI still looks out15:42
rlandyok  - let me know15:42
rlandystill need to promote master, right?15:42
rlandyysandeep|ruck: ^^?15:42
ysandeep|ruckrlandy, https://review.rdoproject.org/r/c/testproject/+/3834815:43
ysandeep|ruckwe still need fs035/20/sc010-kvm internal 15:43
rlandyok - 20 and 35 need rerun15:43
rlandyok - modifying that patch to rerun15:44
rlandyrcastillo|rover: ^^15:44
rcastillo|roverrlandy: ack15:44
rlandyPSI is dead- not sure about internal15:45
rlandymay nedd to skip it to promote15:45
rlandywallaby c8 just promoted15:47
ysandeep|ruckrlandy, rhel8/sc01 is weird - my patch which fixed 17/9 issue is not there in rhel-8 tht rpm15:47
ysandeep|ruckchecking with john15:47
rlandyysandeep|ruck: stuck in a component line?15:47
rlandyor not on dlrn at all?15:47
ysandeep|rucki am checking current(its even not here)15:48
rlandyysandeep|ruck: sounds like dlrn15:50
rlandydelivery15:50
*** marios is now known as marios|out15:52
ysandeep|ruckrlandy, sry should have written full name, I mean checking with jschlueter15:52
rlandylol - too many jon/john people15:53
ysandeep|ruckrlandy, do you see my ping in delivery channel?15:53
ysandeep|rucklooks like many people disconnected15:53
ysandeep|rucklooks like some irc issue15:53
rlandyyep15:54
rlandyysandeep|ruck; I can ping again later15:54
rlandyin case jon got disconnected15:54
rlandylet me know when you are out15:54
rlandywill follow up15:54
ysandeep|ruckrlandy, I am out.. you know the drill.. once dlrn enables back for 17/rhel8 - we promote tripleo and then rerun integration line16:18
rlandyysandeep|ruck; ack16:19
ysandeep|ruckrcastillo|rover, notes for you .. psi is down atm16:22
*** amoralej is now known as amoralej|off16:22
ysandeep|ruckrcastillo|rover, once its back 1) we need to promote integration 17/9 and we are good16:23
rcastillo|roverysandeep|ruck: ack, good night16:24
dviroelysandeep|ruck: o/ 16:25
ysandeep|ruck17/8 once dlrn is running again(its stopped right now) -> we need to promote tripleo -> then intgration line -> again promote all the component -> again integation line16:25
rlandylunch quickly16:25
rlandybiab16:25
ysandeep|ruckgood night guys thanks!16:26
ysandeep|ruckdviroel, rcastillo|rover you can check delivery channel for 17/8 dlrn discussion16:27
rcastillo|roverysandeep|ruck: yeah, following there16:27
*** ysandeep|ruck is now known as ysandeep|out16:28
*** dviroel is now known as dviroel|lunch17:13
rlandyrcastillo|rover: dviroel|lunch: psi looks back17:35
rcastillo|roverrlandy: nice17:35
rlandyrerunning failed rhel-9jobs17:36
rlandyrekicking line - should be easiest17:36
*** dviroel|lunch is now known as dviroel18:11
dviroelI will start deleting all volumes "available" in dev-stage tenant18:33
rlandydviroel++18:40
dviroelrlandy: dasm: review meeting :)19:01
rlandydviroel: talking to dasm - pls start - we will join shortly19:03
dasmin a sec19:03
rlandyrcastillo|rover: master fs020 passed - if fs035 is not a consistent failure, you cam skip that and kvm internal (psi not deploying centos 9 nodes) and promote19:08
rcastillo|roverrlandy: what hash is that? rr script is giving me different failures19:28
rlandyrcastillo|rover: https://review.rdoproject.org/r/c/testproject/+/38348 - was the rerun ...19:29
rcastillo|roverrlandy: nvm just saw19:29
rlandy e8b18652b6c4d5d4b407b9c8bc14063b19:29
rlandyso ...19:29
rlandyrcastillo|rover: python3 roles/rrcockpit/files/telegraf_py3/ruck_rover.py --release master  --distro centos-9 --aggregate_hash tripleo-ci-testing/e8/b1/e8b18652b6c4d5d4b407b9c8bc14063b19:30
rlandy┃ Jobs in promotion criteria required to promo the hash:                           ┃ PASSED History  ┃ FAILURE History ┃ Other History   ┃19:30
rlandy┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩19:30
rlandy│ periodic-tripleo-ci-centos-9-scenario010-kvm-internal-standalone-master          │ 0               │ 0               │ 0               │19:30
rlandy│ periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master                │ 0               │ 4               │ 1               │19:30
rcastillo|roverrlandy: k, thanks, just rerran fs03519:31
rcastillo|roverlet's see if it lets me run kvm19:31
rlandyrcastillo|rover: afaict, kvm job won't start19:32
rlandysee comment on #rhos-ops19:32
rlandyalso fs035 does not have a great history19:32
rlandyif your failures are not consistent ok to skip promote19:33
rlandyaster is 3 days out now19:33
rlandymaster19:33
rcastillo|roverrlandy: k, tentative patch here https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4391819:37
rcastillo|roverwaiting on fs03519:37
rlandyrcastillo|rover: k - let me know when you want me to w+19:37
rcastillo|roverack19:38
* dviroel out20:42
*** dviroel is now known as dviroel|out20:42
* dasm => off21:02
*** dasm is now known as dasm|off21:02
rlandyrcastillo|rover: f035 just reported22:22
rcastillo|roverindeed, let me add it to the patch as well22:23
rcastillo|roverrlandy: pls workflow when you can https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4391822:24
rlandyrcastillo|rover: checking the tempest results22:24
rlandyto see if there is not a real issue22:25
rcastillo|roverlots of tests failed both, but not the same22:26
rlandyrcastillo|rover: not exact;y22:29
rcastillo|roverhard to ignore so many failures22:29
rlandyrcastillo|rover; ok ... so these two are on the same hash ...22:31
rlandyhttps://logserver.rdoproject.org/48/38348/26/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/a92feb0/logs/undercloud/var/log/tempest/stestr_results.html.gz22:31
rlandyhttps://logserver.rdoproject.org/48/38348/26/check/periodic-tripleo-ci-centos-9-ovb-3ctlr_1comp-featureset035-master/a92feb0/logs/undercloud/var/log/tempest/stestr_results.html.gz22:31
rlandy^^ cleanest22:31
rlandyso merging patch22:31
rlandyrcastillo|rover: just fyi - easier to comment out for next time22:32
rlandyrcastillo|rover: I +2'ed and w+22:32
rlandypls revert once promotion22:33
rlandyalso rerunning one failed rho-17 on rhel-9 job22:33
rcastillo|roverrlandy: k sure22:35
*** rlandy is now known as rlandy|bbl22:44
rcastillo|rovernot sure why master promotion hasn't gone through23:43
rcastillo|roverput in the revert anyway23:43
rcastillo|roverhttps://review.rdoproject.org/r/c/rdo-infra/ci-config/+/4368423:43
* rcastillo|rover out23:43
*** rcastillo|rover is now known as rcastillo23:43

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!