| *** rlandy|afk is now known as rlandy|out | 00:51 | |
| *** akahat is now known as akahat|ruck | 05:00 | |
| *** ysandeep|out is now known as ysandeep | 05:05 | |
| *** chkumar|rover is now known as chandankumar | 05:05 | |
| *** undefined is now known as Guest5693 | 05:12 | |
| *** amoralej|off is now known as amoralej | 06:17 | |
| jm1 | happy friday :) | 06:29 |
|---|---|---|
| amoralej | chandankumar, i have https://review.rdoproject.org/r/c/rdoinfo/+/43728 ready to merge | 06:53 |
| amoralej | it's doing major updates of ovn and ovs | 06:53 |
| *** arxcruz is now known as arxcruz|rover | 06:54 | |
| amoralej | ykarel has been testing it and looks fine | 06:54 |
| amoralej | is there any job doing openstack version upgrades? | 06:54 |
| amoralej | i'm afraid those may be affected | 06:54 |
| bhagyashris | folks fyi Network issues observered with Pune and Singapore VPN | 06:55 |
| chandankumar | amoralej: there is only undercloud upgrade job from wallaby to master, in tht patch, ykarel ttested it | 07:02 |
| chandankumar | he covered almost all job | 07:02 |
| chandankumar | I think it's good to go | 07:02 |
| amoralej | nice | 07:02 |
| amoralej | chandankumar, i'll merge it then | 07:03 |
| akahat|ruck | amoralej, ykarel I'm seeing host unrechable https://a23b1dcd976a3689ccdb-24d23f95509aa29f3f112a150d6939cd.ssl.cf2.rackcdn.com/850398/1/check/tripleo-ci-centos-9-undercloud-containers/eaea8b7/logs/undercloud/home/zuul/undercloud_install.log | 07:05 |
| akahat|ruck | can you check before merging it. | 07:05 |
| akahat|ruck | It's against ovn check pathhttps://review.opendev.org/c/openstack/tripleo-quickstart/+/850398 | 07:06 |
| akahat|ruck | amoralej, ykarel it might not related but just want to make sure. | 07:06 |
| akahat|ruck | It's passed on tht: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/850502/1#message-f481c2e3275936dae9f6c9854f3be9e06a0e54ba | 07:07 |
| ykarel | akahat|ruck, that's unrelated failure | 07:08 |
| ykarel | as that test was run at time mirror was not up to date and buggy openvswitch2.17 got installed | 07:09 |
| akahat|ruck | ykarel, ok. | 07:09 |
| akahat|ruck | ykarel, Thanks for confirming. | 07:09 |
| amoralej | ykarel, so, good to merge? | 07:09 |
| ykarel | amoralej, can re vote :) | 07:09 |
| amoralej | good :) | 07:10 |
| akahat|ruck | arxcruz|rover, o/ | 07:15 |
| arxcruz|rover | akahat|ruck hey man! | 07:15 |
| akahat|ruck | arxcruz|rover, let's have sync: https://meet.google.com/acw-bzrw-krh | 07:16 |
| akahat|ruck | jm1, o/ | 07:40 |
| akahat|ruck | jm1, Non integer values in LooseVersion('stable/1.0.0') | 07:41 |
| akahat|ruck | jm1, https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-9-scenario004-standalone-common-rhos-17/9ec934c/job-output.txt | 07:41 |
| *** ysandeep is now known as ysandeep|lunch | 07:44 | |
| jm1 | akahat|ruck: having a look | 07:49 |
| akahat|ruck | jm1, for record i've created lp: https://bugs.launchpad.net/tripleo/+bug/1982552 | 07:51 |
| jm1 | akahat|ruck: this downstream only. rdo is not affected https://logserver.rdoproject.org/openstack-component-common/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-9-scenario004-standalone-common-master/3a2ed24/job-output.txt | 07:53 |
| jm1 | akahat|ruck: do we use lp for downstream? | 07:53 |
| akahat|ruck | jm1, yes. it's downstream. | 07:54 |
| akahat|ruck | jm1, ah.. old habbit.. no we use bugzilla. I'll update. | 07:54 |
| jm1 | akahat|ruck: i think we need ysandeep|lunch here | 07:55 |
| jm1 | akahat|ruck: rhos is behaving different from tripleo here | 07:56 |
| * akahat|ruck lunch | 08:04 | |
| jm1 | akahat|ruck, ysandeep|lunch, arxcruz|rover: updated akahat|ruck's bug report for aoc. this is actually not a bug in aoc but in the downstream job, so we need some help here | 09:29 |
| arxcruz|rover | jm1 ack, thanks for your thoughts on that | 09:30 |
| akahat|ruck | jm1, ack.. I'll re run failed job and wait for results. | 09:31 |
| *** ysandeep|lunch is now known as ysandeep | 09:42 | |
| jpodivin | akahat|ruck: hi. I'm seeing tempest failure in jobs we have derived from tripleo-ci-centos-9-standalone on wallaby https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-standalone-tv-validation&branch=stable%2Fwallaby&skip=0 | 09:54 |
| jpodivin | it's always the same test tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_instance_hard_reboot_after_vol_snap_deletion | 09:54 |
| jpodivin | Is it being tracked/ Or should I file it? | 09:56 |
| akahat|ruck | jpodivin, o/ looking | 10:12 |
| jpodivin | akahat|ruck: I've filed the bug https://bugs.launchpad.net/tripleo/+bug/1982570 | 10:14 |
| jpodivin | all the logs are there | 10:15 |
| jpodivin | and some extra info | 10:15 |
| akahat|ruck | jpodivin, yeah.. this looks like new. | 10:15 |
| akahat|ruck | Thanks for opening lp. | 10:15 |
| akahat|ruck | jpodivin, to unblock you for now i'll add that test to skiplist | 10:15 |
| jpodivin | akahat|ruck: thank you very much :) | 10:15 |
| akahat|ruck | arxcruz|rover, ^^ | 10:18 |
| arxcruz|rover | akahat|ruck ack | 10:20 |
| arxcruz|rover | akahat|ruck jpodivin duplicate of https://bugs.launchpad.net/tripleo/+bug/1980255 | 10:22 |
| jpodivin | arxcruz|rover: that itself is a dupe of https://bugs.launchpad.net/tripleo/+bug/1923194 in turn | 10:24 |
| arxcruz|rover | jpodivin inception!!! | 10:24 |
| jpodivin | :D | 10:24 |
| arxcruz|rover | jpodivin fyi https://review.opendev.org/c/openstack/tempest/+/849562 fix it, need to make some changes as per kopecmartin comment | 10:25 |
| akahat|ruck | arxcruz|rover, in this job different tests are failed in same file: https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-9-standalone-tv-validation&branch=stable%252Fwallaby&skip=0 | 10:25 |
| akahat|ruck | arxcruz|rover, so should we add it to skiplist or not? | 10:25 |
| arxcruz|rover | akahat|ruck yeah, it affect some other tests | 10:25 |
| arxcruz|rover | akahat|ruck that's a good questiion, because it's not all the time, depends on the concurrency | 10:26 |
| arxcruz|rover | let me work on the tempest test | 10:26 |
| akahat|ruck | yeah.. different tests same issue. | 10:27 |
| jpodivin | akahat|ruck, arxcruz|rover: bug marked as duplicate and info posted to the other two. | 10:28 |
| jpodivin | should at least help to keep track of things | 10:29 |
| arxcruz|rover | jpodivin ok, thanks, i tried to mark as dup, but you did it already | 10:29 |
| * jpodivin faster | 10:29 | |
| arxcruz|rover | yeah, i need to work on that patch asap, i was waiting for artom to update it, but he didn't so i'll do it myself | 10:29 |
| arxcruz|rover | we are bleeding on those tests | 10:29 |
| arxcruz|rover | jpodivin | 10:29 |
| arxcruz|rover | btw, i bought a new keyboard, vim is even better now :D | 10:30 |
| jpodivin | What did you get? | 10:30 |
| arxcruz|rover | jpodivin moonlander one | 10:31 |
| arxcruz|rover | jpodivin https://www.zsa.io/moonlander/ | 10:32 |
| jpodivin | cool, futuristic | 10:32 |
| *** rlandy|out is now known as rlandy | 10:33 | |
| arxcruz|rover | it's good, my shoulders now are much better with the splitted layout | 10:33 |
| arxcruz|rover | i don't need to move my hands too much as well | 10:33 |
| arxcruz|rover | rlandy hey, wanna sync? not much happened today though | 10:34 |
| rlandy | arxcruz|rover: yeah | 10:34 |
| jpodivin | I see. I'll have to look into a new soon, so thanks for the tip. | 10:34 |
| jpodivin | arxcruz|rover, akahat|ruck: thanks to you both, for the help. | 10:34 |
| rlandy | akahat|ruck: arxcruz|rover: https://meet.google.com/iqu-bdnu-oni?pli=1&authuser=0 | 10:34 |
| akahat|ruck | jpodivin, np :) | 10:35 |
| arxcruz|rover | akahat|ruck do you still have the link for the scen10 that run the test ? | 10:51 |
| akahat|ruck | arxcruz|rover, yes. one min | 10:51 |
| akahat|ruck | arxcruz|rover, https://code.engineering.redhat.com/gerrit/c/testproject/+/301027/14/.zuul.yaml | 10:53 |
| dviroel | o/ | 11:11 |
| arxcruz|rover | rlandy i put the network component in train for rechec, because checking the latest failres, they were all random failures in different places | 11:18 |
| rlandy | arxcruz|rover: thanks | 11:31 |
| rlandy | ysandeep: hello - pls review: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/44126/1/ci-scripts/infra-setup/roles/rrcockpit/files/telegraf_py3/ruck_rover.py#b607 | 11:31 |
| rlandy | dasm|off: hi - we're looking at your patch ... can you explain more? | 11:32 |
| ysandeep | okay, will check soon | 11:33 |
| rlandy | not sure I understand or agree | 11:34 |
| ysandeep | dviroel, https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_release/30638/rehearse-30638-pull-ci-openstack-k8s-operators-mariadb-operator-master-golangci/1550144330440642560/build-log.txt | 11:42 |
| ysandeep | rlandy, left a comment for dasm|off on https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/44126/1#message-3b0999005bcc1f28eca6f6f1b1b2833e5e0a6611 based on our discussion. | 12:00 |
| dviroel | rlandy: https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/850349 - if you want to review and merge, or wait until monday, up to you | 12:04 |
| * rlandy looks | 12:06 | |
| rlandy | dviroel: will w+ - thanks | 12:07 |
| rlandy | ysandeep: ty | 12:07 |
| ysandeep | dviroel: fyi.. bumping golang version didn't help | 12:09 |
| dviroel | ysandeep: oh, ok, | 12:13 |
| ysandeep | but good news is rest of the config jobs passed which failed earlier. | 12:14 |
| *** amoralej is now known as amoralej|lunch | 12:20 | |
| dviroel | ysandeep: https://prow.ci.openshift.org/pr-history/?org=openshift&repo=release&pr=30638 this? | 12:21 |
| dviroel | the owners is passing now :) | 12:22 |
| ysandeep | yes | 12:22 |
| * akahat|ruck coffee | 12:23 | |
| ysandeep | dviroel: fyi.. this one is different failure from the rest, trying to debug this atm: https://storage.googleapis.com/origin-ci-test/pr-logs/pull/openshift_release/30638/rehearse-30638-pull-ci-openstack-k8s-operators-mariadb-operator-master-golangci/1550445300370378752/build-log.txt | 12:26 |
| dviroel | ysandeep: ok, this is the fun part :) | 12:28 |
| chandankumar | rlandy: 1:1 | 12:31 |
| ysandeep | dviroel, yeah.. working locally http://pastebin.test.redhat.com/1066974 , breaking in ci only | 12:33 |
| chandankumar | rlandy: https://review.rdoproject.org/r/c/testproject/+/44131 | 12:34 |
| dviroel | ysandeep: ack, i will take a look and try on my side too | 12:36 |
| dviroel | ysandeep: checking failures on my hive ocp cluster | 12:36 |
| jm1 | ysandeep: hello :) could we maybe debug this one on monday? https://bugzilla.redhat.com/show_bug.cgi?id=2109807 | 12:46 |
| * jm1 bbl | 12:53 | |
| ysandeep | jm1: is that still an issue? | 12:58 |
| ysandeep | jm1, I see sc04 in manilla just passed: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/build/40d2c48f67ae4d4eb2ad3db5885c2563 | 12:59 |
| ysandeep | akahat|ruck, ^^ ? | 12:59 |
| ysandeep | jm1, akahat|ruck or that is specific to common component ? | 12:59 |
| akahat|ruck | ysandeep, no. it's appears sometimes.. That job where it is reported.. ran again.. but didn't see that issue again | 13:00 |
| akahat|ruck | ysandeep, but sometimes it appears.. to ghost us. | 13:00 |
| ysandeep | ansible galaxy is acting up? | 13:01 |
| akahat|ruck | I'll keep that bz open till monday.. if i found more issues around it. | 13:01 |
| ysandeep | akahat|ruck, ah okay its intermittent.. eww.. they are the worst to debug, just curious - whats the frequency - apart from those 2 logs example - how many jobs are affected? | 13:05 |
| akahat|ruck | ysandeep, only 2 jobs found today.. | 13:05 |
| akahat|ruck | :) | 13:06 |
| ysandeep | ack, lets monitor.. i have never seen this error but i suspect it came from galaxy | 13:06 |
| *** ysandeep is now known as ysandeep|afk | 13:07 | |
| *** amoralej|lunch is now known as amoralej | 13:07 | |
| *** Guest5693 is now known as rcastillo | 13:09 | |
| *** ysandeep|afk is now known as ysandeep | 13:31 | |
| *** ysandeep is now known as ysandeep|mtg | 13:31 | |
| *** dasm|off is now known as dasm | 14:12 | |
| dasm | o/ | 14:12 |
| chandankumar | rlandy: please keep an eye on this one https://review.rdoproject.org/zuul/status#44135 | 14:18 |
| chandankumar | happy weekend people \o | 14:18 |
| rlandy | chandankumar: cool - thanks | 14:19 |
| dasm | ftr: https://review.rdoproject.org/r/c/testproject/+/44135 | 14:19 |
| rlandy | dviroel++ ysandeep|mtg++ nice work | 14:48 |
| rlandy | arxcruz|rover: akahat|ruck: need anything from me? going into a bunch of meetings are after this | 15:16 |
| akahat|ruck | rlandy, no. | 15:16 |
| rlandy | akahat|ruck: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?job_name=periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-17&skip=0 | 15:16 |
| arxcruz|rover | not from my side | 15:16 |
| rlandy | real issue here | 15:16 |
| rlandy | ugh - failing at a bunch of different places | 15:18 |
| akahat|ruck | rlandy, yeah.. i saw that.. but didn't fond any consistent failure | 15:19 |
| rlandy | annoying | 15:19 |
| *** dviroel is now known as dviroel|lunch | 15:25 | |
| arxcruz|rover | akahat|ruck rlandy i'll be around for 30 more minutes | 15:26 |
| *** undefined is now known as Guest5751 | 15:26 | |
| *** ysandeep|mtg is now known as ysandeep|out | 15:26 | |
| *** Guest5751 is now known as rcastillo_ | 15:26 | |
| *** johnsom_ is now known as johnsom | 15:27 | |
| rlandy | arxcruz|rover: k - let me know what you want me to watch after then | 15:29 |
| rlandy | arxcruz|rover: train components still a problem? | 15:30 |
| rlandy | also we can prom wallaby c9 - f we get fs001 to pass | 15:30 |
| arxcruz|rover | rlandy | 15:30 |
| rlandy | oops c8 | 15:30 |
| arxcruz|rover | rlandy random failures | 15:30 |
| arxcruz|rover | master is failing on 65 | 15:31 |
| arxcruz|rover | rlandy i'm rechecking now | 15:31 |
| rlandy | k - thanks | 15:31 |
| arxcruz|rover | sorry, master is failing on 35 and 64 | 15:32 |
| arxcruz|rover | wallaby is failing on 64 | 15:33 |
| arxcruz|rover | rechecking as well | 15:33 |
| rlandy | dviroel|lunch: dasm: rcastillo_: pls will you give https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/42701 another review | 15:33 |
| rlandy | then we can get this sorted for akahat|ruck | 15:33 |
| rcastillo_ | rlandy: ack | 15:36 |
| *** amoralej is now known as amoralej|off | 15:38 | |
| * akahat|ruck offline | 16:17 | |
| rlandy | lunch | 16:33 |
| rlandy | brb | 16:33 |
| *** dviroel|lunch is now known as dviroel | 16:42 | |
| dviroel | rlandy: pls see https://review.opendev.org/c/openstack/tripleo-ci/+/850770/ - fix graph issues | 17:26 |
| dviroel | rlandy: this https://review.opendev.org/c/openstack/tripleo-heat-templates/+/848601 was hitting the issue, conflicting branches within same job def | 17:27 |
| *** dviroel is now known as dviroel|afk | 17:28 | |
| rlandy | looking | 17:29 |
| rlandy | dviroel|afk: thanks w+'ed | 17:32 |
| rlandy | rcastillo_: dasm;looks like downstream gerritt is on weekend break already | 17:39 |
| dasm | :/ | 17:42 |
| *** rcastillo_ is now known as rcastillo | 17:54 | |
| jm1 | leaving for today folks, have a nice weekend :) 🍻 | 18:41 |
| dasm | jm1: o/ | 18:42 |
| dasm | stepping away for a bit. | 18:47 |
| dasm | back | 19:38 |
| rlandy | arxcruz|rover: akahat|ruck: left you note about train tripleo component and 16.2 and 17 ovb failures | 19:45 |
| arxcruz|rover | rlandy you added on the previous rr hackmd, this is the new one https://hackmd.io/lfpJ3NdRTXaTah6t3C1oGw i'll update it | 20:12 |
| rlandy | oops | 20:13 |
| rlandy | will switch | 20:13 |
| arxcruz|rover | rlandy periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-network-train passes, kvm fails, rerunning, also, master should get promoted if not yet, all jobs passed | 20:16 |
| rlandy | arxcruz|rover: added https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/44140 Skip internal-kvm job - will revert | 20:24 |
| rlandy | downstream is down | 20:24 |
| rlandy | kvm master will not promote with it | 20:24 |
| rlandy | skipping | 20:24 |
| arxcruz|rover | rlandy want to skip train network kvm job as well? the failing job passed | 20:25 |
| arxcruz|rover | the featureset001 | 20:25 |
| rlandy | arxcruz|rover: yep - doing thta | 20:31 |
| rlandy | that | 20:31 |
| rlandy | arxcruz|rover: https://review.rdoproject.org/r/c/testproject/+/31165 | 20:32 |
| rlandy | arxcruz|rover: it's late for you | 20:42 |
| rlandy | need to run out for a few | 20:42 |
| *** rlandy is now known as rlandy|biab | 20:42 | |
| *** rlandy|biab is now known as rlandy | 21:17 | |
| rlandy | arxcruz|rover: I updated https://hackmd.io/lfpJ3NdRTXaTah6t3C1oGw | 21:31 |
| rlandy | see note on line 27 | 21:31 |
| rlandy | 37 | 21:31 |
| rlandy | have a good weekend all | 22:03 |
| dasm | see you! | 22:05 |
| rcastillo | going as well | 22:05 |
| rcastillo | talk to you next week! | 22:05 |
| * dasm => offline | 22:11 | |
| dasm | me too :) | 22:11 |
| *** dasm is now known as dasm|off | 22:11 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!