| *** rlandy|ruck is now known as rlandy|out | 01:07 | |
| pojadhav | rlandy|out, ack | 04:28 |
|---|---|---|
| *** ysandeep is now known as ysandeep|lunch | 07:26 | |
| *** bhagyashris_ is now known as bhagyashris | 08:26 | |
| *** ysandeep|lunch is now known as ysandeep | 08:53 | |
| jpodivin | akahat|ruck: Hi, I wanted to ask if you know what could be behind the https://bugs.launchpad.net/tripleo/+bug/1956234 | 09:16 |
| akahat|ruck | jpodivin, few mins.. in middle of something. | 09:26 |
| jpodivin | akahat|ruck: sure, np | 09:26 |
| akahat|ruck | jpodivin, i think it is mirror issue. | 09:55 |
| akahat|ruck | jpodivin, may be amoralej can help us here. ^^ | 09:56 |
| jpodivin | akahat|ruck: yeah, seems like it. Trouble is that it's blocking our check pipeline | 09:56 |
| jpodivin | akahat|ruck: it's strange though, because nobody else seems to be hitting it, just validations. | 09:56 |
| * amoralej checks | 09:56 | |
| amoralej | jpodivin, akahat|ruck yes, looks like an infra issue, it has happen multiple times? | 09:57 |
| jpodivin | amoralej: yes, since monday | 09:58 |
| jpodivin | amoralej, akahat|ruck: https://zuul.opendev.org/t/openstack/builds?job_name=tripleo-ci-centos-8-undercloud-containers&project=openstack/validations-libs | 09:58 |
| jpodivin | there have been problems before, but not on this scale. | 09:59 |
| amoralej | actually, ini the errors since monday, i see three failed jobs, one of them is unrelated to this issue | 10:03 |
| amoralej | right? | 10:03 |
| jpodivin | yep, I believe so | 10:06 |
| jpodivin | The older ones are a different issue. | 10:06 |
| amoralej | https://a79be49d26e60f3be927-f7d1006348fe8b0ba5afe04a60cb30e4.ssl.cf1.rackcdn.com/802901/16/check/tripleo-ci-centos-8-undercloud-containers/6074831/logs/undercloud/home/zuul/undercloud_install.log | 10:06 |
| amoralej | that's the issue in https://zuul.opendev.org/t/openstack/build/60748310309b4379b8e29a6923d13a95/logs | 10:07 |
| jpodivin | amoralej: but that's not really where the problem is. The mirrors are | 10:07 |
| amoralej | then Timeout was reached for http://mirror.centos.org/centos/8/opstools/x86_64/collectd-5/repodata/9fda86d5505b845fa8f03abc506932f2bac9d6a565246e93335babbe649de481-primary.xml.gz | 10:07 |
| amoralej | jpodivin, in that job? | 10:07 |
| amoralej | i didn't find any mirror issue in that particular job | 10:07 |
| jpodivin | amoralej: and the timeout? | 10:08 |
| amoralej | that's a different job | 10:08 |
| amoralej | i'm checking the three failed jobs pointed | 10:08 |
| jpodivin | Well that is what the https://bugs.launchpad.net/tripleo/+bug/1956234 is about | 10:08 |
| jpodivin | Patch doesn't touch mirrors at all, so this shouldn't be happening unless something else changed somewhere . | 10:12 |
| amoralej | jpodivin, i left a comment | 10:21 |
| amoralej | i think it's some kind of infra issue, it could be improved in a couple of ways | 10:22 |
| jpodivin | amoralej: is it related to centos8 EOL by any chance? | 10:23 |
| amoralej | no | 10:28 |
| soniya29 | kopecmartin, arxcruz, chandankumar, ysandeep, rlandy|out, please add/edit today's agenda for the tempest meeting - https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ | 10:28 |
| amoralej | see one of the issues is in centos/8/ but the other one is in centos/8-stream/ | 10:28 |
| jpodivin | amoralej: I still don't understand why are we the only ones hitting this. | 10:32 |
| amoralej | dunno | 10:32 |
| amoralej | i don't see anything special in those jobs | 10:32 |
| jpodivin | amoralej: The build-test-packages task is the only divergence | 10:33 |
| jpodivin | well role | 10:33 |
| amoralej | well, that's a relevant diff | 10:33 |
| amoralej | but iiuc, every tripleo repo is using it, isn't it? | 10:33 |
| amoralej | note building with dlrn means using repos from mirror.centos.org | 10:34 |
| jpodivin | ... not exactly. I went trough the logs of this job for other projects and that role isn't getting triggered there. | 10:34 |
| amoralej | weird | 10:34 |
| amoralej | lemme check tht i.e. | 10:34 |
| jpodivin | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a9f/822835/4/check/tripleo-ci-centos-8-undercloud-containers/a9f6ade/job-output.txt | 10:34 |
| jpodivin | For example oooclient ^ | 10:34 |
| amoralej | mmm mayte oooclient is used from code | 10:35 |
| amoralej | *maybe | 10:35 |
| amoralej | check others | 10:35 |
| amoralej | from tht i.e. | 10:36 |
| jpodivin | same deal. https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_718/822948/2/check/tripleo-ci-centos-8-undercloud-containers/718bdd7/job-output.txt | 10:36 |
| amoralej | jpodivin, see https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a07/822835/4/check/tripleo-ci-centos-9-content-provider/a0788a1/job-output.txt | 10:37 |
| amoralej | for the same review | 10:37 |
| amoralej | it's running buikld-test-packages | 10:37 |
| amoralej | and passing | 10:37 |
| amoralej | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_a07/822835/4/check/tripleo-ci-centos-9-content-provider/a0788a1/logs/delorean_logs/component/tripleo/59/b2/59b283b47863d98041a87c94df6f13bbc4f5df1c_dev/index.html | 10:37 |
| jpodivin | amoralej: why would it run different roles one time than the other: | 10:38 |
| amoralej | well, those are different jobs | 10:38 |
| amoralej | i checked tripleo-ci-centos-8-content-provider | 10:38 |
| jpodivin | amoralej: just noticed the path, right. | 10:38 |
| amoralej | someone in this channel must know :) | 10:38 |
| amoralej | but as the provider create containers and stuff used by other nested jobs | 10:39 |
| amoralej | it may be responsible of building the rpms for those too | 10:39 |
| jpodivin | amoralej: I mean, the role is there, in repo. | 10:39 |
| amoralej | there are parameters | 10:39 |
| amoralej | to enable/disable build-test-packages iirc | 10:40 |
| jpodivin | amoralej: that would still show that the inclusion was skipped. But this gave me an idea. The content provider, validations is the only project of the three not using it. | 10:41 |
| jpodivin | Essentially, whatever is going wrong, content provider shields the other projects from it. | 10:42 |
| jpodivin | somehow | 10:42 |
| jpodivin | I can try some dnm patches to test the hypothesis. | 10:44 |
| chandankumar | dviroel|out: akahat|ruck https://bugs.launchpad.net/tripleo/+bug/1956449 fyi! | 11:06 |
| * bhagyashris stepping out for bit | 11:08 | |
| *** dviroel|out is now known as dviroel|rover | 11:12 | |
| *** rlandy|out is now known as rlandy|ruck | 11:13 | |
| rlandy|ruck | soniya29: added item | 11:17 |
| rlandy|ruck | chandankumar: hey - joined review time | 11:19 |
| rlandy|ruck | akahat|ruck: hello | 11:19 |
| rlandy|ruck | akahat|ruck: what's the status of component lines? | 11:19 |
| chandankumar | rlandy|ruck: joining | 11:20 |
| rlandy|ruck | frenzy_friday: hey - did your testproject pass for scenario010? | 11:32 |
| rlandy|ruck | frenzy_friday: https://code.engineering.redhat.com/gerrit/c/testproject/+/301027 - still failing on zuul cloner | 11:36 |
| rlandy|ruck | pls take a look | 11:36 |
| frenzy_friday | rlandy|ruck, yes, regarding the missing virtualenv I think we need to use ensure-venv role somewhere. Not sure where. Checking | 11:43 |
| dviroel|rover | frenzy_friday: hi, shouldn't be centos-9 here https://review.rdoproject.org/r/c/testproject/+/37738/2/.zuul.yaml#8 | 11:46 |
| frenzy_friday | dviroel|rover, right, I think so. Thanks! updating | 11:48 |
| dviroel|rover | ++ | 11:50 |
| dviroel|rover | chasing master, victoria and train | 12:00 |
| chandankumar | soniya29: skipping tempest meeting | 12:00 |
| soniya29 | chandankumar, okay :) | 12:00 |
| rlandy|ruck | chandankumar: can we close this out? https://trello.com/c/P0vCbwrd/2278-cixlp1956319tripleociproa-error-no-matching-repo-to-modify-ubi-on-rhel-9-container-build | 12:01 |
| chandankumar | rlandy|ruck: waiting for backport https://review.opendev.org/c/openstack/tripleo-common/+/823451 to get merged | 12:02 |
| rlandy|ruck | akahat|ruck: dviroel|rover" let's meet after tempest meeting | 12:02 |
| akahat|ruck | rlandy|ruck, sure. | 12:03 |
| dviroel|rover | ok | 12:03 |
| akahat|ruck | rlandy|ruck, https://bugs.launchpad.net/tripleo/+bug/1956441 | 12:03 |
| rlandy|ruck | chandankumar: can we get sshnaidm to merge that? https://review.opendev.org/c/openstack/tripleo-common/+/823451 | 12:03 |
| rlandy|ruck | akahat|ruck: is that consistent? | 12:04 |
| rlandy|ruck | blocking check/gate/promotions? | 12:04 |
| rlandy|ruck | need CIX? | 12:04 |
| chandankumar | rlandy|ruck: +w https://review.opendev.org/c/openstack/tripleo-quickstart/+/822712 it | 12:04 |
| akahat|ruck | rlandy|ruck, i'm running jobs again.. let's see. | 12:06 |
| *** ysandeep is now known as ysandeep|mtg | 12:22 | |
| arxcruz | ysandeep|mtg: https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/tasks/tempest_resources.yml | 12:25 |
| rlandy|ruck | chandankumar: https://review.opendev.org/c/openstack/tripleo-ansible/+/823147 merged ... can we close out https://trello.com/c/QeKEues3/2271-cixlp1955962tripleociproa-destination-directory-etc-containers-does-not-exist? | 12:33 |
| chandankumar | rlandy|ruck: yes good to close | 12:34 |
| * dviroel|rover gets coffee | 12:35 | |
| akahat|ruck | reviews please https://review.opendev.org/c/openstack/tripleo-quickstart-extras/+/822344 | 12:37 |
| rlandy|ruck | akahat|ruck: dviroel|rover: https://meet.google.com/nmz-prva-snq | 12:43 |
| *** ysandeep|mtg is now known as ysandeep | 12:50 | |
| akahat|ruck | rlandy|ruck, https://review.rdoproject.org/r/c/testproject/+/36394 | 12:51 |
| *** amoralej is now known as amoralej|lunch | 12:58 | |
| kopecmartin | chandankumar: hi, please check this out when you have a moment https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/823493 | 13:01 |
| chandankumar | rlandy|ruck: https://review.opendev.org/c/openstack/ansible-role-collect-logs/+/823493 +w please | 13:04 |
| chandankumar | kopecmartin: thanks! | 13:04 |
| rlandy|ruck | done | 13:05 |
| kopecmartin | chandankumar: rlandy|ruck thank you | 13:06 |
| chandankumar | rlandy|ruck: https://review.rdoproject.org/r/c/rdo-jobs/+/37743 get this merged | 13:12 |
| rlandy|ruck | chandankumar: merged | 13:14 |
| chandankumar | rlandy|ruck: thanks! | 13:14 |
| rlandy|ruck | akahat|ruck: wrt ovb jobs | 13:14 |
| rlandy|ruck | lot so retries | 13:14 |
| rlandy|ruck | network 2022-01-05 12:37:23.694357 | primary | Failed to attach network adapter device to 9a3a12e4-b4a8-4e47-8318-0acc961e48cb (HTTP 500) (Request-ID: req-0d55ae9c-1eb7-4bc0-85ac-1b84b6aa5e7a) | 13:15 |
| akahat|ruck | rlandy|ruck, we hit this issue in many jobs. | 13:15 |
| rlandy|ruck | akahat|ruck: vexxhost | 13:16 |
| rlandy|ruck | akahat|ruck: there were infra issues on vesshost | 13:16 |
| rlandy|ruck | akahat|ruck; pls ping EoD on #rhos-ops | 13:16 |
| rlandy|ruck | akahat|ruck: mhu | 13:17 |
| rlandy|ruck | show him the retries | 13:17 |
| rlandy|ruck | and the network error | 13:17 |
| akahat|ruck | rlandy|ruck, okay. | 13:17 |
| chandankumar | is it 500 error from router? | 13:18 |
| chandankumar | let me get the exact error message | 13:19 |
| *** bhagyashris_ is now known as bhagyashris | 13:19 | |
| jm1 | Hi! RH's repo tripleo-environments often refers to host rdoci-doc.usersys.redhat.com but this hostname cannot be resolved. Has this host been replaced with another one? Does someone know how to access the docs of tripleo-environments? | 13:24 |
| bhagyashris | rlandy|ruck, chandankumar soniya29 ysandeep UA/TC/PM sync | 13:30 |
| bhagyashris | https://meet.google.com/hvj-rmjp-qep?authuser=0 | 13:30 |
| bhagyashris | https://hackmd.io/P5sTPMSBREe5kBS_4jDhIg#2022-01-05-tcuapm-planning-for-sprint-56 | 13:30 |
| dasm | o/ | 13:32 |
| bhagyashris | rlandy|ruck, ^ | 13:33 |
| rlandy|ruck | joining | 13:33 |
| marios | rlandy|ruck: mind if i join? i am missing planning tomorrow am out | 13:35 |
| rlandy|ruck | sure | 13:35 |
| soniya29 | arxcruz, chandankumar, marios, https://review.opendev.org/c/openinfra/python-tempestconf/+/820169 | 13:37 |
| soniya29 | rlandy|ruck, ^^ | 13:38 |
| soniya29 | please review whenever you have time | 13:39 |
| chandankumar | ysandeep: marios rlandy|ruck fyi https://review.rdoproject.org/r/q/topic:%22directord-bootstrap%22+(status:open%20OR%20status:merged) | 13:47 |
| rlandy|ruck | dviroel|rover: rcastillo: hey - can you guys update your jira sprint cards for the compose pinning work | 13:48 |
| dviroel|rover | rlandy|ruck: yes, will do | 13:49 |
| rlandy|ruck | thank you, sir | 13:50 |
| marios | soniya29: adding to review list | 13:56 |
| marios | chandankumar: thanks | 13:57 |
| soniya29 | marios: thanks | 13:57 |
| ysandeep | chandankumar, do you have few mins after this call. | 13:58 |
| chandankumar | ysandeep: yes sure | 13:58 |
| rlandy|ruck | bhagyashris: so you want to cancel scrum tomorrow | 13:58 |
| rlandy|ruck | we'll just go for the planning meeting | 13:59 |
| *** amoralej|lunch is now known as amoralej | 13:59 | |
| rlandy|ruck | ConflictException: 409: Client Error for url: https://network.rdo.mtl2.vexxhost.net/v2.0/ports/c9441337-df7c-41fd-ac09-536f550d9d36, Port c9441337-df7c-41fd-ac09-536f550d9d36 cannot be deleted directly via the port API: has device owner network:ha_router_replicated_interface. | 14:02 |
| rlandy|ruck | akahat|ruck: ^^ | 14:02 |
| rlandy|ruck | OVB errors | 14:02 |
| rlandy|ruck | frenzy_friday: ok- I have a few minutes if you want to meet now | 14:02 |
| rlandy|ruck | ysandeep: will look into the registry push problem on rhos-17 rhel-9 in afternoon | 14:03 |
| rlandy|ruck | will work on it with jason | 14:03 |
| frenzy_friday | rlandy|ruck, yep https://meet.google.com/fyu-bsuj-jsd | 14:03 |
| rlandy|ruck | frenzy_friday: sure - will join in 5 | 14:04 |
| frenzy_friday | thanks | 14:05 |
| chandankumar | rlandy|ruck: qucik merge https://review.rdoproject.org/r/c/rdo-jobs/+/37759 | 14:23 |
| rlandy|ruck | frenzy_friday: I have an email trace with similar error: | 14:27 |
| rlandy|ruck | ahh, I suspect maybe venv not installed on rhel9 node.. | 14:27 |
| rlandy|ruck | View Change | 14:27 |
| rlandy|ruck | comment from ysandeep | 14:27 |
| rlandy|ruck | chandankumar and ysandeep fixed it for c9 | 14:27 |
| rlandy|ruck | https://review.rdoproject.org/r/c/config/+/36113 | 14:28 |
| rlandy|ruck | frenzy_friday: ^^ | 14:28 |
| rlandy|ruck | frenzy_friday: try remove it | 14:29 |
| rlandy|ruck | it's config | 14:29 |
| rlandy|ruck | so you'll have to merge | 14:29 |
| rlandy|ruck | chandankumar: you're not creating a new file for https://review.rdoproject.org/r/c/rdo-jobs/+/37759? | 14:30 |
| rlandy|ruck | wallaby | 14:30 |
| rlandy|ruck | https://review.rdoproject.org/r/c/rdo-jobs/+/37759/1/zuul.d/integration-pipeline-wallaby.yaml | 14:31 |
| rlandy|ruck | ^^ temp change? | 14:31 |
| chandankumar | rlandy|ruck: no I am going to keep it in the same file | 14:31 |
| chandankumar | rlandy|ruck: if you want , I can create a seperate pipeline | 14:31 |
| rlandy|ruck | chandankumar: what's the master situation? | 14:31 |
| chandankumar | rlandy|ruck: it has seperate pipeline | 14:32 |
| chandankumar | for integration | 14:32 |
| chandankumar | let me follow the same for wallaby | 14:32 |
| rlandy|ruck | k | 14:32 |
| rlandy|ruck | chandankumar: we should consider wallaby as if two lines are staying | 14:32 |
| rlandy|ruck | master is a throw away | 14:32 |
| bhagyashris | rlandy|ruck, yup will do that | 14:32 |
| rlandy|ruck | bhagyashris: thank you | 14:33 |
| chandankumar | rcastillo: can you create a new pipeline openstack-periodic-integration-stable1-cs9 for running wallaby jobs? hint: https://review.rdoproject.org/codesearch/?q=openstack-periodic-integration-main-cs9&i=nope&files=&repos= | 14:35 |
| chandankumar | rcastillo: feel free to bug dviroel|rover | 14:35 |
| chandankumar | for the same :-) | 14:36 |
| chandankumar | rcastillo: left notes here https://review.rdoproject.org/r/c/testproject/+/37713/5#message-074dcd4b3e7ace8b4ddccf1b1878d8cfc26672b5 m | 14:41 |
| chandankumar | thank you for picking this work :-) | 14:41 |
| rcastillo | chandankumar: ack. thanks for the help so far! | 14:43 |
| dviroel|rover | akahat|ruck: rlandy|ruck - https://bugs.launchpad.net/tripleo/+bug/1956441 is hitting lot of ovb jobs now | 14:46 |
| rlandy|ruck | dviroel|rover: akahat|ruck: pls check if there are old stacks | 14:49 |
| rlandy|ruck | and if the clean up scripts are working | 14:49 |
| rlandy|ruck | akahat|ruck: dviroel|rover: also get the ticket reported to vexxhost if any | 14:49 |
| rlandy|ruck | and add to bug | 14:49 |
| ysandeep | rlandy|ruck, we got promotion for 17 :) | 14:51 |
| rlandy|ruck | ysandeep: yay! yay! | 14:51 |
| rcastillo | chandankumar: the genisoimage backport conflicts with https://review.opendev.org/c/openstack/ironic-python-agent-builder/+/823431 | 14:51 |
| rcastillo | need both for the ipa image, not sure what way to go here | 14:51 |
| chandankumar | rcastillo: please rebase on top of that | 14:52 |
| chandankumar | using gerrit web ui | 14:53 |
| rcastillo | chandankumar: won't let me because of conflicts | 14:57 |
| chandankumar | rcastillo: you can rebase manually locally and add a note in the commit msg, it is not a clean cherry pick due to rebase on this | 14:57 |
| chandankumar | this way you can test both the changes | 14:57 |
| rcastillo | oh sure, I'll give that a shot | 14:59 |
| chandankumar | see ya | 15:03 |
| frenzy_friday | rlandy|ruck, removed fetch zuul cloner https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-config/+/302065 | 15:07 |
| rlandy|ruck | frenzy_friday: this will be a merge and pray thing ... ysandeep: ^^ you ok with merging this and seeing the impact? | 15:14 |
| ysandeep | rlandy|ruck, checking | 15:15 |
| ysandeep | i remember posting similiar patch | 15:17 |
| rlandy|ruck | ysandeep: yep in rdo | 15:17 |
| rlandy|ruck | this is internal | 15:17 |
| ysandeep | rlandy|ruck, +2 looks good | 15:17 |
| rlandy|ruck | ysandeep: frenzy_friday: ok - I w+'ed | 15:18 |
| rlandy|ruck | let's hope there is no fallout | 15:18 |
| rlandy|ruck | frenzy_friday: ^^ once that merges, you can try your testproject again | 15:25 |
| ysandeep | chandankumar, rlandy|ruck https://bugzilla.redhat.com/show_bug.cgi?id=2037413 reported and cixed | 15:35 |
| ysandeep | rlandy|ruck: if you find anything about registry issue please drop me a comment in tripleo-ci chat channel, leaving for the day, see you tomorrow o/ | 15:40 |
| rlandy|ruck | ysandeep: sure - sorry - not much time to tackle this in my morning | 15:40 |
| rlandy|ruck | will leave you notes | 15:40 |
| rlandy|ruck | also we should be moving to quay.io | 15:40 |
| rlandy|ruck | after chatting with alan | 15:41 |
| ysandeep | for downstream also? | 15:41 |
| ysandeep | rlandy|ruck, ^^ | 15:41 |
| rlandy|ruck | ysandeep: ack- as private repos | 15:41 |
| rlandy|ruck | but not today :) | 15:42 |
| ysandeep | rlandy|ruck: thanks, I wasn't aware of that development | 15:42 |
| *** ysandeep is now known as ysandeep|out | 15:42 | |
| frenzy_friday | rlandy|ruck, rechecked testproject. Hope it moves further now | 15:43 |
| akahat|ruck | rlandy|ruck, https://logserver.rdoproject.org/50/37350/4/check/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-baremetal-wallaby/b6aee90/logs/overcloud-novacompute-0/var/log/extra/services.txt.gz | 15:45 |
| akahat|ruck | search systemd-modules-load.service | 15:45 |
| akahat|ruck | lookslike we are missing device ipmi_si | 15:45 |
| akahat|ruck | dviroel|rover, ^ | 15:46 |
| rlandy|ruck | akahat|ruck: hardware prov | 15:46 |
| dviroel|rover | akahat|ruck: tks, will add to the bug | 15:56 |
| rlandy|ruck | dviroel|rover: have another meeting - then can try help with ^^ | 16:00 |
| rlandy|ruck | let me know how things go | 16:01 |
| dviroel|rover | rlandy|ruck: will be on lunch in 5 min also, be back soon | 16:02 |
| rlandy|ruck | dviroel|rover: k - no rush | 16:02 |
| rlandy|ruck | only one issue atm | 16:03 |
| *** dviroel|rover is now known as dviroel|rover|lunch | 16:03 | |
| rcastillo | c9 wallaby images built just now: https://images.rdoproject.org/centos9/wallaby/rdo_trunk/611d2fc5edb4512559e0750a7fce79b7/ | 16:05 |
| rlandy|ruck | bhagyashris: hello - pls check that there is parity between c8 and c9 in the master component lines | 16:19 |
| rlandy|ruck | minus OVB | 16:19 |
| rlandy|ruck | looks like multinode-ipa is missing | 16:19 |
| bhagyashris | rlandy|ruck, i will check | 16:20 |
| rlandy|ruck | bhagyashris: thank you - pls out in a pacth to add what is missing | 16:20 |
| bhagyashris | rlandy|ruck, i have added multinode-ipa in this patch https://review.rdoproject.org/r/c/rdo-jobs/+/36987 | 16:22 |
| rlandy|ruck | frenzy_friday: https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/301020/1/zuul.d/standalone-jobs.yaml | 16:29 |
| frenzy_friday | rlandy|ruck, https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/302085 - added ntp ip in job def, also in testproj | 16:37 |
| rlandy|ruck | frenzy_friday: great - will merge if that passes | 16:40 |
| *** dviroel|rover|lunch is now known as dviroel|rover\ | 16:47 | |
| dasm | qq. i just sent a patch to tripleo-docs. it worked with http auth. it doesn't work with ssh auth no matter what i'm doing with my ssh keys. | 16:48 |
| dasm | i'm connected through a vpn. is anyone having similar problems? | 16:48 |
| dviroel|rover\ | rlandy|ruck: will be on lunch in 5 min also, be back soo | 16:49 |
| dasm | the message i'm getting is "permission deined (publickey)" | 16:49 |
| rlandy|ruck | dviroel|rover\: sure | 16:49 |
| dviroel|rover\ | rlandy|ruck: gah, sent by mistake | 16:49 |
| dviroel|rover\ | rlandy|ruck: i'm back already :) | 16:49 |
| *** dviroel|rover\ is now known as dviroel|rover | 16:49 | |
| rlandy|ruck | dasm: you should not need vpn to send to triple-docs | 16:49 |
| rlandy|ruck | public repo | 16:49 |
| dasm | i know. just asking if there is anything preventing me from using ssh over vpn. | 16:50 |
| rlandy|ruck | can you point to the patch | 16:50 |
| rlandy|ruck | are you using gerrit? | 16:50 |
| dasm | https://review.opendev.org/c/openstack/tripleo-docs/+/823562 | 16:50 |
| dasm | yes. just disconnected from the vpn, and it still says my ssh keys are invalid. | 16:51 |
| dasm | interesting. | 16:51 |
| dasm | hmm... | 16:51 |
| rcastillo | dasm are you using a default rsa key? I had to generate and add an ed25519 key to get ssh working in gerrit | 16:51 |
| rlandy|ruck | rcastillo: ah - you are correct | 16:52 |
| dasm | i did ssh-keygen | 16:52 |
| rlandy|ruck | there was a posting about that | 16:52 |
| rlandy|ruck | dasm: getting | 16:52 |
| dviroel|rover | dasm: try that | 16:53 |
| dviroel|rover | [dviroel@fedora ~]$ cat .ssh/config | 16:53 |
| dviroel|rover | Host * | 16:53 |
| dviroel|rover | PubkeyAcceptedKeyTypes +rsa-sha2-256,rsa-sha2-512 | 16:53 |
| rlandy|ruck | ssh authentication error with Gerrit | 16:53 |
| rlandy|ruck | dasm: ^^ | 16:53 |
| rlandy|ruck | search openstack-discuss | 16:54 |
| rlandy|ruck | for thread with that title | 16:54 |
| rlandy|ruck | will explain | 16:54 |
| dasm | dviroel|rover: ack, thx. | 16:54 |
| dasm | rlandy|ruck: yeah, ed25519 worked for me. | 16:54 |
| rlandy|ruck | "If that doesn't do the trick, you can add a host entry for | 16:54 |
| rlandy|ruck | review.opendev.org in your ~/.ssh/config file to set | 16:54 |
| rlandy|ruck | PubkeyAcceptedKeyTypes +rsa-sha2-256,rsa-sha2-512 so that it will | 16:54 |
| rlandy|ruck | look for them." | 16:54 |
| rlandy|ruck | ok | 16:54 |
| dasm | thanks, dviroel|rover++ rlandy|ruck++ | 16:55 |
| dasm | i'd never thought about that issue. | 16:55 |
| rlandy|ruck | dasm: neither did I - got if after reading posting | 16:55 |
| dasm | at least it's working now. no more copy/paste for password auth ;) | 16:56 |
| rlandy|ruck | cool | 16:56 |
| rlandy|ruck | rcastillo: you doing ok with image builds? | 16:58 |
| rlandy|ruck | ping if you need help | 16:58 |
| rcastillo | rlandy|ruck: got 3/3 images building now \o/ | 17:02 |
| rlandy|ruck | rcastillo++ nice | 17:05 |
| rlandy|ruck | rcastillo: dasm: ^^ from planning meeting tomorrow, we'll pull both of you for the wallaby c9 work | 17:05 |
| rlandy|ruck | dviroel|rover: ok - my lunch turn now - biab | 17:09 |
| *** rlandy|ruck is now known as rlandy|ruck|lunch | 17:09 | |
| *** rlandy|ruck|lunch is now known as rlandy|ruck | 17:35 | |
| *** akahat|ruck is now known as akahat|out | 17:35 | |
| rlandy|ruck | 2022-01-05 16:12:50 | 2022-01-05 16:12:46.447711 | fa163e13-36cf-80b0-14e3-00000000171a | FATAL | Modules reload | overcloud-controller-1 | error={"changed": false, "msg": "Unable to start service systemd-modules-load.service: Job for systemd-modules-load.service failed because the control process exited with error code.\nSee \"systemctl status systemd-modules-load.service\" and \"journalctl -xe\" for details.\n"} | 17:47 |
| rlandy|ruck | we git the same in check | 17:48 |
| dviroel|rover | humm, I didn't found any ohter error message other then Amol found earlier | 17:49 |
| rlandy|ruck | dviroel|rover: asking on #tripleo | 17:52 |
| rlandy|ruck | I'm a little out of ideas here | 17:52 |
| dviroel|rover | looking | 17:52 |
| rlandy|ruck | hoping the wiz-people there might have some insight | 17:52 |
| *** amoralej is now known as amoralej|off | 18:03 | |
| rlandy|ruck | dviroel|rover: so really I have no idea | 18:19 |
| rlandy|ruck | took some notes in https://bugs.launchpad.net/tripleo/+bug/1956441 | 18:19 |
| rlandy|ruck | will leave a message for steve | 18:19 |
| dviroel|rover | yeah, this one seems tricky | 18:20 |
| rlandy|ruck | left notes for steve in hardware prov chat | 18:22 |
| rlandy|ruck | will try ping him later | 18:22 |
| rlandy|ruck | taking out all of OVB | 18:28 |
| rlandy|ruck | dviroel|rover: rcastillo: ^^ worth a shot at stream pinning? | 18:28 |
| rlandy|ruck | possible to test a previous compose? | 18:28 |
| rlandy|ruck | would have to rebuild images | 18:29 |
| rlandy|ruck | on the OVB job | 18:29 |
| rlandy|ruck | https://composes.centos.org/CentOS-Stream-8-20211202.n.1/compose/BaseOS/x86_64/os/Packages/ has earlier kernel | 18:29 |
| dviroel|rover | rlandy|ruck: rcastillo: we didn't test pinning with centos-8 yet, neither ovbs? | 18:33 |
| dviroel|rover | but worth trying | 18:33 |
| rlandy|ruck | dviroel|rover: just a thought | 18:33 |
| rlandy|ruck | steve will be around shortly | 18:33 |
| dviroel|rover | ok | 18:34 |
| dviroel|rover | rlandy|ruck: looking previous successfull run | 18:37 |
| rcastillo | rlandy|ruck: dviroel|rover: not sure how it'd go, we don't have an older centos8 image | 18:37 |
| dviroel|rover | rlandy|ruck: https://logserver.rdoproject.org/78/823478/1/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/975f7f9/logs/overcloud-novacompute-0/var/log/extra/lsmod.txt.gz - no ipmi modules | 18:37 |
| dviroel|rover | rlandy|ruck: https://logserver.rdoproject.org/78/823478/1/openstack-check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/975f7f9/logs/overcloud-novacompute-0/var/log/extra/services.txt.gz | 18:38 |
| dviroel|rover | Jan 05 08:15:48 overcloud-novacompute-0 systemd[1]: Starting Load Kernel Modules... | 18:39 |
| dviroel|rover | Jan 05 08:15:48 overcloud-novacompute-0 systemd-modules-load[10268]: Inserted module 'br_netfilter' | 18:39 |
| dviroel|rover | Jan 05 08:15:48 overcloud-novacompute-0 systemd-modules-load[10268]: Module 'msr' is builtin | 18:39 |
| dviroel|rover | Jan 05 08:15:48 overcloud-novacompute-0 systemd[1]: Started Load Kernel Modules. | 18:39 |
| dviroel|rover | no ipmi_si module loaded | 18:39 |
| rlandy|ruck | ok | 18:58 |
| rlandy|ruck | was worth a shot | 18:58 |
| *** dviroel|rover is now known as dviroel|rover|afk | 19:00 | |
| rcastillo | have to go afk for a bit. Back in ~1hr | 19:11 |
| rlandy|ruck | ok | 19:13 |
| rlandy|ruck | ysandeep|out: chandankumar: so - reason push containers is failing with rhel-9 is that the job has push_containers: false - so you are not doing the required registry login | 20:05 |
| rlandy|ruck | compare: https://sf.hosted.upshift.rdu2.redhat.com/logs/90/299390/24/check/periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-17/a17e77e/zuul-info/inventory.yaml | 20:05 |
| rlandy|ruck | and https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-build-containers-ubi-8-internal-rhel-8-build-push-upload-rhos-17/e083cbe/zuul-info/inventory.yaml | 20:05 |
| rlandy|ruck | https://code.engineering.redhat.com/gerrit/c/openstack/tripleo-ci-internal-jobs/+/299521/28/zuul.d/tripleo-build-containers.yaml#152 | 20:07 |
| rlandy|ruck | which is weird | 20:07 |
| rlandy|ruck | as it is set there | 20:07 |
| rlandy|ruck | ah - was set to false in the testproject | 20:17 |
| rlandy|ruck | removed that | 20:17 |
| rlandy|ruck | hmmm ... ok - still unauthorized | 20:43 |
| rlandy|ruck | ysandeep|out: chandankumar: ok - worked with jason to add the service account | 22:52 |
| rlandy|ruck | rerunning testproject | 22:52 |
| dasm | \o | 23:17 |
| rlandy|ruck | ysandeep|out: chandankumar: https://sf.hosted.upshift.rdu2.redhat.com/logs/90/299390/25/check/periodic-tripleo-build-containers-ubi-9-internal-rhel-9-build-push-upload-rhos-17/2891af7/logs/containers-built.log | 23:42 |
| rlandy|ruck | images are pushing | 23:42 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!