| *** pojadhav|out is now known as pojadhav|rover | 04:14 | |
| *** ysandeep|out is now known as ysandeep | 05:28 | |
| *** marios is now known as marios|ruck | 06:23 | |
| *** ysandeep is now known as ysandeep|lunch | 07:39 | |
| frenzy_friday | Hey pojadhav|rover, marios|ruck is this a known error: "AttributeError: 'NoneType' object has no attribute 'version_tuple'", "stdout_lines": ["AttributeError: 'NoneType' object has no attribute 'version_tuple'" ? in TASK [python_venv_build : Create the virtualenv (if it does not exist)] | 07:39 |
|---|---|---|
| marios|ruck | frenzy_friday: yeah sec | 07:39 |
| marios|ruck | frenzy_friday: add this in depends-on https://review.opendev.org/c/openstack/tripleo-ci/+/820172/ | 07:40 |
| marios|ruck | frenzy_friday: https://bugs.launchpad.net/tripleo/+bug/1953032 | 07:40 |
| frenzy_friday | marios|ruck, thanks | 07:40 |
| jbadiapa | good morning | 08:13 |
| marios|ruck | o/ | 08:21 |
| marios|ruck | ysandeep|lunch: sshnaidm|afk: * folks can you please vote on https://review.opendev.org/c/openstack/tripleo-ci/+/820172/9#message-95acaed557bb46c7e21b9570c2c77798deca54a9 and we can get it merged today | 08:59 |
| *** pojadhav|rover is now known as pojadhav|rover|afk | 09:14 | |
| *** ysandeep|lunch is now known as ysandeep | 09:19 | |
| frenzy_friday | I need some help with https://review.rdoproject.org/r/c/testproject/+/37023 - I am getting Error running container image prepare: Not found image: https://trunk.registry.rdoproject.org/v2/tripleomastercentos9/openstack-glance-api/manifests/9f93da59213dc8e9f46a7a1e36936945. Do I need to run periodic-tripleo-ci-build-containers-centos-9-push-master first ? | 09:20 |
| sshnaidm|afk | marios|ruck, was it tested somewhere with 7? | 09:20 |
| *** sshnaidm|afk is now known as sshnaidm | 09:20 | |
| sshnaidm | marios|ruck, oh, I see in the patch, nice | 09:20 |
| ysandeep | marios|ruck: revoted, already voted previously on patchset6 | 09:20 |
| marios|ruck | sshnaidm: thanks if you want we can also add the 7 periodics in testproject after the current run finishes | 09:21 |
| marios|ruck | thanks ysandeep | 09:21 |
| marios|ruck | sshnaidm: but yeah it has the 7-train jobs in upstream report | 09:21 |
| sshnaidm | yeah, it's enough I think | 09:22 |
| *** pojadhav|rover|afk is now known as pojadhav|rover | 09:51 | |
| Tengu | hello there! Are there some issues with the container registry on rdoproject? A job is failing while trying to fetch container images - and apparently, not always the same image: https://review.rdoproject.org/r/c/testproject/+/36998 | 10:01 |
| Tengu | frenzy_friday: ah. well. you're hitting the same kind of issues I'm seeing -^^ | 10:02 |
| frenzy_friday | Hey Tengu, yes, I am getting Error running container image prepare: Not found image: https://trunk.registry.rdoproject.org/v2/tripleomastercentos9/openstack-glance-api/manifests/9f93da59213dc8e9f46a7a1e36936945. Do I need to run periodic-tripleo-ci-build-containers-centos-9-push-master first? | 10:04 |
| frenzy_friday | example, in https://review.rdoproject.org/r/c/testproject/+/37023 | 10:04 |
| * frenzy_friday lunch, back in ~1hr | 10:04 | |
| Tengu | frenzy_friday: not sure. seems the image isn't always the same. | 10:05 |
| Tengu | I hit against keystone, then rsyslog | 10:05 |
| Tengu | can't say if the periodic-tripleo-ci-build-containers-centos-9-push-master is needed. Maybe? | 10:05 |
| marios|ruck | Tengu: not sure what's up yet but the line really isn't expected to be stable at this point | 10:12 |
| marios|ruck | Tengu: for now you can try overriding | 10:12 |
| marios|ruck | Tengu: indeed i don't see /9f93da59213dc8e9f46a7a1e36936945 @ https://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomastercentos9/imagestreamtags/ | 10:12 |
| Tengu | marios|ruck: oh, well, it's a test job for another change | 10:12 |
| marios|ruck | Tengu: so you can try and force one of the hashes there like d416170694b13db3586554ba18096332 | 10:13 |
| Tengu | goal is to get https://review.opendev.org/c/openstack/tripleo-ci/+/819845 in - Chandan wanted to get a testjob for it to be 100% sure | 10:13 |
| Tengu | marios|ruck: how? | 10:13 |
| marios|ruck | Tengu: right you can add this to your test job with featureset_override trying to find you example sec | 10:14 |
| marios|ruck | Tengu: like this https://docs.openstack.org/tripleo-docs/latest/ci/chasing_promotions.html#specifying-a-particular-hash | 10:15 |
| marios|ruck | Tengu: and just use dlrn_hash_tag: d416170694b13db3586554ba18096332 | 10:15 |
| Tengu | ah, good! | 10:15 |
| marios|ruck | Tengu: should work at least those containers are there in the registry | 10:15 |
| Tengu | lemme update Chandan job. | 10:15 |
| Tengu | marios|ruck: thanks :). Job updated, let's see. | 10:16 |
| marios|ruck | ack | 10:17 |
| * marios|ruck food biab | 10:33 | |
| jbadiapa | hi, Regarding this patch https://review.opendev.org/c/openstack/tripleo-ansible/+/811474, could we increase the timeout of the tripleo_collectd molecute jobs to 1h30m? | 10:44 |
| jbadiapa | It failed with a timeout, the timeout is 1h and the job is taking a 1h01m to complete | 10:45 |
| jbadiapa | is there any reason to avoid the increase of the timeout? | 10:46 |
| jbadiapa | the jobs https://zuul.opendev.org/t/openstack/job/tripleo-ansible-centos-8-molecule-tripleo_collectd > https://zuul.opendev.org/t/openstack/job/tripleo-ansible-centos-8-base (Timeout: 3600 seconds) | 10:49 |
| *** ysandeep is now known as ysandeep|afk | 11:00 | |
| Tengu | jbadiapa: why is it taking so long? | 11:02 |
| jbadiapa | it's doing several tests at once | 11:02 |
| Tengu | maybe better to see if things could be split? | 11:03 |
| jbadiapa | I talked to chriss and suggested those two options. (splitting the molecule test or increasing the timeout) | 11:05 |
| Tengu | heh, of course, increasing timeout will be prefered - less work ;) | 11:05 |
| *** rlandy|out is now known as rlandy|ruck | 11:08 | |
| jbadiapa | I think so | 11:09 |
| *** dviroel|out is now known as dviroel | 11:11 | |
| rlandy|ruck | marios|ruck: pojadhav|rover: how are things in ruck/rover land? | 11:20 |
| pojadhav|rover | rlandy|ruck, just posted component line status in chat.. checking blockers. | 11:20 |
| rlandy|ruck | pojadhav|rover: k - let's sync when marios|ruck is back | 11:22 |
| pojadhav|rover | yep | 11:22 |
| rlandy|ruck | akahat: pojadhav|rover: also wanted to let you know what's happening with scenraio010 | 11:22 |
| pojadhav|rover | ack | 11:22 |
| marios|ruck | rlandy|ruck: o/ same as yesterday nothing new afaik ... i am digging into containers/images periodics mirror issue can fill you in later | 11:23 |
| marios|ruck | rlandy|ruck: didn't get to the upgrade job yet because of ^^^ i think it has priority | 11:23 |
| marios|ruck | rlandy|ruck: we went ahead with non voting for now anyway to unblock ^ | 11:23 |
| rlandy|ruck | marios|ruck: ack - have not merged the integration criteria yet | 11:24 |
| rlandy|ruck | marios|ruck: ack - you carry on ... you ok if I ping upgrades to see if they want to address this with the downgrade? | 11:26 |
| marios|ruck | rlandy|ruck: sure np with me ;) | 11:26 |
| rlandy|ruck | jfrancoa: hello ... we have a ussuri/victoria promotion blocker ... https://bugs.launchpad.net/tripleo/+bug/1953326 | 11:27 |
| rlandy|ruck | jfrancoa: ^^ can you review the comments on the bug above and advise regarding the suggested downgrade | 11:27 |
| rlandy|ruck | these jobs are also blocking gate - so we are moving them to non-voting | 11:28 |
| rlandy|ruck | marios|ruck: pojadhav|rover: btw: pinged infra re: stuck dlrn | 11:29 |
| rlandy|ruck | no victoria or wallaby builds in 4 days | 11:29 |
| marios|ruck | thanks rlandy|ruck | 11:29 |
| pojadhav|rover | rlandy|ruck, ack | 11:31 |
| dviroel | marios|ruck: morning o/ - do you remember any related issue while building containers? | 11:32 |
| dviroel | marios|ruck: https://logserver.rdoproject.org/76/36976/4/check/periodic-tripleo-ci-centos-9-standalone-master/c902241/logs/undercloud/home/zuul/container-builds/ad9f81ab-a385-4d85-b330-bf34311c8063/base/base-build.log.txt.gz | 11:32 |
| dviroel | marios|ruck: this is showing up when running standalone-master + build_container_images: true | 11:33 |
| marios|ruck | dviroel: looking | 11:33 |
| marios|ruck | dviroel: not seen that ... not immediately clear what the error is here but no haven't seen something like that | 11:34 |
| dviroel | marios|ruck: ok, np - need to see the difference between build-containers and container-build roles | 11:36 |
| dviroel | see if I can spot something | 11:36 |
| Tengu | uho.... https://logserver.rdoproject.org/98/36998/2/check/periodic-tripleo-ci-centos-9-undercloud-containers-master/c2e410a/logs/undercloud/home/zuul/install_packages.sh.log.txt.gz | 11:42 |
| Tengu | seems cs9 line isn't in shape. | 11:43 |
| Tengu | have to go, dog walk, but... meh. | 11:43 |
| ysandeep | hey folks o/ anyone else facing issue with internal vpn? | 11:43 |
| Tengu | ysandeep: nope (using Amsterdam) | 11:44 |
| Tengu | back in a bit. | 11:44 |
| rlandy|ruck | ysandeep: ok by me | 11:44 |
| rlandy|ruck | using Raleigh | 11:44 |
| ysandeep | Tengu, rlandy|ruck thanks! | 11:44 |
| * ysandeep will try other vpns | 11:44 | |
| *** ysandeep|afk is now known as ysandeep | 11:46 | |
| akahat | ack | 11:46 |
| ysandeep | Tengu, thanks Amsterdam worked for me | 11:47 |
| owalsh | marios|ruck: hey, if you have a minute just want to check I'm doing the right thing... | 11:48 |
| marios|ruck | o/ owalsh | 11:48 |
| owalsh | wanted to get CI to cover https://review.opendev.org/c/openstack/tripleo-common/+/820408 so added depends-on for https://review.opendev.org/c/openstack/tripleo-ci/+/820511 | 11:49 |
| * akahat will be back in hour | 11:50 | |
| owalsh | but doesn't seem to have worked. Is that the right ci change? And if yes should I just wait for it to merge (no Depends-On) | 11:51 |
| rlandy|ruck | akahat: will meet with pojadhav|rover - we can talk when you get back | 11:51 |
| marios|ruck | owalsh: looking | 11:52 |
| marios|ruck | owalsh: thinking (yeah i see it builds it at https://d7765034c2c5769142ff-d03ef6adfb032b4f673cb06ee5d0ab26.ssl.cf1.rackcdn.com/820511/1/check/tripleo-ci-centos-8-content-provider/9088da2/logs/report.html the ci one but skips it at tripleo-common https://37df2e11d5754f3dd19d-647e079c0cf961ec5006c71d86c2953a.ssl.cf5.rackcdn.com/820408/2/check/tripleo-ci-centos-8-content-provider/9ff9bdc/logs/report.html ) | 11:53 |
| marios|ruck | owalsh: not sure about the details of our build-test-packages and building tripleo-ci i.e. if we can make that depends-on ci repo work in this way... so instead of digging there, you can just try run what you want with rdo testproject | 11:55 |
| marios|ruck | owalsh: are you familiar with that? | 11:55 |
| rlandy|ruck | ysandeep: fyi - dlrn is stuck on wallaby - so current on rhos-17 is old | 11:55 |
| marios|ruck | owalsh: you can have both the tripleo-common and tripleo-ci change in depends-on ... but gimme sec let me dig a bit more at built-test-packages first | 11:55 |
| owalsh | marios|ruck: ack, thanks! | 11:56 |
| ysandeep | rlandy|ruck, ack thanks for info | 11:56 |
| marios|ruck | owalsh: hmmm it isn't being included... i wonder if it doesn't like the changeid but that should work fine | 11:56 |
| marios|ruck | owalsh: i mean in https://37df2e11d5754f3dd19d-647e079c0cf961ec5006c71d86c2953a.ssl.cf5.rackcdn.com/820408/2/check/tripleo-ci-centos-8-content-provider/9ff9bdc/job-output.txt the 2021-12-06 11:05:43.650564 | primary | "artg_change_list": [ only has 2021-12-06 11:05:43.650646 | primary | "refspec": "refs/changes/08/820408/2" | 11:57 |
| marios|ruck | owalsh: normally it would also contain the tripleo-ci change there | 11:57 |
| marios|ruck | owalsh: can you quickly try with 'normal' url in depends-on? | 11:58 |
| owalsh | marios|ruck: just noticed there are 2 spaces between "Depends-On:" and the change id, wonder if that's the issue | 11:58 |
| marios|ruck | owalsh: maybe but also make it the 'normal' url not change id | 11:58 |
| owalsh | marios|ruck: ack. thanks! | 11:59 |
| marios|ruck | owalsh: ack lets see hope it helps... | 12:00 |
| marios|ruck | owalsh: you can check if it is included by grep artg_change_list as above in the job-output.txt ^^ | 12:00 |
| owalsh | marios|ruck: ack, thanks as always | 12:13 |
| rlandy|ruck | pojadhav|rover: let's sync - https://meet.google.com/hje-vcyh-rru?pli=1&authuser=0 | 12:14 |
| rlandy|ruck | dviroel: hey ... re: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37026 - not sure of arxcruz answered your questions | 12:15 |
| rlandy|ruck | but we should get this patch in | 12:15 |
| rlandy|ruck | dviroel: ^^ can you take it over? | 12:15 |
| dviroel | rlandy|ruck: he didn't, not sure if we need to add DISK_FULL there, we can get this change in, and provide another one if needed? | 12:20 |
| rlandy|ruck | pojadhav|rover: https://trunk.rdoproject.org/centos8-wallaby/report.html | 12:26 |
| rlandy|ruck | marios|ruck: pojadhav|rover will report in the program call tomorrow | 12:28 |
| bhagyashris_ | Tengu, frenzy_friday looks like the image/container with the sha 9f93da59213dc8e9f46a7a1e36936945 is getting archived from our source registry https://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomastercentos9/imagestreamtags | 12:29 |
| bhagyashris_ | may be we will need fresh promotion for cs9 master | 12:30 |
| rlandy|ruck | ysandeep: steve put up patches for uefi on wallaby | 12:31 |
| rlandy|ruck | ysandeep: they are on the review hackmd | 12:31 |
| *** bhagyashris_ is now known as bhagyashris | 12:31 | |
| ysandeep | rlandy|ruck: thanks, I saw your conversation on google channel with him, those patches are on my review list - will check them in next review round | 12:32 |
| rlandy|ruck | ysandeep: thanks - no immediate ruch | 12:33 |
| rlandy|ruck | rush | 12:33 |
| rlandy|ruck | dviroel: wrt DISK_FULL jfrancoa was debugging some tests with that issue | 12:33 |
| rlandy|ruck | it is a failing status | 12:33 |
| rlandy|ruck | but not a job failing status | 12:33 |
| rlandy|ruck | so good to track | 12:33 |
| rlandy|ruck | imho :) | 12:33 |
| rlandy|ruck | rcastillo: hey ... do you have the invite to the OSP all hands? | 12:34 |
| dviroel | rlandy|ruck: i see, so we should be good to merge arxcruz change | 12:38 |
| bhagyashris | dviroel, yes i was also looking at the same code but it's bit confusing i will check later as soon as setup will go up at my side | 12:39 |
| rlandy|ruck | dviroel: ysandeep: ok - so since arxcruz is out, there will not screenshot coming :) | 12:40 |
| rlandy|ruck | dviroel: ysandeep: if we're ok with the logic of the changes in https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37026 | 12:40 |
| rlandy|ruck | pls revote and we can merge and try it | 12:40 |
| bhagyashris | rlandy|ruck, ysandeep jfyi i saw the results yesterday and that's looks good to mee | 12:40 |
| rlandy|ruck | bhagyashris: cool - pls vote there then | 12:41 |
| bhagyashris | rlandy|ruck, yup | 12:41 |
| ysandeep | rlandy|ruck, lets merge that, without screenshot we don't have an easy way to confirm if anything wrong there. | 12:41 |
| ysandeep | revoted +2 | 12:42 |
| dviroel | thanks bhagyashris | 12:42 |
| rlandy|ruck | ysandeep: dviroel: bhagyashris: ok - merging that .... bhagyashris pls keep an eye on the numbers | 12:42 |
| rlandy|ruck | if something looks wrong, pld submit a revert | 12:42 |
| rlandy|ruck | bhagyashris: ^^ | 12:42 |
| bhagyashris | rlandy|ruck, sure | 12:42 |
| rlandy|ruck | thank you! | 12:42 |
| bhagyashris | folks will need cs9 master promotion | 12:44 |
| *** pojadhav|rover is now known as pojadhav|rover|brb | 12:44 | |
| bhagyashris | because the current-tripleo hash https://trunk.rdoproject.org/centos9-master/current-tripleo/delorean.repo.md5 containers archived from our source registry (if you see here https://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomastercentos9/imagestreamtags) and thats causing the issue while running the jobs in the testproject | 12:46 |
| rlandy|ruck | bhagyashris: marios|ruck: do you want to submit a patch to remove criteria | 12:49 |
| rlandy|ruck | and then revert | 12:50 |
| rlandy|ruck | so you get a promotion? | 12:50 |
| marios|ruck | rlandy|ruck: for u/v/upgrade? | 12:50 |
| marios|ruck | rlandy|ruck: i thought we had one already | 12:50 |
| rlandy|ruck | marios|ruck: nah - responding to bhagyashris's c9 comment | 12:50 |
| marios|ruck | rlandy|ruck: or you mean somethign else ... ah master | 12:50 |
| marios|ruck | rlandy|ruck: so we want empty criteria you mean? | 12:50 |
| marios|ruck | pojadhav|rover|brb: can you do it... ah she is away | 12:50 |
| bhagyashris | marios|ruck, yeah for cs9 master | 12:50 |
| marios|ruck | rlandy|ruck: k yeah doing | 12:51 |
| rlandy|ruck | marios|ruck: talking c9 - ack thanks | 12:51 |
| marios|ruck | rlandy|ruck: basically we want it empty right? | 12:51 |
| rlandy|ruck | marios|ruck: don't worry about the rest of ruck/roving pojadhav|rover|brb and I are on that | 12:51 |
| rlandy|ruck | marios|ruck: yep | 12:51 |
| rlandy|ruck | and then revert when promoted | 12:51 |
| marios|ruck | rlandy|ruck: hmmm... we probably want the containers push job though :D | 12:55 |
| marios|ruck | rlandy|ruck: gonna update | 12:55 |
| marios|ruck | and images right | 12:56 |
| rlandy|ruck | whatever can run now | 12:56 |
| rlandy|ruck | and spring current-tripleo | 12:56 |
| marios|ruck | hmmm but images and containers are waiting for fixes still in review/gate | 12:57 |
| marios|ruck | so won't update | 12:57 |
| marios|ruck | rlandy|ruck: ^ | 12:57 |
| marios|ruck | rlandy|ruck: i mean the image/containers jobs are going to fail anyway so cant add hem | 12:57 |
| marios|ruck | them | 12:57 |
| rlandy|ruck | marios|ruck: bhagyashris: looks like we only have dviroel's topic at the community call ... let's chat bout c9 then | 12:58 |
| marios|ruck | k | 12:58 |
| rlandy|ruck | and what we should do there | 12:58 |
| bhagyashris | rlandy|ruck, ack | 12:59 |
| marios|ruck | rlandy|ruck: fwiw... https://review.rdoproject.org/r/c/testproject/+/36817/13#message-92e3f165909ebe0f5faeded43059d32b34796c69 | 12:59 |
| marios|ruck | rlandy|ruck: mirror issue workaround not the tidiest thing in the world but gets us there... https://review.opendev.org/c/openstack/tripleo-ci/+/820850 (plus we'll need https://review.rdoproject.org/r/c/rdo-jobs/+/37067 but we have workaround for now | 13:00 |
| marios|ruck | https://opendev.org/openstack/tripleo-quickstart/src/commit/7cade1571a78f4fa840c62fdecee620e1cf7392f/config/release/tripleo-ci/CentOS-9/promotion-testing-hash-master.yml#L54-L57 | 13:00 |
| rlandy|ruck | marios|ruck: ok - whatever it takes at this point :) | 13:01 |
| rlandy|ruck | pojadhav|rover|brb: fyi ... reruns for failing master, wallaby, train https://review.rdoproject.org/r/c/testproject/+/36254 | 13:12 |
| ysandeep | rlandy|ruck, openstack-ansible-os_tempest xena branch is failing on our tripleo jobs: https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/820821/ | 13:13 |
| ysandeep | rlandy|ruck, ^^ they should just drop tripleo jobs for xena from layout, right? | 13:14 |
| ysandeep | https://github.com/openstack/openstack-ansible-os_tempest/blob/stable/xena/zuul.d/project.yaml | 13:14 |
| *** pojadhav|rover|brb is now known as pojadhav|rover | 13:14 | |
| rlandy|ruck | ysandeep: ack - we don't have such a thing | 13:14 |
| marios|ruck | o/ tripleo-ci please add to reviews list https://review.opendev.org/c/openstack/tripleo-ci/+/820850 we need this for c9 containers/images | 13:14 |
| marios|ruck | rlandy|ruck: ^^ | 13:14 |
| rlandy|ruck | marios|ruck: looks ok - waiting on zuul results to +2 | 13:16 |
| pojadhav|rover | rlandy|ruck, ack | 13:17 |
| rlandy|ruck | ysandeep: marios|ruck: pojadhav|rover: just fyi ... wallaby and victoria dlrn are up to date now | 13:17 |
| pojadhav|rover | rlandy|ruck, great !! | 13:17 |
| rlandy|ruck | we should see new code coming up now | 13:17 |
| ysandeep | ++ | 13:19 |
| bhagyashris | arxcruz, sshnaidm, rlandy, rlandy|ruck, marios, ysandeep, bhagyashris, svyas, soniya29, pojadhav, akahat, chandankumar, frenzy_friday, anbanerj, dviroel, rcastillo, dasm | 13:21 |
| bhagyashris | Tripleo CI community call in 9 mins | 13:21 |
| bhagyashris | feel free to add the topics here https://hackmd.io/MMg4WDbYSqOQUhU2Kj8zNg?both#2021-12-07-Community-Call **@ line 30** | 13:22 |
| bhagyashris | hello folks | 13:30 |
| bhagyashris | community call time | 13:30 |
| bhagyashris | akahat, pojadhav|rover sshnaidm | 13:31 |
| bhagyashris | ^ | 13:31 |
| rlandy|ruck | https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37068 | 13:42 |
| rlandy|ruck | https://review.rdoproject.org/r/c/testproject/+/36817/13#message-92e3f165909ebe0f5faeded43059d32b34796c69 | 13:42 |
| rlandy|ruck | bhagyashris: ^^ pls watch if this hoses the promoter | 13:44 |
| rlandy|ruck | akahat: ^^ | 13:44 |
| marios|ruck | Depends-On: https://review.opendev.org/c/openstack/tripleo-ci/+/820172 | 13:47 |
| marios|ruck | Depends-On: https://review.opendev.org/c/openstack/tripleo-ci/+/820850 | 13:47 |
| Tengu | marios|ruck: where did you find that hash earlier? jobs end in POST_FAILURE issue, apparently due to a package conflict or something. | 13:52 |
| marios|ruck | Tengu: from the registry @ https://trunk.registry.rdoproject.org:8443/oapi/v1/namespaces/tripleomastercentos9/imagestreamtags/ | 14:00 |
| marios|ruck | Tengu: (in community call right now) | 14:01 |
| marios|ruck | Tengu: we are working on fixing the c9 promotion though should be sorted by later today anyway | 14:01 |
| Tengu | marios|ruck: ok! tried to understand how that JSON works, it's... not really convenient, to be honest ;). but lemme know if I can help on the promotion issue :). | 14:01 |
| marios|ruck | Tengu: click the 'raw data' (i guess you're using ff?) | 14:03 |
| marios|ruck | Tengu: at least it is text you can search it | 14:03 |
| Tengu | ah, I'll try that later then | 14:03 |
| * bhagyashris dinner brb | 14:05 | |
| akahat | rlandy|ruck, please +w https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37068 | 14:05 |
| rlandy|ruck | akahat: done | 14:07 |
| rlandy|ruck | jpodivin: matbu: looking at your criteria jobs | 14:07 |
| rlandy|ruck | those jobs are not running in lines yet | 14:08 |
| rlandy|ruck | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-containers-multinode-validation-master | 14:08 |
| rlandy|ruck | last run ^^ 11/09 | 14:08 |
| rlandy|ruck | checking line definitions | 14:08 |
| jpodivin | rlandy|ruck: is there something I should do to help them along? | 14:09 |
| rlandy|ruck | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/component-pipeline-master.yaml | 14:10 |
| rlandy|ruck | - periodic-tripleo-ci-centos-8-standalone-validation-master: | 14:11 |
| rlandy|ruck | dependencies: | 14:11 |
| rlandy|ruck | - periodic-tripleo-centos-8-master-component-validation-promote-consistent-to-component-ci-testing | 14:11 |
| rlandy|ruck | - periodic-tripleo-ci-centos-8-standalone-validation-master: | 14:11 |
| rlandy|ruck | dependencies: | 14:11 |
| rlandy|ruck | - periodic-tripleo-centos-8-master-component-validation-promote-consistent-to-component-ci-testing | 14:11 |
| rlandy|ruck | same job defined twice | 14:11 |
| rlandy|ruck | https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/component-pipeline-wallaby.yaml | 14:12 |
| rlandy|ruck | jpodivin: ^^ do you see those line definitions | 14:12 |
| rlandy|ruck | you need to add the jobs you want to kick in those lines | 14:12 |
| rlandy|ruck | per component | 14:13 |
| rlandy|ruck | your jobs are defined | 14:13 |
| rlandy|ruck | but not defined to kick | 14:13 |
| rlandy|ruck | matbu: jpodivin: ^^ | 14:13 |
| rlandy|ruck | so before you add jobs to criteria, they have to be added to the lines | 14:13 |
| rlandy|ruck | - periodic-tripleo-ci-centos-8-standalone-compute-wallaby-validation: | 14:14 |
| rlandy|ruck | dependencies: | 14:14 |
| rlandy|ruck | - periodic-tripleo-centos-8-wallaby-component-compute-promote-consistent-to-component-ci-testing | 14:14 |
| rlandy|ruck | - periodic-tripleo-centos-8-wallaby-component-compute-promote-consistent-to-component-ci-testing | 14:14 |
| rlandy|ruck | ^^ that one exists | 14:14 |
| rlandy|ruck | jpodivin: ^^ make sense? | 14:16 |
| jpodivin | rlandy|ruck: I understand that one is duplicated | 14:16 |
| rlandy|ruck | and the others are missing | 14:16 |
| jpodivin | rlandy|ruck: so, if I get this right, we should add all of the new jobs in the component-pipeline-* files. | 14:17 |
| jpodivin | is that correct? | 14:17 |
| rlandy|ruck | jpodivin: the compute one seems to be there | 14:18 |
| rlandy|ruck | jpodivin: but yeah - make sure the jobs are in the lines first | 14:18 |
| rlandy|ruck | and then add the criteria | 14:18 |
| rlandy|ruck | your jobs have not run in a month | 14:18 |
| jpodivin | that is embarrassing | 14:19 |
| rlandy|ruck | jpodivin: hardly - we all do that - which is why I know to check for it :) | 14:20 |
| jpodivin | rlandy|ruck: ok, I'll get to patching | 14:25 |
| jpodivin | and thanks for the assist rlandy|ruck++ | 14:25 |
| akahat | rlandy|ruck, marios|ruck o/ we need to revert this patch: https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37068 , we can't keep criteria empty. | 14:49 |
| akahat | this broke promoter: http://promoter.rdoproject.org/promoter_logs/centos9_master.log | 14:50 |
| *** dviroel is now known as dviroel|lunch | 14:50 | |
| rlandy|ruck | marios|ruck is putting in a patch to add just container builds | 14:51 |
| rlandy|ruck | we can merge that | 14:51 |
| akahat | rlandy|ruck, or we can put only one job which is passing: eriodic-tripleo-ci-build-containers-centos-9-push-master | 14:51 |
| akahat | okay cool | 14:52 |
| jpodivin | rlandy|ruck: the pipeline patches are up https://review.rdoproject.org/r/c/rdo-jobs/+/37071 I'm going to rebase the criteria patches on them, just to be sure. | 14:55 |
| jpodivin | well, add a depends on, to be more precise | 14:57 |
| rlandy|ruck | rcastillo: hello | 15:09 |
| rcastillo | rlandy|ruck: hi | 15:13 |
| rlandy|ruck | rcastillo: hey - just checking you have all the all hand meeting on your calendar | 15:14 |
| rlandy|ruck | rcastillo: also - pls see pvt | 15:14 |
| rlandy|ruck | dasm: ^^ sent you the link to the all hands | 15:15 |
| rlandy|ruck | in about 45 mins | 15:15 |
| rcastillo | rlandy|ruck: I do | 15:15 |
| rlandy|ruck | rcastillo: k - good | 15:15 |
| rlandy|ruck | now at least we have you on the right lists and calendars | 15:16 |
| dasm | rlandy|ruck: ack. i didn't get the link, but i see the meeting in my calendar. | 15:17 |
| rlandy|ruck | dasm: great - if you see that, you'll all set on the calendars | 15:19 |
| *** ysandeep is now known as ysandeep|dinner | 15:34 | |
| marios|ruck | rlandy|ruck: you joining el9? | 15:46 |
| rlandy|ruck | akahat: marios|ruck; w+'ed https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37074 | 15:50 |
| rlandy|ruck | akahat: ^^ pls check for this when merged in promoter | 15:50 |
| jbadiapa | is there a problem with zuul? | 16:07 |
| *** dviroel|lunch is now known as dviroel | 16:10 | |
| *** ysandeep|dinner is now known as ysandeep | 16:10 | |
| rlandy|ruck | jbadiapa: which zuul? | 16:14 |
| rlandy|ruck | we have three | 16:14 |
| jbadiapa | https://zuul.opendev.org | 16:15 |
| jbadiapa | rlandy|ruck | 16:15 |
| rlandy|ruck | https://zuul.opendev.org/t/openstack/status is spinning for me now | 16:16 |
| rlandy|ruck | Service Unavailable | 16:17 |
| rlandy|ruck | probably being restarted | 16:17 |
| rlandy|ruck | will check back in a few | 16:17 |
| rlandy|ruck | jbadiapa: https://zuul.opendev.org/t/openstack/status - back | 16:26 |
| jbadiapa | thx | 16:26 |
| dviroel | rlandy|ruck: http://promoter.rdoproject.org/promoter_logs/centos9_master.log | 16:27 |
| rlandy|ruck | dviroel: ack ... waiting for https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/37074 to be on promoter | 16:29 |
| rlandy|ruck | to add one job | 16:29 |
| rlandy|ruck | merged a few minutes ago | 16:29 |
| dviroel | oh, good, didn't see this one | 16:30 |
| ysandeep | rlandy|ruck, do you have few mins, want to showcase node_holder.py new improvements | 16:39 |
| rlandy|ruck | ysandeep: not atm - can you book some time in my calendar? | 16:41 |
| rlandy|ruck | I know it's late for you | 16:41 |
| rlandy|ruck | so tomorrow is ok by me | 16:41 |
| ysandeep | ack, thanks! | 16:42 |
| marios|ruck | rlandy|ruck: pojadhav|rover: going in a few need something before i do? | 16:45 |
| rlandy|ruck | marios|ruck: just checking if we have promoter config | 16:46 |
| rlandy|ruck | hmmm - not yet | 16:46 |
| rlandy|ruck | akahat: ^^ can you check that pull pls | 16:47 |
| bhagyashris | rlandy|ruck, i see on the promoter server new change is not pulled yet | 16:47 |
| rlandy|ruck | bhagyashris: ack - I think that pull is every 10 minutes? | 16:48 |
| bhagyashris | rlandy|ruck, probably | 16:48 |
| bhagyashris | let wait for few mins | 16:49 |
| bhagyashris | rlandy|ruck, now changes pulled on promoter server hope it will run successfully in the next run | 16:56 |
| rlandy|ruck | bhagyashris: thanks | 16:56 |
| ysandeep | rlandy|ruck, pojadhav|rover requested me to debug a ci failure in 16.2 https://sf.hosted.upshift.rdu2.redhat.com/logs/22/295522/1/check/tripleo-rhel-8-buildimage-overcloud-full-rhos-16.2/6d745ca/job-output.txt | 17:02 |
| ysandeep | rlandy|ruck, looks like its coming from because we are removing ansible 2.9 from release file: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/295717/2/config/release/rhos-17.yml#b113 | 17:03 |
| ysandeep | We need to move ci code to 2.11, I didn't get why we are removing this from release file? | 17:03 |
| ysandeep | afaik 8.4 is only be compatible with ansible2.9 | 17:03 |
| rlandy|ruck | ysandeep: pojadhav-: k then we still need it | 17:06 |
| *** marios|ruck is now known as marios|out | 17:06 | |
| rlandy|ruck | should have gotten an install but maybe not in time | 17:06 |
| rlandy|ruck | lunch - brb | 17:06 |
| akahat | rlandy|ruck, checking.. | 17:38 |
| rlandy|ruck | akahat: I think the promoter is busy promoting master c8 | 17:38 |
| akahat | rlandy|ruck, yup. we need to wait until it's done. | 17:39 |
| rlandy|ruck | new config is on there | 17:39 |
| rlandy|ruck | ack | 17:39 |
| ysandeep | rlandy|ruck, fyi.. while working on fs064, we noticed the default nic-config file we use in CI had duplicate default routes on both external and ctlplane network: https://review.opendev.org/c/openstack/tripleo-ansible/+/818529/5#message-a44afc3ef6f8dd3471dc96f51389c697513f9552 , I will soon update that once my this patch get approved. | 18:06 |
| rlandy|ruck | k | 18:07 |
| ysandeep | rlandy|ruck, voted on steve patches: looks good | 18:12 |
| ysandeep | I think we can +w https://review.opendev.org/c/openstack/tripleo-quickstart/+/820459 | 18:13 |
| ysandeep | any objections ^^? | 18:15 |
| * rlandy|ruck reviews the set | 18:16 | |
| ysandeep | tht ones are already +w by Alex, that's the only one on us | 18:17 |
| ysandeep | We need that as soon as tht patch merges | 18:17 |
| rlandy|ruck | hmmm ... we have no hash promoted | 18:18 |
| rlandy|ruck | ysandeep: ack | 18:18 |
| ysandeep | iiuc... Apart from downstream baremetal i think we will be good with these changes, we might need some changes for baremetal. | 18:19 |
| rlandy|ruck | ysandeep: w+'ed it | 18:19 |
| ysandeep | thanks! | 18:20 |
| rlandy|ruck | current-tripleo/2021-11-15 14:47 | 18:20 |
| rlandy|ruck | no master c9 promotion | 18:20 |
| ysandeep | rlandy|ruck, ^^ need help figuring that out? | 18:21 |
| akahat | c8 train promoting: http://promoter.rdoproject.org/promoter_logs/centos8_train.log | 18:22 |
| rlandy|ruck | ysandeep: no promoter is busy wth train | 18:22 |
| rlandy|ruck | akahat: ack - just saw that | 18:22 |
| ysandeep | okay, great! | 18:22 |
| *** ysandeep is now known as ysandeep|out | 18:23 | |
| * ysandeep|out out for the day, see you tomorrow o/ | 18:23 | |
| rlandy|ruck | enqueued wallaby | 18:31 |
| akahat | rlandy|ruck, no such manifest: trunk.registry.rdoproject.org/tripleomastercentos9/openstack-ceilometer-central:9d27ee6e0e4f5143dbc49d4c775c9cd5 | 18:36 |
| akahat | http://promoter.rdoproject.org/promoter_logs/centos9_master.log | 18:36 |
| rlandy|ruck | 2021-12-07 18:41:49,384 4036246 ERROR promoter Containers promote 'aggregate: 9d27ee6e0e4f5143dbc49d4c775c9cd5, commit: dae65ca8df725cf989cf70f142d07efd5ce3cc0f, distro: 0385a1c2f9486db83a8b1d3cbeabfd6da81e77ef, extended: None, component: validation, timestamp: 1638878898' to current-tripleo: Failed promotion | 18:45 |
| rlandy|ruck | akahat: probably the containers were pruned from rdo registry | 18:46 |
| rlandy|ruck | akahat: are containers with that hash on rdo registry? | 18:56 |
| akahat | rlandy|ruck, no. those are not there. | 19:07 |
| dviroel | https://logserver.rdoproject.org/67/36267/37/check/periodic-tripleo-ci-build-containers-centos-9-push-master/cdcd5d6/job-output.txt | 19:10 |
| dviroel | full_hash: 9d27ee6e0e4f5143dbc49d4c775c9cd5 | 19:11 |
| dviroel | should be pushing, right? | 19:11 |
| dviroel | 2021-12-07 16:48:33.337183 | TASK [build-containers : Get contents of openstack repo baseurl for the version hash] | 19:17 |
| dviroel | 2021-12-07 11:48:33.761993 | primary | 6c06225cee172f4363b713b9d2c76d80 | 19:17 |
| akahat | same hash is getting pushed every time? ^^ | 19:23 |
| marios|out | dviroel: looks like issue comes from there https://opendev.org/openstack/tripleo-ci/src/branch/master/roles/build-containers/tasks/main.yaml#L21 | 19:23 |
| marios|out | dviroel: not sure whats going on i will dig more in my morning | 19:23 |
| akahat | "name": "openstack-aodh-api:6c06225cee172f4363b713b9d2c76d80", ... "creationTimestamp": "2021-12-07T16:59:43Z" | 19:23 |
| marios|out | it fetches the hash from the openstack_repo_name (delorean.repo) - i think that task is wrong/needs update | 19:23 |
| dviroel | marios|out: yeah, i'm taking a look there atm | 19:24 |
| marios|out | dviroel: thanks please add me if you post any reviews etc | 19:24 |
| dviroel | marios|out: will do, tks, have a good night ;) | 19:24 |
| marios|out | dviroel: maybe we just need to set buildcontainers_version_hash ? not sure i'm reading teh play correctly | 19:26 |
| marios|out | dviroel: thanks... am watching bob young interview not really working now ;) | 19:27 |
| dviroel | marios|out: ack, comparing with centos-8 one too, to see if this is the flow | 19:28 |
| marios|out | thanks dviroel have a good one o/ | 19:29 |
| dviroel | o/ | 19:29 |
| rlandy|ruck | dviroel: https://review.opendev.org/c/openstack/tripleo-ci/+/820850 - ok to w+? | 19:59 |
| dviroel | rlandy|ruck: yes, you can W+ | 20:01 |
| rlandy|ruck | dviroel: that patch will fail check | 20:06 |
| rlandy|ruck | some timed out jobs | 20:06 |
| rlandy|ruck | 2021-12-07 17:37:38.285 | ERROR:root:overcloud-base has no valid mapping for package lvm2 | 20:09 |
| rlandy|ruck | dviroel: ^^ legit errors here | 20:09 |
| dviroel | ack | 20:15 |
| dviroel | rlandy|ruck: lets hold marios patch for now, I'll amend it with a tentative to provide the correct hash to build-containers role | 20:52 |
| rlandy|ruck | dviroel: thanks | 20:54 |
| rlandy|ruck | program doc is updated | 22:26 |
| rlandy|ruck | dviroel: hmm ... we've got broken promoter now | 23:34 |
| rlandy|ruck | http://promoter.rdoproject.org/promoter_logs/centos8_train.log | 23:34 |
| rlandy|ruck | build-containers role broken everywhere then | 23:35 |
| rlandy|ruck | trunk.registry.rdoproject.org/tripleotraincentos8/centos-binary-heat-api-cfn:79f67d27b580343cdbb196999362461f | 23:35 |
| dviroel | hum, weird | 23:50 |
| rlandy|ruck | dviroel: maybe it's only train | 23:58 |
| dviroel | rlandy|ruck: ack, maybe some artifact is missing | 23:58 |
| rlandy|ruck | dviroel: left marios, pooja, amol, | 23:58 |
| rlandy|ruck | they can review tomorrow | 23:58 |
| dviroel | rlandy|ruck: https://review.rdoproject.org/r/c/testproject/+/36817/13#message-e7a07ee1d812b52a515f02c166d52b2eac52b1c8 | 23:58 |
| dviroel | ^ now it is pushing to the correct hash | 23:59 |
| rlandy|ruck | cool | 23:59 |
| rlandy|ruck | dviroel: late for you | 23:59 |
| rlandy|ruck | dviroel: see you tomorrow | 23:59 |
| dviroel | rlandy|ruck: see you o/ | 23:59 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!