| *** pojadhav|out is now known as pojadhav | 01:42 | |
| *** ysandeep|out is now known as ysandeep | 04:30 | |
| bhagyashris | ykarel|away, hey is the https://review.rdoproject.org down ? | 04:32 |
|---|---|---|
| bhagyashris | now it's working | 04:38 |
| *** ykarel|away is now known as ykarel | 05:02 | |
| chandankumar | pojadhav: hello, added the comment here https://bugzilla.redhat.com/show_bug.cgi?id=1994848#c3 Hope it will help to fix the issue. | 06:14 |
| pojadhav | chandankumar, ack | 06:14 |
| chandankumar | zbr: hello | 06:36 |
| chandankumar | zbr: please have a look at this bug https://bugs.launchpad.net/tripleo/+bug/1940313 when free, thanks :-) | 06:37 |
| ykarel | chandankumar, zbr also check https://review.opendev.org/c/openstack/tripleo-quickstart/+/801776 | 06:50 |
| ykarel | quickstart deploys broken with that | 06:50 |
| * ykarel afk | 06:56 | |
| chandankumar | ykarel: sorry, we missed that case, proposed a fix https://review.opendev.org/c/openstack/tripleo-quickstart/+/804955 to add it back | 07:05 |
| zbr | i think it is wrong to alter collection path, no real reason for doing it | 07:07 |
| zbr | collections can be installed at user or system level using known official paths | 07:07 |
| zbr | not using them open doors for other kind of bugs like shadowing | 07:08 |
| chandankumar | zbr: yes, you made a valid point | 07:16 |
| zbr | how about installing it to one of standard locations? | 07:16 |
| chandankumar | zbr: may be we can add a option in quickstart.sh script to install any collection using ansible-galaxy | 07:17 |
| chandankumar | that will land it to standard location | 07:17 |
| zbr | same applies for roles but there it can be bit more complex to fix (more roles floating around) | 07:17 |
| zbr | is not so easy because we wanted to be able to use depends on, and this is why the install block looks bit more complex | 07:18 |
| zbr | but we could make it a function with params and reuse it | 07:19 |
| chandankumar | zbr: yes, that sounds good | 07:20 |
| chandankumar | zbr: I will update my patch | 07:20 |
| *** jpena|off is now known as jpena | 07:34 | |
| zbr | chandankumar: was the pip install issue on py36 fixed or we still look for solutions? | 07:41 |
| chandankumar | zbr: not yet fixed | 07:42 |
| * zbr thinking about patching pip itself | 07:46 | |
| chandankumar | zbr: Actually depends-on is not working here https://review.opendev.org/c/openstack/python-tripleoclient/+/804887 | 07:46 |
| ykarel | chandankumar, ack | 07:49 |
| *** jpodivin_ is now known as jpodivin | 08:10 | |
| chandankumar | zbr: what about this https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/804890/2#message-291ace911ba6936e9c823ca0c534a7b6ea1141dc ? | 08:19 |
| chandankumar | for current situation if we land the workaround and if it works, we are good to unblock the broken ci | 08:20 |
| *** ysandeep is now known as ysandeep|lunch | 08:26 | |
| *** ykarel is now known as ykarel|lunch | 09:04 | |
| chandankumar | zbr: one query does upstream tox jobs run in a container? | 09:08 |
| *** chem_ is now known as chem | 09:12 | |
| *** chem is now known as Guest4716 | 09:13 | |
| zbr | not usually | 09:13 |
| zbr | i know that rdo does run some jobs inside containers but upstream i am not aware of any job doing so, even if is it possible. | 09:14 |
| zbr | chandankumar: be sure you scrubcribe to https://github.com/pypa/pip/issues/10219 and read my last comment, i added some details. | 09:19 |
| chandankumar | zbr: sure | 09:21 |
| zbr | chandankumar: py36 workaround passed: https://review.opendev.org/c/openstack/python-tripleoclient/+/804887 | 09:46 |
| *** ysandeep|lunch is now known as ysandeep | 09:47 | |
| soniya29 | chandankumar, kopecmartin, arxcruz, ysandeep, rlandy, please add/edit today's agenda for the tempest meeting :- https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ | 10:33 |
| *** ykarel|lunch is now known as ykarel | 10:58 | |
| *** dviroel|ruck|out is now known as dviroel|ruck | 11:12 | |
| *** beagles is now known as eagles | 11:28 | |
| chandankumar | akahat: bhagyashris please have a look at this https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34653 when free, thanks! | 11:33 |
| akahat | chandankumar, ack | 11:33 |
| *** jpena is now known as jpena|lunch | 11:34 | |
| bhagyashris | chandankumar, ack | 11:35 |
| *** rlandy is now known as rlandyrover | 11:36 | |
| *** rlandyrover is now known as rlandy|rover | 11:36 | |
| chandankumar | pojadhav: please have a look at this https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34895 when free, thanks :-) | 11:37 |
| arxcruz | soniya29: i might be late 10 minutes, need to take my daughter at school | 11:41 |
| rlandy|rover | dviroel|ruck: hey - how are things? | 11:42 |
| dviroel|ruck | rlandy|rover: o/ | 11:42 |
| dviroel|ruck | ussuri promoted | 11:42 |
| dviroel|ruck | wallaby failing in two scenario010 jobs | 11:43 |
| dviroel|ruck | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby | 11:43 |
| rlandy|rover | dviroel|ruck: same problem as master? | 11:43 |
| rlandy|rover | scenario010 | 11:43 |
| rlandy|rover | failing consistently? | 11:44 |
| soniya29 | arxcruz, sure | 11:44 |
| dviroel|ruck | rlandy|rover: not sure, will check if the errors are the same | 11:44 |
| dviroel|ruck | rlandy|rover: https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby/2081351/logs/undercloud/var/log/containers/octavia/octavia.log.txt.gz | 11:44 |
| rlandy|rover | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby | 11:46 |
| rlandy|rover | dviroel|ruck: yeah we have consistent failure | 11:46 |
| rlandy|rover | may not be the same cause | 11:46 |
| dviroel|ruck | and for ovn too | 11:46 |
| dviroel|ruck | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-ovn-provider-standalone-wallaby | 11:46 |
| dviroel|ruck | didn't dig into ovn errors yet | 11:47 |
| rlandy|rover | dviroel|ruck: do you want to talk at CIX call? | 11:47 |
| rlandy|rover | cards should be commented | 11:48 |
| dviroel|ruck | rlandy|rover: sure, let me try this time | 11:50 |
| rlandy|rover | dviroel|ruck: k - great - will be on the call if you want me to step in - but you'll be fine | 11:51 |
| rlandy|rover | dviroel|ruck: let's bug the wallaby one separately | 11:51 |
| rlandy|rover | ykarel: hi - what's the status with train? | 11:52 |
| rlandy|rover | we still have the promotions blocked | 11:53 |
| rlandy|rover | are you still working on that? | 11:53 |
| ykarel | rlandy|rover, glance store is not yet in promoted-components | 11:53 |
| rlandy|rover | ykarel: k - holding promotion | 11:56 |
| rlandy|rover | need to check f it's stuck in component lines | 11:56 |
| ykarel | last run i see all train jobs passed | 11:56 |
| * rlandy|rover checks | 11:57 | |
| ykarel | okk next run of periodic-tripleo-centos-8-train-component-glance-promote-to-promoted-components should promote it | 11:58 |
| rlandy|rover | ykarel: https://review.rdoproject.org/zuul/buildset/cb6de673ff7d4c2196ed83fd65008972 https://review.rdoproject.org/zuul/buildset/cb6de673ff7d4c2196ed83fd65008972 | 11:59 |
| rlandy|rover | both have failures | 11:59 |
| ykarel | rlandy|rover, no not that, first we need glance component promotion | 11:59 |
| rlandy|rover | https://review.rdoproject.org/zuul/builds?pipeline=openstack-component-glance | 12:00 |
| rlandy|rover | yeah - that is clear | 12:00 |
| rlandy|rover | ok - got it | 12:00 |
| ykarel | yeap so just need to run periodic-tripleo-centos-8-train-component-glance-promote-to-promoted-components | 12:00 |
| soniya29 | kopecmartin, ysandeep, chandankumar, rlandy|rover, tempest meeting | 12:00 |
| rlandy|rover | we can run that manually or it will kick every two hours | 12:00 |
| rlandy|rover | soniya29: need to be on program call | 12:01 |
| ykarel | so if integration run is about to start before next glance promotion, then ok to run manual | 12:01 |
| rlandy|rover | ysandeep: pojadhav: let's meet later re: downstream components | 12:02 |
| rlandy|rover | we need to clean those up | 12:02 |
| rlandy|rover | after program call | 12:02 |
| ysandeep | rlandy|rover, sure | 12:02 |
| soniya29 | weshay|ruck, tempest meeting? | 12:03 |
| ykarel | okk integration run is in 6 hours, so can wait for auto component promotion, rlandy|rover i hope by tomorrow can unblock promotion | 12:03 |
| ykarel | unless and until manual triggers are done for pipelines | 12:03 |
| rlandy|rover | ykarel: perfect - thank you | 12:03 |
| weshay|ruck | soniya29, yep | 12:04 |
| rlandy|rover | openstack-promote-component just kicked | 12:04 |
| rlandy|rover | we need to watch glance train componnet promotes these | 12:04 |
| rlandy|rover | there | 12:04 |
| dviroel|ruck | rlandy|rover: re https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby | 12:05 |
| dviroel|ruck | only the last run has that octavia error, seems that other ones are failing in the standalone deploy | 12:06 |
| dviroel|ruck | https://logserver.rdoproject.org/86/31586/96/check/periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby/6abaab0/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 12:06 |
| rlandy|rover | dviroel|ruck: real issue | 12:07 |
| rlandy|rover | let's raise it at CIX | 12:07 |
| dviroel|ruck | stanalone deploy: FATAL | print ceph-ansible output in case of failure | undercloud | error={ | 12:08 |
| rlandy|rover | https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-scenario010-standalone-master/7ae37e3/logs/undercloud/var/log/extra/podman/containers/octavia_worker/stdout.log.txt.gz | 12:08 |
| rlandy|rover | ^^ error for master | 12:08 |
| rlandy|rover | different on wallaby | 12:09 |
| rlandy|rover | https://logserver.rdoproject.org/86/31586/96/check/periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby/6abaab0/logs/undercloud/var/log/extra/podman/containers/ | 12:09 |
| rlandy|rover | https://logserver.rdoproject.org/86/31586/96/check/periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby/6abaab0/logs/undercloud/var/log/containers/ | 12:10 |
| rlandy|rover | dviroel|ruck: it may be a similar problem but the error is different | 12:11 |
| rlandy|rover | so I would add a different bug | 12:11 |
| rlandy|rover | the team may link them | 12:12 |
| dviroel|ruck | but looking at https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-scenario010-standalone-wallaby | 12:12 |
| dviroel|ruck | only the last run fails in octavia, the other ones are failing in standalone deploy, something related with ceph deploy | 12:12 |
| chandankumar | rlandy|rover: weshay|ruck https://review.rdoproject.org/r/q/topic:%22jenkins-v2%22+(status:open%20OR%20status:merged) please have a look at this series when free, thanks :-) | 12:12 |
| rlandy|rover | dviroel|ruck: yeah - we can add debug info to th LP | 12:14 |
| rlandy|rover | https://review.rdoproject.org/zuul/buildset/21b02a08411d43449d11f89b491ad2be | 12:18 |
| rlandy|rover | rest of train is working | 12:18 |
| rlandy|rover | that is good | 12:18 |
| *** sshnaidm|pto is now known as sshnaidm | 12:25 | |
| *** jpena|lunch is now known as jpena | 12:32 | |
| rlandy|rover | bhagyashris: hi | 12:38 |
| rlandy|rover | merging the last of dviroel|ruck's patches https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34767 | 12:39 |
| rlandy|rover | not sure why that didn't merge yesterday | 12:39 |
| rlandy|rover | maybe zuul restart | 12:39 |
| rlandy|rover | but you should be good to go there | 12:39 |
| bhagyashris | rlandy|rover, ack | 12:39 |
| rlandy|rover | bhagyashris: the patch is rerunning check | 12:39 |
| bhagyashris | rlandy|rover, ok | 12:39 |
| rlandy|rover | chandankumar: ysandeep: pojadhav: ok to meet now? re: components downstream | 12:41 |
| ysandeep | rlandy|rover, Yes i am available to join | 12:41 |
| bhagyashris | dviroel|ruck, rlandy|rover fyi i have check dviroel|ruck patch https://review.rdoproject.org/r/c/config/+/34709 did some changes. dviroel|ruck I think we should make that ready for review and merge as soon as possible so that we can test that job through testproject patch. | 12:41 |
| rlandy|rover | bhagyashris: k - will look after meeting | 12:41 |
| chandankumar | rlandy|rover: yes | 12:42 |
| rlandy|rover | ysandeep: chandankumar: https://meet.google.com/whi-hnun-zbf?pli=1&authuser=0 | 12:42 |
| ysandeep | pojadhav, ^^ | 12:42 |
| rlandy|rover | pojadhav: around? pls join us | 12:42 |
| bhagyashris | as the job definition is in the config repo ^ so we will need this patch https://review.rdoproject.org/r/c/config/+/34709 merged first and then only we can test through testproject patch | 12:43 |
| dviroel|ruck | bhagyashris: ok, will take a look, thanks | 12:44 |
| bhagyashris | dviroel|ruck, can remove the Work in progress from https://review.rdoproject.org/r/c/config/+/34709 | 12:44 |
| dviroel|ruck | done | 12:44 |
| bhagyashris | dviroel|ruck, thanks once it get merge i will hit the testproject patch to test our change | 12:45 |
| bhagyashris | thanks | 12:45 |
| pojadhav | rlandy|rover, sorry | 12:47 |
| pojadhav | joining | 12:47 |
| dviroel|ruck | rlandy|rover: i'm openning the bug pointing to the ceph deploy error, almost all failures of scenarion010 and scenarion010-ovn-provider had the same error ceph error, only one passed standalone and failed at octavia | 12:49 |
| weshay|ruck | dviroel|ruck, updated your bug.. let's review the settings for ceph | 13:23 |
| weshay|ruck | dviroel|ruck, failed on /usr/sbin/blkid -c /dev/null -s PARTUUID -o value ceph_vg/ceph_lv_wal | 13:24 |
| dviroel|ruck | oh, ok | 13:26 |
| weshay|ruck | dviroel|ruck, so.. I see that volume path removed here: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803859/4/ci/environments/scenario001-standalone.yaml | 13:29 |
| weshay|ruck | perhaps something similar is needed | 13:29 |
| weshay|ruck | I don't see that same exact thing for scenario010 yet | 13:30 |
| weshay|ruck | dviroel|ruck, hrm..https://codesearch.opendev.org/?q=ceph_lv_wal&i=nope&literal=nope&files=&excludeFiles=&repos= | 13:30 |
| weshay|ruck | dviroel|ruck, https://opendev.org/openstack/tripleo-ci/commit/226e4f9b3ef4c33f2885eb461ee38bee1273878c | 13:31 |
| weshay|ruck | dviroel|ruck, ping fmount in #tripleo | 13:34 |
| weshay|ruck | see if he can help trace this down | 13:34 |
| dviroel|ruck | ack, will do | 13:38 |
| ysandeep | weshay|ruck, dviroel|ruck fyi.. to me looks like we are hitting similiar issue in downstream for 17: https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario010-standalone-rhos-17/6157c8e/logs/undercloud/home/zuul/standalone_deploy.log | 13:43 |
| weshay|ruck | ysandeep, ya.. same error https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-17/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario010-standalone-rhos-17/6157c8e/logs/undercloud/var/log/ceph/ceph-volume.log | 13:43 |
| weshay|ruck | rlandy|rover, HA.. PROOF of the why :) | 13:44 |
| weshay|ruck | ysandeep, dviroel|ruck what I haven't found is where ceph_vg/ceph_lv_wal is currently defined for that job | 13:44 |
| rlandy|rover | weshay|ruck: weshay|ruck: sorry - long meeting | 13:48 |
| rlandy|rover | dviroel|ruck: ^^ | 13:49 |
| ysandeep | rlandy|rover, https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/266124 - Temporary remove infrared jobs from 16.2 criteria | 13:59 |
| ysandeep | rlandy|rover: for 17 bm: https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/266129 | 14:08 |
| * pojadhav brb | 14:09 | |
| rlandy|rover | ysandeep: looking | 14:11 |
| zbr | chandankumar: are you working to change tripleo-operator-ansible collection install logic to make it use standard ansible paths? | 14:11 |
| ysandeep | rlandy|rover, also chatting with attila on why their jobs are failing. | 14:12 |
| chandankumar | zbr: yes, will get the patch tomorrow, not yet started | 14:12 |
| zbr | ok, let me know if you need help. the code we recently added on install-deps.sh should give be reusable. | 14:13 |
| rlandy|rover | https://code.engineering.redhat.com/gerrit/c/tripleo-environments/+/266129/2/ci-scripts/dlrnapi_promoter/config/RedHat-8/component/rhos-17.yaml | 14:14 |
| rlandy|rover | checking if ovb passes on those components | 14:14 |
| zbr | dviroel|ruck: i think you can rebase https://review.opendev.org/c/openstack/tripleo-repos/+/804306 now, if you find the time. | 14:22 |
| ysandeep | weshay|ruck, dviroel|ruck hmmm 16.2 sc01/04/10 also failing on same error - https://sf.hosted.upshift.rdu2.redhat.com/logs/openstack-periodic-integration-rhos-16.2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-rhel-8-scenario010-standalone-rhos-16.2/72d22f9/logs/undercloud/home/zuul/standalone_deploy.log | 14:23 |
| dviroel|ruck | ysandeep: weshay|ruck fmount is already working on a fix for wallaby | 14:25 |
| ysandeep | dviroel|ruck, wondering if we are also hitting this on train.. because 16.2 == train | 14:25 |
| dviroel|ruck | we can check with him | 14:26 |
| weshay|ruck | rock | 14:26 |
| *** ykarel is now known as ykarel|away | 14:42 | |
| rlandy|rover | dviroel|ruck++ | 14:42 |
| dviroel|ruck | rlandy|rover: tks for the help | 14:43 |
| dviroel|ruck | :) | 14:43 |
| rlandy|rover | you didn't need any | 14:43 |
| weshay|ruck | rlandy|rover, ysandeep need any help w/ 16.2 or 17? | 14:43 |
| rlandy|rover | weshay|ruck: long story | 14:43 |
| rlandy|rover | see #openstack-pcci | 14:43 |
| rlandy|rover | qe jobs | 14:43 |
| pojadhav | weshay|ruck, rlandy|rover : for a sec can we chat ? | 14:46 |
| rlandy|rover | pojadhav: yes | 14:46 |
| pojadhav | https://meet.google.com/ncb-hutx-hay?authuser=0&hl=en | 14:46 |
| dviroel|ruck | ysandeep: weshay|ruck the fix for train was already merged here vhttps://review.opendev.org/c/openstack/tripleo-heat-templates/+/803861 | 14:51 |
| dviroel|ruck | https://review.opendev.org/c/openstack/tripleo-heat-templates/+/803861 | 14:51 |
| dviroel|ruck | but still missing from 16.2 | 14:51 |
| dviroel|ruck | according to the logs | 14:51 |
| ysandeep | dviroel|ruck, thanks! let me check where is that fix in downstream | 14:55 |
| weshay|ruck | ysandeep, doubt it.. | 14:59 |
| rlandy|rover | arxcruz: hey health meeting | 15:03 |
| bhagyashris | folks review this patch https://review.rdoproject.org/r/c/config/+/34709 when you have free time .. | 15:10 |
| *** ysandeep is now known as ysandeep|away | 15:38 | |
| rlandy|rover | frenzy_friday: k - joining | 15:41 |
| *** jpena is now known as jpena|off | 15:42 | |
| dviroel|ruck | hi bhagyashris, about #34709 | 15:59 |
| dviroel|ruck | i think that we should inherit the playbook from ci-config https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34767/5/playbooks/tripleo-ci-promote-centos-compose/run.yaml | 15:59 |
| dviroel|ruck | it will bring less changes to conf | 15:59 |
| dviroel|ruck | config repo | 15:59 |
| dviroel|ruck | is it possible? | 15:59 |
| dviroel|ruck | also, see that https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34767/5/playbooks/tripleo-ci-promote-centos-compose/run.yaml is now slight different from the one that I added in #34709 | 16:00 |
| dviroel|ruck | there is an ssh_agent role included to work with the secret | 16:00 |
| dviroel|ruck | it seems that is the correct way of dealing with secrets | 16:00 |
| dviroel|ruck | i'm writing these notes in the patch :) | 16:14 |
| dviroel|ruck | rlandy|rover: https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates | 16:39 |
| rlandy|rover | https://d65f6297df4427ee4482-2d1c1acd754ff997a645a979c859a6ed.ssl.cf1.rackcdn.com/783824/10/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/cd771f2/logs/undercloud/home/zuul/install-undercloud.log | 16:42 |
| rlandy|rover | k - will have to see what beaks there | 16:42 |
| rlandy|rover | breaks | 16:42 |
| rlandy|rover | 2021-08-18 15:01:40.936605 | bc764e04-5fe9-024c-467e-000000000de1 | TIMING | tripleo_container_image_prepare : Container image prepare | undercloud | 0:03:18.590114 | 2.05s | 16:43 |
| rlandy|rover | https://d65f6297df4427ee4482-2d1c1acd754ff997a645a979c859a6ed.ssl.cf1.rackcdn.com/783824/10/gate/tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates/cd771f2/logs/undercloud/var/log/tripleo-container-image-prepare.log | 16:44 |
| rlandy|rover | tag unknown | 16:44 |
| rlandy|rover | hmmm is it only this job? | 16:44 |
| rlandy|rover | promotion issue? access issue? | 16:45 |
| weshay|ruck | rlandy|rover, re: gate? | 16:49 |
| weshay|ruck | ya.. master just promoted | 16:49 |
| weshay|ruck | left notes in gerrit | 16:49 |
| dviroel|ruck | i only saw this error in this job | 16:49 |
| jbadiapa | rlandy|rover, I've just updated the launchpad with that issue... https://bugs.launchpad.net/tripleo/+bug/1921285 | 16:51 |
| jbadiapa | I was about to ask on the channel | 16:52 |
| rlandy|rover | jbadiapa: dviroel|ruck: if it's only on one job, then yeah - needs investigation | 16:58 |
| rlandy|rover | a085460a23443cfc28a1dc6c667e110e' | 16:58 |
| rlandy|rover | check if that tag exists | 16:58 |
| jbadiapa | yes, it's on the containers-prepare-parameter.yaml | 16:59 |
| rlandy|rover | 3 hours ago | 17:00 |
| rlandy|rover | master | 17:00 |
| rlandy|rover | ^^ master promotion | 17:00 |
| rlandy|rover | let's rerun and check if it's still an issue | 17:00 |
| rlandy|rover | real question is why only this job | 17:01 |
| jbadiapa | ack, should we wait more or just recheck now? | 17:01 |
| rlandy|rover | now shouldd be fine | 17:01 |
| rlandy|rover | jbadiapa: bug is legit if it's only this job but a rerun may get your patch through | 17:01 |
| rlandy|rover | ping here if now | 17:01 |
| rlandy|rover | not | 17:01 |
| jbadiapa | ack, thx | 17:02 |
| dviroel|ruck | rlandy|rover: weshay|ruck fyi, will be out in ~2h for a medical appoint (19h UTC) | 17:03 |
| rlandy|rover | np | 17:03 |
| rlandy|rover | we'll cover | 17:03 |
| weshay|ruck | aye | 17:03 |
| dviroel|ruck | weshay|ruck: it seams that with fmount fix we are hitting another issue: https://c9efb240a15ba175da34-a2c887b0b0745fbe4c981c51dfe26179.ssl.cf2.rackcdn.com/805034/1/check/tripleo-ci-centos-8-scenario010-standalone/cb05526/logs/undercloud/var/log/ceph/cephadm.log | 17:10 |
| dviroel|ruck | that already has a LP | 17:10 |
| dviroel|ruck | hum, do we have a LP for that? | 17:14 |
| dviroel|ruck | can't find | 17:15 |
| dviroel|ruck | weshay|ruck: not sure if is expected to have this container bump in wallaby too https://opendev.org/openstack/tripleo-quickstart-extras/commit/76a5c4efc66f4e2b6393aef2eead53c03be43896 | 17:41 |
| dviroel|ruck | or if just missing wallaby in the elif releases | 17:41 |
| dviroel|ruck | asked fmount | 17:42 |
| dviroel|ruck | probably no | 17:53 |
| weshay|ruck | rlandy|rover, https://review.opendev.org/c/openstack/releases/+/804871 | 17:59 |
| rlandy|rover | dviroel|ruck: sorry - was in meeting - you all set? need help with ^^? | 18:20 |
| dviroel|ruck | rlandy|rover: yes, for wallaby, https://c9efb240a15ba175da34-a2c887b0b0745fbe4c981c51dfe26179.ssl.cf2.rackcdn.com/805034/1/check/tripleo-ci-centos-8-scenario010-standalone/cb05526/logs/undercloud/var/log/ceph/cephadm.log | 18:22 |
| dviroel|ruck | can't find v6.0.4-stable-6.0-pacific-centos-8-x86_64 | 18:22 |
| rlandy|rover | Error: Error initializing source docker://172.99.67.107:5001/tripleowallaby/daemon:v6.0.4-stable-6.0-pacific-centos-8-x86_64: Error reading manifest v6.0.4-stable-6.0-pacific-centos-8-x86_64 in 172.99.67.107:5001/tripleowallaby/daemon: manifest unknown: manifest unknown | 18:22 |
| rlandy|rover | that was just updated for master | 18:22 |
| * rlandy|rover gets | 18:23 | |
| rlandy|rover | https://github.com/openstack/tripleo-quickstart-extras/commit/b9fa170b1e848d2051883ada0b1fef2817d1f4c3 | 18:23 |
| rlandy|rover | dviroel|ruck: we need to check with jfulton where wallaby lands | 18:23 |
| rlandy|rover | let's go to # tripleo | 18:23 |
| dviroel|ruck | rlandy|rover: according with fmount | 18:24 |
| dviroel|ruck | rlandy|rover: both wallaby and xena (master) use latest pacific | 18:24 |
| rlandy|rover | dviroel|ruck: ok - if that is correct - we need to check tripleo-common | 18:25 |
| dviroel|ruck | https://github.com/openstack/tripleo-quickstart-extras/commit/b9fa170b1e848d2051883ada0b1fef2817d1f4c3 | 18:25 |
| dviroel|ruck | ^ ignore that | 18:25 |
| dviroel|ruck | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_451/805034/1/check/tripleo-ci-centos-8-content-provider/451ff64/logs/undercloud/home/zuul/containers-prepare-parameters.yaml | 18:25 |
| rlandy|rover | ceph_tag: v6.0.0-stable-6.0-pacific-centos-8-x86_64 | 18:26 |
| dviroel|ruck | yes | 18:27 |
| rlandy|rover | https://review.opendev.org/c/openstack/tripleo-heat-templates/+/805034 | 18:28 |
| rlandy|rover | ok I see | 18:29 |
| rlandy|rover | job passed :) | 18:30 |
| dviroel|ruck | but not the standalone-010 ones | 18:32 |
| rlandy|rover | maybe redefined somewhere | 18:33 |
| dviroel|ruck | https://opendev.org/openstack/tripleo-common/src/branch/stable/wallaby/container-images/container_image_prepare_defaults.yaml | 18:34 |
| rlandy|rover | dviroel|ruck++ you found it | 18:35 |
| rlandy|rover | update that | 18:35 |
| rlandy|rover | dviroel|ruck: and also https://opendev.org/openstack/tripleo-common/src/branch/stable/wallaby/container-images/container_image_prepare_defaults.yaml#L29 | 18:35 |
| rlandy|rover | https://opendev.org/openstack/tripleo-common/src/branch/master/container-images/container_image_prepare_defaults.yaml | 18:36 |
| rlandy|rover | should match ^^ | 18:36 |
| dviroel|ruck | will do | 18:38 |
| rlandy|rover | dviroel|ruck: you're a natural at this ruck/rover gig | 18:39 |
| rlandy|rover | we may call you back :) | 18:39 |
| dviroel|ruck | :) fmount also helped me | 18:40 |
| dviroel|ruck | maybe backport this https://review.opendev.org/c/openstack/tripleo-common/+/797226 | 18:41 |
| dviroel|ruck | :) | 18:41 |
| weshay|ruck | I think sagi had a backport | 18:41 |
| weshay|ruck | nope | 18:42 |
| dviroel|ruck | https://review.opendev.org/c/openstack/tripleo-common/+/805007 | 18:43 |
| weshay|ruck | dviroel|ruck++ | 18:44 |
| weshay|ruck | dviroel|ruck, as you have time https://review.rdoproject.org/r/c/rdo-infra/ci-config/+/34999 | 18:45 |
| dviroel|ruck | weshay|ruck: ack | 18:45 |
| weshay|ruck | using the ruck_rover tool w/ release + component to check that makes it easy | 18:45 |
| *** dviroel|ruck is now known as dviroel|ruck|out | 19:07 | |
| rlandy|rover | weshay|ruck: W+ing ^^ | 19:08 |
| weshay|ruck | rlandy|rover, thanks | 19:11 |
| rlandy|rover | dviroel|ruck|out: weshay|ruck: train is running now | 19:12 |
| rlandy|rover | if this passes | 19:12 |
| rlandy|rover | we should edit the train criteria on promoter | 19:13 |
| rlandy|rover | and restart promoter | 19:13 |
| rlandy|rover | checking drln to be sure | 19:13 |
| weshay|ruck | k | 19:13 |
| weshay|ruck | akahat, let's get git pulls back on :))))))) | 19:13 |
| rlandy|rover | some weirdness with periodic-tripleo-centos-8-buildimage-overcloud-full-train | 22:27 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!