| *** rfolco has quit IRC | 00:07 | |
| *** rlandy has quit IRC | 01:19 | |
| *** rfolco has joined #oooq | 01:34 | |
| *** rfolco has quit IRC | 01:49 | |
| *** apetrich has quit IRC | 02:14 | |
| *** skramaja has joined #oooq | 03:50 | |
| *** saneax has joined #oooq | 04:00 | |
| *** skramaja has quit IRC | 04:02 | |
| *** ratailor has joined #oooq | 04:30 | |
| *** kopecmartin is now known as kopecmartin|pto | 04:46 | |
| *** skramaja has joined #oooq | 04:59 | |
| *** ykarel has joined #oooq | 05:01 | |
| *** marios has joined #oooq | 05:05 | |
| *** marios is now known as marios|ruck | 05:05 | |
| *** raukadah is now known as chkumar|rover | 05:06 | |
| *** ysandeep|off is now known as ysandeep | 05:58 | |
| chkumar|rover | marios|ruck: Good morning | 05:59 |
|---|---|---|
| chkumar|rover | marios|ruck: https://review.opendev.org/#/c/744955/1 for https://bugs.launchpad.net/tripleo/+bug/1889122 needed | 05:59 |
| openstack | Launchpad bug 1889122 in tripleo "mirror timeouts in upstream causing undercloud and standalone failures" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 05:59 |
| marios|ruck | chkumar|rover: ack thanks saw the new patches | 06:07 |
| chkumar|rover | marios|ruck: do we want to +w quickly so that it does not get caught againin gate again? | 06:14 |
| *** udesale has joined #oooq | 06:18 | |
| marios|ruck | chkumar|rover: lets try get a vote | 06:19 |
| marios|ruck | anyone around for quick vote please on train cherrypick tripleo-common https://review.opendev.org/#/c/744955/1 need it for the related-bug on review | 06:19 |
| marios|ruck | chkumar|rover: lets give it few mins and if no vote I will +A | 06:19 |
| marios|ruck | chkumar|rover: actually its ok expedited merge policy applies cos it is a cherrypick | 06:20 |
| marios|ruck | chkumar|rover: k will give it 5 mins and +A | 06:20 |
| chkumar|rover | marios|ruck: ok | 06:20 |
| marios|ruck | chkumar|rover: ah you +2 :D | 06:20 |
| marios|ruck | chkumar|rover: k done | 06:20 |
| chkumar|rover | marios|ruck: https://review.opendev.org/#/c/741443/ got badly impacted with rate limit | 06:21 |
| chkumar|rover | in gate | 06:21 |
| chkumar|rover | marios|ruck: thanks! | 06:21 |
| chkumar|rover | marios|ruck: I am going to re-run few ussuri jobs as they got failed due to retry limit | 06:50 |
| marios|ruck | ack chkumar|rover | 06:53 |
| *** skramaja_ has joined #oooq | 07:06 | |
| *** skramaja has quit IRC | 07:06 | |
| *** jpena|off is now known as jpena | 07:13 | |
| *** ysandeep is now known as ysandeep|lunch | 07:18 | |
| *** bogdando has joined #oooq | 07:30 | |
| *** dtantsur|afk is now known as dtantsur | 07:30 | |
| marios|ruck | chkumar|rover: oof https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-main https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable1 https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-integration-stable2 | 07:31 |
| *** apetrich has joined #oooq | 07:32 | |
| chkumar|rover | marios|ruck: for master, it might be network glitch, affected download, let me check timestamp of others failed job | 07:33 |
| chkumar|rover | * for train | 07:34 |
| marios|ruck | chkumar|rover: ussuri looks like problem with dlrn server? | 07:34 |
| marios|ruck | ussuri failed https://review.rdoproject.org/zuul/buildset/2415ed1dde0146159a927472a25ade7f for https://logserver.rdoproject.org/openstack-periodic-integration-stable1/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-ussuri-promote-promoted-components-to-tripleo-ci-testing/33f9395/job-output.txt * 2020-08-06 02:04:09.946585 | primary | dlrnapi_client.rest.ApiException: (500) | 07:35 |
| chkumar|rover | marios|ruck: yes | 07:35 |
| marios|ruck | chkumar|rover: master is node failure | 07:36 |
| *** tosky has joined #oooq | 07:46 | |
| *** udesale has quit IRC | 07:50 | |
| *** udesale has joined #oooq | 07:50 | |
| *** udesale has quit IRC | 08:00 | |
| *** udesale has joined #oooq | 08:00 | |
| chkumar|rover | marios|ruck: we are hitting networking issue on vexxhost, so no point on running jobs on vexxhost now as a testproject also | 08:05 |
| chkumar|rover | for train one, once everything gets merged, we will kick the pipeline on rdo-cloud | 08:06 |
| marios|ruck | chkumar|rover: ack... i am trying to dig into train logs but there are many issues it doesn't make sense | 08:06 |
| marios|ruck | chkumar|rover: i mean for example checkout this standalone-deploy.log https://logserver.rdoproject.org/openstack-periodic-integration-stable2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-train/77c7b75/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 08:08 |
| chkumar|rover | looking into that | 08:14 |
| marios|ruck | chkumar|rover: the whole buildset is a bit of a disaster and there are 2/3 different things here and am not sure if there are any bugs to be filed https://review.rdoproject.org/zuul/buildset/4194b87620d94559becd39ed94490697 | 08:15 |
| chkumar|rover | marios|ruck: yes, | 08:15 |
| marios|ruck | chkumar|rover: they are going to start shouting at us about train very soon | 08:15 |
| marios|ruck | chkumar|rover: we have to sort this out | 08:15 |
| marios|ruck | chkumar|rover: but not sure there is something we can do right now. | 08:16 |
| chkumar|rover | marios|ruck: let me change the nodeset of those jobs pointing to rdo-cloud and rekick it | 08:16 |
| *** ysandeep|lunch is now known as ysandeep | 08:16 | |
| marios|ruck | chkumar|rover: i mean we need to wait for the last things to merge in train | 08:16 |
| marios|ruck | chkumar|rover: and then yeah run in rdo | 08:16 |
| chkumar|rover | till then I can keep the patch ready | 08:16 |
| marios|ruck | chkumar|rover: http://paste.openstack.org/raw/796624/ fyi | 08:19 |
| chkumar|rover | ok | 08:30 |
| *** derekh has joined #oooq | 08:30 | |
| *** arxcruz|off is now known as arxcruz | 08:42 | |
| bogdando | hi folks. Looking into misc logs linked in https://review.opendev.org/#/c/743704 I'm wondering why did we lost /var/log/container-image-prepare.log ?.. | 08:45 |
| bogdando | tripleo-container-image-prepare-ansible.log* | 08:45 |
| bogdando | no more collected? why?.. | 08:45 |
| bogdando | ykarel, weshay|ruck: ^^ | 08:45 |
| bogdando | https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/config/collect-logs.yml#L279 doesn't that work anymore? | 08:46 |
| bogdando | https://opendev.org/openstack/tripleo-ci/src/branch/master/toci-quickstart/config/collect-logs.yml#L278 | 08:47 |
| ykarel | bogdando, let me check, i filed a bug for that earlier, not aware of status | 08:47 |
| ykarel | bogdando, https://bugs.launchpad.net/tripleo/+bug/1866676 | 08:48 |
| openstack | Launchpad bug 1866676 in tripleo "Container updates logs are no longer logged to /var/log/tripleo-container-image-prepare-ansible.log" [Medium,Triaged] | 08:48 |
| bogdando | thanks ykarel | 08:54 |
| chkumar|rover | marios|ruck: re-running train jobs on rdocloud https://review.rdoproject.org/r/#/c/28879/ | 09:15 |
| chkumar|rover | pcs fix landed https://review.opendev.org/#/c/744192/ | 09:15 |
| marios|ruck | chkumar|rover: thanks | 09:16 |
| marios|ruck | chkumar|rover: lets hope we're lucky | 09:16 |
| chkumar|rover | yup fingers crossed | 09:17 |
| marios|ruck | chkumar|rover: you seen/noticed a lot of these '[Zuul] Log Stream did not terminate' ? | 10:13 |
| chkumar|rover | marios|ruck: where | 10:22 |
| chkumar|rover | marios|ruck: not sure something is wrong today | 10:22 |
| chkumar|rover | marios|ruck: https://logserver.rdoproject.org/79/28879/1/check/periodic-tripleo-centos-8-train-containers-build-push/e8d2e75/job-output.txt | 10:22 |
| chkumar|rover | taking more than 1 hour and hitting post failure | 10:22 |
| marios|ruck | chkumar|rover: i am filing bug with the Log Stream did not terminate but there are many examples and 'weird' logs | 10:23 |
| chkumar|rover | marios|ruck: yes, please go ahead | 10:23 |
| marios|ruck | chkumar|rover: from periodics/integration & component | 10:23 |
| chkumar|rover | jpena: ^^ | 10:23 |
| jpena | chkumar|rover, marios|ruck: anything running on vexxhost is going to have trouble until they make networking stable again | 10:24 |
| chkumar|rover | jpena: registry is also on vexxhost na? | 10:24 |
| jpena | yep | 10:24 |
| chkumar|rover | oh | 10:24 |
| jpena | we moved all our infra to vexxhost (except lists.rdo and www.rdo, which are still WIP) | 10:25 |
| jpena | it's been quite stable so far, until we hit today's issues | 10:26 |
| marios|ruck | jpena: ack thanks but i don't think this one is restricted to vexx i have examples from rdo cloud too | 10:27 |
| marios|ruck | chkumar|rover: jpena: there https://bugs.launchpad.net/tripleo/+bug/1890571 | 10:27 |
| openstack | Launchpad bug 1890571 in tripleo "periodic integration/component jobs failing "[Zuul] Log Stream did not terminate"" [Critical,Triaged] | 10:27 |
| jpena | marios|ruck: right, but they are interacting with zuul components running on vexxhost | 10:27 |
| marios|ruck | jpena: i see | 10:27 |
| marios|ruck | jpena: that makes me feel slightly better oddly enough :/ | 10:27 |
| marios|ruck | jpena: cos this is really random the logs are weird nothing makes sense and they are all different | 10:28 |
| jpena | marios|ruck: yes. Every error message you could find is happening today :-/. | 10:28 |
| marios|ruck | chkumar|rover: jpena: few examples of Zuul] Log Stream did not terminate -> fail but the deployment is successful including green tempest run | 10:28 |
| jpena | marios|ruck: were the jobs running on RDO Cloud? | 10:29 |
| jpena | that would make sense | 10:29 |
| marios|ruck | jpena: mix of jobs | 10:29 |
| marios|ruck | jpena: some rdo some vexx | 10:29 |
| marios|ruck | jpena: in the references for the bug | 10:29 |
| marios|ruck | jpena: i updated the description if you want refresh | 10:30 |
| chkumar|rover | jpena: since registry is on vexx, if job runs on rdo cloud or vexxhost, it might got impacted | 10:32 |
| marios|ruck | chkumar|rover: check out the ������ https://logserver.rdoproject.org/openstack-component-baremetal/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-baremetal-master/d1081b0/logs/undercloud/home/zuul/undercloud_install.log.txt.gz | 10:33 |
| chkumar|rover | marios|ruck: it appears to be compiled binary | 10:34 |
| marios|ruck | chkumar|rover: yeah maybe | 10:34 |
| marios|ruck | chkumar|rover: but just another example of 'weirdness' after that 'log stream did not terminate' | 10:35 |
| marios|ruck | i need some kind of therapy after collecting those logs each one is a bit different | 10:35 |
| chkumar|rover | yup | 10:37 |
| *** ysandeep is now known as ysandeep|afk | 10:38 | |
| *** saneax has quit IRC | 10:42 | |
| *** ratailor has quit IRC | 10:57 | |
| *** ysandeep|afk is now known as ysandeep | 11:14 | |
| chkumar|rover | marios|ruck: brb for 30 mins | 11:14 |
| marios|ruck | ack chkumar|rover | 11:19 |
| *** jpena is now known as jpena|lunch | 11:31 | |
| *** sshnaidm is now known as sshnaidm|afk | 11:33 | |
| *** rfolco has joined #oooq | 11:37 | |
| *** ratailor has joined #oooq | 11:37 | |
| weshay|ruck | chkumar|rover, marios|ruck looks like another lovely day in infra | 11:40 |
| chkumar|rover | yes | 11:40 |
| chkumar|rover | today everything is screw up | 11:40 |
| chkumar|rover | i mean in terms of periodic and third party jobs | 11:42 |
| marios|ruck | o/ weshay|ruck hey yeah 'weirdness' today :/ filed a new bug for that see the hackmd | 11:46 |
| weshay|ruck | read through it.. thank you! | 11:47 |
| weshay|ruck | I see javier in #vexxhost | 11:47 |
| *** sshnaidm|afk is now known as sshnaidm | 11:49 | |
| rfolco | bhagyashris, akahat: promoter sync | 12:01 |
| *** udesale_ has joined #oooq | 12:03 | |
| *** udesale has quit IRC | 12:05 | |
| *** saneax has joined #oooq | 12:08 | |
| *** rlandy has joined #oooq | 12:17 | |
| rlandy | ysandeep: weshay|ruck: hello - sent a meeting invite to discuss next steps for 16.2 pipeline after containers are available | 12:21 |
| weshay|ruck | k | 12:21 |
| weshay|ruck | thanks | 12:21 |
| ysandeep | rlandy, ack | 12:21 |
| *** ykarel_ has joined #oooq | 12:26 | |
| *** ykarel has quit IRC | 12:29 | |
| rlandy | chkumar|rover: hey - can you show me where the buildah version is specified/locked for train? | 12:34 |
| rlandy | 16.2 picked up 1.14.9 and it's breaking life' | 12:35 |
| rlandy | as we know it | 12:35 |
| chkumar|rover | rlandy: we donot specify the buildah version we just use the default or getting installed from the repo | 12:35 |
| chkumar|rover | let me check c8 trian job | 12:35 |
| *** jpena|lunch is now known as jpena | 12:36 | |
| chkumar|rover | rlandy: https://logserver.rdoproject.org/87/28687/4/check/periodic-tripleo-ci-build-containers-ubi-8-push-train/ac8b385/logs/system/installed_pkgs.txt - buildah.x86_64 1.11.6-7.module_el8.2.0+305+5e198a41 @AppStream | 12:37 |
| rlandy | right we're using appstream | 12:37 |
| rlandy | 16.2 got all fancy and upgraded on us :( | 12:38 |
| rlandy | ok - for scrum meeting discussion | 12:38 |
| chkumar|rover | rlandy: https://review.opendev.org/#/c/745120/ -> https://review.opendev.org/#/c/745120/1/tripleo_ansible/roles/tripleo-podman/vars/redhat.yml is going to pin podman version for each release | 12:38 |
| chkumar|rover | not sure it is good | 12:38 |
| rlandy | interesting | 12:39 |
| rlandy | let's discuss at meeting | 12:39 |
| rlandy | this is going to be an ongoing issue | 12:39 |
| chkumar|rover | yup | 12:39 |
| *** ratailor has quit IRC | 12:49 | |
| *** ykarel_ is now known as ykarel | 12:54 | |
| rfolco | ping scrum | 13:00 |
| rfolco | arxcruz, rfolco, zbr, sshnaidm, rlandy, marios, ysandeep, bhagyashris, svyas, pojadhav, akahat, weshay, chandankumar | 13:00 |
| rfolco | ping scrum | 13:00 |
| *** rlandy is now known as rlandy|mtg | 13:01 | |
| marios|ruck | weshay|ruck: just killed my upshift vm fyi | 13:11 |
| weshay|ruck | thanks | 13:11 |
| weshay|ruck | rfolco, soniya sends regrets, she has a conflict | 13:13 |
| weshay|ruck | marios|ruck, https://review.opendev.org/#/c/742419/ think you can remove your -2 | 13:16 |
| marios|ruck | weshay|ruck: ack will revisit the series and rebase/as necssary after the call | 13:19 |
| marios|ruck | weshay|ruck: but is that bug fixed? its undercloud upgrade i don'thave any update on that one | 13:19 |
| marios|ruck | weshay|ruck: maybe you mean https://review.opendev.org/#/c/742418/8 | 13:20 |
| weshay|ruck | hrm.. maybe not https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-undercloud-upgrade-ussuri | 13:20 |
| weshay|ruck | I thought I reviewed this job this week and it looked good .. perhaps. not | 13:21 |
| marios|ruck | weshay|ruck: yeah we have an open bug/cix for that one... i think standalone is the next one closest to fixed | 13:31 |
| bogdando | any help with https://review.opendev.org/#/c/745096/1 please? | 13:35 |
| bogdando | ykarel: ^^ | 13:36 |
| rlandy|mtg | chkumar|rover: ok - and we'll have to decide of the patch is downsteam only or upstream rebased to downstream | 13:36 |
| *** TrevorV has joined #oooq | 13:36 | |
| chkumar|rover | rlandy|mtg: you mean these patches na https://review.opendev.org/#/q/topic:tcib/train+(status:open+OR+status:merged) | 13:39 |
| chkumar|rover | ? | 13:39 |
| chkumar|rover | 3 patches still waiting to get it merged | 13:39 |
| rlandy|mtg | chkumar|rover: no - the buildah version lock | 13:39 |
| rlandy|mtg | openstack/tripleo-ansible | 13:39 |
| chkumar|rover | rlandy|mtg: ok got it | 13:39 |
| chkumar|rover | rlandy|mtg: let;s create a upstream one | 13:40 |
| rlandy|mtg | chkumar|rover: I need a downstream one to test first | 13:40 |
| rlandy|mtg | I can abandon it afterwards if it works | 13:40 |
| chkumar|rover | rlandy|mtg: will i create a downstream one? | 13:40 |
| chkumar|rover | give me a min | 13:40 |
| rlandy|mtg | ? not sure what ^^ means | 13:40 |
| rlandy|mtg | oh you'll create it | 13:41 |
| rlandy|mtg | sure | 13:41 |
| *** ykarel_ has joined #oooq | 13:54 | |
| chkumar|rover | rlandy|mtg: https://code.engineering.redhat.com/gerrit/#/c/208286/ | 13:55 |
| chkumar|rover | rlandy|mtg: in the job var put tripleo_buildah_packages: ["<buildah version>"] and depends on | 13:56 |
| chkumar|rover | it will work | 13:56 |
| chkumar|rover | i think | 13:56 |
| rlandy|mtg | chkumar|rover++ | 13:56 |
| *** ykarel has quit IRC | 13:56 | |
| rlandy|mtg | small correction ... | 13:56 |
| * rlandy|mtg will update | 13:56 | |
| rlandy|mtg | thank you | 13:56 |
| *** rlandy|mtg is now known as rlandy | 13:56 | |
| rlandy | chkumar|rover: so ... I define _tripleo_buildah_packages to be the specific buildah package I want in my job vars? | 14:02 |
| * rlandy thinks so | 14:03 | |
| chkumar|rover | rlandy: yes | 14:03 |
| chkumar|rover | rlandy: or you can define tripleo_buildah_packaes | 14:03 |
| chkumar|rover | both will work | 14:03 |
| rlandy | let me try in the job | 14:03 |
| *** ysandeep is now known as ysandeep|brb | 14:19 | |
| *** ysandeep|brb is now known as ysandeep | 14:24 | |
| *** ykarel_ is now known as ykarel | 14:30 | |
| rlandy | ysandeep: meeting | 14:33 |
| ysandeep | rlandy, joining, sry i thought its after half an hour | 14:34 |
| chkumar|rover | rlandy: does it worked | 14:45 |
| chkumar|rover | ? | 14:45 |
| rlandy | chkumar|rover: still running | 14:45 |
| chkumar|rover | rlandy: cool, let me know if works or not | 14:46 |
| rlandy | weshay|ruck: ysandeep: https://review.opendev.org/#/c/744562 | 14:46 |
| rlandy | https://review.opendev.org/#/c/744166/ | 14:46 |
| rlandy | weshay|ruck: ysandeep: ^^ | 14:46 |
| rlandy | https://review.opendev.org/#/c/744786/8/roles/build-containers/templates/build.sh.j2 | 14:47 |
| weshay|ruck | ysandeep, you there? | 14:49 |
| ysandeep | weshay|ruck, connection lost.. reconnecting | 14:50 |
| *** skramaja_ has quit IRC | 14:53 | |
| weshay|ruck | chkumar|rover, marios|ruck should we consider moving all promotion jobs to rdo-cloud? | 14:55 |
| rlandy | chkumar|rover: :( ...no ... did I mess up? | 14:57 |
| rlandy | buildah.x86_64 1.14.9-1.module+el8.2.1+6689+748e6520 @osp-trunk-deps | 14:57 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/40/189440/81/check/periodic-tripleo-build-containers-ubi-8-internal-rhel-8-build-push-upload-rhos-16.2/30b556c/logs/system/installed_pkgs.txt | 14:57 |
| marios|ruck | weshay|ruck: well, not sure that would help yet. per discussion earlier, even with the train 8 jobs that are already switched to rdo, they are using eg registry that is in vexx | 14:58 |
| marios|ruck | weshay|ruck: per discussion with jpena i mean | 14:59 |
| rlandy | weshay|ruck: https://hackmd.io/I_CFKPXHTza-5i2kFDIs5Q | 14:59 |
| rlandy | ysandeep: ^^ | 14:59 |
| marios|ruck | weshay|ruck: so in the end they fail for the same things ie. like in https://bugs.launchpad.net/tripleo/+bug/1890571 we have mix of rdo/vexx | 14:59 |
| openstack | Launchpad bug 1890571 in tripleo "periodic integration/component jobs failing "[Zuul] Log Stream did not terminate"" [Critical,Triaged] | 14:59 |
| weshay|ruck | marios|ruck, ah ya.. dam | 14:59 |
| rlandy | chkumar|rover: I defined it in the job | 15:00 |
| weshay|ruck | chkumar|rover, sshnaidm can we chat for a minute | 15:00 |
| rlandy | maybe I need to change it in the patch | 15:00 |
| chkumar|rover | weshay|ruck: what about moving container registry | 15:00 |
| chkumar|rover | rlandy: let me check | 15:00 |
| sshnaidm | weshay|ruck, yeah | 15:00 |
| chkumar|rover | weshay|ruck: yes | 15:06 |
| weshay|ruck | k.. almost out of my current | 15:06 |
| chkumar|rover | rlandy: updated the tripleo-ansible patch | 15:09 |
| chkumar|rover | rlandy: and rechecked | 15:10 |
| chkumar|rover | let's see | 15:10 |
| rlandy | chkumar|rover: cool - thank you | 15:11 |
| weshay|ruck | rlandy, going to drop.. thanks | 15:13 |
| weshay|ruck | chkumar|rover, sshnaidm https://meet.google.com/izp-dztr-ocu | 15:13 |
| chkumar|rover | rlandy: drop me the notes here, will take a look tomorrow | 15:20 |
| chkumar|rover | see ya | 15:20 |
| *** chkumar|rover is now known as raukadah | 15:20 | |
| rlandy | raukadah: awesome thanks | 15:22 |
| *** jpena is now known as jpena|off | 15:38 | |
| weshay|ruck | marios|ruck, can you please respond to Sergii on opendev-discuss | 15:46 |
| rlandy | weshay|ruck: you addde the 16-2 namespace, right? | 15:46 |
| rlandy | I can't add new users | 15:46 |
| rlandy | to that namespace | 15:46 |
| rlandy | can you? | 15:46 |
| rlandy | hence the unauthorized error | 15:47 |
| weshay|ruck | ah | 15:47 |
| weshay|ruck | sec | 15:47 |
| marios|ruck | weshay|ruck: was getting ready to go but will try to quickly joining | 15:47 |
| marios|ruck | weshay|ruck: oh you mean mailing list | 15:47 |
| marios|ruck | checking | 15:47 |
| marios|ruck | weshay|ruck: well that sounds good to me anyway | 15:48 |
| weshay|ruck | rlandy, adding you now | 15:48 |
| rlandy | weshay|ruck: you me marios as admin | 15:48 |
| rlandy | weshay|ruck: tripleo-ci-registry-user as push, pull, admin | 15:48 |
| weshay|ruck | rlandy, you are now admin | 15:48 |
| weshay|ruck | ysandeep, as well ^ | 15:49 |
| rlandy | ok cool | 15:49 |
| rlandy | weshay|ruck: thanks - added the service user | 15:50 |
| *** ykarel is now known as ykarel|away | 15:50 | |
| marios|ruck | weshay|ruck: done | 15:51 |
| *** marios|ruck has quit IRC | 15:53 | |
| ykarel|away | bogdando, ack | 15:53 |
| rlandy | raukadah: ah - login is a config roe | 15:54 |
| rlandy | role | 15:54 |
| rlandy | doesn't take speculative testing | 15:54 |
| rlandy | won't read the depends-on | 15:54 |
| *** rlandy is now known as rlandy|brb | 15:55 | |
| weshay|ruck | arxcruz, have a minute? | 16:00 |
| *** udesale_ has quit IRC | 16:01 | |
| *** bogdando has quit IRC | 16:18 | |
| *** rlandy|brb is now known as rlandy | 16:22 | |
| rlandy | migi: need your help with registry tokens | 16:33 |
| weshay|ruck | rlandy, what's up? | 16:34 |
| rlandy | weshay|ruck: stil permission denied | 16:34 |
| rlandy | I think we have to go through https://docs.engineering.redhat.com/pages/viewpage.action?spaceKey=KB&title=PSI+Internal+Container+Registry+-+Getting+Started | 16:34 |
| weshay|ruck | rlandy, ah.. this only allows pull | 16:35 |
| * weshay|ruck looks | 16:35 | |
| rlandy | I set admin | 16:35 |
| weshay|ruck | rlandy, admin doesn't give it | 16:36 |
| weshay|ruck | there is a push permission | 16:36 |
| weshay|ruck | look at 17 | 16:36 |
| rlandy | weshay|ruck: I see | 16:36 |
| rlandy | weshay|ruck: how did we get push permissions on 17? | 16:37 |
| weshay|ruck | not sure | 16:37 |
| weshay|ruck | looking | 16:37 |
| weshay|ruck | rlandy, we have it now | 16:41 |
| rlandy | weshay|ruck: still confused | 16:47 |
| weshay|ruck | rlandy, does it work though? | 16:48 |
| rlandy | weshay|ruck: afaict - nothing changed | 16:49 |
| rlandy | did jason do something? | 16:49 |
| weshay|ruck | rlandy, can you rerun? | 16:49 |
| weshay|ruck | rlandy, no.. perhaps it's a policy inherit w/ the user | 16:50 |
| weshay|ruck | and it just took time to update | 16:50 |
| rlandy | if the service user does not pick up tripleorhos-16-2 it will fail imho | 16:50 |
| rlandy | [rlandy@localhost tripleo-ci-internal-config]$ oc describe serviceaccount tripleo-ci-registry-user | 16:50 |
| rlandy | Name: tripleo-ci-registry-user | 16:50 |
| rlandy | Namespace: tripleorhos-17 | 16:50 |
| *** derekh has quit IRC | 17:01 | |
| *** dtantsur is now known as dtantsur|afk | 17:03 | |
| *** ykarel|away has quit IRC | 18:11 | |
| holser | sshnaidm here? | 18:21 |
| sshnaidm | holser, kind of | 18:21 |
| holser | have you seen anything like https://paste.centos.org/view/5e514072 ? | 18:21 |
| holser | sshnaidm I found the issue in docker logs | 18:24 |
| sshnaidm | holser, it may be network problem with downloading repos, just track it down in zuul console | 18:24 |
| sshnaidm | holser, what is that? | 18:24 |
| holser | that message seems to be general | 18:24 |
| holser | https://paste.centos.org/view/9f359711 | 18:26 |
| holser | wrong key on opendev.org | 18:26 |
| sshnaidm | holser, do you have key with password? | 18:27 |
| holser | yeah | 18:30 |
| holser | with passphrase | 18:31 |
| holser | sshnaidm I am thinking we need to rewrite this part and use ssh-agent instead | 18:31 |
| sshnaidm | holser, yeah, but for now you can create a special user in gerrit with key w/o password | 18:32 |
| holser | yeah | 18:32 |
| holser | just have done it | 18:32 |
| rlandy | weshay|ruck: we're pushing containers to 16-2 | 18:41 |
| weshay|ruck | rlandy, whahoo! | 18:43 |
| weshay|ruck | rlandy, any more changes required? We should write this into lore | 18:44 |
| rlandy | weshay|ruck: well you can't always believe what read - in doc | 18:44 |
| rlandy | went on to psi channel | 18:45 |
| rlandy | we recreated the user | 18:45 |
| weshay|ruck | ah.. | 18:46 |
| weshay|ruck | rlandy++ | 18:46 |
| weshay|ruck | ya.. so please write that down in hackmd.. | 18:46 |
| rlandy | yep | 18:47 |
| rlandy | waiting fir it to complete | 18:47 |
| *** sshnaidm is now known as sshnaidm|afk | 18:47 | |
| *** tosky has quit IRC | 19:20 | |
| rfolco | a few patches more to come, but I migrated all the mol jobs with this one https://review.rdoproject.org/r/#/c/28857/ please review weshay|ruck rlandy | 19:44 |
| rlandy | k | 19:44 |
| weshay|ruck | rfolco, although this has been here forever probably.. I find it completely unnecessary to build an openstack container via tripleo | 19:49 |
| weshay|ruck | https://review.rdoproject.org/r/#/c/28857/8/molecule/tripleo_common_integration/converge.yml | 19:49 |
| weshay|ruck | rfolco, maybe I'm misreading it | 19:50 |
| weshay|ruck | but we could have just used a hello_world container | 19:50 |
| weshay|ruck | rfolco, not criticizing your code... just noting it | 19:51 |
| weshay|ruck | rfolco, ? | 19:53 |
| weshay|ruck | am I missing the point of that bit? | 19:53 |
| rfolco | looking | 19:53 |
| rfolco | weshay|ruck, we mock the j2 template we build a hello -like container but running the same workflow we do in a standalone deploy for example | 19:55 |
| rfolco | weshay|ruck, instead of building all default containers, we mock the j2 template and inject it into tripleo-common | 19:55 |
| weshay|ruck | rfolco, ya.. so was that to exercise x86_64 / ppc ? | 19:55 |
| weshay|ruck | rfolco, it's promoter tests.. | 19:56 |
| weshay|ruck | really we should just have a mock container... | 19:56 |
| weshay|ruck | not really mock.. but a single static container | 19:56 |
| weshay|ruck | we're poluting the promoter code w/ tripleo code | 19:56 |
| weshay|ruck | I like the test... but perhaps propose it against ansible-role-container-registry and tripleo-common | 19:57 |
| rfolco | weshay|ruck, its an integration test to check if our promoted containers can be used by openstack image prepare | 19:57 |
| weshay|ruck | the promoter should just care there is a container | 19:57 |
| weshay|ruck | that it can be tagged | 19:57 |
| rfolco | oh yes that was the original idea | 19:57 |
| weshay|ruck | that it's x86_64, or ppc and whether or not to create a manifest | 19:57 |
| rfolco | to run that job on tripleo-common too | 19:57 |
| weshay|ruck | ok | 19:58 |
| weshay|ruck | rfolco, don't forget :) because if that job breaks here.. on a registry or tc change | 19:58 |
| weshay|ruck | that would suck | 19:58 |
| rfolco | weshay|ruck, now that we migrated, we can modify as we wish | 19:58 |
| weshay|ruck | rfolco, what ever happened to our ppc guys... | 19:58 |
| rfolco | we'll have to review each step there and improve | 19:59 |
| weshay|ruck | rfolco, ya.. imho you want to break everything down to it's most simple form for the tests | 20:01 |
| weshay|ruck | and building a container via tripleo and pushing it into a local registry does not seem promoter related.. seems registry and tripleo-common. | 20:01 |
| rfolco | thats why its called tripleo-common-integration, this is the end-to-end integration test... the simple tests are the promote_images and container_push* ones | 20:02 |
| rfolco | weshay|ruck, this test was in a single file ... look https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/promoter/molecule/tripleo-common-integration/converge.yml | 20:03 |
| rfolco | we broke down into separate files, now its easier to modify it if we find better ways to test its integration w/ tripleo-common | 20:05 |
| rfolco | this test was born to check if our manifests could be consumed by deployment workflow in tripleo-common | 20:06 |
| rfolco | hope this makes sense still | 20:07 |
| weshay|ruck | rlandy, where were we having success w/ upshift and fs00$ | 20:18 |
| rlandy | weshay|ruck: https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-component-baremetal&job_name=periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-baremetal-rhos-17 | 20:19 |
| rlandy | note starts passing daily after we promoted | 20:19 |
| rlandy | periodic-tripleo-ci-rhel-8-bm_envA-3ctlr_1comp-featureset001-baremetal-rhos-17 is slightly less stable but also ok | 20:20 |
| rlandy | weshay|ruck: ^^ that what you are looking for? | 20:20 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=openstack-component-clients%20&job_name=periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-clients-rhos-17 | 20:21 |
| rlandy | weshay|ruck: ^^ same deal there | 20:21 |
| *** TrevorV has quit IRC | 20:47 | |
| weshay|ruck | rlandy, hey sorry have been chatting w/ morazi about hardware | 21:03 |
| * weshay|ruck looks | 21:03 | |
| weshay|ruck | rlandy, ah ya.. ok.. looks good :) | 21:03 |
| *** jfrancoa has quit IRC | 21:17 | |
| *** rlandy is now known as rlandy|bbl | 22:16 | |
| arxcruz | weshay|ruck: sorry, did not saw your message, what's up ? | 22:53 |
| *** saneax has quit IRC | 23:50 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!