| *** brault has quit IRC | 01:14 | |
| *** brault has joined #oooq | 01:27 | |
| *** jtomasek has quit IRC | 02:16 | |
| *** ykarel|away has joined #oooq | 02:32 | |
| *** ykarel|away has quit IRC | 02:38 | |
| *** sanjayu_ has joined #oooq | 02:42 | |
| *** brault has quit IRC | 02:53 | |
| *** brault has joined #oooq | 02:53 | |
| *** skramaja has quit IRC | 03:58 | |
| *** dsneddon has quit IRC | 04:01 | |
| *** dsneddon has joined #oooq | 04:06 | |
| *** dsneddon has quit IRC | 04:13 | |
| *** dsneddon has joined #oooq | 04:26 | |
| *** ykarel|away has joined #oooq | 04:27 | |
| *** surpatil has joined #oooq | 04:27 | |
| *** ykarel|away is now known as ykarel | 04:34 | |
| *** soniya29 has joined #oooq | 04:37 | |
| *** skramaja has joined #oooq | 04:48 | |
| *** skramaja has quit IRC | 04:49 | |
| *** skramaja has joined #oooq | 04:59 | |
| *** soniya29 has quit IRC | 05:02 | |
| *** udesale has joined #oooq | 05:12 | |
| *** soniya29 has joined #oooq | 05:16 | |
| *** dsneddon has quit IRC | 05:20 | |
| *** dsneddon has joined #oooq | 05:25 | |
| *** raukadah is now known as chandankumar | 05:27 | |
| *** dsneddon has quit IRC | 05:31 | |
| *** jtomasek has joined #oooq | 05:36 | |
| *** soniya29 has quit IRC | 05:40 | |
| *** ratailor has joined #oooq | 05:47 | |
| *** soniya29 has joined #oooq | 05:47 | |
| *** dsneddon has joined #oooq | 06:06 | |
| *** marios has joined #oooq | 06:13 | |
| *** marios is now known as marios|ruck|stil | 06:13 | |
| *** marios|ruck|stil is now known as marios|ruck | 06:13 | |
| *** ccamacho has quit IRC | 06:39 | |
| *** tesseract has joined #oooq | 06:45 | |
| *** epoojad1 has joined #oooq | 06:58 | |
| *** dsneddon has quit IRC | 07:08 | |
| *** jbadiapa has quit IRC | 07:10 | |
| *** jbadiapa has joined #oooq | 07:10 | |
| *** ratailor_ has joined #oooq | 07:13 | |
| *** ratailor has quit IRC | 07:15 | |
| *** dsneddon has joined #oooq | 07:30 | |
| *** surpatil has quit IRC | 07:31 | |
| *** dsneddon has quit IRC | 07:38 | |
| *** ratailor_ has quit IRC | 07:42 | |
| *** slaweq has joined #oooq | 07:55 | |
| *** slaweq has quit IRC | 07:57 | |
| *** kopecmartin|off is now known as kopecmartin | 08:01 | |
| *** ykarel is now known as ykarel|lunch | 08:02 | |
| *** dsneddon has joined #oooq | 08:04 | |
| *** amoralej|off is now known as amoralej | 08:04 | |
| *** soniya29 has quit IRC | 08:10 | |
| *** dsneddon has quit IRC | 08:12 | |
| *** soniya29 has joined #oooq | 08:34 | |
| *** jpena|off is now known as jpena | 08:36 | |
| *** zbr has quit IRC | 08:37 | |
| *** bogdando has joined #oooq | 08:41 | |
| *** dsneddon has joined #oooq | 08:44 | |
| *** surpatil has joined #oooq | 08:51 | |
| *** ccamacho has joined #oooq | 08:51 | |
| *** ratailor has joined #oooq | 08:52 | |
| *** soniya29 has quit IRC | 09:00 | |
| *** ccamacho has quit IRC | 09:09 | |
| *** brault has quit IRC | 09:10 | |
| *** brault has joined #oooq | 09:10 | |
| *** zbr has joined #oooq | 09:11 | |
| *** akahat has joined #oooq | 09:19 | |
| *** ykarel|lunch is now known as ykarel | 09:29 | |
| *** holser has joined #oooq | 09:30 | |
| *** brault has quit IRC | 09:33 | |
| *** dsneddon has quit IRC | 09:39 | |
| *** rascasoft has joined #oooq | 09:44 | |
| *** rascasoft has quit IRC | 09:44 | |
| *** rascasoft has joined #oooq | 09:45 | |
| *** surpatil has quit IRC | 09:48 | |
| chandankumar | arxcruz: zbr free in the evening for a meeting for ceph-ansible rpm passing decision? | 09:58 |
|---|---|---|
| *** holser has quit IRC | 09:58 | |
| chandankumar | around 5:30 IST? | 09:58 |
| *** holser has joined #oooq | 09:58 | |
| zbr | chandankumar: what decision is needed? bit outside the loop. | 10:05 |
| chandankumar | zbr: https://tree.taiga.io/project/tripleo-ci-board/us/1318?milestone=242589 | 10:06 |
| chandankumar | zbr: finalizing the design on how to pass rpm to the job | 10:06 |
| zbr | chandankumar: do we have a test zuul instance? i would like to play with something: a job that depends on two others, i want to see how it can access results from them. or if we already have an example. | 10:08 |
| *** epoojad1 is now known as epoojad1|afk | 10:08 | |
| chandankumar | arxcruz: has already done | 10:08 |
| chandankumar | zbr: https://review.rdoproject.org/r/#/c/23386/ | 10:09 |
| zbr | let me explain: what if two jobs return the same variables? are dictionaries merged or not? read https://zuul-ci.org/docs/zuul/user/jobs.html#return-values | 10:09 |
| zbr | in job1 returns foo: { "a": 1} and job2 returns foo: { "b": 2 } -- do I get a merged dictionary? | 10:09 |
| zbr | i guess docs explained it. last wins, no merge. | 10:10 |
| *** epoojad1|afk has quit IRC | 10:13 | |
| zbr | if zuul_returns would return: repos: { reponame: { url: .... }} you have my approval! | 10:13 |
| zbr | this would be extendable, so we can return more than one repository to inject in client, easy to parse. | 10:14 |
| arxcruz | zbr: good question, but is it required to be the same variable name? | 10:14 |
| zbr | arxcruz: what do you mean? | 10:14 |
| arxcruz | zbr: actually... | 10:15 |
| arxcruz | zbr: so, we are the ones who set the variable | 10:15 |
| zbr | arxcruz: exactly, and picking this should be future proof. | 10:15 |
| arxcruz | so we can check if the variable already exist, if so, append | 10:15 |
| zbr | hmm, didn't think about this but sounds great, we could chain them | 10:15 |
| zbr | and each child job could add its own repos. | 10:16 |
| zbr | if we need extra properties, like priority, we can extend the dictionary. | 10:16 |
| zbr | one thing that I do not know is: is the repo name/location just a variable or we should measure it as an artifact? | 10:18 |
| chandankumar | arxcruz: zbr Since each of the job have their dependencies and which will be unique so the returned repo will be unique na? | 10:18 |
| arxcruz | chandankumar: we are the responsible to set the variable | 10:18 |
| arxcruz | with the zuul_return module | 10:18 |
| arxcruz | so, we are the responsible if the variable will append or replace | 10:19 |
| zbr | arxcruz: post link to you current zuul_return line | 10:19 |
| arxcruz | zbr: https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/tripleo-ceph-integration/run.yaml#L17-L20 | 10:20 |
| *** dtantsur|afk is now known as dtantsur | 10:20 | |
| chandankumar | arxcruz: zbr does it will help https://pagure.io/zuul-distro-jobs/blob/master/f/roles/repo-fetch/tasks/main.yaml ? | 10:20 |
| zbr | i am inclined to like the use of artifacts dictionary but I have no idea how to parse it, how to distinguish between an artifact which is a yum repo, and one that is another byproduct | 10:21 |
| arxcruz | zbr: this is setting the zuul_return data | 10:21 |
| arxcruz | and this is consuming: | 10:21 |
| arxcruz | https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/run-rpm-build.yaml#L21-L26 | 10:21 |
| zbr | probably we can add "type: yumrepo" inside metadata? | 10:21 |
| zbr | clearly we will need to pick a meaningful repo name, like "ceph-ansible-cr" or ceph-ansible-override | 10:22 |
| zbr | cr from change request (in fact for it is likely to be pr) | 10:22 |
| arxcruz | yup | 10:22 |
| zbr | so maybe "override" wins as is not biased against CR vs PR :D | 10:23 |
| zbr | also add a metadata: { type: rpmrepo } | 10:27 |
| zbr | that will allow us to parse multiple artifacts and install all repos in the future (no need to do it now) | 10:27 |
| *** dsneddon has joined #oooq | 10:35 | |
| *** dsneddon has quit IRC | 10:40 | |
| chandankumar | zbr: arxcruz https://pagure.io/zuul-distro-jobs/pull-request/24 | 10:50 |
| *** mabox has joined #oooq | 10:59 | |
| *** mabox has quit IRC | 11:04 | |
| chandankumar | zbr: arxcruz pagure has one nice feature remote pull request https://docs.pagure.org/pagure/usage/pull_requests.html#remote-git-to-pagure-pull-request | 11:08 |
| *** surpatil has joined #oooq | 11:08 | |
| *** jfrancoa has joined #oooq | 11:09 | |
| *** ccamacho has joined #oooq | 11:30 | |
| arxcruz | chandankumar: so, you've been talking about this pagure, which i have no idea what is, and it wasn't in our design plan | 11:30 |
| arxcruz | honestly, i don't know how this connects with our workflow, if you can explain later on the meeting, would be nice | 11:31 |
| chandankumar | arxcruz: I am working on that to explain the stuff | 11:31 |
| arxcruz | chandankumar: ok | 11:31 |
| chandankumar | arxcruz: I found a bug https://github.com/rdo-infra/rdo-jobs/blob/master/playbooks/tripleo-ceph-integration/run.yaml#L7 | 11:32 |
| chandankumar | arxcruz: it needs to set based on distro | 11:32 |
| chandankumar | we are still hardcoding here | 11:32 |
| arxcruz | chandankumar: it's el8 because wes told us to work on el8 for ceph-ansible | 11:32 |
| arxcruz | we will only test on el8 | 11:32 |
| chandankumar | arxcruz: but for other branches we also need rocky and queens job which will run on centos7 | 11:33 |
| arxcruz | although we can set a variable | 11:33 |
| arxcruz | chandankumar: well, i asked him on the card, and he didn't reply | 11:33 |
| arxcruz | chandankumar: but sure, we can change :) | 11:33 |
| arxcruz | no big deal | 11:33 |
| arxcruz | as soon as we get the final decision for the other branches | 11:34 |
| *** ratailor has quit IRC | 11:34 | |
| *** dsneddon has joined #oooq | 11:36 | |
| chandankumar | arxcruz: zbr jpena let's see the fun here https://review.rdoproject.org/r/#/c/23537/ | 11:38 |
| chandankumar | jpena: Do I need to put zuul-distro jobs as required project? | 11:38 |
| jpena | chandankumar: hm, can't remember. I think we need to, but let's see CI | 11:40 |
| *** dsneddon has quit IRC | 11:41 | |
| chandankumar | jpena: http://logs.rdoproject.org/37/23537/1/check/tripleo-ceph-integration-master/fb7a070/job-output.txt | 11:42 |
| chandankumar | jpena: something is broken on other side | 11:42 |
| jpena | chandankumar: it's a mock issue, I think. | 11:42 |
| chandankumar | jpena: but the pagure distro zuul job worked | 11:42 |
| jpena | yes, it's failing when building the rpm | 11:43 |
| jpena | chandankumar: do you want to build the package for centos 7 or 8? | 11:44 |
| chandankumar | jpena: we want to build the package for both | 11:44 |
| chandankumar | as for stable/3.x branches, we will be running queens or rocky | 11:45 |
| jpena | chandankumar: to fix this, we have a few options. We can set "config_opts['use_bootstrap_container'] = True" in /etc/mock/site-defaults.cfg, or use a centos 8 image for the job instead of centos 7 | 11:49 |
| chandankumar | arxcruz: ^^ | 11:50 |
| arxcruz | chandankumar: we are running it on rhel8 no ? | 11:51 |
| arxcruz | i don't know | 11:51 |
| arxcruz | chandankumar: i don't think it's usefull you implement something before the meeting | 11:51 |
| arxcruz | let's evaluate first, than we check what we need to do | 11:51 |
| arxcruz | otherwise, there's no point for us to meet today | 11:52 |
| jpena | arxcruz: you have "label: rdo-centos-7" in https://review.rdoproject.org/r/#/c/23537/1/zuul.d/ceph-ansible.yaml | 11:52 |
| chandankumar | arxcruz: ok | 11:52 |
| arxcruz | jpena: yeah, the rpm build i though would be any distro and the rpm will built depending of the mock argument | 11:52 |
| arxcruz | mock and distro argument | 11:52 |
| arxcruz | if that's not the case, we can use rhel8, but this is something to discuss before start implement stuff :) | 11:53 |
| jpena | yes, but building for centos/rhel 8 from centos 7 has some quirks | 11:53 |
| jpena | I would suggest to set the option in site-defaults.cfg | 11:53 |
| arxcruz | jpena: so, the best is build directly on rhel8 right ? | 11:53 |
| jpena | arxcruz: that gives you best compatibility, from there you can built for both rhel7 and rhel 8 | 11:54 |
| jpena | buuuut keep in mind that you need to enable EPEL to install mock there | 11:54 |
| *** marios|ruck has quit IRC | 11:55 | |
| arxcruz | chandankumar: did you send the invitation for the meeting? | 11:56 |
| arxcruz | i don't know which time is 5:30 pm ist | 11:56 |
| chandankumar | arxcruz: or we want to do after planning as I am waiting rlandy also | 11:57 |
| arxcruz | chandankumar: up to you | 12:00 |
| *** arxcruz is now known as arxcruz|lunch | 12:01 | |
| chandankumar | arxcruz|lunch: zbr invite sent | 12:02 |
| chandankumar | zbr: sorry let me shift the meeting after planning may be 3 hours later so that rlandy can also join | 12:07 |
| *** rfolco has joined #oooq | 12:11 | |
| *** udesale has quit IRC | 12:14 | |
| *** jpena is now known as jpena|lunch | 12:14 | |
| *** udesale has joined #oooq | 12:14 | |
| weshay|rover | zbr, ykarel looks like the results for https://review.opendev.org/#/c/692316/ match the results https://review.opendev.org/#/c/692223/ w/ the deploy in queens failing on ntp | 12:14 |
| weshay|rover | I don't care which solution is used | 12:14 |
| *** dsneddon has joined #oooq | 12:15 | |
| ykarel | weshay|rover, where u checked results for https://review.opendev.org/#/c/692316/ | 12:15 |
| ykarel | job finished already? | 12:16 |
| * ykarel looks | 12:16 | |
| ykarel | hmm it failed https://logs.rdoproject.org/16/692316/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/9eb4641/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 12:16 |
| ykarel | possibly that var is not applied for some reason | 12:17 |
| *** rlandy has joined #oooq | 12:17 | |
| ykarel | possible due to the place where it's defined | 12:17 |
| ykarel | hmm, the task is skipped | 12:19 |
| ykarel | TASK [ansible-role-container-registry : Disable docker iptables] | 12:19 |
| ykarel | so var is not applied | 12:19 |
| *** dsneddon has quit IRC | 12:19 | |
| weshay|rover | was applied on http://logs.rdoproject.org/23/692223/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/ebcac66/logs/undercloud/etc/sysconfig/ | 12:20 |
| ykarel | but ^^ applied too late in quickstart run | 12:20 |
| ykarel | we need it before the issue get's introuduced | 12:21 |
| ykarel | at the time of container-login playbook | 12:21 |
| weshay|rover | ykarel, before the overcloud deploy no? | 12:21 |
| weshay|rover | when do we need it? | 12:21 |
| ykarel | weshay|rover, https://github.com/rdo-infra/review.rdoproject.org-config/blob/master/playbooks/tripleo-rdo-base/container-login.yaml#L7 | 12:21 |
| weshay|rover | ykarel, so at registry create time eh? | 12:21 |
| ykarel | weshay|rover, yes | 12:22 |
| weshay|rover | aight | 12:22 |
| ykarel | fix needs to be applied before first start of docker | 12:22 |
| ykarel | and that is happening with that playbook in the job | 12:22 |
| weshay|rover | ykarel, ok.. we could read the env var.. and get zuul_branch or zuul_branch_override and set it directly in the vars in the role | 12:23 |
| ykarel | weshay|rover, imo u can set it for all releases, it should not hurt | 12:23 |
| weshay|rover | ykarel, ya.. we could try that as well | 12:23 |
| weshay|rover | zbr, ^ you with us? | 12:24 |
| weshay|rover | ykarel, it looked to me like it applied in your patch | 12:24 |
| ykarel | weshay|rover, where u see? | 12:25 |
| ykarel | link | 12:25 |
| weshay|rover | ykarel, http://logs.rdoproject.org/16/692316/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/9eb4641/logs/undercloud/etc/sysconfig/docker-network.gz | 12:25 |
| ykarel | i see blank file ^^ | 12:26 |
| ykarel | without any iptable setting | 12:26 |
| weshay|rover | really? | 12:30 |
| weshay|rover | DOCKER_NETWORK_OPTIONS=--iptables=false | 12:30 |
| weshay|rover | # /etc/sysconfig/docker-network | 12:30 |
| weshay|rover | ykarel, ^ | 12:31 |
| ykarel | me look again :) | 12:31 |
| ykarel | weshay|rover, i still see th same:- just # /etc/sysconfig/docker-network | 12:32 |
| ykarel | no iptable setting there | 12:32 |
| weshay|rover | here? http://logs.rdoproject.org/16/692316/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/9eb4641/logs/undercloud/etc/sysconfig/docker-network.gz | 12:37 |
| weshay|rover | how can we be seeing two diff things | 12:37 |
| ykarel | same as before for me, no iptables setting | 12:38 |
| ykarel | may be u have some cache | 12:38 |
| rlandy | chandankumar: hi - I changed the wrong var for os_tempest ... " BadRequestException: 400: Client Error for url: https://10.9.120.218:13696/v2.0/subnets.json, Invalid input for allocation_pools. Reason: 'False' is not a valid IP address." | 12:38 |
| chandankumar | rlandy: sorry I went to hibernation mode last night | 12:38 |
| rlandy | np | 12:38 |
| weshay|rover | zbr, what do you see in http://logs.rdoproject.org/16/692316/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/9eb4641/logs/undercloud/etc/sysconfig/docker-network.gz | 12:38 |
| chandankumar | rlandy: is the system on hold? | 12:38 |
| rlandy | the var I changed was the actual tempest network - so that one can be arbitrary | 12:39 |
| ykarel | weshay|rover, could u try downloading, wget http://logs.rdoproject.org/16/692316/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/9eb4641/logs/undercloud/etc/sysconfig/docker-network.gz | 12:39 |
| ykarel | for me with both browser and wget, same result | 12:39 |
| rlandy | the one that needs to be set is the router network | 12:39 |
| * rlandy sees if the box is available | 12:39 | |
| weshay|rover | SOB.. ya.. not there | 12:39 |
| chandankumar | rlandy: it just expects the cidr | 12:40 |
| rlandy | chandankumar: yes - and that worked before and failed on the router ping | 12:40 |
| rlandy | now it's failing earlier' | 12:40 |
| rlandy | on the subnet | 12:40 |
| rlandy | there are allocation pools | 12:40 |
| * rlandy gets | 12:40 | |
| rlandy | the error is correct | 12:40 |
| weshay|rover | ykarel, zbr it is here though http://logs.rdoproject.org/23/692223/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-queens-branch/ebcac66/logs/undercloud/etc/sysconfig/docker-network.gz | 12:41 |
| rlandy | https://github.com/openstack/openstack-ansible-os_tempest/blob/ad4841f70a0ad4ed8090fc3f4c83812204cb59c0/tasks/tempest_resources.yml#L215 | 12:41 |
| rlandy | cidr: "{{ tempest_public_subnet_cidr }}" | 12:41 |
| rlandy | now I have changed that value | 12:42 |
| ykarel | weshay|rover, yes it's there but that's done late with quickstart | 12:42 |
| rlandy | which is not right | 12:42 |
| ykarel | we need it before | 12:42 |
| rlandy | getting box access - sec | 12:42 |
| weshay|rover | chandankumar, ok.. this is fun.. apparently we'll be getting centos8 packages in two weeks | 12:42 |
| chandankumar | rlandy: https://github.com/openstack/tripleo-quickstart-extras/blob/master/playbooks/tempest.yml#L31 | 12:42 |
| *** marios has joined #oooq | 12:42 | |
| *** marios is now known as marios|ruck | 12:42 | |
| rlandy | ack | 12:42 |
| weshay|rover | so next sprint centos8, replace stein, train, master w/ centos-8 | 12:42 |
| rlandy | wrong change | 12:42 |
| chandankumar | weshay|rover: so we will start working on that nextsprint? | 12:42 |
| rlandy | getting | 12:42 |
| chandankumar | rlandy: weshay|rover did you saw the magic here https://pagure.io/zuul-distro-jobs/pull-request/24 and reused here https://review.rdoproject.org/r/#/c/23537/ | 12:43 |
| marios|ruck | chandankumar: o/ hey i just saw your invite. but you sent the invite after the call started... i was afk | 12:44 |
| chandankumar | it was what we wanted to go ahead | 12:44 |
| marios|ruck | chandankumar: if you want to schedule a call better to give more notice than -1 minute ;) | 12:44 |
| weshay|rover | chandankumar, actually.. I'll have to see what Alan is building.. but in theory we should replace stein, train, master but we may just have master | 12:44 |
| weshay|rover | don't know will check | 12:44 |
| chandankumar | marios|ruck: I rescheduled it | 12:44 |
| marios|ruck | chandankumar: ack | 12:45 |
| chandankumar | weshay|rover: ok on centos8 but I was talkin about zuul-distro-job usage | 12:45 |
| marios|ruck | weshay|rover: we must set some time aside to sort out the promoter too this sprint | 12:45 |
| weshay|rover | marios|ruck, ya.. the promoter is in | 12:46 |
| weshay|rover | this sprint | 12:46 |
| *** marios|ruck has quit IRC | 12:47 | |
| weshay|rover | zuul-distro-job? | 12:47 |
| weshay|rover | I'm not in context | 12:47 |
| *** marios has joined #oooq | 12:47 | |
| rlandy | chandankumar: box is available - pm'ing you access | 12:47 |
| *** marios is now known as marios|ruck | 12:47 | |
| marios|ruck | weshay|rover: ack | 12:47 |
| *** ykarel is now known as ykarel|afk | 12:48 | |
| marios|ruck | scrum time? or am i on the wrong call | 13:01 |
| *** Goneri has joined #oooq | 13:01 | |
| rlandy | what time is the meeting today? | 13:01 |
| marios|ruck | rfolco: o/ | 13:02 |
| marios|ruck | scrum | 13:02 |
| chandankumar | when we moved the meeting to 1 hr later? | 13:02 |
| rfolco | marios|ruck, o/ | 13:02 |
| marios|ruck | me and rlandy are at https://maps.google.com/maps?hl=en-GB&q=meet.google.com%2Foiv-geho-mai&source=calendar | 13:02 |
| rfolco | chandankumar, not me | 13:02 |
| marios|ruck | got that url is awful | 13:02 |
| rfolco | wait | 13:03 |
| rfolco | weshay|rover, did you move our planning to 1h from now ? | 13:03 |
| rfolco | I didn't .... | 13:03 |
| weshay|rover | rfolco, I did not | 13:04 |
| weshay|rover | but I'm ready | 13:04 |
| rfolco | weird | 13:04 |
| *** jpena|lunch is now known as jpena | 13:05 | |
| rfolco | weshay|rover, nobody is there at https://meet.google.com/tbr-rgjz-cjc | 13:05 |
| chandankumar | where we should join? | 13:06 |
| marios|ruck | rfolco: we were on the other one | 13:06 |
| *** ykarel|afk is now known as ykarel | 13:06 | |
| weshay|rover | zbr, chandankumar marios|ruck rlandy panda sshnaidm anybody mind if we start early? | 13:06 |
| rlandy | we just dropped | 13:06 |
| marios|ruck | rfolco: there are 2 events in the calendar | 13:06 |
| marios|ruck | rfolco: there was one now | 13:06 |
| marios|ruck | rfolco: and another one in 1 hour | 13:06 |
| marios|ruck | rfolco: you joined the one in an hour | 13:06 |
| rlandy | confusion!!! | 13:06 |
| marios|ruck | rfolco: we joined the one now | 13:06 |
| marios|ruck | WHAT IS GOING ON | 13:06 |
| rlandy | better if we start now | 13:06 |
| marios|ruck | :D | 13:06 |
| rfolco | I apologize my fault | 13:06 |
| rlandy | ok - true or false - we are meeting now??? | 13:07 |
| marios|ruck | rfolco: so should we join the other call | 13:07 |
| marios|ruck | weshay|rover: where? | 13:07 |
| rlandy | which other call? | 13:07 |
| marios|ruck | weshay|rover: are you in scrum now? | 13:07 |
| marios|ruck | rfolco: tell us where to join please | 13:07 |
| weshay|rover | https://meet.google.com/tbr-rgjz-cjc?authuser=1 | 13:07 |
| rfolco | marios|ruck, rlandy sshnaidm panda arxcruz|lunch chandankumar ^ | 13:07 |
| *** arxcruz|lunch is now known as arxcruz | 13:09 | |
| rfolco | arxcruz, sshnaidm please join https://meet.google.com/tbr-rgjz-cjc | 13:09 |
| rfolco | zbr, ^ | 13:09 |
| sshnaidm | rfolco, now?? | 13:09 |
| arxcruz | sorry, summer time ends here | 13:10 |
| rfolco | yes not sure why calendar is wrong | 13:10 |
| *** dsneddon has joined #oooq | 13:15 | |
| *** panda is now known as panda|pto | 13:16 | |
| *** skramaja has quit IRC | 13:19 | |
| *** marios|ruck is now known as marios|call | 13:22 | |
| sshnaidm | rlandy, mind be a ruck firstly? | 13:23 |
| rlandy | sshnaidm: sure | 13:23 |
| *** sshnaidm is now known as sshnaidm|rover | 13:23 | |
| *** akahat has quit IRC | 13:26 | |
| zbr | what is th emeeting id? | 13:27 |
| zbr | on that one i am alone | 13:27 |
| weshay|rover | zbr, https://meet.google.com/tbr-rgjz-cjc?authuser=1 :) | 13:27 |
| *** dtantsur is now known as dtantsur|brb | 13:28 | |
| *** jaosorior has joined #oooq | 13:30 | |
| *** matbu has joined #oooq | 13:45 | |
| *** soniya29 has joined #oooq | 13:59 | |
| *** amoralej is now known as amoralej|lunch | 14:12 | |
| *** dsneddon has quit IRC | 14:19 | |
| *** ykarel is now known as ykarel|away | 14:27 | |
| rfolco | chandankumar, I'll move unfinished work to sprint 18, can I? | 14:35 |
| chandankumar | rfolco: yes | 14:35 |
| *** sanjayu_ has quit IRC | 14:37 | |
| *** dtantsur|brb is now known as dtantsur | 14:40 | |
| *** TrevorV has joined #oooq | 14:42 | |
| *** amoralej|lunch is now known as amoralej | 14:49 | |
| *** dsneddon has joined #oooq | 15:00 | |
| *** soniya29 has quit IRC | 15:06 | |
| *** Trevor_V has joined #oooq | 15:07 | |
| *** surpatil has quit IRC | 15:08 | |
| *** TrevorV has quit IRC | 15:10 | |
| *** bogdando has quit IRC | 15:26 | |
| marios|call | rfolco: rlandy sshnaidm|rover sync in 15 mins ok? | 15:31 |
| *** rlandy is now known as rlandy|ruck | 15:32 | |
| rlandy|ruck | marios|call: ack | 15:32 |
| *** jtomasek has quit IRC | 15:44 | |
| marios|call | rlandy|ruck: rfolco sshnaidm|rover meet on calednar invite | 15:45 |
| sshnaidm|rover | marios|call, 2 mins pls | 15:46 |
| *** jtomasek has joined #oooq | 15:47 | |
| weshay|rover | marios|call, can I jump in? | 15:54 |
| rfolco | marios|call, where? | 15:58 |
| rfolco | I'm coming | 15:58 |
| rfolco | rlandy|ruck, sshnaidm|rover marios|call which channel | 15:59 |
| rlandy|ruck | rfolco: https://meet.google.com/dps-fkdh-izm | 16:00 |
| *** dsneddon has quit IRC | 16:04 | |
| weshay|rover | rlandy|ruck, sshnaidm|rover https://review.rdoproject.org/r/#/c/22878/ | 16:06 |
| *** aakarsh has joined #oooq | 16:08 | |
| *** kopecmartin is now known as kopecmartin|off | 16:10 | |
| *** marios|call is now known as marios | 16:21 | |
| sshnaidm|rover | weshay|rover, do you cover me tomorrow? or anybody else | 16:21 |
| marios | noooooooooooooooooo /me runs screaming | 16:23 |
| rlandy|ruck | sshnaidm|rover: marios: https://review.rdoproject.org/r/#/c/23539/ Update link to ruckrover etherpad for sprint 18 | 16:23 |
| rlandy|ruck | minimal review here | 16:24 |
| marios | rlandy|ruck: ack | 16:24 |
| rlandy|ruck | thank you | 16:24 |
| rlandy|ruck | rfolco: can you help fill in the labels/fields in https://trello.com/c/R3PnXvxQ/1202-cixlp1850549tripleociproa-container-build-raises-wrong-exception | 16:26 |
| rfolco | rlandy|ruck, this is a bug in tripleo-common that builds containers in buildah.py. I am not familiar with these labels there... let me check | 16:28 |
| rfolco | rlandy|ruck, DFG? not sure... its tripleo-common | 16:29 |
| rfolco | rlandy|ruck, us? | 16:29 |
| rfolco | since I am fixing I assigned to us | 16:29 |
| weshay|rover | sshnaidm|rover, I got you | 16:30 |
| rlandy|ruck | deplyoment framework? | 16:30 |
| rlandy|ruck | weshay|rover: ^^? | 16:30 |
| rfolco | I set as oooci | 16:30 |
| *** weshay|rover is now known as weshay | 16:30 | |
| rlandy|ruck | tripleo-common belongs to which dfg | 16:30 |
| weshay | rlandy|ruck, sagi is out on fridays | 16:30 |
| weshay | rlandy|ruck, DF | 16:30 |
| rlandy|ruck | yay | 16:30 |
| rlandy|ruck | good guess | 16:30 |
| rlandy|ruck | weshay; yep - I know | 16:31 |
| rlandy|ruck | I am on as ruck first | 16:31 |
| rlandy|ruck | so no issue | 16:31 |
| rlandy|ruck | weshay: the one day that will be a problem is Nov 15 | 16:31 |
| rlandy|ruck | I am out half day there | 16:31 |
| rlandy|ruck | other than that, I will cover fridays | 16:31 |
| weshay | rlandy|ruck, ok.. I'll cover | 16:31 |
| *** tesseract has quit IRC | 16:33 | |
| sshnaidm|rover | zbr, I think need to fix ansible-lint issue on rdo-infra/ansible-role-tripleo-ci-reproducer repo too | 16:33 |
| rlandy|ruck | weshay: one other question on CIX ... there is only a train label ( and master - do not use) | 16:33 |
| rlandy|ruck | what are master bugs now? | 16:33 |
| zbr | sshnaidm|rover: i already found a fix, i was testing it now with few repos. | 16:33 |
| weshay | rlandy|ruck, ya.. I'll add a ussuri label | 16:34 |
| weshay | master = ussuri | 16:34 |
| sshnaidm|rover | zbr, ack, just don't forget about reproducer repo | 16:34 |
| rlandy|ruck | weshay: ack - ok -thanks | 16:34 |
| rlandy|ruck | marios: one last question before you run - rocky promotions | 16:36 |
| rlandy|ruck | 8 days out - no issue | 16:36 |
| marios | rlandy|ruck: well part of that is the wednesday/weekend thing ... so transient job fails become bigger problem | 16:37 |
| zbr | sshnaidm|rover: watch https://github.com/ansible/ansible-lint/pull/612 and comment on it, there is also a hint on how to test it on a repo before it goes in. | 16:37 |
| marios | rlandy|ruck: but also last week mostly been chasing the train/master so didn't look too long there to be honest | 16:37 |
| rlandy|ruck | marios: ok, np - will look at that | 16:38 |
| marios | rlandy|ruck: you might want to consider changing/adding another weekday into that pipeline maybe | 16:38 |
| marios | rlandy|ruck: to get tue/thu/weekend or something | 16:38 |
| * rlandy|ruck looks at what failed | 16:39 | |
| marios | rlandy|ruck: https://trello.com/c/DvgaBFim/1104-cixlp1843259tripleociproa-periodic-rocky-fs020-job-fails-tempest-tests-tempestscenariotestsecuritygroupsbasicopstestsecuritygrou | 16:39 |
| marios | rlandy|ruck: that is one rocky thing i commented on this sprint | 16:40 |
| marios | rlandy|ruck: https://bugs.launchpad.net/tripleo/+bug/1843259 maybe thats the blocker | 16:40 |
| openstack | Launchpad bug 1843259 in tripleo "Periodic rocky fs020 job fails tempest tests tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_cross_tenant_traffic and tempest.scenario.test_security_groups_basic_ops.TestSecurityGroupsBasicOps.test_multiple_security_groups " [Critical,Fix released] - Assigned to Terry Wilson (otherwiseguy) | 16:40 |
| rfolco | sshnaidm|rover, rlandy|ruck: the (old/current) promoter has rhel8 disabled, and continuous delivery (ansible pull) disabled as well - be aware | 16:40 |
| rlandy|ruck | that card was moved to done | 16:40 |
| rlandy|ruck | rfolco: why is rhel 8 disabled? | 16:41 |
| rfolco | sshnaidm|rover, rlandy|ruck I am trying a new rhel8 promotion in the new promoter so we don't need to stop old one to test the new one | 16:41 |
| marios | rlandy|ruck: (looking at my notes, that was last monday its the only rocky related thing i could find there) | 16:41 |
| * marios shutdown sequence | 16:41 | |
| rfolco | rlandy|ruck, we need to fix the new promoter code that is broken. So we pick rhel8 and try to promote using the new promoter server | 16:42 |
| rfolco | marios, weshay fyi I'm picking new promoter task https://tree.taiga.io/project/tripleo-ci-board/task/1369?kanban-status=1447275 | 16:42 |
| rfolco | chandankumar, ^ | 16:42 |
| rlandy|ruck | rfolco: rhel 8 on train and master | 16:42 |
| rfolco | rlandy|ruck, master | 16:42 |
| marios | rfolco: ack. did you pack for honolulu yet? /me super excited | 16:42 |
| rlandy|ruck | train rhel8 is still going> | 16:42 |
| rfolco | marios, submit the reimbursement first, I'll reach you | 16:43 |
| chandankumar | marios: post the pictures | 16:43 |
| chandankumar | rfolco: reading the bug report | 16:43 |
| rfolco | sshnaidm|rover, rlandy|ruck chandankumar: the new promoter is failing on manifest push auth issue... investigating | 16:44 |
| chandankumar | rfolco: ok | 16:44 |
| rfolco | I am just saying coz now its a sprint thing | 16:44 |
| sshnaidm|rover | rfolco, did it work once? | 16:44 |
| *** dtantsur is now known as dtantsur|afk | 16:45 | |
| rfolco | sshnaidm|rover, not yet, we push containers ok, but manifest push is failing on auth issues not sure why | 16:45 |
| rlandy|ruck | rfolco: I am pretending the new promoter does not exist atm | 16:45 |
| sshnaidm|rover | rfolco, ack, so it's not ruck/rover task for now | 16:45 |
| chandankumar | rfolco: for rhel8, we are using docker there? | 16:45 |
| chandankumar | for manifest | 16:45 |
| rfolco | rlandy|ruck, fair :) | 16:46 |
| rfolco | sshnaidm|rover, yep | 16:46 |
| rlandy|ruck | sshnaidm|rover: not our circus, not our monkeys | 16:46 |
| *** marios is now known as marios|out | 16:46 | |
| rfolco | chandankumar, podman | 16:46 |
| rfolco | oh oh | 16:46 |
| chandankumar | podman does not have manifest support | 16:46 |
| rfolco | chandankumar, I think you found why | 16:46 |
| chandankumar | podman docker another big mess | 16:47 |
| rfolco | chandankumar, I'll double check the workflow and commands used | 16:47 |
| chandankumar | rfolco: ok, cool! | 16:47 |
| sshnaidm|rover | rlandy|ruck, totally :) | 16:50 |
| *** holser has quit IRC | 16:57 | |
| *** jfrancoa has quit IRC | 16:59 | |
| *** marios|out has quit IRC | 17:00 | |
| *** dsneddon has joined #oooq | 17:00 | |
| rfolco | rlandy|ruck, sshnaidm|rover and don't touch our rats on your cages coz I am making experiments w/ them | 17:02 |
| sshnaidm|rover | rfolco, experiments on rats? I'll call greenpeace on you | 17:05 |
| rfolco | sssshhhhh | 17:05 |
| chandankumar | weshay: sshnaidm|rover rfolco rlandy|ruck https://tree.taiga.io/project/tripleo-ci-board/us/1323 we get answers related to zuul distro jobs | 17:06 |
| chandankumar | weshay: anything I missed please add there | 17:07 |
| sshnaidm|rover | chandankumar, I think we can formulate it as "we need to be sure we have an option to merge a fix any time w/o asking or waiting for somebody", so we need to be cores on these repos or sort of | 17:09 |
| chandankumar | sshnaidm|rover: yup | 17:09 |
| chandankumar | now everyone needs a fas account | 17:10 |
| sshnaidm|rover | chandankumar, we had already a lot of breakages from other teams when we started with rdo cloud, wasting hours for finding an issue.. and I wouldn't like to get back to it | 17:10 |
| chandankumar | sshnaidm|rover: we need to sort out these concerns | 17:10 |
| *** ykarel|away has quit IRC | 17:11 | |
| *** udesale has quit IRC | 17:11 | |
| rfolco | rlandy|ruck, chandankumar is branching work done ? | 17:14 |
| rlandy|ruck | rfolco: other than rhel8 | 17:15 |
| rlandy|ruck | still need to sort that out | 17:16 |
| rfolco | rlandy|ruck, ok thx :) | 17:16 |
| chandankumar | rfolco: yup, all from our side | 17:16 |
| chandankumar | tht is also branched | 17:16 |
| rlandy|ruck | rfolco: I did not add check jobs - I think chandankumar did some | 17:16 |
| * rlandy|ruck just checking rocky - will review missing train next | 17:17 | |
| chandankumar | rlandy|ruck: rfolco https://review.opendev.org/687004 | 17:17 |
| rfolco | zbr, what about centos8 node? done? what is the latest on that? | 17:17 |
| chandankumar | rlandy|ruck: we need to work with upgrade team to make sure train job works sorry weshay forgot to add the user story adding now | 17:17 |
| rfolco | sorry guys, I've been heads down on ruckering last 3 weeks, out of loop on some of these work | 17:18 |
| rlandy|ruck | chandankumar: is that sprint or ruck/rover stuff? | 17:18 |
| rfolco | according to my sprint summary this was in plan for last sprint - close-out branching stuff | 17:18 |
| chandankumar | rfolco: it was a part of tech debt mid cycle | 17:20 |
| rfolco | k | 17:20 |
| chandankumar | weshay: rfolco rlandy|ruck us added https://tree.taiga.io/project/tripleo-ci-board/us/1370 do we want to include in this sprint? | 17:20 |
| chandankumar | rfolco: donot know, may be weshay can confirm | 17:21 |
| chandankumar | sorry it is for rlandy|ruck ^^ | 17:21 |
| rlandy|ruck | rfolco: can we just renick you to folco? | 17:22 |
| weshay | chandankumar, I'm chatting w/ mike about that | 17:22 |
| rlandy|ruck | you;d have your own, unique first letter | 17:22 |
| *** chandankumar is now known as alaadin | 17:22 | |
| rlandy|ruck | lol | 17:22 |
| *** alaadin is now known as raukadah | 17:23 | |
| rfolco | rlandy|ruck, folco is not registered, I could not register last time, will try again this cycle | 17:23 |
| rfolco | rlandy|ruck, create a retro card for that | 17:23 |
| rfolco | lol | 17:23 |
| raukadah | rlandy|ruck: rfolco we can add one rule no more names with r | 17:24 |
| rlandy|ruck | raukadah is outlawed | 17:24 |
| rfolco | you guys are lazy, its just one letter | 17:24 |
| raukadah | weshay: ok | 17:24 |
| rlandy|ruck | ok - getting back to rucking | 17:24 |
| *** holser has joined #oooq | 17:24 | |
| *** holser has quit IRC | 17:24 | |
| *** jpena is now known as jpena|off | 17:34 | |
| weshay | rlandy|ruck, sshnaidm|rover ussuri labels added to cix board | 17:38 |
| weshay | master == ussuri | 17:38 |
| sshnaidm|rover | weshay, ack | 17:39 |
| rlandy|ruck | weshay: thanks | 17:39 |
| rlandy|ruck | 2019-10-31 17:04:33,262 30610 INFO promoter Promoting the container images for dlrn hash b52b560fb05102312361a1ce47d71a5de5c19e94 on rocky to current-tripleo | 17:40 |
| rlandy|ruck | woohoo | 17:40 |
| rlandy|ruck | rocky is promoting now | 17:40 |
| rlandy|ruck | sshnaidm|rover: ^^ fyi - so we're good on promotions atm | 17:42 |
| sshnaidm|rover | rlandy|ruck, great! | 17:42 |
| sshnaidm|rover | rlandy|ruck, let's continue this way.. | 17:42 |
| rlandy|ruck | seeing if we can master to run | 17:44 |
| sshnaidm|rover | I hope next week people will be very busy with peking duck and will be calm | 17:44 |
| zbr | sshnaidm|rover: hopefully, https://review.opendev.org/#/c/692418/ will fix it, so you will know how to address it (key is the hook config part) | 17:44 |
| rlandy|ruck | sshnaidm|rover: maybe we should send a few more people to summit - chosen by percentage of code broken in the last sprint | 17:46 |
| raukadah | sshnaidm|rover: rlandy|ruck I will try to collect duck for you | 17:47 |
| rlandy|ruck | raukadah++ | 17:48 |
| sshnaidm|rover | rlandy|ruck, yeah, raukadah anyway doesnt' break anything.. | 17:48 |
| raukadah | but in return, we need a big party at sshnaidm|rover cafe | 17:48 |
| sshnaidm|rover | rlandy|ruck, maybe to not allow him to go | 17:48 |
| sshnaidm|rover | raukadah, which cafe?? | 17:48 |
| raukadah | sshnaidm|rover: your's beer cafe if I remember correctly | 17:49 |
| raukadah | may be weshay can correct ^^ | 17:49 |
| sshnaidm|rover | raukadah, if so, I'm only for it) | 17:49 |
| *** epoojad1 has joined #oooq | 17:50 | |
| weshay | I'm certainly going to ptg only to not break things here | 17:50 |
| weshay | :) | 17:50 |
| raukadah | but we encourage people to break more :-) | 17:51 |
| sshnaidm|rover | zbr, so what is with https://review.opendev.org/#/c/692356/ ? | 17:51 |
| sshnaidm|rover | zbr, sova tests pass there, but linters fails of course,maybe leave just sova test there for now? | 17:52 |
| zbr | sshnaidm|rover: lets merge the linters fix first, and after this I can work on finishing the molecule job (tomorrow) | 17:53 |
| zbr | but we need linters first. | 17:53 |
| zbr | which is likely going to fix yours too | 17:53 |
| sshnaidm|rover | zbr, what is 305? | 17:56 |
| rlandy|ruck | rfolco: the rhel8 scenario check tests are all red - is that expected? | 17:58 |
| rlandy|ruck | tripleo-ci-rhel-8-scenario001-standalone-rdofailure | 17:59 |
| rlandy|ruck | tripleo-ci-rhel-8-scenario002-standalone-rdofailure | 17:59 |
| rlandy|ruck | tripleo-ci-rhel-8-scenario003-standalone-rdofailure | 17:59 |
| rlandy|ruck | tripleo-ci-rhel-8-scenario004-standalone-rdo | 17:59 |
| rlandy|ruck | https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-rhel-8-scenario001-standalone-rdo - since yesterday | 17:59 |
| rfolco | rlandy|ruck, periodic ones were good... don't know about scenario check jobs, it may require some attention | 18:01 |
| *** dsneddon has quit IRC | 18:03 | |
| rlandy|ruck | ok - will look | 18:03 |
| rlandy|ruck | oh the fun begin | 18:03 |
| *** jaosorior has quit IRC | 18:04 | |
| rlandy|ruck | sshnaidm|rover: hey and stein promoted - this is awesome | 18:11 |
| rlandy|ruck | tripleo_common.image.exception.ImageNotFoundException: Not found image: docker://trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582 | 18:13 |
| rlandy|ruck | ^^ consistent on rhel-8 standalone failures | 18:13 |
| rlandy|ruck | docker pull tripleomaster/rhel-binary-aodh-api:19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582 exists | 18:18 |
| rlandy|ruck | sudo docker pull trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582_x86_64 | 18:19 |
| rlandy|ruck | rfolco: ^^ | 18:19 |
| *** amoralej is now known as amoralej|off | 18:19 | |
| rlandy|ruck | the new promoter pushes with manifests, right? | 18:20 |
| rlandy|ruck | the jobs are looking for /trunk.registry.rdoproject.org/tripleomaster/rhel-binary-aodh-api:19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582 | 18:20 |
| rlandy|ruck | the tag is now 19f046112dd8e530fb2ca9338edc5ace27339599_f5dd5582_x86_64 | 18:20 |
| rlandy|ruck | do we need to change the tests or the push??? | 18:21 |
| rlandy|ruck | is that from your test promotions?? | 18:23 |
| rlandy|ruck | will brb | 18:26 |
| weshay | rlandy|ruck, holy crap tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 is passing | 18:35 |
| weshay | https://review.opendev.org/#/c/691886/ | 18:35 |
| weshay | nevermind.. that's stein | 18:36 |
| *** epoojad1 has quit IRC | 18:41 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 18:41 | |
| recheck | [rdo-blog] RDO Train Released → https://blogs.rdoproject.org/2019/10/rdo-train-released/ | 18:51 |
| recheck | [gh-ansible] v2.9.0 → https://github.com/ansible/ansible/releases/tag/v2.9.0 | 18:51 |
| *** rascasoft has quit IRC | 18:55 | |
| *** rascasoft has joined #oooq | 18:55 | |
| *** dsneddon has joined #oooq | 19:00 | |
| rlandy|ruck | lol | 19:01 |
| rlandy|ruck | rfolco: creating lp bug to track ^^ | 19:02 |
| rlandy|ruck | periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-master - passed yay | 19:03 |
| *** dsneddon has quit IRC | 19:04 | |
| rfolco | rlandy|ruck, thx, not sure if this is a dup of https://bugs.launchpad.net/tripleo/+bug/1850527 | 19:05 |
| openstack | Launchpad bug 1850527 in tripleo "new promoter fails to push to registry" [Critical,Triaged] | 19:05 |
| rfolco | rlandy|ruck, the manifest code is being run on current promoter as well, so could be | 19:05 |
| rlandy|ruck | rfolco: only with rhel? | 19:05 |
| rfolco | rlandy|ruck, the new promoter just ran rhel, thats why | 19:06 |
| rlandy|ruck | and the container w/o the manifest does not exist | 19:06 |
| rlandy|ruck | rfolco: this was failing from yesterday | 19:06 |
| rfolco | this is weird | 19:06 |
| rlandy|ruck | so I don;t know if this is because of your experimenting with the new promoter | 19:06 |
| rfolco | rlandy|ruck, file a new one anyway | 19:07 |
| rfolco | no no | 19:07 |
| rfolco | can't be | 19:07 |
| rfolco | I disabled rhel on current promoter, to test it in the new one | 19:07 |
| rfolco | so separate issues | 19:07 |
| rlandy|ruck | rfolco: so then the only way rhel promotes is with the new promoter? | 19:07 |
| rfolco | I am just saying... manifest push might be broken | 19:08 |
| rfolco | Yes, temporarily until we manually switch it back to normal | 19:08 |
| rlandy|ruck | but only on rhel | 19:08 |
| rlandy|ruck | we need to fix this | 19:08 |
| rfolco | I am currently testing it and finding the root cause | 19:08 |
| rfolco | I got some progress | 19:09 |
| rlandy|ruck | sorry - who gets the bug - ruck/rover or you/sprint | 19:09 |
| rlandy|ruck | because currently we have a ton of check jobs failing | 19:09 |
| rfolco | rlandy|ruck, don't know... I am investigating the bug I pasted | 19:09 |
| rfolco | rlandy|ruck, can assign to me as well and will look in a bit | 19:10 |
| rfolco | if its a dup or not | 19:10 |
| rlandy|ruck | ok - I'll create the bug anyways and you ca decide | 19:10 |
| rlandy|ruck | thank you | 19:10 |
| rfolco | yep | 19:10 |
| rfolco | thanks | 19:10 |
| zbr | sshnaidm|afk: [E305] Use shell only when shell functionality is required | 19:23 |
| sshnaidm|afk | zbr, I think most of cases could be converted to command | 19:24 |
| zbr | weshay: or any other, please wf https://review.opendev.org/#/c/692418/1 | 19:26 |
| zbr | we need a new card for next sprint: fix ansible-lint call from pre-commit, current implementation catches only some errors, probably 50% of them. | 19:27 |
| weshay | zbr, adding to the list | 19:28 |
| rlandy|ruck | for once ovb is not the largest contributor to job failures | 19:31 |
| sshnaidm|afk | ansible 2.9.0 is out | 19:36 |
| sshnaidm|afk | rlandy|ruck, let's adopt it? ^^ so we'll not be bored | 19:36 |
| *** sshnaidm|afk is now known as sshnaidm|off | 19:37 | |
| rlandy|ruck | sshnaidm|off: on your shift - by all means | 19:38 |
| weshay | oh lordy.. guys.. my advice is to stop inventing work for a while | 19:52 |
| weshay | zbr, so.. I found our jira | 19:52 |
| zbr | weshay: meaning? | 19:53 |
| weshay | zbr, meaning.. it's happening sooner rather than later.. getting details now | 19:54 |
| rlandy|ruck | something is up with rocky | 19:58 |
| rlandy|ruck | 2019-10-31 19:24:20,178 8038 ERROR promoter QCOW IMAGE UPLOAD FAILED LOGS BELOW: | 19:58 |
| rlandy|ruck | 2019-10-31 19:24:20,179 8038 ERROR promoter ERROR: The promoted hash was not found: https://images.rdoproject.org/centos7/rocky/rdo_trunk/b52b560fb05102312361a1ce47d71a5de5c19e94_64cf2148 | 19:58 |
| rlandy|ruck | ERROR: /home/centos/ci-config/ci-scripts/promote-images.sh failed with 3 exit code | 19:58 |
| rlandy|ruck | ^^ right image does not exist | 19:58 |
| rlandy|ruck | https://review.rdoproject.org/zuul/build/f74ec695cd6e49f79029fa692bebdfc3 - success but no logs | 19:59 |
| rlandy|ruck | weshay: ^^ how is that possible? | 19:59 |
| weshay | rlandy|ruck, we missing a hash on the image server? | 20:00 |
| weshay | rlandy|ruck, probably a storage snafu | 20:00 |
| rlandy|ruck | weshay: ack but rocky is promoting | 20:00 |
| rlandy|ruck | at least it's trying to | 20:00 |
| rlandy|ruck | failing on the image | 20:01 |
| rlandy|ruck | http://logs.rdoproject.org/openstack-periodic-wednesday-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-rocky-upload/f74ec69/ | 20:01 |
| rlandy|ruck | passes | 20:01 |
| *** dsneddon has joined #oooq | 20:01 | |
| rlandy|ruck | but there are no logs links to prove it | 20:01 |
| weshay | CalledProcessError: Command '['bash', '/home/centos/ci-config/ci-scripts/promote-images.sh', '--distro', u'centos', '--distro-version', u'7', u'rocky', 'b52b560fb05102312361a1ce47d71a5de5c19e94_64cf2148', u'current-tripleo']' returned non-zero exit status 3 | 20:01 |
| rlandy|ruck | where do you see that? | 20:02 |
| rlandy|ruck | The requested URL /openstack-periodic-wednesday-weekend/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-rocky-upload/f74ec69/ was not found on this server. | 20:02 |
| weshay | rlandy|ruck, images are not there http://images.rdoproject.org/rocky/rdo_trunk/ | 20:02 |
| rlandy|ruck | weshay: ack | 20:02 |
| weshay | rlandy|ruck, rerun the job w/ testproject on that hash | 20:03 |
| rlandy|ruck | the error is correct | 20:03 |
| rlandy|ruck | weshay: the upload job? | 20:03 |
| rlandy|ruck | sure | 20:03 |
| weshay | aye | 20:03 |
| *** dsneddon has quit IRC | 20:06 | |
| weshay | rlandy|ruck, rhel8 jobs are failing again? | 20:07 |
| weshay | check jobs | 20:08 |
| rlandy|ruck | weshay: ack https://bugs.launchpad.net/tripleo/+bug/1850847 | 20:08 |
| openstack | Launchpad bug 1850847 in tripleo "RHEL8 standalone check jobs are failing to find containers tagged w/o arch in rdoproject registry" [Critical,Triaged] - Assigned to Rafael Folco (rafaelfolco) | 20:08 |
| weshay | ya.. see it | 20:08 |
| weshay | how did that happen? | 20:08 |
| rlandy|ruck | my guess is the new promoter | 20:08 |
| rlandy|ruck | dockerhub has the tag w/o arch | 20:08 |
| rlandy|ruck | rdo registry does not | 20:09 |
| weshay | rfolco, did you guys promote w/ the new promoter? | 20:09 |
| weshay | rlandy|ruck, so we can turn rhel8 back on in the old one | 20:09 |
| rfolco | weshay, no I am reproducing the bug in my rdo instance | 20:09 |
| weshay | k | 20:09 |
| rfolco | weshay, rlandy|ruck feel free to put it back while i am working on the fix | 20:10 |
| rlandy|ruck | promoter is still busy on rocky | 20:10 |
| rlandy|ruck | it wont get to master for a while | 20:10 |
| weshay | rlandy|ruck, ya.. that's not good | 20:10 |
| weshay | not sure if we're spamming docker.io atm.. w/ rocky | 20:10 |
| rlandy|ruck | not good that the promoter is on rocky or that rhel jobs failed? | 20:11 |
| weshay | https://meet.google.com/fhq-qkzj-zbe | 20:11 |
| weshay | rlandy|ruck, ^ | 20:11 |
| rlandy|ruck | weshay: the testproject job is running for image upload | 20:11 |
| weshay | rlandy|ruck, rfolco what I don't understand is | 20:13 |
| weshay | rhel8 has not promoted.. why all of sudden can we not pull containers | 20:13 |
| weshay | any gerrit changes ??? | 20:13 |
| rfolco | we're running old code on old promoter | 20:13 |
| rfolco | continuous delivery is disabled | 20:14 |
| rfolco | if you are relying on any fix or recent code, that isn't going to work | 20:14 |
| rfolco | I don't know, maybe the code that is there is broken also and we entered in a buggy state | 20:15 |
| weshay | rlandy|ruck, oops | 20:33 |
| weshay | pass the gh link again | 20:33 |
| rlandy|ruck | weshay: https://meet.google.com/fhq-qkzj-zbe | 20:33 |
| *** holser has joined #oooq | 20:46 | |
| *** Trevor_V has quit IRC | 20:54 | |
| *** Goneri has quit IRC | 20:58 | |
| *** dsneddon has joined #oooq | 21:02 | |
| *** dsneddon has quit IRC | 21:07 | |
| weshay | rfolco, so.. I think that test attempt on the new promoter broke rhel8 | 21:07 |
| *** rlandy|ruck has quit IRC | 21:09 | |
| *** holser has quit IRC | 21:14 | |
| *** rlandy has joined #oooq | 21:15 | |
| weshay | rlandy, hi | 21:15 |
| weshay | welcome back :) | 21:15 |
| *** rlandy is now known as rlandy|ruck | 21:15 | |
| weshay | rlandy, I think you may need to order a new laptop :) | 21:15 |
| rlandy|ruck | weshay: sorry - crashed again | 21:15 |
| weshay | https://meet.google.com/fhq-qkzj-zbe?authuser=1 | 21:15 |
| rlandy|ruck | this is terrible | 21:15 |
| *** holser has joined #oooq | 21:16 | |
| weshay | rlandy|ruck, https://chrome.google.com/webstore/detail/the-great-suspender/klbibkeccnjlkjkiokjodocebajanakg | 21:17 |
| weshay | rlandy|ruck, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/roles/containers-promote/tasks/main.yaml#L154-L166 | 21:18 |
| weshay | rlandy|ruck, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/roles/containers-promote/tasks/main.yaml#L135-L166 | 21:19 |
| *** holser has quit IRC | 21:21 | |
| rfolco | weshay, :( | 21:25 |
| rfolco | weshay, I'm trying to fix this block https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/container-push/roles/containers-promote/tasks/registry-push.yml#L10-L30 | 21:28 |
| rfolco | weshay, I reproduced the failure executing it separated | 21:29 |
| rfolco | will resume tomorrow or later tonight | 21:29 |
| *** rfolco is now known as rfolco|afk | 21:29 | |
| *** aakarsh has quit IRC | 21:40 | |
| *** holser has joined #oooq | 21:45 | |
| *** jtomasek has quit IRC | 21:47 | |
| *** zbr has quit IRC | 21:58 | |
| *** zbr has joined #oooq | 21:59 | |
| weshay | rlandy|ruck, https://bugs.launchpad.net/tripleo/+bug/1847585 | 22:00 |
| openstack | Launchpad bug 1847585 in tripleo "periodics centos-7-standalone-full-tempest-master + centos-7-ovb-1ctlr_2comp-featureset020-master timeout tempest tests" [Critical,Fix released] - Assigned to yatin (yatinkarel) | 22:00 |
| *** dsneddon has joined #oooq | 22:03 | |
| *** holser has quit IRC | 22:04 | |
| rlandy|ruck | https://review.rdoproject.org/r/#/c/23541/ | 22:04 |
| weshay | validate-tempest/tasks/pre-tempest.yml:73: when: tempest_format in ['container', 'packages'] and release not in ['newton', 'ocata'] | 22:08 |
| *** dsneddon has quit IRC | 22:08 | |
| weshay | https://trunk.rdoproject.org/rhel8-master/previous-current-tripleo/commit.yaml | 22:20 |
| weshay | rlandy|ruck, ^ | 22:20 |
| rlandy|ruck | weshay: https://review.rdoproject.org/r/23542 Test rhel-8 master - can spin on that | 22:38 |
| *** jbadiapa has quit IRC | 22:46 | |
| *** dsneddon has joined #oooq | 23:03 | |
| *** dsneddon has quit IRC | 23:08 | |
| *** rlandy|ruck is now known as rlandy|ruck|bbl | 23:19 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!