| *** holser has quit IRC | 00:07 | |
| *** Goneri has quit IRC | 00:10 | |
| *** jmasud has quit IRC | 01:14 | |
| *** Tengu_ has joined #oooq | 01:16 | |
| *** Tengu has quit IRC | 01:19 | |
| *** jmasud has joined #oooq | 01:21 | |
| *** Tengu has joined #oooq | 01:21 | |
| *** jmasud has quit IRC | 01:22 | |
| *** Tengu_ has quit IRC | 01:22 | |
| *** apetrich has quit IRC | 02:07 | |
| *** ysandeep|away is now known as ysandeep | 03:13 | |
| *** jmasud has joined #oooq | 03:44 | |
| *** ysandeep is now known as ysandeep|afk | 03:47 | |
| *** skramaja has joined #oooq | 04:06 | |
| *** saneax has joined #oooq | 04:18 | |
| *** ratailor has joined #oooq | 04:29 | |
| *** udesale has joined #oooq | 04:33 | |
| bhagyashris | weshay, :) | 04:34 |
|---|---|---|
| *** ykarel|away has joined #oooq | 04:34 | |
| *** ykarel|away is now known as ykarel | 04:44 | |
| *** ykarel_ has joined #oooq | 05:03 | |
| *** ykarel has quit IRC | 05:04 | |
| *** jfrancoa has joined #oooq | 05:11 | |
| *** ykarel_ is now known as ykarel | 05:22 | |
| *** marios has joined #oooq | 05:24 | |
| *** ysandeep|afk is now known as ysandeep | 05:43 | |
| *** udesale_ has joined #oooq | 05:49 | |
| *** udesale has quit IRC | 05:52 | |
| *** udesale__ has joined #oooq | 05:56 | |
| *** udesale_ has quit IRC | 05:58 | |
| *** amoralej|off is now known as amoralej | 06:11 | |
| *** udesale_ has joined #oooq | 06:16 | |
| *** udesale__ has quit IRC | 06:19 | |
| *** apetrich has joined #oooq | 06:43 | |
| *** jtomasek has joined #oooq | 07:02 | |
| *** tosky has joined #oooq | 07:26 | |
| matbu | hey oooq folks, so im a bit lost in the rdo-jobs job def | 07:34 |
| matbu | can someone help me a bit ? | 07:34 |
| marios | o/ matbu what are you looking for/at | 07:35 |
| matbu | marios: hey man o/ | 07:36 |
| matbu | marios: so i'm looking at the component job defintion | 07:37 |
| matbu | but i want to find a way to add my proper validation playbook | 07:37 |
| matbu | marios: like more or less this: https://review.opendev.org/#/c/755513/4/zuul.d/base.yaml | 07:38 |
| matbu | not sure if i'm clear here | 07:38 |
| marios | matbu: which component job you looking at | 07:38 |
| *** holser has joined #oooq | 07:39 | |
| matbu | marios: the component validation | 07:40 |
| matbu | marios: like for example this one: https://github.com/rdo-infra/rdo-jobs/blob/master/zuul.d/component-jobs.yaml#L1134 | 07:40 |
| marios | matbu: ack checking | 07:41 |
| marios | matbu: so, i believe all you need is to check the parenting that one is parenting ultimately from https://github.com/openstack/tripleo-ci/blob/master/zuul.d/base.yaml#L378 | 07:43 |
| matbu | marios: hmm ok, i was lost in the parenting | 07:45 |
| matbu | marios: so just override the vars: playbook will be enough i guess ? | 07:45 |
| marios | matbu: yeah, try this https://github.com/marios/tripleo_ruck_job_tool#install-oooci-jobssh | 07:46 |
| marios | matbu: for parenting its what i use | 07:46 |
| marios | matbu: so i think you could override vars: playbook in the component job itself, or override it in the base job definition i.e. add into https://github.com/openstack/tripleo-ci/blob/master/zuul.d/base.yaml#L378 | 07:47 |
| matbu | marios: ack i will look at your tool thx | 07:47 |
| matbu | marios: and i guess its better to override at the rdo level | 07:48 |
| marios | matbu: np, you give it a name it tells you where it is defined and you can keep copy/pasting the parent name in until you hit tripleo-ci/zuul.d/base.yaml | 07:48 |
| matbu | to avoid adding stuff in tripleo-ci and in tripleo-ci pipeline | 07:48 |
| marios | matbu: ok... am a bit concerned/not sure if it will completely override the parent 'playbooks:' i.e. from tripleo-ci/base ... looking at https://zuul-ci.org/docs/zuul/reference/job_def.html#attr-job.vars | 07:51 |
| marios | matbu: and for reference, here is another example where we override the playbooks in rdo-jobs: https://github.com/rdo-infra/rdo-jobs/blob/258c4dc0fe0056d6c1205e7e840cd28fd51ec306/zuul.d/standalone-jobs.yaml#L496 so you may need to replicate what we have in zuul.d/base as well as add your validations | 07:52 |
| marios | holser: \o/ https://review.opendev.org/#/c/739457/39 | 07:53 |
| holser | marios yeah | 07:53 |
| holser | I've worked with mwhahaha on networking | 07:54 |
| matbu | a green upgrade job | 07:54 |
| holser | marios 1 month of work though | 07:54 |
| matbu | congrats :D | 07:54 |
| marios | man scenarios didn't work though https://review.opendev.org/#/c/757605/ | 07:54 |
| marios | holser: one month ? closer to 4/5 this was the 4th? maybe bug just for ussuri standalone | 07:55 |
| holser | marios that's extra measure | 07:55 |
| holser | right | 07:55 |
| marios | holser: yeah but it is in the depends on | 07:55 |
| holser | well, also stein and train | 07:55 |
| holser | I'll need to get standalone upgrade done for those as well | 07:56 |
| *** jpena|off is now known as jpena | 07:57 | |
| ykarel | marios, we working on a patch to try to clear https://review.opendev.org/#/c/757605/ | 08:07 |
| marios | ykarel: great thanks, i see nova db sync fails but the stdout is empty | 08:07 |
| ykarel | would need a temp patch for migration | 08:07 |
| ykarel | ohhk i just checked tempest fails | 08:08 |
| marios | * https://ca4e422190e719ff4803-b95957069fa081f0196bfab2502640d8.ssl.cf5.rackcdn.com/757605/1/check/tripleo-ci-centos-8-scenario001-standalone/d8a2179/logs/undercloud/home/zuul/standalone_deploy.log | 08:08 |
| ykarel | for that i have idea, for other fails need to check | 08:08 |
| marios | * 2020-10-13 02:48:36.146944 | fa163e56-d600-71ff-f22e-000000003568 | FATAL | Check containers status | standalone | error={"changed": false, "msg": "Failed container(s): ['nova_db_sync'], check logs in /var/log/containers/stdouts/"} | 08:08 |
| marios | 2020-10-13 02:48:36.148326 | fa163e56-d600-71ff-f22e-000000003568 | TIMING | tripleo_container_manage : Check containers status | standalone | 1:14:06.566673 | 46.02s | 08:08 |
| marios | * https://ca4e422190e719ff4803-b95957069fa081f0196bfab2502640d8.ssl.cf5.rackcdn.com/757605/1/check/tripleo-ci-centos-8-scenario001-standalone/d8a2179/logs/undercloud/var/log/containers/stdouts/nova_db_sync.log | 08:08 |
| marios | ykarel: ack thats what i mean ^^ | 08:08 |
| ykarel | marios, ack got it | 08:08 |
| marios | thanks for pushing on that stuff holser almost there... | 08:18 |
| holser | marios yw | 08:18 |
| *** holser has quit IRC | 08:22 | |
| *** udesale__ has joined #oooq | 08:24 | |
| *** pmannidi has quit IRC | 08:24 | |
| *** pmannidi has joined #oooq | 08:24 | |
| *** udesale_ has quit IRC | 08:26 | |
| *** holser has joined #oooq | 08:38 | |
| *** derekh has joined #oooq | 08:44 | |
| *** holser has quit IRC | 08:47 | |
| *** holser has joined #oooq | 08:48 | |
| *** jtomasek has quit IRC | 08:52 | |
| *** jtomasek has joined #oooq | 08:56 | |
| *** udesale_ has joined #oooq | 08:57 | |
| *** udesale__ has quit IRC | 08:59 | |
| *** dtantsur|afk is now known as dtantsur | 09:06 | |
| *** jtomasek has quit IRC | 09:36 | |
| *** holser has quit IRC | 09:54 | |
| *** holser has joined #oooq | 10:21 | |
| marios | biab | 10:43 |
| *** marios has quit IRC | 10:43 | |
| *** udesale has joined #oooq | 10:43 | |
| *** udesale_ has quit IRC | 10:44 | |
| chandankumar | sshnaidm: Hello | 10:57 |
| sshnaidm | chandankumar, hey | 10:57 |
| chandankumar | sshnaidm: regarding installing docker in pre https://review.opendev.org/#/c/752421/76/playbooks/tripleo-buildcontainers/docker.yaml | 10:58 |
| chandankumar | sshnaidm: but docker service is failing to start https://logserver.rdoproject.org/21/29921/7/check/tripleo-ci-centos-7-content-provider-train-test/55e88f2/job-output.txt | 10:59 |
| chandankumar | sshnaidm: there might be some issue with insecure registry part | 10:59 |
| chandankumar | I am not sure | 10:59 |
| sshnaidm | chandankumar, does it go with "become"? | 11:00 |
| sshnaidm | chandankumar, why do you configure a proxy? | 11:01 |
| chandankumar | sshnaidm: https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-docker/tasks/docker-distro.yaml#L4 | 11:01 |
| chandankumar | become is already there | 11:01 |
| chandankumar | sshnaidm: to solve http issue now I think i need to put in container prepare parameter file | 11:01 |
| chandankumar | https://1426d9fa64a2ea83ac45-7e70b99d63ad0f32ef21f8f8ea4f17ac.ssl.cf5.rackcdn.com/752421/41/check/tripleo-ci-centos-7-containers-multinode-queens/572167b/logs/undercloud/home/zuul/containers-prepare-parameter.yaml | 11:02 |
| sshnaidm | chandankumar, let's try w/o proxy | 11:02 |
| sshnaidm | chandankumar, just "ensure-docker" | 11:02 |
| sshnaidm | and let's see | 11:02 |
| sshnaidm | chandankumar, better to run it on rdo and hold a node in this case, because journal is not collected in this step | 11:04 |
| sshnaidm | chandankumar, so we could login and see what are errors | 11:04 |
| sshnaidm | chandankumar, but I think worth to try w/o a proxy once | 11:04 |
| chandankumar | sshnaidm: if i got without proxy, https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-docker/tasks/docker-setup.yaml#L34 | 11:05 |
| chandankumar | fails | 11:05 |
| chandankumar | /etc/docker/daemon.json fail does not exists | 11:06 |
| chandankumar | sshnaidm: https://review.opendev.org/#/c/752421/74..76/playbooks/tripleo-buildcontainers/docker.yaml | 11:06 |
| sshnaidm | chandankumar, I see.. then let's try w/o docker_mirror: '{{ nodepool.public_ipv4 }}:5001' | 11:07 |
| sshnaidm | I don't think it's needed | 11:07 |
| chandankumar | sshnaidm: ok | 11:07 |
| chandankumar | sure | 11:07 |
| sshnaidm | chandankumar, I'll renew the hold on https://review.rdoproject.org/r/#/c/29921/ and let's recheck it there if fails | 11:08 |
| chandankumar | sshnaidm: patch rechecked | 11:10 |
| chandankumar | sshnaidm: http://paste.openstack.org/show/798984/ | 11:26 |
| chandankumar | sshnaidm: Error: docker-ce conflicts with 2:docker-1.13.1-162.git64e9980.el7.centos.x86_64 | 11:27 |
| sshnaidm | chandankumar, so it's installed already | 11:27 |
| chandankumar | for queens, docker is already there somehow | 11:27 |
| chandankumar | for train it is not there | 11:27 |
| sshnaidm | chandankumar, ack | 11:28 |
| sshnaidm | very weird, but ok.. | 11:28 |
| *** jpena is now known as jpena|lunch | 11:31 | |
| soniya29|ruck | weshay: meeting? | 11:31 |
| *** holser has quit IRC | 11:32 | |
| *** holser has joined #oooq | 11:33 | |
| *** holser has quit IRC | 11:34 | |
| *** marios has joined #oooq | 11:38 | |
| weshay | soniya29|ruck, 0/ | 11:38 |
| weshay | sorry I'm late | 11:38 |
| soniya29|ruck | weshay no problem | 11:38 |
| weshay | soniya29|ruck, https://review.opendev.org/#/c/757472/ | 11:40 |
| *** udesale_ has joined #oooq | 11:41 | |
| *** udesale has quit IRC | 11:43 | |
| *** holser has joined #oooq | 11:44 | |
| weshay | soniya29|ruck, ender network_config from template | overcloud-novacompute-0 | error={"changed": false, "msg": "AnsibleUndefinedVariable: 'external_gateway_ip' is undefined"} | 11:45 |
| weshay | https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/eebbe79/logs/undercloud/home/zuul/ansible.log.txt.gz | 11:45 |
| *** holser has quit IRC | 11:46 | |
| weshay | soniya29|ruck, http://codesearch.openstack.org/?q=external_gateway_ip&i=nope&files=&repos= | 11:47 |
| *** holser has joined #oooq | 11:50 | |
| *** ysandeep is now known as ysandeep|brb | 11:56 | |
| *** holser has quit IRC | 11:56 | |
| *** rlandy has joined #oooq | 11:58 | |
| *** rlandy is now known as rlandy|rover | 11:58 | |
| rlandy|rover | soniya29|ruck: weshay: hey - how are things? | 12:02 |
| weshay | 0/ | 12:02 |
| weshay | rlandy|rover, pretty good.. just working w/ soniya29|ruck on the master fs035 error.. looks like a real bug | 12:02 |
| weshay | gate is ok | 12:03 |
| rlandy|rover | weshay: ack- looks like 039 and 002 will fail as well | 12:03 |
| weshay | rlandy|rover, hrm.. check for ""AnsibleUndefinedVariable: 'external_gateway_ip'" in the ansible.log | 12:03 |
| rlandy|rover | weshay: checking logs | 12:04 |
| weshay | rlandy|rover, when we have time today we should work through https://hackmd.io/dTXZH7hYQSKIW7GEYjo8ww?view | 12:04 |
| rlandy|rover | weshay: ack - after I finish the ipa changes | 12:04 |
| weshay | aye | 12:04 |
| rlandy|rover | weshay: funnily enough we did a lot better when I checked yesterday | 12:04 |
| weshay | rlandy|rover, was thinking the content provider could also be a dns server | 12:05 |
| rlandy|rover | build almost all containers | 12:05 |
| rlandy|rover | weshay: we know there are issues with some clouds and that | 12:05 |
| rlandy|rover | let's see if we solve it properly for ipa | 12:05 |
| rlandy|rover | alos | 12:05 |
| rlandy|rover | also | 12:05 |
| rlandy|rover | some clouds use unbound | 12:06 |
| rlandy|rover | so we would have to make that determination on every job | 12:06 |
| rlandy|rover | why the extra step for every job? | 12:06 |
| *** jbadiapa has joined #oooq | 12:06 | |
| rlandy|rover | anyways - let me look at this 035 failure first | 12:06 |
| *** amoralej is now known as amoralej|lunch | 12:07 | |
| *** udesale_ has quit IRC | 12:07 | |
| weshay | ysandeep|brb, ping me if ur back soon if you want to 1-1 | 12:08 |
| weshay | early | 12:08 |
| *** ysandeep|brb is now known as ysandeep | 12:08 | |
| ysandeep | weshay, sure we can have it now.. joining meeting | 12:08 |
| rlandy|rover | weshay: did we try rerunning 035 yet to see if it reproduces? | 12:09 |
| weshay | rlandy|rover, failed the same way 5 times | 12:09 |
| rlandy|rover | oh ok | 12:09 |
| *** rfolco has joined #oooq | 12:10 | |
| rlandy|rover | weshay: look at the difference here ... http://codesearch.openstack.org/?q=external_gateway_ip&i=nope&files=&repos= | 12:10 |
| rlandy|rover | ipv6 is the only one not defaulted | 12:11 |
| rlandy|rover | https://opendev.org/openstack/tripleo-ansible/blame/branch/master/tripleo_ansible/roles/tripleo_network_config/templates/ci/multiple_nics_ipv6.j2 | 12:11 |
| rlandy|rover | ^^ added 6 days ago | 12:11 |
| rlandy|rover | Rabi's patch | 12:12 |
| rfolco | upstream review down ? | 12:12 |
| rfolco | proxy error | 12:12 |
| Tengu | yup | 12:12 |
| rfolco | :( | 12:12 |
| Tengu | already announced on infra | 12:12 |
| Tengu | apparently only web - patches can go through SSH | 12:13 |
| Tengu | not tested yet though. | 12:13 |
| marios | yeah fungi is looking there | 12:16 |
| marios | Tengu: yes i submitted a review when gerrit web was down | 12:17 |
| Tengu | :) | 12:17 |
| *** holser has joined #oooq | 12:19 | |
| *** holser has quit IRC | 12:23 | |
| *** ratailor has quit IRC | 12:26 | |
| *** holser has joined #oooq | 12:29 | |
| rlandy|rover | weshay: soniya29|ruck: fs039 and fs002 don't look like long time errors at first glance - rerunning those | 12:30 |
| *** jpena|lunch is now known as jpena | 12:31 | |
| marios | bac, | 12:31 |
| marios | back now Tengu | 12:31 |
| Tengu | yup | 12:31 |
| rfolco | akahat, ping you join us today? | 12:32 |
| rlandy|rover | weshay: soniya29|ruck: anyone looked at the ussuri failures yet? | 12:35 |
| rlandy|rover | rdocloud registry access? | 12:35 |
| soniya29|ruck | rlandy|rover nope | 12:39 |
| rlandy|rover | soniya29|ruck: see #rhos-ops | 12:39 |
| rlandy|rover | on internal channels | 12:39 |
| soniya29|ruck | rlandy|rover yupp | 12:41 |
| marios | https://review.opendev.org/#/c/753708/21/roles/undercloud-deploy/templates/containers-prepare-parameter.yaml.j2 | 12:41 |
| rfolco | marios, chandankumar sshnaidm this one is good to merge https://review.opendev.org/#/c/756575 | 12:53 |
| rfolco | akahat, ^ | 12:54 |
| *** Goneri has joined #oooq | 12:54 | |
| chandankumar | rfolco: charges applied with * | 12:54 |
| *** pojadhav is now known as pojadhav|afk | 12:54 | |
| rfolco | chandankumar, I have coupons | 12:55 |
| rfolco | discount coupons | 12:55 |
| chandankumar | rfolco: discount coupons not allowed only cash will work | 12:55 |
| rfolco | :( | 12:55 |
| marios | rfolco: what's in it for me? | 12:58 |
| rfolco | marios, I want your +2, nothing else. Our relation is purely moved by one thing: your _+2 | 12:58 |
| marios | 15:58 < marios> rfolco: what's in it for me? | 12:59 |
| rfolco | marios, I lie, I love you. | 12:59 |
| marios | 15:59 < marios> 15:58 < marios> rfolco: what's in it for me? | 12:59 |
| rfolco | marios, maybe you can ask another time | 12:59 |
| weshay | rlandy|rover, ussuri promoted six hours ago | 13:01 |
| rlandy|rover | weshay: it's very red now - access issues | 13:01 |
| rlandy|rover | same as fs002 and fs039 | 13:02 |
| weshay | k | 13:02 |
| rlandy|rover | see #rhos-ops | 13:02 |
| weshay | rlandy|rover, sounds like a network issue | 13:03 |
| *** skramaja has quit IRC | 13:10 | |
| weshay | soniya29|ruck, | 13:12 |
| weshay | 10: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1350 state UNKNOWN qlen 1000 | 13:12 |
| weshay | inet6 fe80::f816:3eff:fee7:d3ed/64 scope link | 13:12 |
| weshay | valid_lft forever preferred_lft forever | 13:12 |
| weshay | https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/aca8750/logs/overcloud-novacompute-0/var/log/extra/network.txt.gz | 13:12 |
| *** amoralej|lunch is now known as amoralej | 13:12 | |
| weshay | so.. not 100% it's relevant but it looks like br-ex got an ipv6 addr previously in passing jobs | 13:12 |
| weshay | soniya29|ruck, can you run a test project job w/ fs035 against https://review.opendev.org/#/c/757849/1/tripleo_ansible/roles/tripleo_network_config/templates/ci/multiple_nics_ipv6.j2 | 13:13 |
| weshay | please | 13:13 |
| soniya29|ruck | weshay sure | 13:13 |
| weshay | soniya29|ruck, ah.. sorry .. I pasted the ipv6 addr | 13:14 |
| weshay | this is the route | 13:14 |
| * weshay looking again at https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset035-master/aca8750/logs/overcloud-novacompute-0/var/log/extra/network.txt.gz | 13:15 | |
| rlandy|rover | weshay: soniya29|ruck: wrt fs035 https://bugs.launchpad.net/tripleo/+bug/1899627 - rabi posted a patch | 13:15 |
| openstack | Launchpad bug 1899627 in tripleo "fs035 jobs are failing in master to deploy the overcloud - "AnsibleUndefinedVariable: 'external_gateway_ip'" [Critical,In progress] - Assigned to Rabi Mishra (rabi) | 13:15 |
| weshay | fe80::/64 dev br-ex proto kernel metric 256 pref medium | 13:15 |
| weshay | soniya29|ruck, so.. it looks like next_hop: {{ external_gateway_ip | default('fe80::1') }} | 13:15 |
| rlandy|rover | unless this is something else | 13:15 |
| weshay | will work https://review.opendev.org/#/c/757849/1/tripleo_ansible/roles/tripleo_network_config/templates/ci/multiple_nics_ipv6.j2 | 13:16 |
| rlandy|rover | there is already a testproject running | 13:16 |
| weshay | soniya29|ruck, you following my steps there? do they make sense? | 13:18 |
| weshay | rlandy|rover, thanks | 13:18 |
| soniya29|ruck | weshay yupp | 13:18 |
| weshay | rlandy|rover, I don't see a tp on rabi's change | 13:19 |
| weshay | https://review.rdoproject.org/r/#/c/29969/ | 13:19 |
| weshay | nvrmind | 13:19 |
| arxcruz | weshay: so, i can't use tripleo quay credentials because... | 13:19 |
| arxcruz | Error: error authenticating creds for "quay.io": too many requests to registry | 13:19 |
| weshay | ah fak | 13:20 |
| weshay | we're screwed | 13:20 |
| weshay | arxcruz, can you spoof your ip? | 13:20 |
| weshay | lolz | 13:20 |
| arxcruz | weshay: it's not related to my ip, it's the user | 13:20 |
| weshay | ah geez | 13:20 |
| arxcruz | weshay: i can use my credentials for example with no problems | 13:20 |
| weshay | hrm | 13:21 |
| weshay | arxcruz, k.. let's chat | 13:21 |
| weshay | https://meet.google.com/obu-qdhm-hyy | 13:22 |
| ysandeep | weshay, rlandy|rover When you have time could you please review and merge https://code.engineering.redhat.com/gerrit/#/c/214931/ & https://code.engineering.redhat.com/gerrit/#/c/214949/ | 13:24 |
| *** pojadhav|afk is now known as pojadhav | 13:27 | |
| chandankumar | arxcruz: does the pushed images are public? if not it will face rate limit issue | 13:29 |
| *** TrevorV has joined #oooq | 13:30 | |
| rfolco | arxcruz, rfolco, zbr, sshnaidm, rlandy, marios, ysandeep, bhagyashris, svyas, pojadhav, akahat, weshay, chandankumar | 13:31 |
| rfolco | ping community call | 13:31 |
| arxcruz | chandankumar: hmmm maybe, i see 4 repos marked as private there | 13:31 |
| rfolco | https://meet.google.com/bqx-xwht-wky?authuser=1 | 13:31 |
| arxcruz | but i did not wanted to change | 13:31 |
| rfolco | https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw#Agenda | 13:31 |
| rfolco | weshay, you join us ? | 13:32 |
| *** jfrancoa has quit IRC | 13:42 | |
| *** jfrancoa has joined #oooq | 13:42 | |
| chandankumar | arxcruz: can we use skopeo directly for pushing images? few questions, we are pulling containers from rdo registry? where annonymouse pull is disabled | 13:50 |
| chandankumar | then pushing to quay | 13:50 |
| weshay | sshnaidm, in 1-1 | 14:06 |
| *** pojadhav is now known as pojadhav|afk | 14:06 | |
| ysandeep | rlandy|rover weshay sshnaidm: new nodeset patch for downstream - https://code.engineering.redhat.com/gerrit/#/c/214952/ | 14:12 |
| *** jfrancoa has quit IRC | 14:13 | |
| *** jfrancoa has joined #oooq | 14:13 | |
| rlandy|rover | ysandeep: thanks - merging | 14:16 |
| ysandeep | thanks :) | 14:17 |
| ysandeep | leaving for the day, when you have time please review this too https://code.engineering.redhat.com/gerrit/#/c/214949/ | 14:19 |
| *** ysandeep is now known as ysandeep|away | 14:25 | |
| weshay | rebased https://review.opendev.org/756133 content-provider failed in the gate | 14:27 |
| weshay | rfolco, can you please put up some patches for content provider on https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/periodic.yaml | 14:28 |
| weshay | rfolco, /me wants to see how well it works there | 14:28 |
| weshay | arxcruz, want to pick it back up? | 14:30 |
| chandankumar | sshnaidm: so queens docker package conflict issue got fixed now | 14:34 |
| arxcruz | weshay: sure | 14:34 |
| chandankumar | sshnaidm: https://review.opendev.org/#/c/752421/81/playbooks/tripleo-buildcontainers/docker.yaml | 14:34 |
| weshay | arxcruz, https://meet.google.com/ors-izwj-kra?authuser=1 | 14:35 |
| chandankumar | weshay: marios sshnaidm rlandy|rover rfolco feel free to review these two patches https://review.opendev.org/752419 and https://review.opendev.org/752421 | 14:35 |
| weshay | rfolco, oh.. periodic has a content-provider.. I didn't see that | 14:35 |
| * weshay looks | 14:35 | |
| sshnaidm | chandankumar++ | 14:36 |
| weshay | rfolco, you there? | 14:46 |
| weshay | rfolco, akahat fyi.. https://zuul.openstack.org/buildset/58b315bfa98e4042ac44d782be0b01ab | 14:46 |
| weshay | doesn't appear this is working https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/periodic.yaml | 14:47 |
| rfolco | weshay, yes sorry I saw your msg | 14:47 |
| weshay | looks like you need to add dependency | 14:47 |
| rfolco | weshay, hmmm yeah this is not looking right | 14:48 |
| rfolco | weshay, I can fix this | 14:48 |
| weshay | ++ | 14:48 |
| chandankumar | rlandy|rover: we donot need the nodepool or host ip now actually, it needs to be added on container prep parameters side , but learned some many thing from that day experiment, thank you :-) | 14:49 |
| marios | chandankumar: noting to check | 14:50 |
| chandankumar | marios: thanks :-) | 14:50 |
| rlandy|rover | chandankumar: good to hear :) | 14:53 |
| *** amoralej is now known as amoralej|off | 15:01 | |
| weshay | akahat, recheck doesn't fix config errors 757544 https://review.opendev.org/#/c/757544/ | 15:04 |
| weshay | rfolco, oh.. one note.. you can probably remove rocky/stein | 15:05 |
| weshay | from periodic | 15:05 |
| chandankumar | weshay: for rocky & stein, we will be keeping only one upgrade job na? | 15:05 |
| weshay | chandankumar, I have two patches up on rocky/stein.. | 15:06 |
| weshay | chandankumar, we'll have to review the patches and what we think is best for us | 15:06 |
| chandankumar | weshay: https://review.opendev.org/#/c/757643/ ? | 15:07 |
| rfolco | weshay, there is only upgrade jobs for stein/train there https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/periodic.yaml | 15:07 |
| weshay | chandankumar, aye.. and one in rdo | 15:08 |
| chandankumar | sshnaidm: we are back in game for c7 | 15:10 |
| chandankumar | both c7 queens and train working | 15:11 |
| chandankumar | content provider | 15:11 |
| sshnaidm | chandankumar, great! | 15:12 |
| rfolco | chandankumar, weshay: whats the problem with https://review.opendev.org/#/c/757889 ? any thoughts ? | 15:13 |
| sshnaidm | chandankumar, commented on https://review.opendev.org/#/c/752421 | 15:15 |
| sshnaidm | rfolco, rebase on https://review.opendev.org/757743 | 15:16 |
| rfolco | sshnaidm, that's it, thank you | 15:16 |
| chandankumar | rfolco: need to add this one tripleo-ci-centos-8-content-provider-ussuri also there i think | 15:16 |
| chandankumar | ok sshnaidm patch will fix it | 15:17 |
| *** ykarel is now known as ykarel|away | 15:18 | |
| *** jmasud has quit IRC | 15:26 | |
| rlandy|rover | weshay: soniya29|ruck: hmmm ... maybe we have a legit issue - https://ci.centos.org/job/tripleo-quickstart-promote-ussuri-current-tripleo-delorean-minimal/ 10 failed jobs | 15:37 |
| rlandy|rover | intropspection | 15:38 |
| rlandy|rover | weshay: also - did we get agreement on removing rocky? | 15:39 |
| weshay | rlandy|rover, have a patch up | 15:39 |
| rlandy|rover | weshay++ | 15:39 |
| *** dtantsur is now known as dtantsur|afk | 15:42 | |
| *** ykarel|away has quit IRC | 15:50 | |
| weshay | hrm.. hit a timeout on content provider.. 2020-10-13 15:15:10.303996 | primary | async task did not complete within the requested time - 5700s | 15:52 |
| weshay | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_7a4/756888/2/gate/tripleo-ci-centos-8-content-provider/7a42ac3/job-output.txt | 15:52 |
| weshay | perhaps we need to bump it | 15:52 |
| weshay | rfolco, ^ | 15:52 |
| rfolco | weshay, ok patch coming | 15:52 |
| weshay | thanks | 15:54 |
| *** pojadhav|afk is now known as pojadhav | 15:56 | |
| *** holser has quit IRC | 15:56 | |
| *** holser has joined #oooq | 15:58 | |
| *** holser has quit IRC | 15:59 | |
| rfolco | weshay, https://review.opendev.org/757905 Bump content provider timeout | 16:00 |
| *** holser has joined #oooq | 16:05 | |
| weshay | rfolco, can't you just override the timeout here? https://opendev.org/openstack/tripleo-ci/src/branch/master/zuul.d/standalone-jobs.yaml#L1039 | 16:14 |
| rfolco | weshay, I tried, but zuul complains... https://review.opendev.org/#/c/757905/1/zuul.d/standalone-jobs.yaml | 16:16 |
| rfolco | weshay, timeout happened while uploading logs, let me try bumping post-timeout | 16:19 |
| rfolco | nevermind, async task did not complete within the requested time - 5700s | 16:21 |
| weshay | rlandy|rover, ysandeep|away fyi.. tracking each 16.2 promotion.. click on promotion details to see what jobs ran http://tripleo-cockpit.usersys.redhat.com/d/KyHCwLHMk/rhos-16-2-full-component-pipeline?orgId=1&fullscreen&panelId=498 | 16:23 |
| rlandy|rover | weshay: ysandeep|away: if the ovb passes will promote again today | 16:24 |
| *** marios is now known as marios|out | 16:32 | |
| *** udesale has joined #oooq | 16:38 | |
| *** udesale has quit IRC | 16:40 | |
| *** marios|out has quit IRC | 16:43 | |
| weshay | rlandy|rover, can you do me a favor and paste in an example repo-query command you are using for https://gitlab.cee.redhat.com/rlandy/repodiff_tools2/-/tree/display_table | 16:52 |
| rlandy|rover | weshay: depends on the type of repo | 16:53 |
| rlandy|rover | module, stream, single repo | 16:54 |
| rlandy|rover | they are different | 16:54 |
| weshay | rlandy|rover, k.. probably worth writing it down vs. search zuul | 16:54 |
| rlandy|rover | weshay: http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git/tree/playbooks/dependency | 16:59 |
| rlandy|rover | both are there | 16:59 |
| *** derekh has quit IRC | 17:00 | |
| arxcruz | weshay: https://quay.io/organization/tripleotestingci | 17:00 |
| *** ykarel|away has joined #oooq | 17:01 | |
| weshay | arxcruz, thanks.. see it | 17:02 |
| *** jmasud has joined #oooq | 17:04 | |
| *** jpena is now known as jpena|off | 17:05 | |
| *** ykarel|away has quit IRC | 17:14 | |
| *** jmasud has quit IRC | 17:18 | |
| *** jfrancoa has quit IRC | 17:21 | |
| rlandy|rover | weshay: ok if I w+ https://review.opendev.org/#/c/757849/"? | 17:25 |
| weshay | rlandy|rover, yes pleaes | 17:29 |
| * rlandy|rover needs to drop car off ... brb | 18:14 | |
| rlandy|rover | sshnaidm: weshay: ^^ | 18:14 |
| *** rlandy|rover is now known as rlandy|rover|brb | 18:14 | |
| weshay | k | 18:14 |
| *** sshnaidm is now known as sshnaidm|afk | 18:24 | |
| rfolco | weshay, any thoughts on what could be happening here ? https://api.us-east.open-edge.io:8080/swift/v1/AUTH_e02c11e4e2c24efc98022353c88ab506/zuul_opendev_logs_2dc/757917/1/check/tripleo-ci-centos-8-content-provider/2dc6a10/logs/undercloud/home/zuul/dlrn.log | 18:52 |
| weshay | rfolco, Error: Failed to download metadata for repo 'advanced-virtualization': Cannot download repomd.xml: Cannot download repodata/repomd.xml: All mirrors were tried | 18:54 |
| weshay | have to look IN the dlrn_logs :) | 18:54 |
| weshay | rfolco, https://api.us-east.open-edge.io:8080/swift/v1/AUTH_e02c11e4e2c24efc98022353c88ab506/zuul_opendev_logs_2dc/757917/1/check/tripleo-ci-centos-8-content-provider/2dc6a10/logs/README.html | 18:55 |
| *** rlandy|rover|brb is now known as rlandy|rover | 18:55 | |
| weshay | link right there :) | 18:55 |
| rfolco | weshay, thank you | 18:56 |
| weshay | :) | 18:56 |
| rlandy|rover | weshay: ysandeep|away: promoting 16.2 https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/builds?pipeline=%09openstack-periodic-integration-rhos-16.2 | 18:57 |
| rlandy|rover | weshay: rfolco: now we're overachieving ??? | 19:00 |
| rlandy|rover | 2020-10-13 14:08:29 | ============= SUMMARY =============== | 19:00 |
| rlandy|rover | 2020-10-13 14:08:29 | Expected................... 89 | 19:00 |
| rlandy|rover | 2020-10-13 14:08:29 | Built...................... 107 | 19:00 |
| rlandy|rover | 2020-10-13 14:08:29 | ===================================== | 19:00 |
| rfolco | rlandy|rover, its a known issue, I'm working on it | 19:01 |
| weshay | rlandy|rover, ya.. what rfolco said :) | 19:01 |
| rfolco | https://review.opendev.org/#/c/755124/ | 19:01 |
| rlandy|rover | weshay: that's 17 | 19:01 |
| rfolco | weshay, rlandy|rover I have a poc working, just need to polish it | 19:01 |
| rlandy|rover | we build all the containers | 19:01 |
| weshay | rlandy|rover, k.. so 17 is back? | 19:01 |
| rlandy|rover | with candidate though | 19:02 |
| rlandy|rover | I think so | 19:02 |
| weshay | rlandy|rover, we need to chat today about WTH w/ osp deps | 19:02 |
| * rlandy|rover trie dthe build/push/upload job now | 19:02 | |
| weshay | this guessing game suuuuuucks | 19:02 |
| rlandy|rover | trying | 19:02 |
| rlandy|rover | weshay: ack - let me rekick job - will ping you when we have results to look at | 19:02 |
| rlandy|rover | like 30 mins? | 19:03 |
| weshay | rlandy|rover, sure.. hey qq | 19:04 |
| weshay | rlandy|rover, do we install the internal redhat cert via a pre in zuul? | 19:04 |
| rlandy|rover | weshay: ack - getting | 19:04 |
| weshay | rlandy|rover, k.. show me in 30 :) | 19:05 |
| rlandy|rover | weshay: preview http://git.app.eng.bos.redhat.com/git/openstack/tripleo-ci-internal-jobs.git/tree/playbooks/configure-rhel.yaml#n57 | 19:05 |
| *** apetrich has quit IRC | 19:06 | |
| weshay | hrm.. looks like I have that.. anyway .. 30min | 19:06 |
| *** TrevorV has quit IRC | 19:26 | |
| *** saneax has quit IRC | 19:32 | |
| rlandy|rover | tripleo-ci-testing/ 2020-10-13 19:54 - holy cow - we're back!! | 19:59 |
| *** jmasud has joined #oooq | 19:59 | |
| rlandy|rover | 17 dockerfiles | 19:59 |
| rlandy|rover | weshay, read to chat when you are | 20:02 |
| rlandy|rover | ready | 20:02 |
| weshay | k | 20:02 |
| rlandy|rover | and new ipa solution | 20:02 |
| weshay | rlandy|rover, getting daughter in headphones | 20:02 |
| weshay | ooooh | 20:03 |
| weshay | https://meet.google.com/acw-vjfx-hwn?authuser=1 | 20:03 |
| *** jbadiapa has quit IRC | 20:20 | |
| *** rfolco has quit IRC | 21:12 | |
| *** rfolco has joined #oooq | 21:14 | |
| weshay | rlandy|rover, that fixed the osp-13 issue! thanks | 22:04 |
| rlandy|rover | weshay: there is an undercloud install issue | 22:08 |
| rlandy|rover | known? | 22:08 |
| weshay | rlandy|rover, I saw | 22:08 |
| rlandy|rover | or we should investigate? | 22:08 |
| weshay | it's probably due to the internal registry or something | 22:08 |
| weshay | will look tomorrow.. we got past todays issue :) | 22:08 |
| *** rfolco has quit IRC | 22:10 | |
| rlandy|rover | ok | 22:11 |
| *** rfolco has joined #oooq | 22:53 | |
| *** rfolco has quit IRC | 23:13 | |
| *** matbu has quit IRC | 23:22 | |
| *** matbu has joined #oooq | 23:23 | |
| *** pmannidi has quit IRC | 23:33 | |
| *** pmannidi has joined #oooq | 23:33 | |
| *** tosky has quit IRC | 23:40 | |
| *** rfolco has joined #oooq | 23:41 | |
| *** Goneri has quit IRC | 23:48 | |
| *** jmasud has quit IRC | 23:54 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!