| *** rf0lc0 has joined #oooq | 00:18 | |
| *** ysandeep|away is now known as ysandeep | 00:26 | |
| *** jmasud has joined #oooq | 00:40 | |
| *** rf0lc0 has quit IRC | 01:06 | |
| *** jmasud has quit IRC | 01:19 | |
| *** jmasud has joined #oooq | 01:22 | |
| *** jmasud has quit IRC | 01:28 | |
| *** ykarel|away has joined #oooq | 01:29 | |
| *** ysandeep is now known as ysandeep|afk | 01:29 | |
| *** ykarel|away is now known as ykarel | 01:31 | |
| *** apetrich has quit IRC | 02:08 | |
| *** udesale has joined #oooq | 03:42 | |
| *** jmasud has joined #oooq | 03:43 | |
| *** jmasud has quit IRC | 03:43 | |
| *** jmasud has joined #oooq | 03:44 | |
| *** udesale_ has joined #oooq | 03:50 | |
| *** udesale has quit IRC | 03:53 | |
| *** ratailor has joined #oooq | 04:10 | |
| *** ykarel_ has joined #oooq | 04:12 | |
| *** ykarel has quit IRC | 04:14 | |
| *** udesale__ has joined #oooq | 04:23 | |
| *** udesale_ has quit IRC | 04:25 | |
| *** udesale has joined #oooq | 04:27 | |
| *** skramaja has joined #oooq | 04:29 | |
| *** udesale__ has quit IRC | 04:30 | |
| *** jfrancoa has joined #oooq | 04:38 | |
| chandankumar | sshnaidm: https://review.opendev.org/#/c/752419/ and https://review.opendev.org/#/c/752421/ are good to go for completing c7 content provider work | 04:40 |
|---|---|---|
| *** ykarel has joined #oooq | 04:49 | |
| *** ykarel_ has quit IRC | 04:51 | |
| *** marios has joined #oooq | 05:02 | |
| *** udesale_ has joined #oooq | 05:03 | |
| *** udesale has quit IRC | 05:05 | |
| *** ykarel has quit IRC | 05:13 | |
| bhagyashris | marios, hi Good morning ! please add this https://review.opendev.org/#/c/757861/ in your review list. Thank you :) | 05:27 |
| marios | bhagyashris: morning ack | 05:31 |
| *** dsneddon has quit IRC | 05:35 | |
| bhagyashris | marios, thanks for review :) but i am not clear about review comment | 05:42 |
| bhagyashris | ohh ok | 05:42 |
| marios | bhagyashris: you need to edit that file | 05:42 |
| bhagyashris | i got it | 05:42 |
| marios | bhagyashris: ac k | 05:42 |
| bhagyashris | sorry i only see file name not full patch that's why i was confused | 05:43 |
| bhagyashris | my bad | 05:43 |
| bhagyashris | s/patch/path | 05:43 |
| marios | bhagyashris: ack np its the same mistake we make everytime we release fwiw :) | 05:44 |
| marios | bhagyashris: i mean the person who posts the emit_releases change usually does what you did (i did it last time too ;)) | 05:44 |
| bhagyashris | marios, ok :) same file name confused me | 05:44 |
| bhagyashris | i see | 05:45 |
| bhagyashris | thanks :) | 05:45 |
| bhagyashris | marios, Done. | 05:48 |
| marios | bhagyashris: ack | 05:51 |
| *** saneax has joined #oooq | 05:58 | |
| *** jtomasek has joined #oooq | 06:01 | |
| chandankumar | marios: https://review.opendev.org/#/c/752419/ and https://review.opendev.org/#/c/752421/ are good to go for completing c7 content provider work | 06:05 |
| marios | chandankumar: will check | 06:06 |
| *** amoralej|off is now known as amoralej | 06:10 | |
| sshnaidm | chandankumar, ++ | 06:25 |
| sshnaidm | let's merge it | 06:25 |
| *** ykarel has joined #oooq | 06:29 | |
| marios | sshnaidm: chandankumar: one more inline question about checking release in ip_address_branch before referencing | 06:31 |
| marios | chandankumar: otherwise lgtm | 06:31 |
| sshnaidm | marios, I'm not sure it makes sense in non-upgrade jobs | 06:32 |
| marios | sshnaidm: it is being referenced on the next line | 06:32 |
| sshnaidm | yeah, and it must be there, if it's not - let's fail this | 06:33 |
| sshnaidm | if release not in registry_ip_address_branch something very wrong happens | 06:34 |
| marios | sshnaidm: right, so better to make it part of the check, and else we fail with some proper message | 06:34 |
| sshnaidm | marios, you'll have message like "registry_ip_address_branch doesn't have 'release'" | 06:34 |
| marios | sshnaidm: any time we reference anything in ansible we are always careful to check that it is defined and give it some default value why is this any different. | 06:35 |
| sshnaidm | marios, I agree, but in this case there is no any "legal" reason for 'release' not to be in registry_ip_address_branch variable. It's sure a bug and we need to fail in this case. | 06:36 |
| *** skramaja_ has joined #oooq | 06:36 | |
| marios | sshnaidm: so then by your logic why do we even have {% if job.registry_ip_address_branch is defined %} | 06:36 |
| marios | sshnaidm: we can remove that one too? | 06:36 |
| *** skramaja has quit IRC | 06:36 | |
| marios | sshnaidm: it will fail 'no attribute registry_ip_address_branch' | 06:36 |
| sshnaidm | because we need to detect if it's consumer job | 06:36 |
| sshnaidm | it's not for guarding there, it's for detecton | 06:37 |
| *** jbadiapa has joined #oooq | 06:38 | |
| *** ysandeep|afk is now known as ysandeep | 06:38 | |
| ykarel | sshnaidm, chandankumar is this known issue in provider jobs? async task did not complete within the requested time - 5700s | 06:48 |
| ykarel | during container builds | 06:48 |
| ykarel | seen in multiple runs | 06:48 |
| sshnaidm | ykarel, not that I'm aware of | 06:49 |
| ykarel | like https://395c71011536eee1042b-03cab02d8b4f23536b6a5db1f6ea92b1.ssl.cf2.rackcdn.com/757605/5/check/tripleo-ci-centos-8-content-provider-train/8dcef7c/job-output.txt | 06:49 |
| ykarel | https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_b0b/757605/5/check/tripleo-ci-centos-8-content-provider-ussuri/b0b7c31/job-output.txt | 06:49 |
| ykarel | sshnaidm, ack please check it fails randomly and impacts gates | 06:49 |
| sshnaidm | ykarel, sure | 06:50 |
| sshnaidm | soniya29|ruck, can you please submit a bug? ^ | 06:50 |
| sshnaidm | unfortunately build log doesn't have timestamps.. | 06:52 |
| ykarel | sshnaidm, also https://5b2971f3dc79fa7015d5-14387ee33f74a27c93fba699ca02403e.ssl.cf5.rackcdn.com/758218/1/check/tripleo-ci-centos-8-containers-multinode-ussuri/d9d24f5/logs/undercloud/home/zuul/undercloud_install.log one | 06:58 |
| ykarel | seperate issue | 06:58 |
| ykarel | build containers and downloaded containers different | 06:59 |
| sshnaidm | ykarel, need to see provider job of it | 07:00 |
| ykarel | 10cbdc7673d1dff9ad577075047fe56c downloading, built ed49f62fc6b62b41dcbd7bf3751b7df2 | 07:00 |
| sshnaidm | ykarel, if it's different dlrn hashes, we have patches for solving it | 07:00 |
| ykarel | yes different dlrn hashes | 07:01 |
| sshnaidm | ykarel, it means was promotion between, afaiu | 07:01 |
| ykarel | not sure i checked last two promotion and atleast it didnt' metched | 07:01 |
| sshnaidm | ykarel, anyways https://review.opendev.org/#/c/758236/ | 07:01 |
| ykarel | 10cbdc7673d1dff9ad577075047fe56c latest promotion, facbbe458004a24e8b46574cc09e0417 second last | 07:02 |
| sshnaidm | ykarel, hmm.. that's weird | 07:02 |
| sshnaidm | ykarel, where is provider job log? | 07:02 |
| ykarel | sshnaidm, https://184bdcb3da6f14107fec-6cf4452aeae42378118fa7e7779efd66.ssl.cf1.rackcdn.com/758218/1/check/tripleo-ci-centos-8-content-provider-ussuri/7d87850/logs/undercloud/var/log/extra/podman/podman_allinfo.log | 07:02 |
| sshnaidm | ykarel, they have some different repos in components | 07:04 |
| sshnaidm | how to know what is md5 hash from delorean.repo? | 07:04 |
| ykarel | sshnaidm, if delorean.repo is unchanged, md5sum <repo path> | 07:05 |
| ykarel | but in ci it's likely changed by adding mirror urls | 07:05 |
| sshnaidm | DLRN_PATH_TAG=current-tripleo/ed/49/ed49f62fc6b62b41dcbd7bf3751b7df2 https://184bdcb3da6f14107fec-6cf4452aeae42378118fa7e7779efd66.ssl.cf1.rackcdn.com/758218/1/check/tripleo-ci-centos-8-content-provider-ussuri/7d87850/logs/undercloud/home/zuul/repo_setup.log | 07:06 |
| sshnaidm | in 2020-10-14 23:01:59 | 07:06 |
| sshnaidm | ykarel, should have been promotion for that ^ | 07:07 |
| ykarel | sshnaidm, checked again ed49f62fc6b62b41dcbd7bf3751b7df2 is second last promotion, but somehow https://trunk.rdoproject.org/centos8-ussuri/previous-current-tripleo/delorean.repo.md5 is not updated | 07:08 |
| ykarel | may be some bug there | 07:08 |
| sshnaidm | ykarel, ok, so it's between promotion as we thought, we have patches to solve it: https://review.opendev.org/#/q/topic:pipeline-tags+(status:open+OR+status:merged) | 07:09 |
| ykarel | facbbe458004a24e8b46574cc09e0417 is 4th last | 07:09 |
| ykarel | sshnaidm, okk, will check those in some time | 07:09 |
| ykarel | fyi i checked using dlrnapi --url https://trunk.rdoproject.org/api-centos8-ussuri promotion-get --promote-name current-tripleo --limit 70|grep aggreg|grep -v null | 07:09 |
| marios | soniya29|ruck: fyi we are switching scen12 non voting https://review.opendev.org/#/c/757898/ for https://review.opendev.org/#/c/757605 (https://launchpad.net/bugs/1895822 ) | 07:10 |
| openstack | Launchpad bug 1895822 in tripleo "centos8 standalone-upgrade-ussuri fails tempest ping router IP" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 07:10 |
| *** ykarel is now known as ykarel|afk | 07:10 | |
| sshnaidm | ykarel|afk, isn't there a more convenient way to see them.. like some updated table somewhere | 07:10 |
| sshnaidm | maybe task for grafana, weshay * | 07:10 |
| sshnaidm | ^ | 07:10 |
| ykarel|afk | sshnaidm, will check later, power out here, laptop going to die | 07:11 |
| sshnaidm | ok :) | 07:11 |
| *** ykarel|afk has quit IRC | 07:11 | |
| *** jpena|off is now known as jpena | 07:19 | |
| *** ksambor has joined #oooq | 07:27 | |
| *** yolanda__ has joined #oooq | 07:27 | |
| chandankumar | marios: may be I will do a followup patch | 07:30 |
| chandankumar | sshnaidm: please press +w on these https://review.opendev.org/#/c/752419/ and https://review.opendev.org/#/c/752421/ | 07:32 |
| *** holser has joined #oooq | 07:38 | |
| sshnaidm | yeah, let's polish it in followups | 07:38 |
| chandankumar | finally we did it | 07:39 |
| chandankumar | thank you marios & sshnaidm :-) | 07:39 |
| sshnaidm | chandankumar, \o/ | 07:39 |
| sshnaidm | chandankumar, marios please review in your time dlrn hashes for consumers: https://review.opendev.org/#/q/topic:pipeline-tags+(status:open+OR+status:merged) | 07:40 |
| *** dtantsur|afk is now known as dtantsur | 07:41 | |
| marios | sshnaidm: ack adding to reviews | 07:42 |
| *** tosky has joined #oooq | 07:42 | |
| ysandeep | Hey folks, Is it expected to run different set of tempest tests in component vs integration line jobs? In below scenario component ovb jobs are running less tempest tests. | 07:45 |
| ysandeep | https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-tripleo-train/77a780c/logs/undercloud/var/log/tempest/stestr_results.html.gz | 07:45 |
| ysandeep | https://logserver.rdoproject.org/openstack-periodic-integration-stable2/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-train/bd07b5c/logs/undercloud/var/log/tempest/stestr_results.html.gz | 07:45 |
| soniya29|ruck | marios ack | 07:46 |
| chandankumar | ysandeep: yes, https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset001.yml#L205 | 07:46 |
| chandankumar | ysandeep: more details here https://opendev.org/openstack/tripleo-quickstart/commit/95ce02b0bb815e0dedb65344b89eb1f15f161d21 | 07:47 |
| ysandeep | chandankumar: thanks a lot! checking | 07:47 |
| *** ysandeep is now known as ysandeep|lunch | 08:03 | |
| soniya29|ruck | sshnaidm ack | 08:04 |
| *** yolanda__ has quit IRC | 08:24 | |
| *** yolanda__ has joined #oooq | 08:24 | |
| bhagyashris | zbr, hi good morning ! | 08:38 |
| *** derekh has joined #oooq | 08:39 | |
| bhagyashris | zbr, victoria full integration pipeline is ready https://review.rdoproject.org/r/#/c/30038/ it will merge soon. whats our plan about pipeline naming 1. should we just renaming the template name or we are going to change pipeline names? | 08:42 |
| zbr | morning! | 08:42 |
| zbr | yep | 08:43 |
| soniya29|ruck | sshnaidm, https://bugs.launchpad.net/tripleo/+bug/1899904 | 08:45 |
| openstack | Launchpad bug 1899904 in tripleo "tripleo-ci-centos-8-content-provider job fails due to async task did not complete within the requested time " [Medium,New] | 08:45 |
| bhagyashris | zbr, are you going to work on that? | 08:48 |
| *** skramaja_ has quit IRC | 08:58 | |
| *** skramaja_ has joined #oooq | 08:58 | |
| sshnaidm | soniya29|ruck, thanks | 09:01 |
| soniya29|ruck | sshanidm :) | 09:01 |
| *** udesale__ has joined #oooq | 09:03 | |
| *** udesale_ has quit IRC | 09:06 | |
| *** ysandeep|lunch is now known as ysandeep | 09:24 | |
| *** jmasud has quit IRC | 09:59 | |
| chandankumar | soniya29|ruck: {"ansible_job_id": "563885111524.168935", "ansible_loop_var": "tripleo_keystone_resources_project_async_result_item", "attempts": 30, "changed": false, "finished": 0, "started": 1, "tripleo_keystone_resources_project_async_result_item": {"ansible_job_id": "563885111524.168935", "ansible_loop_var": "tripleo_keystone_resources_project", "changed": true, "failed": false, "finished": 0, | 10:03 |
| chandankumar | "results_file": "/root/.ansible_async/563885111524.168935", "started": 1, "tripleo_keystone_resources_project": "service"}} | 10:03 |
| chandankumar | soniya29|ruck: https://api.us-east.open-edge.io:8080/swift/v1/AUTH_e02c11e4e2c24efc98022353c88ab506/zuul_opendev_logs_377/752419/52/gate/tripleo-ci-centos-8-containers-multinode-train/3778216/logs/undercloud/home/zuul/overcloud_deploy.log | 10:03 |
| chandankumar | is this one known? seeing currently in gate in one my patch | 10:04 |
| chandankumar | sshnaidm: marios so we are good with job.consumer_job condition with upgrade patch https://review.opendev.org/#/c/753708/ ? | 10:32 |
| sshnaidm | chandankumar, I think so | 10:34 |
| marios | chandankumar: should work ... if it is a consumer job and the registry_ip_address isn't set we have problem it should just fail | 10:34 |
| chandankumar | marios: ok, will wait for results | 10:34 |
| marios | chandankumar: sshnaidm: by the same logic we should fix that https://opendev.org/openstack/tripleo-quickstart/src/commit/bc1be15b1b29b395691fb9a9f3d9baf7ef63f5e1/config/release/tripleo-ci/CentOS-8/master.yml#L15-L18 | 10:35 |
| sshnaidm | marios, yes, and all such conditions | 10:35 |
| marios | sshnaidm: k | 10:36 |
| sshnaidm | I hope to handle this after we merge upgrades, not to mess with rebases | 10:36 |
| sshnaidm | akahat, fyi https://review.opendev.org/#/q/topic:more_tags+(status:open+OR+status:merged) | 10:40 |
| sshnaidm | marios, can you please change it here too? https://review.opendev.org/#/c/755472/12/config/release/tripleo-ci/CentOS-8/train.yml | 10:43 |
| sshnaidm | and ussuri https://review.opendev.org/#/c/755472/12/config/release/tripleo-ci/CentOS-8/ussuri.yml | 10:43 |
| marios | sshnaidm: ack will do in bit trying to figure out layouts for tripleo-ci review | 10:44 |
| *** rascasoft has joined #oooq | 10:44 | |
| sshnaidm | ok | 10:44 |
| *** yolanda__ is now known as yolanda | 10:52 | |
| sshnaidm | chandankumar, I think we have an issue with non-tripleo containers here: https://review.opendev.org/#/c/757186/ | 10:57 |
| sshnaidm | chandankumar, can you take a look at standalone 001 | 10:57 |
| chandankumar | sshnaidm: sure | 10:57 |
| sshnaidm | chandankumar, the job didn't detect any: https://4cea344168f83bb0cf48-0e8dbb243f96d1f5bf8da84efb739aff.ssl.cf2.rackcdn.com/757186/2/check/tripleo-ci-centos-8-content-provider/c7f1da5/logs/undercloud/home/zuul/non_tripleo_containers.log | 10:59 |
| sshnaidm | maybe centos 7/8 issue | 10:59 |
| *** udesale__ has quit IRC | 11:03 | |
| *** ysandeep is now known as ysandeep|afk | 11:07 | |
| chandankumar | weired issue | 11:09 |
| chandankumar | let me reproduce locally | 11:09 |
| soniya29|ruck | chandankumar, i am not aware of this issue | 11:14 |
| *** udesale has joined #oooq | 11:15 | |
| sshnaidm | marios, don't we make a separate template for upgrades? | 11:19 |
| chandankumar | sshnaidm: issue reproduced | 11:19 |
| sshnaidm | chandankumar, ack | 11:20 |
| chandankumar | for train https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/roles/container-build/tasks/non_tripleo_containers.yml#L15 | 11:20 |
| chandankumar | it is not working | 11:20 |
| chandankumar | looking into why | 11:20 |
| sshnaidm | not working in centos8? | 11:20 |
| sshnaidm | chandankumar, I think we don't use kolla on c8 in train | 11:21 |
| chandankumar | sshnaidm: it was my mistake | 11:21 |
| chandankumar | sshnaidm: yes, we donot use kolla for c8 | 11:22 |
| chandankumar | sshnaidm: https://review.opendev.org/758400 | 11:25 |
| marios | sshnaidm: no we haven't done that before... yea we need to discuss i guess you've seen the problem with the multiple content providers | 11:27 |
| marios | sshnaidm: not sure what the right solution is... | 11:27 |
| marios | sshnaidm: i mean we haven't had a separate template for upgrades before | 11:28 |
| sshnaidm | marios, yeah, I think we agreed to have one | 11:28 |
| sshnaidm | on one of mtgs | 11:28 |
| sshnaidm | marios, anyways, it would be better, no need to merge branchful providers to other templates | 11:28 |
| marios | sshnaidm: k, we need to discuss again today then... even if we have own template, the result will be the same, i.e. yhou will include the upgrade template in the same places anyway | 11:28 |
| holser | afternoon folks | 11:28 |
| marios | sshnaidm: so we will still have the providers issue | 11:28 |
| sshnaidm | marios, why would we have providers issue? | 11:29 |
| marios | sshnaidm: i mean multiple providers will run | 11:29 |
| sshnaidm | marios, yeah, with upgrades job it's must | 11:29 |
| marios | sshnaidm: that issue... but lets discuss in 1 hour | 11:29 |
| sshnaidm | but not all patches or repos require upgrades jobs I think | 11:29 |
| sshnaidm | ok, we'll talk in a hour | 11:29 |
| holser | marios quick question.... If change a has Depends-on: B and B has depends-on: C and C is not merged so we'll get > This change depends on a change that failed to merge. Am I correct? | 11:30 |
| holser | marios I am talking about https://review.opendev.org/#/c/739457/ | 11:30 |
| sshnaidm | chandankumar, will it work for both train c7 and c8? https://review.opendev.org/#/c/758400/ | 11:32 |
| chandankumar | sshnaidm: yes | 11:32 |
| sshnaidm | cool | 11:32 |
| *** jpena is now known as jpena|lunch | 11:32 | |
| chandankumar | sshnaidm: sorry | 11:33 |
| sshnaidm | chandankumar, nothing to sorry :) | 11:33 |
| chandankumar | sshnaidm: for c7, I need to call non_tripleo_container playbook in the container builds role | 11:33 |
| chandankumar | more hack | 11:34 |
| chandankumar | for c7 queens - i need to update the tripleo-common file to switch to quay | 11:35 |
| chandankumar | and fix non_tripleo containers for queens | 11:36 |
| *** ratailor has quit IRC | 11:37 | |
| marios | holser: sorry dealing with local issue looking | 11:39 |
| holser | marios thanks a lot | 11:39 |
| marios | holser: problem is that was updated i think https://review.opendev.org/#/c/757605/ | 11:55 |
| marios | holser: so when you update something in the depends-on the parent job will fail with 'failed to merge' | 11:55 |
| marios | holser: is that the questino | 11:55 |
| marios | holser: (sorry was called away again but back now) | 11:55 |
| holser | ok | 11:55 |
| holser | marios right... I guess I updated patches in wrong order | 11:57 |
| holser | so, I'll recheck | 11:57 |
| holser | those patches | 11:57 |
| marios | chandankumar: sshnaidm: we are skipping the content provider call today? something about merging it with scrum? was it for today or i misremember | 11:58 |
| sshnaidm | oh, right | 11:58 |
| chandankumar | marios: yes correct merging with scrum | 11:58 |
| marios | sshnaidm: chandankumar: ack k we can continue after scrum if we need | 11:58 |
| chandankumar | marios: sure | 12:01 |
| *** rlandy has joined #oooq | 12:01 | |
| *** rlandy is now known as rlandy|rover | 12:02 | |
| rlandy|rover | amoralej: hi - thanks for pointing out the issue with priority in the centos8 stream repos. How did setting the priority stop the ceph dependencies from being installed though? | 12:04 |
| amoralej | rlandy|rover, because it forces to install librados2 from appstreams repo | 12:05 |
| amoralej | but python-rados2 is not on that repo, but only in storagesig | 12:05 |
| amoralej | so the version of python-rados2 and librados2 didn't match | 12:05 |
| rlandy|rover | hmm ... ok - we want some mechanism that ensured the dependency repos under test would take preference | 12:06 |
| rlandy|rover | but we did disable the other centos repos so it's probably enough | 12:07 |
| *** amoralej is now known as amoralej|lunch | 12:10 | |
| weshay | hrm.. | 12:10 |
| weshay | 2020-10-15 06:06:27.723661 | fa163ea9-b62f-69a6-9d48-000000000c6b | FATAL | Container image prepare | undercloud | error={"changed": false, "error": "Not found image: http://198.72.124.110:5001/v2/tripleomaster/openstack-swift-container/manifests/09dfd5d2324dabfff402bc47ef288de8", "msg": "Error running container image prepare: Not found image: http://198.72.124.110:5001/v2/tripleomaster/openstack-swift-container/manifests/09dfd5d2324dabfff | 12:10 |
| weshay | 402bc47ef288de8", "params": {}, "success": false} | 12:10 |
| weshay | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5da/757683/1/gate/tripleo-ci-centos-8-undercloud-containers/5da016a/logs/undercloud/home/zuul/undercloud_install.log | 12:10 |
| *** rf0lc0 has joined #oooq | 12:15 | |
| weshay | rf0lc0, 0/ I have to join the osp-ci scrum today.. can you cover our scrum? | 12:16 |
| zbr | chandankumar: weshay: cleanup of integration template names: https://review.rdoproject.org/r/#/q/topic:pipeline-renaming+(status:open+OR+status:merged) | 12:19 |
| rf0lc0 | weshay, yes | 12:20 |
| chandankumar | zbr: will take a look | 12:21 |
| zbr | chandankumar: fyi, no behavior change is made, that is a 3 step rename of template files (as required by zuul). | 12:21 |
| weshay | zbr, k.. needs to be socialized | 12:22 |
| rlandy|rover | soniya29|ruck: weshay: hey - not sure if you saw my question from yesterday re: tempest on periodic-tripleo-ci-centos-7-standalone-stein | 12:22 |
| zbr | weshay: wasn't socializing done two days ago? "any one against?... silence" ;) | 12:22 |
| rlandy|rover | soniya29|ruck: how is this test running for stein if it's in the skip list? | 12:22 |
| rlandy|rover | https://logserver.rdoproject.org/openstack-periodic-integration-stable3/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-7-standalone-stein/c86251c/logs/undercloud/home/zuul/tempest/tempest.html.gz | 12:23 |
| rlandy|rover | weshay: hmm .. now we have the introspection failure on ci.centos master | 12:24 |
| rlandy|rover | will have to look into that today | 12:24 |
| rlandy|rover | so master and ussuri | 12:24 |
| weshay | rlandy|rover, ya.. we may have to run locally | 12:25 |
| weshay | if it works locally .. we can waive the test | 12:25 |
| rlandy|rover | k | 12:27 |
| chandankumar | marios: are we merging patch to tripleo-common in stable/queens branch? | 12:28 |
| chandankumar | I need to quay.io switch patches there | 12:28 |
| chandankumar | *need to send | 12:28 |
| weshay | kopecmartin, 0/ can you please point me at the infraredspec file for collect logs | 12:28 |
| soniya29|ruck | rlandy|rover, let me check | 12:29 |
| chandankumar | weshay: time for 1:1 | 12:30 |
| marios | chandankumar: looks like it? "1 month ago" https://opendev.org/openstack/tripleo-common/commit/84755aaba8449ec4c8d109219fc37907af82c062 | 12:30 |
| marios | chandankumar: https://review.opendev.org/#/q/status:merged+project:openstack/tripleo-common+branch:stable/queens | 12:30 |
| rf0lc0 | marios, chandankumar sshnaidm: joining us ? | 12:32 |
| *** jpena|lunch is now known as jpena | 12:32 | |
| rf0lc0 | https://meet.google.com/jtj-ppwn-wpx?authuser=1 | 12:32 |
| zbr | i realised that using stageN in template names was a big mistake as one cannot simply rename them. for pipelines is debatable, more of a preference but for templates is clear: they need to specify what they contain | 12:32 |
| marios | rf0lc0: we are going to skip it today chandankumar asked us to merge it with scrum | 12:32 |
| marios | chandankumar: you forgot to tell rf0lc0 | 12:32 |
| rf0lc0 | k | 12:32 |
| rf0lc0 | **nice** | 12:33 |
| *** ysandeep|afk is now known as ysandeep | 12:35 | |
| kopecmartin | weshay: hi, what collect logs you mean? the ansible-role-collect-logs or the collect-logs plugin which ispart of infrared repo? | 12:35 |
| weshay | ansible-role-collect-logs | 12:35 |
| kopecmartin | all infrared files are here: https://github.com/openstack/ansible-role-collect-logs/tree/master/infrared_plugin | 12:36 |
| rf0lc0 | akahat, look at comments and depends-on from https://review.opendev.org/#/c/757186 | 12:39 |
| akahat | rf0lc0, ack | 12:42 |
| soniya29|ruck | rlandy|rover, yes..the test is in skiplist and it is failing as well | 12:44 |
| rlandy|rover | soniya29|ruck: how is that possible? | 12:45 |
| rlandy|rover | skip list does not run | 12:45 |
| soniya29|ruck | rlandy|rover, weshay: yeah..i guess skiplist hasn't really ran | 12:46 |
| rlandy|rover | soniya29|ruck: there is a train and master specific list. do we require one for stein? | 12:47 |
| rlandy|rover | soniya29|ruck: can you create a launchpad bug for that so we can track and fix it? don;t know what the exact issue is offhand | 12:48 |
| rlandy|rover | would need to ask arxcruz and friends | 12:48 |
| soniya29|ruck | rlandy|rover, the idea behind the tempest-skiplist was to have one single common skiplist, hence it should be enough but it seems it is not enough | 12:49 |
| chandankumar | weshay: https://bugs.launchpad.net/tripleo/+bug/1899904 | 12:49 |
| openstack | Launchpad bug 1899904 in tripleo "tripleo-ci-centos-8-content-provider job fails due to async task did not complete within the requested time " [Medium,New] | 12:49 |
| * arxcruz reading | 12:49 | |
| rlandy|rover | soniya29|ruck: ack - let's create a bug so we can ask at today's meeting | 12:49 |
| soniya29|ruck | rlandy|rover, give me a sec | 12:49 |
| arxcruz | rlandy|rover: soniya29|ruck yes, it needs one for stein | 12:49 |
| soniya29|ruck | rlandy|rover, let me create the bug and then we can track its progress | 12:50 |
| arxcruz | rlandy|rover: soniya29|ruck i'll create a command to update the skiplist automatically adding the release | 12:50 |
| rlandy|rover | arxcruz: soniya29|ruck: thanks | 12:50 |
| soniya29|ruck | arxcruz: thanks :) | 12:51 |
| *** udesale_ has joined #oooq | 12:54 | |
| ysandeep | weshay, rlandy|rover when you have time please review https://code.engineering.redhat.com/gerrit/#/c/215099/ - | 12:55 |
| ysandeep | Skip image building in fs001 ovb job in integration line | 12:55 |
| ysandeep | Test run is green, Job is downloading images from right place | 12:55 |
| *** udesale has quit IRC | 12:56 | |
| rlandy|rover | soniya29|ruck: here is another tempest one ... | 12:58 |
| rlandy|rover | https://logserver.rdoproject.org/69/29969/7/check/periodic-tripleo-ci-centos-7-ovb-1ctlr_2comp-featureset020-queens/c4e4840/logs/undercloud/home/zuul/tempest/tempest.html.gz | 12:58 |
| weshay | ysandeep, why force periodic? | 12:59 |
| weshay | periodic-tripleo-ci-rhel-8-ovb-3ctlr_1comp-featureset001-internal-rhos-16.2: | 12:59 |
| weshay | vars: | 12:59 |
| weshay | force_periodic: true | 12:59 |
| rlandy|rover | neutron_tempest_plugin.scenario.test_floatingip.FloatingIPQosTest | 12:59 |
| *** amoralej|lunch is now known as amoralej | 12:59 | |
| rlandy|rover | soniya29|ruck: same error in master ... https://logserver.rdoproject.org/openstack-periodic-integration-main/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master/6af1230/logs/undercloud/var/log/tempest/stestr_results.html.gz | 12:59 |
| ysandeep | weshay, oops that was just for testing.. updated the patchset | 12:59 |
| *** udesale__ has joined #oooq | 13:00 | |
| rf0lc0 | arxcruz, rfolco, zbr, sshnaidm, rlandy, marios, ysandeep, bhagyashris, svyas, pojadhav, akahat, weshay, chandankumar | 13:01 |
| weshay | rlandy|rover, mtg soniya29|ruck if only for a minute | 13:01 |
| weshay | please join | 13:01 |
| *** ykarel has joined #oooq | 13:01 | |
| weshay | akahat ^ | 13:01 |
| rlandy|rover | joining | 13:02 |
| akahat | weshay, joining. | 13:02 |
| *** udesale_ has quit IRC | 13:02 | |
| rf0lc0 | zbr, magic | 13:03 |
| rf0lc0 | lol | 13:03 |
| chandankumar | weshay: rf0lc0 https://review.opendev.org/757643 | 13:23 |
| chandankumar | rlandy|rover: rf0lc0 https://review.rdoproject.org/r/#/c/30024/ will give clear picture for rocky & stein | 13:25 |
| arxcruz | chandankumar: https://review.rdoproject.org/r/30019 | 13:27 |
| chandankumar | arxcruz: will take a look, thanks! | 13:27 |
| weshay | kopecmartin, thanks.. so I think this is the variable I'm looking for https://github.com/openstack/ansible-role-collect-logs/blob/master/infrared_plugin/plugin.spec#L94 | 13:31 |
| weshay | where can I see the value for that here https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/QE/view/OSP16.2/job/phase1-16.2_director-rhel-8.3-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph/ | 13:31 |
| weshay | don't see it | 13:31 |
| sshnaidm | rf0lc0, akahat https://review.opendev.org/#/q/topic:more_tags+(status:open+OR+status:merged) | 13:32 |
| weshay | poking around http://rhos-ci-logs.lab.eng.tlv2.redhat.com/logs/rcj/jenkins-phase1-16.2_director-rhel-8.3-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph-63/infrared/ | 13:33 |
| soniya29|ruck | rlandy|rover, arxcruz, weshay, https://bugs.launchpad.net/tripleo/+bug/1899980 | 13:34 |
| openstack | Launchpad bug 1899980 in tripleo "tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port fails in stein inspite of test moved in skiplist" [Undecided,New] | 13:34 |
| migi | weshay: https://code.engineering.redhat.com/gerrit/gitweb?p=rhos-qe-jenkins.git;a=blob;f=jobs/defaults/stages/ir_collect_logs.groovy.inc;h=d8927819f6dc7157bf266befb5968fc6eb1e94c1;hb=HEAD#l103 | 13:35 |
| weshay | migi, it's not in the job log? | 13:35 |
| weshay | thanks btw | 13:35 |
| rlandy|rover | soniya29|ruck: pls see what you think about the queens and master fs020 failures: neutron_tempest_plugin.scenario.test_floatingip.FloatingIPQosTest | 13:35 |
| migi | weshay: it's in the job config | 13:35 |
| migi | weshay: https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/QE/view/OSP16.2/job/phase1-16.2_director-rhel-8.3-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph/configure | 13:36 |
| soniya29|ruck | rlandy|rover sure | 13:36 |
| migi | weshay: -> Pipeline -> Script text field it's there | 13:36 |
| arxcruz | soniya29|ruck: i'll work on that | 13:36 |
| soniya29|ruck | arxcruz: sure | 13:37 |
| rf0lc0 | https://review.rdoproject.org/r/#/q/topic:pipeline-renaming+(status:open+OR+status:merged) | 13:42 |
| kopecmartin | weshay: console output -> https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/view/QE/view/OSP16.2/job/phase1-16.2_director-rhel-8.3-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph/63/console | 13:42 |
| kopecmartin | weshay: then search for collect-artcl.log | 13:42 |
| kopecmartin | and there will be a link for the output of the ansible-*-logs only -> https://rhos-ci-jenkins.lab.eng.tlv2.redhat.com/job/phase1-16.2_director-rhel-8.3-virthost-1cont_1comp_1ceph-ipv4-geneve-ceph/63/artifact/.sh/collect-artcl.log.log | 13:43 |
| kopecmartin | and the job uses these collect_log_types: container,monitoring,network,openstack,system | 13:43 |
| *** TrevorV has joined #oooq | 13:44 | |
| sshnaidm | marios, please review/merge dlrn patches: https://review.opendev.org/#/q/topic:pipeline-tags+(status:open+OR+status:merged) | 13:47 |
| sshnaidm | it's failing jobs | 13:47 |
| marios | sshnaidm: ack will check in bit | 13:47 |
| marios | sshnaidm: checking now | 13:48 |
| bhagyashris | zbr, hi can we add victoria template here https://review.rdoproject.org/r/#/c/30328/ itself | 13:49 |
| zbr | better to keep it separated. use the same topic but another CR | 13:50 |
| arxcruz | soniya29|ruck: https://review.opendev.org/#/c/758430/ this should fix it | 13:51 |
| bhagyashris | ack | 13:51 |
| *** sshnaidm has quit IRC | 13:54 | |
| *** sshnaidm has joined #oooq | 14:00 | |
| soniya29|ruck | rlandy|rover, patch is up for stein which will work for master as well, so need of separate bug right? | 14:05 |
| rlandy|rover | soniya29|ruck: if it's the same fix then one bug is fine | 14:06 |
| soniya29|ruck | rlandy|rover, fix is same | 14:06 |
| rlandy|rover | weshay: you still on the osp meeting? | 14:06 |
| weshay | no.. df now | 14:08 |
| rlandy|rover | okie dokie | 14:10 |
| chandankumar | rf0lc0: https://review.opendev.org/#/c/758400/ will fix train content provide non-tripleo containers issue | 14:19 |
| rf0lc0 | chandankumar, I saw thanks. But docker.io is still on prepare parameter yaml file, so I added this one on top of your patch: https://review.opendev.org/#/c/758214 | 14:20 |
| chandankumar | rf0lc0: logs? | 14:21 |
| rf0lc0 | chandankumar, why prepare parameter is ok on master (ok = registry is from content provider ip), and not in stable branches ? | 14:21 |
| rf0lc0 | chandankumar, with pleasure, sec | 14:21 |
| chandankumar | rf0lc0: it was coming with stable train only | 14:22 |
| rf0lc0 | chandankumar, what was coming? | 14:22 |
| chandankumar | rf0lc0: https://4cea344168f83bb0cf48-0e8dbb243f96d1f5bf8da84efb739aff.ssl.cf2.rackcdn.com/757186/2/check/tripleo-ci-centos-8-content-provider/c7f1da5/logs/undercloud/home/zuul/non_tripleo_containers.log | 14:23 |
| rf0lc0 | chandankumar, look at this change, https://review.opendev.org/#/c/757917/, https://api.us-east.open-edge.io:8080/swift/v1/AUTH_e02c11e4e2c24efc98022353c88ab506/zuul_opendev_logs_ee9/757917/2/check/tripleo-ci-centos-8-scenario004-standalone/ee92344/logs/undercloud/home/zuul/containers-prepare-parameters.yaml | 14:23 |
| rf0lc0 | chandankumar, what I was referring was " namespace: docker.io/tripleotraincentos8" | 14:23 |
| chandankumar | rf0lc0: that is wrong, need to take a look why it is coming | 14:24 |
| rf0lc0 | chandankumar, two different issues | 14:24 |
| rf0lc0 | for this reason I added https://review.opendev.org/#/c/758214/3/roles/extras-common/defaults/main.yml | 14:24 |
| rf0lc0 | chandankumar, but to be honest I don't know why master worked without this | 14:25 |
| rf0lc0 | in the same file, look for the ceph var... | 14:25 |
| rf0lc0 | docker_ceph_namespace: >- | 14:25 |
| rf0lc0 | {% if job.consumer_job|default(false)|bool -%} | 14:25 |
| rf0lc0 | it gets from registry_ip_address | 14:25 |
| chandankumar | rf0lc0: something is overriding it | 14:25 |
| chandankumar | will take a look in some time | 14:25 |
| rf0lc0 | so I did the same for docker_registry_host | 14:26 |
| rf0lc0 | chandankumar, my patch fixed it in testproject... so now I am testing with both your patch and mine, to get the job green.... look the registry_ip_address at https://logserver.rdoproject.org/63/30263/2/check/tripleo-ci-centos-8-scenario004-standalone/c1852ad/logs/undercloud/home/zuul/containers-prepare-parameters.yaml.txt.gz | 14:28 |
| weshay | zbr, rf0lc0 help me verify my understanding.. py3 is available on the latest centos-7 nodepool nodes ya? | 14:30 |
| rf0lc0 | 7.8 yes | 14:31 |
| zbr | yes, but afaik is not installed by default | 14:31 |
| zbr | exactly, before 7.8 we did not had it, at least not py36. | 14:31 |
| rf0lc0 | weshay, installed by default yes in 7.8 | 14:31 |
| rf0lc0 | zbr, the last time I put some debug messages there, it was pre-installed | 14:32 |
| rf0lc0 | but with dual python, python2 prevails | 14:32 |
| zbr | rf0lc0: may or may not be, can change at any type based on what decisions the image gods make. | 14:32 |
| weshay | rf0lc0, zbr k.. I'll have to look but /me wondering if we can setup tq w/ py3 on centos-7 train | 14:32 |
| *** apetrich has joined #oooq | 14:32 | |
| zbr | still, adding bindep.txt cannot hurt. | 14:33 |
| weshay | would help matbu out.. for validations.. /me will look into it | 14:33 |
| rf0lc0 | zbr, weshay, yes, its available anyway, if not in the image | 14:33 |
| rf0lc0 | but keep in mind some packages won't be available, you need to install from pip | 14:34 |
| rf0lc0 | I don't recall which ones, 2 or 3 if I remember right | 14:34 |
| *** udesale_ has joined #oooq | 14:35 | |
| rlandy|rover | bhagyashris: hey your patch 757861,4 is very red in the gate | 14:35 |
| rlandy|rover | legit failure? | 14:36 |
| zbr | rf0lc0: lxml and few others but they are installable using pip. in fact py36 from centos-7 is in a very good shape, better than i was expecting. | 14:36 |
| ysandeep | weshay, to drop ussuri bm job to free envD https://code.engineering.redhat.com/gerrit/#/c/215192/ for 16.2 more jobs | 14:36 |
| rlandy|rover | 2020-10-15 13:34:47.957010 | fa163edd-07fc-7467-e594-000000000bf4 | FATAL | Container image prepare | undercloud | error={"changed": false, "error": "Failed to parse: https://2607:ff68:100:54:f816:3eff:fe87:a228:5001/v2/", "msg": "Error running container image prepare: Failed to parse: https://2607:ff68:100:54:f816:3eff:fe87:a228:5001/v2/", "params": {}, "success": false} | 14:37 |
| rlandy|rover | ysandeep: weshay: I vote yes on that | 14:37 |
| rlandy|rover | we can take the env is we need it | 14:37 |
| *** udesale__ has quit IRC | 14:37 | |
| ysandeep | rlandy|rover: thanks! yes we need it for more jobs.. currently for testing.. | 14:38 |
| rlandy|rover | 2607:ff68:100:54:f816:3eff:fe87:a228:5001/ | 14:40 |
| rlandy|rover | weird | 14:40 |
| *** ysandeep is now known as ysandeep|away | 14:43 | |
| weshay | ysandeep|away, rlandy|rover yes | 14:44 |
| weshay | wf | 14:44 |
| rlandy|rover | ysandeep|away: weshay: if 16.2 passes - will promote today | 14:46 |
| *** skramaja_ has quit IRC | 14:48 | |
| weshay | anyone need anything? | 15:02 |
| rlandy|rover | weshay: any chance you can join the horizon selenium meeting in an hour? | 15:02 |
| * rlandy|rover doesn't want to commit to mpre | 15:03 | |
| rlandy|rover | more | 15:03 |
| weshay | rf0lc0, did there end up being an extra step to enable py3 on c7? | 15:03 |
| rlandy|rover | although we kind of already did | 15:03 |
| weshay | rlandy|rover, sure | 15:03 |
| rlandy|rover | thanks | 15:03 |
| rf0lc0 | weshay, extra step ? | 15:03 |
| weshay | rf0lc0, maybe not an extra step.. /me doesn't remember tbh. | 15:04 |
| weshay | can one just run py3 or py2? | 15:04 |
| weshay | and it's avail? | 15:04 |
| weshay | no scl or other b.s. | 15:04 |
| weshay | you did this for centos7 image builds | 15:04 |
| rf0lc0 | weshay, yes but its very confusing for ansible if you just run modules expecting py3 in a dual python env | 15:05 |
| rf0lc0 | weshay, no need for scl | 15:05 |
| rf0lc0 | weshay, we have everything in 7.8, plus 2 or 3 packages missing to be installed via pip | 15:05 |
| *** jmasud has joined #oooq | 15:14 | |
| weshay | zbr, matbu https://review.opendev.org/758453 worth a shot | 15:21 |
| zbr | weshay: do we have any strong reason preventing us from switching to python3 always, not condition/detection? | 15:24 |
| zbr | do we still need to support old centos, versions with qs? i suspect not quite. | 15:24 |
| weshay | zbr, I think we probably could | 15:24 |
| zbr | no condition means more predictable behavior. | 15:24 |
| weshay | zbr, I can't think of a need to support old centos | 15:24 |
| weshay | queens is supported on rhel7.9 so | 15:25 |
| weshay | matbu, ^ | 15:25 |
| weshay | zbr, the only issue I can think of | 15:25 |
| weshay | is the following | 15:25 |
| weshay | if these jokers.. https://ci.centos.org/view/rdo/view/promotion-pipeline/ | 15:26 |
| weshay | don't have py3 enabled w/o an scl | 15:26 |
| zbr | afaik, with qs we are supposed to support only the latest base os releases, which means 7.x/8.x and we have pure py36 on both without extra repos or scl tricks. | 15:27 |
| zbr | this means there is nothing stopping us from going pure py36 in our requirements. | 15:28 |
| *** chem has quit IRC | 15:28 | |
| zbr | i see only benefits from this | 15:28 |
| weshay | zbr, same here | 15:29 |
| weshay | matbu, ^ just keeping you in the loop | 15:29 |
| marios | brb | 15:30 |
| *** marios has quit IRC | 15:30 | |
| *** chem has joined #oooq | 15:32 | |
| *** marios has joined #oooq | 15:34 | |
| weshay | ykarel++ as always :) | 15:39 |
| ykarel | weshay, what's cat /tmp/foo? | 15:43 |
| ykarel | seems testing bits | 15:43 |
| weshay | oh lord | 15:44 |
| ykarel | also i see some syntax issues | 15:44 |
| weshay | I should pay attention | 15:44 |
| ykarel | missing ;then and = around vars | 15:44 |
| ykarel | missing ;then and = around vars | 15:45 |
| weshay | who even let me in here.. jebus | 15:46 |
| weshay | ykarel, you should kick ban me | 15:47 |
| ykarel | i will fix some other misses | 15:58 |
| weshay | rlandy|rover, mtg | 16:01 |
| chandankumar | rf0lc0: https://review.opendev.org/#/c/758214/3 | 16:12 |
| chandankumar | rf0lc0: few nit picks | 16:12 |
| rf0lc0 | chandankumar, thanks | 16:15 |
| matbu | weshay: thx | 16:18 |
| weshay | rlandy|rover, ok.. done and done | 16:23 |
| rlandy|rover | weshay: thanks ... just don't want to commit to more we can't focus on now | 16:24 |
| *** ysandeep|away is now known as ysandeep | 16:27 | |
| ysandeep | rlandy|rover, ovb job will timeout today as well - https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/021f24e176bb43dba4a65b24daa7e669?logfile=console.log | 16:28 |
| ysandeep | from last few days - seeing jobs sometime stuck on TASK [tripleo-inventory : Ensure gather_facts has been run against localhost] *** | 16:28 |
| *** udesale_ has quit IRC | 16:29 | |
| ysandeep | not sure of reason tbh.. i need more ansible debug skills to troubleshoot these timeouts | 16:29 |
| *** marios is now known as marios|out | 16:32 | |
| *** jpena is now known as jpena|off | 16:35 | |
| *** dtantsur is now known as dtantsur|afk | 16:39 | |
| *** marios|out has quit IRC | 16:39 | |
| *** amoralej is now known as amoralej|off | 16:40 | |
| *** derekh has quit IRC | 17:00 | |
| *** rascasoft has quit IRC | 17:16 | |
| *** ykarel is now known as ykarel|away | 17:19 | |
| ysandeep | rlandy|rover, fyi.. testing a hit and trial for above ovb timeout https://code.engineering.redhat.com/gerrit/215257 .. we hit something similiar for multinode couple of months back and running through single playbook helped. | 17:23 |
| rlandy|rover | ysandeep: sorry - just busy on other channel - will look in a bit | 17:23 |
| ysandeep | rlandy|rover, nw :) will discuss tomorrow.. have a good day ahead! | 17:24 |
| *** ysandeep is now known as ysandeep|away | 17:24 | |
| *** apetrich has quit IRC | 17:32 | |
| weshay | rf0lc0, so.. it may save us time to also make openstack-tox-molecule and tox-linters parents to jobs in tqe | 17:35 |
| weshay | example 753708,26 in the gate now | 17:35 |
| weshay | molecule failed, but integration moves on | 17:36 |
| rf0lc0 | weshay, sorry I don't have context on what you are speaking | 17:37 |
| weshay | rf0lc0, look at the upstream zuul gate | 17:37 |
| weshay | for tripleo | 17:37 |
| weshay | look at the last job.. in there.. 753708,26 | 17:37 |
| weshay | see how openstack-tox-molecule is failed | 17:38 |
| weshay | other repos have the molecule and tox jobs as parents | 17:38 |
| weshay | so if those fail, we fail fast | 17:38 |
| weshay | know what I mean? | 17:38 |
| rf0lc0 | ok, so molecule and tox are dependencies for many jobs in different repos... you want to fail fast on these deps by adding them as deps across all repos | 17:40 |
| rf0lc0 | weshay, did I get the proposal right ? | 17:40 |
| weshay | rf0lc0, well.. I think if we look at something like tripleo-common.. in the layout.. we'll see that parent/child relationship | 17:41 |
| * weshay looks | 17:41 | |
| rf0lc0 | weshay, tripleo-common has *deps_unit_lint... yeah | 17:41 |
| rf0lc0 | so if basic jobs fail, all others are skipped | 17:41 |
| weshay | rf0lc0, https://opendev.org/openstack/tripleo-common/src/branch/master/zuul.d/layout.yaml#L21-L26 | 17:42 |
| rf0lc0 | yep, ok, so apply the same idea across all repos ? | 17:42 |
| weshay | rf0lc0, vs.. https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/zuul.d/layout.yaml | 17:42 |
| weshay | tbh.. I think this should be done in the layout.. | 17:43 |
| weshay | https://opendev.org/openstack/tripleo-quickstart/src/branch/master/zuul.d/layout.yaml | 17:43 |
| rf0lc0 | yes we are speaking of layout | 17:43 |
| weshay | ya.. | 17:43 |
| rf0lc0 | ok, proposing change in ci repos first | 17:44 |
| rf0lc0 | weshay, regarding scen001/scen004, I got a green run in testproject, https://review.rdoproject.org/r/#/c/30263/ | 17:47 |
| rf0lc0 | weshay, includes 2 fixes in tqe ^ | 17:48 |
| weshay | rf0lc0, that registry change looks good, and consistent w/ the variable ALWAYS having the branch name | 17:49 |
| weshay | thanks! | 17:49 |
| rf0lc0 | weshay, cool | 17:50 |
| *** jmasud has quit IRC | 17:57 | |
| rf0lc0 | weshay, click on like and subscribe to my channel for more, don't forget to activate the bell to receive notifications | 18:01 |
| rf0lc0 | https://review.opendev.org/758473 Fastfail on mol/lint jobs | 18:01 |
| rlandy|rover | ugh - linters | 18:01 |
| rf0lc0 | https://review.opendev.org/758475 Fastfail on lint jobs | 18:08 |
| weshay | rf0lc0, The key "dependencies" appears more than once; duplicate keys are not | 18:22 |
| rf0lc0 | oops | 18:23 |
| rf0lc0 | weshay, fixing | 18:23 |
| weshay | rlandy|rover, are we able to hold our own nodes internally? | 18:26 |
| weshay | rlandy|rover, getting yum install errorrs.. https://sf.hosted.upshift.rdu2.redhat.com/logs/78/213478/15/check/tripleo-ci-rhel-7-multinode-1ctlr-featureset010/688be05/logs/undercloud/home/zuul/undercloud_install.log | 18:27 |
| weshay | however the packages are there.. https://sf.hosted.upshift.rdu2.redhat.com/logs/78/213478/15/check/tripleo-ci-rhel-7-multinode-1ctlr-featureset010/688be05/logs/undercloud/var/log/extra/all_available_packages.txt | 18:27 |
| rlandy|rover | weshay: no - but we can log on to them | 18:27 |
| * rlandy|rover looks | 18:28 | |
| weshay | OH.. | 18:28 |
| weshay | you told me that | 18:28 |
| * weshay rekicks | 18:28 | |
| rlandy|rover | np | 18:30 |
| rlandy|rover | weshay: actually I lie - I can hold nodes | 18:31 |
| rlandy|rover | I have a token ... getting | 18:31 |
| * rlandy|rover forgot about that | 18:31 | |
| weshay | rlandy|rover, ooh.. is that something we can doc and share? | 18:31 |
| rlandy|rover | see pvt | 18:31 |
| *** ykarel|away has quit IRC | 18:43 | |
| *** saneax has quit IRC | 18:52 | |
| *** apetrich has joined #oooq | 19:16 | |
| *** apetrich has quit IRC | 19:32 | |
| *** jmasud has joined #oooq | 19:41 | |
| *** TrevorV has quit IRC | 19:52 | |
| *** jmasud has quit IRC | 19:58 | |
| *** jtomasek has quit IRC | 19:59 | |
| *** jmasud has joined #oooq | 20:20 | |
| *** dsneddon has joined #oooq | 20:30 | |
| *** jmasud has quit IRC | 20:36 | |
| *** jmasud has joined #oooq | 20:40 | |
| rlandy|rover | ugh - not liking my ipa changes | 20:58 |
| rlandy|rover | weshay, tox-molecule broken | 21:00 |
| weshay | ha.. was just looking at that | 21:00 |
| weshay | ran it locally | 21:00 |
| weshay | came up w/ a diff error | 21:01 |
| rlandy|rover | https://zuul.openstack.org/builds?job_name=openstack-tox-molecule | 21:01 |
| weshay | ImportError: cannot import name 'ansible_version' | 21:01 |
| rlandy|rover | failed my job | 21:01 |
| rlandy|rover | did we change something? | 21:01 |
| weshay | if it fails in gate.. doubtful | 21:02 |
| weshay | ah.. running from tqe now.. sorry.. I tried tripleo-ci | 21:03 |
| weshay | ya.. recreates | 21:03 |
| *** jfrancoa has quit IRC | 21:04 | |
| weshay | rlandy|rover, all I see on this .. is the molecule-requirments ansible<2.10 | 21:08 |
| weshay | molecule installed: ansi2html==1.5.2,ansible==2.9.14,arrow==0.17.0,attrs==20.2.0,bcrypt==3.2.0,binaryornot==0.4.4,Cerberus==1.3.2,certifi==2020.6.20,cffi==1.14.3,chardet==3.0.4,click==7.1.2,click-completion==0.5.2,click-help-colors==0.8,colorama==0.4.4,cookiecutter==1.7.2,coverage==5.3,cryptography==3.1.1,distro==1.5.0,docker==4.3.1,fasteners==0.15,idna==2.10,importlib-metadata==2.0.0,iniconfig==1.1.1,Jinja2==2.11.2,jinja2-time==0.2.0,MarkupSaf | 21:09 |
| weshay | e==1.1.1,mock==4.0.2,molecule==3.0.8,monotonic==1.5,packaging==20.4,paramiko==2.7.2,pathspec==0.8.0,pexpect==4.8.0,pluggy==0.13.1,poyo==0.5.0,ptyprocess==0.6.0,py==1.9.0,pycparser==2.20,PyNaCl==1.4.0,pyparsing==2.4.7,pytest==6.1.1,pytest-cov==2.10.1,pytest-html==2.1.1,pytest-metadata==1.10.0,pytest-molecule==1.3.1,python-dateutil==2.8.1,python-gilt==1.2.3,python-slugify==4.0.1,PyYAML==5.3.1,requests==2.24.0,selinux==0.2.1,sh==1.13.1,shellingham | 21:09 |
| weshay | ==1.3.2,six==1.15.0,tabulate==0.8.7,text-unidecode==1.3,toml==0.10.1,tree-format==0.1.2,-e | 21:09 |
| weshay | it install 2.9.14 | 21:09 |
| rlandy|rover | weshay: just in the last few hours | 21:09 |
| rlandy|rover | weird | 21:09 |
| rlandy|rover | quickstart and extras | 21:10 |
| weshay | rf0lc0, you have any ideas? | 21:10 |
| rlandy|rover | 2020-10-15 17:29:11.230934 | ubuntu-bionic | ImportError: cannot import name 'ansible_version' | 21:10 |
| rlandy|rover | what is that? | 21:10 |
| weshay | seems like a global | 21:11 |
| weshay | issue | 21:11 |
| weshay | nope.. just us https://zuul.openstack.org/builds?job_name=openstack-tox-molecule | 21:12 |
| rlandy|rover | no obvious changes to tripleo-quickstart or tqe | 21:13 |
| rlandy|rover | weshay: creating LP bug in the mean time | 21:13 |
| *** jbadiapa has quit IRC | 21:17 | |
| rlandy|rover | weshay: https://bugs.launchpad.net/tripleo/+bug/1900033 | 21:19 |
| openstack | Launchpad bug 1900033 in tripleo "openstack-tox-molecule jobs are failing with "ImportError: cannot import name 'ansible_version'"" [Critical,Triaged] | 21:19 |
| rlandy|rover | ansible==2.9.14 on passing jobs as well | 21:23 |
| *** jmasud has quit IRC | 21:25 | |
| rlandy|rover | pytest-molecule==1.3.1 pytest-molecule==1.3.0 | 21:26 |
| rlandy|rover | weshay: ^^ here's the difference | 21:26 |
| rlandy|rover | weshay: putting in change to limit pytest-molecule version | 21:29 |
| rlandy|rover | until we can talk wit zbr | 21:29 |
| rlandy|rover | with | 21:29 |
| rlandy|rover | rf0lc0: zbr: soniya29|ruck: weshay: https://review.opendev.org/#/q/topic:bug1900033+(status:open+OR+status:merged) | 21:40 |
| weshay | aye | 21:41 |
| rlandy|rover | periodic-tripleo-build-containers-ubi-8-internal-rhel-8-build-push-upload-rhos-17 | 21:42 |
| rlandy|rover | success | 21:42 |
| rlandy|rover | wow | 21:42 |
| rlandy|rover | weshay: ^^ | 21:42 |
| rlandy|rover | maybe we can have a debuggable build now | 21:43 |
| weshay | oh.. nice.. first build after we promoted the components? | 21:43 |
| rlandy|rover | 17 is on the run | 21:43 |
| rlandy|rover | no yesterday's failed container builds | 21:43 |
| rlandy|rover | this is the first run with promoted components and passing containers | 21:44 |
| rlandy|rover | so yeah - the first workable build | 21:44 |
| rlandy|rover | image builds passing | 21:44 |
| rlandy|rover | weshay: better luck with rhos-13 job since change t the node? | 21:45 |
| weshay | no.. file is still not there | 21:45 |
| rlandy|rover | k | 21:45 |
| rlandy|rover | we still have another tempest issue in rdo promotions | 21:45 |
| rlandy|rover | will chat with soniya29|ruck tomorrow | 21:45 |
| rlandy|rover | https://zuul.openstack.org/builds?job_name=tripleo-ci-centos-8-containers-multinode | 21:47 |
| rlandy|rover | another one looking less than healthy | 21:47 |
| weshay | that's not a good way determine it | 21:48 |
| rlandy|rover | looking at gate failures | 21:48 |
| * rlandy|rover has to go - will check in later | 22:08 | |
| *** rlandy|rover is now known as rlandy|rover|bbl | 22:09 | |
| *** jmasud has joined #oooq | 22:44 | |
| *** holser has quit IRC | 23:18 | |
| *** holser has joined #oooq | 23:19 | |
| *** tosky has quit IRC | 23:23 | |
| *** jmasud has quit IRC | 23:27 | |
| *** holser has quit IRC | 23:43 | |
| *** ysandeep|away is now known as ysandeep | 23:59 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!