| *** jmasud has quit IRC | 01:10 | |
| *** dsneddon has quit IRC | 01:21 | |
| *** jmasud has joined #oooq | 01:35 | |
| *** jmasud has quit IRC | 01:47 | |
| *** apetrich has quit IRC | 02:09 | |
| *** rlandy has quit IRC | 02:12 | |
| *** jmasud has joined #oooq | 03:26 | |
| *** jmasud has quit IRC | 04:05 | |
| *** jmasud has joined #oooq | 04:07 | |
| *** ysandeep|away is now known as ysandeep | 04:10 | |
| *** ykarel|away is now known as ykarel | 04:13 | |
| *** jmasud has quit IRC | 04:24 | |
| *** jmasud has joined #oooq | 04:42 | |
| *** ratailor has joined #oooq | 04:50 | |
| *** ysandeep is now known as ysandeep|brb | 04:58 | |
| *** jmasud has quit IRC | 05:15 | |
| *** jmasud has joined #oooq | 05:28 | |
| *** ysandeep|brb is now known as ysandeep | 05:29 | |
| *** dsneddon has joined #oooq | 05:42 | |
| *** dsneddon has quit IRC | 05:47 | |
| *** udesale has joined #oooq | 05:57 | |
| *** dpawlik has joined #oooq | 06:03 | |
| *** jfrancoa has joined #oooq | 06:05 | |
| *** marios has joined #oooq | 06:08 | |
| *** jmasud has quit IRC | 06:18 | |
| *** jmasud has joined #oooq | 06:19 | |
| *** yolanda has joined #oooq | 06:28 | |
| *** ccamacho has joined #oooq | 06:37 | |
| *** skramaja has joined #oooq | 06:57 | |
| *** jmasud has quit IRC | 07:01 | |
| *** apetrich has joined #oooq | 07:03 | |
| *** bogdando has joined #oooq | 07:05 | |
| *** jtomasek has joined #oooq | 07:06 | |
| *** matbu has quit IRC | 07:09 | |
| *** matbu has joined #oooq | 07:10 | |
| *** jmasud has joined #oooq | 07:12 | |
| *** jpena|off is now known as jpena | 07:20 | |
| marios | arxcruz: rebase https://review.rdoproject.org/r/#/c/26683/ | 07:23 |
|---|---|---|
| matbu | hi folks, do you have more info about https://bugs.launchpad.net/tripleo/+bug/1874535 | 07:30 |
| openstack | Launchpad bug 1874535 in tripleo "tripleo-validations; unable to discover unit tests, breaking the gate" [Critical,Triaged] | 07:30 |
| matbu | ? | 07:31 |
| matbu | maybe jpena ? | 07:31 |
| matbu | sshnaidm|afk: or | 07:31 |
| marios | o/ matbu panda|ruck is on duty this week | 07:31 |
| marios | matbu: checking the ruck|rover hackmd now | 07:31 |
| matbu | marios: o/ thw how are you ? | 07:32 |
| marios | matbu: so it didn't help ? https://review.opendev.org/#/c/722485/ 4 hours ago merged | 07:32 |
| matbu | s/thw/thx/ | 07:32 |
| marios | matbu: o/ hey man :D hows it going | 07:32 |
| marios | matbu: sorry didn't see somethign in the ruck|rover pad about that | 07:33 |
| jpena | matbu: is this happening consistently on that review? I think I have seen the same behavior elsewhere, but it worked on a recheck (which is a shame) | 07:33 |
| marios | matbu: https://hackmd.io/1pY-KQB_QwOe-a-5oEXTRg panda|ruck can we add that to title | 07:33 |
| *** tosky has joined #oooq | 07:33 | |
| matbu | jpena: yep i think we already see such kind of failure last week :/ | 07:33 |
| marios | matbu: so you're still seeing that after the merge to disable validations? | 07:34 |
| matbu | marios: jpena well my concerns is why it didn't build correctly | 07:35 |
| matbu | it should not failed on unit test discovery | 07:35 |
| matbu | so i'm wondering if it's not an infra thing or .. | 07:35 |
| matbu | i'll check the history there, but afaik there was no new change in that repo recently that could break something | 07:36 |
| marios | matbu: well unless there is some other breaking change upstream | 07:36 |
| marios | matbu: i am not in context with that bug though first time i read it | 07:36 |
| marios | matbu: maybe even in validations-common Tengu are you aware of that bug? https://bugs.launchpad.net/tripleo/+bug/1874535 | 07:37 |
| openstack | Launchpad bug 1874535 in tripleo "tripleo-validations; unable to discover unit tests, breaking the gate" [Critical,Triaged] | 07:37 |
| marios | Tengu: https://review.opendev.org/#/c/722485/2/roles/build-test-packages/defaults/main.yml | 07:38 |
| matbu | marios: yep he is :D we were discussing about it few minutes ago and conclude .... we need CI folks help :D | 07:38 |
| Tengu | marios: yeah, got a ping yesterday from weshay|ruck :/ | 07:38 |
| marios | Tengu: one of those happy ones :D | 07:38 |
| marios | :/ sorry | 07:38 |
| Tengu | yup. | 07:38 |
| Tengu | and I fail to understand why it's failing now, and why | 07:38 |
| Tengu | especially since running 'tox -epy36' here is all green | 07:38 |
| matbu | i will try to build it locally | 07:39 |
| jpena | hm, validations-common... aha, I knew I remembered something | 07:39 |
| jpena | https://trunk.rdoproject.org/centos8-ussuri/component/tripleo/99/05/99050941f49a8752c8b2a7f75af12039f21c1afe_26f41efa/rpmbuild.log | 07:39 |
| marios | Tengu: so why don't you post the revert for that anyway so its ready.. looks like panda|ruck and akahat|ruck are not aware of it yet | 07:39 |
| marios | lets give them some time to investigate | 07:39 |
| jpena | matbu: ^^ so it happened a couple days ago in the trunk.rdo builder. It was just for one commit, the next ones worked fine | 07:39 |
| marios | matbu: Tengu: ack on the green local... so i see now what matbu meant by 'infra' issue might be version of something used | 07:39 |
| Tengu | and I can't use my lab right now, already running things for another topic -.-' | 07:39 |
| jpena | I suspect there's something triggering the issue at random | 07:39 |
| Tengu | jpena: "at random". you know how to whisper soft words to my ears -.-' | 07:40 |
| * jpena hates heisenbugs, too | 07:40 | |
| Tengu | :) | 07:40 |
| Tengu | schrodinbug? | 07:41 |
| *** jtomasek has quit IRC | 07:41 | |
| matbu | jpena: hmm weird | 07:41 |
| jpena | actually yes, wrong scientist | 07:41 |
| Tengu | hehe | 07:41 |
| marios | Tengu: jpena: best part about those is once you've fixed it you still have no idea if you fixed it cos schrodinbug | 07:41 |
| marios | you just wait and see and hope for the best :D | 07:42 |
| Tengu | marios: exactly. a bug in a closed box. | 07:42 |
| marios | Tengu: it both is and isn't fixed | 07:42 |
| marios | all the time | 07:42 |
| Tengu | until you can open that fscking box | 07:42 |
| Tengu | anyway, it was apparently failing on a regular basis since those packages were dropped. | 07:43 |
| marios | me staring at the trace | 07:43 |
| Tengu | so it's maybe not THAT schroedinbug-ish | 07:43 |
| Tengu | (yeah, oe, because รถ in German) | 07:44 |
| matbu | jpena: maybe it's related tot he last commit (victoria unit tests) https://github.com/openstack/validations-common/commits/master | 07:44 |
| matbu | ? | 07:44 |
| marios | Tengu: ? https://github.com/openstack/validations-common/commit/327e7a9b5625dd433b0b1e1d86b704eee05fdbfd | 07:45 |
| Tengu | marios: so we can propose the revert of the deactivation and, then, add a depends-on the thing we think is the solution, right? | 07:45 |
| jpena | matbu: the issue in RDO Trunk happened with commit 99050941f49a8752c8b2a7f75af12039f21c1afe | 07:45 |
| marios | Tengu: that would cause a zuul fail i think i.e. if that template is missing or something or you mean something else | 07:45 |
| marios | Tengu: yes that sounds right | 07:45 |
| Tengu | https://review.opendev.org/722626 | 07:46 |
| Tengu | feel free to edit its commit message to add any needed depends-on, matbu :) | 07:46 |
| Tengu | but please keep the -w in place so that nobody will cry ;) | 07:47 |
| marios | Tengu: how can it already be in merge conflict wtf | 07:47 |
| marios | o_O | 07:47 |
| Tengu | whaattt | 07:47 |
| * marios goes back to bed | 07:47 | |
| marios | exception | 07:47 |
| *** ccamacho has quit IRC | 07:47 | |
| matbu | marios: :D | 07:47 |
| Tengu | lemme do the revert manually. | 07:48 |
| *** ccamacho has joined #oooq | 07:48 | |
| Tengu | oh. | 07:48 |
| Tengu | ok. | 07:48 |
| * marios goes back to reviews | 07:49 | |
| marios | ping if there is somethign i can do/review etc | 07:49 |
| Tengu | so at first the patch did more than just listing those packages | 07:49 |
| matbu | marios: s/reviews/bed/ ? :D | 07:50 |
| * marios can do code reviews in bed | 07:50 | |
| marios | whats the problem | 07:50 |
| Tengu | hehe | 07:50 |
| Tengu | sleep reviews? | 07:50 |
| Tengu | uncoffeeinated reviews | 07:50 |
| Tengu | schroedinview | 07:51 |
| Tengu | :D | 07:51 |
| Tengu | anyway. the patch is now OK, no more merge conflict. still -w in order to lock it down. | 07:51 |
| Tengu | marios: where are those template supposed to be located btw? | 07:52 |
| Tengu | so that we can ensure we actually have this openstacp-python3-victoria-jobs | 07:52 |
| matbu | jpena: so this commit is breaking "sometimes" or every build ? | 07:52 |
| jpena | matbu: sometimes | 07:52 |
| matbu | erf | 07:52 |
| jpena | https://trunk.rdoproject.org/api-centos8-ussuri/api/report.html?package=validations-common | 07:52 |
| Tengu | o_O | 07:53 |
| Tengu | it was apparently failing enough to lock the gate though | 07:53 |
| marios | Tengu: in some zuul.d directory | 07:53 |
| marios | Tengu: upstream i'd have to look for it | 07:53 |
| Tengu | marios: supposed to be in the project itself, or in some other repos? | 07:53 |
| marios | Tengu: could come from another repo that is included in the zuul job | 07:54 |
| matbu | i'm going to add a comment to the LP | 07:54 |
| Tengu | "yay" | 07:54 |
| marios | Tengu: hold on hound should tell us | 07:54 |
| marios | https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/zuul.d/project-templates.yaml | 07:54 |
| marios | Tengu: ^ | 07:54 |
| *** bogdando has quit IRC | 07:54 | |
| Tengu | https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/zuul.d/project-templates.yaml#L457 | 07:55 |
| Tengu | so it's defined. | 07:55 |
| jpena | oh! | 07:55 |
| jpena | I found something | 07:55 |
| Tengu | jpena: share, share! | 07:55 |
| marios | Tengu: yea its used in lots of places http://codesearch.openstack.org/?q=openstack-python3-victoria-jobs | 07:55 |
| jpena | the issue was reproduced outside the DLRN build environment, too | 07:55 |
| matbu | jpena: great ! | 07:55 |
| jpena | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_183/719598/3/gate/openstack-tox-py36/183a9aa/job-output.txt | 07:55 |
| * matbu cross fingers | 07:56 | |
| *** jtomasek has joined #oooq | 07:56 | |
| Tengu | hmm | 07:56 |
| marios | now going back to reviews | 07:56 |
| matbu | jpena: this is the same trace back that is in the LP | 07:57 |
| * Tengu goes back to bed^Wwork | 07:57 | |
| jpena | right | 07:57 |
| *** ysandeep is now known as ysandeep|lunch | 07:57 | |
| jpena | but it's happening on a normal tox-py36 job | 07:57 |
| jpena | on ubuntu | 07:57 |
| Tengu | ubuntu.... | 07:57 |
| * Tengu prepares the flamethrower | 07:57 | |
| chandankumar | panda|ruck: akahat|ruck we are updating ansible version https://review.rdoproject.org/r/#/c/25914/ for ussuri | 07:58 |
| Tengu | .... and now podman is failing on my laptop. pfffff. | 08:03 |
| jpena | Tengu: I recently had an issue with podman after an upgrade, is it complaining about some semaphore/mutexes? | 08:05 |
| Tengu | limits. | 08:05 |
| Tengu | just dropped ~/.config/container directory | 08:05 |
| Tengu | and cleaned the local storage as well. | 08:05 |
| Tengu | lovely podman -.-. | 08:05 |
| Tengu | now: ERROR: Could not open requirements file: [Errno 2] No such file or directory: ' https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt' | 08:06 |
| jpena | right, that's what happened to me. Removing an old config file somewhere under .config/container worked | 08:06 |
| Tengu | so I can't run tox in my ubuntu:bionic container for #reason. great. | 08:08 |
| *** gchamoul has joined #oooq | 08:14 | |
| *** dtantsur|afk is now known as dtantsur | 08:17 | |
| jpena | Tengu: if it helps reproduce, I've been running "tox -epy36" on a fresh git checkout on my laptop (fedora 31), and it failed once after 100 attempts | 08:19 |
| matbu | jpena: 100 attempts :'( | 08:22 |
| Tengu | -.- | 08:25 |
| Tengu | have to jump on some mtg | 08:25 |
| gchamoul | jpena: just checking all the configuration files which could put us on a lead | 08:34 |
| *** derekh has joined #oooq | 08:37 | |
| *** bogdando has joined #oooq | 09:01 | |
| *** ykarel is now known as ykarel|lunch | 09:05 | |
| *** ysandeep|lunch is now known as ysandeep | 09:08 | |
| *** dtantsur is now known as dtantsur|brb | 09:15 | |
| *** jtomasek has quit IRC | 09:20 | |
| *** jmasud has quit IRC | 09:27 | |
| *** jmasud has joined #oooq | 09:39 | |
| *** jmasud has quit IRC | 09:51 | |
| panda|ruck | marios: in containers build, is there a way to access more detailed logs ? like for example, if a container fails on dnf install step, is there a way to get logs on dnf command ? | 09:54 |
| marios | panda|ruck: yea look in logs/ | 09:54 |
| marios | panda|ruck: buildah builds | 09:55 |
| marios | https://sf.hosted.upshift.rdu2.redhat.com/logs/58/196858/71/check/periodic-tripleo-build-containers-rhel-8-master/4998724/logs/buildah-builds/kolla-builds/882b52ca-6f90-4859-aac9-d6a9918d9cfe/docker/ | 09:55 |
| marios | panda|ruck: eg ^ | 09:55 |
| marios | panda|ruck: inside each is the build log | 09:55 |
| panda|ruck | marios: ah thanks https://logserver.rdoproject.org/openstack-periodic-master/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-centos-8-master-containers-build-push/36c9a4a/logs/buildah-builds/kolla-builds/39d7c573-c3fc-4796-91ab-eac4a0646b0d/docker/openvswitch/openvswitch-base/openvswitch-base-build.log | 09:56 |
| panda|ruck | network hiccup probably | 09:56 |
| marios | panda|ruck: ack | 09:56 |
| marios | panda|ruck: have you seen that yet/anywhere else 2020-04-24 09:28:52 | sed: -e expression #1, char 47: unterminated `s' command | 10:07 |
| marios | https://logserver.rdoproject.org/76/26676/3/check/tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039/9b3b503/logs/undercloud/home/zuul/ipa_prep.sh.log.txt.gz | 10:07 |
| marios | panda|ruck: probably not if its ipa specific ... | 10:08 |
| marios | panda|ruck: new thing now happening in the check job which was green until yesterday https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039 | 10:08 |
| panda|ruck | marios: no, akahat|ruck have you seen that ? ^ | 10:08 |
| marios | panda|ruck: ah hold on i think i was looking at a testproject result | 10:09 |
| marios | panda|ruck: so might be something in their patch | 10:09 |
| marios | panda|ruck: never mind... looks like i was ... both fails there are from testproject | 10:09 |
| marios | https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-8-ovb-3ctlr_1comp_1supp-featureset039 | 10:09 |
| panda|ruck | marios: that's what I was lookign for, but the patch is in zuul layout | 10:09 |
| * marios nervous around fs39 | 10:10 | |
| marios | panda|ruck: yeah but has 3 depends on | 10:10 |
| marios | panda|ruck: and i was going crazy, looking in hound/grep for some task, which in fact is only now being added there https://review.opendev.org/#/c/721615/ | 10:10 |
| panda|ruck | marios: yeah, stop looking at check jobs :) | 10:12 |
| marios | panda|ruck: it's fs39... cursed i tell ya | 10:13 |
| *** jbadiapa has joined #oooq | 10:23 | |
| *** jmasud has joined #oooq | 10:27 | |
| panda|ruck | arxcruz: what's the status of tempest component jobs ? are they stable ? do I have to report bugs ? | 10:29 |
| arxcruz | panda|ruck: sorry, what do you mean ? | 10:30 |
| panda|ruck | arxcruz: jobs in the tempest component pipeline are failing, and I wanted to understand if they are supposed to be stable | 10:30 |
| arxcruz | panda|ruck: sorry, i wasn't monitoring these jobs, do you have a link ? | 10:31 |
| arxcruz | to the failure | 10:31 |
| panda|ruck | arxcruz: same failure as others, IP allocation problem | 10:32 |
| *** ykarel|lunch is now known as ykarel | 10:32 | |
| arxcruz | panda|ruck: can i have a look ? | 10:33 |
| panda|ruck | arxcruz: no | 10:33 |
| arxcruz | ok | 10:33 |
| panda|ruck | arxcruz: it's mine, MINE | 10:33 |
| panda|ruck | arxcruz: my tempessssssssst | 10:33 |
| arxcruz | lol | 10:33 |
| panda|ruck | arxcruz: https://logserver.rdoproject.org/openstack-component-tempest/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-tempest-master/d4dab46/logs/undercloud/var/log/tempest/stestr_results.html.gz | 10:34 |
| arxcruz | kopecmartin: do you have any idea? I believe it's concurrency | 10:35 |
| panda|ruck | arxcruz: there's a CIX bug open for that error | 10:35 |
| kopecmartin | there was a bug already with a similar issue i think | 10:36 |
| kopecmartin | let me find it | 10:36 |
| arxcruz | i believe it's because concurrency and the component have small amount of ip to be allocated | 10:36 |
| arxcruz | maybe add more ip availability | 10:36 |
| kopecmartin | https://bugs.launchpad.net/tripleo/+bug/1852770/ | 10:37 |
| openstack | Launchpad bug 1852770 in tripleo "No more IP addresses available on network, neutron_tempest_plugin.scenario.test_security_groups.NetworkSecGroupTest" [High,In progress] - Assigned to wes hayutin (weshayutin) | 10:37 |
| kopecmartin | check the comment #8 | 10:37 |
| arxcruz | kopecmartin: panda|ruck https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/playbooks/tempest.yml#L46 | 10:46 |
| arxcruz | maybe if we increase the ip allocation here... | 10:46 |
| chandankumar | arxcruz: https://opendev.org/openstack/openstack-ansible-os_tempest/src/branch/master/defaults/main.yml#L139 | 10:47 |
| chandankumar | if we check default one | 10:47 |
| chandankumar | they have 50 | 10:47 |
| arxcruz | chandankumar: the default is ommited | 10:47 |
| arxcruz | and we are setting to 20 in our side | 10:47 |
| chandankumar | arxcruz: I mean in example | 10:48 |
| arxcruz | chandankumar: yeah, maybe we should increase ours... | 10:48 |
| chandankumar | arxcruz: yes | 10:48 |
| arxcruz | let me work on that | 10:48 |
| chandankumar | cool, thanks! | 10:48 |
| arxcruz | chandankumar: panda|ruck: https://review.rdoproject.org/r/26775 | 10:55 |
| arxcruz | the test project | 10:55 |
| arxcruz | and https://review.opendev.org/722662 the patch for increase ip allocation | 10:55 |
| Tengu | gchamoul, matbu anything new? | 10:55 |
| gchamoul | can't reproduce locally, with and without tox ... | 10:56 |
| Tengu | meh... | 10:56 |
| gchamoul | trying to get a python3.6 locally | 10:56 |
| Tengu | 'k. will give some more test to the podman +ubuntu:bionic now. | 10:57 |
| gchamoul | because when I run things, I use python3.7 | 10:57 |
| Tengu | I have a small free time right now before new bunch of meetings. | 10:57 |
| chandankumar | arxcruz: if it passes, it can mark it closes bug | 10:58 |
| gchamoul | never see that behaviour locally before, only one time in the gate and it didn't worry myself.... Mistake! | 10:59 |
| arxcruz | chandankumar: yeah, I believe even if it passes, it's better to keep as it is for a few runs to make sure | 10:59 |
| chandankumar | arxcruz: akahat|ruck kopecmartin soniya29 https://hackmd.io/fIOKlEBHQfeTZjZmrUaEYQ?view please add or edit it | 11:06 |
| Tengu | meh... for some reasons I'm unable to run the tox command in an ubuntu:bionic container. | 11:07 |
| gchamoul | Tengu: just need virtualenv | 11:08 |
| Tengu | gchamoul: yeah well, I love having an env as close as possible to what is used ;). | 11:08 |
| Tengu | but since I also happen to have a 3.6 venv... | 11:09 |
| gchamoul | Tengu: to me it's not a tox issue | 11:09 |
| Tengu | source ../venv-3.6/bin/activate | 11:09 |
| Tengu | gchamoul: at least it's something in the python part called by tox. I would find really funky having a broken mirror with wrong version for one of the dependency of the unit tests | 11:12 |
| Tengu | that would explain the randomness. | 11:12 |
| *** dtantsur|brb is now known as dtantsur | 11:14 | |
| *** jmasud has quit IRC | 11:18 | |
| *** jtomasek has joined #oooq | 11:21 | |
| *** jpena is now known as jpena|lunch | 11:28 | |
| *** jtomasek has quit IRC | 11:28 | |
| weshay|ruck | akahat|ruck, panda|ruck give me a couple min please.. I'll be there in a few | 11:28 |
| akahat|ruck | weshay|ruck, okay | 11:29 |
| *** pojadhav is now known as pojadhav|brb | 11:37 | |
| weshay|ruck | akahat|ruck, http://dashboard-ci.tripleo.org/d/Z_UNB29Wz/third-party-dependency-check?orgId=1 | 11:43 |
| kopecmartin | chandankumar: does the issue mean we need to edit spec file and tune the dependencies? https://review.rdoproject.org/r/#/c/26769/ | 11:46 |
| chandankumar | kopecmartin: I think yes, rdo-train spec file, before we updated it to 23.0 | 11:47 |
| chandankumar | you will get the patch | 11:47 |
| chandankumar | *check the rdo-train | 11:47 |
| chandankumar | kopecmartin: https://github.com/rdo-packages/tempest-distgit/commit/1ea2cc50c46fe2783649c003c6bc7fd005e2bd2f | 11:49 |
| chandankumar | kopecmartin: just cherry pick this one | 11:49 |
| chandankumar | and make sure it is available for stein | 11:49 |
| chandankumar | check in deps.yaml file | 11:49 |
| kopecmartin | chandankumar: thanks, i'm on it | 11:49 |
| chandankumar | better it is tagged or not for stein | 11:49 |
| weshay|ruck | panda|ruck, akahat|ruck https://review.opendev.org/#/c/722662 | 11:50 |
| panda|ruck | marios: chandankumar review.enqueue_async(https://review.rdoproject.org/r/26694) | 11:54 |
| marios | panda|ruck: created ticket 123315511s1 | 11:57 |
| panda|ruck | marios: ACK. | 11:57 |
| marios | panda|ruck: RST | 11:58 |
| panda|ruck | marios: TIME_WAIT | 11:59 |
| weshay|ruck | panda|ruck, akahat|ruck https://github.com/rdo-infra/ci-config/tree/master/ci-scripts/infra-setup/roles/rrcockpit | 12:00 |
| weshay|ruck | panda|ruck, akahat|ruck https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/roles/rrcockpit/files/development_script.sh | 12:01 |
| kopecmartin | chandankumar: according to pypi, the dep is part of sphinxcontrib-svg2pdfconverter package which is not in requirements for stein https://github.com/openstack/requirements/tree/stable/stein | 12:02 |
| kopecmartin | chandankumar: what in that case? | 12:03 |
| weshay|ruck | ysandeep, https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic-master&job_name=periodic-tripleo-ci-centos-8-scenario012-standalone-master%09 | 12:10 |
| zbr | weshay|ruck: panda|ruck chandankumar: please take another look at https://review.opendev.org/#/c/722402 -- check comments and see if new set looks good. | 12:10 |
| weshay|ruck | ysandeep, https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/CentOS-8/master.ini#L27 | 12:11 |
| zbr | it still keeps one multinode, 3 standalones, but it does skip them on gate, mainly because these jobs are not really testing the role | 12:11 |
| zbr | including a role with ignore_errors is making almost impossible to fail due to the role itself. | 12:12 |
| weshay|ruck | ysandeep, http://dashboard-ci.tripleo.org/d/wb8HBhrWk/cockpit?orgId=1&var-launchpad_tags=alert&var-releases=master&var-promotion_names=current-tripleo&var-promotion_names=current-tripleo-rdo&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-8-scenario012-standalone | 12:12 |
| chandankumar | zbr: one minute comment | 12:15 |
| zbr | chandankumar: so to run 1+2+10 ? | 12:16 |
| chandankumar | zbr: yes | 12:16 |
| chandankumar | kopecmartin: better we can disbale doc adding a comment about this package | 12:17 |
| chandankumar | kopecmartin: %global with_doc 0 | 12:17 |
| kopecmartin | chandankumar: makes sense, thanks | 12:20 |
| zbr | chandankumar: done! btw, later I plan to add some code to verify.yml to validate that we produced desired files, much more useful than running a full integration and relying on reviewer to manually inspect the collection directory. | 12:20 |
| *** ratailor has quit IRC | 12:21 | |
| panda|ruck | zbr: +1 | 12:25 |
| panda|ruck | zbr: a passing job is not a passing test. | 12:26 |
| *** pojadhav|brb is now known as pojadhav | 12:26 | |
| chandankumar | panda|ruck: commented https://review.rdoproject.org/r/#/c/26694/ | 12:29 |
| rf0lc0 | chandankumar, pls see my comment on https://review.rdoproject.org/r/#/c/26708/ and see if that changes your vote :) | 12:29 |
| panda|ruck | chandankumar: ah, thanks. | 12:30 |
| *** rlandy has joined #oooq | 12:31 | |
| *** jtomasek has joined #oooq | 12:33 | |
| chandankumar | weshay|ruck: https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-ovb-1ctlr_2comp-featureset020-master | 12:34 |
| *** rlandy is now known as rlandy|mtg | 12:35 | |
| *** jpena|lunch is now known as jpena | 12:38 | |
| rf0lc0 | marios, I'll take the other 2 promoter tests to fix (staging-*) as I feel responsible for the mess. If you have time, please help me with the upstream branching :) | 12:41 |
| marios | rf0lc0: i am doing that | 12:41 |
| rf0lc0 | will talk to panda|ruck about the promoter tests that I broke | 12:41 |
| rf0lc0 | marios, nice thanks | 12:41 |
| marios | rf0lc0: i am trying to make sense of what needs -ussuri will post something today anyway | 12:41 |
| marios | https://tree.taiga.io/project/tripleo-ci-board/task/1665 rf0lc0 | 12:42 |
| rf0lc0 | c00l | 12:42 |
| chandankumar | weshay|ruck: arxcruz https://9c98041cc87eb70d9be4-b5689cdf1183af9fcadbc91f7ef3081b.ssl.cf2.rackcdn.com/717995/6/check/tripleo-ci-centos-8-scenario012-standalone/464f8c5/logs/undercloud/var/log/tempest/stestr_results.html | 12:43 |
| panda|ruck | rf0lc0: YOU WHAT ??? | 12:43 |
| panda|ruck | rf0lc0: ship you neck here so I can strangle you in socially distancing. | 12:43 |
| rf0lc0 | panda|ruck, as long as spray alcohol 70% before, I am fine. | 12:44 |
| panda|ruck | chandankumar: marios review.enqueue_async(https://review.opendev.org/722677) | 12:46 |
| panda|ruck | marios: can you offer a REST API for this, it's getting difficult to maintain as RPC | 12:46 |
| marios | panda|ruck: i'll think about it your feedback is very valuable to us | 12:47 |
| chandankumar | rf0lc0: free for few mins? | 12:53 |
| rf0lc0 | chandankumar, yes wanna chat? | 12:54 |
| chandankumar | rf0lc0: yes | 12:54 |
| rf0lc0 | chandankumar, gimme a min to switrch rooms | 12:55 |
| chandankumar | rf0lc0: aye | 12:55 |
| *** jtomasek has quit IRC | 12:58 | |
| rf0lc0 | chandankumar, https://meet.google.com/oiv-geho-mai?authuser=1 | 12:59 |
| *** Goneri has joined #oooq | 13:08 | |
| *** ykarel is now known as ykarel|afk | 13:14 | |
| weshay|ruck | ysandeep, rlandy|mtg https://docs.google.com/document/d/1XxTxgXGdWntjURJBVP2n9ZZ_By8NqRmkbtY3VWcUUcQ/edit | 13:15 |
| chandankumar | rf0lc0: thanks :-) | 13:15 |
| *** pojadhav is now known as pojadhav|away | 13:18 | |
| rf0lc0 | chandankumar, thank you. I just did not understand the next step. after enabling promote to component-ci-testing, what is the next step? | 13:18 |
| *** skramaja has quit IRC | 13:20 | |
| *** amoralej is now known as amoralej|lunch | 13:25 | |
| chandankumar | rf0lc0: http://paste.openstack.org/show/792659/ is clear? | 13:26 |
| chandankumar | regarding ussuri-testing pipeline | 13:26 |
| chandankumar | rf0lc0: then we can add https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/dlrnapi_promoter/config/CentOS-8/component/master.yaml which will create toci testing repo | 13:27 |
| chandankumar | then we can start working on integation pipeline | 13:28 |
| rf0lc0 | chandankumar, right, in other words, first time the pipeline runs, we get consistent, pin to component-ci-testing, and promote to promoted-components. Without any real job to verify if its good or not. We just pick consistent. Because we don't have the artefacts like containers to run these validation jobs in component pipeline. | 13:30 |
| rf0lc0 | chandankumar, is this statement above true ? | 13:30 |
| rf0lc0 | first component promotion will be a lie | 13:31 |
| ysandeep | rlandy|mtg, thank you :) :) | 13:34 |
| *** rlandy|mtg is now known as rlandy | 13:34 | |
| rlandy | ysandeep: sure - happy someone else is working on bm now | 13:34 |
| *** TrevorV has joined #oooq | 13:38 | |
| rf0lc0 | chandankumar, zzzzzzzzz | 13:47 |
| rf0lc0 | chandankumar, 404 | 13:47 |
| chandankumar | rf0lc0: 200 | 13:48 |
| chandankumar | rf0lc0: yes, correct | 13:48 |
| rf0lc0 | \o/ | 13:48 |
| *** ysandeep is now known as ysandeep|afk | 13:50 | |
| arxcruz | panda|ruck: https://logserver.rdoproject.org/75/26775/1/check/periodic-tripleo-ci-centos-8-standalone-full-tempest-scenario-tempest-master/7e14525/logs/stackviz/#/testrepository.subunit/timeline | 13:54 |
| arxcruz | panda|ruck: seems to be working | 13:54 |
| *** udesale_ has joined #oooq | 13:54 | |
| *** ratailor has joined #oooq | 13:56 | |
| *** udesale has quit IRC | 13:57 | |
| *** ykarel|afk is now known as ykarel | 14:07 | |
| panda|ruck | arxcruz: lately all the errors in tempest appear to be intermittent, so I would wait for it not to appear again in the future | 14:10 |
| *** pojadhav|away is now known as pojadhav | 14:12 | |
| *** amoralej|lunch is now known as amoralej | 14:15 | |
| *** udesale_ has quit IRC | 14:19 | |
| chandankumar | rlandy: https://review.opendev.org/#/c/722402/ merge it | 14:47 |
| rlandy | done | 14:48 |
| chandankumar | rlandy: thanks ! | 14:49 |
| marios | rf0lc0: did you get a chance to see containers... i commented on your https://code.engineering.redhat.com/gerrit/#/c/196858/71/playbooks/tripleo-buildcontainers/pre_rhel8.yaml@50 ... then i added the -u -r 8.2 and basically it brings us on par with the 'all repos' case commented https://code.engineering.redhat.com/gerrit/#/c/196858/72/playbooks/tripleo-buildcontainers/pre_rhel8.yaml ... so we are still | 14:51 |
| marios | waiting on the trunk-candidate looks like | 14:51 |
| marios | rlandy: ^^^ | 14:51 |
| marios | rf0lc0: sorry was meant for rlandy | 14:51 |
| marios | rlandy: did you get a chance to see containers... i commented on your https://code.engineering.redhat.com/gerrit/#/c/196858/71/playbooks/tripleo-buildcontainers/pre_rhel8.yaml@50 ... then i added the -u -r 8.2 and basically it brings us on par with the 'all repos' case commented https://code.engineering.redhat.com/gerrit/#/c/196858/72/playbooks/tripleo-buildcontainers/pre_rhel8.yaml ... so we are still | 14:51 |
| marios | waiting on the trunk-candidate looks like | 14:51 |
| * marios spams the channel | 14:51 | |
| rf0lc0 | marios, I made a wtf face :) | 14:52 |
| rlandy | marios: when I checked yesterday, yes | 14:52 |
| rlandy | we can ping lon and ask | 14:52 |
| marios | rlandy: yeah same today i mean earlier when i looked | 14:52 |
| marios | rlandy: but as per https://code.engineering.redhat.com/gerrit/#/c/196858/72/playbooks/tripleo-buildcontainers/pre_rhel8.yaml with containers repo & 8.2 stuff it gets us to 'same' result | 14:52 |
| marios | rlandy: hopefully candidate will give us the rest | 14:52 |
| marios | rlandy: we can wait till Monday and then just merge this then | 14:52 |
| rlandy | marios: we got a response on the ticket to say the action was done so I was expecting an update today | 14:53 |
| marios | rlandy: k well if you get a hold of Lon in your timezone maybe it will be done done by Monday morning i can check it/update | 14:53 |
| rlandy | marios: ack | 14:54 |
| *** ratailor has quit IRC | 15:04 | |
| *** dpawlik has quit IRC | 15:10 | |
| panda|ruck | weshay|ruck: would be a problem to open a bug on the promoter for the error ? I have apossible root cause, a workaround and more permanent fix, but I need to keep track of what I'm doing | 15:20 |
| weshay|ruck | panda|ruck, it's fine to open a bug | 15:21 |
| panda|ruck | weshay|ruck: ok | 15:21 |
| zbr | artcl reviews needed on https://review.opendev.org/#/c/721844/ https://review.opendev.org/#/c/721607/ | 15:37 |
| * marios shutdown | 15:45 | |
| weshay|ruck | marios, helpdesk take care of you? | 15:46 |
| marios | weshay|ruck: not sure that's what i was asking today but don't worry not urgent/problem let's see will ask again if isn't resolved by next week thanks for asking weshay|ruck | 15:46 |
| weshay|ruck | cool | 15:48 |
| *** ysandeep|afk is now known as ysandeep|away | 15:49 | |
| *** marios has quit IRC | 16:05 | |
| zbr | panda|ruck: weshay|ruck chandankumar rlandy : please revote on https://review.opendev.org/#/c/722402/ -- i applied wes request. | 16:05 |
| zbr | we can discuss on monday meeting about challenging that invoked policy | 16:06 |
| weshay|ruck | zbr, is there a specific need for - tripleo-ci-centos-8-scenario010-ovn-provider-standalone | 16:07 |
| weshay|ruck | ? | 16:07 |
| weshay|ruck | or randomly picked | 16:07 |
| zbr | weshay|ruck: not randomly, we have one task specific to it. | 16:07 |
| weshay|ruck | k | 16:07 |
| zbr | martin added one ovn specific task, so it would be useful to keep, at least until we find a way to mock-test it. | 16:08 |
| *** bogdando has quit IRC | 16:26 | |
| *** dtantsur is now known as dtantsur|afk | 16:29 | |
| zbr | weshay|ruck: it will be fun to re-add the votes on this change because the gerrit group change already merged but is waiting for deployment for ~1h, not sure when it will apply | 16:30 |
| zbr | when it does, you will lose access on that two repos, until i add the groups back. i will check again in an hour. | 16:31 |
| *** ykarel is now known as ykarel|away | 16:33 | |
| *** jmasud has joined #oooq | 16:34 | |
| *** pojadhav is now known as pojadhav|away | 16:41 | |
| *** jpena is now known as jpena|off | 16:48 | |
| *** jmasud has quit IRC | 16:53 | |
| *** derekh has quit IRC | 17:00 | |
| *** jmasud has joined #oooq | 17:00 | |
| *** jmasud has quit IRC | 17:12 | |
| *** rlandy is now known as rlandy|brb | 17:14 | |
| chandankumar | weshay|ruck: zbr rlandy|brb rf0lc0 panda|ruck arxcruz https://twitter.com/pyconindia/status/1253732245635952643/photo/1 | 17:47 |
| rf0lc0 | chandankumar, cool | 17:48 |
| weshay|ruck | +1 | 17:52 |
| *** amoralej is now known as amoralej|off | 17:53 | |
| weshay|ruck | rlandy|brb, I think this would continue to work w/ internal builds.. please review https://review.opendev.org/#/c/718433/ | 18:06 |
| *** yolanda has quit IRC | 18:11 | |
| *** rlandy|brb is now known as rlandy | 18:12 | |
| rlandy | weshay|ruck: hi - how do we usually get the ip of the undercloud node? host.primray.ansible_host? | 18:25 |
| rlandy | primary | 18:25 |
| weshay|ruck | rlandy, hrm.. I can look | 18:59 |
| rlandy | undercloud_ip gives 127.0.0.2 :( | 19:01 |
| rlandy | trying ipv4 | 19:01 |
| weshay|ruck | rlandy, give me a link to one your ipa jobs | 19:03 |
| rf0lc0 | weshay|ruck, you know why this did not merge ? https://review.rdoproject.org/r/#/c/26708/ | 19:07 |
| rlandy | weshay|ruck: rechecking using downstream zuul https://code.engineering.redhat.com/gerrit/#/c/195798/4 | 19:16 |
| rlandy | quicker than upstream | 19:16 |
| weshay|ruck | rf0lc0, you got it in the gate now | 19:16 |
| rlandy | last job took the 192 address | 19:17 |
| rlandy | I need it to take te public one I think | 19:17 |
| rf0lc0 | weshay|ruck, yeah depends on patch merged after w+ on that one | 19:17 |
| weshay|ruck | rlandy, what's the cidr you are hoping to find? | 19:18 |
| weshay|ruck | first three octets? | 19:18 |
| weshay|ruck | rlandy, /me suggests working it backwards https://sf.hosted.upshift.rdu2.redhat.com/logs/98/195798/4/check/tripleo-ci-centos-8-standalone-on-multinode-ipa-internal/86e29d1/logs/undercloud/var/log/extra/dump_variables_vars.json | 19:19 |
| rf0lc0 | weshay|ruck, need your fast and furious brain | 19:19 |
| rf0lc0 | weshay|ruck, https://review.rdoproject.org/r/#/c/26800/5/zuul.d/integration-pipeline.yaml | 19:19 |
| rlandy | "10.0.109. | 19:19 |
| rlandy | I think the default will do it | 19:19 |
| weshay|ruck | k.. so that is subnode_1 | 19:20 |
| rf0lc0 | weshay|ruck, can I run these mini pipeline jobs on testproject to speed up container builds and try enabling component jobs ? | 19:20 |
| weshay|ruck | I can trigger the integration line for you | 19:20 |
| weshay|ruck | rf0lc0, don't bother w/ a testproject | 19:21 |
| rf0lc0 | weshay|ruck, let it run on periodic and wait for the containers ? and meanwhile I prepare the rest of ussuri jobs ? | 19:21 |
| weshay|ruck | rf0lc0, looking at ur review | 19:22 |
| rf0lc0 | weshay|ruck, ok... the plan is to run on openstack-periodic-master pipeline, then we move to latest-release with all the other jobs | 19:22 |
| weshay|ruck | rf0lc0, keep working it.. comments | 19:23 |
| rf0lc0 | thank you | 19:23 |
| weshay|ruck | rlandy, for me to help.. we probably need to chat | 19:24 |
| weshay|ruck | rlandy, I suspect ur looking for how the ctrl-plane ip is picked in tq? | 19:24 |
| weshay|ruck | but not sure | 19:24 |
| weshay|ruck | and which variable it's associated with | 19:25 |
| weshay|ruck | rlandy, this bit? https://opendev.org/openstack/tripleo-quickstart/src/branch/master/roles/common/defaults/main.yml#L141-L156 | 19:25 |
| rlandy | weshay|ruck: sec - trying one last option | 19:26 |
| *** jjoyce has quit IRC | 19:27 | |
| *** jfrancoa has quit IRC | 19:27 | |
| rlandy | weshay|ruck: ok - can chat now to explain | 19:28 |
| rf0lc0 | weshay|ruck, this one ? periodic-tripleo-ci-centos-8-ovb-3ctlr_1comp-featureset001-master: | 19:28 |
| *** fuzzball81 has joined #oooq | 19:28 | |
| weshay|ruck | yes.. that is featureset001 | 19:28 |
| weshay|ruck | rlandy, k | 19:29 |
| weshay|ruck | rlandy, https://meet.google.com/gnn-pryg-qdu?authuser=1 | 19:30 |
| weshay|ruck | rlandy, https://opendev.org/openstack/tripleo-quickstart/src/branch/master/config/general_config/featureset039.yml#L216 | 19:53 |
| weshay|ruck | rlandy, https://opendev.org/openstack/tripleo-quickstart-extras/src/branch/master/config/environments/rdocloud.yml#L21-L23 | 19:55 |
| weshay|ruck | rf0lc0, ready for review /merge? | 20:18 |
| rf0lc0 | weshay|ruck, should be, waiting for zuul +1 | 20:18 |
| -openstackstatus- NOTICE: The Zuul project gating service is reporting new patches in merge conflict erroneously due to a configuration error, fix in progress | 20:19 | |
| rf0lc0 | weshay|ruck, wait | 20:19 |
| weshay|ruck | rf0lc0, double check where you are putting the nodedef | 20:20 |
| rf0lc0 | let me check diff on nodeset: upstream-centos-8-vexxhost vs | 20:20 |
| rf0lc0 | exactly | 20:20 |
| weshay|ruck | k | 20:20 |
| rf0lc0 | weshay|ruck, ready | 20:25 |
| weshay|ruck | k | 20:25 |
| *** TrevorV has quit IRC | 20:29 | |
| *** holser has quit IRC | 20:29 | |
| weshay|ruck | rf0lc0, hey.. jump on meet | 20:32 |
| rf0lc0 | weshay|ruck, ok | 20:32 |
| *** jbadiapa has quit IRC | 20:32 | |
| weshay|ruck | rf0lc0, https://meet.google.com/jfh-sokm-hzq?authuser=1 | 20:32 |
| *** fuzzball81 is now known as jjoyce | 20:35 | |
| *** jmasud has joined #oooq | 20:39 | |
| *** holser has joined #oooq | 20:40 | |
| *** holser has quit IRC | 20:42 | |
| *** jmasud has quit IRC | 20:43 | |
| *** ccamacho has quit IRC | 20:51 | |
| *** jmasud has joined #oooq | 21:06 | |
| *** jmasud has quit IRC | 21:09 | |
| *** jmasud has joined #oooq | 21:13 | |
| *** jmasud has quit IRC | 21:14 | |
| *** dsneddon has joined #oooq | 21:24 | |
| *** jmasud has joined #oooq | 21:28 | |
| rf0lc0 | weshay|ruck, shutdown sequence, please see the ussuri progress/task stream on https://hackmd.io/n7eBGiPVR4yoFbz9o-V2bQ?both, taiga is also updated | 21:29 |
| rf0lc0 | tchau | 21:30 |
| rf0lc0 | adios | 21:30 |
| rf0lc0 | ciao | 21:30 |
| *** jmasud has quit IRC | 21:31 | |
| *** rf0lc0 has quit IRC | 21:36 | |
| *** holser has joined #oooq | 21:36 | |
| *** holser has quit IRC | 21:58 | |
| rlandy | weshay|ruck: looking better ... https://sf.hosted.upshift.rdu2.redhat.com/logs/98/195798/5/check/tripleo-ci-centos-8-standalone-on-multinode-ipa-internal/3187313/logs/undercloud/home/zuul/standalone_deploy.log | 22:22 |
| rlandy | gets to standalone deploy step | 22:22 |
| rlandy | 3 | 22:22 |
| *** rlandy has quit IRC | 22:48 | |
| *** rf0lc0 has joined #oooq | 22:55 | |
| -openstackstatus- NOTICE: the This Zuul outage was taken as an opportunity to perform an impromptu maintenance for changing our service deployment model; any merge failures received from Zuul between 19:40 and 20:20 UTC were likely in error and those changes should be rechecked; any patches uploaded between 20:55 and 22:45 UTC were missed entirely by Zuul and should also be rechecked to get fresh test results | 22:56 | |
| *** jmasud has joined #oooq | 23:01 | |
| *** tosky has quit IRC | 23:14 | |
| *** chem has quit IRC | 23:20 | |
| *** chem has joined #oooq | 23:22 | |
| *** jmasud has quit IRC | 23:27 | |
| *** rf0lc0 has quit IRC | 23:30 | |
| *** jmasud has joined #oooq | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!