| *** tosky has quit IRC | 00:02 | |
| *** agopi has joined #oooq | 00:25 | |
| *** rascasoft has joined #oooq | 00:51 | |
| *** sanjayu_ has quit IRC | 01:01 | |
| *** rascasoft has quit IRC | 01:03 | |
| *** skramaja has joined #oooq | 02:27 | |
| *** apetrich has quit IRC | 03:15 | |
| *** ykarel has joined #oooq | 04:19 | |
| *** raukadah is now known as chandankumar | 04:40 | |
| *** sanjayu_ has joined #oooq | 05:19 | |
| *** rlandy has quit IRC | 05:37 | |
| *** udesale has joined #oooq | 05:42 | |
| *** ykarel has quit IRC | 05:54 | |
| *** ykarel has joined #oooq | 05:55 | |
| *** ratailor has joined #oooq | 06:10 | |
| *** jfrancoa has joined #oooq | 06:31 | |
| *** dsneddon has quit IRC | 06:47 | |
| *** quiquell|off is now known as quiquell | 06:52 | |
| *** jtomasek has joined #oooq | 06:53 | |
| *** apetrich has joined #oooq | 07:21 | |
| *** ccamacho has joined #oooq | 07:29 | |
| *** rascasoft has joined #oooq | 07:55 | |
| *** ccamacho has quit IRC | 07:56 | |
| *** kopecmartin|off is now known as kopecmartin | 08:05 | |
| *** ccamacho has joined #oooq | 08:07 | |
| *** amoralej|off is now known as amoralej | 08:15 | |
| *** zbr|ssbarnea has joined #oooq | 08:20 | |
| *** zbr|ssbarnea is now known as zbr | 08:22 | |
| zbr | morning! | 08:23 |
|---|---|---|
| quiquell | zbr: o/ hello there | 08:23 |
| quiquell | zbr: do you have time to sync ? | 08:23 |
| zbr | yees | 08:24 |
| quiquell | marios: ^ | 08:24 |
| quiquell | zbr: Have change the reviews a little to make the job pass | 08:24 |
| quiquell | zbr: well progress more than pass | 08:24 |
| quiquell | talking about f28/py3 containers-build | 08:24 |
| zbr | my first question is you found the correct change to make rdo f28 bld cnt run? | 08:25 |
| marios | o/ | 08:25 |
| quiquell | marios: have time to sync on containers-build ? | 08:25 |
| marios | k | 08:25 |
| quiquell | or we wait for panda|ruck|off ? | 08:25 |
| marios | quiquell: sure lets schedule something maybe send an invite for 1 hour form now give others chance and zbr can review stuff he missed | 08:26 |
| quiquell | marios: perfect | 08:28 |
| marios | zbr: you forgot to add quiquell and panda|ruck|off | 08:28 |
| quiquell | marios: to what ? | 08:28 |
| zbr | done, all | 08:29 |
| marios | quiquell: invite | 08:29 |
| quiquell | ack | 08:29 |
| quiquell | zbr, marios: question Alex's WIP is suppose to pass or is far from that ? | 08:33 |
| marios | quiquell: which one? | 08:33 |
| zbr | yea | 08:33 |
| marios | quiquell: kolla patch? | 08:34 |
| quiquell | marios: yep | 08:34 |
| quiquell | zbr, marios: I am going to add a fix it to nova-base | 08:34 |
| *** tosky has joined #oooq | 08:41 | |
| ykarel | panda|ruck|off, one more promotion blocker :( | 08:51 |
| ykarel | panda|ruck|off, https://github.com/redhat-cip/hardware/pull/76 | 08:52 |
| *** jpena|off is now known as jpena | 08:57 | |
| *** bogdando has joined #oooq | 08:58 | |
| ykarel | panda|ruck|off, reported https://bugs.launchpad.net/tripleo/+bug/1817859 | 09:03 |
| openstack | Launchpad bug 1817859 in tripleo "[fs035][master][promotion] Introspection Fails with TypeError: coercing to Unicode: need string or buffer, bool found" [Undecided,New] | 09:03 |
| ykarel | let me know in case some info is missing | 09:03 |
| ykarel | i was hoping to have promotion today :( | 09:04 |
| zbr | quiquell: i added https://review.rdoproject.org/r/#/c/18936/ as a depends-on of https://review.openstack.org/#/c/636160/ -- to see if it triggers the downstream f28. | 09:05 |
| quiquell | zbr: at the end I am doing a testing review https://review.rdoproject.org/r/#/c/18997/ | 09:06 |
| quiquell | zbr: It will not trigger | 09:06 |
| quiquell | zbr: I remove it because we need a config change | 09:06 |
| quiquell | zbr: we need this https://review.rdoproject.org/r/#/c/18976/ | 09:07 |
| quiquell | zbr: I removed the Depends-On | 09:07 |
| quiquell | zbr: Has do the opposite RDO -> upstream change as Depends-On | 09:07 |
| zbr | quiquell: mine is on config! | 09:07 |
| zbr | why? | 09:07 |
| quiquell | Ahhh you have it already dind't see it | 09:07 |
| quiquell | Will abandom mine | 09:08 |
| zbr | so we have two changes downstream to merge: once on rdo-jobs and one on config. | 09:08 |
| quiquell | Yep | 09:08 |
| quiquell | zbr: ykarel want the job to pass first | 09:08 |
| quiquell | zbr: before merging them | 09:08 |
| quiquell | zbr: That's why I did the test review | 09:08 |
| zbr | quiquell: this will not happen. | 09:08 |
| quiquell | zbr: That's was one of my questions | 09:09 |
| zbr | we need to run the jobs, regardless of their outcome. | 09:09 |
| quiquell | ykarel: ^ | 09:09 |
| quiquell | zbr: Also changed the cherry-pick just to go beyond that | 09:09 |
| ykarel | zbr, you want to run broken jobs? | 09:09 |
| ykarel | not ready job | 09:09 |
| ykarel | quiquell said job is not ready yet | 09:09 |
| zbr | ykarel: yes. and I can explain why. | 09:10 |
| ykarel | so i suggested to get it ready | 09:10 |
| zbr | f28 will probably be mostly broken for their entire life, ~6mo, still we use it to get feedback on other changes that need to be made. | 09:10 |
| quiquell | zbr, ykarel: maybe we can do a middle solution, if everything here https://review.openstack.org/#/c/636160/ is working fine | 09:11 |
| quiquell | then we merge I mean even with failed job | 09:12 |
| ykarel | zbr, what is broken with f28 | 09:12 |
| zbr | quiquell: you don't get it: that change does not trigger fedora28 job!!!! which makes it a totaly useless. | 09:12 |
| zbr | we need to assure that we see fedora-28 running on that change before merging it. | 09:13 |
| ykarel | quiquell, is the job passing on testproject review? | 09:13 |
| zbr | this is why we need to merge the rdo job definitons. | 09:13 |
| quiquell | zbr: I get that that, I mean exercise the job with this https://review.rdoproject.org/r/#/c/18997/ | 09:13 |
| ykarel | okk job is failing ^^, so not ready | 09:14 |
| quiquell | zbr: ensure that the https://review.openstack.org/#/c/636160/ is working fine then we merge | 09:14 |
| quiquell | zbr: we can even register the job to kolla project at RDO | 09:14 |
| quiquell | zbr: know what I mean ? | 09:15 |
| zbr | i know but i also disagree with keeping cyclic dependencies open. | 09:17 |
| quiquell | zbr: dependency is only RDO -> upstream | 09:17 |
| quiquell | zbr: upstream -> RDO is not needed | 09:17 |
| zbr | i want to merge https://review.rdoproject.org/r/#/c/18913/ now while remove its depends on. | 09:17 |
| quiquell | ykarel: It will consume so much resources ? | 09:19 |
| quiquell | ykarel: As I think this was your main concern | 09:19 |
| zbr | i want to see the enablement of fedora-28 jobs on rdo before we merge the upstream changes. f28 bld containers is not a heavy job and it runs only on specific file changes. | 09:20 |
| quiquell | ykarel: what do you think ? | 09:20 |
| quiquell | zbr: Going to remove the RDO -> upstream Depends-On as I was using it to test it at testproject | 09:20 |
| zbr | we use file filters on it, it will run only on the stuff we want and is a cheap one, likely ~30min. | 09:20 |
| quiquell | Let's see if we can merge it today | 09:20 |
| zbr | quiquell: once we merge the two rdo changes it will be very easy to test all changes we do upstream. | 09:21 |
| quiquell | zbr: but we need to convince ykarel so he merge stuff at config project | 09:21 |
| quiquell | ykarel: I think we have do it before like merging non-voting jobs that were WIP | 09:21 |
| zbr | indeed, if we would have talking about a heavy hob that runs on all/many changes I would have being against it. | 09:21 |
| quiquell | ykarel: At least upstream | 09:21 |
| quiquell | zbr: the Depends-On upstream not needed either, going to remove it too, ack ? | 09:22 |
| zbr | this is non-voting and will ever be like this, until we convert it to centos-8... or reach singularity, whicever comes first. | 09:22 |
| zbr | yep | 09:22 |
| quiquell | zbr: updated the test review too https://review.rdoproject.org/r/18997 | 09:23 |
| quiquell | zbr: so we can exercise it while we merge the job | 09:24 |
| quiquell | zbr: is good for explanations | 09:24 |
| zbr | quiquell: btw: if i remember well, the gerrit message meta requires you to avoid empty lines between meta lines, just for the future. | 09:24 |
| quiquell | zbr: ack thanks | 09:24 |
| ykarel | not sure why you guys still sticking to running a broken which you know is not "ready", it's always good to save resources when you know it's wasting | 09:26 |
| ykarel | as i just saw there are around 15 runs in a day | 09:26 |
| ykarel | for centos containers build, and expect same will be running for this broken job | 09:27 |
| quiquell | ykarel: We did the same for the standalone-f28 if I recall correctly | 09:27 |
| quiquell | ykarel: It was also upstream | 09:27 |
| quiquell | ykarel: so we have a job next to the WIP review to check state of the WIP | 09:27 |
| ykarel | quiquell, you mean running broken job and wasting resources? | 09:27 |
| quiquell | ykarel: yep | 09:27 |
| ykarel | :) | 09:27 |
| quiquell | ykarel: was a good tool though | 09:27 |
| zbr | ykarel: 15 runs/day is nothing, is not a deployment job. likely most of them are cause by our changes. | 09:28 |
| quiquell | ykarel: also this job is just building images | 09:28 |
| quiquell | containers I mean | 09:28 |
| ykarel | it's all your, if you want to run it you can, i can just suggest | 09:29 |
| quiquell | ykarel: If we see tha it's getting to much resources we can disable it again and do manual execise with testproject | 09:29 |
| ykarel | my point was to get it ready | 09:29 |
| quiquell | well is going to take 6 month or more | 09:29 |
| ykarel | to get it ready? | 09:30 |
| quiquell | ykarel: is related to Alex's work here https://review.openstack.org/#/c/624838/ | 09:30 |
| quiquell | ykarel: making kolla support f28 and python3 | 09:30 |
| zbr | ykarel: yes, really not joking. | 09:30 |
| quiquell | As say it would be great tool for us | 09:31 |
| quiquell | And we can keep monitoring it to check if it take too much resources | 09:31 |
| marios | zbr: you cant hear me? | 09:32 |
| quiquell | meeting | 09:32 |
| marios | zbr: i can hear you | 09:32 |
| ykarel | quiquell, so why not adding it as an experimental job if it depends on WIP patches? | 09:32 |
| quiquell | ykarel we can do that | 09:33 |
| quiquell | zbr: ^ | 09:33 |
| marios | quiquell: join us | 09:33 |
| quiquell | trying | 09:33 |
| *** derekh has joined #oooq | 09:41 | |
| *** holser_ has joined #oooq | 09:42 | |
| *** holser_ has quit IRC | 09:42 | |
| *** holser_ has joined #oooq | 09:43 | |
| *** ykarel is now known as ykarel|lunch | 09:44 | |
| marios | quiquell: we can't hear you canyou hear us | 09:52 |
| *** chem has joined #oooq | 09:53 | |
| quiquell | zbr: we have to run them in the right places though, don't forget about kolla | 10:14 |
| quiquell | marios: commented here https://review.rdoproject.org/r/#/c/19000/ about how to run check as periodic | 10:19 |
| quiquell | marios: so we use the correct release file | 10:19 |
| marios | quiquell: thank you | 10:20 |
| quiquell | marios: Let me know if there is any issue | 10:21 |
| *** chem has quit IRC | 10:21 | |
| *** chem has joined #oooq | 10:21 | |
| *** panda|ruck|off is now known as panda | 10:23 | |
| *** panda is now known as panda|ruck | 10:23 | |
| *** ykarel|lunch is now known as ykarel | 10:23 | |
| panda|ruck | mmhh, so only fs020 left to fix ? | 10:29 |
| ykarel | for that atleast we have tests added atleast in skip list and is in gate | 10:30 |
| panda|ruck | chandankumar: what could cause tempest to take more than 2:30 hours to complete ? | 10:30 |
| panda|ruck | ykarel: fs020 timed out this time, and I don't see a particular blocking point, it's just tempest being slow ? | 10:31 |
| panda|ruck | s/?// | 10:31 |
| ykarel | panda|ruck, possibly, i tested skip list patch | 10:31 |
| ykarel | and it passed | 10:31 |
| panda|ruck | ykarel: wasn't a question, tempest progresses correctly but every test takes a bit too much | 10:32 |
| chandankumar | panda|ruck: may be new tests got added to the master list | 10:33 |
| chandankumar | by removing from skip list | 10:33 |
| ykarel | panda|ruck, so few months ago it used to take around 1.5 hours for full tests run | 10:33 |
| chandankumar | recently neuton tempest plugin got inclided | 10:33 |
| ykarel | but now it seems to grow | 10:33 |
| chandankumar | included | 10:33 |
| panda|ruck | chandankumar: mmmhh. I'll check how much did it take yesterday | 10:34 |
| chandankumar | or something ate the time | 10:35 |
| panda|ruck | chandankumar: yesteday 2:15 hour | 10:35 |
| panda|ruck | s | 10:35 |
| panda|ruck | too close to timeout threshold | 10:36 |
| chandankumar | is no. of tests runned is same? | 10:36 |
| panda|ruck | chandankumar: should we increase the timeout for periodic jobs at this point ? | 10:36 |
| panda|ruck | chandankumar: I don't know, how meny test did we add yesterday ? | 10:37 |
| chandankumar | panda|ruck: https://review.openstack.org/#/c/635152/ got merged 5 days ago | 10:37 |
| panda|ruck | chandankumar: checking the time 5 days ago | 10:38 |
| panda|ruck | chandankumar: 19th Febr it took 1:30 hour | 10:40 |
| panda|ruck | chandankumar: wow | 10:40 |
| panda|ruck | neutron added 1 more hour | 10:41 |
| panda|ruck | we probably need to bump the timeout at least for the full tempest job | 10:41 |
| panda|ruck | also bump the dead job timeout in the stack cleanup script | 10:41 |
| *** sanjayu_ has quit IRC | 10:45 | |
| quiquell | chandankumar: Have you change your email at RDO gerrit ? | 10:45 |
| *** sanjayu_ has joined #oooq | 10:45 | |
| ykarel | chandankumar, panda|ruck how many tests got added recently | 10:51 |
| *** udesale has quit IRC | 10:53 | |
| panda|ruck | ykarel: the entire neutron plugin, now sure how many are there | 10:58 |
| chandankumar | ykarel: one neutron plugin | 10:59 |
| chandankumar | is has api & scenario tests | 10:59 |
| chandankumar | quiquell: yes | 11:00 |
| chandankumar | panda|ruck: we may increase the time for full tempest fs020 job | 11:00 |
| quiquell | chandankumar: what's the new one ? | 11:01 |
| chandankumar | quiquell: ah no, it got synced from github, chkumar246@gmail.com | 11:03 |
| panda|ruck | chandankumar: we probably need to | 11:03 |
| ykarel | panda|ruck, chandankumar what i saw tests got decreased, seeing SUCCESS jobs:- Now:- Pass 1247 Skip 299, on 13th:- Pass 1459 Skip 215 | 11:07 |
| ykarel | seen SUCCESS job from 13th and today:- https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master | 11:08 |
| ykarel | so it doesn't seem related to tests | 11:09 |
| chandankumar | ykarel: recently ironic got added, | 11:10 |
| ykarel | fs020 overcloud runs ironic? | 11:10 |
| panda|ruck | mmmh fs035 failed introspection too | 11:10 |
| chandankumar | ykarel: yes | 11:10 |
| ykarel | panda|ruck, for that only i shared bug | 11:10 |
| chandankumar | wait one min | 11:11 |
| chandankumar | panda|ruck: ykarel ironic got skipped | 11:11 |
| panda|ruck | chandankumar: uh ? | 11:11 |
| ykarel | afaik ironic is optional, so should not be in default services, thus fs020 will not have it | 11:11 |
| chandankumar | ironic has more than 50 tests skipped | 11:12 |
| panda|ruck | isn't fs20 goal to test everything ? | 11:12 |
| ykarel | test evertying that is deployed | 11:12 |
| panda|ruck | ykarel: what bug ? | 11:13 |
| ykarel | ykarel> panda|ruck|off, reported https://bugs.launchpad.net/tripleo/+bug/1817859 | 11:13 |
| openstack | Launchpad bug 1817859 in tripleo "[fs035][master][promotion] Introspection Fails with TypeError: coercing to Unicode: need string or buffer, bool found" [Undecided,New] | 11:13 |
| panda|ruck | what other change could bring tempest to take 1h more than before ? | 11:14 |
| panda|ruck | ykarel: maybe the number of test decreased, but they take more anyway ? | 11:15 |
| panda|ruck | ykarel: according to performance analysis the OVN is faster than everything else. | 11:15 |
| panda|ruck | so that should not be the issue | 11:15 |
| ykarel | panda|ruck, possibly this is the case , tests are taking longer | 11:17 |
| ykarel | don't we report which tests took most time? | 11:17 |
| quiquell | panda|ruck: can you workflow https://review.rdoproject.org/r/#/c/18913/ ? | 11:18 |
| chandankumar | panda|ruck: ykarel https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/7876026/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-02-27_05_06_02 | 11:18 |
| chandankumar | from timedout one | 11:18 |
| panda|ruck | quiquell: I don't have +2 there sorry ... | 11:19 |
| quiquell | sshnaidm|afk: can you workflow https://review.rdoproject.org/r/#/c/18913/ ? | 11:19 |
| panda|ruck | zbr: quiquell marios sorry for missing the sync these days, but that hour is very difficult for me currently, and probably for all the duration of ruck/roveringr | 11:20 |
| *** ratailor has quit IRC | 11:20 | |
| quiquell | panda|ruck: no problem, we can sync at scrum | 11:20 |
| panda|ruck | the stress increase insomnia. | 11:20 |
| chandankumar | panda|ruck: ykarel here is the issue {2} neutron_tempest_plugin.scenario.test_qos.QoSTest.test_qos_basic_and_update [136.532321s] ... FAILED | 11:21 |
| marios | ack panda|ruck np mainly we include you and rfolco|rover IF you can join like we can't blame you for last minute on the spot scheduling | 11:21 |
| chandankumar | panda|ruck: check the time taken by this tests ^^ | 11:21 |
| chandankumar | 136 mins | 11:21 |
| ykarel | chandankumar, it's just 2.5 minutes | 11:21 |
| panda|ruck | chandankumar: ? | 11:21 |
| ykarel | chandankumar, that is seconds | 11:21 |
| chandankumar | sorry | 11:21 |
| panda|ruck | chandankumar: also, by the time that test starts, 2:30 hours already passed. | 11:22 |
| panda|ruck | chandankumar: ykarel I couldn't find a paritcular tesst that takes too much | 11:23 |
| ykarel | panda|ruck, yes it seems it's not tempest, something else took more time | 11:23 |
| ykarel | i checked 1245 tests were passed when the job timed out | 11:23 |
| ykarel | so only 2 were left | 11:23 |
| panda|ruck | time to knock at the gnocchi's door | 11:23 |
| ykarel | in a passing job i saw test took around 2.29 minutes | 11:23 |
| ykarel | 2.29 hours | 11:24 |
| panda|ruck | ykarel: which is 1h more than 5 days ago | 11:24 |
| ykarel | panda|ruck, can you share the log | 11:24 |
| ykarel | i didn't find tempest completing in 1.5 hours | 11:24 |
| *** sanjayu_ has quit IRC | 11:25 | |
| *** sanjayu_ has joined #oooq | 11:25 | |
| panda|ruck | ykarel: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/6a88858/logs/undercloud/home/zuul/tempest.log.txt.gz | 11:25 |
| panda|ruck | ykarel: 2019-02-19 10:52:44 | Ran: 1346 tests in 5053.0000 sec. | 11:26 |
| ykarel | panda|ruck, but that job is not good | 11:26 |
| ykarel | 98 tests failed there | 11:26 |
| ykarel | good to compare similar results | 11:27 |
| panda|ruck | ykarel: https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master/c068c53/logs/undercloud/home/zuul/tempest.log.txt.gz | 11:30 |
| panda|ruck | ykarel: 2:10 minutes | 11:30 |
| panda|ruck | ykarel: 2019-02-13 15:40:55 | Ran: 1674 tests in 7772.0000 sec. | 11:31 |
| ykarel | panda|ruck, so only 20 minutes diff | 11:31 |
| panda|ruck | I would have imagined a failed test not saving so much time | 11:31 |
| chandankumar | ykarel: what about deploy the overcloud and image preapre step | 11:31 |
| panda|ruck | even taking more if possibile, since it has to report the traceback | 11:32 |
| ykarel | chandankumar, haven't see those | 11:32 |
| panda|ruck | ykarel: with 2:10 we are dangerously close to timeout anyway | 11:32 |
| ykarel | panda|ruck, yes good to increase timeout for fs020 and then ananlyze tempest when we have consistent results | 11:32 |
| ykarel | after few days | 11:33 |
| ykarel | or anaylze with whatever data is there | 11:33 |
| ykarel | is that much time around 2 hour is justified, what i remember from couple of months back tempest used to take around 1.5 hours | 11:34 |
| panda|ruck | arxcruz: ykarel in the end the barbican issue was not the blocking problem for tempest ? | 11:39 |
| panda|ruck | I see there are still patches there, but now we have result anyway | 11:39 |
| *** sanjayu_ has quit IRC | 11:39 | |
| ykarel | panda|ruck, yes it was caused with vitrage_tempest_plugin | 11:39 |
| ykarel | barbican error was not fatal | 11:39 |
| *** sanjayu_ has joined #oooq | 11:40 | |
| *** sanjayu_ has quit IRC | 11:41 | |
| *** sanjayu_ has joined #oooq | 11:42 | |
| *** sshnaidm|afk is now known as sshnaidm | 11:49 | |
| *** sanjayu_ has quit IRC | 11:51 | |
| *** sanjayu_ has joined #oooq | 11:52 | |
| quiquell | sshnaidm: o/ can I help with this ? https://tree.taiga.io/project/tripleo-ci-board/us/760?milestone=220350 | 12:00 |
| quiquell | sshnaidm: or is already in progress ? | 12:00 |
| sshnaidm | quiquell, we cancel this, no need a full tempest job on that | 12:00 |
| quiquell | ack | 12:01 |
| rfolco|rover | panda|ruck, how can I help with promotion? | 12:04 |
| sshnaidm | arxcruz, chandankumar hi, where can I find a generated tempest.conf in the job? | 12:05 |
| panda|ruck | rfolco|rover: do you know how to remove jobs from the criteria ? | 12:06 |
| rfolco|rover | panda|ruck, the ini file... let me try finding it | 12:08 |
| panda|ruck | rfolco|rover: if you find it modify it to remove fs20 and fs53 for master | 12:09 |
| panda|ruck | rfolco|rover: going for lunch | 12:10 |
| rfolco|rover | panda|ruck, ok | 12:10 |
| chandankumar | sshnaidm: ah somethign wrong with temepst container | 12:15 |
| chandankumar | if tests are failing then it got existed | 12:15 |
| chandankumar | sshnaidm: till i fix that | 12:16 |
| chandankumar | sshnaidm: https://logs.rdoproject.org/24/639324/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/ecd2dc2/logs/undercloud/home/zuul/tempest.log.txt.gz#_2019-02-26_18_57_44 | 12:16 |
| chandankumar | sshnaidm: http://paste.openstack.org/show/746420/ -> temepst.conf valyes | 12:16 |
| *** holser_ is now known as holser|lunch | 12:17 | |
| sshnaidm | chandankumar, ok, I think it's worth to collect it | 12:20 |
| sshnaidm | chandankumar, currently I have error in all tests: ConfigParser.NoSectionError: No section: 'global http://logs.rdoproject.org/24/639324/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/ecd2dc2/logs/undercloud/home/zuul/tempest.log.txt.gz | 12:21 |
| sshnaidm | chandankumar, not sure which section "global" should be there | 12:21 |
| rfolco|rover | panda|ruck, https://review.rdoproject.org/r/19023 | 12:22 |
| chandankumar | sshnaidm: https://github.com/openstack/novajoin-tempest-plugin/blob/master/novajoin_tempest_plugin/ipa/ipa_client.py#L78 | 12:23 |
| chandankumar | sshnaidm: it is related to /etc/ipa/default.conf | 12:23 |
| sshnaidm | chandankumar, hmm | 12:23 |
| chandankumar | sshnaidm: does ipa is installed or configured there? | 12:23 |
| sshnaidm | chandankumar, http://logs.rdoproject.org/24/639324/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039/ecd2dc2/logs/undercloud/etc/ipa/default.conf.txt.gz | 12:24 |
| *** amoralej is now known as amoralej|lunch | 12:25 | |
| chandankumar | sshnaidm: let me check locally | 12:25 |
| panda|ruck | rfolco|rover: thanks. looks good | 12:26 |
| sshnaidm | chandankumar, is it from container? Does it have /etc mounted in container then? | 12:26 |
| rfolco|rover | panda|ruck, ok that one was easy. You did the hard part. What's next ? | 12:26 |
| *** udesale has joined #oooq | 12:26 | |
| chandankumar | sshnaidm: yes | 12:29 |
| chandankumar | sshnaidm: it needs to be mounted | 12:29 |
| chandankumar | sshnaidm: otherwise it will not work | 12:30 |
| chandankumar | sshnaidm: so we need to mount /etc/ipa/default.conf to temepst container otherwise it will try to read the files which is not there | 12:31 |
| chandankumar | sshnaidm: but tempest plugin should not depend on these hard coded file path | 12:31 |
| sshnaidm | chandankumar, why don't we mount the whole /etc ?\ | 12:32 |
| chandankumar | sshnaidm: not an issue | 12:32 |
| sshnaidm | chandankumar, what do you mean? | 12:33 |
| ykarel | rfolco|rover, panda|ruck can u triage https://bugs.launchpad.net/tripleo/+bug/1817859 and mark promotion blokcer | 12:33 |
| openstack | Launchpad bug 1817859 in tripleo "[fs035][master][promotion] Introspection Fails with TypeError: coercing to Unicode: need string or buffer, bool found" [Undecided,New] | 12:33 |
| *** jpena is now known as jpena|lunch | 12:36 | |
| sshnaidm | quiquell, did you see this in libvirt reproducer? http://paste.openstack.org/show/746423/ | 12:38 |
| sshnaidm | quiquell, "git: 'review' is not a git command. See 'git --help'.", | 12:38 |
| quiquell | sshnaidm: nope | 12:41 |
| quiquell | sshnaidm: Let me fix it | 12:42 |
| quiquell | sshnaidm: we don't use this at CI that's too much of inception | 12:42 |
| marios | quiquell: zbr cool looks like it got tripleo-ci-testing this time http://logs.rdoproject.org/00/19000/2/check/periodic-tripleo-centos-7-master-containers-build-push/e7ffdf5/etc/yum.repos.d/delorean.repo.txt.gz looks like https://trunk.rdoproject.org/centos7-master/tripleo-ci-testing/delorean.repo | 12:43 |
| quiquell | marios: yep, now tripleo-repos has to take this | 12:44 |
| quiquell | damn now | 12:44 |
| marios | quiquell: it does i mean its working now | 12:44 |
| quiquell | it will build the images with this repo and tag with tripleo-ci-testing | 12:44 |
| quiquell | ack cool | 12:45 |
| quiquell | :-) | 12:45 |
| marios | quiquell: the depends on is pulling in the changes in tripleoci and tripleo-repos | 12:45 |
| quiquell | at least repos are alright | 12:45 |
| quiquell | marios: naif question | 12:45 |
| quiquell | marios: the tripleo-repos what really does ? | 12:46 |
| marios | quiquell: but i had to hard code the string because until the config change merges we cant see it https://review.openstack.org/#/c/638652/6/playbooks/tripleo-buildcontainers/pre.yaml see comment there | 12:46 |
| marios | quiquell: it just pulls the repos and writes them to file. literally is one file. the rest is just validation around the right combinations/allowed | 12:46 |
| marios | quiquell: this is the whole thing https://github.com/openstack/tripleo-repos/blob/master/tripleo_repos/main.py | 12:47 |
| marios | quiquell: one main.py | 12:47 |
| marios | quiquell: and this is the main logic that writes the stuff https://github.com/openstack/tripleo-repos/blob/98a91a87df6b8294a0133809e69830103ec2bd73/tripleo_repos/main.py#L265-L305 | 12:47 |
| marios | end | 12:47 |
| sshnaidm | does anybody know why we use tripleo-repos at all? Instead of repo-setup role | 12:48 |
| quiquell | sshnaidm: that's what I don't get, this think looks like repo-setup | 12:49 |
| sshnaidm | quiquell, moreover, repo-setup supports much more - proxies, etc | 12:50 |
| marios | sshnaidm: i don't know why we use tripleo-repos to be honest. and its confusing that we have tripleo-repos in tripleo-ci then tripleo-repos its own project | 12:50 |
| rfolco|rover | ykarel, done. I don't understand the conclusion on that, though | 12:50 |
| sshnaidm | marios, yeah, it's confusing me too | 12:51 |
| sshnaidm | maybe I missed something | 12:51 |
| ykarel | rfolco|rover, conclusion? you mean the comment | 12:51 |
| ykarel | or the bug itself | 12:51 |
| panda|ruck | marios: quiquell:sshnaidm: because tripleo developers wanted something that could be used outside of quickstart. So until we start making roles independent, things like this will happen. The next victim will probably be build-test-packages. | 12:51 |
| rfolco|rover | ykarel, yes, python comment | 12:51 |
| quiquell | panda|ruck: ack | 12:51 |
| quiquell | marios: do you know why it needs to regenrate the .repo files ? | 12:52 |
| sshnaidm | panda|ruck, what does mean "outside"? What is the difference to git clone quickstart or tripleo-repos? | 12:52 |
| quiquell | marios: they are alredy there | 12:52 |
| marios | quiquell: which? | 12:52 |
| ykarel | rfolco|rover, so fix is posted in python-hardware, but i don't think they test tripleo job, so until we see it passing in our CI we need to obsorve it | 12:52 |
| marios | quiquell: which thing is regenerate which riles. if you mean tripleo-repos i guess just for clean env | 12:52 |
| panda|ruck | sshnaidm: the difference is that to make repo-setup work, you need to know how quickstart works. | 12:52 |
| sshnaidm | panda|ruck, if they want one script, let's make them script with "ansible-playbook repo-setup.yml" | 12:52 |
| ykarel | rfolco|rover, in current promotion run fs035 failed even with the fix, but it seems intermittent introspection failure, /me running it in testproject to verify it | 12:53 |
| sshnaidm | panda|ruck, this role is quite independent | 12:53 |
| quiquell | marios: ack, so they massage the .repo files into clean env and them install the deps | 12:53 |
| quiquell | marios: more or less is that it ? | 12:53 |
| rfolco|rover | ykarel, ok thanks for clarification | 12:54 |
| marios | quiquell: ya its literally that file i posted you above | 12:55 |
| quiquell | marios: yep my brains think that we are also runing repo-setup but we are not | 12:55 |
| *** agopi has quit IRC | 12:59 | |
| quiquell | marios: Was wrong you don't need to simulate periodic since this is not using repo-setup | 12:59 |
| quiquell | marios: but is not bad to do so | 12:59 |
| rfolco|rover | panda|ruck, do we have a bug for fs020 timeout due to neutron tempest tests ? | 13:01 |
| panda|ruck | rfolco|rover: probably not due to neutron at this point, but no | 13:02 |
| quiquell | marios: commented https://review.openstack.org/#/c/638652 | 13:07 |
| quiquell | marios: you don't have to fix it if you don't want | 13:07 |
| marios | quiquell: ack on the periodic & thanks for comment will check | 13:09 |
| quiquell | marios: repo file looks good | 13:09 |
| quiquell | marios: I think you can start to unwip it | 13:09 |
| marios | quiquell: yeah will do in a bit just looking at the f28 periodic one first then i'll start to tidy up the repos stuff | 13:10 |
| marios | zbr: removed you added myself at https://tree.taiga.io/project/tripleo-ci-board/us/651?milestone=220350 | 13:10 |
| quiquell | marios: the f28 is passing now you can add it to the test review | 13:11 |
| quiquell | before it breaks :-) | 13:11 |
| weshay | panda|ruck rfolco|rover why is the hash from last night not promoting atm? | 13:12 |
| marios | quiquell: no i mean the periodic version of it gonna post in a sec | 13:12 |
| quiquell | marios: ack | 13:13 |
| rfolco|rover | weshay, what do you see to make this assumption ? teach me please | 13:15 |
| weshay | rfolco|rover you get my email? | 13:15 |
| rfolco|rover | weshay, yep | 13:15 |
| weshay | k k | 13:15 |
| * weshay opens a tmate | 13:16 | |
| rfolco|rover | weshay, https://review.rdoproject.org/r/19023 | 13:16 |
| *** derekh has quit IRC | 13:22 | |
| weshay | rfolco|rover http://38.145.34.55/master.log | 13:22 |
| weshay | 'review_rdoproject_org', 'repo_url': 'https://trunk.rdoproject.org/centos7/f7/fd/f7fd6cb58b22ad925f10a736215f4f004b497dc7_3f92697b', 'full_hash': 'f7fd6cb58b22ad925f10a736215f4f004b497dc7_3f92697b', 'repo_hash': 'f7fd6cb58b22ad925f10a736215f4f004b497dc7_3f92697b', 'commit_hash': 'f7fd6cb58b22ad925f10a736215f4f004b497dc7'} from tripleo-ci-testing to | 13:22 |
| weshay | current-tripleo, missing successful jobs: [u'periodic-multinode-1ctlr-featureset016', u'periodic-multinode-1ctlr-featureset017', u'periodic-ovb-1ctlr_1comp-featureset020', u'periodic-multinode-1ctlr-featureset019', u'periodic-multinode-1ctlr-featureset018', u'periodic-ovb-3ctlr_1comp-featureset001', u'periodic-ovb-3ctlr_1comp-featureset035'] | 13:22 |
| *** quiquell is now known as quiquell|lunch | 13:23 | |
| weshay | rfolco|rover see that? | 13:23 |
| weshay | centos 2600 2599 0 13:22 ? 00:00:00 python /home/centos/ci-config/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py /home/centos/ci-config/ci-scripts/dlrnapi_promoter/config/master.ini │· | 13:23 |
| rfolco|rover | weshay, I see, its not using centos/master.ini | 13:24 |
| quiquell|lunch | weshay, rfolco|rover: It's config/CentOS-7/master.ini | 13:29 |
| quiquell|lunch | sshnaidm: fix for git review at repro https://review.rdoproject.org/r/19025 | 13:30 |
| rfolco|rover | quiquell|lunch, wasn't | 13:30 |
| rfolco|rover | :) | 13:30 |
| rfolco|rover | see line above | 13:30 |
| quiquell|lunch | Damn... | 13:30 |
| quiquell|lunch | Ack | 13:30 |
| quiquell|lunch | Diagonal reading | 13:30 |
| weshay | rfolco|rover panda|ruck now the right promoter is running http://38.145.34.55/centos7_master.log | 13:33 |
| panda|ruck | weshay: the "right" ? | 13:33 |
| rfolco|rover | panda|ruck, wrong one = 00:00:00 python /home/centos/ci-config/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py /home/centos/ci-config/ci-scripts/dlrnapi_promoter/config/master.ini │· | 13:33 |
| rfolco|rover | right one is centos7/master.ini | 13:34 |
| weshay | panda|ruck which config should it use? | 13:34 |
| panda|ruck | weshay: yes, the centos one | 13:34 |
| weshay | right now .. centos-7/master is running after restarting the service | 13:34 |
| *** skramaja has quit IRC | 13:34 | |
| panda|ruck | weshay: but I saw the one with the right config was running | 13:34 |
| panda|ruck | ok | 13:34 |
| weshay | http://38.145.34.55/centos7_master.log | 13:35 |
| panda|ruck | yeah, because we don't kill what's running | 13:35 |
| panda|ruck | and it's a infinite loop | 13:35 |
| *** agopi has joined #oooq | 13:35 | |
| panda|ruck | never gets updated | 13:35 |
| rfolco|rover | isn't it a service ? | 13:35 |
| weshay | rfolco|rover search that log for to current-tripleo, missing successful | 13:35 |
| rfolco|rover | weshay, what was the point about last night hash ? how did you see it? | 13:36 |
| weshay | panda|ruck say we're promoting today | 13:36 |
| weshay | thanks | 13:37 |
| panda|ruck | we should make that script a cron job | 13:38 |
| *** rlandy has joined #oooq | 13:39 | |
| panda|ruck | or add a killall and start on the ansible playbook | 13:39 |
| *** jpena|lunch is now known as jpena | 13:40 | |
| *** agopi has quit IRC | 13:40 | |
| weshay | ugh.. thank god that's over | 13:41 |
| weshay | rfolco|rover panda|ruck join my blue | 13:41 |
| weshay | http://38.145.34.55/centos7_master.log | 13:51 |
| weshay | '88789e341298af56bb6fe52d384a2746bbcabb53_de2a686c', 'commit_hash': '88789e341298af56bb6fe52d384a2746bbcabb53'} from tripleo-ci-testing to current-tripleo, missing successful jobs: [u'periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-master', u'periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset018-master', u'periodic-tripleo-ci-centos | 13:51 |
| weshay | -7-multinode-1ctlr-featureset017-master', u'periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset019-master'] | 13:51 |
| panda|ruck | rfolco|rover: you have access with the key ssh-rsa | 13:54 |
| panda|ruck | AAAAB3NzaC1yc2EAAAADAQABAAABAQCz37KwYJHVZbOuiSeMndSTH/eSblBWtA8veBkxEtddJ65pdpAQMTNCNh8ZMjDKUS5Y8MZp/8sj9A4wOcFqcGPgilH6yWzm6I1/XU6TDGtRUJRttfqn1W1R5zfHUhF4/ptOuEioU2BhOycbB07HcrCJUEhcV3i3J9IjYXYo7scAgHsX1o0AHcDPP5+2/a3iYjL3EQMCR1uBNfKKnZGb2Uh+O7CXuBbZUbizDOlLIp7BCi6PKukAcXATCG3Bnqx9xynyUn/XZeYJ93wEqLqWl0FD8JlYQMcNcpblc3Mv4Ke8XIp4aPkL54F/yq7omCz3fi53qP8/KYATnQbTJ/tjY9pt rfolco@redhat.com | 13:54 |
| rfolco|rover | panda|ruck, nm, I have it | 13:55 |
| *** quiquell|lunch is now known as quiquell | 13:57 | |
| *** derekh has joined #oooq | 13:57 | |
| weshay | periodic-tripleo-ci-centos-7-standalone-master | 14:02 |
| marios | zbr: quiquell here https://review.rdoproject.org/r/#/c/19026/ ( quiquell note i had to also add to that but not sure we need it for fedora yet python-docker i mean https://review.rdoproject.org/r/#/c/19026/1/playbooks/tripleo-ci-periodic-base/containers-push-registry-login.yaml ) i'll update the taiga in a sec | 14:05 |
| *** agopi has joined #oooq | 14:08 | |
| weshay | panda|ruck periodic-tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039-master | 14:08 |
| *** sanjayu_ has quit IRC | 14:09 | |
| *** vinaykns has joined #oooq | 14:09 | |
| *** sanjayu_ has joined #oooq | 14:09 | |
| *** amoralej|lunch is now known as amoralej | 14:09 | |
| weshay | panda|ruck rfolco|rover periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master | 14:10 |
| quiquell | marios: marios commented https://review.rdoproject.org/r/#/c/19026/ | 14:14 |
| panda|ruck | marios: ^ in case you missed | 14:14 |
| panda|ruck | also marios ^ | 14:14 |
| marios | thanks quiquell and thanks thanks panda|ruck | 14:15 |
| quiquell | rlandy: o/ | 14:20 |
| *** ykarel is now known as ykarel|afk | 14:20 | |
| rlandy | quiquell: hey | 14:24 |
| quiquell | rlandy: I am planning on work at this task https://tree.taiga.io/project/tripleo-ci-board/task/596?kanban-status=1447275 | 14:25 |
| quiquell | rlandy: to test https://code.engineering.redhat.com/gerrit/#/c/163848/ | 14:25 |
| quiquell | rlandy: if that's ok | 14:25 |
| rlandy | quiquell: sure | 14:26 |
| panda|ruck | US owners: tomorrow is last day of design, take a look at the last table in http://38.145.32.196/taigacli/201902271200.txt and see what are the US definitions missing. | 14:26 |
| rlandy | quiquell: you can remove my depends on to test if you want | 14:26 |
| quiquell | rlandy: ack, thanks, also I am testing zuul semphores and works great for the purpose | 14:27 |
| quiquell | rlandy: let me know if I can help reading some failure or something in the current job you are doing | 14:27 |
| rlandy | quiquell: got all the pre's working | 14:28 |
| rlandy | quiquell: job is running v3 playbooks now | 14:28 |
| quiquell | \o/ !!! | 14:29 |
| rlandy | quiquell: got the settings worked out | 14:29 |
| rlandy | quiquell: env_vars didn't work because common overrides them | 14:29 |
| rlandy | had to hack copy the vars to tripleo-ci in the location it was expecting | 14:29 |
| rlandy | anyways, I am working on accessing the hardware from primary | 14:30 |
| rlandy | we have it accessed from loaclhost | 14:30 |
| quiquell | all looks legit | 14:30 |
| rlandy | the problem hits when we try access the virthost within the playbooks | 14:31 |
| rlandy | that is what I am debugging now | 14:31 |
| quiquell | Cannot access 172.0.0.2 | 14:31 |
| quiquell | You mean the quickstart playbooks | 14:31 |
| rlandy | quiquell: yeah - I think it's the way I set up the initial connection | 14:32 |
| rlandy | yes | 14:32 |
| quiquell | rlandy: make the connection local maybe | 14:32 |
| rlandy | quiquell: I actually need to move the initial connection and connection check to root | 14:32 |
| rlandy | that is step one | 14:32 |
| rlandy | because zuul user will get destroyed in libvirt | 14:33 |
| rlandy | setup/cleanup | 14:33 |
| rlandy | then will try this again | 14:33 |
| quiquell | ack | 14:34 |
| quiquell | I will continue win the DLRN part and semaphores | 14:34 |
| rlandy | quiquell: ack - just getting the root connection to work and then I | 14:35 |
| rlandy | will ping you to debug the playbooks | 14:35 |
| quiquell | ack, be careful with libvirt as we are doing nested virtualization | 14:35 |
| quiquell | a nooo we are not :-) | 14:35 |
| quiquell | libvirt over real undercloud hardware | 14:36 |
| quiquell | ack just let me know if we need to debug something | 14:36 |
| panda|ruck | weshay: rfolco|rover https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-rocky/e2e5412/job-output.txt.gz | 14:38 |
| panda|ruck | weshay: rfolco|rover https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-rocky/e2e5412/job-output.txt.gz#_2019-02-27_13_08_06_671648 | 14:38 |
| panda|ruck | reporting success | 14:39 |
| panda|ruck | weshay: rfolco|rover so I think https://bugs.launchpad.net/tripleo/+bug/1817688 is really fixed | 14:41 |
| openstack | Launchpad bug 1817688 in tripleo "job that test ceph in periodic pipeline for stable branches are getting the wrong configuration" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:41 |
| weshay | panda|ruck rfolco|rover 2-27 14:44:14,974 24797 ERROR promoter CONTAINER IMAGE UPLOAD FAILED LOGS BELOW: | 14:48 |
| *** gkadam has joined #oooq | 14:49 | |
| panda|ruck | mmmmhh | 14:50 |
| weshay | panda|ruck rfolco|rover it may just work on the second go | 14:51 |
| weshay | I need to find my link | 14:51 |
| weshay | to the registry | 14:51 |
| panda|ruck | None | 14:52 |
| panda|ruck | message: received unexpected HTTP status: 504 Gateway Time-out | 14:52 |
| panda|ruck | Error pulling trunk.registry.rdoproject.org/tripleomaster/centos-binary-sensu-client | 14:52 |
| panda|ruck | sos it failed to pull one container | 14:52 |
| panda|ruck | the cleanup removed everything | 14:52 |
| panda|ruck | so it will need to pull again | 14:52 |
| weshay | zbr, sshnaidm, rlandy who can we propose as core on config repo in rdo? | 14:59 |
| marios | weshay: quiquell | 14:59 |
| marios | weshay: or panda | 14:59 |
| marios | imho ;) | 14:59 |
| rlandy | weshay: both | 15:00 |
| marios | or both ! | 15:00 |
| weshay | agree.. marios can you propose them both for config and rdo-jobs | 15:00 |
| weshay | on the rdo-llist | 15:00 |
| marios | weshay: ermm i'm not core so not sure i can? | 15:00 |
| marios | s/can/should | 15:00 |
| weshay | zbr that's all you need to do ^ | 15:00 |
| weshay | ya.. sshnaidm or rlandy really should | 15:00 |
| weshay | rlandy can you do that? | 15:00 |
| rlandy | sure | 15:00 |
| weshay | rlandy thanks | 15:01 |
| quiquell | Cool we are doing a lot of stuff at config | 15:01 |
| quiquell | more cores will help | 15:01 |
| rlandy | just watching a job - will do in a few | 15:01 |
| rlandy | quiquell: k - got the root connection working | 15:02 |
| rlandy | trick is only from localhost | 15:02 |
| quiquell | rlandy: virthost not working ? | 15:02 |
| *** jpena is now known as jpena|brb | 15:02 | |
| rlandy | we need connection from primary - watching that | 15:02 |
| rlandy | job is still running - will send log in a bit | 15:02 |
| quiquell | rlandy: this is connection from vm -> undercloud or vm -> undercloid(libvirt) ? | 15:03 |
| bogdando | hi folks. I'm looking for greates of zuul gurus | 15:04 |
| bogdando | https://review.openstack.org/#/c/568543 , for some reason this job seems not included for that graph freezer?.. any ideas how to make it included w/o duplicating its definition https://github.com/openstack-infra/openstack-zuul-jobs/blob/165eaf6cb630a53edfc07307318215061ae088db/zuul.d/jobs.yaml#L467 here?.. | 15:04 |
| panda|ruck | bogdando: it's run only on specific file that change | 15:05 |
| bogdando | panda|ruck: there is an instant message after I update the commit, see graph freezer | 15:05 |
| *** ykarel|afk is now known as ykarel | 15:06 | |
| bogdando | doesn't look like it even comes so far to check for involved files names | 15:06 |
| weshay | rfolco|rover fyi https://hub.docker.com/u/tripleomaster | 15:06 |
| panda|ruck | bogdando: openstack-tox-py27 is run only when we change something in the emit-release-script | 15:06 |
| panda|ruck | bogdando: do you have a few minutes to chat ? | 15:07 |
| bogdando | panda|ruck: yes | 15:07 |
| panda|ruck | bogdando: 3492508669 | 15:08 |
| rlandy | quiquell: k ... we're back here ... https://sf.hosted.upshift.rdu2.redhat.com/zuul/t/tripleo-ci-internal/stream/ac624db9475c45f5b8d36b9131b1731d?logfile=console.log | 15:10 |
| rlandy | zbr: continue, pls | 15:10 |
| zbr | weshay: to easy future dev I think that rlandy or sshnaidm should propose on #rdo to add few more @oooq members to https://review.rdoproject.org/r/#/admin/groups/6,members -- any of them can do the change, but we need to see if anyone is against. | 15:10 |
| fultonj | panda|ruck: https://review.openstack.org/#/c/639408 merged rocky/queens are green. what is status of pike? | 15:11 |
| zbr | we have only two members there and this slows reviewing, especially due to timezone differences. pick anyone you trust and propose. | 15:11 |
| fultonj | panda|ruck: i.e. are some pike jobs green but are the branchless ones not? | 15:12 |
| zbr | i doubt anyone from our team would "go rogue" and abuse that. | 15:12 |
| quiquell | rlandy: btw at the end URLTrigger is implemented just at internal sf, what was the issue with using it ? | 15:12 |
| rlandy | weshay: also ... "on the rdo-list" rdo-list is dead, do you mean rhos-dev? | 15:12 |
| quiquell | ups sorry zbr | 15:12 |
| zbr | i think that using #rdo channel should be enough to get feedback (not against any ML but I do not know it...) | 15:13 |
| quiquell | rlandy: I am moving this to done https://tree.taiga.io/project/tripleo-ci-board/task/781?kanban-status=1447275 | 15:19 |
| quiquell | rlandy: I have create another one to migrate jobs to the dlrn report role | 15:19 |
| rlandy | sure | 15:20 |
| weshay | failed: [localhost] (item=ovn-controller) => {"changed": false, "item": "ovn-controller", "msg": "Error pulling trunk.registry.rdoproject.org/tripleomaster/centos-binary-ovn-controller - code: None message: Get https://trunk.registry.rdoproject.org/v2/tripleomaster/centos-binary-ovn-controller/manifests/b9333e0e6ffd1c5033ee1e33d4756f1831f5c503_a1ff | 15:22 |
| weshay | 18dc: Get https://trunk.registry.rdoproject.org/openshift/token?account=unused&scope=repository%3Atripleomaster%2Fcentos-binary-ovn-controller%3Apull: net/http: request canceled (Client.Timeout exceeded while awaiting headers)"} | 15:22 |
| weshay | rfolco|rover panda|ruck^ | 15:22 |
| quiquell | panda|ruck: make sense that I see two runs of the same job at DLRN for the same repo hash ? | 15:28 |
| *** sanjayu_ has quit IRC | 15:32 | |
| *** sanjayu_ has joined #oooq | 15:32 | |
| *** ccamacho has quit IRC | 15:35 | |
| rlandy | weshay: pls review 'proposing core' email - just sent to you. | 15:35 |
| *** ccamacho has joined #oooq | 15:35 | |
| rlandy | quiquell: none | 15:35 |
| rlandy | ie; no issue in using the URl trigger if you prefer | 15:36 |
| quiquell | rlandy: ack | 15:36 |
| *** ccamacho has quit IRC | 15:41 | |
| panda|ruck | quiquell: where ? | 15:41 |
| *** ccamacho has joined #oooq | 15:41 | |
| panda|ruck | fultonj: rocky is green queens ran too early and didn't pass | 15:41 |
| panda|ruck | fultonj: queens is only run once per day | 15:42 |
| panda|ruck | fultonj: but I think we are good | 15:42 |
| *** dmellado has quit IRC | 15:42 | |
| quiquell | panda|ruck: running a script for the trigger of BM | 15:42 |
| quiquell | panda|ruck: I get this https://paste.fedoraproject.org/paste/QePbpdmfNlD1Gc-mg1hXYg | 15:42 |
| *** dmellado has joined #oooq | 15:43 | |
| quiquell | panda|ruck: I supose is a periodic running at same hash twice | 15:43 |
| panda|ruck | quiquell: no | 15:43 |
| quiquell | I am using master centos7 dlrn url | 15:44 |
| panda|ruck | quiquell: we are reporting all the success twice, one with toci_jobtype as job_id and one with zuul.name as job_id | 15:44 |
| panda|ruck | to maintain backwards compatibility for the promoter server | 15:44 |
| *** jpena|brb is now known as jpena | 15:45 | |
| quiquell | panda|ruck: but what I see here is zuul.id in both | 15:45 |
| panda|ruck | quiquell: we switched to the new configuration just now, so we could stop doing that and report with just the job name | 15:45 |
| quiquell | panda|ruck: well I am filtering but the job_name but it appear twice | 15:45 |
| panda|ruck | quiquell: maybe toci_job type and zuul.id are the same for this particular job | 15:45 |
| quiquell | panda|ruck: ack could be | 15:45 |
| quiquell | also timestamp is very similar so it have to be it | 15:46 |
| quiquell | panda|ruck: thanks! | 15:46 |
| quiquell | have to drop | 15:46 |
| *** quiquell is now known as quiquell|off | 15:46 | |
| panda|ruck | quiquell|off: no wait you're right, thos are two different job, the url is different | 15:47 |
| quiquell|off | panda|ruck: ups that's right | 15:48 |
| weshay | panda|ruck rfolco|rover ok.. :) docker.io is getting updated w/ new images | 15:48 |
| panda|ruck | quiquell|off: we dont' check if a job already reported before reporting it, so if for two periodic runs and the consistent repo doesnt' change | 15:48 |
| weshay | https://hub.docker.com/u/tripleomaster | 15:48 |
| *** ccamacho has quit IRC | 15:48 | |
| panda|ruck | quiquell|off: we are reporting twice | 15:48 |
| *** ccamacho has joined #oooq | 15:48 | |
| quiquell|off | panda|ruck: make sense to prevent that so we don't wait resources ? | 15:49 |
| panda|ruck | weshay: \o/ TCP/IP reliability is highly overrated | 15:49 |
| panda|ruck | :D | 15:49 |
| panda|ruck | quiquell|off: it may, but needs some investigation | 15:49 |
| panda|ruck | quiquell|off: if a pipeline fails for something not related to the hash, it's good to have a second chance | 15:50 |
| panda|ruck | quiquell|off: we can discuss it. | 15:50 |
| *** ccamacho has quit IRC | 15:51 | |
| *** ccamacho has joined #oooq | 15:51 | |
| *** ccamacho has quit IRC | 15:51 | |
| *** ccamacho has joined #oooq | 15:52 | |
| *** ccamacho has quit IRC | 15:54 | |
| *** ccamacho has joined #oooq | 15:54 | |
| rlandy | anyone know how add_host works if the host is already added? | 16:02 |
| rlandy | it overwrites? | 16:02 |
| rlandy | ignores? | 16:02 |
| *** ykarel is now known as ykarel|away | 16:03 | |
| rfolco|rover | sshnaidm, are the latest PRs applied to sova production server ? Looks like the error signature didn't work | 16:04 |
| *** holser|lunch is now known as holser_ | 16:04 | |
| sshnaidm | rfolco|rover, checking it now.. | 16:04 |
| *** ykarel|away is now known as ykarel | 16:04 | |
| panda|ruck | rlandy: just a guess, but with how ansible tries usually to be idempotent, I'd guess it ignores. | 16:06 |
| quiquell|off | rlandy: It ignores It if host is there with the same content | 16:10 |
| *** hubbot1 has joined #oooq | 16:10 | |
| *** udesale has quit IRC | 16:10 | |
| quiquell|off | rlandy: if same with different content it overwrite | 16:10 |
| *** quiquell|off is now known as quiquell|r-- | 16:10 | |
| *** chandankumar is now known as raukadah | 16:11 | |
| quiquell|r-- | panda|ruck: so same applies to VM? | 16:12 |
| quiquell|r-- | BM? | 16:12 |
| sshnaidm | rfolco|rover, should be fine now, ansible pull broke for some reason | 16:13 |
| sshnaidm | we'll see results in next hours | 16:14 |
| rlandy | quiquell|r--: juts trying to figure out what is going on with virthost connection | 16:14 |
| rfolco|rover | sshnaidm, thanks for checking. great | 16:14 |
| rfolco|rover | weshay, ^ fyi | 16:14 |
| rlandy | quiquell|r--: it's added multiple times | 16:14 |
| panda|ruck | quiquell|r--: what's r-- ? and what applies to BM ? | 16:15 |
| panda|ruck | dammit | 16:16 |
| panda|ruck | how do I gat notification from google calendar ?? | 16:16 |
| panda|ruck | I missed the CIX call | 16:16 |
| zbr | is there a way to define "files:" pattern that will match any change from a specific repository? | 16:21 |
| zbr | assume we have repos aaa,bbb,ccc that do trigger some jobs and we have a list of file patterns to filter when this job (template) runs. how can I tell it to run on any change made to "ccc" repository? | 16:22 |
| zbr | exact example: https://github.com/openstack-infra/tripleo-ci/blob/master/zuul.d/base.yaml#L190 | 16:23 |
| zbr | afaik, I cannot override "files" when I list the template. | 16:24 |
| panda|ruck | zbr: you want a centralized configuration that is repo-aware ? | 16:26 |
| zbr | panda|ruck: i guess something like this. | 16:26 |
| quiquell|r-- | zbr: did try [] and * and neither work | 16:26 |
| panda|ruck | zbr: AFAIK the only way is to override the job definition in the repo. | 16:27 |
| *** sanjayu_ has quit IRC | 16:28 | |
| zbr | panda|ruck: mainly I can do it only if I defined a new job that ingerist the other one (override is not allowed) | 16:28 |
| *** saneax has joined #oooq | 16:28 | |
| *** dmellado has quit IRC | 16:32 | |
| *** quiquell|r-- is now known as quiquell|off | 16:32 | |
| *** dmellado has joined #oooq | 16:34 | |
| panda|ruck | promotion surely it's taking its time | 16:42 |
| zbr | added https://review.openstack.org/#/c/639739/ for the patterns | 16:42 |
| fmount | Hi there, running a quickstart deploy I found this bug https://bugs.launchpad.net/tripleo/+bug/1817612 | 16:43 |
| openstack | Launchpad bug 1817612 in tripleo "Wrong containers-prepare-parameter.yaml name in overcloud-deploy.sh template" [Undecided,In progress] - Assigned to Francesco Pantano (fmount) | 16:43 |
| fmount | weshay: can you take a look at it (and the related review)? | 16:43 |
| *** ykarel_ has joined #oooq | 16:43 | |
| fmount | weshay: when you have time of course :D | 16:44 |
| fmount | https://review.openstack.org/#/c/639187/ | 16:44 |
| panda|ruck | fmount: do you know how many of my typo in my life sounded like your nickname ? | 16:46 |
| *** saneax has quit IRC | 16:46 | |
| fmount | ahahahaa | 16:46 |
| *** ykarel has quit IRC | 16:46 | |
| *** gouthamr_ has joined #oooq | 16:46 | |
| fmount | panda|ruck: of course this name was born from a typo | 16:46 |
| fmount | panda|ruck: then became a (bash|zsh) alias, finally I adopted it :D | 16:47 |
| *** gouthamr has quit IRC | 16:48 | |
| *** gouthamr has joined #oooq | 16:50 | |
| *** gouthamr_ has quit IRC | 16:50 | |
| weshay | panda|ruck did you read that bug from fmount? | 16:54 |
| panda|ruck | weshay: yes | 16:54 |
| panda|ruck | weshay: investigating | 16:55 |
| * weshay trying to figure out how in the world ci would not hit this | 16:55 | |
| weshay | http://codesearch.openstack.org/?q=containers-default-parameters&i=nope&files=&repos= | 16:55 |
| panda|ruck | weshay: probably because we are ending up with a different sequence of overrides for the variables. | 16:56 |
| panda|ruck | fmount: what feaatureset are you using ? | 16:56 |
| rlandy | weshay: need to borrow your libvirt brain when you have a moment | 16:56 |
| weshay | fmount can you pass the full command you are using w/ quickstart.sh | 16:56 |
| fmount | panda|ruck:featureset049 | 16:57 |
| fmount | weshay:of course, I'm going to update the bug | 16:57 |
| weshay | rfolco|rover panda|ruck master stein promoted :) | 16:57 |
| weshay | http://rhos-release.virt.bos.redhat.com:3030/rhosp | 16:57 |
| weshay | now we just need to get through ci.centos :) | 16:57 |
| weshay | rfolco|rover panda|ruck /me readd's fs20 as critieria | 16:58 |
| panda|ruck | weshay: ok | 16:58 |
| panda|ruck | weshay: increase the latest hast count too | 16:58 |
| weshay | hash count? | 16:58 |
| weshay | oh | 16:59 |
| weshay | k | 16:59 |
| weshay | to 10? | 16:59 |
| panda|ruck | weshay: yes | 16:59 |
| weshay | k | 16:59 |
| *** ykarel_ is now known as ykarel|away | 16:59 | |
| panda|ruck | and here we go again chasing wher we are defining variables and where we are overriding them | 17:00 |
| panda|ruck | bogdando: I read fungi's comment, he's having the same concerns I had ... | 17:02 |
| weshay | sshnaidm you still around? | 17:02 |
| weshay | qq re: my email about sova | 17:02 |
| sshnaidm | weshay, yeah | 17:02 |
| weshay | sshnaidm I haven't looked through the code that uses the error signatures | 17:03 |
| sshnaidm | weshay, should be fixed, see above ^^ | 17:03 |
| weshay | but is it possible to create a little script to test the signature against a file locally? | 17:03 |
| weshay | oh | 17:03 |
| weshay | kk | 17:03 |
| * weshay reads | 17:03 | |
| rfolco|rover | <sshnaidm> rfolco|rover, should be fine now, ansible pull broke for some reason | 17:03 |
| rfolco|rover | <sshnaidm> we'll see results in next hours | 17:03 |
| rfolco|rover | weshay, ^ this | 17:03 |
| weshay | sshnaidm k.. see it.. | 17:04 |
| weshay | also need to chat w/ you about report portal | 17:04 |
| weshay | at some point | 17:04 |
| weshay | not urgent | 17:04 |
| sshnaidm | weshay, let's chat tomorrow | 17:13 |
| weshay | sshnaidm k.. thanks man | 17:14 |
| sshnaidm | zbr, what's the problem with linters here? http://logs.openstack.org/70/639670/3/check/openstack-tox-linters/c8ceff9/job-output.txt.gz | 17:17 |
| weshay | marios ping when you have a sec | 17:18 |
| bogdando | panda|ruck: I reworked the ropic | 17:18 |
| bogdando | topic | 17:18 |
| zbr | sshnaidm: http://logs.openstack.org/70/639670/3/check/openstack-tox-linters/c8ceff9/job-output.txt.gz#_2019-02-27_14_36_30_635170 | 17:18 |
| *** gkadam has quit IRC | 17:18 | |
| zbr | mainly openssl-devel was not installed | 17:18 |
| sshnaidm | zbr, I see this, but what's the problem? | 17:19 |
| sshnaidm | zbr, is it for all linters? | 17:19 |
| zbr | sshnaidm: yep, wheel ->cryptography->openssl | 17:19 |
| zbr | wheel is needed to build/test packaging | 17:20 |
| sshnaidm | zbr, is there any fix? | 17:20 |
| zbr | sshnaidm: yes, install openssl-devel | 17:20 |
| sshnaidm | zbr, where? | 17:20 |
| sshnaidm | zbr, I'm talking about linters job, where do you want me to install openssl? | 17:21 |
| zbr | sshnaidm: usually this should be done in bindep but it is either missing or bindep didn't do its job. clearly bindep is running. | 17:21 |
| zbr | sshnaidm: bindep.txt is the place. | 17:21 |
| sshnaidm | zbr, you mean all jobs in CI should fail on linters now? | 17:21 |
| zbr | well, this has nothing to do with linters. this is a system dependency | 17:22 |
| sshnaidm | zbr, does it mean it should fail on all jobs now? | 17:23 |
| zbr | sshnaidm: i have no idea but if pip decides that it needs to compile cryptography, it will need the header files | 17:24 |
| zbr | usually these were installed on our infra images by default, maybe they decided to strip them? (making them closer clean image) | 17:25 |
| zbr | but regardless what image we use on CI, i would say is up to each repo owner to assure it has a valid bindep.txt file. so let me look inside it.... | 17:26 |
| zbr | sshnaidm: yep, now I know.... the file is empty. no wonder why it fails. | 17:27 |
| zbr | sshnaidm: and i can tell you why.... | 17:27 |
| zbr | sshnaidm: in the past there was something called bindep-fallback which was deprecated (and mostly removed from most places) -- which was: to install an implicit bindep.txt file if the repository didn't had any bindep.txt file. | 17:28 |
| *** trown is now known as trown|lunch | 17:28 | |
| zbr | i will try to create one bindep.txt now for tripleo-ci | 17:29 |
| panda|ruck | fmount: weshay I'm looking at the jobs we have, and most of them are not passing that file in stable branches, but it's not a fatal miss. | 17:30 |
| panda|ruck | fmount: weshay e.g. https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset016-rocky/e2e5412/logs/undercloud/home/zuul/overcloud-deploy.sh that file is not there, but the job passes | 17:30 |
| weshay | hrm.. because tripleo-common also expects the param's file? | 17:31 |
| *** jfrancoa has quit IRC | 17:32 | |
| fmount | maybe it could be related to the fact that in feature049 we don't have that override? | 17:32 |
| weshay | CONTAINER_DEFAULTS_ENVIRONMENT = ('environments/' | 17:32 |
| weshay | 176 'containers-default-parameters.yaml') | 17:32 |
| bogdando | http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003276.html | 17:32 |
| bogdando | panda|ruck, weshay: ^^ | 17:32 |
| weshay | http://git.openstack.org/cgit/openstack/tripleo-common/tree/tripleo_common/constants.py#n175 | 17:32 |
| * weshay reads | 17:32 | |
| *** dsneddon has joined #oooq | 17:33 | |
| fmount | panda|ruck: weshay I'm running another deployment adding to our scenario (featureset049) this override -> https://pastebin.com/Urnwnz0d | 17:36 |
| *** bogdando has quit IRC | 17:37 | |
| weshay | fmount ah ya | 17:37 |
| panda|ruck | fmount: why that override ? | 17:39 |
| panda|ruck | fmount: I see that used only in fs011 (which is now deprecated) and the variable is not used anywhere that I see | 17:39 |
| fmount | panda|ruck: I know, just looking for other permanent solutions | 17:41 |
| fmount | (mainly playing with variables) | 17:41 |
| panda|ruck | fmount: ok, adding my findings to the bug then | 17:43 |
| fmount | panda|ruck: ok (thx) | 17:43 |
| zbr | sshnaidm: i see others discussing about cryptografy issue on infra, so i suspect is not a problem specific to us | 17:51 |
| *** ykarel|away has quit IRC | 17:51 | |
| *** holser_ has quit IRC | 17:52 | |
| weshay | rlandy can you please send that core proposal email out | 18:00 |
| weshay | not to rhos-dev, rdo-discuss | 18:00 |
| *** panda|ruck is now known as panda|ruck|off | 18:09 | |
| rlandy | weshay: hmmm - not on rdo-discuss | 18:09 |
| rlandy | don't see any trace of where to join either | 18:09 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 18:09 |
| *** jpena is now known as jpena|off | 18:11 | |
| rlandy | see rdo-dev | 18:11 |
| rlandy | weshay: sorry but I don't see where to join rdo-discuss. I see numerous other rdo lists. pls point me to the list. | 18:13 |
| raukadah | rlandy: https://www.rdoproject.org/contribute/mailing-lists/ | 18:14 |
| *** jtomasek has quit IRC | 18:15 | |
| raukadah | rlandy: there is no rdo-discuss list | 18:15 |
| rlandy | raukadah: yes | 18:15 |
| rlandy | hence I am asking | 18:16 |
| rlandy | <weshay> not to rhos-dev, rdo-discuss - afaict the list does not exist | 18:16 |
| *** kopecmartin is now known as kopecmartin|off | 18:16 | |
| rlandy | openstack-discuss or rdo-dev | 18:17 |
| rlandy | that's all I got | 18:17 |
| *** trown|lunch is now known as trown | 18:25 | |
| *** sshnaidm is now known as sshnaidm|afk | 18:28 | |
| weshay | rlandy the rdo-list | 18:39 |
| weshay | rlandy https://www.rdoproject.org/contribute/mailing-lists/ | 18:40 |
| weshay | dev@lists.rdoproject.org | 18:40 |
| rlandy | fine rdo-dev | 18:41 |
| weshay | dev@lists.rdoproject.org | 18:41 |
| weshay | thanks | 18:41 |
| *** amoralej is now known as amoralej|off | 18:43 | |
| rlandy | mail sent | 18:46 |
| weshay | rfolco|rover so we've had one successful run on f28 containers build https://logs.rdoproject.org/60/636160/49/openstack-check/tripleo-build-containers-fedora-28/3969232/logs/build-err.log.txt.gz | 18:53 |
| weshay | is that correct? | 18:53 |
| rfolco|rover | apparently yes | 18:53 |
| weshay | should it still be DNM? https://review.openstack.org/#/c/636160/ | 18:53 |
| weshay | it's listed as ready to review in taiga | 18:54 |
| rfolco|rover | zbr, marios updated the patch... or maybe quiquell|off | 18:55 |
| rfolco|rover | I see some changes to layout that I don't have context... https://review.openstack.org/#/c/636160/49/zuul.d/layout.yaml | 18:55 |
| zbr | weshay: good, point, now that we have the f28 job running and passing, no depends-on, we can remote the DNM. doing it now. | 18:56 |
| rfolco|rover | weshay, the upstream patch should modify tripleo-buildcontainers and tripleo-repos role only. The ci job definition and layout should go away afaik | 18:56 |
| rfolco|rover | zbr, ^ | 18:56 |
| weshay | zbr k.. thanks | 18:56 |
| rfolco|rover | any job definition should go away | 18:57 |
| weshay | zbr can you take ownership of the task etc | 18:57 |
| weshay | so I don't bug rfolco|rover :) | 18:57 |
| rfolco|rover | bug me anytime weshay | 18:57 |
| zbr | feel free to vote on https://review.openstack.org/#/c/636160 now, i only removed DNM. it would be great to put wf before the night. | 18:58 |
| weshay | zbr what does ansible_pkg_mgr resolve to? | 18:59 |
| weshay | yum/dnf | 18:59 |
| weshay | ? | 18:59 |
| zbr | dnf or yum | 18:59 |
| weshay | k | 18:59 |
| weshay | thanks | 18:59 |
| zbr | is the easiest way to distinguish betweet py3/py2 systems. | 18:59 |
| zbr | a side effect, but still valid and future proof | 18:59 |
| weshay | zbr k.. assigned the task to you | 19:01 |
| weshay | rlandy as you have time https://review.rdoproject.org/r/#/c/18975/ | 19:03 |
| zbr | weshay: you remember what I was saying about rdo message trust: check https://review.openstack.org/#/c/638651/ -- and look at last msg from rdo. | 19:18 |
| zbr | it was a -1 ... caused by a NODE_FAILURE ... ;p | 19:18 |
| weshay | zbr not really | 19:18 |
| zbr | weshay: i find in confusing, the node failure is not even listed in the build table. | 19:20 |
| zbr | not to mentiong that user needs to click toogle-ci in order to see the msg. so the cause of failure is very well hidden. | 19:21 |
| rlandy | rfolco|rover: weshay: wrt https://review.rdoproject.org/r/#/c/18975/ - lgtm but depends-on is not reviewed/merged | 19:27 |
| weshay | aye... forgot to hit enter on my comment on that one https://review.openstack.org/#/c/638652/7 | 19:28 |
| zbr | weshay: rlandy we will need to merge https://review.openstack.org/#/c/639746/ on tripleo-ci before being able to merge anything else, to fix linters jobs. | 19:32 |
| weshay | zbr what's wrong w/ bindep and dnf/yum | 19:41 |
| weshay | I guess the bindep role? | 19:42 |
| weshay | confuses dnf/yum | 19:42 |
| zbr | weshay: bindap cannot distiguish between dnf and yum systems at this moment. we need a new tag for that. | 19:42 |
| zbr | it has only "rpm" or "redhat" which are not really what we want. | 19:43 |
| zbr | so far people used workarounds like "centos-7" for "!fedora-28" tags but as you can see this is going nowhere we will spend most of our time keeping bindep conditions updated. | 19:44 |
| weshay | zbr k k | 19:44 |
| zbr | i know how to fix it but i need to get some cores | 19:44 |
| weshay | just need to see a centos box return w/ logs | 19:44 |
| weshay | and I'll +2 and merge | 19:44 |
| weshay | zbr lint passed | 19:44 |
| zbr | weshay: which proves that is fixing the linting issue. anyway, i need to get in touch we pabelanger about bindep | 19:45 |
| weshay | zbr so the role ya? | 19:46 |
| zbr | not sure about role | 19:47 |
| zbr | weshay: got help pabelanger, first bindep change change in the gate. we will find them really useful for reproducer | 19:56 |
| zbr | mainly I plan to enable us to do: python3 -foo [platform:dnf] inside bindep.txt | 19:57 |
| zbr | so we can list different rpm names for yum and dnf systems, and avoid need to mention operating system version | 19:57 |
| *** ccamacho has quit IRC | 20:00 | |
| *** quiquell|off is now known as quiquell | 20:02 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario002-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 20:09 |
| *** jfrancoa has joined #oooq | 20:32 | |
| *** tosky has quit IRC | 20:36 | |
| *** dsneddon has quit IRC | 20:53 | |
| *** dsneddon has joined #oooq | 21:04 | |
| *** jaosorior has quit IRC | 21:08 | |
| *** dsneddon has quit IRC | 21:14 | |
| *** dsneddon has joined #oooq | 21:19 | |
| weshay | rlandy do you know how to use the reproducer w/ a periodic job | 21:20 |
| rlandy | weshay: should be the same ... look in job - should have hash | 21:22 |
| rlandy | weshay: what's the problem? | 21:22 |
| weshay | rlandy it never gets triggered | 21:23 |
| weshay | maybe because it's periodic? | 21:23 |
| rlandy | weshay: you mean from the reproducer env, the job never starts? | 21:23 |
| rlandy | if so it's the zuul.yaml | 21:24 |
| rlandy | can you copy? | 21:24 |
| weshay | rlandy weird.. | 21:26 |
| weshay | it's working for folco | 21:26 |
| rlandy | rfolco|rover deserves something working | 21:26 |
| weshay | heh | 21:26 |
| rlandy | weshay: it's payback for your messiah comment at yesterday's community meeting :) | 21:27 |
| rlandy | pls paste the .zuul.yaml - it's probably malformed | 21:28 |
| rlandy | weshay: when you have a few moments, I'd could use your libvirt brain on bm stuff | 21:40 |
| rlandy | https://sf.hosted.upshift.rdu2.redhat.com/logs/49/163849/34/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/8296fbf/job-output.txt.gz#_2019-02-27_21_12_54_548173 | 21:41 |
| *** vinaykns has quit IRC | 21:47 | |
| *** vinaykns has joined #oooq | 21:48 | |
| *** jfrancoa has quit IRC | 21:54 | |
| *** derekh has quit IRC | 22:09 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario002-standalone, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-scenario012-standalone, tripleo-ci-centos-7-scenario010-multinode-oooq-container @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-ovb-3ctlr_1comp_1supp-featureset039 @ (2 more messages) | 22:10 |
| *** trown is now known as trown|outtypewww | 22:15 | |
| *** agopi has quit IRC | 22:16 | |
| *** agopi has joined #oooq | 22:20 | |
| weshay | rlandy k.. /me looks | 22:24 |
| weshay | sorry.. had to eat.. first meal of the day | 22:24 |
| rlandy | weshay:the problem is that local_working_dir is defined diff on localhost and virthost | 22:24 |
| rlandy | np | 22:24 |
| rlandy | my brain is sort of fried by getting the job this far | 22:25 |
| rlandy | changed: [10.12.52.1 -> localhost] => (item={u'comment': u'ansible_generated_undercloud', u'path': u'/home/zuul/workspace/.quickstart/id_rsa_undercloud'}) | 22:25 |
| rlandy | vs | 22:25 |
| rlandy | (/home/zuul/id_rsa_undercloud): | 22:25 |
| weshay | this should be the same as running quickstart.sh 10.x.x.x. | 22:25 |
| * weshay looks at the invocation | 22:25 | |
| rlandy | toci_quickstart defines local_working_dir as workspace/.quickstart | 22:25 |
| weshay | that's fine | 22:26 |
| rlandy | weshay: ^^ | 22:26 |
| rlandy | so on localhost that remains | 22:26 |
| weshay | this should be the same as running quickstart.sh -w ~/workspace/.quickstart 10.x.x.x. | 22:26 |
| weshay | you want to chat? | 22:26 |
| weshay | 2019-02-27 21:09:22.784616 | primary | +(./toci_quickstart.sh:105): ./quickstart.sh --bootstrap --no-clone --working-dir /home/zuul/workspace/.quickstart --playbook noop.yml --retain-inventory 10.12.52.1 | 22:27 |
| rlandy | try define local_working_dir | 22:27 |
| weshay | home/zuul/workspace/.quickstart/bin/ansible-playbook --tags all --extra-vars @/home/zuul/workspace/.quickstart/config/release/tripleo-ci/CentOS-7/master.yml --extra-vars @/home/zuul/src/git.openstack.org/openstack/tripleo-quickstart/config/nodes/3ctlr_1comp.yml --extra-vars @/home/zuul/src/git.openstack.org/openstack/tripleo-quickstart/config/gener | 22:28 |
| weshay | al_config/featureset001.yml --extra-vars @/home/zuul/src/git.openstack.org/openstack-infra/tripleo-ci/toci-quickstart/config/testenv/baremetal.yml -e toci_vxlan_networking=false -e vxlan_networking=false --extra-vars local_working_dir=/home/zuul/workspace/.quickstart --extra-vars virthost=10.12.52.1 --inventory /home/zuul/workspace/.quickstart/host | 22:28 |
| weshay | s --extra-vars tripleo_root=/home/zuul/src/git.openstack.org/openstack --extra-vars working_dir=/home/zuul --skip-tags tripleo-validations,teardown-all --extra-vars @/home/zuul/workspace/logs/zuul-variables.yaml /home/zuul/workspace/.quickstart/playbooks/baremetal-prep-virthost.yml | 22:28 |
| weshay | ah k.. I see | 22:29 |
| weshay | local_working_dir=/home/zuul/workspace/.quickstart | 22:29 |
| rlandy | pls don't target that box | 22:29 |
| weshay | I wont | 22:29 |
| rlandy | the same var on two hosts registers differently | 22:29 |
| weshay | so.. are you sure it's because of the dir? | 22:29 |
| weshay | and not the key names | 22:30 |
| rlandy | - path: "{{ undercloud_key }}" | 22:30 |
| rlandy | comment: "ansible_generated_undercloud" | 22:30 |
| rlandy | # Copy the undercloud key to the virthost in the ssh_user directory | 22:30 |
| rlandy | - name: copy ssh keys to virthost | 22:30 |
| rlandy | copy: | 22:30 |
| rlandy | src: "{{ undercloud_key }}" | 22:30 |
| weshay | ok | 22:30 |
| weshay | rlandy can we run this w/ debug? | 22:30 |
| weshay | oh wait.. I see | 22:30 |
| weshay | 'path': u'/home/zuul/workspace/.quickstart/id_rsa_virt_power'}) | 22:30 |
| weshay | 'path': u'/home/zuul/workspace/.quickstart/id_rsa_undercloud'}) | 22:30 |
| *** agopi has quit IRC | 22:31 | |
| rlandy | I could override the definitions | 22:31 |
| rlandy | and have them both point to /home/zuul/* | 22:31 |
| weshay | reviewing http://git.openstack.org/cgit/openstack/tripleo-quickstart/tree/roles/libvirt/setup/user/tasks/main.yml#n27 | 22:32 |
| weshay | http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/baremetal-undercloud/defaults/main.yml#n3 | 22:33 |
| weshay | undercloud_key: "{{ local_working_dir }}/id_rsa_undercloud" | 22:33 |
| * rlandy tries override | 22:33 | |
| rlandy | undercloud_key: "/home/{{ undercloud_user }}/id_rsa_undercloud" | 22:34 |
| rlandy | would match in both | 22:34 |
| weshay | weird | 22:34 |
| weshay | https://sf.hosted.upshift.rdu2.redhat.com/logs/49/163849/34/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/8296fbf/logs/10.12.52.1/var/log/extra/dump_variables_vars.json.txt.gz | 22:34 |
| weshay | undercloud_key is not defined | 22:34 |
| rlandy | it is | 22:34 |
| weshay | ah.. maybe not on the virthost | 22:35 |
| weshay | that's the issue | 22:35 |
| weshay | we need to gather_facts? | 22:35 |
| weshay | let's chat | 22:35 |
| rlandy | k | 22:36 |
| weshay | I think you found a booog | 22:36 |
| weshay | rlandy https://sf.hosted.upshift.rdu2.redhat.com/logs/49/163849/34/check/periodic-tripleo-ci-centos-7-baremetal-3ctlr_1comp-featureset001-master/8296fbf/logs/10.12.52.1/var/log/extra/dump_variables_vars.json.txt.gz | 22:37 |
| *** dsneddon has quit IRC | 22:42 | |
| *** vinaykns has quit IRC | 22:43 | |
| *** dsneddon has joined #oooq | 22:43 | |
| sshnaidm|afk | rlandy, weshay, https://review.openstack.org/#/q/topic:atop1+(status:open+OR+status:merged) | 22:49 |
| sshnaidm|afk | rlandy, weshay https://review.openstack.org/#/q/topic:newara+(status:open+OR+status:merged) | 22:49 |
| sshnaidm|afk | in your time please | 22:49 |
| arxcruz | sshnaidm|afk: it's late, you should be drinking beer | 22:53 |
| sshnaidm|afk | arxcruz, who is talking | 22:53 |
| arxcruz | sshnaidm|afk: i had 5 glasses of beer what about you? :P | 22:53 |
| sshnaidm|afk | arxcruz, gehen schlafen! | 22:54 |
| arxcruz | sshnaidm|afk: genau ich wurder in eine minute | 22:55 |
| arxcruz | :D | 22:56 |
| sshnaidm|afk | arxcruz, oktoberfest wurst! | 22:57 |
| * sshnaidm|afk reached limit of german vocabulary | 22:58 | |
| sshnaidm|afk | arxcruz, und zahlen bitte | 22:58 |
| arxcruz | sshnaidm|afk: warum muss ich fur bier zahlen? :D | 23:00 |
| arxcruz | na klar musse ich zahlen | 23:00 |
| arxcruz | sshnaidm|afk: you must visit berlin with weshay :D | 23:00 |
| arxcruz | next week we have openstack something meeting | 23:01 |
| sshnaidm|afk | arxcruz, natulich, Ich muss | 23:03 |
| arxcruz | sshnaidm|afk: wenn ? | 23:03 |
| sshnaidm|afk | gott weiss :) | 23:04 |
| arxcruz | sshnaidm|afk: :D soon I hope, they have good places with good coffee, I found an Israeli coffee place that you would love | 23:07 |
| sshnaidm|afk | arxcruz, yeah, there are a lot of israelis, popular place | 23:07 |
| *** rascasoft has quit IRC | 23:10 | |
| weshay | rlandy undercloud_user: "{{ lookup('env','USER') }}" | 23:21 |
| *** rlandy is now known as rlandy|bbl | 23:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!