| *** tosky has quit IRC | 00:08 | |
| *** hamzy_ has joined #oooq | 00:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, (2 more messages) | 00:26 |
|---|---|---|
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, (2 more messages) | 02:26 |
| *** dsneddon has quit IRC | 02:32 | |
| *** saneax has joined #oooq | 02:41 | |
| *** dsneddon has joined #oooq | 02:57 | |
| *** dsneddon has quit IRC | 03:01 | |
| *** d0ugal has quit IRC | 03:10 | |
| *** apetrich has quit IRC | 03:13 | |
| *** saneax has quit IRC | 03:20 | |
| *** ykarel has joined #oooq | 03:21 | |
| *** dsneddon has joined #oooq | 03:25 | |
| *** dsneddon has quit IRC | 03:31 | |
| *** ykarel has quit IRC | 03:48 | |
| *** udesale has joined #oooq | 03:50 | |
| *** dsneddon has joined #oooq | 03:55 | |
| *** dsneddon has quit IRC | 04:00 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 04:26 |
| *** ykarel has joined #oooq | 04:43 | |
| *** sshnaidm has quit IRC | 04:52 | |
| *** sshnaidm has joined #oooq | 04:55 | |
| *** dsneddon has joined #oooq | 04:56 | |
| *** d0ugal has joined #oooq | 05:04 | |
| *** dsneddon has quit IRC | 05:05 | |
| *** ratailor has joined #oooq | 05:05 | |
| *** ykarel has quit IRC | 05:08 | |
| *** ykarel has joined #oooq | 05:25 | |
| *** udesale has quit IRC | 05:35 | |
| *** udesale has joined #oooq | 05:40 | |
| *** saneax has joined #oooq | 05:43 | |
| *** ratailor_ has joined #oooq | 05:53 | |
| *** ratailor__ has joined #oooq | 05:55 | |
| *** ratailor has quit IRC | 05:56 | |
| *** ratailor_ has quit IRC | 05:58 | |
| *** dsneddon has joined #oooq | 06:01 | |
| *** dsneddon has quit IRC | 06:06 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 06:27 |
| *** dsneddon has joined #oooq | 06:27 | |
| *** dsneddon has quit IRC | 06:32 | |
| *** jfrancoa has joined #oooq | 06:41 | |
| sshnaidm | hubbot1++ | 06:57 |
| hubbot1 | sshnaidm: hubbot1's karma is now 2 | 06:57 |
| *** udesale has quit IRC | 07:11 | |
| *** quiquell|off is now known as quiquell | 07:13 | |
| *** jtomasek has joined #oooq | 07:19 | |
| *** jtomasek has quit IRC | 07:20 | |
| *** jtomase__ has joined #oooq | 07:21 | |
| sshnaidm | quiquell, o/ | 07:22 |
| quiquell | sshnaidm: o/ hello there | 07:23 |
| sshnaidm | quiquell, can you please remind how to do ci in ci? gonna try it now | 07:23 |
| *** udesale has joined #oooq | 07:24 | |
| quiquell | sshnaidm: Can you open the google doc with tripleo infra passwords ? | 07:25 |
| quiquell | sshnaidm: YYou have to get the gerrit user priv keyand put the path with tripleo_ci_gerrit_key | 07:26 |
| sshnaidm | quiquell, ya, opened | 07:26 |
| quiquell | sshnaidm: by default is ~/.ssh/tripleo_ci_gerrit_id_rsa | 07:27 |
| quiquell | sshnaidm: restart and that's all | 07:27 |
| sshnaidm | is it last one in the doc? | 07:27 |
| quiquell | sshnaidm: you will be able to start repro(repro) | 07:27 |
| *** dsneddon has joined #oooq | 07:27 | |
| quiquell | sshnaidm: btw for OVB encryption maybe you can use similar mechanism | 07:28 |
| *** udesale has quit IRC | 07:29 | |
| *** udesale has joined #oooq | 07:29 | |
| *** apetrich has joined #oooq | 07:31 | |
| *** ykarel is now known as ykarel|lunch | 07:32 | |
| *** dsneddon has quit IRC | 07:32 | |
| *** rascasoft has joined #oooq | 07:35 | |
| sshnaidm | quiquell, what do we do with nodepool-setup role? | 07:49 |
| sshnaidm | quiquell, changes there are gonna break current reproducer, and I'd like to keep it live | 07:50 |
| sshnaidm | quiquell, at least while we don't have new one.. | 07:50 |
| quiquell | sshnaidm: Going to discard that, changed needed now are so small than I am going to put them in the repro role | 07:53 |
| quiquell | sshnaidm: Also going to parallelize so we reduce update times | 07:53 |
| sshnaidm | quiquell, cool | 07:53 |
| quiquell | yep forget about the review | 07:54 |
| quiquell | sshnaidm: but libvirt role changes have to be compatible | 07:54 |
| sshnaidm | quiquell, yeah, need to test them for regression | 07:54 |
| sshnaidm | both with old reproducer and quickstart.sh | 07:54 |
| sshnaidm | still think it was a mistake to reuse this part for reproducer.. | 07:55 |
| quiquell | sshnaidm: libvirt too ? | 07:57 |
| quiquell | sshnaidm: I am not going to change nodepool-setup | 07:57 |
| quiquell | sshnaidm: You mean seting livirt with other mechanism like vagrant or something ? | 07:57 |
| quiquell | sshnaidm: or new roles | 07:57 |
| sshnaidm | quiquell, I mean old libvirt code | 07:57 |
| quiquell | sshnaidm: We are in time to change that | 07:57 |
| quiquell | sshnaidm: I don't like it either | 07:58 |
| quiquell | sshnaidm: brb | 07:58 |
| *** quiquell is now known as quiquell|brb | 07:58 | |
| sshnaidm | but you have it working already, so.. | 07:58 |
| quiquell|brb | sshnaidm: now that we have CI we can rework it easily later on | 07:58 |
| sshnaidm | yeah | 07:59 |
| quiquell|brb | sshnaidm: and replace with other stuff | 07:59 |
| quiquell|brb | sshnaidm: not possile before | 07:59 |
| *** kopecmartin|off is now known as kopecmartin | 08:15 | |
| *** dsneddon has joined #oooq | 08:17 | |
| *** jtomase__ has quit IRC | 08:21 | |
| *** jtomasek has joined #oooq | 08:22 | |
| *** dsneddon has quit IRC | 08:22 | |
| *** quiquell|brb is now known as quiquell | 08:23 | |
| quiquell | sshnaidm: maybe we can use one ansible Galaxy role | 08:25 |
| quiquell | There are some good libvirt ones around | 08:25 |
| sshnaidm | quiquell, I don't like things we don't control | 08:26 |
| sshnaidm | quiquell, we have already always breaking puppet modules in ooo... | 08:26 |
| quiquell | Well is like other upstream projects we pin it | 08:26 |
| quiquell | We use exactly working version | 08:26 |
| sshnaidm | maybe we can write a new one :) | 08:26 |
| quiquell | It's a pity there are a lot of good roles there | 08:27 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci- (2 more messages) | 08:27 |
| quiquell | It's like all the upstream pip install we do | 08:27 |
| quiquell | But for roles | 08:27 |
| sshnaidm | quiquell, yeah, maybe we need to consider this | 08:27 |
| quiquell | I think so | 08:28 |
| quiquell | It's better to help at upstream project than maintain and adhoc one | 08:28 |
| chandan_kumar | quiquell: for handling pip installation stuff in a venv you can take a alook at this role https://github.com/openstack/ansible-role-python_venv_build | 08:28 |
| chandan_kumar | it is used in osa side and it is coming as a part of os_tempest | 08:29 |
| *** ykarel|lunch is now known as ykarel | 08:35 | |
| *** tosky has joined #oooq | 08:37 | |
| *** ccamacho has joined #oooq | 08:52 | |
| *** dsneddon has joined #oooq | 08:54 | |
| *** holser_ has joined #oooq | 08:54 | |
| quiquell | sshnaidm: yesterday assigned you the centos7 task | 08:55 |
| quiquell | about repro | 08:55 |
| sshnaidm | quiquell, any other config changes for running ci in ci? | 08:55 |
| sshnaidm | quiquell, ok | 08:56 |
| quiquell | sshnaidm: nope | 08:56 |
| quiquell | sshnaidm: not working ? | 08:56 |
| sshnaidm | quiquell, seems like it doesn't run for me.. http://paste.openstack.org/show/742712/ | 08:56 |
| quiquell | sshnaidm: go to jobs section at your local zuul ensure that you have the job there | 08:57 |
| sshnaidm | quiquell, oops.. seems like schduler crashed | 08:57 |
| sshnaidm | wtf.. "Host '172.18.0.6' is not allowed to connect to this MariaDB server | 08:58 |
| *** dsneddon has quit IRC | 09:01 | |
| *** jpena|off is now known as jpena | 09:01 | |
| quiquell | sshnaidm: that have to be only the web | 09:01 |
| quiquell | sshnaidm: do total wipeout | 09:01 |
| quiquell | sshnaidm: this thing is changing stuff at zuul-config and all | 09:01 |
| quiquell | sshnaidm: if you haven't clean the anon volumes It can give you issues | 09:02 |
| sshnaidm | quiquell, I should be able to set dependency in the local patch on patch in review.rdoproject.org, right? | 09:08 |
| sshnaidm | to test changes in the role | 09:08 |
| quiquell | sshnaidm: yep | 09:08 |
| quiquell | zuul way | 09:08 |
| sshnaidm | interesting to try to test nodepool job https://review.openstack.org/#/c/630649/ | 09:13 |
| *** ratailor_ has joined #oooq | 09:13 | |
| quiquell | sshnaidm: you can use your custom build nodepool with | 09:14 |
| quiquell | nodepool_build role option | 09:14 |
| quiquell | so you point to your change | 09:14 |
| sshnaidm | quiquell, I just built a new container with patched nodepool for now | 09:14 |
| quiquell | sshnaidm: fair enough | 09:14 |
| sshnaidm | quiquell, but CI jobs for my patch fails, maybe to reproduce them.. | 09:14 |
| quiquell | sshnaidm: test first with normal nodepool | 09:16 |
| quiquell | sshnaidm: to ensure that it works | 09:16 |
| quiquell | sshnaidm: then update nodepool | 09:16 |
| *** ratailor__ has quit IRC | 09:16 | |
| sshnaidm | yeah | 09:16 |
| *** dsneddon has joined #oooq | 09:21 | |
| sshnaidm | quiquell, the same error I had before: http://paste.openstack.org/show/742713/ | 09:24 |
| quiquell | sshnaidm: this is on run or startup ? | 09:28 |
| quiquell | sshnaidm: is like required-projects does not include it | 09:28 |
| quiquell | sshnaidm: whell the Depends-On is towards it so it has to include it | 09:28 |
| *** brault has joined #oooq | 09:29 | |
| sshnaidm | quiquell, added depends-on now, let's see | 09:29 |
| sshnaidm | quiquell, was running without it before | 09:29 |
| quiquell | sshnaidm: Let check maybe is something wrong with the connections | 09:29 |
| quiquell | sshnaidm: And we have to qualify them with review.rdoprojects.org | 09:30 |
| quiquell | sshnaidm: to be compatible with RDO | 09:30 |
| quiquell | Let me check because yesterady I was using it | 09:30 |
| quiquell | hummm | 09:32 |
| sshnaidm | quiquell, with dependency works fine | 09:35 |
| sshnaidm | quiquell, but doesn't clone it if no depends-on patch, need to add to required project | 09:35 |
| *** derekh has joined #oooq | 09:37 | |
| quiquell | sshnaidm: I think we have something wrong at the connections | 09:37 |
| quiquell | sshnaidm: we have to replace rdoproject.org with review.rdoproject.org so we have same as RDO and inventory is correct | 09:38 |
| quiquell | sshnaidm: can you try that without the Depends-On ? | 09:38 |
| sshnaidm | quiquell, I think it should be same, it's from config of review.rdoproject.org | 09:39 |
| sshnaidm | quiquell, will try, but not expecting for difference.. | 09:39 |
| quiquell | sshnaidm: Humm was working for me beacuase I have a Depends-On | 09:41 |
| quiquell | :-/ | 09:41 |
| quiquell | Let fix it | 09:41 |
| *** bogdando has joined #oooq | 09:50 | |
| *** saneax has quit IRC | 09:51 | |
| *** saneax has joined #oooq | 09:53 | |
| *** dsneddon has quit IRC | 09:57 | |
| *** chem has joined #oooq | 10:15 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ (2 more messages) | 10:27 |
| *** sshnaidm is now known as sshnaidm|afk | 10:28 | |
| *** dsneddon has joined #oooq | 10:31 | |
| *** dsneddon has quit IRC | 10:36 | |
| *** ratailor__ has joined #oooq | 10:37 | |
| *** ratailor_ has quit IRC | 10:39 | |
| quiquell | ssbarnea|rover, sshnaidm|afk: answered https://review.openstack.org/#/c/629839 | 10:43 |
| quiquell | sshnaidm|afk: found the issue with CI(CI) we need to add required-projects to base job | 10:46 |
| quiquell | sshnaidm|afk: since we are launching the job from diferent project | 10:46 |
| ssbarnea|rover | quiquell: i knew, that's why I didn't comment on this yesterday. Nice work picking it! {{ lookup('env', ... tricks everyone as we forget that it does the lookup on controller. | 10:54 |
| ssbarnea|rover | quiquell: you better fix this in another patch so we can merge it faster. | 10:54 |
| quiquell | ssbarnea|rover: You are hiting the issue with old repro too ? | 10:55 |
| quiquell | ssbarnea|rover: I though it was only at the new zuul repro | 10:55 |
| ssbarnea|rover | i had a similar issue in another place, not sure where. | 10:56 |
| quiquell | ssbarnea|rover: Do you have a libvirt reproducer or quickstart.sh libvirt running ? | 10:56 |
| quiquell | ssbarnea|rover: I have to test this with quickstart.sh | 10:56 |
| ssbarnea|rover | quiquell: but this reminded me that the logic around pub_key is broken, you remember my patch that allowed us to work ssh agent and without an unsafe key on disk? | 10:56 |
| quiquell | ssbarnea|rover: yep | 10:57 |
| ssbarnea|rover | we will have to do the same here, to load the pub key from agent when the key is not on disk. | 10:57 |
| quiquell | ssbarnea|rover: Humm... can you give me a pointer to your fix ? | 10:58 |
| quiquell | ssbarnea|rover: we have a CI on this so we will se if it works very quick | 10:58 |
| ssbarnea|rover | quiquell: yep, look at https://review.openstack.org/#/c/629121/ | 10:58 |
| ssbarnea|rover | quiquell: i am not sure if we will see this on ci because for some reason I do not fully understand our zuul workers do have a key present on their ~/.ssh folder. I would have prefered to use agent there too. | 11:00 |
| ssbarnea|rover | but no need to worry, i will soon add a test for this valid usecase. | 11:00 |
| ssbarnea|rover | just fixed the broken fedora:28 image. | 11:01 |
| quiquell | ssbarnea|rover: nodepool image ? | 11:01 |
| ssbarnea|rover | nope, the docker.io one. | 11:02 |
| quiquell | ssbarnea|rover: ack | 11:02 |
| *** udesale has quit IRC | 11:12 | |
| quiquell | ssbarnea|rover: is this issue important ? http://logs.openstack.org/45/627545/7/check/openstack-tox-linters/bf1e091/job-output.txt.gz#_2019-01-15_11_17_56_568084 | 11:13 |
| *** panda|off is now known as panda | 11:13 | |
| *** dtantsur has joined #oooq | 11:14 | |
| ssbarnea|rover | quiquell: no, is known and outside out power to fix. just a warn to ignore. | 11:18 |
| quiquell | ssbarnea|rover: ack +2 | 11:18 |
| ssbarnea|rover | quiquell: some good news around this https://github.com/pypa/pip/issues/4759 -- aparently fix was merged into pip recently so next version may no longer display it.... and break something else. | 11:20 |
| quiquell | ssbarnea|rover: let's wait for break and cry | 11:21 |
| *** ratailor_ has joined #oooq | 11:26 | |
| *** ratailor__ has quit IRC | 11:29 | |
| *** ratailor__ has joined #oooq | 11:30 | |
| ssbarnea|rover | sshnaidm|afk: can you please wf https://review.openstack.org/#/c/631141/1 ? its dependency was merged. | 11:30 |
| *** ratailor_ has quit IRC | 11:33 | |
| *** ratailor has joined #oooq | 11:34 | |
| *** ratailor__ has quit IRC | 11:35 | |
| * marios biab errand | 11:47 | |
| *** dsneddon has joined #oooq | 11:57 | |
| *** holser_ is now known as holser|lunch | 12:17 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container- (2 more messages) | 12:27 |
| *** panda is now known as panda|lunch | 12:31 | |
| *** jpena is now known as jpena|lunch | 12:35 | |
| quiquell | sshnaidm|afk: I think now the selinux issue with ssh is affecting the fedora28 jobs too :-/ I am going to fix this | 12:40 |
| *** sshnaidm|afk is now known as sshnaidm | 12:41 | |
| *** udesale has joined #oooq | 12:42 | |
| quiquell | sshnaidm: I am going to try with "secrets" from docker compose | 12:49 |
| *** ykarel is now known as ykarel|away | 12:54 | |
| chandan_kumar | weshay: arxcruz|ruck os_tempest requires one more role https://github.com/openstack/ansible-config_template | 13:08 |
| *** dsneddon has quit IRC | 13:12 | |
| weshay | marios, is there any scenario003 job voting atm? | 13:13 |
| weshay | wondering if I can skip it for promotion | 13:13 |
| *** dsneddon has joined #oooq | 13:16 | |
| marios | weshay: scenario003-standalone is non voting for https://bugs.launchpad.net/tripleo/+bug/1811004 | 13:20 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,In progress] - Assigned to Michele Baldessari (michele) | 13:20 |
| marios | weshay: scenario01-2-4 are voting | 13:20 |
| marios | weshay: this what you mean? | 13:20 |
| weshay | scen003 | 13:20 |
| weshay | the promotion job for scen03 is failing | 13:20 |
| weshay | just thinking I can ignore it | 13:21 |
| weshay | as it's not voting right? | 13:21 |
| *** ratailor_ has joined #oooq | 13:21 | |
| weshay | but is the multinode version voting anywhere? | 13:21 |
| *** ratailor has quit IRC | 13:23 | |
| marios | weshay: right | 13:23 |
| marios | weshay: multinode don't think votes anywhere but have to check sec | 13:24 |
| marios | weshay: i have a relevant comment there "OK so we establish these can at least run, 3 fails as expected on the known https://bugs.launchpad.net/tripleo/+bug/1811004 tempest fail" https://review.rdoproject.org/r/#/c/18156/ | 13:24 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,In progress] - Assigned to Michele Baldessari (michele) | 13:24 |
| marios | weshay: jan 11 | 13:25 |
| *** trown|outtypewww is now known as trown | 13:25 | |
| *** ykarel|away has quit IRC | 13:26 | |
| *** ratailor_ has quit IRC | 13:27 | |
| *** holser|lunch is now known as holser_ | 13:27 | |
| marios | https://github.com/openstack-infra/tripleo-ci/blob/ae3f0f9e929e8d156cb2702954a09fd4ba8ae779/zuul.d/multinode-jobs.yaml#L157 weshay | 13:28 |
| *** panda|lunch is now known as panda | 13:28 | |
| *** dsneddon has quit IRC | 13:29 | |
| marios | weshay: looks like yatin found the issue in https://bugs.launchpad.net/tripleo/+bug/1811004 looks like https://review.openstack.org/#/c/626177/3 unproven yet but more i stare at it more likely it seems | 13:29 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,In progress] - Assigned to Michele Baldessari (michele) | 13:29 |
| marios | weshay: if i dont find a nicer way i will post a revert with DNM: testing to see what happens with scen3 but hopefully i find a better way | 13:30 |
| marios | weshay: (before i leave today i mean) | 13:30 |
| * chandan_kumar will be away for few hours. | 13:33 | |
| *** jpena|lunch is now known as jpena | 13:34 | |
| *** dsneddon has joined #oooq | 13:39 | |
| *** rlandy has joined #oooq | 13:39 | |
| weshay | ssbarnea|rover, marios, fyi I'm going to promote master | 13:39 |
| weshay | with or with out fs020 | 13:40 |
| weshay | w/o fs18 | 13:40 |
| weshay | w/o fs30 which is scenario07 | 13:40 |
| weshay | any questions or concerns? | 13:40 |
| *** d0ugal has quit IRC | 13:41 | |
| quiquell | sshnaidm: https://review.rdoproject.org/r/18352 <- inject keys | 13:41 |
| *** d0ugal has joined #oooq | 13:42 | |
| *** dtantsur is now known as dtantsur|brb | 13:42 | |
| *** chem has quit IRC | 13:43 | |
| *** ykarel|away has joined #oooq | 13:47 | |
| *** ykarel|away is now known as ykarel | 13:50 | |
| weshay | ssbarnea|rover, https://review.rdoproject.org/r/#/c/18354/ | 13:55 |
| weshay | ssbarnea|rover, please vote | 13:55 |
| ssbarnea|rover | weshay: done. me out for launch. | 13:57 |
| ykarel | weshay, scenario007 has different issue | 13:57 |
| ykarel | the one for wihch there is tempest patch | 13:57 |
| weshay | ykarel, k | 13:57 |
| weshay | thanks | 13:57 |
| weshay | ykarel, incoming master promotion :) | 13:58 |
| weshay | ykarel, you good w/ that/ | 13:58 |
| weshay | > | 13:58 |
| weshay | ugh | 13:58 |
| weshay | ? | 13:58 |
| weshay | ykarel, https://review.rdoproject.org/r/#/c/18354/ | 13:58 |
| ykarel | weshay, ack if jobs are non voting then it's okk | 13:58 |
| weshay | yup | 13:58 |
| ykarel | but good to fix standalone scen3 | 13:58 |
| ykarel | i seen it's voting at some place | 13:59 |
| weshay | nope | 13:59 |
| weshay | shouldn't be anymore, if it is.. point marios at it :) | 13:59 |
| weshay | brb | 13:59 |
| ykarel | hmm just checked it's non voting, but running at gate | 14:00 |
| ykarel | example https://review.openstack.org/#/c/630918/ | 14:00 |
| marios | weshay: ykarel ? scen 3 should be non voting and not in gate | 14:00 |
| ykarel | marios example https://review.openstack.org/#/c/630918/ | 14:00 |
| marios | ykarel: yeah thanks its mistral | 14:00 |
| marios | ykarel: i'll post a fix for them | 14:01 |
| marios | ykarel: since you're here | 14:01 |
| sshnaidm | quiquell, nice, and what is problem with /var/playbooks? | 14:01 |
| marios | ykarel: about to post a test for the transport url i dind't see one already | 14:01 |
| marios | ykarel: (i.e. checking did you post something?) am trying to make it minimal in tht lets see | 14:01 |
| marios | will post in a sec but the etc/nova.conf on the logs says your'e right | 14:01 |
| marios | ykarel: its even already in bandini comment #10 https://bugs.launchpad.net/tripleo/+bug/1811004 though we didn't realise it at the time | 14:02 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,In progress] - Assigned to Michele Baldessari (michele) | 14:02 |
| marios | :) | 14:02 |
| *** dsneddon has quit IRC | 14:03 | |
| ykarel | marios, ack | 14:03 |
| ykarel | marios, nope /me not posted any patch for it | 14:04 |
| weshay | panda, 1min | 14:04 |
| marios | ykarel: thanks checked tht couldn't see but just checking since you're here :) | 14:04 |
| ykarel | ack :) | 14:06 |
| rlandy | sshnaidm: do we still adjust mtus on ovb? | 14:06 |
| *** Goneri has joined #oooq | 14:08 | |
| sshnaidm | rlandy, I don't think so | 14:08 |
| rlandy | eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1350 .. hmmm we do | 14:08 |
| rlandy | br-ctlplane: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1350 | 14:08 |
| *** chem has joined #oooq | 14:16 | |
| *** vinaykns has joined #oooq | 14:16 | |
| *** d0ugal has quit IRC | 14:18 | |
| *** d0ugal has joined #oooq | 14:18 | |
| quiquell | sshnaidm, rlandy: reproducer CI is failing, I think they have update the fedora28 images and it's affecting it, not sure yet | 14:23 |
| *** Goneri has quit IRC | 14:24 | |
| sshnaidm | quiquell, selinux issues? | 14:24 |
| rlandy | quiquell: k - are we running centos7 as well? | 14:24 |
| rlandy | that one passing? | 14:25 |
| quiquell | rlandy: fedora28 for the moment | 14:25 |
| rlandy | ok | 14:25 |
| quiquell | sshnaidm: I have put this to mitigate the selinux issue but still failing https://review.rdoproject.org/r/#/c/18352/ | 14:25 |
| sshnaidm | rlandy, centos didn't work at all | 14:25 |
| quiquell | sshnaidm: going to test the fix .ssh at centos job | 14:25 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container- (2 more messages) | 14:27 |
| sshnaidm | quiquell, surprisingly I don't have cloud-init working on fedora28 :/ | 14:28 |
| quiquell | sshnaidm: good day bad day | 14:29 |
| sshnaidm | oh, yeah | 14:29 |
| quiquell | we need a little of lithium | 14:29 |
| sshnaidm | quiquell, but it's interesting if images were updated today | 14:30 |
| rlandy | weshay: I think found the missing change for baremetal ... https://github.com/openstack/tripleo-heat-templates/commits/master?after=f89de7a56903fa26c639f7531baeaf421e7910c5+139 - two MTU changes on Dec 30 - we are not using the CI templates | 14:30 |
| rlandy | non-standard config | 14:30 |
| rlandy | if this goes forward we *need* to get on multi-nic and be more standard | 14:31 |
| rlandy | quiquell: we forgot one requirement for job launcher/reproducer - we meed a depends-on on the testproject job | 14:33 |
| rlandy | adding that | 14:33 |
| rlandy | for the original CI'ed change | 14:33 |
| quiquell | rlandy: So we need to passe the change to the role and addit to the git commit | 14:34 |
| quiquell | rlandy: I suspect this is not ready to talk about https://review.openstack.org/#/c/631067/ | 14:34 |
| rlandy | quiquell: no - I got caught up in the baremetal rant I was posting above - it will be ready for tomorrow's discussion | 14:37 |
| rlandy | I need to rethink the vars | 14:37 |
| quiquell | rlandy: ack, no worries I have being swamped today too with this CI thing :-( | 14:37 |
| rlandy | the options are ready for discussion | 14:37 |
| quiquell | sshnaidm, rlandy: centos7 does not come up too at CI :-( | 14:39 |
| quiquell | Drop now | 14:39 |
| quiquell | More pain tomorrow | 14:39 |
| *** quiquell is now known as quiquell|off | 14:39 | |
| *** ssbarnea|rover has quit IRC | 14:40 | |
| *** ssbarnea|rover has joined #oooq | 14:40 | |
| *** gouthamr has quit IRC | 14:54 | |
| sshnaidm | ssbarnea|rover, arxcruz|ruck rdo cloud seems to be down | 14:54 |
| sshnaidm | weshay, ^^ | 14:55 |
| weshay | weee | 14:55 |
| weshay | dam | 14:55 |
| weshay | right when we're promoting | 14:55 |
| weshay | YOU BASTARDS | 14:55 |
| weshay | OMG they killed the cloud | 14:55 |
| *** hubbot1 has quit IRC | 14:56 | |
| *** dmellado has quit IRC | 14:56 | |
| sshnaidm | https://i.imgflip.com/2r9vxe.jpg | 14:57 |
| *** dsneddon has joined #oooq | 15:00 | |
| weshay | sshnaidm++ | 15:02 |
| weshay | sshnaidm++ | 15:02 |
| ssbarnea|rover | rdo cloud is like arthur from rdr2, .... is just a cough | 15:02 |
| ssbarnea|rover | perfect timing for the weekly meeting | 15:03 |
| *** dsneddon has quit IRC | 15:14 | |
| rlandy | in sympathy with our government RDO Cloud decided to shut down as well | 15:15 |
| weshay | lolz | 15:24 |
| *** derekh has quit IRC | 15:30 | |
| *** jfrancoa has quit IRC | 15:33 | |
| weshay | ssbarnea|rover, my god you have a lot of patches | 15:35 |
| weshay | 64 | 15:35 |
| weshay | modified in the last two weeks | 15:36 |
| ssbarnea|rover | weshay: hired two clones in asia do help ;) | 15:36 |
| ssbarnea|rover | ~6/day is not too much, I think. | 15:37 |
| weshay | marios++ | 15:42 |
| weshay | re: https://review.openstack.org/#/c/631227/ | 15:42 |
| marios | weshay: wwel lets see not proven yet but its all ykarel | 15:42 |
| weshay | ykarel++ | 15:43 |
| marios | weshay: it was in the comments from bandini already in https://bugs.launchpad.net/tripleo/+bug/1811004 coment #10 but we didn't realise | 15:43 |
| openstack | Launchpad bug 1811004 in tripleo "tripleo-ci-centos-7-scenario003-standalone fails tempest.scenario.test_minimum_basic.TestMinimumBasicScenario" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 15:43 |
| marios | weshay: diff between amqp vs rabbit in the url | 15:43 |
| marios | well we did but the fix proposed was wrong | 15:43 |
| marios | anyway | 15:44 |
| *** jfrancoa has joined #oooq | 15:45 | |
| *** dtantsur|brb is now known as dtantsur | 15:46 | |
| weshay | ykarel, ssbarnea|rover any thoughts on what DFG or project might be a good target to investigate https://bugs.launchpad.net/tripleo/+bug/1811682 | 15:53 |
| openstack | Launchpad bug 1811682 in tripleo "centos-binary-swift-object failed with cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory"" [Critical,Incomplete] | 15:53 |
| ykarel | weshay, currently not able to open logs | 15:54 |
| ykarel | RDO Cloud | 15:54 |
| ykarel | may be the actual issue is somewhere else | 15:54 |
| ykarel | i have seen libibverbs.so.1: cannot open shared object file: No such file or directory earler as well | 15:54 |
| ykarel | after openvswitch update | 15:55 |
| *** derekh has joined #oooq | 15:55 | |
| ykarel | From the logs there, Networking DFG should be contacted | 15:55 |
| *** udesale has quit IRC | 15:56 | |
| weshay | ykarel, hopefully not a red herring.. | 15:56 |
| weshay | won't know until the cloud is back | 15:56 |
| weshay | ykarel, thanks for the help as usual | 15:56 |
| ssbarnea|rover | weshay: looks like ovs to me. | 15:56 |
| *** saneax has quit IRC | 15:56 | |
| * weshay creates a logstash query | 15:56 | |
| ssbarnea|rover | reminds me that ovs was the no1 cause for rdo cloud outages, every time I heard about one, it was about restarting ovs. | 15:57 |
| weshay | ssbarnea|rover, ykarel clearly it's a red herring http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22cannot%20load%20glue%20library%3A%20libibverbs.so.1%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20voting%3A1 | 16:01 |
| weshay | ssbarnea|rover, if you have time to deep dive on that bug.. it's the wrong trace | 16:02 |
| weshay | panda, I'm back in | 16:03 |
| ykarel | weshay, https://bugzilla.redhat.com/show_bug.cgi?id=1658141 | 16:05 |
| openstack | bugzilla.redhat.com bug 1658141 in dpdk "OVS commands return warning messages with 2.10" [Medium,New] - Assigned to ovs-team | 16:05 |
| ykarel | this is what i was referring to ^^ | 16:05 |
| ssbarnea|rover | weshay: indeed, I see this msg in succesful builds too. | 16:05 |
| *** dsneddon has joined #oooq | 16:10 | |
| marios | mergy? https://review.openstack.org/#/q/topic:centralise_scenario_standalone please thanks for reviews | 16:15 |
| marios | thank you sshnaidm | 16:24 |
| *** dsneddon has quit IRC | 16:26 | |
| *** dsneddon has joined #oooq | 16:29 | |
| *** dsneddon has quit IRC | 16:41 | |
| *** gouthamr has joined #oooq | 16:44 | |
| *** ccamacho has quit IRC | 16:56 | |
| *** jfrancoa has quit IRC | 16:57 | |
| *** dsneddon has joined #oooq | 16:59 | |
| *** bogdando has quit IRC | 17:18 | |
| *** kopecmartin is now known as kopecmartin|off | 17:21 | |
| *** dtantsur is now known as dtantsur|afk | 17:31 | |
| *** ykarel has quit IRC | 17:40 | |
| *** saneax has joined #oooq | 17:49 | |
| ssbarnea|rover | weshay: can you help with https://review.openstack.org/#/c/629537/ ? | 17:51 |
| *** chem has quit IRC | 17:52 | |
| *** derekh has quit IRC | 18:00 | |
| *** jpena is now known as jpena|off | 18:03 | |
| *** trown is now known as trown|lunch | 18:19 | |
| *** holser_ has quit IRC | 18:22 | |
| weshay | ssbarnea|rover, /me looks | 18:51 |
| weshay | ssbarnea|rover, +2 wf | 18:52 |
| ssbarnea|rover | weshay: thanks. hopefully this will resolve that issue. | 18:53 |
| *** jtomasek has quit IRC | 19:12 | |
| *** jtomasek has joined #oooq | 19:13 | |
| *** gouthamr_ has joined #oooq | 19:16 | |
| *** brault has quit IRC | 19:18 | |
| *** trown|lunch is now known as trown | 19:21 | |
| *** dmellado has joined #oooq | 19:22 | |
| *** dtrainor has quit IRC | 19:30 | |
| *** hubbot1 has joined #oooq | 19:32 | |
| *** dtrainor has joined #oooq | 19:37 | |
| *** jaosorior has quit IRC | 19:57 | |
| weshay | tempest.scenario.test_network_basic_ops.TestNetworkBasicOps.test_port_security_macspoofing_port [224.764489s] ... FAILED | 20:06 |
| weshay | on single node | 20:06 |
| * weshay will look | 20:06 | |
| *** zul has quit IRC | 20:09 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container- (2 more messages) | 20:29 |
| *** jtomasek_ has joined #oooq | 20:32 | |
| *** jtomasek has quit IRC | 20:36 | |
| weshay | rlandy, ugh.. sorry.. are you able to ssh to the promoter? | 20:49 |
| rlandy | weshay: let me check | 20:57 |
| rlandy | is there an issue | 20:57 |
| rlandy | weshay: yes - I'm in | 21:00 |
| weshay | man.. | 21:00 |
| rlandy | what do you need? | 21:00 |
| weshay | ok.. | 21:00 |
| rlandy | weshay: did you try ssh as yourself? | 21:00 |
| rlandy | hold on let me tmate | 21:00 |
| weshay | I tried myself, centos and root | 21:00 |
| weshay | ooh thanks | 21:00 |
| weshay | 2019-01-16 20:26:31,098 27661 ERROR promoter Command '['ansible-playbook', '/home/centos/ci-config/ci-scripts/container-push/container-push.yml']' returned non-zero exit status 2 | 21:01 |
| weshay | Traceback (most recent call last): | 21:01 |
| weshay | File "/home/centos/ci-config/ci-scripts/dlrnapi_promoter/dlrnapi_promoter.py", line 145, in tag_containers | 21:01 |
| weshay | env=env, stderr=subprocess.STDOUT).split("\n") | 21:01 |
| weshay | File "/usr/lib64/python2.7/subprocess.py", line 575, in check_output | 21:01 |
| weshay | raise CalledProcessError(retcode, cmd, output=output) | 21:01 |
| weshay | rlandy, ah.. it did promote | 21:49 |
| *** holser_ has joined #oooq | 21:49 | |
| rlandy | how though? | 21:51 |
| rlandy | look whatever good new is good news | 21:51 |
| rlandy | we can push rfolco's changes | 21:52 |
| *** gouthamr has quit IRC | 21:56 | |
| *** gouthamr_ is now known as gouthamr | 21:56 | |
| weshay | rlandy, oh ya | 21:56 |
| weshay | rlandy, shall we? | 21:56 |
| rlandy | weshay: up to you - or I will do it tomorrow morning | 21:57 |
| weshay | meh.. let's do it | 21:57 |
| weshay | rlandy, oh wait wait | 21:58 |
| rlandy | looking at card to see what it left | 21:58 |
| weshay | maybe we let rdo p1 promote as well | 21:58 |
| weshay | or yatin will kill us | 21:58 |
| rlandy | fine ok | 21:58 |
| rlandy | watching | 21:58 |
| weshay | tomorrow hopefully :) | 21:58 |
| * rlandy says small prayer for rdo p1 | 21:59 | |
| *** trown is now known as trown|outtypewww | 22:05 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-scenario003-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container- (2 more messages) | 22:29 |
| *** jtomasek_ has quit IRC | 22:32 | |
| *** jtomasek has joined #oooq | 22:33 | |
| *** jtomasek has quit IRC | 22:42 | |
| *** vinaykns has quit IRC | 22:47 | |
| *** rascasoft has quit IRC | 23:13 | |
| *** rlandy is now known as rlandy|bbl | 23:26 | |
| *** sshnaidm is now known as sshnaidm|afk | 23:43 | |
| *** rascasoft has joined #oooq | 23:46 | |
| *** saneax has quit IRC | 23:48 | |
| *** holser_ has quit IRC | 23:50 | |
| *** rascasoft has quit IRC | 23:51 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
