| *** dsneddon has joined #oooq | 00:19 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario004-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7 (2 more messages) | 00:36 |
|---|---|---|
| *** weshay has quit IRC | 00:59 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci- (2 more messages) | 02:37 |
| *** ykarel has joined #oooq | 03:01 | |
| *** udesale has joined #oooq | 04:00 | |
| *** udesale has quit IRC | 04:00 | |
| *** udesale has joined #oooq | 04:01 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci- (2 more messages) | 04:37 |
| *** ykarel has quit IRC | 05:19 | |
| *** ratailor has joined #oooq | 05:23 | |
| *** skramaja has joined #oooq | 05:28 | |
| *** ratailor_ has joined #oooq | 05:34 | |
| *** ratailor has quit IRC | 05:37 | |
| *** ratailor__ has joined #oooq | 05:37 | |
| *** ratailor_ has quit IRC | 05:40 | |
| *** ykarel has joined #oooq | 05:47 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci- (2 more messages) | 06:37 |
| *** quiquell|off is now known as quiquell | 06:44 | |
| *** jtomasek has joined #oooq | 06:48 | |
| *** saneax has joined #oooq | 06:55 | |
| quiquell | sshnaidm: gm we need to merge tq libvirt changes to merge repro libvirt | 06:58 |
| quiquell | Marios, ssbarnea|rover: ∆ | 06:59 |
| sshnaidm | quiquell, you mean this one? https://review.openstack.org/#/c/629839/ | 07:00 |
| marios | quiquell: ack will check | 07:00 |
| quiquell | The depends on of the RDO patch | 07:00 |
| quiquell | I will ask people to merge the one from zuul job | 07:01 |
| quiquell | Thanks | 07:01 |
| sshnaidm | quiquell, why do we need root here? https://review.openstack.org/#/c/629839/27/roles/libvirt/setup/common/tasks/main.yml | 07:02 |
| sshnaidm | quiquell, marios I need your votes please on https://review.openstack.org/#/c/631955/ , it's critical for OVB CI | 07:03 |
| quiquell | sshnaidm: by default the dir is /opt/vm_images | 07:05 |
| quiquell | Tried to change default and didn't work | 07:05 |
| quiquell | I can try again I don't like become there too | 07:06 |
| sshnaidm | quiquell, did you change libvirt_volume_path and it didn't work? | 07:06 |
| quiquell | Yep | 07:08 |
| marios | sshnaidm: ack | 07:08 |
| sshnaidm | quiquell, it's weird, what does override it? | 07:08 |
| quiquell | I think the problem is the super weird include_vars from over loud | 07:08 |
| quiquell | overcloud | 07:08 |
| quiquell | Hummus like overloud | 07:09 |
| sshnaidm | quiquell, I like hummus | 07:13 |
| sshnaidm | quiquell, but not in overcloud, yeah | 07:13 |
| *** jschlueter has quit IRC | 07:16 | |
| quiquell | sshnaidm: overhummus yikes! | 07:16 |
| sshnaidm | quiquell, panda|off can you take a look please in your time https://review.openstack.org/#/c/631955/ | 07:25 |
| *** ccamacho has joined #oooq | 07:28 | |
| *** apetrich has joined #oooq | 07:49 | |
| *** jfrancoa has joined #oooq | 07:51 | |
| marios | o/ folks grateful for comments and suggestions [docs] https://review.openstack.org/#/c/631024/3 & looks like http://logs.openstack.org/24/631024/3/check/openstack-tox-docs/2df3a00/html/contributor/ci_primer.html | 07:55 |
| *** ykarel is now known as ykarel|lunch | 07:58 | |
| *** dsneddon has quit IRC | 08:01 | |
| *** dsneddon has joined #oooq | 08:02 | |
| *** holser_ has joined #oooq | 08:04 | |
| *** ratailor_ has joined #oooq | 08:08 | |
| *** ratailor__ has quit IRC | 08:11 | |
| *** ykarel|lunch is now known as ykarel | 08:32 | |
| *** gkadam has joined #oooq | 08:34 | |
| *** tosky has joined #oooq | 08:36 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci- (2 more messages) | 08:37 |
| quiquell | marios: commented https://review.openstack.org/#/c/631024/ | 08:38 |
| marios | thanks quiquell | 08:51 |
| marios | quiquell: ill reply but we already have a good section on promotions i link to it already somewhere maybe needs another link | 08:52 |
| quiquell | marios: Yep, promotion section is the place | 08:55 |
| marios | quiquell: thnks | 08:55 |
| quiquell | marios: Maybe link from that part to the section or something | 08:55 |
| quiquell | marios: yw | 08:55 |
| marios | quiquell: yah i will do i replied in there | 08:55 |
| *** saneax is now known as saneax|mtg | 08:59 | |
| *** jpena|off is now known as jpena | 09:03 | |
| *** d0ugal has quit IRC | 09:21 | |
| *** d0ugal has joined #oooq | 09:22 | |
| *** panda|off is now known as panda | 09:29 | |
| *** derekh has joined #oooq | 09:38 | |
| *** bogdando has joined #oooq | 09:41 | |
| *** bogdando has quit IRC | 09:51 | |
| *** holser_ has quit IRC | 09:52 | |
| *** ratailor_ has quit IRC | 09:52 | |
| *** jaosorior has joined #oooq | 09:53 | |
| *** bogdando has joined #oooq | 09:57 | |
| *** dtantsur|afk is now known as dtantsur | 09:57 | |
| *** holser_ has joined #oooq | 09:57 | |
| *** chem has joined #oooq | 09:57 | |
| panda | marios: do you remember why we removed the editor swap files from .gitignore ? | 10:05 |
| marios | panda: line up with other openstack things | 10:06 |
| marios | apparently | 10:06 |
| chandankumar | arxcruz|ruck: Hello | 10:07 |
| marios | panda: i wanted to add to them but was told by alex that we shouldn't in fact lets nuke it | 10:07 |
| chandankumar | arxcruz|ruck: https://review.openstack.org/#/c/631708/ is there a way to set {{ keystone_service_internaluri }} related values automatically in runtime? | 10:07 |
| panda | marios: thanks. I'm -2 a change that is trying to add them again, and I didn't remember the exact reason. | 10:08 |
| marios | panda: sec | 10:10 |
| marios | panda: digging for info in my logs | 10:10 |
| panda | marios: nevermind, your explanation is sufficent | 10:10 |
| marios | https://git.openstack.org/cgit/openstack-dev/cookiecutter/commit/?id=da86f5e042cbe069166d71809d50dea4c8b6aed0 | 10:10 |
| panda | I too was remembering something similar | 10:10 |
| marios | panda: here 18:04 <@mwhahaha> marios: see https://review.openstack.org/#/c/592520/ | 10:10 |
| marios | from november | 10:10 |
| marios | # git config --global core.excludesfile '~/.gitignore' | 10:10 |
| *** ratailor has joined #oooq | 10:12 | |
| *** kopecmartin has joined #oooq | 10:18 | |
| quiquell | sshnaidm, panda: Where do we use the libvirt tq roles ? quicksart libvirt + libvirt reproducer, that's it it ? | 10:19 |
| kopecmartin | arxcruz|ruck, hi, i think that the new time of the meeting would be too late for chandankumar .. .but I'll be here at that time, so I can fill you in on what was discussed then | 10:20 |
| arxcruz|ruck | kopecmartin: ok | 10:20 |
| panda | quiquell: we had a job in ci.centos ages ago. I remember it was failing a alot because we needed a special hardware to test it that was available only 1 run out of 4. Not sure what happened to that job frankly | 10:21 |
| ssbarnea|rover | hi! I am back, please let me know if you already know of a problem. | 10:21 |
| quiquell | panda: So we can forget about it | 10:22 |
| panda | ssbarnea|rover: I have a problem with my kitchen sink. | 10:22 |
| panda | quiquell: no, I can't forget about it, my minds absorbs and retains all kind of useless knowledge. I can try to NOT consider it important. That's the best I can do. | 10:23 |
| ssbarnea|rover | panda: well,i did spend most of the weekend in kitchen, mounting an external exhaust fan, but i would have prefered doing rovering.... | 10:23 |
| panda | ssbarnea|rover: an exhaust fan ? was it chasing you ? are you taht famous ? I think it's illegal to do that to a person | 10:24 |
| sshnaidm | quiquell, the job in ci.centos is actually quickstart libvirt run | 10:24 |
| quiquell | ssbarnea|rover: you have rover your kitchen ? | 10:24 |
| quiquell | sshnaidm: So If I test quickstart libvirt and it works we are good | 10:24 |
| panda | arxcruz|ruck: do you know who natal is ? | 10:25 |
| sshnaidm | quiquell, and libvirt repro | 10:26 |
| arxcruz|ruck | panda: natal is christmas in portuguese | 10:26 |
| quiquell | sshnaidm: ack | 10:26 |
| sshnaidm | quiquell, I was running quickstart.sh with your patch, libvirt part passed ok, failed on undercloud install, but it maybe not related | 10:26 |
| quiquell | sshnaidm: humm ack, going to try it myself | 10:27 |
| sshnaidm | quiquell, oh, it's problem with ntp, sure not related | 10:27 |
| quiquell | sshnaidm: ack then we are good, then let's test repro | 10:28 |
| sshnaidm | quiquell, panda please take a look, it's critical for ovb ci: https://review.openstack.org/#/c/631955/ | 10:28 |
| panda | sshnaidm: had a comment there | 10:30 |
| quiquell | sshnaidm: We don't test ovb changes with build-test-packages ? how do we test them ? | 10:31 |
| panda | arxcruz|ruck: .... | 10:31 |
| quiquell | sshnaidm: humm OVB is very infra... maybe is that | 10:31 |
| panda | arxcruz|ruck: I mean Natal Ngétal, taht is pushing patches on tempestmail | 10:31 |
| ssbarnea|rover | panda: sshnaidm let's help wes merge this https://review.openstack.org/#/c/631825/ -- i tested it. | 10:31 |
| arxcruz|ruck | panda: I think his nickname is Hobbestigrou | 10:32 |
| panda | ssbarnea|rover: I had questions on that | 10:32 |
| arxcruz|ruck | panda: he have a bunch of patches on tq/tqe | 10:32 |
| panda | arxcruz|ruck: I know, but I have questions for him, for example what does he mean with [CORE] in his commit messages. Maybe you knew him. | 10:32 |
| panda | ssbarnea|rover: what are we really testing ? | 10:33 |
| quiquell | ssbarnea|rover: we need libselinux when we are a python3 enviroment, damn... | 10:33 |
| quiquell | pytho2-libselinux I mean | 10:33 |
| panda | ssbarnea|rover: python3 or python2 ? is taht only for detection and then python3 is used ? | 10:33 |
| arxcruz|ruck | panda: I don't know, maybe it's a way to identify easier what about the patches are | 10:33 |
| arxcruz|ruck | panda: some other patches he put [documentation] | 10:33 |
| arxcruz|ruck | I don't mind tbh | 10:33 |
| panda | quiquell: yea, if we end yup testing python2-selinus we are invalidating our tests ... | 10:33 |
| ssbarnea|rover | panda: ansible fails, because ansible uses python2 by default when making a connection, even to localhost. | 10:33 |
| ssbarnea|rover | there is work done into changing what do use, but atm, is kinda hardcoded on python2. | 10:34 |
| panda | ssbarnea|rover: wasn't there a patch to use python3 by default somewhere ? | 10:34 |
| quiquell | ssbarnea|rover: then we have to use ansible-playbook-3 | 10:34 |
| quiquell | ssbarnea|rover: we cannot use ansible-playbook as a moduel I think | 10:34 |
| quiquell | ssbarnea|rover: there were issues with the API | 10:34 |
| panda | ssbarnea|rover: ok, so we are making ansible use python2 or this is just for the detection phase ? | 10:35 |
| ssbarnea|rover | panda: we have no control over that, but this workaround works, tested on fedora 28, latest and rawhide,passed on all. | 10:35 |
| panda | ssbarnea|rover: I'd like to have the ansible bug noted somewhere | 10:35 |
| quiquell | ssbarnea|rover: still, what about using ansible over python3 ? | 10:35 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci- (2 more messages) | 10:37 |
| quiquell | ssbarnea|rover: yep we are using ansible-playbook not ansible-playbook-3 for this case at quickstart.sh | 10:37 |
| quiquell | ssbarnea|rover: and this is part of the f28 efffort we don't use quickstart.sh at CI for test stuff I think | 10:37 |
| quiquell | ssbarnea|rover: we have to do same we did with f28 at quickstart.sh not differnt | 10:38 |
| ssbarnea|rover | quiquell: panda : please read the bug, try to do what user reported and be back with better solutions. | 10:38 |
| *** ratailor has quit IRC | 10:38 | |
| quiquell | ssbarnea|rover: ack | 10:38 |
| panda | ssbarnea|rover: ? | 10:39 |
| ssbarnea|rover | i know the issue quite well as I talked with wes on friday and tested. this is why i supported his patch. from a distance it does not seem the best, but once you go deeper you realise somethign different, please try. | 10:39 |
| ssbarnea|rover | btw, you can easily test this with molecule | 10:39 |
| ssbarnea|rover | in the end is only one simple call of qs, | 10:40 |
| panda | quiquell: ^read what ssbarnea|rover is saying | 10:40 |
| quiquell | panda: I am | 10:40 |
| quiquell | panda: Going to try, best way to understand it | 10:40 |
| ssbarnea|rover | even if you call ansible with python 3, it will endup running with python on the remote, even when the remote is localhost. workaround is to create custom inventory that forces it to use your desired python interpreter. there are multiple ways to call stuff locally with ansible and the results may vary (as we have lots of places over our code, so we cannot fix all of them). | 10:43 |
| quiquell | ssbarnea|rover: ansible_python_interpreter ? | 10:43 |
| quiquell | ssbarnea|rover: My main concern is that it means that we haven't propertly ported quickstart.sh to py3 envs, since we arenot CI it | 10:44 |
| ssbarnea|rover | quiquell: panda : read about this https://github.com/ansible/ansible/issues/45852 -- it should clarify a little bit. also remember that any fix made will not reach our outdated ansible version | 10:44 |
| quiquell | :-/ | 10:44 |
| ssbarnea|rover | shortly, is a tricky business | 10:44 |
| sshnaidm | ssbarnea|rover, I see tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 is broken, do you have a bug about it? | 10:44 |
| quiquell | ssbarnea|rover: yep looks like, let me be a little naif about it, test it | 10:44 |
| ssbarnea|rover | quiquell: sure. anyway this one was not urgent because it affects only people running qs *from* fedora machines, which we do not on ci. | 10:45 |
| panda | ssbarnea|rover: can we add it to this patch ? | 10:46 |
| marios | quiquell: there will be a problem witht he images cos of the key no? | 10:46 |
| panda | ssbarnea|rover: even as a dependent change ? | 10:46 |
| panda | ssbarnea|rover: the ansible link I mean | 10:46 |
| marios | quiquell: i am trying to continue setup for the repro/zuul stuff and just occurred to me the images you shaerd with me and panda with you already virt customize them with your keys | 10:46 |
| panda | ssbarnea|rover: because everyone else is asking the same question "Are we testing it wrong ?" | 10:46 |
| ssbarnea|rover | panda: lets put it in the LP bug., so we avoid rebuilding the patch. ok? | 10:46 |
| quiquell | marios: those images have the team's pub keys on it | 10:46 |
| marios | quiquell: ah that is why sshnaidm was asking for key? | 10:47 |
| panda | marios: didn't start yet at all with the reproducer, planning to do after the morning reviews. | 10:47 |
| quiquell | marios: To be sure try to do a server create with them and access with zuul user you should be able too | 10:47 |
| panda | which already took too much time today | 10:47 |
| ssbarnea|rover | panda: molecule patch is fixing this issue by testing with fedora /end-pr | 10:47 |
| quiquell | ssbarnea|rover: We cannot do ansible_python_interpreter stuff there ? | 10:47 |
| marios | cool thanks quiquell (I recall someone asking me for my key few days ago must ave been for that) | 10:47 |
| marios | ack panda thanks | 10:47 |
| *** ratailor has joined #oooq | 10:48 | |
| quiquell | ssbarnea|rover: Ahh I get it now this is about detecting VIRTHOST distro and the like... | 10:48 |
| panda | ssbarnea|rover: +2 on the patch | 10:48 |
| quiquell | ssbarnea|rover: I mean ansible remote host | 10:48 |
| ssbarnea|rover | quiquell: yes you can, but do you want to add extra messy code to qs.sh? is not fix at all, especially as you may already have an inventory when you call it. i bet you don't want to. | 10:48 |
| quiquell | panda: Yep I am going too | 10:48 |
| quiquell | ssbarnea|rover: I get it now, sorry about my -1 | 10:49 |
| panda | quiquell: ok, was waiting for you to +W, you can do it | 10:49 |
| panda | ssbarnea|rover: but I really want to see a note on the code for this, this is really to weird an it's creating confusion | 10:49 |
| quiquell | panda: Yep I mean, qs.sh is going to be deprectaed so don't add more logic there to detect host stuff and the lke | 10:49 |
| ssbarnea|rover | quiquell++ | 10:50 |
| hubbot1 | ssbarnea|rover: quiquell's karma is now 17 | 10:50 |
| sshnaidm | panda, quiquell responded in https://review.openstack.org/#/c/631955/ | 10:50 |
| panda | quiquell: at this point I'm starting to doubt it, we've been saying "quickstart is deprecated" for the last year and half. Still no suitable alternative. But I agree with you | 10:50 |
| quiquell | sshnaidm: merged | 10:50 |
| quiquell | panda: yep, I am starting to feel the same | 10:50 |
| quiquell | ssbarnea|rover: I mean | 10:51 |
| quiquell | merged ^ | 10:51 |
| sshnaidm | ssbarnea|rover, so, do you know status of tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 ? | 10:51 |
| panda | ssbarnea|rover: how quickly we forget other people | 10:51 |
| quiquell | panda: who are you ? | 10:51 |
| quiquell | panda: I supose alternative to quickstart.sh is just plan playbook ? and just report ansible as a dependency ? | 10:52 |
| ssbarnea|rover | regarding deprecations, I have a proposal: unless we already have a merged partially working alternative, lets not call a piece of code deprecated. Once we merge, I will be happy to replace the cawsay message with a cowsay "deprecated" one ;) | 10:52 |
| quiquell | cowsay++ | 10:52 |
| quiquell | Remember doing a code cata with cowsay :-) | 10:53 |
| quiquell | kata | 10:53 |
| panda | quiquell: ssbarnea|rover the problem was never really deprecating quickstart.sh, but deprecating most of its contents. We wanted to move almost everything taht was there to playbooks | 10:53 |
| panda | not removing the bash access. | 10:53 |
| quiquell | panda: If we just say "requirements: ansible" why don't we just delivery some playbooks ? | 10:53 |
| panda | quiquell: time ? | 10:53 |
| ssbarnea|rover | panda: exactly. this is called refactoring, not reprecation :D | 10:53 |
| panda | quiquell: as always | 10:53 |
| quiquell | panda: we are going to remove everyting to playbooks and roles so time is not issue | 10:54 |
| quiquell | Really using this bash glue code just to remove ansible from requirements does not make sense | 10:54 |
| quiquell | panda: also how much peopleo is using quickstart out of CI ? | 10:55 |
| *** ratailor_ has joined #oooq | 10:55 | |
| ssbarnea|rover | sshnaidm: re fs35, i see it documented on https://review.rdoproject.org/etherpad/p/ruckrover-sprint24 -- by you or arx I guess. | 10:56 |
| panda | quiquell: CI as our team, or as a use target ? | 10:56 |
| quiquell | panda: CI as use target | 10:56 |
| quiquell | panda: All jobs related to tripleo uses quickstart | 10:57 |
| sshnaidm | ssbarnea|rover, not by me sure | 10:57 |
| sshnaidm | ssbarnea|rover, but what is the status?? | 10:57 |
| *** ratailor has quit IRC | 10:57 | |
| sshnaidm | ssbarnea|rover, it's from Dec and not relevant at all | 10:58 |
| quiquell | sshnaidm: stable/1.0 will be obsolete aster everyone uses 2.0 by using master, is that it ? | 10:59 |
| panda | quiquell: if anyone is using it outside the CI scope, he's using it at he's own risk. We had a lot of discussion in the past taht developers were preferring quickstart where they really neede to use tripleo itself | 10:59 |
| quiquell | s/aster/fater/ | 10:59 |
| ssbarnea|rover | sshnaidm: look at last comment, which points to https://review.openstack.org/#/c/576481 which merged few days ago (suspect cause) | 10:59 |
| quiquell | panda: ack | 10:59 |
| sshnaidm | quiquell, I think it will be created 2.0 and master will be future 3.0 | 10:59 |
| sshnaidm | ssbarnea|rover, the job is still faling | 11:00 |
| *** ratailor_ has quit IRC | 11:00 | |
| *** ratailor has joined #oooq | 11:00 | |
| quiquell | sshnaidm: so we will never have stable/stein at OVB project ? | 11:00 |
| sshnaidm | quiquell, nope, these branches are completely different from openstack releases/branches | 11:01 |
| quiquell | sshnaidm: SO we are going to need to change this every time we have a sable branch... | 11:01 |
| quiquell | sshnaidm: And I see stein branch using review from OVB and then master is no good | 11:01 |
| sshnaidm | quiquell, we'll need to add there 2.0 too when it will be stable, that's right | 11:02 |
| sshnaidm | quiquell, but it may not happen at all, for example :) | 11:02 |
| quiquell | sshnaidm: ok, let's think about then, and merge this | 11:02 |
| sshnaidm | quiquell, it's only Ben Nemec that decided about new branch or release there :) | 11:03 |
| quiquell | panda: I am going to workflow OVB review is that ok ? | 11:03 |
| quiquell | panda: We don't really know what's going to happend with future releases | 11:04 |
| panda | quiquell: just a second | 11:04 |
| quiquell | panda: ack, will +2 so you +w | 11:04 |
| panda | sshnaidm: I want to understand, when we gate a change in ovb project, ZUUL_BRANCH is set to stable/1.0, and also zuul.branch ? | 11:04 |
| sshnaidm | panda, it's the same afaik | 11:05 |
| panda | this is bad ... | 11:05 |
| quiquell | ssbarnea|rover: Do you know if libvirt repro is working ? | 11:05 |
| quiquell | sshnaidm: current one I mean not the WIP one | 11:05 |
| panda | changin ZUUL_BRANCH is easy, changing zuul.branch on the other end ... | 11:05 |
| panda | mmhh | 11:05 |
| panda | this may get us in trouble when we want to push forward zuulv3 migration of the scritps | 11:05 |
| ssbarnea|rover | quiquell: it should, it was working last week. send me link to a specific one to test it. | 11:06 |
| quiquell | panda: ZUUL_BRANCH is a reflection of zuul.branch made by zuul | 11:06 |
| sshnaidm | panda, branch: stable/1.0 in https://logs.rdoproject.org/49/631949/3/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/274bf0a/zuul-info/inventory.yaml | 11:06 |
| quiquell | panda: Let's just change this, and do the refactoring later | 11:06 |
| *** ratailor has quit IRC | 11:06 | |
| panda | sshnaidm: ok thanks, I'll try to note this sokmewhere maybe in the epic for zuulv3 migration | 11:06 |
| quiquell | ssbarnea|rover: Just one to test that my libvirt changes for WIP repro is not breaking it | 11:07 |
| sshnaidm | panda, yeah, when changing to zuulv3 we'll have to find solution for feature branches too, and it's not only for OVB | 11:07 |
| panda | quiquell: lol, "I'll do it now, We'll change it later" :P | 11:07 |
| quiquell | ssbarnea|rover: Want to test this one at a libvirt repro https://review.openstack.org/#/c/629839/ | 11:07 |
| sshnaidm | panda, afaik ceph and some other stuff can suffer from this too | 11:07 |
| panda | quiquell: I'm ok with it beacuse we've not actually finished the zuulv3 migration | 11:07 |
| panda | sshnaidm: ok ok. +W, and adding info to the epic | 11:07 |
| quiquell | sshnaidm: at the end we will need a mappingn betwene stable branches and OVB branches | 11:08 |
| sshnaidm | quiquell, panda maybe it's worth to ask zuul folks to support well feature branches | 11:08 |
| sshnaidm | it should be zuul feature | 11:08 |
| quiquell | sshnaidm: Humm but those are not feature branches they are release branches | 11:08 |
| sshnaidm | setting branch for the whole job for all repos - it doesn't work well in this case | 11:09 |
| sshnaidm | quiquell, I mean 1.0/2.0 - are feature branches | 11:09 |
| quiquell | sshnaidm: yep... but still is not a zuul problem | 11:09 |
| sshnaidm | quiquell, or ceph with its jewel, etc | 11:09 |
| sshnaidm | quiquell, zuul set a branch for all repos in job, that's a problem | 11:10 |
| quiquell | sshnaidm: We have to know what version of OVB works with what version of tripleo | 11:10 |
| sshnaidm | quiquell, tbh OVB is not related to tripleo at all | 11:10 |
| quiquell | sshnaidm: and this is not zuul issue we have to map it harcode it | 11:10 |
| sshnaidm | quiquell, OVB is infrastructure for a job | 11:10 |
| quiquell | sshnaidm: Yep I mean, tripleo CI | 11:11 |
| sshnaidm | quiquell, it's like to check that tripleo stein works with zuulv4 | 11:11 |
| quiquell | sshnaidm: not tripleo itself | 11:11 |
| sshnaidm | quiquell, and again, tripleo CI is not related to branches of tripleo | 11:11 |
| quiquell | sshnaidm: I mean, we have some "instrospect" features that may only work with OVB stable/2.0 at stein | 11:11 |
| quiquell | sshnaidm: something like that | 11:11 |
| quiquell | sshnaidm: ahh damnt it | 11:12 |
| quiquell | for get about it | 11:12 |
| quiquell | sshnaidm: brain frat big one | 11:12 |
| quiquell | fart | 11:12 |
| quiquell | Too much coffee this morning | 11:12 |
| sshnaidm | why, it's legitimate questions | 11:12 |
| *** ratailor has joined #oooq | 11:13 | |
| sshnaidm | quiquell, but if OVB will break introspection for example, it will be broken for all releases, just look at it as "hardware preparation" | 11:13 |
| quiquell | sshnaidm: Why all of them ? | 11:14 |
| quiquell | sshnaidm: could be that BMC fix some bug only supperted at new version of introspection | 11:15 |
| panda | ok no more reviews today. REPRODUCER HERE I COME! | 11:18 |
| panda | SSSSPLAAAAAT | 11:18 |
| sshnaidm | quiquell, well, I'm not expert in bmc features for introspection, but may be there is a chance, yeah | 11:20 |
| sshnaidm | quiquell, in this case we can use branch_override variable | 11:20 |
| *** udesale has quit IRC | 11:20 | |
| sshnaidm | quiquell, and to check if branch override exist, and if so - to use it instead of master, or kind of | 11:21 |
| quiquell | sshnaidm: ack | 11:21 |
| chandankumar | sshnaidm: panda arxcruz|ruck this one is good to go https://review.openstack.org/#/c/628421/ no modification needed here | 11:25 |
| panda | chandankumar: this is what you think! | 11:25 |
| chandankumar | panda: other TQE and TOCI changes requires some changes till the tests passes | 11:26 |
| sshnaidm | chandankumar, wow, so many new roles | 11:26 |
| panda | chandankumar: there is a role in ansible to create a virtualenv for ansible ? | 11:27 |
| chandankumar | panda: yes, it is a new role but it does more stuff t | 11:28 |
| chandankumar | panda: https://review.openstack.org/#/c/627482/60/.zuul.yaml@186 | 11:28 |
| chandankumar | panda: installing particular package invenv which python intepreter to choose | 11:29 |
| chandankumar | panda: symlinking a library like libvrit-python or libselinux-python | 11:29 |
| chandankumar | make an venv isolated or with system site packages | 11:29 |
| chandankumar | since we are consuming config_template with os_temepst, we are check with ceph-ansible team to remove these https://github.com/ceph/ceph-ansible/blob/master/plugins/actions/config_template.py and https://github.com/ceph/ceph-ansible/blob/master/library/config_template | 11:30 |
| sshnaidm | ssbarnea|rover, I'd very appreciate if you'd submit a bug about fs035 failing | 11:31 |
| ssbarnea|rover | sshnaidm: isn't the first bullet correct one https://review.rdoproject.org/etherpad/p/ruckrover-sprint24? | 11:32 |
| sshnaidm | ssbarnea|rover, I don't know, you tell me | 11:33 |
| sshnaidm | ssbarnea|rover, it's what ruck and rover do, actually | 11:33 |
| sshnaidm | ssbarnea|rover, is that anybody on it? I see it's promotion blocker, why is it w/o alert? | 11:34 |
| ssbarnea|rover | sshnaidm: i am on call with arx, now.... | 11:34 |
| quiquell | panda: Tell me if the openstack image I shared with you work ok | 11:35 |
| quiquell | panda: They are supose to contain team's pub key | 11:35 |
| panda | quiquell: thanks, I really need a couple of beers. | 11:36 |
| *** ratailor has quit IRC | 11:36 | |
| *** ratailor has joined #oooq | 11:36 | |
| arxcruz|ruck | sshnaidm: bug is opened on 035 | 11:37 |
| arxcruz|ruck | sshnaidm: https://bugs.launchpad.net/tripleo/+bug/1812632 | 11:37 |
| openstack | Launchpad bug 1812632 in tripleo "Overcloud nova compute docker command failing on nova_cellv2_discover_hosts" [Critical,Triaged] | 11:37 |
| sshnaidm | arxcruz|ruck, shouldn't it be with alert if it's promotion blocker? | 11:38 |
| arxcruz|ruck | sshnaidm: done :) | 11:38 |
| sshnaidm | arxcruz|ruck, great, thanks! | 11:38 |
| quiquell | sshnaidm, panda: Do you know why zuul is complaining here ? https://review.rdoproject.org/r/#/c/18279 | 11:38 |
| sshnaidm | ssbarnea|rover, rover actually should investigate it and ping people that can solve it if he can't | 11:39 |
| quiquell | sshnaidm, panda: Everything has being rebased | 11:39 |
| quiquell | depends-on and all | 11:39 |
| sshnaidm | quiquell, maybe circular deps or kind of? | 11:40 |
| sshnaidm | quiquell, or too many deps for zuul :) try to rebase it manually maybe | 11:41 |
| quiquell | Already :-/ | 11:41 |
| ssbarnea|rover | sshnaidm: this is what we are doing, martin is working on that one. | 11:42 |
| marios | panda quiquell: is timing out for me on start zuul and friends (making the key passphrase-less gets me further but still see 500 error at http://localhost:9000/api/tenant/tripleo-ci-reproducer/status with timeout after the 60 attempts http://pastebin.test.redhat.com/699180 i already reluctantly also tried setenforce 0 but same, do i have to do something in firewall maybe otherwise any ideas thanks | 11:51 |
| quiquell | marios: You are using it twoards your openstack tenant ? | 11:54 |
| quiquell | marios: Can you shere your docker-compose logs at ~/tripleo-ci-reproducer ? | 11:55 |
| marios | quiquell: nothing in [m@192 ansible-role-tripleo-ci-reproducer]$ cd ~/tripleo-ci-reproducer/logs/ | 11:55 |
| quiquell | marios: those are logs for job exuctions not for reproducer itself | 11:57 |
| marios | quiquell: and yeah i have ~/.config/openstack/clouds.yaml with the auth stuff for rdo cloud | 11:57 |
| quiquell | marios: but maybe is not a bad idea to use it as reproducer logs too | 11:57 |
| marios | quiquell: so where should i check please | 11:57 |
| *** holser_ is now known as holser|lunch | 11:58 | |
| quiquell | marios: cd ~/tripleo-ci-reproducer; docker-compose logs | 11:58 |
| quiquell | marios: It will show you the "insides" of the repro | 11:58 |
| marios | quiquell: pretty | 11:59 |
| marios | thanks checking quiquell | 11:59 |
| quiquell | marios: Let me know if you can't see anything there | 12:01 |
| marios | quiquell: well is socket error and name does not resolve like http://pastebin.test.redhat.com/699187 | 12:02 |
| quiquell | damn let me know I cannot connect to VPN... | 12:06 |
| chandankumar | marios: quiquell do you know where this var undercloud_network_cidr is used in standalone jobs? | 12:09 |
| marios | https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/undercloud-deploy/README.md#role-network-variables | 12:09 |
| marios | chandankumar: | 12:09 |
| quiquell | marios: Now I se your pastebin, so the zuul web is not working right | 12:10 |
| quiquell | marios: can you do a docker-compose logs zuul-web ? | 12:10 |
| marios | quiquell: sure can trying sec | 12:10 |
| chandankumar | marios: thanks! | 12:11 |
| quiquell | marios: one more thing | 12:11 |
| marios | chandankumar: looks like its set in various places like ./config/environments/ovb-common.yml:5:undercloud_network_cidr: 192.168.24.0/24 | 12:11 |
| quiquell | marios: we can do a blue if you want | 12:11 |
| marios | ./roles/extras-common/defaults/main.yml:36:undercloud_network_cidr: 192.168.24.0/24 | 12:11 |
| marios | ./roles/collect-logs/docs/static/ovb-openstack-cloud/calling-quickstart.rst:55: undercloud_network_cidr: 192.0.2.0/24 | 12:11 |
| marios | etc chandankumar | 12:11 |
| chandankumar | marios: It might be the reason my tests were failing earlier | 12:11 |
| marios | quiquell: sure but i don't want to take up too much of your time let's see how much further i can get like this thanks for the help so far man | 12:12 |
| marios | quiquell: if i don't get anywhere today then maybe tomorrow if panda also finds it useful we can do breakfast sync | 12:12 |
| quiquell | marios: so do one thing | 12:12 |
| marios | panda is buying everyone donuts | 12:12 |
| quiquell | marios: I don't like e-donuts I like r-donuts | 12:12 |
| marios | quiquell: [m@192 tripleo-ci-reproducer]$ docker-compose logs zuul-web | 12:13 |
| marios | ^[[3~WARNING: The no_proxy variable is not set. Defaulting to a blank string. | 12:13 |
| marios | ERROR: No such service: zuul-web | 12:13 |
| quiquell | marios: Maybe it's related to your ssh keys or the like | 12:13 |
| quiquell | marios: try a docker-compose down -v to shut this down | 12:13 |
| quiquell | marios: and then docker-compose up zuul-web | 12:13 |
| quiquell | marios: And see if it start ups | 12:13 |
| marios | quiquell: yah i mean on friday it was failing like startup zuul & friends like http://pastebin.test.redhat.com/698418 | 12:13 |
| marios | quiquell: and i removed the passphrase today from the key and it gets further but fails here now | 12:14 |
| quiquell | marios: just "web" without zuul- sorry | 12:14 |
| quiquell | marios: you are doing good progress | 12:14 |
| ssbarnea|rover | sshnaidm: how about a meeting today after scrum about molecule? or should i leave 30min gap? | 12:14 |
| marios | quiquell: its same error the name does not resolve etc | 12:14 |
| *** panda is now known as panda|lunch | 12:14 | |
| sshnaidm | ssbarnea|rover, I'm fine | 12:14 |
| marios | quiquell: ok thanks lemme poke see if i can do more | 12:14 |
| quiquell | marios: humm try just the scheduler it's like the main service | 12:15 |
| quiquell | marios: docker-compose down -v && docker-compose up scheduler | 12:15 |
| marios | quiquell: ah invalid key | 12:15 |
| quiquell | marios: yep usually is tha t | 12:15 |
| marios | scheduler_1 | raise SSHException("Invalid key") | 12:15 |
| * marios hugs quiquell | 12:15 | |
| marios | thanks | 12:15 |
| quiquell | marios: By default it looks at ~/.ssh/id_rsa | 12:15 |
| marios | but does that mean my key isn't in the image? | 12:15 |
| marios | yeah that is my default key | 12:15 |
| marios | :/ | 12:15 |
| quiquell | marios: Let's do one thing you can override that | 12:16 |
| ssbarnea|rover | sshnaidm: sure, I will send and invite to the team. mean while anyone is welcomed to play with the patch bit more, as it would clearly help. | 12:16 |
| quiquell | marios: Maybe it's and old insecure key or the like and zuul complains | 12:16 |
| marios | quiquell: ack | 12:16 |
| marios | quiquell: will poke | 12:16 |
| marios | thanks | 12:16 |
| marios | ! | 12:16 |
| quiquell | marios: you can try this | 12:16 |
| quiquell | marios: just do a docker-compose up gerrit | 12:17 |
| quiquell | marios: then you have to be able to do | 12:17 |
| quiquell | ssh -p 29418 gerrit query | 12:18 |
| quiquell | for example | 12:18 |
| marios | quiquell: trying the compose down && up scheduler you posted above first (is running) | 12:18 |
| marios | then will try the gerrrit thanks quiquell | 12:18 |
| marios | quiquell: ok cool the scheduler fails as expected ont he ssh key | 12:19 |
| quiquell | marios: just do the docker-compose up gerrit and try to ssh it | 12:19 |
| marios | SSH-2.0-GerritCodeReview_2.16 (SSHD-CORE-2.0.0) | 12:20 |
| marios | Unsupported protocol version: GET / HTTP/1.1 | 12:20 |
| marios | :) | 12:20 |
| marios | quiquell: ^ | 12:20 |
| marios | quiquell: so its there and responds | 12:20 |
| quiquell | marios: humm... | 12:21 |
| quiquell | marios: then have to be upstream or RDO gerrit | 12:21 |
| quiquell | marios: do you have same key for both ? | 12:22 |
| quiquell | marios: try to do the same but for upstream | 12:22 |
| quiquell | and RDO | 12:22 |
| quiquell | with proper user | 12:22 |
| marios | quiquell: yah same key | 12:22 |
| quiquell | marios: Can I see your playbook ? | 12:24 |
| quiquell | marios: wes has kind of same issue even with a new key | 12:25 |
| marios | http://pastebin.test.redhat.com/699201 quiquell | 12:25 |
| *** udesale has joined #oooq | 12:26 | |
| quiquell | try | 12:27 |
| quiquell | ssh -p 29418 marios@review.rdoproject.org gerrit --help | 12:28 |
| quiquell | and | 12:28 |
| quiquell | ssh -p 29418 quiquell@review.rdoproject.org gerrit --help | 12:28 |
| marios | quiquell: nice yeah first one works second fails cos quiquell not marios | 12:28 |
| quiquell | jaja | 12:29 |
| quiquell | well... s/quiquell/marios/ | 12:29 |
| marios | quiquell: did you mean to say upstream gerrit | 12:29 |
| quiquell | ahh yep | 12:29 |
| quiquell | marios@review.openstack.org | 12:29 |
| marios | quiquell: yeah sorry my bad works also | 12:29 |
| marios | cool | 12:29 |
| quiquell | ok so you can ssh the three gerrits | 12:29 |
| quiquell | local/upstream/rdo | 12:29 |
| marios | looks like | 12:30 |
| quiquell | Next step is to do the same but within the scheduler docker image | 12:30 |
| quiquell | marios: but first Can I see the logs from scheduler failing ? | 12:30 |
| marios | docker-compose logs scheduler http://pastebin.test.redhat.com/699196 | 12:31 |
| marios | quiquell: ^ | 12:31 |
| quiquell | marios: I think it's related to passphrase | 12:34 |
| quiquell | https://github.com/paramiko/paramiko/issues/1305 | 12:34 |
| quiquell | Can you generate another pair of keys without passphrase ? | 12:35 |
| quiquell | and use them ? | 12:35 |
| quiquell | We will have to update images too :-( | 12:35 |
| quiquell | But at least scheduler is going to go up | 12:35 |
| marios | quiquell: looking (and thanks ) | 12:37 |
| quiquell | marios: have to be that | 12:37 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, (2 more messages) | 12:37 |
| marios | quiquell: like it used to have one now doesnt? | 12:37 |
| quiquell | marios: there is a pair of vars to override private key path | 12:37 |
| quiquell | marios: at the repro | 12:37 |
| quiquell | marios: what do you mean ? | 12:37 |
| *** ratailor has quit IRC | 12:38 | |
| marios | quiquell: i mean the key i use used to have a passphrase and now it doesn't | 12:41 |
| marios | quiquell:is that the problem | 12:41 |
| marios | quiquell: so we'll try agai with a completely new one without passphrase to begin with and that costs new images | 12:41 |
| quiquell | marios: how did you doo the no-password thing ? | 12:42 |
| quiquell | marios: Maybe we have to pass that to zuul somehow | 12:43 |
| quiquell | marios: looks like preloading the key wigh ssh-agent and making paramiko aware of it, so we need to work this out | 12:45 |
| quiquell | marios: Let's test first with passwordless keys | 12:45 |
| *** ratailor has joined #oooq | 12:47 | |
| marios | quiquell: no i did ssk-keygen -p and followed prompt with empty passphrse | 12:48 |
| marios | ssh-keygen -p | 12:48 |
| marios | quiquell: gonna run afk briefly and then try new keys later if needed i'll do the images tomorrow so you don't have to | 12:49 |
| marios | quiquell: i mean, if you think that will help anyway | 12:49 |
| quiquell | marios: Let's test it | 12:49 |
| quiquell | marios: A lot of people if going to complain | 12:49 |
| quiquell | marios: we have to discover those | 12:49 |
| quiquell | marios: maybe good to write down all thos things for a FAQ | 12:50 |
| ykarel | quiquell, is there some issue in the jobs in patch which i posted recheck | 12:54 |
| ykarel | when i see in zuul status page a job failed, and now i see all SUCCESS: https://review.rdoproject.org/r/#/c/18121 | 12:54 |
| marios | quiquell: ack i have notes i can do that if useful for others maybe in readme but first get it working then do that :) | 12:55 |
| quiquell | ykarel: We have to adjust thinks there, so kind of WIP | 12:56 |
| ykarel | quiquell, ack, just noticed, so shared | 12:56 |
| quiquell | marios: agree | 12:56 |
| quiquell | ykarel: thanks you rock like always | 12:56 |
| *** weshay has joined #oooq | 12:58 | |
| weshay | arxcruz|ruck, ping | 13:01 |
| *** holser|lunch is now known as holser_ | 13:04 | |
| weshay | ssbarnea|rover, panda|lunch anyone know why f28 is down? | 13:09 |
| ykarel | weshay, probably https://bugs.launchpad.net/tripleo/+bug/1812641 | 13:18 |
| openstack | Launchpad bug 1812641 in tripleo "docker-config-scripts/nova_cell_v2_discover_host.py: No such file or directory" [Undecided,In progress] - Assigned to Arx Cruz (arxcruz) | 13:18 |
| *** ratailor has quit IRC | 13:19 | |
| ssbarnea|rover | weshay: arx was also testing a possible patch but we are not sure if it could fix. | 13:20 |
| ykarel | ssbarnea|rover, do you know why that started happening recently? | 13:23 |
| ykarel | i don't see info in the bug | 13:23 |
| ykarel | atleast the patch seems to move forward,f28 is at validate-tempest | 13:24 |
| weshay | ssbarnea|rover, ykarel thanks.. arxcruz|ruck so you have a handle on http://logs.openstack.org/24/624124/4/check/tripleo-ci-fedora-28-standalone/dd3112b/logs/stackviz/#/testrepository.subunit | 13:24 |
| ykarel | weshay, ok so https://review.openstack.org/#/c/576481/ is the cause, this caused two bugs one f28 and other f035 | 13:29 |
| ykarel | both non voting jobs failure ignored | 13:29 |
| ykarel | ssbarnea|rover, ^^ | 13:29 |
| weshay | is there a revert up? | 13:29 |
| ykarel | weshay, bugs are up and people already working on | 13:30 |
| kopecmartin | chandankumar, weshay https://bluejeans.com/3988558640 .. tempest follow up | 13:30 |
| ykarel | arxcruz|ruck, and mschuppert are working | 13:30 |
| weshay | arxcruz|ruck, can you join? | 13:31 |
| arxcruz|ruck | weshay: i'm at CI escalation meeting | 13:31 |
| weshay | arxcruz|ruck, drop that | 13:31 |
| weshay | ssbarnea|rover, go ^ | 13:31 |
| arxcruz|ruck | ok | 13:31 |
| weshay | arxcruz|ruck, thanks | 13:32 |
| *** rlandy has joined #oooq | 13:32 | |
| weshay | panda|lunch, can you please check to w/ Alex if we can get the f28 mix job to be voting | 13:34 |
| *** panda|lunch is now known as panda | 13:38 | |
| panda | weshay: ok | 13:39 |
| weshay | thanks.. obviously after we get it green again | 13:39 |
| panda | slimey f28 | 13:40 |
| panda | https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master/b17b106/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-01-21_07_31_51 | 13:42 |
| panda | [dumb-init] /docker-config-scripts/nova_cell_v2_discover_host.py: No such file or directory | 13:43 |
| jaosorior | are timeouts in this job tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates a known issue? | 13:43 |
| rlandy | quiquell: can you +2 vote here - one less review to carry ... https://review.openstack.org/#/c/622575/ | 13:45 |
| quiquell | rlandy: Do I workflow it too ? | 13:46 |
| rlandy | quiquell: you can | 13:46 |
| rlandy | quiquell: also ... https://review.openstack.org/#/c/631067/16/roles/create-zuul-based-reproducer/templates/reproducer-zuul-based-quickstart.sh.j2 | 13:46 |
| rlandy | wrt your comment there, | 13:46 |
| rlandy | I'll put up a separate patch | 13:47 |
| panda | jaosorior: you may have a better chance to ask ruck and rover directly: arxcruz|ruck ssbarnea|rover ^ | 13:47 |
| rlandy | with all in playbook so we can compare | 13:47 |
| arxcruz|ruck | not my fault | 13:47 |
| arxcruz|ruck | what? | 13:47 |
| arxcruz|ruck | :D | 13:47 |
| quiquell | rlandy: with or without comments better to put the playbook at a separate jinja | 13:47 |
| rlandy | arxcruz|ruck: when you are ruck, it's always your fault | 13:47 |
| weshay | jaosorior, will look shortly, however that is a very common error for upgrades | 13:47 |
| quiquell | rlandy: Ahh that's good idea | 13:47 |
| quiquell | rlandy: nice | 13:47 |
| rlandy | quiquell:sorry - not understanding your first comment | 13:48 |
| quiquell | rlandy: btw, launc job stuff working at CI now it does a hello-reproducer job | 13:48 |
| quiquell | rlandy: so we test full loop at CI | 13:48 |
| panda | arxcruz|ruck: also, have you seen this somewhere else ? https://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master/b17b106/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-01-21_07_31_51 | 13:48 |
| rlandy | quiquell: that's good - in testing the script, we are a out of sync with running pre | 13:48 |
| * rlandy gets | 13:48 | |
| arxcruz|ruck | jaosorior: http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates | 13:48 |
| quiquell | rlandy: nah doesn't matter, good idea about two reviews with bash and wit ansible | 13:48 |
| rlandy | [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all' | 13:48 |
| arxcruz|ruck | jaosorior: doesn't seems to be often | 13:49 |
| rlandy | quiquell: what should we do about pre playbook | 13:49 |
| quiquell | rlandy: other features I am thinking of is making RDO optional | 13:49 |
| rlandy | call that directly from luancher? | 13:49 |
| weshay | jaosorior, http://zuul.openstack.org/builds?job_name=tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates | 13:49 |
| arxcruz|ruck | panda: https://bugs.launchpad.net/tripleo/+bug/1812641 | 13:49 |
| openstack | Launchpad bug 1812641 in tripleo "docker-config-scripts/nova_cell_v2_discover_host.py: No such file or directory" [Undecided,In progress] - Assigned to Arx Cruz (arxcruz) | 13:49 |
| quiquell | rlandy: I don't see a lot of falks exercising RDO jobs | 13:49 |
| arxcruz|ruck | rlandy: true :) | 13:49 |
| rlandy | quiquell: the pre playbook - to start docker etc. | 13:49 |
| rlandy | quiquell: also, what are you defining rdo jobs as? | 13:50 |
| quiquell | rlandy: I mean making connection to RDO gerrit and de base zuul jobs for RDO optional | 13:50 |
| quiquell | rlandy: with a flag or something | 13:51 |
| quiquell | rlandy: If RDO is down repro is not going to work | 13:51 |
| rlandy | quiquell: what would happen with ovb? | 13:51 |
| panda | arxcruz|ruck: great, but what's pyshim ? | 13:51 |
| quiquell | rlandy: I mean RDO activated by default | 13:51 |
| arxcruz|ruck | panda: it's a script that select which version of python to use | 13:51 |
| quiquell | rlandy: As it is, but you can deactivate it with a flag to the repro in case RDO is down so you can run upstream jobs | 13:51 |
| arxcruz|ruck | panda: https://github.com/openstack/tripleo-heat-templates/blob/master/docker_config_scripts/pyshim.sh | 13:51 |
| weshay | jaosorior, any particular reason you are calling that out? | 13:51 |
| panda | arxcruz|ruck: amazing | 13:51 |
| arxcruz|ruck | panda: since the script is not +x it fails if you don't call with pyshim.sh | 13:52 |
| quiquell | rlandy: or we can ha two tenants one for upstream one for RDO | 13:52 |
| rlandy | quiquell: woohoo - correct zuul_yaml spacing - finally ... https://logs.rdoproject.org/67/631067/16/openstack-check/tripleo-ci-centos-7-multinode-1ctlr-featureset010/1a19491/logs/reproducer-zuul-based-quickstart.sh | 13:52 |
| arxcruz|ruck | panda: not sure if your amazing was sarcastic of true... | 13:52 |
| arxcruz|ruck | hehehehehe | 13:52 |
| quiquell | rlandy: \o/ !!! | 13:53 |
| quiquell | rlandy: looks nice!!! | 13:53 |
| quiquell | rlandy: even hackeable :-) | 13:53 |
| rlandy | quiquell: ok - then we are ready for all in an ansible role | 13:53 |
| rlandy | will add that after meeting | 13:53 |
| quiquell | rlandy: sure | 13:53 |
| panda | rlandy: did you use the |indent filter ? | 13:54 |
| rlandy | panda: I tried using |4 | 13:54 |
| rlandy | didn;t work as I wanted | 13:54 |
| rlandy | there were lots of bug reports about that | 13:54 |
| rlandy | I added the spacing to the var in set_fact | 13:54 |
| panda | arxcruz|ruck: I'm never sarcastic. OR AM I ? | 13:54 |
| * rlandy went through a few iterations on this one | 13:55 | |
| arxcruz|ruck | panda: (╯°□°）╯︵ ┻━┻ | 13:55 |
| rlandy | panda: it's supposed to work with block | 13:55 |
| rlandy | maybe I'll try that in the next review | 13:55 |
| rlandy | I just got kind of frustrated with it yesterday and went with what I knew would work | 13:55 |
| *** agopi has joined #oooq | 13:56 | |
| quiquell | rlandy: Do you know if we can use roles from a repo cloned in the same playbook that is going to use the role ? | 13:58 |
| rlandy | quiquell: I don;t see why not | 13:58 |
| rlandy | as long as we get the path correct | 13:58 |
| quiquell | rlandy: ack | 13:58 |
| rlandy | quiquell: we will see shortly | 13:59 |
| rlandy | quiquell: what host do we target with this combined playbook? | 13:59 |
| quiquell | rlandy: if we clone it to roles directory under playbook path we don't need to set anything | 13:59 |
| rlandy | quiquell: it would have to include pre.yaml | 13:59 |
| arxcruz|ruck | I gonna lunch, brb | 13:59 |
| *** ykarel is now known as ykarel|away | 13:59 | |
| jaosorior | weshay: just saw it happen, then I looked at the dashboard and saw another instance in there too. | 13:59 |
| quiquell | rlandy: we can include pre no problem with that | 13:59 |
| rlandy | quiquell: so the script creates the playbook but does not run it | 13:59 |
| rlandy | quiquell: ^^? is that correct? | 14:00 |
| quiquell | rlandy: we have two options | 14:00 |
| rlandy | this is a bit meta :) | 14:00 |
| quiquell | rlandy: scripts create the playbook and runs it | 14:00 |
| quiquell | rlandy: or we just delivery a playbook that people run | 14:00 |
| rlandy | quiquell: let's discuss briefly after/in meeting | 14:00 |
| quiquell | rlandy: we alreadd have ansible as a prereq | 14:00 |
| quiquell | sure | 14:01 |
| rlandy | ok - I'd like to get done with this today' | 14:01 |
| weshay | rlandy, ssbarnea|rover mtg | 14:02 |
| marios | weshay: https://www.youtube.com/watch?v=DbWlJgQOtZo | 14:03 |
| weshay | sshnaidm, hey mtg | 14:04 |
| *** udesale has quit IRC | 14:05 | |
| *** udesale has joined #oooq | 14:06 | |
| chandankumar | marios: http://logs.openstack.org/00/627500/61/check/tripleo-ci-centos-7-standalone-os-tempest/da3993e/job-output.txt.gz#_2019-01-21_13_32_47_138788 | 14:06 |
| chandankumar | marios: https://review.openstack.org/#/c/628415/ Do I need to specifically define the value? | 14:07 |
| marios | chandankumar: in scrum will check in bit | 14:07 |
| *** ykarel|away has quit IRC | 14:08 | |
| *** vinaykns has joined #oooq | 14:11 | |
| *** skramaja has quit IRC | 14:14 | |
| marios | (for after the call thanks) o/ folks scenario standalone promotions jobs https://review.rdoproject.org/r/#/q/topic:standalone-scenario-promotion 3 reviews @ rdo-jobs for scen 1/2&3/4 and then 2 reviews at config which we won't merge i -1 them thanks | 14:25 |
| marios | weshay: sshnaidm ssbarnea|rover arxcruz|ruck panda rlandy quiquell ^^^^ | 14:25 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, (2 more messages) | 14:37 |
| marios | sshnaidm: try click the 1-1 button | 14:46 |
| marios | sshnaidm: in the bj window | 14:46 |
| marios | 1:1 i mean | 14:46 |
| sshnaidm | marios, works, thanks | 14:46 |
| marios | sshnaidm: np (you lose some of the edge but text easier to read ) | 14:48 |
| panda | marios: yeah image scaling in BJ is really bad. | 14:54 |
| ssbarnea|rover | super cool, centos:7 main container got recently broken, try running: "yum makecache fast" on docker run -i -t centos:7 /bin/bash | 14:58 |
| panda | ssbarnea|rover: open a bug, dissect it, then burn it. | 15:02 |
| ssbarnea|rover | panda: yeah, so far dns resolver is not working but I will find out more later. | 15:03 |
| panda | ssbarnea|rover: do you need a flamethrower ? | 15:08 |
| *** ykarel|away has joined #oooq | 15:22 | |
| *** ykarel|away is now known as ykarel | 15:25 | |
| rlandy | quiquell:hey - can we chat for five minutes re: generated playbook | 15:30 |
| panda | marios: what is exactly that timeline missing again ? | 15:30 |
| panda | marios: what updates is not showing ? | 15:30 |
| marios | panda: que? | 15:31 |
| marios | panda: ah | 15:32 |
| marios | panda: like if i update the task or the dod or whatever | 15:32 |
| marios | panda: or strike out some reviews | 15:32 |
| marios | panda: it only shows that you create tasks | 15:32 |
| marios | panda: or stories, or that you relate them to each other | 15:32 |
| weshay | rlandy, panda reminder to merge f28 stuff | 15:33 |
| rlandy | ack | 15:33 |
| rlandy | marios: wrt https://review.rdoproject.org/r/#/q/topic:standalone-scenario-promotion | 15:33 |
| rlandy | two jobs there in merge conflict | 15:33 |
| arxcruz|ruck | jaosorior: do you mind https://review.openstack.org/#/c/632070/ ? :D | 15:33 |
| quiquell | rlandy: sure | 15:33 |
| rlandy | marios: I can merge the job definitions - those are harmless | 15:33 |
| quiquell | rlandy: going to your blue | 15:34 |
| rlandy | quiquell: k | 15:34 |
| quiquell | there | 15:34 |
| ssbarnea|rover | panda: no flamethrower this time, seems like PEBKAC issue related to https://github.com/kubernetes-sigs/kubespray/issues/2002 | 15:47 |
| arxcruz|ruck | jaosorior: thanks! | 15:48 |
| panda | quiquell: we are supporting reproduced only from fedora28 ? | 15:49 |
| *** holser_ has quit IRC | 15:50 | |
| quiquell | panda: centos7 too | 15:57 |
| quiquell | panda: if we want more systems we put CI on it | 15:57 |
| *** quiquell is now known as quiquell|off | 15:58 | |
| *** udesale has quit IRC | 16:01 | |
| *** saneax|mtg has quit IRC | 16:02 | |
| panda | well, I see instruction only for dnf and python3 in the doc | 16:03 |
| panda | maybe I can test what needs to be done for centos7 | 16:04 |
| panda | and document it. | 16:04 |
| weshay | ssbarnea|rover, did you go through molecule today or can/should we reschedule that? | 16:09 |
| ssbarnea|rover | weshay: nope, i got only arx on call. we need to reschedule. | 16:12 |
| rlandy | ssbarnea|rover: panda: anyone else who has been working on standalone ... https://review.rdoproject.org/r/#/q/topic:standalone-scenario-promotion | 16:30 |
| rlandy | the job definitions are harmless to merge | 16:30 |
| rlandy | I can do so - but I haven't been working in this area | 16:30 |
| rlandy | please +1 the reviews if you agree and I'll merge | 16:31 |
| rlandy | only the rdo-jobs ones | 16:31 |
| ssbarnea|rover | rlandy: did them, but i cannot help with more than one +1. | 16:34 |
| rlandy | ssbarnea|rover: that's fine - just wanted more approvals from the people who worked in this area | 16:35 |
| ssbarnea|rover | arxcruz|ruck: weshay: reminder regarding my PTO: this Friday I will be in only for the first part of the day and next Monday fully out. | 16:36 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, (2 more messages) | 16:37 |
| weshay | ssbarnea|rover, k.. thanks | 16:38 |
| rlandy | ssbarnea|rover: in the end I had to -1 anyways https://review.rdoproject.org/r/#/c/18079/10/zuul.d/standalone-jobs.yaml | 16:38 |
| *** gkadam has quit IRC | 16:45 | |
| rlandy | panda: pls ping me when you have a moment to look at reviews to merge | 16:48 |
| weshay | arxcruz++ | 16:49 |
| weshay | ssbarnea|rover, arxcruz|ruck well done guys re: https://review.openstack.org/#/c/632070/2 https://bugs.launchpad.net/tripleo/+bug/1812641 | 16:50 |
| openstack | Launchpad bug 1812641 in tripleo "docker-config-scripts/nova_cell_v2_discover_host.py: No such file or directory" [Undecided,In progress] - Assigned to Arx Cruz (arxcruz) | 16:50 |
| weshay | that's how it's done :) | 16:50 |
| ssbarnea|rover | weshay: question is how that it did slip in? i seen other calls using the shims. or it was just missed when adding the shims? | 16:53 |
| weshay | shims? | 16:54 |
| ssbarnea|rover | pyshim.sh to be correct | 16:54 |
| weshay | ah ya.. hrm... ssbarnea|rover go check the status of the f28 job | 16:55 |
| weshay | then tell me how it slipped through | 16:55 |
| ssbarnea|rover | nv? | 16:55 |
| *** bogdando has quit IRC | 16:55 | |
| weshay | ssbarnea|rover, ya.. nv and failing | 16:55 |
| weshay | panda, did you get in touch w/ alex? | 16:56 |
| rlandy | removing my -1 from f28 stuff | 16:59 |
| panda | weshay: not yet, the job is not green but it's a know failure. | 16:59 |
| panda | rlandy: we can star now, I have a bunch of things to review | 16:59 |
| panda | weshay: can this be discussion for TripleO meeting tomorrow ? | 17:00 |
| rlandy | panda: it's juts a matter of pushing those reviews or not | 17:00 |
| panda | rlandy: I'll do it after we merge the pipeline | 17:00 |
| panda | ok | 17:00 |
| rlandy | panda: I can do it - juts wanted to check with you before | 17:00 |
| panda | rlandy: ok, let's see the list | 17:01 |
| rlandy | ones needs rebase | 17:01 |
| weshay | panda yes please.. let's add it to the agenda | 17:01 |
| panda | rlandy: I can do it | 17:01 |
| panda | weshay: ok | 17:01 |
| rlandy | https://review.rdoproject.org/r/#/c/17738/ | 17:01 |
| rlandy | panda: ^^ I can't rebase that | 17:02 |
| rlandy | unless I did it off line | 17:02 |
| panda | weshay: added | 17:03 |
| rlandy | panda: https://review.rdoproject.org/r/#/c/17991/ - can I merge this? | 17:03 |
| panda | rlandy: no, we can skip that, commented. | 17:07 |
| panda | rlandy: rebasing 17738 | 17:07 |
| rlandy | looking at 17738 | 17:10 |
| weshay | ssbarnea|rover, how do you think we should handle ansible-playbook vs ansible-playbook-3 in https://review.openstack.org/#/c/631825/3/quickstart.sh | 17:15 |
| *** dtantsur is now known as dtantsur|afk | 17:16 | |
| *** kopecmartin is now known as kopecmartin|off | 17:21 | |
| panda | rlandy: 17738 rebased, looks good, I checked the kolla.conf options he added, they look sane. | 17:21 |
| rlandy | panda: ok - let's merge this one and see what happens | 17:25 |
| rlandy | weshay: ssbarnea|rover: arxcruz|ruck: ^^ fyi - merging this | 17:26 |
| rlandy | we can revert if you see issues | 17:26 |
| ssbarnea|rover | weshay: nope. i do not find the -3 really needed. more of an ugly hack. | 17:27 |
| weshay | ssbarnea|rover, k.. open to suggestions if we need to update quickstart.sh further | 17:28 |
| ssbarnea|rover | weshay: few months ago I tried to make ansible be callable as a python module by i got stuck because ansible is made of 3 executables and it fails to reach consensus, people were not against, but.... | 17:28 |
| ssbarnea|rover | weshay: until we have a proof, is not needed. | 17:29 |
| panda | ssbarnea|rover: arxcruz|ruck rlandy I would try to fix if something is broken before reverting. So we can move forward. | 17:29 |
| ssbarnea|rover | panda: i don't get the context? re which issue? | 17:31 |
| panda | ssbarnea|rover: we are merging potentially disrupting changes for the promotion workflow | 17:34 |
| panda | ssbarnea|rover: to inject the f28 pipeline | 17:34 |
| weshay | ssbarnea|rover, arxcruz|ruck if you guys want to sync on ^ I can provide context | 17:35 |
| ssbarnea|rover | weshay: joining now | 17:37 |
| rlandy | panda: ack - agreed | 17:39 |
| weshay | sshnaidm, you around? | 17:50 |
| weshay | sshnaidm, wanted to chat about https://review.openstack.org/#/c/613672/33/zuul.d/layout.yaml when you have time.. thanks! ping me when ever | 17:51 |
| weshay | sshnaidm, probably going to raise it for discussion in next sprints planning meeting | 17:57 |
| weshay | ssbarnea|rover, ^ | 17:57 |
| weshay | https://tree.taiga.io/project/tripleo-ci-board/epic/635 | 17:57 |
| weshay | thanks all | 17:57 |
| *** derekh has quit IRC | 18:01 | |
| *** jpena is now known as jpena|off | 18:13 | |
| *** holser_ has joined #oooq | 18:25 | |
| *** saneax|mtg has joined #oooq | 18:28 | |
| *** saneax|mtg has quit IRC | 18:33 | |
| *** ccamacho has quit IRC | 18:33 | |
| *** saneax|mtg has joined #oooq | 18:33 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, (2 more messages) | 18:37 |
| *** ccamacho has joined #oooq | 18:50 | |
| *** ykarel has quit IRC | 18:53 | |
| *** saneax|mtg is now known as saneax | 18:54 | |
| -openstackstatus- NOTICE: The error causing post failures on jobs has been corrected. It is safe to recheck these jobs. | 19:16 | |
| rlandy | ssbarnea|rover: you're the syntax guy ... when it comes to ansible - inlcude_playbook or include_tasks? | 19:38 |
| rlandy | I'd like to keep the vars defined in all actions | 19:38 |
| sshnaidm | rlandy, it depends what you include - playbook or tasks | 19:48 |
| sshnaidm | weshay, we can talk if you're available | 19:48 |
| rlandy | nvm - I think it has to go into the main playbook anyways | 19:50 |
| weshay | sshnaidm, so I set ssbarnea|rover patch to WIP, created an epic, and we'll discuss it next sprint planning | 19:57 |
| weshay | sshnaidm, I was thinking we could let non-voting jobs in as an experiment.. but better to walk through it in a planning | 19:57 |
| weshay | sshnaidm, sound ok to you? | 19:57 |
| sshnaidm | weshay, sure | 19:58 |
| weshay | cool | 19:58 |
| *** jfrancoa has quit IRC | 20:02 | |
| weshay | arxcruz|ruck, ssbarnea|rover I'm going to send another master promotion through... arxcruz|ruck thanks for fixing 035 | 20:04 |
| *** saneax has quit IRC | 20:22 | |
| *** jtomasek has quit IRC | 20:32 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, (2 more messages) | 20:37 |
| weshay | rlandy, ping.. can you attend a meeting for me tomorrow.. I have to take a day of PTO to run some errands that are due | 20:37 |
| rlandy | weshay: sure - which meeting? | 20:37 |
| weshay | I can tell you what it's all about.. | 20:38 |
| weshay | have a quick sec.. only need 5min max | 20:38 |
| rlandy | sure | 20:38 |
| weshay | rlandy, https://trello.com/b/pthrrzK5/prod-chain-council | 20:39 |
| weshay | rlandy, can I give you a brief description at https://bluejeans.com/4113567798 | 20:40 |
| rlandy | what is going on in that background graphic?? | 20:40 |
| rlandy | joining | 20:40 |
| weshay | lolz | 20:40 |
| weshay | blame mburns | 20:40 |
| *** ssbarnea|bkp2 has joined #oooq | 21:16 | |
| *** ssbarnea|rover has quit IRC | 21:18 | |
| *** honza has quit IRC | 21:19 | |
| *** ccamacho has quit IRC | 21:28 | |
| *** agopi has quit IRC | 22:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, (2 more messages) | 22:37 |
| *** holser_ has quit IRC | 22:45 | |
| *** agopi has joined #oooq | 22:48 | |
| *** vinaykns has quit IRC | 22:48 | |
| *** agopi has quit IRC | 22:53 | |
| *** agopi has joined #oooq | 22:53 | |
| *** dsneddon has quit IRC | 23:21 | |
| *** dsneddon has joined #oooq | 23:22 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!