| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci- (2 more messages) | 00:20 |
|---|---|---|
| *** tosky has quit IRC | 00:21 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario007-multinode-oooq-container, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci- (2 more messages) | 02:20 |
| *** ykarel|away has joined #oooq | 02:45 | |
| *** ykarel|away has quit IRC | 03:05 | |
| *** d0ugal has quit IRC | 03:19 | |
| *** d0ugal has joined #oooq | 03:33 | |
| *** skramaja has joined #oooq | 03:42 | |
| *** ykarel|away has joined #oooq | 03:42 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 04:20 |
| *** ykarel|away has quit IRC | 04:44 | |
| *** udesale has joined #oooq | 04:50 | |
| *** jrist has quit IRC | 04:54 | |
| *** ykarel|away has joined #oooq | 05:00 | |
| *** jrist has joined #oooq | 05:07 | |
| *** ratailor has joined #oooq | 05:18 | |
| *** rascasoft has quit IRC | 05:48 | |
| *** rascasoft has joined #oooq | 05:55 | |
| *** rascasoft has quit IRC | 06:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 06:20 |
| *** saneax has joined #oooq | 06:21 | |
| *** rascasoft has joined #oooq | 06:26 | |
| *** ykarel|away is now known as ykarel | 06:55 | |
| *** kopecmartin has joined #oooq | 07:10 | |
| *** kopecmartin has quit IRC | 07:10 | |
| *** kopecmartin has joined #oooq | 07:14 | |
| *** quiquell|off is now known as quiquell | 07:19 | |
| *** ccamacho has joined #oooq | 07:21 | |
| quiquell | sshnaidm|afk: ping me when you are up | 07:24 |
| *** jfrancoa has joined #oooq | 07:41 | |
| *** gkadam has joined #oooq | 07:43 | |
| *** jfrancoa has quit IRC | 07:45 | |
| *** tosky has joined #oooq | 07:49 | |
| *** jfrancoa has joined #oooq | 08:02 | |
| *** brault has joined #oooq | 08:08 | |
| *** jpena|off is now known as jpena | 08:08 | |
| *** ykarel is now known as ykarel|lunch | 08:09 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 08:20 |
| *** ykarel|lunch is now known as ykarel | 08:57 | |
| arxcruz|ruck | ykarel: hey, regarding https://bugs.launchpad.net/tripleo/+bug/1811185 just saw your comment, thanks! | 08:57 |
| openstack | Launchpad bug 1811185 in tripleo "Connection refused, http://overcloud-controller-0.ctlplane.localdomain:5000/v3/projects?" [Critical,Triaged] | 08:57 |
| ykarel | ack | 08:59 |
| *** holser_ has joined #oooq | 09:15 | |
| *** derekh has joined #oooq | 09:39 | |
| *** derekh has quit IRC | 09:40 | |
| *** derekh has joined #oooq | 09:40 | |
| *** apetrich has quit IRC | 09:42 | |
| *** apetrich has joined #oooq | 09:43 | |
| ssbarnea|rover | arxcruz|ruck: hi! i am back. let me know when you want to sync. | 09:53 |
| arxcruz|ruck | ssbarnea|rover: if you don't mind to see a 4yo girl walking around me all the time, it can be now :) | 09:54 |
| *** jaosorior has quit IRC | 10:00 | |
| ssbarnea|rover | arxcruz|ruck: nopb. i was looking at the mail from wes regarding https://bugs.launchpad.net/tripleo/+bug/1811185 | 10:00 |
| openstack | Launchpad bug 1811185 in tripleo "Connection refused, http://overcloud-controller-0.ctlplane.localdomain:5000/v3/projects?" [Critical,Triaged] - Assigned to Tobias Urdin (tobias-urdin) | 10:01 |
| arxcruz|ruck | ssbarnea|rover: yeah, i'm looking into it | 10:01 |
| arxcruz|ruck | but puppet... | 10:01 |
| ssbarnea|rover | same with me | 10:01 |
| arxcruz|ruck | ssbarnea|rover: I'm pretty confident it's related to https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/keystone/keystone-container-puppet.yaml#L518 | 10:12 |
| arxcruz|ruck | I wish ykarel could enlight us :) | 10:12 |
| *** dtantsur|afk is now known as dtantsur | 10:13 | |
| ssbarnea|rover | arxcruz|ruck: i hope the same. btw, if you can help me merge this it would be really cool: https://review.openstack.org/#/c/629121/ (to use repro without unsafe ssh keys) | 10:13 |
| arxcruz|ruck | ssbarnea|rover: i'm a poor reviewer, not a core | 10:14 |
| ssbarnea|rover | arxcruz|ruck: btw, I spoke with zuul guys yesterday and I found a nice way to test reproducer but i need few hours to validate it. nice and clean (job triggered after the main one runs) | 10:15 |
| arxcruz|ruck | ssbarnea|rover: cool, using zuul api ? | 10:15 |
| ssbarnea|rover | arxcruz|ruck: yes, zuul can have child jobs and pass variables between so the child will know where to pic the repro'.sh file from and use it. | 10:16 |
| ssbarnea|rover | i only need time to play with the feature to validate it. | 10:17 |
| ssbarnea|rover | panda: can you help with the ssh key change? is only 3 lines. i can also post run logs with it. | 10:18 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 10:20 |
| panda | ssbarnea|rover: link again ? | 10:22 |
| ssbarnea|rover | panda: https://review.openstack.org/#/c/629121/5 | 10:22 |
| quiquell | ssbarnea|rover: Since we are moving to a new reproducer, maybe it will take a lot of time from you doing that | 10:27 |
| quiquell | ssbarnea|rover: we will be able to run jobs directly at CI with the new reproducer | 10:27 |
| ssbarnea|rover | quiquell: i am not sure about that, we still need to run reproducer to validate it, right? unless we run it as part of the main job. | 10:28 |
| quiquell | ssbarnea|rover: hummm thinking about it you are right | 10:28 |
| ssbarnea|rover | the idea is kinda simple: "a child job that does extra validation" what it does is irellevant. | 10:29 |
| ssbarnea|rover | and can also be a exit rnd % 2 :D | 10:29 |
| quiquell | ssbarnea|rover: we will have test for the reproducer role, and test that ensure that the reproducer script that call the role is working | 10:29 |
| quiquell | ssbarnea|rover: your stuff can be used for the latest case | 10:29 |
| quiquell | ssbarnea|rover: we don't care about reproducer implementation if the script at jobs have the same interface | 10:29 |
| quiquell | is that it ? | 10:30 |
| panda | ssbarnea|rover: dude, those comments below ... | 10:30 |
| quiquell | Nice thing the stuff about child jobs | 10:30 |
| *** sshnaidm|afk is now known as sshnaidm | 10:31 | |
| ssbarnea|rover | panda: well, overdoing it with the last one. but the become one was needed as this was a clear bug and not obvious. just wanted to be sure someone does not remove it by mistake. | 10:31 |
| panda | ssbarnea|rover: yes, but the comment *BELOW* the line ? you should be ashamed :) | 10:32 |
| panda | ssbarnea|rover: now kneel and recite the 2.6.45 kernel code backwards with me. | 10:33 |
| ssbarnea|rover | panda: already started few months back, but I got stuck at shebang parsing implementation in kernel. | 10:33 |
| panda | ssbarnea|rover: what happens if I don't use an ssh agent ? | 10:37 |
| *** bogdando has joined #oooq | 10:38 | |
| *** skramaja has quit IRC | 10:45 | |
| sshnaidm | ssbarnea|rover, marios can you please add your keys here? https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/infra-setup/tenant_vars/common.yml#L34 | 10:51 |
| sshnaidm | kopecmartin, as well ^^ | 10:52 |
| *** brault has quit IRC | 10:52 | |
| ssbarnea|rover | sshnaidm: why are we doing this? It would be much easier to get ssh keys from github as they expose pub keys there for each account. Meaning: can update them at any time without having to change our code. | 10:52 |
| ssbarnea|rover | we only need to write the list of github accoutns. | 10:52 |
| sshnaidm | ssbarnea|rover, not easier, can you give me your key? | 10:53 |
| sshnaidm | ssbarnea|rover, cat ~/.ssh/id_rsa.pub | 10:54 |
| ssbarnea|rover | sshnaidm: sure, making a CR right now to add it. | 10:54 |
| sshnaidm | ssbarnea|rover, I need it for something else too, can you paste it? | 10:55 |
| sshnaidm | ssbarnea|rover, if you are making patch please remove people that aren't in the team | 10:55 |
| ssbarnea|rover | sshnaidm: https://github.com/ssbarnea.keys | 10:56 |
| quiquell | Damn memory what was the value of "--network" to startup a sever at RDO cloud tenant ? | 10:56 |
| quiquell | with a public ip | 10:56 |
| kopecmartin | sshnaidm, https://github.com/kopecmartin.keys | 10:57 |
| kopecmartin | sshnaidm, do you want me to create a CR too? | 10:57 |
| sshnaidm | kopecmartin, maybe just add all in one CR | 10:57 |
| sshnaidm | less to rebase | 10:58 |
| quiquell | sshnaidm: btw, if we add cloud-init to zuul user to the upstream-... imges we have at RDO kforde will make them public | 10:58 |
| sshnaidm | quiquell, just thought - where we add keys then to image with cloud-init? | 10:59 |
| kopecmartin | sshnaidm, ok, ssbarnea|rover when you're making the CR, can you, please, add my key there too? https://github.com/kopecmartin.keys | 10:59 |
| ssbarnea|rover | kopecmartin: sure. sshnaidm can you please name me the lines I need to remove? | 11:00 |
| sshnaidm | quiquell, I'll test one with real job, I think infra folks complained cloud-init breaks something there.. | 11:00 |
| sshnaidm | ssbarnea|rover, everybody that is not in our team currently | 11:00 |
| quiquell | sshnaidm: nodepool has a key-name option | 11:00 |
| quiquell | sshnaidm: it does same thing as openstack create server --key-name | 11:00 |
| quiquell | sshnaidm: no key to bee added | 11:01 |
| *** jaosorior has joined #oooq | 11:01 | |
| sshnaidm | quiquell, ok, will use it | 11:01 |
| quiquell | sshnaidm: do you have the recipy to add cloud-init to zuul user with virt-customize ? | 11:02 |
| sshnaidm | quiquell, yeah | 11:02 |
| sshnaidm | quiquell, well, we'll see in a minute :D | 11:02 |
| quiquell | sshnaidm: what do you mean ? | 11:02 |
| sshnaidm | quiquell, trying it now.. | 11:03 |
| quiquell | sshnaidm: ahh ok, btw what do we have to pass to --network to create a RDO server with public key ? | 11:03 |
| sshnaidm | quiquell, not sure I understand.. | 11:03 |
| sshnaidm | quiquell, what is external network do you mean? | 11:04 |
| quiquell | sshnaidm: if I do a openstack server create at my tenant, it does not assign a public IP | 11:04 |
| sshnaidm | quiquell, well, you need to allocate floating IP and assign it | 11:04 |
| ssbarnea|rover | someone did some fishy stuff with ci-config like a forced push after changing something locally, got some error. | 11:04 |
| quiquell | sshnaidm: humm I think this was not needed | 11:05 |
| sshnaidm | quiquell, or maybe to try network "38.../24" hot what's its name.. | 11:05 |
| marios | sshnaidm: ok will post a review sec | 11:05 |
| quiquell | sshnaidm: tried that already... not working :-( | 11:05 |
| quiquell | going to ask at RDO | 11:05 |
| quiquell | thanks anyways | 11:05 |
| quiquell | ssbarnea|rover: we cannot do a push -f | 11:06 |
| sshnaidm | quiquell, then I think only floating IP | 11:06 |
| quiquell | sshnaidm: ack | 11:06 |
| sshnaidm | ssbarnea|rover, what do you mean? | 11:06 |
| marios | sshnaidm: sshnaidm https://review.rdoproject.org/r/18183 | 11:08 |
| ssbarnea|rover | sshnaidm: i got some errors with git, nevermin. maybe it was just a local problem. last time when I got this was when someone did a forced push. | 11:08 |
| ssbarnea|rover | quiquell: you are missing from ^^ can you add yourself? | 11:11 |
| ssbarnea|rover | chkumar|out: is also missing, but lucky for us, we can find his key on github. | 11:12 |
| quiquell | ssbarnea|rover: I am already there "ellorent" | 11:13 |
| ssbarnea|rover | oops, bad sight. | 11:13 |
| quiquell | maybe we can remove "adarazs" and "myoung" | 11:13 |
| ssbarnea|rover | what are these usernames? grrit ones, github ones, kerb ones? | 11:13 |
| *** udesale has quit IRC | 11:14 | |
| ssbarnea|rover | quiquell: the change above is already removing them. | 11:14 |
| panda | ssbarnea|rover: arbitrary, but kerb for simplicity | 11:14 |
| *** udesale has joined #oooq | 11:14 | |
| sshnaidm | ssbarnea|rover, it's mails | 11:15 |
| ssbarnea|rover | if anyone likes the idea I can rework the code to download the keys from github, so we only need to keep a map of kerberos->github. | 11:15 |
| quiquell | ssbarnea|rover: ack thanks | 11:15 |
| ssbarnea|rover | i think i already have the code somewhere, did it in the past (with ansible/jinja2) | 11:15 |
| panda | ssbarnea|rover: not worth it | 11:16 |
| ssbarnea|rover | rlandy is missing from the list | 11:16 |
| panda | ssbarnea|rover: and we don't want to rely on github | 11:16 |
| quiquell | ssbarnea|rover: I see rlandy | 11:16 |
| quiquell | line 47 | 11:16 |
| ssbarnea|rover | damn, new google(s) needed | 11:17 |
| ssbarnea|rover | based on feedback received, i will not waste time creating a CR for key downloading. | 11:18 |
| arxcruz|ruck | i give up, i don't know how to track down this puppet thing | 11:32 |
| sshnaidm | why do we still have puppet things | 11:37 |
| quiquell | sshnaidm: all the undercloud is isntall with puppet | 11:45 |
| *** bogdando has quit IRC | 11:45 | |
| *** brault has joined #oooq | 11:46 | |
| *** bogdando has joined #oooq | 11:48 | |
| sshnaidm | it's a bug | 11:49 |
| quiquell | Let's open a LP for it | 11:49 |
| quiquell | ssbarnea|rover: what do you feel about converting all the puppet to ansible ? | 11:50 |
| *** brault has quit IRC | 11:50 | |
| arxcruz|ruck | sshnaidm: i believe they are moving slowly to ansible | 11:53 |
| arxcruz|ruck | but I just can't find where this bullshit is calling keystone admin api instead of keystone internal api | 11:54 |
| *** brault has joined #oooq | 11:57 | |
| *** ratailor has quit IRC | 12:00 | |
| *** holser_ is now known as holser|lunch | 12:01 | |
| quiquell | sshnaidm: mate can you help me with libvirt ? | 12:08 |
| sshnaidm | quiquell, sure | 12:08 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 12:20 |
| quiquell | sshnaidm: https://docs.ansible.com/ansible/latest/modules/virt_pool_module.html | 12:34 |
| *** ykarel is now known as ykarel|away | 12:42 | |
| *** jpena is now known as jpena|lunch | 12:43 | |
| sshnaidm | ssbarnea|rover, can you please address *all* issues there? https://review.rdoproject.org/r/#/c/18182/3 | 12:45 |
| sshnaidm | quiquell, I think there is a "pool resize", not image resizing | 12:46 |
| quiquell | sshnaidm: yep | 12:49 |
| quiquell | sshnaidm: there is no resize for images as ansible module | 12:49 |
| sshnaidm | quiquell, yeah, libvirt in general is not supported in ansible as it seems | 12:50 |
| ssbarnea|rover | sshnaidm: done. | 12:51 |
| *** ykarel|away has quit IRC | 13:00 | |
| *** trown|outtypewww is now known as trown | 13:06 | |
| marios | folks quiquell: panda sshnaidm please needs reviews and if possible merge here https://review.openstack.org/#/q/topic:scen2-standalone-vote+(status:open+OR+status:merged) ( 5 easy reviews to add scen 4 back into vote and then add 2 to gates as it also votes) | 13:07 |
| marios | please thanks | 13:07 |
| quiquell | marios: will do | 13:08 |
| marios | quiquell: the error is explained in the commit message here https://review.openstack.org/629851 (instead of making 2/3 non vote, i made 3/4 :/) | 13:08 |
| marios | quiquell: but lets keep 2 as it is green and i completed the checks in https://tree.taiga.io/project/tripleo-ci-board/task/543?kanban-status=1447276 (and we fixed the containers /services there) | 13:09 |
| marios | quiquell: so the rest 4 are to add to gates thanks | 13:09 |
| quiquell | marios: going to directly workflow | 13:09 |
| quiquell | marios: It's ok or this is too bad | 13:10 |
| marios | quiquell: well no lets not do that | 13:10 |
| marios | quiquell: i mean yes urgent but not fire? | 13:10 |
| marios | quiquell: so as long as we get a ci run on it and merge today should be ok | 13:10 |
| quiquell | marios: +2 then | 13:10 |
| marios | quiquell: thanks man | 13:10 |
| marios | thanks quiquell and weshay ! | 13:12 |
| *** holser|lunch is now known as holser_ | 13:17 | |
| *** jfrancoa has quit IRC | 13:21 | |
| *** ykarel|away has joined #oooq | 13:23 | |
| weshay | ssbarnea|rover, arxcruz|ruck morning guys | 13:24 |
| weshay | ssbarnea|rover, arxcruz|ruck have you guys triaged the errors in the master https://review.rdoproject.org/zuul/status | 13:24 |
| weshay | promotion | 13:24 |
| *** derekh has quit IRC | 13:25 | |
| quiquell | weshay: Fixed the dashboard-ci + promotion-blocker https://review.rdoproject.org/r/#/c/18176/ | 13:29 |
| weshay | hoooo ray!!!! | 13:30 |
| *** ykarel|away is now known as ykarel | 13:30 | |
| quiquell | weshay: But it appears locally but I don't see it at dashboard-ci.tripleo.org | 13:30 |
| weshay | quiquell, I have my thoughts worked out for my two patches too.. just need to resubmit because the json can get dirty | 13:30 |
| quiquell | weshay: Let me know if it works for you | 13:31 |
| quiquell | weshay: ack | 13:31 |
| weshay | k.. will try | 13:31 |
| arxcruz|ruck | weshay: morning, sorry, i was looking into the failure you send the email earlier | 13:31 |
| weshay | arxcruz|ruck, there is a patch up | 13:31 |
| arxcruz|ruck | weshay: where? | 13:31 |
| weshay | arxcruz|ruck, https://review.openstack.org/#/c/629798/2/deployment/keystone/keystone-container-puppet.yaml | 13:31 |
| arxcruz|ruck | jesus... i don't understand this stuff... :( | 13:33 |
| arxcruz|ruck | weshay: i'm checking the rdo zuul now | 13:33 |
| *** saneax has quit IRC | 13:34 | |
| weshay | arxcruz|ruck, thanks... https://review.rdoproject.org/zuul/builds?pipeline=openstack-periodic | 13:35 |
| weshay | is a good view of the previous run | 13:35 |
| marios | thanks for votes panda sshnaidm | 13:36 |
| ssbarnea|rover | ... i am here, trying to keep-up. | 13:36 |
| *** rlandy has joined #oooq | 13:42 | |
| arxcruz|ruck | rlandy: ping | 13:43 |
| rlandy | arxcruz|ruck: hello | 13:43 |
| arxcruz|ruck | rlandy: https://review.openstack.org/#/c/628052 I believe enable ssl here make the featureset020 job start to fail.. | 13:43 |
| arxcruz|ruck | rlandy: http://logs.rdoproject.org/openstack-periodic/git.openstack.org/openstack-infra/tripleo-ci/master/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-rocky/20b2223/logs/undercloud/home/zuul/tempest.log.txt.gz | 13:43 |
| arxcruz|ruck | 2019-01-10 03:58:02 | ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618) | 13:44 |
| rlandy | arxcruz|ruck, https://review.openstack.org/#/c/629186/ | 13:44 |
| arxcruz|ruck | rlandy: \o/ | 13:44 |
| arxcruz|ruck | one less problem :D | 13:44 |
| arxcruz|ruck | weshay: shall I open a bug for this? | 13:44 |
| weshay | sec | 13:45 |
| *** jpena|lunch is now known as jpena | 13:45 | |
| rlandy | arxcruz|ruck: you can just recheck it and try push it through | 13:45 |
| arxcruz|ruck | rlandy: what you mean ? | 13:45 |
| rlandy | Verified | 13:45 |
| rlandy | -1 RDO Third Party CI Zuul | 13:45 |
| * rlandy rechecks | 13:46 | |
| rlandy | rebased | 13:46 |
| arxcruz|ruck | rlandy: oh... got it... | 13:50 |
| quiquell | rfolco: I will connect 10 min later to the meeting | 13:55 |
| rfolco | quiquell, ack | 13:55 |
| rfolco | sprint 4 planning meeting starts in 2 min | 13:58 |
| rfolco | marios, quiquell, sshnaidm, weshay, panda, rlandy, arxcruz, mwhahaha, rfolco, chkumar, ssbarnea, kopecmartin | 13:58 |
| rfolco | agenda https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 13:58 |
| rfolco | bj https://bluejeans.com/5878458097 | 13:58 |
| rfolco | rlandy, ping call | 14:01 |
| quiquell | rfolco: Ok I am back earlier than expected | 14:01 |
| rfolco | quiquell, on time | 14:02 |
| rfolco | starting now | 14:02 |
| rfolco | marios, mic is low to me | 14:11 |
| marios | rfolco: thanks checking | 14:12 |
| *** derekh has joined #oooq | 14:15 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 14:21 |
| weshay | rlandy, fyi https://tree.taiga.io/project/tripleo-ci-board/epic/593 | 14:26 |
| rlandy | weshay: thank you | 14:26 |
| panda | weshay: already tracked here https://tree.taiga.io/project/tripleo-ci-board/epic/592 | 14:29 |
| panda | we need to elect an epic writer ... | 14:30 |
| weshay | panda, that would be me :) | 14:30 |
| *** jfrancoa has joined #oooq | 14:31 | |
| panda | weshay: I'll archive 592 then | 14:31 |
| *** trown is now known as trown|outtypewww | 14:51 | |
| *** ykarel is now known as ykarel|away | 14:55 | |
| *** ccamacho has quit IRC | 15:45 | |
| *** ccamacho has joined #oooq | 15:46 | |
| panda | weshay: rlandy jpena https://review.rdoproject.org/api/18187 https://review.rdoproject.org/api/18186 https://review.rdoproject.org/api/1818 https://review.rdoproject.org/api/18154 | 15:47 |
| panda | fedora mixed job is currently working by mistake | 15:47 |
| rlandy | working by mistake? | 15:48 |
| quiquell | rlandy: Adapted pre.yaml from reproducer to run it a a fres centos7 machine without zuul needed that to test libvirt | 15:49 |
| quiquell | rlandy: https://review.rdoproject.org/r/#/c/18174/ | 15:49 |
| rlandy | panda: https://review.rdoproject.org/api/1818? missing a number there | 15:51 |
| panda | rlandy: sorry https://review.rdoproject.org/api/18184 | 15:52 |
| rlandy | ok | 15:52 |
| panda | rlandy: weshay https://review.rdoproject.org/api/18184 is not needed but it confused me the first time | 15:53 |
| panda | so I'd like it out of the picture, we had those scripts in two different places | 15:53 |
| *** quiquell is now known as quiquell|off | 15:54 | |
| panda | rlandy: yes, working because a combination of bugs leads to a working pipeline | 15:55 |
| panda | rlandy: even if it's not doing that it's supposed to | 15:55 |
| *** ratailor has joined #oooq | 16:05 | |
| rlandy | panda: is there any way to test this all | 16:06 |
| *** udesale has quit IRC | 16:06 | |
| rlandy | rather than letting it run on promotions? | 16:06 |
| panda | rlandy: 18184 no need to test, I don't know why they weren'r remove at the same time as moving them to config. FOr 18187 they only way I see is copy the script locally and see if tehre any sytax error. The generic behavious is to difficult to test without a full run. | 16:09 |
| *** sshnaidm has quit IRC | 16:10 | |
| rlandy | panda: I +2'ed the remove - no problem in that | 16:10 |
| panda | rlandy: for 18186 the scripts where tested individually in the changes to rdo-infra/ci-config, I remember quiquell|off giving a full run on the get-hash.sh script locally to see if it worked | 16:10 |
| rlandy | just trying to avoid another mistake | 16:11 |
| *** sshnaidm has joined #oooq | 16:11 | |
| panda | rlandy: it's config ... there are credentials involved. Frankly I think we made a lot of mistakes becuase we didn't start just merging stuff and see if it didn't work. I delayed and delayed trying to find manually what was going wrong, then a run showed more bugs than I could find in three weeks. | 16:13 |
| rlandy | panda: do you have +2 there or you want me to w+? | 16:16 |
| panda | rlandy: not 18186, found a bug ... | 16:17 |
| * panda facepalm | 16:17 | |
| panda | config blindness | 16:17 |
| panda | rlandy: ok 18186 update, I'm making a diff diff repo/rdo-sf-config/ci-scripts/tripleo-upstream/get-hash.sh repos/rdo-infra-ci-config/ci-scripts/tripleo-upstream/get-hash.sh | 16:20 |
| panda | and now it's consistent | 16:20 |
| * rlandy looks | 16:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 16:21 |
| rlandy | panda: what is this for? | 16:28 |
| rlandy | HASHES_URL=https://trunk.rdoproject.org/${DLRNAPI_DISTRO,,}${DLRNAPI_DISTRO_VERSION}-$RELEASE/$PROMOTE_NAME/commit.yam | 16:28 |
| rlandy | nvm | 16:28 |
| rlandy | I see it | 16:28 |
| panda | rlandy: the curl command | 16:28 |
| rfolco | panda, rlandy weshay: please see https://review.rdoproject.org/r/#/q/topic:f28_promotion+status:open and let me know if I am missing anything. I hope its in good shape now. | 16:31 |
| rfolco | also updated https://tree.taiga.io/project/tripleo-ci-board/us/447 | 16:31 |
| panda | rfolco: where are the baremetal nodes located ? | 16:33 |
| *** ccamacho has quit IRC | 16:33 | |
| panda | I mean rlandy | 16:34 |
| panda | physically located | 16:34 |
| *** dalvarez has joined #oooq | 16:34 | |
| rlandy | https://trunk.rdoproject.org/api-centos7-master-uc - doen't resolve | 16:34 |
| rfolco | panda, lol.... I am beating my head against wall trying to answer your question | 16:34 |
| rlandy | panda: ^^ | 16:35 |
| rlandy | correct? | 16:35 |
| panda | rlandy: checking | 16:35 |
| rlandy | rest does not error | 16:35 |
| bogdando | rlandy: | 16:36 |
| bogdando | hi | 16:36 |
| bogdando | I'm trying to reproduce the promotion job on RDO cloud | 16:36 |
| bogdando | no luck | 16:36 |
| bogdando | rlandy: I want to repro https://review.openstack.org/#/c/629408/ env, any help with that please? | 16:37 |
| bogdando | rlandy: my repro script fails at https://pastebin.com/ZVjB5TA3 :) | 16:37 |
| bogdando | and even to reach that point, I had to apply https://review.openstack.org/#/c/629859/ | 16:37 |
| * rlandy looks | 16:38 | |
| panda | rlandy: in which review are you seeing api-centos7 ? | 16:39 |
| bogdando | yeah, thanks. So it just hangs and does nothing. I guess something is wrong with that mysterious tebroker | 16:39 |
| bogdando | whatever that ./testenv-client does... | 16:39 |
| * bogdando sigh | 16:39 | |
| rlandy | bogdando: well - do you want to run a reproducer or you want to kick a test job on rdo? | 16:39 |
| weshay | bogdando, testenv-client should be gone | 16:39 |
| bogdando | yay | 16:39 |
| weshay | right | 16:39 |
| * rlandy gets fix for reproducer | 16:39 | |
| bogdando | well, I want just a live env with that patch | 16:39 |
| weshay | sshnaidm++ | 16:39 |
| hubbot1 | weshay: sshnaidm's karma is now 10 | 16:39 |
| bogdando | can't do that RDO cloud, not locally with libvirt | 16:40 |
| bogdando | complains about not using it with promo jobs | 16:40 |
| rlandy | bogdando: https://review.openstack.org/#/c/626758/ should fix the reproducer issues | 16:41 |
| bogdando | rlandy: nice thanks | 16:41 |
| bogdando | and would be nice to allow libvirt mode for jobs other than multinode and single node | 16:41 |
| rlandy | so if you add that as an additional patch in your reproducer code | 16:41 |
| bogdando | rlandy++ | 16:42 |
| hubbot1 | bogdando: rlandy's karma is now 41 | 16:42 |
| bogdando | I love that https://review.openstack.org/#/c/626758/5/toci_gate_test.sh | 16:42 |
| panda | weshay: where are our baremetal nodes located ? | 16:42 |
| bogdando | each time we kill an ugly bash a line of code a kitten is born | 16:43 |
| weshay | panda, in raleigh | 16:43 |
| panda | weshay: ok. | 16:43 |
| weshay | I think.. they may have moved after I moved | 16:43 |
| panda | weshay: well if they are in westford, this will be a difficult sprint :) | 16:43 |
| panda | weshay: nevermind, didn't read carefully. | 16:44 |
| rlandy | bogdando: we're in process of reworking the reproducer altogether ... | 16:44 |
| rlandy | panda: sorry - back to your review | 16:45 |
| rlandy | I ran the script best I could in hacked up env | 16:45 |
| weshay | arxcruz|ruck, ssbarnea|rover you guys want to chat? so I can brief on current issues? | 16:47 |
| panda | rlandy: you mean https://review.rdoproject.org/api/18186 ? | 16:47 |
| rlandy | panda: yeah | 16:47 |
| ssbarnea|rover | weshay: probably too late in the day for arxcruz|ruck, but I am here. All i know is currently listed on https://review.rdoproject.org/etherpad/p/ruckrover-sprint24 | 16:48 |
| panda | rlandy: the error is on line 19 | 16:49 |
| panda | ? | 16:49 |
| rlandy | panda; I am not sure if it's an error - the link just does not resolve for me | 16:49 |
| ssbarnea|rover | weshay: bj? | 16:50 |
| panda | rlandy: dlrnapi --url https://trunk.rdoproject.org/api-centos-master-uc repo-get works for me | 16:50 |
| weshay | ssbarnea|rover, let's give arxcruz|ruck a minute, maybe he's around | 16:50 |
| rlandy | panda: ok - then | 16:51 |
| rlandy | maybe it's my side | 16:51 |
| panda | rlandy: in a browser gives me 404 | 16:51 |
| rlandy | panda: what else do you need on those patches? | 16:51 |
| panda | rlandy: but as url for the api, works | 16:51 |
| rlandy | panda: w+? | 16:52 |
| rlandy | didn't quite get if you can | 16:52 |
| weshay | quiquell|off, the bug lists are still not working here :( http://dashboard-ci.tripleo.org/d/cEEjGFFmz/cockpit?orgId=1 | 16:52 |
| panda | rlandy: I can't | 16:52 |
| rlandy | panda: ok - here goes... | 16:54 |
| panda | rlandy: thanks. | 16:54 |
| rlandy | https://review.rdoproject.org/r/#/c/18186 | 16:54 |
| rlandy | https://review.rdoproject.org/r/#/c/18187/ | 16:55 |
| rlandy | https://review.rdoproject.org/r/#/c/18184/ | 16:55 |
| rlandy | https://review.rdoproject.org/r/#/c/18154/ | 16:55 |
| rlandy | panda: ^^ if we land up needing to revert, ping me | 16:55 |
| panda | rlandy: thanks for participating, and stay tunes for the next entertaining bugs. | 16:56 |
| rlandy | can't wait | 16:56 |
| * rlandy should really learn to drink | 16:56 | |
| panda | arxcruz|ruck: ssbarnea|rover I will monitor closely the next periodic runs, if you see some errors that you don't understand, ping me | 16:57 |
| rlandy | panda: https://review.rdoproject.org/r/#/c/18154/ - merge conflict | 17:06 |
| panda | rlandy: rebasing | 17:07 |
| ssbarnea|rover | panda: ok thanks. i will be out for ~1h. | 17:09 |
| *** dtantsur is now known as dtantsur|afk | 17:12 | |
| *** ratailor has quit IRC | 17:12 | |
| panda | rlandy: updated, the change is moving indentation level, so it was too difficult to automatically rebase the other changes. | 17:15 |
| rlandy | panda: ok - let's let ci run and we'll try merge again | 17:15 |
| arxcruz|ruck | panda: ack | 17:19 |
| *** gkadam has quit IRC | 17:21 | |
| bogdando | rlandy: now my repro fails at https://pastebin.com/mwnr3agZ | 17:29 |
| bogdando | there is no such path, wrong hash?.. | 17:30 |
| *** jpena is now known as jpena|off | 17:32 | |
| rlandy | bogdando: that hash probably doesn't exist anymore | 17:42 |
| *** ykarel|away has quit IRC | 17:43 | |
| bogdando | ;( | 17:43 |
| rlandy | unless you need to test that exact hash - just comment out the hash defining line | 17:43 |
| bogdando | will try, thanks | 17:43 |
| rlandy | https://images.rdoproject.org/master/rdo_trunk/ | 17:44 |
| rlandy | those should exist | 17:44 |
| *** panda is now known as panda|off | 17:53 | |
| *** derekh has quit IRC | 18:00 | |
| *** bogdando has quit IRC | 18:00 | |
| ssbarnea|rover | weshay: i am back, ping me if you want to sync | 18:05 |
| weshay | ssbarnea|rover, k | 18:07 |
| weshay | in my blue | 18:07 |
| ssbarnea|rover | weshay: https://review.openstack.org/#/c/629798/ | 18:09 |
| *** kopecmartin is now known as kopecmartin|off | 18:13 | |
| weshay | ssbarnea|rover, 1805186.yaml:2: message:"ImageNotFoundException: Not found image: docker://docker.io/tripleomaster/" AND | 18:14 |
| *** jrist is now known as jrist__ | 18:16 | |
| *** jrist has joined #oooq | 18:17 | |
| *** jrist__ has quit IRC | 18:18 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 18:21 |
| weshay | ssbarnea|rover, http://logs.openstack.org/98/629798/3/check/tripleo-ci-centos-7-containers-multinode/b4b3933/logs/undercloud/var/log/extra/logstash.txt.gz#_2019-01-10_16_23_32 | 18:23 |
| *** brault has quit IRC | 18:25 | |
| *** brault has joined #oooq | 18:27 | |
| *** brault has quit IRC | 18:31 | |
| weshay | ssbarnea|rover, https://docs.google.com/document/d/1i1-_L2mx8_erETAVblchLnymC6-9h7A2x02begy99UQ/edit | 18:38 |
| weshay | ssbarnea|rover, https://review.rdoproject.org/zuul/stream/edea1469f05a4a0fa01bfb62a00e848e?logfile=console.log | 18:40 |
| *** holser_ has quit IRC | 19:08 | |
| *** brault has joined #oooq | 19:10 | |
| *** rascasoft has quit IRC | 19:12 | |
| *** brault has quit IRC | 19:14 | |
| rfolco | weshay, https://review.rdoproject.org/r/#/q/topic:f28_promotion+status:open | 19:23 |
| ssbarnea|rover | weshay: solution missing ImageNotFoundException - https://review.openstack.org/#/c/629921/ thaks to clarkb -- we should all remember this logstash aspect. | 19:30 |
| *** brault has joined #oooq | 19:31 | |
| rlandy | weshay: let me know what you think of https://code.engineering.redhat.com/gerrit/#/c/160382/ - I may have been too aggressive in the clean up | 19:34 |
| rlandy | there is more we can get rid of but I thought I would leave rocky and master around for the moment in case we need pieces of that logic | 19:35 |
| *** brault has quit IRC | 19:36 | |
| weshay | rlandy, what jobs are left after that? | 19:36 |
| weshay | rocky master | 19:37 |
| weshay | I guess | 19:37 |
| rlandy | weshay: yeah - but this is unrelated to the jobs I run | 19:37 |
| rlandy | those were kicked and defined separately | 19:37 |
| rlandy | these jobs were related to the old pipelines | 19:37 |
| rlandy | and upgrades - which we no longer run | 19:37 |
| weshay | rlandy, you are deleting browbeat | 19:37 |
| rlandy | I am ... looking | 19:38 |
| rlandy | only those jobs that run in the promotion pipelines | 19:38 |
| rlandy | http://git.app.eng.bos.redhat.com/git/tripleo-environments.git/tree/jenkins/jobs/tripleo-quickstart/tripleo-quickstart-browbeat.yml | 19:39 |
| rlandy | untouched | 19:39 |
| rlandy | I will add agopi to the review | 19:39 |
| weshay | k.. cool | 19:39 |
| rlandy | weshay: I am not removing job definitions | 19:39 |
| weshay | k | 19:39 |
| rlandy | if it's too aggressive, I can do less | 19:39 |
| rlandy | weshay: is there a pidone rep I should add to the review? | 19:40 |
| weshay | that's a really good question... I haven't heard much from those guys | 19:40 |
| rlandy | weshay:k - I will -1 it until we confirm | 19:41 |
| rlandy | ha - what happened to scale guys? | 19:41 |
| weshay | heh.. probably busy w/ 14 | 19:57 |
| weshay | I have to step out for a bit.. bbl tonight | 19:59 |
| rlandy | ok | 19:59 |
| weshay | ssbarnea|rover, I've workflowed the ovb patch :) | 19:59 |
| weshay | it passed all jobs | 19:59 |
| ssbarnea|rover | good news. | 19:59 |
| ssbarnea|rover | i am facing some kind of container questions regarding my homelab... and I am not sure which direction to pick. so far I was running my home network services as containers on synology dsm but this has an ATOM cpu. | 20:01 |
| ssbarnea|rover | i am sure that it would not scale well for the stuff I want to run there zuul/gerrit/logstash/... | 20:01 |
| *** jfrancoa has quit IRC | 20:06 | |
| rlandy | weshay: it worked - tripleo-ci-testing kick on bm | 20:07 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 20:21 |
| *** zul has quit IRC | 20:35 | |
| *** chem has quit IRC | 21:13 | |
| *** rlandy is now known as rlandy|biab | 21:25 | |
| *** holser_ has joined #oooq | 21:50 | |
| *** holser_ has quit IRC | 22:20 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001, tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-standalone-upgrade, tripleo-ci-centos-7-scenario003-standalone, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053, tripleo-ci-centos-7-containerized-undercloud-upgrades, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035 @ (2 more messages) | 22:21 |
| *** rlandy|biab is now known as rlandy | 22:36 | |
| *** jbadiapa has quit IRC | 23:10 | |
| weshay | rlandy, nice :) | 23:46 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!