| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario000-multinode-oooq- (2 more messages) | 00:03 |
|---|---|---|
| *** dsneddon has joined #oooq | 00:16 | |
| *** dsneddon has quit IRC | 00:21 | |
| *** dsneddon has joined #oooq | 00:53 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario000-multinode-oooq- (2 more messages) | 02:03 |
| *** ykarel has joined #oooq | 03:43 | |
| *** udesale has joined #oooq | 03:49 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario000-multinode-oooq- (2 more messages) | 04:03 |
| *** ykarel has quit IRC | 04:40 | |
| *** ratailor has joined #oooq | 04:48 | |
| *** chandan_kumar is now known as chandankumar | 04:55 | |
| *** ykarel has joined #oooq | 05:01 | |
| chandankumar | quiquell|off: Yes, seen ping from thursday | 05:04 |
| *** brault has quit IRC | 05:06 | |
| *** saneax has joined #oooq | 05:07 | |
| *** ykarel has quit IRC | 05:10 | |
| *** ykarel has joined #oooq | 05:12 | |
| *** agopi has quit IRC | 05:15 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario000-multinode-oooq- (2 more messages) | 06:03 |
| *** quiquell|off has quit IRC | 06:09 | |
| *** jfrancoa has joined #oooq | 06:41 | |
| *** apetrich has joined #oooq | 07:16 | |
| *** quiquell has joined #oooq | 07:24 | |
| quiquell | chandankumar: o/ | 07:24 |
| *** jtomasek has joined #oooq | 07:33 | |
| *** yolanda has joined #oooq | 07:50 | |
| chandankumar | marios_|ruck: quiquell \o/ | 07:55 |
| *** jbadiapa has joined #oooq | 07:57 | |
| *** ykarel is now known as ykarel|lunch | 07:58 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario000-multinode-oooq- (2 more messages) | 08:03 |
| *** skramaja has joined #oooq | 08:04 | |
| *** amoralej|off is now known as amoralej | 08:31 | |
| *** gkadam has joined #oooq | 08:35 | |
| *** gkadam has quit IRC | 08:35 | |
| *** tosky has joined #oooq | 08:36 | |
| *** ykarel|lunch is now known as ykarel | 08:39 | |
| *** ccamacho has quit IRC | 08:42 | |
| *** skramaja_ has joined #oooq | 08:53 | |
| *** skramaja has quit IRC | 08:53 | |
| *** agopi has joined #oooq | 09:01 | |
| *** afazekas is now known as afazekas|sick | 09:08 | |
| *** skramaja_ is now known as skramaja | 09:13 | |
| *** chem has joined #oooq | 09:26 | |
| quiquell | sshnaidm: Hello, do you have a minute ? | 09:27 |
| *** tosky has quit IRC | 09:29 | |
| *** ccamacho has joined #oooq | 09:31 | |
| *** tosky has joined #oooq | 09:33 | |
| *** ssbarnea|rover has joined #oooq | 09:47 | |
| *** derekh has joined #oooq | 09:58 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, master: tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario000-multinode-oooq- (2 more messages) | 10:03 |
| sshnaidm | quiquell, hi, yep | 10:18 |
| sshnaidm | chandankumar, arxcruz|next_yr what do you think about https://review.openstack.org/#/c/509728/ ? | 10:19 |
| chandankumar | sshnaidm: checking | 10:23 |
| chandankumar | sshnaidm: looks good, my last comment is about adding the further functionality in python-tempestconf | 10:24 |
| sshnaidm | chandankumar, can it be added later? | 10:24 |
| chandankumar | for ironic so that anyone can consumed, yes it can be added later | 10:24 |
| *** derekh has quit IRC | 10:46 | |
| *** derekh has joined #oooq | 10:47 | |
| *** dsneddon has quit IRC | 10:52 | |
| sshnaidm | quiquell, btw, zuul containers stopped to work for me. Seems like I pulled a new problematic container. Gerrit fails with java errors, can't be configured ( | 11:05 |
| sshnaidm | quiquell, need to set up our containers versions | 11:05 |
| quiquell | sshnaidm: for gerrit we can, I am working in a review for them to tag it | 11:07 |
| quiquell | sshnaidm: maybe we can tag with the sha1 of the docker hash ? | 11:07 |
| *** udesale has quit IRC | 11:10 | |
| quiquell | sshnaidm: gerrit is not zuul, it has version at your container | 11:11 |
| quiquell | sshnaidm: it's using upstream gerrit I think | 11:11 |
| ssbarnea|rover | marios_|ruck: o/ | 11:22 |
| sshnaidm | quiquell, yeah, but I didn't note version change in gerrit, weird thing | 11:26 |
| quiquell | sshnaidm: let inspect mine | 11:27 |
| quiquell | it's working | 11:27 |
| sshnaidm | quiquell, which tag do you have for gerrit? | 11:27 |
| marios_|ruck | o/ ssbarnea|rover | 11:28 |
| *** dsneddon has joined #oooq | 11:28 | |
| quiquell | sshnaidm: a6301c529fea | 11:29 |
| quiquell | sshnaidm: but it's the latest... I think issue is at zuul iamges | 11:30 |
| ssbarnea|rover | marios_|ruck: i have some bad news regarding the post failure, it just didn't work. | 11:32 |
| ssbarnea|rover | http://logs.openstack.org/81/624381/17/check/tripleo-ci-centos-7-containers-multinode-queens/2cc30b0/job-output.txt.gz | 11:33 |
| sshnaidm | quiquell, maybe.. trying now older version of gerrit image | 11:33 |
| quiquell | sshnaidm: we can pin image id at zuul too, let me pass mine | 11:34 |
| quiquell | sshnaidm: https://paste.fedoraproject.org/paste/aiDxdf894HKHjyJQ0qKpTw | 11:35 |
| quiquell | sshnaidm: If you are able to pin to that, make a review and we merge it | 11:35 |
| marios_|ruck | ssbarnea|rover: looks like https://bugs.launchpad.net/tripleo/+bug/1806699 ? http://logs.openstack.org/81/624381/17/check/tripleo-ci-centos-7-containers-multinode-queens/2cc30b0/job-output.txt.gz#_2018-12-16_19_18_28_525918 | 11:36 |
| openstack | Launchpad bug 1806699 in tripleo "HTTPConnectionPool(host='tempest-sendmail.tripleo.org', port=8080): Max retries exceeded with url" [Undecided,Triaged] | 11:36 |
| sshnaidm | quiquell, ha! previous version of gerrit worked | 11:37 |
| *** rfolco has joined #oooq | 11:37 | |
| quiquell | sshnaidm: can you try to pin all of them ? | 11:37 |
| quiquell | with image id at least | 11:37 |
| quiquell | we can investigate later on about taggin zuul and the like | 11:37 |
| ssbarnea|rover | marios_|ruck: yep the sendmail issue I reported last week, I was told that there is some spam protection in place,... clearly is buggy. | 11:37 |
| sshnaidm | quiquell, yeah, will do | 11:39 |
| sshnaidm | quiquell, need to think where to upload our containers that we'll build for reproducer with SF | 11:39 |
| sshnaidm | quiquell, maybe to create a namespace in docker.io for that | 11:39 |
| quiquell | sshnaidm: take a look here https://tree.taiga.io/project/tripleo-ci-board/task/512?kanban-status=1447274 | 11:40 |
| quiquell | sshnaidm: experiment of using sf RPMs | 11:40 |
| quiquell | sshnaidm: with our repro | 11:41 |
| quiquell | sshnaidm: Was not working, but I think it's not difficult to solve the issues | 11:41 |
| sshnaidm | quiquell, cool, will look | 11:41 |
| quiquell | sshnaidm: TL;DR baze zuul image from centos and sf RPMs | 11:42 |
| quiquell | has attached the docker-compose.yaml and the Dockerfile of the base image | 11:42 |
| *** dsneddon has quit IRC | 11:45 | |
| rfolco | panda, sshnaidm can you please review these quick ones ? https://review.rdoproject.org/r/#/c/17739 and https://review.rdoproject.org/r/#/c/17807 | 11:59 |
| rfolco | the full series: https://review.rdoproject.org/r/#/q/status:open+branch:master+topic:f28_promotion | 11:59 |
| *** dsneddon has joined #oooq | 12:01 | |
| quiquell | sshnaidm: Do you know what 's the vest way to discover if we are in a mixed fedora+centos scenario ? | 12:02 |
| quiquell | panda: ^ Â? | 12:02 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: (2 more messages) | 12:03 |
| sshnaidm | quiquell, what do you mean? | 12:04 |
| quiquell | sshnaidm: Going to bypass container prep for fedora+centos mixed scenarios, it does not make sense | 12:05 |
| quiquell | sshnaidm: to do so I ahave to check if we are in that scenario at the tqe standalone role | 12:05 |
| quiquell | sshnaidm: Like looking at docke rimage name or something | 12:05 |
| sshnaidm | quiquell, I think you can compare docker base image (centos) and /etc/redhat-release, just first thought.. | 12:06 |
| quiquell | sshnaidm: but that's inside docker image, I mean even before it pulls | 12:06 |
| panda | quiquell: I think what rlandy is doing is to set a specific docker namespace | 12:07 |
| sshnaidm | quiquell, well, it should be different place | 12:07 |
| panda | quiquell: so you could compare you current version with the docker_namespace | 12:07 |
| quiquell | panda: ack | 12:08 |
| quiquell | panda: well I can use "name_prefix" from container-prepare-parameters for now I think | 12:10 |
| panda | quiquell: uhm, another thing to consider for when we have the pure fedora container job | 12:11 |
| panda | change "name_prefix" in the containers prepare | 12:11 |
| quiquell | panda: yep | 12:11 |
| panda | rfolco: ^ | 12:11 |
| quiquell | panda, rfolco: http://codesearch.openstack.org/?q=%3A%20centos-binary-&i=nope&files=&repos= | 12:12 |
| rfolco | panda, name_prefix should be an override ? yes, I said "oooovvvveeeerrrrriiiiiddddeeeee" | 12:15 |
| panda | we'll also need to change it to centos7-binary | 12:16 |
| panda | if we're goign to maintain both c7 and c8 versions for stable branches | 12:16 |
| panda | rfolco: then we'll have to override the override | 12:16 |
| sshnaidm | rfolco, need to rebase manually https://review.rdoproject.org/r/#/c/17739 | 12:17 |
| panda | and override the fact that we overriding an overridden overridable | 12:17 |
| quiquell | panda: I was thinking on adding complexity to exercise container prep for fedora28 just for the tripleo updateing | 12:18 |
| quiquell | panda: but I think It does not worth it, at the fedora+centos thing is temporary | 12:18 |
| quiquell | panda: is that right ? | 12:19 |
| panda | quiquell: I will kill all the jobs myself in two months if we don't have the containers. | 12:19 |
| quiquell | panda: With a gun and all ? | 12:20 |
| panda | quiquell: nuke. | 12:20 |
| ssbarnea|rover | quiquell: i have some doubts about its temporary status. sooner or later we will have another fedoraXX to check, who known made another distro name.... It would be better to find a way to leverage the effort. | 12:20 |
| panda | I want the soil to burn, nothing should grow on that area for ages | 12:21 |
| quiquell | ssbarnea|rover: Let's hope kolla infra for fedora is back again when that happends so they have the images out of the box | 12:23 |
| quiquell | ssbarnea|rover: I think problem now with fedora and kolla is like Alex is resurrecting it | 12:23 |
| quiquell | :-) | 12:23 |
| quiquell | Let's bypass for now so we can merge all the f28 stuff | 12:24 |
| panda | why check another fedoraXX ? the only reason we are focusing on 28 is that it's upstream for RHEL8 | 12:24 |
| chandankumar | quiquell: can we move this card to done https://trello.com/c/o1LIqEuH/841-cixtripleo-fedora-28-python-tempestconf-adjustments-for-python3 ? | 12:28 |
| chandankumar | rfolco: Hello | 12:30 |
| quiquell | chandankumar: let's move it it's working now | 12:30 |
| rfolco | chandankumar, o/ | 12:30 |
| chandankumar | quiquell: thanks :-) | 12:30 |
| chandankumar | rfolco: I wanted to talk about these two patches https://review.openstack.org/624081 and https://review.openstack.org/624550 | 12:31 |
| rfolco | chandankumar, refreshing my mind... looking | 12:32 |
| chandankumar | rfolco: here we are removing https://review.openstack.org/#/c/624081/ test_white_regex from fs052 | 12:32 |
| ssbarnea|rover | panda: what i was trying to say is that at some point in the future, we will need to do the same kind of work for RHEL9 which would be based on FedoraXX. Hopefully that time we will be able to reuse our investment. | 12:33 |
| chandankumar | rfolco: ok | 12:34 |
| rfolco | chandankumar, my brain is in loop looking for a clean solution there | 12:36 |
| chandankumar | rfolco: like? | 12:36 |
| rfolco | chandankumar, I don't think this is going to be better than overriding both regex and whitelist. | 12:37 |
| chandankumar | rfolco: so in sceanario standalone review, we are overriding test_white_regex and tempest_whitelist, so you mean to say we donot need above two review? | 12:38 |
| *** udesale has joined #oooq | 12:39 | |
| rfolco | chandankumar, yes, because my first suggestion is technically incorrect: if whitelist is not empty, ignore regex. | 12:39 |
| chandankumar | rfolco: ok, abandoning two reviews | 12:39 |
| rfolco | chandankumar, sorry about the confusion. Thanks for looking at this. | 12:40 |
| chandankumar | rfolco: nope, it is always provides space to learn and experiment things :-) | 12:41 |
| rfolco | panda, name_prefix... should I override in the job definition for f28 promotion jobs ? | 12:41 |
| *** ratailor has quit IRC | 12:45 | |
| *** kopecmartin|off is now known as kopecmartin | 12:47 | |
| chandankumar | panda: rfolco kopecmartin please have a look at this issue https://tree.taiga.io/project/tripleo-ci-board/issue/521 | 12:51 |
| chandankumar | panda: rfolco we need to transfer it to proper dfg maintaining heat | 12:52 |
| *** rlandy has joined #oooq | 12:57 | |
| rlandy | sshnaidm: hi - can we chat for a few after the meeting. I was testing ovb on friday | 12:58 |
| marios | scrum? | 13:00 |
| panda | ssbarnea|rover: the point in the future is 5 years. I can't even imagin how many things will have changed at that point | 13:00 |
| rfolco | ping scrum | 13:01 |
| sshnaidm | rlandy, sure | 13:02 |
| *** amoralej is now known as amoralej|lunch | 13:12 | |
| marios | ssbarnea|rover: call | 13:30 |
| panda | chandankumar: is this only for heat or we need to involve other DFG for the others component as well ? | 13:34 |
| chandankumar | panda: nope | 13:34 |
| chandankumar | panda: just heat | 13:34 |
| chandankumar | sshnaidm: quiquell ssbarnea|rover please have a look at this patch https://review.openstack.org/622865 I need some help | 13:37 |
| quiquell | chandankumar: all jobs railing there | 13:42 |
| panda | rfolco: https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/tripleo-upstream/upload-cloud-images.sh | 13:43 |
| rfolco | panda, cool, thanks | 13:43 |
| quiquell | chandankumar: I think we are nog goint to be able to have a py3 job with devstack until we canno tchange ansible_python_interpreter at zuul | 13:44 |
| sshnaidm | rlandy, let's talk? | 13:44 |
| panda | rfolco: I can modify the script in the same way I modified the other | 13:45 |
| rlandy | sshnaidm: sure - just opening a window to my test box | 13:45 |
| panda | rfolco: to be distro-aware | 13:45 |
| rfolco | panda, L3 should be changed ? | 13:46 |
| ssbarnea|rover | rlandy: regarding reproducer, wans't it expected to self-configure the machine where it is run? like installing deps. | 13:46 |
| rlandy | ssbarnea|rover: ack | 13:46 |
| rlandy | sshnaidm: k - ready to talk when you are | 13:46 |
| rlandy | my blue? | 13:47 |
| sshnaidm | rlandy, https://bluejeans.com/u/sshnaidm/ | 13:47 |
| rfolco | panda, can't find what is the correct url | 13:48 |
| panda | rfolco: L3 should maube be change to account for dnf | 13:48 |
| rfolco | panda, images.rdoproject.org/images/centos7/trunk_rdo | 13:48 |
| panda | rfolco: the correct url for the images in images.r.o ? | 13:48 |
| rfolco | ? | 13:48 |
| rfolco | oops | 13:49 |
| rfolco | http://images.rdoproject.org/master/trunk_rdo | 13:49 |
| chandankumar | quiquell: sure, will wait for job to finish | 13:49 |
| rfolco | no, not this one either :( | 13:49 |
| quiquell | chandankumar:ack | 13:49 |
| panda | rfolco: right now we have https://images.rdoproject.org/master/rdo_trunk/ | 13:49 |
| *** quiquell is now known as quiquell|lunch | 13:49 | |
| rfolco | oh https | 13:49 |
| panda | rfolco: this should be changed to something like https://images.rdoproject.org/Fedora28/master/rdo_trunk/ | 13:49 |
| rfolco | stupid me | 13:50 |
| panda | maybe we can leave master as a symlink to CentOS7/master | 13:50 |
| rfolco | panda, can I try making this ? I know you know how to know to know the knowledge of knowing the know-how to change this | 13:50 |
| panda | rfolco: get out. | 13:51 |
| panda | rfolco: get out now. | 13:51 |
| rfolco | lol | 13:51 |
| rfolco | :( | 13:51 |
| rfolco | panda, panda panda | 13:51 |
| panda | rfolco: it's all yours | 13:51 |
| rfolco | panda, thanks, will add you to the review | 13:52 |
| panda | my home, my car. Yours. | 13:52 |
| panda | rfolco: since you thought this was easy, I'm adding some complexity | 13:57 |
| panda | 1. you need to modify also https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/tripleo-upstream/convert-upload-undercloud.sh | 13:57 |
| rlandy | panda: latest test run got as far as a repo setup failure - which may be legit ... https://logs.rdoproject.org/55/17855/1/check/periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master/c4df844/job-output.txt.gz#_2018-12-17_13_47_19_308377 | 13:58 |
| panda | rfolco: 2. you'll need to work with ronell to modify https://review.openstack.org/#/c/625034/4/config/release/tripleo-ci/Fedora-28/promotion-testing-hash-master.yml@26 and get the proper image url | 14:00 |
| panda | rlandy: progress! | 14:00 |
| panda | rlandy: https://review.openstack.org/#/c/625034/4/config/release/tripleo-ci/Fedora-28/promotion-testing-hash-master.yml@31 perhaps ? | 14:01 |
| *** skramaja has quit IRC | 14:02 | |
| panda | change yum to dnf | 14:02 |
| rlandy | quiquell|lunch: pls see #rhos-ops | 14:02 |
| rlandy | panda: ack - just setting up a meeting re: nodepool image and then will rerun | 14:03 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: (2 more messages) | 14:03 |
| ssbarnea|rover | does anyone knows if is possible to have a single change on gerrit with two depends-on one pointing to upstream gerrit and one to rdo one? | 14:04 |
| *** amoralej|lunch is now known as amoralej | 14:06 | |
| rlandy | rfolco: I am setting up a meeting re:public nodepool images | 14:09 |
| rlandy | rfolco: ca we use your bjs | 14:09 |
| rlandy | as it will be after the community call | 14:10 |
| rfolco | rlandy, yes of course | 14:10 |
| rfolco | panda, ack, adding these to the task | 14:11 |
| rfolco | ssbarnea|rover, rdo gerrit supports depends-on to upstream review.openstack.org | 14:13 |
| rfolco | ssbarnea|rover, the other way around probably doesn't | 14:14 |
| *** quiquell|lunch has quit IRC | 14:15 | |
| *** quiquell has joined #oooq | 14:15 | |
| quiquell | rlandy: ack | 14:15 |
| rlandy | quiquell: no worries - just sent you an invite for thurs | 14:16 |
| rlandy | re: public nodepool images | 14:16 |
| quiquell | yey !!! they are going to have them public ? | 14:18 |
| rlandy | quiquell: we're going to talk about how to do that | 14:19 |
| quiquell | rlandy: ok ok | 14:19 |
| rlandy | quiquell: kforde thinks it's doable but need some thought/process as the image updates daily | 14:19 |
| quiquell | ok, if there is no stopper then we are good | 14:20 |
| rfolco | amoralej, can we make https://images.rdoproject.org/master/rdo_trunk/ distro based ? like https://images.rdoproject.org/centos7/master/rdo_trunk/ symlink to https://images.rdoproject.org/master/rdo_trunk/ and create https://images.rdoproject.org/fedora28/master/rdo_trunk/ | 14:23 |
| *** jpena has joined #oooq | 14:24 | |
| amoralej | jpena, ^ | 14:24 |
| amoralej | i'm ok with it | 14:24 |
| jpena | I just joined the channel, so I'm missing context | 14:25 |
| rfolco | jpena, <rfolco> amoralej, can we make https://images.rdoproject.org/master/rdo_trunk/ distro based ? like https://images.rdoproject.org/centos7/master/rdo_trunk/ symlink to https://images.rdoproject.org/master/rdo_trunk/ and create https://images.rdoproject.org/fedora28/master/rdo_trunk/ | 14:27 |
| rfolco | keep existing default one | 14:27 |
| rfolco | add centos7 and fedora28 dirs, symlink centos7 to existing one. We'll change scripts to upload f28 images to fedora28 url | 14:28 |
| rfolco | jpena, sounds reasonable? | 14:28 |
| jpena | rfolco: sounds good to me. It'll require some small changes to our playbooks and cleanup scripts, but it's ok | 14:29 |
| jpena | while at it, do you have plans to add multiarch support? | 14:29 |
| rfolco | jpena, not me. This may be of interest for tbreeds | 14:33 |
| jpena | rfolco: I'm going to create the directories | 14:36 |
| rfolco | jpena, thank you | 14:37 |
| rlandy | panda: ha ... https://review.rdoproject.org/zuul/stream/f8f143dc92244fed9a4137764bcd3cf8?logfile=console.log .. getting there | 14:37 |
| panda | weeeee | 14:41 |
| panda | deploying the standalone ... expect fireworks | 14:41 |
| rfolco | rlandy, https://media.giphy.com/media/26ybvOkkmathUXss0/giphy.gif | 14:43 |
| quiquell | \o/ !!! | 14:43 |
| quiquell | It's going to fail ta container prep :-) | 14:44 |
| rfolco | don't screw up rlandy's moment | 14:44 |
| * quiquell goes back to his corner | 14:44 | |
| rfolco | /kick quiquell | 14:45 |
| rfolco | :) | 14:45 |
| rlandy | it was nice while it lasted | 14:45 |
| rlandy | quiquell: is that an expected failure? | 14:46 |
| rlandy | or you were just predicting? | 14:46 |
| quiquell | rlandy: yep, fixing it here https://review.openstack.org/#/c/625580 | 14:46 |
| quiquell | rlandy: is the container prep stuff for mixed fedora+centos | 14:46 |
| rlandy | quiquell: cool - will rerun with your change | 14:46 |
| quiquell | rlandy: not working give a sec, have just put in place without local testing it | 14:48 |
| rlandy | quiquell: k - ping when it's ready to do a test run | 14:48 |
| rfolco | panda, I think https://github.com/rdo-infra/ci-config/blob/master/ci-scripts/tripleo-upstream/upload-cloud-images.sh#L3 stays as is... look http://logs.openstack.org/20/619520/21/check/tripleo-ci-fedora-28-standalone/864b99c/logs/undercloud/etc/yum.repos.d/delorean.repo.txt.gz | 14:54 |
| panda | there's no /etc/dnf.repos.d ? :) | 14:58 |
| quiquell | rlandy: should work now https://review.openstack.org/625580 | 14:59 |
| rlandy | k - trying | 14:59 |
| marios | /win 8 | 15:02 |
| panda | /lnx 4 | 15:04 |
| rlandy | no dice | 15:10 |
| panda | quiquell: docker_prep_prefix is search(ansible_distribution|lower | 15:13 |
| panda | quiquell: should be "in" ? | 15:13 |
| panda | docker_prep_prefix is "centos-binary-" ? | 15:13 |
| panda | not even in | 15:14 |
| quiquell | panda: nope is "is" | 15:14 |
| quiquell | search is binary | 15:14 |
| quiquell | panda: used a pipe "|" with search but ansible yell at me with obsolet stuff | 15:15 |
| marios_|ruck | 3 | 15:15 |
| marios_|ruck | gerrit down for everyone or just me .com | 15:15 |
| panda | what's the sintax of serch ? | 15:15 |
| panda | justme.com works for me | 15:16 |
| quiquell | panda: "centos-binary-" | search("fedora") | 15:16 |
| quiquell | panda: give you a warning | 15:16 |
| panda | quiquell: is it the same as regex_search ? | 15:16 |
| panda | quiquell: what's the warning ? | 15:17 |
| panda | because I don't undertand how would it work without the | | 15:17 |
| quiquell | panda: "[DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using `result|search` use `result is search`." | 15:18 |
| * rlandy look s- was just updating other reviews | 15:21 | |
| *** saneax has quit IRC | 15:22 | |
| *** derekh has quit IRC | 15:24 | |
| ssbarnea|rover | rlandy: one side-effect of running reproducer: https://review.openstack.org/#/c/625621/ | 15:25 |
| rlandy | oh the old reproducer | 15:25 |
| rlandy | that's probably fine | 15:26 |
| *** chandankumar is now known as chkumar|out | 15:26 | |
| ssbarnea|rover | i don't really know old/new, is the one i got when I tried to run one job from the gate :D | 15:26 |
| *** ykarel is now known as ykarel|away | 15:26 | |
| panda | quiquell: I think this deployment is failing on a task in tht, not from quickstart-extras | 15:27 |
| panda | quiquell: tripleo-container-image-prepare lo | 15:27 |
| panda | quiquell: https://logs.rdoproject.org/55/17855/1/check/periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master/f8f143d/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-12-17_14_44_25 | 15:27 |
| *** derekh has joined #oooq | 15:27 | |
| quiquell | panda: this one is failing because is using python2 there | 15:28 |
| quiquell | panda: you have two options use the review tha disable container prep (this is the one I paste it here) | 15:28 |
| panda | /o\ | 15:28 |
| quiquell | panda: or use the review that use ansible-playbook-3 there https://review.openstack.org/#/c/625513/ | 15:29 |
| quiquell | panda: but executing container prep of centos7 images using fedora28 is not something that we want :-/ | 15:29 |
| ssbarnea|rover | rlandy: i tried to run it on a freshly installed f28 machine and I had to run it many times just to find more and more missing system deps. my original impression what that it would try to auto-install deps if it detects them as missing. | 15:29 |
| quiquell | panda: we can do a blue if you want so I explain | 15:30 |
| panda | quiquell: how does your patch disable container-prep ? | 15:31 |
| ssbarnea|rover | rlandy: should I continue to report issues and/or attempt to fix them or not really? | 15:32 |
| panda | quiquell: I'm blue | 15:32 |
| ssbarnea|rover | for example I had to do: yum install python-virtualenv ansible python2-openstackclient python2-heatclient python2-lxml | 15:32 |
| rlandy | ssbarnea|rover: sure - absolute;y | 15:32 |
| rlandy | until we have a replacement | 15:32 |
| quiquell | panda: updating the containre prep params yaml | 15:33 |
| ssbarnea|rover | and git, missing too (really fresh machine) | 15:33 |
| quiquell | panda: I mean "NOT" updating it | 15:33 |
| quiquell | panda: In case you don't want | 15:33 |
| ssbarnea|rover | rlandy: ok, just wanted to know I would not invest effort in something to be dropped very soon. | 15:33 |
| rlandy | ssbarnea|rover: I think unless it's a huge effort - yes | 15:34 |
| *** ykarel|away has quit IRC | 15:35 | |
| panda | quiquell: blue blue blue | 15:38 |
| quiquell | ok ok | 15:38 |
| ssbarnea|rover | rlandy: panda it is weird because I do remember we had some CR that was installing dependencies in reproducer but I don't see any code to installed in in the script. wan't this merged, was this feature removed? | 15:38 |
| quiquell | panda: I am at your blue | 15:39 |
| ssbarnea|rover | the bootstrapping code only checks for dependencies, and missing to spot "git", for example. | 15:39 |
| quiquell | panda: you there ? I have to drop soon | 15:42 |
| panda | quiquell: givve 1 minute | 15:42 |
| panda | quiquell: https://logs.rdoproject.org/55/17855/1/check/periodic-tripleo-ci-fedora-28-centos-7-containers-standalone-master/f8f143d/job-output.txt.gz#_2018-12-17_14_36_53_907652 | 15:51 |
| quiquell | panda: https://review.openstack.org/#/c/625513/ | 15:52 |
| quiquell | panda: https://review.openstack.org/#/c/624656/ | 15:52 |
| *** ykarel|away has joined #oooq | 15:52 | |
| ssbarnea|rover | does anyone remember ther resize error with reproducer? guess what, it still reproduce on clean machine, even when running as root. Proof: https://seashells.io/v/mjMHHxsz | 15:55 |
| *** udesale has quit IRC | 15:57 | |
| ssbarnea|rover | I tried to run the failing task using my root use (same user used to run reproducer), and it worked: https://github.com/openstack/tripleo-quickstart/blob/55580cc7ad2123a003f05547b30d47ee8246d135/roles/libvirt/setup/overcloud/tasks/libvirt_nodepool.yml#L12 | 16:01 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: (2 more messages) | 16:03 |
| panda | rlandy: added two more depends-on on the periodic test patch | 16:05 |
| rlandy | panda: looking | 16:05 |
| *** quiquell is now known as quiquell|off | 16:06 | |
| marios_|ruck | sshnaidm: o/ can you recheck here please when you get a chance https://review.openstack.org/#/c/620596/3/zuul.d/layout.yaml | 16:06 |
| marios_|ruck | ykarel|away: when you're next around thanks ^ | 16:07 |
| *** ykarel|away is now known as ykarel | 16:12 | |
| rlandy | panda: I updated the pipeline review .. updating the promotion criteria review | 16:14 |
| ssbarnea|rover | rlandy: the irony is that if I run reproducer as non-root user, it does suceeds resizing the image. only as root it fails. | 16:17 |
| ssbarnea|rover | rlandy: my guess is that the image resize needs to be run by the same user as the one running reproducer. so for root to have become, and for non root not to have become. | 16:18 |
| rlandy | I have hit that issue before | 16:19 |
| rlandy | I am just trying to think how I got around it | 16:19 |
| *** ccamacho has quit IRC | 16:19 | |
| *** ccamacho has joined #oooq | 16:21 | |
| rfolco | rlandy, f28 periodic should use fs99 ? | 16:34 |
| rfolco | rlandy, both pure and mix I mean | 16:34 |
| rlandy | rfolco: only mixed periodic | 16:35 |
| rlandy | pure should use 052 | 16:35 |
| rfolco | rlandy, I left a comment on https://review.openstack.org/#/c/625034/4/config/release/tripleo-ci/Fedora-28/promotion-testing-hash-master.yml@26 anyway, this will need https://review.rdoproject.org/r/17888 | 16:37 |
| marios_|ruck | ssbarnea|rover: o/ see you tomorrow me out | 16:38 |
| rlandy | wait, wait , it ran | 16:39 |
| rlandy | panda: ^^ | 16:39 |
| panda | it ran ? | 16:41 |
| panda | success ? | 16:43 |
| panda | zuul stop lying to me! | 16:43 |
| rfolco | zuul is now known as pinocchio | 16:44 |
| rfolco | zuul left the channel | 16:44 |
| panda | it didn't lie | 16:45 |
| panda | it really passed | 16:45 |
| rfolco | zuul joined the channel | 16:45 |
| panda | now we only need to merge those 10 patches | 16:45 |
| panda | and counting | 16:45 |
| tosky | "I'm not lying" and the queue increases | 16:45 |
| rlandy | panda: updated reviews ... https://review.rdoproject.org/r/#/c/17780/ | 16:45 |
| rlandy | https://review.rdoproject.org/r/#/c/17719/3/zuul.d/tripleo.yaml | 16:45 |
| panda | oh, la fata toskyna | 16:45 |
| rlandy | panda: k - where to from here, oh UA? | 16:46 |
| panda | to infinity .. and bayond! | 16:51 |
| panda | we need to merge all the dependencies, then add the pipeline modification, then the scripts | 16:52 |
| panda | piece of cacke | 16:52 |
| panda | cake | 16:52 |
| rfolco | rlandy, panda SIGSYNC | 16:52 |
| * panda SEGSEGV | 16:52 | |
| rlandy | eek | 16:52 |
| rlandy | panda: k - let's talk about the order of merging | 16:53 |
| rlandy | panda: we have two core votes, we could get quiquell|off's patches in | 16:55 |
| * rlandy checks failures there | 16:55 | |
| panda | rechecking 624656, post-run timed ou | 16:58 |
| rlandy | yeah | 17:01 |
| rlandy | f28 failures there | 17:01 |
| panda | 625513 is working, but it's a miracle :) | 17:04 |
| rlandy | well - t'is the season, right? | 17:12 |
| *** derekh has quit IRC | 17:21 | |
| *** jtomasek has quit IRC | 17:21 | |
| *** trown is now known as trown|lunch | 17:25 | |
| *** dsneddon has quit IRC | 17:26 | |
| rfolco | panda, rlandy: should I make an override for the pedigree job ? https://review.openstack.org/#/c/625034/4/config/release/tripleo-ci/Fedora-28/promotion-testing-hash-master.yml | 17:28 |
| rlandy | rfolco: no fix that review | 17:30 |
| rlandy | it is meant to be for the pure pipeline | 17:30 |
| rlandy | if we need to override for mixed, that should be in the fs | 17:30 |
| ssbarnea|rover | rlandy: panda : rfolco regarding post timeouts, i need some support to make this happen https://review.openstack.org/#/c/625573/ | 17:30 |
| rlandy | what are you changing | 17:30 |
| rfolco | rlandy, https://review.rdoproject.org/r/#/c/17888/1/ci-scripts/tripleo-upstream/upload-cloud-images.sh@13 | 17:32 |
| rfolco | panda, rlandy: and the change that enables distro dir in the image url is https://review.rdoproject.org/r/#/c/17886/ from jpena | 17:34 |
| *** ykarel is now known as ykarel|away | 17:37 | |
| panda | rfolco: ssbarnea|rover or we can also move this forward https://review.openstack.org/617617 , just sayin' | 17:45 |
| rfolco | panda, I'll stop nitpicking there, +1 | 17:46 |
| ssbarnea|rover | panda: here is the deal, i think i found the culprit of the timeouts already so we don't need so much boiler place... which will not work. I already tested. | 17:47 |
| ssbarnea|rover | panda: i already tried to do something very similar. in fact i am now almost sure this is the fix https://review.rdoproject.org/r/#/c/17732/ | 17:49 |
| ssbarnea|rover | you remember the warning from ansible about ignoring ansible.cfg file due to permissions? | 17:49 |
| ssbarnea|rover | guess what,... our ansible.cfg has this https://github.com/openstack/tripleo-quickstart/blob/457e61fb73eb55153cd4b8105c6090b9730c13be/ansible.cfg#L21 | 17:50 |
| ssbarnea|rover | ServerAliveInterval is needed to prevent ansible to get frozen... exactly what happens to us. | 17:50 |
| ssbarnea|rover | but because of wrong file permissions, our jobs are not loading the ansible.cfg file. | 17:50 |
| ssbarnea|rover | ... an working that nobody cared about. | 17:51 |
| rfolco | ssbarnea|rover, why does it happen randomly then? | 17:52 |
| ssbarnea|rover | rfolco: betcause networking, google for it and you will see that is random. | 17:53 |
| panda | also, my patch ensure taht whatever the reason is, we get at least some logs in place .. | 17:53 |
| panda | damn networking! | 17:53 |
| panda | that's it, no more networks from now on. | 17:53 |
| ssbarnea|rover | rfolco: panda read https://review.rdoproject.org/r/#/c/17732/ -- with all proposed workarounds. | 17:53 |
| ssbarnea|rover | :D | 17:54 |
| panda | just usb keys sent through FedEx | 17:54 |
| rfolco | ssbarnea|rover, fair and IMHO panda's patch make it more robust | 17:54 |
| ssbarnea|rover | rfolco: without fixing the frozen issue his patch is not going to help in any way. | 17:54 |
| ssbarnea|rover | when it happens ansible gets frozen, for good. | 17:55 |
| panda | mmhh, pabelanger comment ... | 17:55 |
| ssbarnea|rover | just to be noted: i am not against that patch, but first we need to sort the timeouts. after this we can get that patch. | 17:55 |
| panda | what patch ? we need numbers. | 17:56 |
| *** tosky has quit IRC | 17:57 | |
| ssbarnea|rover | https://review.openstack.org/#/c/620902/ -- is pointing to the two changes that are supposed to address the problem of ansible.cfg file. | 17:58 |
| ssbarnea|rover | once we no longer see warnings about being skipped we should be free of the frozen bug. | 17:58 |
| ssbarnea|rover | panda: what is curious is that the frozen part happens only and always on the same shell task. using "timeout" has zero value as it doesn't help. One alternative could be to use async. | 18:00 |
| ssbarnea|rover | but i prefer to see ansible.cfg issue sorted first, before looking for async. | 18:01 |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: (2 more messages) | 18:03 |
| rfolco | panda, rlandy I am stuck on chicken-n-egg for pure f28 jobs. If we can merge rdo-jobs definition at least, even though not having all the pieces, it does not break anything. | 18:04 |
| rfolco | https://review.rdoproject.org/r/#/q/status:open+branch:master+topic:f28_promotion | 18:04 |
| ssbarnea|rover | going offline for the next hour | 18:06 |
| rlandy | rfolco: what is defined as DISTRO_NAME? | 18:13 |
| rlandy | the answer just says pipeline | 18:14 |
| *** jpena is now known as jpena|off | 18:14 | |
| rfolco | rlandy, override var here https://review.rdoproject.org/r/#/c/17807/4/zuul.d/ovb-jobs.yaml@218 that is exported here https://review.rdoproject.org/r/#/c/17740/7/playbooks/tripleo-ci-periodic-base-upload/post.yaml@26 | 18:15 |
| rfolco | and used here https://review.rdoproject.org/r/#/c/17888/ | 18:15 |
| rfolco | panda, ^ | 18:15 |
| panda | override ? | 18:15 |
| rfolco | nah... var set in job definitio | 18:16 |
| rfolco | lol I am addicted to the word 'overrrrrideeeeeeee' | 18:16 |
| panda | rfolco: -1 | 18:17 |
| panda | rfolco: no need to pass a variable | 18:17 |
| panda | rfolco: you can autodetect | 18:17 |
| rfolco | k | 18:17 |
| panda | rfolco: did you identify the point that calls this script ? | 18:17 |
| rfolco | yes https://review.rdoproject.org/r/#/c/17740/7/playbooks/tripleo-ci-periodic-base-upload/post.yaml@26 | 18:18 |
| rfolco | panda, I think its time to create a discovery role at the beginning and set ansible var for distro... then we just use it (exporting to bash if needed) | 18:19 |
| rfolco | panda, otherwise people will start auto-detecting distro on their bash scripts | 18:19 |
| rlandy | rfolco: panda: maybe we should bj for a few and see what can be merged? | 18:20 |
| rfolco | yes yes | 18:20 |
| panda | rfolco: {{ ansible_distribution }} | 18:20 |
| rfolco | panda, this is not what I need. This returns Fedora, not fedora28 | 18:21 |
| rlandy | panda: including your reviews for the promoter | 18:21 |
| rlandy | 28 is major release | 18:21 |
| panda | rfolco: {{ansible_distribution|lower}}{{ansible_distribution_major_version }} | 18:21 |
| rlandy | correct - version | 18:22 |
| panda | ansible got us covered | 18:22 |
| panda | I'm ready to bj whenever | 18:22 |
| rfolco | panda, you are right. I left. | 18:22 |
| * rlandy as well | 18:22 | |
| rfolco | +1 | 18:22 |
| rlandy | rfolco: your bj? | 18:23 |
| panda | rfolco: you're left and I'm right ? | 18:23 |
| rfolco | my blue | 18:23 |
| rfolco | panda, :) | 18:23 |
| rfolco | panda, get used to my fool puns | 18:23 |
| panda | rfolco: I'm rubber, you're glue. | 18:24 |
| * rlandy waits on bj | 18:25 | |
| *** ykarel|away has quit IRC | 18:25 | |
| rfolco | panda, waiting 4 u | 18:25 |
| *** jfrancoa has quit IRC | 18:27 | |
| ssbarnea|rover | i am back | 18:28 |
| ssbarnea|rover | panda: regarding pabelanger comment, what can i reply? AFAIK, most of tripleo jobs do use that role. | 18:29 |
| *** amoralej is now known as amoralej|off | 18:33 | |
| panda | ssbarnea|rover: is there another way to do the same in non legacy roles ? | 18:35 |
| panda | ssbarnea|rover: otherwise reply what you replied to me | 18:35 |
| panda | ssbarnea|rover: lots of things need to be updated until we can get rid of legacy code | 18:35 |
| ssbarnea|rover | panda: i will do so, i asked because I do not know the reasons and history behind. meanwhile we need to fix old roles in order to move on. i hope that testing will prove to fix that problem. | 18:36 |
| *** trown|lunch is now known as trown | 18:41 | |
| hubbot1 | FAILING CHECK JOBS on master: tripleo-ci-fedora-28-standalone @ https://review.openstack.org/604298, stable/pike: tripleo-ci-centos-7-scenario001-multinode-oooq-container @ https://review.openstack.org/602248, master: tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch, tripleo-ci-centos-7-containers-multinode-rocky, tripleo-ci-fedora-28-standalone @ https://review.openstack.org/560445, stable/rocky: (2 more messages) | 20:03 |
| sshnaidm | ssbarnea|rover, marios_|ruck I changed some points in OVB config, please keep eye on it and tell me if you see something suspicious | 20:07 |
| *** sshnaidm is now known as sshnaidm|off | 20:07 | |
| ssbarnea|rover | sshnaidm|off: any hints around a specific area? i am going out in few minutes anyway but i will check in the morning. | 20:08 |
| sshnaidm|off | ssbarnea|rover, if it doesn't execute in patches - it's suspicious | 20:08 |
| sshnaidm|off | ssbarnea|rover, or it passes a lot - it's critically suspicious | 20:09 |
| sshnaidm|off | it means something is wrong | 20:09 |
| *** kopecmartin is now known as kopecmartin|off | 20:43 | |
| *** dsneddon has joined #oooq | 20:55 | |
| *** hubbot1 has quit IRC | 21:04 | |
| *** dmellado has quit IRC | 21:05 | |
| *** gouthamr has quit IRC | 21:06 | |
| *** gouthamr has joined #oooq | 21:17 | |
| *** yolanda has quit IRC | 21:18 | |
| *** gouthamr_ has joined #oooq | 21:30 | |
| *** gouthamr_ has quit IRC | 21:37 | |
| rlandy | yay - I am compliant | 21:38 |
| *** gouthamr_ has joined #oooq | 21:41 | |
| *** gouthamr_ has quit IRC | 21:46 | |
| *** gouthamr_ has joined #oooq | 21:51 | |
| *** gouthamr_ has quit IRC | 21:56 | |
| *** trown is now known as trown|outtypewww | 21:58 | |
| *** holser_ has joined #oooq | 22:39 | |
| *** holser_ has quit IRC | 22:52 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
