| *** honza has joined #oooq | 00:20 | |
| *** honza is now known as Guest70154 | 00:20 | |
| *** Guest70154 is now known as honza_ | 00:21 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 00:30 |
|---|---|---|
| *** agopi has joined #oooq | 01:42 | |
| *** jaganathan has joined #oooq | 02:06 | |
| *** agopi has quit IRC | 02:17 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 02:30 |
| *** jaganathan has quit IRC | 02:48 | |
| *** udesale has joined #oooq | 04:02 | |
| *** links has joined #oooq | 04:08 | |
| *** holser_ has joined #oooq | 04:11 | |
| *** ccamacho has quit IRC | 04:21 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 04:30 |
| *** ykarel has joined #oooq | 04:49 | |
| *** skramaja has joined #oooq | 04:57 | |
| *** holser_ has quit IRC | 05:04 | |
| *** ccamacho has joined #oooq | 05:25 | |
| *** quiquell|off is now known as quiquell | 05:26 | |
| *** hamzy has quit IRC | 05:31 | |
| quiquell | sshnaidm|rover: Are you there ? | 05:33 |
| *** hamzy has joined #oooq | 05:36 | |
| quiquell | sshnaidm|rover: Started the other day to add --with-ara to python tripleoclient | 05:40 |
| *** ratailor has joined #oooq | 05:57 | |
| *** gvrangan has joined #oooq | 06:15 | |
| *** agopi has joined #oooq | 06:17 | |
| *** udesale_ has joined #oooq | 06:23 | |
| *** udesale has quit IRC | 06:25 | |
| *** gkadam has joined #oooq | 06:26 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 06:30 |
| *** jfrancoa has joined #oooq | 06:36 | |
| quiquell | marios, sshnaidm|rover: sprint16 review for common.yaml vars https://review.openstack.org/#/c/582885/ | 06:37 |
| quiquell | marios, sshnaidm|rover: It's in a good state now | 06:37 |
| *** gkadam has quit IRC | 06:38 | |
| marios | ack quiquell will check in a bit | 06:39 |
| *** udesale__ has joined #oooq | 06:51 | |
| *** udesale_ has quit IRC | 06:54 | |
| *** holser_ has joined #oooq | 06:54 | |
| *** brault has joined #oooq | 06:58 | |
| *** bogdando has joined #oooq | 07:02 | |
| *** amoralej|off is now known as amoralej | 07:09 | |
| marios | sshnaidm|rover o/ can you checkout https://review.openstack.org/#/c/583547/5 when you get a sec thanks. wes and gfidente +2 v4 but i had to recheck for te multinode jobs so added small update (comments) is pretty simple change thanks | 07:16 |
| quiquell | marios: We need to update the script we are going to remove ? damn... | 07:20 |
| marios | quiquell: yah looks like it i mean it needs to be fixed for the bug | 07:21 |
| *** ykarel is now known as ykarel|lunch | 07:27 | |
| chkumar|ruck | quiquell: https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike/2f7ab96/logs/quickstart_collect_logs.txt.gz | 07:33 |
| chkumar|ruck | in the collected logs there is no undercloud folder | 07:33 |
| chkumar|ruck | from logs it is showing quickstart collect logs failed | 07:33 |
| chkumar|ruck | but I am not getting where it got failed in collect logs | 07:33 |
| chkumar|ruck | please have a look | 07:34 |
| quiquell | chkumar|ruck: Let me check | 07:34 |
| quiquell | chkumar|ruck: There is something weird here | 07:35 |
| quiquell | chkumar|ruck: https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike/2f7ab96/ara/ | 07:35 |
| quiquell | chkumar|ruck: Look at the post-logs.yaml, looks like the job was interrupted in the middle of it or the like | 07:36 |
| *** tosky has joined #oooq | 07:37 | |
| quiquell | chkumar|ruck: Nah forget about it, it's a false negative | 07:39 |
| chkumar|ruck | quiquell: nothing odd looks in post-logs.yaml | 07:39 |
| chkumar|ruck | quiquell: ok | 07:40 |
| quiquell | chkumar|ruck: Feels like the build has beign aborted at "collect-logs : Gather the logs to /tmp" | 07:40 |
| *** florianf has joined #oooq | 07:42 | |
| quiquell | chkumar|ruck: Comparing with a working one https://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset022-pike/176872e/logs/quickstart_collect_logs.txt.gz | 07:45 |
| quiquell | chkumar|ruck: collect-logs : Create rsync filter file looks quite different | 07:45 |
| quiquell | looks like overcloud-novacompute-0 and overcloud-controller-0 is missing | 07:46 |
| chkumar|ruck | quiquell: something weired has happened | 07:46 |
| quiquell | chkumar|ruck: It has not beign able to set the nodes up, maybe introspection problems | 07:47 |
| quiquell | being | 07:47 |
| quiquell | chkumar|ruck: Missing PLAY "Inventory the overcloud" | 07:50 |
| chkumar|ruck | quiquell: it got terminated in between then? | 07:50 |
| quiquell | chkumar|ruck: Even "Deploy the overcloud" PLAY has not being executed | 07:50 |
| chkumar|ruck | the job got failed [build-images : run the image build script (direct)] | 07:50 |
| jfrancoa | sshnaidm|rover: o/ do you have a moment for a doubt? | 07:50 |
| quiquell | chkumar|ruck: Damn... we don't have the logs of it | 07:51 |
| chkumar|ruck | quiquell: because undercloud does not get copied | 07:52 |
| quiquell | chkumar|ruck: That's quite broken... | 07:52 |
| quiquell | and it doesn't get copied becouse there is no overcloud nodes | 07:52 |
| *** kopecmartin has joined #oooq | 07:53 | |
| quiquell | chkumar|ruck: Don't know why it does not continue... | 07:55 |
| quiquell | chkumar|ruck: Maybe is a weird conditional somewhere | 07:55 |
| chkumar|ruck | quiquell: let's wait for another run | 07:55 |
| quiquell | chkumar|ruck: Yep... think so | 07:55 |
| *** dtantsur|afk is now known as dtantsur | 08:07 | |
| *** gkadam has joined #oooq | 08:11 | |
| *** gkadam is now known as gkadam-brb | 08:12 | |
| chkumar|ruck | brb for lunch | 08:19 |
| *** dtantsur is now known as dtantsur|bbl | 08:24 | |
| *** holser_ has quit IRC | 08:25 | |
| sshnaidm|rover | jfrancoa, hey, yeah | 08:28 |
| jfrancoa | sshnaidm|rover: it's about the collect-logs role. I added a new line to collect a directory inside undercloud's /tmp/ but I can't find it in the logs | 08:29 |
| jfrancoa | sshnaidm|rover: https://review.openstack.org/#/c/583572/1 | 08:29 |
| jfrancoa | sshnaidm|rover: is it needed to add anything else? or only logs inside /var /home or /etc can be collected? | 08:29 |
| *** ykarel|lunch is now known as ykarel | 08:30 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 08:30 |
| quiquell | sshnaidm|rover: Good morning sir, I have tryied a --with-ara a few days ago | 08:32 |
| quiquell | sshnaidm|rover: https://review.openstack.org/#/c/583537/ | 08:32 |
| quiquell | sshnaidm|rover: python-tripleoclient and tripleo-common change | 08:32 |
| sshnaidm|rover | jfrancoa, will look | 08:34 |
| sshnaidm|rover | quiquell, well, I was doing this too | 08:34 |
| jfrancoa | sshnaidm|rover: thanks a lot, whenever you have some time | 08:34 |
| quiquell | sshnaidm|rover: Didn't know, that's why I am saying, we can drop my tripleo-common part | 08:35 |
| quiquell | and keep the python-tripleoclient renaming the argument | 08:35 |
| quiquell | Read an email about it at the beginning of the week so I started to look at missin part | 08:36 |
| sshnaidm|rover | quiquell, let's sync about grafana and this task too today | 08:36 |
| quiquell | sshnaidm|rover: Absolutly, whenever you have a free brain cycle | 08:37 |
| sshnaidm|rover | quiquell, cool | 08:37 |
| quiquell | just ping me | 08:37 |
| *** tesseract has joined #oooq | 08:41 | |
| sshnaidm|rover | jfrancoa, the file you changed is used for any runs, except of tripleo-ci. For CI we need to change this file: https://github.com/openstack-infra/tripleo-ci/blob/cf6b217b2e4f15edbf08dc60f60845e3eb500abc/toci-quickstart/config/collect-logs.yml | 08:41 |
| sshnaidm|rover | jfrancoa, I know it's confusing, but in CI we have limitations about logs collection, which don't have in local runs.. | 08:42 |
| sshnaidm|rover | jfrancoa, so you can leave that patch and just do the same for CI | 08:42 |
| jfrancoa | sshnaidm|rover: oooh...It is. I'll change it there then, thanks a lot | 08:42 |
| sshnaidm|rover | marios, hi, wrt https://review.openstack.org/#/c/583547/ - I didn't get where is tripleo.sh part there? | 08:43 |
| quiquell | sshnaidm|rover: Spring16 review, ready to merge, added your comments https://review.openstack.org/#/c/582885/ | 08:44 |
| quiquell | test patch here https://review.openstack.org/#/c/583179/ | 08:44 |
| marios | sshnaidm|rover: o/ not sure what you mean, the bug fails because we have old ceph. the problem is the subnode setup is using older ceph still the review updates/fixes it | 08:47 |
| marios | sshnaidm|rover: thanks for checking it, i'll also reply on the review | 08:47 |
| marios | sshnaidm|rover: is the -1 because you want me to s/erase/remove or because you question the need for the patch? | 08:47 |
| sshnaidm|rover | marios, no, I try to understand why it's problem in tripleo.sh, it's not clear neither from bug description, not from commit message | 08:48 |
| *** holser_ has joined #oooq | 08:48 | |
| marios | sshnaidm|rover: problem is that we are still using tripleo.sh --boostrap-subnodes to setup the repos/boosttrap dependencies on subnodes | 08:48 |
| marios | sshnaidm|rover: the bug is about ceph jewel being installed, when it should be luminous | 08:49 |
| marios | sshnaidm|rover: so i updated the code in tripleo.sh that does it | 08:49 |
| *** kopecmartin has quit IRC | 08:49 | |
| *** kopecmartin has joined #oooq | 08:50 | |
| sshnaidm|rover | marios, well, the bug description is awful, no logs, not details, do you have maybe log for that failure? | 08:52 |
| marios | sshnaidm|rover: ack, thanks for the feedback :) I am updating the commmit message now | 08:52 |
| marios | sshnaidm|rover: no log | 08:52 |
| marios | sshnaidm|rover: i was pointed at the bug by weshay | 08:53 |
| sshnaidm|rover | marios, at least in which job does it happen? | 08:53 |
| sshnaidm|rover | marios, how do we know then that this patch solves it? | 08:53 |
| marios | sshnaidm|rover: i guess 'multinode' but i dont have a pointer | 08:53 |
| marios | sshnaidm|rover: well, we don't except it still solves a valid problem with outdated ceph repos (no luminous) | 08:53 |
| marios | sshnaidm|rover: so sure i can make it related-bug for now | 08:54 |
| *** udesale_ has joined #oooq | 08:54 | |
| marios | sshnaidm|rover: and agree we should find some traces | 08:54 |
| marios | sshnaidm|rover: maybe apevec has some pointer | 08:54 |
| sshnaidm|rover | marios, I think we need to have more defined problem, so we could be sure that we solve it. | 08:54 |
| marios | sshnaidm|rover: ack, i just updated and made it 'remove' rather than 'erase' | 08:56 |
| marios | and updated the commit message with more info | 08:56 |
| *** udesale__ has quit IRC | 08:57 | |
| sshnaidm|rover | marios, I updated the bug too | 08:57 |
| sshnaidm|rover | marios, I need to know that we resolve the problem and need to know where to see the result | 08:57 |
| sshnaidm|rover | marios, the bug doesn't have information to do it unfortunately | 08:58 |
| marios | sshnaidm|rover: me too :) | 08:58 |
| marios | refreshing | 08:58 |
| marios | sshnaidm|rover: i updated the commit message to say 'related-bug' | 08:59 |
| marios | sshnaidm|rover: please take another look when you have some time thanks | 08:59 |
| marios | sshnaidm|rover: imo this change is needed anyway, and ceph folks (gfidente) agree | 08:59 |
| marios | sshnaidm|rover: thanks for your review | 09:00 |
| quiquell | chkumar|ruck, sshnaidm|rover: Do we have any issue with reproducer regarding "ansible_private_key_file" undefined ? | 09:01 |
| sshnaidm|rover | marios, I'm totally fine with review, but I can't know what this review does. Until we have info where should we check... | 09:01 |
| marios | sshnaidm|rover: well you can | 09:01 |
| marios | sshnaidm|rover: i mean | 09:01 |
| marios | sshnaidm|rover: take any multinode job and check on the subnodes | 09:02 |
| marios | sshnaidm|rover: do we capture repos i think so right? | 09:02 |
| marios | /etc/yum.repos.d | 09:02 |
| marios | and maybe yum log for the packages | 09:02 |
| marios | sshnaidm|rover: ^%% to see if the ceph is removed? | 09:02 |
| sshnaidm|rover | marios, I don't know if it solved the problem | 09:02 |
| marios | sshnaidm|rover: well the problem is the current release for ceph is set to jewel | 09:03 |
| marios | sshnaidm|rover: but it should definitely be luminous | 09:03 |
| sshnaidm|rover | marios, ok, but all jobs work now | 09:03 |
| sshnaidm|rover | marios, so it doesn't matter actually, we overwrite it | 09:04 |
| sshnaidm|rover | marios, when we run repo-setup role | 09:05 |
| marios | sshnaidm|rover: well looking at logs we still have jewel repos setup | 09:06 |
| marios | sshnaidm|rover: like e.g. here http://logs.openstack.org/85/582385/8/check/tripleo-ci-centos-7-3nodes-multinode/ee445c4/logs/subnode-2/etc/yum.repos.d/ (some other review) vs http://logs.openstack.org/47/583547/4/check/tripleo-ci-centos-7-containers-multinode/3fdd70e/logs/subnode-2/etc/yum.repos.d/ v4 on my review | 09:06 |
| *** panda|off is now known as panda | 09:08 | |
| sshnaidm|rover | marios, but it's repos that we have from infra, and all of them should be disabled: http://logs.openstack.org/85/582385/8/check/tripleo-ci-centos-7-3nodes-multinode/ee445c4/logs/subnode-2/etc/yum.repos.d/centos-ceph-jewel.repo.txt.gz | 09:08 |
| sshnaidm|rover | marios, they have enabled:0 | 09:09 |
| sshnaidm|rover | marios, and this is our repo what we set in repo-setup role: http://logs.openstack.org/85/582385/8/check/tripleo-ci-centos-7-3nodes-multinode/ee445c4/logs/subnode-2/etc/yum.repos.d/quickstart-centos-ceph-luminous.repo.txt.gz | 09:09 |
| sshnaidm|rover | marios, and it's enabled | 09:09 |
| marios | sshnaidm|rover: ah i see | 09:10 |
| marios | sshnaidm|rover: so then why do we override that explicitly in bootstrap-subnodes | 09:10 |
| sshnaidm|rover | marios, so you are actually right about that it happens yet before quickstart runs, but where? how? :D | 09:10 |
| marios | sshnaidm|rover: well in toci_gate_test i have a link in the commit message | 09:11 |
| *** holser_ has quit IRC | 09:11 | |
| marios | sshnaidm|rover: the bug says 'wh is centos-release-ceph-jewel' installed... and my review would also fix that | 09:12 |
| marios | sshnaidm|rover: by trying to remove all the centos-release-ceph-* | 09:13 |
| marios | sshnaidm|rover: but you're right without trace we can't confirm that bug | 09:13 |
| marios | sshnaidm|rover: now, if you think we don't need this code alltogether, then instead lets post a review to remove tripleo.sh --boostrap-subnodes from toci_gate_test.sh? | 09:14 |
| marios | sshnaidm|rover: but i think we still need it , at least there is a task this sprint to port this to ansible instead of using that tripleo.sh | 09:14 |
| sshnaidm|rover | marios, I suspect tripleo.sh is can't be blamed here | 09:14 |
| *** kopecmartin has quit IRC | 09:15 | |
| sshnaidm|rover | marios, look in the bug, you see that it happens actually in ansible run according to piece of log that weshay gave us :) | 09:15 |
| sshnaidm|rover | weshay, we do reverse engineering to your bug :D | 09:15 |
| sshnaidm|rover | marios, so it seems like it runs in pre.yml maybe if not in infra scripts at all | 09:17 |
| *** udesale__ has joined #oooq | 09:17 | |
| marios | sshnaidm|rover: whats 'it' | 09:18 |
| marios | you mean tripleo.sh? | 09:18 |
| marios | sshnaidm|rover: it runs in toci_gate_tes | 09:18 |
| marios | https://github.com/openstack-infra/tripleo-ci/blob/67bffb38c016feaed6cb730687a92b86342938d4/toci_gate_test.sh#L293 | 09:18 |
| chkumar|ruck | quiquell: nope | 09:18 |
| sshnaidm|rover | marios, no, I mean task that failed in bug | 09:18 |
| quiquell | chkumar|ruck: ack | 09:18 |
| marios | sshnaidm|rover: ah | 09:18 |
| *** kopecmartin has joined #oooq | 09:19 | |
| *** udesale_ has quit IRC | 09:20 | |
| sshnaidm|rover | marios, if I understand right links in the bug, it points to multi-node-bridge role that runs in infra, yet before our job. For example: http://logs.openstack.org/98/583198/1/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/c4a6772/job-output.txt.gz#_2018-07-17_11_09_31_355068 | 09:21 |
| quiquell | panda: Are you still in training ? | 09:24 |
| panda | quiquell: not anymore, I'm traianed | 09:26 |
| quiquell | panda: I have the reproducer not working with the sprint16 changes, but not sure if it's because of it | 09:27 |
| quiquell | panda: Can we review it with a bj session Â? | 09:27 |
| sshnaidm|rover | quiquell, I think you don't create workspace in your patch now | 09:27 |
| quiquell | sshnaidm|rover: Silly me you are right... but jobs are working :-/ | 09:28 |
| sshnaidm|rover | quiquell, mm.. maybe it's created in pre.yml that we still use? | 09:29 |
| quiquell | sshnaidm|rover: I will change it to ensure that it's created, that way we can even use this at reproducer | 09:29 |
| quiquell | sshnaidm|rover: Thanks man | 09:29 |
| sshnaidm|rover | quiquell, maybe we need to start to drop things from legacy pre.yml too.. | 09:30 |
| quiquell | panda: Nah reproducer is working, I have some ansible inventory fucked up at my fedora | 09:30 |
| chkumar|ruck | sshnaidm|rover: http://logs.openstack.org/85/564285/31/check/tripleo-ci-centos-7-scenario004-multinode-oooq-container/173d26d/logs/undercloud/var/log/mistral/ceph-install-workflow.log.txt.gz#_2018-07-19_01_39_06_421 | 09:32 |
| chkumar|ruck | docker ceph pull issue known? | 09:32 |
| sshnaidm|rover | chkumar|ruck, how many times did it happen? | 09:35 |
| sshnaidm|rover | quiquell, is there advantage to use ansible_env.HOME instead of ansible_user_dir ? | 09:36 |
| sshnaidm|rover | panda, ^^ | 09:36 |
| quiquell | sshnaidm|rover: Followed advice at #zuul but sure is the same. | 09:37 |
| marios | sshnaidm|rover: please see comment on the review i'll copy/past ehere | 09:37 |
| marios | ack, well I still think this change stands on its own, i.e. we should include luminous in the ceph repo setup done in tripleo.sh --bootstrap | 09:37 |
| marios | I agree the bug is not clear, which is why i made this related-bug and asked for more info on the bug itself and we should definitely get that clarity, but, again imo this change stands. | 09:37 |
| quiquell | sshnaidm|rover: or #openstack-infra, let me find the chat | 09:37 |
| marios | Not sure why you -1 it then please clarify. If it is because "not sure it fixes the bug" then my comments here should negate that? Otherwise please clarify why you -1 so that I know what I can do to make you reconsider thanks | 09:37 |
| marios | sshnaidm|rover: ^ please thanks | 09:38 |
| marios | sshnaidm|rover: and agree on the bug lets try find out more later when apevec/weshay are about | 09:38 |
| sshnaidm|rover | marios, yeah, I think we need to hold on until weshay comes and tells us what is the problem exactly | 09:38 |
| marios | sshnaidm|rover: for the bug, yes, agree | 09:39 |
| sshnaidm|rover | marios, otherwise we'll waste a lot of time on guessing | 09:39 |
| marios | sshnaidm|rover: but what about that review? that is what my questino in | 09:39 |
| chkumar|ruck | sshnaidm|rover: recently happened in noop job on pike, earlier it passed, so asked | 09:39 |
| marios | sshnaidm|rover: i mean | 09:39 |
| quiquell | sshnaidm|rover, panda: 2018-07-16 13:20:06 tristanC you can also create a dir at "{{ ansible_env.HOME }}/workspace" on the test node to get a clean workspace | 09:39 |
| quiquell | at #zuul | 09:39 |
| marios | sshnaidm|rover: its one thing to not vote, or +1 because 'meh' or not sure it fixes etc, but to -1 means you actively disagree with the fix | 09:39 |
| marios | sshnaidm|rover: so please clarify what the disagreement is | 09:39 |
| marios | sshnaidm|rover: so i can fix it | 09:40 |
| quiquell | But I think ansible_user_dir is good too | 09:40 |
| sshnaidm|rover | marios, well, the review doesn't make sense if it doesn't solve a bug. | 09:40 |
| sshnaidm|rover | marios, this code is actually noop | 09:40 |
| marios | sshnaidm|rover: how does it not makes sense. i mean it updates the ceph release setup to include luminous | 09:40 |
| sshnaidm|rover | marios, and in this/next sprint we anyway gonna remove it | 09:40 |
| marios | sshnaidm|rover: yeah i am working on the removal | 09:40 |
| marios | sshnaidm|rover: i mean my sprin 16 task is to make the bootstra-subnodes role | 09:41 |
| sshnaidm|rover | marios, all this is overwritten in next steps, nothing from these repos are used | 09:41 |
| marios | sshnaidm|rover: then why do we have bootstrap subnodes at all ? and why are we bothering to make it ansible | 09:41 |
| quiquell | sshnaidm|rover: Going to replace it with ansible_user_dir, it was the former way | 09:41 |
| sshnaidm|rover | marios, we had it to install ovs bridge | 09:42 |
| sshnaidm|rover | marios, I think you can try just delete it, when we have bridge defined in pre.yml | 09:42 |
| sshnaidm|rover | marios, I'm not up to last changes in sprint, but I think you can try | 09:42 |
| sshnaidm|rover | marios, and all these repos were for installing ovs bridge from openstack and create a vxlan between primary and secondary node | 09:43 |
| marios | sshnaidm|rover: perhaps we should discuss some more on the sprint call this afternoon then if you really think it isn't necessary then we need to really update the task | 09:43 |
| marios | sshnaidm|rover: right now, it doesn't do ehte ovs bridge stuff | 09:43 |
| marios | sshnaidm|rover: (I thought the underclodu-setup role was doing that right?0 | 09:43 |
| marios | sshnaidm|rover: but anyway this is _subnode_ not primary | 09:43 |
| sshnaidm|rover | marios, yep, it's done in pre.yml iirc | 09:44 |
| marios | sshnaidm|rover: this code is only run on subnodes | 09:44 |
| marios | sshnaidm|rover: so anyway it doesn't do that ovs but it does do the ceph repo stuff, remove packages that i guess were conflicting and /etc/pupp/hiera and also creating /dev/loop3 devce for ceph | 09:44 |
| arxcruz | sshnaidm|rover: marios panda https://review.openstack.org/#/c/583659/ this fix scenario002 | 09:45 |
| sshnaidm|rover | marios, almost everything in this script doesn't make sense already, we don't have this puppet stuff, etc.. | 09:45 |
| sshnaidm|rover | marios, yeah, it creates loop dev for ceph, it's important | 09:46 |
| sshnaidm|rover | marios, removing epel and installing heat agents seems make sense.. | 09:46 |
| *** kopecmartin has quit IRC | 09:47 | |
| *** kopecmartin has joined #oooq | 09:48 | |
| sshnaidm|rover | marios, take a look:https://review.openstack.org/#/c/576834/7/toci_gate_test.sh all I remove from toci_gate_test.sh, could be removed from bootstrap too | 09:49 |
| sshnaidm|rover | chkumar|ruck, let's see if it happens again | 09:50 |
| quiquell | panda: Have move the playbooks here https://review.openstack.org/#/c/582466/ | 09:50 |
| quiquell | panda: Do you see any issue with that ? | 09:51 |
| sshnaidm|rover | quiquell, is asnible_user_dir var set by infra somewhere? | 09:51 |
| quiquell | sshnaidm|rover: It's a variable set by ansible, at runtime | 09:52 |
| sshnaidm|rover | chkumar|ruck, how are things going in general? | 09:53 |
| sshnaidm|rover | quiquell, oh, right | 09:53 |
| panda | quiquell: mmmhh | 09:54 |
| ykarel | chkumar|ruck, i remember it faced few days back and u even proposed a patch for it | 09:55 |
| * ykarel finding patch | 09:55 | |
| panda | quiquell: I see what you did. logic is equivalent, result is slightly different | 09:55 |
| chkumar|ruck | sshnaidm|rover: it looks good for today, one failure in periodic job telemetry test failure for pike, one fs27 failure during under in 24 periodic job | 09:55 |
| ykarel | chkumar|ruck, https://review.openstack.org/#/c/581607/ | 09:55 |
| chkumar|ruck | sshnaidm|rover: fs27 passed in periodic | 09:56 |
| panda | quiquell: if you look at the resulting toci_quickstart.sh file, it's way longer. Trying to think about advanteges and disadvantages of this | 09:56 |
| chkumar|ruck | sshnaidm|rover: for telemetry failure we need to wait for next run | 09:56 |
| *** chem has joined #oooq | 09:56 | |
| panda | quiquell: it certainly seem more difficult to read | 09:56 |
| panda | quiquell: but how much do we need to read it ? | 09:57 |
| panda | the result I mean | 09:57 |
| quiquell | panda: YOu mean the output of jinja templates | 09:57 |
| sshnaidm|rover | chkumar|ruck, great, so we are good | 09:57 |
| panda | quiquell: yes | 09:57 |
| quiquell | panda: They are going to disappear in the next sprints | 09:57 |
| chkumar|ruck | ykarel: https://bugs.launchpad.net/tripleo/+bug/1752874/comments/4 last one reported | 09:58 |
| openstack | Launchpad bug 1752874 in tripleo ""Get https://registry-1.docker.io/v2/: dial tcp: lookup registry-1.docker.io on 127.0.0.1:53: server misbehaving" while Trying to pull repository docker.io/ceph/daemon " [Critical,Fix released] - Assigned to John Trowbridge (trown) | 09:58 |
| chkumar|ruck | ykarel: it is hitting again for pike | 09:58 |
| ykarel | chkumar|ruck, yup against this only u propsed patch and we had dicussion that day | 09:58 |
| quiquell | sshnaidm|rover, chkumar|ruck: Saw some timeouts... don't know if they are related to activation of containerized uc | 09:58 |
| panda | quiquell: not toci_quickstart.sh | 09:58 |
| *** honza_ is now known as honza | 10:01 | |
| quiquell | panda: It's not going to take us to much to remove it after this sprint | 10:01 |
| quiquell | panda: Not much people is going to check the jinja2 output. | 10:01 |
| chkumar|ruck | quiquell: yup one at execute tempest and another deploy the overcloud 2-3 timeout till now | 10:01 |
| chkumar|ruck | quiquell: keeping an eye if it exceeds | 10:02 |
| panda | quiquell: replacing toci_quickstart.sh is sgoing to be difficult, we need to refactor the release variables at least. Anyway, the other thing I wanted to investigate is env variable setting in that loop | 10:03 |
| chkumar|ruck | sorry need to findout which one is taking too much time | 10:03 |
| panda | quiquell: if in the loop we are changing some value that is reused in the next cycle, this is going to break it | 10:03 |
| panda | quiquell: are there ? maybe timeout variables ? | 10:03 |
| quiquell | panda: just timeout variables I think, we cannot change them | 10:04 |
| *** kopecmartin has quit IRC | 10:04 | |
| quiquell | panda: Tags is a problem too they are changed by upgrades | 10:04 |
| quiquell | panda: We can rename them and concatenate | 10:04 |
| *** kopecmartin has joined #oooq | 10:07 | |
| quiquell | chkumar|ruck, sshnaidm|rover: rrcockpit maintenance | 10:11 |
| chkumar|ruck | quiquell: ack! | 10:12 |
| sshnaidm|rover | quiquell, sure | 10:13 |
| quiquell | chkumar|ruck, sshnaidm|rover: back online | 10:14 |
| sshnaidm|rover | chkumar|ruck, quiquell I see a lot of node_failure in ovb.. | 10:16 |
| sshnaidm|rover | chkumar|ruck, like last jobs in this patch https://review.openstack.org/#/c/581529/ | 10:17 |
| chkumar|ruck | quiquell: http://paste.opensuse.org/95422203 I might be wrong, I am seeing the same job entry twice | 10:19 |
| sshnaidm|rover | chkumar|ruck, paste opensuse??? | 10:23 |
| sshnaidm|rover | chkumar|ruck, how is it possible?? :D | 10:23 |
| quiquell | sshnaidm|rover, chkumar|ruck: hahaha :-) | 10:23 |
| chkumar|ruck | sshnaidm|rover: is something wrong with the RDO third party job why they are voting as -1 since the job is passing | 10:24 |
| chkumar|ruck | sshnaidm|rover: https://review.openstack.org/#/c/583344/ | 10:24 |
| quiquell | chkumar|ruck: You are right, something is wrong, let me check | 10:24 |
| chkumar|ruck | quiquell: sshnaidm|rover in FOSS, everyone is friend | 10:24 |
| sshnaidm|rover | chkumar|ruck, because one of them failed on NODE_FAILURE | 10:25 |
| quiquell | chkumar|ruck: Just joking | 10:25 |
| sshnaidm|rover | chkumar|ruck, see in comments themselves | 10:25 |
| chkumar|ruck | sshnaidm|rover: yes | 10:25 |
| sshnaidm|rover | chkumar|ruck, I read suse was sold recently (again) to private | 10:26 |
| chkumar|ruck | sshnaidm|rover: yup that is correct | 10:26 |
| sshnaidm|rover | poor company, thrown from one to another.. when we'll buy it already | 10:27 |
| quiquell | chkumar|ruck: I think it's a influxdb hiccup not overwritting some jobs after the restart, let's check if it not more than that | 10:29 |
| chkumar|ruck | quiquell: I think during undercloud install they might be pulled from upstream to undercloud registery so thought to use it | 10:30 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, (1 more message) | 10:30 |
| chkumar|ruck | quiquell: https://review.openstack.org/#/c/549216/ was added earlier it from undercloud registery but removed | 10:31 |
| chkumar|ruck | quiquell: sorry | 10:31 |
| quiquell | ok, just saw some timeouts | 10:32 |
| quiquell | chkumar|ruck, sshnaidm|rover: Going to cleanup builds at rrcockpit | 10:36 |
| quiquell | ok ? | 10:36 |
| quiquell | They will appear again in a few | 10:36 |
| sshnaidm|rover | quiquell, ack | 10:36 |
| *** kopecmartin has quit IRC | 10:41 | |
| *** links has quit IRC | 10:49 | |
| *** jfrancoa is now known as jfrancoa|lunch | 10:52 | |
| *** amoralej is now known as amoralej|lunch | 10:54 | |
| quiquell | Humm when working with "with_items" at ansible | 10:56 |
| quiquell | yum ansible module is faster than package module | 10:56 |
| quiquell | looks like yum module is intelligent and it test it in one go | 10:56 |
| panda | https://review.openstack.org/583916 | 10:56 |
| quiquell | not the case for package module | 10:56 |
| panda | quiquell: any reason to not pass the list directly to the name argument ? | 10:57 |
| quiquell | panda: name doesn't support list at yum module | 10:58 |
| quiquell | panda: Humm "To operate on several packages this can accept a comma separated list of packages or (as of 2.0) a list of packages." | 10:58 |
| quiquell | Right | 10:58 |
| panda | quiquell: yep | 10:59 |
| quiquell | panda: Thanks, now it's even faster | 11:00 |
| *** links has joined #oooq | 11:12 | |
| chkumar|ruck | sshnaidm|rover: Anything we can do for node_failure for ovb fs01 and fs35 jobs? from in last 10 mins, we have more than 12 node failure | 11:17 |
| sshnaidm|rover | chkumar|ruck, working on this right now | 11:19 |
| sshnaidm|rover | chkumar|ruck, you need to join #rhos-ops in internal irc | 11:19 |
| sshnaidm|rover | chkumar|ruck, this channel is for rdo cloud problems :) | 11:19 |
| sshnaidm|rover | chkumar|ruck, oh, you're there, ok | 11:20 |
| *** quiquell is now known as quiquell|lunch | 11:21 | |
| *** udesale__ has quit IRC | 11:28 | |
| *** agopi is now known as agopi|brb | 11:32 | |
| *** kopecmartin has joined #oooq | 11:34 | |
| *** dtantsur|bbl is now known as dtantsur | 12:02 | |
| *** agopi|brb has quit IRC | 12:02 | |
| *** trown|outtypewww is now known as trown | 12:03 | |
| *** amoralej|lunch is now known as amoralej | 12:04 | |
| *** jfrancoa|lunch is now known as jfrancoa | 12:04 | |
| *** holser_ has joined #oooq | 12:22 | |
| *** holser_ has quit IRC | 12:24 | |
| *** holser_ has joined #oooq | 12:25 | |
| panda | rfolco: I still see the problem with openstack/diskimage-builder | 12:30 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, (1 more message) | 12:30 |
| panda | rfolco: https://softwarefactory.usersys.redhat.com/r/#/c/182/ | 12:33 |
| *** ratailor has quit IRC | 12:33 | |
| panda | rfolco: still failing for tripleo-ci | 12:33 |
| panda | rfolco: even after I merged the revert | 12:34 |
| *** ssbarnea1 has quit IRC | 12:36 | |
| rasca | hey folks, I need a core to add a +2 on this https://review.openstack.org/#/c/573255/ to finally merge it, can you help me? | 12:36 |
| *** ssbarnea has joined #oooq | 12:37 | |
| chkumar|ruck | sshnaidm|rover: quiquell|lunch logs.openstack.org not working | 12:39 |
| *** rlandy has joined #oooq | 12:40 | |
| sshnaidm|rover | chkumar|ruck, this question is for #openstack-infra | 12:40 |
| rfolco | panda, odd.. looking | 12:40 |
| panda | rasca: whoa, jinjia supports list operations ? | 12:41 |
| *** quiquell|lunch is now known as quiquell | 12:45 | |
| panda | rfolco: yes it does! | 12:46 |
| rasca | panda, indeed | 12:48 |
| panda | rasca: approved | 12:48 |
| rasca | panda, thanks your majesty | 12:49 |
| panda | rasca: now go, and bring glory to your kingdom | 12:49 |
| rfolco | panda, https://softwarefactory.usersys.redhat.com/r/#/c/189/ | 12:51 |
| rfolco | panda, noop change, no complaints | 12:51 |
| panda | rfolco: ok, merging mine and retesting ... | 12:53 |
| -openstackstatus- NOTICE: logs.openstack.org is offline, causing POST_FAILURE results from Zuul. Cause and resolution timeframe currently unknown. | 12:53 | |
| *** gvrangan has quit IRC | 12:54 | |
| panda | rfolco: bah, working now | 12:55 |
| rfolco | weird | 12:57 |
| quiquell | chkumar|ruck, sshnaidm|rover: This rings any bell ? finger://ze10.openstack.org/e30d658a783340d29e842566e356baab : POST_FAILURE | 13:02 |
| sshnaidm|rover | quiquell, see notice above, logs server is down | 13:02 |
| sshnaidm|rover | openstackstatus/#oooq- NOTICE: logs.openstack.org is offline, causing POST_FAILURE results from Zuul. Cause and resolution timeframe currently unknown. | 13:03 |
| quiquell | sshnaidm|rover: ack | 13:03 |
| myoung | o/ CI scrum starts now :) | 13:03 |
| myoung | weshay: are you joining us? | 13:04 |
| myoung | chkumar|ruck, sshnaidm|rover would one of you please join to give a brief ruck/rover update? | 13:06 |
| chkumar|ruck | myoung: link | 13:07 |
| chkumar|ruck | bj\ | 13:07 |
| sshnaidm|rover | myoung, on other mtg | 13:07 |
| myoung | https://bluejeans.com/7050859455 | 13:07 |
| rasca | rlandy, hi there, you around? Can I book a little slice of your time to talk about reproducers/my validate ha stuff? | 13:08 |
| rlandy | rasca: in meeting - when we are done | 13:09 |
| *** jtomasek has joined #oooq | 13:13 | |
| rasca | rlandy, sure, ping me when you want | 13:14 |
| *** agopi has joined #oooq | 13:15 | |
| *** udesale has joined #oooq | 13:18 | |
| quiquell | sshnaidm|rover: Do you want me to change this https://review.openstack.org/#/c/583861/ to use --ara-report or similar ? or you are already covering it ? | 13:23 |
| sshnaidm|rover | quiquell, let's talk after this meeting | 13:23 |
| quiquell | sshnaidm|rover: ack | 13:23 |
| weshay | ssbarnea, join #sf-dfg internal irc please | 13:24 |
| -openstackstatus- NOTICE: logs.openstack.org is back on-line. Changes with "POST_FAILURE" job results should be rechecked. | 13:37 | |
| *** ccamacho has quit IRC | 13:49 | |
| *** ccamacho1 has joined #oooq | 13:49 | |
| chkumar|ruck | sshnaidm|rover: weshay I am heading home, I will late for call | 13:51 |
| weshay | k | 13:52 |
| weshay | sshnaidm|rover, https://code.engineering.redhat.com/gerrit/#/c/144427/ | 13:58 |
| sshnaidm|rover | weshay, +w | 13:59 |
| weshay | had to drop | 14:03 |
| sshnaidm|rover | quiquell, want to sync now? | 14:04 |
| rlandy | rfolco: can we sync on browbeat? | 14:04 |
| quiquell | sshnaidm|rover: Let's sync | 14:04 |
| rfolco | rlandy, indeed | 14:04 |
| panda | quiquell: need to talk about that card | 14:05 |
| sshnaidm|rover | quiquell, https://bluejeans.com/u/sshnaidm | 14:05 |
| rlandy | rfolco: ok - what do you think we should be doing? | 14:05 |
| rfolco | rlandy, bj ? | 14:05 |
| rlandy | rfolco: ok | 14:05 |
| panda | marios: you have time now to discuss your card ? I have some questions | 14:05 |
| rfolco | rlandy, https://bluejeans.com/5878458097 | 14:06 |
| myoung | panda, chkumar|ruck, are you available to to join weshay and I (briefly) at 3:30 GMT+0? haven't seen response from invite | 14:07 |
| marios | panda: sure | 14:08 |
| marios | panda: call? | 14:08 |
| marios | panda: or do you want to comment on the review with the questions? | 14:08 |
| marios | panda: or here? | 14:08 |
| panda | myoung: yes | 14:08 |
| panda | marios: call | 14:08 |
| marios | panda: sure, gimme | 14:08 |
| marios | panda: sec | 14:09 |
| marios | panda: dropping let me know when you want to talk | 14:11 |
| marios | panda: i commented on https://review.openstack.org/#/c/581026/8/roles/bootstrap-subnodes/tasks/bootstrap.yml@67 about what we discussed please add any more info you want to | 14:25 |
| panda | marios: thanks. | 14:25 |
| *** jfrancoa has quit IRC | 14:28 | |
| sshnaidm|rover | weshay, we talked with marios about your bug: https://bugs.launchpad.net/openstack-infra/+bug/1781255 | 14:28 |
| openstack | Launchpad bug 1781255 in tripleo ""Error: centos-release-ceph-luminous conflicts with centos-release-ceph-jewel-1.0-1.el7.centos.noarch" [Undecided,Incomplete] | 14:28 |
| sshnaidm|rover | weshay, and found it very confusing, non descriptive and without essential details :) | 14:29 |
| weshay | sshnaidm|rover, heh | 14:29 |
| weshay | k.. /me looks | 14:29 |
| sshnaidm|rover | weshay, there is no even link to logs so we couldn't know when and what happens | 14:30 |
| weshay | sshnaidm|rover, I was only hitting that in the reproducer | 14:30 |
| hubbot | FAILING CHECK JOBS on stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci-centos-7-scenario008-multinode-oooq-container, tripleo-ci-centos-7-3nodes-multinode @ https://review.openstack.org/567224, stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, (1 more message) | 14:31 |
| *** quiquell is now known as quiquell|off | 14:32 | |
| sshnaidm|rover | weshay, there is nothing about reproducer in the bug | 14:32 |
| sshnaidm|rover | weshay, can we close it if it's not a bug? | 14:32 |
| weshay | k k.. I'll update the bug, move it to incoplete for now | 14:32 |
| *** arxcruz has quit IRC | 14:41 | |
| *** arxcruz has joined #oooq | 14:44 | |
| *** bogdando has quit IRC | 14:44 | |
| *** jfrancoa has joined #oooq | 14:46 | |
| chkumar|ruck | myoung: you moved the meeting to tomorrow | 14:49 |
| marios | ssbarnea: updated https://review.openstack.org/#/c/578081/12/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2 wdyt when you have time thanks | 14:55 |
| *** ratailor has joined #oooq | 14:56 | |
| myoung | chkumar|ruck: the planning meeting for s17? | 14:59 |
| chkumar|ruck | myoung: sorry mis read it | 14:59 |
| rfolco | rlandy, can't hear you... will drop.... my only suggestion is to cp browbeat from /home/zuul/src to the workspace dir just like we do with tq tqe etc | 15:00 |
| panda | wow 496 active instances in rdocloud | 15:01 |
| rlandy | rfolco: can you point me to the lines of codes where that is | 15:01 |
| rlandy | where we copy tqe from src? | 15:01 |
| weshay | myoung, you joining the tempest squad? | 15:01 |
| rfolco | rlandy, see if we can do this https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-ci/run-v3.yaml#L20 | 15:01 |
| rlandy | rfolco: ok but that is runv3 | 15:02 |
| rlandy | where is that done in legacy ovb? | 15:02 |
| *** dtantsur is now known as dtantsur|afk | 15:02 | |
| rfolco | rlandy, this was previously done by zuul-cloner... is browbeat cloned with zuul-cloner into workspace ? | 15:03 |
| *** ykarel is now known as ykarel|away | 15:04 | |
| rlandy | https://review.rdoproject.org/r/#/c/14808/7/playbooks/legacy/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/run.yaml | 15:04 |
| ssbarnea | marios: replied, one more bug. | 15:04 |
| rlandy | rfolco: /usr/zuul-env/bin/zuul-cloner -v $GIT_SOURCE $ZUUL_PROJECT | 15:06 |
| rfolco | rlandy, checking if it really does that, if breobeat is in workspace | 15:07 |
| *** jfrancoa has quit IRC | 15:08 | |
| rlandy | rfolco: missing here: https://logs.rdoproject.org/17/583717/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/412f3eb/job-output.txt.gz#_2018-07-19_03_39_40_788304 | 15:08 |
| *** jfrancoa has joined #oooq | 15:08 | |
| rlandy | no - I lie, it's there | 15:09 |
| rlandy | https://logs.rdoproject.org/17/583717/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/412f3eb/job-output.txt.gz#_2018-07-19_03_39_40_789214 | 15:09 |
| rfolco | rlandy, https://logs.rdoproject.org/17/583717/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/412f3eb/job-output.txt.gz#_2018-07-19_03_39_38_257307 | 15:10 |
| *** ccamacho1 has quit IRC | 15:10 | |
| rfolco | its there but we don't see if the commit is right | 15:10 |
| panda | rfolco: do we have access credentials for internal sf nodepool ? | 15:10 |
| rlandy | export 'ZUUL_CHANGES=openstack/tripleo-quickstart:master:refs/changes/84/581484/6^openstack/tripleo-quickstart:master:refs/changes/78/583578/1^openstack/tripleo-quickstart-extras:master:refs/changes/88/581488/15^openstack-infra/tripleo-ci:master:refs/changes/76/583576/6^openstack/browbeat:master:refs/changes/17/583717/3' | 15:11 |
| rlandy | rfolco; ^^ correct | 15:11 |
| rfolco | panda, no, cannot log into nodepool afaik :( | 15:11 |
| panda | rfolco: please say yes, please say yes | 15:11 |
| panda | NOOOOO | 15:11 |
| panda | I asked you to say yes | 15:11 |
| rfolco | I just work here sir | 15:11 |
| marios | ssbarnea: thanks :) you're right but virtualenv was the whole reason we added in the first place, will reply on the review and update | 15:12 |
| panda | testing the new path has just become a nightmare | 15:12 |
| rfolco | fill the customer satisfaction form panda | 15:12 |
| ssbarnea | marios: well, I kinda know, still I don't want to introduce new bugs and in this case it does break something that previously was working. | 15:13 |
| rfolco | rlandy, examining what it does... | 15:13 |
| rfolco | rlandy, I am still confused why you need this https://review.openstack.org/#/c/581484/6/quickstart-extras-requirements.txt | 15:14 |
| rfolco | either zuul-cloner is not getting the right commit, or this ^ is messing up with what zuul-cloner did (overriding with master)... then the playbook in .quickstart/browbeat.yml is master | 15:15 |
| rfolco | rlandy, makes any sense what I am saying ? | 15:15 |
| panda | wow, even worse, creating instance in rdocloud nodepool fails | 15:16 |
| marios | ssbarnea: no worries will update to handle both thanks for review | 15:18 |
| rlandy | rfolco: following extras pattern not sure what the diff is | 15:19 |
| ssbarnea | my pleasure, for the non virtualnenv path I would recomment a pip install --user ... --- which should make the script even more reliable. | 15:19 |
| *** gkadam-brb is now known as gkadam | 15:21 | |
| rfolco | rlandy, there is something I am missing... you may explain things like https://logs.rdoproject.org/17/583717/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/412f3eb/job-output.txt.gz#_2018-07-19_03_51_55_992120 | 15:24 |
| *** sshnaidm|rover is now known as sshnaidm|afk | 15:24 | |
| rlandy | looking | 15:28 |
| rlandy | rfolco: ^^ that has to do with the change I making now to skip the rpm build | 15:29 |
| marios | ssbarnea: wdyt then lucky number 13 https://review.openstack.org/#/c/578081/13/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2 | 15:29 |
| rlandy | rfolco: I think the error is not in the cloning but in build/incorporating the zuul change | 15:30 |
| chkumar|ruck | weshay: http://logs.openstack.org/84/581084/10/check/tripleo-ci-centos-7-scenario002-multinode-oooq-container-refstack/d1a158b/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-07-19_14_59_09 | 15:30 |
| rfolco | rlandy, k... I am looking at who does create .quickstart and run things from there... perhaps this thing gets tqe right but not browbeat | 15:30 |
| chkumar|ruck | overcloud deploy has failed twice I am not what is wrong | 15:31 |
| chkumar|ruck | with fs55 which comes from f21 | 15:31 |
| rlandy | rfolco; I think that part is ok | 15:31 |
| rlandy | we will see | 15:31 |
| ssbarnea | marios: but you didn't fix the line 214 which will fail if you dont have ansible or openstack already installed. | 15:31 |
| rfolco | rlandy, it would be good to put some "git show" for debugging what zuul-cloners gets in workspace | 15:31 |
| marios | ssbarnea: yeah i did | 15:33 |
| marios | ssbarnea: it is slightly different | 15:33 |
| marios | ssbarnea: well i hope i did | 15:33 |
| marios | :) | 15:33 |
| rlandy | I would expect to see a gating repo - which is in opt | 15:33 |
| rlandy | we don;t capture opt :( | 15:33 |
| marios | ssbarnea: https://review.openstack.org/#/c/578081/12..13/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2 see the diff 12 ..13 | 15:34 |
| marios | ssbarnea: /win 26 | 15:34 |
| marios | ssbarnea: cool thanks | 15:35 |
| marios | weshay: once upon a time you +2 this please consider re-adding https://review.openstack.org/#/c/578081/13 cc rlandy panda myoung please if you have some review time. not urgent this is sprint-16-etc thanks but been in review for a while. updated today for ssbarnea comments so it also runs on mac \o/ | 15:38 |
| * marios end sell | 15:39 | |
| *** holser__ has joined #oooq | 15:40 | |
| *** ykarel|away has quit IRC | 15:41 | |
| *** holser_ has quit IRC | 15:44 | |
| *** udesale has quit IRC | 15:46 | |
| *** jfrancoa has quit IRC | 15:48 | |
| *** skramaja has quit IRC | 15:52 | |
| *** kopecmartin has quit IRC | 16:01 | |
| *** vinaykns has joined #oooq | 16:04 | |
| vinaykns | Hello channel...I have a question....I couldn't introspect my overcloud nodes...it is waiting forever in looking for messages. | 16:06 |
| *** sshnaidm|afk is now known as sshnaidm|rover | 16:11 | |
| weshay | vinaykns, there is a bug | 16:15 |
| weshay | vinaykns, /me gets | 16:15 |
| weshay | vinaykns, http://bugs.launchpad.net/bugs/1782267 | 16:15 |
| openstack | Launchpad bug 1782267 in tripleo "Stderr: u'Set Chassis Power Control to Up/On failed: Command not supported in present state\n': ProcessExecutionError: Unexpected error while running command." [Critical,Triaged] | 16:15 |
| rlandy | rasca: ready to meet when you are | 16:16 |
| rfolco | rlandy, this is becoming overcomplicated. I'll step back and resume my tasks. I could not find where browbeat-minimal.yml comes from... like other playbooks are copied here: https://logs.rdoproject.org/17/583717/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset053-master/412f3eb/job-output.txt.gz#_2018-07-19_03_44_32_020778 | 16:17 |
| vinaykns | weshay: So is this a blocker from getting quickstart up and running..? | 16:19 |
| rlandy | rfolco:browbeat-minimla comes from browbeat repo | 16:20 |
| weshay | vinaykns, it's a blocker on the vbmc, ironic and libvirt | 16:20 |
| rlandy | copied but setup.cfg | 16:20 |
| weshay | vinaykns, you can use ovb and multinode reproducers atm | 16:20 |
| weshay | they are not impacted | 16:20 |
| rlandy | rfolco: I have a CI run going iwth a change to build-test | 16:20 |
| rlandy | will see where that leads | 16:20 |
| weshay | vinaykns, http://tripleo.org/contributor/reproduce-ci.html | 16:20 |
| rfolco | rlandy, k, sorry for not helping much | 16:20 |
| rlandy | I will probably need to set up a reproducer | 16:21 |
| rlandy | so we can investigate | 16:21 |
| rlandy | rfolco: nothing to be sorry about - we are not done here | 16:21 |
| rlandy | I may still need your expertise | 16:21 |
| rfolco | rlandy, no? I was running away from the problem... | 16:21 |
| rfolco | :) | 16:21 |
| *** agopi is now known as agopi|food | 16:21 | |
| rlandy | rfolco: my friend Sam wrote this book - you may appreciated it after your comment above: https://www.amazon.com/Overcomplicated-Technology-at-Limits-Comprehension/dp/0143131303 | 16:22 |
| *** amoralej is now known as amoralej|off | 16:23 | |
| rfolco | rlandy, looks interesting read | 16:25 |
| *** agopi|food is now known as agopi | 16:25 | |
| *** ratailor has quit IRC | 16:28 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 16:31 |
| vinaykns | weshay: So you meant to say that I can reproduce the stack on a openstack vm.? using the reproduce-ci document. | 16:33 |
| weshay | vinaykns, yes.. you can get a deployment up using the instructions there.. either w/ rdo-cloud or libvirt | 16:34 |
| vinaykns | weshay: Thank you.!! | 16:35 |
| *** dtrainor has quit IRC | 16:35 | |
| weshay | sshnaidm|rover, https://review.openstack.org/#/c/583736/ | 16:37 |
| *** dtrainor has joined #oooq | 16:38 | |
| *** agopi is now known as agopi|food | 16:44 | |
| sshnaidm|rover | weshay, commented | 16:45 |
| *** tesseract has quit IRC | 16:56 | |
| vinaykns | weshay: but can I do customizations on the overcloud using the script..?? | 16:57 |
| weshay | sshnaidm|rover, k.. I'll add bootstrap back... I thought that would be handled | 16:58 |
| weshay | by quickstart.sh.. | 16:58 |
| weshay | but it doesn't appear to be | 16:58 |
| myoung | weshay, chkumar|ruck, panda: is tomorrow @2:30 GMT a good time for sprint 17 pre-planning? with weshay out next week I want to confirm that we have time to cover what's needed. | 17:01 |
| weshay | sshnaidm|rover, ok https://review.openstack.org/583736 | 17:01 |
| myoung | ^^ invite has been sent, fairly open tomorrow on my end | 17:02 |
| weshay | vinaykns, you'd have to be more specific | 17:02 |
| *** links has quit IRC | 17:03 | |
| panda | myoung: yes | 17:03 |
| *** brault has quit IRC | 17:04 | |
| vinaykns | weshay: for ex I need to have qpidrouter daemon listening for oslo notifications messages instead of the default driver. | 17:04 |
| *** brault has joined #oooq | 17:04 | |
| weshay | vinaykns, if you can do it in a an upstream review.. yes | 17:05 |
| sshnaidm|rover | weshay, chkumar|ruck want to sync before I leave? | 17:07 |
| weshay | sshnaidm|rover, anything you need me to take over? | 17:08 |
| sshnaidm|rover | weshay, yep | 17:08 |
| weshay | bah k | 17:08 |
| weshay | going to blue | 17:08 |
| *** trown is now known as trown|lunch | 17:08 | |
| *** brault has quit IRC | 17:09 | |
| *** gkadam has quit IRC | 17:09 | |
| weshay | sshnaidm|rover, I'm in blue | 17:09 |
| *** holser__ has quit IRC | 17:11 | |
| vinaykns | weshay: I have a question...in the document that you have pointed me to follow. It says to source openstack_rc.sh, how would i do that if I am in an openstack vm..?? | 17:24 |
| *** agopi|food is now known as agopi | 17:36 | |
| *** chem has quit IRC | 17:43 | |
| sshnaidm|rover | weshay, 2 stacks failed to delete: 2f5a5f68-b9a2-4a8b-94f8-dc07b5fd72e5 and 46c037ac-d8eb-4f64-82c5-3fb6636c2398 , so I will open a ticker to rdo ops about it | 17:45 |
| panda | rlandy: sshnaidm|rover https://review.openstack.org/584040 works locally on my tenant. Adding tests from internal sh on my tenant. After this passes, I can start moving to rdo | 17:47 |
| sshnaidm|rover | panda, this is tested only in downstream and ovb reproducer | 17:55 |
| sshnaidm|rover | panda, maybe worth to kick jobs in down jenkins to check it after merge to be sure nothing is broken | 17:56 |
| *** sshnaidm|rover is now known as sshnaidm|off | 17:57 | |
| panda | sshnaidm|rover: you mean ovb-manage-stack role ? Since this will be part of CI, we can trigger an ovb job with changes to it | 17:57 |
| *** gvrangan has joined #oooq | 17:58 | |
| panda | hhhmmm, not true | 17:59 |
| panda | will be part of CI but in trusted repo | 17:59 |
| panda | will be difficult to test | 17:59 |
| panda | we'll need to create fictious test .. | 18:00 |
| *** florianf has quit IRC | 18:12 | |
| *** trown|lunch is now known as trown | 18:13 | |
| *** panda is now known as panda|off | 18:20 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000-multinode-oooq-container-upgrades, tripleo-ci- (1 more message) | 18:31 |
| weshay | sshnaidm|off, no need | 18:46 |
| weshay | we can nuke them | 18:46 |
| weshay | rlandy, talking to paul about ovb.. may need you in a few if ur avail | 18:48 |
| rlandy | ok | 18:48 |
| jrist | weshay: I pulled down your https://review.openstack.org/#/c/583042 via master today and am doing another start so I can see if I hit that vbmc failure | 18:53 |
| *** gvrangan has quit IRC | 18:55 | |
| weshay | rlandy, does ovb use https://github.com/openstack/tripleo-quickstart/blob/master/roles/virtbmc/tasks/configure-vbmc.yml | 19:14 |
| rlandy | https://github.com/openstack-infra/tripleo-ci/blob/master/toci-quickstart/config/testenv/ovb.yml#L8 | 19:15 |
| weshay | rlandy, ok.. how is the bmc installed in ovb | 19:15 |
| * weshay forgets | 19:15 | |
| *** tosky has quit IRC | 19:16 | |
| rlandy | https://github.com/cybertron/openstack-virtual-baremetal/blob/master/openstack_virtual_baremetal/openstackbmc.py | 19:16 |
| *** brault has joined #oooq | 19:21 | |
| weshay | rlandy, ping | 19:21 |
| weshay | rlandy, join my blue for 3min | 19:21 |
| *** brault has quit IRC | 19:25 | |
| weshay | jrist, that patch has nothing to do w/ it | 19:30 |
| jrist | I know | 19:30 |
| weshay | jrist, there are no patches atm | 19:30 |
| jrist | there are patches for 7.5 | 19:30 |
| jrist | RHEL | 19:30 |
| weshay | jrist, not that help | 19:30 |
| jrist | but I was wondering if you've seen some for CentOS | 19:30 |
| jrist | ah | 19:30 |
| jrist | :( | 19:30 |
| weshay | jrist, we need to push on the hardware provisioning guys | 19:34 |
| rfolco | myoung, let me know if you want a quick background on why things are done in that particular way... | 19:47 |
| myoung | rfolco: will do, taking a whack at running thru them. when i have more context loaded I'll ping ya later on today or tomorrow AM | 19:48 |
| rfolco | myoung deal | 19:48 |
| jrist | weshay: how do you propose the best way to do that? | 19:52 |
| jrist | who do I need to poke? | 19:52 |
| *** dougbtv has left #oooq | 19:55 | |
| *** myoung is now known as myoung|biab | 20:00 | |
| *** holser_ has joined #oooq | 20:05 | |
| weshay | rlandy, https://review.openstack.org/#/c/584088/ | 20:06 |
| weshay | jrist, can you guys use ovb while we're trying to get this fixed? | 20:11 |
| * weshay needs more logs https://review.openstack.org/#/c/584088/ | 20:11 | |
| jrist | ovb? | 20:12 |
| jrist | you mean rdo cloud? | 20:12 |
| jrist | broken for other reasons afaik | 20:12 |
| jrist | right honza ? | 20:12 |
| *** holser_ has quit IRC | 20:15 | |
| *** holser_ has joined #oooq | 20:15 | |
| weshay | BOOM master is back online w/ bm jobs :) | 20:18 |
| weshay | rlandy, https://rhos-dev-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/all%20top%20level%20multijobs/job/rdo-promote-master-rdo_trunk/ | 20:18 |
| weshay | \0/ | 20:18 |
| weshay | jenkins sooooooooks | 20:18 |
| honza | weshay: jrist https://bugs.launchpad.net/tripleo/+bug/1782588 | 20:18 |
| openstack | Launchpad bug 1782588 in tripleo "OVB configs not set up for containerized undercloud" [High,Triaged] | 20:18 |
| rlandy | ? | 20:18 |
| weshay | honza, wha? | 20:19 |
| rlandy | rfolco: I think I may know what is going on here | 20:19 |
| weshay | honza, we need to work on your bugs | 20:19 |
| jrist | yeah. | 20:19 |
| jrist | +1000 | 20:19 |
| honza | maybe i'm just very incompetent | 20:20 |
| honza | all of this used to work for me, but no more | 20:20 |
| rfolco | rlandy, tell me coz I need to control my OCD level | 20:20 |
| rlandy | weshay: how come, with upgrades, we did not add that repo to requirements? | 20:20 |
| rlandy | rfolco: ^^ | 20:20 |
| honza | weshay: i'm happy to work on it given enough guidance | 20:20 |
| rlandy | look at how upgrades is added | 20:20 |
| weshay | honza, can you paste in the bug how you were using ovb | 20:20 |
| honza | lol, i thought this channel was gonna be closed in favor of #tripleo | 20:21 |
| weshay | honza, aye.. well turns out we generate a lot of traffic | 20:21 |
| rfolco | rlandy, how upgrades relate to browbeat ? | 20:21 |
| rlandy | rfolco: https://github.com/openstack-infra/tripleo-ci/blob/master/toci_gate_test.sh#L226 | 20:22 |
| rlandy | also a standalone repo | 20:22 |
| rlandy | where we source roles and playbooks | 20:22 |
| honza | weshay: done | 20:22 |
| weshay | rlandy, oh man.. we need to nuke devmode | 20:22 |
| rlandy | weshay: pls - nuke it with fire | 20:22 |
| weshay | honza, http://tripleo.org/contributor/reproduce-ci.html | 20:22 |
| * weshay should remove that script have it return 0 | 20:23 | |
| honza | weshay: why do i care about this? | 20:23 |
| honza | weshay: https://docs.openstack.org/tripleo-quickstart/latest/devmode-ovb.html | 20:23 |
| rlandy | weshay: how come we decided on https://github.com/openstack-infra/tripleo-ci/blob/master/toci_gate_test.sh#L226 for upgrades? | 20:24 |
| rlandy | rfolco: I think what is happening is the Depends_On on https://review.openstack.org/#/c/581484/6/quickstart-extras-requirements.txt | 20:25 |
| rlandy | ^^ adds that when we have a change | 20:25 |
| weshay | rlandy, https://review.openstack.org/584097 | 20:25 |
| weshay | honza, https://review.openstack.org/584097 | 20:26 |
| rlandy | rfolco: ^^ so that will pull from master once the change is applied | 20:26 |
| rlandy | as opposed to what upgrades is doing | 20:27 |
| jrist | weshay: can you do one for infrared? :) | 20:27 |
| honza | jrist: YES YES YES | 20:27 |
| * honza reads reproducer-ci docs | 20:27 | |
| rlandy | rfolco: I think I need to remove quickstart-extras-requirements.txt addition | 20:28 |
| honza | weshay: this is very counter intuitive because i'm not reproducing any ci things | 20:28 |
| weshay | jrist, do a reproducer script? | 20:28 |
| rlandy | and do what upgrades is doing | 20:28 |
| honza | oooq isn't a ci-only tool, yo | 20:28 |
| rfolco | rlandy, not sure I got the point. Depends-on does not interfere in requirements. But the opposite might be true. | 20:28 |
| weshay | honza, aye.. I know and I'm sorry for that, but really it's ur best shot | 20:28 |
| weshay | devmode is a dead end | 20:28 |
| weshay | there is no good way to test it | 20:29 |
| weshay | honza, just remove the zuul_change= | 20:29 |
| rlandy | rfolco: forget depends-on | 20:29 |
| weshay | from any given script | 20:29 |
| honza | weshay: so, ... given that i don't have a ci job to go to, to go to the logs dir of... how do i set up an env in ovb? | 20:29 |
| rlandy | look at the change itself | 20:29 |
| weshay | and you'll be fine | 20:29 |
| rlandy | the additional references master | 20:29 |
| rfolco | rlandy, I believe you are going to remove from requirements and use browbeat cloned by zuul-cloner | 20:30 |
| honza | i can do this! | 20:30 |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000 (1 more message) | 20:31 |
| rlandy | rfolco: idk exactly | 20:31 |
| rlandy | bit the two are unrelated | 20:31 |
| *** agopi has left #oooq | 20:31 | |
| *** agopi has joined #oooq | 20:31 | |
| weshay | honza, sec | 20:31 |
| weshay | honza, jrist https://logs.rdoproject.org/45/560445/90/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/24848f0/logs/reproducer-quickstart.sh | 20:32 |
| rfolco | rlandy, I don't understand why browbeat is required at requirements. If the change is on browbeat itself, we need to apply it... its ok to install master for all other changes in other repos... | 20:32 |
| weshay | honza, jrist you guys can change "openstack/tripleo-quickstart-extras:master:refs/changes/45/560445/90" | 20:32 |
| weshay | to be "" | 20:32 |
| * weshay goes to the dentist | 20:32 | |
| honza | bash reproducer-quickstart.sh -w ~/.oooq/rdo-cloud -r | 20:33 |
| honza | returns immediately, does nothing | 20:33 |
| rlandy | rfolco: to pick up the playbooks | 20:33 |
| weshay | honza, https://docs.openstack.org/tripleo-docs/latest/contributor/reproduce-ci.html | 20:33 |
| * weshay away bbl | 20:33 | |
| rlandy | read the comments in the upgrades stuff I linked above | 20:33 |
| honza | rlandy: me? | 20:34 |
| rlandy | honza: no, rfolco | 20:34 |
| honza | rlandy: ha, sorry | 20:34 |
| rlandy | honza: you are welcome to read it as well, but I am not sure you would care to | 20:35 |
| rfolco | I see | 20:35 |
| honza | lol, you have to pass in "true" | 20:42 |
| honza | bash reproducer-quickstart.sh -w ~/.oooq/rdo-cloud -r true -v true | 20:42 |
| honza | it's doing something! | 20:42 |
| honza | is quickstart.sh deprecated, too? | 20:44 |
| weshay | honza, it's going to be :) | 20:46 |
| jrist | whatttt | 20:46 |
| jrist | stop it | 20:46 |
| honza | it's installing packages, somewhere! | 20:46 |
| jrist | :) | 20:46 |
| weshay | jrist, there is a libvirt option to that script too | 20:46 |
| jrist | tell me now | 20:46 |
| weshay | jrist, honza we can't do full runs on tripleo on ci.centos nodes anymore | 20:47 |
| jrist | whattt | 20:47 |
| weshay | the hardware there no longer supports tripleo since really ocata | 20:47 |
| jrist | (╯°□°）╯︵ ┻━┻ | 20:47 |
| weshay | tripleo is tooooo phat | 20:47 |
| honza | lol | 20:47 |
| honza | i just want people to start caring about the non-ci user :) | 20:47 |
| weshay | so .. we have to shift the tooling to be very close to what is executed upstream and in third party | 20:48 |
| jrist | DEVS ARE REAL PEOPLE | 20:48 |
| weshay | hence.. | 20:48 |
| weshay | https://logs.rdoproject.org/45/560445/90/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/24848f0/logs/reproducer-quickstart.sh | 20:48 |
| weshay | ok.. now I really have to go to the dentist | 20:48 |
| honza | weshay: something suceeded, it gave me instructions on how to connect to an undercloud | 20:59 |
| honza | but it's asking me to use a zuul user | 20:59 |
| honza | and the 'undercloud' doesn't have anything on it... | 21:00 |
| *** agopi has quit IRC | 21:05 | |
| honza | weshay: also, the reproducer script in the root of tripleo-quickstart doesn't match the one in your link | 21:06 |
| honza | e.g. i can't override the toci job type | 21:07 |
| *** trown is now known as trown|outtypewww | 21:07 | |
| *** holser_ has quit IRC | 21:12 | |
| *** myoung|biab is now known as myoung | 21:16 | |
| honza | it creates the stack but then i'm stuck | 21:16 |
| honza | we need some human-operator docs cc jrist | 21:17 |
| honza | ... if you beautiful people guide me through the process, then i can write some | 21:17 |
| rlandy | rfolco: weshay: am I correct that the https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/nodepool-setup/ is only used for the reproducer? and not in CI | 21:22 |
| rfolco | dunno | 21:35 |
| rlandy | rfolco: so where is this done in CI itself? | 21:45 |
| *** dtrainor has quit IRC | 21:51 | |
| *** hamzy has quit IRC | 21:53 | |
| *** hamzy has joined #oooq | 21:53 | |
| *** dtrainor has joined #oooq | 21:56 | |
| *** brault has joined #oooq | 21:59 | |
| *** jtomasek has quit IRC | 22:00 | |
| *** brault has quit IRC | 22:03 | |
| rfolco | rlandy, I cannot match most of what is there in upstream ci | 22:20 |
| rfolco | some tasks are legacy and still exists like etc/nodepool | 22:20 |
| rfolco | hard to tell where this is used though | 22:21 |
| *** hubbot has quit IRC | 22:28 | |
| *** hubbot has joined #oooq | 22:29 | |
| hubbot | FAILING CHECK JOBS on stable/ocata: legacy-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-ocata @ https://review.openstack.org/564291, master: tripleo-ci-centos-7-scenario002-multinode-oooq-container, tripleo-ci-centos-7-scenario009-multinode-oooq, legacy-tripleo-ci-centos-7-container-to-container-upgrades-master @ https://review.openstack.org/560445, stable/queens: tripleo-ci-centos-7-scenario000 (1 more message) | 22:31 |
| weshay | rlandy, yes.. that is correct | 22:50 |
| honza | weshay: it says to run /opt/stack/tripleo-ci/toci_gate_test-oooq.sh but that file doesn't exist | 22:53 |
| honza | weshay: is subnode-0 the undercloud? | 22:53 |
| weshay | oh man.. did that patch not merge.. I think it's /opt/stack/tripleo-ci/toci_gate_test.sh | 22:53 |
| weshay | honza, ya.. that is following upstream nodepool naming conventions | 22:54 |
| honza | weshay: ok, cool, it's running something now... | 22:54 |
| honza | fails with "cp: cannot stat ‘/opt/stack/new/tripleo-ci/toci-quickstart/config/testenv/_hosts’: No such file or directory" | 22:54 |
| honza | lol, forgot to source vars | 22:55 |
| weshay | :) | 22:55 |
| honza | installing rpms now | 22:55 |
| honza | weshay: "Playbook run of multinode-undercloud.yml failed" | 22:58 |
| weshay | honza, how are you so lucky | 22:58 |
| weshay | man | 22:58 |
| weshay | honza, got tmate? | 22:58 |
| *** rlandy is now known as rlandy|bbl | 22:59 | |
| honza | weshay: nein | 23:00 |
| weshay | honza, https://review.openstack.org/#/c/578533/ | 23:00 |
| honza | for the subnode? | 23:00 |
| weshay | honza, how are you getting such old code? | 23:00 |
| honza | or you want to play with my laptop? | 23:00 |
| weshay | that was merged 3 weeks ago | 23:00 |
| weshay | source cloudrc.sh; bash -x reproducer-quickstart.sh -w /var/tmp/fs20 --create-virtualenv true -r true -p fs20 | 23:01 |
| honza | weshay: i'm using the reproducer script from tripleo-quickstart | 23:01 |
| honza | not extras | 23:01 |
| weshay | that's an example from my histroy | 23:01 |
| weshay | honza, huh | 23:01 |
| weshay | did you wget the file I pointed you at? | 23:01 |
| honza | i don't have a ci job | 23:01 |
| honza | lol, ok | 23:02 |
| honza | sorry | 23:02 |
| honza | i'll try that! | 23:02 |
| weshay | honza, that's the only way it works dude | 23:02 |
| honza | i thought it was the same file but with some values prefilled | 23:02 |
| honza | weshay: why do you have that file in oooq? .... when it gets generate via that role in -extras? | 23:02 |
| honza | so confused | 23:02 |
| *** vinaykns has quit IRC | 23:02 | |
| * weshay gets again | 23:03 | |
| weshay | wget https://logs.rdoproject.org/45/560445/90/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/ebc929a/logs/reproducer-quickstart.sh | 23:04 |
| honza | running now | 23:04 |
| weshay | source cloudrc.sh; bash -x reproducer-quickstart.sh -w /var/tmp/fs20 --create-virtualenv true -r true -p fs20 | 23:04 |
| honza | weshay: why do i need some random script from a random ci job to install an undercloud in ovb? | 23:04 |
| weshay | honza, it's not a random script, however I just took an ovb job fs001 is the standard | 23:05 |
| weshay | and I took a recent run of it | 23:06 |
| weshay | honza, so I know for sure it passed | 23:06 |
| weshay | honza, because you'll be running the exact same thing as https://logs.rdoproject.org/45/560445/90/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/ebc929a/logs/reproducer-quickstart.sh | 23:06 |
| weshay | as.. | 23:06 |
| weshay | https://logs.rdoproject.org/45/560445/90/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/ebc929a/job-output.txt.gz | 23:06 |
| honza | weshay: is there a way to start a build/run without a reproducer script built by a previous ci job? | 23:07 |
| weshay | honza, the only way your run would fail would be some random infra issue | 23:07 |
| weshay | which could happen | 23:07 |
| honza | right | 23:07 |
| honza | like, how do i bootstrap this? | 23:07 |
| honza | i realize that oooq is used for ci a lot but how do i use it as an installation tool for my dev env? | 23:07 |
| weshay | honza, can you explain that more specifically | 23:07 |
| weshay | bootstrap can mean lots of different things | 23:07 |
| honza | ok | 23:07 |
| honza | so i have creds to an ovb instance | 23:08 |
| weshay | honza, ya.. it's not a finished product for sure.. w/ regards to just bootstraping a devel env | 23:08 |
| weshay | however the important thing is that it's well tested | 23:08 |
| honza | and i'd like to set up a dev env (undercloud, and some blank nodes) | 23:08 |
| honza | what is that recommended way of using oooq to do that? | 23:08 |
| honza | quickstart.sh does that well for libvirt | 23:08 |
| honza | devmode.sh did that beautifully for ovb | 23:08 |
| honza | but devmode.sh is no more | 23:09 |
| weshay | ya.. and quickstart.sh will continue to work however we're going to keep building off of the reproducer script | 23:09 |
| honza | what is the point of the reproducer-script.sh file in tripleo-quickstart? | 23:09 |
| weshay | changing the tires on a plane at 30k feet | 23:09 |
| weshay | honza, so if you need to debug a gerrit review that is failing | 23:09 |
| honza | sorry, reproducer-quickstart.sh | 23:09 |
| weshay | it reproduces that exactly | 23:09 |
| honza | I don't follow. | 23:10 |
| weshay | so if you get a zuul -1 or -2 | 23:10 |
| honza | no, not that | 23:10 |
| honza | that's what the one in -extras does | 23:10 |
| honza | with the template and all | 23:10 |
| weshay | honza, for instance your patch | 23:10 |
| honza | for that job specifically | 23:10 |
| weshay | https://review.openstack.org/#/c/509226/ | 23:10 |
| weshay | is failing | 23:10 |
| honza | when i used that code from oooq you said it was super old | 23:10 |
| weshay | you could recheck all day | 23:11 |
| weshay | that's bad | 23:11 |
| weshay | omg that's an old patch | 23:11 |
| honza | e.g. https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/create-reproducer-script/templates/reproducer-quickstart.sh.j2 | 23:12 |
| weshay | honza, the reproducer script does exactly what devmode used to do | 23:12 |
| weshay | honza, ya.. don't | 23:12 |
| weshay | that's a jinja template | 23:12 |
| weshay | it needs to render | 23:13 |
| honza | WELP | 23:13 |
| honza | weshay: so sorry, i'm an idiot | 23:13 |
| honza | ugggghhhhh | 23:13 |
| honza | so that reproducer file i was getting mad about isn't actually checked into git :( | 23:13 |
| honza | it's so leftover junk | 23:13 |
| weshay | just wget the one I gaave | 23:14 |
| weshay | gave u | 23:14 |
| honza | yes yes yes | 23:14 |
| honza | so, we need a way to get a reproducer-quickstart.sh | 23:15 |
| honza | so, it looks like the part that i'm after doesn't exist yet | 23:16 |
| honza | i want to get that script from a repo somewhere | 23:16 |
| honza | with some nice defaults, for ovb and libvirt | 23:17 |
| honza | if and when this finishes and works i'll be able to keep for a bit, i suppose | 23:17 |
| honza | but eventually it'll get outdated again | 23:17 |
| honza | ... hence the desire for version-controlled file | 23:17 |
| * honza goes afk while undercloud installs | 23:21 | |
| *** vinaykns has joined #oooq | 23:42 | |
| weshay | honza, aye.. I can point you at some jobs that will help.. and yes.. we would like to have that done for folks automatically as well | 23:53 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!