| *** pmannidi has quit IRC | 00:33 | |
| *** pmannidi has joined #openstack-dib | 00:33 | |
| openstackgerrit | Merged openstack/diskimage-builder feature/v2: Semi-automatic doc generation of element dependency https://review.openstack.org/430537 | 00:38 |
|---|---|---|
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder feature/v2: Merge branch 'master' into merge-branch https://review.openstack.org/431237 | 01:00 |
| ianw | ok, that's the last merge i think before a rc ... let's see how CI like its | 01:00 |
| ianw | it | 01:00 |
| *** owalsh has joined #openstack-dib | 02:30 | |
| *** owalsh-afk has quit IRC | 02:31 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder feature/v2: Merge branch 'master' into merge-branch https://review.openstack.org/431237 | 02:37 |
| *** sudipto has joined #openstack-dib | 03:18 | |
| *** sudipto_ has joined #openstack-dib | 03:18 | |
| *** chhavi has joined #openstack-dib | 03:47 | |
| openstackgerrit | Merged openstack/diskimage-builder feature/v2: Merge branch 'master' into merge-branch https://review.openstack.org/431237 | 04:03 |
| *** sudipto has quit IRC | 05:18 | |
| *** sudipto_ has quit IRC | 05:18 | |
| *** owalsh_ has joined #openstack-dib | 05:38 | |
| *** owalsh has quit IRC | 05:39 | |
| *** sudipto has joined #openstack-dib | 05:40 | |
| *** sudipto_ has joined #openstack-dib | 05:40 | |
| andreas-f | ianw / greg: IMHO there are no patches out there that should be merged before v2. | 05:55 |
| ianw | andreas-f: that's good because I just pushed a 2.0.0rc1 tag :) | 05:55 |
| ianw | it seemed to go ok, although it's generated http://tarballs.openstack.org/diskimage-builder/diskimage-builder-2.0.0.0rc1.tar.gz | 05:56 |
| ianw | i do not know why there's an extra ".0" in there | 05:56 |
| andreas-f | There are already some additional refactor-patches - but they should be merged later (tried to document this on the etherpad under 'block device DIB 2.x activities). | 05:57 |
| andreas-f | My proposal is to add a short section in the announcement about deprecations; added a draft in the etherpad ; feel free to change / remove it. | 05:58 |
| ianw | andreas-f: ok, i'm just editing up now. i might leave that because the mail is already getting too long to hold anyone's attention I think & I don't want to create even more confusion that i'm sure there will be | 06:05 |
| andreas-f | Ack. | 06:05 |
| andreas-f | Will run some tests on 'exotic' distributions (Debian, gentoo, opensuse, ...) to see if there are any regressions for them. | 06:18 |
| ianw | ok mail sent, i'm pretty much out for the evening, catch you later! | 06:27 |
| andreas-f | Until later (just starting my breakfast over here ;-) ) | 06:28 |
| *** TheJulia_ has joined #openstack-dib | 06:29 | |
| *** TheJulia has quit IRC | 06:33 | |
| *** TheJulia_ is now known as TheJulia | 06:33 | |
| *** chhavi has quit IRC | 06:33 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Docker matrix build https://review.openstack.org/414347 | 06:35 |
| greghaynes | ianw: andreas-f wooo 2.0 rc :) | 06:52 |
| *** lennyb has joined #openstack-dib | 07:08 | |
| *** noama has joined #openstack-dib | 07:32 | |
| *** pmannidi has quit IRC | 07:55 | |
| *** noama has quit IRC | 08:09 | |
| *** owalsh_ is now known as owalsh-afk | 08:55 | |
| *** openstackgerrit has quit IRC | 09:03 | |
| *** noama has joined #openstack-dib | 09:08 | |
| *** noama_ has joined #openstack-dib | 09:08 | |
| *** noama__ has joined #openstack-dib | 09:09 | |
| *** noama has quit IRC | 09:09 | |
| *** noama__ has quit IRC | 09:09 | |
| *** noama_ has quit IRC | 09:09 | |
| *** noama has joined #openstack-dib | 09:09 | |
| *** chhavi has joined #openstack-dib | 09:10 | |
| *** hashar has joined #openstack-dib | 09:17 | |
| *** chhavi has quit IRC | 09:56 | |
| *** chhavi has joined #openstack-dib | 10:20 | |
| *** sudipto has quit IRC | 11:27 | |
| *** sudipto_ has quit IRC | 11:27 | |
| *** owalsh-afk is now known as owalsh | 12:01 | |
| *** noama has quit IRC | 12:22 | |
| *** chhavi has quit IRC | 12:31 | |
| *** brault|a_ has joined #openstack-dib | 12:56 | |
| *** brault|__ has joined #openstack-dib | 12:58 | |
| *** brault|away has quit IRC | 12:58 | |
| *** brault|__ has quit IRC | 13:02 | |
| *** brault|a_ has quit IRC | 13:04 | |
| *** brault|away has joined #openstack-dib | 13:04 | |
| *** brault|a_ has joined #openstack-dib | 13:06 | |
| *** chhavi has joined #openstack-dib | 13:08 | |
| *** brault|__ has joined #openstack-dib | 13:08 | |
| *** brault|away has quit IRC | 13:08 | |
| *** sudipto has joined #openstack-dib | 13:11 | |
| *** sudipto_ has joined #openstack-dib | 13:11 | |
| *** brault|a_ has quit IRC | 13:11 | |
| *** brault|away has joined #openstack-dib | 13:11 | |
| *** brault|__ has quit IRC | 13:12 | |
| *** brault|away has quit IRC | 13:15 | |
| *** brault|away has joined #openstack-dib | 13:23 | |
| *** brault|away has quit IRC | 13:28 | |
| *** owalsh is now known as owalsh-brb | 14:22 | |
| *** owalsh-brb has quit IRC | 14:26 | |
| *** brault|away has joined #openstack-dib | 14:31 | |
| *** owalsh-brb has joined #openstack-dib | 14:31 | |
| *** owalsh-brb is now known as owalsh | 14:31 | |
| *** brault|away has quit IRC | 14:35 | |
| *** brault|away has joined #openstack-dib | 14:36 | |
| *** brault|a_ has joined #openstack-dib | 14:38 | |
| *** brault|away has quit IRC | 14:40 | |
| *** brault|away has joined #openstack-dib | 14:40 | |
| *** brault|__ has joined #openstack-dib | 14:42 | |
| *** brault|a_ has quit IRC | 14:42 | |
| *** brault|a_ has joined #openstack-dib | 14:45 | |
| *** brault|away has quit IRC | 14:45 | |
| *** brault|__ has quit IRC | 14:47 | |
| *** brault|away has joined #openstack-dib | 14:49 | |
| *** brault|a_ has quit IRC | 14:49 | |
| *** brault|away has quit IRC | 14:54 | |
| *** brault|away has joined #openstack-dib | 14:55 | |
| *** brault|away has quit IRC | 15:01 | |
| *** brault|away has joined #openstack-dib | 15:12 | |
| *** chhavi has quit IRC | 15:27 | |
| *** chhavi has joined #openstack-dib | 16:17 | |
| *** yolanda_ is now known as yolanda | 16:41 | |
| andreas-f | Run some test for v2.0.0rc1 today: https://etherpad.openstack.org/p/dib-v2.0.0rc1-testresults | 17:24 |
| *** brault|away has quit IRC | 17:25 | |
| andreas-f | Major points: centos7 / tox does not work because of version mismatch of virtualenv | 17:25 |
| *** brault|away has joined #openstack-dib | 17:25 | |
| greghaynes | andreas-f: where do you get that traceback, in the image? | 17:26 |
| *** hashar has quit IRC | 17:27 | |
| greghaynes | thats confusing because we dont specify a virtualenv dependency | 17:27 |
| *** brault|away has quit IRC | 17:29 | |
| *** andreas-f has quit IRC | 17:30 | |
| *** andreasf has joined #openstack-dib | 17:30 | |
| andreasf | opensuse-minimal failed completely (but I'm not sure if this is a regression or state of the art). | 17:30 |
| greghaynes | andreasf: do you have any thoughts on the virtualenv thing? We shouldnt be specifying that as a requirement anywhere | 17:31 |
| andreasf | gentoo looks broken: this looks like a regression to me... | 17:32 |
| greghaynes | oh! its tox | 17:32 |
| greghaynes | I wonder if tox is just broken on centos | 17:32 |
| *** sudipto has quit IRC | 17:39 | |
| *** sudipto_ has quit IRC | 17:39 | |
| greghaynes | andreasf: btw, whats different from what you ran vs what our CI should be running? | 17:40 |
| greghaynes | because we should be doing roughly that matrix on every patch | 17:40 |
| andreasf | The CI is only running a subset. | 17:41 |
| greghaynes | ah | 17:41 |
| andreasf | And also not the -minimal elements. | 17:41 |
| greghaynes | oh right :( | 17:41 |
| *** brault|away has joined #openstack-dib | 17:42 | |
| andreasf | But I'm really not sure how important some of the failures are... | 17:43 |
| andreasf | Maybe we should ignore them ;-) | 17:43 |
| *** Administrator__ has joined #openstack-dib | 17:43 | |
| andreasf | BTW: it looks that tox / virtualenv is really broken under Centos7: | 17:44 |
| andreasf | virtualenv devel | 17:44 |
| andreasf | git clone https://git.openstack.org/openstack/diskimage-builder | 17:44 |
| greghaynes | yea thats what it seems liek to me, they are using a tox which depends on a newer virtualenv than they provide | 17:44 |
| andreasf | source devel/bin/activate | 17:44 |
| andreasf | pip install tox | 17:44 |
| andreasf | tox | 17:45 |
| andreasf | pkg_resources.DistributionNotFound: virtualenv>=1.11.2 | 17:45 |
| greghaynes | oh you pip installed tox | 17:45 |
| greghaynes | if so youll have to use virtualenv from pip as well | 17:45 |
| andreasf | Ah ok - so my fault! | 17:45 |
| greghaynes | alternatively, install tox from yum | 17:46 |
| andreasf | Yes - sorry for this; Mixed up things here.... | 17:46 |
| *** zhugaoxiao has quit IRC | 17:46 | |
| andreasf | will fix the script and rerun the tox things this night. | 17:48 |
| andreasf | have to leave now - will be back later. | 17:50 |
| greghaynes | ok | 17:51 |
| *** chhavi has quit IRC | 17:56 | |
| *** brault|away has quit IRC | 18:07 | |
| *** zhugaoxiao has joined #openstack-dib | 18:15 | |
| *** Administrator__ has quit IRC | 18:18 | |
| *** openstackgerrit has joined #openstack-dib | 19:00 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: Docker matrix build https://review.openstack.org/414347 | 19:00 |
| *** owalsh has quit IRC | 19:04 | |
| *** brault|away has joined #openstack-dib | 20:07 | |
| *** brault|away has quit IRC | 20:12 | |
| *** owalsh has joined #openstack-dib | 20:13 | |
| jidar | is there a set of exports commonly used to run dib through a jenkins build? I'm getting errors right now: ERROR: unable to cleanly remove /tmp/dib_build.fLpadDwh/built | 20:54 |
| ianw | jidar: umm, no, no really? | 20:58 |
| ianw | jidar: maybe paste the log to paste.openstack.org? | 20:58 |
| jidar | ianw: is that an expression of disapproval or just answering no | 20:58 |
| ianw | sorry, no, there's no special environment variables or anything | 20:58 |
| jidar | ok, sure I can paste it up. let me clean things up a bit | 20:59 |
| ianw | the unable to remove might have something to do with starting a daemon process during the build that's holding files open ... maybe. i think we've seen similar things to that, but we also have a cleanup loop for that | 21:00 |
| greghaynes | yep its almost certainly something being left open | 21:02 |
| jidar | https://gist.github.com/f85940591a1ec781bf494d0d4f5bd6ae | 21:02 |
| greghaynes | jidar: what all are you building in your image? | 21:02 |
| jidar | puppet and ansible support | 21:02 |
| jidar | for heat anyhow | 21:02 |
| jidar | I'm not sure it's getting there tbh | 21:02 |
| greghaynes | oh, are you whitelisting what sudo commands you can run? | 21:04 |
| greghaynes | because it looks like it doesnt have sudo perms, but it should have failed way earlier if that were the case | 21:04 |
| jidar | jenkins does not (at this point) have sudo rights | 21:04 |
| jidar | let me go fix that | 21:04 |
| ianw | jidar: also, can you run dib with "-x"? | 21:05 |
| jidar | sure thing | 21:05 |
| ianw | but yeah, no sudo would probably explain a termination like that, but "-x" will show you what really happened | 21:05 |
| greghaynes | ah ok, so I think it is failing early on but then we cleanup after the fail and that is failing too | 21:05 |
| ianw | yeah, i think the failure to clean up is a bit of red herring as it's not the first failure | 21:06 |
| jidar | yea, it's already moving further along now | 21:07 |
| jidar | guess I'll prolly setup a slave with sudo rights on it so I don't have to put this on the master | 21:08 |
| ianw | re the virtualenv / tox centos blah blah from before ... this is one of the reasons i want us to integrate with infra more, rather than try making our own images. we've sorted all that sort of thing out | 21:11 |
| greghaynes | is there talk of us making our own images? | 21:12 |
| greghaynes | I havent been following along super closely lately | 21:13 |
| jidar | yea, so I put as much into a venv as possible on this | 21:13 |
| jidar | seemed like the right solution | 21:13 |
| greghaynes | im guessing this is the docker matrix stuff? | 21:14 |
| greghaynes | jidar: I think ianw is referring to an earlier convo, we might be crossing the streams :) | 21:15 |
| jidar | yep, I see that now. thanks :) | 21:15 |
| ianw | jidar: installing into a venv is definitely the right way to go, however :) see also http://lists.openstack.org/pipermail/openstack-dev/2017-February/111833.html | 21:15 |
| ianw | greghaynes: yes, that was what i was referring to. i'm skeptical because because infra platforms have a lot of knowledge behind them, and if we're essentially taking plain upstreams and trying to CI on them, we will hit the same problems infra has over and over | 21:17 |
| greghaynes | I havent really read in all the context to have a useful thought | 21:18 |
| greghaynes | does that patch make N containers, then run dib in them? | 21:18 |
| greghaynes | if so, one really good argument is for local dev and testing | 21:19 |
| greghaynes | e.g. I can run the docker thing locally but I dont have N infra images locally | 21:20 |
| greghaynes | I get what youre saying though, we shouldnt be recreating infra | 21:20 |
| greghaynes | honestly | 21:23 |
| greghaynes | this would be pretty neat to do openstackhealth style | 21:23 |
| greghaynes | run it once a day or so, keep a matrix up on a page | 21:24 |
| ianw | it may be better as a generic test runner like that, which can do anything inside the containers, dib being one option | 21:26 |
| greghaynes | ooo | 21:26 |
| greghaynes | yes | 21:27 |
| jidar | this is likely a stupid question but something that's caused me a bit of trouble, if I need another element included is there any way other than editing the elements of some pre-req to make sure it's input? For instance, I need epel included early on (so that the ansible/puppet stuff is included properly and yum -y install doesn't silently fail grrr) is there no way other than editing one of those | 21:28 |
| jidar | elements file to do so? | 21:28 |
| greghaynes | jidar: if youre having to do that then it sounds like the epel element isnt properly written | 21:28 |
| greghaynes | i.e. its a bug | 21:28 |
| jidar | oh really? the tripleO guys were basically saying we don't include it by default anymore (for reasons) | 21:29 |
| greghaynes | epel should add repos before any of the yum stuff runs | 21:29 |
| greghaynes | Yea, I dunno why, but the idea is all repo setup stuff happens in pre-install.d, all install stuff happens in install.d afterwards | 21:29 |
| jidar | I threw it in here: elements/centos7/element-deps, +epel at the end and everything works | 21:29 |
| greghaynes | oh then just add epel to the command line | 21:29 |
| greghaynes | its equivalent | 21:29 |
| ianw | yeah, we don't ship epel built-in, as you say it can confict with RDO and triplo and stuff | 21:30 |
| jidar | so how do I include it at the command line? I'm not seeing the option in disk-image-builder -h | 21:30 |
| jidar | er, disk-image-create rather | 21:30 |
| greghaynes | just add "epel" on to the end of the command | 21:30 |
| greghaynes | there should be an "elements..." in the -h | 21:31 |
| jidar | oh, because it's an element | 21:31 |
| greghaynes | yep | 21:31 |
| jidar | hahah, yep thanks! | 21:31 |
| andreasf | greg: yes it's the thing with the matrix build what I run this morning - but there seams to be a misunderstanding: I (also) do not want to recreate the infra. | 21:32 |
| andreasf | IMHO currently the knowledge is spread. | 21:33 |
| andreasf | Something is in the dib repo - and others are in other repos. | 21:33 |
| greghaynes | are you referring to the stuff in the jobs definitions or something else? | 21:33 |
| andreasf | If you check something in you can never be sure that it really works in all CI tests | 21:33 |
| andreasf | Not only the job definitions themselfs. | 21:34 |
| andreasf | Its also e.g. about dependencies. | 21:34 |
| andreasf | Why should a puppet module know which prerequisites are needed to run dib? | 21:34 |
| andreasf | This is IMHO something dib should provide. | 21:35 |
| andreasf | So it's the other way round: | 21:35 |
| greghaynes | right, so for that theres two larger solutions that infra has developed / are developing. For the deps theres bindep which I think pabelanger made a patch to add | 21:35 |
| greghaynes | and for the job definitions they are working on a new version of the CI tool which allows job definitions to exist in the repos themselves | 21:35 |
| andreasf | dib should define the dependecies and the CI can use them. | 21:36 |
| greghaynes | yep, thats bindep, let me see if I can find the patch | 21:36 |
| greghaynes | https://review.openstack.org/#/c/391931/ | 21:37 |
| greghaynes | the one comment I had is valid in that we should have our script able to run bindep for a user, but the idea is still what you say - we define the dependencies and then consumers read them | 21:38 |
| andreasf | Good patch - missed it. | 21:38 |
| greghaynes | for the job definitions, zuulv3 is the larger fix for all of openstack for that. For now thats why we made our ./run_functests script so that could be the minimal interface to get to a point where our test definitions are in tree | 21:39 |
| greghaynes | andreasf: did you see the comment about your matrix thing being useful as a more general tool? | 21:40 |
| greghaynes | curious what you think of that | 21:40 |
| andreasf | Yes - maybe you saw my spec about testing? | 21:40 |
| andreasf | I had some toughts about this. | 21:41 |
| greghaynes | https://etherpad.openstack.org/p/DIBMaxtrixDockerBuild or? | 21:41 |
| andreasf | The spec itself is about DIB - but with this idea in mind it can be extended? | 21:41 |
| andreasf | The link you mentioned was one of the first tries. | 21:42 |
| andreasf | The spec proposal: https://review.openstack.org/#/c/429255/ | 21:42 |
| greghaynes | aha, so reproducability is your main issue it seems like? | 21:43 |
| andreasf | There are some aspects: I really would like to be sure that a patch works *before* I commit it... | 21:44 |
| greghaynes | ah, I gave up on that notion when I started workin on openstack :) | 21:44 |
| greghaynes | this is a somewhat common issue though | 21:44 |
| andreasf | ...also we have currently as far as I know no docker test. | 21:45 |
| andreasf | The idea behind the spec is to have a test-runner that can execute test (code) on local host as well as on a (pre-build) docker or vm. | 21:46 |
| andreasf | And then we are back to the beginning: there is a need to easily setup a docker / vm image for DIB testing. | 21:47 |
| greghaynes | ok, so the big concern is that in our CI we really do need to use the provided images. They have a fair amount of modifications to speed up builds and us using our own images is going to make it harder for other openstack folks to debug our CI | 21:48 |
| greghaynes | locally though youre rigth that its not really practical to do something near what a full CI run is | 21:48 |
| greghaynes | additionally though, your deal provides us a lot more coverage than we currently get in CI (because it builds on a lot more distros than we have infra images for) | 21:49 |
| andreasf | Yes - don't want to touch the existing test-infrastructure, but maybe move some 'hard-coded' calls and dependencies (back) to dib. | 21:49 |
| andreasf | There is a long discussion about this coverage and support thing with Ianw in some of these patches. | 21:50 |
| greghaynes | so heres what i wonder - would it be possible to make the matrix runner as an external tool to dib, and then use of it can just be 'distro-matrix-run diskimage-builder/test/run_functests.sh ? | 21:50 |
| greghaynes | basically a generic thing to run a test suite in a matrix of different distros | 21:51 |
| andreasf | Yes - that's the idea behind. | 21:51 |
| greghaynes | ianw: ^ thoughts? | 21:52 |
| greghaynes | on a totally unrelated note, if it isnt brutally obvious yet I have a bit less time to dib nowdays (job situation changed) | 21:53 |
| greghaynes | so.. trying to help where I can but glad to see you all making good progress with me not being around :) | 21:54 |
| andreasf | Yes - saw this and already missed you. | 21:54 |
| greghaynes | haha, I just need to find a way to give some dib talks then I have a good excuse to spend time on it | 21:54 |
| andreasf | Good point: are you goring for a talk in Boston? | 21:56 |
| greghaynes | nope, I was thinking of submitting to some linuxcon conferences though | 21:56 |
| greghaynes | or foss* conferences | 21:57 |
| andreasf | Did you already talks about dib during some conferences? Some days ago I tried to find some - but only find one or two somewhat old. | 21:59 |
| greghaynes | I did one with trhe trove folks at the last summit | 21:59 |
| greghaynes | in barcelona | 21:59 |
| greghaynes | the old ones are probably from the early tripleo days im guessing? | 21:59 |
| andreasf | Will have a look for it. | 22:00 |
| andreasf | Yes - with Robert Collins. | 22:01 |
| greghaynes | right, yep that was a long time ago | 22:01 |
| andreasf | (BTW: centos7/tox still fails; now with 'ERROR: py35: InterpreterNotFound: python3.5' ;-) | 22:01 |
| greghaynes | is py35 pacakged on centos7? | 22:02 |
| greghaynes | if not you may need to run with --skip-missing-interpreters | 22:03 |
| andreasf | Strange; looks that there is no python3 shipped with centos7??? | 22:06 |
| greghaynes | I believe it | 22:06 |
| andreasf | So - I'll go to bed. Good night. | 22:13 |
| greghaynes | gn | 22:13 |
| ianw | sorry, school run | 22:16 |
| greghaynes | congrats on that :) | 22:16 |
| greghaynes | is it amazing? | 22:16 |
| ianw | it's just all over easier because i don't have to do daycare dropoff then school dropoff for the older one :) | 22:17 |
| greghaynes | ah | 22:17 |
| greghaynes | I cant wait until I can drop off for free | 22:17 |
| ianw | heh, don't wish your time away :) but yeah i get it too | 22:18 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!