| marun | mordred: awesome | 00:00 |
|---|---|---|
| *** pabelanger_ has quit IRC | 00:00 | |
| sdague | I'll just make it take a list of log files, so however that can be given, it will provide a report | 00:00 |
| *** pabelanger has joined #openstack-infra | 00:00 | |
| clarkb | mordred: bodepd: we could maybe pick a couple days where we all work on upgrading together and use puppet agent --test --noop liberally to make sure we won't majorly break something | 00:01 |
| mordred | jeblair: ooh... I just had an idea... | 00:01 |
| sdague | ok, now I have 2 plans. Which is good. Tomorrow oslo.* in git in devstack, then stack trace enforcer | 00:01 |
| clarkb | I think the trickiest bit is going to be not breaking a bunch of stuff and putting a change freeze on anything else for a short period so that we can use --noop to show us diffs would be useful | 00:01 |
| mordred | jeblair: and I think you're the only person other than ttx with his head enough in the version madness to tell me if I'm stupid... | 00:01 |
| jeblair | sdague: i like your plans | 00:01 |
| mordred | jeblair: in the version calculation code ... we current have the "commits since last tag" calculation... | 00:02 |
| mordred | jeblair: what if that, instead, was "commits since last release tag" | 00:02 |
| mordred | jeblair: since we have a definition of what a final/release version is | 00:02 |
| sdague | I just feel bad that my other awesome plan, the automatic recheck finder with elastic search with clarkb is back burnered. But I guess that's what the quiet time at the beginning of icehouse is for | 00:02 |
| mordred | so that if you make a milestone release somewhere in the middle | 00:02 |
| clarkb | sdague: thats ok, I have used the quiet time to make elasticsearch not suck so much | 00:03 |
| mordred | it doesn't really do anything to your running tally of commits | 00:03 |
| clarkb | sdague: I really think it is a lot more stable now | 00:03 |
| sdague | cool :) | 00:03 |
| jeblair | mordred: that sounds like it might be too specific for pbr... | 00:03 |
| jeblair | unless it were a setup.py option | 00:03 |
| mordred | jeblair: yeah. I agree. and actually saying it out loud it doesn't solve people's complaint either | 00:03 |
| jeblair | (but that feels weird) | 00:04 |
| mordred | yeah | 00:04 |
| mordred | nevermind | 00:04 |
| clarkb | mordred: should we ever tag things with non version tags upstream? | 00:04 |
| *** dina_belova has quit IRC | 00:04 | |
| mordred | clarkb: milestones | 00:04 |
| clarkb | mordred: milestones are versions | 00:04 |
| mordred | clarkb: but they are prerelease versions | 00:05 |
| clarkb | mordred: at least I thought ttx was going to do them like 2013.2.a1 2013.2.a2 and so on | 00:05 |
| mordred | clarkb: 2013.2.b2 | 00:05 |
| *** jjmb has joined #openstack-infra | 00:05 | |
| *** rfolco has joined #openstack-infra | 00:05 | |
| clarkb | oh I see you specifically asked about release version tags | 00:05 |
| mordred | yah. so - it continues to trip people up that we release 2013.2.a153.g23ab34c and then 2013.2.b2 and then 2013.2.a1.g6ece34 | 00:06 |
| mordred | we explain that it's by design and we make no guarantees about one version to the next | 00:06 |
| *** jjmb has quit IRC | 00:06 | |
| mordred | and, as a project, we don't ... | 00:06 |
| mordred | except ... | 00:06 |
| clarkb | should the version always be prefixed by the tag version? | 00:06 |
| mordred | that sentence did not compute to me | 00:07 |
| clarkb | 2013.2.a152.foo -> 2013.2.b2 -> 2013.2.b2.foo.bar | 00:07 |
| bodepd | mordred: the best thing to do it probably to just upgrade to apache | 00:07 |
| clarkb | mordred: b2 shouldn't go away on the next commit | 00:07 |
| sdague | jeblair: https://review.openstack.org/#/c/39097/ when you get a down minute. | 00:07 |
| bodepd | mordred: you guys have so many parts of the code that use it though... | 00:07 |
| sdague | and I'm off for the evening, see folks in the morning | 00:07 |
| bodepd | mordred: it would require a ton of testing | 00:07 |
| mordred | bodepd: it would not be a small patch | 00:07 |
| clarkb | sdague: good night | 00:07 |
| mordred | ++ | 00:07 |
| mordred | clarkb: so - the thing is, is that as soon as you do that, after the first milestone | 00:08 |
| clarkb | bodepd: ya that is why I was consideringa piecemeal upgrade where we install the latest version of the module as new-apache or some such | 00:08 |
| jeblair | sdague: thanks, and good night! | 00:08 |
| mordred | you'd be doing post-release versioning based on the milestone | 00:08 |
| bodepd | clarkb: so very ugly... | 00:08 |
| mordred | there's no way to know | 00:08 |
| clarkb | bodepd: then you can do one service at a time and have a final change that switch apache to the latest version and s/new-apache/apache/ but pretent I used underscores because puppet doesn't like dashes | 00:08 |
| clarkb | bodepd: indeed | 00:09 |
| clarkb | mordred: you'd only refer to the last tag so after you release 2013.2 you have 2013.2.X | 00:09 |
| mordred | clarkb: right | 00:10 |
| mordred | clarkb: that's post-release versioning | 00:10 |
| mordred | and is how we do libraries | 00:10 |
| mordred | we don't refer to tags in the pre-release versioning, unless it's an exact match | 00:10 |
| bodepd | clarkb: I could also go the cowardly way and just use a file instead of the apache::vhost def | 00:10 |
| mordred | it's the version in the config file + the revcount and sha | 00:10 |
| bodepd | clarkb: it's a but tangled to fix for reals atm | 00:11 |
| clarkb | bodepd: ya :( | 00:11 |
| clarkb | mordred: So the pre and release versioning is different? | 00:11 |
| bodepd | I did wrote a very hopeful rec for an upstream engineer recently where part of the job is upstream work on puppet modules for our CI | 00:11 |
| clarkb | I thought it was pretty naive and was basically if tag on this commit use tag. If not tag on this commit use tag + number of commits + short sha like thing | 00:12 |
| clarkb | I think you use git describe to figure it out | 00:12 |
| mordred | clarkb: for the servers, the logic is "if this version is tagged, the version is the tag, else, use setup.cfg:version + a$revcount + g$gitsha" | 00:12 |
| clarkb | bodepd: ooh | 00:12 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Support multiple triggers https://review.openstack.org/39186 | 00:12 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Add a timer trigger https://review.openstack.org/39187 | 00:12 |
| clarkb | mordred: gotcha | 00:12 |
| bodepd | clarkb: it's all very hopeful and hand-wavey | 00:13 |
| mordred | clarkb: the thing based on git describe is what we use for post-release versioning, which is what you get if you do not specify a version in setup.cfg | 00:13 |
| *** jjmb has joined #openstack-infra | 00:14 | |
| clarkb | mordred: jeblair: fungi: I am going to approve a couple low impact changes that don't have two +2s | 00:15 |
| locke1051 | so... why is the run_mirror.py script in jeepyb? | 00:15 |
| locke1051 | seems like it solves a different problem than jeepyb... | 00:15 |
| clarkb | https://review.openstack.org/#/c/39382 and https://review.openstack.org/#/c/39160/ | 00:15 |
| mordred | clarkb: great | 00:15 |
| clarkb | locke1051: because it was based on the project list which is really what jeepyb seems to revolve around | 00:16 |
| mordred | locke1051: it grew out of "these are things that read projects.yaml" | 00:16 |
| clarkb | locke1051: that is not longer true | 00:16 |
| mordred | locke1051: we have a todo list item to split run mirror into its own thing | 00:16 |
| clarkb | s/that/the mirror being based on the project list/ | 00:16 |
| mordred | locke1051: lifeless wants that too | 00:16 |
| locke1051 | cool | 00:16 |
| lifeless | oh so very badly | 00:16 |
| locke1051 | i was just going to wget it from github | 00:16 |
| mordred | heh | 00:17 |
| mordred | we'll be making it pip installable soon | 00:17 |
| mordred | by itself | 00:17 |
| locke1051 | sweet | 00:18 |
| locke1051 | need any help there? | 00:18 |
| *** krtaylor has joined #openstack-infra | 00:18 | |
| clarkb | I would gladly help and accept a pair of patches that 1. made the mirror script its own project and 2. removed it from jeepyb and 3. updated our jobs to use the new project | 00:19 |
| clarkb | and by pair apparently I really meant three | 00:19 |
| openstackgerrit | A change was merged to openstack-infra/config: enable access for openstack-ci-core https://review.openstack.org/39160 | 00:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Publications use ZUUL_REF instead of ZUUL_BRANCH. https://review.openstack.org/39382 | 00:20 |
| jeblair | it could be done by adding a new project to projects.yaml with the current jeepyb repo set as upstream | 00:20 |
| locke1051 | hokay... so the making new project thing would be an update to the openstack-infra/config stuff i would guess | 00:20 |
| jeblair | and then merge some changes to the new project that remove all the extra stuff | 00:20 |
| locke1051 | thats a good idea | 00:20 |
| jeblair | (for some things we'd go to the trouble of filter-branching, but i kind of think for jeepyb we could just copy and delete) | 00:20 |
| jeblair | it's so small | 00:20 |
| clarkb | ++ | 00:20 |
| locke1051 | yeah not that big | 00:21 |
| jeblair | locke1051: yep, that should just be changes to openstack-infra/config to get all that set up | 00:21 |
| jeblair | locke1051: if you want to do that, that would be great :) | 00:21 |
| jeblair | mordred, lifeless: i assume no one else is in progress on that right now? | 00:22 |
| *** yaguang has joined #openstack-infra | 00:22 | |
| mordred | haven't gotten to it myself yet | 00:22 |
| locke1051 | np, should be a good learning experience | 00:22 |
| lifeless | jeblair: I would have done it but been on leave and then this tech@work thing | 00:23 |
| *** mberwanger has joined #openstack-infra | 00:24 | |
| jeblair | locke1051: http://ci.openstack.org/ should have all the info (and if it doesn't let us know/submit a patch) | 00:24 |
| jeblair | locke1051: perhaps strangely, the 'stackforge' howto may be the best reference as it's sort of a walkthrough for most of what needs to be done (though the new project should go in openstack-infra/) | 00:25 |
| *** dkliban_afk has joined #openstack-infra | 00:25 | |
| mordred | jeblair: hrm. we should fix that ... | 00:26 |
| mordred | you konw - that and the other 100 things we need to do... | 00:26 |
| jeblair | mordred: yeah, well, we used to add 1 openstack project every 6 months; things have changed. | 00:26 |
| mordred | jeblair: remember those days? | 00:28 |
| clarkb | manage_projects revolutionized the project creation business | 00:29 |
| locke1051 | jeblair: ok cool, was looking at previous commits to config for examples as well | 00:29 |
| clarkb | we need that oprah gif. you get a project you get a project projrcts for everyone | 00:29 |
| jeblair | locke1051: great. they're much more consise. :) | 00:30 |
| *** pentameter has quit IRC | 00:31 | |
| *** ^d has quit IRC | 00:36 | |
| *** koolhead17 has quit IRC | 00:41 | |
| locke1051 | so... openstack-infra/run-mirror fine for a name? | 00:41 |
| locke1051 | can't think of anything clever atm | 00:41 |
| jeblair | locke1051: pypi-mirror? | 00:42 |
| jeblair | mordred: ^ | 00:42 |
| clarkb | pypi or cheeseshop | 00:42 |
| mordred | jeblair: I could go with pypi-mirror or run-mirror | 00:43 |
| clarkb | -mirror | 00:43 |
| *** mberwanger has quit IRC | 00:43 | |
| *** zaro0508 has joined #openstack-infra | 00:43 | |
| mordred | less interested in cheeseshop mirrir | 00:43 |
| locke1051 | pypi-mirror is more descriptive | 00:43 |
| *** mberwanger has joined #openstack-infra | 00:45 | |
| marun | mordred: is it easy to create a local openstack pypi mirror? It would be really handy for devstack use. | 00:45 |
| mordred | marun: yup! we have a script - locke1051 ^^ is working on splitting it our right now | 00:46 |
| marun | mordred: awesome! | 00:46 |
| marun | bodepd: ^^ | 00:47 |
| marun | mordred: i've done some work on puppet support for configuring tempest, but actual installation was problematic due to pypi installation flakiness. | 00:48 |
| mordred | marun: check this out: https://review.openstack.org/#/c/38871/ | 00:49 |
| mordred | bodepd: you too | 00:49 |
| mordred | marun, bodepd: it's a set of diskimage-builder elements that you can use to create a qcow2 image that is exactly what devstack-gate nodes look like | 00:49 |
| mordred | and - it runs puppet inside too :) | 00:49 |
| marun | mordred: sweet! | 00:50 |
| *** yaguang has quit IRC | 00:50 | |
| nati_ueno | cool! | 00:51 |
| locke1051 | so this is a little odd... there is already a gerrit ACL file for pypi-mirror | 00:53 |
| locke1051 | https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/gerrit/acls/openstack-infra/pypi-mirror.config | 00:53 |
| jeblair | locke1051: before even the ancient history related to you earlier about it becoming part of jeepyb; it used to be its own project. :| | 00:53 |
| locke1051 | ah | 00:54 |
| locke1051 | planning ahead | 00:54 |
| locke1051 | i just copied over the jeepyb one, had a few changes | 00:54 |
| jeblair | so in fact, that project still exists. | 00:54 |
| locke1051 | mmm | 00:55 |
| jeblair | though it is not on github. | 00:55 |
| locke1051 | is manage-projects going to blow up... | 00:55 |
| *** toddmorey has joined #openstack-infra | 00:56 | |
| jeblair | or maybe that was a different script? | 00:56 |
| jeblair | ah, yes, it seems to be a different project. | 00:56 |
| locke1051 | yeah its still around... looking at it | 00:57 |
| jeblair | https://review.openstack.org/gitweb?p=openstack-infra/pypi-mirror.git;a=tree | 00:57 |
| locke1051 | looks like a fork of the zope build out one | 00:57 |
| jeblair | mordred: any thoughts on how you'd like to handle this? | 00:57 |
| mordred | oh - that. yeah, it's terrible | 00:57 |
| mordred | blow it away | 00:57 |
| clarkb | iirc that was mordred and shrews first attempt | 00:57 |
| mordred | with prejudice | 00:57 |
| locke1051 | lol | 00:57 |
| jeblair | mordred: great. how would you like to blow it away? | 00:58 |
| mordred | jeblair: force push? | 00:58 |
| jeblair | it has gerrit changes | 00:58 |
| *** nati_ueno has quit IRC | 00:58 | |
| mordred | jeblair: does that matter? | 00:58 |
| mordred | or - we could just swap the name | 00:58 |
| jeblair | mordred: you may want to use the gerrit delete-project procedure, and go ahead and delete the changes too. | 00:58 |
| mordred | mirror-pypi | 00:58 |
| *** mberwanger has quit IRC | 00:59 | |
| jeblair | i would rather not have both 'mirror-pypi' and 'pypi-mirror' as projects in our system. | 00:59 |
| mordred | fair | 00:59 |
| locke1051 | yeah that might get confusing | 00:59 |
| jeblair | mordred: my recommendation is since you don't care about the history, delete the project and the associated db records | 00:59 |
| mordred | jeblair: don't we need a restart for that though? | 00:59 |
| jeblair | mordred: yes, we need a short outage. | 00:59 |
| clarkb | mordred: to completely change the subject http://arstechnica.com/business/2013/07/rideshare-drivers-given-citizen-arrest-by-sf-international-airport-officials/ | 00:59 |
| *** dina_belova has joined #openstack-infra | 01:00 | |
| mordred | jeblair: well, we could graft the new branch history in using your keystone trick | 01:00 |
| mordred | jeblair: which would not require downtime | 01:00 |
| mordred | but the changes in gerrit would be legit | 01:01 |
| jeblair | mordred: i don't think the zope mirror script history belongs in the history of this repo | 01:01 |
| mordred | yeah ok | 01:01 |
| clarkb | I agree | 01:01 |
| jeblair | mordred: i think lifeless wants some repos renamed. you might do those at the same time. | 01:02 |
| mordred | jeblair: agree | 01:02 |
| mordred | ok. I'll schedule a downtime | 01:02 |
| mordred | my brain cannot comprehend doing that right now | 01:02 |
| *** woodspa has joined #openstack-infra | 01:02 | |
| jeblair | locke1051: sorry this got complicated. :( we'll schedule gerrit downtime (probably on a friday, weekend, or wee hours) to delete the old project | 01:03 |
| *** mberwanger has joined #openstack-infra | 01:03 | |
| jeblair | locke1051: then things should work as originally expected; you can go ahead and write up a change that imports it as if it's | 01:04 |
| jeblair | er | 01:04 |
| jeblair | imports jeepyb into a new pypi-mirror project as we discussed | 01:04 |
| locke1051 | jeblair: yep still working on it | 01:05 |
| *** dina_belova has quit IRC | 01:05 | |
| locke1051 | actually i think i'm about done | 01:05 |
| *** nati_ueno has joined #openstack-infra | 01:05 | |
| *** changbl has joined #openstack-infra | 01:15 | |
| locke1051 | grr... environment issues | 01:15 |
| openstackgerrit | Maru Newby proposed a change to openstack-dev/pbr: Add support for classmethod console scripts. https://review.openstack.org/39398 | 01:23 |
| marun | mordred: ^^ | 01:24 |
| openstackgerrit | Mathew Odden proposed a change to openstack-infra/config: Create new pypi-mirror project for run_mirror.py https://review.openstack.org/39399 | 01:26 |
| mordred | w00t | 01:26 |
| locke1051 | man that took way too much effort... network died on my dev environment | 01:26 |
| locke1051 | hack to get creative | 01:27 |
| locke1051 | had* | 01:27 |
| mordred | nice | 01:27 |
| mordred | marun: from %(module_name)s import %(import_target)s works when import_target is Class.method ? | 01:27 |
| marun | import_target is attr[0] | 01:27 |
| mordred | marun: nevermind - I can't read | 01:27 |
| marun | :) | 01:28 |
| marun | i added tests, which validates both use-cases | 01:28 |
| marun | added is a strong word. | 01:28 |
| marun | updated | 01:28 |
| *** locke1051 is now known as locke105 | 01:28 | |
| mordred | marun: :) patch looks great. fungi or clarkb could one of you take a quick peek? the patch is for a thing that's breaking neutron currently | 01:29 |
| fungi | lookin | 01:37 |
| *** nati_ueno has quit IRC | 01:41 | |
| *** yaguang has joined #openstack-infra | 01:41 | |
| *** marun has quit IRC | 01:45 | |
| *** afazekas has quit IRC | 01:53 | |
| *** krtaylor has quit IRC | 01:54 | |
| *** nati_ueno has joined #openstack-infra | 01:57 | |
| *** marun has joined #openstack-infra | 01:59 | |
| *** dina_belova has joined #openstack-infra | 02:00 | |
| *** danger_fo is now known as danger_fo_away | 02:03 | |
| *** mberwanger has quit IRC | 02:04 | |
| *** mberwanger has joined #openstack-infra | 02:04 | |
| *** dina_belova has quit IRC | 02:05 | |
| *** afazekas has joined #openstack-infra | 02:05 | |
| *** yaguang has quit IRC | 02:07 | |
| locke105 | so there is a pypi.openstack.org, but i see pypi.slave.openstack.org and also mirror27 and mirror26 | 02:08 |
| locke105 | how are those related... | 02:08 |
| *** mberwanger has quit IRC | 02:09 | |
| *** yaguang has joined #openstack-infra | 02:11 | |
| *** mriedem has joined #openstack-infra | 02:14 | |
| clarkb | locke105: pypi.openstack.org is the actual mirrir. mirror27.slave and mirror26.slave run the mirror script to create a pip cache that is copied to pypi.openstack.org | 02:15 |
| clarkb | locke105: this is done to make sure that if there are different deps across python versions we capture them all | 02:16 |
| locke105 | ok | 02:16 |
| clarkb | fungi: ^ we should add a python33 mirror slave | 02:16 |
| clarkb | locke105: pypi.slave.openstack.org is a poorly named slave that we use to push packages to pypi and maven repos | 02:16 |
| clarkb | it is slightly more trusted than the other slaves | 02:16 |
| fungi | clarkb: that's a very good point. i'll whip one up tomorrow | 02:16 |
| locke105 | whats pypi.slave.openstack.org | 02:16 |
| locke105 | pypi publisher job host? | 02:16 |
| clarkb | locke105: basically. | 02:17 |
| locke105 | ok makes sense now | 02:17 |
| clarkb | it does not run arbitrary code. instead it grabs the sdists build elsewhere and uses curl to shove them into pypi | 02:17 |
| fungi | in an effort not to expose our repository upload credentials in a risky fashion | 02:18 |
| *** changbl has quit IRC | 02:19 | |
| *** toddmorey has quit IRC | 02:19 | |
| locke105 | thats smart | 02:21 |
| *** changbl has joined #openstack-infra | 02:22 | |
| dstufft | What fun is it if you're not exposing your upload creds in a risky fashion ;) | 02:22 |
| *** vijendar has quit IRC | 02:23 | |
| marun | i'm all for consistency, but gating on commit message format without having git-review prevent submission on violations seems like a waste of ci resources | 02:23 |
| clarkb | marun: if you run `tox` prior to pushing to gerrit ti should catch that... | 02:24 |
| marun | i did, and it didn't | 02:24 |
| marun | wait | 02:24 |
| *** vijendar has joined #openstack-infra | 02:24 | |
| marun | so i have to commit before i run tox? | 02:24 |
| clarkb | marun: no | 02:24 |
| clarkb | but if you want it to catch commit message issues yes | 02:24 |
| marun | *sigh* | 02:24 |
| clarkb | (disclaimer I and the infra team have nothing to do with teh commit message stuff) | 02:25 |
| marun | fair enough | 02:25 |
| clarkb | that is all project side and happening with hacking iirc | 02:25 |
| marun | someone clearly has too much free time | 02:25 |
| marun | worrying about pedantic issues like that when there are so many actual quality problems in the projects themselves….well, go figure | 02:26 |
| *** dkliban_afk has quit IRC | 02:26 | |
| * marun goes to remove the period from his commit message summary | 02:26 | |
| clarkb | marun: you should see our commit messages >_> it is definitely not a bikeshed the infra team paints | 02:26 |
| marun | heh | 02:27 |
| marun | clarkb: i marvel at the the inclusiveness of openstack, but it's definitely on the extreme side of it | 02:27 |
| marun | clarkb: i was looking into switching to mutt recently and was surprised at the contrast | 02:28 |
| mordred | clarkb, jeblair we do not, in fact, have a delete a project procedure in the docs | 02:28 |
| *** adalbas has quit IRC | 02:28 | |
| marun | clarkb: the maintainers refuse to accept patches for pretty fundamental stuff, like, say, a sidebar | 02:28 |
| *** rfolco has quit IRC | 02:28 | |
| marun | clarkb: openstack on the other hand, seems to accept anything from anybody, no questions asked. generalizing, but only a little | 02:28 |
| clarkb | mordred: we don't however fungi knows what to do as we did it for the puplications | 02:29 |
| mordred | clarkb: we should write it down | 02:29 |
| mordred | marun: yes. we will. as long as it's styled right :) | 02:30 |
| marun | lol | 02:30 |
| locke105 | i'd say thats pretty accurate, but only as long as you can get someone to actually review it | 02:31 |
| clarkb | we aren't picky but I think we are relatively thorough in infra land. jeblair has imparted good habits on the rest of us | 02:31 |
| clarkb | and swift seems to be picky | 02:32 |
| marun | swift is really picky, to their credit | 02:32 |
| clarkb | tl;dr it depends on the project | 02:32 |
| marun | thought the quality of the codebase isn't all that, at least they keep it stable | 02:32 |
| fungi | mordred: clarkb: i simply patterned it after the project rename queries, but s/update/delete/ basically | 02:32 |
| fungi | and rm the local repo and clones, delete the mirror of it on github... | 02:33 |
| *** prad has joined #openstack-infra | 02:33 | |
| *** jrex_laptop has joined #openstack-infra | 02:34 | |
| fungi | also empty any orphaned groups, if there are any, and set them owned by administrators and not visible | 02:34 |
| fungi | just for cleanliness | 02:34 |
| fungi | on the topic of not exposing sensitive data, i take this as a sign that a measurable proportion of the internet community was surprised to learn that commercial software vendors have little incentive to protect users from domestic spying: https://pbs.twimg.com/media/BQYA_qWCEAIoUFT.png | 02:36 |
| *** dguitarbite has joined #openstack-infra | 02:37 | |
| * locke105 is not surprised at all. | 02:37 | |
| locke105 | security through obscurity | 02:38 |
| fungi | quite a lot of people knew already, but i do find that graph pretty neat | 02:38 |
| *** prad has quit IRC | 02:40 | |
| *** woodspa has quit IRC | 02:43 | |
| *** prad has joined #openstack-infra | 02:43 | |
| *** vipul is now known as vipul-away | 02:46 | |
| mordred | if any of you are on the west coast... | 02:47 |
| clarkb | mordred: ? | 02:47 |
| mordred | my friends, the Bengsons, who are the ones who wrote the opera that I'm working on in SF | 02:47 |
| mordred | have a song on so you think you can dance tonight | 02:47 |
| openstackgerrit | Maru Newby proposed a change to openstack-dev/pbr: Add support for classmethod console scripts https://review.openstack.org/39398 | 02:52 |
| *** vipul-away is now known as vipul | 02:57 | |
| *** melwitt has quit IRC | 02:59 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Fix .mailmap file search location https://review.openstack.org/37562 | 03:00 |
| *** dina_belova has joined #openstack-infra | 03:01 | |
| *** jjmb has quit IRC | 03:04 | |
| *** dina_belova has quit IRC | 03:05 | |
| *** prad has quit IRC | 03:09 | |
| jeblair | mordred: was it in the first 13 minutes? | 03:13 |
| mordred | jeblair: I don't know | 03:13 |
| mordred | I have a youtube clip link though | 03:13 |
| jeblair | mordred: i have also asked mythtv to record it (pvr's are the best -- they watch tv so you don't have to!) | 03:15 |
| mordred | jeblair: nice | 03:15 |
| *** jrex_laptop has quit IRC | 03:20 | |
| *** toddmorey has joined #openstack-infra | 03:20 | |
| *** SergeyLukjanov has joined #openstack-infra | 03:20 | |
| *** prad has joined #openstack-infra | 03:21 | |
| *** prad has quit IRC | 03:21 | |
| *** mriedem has quit IRC | 03:23 | |
| *** toddmorey has quit IRC | 03:27 | |
| *** mberwanger has joined #openstack-infra | 03:28 | |
| *** toddmorey has joined #openstack-infra | 03:31 | |
| *** dguitarbite has quit IRC | 03:32 | |
| *** yaguang has quit IRC | 03:36 | |
| *** yaguang has joined #openstack-infra | 03:48 | |
| zaro | anybody tried using nova image-create lately | 03:52 |
| zaro | ? doesn't seem to work for me. | 03:52 |
| *** krtaylor has joined #openstack-infra | 03:53 | |
| *** vijendar has quit IRC | 03:53 | |
| openstackgerrit | A change was merged to openstack-dev/pbr: Add support for classmethod console scripts https://review.openstack.org/39398 | 04:01 |
| *** dina_belova has joined #openstack-infra | 04:01 | |
| *** ^demon has joined #openstack-infra | 04:06 | |
| *** dina_belova has quit IRC | 04:06 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Replace setuptools_git with a smarter approach https://review.openstack.org/30522 | 04:12 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-dev/hacking: Extract configs-related code to the configs helper https://review.openstack.org/39143 | 04:19 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-dev/hacking: Import exceptions list is now configurable https://review.openstack.org/39140 | 04:19 |
| *** mberwanger has quit IRC | 04:26 | |
| *** marun has quit IRC | 04:27 | |
| *** dkliban_afk has joined #openstack-infra | 04:27 | |
| *** michchap has quit IRC | 04:31 | |
| *** michchap has joined #openstack-infra | 04:31 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-dev/hacking: Add py3k compatibility for config parser https://review.openstack.org/39412 | 04:31 |
| *** jjmb has joined #openstack-infra | 04:32 | |
| *** yaguang has quit IRC | 04:32 | |
| *** boris-42 has joined #openstack-infra | 04:37 | |
| *** boris-42 has quit IRC | 04:37 | |
| *** SergeyLukjanov has quit IRC | 04:45 | |
| *** nayward has joined #openstack-infra | 04:49 | |
| *** yaguang has joined #openstack-infra | 04:52 | |
| *** dkliban_afk has quit IRC | 04:57 | |
| *** dina_belova has joined #openstack-infra | 05:02 | |
| *** nayward has quit IRC | 05:03 | |
| *** dina_belova has quit IRC | 05:07 | |
| *** fungi has quit IRC | 05:08 | |
| *** nayward has joined #openstack-infra | 05:10 | |
| *** reed has quit IRC | 05:11 | |
| *** jaypipes has quit IRC | 05:19 | |
| *** toddmorey has quit IRC | 05:25 | |
| *** jjmb has quit IRC | 05:26 | |
| *** nicedice has quit IRC | 05:46 | |
| *** fungi has joined #openstack-infra | 05:48 | |
| *** dguitarbite has joined #openstack-infra | 05:52 | |
| *** psedlak has joined #openstack-infra | 05:55 | |
| *** toddmorey has joined #openstack-infra | 06:00 | |
| *** dina_belova has joined #openstack-infra | 06:03 | |
| *** mikal has quit IRC | 06:05 | |
| *** psedlak_ has joined #openstack-infra | 06:06 | |
| *** mikal has joined #openstack-infra | 06:06 | |
| *** dina_belova has quit IRC | 06:07 | |
| *** psedlak has quit IRC | 06:10 | |
| *** dina_belova has joined #openstack-infra | 06:13 | |
| *** toddmorey has quit IRC | 06:16 | |
| *** jhesketh has quit IRC | 06:21 | |
| *** toddmorey has joined #openstack-infra | 06:23 | |
| *** yaguang has quit IRC | 06:32 | |
| *** jhesketh has joined #openstack-infra | 06:33 | |
| *** dina_belova has quit IRC | 06:35 | |
| *** jjmb has joined #openstack-infra | 06:37 | |
| *** toddmorey has quit IRC | 06:38 | |
| *** pcrews has joined #openstack-infra | 06:44 | |
| *** yaguang has joined #openstack-infra | 06:45 | |
| *** jjmb has quit IRC | 06:51 | |
| *** GheRivero has quit IRC | 06:59 | |
| *** jjmb has joined #openstack-infra | 06:59 | |
| *** GheRivero has joined #openstack-infra | 06:59 | |
| *** jjmb has quit IRC | 07:03 | |
| *** jjmb has joined #openstack-infra | 07:03 | |
| *** psedlak__ has joined #openstack-infra | 07:03 | |
| *** jjmb has quit IRC | 07:04 | |
| *** jjmb has joined #openstack-infra | 07:04 | |
| *** jaypipes has joined #openstack-infra | 07:04 | |
| *** jjmb has quit IRC | 07:04 | |
| *** jjmb has joined #openstack-infra | 07:05 | |
| *** psedlak_ has quit IRC | 07:07 | |
| *** toddmorey has joined #openstack-infra | 07:15 | |
| openstackgerrit | afazekas proposed a change to openstack-dev/pbr: Hierarchical sphinx API documentation generation https://review.openstack.org/39427 | 07:17 |
| *** yaguang has quit IRC | 07:17 | |
| *** afazekas is now known as afazekas__ | 07:22 | |
| *** psedlak__ is now known as psedlak | 07:24 | |
| *** ^demon has quit IRC | 07:25 | |
| *** nati_ueno has quit IRC | 07:26 | |
| *** nati_ueno has joined #openstack-infra | 07:26 | |
| *** toddmorey has quit IRC | 07:31 | |
| *** nati_ueno has quit IRC | 07:31 | |
| *** yaguang has joined #openstack-infra | 07:36 | |
| *** nati_ueno has joined #openstack-infra | 07:37 | |
| *** nati_ueno has quit IRC | 07:41 | |
| *** toddmorey has joined #openstack-infra | 07:43 | |
| *** dina_belova has joined #openstack-infra | 07:45 | |
| *** hashar has joined #openstack-infra | 07:53 | |
| *** toddmorey has quit IRC | 07:59 | |
| *** Ryan_Lane has quit IRC | 07:59 | |
| *** BobBall_Away is now known as BobBall | 08:01 | |
| *** afazekas has joined #openstack-infra | 08:04 | |
| *** jpich has joined #openstack-infra | 08:06 | |
| *** toddmorey has joined #openstack-infra | 08:15 | |
| *** jjmb has quit IRC | 08:15 | |
| *** jjmb has joined #openstack-infra | 08:16 | |
| *** jjmb has quit IRC | 08:16 | |
| *** jjmb has joined #openstack-infra | 08:16 | |
| *** jjmb has quit IRC | 08:18 | |
| *** jjmb has joined #openstack-infra | 08:18 | |
| *** jjmb has quit IRC | 08:18 | |
| *** jjmb has joined #openstack-infra | 08:20 | |
| *** afazekas has quit IRC | 08:26 | |
| *** toddmorey has quit IRC | 08:31 | |
| *** yaguang has quit IRC | 08:32 | |
| *** yaguang has joined #openstack-infra | 08:32 | |
| *** toddmorey has joined #openstack-infra | 08:35 | |
| chmouel | notmyname, mordred: about probetests and the functional tests we probably would need to remove/adapt the swaio specifics first to get it working on devstack/gating | 08:36 |
| chmouel | s/functional test/on gating/ | 08:36 |
| *** afazekas has joined #openstack-infra | 08:37 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:48 | |
| *** derekh has joined #openstack-infra | 08:49 | |
| *** toddmorey has quit IRC | 08:51 | |
| *** nayward has quit IRC | 08:51 | |
| *** nayward has joined #openstack-infra | 08:52 | |
| *** nayward has quit IRC | 08:57 | |
| *** nayward has joined #openstack-infra | 08:58 | |
| *** toddmorey has joined #openstack-infra | 09:02 | |
| *** nayward has quit IRC | 09:07 | |
| *** nayward has joined #openstack-infra | 09:10 | |
| *** dina_belova has quit IRC | 09:11 | |
| *** toddmorey has quit IRC | 09:18 | |
| *** nayward has quit IRC | 09:21 | |
| *** nayward has joined #openstack-infra | 09:21 | |
| *** toddmorey has joined #openstack-infra | 09:29 | |
| *** dina_belova has joined #openstack-infra | 09:40 | |
| *** toddmorey has quit IRC | 09:45 | |
| *** jjmb has quit IRC | 09:45 | |
| *** jhesketh has quit IRC | 09:56 | |
| *** toddmorey has joined #openstack-infra | 10:00 | |
| *** jhesketh has joined #openstack-infra | 10:09 | |
| *** toddmorey has quit IRC | 10:16 | |
| *** toddmorey has joined #openstack-infra | 10:24 | |
| *** toddmorey has quit IRC | 10:40 | |
| *** toddmorey has joined #openstack-infra | 10:45 | |
| *** yaguang has quit IRC | 10:50 | |
| *** dina_belova has quit IRC | 10:53 | |
| *** dguitarbite has quit IRC | 10:56 | |
| *** weshay has joined #openstack-infra | 10:56 | |
| sdague | man, devstack node exhaustion at 7am EST | 11:01 |
| sdague | that's new | 11:01 |
| sdague | I usually have zul all to myself | 11:02 |
| *** toddmorey has quit IRC | 11:02 | |
| *** weshay has quit IRC | 11:08 | |
| *** weshay has joined #openstack-infra | 11:09 | |
| *** koolhead17 has joined #openstack-infra | 11:12 | |
| *** dina_belova has joined #openstack-infra | 11:12 | |
| *** mikal has quit IRC | 11:19 | |
| *** mikal has joined #openstack-infra | 11:21 | |
| hashar | too many jobs launched :d http://graphite.openstack.org/render/?from=-24hours&height=320&until=now&width=700&bgcolor=ffffff&fgcolor=000000&target=alias(summarize(sumSeries(stats_counts.zuul.job.*),'1h'),'All%20Jobs')&title=Zuul%20Jobs%20Launched%20(per%20Hour)&_t=0.7235209576319903 | 11:21 |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: add update.py testing to requirements https://review.openstack.org/39358 | 11:22 |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: make sqla requirements safe for nova https://review.openstack.org/39461 | 11:26 |
| *** SergeyLukjanov has quit IRC | 11:29 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: add oslo.* to the projects list https://review.openstack.org/39465 | 11:43 |
| *** sandywalsh has quit IRC | 11:44 | |
| *** dina_belova has quit IRC | 11:47 | |
| *** toddmorey has joined #openstack-infra | 11:54 | |
| *** SergeyLukjanov has joined #openstack-infra | 11:57 | |
| *** sandywalsh has joined #openstack-infra | 11:57 | |
| *** dkranz has joined #openstack-infra | 11:58 | |
| *** afazekas has quit IRC | 12:01 | |
| *** CaptTofu has joined #openstack-infra | 12:02 | |
| *** jjmb has joined #openstack-infra | 12:04 | |
| *** psedlak_ has joined #openstack-infra | 12:05 | |
| *** psedlak has quit IRC | 12:05 | |
| *** prad has joined #openstack-infra | 12:07 | |
| *** dina_belova has joined #openstack-infra | 12:07 | |
| *** toddmorey has quit IRC | 12:10 | |
| *** lcestari has joined #openstack-infra | 12:11 | |
| zul | sdague: zul sets off my irc notification thingy ;) | 12:12 |
| sdague | heh | 12:12 |
| *** prad has quit IRC | 12:13 | |
| sdague | mordred, clarkb, fungi, jeblair: https://review.openstack.org/39465 is step #1 in the plan, the rest of global requirements enforcement sits behind that. So if you can take a look early, that would be good. | 12:13 |
| *** prad has joined #openstack-infra | 12:17 | |
| *** dhellmann_ is now known as dhellmann | 12:23 | |
| *** zaro0508 has quit IRC | 12:27 | |
| *** jjmb has quit IRC | 12:32 | |
| *** jjmb has joined #openstack-infra | 12:35 | |
| *** HenryG has joined #openstack-infra | 12:43 | |
| *** rfolco has joined #openstack-infra | 12:45 | |
| *** woodspa has joined #openstack-infra | 12:46 | |
| *** prad has quit IRC | 12:59 | |
| *** dkliban_afk has joined #openstack-infra | 13:03 | |
| *** psedlak__ has joined #openstack-infra | 13:03 | |
| *** psedlak_ has quit IRC | 13:07 | |
| mordred | sdague: could you add oslo* to devstack-vm-update-image too? | 13:07 |
| mordred | sdague: that's the thing that pre-clones repos | 13:07 |
| *** michchap has quit IRC | 13:08 | |
| *** michchap has joined #openstack-infra | 13:08 | |
| *** weshay has quit IRC | 13:09 | |
| mordred | sdague: also, sitepackages = True and -U needed for this? | 13:09 |
| *** krtaylor has quit IRC | 13:09 | |
| *** toddmorey has joined #openstack-infra | 13:10 | |
| *** cppcabrera has joined #openstack-infra | 13:11 | |
| *** cppcabrera has left #openstack-infra | 13:12 | |
| *** mriedem has joined #openstack-infra | 13:13 | |
| *** jjmb1 has joined #openstack-infra | 13:13 | |
| *** psedlak__ is now known as psedla | 13:13 | |
| *** psedla is now known as psedlak | 13:13 | |
| *** toddmorey has quit IRC | 13:15 | |
| *** jjmb has quit IRC | 13:15 | |
| *** anteaya has joined #openstack-infra | 13:20 | |
| *** weshay has joined #openstack-infra | 13:23 | |
| *** jjmb1 has quit IRC | 13:23 | |
| *** changbl has quit IRC | 13:24 | |
| *** dims has joined #openstack-infra | 13:25 | |
| clarkb | good morning | 13:33 |
| fungi | wow, clarkb is up early | 13:35 |
| * fungi double-checks the clock | 13:35 | |
| clarkb | fungi: yes very early | 13:36 |
| *** michchap has quit IRC | 13:36 | |
| *** michchap has joined #openstack-infra | 13:37 | |
| mordred | morning fungi | 13:38 |
| mordred | morning clarkb | 13:38 |
| mordred | fungi: where do we publish our publications now? | 13:38 |
| clarkb | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=29 the number of logs is too damn high | 13:38 |
| clarkb | I am going to manually delete a couple extra indexes today I think | 13:39 |
| clarkb | mordred: docs.openstack.org/infra/publications but the job hasn't run successfully since fungi fixed it | 13:39 |
| fungi | mordred: the next merge to the overview branch of openstack-infra/publications should hopefully appear at http://docs.openstack.org/infra/publications/overview/ now that i've probably got the publish job right | 13:40 |
| mordred | cool | 13:40 |
| *** psedlak_ has joined #openstack-infra | 13:41 | |
| mordred | I will land clarkb's abstract | 13:41 |
| mordred | :) | 13:41 |
| *** psedlak has quit IRC | 13:41 | |
| fungi | stellar | 13:41 |
| fungi | the overview directory *just* appeared on the docs site | 13:42 |
| fungi | so this is a good sign | 13:42 |
| clarkb | magic | 13:42 |
| fungi | i deleted the old '$ZUUL_BRANCH' directory so we didn't leave a mess | 13:43 |
| *** jjmb has joined #openstack-infra | 13:44 | |
| mordred | fungi: it doesn't seem to have loaded the javascript properly | 13:45 |
| mordred | I have all the datas, but it's on a single page | 13:45 |
| clarkb | neat | 13:45 |
| fungi | yeah, i'm looking on the ftp server | 13:46 |
| mordred | http://docs.openstack.org/infra/publications/overview/scripts/slidy.js == unhappy | 13:46 |
| fungi | strangely, everything's there on the server in the correct relative paths | 13:46 |
| *** cppcabrera has joined #openstack-infra | 13:47 | |
| *** cppcabrera has left #openstack-infra | 13:47 | |
| clarkb | mordred: that spits out a 500. Are we not allowed to serve js out of there? | 13:47 |
| fungi | and world-readable | 13:47 |
| mordred | clarkb: I can't imagine why not | 13:48 |
| mordred | omg | 13:48 |
| mordred | maybe there is a top-level .htaccess file that's doing crazy things? | 13:48 |
| mordred | like, that the docs folks put in for some reason? | 13:49 |
| fungi | the css files also aren't available | 13:49 |
| fungi | but the icons, which are also in a relative subdir, are fine | 13:49 |
| fungi | i'll go on a .htaccess hunt | 13:49 |
| *** _TheDodd_ has joined #openstack-infra | 13:50 | |
| clarkb | _TheDodd_: closes-bug properly updated my bug when the change merged. That piece of functionality lgtm | 13:51 |
| fungi | aha, mordred: clarkb: there are .htaccess files in the styles and scripts dirs | 13:51 |
| fungi | check your repos | 13:51 |
| _TheDodd_ | IGTM? | 13:52 |
| _TheDodd_ | Is Good To Merge? | 13:52 |
| _TheDodd_ | Pardon my absolute ignorance. | 13:52 |
| mordred | probably a bad spelling, you usually LGTM is "looks good to me" | 13:52 |
| clarkb | that ^ | 13:53 |
| zaro | cocka-doodle-doo! | 13:53 |
| *** krtaylor has joined #openstack-infra | 13:53 | |
| clarkb | _TheDodd_: checking the other headers with the test bug and updating the gerrit regex is all that remains. Though the gerrit regex mostly works without being updated | 13:53 |
| fungi | reload it now | 13:53 |
| clarkb | fungi: works | 13:54 |
| fungi | fix coming | 13:54 |
| mordred | yay! | 13:54 |
| _TheDodd_ | Lol...gosh. I was reading an I instead of an l. They look identical with the font in Kiwi IRC. | 13:54 |
| clarkb | zaro: oh man, don't remind me of what it is like living with roosters | 13:54 |
| clarkb | they don't just make noise at 6am, they make noise all the time | 13:54 |
| pleia2 | morning | 13:54 |
| clarkb | 2pm why not? 3am never better | 13:54 |
| *** pabelanger has quit IRC | 13:54 | |
| _TheDodd_ | clarkb: yea. The gerrit regex works already, but it could be tightened up so that it will link only the bug number as opposed to the word 'bug' | 13:55 |
| zaro | clarkb: thought you might be homesick | 13:55 |
| clarkb | _TheDodd_: yup | 13:55 |
| _TheDodd_ | This will make it look a bit nicer, but it is totally non-critical. | 13:55 |
| fungi | clarkb: mordred: https://review.openstack.org/39486 | 13:56 |
| pleia2 | ah, those .htaccess files got me too when I uploaded my own oscon slides | 13:57 |
| *** dina_belova has quit IRC | 13:58 | |
| fungi | pleia2: did you discover they were the cause? | 13:58 |
| *** avtar has joined #openstack-infra | 13:58 | |
| pleia2 | fungi: they only caused things to render all wrong | 13:58 |
| *** SergeyLukjanov has quit IRC | 13:58 | |
| pleia2 | single slidedeck | 13:58 |
| fungi | yeah, they definitely do | 13:59 |
| pleia2 | and ended up with regex errors in my apache error.log | 13:59 |
| fungi | after i deleted them via ftp, everything started working | 13:59 |
| * pleia2 nods | 13:59 | |
| *** pentameter has joined #openstack-infra | 14:00 | |
| fungi | sorry they tripped you up! i guess most of us presented from our local filesystems before, so nothing was interpreting the .htaccess rules | 14:00 |
| clarkb | file:/// ftw | 14:00 |
| *** dina_belova has joined #openstack-infra | 14:01 | |
| *** toddmorey has joined #openstack-infra | 14:01 | |
| *** psedlak__ has joined #openstack-infra | 14:01 | |
| fungi | same here | 14:02 |
| zaro | anyone know why cloning a vm on hpcloud isn't working? | 14:02 |
| zaro | i'm using novaclient | 14:02 |
| *** SergeyLukjanov has joined #openstack-infra | 14:03 | |
| fungi | zaro: is it booted? i think we clone our updated devstack vms with the template vm shut down first | 14:04 |
| *** psedlak_ has quit IRC | 14:05 | |
| zaro | fungi: nova list says 'image_snapshot' so seems like the snapshot is getting created. however when i do nova list again the new machine isn't on the list. | 14:06 |
| *** jaypipes has quit IRC | 14:07 | |
| *** CaptTofu has quit IRC | 14:08 | |
| zaro | fungi: command i run is nova image-create $image_id $new_image_name | 14:09 |
| anteaya | my goodness everyone is up early today | 14:12 |
| anteaya | did noone else get caught watching dancing with the stars last night? someone on twitter was jawing on about it | 14:12 |
| _TheDodd_ | No, anteaya, we stayed up all night working on OpenStack. | 14:15 |
| _TheDodd_ | ;) | 14:15 |
| anteaya | you keener you | 14:15 |
| _TheDodd_ | lol | 14:15 |
| anteaya | which is probably why you have stuff that works today, yay you | 14:15 |
| anteaya | :D | 14:15 |
| mordred | anteaya: hehe. what, do you think I was tweeting about that a lot? | 14:16 |
| mordred | :) | 14:16 |
| anteaya | oh just a bit, I wanted to see what the fuss was about, they did a good number, then I went to sleep | 14:16 |
| anteaya | I really like these slides | 14:16 |
| anteaya | I think this one could be updated: http://docs.openstack.org/infra/publications/overview/#%2811%29 to show the pretty zuul progress bars | 14:17 |
| anteaya | all in all, very pretty | 14:18 |
| anteaya | nice to have all that info in one place | 14:19 |
| anteaya | I'll see if I can invoke ttx | 14:21 |
| anteaya | oh ttx... | 14:21 |
| *** pabelanger has joined #openstack-infra | 14:22 | |
| *** afazekas has joined #openstack-infra | 14:22 | |
| fungi | anteaya: i think he's on holiday for the next couple days | 14:23 |
| anteaya | oh okay thanks | 14:23 |
| anteaya | guess I am hitting a wall then with storyboard | 14:23 |
| anteaya | unless anyone else is willing to clone the repo and fire it up (the readme gives the setup instructions) | 14:24 |
| anteaya | the latest changes error out for me when I naviagate to stories | 14:24 |
| anteaya | wanted to find out if it was just me (I might not have completed a configuration step) | 14:25 |
| anteaya | or if it is the code | 14:25 |
| *** burt has joined #openstack-infra | 14:26 | |
| openstackgerrit | Anthony Dodd proposed a change to openstack-infra/jeepyb: This is a functionality test! Do not merge! https://review.openstack.org/39491 | 14:27 |
| *** vijendar has joined #openstack-infra | 14:27 | |
| *** michchap has quit IRC | 14:28 | |
| fungi | zaro: i've not played with nova snapshots before, but i'll try it out myself here in a few minutes | 14:31 |
| *** changbl has joined #openstack-infra | 14:32 | |
| *** datsun180b has joined #openstack-infra | 14:34 | |
| *** jaypipes has joined #openstack-infra | 14:34 | |
| *** markmcclain has joined #openstack-infra | 14:35 | |
| *** cp16net is now known as cp16net|away | 14:37 | |
| *** cp16net|away is now known as cp16net | 14:42 | |
| *** dkranz has quit IRC | 14:43 | |
| anteaya | so the storyboard issue was just me, it turns out I needed to remove the old storyboard.db file and allow manage.py to create a new one | 14:44 |
| pleia2 | glad you got it sorted :) | 14:45 |
| clarkb | anteaya: good to know. I think ttx mentioned he didn't intend on being friendly to upgrades until storyboard was more developed | 14:45 |
| anteaya | mordred informed me thusly | 14:46 |
| anteaya | pleia2: thanks :D | 14:46 |
| *** anneisgentle is now known as annegentle | 14:47 | |
| *** rnirmal has joined #openstack-infra | 14:47 | |
| clarkb | fungi: do you know if the horizontal openstack logo like is used in the overview slides is no longer officially available? That image looks really low res on my monitory but I can't find a replacement | 14:49 |
| mordred | clarkb, fungi: word back on paste.drizzle.org is "let it die" | 14:53 |
| fungi | clarkb: huh... yeah i'm not seeing the old horizontal format logo so maybe they decided not to continue with it for consistency | 14:54 |
| *** rcleere has joined #openstack-infra | 14:54 | |
| clarkb | mordred: that will make jeblair happy | 14:54 |
| mordred | clarkb: that's our last rax classic thing, right? | 14:54 |
| clarkb | mordred: no, apparently puppet dashboard, puppet master, and one other thing are too | 14:54 |
| clarkb | tx.slave | 14:55 |
| clarkb | for tx.slave we can probably just disconnect the slave from jenkins and do a migration on it | 14:55 |
| fungi | all the ones on http://www.openstack.org/brand/openstack-logo/ seem to be the upright/stacked orientation | 14:55 |
| clarkb | zuul/jenkins will end up queuing those jobs | 14:55 |
| clarkb | fungi: ya | 14:55 |
| mordred | clarkb: puppetmaster will be a fun one to move | 14:56 |
| *** dkranz has joined #openstack-infra | 14:56 | |
| clarkb | same for puppet dashboard, reports will fail but shouldn't affect application of stuff | 14:56 |
| mordred | yeah. dashboard seems easy enough | 14:56 |
| clarkb | mordred: well assuming we can go for a period of time without a puppet master (no one is planning any urgent changes right?) we can probably just do that one too | 14:56 |
| mordred | indeed | 14:56 |
| mordred | but can you still have rax migrate servers for you? I think we have to do a re-spin at this point | 14:57 |
| clarkb | mordred: I think you can. Why would they remove that feature? | 14:57 |
| *** dhellmann is now known as dhellmann_ | 14:57 | |
| fungi | yeah i don't think the puppet master will be a pain to replicate. we can just rsync the entire filesystem if we really want | 14:58 |
| clarkb | the alternative to all three is to build new ones, which isn't that bad of a thing either | 14:58 |
| fungi | the reason we didn't move previous servers from legacy to nova, i believe, was for safety (because there weren't backups) | 14:58 |
| *** michchap has joined #openstack-infra | 14:58 | |
| fungi | so if something went wrong and we ended up not being able to boot the vm, it was going to be a lot more downtime to rebuild in a vaccuum | 14:59 |
| clarkb | I am beginning to think we should just rebuild from scratch. Particularly with dashboard. That will reset the DB contents allowing us to set it up with a retention period and so on | 15:00 |
| fungi | but also, because building from scratch both gets you a clean server with no cruft and also exercises your disaster recovery plans | 15:00 |
| *** krtaylor has quit IRC | 15:00 | |
| clarkb | ++. I will fire up a new tx.slave later this morning as that one should be easy | 15:01 |
| * fungi sounds like a mindless enterprise risk assessment consultant suddenly | 15:01 | |
| clarkb | I have now doomed myself to pain | 15:01 |
| clarkb | actuall first https://review.openstack.org/#/c/38846/ should probably go in | 15:01 |
| fungi | we will all suffer with you, clarkb | 15:01 |
| clarkb | but that has the potential for blowing up devstack again | 15:02 |
| jeblair | clarkb: your plan sounds good | 15:02 |
| jeblair | clarkb, fungi: note that slidey does automagic scaling of images, so it may just be scaled more than intended. | 15:02 |
| jeblair | also, i dunno if that was ever an official logo... | 15:03 |
| clarkb | jeblair: ya, I noticed that when I used one of the vertical logos | 15:03 |
| clarkb | jeblair: it stayed really small because it seems to be trying to scale something wide there | 15:03 |
| jeblair | ah, well, actually it's on the homepage so i guess so. | 15:03 |
| *** HenryG has quit IRC | 15:03 | |
| jeblair | (or a slight variant of it is) | 15:04 |
| *** changbl has quit IRC | 15:04 | |
| clarkb | I suppose I can take the vertical one and cut it up to be horizontal. Not sure if that is a breach of the agreement you must agree to before downloading the logo | 15:04 |
| *** changbl has joined #openstack-infra | 15:05 | |
| fungi | clarkb: i strongly suspect it is not allowed. i also strongly suspect nobody who matters will really care | 15:06 |
| jeblair | +1 | 15:06 |
| *** michchap has quit IRC | 15:07 | |
| *** pabelanger has quit IRC | 15:08 | |
| *** dina_belova has quit IRC | 15:08 | |
| *** odyssey4me has joined #openstack-infra | 15:09 | |
| *** w_ has joined #openstack-infra | 15:10 | |
| *** dina_belova has joined #openstack-infra | 15:11 | |
| *** mrodden has quit IRC | 15:11 | |
| *** olaph has quit IRC | 15:13 | |
| *** w_ is now known as olaph | 15:16 | |
| fungi | the spirit of the "do not modify" rules for trademarked logos is to strengthen trademark protection, avoid brand confusion and also prevent people from using really hideous redesigns | 15:18 |
| jeblair | yeah, a i think a copy/paste job is better than making a stick-figure substitute with the logotype "OpenStick". | 15:19 |
| fungi | dunno, that could have some geeky xkcd-like appeal to it ;) | 15:20 |
| jeblair | i can't believe no one has hired randall to draw openstick comics yet. i mean, he was even on contract with ibm for a while. | 15:21 |
| *** mrodden has joined #openstack-infra | 15:22 | |
| clarkb | I almost have something that I thik works | 15:22 |
| *** dina_belova has quit IRC | 15:23 | |
| *** avtar has quit IRC | 15:23 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: add oslo.* to the projects list https://review.openstack.org/39465 | 15:24 |
| sdague | mordred: hows that ^^^ | 15:24 |
| jeblair | i think it's lovely. though now i wonder what happened to mordred's patch to put them in one file. | 15:25 |
| *** michchap has joined #openstack-infra | 15:26 | |
| *** tiamar has joined #openstack-infra | 15:26 | |
| *** michchap_ has joined #openstack-infra | 15:27 | |
| *** pabelanger has joined #openstack-infra | 15:28 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: TextFinder plugin support added https://review.openstack.org/37849 | 15:28 |
| *** UtahDave has joined #openstack-infra | 15:28 | |
| *** SergeyLukjanov has quit IRC | 15:29 | |
| tiamar | hi! I have a problem when executing run_tests scripts. I update to pip 1.4 and setuptools 0.9.8 and uninstalled distribute , but the script always fails when installing the python clients. Any idea of where I should start looking? | 15:29 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: HTML Publisher plugin support added https://review.openstack.org/37876 | 15:29 |
| openstackgerrit | A change was merged to openstack/requirements: Remove d2to1 from list https://review.openstack.org/39144 | 15:29 |
| fungi | tiamar: what errors are you getting? and in which project are you trying to run tests? | 15:30 |
| *** rnirmal has quit IRC | 15:30 | |
| *** michchap has quit IRC | 15:31 | |
| tiamar | fungi: I tried in keystone, nova and cinder - the problems depends on which client pip is downloading | 15:31 |
| *** michchap_ has quit IRC | 15:32 | |
| fungi | tiamar: can you paste the errors you're getting into http://paste.openstack.org/ and give us the resulting url here? i'll see if it looks familiar/obvious | 15:32 |
| tiamar | fungi: will do - but the weird part: I don't have any problems with tox nor with devstack - just with run_tests | 15:33 |
| *** michchap has joined #openstack-infra | 15:34 | |
| tiamar | http://paste.openstack.org/show/42762/ | 15:34 |
| fungi | tiamar: very interesting. it could be something like an old python-setuptools from your distro clashing with newer setuptools from pip in the global system context, thus wouldn't show up under tox because it uses a virtualenv | 15:35 |
| *** michchap_ has joined #openstack-infra | 15:35 | |
| tiamar | I did have setuptools 0.6c11, but I did a apt-get remove --purge on it, and now pip shows setuptools as 0.9.8 | 15:36 |
| fungi | Download error on https://pypi.python.org/simple/setuptools-git/: [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol -- Some packages may not be found! | 15:36 |
| fungi | dstufft: ^ ssl errors between pypi/pip? | 15:36 |
| *** michchap has quit IRC | 15:38 | |
| fungi | tiamar: is there a chance that the machine where you're running that is behind a web proxy of some sort? | 15:39 |
| *** michchap_ has quit IRC | 15:39 | |
| tiamar | fungi: yes, it is. | 15:39 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: make oslo.config gating on devstack/tempest runs https://review.openstack.org/39508 | 15:40 |
| tiamar | but pip does not have trouble downloading other things from pipy | 15:40 |
| fungi | mordred: sdague: do you recall what the proxy workaround was for the pypi ssl errors hp and ibm devs have been seeing? | 15:41 |
| sdague | fungi: recompile git with openssl | 15:41 |
| sdague | or use a different network | 15:41 |
| fungi | oh, it was git not pip the ibm devs were having proxy issues with | 15:42 |
| sdague | yeh | 15:42 |
| sdague | but it was an ssl thing | 15:42 |
| sdague | where a layer 7 filter was getting in the way of something | 15:42 |
| fungi | i think the tripleo/hp folk were seeing https pypi issues with their proxy | 15:42 |
| * fungi nods. i think it had to do with whether the tool, the proxy and the site supported an overlapping set of cryptographic primitives | 15:43 | |
| fungi | but could be mistaken | 15:43 |
| fungi | tiamar: on that machine, are you able to pip install setuptools-git>=0.4 or do you get that same error? | 15:44 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: set encoding UTF-8 for settings sample https://review.openstack.org/37920 | 15:44 |
| fungi | (you'll probably have to quote 'setuptools-git>=0.4' to avoid it being interpreted as a redirect in your shell) | 15:44 |
| openstackgerrit | A change was merged to openstack-infra/storyboard: gitignore /*.egg https://review.openstack.org/37969 | 15:45 |
| fungi | tiamar: looks similar to http://bugs.python.org/setuptools/issue156%3E | 15:47 |
| tiamar | I already have setuptools installed, so the pip command does not print anything | 15:47 |
| jeblair | mordred, fungi, clarkb: https://review.openstack.org/#/c/37945/1 | 15:48 |
| fungi | tiamar: it probably redirected the output of the command to a file named =0.4 if you didn't quote 'setuptools-git>=0.4' in your command line | 15:48 |
| jeblair | mordred, fungi, clarkb: i don't think we've discussed hosting stackforge (and non "#openstack-") channel logs on eavesdrop.openstack.org. | 15:48 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Adding the Sauce OnDemand Plugin https://review.openstack.org/37947 | 15:48 |
| clarkb | jeblair: good point | 15:49 |
| jeblair | mordred, fungi, clarkb: i guess it's okay? it's a bit like tarballs, i suppose. but it's also hosting random user-generated content. | 15:49 |
| tiamar | fungi: I didn't quoted - no errors - says the requirement is up-to-date | 15:49 |
| fungi | jeblair: clarkb: mordred: definitely falls into that same gray area as docs publication for stackforge projects. for tarballs we already decided it was okay... | 15:50 |
| jeblair | fungi: you're right. we decided not to do it for docs, because, well, it's not an openstack doc. | 15:50 |
| clarkb | the docs site comes with a lot more brandng than just the name too | 15:50 |
| clarkb | eavesdrop and tarballs are very barebones mod autoindex served things | 15:51 |
| jeblair | tarballs is required in order to use the build system; docs are not | 15:51 |
| clarkb | however, it is user generated and we could end up hosting links to all kinds of nasty (doubt that will actually happen) | 15:51 |
| jeblair | like paste. :) | 15:51 |
| jeblair | i think i'm okay taking a wait-and-see approach.... | 15:52 |
| fungi | i'm inclined to see eavesdrop and tarballs in a similar light there. it's not so much an official publication as a shared community resource. we already have meetings for stackforge projects taking place in official openstack meeting channels and logged on eavesdrop | 15:52 |
| jeblair | if we get a complaint, we "rm" the file. if we get several complaints, we "rm" the channel. :) | 15:52 |
| clarkb | wfm | 15:52 |
| fungi | agreed | 15:52 |
| fungi | tiamar: okay, so it's probably not pip which is having proxy issues but maybe easy_install/setuptools | 15:53 |
| fungi | the traceback suggests as much | 15:54 |
| jeblair | ttx: https://review.openstack.org/#/c/37977/ do you want to support spaces in project names in storyboard? | 15:54 |
| tiamar | how I could confirm it? | 15:54 |
| fungi | tiamar: do you have a system not behind a proxy where you can test the same run_tests script? | 15:55 |
| tiamar | only at home :) | 15:55 |
| jeblair | clarkb, fungi: https://review.openstack.org/#/c/38755/ | 15:55 |
| jeblair | clarkb, fungi: can you review/aprv that? I'd like to have it in place as i go through several changes that touch layout.yaml. | 15:55 |
| fungi | doing | 15:56 |
| clarkb | reviewing as well | 15:56 |
| tiamar | fungi: I put setuptools-git explicity at requires.txt, and it was downloaded and installed without any errors - but when installing glanceclient it failed - this is just weird. | 15:58 |
| fungi | tiamar: yes, if it's in requires.txt it's being downloaded by different package tools potentially than if it's a transitive dependency | 15:59 |
| tiamar | fungi: right, I obviously didn't know that :P | 16:00 |
| *** dhellmann_ is now known as dhellmann | 16:00 | |
| jeblair | clarkb, fungi: see the patchset n-1 jenkins job output for an example of how that test fails | 16:01 |
| fungi | tiamar: there are multiple package retrieval tools for python, and different ones potentially get used by different packages to satisfy their dependencies at installation. it's a mess | 16:01 |
| clarkb | jeblair: saw that yesterday. it spits out a list at the end | 16:01 |
| * fungi saw it too. very helpful | 16:01 | |
| *** SergeyLukjanov has joined #openstack-infra | 16:03 | |
| fungi | tiamar: also see https://bitbucket.org/pypa/setuptools/issue/52/ssl-errors-with-https-proxy | 16:03 |
| *** vijendar has quit IRC | 16:03 | |
| fungi | tiamar: Ng (in here) opened that, so maybe he can suggest a workaround | 16:03 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Cause virtualenv to be installed into the mirror https://review.openstack.org/38042 | 16:03 |
| tiamar | fungi: thanks! | 16:04 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Add .idea to .gitignore https://review.openstack.org/38121 | 16:04 |
| jeblair | sdague, mordred: where does https://review.openstack.org/#/c/38176/1 fit in to the current plan? | 16:05 |
| clarkb | jeblair: fungi https://review.openstack.org/#/c/39513/ something like that for an updated logo | 16:05 |
| openstackgerrit | A change was merged to openstack-infra/config: Validate that zuul-referenced jobs exist https://review.openstack.org/38755 | 16:05 |
| Ng | tiamar: I don't have a good workaround at this point, although if you manually figure out the dependency chain and use pip to install each part, you won't end up using the setuptools codepath, so you won't hit that bug. | 16:05 |
| *** marun has joined #openstack-infra | 16:06 | |
| tiamar | Ng: that wouldn't be a good solution - more coleagues of me will face this problem and it is better to have run_tests working properly. When I fix it, I come back telling what I did :) | 16:07 |
| Ng | tiamar: I suppose another option would be to somehow force an installation of an older setuptools - the version in distros (e.g. in Ubuntu) is old enough that it has a previous version of the downloading code, which actually works with https and proxies | 16:07 |
| Ng | but I haven't looked into doing that | 16:07 |
| *** nati_ueno has joined #openstack-infra | 16:07 | |
| mordred | jeblair, sdague: I think it still fits in - we want to make sure that it is possible to build a mirror from a given state of requirements | 16:09 |
| mordred | jeblair, sdague: and that that mirror and a given state of pbr will all work to install things into venvs | 16:09 |
| mordred | so I think it's still an addition to the gate-requirements-on-devstack | 16:09 |
| jeblair | mordred: is it ok to merge now or does sequencing need to be rethought? I don't have the whole timeline in my head right now; I have not dedicated many brain cycles to working this out myself. | 16:10 |
| dstufft | fungi: tiamar setuptools 0.7+ doesn't work with proxies IIRC | 16:10 |
| *** adalbas has joined #openstack-infra | 16:10 | |
| dstufft | fungi: tiamar and setup_requires is isntalled by setuptools not pip | 16:10 |
| mordred | jeblair: I believe it should be ok to merge now | 16:11 |
| fungi | dstufft: thanks. it looks like the open bug on setuptools is just waiting on some upstream attention at this point | 16:11 |
| tiamar | dstufft: fungi:but install_venv_common.sh installs setuptools 0.9.8 | 16:11 |
| mordred | dstufft, fungi, tiamar: two things a) setuptools-git as a setup_require is going away b) the workaroud if you are behind a proxy is to install pip and pbr manually first | 16:11 |
| *** dina_belova has joined #openstack-infra | 16:11 | |
| *** zaro0508 has joined #openstack-infra | 16:12 | |
| dstufft | also I just woke up so take everything I say with a grain of salt | 16:12 |
| mordred | tiamar: we should add a manual pip install of pbr and pip into the venv into install_venv_common.sh | 16:12 |
| mordred | because you never don't want it | 16:12 |
| * fungi sends 300 ccs of icmp-coffee-reply to dstufft | 16:13 | |
| clarkb | fungi: that was generous I didn't see an icmp-coffee-request on the wire :) | 16:13 |
| fungi | clarkb: gratuitous icmp | 16:13 |
| clarkb | jeblair: mordred fungi https://review.openstack.org/#/c/38846/ I think we should consider merging that before replacing any rax legacy nodes | 16:14 |
| clarkb | however d-g may bork considering it did last time around | 16:14 |
| *** KernelDog has joined #openstack-infra | 16:14 | |
| mordred | tiamar: https://review.openstack.org/39515 | 16:15 |
| mordred | fungi, dstufft ^^ | 16:15 |
| jeblair | clarkb: then you may want to trigger an image build on one zone after merging that | 16:15 |
| *** yaguang has joined #openstack-infra | 16:15 | |
| *** datsun180b has quit IRC | 16:16 | |
| mordred | jeblair: update-image hasn't worked in over a week | 16:16 |
| mordred | jeblair: because of ... | 16:16 |
| clarkb | mordred: the openvz fix was merged | 16:17 |
| mordred | oh good! | 16:17 |
| clarkb | mordred: was that not sufficient | 16:17 |
| mordred | nevermind | 16:17 |
| mordred | oh yeah - there is another url that's broken | 16:17 |
| clarkb | jeblair: I will do that here in a bit. First I need to catch up on food/caffeine | 16:17 |
| zaro0508 | Clarkb: did you make a release for zmq-plugin? | 16:17 |
| *** krtaylor has joined #openstack-infra | 16:17 | |
| clarkb | zaro0508: not yet I merged the permissions stuff late yesterday. I will try pushing the tag again in a bit | 16:18 |
| tiamar | mordred: thanks! that was super fast, I will try that now. | 16:18 |
| *** datsun180b has joined #openstack-infra | 16:18 | |
| KernelDog | Hey folks, did anyone ever managed to add ESXi (with already bunch of VMs) into their OpenStack setup? | 16:19 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add #savanna to eavesdrop https://review.openstack.org/37945 | 16:19 |
| *** ^d has joined #openstack-infra | 16:20 | |
| *** ^d has joined #openstack-infra | 16:20 | |
| *** toddmorey has quit IRC | 16:20 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Add #savanna to eavesdrop https://review.openstack.org/37945 | 16:20 |
| SergeyLukjanov | hi guys, could you please approve https://review.openstack.org/#/c/37945/ again? (previously failed to merge because of #openstack-qa added before it) | 16:22 |
| fungi | KernelDog: you might want to ask in #openstack-101 (the -infra channel is folks managing the developer community systems) | 16:23 |
| KernelDog | fungi: Thx mate! | 16:24 |
| fungi | KernelDog: also possibly try #openstack | 16:24 |
| bodepd | marun: I also could not get things installed and fell back to my own pip install script | 16:24 |
| bodepd | mordred: I still need timeto wrap my head around disk-image-building. I could use this image instead of having to install the python dependencies for tempest? | 16:25 |
| bodepd | mordred: and then just keep it up-to-date via rsync or something? | 16:26 |
| mordred | bodepd: yup | 16:26 |
| mordred | bodepd: I mainly wanted to show it to you as a combo of dib + puppet - because I know we keep talking about it - but then there were no examples | 16:27 |
| marun | boedpd: using the forthcoming mirror will help | 16:27 |
| marun | sorry, mirror script | 16:27 |
| mordred | bodepd: so - dib + puppet apply and you can still descibe your image content in puppet if you want, but also get the benefits of dib | 16:27 |
| mordred | bodepd: (I find code is nicer than theory for communication some times) | 16:28 |
| SergeyLukjanov | fungi, thank you! | 16:28 |
| fungi | bodepd: i tried to relate your various concerns with dib to mordred after the birthday party, but may not have accurately represented them | 16:28 |
| jeblair | pabelanger: in https://review.openstack.org/#/c/38665/3 is it correct that the supplied modules.conf is not used, but is just there in case someone wants to pass that path in as the modules_conf_source? | 16:29 |
| bodepd | fungi: I think my most general concern is that is simply has not all processed as of yet :) | 16:29 |
| jeblair | pabelanger: (and in our install, we would be passing the one in openstack_project, so it's ignored?) | 16:29 |
| mordred | bodepd: :) | 16:29 |
| bodepd | mordred: isn't there a triple-o session in Seattle coming up? | 16:29 |
| mordred | bodepd: yes | 16:29 |
| mordred | we should probably move this to #tripleo | 16:30 |
| *** michchap_ has joined #openstack-infra | 16:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add #savanna to eavesdrop https://review.openstack.org/37945 | 16:30 |
| *** psedlak__ has quit IRC | 16:30 | |
| *** michcha__ has joined #openstack-infra | 16:31 | |
| *** hashar has quit IRC | 16:32 | |
| *** michchap has joined #openstack-infra | 16:32 | |
| jeblair | https://review.openstack.org/#/c/37987/2/modules/openstack_project/files/zuul/layout.yaml | 16:32 |
| jeblair | mordred, sdague: ^ | 16:32 |
| jeblair | mordred, sdague: the savanna folks want to add a requirements gate to their project | 16:33 |
| *** yaguang has quit IRC | 16:33 | |
| *** sarob has joined #openstack-infra | 16:33 | |
| *** yaguang has joined #openstack-infra | 16:33 | |
| jeblair | mordred, sdague: i guess that's okay, though they don't use the mirror, and i don't expect any auto-update system to auto-update their project | 16:33 |
| *** michchap_ has quit IRC | 16:34 | |
| mordred | agree - although they could probably easily write an opt-in auto-update job for themselves | 16:34 |
| jeblair | mordred, sdague: though perhaps we should consider supporting that. at any rate, i don't think we should let that slow the project down; just bringing it to your attention. | 16:34 |
| mordred | ++ | 16:35 |
| *** sarob has quit IRC | 16:35 | |
| *** pabelanger has quit IRC | 16:35 | |
| mordred | jeblair: it would be interesting to consider how to let a project opt-in to using the restricted mirror | 16:35 |
| fungi | it does seem like a nice-to-have for projects who may be considering applying for incubation but want to show good faith at shoring up their project standardization and consistency with the rest of openstack first | 16:35 |
| *** sarob has joined #openstack-infra | 16:35 | |
| *** michcha__ has quit IRC | 16:35 | |
| mordred | maybe an override list somewhere that would let someone tie themselves to the mirror even if their org wasn't openstack? | 16:36 |
| mordred | could be useful for pbr and hacking too | 16:36 |
| *** michchap_ has joined #openstack-infra | 16:36 | |
| jeblair | yep. i'm going to leave a comment along the lines of 'hey cool, go for it, but understand that this will probably break your gate (but you'll be able to fix it)'. | 16:36 |
| fungi | could trigger off a parameter/envvar in the builder macro? | 16:36 |
| jeblair | mordred: can be added to the select-mirror script. | 16:36 |
| SergeyLukjanov | am I correct that gate-*-requirements checks versions of requirements files using the openstack/requirements repo? | 16:37 |
| mordred | ++ | 16:37 |
| jeblair | fungi: and could look for something like that. | 16:37 |
| * fungi nods | 16:37 | |
| jeblair | SergeyLukjanov: oh hi, i can type in here instead of in the change. :) | 16:37 |
| *** michchap has quit IRC | 16:37 | |
| SergeyLukjanov | hi :) | 16:37 |
| jeblair | SergeyLukjanov: the thing that is about to change is that it's going to be much more strict. | 16:37 |
| *** michchap has joined #openstack-infra | 16:37 | |
| jeblair | SergeyLukjanov: instead of only checking the requirements line that is changed, it will run all the time and check every line | 16:38 |
| jeblair | SergeyLukjanov: which means as soon as something is merged to openstack/requirements, the same change has to be merged to any project that uses the requirements job | 16:38 |
| SergeyLukjanov | jeblair, currently we are syncing our deps with opentstack/requirements | 16:38 |
| jeblair | SergeyLukjanov: there will be a check for all the openstack projects that makes sure that it works with all of them before the change merges to openstack/requirements | 16:38 |
| jeblair | SergeyLukjanov: but it won't check savanna. | 16:38 |
| *** ^d has quit IRC | 16:39 | |
| *** jpich has quit IRC | 16:39 | |
| jeblair | SergeyLukjanov: so what will happen is that when a change merges to requirements, you'll need to update your requirement in savanna to match immediately (openstack projects will need to do the same) | 16:39 |
| clarkb | oh we are going down the route of making requirements strict? I was only half serious when I mentioned we could do that a couple weeks ago | 16:39 |
| jeblair | SergeyLukjanov: but you may also have to _fix_ savanna in the same change if something about the requirement change broke savanna (openstack projects probably won't have to do this) | 16:40 |
| fungi | i for one welcome our new strict requirements checking overlords | 16:40 |
| *** pabelanger has joined #openstack-infra | 16:40 | |
| *** michchap_ has quit IRC | 16:40 | |
| jeblair | clarkb: I'm pretty sure i remember sdague saying "asking nicely doesn't work". | 16:41 |
| jeblair | mordred: ? | 16:41 |
| mordred | yes. although I don't think changing those jobs will be the first thing | 16:41 |
| fungi | instead we will nicely and politely cram it down their throats ;) | 16:41 |
| clarkb | jeblair: :) I think that was my argument | 16:42 |
| *** michchap has quit IRC | 16:42 | |
| mordred | we're going to start with the auto-update-in-devstack + auto-propose-patch | 16:42 |
| *** ^d has joined #openstack-infra | 16:42 | |
| clarkb | I am approving that setuptools change now | 16:42 |
| mordred | I imagine we'll need to run with that for a little bit before we expand the requirements check itself | 16:42 |
| clarkb | if it doesn't get in the 1645 puppet update it will be in at 1700 | 16:42 |
| pabelanger | jeblair: so asterisk_conf_source and modules_conf_source we pass into the asterisk module are likely going to be removed in the next patchset. I'll expose a define in the class that will allow a user to add config files to any module, via the openstack_project::pbx modules | 16:42 |
| *** reed has joined #openstack-infra | 16:43 | |
| *** derekh has quit IRC | 16:43 | |
| clarkb | we should also get eyes on https://review.openstack.org/#/c/39377/ to help the mail queues | 16:43 |
| clarkb | fungi: mordred: rockstar reed ^ | 16:43 |
| SergeyLukjanov | I think that we should strictly fit openstack/requirements in savanna, I'll checking it manually, so it'll be ok | 16:43 |
| reed | clarkb, thanks, I can't really say much about that though | 16:44 |
| * reed has no clue | 16:44 | |
| clarkb | reed: neither can I ... I reviewed the puppet stuff though and puppet look sound | 16:44 |
| jeblair | clarkb, reed: it starts queue runners more often. :) easy peasy. | 16:45 |
| rockstar | Yeah, it's pretty straightforward. | 16:45 |
| openstackgerrit | A change was merged to openstack-infra/config: Ensure latest setuptools after installing pip. https://review.openstack.org/38846 | 16:45 |
| mordred | woot | 16:46 |
| * reed has a clue now | 16:46 | |
| clarkb | 38846 missed 1645 puppet git pull. I am not in a huge hurry so will let it get grabbed at 1700 | 16:47 |
| *** zaro0508 has quit IRC | 16:47 | |
| *** pcrews has quit IRC | 16:47 | |
| jeblair | mordred: for you https://review.openstack.org/#/c/38749/ | 16:47 |
| *** emagana has joined #openstack-infra | 16:48 | |
| jeblair | reed: https://review.openstack.org/#/c/38761/ | 16:48 |
| mordred | jeblair: done | 16:49 |
| openstackgerrit | A change was merged to openstack-infra/config: Enabled gate jobs for openstack-metering cookbook https://review.openstack.org/38730 | 16:49 |
| openstackgerrit | A change was merged to openstack-infra/config: Add queue runner interval option to exim https://review.openstack.org/39377 | 16:50 |
| fungi | i'll spot-check lists.o.o and a couple other servers to make sure that ^ dtrt | 16:50 |
| jeblair | clarkb: don't forget https://review.openstack.org/#/c/38785/ | 16:50 |
| *** toddmorey has joined #openstack-infra | 16:50 | |
| jeblair | fungi: yeah, should be 30m everywhere (which is actually a change for centos hosts, from 1h) | 16:50 |
| openstackgerrit | A change was merged to openstack-infra/config: Add dev-requirements.txt to gate-requirements-install https://review.openstack.org/38749 | 16:50 |
| * fungi nods. consistency ftw | 16:50 | |
| clarkb | jeblair: ya, other things get pushed up the importance list. I will start setting up my test for that | 16:50 |
| jeblair | fungi: should probably show up in the ps cmdline | 16:51 |
| jeblair | "-q30m" or something | 16:51 |
| fungi | yes, it normally does | 16:51 |
| sdague | mordred: ok, so this passes, can we get it sent to the gate - https://review.openstack.org/#/c/39465/ ? | 16:51 |
| sdague | because a lot of other things back up on having that available | 16:51 |
| *** sarob has quit IRC | 16:52 | |
| *** openstack has joined #openstack-infra | 17:19 | |
| *** hub_cap has left #openstack-infra | 17:19 | |
| *** markmcclain1 has quit IRC | 17:19 | |
| *** dina_belova has quit IRC | 17:20 | |
| *** dina_belova has joined #openstack-infra | 17:23 | |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: cease docimpact emails, create bugs only https://review.openstack.org/39118 | 17:23 |
| *** nicedice has joined #openstack-infra | 17:23 | |
| *** markmcclain has joined #openstack-infra | 17:24 | |
| fungi | jeblair: so... the ubuntu servers did reload exim and lists.o.o is therefore running with the new change. something about the centos servers caused it not to reload (looking into it now) | 17:25 |
| jeblair | fungi: thanks; let me know if you want to hand that back off to me | 17:26 |
| fungi | oic, i was watching a centos server where puppet is not running | 17:26 |
| fungi | maybe false alarm | 17:26 |
| fungi | yeah, my bad. it worked there too | 17:27 |
| jeblair | woo | 17:27 |
| openstackgerrit | A change was merged to openstack-infra/config: Add goals and docbook-properties.sh script to openstack-training https://review.openstack.org/39185 | 17:27 |
| * fungi starts puppet back up on git.o.o ;) | 17:27 | |
| jeblair | speaking of back up | 17:27 |
| jeblair | mordred: did you comp the openstackci2 account? | 17:27 |
| mordred | jeblair: yes. I have, at least sent the email relating to comping it | 17:28 |
| jeblair | mordred: thanks! | 17:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Add tempest check and gate for python-heatclient. https://review.openstack.org/38884 | 17:31 |
| *** pcrews has joined #openstack-infra | 17:32 | |
| *** w_ has joined #openstack-infra | 17:33 | |
| *** toddmorey has joined #openstack-infra | 17:33 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/jenkins-job-builder: Allow the gitConfigName and gitConfigEmail values to be used. https://review.openstack.org/36478 | 17:34 |
| *** olaph has quit IRC | 17:35 | |
| *** pabelanger has quit IRC | 17:36 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Allow the gitConfigName and gitConfigEmail values to be used. https://review.openstack.org/36478 | 17:36 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: add oslo.* to the projects list https://review.openstack.org/39465 | 17:38 |
| jeblair | sdague: ^ | 17:39 |
| *** ^d has quit IRC | 17:39 | |
| *** ^d has joined #openstack-infra | 17:40 | |
| *** ^d has joined #openstack-infra | 17:40 | |
| *** pabelanger has joined #openstack-infra | 17:41 | |
| *** afazekas__ is now known as afazekas | 17:42 | |
| *** w_ is now known as olaph | 17:43 | |
| fungi | jeblair: on your https://review.openstack.org/39133 change, see my comment from yesterday... am i misreading how that's supposed to work, or does zuul reload function extensions correctly on the fly now, or are you planning to deploy the change in a controlled fashion? | 17:43 |
| jeblair | fungi: i believe zuul has always supported reloading functions | 17:46 |
| fungi | ahh, okay. i thought you were saying a few weeks ago that you needed to change it to accommodate that. wfm | 17:47 |
| jeblair | i don't remember that :( | 17:47 |
| fungi | discussion about reload() et cetera. i will go jog my memory | 17:48 |
| clarkb | jeblair: you wrote a change to fix that | 17:49 |
| clarkb | jeblair: you had to reimport the functions otherwise changes to them did not take effect | 17:49 |
| *** toddmorey has quit IRC | 17:49 | |
| jeblair | clarkb: that code hasn't substantially changed since 2012-07-10. | 17:51 |
| jeblair | clarkb: there is a variable scoping change that went in with the live-reload change, but that just changed 'self._config_env' to 'config_env'. | 17:51 |
| clarkb | jeblair: then I think there is an outstanding change | 17:54 |
| clarkb | I swear this was noticed after the live reload stuff because we had to restart zuul the last time log paths changed | 17:54 |
| clarkb | sdague: http://logs-dev.openstack.org/31/39531/2/check/gate-nova-python26/30722/logs/screen-c-api.txt.gz does not 404 | 17:54 |
| clarkb | sdague: according to URL lib it results in a 500 | 17:55 |
| *** cppcabrera has joined #openstack-infra | 17:55 | |
| *** cppcabrera has left #openstack-infra | 17:55 | |
| fungi | jeblair: found it. back on the 19th you commented after merging https://review.openstack.org/36320 and seeing it not take effect, that you thought zuul may need a restart to pick up changes in openstack_functions.py | 17:56 |
| jeblair | fungi: i do not think that at this point, after having closely examined the code in question. | 17:56 |
| fungi | okay, great! +2 from me then | 17:57 |
| jeblair | fungi: i wonder if puppet laid down the files in the wrong order | 17:57 |
| jlk | Hunting for re-reviews again. clarkb :) https://review.openstack.org/#/c/36921/ | 18:01 |
| *** andrew_plunk has joined #openstack-infra | 18:01 | |
| andrew_plunk | Hello, I am a developer on the heat team, and I have a server that seems to be blacklisted from being able to submit gerrit reviews | 18:02 |
| andrew_plunk | (the ip seems to be blacklisted). Is there a way to change that? | 18:02 |
| jeblair | fungi, clarkb: http://puppet-dashboard.openstack.org:3000/reports/651768 | 18:02 |
| *** jjmb has quit IRC | 18:02 | |
| mordred | andrew_plunk: we don't blacklist ips ... can you descirbe the issues you're seeing further? | 18:03 |
| mordred | perhaps a pastebin of the errors? | 18:03 |
| jeblair | fungi, clarkb: if that is to be beleived, it did do things in the right order. | 18:03 |
| jeblair | fungi, clarkb: i'm not sure why it would not have taken effect. | 18:03 |
| SergeyLukjanov | are you there any plans to make it possible to trigger jobs in third-party jenkins servers to make this jobs part of the gating process? | 18:04 |
| SergeyLukjanov | I mean if there were even thoughts about it? | 18:04 |
| fungi | jeblair: it looks from the irc logs like you wound up having to recheck some changes from the check queue (failure mode was unspecified but presumably apparent to us at the time) | 18:04 |
| andrew_plunk | mordred: I have the same workspace with gerrit set up on my local laptop and on a cloud server, from my laptop I can submit a gerrit review, but from the server the review hangs after asking for my ssh key password. If you guys do not blacklist ips then I guess it may be a networking problem around my server. | 18:04 |
| mordred | SergeyLukjanov: not to make them gating | 18:04 |
| mordred | SergeyLukjanov: we can certainly trigger (and do) jobs in other servers like how smokestack works | 18:05 |
| mordred | andrew_plunk: yeah - is there a proxy or anything? I know the guys inside of hp have to do special things because hp blocks 29418 | 18:05 |
| *** toddmorey has joined #openstack-infra | 18:06 | |
| fungi | andrew_plunk: any chance it's a rackspace cloud server? we've seen ssh fail via ipv6 from some virtual machines in rackspace | 18:06 |
| mordred | fungi: good call | 18:06 |
| andrew_plunk | fungi: it is a rackspace cloud server | 18:06 |
| fungi | andrew_plunk: specifically, authenticate and then hang indefinitely | 18:06 |
| SergeyLukjanov | mordred, mmm, sounds interesting about triggering jobs, could you please point me on code or resource about how it works? | 18:06 |
| andrew_plunk | yeah that is what I am seeing fungi: | 18:06 |
| dkranz | clarkb: I'm dying with the testr regexp thing. | 18:06 |
| jeblair | andrew_plunk: do you work for rackspace? | 18:06 |
| andrew_plunk | jeblair: yes | 18:07 |
| dkranz | clarkb: Just trying to exclude tags from stuff like | 18:07 |
| dkranz | tempest.api.compute.flavors.test_flavors.FlavorsTestJSON.test_get_flavor_details_for_invalid_flavor_id[gate,negative,slow] | 18:07 |
| *** rnirmal has joined #openstack-infra | 18:07 | |
| fungi | andrew_plunk: can you try the following two commands... 'ping6 -c3 review.openstack.org' and 'ping6 -c3 -Q0x10 review.openstack.org' and compare the results? | 18:07 |
| dkranz | clarkb: The one you gave me yesterday didn't work. | 18:07 |
| clarkb | dkranz: ^(?!.*\[.*\bslow\b.*\]).*$ | 18:07 |
| clarkb | dkranz: you told me before that there would never be other tags on the slow tag :P | 18:08 |
| mordred | SergeyLukjanov: http://ci.openstack.org/third_party.html | 18:08 |
| mordred | SergeyLukjanov: we even have docs! | 18:08 |
| dkranz | clarkb: Sorry. I meant there woujld not be a 'gate' tag as well. | 18:08 |
| jeblair | andrew_plunk: whenever we spin up a server, we test its ipv6 connectivity and if we have this problem, we destroy the server and try again. it's very silly. | 18:08 |
| *** ^d has quit IRC | 18:08 | |
| clarkb | dkranz: so we need a few extra .*'s to match the other tags | 18:08 |
| jeblair | andrew_plunk: i believe fungi opened a ticket about it, and i'm sure we yelled a pvo and troytoman, but it does seem to linger on. | 18:08 |
| clarkb | dkranz: try the regex above | 18:09 |
| SergeyLukjanov | mordred, oh, I read it, just asked because "we can certainly trigger (and do) jobs in other servers" sounds like another approach :) | 18:09 |
| jeblair | s/yelled/provided constructive feedback to/ | 18:09 |
| SergeyLukjanov | mordred, thank you for info | 18:09 |
| andrew_plunk | fungi: the first command gets successful replies, while "ping6 -c3 -Q0x10 review.openstack.org" gets 100% packet loss | 18:09 |
| dkranz | clarkb: Doen'st seem to work. | 18:10 |
| fungi | andrew_plunk: yep! that's the bug. any ipv6 packets with a dscp marking >0x3 get eaten on egress from the vm | 18:10 |
| dkranz | clarkb: There can be any number of tags and the one we are looking for could be anywhere among them. | 18:10 |
| fungi | andrew_plunk: current incarnation of the ticket (we've been pointing out the issue for 8+ months now i think) is 130716-1307023-2 | 18:10 |
| clarkb | dkranz: yeah so the .* should match that. | 18:10 |
| clarkb | dkranz: you can try removing the \bs | 18:10 |
| andrew_plunk | fungi: jeblair: thank you very much for your help! | 18:11 |
| clarkb | dkranz: also make sure that your shell isn't doing anything to them. eg sigle quote the whole thing | 18:11 |
| fungi | andrew_plunk: it only seems to affect a very small percentage of the machines we launch in rackspace, so my guess is it's a localized problem in one segment of that network (maybe a particular switch or even one specific compute node?) | 18:11 |
| dkranz | clarkb: I am single quoting, but still nothing. | 18:12 |
| mordred | SergeyLukjanov: possible bad phrasing on my part ... mainly, smokestack uses that, as does a server at HP that's testing trove | 18:12 |
| jeblair | andrew_plunk: you're welcome, and thank you for yelling at^W^Wproviding constructive feedback to your fellow operations folks about this! :) | 18:12 |
| mordred | and I think some others | 18:12 |
| fungi | andrew_plunk: yes, we'd LOVE to see this issue finally gone | 18:12 |
| mordred | andrew_plunk, jeblair: I just pinged troy about the ipv6 and the nova node slowness problem again | 18:12 |
| clarkb | dkranz: can you paste the full commadn you are running somehwere? | 18:12 |
| jeblair | mordred: awesome. you yell the best. | 18:13 |
| dkranz | clarkb: I was just in the middle of doing that. | 18:13 |
| mordred | jeblair: I've got to be good at something | 18:13 |
| SergeyLukjanov | mordred, afaiu zuul uses this stream too | 18:13 |
| *** odyssey4me has quit IRC | 18:14 | |
| mordred | yup | 18:15 |
| mordred | SergeyLukjanov: in fact, you could totally run your own zuul | 18:16 |
| mordred | and have it follow our stream | 18:16 |
| annegentle | rename question for ya... should http://docs.openstack.org/developer/quantum/ redirect? How do I patch to make that happen? | 18:16 |
| dkranz | clarkb: http://paste.openstack.org/show/42773/ | 18:16 |
| jeblair | annegentle: probably so; and maybe an .htaccess file? | 18:16 |
| fungi | annegentle: you could probably drop a redirect in a .htaccess file there. redirects are generally up to the webserver, so it's either reconfig or that | 18:16 |
| *** jjmb has joined #openstack-infra | 18:17 | |
| clarkb | dkranz: grep may not support negative lookahead without more options | 18:17 |
| annegentle | right I can't remember where that .htaccess file would live though? I guess in openstack-manuals... I'll patch. | 18:17 |
| *** yaguang has quit IRC | 18:17 | |
| SergeyLukjanov | mordred, yep, we tried it, but there are not to much benefits of using it w/o gating, btw we are planning to move our "gate" jobs to be triggered by our own zuul installation | 18:17 |
| mordred | jeblair: ^^ we've discussed before letting a stackforge project define its own thing that could do its gating, but you brought up a good reason why not | 18:19 |
| dkranz | clarkb: OK. That was it. Sorry about that. I tried all but the last with testr but then was trying to be faster. | 18:19 |
| clarkb | dkranz: http://paste.openstack.org/show/42775/ shows how you can test without running testr | 18:20 |
| mordred | also - SergeyLukjanov any reason why to not just use our zuul and put more jobs in the openstack system? (/me not having a specific preference yet, just trying to learn more) | 18:20 |
| clarkb | dkranz: since testr is just using python's re implementation | 18:20 |
| *** melwitt has joined #openstack-infra | 18:20 | |
| dkranz | clarkb: OK, thanks. I'll save that :) | 18:20 |
| clarkb | re.match returns None if it did not match and returns an object otherwise | 18:21 |
| jeblair | mordred: it would have to be exclusive with ours; you can't have two gating systems (at least, not without quite a bit of work). | 18:21 |
| mordred | right | 18:21 |
| SergeyLukjanov | mordred, we have a problem with slaves requirements, to run integration tests for savanna we need an OpenStack that could start at least 4 kvm-backed vms with Hadoop | 18:21 |
| mordred | jeblair: what if a project wanted only their system to run the gate for their project | 18:21 |
| jeblair | mordred: and if a project does not want to use our build infrastructure, that's fine -- don't use our build infrastructure. :) | 18:21 |
| *** toddmorey has quit IRC | 18:22 | |
| jeblair | mordred: if you can run your own jenkins and zuul, you can certainly run your own gerrit. | 18:22 |
| mordred | jeblair: you can - but then it's much harder for other openstack devs to do dev on your project | 18:22 |
| SergeyLukjanov | mordred, we tried to use devstack but there are a lot of false negative results :( | 18:22 |
| mordred | I can see a potential value in using openstack gerrit and all your own testing | 18:23 |
| jeblair | mordred: why is that a problem? | 18:23 |
| jeblair | it's the internet. anyone can connect to anyone. :) | 18:23 |
| mordred | so ... imagine a project such as savanna, which would like to be openstack at some point, and is trying to do things the openstack way | 18:24 |
| jeblair | mordred: well, then it is worth figuring out how to do that in the openstack-infra environment | 18:24 |
| mordred | but whose integration testing needs are not things that are appropriate to solve by us for a stackforge project | 18:24 |
| mordred | yet | 18:24 |
| mordred | like, part of an incubation period might be figuring out how to port in their integration testing | 18:24 |
| jeblair | mordred: why not start that now? | 18:24 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Handle html log annotations. https://review.openstack.org/39547 | 18:25 |
| clarkb | sdague: jeblair ^ that has been tested. I think it would be good to figure out why logs-dev is not returning 404's for invalid log files | 18:25 |
| jeblair | mordred: we are stretched so thin that at the point they are incubated, we aren't going to suddenly have any more time to figure something like that out.. | 18:25 |
| mordred | SergeyLukjanov: potentially good questoin there ... any chance you have access to an openstack that can meet your needs that could be attached to the openstack infra but only used for savanna jobs? | 18:25 |
| mordred | jeblair: good point | 18:25 |
| jeblair | mordred: i think being more inclusive about what we run (recent changes to devstack let you run any job, not just tempest), and try to give people tools to help solve problems when they are small | 18:26 |
| jeblair | is the way to go | 18:26 |
| fungi | and i definitely saw a couple of things in the problem list which are potentially on our wishlist... nested kvm and multi-slave tests | 18:27 |
| *** thomasm has joined #openstack-infra | 18:27 | |
| jeblair | mordred, SergeyLukjanov: the ability to dedicate >1 devstack machine to a test run.... | 18:27 |
| jeblair | fungi: exactly | 18:27 |
| jeblair | mordred, SergeyLukjanov: those are things we'd love to have anyway, so if you can help work on those, everyone benefits | 18:27 |
| mordred | ++ | 18:27 |
| *** andrew_plunk has left #openstack-infra | 18:28 | |
| *** gyee has joined #openstack-infra | 18:28 | |
| SergeyLukjanov | sure, we are diving into the os infra and we'll try to understand how it could be improved to help us | 18:29 |
| *** toddmorey has joined #openstack-infra | 18:29 | |
| mordred | neat! | 18:29 |
| mordred | SergeyLukjanov: yeah - thanks for diving in, btw - it's great to see you guys active | 18:29 |
| jeblair | SergeyLukjanov: yes, thank you very much! | 18:30 |
| *** vipul is now known as vipul-away | 18:30 | |
| SergeyLukjanov | we are working on simplifying integration tests and Hadoop on nested kvm issues in parallel and I hope we'll be able to use smth like devstack gate to integration test savanna... | 18:31 |
| SergeyLukjanov | thank guys for building an amazing infra that could be used by the project like savanna ;) | 18:31 |
| dhellmann | jeblair: are the errors on https://review.openstack.org/39225 related to the change, or is something else going on? | 18:31 |
| mordred | sdague: ok - I've been on phone calls for like a billion years - where are we on requirements and/or what can I do to help | 18:31 |
| sdague | mordred: I have been too :), hold on, let me see where my devstack patches ended up | 18:32 |
| jeblair | dhellmann: related to the change; i just updated the layout check job to verify that jobs specified in zuul layout.yaml are actually defined in jjb config | 18:32 |
| jeblair | dhellmann: if you hit the bottom of http://logs.openstack.org/25/39225/1/check/gate-config-layout/2667/console.html | 18:32 |
| jeblair | dhellmann: you'll see what's missing | 18:32 |
| mordred | fungi, clarkb: https://review.openstack.org/#/c/39461/ ? | 18:33 |
| jeblair | dhellmann: so the change is missing an update to the jjb projects.yaml to create those jobs | 18:33 |
| dhellmann | ah, spotted it | 18:33 |
| fungi | excellent! | 18:33 |
| dhellmann | was looking for "ERROR" | 18:33 |
| dhellmann | :-) | 18:33 |
| sdague | mordred: hm.... why did this just blow up - http://logs.openstack.org/28/39528/1/check/gate-tempest-devstack-vm-full/37885/console.html | 18:33 |
| jeblair | dhellmann: hehe, not a bad idea. :) | 18:33 |
| dhellmann | jeblair: when I can get one of my existing pans out of the fire, I'll try to provide a patch | 18:34 |
| jeblair | dhellmann: maybe better than "ahem, excuse me sir, if it's no trouble, you might consider whether this job is defined?" | 18:34 |
| * dhellmann thought things would get easier when we entered internal betea | 18:34 | |
| mordred | hm | 18:34 |
| sdague | oh, ooops | 18:34 |
| sdague | I lost a part of the seperation | 18:34 |
| dhellmann | jeblair: well, the current message is more clear than "FAIL" so it's a start ;-) | 18:35 |
| mordred | oh good. you know that answer | 18:35 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Handle html log annotations. https://review.openstack.org/39547 | 18:35 |
| clarkb | _TheDodd_: ^ that will test partial-bug for us | 18:35 |
| *** dragondm has joined #openstack-infra | 18:35 | |
| fungi | mordred: sdague: though that change brings to my attention that we're filtering on SQLAlchemy <=0.7.99 instead of <0.8 (any idea why?) | 18:35 |
| sdague | fungi: because 0.8.0beta1 < 0.8 | 18:36 |
| sdague | for.... REASONS | 18:36 |
| sdague | anyway, that's why the pin is like that | 18:36 |
| fungi | sdague: ahh. well, in newpip that ceases to be a danger i think | 18:36 |
| clarkb | ahahahahaha | 18:36 |
| fungi | but good to know | 18:36 |
| sdague | yeh, but we have old pip a lot of places | 18:36 |
| mordred | we can go to <0.8 once we're solidly on pip 1.4 everywhere | 18:36 |
| *** ^demon has joined #openstack-infra | 18:36 | |
| *** ^demon has joined #openstack-infra | 18:36 | |
| dstufft | science | 18:36 |
| *** ^demon is now known as ^d | 18:37 | |
| mordred | sdague: I've told you my plans to make pbr depend on pip 1.4, right? | 18:37 |
| * fungi is a fan of science | 18:37 | |
| dstufft | Hopefully SQLAlchemy never releases 0.7.100 | 18:37 |
| dstufft | or OpenStack will sure look silly! | 18:37 |
| sdague | dstufft: I never write off sqla doing funny things .... | 18:37 |
| sdague | mordred: ok, i need to see if those devstack things go through | 18:38 |
| sdague | then I can fix grenade to do requirements | 18:38 |
| *** cp16net is now known as cp16net|away | 18:38 | |
| sdague | then I think we can see if global update works | 18:38 |
| *** cp16net|away is now known as cp16net | 18:38 | |
| *** pabelanger has quit IRC | 18:38 | |
| sdague | and if so we can bring requirements and oslo.config into the gate | 18:39 |
| sdague | markmc said hold on oslo.messaging right now because no project is actually using it | 18:40 |
| sdague | but just make sure we gate on that before it's allowed to add to nova | 18:40 |
| clarkb | the openvz image link url update just merged. I am building an image in az3 now | 18:40 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Test that custom functions are live-reloaded https://review.openstack.org/39551 | 18:40 |
| jeblair | clarkb, fungi: ^ that works. I am out of ideas of what happened on the 19th. | 18:40 |
| fungi | jeblair: ghosts | 18:41 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add pycadf project https://review.openstack.org/39225 | 18:41 |
| jeblair | clarkb, fungi: I'm inclined to say we should merge it, be prepared to restart it, and examine the logs if it fubars. | 18:41 |
| fungi | wfm | 18:41 |
| fungi | it's not like we're bearing down on a milestone this week, so a little disruption now is better than later | 18:41 |
| clarkb | jeblair: sounds good, and I agree with fungi. better to work out kinks now than later | 18:42 |
| sdague | markmc said he'd make sure to add oslo.messaging before it included in nova, I put him on the oslo.config layout change so they he knew how it was done. I think he's not been in that part of config before | 18:42 |
| clarkb | jeblair: that change feels like it is missing files | 18:44 |
| jeblair | clarkb: hehe it is. | 18:44 |
| fungi | i was just thinking the same thing | 18:44 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add pycadf project https://review.openstack.org/39225 | 18:45 |
| fungi | i forget to git add far too often | 18:45 |
| *** prad has joined #openstack-infra | 18:45 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/zuul: Test that custom functions are live-reloaded https://review.openstack.org/39551 | 18:45 |
| *** pabelanger has joined #openstack-infra | 18:48 | |
| *** pabelanger has quit IRC | 18:48 | |
| *** pabelanger has joined #openstack-infra | 18:48 | |
| *** dina_belova has quit IRC | 18:49 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add pbx and new logstash workers to cacti. https://review.openstack.org/39374 | 18:49 |
| clarkb | jeblair: I think the difference is in your test the files are different so the new stuff gets imported into python properly | 18:50 |
| clarkb | maybe, I have to go see how you are importing the functions from within the files | 18:50 |
| jeblair | clarkb: execfile; shouldn't matter | 18:50 |
| jeblair | clarkb: but i can make the test do that. | 18:51 |
| *** dina_belova has joined #openstack-infra | 18:52 | |
| fungi | mordred: sdague: Double requirement given: oslo.config>=1.2.0a3 (from -r dev-requirements.txt.clean (line 12)) (already in oslo.config>=1.1.0 (from -r requirements.txt.clean (line 33)), name='oslo.config') | 18:52 |
| sdague | ?? | 18:52 |
| jeblair | dhellmann: i don't know what a cadf is, but you feel pretty sure it's not an openstack project (in particular, it is not a component repo of the openstack metering program)? | 18:52 |
| fungi | sdague: on your http://logs.openstack.org/61/39461/1/gate/gate-requirements-install/336/console.html | 18:52 |
| fungi | sdague: no idea how that snuck in there | 18:53 |
| sdague | oh, interesting | 18:53 |
| dhellmann | jeblair: this has been the subject of much discussion. We *may* bring it into the metering program later, but we need to get the code online somewhere to start. At this point, the ceilometer core team is not ready to accept full responsibility for the library. | 18:53 |
| sdague | why didn't we catch that on check? | 18:53 |
| fungi | sdague: it's possible the job changed out from under it | 18:54 |
| sdague | yeh, must have been | 18:54 |
| sdague | that's a good thing to test in the unit tests too once pbr is released | 18:54 |
| fungi | because that's not even a file you touched | 18:54 |
| sdague | actually.... wait | 18:55 |
| sdague | yeh, so that seems like requirements checker is wrong | 18:55 |
| sdague | mordred: can you look at that | 18:55 |
| * mordred reading scrollback | 18:56 | |
| mordred | GAH | 18:56 |
| mordred | sorry - we landed a patch to that script earlier - I don't know how it passed the gate | 18:57 |
| * fungi was just trying to pickaxe his way to the offending change | 18:57 | |
| jeblair | dhellmann: ok, aside from making us move repo locations later... :) there's also integration testing and requirements to consider -- whether it should be tested with the rest of the project and have requirements aligned; if it's a stackforge project, i'd expect it only to be used as a released ribrary. | 18:58 |
| mordred | it's was adding checking the dev-requirements file | 18:58 |
| jeblair | library, even. that's a real dvorak typo, fyi. | 18:58 |
| mordred | sdague, clarkb: I think that the code in the check-reuqirements job should perhaps live in the requirements job | 18:58 |
| mordred | sdague, clarkb: I think that the code in the check-reuqirements job should perhaps live in the requirements repo | 18:58 |
| mordred | so taht we can gate on it | 18:58 |
| mordred | we didn't catch this earlier beause it was change to the job description | 18:58 |
| jeblair | mordred: wfm | 18:58 |
| dhellmann | jeblair: that is what we intend for now. Did I specify the jobs incorrectly to achieve that? | 18:58 |
| fungi | mordred: ahh, yep. i see now that one of the two occurrences is in the dev requires | 18:59 |
| fungi | makes sense to mitigate further disruption like this | 19:00 |
| jeblair | dhellmann: not necessarily; though if you want to use the tag-based release process that we use for most client libraries, you'll want a few more jobs and acls to handle that. | 19:00 |
| dhellmann | jeblair: makes sense; I'll go see what the oslo libs are doing and reproduce that | 19:00 |
| clarkb | mordred: I agree | 19:00 |
| clarkb | mordred: however, keep in mind how do you gate other things on it? | 19:01 |
| jeblair | clarkb: oh you're right. | 19:01 |
| clarkb | mordred: you may end up needing something like the zuul layout checker or the JJB differ | 19:01 |
| jeblair | mordred: what change broke it? | 19:01 |
| dhellmann | jeblair: i see oslo.config has a couple of tarball jobs -- are those required to make the release jobs work? | 19:01 |
| jeblair | dhellmann: yes, that and pypi-jobs | 19:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Change how we gate requirements installs https://review.openstack.org/39558 | 19:01 |
| dhellmann | ok | 19:02 |
| jeblair | ah, that change | 19:02 |
| mordred | jeblair: https://review.openstack.org/#/c/38749/ | 19:02 |
| mordred | yeah | 19:02 |
| *** sdake has quit IRC | 19:02 | |
| mordred | clarkb: so - ACTUALLY - I think that this job can go away once we have the pbr/jeepyb job landed as well as the new stuff in devstack | 19:03 |
| jeblair | mordred: how about this instead | 19:03 |
| mordred | because those will both do the real thing | 19:03 |
| mordred | and then we can just delete this, which was the quick and super helpful thing that got us this far | 19:03 |
| jeblair | mordred: move that into a jenkins_slave_scripts file so what's in jjb is just a stub, then run the requirements check job on changes to that file. | 19:03 |
| *** sparkycollier has joined #openstack-infra | 19:04 | |
| clarkb | mordred: will that provide helpful feedback on why $THING fails? I found the existing script to be handy for that | 19:04 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add pycadf project https://review.openstack.org/39225 | 19:04 |
| clarkb | mordred: it gives precise this version does not match other version info and not in 100k lines of logs | 19:04 |
| mordred | wait... | 19:05 |
| mordred | clarkb: I think we're talking about different jobs | 19:05 |
| clarkb | maybe | 19:05 |
| jeblair | clarkb: yes, and that confused me | 19:05 |
| dhellmann | jeblair: the goal is for us to be as strict as possible, anticipating a move later, so I appreciate this feedback -- let me know if I've missed anything else | 19:05 |
| mordred | gate-requirements-install | 19:05 |
| jeblair | strike everything i said | 19:05 |
| clarkb | ah | 19:05 |
| clarkb | I was thinking you are wanting to get the requirements checker | 19:05 |
| clarkb | because you said 'check-requirements' which is probably the job name | 19:06 |
| mordred | nope. purely I think we're about to have 3 different complete gates | 19:06 |
| jeblair | mordred: i agree that this can probably go away when all that is done. and with that in mind, it's not worth moving gate-requirements-install out of jjb. | 19:06 |
| mordred | yup. completely agree | 19:06 |
| dhellmann | jeblair: ugh, left out the acls | 19:06 |
| *** sdake has joined #openstack-infra | 19:06 | |
| *** sdake has quit IRC | 19:06 | |
| *** sdake has joined #openstack-infra | 19:06 | |
| jeblair | mordred: if you did want to move it, since it only gates one project, it would be fine to move it into requirements repo. | 19:07 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add pycadf project https://review.openstack.org/39225 | 19:08 |
| * dhellmann wonders how hard it would be to make a "start a new stackforge wizard" to generate all of these changes | 19:08 | |
| clarkb | I agree. I was confused before and now believe moving is ++ good assuming we want it | 19:08 |
| *** vipul-away is now known as vipul | 19:09 | |
| clarkb | dhellmann: so, it wouldn't be hard, but I tried to write the documentation with enough detail that people doing it understand the why, how, and what | 19:09 |
| clarkb | rather than just give me project, because eventually you will want $NEWTHING and having a little background is good | 19:09 |
| dhellmann | clarkb: yep, the docs are good, but I seem to keep running into missing pieces | 19:09 |
| dhellmann | the library release stuff, for example, but that may be unusual for stackforge | 19:09 |
| dhellmann | clarkb: https://review.openstack.org/#/c/39225/ | 19:10 |
| dhellmann | clarkb: and I'm not complaining, just thinking out loud | 19:10 |
| dhellmann | this is still way easier than learning how to use all of these separate tools myself :-) | 19:10 |
| mordred | yay! | 19:10 |
| clarkb | dhellmann: I think you mostly have it on the last iteration. One minor detail in review | 19:12 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-infra/config: Add pycadf project https://review.openstack.org/39225 | 19:13 |
| dhellmann | clarkb: thanks | 19:13 |
| dhellmann | clarkb: maybe the best approach is to add more walk-throughs to the docs, to expand on what you've done with stackforge? "So, you want to make a new openstack library" and "So, you're starting a new openstack server project" and that sort of thing | 19:14 |
| dhellmann | I can try to help with that, too, if you think it's worthwhile | 19:14 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Add a mirror33 slave node definition. https://review.openstack.org/39561 | 19:14 |
| clarkb | dhellmann: that may be helpful, treat at more a step by step with lots of info | 19:14 |
| dhellmann | clarkb: right | 19:15 |
| dhellmann | because the basic steps you have for a stackforge project are very clear and easy to follow | 19:15 |
| mordred | honestly, I thnk we could do with more of those in general | 19:16 |
| mordred | we've got more people wanting to do things self-serve outside of stackforge (Which is great) but sometimes a stackforge howto is a great way to get started | 19:17 |
| mordred | or - I've forgotten the steps | 19:17 |
| jeblair | yeah, that howto box is awful small. :) | 19:17 |
| mordred | :) | 19:17 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Update elasticsearch custodial crons. https://review.openstack.org/39562 | 19:17 |
| mriedem | has something changed recently to default tempest/nose to dump debug level messages to the console? | 19:20 |
| mriedem | i'm not using devstack, and not setting up logging.conf | 19:20 |
| clarkb | mriedem: not aware of, but have an example? | 19:20 |
| *** pabelanger has quit IRC | 19:20 | |
| *** sandywalsh has quit IRC | 19:20 | |
| clarkb | oh this is local | 19:20 |
| * clarkb looks at jenkins logs | 19:21 | |
| mriedem | yeah | 19:21 |
| mriedem | clarkb: http://paste.openstack.org/show/42781/ | 19:21 |
| *** nicedice has quit IRC | 19:22 | |
| mriedem | i didn't see any changes to tempest since yesterday that seem related | 19:22 |
| tiamar | mordred, fungi : I got to run tests with run_tests, I had to add d2to1, nose and cffi to install_venv_common. d2to1 due python clients , nose due to nosehtmloutput req. and cffi due some other lib I don't recall. Also had to install libffi-dev (ubuntu) | 19:22 |
| clarkb | mriedem: looks like it. I think we should stop it | 19:22 |
| clarkb | sdague: ^ | 19:22 |
| *** nicedice has joined #openstack-infra | 19:22 | |
| clarkb | because its just noise | 19:22 |
| mriedem | clarkb: k, glad i'm not crazy...thought it was something to do with testr but i'm not running in a virtual env | 19:23 |
| clarkb | http://logs.openstack.org/26/39326/2/check/gate-tempest-devstack-vm-postgres-full/37599/console.html.gz that would also explain why my index for today is gigantic | 19:23 |
| annegentle | anyone here know when the next round of registration codes go out for ATCs? | 19:23 |
| fungi | tiamar: that sounds likely, since setuptools can't https to pypi through your proxy, preinstalling the things it wants gets you there | 19:23 |
| mriedem | clarkb: yup, looks the same to me locally | 19:23 |
| mriedem | clarkb: i'll get a bug opened | 19:24 |
| clarkb | mriedem: thanks | 19:24 |
| jeblair | reed: ^ question from annegentle | 19:24 |
| *** SergeyLukjanov has quit IRC | 19:24 | |
| annegentle | oh yeah reed will know | 19:24 |
| mriedem | clarkb: fyi https://bugs.launchpad.net/tempest/+bug/1207066 | 19:26 |
| uvirtbot | Launchpad bug 1207066 in tempest "Tempest is dumping debug-level messages to console output" [Undecided,New] | 19:26 |
| mriedem | thanks uvirtbot :) | 19:27 |
| *** SergeyLukjanov has joined #openstack-infra | 19:27 | |
| *** vipul is now known as vipul-away | 19:27 | |
| *** vipul-away is now known as vipul | 19:27 | |
| clarkb | mriedem: change I31b460e454123a4211c0a8a1dc91bbfd562ce81a with sha f4a9b0f55fa29f5e1a50777b0090335cb71188f5 seems to be at fault | 19:29 |
| clarkb | I will update the bug | 19:29 |
| mriedem | clarkb: ah, good call | 19:29 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Build a devstack-gate image with diskimage-builder https://review.openstack.org/38871 | 19:29 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Proof of concept of kexec/takeovernode https://review.openstack.org/38887 | 19:29 |
| mrodden | oslo prints to console...? | 19:30 |
| mordred | jeblair: ok - I think the above is now self-contined enough to read | 19:30 |
| reed | annegentle, already went yesterday | 19:30 |
| mordred | tiamar: which project? | 19:30 |
| reed | annegentle, next batch after next milestone | 19:30 |
| annegentle | reed: okie doke | 19:30 |
| tiamar | mordred: keystone | 19:31 |
| mordred | tiamar: and once you do the install_reqs, I could not expect you to need to add anything to your venv by hand | 19:31 |
| mordred | tiamar: what command did you run to add those things? | 19:31 |
| mriedem | mrodden: tempest was changed to use it's logging.conf.sample and i think that dumps to console | 19:31 |
| mriedem | mrodden: https://review.openstack.org/#/c/38879/9/etc/tempest.conf.sample | 19:31 |
| mordred | tiamar: pip install of the requirements.txt and test-requirements.txt files should cause pip to handle everything, and pip should know how to deal with proxies properly | 19:32 |
| tiamar | mordred: I added them to install_venv_common.py | 19:32 |
| clarkb | mriedem: bug updated | 19:32 |
| *** sandywalsh has joined #openstack-infra | 19:32 | |
| mordred | tiamar: that should absolutely not be required and would be a bug | 19:32 |
| fungi | mordred: the issue was that his https proxy was preventing setuptools from pulling in transitive dependencies | 19:33 |
| *** afazekas is now known as afazekas_zz | 19:33 | |
| mordred | fungi: no. | 19:33 |
| mordred | setuptools does not do that on a pip install -r requirements.txt | 19:33 |
| mordred | pip does | 19:33 |
| clarkb | mtreinish: ^ any chance you can look at bug 1207066 ? | 19:33 |
| uvirtbot | Launchpad bug 1207066 in tempest "Tempest is dumping debug-level messages to console output" [Undecided,New] https://launchpad.net/bugs/1207066 | 19:33 |
| *** dina_belova has quit IRC | 19:33 | |
| mrodden | mriedem: oic | 19:33 |
| mordred | although - I can concede that the client libs have likely not removed their d2to1 setup_requires, so that might be needed in install_venv_common as well for the time being | 19:34 |
| mordred | although I'd really like to get new client libs cut without that | 19:34 |
| * mordred starts next annoying project | 19:34 | |
| dhellmann | mordred: you need to open bugs for these things so the projects know to help you out | 19:35 |
| mordred | yeah | 19:35 |
| clarkb | dhellmann: we do open bugs, half the projects get on board, a small group question it and seem on board, and others seem just disinterested... | 19:35 |
| mordred | I really want to get the setuptools_git removing patch landed | 19:35 |
| clarkb | mordred: testr probably isn't going to happen for the rest of the projects at this point during havana | 19:35 |
| mordred | and then bump the underlying depend | 19:35 |
| mordred | clarkb: nope. I agree | 19:35 |
| dhellmann | clarkb: that's still only half that you have to do yourself, right? :-) | 19:36 |
| mordred | well - actually | 19:36 |
| mordred | if sdague and I can get this autoupdate thing going | 19:36 |
| mordred | when aligning on new versions gets hella easier | 19:36 |
| mordred | then | 19:36 |
| *** sandywalsh has quit IRC | 19:37 | |
| fungi | the current concept is that a change merging to openstack/requirements will cause new changes to be automagically proposed to other projects depending on the same package? | 19:37 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Replace setuptools_git with a smarter approach https://review.openstack.org/30522 | 19:39 |
| mordred | fungi: yes | 19:39 |
| mordred | fungi: and in order to land a change to openstack/requirements, syncing all the projects to the new version will have to work in devstack gate | 19:39 |
| fungi | ahh, that's the piece i was missing | 19:40 |
| fungi | okay, so a devstack gate job which edits the requirements of all those projects in-place and then integration-tests them | 19:40 |
| mordred | yup | 19:40 |
| fungi | neat! | 19:41 |
| mordred | so, setup_develop in devstack will run requireents/update.py first | 19:41 |
| * mordred should write the update-all-the-projects job | 19:41 | |
| *** jjmb has quit IRC | 19:42 | |
| *** vipul is now known as vipul-away | 19:42 | |
| mordred | dhellmann: https://review.openstack.org/30522 ? clarkb ? fungi ? anybody want to give love to the longest-lived-pbr-patch-evar? | 19:44 |
| *** vipul-away is now known as vipul | 19:44 | |
| dhellmann | mordred: sure | 19:44 |
| clarkb | mordred: https://jenkins.openstack.org/job/devstack-update-vm-image-hpcloud-az3/172/console can we just delete the openvz stuff? | 19:44 |
| mordred | clarkb: yes | 19:45 |
| * dhellmann cries a little inside every time he sees a global | 19:45 | |
| mordred | actually | 19:45 |
| clarkb | mordred: I think it is still failing because we do the other branches too | 19:46 |
| clarkb | not just master | 19:46 |
| *** jjmb has joined #openstack-infra | 19:47 | |
| fungi | mordred: did we want to go ahead and approve https://review.openstack.org/39558 since it's breaking requirements changes? | 19:47 |
| mordred | fungi: yes! | 19:47 |
| fungi | clarkb: did you want to peek at it first? | 19:48 |
| clarkb | sure | 19:48 |
| fungi | clarkb: approve at will if you're good with it | 19:48 |
| *** jjmb has quit IRC | 19:48 | |
| fungi | clarkb: once that merges we should be able to reverify https://review.openstack.org/39461 | 19:49 |
| clarkb | what does exists-action=w do? | 19:49 |
| mrodden | overwrite if the file exists already | 19:49 |
| fungi | clarkb: overwrites the existing destination, i believe | 19:49 |
| fungi | i remeber looking it up for the mirror script | 19:49 |
| *** jjmb has joined #openstack-infra | 19:49 | |
| fungi | remember | 19:49 |
| *** cppcabrera has joined #openstack-infra | 19:49 | |
| mrodden | otherwise pip will prompt | 19:49 |
| mrodden | bad news for automation | 19:50 |
| clarkb | fungi: mordred approved | 19:50 |
| *** sandywalsh has joined #openstack-infra | 19:51 | |
| openstackgerrit | A change was merged to openstack-infra/config: Change how we gate requirements installs https://review.openstack.org/39558 | 19:52 |
| fungi | i still need to get back to figuring out why testr list-tests seems to be directly invoking the system default python and not heeding the calling environment | 19:52 |
| *** cppcabrera has left #openstack-infra | 19:54 | |
| mordred | woot | 19:55 |
| SergeyLukjanov | are there any thoughts about adding project filter to zuul? | 19:56 |
| SergeyLukjanov | here is a draft - http://f.slukjanov.name/zuul/status.html | 19:57 |
| fungi | oh, or maybe it's still subunit2html.py at fault? its shebang refers directly to /usr/bin/python but we're invoking it as .tox/py33/bin/python .../subunit2html.py ... so i don't think the shebang should matter | 19:57 |
| anteaya | in -meeting the topic is set to driver certification (Meeting topic: cinder) | 19:57 |
| SergeyLukjanov | (I mean zuul dashoard) | 19:57 |
| anteaya | it looks like a bot is not botting properly | 19:57 |
| anteaya | whom shall I notify? | 19:58 |
| *** jjmb has quit IRC | 19:58 | |
| jeblair | anteaya: it just reset it back to the previous topic, which was probably set like that due to a netsplit | 19:59 |
| fungi | anteaya: i'll take a look | 19:59 |
| jeblair | anteaya: probably the topic just needs manual resetting | 19:59 |
| anteaya | ah okay | 19:59 |
| clarkb | fungi: shebang shouldn't matter in that case | 19:59 |
| anteaya | so is that us/-infra or someone else? | 19:59 |
| jeblair | anteaya: us | 19:59 |
| anteaya | grand | 19:59 |
| anteaya | thanks fungi | 19:59 |
| fungi | jeblair beat me to it | 20:01 |
| jeblair | fungi: no i didn't | 20:01 |
| fungi | oh, nevermind. saw you op yourself, but then a meeting started | 20:02 |
| clarkb | a meeting just started so it will have to wait | 20:02 |
| * clarkb goes back to reviewing that pbr change for mordred | 20:02 | |
| fungi | i'll check back in around 2100 and see if it still needs fixing | 20:02 |
| jgriffith | anteaya: bot disappeared half way through meeting on us | 20:02 |
| anteaya | jgriffith: :( | 20:02 |
| jgriffith | anteaya: couldn't endmeeting | 20:02 |
| jgriffith | lemme try again now | 20:03 |
| anteaya | jgriffith: figured it was something like that | 20:03 |
| anteaya | heat has the floor now | 20:03 |
| jeblair | jgriffith: i think the bot is working fine | 20:03 |
| jeblair | jgriffith: i believe it got restarted during your meeting | 20:03 |
| jeblair | probably due to a channel logging change merging | 20:03 |
| jgriffith | jeblair: hmm... so endmeeting, topic, startmeeting, none worked | 20:03 |
| jeblair | jgriffith: it's working now | 20:04 |
| *** dina_belova has joined #openstack-infra | 20:04 | |
| anteaya | guess bot is not acknowledging you as chair right now jgriffith | 20:04 |
| fungi | yeah, at that point the bot probably thought the meeting was stopped. will need to think about how we can handle eavesdrop reconfigurations without negatively impacting meetbot | 20:04 |
| *** jjmb has joined #openstack-infra | 20:04 | |
| fungi | but since after the restart it lost state about the meeting, it ignored endmeeting and so on because it thought there was no meeting | 20:05 |
| jeblair | SergeyLukjanov: nice! i think if you put the entry on the same line as the queue lengths, that might be nice (so it doesn't take up any more vertical space) | 20:05 |
| fungi | maybe meetbot should wait to reload until after endmeeting if a meeting is in progress. would probably require some more advanced signal handling than it has now. i'll file a bug so we don't forget | 20:07 |
| SergeyLukjanov | jeblair, done | 20:07 |
| mordred | SergeyLukjanov: yes! and I agree with jeblair | 20:07 |
| *** nayward has quit IRC | 20:09 | |
| clarkb | mordred: https://review.openstack.org/#/c/30522/ reviewed | 20:09 |
| SergeyLukjanov | I'll cleanup patch and send change request tomorrow with filtering | 20:09 |
| *** dina_belova has quit IRC | 20:09 | |
| * jeblair notes that "infra" works as a filter. :) | 20:09 | |
| jeblair | SergeyLukjanov: cool, thanks! | 20:10 |
| mordred | yay! I got a +1 from clark | 20:10 |
| mordred | now I only need one more victim... | 20:11 |
| *** SergeyLukjanov has quit IRC | 20:12 | |
| *** jjmb1 has joined #openstack-infra | 20:13 | |
| *** adalbas has quit IRC | 20:13 | |
| mordred | clarkb: I responded to your comments/questions | 20:14 |
| *** jjmb has quit IRC | 20:15 | |
| clarkb | mordred: thanks | 20:17 |
| *** jjmb1 has quit IRC | 20:18 | |
| *** hashar has joined #openstack-infra | 20:18 | |
| clarkb | I have neglected to find lunch today. I am going to do that before bad things happen due to a lack offood | 20:19 |
| clarkb | I plan on doing more code review after lunch and/or help fungi debug the py33 subunit testr stuff | 20:20 |
| clarkb | fungi: also worth noting I am not entirely convinced that check that lists the tests works at all. as storyboard ran into issues with it and testr is currently double counting tests | 20:20 |
| clarkb | fungi: it may not be so bad to just remove it | 20:20 |
| fungi | clarkb: appreciated. i haven't really started to debug it yet. my next step was going to be to start browsing testr source looking for embedded references to /usr/bin/python in naughty places | 20:21 |
| mordred | lifeless: ping if/when you get in re: ^^ | 20:21 |
| clarkb | fungi: I just feel semi responsible as the code has proven to be bad in other cases | 20:21 |
| mordred | lifeless: we're having a couple of systemic testr issues that it would be really nice to resolve | 20:21 |
| clarkb | fungi: and honestly tearing it out might be an improvement... | 20:22 |
| *** koolhead17 has quit IRC | 20:22 | |
| clarkb | anyways food now I need it | 20:22 |
| mordred | lifeless: and at the least, being pointed to where/how we might want to fix the issue would be a kindness | 20:22 |
| fungi | it's not stopping the jobs from succeeding, fwiw. it fails and generates a 0 count of found tests, so the run tests count is guaranteed to be greater | 20:22 |
| mordred | yeah. but the thing is - we should be able to tell testr that we would like it to fail if it founds 0 tests or something | 20:23 |
| mordred | instead of what we're doing right now | 20:23 |
| fungi | that too | 20:23 |
| fungi | mordred: clarkb: lifeless: the current reference point is http://logs.openstack.org/30/39330/1/gate/gate-pbr-python33/4/console.html.gz | 20:24 |
| mordred | well the using /usr/bin/python is jut plain evil | 20:26 |
| fungi | it looks like subunit2html.py may actually be calling something which is at fault, since no testr_results.html is created | 20:26 |
| mordred | oh - right | 20:27 |
| fungi | but i can't quite determine where the "/usr/bin/python: No module named subunit" is actually coming from in that case | 20:27 |
| mordred | didn't we already go down this path and figure out that the problem is subunit2html ? | 20:27 |
| clarkb | fungi: really quick, you can go into one of those workspaces and run it by hand to see if you can get more info | 20:28 |
| clarkb | and now I am AFK | 20:28 |
| fungi | i fixed some python2-isms in subunit2html.py, before which things weren't getting this far with that step | 20:28 |
| fungi | good call, clarkb. i'll give that a shot | 20:28 |
| *** krtaylor has quit IRC | 20:29 | |
| fungi | so anyway, because subunit2html.py was complaining about stuff which wasn't py3k-compatible i'm pretty certain it's getting invoked with the right interpreter. but either it's calling something or something is being run after it which is using the system default interpreter instead | 20:29 |
| fungi | mordred: clarkb: lifeless: yeah, so subunt2html.py failing to generate output seems unrelated. the error is being emitted by testr (probably one of its subcommands): http://paste.openstack.org/show/42784/ | 20:34 |
| lifeless | mordred: fungi clarkb etc - I am here. | 20:35 |
| mordred | lifeless: woot! probably just starting with the latest paste | 20:35 |
| mordred | lifeless: is a good place. I think that's more important than the other thing | 20:35 |
| fungi | yeah, simple reproduction there | 20:35 |
| lifeless | you don't have subunit installed in your py33 environment. | 20:35 |
| dkranz | lifeless: I figured out why I was getting errors but it seemed like a bug that the unittest. stuff showed up as a listed test. | 20:35 |
| mordred | lifeless: we do | 20:35 |
| lifeless | or python is mis-set | 20:36 |
| mordred | lifeless: we don't have it installed in the global env, which is what /usr/bin/python is | 20:36 |
| lifeless | note: '/usr/bin/python: No module named subunit' | 20:36 |
| lifeless | thats python 2 | 20:36 |
| mordred | right | 20:36 |
| mordred | yes. we agree with you | 20:36 |
| mordred | now - why is testr trying to call that when we run list-tests | 20:36 |
| fungi | lifeless: .tox/py33/bin/pip freeze says python-subunit==0.0.13 | 20:36 |
| lifeless | it's running | 20:36 |
| lifeless | running=OS_STDOUT_CAPTURE=1 OS_STDERR_CAPTURE=1 OS_TEST_TIMEOUT=60 ${PYTHON:-python} -m subunit.run discover -t ./ . --list | 20:36 |
| lifeless | note that if PYTHON is unset it will be invoking 'python' | 20:37 |
| mordred | oh ffs | 20:37 |
| fungi | so the question is why, when run from an explicit interpreter is it looking to the environment to find the interpreter? | 20:37 |
| lifeless | presumaly you have this set in .testr.conf to let you override python. | 20:37 |
| fungi | we can activate the virtualenv i guess | 20:37 |
| mordred | fungi: yes, I believe we're going to have to do that - OR | 20:37 |
| mordred | fungi: we could do PYTHON=.tox/py33/bin/python | 20:38 |
| lifeless | export PYTHON=python3 | 20:38 |
| mordred | no | 20:38 |
| lifeless | should do it | 20:38 |
| mordred | export PYTHON=.tox/py33/bin/python | 20:38 |
| mordred | it's in a venv, there is no python3 | 20:38 |
| lifeless | mordred: you don't have the stuff in the path ? | 20:38 |
| openstackgerrit | SlickNik proposed a change to openstack-infra/config: Allow trove-core to tag python-troveclient and to push a new version to pypi https://review.openstack.org/39576 | 20:38 |
| lifeless | mordred: huh? | 20:38 |
| mordred | lifeless: this is right after having run tests in a venv | 20:38 |
| mordred | via tox | 20:38 |
| mordred | we get subunit and friends from the tox venv | 20:39 |
| fungi | yep, as expected it works when the evironment points to the interpreter we want. i guess i just assumed there was some reason we were manually specifying the interpreter in the command line and avoiding munging the environment | 20:39 |
| lifeless | oh man, thats going to confuse the hell out of everyone. PEP394 or whatever it is eeek | 20:39 |
| mordred | lifeless: no one other than use touches this | 20:39 |
| mordred | us | 20:39 |
| lifeless | mordred: I mean broadly | 20:40 |
| lifeless | mordred: system python3 being python3, venv python3 being python. | 20:40 |
| mordred | I don't konw what you mean about people being confused | 20:40 |
| anteaya | o/ | 20:40 |
| mordred | ah | 20:40 |
| mordred | gotcha | 20:40 |
| lifeless | mordred: is going to hurt brains | 20:40 |
| mordred | thank you | 20:40 |
| mriedem | looking for some opinion on a tempest bug: https://bugs.launchpad.net/tempest/+bug/1206694 | 20:40 |
| uvirtbot | Launchpad bug 1206694 in tempest "test_neutron_dhcp_agent_list_hosting_net hard-codes the network name" [Undecided,New] | 20:40 |
| mriedem | would anyone prefer re-using this tempest.conf option for that issue? https://github.com/openstack/tempest/blob/master/etc/tempest.conf.sample#L90 | 20:41 |
| mordred | lifeless: it would be nice if testr would set PYTHON to sys.executable in the env it passes if it's not set in the env alread | 20:41 |
| mriedem | or adding a new conf option in the cli section? | 20:41 |
| mordred | lifeless: it would be friendlier to venvs | 20:41 |
| mordred | and venv operation | 20:41 |
| mordred | lifeless: also, I'm glad that the python in the venv is not python3 - it would make it MUCH harder to write general scripts | 20:44 |
| fungi | i may have misread pep-394 the other day, but i thought the takeaway there was that "python" as an interpreter name referred to the one and only python in your path, or python2 if you have both it and python3, until such time as python3 is a viable default | 20:44 |
| *** dina_belova has joined #openstack-infra | 20:44 | |
| *** jjmb has joined #openstack-infra | 20:44 | |
| mordred | lifeless: so - I think we grok that thing - thank you - the other thing is more of a "can you point us in the right direction" thing | 20:45 |
| fungi | so given that the virtualenv in question contains only python3 and no python2, it seems safe to me to refer to it as python (unqualified) | 20:45 |
| sdague | mordred: we have a good pip freeze from a good run? | 20:45 |
| dstufft | fungi: ``python`` is just the default python | 20:45 |
| dstufft | python2 or python3 | 20:45 |
| dstufft | (or python4 in 10 years!) | 20:45 |
| mordred | lifeless: which is - how can we add something to testr so that we don't have to post-sanitycheck a run to make sure it ran tests | 20:45 |
| dstufft | whichever is ystem default (or in venv case, venv default) | 20:45 |
| fungi | dstufft: yeah, more topical is what your default python should be | 20:45 |
| mordred | yes | 20:46 |
| mordred | and awesome | 20:46 |
| dstufft | Python3 :trollmode: | 20:46 |
| mordred | sdague: pip freeze from a good run of ? | 20:46 |
| jeblair | wait, we can't just use /usr/bin/parrot? | 20:46 |
| sdague | http://logs.openstack.org/05/35705/14/check/gate-tempest-devstack-vm-full/37946/console.html seems to indicate the global requirements update is doing a thing now... it just is doing it with a version of deps that don't work | 20:46 |
| dstufft | actually | 20:46 |
| dstufft | PyPy3k | 20:46 |
| dstufft | best python | 20:46 |
| sdague | mordred: devstack gate run | 20:46 |
| mordred | sdague: awesome! | 20:46 |
| mordred | one sec | 20:46 |
| sdague | I expect that because we don't really enforce global requirements, it's just not right | 20:47 |
| fungi | dstufft: the discussion evolved from determining why a tool, when invoked with a specific python interpreter, should go looking for $PYTHON in the environment before calling children | 20:47 |
| lifeless | mordred: not sure I follow | 20:47 |
| lifeless | mordred: testr was erroring in this case | 20:47 |
| mordred | lifeless: different case | 20:47 |
| mordred | lifeless: it's the reason we have this script at all | 20:47 |
| mordred | so fixing it is thing one | 20:47 |
| mordred | but it shouldn't be needed | 20:47 |
| lifeless | mordred: so the reason you have this script is because testr /wasn't/ checking that test listing worked properly. | 20:48 |
| lifeless | mordred: I filed that as a bug and fixed it. | 20:48 |
| mordred | awesome | 20:48 |
| mordred | so you are contending that we should be able to delete this script and be fine now? | 20:48 |
| lifeless | mordred: a related thing is that you might want as policy to say 'there must be /some/ tests' | 20:48 |
| mordred | that would be great | 20:48 |
| lifeless | or there must be 'at least N tests' | 20:48 |
| lifeless | or 'at least Y tests must have claimed to run when we ran tests' | 20:48 |
| mordred | wait - we still might be talking about slightly different things | 20:49 |
| lifeless | I have nothing built-in to testr to do those; all three are policy and shouldn't be defaulted IMO. | 20:49 |
| mordred | if testr runs and find no tests, does it exit with success or failure? | 20:49 |
| *** dina_belova has quit IRC | 20:49 | |
| lifeless | I'd be happy for something to be built in to let you express those policies. | 20:49 |
| _TheDodd_ | clarkb: fungi: mordred: jeblair: Did someone ping me, like, an hour ago or so? | 20:50 |
| lifeless | if testr runs, and test listing works [or is not needed] and no tests are in the output, it considers that a success | 20:50 |
| _TheDodd_ | It's not showing up in my IRC window anymore. | 20:50 |
| mordred | ok. I want to tell it that's a failure | 20:50 |
| lifeless | and the test runners all exit 0 ^ | 20:50 |
| mordred | can you point me in the right direction? | 20:50 |
| jeblair | _TheDodd_: i think clarkb pointed out a change that uses partial-bug | 20:50 |
| mordred | I'm fine with it being a command line option or a config option | 20:50 |
| jeblair | _TheDodd_: that he wrote | 20:50 |
| _TheDodd_ | Thanks, jeblair. | 20:50 |
| mordred | sdague: I do not have a pip freeze ... I think we only produce those in certain situations? | 20:50 |
| sdague | is there a good way I can make one? | 20:51 |
| mordred | jeblair: do you remember how we get a pip freeze out of a devstack run? | 20:51 |
| sdague | even locally | 20:51 |
| mordred | sdague: I _think_ there is an env var you can set for a local run | 20:51 |
| *** jjmb has quit IRC | 20:51 | |
| mordred | sdague: TRACK_DEPENDS | 20:51 |
| lifeless | oh yay... what broke my 3.2 venvs... | 20:51 |
| lifeless | Could not find platform independent libraries <prefix> | 20:51 |
| lifeless | Could not find platform dependent libraries <exec_prefix> | 20:51 |
| mordred | sdague: if [[ $TRACK_DEPENDS = True ]]; then | 20:51 |
| sdague | oh, nice | 20:51 |
| sdague | great | 20:52 |
| mordred | sdague: I do not know when the last time it was used is... | 20:52 |
| sdague | well, I'll run local and find out | 20:52 |
| lifeless | mordred: so, which of the following sounds like it expresses your intent most: | 20:52 |
| lifeless | 'at least 1 test was executed by all the backends combined' | 20:52 |
| lifeless | 'at least 1 test was executed by any backend we spawned' | 20:52 |
| mordred | I don't understand the difference in those | 20:53 |
| lifeless | 'all tests we asked a backend to run were run by it && at least 1 test was run by any backend' | 20:53 |
| sdague | lifeless: is there a reason a backend would spawn if it had no tests to run? | 20:53 |
| lifeless | 'at least 1 test was returned by listing' | 20:53 |
| lifeless | sdague: in non-parallel mode with no test filters we just ask the backend to run and do its thing | 20:53 |
| mordred | 'at least 1 test was returned by listing and was run by some backend' | 20:54 |
| lifeless | mordred: listing only happens in parallel mode | 20:54 |
| sdague | it seems like there should be a sanity check there, we had that issue before where testr choked hard enough that it stopped reporting back, so it seems that any backend needs to return at least 1 result | 20:54 |
| mordred | lifeless: great. so let me try again | 20:54 |
| mordred | lifeless: 'at least 1 test was run' | 20:54 |
| sdague | otherwise the tools can fail on us silently | 20:54 |
| sdague | someone needs to watch the watchers | 20:55 |
| *** zaro0508 has joined #openstack-infra | 20:55 | |
| mordred | right. that's actually the reason we have the wrapper script | 20:55 |
| mordred | :) | 20:55 |
| mordred | lifeless: I'd also like ''all tests we asked a backend to run were run by it && at least 1 test was run by any backend' | 20:56 |
| lifeless | sdague: so, it's totally legitimate for backends to find they can't run a test for some reason - for instance, it's been deleted. | 20:56 |
| sdague | lifeless: why is that legitimate? | 20:56 |
| mordred | load_tests filtering | 20:57 |
| mordred | ? | 20:57 |
| mordred | although wouldn't that affect the list-tests too? | 20:57 |
| lifeless | sdague: in the gap between testr run --parallel doing a list and spawning the backends, someone might save a file deleting a test | 20:57 |
| sdague | if the test files got deleted out from under a test runner, that should be a fail | 20:57 |
| mrodden | so manage_projects is designed to be run on review.openstack.org ? | 20:57 |
| mrodden | the host | 20:57 |
| mordred | ah. yeah. we do not care to support that use case, although I understand that you need to in testr | 20:57 |
| lifeless | sdague: one use case for testr is running under inotify | 20:58 |
| mordred | I would like to fail the test run in that case - or tell testr to return unhappy | 20:58 |
| jeblair | mordred, fungi, clarkb: anyone want to review these zuul changes: https://review.openstack.org/#/c/39387/ https://review.openstack.org/#/c/38942/ ? | 20:58 |
| lifeless | sdague: and reporting back through a background widget | 20:58 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Pass testr and subunit2html.py a PYTHON envvar. https://review.openstack.org/39580 | 20:58 |
| fungi | jeblair: very much so | 20:58 |
| sdague | lifeless: ok, but that seems like that should be optional behavior | 20:58 |
| mordred | lifeless: so I want to be able to tell testr to consider not running tests to be a failure, and to consider backends not running all the tests it asked it to to be a failure | 20:58 |
| lifeless | sdague: so what we're doing is fine tuning the definition of what openstack wants, so I can point mordred-the-cannon at the right code | 20:59 |
| sdague | it's really hard to verify that we ran what we wanted it to run | 20:59 |
| sdague | if we silent pass on a changing repo under us :) | 20:59 |
| mordred | I'd LIKE to say that we want to verify that we ran and got a result for every single test we discovered | 20:59 |
| mordred | and that we found at least 1 test | 20:59 |
| sdague | yeh, that sounds right | 21:00 |
| clarkb | jeblair: yes, I will put them at the top of my list | 21:00 |
| mordred | I wouldn't mind an extra option to indicate the low-water mark for what that number of tests should be | 21:00 |
| mordred | so, "that we found at least $NUMTESTS tests" where NUMTESTS defaults to 1 | 21:01 |
| mordred | or something | 21:01 |
| clarkb | jeblair: sdague https://review.openstack.org/#/c/39571/ can you guys give that one a little priority? | 21:01 |
| mordred | that way we could occasionally set a new low water mark in the tree | 21:01 |
| lifeless | so I'm torn on how to express these. let me get cynthia up and to kindy and I'll noodle on it in my head in the meantime | 21:01 |
| mordred | lifeless: cool. I appreciate that | 21:01 |
| *** dkranz has quit IRC | 21:03 | |
| mordred | jeblair: looking | 21:04 |
| clarkb | mordred: I approved one of them. I will hold off on approving the second so that you can review | 21:05 |
| mordred | reading it now | 21:06 |
| clarkb | the second has also not been reviwed by fungi so double holding off | 21:06 |
| SlickNik | annegentle: have a minute? | 21:06 |
| mordred | jeblair: TOTALLY not important or even a code review - but remind me to show you self.addDetails at some point | 21:06 |
| annegentle | SlickNik: yup | 21:06 |
| SlickNik | annegentle: Wanted to get a link to http://docs.openstack.org/developer/trove/ on the http://docs.openstack.org/ page. | 21:06 |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: Add python26 multiline string assertion back. https://review.openstack.org/38538 | 21:06 |
| SlickNik | annegentle: not sure what the process was, so wanted to ask/check with you. | 21:07 |
| annegentle | SlickNik: sure, just patch openstack-manuals/www/index.html | 21:07 |
| fungi | jeblair: i added a couple quick questions to 38942 | 21:07 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Make a test more robust https://review.openstack.org/39387 | 21:08 |
| jeblair | fungi: answered: two yeses. | 21:08 |
| SlickNik | annegentle: Okay, will do that. Thanks for the info! | 21:08 |
| fungi | poof! aprv | 21:08 |
| jeblair | fungi: yeah, you left clear instructions. :) | 21:08 |
| fungi | that i did! | 21:09 |
| mordred | fungi: NEAT! the py33 gate caught a python3 problem in pbr! | 21:09 |
| mordred | or in a patch, I should say | 21:09 |
| jeblair | i'm going to continue merging zuul-related changes unless anyone wants to review anything else in https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul,n,z | 21:09 |
| fungi | mordred: see, it's paying for itself already ;) | 21:10 |
| mordred | jeblair: fine by me | 21:10 |
| *** hashar has left #openstack-infra | 21:10 | |
| clarkb | jeblair: I think I have managed to review all if not most of them and none had glaring issues | 21:10 |
| clarkb | jeblair: works for me. | 21:10 |
| fungi | feel free, though i'm trying to skim as many as i can | 21:11 |
| fungi | i think i already hit most of them | 21:11 |
| jeblair | fungi: we can always revert/fix-up after the fact. | 21:11 |
| mordred | clarkb: oy | 21:11 |
| mordred | clarkb: file_list = file_list.split(b'\x00') | 21:11 |
| clarkb | mordred: ? | 21:11 |
| mordred | file_list is a utf-8 string | 21:11 |
| mordred | any thoughts? | 21:11 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Stats: break out job runtimes and counts by result https://review.openstack.org/38942 | 21:12 |
| clarkb | mordred: ya, split(b'\x00'.encode('utf8')) | 21:12 |
| clarkb | I think | 21:12 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Copy tempest.log to logs dir after tempest run https://review.openstack.org/39583 | 21:12 |
| *** rfolco has quit IRC | 21:12 | |
| mordred | god. gross | 21:12 |
| mordred | ok | 21:12 |
| clarkb | mordred: test that in an interactive python interpreter | 21:12 |
| fungi | noticed a typo in https://review.openstack.org/39130 but i'll send a fix through while i'm thinking about it since it's already approved | 21:13 |
| clarkb | I always get encode and decode direction mixed up. same issue with deflate and inflate | 21:13 |
| *** krtaylor has joined #openstack-infra | 21:14 | |
| openstackgerrit | A change was merged to openstack-infra/zuul: Use the public gear API to iterate over jobs in the queue https://review.openstack.org/39159 | 21:15 |
| mordred | clarkb: yup. decode | 21:15 |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Replace setuptools_git with a smarter approach https://review.openstack.org/30522 | 21:15 |
| clarkb | they need from_bytearray and to_bytearray aliases :) | 21:16 |
| clarkb | the kernel makes direction of things easy because they do that | 21:16 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/zuul: Fix parameter name typo in NEWS.rst. https://review.openstack.org/39584 | 21:16 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Pass the Job to the parameter function https://review.openstack.org/39130 | 21:17 |
| sdague | clarkb: I'll push this - https://review.openstack.org/#/c/39571/ to the gate as soon as the test results are clean | 21:17 |
| clarkb | sdague: thank you | 21:17 |
| jeblair | fungi: thx | 21:18 |
| sdague | though it might come back while I'm biking home, so maybe a little later than when it comes back :) | 21:18 |
| mtreinish | sdague: I just changed the commit message for the second version | 21:19 |
| mtreinish | the results should be identical to the first revision | 21:19 |
| mtreinish | which were clean | 21:19 |
| * fungi needs to do something about food, and then will come back to reviews | 21:20 | |
| sdague | mtreinish: ok, you sold me | 21:21 |
| sdague | TO THE GATE! | 21:21 |
| clarkb | \o/ | 21:22 |
| clarkb | er I mean CHARGE! | 21:22 |
| sdague | hmmm... something isn't right with oslo.config install apparently on devstack | 21:22 |
| mtreinish | clarkb: ok then you're on deck with https://review.openstack.org/39583 | 21:23 |
| mtreinish | so we don't lose the logs | 21:23 |
| openstackgerrit | A change was merged to openstack-infra/zuul: Fix parameter name typo in NEWS.rst. https://review.openstack.org/39584 | 21:23 |
| clarkb | mtreinish: reviewing | 21:23 |
| clarkb | mtreinish: reviewed | 21:26 |
| mordred | sdague: any luck on requirements - or anything you want me to pick up? | 21:26 |
| *** zaro0508 has quit IRC | 21:26 | |
| mriedem | has anyone ever seen a patch connected to a bug get merged but launchpad didn't change the state of the bug report to 'fix committed'? | 21:28 |
| openstackgerrit | A change was merged to openstack-infra/config: Add new jenkins masters. https://review.openstack.org/38885 | 21:29 |
| jeblair | mriedem: make sure 'hudson-openstack' is in the bug group for the project | 21:29 |
| openstackgerrit | A change was merged to openstack-infra/config: Switch Zuul/JJB to use LOG_PATH https://review.openstack.org/39133 | 21:30 |
| sdague | mordred: no luck yet, I'm getting a funny oslo die locally though | 21:30 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Copy tempest.log to logs dir after tempest run https://review.openstack.org/39583 | 21:30 |
| sdague | I wonder if oslo.config already broke from a4 => a9 | 21:30 |
| mordred | sdague: you know what - once we get to the bottom of it - we're going to be in magical pony land | 21:30 |
| mordred | wow. we so have to get it in the gate | 21:31 |
| sdague | I think there are a few more trolls under the bridge | 21:31 |
| mriedem | jeblair: i'm not sure how to check that, it was a nova patch | 21:31 |
| sdague | before magical pony land | 21:31 |
| mriedem | jeblair: https://review.openstack.org/#/c/35531/ | 21:31 |
| jeblair | mriedem: ah, then it may have something to do with changes recently merged to the bug update script. we'll look into it. | 21:32 |
| mordred | jeblair: eventually, maybe we can add a gearman interface to storyboard and put bug events on the bus... | 21:32 |
| mriedem | jeblair: thanks, this merged about 3 week sago | 21:32 |
| sdague | mordred: that oslo.config break is pure speculation right now | 21:33 |
| jeblair | mriedem: oh, then it wouldn't be that either. | 21:33 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Specify gerrit triggers https://review.openstack.org/39585 | 21:33 |
| jeblair | 21:30 < openstackgerrit> A change was merged to openstack-infra/config: Switch Zuul/JJB to use LOG_PATH https://review.openstack.org/39133 | 21:34 |
| jeblair | btw ^ that is a possibly-break-everything change. | 21:34 |
| *** cp16net is now known as cp16net|away | 21:35 | |
| jeblair | i'm restarting salt-master because it's using 66% of puppetmaster's ram | 21:36 |
| clarkb | ok | 21:36 |
| *** cp16net|away is now known as cp16net | 21:36 | |
| clarkb | I will restart the logstash gearman client once puppet updates that script | 21:36 |
| jeblair | #status log restarted salt-master due to high ram usage (1.3GB) | 21:37 |
| clarkb | jeblair: do you know if the LOG_PATH change got in at 2130 or are you on the puppet master to pull it in manually? | 21:37 |
| jeblair | clarkb: i'm going to pull it manually if needed | 21:37 |
| clarkb | ok | 21:37 |
| jeblair | clarkb: it's up to date | 21:38 |
| *** mriedem has quit IRC | 21:38 | |
| sdague | mordred: no, oslo works in the gate | 21:38 |
| sdague | so... something else is up with my machine | 21:39 |
| sdague | speaking of which is there a way to actually test this - https://review.openstack.org/#/c/39508/1/modules/openstack_project/files/zuul/layout.yaml ? | 21:40 |
| jeblair | sdague: test what about it? | 21:40 |
| sdague | that those jobs will actually run with oslo? | 21:41 |
| sdague | though I guess they are running without | 21:41 |
| sdague | so it shouldn't be an issue | 21:41 |
| jeblair | sdague: the jobs are alreaby being run elsewhere, so the change to add oslo to d-g will test that the jobs work | 21:41 |
| jeblair | (with oslo) | 21:41 |
| sdague | ok | 21:41 |
| sdague | well that already landed | 21:41 |
| *** fifieldt has joined #openstack-infra | 21:41 | |
| sdague | so we should probably bring that in | 21:41 |
| *** sparkycollier has quit IRC | 21:41 | |
| sdague | or at least review it | 21:41 |
| jeblair | sdague: and then separately, adding them to be triggered by oslo should not impact anything, if that's already working | 21:42 |
| *** woodspa has quit IRC | 21:42 | |
| sdague | yep | 21:42 |
| jeblair | clarkb: i think puppet runs ot zuul and jenkins timed out talking to puppetmaster; possibly due to impact from salt. | 21:43 |
| jeblair | it's really crazy. | 21:43 |
| clarkb | jeblair: wow | 21:44 |
| clarkb | jeblair: perhaps try a puppet agent --test by hand to see? | 21:44 |
| jeblair | i'm stopping salt... | 21:44 |
| jeblair | clarkb: that's what i've been running | 21:44 |
| jeblair | #status log stopped salt master due to high load. | 21:45 |
| jeblair | clarkb, mordred: notice: /Stage[main]/Pip/Package[setuptools]/ensure: created | 21:45 |
| jeblair | clarkb, mordred: ^ that happens on every puppet run on every host now | 21:45 |
| SlickNik | annegentle: https://review.openstack.org/#/c/39587/1/www/developer/openstack-projects.html when you get a chance. Thanks! | 21:45 |
| *** ^d has quit IRC | 21:45 | |
| mordred | are we using salt for anything yet? | 21:46 |
| *** dina_belova has joined #openstack-infra | 21:46 | |
| mordred | jeblair: and I'll look in to the setuptools thing | 21:46 |
| jeblair | mordred: we use it to fix things sometimes | 21:46 |
| jeblair | mordred: thanks | 21:46 |
| jeblair | in my unscientific test, after stopping salt master, the next puppet run on zuul ran quickly and did not timeout connecting to puppetmaster | 21:47 |
| jeblair | the zuul config has updated, jjb is running now (it may take a while) | 21:47 |
| jeblair | i would expect jobs launched after this point that complete before jjb finishes to break. | 21:48 |
| jeblair | jjb timed out; running again manually | 21:49 |
| *** changbl has quit IRC | 21:50 | |
| *** pentameter has quit IRC | 21:50 | |
| *** dina_belova has quit IRC | 21:50 | |
| jeblair | it's taking about 4 seconds to update each job. we're probably updated about 500 of them, which would take 30 minutes. | 21:50 |
| clarkb | wow | 21:50 |
| clarkb | last time we changed all of the jobs I had to run it by hand too | 21:51 |
| *** mrodden has quit IRC | 21:51 | |
| *** lcestari has quit IRC | 21:53 | |
| clarkb | FYI I have stopped puppet on logstash.o.o. ZMQ does not like it when you try to connect to names it cannot resolve. If it can't actually connect it is fine and happy and will keep trying until it can connect but apparently if it can't resolve a name that is fatal | 21:55 |
| jeblair | clarkb: ugh | 21:55 |
| clarkb | so I have removed the two new jenkins masters in that config by hand and restarted the daemon. When we have the masters up I will uncomment it. or I will fix the name resolution thing later | 21:55 |
| clarkb | but I don't want to worry about this too much right now in case more important things go sideways in reaction to this change | 21:56 |
| clarkb | jeblair: ya :( I fully expected it to just work because zmq does enough hand holding that you don't expect issues like this | 21:57 |
| *** pabelanger has joined #openstack-infra | 21:59 | |
| *** dhellmann is now known as dhellmann_ | 21:59 | |
| anteaya | jeblair: are you sensing that puppet and salt are not curently working well together? | 22:00 |
| anteaya | or is salt doing something unexpected that is tying up resources? | 22:01 |
| jeblair | anteaya: salt is tying up resources | 22:01 |
| anteaya | okay, is the reason for the resource consumption known? | 22:01 |
| anteaya | or unknown so far? | 22:02 |
| clarkb | jeblair: http://logs.openstack.org/85/39585/1/check/gate-config-layout/2683/console.html did that check run before the zuul change to add triggers merged? | 22:02 |
| *** mrodden has joined #openstack-infra | 22:04 | |
| *** burt has quit IRC | 22:05 | |
| lifeless | mordred: ok so | 22:05 |
| *** vijendar has quit IRC | 22:06 | |
| lifeless | 'detect missing tests' is perhaps best as a run command line argument | 22:06 |
| jeblair | clarkb: the zuul change hasn't merged | 22:06 |
| *** rfolco has joined #openstack-infra | 22:06 | |
| clarkb | jeblair: that'll do it | 22:06 |
| jeblair | clarkb: believe it or not, i'm only breaking one thing at a time right now. :) | 22:06 |
| * mordred believes you | 22:06 | |
| lifeless | 'require at least one test' I am thinking can be a config file option | 22:07 |
| *** thomasm has quit IRC | 22:07 | |
| clarkb | jeblair: just one? :) | 22:07 |
| lifeless | but | 22:07 |
| lifeless | regex filtering interacts with it | 22:08 |
| lifeless | what should 'testr run slow' do if there are no tests matching slow, and 'require at least one test' is set? | 22:08 |
| lifeless | we'll encounter this with tempest I think | 22:08 |
| EmilienM | mordred: hi, I would like to transfer https://github.com/enovance/puppet-heat to StackForge, and I can see you are the owner :) | 22:08 |
| lifeless | one way we could do it is only apply 'require one test' to the listing step; | 22:09 |
| lifeless | mordred: I'd like to mark these as experimental initially | 22:09 |
| mordred | EmilienM: hi! luck for you - we have a guide for this: http://ci.openstack.org/stackforge.html | 22:09 |
| mordred | EmilienM: you'll have tons of fun! | 22:10 |
| EmilienM | mordred: thx a lot, sorry for missing the doc before. | 22:10 |
| clarkb | lifeless: it would fail if no tests match slow | 22:10 |
| mordred | EmilienM: no worries! | 22:10 |
| mordred | lifeless: so, I'm actually fine with these things not affecting users running stuff interactively, tbh | 22:10 |
| clarkb | lifeless: isn't that the desired behavior? you ask for a tool to do something, if it cannot do that something it should error | 22:11 |
| mordred | yeah. I tend to agree with clark - I think an exit 0 if you've asked for it to do a thing is confusing. if I run testr run slow and there are none, that' a failure | 22:12 |
| mordred | BUT | 22:12 |
| mordred | that's not my immediate concern | 22:13 |
| mordred | and I'm fine setting something in the env to switch to the behavior | 22:13 |
| mordred | because I can concede that the command line ui might have different expectations | 22:13 |
| lifeless | mordred: CLI vs env variable doesn't concern me per se | 22:14 |
| *** andrew_plunk has joined #openstack-infra | 22:14 | |
| mordred | me either | 22:14 |
| lifeless | mordred: just want it a) documented and b) documented as experimental | 22:14 |
| mordred | yah | 22:14 |
| *** andrew_plunk has left #openstack-infra | 22:14 | |
| mordred | I have no problem with that | 22:14 |
| lifeless | mordred: doing a and b as CLI options is somewhat more discoverable than env variables. | 22:15 |
| lifeless | mordred: as for the place, testcommand + commands/run | 22:15 |
| *** weshay has quit IRC | 22:18 | |
| openstackgerrit | Steve Baker proposed a change to openstack-infra/config: Create new repo to host legacy heat-cfn client. https://review.openstack.org/38226 | 22:18 |
| openstackgerrit | Emilien Macchi proposed a change to openstack-infra/config: Add heat puppet module to stackforge https://review.openstack.org/39594 | 22:24 |
| dhellmann_ | jeblair: does https://review.openstack.org/#/c/39225/ look ok now? | 22:27 |
| *** dhellmann_ is now known as dhellmann | 22:27 | |
| jeblair | mordred: looks like a hacking bug: https://jenkins.openstack.org/job/gate-tempest-pep8/4567/console | 22:32 |
| EmilienM | mordred: if I understand well, to be core I'll need to file a bug on https://bugs.launchpad.net/openstack-ci/+filebug , right ? | 22:32 |
| mordred | jeblair: looking | 22:32 |
| mordred | jog0: ^^ | 22:32 |
| jeblair | dhellmann: i may have to get back to you on that later | 22:32 |
| dhellmann | jeblair: ak | 22:32 |
| dhellmann | or ack or ok :-) | 22:32 |
| *** rnirmal has quit IRC | 22:33 | |
| *** prad has quit IRC | 22:33 | |
| mordred | jeblair: yeah - looks like an issue with hacking trying to do the import test | 22:33 |
| mordred | I thought we had that test disabled for tempest | 22:34 |
| jeblair | mordred: i particularly like the "raise exception; exit 0" aspect. | 22:34 |
| * jog0 checks | 22:34 | |
| clarkb | EmilienM: its not completely necessary. Submitting the chagne in gerrit will create the group then we can add you | 22:34 |
| clarkb | EmilienM: you can just ping us to do that, however, are you intending on collaborating with the existing stackforge puppet stuff? | 22:35 |
| clarkb | EmilienM: you may just want to use their permissions | 22:35 |
| mordred | jeblair: :) | 22:36 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Handle log paths in logstash gearman client better https://review.openstack.org/39597 | 22:36 |
| EmilienM | clarkb: I'm the main author of the module | 22:36 |
| clarkb | jeblair: ^ and that fixes one other small problem with the log stuff. I have applied locally because puppet has been disabled there | 22:36 |
| *** rfolco has quit IRC | 22:36 | |
| EmilienM | and I want to transfer it to StackForge to collaborate with OpenStack community | 22:36 |
| clarkb | EmilienM: ya so you might want to get in touch with the people doing the other stackforge puppet modules | 22:37 |
| EmilienM | but since I'm the author, and there is still some work to do, I would love to be core on it, since I'm going to push more patchs on the following days | 22:37 |
| EmilienM | clarkb: i'm already in touch with mgagne and other folks | 22:37 |
| *** _TheDodd_ has quit IRC | 22:38 | |
| *** cp16net is now known as cp16net|away | 22:42 | |
| *** vipul is now known as vipul-away | 22:43 | |
| *** vipul-away is now known as vipul | 22:43 | |
| jeblair | clarkb: http://logs.openstack.org/81/39581/1/check/gate-tempest-pep8/65bae6c/ | 22:45 |
| jeblair | clarkb: note the url | 22:46 |
| jog0 | it looks like a bad import or something | 22:46 |
| jog0 | that killed hacking | 22:46 |
| jeblair | clarkb: so i think the LOG_PATH jenkins jobs are looking good (at least the ones that have updated) | 22:47 |
| jeblair | (still running) | 22:47 |
| clarkb | jeblair: yeah, with the change above (39597) I have them going into logstash now | 22:47 |
| *** dina_belova has joined #openstack-infra | 22:47 | |
| *** datsun180b has quit IRC | 22:47 | |
| *** mrodden has quit IRC | 22:48 | |
| jeblair | clarkb: jjb is done, but it encountered some weird errors in the middle, having to do with parsing the cache file... | 22:50 |
| jeblair | clarkb: i'm going to stop puppet on jenkins, rm the jjb cache, and run jjb again | 22:51 |
| clarkb | ok | 22:51 |
| jeblair | clarkb: it should be mostly a noop, but will get any jobs that it may have missed | 22:51 |
| clarkb | yup | 22:51 |
| jeblair | okay that's running now. | 22:52 |
| *** dina_belova has quit IRC | 22:52 | |
| jeblair | i'm going afk for about 30 mins; will check on things when i get back | 22:53 |
| *** jjmb has joined #openstack-infra | 22:53 | |
| *** cp16net|away is now known as cp16net | 22:56 | |
| *** vipul is now known as vipul-away | 22:58 | |
| bodepd | how does hiera know the value of the jenkins_apikey? | 22:58 |
| openstackgerrit | A change was merged to openstack-infra/config: Update elasticsearch custodial crons. https://review.openstack.org/39562 | 22:58 |
| clarkb | bodepd: its in a yaml file that hiera reads | 22:58 |
| bodepd | clarkb: how does it get there? | 22:59 |
| clarkb | we add it by hand | 22:59 |
| bodepd | (seems like a bit of a chicken/egg) | 22:59 |
| bodepd | :( | 22:59 |
| bodepd | can I use password instead? | 22:59 |
| *** vipul-away is now known as vipul | 22:59 | |
| clarkb | bodepd: oh you want to know how to get the jenkins api key? | 22:59 |
| bodepd | I want to know how not to get it :) | 23:00 |
| bodepd | b/c I can spin up a new jenkins instance and setup a user/password | 23:00 |
| clarkb | bodepd: I think the only way to not need an apikey is if you allow unauthenticated users to make changes to your jenkins jobs | 23:00 |
| bodepd | (or at least I know how to do that from puppet) | 23:00 |
| clarkb | as the api uses the apikey iirc | 23:00 |
| * clarkb looks in the python jenkins lib | 23:00 | |
| bodepd | so I always need to install a jenkins, manually get it's key and add it to hiera | 23:01 |
| clarkb | maybe, I am not sure | 23:01 |
| bodepd | k. thanks. | 23:02 |
| clarkb | we are only just now really looking at dynamic jenkins masters again | 23:02 |
| clarkb | managing jenkins programmatically is a giant headache | 23:02 |
| clarkb | bodepd: in the JJB code the value is called password | 23:04 |
| clarkb | bodepd: its possible both will work | 23:04 |
| clarkb | bodepd: I am looking at zuul now | 23:06 |
| *** bnemec has joined #openstack-infra | 23:07 | |
| clarkb | bodepd: fwiw new zuul (tip of master) does not use the jenkins api anymore and instead relies on gearman and the jenkins gearman plugin | 23:07 |
| clarkb | looks like zuul does the same thing. I think it may work if you just put a password into the apikey field | 23:08 |
| *** dhellmann is now known as dhellmann_ | 23:09 | |
| bodepd | clarkb: yeah, it sucks for that | 23:10 |
| *** emagana has quit IRC | 23:10 | |
| bodepd | clarkb: that is what I am hoping to use | 23:10 |
| bodepd | clarkb: is it stable? | 23:10 |
| mordred | clarkb: we might not need that field anymore... | 23:11 |
| *** ladquin has quit IRC | 23:11 | |
| clarkb | mordred: we don't that is a thing that can be cleaned up. I will propose that change | 23:11 |
| *** pcrews has quit IRC | 23:11 | |
| mordred | bodepd: yah - it's been running the openstack gate for a good while now | 23:11 |
| clarkb | bodepd: it is mostly stable. I think there are one or two corner cases that occasionally cause problems but even at ~6k tests a day it isn't bad | 23:11 |
| bodepd | if I use master from config it will install that? | 23:12 |
| bodepd | that is my task for today, is to throw away my old zuul code and use the shiney new zuul | 23:13 |
| bodepd | (well I'm hoping to do that anyways) | 23:13 |
| clarkb | bodepd: if you use master openstack-infra/config and /zuul yes | 23:13 |
| clarkb | bodepd: the apikey in /config is cruft | 23:14 |
| clarkb | so is the user and url I think | 23:14 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Give horizon-ptl ack to tag django_openstack_auth https://review.openstack.org/39605 | 23:15 |
| bodepd | ah, b/c it uses gearman? | 23:16 |
| mordred | clarkb: quick eyes there ^^ gabrielhurley is trying to release and we never gave him rights | 23:16 |
| clarkb | bodepd: yup | 23:18 |
| clarkb | bodepd: gearman does all of the communication between the two | 23:18 |
| clarkb | bodepd: you can setup gearman auth with client ssl certs, we rely on firewall rules only | 23:18 |
| clarkb | not sure if the jenkins gearman plugin does ssl auth | 23:18 |
| mordred | I do not believe it's been added to the java library yet | 23:20 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Cleanup unneeded Jenkins vars. https://review.openstack.org/39607 | 23:20 |
| clarkb | ^ that does the puppet cleanup | 23:20 |
| uvirtbot | clarkb: Error: "that" is not a valid command. | 23:20 |
| clarkb | mordred: that looks correct to me. let me diff against one of the python client libs | 23:21 |
| *** avtar has quit IRC | 23:21 | |
| clarkb | yup +2 | 23:21 |
| * fungi +2'd and aprv'd | 23:24 | |
| bodepd | clarkb: what installs the plugin? | 23:25 |
| bodepd | clarkb: I was reading through the code and didn't find it | 23:25 |
| clarkb | bodepd: now we get into one of two major jenkins pain points | 23:25 |
| openstackgerrit | A change was merged to openstack-infra/config: Give horizon-ptl ack to tag django_openstack_auth https://review.openstack.org/39605 | 23:25 |
| clarkb | bodepd: you do, there is no super automagic way of installing jenkins plugins. You can put the hpi file in /var/lib/jenkins/plugins then on the next jenkins restart it will install the plugin | 23:25 |
| clarkb | bodepd: or you install it through the web UI | 23:25 |
| clarkb | the other major pain point is managing the global config | 23:26 |
| clarkb | bodepd: for your uses copying the hpi file into /var/lib/jenkins/plugins is probably good enough | 23:26 |
| mordred | clarkb, fungi thank you | 23:26 |
| clarkb | bodepd: and since maven repos are hard to deal with you can find our build at http://tarballs.openstack.org/ci/gearman-plugin/ | 23:27 |
| clarkb | you want 0.0.3 right now | 23:27 |
| fungi | bodepd: though we are releasing that jenkins module to jenkins-ci.org so your jenkins should theoretically see it if you resort to beating your head against its terrible, terrible webui | 23:28 |
| fungi | bodepd: https://wiki.jenkins-ci.org/display/JENKINS/Gearman+Plugin | 23:29 |
| clarkb | (zuul comes with its own gearman server so you don't have to worry about that) | 23:30 |
| * fungi likes the word "infrastructor" and will use it if he ever decides to get business cards again | 23:30 | |
| mordred | clarkb: did you see https://review.openstack.org/#/c/39427/ ? afazekas_zz gave us a present | 23:30 |
| fungi | i kept meaning to help fasttrack that. thanks for the reminder | 23:31 |
| clarkb | I did not see it. I am so far behind on review queues right now | 23:31 |
| bodepd | clarkb: this is making me question if I actually even need gearman | 23:31 |
| *** adalbas has joined #openstack-infra | 23:31 | |
| bodepd | clarkb: looks like you guys are uisng it for load balancing | 23:31 |
| bodepd | clarkb: I am no where near needed that | 23:31 |
| jeblair | fungi: i know a really good letterpress printer if you do decide to get cards. :) | 23:32 |
| bodepd | (for my sad little 4 node slave farm :) ) | 23:32 |
| fungi | jeblair: i have one of your samples! | 23:32 |
| clarkb | bodepd: it will be for load balancing eventually but we aren't there yet | 23:32 |
| *** michchap has joined #openstack-infra | 23:32 | |
| bodepd | clarkb: I was going to try to use it b/c that is what you guys do. what if I just have zuul speak directly to jenkins | 23:32 |
| clarkb | bodepd: right now it is one zuul to one jenkins master | 23:32 |
| clarkb | bodepd: you can do that but we stopped supporting that in zuul | 23:33 |
| bodepd | clarkb: then the explanation confused me in terms of why it's even necessary | 23:33 |
| clarkb | bodepd: also, gearman is cleaner for setting up | 23:33 |
| clarkb | its not *necessary* its nicer :) | 23:33 |
| fungi | jeblair: i should actually get the results of gpg2 --fingerprint 48F9961143495829 on a box of cards, per last week's discussion | 23:33 |
| clarkb | bodepd: the jenkins callbacks are clunky and potentially require more complicated firewall rules. gearman gives you a lot of features like percentage of completion, cleaner firewall rules, a bus you can hijack to schedule things outside of zuul and so on | 23:34 |
| fungi | bodepd: it allows us to move the job scheduling further out of jenkins and deeper into zuul, where it can be controlled in more interesting ways | 23:34 |
| bodepd | I will keep going down the gearman path for now | 23:34 |
| bodepd | fungi: clarkb I think the key word in all of that is 'no longer supported' :) | 23:35 |
| jeblair | bodepd: it is (or will be) easier to do an initial setup too; it doesn't require lots of parameter entries in every jenkins job | 23:35 |
| bodepd | jeblair: ah, so you can simply the jjb templates | 23:35 |
| bodepd | I will likely have lots more questions | 23:35 |
| jeblair | fungi, clarkb, zaro: is gearman-plugin available in the jenkins plugin web ui yet? (not sure if the publishing pipeline is all done?) | 23:36 |
| clarkb | jeblair: it is | 23:36 |
| jeblair | bodepd: yes, they are so much simpler now. | 23:36 |
| jeblair | neato! | 23:36 |
| jeblair | we should update the zuul docs to tell people to do that then | 23:36 |
| *** pcrews has joined #openstack-infra | 23:36 | |
| bodepd | http://updates.jenkins-ci.org/download/plugins/gearman-plugin/0.0.3 is solid? | 23:37 |
| *** jrex_laptop has joined #openstack-infra | 23:38 | |
| clarkb | it is the version we are running iirc | 23:38 |
| jeblair | bodepd: yeah, HEAD is 0.0.3 and is what we're running, so that should be the latest and battle tested | 23:38 |
| bodepd | awesome. I have a jenkins::plugin define in my puppet repo | 23:38 |
| bodepd | so, it.might.just.work :) | 23:38 |
| jeblair | clarkb: i don't see any unstable jobs | 23:39 |
| clarkb | jeblair: does that mean jenkins is happy now? | 23:40 |
| jeblair | clarkb: also, the change i was looking at earlier reported: https://review.openstack.org/#/c/39581/ | 23:40 |
| jeblair | clarkb: it seems to have working url to logstash with our hashes at the end | 23:40 |
| jeblair | s/logstash/logs.o.o/ | 23:40 |
| zaro | jeblair: i believe so. last version i saw was 0.0.3 | 23:40 |
| jeblair | clarkb: so yeah, i think it's good | 23:41 |
| jeblair | next up: merge the triggers changes, add a periodic pipeline, disable the jenkins timer on the old jobs, then spin up new jenkinses. | 23:42 |
| jeblair | easy. <sigh> | 23:42 |
| *** adalbas has quit IRC | 23:42 | |
| clarkb | :) its exciting | 23:43 |
| clarkb | jeblair: http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=34 I would like to add another elasticsearch node on Saturday (so that rebalancing isn't too painful) | 23:43 |
| clarkb | jeblair: for disk space | 23:44 |
| clarkb | thoughts, concerns, indifference? | 23:44 |
| jeblair | clarkb: would it make sense to use volumes, or do you want to scale the processing too? | 23:46 |
| jeblair | and ram | 23:46 |
| clarkb | jeblair: volumes would make sense if we had a good story for managing them with config management and if they didn't explode and corrupt filesystems (I may be being over paranoid about them after recent events) | 23:47 |
| clarkb | jeblair: and I figured the extra ram and cpu would be beneficial though not strictly required (the cluster itself seems very stable now) | 23:48 |
| jeblair | clarkb: okiedokie. :) | 23:49 |
| fungi | having a little ram/cpu breathing room seems fine. it doesn't sound like we're terribly overprovisioned there (yet anyway) | 23:49 |
| clarkb | not terribly, but it is a non trivial use of resources too | 23:49 |
| clarkb | and we can think about it for a bit. I don't plan on doing anything until late friday/early saturday | 23:50 |
| * fungi plans to sleep early saturday | 23:50 | |
| clarkb | 10% laptop battery remaining, kernel updated available /me plays with fire | 23:52 |
| mordred | bodepd: if you wanted to share your jenkins::plugin define with us, we would not shoot you in the head | 23:54 |
| mordred | jeblair: wow. I've been distracted enough that I didn't fully realize _why_ you wrote the zuul time code - although now it all makes perfect sense | 23:54 |
| clarkb | woo my isp is dropping packets. I think the fire wins this round | 23:55 |
| jeblair | mordred: :) it feels like everything i do takes at least 6 interlocking steps. | 23:55 |
| mordred | jeblair: maybe you should dance more | 23:55 |
| jeblair | mordred: maybe the bengsons should write me a song | 23:55 |
| mordred | jeblair: you know - if you'd donated more in their last kickstarter campaign, that was one of the prize levels | 23:56 |
| bodepd | I think I snagged it from puppetlabs | 23:56 |
| bodepd | mordred: testing now. I also refactored zuul to work with the latest apache module | 23:57 |
| bodepd | mordred: and I had to add python-pip as a package dep for zuul | 23:57 |
| jeblair | mordred: "masterpieces of immortal art -- created while you wait!" | 23:57 |
| bodepd | mordred: running a final from scratch test now | 23:57 |
| bodepd | mordred: I can submit the pip package and plugins define as a dep, but the apache changes may be more complicated... | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
