| *** SergeyLukjanov has joined #openstack-infra | 00:03 | |
| *** david_h has joined #openstack-infra | 00:05 | |
| *** SergeyLukjanov has quit IRC | 00:09 | |
| *** david_h has quit IRC | 00:13 | |
| *** odyssey4me has joined #openstack-infra | 00:13 | |
| mordred | dstufft: happen to know off the top of your head what the python library entry is that tells me into what path python wants to install things on a given system? | 00:16 |
|---|---|---|
| * mordred is reading pip code because he thinks he's seen it in there before | 00:16 | |
| dstufft | distutils.sysconfig IIRC | 00:16 |
| mordred | dstufft: hrm. that shows me /usr/lib/python2.7/dist-packages on ubuntu - something somewhere tells something to install into /usr/local/ instead doesn't it? | 00:18 |
| mordred | >>> distutils.sysconfig.get_python_lib() | 00:18 |
| mordred | '/usr/lib/python2.7/dist-packages' | 00:18 |
| mordred | (sadly, this is so that I can tell puppet what file to look for to determine whether or not this command has run before) | 00:18 |
| *** cody-somerville has quit IRC | 00:19 | |
| dstufft | mordred: I think you want purelib | 00:20 |
| dstufft | sec looking | 00:20 |
| mordred | dstufft: I'm writing a new pip provider for puppet which ensures os pip and setuptools are not there, and which installs both from ez_setup.py and get-pip.py) | 00:21 |
| mordred | dstufft: while I'm bothering you - do you know if there is a 'good' way I can get get-pip.py to install pip as pip3 when I'm installing with python3? | 00:22 |
| mordred | my thinking was "python3 get-pip.py ; mv /usr/local/bin/pip /usr/local/bin/pip3" | 00:22 |
| mordred | but that's just gross | 00:22 |
| dstufft | pip.locations.distutils_scheme("foobar") | 00:24 |
| dstufft | mordred: it already installs pip as pip3, but also pip and pip3.N | 00:24 |
| mordred | dstufft: hrm. so I guess anyway to get it to not install as pip? | 00:24 |
| dstufft | mordred: not currently | 00:25 |
| mordred | k. fair | 00:25 |
| *** vipul-away is now known as vipul | 00:32 | |
| fungi | mordred: it looks like manage-projects may be broken--any way this could be related to your patch we merged today? http://paste.openstack.org/show/48349/ | 00:35 |
| mordred | fungi: looking | 00:36 |
| mordred | fungi: is that the only ones? | 00:36 |
| fungi | yep | 00:36 |
| fungi | the acl i just merged a change for about an hour ago | 00:36 |
| mordred | fungi: so - new changes | 00:36 |
| mordred | fungi: there shoudl now be a /var/lib/jeepyb with a bunch of repos in it | 00:37 |
| fungi | all of the projects listed are due for an acl to be applied. the /home/gerrit2/acls/openstack/savanna.config is should try to apply is there and correct | 00:37 |
| *** mihgen has quit IRC | 00:37 | |
| fungi | checking the jeepyb cache | 00:37 |
| mordred | which there is and it's got the right things in it... | 00:37 |
| mordred | gerrit2@review:/var/lib/jeepyb/openstack/savanna$ git branch | 00:37 |
| mordred | * config | 00:37 |
| mordred | master | 00:37 |
| mordred | not sure what the problem is - it seems to have worked | 00:37 |
| mordred | fungi: oh - so - fwiw | 00:38 |
| mordred | fungi: run manage-projects as gerrit2 now | 00:38 |
| mordred | oh. or not | 00:38 |
| mordred | hrm | 00:38 |
| fungi | ohhhhh | 00:38 |
| mordred | nevermind | 00:38 |
| mordred | don't listen to me | 00:38 |
| fungi | good policy ;) | 00:38 |
| * mordred running by hand with verbose on | 00:39 | |
| mordred | ok. the repo is on config and the file contents are correct | 00:39 |
| mordred | so, up until this point, it is doing the thing it's supposed to do | 00:40 |
| mordred | fungi: I think there is a bug in the config - | 00:40 |
| mordred | fungi: in that we run manage-projects from puppet as root | 00:40 |
| mordred | but the cron job will expect to be able to run it as gerrit2 | 00:40 |
| mordred | that is not what's at fault here I don't think | 00:40 |
| mordred | DEBUG:paramiko.transport:EOF in transport thread | 00:41 |
| mordred | I'm getting this now when I run it | 00:41 |
| mordred | as gerrit2 | 00:41 |
| mordred | (I chowned the dirs) | 00:41 |
| *** UtahDave has quit IRC | 00:43 | |
| mordred | hrm. ok. no. nevermind. it should run as root and that should always work | 00:44 |
| * mordred shuts up on that subject | 00:44 | |
| * mordred had to trace back to basic principles, sorry | 00:44 | |
| mordred | AAAHHHH | 00:46 |
| mordred | fungi: found it | 00:46 |
| mordred | wow. what a fun sequencing bug | 00:46 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Handle running fetch_config more than once https://review.openstack.org/51422 | 00:48 |
| mordred | fungi: ^^ | 00:48 |
| *** oubiwann_ has joined #openstack-infra | 00:51 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/jeepyb: Put in a reset --hard to clean up https://review.openstack.org/51423 | 00:52 |
| mordred | fungi: it should usually never get into a state where that's necessary, | 00:52 |
| mordred | fungi: in this case, I went in and git a git reset --hard in each repo, then ran manage-projects twice on each projects and all should be well | 00:52 |
| mordred | I've also submitted two patches to fix the issue | 00:52 |
| mordred | I think the original issue was due to the race condition - in that we tried to check out the config refs before they exist - which sadly happens | 00:53 |
| mordred | but because of the new logic, it left us in a state where retrying would also fail | 00:53 |
| mordred | which is what the above patches should fix | 00:53 |
| *** senk has quit IRC | 00:54 | |
| fungi | sorry, was afk bolting things to walls for a few minutes | 00:55 |
| fungi | reading | 00:56 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Handle running fetch_config more than once https://review.openstack.org/51422 | 00:58 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Put in a reset --hard to clean up https://review.openstack.org/51423 | 00:58 |
| fungi | both lgtm | 00:58 |
| fungi | mordred: thanks! | 00:58 |
| dstufft | mordred: the openstack kibana is publically available yea? | 01:02 |
| mordred | yah | 01:03 |
| dstufft | thinking about setting one up to shove all the fastly logs into | 01:03 |
| mordred | dstufft: logstash.openstack.org | 01:03 |
| dstufft | with the ip addresses yanked ofc | 01:03 |
| mordred | dstufft: I can't remember how many ES/LS nodes we're running- clarkb is our master of all of that | 01:04 |
| mordred | it's a non-insignificant collection of machines :) | 01:04 |
| dstufft | heh | 01:04 |
| dstufft | yea we don't have capacity for the ES machines so we'll need to put them on the public cloud (something we generally avoid doing) so I need to go figure out how to handle authentication over the WAN for ES | 01:05 |
| *** wenlock has quit IRC | 01:08 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Make pip install from upstream better https://review.openstack.org/51425 | 01:11 |
| mordred | dstufft: maybe openstack could just spin you up an extra cluster and give you a VPN to it :) | 01:12 |
| mordred | you know - give some back and what not | 01:12 |
| mordred | dstufft: btw - if you're not allergic to ruby, check out the above patch | 01:12 |
| mordred | that's my first stab at replacing all of puppet's thinking on pip installation :) | 01:13 |
| dstufft | gerrit needs a view that shows the entire diff on one page like github does | 01:13 |
| *** odyssey4me has quit IRC | 01:13 | |
| mordred | yes. yes it does | 01:15 |
| mordred | it's HIGH on our wishlist | 01:15 |
| lifeless | right after using current gerrit ;_ | 01:16 |
| mordred | dstufft: https://review.openstack.org/gitweb?p=openstack-infra%2Fconfig.git;a=commitdiff;h=1c7a4c0a5c27ca16d198036382af75d9177cd09b | 01:16 |
| mordred | lifeless: ^^ | 01:16 |
| mordred | that's the page you get to from the "gitweb" link on the gerrit page | 01:16 |
| mordred | you can't COMMENT on it or review from that page | 01:16 |
| mordred | but you can look | 01:16 |
| mordred | lifeless: once we're on current gerrit, _david_ and zaro can make us a one-page-diff page :) | 01:17 |
| lifeless | david | 01:19 |
| mordred | yah | 01:19 |
| mordred | ostrovsky | 01:19 |
| mordred | he's with libreoffice - set up an infra like system for them, and does hacking on gerrit | 01:19 |
| mordred | he and zaro have both been working on elements of the WIP thing upstream | 01:19 |
| *** oubiwann_ has quit IRC | 01:41 | |
| *** oubiwann_ has joined #openstack-infra | 01:50 | |
| *** wenlock has joined #openstack-infra | 02:01 | |
| *** oubiwann_ has quit IRC | 02:19 | |
| *** esker has joined #openstack-infra | 03:25 | |
| *** esker has quit IRC | 03:46 | |
| *** wenlock has quit IRC | 04:04 | |
| *** esker has joined #openstack-infra | 04:19 | |
| *** esker has quit IRC | 04:21 | |
| *** esker has joined #openstack-infra | 04:47 | |
| *** esker has quit IRC | 05:09 | |
| *** zehicle has quit IRC | 05:18 | |
| *** Ryan_Lane has quit IRC | 06:11 | |
| openstackgerrit | Alexander Braverman proposed a change to openstack-infra/jenkins-job-builder: Jenkins Ownership Plugin https://review.openstack.org/51441 | 06:22 |
| *** nicedice has quit IRC | 06:33 | |
| *** nicedice has joined #openstack-infra | 06:34 | |
| *** mihgen_ has joined #openstack-infra | 06:45 | |
| *** dafter has joined #openstack-infra | 06:59 | |
| *** dafter has quit IRC | 06:59 | |
| *** dafter has joined #openstack-infra | 06:59 | |
| *** garyk has quit IRC | 07:02 | |
| *** mihgen_ has quit IRC | 07:06 | |
| *** yolanda has joined #openstack-infra | 07:07 | |
| *** rwsu has quit IRC | 07:22 | |
| *** mihgen has joined #openstack-infra | 07:40 | |
| *** dmakogon_ has joined #openstack-infra | 08:31 | |
| mihgen | hi folks | 08:33 |
| mihgen | could anyone guide me where to look why approved patchsets don't go to master? | 08:33 |
| mihgen | this for example: https://review.openstack.org/#/c/51198/ | 08:33 |
| *** kong has joined #openstack-infra | 08:50 | |
| *** Lingxian has quit IRC | 08:50 | |
| *** thomasbiege has joined #openstack-infra | 08:51 | |
| *** thomasbiege1 has joined #openstack-infra | 09:29 | |
| *** thomasbiege has quit IRC | 09:32 | |
| *** cody-somerville has joined #openstack-infra | 09:39 | |
| *** cody-somerville has quit IRC | 09:44 | |
| *** Lingxian has joined #openstack-infra | 09:55 | |
| *** kong has quit IRC | 09:55 | |
| *** kong has joined #openstack-infra | 10:01 | |
| *** Lingxian has quit IRC | 10:01 | |
| *** boris-42 has joined #openstack-infra | 10:15 | |
| *** salv-orlando has quit IRC | 10:19 | |
| *** fifieldt has joined #openstack-infra | 10:36 | |
| *** beagles has joined #openstack-infra | 10:41 | |
| *** beagles has quit IRC | 10:42 | |
| *** beagles has joined #openstack-infra | 10:42 | |
| *** dmakogon_ has quit IRC | 10:58 | |
| *** Bada has joined #openstack-infra | 11:08 | |
| *** dafter has quit IRC | 11:13 | |
| *** dafter has joined #openstack-infra | 11:13 | |
| *** michchap has joined #openstack-infra | 11:18 | |
| *** dafter has quit IRC | 11:18 | |
| *** Bada has quit IRC | 11:26 | |
| *** cody-somerville has joined #openstack-infra | 11:29 | |
| *** beagles has quit IRC | 11:31 | |
| *** Bada has joined #openstack-infra | 11:48 | |
| *** CaptTofu has joined #openstack-infra | 11:53 | |
| *** salv-orlando has joined #openstack-infra | 11:53 | |
| *** che-arne has joined #openstack-infra | 12:00 | |
| *** jaypipes has quit IRC | 12:03 | |
| *** CaptTofu has quit IRC | 12:05 | |
| *** CaptTofu has joined #openstack-infra | 12:06 | |
| *** thomasbiege1 has quit IRC | 12:07 | |
| *** CaptTofu has quit IRC | 12:10 | |
| *** bodepd_ has joined #openstack-infra | 12:17 | |
| *** bodepd has quit IRC | 12:18 | |
| mordred | mihgen: it's missing +2 votes in the code-review category | 12:23 |
| *** fifieldt has quit IRC | 12:23 | |
| mordred | mihgen: it is required that every patch have at least one +2 in Code Review, although by policy most openstack projects require 2 +2 votes | 12:24 |
| *** dafter has joined #openstack-infra | 12:24 | |
| *** dafter has quit IRC | 12:24 | |
| *** dafter has joined #openstack-infra | 12:24 | |
| mordred | since it's a stackforge project, you're free to make whatever policy on number of +2s you'd like | 12:24 |
| sdague | mordred: so I was thinking about the redo on the log filter that htmlifies our logs.o.o. What would be really cool is if we could CD master to logs-dev.o.o, and CD a stable branch to logs.o.o. However, given python's tendancy to only like one version of things installed, I'm not sure how to accomplish that | 12:31 |
| *** dafter has quit IRC | 12:36 | |
| *** dafter has joined #openstack-infra | 12:37 | |
| *** dafter has quit IRC | 12:37 | |
| *** dafter has joined #openstack-infra | 12:38 | |
| *** dafter has joined #openstack-infra | 12:38 | |
| mordred | sdague: two servers? | 12:40 |
| sdague | right now we're solving it with vhosts | 12:41 |
| mordred | hrm | 12:41 |
| mordred | well... | 12:41 |
| sdague | I guess we could solve it with two servers, but that seems heavy | 12:41 |
| mordred | virtualenvs | 12:41 |
| mordred | we were talking yesterday about our python stuff and turning it into debs | 12:42 |
| mordred | one of the things that came out of that was using dh-virtualenv | 12:42 |
| mordred | to make debs that contain a virtualenv in them | 12:42 |
| mordred | so, if we did that, we could make a deb called logfilter-master and a deb called logfilter | 12:43 |
| sdague | I suppose, there is a reason you want to do that instead of CD for these tools (or that would be triggered at the post of every merge?) | 12:43 |
| sdague | http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/ - so looks like it would be doable with a wrapper | 12:44 |
| mordred | that would be triggered at the post of every merge | 12:46 |
| mordred | and I mention that as an option purely because the alternative, teaching puppet about virtualenvs, seems like more work to me :) | 12:47 |
| sdague | :) | 12:50 |
| *** Bada has quit IRC | 12:54 | |
| *** rickerc_ has joined #openstack-infra | 12:57 | |
| *** rickerc has quit IRC | 12:59 | |
| *** rickerc_ is now known as rickerc | 12:59 | |
| *** CaptTofu has joined #openstack-infra | 13:01 | |
| *** esker has joined #openstack-infra | 13:02 | |
| *** esker has quit IRC | 13:07 | |
| boris-42 | mordred ping | 13:16 |
| boris-42 | mordred is it complex to add numphy to openstack dependencies ?) http://logs.openstack.org/50/51450/2/check/gate-rally-pep8/ff7f563/console.html | 13:16 |
| boris-42 | requirements* | 13:17 |
| boris-42 | sdague ^ | 13:17 |
| sdague | boris-42: if you are building on a devstack node, it will be installed | 13:19 |
| boris-42 | sdague actually as you see it failed.. | 13:20 |
| sdague | yes, that's not a devstack node, that's a unit test one | 13:20 |
| boris-42 | sdague I mean I add requirements to Rally and put on review.. | 13:20 |
| boris-42 | sdague so how can I fix it ?) | 13:20 |
| sdague | I thought the rally testing was going to be in devstack env? | 13:20 |
| boris-42 | sdague not sure that I understand=). There are 2 different tasks | 13:21 |
| boris-42 | sdague Rally project it self | 13:21 |
| boris-42 | sdague and testing existing deployments | 13:22 |
| *** CaptTofu has quit IRC | 13:24 | |
| boris-42 | sdague so as Rally is separated project. I would like to add all dependencies in requirements.txt | 13:24 |
| sdague | ok, this is triggered by trying to add matplotib. Do you have code using that yet? | 13:24 |
| *** CaptTofu has joined #openstack-infra | 13:24 | |
| boris-42 | sdague I will have soon.. | 13:25 |
| boris-42 | sdague I am just interesting before starting writing code, is it possible to add this dependency.. | 13:26 |
| *** DennyZhang has joined #openstack-infra | 13:26 | |
| sdague | boris-42: the biggest issue is you need a lot of headers to compile it | 13:27 |
| sdague | I don't know if the unit test nodes have those | 13:27 |
| sdague | you could always try just adding it to your requirements file to find out | 13:27 |
| sdague | compiling numpy is about a 3 minute process IIRC, that's what we saw when we were mistakenly building it in devstack (vs. package install) | 13:28 |
| *** CaptTofu has quit IRC | 13:29 | |
| *** CaptTofu has joined #openstack-infra | 13:29 | |
| mordred | once we are building wheels in our mirror - this will be easier | 13:30 |
| boris-42 | sdague mordred so what should I do know?) try to find another lib?) | 13:32 |
| sdague | boris-42: what are you trying to accomplish? | 13:33 |
| sdague | in general I'm a very big fan of matplotlib | 13:34 |
| sdague | so I'd say if you need graphing libraries, that's the right direction | 13:34 |
| sdague | but I don't have much context :) | 13:35 |
| boris-42 | sdague yes I need to process a lot of statistics data into graphics=) | 13:36 |
| boris-42 | sdague ok I think that i will make some check that matplotlib is installed | 13:36 |
| boris-42 | sdague and will use try_import_module and for test mocks=) | 13:37 |
| sdague | boris-42: what are you going to test on the other side of the matplotlib import? | 13:37 |
| *** oubiwann_ has joined #openstack-infra | 13:37 | |
| *** yaguang has joined #openstack-infra | 13:37 | |
| sdague | honestly, it seems like the toolchain that actually does the graphs should be seperate enough from the tools that generate the data, that you wouldn't have to mix all their tests oddly | 13:38 |
| *** CaptTofu has quit IRC | 13:38 | |
| boris-42 | sdague actually I would like to make all-in-one | 13:39 |
| *** CaptTofu has joined #openstack-infra | 13:39 | |
| sdague | that seems less useful | 13:39 |
| boris-42 | sdague no it seems like easy to use=) | 13:39 |
| sdague | the visualization of the data should take place after it's collected, so you can change the visualization on existing data | 13:39 |
| boris-42 | sdague the raw data is saved into DB | 13:40 |
| boris-42 | sdage ahh | 13:40 |
| boris-42 | sdague I thought another =) | 13:40 |
| boris-42 | sdague the deploying, benchmarking, result collection & result processing are separated things | 13:41 |
| *** DennyZhang has quit IRC | 13:41 | |
| boris-42 | sdague but they are parts of one application, called Rally | 13:42 |
| boris-42 | sdague so I was speaking about testing of drawing graphs | 13:43 |
| boris-42 | sdague that I will use mocks to mock matplotlib module | 13:43 |
| *** CaptTofu has quit IRC | 13:43 | |
| sdague | yeh, I was just trying to understand what testing of matplotlib calls would be like. It's sort of a complex thing to test the output off, as it outputs graphics. I guess you could test the datastructure before the plot call, but I'm not sure how stable matplotlib internals are | 13:46 |
| *** beagles has joined #openstack-infra | 13:50 | |
| boris-42 | sdague at lest to test that there is no typos in my code=) | 13:54 |
| boris-42 | least* | 13:54 |
| sdague | ok, fair enough | 13:54 |
| boris-42 | sdague so in feature it will be possible to add matplotlib in requirements?) | 13:55 |
| sdague | did you try putting numpy in your requirements file? | 13:57 |
| boris-42 | sdague no but I will try now | 13:57 |
| sdague | try that first | 13:57 |
| sdague | if it doesn't help, can try to figure out something next week | 13:57 |
| *** esker has joined #openstack-infra | 14:02 | |
| *** dizquierdo has joined #openstack-infra | 14:03 | |
| boris-42 | sdague doesn't help http://logs.openstack.org/50/51450/4/check/gate-rally-pep8/0c6a6c9/console.html | 14:06 |
| boris-42 | sdague here is the patch set https://review.openstack.org/#/c/51450/ | 14:06 |
| *** esker has quit IRC | 14:06 | |
| sdague | yeh, you are definitely requiring more out of the unit test nodes than they are designed to support | 14:08 |
| sdague | as a stackforge project, that's sort of what you get. I'd just say seperate the tests out so it is easy to exclude those in CI | 14:09 |
| boris-42 | sdague one thing that I would like is to install these dependency automatically when users will run puthon setup.py install | 14:11 |
| boris-42 | sdague I think that I will found some hack to make it works. | 14:11 |
| sdague | boris-42: honestly, they should install numpy and matplotlib from packages | 14:11 |
| sdague | not from pip | 14:11 |
| sdague | numpy is basically written in C, and really needs other math libs to be useful | 14:12 |
| sdague | so that really should come from the base OS, not from pip | 14:12 |
| boris-42 | sdague okay then wiki=) | 14:13 |
| boris-42 | sdague thank you for help | 14:13 |
| sdague | no prob | 14:13 |
| *** thomasbiege has joined #openstack-infra | 14:45 | |
| *** sileht has quit IRC | 14:46 | |
| clarkb | fungi I will put a gameplan on etherpad-dev in a few minutes | 15:07 |
| *** beagles has quit IRC | 15:09 | |
| *** zul has quit IRC | 15:14 | |
| *** yaguang has quit IRC | 15:17 | |
| *** zul has joined #openstack-infra | 15:17 | |
| *** dizquierdo has quit IRC | 15:20 | |
| *** dkranz has joined #openstack-infra | 15:22 | |
| *** beagles has joined #openstack-infra | 15:23 | |
| *** esker has joined #openstack-infra | 15:25 | |
| *** dmakogon_ has joined #openstack-infra | 15:27 | |
| *** resker has joined #openstack-infra | 15:27 | |
| *** guohliu has quit IRC | 15:27 | |
| *** esker has quit IRC | 15:29 | |
| clarkb | fungi: https://etherpad-dev.openstack.org/p/etherpad-upgrade | 15:30 |
| *** resker has quit IRC | 15:32 | |
| *** dkranz has quit IRC | 15:34 | |
| jeblair | clarkb: good morning | 15:35 |
| *** jeblair is now known as ncsamosaic | 15:35 | |
| ncsamosaic | i have joined the etherpad | 15:35 |
| ncsamosaic | clarkb: wow today's date is 2013 1013. that's neat. | 15:37 |
| clarkb | ncsamosaic: good morning | 15:37 |
| ncsamosaic | clarkb: aww i thought we were doing a new thing with nicks. i was hoping fungi would use pick gopher. | 15:38 |
| *** cody-somerville has quit IRC | 15:38 | |
| clarkb | ncsamosaic: sorry that was a hold over from when I was testing etherpad-dev. It made keeping track of which browser did what easier :) | 15:39 |
| *** ncsamosaic is now known as jeblair | 15:39 | |
| jeblair | oh well | 15:40 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Revert "Remove tripleo nodepool config" https://review.openstack.org/51464 | 15:42 |
| mordred | oh hi! | 15:42 |
| openstackgerrit | A change was merged to openstack-infra/nodepool: Rename ASRT -> AGT https://review.openstack.org/51267 | 15:42 |
| jeblair | clarkb: i'm going to work on nodepool unless you assign me a task; sound good? | 15:43 |
| clarkb | mordred: https://etherpad-dev.openstack.org/p/etherpad-upgrade and join the party | 15:43 |
| clarkb | jeblair: sure, I have listed fungi on things because he said he would be around | 15:43 |
| jeblair | clarkb: yep. i was unsure at the time, but i'm here for a bit. :) | 15:44 |
| clarkb | great will keep that in mind (perhaps for testing things) | 15:44 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Remove tripleo nodepool config" https://review.openstack.org/51464 | 15:44 |
| jeblair | I'm going to put jenkins in shutdown for a few mins | 15:45 |
| mordred | ooh! I lke the ordered lists | 15:45 |
| mordred | jeblair: cool | 15:45 |
| mordred | btw- I landed the outstanding gerrit changes after checking that they continued to operate | 15:46 |
| jeblair | (i realize the etherpad plan involves merging changes -- this won't be for long) | 15:46 |
| clarkb | jeblair: ok, and merging those etherpad changes isn't required to effect the upgrade, its there to make sure that going forward puppet doesn't do something unexpected | 15:46 |
| *** thomasbiege has quit IRC | 15:47 | |
| *** beagles has quit IRC | 15:48 | |
| jeblair | jenkins is out of shutdown mode and nodepool is running git master | 15:48 |
| mordred | woot | 15:50 |
| clarkb | \o/ | 15:50 |
| fungi | clarkb: i am | 15:50 |
| mordred | now if the goverment could be as good as jenkins | 15:50 |
| fungi | just got back from saturday workout with 10 minutes to spare ;) | 15:51 |
| jeblair | fungi: that was a LONG workout! | 15:51 |
| fungi | eh, 10 miles | 15:51 |
| jeblair | fungi: it went into sunday! | 15:51 |
| fungi | oh, right. it IS sunday :/ | 15:51 |
| fungi | ha | 15:51 |
| clarkb | fungi: https://etherpad-dev.openstack.org/p/etherpad-upgrade if you haven't seen it yet. Let me know if that looks sane | 15:51 |
| jeblair | fungi's daily workouts take 2 days! | 15:52 |
| fungi | hee | 15:52 |
| mordred | btw: https://review.openstack.org/#/c/46015/1 has 3 core +2's and 3 +1s. I think it's one of the most positively reviewed patches every | 15:52 |
| fungi | clarkb: yup, saw it in scrollback and was just pulling it up | 15:52 |
| fungi | will pull up rackspace and update dns momentarily when you're ready | 15:52 |
| fungi | clarkb: plan looks sane to me | 15:54 |
| clarkb | great | 15:54 |
| fungi | btw what are ozone.etherpad.openstack.org and openstack.etherpad.openstack.org? old cruft? | 15:55 |
| mordred | fungi: ozone is that rackspace shared our etherpad server with us back in the day | 15:56 |
| mordred | for one of their internal projects | 15:56 |
| mordred | I'm not sure if they ever migrated that to a server not in the openstack account- but I believe that entry was added to help with the split | 15:56 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Fix another wrong use of server_id https://review.openstack.org/51465 | 15:57 |
| fungi | maybe those two address records can be deleted now? they just redirect to etherpad.o.o anyway | 15:57 |
| jeblair | we should probably check with them | 15:58 |
| jeblair | and probably not delete the server | 15:58 |
| fungi | so leave them pointed to the old etherpad server for now or point them to the new one? | 15:58 |
| fungi | okay the former. got it | 15:58 |
| fungi | i'll also add an aaaa for the etherpad.o.o since we don't have one currently | 15:59 |
| jeblair | i vote we leave them pointed at the old server and leaving the old server shut down; that will flush out any problems. | 15:59 |
| clarkb | ++ | 15:59 |
| mordred | yup | 15:59 |
| clarkb | I don't intend on deleting the server immediately if for no other reason than it contains database dumps | 16:00 |
| fungi | oh, actually, i think those other names will likely continue to work even if we delete since they're pointed to a different ip address which is hosting the redirect by name anyway? | 16:00 |
| fungi | but we can leave it up regardless | 16:00 |
| jeblair | i just verified that the SIGUSR2 thread dump feature in nodepool works and dumps thread stacktraces to the debug log | 16:02 |
| mordred | jeblair: neat! | 16:02 |
| *** esker has joined #openstack-infra | 16:02 | |
| fungi | that's excellent | 16:02 |
| mordred | jeblair: it's almost like you've used programming | 16:02 |
| clarkb | ok enough talking its past time to start :) | 16:03 |
| * mordred stops talking | 16:03 | |
| clarkb | I am shutting down etherpad services on both servers | 16:03 |
| jeblair | #status alert etherpad is down for software upgrade and migration | 16:03 |
| fungi | shall i #status alert? | 16:03 |
| * jeblair fixes statusbot | 16:03 | |
| fungi | heh. simultaneous thought | 16:04 |
| clarkb | fungi: that is done, you should be good to start DNS changes (I am using IP addresses on my end so that work won't interrupt me) | 16:04 |
| mordred | maybe we should just restart statusbot on a cron | 16:04 |
| fungi | working on it now | 16:04 |
| *** openstackstatus has joined #openstack-infra | 16:04 | |
| openstackgerrit | A change was merged to openstack-dev/hacking: Allow APL text to start on the first text column https://review.openstack.org/49387 | 16:04 |
| jeblair | mordred: or we could fix it | 16:04 |
| jeblair | #status alert etherpad is down for software upgrade and migration | 16:05 |
| openstackstatus | NOTICE: etherpad is down for software upgrade and migration | 16:05 |
| mordred | jeblair: oh. that would be using programming again. you and your science | 16:05 |
| *** ChanServ changes topic to "etherpad is down for software upgrade and migration" | 16:05 | |
| fungi | dns is in place | 16:05 |
| jeblair | clarkb: did you see mordred's update of your mysqldump command? | 16:05 |
| fungi | forward anyway. adding reverse (though that could have been safely done in advance) | 16:05 |
| mordred | dns has already migrated for me | 16:06 |
| clarkb | jeblair: I did | 16:06 |
| clarkb | jeblair: and I used the --opt flag | 16:07 |
| clarkb | fungi: yeah I should've done reverse DNS earlier sorry | 16:07 |
| fungi | no worries. easy enough to do now while i'm at it, just needs a separate account | 16:07 |
| *** esker has quit IRC | 16:07 | |
| mordred | we should make better 503 pages for our apaches | 16:07 |
| mordred | with like a logo and a fail unicorn or something | 16:08 |
| fungi | failicorn | 16:08 |
| fungi | i find it amusing that rackspace's dns management webui assumes you can have one and only one ptr per ip address | 16:10 |
| fungi | the list of rackspace dns management assumptions continues to grow | 16:10 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/nodepool: Fix another wrong use of server_id https://review.openstack.org/51465 | 16:11 |
| clarkb | I am applying the DB dump to the trove server now | 16:12 |
| clarkb | its half a gig so it takes a little bit of time | 16:12 |
| fungi | sure. i'm going to go back over those two changes while i have a moment | 16:12 |
| jeblair | mordred: http://paste.openstack.org/show/48379/ | 16:12 |
| mordred | jeblair: fascintating | 16:13 |
| mordred | SpamapS: ^^ | 16:13 |
| mordred | SpamapS: we seem to have floating ip problems in the grizzly cloud | 16:13 |
| jeblair | mordred: i think someone needs to sit down with nodepool, novaclient, and the tripleo poc; i think it behaves in a new and exciting way that hpcloud did not when it used floating ips. | 16:13 |
| mordred | jeblair: I agree | 16:13 |
| jeblair | mordred: i would _guess_ that it's not so much a problem with the cloud, but rather nodepool makes assumptions that are inconsistent with it | 16:14 |
| mordred | jeblair: ++ | 16:14 |
| mordred | SpamapS, lifeless: ^^ we've hit enough issues with this that I think someone needs to do what jeblair is saying above and iterate with a nodepool until it can actually spin up nodes on our grizzly cloud | 16:14 |
| fungi | jeblair: also, i have the new tripleo poc cloud accounts for nodepool/infra in the usual place, but didn't add/update anything in hiera yet since i wasn't sure what the state of that is | 16:14 |
| clarkb | fungi: oh just occurred to me, if someone has the old etherpad-dev host IP handy (nova list should show it) can you make sure that puppet is disabled there (though I intend on deleting the host tomorrow) | 16:15 |
| jeblair | fungi: is that for the cd-master cloud? | 16:15 |
| fungi | jeblair: yes | 16:15 |
| clarkb | I updated hiera | 16:16 |
| fungi | clarkb: i can kill puppet on the old etherpad-dev, sure | 16:16 |
| jeblair | fungi: cool; yeah, i don't think we want to do anything much with it at the moment, at least until it stops deleting servers every 40 minutes | 16:16 |
| fungi | jeblair: makes sense ;) | 16:16 |
| mordred | jeblair: agreed. lifeless mainly wanted to get our account(s) in place | 16:16 |
| mordred | and get us the info for them | 16:16 |
| clarkb | ok mysql is done doing its thing I am going to start etherpad-lite on the new server now | 16:16 |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Revert "Revert "Remove tripleo nodepool config"" https://review.openstack.org/51469 | 16:16 |
| mordred | once it's actually, you know upgrading rather than replacing, it will be a much more interesting cloud | 16:17 |
| jeblair | mordred: ++ | 16:17 |
| mordred | (to be fair, it's an interesting cloud, as it is an auto-deployed openstack master cloud - but not interesting as a nodepool source) | 16:17 |
| clarkb | https://etherpad.openstack.org/p/etherpad-upgrade care to join that pad and we can do some testing? | 16:18 |
| clarkb | everyone should see the "Style" dropdown which is only on the new server and not the old | 16:18 |
| fungi | looks pretty broken at the moment | 16:18 |
| fungi | possible i'm on the old ip address still | 16:19 |
| jeblair | fungi: really? we're all talking about you | 16:19 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Revert "Remove tripleo nodepool config"" https://review.openstack.org/51469 | 16:19 |
| * fungi clears caches | 16:19 | |
| mordred | ODF export works | 16:20 |
| *** dizquierdo has joined #openstack-infra | 16:21 | |
| clarkb | export then import won't result in 1:1 data, but I wasn't too worried about that | 16:21 |
| clarkb | I think most folks are interested in exports | 16:21 |
| mordred | I just tested that | 16:21 |
| mordred | it sucks | 16:21 |
| mordred | https://etherpad.openstack.org/p/etherpad-upgrade-import | 16:21 |
| clarkb | and the new redirects I put in place are working for folks? | 16:22 |
| jeblair | https://wiki.openstack.org/wiki/Summit/Havana/Etherpads | 16:22 |
| jeblair | is a good way to test ^ | 16:22 |
| jeblair | clarkb: so far looks good | 16:22 |
| fungi | seems working | 16:23 |
| clarkb | jeblair: yup I have opened 3 semi randomly and they look good | 16:23 |
| mordred | yup | 16:24 |
| clarkb | I would like to keep old etherpad.o.o around until I sort out what to do with the DB dumps on that host (I don't want to lose 30 days of dumps be deleting the host), but the new host looks good from what I can see | 16:24 |
| mordred | ++ | 16:25 |
| jeblair | clarkb: status ok? | 16:25 |
| clarkb | jeblair: yup | 16:25 |
| jeblair | #status ok etherpad migration complete | 16:25 |
| openstackstatus | NOTICE: etherpad migration complete | 16:25 |
| *** ChanServ changes topic to "Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack.org/ | Bugs https://launchpad.net/openstack-ci | Code https://git.openstack.org/cgit/openstack-infra/" | 16:25 | |
| fungi | puppet is now stopped on the old etherpad-dev btw | 16:26 |
| clarkb | fungi: thanks | 16:26 |
| mordred | jeblair: we should make a job that tries to run nodepool against a devstack install | 16:26 |
| fungi | are we ready to merge those config changes now? | 16:26 |
| clarkb | fungi: yes | 16:26 |
| jeblair | mordred: !! | 16:26 |
| *** aspiers has quit IRC | 16:28 | |
| openstackgerrit | A change was merged to openstack-infra/config: Update etherpad and etherpad puppet manifests. https://review.openstack.org/45360 | 16:29 |
| openstackgerrit | A change was merged to openstack-infra/config: Tune apache for etherpad lite. https://review.openstack.org/50700 | 16:29 |
| fungi | those should be on the puppet master now | 16:30 |
| fungi | puppeting time, clarkb? | 16:31 |
| fungi | similar to pudding time i think | 16:31 |
| * fungi suddenly gets a primus song stuck in his head | 16:31 | |
| clarkb | fungi: yup, I will run puppet by hand to make sure it still applies cleanly then start the daemons | 16:31 |
| clarkb | fungi: done | 16:34 |
| clarkb | I think that is it | 16:34 |
| *** dizquierdo has quit IRC | 16:35 | |
| clarkb | thanks everyone | 16:35 |
| jeblair | clarkb: thank you! | 16:35 |
| jeblair | i started puppet on nodepool | 16:35 |
| *** oubiwann_ has quit IRC | 16:37 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-dev/pbr: Use wheels for installation https://review.openstack.org/48803 | 16:37 |
| clarkb | fungi: do you have old etherpad-dev's ipv4 address handy? | 16:37 |
| clarkb | fungi: I want to note that so that I don't have to look it up each time | 16:37 |
| fungi | clarkb: sure | 16:39 |
| fungi | just a sec | 16:39 |
| fungi | clarkb: 166.78.1.246 | 16:39 |
| clarkb | fungi: ty | 16:40 |
| *** sileht has joined #openstack-infra | 16:44 | |
| clarkb | I am going to resume my regularly scheduled weekend now if there isn't anything else | 16:45 |
| * fungi is already busy doing the same | 16:47 | |
| fungi | enjoy! | 16:47 |
| *** thomasbiege has joined #openstack-infra | 16:59 | |
| *** thomasbiege has quit IRC | 17:03 | |
| *** senk has joined #openstack-infra | 17:16 | |
| *** dcramer_ has quit IRC | 17:17 | |
| *** wenlock has joined #openstack-infra | 17:18 | |
| *** Bada has joined #openstack-infra | 17:20 | |
| *** Bada has quit IRC | 17:20 | |
| *** wenlock has quit IRC | 17:39 | |
| *** salv-orlando has quit IRC | 17:40 | |
| *** salv-orlando has joined #openstack-infra | 17:41 | |
| *** thomasbiege has joined #openstack-infra | 17:46 | |
| *** salv-orlando has quit IRC | 17:51 | |
| *** thomasbiege has quit IRC | 17:57 | |
| *** esker has joined #openstack-infra | 18:02 | |
| *** thomasbiege has joined #openstack-infra | 18:02 | |
| *** sarob has joined #openstack-infra | 18:05 | |
| *** nicedice has quit IRC | 18:05 | |
| *** nicedice has joined #openstack-infra | 18:05 | |
| *** esker has quit IRC | 18:07 | |
| *** dcramer_ has joined #openstack-infra | 18:30 | |
| SpamapS | re floating ips on the grizzly and trunk clouds.. I think we just have to create them. | 18:33 |
| SpamapS | As in, we're just not doing it right now | 18:33 |
| *** yamahata has joined #openstack-infra | 18:37 | |
| *** senk has quit IRC | 18:43 | |
| mihgen | mordred: thanks! enjoing how it is landing to master :) | 18:43 |
| *** senk has joined #openstack-infra | 18:44 | |
| *** david_h has joined #openstack-infra | 18:48 | |
| *** wenlock has joined #openstack-infra | 18:53 | |
| *** sarob has quit IRC | 18:55 | |
| *** sarob has joined #openstack-infra | 18:56 | |
| *** beagles has joined #openstack-infra | 18:57 | |
| *** sarob has quit IRC | 19:00 | |
| *** david_h has quit IRC | 19:02 | |
| *** esker has joined #openstack-infra | 19:02 | |
| *** thomasbiege has quit IRC | 19:03 | |
| mordred | SpamapS: I'm sure you're right - but I think we may want to actually ensure that the grizzly cloud has the things that nodepool is expecting before we try enabling it in the infra config next | 19:03 |
| *** esker has quit IRC | 19:07 | |
| *** cody-somerville has joined #openstack-infra | 19:27 | |
| SpamapS | mordred: I didn't know we were trying to do that actually. | 19:37 |
| SpamapS | mordred: I thought we would just try and make the cd cloud stable and use that. | 19:37 |
| *** krtaylor has quit IRC | 19:38 | |
| *** wenlock has quit IRC | 19:44 | |
| *** dmakogon_ has quit IRC | 19:48 | |
| lifeless | SpamapS: mordred: we're not going to be changing the gtrizzly cloud | 19:48 |
| lifeless | SpamapS: mordred: but if there are things nodepool needs in it that are missing, we'll make sure we put them in the cd-overcloud asap | 19:49 |
| SpamapS | lifeless: I believe you were working on what is missing from the grizzly cloud, which is a floating ip pool for tenants. | 19:52 |
| *** salv-orlando has joined #openstack-infra | 19:59 | |
| *** beagles has quit IRC | 20:03 | |
| *** sarob has joined #openstack-infra | 20:06 | |
| *** sarob has quit IRC | 20:11 | |
| *** mriedem has joined #openstack-infra | 20:18 | |
| lifeless | SpamapS: huh, grizzly has a floating ip pool | 20:22 |
| lifeless | SpamapS: totally usable, ditto the cd-overcloud | 20:22 |
| lifeless | jeblair: ^ | 20:22 |
| lifeless | you need to ask for it via neutron | 20:23 |
| *** esker has joined #openstack-infra | 20:30 | |
| *** dcramer_ has quit IRC | 20:30 | |
| lifeless | mordred: it would be cool if run-mirror could post-process the mirror to have wheels | 20:34 |
| mordred | lifeless: wheels are coming | 20:40 |
| mordred | lifeless: ah. neutron. well, as you may imagine, nodepool does not currently have any neutron support | 20:41 |
| clarkb | I thought nova is supposed to proxy those things to neutron | 20:42 |
| mordred | clarkb: maybe that has changed in grizzly/havanan? | 20:42 |
| mordred | lifeless: do you have any examples of doing it with neutron instead of nova? | 20:43 |
| lifeless | ok | 20:43 |
| clarkb | I don't think so. I thought it was a partial attempt at a migration path | 20:43 |
| lifeless | clarkb: mordred: http://git.openstack.org/cgit/openstack/tripleo-incubator/tree/scripts/devtest_overcloud.sh#n154 | 20:44 |
| mordred | lifeless: is it possile that you are not running the nova floating-ip extension? | 20:44 |
| lifeless | mordred: we're running defaults | 20:44 |
| mordred | ah | 20:44 |
| mordred | ok | 20:44 |
| lifeless | mordred: which is by design | 20:44 |
| mordred | fascinating | 20:44 |
| lifeless | if this is something that should be on by default, our preference is to fix nova | 20:44 |
| mordred | so, floating-ip extension for nova is one of the "default" extensions :) | 20:44 |
| lifeless | how does one query what extensions are enabled ? | 20:45 |
| lifeless | (I mean, all our config glue etc is open, so you can just check there, but perhaps there is an API to check?) | 20:45 |
| mordred | client.get('/extensions') | 20:45 |
| * mordred looking through code right now | 20:46 | |
| lifeless | nova list-extensions | 20:46 |
| mordred | btw - I agree with running with defaults. just saying - I think the default for nova ihere is broken | 20:47 |
| lifeless | so if it's something we just turn on - cool | 20:48 |
| mordred | lifeless: currently, the only client lib that nodepool is using is novaclient | 20:49 |
| *** senk has quit IRC | 20:51 | |
| mordred | if not ip and self.manager.hasExtension('os-floating-ips'): | 20:51 |
| mordred | ip = self.manager.addPublicIP(server['server_id']) | 20:51 |
| mordred | ^^ that's the code | 20:51 |
| mordred | and somehow, on the grizzly cloud, it's getting to the second line | 20:52 |
| mordred | (we do switch on the capability because rackspace doesn't have floating-ips | 20:52 |
| mordred | so the question now is - why is the cloud reportingt os-floating-ips but then reutrning | 20:53 |
| mordred | NotFound: FloatingIpPoolNotFound: Floating ip pool not found. (HTTP 404) (Request-ID: req-9dde3faa-8ab3-4d62-b6bc-782c92bc4d6c) | 20:53 |
| lifeless | mordred: I am going to guess at some magic setting like 'the network to pull floating-ips from needs name X' | 21:01 |
| lifeless | mordred: or | 21:01 |
| lifeless | mordred: just not proxying it to neutron at all | 21:01 |
| mordred | lifeless: I'm not sure what you mean | 21:02 |
| lifeless | is addPublicIP in the nova API ? | 21:06 |
| mordred | lifeless: no. that's in the proveder manager in nodepool | 21:06 |
| mordred | because, you guessed it, our clouds are not compatible | 21:07 |
| lifeless | ok, floating_ips.create | 21:08 |
| mordred | lifeless: I see this CONF.import_opt('default_floating_pool', 'nova.network.floating_ips') | 21:08 |
| mordred | in nova/network/neutronv2/api.py | 21:09 |
| lifeless | yeah | 21:09 |
| lifeless | I'm in the middle of nova/api/openstack/compute/contrib/floating_ips.py atm | 21:09 |
| * mordred trying to read through as well | 21:09 | |
| mordred | jeez this stuff is spaghetti | 21:09 |
| lifeless | defaults to 'nova' | 21:10 |
| lifeless | thats not a super useful value | 21:10 |
| lifeless | if I poke the grizzly cloud by hand | 21:10 |
| lifeless | can you test easily? | 21:10 |
| mordred | yup | 21:10 |
| mordred | hang on - lemme make aq uick test | 21:10 |
| lifeless | looks like you can pass pool in too | 21:11 |
| mordred | yes. I can recreate the error locally | 21:11 |
| mordred | lifeless: btw - I would not mind learning what you do in this case | 21:12 |
| mordred | at some point I might actually want to start learning something about openstack | 21:12 |
| lifeless | so I'e added default_floating_pool under [DEFAULT] on the grizzly cloud | 21:13 |
| lifeless | it's just bouncing all the apis | 21:14 |
| lifeless | done | 21:14 |
| *** esker has quit IRC | 21:14 | |
| lifeless | in principle, if we read it right, that should work on the grizzly poc now | 21:14 |
| *** mriedem has quit IRC | 21:14 | |
| mordred | testing | 21:14 |
| mordred | heh | 21:14 |
| mordred | ERROR: HTTPSConnectionPool(host='poc.tripleo.org', port=10000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 2] No such file or directory) | 21:14 |
| lifeless | nova list works for me | 21:15 |
| lifeless | you may have hit it while it was restarting ? | 21:15 |
| lifeless | mordred: still go no? | 21:17 |
| lifeless | https://bugs.launchpad.net/tripleo/+bug/1239450 | 21:18 |
| uvirtbot | Launchpad bug 1239450 in tripleo "nova floating-ip extension doesn't work on overcloud" [High,Triaged] | 21:18 |
| mordred | worked! | 21:18 |
| lifeless | ok | 21:18 |
| lifeless | so that's in place on the grizzly cloud | 21:18 |
| lifeless | and we'll get it in place on the cd-overcloud shotly | 21:19 |
| lifeless | shortly | 21:19 |
| mordred | neat. it's like we've learned something! | 21:19 |
| clarkb | what was the fix? creating a nova ip pool? | 21:19 |
| * mordred is also now set up locally to talk to the infra account on the grizzly cloud | 21:19 | |
| mordred | so I'll work on testing nodepool | 21:19 |
| lifeless | 10:13 < lifeless> so I'e added default_floating_pool under [DEFAULT] on the grizzly cloud | 21:19 |
| lifeless | clarkb: ^ | 21:19 |
| mordred | clarkb: he had to configure nova to talk to neutron for hte pool | 21:20 |
| lifeless | it defaults to 'nova', and the external network we're making in neutron is called 'ext-net' | 21:20 |
| lifeless | nova !- ext-net | 21:20 |
| clarkb | gotcha | 21:20 |
| *** senk has joined #openstack-infra | 21:20 | |
| mordred | clarkb: if I run a nodepool on my laptop, do I _have_ to have a zmq and a zuul now? | 21:27 |
| clarkb | mordred: yes I think so | 21:32 |
| *** cody-somerville has quit IRC | 21:32 | |
| clarkb | because gearman doesn't really allow you to inspect finished jobs, just the queues | 21:33 |
| *** jhesketh has joined #openstack-infra | 21:33 | |
| *** jhesketh has quit IRC | 21:34 | |
| *** jhesketh has joined #openstack-infra | 21:34 | |
| mordred | ah. there is a fake-servers.py in tools | 21:36 |
| *** dafter has quit IRC | 21:38 | |
| *** Ryan_Lane has joined #openstack-infra | 21:40 | |
| sdague | mordred: that cloning on the cgit cluster seems to still be there - https://review.openstack.org/#/c/51484/1 | 21:46 |
| sdague | clarkb: also ^^^ that's the keystone support, so monday we should figure out the CD strategy for landing changes from this repo to the log servers | 21:46 |
| mordred | ++ | 21:49 |
| clarkb | I like the idea of two virtualenvs, then run the different executables in each vhost | 21:50 |
| mordred | good. now we just have to engineer that | 21:50 |
| mordred | clarkb: did you see/care about any of hte dh_virtualenv conv from yesterday? | 21:50 |
| clarkb | no | 21:51 |
| clarkb | I didn't try catching up on that scrollback | 21:51 |
| mordred | fair | 21:51 |
| mordred | clarkb: check out https://github.com/spotify/dh-virtualenv | 21:51 |
| mordred | clarkb: suppress your gag reflex for a bit too | 21:51 |
| mordred | clarkb: because I think we all have the initial reaction of "EW" | 21:52 |
| mordred | but give it an honest read - I think it might be a decently practical solution for some of our things | 21:52 |
| clarkb | what does it do that pip and virtualenv don't? | 21:52 |
| mordred | clarkb: it makes a debian package that contains the virtualenv - so we could install the deb with puppet easily | 21:52 |
| mordred | clarkb: it edits the shbangs so that you don't have to active the virtualenv to use scripts installed in it | 21:53 |
| mordred | so I think it would be potentially super easy to make trunnk and branch packages of infra things for our consumption only that way | 21:54 |
| mordred | I may give a shot at it for os-loganalyze and see what it looks like | 21:54 |
| mordred | (locally, of course - I'm about to be on a flight to argentina | 21:55 |
| mordred | ) | 21:55 |
| *** aspiers has joined #openstack-infra | 21:55 | |
| clarkb | I see, so it gives us easy package installs with virtualenvs and not needing to do tons of packaging | 21:55 |
| mordred | yah | 21:56 |
| clarkb | mordred: don't forget the call tomorrow | 21:56 |
| clarkb | 9am PST iirc | 21:56 |
| mordred | clarkb: there is a cal tomorrow? | 21:56 |
| mordred | clarkb: yes! | 21:57 |
| mordred | I'll be there. | 21:57 |
| mordred | thanks | 21:57 |
| mordred | clarkb, lifeless: I have a nodepool up and trying to do things with the grizzly cloud | 22:01 |
| mordred | lifeless: I'm getting ConnectionError: HTTPSConnectionPool(host='poc.tripleo.org', port=10000): Max retries exceeded with url: /v2.0/tokens (Caused by <class 'socket.error'>: [Errno 2] No such file or directory) | 22:02 |
| mordred | but that could be due to airpane wifi | 22:02 |
| lifeless | mordred: looks fine to me, so yes. | 22:02 |
| lifeless | mordred: where are you flying now :) | 22:02 |
| mordred | lifeless: k. it's getting farther now | 22:03 |
| mordred | lifeless: this instant to atlanta - in a couple of hours to argentina | 22:03 |
| lifeless | mordred: if I can suggest; open a screen session somewhere | 22:03 |
| lifeless | mordred: e.g. in that grizzly cloud itself | 22:03 |
| mordred | lifeless: yah. well, that's a whole different kind of pain - but yup | 22:04 |
| *** aspiers has quit IRC | 22:04 | |
| *** slong has joined #openstack-infra | 22:06 | |
| mordred | jeblair: https://review.openstack.org/#/c/51465/2/nodepool/nodepool.py doesn't seem to work either - I get: | 22:06 |
| mordred | ip = self.manager.addPublicIP(server[server_id]) | 22:07 |
| mordred | KeyError: u'fdf82596-5d82-427a-9416-d0c2b452c700' | 22:07 |
| mordred | jeblair: but that is a valid server | 22:07 |
| clarkb | mordred: is that the bug that pabelanger's fix fixes? | 22:07 |
| mordred | jeblair: the KeyError seems to be something from the python side? | 22:07 |
| * mordred is landing | 22:07 | |
| mordred | clarkb: maybe? | 22:08 |
| mordred | clarkb: I'll try applying his patches too | 22:08 |
| lifeless | is it partial-fix or partial-bug ? | 22:14 |
| lifeless | clarkb: ^ | 22:14 |
| clarkb | lifeless: both! let me check the one we have documetned | 22:18 |
| *** sarob has joined #openstack-infra | 22:19 | |
| clarkb | "partial-bug" | 22:19 |
| *** michchap has quit IRC | 22:20 | |
| lifeless | thanks | 22:20 |
| *** sarob has quit IRC | 22:23 | |
| *** michchap has joined #openstack-infra | 22:23 | |
| *** erfanian has joined #openstack-infra | 22:24 | |
| *** jhesketh__ has joined #openstack-infra | 22:41 | |
| pabelanger | clarkb, mordred yes | 22:48 |
| pabelanger | addPublicIP(server_id) I believe | 22:48 |
| *** sarob has joined #openstack-infra | 22:49 | |
| *** cody-somerville has joined #openstack-infra | 22:51 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add os-loganalyze to #openstack-infra https://review.openstack.org/51489 | 22:55 |
| jeblair | 22:09 < mordred> ip = self.manager.addPublicIP(server[server_id]) | 22:57 |
| jeblair | mordred: that's from patchset 1 of that change | 22:57 |
| jeblair | mordred: patchset 2 (which you linked to) fixes that | 22:58 |
| *** sarob has quit IRC | 22:58 | |
| sdague | jeblair: the cgit mirroring still acting up? | 23:02 |
| jeblair | sdague: i was not aware that it was | 23:02 |
| sdague | jeblair: https://review.openstack.org/#/c/51491/ | 23:02 |
| sdague | I seem to hit it about 25% of the time | 23:03 |
| sdague | http://logs.openstack.org/91/51491/1/gate/gate-os-loganalyze-python27/0308009/console.html | 23:03 |
| jeblair | sdague: ah, then it's likely one of the servers didn't get your project replicated | 23:03 |
| sdague | yeh, fungi was looking at it yesterday morning, and I thought he'd applied a fix, maybe a server was missing from there | 23:03 |
| sdague | from that | 23:03 |
| jeblair | sdague: weird, it's on all the servers, but git02 seems to have different refs | 23:05 |
| jeblair | http://git02.openstack.org:8080/cgit/openstack-infra/os-loganalyze | 23:05 |
| jeblair | sdague: i just forced replication and git02 is now in sync with the others | 23:05 |
| sdague | ok, cool | 23:06 |
| sdague | any idea why it seems to keep tripping over it? | 23:06 |
| jeblair | sdague: no; i'm curious as to what fungi did to fix it. if we did the same thing, we have a problem. | 23:06 |
| sdague | some of it is in backlog from yesterday | 23:07 |
| jeblair | sdague: ah, i think the actual problem is the git repo on the jenkins slave | 23:08 |
| sdague | ok | 23:08 |
| jeblair | sdague: it may have cloned while the repo had no content | 23:08 |
| sdague | oh, then remains stale? | 23:09 |
| jeblair | sdague: yep, 'git reset --hard' doesn't work on a repo with no commits | 23:09 |
| sdague | precise35 seems to be the one causing me grief at the moment | 23:09 |
| jeblair | sdague: we should probably just have gerrit-git-prep 'rm -fr .' if 'git reset --hard' fails | 23:09 |
| jeblair | sdague: i manually did that on precise35, so that should improve your odds of success | 23:10 |
| sdague | cool, thanks | 23:10 |
| sdague | reverify again, we'll see :) | 23:10 |
| sdague | seems to have worked, it actually ran on precise35 that time | 23:13 |
| sdague | thanks | 23:13 |
| jeblair | sdague: np | 23:14 |
| openstackgerrit | lifeless proposed a change to openstack-infra/nodepool: Don't use max_overflow with sqlite https://review.openstack.org/49639 | 23:16 |
| openstackgerrit | lifeless proposed a change to openstack-infra/nodepool: Add the ability to pass pool for AddPublicIP https://review.openstack.org/49833 | 23:16 |
| *** cody-somerville has quit IRC | 23:16 | |
| pabelanger | lifeless, testing out the patches? | 23:19 |
| lifeless | pabelanger: gerrit whinged, it needed a rebase | 23:20 |
| lifeless | pabelanger: I'm not sure why you have it built on the sqlite change; but I didn't want to fiddle too much | 23:20 |
| lifeless | pabelanger: so I just pulled it down, rebased and pushed back up | 23:20 |
| *** sarob has joined #openstack-infra | 23:24 | |
| pabelanger | lifeless, ya, the sqlite stuff could be rebased out. It was more to get tox going locally cause I was too lazy to setup mysql | 23:26 |
| lifeless | ok, jeblair seemed concerned about that | 23:27 |
| lifeless | so let me rebase it out | 23:27 |
| pabelanger | cool here | 23:28 |
| openstackgerrit | lifeless proposed a change to openstack-infra/nodepool: Add the ability to pass pool for AddPublicIP https://review.openstack.org/49833 | 23:28 |
| lifeless | jeblair: / clarkb: ^ should be good to approve now | 23:28 |
| *** sarob has quit IRC | 23:33 | |
| clarkb | mordred: you are missing a great saints game | 23:38 |
| *** senk has quit IRC | 23:43 | |
| *** senk has joined #openstack-infra | 23:43 | |
| sdague | well except for them folding at the end :) | 23:48 |
| *** senk has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!