| tchaypo | my paste buffer is twins! | 00:00 |
|---|---|---|
| *** morazi has quit IRC | 00:00 | |
| tchaypo | https://bugs.launchpad.net/tripleo/+bug/1301220 is the useful bit | 00:00 |
| uvirtbot | Launchpad bug 1301220 in tripleo "pip can't find markupsafe when running devtest_seed.sh" [High,Triaged] | 00:00 |
| lifeless | tchaypo: so what happens if you do session.get of a url which 404s | 00:00 |
| lifeless | tchaypo: does it raise ? | 00:00 |
| lifeless | except requests.HTTPError as exc: | 00:00 |
| lifeless | level = 2 if exc.response.status_code == 404 else 1 | 00:00 |
| lifeless | cls._handle_fail(req, link, exc, url, cache=cache, level=level) | 00:00 |
| lifeless | looks like thats the 404 handler | 00:01 |
| lifeless | so yeah, requests bug - getting a path which is not there should be a 404 not an OSError | 00:01 |
| lifeless | first stage workaround would be to add an OSError except there I think - I'm concurring with your thinking | 00:02 |
| tchaypo | not a requests bug - download.py implements its own adapter that overrides BaseAdapter from requests | 00:02 |
| *** eguz has quit IRC | 00:02 | |
| tchaypo | so I think this is a pip bug - its Adapter should return a 404 if the stat fails | 00:02 |
| lifeless | oh okay | 00:03 |
| lifeless | also aiieeee why do folk make this stuff so complicated | 00:03 |
| tchaypo | but you're right: if download.py:149 raised, index.py:670 would fail and execution would drop to the "except requests.HTTPError" on 689 that you pointed out | 00:05 |
| tchaypo | and from that point we should be fine | 00:05 |
| *** andreaf2 has quit IRC | 00:06 | |
| tchaypo | Returning an HTTPError for something that didn't touch HTTP feels a little bit wrong, but no wronger than using requests to read the local filesystem | 00:06 |
| *** blamar has quit IRC | 00:06 | |
| tchaypo | looking at File "/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/sessions.py", line 486, in send | 00:08 |
| tchaypo | r = adapter.send(request, **kwargs)y | 00:08 |
| tchaypo | bah | 00:08 |
| tchaypo | looking at the latest in github, this is still the case now, so I'll get them a pull request this afternoon | 00:08 |
| tchaypo | my truck is here | 00:08 |
| StevenK | Your truck? | 00:08 |
| *** blamar has joined #tripleo | 00:09 | |
| *** maru_afk is now known as marun | 00:20 | |
| *** yamahata has joined #tripleo | 00:22 | |
| *** blamar has quit IRC | 00:26 | |
| *** matsuhashi has joined #tripleo | 00:28 | |
| *** blamar has joined #tripleo | 00:32 | |
| greghaynes | aye, looks like my issue isnt a scaling problem, there just exists both a NovaCompute0Config and NovaComputeConfig and the latter isnt being pulled into the resulting template :/ | 00:39 |
| *** rlandy has quit IRC | 00:40 | |
| lifeless | tchaypo: once we have a fix, can we arrange to monkeypatch the pip we use to have it? | 00:43 |
| *** julim has quit IRC | 00:55 | |
| *** julim has joined #tripleo | 00:56 | |
| *** julim has quit IRC | 01:00 | |
| StevenK | Hm | 01:02 |
| *** matsuhashi has quit IRC | 01:05 | |
| *** matsuhashi has joined #tripleo | 01:06 | |
| *** julim has joined #tripleo | 01:07 | |
| StevenK | Hm | 01:07 |
| Lotus907efi | lifeless: I am working on a fairly lengthy document that explains how to build a precise based qcow2 image that can then be booted using the nova boot command onto a baremetal node. | 01:07 |
| StevenK | os-cloud-config is in the cache, but unbuilt, so I can't call any scripts | 01:08 |
| lifeless | Lotus907efi: from your custom repo, right ? | 01:08 |
| Lotus907efi | it is based partially on the directions you provided a while ago on how to boot a baremetal node | 01:08 |
| Lotus907efi | well that part is later. this is a fairly long document and I am just up to the part of doing it with plain precise | 01:09 |
| lifeless | Lotus907efi: ok. Why is it long? | 01:09 |
| StevenK | And setup.py build doesn't give me scripts either | 01:09 |
| Lotus907efi | afterwards I will add the bits on how to do the same thing from our private repo | 01:09 |
| StevenK | Oh, wait | 01:09 |
| Lotus907efi | lifeless: dude it is a fairly complex operation | 01:09 |
| *** cwolferh_ has joined #tripleo | 01:09 | |
| Lotus907efi | wait until I have saved something and then I will send you the link | 01:10 |
| lifeless | Lotus907efi: ok :) | 01:10 |
| Lotus907efi | if you think then think I could have covered it in a shorter document and completely described all of the steps, you can let me know what I can cut out. :D | 01:10 |
| *** yamahata has quit IRC | 01:11 | |
| Lotus907efi | if you then think I could have covered it in a shorter document and completely described all of the steps, you can let me know what I can cut out. :D | 01:11 |
| *** julim has quit IRC | 01:12 | |
| Lotus907efi | and it will be even longer once I get to the directions on how to do this from our custom private repo | 01:12 |
| *** cwolferh has quit IRC | 01:12 | |
| Lotus907efi | since for instance that required me to create a new $HOME/.cache/image-create/precise-server-cloudimg-amd64-root.tar.gz file that matched our repo contents | 01:13 |
| *** TravT has quit IRC | 01:14 | |
| *** geerdest has quit IRC | 01:24 | |
| Lotus907efi | lifeless: I e-mailed the link to the wiki page | 01:27 |
| Lotus907efi | just now | 01:28 |
| Lotus907efi | it is not complete yet but is about 70% done I think | 01:28 |
| *** nosnos has joined #tripleo | 01:29 | |
| lifeless | k, I'll dig up my OATH and have a look shortly | 01:30 |
| *** julim has joined #tripleo | 01:37 | |
| *** ilives has joined #tripleo | 01:41 | |
| *** newell_ has quit IRC | 01:42 | |
| Lotus907efi | your OATH ? | 01:46 |
| StevenK | Lotus907efi: VPN token | 01:48 |
| *** ilives has quit IRC | 01:48 | |
| *** ilives has joined #tripleo | 01:49 | |
| Lotus907efi | oh | 01:54 |
| lifeless | Lotus907efi: it seems to repeat itself | 02:08 |
| lifeless | Lotus907efi: e.g. there are two 'goal' sections | 02:09 |
| lifeless | with the same goal | 02:09 |
| lifeless | actually 8 goal sections | 02:09 |
| lifeless | Lotus907efi: this seems to document setting up a baremetal cloud, which I would make the starting environment | 02:10 |
| lifeless | Lotus907efi: since you don't do that per image you boot | 02:10 |
| lifeless | begging for reviews - https://review.openstack.org/#/c/84073/ | 02:27 |
| *** cwolferh_ has quit IRC | 02:28 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Make it possible to pass real hardware in. https://review.openstack.org/83188 | 02:29 |
| tchaypo | Okay, all | 02:32 |
| tchaypo | My boxen are in storage now | 02:32 |
| tchaypo | The cardboard kind | 02:32 |
| tchaypo | Now | 02:32 |
| tchaypo | To get some food and a desk so I can get back to work | 02:32 |
| StevenK | tchaypo: Moving? | 02:34 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Change templateUrl to absolute path https://review.openstack.org/84926 | 02:37 |
| tchaypo | Had a passle of boxes that hadn't been unpacked since the last move 12 months ago | 02:39 |
| tchaypo | Decided to put them in external storage to free up cupboard and floor space for things that actually get touched | 02:39 |
| *** julim has quit IRC | 02:50 | |
| *** weshay has joined #tripleo | 02:53 | |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1290488 | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| uvirtbot | Launchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete] | 03:00 |
| *** spzala has quit IRC | 03:02 | |
| tchaypo | lifeless: Although it was phrased as a query, I'm going to interpret your hint about the monkey-patch as a suggestion | 03:04 |
| lifeless | tchaypo: works for me | 03:04 |
| tchaypo | ;) | 03:05 |
| *** matsuhashi has quit IRC | 03:05 | |
| tchaypo | and now i have a burger, bbs. | 03:06 |
| SpamapS | greghaynes: the OS::Heat::StructuredConfig does not scale out, it is one config to many deployments | 03:11 |
| SpamapS | greghaynes: OS::Heat::StructuredDeployment should scale out. | 03:11 |
| *** sdake has quit IRC | 03:14 | |
| *** nosnos has quit IRC | 03:23 | |
| greghaynes | SpamapS: Makes sense - I was incorrect on that being a scaling issue | 03:35 |
| greghaynes | I dont think the NovaComputeConfig is included anywhere though and is referred to by NovaCompute0Config | 03:36 |
| *** eghobo has joined #tripleo | 03:54 | |
| *** rpodolyaka1 has joined #tripleo | 04:04 | |
| tchaypo | I just had to "cd pip/pip" | 04:05 |
| tchaypo | i feel very british all of a sudden | 04:05 |
| pleia2 | haha | 04:06 |
| *** ramishra has joined #tripleo | 04:09 | |
| *** nosnos has joined #tripleo | 04:09 | |
| *** ramishra_ has joined #tripleo | 04:09 | |
| tchaypo | well blow me down with a feather | 04:13 |
| tchaypo | i just had cause to use try .. except .. else | 04:13 |
| *** ramishra has quit IRC | 04:13 | |
| tchaypo | lifeless: I'm scareded now | 04:15 |
| *** matsuhashi has joined #tripleo | 04:15 | |
| tchaypo | I write some code and it seems to have worked first try. | 04:15 |
| lifeless | tchaypo: welcome to the house of fun | 04:15 |
| tchaypo | granted, it's a total of 5 lines | 04:15 |
| tchaypo | but still. | 04:15 |
| lifeless | mordred: ^ we're making pip work properly with file:/// mirrors | 04:15 |
| *** akuznetsov has joined #tripleo | 04:19 | |
| tchaypo | http://paste.openstack.org/show/74921/ | 04:20 |
| tchaypo | much satisfactorier | 04:20 |
| StevenK | tchaypo: try ... except ... finally ... else ? | 04:20 |
| tchaypo | not had to do that yet | 04:22 |
| tchaypo | https://github.com/pypa/pip/pull/1704 | 04:24 |
| tchaypo | so my next question is | 04:24 |
| tchaypo | s/question is/thing to figure out is/ | 04:25 |
| tchaypo | how to monkey patch in this case. | 04:25 |
| lifeless | tchaypo: you could pip install that branch of pip ? | 04:26 |
| lifeless | for the evil | 04:26 |
| tchaypo | find the element that installs pip and tweak it to slap that patch on top? | 04:26 |
| lifeless | tchaypo: I think we install pip a few places | 04:26 |
| tchaypo | yeah, I *could* but that doesn't scale. having a directory of patches to slap on does | 04:26 |
| lifeless | tchaypo: and we fail to install pip in some places because its installed implictly by making a virtualenv | 04:26 |
| tchaypo | there's an element called pip-and-virtualenv | 04:26 |
| tchaypo | silly me assumed that would be the place that installs pip and virtualenv | 04:27 |
| lifeless | tchaypo: yeah, I'm not sure everything uses it yet is all. STill - you'll know when you've stabbed it hard enough | 04:27 |
| lifeless | SpamapS: you've gone soft in your review dotage | 04:27 |
| StevenK | steven@undermined:~/openstack/openstack/tripleo-incubator% register-nodes.py --service-host seed --nodes <(jq '[.nodes[0]]' $TE_DATAFILE) | 04:27 |
| StevenK | steven@undermined:~/openstack/openstack/tripleo-incubator% | 04:27 |
| StevenK | \o/ | 04:27 |
| *** BadCub01 has quit IRC | 04:28 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Add a register-nodes command and associated API https://review.openstack.org/84933 | 04:28 |
| lifeless | SpamapS: not register-nodes ... ? | 04:29 |
| lifeless | SpamapS: as in, why the .py ? | 04:29 |
| *** michchap has quit IRC | 04:29 | |
| *** ramishra_ has quit IRC | 04:29 | |
| *** michchap has joined #tripleo | 04:29 | |
| tchaypo | lifeless: i think i might do the patching in the pypi element | 04:30 |
| StevenK | lifeless: scripts/register-nodes is already shipped by -incubator | 04:30 |
| tchaypo | that should solve the problem for xuhaiwei and anyone else we care about | 04:30 |
| xuhaiwei | the patch is posted? | 04:31 |
| tchaypo | xuhaiwei: https://github.com/jamezpolley/pip/commit/66dff9a8a3bf95d105792f2aae3dd7caa65b5253 | 04:32 |
| tchaypo | but that's just a patch for pip; it's presumably going to take Some Time for that to be accepted, merged, and released; so in the meantime I'm going to try to make the pypi element monkey-patch pip in the chroot | 04:33 |
| tchaypo | that's a patch we should be able to get live tomorrow if lifeless can talk someone else into givint it a +2 | 04:33 |
| tchaypo | ... and even if i can't get it live, it's a patch you'll be easily able to pull from gerrit to fix your particular issue | 04:34 |
| *** ramishra_ has joined #tripleo | 04:34 | |
| clarkb | oh is this the bug that apache fixes? | 04:34 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Build os-cloud-config in setup-clienttools https://review.openstack.org/84935 | 04:35 |
| tchaypo | clarkb: this is a bug caused by pip not handling a failure to stat a file in the same way it would handle a 404 returned from apache | 04:35 |
| mordred | lifeless: yay! | 04:36 |
| tchaypo | should I try to be fancy and locate pip/download.py, or should i just hardcode the location? | 04:36 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Link register-nodes in setup-clienttools https://review.openstack.org/84936 | 04:36 |
| clarkb | tchaypo: right, I am suggesting that a web server fixes this problem in a much less hacky way than monkey patching pip does | 04:37 |
| StevenK | lifeless: So, .py because scripts already includes a register-nodes, and you can't overwrite a file with a symlink | 04:37 |
| clarkb | tchaypo: then when pip cuts a release remove the web server | 04:37 |
| tchaypo | clarkb: alternatively, https://github.com/jamezpolley/pip/commit/66dff9a8a3bf95d105792f2aae3dd7caa65b5253 makes pip handle the stat failure | 04:37 |
| StevenK | lifeless: Can you chime in on https://review.openstack.org/#/c/84330/ ? | 04:38 |
| *** killer_prince has joined #tripleo | 04:39 | |
| *** matsuhashi has quit IRC | 04:40 | |
| tchaypo | clarkb: I agree that it's not something I'm likely to run into myself; i can't imagine not having a web-based python mirror handy, unless I'm on a plane, in which case I wouldn't be coding | 04:40 |
| tchaypo | or running image-create | 04:40 |
| tchaypo | but pip says it can do a thing, I think it's reasonable to expect it to do that thing | 04:41 |
| tchaypo | "just use the pypi element and things will be cached" is smoething i feel comfortable telling everyone | 04:41 |
| clarkb | tchaypo: I agree... I am suggesting you shouldn't monkey patch pip in the interem | 04:41 |
| clarkb | you should use a web server instead | 04:41 |
| tchaypo | I don't have the problem, xuhaiwei | 04:42 |
| tchaypo | does | 04:42 |
| tchaypo | and now that i think about it, the web server probably doesn't even require running bandersnatch | 04:42 |
| xuhaiwei | tchaypo: you mean you have fixed it? | 04:42 |
| xuhaiwei | it's a bug of pip? | 04:43 |
| mordred | tchaypo: I'm very excited about your pip patch | 04:43 |
| *** weshay has quit IRC | 04:43 | |
| tchaypo | really? | 04:44 |
| tchaypo | because I'm getting exccited about clarkb's idea | 04:44 |
| mordred | I want BOTH THINGS | 04:44 |
| mordred | I want for file:// and http:// to not behave differently, because that's just a good thing | 04:44 |
| clarkb | ++ | 04:44 |
| tchaypo | spinning up nginx, point the docroot at ~/.cache/image-create/pypi/mirror | 04:44 |
| mordred | but also, sticking a web server and mod_autoindex makes it work now | 04:44 |
| tchaypo | index.html is already built | 04:44 |
| mordred | nod | 04:45 |
| mordred | sorry - I forgot - in pypi.o.o we delete index.html and use autoindex | 04:45 |
| mordred | because we keep a cumulative mirror | 04:45 |
| tchaypo | yeah sure | 04:45 |
| clarkb | mordred: and 3 different sources | 04:45 |
| mordred | but for the build you just did - pointing nginx at the dir is totes fine | 04:45 |
| mordred | as a temp workaround | 04:45 |
| mordred | until the patch landes | 04:45 |
| tchaypo | but for xuhaiwei who efectively has no internets and wants to be able to build entirely out of ~/.cache/ | 04:46 |
| *** matsuhashi has joined #tripleo | 04:46 | |
| tchaypo | xuhaiwei: If I suggested we fix your problem by installing nginx and having it serve the contents of your ~/.cache/image-create/pypi/mirror - how scary does that sound to you? | 04:47 |
| tchaypo | I think I can walk you through it - I did something fairly similar just a few days ago so i have an nginx.conf lying around | 04:48 |
| mordred | tchaypo: I have pinged dstufft in #openstack-infra about your patch - he's a pip maintainer | 04:48 |
| xuhaiwei | tchaypo: It's ok for me, i think :) | 04:48 |
| *** ramishra_ has quit IRC | 04:49 | |
| tchaypo | xuhaiwei: cool :) I'm going to agree with clarkb that that's probably the simplest thing for you to do - get nginx serving and then change your $PYPI_MIRROR to point at nginx | 04:49 |
| *** ramishra has joined #tripleo | 04:51 | |
| lifeless | StevenK: but we don't need both | 04:51 |
| lifeless | StevenK: so the commit that installs os-c-c in tools could fix that up | 04:52 |
| xuhaiwei | tchaypo: could you explain it for me? I am a little confused | 04:52 |
| lifeless | tchaypo: fancy schmancy | 04:52 |
| xuhaiwei | tchaypo: the link you gave me just now, couldn't solve the problem? | 04:53 |
| tchaypo | xuhaiwei: it will, but only after the pip maintainers accept the patch, and there's a new release of pip that contains the patch | 04:54 |
| xuhaiwei | can't do it by hand? | 04:54 |
| tchaypo | actually, you could | 04:55 |
| lifeless | it needs to be in tree to fix it | 04:55 |
| tchaypo | i take that back, it'd be quite hard to do it by hand | 04:55 |
| xuhaiwei | another way, use nginx? | 04:55 |
| lifeless | tchaypo: so, I'm going to argue against nginx / apache | 04:55 |
| tchaypo | because you need to patch the pip that gets installed inside the chroot. | 04:55 |
| StevenK | lifeless: I can not work out how to get scripts built in place | 04:55 |
| lifeless | tchaypo: because a) its installing a listening service on users workstations | 04:56 |
| tchaypo | having a monkey-patch inside the element so that it can be patched after it's installed but before it's used is really the only way | 04:56 |
| lifeless | tchaypo: b) not everyone is familiar with, nor wants to operate a webserver locally | 04:56 |
| lifeless | tchaypo: c) dstufft just said your patch is the bom | 04:56 |
| lifeless | b | 04:56 |
| tchaypo | yeah, okay. | 04:57 |
| greghaynes | maybe make this an extra element that people can opt-in to via DIB_EXTRA_ARGUMENTS? | 04:57 |
| tchaypo | the pypi-but-more-magical element? | 04:58 |
| xuhaiwei | tchaypo: so if using nginx, how to do it? | 04:58 |
| greghaynes | haha | 04:59 |
| lifeless | tchaypo: you could make it a fixed-pypi element yes, which we can delete once its merged, if that feels better | 04:59 |
| tchaypo | xuhaiwei: lifeless has convinced me that's a bad idea | 04:59 |
| tchaypo | or at least, in the time it would take me to explain it to you I could just write some code that fixes the problem without needing to set up nginx | 04:59 |
| *** yamahata has joined #tripleo | 05:02 | |
| *** jcoufal has joined #tripleo | 05:03 | |
| xuhaiwei | tchaypo: ok, that's a good idea | 05:08 |
| *** Rakesh5 has joined #tripleo | 05:08 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Install kpartx on Ubuntu https://review.openstack.org/83372 | 05:11 |
| *** yamahata has quit IRC | 05:12 | |
| *** yamahata has joined #tripleo | 05:12 | |
| lifeless | StevenK: you can't. Why do you want that ? | 05:19 |
| StevenK | lifeless: Because we probably want to call the CLI scripts from incubator and toci | 05:20 |
| *** eghobo has quit IRC | 05:20 | |
| StevenK | lifeless: And indeed, that was my plan with this register-nodes CLI | 05:22 |
| *** sdake has joined #tripleo | 05:23 | |
| tchaypo | root@bill:/# locate download.py | grep pip | grep opt.stack.venvs | grep -v pyc | wc -l | 05:23 |
| tchaypo | 15 | 05:23 |
| lifeless | tchaypo: for any given venv its static w.r.t. the venv root | 05:25 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-image-elements: Monkey-patch pip to handle file:// urls gracefully https://review.openstack.org/84946 | 05:32 |
| tchaypo | okay, while image-create is running to see if that's going to work, I'm going to relocate. Back in 20 or so. | 05:35 |
| lifeless | also lol - https://github.com/mitsuhiko/jinja2/pull/226 | 05:35 |
| lifeless | StevenK: tchaypo: would you guys like a f2f meetup in a few weeks time? Lynne will be heading AU-ward | 05:36 |
| lifeless | tchaypo: I don't think https://review.openstack.org/#/c/84946/1/elements/pip-enable-file-urls/install.d/02-install.d will work | 05:37 |
| StevenK | lifeless: For a day, few days, week? | 05:39 |
| lifeless | StevenK: probably a thursday/friday workwise anyhow | 05:39 |
| lifeless | likely there for the weekend to socialise too | 05:39 |
| StevenK | lifeless: Sounds brilliant to me | 05:40 |
| tchaypo | I didn't mean to disappear quite that suddenly | 05:44 |
| tchaypo | But I've been having my iphone for wifi and it chose that exact second to hit 0% | 05:45 |
| StevenK | lifeless: Dinner with Lynne, you and C one night would be excellent too | 05:45 |
| tchaypo | lifeless: I'll be surprised if it does but it was a convenient time to move | 05:46 |
| tchaypo | lifeless: F2f sounds good | 05:46 |
| lifeless | StevenK: if she can fit it in | 05:48 |
| lifeless | StevenK: otherwise I'll come sob on your shoulder about how my wife abandons me in sydney for her family and friends | 05:48 |
| *** CaptTofu has joined #tripleo | 05:52 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Relabel /tftpboot if selinux is used https://review.openstack.org/84949 | 05:52 |
| StevenK | lifeless: Haha | 05:53 |
| *** jprovazn has joined #tripleo | 05:53 | |
| StevenK | lifeless: Well, dinner with you works too :-) | 05:53 |
| lifeless | (she may be heading out blacktown way - tricky to do dinner with C with that latency | 05:54 |
| tchaypo | well stevek is at parra | 05:54 |
| StevenK | Which is on the way | 05:55 |
| lifeless | yes | 05:55 |
| lifeless | 30m vs 2hr | 05:55 |
| lifeless | 'on the way' | 05:55 |
| StevenK | lifeless: IE, you have to pass by Parramatta to get to Blacktown | 05:55 |
| lifeless | StevenK: yes, got that | 05:55 |
| *** andreaf2 has joined #tripleo | 05:57 | |
| *** Guest13973 is now known as ohadlevy | 05:58 | |
| *** ohadlevy has joined #tripleo | 05:58 | |
| StevenK | Huh. https://review.openstack.org/#/c/83695/1 << I thought that already worked | 06:00 |
| tchaypo | i can confirm that althoguh the patch applies cleanly, this only fixes system pip not pip inside any of the virtualenvs | 06:00 |
| tchaypo | even virtualenvs created after patching system pip. | 06:00 |
| tchaypo | i believe virtualenv has its own copy of pip... | 06:01 |
| tchaypo | or does it just download and install a copy?a | 06:02 |
| tchaypo | maybe a better tactic would be to update the archive in the pypi mirror | 06:02 |
| *** andreaf_ has joined #tripleo | 06:05 | |
| *** andreaf_ has quit IRC | 06:07 | |
| *** andreaf_ has joined #tripleo | 06:07 | |
| *** andreaf has quit IRC | 06:08 | |
| *** andreaf_ has quit IRC | 06:09 | |
| *** andreaf has joined #tripleo | 06:09 | |
| *** andreaf has quit IRC | 06:10 | |
| *** andreaf has joined #tripleo | 06:10 | |
| jprovazn | lifeless, hi, do I need extra special premissions to be able to add comments to trello cards? Currrently I can do subscribe, print and export only | 06:12 |
| *** andreaf2 has quit IRC | 06:12 | |
| tchaypo | jprovazn: yeah, you need to be a member of the group | 06:12 |
| jprovazn | tchaypo, thanks, do you know who has admin perms to add me to the group? | 06:13 |
| jprovazn | I guess lifeless :) | 06:13 |
| lifeless | tchaypo: ooo I like that strategy | 06:13 |
| lifeless | jprovazn: whats your username? | 06:14 |
| tchaypo | lifeless, GheRivero should be online soon, cmsj should be online later | 06:14 |
| jprovazn | lifeless, janprovaznik | 06:14 |
| tchaypo | lifeless: i hate it. it's unsimple. | 06:15 |
| tchaypo | i think this means i need to find the bit of pypi-mirror that does the "python setup.py blahblah" to build the wheel and patch that to have a special case for this one repo | 06:16 |
| *** rpodolyaka1 has quit IRC | 06:18 | |
| tchaypo | but wait, this is unix | 06:19 |
| tchaypo | i bet once i find out what a wheel is it's going to turn out to be a .tgz that i can unpack, mess with, and repack | 06:19 |
| lifeless | tchaypo: its a zip but yes | 06:19 |
| lifeless | tchaypo: but thats evil | 06:19 |
| lifeless | tchaypo: lets backup a second | 06:19 |
| tchaypo | *shrug* how do you think you edit email templates for jira? | 06:20 |
| lifeless | tchaypo: pip reads from pypi indices | 06:20 |
| lifeless | tchaypo: LALALALALALALA | 06:20 |
| lifeless | tchaypo: anyone can publish a pypi index | 06:20 |
| lifeless | tchaypo: with any amount of files in it | 06:20 |
| lifeless | tchaypo: as you know, from reading the code, you don't need index.html, for local disk bare directories are fine | 06:20 |
| lifeless | tchaypo: AIUI anyhow | 06:21 |
| tchaypo | so our element has a second pypi mirror with a second wheel that we fake having a higher number so it gets preferred? | 06:21 |
| lifeless | tchaypo: anyhow, point is, why not make a pre-release sdist of pip | 06:21 |
| tchaypo | aiui we had a change land a few days ago so that we don't use pre-releases any more | 06:21 |
| lifeless | tchaypo: yes, unless you opt into them | 06:21 |
| tchaypo | *thinks* actually that was specifically for tox, which used to prefer pre-releasees and other crazy things | 06:21 |
| *** rpodolyaka1 has joined #tripleo | 06:22 | |
| tchaypo | so in this case I'd have to make virtualenv prefer pre-releases when it ses up the env? | 06:22 |
| lifeless | tchaypo: so all you need to do is a) add the index in the pypi url list - using the existing variables | 06:22 |
| *** rpodolyaka1 has quit IRC | 06:23 | |
| *** rpodolyaka1 has joined #tripleo | 06:23 | |
| lifeless | tchaypo: and b) install pip from that with a specific version | 06:23 |
| *** rpodolyaka1 has quit IRC | 06:23 | |
| lifeless | tchaypo: or for full automation | 06:23 |
| *** rpodolyaka1 has joined #tripleo | 06:24 | |
| *** rpodolyaka1 has quit IRC | 06:24 | |
| tchaypo | afaict updating system pip isn't sufficient; when you 'virtualenv blah'. virtualenv looks for its preferred version of pip | 06:24 |
| *** rpodolyaka1 has joined #tripleo | 06:24 | |
| tchaypo | i can see how the method you outline updates the sytem pip but afaict the one inside the virtualenv will still be sad | 06:24 |
| tchaypo | it'd be nice if i suddenly realised that the system pip is all i need. | 06:25 |
| *** rpodolyaka1 has quit IRC | 06:25 | |
| lifeless | tchaypo: oh so | 06:25 |
| lifeless | let me be more specific | 06:25 |
| lifeless | after we upgrade the system pip | 06:25 |
| lifeless | do this | 06:25 |
| tchaypo | ah, one sec | 06:26 |
| lifeless | pip install --no-use-wheel -d /tmp/foobarbaz -U pip | 06:26 |
| lifeless | I think that will get you a source tree correctly | 06:26 |
| lifeless | patch that source tree | 06:26 |
| lifeless | then | 06:26 |
| lifeless | run pip install /tmp/foobarbaz | 06:26 |
| tchaypo | found it | 06:26 |
| lifeless | and after we make every virtualenv | 06:26 |
| lifeless | run run pip install /tmp/foobarbaz | 06:27 |
| tchaypo | virtualenv gets pip from /usr/local/lib/python2.7/dist-packages/virtualenv_support/pip-1.5.4-blahblah.whl | 06:27 |
| lifeless | I think that will give you what you need across the board without evil | 06:27 |
| lifeless | jprovazn: added | 06:28 |
| jprovazn | lifeless, thanks | 06:28 |
| tchaypo | okay, I've confirmed that virtualenv compiles a list of SEARCH_DIRS and looks inside them for anything matching "pip-*whl" and installs the first one it finds | 06:32 |
| tchaypo | and the list of SEARCH_DIRS seems to include | 06:33 |
| tchaypo | *drumroll* | 06:33 |
| tchaypo | . | 06:33 |
| *** matsuhashi has quit IRC | 06:36 | |
| tchaypo | okay, so it looks like to do it that way, we need to drop our special wheel either in /usr/local/lib/python2.7/dist-packages/virtualenv_support (trampling on the existing file) or .; . is hard unless we assume virtualenv is always run with $PWD=/root | 06:37 |
| *** CaptTofu has quit IRC | 06:37 | |
| *** CaptTofu has joined #tripleo | 06:38 | |
| *** matsuhashi has joined #tripleo | 06:39 | |
| tchaypo | so getting back to your idea - it sounds like the challenge there is finding a way to make virtualenv always do "pip install --pre -u pip" or "pip install pip==1.5.4.blah" after it creates the virtualenv | 06:39 |
| StevenK | Patch virtualenv so you can patch pip. WCPGW | 06:40 |
| tchaypo | WCPGW? | 06:41 |
| tchaypo | oh. | 06:41 |
| tchaypo | well my latest greatest idea is to patch | 06:41 |
| *** CaptTofu has quit IRC | 06:42 | |
| tchaypo | ... but it seems that activate.sh is not cloned from a template file, so i can't just patch that. | 06:42 |
| tchaypo | but how fun would it be if activate.sh did "pip install --pre -u pip" on every run! wheee! | 06:43 |
| lifeless | tchaypo: huh | 06:47 |
| lifeless | tchaypo: hell no | 06:47 |
| lifeless | tchaypo: so I don't understandw hy you need to change virtualenv | 06:47 |
| *** sdake has quit IRC | 06:48 | |
| lifeless | tchaypo: *all* the virtualenvs we make are in tripleo-image-elements/elements/*/install.d/* | 06:48 |
| lifeless | tchaypo: we don't need to fix this for virtualenvs we don't make. | 06:48 |
| *** mrunge has joined #tripleo | 06:48 | |
| tchaypo | was assuming there'd be many of them | 06:49 |
| lifeless | tchaypo: grep 'virtualenv' . -r :> | 06:49 |
| lifeless | I think there are twelve | 06:49 |
| tchaypo | 08:49:44 0 1 polleyj@bill:~/.cache/tripleo$ | 06:50 |
| tchaypo | -bash: /dev/null: Permission denied | 06:50 |
| tchaypo | 08:49:46 0 1 polleyj@bill:~/.cache/tripleo$ | 06:50 |
| tchaypo | -bash: /dev/null: Permission denied | 06:50 |
| tchaypo | 08:49:54 0 1 polleyj@bill:~/.cache/tripleo$ | 06:50 |
| tchaypo | 08:50:30 0 0 polleyj@bill:~/.cache/tripleo$ ls -l /dev | 06:50 |
| tchaypo | total 0 | 06:50 |
| tchaypo | -rw-r--r-- 1 root root 0 Apr 3 08:50 null | 06:50 |
| tchaypo | drwxr-xr-x 2 root root 0 Mar 22 03:53 pts | 06:50 |
| tchaypo | -bash: /dev/null: Permission denied | 06:50 |
| tchaypo | so that's a thing | 06:50 |
| StevenK | What have you done to your poor machine? | 06:52 |
| tchaypo | run create-image | 06:53 |
| tchaypo | with "break=after-error" set, then exited out of the chroot and ^Ced as it tried to clean up | 06:54 |
| *** e0ne has joined #tripleo | 06:54 | |
| *** rdopieralski has joined #tripleo | 06:55 | |
| lifeless | tchaypo: ah yes you may have rm -rf'd /dev | 06:55 |
| lifeless | bbs | 06:55 |
| tchaypo | such empathy. much compassion. wow | 06:55 |
| StevenK | Haha | 06:55 |
| tchaypo | ;) | 06:55 |
| tchaypo | fwiw i actually count about 5 calls to virtualenv in tripleo-image-elements/elements/*/install.d/* (with grep -r as some of the scripts are in subdirs of install.d) | 06:56 |
| tchaypo | so. I'm going to go away for a while - head home, have dinner, that kind of thing. I'll be back in a few hours with a plan to patch ALL THE VIRTUALENVS | 06:57 |
| tchaypo | and maybe fix /dev | 06:57 |
| *** xuhaiwei has quit IRC | 07:00 | |
| *** xuhaiwei has joined #tripleo | 07:02 | |
| *** zigo has quit IRC | 07:04 | |
| *** d0ugal has joined #tripleo | 07:04 | |
| *** d0ugal has quit IRC | 07:04 | |
| *** d0ugal has joined #tripleo | 07:04 | |
| *** sdague has quit IRC | 07:05 | |
| *** sdague has joined #tripleo | 07:05 | |
| *** Hazelesque has quit IRC | 07:07 | |
| *** zigo has joined #tripleo | 07:08 | |
| *** Hazelesque_ has joined #tripleo | 07:08 | |
| GheRivero | morning all | 07:08 |
| *** mikal has quit IRC | 07:10 | |
| *** mikal has joined #tripleo | 07:10 | |
| lifeless | good news everyone we're now only 15hrs behind | 07:13 |
| lifeless | morning GheRivero ! | 07:14 |
| lifeless | GheRivero: how is the perf optimisation stuff coming along? | 07:14 |
| lifeless | GheRivero: I am about to send you and the galway folks a mail | 07:14 |
| GheRivero | morning lifeless, from the tripleo-incubator story, it's mostly done (just a couple of more tests I want to do) | 07:16 |
| lifeless | GheRivero: cool (mail sent btw) | 07:17 |
| GheRivero | With the test machine they give, from a 1419 build time, I managed to reduce it to 908 | 07:18 |
| lifeless | thats pretty nice | 07:19 |
| GheRivero | I resume most of it in the last comment on the bug | 07:19 |
| lifeless | ok, I'll peek later | 07:19 |
| GheRivero | https://bugs.launchpad.net/tripleo/+bug/1295732/comments/35 | 07:19 |
| uvirtbot | Launchpad bug 1295732 in tripleo "tripleo devtest overcloud startup takes too long " [High,In progress] | 07:19 |
| *** rdopieralski has quit IRC | 07:21 | |
| *** ifarkas has joined #tripleo | 07:22 | |
| *** rdopieralski has joined #tripleo | 07:22 | |
| *** jtomasek has joined #tripleo | 07:23 | |
| *** e0ne_ has joined #tripleo | 07:25 | |
| *** ramishra has quit IRC | 07:26 | |
| *** rpodolyaka1 has joined #tripleo | 07:28 | |
| *** e0ne has quit IRC | 07:29 | |
| *** hashar has joined #tripleo | 07:29 | |
| *** marun is now known as marun_afk | 07:29 | |
| *** d0ugal has quit IRC | 07:34 | |
| *** martyntaylor has quit IRC | 07:43 | |
| jprovazn | SpamapS or lifeless: what is the issue with rebooting a heat-engine node (if multiple heat-engines are used)? I tried search more about this, but no luck | 07:46 |
| lifeless | jprovazn: dirty little secret | 07:48 |
| lifeless | jprovazn: if there is a stack-update | 07:48 |
| lifeless | jprovazn: and the engine is interrupted | 07:48 |
| lifeless | jprovazn: game over | 07:48 |
| *** ramishra has joined #tripleo | 07:48 | |
| lifeless | jprovazn: so if we can guarantee a particular engine isn't in use, rebooting the node is fine. I don't know if there is a heat disable-engine API call | 07:49 |
| jprovazn | lifeless, and stack-create would alive engine's interruption or is it the same case? | 07:50 |
| lifeless | a create is an update from null | 07:51 |
| lifeless | so same case | 07:51 |
| *** martyntaylor has joined #tripleo | 07:52 | |
| *** rpodolyaka1 has quit IRC | 07:52 | |
| jprovazn | lifeless, SpamapS said this is "massive architectural problem" - I wonder if there is a chance this could be solved in "reasonably short time" | 07:52 |
| jprovazn | by solved I mean solved on heat side | 07:53 |
| lifeless | jprovazn: I think we can get something good enough for deploys easily (a disable API to let use quiesce nodes), but we have much larger work to make heat truely distributed and safe for this | 07:54 |
| jprovazn | lifeless, hm, and using pacemaker in a/p mode wouldn't help because if stack-update is interrupted then same situation occurs no matter if engine is respawned somewhere else | 07:59 |
| lifeless | right | 07:59 |
| lifeless | its not a multi-node problem | 07:59 |
| lifeless | its a heat problem | 07:59 |
| jprovazn | lifeless, yes, thanks for clarification | 08:00 |
| *** jistr has joined #tripleo | 08:01 | |
| *** akuznetsov has quit IRC | 08:01 | |
| *** derekh has joined #tripleo | 08:05 | |
| derekh | lifeless: morning, I see we're processing the backlog :-) | 08:05 |
| derekh | I'm not in office yet but just observing things from the train, 6 of the last 12 failures have happened on the same testenv for the same reason | 08:07 |
| *** akuznetsov has joined #tripleo | 08:07 | |
| *** yassine has joined #tripleo | 08:08 | |
| derekh | http://logs.openstack.org/94/83194/5/check-tripleo/check-tripleo-overcloud-precise/2cea869/console.html | 08:08 |
| derekh | 2014-04-03 04:53:18.812 | Permission denied (publickey). | 08:08 |
| *** lucasagomes has joined #tripleo | 08:09 | |
| derekh | Is it possible we have a rogue instance out there reporting to be 192.168.1.65? | 08:10 |
| derekh | And this one is weird, http://logs.openstack.org/05/84705/1/check-tripleo/check-tripleo-overcloud-precise/0134423/console.html | 08:11 |
| derekh | the ssh key worked to destory the VM before boot seed vm "2014-04-03 05:27:54.297 | Domain seed_4 destroyed" | 08:12 |
| derekh | but the same key failed with "Permission denied (publickey)" further down | 08:12 |
| *** d0ugal has joined #tripleo | 08:13 | |
| *** d0ugal has quit IRC | 08:13 | |
| *** d0ugal has joined #tripleo | 08:13 | |
| derekh | lifeless: so I'm thinking we either "nova rebuild" that instance, or leave it causing failures and I can figure it out later when I'm in, thoughts? | 08:13 |
| *** bauzas has joined #tripleo | 08:17 | |
| derekh | lifeless: actually new plan, I'm going to kill the workers on that testenv so its stops accepting jobs and can leave it running | 08:18 |
| *** rpodolyaka1 has joined #tripleo | 08:18 | |
| *** rpodolyaka1 has quit IRC | 08:19 | |
| derekh | done | 08:21 |
| *** lxsli has quit IRC | 08:22 | |
| *** martyntaylor has quit IRC | 08:29 | |
| *** hashar has quit IRC | 08:31 | |
| *** derekh has quit IRC | 08:33 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed a change to openstack/tripleo-incubator: Add trap to clean up openstack-tools folder on failure https://review.openstack.org/82763 | 08:35 |
| openstackgerrit | tom-howley proposed a change to openstack/tripleo-heat-templates: Add rabbitmq keepalive and keepalive parameters to overcloud/undercloud template https://review.openstack.org/81934 | 08:36 |
| *** gcha has joined #tripleo | 08:38 | |
| openstackgerrit | tom-howley proposed a change to openstack/tripleo-heat-templates: Add rabbitmq keepalive and keepalive parameters to overcloud/undercloud template https://review.openstack.org/81934 | 08:38 |
| *** athomas has joined #tripleo | 08:40 | |
| rcarrillocruz | Ng , lifeless, GheRivero: morning morning, what do you think about my inline comment on https://review.openstack.org/#/c/84801/ ? you think we should maintain some sort of supported platforms matrix for devtest ? | 08:41 |
| *** lynxman has quit IRC | 08:41 | |
| *** lynxman has joined #tripleo | 08:42 | |
| *** martyntaylor has joined #tripleo | 08:44 | |
| GheRivero | I agree with the idea as long as it's completed. I'll wait for the REdHat rack to came up with their CI and Fedora testing to complete the list | 08:44 |
| Goneri | g | 08:44 |
| *** pblaho has joined #tripleo | 08:45 | |
| lifeless | rcarrillocruz: isn't that what tchaypo's patch does ? | 08:46 |
| *** jp_at_hp has joined #tripleo | 08:46 | |
| rcarrillocruz | i think i'd rather rename the bug to put something like 'add supported platforms matrix to documentation'. As is now ( filed it myself) it's titled as 'Update documentation to state devtest does not support Ubuntu 12.04' | 08:47 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Overcloud initialization https://review.openstack.org/83340 | 08:48 |
| rcarrillocruz | lifeless: the patchset he linked is in abandoned state. I better get in touch with him to agree on which one to work on, no point in having two different changes implementing pretty much the same thing | 08:49 |
| lifeless | tchaypo: ^ :) | 08:50 |
| tchaypo | yeah, so | 08:51 |
| tchaypo | if it's the change I'm thinking of, I commented on it last night | 08:52 |
| tchaypo | that change and mine both do roughly the same thing but in different places; I can't see any reason to prefer one over the other | 08:52 |
| *** hashar has joined #tripleo | 08:52 | |
| rcarrillocruz | oh | 08:52 |
| rcarrillocruz | you're here | 08:52 |
| rcarrillocruz | :D | 08:52 |
| tchaypo | as long as we don't give the impression that ubuntu is the only platform that works | 08:53 |
| rcarrillocruz | hmm | 08:53 |
| rcarrillocruz | ok | 08:53 |
| tchaypo | part of the reason I abandoned mine is that I was aiming for a list of "platforms which have CI testing", and we still can't quite say that Fedora falls on that list | 08:53 |
| tchaypo | rcarrillocruz' change is simply a list of "platforms the dev team regularly uses/tests", which is a lower bar - we can put fedora in that today | 08:54 |
| lifeless | tchaypo: you need to click on 'restore change' on the review | 08:54 |
| lifeless | tchaypo: or its locked for everyone | 08:54 |
| *** e0ne_ has quit IRC | 08:54 | |
| tchaypo | hah. | 08:54 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Loading template parameters from tuskar-api https://review.openstack.org/84427 | 08:54 |
| *** e0ne has joined #tripleo | 08:55 | |
| tchaypo | I shall go ahead and do that, but I don't intend to pursue mine until we get CI happening on fedora.. and if rcarrillocruz already has a roughly equivalent patch in I won't duplicate it | 08:55 |
| tchaypo | done | 08:56 |
| rcarrillocruz | i wonder if you could keep yours for the devtest script documentation itself and i keep mine for just the README | 08:57 |
| rcarrillocruz | alternatively, we get rid of either changeset and we address the documentation in both places? | 08:57 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Hide non working scaling https://review.openstack.org/84413 | 08:58 |
| tchaypo | It might even make sense to do yours now and mine later | 08:58 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/diskimage-builder: doc: indentation rules for element https://review.openstack.org/84960 | 08:58 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/diskimage-builder: clean up: fix some indent not multiple of 2 https://review.openstack.org/84693 | 08:58 |
| tchaypo | in some ways the "here's what we do CI on" is always going to be a subset of "here's stuff the devs are fairly sure works" | 08:58 |
| tchaypo | and neither list is going to include precise, and that should mean fewer people wasting time trying to use it :) | 08:58 |
| rcarrillocruz | k, will do that. however, i'm not aware of the platforms that work or not, since i only use ubuntu. i could send an email to the mailing list to ask people to put their experiences: this works, this does not | 08:59 |
| tchaypo | one thing that neither of our changes address is min hardware | 08:59 |
| rcarrillocruz | otherwise i'll get -1 from people saying 'hey, my distro-version FOO does work and you did not put it' | 08:59 |
| tchaypo | at least, a note that you X vms at YGb each means you need a minimum of ZGb ram; and unless you have real baremetal to use as nodes you really need phyiscal hardware, not a VM | 09:00 |
| tchaypo | I think dan prince and maybe others have listed specific versions on comments on mine | 09:01 |
| tchaypo | If not, I think that just saying something like "Recentish Fedora" is a good start, and follow that with a note on the review asking people to list specific versions they'd like to add | 09:02 |
| *** untriaged-bot has joined #tripleo | 09:02 | |
| untriaged-bot | Untriaged bugs so far: | 09:02 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1290488 | 09:02 |
| uvirtbot | Launchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete] | 09:02 |
| *** untriaged-bot has quit IRC | 09:02 | |
| *** markmc has joined #tripleo | 09:04 | |
| *** matsuhashi has quit IRC | 09:07 | |
| *** athomas has quit IRC | 09:08 | |
| *** e0ne_ has joined #tripleo | 09:08 | |
| *** jcoufal has quit IRC | 09:09 | |
| *** jcoufal has joined #tripleo | 09:10 | |
| *** xuhaiwei has quit IRC | 09:11 | |
| *** e0ne has quit IRC | 09:12 | |
| *** matsuhashi has joined #tripleo | 09:12 | |
| *** athomas has joined #tripleo | 09:13 | |
| openstackgerrit | Jon-Paul Sullivan proposed a change to openstack/diskimage-builder: Standardise manifest creation and retrieval https://review.openstack.org/83848 | 09:13 |
| *** ramishra has quit IRC | 09:13 | |
| *** morganfainberg is now known as morganfainberg_Z | 09:15 | |
| openstackgerrit | Loganathan Parthipan proposed a change to openstack/tripleo-incubator: Make libvirt volume pool configurable https://review.openstack.org/83409 | 09:16 |
| *** ilives has quit IRC | 09:18 | |
| *** ilives has joined #tripleo | 09:18 | |
| *** gcha has quit IRC | 09:28 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Change templateUrl to absolute path https://review.openstack.org/84926 | 09:32 |
| *** derekh has joined #tripleo | 09:33 | |
| lifeless | derekh: hi :) | 09:35 |
| derekh | lifeless: hi ya | 09:35 |
| *** ramishra has joined #tripleo | 09:37 | |
| andrearosa | SpamapS: questionf for you about default_ephemeral_format, are u available? | 09:43 |
| derekh | lifeless: so I killed the workers on that test env host, will try and figure out the problem now | 09:44 |
| openstackgerrit | Om Kumar proposed a change to openstack/diskimage-builder: Disable splashimage for legacy grub. https://review.openstack.org/84973 | 09:46 |
| *** akrivoka has joined #tripleo | 09:47 | |
| *** yassine has quit IRC | 09:48 | |
| derekh | wtf, my instance on the ci-overcloud got the tripleo-bm-test ip on eth0 | 09:48 |
| lifeless | derekh: yeah there is something messed up | 09:50 |
| lifeless | derekh: I think arosens patch is broken | 09:50 |
| lifeless | derekh: the one I've manually applied | 09:50 |
| derekh | ok, so is nodepool working with a workaround? | 09:51 |
| lifeless | nodepool is working | 09:53 |
| lifeless | the patch I pulled in is in nova trunk | 09:53 |
| derekh | lifeless: I mean the instances nodepool creates do those no have switched nics aswell? | 09:56 |
| lifeless | derekh: https://review.openstack.org/#/q/Ia9e71364bca6cbc24ebc1c234e6a5af14f51cd62,n,z | 09:57 |
| lifeless | derekh: random | 09:57 |
| lifeless | derekh: but mainly ok | 09:57 |
| lifeless | derekh: sec | 09:57 |
| derekh | lifeless: ahh ok | 09:57 |
| lifeless | derekh: we were seeing this badly | 09:57 |
| lifeless | derekh: so I landed that patch and it got a bit better | 09:58 |
| lifeless | derekh: but I'm thinking its still broken / worse somehow | 09:58 |
| howleyt | What do people use for validating their yaml templates? I want to locate source of this error: ERROR: Incorrect arguments to "Fn::Join" should be: "Fn::Join" : [ " ", [ "str1", "str2"]] | 09:58 |
| howleyt | Trying kwalify at the minute. | 09:59 |
| derekh | lifeless: I see, ok, suppose nodepool just keeps trying until it finds a good instance | 09:59 |
| *** matsuhashi has quit IRC | 09:59 | |
| *** matsuhashi has joined #tripleo | 09:59 | |
| *** d0ugal has quit IRC | 10:00 | |
| openstackgerrit | jan grant proposed a change to openstack/tripleo-image-elements: Ensure the (block) loop device is available. https://review.openstack.org/83383 | 10:05 |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Read default_ephemeral_format from nova.conf https://review.openstack.org/84979 | 10:09 |
| killer_prince | lifeless: hi, you gave me a comment for grub-pc package installation to be guarded by ARCH check. can you please elaborate a bit.. | 10:09 |
| killer_prince | does it mean that there is no grub pc fo ARM..? | 10:09 |
| killer_prince | sorry.. i am new to ARM world... | 10:10 |
| lifeless | killer_prince: there are lots of different grubs | 10:11 |
| killer_prince | aha.. so would not that be controlled by the map-packages as in which grub to install.. | 10:11 |
| lifeless | right | 10:12 |
| lifeless | e.g. | 10:12 |
| lifeless | grub-ieee1275 | 10:12 |
| lifeless | or coreboot | 10:12 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Change templateUrl to absolute path https://review.openstack.org/84926 | 10:12 |
| *** martyntaylor has quit IRC | 10:13 | |
| killer_prince | okay... so then if its going to be controlled by map-packages, do we still nedd to guard it using arch check.. | 10:13 |
| derekh | yup, two instances out there somewhere with the same IP | 10:13 |
| derekh | ARPING 192.168.1.65 | 10:13 |
| derekh | 60 bytes from 78:e7:d1:23:93:15 (192.168.1.65): index=0 time=858.068 usec | 10:13 |
| derekh | 60 bytes from 78:e7:d1:23:95:25 (192.168.1.65): index=1 time=955.105 usec | 10:13 |
| lifeless | derekh: nuuuts | 10:14 |
| *** martyntaylor has joined #tripleo | 10:15 | |
| derekh | lifeless: found it | 10:15 |
| lifeless | power em both offf:P | 10:15 |
| derekh | lifeless: it was ahem testenv-derekh-testenv0-q3eu72x2ydjh | 10:15 |
| derekh | lifeless: which was allocated IP's from the old ci-undercloud | 10:15 |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Read default_ephemeral_format from nova.conf https://review.openstack.org/84979 | 10:16 |
| lifeless | derekh: ahahahhaahhaahahahahahahahhaahahahahahhahahahahahahahahah | 10:17 |
| *** ramishra has quit IRC | 10:17 | |
| *** ramishra has joined #tripleo | 10:17 | |
| *** d0ugal has joined #tripleo | 10:17 | |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tripleo-image-elements: Add support for controlling concurrency https://review.openstack.org/84982 | 10:18 |
| killer_prince | lifeless: another doubt is that how about vm element, i do not see any arch check guarding grub package installation... but the image can be used for baremetal too... | 10:18 |
| derekh | workers on testenv2 restarted all should be good again | 10:20 |
| lifeless | killer_prince: not with ironic or nova-bm today | 10:24 |
| lifeless | killer_prince: its a defect, perhaps we should ignore it for your patch too for now | 10:24 |
| killer_prince | lifeless: umm.. okay.. | 10:27 |
| killer_prince | lifeless: also, were you able to review https://review.openstack.org/#/c/79873/. This is a dependency for localboot element.. | 10:28 |
| *** e0ne_ has quit IRC | 10:30 | |
| *** hashar has quit IRC | 10:33 | |
| openstackgerrit | Om Kumar proposed a change to openstack/tripleo-image-elements: Adds local storage Boot support in case PXE boot fails. https://review.openstack.org/79289 | 10:37 |
| *** akrivoka has quit IRC | 10:38 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Catching 500 from heat during stack-create https://review.openstack.org/84988 | 10:38 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Catching 500 from heat during stack-create https://review.openstack.org/84988 | 10:39 |
| *** ramishra has quit IRC | 10:41 | |
| *** ramishra has joined #tripleo | 10:42 | |
| openstackgerrit | Om Kumar proposed a change to openstack/diskimage-builder: Refactor code to select boot kernel https://review.openstack.org/79873 | 10:44 |
| *** matsuhashi has quit IRC | 10:48 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Change templateUrl to absolute path https://review.openstack.org/84926 | 10:49 |
| *** gcha has joined #tripleo | 10:50 | |
| *** matsuhashi has joined #tripleo | 10:55 | |
| *** e0ne has joined #tripleo | 11:09 | |
| tchaypo | lifeless: I was thinking about how to distribute the patched wheel; but I've realised that it's probably simpler to distribute the patch. If we're going to be running an action inside the virtualenv after creating it anyway, that action may as well be "apply this patch to this file". The location of that file is very predictable inside the virtualenv, so it's probably simpler to patch there than to patch the | 11:10 |
| tchaypo | system copy anywa | 11:10 |
| tchaypo | y | 11:10 |
| tchaypo | unfortunately between a 5am start and 90 minutes of lugging boxes I don't have enough brain left | 11:11 |
| lifeless | tchaypo: you don't need to patch the wheel | 11:11 |
| *** e0ne_ has joined #tripleo | 11:11 | |
| lifeless | tchaypo: just the sdist and install from a path - or am I missing somethig ? | 11:11 |
| tchaypo | I don't think there's even a need to install. | 11:12 |
| tchaypo | when the virtualenv is created, virtualenv will install pip | 11:12 |
| tchaypo | and the file we're interested in will always be /lib/pythonX.Y/site-packages/pip/download.py | 11:13 |
| tchaypo | can just patch in place | 11:13 |
| lifeless | if you do that | 11:13 |
| *** e0ne__ has joined #tripleo | 11:13 | |
| lifeless | the manifest of versions will be wrong | 11:13 |
| *** e0ne_ has quit IRC | 11:13 | |
| *** e0ne has quit IRC | 11:13 | |
| *** matsuhashi has quit IRC | 11:15 | |
| *** akrivoka has joined #tripleo | 11:15 | |
| tchaypo | true. | 11:15 |
| *** hashar has joined #tripleo | 11:17 | |
| *** e0ne__ has quit IRC | 11:18 | |
| tchaypo | okay; so to keep the manifest more accurate, the we would need to grab something (the wheel from virtualenv_support?); unpack, patch download.py, patch setup.py/MANIFEST.in to reflect our hacked-up version number, re-pack, and drop that in the mirror | 11:18 |
| tchaypo | then pip should be able to install it even if MarkupSafe is broken because it won't have a case issue | 11:19 |
| lifeless | 19:26 < lifeless> pip install --no-use-wheel -d /tmp/foobarbaz -U pip | 11:19 |
| lifeless | 19:26 < lifeless> I think that will get you a source tree correctly | 11:19 |
| lifeless | 19:26 < lifeless> patch that source tree | 11:19 |
| lifeless | 19:26 < lifeless> then | 11:19 |
| lifeless | 19:26 < lifeless> run pip install /tmp/foobarbaz | 11:19 |
| lifeless | 19:26 < tchaypo> found it | 11:19 |
| lifeless | and yes, have the patch fudge the version too | 11:19 |
| tchaypo | ah goody | 11:20 |
| *** yassine has joined #tripleo | 11:20 | |
| tchaypo | as well as /dev being emtpy, so as ~/.cache/image-create/pypi/mirror | 11:21 |
| lifeless | hah yes | 11:21 |
| tchaypo | I'm hoping this doesn't turn into me having to rebuild my machine | 11:21 |
| *** matsuhashi has joined #tripleo | 11:21 | |
| tchaypo | but I don't know how you're awake, I'm really too tired. | 11:21 |
| tchaypo | thanks for the hints, I'll chase them in the morning | 11:22 |
| lifeless | night, I'm off too. | 11:22 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Bridge physical interface to the seed. https://review.openstack.org/84083 | 11:26 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: SSH key and virtual_power_driver not used on H/W https://review.openstack.org/83770 | 11:27 |
| lifeless | derekh: btw after the reboot - needed to do 'ethtool -K eth2 gro off' | 11:28 |
| lifeless | down to 9 hours backlog | 11:29 |
| *** d0ugal has quit IRC | 11:30 | |
| *** jcoufal has quit IRC | 11:31 | |
| lifeless | derekh: actually, when did you see the last wrong-order nic? was it an old log or a current VM? WIll look for answer later. | 11:32 |
| lifeless | GheRivero: / Ng: https://review.openstack.org/#/c/84073/ has passed CI, just needs another +2 | 11:35 |
| GheRivero | lifeless: has it pass the CI? | 11:37 |
| lifeless | GheRivero: yes - the ironic undercloud failure is systemic (see my mail on monday to -infra and -dev about it), and the f20 stuff isn't working at all yet | 11:37 |
| lifeless | derekh: ^ reminds me - can you please make the f20 job explicitly nonvoting until its fully live ? | 11:38 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Performance tweaks for testenv deploy script. https://review.openstack.org/84073 | 11:38 |
| GheRivero | ok then | 11:38 |
| lifeless | we should do the same to the ironic undercloud job | 11:38 |
| * lifeless really collapses | 11:41 | |
| *** e0ne has joined #tripleo | 11:42 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Catching 500 from heat during stack-create https://review.openstack.org/84988 | 11:43 |
| openstackgerrit | Michael Kerrin proposed a change to openstack/tripleo-incubator: Indent code so that is readable https://review.openstack.org/85000 | 11:43 |
| *** matsuhashi has quit IRC | 11:52 | |
| *** weshay has joined #tripleo | 11:56 | |
| *** mrunge has quit IRC | 11:58 | |
| *** yamahata has quit IRC | 12:10 | |
| *** StevenK_ has joined #tripleo | 12:10 | |
| *** StevenK has quit IRC | 12:10 | |
| openstackgerrit | Jon-Paul Sullivan proposed a change to openstack/tripleo-incubator: Enable selective dib configuration per image https://review.openstack.org/85009 | 12:18 |
| openstackgerrit | Michael Kerrin proposed a change to openstack/tripleo-incubator: Indent code so that is readable https://review.openstack.org/85000 | 12:19 |
| *** noslzzp has joined #tripleo | 12:23 | |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Fix permissions on files in /etc/mysql https://review.openstack.org/85010 | 12:24 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Don't show image parameters to user https://review.openstack.org/85012 | 12:28 |
| shadower | what's up with check-tripleo-overcloud-f20 (status NOT_REGISTERED) jenkins job? | 12:29 |
| shadower | should we treate that as voting or ignore it (we=reviewers) | 12:30 |
| * shadower doesn't remember an email about that one | 12:30 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-heat-templates: Add logstash HOT https://review.openstack.org/83422 | 12:30 |
| *** morazi has joined #tripleo | 12:31 | |
| *** sballe has joined #tripleo | 12:31 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-image-elements: Allow configuration of extended logging https://review.openstack.org/83353 | 12:31 |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-image-elements: Add logstash-{agent,solo} elements https://review.openstack.org/83354 | 12:32 |
| *** geerdest has joined #tripleo | 12:33 | |
| openstackgerrit | Dirk Mueller proposed a change to openstack/tripleo-image-elements: Excplicitly install dhcp-agent https://review.openstack.org/74991 | 12:34 |
| *** jdob has joined #tripleo | 12:37 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-image-elements: Add logstash-{agent,solo} elements https://review.openstack.org/83354 | 12:38 |
| openstackgerrit | Alexis Lee proposed a change to openstack/tripleo-image-elements: Allow configuration of extended logging https://review.openstack.org/83353 | 12:39 |
| *** killer_prince has quit IRC | 12:45 | |
| *** ramishra has quit IRC | 12:50 | |
| *** nosnos has quit IRC | 12:50 | |
| *** matty_dubs|gone is now known as matty_dubs | 12:51 | |
| *** ramishra has joined #tripleo | 12:53 | |
| *** dprince has joined #tripleo | 12:55 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/tripleo-incubator: Make setup-clienttools optional https://review.openstack.org/85024 | 13:00 |
| *** StevenK_ is now known as StevenK | 13:02 | |
| derekh | lifeless: the instance with the mixed up nics is gone now, it was one I started manually to test something | 13:03 |
| derekh | lifeless: will take a look at the f20 job | 13:03 |
| *** julim has joined #tripleo | 13:15 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/tripleo-incubator: Make setup-clienttools optional https://review.openstack.org/85024 | 13:21 |
| *** killer_prince has joined #tripleo | 13:23 | |
| *** d0ugal has joined #tripleo | 13:23 | |
| *** d0ugal has quit IRC | 13:23 | |
| *** d0ugal has joined #tripleo | 13:23 | |
| *** d0ugal has quit IRC | 13:24 | |
| *** d0ugal has joined #tripleo | 13:25 | |
| *** d0ugal has quit IRC | 13:25 | |
| *** d0ugal has joined #tripleo | 13:25 | |
| *** lblanchard has joined #tripleo | 13:25 | |
| *** weshay has quit IRC | 13:27 | |
| *** weshay has joined #tripleo | 13:30 | |
| *** ramishra has quit IRC | 13:35 | |
| *** geerdest_ has joined #tripleo | 13:37 | |
| *** geerdest has quit IRC | 13:40 | |
| *** geerdest_ is now known as geerdest | 13:40 | |
| *** ilives has quit IRC | 13:48 | |
| *** Hazelesque_ is now known as Hazelesque | 13:52 | |
| *** ilives has joined #tripleo | 13:52 | |
| *** sdake__ is now known as sdake | 13:56 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Read default_ephemeral_format from nova.conf https://review.openstack.org/84979 | 14:18 |
| *** Rakesh5 has quit IRC | 14:20 | |
| *** pblaho has quit IRC | 14:21 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Allow set notification_topics, notify_api_defaults https://review.openstack.org/84672 | 14:25 |
| *** mrunge has joined #tripleo | 14:29 | |
| openstackgerrit | Dougal Matthews proposed a change to openstack/tripleo-incubator: Register the Tuskar Service in the Undercloud https://review.openstack.org/85050 | 14:34 |
| *** TravT has joined #tripleo | 14:35 | |
| *** CaptTofu has joined #tripleo | 14:36 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Relabel /tftpboot if selinux is used https://review.openstack.org/84949 | 14:36 |
| *** weshay has quit IRC | 14:38 | |
| *** mrunge has quit IRC | 14:40 | |
| openstackgerrit | Dougal Matthews proposed a change to openstack/tripleo-incubator: Register the Tuskar Service in the Undercloud https://review.openstack.org/85050 | 14:43 |
| openstackgerrit | Michael Kerrin proposed a change to openstack/tripleo-image-elements: WIP - enable elements to be replayable https://review.openstack.org/81495 | 14:44 |
| *** rlandy has joined #tripleo | 14:45 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Remove mostly empty directories https://review.openstack.org/81368 | 14:46 |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/tuskar: Use alembic for database migrations https://review.openstack.org/80338 | 14:49 |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Allow setting of rpc_all configuration options https://review.openstack.org/85059 | 14:51 |
| *** bauzas has quit IRC | 14:52 | |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Make MySQL root password configurable https://review.openstack.org/81392 | 14:52 |
| *** viktors has joined #tripleo | 14:53 | |
| *** rdopieralski has quit IRC | 14:55 | |
| *** morazi has quit IRC | 14:57 | |
| *** rlandy has quit IRC | 15:00 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1300693 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1290488 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1301841 | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| uvirtbot | Launchpad bug 1300693 in tripleo "devtest.sh run leaves 3 default security groups" [Undecided,Incomplete] | 15:00 |
| uvirtbot | Launchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete] | 15:00 |
| uvirtbot | Launchpad bug 1301841 in tripleo "default_ephemeral_format not set for seed and undercloud" [Undecided,In progress] | 15:00 |
| *** rlandy has joined #tripleo | 15:00 | |
| *** morazi has joined #tripleo | 15:02 | |
| *** martyntaylor has quit IRC | 15:02 | |
| *** hashar has quit IRC | 15:05 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/tripleo-incubator: Move setup-clienttools to devtest_setup.sh https://review.openstack.org/85024 | 15:05 |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Allow setting of logging configuration options https://review.openstack.org/85066 | 15:08 |
| *** weshay has joined #tripleo | 15:08 | |
| *** dguerri_ is now known as _dguerri | 15:11 | |
| *** _dguerri is now known as dguerri_ | 15:12 | |
| *** martyntaylor has joined #tripleo | 15:13 | |
| openstackgerrit | Jon-Paul Sullivan proposed a change to openstack/tripleo-incubator: Enable selective dib configuration per image https://review.openstack.org/85009 | 15:19 |
| *** d0ugal_ has joined #tripleo | 15:20 | |
| *** d0ugal has quit IRC | 15:20 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Mising import url https://review.openstack.org/85072 | 15:21 |
| *** sdake_ has joined #tripleo | 15:22 | |
| *** sdake_ has quit IRC | 15:22 | |
| *** sdake_ has joined #tripleo | 15:22 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Create and use libvirtd group for package install https://review.openstack.org/82529 | 15:26 |
| *** Vinsh has joined #tripleo | 15:27 | |
| *** yamahata has joined #tripleo | 15:28 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Mising import url https://review.openstack.org/85072 | 15:30 |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar-ui: Missing import url https://review.openstack.org/85072 | 15:35 |
| *** CaptTofu has quit IRC | 15:40 | |
| *** lxsli has joined #tripleo | 15:41 | |
| *** lxsli has quit IRC | 15:41 | |
| *** andreaf has quit IRC | 15:41 | |
| *** lxsli has joined #tripleo | 15:42 | |
| *** newell_ has joined #tripleo | 15:45 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-heat-templates: Set the glance_port as Heat configurable https://review.openstack.org/85081 | 15:47 |
| *** eghobo has joined #tripleo | 15:47 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Read glance_port from nova.conf https://review.openstack.org/85082 | 15:48 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-heat-templates: Expose dnsmasq options https://review.openstack.org/82803 | 15:50 |
| *** d0ugal_ has quit IRC | 15:50 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Expose dnsmasq options https://review.openstack.org/82804 | 15:50 |
| *** d0ugal_ has joined #tripleo | 15:51 | |
| *** dguerri_ is now known as _dguerri | 15:53 | |
| *** d0ugal_ has quit IRC | 15:55 | |
| *** d0ugal_ has joined #tripleo | 15:55 | |
| *** d0ugal_ is now known as d0ugal | 15:56 | |
| *** _dguerri is now known as dguerri_ | 15:56 | |
| *** matty_dubs is now known as matty_dubs|lunch | 15:58 | |
| *** petertoft has joined #tripleo | 15:58 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Allow setting of glance_protocol https://review.openstack.org/85087 | 15:59 |
| *** marun_afk is now known as marun | 16:01 | |
| *** andreaf has joined #tripleo | 16:03 | |
| *** cwolferh has joined #tripleo | 16:07 | |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-heat-templates: Add enable_tunneling to swift storage metadata https://review.openstack.org/83417 | 16:09 |
| *** d0ugal has quit IRC | 16:11 | |
| *** jistr has quit IRC | 16:13 | |
| *** morazi has quit IRC | 16:16 | |
| *** csd_ has joined #tripleo | 16:20 | |
| *** AaronGr has joined #tripleo | 16:21 | |
| *** csd_ is now known as csd | 16:23 | |
| *** morazi has joined #tripleo | 16:25 | |
| *** yassine has quit IRC | 16:26 | |
| *** jprovazn has quit IRC | 16:27 | |
| *** csd_ has joined #tripleo | 16:29 | |
| *** csd has quit IRC | 16:30 | |
| *** csd_ has quit IRC | 16:30 | |
| *** csd has joined #tripleo | 16:30 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/tripleo-image-elements: Add debug to ironic conf file. DO NOT MERGE https://review.openstack.org/85101 | 16:33 |
| *** stevehuang has quit IRC | 16:36 | |
| *** ifarkas has quit IRC | 16:36 | |
| *** killer_prince has quit IRC | 16:42 | |
| *** e0ne has quit IRC | 16:49 | |
| *** e0ne has joined #tripleo | 16:50 | |
| *** mclaren has joined #tripleo | 16:51 | |
| *** viktors has quit IRC | 16:53 | |
| bnemec | Hey all, https://review.openstack.org/#/c/84876/ is a pretty trivial change for something that is breaking devtest on Fedora right now. | 16:53 |
| *** e0ne has quit IRC | 16:54 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Install bridge-utils on compute nodes https://review.openstack.org/84876 | 16:55 |
| *** lucasagomes has quit IRC | 16:56 | |
| SpamapS | lifeless: whatever you were trying to say to me last night about review softness or register nodes or why the .py.. ????? read backscroll, still don't know what you were saying. | 16:58 |
| *** andreaf has quit IRC | 16:59 | |
| Lotus907efi | SpamapS: does lifeless occasionally mumble-type in his sleep and the result ends up on here? :) | 17:08 |
| *** derekh has quit IRC | 17:08 | |
| SpamapS | Lotus907efi: it's entirely possible | 17:11 |
| SpamapS | but seeing as it was about 18:00 in his TZ, I'm guessing he was just brain dumping | 17:11 |
| *** gcha has quit IRC | 17:14 | |
| *** ilives has quit IRC | 17:17 | |
| *** ilives has joined #tripleo | 17:18 | |
| *** killer_prince has joined #tripleo | 17:21 | |
| *** athomas has quit IRC | 17:21 | |
| rcarrillocruz | guys, you getting check-tripleo-ironic-undercloud-precise job failures? two different reviews failing in that job, and my changesets are not related to ironic... | 17:22 |
| Lotus907efi | yeah, I am seeing that too for one of my reviews | 17:24 |
| bnemec | rcarrillocruz: Lotus907efi: I believe that's a known issue. | 17:24 |
| bnemec | Let me see if I can find the ML post about it. | 17:24 |
| rcarrillocruz | the console output shows: | 17:26 |
| bnemec | Here: http://lists.openstack.org/pipermail/openstack-dev/2014-March/031242.html | 17:26 |
| rcarrillocruz | 2014-04-03 15:41:02.025 | Command output matched 'CREATE_FAILED'. Exiting... | 17:26 |
| rcarrillocruz | top man | 17:26 |
| rcarrillocruz | thx bnemec | 17:27 |
| rcarrillocruz | i'll look into it | 17:27 |
| *** jang1 has joined #tripleo | 17:28 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Rename node profile to flavor https://review.openstack.org/85116 | 17:29 |
| slagle | jomara: any updated builds to try yet? | 17:30 |
| openstackgerrit | Ana Krivokapic proposed a change to openstack/tuskar-ui: Add missing Nodes heading https://review.openstack.org/85117 | 17:31 |
| jomara | slagle: no, working through new requirements | 17:31 |
| jomara | gotta do some systemd crap im not familiar with | 17:31 |
| * jomara sobs | 17:31 | |
| slagle | jomara: do you need a new release of python-tuskarclient? | 17:32 |
| slagle | i'll just go ahead and do it, i believe there was a commit or 2 | 17:32 |
| *** morganfainberg_Z is now known as morganfainberg | 17:34 | |
| Lotus907efi | rcarrillocruz: are you seeing failures in check-tripleo-overcloud-precise and check-tripleo-overcloud-f20 that seem unrelated to any changes you made too? | 17:34 |
| jomara | slagle: oh, thanks, do you need a new build of that? | 17:35 |
| jomara | slagle: im still waiting on fedora scm...its beiung slow today | 17:35 |
| *** matty_dubs|lunch is now known as matty_dubs | 17:35 | |
| jomara | i submitted the request at like 930 | 17:35 |
| *** rainya has quit IRC | 17:43 | |
| slagle | jomara: i don't know that I *need* an update to that | 17:44 |
| slagle | haven't hit any issues yet | 17:44 |
| rcarrillocruz | Lotus907efi: i had two reviews that had +1 from Jenkins. Then the CI system re-ran jobs and then the check-tripleo-ironic-undercloud job failed | 17:44 |
| rcarrillocruz | https://review.openstack.org/#/c/82763/ | 17:44 |
| rcarrillocruz | and | 17:44 |
| rcarrillocruz | https://review.openstack.org/#/c/84801/ | 17:45 |
| jomara | slagle: that, meaning python-tuskarclient? | 17:45 |
| rcarrillocruz | so it seems that some change put in the CI for those jobs have some problems | 17:45 |
| Lotus907efi | hmm, I had a review with a +1 as well but now three jobs are failing | 17:45 |
| rcarrillocruz | same job failing? | 17:46 |
| Lotus907efi | I looked at the logs and cannot find anything rlated to my change | 17:46 |
| Lotus907efi | related | 17:46 |
| rcarrillocruz | in the console log the common factor for my two reviews is: | 17:46 |
| *** rlandy has quit IRC | 17:46 | |
| Lotus907efi | check-tripleo-ironic-undercloud, check-tripleo-overcloud-precise and check-tripleo-overcloud-f20 | 17:46 |
| rcarrillocruz | 2014-04-03 15:41:02.025 | Command output matched 'CREATE_FAILED'. Exiting... | 17:47 |
| slagle | jomara: yes | 17:49 |
| Lotus907efi | they both look like an undercloud creation failure | 17:51 |
| Lotus907efi | the failure I am seeing with check-tripleo-overcloud-precise also has to do with undercloud coming up but is not exactly the same | 17:53 |
| *** akrivoka has quit IRC | 17:53 | |
| Lotus907efi | the error I see is: ERROR: the main setup script run by this job failed - exit code: 1 | 17:54 |
| rcarrillocruz | well, i also get that following to the error I pasted above | 17:56 |
| rcarrillocruz | so this looks like we are hitting the same issue | 17:56 |
| Lotus907efi | and this is just after a "nova list" of the undercloud which is in a state of BUILDING but has a POWER_STATE of NOSTATE | 17:57 |
| Lotus907efi | which I understand is a problem (NOSTATE for power state) | 17:57 |
| Lotus907efi | NOSTATE that is | 17:57 |
| *** derekh has joined #tripleo | 17:58 | |
| *** petertoft has quit IRC | 17:58 | |
| Lotus907efi | above this in my console.log I see: + wait_for_stack_ready 220 10 undercloud Timing out after 2200 seconds: COMMAND=heat stack-show undercloud | awk '/stack_status / { print $4 } | 17:59 |
| Lotus907efi | isn't there something wrong if that wait_for_stack_ready command times out ? | 18:00 |
| bnemec | Lotus907efi: Yes, generally | 18:00 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-incubator: Deploy overcloud images from seed https://review.openstack.org/85126 | 18:01 |
| bnemec | When a PXE deploy gets hung up in the BUILDING state, it likely means that some piece of the deployment failed (PXE booting, baremetal deploy, that sort of thing). | 18:01 |
| bnemec | At least in my experience. | 18:01 |
| *** rwsu has quit IRC | 18:03 | |
| *** jprovazn has joined #tripleo | 18:04 | |
| jprovazn | SpamapS, hi | 18:06 |
| jprovazn | SpamapS, https://review.openstack.org/#/c/81666/ question | 18:06 |
| jprovazn | SpamapS, is this expected error? http://paste.openstack.org/show/75001/ | 18:07 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-incubator: Utility to retrieve images from a given URL https://review.openstack.org/85130 | 18:08 |
| *** akuznetsov has quit IRC | 18:12 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Rename node profile to flavor https://review.openstack.org/85116 | 18:13 |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack/tripleo-image-elements: Configure logging for keystone https://review.openstack.org/84847 | 18:14 |
| *** phschwartz has quit IRC | 18:17 | |
| *** akuznetsov has joined #tripleo | 18:17 | |
| *** phschwartz has joined #tripleo | 18:17 | |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack/tripleo-incubator: Configurable Keystone token provider https://review.openstack.org/84808 | 18:19 |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack/tripleo-heat-templates: Configurable Keystone token provider https://review.openstack.org/84807 | 18:20 |
| *** phschwartz has quit IRC | 18:21 | |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack/tripleo-image-elements: Configurable Keystone token provider https://review.openstack.org/84802 | 18:22 |
| *** phschwartz has joined #tripleo | 18:22 | |
| *** akuznetsov has quit IRC | 18:28 | |
| *** derekh has quit IRC | 18:37 | |
| *** akuznetsov has joined #tripleo | 18:38 | |
| *** derekh has joined #tripleo | 18:39 | |
| *** killer_prince has quit IRC | 18:42 | |
| *** jang1 has quit IRC | 18:47 | |
| *** marun is now known as marun_afk | 18:50 | |
| *** derekh has quit IRC | 18:54 | |
| *** lazy_prince has joined #tripleo | 18:56 | |
| *** lazy_prince is now known as killer_prince | 18:56 | |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Add node filter https://review.openstack.org/85146 | 19:02 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Add node filter https://review.openstack.org/85146 | 19:02 |
| *** akuznetsov has quit IRC | 19:06 | |
| *** stevehuang has joined #tripleo | 19:06 | |
| *** derekh has joined #tripleo | 19:11 | |
| *** csd_ has joined #tripleo | 19:12 | |
| *** csd has quit IRC | 19:15 | |
| *** jprovazn has quit IRC | 19:21 | |
| *** jp_at_hp has quit IRC | 19:26 | |
| *** e0ne has joined #tripleo | 19:31 | |
| lifeless | SpamapS: the only bit I intended to address to you was reviews; I hit 3/4 of yours where you'd +2'd with defects | 19:32 |
| lifeless | SpamapS: look at your review mail leading up to that time, you should see my -1's | 19:33 |
| *** jp_at_hp has joined #tripleo | 19:35 | |
| greghaynes | Vinsh: Hey | 19:36 |
| Vinsh | Hello | 19:36 |
| greghaynes | Vinsh: So looks like jprovazn is the one who wrote the haproxy element | 19:37 |
| greghaynes | not totally punting the problem to him, just a heads up if you see him around | 19:37 |
| greghaynes | hes been doing a lot of the HA work | 19:37 |
| Vinsh | gotcha, will kepe an eye open | 19:37 |
| lifeless | GheRivero: hey - got anything for me to run with ? | 19:38 |
| Vinsh | to restate what we were talking about elsewhere... I have a case where haproxy is setup on a controller node. | 19:38 |
| Vinsh | What I am wondering.. is how does one go about configuring haproxy.. in heat maybe? so that we can define what services it should proxy for. | 19:38 |
| greghaynes | Vinsh: If you check out the haproxy element, specifically the haproxy.cfg it looks like there is a template for haproxy.services | 19:39 |
| greghaynes | AFAICT its not actually used yet, so disclaimer on its functionality | 19:39 |
| Vinsh | ah-hah.. so this section starting "{{#haproxy.services}} | 19:40 |
| Vinsh | listen {{name}} | 19:40 |
| Vinsh | " | 19:40 |
| greghaynes | but you should be able to add that a resource haproxy in the heat template with those properties | 19:40 |
| greghaynes | yes | 19:40 |
| GheRivero | lifeless: https://review.openstack.org/#/c/85130/ and https://review.openstack.org/#/c/85126/ | 19:40 |
| Vinsh | ok. just getting up to speed on heat. But your saying it will look in its templates.. find a section there for "haproxy.services" and then configure based on that? | 19:41 |
| GheRivero | lifeless: a bit of plumming is still missing but that should be minimun. I'll be back in 30m | 19:41 |
| greghaynes | Yes. Those properties are filled out based on things youll see in os-collect-config --print (useful for debugging) which are spefified from the heat template | 19:42 |
| Vinsh | right on | 19:42 |
| Vinsh | for a small config.. like out of the box devtest.. running haproxy on 1-many controller nodes might be fine | 19:42 |
| Vinsh | but.. thinking of scale.. we might need to define 2+ nodes used solely as haproxy nodes in the undercloud | 19:43 |
| Vinsh | I /think/ :) | 19:43 |
| lifeless | Vinsh: in the undercloud? why? | 19:44 |
| lifeless | Vinsh: I mean, why would the undercloud have so much load? | 19:44 |
| Vinsh | Maybe I used the wrong cloud there.. | 19:44 |
| Vinsh | i'm thinking.. that you have nova/glance/keystone running in over cloud.. | 19:45 |
| Vinsh | typically thats all on an undercloud compute node | 19:45 |
| Vinsh | right? | 19:45 |
| Vinsh | then there are the "notcompute" controller nodes in undercloud | 19:45 |
| Vinsh | which is where haproxy is running in my setup now.. to proxy the api services of other overcloud items | 19:46 |
| lifeless | Vinsh: I think you have the cloud layers a bit tangled | 19:47 |
| *** jp_at_hp1 has joined #tripleo | 19:47 | |
| Vinsh | Yeah. | 19:47 |
| lifeless | Vinsh: the overcloud /all/ runs as instances on the undercloud. Which means all the overcloud is baremetal machines. | 19:47 |
| lifeless | the 'undercloud compute node' is just nova-barmetal, it doesn't host any vms | 19:48 |
| lifeless | today at least; there are high level sketches of what a more flexible deployment substrate would look like | 19:48 |
| *** jp_at_hp has quit IRC | 19:48 | |
| openstackgerrit | Jon-Paul Sullivan proposed a change to openstack/tripleo-incubator: Add the select-cloud script https://review.openstack.org/82807 | 19:49 |
| openstackgerrit | Jon-Paul Sullivan proposed a change to openstack/tripleo-incubator: Standardise location of environment password/rc files. https://review.openstack.org/83250 | 19:49 |
| Vinsh | Ok. detangled now. So what I should have said is "in the overcloud we have controller nodes that right now also run haproxy, but looking forward we might need a dedicated overcloud node(s) to run haproxy.. in a larger scale. | 19:51 |
| greghaynes | possible, and wouldnt be hard to do, but id expect lots of other things to fall over from load before hitting that issue. | 19:52 |
| lifeless | Vinsh: certainly, in principle. The rule of thumb for haproxy is about 100K tcp connections per second per instance | 19:52 |
| lifeless | Vinsh: so, at sufficiently high scale, certainly. | 19:52 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv https://review.openstack.org/79989 | 19:54 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Openstack-clients: don't hard code venv https://review.openstack.org/79988 | 19:54 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS... https://review.openstack.org/76966 | 19:54 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element https://review.openstack.org/76967 | 19:54 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Horizon: dynamically set config time env vars https://review.openstack.org/82611 | 19:54 |
| lifeless | Vinsh: (and those figures are from 2009 :)) | 19:54 |
| *** ilives has quit IRC | 19:55 | |
| *** ilives has joined #tripleo | 19:56 | |
| *** rwsu has joined #tripleo | 19:56 | |
| derekh | lifeless: taking a quick glance at the nodepool code, I'm thinking (kind of hoping) it will start creating nodes once the backlog is cleared as each CreateServerTask task is added to a queue | 19:58 |
| derekh | lifeless: the f20 instances that is | 19:58 |
| clarkb | derekh: that was one of my theories previously, jeblair is checking the allocator code to see if there is a bug in that | 20:00 |
| clarkb | derekh: I guess we will see :) | 20:00 |
| *** ohadlevy has quit IRC | 20:01 | |
| *** ohadlevy_ has joined #tripleo | 20:01 | |
| derekh | clarkb: fingers crossed | 20:01 |
| *** TravT|2 has joined #tripleo | 20:02 | |
| *** TravT has quit IRC | 20:02 | |
| *** ilives has quit IRC | 20:03 | |
| *** ilives has joined #tripleo | 20:05 | |
| *** dprince has quit IRC | 20:06 | |
| *** CaptTofu has joined #tripleo | 20:07 | |
| *** rwsu has quit IRC | 20:07 | |
| *** rwsu has joined #tripleo | 20:09 | |
| lifeless | GheRivero: cool, so I think I'd like to pull the list of images out of that, and make it also proxy to dib, so we have one interface. | 20:10 |
| lifeless | GheRivero: and the long planned 'we should tag images' stuff gets a home to live in. | 20:10 |
| *** rwsu has quit IRC | 20:11 | |
| slagle | lifeless: hey, so i have permissions to push signed tags for the releases, but i can't create branches for the stable branches | 20:11 |
| slagle | lifeless: does someone in infra have to do that? | 20:11 |
| *** rwsu has joined #tripleo | 20:11 | |
| lifeless | slagle: no idea, I've never been involved with that | 20:11 |
| slagle | k, will ask around | 20:12 |
| derekh | slagle: I think you can do it, I've seme to remember doing for packstack before, one sec | 20:13 |
| GheRivero | and back | 20:13 |
| lifeless | GheRivero: so e.g build-seed-vm woud decide the elements it wants, and ask a helper that knows it can download-or-build to 'get me the image' | 20:14 |
| slagle | derekh: i tried via gerrit ui and it just says "can not create refs/heads/stable/icehouse" | 20:14 |
| derekh | slagle: in here ? https://review.openstack.org/#/admin/projects/openstack/tripleo-image-elements,branches | 20:14 |
| slagle | then tried creating from cli as well, then pushing the branch to the gerrit remove | 20:14 |
| slagle | remote | 20:14 |
| slagle | and i get rejected | 20:14 |
| slagle | derekh: yea | 20:14 |
| GheRivero | lifeless: do you want to have different images with different elements? | 20:15 |
| lifeless | GheRivero: I'm thinking about several long term threads that have turned up | 20:16 |
| lifeless | GheRivero: one is what e.g. slagle has done with precanned images that can be downloaded | 20:16 |
| derekh | slagle: hmm, maybe something has changed, when I look at the packstack version of the page I see two textboxes to enter new branch details (I don't see it for tripleo projects), but can't try it out because no branch needed | 20:16 |
| lifeless | GheRivero: another is that we need to be able to tell if an image we download meets the elements+arch we wanted | 20:16 |
| lifeless | GheRivero: don't have to solve it all at once, but I think a little tweak to the structure will make it clearer how to solve in future | 20:17 |
| lifeless | GheRivero: also I want to avoid having a centralised list of images to maintain, since the image list depends on the scenario; so the acquire-and-cache layer should be input driven | 20:17 |
| GheRivero | lifeless: for quick testing I was using the Galway jenkis artifacts from the last successful build | 20:17 |
| GheRivero | ok, I got the broader idea now | 20:18 |
| lifeless | morganfainberg: fix it how ? :P | 20:19 |
| *** julim has quit IRC | 20:19 | |
| geerdest | lifeless: re https://review.openstack.org/#/c/84189/ where can i put horizon for undercloud to get a +2? | 20:21 |
| lifeless | geerdest: devtest_undercloud :) | 20:21 |
| lifeless | geerdest: lets see if that was updated overnight | 20:21 |
| geerdest | so my patch set 1 then? | 20:22 |
| lifeless | geerdest: yes, though why are you not adding it to overcloud too ? | 20:23 |
| geerdest | its already in overcloud | 20:23 |
| lifeless | ahh! | 20:25 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Modified deployment creation message https://review.openstack.org/85163 | 20:25 |
| lifeless | geerdest: so yes, your patch set one. | 20:25 |
| *** rwsu has quit IRC | 20:27 | |
| lifeless | GheRivero: what do you think? | 20:28 |
| openstackgerrit | Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Clarified initialization message https://review.openstack.org/85166 | 20:28 |
| openstackgerrit | Jay Dobies proposed a change to openstack/tuskar: Fixes for Tuskar API Keystone authentication https://review.openstack.org/84827 | 20:28 |
| openstackgerrit | Trent Geerdes proposed a change to openstack/tripleo-incubator: Adding horizon element to undercloud https://review.openstack.org/84189 | 20:29 |
| *** rwsu has joined #tripleo | 20:31 | |
| GheRivero | lifeless: I like the idea, it's doable, but don't know it could be worth it to do it know or wait for the "refactoring' | 20:31 |
| lifeless | GheRivero: I'll have a play today | 20:33 |
| GheRivero | lifeless: but wtf, let's give it a try | 20:33 |
| openstackgerrit | Ghe Rivero proposed a change to openstack/tripleo-incubator: Deploy overcloud images from seed https://review.openstack.org/85126 | 20:34 |
| *** derekh has quit IRC | 20:35 | |
| *** rainya- has joined #tripleo | 20:40 | |
| *** csd_ is now known as csd | 20:43 | |
| *** marun_afk is now known as marun | 20:45 | |
| slagle | lifeless: fyi, i proposed this change to infra/config to give us permissions to create branches: https://review.openstack.org/#/c/85171/ | 20:47 |
| lifeless | ack | 20:49 |
| *** dkehn__ has joined #tripleo | 20:51 | |
| *** dkehn has quit IRC | 20:51 | |
| *** dkehn_ has quit IRC | 20:52 | |
| *** dkehn has joined #tripleo | 20:53 | |
| *** dkehn__ is now known as dkehn_ | 20:54 | |
| *** CaptTofu has quit IRC | 20:57 | |
| *** CaptTofu has joined #tripleo | 20:58 | |
| *** TravT|2 has quit IRC | 20:59 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:00 | |
| *** untriaged-bot has joined #tripleo | 21:01 | |
| untriaged-bot | Untriaged bugs so far: | 21:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302148 | 21:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1300693 | 21:01 |
| uvirtbot | Launchpad bug 1302148 in tripleo "error: Unable to add port vnet2 to OVS bridge br-ctlplane: Operation not permitted" [Undecided,New] | 21:01 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1290488 | 21:01 |
| uvirtbot | Launchpad bug 1300693 in tripleo "devtest.sh run leaves 3 default security groups" [Undecided,Incomplete] | 21:01 |
| uvirtbot | Launchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete] | 21:01 |
| *** untriaged-bot has quit IRC | 21:01 | |
| *** lblanchard has quit IRC | 21:01 | |
| *** CaptTofu has quit IRC | 21:02 | |
| *** markmc has quit IRC | 21:04 | |
| lifeless | slagle: cool [ptl candidacy] | 21:09 |
| *** jdob has quit IRC | 21:10 | |
| *** dkehn__ has joined #tripleo | 21:15 | |
| *** dkehn__ has quit IRC | 21:16 | |
| *** dkehn__ has joined #tripleo | 21:16 | |
| *** dkehn has quit IRC | 21:16 | |
| *** dkehn_ has quit IRC | 21:16 | |
| *** dkehn has joined #tripleo | 21:18 | |
| *** weshay has quit IRC | 21:19 | |
| *** rwsu has quit IRC | 21:22 | |
| *** martyntaylor has left #tripleo | 21:22 | |
| *** dkehn_ has joined #tripleo | 21:23 | |
| *** yassine has joined #tripleo | 21:25 | |
| *** dkehn__ has quit IRC | 21:25 | |
| *** dkehn has quit IRC | 21:25 | |
| *** dkehn has joined #tripleo | 21:26 | |
| *** yassine has quit IRC | 21:27 | |
| *** yassine has joined #tripleo | 21:27 | |
| *** yassine has quit IRC | 21:30 | |
| *** dkehn__ has joined #tripleo | 21:32 | |
| *** dkehn has quit IRC | 21:33 | |
| *** dkehn_ has quit IRC | 21:34 | |
| openstackgerrit | john Brooker proposed a change to openstack/tripleo-incubator: Undercloud heat on Bare-metal https://review.openstack.org/85180 | 21:34 |
| *** dkehn has joined #tripleo | 21:36 | |
| *** weshay has joined #tripleo | 21:37 | |
| *** dkehn__ is now known as dkehn-x | 21:37 | |
| openstackgerrit | john Brooker proposed a change to openstack/tripleo-incubator: Allow custom IP ranges to be configured for under and overcloud https://review.openstack.org/85181 | 21:38 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/diskimage-builder: avoid privilege issue https://review.openstack.org/85183 | 21:39 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/diskimage-builder: check element list during the start up https://review.openstack.org/85184 | 21:40 |
| lifeless | devananda: slagle: SpamapS: GheRivero: jogo: Ng: cloud is down - and I have meetings I need to hand off to someone | 21:41 |
| *** CLOUDOUTAGE has joined #tripleo | 21:43 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 21:43 |
| *** CLOUDOUTAGE has quit IRC | 21:43 | |
| lifeless | so now I think we have network fail on the undercloud | 21:49 |
| lifeless | stopping the overcloud control booting | 21:49 |
| lifeless | looping it - would like to avoid rebooting the udnercloud for now | 21:49 |
| openstackgerrit | john Brooker proposed a change to openstack/tripleo-incubator: Detect baremetal undercloud and change heat. https://review.openstack.org/85180 | 21:50 |
| *** jtomasek has quit IRC | 21:52 | |
| *** matty_dubs|gone is now known as matty_dubs | 21:56 | |
| *** noslzzp has quit IRC | 21:56 | |
| *** slagle has quit IRC | 21:57 | |
| *** slagle has joined #tripleo | 21:58 | |
| *** noslzzp has joined #tripleo | 21:58 | |
| lifeless | all undercloud comms down | 22:01 |
| openstackgerrit | john Brooker proposed a change to openstack/tripleo-incubator: Detect baremetal undercloud and change heat. https://review.openstack.org/85180 | 22:02 |
| *** noslzzp has quit IRC | 22:03 | |
| *** marun is now known as marun_afk | 22:03 | |
| *** noslzzp has joined #tripleo | 22:10 | |
| *** noslzzp has quit IRC | 22:14 | |
| *** CLOUDOUTAGE has joined #tripleo | 22:14 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 22:14 |
| *** CLOUDOUTAGE has quit IRC | 22:14 | |
| tchaypo | morning | 22:15 |
| tchaypo | sudo won't let me enter a password because i don't have a tty | 22:15 |
| greghaynes | better go find one | 22:16 |
| tchaypo | easy | 22:17 |
| tchaypo | sudo mknod ... oh wait. | 22:17 |
| tchaypo | I have a Wyse 85 sitting under my desk | 22:17 |
| greghaynes | hahaha | 22:18 |
| tchaypo | oh goody, /dev/null is a standard file | 22:24 |
| SpamapS | This is getting a little bit ridiculuos | 22:25 |
| *** rwsu has joined #tripleo | 22:26 | |
| *** weshay has quit IRC | 22:26 | |
| *** UtahDave has joined #tripleo | 22:27 | |
| *** dkehn has quit IRC | 22:35 | |
| *** dkehn-x has quit IRC | 22:36 | |
| lifeless | power | 22:39 |
| lifeless | bah | 22:39 |
| *** e0ne has quit IRC | 22:43 | |
| *** yamahata has quit IRC | 22:44 | |
| SpamapS | lifeless: just now logging in to boxes to dig around | 22:45 |
| *** CLOUDOUTAGE has joined #tripleo | 22:45 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 22:45 |
| *** CLOUDOUTAGE has quit IRC | 22:45 | |
| lifeless | SpamapS: hi | 22:48 |
| lifeless | SpamapS: I have a textcons open on the undercloud | 22:48 |
| lifeless | SpamapS: its not getting DHCP back from the seed | 22:48 |
| lifeless | SpamapS: to boot | 22:48 |
| SpamapS | lifeless: ok. and the seed? | 22:48 |
| SpamapS | and why is it booting? | 22:48 |
| lifeless | SpamapS: I have started dnsmasq-bm on the seed | 22:48 |
| lifeless | SpamapS: because its still running old kernel and *its* networking caput | 22:48 |
| SpamapS | please don't tell me hardware problems | 22:48 |
| lifeless | SpamapS: cmd_alloc | 22:48 |
| lifeless | SpamapS: old kernels all around | 22:48 |
| SpamapS | I don't know what that means, but I believe you that it needed a bounce. ;) | 22:49 |
| lifeless | I tried to rmmod the mellanox module and it wedged | 22:49 |
| *** rollerj has joined #tripleo | 22:49 | |
| SpamapS | lifeless: yeah, ye olde mellanox suckage.. | 22:49 |
| lifeless | SpamapS: we really need to drop new kernels on both the undercloud and overcloud I think | 22:49 |
| SpamapS | lifeless: we really need to start thinking about how much operating this cloud is costing us. :-P | 22:52 |
| *** marun_afk is now known as marun | 22:52 | |
| lifeless | SpamapS: so seed | 22:53 |
| lifeless | SpamapS: my next step was to be tcpdumping on the seed (.168) and making sure it sees DHCP requests from the undercloud | 22:53 |
| lifeless | SpamapS: I've ctrl-B'd the undercloud on the ipxe prompt to iterate quickly without waiting for 10m power cycles | 22:54 |
| lifeless | which it doesn't seem to be... | 22:54 |
| *** morazi has quit IRC | 22:57 | |
| SpamapS | weird | 22:57 |
| lifeless | 22:55:02.881233 IP 0.0.0.0.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from 52:54:00:f0:92:6d (oui Unknown), length 300 | 22:57 |
| lifeless | DHCP (net1 78:e7:d1:23:9d:7d)................................ Connection timed o | 22:57 |
| SpamapS | lifeless: is the seed failing to respond then? | 22:59 |
| lifeless | greghaynes: stephen halpin may ping you about being able to deploy just mysql-cluster - I'm thinking if you give him a sliced-down version of your hand crafted template it might be more than sufficient | 23:00 |
| *** UtahDave has left #tripleo | 23:01 | |
| *** CaptTofu has joined #tripleo | 23:02 | |
| lifeless | SpamapS: different macs | 23:04 |
| lifeless | SpamapS: it might even be physical | 23:04 |
| *** yamahata has joined #tripleo | 23:04 | |
| SpamapS | oh | 23:05 |
| *** matsuhashi has joined #tripleo | 23:05 | |
| SpamapS | lifeless: ok. I'm at a coffee shop and need to be moving back to home office shortly. Will be available again in about 45 min. | 23:06 |
| *** dkehn has joined #tripleo | 23:06 | |
| *** matty_dubs is now known as matty_dubs|gone | 23:07 | |
| *** dkehn is now known as dkehnx | 23:07 | |
| tchaypo | did an apt-get update, rebooted laptop to fix weird graphics errors i was getting. now it doesn't believe my monitor can go above 1024x768 | 23:09 |
| *** dkehn has joined #tripleo | 23:09 | |
| *** e0ne_ has joined #tripleo | 23:15 | |
| *** CLOUDOUTAGE has joined #tripleo | 23:16 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 23:16 |
| *** CLOUDOUTAGE has quit IRC | 23:16 | |
| greghaynes | lifeless: ok | 23:18 |
| *** e0ne_ has quit IRC | 23:20 | |
| lifeless | seed virsh host sees the requests | 23:21 |
| lifeless | seed vm isn't, not sure why yet | 23:21 |
| lifeless | ahha was dupming wrong interface | 23:21 |
| lifeless | seed sees it | 23:22 |
| openstackgerrit | tom-howley proposed a change to openstack/tripleo-image-elements: Add keepalive option to rabbitmq config https://review.openstack.org/81920 | 23:28 |
| lifeless | seed dnsmasq was out of ips have fixed and updated admin notes | 23:38 |
| *** yamahata has quit IRC | 23:38 | |
| *** eghobo has quit IRC | 23:42 | |
| *** CLOUDOUTAGE has joined #tripleo | 23:48 | |
| CLOUDOUTAGE | lifeless devananda Ng SpamapS jog0 GheRivero derekh dprince slagle -- ci-overcloud currently down https://etherpad.openstack.org/p/cloud-outage | 23:48 |
| *** CLOUDOUTAGE has quit IRC | 23:48 | |
| *** CaptTofu has quit IRC | 23:49 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!