| *** chuckC has quit IRC | 00:04 | |
| *** eguz has quit IRC | 00:11 | |
| *** matsuhashi has joined #tripleo | 00:19 | |
| lifeless | SpamapS: how did you go with CI ? | 00:24 |
|---|---|---|
| *** nati_ueno has quit IRC | 00:30 | |
| *** morazi has joined #tripleo | 00:38 | |
| *** nati_ueno has joined #tripleo | 00:41 | |
| greghaynes | I was expecting to find a nasty set of functions for the current heat stack-update routine and instead found a nice 10-liner :) | 00:44 |
| *** spzala has quit IRC | 00:46 | |
| lifeless | SpamapS: jobs aren't running | 00:47 |
| *** nati_uen_ has joined #tripleo | 00:59 | |
| greghaynes | \O/ I think I finally understand that heat convergence IRC over etherpad convo | 01:00 |
| greghaynes | now back to HA | 01:00 |
| *** newell has quit IRC | 01:01 | |
| *** nati_ueno has quit IRC | 01:03 | |
| SpamapS | lifeless: oh, nodes were spinning up.. are they unrecahable? | 01:03 |
| SpamapS | 2014-04-22 01:11:30,504 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [2/120s]: url error [[Errno 113] No route to host] | 01:05 |
| SpamapS | doh | 01:05 |
| SpamapS | lovely | 01:15 |
| SpamapS | novacompute5 appears to have frozen | 01:15 |
| SpamapS | resetting | 01:18 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Add a register-nodes command and associated API https://review.openstack.org/84933 | 01:21 |
| *** marun has quit IRC | 01:26 | |
| lifeless | SpamapS: thanks | 01:29 |
| SpamapS | begs the question.. wtf | 01:30 |
| StevenK | cody-somerville: Hai. Ping me when you're at your keys | 01:30 |
| *** nosnos has joined #tripleo | 01:36 | |
| *** rwsu has quit IRC | 01:47 | |
| *** chuckC has joined #tripleo | 01:53 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Make pull-tools ignore errors if USE_CACHE is set. https://review.openstack.org/89455 | 01:55 |
| tchaypo | *facepalm* | 01:59 |
| tchaypo | devtest.sh forcibly sets USE_CACHE to 0 | 01:59 |
| tchaypo | and requires that I pass -c to turn it back on | 01:59 |
| tchaypo | *shrug* i can live with that | 02:00 |
| tchaypo | :aq | 02:00 |
| *** chuckC has quit IRC | 02:06 | |
| cody-somerville | StevenK: pong. :) | 02:11 |
| *** morazi has quit IRC | 02:17 | |
| *** nati_uen_ has quit IRC | 02:27 | |
| *** nati_ueno has joined #tripleo | 02:33 | |
| lifeless | tchaypo: user interfaces, FTW | 02:39 |
| *** weshay has quit IRC | 02:43 | |
| *** nati_ueno has quit IRC | 02:45 | |
| lifeless | popping out for a little, if I'm needed I'm on hangouts / mobile phone. | 02:49 |
| *** nati_uen_ has joined #tripleo | 02:49 | |
| *** eghobo has joined #tripleo | 02:53 | |
| *** nati_uen_ has quit IRC | 02:55 | |
| tchaypo | lifeless: i was going to ask if you thought it was worth tweaking to respect USE_CACHE... | 02:59 |
| tchaypo | but then I realised you'd tell me that the best way to find out would be to submit a patch for review | 03:00 |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 03:00 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1309027 | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| uvirtbot | Launchpad bug 1309027 in tripleo "rabbitmq nodes repeatedly reporting the master node as being 'down' then 'up'" [Undecided,New] | 03:00 |
| *** jpeeler has quit IRC | 03:06 | |
| *** matsuhashi has quit IRC | 03:20 | |
| *** ramishra has joined #tripleo | 03:21 | |
| *** jml has quit IRC | 03:21 | |
| lifeless | tchaypo: ah no, so its deliberate | 03:21 |
| lifeless | tchaypo: the rationalise-ui goal is to remove env variables | 03:22 |
| lifeless | tchaypo: in favour of crisp clear interfaces, with plumbing-access reserved for special cases | 03:22 |
| *** jml has joined #tripleo | 03:23 | |
| *** ramishra has quit IRC | 03:23 | |
| *** ramishra has joined #tripleo | 03:24 | |
| greghaynes | lifeless: for the VIP's on control node, do you know if theres any WIP for that? | 03:24 |
| greghaynes | or anyone who was looking at it | 03:24 |
| lifeless | greghaynes: I don't believe so | 03:24 |
| tchaypo | what does plumbing-access mean in this case? | 03:24 |
| lifeless | greghaynes: I was hoping to do so, but ETIME | 03:25 |
| lifeless | tchaypo: e.g. setting PYPI_FUNKY_SETTING or something | 03:25 |
| lifeless | tchaypo: https://etherpad.openstack.org/p/tripleo-incubator-rationalise-ui | 03:25 |
| greghaynes | awesome. How granular should we make each vip? One per node, one per service, something inbetween? | 03:25 |
| lifeless | greghaynes: we can alwys change in future | 03:26 |
| greghaynes | ok. I kind of want to do minimal to make merge.py happy with scaling control node | 03:26 |
| greghaynes | so probably one for the node | 03:26 |
| tchaypo | oh yes - the things we expect users to twiddle are flags, the deeply embedded should-only-need-to-tweak-this-in-strange-corner-cases stuff is okay to still have an env variable | 03:27 |
| lifeless | tchaypo: or to have a passthrough facility in the json schemas we come up with | 03:28 |
| lifeless | greghaynes: I'd say 'one for all services' or something for your start point | 03:28 |
| lifeless | greghaynes: because one per node is definitely broken :) | 03:28 |
| greghaynes | haha, right | 03:28 |
| greghaynes | doesnt really buy much if you do one per node ;) | 03:29 |
| lifeless | exactly ;) | 03:31 |
| *** nosnos has quit IRC | 03:32 | |
| *** cwolferh has quit IRC | 03:37 | |
| tchaypo | I never cease to be amazed by how much the battery life estimates increase from "powertop --auto-tune" | 03:48 |
| tchaypo | also by just how much USE_CACHE still downloads fresh content from the interweb | 03:48 |
| *** cwolferh has joined #tripleo | 03:48 | |
| lifeless | tchaypo: USE_CACHE shouldn't do anything (other than the install-deps stuff, which if you setup your local pip conf shouldn't grab anything from the net | 03:49 |
| *** nati_ueno has joined #tripleo | 03:49 | |
| *** ramishra_ has joined #tripleo | 03:51 | |
| *** ramishra has quit IRC | 03:55 | |
| tchaypo | "shouldn't do anything" meaning "shouldn't touch the net" or "shouldn't change anything"? | 03:59 |
| *** ramishra has joined #tripleo | 03:59 | |
| *** TravT has quit IRC | 04:00 | |
| lifeless | tchaypo: shouldn't touch the net | 04:01 |
| *** tzumainn has quit IRC | 04:01 | |
| lifeless | tchaypo: assuming local pip and ubuntu mirror | 04:01 |
| tchaypo | I'm using devtest.sh directly rather than the sub-scripts - does that assumption still apply? | 04:01 |
| tchaypo | I was assuming it didn't, since the first failure I noticed is that it does an apt-get update and then barfs if one of the upstreams can't be reached | 04:02 |
| *** ramishra_ has quit IRC | 04:02 | |
| *** nati_ueno has quit IRC | 04:03 | |
| tchaypo | but maybe "local ubuntu mirror" covers that - I've updated by main ubuntu repos to point to my local mirror but haven't changed the ppas and such yet. I've got them mirrored just haven't changed sources.list | 04:03 |
| lifeless | tchaypo: the only place I know that would still hit the net with -c is install-dependencies, and then only if you don't have local mirrors | 04:03 |
| StevenK | tchaypo: Uh, yeah. Now your review is making me WTF, because pull-tools isn't even called if USE_CACHE is not 0 | 04:05 |
| StevenK | And is using ==, grrr | 04:05 |
| StevenK | lifeless: Is it useful to argue for my viewpoint of I'm happy to target bash, but would ask we nack bashisms iff the POSIX way is easy | 04:06 |
| StevenK | (Examples: [[ versus [ ; == versus =) | 04:06 |
| tchaypo | but maybe the apt-get behaviour I saw was before I figured out I need to use -c, setting USE_CACHE is ignored | 04:07 |
| tchaypo | StevenK: my understanding is that [[ is better than [ in some cases (and worse in a few), but I don't remember details. | 04:07 |
| lifeless | StevenK: I'm a huge [ = ] bigot :) | 04:08 |
| lifeless | StevenK: what review are you reading? | 04:08 |
| tchaypo | in any case, patch abandoned. | 04:08 |
| tchaypo | 89455 | 04:08 |
| StevenK | lifeless: The counter-point is set -o pipefail, but POSIX doesn't have anything close, and trying to implement it in POSIX is liable to make people's eyes bleed | 04:09 |
| tchaypo | lifeless: bigoted in favour of, or against? | 04:09 |
| lifeless | tchaypo: I far prefer [ = ] to [[ == ]] | 04:10 |
| StevenK | HAH | 04:10 |
| SpamapS | hm | 04:10 |
| StevenK | == is a pattern match | 04:10 |
| SpamapS | novacompute9 paniced | 04:10 |
| tchaypo | 98-source-repositories is currently barfing on me because it gets an answer it doesn't like talking to a git server (i think it's git.openstack.org) | 04:10 |
| lifeless | tchaypo: what doth the error read ? | 04:11 |
| lifeless | SpamapS: kernel ? | 04:11 |
| SpamapS | yeah | 04:11 |
| tchaypo | give me a minute and I'll have it for you again. | 04:12 |
| SpamapS | http://paste.openstack.org/show/76588/ | 04:12 |
| SpamapS | lifeless: https://review.openstack.org/#/c/88384/ <-- trusty passed CI.. before CI went into the weeds again | 04:12 |
| StevenK | Switched to a new branch 'double-things-make-me-double-sad' | 04:13 |
| SpamapS | 2014-04-22 04:11:22,278 - util.py[WARNING]: 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [116/120s]: url error [[Errno 113] No route to host] | 04:13 |
| SpamapS | I think might need to restart openvswitch agent | 04:13 |
| SpamapS | on the compute nodes | 04:13 |
| *** nati_ueno has joined #tripleo | 04:18 | |
| tchaypo | http://paste.openstack.org/show/Hd3A3wlOni8yYrGeMcaB/ | 04:18 |
| tchaypo | i think i can see the problem | 04:18 |
| tchaypo | -c can | 04:19 |
| tchaypo | -c can't use cached images if the images haven't been built yet. | 04:19 |
| StevenK | tchaypo: Yeah, so if you're going to hack on a plane, then you'd probably build the images before you leave, and then use -c while disconncted | 04:21 |
| StevenK | *disconnected | 04:21 |
| *** dkehn_ has joined #tripleo | 04:21 | |
| StevenK | And there's an option for building only that escapes me | 04:21 |
| SpamapS | I think that's new | 04:21 |
| lifeless | SpamapS: cool | 04:22 |
| lifeless | SpamapS: no route to host - metadata agent on the router | 04:22 |
| lifeless | SpamapS: *or* | 04:22 |
| lifeless | SpamapS: is gro off ? | 04:22 |
| lifeless | SpamapS: on the control node | 04:22 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Remove obvious bashisms like [[ and == https://review.openstack.org/89471 | 04:23 |
| SpamapS | lifeless: generic-receive-offload: on | 04:24 |
| SpamapS | doh | 04:24 |
| SpamapS | thought we had that in a post-up | 04:24 |
| lifeless | SpamapS: apparently not | 04:24 |
| *** dkehnx has quit IRC | 04:25 | |
| SpamapS | ah looks like I had to restart openvswitch-switch | 04:26 |
| SpamapS | but.. why... ?? | 04:26 |
| tchaypo | I think for now I'm going to switch tack and resume hacking on my hetzner box, and finish off building images on my laptop once I'm on proper wifi tonight | 04:26 |
| tchaypo | well, first I'm going to finish getting through the weekend email/review backlog | 04:27 |
| StevenK | tchaypo: Proper wifi? | 04:27 |
| StevenK | lifeless: You were talking about being in Sydney with L and C soonish, is that still happening pre-summit ? | 04:28 |
| tchaypo | as opposed to being in the back of a moving car on 4G | 04:28 |
| StevenK | tchaypo: Who kidnapped you, and do I need to send help? | 04:28 |
| *** matsuhashi has joined #tripleo | 04:28 | |
| *** nati_ueno has quit IRC | 04:29 | |
| lifeless | StevenK: next week | 04:29 |
| tchaypo | very polite kidnappers, to allow me to continue to work and communicate while they're taking me into the coutnryside | 04:29 |
| SpamapS | + os-apply-config --key keystone.db --type dsn | 04:30 |
| SpamapS | /usr/local/bin/cleanup-keystone-tokens.sh: 4: /usr/local/bin/cleanup-keystone-tokens.sh: os-apply-config: not found | 04:30 |
| SpamapS | + KEYSTONE_DB= | 04:30 |
| SpamapS | our cleanup tokens thing is still not working right | 04:30 |
| SpamapS | doh | 04:30 |
| lifeless | whee | 04:30 |
| lifeless | StevenK: shared the tripit with you | 04:31 |
| SpamapS | novacompute9 is backonline, seems to be booting vms | 04:31 |
| StevenK | tchaypo: Haha | 04:31 |
| lifeless | -> picking up C | 04:31 |
| StevenK | lifeless: I'll talk to Sarah about dinner with you/you and L sometime next week, then | 04:32 |
| *** SpamapS changes topic to "FIREDRILL: CI down - Hardware issues and kernel panics causing mayhem | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 04:32 | |
| greghaynes | Now with 100% real kernel panics | 04:33 |
| SpamapS | It's what plants crave | 04:33 |
| greghaynes | oh the irony of that movie title | 04:34 |
| * StevenK twitches over memories of Idiocracy | 04:34 | |
| SpamapS | Put this one in your ear, this one in your mouth, and this one in your butt. | 04:34 |
| SpamapS | no wait.. | 04:35 |
| SpamapS | It says you're totally tarded. That's ok, my sister's tarded.. she's a pilot. | 04:35 |
| StevenK | SpamapS, greghaynes: One comment I heard about the movie is that the actor on "Ow My Balls!" is the smartest guy since he knows he is an actor | 04:36 |
| *** akuznetsov has joined #tripleo | 04:36 | |
| greghaynes | lol | 04:36 |
| SpamapS | that sounds like a quite rich beer conversation to be had | 04:36 |
| *** nosnos has joined #tripleo | 04:37 | |
| StevenK | I may have to copy it my laptop so I can study for the conversation while on the plane | 04:37 |
| SpamapS | so it looks like there's something horribly wrong between ovs and neutron-openvswitch-agent .. | 04:38 |
| SpamapS | hah | 04:43 |
| SpamapS | so dhcp seemed wedged | 04:43 |
| SpamapS | unwedged it.. and now neutron-server is _pwned_ | 04:44 |
| *** sdake_ has quit IRC | 04:45 | |
| SpamapS | oo.. I see slaves starting | 04:50 |
| *** chuckC has joined #tripleo | 04:52 | |
| *** morganfainberg is now known as morganfainberg_Z | 04:55 | |
| SpamapS | https://jenkins03.openstack.org/job/check-tripleo-overcloud-precise/421/ | 04:55 |
| SpamapS | CI jobs flowing again | 04:55 |
| *** SpamapS changes topic to "tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 04:56 | |
| greghaynes | \O/ | 04:56 |
| *** Rakesh6 has joined #tripleo | 04:56 | |
| SpamapS | We really need HA... | 04:58 |
| SpamapS | and better hardware | 04:58 |
| SpamapS | the amount these go down is just insane | 04:58 |
| SpamapS | perhaps we should ask a hardware company known for having awesome servers to provide us with some of their awesome servers. :) | 04:58 |
| *** morganfainberg_Z is now known as morganfainberg | 04:59 | |
| cody-somerville | Someone should just try ordering some and see what happens ;p | 04:59 |
| StevenK | SpamapS: Totally | 04:59 |
| StevenK | SpamapS: What are they, out of interest? | 05:00 |
| *** matsuhas_ has joined #tripleo | 05:02 | |
| *** matsuhashi has quit IRC | 05:05 | |
| *** yamahata has joined #tripleo | 05:06 | |
| *** akuznetsov has quit IRC | 05:08 | |
| StevenK | greghaynes: I did mean POSIX in the commit message | 05:09 |
| *** nati_ueno has joined #tripleo | 05:09 | |
| greghaynes | Really? that reads odd.... not that its a big deal | 05:09 |
| StevenK | greghaynes: And =~ is a bashism, but POSIX doesn't have a analogue | 05:09 |
| greghaynes | yea, so doesnt that need to be left as bashism? | 05:10 |
| greghaynes | as in [[ ]] | 05:10 |
| StevenK | Let's find out | 05:11 |
| StevenK | ./tripleorc:8: ~ not found | 05:11 |
| StevenK | Handy | 05:11 |
| *** yamahata has quit IRC | 05:12 | |
| StevenK | greghaynes: I will try and reflow the commit message | 05:12 |
| *** yamahata has joined #tripleo | 05:12 | |
| greghaynes | eh, really dont mind :p | 05:12 |
| greghaynes | My nit there is the if and only if actually | 05:12 |
| *** akuznetsov has joined #tripleo | 05:12 | |
| StevenK | greghaynes: I'm trying to avoid people (including myself) writing eye bleeding shell to avoid set -o pipefail | 05:13 |
| greghaynes | hehe | 05:14 |
| StevenK | greghaynes: There are a number of things that POSIX doesn't support easily or well. [[ == ]] is not one of them | 05:14 |
| greghaynes | yes, so I think id say 'support POSIX if its easy' | 05:14 |
| greghaynes | I think youre right though | 05:15 |
| greghaynes | and I should remove that comment | 05:15 |
| greghaynes | Im still curious about the bashism though - I wrote a little test.sh that didnt seem happy about doing =~ in [ ] | 05:15 |
| StevenK | TripleO is targetting bash for scripts, but I think we should try and | 05:16 |
| StevenK | avoid bashisms, preferring POSIX constructs if they are easy-to-use. | 05:16 |
| StevenK | greghaynes: ^ | 05:16 |
| greghaynes | hehe, yes, /me admits sillyness on review | 05:16 |
| StevenK | greghaynes: Have you met /usr/bin/[ ? | 05:17 |
| greghaynes | indeed | 05:17 |
| StevenK | + elif [ -z "$SUCCESSFUL_MATCH_OUTPUT" ] && [ $STATUS -eq 0 ]; then | 05:18 |
| StevenK | Tempted to switch that to -a | 05:18 |
| StevenK | Like other places | 05:18 |
| StevenK | [ ] && [ ] is POSIX, but bloody hell -a has existed for YEAS | 05:19 |
| StevenK | *YEARS | 05:19 |
| greghaynes | I actually have been doing [ ] && [] | 05:20 |
| greghaynes | but not for any good reason | 05:20 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Remove obvious bashisms like [[ and == https://review.openstack.org/89471 | 05:20 |
| StevenK | greghaynes: Now you can shout POSIX | 05:20 |
| greghaynes | hehe | 05:21 |
| *** sdake_ has joined #tripleo | 05:22 | |
| greghaynes | StevenK: actually, wait_for has the same thing going on | 05:23 |
| StevenK | Yeah, I see it | 05:23 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Remove obvious bashisms like [[ and == https://review.openstack.org/89471 | 05:24 |
| *** akuznetsov has quit IRC | 05:26 | |
| *** ramishra has quit IRC | 05:26 | |
| *** ramishra has joined #tripleo | 05:27 | |
| *** akuznetsov has joined #tripleo | 05:28 | |
| greghaynes | lifeless: ran into http://goo.gl/f991oE in my fees and seems similar to the heat-convergence proposed design | 05:30 |
| greghaynes | er, in my feed | 05:31 |
| *** akuznetsov has quit IRC | 05:34 | |
| *** akuznetsov has joined #tripleo | 05:36 | |
| *** akuznetsov has quit IRC | 05:43 | |
| *** akuznetsov has joined #tripleo | 05:49 | |
| *** akuznetsov has quit IRC | 05:49 | |
| *** nati_ueno has quit IRC | 05:49 | |
| *** akuznetsov has joined #tripleo | 05:51 | |
| *** jeremydei has quit IRC | 05:53 | |
| *** jeremydei has joined #tripleo | 05:54 | |
| *** jtomasek has joined #tripleo | 05:58 | |
| lifeless | greghaynes: heh, yes | 05:59 |
| *** akuznetsov has quit IRC | 06:01 | |
| *** matsuhas_ has quit IRC | 06:02 | |
| *** jcoufal has joined #tripleo | 06:07 | |
| *** matsuhashi has joined #tripleo | 06:10 | |
| *** ifarkas has joined #tripleo | 06:10 | |
| *** nati_ueno has joined #tripleo | 06:12 | |
| *** viktors has joined #tripleo | 06:16 | |
| *** yamahata has quit IRC | 06:19 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Store seed details in the JSON https://review.openstack.org/79051 | 06:31 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/diskimage-builder: Small fixes for dhcp-all-interfaces https://review.openstack.org/88131 | 06:40 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Build os-cloud-config in setup-clienttools https://review.openstack.org/84935 | 06:41 |
| openstackgerrit | Juerg Haefliger proposed a change to openstack/diskimage-builder: Add sysv support to elements/dhcp-all-interfaces https://review.openstack.org/86299 | 06:43 |
| greghaynes | marios: ping | 06:44 |
| greghaynes | mordred: Need to hed to bed, but added a (hopefully not too confusing) response to https://review.openstack.org/#/c/83296/. Thanks again! | 06:46 |
| greghaynes | er, marios ^ | 06:46 |
| *** jprovazn has joined #tripleo | 06:52 | |
| *** viktors has quit IRC | 06:58 | |
| marios | greghaynes: thanks | 07:03 |
| * marios looks | 07:03 | |
| *** e0ne has joined #tripleo | 07:03 | |
| *** chuckC has quit IRC | 07:04 | |
| *** pblaho has joined #tripleo | 07:10 | |
| *** e0ne has quit IRC | 07:11 | |
| *** viktors has joined #tripleo | 07:13 | |
| rpodolyaka1 | morning tripleo | 07:14 |
| marios | o/ | 07:14 |
| *** eghobo has quit IRC | 07:15 | |
| *** e0ne has joined #tripleo | 07:21 | |
| *** shardy_afk is now known as shardy | 07:30 | |
| *** hashar has joined #tripleo | 07:31 | |
| *** jcoufal has quit IRC | 07:33 | |
| *** jcoufal has joined #tripleo | 07:40 | |
| *** mrunge has joined #tripleo | 07:42 | |
| *** ohadlevy_ is now known as ohadlevy | 07:47 | |
| *** ohadlevy has joined #tripleo | 07:47 | |
| lifeless | morning rpodolyaka1 | 07:51 |
| *** nati_ueno has quit IRC | 07:54 | |
| *** giulivo has joined #tripleo | 07:55 | |
| Ng | morning | 07:56 |
| *** andreaf has joined #tripleo | 07:59 | |
| *** e0ne has quit IRC | 07:59 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Remove quota restrictions on undercloud's nova https://review.openstack.org/87126 | 08:00 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Make time out 60 mins like the Heat default. https://review.openstack.org/87118 | 08:00 |
| *** e0ne has joined #tripleo | 08:01 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: SSH key and virtual_power_driver not used on H/W https://review.openstack.org/83770 | 08:01 |
| *** derekh has joined #tripleo | 08:03 | |
| *** jtomasek has quit IRC | 08:04 | |
| *** jtomasek has joined #tripleo | 08:04 | |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 08:04 |
| *** Matt1 has quit IRC | 08:05 | |
| *** andreaf has quit IRC | 08:12 | |
| *** jistr has joined #tripleo | 08:12 | |
| *** andreaf has joined #tripleo | 08:13 | |
| *** martyntaylor has joined #tripleo | 08:14 | |
| *** lucasagomes has joined #tripleo | 08:19 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Allows user-specified PXE timeout for overcloud https://review.openstack.org/86605 | 08:19 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Correct hard-coded IP ranges https://review.openstack.org/86308 | 08:24 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Bridge physical interface to the seed. https://review.openstack.org/84083 | 08:28 |
| *** panda has joined #tripleo | 08:33 | |
| *** bauzas has joined #tripleo | 08:34 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Fix up ensure-test-env https://review.openstack.org/88472 | 08:35 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Move the testenv json file to /etc https://review.openstack.org/88473 | 08:35 |
| openstackgerrit | A change was merged to openstack/python-tuskarclient: Remove py3kcompat module https://review.openstack.org/87642 | 08:37 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Fixing manifest.in https://review.openstack.org/86916 | 08:39 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Delete horizon.d3linechart.js https://review.openstack.org/86379 | 08:42 |
| *** e0ne has quit IRC | 08:44 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Add Django 1.5 and 1.6 to tox.ini https://review.openstack.org/87558 | 08:45 |
| *** hashar has quit IRC | 08:45 | |
| *** akrivoka has joined #tripleo | 08:52 | |
| *** e0ne has joined #tripleo | 08:53 | |
| *** hashar has joined #tripleo | 08:54 | |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 09:00 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1309027 | 09:00 |
| uvirtbot | Launchpad bug 1309027 in tripleo "rabbitmq nodes repeatedly reporting the master node as being 'down' then 'up'" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| *** ramishra has quit IRC | 09:01 | |
| *** tzumainn has joined #tripleo | 09:05 | |
| *** jogo has quit IRC | 09:07 | |
| *** jogo has joined #tripleo | 09:07 | |
| openstackgerrit | Victor Sergeyev proposed a change to openstack/tripleo-incubator: Fixes for manual installation https://review.openstack.org/87955 | 09:09 |
| *** ramishra has joined #tripleo | 09:10 | |
| *** akuznetsov has joined #tripleo | 09:14 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Haproxy should listen only on vip https://review.openstack.org/89517 | 09:20 |
| openstackgerrit | therese-mchale proposed a change to openstack/tripleo-image-elements: Create ~/.my.cnf with root user/password https://review.openstack.org/89520 | 09:41 |
| *** dshulyak has joined #tripleo | 09:42 | |
| *** jang1 has joined #tripleo | 09:43 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-image-elements: Nasty hack to work around pip's case-sensitivity https://review.openstack.org/84946 | 09:44 |
| *** martyntaylor has left #tripleo | 09:49 | |
| *** petertoft has joined #tripleo | 09:51 | |
| *** markmc has joined #tripleo | 09:52 | |
| openstackgerrit | Cian O'Driscoll proposed a change to openstack-infra/tripleo-ci: Test the upgrade codepath works as well. https://review.openstack.org/87758 | 09:58 |
| *** dkehn__ has joined #tripleo | 10:07 | |
| *** dkehn_ has quit IRC | 10:10 | |
| *** e0ne has quit IRC | 10:20 | |
| proffalken | SpamapS: on https://review.openstack.org/#/c/85557/5/elements/rabbitmq-server/os-refresh-config/post-configure.d/40-rabbitmq you've said that hostname -s will suffice | 10:23 |
| openstackgerrit | Cian O'Driscoll proposed a change to openstack/tripleo-image-elements: Store ssh host keys on ephemeral partition https://review.openstack.org/89529 | 10:23 |
| proffalken | does hostname -s automatically lowercase then? | 10:23 |
| proffalken | or have I missed something else in your comment that does this? | 10:23 |
| howleyt | Can I get some core reviews of these please?: https://review.openstack.org/#/c/81920/ and https://review.openstack.org/#/c/81934/ | 10:28 |
| lxsli | greghaynes: Hi Greg, for t-i-e element openjdk-7-jre you said to update packages-map, did you mean map-packages in various dib elements? If so that'll have to be a separate review, happy to do it though | 10:29 |
| derekh | Ng: would you have any idea if there is net issues currently where the HP rack is hosted, getting a fair few nova api timeouts to the ci-overcloud, the timeouts don't seem to be happening if using the nova client from the rack itself. | 10:32 |
| *** hashar has quit IRC | 10:33 | |
| Ng | derekh: hmm, if there were an issue there it wouldn't automatically drift across my radar. how long's it been going on/ | 10:33 |
| tchaypo | Ng: oh hai | 10:34 |
| Ng | tchaypo: hey | 10:34 |
| derekh | Ng: I've been trying to figure out problems for the last 2 hours, and have been geting timeouts all along, so at least 2 hours but I don't know how much longer | 10:37 |
| Ng | ouch | 10:37 |
| Ng | derekh: but ssh into the rack is stable/ | 10:37 |
| derekh | Ng: the zuul queue still has things in it from yesterday, which is what I was looking into | 10:37 |
| Ng | why can I not type a question mark today | 10:37 |
| derekh | Ng: which may be related | 10:38 |
| Ng | hmm | 10:38 |
| Ng | I'll check with our NOC folks | 10:38 |
| derekh | Ng: ssh is working (although does seem slow but I could be just imagining that) | 10:38 |
| openstackgerrit | Alexis Lee proposed a change to openstack/diskimage-builder: Map openjdk-7-jre to RHEL+SUSE https://review.openstack.org/89531 | 10:42 |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 10:48 |
| *** akrivoka has quit IRC | 10:48 | |
| *** akrivoka has joined #tripleo | 10:51 | |
| *** e0ne has joined #tripleo | 10:58 | |
| *** derekh has quit IRC | 11:01 | |
| *** e0ne has quit IRC | 11:03 | |
| *** e0ne has joined #tripleo | 11:14 | |
| *** lucasagomes is now known as lucas-hungry | 11:22 | |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: mysql-common requires python MySQLdb https://review.openstack.org/83969 | 11:27 |
| *** e0ne has quit IRC | 11:52 | |
| *** pblaho has quit IRC | 11:57 | |
| *** killer_prince has quit IRC | 11:57 | |
| *** pblaho has joined #tripleo | 11:57 | |
| *** jcoufal has quit IRC | 12:04 | |
| *** jcoufal has joined #tripleo | 12:04 | |
| *** rpodolyaka1 has quit IRC | 12:04 | |
| *** jdob has joined #tripleo | 12:06 | |
| *** morazi has joined #tripleo | 12:09 | |
| *** jcoufal has quit IRC | 12:09 | |
| *** jdob has quit IRC | 12:09 | |
| *** jcoufal has joined #tripleo | 12:09 | |
| *** jdob has joined #tripleo | 12:09 | |
| *** jcoufal has quit IRC | 12:14 | |
| *** jcoufal has joined #tripleo | 12:14 | |
| *** jcoufal has quit IRC | 12:15 | |
| *** jcoufal has joined #tripleo | 12:15 | |
| openstackgerrit | Cian O'Driscoll proposed a change to openstack/tripleo-image-elements: Store ssh host keys on ephemeral partition https://review.openstack.org/89529 | 12:17 |
| *** e0ne has joined #tripleo | 12:17 | |
| *** jcoufal has quit IRC | 12:18 | |
| *** jcoufal has joined #tripleo | 12:19 | |
| *** jcoufal has quit IRC | 12:20 | |
| *** jcoufal has joined #tripleo | 12:20 | |
| *** lucas-hungry is now known as lucasagomes | 12:20 | |
| *** hashar has joined #tripleo | 12:23 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Add selenium test-case https://review.openstack.org/86347 | 12:24 |
| *** e0ne_ has joined #tripleo | 12:24 | |
| *** e0ne has quit IRC | 12:26 | |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 12:33 |
| *** Hazelesque has quit IRC | 12:35 | |
| *** weshay has joined #tripleo | 12:35 | |
| *** Hazelesque has joined #tripleo | 12:35 | |
| *** derekh has joined #tripleo | 12:41 | |
| *** matsuhashi has quit IRC | 12:43 | |
| *** matsuhashi has joined #tripleo | 12:43 | |
| *** jpeeler has joined #tripleo | 12:46 | |
| *** jpeeler has joined #tripleo | 12:46 | |
| *** matsuhashi has quit IRC | 12:48 | |
| *** blamar has quit IRC | 12:48 | |
| *** nosnos has quit IRC | 12:48 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-image-elements: Haproxy should listen only on vip https://review.openstack.org/89517 | 12:48 |
| *** rcarrillocruz1 has joined #tripleo | 12:49 | |
| *** rcarrillocruz has quit IRC | 12:49 | |
| *** blamar has joined #tripleo | 12:49 | |
| *** Rakesh6 has quit IRC | 12:50 | |
| *** nosnos has joined #tripleo | 12:51 | |
| *** nosnos has quit IRC | 12:52 | |
| *** bauzas has quit IRC | 12:52 | |
| *** bauzas has joined #tripleo | 12:53 | |
| *** viktors has quit IRC | 12:54 | |
| *** e0ne has joined #tripleo | 12:54 | |
| *** viktors has joined #tripleo | 12:56 | |
| openstackgerrit | Peter Belanyi proposed a change to openstack/python-tuskarclient: Implement finding by name https://review.openstack.org/88345 | 12:56 |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-heat-templates: Introduce configurable virtual ip in templates https://review.openstack.org/89556 | 12:57 |
| *** dkehn__ is now known as dkehnx | 12:57 | |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Read libvirt block_migration_flag from nova.conf https://review.openstack.org/84657 | 12:57 |
| *** ramishra has quit IRC | 12:58 | |
| *** lazy_prince has joined #tripleo | 12:58 | |
| *** lazy_prince is now known as killer_prince | 12:58 | |
| *** e0ne_ has quit IRC | 12:58 | |
| *** dprince has joined #tripleo | 12:59 | |
| *** e0ne_ has joined #tripleo | 13:02 | |
| *** sseago has quit IRC | 13:04 | |
| *** e0ne has quit IRC | 13:06 | |
| *** pblaho has quit IRC | 13:08 | |
| *** bauzas has quit IRC | 13:10 | |
| *** ccrouch has joined #tripleo | 13:14 | |
| *** matty_dubs|gone is now known as matty_dubs | 13:23 | |
| derekh | Ng: the api problem I was observing this morning seems to be gone, although we still have a lot of vm's going to the error state | 13:25 |
| Ng | derekh: the cloud giveth, the cloud taketh away | 13:26 |
| Ng | derekh: any indication what's causing the erroring? | 13:26 |
| derekh | Ng: nothing obvious :-( | 13:27 |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 13:27 |
| *** jprovazn has quit IRC | 13:28 | |
| *** jprovazn has joined #tripleo | 13:30 | |
| *** ramishra has joined #tripleo | 13:30 | |
| dprince | slagle/derekh: you guys approved this before https://review.openstack.org/#/c/76966/ | 13:32 |
| dprince | derekh: thanks! | 13:37 |
| derekh | dprince: np | 13:37 |
| *** julim has joined #tripleo | 13:42 | |
| *** rpodolyaka has joined #tripleo | 13:52 | |
| *** mrunge has quit IRC | 13:55 | |
| *** e0ne_ has quit IRC | 13:55 | |
| *** e0ne has joined #tripleo | 13:55 | |
| *** e0ne has quit IRC | 13:59 | |
| funzo | derekh: hello. i saw some work you had done on nova-docker. I'm interested in working on nova-docker, are you still involved in that effort? | 14:00 |
| derekh | funzo: cool, yup I'm just waiting on the ci job to be created, so we can then build on the what is there https://review.openstack.org/#/c/86910/ | 14:02 |
| *** e0ne has joined #tripleo | 14:03 | |
| *** rwsu has joined #tripleo | 14:09 | |
| *** CaptTofu has quit IRC | 14:10 | |
| *** CaptTofu has joined #tripleo | 14:11 | |
| *** jprovazn has quit IRC | 14:12 | |
| *** jprovazn has joined #tripleo | 14:13 | |
| *** CaptTofu has quit IRC | 14:15 | |
| *** CaptTofu has joined #tripleo | 14:17 | |
| *** bauzas has joined #tripleo | 14:17 | |
| *** spzala has joined #tripleo | 14:17 | |
| hewbrocca | funzo: wooo nova-docker | 14:22 |
| hewbrocca | funzo: Slow is also working on it | 14:22 |
| *** morganfainberg is now known as morganfainberg_Z | 14:24 | |
| funzo | hewbrocca: awesome! | 14:27 |
| hewbrocca | Yes | 14:27 |
| hewbrocca | funzo: getting the band back together :) | 14:27 |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-incubator: Use config baremetal seed ip addr not 192.0.2.1 https://review.openstack.org/87276 | 14:27 |
| funzo | hewbrocca: srsly. derekh I will crash the thursday hangout to listen in, maybe we will need the same work done and can join forces | 14:27 |
| *** newell has joined #tripleo | 14:28 | |
| *** jprovazn has quit IRC | 14:29 | |
| funzo | derekh: other than the CI job creation, do you know what's next? from the mailing list, I gathered that nova-docker needs test coverage before it can be considered for the nova compute service | 14:29 |
| *** jcoufal has quit IRC | 14:29 | |
| funzo | hewbrocca: derekh if we use this channel to coordinate nova-docker effort, is it too much noise? should we be talking about it in a different channel? | 14:30 |
| derekh | I think test coverage is the main thing, yes lets take this out of #tripleo :-) | 14:31 |
| funzo | derekh: any opposition to creating #nova-docker? | 14:32 |
| derekh | funzo: fire ahead, brb | 14:32 |
| *** e0ne_ has joined #tripleo | 14:32 | |
| hewbrocca | funzo: you might ping Slow when he gets on and let him know | 14:32 |
| *** sseago has joined #tripleo | 14:32 | |
| funzo | derekh: hewbrocca: #nova-docker created | 14:32 |
| *** e0ne has quit IRC | 14:36 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Make bin/ensure-bridge executable https://review.openstack.org/89593 | 14:38 |
| openstackgerrit | Ryan Moore proposed a change to openstack/tripleo-image-elements: Restructure the nova.conf to match documentation https://review.openstack.org/83821 | 14:41 |
| *** marun has joined #tripleo | 14:41 | |
| *** sseago has quit IRC | 14:41 | |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add allow_empty to metadata header https://review.openstack.org/89597 | 14:46 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add mode to metadata header https://review.openstack.org/89598 | 14:46 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add metadata header https://review.openstack.org/82038 | 14:46 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Allow multiple passes of template evaluation https://review.openstack.org/88308 | 14:46 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add metadata header https://review.openstack.org/82038 | 14:50 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add allow_empty to metadata header https://review.openstack.org/89597 | 14:50 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Allow multiple passes of template evaluation https://review.openstack.org/88308 | 14:50 |
| openstackgerrit | Alexis Lee proposed a change to openstack/os-apply-config: Add mode to metadata header https://review.openstack.org/89598 | 14:50 |
| *** sseago has joined #tripleo | 14:55 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1302881 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1309027 | 15:00 |
| uvirtbot | Launchpad bug 1302881 in tripleo "incloud CIDR can overlap custom baremetal-network" [Undecided,Incomplete] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1311103 | 15:00 |
| uvirtbot | Launchpad bug 1309027 in tripleo "rabbitmq nodes repeatedly reporting the master node as being 'down' then 'up'" [Undecided,New] | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| uvirtbot | Launchpad bug 1311103 in tripleo "undercloud fails with UNDERCLOUD_TUSKAR_PASSWORD: unbound variable" [Undecided,New] | 15:00 |
| *** marun has quit IRC | 15:02 | |
| *** derekh has quit IRC | 15:02 | |
| *** nekron99 has quit IRC | 15:08 | |
| *** jcoufal has joined #tripleo | 15:12 | |
| openstackgerrit | Dmitry Shulyak proposed a change to openstack/tripleo-incubator: Add virtual ip create to devtest_overcloud.sh https://review.openstack.org/89613 | 15:23 |
| dshulyak | hi guys, i've added WIP patch that allocates virtual ip from baremetal network range ^^ | 15:25 |
| dshulyak | and I would like to discuss how we should approach its creation | 15:26 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Add set-pip-vars script to simplify pip manifests https://review.openstack.org/83194 | 15:26 |
| dprince | dshulyak: it might be nice to see that integrated into the JSON format here: http://git.openstack.org/cgit/openstack/tripleo-incubator/tree/scripts/devtest_setup.sh#n115 | 15:27 |
| dprince | dshulyak: I suppose this is really just overcloud though so maybe the env variable is acceptable for now? | 15:29 |
| *** petertoft has quit IRC | 15:30 | |
| *** hashar has quit IRC | 15:30 | |
| *** hashar has joined #tripleo | 15:31 | |
| dshulyak | dprince: i think env variable is good enough, but i will check how it can be integrated better | 15:32 |
| dshulyak | dprince: should we hardcode it outside of network range? | 15:32 |
| dshulyak | baremetal network | 15:33 |
| dshulyak | or allocate it via neutron? | 15:34 |
| *** hashar has quit IRC | 15:34 | |
| dprince | dshulyak: if we are going to pull this from the undercloud range then doing so dynamically would be best. | 15:34 |
| dprince | dshulyak: the value you have coded in now will likely almost always file right (because the single undercloud node would usually get that one?) | 15:35 |
| dshulyak | dprince: thanks, it seems i was right with my approach, i will polish it a bit and send on review | 15:46 |
| *** jpeeler has quit IRC | 15:46 | |
| *** jistr has quit IRC | 15:48 | |
| *** jpeeler has joined #tripleo | 15:48 | |
| *** jistr has joined #tripleo | 15:50 | |
| goneri | bnemec: hi! thanks for the review, FYI Jenkins validated the patch https://review.openstack.org/#/c/86955/ | 15:50 |
| *** hashar has joined #tripleo | 15:50 | |
| *** chuckC has joined #tripleo | 15:51 | |
| *** eghobo has joined #tripleo | 15:51 | |
| *** viktors is now known as viktors_away | 15:52 | |
| bnemec | goneri: Actually, the tripleo-check job hasn't finished yet because of the CI issues we were having yesterday. | 15:52 |
| *** derekh has joined #tripleo | 15:53 | |
| *** jpeeler has quit IRC | 15:54 | |
| *** jpeeler has joined #tripleo | 15:54 | |
| goneri | bnemec: oh, I misunderstood | 15:55 |
| *** pbelanyi_ has joined #tripleo | 15:56 | |
| openstackgerrit | Phil Neal proposed a change to openstack/tripleo-incubator: Add Ceilometer to control node by default https://review.openstack.org/89625 | 15:57 |
| *** mrunge has joined #tripleo | 15:58 | |
| *** morazi has quit IRC | 15:58 | |
| *** morazi has joined #tripleo | 16:02 | |
| *** bauzas has quit IRC | 16:02 | |
| openstackgerrit | A change was merged to openstack/python-tuskarclient: Implement finding by name https://review.openstack.org/88345 | 16:09 |
| *** e0ne_ has quit IRC | 16:09 | |
| *** matty_dubs is now known as matty_dubs|lunch | 16:11 | |
| *** jistr has quit IRC | 16:12 | |
| *** lucasagomes is now known as lucas-afk | 16:13 | |
| *** julim has quit IRC | 16:15 | |
| *** mrunge has quit IRC | 16:15 | |
| *** cwolferh has quit IRC | 16:20 | |
| *** julim has joined #tripleo | 16:20 | |
| *** cwolferh has joined #tripleo | 16:20 | |
| *** vinsh has quit IRC | 16:21 | |
| *** hashar has quit IRC | 16:26 | |
| *** bauzas1 has joined #tripleo | 16:31 | |
| *** vinsh has joined #tripleo | 16:34 | |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: RabbitMQ - Ensure node name comparisons are case insensitive https://review.openstack.org/85557 | 16:37 |
| *** markmc has quit IRC | 16:37 | |
| *** morazi has quit IRC | 16:38 | |
| bnemec | jogo: Regarding https://review.openstack.org/#/c/87978/2 , I'm a little confused as to why anyone would have compute nodes without proper network connectivity in the first place. | 16:39 |
| bnemec | I think there's more information needed about the use case they're trying to address there before we can decide where it needs to live. | 16:39 |
| jogo | bnemec: if network fails? | 16:39 |
| jogo | bnemec: re: hardware failures. | 16:40 |
| bnemec | jogo: Yeah, I guess I'm not sure monitoring is part of the tripleo mission. | 16:41 |
| bnemec | That seems like a separate thing, that tripleo could then potentially use. | 16:41 |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: RabbitMQ - Increase restart timeouts https://review.openstack.org/89640 | 16:44 |
| openstackgerrit | Kiall Mac Innes proposed a change to openstack/tripleo-image-elements: RabbitMQ - Consistent PID file location https://review.openstack.org/85604 | 16:44 |
| bnemec | jogo: Also, we wouldn't want to schedule around a disconnected compute node, would we? Isn't the right thing to do if you detect a broken node to evacuate all the instances on it and disable it completely? | 16:46 |
| bnemec | Maybe I just need to comment on the review. :-) | 16:46 |
| jogo | bnemec: yup. this is a more general problem | 16:47 |
| jogo | bnemec: how do I detect bad nodes and take them out of rotaation | 16:48 |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Remove quota restrictions on undercloud's nova https://review.openstack.org/87126 | 16:50 |
| *** morazi has joined #tripleo | 16:51 | |
| *** derekh has quit IRC | 16:51 | |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Remove quotas on seed and undercloud's nova https://review.openstack.org/87126 | 16:52 |
| *** ramishra has quit IRC | 16:52 | |
| *** nekron99 has joined #tripleo | 16:52 | |
| *** ramishra has joined #tripleo | 16:53 | |
| *** ramishra has quit IRC | 16:53 | |
| *** ramishra has joined #tripleo | 16:53 | |
| *** jcoufal has quit IRC | 16:55 | |
| *** bauzas1 has quit IRC | 16:57 | |
| *** nekron99 has quit IRC | 16:57 | |
| *** matty_dubs|lunch is now known as matty_dubs | 17:02 | |
| *** pbelanyi_ has quit IRC | 17:03 | |
| *** mrunge has joined #tripleo | 17:03 | |
| openstackgerrit | sowmini proposed a change to openstack/tripleo-image-elements: Config updates for dhcp, metadata, l3_agent https://review.openstack.org/89644 | 17:06 |
| *** eghobo has quit IRC | 17:07 | |
| *** eghobo has joined #tripleo | 17:07 | |
| SpamapS | dprince: I notice that most jobs in the CI queue are waiting on f20 .. is the RH region up and stable? | 17:08 |
| dprince | SpamapS: yes, https://review.openstack.org/#/c/83057/ | 17:11 |
| *** marun has joined #tripleo | 17:14 | |
| *** nati_ueno has joined #tripleo | 17:14 | |
| *** dkehn has quit IRC | 17:19 | |
| *** dkehn has joined #tripleo | 17:20 | |
| SpamapS | dprince: wait so how are the current f20's happening? | 17:22 |
| greghaynes | lxsli: Yep, the map-packages. Youre right about the separate review but that one would need to land first to keep CI passing | 17:23 |
| dprince | SpamapS: the intent is that we'll be another provider who can run any distro | 17:24 |
| dprince | SpamapS: current F20 runs are running on HP sir :) | 17:24 |
| SpamapS | dprince: OH :) | 17:24 |
| dprince | SpamapS: sorry, I missed the gist of your initial question. Sounds like F20's are just slow then? | 17:25 |
| dprince | SpamapS: let me poke around, derekh would know better but I might be able to have a look too | 17:25 |
| *** morganfainberg_Z is now known as morganfainberg | 17:26 | |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 17:29 |
| *** giulivo has quit IRC | 17:31 | |
| *** panda has quit IRC | 17:35 | |
| *** panda has joined #tripleo | 17:35 | |
| *** dkehn has quit IRC | 17:36 | |
| *** epim has joined #tripleo | 17:37 | |
| *** dkehn has joined #tripleo | 17:37 | |
| openstackgerrit | Coleman Corrigan proposed a change to openstack/tripleo-image-elements: reset-db to get all db parameters from the config https://review.openstack.org/88340 | 17:42 |
| SpamapS | dprince: I keep forgetting that the testenv's are where the OS stuff happens anyway | 17:42 |
| *** olaph has joined #tripleo | 17:46 | |
| *** martyntaylor has joined #tripleo | 17:47 | |
| *** sdake is now known as sdake-afk | 17:49 | |
| lifeless | Morning | 17:53 |
| greghaynes | 'morning | 17:53 |
| *** chuckC has quit IRC | 17:57 | |
| *** absubram has joined #tripleo | 17:58 | |
| absubram | hi.. I'm trying out devtest and I keep hitting this timeout issue when it runs the heat/requirements | 17:59 |
| absubram | Timeout: (<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x9336cac>, 'Connection to pypi.python.org timed out. (connect timeout=15)') | 17:59 |
| absubram | anyone hit this also? | 17:59 |
| *** mrunge has quit IRC | 18:03 | |
| lifeless | absubram: are you behind a corporate firewall ? | 18:05 |
| *** sdake-afk is now known as sdake | 18:05 | |
| lifeless | absubram: you may want to create a local pypi-mirror, there are some docs for that in diskimage-builder/elements/pypi/README.md | 18:06 |
| *** chuckC has joined #tripleo | 18:09 | |
| absubram | lifeless: ah.. thanks.. that might be it.. let me give this a shot | 18:10 |
| *** pblaho has joined #tripleo | 18:13 | |
| *** eguz has joined #tripleo | 18:15 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Slight increase in testenv disk space https://review.openstack.org/88459 | 18:17 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Slight increase in testenv disk space https://review.openstack.org/88459 | 18:17 |
| SpamapS | lxsli: Nice job on the o-a-c header spec. I added some comments. | 18:18 |
| SpamapS | lxsli: I suggest sending a link to the etherpad/blueprint to openstack-dev for wider review. | 18:19 |
| *** eghobo has quit IRC | 18:19 | |
| lifeless | https://flynn.io/docs/architecture | 18:19 |
| *** jtomasek has quit IRC | 18:20 | |
| lifeless | lxsli: what tz are you in ? | 18:20 |
| *** akuznetsov has quit IRC | 18:21 | |
| *** dshulyak_ has joined #tripleo | 18:21 | |
| *** ramishra has quit IRC | 18:22 | |
| *** ramishra has joined #tripleo | 18:23 | |
| *** jprovazn has joined #tripleo | 18:23 | |
| *** ramishra_ has joined #tripleo | 18:26 | |
| *** martyntaylor has left #tripleo | 18:26 | |
| *** ramishra has quit IRC | 18:27 | |
| *** jtomasek has joined #tripleo | 18:28 | |
| openstackgerrit | Ben Nemec proposed a change to openstack/os-refresh-config: Use FHS-compliant default base directory https://review.openstack.org/89667 | 18:31 |
| openstackgerrit | Ben Nemec proposed a change to openstack/os-apply-config: Use FHS-compliant default template location https://review.openstack.org/89668 | 18:31 |
| *** dshulyak_ has quit IRC | 18:35 | |
| *** chuckC has quit IRC | 18:36 | |
| *** nekron99 has joined #tripleo | 18:37 | |
| NobodyCam | question. has anyone tested with the opensuse element? how about suse enterprize? | 18:40 |
| *** hashar has joined #tripleo | 18:41 | |
| *** ramishra_ has quit IRC | 18:41 | |
| *** ramishra has joined #tripleo | 18:42 | |
| greghaynes | I tried it a while back, put up patches to fix a few super low-hanging-fruit bugs but then ran into other issues | 18:42 |
| greghaynes | pretty easy to test though - just set NODE_DIST | 18:42 |
| NobodyCam | greghaynes: ack.... | 18:47 |
| NobodyCam | :) | 18:47 |
| *** ramishra has quit IRC | 18:47 | |
| *** openstackgerrit has quit IRC | 18:49 | |
| *** chuckC has joined #tripleo | 18:49 | |
| lifeless | anyone wanting a cheap make-CI-faster thing - fix horizon to not recompress stylesheets etc on every os-c-c run. | 18:49 |
| *** openstackgerrit has joined #tripleo | 18:49 | |
| tteggel | lifeless: lxsli is UK so UTC +1 | 18:54 |
| lifeless | tteggel: thanks | 18:54 |
| *** nati_ueno has quit IRC | 18:58 | |
| lifeless | meeting time! | 18:59 |
| jdob | o/ | 18:59 |
| jdob | damnit, wrong window | 19:00 |
| *** dprince has quit IRC | 19:00 | |
| *** jistr has joined #tripleo | 19:01 | |
| *** derekh has joined #tripleo | 19:03 | |
| *** nati_ueno has joined #tripleo | 19:04 | |
| *** jcoufal has joined #tripleo | 19:05 | |
| *** GheAway is now known as GheRivero | 19:07 | |
| lifeless | rpodolyaka: joinging the meeting ? | 19:12 |
| *** nati_ueno has quit IRC | 19:14 | |
| *** e0ne has joined #tripleo | 19:19 | |
| *** dprince has joined #tripleo | 19:19 | |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Add passthrough configuration glue. https://review.openstack.org/87843 | 19:20 |
| *** nati_ueno has joined #tripleo | 19:23 | |
| *** akuznetsov has joined #tripleo | 19:27 | |
| *** akuznetsov has quit IRC | 19:31 | |
| *** hewbrocca has quit IRC | 19:39 | |
| *** pblaho has quit IRC | 19:55 | |
| *** markmc has joined #tripleo | 19:57 | |
| tchaypo | I think that defaulting to things that one set of production-hardened people want won't always remove the need to make something configureable | 20:01 |
| *** jcoufal has quit IRC | 20:01 | |
| *** hewbrocca has joined #tripleo | 20:01 | |
| SpamapS | tchaypo: absolutely, but it reduces the priority for working passthrough. :) | 20:01 |
| *** derekh has quit IRC | 20:03 | |
| tchaypo | if only because we can defer adding it until we know we need it | 20:06 |
| tchaypo | We should just go for the simpler option | 20:06 |
| *** TravT has joined #tripleo | 20:07 | |
| tchaypo | invent an Artificial General Intelligence to create the config files for us. | 20:07 |
| greghaynes | really we should just ship a grammar for the config file in a given template :p | 20:08 |
| SpamapS | so.. CI.. ok.. does look like the compute nodes are MIA | 20:09 |
| SpamapS | ok I'm going to need some lunch in me before I attack this.. again. | 20:10 |
| greghaynes | +1 on lunch -> noms | 20:10 |
| tchaypo | greghaynes: one grammar config file? | 20:11 |
| tchaypo | hang on, let me come back and re-parse that later. Still not sufficiently caffeinated | 20:12 |
| greghaynes | just map each config file to a BNF grammar | 20:12 |
| greghaynes | (not serious) | 20:12 |
| tchaypo | we can take the sendmail approach | 20:13 |
| greghaynes | haha | 20:13 |
| tchaypo | have heat provide a dict, which we parse through a series of M4 macros into a turing-complete config language | 20:13 |
| SpamapS | tchaypo: I KEEEEEL YOU | 20:14 |
| SpamapS | tchaypo: http://fc02.deviantart.net/fs46/f/2009/220/5/1/I_KEEL_YOU_by_burning_blizzard.jpg | 20:14 |
| tchaypo | c'mon, have you met eric? he's a lovely guy. | 20:15 |
| tchaypo | why are you hating on his seemed-sensible-at-the-time choices? :p | 20:16 |
| *** andreaf_ has joined #tripleo | 20:16 | |
| tchaypo | oh yeah, our config-complete language has to be whitespace-significant too | 20:17 |
| *** andreaf_ has quit IRC | 20:19 | |
| *** andreaf_ has joined #tripleo | 20:19 | |
| *** sballe_ has joined #tripleo | 20:19 | |
| *** ifarkas has quit IRC | 20:19 | |
| *** andreaf has quit IRC | 20:20 | |
| *** jistr has quit IRC | 20:20 | |
| *** sballe has quit IRC | 20:23 | |
| tchaypo | s/config-complete/turing-complete/ | 20:24 |
| *** TravT has quit IRC | 20:25 | |
| *** chuckC has quit IRC | 20:27 | |
| *** julim has quit IRC | 20:30 | |
| *** nekron99 has quit IRC | 20:32 | |
| jprovazn | SpamapS, greghaynes: hi, to make sure I'm not duplicating work, anyone of you is looking at makng control node scalable when generating overcloud heat template? | 20:36 |
| tchaypo | i think greghaynes isworking on something like hat | 20:36 |
| greghaynes | I was just about to get started but the more the merrier | 20:37 |
| greghaynes | Right now it seems like we need to do the VIP's for each service before that will work | 20:37 |
| jprovazn | greghaynes, well, same VIP can be used by multiple services | 20:39 |
| greghaynes | Yep, actually had that chat with lifeless yesterday - was planning on just one vip for all the services for nows | 20:40 |
| jprovazn | greghaynes, I don't expect it to be usable now, but I think it would make testing of HA work easier if everybody could simply deploy >1 control node | 20:40 |
| tchaypo | +1 | 20:40 |
| greghaynes | Yep | 20:40 |
| greghaynes | I *think* thats really the only thing blocking merge.py 'just working' | 20:41 |
| tchaypo | if we can roll out >1 control node we can find the bits that are missing/broke and start hacking on them | 20:41 |
| jprovazn | greghaynes, great, it's EOD for me anyway. I can ping you tomorrow and check with you if I help is needed on this | 20:47 |
| *** hashar has quit IRC | 20:47 | |
| jprovazn | greghaynes, or just pick something else | 20:47 |
| jprovazn | s/pick/I'll pick/ :) | 20:48 |
| *** e0ne has quit IRC | 20:50 | |
| *** TravT has joined #tripleo | 20:51 | |
| greghaynes | jprovazn: Ill see how far I get today, and ill try to at a minimum get a WIP up for you by your tomorrow morning | 20:54 |
| *** TravT has quit IRC | 20:55 | |
| *** TravT has joined #tripleo | 20:55 | |
| jprovazn | greghaynes, cool | 20:58 |
| *** nati_ueno has quit IRC | 20:59 | |
| *** epim has quit IRC | 21:00 | |
| *** nati_ueno has joined #tripleo | 21:00 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:03 | |
| *** jdob has quit IRC | 21:04 | |
| *** jprovazn has quit IRC | 21:10 | |
| *** jtomasek has quit IRC | 21:12 | |
| vinsh | greghaynes, got 3 control nodes up here.. but the 2 nova nodes failed to boot :-/ | 21:16 |
| vinsh | this isn't a flexible method.. just a long section duplicated overcloud-source.yaml. | 21:17 |
| vinsh | having issues getting the rabbits to cluster. | 21:17 |
| vinsh | hm. well on this run I got 2 computes a 1/3 controls up. Random. | 21:17 |
| vinsh | ugh. this thing again "++ os-apply-config --key db-password | 21:19 |
| vinsh | [2014/04/22 09:18:22 PM] [ERROR] key db-password does not exist in ['/var/lib/os-collect-config/heat_local.json', '/var/lib/os-collect-config/ec2.json', '/var/lib/os-collect-config/cfn.json'] | 21:19 |
| vinsh | + db_pass= | 21:19 |
| vinsh | " | 21:19 |
| lifeless | vinsh: failed to get metadata | 21:19 |
| lifeless | vinsh: this is (IMO) a bug in os-collect-config - it fails open rather than closed. | 21:19 |
| vinsh | Yeah.. I told devtest_overcloud to wait for 5 nodes.. "computescale+3" but that never works. | 21:20 |
| lifeless | vinsh: but it might be a design limitation rather than a bug - just changing it could lead to permanently wedged machines | 21:20 |
| vinsh | so I have to leave it at compute+1.. and I guess it doesn't inject that data in time? | 21:20 |
| lifeless | vinsh: not sure what the issue triggering this is for you, but we don't inject any data, its all pull-based | 21:20 |
| vinsh | oh... so it means whatever is in my template isn't complete enough to provide that info | 21:21 |
| lifeless | likely, yes | 21:21 |
| vinsh | yet.. there it is " db-password: unset" | 21:22 |
| lifeless | on the box | 21:23 |
| lifeless | run os-collect-config --print -v | 21:23 |
| SpamapS | lifeless: I agree on the failing open | 21:23 |
| vinsh | lifeless, thanks! thats the command I was brainfarting. | 21:23 |
| SpamapS | lifeless: we need a good way to notify about failure though, since it is the way we configure signalling. | 21:24 |
| lifeless | SpamapS: and bootstrap and so on | 21:25 |
| lifeless | SpamapS: the thing that leaps to mind for me is a different orc script dir for bootstrap | 21:25 |
| lifeless | SpamapS: vs 'all sources up' | 21:25 |
| SpamapS | lifeless: I've thought about that too. | 21:25 |
| lifeless | but I haven't thought deeply about this at all | 21:25 |
| lifeless | so its probably bong | 21:26 |
| SpamapS | lifeless: no I think we should manage a limited subset of things that need to happen before everything else happens. | 21:26 |
| SpamapS | lifeless: It could work as symlinks to the regular orc dir | 21:26 |
| *** absubram has quit IRC | 21:26 | |
| SpamapS | as in, bootstrap/pre-configure.d/##-something -> /opt/stack/os-config-refresh/pre-configure.d/##-something | 21:27 |
| SpamapS | lifeless: that would also solve the signalling problem as we could have special bootstrappy signalling instructions in the bootstrap error.d | 21:29 |
| SpamapS | assuming https://review.openstack.org/#/c/83634/ ever lands | 21:29 |
| SpamapS | and with that, I remember that I'm supposed to be investigating downed CI machines | 21:30 |
| *** e0ne has joined #tripleo | 21:30 | |
| *** jang1 has quit IRC | 21:31 | |
| lifeless | SpamapS: https://review.openstack.org/#/c/83634/2//COMMIT_MSG | 21:33 |
| SpamapS | lovely | 21:33 |
| lifeless | SpamapS: I did note it in patch set 1 for you :) | 21:34 |
| lifeless | SpamapS: I also +2'd :0 | 21:34 |
| *** e0ne has quit IRC | 21:34 | |
| SpamapS | yeah mega doh | 21:34 |
| SpamapS | novacompute8 looks frozen | 21:34 |
| SpamapS | two different boxen like this now | 21:35 |
| *** dprince has quit IRC | 21:35 | |
| SpamapS | lifeless: I think we need some kind of incident log. Any ideas on how we should do that? | 21:35 |
| *** adam_g has left #tripleo | 21:35 | |
| SpamapS | Back in the day, I used to use nagios for that. | 21:35 |
| SpamapS | Since you could do passive checks and we'd have a "KNOWN_OUTTAGE" check which we'd twiddle off and on with information about why. | 21:36 |
| *** weshay has quit IRC | 21:36 | |
| SpamapS | quite helpful for mining for "oh look every tuesday box 7 gets hot and dies" | 21:37 |
| SpamapS | resetting novacompute8 | 21:40 |
| lifeless | SpamapS: journal items in the etherpad for now? | 21:41 |
| lifeless | DATE MACHINE ISSUE | 21:42 |
| SpamapS | the etherpad? | 21:46 |
| SpamapS | we have so many.. :P | 21:47 |
| lifeless | the one for outages | 21:47 |
| openstackgerrit | Chris Krelle proposed a change to openstack/diskimage-builder: update opensuse element to have correct BASE_IMAGE_NAME https://review.openstack.org/89703 | 21:47 |
| lifeless | SpamapS: https://etherpad.openstack.org/p/cloud-outage | 21:47 |
| SpamapS | the one not prefixed with tripleo .. of course | 21:48 |
| lifeless | don't blame me :P | 21:48 |
| SpamapS | I blame the huns | 21:50 |
| vinsh | Hey.. where are heat errors logged.. say I run devtest_overcloud .. and nodes go right to "ERROR" | 21:52 |
| vinsh | where do I dig in to find out why? | 21:52 |
| lifeless | heat event-list overcloud | 21:53 |
| SpamapS | vinsh: nova show <name|uuid> | 21:53 |
| lifeless | then heat event-show overcloud <event-id> | 21:53 |
| SpamapS | oh right first might want to read event-list | 21:53 |
| vinsh | ah-hah "| fault | {"message": "No valid host was found. ", "code": 500, "details": " File \"/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py\", line 108, in schedule_run_instance |" | 21:53 |
| vinsh | i'm doing things that confuse it.. trying to get multiple control nodes up :) | 21:53 |
| SpamapS | That means it didn't have enough. | 21:54 |
| lifeless | vinsh: before you start | 21:54 |
| lifeless | vinsh: check nova hypervisor-stats | 21:54 |
| vinsh | yes.. I was checking that.. only has 3 nodes | 21:54 |
| lifeless | vinsh: hypervisor count - running vms == available nodes | 21:54 |
| vinsh | ohhh OK | 21:55 |
| SpamapS | well crap.. have to go to the DMV to renew my driver's license | 21:55 |
| SpamapS | damn procrastination | 21:55 |
| vinsh | SO. whats the trick to up that count? which script sets that up? | 21:55 |
| * SpamapS will be back in a few.. annoyed most likely.. ;) | 21:55 | |
| vinsh | :) | 21:55 |
| lifeless | vinsh: setup-baremetal / register-nodes | 21:55 |
| vinsh | OK.. so I bet that devtest_undercloud calls that out.. and the fact that I am trying to brew more nodes after the fact means I need to back track and tweak that. | 21:57 |
| vinsh | Thanky'all. | 21:57 |
| vinsh | I can't tell heat to bring up 3 control nodes.. if it only has 1 to work with :) | 21:58 |
| vinsh | I didn't realize in which phase of this that was done till now. SO much to learn still. | 21:58 |
| lifeless | you could scale the hypervisor count to 0 | 21:58 |
| lifeless | I don't know if that would work | 21:58 |
| lifeless | but in principle ... :) | 21:58 |
| *** sballe_ has quit IRC | 21:59 | |
| vinsh | i'l give it more G's 4G 5G | 21:59 |
| vinsh | up the G's. | 21:59 |
| vinsh | higher octane. done. | 21:59 |
| *** akrivoka has quit IRC | 22:01 | |
| greghaynes | I wonder if youre hitting the disk space bug I ran into | 22:01 |
| greghaynes | I bet you are, actually | 22:01 |
| lifeless | SpamapS: ok, do you need help with the cloud ? | 22:02 |
| lifeless | I'm finally out of my morning meeting hell | 22:02 |
| vinsh | greghaynes, ah, hence your patches for "slightly" more space. | 22:02 |
| lifeless | clarkb: speaking of; how goeth the region bringup ? | 22:02 |
| greghaynes | I mean, youre first hitting the lack of bm nodes, but secondly youre going to hit the space bug | 22:02 |
| clarkb | lifeless: got distracted by gerrit stuff, about to approve the nodepool config change now | 22:03 |
| greghaynes | vinsh: yes. That one shows as scheduler couldnt find node even though one is available and then if you look in nova logs youll see an out of space error | 22:03 |
| lifeless | greghaynes: vinsh: oh; use ironic. | 22:03 |
| *** nati_ueno has quit IRC | 22:04 | |
| * vinsh applies ironic. its super metal-y. | 22:05 | |
| greghaynes | oh, ironic doesnt do the iscsi + dd deal? | 22:06 |
| vinsh | https://s3.amazonaws.com/uploads.hipchat.com/8522/39755/3pTwqnmkI7NNg1V/APT.jpg | 22:06 |
| lifeless | greghaynes: it does, but same image for all nodes | 22:06 |
| greghaynes | ah. That would help | 22:06 |
| lifeless | greghaynes: indeedily | 22:06 |
| *** sseago has quit IRC | 22:10 | |
| vinsh | greghaynes, when you have a minute or 2.. could you translate that for me? | 22:11 |
| vinsh | Id like to understand what he means :) | 22:11 |
| greghaynes | in devtest_variables therse a USE_IRONIC var. About the dd thing, we image bm nodes by mounting their drive over iscsi and dd'ing an image onto them | 22:12 |
| greghaynes | well, not just bm nodes | 22:13 |
| *** derekh has joined #tripleo | 22:13 | |
| vinsh | Yeah, I got that part.. have watched dd progress even | 22:13 |
| greghaynes | cool. So currently we make a copy of the image per node when we are deploying from undercloud | 22:14 |
| greghaynes | but theres really no reason to do that | 22:14 |
| vinsh | since its same image for all 3 control nodes right? | 22:15 |
| greghaynes | yep | 22:15 |
| *** epim has joined #tripleo | 22:20 | |
| *** epim has quit IRC | 22:20 | |
| *** sseago has joined #tripleo | 22:22 | |
| *** markmc has quit IRC | 22:23 | |
| *** nati_ueno has joined #tripleo | 22:24 | |
| clarkb | images just started building in red hat cloud | 22:25 |
| vinsh | greghaynes, so we would want to set "OVERCLOUD_DIB_EXTRA_ARGS="$OVERCLOUD_DIB_EXTRA_ARGS nova-ironic" | 22:26 |
| vinsh | vs -baremetal ? | 22:26 |
| vinsh | oh nevermind | 22:26 |
| vinsh | I see the flag. | 22:27 |
| greghaynes | theres a bit more | 22:27 |
| greghaynes | yep, exactly | 22:27 |
| *** andreaf_ has quit IRC | 22:29 | |
| lifeless | vinsh: yeah redo the undercloud | 22:31 |
| lifeless | vinsh: and the deploy ramdisk | 22:31 |
| vinsh | It looks like I have to add more entries to "testenv.sh" for it to provide more nodes to deploy to? | 22:32 |
| lifeless | vinsh: devtest_testenv can do that for you | 22:32 |
| lifeless | vinsh: I don't think we've got it fully automated to do additive stuff though, so some assembly may be required | 22:32 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-incubator: Add OVERCLOUD_CONTROLSCALE support https://review.openstack.org/83296 | 22:38 |
| greghaynes | relevant patch ^ | 22:38 |
| derekh | clarkb: cool, thanks | 22:43 |
| *** nati_ueno has quit IRC | 22:43 | |
| *** newell has quit IRC | 22:50 | |
| *** morazi has quit IRC | 22:50 | |
| *** epim has joined #tripleo | 22:50 | |
| davidlenwell | happy birthday SpamapS | 22:54 |
| devananda | lifeless: hi! do you have a fairly recent tripleo deck that I can pilfer some slides from? :-D | 22:56 |
| vinsh | lifeless, do you have an example of a nodes list to feed devtest_testenv? | 22:57 |
| *** lucas-afk has quit IRC | 22:57 | |
| vinsh | It seems to only start with the entry for the seed vm.. then grows during the run | 22:57 |
| lifeless | vinsh: nodes doesn't include the seed vm at all | 22:57 |
| lifeless | vinsh: devtest_testenv has the code to create vm based nodes, and sets up the list for you | 22:58 |
| lifeless | devananda: LCA? | 22:58 |
| lifeless | devananda: but that wasn't really tripleo specific; whats the topic ? | 22:58 |
| vinsh | lifeless, Ok, so how do I tell it to create more nodes? | 22:58 |
| devananda | lifeless: openstack bare metal things. it's a ~25min overview at linuxTAG | 22:58 |
| greghaynes | vinsh: check out that patch not far up in scrollback | 22:59 |
| greghaynes | vinsh: specifically the edit in _testenv | 22:59 |
| vinsh | ACK. doing so now. | 22:59 |
| greghaynes | it might be enlightening on what it does :) | 22:59 |
| vinsh | a-hah. light bulb goes on. | 23:00 |
| *** nati_ueno has joined #tripleo | 23:02 | |
| clarkb | derekh: red hat cloud seems to be doing things with new images | 23:03 |
| clarkb | f20 images are already being used and precise nodes should show up momentarily | 23:03 |
| lifeless | \o/ | 23:03 |
| derekh | clarkb: woot, :-) | 23:04 |
| derekh | clarkb: I see them being created havn't seen any getting used in the zuul queue yet though ( I think) | 23:05 |
| derekh | precise nodes being created now | 23:07 |
| bnemec | -2 all the things! | 23:08 |
| bnemec | That should help our review stats a bit. :-)\ | 23:08 |
| greghaynes | suddenly we have the lowest collective approval stats | 23:08 |
| lifeless | HATERS UNITE | 23:09 |
| derekh | clarkb: looks like the jobs are failing as soon as they start logs.openstack.org/57/84657/3/check-tripleo/check-tripleo-overcloud-f20/084e46f/console.html | 23:10 |
| derekh | brb | 23:10 |
| clarkb | looks like the remote end is killing the ssh connection? | 23:10 |
| clarkb | derekh: seems like this one https://jenkins05.openstack.org/job/check-tripleo-overcloud-precise/491/console is working | 23:12 |
| clarkb | so not all of them are dying | 23:12 |
| clarkb | derekh: is it possible that fail you have there is related to a job cancellation? | 23:12 |
| clarkb | derekh: looking at that change I don't see it reporting to gerrit | 23:12 |
| derekh | clarkb: ahh ok, possibly | 23:15 |
| clarkb | I think we are ok | 23:15 |
| derekh | clarkb: yup, I've found others that are running | 23:17 |
| lifeless | SpamapS: ping | 23:18 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Correct hard-coded IP ranges https://review.openstack.org/86308 | 23:19 |
| *** eguz has quit IRC | 23:36 | |
| *** eghobo has joined #tripleo | 23:36 | |
| SpamapS | lifeless: ok back | 23:41 |
| vinsh | greghaynes, 3 control nodes up now. nice! | 23:42 |
| vinsh | rabbit.. not well though | 23:42 |
| vinsh | DIAGNOSTICS | 23:42 |
| vinsh | =========== | 23:42 |
| vinsh | nodes in question: ['rabbit@overcloud-controller1-q4fqxcrhepw3,overcloud-controller2-56gybpwerze3'] | 23:42 |
| vinsh | hosts, their running nodes and ports: | 23:42 |
| vinsh | - unable to connect to epmd on overcloud-controller1-q4fqxcrhepw3,overcloud-controller2-56gybpwerze3: nxdomain (non-existing domain) | 23:42 |
| vinsh | probably because those are some bogus names... can't resolve them maybe. | 23:42 |
| clarkb | that is exactly what I would expect nxdomain to imply | 23:43 |
| vinsh | current node details: | 23:44 |
| vinsh | - node name: 'rabbitmqctl18084@overcloud-controller0-h7elsdlhfkdq' | 23:44 |
| vinsh | - home dir: /mnt/state/var/lib/rabbitmq | 23:44 |
| vinsh | - cookie hash: SPE7wCKCwmg57PQAlN3GQQ== | 23:44 |
| vinsh | strange names anyway on itself. | 23:44 |
| SpamapS | lifeless: so it looks like what may be happening is we get network blips.. and that makes openvswitch/gre freak out. Restarting openvswitch-switch seems to fix things. | 23:44 |
| SpamapS | lifeless: hard to know without good monitoring | 23:44 |
| SpamapS | # ls -l /var/log/upstart|wc -l | 23:46 |
| SpamapS | 84913 | 23:46 |
| SpamapS | thats also.. concerning | 23:47 |
| SpamapS | # cat /var/log/upstart/dhcp-all-interfaces-qvo058d5d26-a5.log | 23:47 |
| SpamapS | Inspecting interface: qvo058d5d26-a5...Device has generated MAC, skipping. | 23:47 |
| SpamapS | We may want to suppress that | 23:47 |
| *** CaptTofu has quit IRC | 23:51 | |
| *** CaptTofu has joined #tripleo | 23:51 | |
| greghaynes | vinsh: awesome! | 23:51 |
| vinsh | -ish ;) | 23:52 |
| *** CaptTofu has quit IRC | 23:56 | |
| lifeless | SpamapS: so restarting ovs on the compute nodes or control plane? | 23:59 |
| SpamapS | lifeless: compute nodes | 23:59 |
| lifeless | SpamapS: I suspect mellanox then | 23:59 |
| SpamapS | lifeless: the symptom shows as 0 traffic on br-tun | 23:59 |
| SpamapS | lifeless: agreed | 23:59 |
| SpamapS | lifeless: so.. options.. 1) manually upgrade mellanox on all of them | 23:59 |
| SpamapS | 2) deploy trusty | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
