Friday, 2014-01-17

SpamapS	right I get that is why it is broken	00:00
SpamapS	but I'm suggesting that we have a nice data store right there that is distributed and related to what we're doing... :-P	00:01
* SpamapS curses 8 minute posts		00:01
lifeless	SpamapS: I agree	00:03
lifeless	SpamapS: fast and loose on patches	00:03
lifeless	now I've popped my stack, can you assplain ?	00:03
SpamapS	lifeless: well in a CD environment, you want to be as close to upstream as possible, but sometimes upstream just won't take your patches, or you have blocking things that don't let you land them in time...	00:04
SpamapS	lifeless: you would generally have a buffer for those patches, where you can track the delta and manage it. We are just cherry picking and praying a bit.	00:05
lifeless	SpamapS: yes, we should be cd-testing what we deploy	00:06
lifeless	SpamapS: which implies cd deploying undercloud	00:06
lifeless	SpamapS: I think your point is entirely valid but behind a dependency chain...	00:06
lifeless	that I'm not sure how to brea through	00:06
SpamapS	yeah its really damn long too.. :-P	00:06
SpamapS	lifeless: I'm calling out what makes me worry. And a problem that worries me, behind a long dependency chain... perhaps means it deserves attention?	00:07
SpamapS	Or I need fears allayed. :-P	00:07
*** vipul-away is now known as vipul		00:08
*** CaptTofu has joined #tripleo		00:09
SpamapS	2014-01-17 00:15:13.349 3355 WARNING keystone.openstack.common.versionutils [-] Deprecated: v2 API is deprecated as of Icehouse in favor of v3 API and may be removed in K.	00:12
*** cd-undercloud has joined #tripleo		00:12
cd-undercloud	************ overcloud complete status=1 **********	00:12
*** cd-undercloud has quit IRC		00:12
SpamapS	lots of warning spewing	00:12
SpamapS	tripleo-cd stopped to debug	00:13
StevenK	rsync: mkstemp "/var/lib/libvirt/images/.seed.qcow2.Gb4RR4" failed: Permission denied (13)	00:13
StevenK	rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1070) [sender=3.0.9]	00:13
SpamapS	ah	00:13
SpamapS	lifeless: ok so same problem.. db was initialized again	00:13
SpamapS	lifeless: BUT.. my patch to allow update to wait on the waitconditions works so thats nice. :)	00:14
SpamapS	lifeless: are you going to work on the patch to stop the database-reinitialized-over-and-over problems?	00:15
openstackgerrit	lifeless proposed a change to openstack/tripleo-image-elements: Preserve 'did we init openstack' state file. https://review.openstack.org/67339	00:16
lifeless	SpamapS: no	00:16
lifeless	:P	00:16
SpamapS	lifeless: ok, I'm happy to	00:16
lifeless	SpamapS: that was irony	00:17
SpamapS	lifeless: :)	00:17
lifeless	https://review.openstack.org/67339 <- is said patch	00:17
lifeless	written but not pushed 20m ago	00:17
SpamapS	:)	00:18
lifeless	you can include multiline strings in yaml, right?	00:18
SpamapS	async comms: not for the faint of heart	00:18
SpamapS	lifeless: definitely	00:18
SpamapS	lifeless: only question is whether you want the newlines embedded (use >) or not (use \|)	00:19
lifeless	SpamapS: if i have a file on disk	00:20
lifeless	and I need to pass its contents in as a heat parameter	00:20
lifeless	say an SSL certificate	00:20
lifeless	...	00:20
SpamapS	actually	00:20
SpamapS	\| if you want ot keep the newlines	00:20
SpamapS	> if you want them removed	00:20
SpamapS	I always forget	00:20
SpamapS	lifeless: $() for now :(	00:21
SpamapS	lifeless: I think steve baker is working on some magic for that	00:21
SpamapS	lifeless: OR, you could make it a nested stack	00:21
SpamapS	lifeless: OR, you could upload said file to a secure location and wget it from inside your instances..	00:21
SpamapS	lifeless: note that secrets should never be passed in _server_ metadata (ec2 metadata service will share them with whomever wants them on the box) .. they can however be done in a separate LaunchConfiguration safely.	00:22
SpamapS	tho when we decide it is time to harden, we should probably lock down ec2 metadata	00:23
*** vipul is now known as vipul-away		00:24
lifeless	SpamapS: if anyone is on the box	00:24
lifeless	SpamapS: we're fucked	00:24
lifeless	SpamapS: though I agree in principle	00:24
lifeless	SpamapS: but I'll note that if you're on the box, you can read metadata to bootstrap up to full access	00:24
lifeless	SpamapS: so you're fucked	00:24
SpamapS	lifeless: right but hardening == reducing it to "we're tickled a lot" right?	00:25
SpamapS	read which metadata?	00:25
*** jhurlbert has quit IRC		00:25
SpamapS	Assuming non-root here.. local root is definitely "oh noes" :)	00:25
lifeless	hmm	00:26
lifeless	so I now have a meta-template problem	00:26
lifeless	SpamapS: non-root can read the ec2 metadata	00:26
lifeless	SpamapS: ec2 metadata lets you read heat metadata	00:26
lifeless	SpamapS: I want to write one or more files with names determined by a key in the heat metadata	00:27
lifeless	SpamapS: in principle that is - this is stunnel4	00:27
lifeless	I'm going to make it less generic for now	00:27
SpamapS	lifeless: right so we should be shutting down ec2 metadata as soon as we have read it.	00:30
SpamapS	lifeless: null routing it I mean. cloud-init will do this for us actually.	00:30
lifeless	SpamapS: mmmm, I am really not convinced	00:30
SpamapS	a files with keys from metadata.. we have a bug for that in oac	00:31
lifeless	SpamapS: and I don't want to rathole in this right now	00:31
* SpamapS boards up the rat hole		00:31
SpamapS	lifeless: ok so please do lend some feedback to the design discussed in https://bugs.launchpad.net/os-apply-config/+bug/1258351	00:32
uvirtbot	Launchpad bug 1258351 in os-apply-config "Allow meta templates which can generate moustache-derived filenames" [Wishlist,In progress]	00:32
lifeless	SpamapS: remind me why heat needs it's own SSL ?	00:33
SpamapS	lifeless: because https://review.openstack.org/#/c/64142/ has not landed yet, which fixes https://launchpad.net/bugs/1235555	00:34
uvirtbot	Launchpad bug 1235555 in heat "Heat API cannot cope with being behind an SSL terminator" [High,In progress]	00:34
*** vipul-away is now known as vipul		00:34
SpamapS	though frankly.. stunnel is just a temporary solution right?	00:34
lifeless	SpamapS: why do you say that?	00:34
lifeless	SpamapS: separately, why do you want use-ephemeral (plumbing) to depend on ssh-ephemeral (specific service)	00:35
SpamapS	lifeless: ssh: because we always want SSH, and that will make sure the ssh-ephemeral bits always get pulled in when use-ephemeral gets pulled in.	00:36
SpamapS	I just don't see us ever building with, say, mysql, but then not wanting ssh-ephemeral	00:38
lifeless	SpamapS: so I think a better thing to do would be to put ssh-ephemeral as a dep of base	00:38
lifeless	SpamapS: for instance	00:38
SpamapS	stunnel I'm asking because stunnel is not very efficient in my experience (it works, but it forks a lot IIRC)	00:38
lifeless	ok, so I am not attached to stunnel	00:38
lifeless	I just don't want by-hand.	00:38
SpamapS	Have had this recommended to me by DreamHost peeps: https://github.com/bumptech/stud	00:38
SpamapS	lifeless: base is in dib.. that doesn't seem right :)	00:39
lifeless	isn't use-ephemeral in dib too ?	00:39
SpamapS	no	00:39
SpamapS	its our own tripleo crazy sauce	00:39
openstackgerrit	Ryan Brady proposed a change to openstack/tripleo-image-elements: Updates boot-stack iptables file https://review.openstack.org/66571	00:39
SpamapS	isn't it?	00:39
SpamapS	yeah	00:39
lifeless	ah yes	00:40
lifeless	hmm	00:40
lifeless	so	00:40
lifeless	I think it will be confusing to have an element that is depended on by the thing it uses	00:40
lifeless	rather than the reverse	00:40
SpamapS	and making everybody who uses diskimage-builder go /mnt crazy is heavy handed	00:40
lifeless	sure	00:40
SpamapS	well it is really a circular dep	00:40
lifeless	its not though; you're arguing a policy dep	00:41
SpamapS	and if we were in debian land I could see it as a use-ephemeral recommending ssh-ephemeral :)	00:41
SpamapS	but does it _very_ make sense to have some things using ephemeral, but not SSH? ever?	00:41
SpamapS	s/_very_/ever/	00:41
lifeless	so	00:41
lifeless	Whats unique about ssh	00:42
lifeless	its already installed	00:42
lifeless	[for some images]	00:42
lifeless	and it's in every image	00:42
lifeless	for other software I've been arguing that we don't need sometimes=/mnt sometimes /	00:42
lifeless	but rather that saying /mnt always is fine	00:42
*** michchap_ has quit IRC		00:43
*** michchap has joined #tripleo		00:43
SpamapS	Right, use-ephemeral is only for "sometimes" ...	00:44
SpamapS	so I can see it working this way too.	00:44
SpamapS	this way meaning the way the patch has it now	00:45
lifeless	what do you mean sometimes? It's unconditional for us :)	00:45
SpamapS	I'm just wary of having to remember to add 'ssh-ephemeral'	00:45
lifeless	so I've done a review tehre	00:45
lifeless	what about just doing this in use-ephemeral	00:45
lifeless	if its not an optional extra, why is it extra	00:46
SpamapS	its just separation of concerns at that point	00:46
lifeless	separate scripts ;)	00:47
openstackgerrit	lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343	00:50
*** CaptTofu has quit IRC		00:53
StevenK	Can someone show me ls -ld /var/lib/libvirt/images on an Ubuntu machine, I want to check if my perms are wrong	00:55
SpamapS	lifeless: hm.. do we really want to be passing private SSL keys in	00:55
SpamapS	drwx--x--x 2 root root 4096 Jan 6 15:59 /var/lib/libvirt/images	00:55
SpamapS	StevenK: ^	00:55
StevenK	Hmph	00:55
StevenK	I'm getting a EPERM from rsync during devtest.sh	00:56
SpamapS	StevenK: hm.. that looks like a special case where $HOST_IP is set..	00:59
SpamapS	StevenK: otherwise sudo is used to cp rather than rsync	01:00
SpamapS	I have not run devtest entirely on my machine since you guys exploded it thouigh	01:00
lifeless	assplode!	01:00
lifeless	SpamapS: we really want ssl to just work	01:01
lifeless	SpamapS: we pass a tonne of other secrets in, why are these ones special?	01:01
SpamapS	lifeless: its a twitch muscle thing... may not have merit :)	01:02
* StevenK drops the JSON cache		01:02
StevenK	Let's see	01:02
SpamapS	lifeless: also the other secrets grant access and can be revoked on compromise. keys protect comms and the traffic they protected cannot be reclaimed and re-encrypted.	01:03
SpamapS	lifeless: but I totally acknowledge that to do it "right" requires {{pain}}	01:04
lifeless	SpamapS: I don't see how thats a semantic difference. While the secret is compromised and not detected data is disclosed	01:04
lifeless	when the compromise is detected, the secret is replaced, and no further data is disclosed.	01:04
SpamapS	lifeless: you cannot detect somebody using your key to decrypt your traffic. You can detect somebody using your password to login.	01:05
SpamapS	They're not oceans apart.	01:05
SpamapS	But they are two different seas of fail. :)	01:05
*** CaptTofu has joined #tripleo		01:05
lifeless	mmm	01:05
lifeless	I could quibble on both your assertions there.	01:05
SpamapS	But don't think i'm adamant. I just want to think it through.	01:05
lifeless	detecting hacked logins is nontrivial	01:06
lifeless	it's serious ML territory	01:06
lifeless	witness that google's best visible effort so far on that is to expose last-site to users of gmail	01:06
SpamapS	So I've been through compromises where passwords were breached and the way the compromise was confirmed was that the passwords were being used from !(the clients' office).	01:07
SpamapS	I have no idea if my SSL keys have ever been hacked... and I don't know how i would be able to know. :-P	01:07
SpamapS	Anyway I need bigger stronger boards on that rat-hole. ;)	01:08
SpamapS	My point is not to block, but to talk. :)	01:08
SpamapS	I think encryption keys are generally held as more important to keep private than passwords.	01:09
SpamapS	I am willing to capitulate that to spend time keeping them more secret would be complex.	01:09
lifeless	so we could scp them up	01:10
lifeless	but that makes the admin console a SPOF for them	01:10
lifeless	and requires a guarded machine that admins log into	01:10
SpamapS	Or investigate barbican, which is designed to protect such things.	01:10
lifeless	which is heat, no? :)	01:10
lifeless	barbican isn't incubated yet is it?	01:11
lifeless	I remember angst about that just recently	01:11
SpamapS	IIRC Barbican specifically only ever lets you see a key when an admin has authorized it.. the theory being that a machine which consumes the key does not have the ability to control _when_ it consumes the key.	01:11
SpamapS	But that is rat-holey too	01:11
SpamapS	lifeless: I think having the key generated on one box, and scp'd from it to any others that need it, would work....	01:13
SpamapS	lifeless: the second process of CSR->CA signing is somewhat unrelated.	01:13
SpamapS	of course now the SCP calls into question how they exchange SSH keys and we go WAY back to a concern I had a year ago about why there still isn't a thing which makes sure OpenStack nodes expose their host keys securely.	01:14
lifeless	right	01:14
lifeless	this shit is HARD	01:14
lifeless	I think this is a 'get something and iterate' situation	01:14
SpamapS	lifeless: ok I relinquish my -1 on the condition that there be a bug to tackle secure key distribution.	01:14
lifeless	SpamapS: can you file it :)	01:15
SpamapS	The recent angst over Barbican was mostly that it was largely a Rackspace driven project.	01:15
SpamapS	We could remove _that_ angst by contributing. :)	01:15
lifeless	no, that wasn't it	01:15
SpamapS	lifeless: yeah I'm filing it.	01:15
lifeless	it was about release times etc	01:15
SpamapS	well the other angst was they used celery	01:15
*** tserong_ is now known as tserong		01:16
SpamapS	lifeless: https://bugs.launchpad.net/tripleo/+bug/1269999	01:18
uvirtbot	Launchpad bug 1269999 in tripleo "Provide a way to distribute keys securely" [Medium,Triaged]	01:18
SpamapS	lifeless: I will not -1 that part again :) now.. fix your yaml :)	01:18
openstackgerrit	lifeless proposed a change to openstack/tripleo-heat-templates: Add SSL configuration metadata for overclouds. https://review.openstack.org/67348	01:19
lifeless	SpamapS: but I mades it looks pretty	01:19
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Make it possible to customise overcloud names. https://review.openstack.org/66467	01:21
openstackgerrit	lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343	01:24
SpamapS	lifeless: its still wrong....	01:26
lifeless	SpamapS: merge.py likes it	01:27
SpamapS	lifeless: its right in tripleo-heat-templates .. but not in README.md	01:28
lifeless	SpamapS: I don't understand, whats wrong	01:28
lifeless	SpamapS: the quotes?	01:28
SpamapS	lifeless: this is why I wrote that tool that I will some day write tests for btw.. ;)	01:28
SpamapS	lifeless: I wrote an example	01:29
lifeless	ah, got it	01:29
openstackgerrit	lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343	01:29
*** sdake has quit IRC		01:33
lifeless	hmm	01:41
lifeless	does bash have a ternary operator?	01:41
StevenK	[[ $b = 5 ]] && a="$c" \|\| a="$d" ?	01:42
lifeless	twitch	01:42
lifeless	but yeah I guess	01:42
StevenK	So, no ?:	01:43
* StevenK pouts at devtest		01:44
StevenK	Stop using rsync, damn it	01:44
lifeless	do you have a host-ip in your json ?	01:45
StevenK	I killed the json file before this run	01:46
StevenK	And yes, I do	01:46
StevenK	The IP is virbr0	01:46
lifeless	thats why it's using rsync	01:47
lifeless	it may be that I killed the non-rsync path entirely by mistake	01:47
openstackgerrit	A change was merged to openstack/tripleo-incubator: Cleanup use of SEED_IP. https://review.openstack.org/66256	01:47
StevenK	The non-rsync path still exists	01:47
lifeless	yes but the conditions for it may never be met	01:48
StevenK	I don't really to git bisect devtest :-)	01:48
* StevenK tries to revert 03adc448 from his tree		01:50
* SpamapS tries a clean run of devtest		01:52
*** nosnos has joined #tripleo		02:04
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Permit deploying an overcloud with SSL endpoints. https://review.openstack.org/67356	02:04
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Make it possible to customise overcloud names. https://review.openstack.org/66467	02:04
lifeless	ok in theory thats a tonne of manual foo avoided	02:06
lifeless	SpamapS: I'm gonig to take over tripleocd temporarily, ok ?	02:06
SpamapS	lifeless: yes. It won't work right again until we fix the "delete the db every time" part	02:10
lifeless	SpamapS: so +2 the commit :P	02:10
SpamapS	lifeless: you going to cherry pick to glory?	02:10
*** CaptTofu has quit IRC		02:10
lifeless	SpamapS: yes	02:10
lifeless	SpamapS: incubator templates and elements	02:10
lifeless	trying to recover ci-overcloud quickly	02:10
SpamapS	ok	02:10
SpamapS	I have a local change committed in the templates at the moment..	02:11
SpamapS	need to get that in review but it will be blocked on landing OS::Heat::UpdateWaitConditionHandle in Heat	02:11
lifeless	SpamapS: oh, so I review -d'd it away already.	02:11
SpamapS	lifeless: ok, your wait_for stack-ready won't wait long enough then	02:11
lifeless	SpamapS: thats ok, I'm not running the whole thing	02:12
SpamapS	lifeless: want a -x'able review #?	02:12
*** newell has quit IRC		02:13
openstackgerrit	Clint "SpamapS" Byrum proposed a change to openstack/tripleo-heat-templates: Use OS::Heat::UpdateWaitConditionHandle https://review.openstack.org/67358	02:15
lifeless	SpamapS: let me guess; it will hand in update-in-progress?	02:15
lifeless	ah nuts, stackname patch is incomplete	02:16
SpamapS	-> family time	02:19
lifeless	https://bugs.launchpad.net/tripleo/+bug/1270012 is going to bite us when we go to more nodes	02:20
uvirtbot	Launchpad bug 1270012 in tripleo "setup-endpoints doesn't work with internal + public networks" [High,Triaged]	02:20
*** julim has joined #tripleo		02:22
*** julim has quit IRC		02:24
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Permit deploying an overcloud with SSL endpoints. https://review.openstack.org/67356	02:28
openstackgerrit	A change was merged to openstack/python-tuskarclient: Corrected the project name in a comment https://review.openstack.org/67174	02:29
lifeless	service mysql start	02:32
lifeless	df: '/var/lib/mysql/.': No such file or directory	02:32
lifeless	/etc/init/mysql.conf: LC_ALL=C BLOCKSIZE= df --portability /var/lib/mysql/. \| tail -n 1 \| awk '{ exit ($4<4096) }'	02:33
lifeless	blink	02:33
lifeless	StevenK: low hanging fruit : https://bugs.launchpad.net/tripleo/+bug/1270017	02:34
uvirtbot	Launchpad bug 1270017 in tripleo "mysql fails to start due to df of missing directory" [Critical,Triaged]	02:34
StevenK	lifeless: Still fiddling with devtest -- looks like 03adc44 is the smoking gun	02:37
lifeless	StevenK: which one was that	02:38
StevenK	"Use the test env JSON to detect remote VM hosts."	02:38
StevenK	I think we need to sanity check the value against host IPs	02:38
lifeless	hmm	02:38
lifeless	Jan 17 02:38:42 ci-overcloud-notcompute-3st54nnok5p5 kernel: [14732.306037] Buffer I/O error on device sda1, logical block 66885	02:39
lifeless	tad worrying	02:39
lifeless	StevenK: what sort of sanity do you have in mind ?	02:39
StevenK	lifeless: Check the list of local IPs, and if HOST_IP is in it, set it back to ""	02:39
lifeless	StevenK: in boot-seed-vm specifically?	02:40
StevenK	Yeah	02:41
lifeless	ok	02:41
lifeless	seems plausible	02:41
StevenK	I think that's what was tripping me up, given the host_ip in in the JSON was a local bridge interface	02:42
StevenK	Hm, now how to drop HEAD	02:44
lifeless	[14787.186531] end_request: critical target error, dev sda, sector 535081	02:50
lifeless	[14787.186565] Buffer I/O error on device sda1, logical block 66885	02:50
lifeless	I see another JIRA ticket coming up	02:50
StevenK	Haha	02:50
StevenK	Hmm, I really don't like the idea of 'ip a l \| grep inet \| tr .. \| cut ' :-(	02:51
lifeless	StevenK: if I may suggest another approach	02:52
lifeless	StevenK: edit the schema two have two separate variables	02:52
lifeless	StevenK: one for nova's use (host-ip today) and one for 'starting the seed stuff' which devtest-testenv would set to ''	02:53
lifeless	StevenK: but tripleo-ci can set to $host-ip	02:53
StevenK	lifeless: How about a bool to even perform it? tripleo-ci can set it, and devtest can have it set to false	02:54
lifeless	perform what?	02:54
StevenK	HOST_IP=$(OS_CONFIG_FILES=$TE_DATAFILE os-apply-config --key host-ip --type netaddress --key-default '')	02:55
StevenK	That	02:55
lifeless	I don't understand why a bool would be better	02:55
lifeless	either way its in the test environment descriptor, right?	02:55
lifeless	I mean, it's up to you; I'm just expressing confusion	02:56
*** AaronGr is now known as AaronGr_Zzz		02:57
* StevenK is checking where os-apply-config is coming from, to see if we can make that smarter		02:58
lifeless	oac is one of our tools	02:58
lifeless	the file its reading is the json file	02:58
lifeless	StevenK: I still don't understand what smarts you're referring to	02:58
StevenK	lifeless: Something sets host_ip in the JSON to the local interface bridge. If it stopped doing that, boot-seed-vm wouldn't need to change.	02:59
lifeless	StevenK: if it stopped doing that the undercloud would break	03:01
lifeless	StevenK: as would the seed	03:01
lifeless	StevenK: because the nova baremetal power driver in those vms uses that ip to ssh to to query and set the power state of the other vms	03:01
lifeless	StevenK: so that ip has to be there.	03:02
lifeless	StevenK: we can either do a heuristic to avoid the rsync; or signal explicitly somehow.	03:02
StevenK	The check we do so far is if it's set, rsync to it, assuming that the user can write to /var/lib/libvirt/images, otherwise use sudo cp	03:03
lifeless	right, that check is clearly wrong	03:03
StevenK	Ah ha.	03:04
StevenK	You didn't change that bit of boot-seed-vm	03:04
StevenK	lifeless: So now that we both understand the problem, what do you think we should do?	03:04
lifeless	StevenK: I think we should signal somehow to boot-seed-vm whether we expect to be cping or rsyncing	03:05
lifeless	StevenK: one way is to use a different JSON key to say where to rsync to (if we might want that to be a different IP address than the VMs use)	03:05
lifeless	- I think this is quite likely to be very useful	03:06
StevenK	Right	03:06
lifeless	StevenK: another way would be a boolean saying 'attempt rsync'	03:06
lifeless	StevenK: let me draw a quick network diagram	03:06
lifeless	StevenK: (on https://etherpad.openstack.org/p/tripleo-test-cluster)	03:07
lifeless	StevenK: at the bottom	03:07
StevenK	Right.	03:08
lifeless	dunno which you think is better	03:09
StevenK	lifeless: So we add a rsync_host JSON key, and if that's set, we do rsync to it, otherwise sudo cp	03:09
lifeless	StevenK: rsync and do the libvirt operations over ssh in boot-seed-vm	03:10
StevenK	In which case rsync_host is a bad name.	03:10
lifeless	StevenK: all those operations inside boot-seed-vm are either local or remote; I don't think we should mix and match	03:10
lifeless	[unless we have a really good reason]	03:11
StevenK	lifeless: We already do the VM operations using qemu+ssh if VM_HOST is set	03:12
lifeless	StevenK: indeed, but I think we shouldn't if we're not rsyncing	03:14
lifeless	e.g. the condition for 'should we rsync' and 'should we use virsh+ssh' is the same, no ?	03:14
StevenK	lifeless: Yes, I don't disagree. I'm just saying my choice of a JSON key of 'rsync_host' is bad and I should come up something better. Maybe a literal 'vm_host'	03:15
lifeless	StevenK: I think this is specific to boot-seed-vm	03:16
lifeless	StevenK: because after we handoff to the seed, we're now in its-an-API mode	03:16
lifeless	StevenK: thus my suggestion of boot-seed-vm-ip	03:16
StevenK	lifeless: That boolean you mention being 'Stuff should be done remotely' ?	03:20
lifeless	StevenK: yes	03:22
lifeless	I dont	03:23
lifeless	I don't have a good name for it	03:23
lifeless	just sketching the implementations I can see so far	03:23
lifeless	I have to pop out and pick up C and dinner	03:23
lifeless	StevenK: my suggestion FWIW: pick one and get it done and working and up for review; you will get feedback :)	03:28
StevenK	Heh	03:28
*** lifeless changes topic to "lifeless has the tripleo-cd lock \| tripleo-cd running preserve-ephemeral WIP patches \| Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"		03:29
lifeless	StevenK: in general if you can't decide between N implementations, take the least effort one and let reviewers find reasons why a different impl would have been better	03:29
*** ccrouch has joined #tripleo		03:40
*** ramishra has joined #tripleo		03:42
*** weshay has quit IRC		03:42
openstackgerrit	Steve Kowalik proposed a change to openstack/tripleo-incubator: Add a new remote operations flag to boot-seed-vm https://review.openstack.org/67366	03:51
SpamapS	rsync: mkstemp "/var/lib/libvirt/images/.seed.qcow2.abv9KH" failed: Permission denied (13)	03:55
SpamapS	rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1070) [sender=3.0.9]	03:55
SpamapS	real38m51.353s	03:55
SpamapS	<sigh>	03:55
StevenK	Yes, exactly.	03:56
SpamapS	hrm	03:58
SpamapS	new flag..	03:58
*** coolsvap has joined #tripleo		04:00
SpamapS	StevenK: why are we not just running with host-ip="" ?	04:00
StevenK	SpamapS: Even starting with no data file still ends up with a local bridge IP in host-ip	04:01
*** CaptTofu has joined #tripleo		04:11
*** CaptTofu has quit IRC		04:15
*** sdake has joined #tripleo		04:16
*** coolsvap_away has joined #tripleo		04:20
*** coolsvap has quit IRC		04:21
*** coolsvap_away is now known as coolsvap		04:21
*** vkozhukalov has joined #tripleo		04:21
lifeless	StevenK: reviewed ;)	04:28
lifeless	StevenK: a nuance I think I failed to communicate is that all of this needs to live in the JSON file	04:28
lifeless	StevenK: a given devtest run doesn't have any other way of knowing whether its local or remote	04:28
lifeless	StevenK: so parameters and env variables are right out	04:28
lifeless	SpamapS: because host-ip has to be set for the nova bm driver to power on the undercloud	04:29
StevenK	lifeless: However, I figured out why devtest yesterday died horribly -- ENOSPC in /var. Currently at "Waiting for the overcloud stack to be ready", so I think I'm almost done.	04:29
StevenK	/dev/mapper/sys-var 99G 32G 63G 34% /var	04:32
StevenK	(It was a 20G partition this time yesterday)	04:32
SpamapS	lifeless: ah so this goes back to the age old "is this running on one of my IP's?" problem. :)	04:33
lifeless	SpamapS: yes; see backscroll + the etherpad for analysis	04:33
openstackgerrit	lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343	04:37
lifeless	SpamapS: did you see - fucked hard disk :(	04:37
StevenK	scripts/devtest.sh --trash-my-machine 1104.21s user 284.02s system 13% cpu 2:47:46.94 total	04:39
lifeless	SpamapS: Ng: oh, I suspose it's possible the montyX machines have <<< 2TB disks; didn't think to check that	04:43
lifeless	but I would have expected dd to fail if that was the case	04:43
StevenK	So looks like I have devtest	04:44
StevenK	lifeless: I can pause the 4 VMs?	04:45
openstackgerrit	lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343	04:45
lifeless	congrats, you've now deployed OpenStack. Have a cigar.	04:45
lifeless	StevenK: yes	04:45
StevenK	lifeless: Is order important?	04:45
lifeless	StevenK: not really	04:46
StevenK	baremetal_1 is spinning with ~100CPU	04:46
lifeless	StevenK: welcome to openstack	04:46
lifeless	StevenK: its running a qemu vm inside it	04:46
lifeless	StevenK: I suspect	04:46
StevenK	Now to recover my RAM	04:47
*** morazi has quit IRC		04:51
openstackgerrit	Steve Kowalik proposed a change to openstack/tripleo-incubator: Add a new remote ops JSON value to boot-seed-vm https://review.openstack.org/67366	04:59
*** ramishra has quit IRC		05:08
openstackgerrit	Steve Kowalik proposed a change to openstack/tripleo-incubator: Add a new remote ops JSON value to boot-seed-vm https://review.openstack.org/67366	05:12
*** rpodolyaka1 has joined #tripleo		05:40
*** rushiagr has joined #tripleo		05:53
*** bauzas has joined #tripleo		05:57
*** CaptTofu has joined #tripleo		06:12
*** CaptTofu has quit IRC		06:16
*** rpodolyaka1 has quit IRC		06:31
clarkb	SpamapS: lol "virtualization"	06:33
*** ramishra has joined #tripleo		06:36
*** e0ne has joined #tripleo		06:38
*** vkozhukalov has quit IRC		06:52
*** asalkeld has joined #tripleo		06:53
asalkeld	hi, when you run image-builder what is available in to post-install.d/ ?	06:54
lifeless	asalkeld: whatever you want	06:54
asalkeld	I am trying to install some custom files via root.d	06:54
asalkeld	in /root/app/..	06:54
lifeless	asalkeld: root.d is really for the base image	06:54
lifeless	asalkeld: it runs before any of the general infra is available	06:55
asalkeld	but it's not there when I run post-install.d	06:55
asalkeld	I cd /root/app	06:55
lifeless	asalkeld: so, we could debug this, but fundamentally you're doing it wrong using root.d	06:55
lifeless	asalkeld: so lets fix that first	06:55
asalkeld	ok, cool	06:55
asalkeld	so I want a local repo (solum app)	06:56
asalkeld	basically a dircetory	06:56
asalkeld	with some user files	06:56
asalkeld	at what stage would you insert that?	06:56
lifeless	install.d	06:57
asalkeld	ok	06:57
lifeless	its for installing things	06:57
asalkeld	fail enough	06:57
asalkeld	lol	06:57
asalkeld	fair enough	06:57
*** rpodolyaka1 has joined #tripleo		06:57
asalkeld	btw: I am writing a script to convert heroku apps into tripelo elements	06:58
asalkeld	so we can build images for those apps	06:58
lifeless	root.d is for basic operations with no abstractions	07:01
lifeless	-> untarring a root fs	07:01
lifeless	pre-install is for setting up abstractions	07:01
lifeless	install is for ... installing	07:01
lifeless	post-post for dealing with gnarly things, etc	07:01
asalkeld	thanks, I am moving it into install.d	07:02
*** jprovazn has joined #tripleo		07:04
*** bauzas has quit IRC		07:04
*** mrunge has joined #tripleo		07:06
asalkeld	lifeless, http://paste.openstack.org/show/61419/	07:15
asalkeld	lifeless, http://paste.openstack.org/show/61418/	07:15
lifeless	asalkeld: cool	07:21
asalkeld	super rough	07:21
*** rpodolyaka1 has left #tripleo		07:27
*** jcoufal has joined #tripleo		07:27
SpamapS	clarkb: any time I can entertain... :)	07:34
SpamapS	asalkeld: wait you want to pull in a local dir into the chroot? extra-data.d is for that IIRC	07:35
asalkeld	ok, well I just put the files under install.d for now	07:35
SpamapS	asalkeld: ah if you can embed the files in the element then that is a simpler solution yes	07:36
lifeless	asalkeld: so yeah, to expand on SpamapS's point - you should be abel to write an element that itself directly embeds, using extra-data + install	07:40
*** rushiagr has quit IRC		07:40
lifeless	no currying involved	07:40
asalkeld	ok	07:40
asalkeld	does "install-packages" convert X-dev into X-devel on rpm distros?	07:41
lifeless	not systematically	07:45
lifeless	but it could	07:45
lifeless	it translates things we know are different and have used	07:45
asalkeld	it doesn't change ruby-dev into ruby-devel	07:46
asalkeld	(from what I saw)	07:46
SpamapS	asalkeld: in the os elements, map-packages controls that	07:48
asalkeld	k, looking	07:48
*** e0ne has quit IRC		07:48
SpamapS	asalkeld: so yeah, you have to add each mapping :-/	07:48
asalkeld	can I do that in my own element?	07:49
lifeless	asalkeld: no, not currently (and it's not as simple as just writing the code)	07:50
lifeless	SpamapS: a systematic dev -> devel might be useful	07:50
openstackgerrit	Jan Provaznik proposed a change to openstack/tripleo-image-elements: Add haproxy element https://review.openstack.org/61374	07:51
SpamapS	lifeless: better than nothing for sure	07:51
openstackgerrit	Steve Kowalik proposed a change to openstack/tripleo-incubator: Add a new remote ops JSON value to boot-seed-vm https://review.openstack.org/67366	07:55
*** erwan_ta- is now known as erwan_taf		08:04
*** bauzas has joined #tripleo		08:08
openstackgerrit	Angus Salkeld proposed a change to openstack/diskimage-builder: Convert -dev to -devel in fedora/map-packages https://review.openstack.org/67397	08:11
rpodolyaka	morning tripleo	08:11
*** ramishra has quit IRC		08:12
*** CaptTofu has joined #tripleo		08:12
*** CaptTofu has quit IRC		08:17
lifeless	SpamapS: where did the patch to use sensible modules for heat w/ cloud-init get to?	08:17
*** rdopieralski has joined #tripleo		08:19
*** jcoufal has quit IRC		08:21
lifeless	rpodolyaka: how goes nova?	08:22
rpodolyaka	lifeless: hey! I updated the series fixing the rebase glitch	08:25
rpodolyaka	lifeless: I believe, dprince already +2 one of the patches	08:26
lifeless	sweet	08:26
rpodolyaka	lifeless: looking through the jenkins failure logs now	08:26
*** vkozhukalov has joined #tripleo		08:27
rpodolyaka	lifeless: hmm, failure during rebuild. though is dated december the 18 last year (https://bugs.launchpad.net/nova/+bug/1262153). So this must not be caused by our merged patch	08:30
uvirtbot	Launchpad bug 1262153 in nova "test_rebuild_server_in_error_state spurious error" [Undecided,New]	08:30
*** jcoufal has joined #tripleo		08:31
*** rushiagr has joined #tripleo		08:31
lifeless	rpodolyaka: indeed	08:33
rpodolyaka	lifeless: ok, so I'll recheck it for now and run devtest with the updated series, and then will come back to this later	08:35
*** coolsvap has quit IRC		08:35
*** coolsvap has joined #tripleo		08:35
rpodolyaka	lifeless: I'm wondering if it's related to detaching of block devices, we accidentally found earilier	08:35
lifeless	rpodolyaka: would make sense to me :)	08:39
lifeless	shooting block devices in the head is not recommended	08:40
rpodolyaka	:)	08:40
*** e0ne has joined #tripleo		08:45
*** e0ne_ has joined #tripleo		08:46
*** jtomasek has joined #tripleo		08:47
*** e0ne has quit IRC		08:51
lifeless	and the rootwrap fail is back	08:53
*** ogelbukh has quit IRC		08:55
jprovazn	Hi, should hostname resolving work for overcloud nodes? e.g. should notcompute node be able to get compute node IP? I'm getting:	09:00
jprovazn	[root@overcloud-notcompute-qz3apzcrdxq6 ~]# ping overcloud-NovaCompute0-vhl4yeif6qee	09:00
jprovazn	ping: unknown host overcloud-NovaCompute0-vhl4yeif6qee	09:00
lifeless	jprovazn: no	09:00
lifeless	we don't have anything setup to glue nova hostnames into dns	09:01
jprovazn	lifeless, thanks	09:01
SpamapS	DNS is for wussies	09:01
SpamapS	;)	09:01
jprovazn	:)	09:02
*** vkozhukalov has quit IRC		09:03
* SpamapS sleeps		09:04
lifeless	SpamapS: orly	09:04
lifeless	SpamapS: I'll believe it when I don't see you	09:04
*** ramishra has joined #tripleo		09:06
*** bauzas has quit IRC		09:07
*** bauzas has joined #tripleo		09:07
*** jistr has joined #tripleo		09:14
*** vkozhukalov has joined #tripleo		09:18
*** rushiagr is now known as rushiagr_away		09:20
openstackgerrit	A change was merged to openstack/tripleo-heat-templates: Add optional cinder storage node to overcloud https://review.openstack.org/62735	09:20
*** markmc has joined #tripleo		09:22
*** max_lobur_afk is now known as max_lobur		09:23
*** pblaho has joined #tripleo		09:26
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Permit deploying an overcloud with SSL endpoints. https://review.openstack.org/67356	09:32
*** ProfFalken has joined #tripleo		09:35
openstackgerrit	Dirk Mueller proposed a change to openstack/python-tuskarclient: Use implicit dependencies of Hacking for pep8, pyflakes, flake8 https://review.openstack.org/67128	09:38
*** rushiagr_away is now known as rushiagr		09:39
rpodolyaka	lifeless: still around?	09:39
lifeless	rpodolyaka: hi	09:40
lifeless	rpodolyaka: yes, just wedging my keystone :)	09:40
rpodolyaka	lifeless: a quick question on tripleo-cd. I believe we are using some script to flush expired keystone tokens, are we? I can't find it in our repos. Does it only exist on the tripleo-cd undercloud machine?	09:40
lifeless	rpodolyaka: look in tripleo-image-elements/elements/keystone	09:41
rpodolyaka	lifeless: thanks! just found it too. sorry for bothering you :(	09:42
lifeless	rpodolyaka: lol np	09:47
* lifeless wonders where derekh is :)		09:47
openstackgerrit	Radomir Dopieralski proposed a change to openstack/tuskar-ui: Deployment design form https://review.openstack.org/67155	09:49
lifeless	SpamapS: you'll love this - time sync causing agent failures	09:57
lifeless	SpamapS: because the agent supplies it's own heartbeat time..	09:57
*** martyntaylor has joined #tripleo		10:00
*** martyntaylor has quit IRC		10:01
openstackgerrit	Radomir Dopieralski proposed a change to openstack/tuskar-ui: Deployment design form https://review.openstack.org/67155	10:02
*** martyntaylor has joined #tripleo		10:03
*** lifeless changes topic to "tripleo-cd running preserve-ephemeral WIP patches \| Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"		10:11
*** CaptTofu has joined #tripleo		10:13
openstackgerrit	lifeless proposed a change to openstack/tripleo-heat-templates: Expose bridge_mappings in the overcloud. https://review.openstack.org/67414	10:16
lifeless	night all	10:16
rpodolyaka	night	10:17
*** CaptTofu has quit IRC		10:18
*** pblaho has quit IRC		10:21
*** athomas has joined #tripleo		10:23
*** ramishra has quit IRC		10:37
*** vkozhukalov has quit IRC		10:46
*** nosnos has quit IRC		10:53
*** vkozhukalov has joined #tripleo		10:58
*** lucasagomes has joined #tripleo		11:00
openstackgerrit	A change was merged to openstack/tuskar: Tuskar domain model updates for wireframes https://review.openstack.org/66062	11:02
*** derekh has joined #tripleo		11:05
*** panda_ has joined #tripleo		11:07
*** martyntaylor has quit IRC		11:09
*** panda__ has quit IRC		11:10
*** d0ugal has joined #tripleo		11:13
*** athomas has quit IRC		11:17
*** athomas has joined #tripleo		11:26
*** rlandy has joined #tripleo		11:27
*** vipul is now known as vipul-away		11:31
*** rushiagr2 has joined #tripleo		11:34
*** rushiagr has quit IRC		11:34
*** rushiagr3 has joined #tripleo		11:36
*** rushiagr2 has quit IRC		11:36
*** rushiagr3 has quit IRC		11:40
*** rushiagr3 has joined #tripleo		11:40
*** rushiagr3 is now known as rushiagr_away		11:54
*** jcoufal has quit IRC		11:56
*** martyntaylor has joined #tripleo		11:57
*** vkozhukalov has quit IRC		12:00
*** asalkeld has quit IRC		12:01
*** rushiagr_away has quit IRC		12:08
*** vkozhukalov has joined #tripleo		12:12
*** pblaho has joined #tripleo		12:12
*** CaptTofu has joined #tripleo		12:14
*** rushiagr_away has joined #tripleo		12:15
*** vkozhukalov has quit IRC		12:17
*** CaptTofu has quit IRC		12:19
*** rushiagr_away has quit IRC		12:24
*** rushiagr_away has joined #tripleo		12:28
*** vkozhukalov has joined #tripleo		12:32
*** jcoufal has joined #tripleo		12:33
*** rushiagr_away has quit IRC		12:39
*** rushiagr_away has joined #tripleo		12:45
*** derekh has quit IRC		12:46
*** CaptTofu has joined #tripleo		12:55
openstackgerrit	Sascha Peilicke proposed a change to openstack/tuskar: Sync with global requirements https://review.openstack.org/67061	12:56
*** rushiagr_away is now known as rushiagr3		13:04
*** d0ugal has quit IRC		13:05
*** d0ugal has joined #tripleo		13:05
openstackgerrit	A change was merged to openstack/python-tuskarclient: Use implicit dependencies of Hacking for pep8, pyflakes, flake8 https://review.openstack.org/67128	13:08
*** coolsvap has quit IRC		13:09
*** markmc has quit IRC		13:11
*** jcoufal has quit IRC		13:21
*** jdob has joined #tripleo		13:22
openstackgerrit	Ryan Brady proposed a change to openstack/tripleo-image-elements: Updates iptables rule for MySQL element https://review.openstack.org/66569	13:27
*** rushiagr3 has quit IRC		13:28
*** mrunge has quit IRC		13:31
*** pblaho has quit IRC		13:37
*** morazi has joined #tripleo		13:37
*** morazi has quit IRC		13:37
*** pblaho has joined #tripleo		13:37
openstackgerrit	Ryan Brady proposed a change to openstack/tripleo-image-elements: Updates cinder iptables rules https://review.openstack.org/67458	13:40
*** markmc has joined #tripleo		13:40
*** jcoufal-m has joined #tripleo		13:43
*** julim has joined #tripleo		13:44
*** jcoufal-m_ has joined #tripleo		13:45
*** jcoufal-m_ has quit IRC		13:45
*** jcoufal-m_ has joined #tripleo		13:45
*** jcoufal-m has quit IRC		13:49
openstackgerrit	Ryan Brady proposed a change to openstack/tripleo-image-elements: Updates gearman iptables rule https://review.openstack.org/67460	13:49
*** jcoufal-m_ has quit IRC		13:53
*** lblanchard has joined #tripleo		13:54
*** dprince has joined #tripleo		13:55
*** markmc has quit IRC		13:56
*** markmc has joined #tripleo		13:59
*** CaptTofu has quit IRC		14:00
*** CaptTofu has joined #tripleo		14:00
*** jcoufal has joined #tripleo		14:01
*** jayg\|g0n3 is now known as jayg		14:02
*** CaptTofu has quit IRC		14:05
*** morazi has joined #tripleo		14:06
*** rushiagr3 has joined #tripleo		14:08
*** rushiagr3 is now known as rushiagr		14:09
*** spzala has joined #tripleo		14:10
openstackgerrit	Ryan Brady proposed a change to openstack/tripleo-image-elements: Adds iptables element https://review.openstack.org/67163	14:18
rbrady	jprovazn: just added a readme	14:19
jprovazn	rbrady, ah, great	14:20
*** rlandy has quit IRC		14:27
*** jtomasek has quit IRC		14:33
*** jistr has quit IRC		14:33
*** pblaho has quit IRC		14:36
*** jistr has joined #tripleo		14:37
*** ftcjeff has joined #tripleo		14:39
*** matty_dubs\|gone is now known as matty_dubs		14:45
openstackgerrit	Sascha Peilicke proposed a change to openstack/tuskar: Sync with global requirements https://review.openstack.org/67061	14:46
*** jtomasek has joined #tripleo		14:48
*** coolsvap has joined #tripleo		14:53
*** jog0 is now known as flashgordon		14:56
*** marun has joined #tripleo		14:57
*** pblaho has joined #tripleo		15:02
*** lucasagomes is now known as lucas-hungry		15:09
*** jergerber has joined #tripleo		15:12
*** lblanchard has quit IRC		15:13
*** lblanchard has joined #tripleo		15:13
*** jtomasek has quit IRC		15:13
*** aweiteka has joined #tripleo		15:16
*** jistr has quit IRC		15:16
*** lblanchard has quit IRC		15:20
*** lblanchard has joined #tripleo		15:20
*** CaptTofu has joined #tripleo		15:24
*** jistr has joined #tripleo		15:25
*** jtomasek has joined #tripleo		15:26
*** bcrochet has quit IRC		15:28
*** bcrochet has joined #tripleo		15:30
*** jergerber has left #tripleo		15:36
*** marun has quit IRC		15:36
*** marun has joined #tripleo		15:36
*** bcrochet has quit IRC		15:40
*** bcrochet has joined #tripleo		15:43
*** jprovazn is now known as jprovazn_afk		15:44
*** bnemec is now known as beekneemech		15:45
*** jistr has quit IRC		15:45
*** ProfFalken has quit IRC		15:51
*** panda__ has joined #tripleo		15:55
SpamapS	lifeless: hah! That is rich. :)	15:55
*** jcoufal has quit IRC		15:56
SpamapS	lifeless: we have an NTP element, so we just need to decide how that is going to work. I think simplest thing would be for the undercloud to broadcast on all bridges	15:56
*** pblaho has quit IRC		15:56
SpamapS	lifeless: but less scalable would be to just point all servers at the undercloud controller(s)	15:57
*** panda_ has quit IRC		15:58
*** lucas-hungry is now known as lucasagomes		16:01
*** akrivoka has joined #tripleo		16:01
*** ifarkas has quit IRC		16:04
*** marun has quit IRC		16:05
*** marun has joined #tripleo		16:05
openstackgerrit	James Slagle proposed a change to openstack/diskimage-builder: Update to Fedora 20 https://review.openstack.org/67505	16:07
openstackgerrit	Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Initial implementation of deployment detail pages https://review.openstack.org/67322	16:07
*** edmund has joined #tripleo		16:10
*** vipul-away is now known as vipul		16:12
openstackgerrit	James Slagle proposed a change to openstack/diskimage-builder: Update to Fedora 20 https://review.openstack.org/67505	16:13
Ng	lifeless: do I need to file a ticket to get disks upgraded in the monty machines?	16:13
Ng	lifeless: (or do I need to go and poke at the machines to decide if a ticket needs to be filed?)	16:15
*** anteaya is now known as tired		16:21
*** tired is now known as very_tired		16:22
openstackgerrit	Tzu-Mainn Chen proposed a change to openstack/tuskar-ui: Initial implementation of deployment detail pages https://review.openstack.org/67322	16:23
*** vipul is now known as vipul-away		16:27
*** newell has joined #tripleo		16:30
*** marun has quit IRC		16:31
*** jtomasek has quit IRC		16:31
*** vipul-away is now known as vipul		16:31
*** marun has joined #tripleo		16:32
*** e0ne_ has quit IRC		16:32
*** AaronGr_Zzz is now known as AaronGr		16:45
*** bauzas has quit IRC		16:45
*** rushiagr is now known as rushiagr_away		16:48
*** rushiagr_away is now known as rushiagr		16:48
*** gothicmindfood has joined #tripleo		16:55
*** akrivoka has quit IRC		16:55
*** gothicmindfood has left #tripleo		16:56
*** coolsvap has quit IRC		16:57
*** sgrasley has joined #tripleo		16:58
*** coolsvap has joined #tripleo		16:59
*** vkozhukalov has quit IRC		17:00
*** taps_ has joined #tripleo		17:02
*** jdob has quit IRC		17:14
lifeless	Ng: good question :)	17:14
*** matty_dubs is now known as matty_dubs\|lunch		17:15
Ng	lifeless: re the borked fan one, a borked fan was replaced, but there is still an error LED showing, so another technician will be looking at the machine again later after the shift handovers and whatever other stuff they have to do that's higher priority	17:16
lifeless	Ng: I think some investigation is a good idea, since we have lots of spare machines at the moment	17:16
Ng	lifeless: k	17:16
lifeless	Ng: if you update the bm_node record to change service_host from broken-disk to undercloud, then the nova-bm api will be able to deploy to it again	17:18
lifeless	Ng: and you can spin up machines on the undercloud with nova boot until it's picked, then poke around	17:18
Ng	nice, will do so when they report the error LED is cleared	17:18
lifeless	Ng: is there an error LED on the one with the broken disk ?	17:19
lifeless	Ng: (I felt like that was a non-sequitor)	17:19
*** annashen has quit IRC		17:20
Ng	lifeless: oh, I thought you meant you'd put labelled the fanbork machine with broken-disk (which did seem odd, should have clarified :)	17:21
*** kiall has quit IRC		17:21
Ng	I am not aware of an error light on the one spewing IO errors, but I haven't asked if it's showing one :)	17:21
Ng	I was going to log into it in a bit and see what I could find out about its disks	17:21
lifeless	Ng: no - if you look at nova baremetal-node-list \| grep -v undercloud	17:22
lifeless	you can see we have one broken-disk, one broken-fan and one disabled-but-we-don't-know-whats-wrong-yet	17:23
*** kiall has joined #tripleo		17:24
Ng	lifeless: gotcha	17:24
Ng	lifeless: I had some idle wonders earlier about how to catch situations where a flavor is wrong, or even whether we should want to do that.	17:25
Ng	like maybe the deploy helper could check the size of the block device iscsi has given it	17:25
*** annashen has joined #tripleo		17:26
Ng	(assuming that SEEK_END on an iscsi target isn't a horrifyingly slow operation)	17:26
lifeless	Ng: I think thats likely a good idea, since right now we don't trigger a write to the last block at all	17:30
lifeless	Ng: I should have done lsblk before powering off - but! I'm fairly sure sfdisk would have whined if it was a simple 'disk is too small for the partitions' situation	17:31
Ng	lifeless: you'd hope so :)	17:31
Ng	some of those tools are pretty happy to let you shoot all of your feet off though	17:31
Ng	and they will only enumerate your toes in units of imperial cylinder sectors, before you do	17:32
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Permit deploying an overcloud with SSL endpoints. https://review.openstack.org/67356	17:35
openstackgerrit	lifeless proposed a change to openstack/tripleo-incubator: Make it possible to customise overcloud names. https://review.openstack.org/66467	17:35
*** rdopieralski has quit IRC		17:42
*** edmund has quit IRC		17:46
*** e0ne has joined #tripleo		17:48
*** taps_ has quit IRC		17:48
openstackgerrit	Ben Nemec proposed a change to openstack/tripleo-incubator: Use TE_DATAFILE if JSONFILE is not set https://review.openstack.org/67535	17:50
*** martyntaylor has quit IRC		17:50
*** athomas has quit IRC		17:51
*** lsmola_ has quit IRC		17:54
*** matty_dubs\|lunch is now known as matty_dubs		17:54
*** taps has joined #tripleo		17:59
*** jdob has joined #tripleo		18:04
Ng	lifeless: monty0001 has a single 2TB SAS disk, presented as a mumble-less-than-2TB logical drive	18:08
Ng	the BIOS claims it will present as 1863GB	18:09
Ng	(and as far as the controller is concerned, the disk is not aware of any health issues, but, lol SMART)	18:10
*** d0ugal has quit IRC		18:16
*** marun has quit IRC		18:22
*** marun has joined #tripleo		18:22
*** max_lobur is now known as max_lobur_afk		18:30
*** CaptTofu has quit IRC		18:31
*** marun has quit IRC		18:35
*** marun has joined #tripleo		18:35
*** rushiagr has quit IRC		18:49
*** rushiagr has joined #tripleo		18:51
*** vkozhukalov has joined #tripleo		18:53
*** rushiagr has quit IRC		18:53
openstackgerrit	A change was merged to openstack/tuskar-ui: Initial implementation of deployment detail pages https://review.openstack.org/67322	18:54
*** marun has quit IRC		19:04
*** marun has joined #tripleo		19:05
*** julim has quit IRC		19:05
*** julim has joined #tripleo		19:06
*** vipul is now known as vipul-away		19:09
*** vipul-away is now known as vipul		19:09
*** panda__ has quit IRC		19:09
*** panda__ has joined #tripleo		19:15
*** noslzzp has joined #tripleo		19:17
*** CaptTofu has joined #tripleo		19:19
*** jistr has joined #tripleo		19:24
*** jprovazn_afk has quit IRC		19:25
cody-somerville	SpamapS: ping. You aroubnd?	19:28
*** marun has quit IRC		19:28
*** marun has joined #tripleo		19:29
*** vipul is now known as vipul-away		19:35
*** vipul-away is now known as vipul		19:35
*** vipul is now known as vipul-away		19:35
*** bauzas has joined #tripleo		19:37
*** marun has quit IRC		19:41
*** marun has joined #tripleo		19:41
openstackgerrit	James Slagle proposed a change to openstack/tripleo-incubator: Update write-tripleorc. https://review.openstack.org/67557	19:43
* Ng returns from dinner/kids		19:44
*** vipul-away is now known as vipul		19:47
Ng	congrats mordred for being voted overlord space commander of all the realms of openstack	19:48
pleia2	hehe	19:48
*** AaronGr is now known as aarongr_afk		19:49
*** vipul is now known as vipul-away		19:50
*** ftcjeff has quit IRC		19:57
*** erwan_taf has quit IRC		19:58
mordred	woot!	19:59
*** erwan_taf has joined #tripleo		19:59
*** markmc has quit IRC		20:00
*** marun has quit IRC		20:00
*** marun has joined #tripleo		20:00
*** csd has joined #tripleo		20:03
clarkb	we have results?	20:04
*** lucasagomes has quit IRC		20:04
clarkb	I haven't seen email or anything about it yet	20:06
SpamapS	lifeless: hah, you said in 10 lines what I took 5 paragraphs to say. bravo.	20:09
*** ftcjeff has joined #tripleo		20:10
*** marun has quit IRC		20:15
*** marun has joined #tripleo		20:15
*** bauzas has quit IRC		20:35
*** bauzas has joined #tripleo		20:38
*** jistr has quit IRC		20:40
*** marun has quit IRC		20:41
*** marun has joined #tripleo		20:41
*** vipul-away is now known as vipul		20:47
*** dprince has quit IRC		20:49
Ng	hmm	20:50
Ng	raise exception.NoValidHost(reason="")	20:51
Ng	thanks scheduler, I probably don't need to know the reason, do I ;)	20:51
Ng	--availability_zone_zone=nova::blah is how you force an instance onto a particular BM node 'blah', right?	20:54
Ng	the scheduler seems to hate me doing that	20:54
*** marun has quit IRC		21:01
*** marun has joined #tripleo		21:01
*** marun has quit IRC		21:14
*** marun has joined #tripleo		21:14
* Ng sends some important email and ejects		21:20
*** noslzzp has quit IRC		21:21
*** rbrady has quit IRC		21:25
*** marun has quit IRC		21:26
*** marun has joined #tripleo		21:27
*** UtahDave has joined #tripleo		21:28
*** vipul is now known as vipul-away		21:31
*** marun has quit IRC		21:37
*** marun has joined #tripleo		21:38
*** vipul-away is now known as vipul		21:43
*** rbrady has joined #tripleo		21:44
*** d0ugal has joined #tripleo		21:44
*** rbrady has quit IRC		21:44
*** aarongr_afk is now known as AaronGr		21:45
*** d0ugal has quit IRC		21:45
*** d0ugal has joined #tripleo		21:45
*** d0ugal has quit IRC		21:46
*** d0ugal has joined #tripleo		21:46
*** d0ugal has quit IRC		21:46
*** vipul is now known as vipul-away		21:47
*** lblanchard has quit IRC		21:50
*** sdake has quit IRC		21:52
*** beekneemech has quit IRC		21:52
*** derekh has joined #tripleo		21:54
*** marun has quit IRC		21:55
*** marun has joined #tripleo		21:55
*** UtahDave has quit IRC		21:57
*** matty_dubs is now known as matty_dubs\|gone		21:59
*** d0ugal has joined #tripleo		22:01
*** vipul-away is now known as vipul		22:04
*** vipul is now known as vipul-away		22:04
*** jcooley_ has joined #tripleo		22:10
*** d0ugal has quit IRC		22:10
*** CaptTofu has quit IRC		22:11
*** jayg is now known as jayg\|g0n3		22:12
*** UtahDave has joined #tripleo		22:14
*** CaptTofu has joined #tripleo		22:16
*** aweiteka has quit IRC		22:19
*** vkozhukalov has quit IRC		22:24
*** bnemec has joined #tripleo		22:24
*** marun has quit IRC		22:28
*** very_tired is now known as anteaya		22:28
*** marun has joined #tripleo		22:29
*** ccrouch has quit IRC		22:30
*** ccrouch has joined #tripleo		22:30
*** ccrouch has quit IRC		22:35
*** ccrouch has joined #tripleo		22:37
*** jamezpolley has quit IRC		22:40
*** jdob has quit IRC		22:43
*** morazi has quit IRC		22:44
*** flashgordon is now known as jog0		22:50
*** ftcjeff has quit IRC		22:50
*** marun has quit IRC		22:50
*** bauzas has quit IRC		23:08
openstackgerrit	A change was merged to openstack/tripleo-incubator: Make it possible to customise overcloud names. https://review.openstack.org/66467	23:09
openstackgerrit	Derek Higgins proposed a change to openstack-infra/tripleo-ci: Fix the metadata resource name https://review.openstack.org/67599	23:16
openstackgerrit	Derek Higgins proposed a change to openstack-infra/tripleo-ci: Change metadata names to match the heat template https://review.openstack.org/67600	23:16
openstackgerrit	Derek Higgins proposed a change to openstack-infra/tripleo-ci: Change to tripleo-ci directory https://review.openstack.org/67221	23:16
openstackgerrit	Derek Higgins proposed a change to openstack-infra/tripleo-ci: Get IP for Test Host from neutron https://review.openstack.org/66942	23:16
openstackgerrit	Derek Higgins proposed a change to openstack-infra/tripleo-ci: Change path to tripleo-incubator https://review.openstack.org/67222	23:16
openstackgerrit	Derek Higgins proposed a change to openstack-infra/tripleo-ci: Get seed IPs from configured network https://review.openstack.org/66943	23:16
*** CaptTofu has quit IRC		23:24
*** CaptTofu has joined #tripleo		23:25
*** CaptTofu has quit IRC		23:29
*** jcooley_ has quit IRC		23:32
*** jcooley_ has joined #tripleo		23:39
*** vipul-away is now known as vipul		23:42
*** jcooley_ has quit IRC		23:44
*** boris-42 has quit IRC		23:45
*** jamezpolley has joined #tripleo		23:51
*** shadower has quit IRC		23:55
*** jhurlbert has joined #tripleo		23:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!