| *** yamahata has joined #tripleo | 00:03 | |
| *** yfaykya has quit IRC | 00:03 | |
| *** yfaykya has joined #tripleo | 00:05 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack/os-cloud-config: Fix niggles with the documentation https://review.openstack.org/97653 | 00:09 |
|---|---|---|
| openstackgerrit | Derek Higgins proposed a change to openstack/diskimage-builder: Export IMAGE_ELEMENT after finding its value https://review.openstack.org/97654 | 00:10 |
| *** yfaykya has quit IRC | 00:10 | |
| saurabhs | lifeless: can you suggest me something for above query | 00:10 |
| greghaynes | dshulyak: Im rebasing some of your haproxy patches off master to pull in some crit fixes | 00:10 |
| greghaynes | saurabhs: just cp in os-refresh-config | 00:11 |
| greghaynes | or in install.d | 00:11 |
| *** yfaykya has joined #tripleo | 00:11 | |
| saurabhs | for doing a cp in os-refresh-config that file has to be on the image right? how do I get it in there, currently I was using os-apply-config but if its a jinja template then os-apply-config modifies it | 00:13 |
| saurabhs | greghaynes | 00:13 |
| saurabhs | : | 00:13 |
| *** yfaykya has quit IRC | 00:15 | |
| openstackgerrit | xu-haiwei proposed a change to openstack/tripleo-incubator: Always save undercloud and overcloud password at $TRIPLEO_ROOT https://review.openstack.org/97657 | 00:17 |
| greghaynes | saurabhs: during install.d you should have access to the element and the chroot, you can install/cp the file from the element into the chroot then | 00:17 |
| * greghaynes should probably figure out how that actually works | 00:17 | |
| *** yfaykya has joined #tripleo | 00:17 | |
| greghaynes | we do this in elements/mariadb/install.d/10-mariadb for mysql-set-server-id.upstart | 00:18 |
| saurabhs | greghaynes: thanks I will look at it | 00:18 |
| saurabhs | greghaynes: so where does mysql-set-server-id.upstart come from | 00:20 |
| saurabhs | install $(dirname $0)/mysql-set-server-id.upstart /etc/init/mysql-set-server-id.conf | 00:20 |
| greghaynes | its in that element | 00:20 |
| greghaynes | no its not | 00:21 |
| greghaynes | its in mysql-common | 00:21 |
| greghaynes | in install.d | 00:21 |
| *** yfaykya has quit IRC | 00:22 | |
| *** matsuhashi has joined #tripleo | 00:23 | |
| *** xuhaiwei has joined #tripleo | 00:23 | |
| *** yfaykya has joined #tripleo | 00:23 | |
| tchaypo | jogo: any chance you have some local changes to devtest.sh? | 00:26 |
| *** olaph has quit IRC | 00:26 | |
| lifeless | ok am back | 00:28 |
| lifeless | saurabhs: hi | 00:28 |
| *** yfaykya has quit IRC | 00:28 | |
| lifeless | saurabhs: what sort of data do you want to pull in | 00:28 |
| lifeless | saurabhs: is it dynamic or static - does it change with configuration, or with the deploy, or with the software? | 00:28 |
| *** yfaykya has joined #tripleo | 00:30 | |
| jogo | tchaypo: no, not really sure what happened | 00:30 |
| jogo | others couldn't repreduce so maybe it was just me or something | 00:31 |
| *** shakayumi has joined #tripleo | 00:31 | |
| *** shakayumi has joined #tripleo | 00:32 | |
| *** nati_uen_ has quit IRC | 00:32 | |
| *** nati_ueno has joined #tripleo | 00:33 | |
| saurabhs | lifeless: it will probably change per deploy but its a jinja template that will change | 00:34 |
| *** yfaykya has quit IRC | 00:34 | |
| *** bcrochet is now known as bcrochet|g0ne | 00:34 | |
| lifeless | saurabhs: when would it change? | 00:35 |
| *** yfaykya has joined #tripleo | 00:36 | |
| saurabhs | lifeless: depending upon the flavor | 00:36 |
| lifeless | flavor? | 00:36 |
| saurabhs | nova flavor | 00:36 |
| lifeless | I'm thoroughly confused :) | 00:36 |
| lifeless | flavors are API defined | 00:36 |
| saurabhs | lifeless: depending upon the flavor size of the instance we will change some values in the config which is again a jinja template | 00:37 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Add details of which OS releases are tested in CI https://review.openstack.org/92749 | 00:37 |
| lifeless | saurabhs: so I deploy a thing X, you want to change configuration within X based on the nova flavor of the machine X was deployed to ? | 00:38 |
| *** nati_ueno has quit IRC | 00:38 | |
| saurabhs | lifeless: yes | 00:39 |
| lifeless | saurabhs: ok, so why would the template change, not the output of the template? | 00:39 |
| *** nati_ueno has joined #tripleo | 00:39 | |
| lifeless | saurabhs: it sounds like this is exactly what the template should be handling | 00:39 |
| *** yfaykya has quit IRC | 00:40 | |
| saurabhs | lifeless: so depending upon the nova flavor of the taskmanage template would get some specific values which it self is a template that taskmanager will in turn use to configure guest agents depending upon the nova flavor specific to the guestagent | 00:41 |
| *** yfaykya has joined #tripleo | 00:42 | |
| lifeless | saurabhs: what stops it being just one template ? | 00:42 |
| *** shakayumi has quit IRC | 00:45 | |
| *** kevinbenton has joined #tripleo | 00:45 | |
| lifeless | saurabhs: anyhow to answer your question, run-time delivery of content can be done via | 00:45 |
| lifeless | - oac templates | 00:46 |
| lifeless | - out of band (e.g. download from a server somewhere) | 00:46 |
| *** yfaykya has quit IRC | 00:46 | |
| lifeless | thats really about it - you can trigger the latter from your agent / api service / orc script | 00:46 |
| *** ddieterly has joined #tripleo | 00:46 | |
| lifeless | but it seems like you have a circuitious approach to something that would make a single simple template | 00:46 |
| saurabhs | lifeless: what is oac template | 00:47 |
| *** nati_ueno has quit IRC | 00:47 | |
| *** morazi has joined #tripleo | 00:47 | |
| *** nati_ueno has joined #tripleo | 00:48 | |
| lifeless | saurabhs: os-apply-config, the minimal template layer we use to write nova.conf etc | 00:48 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Clean up all outstanding ReST errors and warnings https://review.openstack.org/97129 | 00:48 |
| saurabhs | lifeless: thanks | 00:48 |
| *** yfaykya has joined #tripleo | 00:48 | |
| *** yfaykya has quit IRC | 00:52 | |
| *** yamahata has quit IRC | 00:53 | |
| *** yamahata has joined #tripleo | 00:53 | |
| *** yfaykya has joined #tripleo | 00:54 | |
| *** morazi has quit IRC | 00:56 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Clean up any existing environment first https://review.openstack.org/92038 | 00:59 |
| *** yfaykya has quit IRC | 00:59 | |
| *** yfaykya has joined #tripleo | 01:00 | |
| *** yfaykya has quit IRC | 01:05 | |
| *** yfaykya has joined #tripleo | 01:06 | |
| *** yfaykya has quit IRC | 01:11 | |
| *** yfaykya has joined #tripleo | 01:12 | |
| *** yfaykya has quit IRC | 01:17 | |
| *** nati_uen_ has joined #tripleo | 01:18 | |
| *** yfaykya has joined #tripleo | 01:19 | |
| *** nati_ueno has quit IRC | 01:22 | |
| *** yfaykya has quit IRC | 01:23 | |
| *** yfaykya has joined #tripleo | 01:25 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-heat-templates: Add custom public IP support to the undercloud. https://review.openstack.org/96917 | 01:25 |
| *** spzala has quit IRC | 01:26 | |
| *** yfaykya has quit IRC | 01:29 | |
| *** yfaykya has joined #tripleo | 01:31 | |
| *** cwolferh has quit IRC | 01:35 | |
| *** yfaykya has quit IRC | 01:35 | |
| *** yfaykya has joined #tripleo | 01:37 | |
| *** nosnos has joined #tripleo | 01:38 | |
| *** jp_at_hp has quit IRC | 01:39 | |
| *** nati_uen_ has quit IRC | 01:39 | |
| *** nati_ueno has joined #tripleo | 01:39 | |
| *** yfaykya has quit IRC | 01:42 | |
| *** yfaykya has joined #tripleo | 01:43 | |
| *** yfaykya has quit IRC | 01:47 | |
| *** yfaykya has joined #tripleo | 01:49 | |
| *** yfaykya has quit IRC | 01:54 | |
| *** yfaykya has joined #tripleo | 01:55 | |
| *** spzala has joined #tripleo | 02:01 | |
| *** jml has quit IRC | 02:04 | |
| *** jml has joined #tripleo | 02:11 | |
| *** ramishra has joined #tripleo | 02:21 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Use seed ip instead of gateway ip for ssh-keygen. https://review.openstack.org/94219 | 02:23 |
| *** spzala has quit IRC | 02:28 | |
| *** noslzzp has joined #tripleo | 02:41 | |
| *** tzumainn has quit IRC | 02:46 | |
| *** ramishra has quit IRC | 02:48 | |
| *** xuhaiwei has quit IRC | 02:52 | |
| jogo | silly question, how do I add metadata to a heat template? | 02:57 |
| *** ramishra has joined #tripleo | 02:58 | |
| lifeless | jogo: what sort of metadata ? | 02:58 |
| jogo | lifeless: I am trying to manaully exercise the basic heatworkflow that we want to exercise in large-ops | 02:59 |
| jogo | this is my first time using heat | 02:59 |
| *** untriaged-bot has joined #tripleo | 03:00 | |
| untriaged-bot | Untriaged bugs so far: | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 03:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 03:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1326016 | 03:00 |
| *** untriaged-bot has quit IRC | 03:00 | |
| uvirtbot | Launchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,New] | 03:00 |
| jogo | I was thinking for starters create a simple template with some metadata and run 'heat --debug resource-metadata' | 03:00 |
| jogo | then run a bunch of heat resource-metadata's in parallel | 03:01 |
| jogo | to get a sense of thigns | 03:01 |
| lifeless | jogo: ah | 03:01 |
| lifeless | checkout tripleo-heat-templates | 03:01 |
| jogo | if that all worked I was gonna wire that into the large-ops test as a POC | 03:01 |
| lifeless | make overcloud.yaml | 03:01 |
| lifeless | with OVERCLOUD_SCALE=100 | 03:02 |
| lifeless | or something | 03:02 |
| lifeless | that will get you a template that is realistic | 03:02 |
| jogo | I was hoping for something small for starters | 03:02 |
| lifeless | SCALE=2 then | 03:02 |
| lifeless | you'll need a template with software-config | 03:03 |
| lifeless | deployments | 03:03 |
| lifeless | and machines | 03:03 |
| lifeless | and launchconfigurations | 03:03 |
| lifeless | it will be a lot easier to take one that has that than to start from scratch | 03:03 |
| lifeless | you can dial the scale down as low as you want to make it likely to work up-front | 03:04 |
| jogo | would any of those work with the fake virt driver? | 03:04 |
| lifeless | all of them | 03:04 |
| greghaynes | probably mean OVERCLOUD_COMPUTESCALE | 03:05 |
| lifeless | as I said, the recipe was fake virt + scaled template + the kill-heat script | 03:05 |
| jogo | so I want to start even smaller so I sort of understand it | 03:06 |
| lifeless | ok | 03:07 |
| lifeless | well, the way I'd do it is to study the working template | 03:07 |
| lifeless | anyhow, metadata | 03:08 |
| lifeless | you can do it via launchconfiguration or deployments of configuration | 03:08 |
| lifeless | and we do both | 03:08 |
| *** rwsu has quit IRC | 03:10 | |
| lifeless | nuts | 03:24 |
| lifeless | COMMAND=ironic node-set-power-state 446b2b8c-4cb3-4058-a0fd-2efb6e1e7a31 off | 03:24 |
| lifeless | OUTPUT=Node 446b2b8c-4cb3-4058-a0fd-2efb6e1e7a31 is locked by host undercloud-undercloud-n3af3lhhuitr.novalocal, please retry after the current operation is completed. (HTTP 409) | 03:24 |
| lifeless | adam_g: IIRC we did a workaround for this ? | 03:24 |
| lifeless | ah yes, we did, but 10 seconds apparently not enough | 03:25 |
| *** nosnos has quit IRC | 03:29 | |
| *** matsuhashi has quit IRC | 03:31 | |
| *** vinsh has joined #tripleo | 03:39 | |
| jogo | lifeless: got a simple example working. so what element of heat is falling over serving metadata? | 03:45 |
| lifeless | jogo: everything | 03:45 |
| lifeless | jogo: its hard to pinpoint, which is why we want a representative test, not a minimal test | 03:46 |
| lifeless | jogo: it was previously just raw performance issues, which would show up in the server and the API | 03:46 |
| jogo | lifeless: make sense. I want to build up to learn about heat a bit | 03:46 |
| lifeless | a lot has improved though | 03:46 |
| *** ddieterly has quit IRC | 03:46 | |
| *** ddieterly has joined #tripleo | 03:47 | |
| *** sballe has joined #tripleo | 03:47 | |
| openstackgerrit | Om Kumar proposed a change to openstack/diskimage-builder: Replacing deploy to deploy-baremetal in README.md https://review.openstack.org/95655 | 03:48 |
| openstackgerrit | Om Kumar proposed a change to openstack/diskimage-builder: Refactor code to select boot kernel https://review.openstack.org/79873 | 03:49 |
| StevenK | tchaypo: Did you see I closed https://bugs.launchpad.net/os-cloud-config/+bug/1317749 this morning? | 03:59 |
| uvirtbot | Launchpad bug 1317749 in os-cloud-config "Documentation link is a 404" [High,Fix released] | 03:59 |
| jogo | lifeless: just using http://paste.ubuntu.com/7584811/ | 04:00 |
| jogo | and calling 'heat resource-metadata stack server' 50 times at once the slowest run takes 11 seconds | 04:00 |
| lifeless | jogo: right, so as long as it takes < 30 seconds reliably, its usable | 04:01 |
| lifeless | jogo: on a 24 core machine with 24 heat-engine processes | 04:01 |
| lifeless | jogo: oh | 04:02 |
| lifeless | no, that should be fine | 04:03 |
| StevenK | lifeless: Review of https://review.openstack.org/97653 please, the docs up on docs.o.o/developer/os-cloud-config look pretty terrible | 04:03 |
| lifeless | its a bit of a simple template | 04:03 |
| lifeless | jogo: what deployment are you testing against ? | 04:04 |
| jogo | devstack | 04:06 |
| jogo | with fake virt backend | 04:06 |
| jogo | so just hit something funny | 04:06 |
| jogo | I get a stacktrace if I try to use the same template twice | 04:06 |
| jogo | http://paste.ubuntu.com/7584849/ | 04:06 |
| lifeless | have you deleted the stack in the interim ? | 04:08 |
| lifeless | There is a deterministic named user in keystone used for the delegated access control | 04:09 |
| jogo | I have not deleted the other stack | 04:09 |
| jogo | I wanted to create two stacks | 04:09 |
| jogo | at once | 04:09 |
| jogo | with different names | 04:09 |
| lifeless | its more than a bit of a bug :) | 04:09 |
| jogo | and the same template | 04:09 |
| lifeless | SpamapS: do you have the bug # | 04:09 |
| lifeless | jogo: so AIUI if you rename the server it should be fine | 04:10 |
| jogo | anyway I have to head AFK for a bit | 04:10 |
| jogo | lifeless: I don't want to have to rename the server, that is just silly | 04:11 |
| jogo | server names are not unique | 04:11 |
| lifeless | jogo: yes, it is. | 04:11 |
| lifeless | jogo: but as I explained, a) its a bug, b) thats the workaround. | 04:11 |
| jogo | yeah that will work for now | 04:11 |
| jogo | so for a basic test I am thinking: | 04:11 |
| jogo | spin up n copies of a simple template | 04:11 |
| lifeless | server names should be unique, but thats a different discussion | 04:12 |
| lifeless | n copies of a single template is entirely uninteresting to me | 04:12 |
| stevebaker | lifeless: if you take the "name: Single Compute Instance" out of that template then you won't hit https://bugs.launchpad.net/heat/+bug/1324499 | 04:12 |
| uvirtbot | Launchpad bug 1324499 in heat "physical_resource_name of Nova server should not be overriden" [High,In progress] | 04:12 |
| lifeless | its worth while doing in the broad set of tests to run, but its totally different workload to the ones we need to qualify for Helion | 04:12 |
| jogo | run m copies of 'heat-resource-metadata' at once and see what happens (watch for timeouts) | 04:12 |
| jogo | lifeless: true, I want to keep the size of the code to drive the test fairly small and self contained (for now) so didn't want a massive template, unless its autogenerated from something small or something | 04:13 |
| lifeless | jogo: well, thats what you would get if you used tripleo-heat-templates - no code to write at all | 04:14 |
| lifeless | jogo: git clone; make with a scale, done. | 04:14 |
| jogo | thats a lot of complexity to add to the gate for now | 04:14 |
| jogo | anyway now that I got the basic thing working I will try that use case too | 04:15 |
| lifeless | stevebaker: thanks | 04:15 |
| jogo | time to go AFK for abit o/ | 04:15 |
| lifeless | jogo: ciao | 04:15 |
| lifeless | stevebaker: do you need anything from me on that memoisation patch? | 04:16 |
| stevebaker | lifeless: a fass-forward button on the gate | 04:16 |
| stevebaker | fast | 04:16 |
| StevenK | Haha | 04:16 |
| lifeless | stevebaker: fasssssssst | 04:16 |
| stevebaker | sigh, and a recheck bot | 04:17 |
| StevenK | We're still recovering from setuptools kills the entire world | 04:17 |
| StevenK | check is down to 102, which is good. | 04:17 |
| StevenK | gate is 80, which isn't. | 04:17 |
| lifeless | I missed the setuptools excitment | 04:18 |
| lifeless | what happened ? | 04:18 |
| *** ramishra has quit IRC | 04:19 | |
| StevenK | lifeless: https://bitbucket.org/pypa/setuptools/issue/213/newest-setuptools-breaks-installation-of | 04:19 |
| StevenK | I spent all of Monday trying to workaround it and getting bitten in various ways | 04:20 |
| lifeless | nice (not) | 04:21 |
| lifeless | I was doing queens birtthday:) | 04:21 |
| StevenK | That's this coming Monday for NSW | 04:23 |
| *** matsuhashi has joined #tripleo | 04:25 | |
| *** nosnos has joined #tripleo | 04:26 | |
| stevebaker | lifeless: are you still having metadata polling issues on templates other than a high OVERCLOULD_COMPUTESCALE? | 04:27 |
| lifeless | stevebaker: we're trying to get out in front | 04:28 |
| lifeless | stevebaker: target scale of 2000 pollers | 04:28 |
| *** rcarrill` has joined #tripleo | 04:29 | |
| *** rcarrillocruz has quit IRC | 04:32 | |
| stevebaker | lifeless: COMPUTESCALE=100 leads to a 243279 line template, there must be a better way of composing that | 04:33 |
| *** ramishra has joined #tripleo | 04:33 | |
| greghaynes | :) | 04:39 |
| greghaynes | Thats probably the n^2 hosts: properties | 04:40 |
| *** pcrews has quit IRC | 04:44 | |
| stevebaker | it is. I would highly recommend moving the hosts building to a single config resource rather than n deployment resources | 04:44 |
| *** jml has quit IRC | 04:46 | |
| *** jml has joined #tripleo | 04:47 | |
| *** nati_ueno has quit IRC | 04:55 | |
| *** nati_uen_ has joined #tripleo | 04:57 | |
| *** nati_uen_ has quit IRC | 04:57 | |
| *** shakamunyi has joined #tripleo | 05:04 | |
| *** rameshg87 has joined #tripleo | 05:07 | |
| *** Kiall has quit IRC | 05:15 | |
| *** rakesh_hs has joined #tripleo | 05:15 | |
| lifeless | stevebaker: like https://review.openstack.org/#/c/93817/ ? | 05:18 |
| lifeless | stevebaker: SpamapS said he thinks this is wrong .... | 05:18 |
| lifeless | stevebaker: personally, I'm with you | 05:18 |
| lifeless | greghaynes: ^ | 05:18 |
| *** Kiall has joined #tripleo | 05:19 | |
| *** saurabhs has left #tripleo | 05:20 | |
| greghaynes | Well, if thats how large the file is at 100 lines, 2k is going to be a pretty big number | 05:21 |
| greghaynes | er, at 100 hosts | 05:21 |
| lifeless | yes | 05:22 |
| lifeless | greghaynes: care to do ze honours? | 05:23 |
| greghaynes | hrm? | 05:23 |
| lifeless | greghaynes: resurrect spamaps patch | 05:24 |
| greghaynes | ah, I have this power? | 05:24 |
| lifeless | now, wtf is this undercloud having instances go straight to error | 05:24 |
| lifeless | greghaynes: review -d 93817 | 05:25 |
| * StevenK pouts at the CI failure in 97653 | 05:25 | |
| stevebaker | I commented on 93817. 243279 line overcloud.yaml vs 14596 line | 05:25 |
| greghaynes | oh, I am seeing the same for my undercloud | 05:26 |
| lifeless | greghaynes: ruh roh | 05:26 |
| greghaynes | was just digging into it | 05:26 |
| lifeless | greghaynes: spamaps patch may do too much | 05:26 |
| lifeless | but its an example | 05:26 |
| *** rdopiera has joined #tripleo | 05:26 | |
| lifeless | greghaynes: no valid host ? | 05:26 |
| greghaynes | yep | 05:27 |
| lifeless | my image was built a few days back | 05:27 |
| lifeless | may 29th | 05:27 |
| greghaynes | hrm. I have a fairly large patch stack going on so im kind of all over | 05:28 |
| greghaynes | (woo HA patches) | 05:28 |
| lifeless | \o/ | 05:28 |
| lifeless | HAundercloud ? | 05:28 |
| greghaynes | no, but I ran out of /var and... yea | 05:28 |
| greghaynes | then my undercloud didnt come back | 05:29 |
| lifeless | oops | 05:29 |
| lifeless | how did /var run out ? | 05:29 |
| *** jtomasek has joined #tripleo | 05:30 | |
| greghaynes | we must put images there at some point in build? I was making more overcloud nodes than normal | 05:30 |
| lifeless | 2014-06-04 05:30:16.307 26433 DEBUG nova.scheduler.filters.ram_filter [req-9560ef14-edb1-4890-9005-8961dc1fbcc8 None] (undercloud, a5318eae-3d81-4d62-9531-e5700c71e378) ram:0 disk:0 io_ops:1 instances:1 does not have 98304 MB usable ram, it only has 0.0 MB usable ram. host_passes /opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/scheduler/filters/ram_filter.py:60 | 05:30 |
| StevenK | /dev/mapper/sys-var 99G 11G 84G 11% /var | 05:30 |
| StevenK | That's why I do that ^ | 05:30 |
| greghaynes | yea, I had to lvextend | 05:31 |
| StevenK | \o/ lvm | 05:31 |
| lifeless | /tftproot | 05:31 |
| lifeless | could be the issue | 05:31 |
| StevenK | /tftproot makes me sad | 05:31 |
| StevenK | Because damn the FHS | 05:31 |
| tchaypo | http://docs.openstack.org/developer/hacking/ says to "order your imports according to the following template", then blunders onward without providing a template. | 05:31 |
| lifeless | it needs to move to /mnt/stte anyhow | 05:31 |
| tchaypo | oh no, there, sthe template, a couple of paragraphs below | 05:32 |
| StevenK | lifeless: Would you like me to poke at that? | 05:32 |
| lifeless | StevenK: sure | 05:33 |
| lifeless | I wonder if Ironic is exporting the wrong amount of memory | 05:34 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Fix jshint warnings https://review.openstack.org/97467 | 05:34 |
| lifeless | free_ram_mb | 4423680 / memory_mb | 4423680 | 05:35 |
| lifeless | so it *seems* ok | 05:35 |
| lifeless | memory_mb_used | 0 | 05:35 |
| StevenK | pre-start script | 05:36 |
| StevenK | mkdir -p /tftpboot | 05:36 |
| * StevenK whimpers | 05:36 | |
| *** jtomasek has quit IRC | 05:38 | |
| *** pcrews has joined #tripleo | 05:44 | |
| *** noslzzp has quit IRC | 05:45 | |
| lifeless | this might do it | 05:49 |
| lifeless | memory_mb_limit = total_usable_ram_mb * ram_allocation_ratio | 05:49 |
| lifeless | nope, ram_allocation_ratio is 1.0 | 05:49 |
| *** lazy_prince has joined #tripleo | 05:49 | |
| *** pcrews has quit IRC | 05:56 | |
| *** jml has quit IRC | 05:59 | |
| lifeless | greghaynes: interesting, --num-instances 2 and I've got it spawning | 06:02 |
| *** jtomasek has joined #tripleo | 06:03 | |
| lifeless | count 20 doing stuff too | 06:03 |
| lifeless | v odd | 06:03 |
| *** andreaf has joined #tripleo | 06:03 | |
| *** jml has joined #tripleo | 06:04 | |
| lifeless | jml: oh hai | 06:06 |
| *** nati_ueno has joined #tripleo | 06:10 | |
| SpamapS | lifeless: ok so the reason I think the patch I did on config vs. deployment is wrong is that it destroys composability. | 06:12 |
| SpamapS | lifeless: if we can pull configs from other configs, I might change my mind on that. | 06:12 |
| SpamapS | stevebaker: ^^ | 06:16 |
| SpamapS | lifeless: did we redeploy the R1 jump host too? | 06:17 |
| SpamapS | lifeless: I can't seem to SSH to it. | 06:17 |
| *** pcrews has joined #tripleo | 06:21 | |
| SpamapS | lifeless: left thoughts on https://review.openstack.org/#/c/93817/ | 06:28 |
| *** nati_ueno has quit IRC | 06:28 | |
| SpamapS | aha, it moved to .2 | 06:29 |
| *** lsmola has quit IRC | 06:40 | |
| lifeless | SpamapS: bastion.hp1.tripleo.org | 06:46 |
| lifeless | SpamapS: so I don't understand in pragmatic terms what composability it affects | 06:46 |
| lifeless | SpamapS: we can compose deployments to bring separate keys in | 06:47 |
| SpamapS | except the structure is entirely contained in the configs | 06:47 |
| lifeless | but each deployment brings its own config | 06:48 |
| SpamapS | so if you reference the controller IP directly in the config, you can't reuse it in a topology that has a split-out rabbitMQ server. | 06:48 |
| lifeless | paraphrasing | 06:48 |
| lifeless | 'if the set of hosts we need to scatter-gather changes, we need a different scatter-gather expression' ? | 06:48 |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Fixes to swift-storage-source.yaml https://review.openstack.org/92620 | 06:48 |
| SpamapS | No | 06:49 |
| SpamapS | That's obviously true. | 06:49 |
| lifeless | but that seems to be the only implication from what I understand stevebaker to be suggesting | 06:49 |
| SpamapS | If the set of hosts we need to scatter-gather changes, we must compose a whole new _configuration structure_. | 06:49 |
| lifeless | SpamapS: why? We can have as many independent configs as we want | 06:50 |
| SpamapS | Unless we want to play the ordering/overlay game. | 06:50 |
| SpamapS | lifeless: I urge you to author some templates before making assumptions about them being sane. | 06:50 |
| lifeless | SpamapS: uhm, I have done and do - you're not the only tht author :) | 06:50 |
| SpamapS | lifeless: Ok, so config objects are intended to be reusable. | 06:51 |
| SpamapS | but they cease to be reusable if the reference servers. | 06:51 |
| SpamapS | they | 06:51 |
| lifeless | so what I think you're getting at | 06:52 |
| SpamapS | Now, we can make a "topologyConfig" object or something, and have that reference servers, and then just reference it in a deployment with a later lexical sort than the generic one... | 06:52 |
| lifeless | is that a config for Nova with a string reference to ControlPlane0 rather than an input, can only be used when ControlPlane0 is defined to something sensible | 06:52 |
| SpamapS | but zomg they're already hard to understand without also having to order the deployments in your head. :-P | 06:53 |
| SpamapS | lifeless: precisely | 06:53 |
| lifeless | SpamapS: Sadly the deployment design is ordered; we should do something to make that easier to manage. | 06:53 |
| lifeless | SpamapS: so, I believe stevebaker in this case is suggesting that: | 06:53 |
| lifeless | a) we make a new config that defines hosts | 06:53 |
| lifeless | b) a new deployment of that | 06:54 |
| lifeless | c) *we do not define hosts /anywhere else/ | 06:54 |
| lifeless | that is, the configs are disjoint | 06:54 |
| SpamapS | lifeless: which is especially frustrating if you want to reference ControlPlane0 for everything except rabbit... | 06:54 |
| SpamapS | lifeless: yeah, basically we can have a single config that is "the topology" | 06:54 |
| lifeless | that mitigates the ordering problem a bit | 06:55 |
| *** noslzzp has joined #tripleo | 06:56 | |
| SpamapS | Indeed. And there can still be references to those inputs in the generic configs. | 06:56 |
| lifeless | SpamapS: you can use a config as input to another config ? | 06:56 |
| SpamapS | in generic deployments, they'll be set to '*SEE TOPOLOGY*' or something. | 06:56 |
| SpamapS | lifeless: you can get the 'config' attribute yes | 06:56 |
| SpamapS | lifeless: but I am 99% sure it will always be a string | 06:57 |
| lifeless | so | 06:57 |
| lifeless | high level | 06:57 |
| lifeless | we have a fairly severe scaling issue right now | 06:57 |
| lifeless | we want to get past it | 06:57 |
| SpamapS | Yeah, are we not confident that the caching will help? | 06:58 |
| lifeless | can we try to do this the right way, and if it doesn't seem tacklable in a couple days, do the thing stevebaker suggested | 06:58 |
| lifeless | SpamapS: n^2 >> caching | 06:58 |
| SpamapS | We have a lot of plates spinning. | 06:58 |
| SpamapS | Working on this means letting one fall. | 06:58 |
| lifeless | SpamapS: you seem worried about the long term implications of doing a short term patch | 06:59 |
| SpamapS | I'm convinced that it is doable in a way that won't make me wretch. | 06:59 |
| lifeless | SpamapS: I respect your instincts, but I'm worried about us just stopping cold | 06:59 |
| SpamapS | I am, however, not convinced that caching won't be a big enough win to take the pressure off. | 06:59 |
| lifeless | SpamapS: so I'm trying to find a balanced path that lets us move forward | 06:59 |
| marios | q: gerrit apparently no longer 'times out' reviews due to inactivity. I am pretty sure this one of the things pounding our review queue. Example: https://review.openstack.org/#/c/92853/ - I don't think anything further is going to happen here. Should I -2 it (will that no longer be counted in review queue times?)? | 06:59 |
| lifeless | merge.py has some n^2 stuff in it too I think | 06:59 |
| lifeless | marios: workflow -1 it | 07:00 |
| SpamapS | merge.py has miracle whip in it | 07:00 |
| clarkb | no you should abandon if the change will never merge. WIP if it needs more time | 07:00 |
| *** noslzzp has quit IRC | 07:00 | |
| lifeless | marios: -2 is 'can not merge until you change your mind' | 07:00 |
| marios | lifeless: right, but what clarkb said | 07:00 |
| lifeless | clarkb: can we abandon someone elses patch ? | 07:00 |
| clarkb | should be able to as core | 07:01 |
| lifeless | marios: that patch looks legitimte-but-needs-work to me, no ? | 07:01 |
| SpamapS | http://theoatmeal.com/blog/miracle_whip for reference ;) | 07:01 |
| clarkb | unless that is amagic button for me | 07:01 |
| marios | lifeless: well, clint and ryan say that this is already in os-svc-install (comments @ https://review.openstack.org/#/c/92853/1/elements/collectl/install.d/85-collectl ) | 07:01 |
| *** giulivo has joined #tripleo | 07:01 | |
| marios | lifeless: and may 23rd is a long time ago | 07:01 |
| lifeless | clarkb: oh, it is there isn't it. | 07:01 |
| lifeless | marios: they say 'May 9 6:34 AM | 07:02 |
| lifeless | This is already in os-svc-install element via os-svc-enable, why not use that?' | 07:02 |
| *** mrunge has joined #tripleo | 07:02 | |
| lifeless | marios: which is not 'no change needed' its 'patch needs to be altered to do what it does differently' | 07:02 |
| SpamapS | correct | 07:02 |
| lifeless | marios: in terms of dates, summit + running around in circles fixing testbeds within HP- I rather suspect Ng would be delighted to fix the patch up | 07:02 |
| marios | lifeless: ok. yeah the question was also more general (trying to clean up old reviews - recheck bugs etc etc) | 07:03 |
| lifeless | marios: anyhow, the protocol for removing from the dashboard is workflow -1 unless you're really sure they are not coming back / its not relevant in which case abandon. IMO. | 07:03 |
| marios | lifeless: i guess -1 should get the authors attention as well | 07:03 |
| marios | k, thanks | 07:03 |
| lifeless | -2 is 'merge it over my dead body' - its sticky regardless of new uploads, so its a very big hammer. | 07:04 |
| SpamapS | lifeless: did you ever get the fake-backed-heat + kill-heat-via-occ thing going btw? | 07:04 |
| lifeless | jogo has a devstack based one | 07:04 |
| lifeless | I've just gotten the undercloud doing stuff now | 07:04 |
| lifeless | though I suspect we're about to wedge the BMCs since we don't have the ironic IPMI rate limit patch yet. | 07:05 |
| SpamapS | I'd love it if hpcloud had a devstack image available publicly for devs to use. | 07:05 |
| *** jprovazn has joined #tripleo | 07:07 | |
| *** eghobo has joined #tripleo | 07:08 | |
| jogo | SpamapS: I have the heat + fake backend set up, I am not suing occ just yet though | 07:08 |
| lifeless | Killed | 07:09 |
| lifeless | make: *** [overcloud.yaml] Error 137 | 07:09 |
| jogo | SpamapS: what do you expect the failure mode to be? just a slow down? | 07:09 |
| lifeless | 'woops' | 07:09 |
| SpamapS | jogo: exponential slow down | 07:10 |
| lifeless | so 100 hypervisors = 240K line template | 07:10 |
| lifeless | 110 = 290K line template | 07:11 |
| SpamapS | jogo: every deployment (so ever server) will make the stack twice as slow to deploy | 07:11 |
| lifeless | SpamapS: | 07:11 |
| SpamapS | every | 07:11 |
| SpamapS | Maybe we should just try to fix resource groups.. ;) | 07:11 |
| lifeless | 120=345K lines | 07:11 |
| SpamapS | right that is because we have every-host-references-every-host | 07:12 |
| SpamapS | that single bit.. | 07:12 |
| SpamapS | we can put in a single config | 07:12 |
| jogo | SpamapS: oh good to know, I am trying the autoscaling group | 07:12 |
| jogo | so I can do an autoscaling group of say 100? | 07:13 |
| jogo | and that should slow way down? | 07:13 |
| lifeless | jogo: well, this is where testing the tripleo template specifically is what we /need/ | 07:13 |
| SpamapS | lifeless: how about we just try putting the hosts thing in a single config.. leaving the rest untoutched | 07:13 |
| SpamapS | untouched | 07:13 |
| lifeless | SpamapS: thats all I *ever* suggested :) | 07:13 |
| SpamapS | lifeless: well I misunderstood | 07:13 |
| jogo | lifeless: yeah, I'll save that for tmorrow | 07:13 |
| lifeless | SpamapS: 130 hypervisors -> 401K lines | 07:13 |
| * SpamapS starts new simpler patch | 07:13 | |
| SpamapS | lifeless: I bet its eating ridiculous amounts of RAM while running merge.py too | 07:14 |
| SpamapS | unless we forced cyaml | 07:14 |
| lifeless | SpamapS: yeah, I killed libvirt and a few other daemons trying to build a 2000 hypervisor template | 07:14 |
| SpamapS | nice | 07:14 |
| lifeless | 'nice' | 07:14 |
| SpamapS | gah no | 07:14 |
| SpamapS | lifeless: yeah we don't use libyaml | 07:14 |
| SpamapS | so.. purepython one... | 07:14 |
| lifeless | SpamapS: nah, it hadnt gotten to the yaml point | 07:15 |
| lifeless | its not serialisation fail | 07:15 |
| lifeless | its n^2 calculating the in memory structure too | 07:15 |
| SpamapS | we parse nova-compute-instance.yaml over and over | 07:15 |
| SpamapS | FileInclude's | 07:15 |
| * SpamapS REFUSES to entertain the idea of optimizing merge.py | 07:16 | |
| SpamapS | fix the HOTs | 07:16 |
| lifeless | is the FileInclude below a mergemap ? | 07:16 |
| adam_g | oh, nice. left devtest on loop on 405, went to see aliens and came home to successful runs | 07:16 |
| lifeless | I'm fairly sure we don't reparse | 07:16 |
| jogo | ux | 07:16 |
| SpamapS | adam_g: mostly they only come at night. Mostly. | 07:16 |
| SpamapS | lifeless: I hope so. :) | 07:17 |
| adam_g | SpamapS, the aliens or the successful devtests? | 07:17 |
| lifeless | SpamapS: IMBVW | 07:17 |
| *** matsuhashi has quit IRC | 07:17 | |
| *** xuhaiwei has joined #tripleo | 07:17 | |
| SpamapS | lifeless: does not matter. We should not be doing this at this scale. | 07:17 |
| lifeless | you and your reasons :) | 07:18 |
| *** matsuhashi has joined #tripleo | 07:18 | |
| adam_g | lifeless, RE: locked nodes. i haven't seen that in a long time and IIRC nick reported this fixed it? https://review.openstack.org/#/c/95573/ | 07:19 |
| SpamapS | yaml really confuses diff sometimes | 07:20 |
| SpamapS | http://paste.ubuntu.com/7585626/ | 07:20 |
| lifeless | adam_g: saw it fairly reliably registering 46 nodes | 07:20 |
| SpamapS | lifeless: ^^ | 07:20 |
| lifeless | sigh at sso handshake for plain text versions | 07:21 |
| lifeless | SpamapS: looks plausible | 07:22 |
| SpamapS | yeah I stopped using paste.o.o because it was down too often | 07:22 |
| SpamapS | lifeless: testing it now with COMPUTESCALE=100 | 07:22 |
| SpamapS | with 10 computescale it is only 4 less lines of yaml | 07:23 |
| *** eghobo has quit IRC | 07:24 | |
| SpamapS | doh | 07:24 |
| * SpamapS fails | 07:24 | |
| lifeless | ok this is weird | 07:28 |
| lifeless | nova list | grep hw-t | wc -l | 07:28 |
| lifeless | 44 | 07:28 |
| lifeless | ironic node-list | grep False | wc -l | 07:28 |
| lifeless | 45 | 07:28 |
| lifeless | but i registered 46 before | 07:28 |
| lifeless | oh hmmm | 07:29 |
| lifeless | SpamapS: have you been on the bastion and toggling things? | 07:29 |
| SpamapS | lifeless: no | 07:29 |
| SpamapS | lifeless: I cleaned up an old nagios config that was spamming me, that is all | 07:29 |
| lifeless | | last_error | During sync_power_state, max retries exceeded for node 3f9df53e- | | 07:30 |
| lifeless | | | 5a76-47b3-a1c4-f596822bb43d, node state None does not match expected | | 07:30 |
| lifeless | | | state 'power on'. Updating DB state to 'None' Switching node to | | 07:30 |
| lifeless | | | maintenance mode. | | 07:30 |
| SpamapS | lifeless: ok I have a patch that is a significant reduction in size... but I'm worried about circular deps. Spinning up a Heat to try it out | 07:41 |
| SpamapS | $ wc -l 100.orig.overcloud.yaml overcloud.yaml | 07:42 |
| SpamapS | 243286 100.orig.overcloud.yaml | 07:42 |
| SpamapS | 20494 overcloud.yaml | 07:42 |
| lifeless | much better :) | 07:42 |
| lifeless | devananda: filing a bug about the boot-> maintenance mode thing in Ironic | 07:42 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Compute all hosts matrix once https://review.openstack.org/97703 | 07:43 |
| *** pblaho has joined #tripleo | 07:45 | |
| SpamapS | https://jenkins04.openstack.org/job/check-tripleo-ironic-seed-precise/885/console ... build running.. by the time I get a new devtest up it will have validated or rejected my template :-P | 07:46 |
| lifeless | you should be able to just stack-update an existing devtest | 07:47 |
| SpamapS | lifeless: that would assume I have an existing devtest | 07:48 |
| lifeless | aye | 07:48 |
| SpamapS | probably worth building a new one | 07:48 |
| SpamapS | I have old images | 07:48 |
| *** andreaf has quit IRC | 07:55 | |
| *** derekh_ has joined #tripleo | 07:59 | |
| *** ifarkas has joined #tripleo | 08:00 | |
| derekh_ | hmm, ci jobs failing all over the place for the last 8 hours | 08:02 |
| *** shakamunyi has quit IRC | 08:10 | |
| *** jistr has joined #tripleo | 08:10 | |
| *** athomas has joined #tripleo | 08:12 | |
| *** jml__ has joined #tripleo | 08:16 | |
| *** jml__ has quit IRC | 08:20 | |
| SpamapS | derekh_: I've seen this twice now: | 08:23 |
| SpamapS | 2014-06-04 08:18:08.094 | | 5953041e-d598-4a25-90f4-d80c24bb994b | undercloud-undercloud-esu4mnhbtscz | ERROR | - | NOSTATE | | | 08:23 |
| SpamapS | derekh_: where the undercloud node failed to even provision | 08:23 |
| SpamapS | lifeless: the overcloud job worked with my modified template | 08:23 |
| lifeless | SpamapS: \o/ ship it | 08:24 |
| derekh_ | SpamapS: ya, running it locally now to see if I can get more details | 08:24 |
| SpamapS | weird | 08:26 |
| SpamapS | seed logs only show his | 08:26 |
| SpamapS | 2014-06-04 08:15:36.778 4685 WARNING nova.scheduler.driver [req-4eea42fc-e949-43c3-83ea-7a4a3c7c4de4 None] [instance: 5953041e-d598-4a25-90f4-d80c24bb994b] Setting instance to ERROR state. | 08:26 |
| *** ramishra has quit IRC | 08:26 | |
| *** lucasagomes has joined #tripleo | 08:26 | |
| SpamapS | but no reason given | 08:26 |
| *** pelix has joined #tripleo | 08:27 | |
| mordred | SpamapS: reasons are for wussies | 08:28 |
| SpamapS | wussies are the reason | 08:28 |
| mordred | wussies are in season | 08:28 |
| giulivo | I second SpamapS on wussies being the reason, generally speaking | 08:28 |
| derekh_ | Jun 04 04:35:15 host-192-168-1-117 neutron-dhcp-agent[2987]: Command: ['sudo', '/usr/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-0cedc0c1-8810-4d19-b06c-3129a1b69dd4', 'ip' | 08:28 |
| derekh_ | Jun 04 04:35:15 host-192-168-1-117 neutron-dhcp-agent[2987]: Stderr: 'Cannot open network namespace "qdhcp-0cedc0c1-8810-4d19-b06c-3129a1b69dd4": No such file or directory\n' | 08:28 |
| SpamapS | wwhhaaaaa | 08:29 |
| *** markmc has joined #tripleo | 08:30 | |
| SpamapS | derekh_: same error here actually | 08:30 |
| SpamapS | derekh_: so.. new neutron bug maybe? | 08:30 |
| SpamapS | that would cause PXE boot error among other things | 08:30 |
| SpamapS | https://bugs.launchpad.net/neutron/+bug/1286209 might be related | 08:32 |
| uvirtbot | Launchpad bug 1286209 in neutron "unhandled trace if no namespaces in metering agent" [Undecided,Confirmed] | 08:32 |
| derekh_ | SpamapS: possibly | 08:33 |
| SpamapS | derekh_: if there's a race between dhcp agent and l3 agent, that might be the issue | 08:33 |
| SpamapS | on overclouds.. | 08:34 |
| SpamapS | metadata and l3 are started immediately after dhcp | 08:35 |
| derekh_ | I don't see any neutron merges that would fit the timeline, failures seem to have started around 1 or 2 AM UTC | 08:35 |
| *** IvanBerezovskiy1 has joined #tripleo | 08:35 | |
| SpamapS | on undercloud, there is no l3 or metadata agent | 08:35 |
| SpamapS | but openvswitch agent starts right after dhcp | 08:36 |
| openstackgerrit | A change was merged to openstack/tuskar: Temporary fix for SQLite compatibility https://review.openstack.org/94651 | 08:37 |
| SpamapS | I would actually expect neutron-dhcp-agent to create the netns | 08:37 |
| derekh_ | gent.dhcp_agent Timeout: Timeout while waiting on RPC response - topic: "q-plugin", RPC method: "report_state" info: "<unknown>" | 08:37 |
| *** e0ne has joined #tripleo | 08:38 | |
| *** matsuhashi has quit IRC | 08:42 | |
| lifeless | devananda: https://bugs.launchpad.net/ironic/+bug/1326279 | 08:43 |
| uvirtbot | Launchpad bug 1326279 in ironic "nova operations can lead to nodes entering maintenance mode" [Undecided,New] | 08:43 |
| jml | lifeless: hi. just lurking :) | 08:43 |
| lifeless | jml: :) | 08:43 |
| SpamapS | oh weird.. dhcp agent doesn't ensure_namespace | 08:43 |
| lifeless | SpamapS: ahahahahaahah BUGS | 08:43 |
| SpamapS | openvswitch-agent does it I think | 08:43 |
| lifeless | we need a cross service dependency I guess? | 08:44 |
| SpamapS | yeah we might just need to order things | 08:44 |
| SpamapS | and it may just be a race that got worse or something | 08:44 |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Add passthrough configuration glue. https://review.openstack.org/91810 | 08:46 |
| mordred | SpamapS: I'm going to punch you in the face | 08:46 |
| mordred | SpamapS: there is a class in heat I want to use ... but it's eversoslightly tied too closely to heat | 08:46 |
| mordred | SpamapS: so I think it would wind up being weird | 08:47 |
| mordred | SpamapS: which means I'm going to blame you | 08:47 |
| mordred | SpamapS: and punch you in the face | 08:47 |
| SpamapS | mordred: I'm quakin in my little space boots | 08:47 |
| mordred | SpamapS: OpenStackClients | 08:47 |
| SpamapS | mordred: oh yeah that one is nice | 08:47 |
| SpamapS | It probably started out entirely generic | 08:48 |
| mordred | SpamapS: awesome class - because, you know, figuring out how the heck to interact with services in python is impossible | 08:48 |
| SpamapS | but over time, in tree.. it rots | 08:48 |
| mordred | SpamapS: _get_client_option | 08:48 |
| mordred | is the thing that kills it | 08:48 |
| *** pblaho has quit IRC | 08:48 | |
| *** mrunge has quit IRC | 08:48 | |
| *** sballe has quit IRC | 08:48 | |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** IvanBerezovskiy1 has quit IRC | 08:48 | |
| *** goneri has quit IRC | 08:48 | |
| *** pblaho has joined #tripleo | 08:48 | |
| *** IvanBerezovskiy has joined #tripleo | 08:48 | |
| * mordred cries | 08:48 | |
| *** goneri has joined #tripleo | 08:49 | |
| *** mrunge has joined #tripleo | 08:49 | |
| *** IvanBerezovskiy has left #tripleo | 08:49 | |
| *** sballe has joined #tripleo | 08:49 | |
| mordred | SpamapS: of course, I suppose I could just cut and paste it... | 08:49 |
| *** IvanBerezovskiy has joined #tripleo | 08:49 | |
| SpamapS | mordred: that looks like it wants to be in oslo somewhere | 08:49 |
| *** matsuhashi has joined #tripleo | 08:50 | |
| mordred | yah | 08:50 |
| mordred | or, in python-openstacksdk | 08:50 |
| SpamapS | even the _get_client_option is only tied to heat because thats where the things are defined.. but that could be entirely out of heat and still used in an identical way | 08:50 |
| *** openstackgerrit has joined #tripleo | 08:50 | |
| SpamapS | mordred: good luck with that | 08:51 |
| * SpamapS puts some meat on his black eye | 08:51 | |
| mordred | SpamapS: why would I need to pass in service_type='image' to glanceclient.Client ? | 08:51 |
| mordred | SpamapS: I mean, wouldn't it - you know - just be image? | 08:51 |
| SpamapS | mordred: soon, it will be images, templates, murano Apps.. solum "things" | 08:52 |
| * mordred punches someone | 08:52 | |
| SpamapS | derekh_: I'm going to submit a workaround patch which just makes sure ovs agent starts before dhcp agent | 08:53 |
| *** e0ne has quit IRC | 08:53 | |
| SpamapS | derekh_: I hypothesize that this is just one of those times where something non-obvious has sped up or slowed down one of the racers and made the race less predictable. | 08:54 |
| *** lokesh184 has joined #tripleo | 08:54 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Revert : Revert "Convert address to str in fixed_ip_obj.associate" https://review.openstack.org/97717 | 08:54 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: Updates context module from oslo-incubator https://review.openstack.org/97718 | 08:55 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: Updates importutils module from oslo-incubator https://review.openstack.org/97719 | 08:55 |
| openstackgerrit | Petr Blaho proposed a change to openstack/tuskar: Updates gettextutils module from oslo-incubator https://review.openstack.org/97720 | 08:55 |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-image-elements: Ensure neutron-openvswitch-agent starts first https://review.openstack.org/97721 | 08:55 |
| derekh_ | SpamapS: ok cool, I'm testing a revert of the only commit I see that look remotly relevant ^^ (no real reason to suspect it) | 08:55 |
| *** e0ne has joined #tripleo | 08:57 | |
| SpamapS | derekh_: if 97721 passes, I'll recheck it a couple more times | 08:57 |
| derekh_ | SpamapS: the queue is fairly short so they shouldn't be stuck in it all day | 08:57 |
| derekh_ | SpamapS: sounds good | 08:57 |
| SpamapS | Indeed \o/ for that | 08:58 |
| *** jp_at_hp has joined #tripleo | 08:58 | |
| derekh_ | Everybody, it would help things if you held off on pushing to gerrit while we try and get a fix through the queue | 08:58 |
| SpamapS | derekh_: but, don't go shoutin about it :) | 08:58 |
| derekh_ | SpamapS: does that patch change anything? | 08:59 |
| *** untriaged-bot has joined #tripleo | 09:00 | |
| untriaged-bot | Untriaged bugs so far: | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1326289 | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1288178 | 09:00 |
| uvirtbot | Launchpad bug 1326289 in tripleo "Failing to launch instances : Cannot open network namespace" [Undecided,New] | 09:00 |
| uvirtbot | Launchpad bug 1288178 in nova "Sync new policy from oslo" [High,In progress] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 09:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 09:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1326016 | 09:00 |
| uvirtbot | Launchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,New] | 09:00 |
| *** untriaged-bot has quit IRC | 09:00 | |
| derekh_ | SpamapS: ok, I see it, gerrit doesn't make the rename obvious | 09:01 |
| *** andreaf has joined #tripleo | 09:05 | |
| *** shakamunyi has joined #tripleo | 09:06 | |
| *** SpamapS has quit IRC | 09:08 | |
| *** pblaho has quit IRC | 09:09 | |
| *** shakamunyi has quit IRC | 09:10 | |
| *** rcarrillocruz has joined #tripleo | 09:11 | |
| *** rcarrill` has quit IRC | 09:11 | |
| *** SpamapS has joined #tripleo | 09:11 | |
| SpamapS | derekh_: it doesn't? | 09:13 |
| shadower | SpamapS, derekh_: in the meantime, should we just do "recheck no bug" or is this filed somewhere? | 09:13 |
| derekh_ | shadower: its filed, one sec | 09:13 |
| SpamapS | It says renamed on the summary and detail pages. :-P | 09:14 |
| derekh_ | shadower: https://bugs.launchpad.net/tripleo/+bug/1326289 but don't bother rechecking until its fixed | 09:14 |
| uvirtbot | Launchpad bug 1326289 in tripleo "Failing to launch instances : Cannot open network namespace" [Undecided,New] | 09:14 |
| shadower | derekh_: ok thanks | 09:15 |
| derekh_ | SpamapS: ya ya | 09:15 |
| SpamapS | derekh_: doh, I should have referenced that | 09:17 |
| SpamapS | derekh_: well instead of a recheck I'll push that after it succeeds. | 09:17 |
| derekh_ | me too ... | 09:18 |
| SpamapS | derekh_: if they both fix it.. I think we can land my t-i-e workaround, and push the revert back up to nova | 09:19 |
| SpamapS | but I'm really skeptical of the nova revert | 09:19 |
| SpamapS | since our problem is that dhcp agent fails to start | 09:19 |
| giulivo | anyone running on F20 and seeing this: "unsupported configuration: SATA is not supported with this QEMU binary" ? | 09:19 |
| openstackgerrit | Yongli He proposed a change to openstack/tuskar: Rewrite nova policy to use the new changes of common policy https://review.openstack.org/97729 | 09:20 |
| giulivo | which basically prevents any VM (including seed) to start? | 09:20 |
| derekh_ | SpamapS: yup sounds good, as I say I have no reason really to suspect it just matching timelines | 09:20 |
| SpamapS | derekh_: should be interesting :) | 09:20 |
| * SpamapS stares at https://jenkins05.openstack.org/job/check-tripleo-undercloud-precise/918/console and wills it to GO FASTURRRR | 09:21 | |
| SpamapS | sleep comes once I get a result | 09:21 |
| SpamapS | I can't sleep until I have data to dream on | 09:21 |
| openstackgerrit | Yongli He proposed a change to openstack/tuskar: Rewrite policy to use the new changes of common policy https://review.openstack.org/97729 | 09:22 |
| *** ramishra has joined #tripleo | 09:23 | |
| *** lsmola has joined #tripleo | 09:24 | |
| lifeless | giulivo: thats odd :) | 09:30 |
| lifeless | giulivo: but I think I've heard it before, perhaps check google? | 09:31 |
| derekh_ | SpamapS: ok, ran it in debug mode and looks like my node is being filtered out by ComputeCapabilitiesFilter http://paste.openstack.org/raw/82736/ | 09:31 |
| giulivo | lifeless, hehe indeed, I was looking into a recent change which aimed at making the bus type configurable | 09:31 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Revert : Remove traces of now unused host capabilities from scheduler https://review.openstack.org/97732 | 09:31 |
| derekh_ | SpamapS: so ^ | 09:31 |
| giulivo | but doesn't seem to affect this as it was set to sata by default before that so ... | 09:31 |
| openstackgerrit | Ramakrishnan G proposed a change to openstack/diskimage-builder: Add dhcp support for ramdisk element. https://review.openstack.org/97301 | 09:31 |
| derekh_ | that one looks relevant at least :-) | 09:31 |
| SpamapS | derekh_: well THAT looks more likely | 09:31 |
| lifeless | giulivo: we'll need to make some improvements to ironic to use arbitrary disk buses | 09:32 |
| SpamapS | derekh_: and I was just watching and it failed way too fast to be DHCP failing | 09:32 |
| SpamapS | derekh_: frustrating because I looked at that patch and it looked like it was taking nova-bm into account. | 09:33 |
| giulivo | lifeless, I don't have much hardware to play with ironic yet though | 09:33 |
| *** ramishra has quit IRC | 09:33 | |
| SpamapS | derekh_: I'm 99% sure that's our problem | 09:34 |
| derekh_ | SpamapS: you did more then me so | 09:34 |
| SpamapS | derekh_: damn, a 'check experimental' would have revealed this I think | 09:34 |
| derekh_ | SpamapS: ya looks likely, will propose a revert to nova if it passes | 09:34 |
| SpamapS | and was quite the prudent thing to do | 09:34 |
| derekh_ | SpamapS: yup, | 09:34 |
| lifeless | giulivo: hmm, applies to vms too :) | 09:35 |
| SpamapS | ok, sleep | 09:35 |
| lifeless | giulivo: its the ephemeral mapping dev name | 09:35 |
| SpamapS | data received | 09:35 |
| SpamapS | derekh_: thanks! | 09:35 |
| derekh_ | SpamapS: no prob | 09:36 |
| *** shakamunyi has joined #tripleo | 09:36 | |
| giulivo | lifeless, wait maybe I'm still missing some pieces of the puzzle on the topic ... if you meant to do that with VMs, how is Ironic affected by the disk bus type? | 09:37 |
| lifeless | giulivo: because the bus type defines the device name | 09:37 |
| lifeless | and the device name is needed by the ephemeral mapping metadata | 09:37 |
| giulivo | oh okay, that is the piece I'm missing I think ... ephemeral mapping metadata? | 09:38 |
| *** ramishra has joined #tripleo | 09:40 | |
| *** shakamunyi has quit IRC | 09:41 | |
| *** e0ne has quit IRC | 09:43 | |
| *** akrivoka has joined #tripleo | 09:43 | |
| *** e0ne has joined #tripleo | 09:43 | |
| *** e0ne_ has joined #tripleo | 09:44 | |
| lifeless | giulivo: nova exports the device path that the ephemeral partition -where we store state- is | 09:46 |
| lifeless | e.g. /dev/sda1 | 09:46 |
| giulivo | ah, now I see that too | 09:48 |
| *** e0ne has quit IRC | 09:48 | |
| giulivo | so that would break if we were to switch to other bus types | 09:48 |
| giulivo | (well, to non sdX) | 09:48 |
| lifeless | right | 09:49 |
| *** martyntaylor has joined #tripleo | 09:50 | |
| *** StevenK has quit IRC | 09:53 | |
| *** StevenK has joined #tripleo | 09:53 | |
| *** jcoufal has joined #tripleo | 10:01 | |
| *** nosnos has quit IRC | 10:08 | |
| *** dtantsur is now known as dtantsur|brb | 10:11 | |
| *** athomas has quit IRC | 10:12 | |
| openstackgerrit | Ramakrishnan G proposed a change to openstack/tripleo-specs: Add design spec for deploy-ironic-bm element https://review.openstack.org/97736 | 10:13 |
| *** ramishra has quit IRC | 10:17 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Separate Heat BM and VM configs for Nova-BM. https://review.openstack.org/90033 | 10:18 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Make a separation between --heat-env https://review.openstack.org/93091 | 10:18 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 10:19 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-heat-templates: Remove the tuskar-source template https://review.openstack.org/97739 | 10:20 |
| *** athomas has joined #tripleo | 10:21 | |
| *** e0ne_ has quit IRC | 10:22 | |
| *** matsuhashi has quit IRC | 10:24 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Allow users to use swift out of the box https://review.openstack.org/92391 | 10:24 |
| *** xuhaiwei has quit IRC | 10:28 | |
| *** bcrochet|g0ne is now known as bcrochet | 10:33 | |
| *** sballe has quit IRC | 10:35 | |
| *** shakamunyi has joined #tripleo | 10:37 | |
| lifeless | derekh_: hi | 10:38 |
| lifeless | derekh_: so, I brought up an undercloud | 10:38 |
| lifeless | derekh_: and am trying to identify broken machines | 10:39 |
| lifeless | derekh_: I've now got several stuck instancse likes 9df33a91-4be3-429c-a712-35cd60a1e101 | 10:39 |
| derekh_ | lifeless: ok, anything I can do to help? | 10:39 |
| lifeless | derekh_: note down the node uuids that failed, delete all the instances, poke and file bugs if they don't delete cleanly, then do a nova boot matched to capacity | 10:40 |
| lifeless | derekh_: there's a screen session (screen -xr to join) | 10:40 |
| *** sballe has joined #tripleo | 10:40 | |
| lifeless | derekh_: record the node uuids in the etherpad that are bad, rinse and repeat until we can reliably deploy the available non-maintenance-mode capacity | 10:41 |
| lifeless | derekh_: then we translate the node uuids back to IPMI addresses, and I'll get dcops tickets filed on the bad ones | 10:41 |
| *** shakamunyi has quit IRC | 10:41 | |
| lifeless | derekh_: and we'll have hopefully 40 odd machines to build the cloud from | 10:42 |
| lifeless | derekh_: I'm going to pinch one tomorrow for load testing | 10:42 |
| derekh_ | lifeless: cool will do | 10:42 |
| *** sballe_ has joined #tripleo | 10:42 | |
| derekh_ | lifeless: which etherpad | 10:43 |
| lifeless | derekh_: https://etherpad.openstack.org/p/tripleo-ci-hp1-rebuild | 10:43 |
| derekh_ | lifeless: ok | 10:43 |
| derekh_ | lifeless: should I be sshing to the basion ? what user? | 10:45 |
| derekh_ | lifeless: got it | 10:45 |
| *** sballe has quit IRC | 10:45 | |
| lifeless | derekh_: stack@bastion.hp1.tripleo.org | 10:46 |
| lifeless | (I think I put that in the cloud admin spreadsheet FWIW) | 10:47 |
| lifeless | then screen -xr | 10:47 |
| *** yamahata has quit IRC | 10:47 | |
| derekh_ | yup, I'm in, thanks | 10:47 |
| *** matsuhashi has joined #tripleo | 10:47 | |
| *** sballe_ has quit IRC | 10:48 | |
| *** SpamapS has quit IRC | 10:50 | |
| *** akrivoka has quit IRC | 10:51 | |
| derekh_ | lifeless: no tables in the nova_bm db, am I looking in the wrong place | 10:51 |
| *** SpamapS has joined #tripleo | 10:52 | |
| lifeless | derekh_: yes :) | 10:52 |
| derekh_ | lifeless: ok, switching to ironic mode | 10:52 |
| derekh_ | lifeless: havn't used ironic really, so will be a bit of a learning experiance :-) | 10:53 |
| lifeless | oy vey :) | 10:54 |
| lifeless | look at register-nodes for a primer | 10:54 |
| derekh_ | lifeless: if your still there (or anybody else) , I think we should merge https://review.openstack.org/#/c/97732/1 , I'll update the bug and propose the revert to nova | 10:59 |
| derekh_ | undercloud failed because of a net problem | 10:59 |
| lifeless | but it got further ? | 10:59 |
| derekh_ | lifeless: nope, it didn't get to the deploy phase (failed building undercloud image) | 11:00 |
| derekh_ | lifeless: but this is the first overcloud job that has passed in about 10 hours | 11:00 |
| lifeless | derekh_: where was it failing without the revert? | 11:00 |
| lifeless | oh | 11:00 |
| lifeless | so yes, it got further ;) | 11:00 |
| openstackgerrit | Eamonn O'Toole proposed a change to openstack/tripleo-heat-templates: Add SwiftDeviceRange parameter to Swift yaml templates https://review.openstack.org/97752 | 11:01 |
| derekh_ | lifeless: sorry yes overcloud job got further, undercloud failed for other reason | 11:01 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Add revert hook point https://review.openstack.org/94156 | 11:01 |
| lifeless | derekh_: I agree; have +2 it | 11:02 |
| derekh_ | lifeless: ok thanks, any other core https://review.openstack.org/#/c/97732/1 | 11:02 |
| lifeless | derekh_: you can do it | 11:03 |
| derekh_ | lifeless: done | 11:03 |
| lifeless | its why we have the exceptions in the review guidelines | 11:03 |
| derekh_ | okie dokie | 11:03 |
| openstackgerrit | A change was merged to openstack-infra/tripleo-ci: Revert : Remove traces of now unused host capabilities from scheduler https://review.openstack.org/97732 | 11:04 |
| *** ramishra has joined #tripleo | 11:05 | |
| openstackgerrit | Eamonn O'Toole proposed a change to openstack/tripleo-heat-templates: Add metadata and change swift.devices to enable zones https://review.openstack.org/97758 | 11:09 |
| *** sdague has left #tripleo | 11:09 | |
| *** lokesh184 has quit IRC | 11:13 | |
| *** loki184 has joined #tripleo | 11:16 | |
| openstackgerrit | Robert Parker proposed a change to openstack/tripleo-heat-templates: Setup SSL for Ceilometer https://review.openstack.org/96257 | 11:17 |
| *** rameshg87 has left #tripleo | 11:17 | |
| *** ramishra has quit IRC | 11:21 | |
| *** ramishra has joined #tripleo | 11:21 | |
| *** dtantsur|brb is now known as dtantsur | 11:23 | |
| *** matsuhashi has quit IRC | 11:25 | |
| *** matsuhashi has joined #tripleo | 11:26 | |
| *** ramishra has quit IRC | 11:26 | |
| *** matsuhashi has quit IRC | 11:29 | |
| *** matsuhashi has joined #tripleo | 11:29 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/diskimage-builder: Export IMAGE_ELEMENT after finding its value https://review.openstack.org/97654 | 11:30 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-specs: Deploy cloud hypervisor type https://review.openstack.org/94586 | 11:33 |
| openstackgerrit | Robert Parker proposed a change to openstack/tripleo-heat-templates: Setup SSL for Ceilometer https://review.openstack.org/96257 | 11:33 |
| *** shakamunyi has joined #tripleo | 11:38 | |
| robsparker | notify I | 11:39 |
| TheJulia | win 7 | 11:40 |
| *** yamahata has joined #tripleo | 11:41 | |
| *** loki184 has quit IRC | 11:42 | |
| lifeless | derekh_: with nova delete you don't need -n 1 | 11:42 |
| lifeless | it takes n-args | 11:42 |
| lifeless | faster, less token creation | 11:42 |
| *** shakamunyi has quit IRC | 11:43 | |
| derekh_ | lifeless: ok, thanks | 11:43 |
| pbelanyi | rdopiera: should we add the jshint test for tuskar-ui as voting to the gate? since you've fixed all issues that it had | 11:43 |
| pbelanyi | (and is this the right channel for tuskar-ui stuff, now that #tuskar is supposed to be dead?) | 11:43 |
| rdopiera | pbelanyi: yeah, but I would wait a little, to see how well it handles things | 11:47 |
| *** lucasagomes is now known as lucas-hungry | 11:47 | |
| pbelanyi | rdopiera: okay, then I'll leave the patch as it is, and change it when we see that it actually works fine | 11:48 |
| rdopiera | pbelanyi: you could start working on a similar patch for horizon :) | 11:49 |
| *** yamahata has quit IRC | 11:50 | |
| *** yamahata has joined #tripleo | 11:50 | |
| pbelanyi | rdopiera: I already have this https://review.openstack.org/#/c/97237/ | 11:50 |
| pbelanyi | and I want to get it through before trying to add it to infra | 11:51 |
| rdopiera | pbelanyi: seems like Tatiana didn't see your change to the commit message | 11:52 |
| pbelanyi | rdopiera: yeah, maybe I'll ping her, but jenkins is nacking it too :/ | 11:53 |
| rdopiera | pbelanyi: http://logs.openstack.org/37/97237/1/check/gate-horizon-requirements/3936e9c/console.html | 11:54 |
| pbelanyi | rdopiera: 'Requirement nodeenv not in openstack/requirements'... but then how did it work for tuskar-ui? | 11:56 |
| *** e0ne has joined #tripleo | 11:56 | |
| rdopiera | pbelanyi: it doesn't have that check | 12:00 |
| *** weshay has joined #tripleo | 12:00 | |
| rdopiera | pbelanyi: I would ask around in #openstack-infra | 12:01 |
| pbelanyi | rdopiera: it seems we'd have to add nodeenv to openstack/requirements https://wiki.openstack.org/wiki/Requirements | 12:02 |
| *** e0ne_ has joined #tripleo | 12:03 | |
| *** tzumainn has joined #tripleo | 12:04 | |
| *** e0ne has quit IRC | 12:05 | |
| *** morazi has joined #tripleo | 12:06 | |
| *** akrivoka has joined #tripleo | 12:08 | |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-heat-templates: Fix incorrect indentation for bootstack_nodeid https://review.openstack.org/97770 | 12:16 |
| *** jistr is now known as jistr|eng | 12:18 | |
| *** ramishra has joined #tripleo | 12:21 | |
| *** jistr|mobi has joined #tripleo | 12:21 | |
| *** jdob has joined #tripleo | 12:22 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Allow install mariadb from RDO repository https://review.openstack.org/87295 | 12:23 |
| *** ddieterly has quit IRC | 12:27 | |
| *** ddieterly has joined #tripleo | 12:27 | |
| jang | hi guys | 12:30 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Separate Nova Compute drivers https://review.openstack.org/97772 | 12:35 |
| openstackgerrit | James Slagle proposed a change to openstack/tripleo-image-elements: Add nova-docker element https://review.openstack.org/97773 | 12:35 |
| *** lazy_prince has quit IRC | 12:38 | |
| *** shakamunyi has joined #tripleo | 12:39 | |
| *** shakamunyi has quit IRC | 12:44 | |
| openstackgerrit | Jan Provaznik proposed a change to openstack/tripleo-image-elements: Explicitly specify mysql node address https://review.openstack.org/97775 | 12:46 |
| *** dprince has joined #tripleo | 12:49 | |
| *** ddieterly has quit IRC | 12:58 | |
| giulivo | lifeless, is the purpose of REMOTE_OPERATIONS to run devtest on a system which is not the one hosting the seed/undercloud/overcloud VMs ? | 13:00 |
| *** lucas-hungry is now known as lucasagomes | 13:02 | |
| *** shakamunyi has joined #tripleo | 13:03 | |
| *** ramishra has quit IRC | 13:07 | |
| *** mrunge has quit IRC | 13:09 | |
| *** markmc has quit IRC | 13:09 | |
| *** ramishra has joined #tripleo | 13:10 | |
| openstackgerrit | Tom Hancock proposed a change to openstack/diskimage-builder: Add missing dollar https://review.openstack.org/97784 | 13:17 |
| slagle | giulivo: yes. CI uses that | 13:17 |
| slagle | giulivo: in theory, it should work for manual devtest runs as well, although i haven't tried it | 13:17 |
| giulivo | slagle, thanks! I'm trying to use it too now | 13:18 |
| *** julim has joined #tripleo | 13:18 | |
| giulivo | manual devtest :) | 13:18 |
| slagle | cool | 13:18 |
| *** jistr|eng is now known as jistr | 13:20 | |
| *** jistr|mobi has quit IRC | 13:20 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Add an overview diagram to the userguide https://review.openstack.org/96428 | 13:20 |
| openstackgerrit | Andrea Rosa proposed a change to openstack/tripleo-image-elements: Specify nova configuration file with --config-dir https://review.openstack.org/97785 | 13:26 |
| *** ddieterly has joined #tripleo | 13:29 | |
| StevenK | jistr: Hai, do you want to +A a os-cloud-config docs change that finally passed CI? https://review.openstack.org/#/c/97653/ | 13:33 |
| *** matty_dubs|gone is now known as matty_dubs | 13:33 | |
| jistr | StevenK: done. i already looked at it before but didn't +2 because it had enough of those :) | 13:34 |
| StevenK | jistr: Haha, thanks. :-) | 13:35 |
| *** jprovazn has quit IRC | 13:37 | |
| *** vinsh has quit IRC | 13:38 | |
| dprince | derekh: Are you looking into the Nova issues around the scheduler revert https://review.openstack.org/#/c/97732/1/toci_devtest.sh | 13:40 |
| *** vinsh has joined #tripleo | 13:41 | |
| derekh_ | dprince: https://bugs.launchpad.net/tripleo/+bug/1326289 and https://review.openstack.org/#/c/97757/ | 13:45 |
| uvirtbot | Launchpad bug 1326289 in tripleo "Failing to launch instances : Filter ComputeCapabilitiesFilter returned 0 hosts" [Critical,Triaged] | 13:45 |
| derekh_ | dprince: mentioned it on #nova earlier aswell, will pop in there now again | 13:46 |
| dprince | derekh_: sent | 13:46 |
| dprince | derekh_: thanks, that was what I was looking for :) | 13:46 |
| *** ramishra has quit IRC | 13:47 | |
| derekh_ | dprince: no prob, thanks | 13:47 |
| *** ramishra has joined #tripleo | 13:47 | |
| openstackgerrit | A change was merged to openstack/os-cloud-config: Fix niggles with the documentation https://review.openstack.org/97653 | 13:48 |
| *** ramishra has quit IRC | 13:52 | |
| TheJulia | /win 6 | 13:52 |
| TheJulia | doh | 13:52 |
| *** jdob has quit IRC | 13:53 | |
| *** e0ne_ has quit IRC | 13:56 | |
| *** jdob has joined #tripleo | 13:56 | |
| *** e0ne has joined #tripleo | 13:57 | |
| *** jprovazn has joined #tripleo | 14:01 | |
| *** jtomasek has quit IRC | 14:05 | |
| *** jtomasek has joined #tripleo | 14:08 | |
| *** funzo has quit IRC | 14:09 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Add svc-map element. https://review.openstack.org/96770 | 14:12 |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Configure passthrough in swift config files https://review.openstack.org/92628 | 14:15 |
| *** funzo has joined #tripleo | 14:16 | |
| *** pblaho has joined #tripleo | 14:16 | |
| *** rpodolyaka1 has joined #tripleo | 14:16 | |
| openstackgerrit | Tom Hancock proposed a change to openstack/tripleo-incubator: Rebuild, but dont reconfigure undercloud services https://review.openstack.org/97799 | 14:18 |
| *** rakesh_hs has quit IRC | 14:19 | |
| jp_at_hp | derekh_: can you be more specific about what your use case is from your comments on https://review.openstack.org/#/c/85418/ please? | 14:21 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Name disable-requiretty files uniquely https://review.openstack.org/94686 | 14:23 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add unit test to enforce unique file names. https://review.openstack.org/94705 | 14:23 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Name 03-mariadb files uniquely https://review.openstack.org/97801 | 14:23 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Name 03-mariadb files uniquely https://review.openstack.org/97801 | 14:24 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Add unit test to enforce unique file names. https://review.openstack.org/94705 | 14:24 |
| *** rdopiera has quit IRC | 14:28 | |
| derekh_ | jp_at_hp: iirc I changed tripleo-incubator to a branch called pp , then I cherry-picked the patch into it and ran a full dev test (I forget if I did it with -c or not), if you can't reproduce it I can try set it up again and reproduce | 14:29 |
| derekh_ | How do you put a ironic node into maintenance mode ? | 14:31 |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Store swift data, db & objects on /mnt filesystem https://review.openstack.org/89847 | 14:38 |
| *** morazi has quit IRC | 14:39 | |
| *** jprovazn is now known as jprovazn_afk | 14:40 | |
| *** rwsu has joined #tripleo | 14:40 | |
| *** andreaf has quit IRC | 14:41 | |
| openstackgerrit | Gerry Drudy proposed a change to openstack/tripleo-image-elements: Move swift logs to logs in /var/log/swift https://review.openstack.org/91533 | 14:42 |
| *** matsuhashi has quit IRC | 14:49 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Compute all hosts matrix once https://review.openstack.org/97703 | 14:49 |
| *** sballe has joined #tripleo | 14:50 | |
| *** morazi has joined #tripleo | 14:51 | |
| *** sballe_ has joined #tripleo | 14:52 | |
| *** sballe has quit IRC | 14:56 | |
| *** sballe__ has joined #tripleo | 14:57 | |
| *** ramishra has joined #tripleo | 14:58 | |
| *** untriaged-bot has joined #tripleo | 15:00 | |
| untriaged-bot | Untriaged bugs so far: | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326336 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326337 | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tuskar/+bug/1288178 | 15:00 |
| uvirtbot | Launchpad bug 1326336 in diskimage-builder "Do You Know These Things About Your Babys Skin" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326387 | 15:00 |
| uvirtbot | Launchpad bug 1326337 in diskimage-builder "Skin Care Basics That Everyone" [Undecided,New] | 15:00 |
| uvirtbot | Launchpad bug 1288178 in nova "Sync new policy from oslo" [High,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326391 | 15:00 |
| uvirtbot | Launchpad bug 1326387 in diskimage-builder "Tips For A Healthy And Shining Skin" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1326016 | 15:00 |
| uvirtbot | Launchpad bug 1326391 in diskimage-builder "All Natural Skin Care Treatments For Your Particular Skin Type" [Undecided,New] | 15:00 |
| uvirtbot | Launchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,In progress] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326357 | 15:00 |
| uvirtbot | Launchpad bug 1326357 in diskimage-builder "Treatment Options For Skin Cancer" [Undecided,New] | 15:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326319 | 15:00 |
| uvirtbot | Launchpad bug 1326319 in diskimage-builder "Add support for deploying for ironic from baremetal node" [Undecided,New] | 15:00 |
| *** sballe_ has quit IRC | 15:00 | |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 15:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 15:00 |
| *** untriaged-bot has quit IRC | 15:00 | |
| slagle | lol, silly diskimage-builder | 15:01 |
| *** ramishra has quit IRC | 15:03 | |
| *** sballe__ has quit IRC | 15:06 | |
| *** martyntaylor has quit IRC | 15:06 | |
| jp_at_hp | derekh_: so it is a local branch with no remote. So I think the change is that the best effort is applied, and otherwise we don't bail. Sound reasonable? | 15:08 |
| derekh_ | jp_at_hp: yup, sounds good to me, it should include the commit has at least | 15:11 |
| derekh_ | *hash | 15:11 |
| jp_at_hp | i'll take a look | 15:12 |
| jp_at_hp | it's a thorny problem, as it is intended more for ci bots than for transient developer scenarios | 15:12 |
| jp_at_hp | I think it also shows a weakness in the source-repositories input file, but that's probably for another day | 15:12 |
| * derekh_ put a lot of thought into that :-( | 15:13 | |
| *** spiffxp- has quit IRC | 15:13 | |
| derekh_ | I have a sudden urge to go out and buy some skin cream | 15:14 |
| *** SlickNik has quit IRC | 15:14 | |
| *** SlickN1k is now known as SlickNik | 15:14 | |
| jp_at_hp | derekh_: oops - sorry ! I may be just thinking about the git case and not all of the other cases it caters for ;-) | 15:14 |
| * bnemec races slagle to kill the spam bugs :-) | 15:15 | |
| derekh_ | jp_at_hp: I was only joking, its very possible I didn't think of loads of things | 15:15 |
| jp_at_hp | That sounds like something I'd do - not think about things that is :D | 15:16 |
| bnemec | derekh_: So CI should be working again right? Hoping I didn't just trigger a bunch of doomed rechecks... | 15:19 |
| slagle | bnemec: i won, but there's definitely some good info in there about skin care | 15:20 |
| derekh_ | bnemec: yup, your good | 15:21 |
| bnemec | slagle: Heh | 15:22 |
| bnemec | derekh_: Cool, thanks | 15:22 |
| *** nati_ueno has joined #tripleo | 15:23 | |
| *** IvanBerezovskiy has left #tripleo | 15:27 | |
| *** dtantsur is now known as dtantsur|afk | 15:29 | |
| *** olaph has joined #tripleo | 15:30 | |
| *** rcarrill` has joined #tripleo | 15:31 | |
| *** rcarrillocruz has quit IRC | 15:34 | |
| *** cwolferh has joined #tripleo | 15:35 | |
| *** rcarrill` is now known as rcarrillocruz | 15:37 | |
| *** markmc has joined #tripleo | 15:38 | |
| *** rcarrill` has joined #tripleo | 15:41 | |
| *** yamahata has quit IRC | 15:43 | |
| *** rcarrillocruz has quit IRC | 15:43 | |
| *** jistr has quit IRC | 15:51 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/93253 | 15:52 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/96234 | 15:52 |
| openstackgerrit | Giulio Fidente proposed a change to openstack/tripleo-incubator: Improve support for REMOTE_OPERATIONS in manual runs https://review.openstack.org/97839 | 15:58 |
| *** rcarrill` is now known as rcarrillocruz | 15:59 | |
| *** matty_dubs is now known as matty_dubs|lunch | 15:59 | |
| *** ramishra has joined #tripleo | 15:59 | |
| *** nati_ueno has quit IRC | 15:59 | |
| *** markmc has quit IRC | 16:02 | |
| *** e0ne has quit IRC | 16:03 | |
| *** ramishra has quit IRC | 16:04 | |
| *** pblaho has quit IRC | 16:06 | |
| *** ramishra has joined #tripleo | 16:12 | |
| *** ddieterl_ has joined #tripleo | 16:12 | |
| *** ddieterly has quit IRC | 16:14 | |
| *** jcoufal has quit IRC | 16:17 | |
| *** ewindisch has quit IRC | 16:18 | |
| *** ramishra has quit IRC | 16:23 | |
| *** ramishra has joined #tripleo | 16:24 | |
| *** ramishra has quit IRC | 16:27 | |
| *** ramishra has joined #tripleo | 16:27 | |
| *** ramishra has quit IRC | 16:27 | |
| *** ewindisch has joined #tripleo | 16:29 | |
| *** ramishra has joined #tripleo | 16:34 | |
| *** martyntaylor has joined #tripleo | 16:36 | |
| *** matty_dubs|lunch is now known as matty_dubs | 16:37 | |
| derekh_ | lifeless: 3 instance in error state, couldn't delete them (their holding 6 ironic nodes) https://bugs.launchpad.net/ironic/+bug/1326364 | 16:42 |
| uvirtbot | Launchpad bug 1326364 in ironic "Instances assosiated with multiple ironic nodes" [Undecided,Incomplete] | 16:42 |
| derekh_ | lifeless: after debugging that for a bit I did a nova boot of 39 instances, all but 2 are finbished spawning (I suspect they'll timout) | 16:43 |
| derekh_ | lifeless: but currently waiting that out | 16:43 |
| derekh_ | lifeless: mainly because I couldn't figute out how to put a node into maintanance mode | 16:43 |
| *** ewindisch has quit IRC | 16:48 | |
| *** ramishra has quit IRC | 16:50 | |
| *** saurabhs has joined #tripleo | 16:52 | |
| *** derekh_ has quit IRC | 16:56 | |
| d0ugal | jdob: You around? | 16:58 |
| openstackgerrit | Robert Parker proposed a change to openstack/tripleo-incubator: Setting the metering secrets https://review.openstack.org/94201 | 17:01 |
| openstackgerrit | Flint Calvin proposed a change to openstack/tripleo-incubator: Setting the metering secrets https://review.openstack.org/94201 | 17:06 |
| openstackgerrit | Jon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-incubator: Create a manifest of tools pulled https://review.openstack.org/85418 | 17:12 |
| *** shakamunyi has quit IRC | 17:13 | |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-specs: TripleO network configuration https://review.openstack.org/97859 | 17:22 |
| *** lucasagomes is now known as lucas-dinner | 17:24 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-heat-templates: Compute all hosts matrix once https://review.openstack.org/97703 | 17:24 |
| *** jang1 has joined #tripleo | 17:25 | |
| *** jcoufal has joined #tripleo | 17:33 | |
| *** jp_at_hp has quit IRC | 17:33 | |
| *** nati_ueno has joined #tripleo | 17:34 | |
| *** akrivoka has quit IRC | 17:34 | |
| greghaynes | SpamapS: What was the nova-bm fix for the overnight breakage? | 17:34 |
| greghaynes | having a hard time finding in sb | 17:35 |
| greghaynes | oh hey, a lp bug. thats probably helpful | 17:36 |
| greghaynes | ah, so we just have to run with revert for now? | 17:37 |
| dprince | can I get some core eyes on https://review.openstack.org/#/c/97540/1 | 17:37 |
| SpamapS | greghaynes: It will likely look a lot like the ironic fix | 17:37 |
| dprince | ^^ super simple easy fix for the sysctl.d .conf file name | 17:38 |
| SpamapS | greghaynes: https://review.openstack.org/#/c/97447/4 <-- probably could use a patch in Nova if it isn't fixed already | 17:38 |
| greghaynes | looks like https://review.openstack.org/#/c/97806/ | 17:39 |
| *** e0ne has joined #tripleo | 17:40 | |
| SpamapS | dprince: I might let a type-o slide in this case, but not the exact opposite word. ;) | 17:41 |
| greghaynes | haha, I was debating whether I was going to comment on that | 17:42 |
| *** e0ne has quit IRC | 17:42 | |
| greghaynes | glad you did instead | 17:42 |
| *** e0ne has joined #tripleo | 17:43 | |
| *** e0ne has quit IRC | 17:43 | |
| *** eghobo has joined #tripleo | 17:44 | |
| dprince | SpamapS: snap, prefixes... suffixes who cares! | 17:46 |
| * dprince fixes it | 17:46 | |
| nealph | lsmola: https://review.openstack.org/#/c/94730/ finally made it through the guantlet of tests. thanks for the re-check...it took a couple more kicks to get it through.:) | 17:47 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Name sysctl.d files with .conf https://review.openstack.org/97540 | 17:47 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update bootstack to use sysctl-set-value. https://review.openstack.org/84600 | 17:47 |
| dprince | SpamapS: There were no changes on https://review.openstack.org/#/c/84600/ between these patchsets. The last one passed so assuming something stupid doesn't happen it should be fine. I've waited 2 months for that one so what is another day! | 17:51 |
| dprince | SpamapS: thanks for reviewing those though, at least I can (almost) send them once CI passes | 17:54 |
| *** jistr has joined #tripleo | 17:55 | |
| SpamapS | dprince: yeah.. and then you get to wait 14 hours for the gate. :-P | 17:56 |
| *** morazi has quit IRC | 18:04 | |
| *** e0ne has joined #tripleo | 18:04 | |
| *** pelix has quit IRC | 18:08 | |
| *** shakayumi has joined #tripleo | 18:08 | |
| *** shakayumi has quit IRC | 18:08 | |
| *** athomas has quit IRC | 18:08 | |
| *** shakayumi has joined #tripleo | 18:09 | |
| *** shakayumi has quit IRC | 18:09 | |
| *** akuznetsov has joined #tripleo | 18:09 | |
| *** e0ne has quit IRC | 18:11 | |
| *** jprovazn_afk has quit IRC | 18:11 | |
| *** shakayumi has joined #tripleo | 18:18 | |
| *** morazi has joined #tripleo | 18:19 | |
| *** nati_uen_ has joined #tripleo | 18:23 | |
| *** nati_ueno has quit IRC | 18:26 | |
| greghaynes | dprince: youre going to not like me for pointing it out with 3 +3's, but theres a typo on https://review.openstack.org/#/c/84600/ I left a note about | 18:27 |
| greghaynes | er, 3 +2's | 18:27 |
| *** jdob has quit IRC | 18:28 | |
| *** jdob has joined #tripleo | 18:28 | |
| dprince | greghaynes: what can I say... I like it when my 'boots tap' | 18:29 |
| greghaynes | oh, thats better than what I thought. "boot stap!" | 18:29 |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Setup SSL for Ceilometer https://review.openstack.org/96257 | 18:37 |
| openstackgerrit | Dan Prince proposed a change to openstack/tripleo-image-elements: Update bootstack to use sysctl-set-value. https://review.openstack.org/84600 | 18:37 |
| *** jcoufal has quit IRC | 18:42 | |
| openstackgerrit | Ryan Brady proposed a change to openstack/diskimage-builder: Add svc-map element. https://review.openstack.org/96770 | 18:46 |
| lifeless | morning | 18:46 |
| greghaynes | O/ | 18:47 |
| *** dprince has quit IRC | 18:48 | |
| *** akuznetsov has quit IRC | 18:48 | |
| *** e0ne has joined #tripleo | 18:49 | |
| *** akuznetsov has joined #tripleo | 18:49 | |
| *** jistr has quit IRC | 18:55 | |
| openstackgerrit | Michael Tupitsyn proposed a change to openstack/tripleo-incubator: Do not create admin user if it exists already https://review.openstack.org/96307 | 18:57 |
| *** rcarrillocruz has quit IRC | 19:00 | |
| *** rcarrillocruz has joined #tripleo | 19:06 | |
| *** dprince has joined #tripleo | 19:07 | |
| lifeless | hi | 19:10 |
| lifeless | can someone test out https://review.openstack.org/#/c/97806/ with nova-bm (its a roll-forward fix for the regresssion) | 19:11 |
| greghaynes | lol wtf, just got this in a devtest run: http://paste.ubuntu.com/7589619/ | 19:11 |
| greghaynes | I think github is having a hard time today | 19:12 |
| lifeless | greghaynes: repo has been deleted will cause that | 19:12 |
| lifeless | greghaynes: or renamed | 19:12 |
| *** andreaf has joined #tripleo | 19:12 | |
| greghaynes | hrm, well I did get a spurious github 500 in previous run | 19:12 |
| *** rpodolyaka1 has quit IRC | 19:13 | |
| greghaynes | hah, this is on a build with patch you linked ;) | 19:13 |
| *** akuznetsov has quit IRC | 19:23 | |
| devananda | lifeless: on https://bugs.launchpad.net/ironic/+bug/1320513, it would be very helpful right now to know whether this is regarding the timing between ipmitool repeating/retrying the same request | 19:24 |
| uvirtbot | Launchpad bug 1320513 in ironic "IPMI commands are sent / queried too fast" [High,In progress] | 19:24 |
| devananda | lifeless: or time between discreet requests | 19:24 |
| devananda | lifeless: addressing the former requires upgrading your version of ipmitool. the latter can be easily done in python | 19:24 |
| *** morazi has quit IRC | 19:24 | |
| devananda | lifeless: fwiw, ipmitool 1.8.11 (packaged with ubuntu 12.04) will retry a single command 16 times with an interval between 1 and 5 seconds | 19:26 |
| devananda | lifeless: and there is, as far as i can tell, no way to change that (besides upgrading to 1.8.12) | 19:26 |
| *** spzala has joined #tripleo | 19:28 | |
| *** rpodolyaka1 has joined #tripleo | 19:29 | |
| openstackgerrit | Erik Colnick proposed a change to openstack/tripleo-image-elements: Add l3-triple.filters file for tripleo installations. https://review.openstack.org/97887 | 19:30 |
| *** nati_uen_ has quit IRC | 19:33 | |
| *** morazi has joined #tripleo | 19:38 | |
| *** nati_ueno has joined #tripleo | 19:39 | |
| greghaynes | lifeless: that patch fixes the issue for me | 19:40 |
| *** michchap has quit IRC | 19:40 | |
| *** jp_at_hp has joined #tripleo | 19:41 | |
| *** TimuRIo has joined #tripleo | 19:42 | |
| lifeless | greghaynes: can you comment to that effect on https://review.openstack.org/#/c/97806/ ? | 19:45 |
| lifeless | greghaynes: and then we go to #openstack-nova nd agitate for it | 19:45 |
| lifeless | dprince: ^ | 19:46 |
| dprince | lifeless: ack, haven't tested it yet myself. will have a look though | 19:47 |
| *** e0ne has quit IRC | 19:47 | |
| lifeless | devananda: ack; uhm, since they solved it years back I'm going to go with discrete. | 19:47 |
| *** vinsh has quit IRC | 19:47 | |
| lifeless | devananda: and since nova-bm didn't do it either, again discrete | 19:47 |
| * dprince runs check experimental on it... | 19:48 | |
| lifeless | dprince: won't help | 19:48 |
| lifeless | dprince: we have a revert in toci | 19:48 |
| lifeless | dprince: so CI isn't blocked; the downside is that means we can't tell | 19:48 |
| dprince | lifeless: I think our revert is looks for the ticket? no? | 19:48 |
| * dprince checks | 19:48 | |
| lifeless | dprince: oh maybe | 19:48 |
| openstackgerrit | Jay Dobies proposed a change to openstack/tripleo-specs: Spec for Tuskar Juno REST API changes https://review.openstack.org/94720 | 19:48 |
| * dprince thinks derekh is smarter than we are | 19:48 | |
| lifeless | naturally | 19:49 |
| lifeless | ah nice | 19:49 |
| lifeless | bug.* $3 | 19:49 |
| dprince | yep, I'll run experimental and we'll just see :) | 19:49 |
| lifeless | dprince: so yeah, check experimental will work | 19:49 |
| lifeless | we should make temprevert echo its status more clearly | 19:50 |
| SpamapS | with the f20 jobs.. | 19:52 |
| SpamapS | is there no way to look at the journal using the logs tarballs? | 19:52 |
| * SpamapS shoots systemd in the foot | 19:52 | |
| lifeless | ask lennart | 19:52 |
| dprince | SpamapS: I believe derekh is going to standardize our log locations w/ some journalctl foo | 19:53 |
| dprince | SpamapS: he mentioned something about it on Joe's patch for the elastic recheck stuff... | 19:54 |
| SpamapS | dprince: I pointed him to a way to have rsyslog pull the journal out into actual usable logs. | 19:54 |
| dprince | Spamaps: send me https://review.openstack.org/#/c/84600/ | 19:58 |
| SpamapS | dprince: done | 20:02 |
| lifeless | devananda: I think a good first approximation is to stop using attempts=3, put a global tracker in place and ensure we do sleep(5) between calls to the same ipmi endpoint | 20:03 |
| lifeless | devananda: tunnelling is going to be trickier to support | 20:03 |
| *** nati_ueno has quit IRC | 20:04 | |
| *** nati_uen_ has joined #tripleo | 20:07 | |
| *** nati_ueno has joined #tripleo | 20:09 | |
| dprince | SpamapS: any reason not to +1 approve https://review.openstack.org/#/c/84600. The dependent patch is also already a +1 right? | 20:10 |
| *** nati_uen_ has quit IRC | 20:10 | |
| dprince | SpamapS: I'm fine doing it myself, just wondering why you didn't? | 20:10 |
| lifeless | dprince: so that nova bug | 20:11 |
| lifeless | dprince: we're going to need a similar change to ironic | 20:11 |
| lifeless | no? | 20:11 |
| *** rpodolyaka1 has quit IRC | 20:14 | |
| dprince | lifeless: yes, we will. Want me to pick that up? /me is only using Ironic now anyways... | 20:20 |
| dprince | lifeless: this is just another example of why that code belongs in Nova | 20:21 |
| SpamapS | I thought somebody already fixed ironic | 20:22 |
| dprince | SpamapS: not sure, I didn't check. Just answering questions :) | 20:22 |
| SpamapS | https://review.openstack.org/#/c/97447/4 | 20:23 |
| * SpamapS must go to the dentist now.. bbl | 20:23 | |
| lifeless | ah cool | 20:24 |
| lifeless | just waiting on zuuuuuuuuuuuuuul | 20:25 |
| jerryz | lifeless: i made a few comments about progress on the infra-specs change you did. | 20:27 |
| lifeless | jerryz: cool | 20:29 |
| jerryz | lifeless: nodepool is working now , by parsing proxy info from nodepool.yaml. Jenkins could work too, but not with prefix command, but running scripts from master. | 20:30 |
| dprince | lifeless: The Ironic change is going to conflict with the Nova revert we are already doing as well. (this is why rolling forwards is bad sometimes... because it is everyone's race) | 20:31 |
| dprince | lifeless: they are in fact next to each other in the merge queue with the Ironic change landing first( 97447) and the nova change landing second (97757) | 20:32 |
| lifeless | dprince: the nova revert breaks the new ironic code? | 20:33 |
| dprince | lifeless: we can -2 one of them now... but it'll cost the merge queue which is already painful at this point | 20:33 |
| lifeless | dprince: or the nova roll-forward breaksthe new ironic code? | 20:34 |
| dprince | lifeless: I haven't confirmed that, but based on Han's assesment on the review I think yes | 20:34 |
| dprince | lifeless: we could just wait and see | 20:34 |
| lifeless | this one - You're welcome! | 20:34 |
| lifeless | This last failure looks unrelated. Haven't seen it with previous patchsets. | 20:34 |
| lifeless | ? | 20:34 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/tripleo-image-elements: indent using 4 spaces (1/3) https://review.openstack.org/93206 | 20:39 |
| *** vinsh has joined #tripleo | 20:44 | |
| *** TravT has joined #tripleo | 20:44 | |
| *** whayutin_ has joined #tripleo | 20:45 | |
| *** weshay has quit IRC | 20:48 | |
| *** untriaged-bot has joined #tripleo | 21:00 | |
| untriaged-bot | Untriaged bugs so far: | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/diskimage-builder/+bug/1326319 | 21:00 |
| uvirtbot | Launchpad bug 1326319 in diskimage-builder "Add support for deploying for ironic from baremetal node" [Undecided,New] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1326016 | 21:00 |
| uvirtbot | Launchpad bug 1326016 in tripleo "Different configuration files for different nova services are merged together not allowing to specify specific value for a specific service" [Undecided,In progress] | 21:00 |
| untriaged-bot | https://bugs.launchpad.net/tripleo/+bug/1325762 | 21:00 |
| uvirtbot | Launchpad bug 1325762 in tripleo "intermittent tftp issues using physical hardware" [Undecided,New] | 21:00 |
| *** untriaged-bot has quit IRC | 21:00 | |
| *** jdob has quit IRC | 21:01 | |
| *** matty_dubs is now known as matty_dubs|gone | 21:02 | |
| *** e0ne has joined #tripleo | 21:02 | |
| *** e0ne has quit IRC | 21:03 | |
| jogo | where does systemd log openstack logs in toci-ci | 21:13 |
| jogo | I can't find anything in the seed_* tarball | 21:14 |
| *** jang1 has quit IRC | 21:16 | |
| jogo | dprince: ^ | 21:18 |
| slagle | i believe you have to run journalctl locally and point it at the journal files from the tarball | 21:19 |
| slagle | journalctl --file /path/to/var/log/journal... | 21:19 |
| *** derekh_ has joined #tripleo | 21:20 | |
| jogo | slagle: do our CI nodes have journalctl on them? | 21:20 |
| derekh_ | jogo: the fedora ones will | 21:21 |
| slagle | yes, the fedora ones | 21:21 |
| jogo | I am gonna leave that for a dependant patch then | 21:21 |
| jogo | re: https://review.openstack.org/#/c/96308/4 | 21:21 |
| derekh_ | jogo: which run the fedora jobs so your golden | 21:21 |
| *** cwolferh has quit IRC | 21:21 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/tripleo-ci: Extract upstart logs so they can be loaded into logstash https://review.openstack.org/96308 | 21:22 |
| jogo | unless someone wants to ammend my patch to work with fedora ^ | 21:22 |
| derekh_ | jogo: I'm happy to merge that and can follow up with fedora coverage in the morning | 21:23 |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-specs: Moving TripleO Heat Templates towards native Heat https://review.openstack.org/97939 | 21:23 |
| jogo | derekh_: WFM | 21:24 |
| slagle | derekh_: hey, do you think it's worth running journalctl --full > systemd_logs.txt just so that things are a bit easier to introspect? | 21:25 |
| *** cwolferh has joined #tripleo | 21:26 | |
| *** nati_ueno has quit IRC | 21:26 | |
| derekh_ | slagle: I was going to do something on a unit specific basis so we would end up with a log file per unit, that way the same logstash signatures could be used | 21:26 |
| slagle | ah, sounds good | 21:27 |
| derekh_ | slagle: will push up a untested WIP of what I'm talking about in a few minutes, and fully test in the morning | 21:27 |
| *** julim has quit IRC | 21:32 | |
| *** martyntaylor has quit IRC | 21:38 | |
| *** andreaf has quit IRC | 21:40 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/tripleo-ci: Write log file for each systemd unit https://review.openstack.org/97947 | 21:48 |
| derekh_ | something like that might work... | 21:48 |
| lifeless | derekh_: o/ | 21:55 |
| *** jtomasek has quit IRC | 22:10 | |
| *** elynn has quit IRC | 22:11 | |
| *** elynn has joined #tripleo | 22:13 | |
| *** SpamapS has quit IRC | 22:14 | |
| *** SpamapS has joined #tripleo | 22:18 | |
| *** SpamapS has quit IRC | 22:18 | |
| *** SpamapS has joined #tripleo | 22:18 | |
| derekh_ | lifeless: hi, sorry didn't get as far as I hoped today, kinda got stuck trying to figure out how to put a node into maintanance mode | 22:23 |
| derekh_ | lifeless: so how does one do that / | 22:23 |
| derekh_ | ? | 22:23 |
| *** spzala has quit IRC | 22:24 | |
| lifeless | derekh_: ironic node-update $uuid add maintenance=True, IIRC | 22:25 |
| derekh_ | lifeless: thank, no amount of googling was helping me | 22:29 |
| *** akuznetsov has joined #tripleo | 22:31 | |
| *** whayutin_ has quit IRC | 22:36 | |
| *** jp_at_hp has quit IRC | 22:39 | |
| *** dprince has quit IRC | 22:40 | |
| *** derekh_ has quit IRC | 22:50 | |
| *** shakayumi has quit IRC | 22:55 | |
| *** shakayumi has joined #tripleo | 22:59 | |
| *** olaph has quit IRC | 23:02 | |
| *** saurabhs has left #tripleo | 23:02 | |
| *** ddieterl_ has quit IRC | 23:04 | |
| *** lucas-dinner has quit IRC | 23:10 | |
| *** shakayumi has quit IRC | 23:15 | |
| *** TimuRIo has quit IRC | 23:19 | |
| lifeless | StevenK: https://bugs.launchpad.net/tripleo/+bug/1326581 might be fun | 23:32 |
| uvirtbot | Launchpad bug 1326581 in tripleo "all nodes in nodes.json must be in good working order" [Wishlist,Triaged] | 23:32 |
| lifeless | StevenK: after we start using the os-cloud-config that is; can I help with that at all ? | 23:32 |
| StevenK | lifeless: Biggest blocker to a -incubator branch to switch to register-nodes from os-cloud-config is https://review.openstack.org/#/c/95441/ | 23:35 |
| SpamapS | Wom 19 | 23:36 |
| lifeless | StevenK: I've +A but in general I'd love to see more docs on such things | 23:39 |
| lifeless | StevenK: folk shouldn't need to read the code to see it does filtering like that | 23:39 |
| StevenK | lifeless: Hm, yes, you're right, I should have added a note about it to usage. | 23:41 |
| openstackgerrit | Nicholas Randon proposed a change to openstack/tripleo-incubator: Wait for ssh demon to become active https://review.openstack.org/96515 | 23:45 |
| openstackgerrit | A change was merged to openstack/os-cloud-config: Ignore pm_password if it's >255 characters https://review.openstack.org/95441 | 23:46 |
| *** michchap has joined #tripleo | 23:48 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!