| *** cd-undercloud has joined #tripleo | 00:32 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 00:32 |
|---|---|---|
| *** cd-undercloud has quit IRC | 00:32 | |
| lifeless | SpamapS: are you fiddling ? | 00:42 |
| SpamapS | lifeless: yes I am watching textcons on monty0005 | 00:44 |
| SpamapS | lifeless: deleted the stack so I can watch it be re-created | 00:44 |
| lifeless | ok | 00:44 |
| SpamapS | apologies if you were in the middle of something :-/ | 00:45 |
| SpamapS | I was poking around at both servers and getting absolutely nowhere | 00:45 |
| lifeless | well you'd said the metadata server was broken | 00:45 |
| SpamapS | I thought it was | 00:45 |
| lifeless | so I was going to debug that | 00:45 |
| SpamapS | but one of the two did get metadata | 00:45 |
| SpamapS | neither, however, had the right authorized_keys | 00:45 |
| lifeless | and .177 has dropped off the net somehow | 00:46 |
| *** jcooley_ has joined #tripleo | 00:53 | |
| SpamapS | oh good monty0005 picked.. watching it post | 01:09 |
| SpamapS | lifeless: looked like nova-api just needed a restart | 01:17 |
| *** cd-undercloud has joined #tripleo | 01:24 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 01:24 |
| *** cd-undercloud has quit IRC | 01:24 | |
| SpamapS | \o/ | 01:24 |
| *** NearlyFunctional has quit IRC | 01:26 | |
| lifeless | SpamapS: crack open the champagne | 01:36 |
| SpamapS | lifeless: need 2 in a row :) | 01:36 |
| SpamapS | and too late.. already a little tipsy on red wine :) | 01:36 |
| lifeless | oh noes | 01:37 |
| lifeless | SpamapS: so if it updates ok | 01:38 |
| lifeless | SpamapS: then we crack a champagne open :) | 01:38 |
| lifeless | I have a bottle of moet in the fridge... | 01:39 |
| *** NearlyFunctional has joined #tripleo | 01:45 | |
| jog0 | lifeless: I see I broke the cloud? you weren't running with fatal_deprecations=True? | 01:51 |
| lifeless | jog0: thats a thing? | 01:52 |
| lifeless | jog0: actually, there's more though - not your fault - the neutron driver doesn't check it's configured correctly | 01:53 |
| lifeless | jog0: which has mystified other people before me :) | 01:53 |
| lifeless | StevenK: tomorrow, I wonder if you might take this patch https://review.openstack.org/#/c/69060/ off my hands? | 01:55 |
| lifeless | SpamapS: update-in-progress | 01:56 |
| jog0 | lifeless: its a thing, not sure if it works | 01:57 |
| jog0 | last I checked it didn't | 01:57 |
| lifeless | heh | 01:59 |
| lifeless | that might explain why.... | 01:59 |
| lifeless | its waiting for overcloud stack ready | 02:01 |
| lifeless | overcloud hosts are down ... | 02:04 |
| lifeless | ssh wise | 02:04 |
| StevenK | lifeless: I would, except tomorrow is Australia Day | 02:04 |
| lifeless | doh | 02:04 |
| lifeless | well we'll see if I get to it | 02:04 |
| lifeless | otherwise tuesday :) | 02:04 |
| lifeless | just trying not to be a bottleneck | 02:04 |
| lifeless | SpamapS: are you on the textcons on monty0005 | 02:06 |
| lifeless | SpamapS: if so, please exit and use the ilo shared screen session instead | 02:06 |
| lifeless | SpamapS: update complete! | 02:08 |
| lifeless | ERROR: Unable to sign token. (HTTP 500) | 02:13 |
| lifeless | ,None | 02:13 |
| *** cd-undercloud has joined #tripleo | 02:13 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 02:13 |
| *** cd-undercloud has quit IRC | 02:13 | |
| lifeless | stopped to analyse | 02:13 |
| lifeless | keystone service-list | 02:14 |
| lifeless | Authorization Failed: Unable to sign token. (HTTP 500) | 02:14 |
| lifeless | 2014-01-25 22:02:07.059 3375 WARNING keystone.openstack.common.versionutils [-] | 02:16 |
| lifeless | Deprecated: keystone.common.kvs.Base is deprecated as of Icehouse in favor of ke | 02:16 |
| lifeless | ystone.common.kvs.KeyValueStore and may be removed in J. | 02:16 |
| lifeless | wtf | 02:16 |
| lifeless | keystone certificate isn't preserved | 02:16 |
| lifeless | SpamapS: ping; need +2 for great good | 02:41 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Move keystone os-config-applier to os-apply-config https://review.openstack.org/69159 | 02:41 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Preserve keystone tokens on rebuilds. https://review.openstack.org/69160 | 02:41 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Remove deprecated option. https://review.openstack.org/69161 | 02:41 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343 | 02:41 |
| lifeless | SpamapS: https://review.openstack.org/69160 specifically | 02:42 |
| SpamapS | lifeless: ack | 02:44 |
| lifeless | SpamapS: if we can land that, I think tripleo-cd will give us upgrades | 02:44 |
| SpamapS | lifeless: what writes those files? init-keystone? | 02:44 |
| lifeless | SpamapS: yeah. Oh nuts I need to make the dir. One sec | 02:45 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: SSL support for openstack services. https://review.openstack.org/67343 | 02:46 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Remove deprecated option. https://review.openstack.org/69161 | 02:46 |
| lifeless | SpamapS: ^ there | 02:46 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-image-elements: Preserve keystone tokens on rebuilds. https://review.openstack.org/69160 | 02:46 |
| lifeless | SpamapS: so yeah, init-keystone runs pki_setup which makes those files | 02:47 |
| lifeless | SpamapS: I tested this on the overcloud thats there now | 02:47 |
| lifeless | I'm sure there will be more glitches to get to acutally fully usable | 02:48 |
| lifeless | but keystone user-list works with that | 02:48 |
| lifeless | neutron port-list doesn't yet, but I suspect that may be because we *didn't* preserve everything | 02:48 |
| SpamapS | lifeless: right probably some cached tokens or something | 02:51 |
| lifeless | l3 agent failing to restart | 02:52 |
| SpamapS | lifeless: so +2 or policy-breaking +2/+A ... I'm kind of in the "if it fixes the cd cloud shortcuts are justified" camp | 02:53 |
| lifeless | SpamapS: +A is within policy | 02:53 |
| lifeless | SpamapS: we explicitly arranged for this | 02:53 |
| lifeless | go check the mails to -dev and/or the meetings | 02:54 |
| lifeless | minutes | 02:54 |
| lifeless | 2014-01-25 22:47:45.896 8292 WARNING keystoneclient.middleware.auth_token [-] signing_dir mode is 0770 instead of 0700 | 02:55 |
| lifeless | 2014-01-25 22:47:59.968 8292 WARNING keystoneclient.middleware.auth_token [-] Verify error: Command 'openssl' returned non-zero exit status 4 | 02:55 |
| lifeless | 2014-01-25 22:47:59.968 8292 WARNING keystoneclient.middleware.auth_token [-] Authorization failed for token 0e9c2e6c7e06fe3ed3b4fd27f2fdbdd5 | 02:55 |
| lifeless | 2014-01-25 22:48:00.136 8292 WARNING keystoneclient.middleware.auth_token [-] Verify error: Command 'openssl' returned non-zero exit status 4 | 02:55 |
| lifeless | looks like cached | 02:55 |
| lifeless | now to find where they are stored | 02:55 |
| lifeless | oh yuck | 02:58 |
| lifeless | keystone-signing-* | 02:58 |
| lifeless | in /tmp | 02:58 |
| lifeless | rm -rf /tmp/keystone-signing-* :) | 02:59 |
| lifeless | 2014-01-25 22:55:07.818 8796 TRACE nova.api.openstack Unauthorized: Unknown auth strategy | 03:00 |
| *** CaptTofu has joined #tripleo | 03:03 | |
| lifeless | oh ffs l3 agent | 03:07 |
| lifeless | you and rootwrap /really/ don't get on do you | 03:07 |
| lifeless | right, thats l3 fixed | 03:08 |
| lifeless | it would be nice if keystone logged API calls | 03:18 |
| * lifeless files a bug | 03:18 | |
| lifeless | SpamapS: did you run away? | 03:18 |
| lifeless | API calls to nova work | 03:20 |
| lifeless | api calls to keystone work | 03:20 |
| lifeless | api calls to glance work | 03:20 |
| lifeless | neutron is fucked up | 03:20 |
| lifeless | ahha /mnt/state/var/ | 03:26 |
| lifeless | lib/neutron/keystone-signing | 03:26 |
| lifeless | fixed it | 03:33 |
| lifeless | so yeah, my commit will fix it | 03:33 |
| lifeless | SpamapS: oh, you didn't approve the dependency! | 03:36 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Move keystone os-config-applier to os-apply-config https://review.openstack.org/69159 | 03:36 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Preserve keystone tokens on rebuilds. https://review.openstack.org/69160 | 03:36 |
| lifeless | tripleo-cd restarted | 03:40 |
| *** UtahDave has joined #tripleo | 03:46 | |
| SpamapS | lifeless: sorry.. it's "kids being insane and whining" time right now | 03:50 |
| SpamapS | lifeless: may or may not be back in the next 10 hours ;) | 03:51 |
| clarkb | *whinging ftfy | 03:57 |
| *** UtahDave has quit IRC | 04:19 | |
| lifeless | SpamapS: no worries; I approved the dependency | 04:19 |
| lifeless | SpamapS: gtd :) | 04:19 |
| *** cd-undercloud has joined #tripleo | 04:21 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 04:21 |
| *** cd-undercloud has quit IRC | 04:21 | |
| lifeless | nuts | 04:22 |
| lifeless | 409-{u'NeutronError': {u'message': u'Security group rule already exists. Group id is e6e55d4e-09c6-4e25-9af7-019 | 04:22 |
| lifeless | 6dd1a488a.', u'type': u'SecurityGroupRuleExists', u'detail': u''}} | 04:22 |
| lifeless | ahha | 04:22 |
| lifeless | is bug | 04:22 |
| lifeless | | a9e76773-7ac5-47ef-8626-6442409642d2 | demo | ACTIVE | None | NOSTATE | default-net=10.0.0.2, 138.35.77.132 | | 04:22 |
| *** boris-42 has joined #tripleo | 04:23 | |
| *** weshay has quit IRC | 04:24 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Don't double-create sec groups and VMs on rebuild https://review.openstack.org/69165 | 04:32 |
| lifeless | hmm | 04:35 |
| lifeless | ls /mnt/state/var/lib/nova/instances/ -l | 04:35 |
| lifeless | total 8 | 04:35 |
| lifeless | drwxr-x--- 2 nova kvm 4096 Jan 26 04:20 _base | 04:35 |
| lifeless | thats a bit new | 04:35 |
| lifeless | but | 04:36 |
| lifeless | last mkfs was at 0100 on the deploy server | 04:36 |
| lifeless | so | 04:37 |
| lifeless | /mnt was not mounted | 04:37 |
| lifeless | -> was not preserved | 04:37 |
| lifeless | on the compute node | 04:37 |
| *** UtahDave has joined #tripleo | 04:37 | |
| lifeless | I wonder if clints workaround is only configured on one server | 04:38 |
| *** xuhanp has joined #tripleo | 04:42 | |
| *** yamahata_ has quit IRC | 04:42 | |
| *** yamahata_ has joined #tripleo | 04:42 | |
| *** jomara_ has joined #tripleo | 04:43 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Workaround bug 1263294 for all images we build. https://review.openstack.org/69167 | 04:44 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Don't double-create sec groups and VMs on rebuild https://review.openstack.org/69165 | 04:44 |
| lifeless | monkeypatching that into place | 04:44 |
| *** jomara has quit IRC | 04:49 | |
| *** UtahDave has quit IRC | 04:51 | |
| *** ramishra has joined #tripleo | 04:54 | |
| *** UtahDave has joined #tripleo | 04:59 | |
| *** noslzzp has joined #tripleo | 05:00 | |
| *** ramishra has quit IRC | 05:13 | |
| *** ramishra has joined #tripleo | 05:13 | |
| *** ramishra has quit IRC | 05:17 | |
| *** jcooley_ has quit IRC | 05:21 | |
| *** CaptTofu has quit IRC | 05:27 | |
| *** CaptTofu has joined #tripleo | 05:27 | |
| *** CaptTofu has quit IRC | 05:32 | |
| *** matsuhashi has joined #tripleo | 05:32 | |
| *** coolsvap has joined #tripleo | 05:38 | |
| SpamapS | lifeless: ugggh | 05:39 |
| *** cd-undercloud has joined #tripleo | 05:40 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 05:40 |
| *** cd-undercloud has quit IRC | 05:40 | |
| lifeless | SpamapS: yeah, thats what I said | 05:40 |
| lifeless | timed out after uploading the user image | 05:42 |
| SpamapS | lifeless: I wonder if we can get bash to tell us what line of devtest_overcloud failed. :) | 05:42 |
| lifeless | if we run devtest_overcloud.sh with -x | 05:43 |
| lifeless | it seems to have all come up though | 05:43 |
| lifeless | nova hypervisor-stats | 05:46 |
| lifeless | +----------------------+-------+ | 05:46 |
| lifeless | | Property | Value | | 05:46 |
| lifeless | +----------------------+-------+ | 05:46 |
| lifeless | | count | 0 | | 05:46 |
| lifeless | not quite | 05:46 |
| lifeless | nova service-list --binary nova-compute | 05:46 |
| lifeless | ERROR: list index out of range | 05:46 |
| lifeless | no nova-compute | 05:46 |
| lifeless | SpamapS: did you log out of the ilo ? | 05:46 |
| lifeless | SpamapS: since they are single use for textcons, its *much* better to use the shared screen | 05:46 |
| *** noslzzp has quit IRC | 05:46 | |
| *** matsuhashi has quit IRC | 05:48 | |
| *** matsuhashi has joined #tripleo | 05:48 | |
| *** noslzzp has joined #tripleo | 05:49 | |
| SpamapS | lifeless: yeah I logged out of the ilo when you texted me :) | 05:49 |
| lifeless | SpamapS: thanks | 05:49 |
| SpamapS | or hangout'd me | 05:49 |
| SpamapS | I'm android now btw.. | 05:49 |
| lifeless | Ething :) | 05:49 |
| SpamapS | so I can't tell the difference :) | 05:49 |
| * SpamapS hugs his shiny blue motoX | 05:50 | |
| lifeless | 2014-01-26 05:52:48.833 2746 WARNING nova.virt.libvirt.driver [req-0a552c44-523d-458a-bbba-d9af7962efb9 None None] Cannot update service status on host: overcloud-novacompute0-trfhdvlqzrex,since it is not registered. | 05:50 |
| lifeless | 2014-01-26 05:52:18.655 2746 ERROR nova.openstack.common.rpc.common [req-157e1dd1-a188-4dfb-bc2c-40b340650745 None None] AMQP server on 10.10.16.171:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. | 05:51 |
| lifeless | 2014-01-26 05:52:18.655 2746 ERROR nova.openstack.common.rpc.common [req-157e1dd1-a188-4dfb-bc2c-40b340650745 None None] AMQP server on 10.10.16.171:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 30 seconds. | 05:51 |
| lifeless | bah | 05:51 |
| lifeless | anyhow, nova compute | 05:51 |
| SpamapS | hm | 05:51 |
| SpamapS | the ECONNREFUSED are just "ordering problems" | 05:51 |
| lifeless | =INFO REPORT==== 26-Jan-2014::01:46:29 === | 05:51 |
| lifeless | accepting AMQP connection <0.6332.0> (10.10.16.174:33735 -> 10.10.16.171:5672) | 05:51 |
| lifeless | =WARNING REPORT==== 26-Jan-2014::01:46:31 === | 05:51 |
| lifeless | closing AMQP connection <0.6332.0> (10.10.16.174:33735 -> 10.10.16.171:5672): | 05:51 |
| lifeless | connection_closed_abruptly | 05:51 |
| SpamapS | the abrupt was probably me just now :) | 05:52 |
| *** matsuhashi has quit IRC | 05:52 | |
| SpamapS | root@overcloud-novacompute0-trfhdvlqzrex:~# netstat -tnp | grep '171:5672.*ESTAB'|wc -l | 05:52 |
| SpamapS | 6 | 05:52 |
| *** matsuhashi has joined #tripleo | 05:52 | |
| lifeless | its registered now | 05:52 |
| lifeless | | nova-compute | overcloud-novacompute0-trfhdvlqzrex | nova | enabled | up | 2014-01-26T01:47:42.000000 | None | | 05:53 |
| SpamapS | lifeless: ah, did you have to do something? | 05:53 |
| SpamapS | lifeless: or was it just late to the party? | 05:53 |
| lifeless | I did a restart of nova-compute. | 05:53 |
| lifeless | but | 05:53 |
| lifeless | | nova-compute | overcloud-novacompute0-trfhdvlqzrex | nova | enabled | up | 2014-01-26T01:47:42.000000 | None | | 05:53 |
| lifeless | note the timestamp | 05:53 |
| lifeless | its now 53 | 05:53 |
| lifeless | checking clock deltas | 05:53 |
| lifeless | date | 05:53 |
| lifeless | Sun Jan 26 05:56:38 UTC 2014 | 05:53 |
| lifeless | compute | 05:53 |
| lifeless | $ date | 05:53 |
| lifeless | Sun Jan 26 01:48:50 UTC 2014 | 05:53 |
| lifeless | control | 05:54 |
| SpamapS | we really need to just get ntp up | 05:54 |
| SpamapS | hahahaha | 05:54 |
| SpamapS | oh wow | 05:54 |
| lifeless | one of these is not like the other | 05:54 |
| lifeless | I think the undercloud should offer NTP to it's tenants | 05:54 |
| SpamapS | lifeless: so do we go through the pain of secure broadcast NTP or just point all the boxes at the undercloud compute node? | 05:54 |
| lifeless | since we can't just force the virt machine clock to be right | 05:54 |
| lifeless | SpamapS: get the damn thing going then iterate? | 05:55 |
| SpamapS | I've always found NTP's security things to be a royal pain | 05:55 |
| lifeless | brought the 01 clock forward to the current | 05:56 |
| lifeless | so the time thing was the failure I think | 05:58 |
| lifeless | IMBW | 05:58 |
| lifeless | SpamapS: /dev/sda1 isn't mounted on compute | 06:01 |
| SpamapS | lifeless: of course not :( | 06:02 |
| lifeless | cat /etc/dib_arguments | 06:02 |
| lifeless | ubuntu -a amd64 -o /opt/stack/overcloud-compute nova-compute nova-kvm neutron-openvswitch-agent os-collect-config dhcp-all-interfaces pypi-openstack pip-cache stackuser pypi | 06:02 |
| lifeless | i don't see use-ephemeral there | 06:02 |
| lifeless | so either my patch wasn't applied | 06:02 |
| lifeless | or $wtf | 06:02 |
| lifeless | poking | 06:02 |
| SpamapS | I think I expected we'd add use-ephemeral to all elements which store in /mnt/state | 06:02 |
| lifeless | SpamapS: feel free to +A these patches too | 06:03 |
| SpamapS | but then forgot that I expected that :_p | 06:03 |
| lifeless | I think we do, but the workaround being in it is ... | 06:03 |
| lifeless | awkward | 06:03 |
| lifeless | mostly do | 06:03 |
| lifeless | anyhow | 06:03 |
| lifeless | let me delete the overcloud | 06:04 |
| lifeless | indeed, use-ephemeral wasn't included, so my patch is broken or wasn't used | 06:05 |
| *** matsuhashi has quit IRC | 06:06 | |
| *** akuznetsov has joined #tripleo | 06:08 | |
| *** matsuhashi has joined #tripleo | 06:10 | |
| lifeless | oh we're not importing _variables yes | 06:10 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Workaround bug 1263294 for all images we build. https://review.openstack.org/69167 | 06:13 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Don't double-create sec groups and VMs on rebuild https://review.openstack.org/69165 | 06:13 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-heat-templates: Allow setting a single NTP Server https://review.openstack.org/69173 | 06:15 |
| lifeless | SpamapS: oh you misunderstood me | 06:16 |
| lifeless | dhcp can advertise ntp-servers | 06:16 |
| lifeless | real simple | 06:16 |
| SpamapS | lifeless: can we actually do that in neutron DHCP though? | 06:16 |
| SpamapS | and.. | 06:16 |
| SpamapS | how do we make ntpd pick that up? | 06:16 |
| lifeless | I rather suspect linux DHCP does it already | 06:17 |
| lifeless | SpamapS: we can right now set extra options globally for all networks | 06:18 |
| lifeless | but we can do per-port things in code with a little work | 06:18 |
| lifeless | I guess I'm seeing this as a nova baremetal feature for parity with virt hypervisors | 06:19 |
| lifeless | e.g. ironic thing | 06:19 |
| lifeless | so maybe the heat template is needed for nwo | 06:19 |
| SpamapS | http://support.ntp.org/bin/view/Support/ConfiguringNTP#Section_6.12. explains it | 06:19 |
| SpamapS | lifeless: no I think this will work but we have to change the ntp element to use /etc/ntp.conf.dhcp | 06:19 |
| lifeless | yeah I see | 06:20 |
| SpamapS | oh hm | 06:20 |
| SpamapS | Ubuntu/Debian seem to configure this automatically. | 06:20 |
| lifeless | see http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html# under 'Create /etc/neutron/dnsmasq-neutron.conf, and add these values to lower the MTU size on instances and prevent packet fragmentation over the GRE tunnel: | 06:21 |
| *** tzumainn has joined #tripleo | 06:21 | |
| *** matsuhashi has quit IRC | 06:21 | |
| lifeless | same thing can set ntp servers | 06:21 |
| lifeless | let me monkey this onto the undercloud | 06:21 |
| SpamapS | README.Debian.gz claims that /var/lib/ntp/ntp.conf.dhcp will just be used automatically | 06:22 |
| *** matsuhas_ has joined #tripleo | 06:23 | |
| lifeless | right, added to dnsmasq config | 06:23 |
| lifeless | by | 06:23 |
| lifeless | editing /etc/neutron/dnsmasq-neutron.conf | 06:23 |
| lifeless | and setting to | 06:23 |
| SpamapS | lifeless: I've never used DHCP for NTP.. always went the extra step and did secure broadcast. | 06:24 |
| lifeless | dhcp-option-force=option:ntp-server,10.10.16.169 | 06:24 |
| *** akuznetsov has quit IRC | 06:24 | |
| SpamapS | like, extra 2 steps really because I never even knew DHCP could do NTP :) | 06:24 |
| SpamapS | or actually I think somewhere in the back of my head I recall Win2k doing that. | 06:24 |
| *** rpodolyaka1 has joined #tripleo | 06:25 | |
| *** akuznetsov has joined #tripleo | 06:27 | |
| *** jcooley_ has joined #tripleo | 06:29 | |
| lifeless | oh good ntpd is running :) | 06:30 |
| lifeless | SpamapS: did you just install it ?:)( | 06:30 |
| lifeless | s/(// | 06:30 |
| SpamapS | a while back I started working on this and then ooh shiny | 06:30 |
| lifeless | ok, so done on undercloud but needs automation | 06:32 |
| lifeless | and then a proper feature in e.g. ironic / nova | 06:32 |
| lifeless | or possibly just a neutron subnet setting | 06:32 |
| lifeless | dkehn: ^ what do you think, add a feature to set the ntp servers for a subnet in neutron ? | 06:33 |
| lifeless | ok, time to cook dinner | 06:33 |
| SpamapS | lifeless: I think "set dhcp options" seems a valid thing | 06:36 |
| SpamapS | lifeless: widely consumable for whatever reason | 06:37 |
| SpamapS | lifeless: we will need to add the ntp element to the element list I think, but then in theory it will pick up the DHCP configured server | 06:37 |
| *** matsuhas_ has quit IRC | 06:38 | |
| *** matsuhashi has joined #tripleo | 06:39 | |
| lifeless | SpamapS: oh ntptime isn't even in the base? | 06:40 |
| lifeless | nuts | 06:40 |
| lifeless | yes | 06:40 |
| *** jcooley_ has quit IRC | 06:45 | |
| *** ramishra has joined #tripleo | 06:46 | |
| *** ramishra has quit IRC | 06:51 | |
| *** cd-undercloud has joined #tripleo | 07:01 | |
| cd-undercloud | ************** overcloud complete status=2 ************ | 07:01 |
| *** cd-undercloud has quit IRC | 07:01 | |
| lifeless | oh what now | 07:01 |
| lifeless | awk: line 1: syntax error at or near } | 07:03 |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Fix last commit... https://review.openstack.org/69174 | 07:06 |
| *** ramishra has joined #tripleo | 07:06 | |
| *** ramishra_ has joined #tripleo | 07:07 | |
| *** rlandy has joined #tripleo | 07:07 | |
| *** noslzzp has quit IRC | 07:07 | |
| *** rlandy has quit IRC | 07:07 | |
| lifeless | SpamapS: ^ :( | 07:08 |
| lifeless | SpamapS: so why do we need the ntp element ? | 07:10 |
| *** rpodolyaka1 has quit IRC | 07:11 | |
| *** ramishra has quit IRC | 07:11 | |
| *** akuznetsov has quit IRC | 07:17 | |
| *** rlandy has joined #tripleo | 07:24 | |
| *** ramishra_ has quit IRC | 07:24 | |
| *** CaptTofu has joined #tripleo | 07:28 | |
| *** CaptTofu has quit IRC | 07:33 | |
| *** akuznetsov has joined #tripleo | 07:36 | |
| *** akuznetsov has quit IRC | 07:36 | |
| *** cd-undercloud has joined #tripleo | 08:04 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 08:04 |
| *** cd-undercloud has quit IRC | 08:04 | |
| *** rpodolyaka1 has joined #tripleo | 08:20 | |
| *** xuhanp has quit IRC | 08:36 | |
| *** cd-undercloud has joined #tripleo | 08:57 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 08:57 |
| *** cd-undercloud has quit IRC | 08:57 | |
| *** matsuhashi has quit IRC | 09:03 | |
| *** matsuhashi has joined #tripleo | 09:09 | |
| *** e0ne has quit IRC | 09:26 | |
| SpamapS | lifeless: ntp installs ntp :) | 09:27 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Fix last commit... https://review.openstack.org/69174 | 09:29 |
| *** CaptTofu has joined #tripleo | 09:29 | |
| *** e0ne has joined #tripleo | 09:29 | |
| *** CaptTofu has quit IRC | 09:34 | |
| lifeless | SpamapS: yes but | 09:36 |
| lifeless | ping: unknown host Ip | 09:37 |
| lifeless | hah I failed something somewhere | 09:38 |
| SpamapS | I'm surprised how long it takes between dhcp for the interfaces and sshd being up. | 09:39 |
| lifeless | entropy? | 09:40 |
| SpamapS | [ 159.773457] openvswitch: Open vSwitch switching datapath 1.10.2, built Jan 26 2014 09:17:31 | 09:41 |
| SpamapS | [ 279.781621] type=1400 audit(1390729303.987:11): apparmor="STATUS" operation="profile_replace" parent=1742 profile="unconfined" name="/ | 09:41 |
| SpamapS | actually [ 29.025475] mlx4_en: eth2: frag:0 - size:1526 prefix:0 align:0 stride:1536 | 09:41 |
| SpamapS | virtually identical lag on both... dmesg is basically identical too | 09:43 |
| lifeless | hmm | 09:43 |
| lifeless | wtf | 09:43 |
| lifeless | nova floating-ip-list | 09:43 |
| lifeless | +---------------+-----------+----------+---------+ | 09:43 |
| lifeless | | Ip | Server Id | Fixed Ip | Pool | | 09:43 |
| lifeless | +---------------+-----------+----------+---------+ | 09:43 |
| lifeless | | 138.35.77.132 | | 10.0.0.2 | ext-net | | 09:43 |
| lifeless | +---------------+-----------+----------+---------+ | 09:43 |
| lifeless | | awk '$8="ext-net" {print $2}' | 09:44 |
| lifeless | 09:44 | |
| lifeless | Ip | 09:44 |
| lifeless | 09:44 | |
| lifeless | 138.35.77.132 | 09:44 |
| lifeless | ugh, = vs == I suspect | 09:46 |
| lifeless | yeah, thats it | 09:46 |
| SpamapS | doohhh | 09:46 |
| SpamapS | lifeless: neutron has more shell friendly bits btw | 09:47 |
| lifeless | SpamapS: ish, but yeah | 09:47 |
| SpamapS | $ neutron floatingip-list -f csv | 09:47 |
| SpamapS | "id","fixed_ip_address","floating_ip_address","port_id" | 09:47 |
| SpamapS | "f47f0223-034f-48b3-b5e2-f177d837fd52","10.0.0.2","138.35.77.132","0c7875a1-bf5c-4339-9d7a-cfb19eac42ea" | 09:47 |
| lifeless | also nova floating-ip-list doesn't record the association even though the port is assigned | 09:47 |
| SpamapS | $ neutron floatingip-list -f csv -c floating_ip_address | 09:48 |
| SpamapS | "floating_ip_address" | 09:48 |
| SpamapS | "138.35.77.132" | 09:48 |
| *** cd-undercloud has joined #tripleo | 09:48 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 09:48 |
| *** cd-undercloud has quit IRC | 09:48 | |
| SpamapS | lifeless: this is apparentl due to using cliff.. which I submitted a patch for almost 2 weeks ago .. no review yet. :-P | 09:49 |
| lifeless | which is? | 09:49 |
| SpamapS | lifeless: having -f csv | 09:49 |
| SpamapS | lifeless: I'd also like to have --no-headers | 09:50 |
| SpamapS | when it is showing a single record you get -f shell | 09:51 |
| lifeless | neutron floating-ip-list -f csv -c floating_ip_address | 09:51 |
| lifeless | Invalid command u'-f' | 09:51 |
| SpamapS | which spits out variable assignments | 09:51 |
| SpamapS | lifeless: floatingip-list | 09:51 |
| SpamapS | they screwed that one up bad | 09:51 |
| SpamapS | lifeless: no champagne today | 09:52 |
| lifeless | gnar | 09:52 |
| SpamapS | lifeless: see you on the flip side | 09:52 |
| lifeless | need to get rid of quotes | 09:52 |
| * SpamapS crashes | 09:52 | |
| lifeless | this way is too hard | 09:52 |
| lifeless | night | 09:52 |
| SpamapS | lifeless: --quote=none | 09:52 |
| SpamapS | clint@undercloud-notcompute-jws3awlsb2kh:~$ neutron floatingip-list -f csv -c floating_ip_address --quote=none | 09:53 |
| SpamapS | floating_ip_address | 09:53 |
| SpamapS | 138.35.77.132 | 09:53 |
| lifeless | hmm | 09:55 |
| lifeless | ssh not running on .171?! | 09:55 |
| lifeless | and bad keys on on 100.10.16.174 | 09:58 |
| lifeless | ssh running on the public ip. wtf | 10:00 |
| lifeless | nope | 10:01 |
| lifeless | I have a bad 10.10.16.171 I think | 10:01 |
| lifeless | or there are two | 10:01 |
| lifeless | SpamapS: argh - bm-aw1az2-freecloud0033 'also possible TFTP/pxe problems, clint investigating' has that IP | 10:02 |
| *** e0ne has quit IRC | 10:04 | |
| *** e0ne_ has joined #tripleo | 10:05 | |
| lifeless | SpamapS: least the hypervisor-stats came back this time | 10:10 |
| jamezpolley | So now that I've signed up to the lists -does anyone have a useful set of gmail filters to apply labels based on the [tags] in the subjects, or should I create one? | 10:13 |
| lifeless | do what now? | 10:13 |
| lifeless | since we're deploying, we're basically interested in every topic | 10:14 |
| jamezpolley | good point. I shall just create an openstack label for now | 10:14 |
| lifeless | lol | 10:18 |
| lifeless | [ 2.254959] init: Error while reading from descriptor: Bad file descriptor | 10:18 |
| lifeless | [ 2.256220] init: mountall-shell post-stop process (235) terminated with status 6 | 10:18 |
| lifeless | [ 2.258389] init: error.c:219: Assertion failed in _nih_error_raise_system: errno > 0 | 10:18 |
| lifeless | [ 2.260116] init: Caught abort, core dumped | 10:18 |
| lifeless | [ 2.261191] init: startpar-bridge (mountall-shell--stopped) main process (237) terminated with status 6 | 10:18 |
| lifeless | this vm is rather unhappy | 10:18 |
| lifeless | [ 2.134167] init: error.c:219: Assertion failed in _nih_error_raise_system: errno > 0 | 10:19 |
| lifeless | [ 2.137194] init: Caught abort, core dumped | 10:19 |
| lifeless | [ 2.244318] init: mountall-shell main process (227) terminated with status 6 | 10:19 |
| lifeless | [ 2.245649] init: startpar-bridge (mountall-shell--started) main process (229) terminated with status 6 | 10:19 |
| lifeless | [ 2.247401] init: startpar-bridge (console-setup--stopped) main process (231) terminated with status 6 | 10:19 |
| lifeless | [ 2.249229] init: startpar-bridge (plymouth-stop--stopped) main process (233) terminated with status 6 | 10:19 |
| lifeless | [ 2.251792] init: error.c:219: Assertion failed in _nih_error_raise_system: errno > 0 | 10:19 |
| lifeless | [ 2.253507] init: Caught abort, core dumped | 10:19 |
| jamezpolley | quit | 10:27 |
| *** e0ne_ has quit IRC | 10:27 | |
| *** UtahDave has quit IRC | 10:27 | |
| * jamezpolley sighs | 10:28 | |
| lifeless | :) | 10:28 |
| *** e0ne has joined #tripleo | 10:28 | |
| jamezpolley | okay, tinkering with irc from an ipad on a long weekend is a bad idea. I'm going to stop it now. | 10:28 |
| *** UtahDave has joined #tripleo | 10:35 | |
| *** cd-undercloud has joined #tripleo | 10:46 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 10:46 |
| *** cd-undercloud has quit IRC | 10:46 | |
| *** coolsvap has quit IRC | 10:47 | |
| openstackgerrit | lifeless proposed a change to openstack/tripleo-incubator: Further work towards status=0 on updates. https://review.openstack.org/69184 | 10:53 |
| *** e0ne has quit IRC | 11:08 | |
| *** matsuhashi has quit IRC | 11:17 | |
| *** CaptTofu has joined #tripleo | 11:30 | |
| *** CaptTofu has quit IRC | 11:34 | |
| *** matsuhashi has joined #tripleo | 11:40 | |
| *** cd-undercloud has joined #tripleo | 11:41 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 11:41 |
| *** cd-undercloud has quit IRC | 11:41 | |
| *** matsuhashi has quit IRC | 11:53 | |
| *** matsuhashi has joined #tripleo | 11:54 | |
| *** matsuhashi has quit IRC | 11:55 | |
| *** matsuhas_ has joined #tripleo | 11:55 | |
| *** e0ne has joined #tripleo | 12:05 | |
| *** michchap has quit IRC | 12:28 | |
| *** michchap has joined #tripleo | 12:29 | |
| *** cd-undercloud has joined #tripleo | 12:31 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 12:31 |
| *** cd-undercloud has quit IRC | 12:31 | |
| *** matsuhas_ has quit IRC | 12:33 | |
| *** michchap has quit IRC | 12:34 | |
| *** matsuhashi has joined #tripleo | 12:34 | |
| *** CaptTofu has joined #tripleo | 12:34 | |
| *** matsuhashi has quit IRC | 13:05 | |
| *** matsuhashi has joined #tripleo | 13:06 | |
| *** cd-undercloud has joined #tripleo | 13:17 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 13:17 |
| *** cd-undercloud has quit IRC | 13:17 | |
| *** matsuhashi has quit IRC | 13:18 | |
| *** CaptTofu has quit IRC | 13:26 | |
| *** CaptTofu has joined #tripleo | 13:27 | |
| *** CaptTofu has quit IRC | 13:32 | |
| *** michchap has joined #tripleo | 14:00 | |
| *** cd-undercloud has joined #tripleo | 14:03 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 14:03 |
| *** cd-undercloud has quit IRC | 14:03 | |
| *** michchap has quit IRC | 14:05 | |
| *** michchap has joined #tripleo | 14:38 | |
| *** michchap has quit IRC | 14:43 | |
| *** cd-undercloud has joined #tripleo | 14:49 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 14:49 |
| *** cd-undercloud has quit IRC | 14:49 | |
| *** erwan_taf has quit IRC | 15:17 | |
| *** erwan_taf has joined #tripleo | 15:18 | |
| *** e0ne has quit IRC | 15:29 | |
| *** michchap has joined #tripleo | 15:30 | |
| *** michchap has quit IRC | 15:35 | |
| *** cd-undercloud has joined #tripleo | 15:35 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 15:35 |
| *** cd-undercloud has quit IRC | 15:35 | |
| *** michchap has joined #tripleo | 15:39 | |
| *** michchap has quit IRC | 15:43 | |
| *** rpodolyaka1 has quit IRC | 15:44 | |
| SpamapS | jamezpolley: Not sure gmail can do the "kill this thread" trick.. I'd be lost without that one. :-P | 15:50 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Further work towards status=0 on updates. https://review.openstack.org/69184 | 15:55 |
| SpamapS | Conflict occurred attempting to store role grant. User 2a07aef214054eceab7f19106885a2f9 already has role 196f60afd87147c282dae3da60d47310 in tenant 2b6216d7c0c94250b0883fb26c8dc847 (HTTP 409) | 15:56 |
| SpamapS | ugh | 16:12 |
| SpamapS | python-keystoneclient's test_shell.py is.. wow | 16:12 |
| SpamapS | was thinking we should just add a --assert option to python-keystoneclient which would accept 409's as success. But not sure I'm ready to dive into this test suite.. wow | 16:13 |
| *** mestery has quit IRC | 16:16 | |
| *** mestery has joined #tripleo | 16:16 | |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Fix fail on user role conflict https://review.openstack.org/69206 | 16:21 |
| *** cd-undercloud has joined #tripleo | 16:21 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 16:21 |
| *** cd-undercloud has quit IRC | 16:21 | |
| SpamapS | lifeless: hey .. reading assert-users... I was just thinking.. why don't we just use Heat for that? We can run a stack per user.. | 16:31 |
| *** rpodolyaka1 has joined #tripleo | 16:31 | |
| SpamapS | lifeless: could replace assert-user with a template, and assert-users would just call heat stack-create in a loop | 16:32 |
| *** rpodolyaka1 has quit IRC | 16:38 | |
| *** michchap has joined #tripleo | 16:40 | |
| *** shardy_afk is now known as shardy | 16:40 | |
| *** michchap has quit IRC | 16:40 | |
| *** michchap has joined #tripleo | 16:41 | |
| *** michchap has quit IRC | 16:45 | |
| *** rpodolyaka1 has joined #tripleo | 16:50 | |
| *** cd-undercloud has joined #tripleo | 17:10 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 17:10 |
| *** cd-undercloud has quit IRC | 17:10 | |
| *** ccrouch has quit IRC | 17:14 | |
| *** brownelj has quit IRC | 17:28 | |
| *** akuznetsov has joined #tripleo | 17:36 | |
| *** michchap has joined #tripleo | 17:41 | |
| *** michchap has quit IRC | 17:46 | |
| *** cd-undercloud has joined #tripleo | 17:57 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 17:57 |
| *** cd-undercloud has quit IRC | 17:57 | |
| SpamapS | hrm | 18:04 |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Fix fail on user role conflict https://review.openstack.org/69206 | 18:07 |
| *** rpodolyaka1 has quit IRC | 18:13 | |
| fungi | general note to the crowd (anyone who's managing the ci overcloud): nodepool is unable to ssh to the slaves it's launching again. probably continuing fallout from https://launchpad.net/bugs/1271344 | 18:18 |
| *** akuznetsov has quit IRC | 18:22 | |
| *** sdague has quit IRC | 18:35 | |
| *** sdague has joined #tripleo | 18:36 | |
| *** akuznetsov has joined #tripleo | 18:40 | |
| *** michchap has joined #tripleo | 18:42 | |
| *** michchap has quit IRC | 18:42 | |
| *** michchap has joined #tripleo | 18:42 | |
| fungi | oh, actually no, this time it's "Timeout waiting for waiting for server" http://paste.openstack.org/show/61906/ | 18:44 |
| *** akuznetsov has quit IRC | 18:45 | |
| *** cd-undercloud has joined #tripleo | 18:45 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 18:45 |
| *** cd-undercloud has quit IRC | 18:45 | |
| fungi | so maybe something related to the new tripleo-precise image built by nodepool after the recent tripleo prep tooling changes merged | 18:46 |
| *** michchap has quit IRC | 18:47 | |
| *** e0ne has joined #tripleo | 18:48 | |
| fungi | anyway, any of your changes which get gate-tripleo-deploy job runs are getting indefinitely stuck in zuul's check pipeine waiting for available workers until someone has time to look into it | 18:52 |
| *** rpodolyaka1 has joined #tripleo | 19:07 | |
| *** sdake has quit IRC | 19:15 | |
| *** e0ne_ has joined #tripleo | 19:15 | |
| *** e0ne has quit IRC | 19:18 | |
| *** cd-undercloud has joined #tripleo | 19:31 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 19:31 |
| *** cd-undercloud has quit IRC | 19:31 | |
| SpamapS | \o/ | 19:32 |
| SpamapS | \o/\o/\o/ thats an update success!!! | 19:32 |
| lifeless | \o/ | 19:36 |
| lifeless | SpamapS: uhm, assert-users as heat temlpate - interesting if baroque | 19:37 |
| lifeless | SpamapS: clearly I got up just in time for drinking | 19:37 |
| lifeless | fungi: hi, wassup ? | 19:38 |
| fungi | lifeless: see above. nodes timing out on launch in ci overcloud | 19:38 |
| lifeless | ah, I ee error state on the instances | 19:39 |
| lifeless | no valid host was found | 19:39 |
| fungi | i haven't had time to investigate via novaclient | 19:39 |
| lifeless | I can't ping the hypervisor | 19:40 |
| fungi | that sounds double plus ungood | 19:40 |
| lifeless | [798032.623528] EXT4-fs (nbd15p1): error count: 2 | 19:41 |
| lifeless | [798032.623560] EXT4-fs (nbd15p1): initial error at 1389949008: ext4_mb_generate | 19:41 |
| lifeless | _buddy:755 | 19:41 |
| lifeless | [798032.623609] EXT4-fs (nbd15p1): last error at 1390270787: ext4_lookup:1437: i | 19:41 |
| lifeless | node 2 | 19:41 |
| lifeless | eth2 has been given the undercloud IP -> I bet its dhcp-all-interfaces breaking us again | 19:42 |
| SpamapS | lifeless: well if things are working, the cork pops in about 45 minutes :) | 19:42 |
| lifeless | I can ping it now | 19:42 |
| *** michchap has joined #tripleo | 19:42 | |
| *** michchap has quit IRC | 19:42 | |
| lifeless | fungi: please delete + image delete | 19:42 |
| lifeless | fungi: should be fine now | 19:43 |
| *** michchap has joined #tripleo | 19:43 | |
| lifeless | fungi: https://bugs.launchpad.net/tripleo/+bug/1272969 | 19:47 |
| lifeless | SpamapS: so what further work was needed to fix it ? | 19:47 |
| *** michchap has quit IRC | 19:48 | |
| lifeless | SpamapS: oh, os-assert-users bug? | 19:48 |
| lifeless | SpamapS: so, I could imagine one heat stack for admin users and one stack for regular users and update the stack to update things | 19:50 |
| lifeless | SpamapS: but a loop doing one-per-user seems fugly | 19:50 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Fix fail on user role conflict https://review.openstack.org/69206 | 19:50 |
| SpamapS | lifeless: yea I'm looking at how we could make heat smart enough to take an array and turn it into multiple resources | 19:51 |
| SpamapS | lifeless: that would also serve the nova compute case. | 19:51 |
| SpamapS | since then you could just give them uuid's and add/remove them by uuid | 19:52 |
| SpamapS | and give them credentials by uuid | 19:52 |
| SpamapS | Because I really dislike the "give me 8 clones" mode that we have now. | 19:52 |
| fungi | lifeless: thanks--doing | 19:53 |
| SpamapS | lifeless: anyway back to users.. agreed that we could just generate two stacks from the data file. | 19:53 |
| jamezpolley | SpamapS: it has mute, which is fairly similar.It used to not work if people CCed you but i think it got buffed a few yeats ago | 20:05 |
| *** werebutt has joined #tripleo | 20:06 | |
| *** werebutt has left #tripleo | 20:06 | |
| SpamapS | jamezpolley: I want the list reference only to be squelched | 20:06 |
| SpamapS | jamezpolley: anything more greedy than that will result in me missing important things. | 20:07 |
| jamezpolley | *checks* now I remember - the buff was that if you're already in the CC list and you mute, it will stay muted; if you mute and someone later CCs you it will still unmute. Previously if it you had already been CCd (eg, because you commented early in the thread) you could never mute it afterward | 20:12 |
| lifeless | SpamapS: monkeypatch in incubator removed | 20:13 |
| SpamapS | lifeless: \o/ | 20:14 |
| SpamapS | k time to do Sunday stuff | 20:14 |
| *** cd-undercloud has joined #tripleo | 20:20 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 20:20 |
| *** cd-undercloud has quit IRC | 20:20 | |
| lifeless | FUCK YEAH | 20:20 |
| lifeless | SpamapS: go get blitzed man | 20:20 |
| *** sdake has joined #tripleo | 20:30 | |
| *** michchap has joined #tripleo | 20:43 | |
| *** michchap has quit IRC | 20:43 | |
| *** michchap has joined #tripleo | 20:44 | |
| *** michchap has quit IRC | 20:49 | |
| mordred | lifeless: ++ | 21:04 |
| *** cd-undercloud has joined #tripleo | 21:07 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 21:07 |
| *** cd-undercloud has quit IRC | 21:07 | |
| greghaynes | lifeless: Hey, so im trying to do the seed vm -> heat stack create overcloud and nova fails with one of the nodes while booting the other, then the other node fails to boot with : No valid host was found. ', u'code': 500, u'details': u' File "/opt/stack/venvs/nova/local/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 107, in schedule_run_instance | 21:21 |
| greghaynes | After some reading best guess is it wants resources the seed vm cant provide (doing qemu in the seed vm) | 21:21 |
| greghaynes | not sure if thats an insane way to do it, or if theres some tricks to make a seed vm with needed resources, or im totally off target | 21:22 |
| lifeless | greghaynes: so the seed is a baremetal cloud, qemu shouldn't matter | 21:22 |
| lifeless | greghaynes: remember its booting other VM's via libvirt + ssh | 21:22 |
| lifeless | greghaynes: (or physical machines, if you're using physical machines ;)) | 21:22 |
| greghaynes | aha! so its not qemu in a vm'ing | 21:23 |
| lifeless | greghaynes: with your seed cloud active | 21:23 |
| lifeless | greghaynes: nova baremetal-node-list | 21:23 |
| lifeless | that should return a node | 21:23 |
| lifeless | then nova hypervisor-stats | 21:23 |
| lifeless | which should return a count 1 hypervisor with some size | 21:23 |
| lifeless | nova flavor-list | 21:23 |
| lifeless | should have the sum of the disk space match the disk space returned from hypervisor-stats (easy to divide by 1 ;)) | 21:24 |
| greghaynes | yep, so used memory and used gb all ads up to 100% used | 21:24 |
| lifeless | nova list | 21:25 |
| lifeless | see what you have running ? | 21:25 |
| greghaynes | Yep, and they both have errord out | 21:25 |
| lifeless | one thing that stands out to me is that you said seed->overcloud | 21:25 |
| lifeless | which is not usual | 21:25 |
| lifeless | usually folk doing the devtest story do seed -> undercloud | 21:25 |
| greghaynes | yes, I thought I remembered you suggesting that as a workflow | 21:25 |
| lifeless | because their seed will only have one physical node registered | 21:26 |
| lifeless | but the overcloud needs two (and soon 4) | 21:26 |
| lifeless | greghaynes: its a thing you can do to optimise resource usage | 21:26 |
| greghaynes | gotcha | 21:26 |
| lifeless | but you'll need to register mode baremetal nodes | 21:26 |
| greghaynes | makes sense :) | 21:27 |
| *** rpodolyaka1 has quit IRC | 21:39 | |
| *** michchap has joined #tripleo | 21:44 | |
| *** michchap has quit IRC | 21:44 | |
| *** michchap has joined #tripleo | 21:45 | |
| *** marun has joined #tripleo | 21:47 | |
| *** michchap has quit IRC | 21:50 | |
| *** cd-undercloud has joined #tripleo | 21:54 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 21:54 |
| *** cd-undercloud has quit IRC | 21:54 | |
| *** marun has quit IRC | 21:56 | |
| *** marun has joined #tripleo | 22:02 | |
| lifeless | bbiab, running some erransd | 22:06 |
| *** sdake has quit IRC | 22:25 | |
| *** e0ne_ has quit IRC | 22:27 | |
| *** AaronGr_Zzz is now known as AaronGr | 22:28 | |
| *** marun has quit IRC | 22:32 | |
| *** cd-undercloud has joined #tripleo | 22:40 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 22:40 |
| *** cd-undercloud has quit IRC | 22:40 | |
| *** michchap has joined #tripleo | 22:45 | |
| *** michchap has quit IRC | 22:45 | |
| *** michchap has joined #tripleo | 22:45 | |
| *** michchap has quit IRC | 22:50 | |
| *** cd-undercloud has joined #tripleo | 23:27 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 23:27 |
| *** cd-undercloud has quit IRC | 23:27 | |
| *** e0ne has joined #tripleo | 23:38 | |
| *** michchap has joined #tripleo | 23:41 | |
| *** e0ne has quit IRC | 23:43 | |
| *** UtahDave has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!