| *** julim has joined #tripleo | 00:02 | |
| *** julim has quit IRC | 00:06 | |
| *** matsuhashi has joined #tripleo | 00:18 | |
| *** tzumainn has quit IRC | 00:26 | |
| *** CaptTofu has quit IRC | 00:33 | |
| *** CaptTofu has joined #tripleo | 00:34 | |
| *** nosnos has joined #tripleo | 00:34 | |
| *** CaptTofu has quit IRC | 00:38 | |
| *** UtahDave has joined #tripleo | 00:41 | |
| *** UtahDave has quit IRC | 00:56 | |
| *** edmund has joined #tripleo | 01:08 | |
| *** krotscheck has quit IRC | 01:17 | |
| *** CaptTofu has joined #tripleo | 01:23 | |
| *** CaptTofu has quit IRC | 01:23 | |
| *** CaptTofu has joined #tripleo | 01:24 | |
| *** CaptTofu has quit IRC | 01:24 | |
| *** CaptTofu has joined #tripleo | 01:25 | |
| *** anteaya has quit IRC | 01:43 | |
| *** noslzzp has quit IRC | 01:48 | |
| *** cwolferh has quit IRC | 02:30 | |
| *** cwolferh has joined #tripleo | 02:46 | |
| *** MarkAtwood has quit IRC | 03:07 | |
| *** matsuhashi has quit IRC | 03:20 | |
| *** matsuhashi has joined #tripleo | 03:21 | |
| *** matsuhashi has quit IRC | 03:25 | |
| *** edmund has quit IRC | 03:37 | |
| *** cwolferh has quit IRC | 03:46 | |
| *** spzala has quit IRC | 03:51 | |
| *** jcoufal has joined #tripleo | 04:06 | |
| *** ccrouch is now known as ccrouch-afk | 04:07 | |
| *** akuznetsov has joined #tripleo | 04:24 | |
| *** openstackgerrit has quit IRC | 04:36 | |
| *** openstackgerrit has joined #tripleo | 04:36 | |
| *** matsuhashi has joined #tripleo | 04:42 | |
| *** MarkAtwood has joined #tripleo | 04:43 | |
| *** MarkAtwood has left #tripleo | 04:49 | |
| *** rpodolyaka1 has joined #tripleo | 04:49 | |
| *** rpodolyaka1 has quit IRC | 04:52 | |
| *** rpodolyaka1 has joined #tripleo | 04:53 | |
| tserong | lifeless, FYI, i'll be in HK for the openstack summit, along with at least half a dozen other SUSE crew | 05:00 |
|---|---|---|
| lifeless | tserong: sweet | 05:02 |
| lifeless | tserong: hey, there are folk in HP asking after suse support in dib | 05:02 |
| lifeless | tserong: I said you had been doing some stuff but there wasn't a 'just works' element yet. | 05:02 |
| *** matsuhashi has quit IRC | 05:02 | |
| tserong | yeah, i' | 05:02 |
| lifeless | tserong: and that I would enquire ;) | 05:02 |
| tserong | heh | 05:02 |
| tserong | i've got a partial implementation of an opensuse element | 05:02 |
| tserong | which i was hacking on week before last | 05:02 |
| *** matsuhashi has joined #tripleo | 05:03 | |
| tserong | it did actually build something, but that thing didn't boot :-/ | 05:03 |
| tserong | i didn't commit it anywhere public yet though (it's a bit more WIPy than I'd like) | 05:04 |
| tserong | i was hoping to squeeze in a bit of cleanup on it this week before going to HK | 05:04 |
| tserong | but i've also got a pile of other stuff to get through, so i'll have to see how i go :-/ | 05:05 |
| devananda | lifeless: hi! #'s question for one o fmy slides -- how often per day is tripleo deploying a new cloud? | 05:06 |
| *** matsuhashi has quit IRC | 05:08 | |
| rpodolyaka1 | morning guys | 05:08 |
| *** coolsvap has joined #tripleo | 05:09 | |
| rpodolyaka1 | I believe, it takes about an hour to build and deploy overcloud, so 24 times per day | 05:10 |
| devananda | ty | 05:11 |
| rpodolyaka1 | maybe, a few clouds more :) | 05:11 |
| lifeless | devananda: yeah | 05:23 |
| lifeless | tserong: ok | 05:23 |
| lifeless | tserong: let me know if you need input | 05:23 |
| *** rushiagr has joined #tripleo | 05:26 | |
| lifeless | hmm | 05:27 |
| lifeless | tripleo-cd status is suspiciously quiet | 05:27 |
| lifeless | # service tripleo-cd status | 05:27 |
| lifeless | tripleo-cd stop/waiting | 05:27 |
| tserong | lifeless, thanks, will do | 05:27 |
| lifeless | error: Couldn't resolve host 'git.openstack.org' while accessing https://git.openstack.org/cgit/openstack/diskimage-builder/info/refs | 05:28 |
| lifeless | fatal: HTTP request failed | 05:28 |
| lifeless | *************************************************** | 05:28 |
| lifeless | DNS again | 05:28 |
| lifeless | who wants to be on mellanox firmware EPIC FAIL again ? | 05:28 |
| lifeless | seems reliable in a while-true loop | 05:28 |
| lifeless | so, starting it up again | 05:29 |
| lifeless | tserong: I don't know if Hari will be in HK, if he is I'll introduce you | 05:30 |
| lifeless | tserong: since his team is responsible for getting a bunch of vendor images together | 05:31 |
| tserong | ok, cool | 05:31 |
| tserong | the openSUSE image I've been basing off, I generated on build.opensuse.org in my home project | 05:32 |
| tserong | http://download.opensuse.org/repositories/home:/tserong:/Images/images/ | 05:32 |
| tserong | but (without having spoken to anyone else about this), I don't see any reason we couldn't publish more official-ish | 05:32 |
| tserong | images in the Cloud:OpenStack project on OBS (somewhere in http://download.opensuse.org/repositories/Cloud:/OpenStack:/) | 05:33 |
| tserong | then DIB would grab one of those, much the same as it does for ubuntu, and do its magic on top | 05:33 |
| lifeless | yeah | 05:36 |
| lifeless | that would be idea :) | 05:36 |
| lifeless | ideal. | 05:36 |
| rushiagr | hi folks | 05:44 |
| rushiagr | I am trying the dev_test setup for tripleo | 05:44 |
| rushiagr | I thought while it is downloading packages, I'll ask a question | 05:45 |
| rushiagr | Q: Is my laptop (8gb ram, i7 processor) enough for the devtest environment? I see that it mentions that it will create 5 VMs! | 05:45 |
| rpodolyaka1 | rushiagr: I think, you are a bit short of RAM and might possibly end up using swap | 05:46 |
| rushiagr | rpodolyaka1: can I make it just 3 VMs by changing some config? one for seed, one undercloud and the other overcloud? | 05:47 |
| rpodolyaka1 | rushiagr: though you might try to lower the mem requirements for created VMs (NODE_MEM env variable) | 05:47 |
| lifeless | rushiagr: it's four VM's | 05:47 |
| lifeless | rushiagr: the 5th is a qemu instance within the overcloud compute node | 05:48 |
| rushiagr | 4 + 1 seed, isnt it? | 05:48 |
| lifeless | rushiagr: so it has no direct footprint on the host. | 05:48 |
| lifeless | seed, undercloud, overcloud control plane, overcloud compute. | 05:48 |
| rushiagr | lifeless: ah, okay | 05:48 |
| lifeless | also, once the undercloud is running, you can 'save' the seed via virt-manager | 05:48 |
| lifeless | well | 05:48 |
| lifeless | if you do some routing stuff, I shouldn't have mentioned that right now; it's in the advanced fiddling category ;) | 05:49 |
| rushiagr | lifeless: yeah :) | 05:49 |
| rushiagr | lifeless: that made me feel I know so little :/ | 05:50 |
| * rushiagr hopes NODE_MEM=512 would be enough | 05:52 | |
| lifeless | openstack won't work with tht | 05:53 |
| lifeless | openstack services are fat memory hogs | 05:53 |
| rushiagr | owh, 1G a good compromise? | 05:53 |
| lifeless | 2G | 05:53 |
| rpodolyaka1 | rushiagr: 1.5-1.7 could work, I think | 05:53 |
| lifeless | I have a running undercloud here | 05:53 |
| lifeless | KiB Mem: 98997760 total, 41625076 used, 57372684 free, 2194164 buffers | 05:53 |
| rpodolyaka1 | but 2GB is known to work great | 05:54 |
| rushiagr | I have only 8 G ram, so I'm trying to optimise it as much as I can | 05:54 |
| lifeless | rushiagr: yeah :( | 05:54 |
| lifeless | rushiagr: we've had deploy failures due to the node being slow due to swapping with 1G nodes | 05:54 |
| lifeless | rushiagr: we then tried 1.5G - same | 05:54 |
| lifeless | rushiagr: be sure to use a 32-bit build, 64-bit is a waste for this scenario | 05:55 |
| lifeless | rushiagr: 1.5G may work w/32bit | 05:55 |
| rushiagr | cant I configure the undercloud nodes with a different memory than the overcloud, seed nodes | 05:55 |
| rushiagr | lifeless: good advice, will follow that | 05:55 |
| lifeless | rushiagr: in theory yes, but we haven't put that sort of complexity into the dev harness | 05:56 |
| rushiagr | lifeless: okay, that's exactly the ans I was looking for | 05:56 |
| lifeless | so yeah this machine | 05:57 |
| lifeless | 400M for nova compute, 245M resident | 05:57 |
| lifeless | 400M for conductor, 230M resident | 05:57 |
| lifeless | 2.5G for rabbitmq, 210M resident | 05:57 |
| lifeless | 4.3G for mysqld, 200M resident | 05:57 |
| lifeless | 200M for nova-api, 140M resident | 05:58 |
| lifeless | meh, let me copy-paste | 05:58 |
| *** tzumainn has joined #tripleo | 05:58 | |
| rushiagr | well, I'll do with 1.7G, and let you people know | 05:58 |
| lifeless | http://paste.ubuntu.com/6322429/ | 05:59 |
| rushiagr | what do you mean by resident? | 05:59 |
| lifeless | actual RAM used | 05:59 |
| lifeless | the virt column includes memory mapped libraries and files | 05:59 |
| lifeless | and swapped out pages | 05:59 |
| lifeless | and ram | 05:59 |
| lifeless | so virt-res = stuff on disk that this program may access, res = currently in ram | 05:59 |
| rushiagr | ahh, okay | 06:00 |
| lifeless | it's common to have lots of shared pages amongst programs using the same libraries | 06:00 |
| lifeless | so you can't add the virt column up to get a sane total | 06:00 |
| lifeless | but you can add the res column up | 06:00 |
| rushiagr | I misunderstood that resident = disk-resident. Completely opposite | 06:00 |
| rushiagr | lifeless: yeah, good to know | 06:01 |
| lifeless | anyhow, point is that I can visually see 1.9G of memory resident on this 64 bit node | 06:01 |
| rushiagr | hmmm.. I'll start with 1.7G RAM and 32bit machines.. | 06:02 |
| lifeless | note that deploying bare metal images involves about 2.5G of IO too | 06:03 |
| lifeless | reading the qcow out of glance | 06:03 |
| lifeless | writing it to a temp file | 06:03 |
| lifeless | converting it to a raw file | 06:03 |
| lifeless | then it gets copied to the target via iscsi | 06:03 |
| rushiagr | that should be okay i guess, I have 1TB HDD | 06:05 |
| *** akuznetsov has quit IRC | 06:05 | |
| lifeless | rushiagr: right, point was that that goes into the pagecache overhead of the relevant VM | 06:06 |
| lifeless | e.g. the seed | 06:06 |
| rushiagr | ah okay | 06:07 |
| *** jcoufal has quit IRC | 06:28 | |
| *** jeckersb_gone has quit IRC | 06:28 | |
| *** GheRivero has quit IRC | 06:28 | |
| *** akuznetsov has joined #tripleo | 06:30 | |
| *** GheRivero has joined #tripleo | 06:30 | |
| *** jeckersb_gone has joined #tripleo | 06:31 | |
| *** rpodolyaka1 has quit IRC | 06:37 | |
| *** jcoufal has joined #tripleo | 06:37 | |
| *** akuznetsov has quit IRC | 06:40 | |
| *** cd-undercloud has joined #tripleo | 06:40 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 06:40 |
| *** cd-undercloud has quit IRC | 06:40 | |
| lifeless | ruhroh | 06:41 |
| lifeless | timed out waiting for the overcloud stack to be ready | 06:41 |
| lifeless | I bet one or more of the new machines I registered is fail. | 06:42 |
| lifeless | hmmm, both active | 06:42 |
| lifeless | I can ping both | 06:42 |
| lifeless | I can ssh into the compute node | 06:43 |
| lifeless | I can ssh into the not compute node | 06:43 |
| lifeless | it's vlan hasn't come up | 06:43 |
| lifeless | [2013-10-29 02:20:44,975] (os-refresh-config) [ERROR] Aborting... | 06:44 |
| lifeless | 2013-10-29 02:20:44.979 1683 ERROR os-collect-config [-] Command failed, will not cache new data. Command 'os-refresh-config' returned non-zero exit status 1 | 06:44 |
| lifeless | 2013-10-29 02:20:44.979 1683 WARNING os-collect-config [-] Sleeping 30.00 seconds before re-exec. | 06:44 |
| lifeless | last thing logged at 20 paste | 06:44 |
| lifeless | 20 past | 06:44 |
| lifeless | but it's running: | 06:44 |
| lifeless | service os-collect-config status | 06:44 |
| lifeless | os-collect-config start/running, process 1683 | 06:44 |
| lifeless | cat /var/lib/heat-cfntools/cfn-metadata-server | 06:44 |
| lifeless | http://10.10.16.169:8000 | 06:45 |
| lifeless | so that looks ok | 06:45 |
| lifeless | and it can ping it | 06:45 |
| lifeless | heat-local data is there | 06:45 |
| lifeless | I'm confused why this didn't recover | 06:45 |
| lifeless | the clock is out by 20m | 06:46 |
| lifeless | I wonder if that breaks heat auth | 06:46 |
| SpamapS | lifeless: some orc script being silent? | 06:46 |
| SpamapS | lifeless: heat auth is not time based | 06:46 |
| lifeless | 2013-10-29 02:23:05.698 1683 ERROR os-collect-config [-] Command failed, will not cache new data. Command 'os-refresh-config' returned non-zero exit status 1 | 06:46 |
| lifeless | 2013-10-29 02:23:05.698 1683 WARNING os-collect-config [-] Sleeping 30.00 seconds before re-exec. | 06:46 |
| lifeless | SpamapS: keystone tokens | 06:46 |
| lifeless | SpamapS: ? | 06:46 |
| SpamapS | lifeless: no it uses ec2 keypairs | 06:46 |
| lifeless | log into 10.10.16.174 and have a poke | 06:46 |
| lifeless | SpamapS: ^ | 06:47 |
| lifeless | SpamapS: it's written a seemingly valid /etc/os-collect-config.conf | 06:47 |
| lifeless | path = notcomputeConfig.Metadata | 06:47 |
| lifeless | etc | 06:47 |
| SpamapS | lifeless: key I added isn't working | 06:47 |
| lifeless | SpamapS: thats too opaque for me | 06:47 |
| lifeless | SpamapS: other than 'blame SpamapS' | 06:48 |
| SpamapS | spamaps-clint-HP is not working to login to cd-undercloud.tripleo.org | 06:48 |
| lifeless | SpamapS: oh, cause thats manually synced and I had not | 06:48 |
| SpamapS | I was hoping you'd say that :) | 06:48 |
| lifeless | sec | 06:48 |
| *** tzumainn has quit IRC | 06:49 | |
| lifeless | ok try cd-undercloud again ? | 06:50 |
| SpamapS | ty | 06:51 |
| lifeless | I'll do the bastion in a sec | 06:51 |
| *** rdopieralski has joined #tripleo | 06:52 | |
| lifeless | nova updated | 06:53 |
| lifeless | now the bastion | 06:53 |
| SpamapS | lifeless: looks like something was definitely wonky.. cfn data source is working now | 06:53 |
| lifeless | SpamapS: what fixed it? | 06:53 |
| SpamapS | time? | 06:54 |
| *** jtomasek has joined #tripleo | 06:54 | |
| SpamapS | as in, something went slow | 06:54 |
| lifeless | I didn't reset the clock | 06:54 |
| lifeless | SpamapS: try the bastion host ? | 06:54 |
| lifeless | SpamapS: hang on | 06:54 |
| lifeless | SpamapS: you sshed into the undercloud | 06:54 |
| SpamapS | lifeless: aye | 06:54 |
| lifeless | SpamapS: how can you tell the cfn data source is working? You're on the wrong machien... | 06:54 |
| SpamapS | lifeless: I'm on 10.10.16.174 | 06:55 |
| SpamapS | I bounced | 06:55 |
| lifeless | 2013-10-29 02:31:30.601 1683 ERROR os-collect-config [-] Command failed, will not cache new data. Command 'os-refresh-config' returned non-zero exit status 1 | 06:55 |
| lifeless | 2013-10-29 02:31:30.602 1683 WARNING os-collect-config [-] Sleeping 30.00 seconds before re-exec. | 06:55 |
| lifeless | still looks broken to me | 06:55 |
| SpamapS | do we have two 10.10.16.174's? | 06:55 |
| lifeless | SpamapS: how did you bounce? Your key isn't on 10.10.16.174... | 06:55 |
| lifeless | SpamapS: whats the hostname you see ? | 06:56 |
| SpamapS | heat-admin@undercloud-notcompute-jws3awlsb2kh:~$ | 06:56 |
| lifeless | thats the undercloud | 06:56 |
| lifeless | note the letters u-n-d-e-r | 06:56 |
| lifeless | :P | 06:56 |
| SpamapS | oh did not catch the $? == 1 | 06:56 |
| SpamapS | no | 06:56 |
| SpamapS | I didn't note any letters | 06:57 |
| lifeless | stack name in the hostname is super useful | 06:57 |
| lifeless | anyhow | 06:57 |
| SpamapS | yes it is | 06:57 |
| lifeless | I've now added your key to the 174 host | 06:57 |
| lifeless | so you should be able to log in there | 06:57 |
| lifeless | and to the bastion which has ilo access setup etc | 06:57 |
| lifeless | done dan's too while there | 06:57 |
| lifeless | except bastion, we dont' have non-HP on there yet. | 06:57 |
| lifeless | leaving it with you for a little (insomniac!) while I sooth Cynthia | 06:59 |
| SpamapS | | notcompute | 37443 | ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-85b74461-de49-4fed-8601-62ae42d4b5a4) | CREATE_FAILED | 2013-10-29T05:58:36Z | | 07:00 |
| SpamapS | lifeless: looks like novaclient exploded and heat stopped stack creation. | 07:01 |
| *** coolsvap has quit IRC | 07:04 | |
| SpamapS | lifeless: no visible logs of that request ID anywhere on undercloud.. | 07:05 |
| SpamapS | lifeless: I recommend starting tripleo-cd again. I'm off to bed. | 07:06 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 07:20 |
| rdopieralski | lifeless: do you have a moment to talk about this patch ^^ ? | 07:20 |
| *** jtomasek has quit IRC | 07:25 | |
| lifeless | SpamapS: ok, thanks | 07:28 |
| lifeless | SpamapS: since it's late, I'm going to stab the firmware thing then restart | 07:28 |
| lifeless | rdopieralski: sure | 07:30 |
| *** marios has quit IRC | 07:31 | |
| *** marios has joined #tripleo | 07:31 | |
| rdopieralski | lifeless: do you think I should add werkzeug to tuskar-ui's and horizon's dependencies just for that function, or should I rather rewrite it on my own using the same principles as this one? | 07:33 |
| rdopieralski | lifeless: note, that werkzeug is a http library, unrelated to caching | 07:34 |
| rdopieralski | lifeless: I can write my own code that does that -- actually, if you look at the previous patches, I did -- but this trick is quite smart, so I figured it would be nice to give credit | 07:35 |
| lifeless | rdopieralski: there are many such things out there | 07:39 |
| lifeless | rdopieralski: I don't think we want to add yet another | 07:39 |
| lifeless | rdopieralski: reuse is good. | 07:39 |
| *** bauzas has joined #tripleo | 07:39 | |
| rdopieralski | lifeless: you didn't answer | 07:41 |
| lifeless | rdopieralski: I did in the review | 07:42 |
| * rdopieralski looks | 07:42 | |
| lifeless | rdopieralski: I don't know if adding werkzeug is the right thing | 07:43 |
| lifeless | rdopieralski: but copying the code is the wrong thing | 07:43 |
| lifeless | rdopieralski: writing a new caching decorator is the wrong thing | 07:43 |
| rdopieralski | "but it's work it" <-- what does that mean? | 07:43 |
| lifeless | thats me failing to type. Sorry! | 07:43 |
| lifeless | but, lets work through it | 07:44 |
| lifeless | is what I should have written. | 07:44 |
| rdopieralski | ok, thanks, I will do something more productive then and maybe someday when we have more time for politics we will talk about including werkzeug in the dependencies | 07:44 |
| lifeless | I don't expect it to be very political | 07:45 |
| lifeless | but it does require changing global-requirements | 07:45 |
| lifeless | and consensus on the new dep | 07:45 |
| lifeless | If you don't have time to do that now, I can understand that, but please don't be put off by the fact we're part of a larger ecosystem ;) | 07:45 |
| rdopieralski | can't help it | 07:46 |
| rdopieralski | I will just find some other part of code that I can somehow improve | 07:46 |
| rdopieralski | and hopefully it won't attract too much attention | 07:46 |
| lifeless | rdopieralski: I'm lost, I don't understand the objection to putting a one line patch into global-requirements and pinging a couple of folk about it | 07:47 |
| lifeless | rdopieralski: receiving attention in this project is generally a good thing; 1500 devs... | 07:47 |
| rdopieralski | lifeless: no objection | 07:47 |
| rdopieralski | well, I don't like attention, I like fixing stuff | 07:48 |
| lifeless | ok, so lets fix this? | 07:48 |
| rdopieralski | I can't think of a commit message that would justify pulling in werkzeug just for one of its internal utility function. | 07:50 |
| lifeless | rdopieralski: how big is werkzeug ? | 07:51 |
| *** jprovazn has joined #tripleo | 07:51 | |
| lifeless | rdopieralski: AIUI it's a library, it's *meant* to be reused. | 07:51 |
| rdopieralski | several thousand lines, afair | 07:51 |
| lifeless | rpodolyaka: so the commit message should just be 'add werkzeug' | 07:51 |
| lifeless | several thousand lines - tiny. | 07:51 |
| lifeless | not 100 lines tiny, but still not big. | 07:51 |
| rdopieralski | yeah, it's a small http header parsing and web utilities library | 07:51 |
| lifeless | so lets reuse it. | 07:52 |
| rdopieralski | pretty much all that is in django.core | 07:52 |
| rdopieralski | ok, let me try it | 07:52 |
| lifeless | it'll take 5 minutes to prep and put forward | 07:52 |
| *** martyntaylor has joined #tripleo | 07:52 | |
| *** martyntaylor has left #tripleo | 07:52 | |
| lifeless | worst that can happen is soemone has a good reason not to use it | 07:52 |
| lifeless | if that happens, we've lost 5 minutes, oh well, we can move on. | 07:52 |
| lifeless | let me just check the license is sympatico for OpenStack | 07:53 |
| lifeless | BSD | 07:53 |
| lifeless | fine | 07:53 |
| lifeless | if folk complain about it's size, I'll poke Armin about splitting it apart | 07:54 |
| lifeless | it is a decently sized tarball, I'm not sure why | 07:54 |
| lifeless | I want a round tuit to run https://pypi.python.org/pypi/radon against nova ;) | 07:55 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 07:55 |
| rdopieralski | lifeless: the packaging people will kill me :P | 07:56 |
| rdopieralski | lifeless: so what's the process now? | 07:57 |
| rdopieralski | lifeless: same as with any change? | 07:57 |
| *** coolsvap has joined #tripleo | 07:57 | |
| lifeless | rdopieralski: make a change to the openstack-infra/global-requirements repo adding it | 07:57 |
| lifeless | rdopieralski: add a reviewer (me, so I know the url to help discuss it) and add a review (the horizon PTL) | 07:58 |
| lifeless | rdopieralski: wait for reviews; if it gets in then the tuskar-ui change is a no-brainer | 07:58 |
| lifeless | bbiab | 07:59 |
| *** rushiagr has quit IRC | 08:02 | |
| rdopieralski | I can't see global-requirements at https://github.com/openstack-infra | 08:04 |
| *** jcoufal_ has joined #tripleo | 08:05 | |
| rpodolyaka | it's here git://git.openstack.org/openstack/requirements | 08:06 |
| rpodolyaka | or here https://github.com/openstack/requirements | 08:07 |
| *** lsmola has joined #tripleo | 08:07 | |
| rdopieralski | thanks | 08:07 |
| *** jcoufal has quit IRC | 08:08 | |
| rpodolyaka | rdopieralski: sorry for pointing you to the 'long way' instead of short one :( | 08:08 |
| rdopieralski | rpodolyaka: what do you mean? | 08:09 |
| rpodolyaka | rdopieralski: reusing of Werkzeug decorator instead yours one | 08:09 |
| rdopieralski | nah, I should have done it from the beginning | 08:09 |
| rdopieralski | I'm just too lazy when it comes to interacting with people | 08:09 |
| rpodolyaka | but it's community, it's open source, you have to talk to people here :) | 08:10 |
| rdopieralski | thank you for the help with it guys, by the way, I really appreciate it | 08:10 |
| rpodolyaka | np, we are glad to help | 08:10 |
| rpodolyaka | the more people we had, the more task we can finish ;) | 08:11 |
| rpodolyaka | *have | 08:11 |
| * rpodolyaka is typing too bad today | 08:11 | |
| *** ifarkas has joined #tripleo | 08:11 | |
| *** marun has joined #tripleo | 08:13 | |
| lifeless | rpodolyaka: +1 | 08:15 |
| rdopieralski | this is a little embarassing, but who is horizon's ptl? | 08:16 |
| rdopieralski | Gabriel Hurley? | 08:17 |
| rpodolyaka | yep | 08:17 |
| rpodolyaka | afaik | 08:17 |
| rpodolyaka | :) | 08:17 |
| *** akuznetsov has joined #tripleo | 08:17 | |
| rdopieralski | it's hard to find on the wiki :) | 08:17 |
| rpodolyaka | I found the letter in the mailbox - " September 2013 PTL elections - Final results" :) | 08:18 |
| rdopieralski | now, I suppose the best way to have my patches to horizon reviewed is to come to the horzion meeting and mention them? | 08:20 |
| lifeless | in theory no | 08:20 |
| lifeless | in practice, perhaps :) | 08:20 |
| rdopieralski | or at least mention the blueprints | 08:20 |
| ogelbukh | rdopieralski: you could use https://review.openstack.org/#/admin/groups/ | 08:20 |
| ogelbukh | :) | 08:20 |
| ogelbukh | just look for -ptl groups | 08:21 |
| rdopieralski | oh, nice, thanks! | 08:21 |
| ogelbukh | rdopieralski: np ) | 08:21 |
| *** marun has quit IRC | 08:27 | |
| *** nosnos_ has joined #tripleo | 08:29 | |
| *** nosnos has quit IRC | 08:30 | |
| *** marun has joined #tripleo | 08:33 | |
| *** marun has quit IRC | 08:33 | |
| *** cd-undercloud has joined #tripleo | 08:38 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 08:38 |
| *** cd-undercloud has quit IRC | 08:38 | |
| rpodolyaka | ouch | 08:39 |
| *** jistr has joined #tripleo | 08:57 | |
| *** nosnos has joined #tripleo | 09:02 | |
| *** nosnos_ has quit IRC | 09:03 | |
| *** jtomasek has joined #tripleo | 09:03 | |
| *** marun has joined #tripleo | 09:03 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 09:05 |
| *** derekh has joined #tripleo | 09:09 | |
| *** markmc has joined #tripleo | 09:09 | |
| Ng | morning | 09:10 |
| ogelbukh | Ng: good morning | 09:16 |
| rpodolyaka | o/ | 09:16 |
| shadower | hey | 09:17 |
| *** rushiagr2 has joined #tripleo | 09:17 | |
| Ng | hey hey | 09:21 |
| *** akuznetsov has quit IRC | 09:21 | |
| *** lucasagomes has joined #tripleo | 09:24 | |
| *** akuznetsov has joined #tripleo | 09:25 | |
| *** rushiagr2 has quit IRC | 09:27 | |
| *** akuznetsov has quit IRC | 09:30 | |
| *** cd-undercloud has joined #tripleo | 09:33 | |
| cd-undercloud | ************** overcloud complete status=0 ************ | 09:33 |
| *** cd-undercloud has quit IRC | 09:33 | |
| *** rushiagr3 has joined #tripleo | 09:34 | |
| rpodolyaka | \o/ | 09:34 |
| lifeless | \o/ | 09:36 |
| *** jcoufal_ is now known as jcoufal | 09:40 | |
| *** epim has joined #tripleo | 09:52 | |
| *** epim has quit IRC | 09:56 | |
| derekh | lifeless: should there be a triple section on https://wiki.openstack.org/wiki/Summit/Icehouse/Etherpads for us to create etherpads under ? | 10:00 |
| lifeless | derekh: yes! | 10:00 |
| derekh | lifeless: k, I can create it if you want | 10:01 |
| lifeless | derekh: fantastic :) | 10:03 |
| *** akuznetsov has joined #tripleo | 10:06 | |
| derekh | lifeless: done https://wiki.openstack.org/wiki/Summit/Icehouse/Etherpads#Tuesday_6 | 10:09 |
| lifeless | thanks ;) | 10:09 |
| * lifeless is a big believer in delegation | 10:10 | |
| *** nosnos has quit IRC | 10:31 | |
| *** nosnos has joined #tripleo | 10:32 | |
| *** nosnos has quit IRC | 10:36 | |
| *** cd-undercloud has joined #tripleo | 10:42 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 10:42 |
| *** cd-undercloud has quit IRC | 10:42 | |
| lifeless | nuts | 10:42 |
| lifeless | derekh: now mightbe the time to exercise your sysadmin access :) - see why it's failing | 10:43 |
| derekh | lifeless: k, will take a look around | 10:43 |
| lifeless | I reset the firmware a couple hours back, and thats usually stable for days | 10:43 |
| *** ehelms-afk is now known as ehelms | 10:43 | |
| lifeless | last three runs were fail, pass, fail | 10:43 |
| derekh | [2013/10/29 06:42:19 AM] [ERROR] key db-password does not exist in ['/var/run/os-collect-config/heat_local.json', '/var/run/os-collect-config/ec2.json'] | 10:50 |
| derekh | + db_pass= | 10:50 |
| derekh | still digging into whats caused that | 10:51 |
| lifeless | derekh: ok so that should happen once on boot | 10:51 |
| lifeless | then it gets the real config from cfn | 10:51 |
| lifeless | derekh: one reason that can keep occuring is if heat thinks the instance deploy failed | 10:52 |
| lifeless | derekh: e.g. heat event-list overcloud | 10:52 |
| lifeless | | notcompute | 37620 | CREATE aborted | CREATE_FAILED | 2013-10-29T09:58:46Z | 10:52 |
| lifeless | | NovaCompute0 | 37619 | ClientException: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request-ID: req-f4603762-9da9-4f79-b7d8-8f53fc1c5875) | CREATE_FAILED | 2013-10-29T09:58:46Z | 10:52 |
| lifeless | derekh: so, question is why did nova barf | 10:53 |
| *** rushiagr3 has quit IRC | 10:55 | |
| derekh | lifeless: 2013-10-29 09:58:45,790.790 19625 TRACE nova.api.openstack ConnectionFailed: Connection to neutron failed: Maximum attempts reached | 10:56 |
| lifeless | derekh: ahha | 10:56 |
| lifeless | derekh: stabbity stab stab stab | 10:56 |
| derekh | lifeless: hmm, looks like this is relevant in neutron-server.log | 11:00 |
| derekh | 2013-10-29 09:57:06.448 19414 WARNING neutron.db.agentschedulers_db [-] Fail scheduling network {'status': u'ACTIVE', 'subnets': [u'18b96b8d-c989-4820-b330-0c0f7c19c8d5'], 'name': u'ctlplane', 'provider:physical_network': u'ctlplane', 'admin_state_up': True, 'tenant_id': u'4956c533154c476799c688eda7ed65ab', 'provider:network_type': u'flat', 'router:external': False, 'shared': False, 'id': u'e85b44c7-1136-4217-954e-cdf0acdddfe1', 'provider:segmentation_id': | 11:00 |
| derekh | None} | 11:00 |
| derekh | happened at the same time as the heat fail event | 11:01 |
| derekh | 2013-10-29 09:57:50.587 18834 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']. Exception: | 11:04 |
| rpodolyaka | is dhcp agent running? | 11:05 |
| derekh | yup | 11:05 |
| lifeless | derekh: what exception ? | 11:05 |
| *** che-arne has joined #tripleo | 11:05 | |
| derekh | Oct 29 09:57:41 undercloud-notcompute-jws3awlsb2kh kernel: [1826679.931936] Buffer I/O error on device loop0p1, logical block 2264960 | 11:05 |
| lifeless | derekh: I think thats the stuff-stuck-on-unmount thing showing up lazily | 11:06 |
| lifeless | we have tonnnnes of that in the logs | 11:06 |
| derekh | ok | 11:06 |
| lifeless | not great, but not correlated with failures over the last couple weeks | 11:06 |
| lifeless | 00:04 < derekh> 2013-10-29 09:57:50.587 18834 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute | 11:07 |
| lifeless | ['ovs-vsctl', '--timeout=2', 'list-ports', 'br-int']. Exception: | 11:07 |
| lifeless | what exception followed it ? | 11:07 |
| derekh | just the exit code from ovs-vsctl http://paste.openstack.org/show/50117/ | 11:08 |
| rpodolyaka | hmm, does br-int exist at all?| | 11:08 |
| lifeless | rpodolyaka: should do, this is the undercloud node which has been doing all the deploys :) | 11:09 |
| derekh | rpodolyaka: yup and that command is working for me now | 11:10 |
| lifeless | lets capture this in a bug at minimum | 11:10 |
| *** jtomasek has quit IRC | 11:10 | |
| lifeless | I forsee someone climbing through ovs-vsctl code to see what causes this :) | 11:10 |
| derekh | ok, will gather up the details | 11:11 |
| rpodolyaka | lifeless: ah, I forgot for a minute that we didn't deploy undercloud :) | 11:11 |
| *** jtomasek has joined #tripleo | 11:14 | |
| lifeless | ok, past-midnight gnight; see you all at the meeting ;) | 11:24 |
| derekh | https://bugs.launchpad.net/tripleo/+bug/1245852 | 11:24 |
| derekh | lifeless: night ^ bug filed, will see if I can find anything more out | 11:25 |
| rpodolyaka | interesting, my undercloud fails to get CFN metadata | 11:34 |
| rpodolyaka | heat-admin@undercloud-notcompute-gvqnorbsopna:/var/run/os-collect-config$ sudo os-collect-config | 11:34 |
| rpodolyaka | 2013-10-29 11:33:42.425 31715 WARNING os_collect_config.cfn [-] 400 Client Error: InvalidParameterValue | 11:34 |
| rpodolyaka | 2013-10-29 11:33:42.425 31715 WARNING os-collect-config [-] Source [cfn] Unavailable. | 11:34 |
| rpodolyaka | heh, | notcomputeConfig | 9 | TypeError: Index to "Fn::Select" should be a string: "Fn::Select": ["4", ["str1", "str2"]] | CREATE_FAILED | 2013-10-29T11:10:52Z | 11:37 |
| rpodolyaka | the funny part is that corresponding commit to heat has been merged recently (https://review.openstack.org/#/c/54063/3/heat/engine/template.py) | 11:41 |
| * rpodolyaka investigating | 11:41 | |
| openstackgerrit | Ivan Berezovskiy proposed a change to openstack/diskimage-builder: Check of existence of directory 'lost+found' https://review.openstack.org/54311 | 11:44 |
| rpodolyaka | aha, and we are using int values in Fn::Select calls in our templates | 11:44 |
| *** jtomasek has quit IRC | 11:44 | |
| rpodolyaka | so this must be the reason | 11:44 |
| derekh | rpodolyaka: I just deployed a fresh devtest this morning and didn't get this error | 11:45 |
| rpodolyaka | derekh: could you please check if you have commit 9df15c134261f6b0675d5039562b3a893e879e44 in heat git repo? | 11:45 |
| derekh | rpodolyaka: nope don't have it, looks like DIB_OFFLINE=1 so it mustn't have gotten pulled down | 11:48 |
| rpodolyaka | derekh: aha, btw does "heat event-show overcloud" in tripleo-cd cloud show something similar to this error? | 11:49 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 11:49 |
| *** akuznetsov has quit IRC | 11:49 | |
| rpodolyaka | derekh: but we don't upgrade heat in undercloud, so it should be ok | 11:49 |
| rpodolyaka | so it's a bug, but it should not affect tripleo-cd | 11:50 |
| *** cd-undercloud has joined #tripleo | 11:51 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 11:51 |
| *** cd-undercloud has quit IRC | 11:51 | |
| *** markmc has quit IRC | 11:51 | |
| derekh | rpodolyaka: agreed, cd error looks different http://paste.openstack.org/show/50119/ | 11:51 |
| rpodolyaka | ok, so Heat docs say that selector can be an integer value (http://docs.openstack.org/developer/heat/template_guide/functions.html#fn-select), going to file/fix a bug in Heat then | 11:58 |
| *** akuznetsov has joined #tripleo | 12:05 | |
| *** rushiagr has joined #tripleo | 12:05 | |
| *** ccrouch-afk is now known as ccrouch | 12:09 | |
| *** akuznetsov has quit IRC | 12:11 | |
| *** akuznetsov has joined #tripleo | 12:21 | |
| *** jdob has joined #tripleo | 12:23 | |
| *** john-n-seattle2 has quit IRC | 12:26 | |
| *** lucasagomes is now known as lucas-hungry | 12:27 | |
| rushiagr | is the devtest setup need to be done as root? | 12:40 |
| rushiagr | s/is/does/ | 12:40 |
| slagle | rushiagr: no, it uses sudo where it needs to | 12:41 |
| *** dprince has joined #tripleo | 12:50 | |
| rushiagr | slagle: I remember that the command at point number 14 of this page http://docs.openstack.org/developer/tripleo-incubator/devtest.html gave me some trouble | 12:51 |
| rushiagr | slagle: the output gave me a 'permission denied' somewhere in between | 12:52 |
| slagle | rushiagr: did it still build an image successfully? | 12:52 |
| rushiagr | slagle: so I ran it via sudo (and probably some other commands too), which now is giving a 'permission denied' on the execution of 'devtest-seed-vm' line on devtest_seed page | 12:53 |
| rushiagr | slagle: yes, it built it successfully | 12:54 |
| slagle | rushiagr: you shouldn't have to add a 'sudo' anywhere. the called scripts themsevles use sudo where it's needed | 12:55 |
| rushiagr | hmm.. I'll do everything all over again | 12:56 |
| rushiagr | slagle: thanks | 12:56 |
| slagle | rushiagr: it's possible there could be one missing. might be worth filing a bug. or if you have a log you could paste, that could help too | 12:56 |
| rushiagr | I think my usage of 'sudo' while building images might have given a 'permission denied' at later stages. I'll try again and keep this channel posted if I find issues still | 12:57 |
| *** jdob has quit IRC | 12:57 | |
| *** jdob has joined #tripleo | 12:57 | |
| *** akuznetsov has quit IRC | 12:58 | |
| *** cd-undercloud has joined #tripleo | 12:59 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 12:59 |
| *** cd-undercloud has quit IRC | 12:59 | |
| openstackgerrit | Roman Podoliaka proposed a change to openstack/tripleo-heat-templates: Use string selector values in Fn::Select calls https://review.openstack.org/54322 | 12:59 |
| rushiagr | I am hoping running 'cleanup-env' wont clean up my downloaded images. Am I correct? I havent set up proxies yet, and dont want that 200 something megs image to be downloaded | 13:00 |
| rpodolyaka | rushiagr: it won't, it just deletes created VMs | 13:01 |
| rushiagr | rpodolyaka: thanks | 13:01 |
| rpodolyaka | rushiagr: but you'd better set up an http proxy anyway :) | 13:02 |
| rushiagr | rpodolyaka: looks like I should. I wanted to sprint through the devtest setup, but I am realising going slowly but steadily will give me lesser trouble | 13:03 |
| rpodolyaka | rushiagr: btw, one of the last commit to Heat breaks our devtest story | 13:05 |
| rpodolyaka | https://bugs.launchpad.net/heat/+bug/1245863 | 13:06 |
| rpodolyaka | I proposed two different fixes, waiting on input from Heat guys | 13:06 |
| rushiagr | rpodolyaka: oh :( | 13:07 |
| rpodolyaka | though it's easy to overcome | 13:07 |
| rushiagr | rpodolyaka: nice to know. I think changing the template files temporarily would fix this problem (whenever it will occur for me) | 13:07 |
| rushiagr | rpodolyaka: yeah | 13:07 |
| rushiagr | http://paste.openstack.org/show/50128/ | 13:08 |
| rushiagr | any help on this? except wiping off these files and downloading the images again? | 13:09 |
| *** julim has joined #tripleo | 13:09 | |
| *** anteaya has joined #tripleo | 13:10 | |
| rpodolyaka | rushiagr: try "sudo chown your_user_name:your_user_name ~/.cache/image-create/" first | 13:11 |
| rpodolyaka | oh, I forgot -R | 13:11 |
| rpodolyaka | to make it recursive | 13:11 |
| rushiagr | okay, trying | 13:11 |
| *** jdob has quit IRC | 13:11 | |
| *** jdob has joined #tripleo | 13:12 | |
| *** jeckersb_gone is now known as jeckersb | 13:12 | |
| rushiagr | rpodolyaka: that worked! it is now downloading packages | 13:13 |
| rpodolyaka | :) | 13:13 |
| *** morazi has joined #tripleo | 13:14 | |
| *** akuznetsov has joined #tripleo | 13:14 | |
| *** tzumainn has joined #tripleo | 13:17 | |
| * rushiagr just asked his colleague and got his groups and permissions myths busted | 13:17 | |
| rushiagr | I knew only two things till today -- use a 'sudo' or do a recursive 'chmod 777' :D | 13:18 |
| rpodolyaka | Ж) | 13:20 |
| rpodolyaka | :) | 13:20 |
| *** john-n-seattle has joined #tripleo | 13:27 | |
| *** jayg|g0n3 is now known as jayg | 13:27 | |
| dkehn | morning all | 13:31 |
| *** akuznetsov has quit IRC | 13:33 | |
| *** akuznetsov has joined #tripleo | 13:35 | |
| rushiagr | dkehn: good morning :) | 13:42 |
| rushiagr | rpodolyaka: is it okay if I set up squid proxy after I create the seed VM? the devtest_seed thingy is still downloading packages | 13:47 |
| rpodolyaka | rushiagr: yep, packages will also be downloaded when building images for undercloud/overcloud/user VMs | 13:48 |
| rushiagr | rpodolyaka: okay. Will the packages downloaded now (for seed VM) be cached for these VMs? or only after setting up the squid proxy will they be cached? | 13:50 |
| rushiagr | I doubt this will happen. But if you can confirm, I can stop the package downloads for seed VM right now | 13:52 |
| *** coolsvap has quit IRC | 13:54 | |
| rpodolyaka | rushiagr: nope, it will take effect only when you set http_proxy env variable and build a new image | 13:58 |
| rpodolyaka | rushiagr: though I can see fedora packages being cached in .cache/image-create/yum/. Not sure, where we set up this manual caching | 13:59 |
| *** lucas-hungry is now known as lucasagomes | 13:59 | |
| rushiagr | rpodolyaka: okay. Thanks | 14:00 |
| rpodolyaka | rushiagr: but I can't see any *.deb files in image-create and subdir | 14:00 |
| rpodolyaka | so http proxy is a universal solution | 14:01 |
| openstackgerrit | Ivan Berezovskiy proposed a change to openstack/diskimage-builder: Check existence of directory 'lost+found' before attempting to remove it https://review.openstack.org/54311 | 14:05 |
| *** noslzzp has joined #tripleo | 14:08 | |
| *** cd-undercloud has joined #tripleo | 14:09 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 14:09 |
| *** cd-undercloud has quit IRC | 14:09 | |
| rushiagr | rpodolyaka: yeah.. I am doing just that | 14:12 |
| *** julim has quit IRC | 14:12 | |
| *** spzala has joined #tripleo | 14:15 | |
| *** akuznetsov has quit IRC | 14:17 | |
| dkehn | /window/show 5 | 14:19 |
| NobodyCam | good morning TripleO | 14:29 |
| NobodyCam | morning dkehn | 14:29 |
| *** jergerber has joined #tripleo | 14:29 | |
| *** blamar has quit IRC | 14:30 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 14:31 |
| *** akuznetsov has joined #tripleo | 14:34 | |
| *** blamar has joined #tripleo | 14:34 | |
| openstackgerrit | Ivan Berezovskiy proposed a change to openstack/diskimage-builder: Check existence of directory 'lost+found' before attempting to remove it https://review.openstack.org/54311 | 14:39 |
| *** che-arne has quit IRC | 14:47 | |
| *** edmund has joined #tripleo | 14:53 | |
| *** edmund has quit IRC | 14:53 | |
| openstackgerrit | Ivan Berezovskiy proposed a change to openstack/diskimage-builder: Check existence of directory 'lost+found' https://review.openstack.org/54311 | 14:54 |
| *** spzala has quit IRC | 14:55 | |
| *** coolsvap has joined #tripleo | 14:58 | |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-image-elements: Rename heat_watch_server_url to watch_server_url https://review.openstack.org/54347 | 14:59 |
| openstackgerrit | Derek Higgins proposed a change to openstack/tripleo-heat-templates: Rename heat_watch_server_url to watch_server_url https://review.openstack.org/54348 | 14:59 |
| *** openstackgerrit has quit IRC | 15:00 | |
| *** openstackgerrit has joined #tripleo | 15:01 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Check existence of directory 'lost+found' https://review.openstack.org/54311 | 15:04 |
| *** julim has joined #tripleo | 15:05 | |
| SpamapS | rdopieralski: so is the end of the story with https://review.openstack.org/#/c/53859/ that django already has this cached_property decorator, we don't need to even import werkzeug? | 15:07 |
| rdopieralski | SpamapS: yes, but the django's one is slightly broken, so it doesn't work well with mox, so I need to work around it in tuskar-ui tests | 15:10 |
| SpamapS | rdopieralski: _ahhh_ | 15:10 |
| SpamapS | rdopieralski: and have we reported that bug to django? :) | 15:10 |
| rdopieralski | SpamapS: but I'm getting there :) | 15:10 |
| rdopieralski | SpamapS: no, because I just found it | 15:10 |
| rdopieralski | I will report it, and propose a patch | 15:11 |
| rdopieralski | but we still need the workaround for earlier versions | 15:11 |
| SpamapS | rdopieralski: yeah, totally. | 15:11 |
| * rdopieralski goes back to recursive yak shaving | 15:11 | |
| *** john-n-seattle has quit IRC | 15:12 | |
| SpamapS | rdopieralski: well thanks for staying the course. I thought copying the code was o-k but I figured you had a better reason than "don't want to change requirements" ;) | 15:12 |
| rpodolyaka | SpamapS: you might want to check this one https://bugs.launchpad.net/heat/+bug/1245863 :) | 15:13 |
| rpodolyaka | s/one/bug/ | 15:14 |
| SpamapS | Ugh yeah I totally missed the regression introduced there. | 15:15 |
| SpamapS | glaringly obvious when you know it is there. DOH. | 15:15 |
| SpamapS | why would anyone use "4" and not 4. :-P | 15:15 |
| rpodolyaka | php guys? :) | 15:16 |
| SpamapS | hah probably. :) | 15:17 |
| SpamapS | rpodolyaka: fix approved, now it just has to wind its way through the gate | 15:17 |
| rpodolyaka | thanks :) | 15:18 |
| SpamapS | no thank **YOU** | 15:18 |
| rpodolyaka | np :) | 15:18 |
| *** cd-undercloud has joined #tripleo | 15:21 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 15:21 |
| *** cd-undercloud has quit IRC | 15:21 | |
| SpamapS | still barfing on neutron rate limits or something | 15:22 |
| derekh | SpamapS: https://bugs.launchpad.net/tripleo/+bug/1245852 | 15:23 |
| *** SpamapS changes topic to "CRITICAL BUG: 1245852 | Using OpenStack to deploy OpenStack; meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 15:24 | |
| *** che-arne has joined #tripleo | 15:25 | |
| *** ruhe has joined #tripleo | 15:26 | |
| rpodolyaka | hmm, interesting. why do we have two interfaces for dhcp if there is only one neutron network? | 15:27 |
| *** john-n-seattle has joined #tripleo | 15:27 | |
| rpodolyaka | neutron doesn't clean up for himself? | 15:28 |
| SpamapS | rpodolyaka: that may be the problem | 15:29 |
| rpodolyaka | on the other hand, we don't configure ctlplane network in devtest_overcloud.sh | 15:30 |
| rpodolyaka | so it's not recreated on each deploy of overcloud | 15:30 |
| *** akuznetsov has quit IRC | 15:30 | |
| * rpodolyaka is confused | 15:30 | |
| SpamapS | rpodolyaka: networking makes my head explode. :-P | 15:31 |
| SpamapS | I almost always have to draw it. | 15:32 |
| *** akuznetsov has joined #tripleo | 15:32 | |
| *** marun has quit IRC | 15:34 | |
| *** jprovazn has quit IRC | 15:40 | |
| *** dprince has quit IRC | 15:49 | |
| openstackgerrit | Tomas Sedovic proposed a change to openstack/tripleo-image-elements: Add Horizon element https://review.openstack.org/50918 | 15:49 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 15:50 |
| SpamapS | derekh: so I imagine you're near EOD... are you still trying to work that bug or should I pick it up? | 15:53 |
| *** cwolferh has joined #tripleo | 15:54 | |
| SpamapS | 08:54 < openstackgerrit> A change was merged to openstack/heat: Make Fn::Select accept an integer selector again https://review.openstack.org/54321 | 15:55 |
| SpamapS | rdopieralski: ^^ | 15:55 |
| rdopieralski | SpamapS: ? | 15:55 |
| derekh | SpamapS: I'll be here for another bit but havn't gone back since my last update to the bug, I'll poke around on it a bit more now for a bit but feel free to pick it up also | 15:55 |
| *** akuznetsov has quit IRC | 15:56 | |
| SpamapS | rdopieralski: doh sorry, too many r's | 15:57 |
| SpamapS | rpodolyaka: ^^ | 15:57 |
| SpamapS | derekh: ok I will likely pick it up after the meeting. | 15:57 |
| *** cwolferh has quit IRC | 16:00 | |
| derekh | SpamapS: ok cool | 16:00 |
| *** ruhe has quit IRC | 16:01 | |
| derekh | If I discover anythin else before that I'll add it to the bug | 16:01 |
| *** julim has quit IRC | 16:02 | |
| *** spzala has joined #tripleo | 16:09 | |
| rpodolyaka | SpamapS: cool ;) | 16:10 |
| *** ruhe has joined #tripleo | 16:12 | |
| *** cwolferh has joined #tripleo | 16:12 | |
| *** ifarkas has quit IRC | 16:17 | |
| *** ruhe has quit IRC | 16:21 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Refactor the cached api calls into @cached_property https://review.openstack.org/53815 | 16:25 |
| rushiagr | seed vm up and running \\o// | 16:29 |
| SpamapS | rushiagr: ^5 | 16:30 |
| rushiagr | SpamapS: :) | 16:30 |
| *** rdopieralski has quit IRC | 16:31 | |
| *** MarkAtwood has joined #tripleo | 16:31 | |
| *** edmund has joined #tripleo | 16:31 | |
| *** cd-undercloud has joined #tripleo | 16:32 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 16:32 |
| *** cd-undercloud has quit IRC | 16:32 | |
| *** akuznetsov has joined #tripleo | 16:36 | |
| *** jcoufal has quit IRC | 16:39 | |
| *** viktors has left #tripleo | 16:39 | |
| openstackgerrit | Clint Byrum proposed a change to openstack/os-apply-config: Preserve symlinks when writing output https://review.openstack.org/54377 | 16:42 |
| *** dprince has joined #tripleo | 16:48 | |
| *** ruhe has joined #tripleo | 16:51 | |
| *** rushiagr has quit IRC | 16:51 | |
| *** marun has joined #tripleo | 16:54 | |
| *** julim has joined #tripleo | 17:01 | |
| *** krotscheck has joined #tripleo | 17:05 | |
| *** ruhe has quit IRC | 17:13 | |
| *** UtahDave has joined #tripleo | 17:15 | |
| *** jcoufal has joined #tripleo | 17:15 | |
| Ng | one week today it shall be TripleO Tuesday :D | 17:19 |
| *** blamar has quit IRC | 17:19 | |
| *** blamar has joined #tripleo | 17:20 | |
| SpamapS | Ng: woooot | 17:25 |
| slagle | i wonder if tripleo's will deliver burgers to the conference | 17:26 |
| openstackgerrit | Jiri Stransky proposed a change to openstack/tripleo-incubator: Respect storage pool path in create-nodes script https://review.openstack.org/54385 | 17:27 |
| Ng | slagle: I am going to go to a TripleO's at some point, irregardless of their delivery abilities :) | 17:27 |
| Ng | and I would like to take many of us with me :) | 17:27 |
| slagle | oh i'm going for sure :) | 17:27 |
| *** rpodolyaka1 has joined #tripleo | 17:35 | |
| *** derekh has quit IRC | 17:38 | |
| *** cd-undercloud has joined #tripleo | 17:42 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 17:42 |
| *** cd-undercloud has quit IRC | 17:42 | |
| *** jistr has quit IRC | 17:46 | |
| hewbrocca | Ng: what is TripleO's???? | 17:50 |
| Ng | hewbrocca: there's a Canadian burger chain that has some outlets in Hong Kong, called Triple-O's | 17:50 |
| hewbrocca | holy | 17:50 |
| hewbrocca | when are we going??? | 17:50 |
| Ng | hehe | 17:50 |
| hewbrocca | I wonder if they have t-shirts | 17:51 |
| Ng | that would be pretty cool | 17:51 |
| hewbrocca | srsly | 17:51 |
| hewbrocca | OK this is also funny: | 17:52 |
| hewbrocca | http://www.boston.com/travel/boston/mob/the_tour?pg=6 | 17:52 |
| hewbrocca | hmmm | 17:54 |
| hewbrocca | they appear to have t-shirts and hats, but only for employees? | 17:54 |
| hewbrocca | https://www.tripleos.com/about-us | 17:54 |
| *** ifarkas has joined #tripleo | 17:55 | |
| *** julim_ has joined #tripleo | 18:00 | |
| *** krotscheck has quit IRC | 18:01 | |
| *** julim has quit IRC | 18:01 | |
| pleia2 | lifeless: I left some notes under iteration 2, could use some feedback particularly on the first grey comment | 18:08 |
| * Ng dinners | 18:08 | |
| lifeless | pleia2: k | 18:09 |
| SpamapS | lifeless: 'morning | 18:16 |
| lifeless | SpamapS: blergh :P. C woke at 0530 | 18:16 |
| SpamapS | lifeless: she got you a whole extra hour of daytime! | 18:16 |
| *** krotscheck has joined #tripleo | 18:19 | |
| *** ehelms is now known as ehelms-afk | 18:19 | |
| lifeless | meeting time! | 19:01 |
| *** anteaya has quit IRC | 19:03 | |
| *** jprovazn has joined #tripleo | 19:05 | |
| *** cd-undercloud has joined #tripleo | 19:05 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 19:05 |
| *** cd-undercloud has quit IRC | 19:05 | |
| *** anteaya has joined #tripleo | 19:08 | |
| *** lucasagomes has quit IRC | 19:10 | |
| *** che-arne has quit IRC | 19:13 | |
| lifeless | Ng: are you here for the meeting ? | 19:15 |
| rpodolyaka1 | looks like somebody is going to wear a UTC watch :) | 19:15 |
| *** coolsvap has quit IRC | 19:17 | |
| *** athomas has joined #tripleo | 19:19 | |
| *** martyntaylor has joined #tripleo | 19:19 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Add James Slagle to tripleo-cd-admins. https://review.openstack.org/53587 | 19:19 |
| *** marun has quit IRC | 19:30 | |
| Ng | lifeless: I am now. we've come off daylight savings, so the meeting is now exactly co-incidental with baby-bedtime :/ | 19:37 |
| lifeless | Ng: ugh | 19:40 |
| Ng | yeah :/ | 19:40 |
| lifeless | Ng: perhaps you can paste notes from you for the meeting here in advance next meeting? | 19:40 |
| Ng | lifeless: good plan | 19:40 |
| dkehn | Ng: after running devtest.sh a few time, can one jsut go to the devtest_seed.sh and othere scripts easily | 19:45 |
| dkehn | Ng: assuming executing all the exports in there | 19:46 |
| Ng | dkehn: yeah if your environment variables are all set up | 19:46 |
| dkehn | Ng: trying to nail down the ml2 piece | 19:47 |
| *** vipul is now known as vipul-away | 19:49 | |
| *** marios has quit IRC | 19:51 | |
| *** marios has joined #tripleo | 19:51 | |
| *** vipul-away is now known as vipul | 20:05 | |
| *** athomas has quit IRC | 20:05 | |
| *** rpodolyaka1 has quit IRC | 20:11 | |
| *** cd-undercloud has joined #tripleo | 20:28 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 20:28 |
| *** cd-undercloud has quit IRC | 20:28 | |
| *** jdob has quit IRC | 20:31 | |
| *** boris-42_ has joined #tripleo | 20:32 | |
| *** spzala has quit IRC | 20:35 | |
| *** boris-42 has quit IRC | 20:35 | |
| *** markmc has joined #tripleo | 20:35 | |
| *** rdopieralski has joined #tripleo | 20:38 | |
| *** julim_ has quit IRC | 20:47 | |
| *** jprovazn has quit IRC | 21:01 | |
| *** lsmola_ has joined #tripleo | 21:02 | |
| SpamapS | lifeless: sorry was offline accidentally due to my phone not wanting to tether in saucy. | 21:03 |
| *** ifarkas has quit IRC | 21:07 | |
| *** akuznetsov has quit IRC | 21:07 | |
| *** jergerber has quit IRC | 21:07 | |
| *** lsmola has quit IRC | 21:07 | |
| *** kiall has quit IRC | 21:07 | |
| *** Ng has quit IRC | 21:07 | |
| lifeless | SpamapS: doh | 21:08 |
| *** edmund has quit IRC | 21:12 | |
| *** akuznetsov has joined #tripleo | 21:14 | |
| *** jergerber has joined #tripleo | 21:14 | |
| *** kiall has joined #tripleo | 21:14 | |
| *** Ng has joined #tripleo | 21:14 | |
| *** hobana.freenode.net sets mode: +v Ng | 21:14 | |
| SpamapS | lifeless: have you looked at the cd cloud again yet? Just about to pick that back up. | 21:17 |
| lifeless | SpamapS: no, still in meetings | 21:18 |
| lifeless | SpamapS: you saw the bug / ticket in trello? | 21:18 |
| *** kiall has quit IRC | 21:24 | |
| *** kiall has joined #tripleo | 21:26 | |
| SpamapS | lifeless: yeah, on it | 21:28 |
| *** ehelms-afk is now known as ehelms | 21:28 | |
| openstackgerrit | Jenkins proposed a change to openstack/os-collect-config: Updated from global requirements https://review.openstack.org/54426 | 21:32 |
| lifeless | SpamapS: any thoughts? | 21:50 |
| SpamapS | lifeless: I was pondering rolling back to the last commit that worked. | 21:50 |
| SpamapS | lifeless: as in roll back all git trees | 21:50 |
| lifeless | SpamapS: huh | 21:51 |
| lifeless | SpamapS: why? we know its neutron shi**ing itself | 21:51 |
| SpamapS | lifeless: it may be due to changes elsewhere. | 21:51 |
| SpamapS | have not really dug into the logs though so :-P | 21:51 |
| lifeless | SpamapS: but we're not deploying to the undercloud | 21:51 |
| lifeless | SpamapS: abstractions are useful, they let us totally ignore things contained within ;) | 21:52 |
| *** cd-undercloud has joined #tripleo | 21:52 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 21:52 |
| *** cd-undercloud has quit IRC | 21:52 | |
| SpamapS | lifeless: I'm sorry I thought it was overcloud neutron having problems... which explains why I was getting nowhere finding the errors. | 21:52 |
| lifeless | SpamapS: heat reported nova fail; nova logs report neutron fail. | 21:52 |
| lifeless | SpamapS: take 5, have coffee :> | 21:53 |
| * SpamapS has been trying to kill other distractions while digging in | 21:53 | |
| *** julim has joined #tripleo | 21:53 | |
| *** martyntaylor has left #tripleo | 22:08 | |
| *** morazi has quit IRC | 22:12 | |
| *** jayg is now known as jayg|g0n3 | 22:14 | |
| SpamapS | lifeless: I'm wondering if we just have a stuck ovs port on br-int | 22:14 |
| SpamapS | Oct 29 22:12:22|408190|netdev_linux|WARN|ioctl(SIOCGIFINDEX) on tap8e1c39ef-b8 device failed: No such device | 22:14 |
| SpamapS | Oct 29 22:12:33|408191|netdev|WARN|Dropped 45 log messages in last 12 seconds (most recently, 1 seconds ago) due to excessive rate | 22:14 |
| lifeless | ip link show dev tap8e1c39ef-b8 | 22:14 |
| lifeless | Device "tap8e1c39ef-b8" does not exist. | 22:14 |
| lifeless | no such port | 22:14 |
| SpamapS | shows in list-ports for br-int | 22:15 |
| lifeless | ovs-vsctl show | grep tap | 22:15 |
| lifeless | Port "tap8e1c39ef-b8" | 22:15 |
| lifeless | Interface "tap8e1c39ef-b8" | 22:15 |
| lifeless | so how did it get there is a question | 22:15 |
| SpamapS | lifeless: neutron bug? ovs bug? | 22:15 |
| lifeless | I suspect the firmware stuff | 22:15 |
| lifeless | pulling eth2 out from under it and then readding it | 22:15 |
| lifeless | could well have tickled something | 22:15 |
| SpamapS | yeah we may need to be more careful and quiesce the ovs daemons then | 22:15 |
| lifeless | I think we'll want to forward this to neutron, but I agree lets remove that port from ovs and see what happens | 22:16 |
| SpamapS | lifeless: ok ovs thinks it is gone. | 22:17 |
| SpamapS | just in time to delete the old overcloud | 22:18 |
| SpamapS | | c68d20dc-2d81-42d8-bff3-600d2312a508 | overcloud-notcompute-nnbzko2heo7k | BUILD | deleting | NOSTATE | ctlplane=10.10.16.174 | | 22:24 |
| *** tzumainn has quit IRC | 22:29 | |
| SpamapS | oh m I guess that still works | 22:30 |
| lifeless | we've 37 or so machines available | 22:33 |
| lifeless | a wedged one isn't the ned of the world | 22:33 |
| *** rdopieralski has quit IRC | 22:46 | |
| *** lsmola_ has quit IRC | 22:47 | |
| *** tzumainn has joined #tripleo | 22:54 | |
| *** spzala has joined #tripleo | 23:01 | |
| *** cd-undercloud has joined #tripleo | 23:13 | |
| cd-undercloud | ************** overcloud complete status=1 ************ | 23:13 |
| *** cd-undercloud has quit IRC | 23:13 | |
| lifeless | nup | 23:21 |
| lifeless | at least heat-list shows now errors this time | 23:21 |
| lifeless | SpamapS: ^ | 23:21 |
| lifeless | s/now/no/ | 23:21 |
| *** UtahDave has left #tripleo | 23:22 | |
| *** sdake has quit IRC | 23:25 | |
| *** jergerber has quit IRC | 23:31 | |
| *** jcoufal has quit IRC | 23:43 | |
| *** tzumainn has quit IRC | 23:47 | |
| *** devanand1 has joined #tripleo | 23:53 | |
| *** marios_ has joined #tripleo | 23:53 | |
| *** MarkAtwood has quit IRC | 23:54 | |
| *** markmc has quit IRC | 23:56 | |
| *** jcooley_ has joined #tripleo | 23:57 | |
| *** marios has quit IRC | 23:58 | |
| *** devananda has quit IRC | 23:58 | |
| *** lynxman has quit IRC | 23:58 | |
| *** jesusaurus has quit IRC | 23:58 | |
| *** jcooley has quit IRC | 23:58 | |
| *** jcooley_ is now known as jcooley | 23:58 | |
| *** lynxman has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!