Saturday, 2014-03-08

*** CaptTofu has joined #tripleo00:05
openstackgerritChris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud  https://review.openstack.org/7296900:07
*** rpodolyaka has quit IRC00:11
*** rpodolyaka has joined #tripleo00:12
*** tserong has joined #tripleo00:41
*** CaptTofu has quit IRC00:50
*** CaptTofu has joined #tripleo00:50
*** CaptTofu has quit IRC00:55
*** rbrady-afk is now known as rbrady00:56
rbradylifeless, SpamapS: do you have any time to review my 4 cinder/nfs patches this afternoon?00:57
*** rpodolyaka has quit IRC01:00
*** vkozhukalov has quit IRC01:05
*** blamar has joined #tripleo01:08
*** cwolferh has quit IRC01:10
*** rbrady is now known as rbrady-afk01:16
*** openstack has joined #tripleo01:22
*** coolsvap has quit IRC01:44
*** CaptTofu has joined #tripleo01:58
*** rpodolyaka has joined #tripleo02:01
*** sdake has quit IRC02:02
*** CaptTofu has quit IRC02:03
*** sdake_ has quit IRC02:03
*** rpodolyaka has quit IRC02:05
*** tzumainn has quit IRC02:07
*** blamar has quit IRC02:10
lifelessrbrady-afk: I'm travelling and jprovazn's rabbit patches are my top of queue02:16
*** rbrady-afk is now known as rbrady02:16
rbradylifeless: alright02:16
rbradylifeless: safe travels and thanks for the help02:17
lifelessrbrady: np; will look after I get through jan's work02:18
lifelessthanks02:18
*** rbrady is now known as rbrady-afk02:18
*** sdake has joined #tripleo02:19
*** sdake has quit IRC02:19
*** sdake has joined #tripleo02:19
*** sdake has quit IRC02:36
*** tserong has quit IRC02:41
*** tserong has joined #tripleo02:41
*** tserong has joined #tripleo02:41
*** newell has quit IRC02:43
*** sdake has joined #tripleo02:47
*** mestery has quit IRC03:16
*** ramishra has joined #tripleo03:39
*** CaptTofu has joined #tripleo03:46
*** CaptTofu has quit IRC03:51
killer_princeI have a doubt. when image is generated using dib, i see that the kernel and ramdisks are updated in the image. and old copies of kernel and ramdisk is left on the image. is there a reason why its not cleaned up?04:00
killer_princeis it a good idea to cleanup the image by deleting old kernel and ramdisk when generating image using dib..04:12
killer_princeor is there an element already to do this cleanup..?04:16
jammyjamkiller_prince: it sounds like you think that leaving the old copies of the kernel around is going to be a problem04:28
jammyjamnormally the first reason it would bother me is wasting disk space - nothing worse than a box with a full /boot that can't do anything any more. Second reason it would bother me is doubt over whether the kernel that's coming up next time is the same as what's running now04:29
jammyjamI don't think either of those are going to be a problem in this case?04:29
jammyjamso that just leaves tidiness, right?04:29
jammyjamI think tidiness alone is sufficient reason to want to only have current kernel/ramdisk on the image. There's no need to have a "last known good" image that I can think of, not when we're building a master that will be tested before being deployed04:30
killer_princejammyjam: I am writing an element that will install bootloader on disk when instance is cretaed. so the way grub installer works is that it picks up the kernel and ramdisk with latest time stamps as default boot entry. and this results in some cases where instead of picking up generic kernel, it picks up virtual kernel.04:32
killer_princejammyjam: and there is where my problem starts. when i boot such image on baremetal, it will not detect all devices and even result in kernel panic..04:33
killer_princeso wanted to check if there is a reason to keep those kernels on the disk..04:34
jammyjamThis is the project to enable the nodes to boot from bare metal if pxe-boot fails?04:34
killer_princeyes..04:34
jammyjamIt sounds to me as though there's two possible solutions here - either make sure that your intended kernel/ramdisk are the only ones around, or start tinkering with the grub config to be more explicit about what it should pick up04:35
killer_princeyup.. i want to keep it simple and avoid future maintainance if grub config format changes.. so was looking for the former solution..04:37
killer_princejammyjam: what is your opinion on this..04:39
jammyjammy considered opinion, after thinking about it for 20 minutes?04:40
killer_princejammyjam: take your time..04:41
jammyjammy understanding is that the baremetal boot-off-disk is not intended for regular production use, it's only there as a fallback in case pxeboot wasn't available right then - it's only intended to get the box back up so that it's contactable and we can figure out what's wrong with it and reboot once pxe is available04:42
jammyjamis that right?04:42
jammyjamIf so, you probably (A) don't want it to boot up from a virtual kernel, and (B) don't want your fallback system to fall back to an older kernel. Cleaning them up sounds like the right thing to do to me.04:44
killer_princepartially correct. however i have seen customers who would like to use PXE only for deployment and not for production use.. they simply insist that they want the instances to boot from the disk rather than pxe.. (not sure why)..04:44
*** ramishra has quit IRC04:44
jammyjamI don't know if there's an existing element though04:44
*** weshay has quit IRC04:45
jammyjamI can think of a few possibilities - maybe they think booting from disk will be faster and more robust? Maybe the machines get moved to another part of the network after boot and can't reach the PXE server any more?04:45
killer_princemay be.. so my next doubt would be shall i write a seperate element to do this cleanup or shall i include this in my element itself..04:45
jammyjamMaybe they have security auditors who are worried that doing PXE boot every time means you can't be sure what code is running on the machine?04:46
jammyjamThat's a question I don't have an answer for :) Either way we can move it later..04:46
killer_princethey had some security concerns and some scalability concerns..04:46
killer_princeokay... then i guess i iwll include them in my elemnt for now, if needed we can rework later to a seperate element..04:47
*** jammyjam is now known as tchaypo05:18
*** noslzzp has quit IRC05:33
*** CaptTofu has joined #tripleo05:35
*** CaptTofu has quit IRC05:39
*** sdake_ has joined #tripleo06:01
*** sdake_ has quit IRC06:01
*** sdake_ has joined #tripleo06:01
*** tzumainn has joined #tripleo06:02
* SpamapS finally returns bleh06:31
SpamapSI think there are plenty of valid reasons to want to boot from disk later on. It just increases the testing burden to test both scenarios.06:36
SpamapSkiller_prince: ^^06:36
killer_princeSpamapS: you mean testing the element or testing instances..06:38
SpamapSkiller_prince: I mean testing the image.06:38
killer_princeSpamapS: well my approach is that when this element is used, it will continue to boot from PXE. but it will fall back to localdisk only when pxe fails..06:40
killer_princeSpamapS: and so for testing all we need to do is remove pxe config from tftp and it should come up from localdisk.06:41
killer_princeSpamapS: and the boot loader will get installed upon first pxe boot of instance..06:41
SpamapSkiller_prince: "it will fall back" is a bios problem.. not a user space image controllable problem.. AFAIK06:42
SpamapSkiller_prince: which is one reason why it is harder to test06:42
killer_princeSpamapS: well thats true. but once it falls back, disk should have bootloader configured properly so that it can continue booting from localdisk.06:47
killer_princeSpamapS: i agree, its hard to test. but that should not be the reason not to have it there..06:47
killer_princemy opinion..06:47
SpamapSkiller_prince: it is not a reason to not do it. It is a reason to require more resources to get it done. ;)06:49
killer_princeSpamapS: aha.. that way06:51
*** ramishra has joined #tripleo06:57
*** akuznetsov has quit IRC07:19
*** CaptTofu has joined #tripleo07:23
*** CaptTofu has quit IRC07:28
*** GheRivero has quit IRC07:34
*** jtomasek has quit IRC07:46
*** killer_prince is now known as lazy_prince07:55
*** GheRivero has joined #tripleo08:35
*** nijaba_ has joined #tripleo08:49
*** jeblair_ has joined #tripleo08:52
*** Slower_ has joined #tripleo08:52
*** dtantsur_ has joined #tripleo08:54
*** jeblair has quit IRC08:56
*** jrist has quit IRC08:56
*** Slower has quit IRC08:56
*** dtantsur has quit IRC08:56
*** nijaba has quit IRC08:56
*** greghaynes has quit IRC08:56
*** dtantsur_ is now known as dtantsur08:56
*** ramishra has quit IRC08:59
*** jrist has joined #tripleo09:01
*** greghaynes has joined #tripleo09:02
*** CaptTofu has joined #tripleo09:11
*** rcarrillocruz1 has joined #tripleo09:11
*** rcarrillocruz has quit IRC09:14
*** rpodolyaka has joined #tripleo09:14
*** rcarrillocruz has joined #tripleo09:15
*** CaptTofu has quit IRC09:16
*** ramishra has joined #tripleo09:16
*** rcarrillocruz1 has quit IRC09:17
*** vkozhukalov has joined #tripleo09:29
*** ramishra has quit IRC09:34
*** ramishra has joined #tripleo09:49
*** rcarrillocruz1 has joined #tripleo09:52
*** rpodolyaka has quit IRC09:53
*** rcarrillocruz has quit IRC09:55
*** vkozhukalov has quit IRC10:00
*** ramishra has quit IRC10:11
*** CaptTofu has joined #tripleo10:51
*** akuznetsov has joined #tripleo11:07
*** akuznetsov has quit IRC11:15
*** martyntaylor has joined #tripleo11:18
*** d0ugal has joined #tripleo11:18
*** d0ugal has joined #tripleo11:18
*** martyntaylor has left #tripleo11:32
*** rcarrillocruz1 has quit IRC12:12
*** rcarrillocruz1 has joined #tripleo12:15
*** CaptTofu has quit IRC12:29
*** killer_p- has joined #tripleo14:46
*** killer_p- is now known as killer_prince14:46
*** lazy_prince has quit IRC14:46
*** tzumainn has quit IRC15:00
*** jeblair_ is now known as corvus15:40
*** ramishra has joined #tripleo16:22
*** ramishra has quit IRC16:26
*** bauzas has joined #tripleo17:14
lifelessmorning17:15
*** jang1 has joined #tripleo17:28
*** akuznetsov has joined #tripleo17:40
*** petertoft has joined #tripleo17:47
mordredmorning lifeless18:00
mordredlifeless: how was the flight back?18:00
lifelessmordred: it was ok; I managed to finangle an upgrade (200 miles spent as a bid; elite inflates that by 50% = win18:04
mordredlifeless: woot18:06
* lifeless disables the CRDA udev rule18:07
lifelesshmm, not that one18:09
lifelessmore fiddling needed18:09
lifelessbut I am getting closer to tracking this EPIC FAIL down18:09
lifelessAP roams18:11
lifelessregulatory domain gets updated to world18:11
lifelessthen back to nz18:11
lifelessand the card goes 'fuckit, I'm confused'18:11
lifelessbecause the channel that was in use gets blocked by the temporary world domain18:12
lifeless*I* think18:12
lifelessand the card drops off the ssid18:12
lifelessI'm happy because to have a working theory now, since this has been pissing me off for years :)18:14
lifelessbah, english, do I speak it?18:15
lifelessor possibly, its merely changing crda that glitches things. more investigation needed.18:19
lifelessok, ttg18:29
*** jang1 has quit IRC18:33
*** akuznets_ has joined #tripleo19:53
*** akuznetsov has quit IRC19:55
SpamapSlifeless: regarding the mv and space allocation, I believe the problem is that the loopdev/ext4 is allocating space faster than the mv is freeing it, which is why we haven't seen the out of space until we got to the point where the tmpfs was already 75%+ full before we started the mv.20:12
*** hashar has joined #tripleo20:18
SpamapSlifeless: is there any reason we build and upload the 'user' image in devtest instead of just uploading saucy?20:19
*** akuznets_ has quit IRC21:10
*** hashar has quit IRC21:10
*** akuznetsov has joined #tripleo21:17
*** hashar has joined #tripleo21:18
*** andreaf has joined #tripleo21:23
*** CaptTofu has joined #tripleo21:30
*** andreaf has quit IRC21:31
*** petertoft has quit IRC21:43
*** steels has joined #tripleo21:49
SpamapSwe should have the "waiting for the overcloud stack to be ready" tail os-collect-config logs on all the nodes21:55
SpamapSor at least spit out the heat event list21:55
* SpamapS goes to do saturday things21:55
*** hashar has quit IRC21:56
*** CaptTofu has quit IRC22:21
*** CaptTofu has joined #tripleo22:22
*** d0ugal has quit IRC22:23
*** CaptTofu has quit IRC22:26
lifelessSpamapS: we could upload cirros for that matter22:35
lifelessSpamapS: logstash :)22:35
mordredlifeless: ++23:20
mordredlifeless: poke jesusaurus about that - he's on the hook on the hp cloud side for all things logstash23:21
*** CaptTofu has joined #tripleo23:23
jesusaurus<_<23:24
jesusaurus>_>23:25
*** CaptTofu has quit IRC23:28
* mordred throws bunnyrabbit at jesusaurus23:32
SpamapS2014-03-08 22:06:28.974 | Build timed out (after 145 minutes). Marking the build as failed.23:41
SpamapSthings were working otherwise23:41
SpamapShm we need the same thing as seed_logs.tgz  but for overcloud23:42
mordredSpamapS: BUILD FASTER23:42
SpamapSmuch vrooom23:43
SpamapSmordred: local pypi/apt cache will help a bit23:43
SpamapSmordred: and wheels23:43
mordredword23:43
SpamapSand cloud image for that matter23:43
SpamapSmordred: https://etherpad.openstack.org/p/tripleo-parallel-test FYI23:44
SpamapSbut anyway..23:44
SpamapSI think so we can start approving changes we should just allow 20 more minutes23:44
mordredSpamapS: I don't believe in testing things parallel23:44
SpamapSmordred: it should be tripleo-tests-are-zomg-slow23:45
SpamapSits just a bitch fest list of all the things we should be doing :)23:45
*** bauzas has quit IRC23:46
SpamapSanyway...23:46
SpamapSmuch weekend23:46
SpamapSso sun23:46
SpamapSvery outside23:46
*** rcarrillocruz has joined #tripleo23:48
*** rcarrillocruz1 has quit IRC23:49

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!