Thursday, 2013-12-05

*** rongze has quit IRC00:02
*** openstackgerrit has quit IRC00:06
*** noslzzp has quit IRC00:06
*** openstackgerrit has joined #tripleo00:06
*** morazi has quit IRC00:07
SpamapSow00:08
SpamapSIllegal OpCode00:09
SpamapSon the console of one of the servers.00:09
SpamapSthat _can't_ be good00:09
*** cd-undercloud has joined #tripleo00:11
cd-undercloud************** overcloud complete status=1 ************00:11
*** cd-undercloud has quit IRC00:11
*** CaptTofu has joined #tripleo00:20
SpamapShrm, now DHCP doesn't seem to be reaching undercloud00:21
* SpamapS -> kids00:21
*** matsuhashi has joined #tripleo00:21
*** epim has quit IRC00:26
*** martyntaylor has quit IRC00:32
*** cd-undercloud has joined #tripleo00:36
cd-undercloud************** overcloud complete status=1 ************00:36
*** cd-undercloud has quit IRC00:36
*** krotscheck has quit IRC01:21
*** sballe has quit IRC01:24
*** rongze has joined #tripleo01:28
*** matsuhashi has quit IRC01:31
*** rongze has quit IRC01:33
*** matsuhas_ has joined #tripleo01:33
*** cd-undercloud has joined #tripleo01:34
cd-undercloud************** overcloud complete status=1 ************01:34
*** cd-undercloud has quit IRC01:34
*** nosnos has joined #tripleo01:36
*** flwang has quit IRC01:49
*** cd-undercloud has joined #tripleo01:59
cd-undercloud************** overcloud complete status=1 ************01:59
*** cd-undercloud has quit IRC01:59
*** boris-42 has quit IRC02:02
*** openstack has joined #tripleo02:18
*** openstackgerrit has quit IRC02:24
*** openstackgerrit has joined #tripleo02:24
*** jayg is now known as jayg|g0n302:30
*** jcooley_ has joined #tripleo02:35
*** matsuhas_ has quit IRC02:35
*** flwang has joined #tripleo02:50
*** rongze has joined #tripleo02:53
*** vkozhukalov has joined #tripleo02:55
*** rongze_ has joined #tripleo02:55
*** cd-undercloud has joined #tripleo02:58
cd-undercloud************** overcloud complete status=1 ************02:58
*** cd-undercloud has quit IRC02:58
*** rongze has quit IRC02:58
*** jcooley_ has quit IRC03:17
*** sdake_ has quit IRC03:21
*** cd-undercloud has joined #tripleo03:23
cd-undercloud************** overcloud complete status=1 ************03:23
*** cd-undercloud has quit IRC03:23
*** CaptTofu has quit IRC03:24
openstackgerritJun Jie Nan proposed a change to openstack/diskimage-builder: Fix no busybox symlinks issue on rhel  https://review.openstack.org/5994003:28
lifelesspython03:31
*** CaptTofu has joined #tripleo03:36
*** cwolfe__ has joined #tripleo03:45
*** cwolferh_ has quit IRC03:47
*** tzumainn has quit IRC03:59
*** tzumainn has joined #tripleo04:05
*** matsuhashi has joined #tripleo04:14
*** cd-undercloud has joined #tripleo04:23
cd-undercloud************** overcloud complete status=1 ************04:23
*** cd-undercloud has quit IRC04:23
*** michchap_ has joined #tripleo04:24
*** michchap has quit IRC04:26
*** sdake_ has joined #tripleo04:29
*** rongze_ has quit IRC04:35
*** michchap has joined #tripleo04:35
*** CaptTofu has quit IRC04:36
*** CaptTofu has joined #tripleo04:36
*** michchap_ has quit IRC04:38
*** cd-undercloud has joined #tripleo04:47
cd-undercloud************** overcloud complete status=1 ************04:47
*** cd-undercloud has quit IRC04:47
*** bauzas has quit IRC04:55
*** marun has joined #tripleo05:03
SpamapSlifeless: traffic is not making it from br-ctlplane into the DHCP netns. Halp?05:18
lifelessSpamapS: hi05:25
lifelessSpamapS: on host?05:25
*** rushiagr has joined #tripleo05:27
lifelessSpamapS: assuming cd-undercloud-notcompute05:38
lifelessSpamapS: ovs show + ovs ofctl dump-flows is useful here05:38
lifelessSpamapS: we expect br-ctlplane -> br-int and from there a tap device into the agents05:39
lifelessSpamapS: so from the netns we can see the tap device is tap56efc4c6-5505:40
lifelesswhich is indeed plugged into br-int05:40
lifelesswith tag 105:40
lifelessSpamapS: we can see that br-int has 3 flows, dropping traffic from ports 2 and 3 and flooding everything else05:41
lifelessSpamapS: ovs-ofctl show br-int gets you port ids05:42
lifeless2 and 3 are taps but not the dhcp one05:42
lifelessSpamapS: so it looks fine to me05:42
lifelessSpamapS: nothing obvious in iptables-save that would cause issues05:43
*** cd-undercloud has joined #tripleo05:46
cd-undercloud************** overcloud complete status=1 ************05:46
*** cd-undercloud has quit IRC05:46
*** sdake_ has quit IRC05:49
*** noslzzp has joined #tripleo06:09
*** cd-undercloud has joined #tripleo06:11
cd-undercloud************** overcloud complete status=1 ************06:11
*** cd-undercloud has quit IRC06:11
*** matsuhashi has quit IRC06:14
*** rushiagr2 has joined #tripleo06:14
*** matsuhashi has joined #tripleo06:14
*** rushiagr has quit IRC06:15
*** rpodolyaka1 has joined #tripleo06:17
*** rushiagr2 is now known as rushiagr06:24
openstackgerritMark McLoughlin proposed a change to openstack/diskimage-builder: Log the repository fetches in source-repositories  https://review.openstack.org/6003506:30
*** agordeev has joined #tripleo06:43
*** vkozhukalov has quit IRC06:48
lifelessSpamapS:06:50
lifeless# tcpdump -enli br-int port bootpc06:50
lifelesstcpdump: br-int: That device is not up06:50
*** pblaho has joined #tripleo06:50
lifelessok, so ifconfig that up; now I see it on br-int06:50
StevenKlifeless: ITYM 'ip l s dev <blah> up' ? :-)06:51
lifelessStevenK: 'meh'06:51
StevenKHaha06:52
lifelessso06:52
lifelessmy suspicion would be the process for tap56efc4c6-55 is awol06:52
*** noslzzp has quit IRC06:52
*** rongze has joined #tripleo06:54
*** flwang has left #tripleo06:55
*** akuznetsov has quit IRC06:55
*** cd-undercloud has joined #tripleo07:10
cd-undercloud************** overcloud complete status=1 ************07:10
*** cd-undercloud has quit IRC07:10
*** rpodolyaka1 has quit IRC07:11
*** kmsWork_ has joined #tripleo07:17
*** matsuhashi has quit IRC07:17
*** kmsWork has quit IRC07:19
*** akuznetsov has joined #tripleo07:23
*** akuznetsov has quit IRC07:26
openstackgerritJun Jie Nan proposed a change to openstack/diskimage-builder: Fix no busybox symlinks issue on rhel  https://review.openstack.org/5994007:26
*** tzumainn has quit IRC07:29
*** jcoufal has joined #tripleo07:30
*** cd-undercloud has joined #tripleo07:34
cd-undercloud************** overcloud complete status=1 ************07:34
*** cd-undercloud has quit IRC07:34
*** boris-42 has joined #tripleo07:35
*** ifarkas has joined #tripleo07:50
*** lsmola has joined #tripleo07:57
rpodolyakamorning tripleo07:59
*** akuznetsov has joined #tripleo08:07
*** jprovazn has joined #tripleo08:16
*** ifarkas has quit IRC08:21
*** GheRivero has joined #tripleo08:23
*** panda has joined #tripleo08:26
*** vipul is now known as vipul-away08:27
*** ifarkas has joined #tripleo08:27
*** bauzas has joined #tripleo08:30
*** vkozhukalov has joined #tripleo08:32
*** cd-undercloud has joined #tripleo08:34
cd-undercloud************** overcloud complete status=1 ************08:34
*** cd-undercloud has quit IRC08:34
*** vipul-away is now known as vipul08:37
*** rdopieralski has joined #tripleo08:38
*** SlickNik has quit IRC08:39
*** SlickNik has joined #tripleo08:40
*** akrivoka has joined #tripleo08:41
*** boris-42 has quit IRC08:57
*** cd-undercloud has joined #tripleo08:58
cd-undercloud************** overcloud complete status=1 ************08:58
*** cd-undercloud has quit IRC08:58
*** jistr has joined #tripleo08:58
Ngmorning09:02
lifelessmorning ng09:09
lifelessSpamapS: (hiatus with me is C, obviously :P)09:09
*** athomas has joined #tripleo09:10
*** boris-42 has joined #tripleo09:16
lifelessSpamapS: and yeah, my suspicion was right - service neutron-openvswitch-agent restart, et voila09:20
*** lucasagomes has joined #tripleo09:25
*** jtomasek has joined #tripleo09:27
*** jcoufal has quit IRC09:30
lifelessrpodolyaka: around?09:32
rpodolyakalifeless: yep09:32
rpodolyakareview time :)09:33
lifelessrpodolyaka: so hey, wanted to sync on rebuild09:36
rpodolyakasure09:36
lifelessrpodolyaka: we chatted earlier in the week about the disconnect we had09:36
rpodolyakaso, I tried to push it down the stack09:36
rpodolyakacuriously, tests work09:36
*** jtomasek has quit IRC09:36
lifelessrpodolyaka: I'm wondering how the revamp is going, and wanted to suggest that I spend tomorrow afternoon hacking on it09:36
rpodolyakaboth unit and devstack gates09:36
lifelessrpodolyaka: cool09:37
lifelessship it ! ;)09:37
rpodolyakahttps://review.openstack.org/#/c/59468/09:37
rpodolyakahave you already seen this one?09:37
rpodolyaka+ I made small fix to your patch adding a table column09:38
lifelessawesome09:39
rpodolyakaok, so I'm going to write a commit message for the rebuild patch and remove the WIP status09:40
lifelessrpodolyaka: what happened to all the API bits - are they sitting locally for now?09:40
rpodolyakalifeless: yes09:40
lifelessok09:40
rpodolyakalifeless: going to hack on this today09:40
lifelesssweet09:40
lifelessif you push everything - WIP and non-WIP when you're done tonight, I'll try to hack on it my tomorrow09:41
rpodolyakagot side tracked yesterday pinging neutron team09:41
lifelessI'm itching to get some code written :)09:41
rpodolyakacool!09:41
rpodolyakame too!09:41
rpodolyaka:)09:41
* rpodolyaka got a small team here at mirantis09:41
lifelessah, the perils of being a team lead?09:41
rpodolyakayeah... mentoring is hard :)09:42
*** max_lobur_afk is now known as max_lobur09:45
*** jtomasek has joined #tripleo09:50
lifelessok, night all09:55
*** cd-undercloud has joined #tripleo09:57
cd-undercloud************** overcloud complete status=1 ************09:57
*** cd-undercloud has quit IRC09:57
*** jcoufal has joined #tripleo09:58
*** athomas has quit IRC10:00
*** akrivoka has quit IRC10:01
*** akrivoka has joined #tripleo10:02
*** jtomasek has quit IRC10:04
*** lsmola has quit IRC10:05
*** martyntaylor has joined #tripleo10:09
openstackgerritJun Jie Nan proposed a change to openstack/diskimage-builder: Add examples for ramdisk-image-create  https://review.openstack.org/6019310:12
*** derekh has joined #tripleo10:13
*** hewbrocca has joined #tripleo10:16
*** lsmola has joined #tripleo10:17
*** jtomasek has joined #tripleo10:20
*** michchap has quit IRC10:30
*** michchap has joined #tripleo10:31
*** jprovazn has quit IRC10:41
*** akrivoka has quit IRC10:46
*** akrivoka has joined #tripleo10:47
*** boris-42 has quit IRC10:47
*** lsmola has quit IRC10:49
*** athomas has joined #tripleo10:55
*** markmc has joined #tripleo10:55
*** cd-undercloud has joined #tripleo10:59
cd-undercloud************** overcloud complete status=0 ************10:59
*** cd-undercloud has quit IRC10:59
Ng:)11:01
*** boris-42 has joined #tripleo11:02
*** lsmola has joined #tripleo11:02
*** rongze has quit IRC11:12
*** jprovazn has joined #tripleo11:15
*** akrivoka has quit IRC11:18
*** akrivoka has joined #tripleo11:33
*** jcoufal has quit IRC11:36
vkozhukalovtried to build image with diskimage-builder, included some tripleo elements from tripleo-image-elements, scripts failes with the error "DIB_INIT_SYSTEM: unbound variable". the question is how to use this variable?11:47
*** morazi has joined #tripleo11:48
*** rha has joined #tripleo11:48
vkozhukalovand why is it not installed into some default?11:50
GheRiverovkozhukalov: i'll take a look to it11:52
GheRiverocan you paste the command you were using?11:52
*** morazi has quit IRC11:53
*** cd-undercloud has joined #tripleo11:53
cd-undercloud************** overcloud complete status=0 ************11:53
*** cd-undercloud has quit IRC11:53
vkozhukalovGheRivero, disk-image-create -x -u -a amd64 -o /var/tmp/ieS8rohk vm ironic 2>&1 | tee /var/tmp/ieS8rohk.log11:54
vkozhukalovGheRivero, error line in log is "/tmp/in_target.d/install.d/10-os-collect-config: line 14: DIB_INIT_SYSTEM: unbound variable"11:55
GheRiverocan you see something like: Please include at least one distribution root element. in the logs?11:56
vkozhukalovGheRivero, no, i can't11:57
*** rongze has joined #tripleo11:57
vkozhukalovGheRivero, I can send you the log file if you need.11:59
GheRiveroyeah, you can paste it. Is the ironic element in a review?11:59
GheRiveroforget it12:00
vkozhukalovGheRivero, it is already in master.12:01
GheRiveroyeah. i forgot to update my local copy :)12:02
*** morazi has joined #tripleo12:06
GheRiverovkozhukalov: did you paste the output?12:09
vkozhukalovGheRivero, log file http://rghost.net/5070412912:11
GheRiveroserver not found :(12:12
GheRiverowhen downloading it12:12
vkozhukalovGheRivero, one min12:12
vkozhukalovGheRivero, https://drive.google.com/file/d/0BybDDjx4oqkYTEphT2hDX1FzTkU/edit?usp=sharing12:14
GheRiveroyou can also use paste.openstack.org12:14
GheRivero:)12:14
vkozhukalovGheRivero, ok, will know12:14
*** rongze has quit IRC12:17
GheRiveroi just build it without problems...12:19
GheRiverowill check env variables12:19
openstackgerritA change was merged to openstack/diskimage-builder: Fix no busybox symlinks issue on rhel  https://review.openstack.org/5994012:24
GheRiverovkozhukalov: do you have dib updated?12:26
*** rongze has joined #tripleo12:35
vkozhukalovGheRivero, wow. You are right. Im sorry I have not updated my dib.12:37
vkozhukalovGheRivero, thanks a lot lot lot12:37
GheRivero:)12:37
GheRiveroi hate this computer (athlon k7). My HD dies yesterday and still waiting for a replacement12:39
*** rongze_ has joined #tripleo12:41
*** rongze has quit IRC12:42
*** nosnos has quit IRC12:45
*** cd-undercloud has joined #tripleo12:49
cd-undercloud************** overcloud complete status=0 ************12:49
*** cd-undercloud has quit IRC12:49
*** CaptTofu has quit IRC12:51
*** CaptTofu has joined #tripleo12:51
shadowerhave you ever run out of memory on the overcloud-in-a-vm?12:55
shadowerI'm getting 'ERROR: An unexpected error prevented the server from fulfilling your request. [Errno 12] Cannot allocate memory (HTTP 500)' after a few redeploys of the overcloud12:55
shadowerthis on a standard devtest, 64bit vms, 2048 RAM12:56
*** athomas has quit IRC12:57
ifarkasshadower, yeah, I sometimes had this issue and had to rebuild the undercloud13:00
jprovaznshadower, I think I've seen this about a month ago, yesterday I've hit this on undercloud vm - for undercloud the issue is with keystone and bunch of generated tokens in DB (which keystone seems to keep also in memory)13:00
ifarkasshadower, I redeployed the it, doubled the available ram and it has be up for a couple weeks13:01
jprovaznshadower, could you check memory on your vm and ideally also check size of "token" table in keystone db?13:01
shadowerjprovazn: it is keystone that's hitting the issue13:02
jprovaznah :)13:02
*** rbrady has joined #tripleo13:02
jprovaznshadower, and which processes take msot of mem?13:03
shadowerjprovazn: I've about 170mb of free memory, keystone-all takes the most mem according to top13:03
openstackgerritA change was merged to openstack/diskimage-builder: Add examples for ramdisk-image-create  https://review.openstack.org/6019313:04
shadowerand when I check the logs, the issue is in keystone-all in issue_v2_token13:05
shadowermore specifically, there is  a os.fork() command that fails13:05
openstackgerritA change was merged to openstack/diskimage-builder: Log the repository fetches in source-repositories  https://review.openstack.org/6003513:06
jprovaznshadower, and how much mem does keystone take atm? more than 170M?13:09
shadowerjprovazn: that's the RES column, right?13:10
jprovaznshadower, yes13:10
jprovaznshadower, when fork is done, memory is copied to the new process, so if parent keystone process is big enough, it might be the reason13:11
shadowerabout 450mb if I read it rigt13:11
shadoweryeah13:11
jprovaznshadower, as a hotfix, you can try:13:12
jprovaznmysql -u keystone -p keystone13:12
jprovazn(password 'unset')13:12
jprovaznthen "delete from token;" (this works for me for undercloud, not sure if you have some tokens in overcloud)13:13
jprovaznif yes, something like "delete from token where expire>some_Date" might be better13:13
shadowerjprovazn: my issue is in the undercloud too13:13
jprovaznshadower, ah, I read overcloud above13:13
jprovaznshadower, ok, in that case it should work13:13
jprovaznthen restart keystone13:13
shadowerI wrote it wrong, sorry13:13
shadowerjprovazn: sweet, thanks!13:16
jprovaznshadower, yaw13:17
jprovazngoing to look what exactly causes this "token spree"13:17
shadoweryeah if keystone's leaking memory like that, surely that's a bug. Especially if it needs to fork on every os cli request13:17
shadowerthe forking seems to be needed because of openssl signing13:18
*** lsmola has quit IRC13:22
*** CaptTofu has quit IRC13:25
*** CaptTofu has joined #tripleo13:25
*** CaptTofu has quit IRC13:35
*** jdob has joined #tripleo13:35
*** CaptTofu has joined #tripleo13:35
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New structure of panels  https://review.openstack.org/6023113:38
rdopieralskiwheee13:39
*** noslzzp has joined #tripleo13:40
rdopieralskiboilerplate :)13:41
*** cd-undercloud has joined #tripleo13:42
cd-undercloud************** overcloud complete status=42 ************13:42
*** cd-undercloud has quit IRC13:42
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New structure of panels  https://review.openstack.org/6023113:47
*** athomas has joined #tripleo13:48
*** ccrouch has joined #tripleo13:50
*** jistr has quit IRC13:57
*** jistr has joined #tripleo13:58
*** dprince has joined #tripleo13:59
*** morazi has quit IRC14:01
*** tzumainn has joined #tripleo14:02
*** jcoufal has joined #tripleo14:03
*** julim has joined #tripleo14:12
*** morazi has joined #tripleo14:12
openstackgerritRalf Haferkamp proposed a change to openstack/diskimage-builder: Add support for building openSUSE images  https://review.openstack.org/5615714:20
*** matty_dubs|gone is now known as matty_dubs14:21
*** jayg|g0n3 is now known as jayg14:28
*** cd-undercloud has joined #tripleo14:33
cd-undercloud************** overcloud complete status=0 ************14:33
*** cd-undercloud has quit IRC14:33
*** john-n-seattle1 has quit IRC14:33
NobodyCamgood morning TripleO14:37
*** jdob has quit IRC14:38
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New structure of panels  https://review.openstack.org/6023114:40
*** sballe has joined #tripleo14:49
*** jdob has joined #tripleo14:51
*** john-n-seattle1 has joined #tripleo14:53
*** jcoufal has quit IRC14:56
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: New structure of panels  https://review.openstack.org/6023114:58
*** sdake has quit IRC15:02
*** sdake has joined #tripleo15:03
*** GheRivero has quit IRC15:04
*** pblaho has quit IRC15:08
*** marun has quit IRC15:23
*** cd-undercloud has joined #tripleo15:25
cd-undercloud************** overcloud complete status=0 ************15:25
*** cd-undercloud has quit IRC15:25
*** jergerber has joined #tripleo15:27
SpamapSawright.. we're deployin clouds again :)15:50
derekhhip hip15:57
*** jcooley_ has joined #tripleo15:58
*** jprovazn has quit IRC15:58
*** jcooley_ has quit IRC15:59
*** vkozhukalov has quit IRC16:00
SpamapSderekh: now you're just asking for it. ;)16:03
SpamapSso now I feel like we have to attack the mellanox issues..16:04
*** jcooley_ has joined #tripleo16:06
*** boris-42 has quit IRC16:11
*** jprovazn has joined #tripleo16:12
*** jcooley_ has quit IRC16:12
*** jcooley_ has joined #tripleo16:13
*** ifarkas has quit IRC16:14
*** max_lobur is now known as max_lobur_afk16:15
*** cd-undercloud has joined #tripleo16:17
cd-undercloud************** overcloud complete status=0 ************16:17
*** cd-undercloud has quit IRC16:17
*** jcooley_ has quit IRC16:18
*** akrivoka has quit IRC16:23
SpamapSdprince: you working on enable/disable today?16:24
dprinceSpamapS: I've had some interruptions, but I probably can16:26
dprinceSpamapS: were you anxious to jump in there?16:26
SpamapSdprince: I need to do some reviews and then if you're not started I'll grab the baton back.16:27
SpamapSdprince: probably ~2 hours16:28
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add an ntp element  https://review.openstack.org/6007916:29
*** rongze_ has quit IRC16:37
*** akuznetsov has quit IRC16:44
*** bauzas has quit IRC16:45
dprinceSpamapS: so with the suggested new approaches we would need to enable use-ephemeral then?16:46
* dprince has not used it before...16:46
SpamapSdprince: right, you need to store state in places that use-ephemeral knows about16:47
*** cwolfe__ has quit IRC16:50
dprinceSpamapS: So we aren't doing that for our config files today (yet) so I feel sort of like this could come after we fix service enable/disable16:51
*** cwolferh has joined #tripleo16:51
*** jcooley_ has joined #tripleo16:51
dprinceSpamapS: perhaps you already agree and I'm just assuming we'd do this first. Let me just throw some code up there.16:52
*** jcooley_ has quit IRC16:54
*** jcooley_ has joined #tripleo16:55
*** jcooley_ has quit IRC16:57
*** athomas has quit IRC16:57
*** jcooley_ has joined #tripleo16:57
SpamapSdprince: I do agree, and it is also on my todo list to do it for the config files :)16:58
SpamapSdprince: either can come first.16:58
SpamapSI should say it is on my very short-term list16:58
dprinceSpamapS: agree we do16:58
*** rdopieralski has quit IRC17:01
*** jcooley_ has quit IRC17:02
*** athomas has joined #tripleo17:04
*** jcooley_ has joined #tripleo17:06
*** markmc has quit IRC17:06
openstackgerritA change was merged to openstack/tuskar-ui: New structure of panels  https://review.openstack.org/6023117:09
*** CaptTofu has quit IRC17:10
*** CaptTofu has joined #tripleo17:10
*** CaptTofu has quit IRC17:12
*** CaptTofu has joined #tripleo17:12
*** cd-undercloud has joined #tripleo17:12
cd-undercloud************** overcloud complete status=0 ************17:12
*** cd-undercloud has quit IRC17:12
*** matty_dubs is now known as matty_dubs|lunch17:13
*** julim has quit IRC17:16
*** julim has joined #tripleo17:17
*** martyntaylor has quit IRC17:18
openstackgerritA change was merged to openstack/diskimage-builder: Add support for building openSUSE images  https://review.openstack.org/5615717:24
derekhopenSUSE welcome to disk-image-builder :-)17:26
Ng\o/17:27
derekhpleia2: so I booted up a sample/manual test env host and geard server and started running commands (for the easy stuff) https://etherpad.openstack.org/p/tripleo-test-cluster17:29
pleia2derekh: great17:30
*** jtomasek has quit IRC17:31
derekhpleia2: have manually started setting up 2 test envs on the host (see commands in etherpad), we now have http://paste.openstack.org/show/54525/17:31
derekhpleia2: don't be fooled into thinking those VM's do anything, their blank disks17:32
derekh;-)17:32
pleia2derekh: so with boot-seed-vm I think we need two conditions added, 1. don't run configure-vm 2... browsing now because I forgot (coffee is brewing as we speak, yay!)17:34
derekhpleia2: dprince so I think its mostly neutron hokus pokus from here, I'll pick up on that in the morning, or if ye want to try out anything fire ahead and add what you did yo the etherpad17:35
pleia2derekh: great, I'll give it a spin now17:35
derekhpleia2: ya we want one boot-seed-vm todo everything except run DIB and the other to only run DIB17:36
derekhpleia2: actually the second condition is done by passing in the node to not boot VM17:36
pleia2derekh: so I was thinking I'd actually run boot-seed-vm on the nodepool node17:36
derekhpleia2: kinda17:36
pleia2where the image is created, we're just not creating the configure-vm stuff because that's already done17:37
derekhpleia2: yup, you right, we need to run it on the nodepool node to create the image17:37
derekhpleia2: and on the testenv host to do everything except create the image17:37
pleia2derekh: right17:37
derekhpleia2: the dirty hack in the etherpad create a noop disk-image-builder so that boot-seed-vm does its stuff on the test env but doesn't create the image17:38
pleia2derekh: gotcha, so we need a couple patches in boot-seed-vm so we can support both ends of creation17:39
pleia2I'll work on that today too17:39
derekhpleia2: yup, If its going to be a mess to decouple we don't really need a patch for the nodepool end, we could just use boot-seed-vm with -n so it doesn't start the VM, it doesn't really matter if it does things like configure-vm as we'll be throwing this VM away after the test17:41
*** jistr has quit IRC17:41
derekhpleia2: but I'm sure people with say that ugly, so we should try and do both ends17:42
pleia2I'll see how painful it is :)17:42
*** jcooley_ has quit IRC17:47
*** jcooley_ has joined #tripleo17:48
*** panda has quit IRC17:49
*** jcooley_ has quit IRC17:52
*** jcooley_ has joined #tripleo17:56
*** matty_dubs|lunch is now known as matty_dubs17:56
* Ng bows out for some feeding17:58
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable some services after installing daemon's...  https://review.openstack.org/5929917:59
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Call os-svc-enable in post-configure.d  https://review.openstack.org/5929817:59
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable services with os-svc-enable  https://review.openstack.org/5929717:59
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Don't enable services in os-svc-install  https://review.openstack.org/5930017:59
dprinceSpamapS: okay. I messed with upstart a bit... and I think I'm confused a bit. Clearly you are more the expert here so what I did in my patch series is make os-svc-enable a noop for upstart for now.18:00
dprinceSpamapS: So the existing patch series affect systemd only, which I think eliminates the concerns you guys have. Perhaps you can append your implementation at the end and we'll be done with this first step?18:00
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable some services after installing daemon's...  https://review.openstack.org/5929918:03
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Call os-svc-enable in post-configure.d  https://review.openstack.org/5929818:03
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Enable services with os-svc-enable  https://review.openstack.org/5929718:03
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Don't enable services in os-svc-install  https://review.openstack.org/5930018:03
*** akuznetsov has joined #tripleo18:04
*** cd-undercloud has joined #tripleo18:06
cd-undercloud************** overcloud complete status=0 ************18:06
*** cd-undercloud has quit IRC18:06
*** vkozhukalov has joined #tripleo18:08
*** derekh has quit IRC18:10
*** csd_ has joined #tripleo18:18
SpamapSdprince: will review shortly18:18
*** csd_ is now known as csd18:18
*** shardy is now known as shardy_afk18:19
*** csd has quit IRC18:26
lifelessmorning18:32
lifelessoh look, success18:32
*** rbrady has left #tripleo18:35
*** rbrady has joined #tripleo18:36
*** athomas has quit IRC18:38
*** csd has joined #tripleo18:38
*** krotscheck has joined #tripleo18:39
*** sdake has quit IRC18:44
*** sdake has joined #tripleo18:45
*** john-n-seattle has quit IRC18:45
*** john-n-seattle has joined #tripleo18:49
*** lucasagomes has quit IRC18:51
*** cd-undercloud has joined #tripleo18:58
cd-undercloud************** overcloud complete status=0 ************18:58
*** cd-undercloud has quit IRC18:58
lifelesstwice in a row18:59
lifelessO18:59
lifelessM18:59
lifelessG18:59
* SpamapS self ^5's19:01
SpamapSwhich I think is just called a clap19:01
lifelessthats a very very special clap19:03
SpamapSlifeless: where do you think that element tester thing should live?19:06
lifelessSpamapS: well so far it's testing oac templates.19:06
SpamapSlifeless: I feel like embedded metadata in README.md plus os-apply-config is specific to tripleo-image-elements19:06
*** jrist has joined #tripleo19:07
SpamapSlifeless: my second order of that btw is to allow element authors to save the output as .expected and then we test the result.19:07
lifelessSpamapS: so, tie is a leaf node in our code graph19:07
lifelessSpamapS: I think it should stay that way19:07
SpamapSagreed19:07
*** sdake_ has joined #tripleo19:07
lifelessSpamapS: I'd be happy with it in oac for now.19:07
SpamapSso maybe a tripleo-dev-tools ?19:07
SpamapSoh19:07
SpamapShmmm19:07
lifelessSpamapS: it's not a perfect fit19:07
SpamapSIt's an o-k fit19:08
*** derekh has joined #tripleo19:08
lifelessSpamapS: but I think it's less bad than in tie19:08
*** noslzzp has quit IRC19:08
SpamapSlifeless: truth be told there is nothing element specific about the code other than README.md and the directory named os-apply-config .. so yeah we can just separate those two bits19:08
SpamapSand yeah, I totally hated writing that without tests19:09
SpamapSputting it in oac means it can have real tests :)19:09
*** noslzzp has joined #tripleo19:09
*** rushiagr has quit IRC19:13
*** noslzzp has quit IRC19:14
*** jtomasek has joined #tripleo19:18
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/os-apply-config: Log parsing errors from pystache as errors  https://review.openstack.org/6004719:23
*** derekh has quit IRC19:25
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add an ntp element  https://review.openstack.org/6007919:26
*** rpodolyaka1 has joined #tripleo19:31
*** jrist has quit IRC19:32
*** bauzas has joined #tripleo19:36
*** jtomasek has quit IRC19:43
SpamapSdprince: I'm trying to find where systemd writes the symlinks from the enable/disable command. Help?19:44
*** jcooley_ has quit IRC19:44
SpamapSlooks like the pkg-config variable 'systemdsystemconfdir' has it19:46
*** jcoufal has joined #tripleo19:46
dprinceSpamapS: /etc/systemd/system/multi-user.target.wants19:46
dprinceSpamapS: so presumably that will eventually be on /mnt/state as well... once we start actually using the use-ephemeral element?19:48
dprinceSpamapS: But for now... (in my existing service enable series) I think we can leave it alone19:48
openstackgerritA change was merged to openstack/tripleo-image-elements: Add an ntp element  https://review.openstack.org/6007919:49
SpamapSRight. THat will likely have to be one of those symlinked to /mnt things.19:49
SpamapSdprince: I think I'll do my upstart implementation as a standalone utility in os-svc-install and then you can just rebase off that commit. I'd rather not merge the upstart noop if we don't have to. Starting on it now.19:51
*** cd-undercloud has joined #tripleo19:51
cd-undercloud************** overcloud complete status=0 ************19:51
*** cd-undercloud has quit IRC19:51
dprinceSpamapS: Fine. In fairness to myself I did implement a mostly working version for upstart in its current form. And this could have come later. But I'm happy to rebase. So long as we move forwards and don't overly intertwine service enable and read only features that have been talked about here. They are related... yes but not having Mega commits makes reviews easier IMO.19:57
SpamapSdprince: we're very close to the readonly and ephemeral disk separation, it is not just pie in the sky.19:58
SpamapSdprince: and mega commits are definitely _bad_19:59
dprinceSpamapS: Sure. I understand that now. I just think things in this service enable series have been held up for no reason. It isn't like it would have broken anything....19:59
SpamapSdprince: right, I think it is more like "let's not make state/software separation harder than it already is going to be.20:00
*** jcooley_ has joined #tripleo20:01
SpamapSdprince: really.. just about 20 minutes from testing the upstart solution myself. :)20:01
dprinceSpamapS: my take is... its all in /etc so lets just do it at the same time ;)20:01
SpamapSdprince: and the opposing take is "let's not create more churn for what we're planning to do basically tomorrow"20:07
*** julim has quit IRC20:16
lifelessand another success, cool.20:17
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add a service enable/disable mechanism for upstart  https://review.openstack.org/6034120:21
SpamapSdprince: ^^20:21
SpamapSuntested yet20:21
SpamapSbuilding an image to test it with now20:21
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add a service enable/disable mechanism for upstart  https://review.openstack.org/6034120:26
* SpamapS cleaned up upstart job env statement20:26
rpodolyaka1lifeless: hey!20:29
rpodolyaka1lifeless: so this is what I've got so far https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:bp/baremetal-preserve-epehemeral,n,z20:30
dprinceSpamapS: I left some comments, mainly weren't there supposed to be some changes to the actual upstart script in os-svc-daemon too?20:30
SpamapSdprince: there is one change.20:32
dprinceSpamapS: right, where does that TRIPLEO export get used then?20:32
rpodolyaka1lifeless: this should be considered as drafts... have to be heavily refactored (the patch pushing rebuild down the stack is really ugly, we probably will want to add some helpers instead of callbacks). And the last one (bringing in-rebuild for baremetal) is broken due to incorrect db query20:32
SpamapSdprince: 'start on starting TRIPLEO=1' is a matcher. It matches any 'starting' event with "TRIPLEO=1" in its environment.20:33
SpamapSdprince: the instance $JOB makes upstart start one of those for every unique $JOB20:33
SpamapSdprince: and the 'task' makes upstart block the 'starting' event until the script section exits.20:34
dprinceSpamapS: ah, cool. well I suppose that works then. couple other minor things but if this works then great20:34
*** jcooley_ has quit IRC20:35
SpamapSjust spotted a bug in the upstart conf20:35
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add a service enable/disable mechanism for upstart  https://review.openstack.org/6034120:36
*** jcooley_ has joined #tripleo20:41
*** dprince has quit IRC20:41
*** cd-undercloud has joined #tripleo20:45
cd-undercloud************** overcloud complete status=0 ************20:45
*** cd-undercloud has quit IRC20:45
openstackgerritA change was merged to openstack/os-apply-config: Log parsing errors from pystache as errors  https://review.openstack.org/6004720:45
*** cwolferh has quit IRC20:47
*** jprovazn has quit IRC20:50
rbradyslagle: I had to add "export LIBVIRT_DEFAULT_URI=${LIBVIRT_DEFAULT_URI:-"qemu:///system"}" to step 6 of https://gist.github.com/slagle/981b279299e91ca91bd9 to get undercloud running.  So far overcloud fails.20:50
*** julim has joined #tripleo20:56
rpodolyaka1night all20:58
*** rpodolyaka1 has left #tripleo20:58
*** noslzzp has joined #tripleo20:59
*** jcooley_ has quit IRC20:59
lifelessand is that another success I see?21:03
*** cd-undercloud has joined #tripleo21:37
cd-undercloud************** overcloud complete status=0 ************21:37
*** cd-undercloud has quit IRC21:37
SpamapSlifeless: and another21:37
SpamapSalso it isn't slower anymore21:37
SpamapS'bout the same. I wonder if we just needed to get everything cached21:38
*** sdake has quit IRC21:40
*** sdake has joined #tripleo21:40
*** sdake has quit IRC21:40
*** sdake has joined #tripleo21:40
lifelessSpamapS: or unfuck the network21:41
lifelessexcuse the french21:41
SpamapScould very well be21:48
SpamapSlifeless: any movement on that JIRA ticket? I haven't yet made the loop back around to try it out yet.21:48
lifelessSpamapS: I haven't checked, I'm not subscribed to it21:50
SpamapShm... building an image with diskimage-builder and the vm element didn't result in an image that just boots in kvm.21:52
SpamapSn/m21:54
SpamapSdoh no stackuser21:54
*** epim has joined #tripleo21:54
*** jcooley_ has joined #tripleo22:12
*** jdob has quit IRC22:14
*** jcoufal has quit IRC22:17
*** ccrouch has quit IRC22:17
*** matty_dubs is now known as matty_dubs|gone22:29
*** cd-undercloud has joined #tripleo22:30
cd-undercloud************** overcloud complete status=42 ************22:30
*** cd-undercloud has quit IRC22:30
*** noslzzp has quit IRC22:41
*** openstackgerrit has quit IRC22:48
*** openstackgerrit has joined #tripleo22:48
*** epim has quit IRC22:51
*** epim has joined #tripleo22:58
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Need libssl-dev to install glance standalone  https://review.openstack.org/6037422:58
SpamapSnooooo not 4222:59
* SpamapS shakes fist at Neutron22:59
SpamapS\o/ tripleo-image-elements and tripleo-heat-templates should not comment on tripleo bugs. :) https://review.openstack.org/6027823:09
lifelessnot?23:13
*** jcooley_ has quit IRC23:14
*** jcoufal-mob has joined #tripleo23:16
*** bauzas has quit IRC23:17
*** jcooley_ has joined #tripleo23:23
*** jcoufal-mob has quit IRC23:24
*** cd-undercloud has joined #tripleo23:25
cd-undercloud************** overcloud complete status=0 ************23:25
*** cd-undercloud has quit IRC23:25
*** vkozhukalov has quit IRC23:26
SpamapSlifeless: s/not/now/23:26
*** jcooley_ has quit IRC23:42
*** jcooley_ has joined #tripleo23:43
*** openstackgerrit has quit IRC23:47
*** openstackgerrit has joined #tripleo23:47
lifelessok, next deploy that succeeds I'm going to halt the service to validate some of the toci assumptions23:50
lifelessspecifically about vms and provider networks23:50
lifelessif anyone wants to collaborate on that (e.g. dan, who is offline :P, or pleia2 ) - just shout23:51
pleia2lifeless: I'm around23:53
lifelesspleia2: cool23:54
lifelessso it will be ~30m before it hits23:54
lifelessonce it does I plan to stop tripleo-cd23:54
*** jcooley_ has quit IRC23:55
lifelessthen review the deployed neutron setup, add a dedicated test tenant, and a provider network it has access too23:55
lifelessand then see if we can do the funky chicken with an instance spawned on that network23:55
pleia2great23:55
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-image-elements: Add a service enable/disable mechanism for upstart  https://review.openstack.org/6034123:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!