Thursday, 2014-08-14

lifelesspossibly even just nova show will let us diagnose this00:00
lifelesscinerama: what does 'nova show' output for theinstance?00:00
cineramalifeless: any field in particular or shall i dump it into a pastebin?00:01
lifelesspastebin00:01
lifelesstchaypo: ok, so next step is to deploy an undercloud00:02
lifelesstchaypo: I haven't checked ng's patches, but you were saying it was missing the undercloud metadata?00:02
tchayponeutron subnet-list shows me the subnets I expected to see based on the spreadsheet...00:02
tchaypoexcept that the allocation_pool for the public subnet runs from .4 to .4 which is a little smaller than I expected00:02
tchaypoalthoguh thinking about it, we only plan to expose one public VIP, right? so that's probably fine00:03
lifelesstchaypo: look at the equivalent on hp100:03
lifeless| bfee35b1-2641-4599-b68c-6f799ed83b24 |      | 138.35.77.0/25  | {"start": "138.35.77.4", "end": "138.35.77.4"}   |00:03
lifelessso yeah I think fine00:04
tchaypoyeah, https://review.openstack.org/#/c/109653/4/elements/tripleo-cd/configs/hp2_ci_env.json - missing values for NeutronPublicInterface and HypervisorNeutronPublicInterface00:04
tchaypoi think it should be "int_public" for both?00:05
lifelesstchaypo: why do you think that?00:06
cineramalifeless: http://paste.ubuntu.com/8040620/00:07
lifelesscinerama: so the flavour is interesting, its not one our tooling makes :)00:08
tchaypohang on, let me compare with hp100:08
lifelesscinerama: and the name | name                                 | demo                                                                                                                                |00:08
tchaypolifeless: maybe it's an ansiblized flavour?00:08
lifelesscinerama: makes me think this is a libvirt VM instance, not an Ironic or Nova-BM instance.00:08
lifelesstchaypo: if the ansible experiment is making new flavours, they're off in the weeds IMO :)00:08
cineramalifeless: :) indeed - i created a flavour that allowed ephemeral storage and nova booted a copy of the user image we build in devtest_overcloud00:09
tchaypookay, I'm wrong.00:09
tchaypoand.. confused now. the hp1_ci_env.json lists eth2 for those parameters, but i don't see an eth2 on the bastion or on the seed00:10
lifelesscinerama: libvirt hasn't had --preserve-ephemeral implemented for it yet.00:10
lifelesscinerama: 'someone' needs to do that00:10
mordredlifeless: phil day said he's going to look in to it00:11
*** nosnos has joined #tripleo00:11
lifelessmordred: I know00:11
lifelessmordred: in his copious spare time00:11
lifelesscinerama: I haven't looked to see if there is a pending patch up yet00:12
cineramalifeless: so we're using libvirt by default?00:13
*** gfidente has quit IRC00:15
lifelesscinerama: we deploy three clouds00:15
lifelesscinerama: seed, undercloud aka deployer, overcloud aka workload00:15
cineramalifeless: i actually knew that! :)00:15
lifelesscinerama: today, the seed has only one hypervisor in user - nova-bm or ironic00:15
lifelesscinerama: ditto the deployer cloud00:15
lifelesscinerama: the workload cloud also has only one, and thats libvirt with either kvm or qemu as its libvirt instance type00:16
lifelessverbosity for hopefully increased clarity00:16
cineramalifeless: ah, i think i see00:16
lifelesss/user/use/ above00:17
lifelesstchaypo: so, eth200:17
lifelesstchaypo: is referring to the ethernet device in use within the machines that are deployed00:17
lifelesstchaypo: the seed is a vm and not relevant to these parameters00:18
tchaypothe baremetal the vms run on?00:18
lifelesstchaypo: PARSE ERROR00:18
lifelesstchaypo: the bastion has been installed with biosdevname for whatever reason00:18
tchaypoactually, speaking of parse errors, i need to step back to an earlier point00:19
lifelesstchaypo: all our code to date has been homogeneous w.r.t. network configuration. We may find bringing up hp2 to involve nontrivial feature work. At least, I hope it does :)00:19
tchaypowe were talking about o-r-c calling _testenv.sh00:20
tchaypoand how the o-r-c configs get dropped by d-i-b as it's building the image00:20
*** jml has quit IRC00:20
tchaypomy question is - what image is it building?00:20
lifelesstchaypo: the testenv image00:20
tchaypoI think I'm missing something very basic here00:20
tchaypowhat is the testenv image used for?00:21
tchaypoIf it's running _testenv.sh to create the testenv, but it can't come up till the testenv exists, don't we have a chicken/egg issue?00:21
tchaypoi think you can see why i think there's something i'm missing00:22
lifelesshave you read the tripleo-ci docs?00:22
lifelessif not, do so - later, while I'm at gym with Cynthia00:22
*** jml has joined #tripleo00:22
*** yamahata has joined #tripleo00:23
tchaypoi thought i had. it sounds like a refresh is in order00:23
lifelessbut to answer00:24
lifelessthe testenv image is the machine image we run to deploy testenvs in the region00:24
lifelessa testenv machine hosts some N testenvs00:24
*** eguz has joined #tripleo00:24
*** eguz has quit IRC00:24
tchaypotestenv machine == baremetal node?00:25
lifelesstchaypo: they are yes00:26
lifelessbecause they host VMs00:26
tchaypoyep, i see my error, i think.00:27
*** eghobo has quit IRC00:28
lifelessaka testenv worker00:29
tchaypoI had a diagram in my head in which the bastions ran a seed vm that controlled many under/overcloud pairs running on the bm00:29
tchaypoI don't know where i got that from00:29
lifelessthats an interesting diagram00:29
lifelessbut no :)00:29
lifelessbastion hosts seed; seed deploys undercloud, undercloud deploys testenvs, undercloud deploys ci-overcloud00:30
*** penick has quit IRC00:30
lifelesstchaypo: ok so back to the hp2 undercloud00:32
lifelesstchaypo: I don't know the best way for discovery-learning here00:33
lifelesstchaypo: so I'm just going to say it ;)00:33
lifelesstchaypo: you need to find out what the in-machine layout of ethernet cards is00:33
lifelesstchaypo: I usually do this by deploying an image which will fail, but then logging on in the console with stack:stack00:33
* lifeless looks around for cores00:35
lifelessbnemec: slagle: greghaynes: SpamapS: https://review.openstack.org/#/c/110492/ [we are running this already, things would be stuffed if we weren't]00:35
openstackgerritA change was merged to openstack-infra/tripleo-ci: Explicitly set NODE_ARCH to i386  https://review.openstack.org/10634200:37
openstackgerritA change was merged to openstack-infra/tripleo-ci: Allow new virsh commands from ironic  https://review.openstack.org/11049200:37
lifelesshttps://review.openstack.org/#/c/108997/00:38
lifelessalso needs such a review00:38
lifelessthe failures are before we renamed things - its actually passed CI00:38
lifelessbnemec: slagle: greghaynes: SpamapS: ^00:38
tchaypolifeless: yep, so what i was just doing matches that, i think00:40
tchaypoon hp1 testenv.json has a seed-ip, so i was looking at that seed00:40
tchaypoand then it has a single undercloud node so i was looking at that00:41
tchaypoand it has an eth200:41
*** rwsu has quit IRC00:42
tchaypo.. hang on, i may have just understood something else00:42
*** cwolferh has quit IRC00:43
openstackgerritA change was merged to openstack-infra/tripleo-ci: Use baremetal_full to get the entire disk  https://review.openstack.org/10899700:43
tchaypoin my mind i've been conflating the ci and cd clouds00:43
bnemeclifeless: I'm going to have to take your and Derek's word for it that baremetal_full is a thing that exists. :-)00:44
tchaypobut if i've just understood correctly, the cd overcloud hands out instances to jenkins which then deploys a full seed/undercloud/overcloud stack inside the instance to test out a change00:44
lifelessbnemec: it does in hp100:46
lifelessbnemec: I *thought* we added automation to make it00:46
bnemeclifeless: I didn't find anything in tripleo-ci or my local repos.00:46
*** nosnos has quit IRC00:46
bnemecMaybe that change is in review limbo too. :-)00:46
lifelessbnemec: could be00:46
tchaypoI'm getting ready for a driving lesson in a few minutes00:46
lifelessbnemec: we'll deal if it is00:46
lifelesstchaypo: there is no cd overcloud today; its in hiatus.00:47
lifelesstchaypo: the ci-overcloud hands out a VM to nodepool00:47
lifelesstchaypo: the test cloud we deploy is deployed in a testenv [hosted on a testenv worker as discussed before]00:47
lifelesstchaypo: the VM that was handed out to nodepool is the jenkins slave for the test run and is responsible for running devtest scripts, building images etc, but it doesn't host vms00:48
tchaypoi think you mean - there's no cd overcloud in hp1, which is why it's out of action?00:49
tchaypooh wait00:50
tchayporight. so we have preconfigured testenvs deployed on the testenv workers. ci-overcloud hands out a vm to be a jenkins slave, that slave runs the devtest scripts pointed at the preconfigured testenv00:51
*** Yanyanhu has joined #tripleo00:52
tchayposo the o-r-c thing that calls _testenv.sh would be run at the point when we're pre-building the testenvs; it's not run once per devtest run00:52
lifelessright00:53
*** rwsu has joined #tripleo00:55
*** rwsu has quit IRC00:57
*** rwsu has joined #tripleo00:57
tchaypodriving instructor calls. I'll re-re-read the docs again, then make a guess at some values and go ahed with an undercloud deploy to see where it fails00:58
*** saurabhs has quit IRC00:58
*** jang1 has joined #tripleo01:02
*** eghobo has joined #tripleo01:02
*** eghobo has quit IRC01:07
SpamapS 4374 root      20   0  435652 380400  33776 R  98.1  9.2 199:18.33 corosync01:19
*** pradeep has joined #tripleo01:19
SpamapSwtf is it doing?01:19
*** pradeep has quit IRC01:23
*** nosnos has joined #tripleo01:38
*** cody-somerville has joined #tripleo01:55
*** cody-somerville has quit IRC01:55
*** cody-somerville has joined #tripleo01:55
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Remove hard-coded ports form ring builder script  https://review.openstack.org/11245502:00
jogocinerama: I was AFK for a bit but I'll be around tomorrow02:09
*** rushiagr_away is now known as rushiagr02:11
*** noslzzp has joined #tripleo02:17
*** tserong has quit IRC02:18
*** ramishra has joined #tripleo02:20
*** tserong has joined #tripleo02:23
*** tserong has joined #tripleo02:23
*** cwolferh has joined #tripleo02:29
*** rushiagr is now known as rushiagr_away02:30
lifelesstchaypo: am back02:33
SpamapSis there some reason I'd have super bursty SSH to any of the vms on a devtest box?02:34
SpamapSwondering if there's some routing insanity02:34
lifelessSpamapS: no, it should be reliable and robust02:34
lifelessSpamapS: the user vms will be qemu of course02:35
*** jdonalds has joined #tripleo02:35
lifelessSpamapS: if you mean the cloud nodes they are routed via the seed, if your seed is overloaded that might cause latency02:35
SpamapSssh'ing into the undercloud from the host machine.. it stutters a lot02:37
SpamapSseed is at 1.37 load w/ 2CPU's and 4G RAM... lots free.02:38
SpamapSbut ssh to the seed shows the same stuttering02:38
SpamapSfeels like buffer bloat02:39
SpamapS(host is an SL390 from hp2 btw)02:40
*** cody-somerville has quit IRC02:41
SpamapSstarting over with x86_64 ... :-P02:43
lifelessSpamapS: interesting :/02:47
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135663703:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126303:00
uvirtbotLaunchpad bug 1356637 in tripleo "os-refresh-config fails in hosts element when update is different" [Undecided,New]03:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604003:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589803:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/134784803:00
uvirtbotLaunchpad bug 1347848 in tripleo "/etc/hosts information about cloud hsots lost on reboot" [Undecided,In progress]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135623303:00
*** ramishra has quit IRC03:00
uvirtbotLaunchpad bug 1356233 in tripleo "Error in deploying a new node" [Undecided,New]03:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/135633903:00
uvirtbotLaunchpad bug 1356339 in diskimage-builder "Add support for vmedia booting in deploy-ironic element" [Undecided,New]03:00
*** untriaged-bot has quit IRC03:00
*** ramishra has joined #tripleo03:01
*** marun has quit IRC03:04
*** Ng has quit IRC03:05
*** Ng has joined #tripleo03:05
*** ChanServ sets mode: +v Ng03:05
*** dsneddon has quit IRC03:12
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Add set -o pipefail to scripts  https://review.openstack.org/10929803:21
*** rbrady has quit IRC03:24
*** nosnos has quit IRC03:32
*** mkerrin has quit IRC03:43
*** noslzzp has quit IRC03:47
*** nealph_ has joined #tripleo03:49
*** wendar_ has joined #tripleo03:49
*** mkerrin has joined #tripleo03:50
*** cwolferh has quit IRC03:51
*** jdonalds has quit IRC03:51
*** mikal_ has joined #tripleo03:51
*** jomara_ has joined #tripleo03:51
*** Kiall has quit IRC03:52
*** cinerama has quit IRC03:52
*** jomara has quit IRC03:52
*** mikal has quit IRC03:52
*** mordred has quit IRC03:52
*** sdake has quit IRC03:52
*** jroll has quit IRC03:52
*** wendar has quit IRC03:52
*** dguerri`afk has quit IRC03:52
*** nealph has quit IRC03:52
*** dguerri has joined #tripleo03:52
*** jdonalds has joined #tripleo03:53
*** uberj has quit IRC03:53
*** mordred has joined #tripleo03:53
*** uberj_ has joined #tripleo03:53
*** cinerama has joined #tripleo03:54
*** Kiall has joined #tripleo03:54
*** jroll has joined #tripleo03:54
*** pcrews has quit IRC03:54
*** sdake has joined #tripleo03:54
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/os-collect-config: Allow using non-default collectors  https://review.openstack.org/11411603:54
*** mkerrin has quit IRC03:55
*** jdonalds has quit IRC03:59
*** nosnos has joined #tripleo04:01
*** pradeep has joined #tripleo04:01
*** blkperl has quit IRC04:02
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/os-collect-config: Allow using non-default collectors  https://review.openstack.org/11411604:05
*** dguerri_ has joined #tripleo04:06
*** dguerri has quit IRC04:07
lifelesstchaypo: ok so let me know when you return04:18
lifelessSpamapS: did you sort thy issue out?04:19
SpamapSlifeless: rebuilding from scratch w/ amd6404:20
tchaypolifeless: oddly i had just sat down as your message came through04:21
SpamapSlifeless: we _really_ need to get Ironic to retry on errors. One unknown PXE hiccup leads to a lot of operational pain. :-P04:22
* SpamapS drinks second cup of coffee at 21:23... one of those nights04:23
SpamapSoh wow, yeah HUGE difference w/ these new nodes04:25
SpamapSsomething was very wrong with that devtest04:25
SpamapSdunno if it was RAM starvation or amd64 vs. i386 .. but this one is positively zippy04:27
greghaynes:/04:27
greghaynesthats scary04:27
SpamapSI think it is possible it was the corosync stuff that got reverted to04:28
SpamapStoo04:28
clarkbdid you remove firewalld?04:28
SpamapSclarkb: I did nothing explicit04:29
clarkbapparently that makes tempest run at least 3x faster04:29
SpamapSexcept double RAM, and go from i386 -> amd6404:29
SpamapSno I think the boxes were either slammed with CPU or there was some kind of networking hiccup04:29
clarkbits a libvirt bug talking to firewalld apparently and probably not related to your thing04:29
SpamapSI forgot to check and see if maybe os-collect-config was spinning endlessly reconfiguring my bridges04:30
SpamapSthat might have done it.04:30
lifelessclarkb: firewalld?04:32
lifelessclarkb: tempest?04:32
lifelessclarkb: I think you're talking about something utterly different04:32
clarkbI know04:33
clarkbI just cant get over it04:33
lifelesscan't get enough can't get enough no?04:33
clarkbits one of those bugs that never would have been sorted without danpb pulling out libvirt knowledge04:33
clarkbor at least would have eaten a ton of time04:34
lifelesscertainly04:36
lifelesstchaypo: so04:36
lifelesstchaypo: where are we?04:36
tchaypothere's an undercloud a-building04:39
lifelessSpamapS: how have you reserved your node in hp2 ?04:39
*** bmahalakshmi has joined #tripleo04:40
tchayposaid undercloud is a-building with just a standard invocation of "devtest_underclould.sh $TE_DATAFILE"04:41
tchaypoi'm about 80% sure it's going to fail04:41
SpamapSlifeless: a little english bird slipped me an IP or two.04:44
tchaypoSpamapS: you confused me there04:45
tchaypoI was thinking "Why just IP? Why not IPA? Is the "A" just assumed now? Was it something else, maybe an india pale lager? What would that even mean?"04:45
SpamapSpay no attention to the two nodes we stole.. er.. appropriated...04:45
tchaypobut apparently you aren't talking about beer04:45
SpamapStchaypo: and clearly you are. :)04:47
*** dshulyak has joined #tripleo04:50
tchaypohuh. that's not an error i was expecting04:53
tchaypoError: OS credentials not found. Please save them to ~/stackrc.04:54
lifelessSpamapS: letme rephrase04:54
lifelessSpamapS: we're redeploying it at the moment04:54
tchaypoand it's all well and good to tell me to ignore them04:54
lifelessSpamapS: I don't want to overwrite what you're doing04:54
tchaypobut from the information you've provided I'm not going to recognise them if I see them, and I don't know if the scripts will either04:54
lifelessSpamapS: but if its not in the toolchain from wiki page data dump through to nodes.json, we will04:54
SpamapSlifeless: I've been told they were removed from machine-information04:55
SpamapSCan give macs04:55
*** pradipta_away is now known as pradipta04:55
lifelessSpamapS: I see them, you should be fine04:55
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Remove hard-coded ports form ring builder script  https://review.openstack.org/11245504:56
SpamapSlifeless: ah thanks :)04:56
*** mikal_ is now known as mikal04:58
*** sseago_ has joined #tripleo04:58
tchaypooh, the two lines starting with "cmsjreserved"?05:00
tchaypothose lines don't parse when we run the tools05:00
*** jdonalds has joined #tripleo05:05
*** jdonalds has quit IRC05:07
*** mkerrin has joined #tripleo05:08
*** nijaba_ has joined #tripleo05:08
*** jdonalds has joined #tripleo05:10
*** Hazelesque has quit IRC05:11
*** nijaba has quit IRC05:11
*** Hazelesque has joined #tripleo05:11
*** Kiall has quit IRC05:12
*** Kiall has joined #tripleo05:12
*** mkerrin has quit IRC05:13
*** rushiagr_away is now known as rushiagr05:19
*** pensu has joined #tripleo05:22
*** rdopieralski has joined #tripleo05:25
openstackgerritRamakrishnan G proposed a change to openstack/diskimage-builder: Add vmedia boot support for deploy-ironic element  https://review.openstack.org/11412405:26
*** rushiagr is now known as rushiagr_away05:31
*** rushiagr_away is now known as rushiagr05:31
*** pradeep has quit IRC05:35
*** pradeep has joined #tripleo05:36
*** rameshg87 has joined #tripleo05:37
tchaypoho hum. Why do I not have OS_AUTH_URL etc in my environment or in a file?05:42
*** dshulyak has quit IRC05:44
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394605:45
*** dshulyak has joined #tripleo05:46
tchaypobecause i didn't source the seedrc from inside the incubator repo that has those things in05:49
*** mkerrin has joined #tripleo05:50
lifelesstchaypo: they start with #05:51
lifelesstchaypo: is I think the key thing05:51
tchaypothe script i was looking at uses awk (or something) to pull out tab-seperated fields, those lines have an extra field so it gets nonsense instead of an ip/mac/etc05:52
*** SpamapS has quit IRC05:53
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394605:56
openstackgerritRamakrishnan G proposed a change to openstack/diskimage-builder: Add vmedia boot support for deploy-ironic element  https://review.openstack.org/11412405:59
tchaypoI hate being half-sick. I feel mostly fine at the moment until I turn my head and then I come over all dizzy... and then I want to collapse asleep. I wish I could either be properly sick or not sick at all.06:01
*** mkerrin has quit IRC06:02
*** rameshg87 has quit IRC06:03
*** pradeep1 has joined #tripleo06:08
*** pradeep has quit IRC06:09
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394606:10
*** mkerrin has joined #tripleo06:11
lifelesstchaypo: it uses python06:13
lifelesstchaypo:     if line.strip() == '' or line[0] in '#-' or line.strip().split()[0]=='id':06:13
*** ramishra has quit IRC06:14
*** ramishra has joined #tripleo06:14
lifelessmkerrin: hi06:15
tchaypoClose to awk ;)06:18
*** dshulyak has quit IRC06:20
tchaypoI know what I did wrong. I forget to plug the holes on ng's template. Oh well, my guess would probably  have been wrong anyway.06:20
lifelesstchaypo: well you need the machine running to inspect06:23
lifelesssince our tooling doesn't just DTRT06:23
lifeless(which you may remember victor_lowther proposing atthe midcycle)06:23
tchaypowell yess06:24
*** loki184 has joined #tripleo06:26
* tchaypo waits some more06:26
tchaypoI'm not sure that I do remember the bit you're thinking of06:26
*** pradipta is now known as pradipta_away06:28
*** k4n0 has joined #tripleo06:30
lsmolalifeless: ping06:35
*** ramishra has quit IRC06:40
*** jprovazn has joined #tripleo06:42
*** SpamapS has joined #tripleo06:47
*** SpamapS has quit IRC06:47
*** SpamapS has joined #tripleo06:47
*** jtomasek_ has joined #tripleo07:01
*** jdonalds has quit IRC07:05
*** dtantsur|afk is now known as dtantsur07:06
*** jdonalds has joined #tripleo07:08
*** jdonalds has joined #tripleo07:08
*** jdonalds has quit IRC07:10
mkerrinlifeless: hi, just got in07:12
*** mkerrin has quit IRC07:22
tchaypong: hp2-gumpf?07:28
*** jcoufal has joined #tripleo07:28
lifelesslsmola: hi07:30
*** mkerrin has joined #tripleo07:30
lsmolalifeless: hola07:30
lsmolalifeless: I did responded to the blueprint07:30
lsmolalifeless: I think only issue I see is in the sharded deployment, which I have no info about07:31
lsmolalifeless: though hopefully making the init inside template optional + everything backwards compatible should address all your issues I assume07:32
lifelesstchaypo: e.g. we say 'eth2' or 'em1' rather than 'first NIC' or 'first 10G NIC'07:32
*** gfidente has joined #tripleo07:32
lifelesslsmola: I don't understand why we need this at all07:33
lifelesslsmola: the UI is backed by the API07:33
lsmolalifeless: the UI can't access any passwords now07:34
lifelesslsmola: and my understanding of password management stuff was that we were going to be *no worse* than the current code.07:34
lifelesslsmola: so that sounds like the API has regressed07:34
lsmolalifeless: yes, the is no barbican support anymore07:34
lifelesslsmola: I mean the incubator code07:34
tchaypolifeless: a, "first NIC" is what you mean by "the right thing07:34
tchaypo"07:34
lsmolalifeless: so why is this worse? it just takes the shellscript from the incubator and executes it on a controller machine07:36
openstackgerritJames Polley proposed a change to openstack/tripleo-image-elements: Add hp2 region.  https://review.openstack.org/10965307:36
lifelesslsmola: so we *used* to do this07:36
lifelesslsmola: and it was a PITA07:36
lsmolalifeless: all as part of the Heat deployment, which is great for the UI, we deploy cloud and we get working cloud as output07:36
lifelesslsmola: it took use ages to undo it, I really don't want us to see it brought back in07:37
lifelesslsmola: clearly if we have to have it we'll have to have it07:37
lsmolalifeless: :-) well we would like to have it07:37
lsmolalifeless: as long it is optional, you can easily delete it in one patch07:38
lsmolalifeless: :-)07:38
lsmolalifeless: I believe the stuff from past was more complex because of the pki setup07:38
lifelesslsmola: its not optional if tuskar depends on it07:38
lsmolalifeless: right now it's really just few API call, that can be called from anywhere07:38
*** sballe has joined #tripleo07:39
lsmolalifeless: well it's more UI think, we can't access the passwords as the CLI version07:39
lsmolalifeless: well it would be optional just for UI07:39
lifelesslsmola: but thats why the API has a data store07:40
lsmolalifeless: I mean you can disable it from UI and init the cloud yourself07:40
lsmolalifeless: yeah, but store for secrets is likely to come in K07:40
lifelesslsmola: that means that the UI now has to have an option to disable it07:40
lsmolalifeless: we need something that will work now07:41
lifelesslsmola: rather than the API running os-cloud-config stuff07:41
lifelesslsmola: so tell me why storing these parameters in the same store as our other parameters won't work07:41
lsmolalifeless: well I would just make it a HEat parameter InitKEystone, that would invoke another software config07:41
lsmolalifeless: for the UI?07:42
lsmolalifeless: tuskar API is now storing the parameters without Passwords AFAIK07:42
lifelesslsmola: so why isn't is storing the password parameters?07:43
lsmolalifeless: so do you think it would be cleaner to rather store the passwords there in a raw form?07:43
lsmolalifeless: because it's not suppose to be secret store07:43
lifelesslsmola: thats interesting07:43
lifelesslsmola: but really, *why*07:43
lifelesslsmola: there are lots of issues with secret management07:44
lsmolalifeless: we should have barbican for that07:44
lifelesslsmola: forget should for a second07:44
lifelesslsmola: lets actually analyse holistically07:44
lifelesslsmola: *everything* is in heat today07:44
*** jistr has joined #tripleo07:44
lsmolalifeless: well adding another issue with secret management doesn't seems right07:45
lifelesslsmola: any store that is less secure than that can be argued to be a problem07:45
lifelesslsmola: but any store that is more secure doesn't actually help anything07:45
lsmolalifeless: well yeah, but heat is about to fix their secret issues07:46
lsmolalifeless: so it's storing passwords in Tuskar-API vs Init inside Heat07:47
openstackgerritA change was merged to openstack/os-cloud-config: Updated from global requirements  https://review.openstack.org/10658907:47
lifelesslsmola: so once they do there's no issue for tuskar, right?07:47
lsmolalifeless: I guess, when we have secret store from where we can get the passwords, it should be solved07:48
lsmolalifeless: not sure if Heat is actually planning to give us the passwords07:48
lifelesslsmola: well its our parameter, so I'd hope so ;P07:49
lsmolalifeless: but we can do Barbican on Tuskar-API side, though so far there is no spec for that07:49
d0ugallsmola: There is actually :)07:49
lifelesslsmola: so why are we doing this work now? If the answer is because aesthetically we'd rather not store the passwords that are *already stored by the cloud below* in a service *in the cloud below*07:49
d0ugallsmola: Barbican is covered in the alternatives for the storage spec.07:49
lsmolalifeless: plus we will need also some user creadential from overcloud, so we can get e.g. Overcloud Ceilometer data, etc. so it will be needed07:49
lifelessthen I think thats a fairly substantial distraction from the things that matter07:49
lsmolad0ugal: ok07:50
lsmolalifeless: :-) yeah when you say it that way07:51
d0ugalThe plan is not to store sensitive data within Tuskar, but it isn't actually enforced now. So, we could use it until Heat helps us out more.07:51
lsmolalifeless: I guess it's less work07:52
lifelesswe're working on ssl everywhere by default07:52
lsmolalifeless: I don't like another secret mess, but if everybody will be ok with that07:52
lifelesslsmola: its not a new mess07:52
lsmolalifeless: and it will be fixed in K07:52
lifelesslsmola: we have the issue on the admin console now07:52
lsmolalifeless: ok, that is true :-)07:53
tchaypolifeless: i was looking at repo_refs on hp1 so i could catch hp2 up with changes you've made today, and i noticed a few patches that have merged, so i took them out07:53
lsmolalifeless: so as long as everybody is fine with us storing passwords in raw form in the Tuskar API07:53
lsmolad0ugal: ^07:54
lifelesslsmola: I can't promise noone will be unhappy07:54
d0ugalI wouldn't say I'm "fine" with it, but I think it's tolerable for now :)07:54
lifelesslsmola: but limited time, delivery etc07:54
lsmolalifeless: well I am quite unhappy, though it means less work for me, so that makes it kind of even :-D07:54
d0ugallol07:54
lsmolalifeless: yeah given we have only few weeks till this should be solved somehow07:55
openstackgerritLennart Regebro proposed a change to openstack/tuskar-ui: Fixed various styling issues  https://review.openstack.org/11397007:55
lifelesslsmola: getting in-instance stuff *right* and dealing with upgrade and race conditions and quorum issues is *hard07:55
lsmolalifeless: and given I need you +2 for the spec, I guess it's kind of decided07:55
lsmolalifeless: :-D07:55
lifelesslsmola: thats why we moved this out of the instance in the first place07:55
lifelesslsmola: you don't need me specifically anymore07:56
lifelesslsmola: see meeting minutes from 2 days ago07:56
lsmolalifeless: ah, I dodn't see that :-)07:56
d0ugallsmola: Since you have more time now, can I get help with something for a minute? ;)07:56
lsmolalifeless: ok so now I just need to convert the setup-enpoints for the os-cloud-config one07:56
lifelesswell, its changing in a coupel days, but yeah07:57
lsmolalifeless: should I convert the spec for that, or it doesnt need spec, as it is already half way done?07:57
openstackgerritLennart Regebro proposed a change to openstack/tuskar-ui: Fixed various styling issues  https://review.openstack.org/11397007:58
lifelesslsmola: setup-endpoints?07:58
lsmolalifeless: yes07:58
*** regebro has quit IRC07:58
*** regebro has joined #tripleo07:58
lifelesslsmola: I'm halfway through reviewing your os-cloud-config patches07:58
lifelesslsmola: what thing are you referring to here07:58
lsmolalifeless: yes07:59
lsmolalifeless: we need the os-cloud config, so we are able to call it from the UI07:59
lifelesslsmola: from the API surely07:59
lsmolalifeless: actually it will be called most likely from the UI, tied to some initialize button08:00
lifelesslsmola: the UI is meant to be browser skin only, right?08:00
lifelesslsmola: that doesn't make sense to me.08:00
lsmolalifeless: it's just API calls so..08:00
lifelesslsmola: tuskarclient needs to be functionally equivalent08:00
lifelesslsmola: if its in the UI, we're writing the code twice.08:00
lsmolalifeless: well tuskarclient is just for planning08:00
lifelesslsmola: lets talk about CI08:01
lsmolalifeless: so when done in CLI, you will call os-cloud-config directly08:01
lifelesslsmola: very soon we want to do the deployer and workload cloud deploys we do in CI via tuskar.08:01
lsmolalifeless: like the devtest does it now08:01
lifelesslsmola: which means via tuskarclient and the tuskar API08:01
lsmolalifeless: well actually tuskar is just for planning, so you will get templates form tuskar and you will call heat08:02
lsmolalifeless: the spec of tuskar has grown very small08:02
lifelessso thats a good thing08:02
lsmolalifeless: it no longer proxies another services08:02
tchaypo+108:03
lsmolalifeless: though again, at some point the init code could move somewhere08:04
lifelesslsmola: I don't have any deep objection to calling os-cloud-config CLIs directly08:04
lifelesslsmola: it just surprised me08:04
lsmolalifeless: it's several seconds operation, which is better to be packed in some workflow engine on the API side08:04
lsmolalifeless: actually we will call the python functions, not the CLIs AFAIK08:05
lifelesslsmola: tuskar[ui/api] should call the python functions yes08:05
lsmolalifeless: yeah the original plan was that tuskar will pack it whole in a workflow, which might happen in the future08:06
lsmolalifeless: cause now we will have to have init button in the UI08:06
tchaypolifeless: 113154 has merge conflicts with master, wheee08:07
lifelesslsmola: I don't see why there needs to be a button08:07
lsmolalifeless: well UI doesn't know when the stack is created08:07
lifelesslsmola: when heat goes stack-ready, chain into the other bits08:07
tchaypolifeless: did something change with that? are we still planning to test/use the rollback?08:07
lsmolalifeless: yeah that would mean a waiting loop in UI code, which would be very bad08:07
lsmolalifeless: actually it would be possible only on client side AFAIK08:08
lifelesstchaypo: heat fixed it08:08
lsmolalifeless: proper workflow using taskflow e.g. could be implemented only in API08:09
lifelesstchaypo: we need to make sure we can deploy current image with old stack08:09
regebroAh, I see. The recommended Squid proxy settings will check if the target has updated. And if it fails to connect to the target, squid will give an error.08:09
regebroThat's useless. Damn. Do I need to learn Squid now?08:09
lifelesslsmola: so sure, or perhaps we teach heat how to talk to new apis dynamically08:10
jangnobody should have to learn squid08:10
lsmolalifeless: I think SpamapS has some solution in mind08:10
lsmolalifeless: turning os-cloud-config to daemon08:11
lifelessSpamapS: ^08:11
lifelesslsmola: that surprises me, but possibly08:11
lifelessI know SpamapS is keen on orchestrating the new cloud via heat08:11
lsmolalifeless: ok, we will see in K08:11
lifelesswe just have some engineering to do to make it possible08:11
lsmolalifeless: ok, so I am abandoning the spec, I will finish the setup-endpoints without spec :-)08:13
lsmolalifeless: I will probably also check if guys are ok storing secrets in Tuskar-API, but I think it was mostly me against :-)08:14
tchaypolifeless: so drop that change from the repo_refs?08:14
lifelesslsmola: ok08:14
d0ugallsmola: Would be have time to look at the failure in https://review.openstack.org/#/c/113925/? An old test you wrote seems to have stopped passing. Unrelated to my change.08:15
lifelesstchaypo: its more complex than that08:15
lsmolad0ugal: ok. will do08:15
lifelesstchaypo: since we need to reconcile a number of patches08:15
tchaypolifeless: excellent, i love complex things08:16
lifelesstchaypo: so 113214 isn't needed08:18
lifelesstchaypo: 113154 will go yes08:18
lifelesstchaypo: 112778 will still be needed08:18
lifelesstchaypo: you won't need 113148 but its still needed for hp108:18
lifelesstchaypo: 103454 needs to be toggled to its HOT form08:18
lifelesstchaypo: 108208 likewise08:18
lifelesstchaypo: and 113723 will be needed08:18
lsmolad0ugal: so why do you need that code? I thought Tuskar is not using merge.py anymore08:20
d0ugallsmola: It's used by v1 of the api, we still have that.08:20
d0ugalI personally don't need it :)08:21
lifelesshas tuskar reimplemented merge.py?08:21
lifelesslsmola: d0ugal: we still have things requiring merge.py08:21
lsmolad0ugal: I thought v1 api is not working08:21
lifelesswhich the deprecate-merge-py spec calls out as being needed, from memory08:21
lsmolalifeless: no we are using Hot and building master template08:22
d0ugallifeless: not reimplemented, jdob has been working in that area - I'd need to check with him to get you an accurate answer :)08:22
lsmolalifeless: so yeah, the new tuskar is not using merge.py, or should not be08:22
d0ugallsmola: I'm not sure what the policy is for removing v1 of the API.08:22
lifelesslsmola: see 3e35d53in t-h-t08:22
lifelesslsmola: how is scaling being catered to?08:23
lsmolalifeless: scaling is done using resource group08:23
lsmolalifeless: that wraps the provider resource08:24
*** yamahata has quit IRC08:26
lifelesslsmola: so a dependency in the spec08:26
lifelesslsmola: is the removal of nodes for scale down - specific nodes08:27
lsmolalifeless: yes08:27
lifelesslsmola: 3e35d53 in t-h-t demonstrates doing that with merge.py; has it been solved for heat yet?08:27
lifelesslsmola: if not, I don't think we can use resource groups.08:27
lsmolalifeless: I believe that should be finished on heat side08:27
lsmolalifeless: yes, I am aware of that issue08:27
*** wuhg has joined #tripleo08:27
lsmolalifeless: I mean I believe it should be finished in J08:28
lifelesslsmola: ok, that will be great08:28
lsmolalifeless: I agree :-)08:28
*** ohadlevy has quit IRC08:28
lifelesslsmola: if it doesn't land08:29
lifelesslsmola: what do we do?08:29
lsmolalifeless: well I believe the plan is it has to08:29
lsmolalifeless: tuskar is being prepared for that08:30
loki184lifeless: Infra team needs +1 from you to merge this patch https://review.openstack.org/#/c/110562/ can you please look at it .08:30
lsmolalifeless: :-) so we will just make sure it does, we have the man power I believe08:30
lifelesslsmola: plans like that make me nervous08:31
lsmolalifeless: yes, you are not alone :-)08:31
SpamapS"march of death"08:31
lsmolalifeless: well if that will not work out, we will have to hack it and use partially merge.py probably08:32
lsmolaSpamapS: hello :-) don't you have middle of the night?08:32
lifelessloki184: sorry, why do they want my +1?08:32
lifelessloki184: the mechanics of correct layout are an -infra issue08:33
SpamapSlsmola: yes. I also have a bit of a deadline. :-P08:33
*** bmahalakshmi2 has joined #tripleo08:33
*** bmahalakshmi has quit IRC08:33
lsmolaSpamapS: :-)08:34
loki184lifeless: Because this tripleo blueprint and  infra core-reviewers commented that they want +1 from tripleo ptl08:34
lifelessloki184: so, I've reviewed it multiple times for you - tell you what, when it has +2 from an infra core08:36
lifelessloki184: I'll happily slap a +1 tripleo wants this badge on it08:36
lsmolad0ugal: hm but I don't know how code that break this got in08:36
lsmolad0ugal: the comparing doesn't fit because of 'BlockStorage0': (12, frozenset([])08:36
d0ugalyeah, which is weird08:37
loki184lifeless: Ok Ty :)08:37
d0ugallsmola: I can't see anything make a frozen set.08:37
d0ugallsmola: All I can think is a dependency has updated/changed?08:37
lsmolad0ugal: probably, but that should not pass the CI right, if there is a test failure it should not pass08:38
d0ugaltrue08:38
tchaypoand 105878?08:39
d0ugallsmola: Could a change in t-h-t cause issues? as it's not versioned. https://github.com/openstack/tuskar/blob/master/requirements.txt#L808:40
tchaypono that merged already08:40
lifelesstchaypo: 105878 == 113214 - see the comments in hp108:40
lsmolad0ugal: it should not, that test just checks how the merge.py is called08:40
lsmolad0ugal: ah08:41
lsmolad0ugal: ok it's probably merge.py change08:41
*** derekh has joined #tripleo08:42
lifelessI changed an internal data structure w/in merge.py08:42
lsmolad0ugal: this is actually calling function from merge py https://github.com/openstack/tuskar/blob/745534460a7f0a5294cf8895844aeacceda398dd/tuskar/heat/template_tools.py#L6908:42
d0ugalaha08:42
lsmolad0ugal: and seems like that has changed08:42
d0ugalok, cool, that's fine.08:42
d0ugalI'll update the test08:42
lifelessyeah08:43
lifelessthe default values there08:43
lifelessaren't the result of parsing08:43
lifelessyou should parse them to get a set of odefaults08:43
lifelessdefaults = merge.parse_scale("NovaCompute=0") etc08:43
lsmolad0ugal: but it probably means it is breaking v1, so maybe you should rather fix that function08:43
d0ugallsmola: True but I can barely remember how that bit works :/08:44
cian_lifeless: have you any comment on https://bugs.launchpad.net/tripleo/+bug/135628308:44
uvirtbotLaunchpad bug 1356283 in tripleo "check-tripleo-novabm-overcloud-f20-nonha gate job is setting USE_IRONIC=0" [Undecided,Invalid]08:44
derekhlifeless: looks like we hit port quota's / limits, taking a look08:44
cian_probably not a bug but more of a feature request08:44
lsmolad0ugal: try it, if you will hit the wall, I can look on it08:45
lifelesscian_: its meant to set USE_IRONIC=008:45
lifelesscian_: its a novabm test job08:45
lsmolad0ugal: though it was marios writing that one I think, so I would have to debug it too :-)08:45
lifelessderekh: oh hai08:46
cian_i'm fine with that but shouldn't we be moving to gating on ironic and not bm08:46
d0ugallsmola: heh, ok, I'll take a loook.08:46
d0ugalWe should figure out what we are doing with v1 too, if we are planning to keep it.08:46
lifelesscian_: while we support nova-bm we need to test it08:46
lsmolad0ugal: ok, let me know if you succeed, will help you merge it asap, as it is probably blocking all patches, right?08:46
*** loki184 has quit IRC08:46
lifelesscian_: we test ironic too08:46
d0ugallsmola: yeah, everything is failing :)08:46
lsmolad0ugal: yaay we should celebrate it as Fail Day08:47
cian_check-tripleo-ironic-overcloud-precise-ha (non-voting)08:47
cian_i see thanks08:47
lifelesscian_: we're trying to increase capacity in CI right08:48
lifelesscian_: we have hp1 going through burnin right now08:48
cian_np, appreciate the problem with machine resources08:49
lifelessderekh: yeah sorry I didn't get deep in today08:51
lifelessderekh: 23/500 passed08:51
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Captionless data table  https://review.openstack.org/11417008:51
tchaypourr.08:52
derekhlifeless: no probs, ya as far as I can see, most of the failures were port limits I'll rekick it once I get that sorted08:52
tchayposo 103454 is going to need to be rebased off of 113154 obviously08:52
derekhlifeless: nothing went to error state08:52
lifelesstchaypo: urrr ?08:52
tchaypoif it's going to be hotified.08:52
derekhlifeless: which is nice08:52
lifelesstchaypo: theres a version in it's history08:52
*** andreaf_ has quit IRC08:52
lifelesstchaypo: which you can just switch back to08:52
tchaypolifeless: ooh, i like it when i can cheat08:52
tchaypo... and there are only two patchsets in its history so it's not hard to figure out which08:53
lifelesstchaypo: erm08:53
tchaypowait08:53
* tchaypo looks back at the right review08:53
tchaypodammit.08:53
*** markmc has joined #tripleo08:53
tchayposlightly more than 2.08:53
tchaypo28 looks like the last hot version08:54
tchaypoand everything since has been rebase..08:54
openstackgerritJames Polley proposed a change to openstack/tripleo-heat-templates: Expose public_interface_tag support.  https://review.openstack.org/10345408:55
lifelessI think it might have a patch under it08:55
lifelessor 108208 yeah08:55
tchayponot sure what you mean about 10820808:56
tchaypobut it is the other one i need to fix..08:56
tchaypoah, and it has a dependency08:56
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Symlink to backup dir until ephemeral storage available  https://review.openstack.org/11395708:57
lifelessit also has a HOT version in hsitory08:58
tchaypoindeed. number 6 in this case.08:59
openstackgerritJames Polley proposed a change to openstack/tripleo-heat-templates: Fixup network configuration for ext-net  https://review.openstack.org/10821308:59
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126309:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604009:00
*** andreaf has joined #tripleo09:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589809:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/134784809:00
uvirtbotLaunchpad bug 1347848 in tripleo "/etc/hosts information about cloud hsots lost on reboot" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135663709:00
uvirtbotLaunchpad bug 1356637 in tripleo "os-refresh-config fails in hosts element when update is different" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/135633909:00
uvirtbotLaunchpad bug 1356339 in diskimage-builder "Add support for vmedia booting in deploy-ironic element" [Undecided,In progress]09:00
*** untriaged-bot has quit IRC09:00
*** bmahalakshmi2 has quit IRC09:00
tchaypohttps://review.openstack.org/#/c/108208/10..11/overcloud-source.yaml09:03
tchaypoversion 10 looks approximately hot, but it has some oddness around line 40009:03
tchaypoFn::Select and so on09:03
tchaypoam I misreading, or is that a not-quite-right merge?09:03
*** sballe_ has joined #tripleo09:04
lsmolalifeless: no need to be backwards compatible here right https://review.openstack.org/#/c/113839/5/os_cloud_config/keystone.py09:04
lsmolalifeless: AFAIK it was never used09:05
*** sballe has quit IRC09:05
*** cian_ has quit IRC09:05
lsmolalifeless: also it doesn't have CLI interface, so it was not definitely used in devtest09:05
*** andreaf has quit IRC09:06
gfidentetchaypo, so that depends on hot migration being reverted?09:06
lifelesslsmola: its a public function in a public module ina  released codebase09:06
tchaypogfidente: patchset 11 does, but we're not reverting hot, so09:06
tchaypoI'm reverting back to 10, which is the HOT version09:07
lsmolalifeless: so what is the way to deprecate it?09:07
lsmolalifeless: cause nobody is using that, plus the heat init is wrong09:07
gfidentetchaypo, ah okay, thought I was missing something09:08
lifelesslsmola: exactly, we deprecate it09:08
*** bmahalakshmi has joined #tripleo09:08
lifelesslsmola: I don't know if there is a stock deprecated decorator in openstack09:08
lifelesslsmola: but if there is use it, otherwise just issut a warnings.warn09:08
*** sballe_ has quit IRC09:09
lsmolalifeless: ok so just LOG.warn('This function is deprecated')09:10
lxslilifeless: thanks for your comment re: eph storage symlinks09:11
lxslilifeless: can you explain why this'll break with readonly roots? is it our use of mktemp?09:12
lxslilifeless: + is your proposal to do the move + symlinks in a use-ephemeral/post-install.d script?09:13
lxslihttps://review.openstack.org/#/c/113957/09:14
lifelesslxsli: because changing something on a readonly fs is kindof hard09:14
openstackgerritJames Polley proposed a change to openstack/tripleo-heat-templates: Permit specifying VLAN mappings to overclouds.  https://review.openstack.org/10820809:15
lxslichanging what? oh the symlinks.09:15
lifelesslxsli: yes post-install.d would be fine, or possibly even cleanup.d, I haven't reviewedthe phases carefully or I'd have suggested a precise one09:15
lxsliRight you are, cheers09:15
*** yamahata has joined #tripleo09:16
d0ugallsmola: I can't get this to error locally :/09:16
lifelessd0ugal: did my suggested fix make sense?09:17
lifelessd0ugal: make sure you have tip merge.py09:17
*** ohadlevy has joined #tripleo09:17
d0ugallifeless: ah, I think I missed the end of your suggestion, looking now.09:17
*** sballe has joined #tripleo09:18
*** loki184 has joined #tripleo09:20
*** akrivoka has joined #tripleo09:22
pensuHi all, please share your thoughts: http://lists.openstack.org/pipermail/openstack/2014-August/008896.html09:23
d0ugalDamn pip cache, now it's failing locally :)09:24
*** sballe has quit IRC09:24
lxslilifeless: is there a spec for RO FS?09:28
*** Yanyanhu has quit IRC09:29
lifelesslxsli: no, its really future work, but we try not to make our future lives harder09:29
lifelesslxsli: and in this case doing something safe is simple09:29
lifelesslxsli: (redoing the symlink at runtime would also be racy)09:29
lxslilifeless: yeah agreed09:29
*** andreaf has joined #tripleo09:30
*** andreaf has quit IRC09:30
*** andreaf has joined #tripleo09:30
*** rlandy has joined #tripleo09:38
*** dtantsur is now known as dtantsur|lunch09:39
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Update the calling to merge.parse_scaling  https://review.openstack.org/11418009:44
d0ugallifeless: So, I'm not sure I fully understood - but I did manage to make it pass with ^09:44
d0ugalit feels hacky tho'09:44
d0ugallsmola: I think you know this bit better than me ^09:44
*** ramishra has joined #tripleo09:45
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Update the call to merge.parse_scaling  https://review.openstack.org/11418009:46
lifelessd0ugal: ok so what I'm saying is that line 60 is the bug09:46
d0ugalah09:47
lsmolalifeless: oh, the format changed?09:47
d0ugalSo our mock is wrong too then :)09:47
lifelessd0ugal: reviewed09:49
d0ugalthanks!09:49
lifelesslsmola: the internal structure yeah09:49
lsmolalifeless: ok09:49
lifelesslsmola: arguably a mistake, but there we go09:49
lsmolalifeless: :-)09:50
*** lucasagomes has joined #tripleo10:02
*** andreaf has quit IRC10:02
*** andreaf has joined #tripleo10:03
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Update the call to merge.parse_scaling  https://review.openstack.org/11418010:05
d0ugallifeless, lsmola: ^ I think that makes more sense but it still feels somewhat brittle.10:05
lsmolad0ugal: if the tests are passing I guess it's fine10:06
d0ugalI just mean if the internal structure changes again our mock check will fail.10:07
d0ugalbut given we may need to make changes in that case, I guess it's useful to know.10:07
lifelessd0ugal: looks fine to me10:08
lsmolad0ugal: well yeah, we are using the merge.py python interface, so hopefully it will not change much10:08
lifelessI'm not sure why the mock needs to know the structure10:09
lifelessmerge.py's stuff should be trivial to feed real data and avoid mocking10:09
lifelessanyhoq10:09
d0ugalTrue, I'm not sure, I was just adapting what was there :)10:09
lifeless+2'd10:10
d0ugalThanks10:10
lifelessnot sure if there is CI there10:10
lifelessso someone else can +A10:10
lsmolad0ugal: no tuskar is still not in CI, thought the runs are there10:10
lsmolalifeless: ^10:11
lsmolad0ugal: ok approving10:11
*** pradeep1 has quit IRC10:11
openstackgerritA change was merged to openstack/tuskar-ui: Updated from global requirements  https://review.openstack.org/11407510:11
*** nosnos has quit IRC10:16
*** ramishra has quit IRC10:19
*** ramishra has joined #tripleo10:21
openstackgerritA change was merged to openstack/tuskar: Update the call to merge.parse_scaling  https://review.openstack.org/11418010:24
openstackgerritAna Krivokapic proposed a change to openstack/tuskar-ui: Filter powered on/off nodes correctly  https://review.openstack.org/11419110:28
*** pelix has joined #tripleo10:29
*** ramishra has quit IRC10:30
*** ramishra has joined #tripleo10:31
openstackgerritMatthew Gilliard proposed a change to openstack/os-cloud-config: Allow capabilities to be set on ironic nodes  https://review.openstack.org/11004810:35
jprovaznlifeless: you around?10:36
*** pblaho has joined #tripleo10:41
openstackgerritJiri Tomasek proposed a change to openstack/tuskar-ui: Style the node details page according to the wireframes.  https://review.openstack.org/11349810:42
openstackgerrittherese-mchale proposed a change to openstack/tripleo-heat-templates: Set the MySQL root password  https://review.openstack.org/10256210:47
openstackgerritNicholas Randon proposed a change to openstack/tripleo-heat-templates: Add strict dependencies to the undercloud template  https://review.openstack.org/11419710:49
*** ramishra has quit IRC10:51
openstackgerritNicholas Randon proposed a change to openstack/tripleo-heat-templates: Add strict dependencies to the undercloud template  https://review.openstack.org/11419710:52
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394610:52
*** dtantsur|lunch is now known as dtantsur10:52
openstackgerritNicholas Randon proposed a change to openstack/tripleo-heat-templates: Add strict dependencies to the undercloud template  https://review.openstack.org/11419710:52
openstackgerritNicholas Randon proposed a change to openstack/tripleo-heat-templates: Add strict dependencies to the undercloud template  https://review.openstack.org/11419710:53
openstackgerritLennart Regebro proposed a change to openstack/tuskar-ui: Fixed various styling issues  https://review.openstack.org/11397011:01
openstackgerritA change was merged to openstack/tripleo-image-elements: Custom policy for nova-api tmpfs errors  https://review.openstack.org/10744711:02
*** andreaf has quit IRC11:05
openstackgerritA change was merged to openstack/tripleo-image-elements: Custom policy for ssh-keygen failure  https://review.openstack.org/10723311:05
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394611:07
*** yamahata has quit IRC11:09
*** loki184 has quit IRC11:10
*** ramishra_ has joined #tripleo11:18
*** ramishra_ has quit IRC11:31
*** julim has joined #tripleo11:33
*** julim has quit IRC11:34
*** ramishra_ has joined #tripleo11:35
*** lblanchard has joined #tripleo11:36
*** lucasagomes is now known as lucas-hungry11:38
*** bmahalakshmi has quit IRC11:40
openstackgerritAna Krivokapic proposed a change to openstack/tuskar-ui: Filter powered on/off nodes correctly  https://review.openstack.org/11419111:40
*** morazi has joined #tripleo11:46
*** ramishra_ has quit IRC11:47
*** noslzzp has joined #tripleo11:48
*** lucas-hungry is now known as lucasagomes11:53
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394612:02
*** weshay has joined #tripleo12:05
*** jdob has joined #tripleo12:07
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.context  https://review.openstack.org/11290412:11
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.fixture  https://review.openstack.org/11290512:11
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.log_handler  https://review.openstack.org/11290612:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.network_utils  https://review.openstack.org/11290712:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.cliutils  https://review.openstack.org/11290812:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused/unfinished tuskar.manager  https://review.openstack.org/11320512:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.rpc  https://review.openstack.org/11289812:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.service  https://review.openstack.org/11289912:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.processutils  https://review.openstack.org/11290012:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.notifier  https://review.openstack.org/11285812:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.loopingcall  https://review.openstack.org/11290112:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.threadgroup  https://review.openstack.org/11290212:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.periodic_task  https://review.openstack.org/11285612:12
openstackgerritDougal Matthews proposed a change to openstack/tuskar: Remove unused openstack.common.eventlet_backdoor  https://review.openstack.org/11290312:12
openstackgerritJiri Tomasek proposed a change to openstack/tuskar-ui: Style the node details page according to the wireframes.  https://review.openstack.org/11421812:14
*** julim has joined #tripleo12:15
*** dprince has joined #tripleo12:21
openstackgerritJiri Tomasek proposed a change to openstack/tuskar-ui: Style the node details page according to the wireframes.  https://review.openstack.org/11421812:22
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: reenable boot an instance from a volume  https://review.openstack.org/11297912:23
openstackgerritJiri Tomasek proposed a change to openstack/tuskar-ui: Style the node details page according to the wireframes  https://review.openstack.org/11421812:24
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Do not run pki init as default  https://review.openstack.org/11324012:29
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create tenant idempotent  https://review.openstack.org/11385212:29
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Allow setup services endpoints  https://review.openstack.org/11388212:29
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create keystone endpoint idempotent  https://review.openstack.org/11388312:29
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create roles idempotent  https://review.openstack.org/11383912:29
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make setup endpoint idempotent  https://review.openstack.org/11282512:29
gfidentederekh, out of my curiosity, I can see some ha ci jobs starting to pass now which is nice12:31
gfidenteI wonder if it would make any sense to simulate failure of a node too, in the job? maybe stop of the nodes from undercloud (or a controller and a compute) and relaunch a user image?12:32
derekhgfidente: yup, looks like about 1/2 are passing12:32
derekhgfidente: its probably worth thinking about once the job itself is reliably passing12:33
gfidenteyeah, agreed12:33
gfidentethat would be awesome to me though, I'll keep it in my todolist to see if at some point turns to be good to talk about12:34
*** pensu has quit IRC12:35
*** andreaf has joined #tripleo12:36
openstackgerritRadomir Dopieralski proposed a change to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394612:38
openstackgerritA change was merged to openstack/tuskar-ui: Style the node details page according to the wireframes  https://review.openstack.org/11421812:40
*** pblaho_ has joined #tripleo12:41
*** pblaho has quit IRC12:44
lsmolalifeless: what is  VWS, can't google anything like that at pep812:45
openstackgerritA change was merged to openstack/tuskar-ui: Filter powered on/off nodes correctly  https://review.openstack.org/11419112:54
*** noslzzp has quit IRC12:54
openstackgerritPetr Blaho proposed a change to openstack/python-tuskarclient: Adds empty RoleManager to v2 client  https://review.openstack.org/11423512:56
openstackgerritPetr Blaho proposed a change to openstack/python-tuskarclient: Adds binding for POST /v2/plans/:plan_uuid/roles  https://review.openstack.org/11423612:56
openstackgerritPetr Blaho proposed a change to openstack/python-tuskarclient: Adds binding DELETE /v2/plans/:plan/roles/:role  https://review.openstack.org/11423712:56
openstackgerritPetr Blaho proposed a change to openstack/python-tuskarclient: Adds binding for GET /v2/roles  https://review.openstack.org/11423812:56
*** pblaho_ has quit IRC12:57
*** pblaho_ has joined #tripleo12:57
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Do not run pki init as default  https://review.openstack.org/11324012:58
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create tenant idempotent  https://review.openstack.org/11385212:58
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create keystone endpoint idempotent  https://review.openstack.org/11388312:58
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make create roles idempotent  https://review.openstack.org/11383912:58
openstackgerritLadislav Smola proposed a change to openstack/os-cloud-config: Make setup endpoint idempotent  https://review.openstack.org/11282512:58
openstackgerritLennart Regebro proposed a change to openstack/tuskar-ui: Fixed various styling issues  https://review.openstack.org/11397013:02
*** noslzzp has joined #tripleo13:08
*** dshulyak has joined #tripleo13:11
openstackgerritA change was merged to openstack/tuskar-ui: Plan editing form  https://review.openstack.org/11394613:12
*** pblaho_ is now known as pblaho|afk13:17
*** jcoufal has quit IRC13:19
*** dshulyak has quit IRC13:19
*** jcoufal has joined #tripleo13:19
*** jprovazn has quit IRC13:21
jistrjdob: Btw i think it's better to just submit a new revision rather than submit patches on top, to make it an atomic operation -- assert that we don't then abandon the fix for example. If we already know we need to fix something in a patch that's not in master yet, we should just fix it in that patch. Giving +2 this time not to bikeshed. We've already had this discussion in Tuskar before :)13:22
jistr^ re fix-ups on patches as dependent patches, rather than just new revisions13:22
*** akrivoka has quit IRC13:22
jdobjistr: ya, i didn't intend to do that workflow13:23
jdobi kinda screwed things up13:23
jdoboh oh, you mean the namespace fix thing13:23
*** akrivoka has joined #tripleo13:23
jdobi hadn't thought about it; forgot the initial writing of that code was still out for review13:24
jdobi see your point if I had fixed it in response to your comments13:24
*** pblaho|afk is now known as pblaho_13:24
*** sballe has joined #tripleo13:24
*** jprovazn has joined #tripleo13:25
*** pblaho__ has joined #tripleo13:28
*** pblaho_ has quit IRC13:29
*** pblaho__ is now known as pblaho13:29
*** jistr has quit IRC13:30
openstackgerritNicholas Randon proposed a change to openstack/tripleo-heat-templates: Add strict dependencies to the undercloud template  https://review.openstack.org/11419713:31
*** pblaho is now known as pblaho|meeting13:32
*** jistr has joined #tripleo13:32
*** sballe has quit IRC13:33
*** andreaf_ has joined #tripleo13:38
*** rushiagr is now known as rushiagr_away13:39
*** andreaf has quit IRC13:41
openstackgerritA change was merged to openstack/tuskar-ui: Fixed various styling issues  https://review.openstack.org/11397013:45
*** pcrews has joined #tripleo13:47
*** sballe has joined #tripleo13:54
*** ccrouch1 has quit IRC14:03
*** pensu has joined #tripleo14:04
*** jang2 has joined #tripleo14:07
*** jang has quit IRC14:08
*** jang1 has quit IRC14:10
*** jang2 has quit IRC14:10
*** jang has joined #tripleo14:12
*** jang has quit IRC14:14
*** jang has joined #tripleo14:15
*** jang has quit IRC14:18
*** ccrouch has joined #tripleo14:20
*** jang has joined #tripleo14:20
*** jang has quit IRC14:22
*** jang has joined #tripleo14:23
*** rdopieralski has quit IRC14:25
*** andreaf_ is now known as andreaf14:39
*** rushiagr_away is now known as rushiagr14:41
*** pensu has quit IRC14:42
*** k4n0 has quit IRC14:45
*** nealph_ is now known as nealph14:48
*** cwolferh has joined #tripleo14:49
*** vinsh has joined #tripleo14:52
*** cwolferh has quit IRC14:59
*** marun has joined #tripleo15:00
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/135633915:00
uvirtbotLaunchpad bug 1356339 in diskimage-builder "Add support for vmedia booting in deploy-ironic element" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135663715:00
uvirtbotLaunchpad bug 1356637 in tripleo "os-refresh-config fails in hosts element when update is different" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135126315:00
uvirtbotLaunchpad bug 1351263 in tripleo "Seed init failed, nova list returned 503" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604015:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589815:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
*** ddieterly has joined #tripleo15:00
*** mestery is now known as mestery_afk15:01
openstackgerritA change was merged to openstack-infra/tripleo-ci: Add /mnt/state/var/log contents to indexed test logs.  https://review.openstack.org/9925015:03
*** marun has quit IRC15:04
*** marun has joined #tripleo15:04
openstackgerritA change was merged to openstack/tripleo-image-elements: Fix Fedora builds on Ubuntu  https://review.openstack.org/10898615:05
*** jprovazn has quit IRC15:05
SpamapSsomething is going horribly wrong on the seed still15:05
*** marun has quit IRC15:05
SpamapSafter a few hours, getting the same bursty networking15:05
SpamapSand a load average of 10+15:06
*** jcoufal has quit IRC15:06
*** jcoufal has joined #tripleo15:06
*** marun has joined #tripleo15:07
*** pblaho|meeting is now known as pblaho15:08
*** pblaho has quit IRC15:08
*** pblaho has joined #tripleo15:08
vinshhey greghaynes, SpamapS ... tomorrow is my last day on openstack with HPQ.  Starting monday I'll be openstack with TWC.15:08
*** akrivoka has quit IRC15:09
*** marun has quit IRC15:09
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: Use Percona Packages for Ubuntu and Debian  https://review.openstack.org/9013415:09
SpamapSvinsh: very interesting!15:09
vinshWill find out if I end up bringing some /heat/ into TWC sometime soon15:10
*** akrivoka has joined #tripleo15:10
SpamapSvinsh: Hopefully you'll still let us buy you a beer on the corporate card next time we hang. ;)15:10
vinshI have word the I have a ticket to Paris already!  hope so!15:10
*** jcoufal has quit IRC15:10
*** dprince has quit IRC15:11
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: MySQL upstart script setup  https://review.openstack.org/9996415:12
*** marun has joined #tripleo15:12
*** jistr has quit IRC15:12
*** jistr has joined #tripleo15:13
*** pblaho_ has joined #tripleo15:16
*** marun has quit IRC15:17
*** marun has joined #tripleo15:17
*** mestery_afk is now known as mestery15:19
jmoffittquestion for any tuskar folks who might be about... I pulled down the latest UI yesterday, found that it now has an immutable default config (which is fine for my purposes), but also that if you hit deploy the UI breaks with the following exception: Exception Value: 'NoneType' object has no attribute 'status'    is this a known issue? is there a fix in the pipeline?15:20
*** pblaho has quit IRC15:20
*** uberj_ is now known as uberj15:20
*** Hefeweiz1n is now known as Hefeweizen15:21
*** ddieterly has quit IRC15:25
*** yamahata has joined #tripleo15:27
*** pblaho_ has quit IRC15:30
*** sballe_ has joined #tripleo15:33
*** lblanchard has quit IRC15:33
*** lblanchard has joined #tripleo15:34
*** jistr has quit IRC15:34
*** dtantsur is now known as dtantsur|afk15:36
*** sballe has quit IRC15:36
openstackgerritDerek Higgins proposed a change to openstack/tripleo-image-elements: Increase the quotas for nodepool on hp1  https://review.openstack.org/11427615:37
*** markmc_ has joined #tripleo15:37
*** ByPasS has quit IRC15:37
derekhlifeless: big hammer is running again (hit problems today with various quotas and limits), ^^ will up the quotas to what I think we need15:38
derekhlifeless: but I also hit a floatingip limit 138.35.77.0/2515:38
derekhso currently running big hammer with 100 sim processes15:38
*** ByPasS has joined #tripleo15:39
*** markmc has quit IRC15:40
regebrono but seriously, how long does this undercloud take to create?15:43
regebroMeh!15:43
*** cody-somerville has joined #tripleo15:43
*** cody-somerville has joined #tripleo15:43
derekhregebro: here is an example from our CI jobs http://logs.openstack.org/42/114242/1/check-tripleo/check-tripleo-ironic-undercloud-precise-nonha/a6c42fb/console.html15:45
derekhregebro: just over an hour for seed and undercloud15:45
regebroderekh: I was just going to say that I think it's been waiting for the undercloud to start for an hour now.15:46
regebroderekh: And it timed out.15:46
regebroOh well, nothing to do, let's try that again then.15:47
openstackgerritA change was merged to openstack/tripleo-heat-templates: Fix BLOCKSTORAGESCALE and SWIFTSTORAGESCALE in Makefile  https://review.openstack.org/11206215:47
derekhregebro: yup, something wrong there, in the CI jobs the wait for undercloud is about 8minutes15:48
derekhhttp://logs.openstack.org/42/114242/1/check-tripleo/check-tripleo-ironic-undercloud-precise-nonha/a6c42fb/console.html#_2014-08-14_14_31_03_15415:48
regebroI'll try to run each bit of devtest.sh separately then.15:48
openstackgerritA change was merged to openstack/tripleo-heat-templates: Add BlockStorage and SwiftStorage nodes into hosts  https://review.openstack.org/11206415:50
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-heat-templates: Remove unneeded neutron config parts from block-storage.yaml  https://review.openstack.org/11206715:54
*** sballe has joined #tripleo16:06
*** markmc_ has quit IRC16:06
*** sballe__ has joined #tripleo16:07
*** sballe_ has quit IRC16:08
*** sballe has quit IRC16:11
*** cwolferh has joined #tripleo16:16
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-heat-templates: Use VIP for rabbit/keystone and mysql in block-storage.yaml  https://review.openstack.org/11206916:20
*** sballe__ has quit IRC16:21
*** jcoufal has joined #tripleo16:30
*** dprince has joined #tripleo16:32
*** rollerj has joined #tripleo16:33
rollerjhello all!  has anybody seen issues when building the undercloud where the "success" signal in 99-refresh-completed fails with a "ResourceFailure: AuthorizationFailure: Authorization failed: Internal Server Error (HTTP 500)"?16:34
*** akrivoka has quit IRC16:35
SpamapSrollerj: I had a success about 11 hours ago that did not experience problems sending that signal16:35
*** ramishra has joined #tripleo16:37
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-heat-templates: Make BlockStorage deployment dependent on controller0AllNodes  https://review.openstack.org/11207316:39
*** pelix has quit IRC16:40
rollerji know that this is not a general issue, but i am running into it in my own builds.  i'm fishing for ideas around what might cause this behavior to know where to look next (have already tried the rebuild, reboot, retry, loop several times)16:41
*** pelix has joined #tripleo16:41
*** ramishra has quit IRC16:41
*** yamahata has quit IRC16:43
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-heat-templates: Ensure glance.host and glance.cinder are set for BlockStorage nodes  https://review.openstack.org/11207416:44
*** derekh has quit IRC16:45
*** dshulyak has joined #tripleo16:47
*** dshulyak has quit IRC16:51
*** saurabhs has joined #tripleo16:51
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Symlink to backup dir until ephemeral storage available  https://review.openstack.org/11395716:52
*** lucasagomes has quit IRC16:52
SpamapSI wonder if this weirdness with the seed networking is the horrible trusty kernel16:53
*** sballe has joined #tripleo16:55
*** sballe has quit IRC16:56
*** sballe has joined #tripleo16:56
*** sballe has quit IRC16:56
*** dshulyak has joined #tripleo16:57
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Symlink to backup dir until ephemeral storage available  https://review.openstack.org/11395716:57
*** sballe has joined #tripleo16:57
*** penick has joined #tripleo17:02
openstackgerritA change was merged to openstack/tripleo-incubator: reenable boot an instance from a volume  https://review.openstack.org/11297917:02
saurabhshey guys is there a os-create-db, elements or command in tripleO (to help create databases on non-local mysql).17:05
saurabhsI was looking at a patch https://review.openstack.org/#/c/96448/ and it was abandoned for os-create-db but I am not abel to locate where this os-create-db is used.17:05
saurabhscan somebody point me to some example of this os-create-db17:06
*** jang1 has joined #tripleo17:06
slaglesaurabhs: it's just a small helper script in the os-svc-install element in tripleo-image-elements17:09
slagleit will only create local db's17:09
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-heat-templates: Replace occurrences of list_join with Fn::Join  https://review.openstack.org/11430017:11
saurabhsslagle: thanks. is there anything that allows us to create database for the service if mysql is not running locally. how do we handle this?17:11
SpamapSam I the only one seeing _awful_ performance on Ubuntu?17:14
SpamapScorosync pegging a CPU17:15
SpamapS(could be the host OS and kvm issues, I really don't know)17:15
openstackgerritMichael Tupitsyn proposed a change to openstack/tripleo-incubator: Idempotency for keystone initialization  https://review.openstack.org/10617517:15
slaglesaurabhs: that exists today in tripleo, i don't really think so17:16
openstackgerritMichael Tupitsyn proposed a change to openstack/tripleo-image-elements: Keystone/LDAP integration  https://review.openstack.org/10616717:16
openstackgerritMichael Tupitsyn proposed a change to openstack/tripleo-image-elements: Keystone LDAP domain configuration  https://review.openstack.org/10811717:16
saurabhsslagle: thanks17:19
*** wuhg has quit IRC17:22
openstackgerritGiulio Fidente proposed a change to openstack/tripleo-heat-templates: Passthrough and allNodesConfig for BlockStorage nodes  https://review.openstack.org/11207517:25
*** penick has quit IRC17:27
greghaynesSpamapS: I am not seeing those kinds of perf issues17:30
*** penick has joined #tripleo17:31
*** ramishra has joined #tripleo17:37
*** ramishra has quit IRC17:42
SpamapSgreghaynes: I didn't for the first 3 hours my devtest was up17:42
SpamapSbut I just left it doing nothing17:42
SpamapSand now it is unusable17:42
greghaynesmine is running overnight... youre just sshing into things and seeing lag?17:43
SpamapSsuper high load17:44
SpamapSnetwork lag17:44
greghaynesso I think I was seeing that on my older cpu box17:44
* greghaynes kicks off build there17:44
SpamapSmodel name      : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz17:45
SpamapStimes 24 :-P17:45
greghaynesinterestingly, my 'older cpu' box is hp hardware, the other (without lag issues) is not17:46
SpamapSwait what.. isn't 'vt' a flag that is needed for VM perf?17:46
SpamapSvmx17:47
*** dsneddon has joined #tripleo17:47
SpamapSok17:47
*** regebro has quit IRC17:50
*** pelix has quit IRC17:52
*** panda has quit IRC18:06
*** panda has joined #tripleo18:06
*** sballe has quit IRC18:08
NobodyCamquestion, after an update I am now getting "Could not find admin role" while attempting to setup the seed vm, is there something I need to do for that?18:14
*** andreaf has quit IRC18:18
*** andreaf has joined #tripleo18:19
*** rlandy has quit IRC18:20
*** jtomasek_ has quit IRC18:23
*** dprince has quit IRC18:26
*** rlandy has joined #tripleo18:29
*** pensu has joined #tripleo18:32
*** pblaho_ has joined #tripleo18:33
*** nealph_ has joined #tripleo18:33
*** nealph has quit IRC18:36
openstackgerritJulia Kreger proposed a change to openstack/tripleo-image-elements: Addition of basic ansible install  https://review.openstack.org/11300718:37
*** chuckC has quit IRC18:38
*** ramishra has joined #tripleo18:38
*** ramishra has quit IRC18:42
gfidentethis is fixing a problem in CI, should be merged ASAP https://review.openstack.org/#/c/114300/18:48
* gfidente salute everyone18:48
bnemecslagle: lifeless: greghaynes: SpamapS: ^18:49
bnemecPlease fix my screwup...18:49
greghaynesoh, was there just a race condition with merging?18:51
greghayneswhere we fixed the fn::join's then merged something that added list_join?18:51
bnemecgreghaynes: Yeah, the CI pass happened before that change, and somehow didn't conflict with the change itself.18:51
greghaynesok, +2'd18:52
openstackgerritA change was merged to openstack/tripleo-heat-templates: Replace occurrences of list_join with Fn::Join  https://review.openstack.org/11430018:52
bnemecgreghaynes: Thanks!18:52
greghayneslol, CI be dammed18:52
bnemecgfidente: ^18:52
slaglegreghaynes: heh, it passed CI18:52
greghaynesoh!18:52
slaglecheck out the new Toggle CI hawtness18:52
greghayneshah, still getting used to that18:53
greghaynesAnd you know its good because it even passed the HA CI18:53
bnemec\o/ :-)18:53
*** cwolferh_ has joined #tripleo18:56
*** dprince has joined #tripleo18:58
*** cwolferh has quit IRC18:58
*** pensu has quit IRC19:00
*** pensu has joined #tripleo19:01
*** cwolfe__ has joined #tripleo19:02
openstackgerritGregory Haynes proposed a change to openstack/os-collect-config: Exit with error status on failed command  https://review.openstack.org/11133419:03
*** cwolferh_ has quit IRC19:04
*** chuckC has joined #tripleo19:10
*** loki184 has joined #tripleo19:13
*** dshulyak has quit IRC19:26
*** loki184 has quit IRC19:36
*** ByPasS has quit IRC19:37
*** ramishra has joined #tripleo19:39
*** markmc has joined #tripleo19:41
*** ramishra has quit IRC19:44
lifelessmorning19:47
dprincelifeless: morning19:50
openstackgerritJay Dobies proposed a change to openstack/tuskar: Wires up REST API to underlying Tuskar logic  https://review.openstack.org/11398819:51
openstackgerritJay Dobies proposed a change to openstack/tuskar: Wires up REST API to underlying Tuskar logic  https://review.openstack.org/11398819:52
*** penick has quit IRC19:59
*** dshulyak has joined #tripleo20:11
*** andreaf_ has joined #tripleo20:14
*** andreaf has quit IRC20:17
*** rlandy has quit IRC20:18
*** dshulyak has quit IRC20:20
*** dshulyak has joined #tripleo20:20
*** jml has quit IRC20:27
*** jml has joined #tripleo20:27
openstackgerritOpenStack Proposal Bot proposed a change to openstack/os-cloud-config: Updated from global requirements  https://review.openstack.org/11433220:28
openstackgerritOpenStack Proposal Bot proposed a change to openstack/tuskar-ui: Updated from global requirements  https://review.openstack.org/11433420:30
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Remove disable_interface from dhcp-all-interfaces  https://review.openstack.org/11433920:33
openstackgerritDan Prince proposed a change to openstack/diskimage-builder: Formatting fixes for dhcp-all-interfaces  https://review.openstack.org/11434020:33
*** ramishra has joined #tripleo20:40
openstackgerritA change was merged to openstack/tripleo-incubator: Base decision to use MariaDB on NODE_DIST  https://review.openstack.org/11355720:40
*** dshulyak has quit IRC20:40
openstackgerritA change was merged to openstack/tripleo-image-elements: Remove hard-coded ports form ring builder script  https://review.openstack.org/11245520:42
*** ramishra has quit IRC20:44
*** rushiagr is now known as rushiagr_away20:47
openstackgerritA change was merged to openstack-infra/tripleo-ci: Remove unneeded CI workarounds  https://review.openstack.org/11319520:48
*** ccrouch has quit IRC20:48
openstackgerritNicholas Randon proposed a change to openstack/tripleo-image-elements: Handle double path registration  https://review.openstack.org/11354620:49
*** penick has joined #tripleo20:50
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in more elements  https://review.openstack.org/10876220:53
openstackgerritJulia Kreger proposed a change to openstack/tripleo-image-elements: Addition of basic ansible install  https://review.openstack.org/11300720:54
*** jdob has quit IRC20:55
*** sseago_ has quit IRC20:55
*** sseago__ has joined #tripleo20:55
*** sseago_ has joined #tripleo20:56
*** jcoufal has quit IRC20:58
clarkbhello tripleo20:59
clarkbnext week I am going to switch the rest of the world over to trusty test nodes as the default test node20:59
*** sseago__ has quit IRC21:00
clarkbtripleo clouds are a bit different so I have been treating it as a separate thing21:00
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/135633921:00
uvirtbotLaunchpad bug 1356339 in diskimage-builder "Add support for vmedia booting in deploy-ironic element" [Undecided,In progress]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135663721:00
clarkbbut I should do the same for you all as well21:00
uvirtbotLaunchpad bug 1356637 in tripleo "os-refresh-config fails in hosts element when update is different" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135589821:00
uvirtbotLaunchpad bug 1355898 in tripleo "mariadb doesn't start after an upgrade" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/135604021:00
uvirtbotLaunchpad bug 1356040 in tripleo "Glance entry in iptables needs 9191 entry for Fedora" [Undecided,Fix committed]21:00
*** untriaged-bot has quit IRC21:00
clarkbthere are two pieces of this. First is updating the nodes that we boot in the test clouds to get access to your test envs. then of course the test envs21:01
clarkbcan I just do the jenkins nodes or do we need to test all that?21:01
*** penick has quit IRC21:02
lifelessclarkb: JFDI I think21:03
clarkblifeless: cool21:03
*** ccrouch has joined #tripleo21:03
clarkbI need to make sure we have trusty base images there and build one before I switch anything so it won't happen immediately but I will go ahead and full steam this21:03
clarkblifeless: then I would leave it up to you guys to update the test envs right?21:05
clarkblifeless: so that can actually happen completely independently which is nice21:05
*** penick has joined #tripleo21:07
lifelessclarkb: the testenvs don't need changing21:08
lifelessclarkb: we already build and test with trusty21:08
lifelessclarkb: there would be a DIB_RELEASE variable in modules/openstack_project/files/jenkins_job_builder/config/tripleo.yaml to override that21:08
clarkbI see21:08
lifelessthe testenv workers are irrelevant as they are vm hosts21:09
lifelessthe testenv vms themselves are blank slates we deploy to21:09
greghayneslifeless: Why are our tests named tripleo-*-precise?21:10
clarkblifeless: pleia2: nova image list says there is no trusty image. Do we have glance access or do you provide those images for us and I can bug you for an image?21:12
pleia2last I knew the infra account would just upload the images to glance on the tripleo server it needed to use21:16
clarkbpleia2: thanks21:16
pleia2glance image-create etc21:16
clarkbyup, I don't do it often but its pretty straightforward21:17
* pleia2 nods21:17
*** dprince has quit IRC21:19
lifelessgreghaynes: because the jenkins slave is precise21:25
lifelessgreghaynes: when clarkb upgrades things he'll rename the jobs too21:25
greghaynesah, so that precise test is actually testing trusty21:25
lifelessmore specifically21:26
lifelessits testing the default behavior for built OS when the host is precise21:26
lifelesswhich today is trusty21:26
lifelessbut has previously been saucy, for instance21:26
greghaynesgotcha21:26
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack/tripleo-image-elements: Add an element to restore ssh keys from /mnt/state  https://review.openstack.org/11436021:30
*** openstackgerrit has quit IRC21:46
*** gfidente has quit IRC21:47
*** openstackgerrit has joined #tripleo21:47
*** jang1 has quit IRC21:51
*** ramishra has joined #tripleo21:54
*** andreaf has joined #tripleo21:55
*** andreaf has quit IRC21:56
*** andreaf has joined #tripleo21:56
*** andreaf has quit IRC21:56
*** andreaf has joined #tripleo21:57
*** andreaf_ has quit IRC21:58
*** ramishra has quit IRC21:59
tchaypogood morning22:00
lifelessmorning22:05
lifelesshows your ear?22:05
*** noslzzp has quit IRC22:05
*** noslzzp has joined #tripleo22:06
*** pensu has quit IRC22:06
*** noslzzp has quit IRC22:06
tchayporinging22:10
tchaypopainkillers are working though, and its continuing to get better22:11
tchaypothe main value I'm seeing from prep_source_repos so far is that it finds all the merge conflicts we're going to have to fix one day anyway22:13
* tchaypo removes tongue from cheek22:13
*** morazi has quit IRC22:13
openstackgerritJames Polley proposed a change to openstack/tripleo-heat-templates: Expose public_interface_tag support.  https://review.openstack.org/10345422:14
tchaypoonce i get prep_source_repos able to apply everything cleanly I'll kick off an undercloud build and then do my expense report before it gets any more overdue and causes Ng any more hassle22:15
tchaypooh, poop. I seem to have missed the cutoff for changing my password.22:16
greghaynesRIP22:16
tchaypoit was good while it lasted22:18
tchaypowoot, prep_source_repos applied cleanly!22:18
tchaypogo, undercloud, go!22:18
* greghaynes doing something pretty nasty for ssh host key restoration22:19
lifelessgreghaynes: you saw SpamapS patch ?22:20
tchaypossh host key restoration?22:20
greghaynesyep22:20
greghaynesim adding a pretty silly hack to it22:20
lifeless09:30 < openstackgerrit> Clint 'SpamapS' Byrum proposed a change to openstack/tripleo-image-elements: Add an22:20
lifeless                         element to restore ssh keys from /mnt/state  https://review.openstack.org/11436022:20
greghaynestheres this issue that external services need a way to check for when the ssh key has been restored since there will be a brief period where the host key is invalid22:21
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Add an element to restore ssh keys from /mnt/state  https://review.openstack.org/11436022:21
greghaynes^22:21
greghaynesDo not advise reading on a full lunch22:22
tchaypoit's 8:22am, no full lunches have been had22:23
tchaypoah22:23
tchaypojust from the title i can make a guess22:23
tchaypothis is for when a node gets updated with a new image22:23
tchaypowe want to preserve the old ssh host keys, so the rest of the world doesn't have too much of a tantrum?22:24
greghaynesYes, I might be trying to hijack the ssh version string to allow us to know when the host key has been updated :)22:24
tchaypoit all sounded reasonable22:24
tchaypowhen i thought you were storing the host keys on /mnt/state22:24
tchaypobut then you said "verson string" and I got sad22:24
*** markmc has quit IRC22:25
tchayposo the idea here is that if we've fixed the host keys, the version string should let us know22:26
tchaypoif we haven't fixed it yet, we'll know because (A) the host key is wrong, and (B) there's no version string22:26
tchaypois that right?22:27
greghaynesmostly - itll still have a version string, most tools get pretty mad and (rightfully) exit with failure on hostkey validation error22:27
greghaynesso this way there can be a check infront of that of "has the key been restored yet"22:28
tchayposo we're never going to be *trusting* that versionstring for anything22:29
tchaypoie, we're never going to be saying "oh, but it says its changed, so that's okay, I'll just accept the new key"22:29
tchaypoif we see a wrong key we're going to be sitting back and waiting for the MitM to give up and go away and/or the key to be restored22:30
greghaynesprobably not, hopefully this is just a hack for a short term solution22:30
greghaynesI just like how easy preserving these keys seems, but then turns into a giant mess22:30
*** penick has quit IRC22:31
*** penick has joined #tripleo22:33
tchaypoinstead of preserving them, we could just use the first mac address as a seed, so we get the same randomly-generated key every time22:34
* tchaypo sets mode +troll22:34
tchaypoCannot update a failed stack22:34
greghaynes"randomly-generated"22:35
greghaynesdef rand(): return 422:35
greghaynes?22:35
SpamapSJust return "Red Dwarf" .. its pretty random22:38
greghayneswhat, our pacemaker element has a sleep 30 in it?22:40
SpamapSsure why not22:40
SpamapSsleeps make the world go round22:40
*** jml has quit IRC22:43
*** jml has joined #tripleo22:47
openstackgerritJulia Kreger proposed a change to openstack/tripleo-image-elements: Addition of basic ansible install  https://review.openstack.org/11300722:51
openstackgerritA change was merged to openstack/tripleo-image-elements: Add rdo-release element  https://review.openstack.org/10459822:53
*** ramishra has joined #tripleo22:55
*** ramishra has quit IRC22:59
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in fedora  https://review.openstack.org/10844122:59
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in more elements  https://review.openstack.org/10876222:59
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: foo  https://review.openstack.org/11438523:00
*** penick has quit IRC23:03
*** chuckC has quit IRC23:08
*** ccrouch has quit IRC23:10
*** penick has joined #tripleo23:10
*** penick has quit IRC23:11
*** penick has joined #tripleo23:12
openstackgerritJonathan Brownell proposed a change to openstack/tripleo-incubator: Exposed fixed VIP address to Heat ControlFixedIPs parameter  https://review.openstack.org/11439123:21
*** ccrouch has joined #tripleo23:23
*** jml has quit IRC23:25
*** jml has joined #tripleo23:28
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in fedora  https://review.openstack.org/10844123:41
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: package-installs for pre-install.d/post-install.d  https://review.openstack.org/10843723:41
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Use package-installs in more elements  https://review.openstack.org/10876223:41
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Add an element to restore ssh keys from /mnt/state  https://review.openstack.org/11436023:50
*** ramishra has joined #tripleo23:56
*** chuckC has joined #tripleo23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!