Friday, 2014-03-28

Lotus907efiif I look around in the devtest scripts will it be semi-obvious how to extract the ramdisk and kernel?00:00
StevenKIf you're using dib, it has a baremetal element to do so. dib also has a deprecated script to pull a kernel and initramfs from an image.00:01
*** shardy has quit IRC00:02
StevenKlifeless: iff CI is up, do you want to review https://review.openstack.org/#/c/81691/ ?00:02
*** shardy has joined #tripleo00:03
Lotus907efiStevenK: thanks00:04
*** xuhaiwei has joined #tripleo00:20
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/os-refresh-config: Add basic unit tests  https://review.openstack.org/8363300:26
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/os-refresh-config: Add error handler to os-refresh-config  https://review.openstack.org/8363400:26
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/tripleo-heat-templates: WIP: Switch overcloud to software-config  https://review.openstack.org/8166600:27
SpamapSok, 81666 is blocked by 83614 now00:28
lifelessSpamapS: so, ironic00:30
*** marun has quit IRC00:30
*** matsuhashi has joined #tripleo00:31
lifelessStevenK: it is up, but that patch interacts with the ironic one00:37
lifelessStevenK: so I'm going to try to push that a little first00:37
lifelessdib-run-parts Thu Mar 27 19:31:33 UTC 2014 Running /opt/stack/os-config-refresh/post-configure.d/74-swift-proxy00:40
lifelessstop: Unknown instance:00:40
lifelessstart: Job failed to start00:40
lifeless[2014-03-27 19:31:36,240] (os-refresh-config) [ERROR] during post-configure phase. [Command '['dib-run-parts', '/opt/stack/os-config-refresh/post-configure.d']' returned non-zero exit status 1]00:40
lifelessEWUT00:40
lifelessI don't see anyway for that to be caused by the ironic patch00:40
lifeless    fileobj = self.myfileobj = __builtin__.open(filename, mode or 'rb')00:42
lifelessIOError: [Errno 13] Permission denied: '/etc/swift/object.ring.gz'00:42
StevenKBlink00:44
lifelessnotcompute was 192.0.2.4 if you want the logs yourself00:46
lifelessrequested mode was rb00:46
lifeless    gz_file = GzipFile(filename, 'rb')00:46
lifelessthe files there00:46
StevenKAnd readable, I'm guessing00:53
StevenKAnd /etc/swift has the x bit set?00:53
*** derekh has joined #tripleo00:57
SpamapSlifeless: yes00:57
SpamapSlifeless: so it is o-k that it failed lots of stuff?00:57
lifelessStevenK: can't tell, vm is torn down at the end of job00:58
lifelessSpamapS: so if only ironic undercloud failed, I would say yes00:59
lifelessSpamapS: but theres no reason for nova bm overcloud to have failed.00:59
SpamapSIOError: [Errno 13] Permission denied: '/etc/swift/object.ring.gz'00:59
SpamapSwtf?00:59
lifelessSpamapS: welcome to my world01:00
SpamapShaha01:00
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Revert https://review.openstack.org/#/c/83209/3  https://review.openstack.org/8363601:01
SpamapShttp://git.openstack.org/cgit/openstack/swift/commit/?id=4b075c107201e3064418d3e5dc41f0ec5ca02e7d01:01
SpamapSlifeless: merged today01:01
derekhthat swift error ^^ maybe related01:01
derekhlifeless: SpamapS: runing test revert https://review.openstack.org/#/c/83209/301:02
SpamapSoh hahaha01:02
*** dguerri has quit IRC01:02
SpamapSI see now that I just found the same error01:02
SpamapSseriously.. this is .. insane01:03
derekhSpamapS: yup, this will get annoying v.fast , any watch this space https://review.openstack.org/#/c/83636/ I'll be in bed01:03
StevenKHm01:03
lifelessderekh: are other jobs failing in the same way?01:04
SpamapShttps://review.openstack.org/83614 failed that way01:04
lifelessok01:05
derekhlifeless: I looked at 2 and saw the same error, forget which reviews they were for01:05
lifelessso01:05
*** dguerri has joined #tripleo01:05
lifelessSpamapS: I'd like to land the ironic support one then01:05
SpamapSlet's submit the actual upstream revert yes?01:05
lifelesslets wait for derekh's patch01:06
derekhI'm gonna guess the ring file is now being created by root with -rw-------. as its now using NamedTemporaryFile01:08
derekhso now swift can't read it01:08
lifelessyeah01:08
lifelessso thats actually not bad01:08
lifelessif we chown it01:08
SpamapSExactly01:08
SpamapSNot sure if we can chown it at the right time01:09
*** ramishra has joined #tripleo01:09
lifelesselements/swift/os-refresh-config/configure.d/73-swift01:09
lifelesspresumably at the end of that01:10
SpamapSWhy wouldn't this have broken us before?01:10
derekhSpamapS: because the swift user only neads to read the file and root was creating it with whatever umask allowed01:11
SpamapShm01:12
SpamapSok..01:12
derekhyup, that would do it, I remember a similar situation with the keystone pki generation  stuff and they had some kind of a runaskeystone paramater in keystone-manage01:13
SpamapSderekh: I just figure if we have problems, other deployers will run into similar "this is a breaking change" problems.01:13
derekhSpamapS: yup agreed they would01:13
SpamapSI think..01:13
SpamapSthe problem is NamedTemporaryFile is more strict than open('...', 'wb')01:14
SpamapStesting that theory now01:14
*** cwolferh_ has quit IRC01:14
SpamapSthat is it01:15
SpamapSderekh: this is going to break a lot of people I think.01:15
SpamapSBut.. actually I think the right answer _is_ to chmod it.01:16
lifelessindeed01:17
lifelessso01:17
SpamapShttps://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/swift#n28501:17
SpamapSdevstack chown -R's the whole conf dir01:17
lifelessMEGA HAMMER DUDE01:17
lifelessderekh: SpamapS: so - I'd like to land https://review.openstack.org/#/c/81959/01:18
lifelessreasoning - the swift failure is orthogonal01:18
lifelessthis patch has run locally just fine for me01:18
SpamapSI agree that it is out of band01:18
lifelessand getting even the seed up is immediate useful feedback for ironic01:18
*** ProfFalken has quit IRC01:20
*** ProfFalken has joined #tripleo01:21
derekhlifeless: looking, I'd really like to wait for a pass (unless there is something pressing)01:22
lifelessderekh: other conflicting patches :(01:23
derekhlifeless: so its needed for another patch to pass ?01:26
lifelessderekh: there are other patches that have to rebase on top of this, or vice verca01:26
*** eghobo has joined #tripleo01:27
derekhlifeless: ok +2 added but I mainly looked to find possible problems with nova-bm case, i.e. I'm trusting you on the ironic case01:31
Lotus907efiI am trying to run: heat stack-update  undercloud -f $TRIPLEO_ROOT/tripleo-heat-templates/undercloud-vm.yaml  -P  "PowerUserName=$(whoami);AdminToken=${UNDERCLOUD_ADMIN_TOKEN};AdminPassword=${UNDERCLOUD_ADMIN_PASSWORD};GlancePassword=${UNDERCLOUD_GLANCE_PASSWORD};HeatPassword=${UNDERCLOUD_HEAT_PASSWORD};NeutronPassword=${UNDERCLOUD_NEUTRON_PASSWORD};NovaPassword=${UNDERCLOUD_NOVA_PASSWORD};BaremetalArch=${NODE_ARCH};PowerManager=$POWER_MAN01:31
Lotus907efiand it keeps returning an error about ERROR: The Parameter (PowerSSHPrivateKey) was not provided01:32
derekhok, night all01:32
*** derekh has quit IRC01:32
Lotus907efiI am guessing the PowerSSHPrivateKey is the ssh key for the user running the command01:33
Lotus907efibut do I provide a path to the key or the text contents of the key itself?01:33
lifelessLotus907efi: no, its the ssh key to use to ssh back into the host hosting the emulated bm hosts01:34
Lotus907efiyes, but how do I pass the right value into the parameter?01:34
*** ramishra has quit IRC01:34
*** ramishra has joined #tripleo01:34
Lotus907efii.e. is it "PowerSSHPrivateKey=/path/to/ssh/key/file" ?01:35
Lotus907efior do I have to cat the contents of the keyfile into the variable?01:36
lifelessif you look higher up in devtest_undercloud.sh you can see how its set01:36
*** nosnos has joined #tripleo01:37
Lotus907efiok, I think I found it. os-apply-config -m $TE_DATAFILE --key ssh-key --type raw01:38
*** xuhaiwei has quit IRC01:39
*** rpodolyaka has joined #tripleo01:39
Lotus907eficool, the command now works01:40
* Lotus907efi needs to update notes on this process01:40
*** yamahata has joined #tripleo01:43
*** rpodolyaka has quit IRC01:43
Lotus907efilifeless: want me to send the updated direction back to you are are they soon going to be nearly worthless once ironic superceeds nova-baremetal ?01:45
*** xuhaiwei has joined #tripleo01:47
lifelessLotus907efi: improvements are definitely welcome01:51
lifelessSpamapS: +2 but there is a typo https://review.openstack.org/#/c/83634/01:59
Lotus907efilifeless: when I run the nova boot --flavor baremetal --image blah  ------ is blah an image name or an image ID?02:02
lifelessLotus907efi: either02:02
Lotus907efioh, ok02:02
lifelessahha02:04
lifelessNobodyCam: I know why the ironic undercloud job fails in CI02:04
Lotus907efidoes the image have to be known to the seed or the undercloud node?02:05
Lotus907efiI see the image listed when I have sourced the seedrc file but not when I source the undercloudrc file02:05
lifelessLotus907efi: it has to be registed with the glance for the cloud you're deploying it into02:05
Lotus907efihmmm02:05
lifelesse.g. if you're deploying it from the undercloud, load it into the undercloud glance02:05
Lotus907efiok02:06
mordredlifeless: does the ironic undercloud job fail in CI because of bunnyrabbits?02:07
Lotus907efidoes scripts/load-image load an image into the seed by default?02:07
Lotus907efibunnyrabbits ?  :)02:08
Lotus907efiwow, I think my baremetal node may be doing something02:11
lifelessmordred: NobodyCam: devananda: https://bugs.launchpad.net/openstack-ci/+bug/129873102:12
uvirtbotLaunchpad bug 1298731 in openstack-ci "ssh command limits for toci virsh don't match ironic commands" [Undecided,New]02:12
lifelessLotus907efi: it loads it into whatever cloud you have sourced credentials for02:12
Lotus907efiyeah, for some reason I must have sourced the seed creds when I did it first time02:13
*** newell has quit IRC02:14
lifelessNobodyCam: so to fix it, you need to setup a restricted ssh key with the ci_commands file, and then get ironic running against it02:15
openstackgerritA change was merged to openstack/tripleo-incubator: Add Ironic support to incubator.  https://review.openstack.org/8195902:15
*** fandi has joined #tripleo02:18
*** Matt2 has quit IRC02:20
*** morazi has quit IRC02:23
*** jammyjam has quit IRC02:25
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Ensure the swift ring is readable by swift.  https://review.openstack.org/8364502:33
*** fandi has quit IRC02:33
lifelessSpamapS: ^ fingers crossed02:33
*** tchaypo has joined #tripleo02:42
*** ramishra has quit IRC02:43
*** fandi has joined #tripleo02:49
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Completely subsume POWER_MANAGER into testenv  https://review.openstack.org/8169102:50
*** ramishra has joined #tripleo02:53
xuhaiweiI saw this error when running devtest_setup.sh02:54
xuhaiweiCould not find any downloads that satisfy the requirement pip in ./tripleo-incubator/openstack-tools/lib/python2.7/site-packages (from pbr>=0.5.21,<1.0->python-cinderclient)02:54
xuhaiweiwhat's the problem02:54
lifelesspossibly easy-install failing? Possibly you're behind  mirror? The real error will be somewhere in the 100 or so lines preceeding that02:55
xuhaiweiI'm not using mirror02:56
xuhaiweijust followed this http://docs.openstack.org/developer/tripleo-incubator/devtest.html02:56
xuhaiweidoing this devtest_setup.sh --trash-my-machine02:57
lifelesssorry, I mean behind proxy02:58
lifelessanyhow, there will be much more output than that one line, and from the rest we can probably make a reasonable guess about the issue02:58
xuhaiweiprint all of them here?02:59
lifelesspastebin it please03:00
lifelesspleia2: https://blueprints.launchpad.net/tripleo/+spec/ci-mirrors for posterity03:01
*** eghobo has quit IRC03:01
xuhaiweiFetched 95.8 MB in 51s (1,852 kB/s) Extracting templates from packages: 100% Selecting previously unselected package bridge-utils. (Reading database ... 49141 files and directories currently installed.) Unpacking bridge-utils (from .../bridge-utils_1.5-2ubuntu7_amd64.deb) ... Selecting previously unselected package libsigsegv2. Unpacking libsigsegv2 (from .../libsigsegv2_2.9-4ubuntu2_amd64.deb) ... Processing triggers for man-db ..03:04
lifelessStevenK: so - https://review.openstack.org/#/c/79051 - is that ready to roll ?03:04
*** matsuhashi has quit IRC03:04
StevenKxuhaiwei: paste.openstack.org, *not* here03:05
lifelessxuhaiwei: ho - http://paste.openstack.org/ please.03:05
xuhaiweisorry03:05
StevenKlifeless: I'd like to test it harder, but we can do that using CI if you wish.03:06
StevenKlifeless: It may end up confusing people, since seed-route-dev will be right, and seed.route-dev will always be 'virbr0' until we change toci03:06
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Remove swift container sync service  https://review.openstack.org/8262503:07
xuhaiweiI have pasted the log03:07
StevenKxuhaiwei: Now you need to paste the URL for that paste here03:08
lifelessStevenK: transient, but yes.03:08
*** ramishra has quit IRC03:08
xuhaiweihttp://paste.openstack.org/show/74514/03:08
StevenKlifeless: Sure. I say it because the branch isn't a case of "Everything is perfect and we don't need to do more work" :-)03:09
*** matsuhashi has joined #tripleo03:10
lifelessxuhaiwei: is there anything interesting in /tmp/tmpEp7iCA ?03:12
xuhaiweiit ends with this03:13
xuhaiweiException information: Traceback (most recent call last):   File "/home/system/.cache/tripleo/tripleo-incubator/openstack-tools/local/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main     status = self.run(options, args)   File "/home/system/.cache/tripleo/tripleo-incubator/openstack-tools/local/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run     requirement_set.prepare_files(finder, force_ro03:13
xuhaiweirequirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)   File "/home/system/.cache/tripleo/tripleo-incubator/openstack-tools/local/lib/python2.7/site-packages/pip/req.py", line 1176, in prepare_files     raise not_found DistributionNotFound: No distributions at all found for pip in ./tripleo-incubator/openstack-tools/lib/python2.7/site-packages (from pbr>=0.5.21,<1.0->python-cinderclient)03:14
lifelessxuhaiwei: use pastebin please03:15
xuhaiweiok03:15
*** nosnos has quit IRC03:17
openstackgerritA change was merged to openstack/tripleo-image-elements: Add support to deploy-testenv for multiple environments  https://review.openstack.org/8168603:19
xuhaiweiI updated it03:20
xuhaiweihttp://paste.openstack.org/show/74516/03:20
xuhaiweiSorry, I will be off for about 1 hour03:22
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update nova's selinux policies  https://review.openstack.org/8297603:22
lifelessxuhaiwei: thats fine. You could have just pasted the new stuff - it gets a unique url from each paste.03:22
lifelessxuhaiwei:03:23
lifelessCould not fetch URL https://pypi.python.org/simple/pip/: timed out03:23
xuhaiweiyes03:24
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update nova's selinux policies  https://review.openstack.org/8297603:24
lifelessxuhaiwei: thats the problem. Just try again03:24
xuhaiweiok03:24
lifelessok, C time, back in a few hours03:25
*** matsuhashi has quit IRC03:31
*** eghobo has joined #tripleo03:35
*** xuhaiwei has quit IRC03:37
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update glance's selinux policies  https://review.openstack.org/8297803:41
*** Rakesh5 has joined #tripleo03:46
*** ProfFalken has quit IRC03:47
*** ProfFalken has joined #tripleo03:48
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update neutron's selinux policies  https://review.openstack.org/8298003:52
*** matsuhashi has joined #tripleo03:53
*** untriaged-bot has joined #tripleo04:00
untriaged-botUntriaged bugs so far:04:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129846504:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129048804:00
uvirtbotLaunchpad bug 1298465 in tripleo "Baremetal recreation may fool the heat stack." [Undecided,Confirmed]04:00
*** untriaged-bot has quit IRC04:00
uvirtbotLaunchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete]04:00
* tchaypo feels nagged04:00
tchaypoThe first one had status "confirmed", i changed to "triaged" and set priority to medium04:04
tchaypothe second one has status "incomplete"; I'd characterise it as a vague description of a maybe-problem on a system that has been torn down we we can't reproduce04:05
tchaypoI don't know what to do with that one that will make untriaged-bot happy and also accurately represent the state of the issue04:05
*** nosnos has joined #tripleo04:06
StevenKIncomplete is the right status, yes04:06
*** xuhaiwei has joined #tripleo04:06
*** killer_prince has joined #tripleo04:09
StevenKBlah, I'm in dependency hell04:10
StevenKArgh, my notifications have changed to the undisimisble ones04:10
tchayponotifications?04:11
StevenKYeah, things that pop up in the corner, usually can be closed04:11
tchaypooh04:12
tchayposo nothing to do with the dependencies04:12
StevenKNo, they are seperate problems.04:14
*** vkozhukalov has joined #tripleo04:15
tchaypoin other news, I asked iotop to do its thing, and now my laptop is telling me it thinks it can get 4 hours out of the slice instead of 2.75 hours04:15
lifelesstchaypo: powertop or iotop?04:16
tchaypopowertop04:19
lifelesstchaypo: the untriaged bot shouldn't be nagging on the second04:20
lifelesstchaypo: the first I disagree with your assessment - but thanks for making it - I've asked for more info from the reporter04:20
tchaypodisagree in what way?04:21
tchaypoI'm guessing 30% you think I gave it the wrong priority, 70% you think it's got sufficient information to be useful04:22
openstackgerritRichard Su proposed a change to openstack/tripleo-image-elements: Update keystone's selinux policies  https://review.openstack.org/8298104:22
lifelesstchaypo: a) see the report :) b) 70% was right04:25
lifelessinsufficient that is04:26
StevenKlifeless: So the seed patch can't land, it requires setup-seed-vm to be passed a JSON file, so the easiest way to get that is to land the toci change that uses devtest_testenv, and that is blocked by https://review.openstack.org/#/c/83267/04:27
lifelessthere's a reasonable -1 on that patch04:29
lifelessperhaps not to the level jp asks for as other docs cover that04:29
tchaypolifeless: yeah, you're right. I only looked at the fact that it was confirmed and "fixed" it to used triage, didn't stop to think about whether it should even have been confirmed04:29
lifelessbut certainly a hl view04:29
StevenKlifeless: Yes, I was going to ask for your help about that04:29
StevenK"Name of the bridge device to connect to baremetal nodes." ?04:31
lifelessStevenK: see the review04:32
lifelessStevenK: just commented there for you04:32
StevenKOH04:33
StevenKI don't even need it?04:33
StevenKI can just use brbm${NUM}04:33
lifeless:)04:33
*** fandi has quit IRC04:37
adam_ghas anyone else tried to build images on trusty?04:47
StevenKI can run up a test fairly quickly with my local mirror04:48
lifelessadam_g: yes, they hit troubl04:48
lifelessadam_g: I believe it was tchaypo04:48
adam_goh04:48
StevenKtchaypo is the trouble they hit?04:48
lifelessthats one interpretation04:48
StevenKHaha04:48
*** fandi has joined #tripleo04:49
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Pass the baremetal bridge to create-nodes  https://review.openstack.org/8326704:49
StevenKBlah. Forgot to change the topic.04:49
lifelessdoes anything use 'virt_type' nowadays ? (in nodes)04:49
adam_g-bash: /usr/local/bin/os-apply-config: cannot execute binary file: Exec format error04:49
StevenKadam_g: That's amd64 vs i38604:50
lifelessadam_g: amd64 image on an i386 host ?04:50
StevenKlifeless: I added it because NobodyCam asked me to04:50
lifelessStevenK: how long ago ?04:50
adam_gamd64 image built on an amd64 box running on an amd64 host04:50
tchaypoadam_g: I didn't have any issues, I was going to suggest and64 vs i386 but if I say that now I'll just look like I'm copying04:50
adam_give tried building i38604:50
adam_gweird mixed results, sometimes my python environment is fubard, othertimes shared libraries04:51
tchaypoPulling i386 wheels?04:51
*** Shrews has quit IRC04:51
StevenKlifeless: At the sprint04:51
lifelessStevenK: ah, before I jumped on the grenade.04:52
*** Shrews has joined #tripleo04:52
adam_gtchaypo, is that something easy to determine from a dib-seed.log?04:52
lifelesswheels know their own arch04:54
lifelessits in the name04:54
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Delete unused 'virt_type' key from nodes list.  https://review.openstack.org/8365904:54
lifelessadam_g: is it broken right now?04:55
adam_glifeless, i still have a busted seed up and runnign, yes04:55
lifelessadam_g: ok so04:55
lifelessadam_g: use head /usr/local/bin/os-apply-config to find the actual binary it is running04:55
lifelessadam_g: then file on that to find out what its arch is04:55
adam_glifeless, thats the weird thing04:56
lifelessadam_g: one weird thing?04:57
lifelessBrought you by memes R us04:57
adam_ghttp://paste.ubuntu.com/7166603/04:57
StevenKWhat about ls -lh /opt/stack/venvs/os-apply-config/bin/os-apply-config04:57
lifelessadam_g: its a symlink04:57
lifelessadam_g: so, follow the chain :)04:58
StevenKHowever, cat would have, so it's broken04:58
adam_glifeless, yes--the cat is on the link target04:58
adam_gsource04:58
lifelessadam_g: I meant stat it04:58
StevenKadam_g: But you didn't stat the source04:58
lifelesstarget the thing its pointing at04:58
lifelessyou statted the source04:58
adam_ghttp://paste.ubuntu.com/7166617/04:59
lifeless195 bytes05:00
StevenKcat -A05:00
lifelessod it ?05:00
lifelessor that05:00
StevenKI tend to reach for cat -A first, but od would work too05:01
StevenKlifeless: Hm, I thought you said CI was up. Should we fix the topic, then?05:02
lifelessplease05:02
lifelessit is05:02
*** StevenK changes topic to "tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"05:02
adam_glotsa 0s http://paste.ubuntu.com/7166631/05:05
*** rpodolyaka has joined #tripleo05:07
xuhaiweiCould not fetch URL https://pypi.python.org/simple/pip/: timed out Is it a proxy problem?05:08
xuhaiweiMy proxy seem go well05:09
*** nati_ueno has quit IRC05:09
*** tchaypo has quit IRC05:12
lifelessxuhaiwei: it can be, or networking. Or sometimes the pypi CDN has trouble. This is why we recommend the use of a mirror.05:13
lifelessadam_g: so - your filesystem is corrupt.05:13
xuhaiweiwhat is the mirror?05:14
lifelessadam_g: is that a trusty build host ? or just a trusty image?05:14
adam_glifeless, seems that way eh?05:14
lifelessxuhaiwei: it is a copy of (some of) the files needed to install05:14
adam_gtrusty image built on a trusty05:14
adam_gwill try building on another machine later or tomorrow05:14
lifelessadam_g: ok, so whats the bet trusty 'fixed' something.05:14
lifelessadam_g: and exposed a latent race :(. Or just broke it.05:15
lifelessadam_g: file a bug  on dib?05:15
adam_gi will later/tomorrow. late here & wife is needs ride home from work05:15
lifelessciao05:15
*** jamezpolley has joined #tripleo05:19
*** jamezpolley is now known as tchaypo05:20
*** marun has joined #tripleo05:27
lifelesswheee that was some review05:29
lifelesshmm uts05:31
lifelesshttps://review.openstack.org/#/c/83645/ still failed05:31
*** tchaypo has quit IRC05:33
lifelessu+r != g+r05:37
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Ensure the swift ring is readable by swift.  https://review.openstack.org/8364505:37
StevenKHeh05:37
lifelessyeah05:39
StevenKI tend to use the octal codes for that reason, TBH05:40
StevenKLess confusion versus what the current state is, it's "You should have these perms, go do it"05:40
lifelessfair point05:40
openstackgerritlifeless proposed a change to openstack/tripleo-incubator: Update documentation.  https://review.openstack.org/8274105:42
*** killer_prince has quit IRC05:43
StevenKlifeless: I think https://review.openstack.org/#/c/83267/ has passed enough of the CI for a review.05:44
lifelessgreghaynes: https://review.openstack.org/#/c/83296/05:44
*** tchaypo has joined #tripleo05:45
greghaynesNot sure percona and I agree on what statically linking openssl means: http://paste.ubuntu.com/7166774/05:51
greghaynesalso woo for differently named .so's05:53
tchaypoubuntu pastebin? o_O05:53
greghaynespastebuntu05:53
tchaypocue jokes about how the one feature openstack really lacks is PastebinAsAService05:54
greghaynesIf only we could think of a witty name for it05:54
tchaypocirf05:55
tchayposhort for circular file05:55
tchaypoor lenticular05:55
tchaypobecause lenticular clouds are approximately cylindrical05:56
openstackgerritGregory Haynes proposed a change to openstack/tripleo-incubator: Add OVERCLOUD_NOTCOMPUTESCALE support  https://review.openstack.org/8329605:58
greghaynesIs there a non haenous way to fix the .so issue in that paste?05:59
greghaynesI can think of some things im ashamed of that ive done in the past...05:59
tchaypo*heinous05:59
greghaynesty06:00
greghaynesreguarding ln -s06:00
tchaypounfortunately i have no helpful suggestions, just spelling nitpicks06:00
*** tchaypo has quit IRC06:11
*** tchaypo has joined #tripleo06:14
tchaypookay, I'm almost happy with weehat now06:15
*** vkozhukalov has quit IRC06:15
*** vkozhukalov1 has joined #tripleo06:15
tchaypo*weechat06:15
*** rpodolyaka has quit IRC06:17
lifelessgreghaynes: install the libraries ?06:23
lifelessgreghaynes: or are those so's not available ?06:23
greghaynesthey are installed, just wrong versions06:25
greghayneslibssl1.0.0 on ubuntu06:26
greghaynesor at least our cloud images06:26
greghaynesI suspect its actually the write so just naming mismatch but :/06:26
greghayneser, I suspect its actually the right version06:26
*** eghobo has quit IRC06:27
lifelessgreghaynes: didn't we land percona? Is this a follow-up patch ?06:30
greghaynesyes. one moment and all will be made (somewhat more) clear with an incoming review06:31
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Enable Galera clustering  https://review.openstack.org/8367506:33
*** epim has joined #tripleo06:34
greghaynesso, turns out that if you dont name a file in the mysql conf.d *.cnf it doesnt load it. As a result we were running percona but not loading the galera .so.06:34
greghaynes\O/06:34
greghaynesoh wait, that config.d thing was just a local thing ive been battling with, it wasnt commited.06:36
greghaynesAs a result of it (xtradb-cluster.cnf) not being commited though we werent loading the galera .so06:36
*** dividehex has quit IRC06:37
*** epim has quit IRC06:37
openstackgerritGregory Haynes proposed a change to openstack/tripleo-image-elements: Enable Galera clustering  https://review.openstack.org/8367506:38
*** dividehex has joined #tripleo06:42
lifelessgreghaynes: so we expect that to arf?06:46
greghaynesThe patch?06:46
lifelessyes06:46
greghaynesyes06:46
lifelesssymlinks if its really he same version seems ok to me, if a little puke inducing06:47
lifelessgreghaynes: have you filed a bug upstream?06:47
greghaynesactually, I should probably run strings on it to make sure it really is the same version06:47
greghaynesand figure out what they were running when they built06:48
greghaynesI think I found the bug filed. *checks*06:48
*** vkozhukalov1 has quit IRC06:49
greghayneshrm https://bugs.launchpad.net/percona-xtradb-cluster/+bug/121385506:50
uvirtbotLaunchpad bug 1213855 in percona-xtradb-cluster "Binary build is linked against libssl.so.10 and libcrypto.so.10 making it hard to run on e.g. SLES11" [Medium,Fix released]06:50
lifelessreleased eh06:51
lifelesswtf do they call this ?06:51
greghaynes"bug released"06:51
*** ramishra has joined #tripleo06:52
*** ramishra has quit IRC06:52
lifelessLOL06:52
lifelessgreghaynes: see comment 906:53
*** ramishra has joined #tripleo06:53
openstackgerritOm Kumar proposed a change to openstack/diskimage-builder: Fix Grub configurations for Fedora images built on a UEFI host.  https://review.openstack.org/8334206:54
greghaynesYes, I am using the TESTING repo for that reason - wasnt even starting up before without symlink fix06:54
lifelessgreghaynes: oh, so it does work ?06:55
greghaynesI suspect that if I wasnt I would get the crash thats posted above there06:55
greghayneswell, that bug is worked around successfully, yes06:55
lifelessthere is a comment that the fully dynamic build works ok06:55
lifelesswith the symlinks06:55
*** killer_prince has joined #tripleo06:56
greghaynesI just wasnt very happy with the solution06:56
*** rdopieralski has joined #tripleo06:58
openstackgerritA change was merged to openstack/tuskar-ui: Make @handle_errors support any kind of function  https://review.openstack.org/7815307:00
*** jtomasek has joined #tripleo07:13
*** mrunge has joined #tripleo07:17
*** jprovazn has joined #tripleo07:18
*** ramishra has quit IRC07:19
*** matsuhashi has quit IRC07:20
*** matsuhas_ has joined #tripleo07:24
*** rpodolyaka has joined #tripleo07:25
lifelessnuts https://review.openstack.org/#/c/83645/ failed07:29
openstackgerritlifeless proposed a change to openstack/tripleo-image-elements: Ensure the swift ring is readable by swift.  https://review.openstack.org/8364507:32
StevenKHeh, handy07:33
StevenKI was for the BIG hammer approach if that patch set fails07:33
*** funzo_ has joined #tripleo07:38
*** funzo has quit IRC07:40
GheRiveromorning all07:41
tchaypomorning ghe07:41
*** rpodolyaka has quit IRC07:43
openstackgerritA change was merged to openstack/tripleo-image-elements: Ensure the swift ring is readable by swift.  https://review.openstack.org/8364507:44
*** rpodolyaka has joined #tripleo07:45
*** rpodolyaka has quit IRC07:46
*** akuznetsov has joined #tripleo08:02
jprovaznlifeless, Hi, are you around?08:03
*** bauzas has joined #tripleo08:05
*** athomas has joined #tripleo08:05
*** gcha has joined #tripleo08:08
*** vkozhukalov has joined #tripleo08:16
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Making overcloud init work  https://review.openstack.org/8334008:25
*** hashar has joined #tripleo08:26
*** lazy_prince has quit IRC08:29
Gonerihi all, https://review.openstack.org/#/c/82315/ has finally its +1 from Jenkins. Can someone approve the change? :)08:31
*** jcoufal has joined #tripleo08:32
*** jistr has joined #tripleo08:33
*** lazy_prince has joined #tripleo08:34
*** rpodolyaka1 has joined #tripleo08:42
*** viktors has joined #tripleo08:42
rpodolyaka1morning all08:43
*** rpodolyaka1 is now known as rpodolyaka08:43
openstackgerritRyan Moore proposed a change to openstack/tripleo-image-elements: Allow setting of compute configuration options  https://review.openstack.org/8343808:43
*** 77CAADEUV has joined #tripleo08:46
*** rpodolyaka1 has joined #tripleo08:46
*** 77CAADEUV has quit IRC08:47
*** ifarkas has joined #tripleo08:48
*** jistr has quit IRC08:49
*** rpodolyaka1 has quit IRC08:51
*** matsuhas_ has quit IRC08:53
*** hashar has quit IRC08:55
*** hashar has joined #tripleo08:55
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-incubator: WIP - configure a node ephemeral disk  https://review.openstack.org/8277608:57
*** matsuhas_ has joined #tripleo08:57
*** jistr has joined #tripleo08:59
lifelessjprovazn: hi09:00
jprovaznlifeless, one question about this card: https://trello.com/c/DaIs1zxb/82-neutron-ha-redundant-environment09:01
jprovaznlifeless, in Sunnyvalle we were thinking about using A/P with Pacemaker. this is one of options on the card09:02
jprovaznlifeless, my concern is that another possible option is graceful migration of networks between nodes09:03
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-image-elements: Make enabled_apis option in nova.conf configurable  https://review.openstack.org/8335509:04
jprovaznlifeless, which might be waste of time - once HA is done for L3 agent, it wouldn't be required09:04
lifelessjprovazn: thats still A/P right ?09:04
lifelessGheRivero: hey, the slow startup bug - on a call with the guys now; are you instrumenting things a little per my suggestions?09:05
jprovaznlifeless, you mean migration? yes - some kind of graceful A/P09:05
GheRiverolifeless: sorry the delay. I had a small kitchen accident yesterday. Nothing serious but my left hand fingers will be a little unconfortable in the keyboard a couples of days.09:07
jprovaznlifeless, so having solution which supportes no-graceful situations would be nicer09:07
GheRiverolifeless: http://paste.openstack.org/show/74527/09:07
GheRiveroThat's what I have so far from the devtest host09:08
*** pblaho has joined #tripleo09:08
*** derekh has joined #tripleo09:09
*** e0ne has joined #tripleo09:12
*** xuhaiwei has quit IRC09:13
*** yassine has joined #tripleo09:13
derekhlifeless: You fix for the rin file seems to have had the desired effect, we are now getting further but now having problems pinging the user image09:13
derekhthis also happened on my test revert before I left, so we have a new probelm somewhere09:14
derekhfrom nova-api.,log http://paste.fedoraproject.org/89452/98068139/  related ?09:15
openstackgerritjan grant proposed a change to openstack/diskimage-builder: Parameterise PXE kernel and initrd selection  https://review.openstack.org/8339009:16
lifelessderekh: should we land my fix?09:17
lifelessderekh: regardless?09:17
derekhlifeless: looks like we already did09:17
*** d0ugal has quit IRC09:19
lsmola_lifeless: the change looked alright to me so I approved09:19
*** d0ugal has joined #tripleo09:19
*** d0ugal has quit IRC09:19
*** d0ugal has joined #tripleo09:19
lsmola_lifeless: I am observing new patches, but there is 99% chance it works :-D09:19
derekhI think we would have merged it anyways so we can debug the new problem09:21
lsmola_derekh: yeah it's not like merging it could break anything :-)09:21
*** e0ne has quit IRC09:22
*** jcoufal has quit IRC09:25
*** jcoufal has joined #tripleo09:26
*** giulivo has joined #tripleo09:26
*** akrivoka has joined #tripleo09:28
*** lucasagomes has joined #tripleo09:28
*** e0ne has joined #tripleo09:28
*** e0ne_ has joined #tripleo09:28
derekhhmm neutron-server.log09:29
derekh2014-03-28 08:49:09.768 5431 ERROR neutron.notifiers.nova [-] Failed to notify nova on events: [{'status': 'completed', 'tag': u'a51654ca-cab8-4013-93b5-260a85c4f154', 'name': 'network-vif-plugged', 'server_uuid': u'ec8fd597-d5c0-4e3f-8cc7-e0672c9827e4'}]09:29
*** markmc has joined #tripleo09:31
openstackgerritTom Hancock proposed a change to openstack/tripleo-incubator: Make ELEMENTS_PATH user-configurable  https://review.openstack.org/8369509:32
*** e0ne has quit IRC09:32
lifelessderekh: thats unrelated - its the not yet fixed thing to fix races09:33
derekhlifeless: k09:33
lifelessderekh: we were discussing with arosen how to make that work, given we need a tenant name not id.09:33
lifelessderekh: neutron-metadata-agent would be the place to look I think, but its 2233 and i have a cold coming on so I'm going to do one last review and bail09:34
derekhlifeless: ok, I should have recognized it09:34
derekhlifeless: ok, I'll keep poking around09:34
*** jcoufal has quit IRC09:37
*** jcoufal has joined #tripleo09:37
lifelessgnight all09:40
*** jp_at_hp has joined #tripleo09:40
*** funzo_ has quit IRC09:43
*** funzo has joined #tripleo09:46
*** rpodolyaka1 has joined #tripleo09:47
lsmola_lifeless: good night09:48
*** rpodolyaka1 has quit IRC09:52
*** andreaf has quit IRC09:58
*** untriaged-bot has joined #tripleo10:00
untriaged-botUntriaged bugs so far:10:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129048810:00
uvirtbotLaunchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete]10:00
*** untriaged-bot has quit IRC10:00
*** matsuhas_ has quit IRC10:00
jp_at_hplifeless: are you still around?  I wanted to chat about https://review.openstack.org/#/c/82807/ to cover it quickly rather than bouncing review comments back and forth for a week...10:00
StevenKjp_at_hp: You missed him by 20 minutes, I'm afraid.10:01
jp_at_hp:( boo.10:01
jp_at_hpThanks SteveK10:01
jp_at_hpHmmm, maybe that shoud be thanks StevenK...10:02
jp_at_hpActually - since you are here StevenK - I'd love to chat about what's changing with the testenv.json, and the various condfigurationfiles devtest uses...10:02
StevenKjp_at_hp: It's 9pm and I'm about to disappear for dinner in about 3 minutes, so it will have to be really quick.10:03
jp_at_hpOK, so I guess given that, is there a design or writeup of end-goal or next steps anywhere so that I can help out, given it's something I'm very interested in?10:04
jp_at_hpAnd if not, is either https://etherpad.openstack.org/p/tripleo-devtest.sh-refactoring-blueprint or https://etherpad.openstack.org/p/tripleo-incubator-rationalise-ui an appropriate place to add things so that we can keep in sync?10:05
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Making overcloud init work  https://review.openstack.org/8334010:06
killer_princeCan anyone point me to the work being done related to UEFI in ironic and where can I find more details related to it.10:06
StevenKjp_at_hp: End goal is moving things into os-cloud-config and then calling that from devtest. Currently, I'm working on getting toci using devtest_testenv, and then pulling the seed details into the JSON.10:07
jp_at_hpCool, thanks StevenK - is there anything written down around os-cloud-config anywhere, or just a README in the code?10:08
StevenKjp_at_hp: So it will be things like setup-baremetal and registering nodes that will move across -- and will be refactored into Python and tested as well.10:09
jp_at_hpThat's great - but are the goals of os-cloud-config listed in any meeting minutes, or blueprint, or anything like that so that I can read up on it while you're offline having dinner (which I hope I'm not making you late for - but this is really useful info for me)10:11
StevenKjp_at_hp: I'm not certain about that bit, sorry.10:12
dtantsurslagle, hi. Does instack create configuration files for openstack services, if yes, where and how? I'm asking because after reboot, neither keystone, nor heat could find their configuration files; glance also has some problems10:12
*** yamahata has quit IRC10:12
jp_at_hp@StevenK - thanks!  I'll see if anyone does...10:13
dtantsurslagle, e.g. keystone is expecting /etc/keystone-paste.ini10:13
*** nosnos has quit IRC10:16
*** e0ne_ has quit IRC10:16
*** e0ne has joined #tripleo10:23
*** andreaf has joined #tripleo10:23
*** funzo has quit IRC10:28
*** Matt3 has joined #tripleo10:29
*** ProfFalken has quit IRC10:34
*** ProfFalken has joined #tripleo10:34
*** funzo has joined #tripleo10:34
*** jcoufal has quit IRC10:34
*** jcoufal has joined #tripleo10:35
janganyone here - missing rabbit in a brand new overcloud image. Anyone know the culprit commit?10:36
openstackgerritA change was merged to openstack/python-tuskarclient: Add API binding for the template_parameters action  https://review.openstack.org/7763710:37
openstackgerritA change was merged to openstack/python-tuskarclient: Add CLI support for the template_parameters action  https://review.openstack.org/7764910:37
tteggeljang: looking. http://goodsquishy.com/downloads/tripleo-jobs.html may be of use to narrow time time10:41
jprovaznjang, did you run devtest when building overcloud image or did you build the image "manually"?10:47
openstackgerritAlexis Lee proposed a change to openstack/tripleo-image-elements: Allow configuration of extended logging  https://review.openstack.org/8335310:48
jprovaznjang, because when building an image, you have to explicitly include rabbitmq (or qpid) - for devtest_overcloud.sh it's done by:10:48
jprovaznOVERCLOUD_CONTROL_DIB_EXTRA_ARGS=${OVERCLOUD_CONTROL_DIB_EXTRA_ARGS:-'rabbitmq-server'}10:48
*** CaptTofu has quit IRC10:52
*** CaptTofu has joined #tripleo10:52
*** funzo_ has joined #tripleo10:54
*** funzo has quit IRC10:56
*** CaptTofu has quit IRC10:56
*** marios_ has quit IRC11:02
*** marios has joined #tripleo11:03
jangah, right.11:05
jangthanks.11:05
jangI did it with devtest, but I need to couble-check what defaults devtest_variables might be inserting.11:06
jangah-ha, git it, thanks. Polluted shell env var meant that rabbit wasn't making its way into the build.11:11
jangon the subject of recent work: /etc/swift/*.gz files appear to have the wrong file permissions. Also something from last night?11:12
openstackgerritSascha Peilicke proposed a change to openstack/tuskar-ui: Honor user pip configuration  https://review.openstack.org/8371411:12
*** marios has quit IRC11:16
*** marios has joined #tripleo11:16
*** morganfainberg is now known as morganfainberg_Z11:20
*** julim has joined #tripleo11:21
*** julim has quit IRC11:22
*** julim has joined #tripleo11:31
slagledtantsur: it does11:32
*** Shrews has quit IRC11:33
tteggeldoes anyone know why overcloud ci is failing? local devtest run is fine.11:33
*** Shrews has joined #tripleo11:33
slagledtantsur: there is no /etc/keystone-paste.ini though. why do you think it's needed? is there something in the logs?11:34
openstackgerritjan grant proposed a change to openstack/diskimage-builder: Parameterise PXE kernel and initrd selection  https://review.openstack.org/8339011:34
*** marios has quit IRC11:36
*** marios has joined #tripleo11:37
derekhFIREDRILL ci-overcloud jobs failing | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt11:40
derekhci overcloud failing, No idea why, userimage is booting but can't be pinged, anybody seeing this locally?11:41
GheRiverobuilding a full tripleo stack11:44
*** mestery has joined #tripleo11:47
openstackgerritJames Slagle proposed a change to openstack/diskimage-builder: Make $GIT_MANIFEST configurable  https://review.openstack.org/8372011:47
*** rbrady1 is now known as rbrady11:47
jp_at_hpderekh: any chance you could look over https://review.openstack.org/#/c/80617/ please?11:58
jp_at_hpAny other core reviewers able to take a look at https://review.openstack.org/#/c/80617/ please?  It's been in limbo for a little while...11:59
*** akrivoka has quit IRC11:59
derekhjp_at_hp: will do, just gotta figure out busted CI first, then will giver it a look over11:59
*** akrivoka has joined #tripleo12:00
*** dguerri is now known as _dguerri12:01
*** fandi has quit IRC12:04
*** weshay has quit IRC12:06
*** weshay has joined #tripleo12:06
*** marios has quit IRC12:06
*** mrunge has quit IRC12:08
*** marios has joined #tripleo12:10
*** weshay has quit IRC12:10
jp_at_hpta muchly derekh12:11
*** weshay has joined #tripleo12:12
jp_at_hpslagle: I like the idea of allowing people to specify a specific name for the manifests, or at least a specific place to collect them outside of the image.  Did you see the notes in https://etherpad.openstack.org/p/tripleo-incubator-rationalise-ui?12:12
*** hewbrocca has joined #tripleo12:13
slaglejp_at_hp: i have not. will check it out12:13
slaglewas just going for something simple in the short term :)12:13
*** morazi has joined #tripleo12:14
dtantsurslagle, in logs I only head like "keystone cannot find keystone-paste.ini, and I just guessed it should go to /etc12:16
dtantsur* head = had12:17
*** killer_prince has quit IRC12:18
*** lazy_prince is now known as killer_prince12:18
jp_at_hpslagle: simple is good - it usually means I can understand it :D12:19
slagledtantsur: maybe pastebin me the logs and what commands you ran?12:22
*** mestery has quit IRC12:22
dtantsurslagle, I didn't ran anything, just set up instack, after sometime rebooted and got: keystone-all[2573]: 2014-03-28 11:05:24.356 2573 CRITICAL keystone [-] ConfigFileNotFound: The Keystone configuration file keystone-paste.ini could not be found.12:23
dtantsurthat's the only interesting thing in logs, I'm afraid12:23
dtantsuror what logs do you need?12:23
*** lucasagomes is now known as lucas-hungry12:26
derekhGheRivero yours is still building ? user instance has its IP, is responding to arp requests but icmp isn't reacing its tap device, still digging12:28
GheRiveroIt finished properly, but was built with --offline, building another one without it just in case12:34
slagledtantsur: what did you do to set it up?12:36
*** decede has joined #tripleo12:37
dtantsurslagle, I think I exactly followed this one: https://github.com/agroup/instack-undercloud/blob/master/README-virt.md I don't remember doing anything outstanding that could affect startup12:37
*** jp_at_hp has quit IRC12:40
slagledtantsur: send me ~/.instack/install-undercloud.log12:40
dtantsurslagle, https://dl.dropboxusercontent.com/u/1730743/install-undercloud.log12:42
*** hashar has quit IRC12:46
*** akrivoka_ has joined #tripleo12:48
*** akrivoka has quit IRC12:51
*** lblanchard has joined #tripleo12:51
*** akrivoka_ has quit IRC12:52
*** akrivoka_ has joined #tripleo12:53
*** rlandy has joined #tripleo12:54
*** jdob has joined #tripleo12:56
*** akrivoka_ has quit IRC12:58
*** akrivoka has joined #tripleo12:58
openstackgerritLadislav Smola proposed a change to openstack/tuskar-ui: Making overcloud init work  https://review.openstack.org/8334012:59
*** dprince has joined #tripleo13:05
openstackgerritDerek Higgins proposed a change to openstack/tripleo-incubator: ssh to user instance  https://review.openstack.org/8373613:06
derekhssh seems to work and ping doesn't, lets see if ^^ works13:06
openstackgerritA change was merged to openstack/tripleo-image-elements: Adds mariadb element  https://review.openstack.org/7988413:06
derekhGheRivero: can you let me know if you see that behavior?13:07
openstackgerrittherese-mchale proposed a change to openstack/tripleo-image-elements: Secure the mysql installation  https://review.openstack.org/8057013:09
*** funzo_ is now known as funzo13:12
*** zigo has quit IRC13:13
*** zigo has joined #tripleo13:14
*** CaptTofu has joined #tripleo13:15
openstackgerritDerek Higgins proposed a change to openstack-infra/tripleo-ci: Revert https://review.openstack.org/#/c/83391/  https://review.openstack.org/8363613:15
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Stop using all ports option w/ neutron-ovs-cleanup  https://review.openstack.org/6993713:21
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Refactor ensure-bridge for persistance  https://review.openstack.org/6991813:21
*** matty_dubs|gone is now known as matty_dubs13:27
*** jprovazn has quit IRC13:30
*** Rakesh5 has quit IRC13:32
openstackgerritAlexis Lee proposed a change to openstack/tripleo-image-elements: Add logstash-{agent,solo} elements  https://review.openstack.org/8335413:41
openstackgerritAlexis Lee proposed a change to openstack/tripleo-heat-templates: Add logstash HOT  https://review.openstack.org/8342213:42
*** lucas-hungry is now known as lucasagomes13:43
openstackgerritAlexis Lee proposed a change to openstack/tripleo-heat-templates: Add logstash HOT  https://review.openstack.org/8342213:45
openstackgerritStuart McLaren proposed a change to openstack/diskimage-builder: apt-conf: uninitialised variables fix  https://review.openstack.org/8374613:45
*** noslzzp has joined #tripleo13:46
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: TEST ONLY: make nova depend on common-venv  https://review.openstack.org/7998913:46
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Openstack-clients: don't hard code venv  https://review.openstack.org/7998813:46
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Wire in _EXTRA_INSTALL_OPTS...  https://review.openstack.org/7696613:46
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Add a new common-venv element  https://review.openstack.org/7696713:46
openstackgerritDan Prince proposed a change to openstack/tripleo-image-elements: Horizon: dynamically set config time env vars  https://review.openstack.org/8261113:46
*** zigo has quit IRC13:47
*** zigo has joined #tripleo13:47
*** zigo has quit IRC13:51
openstackgerritAlexis Lee proposed a change to openstack/tripleo-image-elements: Add logstash-{agent,solo} elements  https://review.openstack.org/8335413:52
lsmola_rwsu: hello13:53
lsmola_rwsu: are you around?13:54
*** zigo has joined #tripleo13:55
*** jistr has quit IRC13:57
*** e0ne has quit IRC13:58
*** jp_at_hp has joined #tripleo13:58
*** zigo has quit IRC13:59
*** zigo has joined #tripleo13:59
*** CaptTofu has quit IRC13:59
*** jistr has joined #tripleo13:59
*** CaptTofu has joined #tripleo14:00
*** CaptTofu has quit IRC14:00
Ngmorning14:06
*** derekh changes topic to "FIREDRILL ci-overcloud jobs failing | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"14:08
*** slagle has quit IRC14:13
*** slagle has joined #tripleo14:15
*** openstack has joined #tripleo14:21
*** jistr has joined #tripleo14:21
*** jtomasek has quit IRC14:24
*** bnemec is now known as beekneemech14:25
*** derekh changes topic to "FIREDRILL ci-overcloud jobs failing https://bugs.launchpad.net/tripleo/+bug/1299022 | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"14:27
rwsulsmola_: hi14:28
lsmola_rwsu: hello14:28
lsmola_rwsu: you were the one who pushed the separate swift node right? :-)14:28
lsmola_rwsu: do you have somewhere the right code for generating the swift image?14:29
lsmola_rwsu: I would like to test it14:29
openstackgerritDerek Higgins proposed a change to openstack/tripleo-incubator: ssh to user instance  https://review.openstack.org/8373614:32
rwsulsmola_: yes, you just need to add swift-storage to the image you want to use, something like http://paste.openstack.org/show/74553/14:36
* rwsu reboots14:38
*** rwsu has quit IRC14:38
*** e0ne has joined #tripleo14:38
*** rwsu has joined #tripleo14:41
*** e0ne has quit IRC14:43
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: Make it possible to pass real hardware in.  https://review.openstack.org/8318814:56
jp_at_hpAre there any cores that can add a second +2 to this please? https://review.openstack.org/#/c/83171/  I need to start writing a following patch, and would prefer that it is merged...14:57
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: SSH key and virtual_power_driver not used on H/W  https://review.openstack.org/8377014:57
*** Matt3 has quit IRC14:57
*** jtomasek has joined #tripleo15:01
derekhGot a solution for devtest_overcloud change15:02
derekhneutron security-group-rule-create default --protocol icmp --direction ingress --port-range-min 8 --port-range-max 815:02
derekhto15:02
derekhneutron security-group-rule-create --protocol icmp --direction ingress default15:02
derekhwill send a patch in a few minutes once I figure out for sure why the port numbers are in there15:02
*** thouveng has quit IRC15:03
*** CaptTofu has joined #tripleo15:07
*** yamahata has joined #tripleo15:10
*** e0ne has joined #tripleo15:12
openstackgerritAlexis Lee proposed a change to openstack/diskimage-builder: Add ability to upload a new GPG into the apt keyring for the image  https://review.openstack.org/8359615:14
lsmola_rwsu: thanks, will try that15:15
*** rlandy has quit IRC15:17
*** Matt2 has joined #tripleo15:20
openstackgerritAlexis Lee proposed a change to openstack/diskimage-builder: Add ability to add extra apt keys  https://review.openstack.org/8359615:20
*** sgrasley_ has quit IRC15:23
*** hashar has joined #tripleo15:26
*** sgrasley has joined #tripleo15:30
*** AaronGr has left #tripleo15:33
openstackgerritDerek Higgins proposed a change to openstack/tripleo-incubator: Only set port-range-min in icmp security rule  https://review.openstack.org/8373615:33
openstackgerritDerek Higgins proposed a change to openstack/tripleo-incubator: Only set port-range-min in icmp security rule  https://review.openstack.org/8373615:35
*** noslzzp has quit IRC15:35
GheRiverosome data about tripleo times with different params https://docs.google.com/spreadsheets/d/1-e6aIv9RCeDR5QKQuwNOt1qVhDsKGv0HtICZmKCVyis/edit?usp=sharing15:39
*** eghobo has joined #tripleo15:41
openstackgerritjan grant proposed a change to openstack/diskimage-builder: Parameterise PXE kernel and initrd selection  https://review.openstack.org/8339015:43
derekhGheRivero: just incase your still looking I think  https://review.openstack.org/83736 should solve the problem15:49
openstackgerritAndrea Rosa proposed a change to openstack/tripleo-image-elements: Allow settings for Nova API  https://review.openstack.org/8379415:53
*** nati_ueno has joined #tripleo15:54
openstackgerritAndrea Rosa proposed a change to openstack/tripleo-image-elements: Allow settings for Nova API  https://review.openstack.org/8379415:55
*** CaptTofu has quit IRC15:55
*** akuznetsov has quit IRC15:56
*** jcoufal has quit IRC15:57
*** cwolferh_ has joined #tripleo15:59
*** untriaged-bot has joined #tripleo16:00
untriaged-botUntriaged bugs so far:16:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129048816:00
uvirtbotLaunchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete]16:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129894916:00
uvirtbotLaunchpad bug 1298949 in tripleo "tripleorc is not available to debug a failed run" [Undecided,New]16:00
*** untriaged-bot has quit IRC16:00
sgrasleystack16:00
*** sgrasley has quit IRC16:01
*** athomas has quit IRC16:01
*** rdopieralski has quit IRC16:02
jp_at_hpslagle: on the git manifest naming - I take it you are running d-i-b without an image name then?  And that if you ran it with a specific image name you would get a better name out?  I want to learn a little more about your use-case to understand what you are seeing...16:05
*** jogo is now known as flashgordon16:07
*** matty_dubs is now known as matty_dubs|lunch16:09
Gonerihi, can a core put a final +2 here: https://review.openstack.org/#/c/82315/ there is already two +2 and a +116:14
*** bauzas has quit IRC16:17
*** epim has joined #tripleo16:18
devanandamorning, all16:20
openstackgerritAlexis Lee proposed a change to openstack/diskimage-builder: Add ability to add extra apt keys  https://review.openstack.org/8359616:22
slaglejp_at_hp: i am using -o. I get a git manifest named like this: .cache/image-create/source-repositories/dib-git-manifest-fedora_L664agm016:23
openstackgerritA change was merged to openstack/tripleo-incubator: doc: ReST syntax changes to improve readability  https://review.openstack.org/8231516:24
*** nati_ueno has quit IRC16:27
Goneriderekh: thanks16:27
derekhGoneri: np16:27
slaglejp_at_hp: oh i see now that gets mv'd at the end :)16:28
slaglejp_at_hp: still...i want my own path instead of manifests littering my git tree16:30
slaglei often use dib from a checkout16:30
slaglejp_at_hp: they're meant to be vauable artificats right? what's the objection to choosing where they get saved?16:30
jp_at_hpslagle: yes - I'm about to make the change to put all manifests into a single directory in $TRIPLEO_ROOT as per the etherpad16:31
slaglejp_at_hp: sure...i don't have a $TRIPLEO_ROOT when i use diskimage-builder16:31
slagleit's standalone software too16:31
jp_at_hpAnd then a single variable naming that dir would get you the git + pip + dpkg manifests (which you woun't have) in a single place...16:31
jp_at_hpslagle: sure - I get you, but there are more than just git manifests16:32
jp_at_hpso I think your change should be a var to name the manifests save directory, not just modifying individual manifests piecemeal...16:33
slaglei *guess* defining a $TRIPLEO_ROOT would be sufficient, one env var is as good as another, but the naming is bad if you're not using tripleo-incubator16:33
slagleit makes no sense in a pure dib context16:34
jp_at_hpslagle: sorry - I'm not sure I'm saying the right things, as I think there is still some confusion between us :(16:34
jp_at_hpSo.  Pure dib context.  I am about to make a change that will...  create a /etc/dib-manifests directory in the image16:35
*** vkozhukalov has quit IRC16:35
jp_at_hpcopy all manifests generated (currently git, pip* and dpkg*) into /etc/dib-manifests in the image (* == review in-flight)16:36
slagleok, that sounds good so far16:36
jp_at_hpadd a new element (manifests) which will copy the /etc/dib-manifests directory to $(dirname $IMAGE_NAME)/dib-manifests-$IMAGE_NAME16:37
derekhjp_at_hp: review done, just a small typo16:38
slaglejp_at_hp: ok, so i'd like $(dirname $IMAGE_NAME) there to be configurable16:38
jp_at_hpslagle: Yes!!  That's what I was trying to get to :D16:39
slaglejp_at_hp: ok, i think we're on the same page now :)16:39
jp_at_hpCool :D16:40
slaglejp_at_hp: so, i'll happily abandon what i've put up :)16:40
slaglefeel free to point me at your reviews when you've got them submitted16:40
jp_at_hpSo yes - I think you're review in the context you knew about was great...  I was just trying to tell you what was coming, as soon as I write it now...16:40
jp_at_hpslagle: will do, thanks.16:40
*** lucasagomes has quit IRC16:41
openstackgerritMichael Kerrin proposed a change to openstack/tripleo-incubator: Configure a root disk & ephemeral disk for baremetal nodes  https://review.openstack.org/8277616:42
openstackgerritJon-Paul Sullivan proposed a change to openstack/tripleo-incubator: Allow set-source-vars to consume a git manifest  https://review.openstack.org/8061716:43
*** akuznetsov has joined #tripleo16:43
jp_at_hpderekh: https://review.openstack.org/#/c/80617/ is updated.  Just gotta wait for CI now :(16:43
*** lucasagomes has joined #tripleo16:43
*** hashar has quit IRC16:45
derekhjp_at_hp: hopefully it will be back functional in about 30 minutes16:46
jp_at_hpderekh: great work looking after the ci jobs, thanks.  I know it isn't easy16:47
SpamapSo/16:48
pleia2derekh: hey, we have f20s! have we tried anything with them yet? :)16:48
derekhjp_at_hp: thanks, seems to be a never ending roller coaster16:48
derekhpleia2: there waiting on jenkins the jobs to be added https://review.openstack.org/#/q/status:open+project:openstack-infra/config+branch:master+topic:add-f20-jobs,n,z16:48
pleia2derekh: thanks, having a look now16:49
*** akuznetsov has quit IRC16:50
openstackgerritRyan Moore proposed a change to openstack/tripleo-image-elements: Restructure the nova.conf to match documentation  https://review.openstack.org/8382116:54
*** akuznetsov has joined #tripleo16:56
*** matty_dubs|lunch is now known as matty_dubs16:58
*** akuznetsov has quit IRC17:00
*** epim has quit IRC17:01
*** newell has joined #tripleo17:02
*** noslzzp has joined #tripleo17:05
*** akuznetsov has joined #tripleo17:08
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: SSH key and virtual_power_driver not used on H/W  https://review.openstack.org/8377017:09
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: Make it possible to pass real hardware in.  https://review.openstack.org/8318817:09
*** pblaho has quit IRC17:09
* derekh finally got around to building a local pypi mirror17:10
*** nati_ueno has joined #tripleo17:11
beekneemechderekh: Me too.  It cut about nine minutes off my devtest runs. :-)17:11
beekneemechNow I just need a local fedora mirror and traffic will never have to leave my house for image builds. ;-)17:12
derekhbeekneemech: ya, I got about 12 minutes (sample set of 1), I was expecting something better...17:13
derekhbeekneemech: I think the rpm's should be getting cached locally17:14
derekhbeekneemech: but, if your looking to use squid with yum repo's17:14
beekneemechYeah, that's true.  I'm probably not hitting the external yum repos much anyway.17:15
derekhbeekneemech: and find it usless because each requests goes to a different mirror, so no benifit17:15
derekhbeekneemech: try https://github.com/derekhiggins/scripts/blob/master/squid/yum_mirror_rewriter17:15
derekhbeekneemech: rewrites all yum requests to a single mirror so the cache is useful17:16
*** ifarkas has quit IRC17:16
beekneemechderekh: Oh, nice!  I've just been overriding the yum repo in the image, but having squid do it automatically would be sweet.17:17
*** e0ne has quit IRC17:18
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/os-apply-config: Ignore non-mapping json when merging.  https://review.openstack.org/8383117:18
*** CaptTofu has joined #tripleo17:22
*** ProfFalken has quit IRC17:23
*** epim has joined #tripleo17:23
*** ProfFalken has joined #tripleo17:24
*** epim_ has joined #tripleo17:27
*** _dguerri has quit IRC17:28
*** ProfFalken has quit IRC17:28
*** epim has quit IRC17:28
*** epim_ is now known as epim17:28
*** Kiall has quit IRC17:29
*** Shrews has quit IRC17:29
openstackgerritNicholas Randon proposed a change to openstack/tripleo-incubator: SSH key and virtual_power_driver not used on H/W  https://review.openstack.org/8377017:32
*** akrivoka has quit IRC17:32
openstackgerritClint "SpamapS" Byrum proposed a change to openstack/os-collect-config: Restore 'deployments' key to cfn Metadata  https://review.openstack.org/8383717:32
openstackgerritGonéri Le Bouder proposed a change to openstack/diskimage-builder: fix grub2 installation on Debian Wheezy  https://review.openstack.org/8350617:33
*** jistr has quit IRC17:34
*** stevehuang has joined #tripleo17:38
*** morganfainberg_Z is now known as morganfainberg17:39
*** markmc has quit IRC17:42
*** giulivo has quit IRC17:42
*** akuznetsov has quit IRC17:48
derekhok, that passed the overcloud job https://review.openstack.org/#/c/83736/17:49
derekhit failed the undercloud ironic for something I assume is unrelated as I only touched devtest_overcloud17:50
derekhmerge  to unblock CI ?17:50
derekhNg: SpamapS slagle GheRivero lifeless ^17:52
SpamapSderekh: we've been treating ironic undercloud as non voting because there are some unresolved issues there17:53
derekhlifeless: I'm guessing you still have something to do to fix the ironic undercloud job?17:53
derekhSpamapS: ahh ok, cool17:53
SpamapSderekh: so yes, full steam ahead17:54
SpamapSboom17:55
Ngboom!17:55
SpamapS+A's17:55
NgHEAD shot17:55
derekhSpamapS: Ng thanks17:55
openstackgerritA change was merged to openstack/tripleo-incubator: Only set port-range-min in icmp security rule  https://review.openstack.org/8373617:55
Ngone in the HEAD, two in the trunk17:55
derekhOk, I'm outta here, have a good weekend17:56
*** derekh has quit IRC17:56
*** mkerrin has quit IRC18:02
*** lucasagomes has left #tripleo18:09
openstackgerritJay Dobies proposed a change to openstack/python-tuskarclient: Changed role count assignment to use role name  https://review.openstack.org/8005218:10
openstackgerritJon-Paul Sullivan proposed a change to openstack/diskimage-builder: Standardise manifest creation and retrieval  https://review.openstack.org/8384818:12
openstackgerritJon-Paul Sullivan proposed a change to openstack/diskimage-builder: Write a dpkg manifest to list installed packages  https://review.openstack.org/8317118:12
jp_at_hpslagle: You will want to review this one: https://review.openstack.org/#/c/83848/18:14
openstackgerritJon-Paul Sullivan proposed a change to openstack/tripleo-image-elements: Standardise manifest creation and retrieval  https://review.openstack.org/8385018:14
*** jp_at_hp has quit IRC18:16
*** Guest77682 has joined #tripleo18:19
*** andreaf has quit IRC18:20
*** vkozhukalov has joined #tripleo18:30
*** morazi has quit IRC18:36
*** morazi has joined #tripleo18:38
SpamapSshadower: thanks for the feedback on the change to softwareconfig templates btw. Very useful.18:46
*** openstackgerrit has quit IRC18:48
*** openstackgerrit has joined #tripleo18:48
*** dguerri has joined #tripleo18:50
*** kiall has joined #tripleo18:52
*** ProfFalken has joined #tripleo18:54
*** shakayumi has joined #tripleo18:58
*** jtomasek has quit IRC19:02
*** jang1 has joined #tripleo19:02
*** e0ne has joined #tripleo19:07
SpamapSdprince: did you get anywhere with the ovs mac weirdness?19:12
SpamapSdprince: I got distracted away from it a lot19:12
dprinceSpamapS: I got it.19:12
dprinceSpamapS: Will send a patch in a second...19:13
dprinceSpamapS: Problem was libvirt actually. It automatically sets the 2nd MSB of a MAC high... making it a locally administered MAC address.19:14
*** CaptTofu has quit IRC19:14
*** CaptTofu has joined #tripleo19:14
dprinceSpamapS: My solution is to fix configure-vm so we set our own MAC addresses for the bare metal network...19:15
dprinceSpamapS: so that they more accurately reflect real hardware...19:16
SpamapSdprince: OH!19:16
SpamapSdprince: that all makes sense actually...19:16
SpamapScasuallly you want a locally administered MAC19:16
SpamapSbut when emulating real servers.. not so much :)19:17
dprinceSpamapS: If I hadn't developed this on real baremetal (my local machines) I would have realized this sooner perhaps.19:17
dprinceSpamapS: anyways I knew it worked... this is just a test environment problem19:17
SpamapSdprince: well there's yer problem :)19:17
SpamapS"Success is a poor teacher. -- Bill Gates"19:18
dprinceSpamapS: well, if everyone did things the same way we very well may have conveined on a different solution here as well :)19:18
*** CaptTofu has quit IRC19:19
*** CaptTofu has joined #tripleo19:20
*** CaptTofu has quit IRC19:20
openstackgerritDan Prince proposed a change to openstack/tripleo-incubator: Generate better bm network mac addresses for VMs  https://review.openstack.org/8386719:28
dprinceSpamapS: ^^19:28
dprinceSpamapS: one that is in we'll just need to rebuild *all* the test-env workers on the undercloud... and then my patch should work :)19:29
dprinceSpamapS: This will also make it so the brbm MAC might change on the host machines though... so I'm not entirely sure it won't break something subtle. But I don't think we rely on that for anything ATM19:31
dprinceSpamapS: in any case this is better testing so I think whatever we need to do to make this happen should happen19:32
*** jcoufal has joined #tripleo19:36
*** jcoufal has quit IRC19:38
openstackgerritSteven DuChene proposed a change to openstack/diskimage-builder: Fixed test to see if DIB_ADD_APT_KEY variable was set in 01-copy-apt-keys  https://review.openstack.org/8387719:51
openstackgerritSteven DuChene proposed a change to openstack/diskimage-builder: Add ability to add extra apt keys  https://review.openstack.org/8359619:56
lifelessdprince: the brbm bridge isn't currently accessed by anything on the host machine19:56
dprincelifeless: yeah. I commented as much19:56
lifelessdprince: it doesn't have an ip etc; we may change that in future, but for now its fine19:56
dprincelifeless: ah, cool. So you buy this?19:56
lifelessdprince: haven't read the patch yet, but changing the generated macs to look real is a big +1 conceptually from me19:57
lifelessdprince: have you reworked the other patch to not use the /e/n/i ovs support?19:57
dprincelifeless: was just trying to think ahead and avoid another (cryptic) problem19:57
dprincelifeless: no, I'm not sure I can sir19:57
lifelessdprince: ok, I'll take a look Monday19:58
dprincelifeless: the thing about eni is you either use it or you don't19:58
lifelessdprince: whats the # again ?19:58
dprincelifeless:  https://review.openstack.org/#/c/69918/19:58
lifelessdprince: I'm not saying 'dont use eni', I'm saying 'dont use the bad eni glue'19:59
lifelessdprince: just means the stanza is written differently and we let ovs maintain the state19:59
dprincelifeless: so the Fedora scripts behave the same way (i.e. it removes the bridge if you ifdown the bridge)19:59
lifelessdprince: should I push up another iteration when I look at it, or do you want me to send you a patch out of band ?19:59
lifelessdprince: sounds like using the fedora stuff will be a mistake too20:00
openstackgerritRyan Brady proposed a change to openstack/tripleo-image-elements: Fix tgt target in cinder element  https://review.openstack.org/8180420:00
openstackgerritGregory Haynes proposed a change to openstack/tripleo-heat-templates: Add initial support for galera clustering  https://review.openstack.org/8388320:01
dprincelifeless: I might prefer to see these side/by/side20:01
dprincelifeless: I'm not sure the init scripts will like it if you don't have them both configured though (haven't tested it)20:02
*** gcha has quit IRC20:03
dprincelifeless: On Fedora upping the interface tries to make sure the bridge is also up20:03
dprincelifeless: My guess is Ubuntu does the same20:04
dprincelifeless: in any case we bounce Neutron on a config refresh anyway so everything will get recreated20:04
*** dprince has quit IRC20:06
*** epim has quit IRC20:07
lifelessnuts hes gone20:07
lifelessI was going to say I'm very confused20:07
lifelessif the goal is to preserve state, we should20:07
lifelessif we don't want to preserve state, just putting the ovs db in tmpfs would do that :)20:07
*** gcha has joined #tripleo20:08
SpamapSlifeless: the bad ENI glue deserves some upstream fist shaking / bug filing before we abandon hope of it working.20:08
SpamapSlifeless: entirely possible that Debian doesn't understand.20:09
lifelessSpamapS: I'm sure they don't, but we need something that works now20:11
lifelessSpamapS: so we should shake in parallel20:11
SpamapSRight, I prefer to start upstream. For all we know they've a fix in-flight.20:12
*** yassine has quit IRC20:16
*** vkozhukalov has quit IRC20:21
*** morazi_ has joined #tripleo20:22
*** morazi has quit IRC20:25
lifelessSpamapS: https://bugs.debian.org/cgi-bin/pkgreport.cgi?ordering=normal;archive=0;src=openvswitch;dist=unstable;repeatmerged=020:26
*** julim has quit IRC20:27
lifelessSpamapS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=701760 will affect us20:27
uvirtbotDebian bug 701760 in openvswitch-switch "openvswitch-switch: problem interacting with libvirt" [Important,Open]20:27
lifelessSpamapS: open for > 1 year20:28
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Enable unit testing  https://review.openstack.org/8388820:30
openstackgerritBen Nemec proposed a change to openstack/tripleo-image-elements: Add unit tests for os-svc-daemon  https://review.openstack.org/8388920:30
lifelessSpamapS: may https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=68053820:30
uvirtbotDebian bug 680538 in openvswitch-brcompat "openvswitch - brcompat unable to overtake bridge configured by ifupdown" [Normal,Open]20:30
lifelessSpamapS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=68651820:31
uvirtbotDebian bug 686518 in openvswitch-switch "openvswitch-switch: No network configuration with openvswitch" [Normal,Open]20:31
lifelessSpamapS: there isn't a precise bug listed20:32
*** julim has joined #tripleo20:33
openstackgerritSteven DuChene proposed a change to openstack/diskimage-builder: Add ability to add extra apt keys  https://review.openstack.org/8359620:36
lifelessthe thing I think people miss is that openvswitch is not like brutils20:38
lifelessSpamapS: anyhow its saturday here, I just don't want us to half-ass this; either we're stateless in which case we should be stateless; or we're stateful in which case we should be stateful.20:39
*** eguz has joined #tripleo20:52
*** CaptTofu has joined #tripleo20:52
*** andreaf has joined #tripleo20:54
openstackgerritA change was merged to openstack/diskimage-builder: Write a dpkg manifest to list installed packages  https://review.openstack.org/8317120:55
*** eghobo has quit IRC20:56
*** julim has quit IRC20:59
*** matty_dubs is now known as matty_dubs|gone21:01
*** mestery has joined #tripleo21:02
*** lblanchard has quit IRC21:06
*** rpodolyaka1 has joined #tripleo21:09
NobodyCamlifeless: I think just a minor change to the ci_commands regex would fix... but I have no idea on how to actually test it.. http://paste.openstack.org/show/G5usxK1nMdJ7fbe8DkpM/21:10
*** jdob has quit IRC21:11
*** rpodolyaka1 has quit IRC21:18
devanandaSpamapS: hi! have a few ironic patches you may be interested in, if'n your testing ironic stuff today21:20
SpamapSdevananda: I'm just trying to keep my patches moving forward in the review queue and doing reviews today....21:24
devanandaSpamapS: if you want to review stuff in ironic, there's a few RC-targeted patches in flight that fix things for tripleo .... :)21:24
SpamapSNg: btw, https://review.openstack.org/#/c/83614/ is blocking my software-config conversion work.21:25
SpamapSGheRivero: ^^21:25
SpamapSslagle: ^^21:25
* Ng looks21:25
SpamapSdevananda: yay21:25
*** sballe_ has joined #tripleo21:26
NobodyCamSpamapS: also don't know if you saw my comment to lifeless. I think a minor regex change will fix the ssh bug lifeless reported in ci21:27
*** stevehuang has quit IRC21:28
SpamapSNobodyCam: patch #?21:29
*** sballe has quit IRC21:29
lifelessSpamapS: 10:10 < NobodyCam> lifeless: I think just a minor change to the ci_commands regex would fix... but I have no idea on how to actually test it..21:32
lifeless                   http://paste.openstack.org/show/G5usxK1nMdJ7fbe8DkpM/21:32
NobodyCamSpamapS: just getting it ready now21:34
*** morazi_ has quit IRC21:37
*** eguz has quit IRC21:39
openstackgerritChris Krelle proposed a change to openstack-infra/tripleo-ci: Allow ironic to use ssh commands  https://review.openstack.org/8390621:42
NobodyCamSpamapS: lifeless ^^^ I do not know how to test that localy21:43
lifelessNobodyCam: configure the ssh virtual power key on your laptop using that21:48
lifelessNobodyCam: then with USE_IRONIC=1 run devtest21:48
lifelessNobodyCam: but customise your testenv.json first21:48
lifelessroughly21:48
lifelessactually no need to customise testenv.json21:49
lifelessjust modify your .ssh/authorized_keys file to use ci_commands21:49
lifelessthen run with USE_IRONIC=121:49
NobodyCam:) doh... yes... doing now21:50
jang1heh. lesson learned today: make sure your ipmi addresses line up with your intended targets. Yes, I rebooted my workstation.21:51
lifelessjang1: :)21:52
lifelesswelcome to inception21:52
*** e0ne_ has joined #tripleo21:55
SpamapSjang1: at least you only _rebooted_ it.21:55
*** e0ne has quit IRC21:57
*** untriaged-bot has joined #tripleo22:00
untriaged-botUntriaged bugs so far:22:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/129048822:00
*** untriaged-bot has quit IRC22:00
uvirtbotLaunchpad bug 1290488 in tripleo "Baremetal: Invalid credentials" [Undecided,Incomplete]22:00
*** e0ne_ has quit IRC22:03
*** e0ne has joined #tripleo22:04
*** geerdest has quit IRC22:04
*** eghobo has joined #tripleo22:13
*** eghobo has quit IRC22:13
*** eghobo has joined #tripleo22:13
jang1I would have reimaged it but nova bm died.22:14
NobodyCamlifeless: SpamapS: I think it was able to test correctly. and it worked: http://paste.openstack.org/show/NkpV3FYoqcsvBw186Abu/22:14
jang1lifeless: if I can bend your ear for a couple of minutes: you said that the heat callback URIs were signed. How's the key generated for that? I take it that launching the same seed disk image a second time won't lead to the same signatures>22:15
jang1still trying to figure out what's going on with our bizarre baremetal misbehaviour22:15
lifelessSpamapS: ^22:17
lifelessjang1: I haven't read the deep plumbing but22:17
lifelessjang1: the waitcondition url is unique to a stack + wait condition instance22:17
lifelessjang1: stacks have a uuid, so new seed -> new stack - must be unique22:18
lifelessjang1: beyond that, if the wait condition resource is recreated I'm moderately sure it would be unique there as well22:18
*** morazi has joined #tripleo22:19
jang1hrm.22:20
jang1I've been chasing another bug today (traces from the nova bm image download) - will pick it up on Monday.22:21
*** marun has quit IRC22:21
jang1although, all of Galway _is_ asleep right now...22:21
lifelessgood time to use their workstations as  atest cluster22:22
NobodyCamoh can we deploy a seti search client on them while they sleep?22:23
*** marun has joined #tripleo22:23
*** andreaf has quit IRC22:24
jang1I thought bitcoin mining was all the rage these days22:26
NobodyCamlifeless: undercloud deployed and loading up all the overcloud files... testing with lifeless method has worked22:26
NobodyCam a6d0f7ee-1a24-47b8-abf2-198657449190 | overcloud  | CREATE_IN_PROGRESS | 2014-03-28T22:27:14Z22:27
lifelessNobodyCam: and it used ironic (e.g. ironic node-list against the undercloud works?)22:31
lifelessNobodyCam: and - finalyl - try ssh localhost -i ~/.ssh/virtual-power-key and make sure that that auths ok but errors because the command isn't permitted22:32
NobodyCamlifeless: http://paste.openstack.org/show/fXEElTX4EG2C48dkYwOU/22:33
NobodyCamugg got me with the last one... I was able to auth... which is very odd22:35
lifelessshould be able to auth22:35
NobodyCams/auth/login22:35
lifelessshouldn't be able to login22:35
lifelessso, the virtual-power-key line in your authorized_keys file is wrong :)22:36
lifelessor try22:36
lifelessssh localhost -i ~/.ssh/virtual-power-key ls22:36
lifeless(maybe we wrongly let interactive connections through)22:36
SpamapSjang1: the key for the signature is an ec2 credential created attached to an ephemeral keystone user that can only do one thing (signal the waitcondition)22:36
lifelesswhich would be a separate bug22:36
NobodyCamnobodycam@nobodycam-HP-EliteBook-8460p:~$ ssh localhost -i ~/.ssh/id_rsa_virt_power ls22:38
NobodyCamUnknown command: ls22:38
NobodyCamretesting the login bit22:38
NobodyCamlifeless: nope couldn't log in http://paste.openstack.org/show/KXB0dDhTBmvBrZ35eRow/22:39
NobodyCamI just cut and pastd you example and then recalled I use a different key name.. so it nicly defaulted to my id_rsa file22:40
lifelessNobodyCam: did devtest use thay key?22:40
NobodyCamdevtest should have used the correct.. but I am already dbl checking22:41
jang1SpamapS: thanks. Okay, back to the drawing board.22:41
NobodyCamdbl check'd devtest used the correct key!22:44
jang1hrm. How does the callback URI get passed to the undercloud instance? Is it via metadata, or injected into the image via file injection?22:44
lifeless+2'd22:44
lifelessSpamapS: ^22:45
lifelessjang1: heat metadata22:45
lifelessjang1: file injection is the devils work22:45
*** eguz has joined #tripleo22:48
jang1okay. Sorry, I've not had a chance to look at the metadata path. If I'm an undercloud running on a particular mac address, where nova BM will be bringin up an instance, what stops me from reading the callback uri from the metadata service?22:48
*** eghobo has quit IRC22:52
jang1(actually this is trivial for me to try. I can configure the re-launched seed with the wrong ipmi password so it can't reboot the undercloud. Let me have a fiddle with this.)22:54
lifelessec2 metadata holds creds to heat metadata22:59
lifelessheat metadata has the wait cond url22:59
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: set -e all the things  https://review.openstack.org/8392723:04
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Ensure scripts are set -u  https://review.openstack.org/8392823:04
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Check for set -o pipefail  https://review.openstack.org/8392923:04
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: dib-lint does not work with set -e  https://review.openstack.org/8393023:04
openstackgerritBen Nemec proposed a change to openstack/diskimage-builder: Make sure all scripts are set -e  https://review.openstack.org/8163723:04
*** gcha has quit IRC23:05
*** SlickNik is now known as gerrit23:06
*** gerrit is now known as Guest3549423:06
*** Guest35494 is now known as SlickNik23:07
lifelessjang1: so - I haven't rechecked the bug to see what symptoms you're having23:08
lifelessjang1: but23:08
lifelessjang1: if heat believes the instance is deployed before it has23:08
lifelessjang1: a possible process would be ec2 -> updated heat creds -> success23:09
lifelessjang1: using Ironic will fix this23:09
NobodyCamhappy dance: http://paste.openstack.org/show/t38Hj7wkxmfB4aT8WQJQ/23:11
* NobodyCam notes his laptop is not all that fast23:11
*** CaptTofu has quit IRC23:12
jang1what it _looked_ like to us was that, with the old undercloud running, the new seed stood up. It got ready to launch a new stack, put metadata available, which the old undercloud got hold of and used (calling back to heat) before NBM had a chance to shut it down and deploy the new image to it.23:12
jang1but that was mostly surmising from the state of heat afterwards.23:12
*** CaptTofu has joined #tripleo23:12
jang1anyway, it's bedtime here. _("night") all.23:13
lifelessnight!23:13
*** jang1 has quit IRC23:14
*** CaptTofu has quit IRC23:16
* greghaynes suddenly understands a lot more after finding the os-apply-config element23:30
greghaynesNow im just confused about why we would ever run service mysql restart before the config applier has run...23:31
*** CaptTofu has joined #tripleo23:44
*** slagle has quit IRC23:46
*** slagle has joined #tripleo23:47
*** julim has joined #tripleo23:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!