| *** xuhaiwei has joined #tripleo | 00:04 | |
| * StevenK stabs the nvidia kernel module | 00:07 | |
| *** matsuhashi has joined #tripleo | 00:12 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Remove $(dirname $0) from devtest.sh docs. https://review.openstack.org/79517 | 00:13 |
|---|---|---|
| tchaypo | StevenK: do tell | 00:14 |
| StevenK | tchaypo: Display lockups with NVRM: Xid (0000:01:00): 8, Channel 00000001 | 00:14 |
| tchaypo | where are you seeing that? | 00:23 |
| StevenK | dmesg, before I rebooted | 00:23 |
| StevenK | Which is the only way to get the card and the kernel module back into sync | 00:23 |
| *** matsuhashi has quit IRC | 00:31 | |
| lifeless | StevenK: what driver? | 00:35 |
| lifeless | wendar: on the offchance you're still up - you have mail | 00:38 |
| StevenK | lifeless: nvidia 304.88 | 00:41 |
| lifeless | StevenK: hmm, I have radeon on this laptop. should I change? | 00:47 |
| StevenK | lifeless: Sorry, I was complaining about my desktop, not my laptop | 00:47 |
| *** matsuhashi has joined #tripleo | 00:52 | |
| lifeless | ah, kk | 00:54 |
| *** fandi has quit IRC | 00:58 | |
| tchaypo | slagle: I don't suppose you're around? | 01:02 |
| xuhaiwei | lifeless: I am confused about this in the tripleo-incubator README, ' os-apply-config reads a JSON metadata file and generates templates', what are the 'templates' here? | 01:06 |
| lifeless | xuhaiwei: moustache templates | 01:07 |
| xuhaiwei | and ' moustache templates' are? | 01:07 |
| lifeless | xuhaiwei: moustache templates! | 01:07 |
| lifeless | http://lmgtfy.com/?q=moustache+templates | 01:08 |
| *** CaptTofu has joined #tripleo | 01:12 | |
| *** CaptTofu has quit IRC | 01:17 | |
| xuhaiwei | lifeless: I think I have understood mustache templates, but what are these templates for? used by heat to do the configuration? | 01:25 |
| lifeless | xuhaiwei: by oac to capture the metadata json into config files | 01:26 |
| *** CaptTofu has joined #tripleo | 01:26 | |
| xuhaiwei | oac read a JSON metadata file and generates templates, so the templates are the config files, right? | 01:27 |
| tchaypo | xuhaiwei: welcome back | 01:27 |
| xuhaiwei | tchaypo: good morning, it's not morning in your place maybe | 01:29 |
| StevenK | Nope, it's been the afternoon for 30 minutes | 01:29 |
| xuhaiwei | here is UTC+9 | 01:30 |
| *** nosnos has joined #tripleo | 01:30 | |
| tchaypo | lifeless: I'm fairly certain that https://bugs.launchpad.net/diskimage-builder/+bug/1266550 is just caused by using a version of pip <1.5 - is there anything we should do about this? I'm inclined to close it as not being our problem, but I'm wondering if we specify a version of pip somewhere.. | 01:31 |
| lifeless | mordred: does pip 1.5+ work with openstack stuff ? | 01:32 |
| mordred | lifeless: it should | 01:32 |
| mordred | we're not using it in the gate yet because of tox fail | 01:32 |
| tchaypo | mordred: so I'm interpreting that as also meaning that you don't "pip install diskimage-builder" in the gate? | 01:33 |
| lifeless | tchaypo: thats correct | 01:34 |
| xuhaiwei | lifeless: it's really vague here, after hearing your explanation, my understanding is oac uses JSON metadate to generate mustache templates, and the mustache templates are used by heat to do the configuration for openstack, am I right? | 01:34 |
| tchaypo | I'm going to go ahead and close | 01:35 |
| tchaypo | hrm. Won | 01:35 |
| lifeless | xuhaiwei: no, heat doesn't do any configuration. its a web service. configuration happens locally. | 01:35 |
| tchaypo | "won't fix" or "invalid"? | 01:35 |
| lifeless | xuhaiwei: heat exports the metadata for the machine. oac turns that into files on disk | 01:36 |
| tchaypo | I think this is invalid - as far as i can tell, it's not a bug with DIB nor with tripleo | 01:36 |
| xuhaiwei | lifeless: and the 'files' are used by openstack? | 01:36 |
| *** shakamunyi has joined #tripleo | 01:38 | |
| tchaypo | If I understand oac correctly, the files it's generating will generally be the config file for the thing being configured | 01:38 |
| tchaypo | if you look at the examples in the README (https://pypi.python.org/pypi/os-apply-config has a handy web version), you'll see that it generates /etc/mysql/mysql.conf | 01:39 |
| xuhaiwei | tchaypo: for example, like this? connection = mysql://{{keystone.database.user}}:{{keystone.database.password}@{{keystone.database.host}}/keystone | 01:42 |
| tchaypo | Yep, that seems to be a mustache template. The README notes that there's an example tree at http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/keystone/os-apply-config/etc | 01:47 |
| tchaypo | http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/keystone/os-apply-config/etc/keystone/keystone.conf is a template, which looks like it should end up at /etc/keystone/keystone.conf | 01:47 |
| xuhaiwei | I found it, you mean the keystone.conf is generated by os-apploy-config? sounds reasonable | 01:51 |
| xuhaiwei | but why is there only configuration file for keystone? | 01:53 |
| tchaypo | That particular example seems to be for the "keystone" element | 01:53 |
| *** CaptTofu has quit IRC | 01:53 | |
| *** CaptTofu has joined #tripleo | 01:54 | |
| xuhaiwei | ok, that makes sense for me, thank you | 01:54 |
| tchaypo | "element" in this case being a thing diskimage-builder uses when building an disk image | 01:54 |
| tchaypo | If you jump up a few levels you'll see that other elements have their own trees | 01:55 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Teach testenv about numbers and bridges https://review.openstack.org/80874 | 01:55 |
| tchaypo | eg http://git.openstack.org/cgit/openstack/tripleo-image-elements/tree/elements/debian-mirror/os-apply-config for the debian-mirror element | 01:55 |
| tchaypo | I assume something must munge all these trees together to produce a single unified tree for o-a-c to run over | 01:55 |
| * StevenK makes a note to destroy his wireless AP | 01:57 | |
| tchaypo | why? | 01:57 |
| StevenK | I can't scp over IPv6 when connected to it | 01:57 |
| StevenK | But IPv4 works fine | 01:57 |
| tchaypo | Do you speak dutch, perchance? | 01:58 |
| StevenK | No? | 01:58 |
| *** CaptTofu has quit IRC | 01:58 | |
| tchaypo | line 4 of devtest_testenv.sh has "VM's" | 01:59 |
| tchaypo | using 's to form a plural is correct/common in dutch | 01:59 |
| StevenK | Added in 578f9037 by lifeless | 02:00 |
| lifeless | I'm ' special | 02:01 |
| tchaypo | in other news, every time I see getopt (and the accompanying TEMP= and while true; do) I have a little cry. | 02:01 |
| StevenK | tchaypo: What would you prefer? | 02:01 |
| tchaypo | The problem is, there's nothing else that's as portable, so I can't point to anything I'd prefer to use | 02:02 |
| xuhaiwei | tchaypo: your explanation of oac makes sense to me, thank you. | 02:03 |
| StevenK | Bleh, I think I'm missing : for n and p | 02:04 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Teach testenv about numbers and bridges https://review.openstack.org/80874 | 02:06 |
| tchaypo | so | 02:07 |
| tchaypo | here's my manyth newbie question for the day | 02:08 |
| tchaypo | what's toci? | 02:08 |
| greghaynes | I always assumed triple o CI | 02:08 |
| tchaypo | oh. We're either talking about a mesoamerican deity, or TripleO CI | 02:08 |
| lifeless | tchaypo: TripleO-CI | 02:09 |
| tchaypo | it makes me sad that googling for this things always finds the github repo and never the git.openstack repo | 02:09 |
| lifeless | TOCI | 02:09 |
| * tchaypo -> loonch | 02:21 | |
| *** weshay has quit IRC | 02:35 | |
| *** CaptTofu has joined #tripleo | 02:42 | |
| *** matsuhashi has quit IRC | 02:48 | |
| *** tserong has quit IRC | 02:52 | |
| *** matsuhas_ has joined #tripleo | 02:52 | |
| *** tserong has joined #tripleo | 02:53 | |
| *** tserong has joined #tripleo | 02:53 | |
| *** killer_prince is now known as lazy_prince | 03:02 | |
| *** dkehn has joined #tripleo | 03:04 | |
| *** dkehn_ has quit IRC | 03:06 | |
| lifeless | I hate nbd | 03:17 |
| lifeless | 2911 root 20 0 59788 1768 888 R 98.5 0.1 416:30.47 qemu-nbd | 03:17 |
| *** matsuhas_ has quit IRC | 03:22 | |
| *** dkehn is now known as dkehn_ | 03:23 | |
| *** dkehn__ is now known as dkehn | 03:23 | |
| lifeless | I really want to give bcache a spin | 03:27 |
| lifeless | but you have to reformat your block devices | 03:27 |
| openstackgerrit | Steve Kowalik proposed a change to openstack/tripleo-incubator: Switch to using jq to write out initial JSON https://review.openstack.org/80884 | 03:28 |
| lifeless | ahha | 03:28 |
| lifeless | https://github.com/g2p/blocks | 03:28 |
| StevenK | lifeless: Thoughts on ^ ? | 03:30 |
| StevenK | lifeless: Not looking for an actual review, just a high-level looks good or 'wtf are you smoking' | 03:30 |
| *** xuhaiwei has quit IRC | 03:31 | |
| lifeless | StevenK: one quibble | 03:32 |
| StevenK | lifeless: It turns up in the JSON file as "ssh-key": "-----BEGIN RSA PRIVATE KEY-----\nMIIE... | 03:34 |
| lifeless | StevenK: does jq / python read the resulting json ok ? | 03:37 |
| StevenK | % jq ".[\"ssh-key\"]" < testenv.json | 03:41 |
| StevenK | "-----BEGIN RSA PRIVATE KEY-----\nMIIE... | 03:41 |
| StevenK | >>> f['ssh-key'] | 03:42 |
| StevenK | u'-----BEGIN RSA PRIVATE KEY-----\nMIIE... | 03:42 |
| lifeless | awesome | 03:44 |
| *** CaptTofu has quit IRC | 03:48 | |
| StevenK | {"ssh-key": "-----BEGIN RSA PRIVATE KEY-----\nMIIE... is what we get from the current python call, so it's perfect | 03:49 |
| StevenK | lifeless: Feel like approving https://review.openstack.org/#/c/77741/ so I can follow the same pattern in toci? | 03:52 |
| StevenK | Without having to do a horrible rebase later | 03:53 |
| *** xuhaiwei has joined #tripleo | 04:08 | |
| *** matsuhashi has joined #tripleo | 04:43 | |
| *** lazy_prince has quit IRC | 04:46 | |
| *** akuznetsov has joined #tripleo | 04:47 | |
| *** lazy_prince has joined #tripleo | 04:57 | |
| *** lazy_prince is now known as killer_prince | 04:58 | |
| tchaypo | StevenK: I'll verify this and add a comment on the review, but I believe you want <<< rather than << | 05:05 |
| tchaypo | afaik the difference is that << doesn't do any parameter substitution | 05:06 |
| StevenK | tchaypo: <<< is different again | 05:13 |
| *** killer_prince2 has joined #tripleo | 05:13 | |
| *** killer_prince2 is now known as lazy_prince | 05:13 | |
| tchaypo | my testing shows me I'm wrong, the code as written does the expected. | 05:13 |
| tchaypo | oh. I see. | 05:16 |
| tchaypo | The mange doesn't say that no expansion is performed on the here document, it says no expansion is performed on word (or in the case of <<<, it is performed) | 05:17 |
| tchaypo | don't mind me. | 05:17 |
| tchaypo | So my next thing is - afaict using simple "cat <<EOF > JSON_FILE" should suffice, why invoke jq? | 05:17 |
| tchaypo | My assumption is that you're expecting that jq will barf if the input isn't valid json, and it's better to break at this point than to break later on when something is reading the file and the poor user will be stuck trying to figure out where this came from? | 05:18 |
| StevenK | tchaypo: Because it will get extended later | 05:20 |
| StevenK | This is a first shot spilt it out first as a seperate branch | 05:20 |
| tchaypo | define "it" | 05:20 |
| StevenK | tchaypo: It being the work I'm doing on testenv | 05:22 |
| *** morganfainberg_Z is now known as morganfainberg | 05:34 | |
| lifeless | StevenK: have we landed the branch to consume that now? | 05:34 |
| *** akuznetsov has quit IRC | 05:44 | |
| StevenK | lifeless: That consumes nodes? The current HEAD of incubator will deal with either | 05:47 |
| *** CaptTofu has joined #tripleo | 05:49 | |
| *** nosnos has quit IRC | 05:54 | |
| *** nosnos has joined #tripleo | 05:54 | |
| *** CaptTofu has quit IRC | 05:54 | |
| *** matsuhashi has quit IRC | 05:56 | |
| *** matsuhashi has joined #tripleo | 05:57 | |
| *** matsuhashi has quit IRC | 05:57 | |
| lazy_prince | Hi all, in case if you get time, pls review https://review.openstack.org/#/c/79873/ it has passed ci and is waiting for review. | 05:58 |
| *** matsuhashi has joined #tripleo | 06:00 | |
| openstackgerrit | Steve Kowalik proposed a change to openstack-infra/tripleo-ci: Change how the JSON is updated for readability https://review.openstack.org/80905 | 06:02 |
| *** rpodolyaka has joined #tripleo | 06:02 | |
| *** akuznetsov has joined #tripleo | 06:02 | |
| *** matsuhashi has quit IRC | 06:09 | |
| *** matsuhashi has joined #tripleo | 06:10 | |
| *** matsuhas_ has joined #tripleo | 06:12 | |
| *** matsuhashi has quit IRC | 06:14 | |
| *** rpodolyaka has quit IRC | 06:16 | |
| *** rpodolyaka has joined #tripleo | 06:16 | |
| *** rpodolyaka has quit IRC | 06:19 | |
| *** nosnos has quit IRC | 06:24 | |
| *** nosnos has joined #tripleo | 06:25 | |
| *** matsuhas_ has quit IRC | 06:25 | |
| *** matsuhashi has joined #tripleo | 06:25 | |
| tchaypo | ho hum. In my change to rid devtest.html of telling the user to use $(dirname $0) I've introduced a chicken/egg | 06:27 |
| tchaypo | sure, source devtest_variables will set $TRIPLEO_ROOT and prepend $TRIPLE_ROOT/triple-incubator/scripts to the path | 06:28 |
| tchaypo | and that's all well and good *if you've run at least devtest_setup.sh once before* | 06:28 |
| tchaypo | but the very first time, $TRIPLEO_ROOT is empty | 06:28 |
| lifeless | StevenK: and you successfully tested the testenv patch ? | 06:37 |
| lifeless | tchaypo: how did it pass CI then ? | 06:38 |
| *** rpodolyaka has joined #tripleo | 06:38 | |
| *** matsuhashi has quit IRC | 06:39 | |
| *** matsuhashi has joined #tripleo | 06:40 | |
| tchaypo | lifeless: http://logs.openstack.org/17/79517/3/check-tripleo/check-tripleo-seed-precise/ca9ce2f/devstack-gate-setup-workspace-new.txt.gz#_2014-03-13_23_06_40_180 | 06:43 |
| tchaypo | which is mentioned at http://logs.openstack.org/17/79517/3/check-tripleo/check-tripleo-seed-precise/ca9ce2f/console.html.gz#_2014-03-13_23_06_27_711 | 06:44 |
| tchaypo | it looks like safe-devstack-vm-gate-wrap.sh takes a shortcut and does a git clone into /opt/stack/new/tripleo-incubator | 06:45 |
| tchaypo | so by the time we source devtest_variables.sh $TRIPLEO_ROOT/tripleo-incubator/scripts has the scripts in | 06:46 |
| tchaypo | but tha won't be true for someone setting up their environment for the first time | 06:46 |
| lifeless | tchaypo: are we broken right now? | 06:52 |
| lifeless | tchaypo: if so, we should revert the change asap and then you can mull it over at leisure | 06:52 |
| tchaypo | we're broken right now for anyone who clones onto a new machine which doesn't have $TRIPLEO_ROOT already existing | 06:53 |
| tchaypo | does git review have a magic for reverting, or do I just do a standard git revert; git review? | 06:53 |
| tchaypo | *reads docs* | 06:53 |
| StevenK | tchaypo: You'll have to git revert ; git ci --amend ; git review | 06:53 |
| StevenK | Revert doesn't add the Change-id that gerrit wants | 06:53 |
| tchaypo | need to amend to get the.. yep. | 06:54 |
| StevenK | lifeless: derekh tested patch set 2 for sure, not certain about 3, and I never managed to successfully test any of them | 06:55 |
| tchaypo | should I revert my change, or the change that merged mmy change into master? | 06:55 |
| StevenK | Your change | 06:55 |
| tchaypo | I'm thinking that it would be clearer if I do the former | 06:55 |
| StevenK | You can't revert a merge commit | 06:55 |
| StevenK | I think git will yell | 06:55 |
| *** rpodolyaka has quit IRC | 06:56 | |
| *** rpodolyaka has joined #tripleo | 06:58 | |
| *** rpodolyaka has quit IRC | 07:00 | |
| xuhaiwei | I don't see blueprint for tripleo-image-elements, does it mean almost all the functions are already done? | 07:01 |
| StevenK | Feature work doesn't require blueprints, we use trello to track that | 07:02 |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Revert "Remove $(dirname $0) from devtest.sh docs." https://review.openstack.org/80914 | 07:02 |
| *** rpodolyaka has joined #tripleo | 07:03 | |
| * tchaypo looks at trello | 07:05 | |
| xuhaiwei | ok, I looked at trello once on the last Friday, it's not easy for me to understand it | 07:06 |
| StevenK | tchaypo: Re your review, I'm tempted to switch it to cat <file> in a later patch, but I need to talk to derek about it | 07:06 |
| *** rpodolyaka has quit IRC | 07:07 | |
| tchaypo | I think I'd prefer jq, on balance. It's a smidge more heavyweight, and in the normal case we get no benefit | 07:07 |
| tchaypo | but in the case where we've somehow managed to screw up the json I think it's probably better to find out about it early | 07:07 |
| StevenK | tchaypo: I'm talking about the python one-liner that's embedded | 07:07 |
| tchaypo | StevenK: are you able to add me to the trello org? | 07:07 |
| tchaypo | oh, that one | 07:07 |
| tchaypo | meh. out-of-scope for that change. | 07:08 |
| StevenK | If we want to continue to replace \n with \\n, we could use sed -e ':a;N;$!ba;s/\n/\\n/g' /root/.ssh/test_env_rsa , but I think that's worse | 07:08 |
| lifeless | xuhaiwei: look under tripleo, not tripleo-image-elements | 07:08 |
| StevenK | tchaypo: I bugged lifeless to add me | 07:09 |
| lifeless | xuhaiwei: but most of our work is too fine grained to make blueprints, we do have blueprints for big things like HA design | 07:09 |
| tchaypo | lifeless: *bug**bug* | 07:09 |
| lifeless | tchaypo: for what? | 07:09 |
| tchaypo | lifeless: add me to the org in trello | 07:09 |
| lifeless | whats your trello uid ? | 07:09 |
| xuhaiwei | lifeless: look the bp under tripleo? | 07:10 |
| StevenK | xuhaiwei: As in https://blueprints.launchpad.net/tripleo | 07:10 |
| tchaypo | lifeless: it's now tchaypo | 07:10 |
| xuhaiwei | ok | 07:11 |
| lifeless | tchaypo: done | 07:13 |
| tchaypo | woo. Should I add a card for https://review.openstack.org/80914 (and the reimplementation that's going to follow) or is that too small a piece of work? | 07:14 |
| tchaypo | I'm thinking that since I've discovered a way in which the ci environment doesn't mirror what we tell users to do, this probably warrants a bug and a card | 07:14 |
| lifeless | it certainly warrants a bug | 07:16 |
| lifeless | tchaypo: but is it something to worry about right now, vs e.g. HA ? | 07:16 |
| lifeless | tchaypo: or reuse-on-baremetal ? | 07:16 |
| lifeless | tchaypo: or provisioning mirrors to make builds more reliable? | 07:16 |
| lifeless | Ng: / GheRivero: https://review.openstack.org/#/c/80914/ | 07:17 |
| lifeless | ok, I'm awl for a bit | 07:18 |
| tchaypo | it's ~4 lines of code to fix; to my mind that's so trivial I may as well do it now so I can stop thinking about it | 07:18 |
| tchaypo | vs coming back in N weeks and having to recreate mental state | 07:18 |
| *** jprovazn has joined #tripleo | 07:32 | |
| *** e0ne has joined #tripleo | 07:46 | |
| *** CaptTofu has joined #tripleo | 07:50 | |
| lifeless | tchaypo: 4 lines is not really worth a card then :) | 07:51 |
| tchaypo | urr. obviously that's not a few lines to make the CI env more like a naive user env. It's a few lines to make the naive user env work, and then I think we can drop this and move on | 07:51 |
| lifeless | tchaypo: I mean you can, but its not really a team coordination issue | 07:51 |
| tchaypo | that's spooky. I go away for 30 minutes and then just as I'm typing a response to myself you preempt me and make it look like I'm responding to you | 07:51 |
| tchaypo | you're going to be one very annoying parent if you can pull off that trick often enough :) | 07:52 |
| lifeless | :) | 07:52 |
| lifeless | just won a CS:GO game 16:1, which is unheard of :) | 07:52 |
| *** CaptTofu has quit IRC | 07:55 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Remove $(dirname $0) from devtest.sh docs. https://review.openstack.org/80920 | 07:55 |
| *** e0ne has quit IRC | 07:56 | |
| openstackgerrit | James Polley proposed a change to openstack/tripleo-incubator: Remove $(dirname $0) from devtest.sh docs. https://review.openstack.org/80920 | 07:57 |
| *** rpodolyaka has joined #tripleo | 07:57 | |
| rpodolyaka | morning tripleo | 07:57 |
| *** giulivo has joined #tripleo | 07:59 | |
| * tchaypo -> dinner | 08:00 | |
| *** nijaba has joined #tripleo | 08:02 | |
| *** nijaba has quit IRC | 08:02 | |
| *** nijaba has joined #tripleo | 08:02 | |
| *** rcarrillocruz has joined #tripleo | 08:02 | |
| *** rpodolyaka1 has joined #tripleo | 08:03 | |
| rcarrillocruz | hmm, guys, some change apparently broke devtest to be run unattended? | 08:03 |
| rcarrillocruz | vagrant@precise64:~/tripleo-incubator/scripts$ ./devtest.sh | 08:03 |
| rcarrillocruz | Not running - this script is destructive and requires --trash-my-machine to run. | 08:03 |
| rcarrillocruz | vagrant@precise64:~/tripleo-incubator/scripts$ ./devtest.sh --trash-my-machine | 08:03 |
| rcarrillocruz | ./devtest.sh: line 144: devtest_setup.sh: command not found | 08:03 |
| rcarrillocruz | vagrant@precise64:~/tripleo-incubator/scripts$ | 08:03 |
| *** gcha has joined #tripleo | 08:05 | |
| *** rpodolyaka1 has quit IRC | 08:07 | |
| GheRivero | morning all | 08:10 |
| rpodolyaka | o/ | 08:10 |
| lifeless | rcarrillocruz: yes, fixed by https://review.openstack.org/#/c/80914/ which GheRivero should be landing soon :) | 08:12 |
| GheRivero | done | 08:12 |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: Revert "Remove $(dirname $0) from devtest.sh docs." https://review.openstack.org/80914 | 08:12 |
| *** rcarrillocruz1 has joined #tripleo | 08:13 | |
| *** nosnos has quit IRC | 08:13 | |
| *** rcarrillocruz has quit IRC | 08:13 | |
| *** nosnos has joined #tripleo | 08:14 | |
| *** matsuhashi has quit IRC | 08:17 | |
| *** matsuhashi has joined #tripleo | 08:17 | |
| *** rdopieralski has joined #tripleo | 08:18 | |
| *** Lotus907efi1 has quit IRC | 08:21 | |
| *** rcarrillocruz1 has quit IRC | 08:21 | |
| *** openstack has quit IRC | 08:21 | |
| *** openstack has joined #tripleo | 08:30 | |
| *** openstackstatus has joined #tripleo | 08:30 | |
| *** andreaf has joined #tripleo | 08:31 | |
| *** e0ne has joined #tripleo | 08:32 | |
| *** nosnos has quit IRC | 08:34 | |
| *** jcoufal has joined #tripleo | 08:36 | |
| *** fandi has joined #tripleo | 08:37 | |
| *** jistr has joined #tripleo | 08:40 | |
| *** e0ne_ has joined #tripleo | 08:44 | |
| *** morganfainberg is now known as morganfainberg_Z | 08:45 | |
| *** e0ne has quit IRC | 08:48 | |
| *** dividehex has quit IRC | 08:50 | |
| *** fandi has quit IRC | 08:54 | |
| *** vkozhukalov_ has joined #tripleo | 08:55 | |
| *** petertoft has joined #tripleo | 08:56 | |
| *** matsuhas_ has quit IRC | 08:56 | |
| *** dividehex has joined #tripleo | 08:58 | |
| *** martyntaylor has joined #tripleo | 08:59 | |
| *** lucasagomes has joined #tripleo | 09:02 | |
| *** rpodolyaka1 has joined #tripleo | 09:03 | |
| *** andreaf2 has joined #tripleo | 09:04 | |
| *** lucasagomes has quit IRC | 09:05 | |
| *** lucasagomes has joined #tripleo | 09:05 | |
| *** mrunge has joined #tripleo | 09:05 | |
| *** matsuhashi has joined #tripleo | 09:06 | |
| Ng | morning | 09:07 |
| *** andreaf has quit IRC | 09:07 | |
| *** rpodolyaka1 has quit IRC | 09:08 | |
| *** martyntaylor has left #tripleo | 09:09 | |
| *** lucasagomes has quit IRC | 09:09 | |
| *** rcarrillocruz1 has joined #tripleo | 09:11 | |
| *** rcarrillocruz has quit IRC | 09:12 | |
| *** nosnos has joined #tripleo | 09:14 | |
| *** rcarrillocruz has joined #tripleo | 09:18 | |
| *** rcarrillocruz1 has quit IRC | 09:20 | |
| *** jcoufal has quit IRC | 09:20 | |
| *** xuhaiwei has quit IRC | 09:34 | |
| *** martyntaylor has joined #tripleo | 09:39 | |
| *** jtomasek has joined #tripleo | 09:42 | |
| *** fandi has joined #tripleo | 09:48 | |
| *** CaptTofu has joined #tripleo | 09:51 | |
| *** vkozhukalov_ has quit IRC | 09:51 | |
| *** nosnos_ has joined #tripleo | 09:55 | |
| *** CaptTofu has quit IRC | 09:56 | |
| *** nosnos has quit IRC | 09:58 | |
| *** derekh has joined #tripleo | 10:02 | |
| *** martyntaylor has left #tripleo | 10:02 | |
| derekh | I'm not here right now so caon't look into it but FYI no overcloud jobs have passed since friday evening http://goodsquishy.com/downloads/tripleo-jobs.html | 10:03 |
| derekh | setting firedrill, so it gains attention | 10:03 |
| *** derekh changes topic to "FIREDRILL overcloud ci job not passing http://goodsquishy.com/downloads/tripleo-jobs.html | TripleO | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 10:04 | |
| derekh | that page with the jobs listed updates every 20 minutes, | 10:04 |
| derekh | gotta go | 10:04 |
| *** derekh has quit IRC | 10:04 | |
| *** nosnos_ has quit IRC | 10:10 | |
| *** martyntaylor has joined #tripleo | 10:10 | |
| *** e0ne_ has quit IRC | 10:12 | |
| *** nosnos has joined #tripleo | 10:12 | |
| *** martyntaylor has quit IRC | 10:13 | |
| *** e0ne has joined #tripleo | 10:14 | |
| *** tteggel has quit IRC | 10:18 | |
| *** tteggel has joined #tripleo | 10:18 | |
| *** akrivoka has joined #tripleo | 10:19 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Correct permissions on /tftpdboot/pxelinux.cfg directory https://review.openstack.org/80595 | 10:19 |
| openstackgerrit | A change was merged to openstack/tripleo-heat-templates: Change nova compute driver location https://review.openstack.org/80601 | 10:21 |
| *** pbelanyi has joined #tripleo | 10:27 | |
| *** matsuhashi has quit IRC | 10:38 | |
| *** rcarrillocruz1 has joined #tripleo | 10:42 | |
| *** rcarrillocruz has quit IRC | 10:45 | |
| *** nosnos has quit IRC | 10:46 | |
| *** nosnos has joined #tripleo | 10:47 | |
| *** matsuhashi has joined #tripleo | 10:54 | |
| *** vkozhukalov_ has joined #tripleo | 10:57 | |
| *** rpodolyaka1 has joined #tripleo | 11:01 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Add link for Ironic rootwrap https://review.openstack.org/80719 | 11:01 |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Change nova scheduler driver location https://review.openstack.org/80699 | 11:01 |
| *** xuhaiwei has joined #tripleo | 11:02 | |
| *** rpodolyaka1 has quit IRC | 11:05 | |
| *** matsuhashi has quit IRC | 11:05 | |
| openstackgerrit | jan grant proposed a change to openstack/tripleo-incubator: Use a pypi mirror for the bootstrap process. https://review.openstack.org/80949 | 11:08 |
| *** jcoufal has joined #tripleo | 11:09 | |
| *** openstackgerrit has quit IRC | 11:10 | |
| *** openstackgerrit has joined #tripleo | 11:11 | |
| *** matsuhashi has joined #tripleo | 11:13 | |
| rcarrillocruz1 | guys, is there any reason why when we run devtest.sh we clone tripleo-incubator into the .cache folder? I think it would be good to have an option to simply copy the already cloned tripleo-incubator files from where we run devtest in first instance. That way any modification we do to devtest scripts will be tested, as is now no matter what we modify in tripleo-incubator scripts will be taken into account, as we clone tripleo-incub | 11:18 |
| rcarrillocruz1 | unless my workflow is broken, any thoughts? | 11:18 |
| jang | I've a change to do something like this. | 11:18 |
| jang | https://review.openstack.org/#/c/78371/ | 11:19 |
| jang | I use a directory of checked-out git repos (including tripleo-incubator) and have a small sourced file that sets DIB_REPOLOCATION and DIB_REPOREF for each of them. Then if I run devtest from inside that directory, an equivalent checkout (based on that repo) happens to my ~/.cache | 11:20 |
| *** andreaf2 has quit IRC | 11:21 | |
| rcarrillocruz1 | nice, i wondered if it was just me that found this a bit annoying to circumvent | 11:22 |
| rcarrillocruz1 | thanks jang | 11:22 |
| jang | you're welcome. | 11:29 |
| jang | (and also welcome to give that a +1 if you find it useful, heh :-) ) | 11:30 |
| *** matsuhashi has quit IRC | 11:31 | |
| jang | rcarrillocruz1: I use it with this: http://pastebin.com/nmkaak9n as a part of the script that sets up my environment before running devtest.sh | 11:32 |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Node profile suggestions https://review.openstack.org/78563 | 11:32 |
| *** rcarrillocruz1 has quit IRC | 11:35 | |
| *** fandi has quit IRC | 11:38 | |
| *** lazy_prince2 has quit IRC | 11:45 | |
| *** e0ne has quit IRC | 11:45 | |
| *** weshay has joined #tripleo | 11:46 | |
| *** e0ne has joined #tripleo | 11:49 | |
| *** sdake has quit IRC | 11:49 | |
| *** CaptTofu has joined #tripleo | 11:52 | |
| *** lazy_prince has joined #tripleo | 11:54 | |
| *** lazy_prince is now known as killer_prince | 11:54 | |
| *** martyntaylor has joined #tripleo | 11:55 | |
| *** CaptTofu has quit IRC | 11:56 | |
| *** xuhaiwei has quit IRC | 11:59 | |
| *** CaptTofu has joined #tripleo | 12:02 | |
| *** dprince has joined #tripleo | 12:03 | |
| *** rdopiera1ski is now known as rdopieralski | 12:04 | |
| *** lynxman has quit IRC | 12:04 | |
| *** lynxman has joined #tripleo | 12:05 | |
| *** e0ne_ has joined #tripleo | 12:08 | |
| *** gonery has joined #tripleo | 12:11 | |
| *** e0ne has quit IRC | 12:12 | |
| openstackgerrit | Radomir Dopieralski proposed a change to openstack/tuskar-ui: Fix spelling in a comment in test data https://review.openstack.org/79041 | 12:13 |
| *** rdopieralski has quit IRC | 12:14 | |
| *** rdopieralski has joined #tripleo | 12:16 | |
| *** matsuhashi has joined #tripleo | 12:27 | |
| *** CaptTofu has quit IRC | 12:31 | |
| *** martyntaylor1 has joined #tripleo | 12:32 | |
| *** e0ne has joined #tripleo | 12:32 | |
| *** martyntaylor has quit IRC | 12:35 | |
| *** e0ne_ has quit IRC | 12:36 | |
| *** matsuhashi has quit IRC | 12:37 | |
| *** nosnos_ has joined #tripleo | 12:37 | |
| *** nosnos has quit IRC | 12:37 | |
| *** matsuhashi has joined #tripleo | 12:37 | |
| *** matsuhashi has quit IRC | 12:42 | |
| *** e0ne has quit IRC | 12:45 | |
| *** e0ne has joined #tripleo | 12:45 | |
| *** pblaho has quit IRC | 12:48 | |
| *** Guest26078 is now known as ohadlevy | 12:50 | |
| *** ohadlevy has quit IRC | 12:50 | |
| *** ohadlevy has joined #tripleo | 12:50 | |
| *** CaptTofu has joined #tripleo | 12:50 | |
| *** weshay has quit IRC | 12:54 | |
| *** sseago_ has joined #tripleo | 12:54 | |
| *** CaptTofu has quit IRC | 12:54 | |
| *** kiall_ is now known as Kiall | 12:59 | |
| *** rbrady1 is now known as rbrady | 13:00 | |
| *** matsuhashi has joined #tripleo | 13:03 | |
| *** lblanchard has joined #tripleo | 13:04 | |
| *** matsuhashi has quit IRC | 13:05 | |
| *** matsuhashi has joined #tripleo | 13:05 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Move install type enablement into base element https://review.openstack.org/78794 | 13:07 |
| *** matsuhashi has quit IRC | 13:10 | |
| *** nosnos_ has quit IRC | 13:12 | |
| *** dkehn__ is now known as dkehn | 13:14 | |
| *** jtomasek has quit IRC | 13:17 | |
| *** hashar has joined #tripleo | 13:19 | |
| *** julim has joined #tripleo | 13:22 | |
| *** jtomasek has joined #tripleo | 13:32 | |
| *** blamar has joined #tripleo | 13:39 | |
| *** weshay has joined #tripleo | 13:43 | |
| *** jpeeler has joined #tripleo | 13:47 | |
| *** jpeeler has joined #tripleo | 13:47 | |
| *** slagle has quit IRC | 13:58 | |
| *** lazy_prince has joined #tripleo | 13:59 | |
| *** killer_prince has quit IRC | 14:00 | |
| *** lazy_prince is now known as killer_prince | 14:00 | |
| *** slagle has joined #tripleo | 14:00 | |
| *** jcoufal has quit IRC | 14:03 | |
| *** jcoufal has joined #tripleo | 14:04 | |
| *** rwsu has joined #tripleo | 14:07 | |
| *** killer_prince is now known as lazy_prince | 14:10 | |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Node profile suggestions https://review.openstack.org/78563 | 14:10 |
| openstackgerrit | A change was merged to openstack/tuskar-ui: Fix spelling in a comment in test data https://review.openstack.org/79041 | 14:10 |
| *** victor_lowther__ has quit IRC | 14:11 | |
| *** victor_lowther__ has joined #tripleo | 14:12 | |
| *** pblaho has joined #tripleo | 14:21 | |
| openstackgerrit | Ladislav Smola proposed a change to openstack/tuskar: Return template attributes https://review.openstack.org/77555 | 14:30 |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow set-source-vars to consume a git manifest https://review.openstack.org/80617 | 14:31 |
| openstackgerrit | Matthew Gilliard proposed a change to openstack/tripleo-incubator: Allow set-source-vars to consume a git manifest https://review.openstack.org/80617 | 14:33 |
| *** julim_ has joined #tripleo | 14:33 | |
| *** julim has quit IRC | 14:36 | |
| *** lazy_prince has quit IRC | 14:40 | |
| *** rcarrillocruz has joined #tripleo | 14:54 | |
| gonery | Hi, I'm a bit lost with the ${foo}-package-install and ${foo}-source-install difference. Do you have any point that explain how it works? | 15:01 |
| jang | I forget where the details are - the source-repositories element? | 15:07 |
| *** newell has joined #tripleo | 15:07 | |
| SpamapS | gonery: It is actually about to change. | 15:07 |
| *** tburnes has joined #tripleo | 15:07 | |
| gonery | I finally found that: https://github.com/openstack/diskimage-builder/blob/master/README.md#install-types | 15:09 |
| gonery | SpamapS: how, I thought it was an ongoing changes | 15:09 |
| *** killer_prince has joined #tripleo | 15:10 | |
| SpamapS | gonery: ohh! that just landed :) | 15:10 |
| SpamapS | https://review.openstack.org/#/c/78794/ | 15:11 |
| jang | that's much neater. | 15:12 |
| gonery | ah! awesome. I was getting mad =) | 15:13 |
| *** vkozhukalov_ has quit IRC | 15:14 | |
| *** andreaf has joined #tripleo | 15:14 | |
| SpamapS | Yeah that is only 2 hours old :) | 15:15 |
| *** slagle has quit IRC | 15:16 | |
| gonery | I'm discovering TripleO and I did some (minor) changes on the doc like this one https://review.openstack.org/#/c/80538/ | 15:16 |
| *** athomas has joined #tripleo | 15:17 | |
| *** slagle has joined #tripleo | 15:17 | |
| gonery | I've the feeling this documentation is a bit outdated. Does it make sense to push them? | 15:17 |
| slagle | gonery: which docs?. what's in the diskimage-builder README should be accurate | 15:20 |
| slagle | looks like some markdown syntax errors slipped through though | 15:20 |
| slagle | if you look at the source it might be clearer | 15:20 |
| slagle | i'll push a fix for the md errors | 15:21 |
| gonery | ok so, I continue my changes on tripleo-incubator/doc/source/deploying.rst | 15:22 |
| gonery | thanks | 15:22 |
| *** shakamunyi has quit IRC | 15:31 | |
| openstackgerrit | A change was merged to openstack/tripleo-incubator: doc: correctly index the disk-image-create calls https://review.openstack.org/80538 | 15:32 |
| jeblair | there are 9 tripleo-test-cloud hosts in nodepool that have been in the delete state for a long time (38-212 hours) | 15:33 |
| jeblair | lifeless, davidlenwell, Ng: ^ | 15:34 |
| *** akuznetsov has quit IRC | 15:36 | |
| *** sballe has joined #tripleo | 15:39 | |
| *** CaptTofu has joined #tripleo | 15:41 | |
| *** andreaf has quit IRC | 15:42 | |
| SpamapS | slagle: oh markdown errors, that reminds me that we need to land my markdown validator in dib :) | 15:43 |
| SpamapS | gonery: and yes, deploying.rst has not seen much revision lately | 15:43 |
| SpamapS | jeblair: davidlenwell is not associated ;) | 15:43 |
| SpamapS | jeblair: https://git.openstack.org/cgit/openstack/tripleo-incubator/tree/tripleo-cloud/tripleo-cd-admins <-- the source of truth | 15:44 |
| SpamapS | jeblair: and I'm looking into it now | 15:44 |
| *** lblanchard has quit IRC | 15:45 | |
| *** lblanchard has joined #tripleo | 15:46 | |
| SpamapS | jeblair: the ones stuck in deleted are known neutron races that require nova-compute to be restarted.. they should start disappearing soon | 15:48 |
| *** jprovazn has quit IRC | 15:49 | |
| jeblair | SpamapS: davidlenwell has a voice | 15:49 |
| SpamapS | jeblair: oh we haven't been managing voice | 15:51 |
| SpamapS | good idea though :) | 15:52 |
| SpamapS | we should tie tripleo-cd-admins into a bot or something :-P | 15:52 |
| openstackgerrit | James Slagle proposed a change to openstack/diskimage-builder: Correct README.md markdown errors https://review.openstack.org/81003 | 15:52 |
| *** marun has joined #tripleo | 15:53 | |
| jeblair | SpamapS, lifeless: feel free to self-manage the channel access list (we have a script that makes sure that the infra bots have what they need now) | 15:54 |
| *** andreaf has joined #tripleo | 15:54 | |
| jeblair | SpamapS, lifeless: i'm personally not a fan of voicing, but regardless, it's probably better to be consistent one way or the other. | 15:55 |
| jeblair | SpamapS: anyway, cool, thanks for looking into it. | 15:55 |
| *** pbelanyi has quit IRC | 15:59 | |
| SpamapS | jeblair: all gone | 16:06 |
| *** jtomasek has quit IRC | 16:06 | |
| *** akrivoka has quit IRC | 16:10 | |
| *** jrist has joined #tripleo | 16:12 | |
| *** rdopieralski has quit IRC | 16:13 | |
| *** CaptTofu has quit IRC | 16:16 | |
| *** CaptTofu has joined #tripleo | 16:17 | |
| Ng | SpamapS: we also need a playbook to start being worked up, atm I wouldn't know what to do with stuck nodepool stuff | 16:17 |
| Ng | I'm not even sure I know what nodepool is :) | 16:17 |
| *** Hefeweizen has quit IRC | 16:18 | |
| SpamapS | Ng: it is stuck nova stuff | 16:18 |
| SpamapS | Ng: and agreed, the play for things stuck with neutron errors that won't delete is | 16:19 |
| SpamapS | $ . stackrc ; for i in $(nova list | grep ci-overcloud-NovaCompute|awk -F\| '{split($7,x,"=") ; print x[2]}') ; do ssh heat-admin@$i sudo service nova-compute restart ; done | 16:19 |
| SpamapS | Ng: though I suspect we can eliminate those by updating nova | 16:20 |
| SpamapS | we should probably get _actual_ CD going for our cloud :-P | 16:20 |
| * SpamapS looks at Heat and curses | 16:20 | |
| Ng | heh | 16:20 |
| SpamapS | no retry.. no graceful shutdown.. I want to replace Heat with a moderately long bash script | 16:20 |
| *** jtomasek has joined #tripleo | 16:20 | |
| *** akuznetsov has joined #tripleo | 16:21 | |
| *** CaptTofu has quit IRC | 16:22 | |
| *** jcoufal has quit IRC | 16:24 | |
| *** cwolferh_ is now known as cwolferh | 16:25 | |
| *** e0ne has quit IRC | 16:25 | |
| *** e0ne has joined #tripleo | 16:25 | |
| * SpamapS quits whining and attacks graceful shutdown :-P | 16:25 | |
| *** jrist has quit IRC | 16:27 | |
| openstackgerrit | Chris Krelle proposed a change to openstack/tripleo-image-elements: Install Ironic and IronicClient in to nova's venv https://review.openstack.org/80725 | 16:30 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/tripleo-incubator: doc: deploying.rst improvement https://review.openstack.org/81011 | 16:34 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/tripleo-incubator: Rest syntax fixes https://review.openstack.org/81012 | 16:34 |
| SpamapS | hm | 16:35 |
| SpamapS | swift logs to syslog by default? | 16:35 |
| SpamapS | but everybody else goes to stdout? | 16:35 |
| *** hashar has quit IRC | 16:36 | |
| SpamapS | https://bugs.launchpad.net/tripleo/+bug/1293663 filed.. perhaps actually a swift bug | 16:39 |
| clarkb | SpamapS: it is inconsistent :) | 16:40 |
| clarkb | and somewhat intentionally so I guess. I gave up on making them be like everyone else a while back | 16:41 |
| SpamapS | jogo: https://review.openstack.org/#/c/80644/ <-- an example of nova failing to spin up instances because neutron-openvswitch-agent wasn't quite ready.. I think. | 16:41 |
| clarkb | swift is more interested in consistency across their releases than consistency with openstack (which isn't a bad thing, just different priorities) | 16:41 |
| SpamapS | jogo: or it is because neutron-server wasn't ready. Having a hard time following the execution. | 16:42 |
| SpamapS | I want a tool that takes all of the logs in a dir like that and just sorts them by timestamp into one big log. | 16:42 |
| SpamapS | clarkb: sucks for operators though | 16:42 |
| clarkb | SpamapS: yup | 16:42 |
| SpamapS | clarkb: and it hurts their adoption IMO | 16:42 |
| SpamapS | clarkb: so many openstack operators go "oh.. swift? Yeah, we might do that someday" | 16:42 |
| clarkb | SpamapS: I completely agree as the "operator" in this case needing to deal with swift logs | 16:43 |
| SpamapS | even though having object storage is super important for writing cloud apps | 16:43 |
| *** martyntaylor has joined #tripleo | 16:44 | |
| davidlenwell | jeblair: thank you for standing up for me.. but yeah .. I don't care about that thing .. ;) | 16:45 |
| SpamapS | davidlenwell: nice latency ;) | 16:46 |
| davidlenwell | SpamapS: I was moving to sf over the weekend .. have not touched irc in a while | 16:46 |
| *** martyntaylor1 has quit IRC | 16:47 | |
| *** rcarrillocruz1 has joined #tripleo | 16:48 | |
| *** martyntaylor has quit IRC | 16:49 | |
| SpamapS | davidlenwell: I reject your excuse, and I fire you from #tripleo. ;) | 16:49 |
| *** rcarrillocruz has quit IRC | 16:50 | |
| *** akuznetsov has quit IRC | 16:54 | |
| davidlenwell | SpamapS: was i hired by tripleo? | 16:56 |
| *** bauzas has quit IRC | 16:56 | |
| SpamapS | davidlenwell: yes, covertly. Just before I fired you. | 16:57 |
| SpamapS | davidlenwell: so congratulations! and condolences. ;) | 16:58 |
| *** sseago_ has quit IRC | 16:58 | |
| davidlenwell | :) :( | 16:58 |
| SpamapS | Crap.. CI may be broken | 17:00 |
| SpamapS | Seen two instances where we fail to reach the booted instance | 17:00 |
| *** jistr has quit IRC | 17:03 | |
| SpamapS | looks like no overcloud jobs have passed since Friday | 17:03 |
| SpamapS | oh | 17:04 |
| SpamapS | haha | 17:04 |
| * SpamapS reads /topic | 17:04 | |
| slagle | see topic :) | 17:04 |
| slagle | i've been trying to run through devtest locally | 17:04 |
| * SpamapS should probably _start_ his day with email, rather than open it 2 hours in | 17:05 | |
| *** e0ne has quit IRC | 17:13 | |
| *** morganfainberg_Z is now known as morganfainberg | 17:14 | |
| *** bauzas has joined #tripleo | 17:15 | |
| *** CaptTofu has joined #tripleo | 17:21 | |
| *** akuznetsov has joined #tripleo | 17:26 | |
| *** rcarrillocruz has joined #tripleo | 17:26 | |
| *** rcarrillocruz1 has quit IRC | 17:27 | |
| *** akuznetsov has quit IRC | 17:30 | |
| *** mrunge has quit IRC | 17:33 | |
| *** jtomasek has quit IRC | 17:35 | |
| *** akuznetsov has joined #tripleo | 17:38 | |
| *** CaptTofu has quit IRC | 17:42 | |
| *** vkozhukalov_ has joined #tripleo | 17:45 | |
| *** pblaho has quit IRC | 17:50 | |
| *** john-n-seattle has joined #tripleo | 17:52 | |
| *** petertoft has quit IRC | 17:54 | |
| *** john-n-seattle has left #tripleo | 17:55 | |
| lifeless | orning | 17:55 |
| lifeless | dprince: hi; I've been keeping an eye out for the post-ci-sprint cleanup patches but (probably me being blind) haven't spotted them... | 17:55 |
| dprince | lifeless: well. I'm still in the fix DHCP hole. | 17:56 |
| SpamapS | slagle: any progress on devtest fails? | 17:56 |
| dprince | lifeless: so I haven't sent much up yet... although I plan to tackle it this week | 17:56 |
| slagle | SpamapS: my overcloud images are building now | 17:57 |
| Lotus907efi | how is devtest failing for slagle? I am curious if it is similar to the devtest fails I am seeing | 17:57 |
| slagle | then we'll see | 17:57 |
| slagle | Lotus907efi: in the CI environment, you can't ping instances on the overcloud | 17:57 |
| Lotus907efi | oh | 17:58 |
| slagle | Lotus907efi: was just trying to see if it repro's locally or not | 17:58 |
| Lotus907efi | my builds are not getting that far | 17:58 |
| lifeless | dprince: ah cool; do you need any help ? | 17:58 |
| lifeless | I bet its qemu-nbd sucking | 17:59 |
| dprince | lifeless: maybe. I'm trying to get access to our Force 10 switches ATM. Once I have that I should be able to help myself more easily. | 17:59 |
| Lotus907efi | I have a problem with a devtest system behind a firewall and easy_install of a dependent module called pbr failing | 17:59 |
| * slagle notes that he hasn't run devtest.sh on fedora in a little while, and it's an epic fail currently | 17:59 | |
| lifeless | I've seen it hit 99% CPU routinely trying to boot the guest vm the last day or so | 17:59 |
| openstackgerrit | Thomas Goirand proposed a change to openstack/tuskar-ui: Fix deprecated use of django.conf.urls.defaults https://review.openstack.org/81035 | 17:59 |
| dprince | lifeless: but yes. another set of eyes (possibly better than mine) would be nice once I'm in the drivers seat... | 17:59 |
| dprince | lifeless: do you guys have full access to your HP switches? | 18:00 |
| * Ng dinners | 18:00 | |
| SpamapS | dprince: no | 18:02 |
| dprince | SpamapS: okay. good to know | 18:03 |
| * dprince wants the gun | 18:03 | |
| *** rcarrillocruz1 has joined #tripleo | 18:05 | |
| *** tburnes has quit IRC | 18:06 | |
| *** akuznetsov has quit IRC | 18:06 | |
| *** akuznetsov has joined #tripleo | 18:06 | |
| *** rcarrillocruz has quit IRC | 18:07 | |
| Lotus907efi | lifeless: this problem with the devtest.sh run failing on a system behind a proxy firewall is pretty serious for us. Think I should file a bug against something? | 18:10 |
| *** cwolferh_ has joined #tripleo | 18:11 | |
| *** rcarrillocruz has joined #tripleo | 18:12 | |
| *** rcarrillocruz1 has quit IRC | 18:12 | |
| *** cwolferh has quit IRC | 18:14 | |
| lifeless | dprince: no access to the switches at all | 18:15 |
| lifeless | Lotus907efi: is this the mirror issue or pbr issue? | 18:15 |
| *** rcarrillocruz1 has joined #tripleo | 18:15 | |
| Lotus907efi | no, I already filed a bug for the run-mirror issue | 18:15 |
| Lotus907efi | this is the pbr module issue | 18:16 |
| *** rcarrillocruz has quit IRC | 18:16 | |
| Lotus907efi | the bug for the run-mirror issue with failing to build httpretty is openstack-ci/+bug/1293717 | 18:17 |
| Lotus907efi | is there someway I can turn on some sort of extended debugging to see if easy_install is even trying to find or access a .pydistutils.cfg file | 18:18 |
| Lotus907efi | ? | 18:18 |
| Lotus907efi | I would hate to have to run the whole devtest.sh thing inside of strace to see if a file open takes place | 18:19 |
| Lotus907efi | the output would be enormous | 18:19 |
| lifeless | Lotus907efi: so you've got a ~/.pydistutils.cfg containing [easy_install]\nindex_url = <local mirror> ? | 18:28 |
| Lotus907efi | yes | 18:28 |
| lifeless | and what does the failure look like ? | 18:29 |
| Lotus907efi | I have a pastebin of the complete run output if that would help | 18:29 |
| lifeless | yes | 18:30 |
| *** Hefeweizen has joined #tripleo | 18:30 | |
| Lotus907efi | it looks the same as without the .pydistutils.cfg file | 18:30 |
| *** blamar has quit IRC | 18:30 | |
| lifeless | is the .pydistutils.cfg file readable by the user? e.g. if you're running as 'fred', it should be in fred's homedir and have at least mode u+r | 18:30 |
| lifeless | anyhow, point me at the pastebin please :) | 18:31 |
| Lotus907efi | http://pastebin.com/AZsnNLVi | 18:31 |
| lifeless | clarkb: SpamapS: I wonder if a new qemu landed in saucy or something to trigger this breakage. Perhaps we should configure up guestfish. How hard is that ? | 18:31 |
| Lotus907efi | I am running devtest.sh as root on this system and the file is in root's home dir | 18:31 |
| lifeless | /opt/stack/venvs/os-refresh-config/build/os-refresh-config | 18:32 |
| lifeless | thats not failing in your user area | 18:32 |
| lifeless | thats inside the image build | 18:32 |
| lifeless | which is a chroot | 18:32 |
| lifeless | Building elements: ramdisk base ubuntu deploy stackuser apt-sources use-ephemeral pypi use-ephemeral | 18:34 |
| lifeless | thats good | 18:34 |
| lifeless | it should set a index url | 18:34 |
| lifeless | but | 18:34 |
| *** sdake_ is now known as sdake | 18:34 | |
| lifeless | di you have PYPI_MIRROR_URL set ? | 18:34 |
| lifeless | looks like it | 18:35 |
| Lotus907efi | yes | 18:35 |
| lifeless | + /opt/stack/venvs/os-refresh-config/bin/pip install -U os-refresh-config | 18:35 |
| lifeless | Downloading/unpacking os-refresh-config | 18:35 |
| lifeless | http://15.126.28.132/pypi/mirror/os-refresh-config/ uses an insecure transport scheme (http). Consider using https if 15.126.28.132 has it available | 18:35 |
| lifeless | Downloading os-refresh-config-0.1.1.tar.gz | 18:35 |
| Lotus907efi | everything BEFORE the easy_install of pbr gets pulled from the pypi mirror I have setup | 18:35 |
| Lotus907efi | so if it is a chroot does the .pydistutils.cfg need to be placed inside the chroot environment somewhere? | 18:37 |
| slagle | SpamapS: from a failed ci overcloud job log: http://paste.openstack.org/show/73682/ | 18:37 |
| slagle | and i'm seeing something similar locally as well | 18:37 |
| slagle | could be this is related: https://review.openstack.org/#/c/74832/ | 18:38 |
| lifeless | Lotus907efi: is pbr in your local mirror ? | 18:39 |
| slagle | yea, exact same traceback in my local environment which is using all Fedora | 18:39 |
| Lotus907efi | yes, I checked for that | 18:39 |
| slagle | so something "upstream" is hosing us i suspect | 18:39 |
| Lotus907efi | I did a find and it is there | 18:39 |
| lifeless | slagle: SpamapS: interesting. Ok I have a suggestion. | 18:40 |
| lifeless | slagle: SpamapS: finish migrating to ml2. We're running an unsupported combo at the moment. | 18:40 |
| lifeless | slagle: SpamapS: specifically we're running the ml2 plugin on the network node, but the ovs plugin on the hypervisor. | 18:40 |
| lifeless | we should be running ml2 in both places | 18:40 |
| *** blamar has joined #tripleo | 18:41 | |
| lifeless | I'm speculating that the mismatch we have is affecting vif plug callbacks | 18:42 |
| slagle | lifeless: will see if i can switch over my running devtest | 18:42 |
| slagle | and see if that helps | 18:42 |
| lifeless | Lotus907efi: can you make a virtualenv and then pip install pbr within it successfully ? | 18:42 |
| slagle | it's definitely that change i linked that is causing the "failure" | 18:42 |
| lifeless | reading that change | 18:43 |
| lifeless | righto | 18:43 |
| lifeless | so also check with markmcclain perhaps about whether the neutron side is all landed ? | 18:43 |
| greghaynes | I was getting pbr fails earlier when using run_mirror + the binary mirror dir, havent dug much into it yet but using the regular mirror dir seemed to work for now | 18:45 |
| lifeless | Lotus907efi: if you export break=before-install and run devtest again, it will drop you into a shell after doing the pypi configuration | 18:46 |
| Lotus907efi | what do you mean by "binary mirror dir" verses "regular mirror dir" ??? | 18:46 |
| openstackgerrit | Chris Krelle proposed a change to openstack/tripleo-incubator: Add Ironic bits to _undercloud https://review.openstack.org/72969 | 18:46 |
| greghaynes | theres a /Ubuntu-13.10 dir that run-mirror makes | 18:46 |
| lifeless | oh ! I see | 18:46 |
| lifeless | Lotus907efi: so when you did 'find' | 18:46 |
| lifeless | Lotus907efi: can you pastebin the pbr files you found ? | 18:46 |
| Lotus907efi | sure | 18:46 |
| lifeless | if its all wheels, its not in the mirror as far as easy_install is concerned (AFAIK) | 18:47 |
| greghaynes | ahh | 18:47 |
| Lotus907efi | http://paste.openstack.org/show/73684/ | 18:47 |
| *** e0ne has joined #tripleo | 18:48 | |
| Lotus907efi | there is a pbr tar.gz file under a Ubuntu-13.10 directory | 18:48 |
| Lotus907efi | hmm, maybe not | 18:49 |
| greghaynes | There was for me, but still fail | 18:49 |
| Lotus907efi | there is a pbr-0.6-py2.py3-none-any.whl file under Ubuntu-13.10/pbr | 18:49 |
| Lotus907efi | but under the mirror dir there is a pbr/pbr-0.6.tar.gz file | 18:50 |
| greghaynes | Yes, I imagine lifeless has something in mind with the easy_install not knowing how to handle wheels | 18:50 |
| lifeless | well, you're pointing at mirror, not mirror/Ubuntu-13.10 right? so that should be ok | 18:50 |
| Lotus907efi | yes, I am pointing at mirror | 18:50 |
| Lotus907efi | from my devtest_variables.sh file: export PYPI_MIRROR_URL=http://15.126.28.132/pypi/mirror | 18:51 |
| *** tburnes has joined #tripleo | 18:51 | |
| *** rcarrillocruz has joined #tripleo | 18:52 | |
| Lotus907efi | and the .pydistutils.cfg has index_url = http://15.126.28.132/pypi/mirror/ | 18:52 |
| lifeless | Lotus907efi: the one inside the image? at this point in the process the one in your root user home dir is irrelevant | 18:53 |
| lifeless | Lotus907efi: diskimage-builder chroots | 18:53 |
| *** rcarrillocruz1 has quit IRC | 18:54 | |
| Lotus907efi | so as I asked before does the .pydistutils.cfg file have to be moved under the chroot somehow? | 18:54 |
| lifeless | no, the pypi element creates it | 18:54 |
| lifeless | but we can debug whats going on | 18:54 |
| Lotus907efi | how can I check if that is actually happening? | 18:54 |
| lifeless | if you export break=before-install | 18:54 |
| Lotus907efi | ok | 18:54 |
| lifeless | then run devtest | 18:54 |
| lifeless | it will drop into bash | 18:54 |
| lifeless | between the pre-install.d and install.d phases | 18:55 |
| Lotus907efi | alright and the bash shell will be inside the chroot? | 18:55 |
| lifeless | yes | 18:55 |
| lifeless | you can then make a virtualenv in there | 18:55 |
| lifeless | and try to pip install pbr | 18:55 |
| Lotus907efi | ok, then I just check for the existence of the file | 18:55 |
| lifeless | or easy_install pbr | 18:55 |
| Lotus907efi | how do I make a virtualenv? I have not done that before | 18:55 |
| lifeless | and strace or whatnot in a very granular fashion | 18:55 |
| lifeless | get into the shell | 18:55 |
| lifeless | I'll lookup some details for you | 18:56 |
| Lotus907efi | alright, let me give this a try | 18:56 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Use xtradb-cluster in place of mysql https://review.openstack.org/75540 | 18:56 |
| lifeless | once you're in the shell | 18:57 |
| Lotus907efi | so just in the shell environment before I run devtest.sh --trash-my-machine I do "export break=before-install" then run devtest.sh --trash-my-machine | 18:57 |
| lifeless | yes | 18:57 |
| lifeless | once you're in the shell | 18:57 |
| lifeless | cd to /tmp/in_target.d/install.d/ | 18:57 |
| lifeless | we'll run a few helper scripts that should succeed | 18:57 |
| Lotus907efi | ok, I am going to let this go while I make a quick run to the post office | 18:57 |
| Lotus907efi | I have to drop something off for my stepson | 18:57 |
| lifeless | 00-baseline-environment and 00-ccache and 01-install-pip | 18:58 |
| lifeless | sure | 18:58 |
| lifeless | after they succeed | 18:58 |
| Lotus907efi | and I will check in as soon as I get back | 18:58 |
| lifeless | we can step through the behaviour of 01-os-refresh-config line by line | 18:58 |
| *** rpodolyaka1 has joined #tripleo | 19:02 | |
| SpamapS | lifeless: sure that makes sense. I really haven't ever looked closely. | 19:03 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Use xtradb-cluster in place of mysql https://review.openstack.org/75540 | 19:14 |
| greghaynes | -> noms | 19:14 |
| *** weshay has quit IRC | 19:19 | |
| *** petertoft has joined #tripleo | 19:24 | |
| *** vkozhukalov_ has quit IRC | 19:25 | |
| SpamapS | slagle: over here now.. so we have a lot of settings in ovs_neutron_plugin.ini only... | 19:29 |
| SpamapS | slagle: feels like it would be helpful to have an actual neutron expert review that | 19:30 |
| *** fandi has joined #tripleo | 19:30 | |
| *** jang1 has joined #tripleo | 19:31 | |
| *** hashar has joined #tripleo | 19:48 | |
| openstackgerrit | A change was merged to openstack/diskimage-builder: Support adding DHCP interfaces one at a time. https://review.openstack.org/78711 | 19:52 |
| *** fandi has quit IRC | 19:54 | |
| openstackgerrit | A change was merged to openstack/tripleo-image-elements: Install Ironic and IronicClient in to nova's venv https://review.openstack.org/80725 | 19:57 |
| *** weshay has joined #tripleo | 19:59 | |
| openstackgerrit | Richard Su proposed a change to openstack/tripleo-image-elements: Update selinux policies for mysqld https://review.openstack.org/74607 | 20:00 |
| *** dprince has quit IRC | 20:01 | |
| lifeless | Ng: aiee why are you landig CI-failed patches. | 20:08 |
| Ng | lifeless: I realised I was being stupid after those two and stopped :/ | 20:09 |
| * SpamapS gives Ng a blanket and walks him over to the fire.. there there.. it's Monday... have some soup | 20:09 | |
| * SpamapS motions to the executioner to start sharpening the axe behind Ng's back | 20:10 | |
| greghaynes | I wouldnt trust that soup, either | 20:10 |
| Ng | hah | 20:10 |
| Ng | I was intending on spending most of the evening getting things landed, and got a little ahead of myself | 20:11 |
| lifeless | Ng: it's usual to read the jenkins comments for tripleo-ci :) | 20:11 |
| SpamapS | greghaynes: it's just kava.. we don't want him to fight... ;) | 20:12 |
| Ng | lifeless: I did see them, but I thought "oh right, CI is broken" and then two patches later I was mid-review when it hit me | 20:13 |
| Ng | sorry | 20:13 |
| Ng | apparently being married makes me stupider | 20:13 |
| lifeless | :) | 20:14 |
| *** akuznetsov has quit IRC | 20:14 | |
| SpamapS | Ng: well we could have told you that... | 20:15 |
| lifeless | marriage is proof of stupidity? Not sure I agree... | 20:15 |
| SpamapS | No it is the beginning of stupidity. | 20:16 |
| SpamapS | "fat, dumb, and happy.." | 20:16 |
| lifeless | SpamapS: ah, thats sugar :) | 20:17 |
| * SpamapS goes for a quick walk 'round the block to counter the first one | 20:17 | |
| SpamapS | lifeless: you're like a dog on a bone | 20:17 |
| Ng | a low sugar bone | 20:17 |
| lifeless | SpamapS: since I just finished breakfast - 2 rashers of bacon and three eggs... I sure am | 20:17 |
| *** rcarrillocruz1 has joined #tripleo | 20:17 | |
| lifeless | SpamapS: I've lost another 2Kg since the sprint | 20:17 |
| Ng | I still have no idea how you make that work, I can barely find time to get a single piece of jam toast in my face in the entire kids-breakfasting process | 20:18 |
| SpamapS | I eat before the children wake. | 20:18 |
| lifeless | Ng: cynthia had an egg, 1/2 rasher of bacon and is topping up with a yoghurt | 20:18 |
| *** rcarrillocruz has quit IRC | 20:18 | |
| Ng | SpamapS: good grief, I'd be getting up yesterday | 20:18 |
| lifeless | Ng: of course, I let Cynthia eat herself... :) | 20:19 |
| Ng | I'd pass myself on the way to bed | 20:19 |
| SpamapS | Usually 1 egg and either a yogurt or chunk of buttered toast. | 20:19 |
| SpamapS | Ah, my children get up at 0645 - 0700 | 20:19 |
| SpamapS | I try to wake around 0630 | 20:19 |
| *** rcarrillocruz has joined #tripleo | 20:19 | |
| Ng | 0645. LUXURY! | 20:19 |
| SpamapS | Training. | 20:19 |
| SpamapS | and.. letting them stay up until 2000, which is a bit unorthodox. | 20:20 |
| Ng | hah | 20:20 |
| Ng | Training. | 20:20 |
| Ng | and cheating! | 20:20 |
| SpamapS | but seriously, my 2 year old is the laziest baby ever. He has slept 10 - 12 hours straight through the night since he was 12 months old. | 20:20 |
| SpamapS | and 8+ hours in a row since he was 6 months | 20:21 |
| SpamapS | so also: genes. | 20:21 |
| SpamapS | ok.. walk.. must get back before devtest finally finishes | 20:21 |
| *** rcarrillocruz1 has quit IRC | 20:22 | |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/tripleo-incubator: Rest syntax fixes https://review.openstack.org/81012 | 20:36 |
| openstackgerrit | Gonéri Le Bouder proposed a change to openstack/tripleo-incubator: doc: deploying.rst improvement https://review.openstack.org/81011 | 20:36 |
| slagle | ugh, how are we supposed to get the admin tenant id into neutron.conf? | 20:37 |
| *** julim_ has quit IRC | 20:37 | |
| slagle | that's not known until init-keystone has run | 20:39 |
| Lotus907efi | lifeless: back from post office | 20:39 |
| Lotus907efi | but I do not have a shell prompt | 20:39 |
| Lotus907efi | it says: --------------------- END PROFILING --------------------- | 20:39 |
| Lotus907efi | Starting debug shell. Exit to resume building. | 20:39 |
| Lotus907efi | At stage before-install | 20:39 |
| Lotus907efi | but when I hit enter/return I do not get any sort of shell prompt | 20:40 |
| Lotus907efi | am I supposed to have a shell prompt here? | 20:42 |
| *** slagle changes topic to "FIREDRILL overcloud ci job not passing (https://bugs.launchpad.net/tripleo/+bug/1293782) | TripleO | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt" | 20:43 | |
| lifeless | Lotus907efi: you're in a shell :) | 20:45 |
| lifeless | Lotus907efi: type 'ls' and enter | 20:45 |
| Lotus907efi | hmm, ok - weird but ok | 20:46 |
| lifeless | Lotus907efi: one reason you might not have a prompt is if stdout is a pipe, and I have a terrible suspicion devtest has done that to you | 20:46 |
| lifeless | Lotus907efi: please file a bug about that | 20:46 |
| *** victor_lowther__ is now known as victor_lowther | 20:46 | |
| Lotus907efi | umm, I redirected stdout to a tee command so I could capture a log | 20:47 |
| Lotus907efi | whoops, my bad | 20:47 |
| Lotus907efi | ok, in root's home dir in this chroot env there is indeed a .pydistutils.cfg file | 20:51 |
| Lotus907efi | and it seems to have the right stuff in it | 20:51 |
| *** rpodolyaka1 has quit IRC | 20:52 | |
| *** rpodolyaka1 has joined #tripleo | 20:53 | |
| Lotus907efi | so I have done "cd /tmp/in_target.d/install.d/" and I see lots of stuff there. You mentioned running 00-baseline-environment and 00-ccache and 01-install-pip | 20:53 |
| Lotus907efi | should I do that now? | 20:53 |
| lifeless | yes | 20:55 |
| Lotus907efi | already ran 00-baseline-environment and 00-ccache | 20:56 |
| Lotus907efi | now going to run 01-install-pip | 20:56 |
| *** rpodolyaka1 has quit IRC | 20:57 | |
| Lotus907efi | crap, got an error when it tried to get setuptools | 20:59 |
| Lotus907efi | Could not find any downloads that satisfy the requirement setuptools | 20:59 |
| Lotus907efi | Some externally hosted files were ignored (use --allow-external setuptools to allow). | 20:59 |
| Lotus907efi | Cleaning up... | 20:59 |
| Lotus907efi | No distributions at all found for setuptools | 20:59 |
| Lotus907efi | that worked before just fine | 20:59 |
| lifeless | how very odd | 21:00 |
| Lotus907efi | hmm | 21:00 |
| Lotus907efi | I know why | 21:00 |
| Lotus907efi | I have proxy env varis set | 21:00 |
| Lotus907efi | and it did not pick up the right no_proxy one with the mirror system listed | 21:01 |
| Lotus907efi | let me try something here | 21:01 |
| Lotus907efi | ok, I fixed it | 21:03 |
| Lotus907efi | ok, 01-install-pip finished successfully | 21:03 |
| Lotus907efi | it said "Successfully installed virtualenv" | 21:04 |
| Lotus907efi | so what now? | 21:04 |
| lifeless | well | 21:05 |
| lifeless | lets try the 01-os-refresh-config script | 21:06 |
| lifeless | in case that thing you fixed fixes it too | 21:06 |
| *** e0ne has quit IRC | 21:06 | |
| *** e0ne has joined #tripleo | 21:07 | |
| *** dguerri_ is now known as dguerri | 21:08 | |
| *** rcarrillocruz1 has joined #tripleo | 21:09 | |
| *** lblanchard has quit IRC | 21:09 | |
| Lotus907efi | hmmm, so I stopped the whole process and re-ran devtest.sh --trash-my-machine without my redirection piped to a tee command | 21:09 |
| Lotus907efi | and I still do not get a shell prompt | 21:10 |
| lifeless | Lotus907efi: right, devtest does it's own tees | 21:10 |
| lifeless | Lotus907efi: thats why I asked you to file a bug :) | 21:10 |
| Lotus907efi | but anyway I cd'ed to /tmp/in_target.d/install.d/ and then ran 00-baseline-environment | 21:10 |
| *** e0ne has quit IRC | 21:10 | |
| Lotus907efi | then I ran 00-ccache | 21:11 |
| Lotus907efi | but it has not returned yet | 21:11 |
| Lotus907efi | and I cannot enter any more commands | 21:11 |
| *** hashar is now known as hasharMeeting | 21:11 | |
| lifeless | odd ;) | 21:11 |
| Lotus907efi | ok, I will file a bug on this no shell prompt | 21:12 |
| *** rcarrillocruz has quit IRC | 21:12 | |
| Lotus907efi | ok, it just took quite a while | 21:12 |
| Lotus907efi | no I can do "ls" again | 21:12 |
| Lotus907efi | now going to run 01-install-pip | 21:12 |
| Lotus907efi | ok, that ran ok | 21:13 |
| Lotus907efi | now for 01-os-refresh-config | 21:13 |
| Lotus907efi | Command python setup.py egg_info failed with error code 1 in /opt/stack/venvs/os-refresh-config/build/os-refresh-config | 21:14 |
| Lotus907efi | socket.timeout: timed out | 21:14 |
| *** e0ne has joined #tripleo | 21:15 | |
| Lotus907efi | in the apache access log on my mirror system the last two lines in the log say: | 21:15 |
| Lotus907efi | 10.23.16.147 - - [17/Mar/2014:15:13:38 -0600] "GET /pypi/mirror/os-refresh-config/ HTTP/1.1" 200 448 "-" "pip/1.5.4 CPython/2.7.5+ Linux/3.11.0-12-generic" | 21:15 |
| Lotus907efi | 10.23.16.147 - - [17/Mar/2014:15:13:38 -0600] "GET /pypi/mirror/os-refresh-config/os-refresh-config-0.1.1.tar.gz HTTP/1.1" 200 8722 "-" "pip/1.5.4 CPython/2.7.5+ Linux/3.11.0-12-generic" | 21:15 |
| *** rwsu has quit IRC | 21:16 | |
| Lotus907efi | so it fetched pypi/mirror/os-refresh-config/os-refresh-config-0.1.1.tar.gz | 21:16 |
| Lotus907efi | but nothing after that | 21:16 |
| lifeless | ok | 21:16 |
| lifeless | so thats the problem reproduced | 21:17 |
| lifeless | now | 21:17 |
| lifeless | if you look in /opt/stack/venvs/ | 21:17 |
| lifeless | and delete os-refresh-config | 21:17 |
| lifeless | we're ready to debug step by step | 21:17 |
| Lotus907efi | it did catch a pip.log file with the failure in it | 21:17 |
| Lotus907efi | ok | 21:17 |
| lifeless | so the script we're going to be stepping through is elements/os-refresh-config/install.d/01-os-refresh-config in tripleo-image-elements (but I'm going to feed you commands one at a time) | 21:18 |
| lifeless | once you've deleted /opt/stack/venvs/os-refresh-config | 21:18 |
| lifeless | then | 21:18 |
| lifeless | virtualenv --setuptools /opt/stack/venvs/os-refresh-config | 21:18 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/pip install -U pip | 21:18 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/pip install -U 'setuptools>=1.0' | 21:18 |
| Lotus907efi | ok, let me do this one step at a time | 21:19 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/pip install -U pbr | 21:19 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/pip install -U os-refresh-config | 21:19 |
| Lotus907efi | so /opt/stack/venvs/os-refresh-config is a directory | 21:19 |
| Lotus907efi | with stuff under it | 21:20 |
| Lotus907efi | you want me to delete it all? | 21:20 |
| Lotus907efi | the directory and all of it's contents? | 21:21 |
| lifeless | yes | 21:21 |
| Lotus907efi | ok, removing whole directory | 21:22 |
| *** rpodolyaka1 has joined #tripleo | 21:23 | |
| Lotus907efi | ran "virtualenv --setuptools /opt/stack/venvs/os-refresh-config" and it said it said: "Installing setuptools, pip...done." | 21:23 |
| Lotus907efi | so onto the next command | 21:23 |
| Lotus907efi | and "/opt/stack/venvs/os-refresh-config/bin/pip install -U pip" said: Requirement already up-to-date: pip in /opt/stack/venvs/os-refresh-config/lib/python2.7/site-packages | 21:24 |
| Lotus907efi | Successfully installed setuptools | 21:24 |
| Lotus907efi | Successfully installed pbr | 21:25 |
| Lotus907efi | Successfully installed os-refresh-config | 21:25 |
| Lotus907efi | now what? | 21:25 |
| lifeless | ok so we have a workaround | 21:26 |
| lifeless | but lets see if we can get more data on the failure | 21:26 |
| lifeless | delete the directory again | 21:26 |
| lifeless | then do | 21:26 |
| Lotus907efi | alright | 21:26 |
| lifeless | virtualenv --setuptools /opt/stack/venvs/os-refresh-config | 21:26 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/pip install -U pip | 21:26 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/pip install -U 'setuptools>=1.0' | 21:26 |
| lifeless | /opt/stack/venvs/os-refresh-config/bin/easy_install pbr | 21:27 |
| *** rpodolyaka1 has quit IRC | 21:27 | |
| Lotus907efi | whoops, seem to have a problem with the install pbr | 21:29 |
| Lotus907efi | Reading http://15.126.28.132/pypi/mirror/pbr/ | 21:29 |
| Lotus907efi | error: None | 21:29 |
| Lotus907efi | it worked before | 21:29 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Add mysql-mariadb-compat element https://review.openstack.org/78034 | 21:29 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Add cinder-lio element https://review.openstack.org/78463 | 21:29 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Don't install percona-toolkit if distro is RHEL7 https://review.openstack.org/78032 | 21:29 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Factor out tgt-specific parts of cinder element https://review.openstack.org/78462 | 21:29 |
| openstackgerrit | Ben Nemec proposed a change to openstack/tripleo-image-elements: Allow os-svc-daemon to specify dir name with -d option https://review.openstack.org/78033 | 21:29 |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Add RHEL 7 image element https://review.openstack.org/74146 | 21:30 |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Add redhat-common element https://review.openstack.org/74866 | 21:30 |
| openstackgerrit | Ben Nemec proposed a change to openstack/diskimage-builder: Use redhat-common in fedora element https://review.openstack.org/74867 | 21:30 |
| Lotus907efi | easy_install pbr failed | 21:30 |
| lifeless | Lotus907efi: great | 21:30 |
| *** rwsu has joined #tripleo | 21:31 | |
| Lotus907efi | that's great? | 21:31 |
| lifeless | strace -o /tmp/pbr-fail.log -s 512 /opt/stack/venvs/os-refresh-config/bin/easy_install pbr | 21:31 |
| lifeless | Lotus907efi: means we have a small codepath to analyse | 21:31 |
| Lotus907efi | ah | 21:31 |
| Lotus907efi | ok, so run that strace? | 21:33 |
| Lotus907efi | want me to try and redirect the output to a file? | 21:34 |
| Lotus907efi | oh, nevermind | 21:34 |
| Lotus907efi | I see the output file now | 21:34 |
| *** beekneemech is now known as bnemec | 21:36 | |
| Lotus907efi | that file is 800+KB | 21:36 |
| Lotus907efi | you want it put somewhere? | 21:36 |
| Lotus907efi | hmm, looking at the bottom of that file I see where there was a stat attempted of /root/.pypirc which did not exist but I do not see a mention of a .pydistutils.cfg so far | 21:40 |
| Lotus907efi | hmmm, way way earlier in the output I see .pydistutils.cfg being opened | 21:42 |
| Lotus907efi | and something gets the index_url = http://15.126.28.132/pypi/mirror setting | 21:42 |
| lifeless | Lotus907efi: please do put it somewhere | 21:47 |
| lifeless | StevenK: C is home sick today | 21:47 |
| lifeless | greghaynes: ^ | 21:48 |
| lifeless | I'm looking for a volunteer to insert | 21:48 |
| lifeless | /opt/stack/venvs/$thing/bin/pip install -U pbr | 21:48 |
| Lotus907efi | is 800+KB too big to pastebin? it seems like the last 100 - 200 lines is where the interesting stuff takes place anyway | 21:48 |
| lifeless | into all our venv creation stuff right after the setuptools | 21:48 |
| * greghaynes reads context | 21:48 | |
| lifeless | Lotus907efi: if you compress it and attach it to the bug | 21:48 |
| lifeless | Lotus907efi: that would be great | 21:48 |
| lifeless | Lotus907efi: if there isn't a bug, please file one on http://pad.lv/b/tripleo | 21:49 |
| *** petertoft has quit IRC | 21:49 | |
| Lotus907efi | ok, I can do that .... except I don't actually have a bug open on this particular problem as of yet | 21:49 |
| Lotus907efi | but I can open one | 21:49 |
| Lotus907efi | ok, filling bug | 21:49 |
| lifeless | my proposal here is that we simultaneously workaround the problem by pip installing pbr and gather data to improve tooling or whatever to avoid the problem if someone triggers easy_install some other way | 21:49 |
| *** rcarrillocruz has joined #tripleo | 21:54 | |
| greghaynes | Where do we currently do the easy_install pbr from? | 21:56 |
| *** rcarrillocruz1 has quit IRC | 21:56 | |
| greghaynes | I notice you wre asking Lotus907efi to to it just before the o-r-c install... its a dep from that? | 21:57 |
| Lotus907efi | yes | 21:57 |
| * greghaynes rtfs's o-r-c | 21:58 | |
| *** CaptTofu has joined #tripleo | 21:58 | |
| greghaynes | ok, I dont get why its easy_installing rather than pip installing if we are pip installing o-r-c in the first place | 22:00 |
| *** bauzas has quit IRC | 22:00 | |
| greghaynes | but yes for fast fix we can jsut stick that as a pip install beforehand... | 22:00 |
| *** e0ne has quit IRC | 22:02 | |
| greghaynes | Lotus907efi: Got a bug #? | 22:05 |
| Lotus907efi | working on it now | 22:06 |
| Lotus907efi | describing the conditions is complex | 22:06 |
| greghaynes | ty. Yep :) | 22:06 |
| Lotus907efi | since this only seems to occur when the system is behind a firewall and requires proxy settings | 22:06 |
| SpamapS | Lotus907efi: specifically there is some weirdness with the HP corporate firewall and the way pip/easy_install download via https | 22:07 |
| SpamapS | Lotus907efi: I'd be curious to see if you had the same problems bouncing through the SOCKS proxy. | 22:07 |
| SpamapS | which is effectively identical when you're doing https anyway | 22:08 |
| Lotus907efi | hmmm | 22:08 |
| * SpamapS grumbles something about allowing SSH out but not HTTPS ... | 22:08 | |
| Lotus907efi | I don't think I have tried a socks proxy | 22:08 |
| greghaynes | The require https proxy baffles me | 22:09 |
| SpamapS | Lotus907efi: there's some instructions in the internal wiki's.. I'll dig them out if you don't have them at hand. | 22:09 |
| Lotus907efi | https://bugs.launchpad.net/tripleo/+bug/1293812 | 22:10 |
| Lotus907efi | file bug: https://bugs.launchpad.net/tripleo/+bug/1293812 | 22:10 |
| jang1 | Lotus907efi: I've something that works if you have a pypi mirror to point at: brief summary notes on the wiki under my personal space (jan grant) | 22:10 |
| Lotus907efi | I do have a pypi mirror | 22:11 |
| jang1 | by "that works" I mean the mob in Bristol have been going through a similar degree of pain, but it's working now. | 22:11 |
| greghaynes | haha - my browser happily tries to open the bz2 as a text file | 22:11 |
| greghaynes | on the bug report | 22:11 |
| greghaynes | Methinks the lp server doesnt mimetype that correctly | 22:11 |
| Lotus907efi | hmm, not my fault if your browser has broken-ness :) | 22:12 |
| Lotus907efi | jang1: where do I find these notes? | 22:12 |
| jang1 | hang on, let me see if I can see it from here... | 22:13 |
| Lotus907efi | you have to put on your glasses to see it from there? :D | 22:13 |
| Lotus907efi | or you have to use binoculars? | 22:13 |
| jang1 | I'm not on any vpns at the mo. | 22:14 |
| Lotus907efi | ok, I was just kidding | 22:14 |
| Lotus907efi | just punning on the "can see it from here" | 22:14 |
| jang1 | something I discovered today was that if you have "extra-index-url" set in your pip conf it seems to leak into, and bugger up, the chroot environments | 22:15 |
| jang1 | so, ditch that too. | 22:15 |
| Lotus907efi | I don't think I have that | 22:15 |
| Lotus907efi | I have index_url in .pydistutils.cfg | 22:16 |
| Lotus907efi | but not extra-index-url | 22:16 |
| SpamapS | jang1: that seems like a bug.. I don't see anywhere in any extra-data.d scripts that would pull in the local pip.conf | 22:17 |
| jang1 | no, it surprised me too - I thought those were pristinely generated. | 22:17 |
| jang1 | at the time I just said "... don't do that, rely on PYPI_MIRROR_URL" and left it at that. | 22:17 |
| jang1 | I didn't dig into where ti came from, but if I've time tomorrow I'll have an experiment. | 22:18 |
| jang1 | since most of tomorrow will be spent waiting on debootstrap, I should have plenty of time :-) | 22:18 |
| Lotus907efi | ok, am I still filling a bug on the fact that devtest.sh does not actually provide a shell prompt when "export break=before-install" is set? | 22:18 |
| SpamapS | jang1: could you report the bug now? It's ok if it isn't supported by much evidence.. but if others who subscribe to the bugs have seen it it may remind them to report | 22:18 |
| jang1 | will do. | 22:18 |
| SpamapS | jang1: I'd report against tripleo .. but it may be diskimage-builder too | 22:19 |
| jang1 | done | 22:22 |
| jang1 | and now, it's a school night, and I'm contractually pobliged to retire. | 22:22 |
| jang1 | night all. | 22:22 |
| SpamapS | argh.. my overcloud failed with NoValidHost | 22:22 |
| *** jang1 has quit IRC | 22:23 | |
| *** rpodolyaka1 has joined #tripleo | 22:24 | |
| SpamapS | wait_for 60 1 [ "\$(nova hypervisor-stats | awk '\$2==\"count\" { print \$4}')" != "0" ] | 22:24 |
| SpamapS | still racey | 22:24 |
| SpamapS | it was likely 2, instead of 3 | 22:24 |
| Lotus907efi | filled bug on the no shell prompt from devtest.sh when break=before-install is set: https://bugs.launchpad.net/tripleo/+bug/1293821 | 22:27 |
| *** rcarrillocruz1 has joined #tripleo | 22:28 | |
| *** rpodolyaka1 has quit IRC | 22:28 | |
| *** rcarrillocruz has quit IRC | 22:29 | |
| *** e0ne has joined #tripleo | 22:30 | |
| * tchaypo finally has suspend/resume on his laptop | 22:33 | |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81101 | 22:33 |
| greghaynes | Lotus907efi: can you see if that ^ fixes it for you | 22:33 |
| greghaynes | Fixed my issue when trying to use the binary pip mirror as well | 22:34 |
| SpamapS | tchaypo: bragger | 22:34 |
| greghaynes | suspend/resume 'just worked' on ubuntu for me | 22:35 |
| greghaynes | wow, devtest is silly fast with that fix and binary pip mirror | 22:36 |
| SpamapS | greghaynes: yeah, binary pip mirror is huge, I want to get us using that in CI | 22:37 |
| tchaypo | It just worked as soon as i enabled the nvidia driver instead of using noveau | 22:38 |
| greghaynes | Oh, youre on a different laptop than the rest of us? | 22:38 |
| tchaypo | 8570w | 22:38 |
| *** e0ne has quit IRC | 22:38 | |
| greghaynes | yep | 22:39 |
| tchaypo | don't know what you have so I'm not sure if that's different | 22:39 |
| greghaynes | 9470m | 22:39 |
| greghaynes | Is it the huge one adam_g has also? | 22:39 |
| SpamapS | tchaypo: oh lord.. noveau is terrible | 22:39 |
| SpamapS | but | 22:39 |
| SpamapS | I'm sad that those new ones are Nvidia | 22:39 |
| SpamapS | I rather like Intel. | 22:39 |
| tchaypo | greghaynes: oh no, that's a bigger number, I'm jealous now | 22:39 |
| greghaynes | haha | 22:39 |
| *** openstackgerrit has quit IRC | 22:39 | |
| SpamapS | 9470m are the EliteBook's | 22:40 |
| *** openstackgerrit has joined #tripleo | 22:40 | |
| tchaypo | you mean utlrabook? | 22:40 |
| Lotus907efi | hmmm, let me look at it and see | 22:40 |
| * greghaynes jsut got slice battery in mail today | 22:40 | |
| adam_g | yeah nvidia/nouveau has been giving me headaches on my zbook | 22:40 |
| tchaypo | oh, i see, elitebook, is the HP custom name for an ultrabook? | 22:40 |
| greghaynes | tchaypo: not really - its more like a mac clone | 22:40 |
| adam_g | in fact, its currently sitting in its initramfs until i can fix up modprobe.conf | 22:41 |
| tchaypo | greghaynes, adam_g - I've been adding notes to https://etherpad.openstack.org/p/tripleo-newdev-notes - can you guys spot anything obvious that I'm missing? | 22:42 |
| greghaynes | nice! | 22:44 |
| *** hasharMeeting has quit IRC | 22:44 | |
| greghaynes | ooo, this devtest run might clock in at under 20mins | 22:47 |
| * greghaynes crosses fingers | 22:47 | |
| *** andreaf has quit IRC | 22:50 | |
| *** michchap has joined #tripleo | 22:50 | |
| adam_g | tchaypo, lgtm | 22:50 |
| cody-somerville | What kind of problems are you guys having with the zbooks? | 22:53 |
| *** michchap_ has quit IRC | 22:53 | |
| tchaypo | cody-somerville: can't get them in apj yet, so my problem is that I've got an 8570w instead (which seems perfectly functional, if a bit overweight). lifeless seems pretty happy with his | 22:55 |
| *** newell has quit IRC | 22:56 | |
| cody-somerville | ugh... | 22:57 |
| cody-somerville | tchaypo: Who told you they weren't available in APJ? | 22:57 |
| adam_g | cody-somerville, no issues with the laptop itself, just general nvidia/nouveau/linux driver PITA | 22:58 |
| cody-somerville | adam_g: Do you have the zBook 14? | 22:59 |
| tchaypo | cody-somerville: extrapolation from lifeless mentioning he wasn't able to get one here, and from chris ordering me a zbook but the 8570w getting delivered instead | 22:59 |
| adam_g | cody-somerville, 15 | 22:59 |
| Shrews | cody-somerville: loving mine | 23:01 |
| cody-somerville | adam_g: Part of the problem might be that the zbook 15 has hybrid graphics | 23:02 |
| cody-somerville | adam_g: Have you not had any luck telling it to just use the integrated Intel HD Graphics 4600? | 23:02 |
| cody-somerville | Shrews: You have the 14 though, right? | 23:02 |
| Shrews | cody-somerville: correct | 23:03 |
| adam_g | cody-somerville, have not tried that. ive just been using the proprietary nvidia drivers and they work okay, i just need to remember to get all the post-upgrade DKMS stuff in place before i upgrade my kernel next time | 23:03 |
| cody-somerville | So there's like five different laptops being used by our team now. I don't think that was intended. | 23:04 |
| SpamapS | Yeah | 23:04 |
| SpamapS | we had the 9470m's and then the zbook was a trial | 23:04 |
| cody-somerville | Ng: ping | 23:05 |
| *** blamar has quit IRC | 23:07 | |
| openstackgerrit | Clint "SpamapS" Byrum proposed a change to openstack/tripleo-incubator: Wait for enough hosts to be available in nova https://review.openstack.org/81106 | 23:10 |
| openstackgerrit | Gregory Haynes proposed a change to openstack/tripleo-image-elements: Explicitly pip install pbr before its installed as dependency https://review.openstack.org/81101 | 23:11 |
| Lotus907efi | greghaynes: I think you need to put the temp fix into os-collect-config as well | 23:12 |
| greghaynes | Yep, just ran into that - should be fixed in that update ^? | 23:13 |
| *** blamar has joined #tripleo | 23:13 | |
| cody-somerville | adam_g: tchaypo: Hey. Can you both send me the SKU Number of your laptops? | 23:15 |
| adam_g | cody-somerville, sure | 23:15 |
| *** sseago_ has joined #tripleo | 23:23 | |
| *** rpodolyaka1 has joined #tripleo | 23:24 | |
| *** arosen has joined #tripleo | 23:25 | |
| arosen | Hi, I'm looking at fixing this bug: https://bugs.launchpad.net/tripleo/+bug/1293782 but i had a quick question | 23:28 |
| arosen | it looks like tripleo-image-elements doesn't actually create keystone endpoints in that repo | 23:28 |
| *** rpodolyaka1 has quit IRC | 23:29 | |
| tchaypo | greghaynes: could the pbr issue you're looking at be caused by out-of-order dependencies? | 23:34 |
| tchaypo | I'm looking at https://bugs.launchpad.net/diskimage-builder/+bug/1292514 | 23:35 |
| lifeless | cody-somerville: zbook14 APJ - we had mail from the fulfillment person saying they weren't available | 23:36 |
| lifeless | greghaynes: yes, pip install $foo for setup_requires invokes easy_install on missing deps | 23:36 |
| lifeless | greghaynes: so manually installing pbr is a workaround | 23:37 |
| lifeless | greghaynes: and the strace should let us figure out why easy_install is bypassing the selected mirror | 23:37 |
| *** xuhaiwei has joined #tripleo | 23:37 | |
| lifeless | greghaynes: that patch needs to be much more extensive | 23:38 |
| cody-somerville | lifeless: the e-mail you got was that the particular model I ordered for you in APJ was not available. They did later provide me with a model number for zbook 14 that is available in New Zealand. | 23:38 |
| lifeless | greghaynes: heat-cfntools | 23:38 |
| lifeless | cody-somerville: ah! wow we make our life hard don't we :( | 23:38 |
| tchaypo | cody-somerville: in my case, they confirmed to chris they were shipping a zbook and then confirmed that they *had* shipped the 8570w, no explanation | 23:39 |
| lifeless | greghaynes: basically grep for bin\/pip install and plaster it everywhere | 23:39 |
| greghaynes | haha, ok | 23:40 |
| *** CaptTofu has quit IRC | 23:41 | |
| *** CaptTofu has joined #tripleo | 23:41 | |
| Lotus907efi | I clean out my sources and started completely from scratch with a fresh checkout of HEAD | 23:43 |
| greghaynes | tchaypo: maybe but that seems unlikely | 23:43 |
| Lotus907efi | and now python-cinderclient is also failing due to easy_install of pbr | 23:43 |
| greghaynes | Yep, I think thats what lifeless was getting at with plaster it everywhere | 23:44 |
| Lotus907efi | yeah, crud | 23:44 |
| Lotus907efi | I thought we were getting close | 23:44 |
| greghaynes | eh, I think he already did the hard part in finding the workaround | 23:45 |
| *** CaptTofu has quit IRC | 23:46 | |
| * tchaypo reads dib-seed.log | 23:49 | |
| * tchaypo realises just how many random env vars it relies on | 23:49 | |
| tchaypo | it==diskimage-builder | 23:50 |
| openstackgerrit | Aaron Rosen proposed a change to openstack/tripleo-image-elements: Add configuration required for neutron-nova events https://review.openstack.org/81120 | 23:51 |
| greghaynes | Lotus907efi: with the latest revision I dont think python-cinderclient should fail | 23:51 |
| Lotus907efi | ok | 23:51 |
| greghaynes | there are other things that should though, so maybe a couple more... | 23:51 |
| Lotus907efi | so where is the python-cinderclient code at anyway? | 23:53 |
| *** blamar has quit IRC | 23:53 | |
| Lotus907efi | I am not seeing any cinderclient code changes mentioned in https://review.openstack.org/#/c/81101/ | 23:54 |
| greghaynes | Lotus907efi: I hope the install-openstack-client change takes care of the cinderclient install | 23:59 |
| greghaynes | did on my box... | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!