| *** sdake has joined #openstack-ansible | 00:01 | |
| coolj | shanecc: just to document what we figured out, the nova-key tag has to be used at the same time as the nova-key-distribute tag to populate the nova_pubkey host var | 00:01 |
|---|---|---|
| *** sdake has quit IRC | 00:03 | |
| *** sdake has joined #openstack-ansible | 00:07 | |
| *** elo has quit IRC | 00:08 | |
| *** sacharya has joined #openstack-ansible | 00:11 | |
| *** mrda has joined #openstack-ansible | 00:11 | |
| *** sacharya has quit IRC | 00:15 | |
| *** mss has quit IRC | 00:16 | |
| *** jaybeers has quit IRC | 00:17 | |
| *** rady has quit IRC | 00:33 | |
| BjoernT | coolj: this task is one of those who can't be limited. The iteration over the nova_compute group runs into the situation were it puts one key from the limited host into the ansible fact and the loop inside the distribution task try to write a key from all host inside the group to the authorized_keys file | 00:35 |
| coolj | yeah, worked after running with --tags nova-key,nova-key-distribute :D | 00:38 |
| *** sdake has quit IRC | 01:10 | |
| bgmccollum | with or without limit? | 01:29 |
| coolj | shanecc: ^ | 01:29 |
| bgmccollum | i hate limit | 01:29 |
| bgmccollum | burn it | 01:29 |
| bgmccollum | please | 01:29 |
| *** karimb has quit IRC | 01:30 | |
| coolj | can limits be useful :P | 01:32 |
| coolj | ansible rabbit --limit '*704118*,*704117*' -m shell -a 'service rabbitmq-server stop' | 01:32 |
| coolj | ansible rabbit --limit '*704118*,*704117*' -m shell -a 'service rabbitmq-server start' | 01:32 |
| coolj | err limits can* | 01:32 |
| bgmccollum | coolj: limits are something useful if you know the "limits" of using them... | 01:33 |
| bgmccollum | adhoc ansible commands...yeah i can get behind that...with playbooks...expect epic failures | 01:34 |
| bgmccollum | so many interdependencies of playbooks expecting certain hosts to exists, and certain tasks to have run | 01:34 |
| bgmccollum | its safer to target with tags, and not limits... | 01:35 |
| bgmccollum | and dinner | 01:35 |
| shanecc | originally it was without --limit | 01:37 |
| shanecc | just ran setup-everything.yml with no limits | 01:37 |
| *** BjoernT has quit IRC | 01:45 | |
| coolj | all this no limit talk reminds me of bad 90's hip hop | 01:51 |
| *** sacharya has joined #openstack-ansible | 01:54 | |
| *** sdake has joined #openstack-ansible | 01:56 | |
| *** sacharya has quit IRC | 01:59 | |
| shanecc | do doing os-nova-install with --tags nova-key,nova-key-distribute completes. then running os-nova-install by itself completes | 02:06 |
| shanecc | now running setup-everything, should finish now I think | 02:06 |
| shanecc | thanks coolj! | 02:06 |
| shanecc | still not sure what caused it to begin with | 02:06 |
| *** Bjoern_ has joined #openstack-ansible | 02:11 | |
| *** sdake has quit IRC | 02:15 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Functional backport for the new repo-build process https://review.openstack.org/244215 | 02:31 |
| *** rebase has quit IRC | 02:41 | |
| *** sdake has joined #openstack-ansible | 02:44 | |
| openstackgerrit | Paul Belanger proposed openstack/openstack-ansible: Rename dev-requirements.txt to test-requirements.txt https://review.openstack.org/244404 | 02:50 |
| pabelanger | ohai | 02:50 |
| pabelanger | just a heads up, I am working on standardizing the ansible-lint gate job across all modules. So, ^ is just a cosmetic change to help move openstack-ansible into that process | 02:51 |
| pabelanger | basically, we'll be adding a tox -eansible-lint job launch the lint process. Over having JJB macros or embedded scripts deal with the installation process | 02:52 |
| pabelanger | I should also note, you current gate-ansible-lint is not in the gate pipeline either. Not sure if that is intended or not | 02:55 |
| cloudnull | pabelanger: that lint job is being called from our general gate check script and we're actively moving toward simply using tox | 02:58 |
| pabelanger | cloudnull: Ya, I have some code up right now for review. If you are interested: https://review.openstack.org/#/c/244403/ | 02:59 |
| pabelanger | then the plan will be to kick the gate-{project}-ansible-lint job to use tox over sudo pip install ansible-lint | 03:00 |
| pabelanger | from JJB | 03:00 |
| cloudnull | nice | 03:01 |
| cloudnull | we should steal that too :) | 03:02 |
| pabelanger | Ya. I just added my first ansible-role into OpenStack and want to work with this team to better condense the testing infrastructure | 03:02 |
| pabelanger | https://github.com/openstack/ansible-role-nodepool FYI | 03:02 |
| pabelanger | unofficial project right now | 03:03 |
| pabelanger | still trying to figure out where it should live | 03:03 |
| pabelanger | but we already have 2 other teams using ansible, so works better if we are all using the same testing tools | 03:03 |
| cloudnull | ++ | 03:03 |
| cloudnull | we have a pr to project-config to move some of our roles into standalone roles https://review.openstack.org/#/c/243952/ | 03:04 |
| cloudnull | in the role break out we're currently have https://github.com/os-cloud/openstack-ansible-lxc_container_create/blob/master/tox.ini#L79-L97 for lint + functional testing | 03:05 |
| cloudnull | sorry ; syntax checks, lint, functional https://github.com/os-cloud/openstack-ansible-lxc_container_create/blob/master/tox.ini#L66-L97 | 03:06 |
| cloudnull | one of the the goals for this cycle is to move all of the roles that are beneficial to everyone into standalone repos. | 03:06 |
| cloudnull | so far we've tackled the base host roles , next infra, then OS_.* | 03:07 |
| pabelanger | what is ansible-role-requirements.yml ? | 03:07 |
| pabelanger | Ah | 03:07 |
| pabelanger | found it | 03:07 |
| pabelanger | cloudnull: okay good, so we are somewhat inline. tox -eansible-lint and tox -eansible-syntaxx | 03:08 |
| pabelanger | syntax* | 03:08 |
| cloudnull | yesir, but we should add the lint job to our main repo pipeline too | 03:09 |
| pabelanger | mind you, you don't likely want to install from galaxy for your modules. zuul-cloner might be better to get cross-repo dependencies working. | 03:09 |
| pabelanger | but I haven't tested that yet | 03:09 |
| pabelanger | either way, looks good | 03:10 |
| cloudnull | we've taken the approach to use the galaxy command however we're pointing the ref to a git repo | 03:11 |
| cloudnull | its worked well so far, however ill look at zuul cloner | 03:11 |
| cloudnull | like so https://github.com/os-cloud/openstack-ansible-lxc_container_create/blob/master/tests/ansible-role-requirements.yml | 03:12 |
| cloudnull | is what we've done thus far | 03:12 |
| *** shanecc has quit IRC | 03:14 | |
| pabelanger | Right, zuul-cloner is great because you could have a cross repo change between lxc_container_create and apt_package_pinning (for example) with out having to push untested code up to ansible-galaxy first. But like I said, i haven't fully tested it with my playbooks yet. Will need to get my 2nd module up to try it | 03:15 |
| cloudnull | pabelanger: do you know if general OpenStack has a galaxy account that we can publish roles to ? | 03:16 |
| cloudnull | at present we're just assuming the roles will be in git | 03:16 |
| pabelanger | cloudnull: not yet :) I'm actually going to add something tomorrow. Want to bounce something off -infra roots first | 03:17 |
| cloudnull | but if theres a place we can publish roles within https://galaxy.ansible.com/ that'd be great | 03:17 |
| pabelanger | Ya | 03:17 |
| cloudnull | ah | 03:17 |
| cloudnull | well if I or anyone here can help please let us know | 03:17 |
| pabelanger | When is your weekly meetings? | 03:18 |
| * pabelanger googles | 03:18 | |
| cloudnull | Thursday at 16:00 UTC. | 03:18 |
| pabelanger | tomorrow | 03:18 |
| cloudnull | https://wiki.openstack.org/wiki/Meetings/openstack-ansible | 03:18 |
| pabelanger | Adding it to calenar | 03:18 |
| pabelanger | calender* | 03:18 |
| cloudnull | sweet | 03:19 |
| pabelanger | cloudnull: But ya, I see it working the same way we do pypi. Likely the same account name too | 03:21 |
| pabelanger | think we use openstackci | 03:21 |
| pabelanger | okay, getting late here. I'll be around more to help on the infra side of things. Will be at the meeting tomorrow to for intros | 03:23 |
| cloudnull | sweet ! | 03:30 |
| cloudnull | have a good one | 03:31 |
| cloudnull | ttyt | 03:31 |
| *** mss has joined #openstack-ansible | 03:34 | |
| *** mss has quit IRC | 03:39 | |
| *** sdake has quit IRC | 03:39 | |
| *** sdake has joined #openstack-ansible | 03:40 | |
| *** Bjoern_ is now known as Bjoern_zZzZzZzZ | 03:40 | |
| *** Bjoern_zZzZzZzZ has quit IRC | 03:41 | |
| *** Bjoern_ has joined #openstack-ansible | 03:44 | |
| *** devl has joined #openstack-ansible | 03:44 | |
| openstackgerrit | Merged openstack/openstack-ansible: Deployment fix when container_networks are not set https://review.openstack.org/241293 | 03:45 |
| bgmccollum | cloudnull: mind if queue up the backport ^ to kilo? | 03:47 |
| cloudnull | that'd be great | 03:47 |
| cloudnull | ++ liberty if not already done | 03:47 |
| bgmccollum | been running the patch for a while now...works great | 03:47 |
| bgmccollum | will do | 03:48 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Deployment fix when container_networks are not set https://review.openstack.org/244424 | 03:49 |
| cloudnull | ^ done | 03:49 |
| bgmccollum | too quick | 03:49 |
| bgmccollum | spurs game just came on...lost focus | 03:50 |
| cloudnull | https://review.openstack.org/#/c/244424/ and https://review.openstack.org/#/c/244423/ | 03:50 |
| cloudnull | ha | 03:50 |
| * cloudnull doesnt watch basket ball | 03:50 | |
| bgmccollum | need to setup my liberty build jobs | 03:56 |
| *** Bjoern_ is now known as Bjoern_zZzZzZzZ | 03:58 | |
| *** Bjoern_zZzZzZzZ has quit IRC | 04:03 | |
| *** Bjoern_ has joined #openstack-ansible | 04:05 | |
| *** mss has joined #openstack-ansible | 04:10 | |
| *** mss has quit IRC | 04:15 | |
| *** elo has joined #openstack-ansible | 04:41 | |
| *** Bjoern_ has quit IRC | 04:48 | |
| *** rebase has joined #openstack-ansible | 04:50 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Functional backport for the new repo-build process https://review.openstack.org/244215 | 04:55 |
| *** mss has joined #openstack-ansible | 05:07 | |
| *** alkari has joined #openstack-ansible | 05:12 | |
| *** fawadkhaliq has joined #openstack-ansible | 05:14 | |
| *** sacharya has joined #openstack-ansible | 05:19 | |
| miguelgrinberg | alextricity: I reproduced the ec2tokens bug on my AIO, filed a bug for it: https://bugs.launchpad.net/openstack-ansible/+bug/1515485. Still haven't found the problem, will continue tomorrow. | 05:31 |
| openstack | Launchpad bug 1515485 in openstack-ansible "Heat CFN signals do not pass authorization" [Undecided,New] | 05:31 |
| *** javeriak has joined #openstack-ansible | 06:18 | |
| *** alkari has quit IRC | 06:23 | |
| *** ashishb has joined #openstack-ansible | 06:27 | |
| *** phiche has joined #openstack-ansible | 06:43 | |
| *** phiche1 has joined #openstack-ansible | 06:46 | |
| *** phiche has quit IRC | 06:48 | |
| *** jhesketh has quit IRC | 06:54 | |
| *** sacharya has quit IRC | 06:54 | |
| *** jhesketh has joined #openstack-ansible | 06:59 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Functional backport for the new repo-build process https://review.openstack.org/244215 | 07:00 |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated the py_pkgs lookup plugin for multi source https://review.openstack.org/241483 | 07:02 |
| *** sirushti has quit IRC | 07:14 | |
| *** sirushti has joined #openstack-ansible | 07:17 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Updated the py_pkgs lookup plugin for multi source https://review.openstack.org/241483 | 07:23 |
| *** phiche1 has quit IRC | 07:24 | |
| openstackgerrit | Kevin Carter proposed openstack/openstack-ansible: Functional backport for the new repo-build process https://review.openstack.org/244215 | 07:26 |
| *** phiche has joined #openstack-ansible | 07:30 | |
| *** mss has quit IRC | 07:33 | |
| prometheanfire | cloudnull: go to bed | 07:36 |
| cloudnull | prometheanfire: go to bed | 07:37 |
| prometheanfire | just woke up | 07:38 |
| cloudnull | ha | 07:39 |
| cloudnull | your an early riser :) | 07:39 |
| prometheanfire | why are there no fast foods open at night? | 07:40 |
| cloudnull | because texas | 07:41 |
| prometheanfire | :( | 07:41 |
| cloudnull | theres always Jims | 07:42 |
| prometheanfire | true | 07:44 |
| prometheanfire | also ihop | 07:44 |
| prometheanfire | about the same quality | 07:44 |
| *** slotti has joined #openstack-ansible | 07:46 | |
| cloudnull | I'm going to go crash for a bit. See you in the after now. | 07:53 |
| *** sacharya has joined #openstack-ansible | 07:55 | |
| *** sacharya has quit IRC | 07:59 | |
| *** egonzalez has joined #openstack-ansible | 08:13 | |
| *** sdake has quit IRC | 08:24 | |
| *** sdake has joined #openstack-ansible | 08:25 | |
| *** elo has quit IRC | 08:26 | |
| *** elo has joined #openstack-ansible | 08:26 | |
| *** rebase has quit IRC | 08:28 | |
| *** javeriak has quit IRC | 08:28 | |
| *** javeriak has joined #openstack-ansible | 08:28 | |
| *** rebase has joined #openstack-ansible | 08:28 | |
| *** rebase has quit IRC | 08:29 | |
| *** mss has joined #openstack-ansible | 08:30 | |
| *** javeriak has quit IRC | 08:33 | |
| *** javeriak has joined #openstack-ansible | 08:34 | |
| *** mss has quit IRC | 08:35 | |
| *** gparaskevas has joined #openstack-ansible | 08:40 | |
| *** mpavone has joined #openstack-ansible | 08:43 | |
| *** karimb has joined #openstack-ansible | 08:48 | |
| *** sdake has quit IRC | 08:49 | |
| *** sdake has joined #openstack-ansible | 08:49 | |
| *** pcaruana has joined #openstack-ansible | 08:53 | |
| *** sdake has quit IRC | 09:00 | |
| *** sdake has joined #openstack-ansible | 09:07 | |
| *** tiagogomes_ has joined #openstack-ansible | 09:29 | |
| *** openstackgerrit has quit IRC | 09:46 | |
| *** sdake has quit IRC | 09:46 | |
| *** openstackgerrit has joined #openstack-ansible | 09:47 | |
| *** karimb has quit IRC | 09:47 | |
| *** karimb has joined #openstack-ansible | 09:48 | |
| openstackgerrit | Merged openstack/openstack-ansible: Fixing /etc/rabbitmq permission bug https://review.openstack.org/243227 | 10:07 |
| *** gparaskevas_ has joined #openstack-ansible | 10:13 | |
| *** openstackgerrit has quit IRC | 10:16 | |
| *** openstackgerrit has joined #openstack-ansible | 10:17 | |
| *** sacharya has joined #openstack-ansible | 10:21 | |
| *** sacharya has quit IRC | 10:25 | |
| *** sdake has joined #openstack-ansible | 10:25 | |
| *** mss has joined #openstack-ansible | 10:32 | |
| mattt | odyssey4me cloudnull : updated https://review.openstack.org/#/c/240560/ w/ a question, can you guys offer some input? | 10:32 |
| *** mss has quit IRC | 10:36 | |
| *** tiagogomes_ has quit IRC | 10:38 | |
| *** tiagogomes_ has joined #openstack-ansible | 10:38 | |
| *** sdake has quit IRC | 10:41 | |
| *** sdake has joined #openstack-ansible | 10:41 | |
| *** javeriak_ has joined #openstack-ansible | 10:46 | |
| *** lkoranda_ has joined #openstack-ansible | 10:48 | |
| *** javeriak has quit IRC | 10:50 | |
| *** lkoranda has quit IRC | 10:51 | |
| *** lkoranda_ has quit IRC | 10:53 | |
| *** sdake has quit IRC | 10:57 | |
| *** sdake has joined #openstack-ansible | 10:57 | |
| *** lkoranda has joined #openstack-ansible | 10:58 | |
| *** javeriak_ has quit IRC | 11:04 | |
| *** vdo has joined #openstack-ansible | 11:05 | |
| *** tlian has joined #openstack-ansible | 11:09 | |
| *** javeriak has joined #openstack-ansible | 11:12 | |
| *** sdake has quit IRC | 11:13 | |
| *** sdake has joined #openstack-ansible | 11:13 | |
| *** karimb_ has joined #openstack-ansible | 11:14 | |
| *** karimb has quit IRC | 11:14 | |
| *** mgoddard_ has quit IRC | 11:19 | |
| *** mgoddard_ has joined #openstack-ansible | 11:19 | |
| *** toddnni has quit IRC | 11:28 | |
| *** sdake has quit IRC | 11:29 | |
| *** fawadkhaliq has quit IRC | 11:35 | |
| *** cloudtrainme has joined #openstack-ansible | 11:44 | |
| odyssey4me | hughsaunders git-harry can I ask you guys to cast your expert pythonic eyes on https://review.openstack.org/241483 ? | 11:47 |
| hughsaunders | odyssey4me: erm not sure I ever had those, but I'll use my slightly sleepy eyes instead | 11:47 |
| *** javeriak has quit IRC | 11:56 | |
| openstackgerrit | Merged openstack/openstack-ansible: Wait for container SSH https://review.openstack.org/243668 | 12:04 |
| *** javeriak has joined #openstack-ansible | 12:12 | |
| *** Jezogwza has joined #openstack-ansible | 12:26 | |
| *** gparaskevas_ has quit IRC | 12:28 | |
| *** Jezogwza_ has joined #openstack-ansible | 12:31 | |
| *** mss has joined #openstack-ansible | 12:32 | |
| *** Jezogwza has quit IRC | 12:33 | |
| *** Jezogwza_ is now known as Jezogwza | 12:33 | |
| *** mss has quit IRC | 12:37 | |
| *** sacharya has joined #openstack-ansible | 12:45 | |
| *** gparaskevas has quit IRC | 12:48 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Functional back port of the Liberty repo-build process https://review.openstack.org/244215 | 12:49 |
| *** sacharya has quit IRC | 12:49 | |
| *** javeriak has quit IRC | 12:53 | |
| *** vdo has quit IRC | 12:56 | |
| tiagogomes_ | :w | 12:56 |
| *** cloudtrainme has quit IRC | 12:58 | |
| *** vdo has joined #openstack-ansible | 12:59 | |
| *** cloudtrainme has joined #openstack-ansible | 13:00 | |
| *** javeriak has joined #openstack-ansible | 13:01 | |
| openstackgerrit | Merged openstack/openstack-ansible: Fix race condition for /openstack directories https://review.openstack.org/243916 | 13:11 |
| openstackgerrit | Merged openstack/openstack-ansible: Adding docs/comments for glance config https://review.openstack.org/243661 | 13:11 |
| *** woodard has joined #openstack-ansible | 13:14 | |
| *** woodard has quit IRC | 13:14 | |
| *** Jezogwza has quit IRC | 13:15 | |
| *** woodard has joined #openstack-ansible | 13:15 | |
| openstackgerrit | Merged openstack/openstack-ansible: Rename dev-requirements.txt to test-requirements.txt https://review.openstack.org/244404 | 13:17 |
| *** vdo has quit IRC | 13:23 | |
| *** tlian has quit IRC | 13:23 | |
| *** tlian has joined #openstack-ansible | 13:24 | |
| *** moravec has quit IRC | 13:37 | |
| *** javeriak_ has joined #openstack-ansible | 13:38 | |
| *** javeriak has quit IRC | 13:38 | |
| *** vdo has joined #openstack-ansible | 13:39 | |
| *** sdake has joined #openstack-ansible | 13:46 | |
| *** mgariepy has quit IRC | 13:47 | |
| *** mgariepy has joined #openstack-ansible | 13:50 | |
| odyssey4me | hughsaunders https://review.openstack.org/#/c/241322/ | 13:55 |
| odyssey4me | hughsaunders https://review.openstack.org/#/c/241592/3/doc/source/releasenotes/notes/rabbitmq-ssl-bfde279f608c1c58.yaml,cm | 13:55 |
| mhayden | buenos dias | 13:55 |
| mhayden | odyssey4me: glad you're back! :) | 13:56 |
| hughsaunders | git-harry: https://review.openstack.org/#/c/241322/1/test-requirements.txt,cm | 13:56 |
| odyssey4me | mhayden o/ | 13:56 |
| odyssey4me | trying to catch up! | 13:56 |
| *** gparaskevas has joined #openstack-ansible | 14:00 | |
| *** sdake has quit IRC | 14:02 | |
| *** cloudtrainme has quit IRC | 14:04 | |
| *** sdake has joined #openstack-ansible | 14:06 | |
| *** javeriak_ has quit IRC | 14:06 | |
| *** scarlisle has joined #openstack-ansible | 14:11 | |
| *** javeriak has joined #openstack-ansible | 14:11 | |
| *** cloudtrainme has joined #openstack-ansible | 14:12 | |
| *** woodard has quit IRC | 14:12 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Add reno for release notes management https://review.openstack.org/241592 | 14:13 |
| *** Bjoern_ has joined #openstack-ansible | 14:16 | |
| *** Bjoern_ is now known as Bjoern_zZzZzZzZ | 14:17 | |
| *** woodard has joined #openstack-ansible | 14:18 | |
| *** Bjoern_zZzZzZzZ is now known as Bjoern_ | 14:19 | |
| *** Bjoern_ is now known as Bjoern_zZzZzZzZ | 14:19 | |
| *** sdake has quit IRC | 14:21 | |
| *** ashishb has quit IRC | 14:21 | |
| *** sdake has joined #openstack-ansible | 14:22 | |
| *** jimchou has joined #openstack-ansible | 14:24 | |
| *** jmckind has joined #openstack-ansible | 14:24 | |
| *** KLevenstein has joined #openstack-ansible | 14:27 | |
| *** itsuugo has joined #openstack-ansible | 14:30 | |
| *** fawadkhaliq has joined #openstack-ansible | 14:31 | |
| *** jmckind is now known as jmckind_ | 14:31 | |
| *** jwitko has quit IRC | 14:32 | |
| *** mss has joined #openstack-ansible | 14:33 | |
| openstackgerrit | Merged openstack/openstack-ansible: Avoid requesting get-pip.py multiple times https://review.openstack.org/243804 | 14:34 |
| cloudnull | o/ mornings | 14:35 |
| *** sdake has quit IRC | 14:37 | |
| *** Bjoern_zZzZzZzZ is now known as Bjoern_ | 14:37 | |
| *** sdake has joined #openstack-ansible | 14:38 | |
| openstackgerrit | Jesse Pretorius proposed openstack/openstack-ansible: Avoid requesting get-pip.py multiple times https://review.openstack.org/244686 | 14:38 |
| *** moravec has joined #openstack-ansible | 14:38 | |
| *** mss has quit IRC | 14:38 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible: Update neutron_migrations_facts https://review.openstack.org/240560 | 14:40 |
| cloudnull | Its a little out-of-band however if anyone has time to do some reviews on https://github.com/ansible/ansible/pull/12555 it would be much appreciated. | 14:40 |
| cloudnull | its a port for config_template into ansible 2.0 | 14:40 |
| *** javeriak has quit IRC | 14:42 | |
| *** itsuugo has quit IRC | 14:43 | |
| mhayden | odyssey4me: thanks for the backports | 14:44 |
| odyssey4me | mhayden :) I think it's time we setup a stable-maint group to focus on proactive backporting and the reviews thereof | 14:45 |
| *** mss has joined #openstack-ansible | 14:45 | |
| odyssey4me | cloudnull o/ | 14:45 |
| mhayden | true -- i've been actively backporting things i find in there but you're too quick | 14:45 |
| odyssey4me | hughsaunders perhaps you should take a gander at the ansible PR, as you've been a submitter yourself :) | 14:46 |
| hughsaunders | ok | 14:46 |
| odyssey4me | mhayden I'd prefer to delegate the task, but I find that people tend to forget to do it unless it's a specific responsibility or they care about the backport being done for a specific reason | 14:47 |
| *** mss has quit IRC | 14:49 | |
| bgmccollum | kinda wish gerrit would auto propose backports for things landed in master...then you could just tick some yes / no buttons and not have to think much about it | 14:50 |
| odyssey4me | bgmccollum we could probably setup something like that using the propose bot - perhaps if there's a linked bug, then propose a backport | 14:51 |
| bgmccollum | and if it applies cleanly, and passes jenkins...then you basically "do it"... | 14:52 |
| odyssey4me | but the thing is that branches other than master are meant to be stable, and therefore should not require as many backport proposals | 14:52 |
| bgmccollum | true | 14:52 |
| odyssey4me | but yeah, considering that some projects (like neutron) are starting to implement proactive backport policies... perhaps something like that would be a good suggestion | 14:53 |
| odyssey4me | it's hardly a burden on humans though | 14:53 |
| bgmccollum | i think having the review in your face elicits a response though | 14:53 |
| *** jmckind_ has quit IRC | 14:53 | |
| *** sdake has quit IRC | 14:53 | |
| bgmccollum | youre right its not much of a burder...but as you said, we tend to not pay attention / forget | 14:53 |
| bgmccollum | ive been ushering lots of backports to kilo because they're things affecting me specifically, and things i need to do im my job... | 14:54 |
| *** javeriak has joined #openstack-ansible | 14:58 | |
| *** b3rnard0 is now known as b3rnard0_meeting | 15:00 | |
| *** spotz_zzz is now known as spotz | 15:02 | |
| *** jmckind has joined #openstack-ansible | 15:12 | |
| *** woodard has quit IRC | 15:14 | |
| odyssey4me | bgmccollum bugfix backports are always welcome | 15:16 |
| mhayden | so i'd like to avoid messing with the ansible aio playbook again since it's already +2'd -- should i write docs and make that commit dependent on the AIO commit? | 15:16 |
| odyssey4me | mhayden yes, absolutely | 15:17 |
| mhayden | odyssey4me: okay, spiffy | 15:17 |
| mhayden | i'm still a bit green when it comes to those dependency linkages in gerrit, so i'm sure i'll mess something up | 15:17 |
| odyssey4me | the existing patch is a like for like replacement - patches on top of that can change functionality in a surgical way | 15:17 |
| odyssey4me | mhayden git review -d <review id>, then git checkout -b <new branch id> | 15:18 |
| mhayden | where <review id> is the existing AIO review, i assu,e? | 15:18 |
| odyssey4me | mhayden yep | 15:18 |
| mhayden | s/assu,e/assume/ | 15:18 |
| mhayden | okay | 15:18 |
| mhayden | thanks | 15:19 |
| *** oneswig has joined #openstack-ansible | 15:23 | |
| *** mpavone has quit IRC | 15:26 | |
| mhayden | odyssey4me: so i end up with this: https://gist.github.com/major/e9740a2731811d9b6717 | 15:26 |
| mhayden | did i miss a step there? | 15:27 |
| odyssey4me | nope, that's exactly right | 15:27 |
| mhayden | oh, so i say yes there? | 15:27 |
| odyssey4me | as the other one hasn't merged, it'll show | 15:27 |
| mhayden | gotcha | 15:27 |
| odyssey4me | yes | 15:27 |
| mhayden | thanks! | 15:27 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible: Use Ansible AIO playbook for run-aio-build.sh https://review.openstack.org/244708 | 15:27 |
| *** sdake has joined #openstack-ansible | 15:35 | |
| *** jmckind_ has joined #openstack-ansible | 15:36 | |
| *** jmckind_ has quit IRC | 15:37 | |
| *** jmckind has quit IRC | 15:38 | |
| *** jmckind_ has joined #openstack-ansible | 15:38 | |
| *** jmckind_ has quit IRC | 15:39 | |
| *** sdake_ has joined #openstack-ansible | 15:39 | |
| prometheanfire | cloudnull: you in? | 15:39 |
| *** mpavone has joined #openstack-ansible | 15:39 | |
| *** sdake has quit IRC | 15:39 | |
| *** Bjoern_ is now known as BjoernT | 15:40 | |
| *** fawadkhaliq has quit IRC | 15:40 | |
| *** jmckind has joined #openstack-ansible | 15:40 | |
| *** jmckind has quit IRC | 15:41 | |
| *** jmckind has joined #openstack-ansible | 15:42 | |
| cloudnull | i is | 15:42 |
| cloudnull | ^ prometheanfire | 15:42 |
| *** jmckind has quit IRC | 15:42 | |
| *** jmckind has joined #openstack-ansible | 15:43 | |
| *** sdake has joined #openstack-ansible | 15:43 | |
| *** jmckind has quit IRC | 15:44 | |
| *** jmckind has joined #openstack-ansible | 15:44 | |
| *** jmckind has quit IRC | 15:45 | |
| prometheanfire | cloudnull: castle? | 15:46 |
| prometheanfire | cloudnull: have a stupid variable access thing not working | 15:47 |
| *** sdake_ has quit IRC | 15:47 | |
| prometheanfire | https://travis-ci.org/rcbops/rpc-openstack/builds/90755960 | 15:47 |
| prometheanfire | https://github.com/prometheanfire/rpc-openstack/blob/fixing-the-submodules/rpcd/playbooks/repo-fetcher.yml | 15:47 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible: Updating AIO docs for Ansible playbook https://review.openstack.org/244720 | 15:49 |
| *** jmckind has joined #openstack-ansible | 15:50 | |
| cloudnull | prometheanfire: this is wrong https://github.com/prometheanfire/rpc-openstack/blob/fixing-the-submodules/rpcd/playbooks/repo-fetcher.yml#L36 | 15:52 |
| odyssey4me | community meeting in 7 mins in #openstack-meeting-4 cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, mancdaz, dolphm, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva | 15:53 |
| prometheanfire | ah, see it now | 15:53 |
| prometheanfire | thanks | 15:53 |
| *** woodard has joined #openstack-ansible | 15:54 | |
| cloudnull | np | 15:54 |
| *** cloudtrainme has quit IRC | 15:56 | |
| *** mss has joined #openstack-ansible | 15:59 | |
| odyssey4me | community meeting now in #openstack-meeting-4 cloudnull, mattt, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, Sam-I-Am, odyssey4me, serverascode, rromans, mancdaz, dolphm, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva | 16:01 |
| mattt | o/ | 16:01 |
| mattt | oh derp | 16:01 |
| *** b3rnard0_meeting is now known as b3rnard0 | 16:07 | |
| cloudnull | has anyone seen this in the wild for kilo https://bugs.launchpad.net/python-glanceclient/+bug/1479296 -- recent deployment of OSA kilo and I seem to be hitting that | 16:08 |
| openstack | Launchpad bug 1476770 in openstack-ansible "duplicate for #1479296 _translate_from_glance fails with "AttributeError: id" in grenade" [High,In progress] - Assigned to Jesse Pretorius (jesse-pretorius) | 16:08 |
| *** sigmavirus24_awa is now known as sigmavirus24 | 16:10 | |
| *** jmckind has quit IRC | 16:17 | |
| *** jmckind_ has joined #openstack-ansible | 16:17 | |
| cloudnull | so it seems that we're potentially seeing -- TRACE nova.api.openstack HTTPMultipleChoices: HTTPMultipleChoices (HTTP 300) Requested version of OpenStack Images API is not available. | 16:19 |
| cloudnull | stevelle sigmavirus24 ? | 16:19 |
| *** egonzalez has quit IRC | 16:19 | |
| cloudnull | tagged versions seem to be fine / head seems to be mad | 16:20 |
| *** shanecc has joined #openstack-ansible | 16:20 | |
| *** ashishb has joined #openstack-ansible | 16:22 | |
| cloudnull | so ignore the previous TRACE that was self inflicted, however this still remains -- TRACE nova.api.openstack AttributeError: id | 16:23 |
| cloudnull | which is coming out of /usr/local/lib/python2.7/dist-packages/glanceclient/openstack/common/apiclient/base.py | 16:23 |
| *** egonzalez has joined #openstack-ansible | 16:24 | |
| sigmavirus24 | hm | 16:25 |
| sigmavirus24 | cloudnull: is it reproducible in venvs? | 16:25 |
| sigmavirus24 | Because it shouldn't be | 16:25 |
| *** toddnni has joined #openstack-ansible | 16:27 | |
| *** mpavone has quit IRC | 16:27 | |
| cloudnull | i have not tried it | 16:29 |
| cloudnull | thats using kilo | 16:29 |
| sigmavirus24 | ah right | 16:29 |
| sigmavirus24 | kilo didn't get the venv support, right? | 16:29 |
| cloudnull | no , not yet , if needed we can do it but its not a thing at this point | 16:29 |
| *** phiche has quit IRC | 16:33 | |
| *** sacharya has joined #openstack-ansible | 16:39 | |
| sigmavirus24 | cloudnull: did our meeting time change? | 16:39 |
| cloudnull | no but daylight savings did :) | 16:39 |
| cloudnull | 1600 UTC | 16:39 |
| *** sdake has quit IRC | 16:40 | |
| *** fawadkhaliq has joined #openstack-ansible | 16:40 | |
| *** sdake has joined #openstack-ansible | 16:42 | |
| *** phiche has joined #openstack-ansible | 16:43 | |
| logan2 | hmm i am seeing https://bugs.launchpad.net/ubuntu/+source/python-eventlet/+bug/1452312 on kilo deploys using syslog for openstack services :( | 16:43 |
| openstack | Launchpad bug 1452312 in python-eventlet (Ubuntu Wily) "glance-registry process spins if rsyslog restarted with syslog logging enabled" [High,Fix released] | 16:43 |
| logan2 | restart rsyslog, all openstack processes spin at 100% cpu until restarted | 16:44 |
| *** slotti has quit IRC | 16:44 | |
| *** mgoddard__ has joined #openstack-ansible | 16:45 | |
| *** Mudpuppy has joined #openstack-ansible | 16:45 | |
| *** mgoddard_ has quit IRC | 16:48 | |
| cloudnull | logan2: is registry setup to log to syslog directly ? | 16:52 |
| odyssey4me | cloudnull was that kilo using repo-build, or using repo-clone? | 16:52 |
| cloudnull | build | 16:52 |
| odyssey4me | cloudnull so the kilo branch has not had a sha update in some time - it may be time to tag, then do a sha/tag update and figure that out | 16:53 |
| openstackgerrit | Nolan Brubaker proposed openstack/openstack-ansible: Add testing for dynamic_inventory.py https://review.openstack.org/242225 | 16:53 |
| odyssey4me | I expect that it probably relates to requests/urllib/apt nonsense | 16:53 |
| logan2 | its not glance registry, i am doing openstack->syslog on compute nodes, so what im seeing affected so far are cinder-volume and nova-api-metadata | 16:54 |
| logan2 | im not seeing it on my manual non-OSA compute nodes which run python-eventlet 0.16.1-0ubuntu1.1~cloud0 from ubuntu cloud kilo repo | 16:55 |
| odyssey4me | ah yes, logan2 this is why we don't configure the services to push logs via syslog directly | 16:55 |
| odyssey4me | it is very poorly supported upstream | 16:55 |
| oneswig | It looks awfully like a long-term unresolved issue: https://bugs.launchpad.net/nova/+bug/1101404 | 16:55 |
| openstack | Launchpad bug 1101404 in OpenStack Compute (nova) "nova syslog logging to /dev/log race condition in python 2.6 and 2.7" [Medium,Confirmed] | 16:55 |
| logan2 | frustrating | 16:56 |
| odyssey4me | it may get better, but we instead log local and use rsyslog to ship the logs | 16:56 |
| logan2 | hmm I see that. interesting.. does it wipe or roll the logs continuously as it ships them | 16:57 |
| logan2 | local logging is a big concern in this env as the compute nodes are diskless | 16:58 |
| *** mgoddard has joined #openstack-ansible | 16:58 | |
| *** mgoddard__ has quit IRC | 16:58 | |
| *** moravec has quit IRC | 16:59 | |
| odyssey4me | logan2 as far as I know we have log rotation setup for all containers and hosts, including the rsyslog container | 16:59 |
| cloudnull | it logs and rotates on a schedule | 16:59 |
| odyssey4me | I expect that you may wish to tweak those settings to make them a limited size | 16:59 |
| logan2 | got it so logrotate handles the rotation there not rsyslog as it ships | 16:59 |
| cloudnull | correct | 17:00 |
| *** jwitko has joined #openstack-ansible | 17:00 | |
| *** daneyon has joined #openstack-ansible | 17:00 | |
| logan2 | ok I will look at goign that route thanks | 17:00 |
| cloudnull | the rotate bits are here https://github.com/openstack/openstack-ansible/blob/master/playbooks/roles/rsyslog_client/templates/os_aggregate_storage.j2 | 17:01 |
| cloudnull | probably want to rotate daily, with a rotate of 1 | 17:01 |
| *** daneyon has quit IRC | 17:01 | |
| *** daneyon has joined #openstack-ansible | 17:02 | |
| *** daneyon_ has joined #openstack-ansible | 17:03 | |
| *** moravec has joined #openstack-ansible | 17:04 | |
| *** pcaruana has quit IRC | 17:06 | |
| *** daneyon has quit IRC | 17:06 | |
| *** sdake_ has joined #openstack-ansible | 17:07 | |
| *** sdake has quit IRC | 17:09 | |
| *** karimb_ has quit IRC | 17:10 | |
| *** galstrom_zzz is now known as galstrom | 17:11 | |
| *** javeriak has quit IRC | 17:12 | |
| *** javeriak has joined #openstack-ansible | 17:12 | |
| cloudnull | odyssey4me: so i rebuilt using new versions of services from kilo , same issue | 17:13 |
| cloudnull | sigmavirus24: also tried to run nova compute from a venv , same issue . | 17:14 |
| odyssey4me | cloudnull if you repo-clone instead of build? | 17:14 |
| odyssey4me | I'm thinking that there may be a new requirements cap needed which may already be upstream somewhere | 17:14 |
| sigmavirus24 | hm | 17:15 |
| sigmavirus24 | so I know the fundmental problem | 17:15 |
| sigmavirus24 | It's a known issue with a weird interaction between very old versions of requests and very new versions of urllib3 | 17:15 |
| *** javeriak_ has joined #openstack-ansible | 17:16 | |
| *** javeriak has quit IRC | 17:17 | |
| logan2 | well at least the syslog thing appears to be a quick patch to eventlet. https://github.com/eventlet/eventlet/commit/ed0655333007fe97624b745e92deb407d192022c -- it hasn't made it into a release yet but i guess i will test it locally and maybe look at how easy it would be to deploy it that way with osa | 17:20 |
| cloudnull | sigmavirus24: so i have requests 2.7.0 installed ? | 17:22 |
| cloudnull | though i have also tried 2.8.1 | 17:23 |
| cloudnull | urllib3 (1.10.4) | 17:24 |
| sigmavirus24 | hm | 17:24 |
| sigmavirus24 | cloudnull: that's bizarre | 17:24 |
| sigmavirus24 | so | 17:24 |
| sigmavirus24 | hm | 17:24 |
| sigmavirus24 | cloudnull: can you PM me the IP and put my SSH key on there? | 17:25 |
| *** alkari has joined #openstack-ansible | 17:26 | |
| *** itsuugo has joined #openstack-ansible | 17:28 | |
| *** aojea_ has joined #openstack-ansible | 17:28 | |
| *** sdake_ has quit IRC | 17:28 | |
| *** sdake has joined #openstack-ansible | 17:29 | |
| openstackgerrit | Merged openstack/openstack-ansible: Choose virt type automatically https://review.openstack.org/243098 | 17:30 |
| *** sdake has quit IRC | 17:30 | |
| *** sdake has joined #openstack-ansible | 17:31 | |
| *** itsuugo has quit IRC | 17:31 | |
| *** aojea_ has quit IRC | 17:31 | |
| sigmavirus24 | So cloudnull this is bizarre that even in a venv it's still happening | 17:31 |
| cloudnull | its strange | 17:32 |
| *** moravec1 has joined #openstack-ansible | 17:33 | |
| cloudnull | I uninstalled the nova and glance client from the host | 17:33 |
| sigmavirus24 | okay | 17:33 |
| sigmavirus24 | just checking | 17:33 |
| cloudnull | its all running in a venv for testing | 17:33 |
| *** moravec has quit IRC | 17:34 | |
| cloudnull | so https://bugs.launchpad.net/python-glanceclient/+bug/1479296 which is a duplicate bug seems to indicate that its caused by nova defaulting to glance v1 | 17:35 |
| openstack | Launchpad bug 1476770 in openstack-ansible "duplicate for #1479296 _translate_from_glance fails with "AttributeError: id" in grenade" [High,In progress] - Assigned to Jesse Pretorius (jesse-pretorius) | 17:35 |
| cloudnull | and setting glance v2 makes things happier | 17:35 |
| cloudnull | however thats not an option in nova-boot | 17:35 |
| *** egonzalez has quit IRC | 17:39 | |
| *** phiche has quit IRC | 17:41 | |
| *** phiche has joined #openstack-ansible | 17:41 | |
| *** ashishb has quit IRC | 17:42 | |
| *** ashishb has joined #openstack-ansible | 17:43 | |
| *** ashishb has quit IRC | 17:43 | |
| *** ashishb has joined #openstack-ansible | 17:44 | |
| *** ashishb has quit IRC | 17:44 | |
| cloudnull | sigmavirus24: so i put the api in a venv too and same thing | 17:44 |
| sigmavirus24 | sorry cloudnull. was pulled into a vid call | 17:44 |
| sigmavirus24 | which log should I be looking for? | 17:44 |
| *** ashishb has joined #openstack-ansible | 17:44 | |
| *** ashishb has quit IRC | 17:44 | |
| sigmavirus24 | also nova API doesn't support glance v2 | 17:44 |
| sigmavirus24 | Might make that into mitaka | 17:44 |
| sigmavirus24 | But that's not guaranteed | 17:44 |
| *** ashishb has joined #openstack-ansible | 17:45 | |
| *** ashishb has quit IRC | 17:45 | |
| cloudnull | log entry /openstack/venvs/nova-11.2.5/lib/python2.7/site-packages/glanceclient/openstack/common/apiclient/base.py -- AttributeError: id | 17:45 |
| cloudnull | i have the log up on the tmux session | 17:45 |
| sigmavirus24 | thanks | 17:46 |
| *** tiagogomes_ has quit IRC | 17:48 | |
| sigmavirus24 | this is very bizarre | 17:51 |
| *** sdake has quit IRC | 17:54 | |
| *** Mudpuppy has quit IRC | 17:54 | |
| *** oneswig has quit IRC | 17:58 | |
| *** moravec1 has quit IRC | 17:59 | |
| cloudnull | sigmavirus24: this is the diff between what is on this box and what is up in the 11.2.5 repo on rpc-repo.rax http://paste.openstack.org/show/478708/ | 18:00 |
| *** yanshg has joined #openstack-ansible | 18:00 | |
| sigmavirus24 | so cloudnull in the past the problem was related to `python-requests` being installed by apt | 18:00 |
| sigmavirus24 | and urllib3 being newer than the version that apt would install | 18:00 |
| sigmavirus24 | And that conflicting with glanceclient/nova-compute on a host | 18:00 |
| sigmavirus24 | This has been seen in customer environments | 18:01 |
| sigmavirus24 | This is also what (if I remember) partially motivated the implementation of virtualenvs | 18:01 |
| *** shanecc has quit IRC | 18:01 | |
| *** sdake has joined #openstack-ansible | 18:02 | |
| *** phiche1 has joined #openstack-ansible | 18:02 | |
| cloudnull | so not everything is in a venv in my test box | 18:03 |
| *** javeriak_ has quit IRC | 18:03 | |
| cloudnull | just nova compute/nova apii | 18:03 |
| sigmavirus24 | Yeah | 18:03 |
| sigmavirus24 | that's still bizarre | 18:03 |
| cloudnull | and their only there becuase i just did that | 18:03 |
| cloudnull | I also just uninstalled python-requests to rule that out | 18:03 |
| *** phiche has quit IRC | 18:04 | |
| *** Mudpuppy has joined #openstack-ansible | 18:04 | |
| sigmavirus24 | yeah | 18:04 |
| *** javeriak has joined #openstack-ansible | 18:05 | |
| sigmavirus24 | so weird that glance works | 18:06 |
| sigmavirus24 | But nova-api can't use glance | 18:06 |
| sigmavirus24 | *glanceclient | 18:06 |
| *** moravec has joined #openstack-ansible | 18:08 | |
| sigmavirus24 | lol cloudnull not sticking around the tmux session? | 18:13 |
| sigmavirus24 | i was just digging into that container :P | 18:13 |
| cloudnull | ha ha . | 18:13 |
| cloudnull | i was trying to stay out of your way | 18:13 |
| cloudnull | i guess that didnt work :p | 18:13 |
| sigmavirus24 | lol no worries | 18:13 |
| cloudnull | so i installed glanceclient 0.18.0 instead of 0.17.1 | 18:14 |
| cloudnull | no change | 18:14 |
| sigmavirus24 | yeah I see that | 18:15 |
| cloudnull | maybe its nova | 18:16 |
| cloudnull | we have nova (2015.1.3.dev1) | 18:16 |
| cloudnull | and maybe its busted | 18:16 |
| sigmavirus24 | I mean it shouldn't be | 18:16 |
| sigmavirus24 | the code that's getting the AttributeError is deep in glanceclient | 18:16 |
| cloudnull | but only after File "/openstack/venvs/nova-11.2.5/lib/python2.7/site-packages/nova/image/glance.py", line 545, in _extract_attributes | 18:17 |
| *** sdake has quit IRC | 18:18 | |
| cloudnull | in the api container im going to install 2015.1.2 if you dont ming | 18:18 |
| cloudnull | *mind | 18:18 |
| *** phiche1 has quit IRC | 18:20 | |
| *** javeriak_ has joined #openstack-ansible | 18:21 | |
| cloudnull | so once again, no change... | 18:22 |
| *** javeriak has quit IRC | 18:23 | |
| *** phiche has joined #openstack-ansible | 18:23 | |
| sigmavirus24 | yeah | 18:24 |
| sigmavirus24 | sorry | 18:24 |
| sigmavirus24 | yeah I don't think it's glance-api causing the problem | 18:25 |
| sigmavirus24 | like using glanceclient from the command-line shows the service is fine | 18:25 |
| *** toddnni has quit IRC | 18:25 | |
| sigmavirus24 | cloudnull: so http://paste.openstack.org/show/478715/ explains why it's something glanceclient/requests related | 18:28 |
| sigmavirus24 | but not quite because it comes out of glanceclient at the end there | 18:28 |
| sigmavirus24 | To be clear, the issue that appeared upstream was fixed by pinning to an older urllib3 | 18:29 |
| sigmavirus24 | but nova-api-os-compute container doesn't have it installed at all | 18:29 |
| sigmavirus24 | so | 18:29 |
| cloudnull | ... :( | 18:29 |
| javeriak_ | hey guys \o | 18:30 |
| cloudnull | hi javeriak_ | 18:30 |
| javeriak_ | been trying to find documentation on upgrades, do we have anything official | 18:30 |
| javeriak_ | like as in from juno --> kilo | 18:30 |
| cloudnull | juno > kilo ? | 18:30 |
| javeriak_ | yep | 18:31 |
| cloudnull | http://docs.openstack.org/developer/openstack-ansible/kilo/upgrade-guide/index.html | 18:31 |
| javeriak_ | perfect, thanks! | 18:31 |
| cloudnull | process outline here | 18:31 |
| cloudnull | http://docs.openstack.org/developer/openstack-ansible/kilo/upgrade-guide/process.html | 18:31 |
| javeriak_ | btw cloudnull, saw your tokyo video, very cool | 18:31 |
| cloudnull | you mean me rambling like an idot :) | 18:32 |
| cloudnull | *idiot | 18:32 |
| javeriak_ | haha, it was cool | 18:32 |
| cloudnull | you should come to the next summit | 18:33 |
| cloudnull | its in austin | 18:33 |
| cloudnull | it'd be great to put faces to names :) | 18:33 |
| cloudnull | tyvm btw | 18:34 |
| *** alkari has quit IRC | 18:34 | |
| javeriak_ | Yea that would be great, I'm planning to hopefully | 18:34 |
| cloudnull | you'll be ATC this go around , if you wern't already | 18:34 |
| cloudnull | so free ticket ;) | 18:34 |
| sigmavirus24 | cloudnull: even nova image-list works | 18:35 |
| sigmavirus24 | wtf | 18:35 |
| javeriak_ | yea already was this time around too, was abroad so couldnt make arrangements | 18:35 |
| *** alkari has joined #openstack-ansible | 18:35 | |
| cloudnull | ah , yea for sure. | 18:35 |
| cloudnull | sigmavirus24: idk... maybe it doesnt like requests 2.8.1 ? | 18:35 |
| sigmavirus24 | cloudnull: idk either | 18:36 |
| sigmavirus24 | 2.8.x does have urllib3 1.11.0 in it though | 18:37 |
| sigmavirus24 | but 2.7.0 was what you were using before and it was still bjorked, right? | 18:37 |
| *** itsuugo has joined #openstack-ansible | 18:37 | |
| cloudnull | FML... | 18:38 |
| cloudnull | so pinning to 2.7.0 works | 18:39 |
| *** fawadkhaliq has quit IRC | 18:39 | |
| cloudnull | the build was pulling in 2.8.1 | 18:39 |
| cloudnull | i reinstalled 2.7.0 on compute/api | 18:40 |
| cloudnull | https://github.com/openstack/requirements/blob/stable/kilo/global-requirements.txt#L143 | 18:41 |
| cloudnull | theres no cap. | 18:41 |
| cloudnull | and https://github.com/openstack/requirements/blob/stable/kilo/upper-constraints.txt hasnt been updated in 4 mo | 18:42 |
| sigmavirus24 | huh | 18:43 |
| sigmavirus24 | propose an update tehn | 18:43 |
| sigmavirus24 | *then | 18:43 |
| sigmavirus24 | I'm surprised the gate didn't see this yet | 18:43 |
| sigmavirus24 | Oh https://github.com/openstack/requirements/blob/stable/kilo/upper-constraints.txt#L256 is there | 18:44 |
| cloudnull | it is however in kilo we're not using that | 18:45 |
| cloudnull | and at last check https://github.com/openstack/requirements/blob/stable/kilo/upper-constraints.txt#L176 still breaks keystone | 18:45 |
| cloudnull | yup https://github.com/openstack/keystone/blob/stable/kilo/requirements.txt#L25 | 18:46 |
| cloudnull | im going to rebuild the repo and force reinstall some things and see if i can make it happy | 18:46 |
| miguelgrinberg | alextricity: hey, if possible, could you switch this lab you were using yesterday to uuid tokens and retry the software-config heat template? It seems those 403s are caused by an upstream problem with fernet tokens. | 18:47 |
| *** sdake has joined #openstack-ansible | 18:49 | |
| *** sdake has quit IRC | 18:51 | |
| *** shanecc has joined #openstack-ansible | 18:53 | |
| *** toddnni has joined #openstack-ansible | 19:15 | |
| logan2 | building eventlet from git w/ that fix commit fixed the rsyslog restart issue. awesome how easy it was to get eventlet building from that github tag | 19:18 |
| *** spotz is now known as spotz_zzz | 19:20 | |
| *** spotz_zzz is now known as spotz | 19:21 | |
| *** sdake has joined #openstack-ansible | 19:22 | |
| *** spotz is now known as spotz_zzz | 19:25 | |
| *** spotz_zzz is now known as spotz | 19:25 | |
| pabelanger | Question about tox -eansible-syntax and tox -eansible-lint. Is there a good argument to keep them as 2 different environments or could we run both ansible-syntax and ansible-lint in the same task | 19:25 |
| pabelanger | eg: tox -eansible-lint runs both ansible-lint and ansible-syntax | 19:25 |
| dolphm | miguelgrinberg: what problem with fernet tokens? | 19:28 |
| miguelgrinberg | dolphm: haven't figured out exactly what the problem is, but heat cannot validate ec2tokens when keystone is configured for fernet | 19:29 |
| *** javeriak has joined #openstack-ansible | 19:29 | |
| dolphm | miguelgrinberg: oh weird, that's a new one | 19:29 |
| dolphm | ec2 api throws a 403? | 19:30 |
| miguelgrinberg | this is kilo, btw, no problem in L or master | 19:30 |
| dolphm | miguelgrinberg: hmm | 19:30 |
| miguelgrinberg | yes, heat gets back 403s | 19:30 |
| miguelgrinberg | switching to uuid everything starts to work | 19:30 |
| cloudnull | pabelanger: that sounds sensible to me | 19:30 |
| cloudnull | theres really no reason not to run the two together | 19:31 |
| pabelanger | cloudnull: right. And it will also reduce the pressure in gate since we don't have to launch 2 nodes each change | 19:31 |
| *** moravec1 has joined #openstack-ansible | 19:31 | |
| *** alkari has quit IRC | 19:31 | |
| *** moravec has quit IRC | 19:32 | |
| dolphm | miguelgrinberg: no federation involved or anything, right? | 19:32 |
| *** javeriak_ has quit IRC | 19:33 | |
| miguelgrinberg | dolphm: no, nothing like that, single cloud | 19:33 |
| cloudnull | +1 pabelanger that should be better all around . | 19:33 |
| pabelanger | cloudnull: okay, I'll just use tox -eansible-lint for now | 19:34 |
| dolphm | miguelgrinberg: i'm opening an Incomplete bug for now. any other details? | 19:35 |
| *** javeriak_ has joined #openstack-ansible | 19:35 | |
| miguelgrinberg | dolphm: I wrote a bug last night: https://bugs.launchpad.net/openstack-ansible/+bug/1515485 | 19:36 |
| openstack | Launchpad bug 1515485 in openstack-ansible "Heat CFN signals do not pass authorization" [Undecided,New] | 19:36 |
| miguelgrinberg | sounds like this isn't OSA's fault, but I'm not 100% sure yet | 19:36 |
| dolphm | miguelgrinberg: oh, perfect. i'll add keystone just in case | 19:36 |
| dolphm | miguelgrinberg: the bug says you haven't tested liberty - have you tested it since then? | 19:38 |
| *** itsuugo has quit IRC | 19:38 | |
| dolphm | (you said no problem in L earlier) | 19:38 |
| miguelgrinberg | yes, I have | 19:38 |
| *** javeriak has quit IRC | 19:39 | |
| dolphm | miguelgrinberg: oh *heat* is returning the 403 -- i couldn't think of why keystone might be returning a 403 (we don't use 403 much) | 19:39 |
| lbragstad | i was just thinking that, i wasn't sure there was anything fernet specific that would return a 403 | 19:40 |
| dolphm | miguelgrinberg: do you have keystone logs for "2015-11-12 05:13:34.493 1862 INFO heat.api.aws.ec2token [-] Authenticating with http://172.29.236.100:5000/v3/ec2tokens" | 19:40 |
| miguelgrinberg | dolphm: correct, heat fails to validate the ec2 token and returns a 403 | 19:40 |
| *** javeriak has joined #openstack-ansible | 19:40 | |
| miguelgrinberg | dolphm: I think so, let me see what I find | 19:41 |
| miguelgrinberg | up until this morning I assumed it was an OSA configuration problem, but someone responded on the bug and gave me the clue about fernet | 19:42 |
| *** javeriak_ has quit IRC | 19:43 | |
| dolphm | miguelgrinberg: any chance heat would be doing something to trigger token revocation in keystone? (disabling or deleting things? unassigning roles?) | 19:44 |
| dolphm | miguelgrinberg: and any idea if the 403's are transient or consistent? | 19:44 |
| miguelgrinberg | I don't think heat deal with revocation at all. The 403s were consistent, failed every time. | 19:44 |
| *** sdake_ has joined #openstack-ansible | 19:46 | |
| miguelgrinberg | dolphm: keystone stacktrace for the log entry you referenced above: http://paste.openstack.org/show/478721/ | 19:47 |
| dolphm | miguelgrinberg: yay, an easy backtrace! | 19:47 |
| dolphm | miguelgrinberg: i'll add to the bug | 19:48 |
| dolphm | miguelgrinberg: definitely keystone's fault | 19:48 |
| *** sdake has quit IRC | 19:49 | |
| dolphm | lbragstad: this is the code that you refactored between kilo and liberty, which we didn't backport, right? | 19:49 |
| dolphm | lbragstad: or, it appears we did not backport if the problem is only in liberty | 19:49 |
| dolphm | err, kilo* | 19:49 |
| miguelgrinberg | dolphm: does it make sense that this happens with fernet but not uuid? | 19:53 |
| dolphm | miguelgrinberg: yes, absolutely | 19:53 |
| miguelgrinberg | oh okay, great! | 19:53 |
| dolphm | miguelgrinberg: pretty sure i've seen this backtrace before - and there's already a fix out there. challenge will be finding it and figuring out why it's not in kilo | 19:53 |
| lbragstad | dolphm it could very well be | 19:53 |
| lbragstad | dolphm looking at the trace | 19:53 |
| dolphm | miguelgrinberg: i trust this is the latest stable/kilo, and not 3 month old code :P | 19:54 |
| miguelgrinberg | dolphm: I installed the osa/kilo branch yesterday | 19:54 |
| dolphm | miguelgrinberg: we landed (literally) 20 or so backports for fernet to stable/kilo, and only excluded about 3 patches because they didn't seem backportable. that makes our shopping list much shorter, though. | 19:54 |
| *** linggao has joined #openstack-ansible | 19:55 | |
| miguelgrinberg | dolphm: alextricity has a customer lab that has the problem, he's the one that found it | 19:55 |
| lbragstad | i think i found the difference | 19:55 |
| lbragstad | this is master - https://github.com/openstack/keystone/blob/master/keystone/token/providers/common.py#L550 | 19:56 |
| lbragstad | this is kilo - https://github.com/openstack/keystone/blob/stable/kilo/keystone/token/providers/fernet/core.py#L184 | 19:56 |
| lbragstad | kilo doesn't check if auth_context is None before performing a .get() | 19:56 |
| dolphm | lbragstad: git blame that one and backport! | 19:56 |
| lbragstad | can we backport this - https://github.com/openstack/keystone/commit/91a0b29809cb71c1b2df1642d6c34a3f60a801ed ? | 19:57 |
| dolphm | lbragstad: oh wait - that backtrace is saying there is no auth context! | 19:58 |
| lbragstad | dolphm auth_context is None | 19:58 |
| lbragstad | instead of an empty dict or something like that | 19:59 |
| *** moravec has joined #openstack-ansible | 19:59 | |
| lbragstad | we're protected from that in liberty and master because of `if auth_context` | 19:59 |
| dolphm | lbragstad: but why is there no auth context?! | 19:59 |
| *** moravec1 has quit IRC | 19:59 | |
| dolphm | because the ec2 controllers are calling us without it or something? | 19:59 |
| lbragstad | that could be | 20:00 |
| lbragstad | checking the heat keystone client code | 20:00 |
| lbragstad | which is here ? https://github.com/openstack/heat/blob/d7957587e301dce88ae13acc3df94bae6ebee96a/heat/api/aws/ec2token.py#L161 | 20:01 |
| sigmavirus24 | cloudnull: that failure you were seeing was on head of osa kilo right? | 20:02 |
| cloudnull | yes however in testing a kilo world without yaprt. which seems to be busted because nobody has been maintaining the upper-constraints file in kilo | 20:04 |
| openstackgerrit | Merged openstack/openstack-ansible: Added logging for haproxy to rsyslog https://review.openstack.org/239505 | 20:05 |
| *** javeriak_ has joined #openstack-ansible | 20:06 | |
| lbragstad | dolphm auth_context should start somewhere around here - https://github.com/openstack/keystone/blob/stable/kilo/keystone/auth/controllers.py#L85 | 20:07 |
| *** javeria__ has joined #openstack-ansible | 20:09 | |
| *** javeriak has quit IRC | 20:09 | |
| *** moravec has quit IRC | 20:09 | |
| *** javeriak_ has quit IRC | 20:12 | |
| *** sdake_ has quit IRC | 20:14 | |
| lbragstad | miguelgrinberg are you able to determine if auth_context is actually set in liberty/master versus kilo here - https://github.com/openstack/keystone/blob/stable/kilo/keystone/token/providers/fernet/core.py#L184 | 20:16 |
| *** sdake has joined #openstack-ansible | 20:17 | |
| miguelgrinberg | lbragstad: I can add logging to find out, sure | 20:17 |
| lbragstad | miguelgrinberg we know that auth_context is None in kilo due to the bug | 20:17 |
| lbragstad | just curious if there is something setting it in liberty or master | 20:17 |
| lbragstad | before it reaches that point | 20:17 |
| miguelgrinberg | okay, give a few min. | 20:18 |
| lbragstad | miguelgrinberg awesome, thank you | 20:18 |
| *** javeriak has joined #openstack-ansible | 20:23 | |
| *** moravec has joined #openstack-ansible | 20:25 | |
| miguelgrinberg | lbragstad: just to make sure I'm looking at the right place, in post-kilo releases the code moved to token/providers/common.py, correct? | 20:25 |
| lbragstad | miguelgrinberg correct | 20:25 |
| lbragstad | lbragstad that was something we landed after liberty opened for development, along with a host of other things | 20:26 |
| *** javeriak has quit IRC | 20:26 | |
| *** javeria__ has quit IRC | 20:27 | |
| linggao | Hi all, I am new to ansible and am installing all-in-one openstack in my local ubuntu x86_64 server. I copied openstack_user_config.yml.aio to openstack_user_config.yml. Do I need to change the all the network settings in the file to match the ip address of my localhost? | 20:27 |
| linggao | Looks like there are 3 or 4 networks specified in that file. | 20:28 |
| lbragstad | miguelgrinberg i'm thinking that if auth_context is None in kilo, libery, and master. Then it's definitely the fact that we don't do 'if auth_context'. But I wanted to make sure that it wasn't not getting hit because of how heat may have been interacting with keystone | 20:31 |
| *** moravec has quit IRC | 20:32 | |
| miguelgrinberg | lbragstad: the context is None also in master | 20:34 |
| lbragstad | miguelgrinberg great | 20:35 |
| lbragstad | miguelgrinberg i have a backport for you - https://review.openstack.org/#/c/244853/ | 20:35 |
| lbragstad | miguelgrinberg that passes tests and pep8 locally on stable/kilo | 20:35 |
| lbragstad | miguelgrinberg that should include the auth_context None check that you need | 20:35 |
| miguelgrinberg | lbragstad: okay, I'll install on my kilo box and let you know in a bit | 20:36 |
| *** moravec has joined #openstack-ansible | 20:38 | |
| *** jmckind_ has quit IRC | 20:48 | |
| *** moravec1 has joined #openstack-ansible | 20:53 | |
| *** sdake has quit IRC | 20:54 | |
| *** moravec has quit IRC | 20:54 | |
| *** cloudtrainme has joined #openstack-ansible | 20:55 | |
| *** moravec1 has quit IRC | 20:57 | |
| *** sdake has joined #openstack-ansible | 21:11 | |
| *** sdake_ has joined #openstack-ansible | 21:15 | |
| *** sdake has quit IRC | 21:17 | |
| *** itsuugo has joined #openstack-ansible | 21:17 | |
| *** toddnni has quit IRC | 21:23 | |
| *** woodard has quit IRC | 21:37 | |
| *** jmckind has joined #openstack-ansible | 21:37 | |
| *** jmckind has quit IRC | 21:38 | |
| *** jmckind has joined #openstack-ansible | 21:38 | |
| *** jmckind has quit IRC | 21:39 | |
| *** jmckind has joined #openstack-ansible | 21:40 | |
| *** jmckind has quit IRC | 21:46 | |
| *** jhesketh has quit IRC | 21:50 | |
| *** jhesketh has joined #openstack-ansible | 21:53 | |
| *** itsuugo has quit IRC | 22:11 | |
| *** sacharya has quit IRC | 22:19 | |
| *** gparaskevas has quit IRC | 22:34 | |
| *** rady has joined #openstack-ansible | 22:34 | |
| *** Mudpuppy has quit IRC | 22:40 | |
| *** phiche has quit IRC | 22:42 | |
| *** sdake_ has quit IRC | 22:44 | |
| *** rady has quit IRC | 22:47 | |
| *** sshen has joined #openstack-ansible | 22:53 | |
| *** devl has quit IRC | 22:57 | |
| *** sigmavirus24 is now known as sigmavirus24_awa | 22:59 | |
| *** galstrom is now known as galstrom_zzz | 23:08 | |
| *** toddnni has joined #openstack-ansible | 23:08 | |
| *** devl has joined #openstack-ansible | 23:09 | |
| *** mss_ has joined #openstack-ansible | 23:11 | |
| *** mss has quit IRC | 23:14 | |
| *** KLevenstein has quit IRC | 23:15 | |
| *** darrenc is now known as darrenc_afk | 23:18 | |
| *** sdake has joined #openstack-ansible | 23:23 | |
| *** sdake_ has joined #openstack-ansible | 23:26 | |
| *** sdake has quit IRC | 23:28 | |
| *** darrenc_afk is now known as darrenc | 23:39 | |
| *** woodard has joined #openstack-ansible | 23:41 | |
| miguelgrinberg | lbragstad: patch seems to work fine. Thanks! | 23:46 |
| *** woodard has quit IRC | 23:51 | |
| *** woodard has joined #openstack-ansible | 23:52 | |
| *** spotz is now known as spotz_zzz | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!