Friday, 2017-09-08

*** shardy_afk has quit IRC00:01
*** woodard_ has joined #openstack-ansible00:08
*** woodard has quit IRC00:08
*** shardy_afk has joined #openstack-ansible00:09
*** weezS_ has joined #openstack-ansible00:09
*** weezS has quit IRC00:10
*** weezS_ is now known as weezS00:10
*** weezS has quit IRC00:19
*** galstrom_zzz is now known as galstrom00:22
*** thorst has joined #openstack-ansible00:27
*** yifei has joined #openstack-ansible00:28
*** galstrom is now known as galstrom_zzz00:28
*** shardy_afk has quit IRC00:33
*** zcourts has joined #openstack-ansible00:33
*** zcourts_ has quit IRC00:36
*** agrebennikov has quit IRC00:38
*** shardy_afk has joined #openstack-ansible00:41
*** hamzy has joined #openstack-ansible01:08
*** thorst has quit IRC01:16
*** thorst has joined #openstack-ansible01:28
*** exodusftw has quit IRC01:32
*** hw_wutianwei has joined #openstack-ansible01:33
*** exodusftw has joined #openstack-ansible01:38
*** thorst has quit IRC01:46
*** thorst has joined #openstack-ansible02:07
*** thorst has quit IRC02:07
*** exodusftw has quit IRC02:14
*** japestinho has joined #openstack-ansible02:20
*** exodusftw has joined #openstack-ansible02:22
*** markvoelker has quit IRC03:12
*** udesale has joined #openstack-ansible03:41
*** gkadam__ has joined #openstack-ansible03:50
*** thorst has joined #openstack-ansible04:08
openstackgerritLogan V proposed openstack/openstack-ansible master: [WIP] Pin ceph apt packages in playbook  https://review.openstack.org/50124904:12
*** thorst has quit IRC04:13
*** gouthamr has quit IRC04:19
*** chhavi has joined #openstack-ansible04:30
*** marst has joined #openstack-ansible04:45
*** marst has quit IRC04:50
*** chyka has joined #openstack-ansible04:57
*** chyka has quit IRC05:02
*** thorst has joined #openstack-ansible05:09
*** DanyC has joined #openstack-ansible05:11
*** markvoelker has joined #openstack-ansible05:13
*** thorst has quit IRC05:14
*** reset has joined #openstack-ansible05:27
*** DanyC has quit IRC05:44
*** markvoelker has quit IRC05:47
*** chhavi has quit IRC05:56
csmartvvkhfkhlllceikdghhjkfflibgitlenlfrlljbfjeerj05:57
csmartfortunately not a password :-)05:57
*** drifterza has joined #openstack-ansible06:00
*** dxiri has quit IRC06:06
*** dxiri has joined #openstack-ansible06:07
*** thorst has joined #openstack-ansible06:10
*** thorst has quit IRC06:15
*** Oku_OS-away is now known as Oku_OS06:16
neithcsmart: cat ? :D06:32
*** markvoelker has joined #openstack-ansible06:44
*** armaan has joined #openstack-ansible06:52
*** Taseer has joined #openstack-ansible06:54
*** mudpuppy has quit IRC06:57
*** mudpuppy has joined #openstack-ansible06:57
*** yolanda has joined #openstack-ansible07:07
*** thorst has joined #openstack-ansible07:11
*** thorst has quit IRC07:16
*** markvoelker has quit IRC07:18
*** DanyC has joined #openstack-ansible07:39
hwoaranggood morning07:40
*** DanyC has quit IRC07:44
*** armaan has quit IRC07:48
*** DanyC has joined #openstack-ansible07:54
*** pbandark has joined #openstack-ansible08:04
*** shardy_afk is now known as shardy08:07
andymccrmorning!08:09
andymccrready to go: https://review.openstack.org/#/c/500569/08:09
*** thorst has joined #openstack-ansible08:12
*** markvoelker has joined #openstack-ansible08:15
*** thorst has quit IRC08:17
*** vnogin has joined #openstack-ansible08:18
odyssey4meBOOM!08:27
evrardjp everytime they drop the bombs! ...08:29
evrardjpgood morning08:29
odyssey4mere: https://review.openstack.org/501641 - are we sure we actually use shade in ocata anywhere?08:30
evrardjpyes08:31
evrardjpgit checkout stable/ocata on tempest08:31
evrardjpgrep os_image:08:31
*** aanorbel has joined #openstack-ansible08:31
evrardjpon os_tempest*]08:31
odyssey4meah ok08:31
evrardjpnot needed for N08:32
odyssey4mewell, considering that it's not in u-c, I wonder if it might not work if we add the constraint here: https://github.com/openstack/openstack-ansible-os_tempest/blob/stable/ocata/defaults/main.yml#L15108:33
odyssey4methat way we won't be installing shad everywhere in our role tests (g-r-p is used as requirements in role tests)08:34
neithmorning !08:34
evrardjpyeah I thought of that after the conversation we had yesterday08:35
evrardjpwe are using it also in the role test of glance08:37
evrardjpin O08:37
odyssey4mebut that's just in the test, where it is installed on its own and is isolated08:37
odyssey4meboth those places can use a pinned version08:37
evrardjpoctavia is not in O08:38
odyssey4methere is no need for the version they use to be updated constantly08:38
evrardjpmagnum is using it too08:38
*** electrofelix has joined #openstack-ansible08:38
odyssey4meoh bother, it just keeps on giving :p08:38
evrardjphaha08:38
evrardjptoo bad pip isn't smarter08:39
evrardjpwell wait08:39
neithI upgraded to 14.2.8, this time run the bootstrap script, played setup host and setup-infra.yml without any issues, now setup-openstack.yml fails cause the variable ansible_local['openstack_ansible'] is not defined , so I checked and discoverd it should exists in /etc/ansible/facts.d however there are no files in  this dir in almost all container but repo_container . I replayed setup-infra.yml but still no08:47
neithfiles in /etc/ansible/facts.d of containers. Any way to force the creation of /etc/ansible/facts.d files?08:47
evrardjpthen there was something wrong in one of your previous plays08:48
neithevrardjp: looking again in logs08:48
evrardjpare you running last version of N ?08:49
*** markvoelker has quit IRC08:49
odyssey4meneith have you also ensured that your roles are up to date?08:49
evrardjpodyssey4me: what's the approach you'd prefer to take?08:49
neithodyssey4me: I did08:49
odyssey4meneith oh I see you did08:50
odyssey4meevrardjp dunno, that will work for now I guess - can see how it goes08:50
neithevrardjp: I only have this08:51
neithhttps://www.irccloud.com/pastebin/R18WgxUZ/08:51
evrardjpodyssey4me: yeah. But we'll be stuck forever with that. Let's see later indeed08:51
evrardjpneith: I am more talking about tasks that drop these facts files08:52
neithls08:52
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server master: Add release note for RabbitMQ ulimit  https://review.openstack.org/50146008:53
evrardjpmhayden: with ARA now part of our world, do we still need that debug_message_collector?08:54
odyssey4meneith that error has nothing to do with local facts: https://github.com/openstack/openstack-ansible-repo_build/blob/14.2.8/tasks/repo_build_wheels.yml#L69-L8108:55
odyssey4medid you perhaps delete some folders and no re-run the repo-server playbook?08:55
neithI re run setup-hosts.yml then setup-infrastrucutre.yml08:56
neithgrep "failed=1" in my logs returns nothing08:58
neithhttps://paste.ubuntu.com/25488748/08:59
*** aanorbel has quit IRC08:59
odyssey4meneith your report is conflicting - you showed https://www.irccloud.com/pastebin/R18WgxUZ/ as an error, which is a repo container error that has nothing to do with local facts... but your earlier message says that tasks are failing with errors saying ansible_local['openstack_ansible'] is not defined is not defined... and you are now saying that no tasks are showing as failed...09:01
odyssey4meI'm confused09:01
odyssey4meok, I think I see what's going on with https://www.irccloud.com/pastebin/R18WgxUZ/ at least - it's just a deprecationw arning because the os-releases folder is empty and the previous task had no results09:02
odyssey4methat's cosmetic and not of concern09:02
odyssey4meso your repo build is fine09:02
odyssey4meI see that the playbook log you posted relates to the repo build, so that explains the 'no tasks failed' report09:03
odyssey4meso which tasks are actually failing?09:03
neithevrardjp: told the absence might be because of a failing taks in setup-infra thats why I pasted it to show you that there where no error09:03
openstackgerritMerged openstack/openstack-ansible stable/ocata: Pin Shade to last known version  https://review.openstack.org/50164109:03
neithhttps://www.irccloud.com/pastebin/VSRg8SKX/09:04
neiththis is where it fails running setup-openstack.yml09:04
neithas I said the var is not defined cause /etc/ansible/facts.d on this container is empty09:05
neithlike most other containers09:05
odyssey4methat fact is created here: https://github.com/openstack/openstack-ansible-os_keystone/blob/14.2.8/tasks/keystone_install.yml#L112-L14009:06
odyssey4meis that task running?09:06
odyssey4mewe went through this process the other day - please verify that the task is running, and if not then try to debug why by checking each condition09:06
neithodyssey4me: I'll verify again09:07
odyssey4meodd though, because there should be a fact set from the lxc-container-create playbook too, which adds the container variant used to create the container09:07
odyssey4mehttps://github.com/openstack/openstack-ansible-lxc_container_create/blob/14.2.8/tasks/container_create.yml#L461-L46609:08
odyssey4meyou may wish to verify that's also being executed09:08
neithodyssey4me: well I might move /etc/ansible from the deployement host and rebootstrap09:08
*** vnogin has quit IRC09:13
*** thorst has joined #openstack-ansible09:13
*** thorst has quit IRC09:17
evrardjpneith: I didn't mean a failed a task only, some task for fact could have been skipped09:18
evrardjpI wanted to have an overview of the log to discover more09:19
evrardjpodyssey4me: maybe we should just have a mechanism to bail out if there is no <whatever> local fact09:20
evrardjpexample, we expect this fact to be setup, fail if you don't have it09:20
*** yifei has quit IRC09:21
*** vnogin has joined #openstack-ansible09:23
hwoarangthis https://review.openstack.org/#/c/501273/ appear to be stuck. How do you normally unblock it? with a 'recheck' i suppose?09:28
*** shardy is now known as shardy_mtg09:29
evrardjphwoarang: done09:30
hwoaranghmm09:31
hwoarangbut it already have W+109:31
hwoarangstrange09:31
hwoarang*had09:31
odyssey4meevrardjp that's exactly what it's doing09:33
odyssey4methe install tasks add it if it's not there, later tasks will fail if it is not09:33
evrardjpgreat09:33
evrardjpi am just worried about neith issue09:34
evrardjpI should worry I guess :p09:34
odyssey4mehwoarang if it was dependent on another patch, then it needs an event of some kind for zuul to pick it up09:34
odyssey4meso removing and re-adding workflow, or adding another workflow does the trick... as will a recheck09:34
odyssey4menot sure if that was the case here - I don't see 'depends-on' in the commit message09:35
hwoarangodyssey4me: ah ok thank you09:36
evrardjpyeah that was weird, I thought of a rebase on top of it, but I don't see anything in "related" either09:36
evrardjprecheck does a recheck, so +w in this case was better09:37
hwoarangodyssey4me: shall https://review.openstack.org/#/c/497196/2 be just about the ceph_client inclusion?09:39
odyssey4mehwoarang - oh, silly me, well spotted there - I didn't realise that the image name there was ocata :p09:41
odyssey4meyeah, lemme revise09:41
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: Cleaning up lxc-net tasks  https://review.openstack.org/50127309:42
odyssey4mehwoarang abandoned mine and restored yours09:44
odyssey4methanks for that09:44
hwoarangah ok lets see09:45
evrardjpohoh what a sync odyssey4me09:46
*** mrch has joined #openstack-ansible09:52
*** vnogin has quit IRC10:03
*** vnogin has joined #openstack-ansible10:05
*** szaher has quit IRC10:12
*** vnogin has quit IRC10:12
*** vnogin has joined #openstack-ansible10:13
*** thorst has joined #openstack-ansible10:14
*** szaher has joined #openstack-ansible10:15
*** thorst has quit IRC10:18
openstackgerritMerged openstack/openstack-ansible stable/pike: Bump SHA's prior to releasing 16.0.0  https://review.openstack.org/50056910:21
evrardjpcould I get some reviews on https://review.openstack.org/#/c/494438/  this way it gets out of the way?10:21
evrardjpandymccr: ^ woot merged!10:21
evrardjpyou propose I guess?10:21
evrardjpodyssey4me: do you have an idea why http://status.openstack.org/openstack-health/#/job/periodic-openstack-ansible-deploy-translations-master-ubuntu-xenial first link in recent run is reported fail, while the logs say success, and ARA seems to have runned fine?10:24
odyssey4meevrardjp no idea10:26
andymccrevrardjp: nice10:26
andymccrevrardjp: heading into the office then we release!10:26
evrardjpahaha great10:26
evrardjpandymccr: on top of that ubuntu jobs seem to have passed our testing. All of them10:27
evrardjpfor pike10:27
evrardjpupgrade and all10:27
odyssey4menone of them ran today yet10:28
evrardjpoh yeah, that's 7th.10:30
evrardjpanyway, good state :p10:30
evrardjpodyssey4me: how is the planning done ? With the jobs I mean10:31
odyssey4meplanning?10:31
evrardjpShould we propose when they run, or it's a best effort thing?10:31
odyssey4methey're scheduled daily, but when they actually execute depends on resources available... the jobs have started, but no nodes have been available yet10:32
odyssey4meI don't know when the daily scheduled run is set - I think it's some sort of global setting.10:32
evrardjpok10:33
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Add release note for RabbitMQ ulimit  https://review.openstack.org/50146110:33
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/ocata: Add release note for RabbitMQ ulimit  https://review.openstack.org/50146210:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-rabbitmq_server stable/newton: Add release note for RabbitMQ ulimit  https://review.openstack.org/50146310:34
*** udesale has quit IRC10:35
openstackgerritMerged openstack/openstack-ansible master: Make pkg_locations overridable easier  https://review.openstack.org/46331510:36
openstackgerritMerged openstack/openstack-ansible stable/pike: scripts: scripts-library.sh: Fix tracepath logging  https://review.openstack.org/50112510:36
hwoaranghow do you normally use the ara output? is there a server running already or do you examine results locally?10:38
odyssey4mehwoarang IIRC we only install it and generate the static results in nodepool10:42
*** stuartgr has joined #openstack-ansible10:43
-openstackstatus- NOTICE: Our CI systems experience a hickup, no new jobs are started. Please stay tuned and wait untils this resolved.10:46
*** markvoelker has joined #openstack-ansible10:46
evrardjpahah we get the first signs :)10:46
evrardjpwell I am maybe too fast to call wolf10:46
hwoarangodyssey4me: when can I see these results?10:49
hwoarang*where10:49
hwoarangi am missing some pieces of the puzzle and i can't find much on google10:50
hwoarangoh found them10:57
hwoarangthis is so cool...10:57
* hwoarang is obviously catching up with different bits and pieces10:57
*** pbandark1 has joined #openstack-ansible11:00
*** pbandark has quit IRC11:01
*** pbandark1 is now known as pbandark11:01
*** gparaskevas has joined #openstack-ansible11:05
csmartneith: yubikey ;-)11:06
*** vnogin has quit IRC11:06
*** jbadiapa has quit IRC11:09
*** jbadiapa has joined #openstack-ansible11:09
*** vnogin has joined #openstack-ansible11:10
*** thorst has joined #openstack-ansible11:14
*** thorst has quit IRC11:19
*** markvoelker has quit IRC11:20
odyssey4mehwoarang yeah ARA is great - when 1.0 hits it will also show the vars/inventory as seen by the playbook11:20
odyssey4meit's going to make troubleshooting a lot easier11:20
evrardjpthis will be MASSIVE for us11:21
evrardjpwe have so many variables...11:21
*** Taseer has quit IRC11:27
*** vnogin has quit IRC11:30
*** smatzek has joined #openstack-ansible11:32
neithARA seems great indeed, thks for the tip11:33
*** vnogin has joined #openstack-ansible11:33
*** drifterza has quit IRC11:43
*** qiliang27 has quit IRC11:47
odyssey4meneith a word of warning, ARA has jinja requirements which are higher than those compatible with Ansible 2.1.x, so it doesn't work with Newton11:55
neithodyssey4me: I won"t use it for OSA, but in my jenkins CI for k8S, thks for the advise11:56
*** thorst has joined #openstack-ansible12:02
* dmsimard ara senses tingling12:04
openstackgerritLogan V proposed openstack/openstack-ansible master: [WIP] Pin ceph apt packages in playbook  https://review.openstack.org/50124912:10
*** fellypefca has joined #openstack-ansible12:11
*** fellypefca has left #openstack-ansible12:12
*** shardy_mtg is now known as shardy12:16
*** markvoelker has joined #openstack-ansible12:16
*** gparaskevas has quit IRC12:17
*** gkadam__ has quit IRC12:31
*** markvoelker has quit IRC12:34
*** markvoelker has joined #openstack-ansible12:34
*** SerenaFeng has joined #openstack-ansible12:37
*** woodard_ has quit IRC12:45
*** dxiri has quit IRC12:45
*** woodard has joined #openstack-ansible12:45
*** dxiri has joined #openstack-ansible12:46
dmsimardevrardjp: the thing about variables is that they are compiled just-in-time, and they can potentially change all the time12:49
dmsimardevrardjp: ara will pick up var *files* contents (just like it does for playbook and task files)12:49
dmsimardara could maybe, technically, record the hostvars of a host when recording a result (for each result) but that would get heavy really quickly12:50
*** dxiri has quit IRC12:50
dmsimardperhaps eventually for a debug/verbose mode in ara12:50
dmsimardodyssey4me: ^12:51
odyssey4medmsimard ah, I was wondering what was actually going to happen there12:52
odyssey4methat makes sense12:52
odyssey4mefull debug would be nice, but as you say - heavy12:52
dmsimardyeah, if we're talking about 2000 tasks on 5 hosts (10 000 results) that's 10 000 * hostvars :)12:53
dmsimardand hostvars is pretty verbose, it contains far too many things12:53
andymccrso we need to start deploying big data!12:53
dmsimardhostvars contains almost the entire context12:53
odyssey4methis is very probably part of why our deploy times are so long12:55
*** SerenaFeng has quit IRC12:56
*** SerenaFeng has joined #openstack-ansible13:02
evrardjp[m]at least hostvars are a subset of vars...13:07
*** vnogin has quit IRC13:10
mgariepymorning everyone13:12
andymccrmornin mgariepy13:15
mgariepywhat's up ?13:15
*** Dinesh_Bhor has quit IRC13:16
*** vnogin has joined #openstack-ansible13:18
mgariepyanyone have a few minutes to review leap upgrade patches ? https://review.openstack.org/#/c/501765/ and https://review.openstack.org/#/c/500839/13:19
openstackgerritAndy McCrae proposed openstack/openstack-ansible master: Bump openstack_release version  https://review.openstack.org/50205813:23
andymccrahh wait13:23
andymccri am derping13:23
odyssey4melol13:23
andymccri was sure i had set that. wow its a good thing this is my last one ;P13:24
odyssey4mewrong branch13:24
odyssey4meclearly you need to step away, into a bar :p13:24
mhaydenhwoarang: looking at https://review.openstack.org/496376 -- i'm keeping the same default for opensuse13:25
mhaydenit was always set to install from 'file'13:26
*** hw_wutianwei has quit IRC13:26
mhaydeni just moved that default from defaults/main.yml -> vars/suse.yml13:26
hwoarangoh13:27
hwoarangi got confused then :(13:27
*** gouthamr has joined #openstack-ansible13:27
andymccrhaha maybe i should do the release from a bar :D quick access to celebratory drinks13:27
evrardjphaha13:28
evrardjpI think it's worth celebrating!13:28
hwoarangmhayden: but for the tests it should always be 'file' so you always know what you install during upgrade etc. what do you think?13:28
mhaydenwell, the CentOS method uses distro for both now though13:28
mhaydenwith pins13:28
mhaydeni'd like to convery ubuntu to that soon as well13:29
mhaydenso it would be good to test CentOS in the same way that it would be installed by default (using distro)13:29
hwoarangbut the test will break if the repository suddently updates the rabbitmq version13:29
mhaydenhwoarang: that's why 'yum versionlock' is used13:30
hwoaranghmm13:30
mhaydenwe were already doing that for erlang13:30
mhaydenso i am proposing we do it for rabbitmq, too13:30
*** chhavi has joined #openstack-ansible13:30
andymccroh i forgot we actually just do an rc13:31
andymccrand thats all really13:31
*** smatzek has quit IRC13:31
hwoarangi must be missing something13:31
hwoarangin the tests you try stuff like 'install version 3.6.6' and then update to '3.6.9' and check that 3.6.9 is running. this all works well if you install the rpms directly. but if the repo has 3.6.11 then the test will fail since you are querying for the wrong version13:32
mhaydenthe current process for CentOS is: install a billion erlang packages from upstream w/ version locks13:33
mhaydenalso, rabbitmq is installed from a file13:33
hwoarangyes13:33
mhaydenso my proposal is: use the RabbitMQ recommended single erlang package with HiPE from their repo (with version locks)13:33
mhaydenso that's a slight modification13:33
mhaydenmodification #2 is to install rabbitmq from the RabbitMQ recommended repo, using versionlock13:33
mhaydeninstead of from a file13:34
mhaydenthis saves time, improves rabbitmq performance, and reduces package installs for centos13:34
hwoarangok then13:34
mhaydenit also avoids the situation we were in where openstack-ocata and epel both provided erlang13:34
mhaydenand they conflicted13:34
mhaydenit distances us from that problem more13:35
hwoarangi see13:35
mhaydeni should probably add a release note in that patch13:35
*** galstrom_zzz is now known as galstrom13:36
mhaydenhwoarang: i put a -1 on it until i can add a reno that explains it13:36
*** chas has joined #openstack-ansible13:38
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: [WIP] Remove RHEL 6 STIG content  https://review.openstack.org/50188313:39
hwoarangi have absolutely no clue how the tests pass on centos13:40
mhaydenhwoarang: the version locks are adjusted during the deploy13:40
mhaydenthey're set to 3.6.6 initially13:40
mhaydenso only 3.6.6 gets on there13:40
mhaydenthen the locks are changed to 3.6.913:40
mhaydenand rabbitmq gets updated13:40
hwoarangoh so upstream repo provides multiple versions?13:41
mhaydenyeah13:41
hwoarangahh13:41
odyssey4methat rabbitmq role should be changed to do deploys and upgrades seperately for tests13:41
mhayden3.6.11 back into the 3.5's13:41
*** vnogin has quit IRC13:41
odyssey4meit'd make it easier to understand what's going on there13:41
mhaydenodyssey4me: yessir13:41
odyssey4meperhaps a leaf can be taken from the mariadb book and someone can do it?13:41
odyssey4memariadb role I mean == galera_server13:42
odyssey4methat misnaming still irks me13:42
hwoarangi could give it a shot once this settles down13:42
*** jbadiapa has quit IRC13:43
openstackgerritMajor Hayden proposed openstack/openstack-ansible-rabbitmq_server master: Optimize CentOS Erlang installation  https://review.openstack.org/49637613:45
-openstackstatus- NOTICE: nodepool issue related to bad images has been resolved, builds should be coming back online soon. Restarted gerrit due to reasons. Happy Friday.13:45
openstackgerritMajor Hayden proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade to RabbitMQ 3.6.11  https://review.openstack.org/49680113:46
*** BjoernT has joined #openstack-ansible13:46
mhaydenhwoarang: i'll get your stuff into that pip_install patch13:48
hwoarangthank you! lets see13:48
mhaydenevrardjp: for https://review.openstack.org/499601 -- are you suggesting that we don't install these packages at all?13:48
mhaydeni'd be curious to see if that works13:48
*** vnogin has joined #openstack-ansible13:49
odyssey4memhayden they ere added for SNI support13:49
odyssey4meif we ship our ansible requirements in a venv, then sure - they won't be needed13:49
mhaydenah, that's right13:49
* mhayden knew there was a reason13:50
odyssey4mebut for now, we need SNI support in the system packages13:50
evrardjpodyssey4me: mhayden precisely.13:50
odyssey4mewhat may be worth doing is not using get-pip and using a system package... then trying to ensure we use backports or whatever to get a reasonable pip versin13:50
evrardjpyeah13:50
mhaydenodyssey4me: might be worth trying13:50
evrardjpwell not mix and match would be good, but We do what we have to do until we arrive to a good place13:51
odyssey4mealthough the general idea - or one of them - in https://review.openstack.org/501296 is to rather put what we need into venvs, ships those around and then only install the smallest footprint possible on the hosts (should only need python-virtualenv)13:51
odyssey4mein the venv we can set our pip version and all that13:51
neithodyssey4me: well to conclude, I just runned setup-hosts.yml, setup-infrastructure.yml and setup-openstack.yml and it worked. So I have no clue what was the issue13:51
odyssey4meneith was that after rebootstrapping with a clean /etc/ansible start?13:52
odyssey4mesetup-hosts would have done the lxc-container-create bits, putting that local fact down... the others would have done the rest in sequence13:52
odyssey4meit's plausible that your previous attempts using various limits/tag mixes got it all confused13:53
*** smatzek has joined #openstack-ansible13:53
neithodyssey4me: no I did not clean /etc/ansible as I wanted to try, I really wanted to find out the issue13:53
*** dxiri has joined #openstack-ansible13:53
openstackgerritMerged openstack/openstack-ansible-rsyslog_client master: Test postrotate works  https://review.openstack.org/49443813:53
neithodyssey4me: yes I assume so13:53
odyssey4meodd - well, hopefully that is that, no more trouble :)13:53
neithodyssey4me: Yes thats awesome13:53
neithyou are all awesome13:53
neiththks again13:54
odyssey4merepeat after me - when you check out a new version, bootstrap ansible :) ;)13:54
odyssey4mehappy friday!13:54
mgariepylol13:54
neithodyssey4me: I wrote it down in bold red in our wiki13:54
neithlol13:54
odyssey4meheh, good show13:54
openstackgerritMajor Hayden proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip  https://review.openstack.org/49960113:55
odyssey4memaybe one day we can implement some sort of packaging so that this doesn't have to get used with all this developer workflow13:55
neithI have now new skills with galera and rabbitmq clusters ;)13:55
openstackgerritMajor Hayden proposed openstack/openstack-ansible stable/pike: Suppress curl warning w/shell module  https://review.openstack.org/50126013:56
odyssey4meoh wonderful - hopefully you can help make what we have better then, because I'm sure it's over complicated13:56
openstackgerritMajor Hayden proposed openstack/openstack-ansible stable/ocata: Suppress curl warning w/shell module  https://review.openstack.org/50126113:57
neithnext goal upgrade from N to O13:57
mhaydenevrardjp / andymccr: feel free to throw in your $0.02 on this one for project-config -> https://review.openstack.org/50188013:57
*** dxiri has quit IRC13:58
andymccrmhayden: seems ok to me14:00
andymccrmhayden:  as an fyi you will probably stop being able to gate against trusty at some point in the near future14:00
mhaydenandymccr: i figured :/14:00
mhaydenwe'll just cross that bridge when we get there i guess14:01
andymccri kinda hope it stays around until at least the end of newton :)14:01
hwoarangmhayden: thank you for all the explanations in rabbitmq. i am going to cleanup suse based on your patch as well14:02
mhaydenhwoarang: woot!14:02
*** esberglu has joined #openstack-ansible14:02
sauloaislanMorning!14:02
*** SerenaFeng has quit IRC14:03
*** SerenaFeng has joined #openstack-ansible14:03
*** SerenaFeng has quit IRC14:03
openstackgerritMerged openstack/ansible-hardening stable/pike: Check apparmor_status output  https://review.openstack.org/50187714:05
openstackgerritMerged openstack/ansible-hardening stable/pike: Add release note for F26 support  https://review.openstack.org/50005514:06
*** dxiri has joined #openstack-ansible14:12
*** dxiri_ has joined #openstack-ansible14:15
*** agrebennikov has joined #openstack-ansible14:18
*** dxiri has quit IRC14:19
*** dxiri_ has quit IRC14:20
*** marst has joined #openstack-ansible14:22
*** dave-mccowan has joined #openstack-ansible14:23
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests master: Remove ubuntu-14 reference from bindep  https://review.openstack.org/50207014:26
*** vnogin has quit IRC14:26
*** vnogin has joined #openstack-ansible14:28
hwoarangmhayden: i am curious. since you install rabbitmq from an external repo (ie no yum/centos repos) shouldn't the install method me 'external_repo' ?14:29
hwoarangit seems that distro==external_repo for yum. maybe that's by design i suppose since the official repos do not have rabbitmq-server14:30
hwoarangi really shouldn't be doing that on a friday afternoon!14:33
*** mrch has quit IRC14:34
mgariepyhwoarang, rdo repo rabbitmq pkg14:38
*** cshen has quit IRC14:41
hwoaranghmm14:42
mgariepyeverything is complicated with centos :) haha14:43
hwoarangtrue14:44
hwoarangi will comment on the bug and then i will shut up for today. i will have the whole next week to bug you anyway14:45
mhaydennext week should be fun!14:45
mgariepyis cloudnull on vacation ?14:46
mhaydenindeed14:46
mhayden:/14:46
mgariepy:(14:46
mhaydenin places much more fun than Denver14:46
mgariepyhehe14:46
mhaydenif you live in Denver, what are you called?14:46
mhaydenDenvites? Denvernarians? Denvernians?14:46
*** Taseer has joined #openstack-ansible14:47
mgariepyDenverite14:47
asettlemhayden: Denverites would be where my brain went14:47
asettlemgariepy: jinx14:47
mgariepywell wikipedia served me well.14:47
* mhayden yields to asettle since her mastery of English is much better than his14:47
asettleHahahhahaha14:47
asettleI really wouldn't go that far14:47
mgariepyMy french is better;p14:48
asettleShow off :p14:48
andymccrnot sure thats showing off14:48
asettleOh but it is.14:49
mgariepyare all of you going to sydney ?14:50
odyssey4methe likelihood for me is perhaps less than zero14:51
andymccri imagine there will be some representation but not a great amount14:52
hwoarangi'm not14:53
mgariepyit's kinda very far :)14:53
openstackgerritMerged openstack/openstack-ansible-tests master: Remove RDO install in tests repo  https://review.openstack.org/50187314:55
mhaydenso it was suggested that we remove the debug message collector plugin since we have ARA14:56
mhaydeni'm happy to do that unless people have strong reservations about keeping it14:56
openstackgerritMajor Hayden proposed openstack/openstack-ansible-tests stable/pike: Remove RDO install in tests repo  https://review.openstack.org/50208114:57
mhaydenalso, if y'all can help me get these backports merged, it will please the centos users ;) https://review.openstack.org/#/q/1d206a59f70dd52d6966016cc2a500953692cfaf,n,z14:59
*** shardy is now known as shardy_mtg15:00
mgariepymhayden, got my +15:01
mgariepy215:02
*** x0ff has joined #openstack-ansible15:03
*** agrebennikov has quit IRC15:04
*** agrebennikov has joined #openstack-ansible15:04
*** Oku_OS is now known as Oku_OS-away15:05
*** chhavi has quit IRC15:06
x0ffafter neutron-config my neutron-server reports no agents alive15:07
*** chhavi has joined #openstack-ansible15:07
mhaydenmgariepy: gracias ;)15:07
mhaydenand thanks to ol' hwoarang, too15:07
x0ffagents are started and send rpcs to neutron but get timeouts15:07
x0ffhow can i find out what's going on when there is nothing releated in debug logs on agents and server?15:09
openstackgerritMarc Gariépy (mgariepy) proposed openstack/openstack-ansible-ops master: Create a var file to be sourced with vars.sh  https://review.openstack.org/50176515:10
*** pbandark1 has joined #openstack-ansible15:14
*** Taseer has quit IRC15:14
*** pbandark has quit IRC15:15
*** pbandark1 is now known as pbandark15:15
openstackgerritMajor Hayden proposed openstack/openstack-ansible-plugins master: Remove debug_message_collector plugin  https://review.openstack.org/50209415:16
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_neutron master: Ensure QoS extension is enabled  https://review.openstack.org/49458415:18
jamesdentonIn the os_neutron role, the iputils-arping package is a dependency defined under neutron_distro_packages in vars/ubuntu-16.04.yml, but i'm finding in some environments it is not installed. The distro-default 'arping' remains15:21
jamesdentonAnyone have a Newton environment handy and can run 'arping -v/-V'?15:21
*** chyka has joined #openstack-ansible15:22
*** DanyC has quit IRC15:27
*** DanyC has joined #openstack-ansible15:28
*** drifterza has joined #openstack-ansible15:28
*** DanyC has quit IRC15:28
*** shardy_mtg is now known as shardy15:37
mgariepyjamesdenton, http://paste.openstack.org/show/620732/15:39
jamesdentonThanks, mgariepy. That's OSA v14?15:40
mgariepythat's for neutron_agent container15:40
mgariepy14.2.7 + some patches15:40
jamesdentonperfect, thank you15:40
openstackgerritMerged openstack/openstack-ansible-lxc_hosts stable/pike: Disable lxc-net on CentOS  https://review.openstack.org/50127515:42
*** pbandark has quit IRC15:50
*** vnogin has quit IRC15:51
*** pbandark has joined #openstack-ansible15:52
*** gfa has quit IRC16:02
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Queens docs cleanup  https://review.openstack.org/50210916:03
openstackgerritMajor Hayden proposed openstack/ansible-hardening stable/pike: Final docs updates for Pike  https://review.openstack.org/50211016:05
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Queens doc updates + removal of RHEL 6 STIG  https://review.openstack.org/50188316:07
spotzmhayden: you sure about lead 42.3?16:24
*** woodard has quit IRC16:24
spotzI mean leap16:24
*** dxiri has joined #openstack-ansible16:27
*** szaher has quit IRC16:31
*** szaher has joined #openstack-ansible16:37
*** vnogin has joined #openstack-ansible16:43
*** shardy has quit IRC16:47
*** vnogin has quit IRC16:47
taskeris there a way to instruct the repo playbooks to not build everytime it runs? nothing's changed in the last hour, I don't need a complete rebuild.16:50
*** DanyC has joined #openstack-ansible17:11
*** esberglu has quit IRC17:11
*** szaher has quit IRC17:15
*** chhavi has quit IRC17:17
*** x0ff has quit IRC17:27
mhaydenspotz: according to hwoarang, the hardening role works fine on leap 42.2/3, hence the 42.x in the docs17:28
spotzokie17:29
*** vnogin has joined #openstack-ansible17:41
*** esberglu has joined #openstack-ansible17:41
*** dsantoro has joined #openstack-ansible17:43
dsantoroI would like to install DVR on top of an environment created with openstack-ansible ans using linux-bridges, is that possible or I need to use Open vSwitch ?17:45
*** vnogin has quit IRC17:45
mhaydenjamesdenton: on https://bugs.launchpad.net/neutron/+bug/1715734 are you submitting a patch for that?17:45
openstackLaunchpad bug 1715734 in neutron "Gratuitous ARP for floating IPs not so gratuitous" [Medium,In progress] - Assigned to James Denton (james-denton)17:45
*** esberglu has quit IRC17:45
openstackgerritMajor Hayden proposed openstack/openstack-ansible-plugins master: Remove debug_message_collector plugin  https://review.openstack.org/50209417:46
mhaydendsantoro: we use linuxbridge by default17:47
mhaydeni have a cloud deployed with it as well17:47
jamesdentonmhayden I submitted a patch for Neutron, but abandoned it when i found it wasn't Neutron's fault. Signs point to OSA or RPC right now, so trying to track that down. Something, somewhere, is installing the wrong arping package17:47
mhaydenah okay17:47
mhaydenjamesdenton: newton?17:48
jamesdentoneverything?17:48
jamesdenton:)17:48
mhaydenjamesdenton: https://github.com/openstack/openstack-ansible-os_neutron/blob/stable/newton/vars/ubuntu-14.04.yml#L3217:48
mhaydenhttps://github.com/openstack/openstack-ansible-os_neutron/blob/stable/newton/vars/ubuntu-16.04.yml#L3217:48
jamesdentonyeah, i saw that. It may be support playbooks after the fact17:48
mhaydenoh17:48
*** electrofelix has quit IRC17:50
dsantoro@mhayden: I saw linux-bridge is used by default by openstack-ansible. But is it possibile to add a DVR (Ditributed Virtual Router) functionaliy using linux-bridge ?17:51
mhaydendsantoro: i don't think we have that supported quite yet17:51
openstackgerritMatt Thompson proposed openstack/openstack-ansible-ops master: [WIP] MNAIO updates  https://review.openstack.org/50213217:51
mhaydenjamesdenton: here's what i'm seeing in ocata -> https://gist.githubusercontent.com/major/0f92cc3e652830dbf6524a825b70c056/raw/30bf2a915ec0472f463780222cec8f09fa5fa7db/gistfile1.txt17:52
mhaydenthat's on CentOS, of course17:52
dsantoro@mhayden: thanks, I will setup an environment using Open vSwitch in this case17:52
mhaydeni think we have some folks on this channel that use OVS now17:52
mhaydenjamesdenton: https://github.com/openstack/neutron/blob/90b3586176aeeaf95ef670ac721a897774550ede/neutron/tests/common/net_helpers.py#L155-L16617:53
mhaydenlooks like it expects 'arping' to be present17:53
jamesdenton@mhayden So the syntax is good. The issue lies with the binaries that get installed via iputils-arping vs arping. The behavior is different, even though the flags are the same. Similar enough so that Neutron doesn't complain but it doesn't do what you want.17:53
mhaydenjamesdenton: trusty or xenial?17:54
* mhayden hopes for xenial17:54
jamesdentonhehe, xenial. But iputils-arping and arping packages have the same binary name: arping. Both do similar things, but the flags are different and the generated arp packet is different. Neutron needs iputils-arping, OSA should install iputils-arping, but i've noticed in RPC environments arping is the one installed.17:56
*** dsantoro has quit IRC18:00
*** dsantoro has joined #openstack-ansible18:00
jmccrorytasker it shouldn't as long as you're not overriding these somewhere, https://github.com/openstack/openstack-ansible-repo_build/blob/master/defaults/main.yml#L55-L68, and you're on a recent enough release that included those variables18:01
mhaydenjamesdenton: wow18:01
*** poopcat has joined #openstack-ansible18:03
logan-that sounds like a blast to troubleshoot18:03
*** poopcat has quit IRC18:07
*** albertcard1 has joined #openstack-ansible18:07
*** chas has quit IRC18:08
*** dsantoro has quit IRC18:08
*** chas has joined #openstack-ansible18:09
* mhayden just updated his CentOS deployment to the head of Ocata this morning -- all is well18:09
mhaydenthat lxc-net fix is helpful18:09
jamesdentonWhen is the next Pike tag dropped? This unbound bug gets me every time18:10
*** woodard has joined #openstack-ansible18:11
*** chas has quit IRC18:13
*** albertcard1 is now known as poopcat18:19
openstackgerritMarc Gariépy (mgariepy) proposed openstack/openstack-ansible-ops master: Echo warning to export TODOLIST var.  https://review.openstack.org/50176518:20
spotzjamesdenton: It should be next week I think for us18:20
jamesdentonthanks spotz18:21
*** sauloaislan has quit IRC18:26
*** woodard has quit IRC18:28
*** stuartgr has left #openstack-ansible18:30
*** vishwana_ has quit IRC18:35
*** esberglu has joined #openstack-ansible18:35
*** esberglu has quit IRC18:40
*** dxiri has quit IRC18:42
*** esberglu has joined #openstack-ansible18:46
*** strattao has quit IRC19:08
*** galstrom is now known as galstrom_zzz19:08
*** strattao has joined #openstack-ansible19:32
openstackgerritMatt Thompson proposed openstack/openstack-ansible-ops master: [WIP] multi-node-aio updates  https://review.openstack.org/50213219:50
*** lucasxu has joined #openstack-ansible20:14
*** strattao has quit IRC20:16
*** drifterza has quit IRC20:21
openstackgerritLogan V proposed openstack/openstack-ansible-lxc_hosts master: Do not install recommended packages  https://review.openstack.org/50217220:21
openstackgerritLogan V proposed openstack/openstack-ansible master: [WIP] Pin ceph apt packages in playbook  https://review.openstack.org/50124920:24
*** smatzek has quit IRC20:25
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server stable/ocata: Add release note for RabbitMQ ulimit  https://review.openstack.org/50146220:26
*** BjoernT has quit IRC20:28
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server stable/newton: Add release note for RabbitMQ ulimit  https://review.openstack.org/50146320:28
openstackgerritMajor Hayden proposed openstack/openstack-ansible-lxc_hosts stable/pike: tasks: lxc_net: Disable the lxc-net service on SUSE  https://review.openstack.org/50217820:37
openstackgerritMajor Hayden proposed openstack/openstack-ansible-lxc_hosts stable/pike: Cleaning up lxc-net tasks  https://review.openstack.org/50217920:38
mhaydenokay folks, i'm looking forward to seeing some of y'all next week in Colorado! :)20:39
mhaydenwell i would look forward to seeing all of you20:39
mhaydenbut only some of you are coming20:39
mhaydenyou know what i mean20:39
mhaydeni figured andymccr would give me grief for phrasing20:39
*** lucasxu has quit IRC20:44
*** strattao has joined #openstack-ansible20:46
*** woodard has joined #openstack-ansible20:53
*** chas has joined #openstack-ansible20:55
*** esberglu has quit IRC20:55
*** esberglu has joined #openstack-ansible20:55
*** gouthamr has quit IRC20:56
*** aludwar has quit IRC20:59
*** aludwar has joined #openstack-ansible20:59
*** aludwar has quit IRC20:59
*** aludwar has joined #openstack-ansible21:00
*** esberglu has quit IRC21:00
*** jwitko has quit IRC21:10
*** vnogin has joined #openstack-ansible21:19
*** gouthamr has joined #openstack-ansible21:24
*** thorst has quit IRC21:32
*** marst has quit IRC21:32
*** ying_zuo has left #openstack-ansible21:39
*** chas has quit IRC21:51
*** chas has joined #openstack-ansible21:52
*** chas has quit IRC21:56
*** BjoernT has joined #openstack-ansible22:05
*** BjoernT has quit IRC22:13
*** vnogin has quit IRC22:13
csmartmhayden: have fun22:16
*** thorst has joined #openstack-ansible22:32
*** vnogin has joined #openstack-ansible22:36
*** thorst has quit IRC22:37
*** dave-mccowan has quit IRC22:39
*** galstrom_zzz is now known as galstrom22:43
*** galstrom is now known as galstrom_zzz22:44
*** galstrom_zzz is now known as galstrom22:46
*** galstrom is now known as galstrom_zzz22:55
openstackgerritMerged openstack/openstack-ansible-lxc_hosts stable/ocata: Disable lxc-net on CentOS  https://review.openstack.org/50127723:02
*** thorst has joined #openstack-ansible23:14
*** chyka has quit IRC23:18
*** thorst has quit IRC23:19
*** SerenaFeng has joined #openstack-ansible23:21
*** SerenaFeng has quit IRC23:24
*** galstrom_zzz is now known as galstrom23:30
*** vnogin_ has joined #openstack-ansible23:42
*** vnogin has quit IRC23:44
*** galstrom is now known as galstrom_zzz23:47
*** askb has quit IRC23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!