Monday, 2015-08-03

*** markvoelker has quit IRC00:02
*** h0mer has joined #openstack-ansible01:46
*** markvoelker has joined #openstack-ansible01:59
*** markvoelker has quit IRC02:03
*** heww has joined #openstack-ansible02:53
*** heww has quit IRC02:54
*** heww has joined #openstack-ansible02:56
hewwhi, all, last week I add trove to os-ansible-deployment myself, and everything works well except that trove guestagent will access rabbitmq cluster to tell trove taskmanager its status.03:02
hewwtrove guestagent which run in nova instance can't connect to rabbitmq cluster using lxc interface linked with br-mgmt03:04
hewwthen I add rabbitmq cluster to haproxy, and trove guestagent connect to rabbit cluster through external_lb_vip_address by haproxy03:06
hewwafter that trove works well03:06
hewwbut, anyone can help me so that trove guestagent can connect to rabbitmq cluster by br-mgmt network which value is {{ rabbitmq_servers }} like other roles.03:08
*** alejandrito has quit IRC03:12
*** ccrouch has left #openstack-ansible03:25
*** JRobinson__ is now known as JRobinson__afk03:36
*** markvoelker has joined #openstack-ansible04:00
*** markvoelker has quit IRC04:04
*** JRobinson__afk is now known as JRobinson__04:04
*** daneyon has joined #openstack-ansible04:43
*** daneyon has quit IRC04:43
*** daneyon has joined #openstack-ansible04:44
*** fawadkhaliq has joined #openstack-ansible05:23
*** heww has quit IRC05:31
*** harlowja has quit IRC05:44
*** daneyon has quit IRC05:48
*** daneyon has joined #openstack-ansible05:49
*** heww has joined #openstack-ansible05:51
*** fawadk has joined #openstack-ansible05:56
*** fawadkhaliq has quit IRC05:58
*** daneyon has quit IRC05:58
*** heww has quit IRC06:00
*** markvoelker has joined #openstack-ansible06:01
*** daneyon has joined #openstack-ansible06:02
*** h0mer has quit IRC06:02
*** javeriak has joined #openstack-ansible06:04
*** markvoelker has quit IRC06:05
*** daneyon has quit IRC06:08
*** daneyon has joined #openstack-ansible06:09
*** javeriak has quit IRC06:19
*** javeriak has joined #openstack-ansible06:19
*** daneyon_ has joined #openstack-ansible06:25
*** fawadk has quit IRC06:27
*** daneyon has quit IRC06:28
*** daneyon_ has quit IRC06:41
*** javeriak has quit IRC06:47
*** javeriak has joined #openstack-ansible06:49
*** JRobinson__ has quit IRC07:06
*** javeriak_ has joined #openstack-ansible07:11
*** javeriak has quit IRC07:11
*** javeriak_ has quit IRC07:14
*** javeriak_ has joined #openstack-ansible07:19
*** javeriak_ has quit IRC07:23
*** javeriak has joined #openstack-ansible07:24
*** pilgrimstack has joined #openstack-ansible07:42
*** javeriak has quit IRC07:43
*** javeriak has joined #openstack-ansible07:43
*** javeriak has quit IRC07:47
*** javeriak has joined #openstack-ansible07:49
*** gparaskevas has joined #openstack-ansible08:00
*** fawadkhaliq has joined #openstack-ansible08:01
*** markvoelker has joined #openstack-ansible08:02
*** markvoelker has quit IRC08:06
*** subscope has joined #openstack-ansible08:10
odyssey4mejaveriak regarding your instance build failure one in three times - are you using swift or a file based glance back-end?08:22
javeriakodyssey4me: nope, i just uploaded a regular cirros image through horizon08:23
odyssey4mejaveriak but what is your glance configured to use as a back-end?08:23
odyssey4meie what is the glance store?08:23
javeriakodyssey4me: not sure, this a standard OSAD deployed cluster, where can I check?08:25
odyssey4mejaveriak if it's a multi-node deployment, then there is no such thing as a 'standard' build - you have many options to choose from :)08:26
odyssey4megrep ^glance_default_store /etc/openstack_deploy/user_variables.yml08:27
*** fawadkhaliq has quit IRC08:28
javeriakglance_default_store: file08:29
*** fawadkhaliq has joined #openstack-ansible08:30
odyssey4meok, so what's happening there is that glance is using the file system to store images08:30
odyssey4meyou have three controllers, with haproxy load balancing between them08:30
odyssey4meif that file store isn't on a shared file system back-end (like an nfs share) then the image is only present once per three calls due to the load balancing08:31
odyssey4meit's best to use a shared backing store for glance, like an nfs share, swift, ceph, etc08:31
odyssey4meyou can work around the issue by doing one of the following: 1) implementing a mechanism to replicate your file store between controllers - something like rsync or whatever08:32
odyssey4me2) shutting down the two glance services that don't have the data required - not a great workaround, but it'll do for testing08:32
odyssey4me3) implementing a different load balancing algorithm for load balancing that always forwards glance api traffic to the same back-end (probably best implemented alongside option 1)08:34
odyssey4me4 switching to a shared back-end data store for glance images (the best option)08:34
javeriakso that means a vm creation call should fail every now and then whenever the calls reache a controller not hosting the image-file?08:35
odyssey4mejaveriak every time the api call asks for an image for which the container servicing the api call is not there, the call will fail08:35
odyssey4methe LB is set to use a round robin LB scheme as I recall, so it makes sense that it'll be a 1 in 3 success08:36
javeriakodyssey4me: yes that makes sense, but since this is how we always deploy through OSAD (ie 3 -controllers), doesnt it implement any of these solutions by default? since this issue will always be there08:42
odyssey4mejaveriak most deployers will use a shared file system back-end, or swift - and ceph is being worked on as another option for 11.2.008:45
odyssey4mejaveriak another option, which RAX uses in its deployments in its own DC's is to use cloudfiles as a back-end08:46
odyssey4methe shared file system back-end of choice is normally some sort of NFS bac-end08:46
javeriakodyssey4me: right, thanks for the explanation! I'll go investigate and see whats the best implementation for us08:48
*** fawadkhaliq has quit IRC09:03
*** fawadkhaliq has joined #openstack-ansible09:06
*** javeriak has quit IRC09:09
*** javeriak has joined #openstack-ansible09:11
*** gparaskevas has quit IRC09:21
*** gparaskevas has joined #openstack-ansible09:27
gparaskevasHello there!09:29
*** fawadkhaliq has quit IRC09:32
gparaskevasodyssey4me: Hello how aare you? I am beggining to do the contrib on that https://bugs.launchpad.net/openstack-ansible/+bug/1446432 . I will clone master i will create a new branch with the issue id and i will then commit?09:32
openstackLaunchpad bug 1446432 in openstack-ansible "Enable HAProxy stats Web UI" [Wishlist,In progress] - Assigned to George Paraskevas (gparaskevas)09:32
*** fawadkhaliq has joined #openstack-ansible09:33
evrardjpgood morning everyone09:44
evrardjpquick question odyssey4me: for the commit I'm owner (but I didn't do the last commit), is it allowed to review it?09:44
*** fawadkhaliq has quit IRC09:45
evrardjpIt's weird to review your own commit, but it's not only mine at the end...09:45
*** fawadkhaliq has joined #openstack-ansible09:45
svgevrardjp: welcome to gerrit club09:48
evrardjphehe thanks svg09:48
evrardjpnow, I'll try to review the ceph integration09:50
*** fawadkhaliq has quit IRC09:58
gparaskevascan someone help me with i commit on a bug please?10:01
*** markvoelker has joined #openstack-ansible10:02
*** markvoelker has quit IRC10:07
*** Ti-mo has quit IRC10:08
*** jmccrory has quit IRC10:08
*** logan2 has quit IRC10:08
*** jmccrory has joined #openstack-ansible10:08
*** Ti-mo- has joined #openstack-ansible10:08
*** logan2 has joined #openstack-ansible10:08
evrardjpgparaskevas: I'm not very experienced with that, but yes, cloning and then branching seems a good approach10:09
gparaskevasevrardjp: ok i have done that but now i am trying to git review and i have some error there.10:09
evrardjpThere are documentation about the best practices for the commits (messages and content)10:10
evrardjphttps://wiki.openstack.org/wiki/GitCommitMessages10:10
evrardjpplease also read this: https://wiki.openstack.org/wiki/How_To_Contribute10:11
evrardjpthe workflow is explained here: http://docs.openstack.org/infra/manual/developers.html10:11
evrardjp(it also explains the agreement that you have to sign for openstack contributions)10:12
evrardjpok gparaskevas10:12
evrardjpgit review -s was working fine?10:12
gparaskevasit seems i shoud create a remote10:12
evrardjpok10:12
evrardjpmaybe you try with the sandbox first: http://docs.openstack.org/infra/manual/sandbox.html#sandbox10:13
javeriakgparaskevas : try this as well, it says nothing about needing to create a sepeerate branch though, you should be able to commit directly with the appropriate Bug ref in the commit-message, http://docs.openstack.org/infra/manual/developers.html#development-workflow10:13
evrardjpgit review -s handles the setup and the creation of the remotes10:13
odyssey4megparaskevas the workflow outlined by evrardjp is correct10:14
evrardjpthere is a debug flag on git review that can help you10:14
odyssey4metypically git clone the repo, then git checkout master; then git checkout -b bug/<bug number>, then make changes and commit, then git review to push it up10:15
evrardjpbut keep in mind that git review -s could fail if you didn't do the previous steps explained in the developpers.html page10:15
odyssey4me++10:15
evrardjp(like creating the accounts, signing the agreements...)10:15
odyssey4meevrardjp usually as the owner you can review if someone's changed your patch, but you shouldn't +2 your own patch (only cores can +2 though)10:16
odyssey4meif your patch has been updated by someone else, then you can add a vote with comment that you're happy with the later changes and have tested, etc10:17
evrardjpI didn't intend (never!) to do a +210:17
evrardjpodyssey4me: seems logical... I was wondering if there was a netiquette to respect10:17
evrardjpjust common sense10:17
evrardjp\o/10:17
odyssey4me:)10:18
*** daneyon has joined #openstack-ansible10:18
evrardjpodyssey4me: Quick question about the launchpad milestones... At the moment (with the delay) they are wrongly ordered (11.1.1 before 11.1.0)... won't it be a problem later?10:20
evrardjpmaybe the time estimation on launchpad should be changed...10:20
evrardjpdetails, details...10:20
odyssey4meevrardjp the ordering will be fixed when we set a date for the milestones that don't currently have one10:21
*** fawadkhaliq has joined #openstack-ansible10:21
odyssey4mewe'll discuss the dates on thu at the meeting - let me quickly add that to the agenda10:22
evrardjpI wasn't planning to add you more work, just concerned about possible future issues (if any) and wanted to learn by the same occasion10:22
*** javeriak has quit IRC10:23
evrardjpanyway gparaskevas: is your git review -s working ?10:23
*** daneyon has quit IRC10:23
gparaskevasevrardjp: no :( it asks for username i fill it in but then says cannot find your gerrit10:24
gparaskevasevrardjp: i have created a username gparask i have added my ssh key wich it asked me for as well the first time i git review on the consle shows the correct repo...something's missing10:25
evrardjpon the gerrit setting, there is "generate password" under "HTTP password"10:26
*** javeriak has joined #openstack-ansible10:26
evrardjpoh ok10:27
gparaskevasyes10:27
*** javeriak has quit IRC10:27
evrardjp(just making sure this is filled in too first)10:27
gparaskevasi havent create a password..10:27
gparaskevasshould i?10:27
gparaskevasi thought that it worked with keys10:27
evrardjpyou should10:28
evrardjpin all cases you should10:28
evrardjpyou'll use ssh but you can also use HTTPS10:28
evrardjpit depends what you prefer10:28
evrardjpbut you still should do it10:28
evrardjpthen try to use your git review with -v (to be verbose)10:31
evrardjpso git review -s -v10:31
evrardjpand make sure everything mentionned here is done: http://docs.openstack.org/infra/manual/developers.html#account-setup10:31
evrardjp(I said you should do it because it never hurts)10:32
gparaskevasok let me check!10:34
gparaskevasok i did the git review succesfully(it didnt promt any messages at lease this time) where can i see any changes on that?10:55
*** pilgrimstack1 has joined #openstack-ansible10:57
*** pilgrimstack has quit IRC10:57
gparaskevasodyssey4me: i dit a git review after i have done all the steps. Commit message also included the solves tag ans the branch name was bug/issueid10:58
odyssey4megparaskevas ok, I don't see the review though?11:03
odyssey4meevrardjp thank you for helping gparaskevas :)11:03
*** pilgrimstack1 has quit IRC11:11
*** pilgrimstack has joined #openstack-ansible11:12
*** javeriak has joined #openstack-ansible11:20
odyssey4megparaskevas so you've done 'git commit' and completed the message, did you then do a 'git review' afterwards to push it up?11:22
*** jaypipes has joined #openstack-ansible11:27
*** javeriak has quit IRC11:28
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment-specs: Add tox generated files to .gitignore  https://review.openstack.org/20844011:29
gparaskevasodyssey4me: i dit a git review yes but i cannot see it either so let me check if the gerrit is correct  on my config11:29
*** javeriak has joined #openstack-ansible11:29
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment-specs: Added spec to upgrade MariaDB to v10  https://review.openstack.org/20370811:31
*** markvoelker has joined #openstack-ansible11:33
*** markvoelker has quit IRC11:38
odyssey4memattt can you trouble you for a quick backport review? https://review.openstack.org/20801211:41
matttodyssey4me: sure looking11:41
*** javeriak has quit IRC11:46
*** javeriak_ has joined #openstack-ansible11:48
gparaskevasodyssey4me: it seems like i am not doing something right. I create account based on the same email for openstack, gerrit and launchpad(review and bugs using the same launchpad account) i assigned the bug to me. I cloned the master repo and i created a new branch for the bug. then i commited with details and tags and then i git reviewed some error regadin my gerrit11:50
gparaskevasodyssey4me: then i vreated a gerrit for my repo with my username then i git review again and then no errors...11:51
gparaskevasodyssey4me: but there is still  no code shown anywhere11:51
odyssey4megparaskevas is the configure gerrit-username the same as your launchpad name?11:51
gparaskevaslet me see11:52
odyssey4mecan you perhaps pastebin the output ?11:52
gparaskevassure11:52
odyssey4methanks mattt11:53
matttnp!11:54
*** markvoelker_ has joined #openstack-ansible11:54
*** javeriak_ has quit IRC11:55
*** javeriak has joined #openstack-ansible11:55
matttreview.openstack.org painfully slow for anyone else?11:56
gparaskevasodyssey4me: http://paste.openstack.org/show/406736/ check my details. my gerrit username is gparask because i couldnt write gparaskevas like on launchpad11:56
gparaskevasmattt: yes painfully slow11:57
odyssey4megparaskevas are you able to log into the gerrit review website? ie review.openstack.org ?11:58
gparaskevasyes i am currently logged in11:58
odyssey4megparaskevas is that with the shorter username?11:59
odyssey4megparaskevas go to https://review.openstack.org/#/settings/ - what does it show for your username?11:59
gparaskevasodyssey4me: gparask11:59
svgHi all12:00
svgmattt: What is now left to do for the ceph review?12:01
odyssey4meok, can you pastebin the output from 'git review -v -s' ?12:01
matttsvg: not much, i think we're good to go12:01
svgWhich means..?12:02
svgI guess this needs two +2 reviews now?12:02
matttsvg: yeah, i'm just reviewing some of the changes that went in over the weekend, then I'll +212:02
svgOn a sidenote, I need to prepare a kilo version with the ceph patch, for internal testing12:03
gparaskevasodyssey4me: http://paste.openstack.org/show/406739/12:03
odyssey4mesvg if this is approved, it can be backported with a simple cherry-pick12:03
svgI'm wondering if i just backport the thing, ot if there might be anything else i need to take into account atm.12:03
odyssey4mesvg for your own internal test you should just be able to do a cherry-pick on top of your own checkout12:04
*** fawadkhaliq has quit IRC12:04
svgBecause me is going two weeks to France by the end of the week, and I'd like to allow $cowroker to not need me :)12:04
odyssey4mebut for now, don't backport until we have the master patch merged12:04
odyssey4meconsidering our delays for 11.1.0 it would seem that this may be able to be brought forward from 11.2.0 to 11.1.0 :)12:05
svgThat would be handy.12:05
svgI'm just trying to get an idea of the time frame here, as I not much time for the internal stuff anymore?12:06
mattti'd love for people to give this a bit of a workout in master before we backport to kilo12:07
matttbut assuming we don't discover any issues it should be a straightforward backport12:07
odyssey4mesvg we'll re-discuss the time-frame for releasing 11.1.0 a little later today when we have as many of the cores online as possible12:07
*** daneyon has joined #openstack-ansible12:07
svgok, thx, then we will know more today.12:07
*** daneyon has quit IRC12:12
*** woodard has joined #openstack-ansible12:13
matttevrardjp: you're about 10 mins too late on your comments :)12:20
matttevrardjp: this review has been in flight for over 3 months, we wanted to get what is there merged now and we can improve where necessary with subsequent patches12:21
evrardjpsorry, hard to follow you guys when in a meeting ;)12:21
evrardjpthat's no problem then12:22
evrardjpthere is always room for improvement, right? :D12:22
matttevrardjp: yes! and smaller changes will now be much easier to get through, so please do help where you're able12:22
evrardjpmy comments was more a way to open the discussion too12:24
odyssey4meevrardjp the comments are valuable, but a lot easier when done in line at the location in question12:26
odyssey4mehave you seen that you can comment on a specific line in a file?12:27
evrardjpI've seen that option, but was wondering what was the impact of doing it... Thought the basic text was a good start12:27
evrardjpwill do it differently next time12:28
odyssey4meand yes, the patch is on its way to merging into master - so once it's merged you can log specific bugs and work on patches which are targeted at certain topics - it's a lot easier when working on something small that targets a specific focal point12:28
*** woodard has quit IRC12:28
odyssey4meevrardjp see for example https://review.openstack.org/#/c/207793/12:28
odyssey4mein the os-neutron-install change there was a comment on a specific line which is seen in context12:29
odyssey4methis makes it easier to discuss in the right context12:29
*** woodard has joined #openstack-ansible12:29
*** woodard has quit IRC12:30
*** woodard has joined #openstack-ansible12:31
*** alextricity has quit IRC12:38
*** alextricity has joined #openstack-ansible12:38
gparaskevasodyssey4me: so is it that my gerrit username is gparask and my launchpad username is gparaskevas? is that the issue?12:39
odyssey4megparaskevas no idea - both my usernames are the same, and mine is longer than yours, so I'm not sure why you made yours shorter?12:43
gparaskevasodyssey4me: i dont know if its a matter of length i just couldnt set gparaskevas as my username on gerrit...12:44
odyssey4mecan you paste both the console output all the way from the line where you actually write the command?12:44
odyssey4mealso add a git review output from your branch12:44
odyssey4mehmm, let me check something12:45
gparaskevashttp://paste.openstack.org/show/406746/12:46
gparaskevassure12:46
odyssey4meif you execute 'git commit -a --amend' then pastebin the commit edit that's shown?12:46
gparaskevashttp://paste.openstack.org/show/406747/12:47
odyssey4meok, there's no change-id line which means that git review isn't working properly12:49
odyssey4menormally a post-commit hook would add a change id to your commit message12:50
gparaskevasmm thats true12:50
gparaskevasso?12:50
gparaskevascan i add it manualy?12:50
openstackgerritMerged stackforge/os-ansible-deployment: Enable Horizon to consume a Keystone v3 API endpoint  https://review.openstack.org/20801212:50
odyssey4meno, it's dynamically generated12:50
*** tlian has joined #openstack-ansible12:50
odyssey4megparaskevas so that means that your git review plugin isn't properly configured for the repo, which is odd12:53
gparaskevasodyssey4me: i did that because i had some problems..:git remote add gerrit ssh://gparask@review.openstack.org:29418/stackforge/os-ansible-deployment.git12:55
gparaskevaslook at what problems i had in the begginging12:55
odyssey4mewhat version of git-review do you have installed? ie: pip freeze | grep review12:56
gparaskevasgit-review-1.24-5.fc22.noarch12:56
gparaskevasgit-review==1.2412:57
gparaskevasi did install it with yum12:57
odyssey4meplease paste the output (including command executed) of git review?12:59
gparaskevasodyssey4me: git review -v -d : http://paste.openstack.org/show/406749/13:00
*** markvoelker_ has quit IRC13:02
odyssey4meright, now that makes sense13:03
odyssey4meit can't connect to gerrit on that port via ssh - do you have a firewall perhaps blocking outgoing high ports?13:03
odyssey4meor unknown ports?13:04
*** markvoelker has joined #openstack-ansible13:04
* odyssey4me you may wish to try setting this up: http://docs.openstack.org/infra/manual/developers.html#accessing-gerrit-over-https13:04
odyssey4megparaskevas ^13:05
gparaskevasodyssey4me: ok13:05
openstackgerritMerged stackforge/os-ansible-deployment: Add Ceph/RBD support to playbooks  https://review.openstack.org/18195713:16
* svg http://i61.tinypic.com/2d1jne0.jpg13:19
svgThank you all for your support! ^^13:19
*** woodard has quit IRC13:21
matttsvg: more like http://www.quickmeme.com/img/4a/4a5c519396c22eef4722df1c81c2d589854e5bfe5982e1824d7d6e6d8a2650fd.jpg13:21
svgI'm not that complicated guy :)13:25
*** woodard has joined #openstack-ansible13:25
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment-specs: Added spec to upgrade MariaDB to v10  https://review.openstack.org/20370813:27
*** bapalm has joined #openstack-ansible13:27
*** TheIntern has joined #openstack-ansible13:30
*** KLevenstein has joined #openstack-ansible13:33
*** richoid has quit IRC13:33
gparaskevasodyssey4me: i tried with https but it still goes over ssh for some reason13:33
odyssey4medid you remove the gerrit remote after configuring the scheme and port?13:34
odyssey4meie git remote rm gerrit13:34
gparaskevasyes :(13:34
odyssey4medo a fresh 'git config --list' and 'git remote -v' and pastebin it?13:35
gparaskevasodyssey4me: http://paste.openstack.org/show/406754/13:37
odyssey4methat looks right, now execute: git review -v -s13:39
-openstackstatus- NOTICE: The Gerrit service on review.openstack.org has been restarted in an attempt to improve performance.13:39
*** sdake has joined #openstack-ansible13:41
gparaskevasodyssey4me: http://paste.openstack.org/show/406757/13:44
odyssey4megparaskevas although I don't think it'll help - try removing the clone and re-cloning it?13:48
gparaskevasod odyssey4me ok13:48
odyssey4megparaskevas also, have you generated the http password? https://review.openstack.org/#/settings/http-password13:49
gparaskevasyes but when i changed from ssh to http it only asked me for usernmae and never asked me from password on the commandline13:50
odyssey4meyeah, that's because it's still trying to use ssh - as long as you've done that then it's fine13:50
*** bapalm_ has joined #openstack-ansible13:51
odyssey4meif re-cloning and trying again doesn't work, try : git config remote.gerrit.url='https://gparask@review.openstack.org/stackforge/os-ansible-deployment.git'13:53
odyssey4methen try to git review -s again13:53
*** sigmavirus24_awa is now known as sigmavirus2413:54
*** spotz_zzz is now known as spotz13:54
*** bapalm has quit IRC13:54
*** daneyon has joined #openstack-ansible13:56
*** javeriak has quit IRC13:57
gparaskevasodyssey4me: i am not running anything as root nor am i useing sudo13:59
*** Mudpuppy has joined #openstack-ansible14:00
*** daneyon has quit IRC14:01
odyssey4megparaskevas that should be fine - I don't use sudo either, nor am I running as root14:01
*** spotz is now known as spotz_zzz14:03
*** spotz_zzz is now known as spotz14:04
*** ccrouch has joined #openstack-ansible14:05
*** ccrouch has left #openstack-ansible14:05
gparaskevasodyssey4me: i am recloning from github14:05
gparaskevasodyssey4me: i am redoing the changes ad i will tell you once more14:06
*** ccrouch has joined #openstack-ansible14:11
*** ccrouch has left #openstack-ansible14:11
*** prad has joined #openstack-ansible14:11
*** prad has quit IRC14:11
*** pilgrimstack has quit IRC14:14
*** pilgrimstack has joined #openstack-ansible14:14
*** sdake has quit IRC14:16
*** sdake has joined #openstack-ansible14:24
evrardjpjust to be sure gparaskevas: you're using the same email for gerrit and for launchpad? Is that the same email address used to sign the (C)CLA?14:25
gparaskevasevrardjp: yes...14:25
gparaskevasi have loged in with the same email address on all services14:25
evrardjpso the (C)CLA has been signed too14:26
evrardjp(I ask this because I got the same message with CCLA not signed)14:26
sigmavirus24evrardjp: ICLA is what gparaskevas signs, CCLA is what his employer signs I think14:26
evrardjpindeed14:26
sigmavirus24(I := Individual, C := Corporate)14:26
evrardjpI thought it was CLA and CCLA14:26
evrardjpbut yes, that's the idea14:27
sigmavirus24different Apache projects use different shorthand14:27
evrardjpI just wanted to make sure <something> was signed14:27
gparaskevasnothing is signed i am sorry14:27
gparaskevaswait14:27
sigmavirus24:D14:27
gparaskevaswhen i created the opensatck account is asked me for what company i work for14:27
gparaskevasthis is all about?14:28
gparaskevasit has to do with the afiliaton?14:28
evrardjpcould you go there? https://review.openstack.org/#/settings/agreements14:29
evrardjpand see if something is "verified" ?14:29
gparaskevasnothing14:30
evrardjpthat's your issue14:30
gparaskevasfinaly14:30
evrardjpI asked if you read the page completely at the start of the questions ;)14:31
evrardjpit's written :D14:31
evrardjpI had the same issue14:31
*** prad has joined #openstack-ansible14:36
gparaskevasevrardjp: first of all thank you! i send an email to my boss to see it through and i will try again after i sign it14:38
openstackgerritSerge van Ginderachter proposed stackforge/os-ansible-deployment: Enable Neutron lbaas-haproxy  https://review.openstack.org/20851914:51
*** javeriak has joined #openstack-ansible14:56
jwitkohey guys, it looks like my  ELK stack is spazzing out.  I have messages being spammed at ridiculous rates about errors that happened days ago....   Anyone ever deal with something like this before?14:56
*** yaya has joined #openstack-ansible14:57
*** javeriak_ has joined #openstack-ansible15:00
*** phalmos has joined #openstack-ansible15:00
openstackgerritSerge van Ginderachter proposed stackforge/os-ansible-deployment: Enable Neutron lbaas-haproxy  https://review.openstack.org/20851915:00
*** javeriak has quit IRC15:01
odyssey4mejwitko sure, thought it depends on what version of everything you're using, of course15:08
jwitkoKilo15:08
jwitkosorry,  11.0.4 osad  Kilo branch15:08
odyssey4mejwitko have you used the ELK stack from rpc-openstack then? because OSAD does not include the ELK stack15:09
jwitkodoh.  yes15:09
odyssey4meso it all depends on how your log shipping is happening - if not via beaver, then logstash doesn't keep up well15:09
jwitkoits using Beaver15:09
odyssey4meif it's only coming in via rsyslog then it's likely that you're having sporadic log shipping failures which cause a backup - that's why we switched to beaver15:10
odyssey4meother than that, you'll have to isolate whether you're getting resubmissions of old events, or whether it's not keeping up with new events15:10
odyssey4methen isolate whether the issue is logstash not keeping up with processing, or whether elasticsearch is not keeping up with storing15:11
javeriak_hey guys, due to some power trips a deployment of mine has gone into a bad state, i cant atm figure out what else to check, all nova services show as Down, indivdually they are in the running states, i tried restarting and no luck, also haproxy stats show all backends as UP, any pointers on what else I could check15:11
jwitkolooks like iv’e got some work ahead of me15:11
jwitkothanks odyssey4me15:11
*** alextricity_r has joined #openstack-ansible15:15
javeriak_I do see alot of ECONNREFUSED, oslo.messaging._drivers.impl_rabbit AMQP server on 172.20.190.154:5672 is unreachable in nova logs15:15
sigmavirus24javeriak_: what version?15:15
alextricity_rMorning! Does anyone know where the OSAD docs are?15:15
sigmavirus24javeriak_: is rabbit even up?15:15
alextricity_rI remember viewing them in the past, but I lost the link..15:15
sigmavirus24alextricity_r: I believe osad.readthedocs.org15:15
evrardjpjaveriak_: check everywhere your rabbit status15:15
mgariepyalextricity_r, http://osad.readthedocs.org/en/latest/15:16
alextricity_rsigmavirus24: mgariepy Thanks :)15:16
odyssey4mealextricity_r dotted over a few places - that's one place, the rest are in example configs and the main defaults files in each role15:16
sigmavirus24javeriak_: also I think I remember hearing that if you restart rabbit you have to then restart nova because of a channels issue in oslo.message15:16
sigmavirus24*messaging15:16
odyssey4mejaveriak_ sounds ot me like your rabbitmq may be down15:16
evrardjpalextricity_r: make html on your git clone in the docs section15:16
evrardjpfar better :)15:17
sigmavirus24I don't think the issue was that rabbit was unreachable though15:17
evrardjpless friendly but always available15:17
evrardjpalextricity_r: check not only if a process is running, but also how it behaves15:18
sigmavirus24evrardjp: I believe you meant that for javeriak_ ?15:19
javeriak_what container does rabbit live in guys?15:19
javeriak_i havnt been able to find the service yet15:20
evrardjpsigmavirus24: yes Indeed15:20
javeriak_sigmavirus24: I'm on 10.1.1015:20
evrardjpend of the day, I overflowed :p15:20
sigmavirus24evrardjp: infra1_rabbit_mq_container for example15:22
sigmavirus24evrardjp: understandable15:22
evrardjpsigmavirus24:  ;)15:22
javeriak_also btw is there a way to restart services from within the main infra nodes, as in not have to go into respective containers, something like all the symlinked logs you can get under /openstack/logs15:23
javeriak_sigmavirus24: ofcourse :P, i should have seen that15:23
evrardjpit shouldn't happen, openstack doesn't need a centralized reboot or any reboot at all ;)15:24
sigmavirus24javeriak_: most of the services should start when the container comes up so you could do `lxc-stop -n container; lxc-start -d -n container`15:25
evrardjpjaveriak_: to test the rabbitmq, I usually start with checking peers (rabbitmqctl cluster_status) then checking the policies (rabbitmqctl list_policies)15:25
sigmavirus24the -d is import so it goes into daemon mode15:25
javeriak_sigmavirus24: yes im familiar with the lxc commands, so rabbitmq does indeed report that its not running15:26
odyssey4mejaveriak_ use ansible, eg: ansible <group name> -m shell -a '<command to run>'15:27
sigmavirus24odyssey4me: that's the easy way15:27
sigmavirus24psh15:27
odyssey4mejaveriak_ a specific example: ansible nova_api_os_compute -m shell -a 'service nova-api-os-compute restart'15:29
odyssey4me(run from your playbooks directory)15:29
javeriak_odyssey4me: ah right thats much easier , thanks15:31
odyssey4mesigmavirus24 palendae I think we have a ordering issue in the upgrade script - either that or one of the patches isn't right15:31
odyssey4methe /tmp/fix_juno_log_rotate play fails because it's trying to act on the cinder_scheduler, nova_console, repo and rsyslog containers... none of which exist until the lxc-containers-create play runs15:34
odyssey4meor maybe this one should also be || true15:34
javeriak_so rabbitmq failed to start, getting unhelpful crash reports...15:35
sigmavirus24odyssey4me: palendae is on voluncation15:35
odyssey4meoh dear15:36
sigmavirus24odyssey4me: i was seeing the same thing but my may concern was that we had fixed the syntax problems in run-upgrade.sh15:36
odyssey4methx sigmavirus2415:36
*** gparaskevas has quit IRC15:37
*** phalmos has quit IRC15:40
odyssey4mesigmavirus24 mattt d34dh0r53 unfortunately we're the only cores available right now for voting, so instead of trying to release 11.1.0 today I suggest that we do so on Thu after the community meeting15:44
*** daneyon has joined #openstack-ansible15:45
matttodyssey4me: sounds fine to me15:46
evrardjpmattt: I'm not sure to have followed the ceph commits right now, will it be merged in the 11.1.0 ?15:47
evrardjpor do you wish more testing on master?15:47
matttevrardjp: my personal opinion is for it to go into 11.215:48
evrardjpthere is no expected date for 11.2 :)15:49
*** daneyon has quit IRC15:50
evrardjpI'll try my chances with master then15:50
evrardjpor cherry picking15:50
evrardjpthanks for the info15:50
matttevrardjp: yeah cherry-picking should work15:50
matttevrardjp: it may go into 11.1, but like i say i'd rather see more people bash it a bit before we backport to kilo15:50
*** daneyon has joined #openstack-ansible15:51
*** daneyon has quit IRC15:51
*** daneyon has joined #openstack-ansible15:51
odyssey4meevrardjp cherry pick to the kilo branch is the best strategy15:53
evrardjpI think too15:53
odyssey4methe master branch uses the upstream master branches which are massively in flux, wherea the kilo branch tracks the stable/kilo branches upstream15:53
evrardjpthis also helps give feedback for future merges15:53
*** openstackgerrit_ has joined #openstack-ansible15:54
evrardjpodyssey4me: that's why I was afraid :p15:54
*** daneyon_ has joined #openstack-ansible15:54
*** daneyon has quit IRC15:56
odyssey4mesigmavirus24 perhaps it's best to rebase https://review.openstack.org/207936 on top of https://review.openstack.org/207687 ?15:58
sigmavirus24odyssey4me: ah yeah, it'll cause a merge conflict but i'll fix it up as soon as the other merges15:59
sigmavirus24it's a stupid merge conflict too because git is bad at merging things sometimes15:59
odyssey4meyeah16:00
*** ccrouch has joined #openstack-ansible16:04
*** ccrouch has left #openstack-ansible16:04
*** phalmos has joined #openstack-ansible16:04
odyssey4mesigmavirus24 odd, I can't seem to find where '/tmp/fix_juno_log_rotate.yml' is being created?16:06
sigmavirus24odyssey4me: it's inline in run-upgrade.sh I thought16:06
sigmavirus24odyssey4me: https://github.com/stackforge/os-ansible-deployment/blob/kilo/scripts/run-upgrade.sh#L57916:07
*** galstrom_zzz is now known as galstrom16:07
odyssey4mesigmavirus24 ah, lovely - it's in kilo and not master :/16:08
sigmavirus24odyssey4me: "fix juno"16:09
sigmavirus24only needs to be in kilo16:09
odyssey4mesigmavirus24 yeah, although I'm concerned at our divergence without a proper review of what needs to be where16:10
sigmavirus24odyssey4me: the problem is for upgrades from juno to kilo, not juno to liberty (since we're not supporting upgrades from juno to liberty)16:10
odyssey4mesigmavirus24 agreed, but we're fixing several upgrade things directly in the kilo branch which are also useful for future upgrades16:11
odyssey4menever mind though, for liberty I think we need to revamp this whole method16:11
odyssey4methe monolithic method must die16:12
d34dh0r53odyssey4me: I'm out of the office starting Thu16:12
openstackgerritMerged stackforge/os-ansible-deployment: Properly parse and quote arguments in upgrade script  https://review.openstack.org/20768716:12
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Properly parse and quote arguments in upgrade script  https://review.openstack.org/20854616:13
odyssey4med34dh0r53 sigmavirus24 ^ forward port to master from kilo16:13
*** javeriak_ has quit IRC16:13
sigmavirus24I'll rebase the other one now16:14
*** sdake has quit IRC16:16
openstackgerritIan Cordasco proposed stackforge/os-ansible-deployment: Use playbook status to report upgrade failure  https://review.openstack.org/20793616:16
sigmavirus24odyssey4me: rebased ^16:16
*** fawadkhaliq has joined #openstack-ansible16:16
odyssey4med34dh0r53 ^^16:17
d34dh0r53looks good16:17
*** sdake has joined #openstack-ansible16:21
*** javeriak has joined #openstack-ansible16:22
*** javeriak_ has joined #openstack-ansible16:26
*** javeriak has quit IRC16:31
*** phalmos has quit IRC16:33
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Force the juno log rotate fix task to always succeed  https://review.openstack.org/20855716:33
*** alop has joined #openstack-ansible16:33
*** phalmos has joined #openstack-ansible16:35
odyssey4mesigmavirus24 ^ that should do the trick to fix the issue I found... I'm rebuilding an AIO to verify.16:37
*** TheIntern has quit IRC16:37
sigmavirus24I have one up so I'll test that16:39
*** richoid has joined #openstack-ansible16:40
*** sdake has quit IRC16:43
*** sdake has joined #openstack-ansible16:47
*** harlowja has joined #openstack-ansible17:03
*** phalmos has quit IRC17:05
*** yaya_ has joined #openstack-ansible17:09
*** yaya has quit IRC17:09
*** yaya_ is now known as yaya17:09
sigmavirus24odyssey4me: so far so good on the upgrade with that change17:21
odyssey4mesigmavirus24 excellent :)17:36
sigmavirus24well I mean17:42
*** TheIntern has joined #openstack-ansible17:42
sigmavirus24galera is still restarting17:42
sigmavirus24but17:42
odyssey4mesigmavirus24 did you set galera_ignore_cluster_state to true in user_variables ?17:43
sigmavirus24damnit17:43
sigmavirus24I knew I forgot something17:43
odyssey4meif not, then your upgrade will fail17:44
sigmavirus24yep17:44
openstackgerritTom Cameron proposed stackforge/os-ansible-deployment: Add new roles to create a router container and custom routes  https://review.openstack.org/20368317:44
sigmavirus24it just failed as you said that17:44
odyssey4mesigmavirus24 ok, I have an upgrade test running now17:47
sigmavirus24I'm rebuilding Juno AIOs as we speak to test out the ugprades17:48
odyssey4mehmm, I forgot to lower the number of forks in ansible.cfg17:48
odyssey4mewithout doing that often results in ssh connection failures :/17:49
odyssey4methere we go, the upgrade script has passed the stage where it failed previously :)17:50
*** javeriak_ has quit IRC17:57
openstackgerritMerged stackforge/os-ansible-deployment: Change ansible forks used  https://review.openstack.org/20747417:59
*** javeriak has joined #openstack-ansible18:01
odyssey4med34dh0r53 https://review.openstack.org/20855718:02
*** javeriak has quit IRC18:05
*** javeriak has joined #openstack-ansible18:05
*** openstackgerrit_ has quit IRC18:07
d34dh0r53odyssey4me: sigmavirus24 should I merge this, or wait until sigmavirus24's test is doine?18:11
d34dh0r53looks like it should get past that point to me now18:12
sigmavirus24d34dh0r53: up to you,18:12
sigmavirus24It'll take a while for these AIOs to build before I can even do a full upgrade run18:12
d34dh0r53k18:12
*** phalmos has joined #openstack-ansible18:15
*** javeriak has quit IRC18:16
*** harlowja has quit IRC18:21
*** harlowja has joined #openstack-ansible18:21
*** yaya has quit IRC18:29
*** javeriak has joined #openstack-ansible18:29
*** fawadkhaliq has quit IRC18:31
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Move ansible logging to gate-check script only  https://review.openstack.org/20750218:34
odyssey4mesigmavirus24 meh, now it failed on the setup-infrastructure play during the galera install :/18:36
openstackgerritMerged stackforge/os-ansible-deployment: Use playbook status to report upgrade failure  https://review.openstack.org/20793618:40
*** javeriak has quit IRC18:41
sigmavirus24odyssey4me: I'll cherrypick that forward18:42
*** TheIntern has quit IRC18:44
openstackgerritIan Cordasco proposed stackforge/os-ansible-deployment: Use playbook status to report upgrade failure  https://review.openstack.org/20860718:46
*** TheIntern has joined #openstack-ansible18:46
odyssey4mesigmavirus24 fyi a simple re-run of that playbook worked perfectly18:46
sigmavirus24odyssey4me: d34dh0r53 ^ forward port18:46
openstackgerritJesse Pretorius proposed stackforge/os-ansible-deployment: Move ansible logging to gate-check script only  https://review.openstack.org/20750219:22
*** sdake has quit IRC19:28
odyssey4mesigmavirus24 happy to +w this one? https://review.openstack.org/20855719:38
odyssey4meI think that'll be the last of the required patches for a decent QE test run19:39
sigmavirus24yeah my last AIO failed somewhere unrelated19:39
*** galstrom is now known as galstrom_zzz20:00
*** sdake has joined #openstack-ansible20:01
*** galstrom_zzz is now known as galstrom20:01
*** alextricity_r has quit IRC20:03
*** sdake_ has joined #openstack-ansible20:10
*** sdake has quit IRC20:13
*** yaya has joined #openstack-ansible20:35
openstackgerritMerged stackforge/os-ansible-deployment: Force the juno log rotate fix task to always succeed  https://review.openstack.org/20855720:44
*** bapalm_ has quit IRC21:00
*** woodard has quit IRC21:08
*** yaya has quit IRC21:12
*** daneyon_ has quit IRC21:14
*** daneyon has joined #openstack-ansible21:15
*** pilgrimstack has quit IRC21:16
*** pilgrimstack1 has joined #openstack-ansible21:16
*** galstrom is now known as galstrom_zzz21:21
*** galstrom_zzz is now known as galstrom21:35
*** TheIntern has quit IRC21:36
*** yaya has joined #openstack-ansible21:39
*** yaya has quit IRC21:50
Sam-I-Ammoo22:03
*** Mudpuppy has quit IRC22:06
*** yaya has joined #openstack-ansible22:09
*** KLevenstein has quit IRC22:18
bgmccollumodyssey4me, gparaskevas: git review using HTTP passwords requires git-review 1.25 or newer22:23
*** spotz is now known as spotz_zzz22:28
sigmavirus24bgmccollum: that's a good fun fact22:32
sigmavirus24I'll remember it :D22:32
*** JRobinson__ has joined #openstack-ansible22:34
*** galstrom is now known as galstrom_zzz22:37
*** yaya has quit IRC22:38
*** yaya has joined #openstack-ansible23:14
*** darrenc has quit IRC23:23
*** JRobinson__ has quit IRC23:23
*** darrenc has joined #openstack-ansible23:23
*** sigmavirus24 is now known as sigmavirus24_awa23:24
*** darrenc has quit IRC23:35
*** darrenc has joined #openstack-ansible23:35
*** phalmos has quit IRC23:36
*** darrenc is now known as darrenc_afk23:45

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!