Tuesday, 2014-09-16

fungilifeless: no, my client was smart enough to split my comment onto a second line00:00
anteayabecause I haven't figured out how to deal with that yet - with certainty00:00
dimslifeless: did you invoke me? :)00:00
anteayahey dims00:00
dimshi anteaya00:00
jrollanteaya: is there something wrong with "show me your contribution and I'll give you a ballot" for those?00:00
anteayayou haven't been invoked in a while00:00
lifelessdims: i did not00:00
anteayajroll: yes, that was the old system00:00
lifelessdims: lazy copy-pate :)00:00
*** emagana has quit IRC00:01
fungijroll: that's what i'm proposing we move to for all voters next cycle00:01
dimslifeless:  hehe00:01
anteayajroll: now since they can resign their membership I now have to validate they were a member during specific dates00:01
jrollanteaya: ugh :/00:01
anteayaand while I might be able to validate they are there if they are, I don't know how to validate they are not there if they are not00:01
anteayajroll: now you get it00:01
anteayawhereas I can when it was just gerrit00:02
anteayawhen on and on with some person who insisted they could vote last round00:02
anteayatold them 5 different time they were ineligible, kept coming back insisting they were00:02
fungianteaya: to be fair, the queries we used to gess this from gerrit were also not 100% accurate since they depended on the change owner hopefully being the commit author, which reasonably often still isn't the case00:02
anteayaI *could* prove they were not last time, I don't know how to this time00:02
*** unicell has quit IRC00:03
anteayafungi: I put it on them to provide me a sha00:03
mikalSo... I think I cans cript the foundation membership check as long as the check is by name not email address00:03
*** mbacchi has quit IRC00:03
anteayaand the shas provided were for abandoned patches00:03
mikalhttp://www.openstack.org/community/members/results?mq=michael+still&locale=en_US&action_getResults=Go 302 redirects to a profile if the person is a member00:03
*** melwitt has quit IRC00:03
anteayamikal: if there is only one person with that name, that might work00:04
mikalDo we have an example of a resigned member I could check?00:04
jrollmikal: then you'll run into mike vs michael etc, I bet00:04
anteayajroll: you guessed it00:04
*** hdd_ has quit IRC00:04
mikalNot much I can do about that00:04
anteayamikal: I have no idea, foundation doesn't give me that information00:04
* jroll is never sure if he is registered as jim or james in certain places00:04
fungipeople named michael who spell their name mikal00:04
jrollhehe00:04
anteayajroll: yup00:04
mordredjesusaurus: new lesson - when you pick up patches that I wrote, you USUALLY have to rewrite them00:04
*** dannywilson has joined #openstack-infra00:05
anteayaor people who can't be bothered to get a password reset link and just make up a new profile00:05
jrollfungi: then there's my friend micheal (yes e before a), where he is the only person that can spell his name00:05
fungipretty awesome00:05
clarkbfungi: should we worry about this new ossn at all?00:05
clarkbthey are just test nodes getting floating ips so maybe not00:05
jesusaurusmordred: very true, but this was a simple one00:06
mordredjesusaurus: the simplest ones are the ones that kill yo00:06
mikalSigh00:07
mikalI am starting to feel like this is all too hard00:07
*** boden has joined #openstack-infra00:07
anteayamikal: yup00:07
jrollmikal: everything is hard, let's go drink beer00:07
*** signed8bit has quit IRC00:07
anteayamikal: so the other thing you can do, is email all your nova folks and tell them if they want to be extra atcs for nova they need to send you an email with a link to their foundation profile00:08
anteayaand based on dhellmann's comment on your patch hes if find the edge cases in scripting the scraping of the foundation website too00:09
fungii'm to the point where i think if we're going to build automated systems to determine whether someone's an eligible voter for a program, we should provide an opt-in interface where they can register to vote (which then checks their individual eligibility for a program election on the backend)00:09
jrollmikal: where's that extra atc patch?00:09
anteayait isn't the majority I am worried about, it is the edge cases, the ones the scripts miss00:10
clarkbfungi: agreed and there is lots of data out there saying opt in vote by mail(remotely systems of some sort) work really well00:10
jrollmikal: I'm willing to bet most of them are easy finds, and there's only a couple to verify00:10
jroll(most are ironic contributors already)00:10
anteayafungi clarkb that doesn't help us this round, but do offer it after the eleciton season since it will take all of a release at least to square with the charter00:10
fungii'm a little worried about tying technical voting to anything in the foundation member database right now, since we are an open project and currently there is nothing open about the system managing and tracking that data00:11
anteayafungi: me too00:11
anteayabut it is in the charter00:11
jrollmikal: gah, maybe not, that's a lot of extra ATCs00:11
anteayaand I can't change it 2 weeks before elections start00:11
*** boden has quit IRC00:12
openstackgerritA change was merged to openstack-infra/config: Switch infra to new puppet apply job  https://review.openstack.org/12167500:12
fungithough i'm probably just going to grip tightly to my grass skirt and hope we can all safely hula through00:12
anteayafungi: go you00:13
anteayapluto goes direct on the 23rd so one more week to go00:13
jogohere are some raw recheck numbers from nova: http://paste.openstack.org/show/111954/00:14
anteayaand jogo the squirrel is here to distract us with data00:14
anteayathanks jogo :D00:14
fungianteaya: mikal: though perhaps one option is for the tc to pass a resolution stating that for this election it's going to ignore the possibility that contributors may have renounced their foundation citizenship, for the purposes of determining eligible voters?00:15
anteayacan we get lines of code in there somewhere?00:15
anteayafungi: do propose, the tc agenda has been drawn up tomorrow and we only have tomorrow's meeting and one more before election season kicks off00:16
anteayawe can't change the rules for the roll after sept 2600:16
anteayafungi: good luck00:16
*** bknudson has joined #openstack-infra00:16
jogoanteaya: anteaya this is number of rechecks I don't *think* LOC is relevant for this00:16
fungimight carry more weight if proposed by an actual tc member (which i am not). i'm willing to just try and make it work00:17
anteayajogo: okay00:17
anteayafungi: well you have mikal and perhaps mordred to convince00:17
*** bdpayne has quit IRC00:17
anteayabut I am disappointed that election buisness is not higher on tomorrow's tc agenda00:17
jogoanteaya: which is to say not sure how to get LOC yet00:17
anteayabut I hadn't mentioned to ttx raising its place in the agenda, so he probably isn't aware of how I feel00:18
anteayajogo: ah00:18
fungioh, there are plenty of tc members sleeping in this channel, but anyway i plan to just try to power through with what we have and come up with better solutions for next cycle as soon as possible once this one is behind us00:18
anteayajogo: thanks for the honesty, it is refreshing00:18
anteayaand I am still tying my boat to being in compliance with the charter00:19
jogoanteaya: np :)00:19
anteayathough I do have a drowning feeling through doing so00:19
anteayajogo: can you sort by recheck number?00:20
jogoanteaya: yeah, right now I am trying to figure out how to best slice the data00:20
jogoit is sorted by merge time00:20
anteayajogo: k, slice away00:20
jogomost recent patches at the top00:20
anteayaah, okay00:21
fungiclarkb: that ossn has no real implications for us. it's more of a "zomg i removed the floating ip for my sooper-sensitive server and people who were connected through it stayed connected"00:21
clarkbjogo: I think you can get the diff stats fro mgerrit api, but I may be mistaken. worst case you fetch change and diff it directly00:21
jogoanteaya: I am thinking (#of rechecks)/(# of revisions)00:21
arosenfungi:  clarkb do you guys know if nodepool has anything in it to figure out / avoid bad image snapshots?00:21
clarkbfungi: well it doesn't seem to make a statement as too what actually happens just that a thing happens00:21
anteayaI can't wait to see how that looks00:21
jogoclarkb: yeah I am sure there is a way, just haven't dug in yet00:21
clarkbfungi: if the image build process fails (returns 1 or api 500s etc) then it ditched the image00:22
clarkber arosen ^00:22
arosenFrom time to time it seems like we have an infra problem where the snapshot that gets recreated results in a grub error. Though it doesn't always happen.00:22
clarkbarosen: so you acn add checks in that do that00:22
arosenclarkb: unfortunately  there doesn't seem to be an api error :(00:22
fungiarosen: it has bash errorexit and some attempts at interpreting puppet detailed exit codes00:22
jogoI am a little surprised by how many revisions a patch has00:22
arosenit seems like everything works but when the image is used the vm is just stuck hanging on grub with image not found.00:22
*** JayJ____ has joined #openstack-infra00:23
clarkbarosen: I would try removing our fix grub script from your builds to see if that fixes you00:23
clarkbarosen: since we do mess with grub in the images00:23
clarkbthen we can work from there if that does indeed fix it00:23
arosenclarkb:  can you point me to where i'd need to do that?00:23
clarkbarosen: yup one sec00:23
arosenclarkb:  the weird thing is it seems to work 60% of the time.00:23
fungiarosen: there's been some discussion around having an acceptance test option in nodepool for images (we had one for devstack-gate back when it did the image management itself), but support has not been implemented00:24
aroseni haven't tried to debug this from the underlying infrastructure yet though.00:24
clarkbarosen: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/restrict_memory.sh is the script that does it. just comment out or delete calls to it in the other scripts00:25
*** SumitNaiksatam has quit IRC00:25
clarkbor comment out the script contents I suppose00:25
fungialso worth noting, back when devstack-gate ran devstack as an image acceptance test, it was not the most reliable (fraught with false negatives because of openstack bugs encountered by devstack)00:25
arosenclarkb:  as in line 29-31?00:25
aroseni mean 21-3100:25
clarkbarosen: and the stuff above it00:25
anteayaI am going to try being not on the computer for a while00:25
clarkbarosen: both sets. they handle grub 1 and grub 200:26
arosenk00:26
arosendone00:26
*** terryw has quit IRC00:30
*** rmcall has quit IRC00:30
mikalanteaya / fungi: so...00:31
mikalanteaya: what if I used git commits to incubated / graduated repos as a test of foundation membership?00:31
mikalanteaya: its as flawed as the electrol role system now00:31
fungimikal: you'd want to compare them to the change owner which i think gerrit reflects in the git notes00:32
*** pballand has quit IRC00:32
jrollthat seems pretty wrong00:32
jroll(the incubated/graduated check)00:32
mikalfungi: well, when jenkins merges you get the original author in the merged commit, right?00:32
fungihuh, actually change owner is not in the git notes for a change00:33
fungimikal: in the git commit metadata you get information on what the "author" of te last patchset was (which probably hasn't been changed since the first patchset but isn't actually required to be a gerrit account and therefore not required to be a foundation member) and also the committer for the most recent patchset (who is guaranteed to be a gerrit account/foundation member but not guaranteed to be00:34
fungithe originator/owner of the change)00:34
fungiso if you can find the e-mail address in committer fields for recent changes then it should be a foundation member00:35
fungias long as that's all you're trying to match up00:35
*** salv-orlando has quit IRC00:36
*** bhuvan has quit IRC00:37
*** JayJ____ has quit IRC00:37
mikalfungi: I get the email address in the merged commit too. Or is that what yuo're talking about?00:39
jogoI take it someone already noticed zuul is reporting Queue lengths: 862 events, 48 results.00:40
jogofungi: ^00:40
fungimikal: if you look at git log --pretty=fuller you'll see there are two people associated with a commit (often the same, but not guaranteed to be)00:40
fungijogo: merge-check00:41
mikalSigh00:41
jogofungi: ? its empty00:41
mikalI am so close on rage quitting on this00:41
fungimikal: yeah, the voter eligibility requirements are sort of abstract and not grounded in the reality of the data we have00:42
*** psedlak has quit IRC00:43
*** dane_leblanc has quit IRC00:45
fungimikal: we had more or less worked out a reasonable compromise based on scraping data from sources we had and letting people who felt they were omitted letting us know so we could add them, but now we've introduced a distinct risk that we'll authorize ineligible voters (because foundation memberships are finally expiring)00:45
*** rhsu1 has quit IRC00:47
*** JayJ____ has joined #openstack-infra00:51
*** StevenK_ is now known as StevenK00:51
*** dannywilson has quit IRC00:52
*** SumitNaiksatam has joined #openstack-infra00:52
*** dannywilson has joined #openstack-infra00:52
openstackgerritAaron Rosen proposed a change to openstack-infra/nodepool: Fix attribute error state if node not found  https://review.openstack.org/12172100:55
openstackgerritNikhil Manchanda proposed a change to openstack-infra/config: Create a trove-specs repository  https://review.openstack.org/12145700:56
*** dannywilson has quit IRC00:57
mordredfungi: can I propose something?00:59
lifelessmordred: can you?01:00
mordredlifeless: can I?01:00
mordredfungi: since foundation membership has no qualification, rather than requring membership for commits and votes - what if commiting were one of the methods you could sign up for membership01:00
mordredfungi: it's no more or less insane than a web form randomly01:01
mordredso any subsequent commit would re-up you01:01
mordredso since you have to be an ATC to vote, we know that everyone has commited in the last year01:01
mordredipso facto - it'snot possible to have inelligible voters in an ATC-base election01:02
fungimordred: sounds sane to me, though i suppose it could be abused by someone committing under lots of different names, the same could hold true now with additional member profiles01:02
mordredwell, for our elections we're condorcet - so ballot stuffing is way harder01:02
lifelesssybil for ever!01:03
mordredI guess in a single project PTL ...01:03
mikalfungi: well, commits are already hand reviewed01:03
fungithat too. and board elections don't care about atc status anyway01:03
mordredfungi: yah. I'm only talkin gabout ours01:03
mikalSurely we'd notice someone branch stacking?01:03
mordredthe "I signed up on the website and ignore it" folks should have the same old procedure01:03
mordredmikal: also, can you imagine building cred in nova for 10 different people from scratch?01:04
fungiif the tc has the power to decide that without going before the board and modifying foundation documents, i recommend just making a proclamation01:04
mikalmordred: omg, I can't even do it for one...01:04
fungii'd be willing to back any assertion that because the foundation member system is not open it's inscrutable and unfit for being considered as technical election criteria01:06
fungiand that we'll just make do with what we have (git commit data)01:06
*** rhsu has joined #openstack-infra01:07
mordredfungi: ++01:07
*** boden has joined #openstack-infra01:08
*** harlowja_at_home has joined #openstack-infra01:08
*** tomoe has joined #openstack-infra01:08
fungibasically that's what we did up until now, the current crisis (if it is one) is around people who would have qualified as atcs ceasing to be foundation members because they didn't vote in board elections01:09
mikalDo we require them to be foundation members to commit code?01:09
mikalShould we be revoking CLAs?01:09
fungiand voting in a board election to remain eligible to vote in tc and ptl elections is sort of a silly requirement01:09
*** Longgeek has joined #openstack-infra01:10
fungimikal: technical we have no real reason to require foundation membership to commit code under the cla, but we have inadvertently built a system that enforces that they are at the time they first sign the cla01:10
fungiwhich many of us would like to fix01:10
mordredmikal: we could also all rage quite and move to an island01:11
fungithe main problems being that severing that tie makes it harder to confirm they're foundation members for technical election purposes and for free summit passes01:11
mikalmordred: please send me a brochure for said island01:12
fungii think the summit pass problem is more tractable that the voter eligibility one01:12
fungier, than01:12
mordredfungi: I believe they're both solved01:12
mikalGiven we treat summit discount codes like party favours anyway01:12
mordredan ATC must start off as a Foundation Member01:12
*** boden has quit IRC01:12
mordredan inactive foundation member must only "express interest" in having their status continue01:13
mordreda commit is an active interest01:13
fungimikal: you could move to my island. i'm sure the local chamber of commerce can furnish me with plenty of brochures to forward along01:13
mordredtherefore, any commit comes from a person asserting that they are a foundation member by definition making them an active foundation member01:13
mikalfungi: is your island in America?01:13
fungimikal: technically, yes. bummer01:13
mordredthis means ATC and summit pass and voting can all be solved purely from gerrit01:14
*** yaguang has joined #openstack-infra01:14
mikalSo how does the openid thing play into this?01:14
mikalIts the long term proposed solution, right?01:14
mordrednot really01:15
mordredit doesn't solve a ton01:15
mordredother than != launchpad01:15
fungimikal: the long-term idea is that you have an openstack systems account which gerrit authenticates against and which the foundation member database also links to01:15
mikalOk01:15
mikalI thought it was the longer term membership db thing as well01:15
*** amcrn has quit IRC01:16
fungiso it ideally provides a unique mapping between gettit accounts and foundation membership, though in actuality it doesn't prevent the multiple accounts problem if they abandon and sign up again01:16
fungis/gettit/gerrit/01:16
*** psedlak has joined #openstack-infra01:17
fungibut that way everyone who uses our openid-authenticated systems has an openstack systems account, which may or may not have one and only one foundation membership associated with it, which may be active or inactive at any point in time01:17
*** Vega has joined #openstack-infra01:17
*** Vega is now known as Guest3017601:18
*** Guest30176 is now known as LuckyVega01:18
tchayposo my gertty seems to be sad, it has a whole bunch of changes where it has no owner (and comments with no author and so on)01:18
tchaypois there anyway I can prod it into re-downloading data about those changes, or do i need to blow away my db and start again?01:19
fungibasically it arises by making the foundation membership system also authenticate against the same registry as the other systems. so a strong coupling as opposed to the weak correlation we try to assemble presently01:19
fungitchaypo: did it get into that situation following an upgrade, or just spontaneously?01:19
*** dimtruck is now known as zz_dimtruck01:20
tchaypounknown. There were certainly upgrades before I noticed it, but I don't know how long it's been in that state01:20
tchaypoI think it's been sitting there for a while - I've been avoiding looking at the particular change that i see most often, because looking at it crashes gertty01:22
openstackgerritA change was merged to openstack-infra/config: Remove test.sh  https://review.openstack.org/12167601:25
*** ramashri has quit IRC01:25
fungitchaypo: which change and are you running the most recent point release? i think there was a fix for crashing on certain non-ascii data in fields which shouldn't have had non-ascii data in them but might anyway because internets01:25
*** marcoemorais has quit IRC01:26
*** unicell has joined #openstack-infra01:26
fungiyeah, https://review.openstack.org/119410 merged after the 1.0.0 release01:27
*** gyee has quit IRC01:27
fungiso if you didn't upgrade to 1.0.1 or later that's a likely cause of change-specific crashes01:27
mikalfungi: what would you think about extra-atcs output like this? http://paste.openstack.org/show/111982/01:28
*** miqui has quit IRC01:29
tchaypoFwiw I've been running from trunk rather than a released version01:29
tchaypoAnd I've been toying with various patches applied on top01:29
mikalanteaya: ^--- (but I suspect you've wandered off)01:29
fungimikal: right now the script parses that file line-by-line, so while that might work, some sort of standardized structure like yaml would still be better01:29
fungiit already has to grok yaml since it uses the programs.yaml file to figure out which projects belong to what programs01:30
mikalI agree01:30
mikalI felt like tweaking the file format was a step too far though01:30
mikalGiven the level of pain this relatively simple thign has already caused01:30
mikalWhat I am trying to provide is an audit log of how I decided they were an "eligible voter"01:31
fungimikal: well, if there were some declaration that we could ignore the foundation member database for technical elections then i would just build a database of related e-mail addresses based on what people had as primary/alternate addresses in their gerrit accounts and then just scrape author fields and co-authored-by headers out of commit messages01:32
mikalWhile I like that plan...01:33
fungi(build an ephemeral database as an in-memory dict that is)01:33
mikalRegardless, the curent status quo is I can use a commit in the past as a measure of if they've _ever_ been a foundation member, yes?01:33
*** Longgeek has quit IRC01:33
fungimikal: i think so, yes. if there are any corner cases, they'll be rare01:34
*** Longgeek has joined #openstack-infra01:34
mikalAnd pre-existing01:34
mikali.e. the current voter process already suffers from those01:34
fungithinks like committers on imported project repositories where the project managers should have confirmed icla had been signed by all previous committers but missed one01:35
fungiand yeah, possibly would also have gotten picked up in the current implementation anyway01:36
fungiso modulo changing the actual file structure substantially, what you have there should be fine and i can tweak the current script to do something with that01:36
*** melwitt has joined #openstack-infra01:37
mikalOk, I have gone and hand checked the foundation profile id for each of the ones needing hand verification as well01:40
*** armax has left #openstack-infra01:41
fungimikal: thanks, that'll be a big help. hopefully that satisfies anteaya's desire to be able to cross-check them in the event of a dispute01:41
*** pcrews has quit IRC01:42
fungimikal: the point i was trying to make before, which i hopefully was clear enough on, is that if this were *simply* a matter of scraping git commit messages to populate co-author lists then i'd just make the current script do that for us and be done with it01:44
mikalfungi: so, my secret agenda is that I've been asking for something like this for a year01:45
mikalSo if I have to flail to get my authors recognized by the system, then so be it01:45
mikalhttps://review.openstack.org/#/c/119666 has the new version now01:45
fungithanks01:45
fungiand yeah, perhaps it's something we can strive to get clarified in the appropriate documents such that we can eventually do just that01:46
*** xchu has joined #openstack-infra01:46
mordredfungi: or pictures of cats01:46
fungimordred: yes, with funny captions. the very underpinnings of the internet itself01:47
*** xchu has quit IRC01:48
tchaypoSo I'm assuming it's probably not something worth chasing as a bug in Gertty, it's probably caused (or at least exacerbated by) the unusual way I'm using it01:49
tchaypoI think I should just go ahead and nuke the db and let it pull everything down again01:49
fungitchaypo: are you running 1.0.2?01:49
fungioh, you said trunk01:50
fungii totally missed your later comments01:50
tchaypo:)01:50
*** xchu has joined #openstack-infra01:51
fungitrunk last sync'd how recently?01:51
*** xchu has quit IRC01:51
*** jpich_ has joined #openstack-infra01:54
*** dane_leblanc has joined #openstack-infra01:54
*** harlowja_at_home has quit IRC01:57
*** jpich has quit IRC01:57
*** craigbr has joined #openstack-infra01:59
*** yamahata has quit IRC02:01
*** Daisy has joined #openstack-infra02:01
*** yamahata has joined #openstack-infra02:01
*** LuckyVega has quit IRC02:04
*** praneshp has quit IRC02:06
*** boden has joined #openstack-infra02:08
*** yamamoto has quit IRC02:09
tchaypoYesterday or the day before02:10
*** craigbr has quit IRC02:10
*** rhsu has quit IRC02:11
*** boden has quit IRC02:13
*** bdpayne has joined #openstack-infra02:14
*** JayJ____ has quit IRC02:15
*** Ryan_Lane has quit IRC02:18
anteayamikal back now I needed a bit of a break02:19
anteayamikal: your patch is the height of loveliness02:20
anteayamikal: it is the most beautiful thing i have seen in several days02:20
mikalanteaya: LOL02:20
anteayamikal: it is indeed the exact thing to allay my concerns02:20
mikalanteaya: ok, so good. As long as it meets your needs I am happy.02:20
anteayait does very much02:20
mikalExcellent02:21
anteayaplease be very happy02:21
mikalI shall eat the lunch of the victor then02:21
anteayaI will comment to that effect on the patch02:21
mikal(Lentil pasta is clearly the lunch of winners)02:21
anteayamikal: if you need more lunch, go ahead I guess02:21
anteayamikal: it is02:21
mikalHeh, I haven't gotten as far as lunch yet... Too disorganized.02:21
*** adalbas has quit IRC02:22
anteayamikal: well I am sure I didn't help matters but I am so very grateful02:23
*** otter768 has joined #openstack-infra02:23
anteayamikal: I commented02:23
anteayafungi: I would so back you backing this assertion -> i'd be willing to back any assertion that because the foundation member system is not open it's inscrutable and unfit02:28
anteayafor being considered as technical election criteria02:28
*** dims has quit IRC02:29
anteayaoh and if rage quitting were an option I would have been gone some time ago02:29
*** dims has joined #openstack-infra02:29
mikalSo, its probably too late to fix voting for Kilo02:29
mikalBut we should attack it early in Kilo so Lemming is right02:30
*** dims has quit IRC02:30
*** dims has joined #openstack-infra02:30
*** dims has quit IRC02:30
*** dims has joined #openstack-infra02:31
anteayamikal: I am all for limping along through kilo as long as my heart still believes we are in compliance with the tc charter and having a full on rearrangement of how we do things for elections and how to get the structure in place to make that happen02:34
*** jpich_ is now known as jpich02:35
anteayaI have no horse in the race about what the structure is as long as it is documented so elections can be conducted in compliance with it02:35
*** dims has quit IRC02:35
*** yamamoto has joined #openstack-infra02:35
*** hdd_ has joined #openstack-infra02:45
*** praneshp has joined #openstack-infra02:48
morganfainbergso.. i have a random question someone here might provide me some insight on...02:48
morganfainbergnot so random actually02:48
*** praneshp_ has joined #openstack-infra02:49
morganfainberghttps://review.openstack.org/#/c/119452 - this is a replacement for the memcache lib with something that doesn't leak resources to the point where you could run ourt of FDs, however, it assumes eventlet and multuple threads (works single threaded but no real benefit to testing it single threaded)02:49
*** patrickeast has quit IRC02:50
*** carl_baldwin has joined #openstack-infra02:50
SlickNikanteaya: Updated https://review.openstack.org/#/c/121457 as per the suggestions. Thanks for the reviews!02:50
morganfainbergi'm trying to figure out the best way to test the threading fixes ... any... uh.. suggestions? its not something i typically have tried to test (multiple worker capable code) in the OpenStack test suites02:50
clarkbmorganfainberg: I think nova has worker thread tests02:51
clarkbmorganfainberg: you might grep around in that tree?02:51
morganfainbergclarkb, i'll go hunting.02:51
morganfainbergclarkb, thanks02:51
clarkbmorganfainberg: I want to say they have artifical sequencing points where you can check specific behaviors02:52
SlickNikWould also appreciate some core eyeballs on https://review.openstack.org/#/c/121457/ when folks get a chance, thanks!02:52
morganfainbergclarkb, yeah i'm dealing with dealing with inter-thread communication and locking (save me)02:52
anteayaSlickNik: thank you02:52
*** jamielennox is now known as jamielennox|away02:52
*** praneshp has quit IRC02:52
*** praneshp_ is now known as praneshp02:52
morganfainbergclarkb, but basically memcache + eventlet = bad bad bad bad bad news02:52
morganfainbergpython-memcached is not even close to eventlet safe02:53
morganfainbergs/memcached/memcache02:53
*** Ryan_Lane has joined #openstack-infra02:58
*** sabeen has quit IRC02:59
*** amotoki has joined #openstack-infra03:03
*** amotoki has quit IRC03:03
*** carl_baldwin has quit IRC03:04
*** grue_pm has joined #openstack-infra03:07
anteayaSlickNik: figured I could sell you on "new-project" :D03:08
*** boden has joined #openstack-infra03:09
anteayaSlickNik: I don't think we need tarballs for specs03:09
anteayaSlickNik: sorry I didn't notice it before03:10
*** otter768 has quit IRC03:13
*** oanufriev has quit IRC03:14
*** boden has quit IRC03:14
anteayaSlickNik: yeah, so I am -1 on the tarballs line and +1 on everything else03:15
anteayaand I am calling it a night03:15
anteayait is a night03:15
anteayag'night03:15
*** Ryan_Lane has quit IRC03:17
*** oanufriev has joined #openstack-infra03:22
*** harlowja_at_home has joined #openstack-infra03:26
*** yaguang has quit IRC03:28
*** wenlock has joined #openstack-infra03:31
*** zz_dimtruck is now known as dimtruck03:32
*** daya_k has joined #openstack-infra03:39
*** daya_k_ has joined #openstack-infra03:40
*** koolhead17 has joined #openstack-infra03:40
*** bhuvan has joined #openstack-infra03:40
*** daya_k has quit IRC03:44
*** daya_k_ is now known as daya_k03:44
*** chair6 has left #openstack-infra03:44
*** 20WABJQK7 has quit IRC03:47
*** arnaud has quit IRC03:47
* krotscheck has discovered that debugging an import tool is difficult when you cannot access the server form which you’r eimporting03:52
krotscheckEvening hack night, go!03:52
*** rushiagr_away is now known as rushiagr03:56
*** esker has joined #openstack-infra03:58
openstackgerritNikhil Manchanda proposed a change to openstack-infra/config: Create a trove-specs repository  https://review.openstack.org/12145704:01
*** daya_k_ has joined #openstack-infra04:03
*** daya_k has quit IRC04:04
*** daya_k_ is now known as daya_k04:04
*** tomoe has quit IRC04:04
*** garyh has quit IRC04:10
*** boden has joined #openstack-infra04:10
*** pballand has joined #openstack-infra04:13
*** boden has quit IRC04:15
*** oomichi has joined #openstack-infra04:16
*** yaguang has joined #openstack-infra04:16
*** eharney_ has quit IRC04:18
*** laxity has quit IRC04:21
*** baoli has quit IRC04:22
tchaypoffs04:23
* tchaypo takes some deep breaths04:23
*** tomoe has joined #openstack-infra04:26
*** otherwiseguy has joined #openstack-infra04:32
*** esker has quit IRC04:34
*** hdd_ has quit IRC04:36
*** Ryan_Lane has joined #openstack-infra04:37
*** laxity has joined #openstack-infra04:37
*** arnaud has joined #openstack-infra04:42
*** rhsu has joined #openstack-infra04:42
*** rushiagr is now known as rushiagr_away04:43
*** yfried|afk has quit IRC04:43
*** vigneshvar has joined #openstack-infra04:49
*** harlowja_at_home has quit IRC04:50
*** Longgeek has quit IRC04:50
*** daya_k_ has joined #openstack-infra04:51
*** arnaud has quit IRC04:52
*** dimtruck is now known as zz_dimtruck04:52
*** daya_k has quit IRC04:54
*** daya_k_ is now known as daya_k04:54
*** signed8bit has joined #openstack-infra04:56
jogohttp://status.openstack.org/elastic-recheck/data/uncategorized.html hasn't updated in a few days04:58
jogolast update  2014-09-13T20:5004:58
jogoclarkb: ^04:58
*** Ryan_Lane has quit IRC04:58
*** harlowja_at_home has joined #openstack-infra04:58
*** signed8bit has quit IRC04:59
*** harlowja_ is now known as harlowja_away04:59
*** signed8bit has joined #openstack-infra04:59
*** stevemar has quit IRC05:03
openstackgerritmelanie witt proposed a change to openstack-dev/pbr: add the ability to get version from METADATA file  https://review.openstack.org/12175105:05
*** pballand has quit IRC05:06
*** sandywalsh has quit IRC05:07
*** tomoe has joined #openstack-infra05:08
*** Longgeek has joined #openstack-infra05:08
clarkbjogo k. I just got internets up in new place can look tomorrow05:08
*** grue_pm has quit IRC05:08
*** Longgeek has quit IRC05:09
*** Longgeek_ has joined #openstack-infra05:09
*** tomoe has quit IRC05:09
*** tomoe has joined #openstack-infra05:10
*** zhiyan_ is now known as zhiyan05:10
*** boden has joined #openstack-infra05:11
*** boden has quit IRC05:16
jogokk05:16
jogoclarkb: new place in portland?05:16
*** rushiagr_away is now known as rushiagr05:16
clarkbyup05:17
*** signed8bit has quit IRC05:18
*** bdpayne has quit IRC05:19
*** bdpayne has joined #openstack-infra05:20
*** nelsnelson has joined #openstack-infra05:22
jogoclarkb: nice, enjoy05:26
*** yamahata has quit IRC05:27
*** yamahata has joined #openstack-infra05:28
*** grue_pm has joined #openstack-infra05:31
*** grue_pm has quit IRC05:32
*** ppai has joined #openstack-infra05:35
*** yfried|afk has joined #openstack-infra05:40
*** otherwiseguy has quit IRC05:44
*** otherwiseguy has joined #openstack-infra05:44
*** harlowja_at_home has quit IRC05:45
*** mmaglana has quit IRC05:45
*** imcsk8_ has quit IRC05:46
*** imcsk8 has joined #openstack-infra05:46
*** otherwiseguy has quit IRC05:50
*** vigneshvar has quit IRC05:57
*** vigneshvar has joined #openstack-infra06:00
*** reed has quit IRC06:01
*** yfried has joined #openstack-infra06:02
*** yfried|afk has quit IRC06:03
*** renlt has joined #openstack-infra06:07
*** bdpayne has quit IRC06:09
*** renlt has quit IRC06:10
*** renlt has joined #openstack-infra06:11
*** yfried_ has joined #openstack-infra06:11
*** yfried_ is now known as yfried|afk06:11
*** boden has joined #openstack-infra06:12
*** renlt has quit IRC06:12
*** gpanda has joined #openstack-infra06:12
*** yfried has quit IRC06:14
openstackgerritA change was merged to openstack-infra/zuul: Display only first 7 chars for git commit id  https://review.openstack.org/8355606:16
*** boden has quit IRC06:16
*** loki184 has joined #openstack-infra06:17
*** flaper87|afk is now known as flaper8706:17
*** dmellado has joined #openstack-infra06:17
*** LinuxJedi has quit IRC06:22
*** wenlock has quit IRC06:24
*** ashaeron has joined #openstack-infra06:25
*** e0ne has joined #openstack-infra06:29
*** denis_makogon has joined #openstack-infra06:31
*** MaxV has joined #openstack-infra06:34
*** LinuxJedi has joined #openstack-infra06:37
*** e0ne has quit IRC06:37
*** KanagarajM has joined #openstack-infra06:39
*** nelsnelson has quit IRC06:41
*** MaxV has quit IRC06:41
*** e0ne has joined #openstack-infra06:41
*** jyuso has joined #openstack-infra06:43
*** e0ne has quit IRC06:43
*** nelsnelson has joined #openstack-infra06:44
*** pkoniszewski has joined #openstack-infra06:46
*** HeOS has joined #openstack-infra06:48
*** yolanda has joined #openstack-infra06:54
*** bhuvan has quit IRC06:54
*** arxcruz has joined #openstack-infra07:00
*** nadya has joined #openstack-infra07:00
*** nadya is now known as Guest1331907:01
*** rhsu has quit IRC07:06
*** KanagarajM2 has joined #openstack-infra07:06
*** KanagarajM has quit IRC07:07
*** mpaolino has joined #openstack-infra07:07
*** rcarrillocruz has quit IRC07:07
*** rcarrillocruz has joined #openstack-infra07:08
*** bradjones has quit IRC07:09
*** boden has joined #openstack-infra07:12
*** daya_k_ has joined #openstack-infra07:13
*** daya_k has quit IRC07:14
*** daya_k_ is now known as daya_k07:14
*** bradjones has joined #openstack-infra07:14
*** bradjones has quit IRC07:15
*** bradjones has joined #openstack-infra07:15
*** boden has quit IRC07:17
*** yamamoto has quit IRC07:20
*** Guest81828 has joined #openstack-infra07:23
*** daya_k has quit IRC07:24
*** bogdando has quit IRC07:26
openstackgerritCarl Loa Odin proposed a change to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality  https://review.openstack.org/9105107:26
*** shardy_z is now known as shardy07:28
*** jp_at_hp has joined #openstack-infra07:32
*** MaxV has joined #openstack-infra07:33
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/config: Add settings.xml parameter to Jenkins slave class  https://review.openstack.org/12156007:33
*** jpich has quit IRC07:35
*** hashar has joined #openstack-infra07:35
*** jlibosva has joined #openstack-infra07:40
*** yamamoto has joined #openstack-infra07:45
*** mrmartin has joined #openstack-infra07:45
*** dizquierdo has joined #openstack-infra07:47
*** jistr has joined #openstack-infra07:50
*** jgallard has joined #openstack-infra07:50
*** berendt has joined #openstack-infra07:52
*** ihrachyshka has joined #openstack-infra07:53
*** melwitt has quit IRC07:55
*** tomoe has quit IRC07:56
openstackgerritA change was merged to openstack-dev/pbr: Refactor oneline changelog walking.  https://review.openstack.org/10827107:58
*** derekh has joined #openstack-infra07:59
*** zz_johnthetubagu is now known as johnthetubaguy08:03
*** e0ne has joined #openstack-infra08:08
mikais there a way to use "Submit" (to actually merge a change) with gertty? 'gertty --print-keymap' doesn't list something like that, any hints - jeblair or fungi?08:08
*** IvanBerezovskiy has joined #openstack-infra08:08
*** johnthetubaguy is now known as zz_johnthetubagu08:09
*** e0ne has quit IRC08:09
*** boden has joined #openstack-infra08:13
*** Longgeek_ has quit IRC08:13
*** Longgeek has joined #openstack-infra08:14
*** andreykurilin_ has joined #openstack-infra08:14
*** zz_johnthetubagu is now known as johnthetubaguy08:16
*** boden has quit IRC08:17
*** boden has joined #openstack-infra08:27
*** koolhead17 has quit IRC08:27
*** ociuhandu has quit IRC08:27
*** Kim_ has joined #openstack-infra08:28
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack-infra/config: Add tripleo-ansible to stackforge  https://review.openstack.org/12177608:30
*** tomoe has joined #openstack-infra08:32
*** andreykurilin_ has quit IRC08:35
Kim_My CI system advertise "build succeeded"  in some commits that should not run at all according to my filter policies at layout.yaml, is someone have any idea?08:36
*** flaper87 is now known as flaper87|afk08:37
*** flaper87|afk is now known as flaper8708:37
coolsvapmy devstack patch https://review.openstack.org/#/c/120351/ returned -1 since the check-tempest-dsvm-postgres-full gate is unstable, looks like its getting 404 for ImagesMetadataTestJSON tests, should I do a recheck?08:39
*** pelix has joined #openstack-infra08:43
*** cnesa12 has joined #openstack-infra08:50
openstackgerritThierry Carrez proposed a change to openstack-infra/release-tools: Add oslo_release.sh script  https://review.openstack.org/12152708:51
*** Guest13319 has quit IRC08:53
*** flaper87 is now known as flaper87|afk08:56
*** e0ne has joined #openstack-infra09:00
*** BobBall_AWOL is now known as BobBall09:02
*** amotoki has joined #openstack-infra09:04
*** cdent has joined #openstack-infra09:05
*** flaper87|afk is now known as flaper8709:05
*** jgallard has quit IRC09:12
*** jgallard has joined #openstack-infra09:15
*** KanagarajM2 has quit IRC09:23
*** skolekonov has joined #openstack-infra09:28
*** marun has joined #openstack-infra09:30
*** marun has quit IRC09:31
*** KanagarajM has joined #openstack-infra09:34
*** koolhead17 has joined #openstack-infra09:34
*** koolhead17 has joined #openstack-infra09:35
*** daya_k has joined #openstack-infra09:43
Kim_My CI system advertise "build succeeded"  in some commits that should not run at all according to my filter policies at layout.yaml, is someone have any idea?09:44
*** Guest81828 has quit IRC09:46
*** luqas has quit IRC09:47
*** Longgeek has quit IRC09:48
*** yamamoto has quit IRC09:52
*** yfried|afk is now known as yfried_09:55
*** nadya_ has joined #openstack-infra09:56
*** flaper87 is now known as flaper87|afk09:57
*** KanagarajM2 has joined #openstack-infra10:00
*** e0ne has quit IRC10:01
*** pcm_ has joined #openstack-infra10:01
*** unicell1 has joined #openstack-infra10:01
*** pcm_ has quit IRC10:01
*** KanagarajM has quit IRC10:02
*** pcm_ has joined #openstack-infra10:02
*** unicell has quit IRC10:02
*** ZZelle has quit IRC10:03
*** ZZelle has joined #openstack-infra10:03
*** afazekas has joined #openstack-infra10:03
*** e0ne has joined #openstack-infra10:04
*** yamamoto has joined #openstack-infra10:04
*** tomoe has quit IRC10:04
*** Daisy has quit IRC10:04
*** tomoe has joined #openstack-infra10:05
*** KanagarajM2 has quit IRC10:06
*** mflobo_ has quit IRC10:07
*** mflobo has joined #openstack-infra10:08
*** praneshp has quit IRC10:08
*** penguinRaider has quit IRC10:08
*** penguinRaider has joined #openstack-infra10:08
*** tomoe has quit IRC10:09
*** rushiagr is now known as rushiagr_away10:09
*** gpanda has quit IRC10:11
*** jyuso has quit IRC10:13
*** amuller has joined #openstack-infra10:13
*** dalgaaf__ has joined #openstack-infra10:13
*** jgallard has quit IRC10:17
*** rlandy has joined #openstack-infra10:19
*** jcoufal-mobile has joined #openstack-infra10:25
*** jcoufal-mobile has quit IRC10:30
*** koolhead17 has quit IRC10:30
*** penguinRaider has quit IRC10:33
*** cnesa12 has quit IRC10:45
*** yfried_ has quit IRC10:45
*** hashar has quit IRC10:45
*** penguinRaider has joined #openstack-infra10:46
*** salv-orlando has joined #openstack-infra10:49
*** Longgeek has joined #openstack-infra10:49
*** Kim_ has quit IRC10:51
*** ildikov has joined #openstack-infra10:52
*** flaper87|afk is now known as flaper8710:53
*** Longgeek has quit IRC10:55
*** daya_k has quit IRC10:55
*** dims has joined #openstack-infra10:56
*** kim_ has joined #openstack-infra10:58
*** pblaho has joined #openstack-infra11:00
*** _shaps_ has quit IRC11:01
*** mwagner_lap has quit IRC11:03
flaper87SergeyLukjanov: fungi jeblair hey guys, do you think we can move this forward? https://review.openstack.org/#/c/120437/11:03
flaper87I haven't hat a chance to manual test this11:03
flaper87I've been meaning to, FWIW.11:03
*** hashar has joined #openstack-infra11:07
*** marun has joined #openstack-infra11:09
kashyapttx, Nicely expressed about the 25% chunks discussion of bugs across IRC channels.11:13
*** tomoe has joined #openstack-infra11:14
*** yfried_ has joined #openstack-infra11:14
*** yfried_ is now known as yfried|afk11:14
*** yfried has joined #openstack-infra11:17
*** jgallard has joined #openstack-infra11:17
*** yaguang has quit IRC11:19
*** yfried|afk has quit IRC11:20
*** zul has quit IRC11:21
*** zul has joined #openstack-infra11:22
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/config: Add support for multi-module maven jobs  https://review.openstack.org/12181111:23
*** rushiagr_away is now known as rushiagr11:23
*** tomoe has quit IRC11:26
*** tomoe has joined #openstack-infra11:26
kim_My CI system advertise "Build succeeded." on a commit from branch that it should not check at all...does any one have any idea why?11:27
*** tomoe has quit IRC11:30
marunsdague: hi!11:33
*** daya_k has joined #openstack-infra11:33
chmouelhello any ideas how come that bug got assigned to me https://bugs.launchpad.net/grenade/+bug/1365105 (without me asking) ?11:33
uvirtbotLaunchpad bug 1365105 in devstack "grenade fails to upgrade keystone when mod wsgi is enabled" [Undecided,In progress]11:33
marunsdague: I have a few questions about improving devstack support for functional testing when you have a moment.11:34
marunsdague: Namely the best way to install dependencies only, and how to install both postgres and mysql to support db-specific migration testing.11:34
*** flaper87 is now known as flaper87|afk11:38
*** arxcruz has quit IRC11:42
*** kgiusti has joined #openstack-infra11:43
*** weshay has joined #openstack-infra11:44
*** mrmartin has quit IRC11:45
rcarrillocruzzaro: hi there, i pushed a change for multimodule maven jobs, appreciate if you could review https://review.openstack.org/#/c/121811/111:48
*** tomoe has joined #openstack-infra11:48
*** mmedvede has joined #openstack-infra11:48
*** flaper87 has joined #openstack-infra11:49
*** loki184 has quit IRC11:50
*** tomoe has quit IRC11:50
*** Longgeek has joined #openstack-infra11:50
*** luqas has joined #openstack-infra11:51
*** tomoe has joined #openstack-infra11:51
*** baoli has joined #openstack-infra11:53
*** Longgeek has quit IRC11:55
*** tomoe has quit IRC11:55
*** arxcruz has joined #openstack-infra11:57
*** pblaho has quit IRC12:03
*** koolhead17 has joined #openstack-infra12:03
*** koolhead17 has joined #openstack-infra12:03
*** flaper87 has quit IRC12:06
*** pblaho has joined #openstack-infra12:08
openstackgerritA change was merged to openstack-dev/pbr: Stop using intersphinx  https://review.openstack.org/12132912:08
*** jcoufal has joined #openstack-infra12:10
*** marun has quit IRC12:14
*** JayJ____ has joined #openstack-infra12:14
*** amotoki has quit IRC12:17
*** dims has quit IRC12:18
*** dims has joined #openstack-infra12:19
*** jp_at_hp has quit IRC12:19
HeOSHi to all! What about https://review.openstack.org/120747 and https://review.openstack.org/117199 ?12:19
*** jp_at_hp has joined #openstack-infra12:19
*** hashar_ has joined #openstack-infra12:21
*** dkranz has joined #openstack-infra12:22
*** ihrachyshka_ has joined #openstack-infra12:22
*** pblaho is now known as pblaho|afk12:22
*** aysyanne has joined #openstack-infra12:23
*** pblaho|afk is now known as pblaho12:24
*** jp_at_hp1 has joined #openstack-infra12:24
*** hashar has quit IRC12:24
*** pblaho has quit IRC12:24
*** jp_at_hp has quit IRC12:24
*** ihrachyshka has quit IRC12:25
*** doug-fish has left #openstack-infra12:25
*** jp_at_hp has joined #openstack-infra12:27
*** jp_at_hp1 has quit IRC12:28
*** tomoe has joined #openstack-infra12:29
SergeyLukjanovjeblair, fungi, clarkb, mordred, I'm on vacation now and will be traveling to CA tomorrow, so, unable to attend today's meeting12:29
*** tomoe has quit IRC12:30
*** hashar_ has quit IRC12:30
*** cnesa12 has joined #openstack-infra12:30
*** tomoe has joined #openstack-infra12:31
*** pblaho has joined #openstack-infra12:31
*** doug-fish has joined #openstack-infra12:33
*** tomoe has quit IRC12:35
*** mwagner_lap has joined #openstack-infra12:38
*** hashar has joined #openstack-infra12:39
*** Guest81828 has joined #openstack-infra12:41
*** zz_jgrimm is now known as jgrimm12:41
*** bmwiedemann has quit IRC12:42
*** bswartz has quit IRC12:43
*** bmwiedemann has joined #openstack-infra12:43
*** adalbas has joined #openstack-infra12:45
*** mriedem has joined #openstack-infra12:46
*** dprince has joined #openstack-infra12:46
*** tomoe has joined #openstack-infra12:47
*** mriedem has quit IRC12:48
*** juzuluag has joined #openstack-infra12:48
*** mjturek has joined #openstack-infra12:48
*** mjturek has left #openstack-infra12:49
*** mjturek has joined #openstack-infra12:49
*** Longgeek has joined #openstack-infra12:51
*** dkranz has quit IRC12:52
*** hashar has quit IRC12:52
*** jgallard has quit IRC12:52
*** ociuhandu has joined #openstack-infra12:52
*** hashar has joined #openstack-infra12:55
*** mriedem has joined #openstack-infra12:56
*** Longgeek has quit IRC12:57
*** KanagarajM has joined #openstack-infra12:58
*** sdake has joined #openstack-infra13:00
*** tomoe has quit IRC13:02
kim_ My CI system advertise "Build succeeded." on a commit from branch that it should not check at all...does any one have any idea why?13:02
*** tomoe has joined #openstack-infra13:03
*** dane_leblanc has quit IRC13:05
*** pblaho has quit IRC13:06
*** tomoe has quit IRC13:07
*** mbacchi has joined #openstack-infra13:07
*** Longgeek has joined #openstack-infra13:10
*** radez_g0n3 is now known as radez13:12
*** dustins has joined #openstack-infra13:13
*** changbl_ has quit IRC13:13
*** bknudson has quit IRC13:16
*** pblaho has joined #openstack-infra13:18
*** koolhead17 has quit IRC13:18
*** isviridov is now known as isviridov_at_met13:20
fungikim_: i'd need to see the change it commented on and a copy of your layout.yaml file13:21
fungiSergeyLukjanov: thanks for the heads up, and enjoy your vacation13:21
*** pblaho has quit IRC13:23
*** pblaho has joined #openstack-infra13:24
fungilovely... jenkins06 has died13:27
kim_fungi: http://paste.openstack.org/show/112153/13:27
kim_fungi: Thanks13:27
openstackgerritA change was merged to openstack-infra/config: Upgrade to puppetlabs-postgresql 4.0.0  https://review.openstack.org/12168513:28
*** juzuluag has quit IRC13:28
*** craigbr has joined #openstack-infra13:33
*** rwsu has quit IRC13:33
*** salv-orlando has quit IRC13:34
*** craigbr has quit IRC13:34
*** julim has joined #openstack-infra13:34
*** mfer has joined #openstack-infra13:34
*** daya_k has quit IRC13:34
*** craigbr has joined #openstack-infra13:34
*** zz_gondoi is now known as gondoi13:35
*** dane_leblanc has joined #openstack-infra13:35
openstackgerritA change was merged to openstack-infra/config: Add bare-f20 nodes  https://review.openstack.org/11739713:36
*** e0ne has quit IRC13:36
fungi#status log jenkins ran out of jvm memory on jenkins06 at 01:42:20 http://paste.openstack.org/show/112155/13:38
openstackstatusfungi: finished logging13:38
*** sabeen has joined #openstack-infra13:38
fungijudging from http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=1456&rra_id=all the system was just shy of 13g active memory when it died13:40
*** otherwiseguy has joined #openstack-infra13:41
fungiluckily we don't *appear* to have any hung jobs from it13:41
*** miqui has joined #openstack-infra13:42
*** e0ne has joined #openstack-infra13:42
fungikim_: do you have a link to the non-master-branch change on which your system commented?13:43
*** wenlock has joined #openstack-infra13:43
*** r-daneel__ has joined #openstack-infra13:48
*** amotoki has joined #openstack-infra13:48
kim_fungi: https://review.openstack.org/#/c/120159/13:49
*** dmsimard_away is now known as dmsimard13:55
fungikim_: is it possible that you've overridden it in local_functions.py(set_log_path)?13:58
fungikim_: another possibility is that perhaps the branch filter gets ignored if a files filter is used13:58
*** oomichi has quit IRC13:59
*** nelsnelson has quit IRC14:01
*** nelsnelson has joined #openstack-infra14:02
kim_fungi: I do not think that set_log_path override my filter policy14:02
fungikim_: in that case it may be a bug/limitation when combining a branch filter with a files filter14:03
kim_fungi: http://paste.openstack.org/show/112169/14:03
fungikim_: yeah, your set_log_path() looks pretty normal so i don't think it's involved14:03
*** craigbr has quit IRC14:04
kim_fungi:  You think that there is a bug? in zuul? I do not see any mantion for such a bug in the web14:04
fungikim_: right, i mean you may have just discovered a new bug14:04
*** bswartz has joined #openstack-infra14:04
fungii'll see if i can write a regression test to confirm14:05
*** dkranz has joined #openstack-infra14:05
*** pcrews has joined #openstack-infra14:05
*** wenlock has quit IRC14:06
*** pblaho_ has joined #openstack-infra14:07
*** pblaho_ has quit IRC14:08
kim_fungi: The most strange, it works fine when I configre the manager from: IndependentPipelineManager to: DependentPipelineManager14:08
*** otherwiseguy has quit IRC14:09
*** otherwiseguy has joined #openstack-infra14:10
*** pblaho has quit IRC14:11
fungikim_: oh! i see what's going on14:12
*** dangers_away is now known as dangers14:12
fungikim_: job filters prevent a particular job from being run, but they don't prevent zuul from acting on and commenting on a change14:13
kim_fungi: job filter=branch+file filter?14:14
fungikim_: if you look at the comment zuul left on the stable branch change and compare it to a comment it left on a master branch change, you'll see that on the stable change it successfully ran no tests and reported as much14:14
fungikim_: i think you need some sort of pipeline filter rather than a job filter14:15
*** pblaho has joined #openstack-infra14:15
*** Vek__ has quit IRC14:15
*** andreaf_ has joined #openstack-infra14:16
*** andreaf has quit IRC14:16
*** Longgeek has quit IRC14:16
*** andreaf_ is now known as andreaf14:16
*** andreaf_ has joined #openstack-infra14:17
*** salv-orlando has joined #openstack-infra14:18
kim_fungi: By job filter you mean branch + file filter?14:18
*** yfried has quit IRC14:18
*** pblaho has quit IRC14:18
fungikim_: see the trigger gerrit branch option documented in http://ci.openstack.org/zuul/zuul.html#pipelines14:19
*** pblaho has joined #openstack-infra14:20
*** pblaho has quit IRC14:20
*** pblaho has joined #openstack-infra14:20
fungikim_: it doesn't appear we have a similar option to filter pipeline triggers on change file patterns though, so you might be stuck leaving empty reports on changes which don't touch any files matching your job files filter14:20
*** pblaho has quit IRC14:21
*** Longgeek has joined #openstack-infra14:21
*** Longgeek has quit IRC14:21
fungikim_: or you could submit a feature to zuul for a gerrit files pipeline trigger14:21
*** stevemar has joined #openstack-infra14:23
*** eharney_ has joined #openstack-infra14:24
*** eharney_ is now known as eharney14:24
kim_fungi: but when I eliminate the files filter section by delete the files section from layout.yaml I still have the same issue...14:25
*** andreaf has quit IRC14:26
*** garyh has joined #openstack-infra14:26
fungikim_: yes, that's a separate problem14:27
fungikim_: to avoid reporting on stable branch changes, you need to add a branch filter to the gerrit trigger in your pipeline14:27
fungikim_: per the documentation i linked above14:28
*** YorikSar_ is now known as YorikSar14:29
*** tonytan4ever has joined #openstack-infra14:31
*** rwsu has joined #openstack-infra14:31
jeblairmika: no there isn't a way to submit with gertty.  i think we would need a schema change to add a pending submit action, then a new sync action to actually send it.  not too hard, but not quite trivial.14:31
jeblairmika: (we don't use submit with openstack (our CI system is the only user who can submit), so that wasn't high on my list for the first version)14:31
*** david-lyle has joined #openstack-infra14:32
*** jgrimm is now known as zz_jgrimm14:33
*** krtaylor has quit IRC14:34
*** pkoniszewski has quit IRC14:35
*** lttrl has joined #openstack-infra14:37
*** tonytan4ever has quit IRC14:39
*** zz_dimtruck is now known as dimtruck14:39
*** ppai has quit IRC14:39
*** andreaf has joined #openstack-infra14:43
openstackgerritA change was merged to stackforge/gertty: Allow specifying a config file  https://review.openstack.org/12139314:44
openstackgerritA change was merged to stackforge/gertty: Fix some username related problems  https://review.openstack.org/12139414:45
*** pblaho has joined #openstack-infra14:45
anteayakim_: which ci is yours?14:45
openstackgerritA change was merged to stackforge/gertty: Associate orphan messages with revision 1  https://review.openstack.org/12139514:45
*** sandywalsh has joined #openstack-infra14:46
jeblairanteaya: mellanox14:46
anteayaSergeyLukjanov: safe travels14:47
openstackgerritA change was merged to stackforge/gertty: Handle variable labels in change list  https://review.openstack.org/12139614:47
anteayajeblair: thanks14:47
jeblairkrotscheck: had a question inline on 11926614:48
anteayaI'm still looking to talk to the vArmour ci folks14:48
kim_anteaya: We aew from Mellanox14:50
kim_fungi: thanks14:50
anteayakim_: thank you14:52
anteayakim_: thanks for investigating what your system is doing14:52
*** bdpayne has joined #openstack-infra14:53
openstackgerritA change was merged to openstack-infra/config: Remove old projects yaml jobs  https://review.openstack.org/11668814:53
*** tonytan4ever has joined #openstack-infra14:53
kim_anteaya: :)14:54
*** FallenPegasus has joined #openstack-infra14:54
*** marun has joined #openstack-infra14:55
anteayakim_: no really most people wait for their system to be disabled before they ask any questions, I didn't know your system was doing this so good work, I am impressed14:56
openstackgerritJordan OMara proposed a change to openstack-infra/config: Add xstatic-angular-fileupload support  https://review.openstack.org/12010114:56
anteayanow your system *is* currently disabled but for other reasons14:56
*** ihrachyshka_ has quit IRC14:57
fungiit's worth noting that the lack of a files option for pipeline triggers in zuul is probably something we'd be glad to support if someone wrote up the patches to add and test it, we just haven't needed to maintain file-specific pipelines in our production environment and so never implemented it14:57
*** jgallard has joined #openstack-infra14:58
anteayafungi: would that be worthy of a bug, so then I can post the url in third party meetings?14:58
anteayaI can file the bug if you think so14:58
kim_anteaya: We are dedicate for CI and ask question mainly when we are enable14:59
*** ildikov_ has joined #openstack-infra14:59
fungifeature request14:59
anteayafungi: feature request in lp or storyboard?14:59
anteayakim_: great, well asking questions anytime is wonderful - thank you15:00
anteayakim_: can you ever attend the third party meetings that happen on Mondays at 1800 UTC?15:00
*** tomoe has joined #openstack-infra15:00
*** wenlock has joined #openstack-infra15:00
*** ildikov has quit IRC15:01
*** esker has joined #openstack-infra15:02
fungianteaya: looks like lp still? the zuul bugs don't seem to be primarily in sb yet15:02
kim_anteaya: I attendees this meeting all the times (omrim) I have been at the meeting last night..15:02
anteayafungi: kk, thanks15:03
anteayakim_: ah I didn't realize you also were omrim15:03
anteayakim_: and yes, you do attend and thank you15:03
*** bdpayne has quit IRC15:08
kim_anteaya: I validate my fix in my CI playground and it works, can you enable Mellanox CI system?15:09
*** craigbr has joined #openstack-infra15:09
anteayakim_: well mellanox working wasn't the issue, the issue if I recall was voting too many times - which you have demonstrated you will address by following a different process when you backup your servers in future and not having urls to logs in your comments15:10
openstackgerritNikolay Fedotov proposed a change to openstack-infra/nodepool: Remove hardcoded root path of a jenkins node  https://review.openstack.org/12189615:10
anteayakim_: do you have urls to test logs in your comments now?15:10
*** pelix has quit IRC15:11
anteayalaunchpad server appears cranky15:12
kim_anteaya: Yes15:12
*** amuller_ has joined #openstack-infra15:13
*** otherwiseguy has quit IRC15:13
kim_anteaya: I will explain what was wrong in the email you sent me so everyone will have this knowledge too...15:13
anteayafungi: https://bugs.launchpad.net/zuul/+bug/137010515:13
uvirtbotLaunchpad bug 1370105 in zuul "feature request: gerrit files option for pipeline triggers" [Undecided,New]15:13
anteayakim_: thank you I appreciate this15:13
*** juzuluag has joined #openstack-infra15:14
anteayafungi: if you would be so kind: 9732 mellanox Mellanox CI mlnx-openstack-ci@dev.mellanox.co.il15:14
*** dannywilson has joined #openstack-infra15:14
*** pelix has joined #openstack-infra15:15
*** otherwiseguy has joined #openstack-infra15:15
*** amuller has quit IRC15:16
fungianteaya: kim_: re-activated15:17
kim_anteaya: Thanks15:17
kim_fungi: thanks15:17
*** hdd has joined #openstack-infra15:20
anteayakim_: please show me the url of a patch that your system comments on so I can confirm the url is in the comments and the logs are browsable15:20
anteayathe url for your test logs is in the comment15:21
anteayafungi: I have added the url for the zuul feature request in next week's third party meeting: https://wiki.openstack.org/wiki/Meetings/ThirdParty#09.2F22.2F1415:22
*** zz_jgrimm is now known as jgrimm15:22
*** loki184 has joined #openstack-infra15:22
anteayafungi: have you time to look at this patch which moves a dvr test job to the check queue? https://review.openstack.org/#/c/120603/ 8 +1's and a +215:24
*** esker has quit IRC15:24
kim_anteaya: I will send you when I have results for all branches it will take a hour or two..15:26
*** afazekas has quit IRC15:28
jogogot my first number, over the last 400 merged patches in nova we have done 0.67 rechecks/reverifies per patch revision15:29
anteayakim_: thanks15:30
anteayajogo: interesting15:30
*** loki184 has quit IRC15:30
openstackgerritA change was merged to openstack-infra/zuul: Add check for ref being a string before applying regex  https://review.openstack.org/11657715:31
anteayajogo: any idea how many of those were initiated when the code was actually breaking things but the dev didnt' look at the logs to find out?15:31
*** flaper87|afk is now known as flaper8715:31
anteayaversus those initiated because the vm failed to buiid for instance15:32
jogoanteaya: no idea15:32
anteayaokay15:32
anteayaso two factors in that number15:32
jogoanteaya: but good question, my *hunch* is that is a minority case15:32
anteaya1) legitimate system failurs15:32
anteaya2) poor dev habits15:32
anteayawell you went digging to aliviate hunches15:33
jogoanteaya: so how do I sort the two cases out?15:33
anteayawhich was the motivation for this as of yesterday15:33
*** atiwari has joined #openstack-infra15:33
anteayajogo: that is a very good question15:33
*** esker has joined #openstack-infra15:33
anteayawell one would be if the same error hit a patch series more than once15:33
sdaguedid somone kill git ?15:33
anteayafor instance15:33
anteayahttp://git.openstack.org/cgit renders for me15:34
jeblairjogo: at least one job failure in common on both sides of a recheck (all for the same patchset) suggests someone did a blind recheck on a bug their patch introduced.15:34
jogoanteaya: I can't easily detect what the error was15:34
anteayasdague: what are you seeing15:34
sdaguehttp://logs.openstack.org/81/120781/6/check/gate-nova-pep8/6f676c1/console.html15:34
sdague100% of all my tests on that change failed15:34
jogojeblair: yeah I like that, thanks15:34
jeblairsdague: cloning from git.o.o works for me15:34
anteayasdague: when was that from?15:35
sdaguehttps://review.openstack.org/#/c/120781/15:35
sdaguevery recently15:35
anteayasdague: jenkis06 fell over not long ago15:35
sdaguean across the board failure15:35
jeblairsdague: that's from hpcloud-b1; where were the other failures from?15:35
anteayasdague: https://wiki.openstack.org/wiki/Infrastructure_Status15:36
*** mmaglana has joined #openstack-infra15:36
*** DuncanT-away is now known as DuncanT15:36
sdaguehttp://logs.openstack.org/81/120781/6/check/check-tempest-dsvm-neutron-full/94a79a9/console.html b215:36
openstackgerritMikhail S Medvedev proposed a change to openstack-infra/elastic-recheck: Disambiguate filename:<file> from tag:<file> in query guidelines  https://review.openstack.org/12190815:36
sdaguehttp://logs.openstack.org/81/120781/6/check/check-tempest-dsvm-neutron-heat-slow/e7f7fcb/console.html b515:36
*** mmaglana has quit IRC15:36
*** mpaolino has quit IRC15:36
jeblairsdague: so an across-the-board hp, rax, or Internet networking issue15:36
jeblairsdague: or an issue with git.o.o (the load balancer)15:37
*** IvanBerezovskiy has left #openstack-infra15:37
sdagueI see b415:37
jeblairall 5 git servers and the lb work for me atm.15:37
sdagueI didn't see a rax node in the fails, b1, b2, b4, b515:37
openstackgerritA change was merged to openstack-infra/zuul: Rename doc environment to docs  https://review.openstack.org/11991115:37
sdagueanything recorded on the git side?15:38
jeblairsdague: was there a rax node that succeeded?15:38
sdaguejeblair: yes15:38
sdagueyeh, the 2 successes were on rax15:38
jeblairhttp://cacti.openstack.org/cacti/graph.php?action=zoom&local_graph_id=862&rra_id=0&view_type=tree&graph_start=1410877161&graph_end=1410877790&graph_height=120&graph_width=500&title_font_size=1015:38
sdagueall of the git servers in rax?15:38
jeblairgit.o.o's traffic doubled from 150 to 300mbps around that time15:38
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Error on duplicate job names being found  https://review.openstack.org/5673415:38
jeblairsdague: yes15:39
*** Steap has quit IRC15:39
*** Steap_ has joined #openstack-infra15:39
*** krtaylor has joined #openstack-infra15:39
pelixzaro: think I've caught the various possible duplicate situations for projects with the above change, but the tests added probably need to be checked just to make sure I haven't missed something!15:39
*** daya_k has joined #openstack-infra15:39
sdagueok, well I'll just recheck15:40
sdaguethe across the board made me wonder if it was a longer running issue15:40
jeblairhttp://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=12515:41
jeblairgit05 has some gaps; it may have been kicked out of the lb15:41
*** marcoemorais has joined #openstack-infra15:42
*** rushiagr is now known as rushiagr_away15:43
*** ashaeron has quit IRC15:46
*** vigneshvar has quit IRC15:47
*** jerryz_ has joined #openstack-infra15:47
*** e0ne has quit IRC15:48
*** ildikov_ has quit IRC15:49
*** tomoe has quit IRC15:50
*** tomoe has joined #openstack-infra15:50
*** jgallard has quit IRC15:52
*** esker has quit IRC15:52
sdaguefungi: I think I'm giving up on - https://review.openstack.org/#/c/117609/15:53
*** shashankhegde has joined #openstack-infra15:53
*** jerryz_ has quit IRC15:54
*** dannywilson has quit IRC15:54
sdaguetoo many clients updated things that aren't back in havana15:54
*** tomoe has quit IRC15:54
*** dannywilson has joined #openstack-infra15:55
*** garyh has quit IRC15:56
*** andreykurilin has joined #openstack-infra15:56
*** kim_ has quit IRC15:58
*** ociuhandu_ has joined #openstack-infra15:59
*** dannywilson has quit IRC15:59
sdaguejeblair: yep... still failing - http://logs.openstack.org/82/120782/7/check/gate-nova-pep8/d932ce0/console.html16:00
*** bknudson has joined #openstack-infra16:00
*** arxcruz has quit IRC16:00
zaropelix: cool. will take a look.16:01
sdagueoh, sorry, that was just delayed email getting to me16:01
*** dimtruck is now known as zz_dimtruck16:02
*** ociuhandu_ has quit IRC16:02
*** dannywilson has joined #openstack-infra16:02
*** kgiusti is now known as kgiusti|afk16:03
*** bdpayne has joined #openstack-infra16:04
fungisdague: well, also, havana is about to get put down16:04
krotscheckjeblair: tx- will look at 119266 right now.16:04
sdaguefungi: yeh16:04
*** SumitNaiksatam has quit IRC16:04
*** bdpayne_ has joined #openstack-infra16:06
*** ramashri has joined #openstack-infra16:06
krotscheckjeblair: Your comment on https://review.openstack.org/#/c/119266/4/storyboard/notifications/publisher.py - I’m not quite getting the context there, could you explain the python things I’m not getting there?16:07
*** garyh has joined #openstack-infra16:08
*** bdpayne has quit IRC16:08
jeblairkrotscheck: sorry, be back in 5 mins16:09
krotscheckjeblair: No worries, I’m sitting in a keynote.16:09
daya_kfungi, anteaya:  quick reminder to consider my request to re-enable ibm sdnve ci16:09
*** Guest81828 has quit IRC16:10
*** virmitio has joined #openstack-infra16:10
*** rhsu has joined #openstack-infra16:11
*** Vek__ has joined #openstack-infra16:12
*** changbl_ has joined #openstack-infra16:13
jeblairkrotscheck: so first thing: i think there's a bug in gertty and a comment i left is missing.  that's very disturbing and i will fix asap.16:15
anteayadaya_k: http://lists.openstack.org/pipermail/third-party-announce/2014-September/000008.html do you have browsable logs?16:15
*** marun_ has joined #openstack-infra16:15
krotscheckjeblair: That makes more sense :)16:15
*** maxbit has joined #openstack-infra16:16
daya_kanteaya : yes, in my email i have a link with an example,16:16
daya_kanteaya :  http://sng01.objectstorage.softlayer.net/v1/AUTH_bbc5e0c7-0d10-46a0-a57a-29607ab57956/sdnve-openstack-neutron-plugin/1843/16:16
jeblairkrotscheck: the missing comment, which i have replaced, is about using a list instead of a set16:16
jeblairkrotscheck: "Should this be a list instead of a set (which is unordered) so that the queued messages are still sent in the original order?"16:16
jeblairkrotscheck: as the "_pending" attribute16:17
anteayadaya_k: nope, I was just prompted to download apache_config16:17
anteayadaya_k: on firefox16:17
*** marun has quit IRC16:17
*** skolekonov has quit IRC16:17
daya_kanteaya : yeah that one seems to have slipped through, is that a req to have the apache config?16:18
daya_kanteaya : its a file of size 0, i'll look at removing it from the upload if its not required16:19
krotscheckjeblair: And that gives context to the second comment, excellent.16:19
*** shashankhegde has quit IRC16:20
anteayadaya_k: what repos are you testing?16:20
anteayadaya_k: rabbitmq16:21
daya_kanteaya : meaning what triggers ? openstack/neutron on master, icehouse and havana16:21
daya_kyeah, thats also always been 0, also the sudoers.d file. i left it as is from devstack-gate cleanup, but i havent seen any content in it.16:22
daya_kanteaya : since they are all 0, can i just not upload these?16:23
anteayadaya_k: you have to ask neutron16:24
anteayadaya_k: neutron decides what files they want to see16:24
anteayaI just need to ensure that was is available is browsable16:24
pleia2I have a short connection during the meeting today (chaos!), but I'll be home tomorrow :)16:24
anteayapleia2: yay16:24
anteayapleia2: how are you doing?16:24
pleia2anteaya: doing well, thanks16:25
anteayapleia2: how is flordia16:25
pleia2so hot!16:25
anteayaCanada isn't16:25
anteaya:D16:25
pleia2I hear there has been snow!16:25
daya_kanteaya : they are not on the neutron third party wiki - https://wiki.openstack.org/wiki/NeutronThirdPartyTesting#Log_.26_Test_Results_Filesystem_Layout16:25
anteayaout west yes16:25
* pleia2 nods16:25
anteayadaya_k: good investigation16:25
anteayadaya_k: ensure you have all logs required in the wiki then16:25
* pleia2 goes back to airport things16:26
anteayadaya_k: then link me to another sample16:26
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Make JJB python 3 compatible  https://review.openstack.org/11870316:26
daya_kanteaya : ok, will do, thanks16:26
anteayadaya_k: thank you16:26
openstackgerritA change was merged to openstack-dev/pbr: Only consider tags that look like versions.  https://review.openstack.org/11440316:29
*** nadya_ has quit IRC16:29
*** LinuxJedi has quit IRC16:29
*** LinuxJedi has joined #openstack-infra16:30
*** marcoemorais has quit IRC16:32
*** marcoemorais has joined #openstack-infra16:32
*** penguinRaider has quit IRC16:32
pelix@zaro does https://review.openstack.org/#/c/118312/ look good? PMD publisher16:33
*** MaxV has quit IRC16:33
openstackgerritMikhail S Medvedev proposed a change to openstack-infra/elastic-recheck: Disambiguate filename:<file> from tag:<file> in query guidelines  https://review.openstack.org/12190816:34
*** otherwiseguy has quit IRC16:35
*** otherwiseguy has joined #openstack-infra16:36
*** mpaolino has joined #openstack-infra16:36
clarkbo/16:40
fungiu can haz internets16:40
anteayahey clarkb16:40
* anteaya mind now says clarkb brisket16:41
clarkbfungi: I haz internets16:41
clarkbfungi: its amazing but I somehow managed to work around comcasts wait 5 business days deal16:41
anteayayay internets16:42
*** jgrimm is now known as zz_jgrimm16:42
fungiclarkb: come to north carolina--they don't even make you wait that long to buy a gun ;)16:42
clarkbthe trick is get a modem yourself and have all the cables you need. sign up pay $10 for the install kit that takes 5 business days to arrive then immediately call them with your new account number and set up on your modem16:42
clarkbapparently there is no way to say don't send me the thing I don't need16:43
* anteaya notes to stay away from north carolina16:44
*** denis_makogon has quit IRC16:44
fungianteaya: yeah, i don't recommend it for the most part16:44
fungianteaya: except it contains awesome people like olaph and Shrews16:44
clarkbbest part is all of my packets still go through seattle16:44
anteayafungi: I might reconsider for the awesome people then16:45
anteayaclarkb: they miss it apparently16:45
anteayalike breadcrumbs from hansel and gretel16:45
fungiclarkb: just as well. most of your mail will be going through seattle for a while too16:45
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Support lazy resolving of include yaml tags  https://review.openstack.org/6358016:46
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Retain user specified order of post build scripts  https://review.openstack.org/11817016:47
clarkbfungi: so I don't expect https://review.openstack.org/#/c/120228/ to make it into juno16:48
clarkbfungi: if only because its rootwrap and handlign all the pythons is a little bit of work16:48
clarkbfungi: any thoughts on that? swithing in juno would be sooooo nice though :/16:48
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Retain user specified order of post build scripts  https://review.openstack.org/11817016:48
*** esker has joined #openstack-infra16:49
fungiclarkb: makes sense... might be saner to not support py3k in rootwrap until it can be addressed. that would allow us to press forward with dropping the py33 custom hackery16:49
*** bhuvan has joined #openstack-infra16:49
* anteaya goes for a walk16:49
clarkbfungi: ya thats a thought. would need to check if any of the servers support py3k and depend on rootwrap though16:49
*** esker has quit IRC16:50
*** jistr has quit IRC16:50
jeblairadam_g: in 112134 you say it needs to be coordinated with 115803; 803 is WIP -- what's the story?16:50
jeblairadam_g: oh, it looks like 134 needs to merge first then 803, yeah?16:51
*** mpaolino has quit IRC16:52
*** rhsu has quit IRC16:52
HeOSfolks, merge please https://review.openstack.org/120747 and https://review.openstack.org/117199? :)16:52
clarkbfungi: do we have changes to switch pypy over yet? I suppose that required new images I should check we have those16:52
fungiclarkb: i believe the change hasn't merged to add it to images yet16:53
*** jaypipes has joined #openstack-infra16:53
clarkbfungi: oh right we got busy with puppet16:53
fungiclarkb: though 120906 so at least all the projects should work on trusty for now16:53
*** ihrachyshka has joined #openstack-infra16:53
fungier, though 120906 merged16:53
fungi(your zaqar fix)16:53
fungis/fix/workaround/16:53
clarkbHeOS: 120747 is approved. looks like it may have gotten lost by zuul yesterday? in any case a recheck should send it through the gate and merge if it passes16:54
fungiodd for an event like that to go missing... checking zuul's logs to find out why16:55
fungioh! i bet i know16:56
fungicheck result not new enough, so not considered16:56
fungii guessed wrong16:57
fungihttp://paste.openstack.org/show/112236/16:57
fungichecking gerrit logs now to see if it blew up16:57
clarkbHeOS: for the jeepyb change, will that continue to work with the version of gerrit we have deployed?16:58
rfolcocould somebody help me understand if upstream images have cached git repos for the tested project while running devstack-gate jobs ?16:59
clarkbHeOS: I wonder if jeepyb should check the gerrit version and switch behavior based on that16:59
fungirfolco: yes, they do16:59
jeblairrfolco: yes, but they are updated at the start of the job16:59
*** kgiusti|afk is now known as kgiusti16:59
clarkbHeOS: gerritlib has a get gerrit version call that works on gerrit >=2.5 iirc16:59
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: wrappers: fixed locks for empty collection  https://review.openstack.org/10709116:59
fungirfolco: the instances boot from snapshots with repositories usually no more than a day out of date, and then update the remaining delta when a job runs17:00
*** derekh has quit IRC17:00
rfolcothanks jeblair and fungi... one more: is devstack deployed while the job runs (on prepare_node_devstack) ?17:00
HeOSclarkb, is you version 2.8.4-15-g6dc8444?17:00
*** hashar has quit IRC17:01
fungirfolco: yes, devstack's setup runs at the start of the job, via a set of scripts in the devstack-gate project17:01
fungirfolco: prepare_node_devstack.sh is not run ion the context of a job17:01
clarkbHeOS: yes, but we also have users running 2.417:01
fungirfolco: that is run to pre-cache files and packages devstack might need when nodepool builds images for it17:01
clarkbHeOS: I left my question in a comment on the change17:02
rfolcofungi, I am just trying to understand what I should include in the image, what nodepool does, and what the job devstack-gate does17:02
*** patrickeast has joined #openstack-infra17:02
adam_gjeblair, yea, that is correct17:03
HeOSclarkb, thanks, i will investigate this question tomorrow. :)17:03
*** shashankhegde has joined #openstack-infra17:03
fungirfolco: basically nodepool's prep scripts collect anything which can safely be provided ahead of time which devstack is likely to need and cache those in the snapshot, then devstack itself installs and uses those local caches as long as they're still relevant (for example, if ubuntu package versions on the mirrors haven't changed since they were cached on disk)17:03
clarkbHeOS: thank you for helping us fix this bug17:03
*** harlowja_away is now known as harlowja_17:04
*** sarob_ has joined #openstack-infra17:04
*** esker has joined #openstack-infra17:04
*** sarob_ has quit IRC17:04
clarkbjogo: ok I am looking at uncategorized thing now17:04
HeOSclarkb, :) It's the fun for me. :)17:04
rfolcofungi, thanks for clarifying.17:05
*** esker has quit IRC17:06
clarkbjogo: according to the timestamp on the file on the server it updated ~45 minutes ago17:06
rfolcofungi, something is going wrong with my cached repos, its making confusion with refs for the projects... http://paste.openstack.org/show/112238/17:06
clarkbjogo: maybe you cached an old version and we should set a shorter ttl on that file?17:06
*** dannywilson has quit IRC17:06
clarkbjogo: more specifics would help. let me know if you have them17:07
clarkbof course someone may have fixed it early this morning and I am just derping17:07
*** vigneshvar has joined #openstack-infra17:07
rfolcofungi, L106 fatal: Couldn't find remote ref refs/zuul/master/Z8a8e82072d324931ae960c7c81fa93ba17:07
*** dannywilson has joined #openstack-infra17:07
rfolcofungi, this should be nova's patch ref, not devstack-gate...17:07
fungirfolco: in case it's running in a dependent pipeline, devstack-gate trues to get the zuul_ref for all projects it's using17:08
fungier, tries17:08
fungiokay, so here's what went wrong in gerrit which caused zuul not to enqueue that approved change: http://paste.openstack.org/show/112239/17:08
adam_gjeblair, any chance you might hit https://review.openstack.org/#/c/118700/ while you're looking at ironic things?17:08
*** Ryan_Lane has joined #openstack-infra17:09
clarkbfungi: nice17:09
jrolloh, are we looking at ironic things? /me has some patches17:09
jeblairadam_g: it's 2 changes further down in my list :)17:09
clarkbfungi: so jdbc and externally hosted mysql had a disagreement17:09
*** flaper87 is now known as flaper87|afk17:09
fungii guess we'll be on the lookout for more of those just in case, but probably not much else we should do for now17:10
*** luqas has quit IRC17:10
openstackgerritA change was merged to openstack-infra/config: Move puppetmaster_boostrap to tools dir  https://review.openstack.org/12167717:10
jrolljeblair: if you don't mind adding these to your list :) https://review.openstack.org/#/q/status:open+project:openstack-infra/devstack-gate+branch:master+topic:ironic-agent,n,z17:10
adam_gjeblair, :)17:10
jeblairjroll: my list is generated automatically by gertty :)17:11
*** dannywilson has quit IRC17:11
jrolljeblair: heh, ok, thought you might be on an ironic kick :P17:12
*** dannywilson has joined #openstack-infra17:12
jrollthey are small, I promise! :P17:12
jeblairjroll: they are somewhere near position 100 out of 22517:13
clarkbfungi: I assume jeblair should get to https://review.openstack.org/#/c/118859/ today then we can rebuild images and move pypy as soon as possible17:13
jrollheh17:13
jeblairclarkb: it's right after adam_g's :)17:14
clarkbjeblair: woot17:14
*** rhsu has joined #openstack-infra17:15
fungiclarkb: sure. that switch ought to be fair game. i'll whip up the (very trivial) patch for it17:15
*** tonytan4ever has quit IRC17:16
jeblairfungi: remind me why we have special workers for py3k and why we don't need them for py34?17:17
fungijeblair: primarily because we didn't want to install unofficial packages on our normal precise workers, but also because at the time we were still relying on puppet to install distro-provided packages for things like pip and tox and they were designed to be interpreter-specific17:19
*** HeOS has quit IRC17:20
fungirecall that precise had python 3.2, so we needed a backport ppa for 3.317:20
clarkbalso didn't we use ppa for pypy because precise didn't have it or it was really really old17:20
jeblairfungi: do we do any 'pip install foo' with puppet on slaves?17:21
*** med_` has joined #openstack-infra17:21
fungithe pypy package in precise was also deemed too old to be suitable (extremely buggy preview release apparently)17:21
*** vhoward has left #openstack-infra17:21
*** emagana has joined #openstack-infra17:21
*** med_` is now known as medberry17:21
*** medberry has quit IRC17:21
*** medberry has joined #openstack-infra17:21
*** loki184 has joined #openstack-infra17:22
*** esker has joined #openstack-infra17:22
fungijeblair: if we do, it will install them into the import path of the default python interpreter so having other interpreters present doesn't break that17:22
clarkbI think we can probably collapse the two python unittest slave images17:23
*** loki184 has quit IRC17:23
fungijeblair: and for py34/pypy jobs they ought to be getting all their python libraries via tox in a venv rather than from the system install path17:23
clarkb(is that what jeblair is getting to?)17:23
*** medberry is now known as med_17:23
fungicollapse which two python unit test slave images?17:23
jeblairyeah, just trying to understand what to expect :)17:24
fungiif you mean get rid of the py3k images, yes that's the point of this exercise17:24
jeblairfungi: will 27 still be the default interpreter?17:24
fungijeblair: yes, in fact if you log into one of our bare-trusty nodes you'll see they already have python3.4 installed by default17:24
jeblairfungi, clarkb: and finally, do you want me to aprv that change now?17:24
jeblairhey it's even on my workstation :)17:25
clarkbjeblair: yup. fwiw upstream Python says make python3.x python3 in the default path so that python2 doesn't break17:25
clarkband python2 continues to be `python`17:25
fungithat change is mostly to get the dev headers and also to be explicit about teh fact that we're using 3.4 rather than just relying on it happening to be preinstalled17:25
*** hdd has quit IRC17:25
clarkbarch does not do this and apparently gentoo does something more like /etc/alternatives to switc between them17:25
jeblairaprvd17:25
*** ChuckC has quit IRC17:25
fungiand to pull in pypy and its dev headers too17:25
clarkbjeblair: ty17:26
*** ChuckC has joined #openstack-infra17:26
*** kevinbenton has quit IRC17:27
*** med_ is now known as med17:27
*** lttrl has quit IRC17:27
*** dizquierdo has quit IRC17:27
*** simonmcc has quit IRC17:28
*** zz_dimtruck is now known as dimtruck17:28
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Start running all PyPy jobs on Ubuntu Trusty  https://review.openstack.org/12193117:29
*** lyxus has quit IRC17:29
*** simonmcc has joined #openstack-infra17:29
*** cnesa12 has quit IRC17:29
clarkbjogo: please see comment on 12134317:30
clarkber 12124217:30
jeblairfungi: in 119203 why did you put '.git' on the end of the repo url?17:30
*** emagana has quit IRC17:31
*** med is now known as med_17:31
*** baoli has quit IRC17:31
*** emagana has joined #openstack-infra17:31
*** lyxus has joined #openstack-infra17:32
*** e0ne has joined #openstack-infra17:32
fungijeblair: out of habit when canonicalizing the URLs. i thought .git suffix was typical to denote that it's a bare repo17:32
*** emagana has quit IRC17:32
*** daya_k has quit IRC17:32
*** emagana has joined #openstack-infra17:32
openstackgerritMikhail S Medvedev proposed a change to openstack-infra/elastic-recheck: Disambiguate filename:<file> from tag:<file> in query guidelines  https://review.openstack.org/12190817:32
*** baoli has joined #openstack-infra17:32
*** kevinbenton has joined #openstack-infra17:32
jeblairfungi: otoh, we don't actually recommend anyone do that when cloning from our servers (it's kind of user-unfriendly)17:33
*** daneleblanc has joined #openstack-infra17:33
*** pblaho has quit IRC17:34
*** dane_leblanc has quit IRC17:34
*** yfried has joined #openstack-infra17:34
*** yfried is now known as yfried|afk17:34
fungii'm happy to format git urls in whatever way has the greatest consensus/consistency17:34
*** pblaho has joined #openstack-infra17:35
openstackgerritA change was merged to openstack-infra/config: Add Python 3.4 and PyPy to Ubuntu Trusty slaves  https://review.openstack.org/11885917:35
*** lttrl has joined #openstack-infra17:35
clarkbbknudson: re 116404 I do not see apache logs in the logs collected by the check jobs. Do you know why that is happening?17:35
openstackgerritJeremy Stanley proposed a change to openstack-infra/config: Use git farm for layout job cloning  https://review.openstack.org/11920317:36
*** bdpayne_ has quit IRC17:36
clarkbbknudson: that change should be self testing17:36
jeblairit's clearly not a big deal, i just don't really want people to think that the git farm requires that and is therefore less friendly than github17:36
fungimakes sense17:36
bknudsonclarkb: this is a launchpad bug?17:36
fungiimage and perception are important17:36
clarkbbknudson: no sorry, the change17:36
*** vigneshvar has quit IRC17:36
jeblairsdague: can you review 119332 please?17:37
bknudsonclarkb: I was requested to make this change when I posted a change to devstack to log keystone in apache... let me find it.17:37
bknudsonclarkb: https://review.openstack.org/#/c/116133/ -- see the comment by ianw17:38
clarkbbknudson: not questioning why you wrote it, pointing out it doesn't seem to work as intended and i am not immediately sure why17:39
bknudsonclarkb: ah... let me poke around... I'm not sure what I should be looking at since not familiar with it... but I assume I should find something in the tempest logs now.17:39
clarkbbknudson: in the job logs yes17:40
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Extend trigger_builds functionality  https://review.openstack.org/9105117:40
clarkboh there it is http://logs.openstack.org/04/116404/3/check/check-tempest-dsvm-full/4c995ff/logs/devstack-gate-cleanup-host.txt.gz#_2014-09-10_22_19_57_22417:40
clarkbbknudson: I think it is /var/log/apache on ubuntun and /var/log/httpd on fedora.17:41
* clarkb comments on change17:41
*** ramashri has quit IRC17:41
*** esker has quit IRC17:41
bknudsonclarkb: crap17:41
clarkbbknudson: however the fedora job didn't seem to grab them either17:41
clarkbso maybe other issues there17:41
bknudsonclarkb: so it's got the fedora correct?17:41
clarkber wait no it is apach217:42
clarkbI am confused now17:42
bknudsonclarkb: my system has /var/log/apache217:42
clarkbbknudson: no it didn't get fedora either /me logs into a test box17:42
bknudsonclarkb: and it's running with sudo, but it looks like it has the filename in ''17:42
clarkbbknudson: I will poke around on some test nodes and see if I can get a better answer17:43
bknudsonclarkb: if I can't use * then I'll just put the logs I need17:43
*** bdpayne has joined #openstack-infra17:43
bknudsonclarkb: I assume the code worked before, and the horizon_error.log was saved...17:44
clarkbya I think it did but I don't tend to look at jobs that closely so I could be completely wrong17:44
*** emagana has quit IRC17:45
bknudsonclarkb: this one has horizon_error.txt.gz, so that must have been working: http://logs.openstack.org/12/119512/1/check/check-tempest-dsvm-full/6b7320b/logs/17:45
clarkbbknudson: I think you may be right. the * glob isn't being expanded for some reason17:45
*** emagana has joined #openstack-infra17:46
*** bdpayne_ has joined #openstack-infra17:46
*** tonytan4ever has joined #openstack-infra17:46
*** emagana has quit IRC17:46
*** emagana_ has joined #openstack-infra17:47
*** bdpayne has quit IRC17:48
bknudsonclarkb: looking at the logs generated, I think we're already getting everything due to the changes in devstack.17:48
bknudsonclarkb: so I think https://review.openstack.org/#/c/116404 can be abandoned.17:48
clarkbbknudson: oh so we get them anyways? have a link?17:48
*** jaypipes has quit IRC17:49
*** carl_baldwin has joined #openstack-infra17:49
bknudsonif you look here: http://logs.openstack.org/04/116404/3/check/check-tempest-dsvm-full/4c995ff/logs/ -- it's got screen-key.txt.gz screen-key-access.txt.gz17:49
clarkbbknudson: those are the screen logs though not the apache keystone logs unless I am mistaken17:49
bknudsonclarkb: the point of https://review.openstack.org/#/c/116404 was to pick up the keystone access log.17:49
bknudsonclarkb: the screen logs are tail -f of the apache logs.17:50
clarkbbknudson: oh ok17:50
clarkbthen yes I think that change can be abandoned, unless you prefer to grab te logs directly rather than tail-f17:50
bknudsonso I'm just going to abandon 116404. I didn't realize that would happen.17:50
*** zz_jgrimm is now known as jgrimm17:51
*** tonytan4ever has quit IRC17:51
*** jlibosva has quit IRC17:51
bknudsonclarkb: Sorry, I should have been looking myself to see if the logs were actually getting collected. I just assumed it worked.17:52
clarkbbknudson: np, this is why we have code review :)17:52
sdaguejeblair: sure, so honestly like I said yesterday, I don't see any value in the pg jobs. They are slow and no one is looking into them. So adding more seems.... odd.17:53
clarkbya I hope to stash those graphs somewhere then send mail about it today17:53
*** andreykurilin_ has joined #openstack-infra17:54
*** ihrachyshka has quit IRC17:55
*** slagle_ is now known as slagle17:55
*** vigneshvar has joined #openstack-infra17:56
*** pelix has quit IRC17:56
*** andreykurilin has quit IRC17:56
*** ramashri has joined #openstack-infra17:57
clarkbjeblair: krotscheck should 97521 be abandoned?17:58
clarkbor WIP'd?17:58
*** amuller__ has joined #openstack-infra17:59
* krotscheck is ambivalent. SergeyLukjanov?17:59
krotscheckclarkb: If not enough people want it strongly, It should be abandoned. Otherwise, the people who want it should take ownership.18:00
anteayajeblair: I have added a link to https://review.openstack.org/#/c/121650/ under the config repo split priority item on today's meeting agenda18:00
*** tonytan4ever has joined #openstack-infra18:00
*** johnthetubaguy is now known as zz_johnthetubagu18:01
*** yfried|afk has quit IRC18:02
*** ihrachyshka has joined #openstack-infra18:02
dkranzclarkb: I'm pulling pretty recent log file names from logstash and get File Not Found like with http://logs.openstack.org/81/120781/2/check/check-tempest-dsvm-full/8f480f0/console.html18:03
dkranzclarkb: Is there a known problem with this?18:03
*** amuller_ has quit IRC18:03
fungijogo: oh, since i saw your comments in scrollback about elastic recheck overnight i looked into it this morning18:03
dkranzclarkb: This log file was from 9/1118:03
*** AJaeger has joined #openstack-infra18:04
mtreinishdkranz: that link is just missing a .gz on the end...18:04
clarkbdkranz: what query are you using?18:05
clarkboh you mean logs.o.o failed18:05
clarkbya thats a thing that osloganalyze broke18:05
dkranzmtreinish, clarkb : thanks guys18:05
fungijogo: for some reason it, at one point, started to dogpile cron runs (i think the first of them was hung and holding back the rest) so status.o.o had 512 e-r processes waiting. i killed them all, ran it by hand as the recheck user, it worked fine, and the cron runs since seem to terminate normally18:05
clarkbpreviously apache would serve you the file but now osloganalyse needs to check if the compressed file is available18:05
*** ZZelle_ has joined #openstack-infra18:05
clarkbfungi: thanks. I saw this morning that the files were up to date and figured someone may have fixed it18:06
fungii'll keep an eye out for any recurrences and see if i can narrow down what triggers that, if possible18:07
anteayaI'm looking to confirm openstack/config has no tags (apparently git filter branch does interesting things with tags) I have searched in the log for the message "tag" and can't see any tags in the results: http://git.openstack.org/cgit/openstack-infra/config/log/?qt=grep&q=tag18:07
anteayahow else can I confirm config has no tags?18:07
clarkbanteaya: run `git tag`18:07
anteayakk18:07
clarkbempty output means no tags18:07
anteayaempty output18:07
fungiit should return nothing. you can compare to, for example, python-novaclient which should have lots18:07
anteayathank you18:07
anteayakk18:07
fungialso you can compare it to other projects in cgit's webui. the refs summary normally lists any tags if present18:08
anteayaI had a copy of python-neutronclient handy, and it too has many tags18:08
anteayathanks18:08
*** julim has quit IRC18:08
* anteaya checks the refs in the gui18:08
anteayaI see them, thank you18:09
* anteaya goes back to git filter branch docs18:09
fungianteaya: yeah, cgit appears to omit the "tag" list in the summary if there are none18:09
anteayaI see the same18:09
adam_gclarkb, any chance you can hit this if you get a sec? your +2 from last week fell off at some point: https://review.openstack.org/11213418:10
jeblairanteaya: thanks18:11
*** jp_at_hp has quit IRC18:11
*** ihrachyshka has quit IRC18:12
*** julim has joined #openstack-infra18:12
*** yfried|afk has joined #openstack-infra18:12
anteayajeblair: sure, that patch is mostly at the stage to ensure I have the tests correct, then I need to practice filter branch and figure out where to put the filtered repo so manage-projects can consume it18:13
anteayabut we can save that for meeting if you prefer18:13
clarkbadam_g: done18:14
clarkbanteaya: usually throwing things on github is fine18:14
clarkbanteaya: but any other publicly accessible git repo will work too18:14
*** baoli has quit IRC18:15
anteayaokay I can do that18:15
*** otherwiseguy has quit IRC18:16
openstackgerritAndrew Lazarev proposed a change to openstack/requirements: Bump sahara client to 0.7.2  https://review.openstack.org/12194218:17
*** esker has joined #openstack-infra18:18
fungiclarkb: i have a screen session going as root on nodepool now with all the bare-trusty images updating in separate windows, fyi18:18
Adri2000anyone familiar with the wiki can explain why the level 2 title in there https://wiki.openstack.org/wiki/IRC appear before the first table when it should be before the second table?18:18
clarkbfungi: great18:18
adam_gclarkb, thanks! if you're feeling charitable, this should help finish off our sideways testing until we make it voting: https://review.openstack.org/#/c/118700/18:18
*** hdd has joined #openstack-infra18:19
anteayaAdri2000: hi18:20
Adri2000hi anteaya18:20
anteayaAdri2000: so you want to create a seperate table for non-english irc channel?18:20
anteayachannels18:20
*** yamahata has quit IRC18:20
*** ihrachyshka has joined #openstack-infra18:20
Adri2000anteaya: yes, what do you think?18:20
Adri2000I suppose we'll see more channels than just the french one in the near future18:21
openstackgerritA change was merged to openstack-infra/config: Remove partial mirror from static.o.o  https://review.openstack.org/11946518:21
anteayawell I am not a wiki expert by any means18:21
anteayabut honestly I would just add it to the original table18:21
anteayaand state that it is the french language channel18:21
anteayathen people can be in that channel if they want18:21
anteayasame as they can be in this channel if they want to talk about infrastruture matters18:22
anteayaI think creating a language division will inspire people to create divisions based on other categories18:22
anteayato the extent that the page will become complex and hinder its usefulness18:23
anteayathose are my thoughts18:23
fungirax-ord update failed on an ssh timeout. awesome18:23
*** cdent has quit IRC18:23
*** flaper87|afk is now known as flaper8718:25
openstackgerritA change was merged to openstack-infra/config: Consolidate and rename Ironic jobs  https://review.openstack.org/11213418:26
Adri2000anteaya: well, ok, let's do that for now and maybe reconsider when we have more channels like this (-fr is the first to be added to infra). the idea was to have a list for "topic" channels and a list for "language" channels, that's where I see a difference18:26
*** KanagarajM has quit IRC18:26
anteayathat is your choice18:26
*** pblaho has quit IRC18:28
openstackgerritA change was merged to openstack-infra/config: Set docimpact-group for ceilometer and trove  https://review.openstack.org/11954618:28
*** cnesa12 has joined #openstack-infra18:28
*** prad has joined #openstack-infra18:28
openstackgerritA change was merged to openstack-infra/config: Adding Bandit project to StackForge  https://review.openstack.org/11986518:30
clarkbadam_g: that change just got a note from zuul saying it needs a rebase18:30
adam_gclarkb, oh party, thanks18:30
clarkbbut I was reviewing it :)18:31
*** ihrachyshka has quit IRC18:32
openstackgerritAdam Gandelman proposed a change to openstack-infra/config: Adds support for Ironic parallel Tempest testing  https://review.openstack.org/11870018:35
*** otherwiseguy has joined #openstack-infra18:35
clarkbadam_g: you left merge tags in at least one file :P18:36
sandywalshjeblair: thanks for the +2's!18:36
adam_gclarkb, yeah.. fixing now18:36
jeblairsandywalsh: np :)  thanks for the projects :)18:37
sandywalsh:)18:37
ekarlsophschwartz: you around boss ?18:38
*** e0ne has quit IRC18:38
openstackgerritRicardo Carrillo Cruz proposed a change to openstack-infra/config: Add settings.xml parameter to Jenkins slave class  https://review.openstack.org/12156018:38
*** amuller__ has quit IRC18:38
*** esker has quit IRC18:38
*** AJaeger has quit IRC18:40
openstackgerritA change was merged to openstack-infra/config: Move check-tempest-dsvm-neutron-dvr to check queue  https://review.openstack.org/12060318:40
fungiclarkb: did you see zaro's response to your comment on 88432 (back in may)?18:42
openstackgerritAdam Gandelman proposed a change to openstack-infra/config: Adds support for Ironic parallel Tempest testing  https://review.openstack.org/11870018:42
*** mrmartin has joined #openstack-infra18:43
clarkbfungi: pretty sure jeblair put those overrides in place because it was too noisy18:43
clarkbfungi: that change really confused me and  Ithink the -1 is still appropriate18:43
clarkbwe do gigabytes of logs with gearman at debug level iirc18:43
fungioh, right18:43
openstackgerritA change was merged to openstack-infra/config: Allow puppet to be automatically upgraded  https://review.openstack.org/12120218:44
fungigrr, new project approvals killed most of the running image updates18:45
zarofungi, clarkb : it's been a while, but what i remember is jeblair requested that we turn up gearman logging. don't remember why anymore though.18:45
jeblairfungi: oh sorry18:45
jeblairfungi: why were image updates running?18:45
fungijeblair: refreshing bare-trusty for pypy packages18:46
fungijeblair: to accommodate https://review.openstack.org/12193118:46
fungianyway, not a huge deal. i always forget that new project approvals smack nodepool image updates until i cause them18:47
openstackgerritA change was merged to openstack-infra/config: Revert "Drop 90 servers from rax-dfw"  https://review.openstack.org/12121018:47
anteayafungi: why is that?18:47
*** carl_baldwin has quit IRC18:48
jeblairfungi: well, i'm trying to work through a 2 week review backlog.  i could just not aprv anything, but i'm not sure that's a good idea.18:48
fungiwe iterate over a list of projects to clone, but those projects may not have been added to gerrit between when the list is updated and when puppet runs next18:48
fungijeblair: no! i wasn't attempting to disparage18:48
anteayafungi: ah18:48
jeblairfungi: no, it's okay, really, i can stop reviewing!  ;)18:49
jeblair<jk>18:49
fungiheh18:49
jeblairzaro, clarkb: i think we were trying to get to the bottom of this bug: https://bugs.launchpad.net/openstack-ci/+bug/128437118:50
uvirtbotLaunchpad bug 1284371 in openstack-ci "gate error: Looks like the node went offline during the build. Check the slave log for the details.FATAL: /var/lib/jenkins/logs/slaves/*/slave.log (No such file or directory)" [High,Triaged]18:50
jeblairzaro, clarkb: so we wanted to see when/why/if gearman-plugin was choosing to deregister jobs18:50
krotscheckInfra just got a callout and round of applause here at OpenStack SV :)18:50
anteayakrotscheck: yay18:51
clarkbkrotscheck: oh?18:51
krotscheckYep- William Franklin basically said: Hey, there’s all these people who make sure that you can commit things. Let’s appreciate them a bit :)18:51
*** ociuhandu has quit IRC18:52
SpamapSnice18:52
fungigood ol' bill18:52
jeblairwe also work very hard to make sure people can't commit (some) things  ;)18:53
*** vigneshvar has quit IRC18:53
mrmartin;)18:53
*** cdent has joined #openstack-infra18:53
*** HeOS has joined #openstack-infra18:54
*** nadya__ has joined #openstack-infra18:55
*** signed8bit has joined #openstack-infra18:57
SpamapSjeblair: so true. :)18:58
adam_gclarkb, https://review.openstack.org/#/c/118700/ rebased18:59
jheskethMorning18:59
openstackgerritKhai Do proposed a change to openstack-infra/gearman-plugin: change logging level  https://review.openstack.org/8262718:59
jeblairjhesketh: morning!18:59
anteayamorning jhesketh18:59
jeblairit's about meeting time18:59
anteayajhesketh: well actually more like middle fo the night, is it not?19:00
jheskethanteaya: 5am, not sure what you'd call that19:00
jheskethprobably closer to morning19:00
anteayajhesketh: I could go either way19:00
zaroclarkb: my memory has been jogged.  88432 is actually a no-op really.  gearman is already set to to log in DEBUG.  this change _keeps_ gearman logging in DEBUG mode only after change 82627 merges. jeblair suggested in 82627 (PS2)19:01
*** emagana_ has quit IRC19:03
*** marun_ has quit IRC19:04
dhellmannmordred, lifeless, fungi, sdague, jeblair : considering whether to make a pbr release today (0.11) and cut 1.0 thursday. The changes include a bug fix for the semver processing of tags that don't look like version numbers. thoughts/ http://paste.openstack.org/show/112270/19:04
clarkbdon't we pin pbr to <1.0 in a lot of places?19:05
*** vigneshvar has joined #openstack-infra19:06
*** signed8bit has quit IRC19:06
lifelessdhellmann: I reallllly want a release of pbr done19:06
dhellmannclarkb: good point, but we can fix that because 0.11 and 1.0 would have the same bits in the dist19:07
lifelessclarkb: if we're pinned thats fine19:07
lifelessclarkb: we can go and unpin after 1.0 doesn't break the world19:07
dhellmannlifeless: I do too, but I also don't want things to break this week.19:07
lifelessdhellmann: this is just releasing to the world what CI has been using for a couple months already19:07
lifelessdhellmann: because CI tests with master19:08
dhellmannlifeless: every time we say that, things break19:08
dhellmannlifeless: we do not actually *install* openstack in any of our test jobs19:08
lifelessdhellmann: tripleo does19:08
*** tonytan4ever has quit IRC19:09
lifelessdhellmann: and the pbr integration test does19:09
dhellmanndo those jobs run regularly? really asking, I have no idea how those are hooked up19:09
lifelessdhellmann: tripleo jobs are hooked up on changes to tripleo and ironic branches today19:09
lifelessdhellmann: pbr's integration job is hooked up on changes to pbr19:09
lifelessdhellmann: both pip install things :)19:10
dhellmannyeah, I forgot about that one19:10
lifelessdhellmann: I want to run the pbr integration job much more broadly (or make devstack pip install -e)19:10
dhellmanndevstack does run pip install -e but that doesn't actually build the packages so if we, for example, don't catch that neutron is missing a file they tell us to package then that breaks somewhere other than a neutron job19:11
lifelessdhellmann: actually pip install -e does catch that19:11
lifelessdhellmann: I know because neutrons missing file broke my patches to pbr while their trunk was broken19:11
dhellmannreally, because I thought we had a case this cycle where it didn't?19:12
lifelessdhellmann: we don't run the pbr integration tests in response to neutron changes19:12
lifelessdhellmann: which is how that case got into neutron master19:12
dhellmannthat was a bug in pbr, then?19:12
lifelessno19:12
*** timo__ has joined #openstack-infra19:12
lifelesslet me try explaining again19:13
lifelessthere is a test that catches setup.* issues in openstack19:13
lifelessit only runs on pbr changes19:14
lifelessneutron made a change that broken setup.*19:14
lifelessbecause the test that would have caught it doesn't run on neutron changes, the change landed in neutron19:15
dhellmannthat much I understood. I thought you said pip install -e would have caught that problem.19:15
lifelessthen those tests were broken for everyone else19:15
lifelessanyhow, its moot right?19:16
lifelesswe have tests that caught the neutron setup issue, they just don't run in response to neutron changes19:16
lifelessnot a bug in pbr, possibly a bug in what tests we run when19:16
dhellmannyeah19:16
*** craigbr has quit IRC19:17
*** rlandy has quit IRC19:18
dhellmannlifeless: so what's our mitigation plan if something does break? how do we roll back to the previous version?19:19
openstackgerritMark Vanderwiel proposed a change to openstack-infra/config: Remove unnecessary metadata check  https://review.openstack.org/12195819:19
lifelessdhellmann: we don't. We do a commit to fix (perhaps by undoing everything since 0.10.0) and land that and release 0.12.019:20
*** ramashri has quit IRC19:20
lifelessdhellmann: or possibly, though I haven't tested it, tag the 0.10.0 rev a second time as 0.12.0 and push that19:20
lifelessI'm not at all sure that that would DTRT - but we could manually push such an sdist to pypi19:21
dhellmannyeah19:21
dhellmannI'd like to hear from some others because of the timing. sdague, mordred, jeblair, fungi, clarkb, anyone else?19:22
*** harlowja_ has quit IRC19:22
*** harlowja has joined #openstack-infra19:22
openstackgerritMark Vanderwiel proposed a change to openstack-infra/config: Remove unnecessary metadata check  https://review.openstack.org/12195819:22
fungidhellmann: infra is in its weekly meeting, but will try to discuss after this and tc/project meetings wrap up19:22
clarkbmeeting now, but can chime in after that19:22
jeblairspeaking of timing, we're all in a meeting :)19:22
dhellmannheh, sorry, all19:22
*** yfried|afk has quit IRC19:23
*** yfried|afk has joined #openstack-infra19:23
*** pcrews has quit IRC19:25
*** rkukura has quit IRC19:29
*** rkukura has joined #openstack-infra19:31
dansmithdstufft: clarkb assured me you could unfsck my pip stuff in mere minutes.. you around for some questions? :)19:31
lifelessdansmith: wassup?19:32
lifelessdansmith: I'm not dstufft but I might have some ideas19:32
dansmithlifeless: damn right you're not! :)19:32
*** praneshp has joined #openstack-infra19:32
dansmithso, I have issues running nova unit tests in tox19:32
dansmithwhen I run tox -epy27, I get this immediately:19:33
dansmithpkg_resources.DistributionNotFound: virtualenv>=1.11.219:33
dansmithyet:19:33
dansmithtox==1.7.219:33
dansmithvirtualenv==1.11.619:33
dansmithtried nuking /usr/local/lib/python2.7/dist-packages and starting over,19:33
dansmithgot right back to the same spot19:34
*** sdake has quit IRC19:34
dhellmanndansmith: are you using a pypi mirror?19:35
dansmithdhellmann: not intentionally, that I know of :)19:35
dhellmannheh19:35
*** bdpayne has joined #openstack-infra19:35
dansmithdhellmann: how do I tell?19:35
lifelessdansmith: is there a .tox/py27 directory19:35
dhellmannhmm, do we work with tox 1.7.2 now? I remember an issue that had me pinned to 1.6.119:35
*** koolhead17 has joined #openstack-infra19:36
lifelessdhellmann: I believe so19:36
dansmithlifeless:  yeah, tried nuking that several times, let me do it again19:36
lifelessdansmith: ok so nuke that19:36
dhellmannyeah, that error was different, but that caught my eye19:36
lifelessdansmith: then19:36
dansmithlifeless: yeah, nuked, same deal19:36
dansmithlet me pastebin the whole trace19:36
lifelessdansmith: cat ~/.pip/pip.conf19:36
dansmithI've nuked ~/.pip too19:36
*** koolhead17 has quit IRC19:36
dansmithhttp://pastebin.com/TDFYWeCS19:37
lifelessdansmith: finally set | grep PIP19:37
*** yolanda has quit IRC19:37
dansmithlifeless: nothing in my env19:37
lifelessah ok19:37
lifelessso your tox is bust19:37
lifelessdansmith: try sudo pip install -U tox19:37
*** yolanda has joined #openstack-infra19:37
dansmithlifeless: I've already tried nuking /usr/local/lib/python2.7/dist-packages19:38
dansmithwhich is where tox lives19:38
dansmithand I don't have a system tox packagfe installed19:38
dhellmannhow about: pip install -U tox19:38
lifelessdansmith: so19:38
*** bdpayne_ has quit IRC19:38
dansmithdhellmann: reports up-to-date19:38
lifelessdansmith: what I'm asking is different to that, in principle19:39
dhellmanndansmith: I think you're making this hard on purpose :-)19:39
dansmithwha? :)19:39
lifelessdansmith: so you tried sudo pip install -U tox19:39
dansmithlifeless: http://pastebin.com/HBCt8ZQE19:39
dansmithyes19:39
dhellmannI think lifeless is right, and we need to completely get rid of your tox and then reinstall it19:39
dhellmannhow about: pip uninstall tox && pip install tox19:40
dansmithdone, same error19:40
lifelessdhellmann: I think a larger fire is needed - sudo pip uninstall tox virtualenv py && sudo pip install tox19:40
*** bdpayne has quit IRC19:40
dhellmannactually you might need to run the uninstall step a couple of times if you have some weirdness going on19:40
*** koolhead17 has joined #openstack-infra19:40
dansmithsecond uninstall claims no tox installed19:40
dhellmannremoving dist-packages ought to do most of that19:40
dansmithsame for virtualenv19:41
dansmithdhellmann: right, you saw that I've done that once already, right?19:41
dhellmannand the installing it gets you back to the same error about virtualenv missing?19:41
dansmithyes19:41
dhellmanndansmith: yeah19:41
dansmithafter removing,19:41
dansmithI installed the system's pip,19:41
dansmithpip install -U pip19:41
dansmithwhich gets me a working new pip in /usr/local19:41
dansmiththen pip install tox,19:41
dansmithwhich pulls in virtualenv,19:41
dansmithand then tox -r blows up as above19:42
dhellmannok, I usually use get-pip to install pip but I'm not sure that would make a difference19:42
dansmithwell, I can try it if you want, but this is all I've ever done19:42
*** esker has joined #openstack-infra19:42
*** daneleblanc has quit IRC19:42
dhellmannnah, I don't think that's it19:42
lifelesswhat version of py is installed ?19:42
dansmith(master)% python -V19:43
dansmithPython 2.7.319:43
*** bradjones has quit IRC19:43
dansmithDISTRIB_DESCRIPTION="Ubuntu 12.04.3 LTS"19:43
dhellmanndansmith: no, I think lifeless means literally the package "py"19:43
lifelessdansmith: no, py - should be in pip freeze19:43
dansmithheh, sorry19:43
dansmithpy comes in with tox, fwiw, but:19:43
dansmithpy==1.4.2419:43
lifelessdansmith: right thats why I want to know :)19:43
lifelessso thats shiny and new19:44
lifelessits not in my pip mirror yet19:44
dhellmannyou have newer versions of all of these things than I have19:44
lifelesstry19:44
dansmithorly19:44
lifelesspip install py==1.4.2319:44
dhellmannall 3 of them, fwiw19:44
dansmithso *maybe* I'm not crazy? :)19:44
lifelesspip install tox19:44
lifelessthen see what happens19:44
dhellmanndansmith: you're an openstack developer, so I think you have to give up on not being crazy19:45
lifelesshmm, my mirror must be stale, as 1.4.24 hit pypi on the 5th19:45
dansmithdhellmann: heh19:45
*** rkukura has quit IRC19:45
dansmithlifeless: same deal19:45
dansmithlifeless: it pulled in the same version of virtualenv as reported above, of course19:46
*** e0ne has joined #openstack-infra19:46
*** shashankhegde has quit IRC19:46
openstackgerritK Jonathan Harker proposed a change to openstack-infra/config: Clean up bashate failures  https://review.openstack.org/11894419:46
lifelessdansmith: ok19:46
dansmithoh check it19:47
dansmithvirtualenv 1.11.5 works19:47
lifelessso with py 1.4.23 and your version of virtualenv and tox, tox seems happy to me19:47
dansmithapparently19:47
dansmith1.11.6 not so much19:47
lifeless1.11.6 is fine for me19:47
*** shashankhegde has joined #openstack-infra19:47
lifelessbut whatevs :)19:47
dhellmannweird, that package is months old19:47
dansmithwell, I uninstalled virtualenv and installed the version from one of my other boxes that does work, and now I'm building tox19:47
dansmither, .tox19:48
dstufftare you using a super old version of setuptools19:50
dstufftlike pre 0.6.uhhh24?19:50
dansmithii  python-setuptools                0.6.24-1ubuntu1                             Python Distutils Enhancements (setuptools compatibility)19:50
*** dannywilson has quit IRC19:50
*** esker has quit IRC19:51
*** marun_ has joined #openstack-infra19:51
*** bcourt has joined #openstack-infra19:51
*** dannywilson has joined #openstack-infra19:51
dansmithdstufft: ^19:51
dstufftdansmith: try pip install --no-use-wheel19:52
dstufftit might be 0.6.2819:52
dstufftis this precise19:52
dstufftyou can't install from wheels on old versions of setuptools correctly because it doesn't recognize the wheel metadata19:52
dstufft-> has to go19:52
dansmithdstufft: but you saw that I'm working with virtualenv 1.11.5 and py1.4.23, right?19:52
dansmithdstufft: ah, okay, well, I'm working for now19:52
dansmithwill try the above with fresh packages later19:53
lifelessdansmith: that would make sense19:53
dansmithdstufft: lifeless: dhellmann: thanks19:53
*** maxbit has quit IRC19:53
lifelessdansmith: because there is a wheel of 1.11.619:53
lifelesshmmm there is one of 1.11.5 too though19:53
lifelessah19:53
lifelesspy3 only19:53
lifelessits the wheel thing19:53
*** tonytan4ever has joined #openstack-infra19:53
lifeless1.11.5 has a py3 only wheel, 1.11.6 has a py2.py3 wheel.19:54
lifelessdstufft: dansmith: ^19:54
dansmithah19:54
dansmithmakes sense19:54
dansmithglad I'm not crazy19:54
*** torandu has joined #openstack-infra19:56
*** esker has joined #openstack-infra19:56
dansmithwell, because of this anyway19:56
dansmithclarkb: you called that one pretty good19:56
*** bookwar has quit IRC19:56
*** esker has quit IRC19:57
*** mfer has quit IRC19:57
*** pcrews has joined #openstack-infra19:58
*** e0ne has quit IRC20:00
*** nadya__ has quit IRC20:00
krotscheck….that moment when you’re putting off lunch and going to the bathroom because you have something on the infra agenda, and then time runs out…. [/sadtrombone]20:00
krotscheckAhh well, next week.20:01
anteayakrotscheck: yeah, it hurts doesn't it20:01
*** isviridov_at_met is now known as isviridov20:01
anteaya:(20:01
clarkbkrotscheck: happy to talk about it here if it helps/20:01
clarkbdansmith: :)20:01
*** bookwar has joined #openstack-infra20:01
clarkbdansmith: dstufft is the python packaging magician20:01
dansmithheh20:01
krotscheckanteaya: Not as much as my stomach growling! I’m gonna go do that :D20:01
dansmithsomeday I'll be a magician of something20:01
anteayakrotscheck: yes, eat vegetables!20:01
jeblairkrotscheck: who did you need to have around for that topic?20:02
*** esker has joined #openstack-infra20:02
anteayawell I am curious about where the names are coming from on the storyboard groups20:02
anteayaI don't know who needs to be here for that20:03
openstackgerritKen Giusti proposed a change to openstack-infra/config: Add a test to verify oslo.messaging's AMQP 1.0 messaging protocol support  https://review.openstack.org/11575220:03
*** ramashri has joined #openstack-infra20:03
*** unicell1 has quit IRC20:03
*** e0ne has joined #openstack-infra20:04
*** eharney has quit IRC20:04
*** mestery has quit IRC20:04
*** mestery has joined #openstack-infra20:05
*** weshay has quit IRC20:05
fungimrmartin: i'm looking at bug 1369710 and have confirmed that libmcrypt4 and php5-mcrypt are installed on our bare-trusty nodes in all providers. i'm currently trying to reproduce the failure20:06
uvirtbotLaunchpad bug 1369710 in openstack-ci "openstackid release fails on trusty node" [Medium,In progress] https://launchpad.net/bugs/136971020:06
*** kgiusti has quit IRC20:07
mrmartinfungi: seems strange20:07
mrmartinfungi: and additionally we need to test the openstackid against the trusty php packages, because there was a version change between precise and trusty20:07
clarkbdhellmann: lifeless so for a pbr release. I think it is relatively low risk, and if there are specific areas where we need more coverage we can do that by hand if necessary (but should also add jobs if possible)20:08
*** andreaf has quit IRC20:08
*** esker has quit IRC20:09
*** andreaf has joined #openstack-infra20:09
ianwclarkb: over the next few hours, i will validate a) if debootstrap is needed (probably a merge error) and b) if the permissions need to be set like that in https://review.openstack.org/#/c/102583/11 .  this does not seem like it should be a blocker20:10
*** esker has joined #openstack-infra20:11
bswartzanteaya: ping20:11
anteayahello bswartz20:11
anteayahow can I help?20:11
bswartzanteaya: I sent a request to third-party-requests and it went into a black hole as far as I can tell20:11
jheskethbtw, I have a bunch of zuul changes up for review if people have time :-) https://review.openstack.org/#/q/status:open+project:openstack-infra/zuul+owner:%22Joshua+Hesketh+%253Cjoshua.hesketh%2540rackspace.com%253E%22,n,z20:12
bswartzdid I miss a response somewhere?20:12
bswartzor does it normally take time?20:12
bswartzI'm assuming you're the one that handles them... is that right?20:12
clarkbianw: I don't think debootstrap is needed. let me know what you find out (dib starts with an ubuntu base image that it mounts and chroots into on its own so no deboostrap necessary)20:12
jheskethI'm particularly interested in feedback on 121528 as it relates to recent refactor discussions20:12
clarkbianw: also lifeless may know the answer to that off the top of his head20:12
lifelessianw: debootstrap is only needed for Debian20:13
anteayabswartz: netapp manilla ci? http://lists.openstack.org/pipermail/third-party-requests/2014-September/thread.html20:13
lifelessianw: in the context of dib20:13
bswartzyes that's mine20:13
anteayabswartz: except you won't be using the name mailla20:13
clarkblifeless: oh it is needed ok20:13
anteayabswartz: not a black hole20:13
lifelessclarkb: for *debian*20:13
clarkblifeless: oh not ubuntu20:13
lifelessclarkb: Ubuntu has pre-built images20:13
anteayasent yesterday20:13
clarkblifeless: roger20:13
*** baoli has joined #openstack-infra20:13
bswartzanteaya: where did you send the response? I seem to have missed it20:14
*** JayJ____ has quit IRC20:14
anteayabswartz: please reply to that post with the name of the driver or plugin you are verifying20:14
clarkbok I am going to find the second half of my lunch20:14
anteayabswartz: no reply yet20:14
anteayabswartz: someone gets to requests about once a week20:14
bswartzokay that's what I was wondering20:14
*** JayJ____ has joined #openstack-infra20:14
bswartzI'll add more detail to my request20:14
fungimrmartin: yeah, it's reproducible and my first guess is that it's looking for the wrong php version20:14
bswartzyou want the driver name?20:14
anteayabswartz: I don't yet have permissions to do it myself so I have to wait for someone else to have a minute20:14
anteayabswartz: yes, please add the driver name in a follow up email to that thread20:15
bswartzis there an issue with the name "NetApp Manila CI"? This is to distinguish it from the "NetApp CI" which is currently cinder-specific20:15
mtreinishfungi, clarkb: I'm trying to push a merge commit for a tempest->tempest-lib migration. I'm using: http://git.openstack.org/cgit/openstack/tempest-lib/tree/tools/migrate_from_tempest.sh to add the commit history.20:15
mtreinishthe issue I'm hitting is gerrit rejects the push saying I'm not the committer on the bottom patch20:15
*** hdd has quit IRC20:15
mtreinishdoes that mean I need to update the committer for each commit before I do a migration?20:16
*** berendt has quit IRC20:16
anteayabswartz: there is an issue with using an openstack program name in a ci account name yes20:16
mtreinishWhile that shouldn't be too hard to do, I'm hoping there is a magic way to avoid doing that in the script20:16
fungimtreinish: pushing merge commits is a special permission (mainly to avoid people doing it accidentally from a muddy local branch). what was the reason behind not importing from a filtered copy of the original repo?20:17
anteayabswartz: which we address by using the driver name, which you are going to provide, yes?20:17
bswartzanteaya: okay I'll suggest a different name20:17
zarorcarrillocruz: made a suggestion in 12156020:17
fungimtreinish: also, you'd another special permission to impersonate the committer if they're not all committed by you20:18
anteayabswartz: no, we create the name20:18
bswartzthe driver name's between cinder and manila are the same, so I'll need to add something else to distinguish the 220:18
anteayabswartz: please provide the driver name20:18
bswartzthe driver name is just netapp20:18
mtreinishfungi: that's what the script does, we're doing it piecewise by functionality. Because there is a lot of refactor needed to make somethings work20:18
*** e0ne has quit IRC20:18
bswartzbut the manila netapp driver and the cinder netapp driver are different20:19
*** esker has quit IRC20:19
mtreinishfungi: or at least that's what I attempted to script20:19
fungimtreinish: oh, sounds like maybe the merge commits its complaining about are ones from the previous repo20:19
*** bdpayne has joined #openstack-infra20:20
*** dprince has quit IRC20:20
anteayabswartz: okay please provide the details of how they are different in your email20:20
fungimtreinish: normally we import that when the project is created in gerrit, to avoid all this nastiness. have there been any reviews in the new project yet?20:20
mtreinishfungi: oh, sorry I wasn't clear it's complaing about the commits from the prev repo not being committed by me20:20
mrmartinfungi: looks strange, because the php5-mcrypt package name is the same in trusty http://packages.ubuntu.com/search?keywords=mcrypt&searchon=names&suite=trusty&section=all20:20
mtreinishfungi: there have been a few20:20
mtreinishthis is the first time we're trying to migrate after I moved it into gerrit20:20
fungijeblair: clarkb: ^ suggestions on how to untangle tempest-lib? apparently it has some commits and the desire is to push a merge commit of a filtered branch into the current one20:21
*** bdpayne_ has joined #openstack-infra20:21
mtreinisha filtered branch of another repo...20:22
fungiright20:22
clarkbwe dont combine repos...20:22
mtreinishif it's too crazy I can just make the executive decision and say we can't move the commit history when we do a migration20:22
fungiif there weren't already existing commits in the target repo i'd say just force push the replacement repo into it20:22
clarkbI mean we can but I think keystone was like first and only20:22
jeblairmtreinish: did you talk to us about this before you started?20:23
mtreinishyes, a couple of weeks ago after I saw the neutron incubator thread20:23
mtreinishalthough I'm guessing I wasn't exactly clear with my intended path here20:24
fungithe neutron incubator thread was about all sorts of wrong ideas about recombining git repositories20:24
jeblairmtreinish: a link to that discussion would be nice.  afaik, you prepared a repo for import and we imported it.20:24
*** otherwiseguy has quit IRC20:24
*** bdpayne has quit IRC20:25
*** baoli has quit IRC20:25
*** baoli has joined #openstack-infra20:26
*** mbacchi has quit IRC20:26
*** paul-- has joined #openstack-infra20:26
*** denis_makogon has joined #openstack-infra20:27
mtreinishjeblair: http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2014-08-28.log at 2014-08-28T19:42:0520:27
mtreinishalthough it looks like I misunderstood clarkb20:27
*** radez is now known as radez_g0n320:27
*** otherwiseguy has joined #openstack-infra20:28
jeblairmtreinish: that was on the date you uploaded the change to import tempest-lib.  i believe we all understood you to be asking whether it's okay to filter-branch everything you need into the staging repo so it can be imported.20:30
jeblairand it is.  and you've done that.  and we can't import it a second time.20:30
*** boden has quit IRC20:32
anteayabswartz: if you want to merge the two systems why not use the one account to test both drivers?20:32
mtreinishjeblair: yeah it's my mistake for not being clear and mistunderstanding. Ok, I guess we'll just cp code and generate a list of sha1s for commit messages or something like that instead20:32
fungiin retrospect it sounds like mtreinish wanted to filter branch multiple times in stages and merge in over the course of a piecemeal migration from tempest to tempest-lib20:32
*** boden has joined #openstack-infra20:32
mtreinishfungi: yeah that's what I meant20:32
fungirather than all in one shot20:32
fungiwhich won't really work. that gets extremely messy20:32
jeblairthat's a shame. because you're loosing the history of your code.20:32
*** dannywilson has quit IRC20:33
*** julim has quit IRC20:34
bswartzanteaya: since the systems are separate hardware, separate software, I don't want issues with one system to cause you guys to disable both20:34
*** dalgaaf__ has quit IRC20:34
*** dannywilson has joined #openstack-infra20:34
mtreinishjeblair: yeah it is. But, the history will still be there in the tempest repo though20:34
anteayabswartz: your call20:35
anteayabswartz: what is NAS?20:35
*** isviridov is now known as isviridov_away20:35
bswartzNetwork Attached Storage -- it's a shorthand for shared file systems20:35
bswartzsuch as NFS and CIFS20:35
fungi(and afs!)20:35
bswartzfungi: yse20:36
* fungi couldn't resist20:36
anteayamore acronyms sigh20:36
ianwclarkb / lifeless : cool, from a quick manual grep of git log -p, elements/debian/root.d/08-debootstrap is the only place debootstrap has ever seemed to be used.  so we have to conclude that whatever yolanda was seeing requiring debootstrap was a bug that needs fixing elsewhere20:36
*** dalgaaf__ has joined #openstack-infra20:36
anteayawell the email didnt' give me what I was hoping for in terms of understanding the difference between the two drivers, but I'll do my best20:36
*** boden has quit IRC20:36
anteayabswartz: might take a week, in the mean time you can use your personal gerrit account to listen to the gerrit stream and exercise itself20:37
bswartzanteaya: they're drivers for the same product but for different openstack projects20:37
anteayabswartz: so it is fit when your account is available20:37
anteayabswartz: right, that much I got20:37
bswartzanteaya: sure, that's what I've been doing -- I didn't know whether requests were typically answered within a day or within a month so I wanted to ask20:38
bswartznow I've got my answer20:38
fungimrmartin: the php5-mcrypt package seems to install /usr/lib/php5/20121212/mcrypt.so but maybe it's looking in the wrong place?20:38
*** sballe has joined #openstack-infra20:38
anteayabswartz: glad to hear you aren't blocked by the turn around time20:38
anteayabswartz: hanging out in this channel is a good idea, having you available to answer netapp concerns with get you ci diabled less frequently20:39
anteayabswartz: also you learn useful stuff20:39
anteayaon a good day20:39
mrmartinfungi: I'll try to do a clean trusty based deployment in vagrant, and let's see what happens20:39
bswartzanteaya: you may want to document what the typical response time is in this guide: http://ci.openstack.org/third_party.html#requesting-a-service-account20:39
bswartzto avoid more impatient people like me coming in here and shouting20:39
anteayabswartz: do offer a patch20:40
anteayabswartz: I can teach you how20:40
bswartzI've been lurking in this channel for a few weeks now20:40
anteayabswartz: great20:40
anteayabswartz: made it to any third party meetings yet?20:40
bswartzoh God not more meetings20:40
anteayathats what I say20:40
bswartzlol20:40
anteayain one now20:40
bswartzwhen is the infra weekly meeting20:40
bswartzI can put it on my calendar at least20:41
anteaya1900 UTC20:41
bswartztuesdays?20:41
anteayahttps://wiki.openstack.org/wiki/Meetings20:41
anteayayes20:41
anteayaand please attend the third party meetings20:41
anteayathird party business should be discussed at the third party meeting prior to having an agenda item on the infra meeting20:42
anteayait helps to focus the item by the time infra hears about it20:42
anteayasince usually the first question is, what do the other third party folks think?20:42
bswartzokay looks like I just missed it20:42
anteayayes20:43
anteayatc meeting is 3/4 over and project is next20:43
fungibswartz: speaking of calendars, we agreed at today's infra team meeting to do the manila project moves from stackforge to openstack namespace at 20:30 utc this friday. i'll send a message to the -dev ml with details shortly, but might as well run it by you while you're in here20:43
bswartzfungi: sounds excellent -- I never found out why it was delayed though20:43
fungibswartz: delayed?20:44
*** marun_ has quit IRC20:44
fungibswartz: oh, as in why we waited until now. we usually try to schedule it during quiet periods since it requires shutting down zuul and gerrit for a few minutes20:44
bswartzI was told by vponomaryov that you planned to do it 2 weeks ago20:44
anteayawe asked for manilla representation each time it was presented on the infra agenda20:44
bswartzah20:44
anteayanoone attended20:45
bswartzokay so I really should be there20:45
anteayathat would be great20:45
bswartzand I have a recurring calendar conflict I'll need to sort out20:45
*** vigneshvar has quit IRC20:45
fungibswartz: we discussed scheduling for it in the meeting once i had confirmed with you and vponomaryov that you were okay with it whenever, but the feature freeze rush had the gate fairly busy and most of us weren't around to do it last weekend20:45
*** koolhead17 has quit IRC20:46
bswartzwe're getting close to rc1 now...20:46
*** cdent has quit IRC20:46
bswartzI suppose there's never a great time20:46
bswartzjust have to bite the bullet20:46
fungiyeah, there's never a great time for these sorts of things unless projects start getting their incubations accepted at the start of the cycle rather than near the end20:46
*** signed8bit has joined #openstack-infra20:46
fungior new program applications in this case20:46
fungibut those always seem to happen closer to release time20:47
* krotscheck has had his vegetables20:48
krotscheckanteaya: So, the history of the groups thing is, as I gathered, the following.20:48
krotscheckFirst, groups were added to jeepyb and review.projects.yaml (by mordred I think) in anticipation of them being a storyboard thing.20:49
krotscheckAt that point, the term was named “group"20:49
krotscheckThen, at some point later,  it was used to assist with launchpad things which I’m not entirely clear on.20:50
krotscheckDuring all this time, bitrot may or may not have happened.20:50
*** adalbas has quit IRC20:50
*** signed8bit has quit IRC20:50
krotscheckThen we actually came around and designed/built project groups, and it turns out that we need a single project to be able to join more than one group.20:50
krotscheck(And by we I mean StoryBoard)20:51
*** signed8bit has joined #openstack-infra20:51
krotscheckThat’s where my two patches then came into play, modifying “group” to “groups"20:51
krotscheckHowever I did this without knowing what the launchpad thing was that happened since the original implementation.20:51
* krotscheck is still not quite certain on what that is.20:52
*** dkehnx1 has joined #openstack-infra20:52
*** marcoemorais has quit IRC20:52
krotscheckSo now the discussion you started has split into two- What should we name groups (oh look, bitrot), and how should we (if at all) distinguis between launchpad groupy things and storyboard groupy things.20:52
*** marcoemorais has joined #openstack-infra20:52
krotscheckAnd they lived happily ever after.20:52
*** marcoemorais1 has joined #openstack-infra20:54
*** craigbr has joined #openstack-infra20:54
*** SumitNaiksatam has joined #openstack-infra20:55
*** mwagner_lap has quit IRC20:56
*** andreykurilin_ has quit IRC20:57
fungikrotscheck: minor historical edit there20:57
*** marcoemorais has quit IRC20:58
fungikrotscheck: we had a "launchpad" parameter we were using to track what we now call "group" in that file20:58
fungiit was renamed in anticipation of storyboard's group feature, and in an effort to cease advertising lp in confusing ways in our config20:58
*** e0ne has joined #openstack-infra20:58
krotscheckThat makes sense.20:58
*** gokrokve has joined #openstack-infra20:58
fungibut it did already exist in some form20:59
krotscheckOk, so the question is, do the modifications I made in jeepyb and config address the feature which that supports?20:59
*** groknix has joined #openstack-infra20:59
*** bcourt has quit IRC20:59
clarkbkrotscheck: which modifications?21:00
*** tonytan4ever has quit IRC21:00
*** signed8bit has quit IRC21:00
krotscheckclarkb: https://review.openstack.org/#/c/111814/ and https://review.openstack.org/#/c/111815/21:00
ianwjeblair / clarkb : in https://review.openstack.org/#/c/102583 (Add pre-requisites needed for diskimage-builder) between patchset 4 & 5 "file { '/var/lib/nodepool':" changed to "file {'/opt/nodepool".  i think it is just a typo21:01
*** signed8bit has joined #openstack-infra21:01
*** marcoemorais1 has quit IRC21:02
*** marcoemorais has joined #openstack-infra21:02
*** dkliban is now known as dkliban_afk21:02
*** marcoemorais has quit IRC21:02
*** marcoemorais has joined #openstack-infra21:03
*** dustins has quit IRC21:03
*** melwitt has joined #openstack-infra21:03
openstackgerritBen Swartzlander proposed a change to openstack-infra/config: Document typical response time for third-party-requests is one week  https://review.openstack.org/12198021:03
clarkbkrotscheck: that seems like jsut a syntactical change... we are just adding some 's's ?21:03
krotscheckclarkb: It does support multiple groups now, so there’s a loop there.21:03
clarkbkrotscheck: I think so, if yo ujust feed it a list in yaml you ge ta list in python21:04
krotscheckEverything else is syntactical21:04
*** bswartz has quit IRC21:04
*** otherwiseguy has quit IRC21:04
krotscheckclarkb: I just don’t want the addition of that loop to break things elsewhere, which jeblair suggested.21:05
krotscheckAgain, I don’t know what’s going on.21:05
clarkbthat loop?21:05
clarkbthe loop in update blueprint?21:06
krotscheckclarkb: Does the addition of “hey we’re looping over several groups rather than one” break things in jeepyb21:06
krotscheckOr  in other things.21:06
*** ociuhandu has joined #openstack-infra21:06
clarkbya ok that loop. so to me that should be the only change we are making...21:06
*** signed8bit is now known as signed8bit_ZZZzz21:06
clarkbbut this is fine too21:07
clarkbits just a bit harder to understand the problem we are solving21:07
*** aysyanne has quit IRC21:07
openstackgerritIan Wienand proposed a change to openstack-infra/config: Add pre-requisites needed for diskimage-builder  https://review.openstack.org/10258321:08
anteayakrotscheck: so here is what I am hearing, there was a motivation to do a thing and there was already a thing and the thing that is being done needs to be different from the thing that was21:09
clarkbianw: /var/lib/nodepool is tpyically an ok location for images, except / is really tiny on our nodes and we mount the ephemeral drive on /opt21:09
clarkbianw: maybe we can have a /opt/dib/ or /opt/nodepool_dib or similar? /me double checks nodepool node21:09
anteayakrotscheck: and noone seems to know where the actual list of proposed groups came from like openstack-api-site21:10
krotscheckanteaya: Plus bitrot.21:10
anteayaand group python-cliff21:10
krotscheckanteaya: Well, it sounds like that came from the old ‘launchpad’ parameter, what’s the history of that?21:10
anteayaplus bitrot21:10
clarkbianw: nevermind its a good thing I double checked. nodepool / is giant21:10
clarkbapparently it hasn't been rebuilt21:10
anteayaand is there a link to the old launchpad parameter21:10
fungianteaya: many (most?) of those are the lp project names corresponding to those git repositories21:10
anteayaso python-cliff was an lp project name?21:11
fungianteaya: or possibly an error we missed in code review21:11
anteayathat is what I am trying to find out21:11
ianwclarkb: ahh, that makes sense as to how that came to be , i'm guessing modred put it on /opt and chose the logical name, except that's where the checkout is done21:11
*** marcusvrn has quit IRC21:11
anteayasince i believe cliff is in oslo now21:11
fungianteaya: nope, it's real... https://bugs.launchpad.net/python-cliff21:11
anteayaand should be in the oslo group21:11
clarkbianw: however if we use /opt then we will get what we want if/when we rebuild that host21:11
fungianteaya: that seems to be the bug tracker in lp for openstack/cliff21:11
clarkbianw: probably worth setting to /opt/something now just to avoid future pain21:12
dhellmannanteaya: cliff was taken on lp, so I used python-cliff21:12
dhellmannsame for stevedore21:12
anteayahttp://git.openstack.org/cgit/openstack/governance/tree/reference/programs.yaml#n16621:12
ianwclarkb: mordred did make the comment "Using the ephemeral partition seems perfect for this as we can always rebuild an image so they don't have to stick around say after a nodepool host rebuild." (i pasted that into the commit log)21:12
anteayadhellmann: so you want the storyboard group for those to be cliff and stevedore or oslo?21:12
anteayadhellmann: because in my mind it should be olso21:13
clarkbianw: ya so I think we should put the path under /opt21:13
jeblairkrotscheck: btw, the multiple group thing might actually solva e problem that dhellmann and ttx have been having with jeepyb finding oslo blueprints in launchpad :)21:13
anteayadhellmann: context https://review.openstack.org/#/c/111815/3/modules/openstack_project/files/review.projects.yaml21:13
clarkband update the commit message21:13
fungianteaya: the group (formerly launchpad) option in the jeepyb projects.yaml file presently serves two purposes... for identifying when the name of a project is different on lp vs in git/gerrit, and for multiple git repositories sharing one lp project name (effectively different manifestations of the same need)21:13
krotscheckjeblair: yay unexpected fixes!21:13
anteayafungi: okay21:14
ianwclarkb: alright, lucky revision 13 coming up :)21:14
dhellmannanteaya: we have an oslo project group in launchpad, and separate projects for each library. I would like storyboard to be the same, I think, but let me think about it after this meeting is done.21:14
anteayadhellmann: kk21:14
jeblairdhellmann: that's the design in storyboard -- projects can be members of multiple groups21:14
jeblairdhellmann: so in sb, individual projects will be members of both oslo and openstack21:15
clarkbianw: and commented on the change direcntly21:15
anteayafungi: that is in jeepyb and launchpad this is for review.projects.yaml and storyboard21:15
anteayafungi: isn't now the time to make a change if a change needs to be made?21:15
fungianteaya: review.projects.yaml is jeepyb21:15
dhellmannjeblair, anteaya : ok, that sounds like what we want then21:15
fungianteaya: i never disputed the need to change things, just trying to provide historical reference21:16
anteayafungi: you mean that file is consumed by jeepyb?21:16
anteayafungi: ah okay21:16
dhellmannjeblair, mordred, fungi : if you have reasons for me not to do a pbr release in the morning, let me know, otherwise I'll go ahead and cut 0.1121:16
fungianteaya: it's why i called it "jeepyb's projects.yaml" (to differentiate it from jjb's projects.yaml which we also have)21:16
*** hashar has joined #openstack-infra21:17
*** juzuluag has quit IRC21:17
fungidhellmann: nothing i know of, unless the metadata 2.0 thing melwitt broughth up needs critical attention first?21:17
anteayafungi: I think I may be confused I am referring to https://review.openstack.org/#/c/111815/3/modules/openstack_project/files/review.projects.yaml21:17
anteayafungi: which I don't have in my head as jeepyb's projects.yaml21:17
fungianteaya: yep. that's jeepyb's projects.yaml21:17
anteayaah so that is a failing on my end21:18
krotscheckmordred is currently leading a panel.21:18
anteayaI have not heretofore refered to it as such21:18
anteayakrotscheck: how is he doing?21:18
fungianteaya: jeepyb is what consumes it. it's used by manage_projects.py and update_bug.py21:18
krotscheckanteaya: He’s doing his best to get his panelists to call each others names.21:18
anteayafungi: ah, I shall endevour to remember that, thanks21:18
dhellmannfungi: I'm not sure which bug report that is?21:18
anteayakrotscheck: there's a fight in the making21:19
anteayakrotscheck: do keep score21:19
krotscheckRight now it’s 0-0-0-021:19
fungianteaya: it's now starting to be increasingly used by more things, i was just trying not to let it be confused with the projects.yaml in our jjb configuration21:19
anteayakrotscheck: ha ha ha21:19
anteayafungi: I will go looking to make sure I know which file you are referencing21:19
fungianteaya: without typing a mile-long pathname to differentiate them anyway ;)21:20
*** mmedvede has quit IRC21:20
anteayaright21:20
melwittdhellmann, fungi: this is the bug report https://bugs.launchpad.net/python-novaclient/+bug/136917921:20
uvirtbotLaunchpad bug 1369179 in pbr "nova list crashes on startup" [Undecided,In progress]21:20
fungidhellmann: bug 136917921:20
uvirtbotLaunchpad bug 1369179 in pbr "nova list crashes on startup" [Undecided,In progress] https://launchpad.net/bugs/136917921:20
fungihah, melwitt beat me to it21:20
*** dannywilson has quit IRC21:21
anteayafungi: so this is jjb projects.yaml http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/projects.yaml21:21
*** hashar has quit IRC21:21
anteayaand review.projects.yaml is jeepyb's projects.yaml21:21
anteayaokay21:21
melwittfungi: whoops :)21:21
dhellmannmelwitt: is that only on os x or is it only with a particular version of setuptools?21:21
anteayaI had just called them projects.yaml and review.projects.yaml21:22
openstackgerritIan Wienand proposed a change to openstack-infra/config: Add pre-requisites needed for diskimage-builder  https://review.openstack.org/10258321:22
anteayamelwitt: no no, if you can beat fungi to a url you get a cookie21:22
melwittdhellmann: it's not particular to os x. the conditions for it happening is if novaclient gets installed under dist-packages I see it has METADATA file instead of PKG-INFO file *and* if git isn't installed. otherwise pbr will use git to find the version21:22
jeblairsdague: can you review 116129 please?21:22
* anteaya gives melwit a cookie21:22
melwitthaha, thanks anteaya!21:23
anteaya:D21:23
*** miqui has quit IRC21:23
dhellmannmelwitt: Can you tell me which version of setuptools you have installed?21:23
melwittdhellmann: I was able to get the same behavior using a vagrant virtualbox vm running ubuntu/precise64 as it comes without git installed21:23
melwittdhellmann: checking...21:23
*** eharney has joined #openstack-infra21:24
clarkbianw: thank you21:24
clarkbjeblair: I believe 102583 is now ready to have your -2 removed and be rereviewed21:24
jeblairclarkb: +=421:25
*** mrmartin has quit IRC21:25
ianwclarkb: images-dir in https://review.openstack.org/#/c/88479/41/modules/openstack_project/templates/nodepool/nodepool.yaml.erb -- is that a symlink?  hunting to see where it might be setup21:25
jeblairclarkb: is the child change ready for review?21:25
jeblairsounds like no :)21:26
*** gokrokve has quit IRC21:26
*** gokrokve has joined #openstack-infra21:27
melwittdhellmann: on the precise vm it's 0.6c11. I'm still trying to figure out how to find the version on mac os x :|21:28
dhellmannmelwitt: "pip freeze | grep setuptools" should say, I think21:28
jeblairsdague: would be nice for you to revisit 117637 too21:29
*** flaper87 is now known as flaper87|afk21:29
clarkbjeblair: the child change should probably be grabbed and tested before further review since it may have rotted21:29
ianwclarkb: ok, so /etc/nodepool/elements is setup in https://review.openstack.org/#/c/88479/41/modules/openstack_project/manifests/nodepool_prod.pp ; seems that should probably symlink /etc/nodepool/images to /opt/nodepool_dib ... or change image-dir21:30
*** SumitNaiksatam has quit IRC21:30
jeblairclarkb: but also it may need some thoughts about paths which i think ianw was getting at21:30
jeblairianw: would prefer changing image-dir21:30
*** FallenPegasus has quit IRC21:30
clarkbianw: I think we shoudl just change imagedir21:30
*** mestery has quit IRC21:31
jeblairanteaya: did someone disable vmware nsx ci?21:31
jeblairanteaya: because they voted on a devstack-gate change21:32
anteayaI will check the lastest21:32
dhellmannmelwitt: I wonder if we should just fall back to asking setuptools for a version instead of writing that loop ourselves21:32
*** mestery has joined #openstack-infra21:32
melwittdhellmann: I don't get output for setuptools on mac when I show pip freeze. I don't typically run anything of note python on my mac, I only did it to repro the issue in the bug report... so I'm not sure how I have setuptools here21:32
jeblairanteaya: https://review.openstack.org/#/c/120817/21:32
dhellmannmelwitt: you might be getting it as a temporary drive-by when pbr runs. Let's move to #openstack-oslo to free up this channel for the infra team21:32
anteayajeblair: fungi re-enabled them: http://lists.openstack.org/pipermail/third-party-announce/2014-September/000012.html21:33
melwittdhellmann: are you referring to querying pkg_resources (is that the same as asking setuptools)?21:33
jeblairanteaya: this was since then (only yesterday)21:33
*** sballe has quit IRC21:33
*** dannywilson has joined #openstack-infra21:33
ianwclarkb: ahh, yolanda's comment in https://review.openstack.org/#/c/88479 says she found a problem in devstack nodes for Debian; that explains why she was requiring debootstrap.  i think she was working on some hp debian derivative support, so that makes sense21:33
clarkbmelwitt: pip freeze never outputs the setuptools version. try pip list21:33
anteayajeblair: ah then not that I know of21:33
melwittthanks clarkb21:33
anteayajeblair: do you want them not to vote on devstack-gate?21:33
anteayasalv-orlando: ping21:33
clarkbmelwitt: its one of those things that just is apparently and it can't be changed for raisins :(21:34
clarkbianw: that makes sense21:34
melwittclarkb: hehe, noted!21:34
fungianteaya: rhsu seems to possible be around again now too21:34
jeblairanteaya: no third party systems gets to vote on projects unless they are invited to.  infra has invited no one to vote on its projects.21:34
fungier, possibly21:34
anteayajeblair: okay thanks21:35
rhsuanteaya: hi im here21:35
anteayarhsu: better check your system21:35
anteayarhsu: you are voting on devstack-gate and you have not been invited to do so: https://review.openstack.org/#/c/120817/21:35
anteayarhsu: so either disable your system or remove devstack-gate whichever is faster21:36
anteayathen we can discuss21:36
rhsuoops looks like that's the neutron CI21:36
rhsulet me try to get in touch with salv-orlando21:36
*** lttrl has quit IRC21:36
jeblairanteaya: i have removed 'vmware nsx ci' from https://review.openstack.org/#/admin/groups/91,members21:36
jeblairsalv-orlando: ^21:37
jeblairfeel free to add them back when it's configured to vote on only the projects it has been invited to vote on21:37
anteayarhsu: jeblair is faster than you21:37
*** dkranz has quit IRC21:37
rhsunooo haha21:37
anteayarhsu: you now have time to figure out what is going on and fix ti21:37
*** mflobo has quit IRC21:38
anteayajeblair: thanks21:39
*** morganfainberg is now known as CaptainMorgan21:39
*** CaptainMorgan is now known as morganfainberg21:39
rhsuanteaya/jeblair: is there any way you can unblock vmware nsx ci posting on Nova? salv-orlando and i are running separate CI systems under the same account.21:39
anteayait can post comments but it can't vote21:40
rhsuhmm i see21:40
jeblairrhsu: not currently no, sorry; the acls are global.21:41
*** lttrl has joined #openstack-infra21:41
jeblairmtreinish: can you look at 121600 ?21:41
rhsuanteaya/jeblair: got it. i'll try to get hold of salv-orlando to see if he can address the issue.21:42
anteayarhsu: thanks21:42
anteayajeblair fungi so mestery agrees vArmour is having too many problems to retain voting status, so if I remove them from https://review.openstack.org/#/admin/groups/91,members they remain in https://review.openstack.org/#/admin/groups/270,members correct?21:43
fungianteaya: yes21:44
anteayaso just select teh check box in the gui and click the delete button?21:44
fungianteaya: that's it, yep!21:44
anteayak21:44
anteayathanks21:44
*** denis_makogon has quit IRC21:45
anteayarhsu: you probably should adjust your status on your wikipage and the thirdpartysystems page21:45
rhsuanteaya: will do!21:47
anteayathanks21:48
*** cnesa12 has quit IRC21:48
*** stevemar has quit IRC21:49
mtreinishjeblair: comment left21:50
mtreinishbut I'm fine with doing that as a first step21:50
*** pcm_ has quit IRC21:51
*** torandu has quit IRC21:51
jeblairmtreinish: haha, any chance you can re-paste me the number of that change? :)21:52
* jeblair wishes for tagged command queuing for review requests :)21:52
jeblairah found it: 12160021:52
jeblairmtreinish: nm21:52
anteayaSpamapS: would you like to lose this branch prior to the stackforge import? https://github.com/SpamapS/tripleo-ansible/tree/fiddling21:53
fungiclarkb: oops! Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to parse template unattended_upgrades/50unattended-upgrades.erb: Filepath: /opt/config/production/modules/unattended_upgrades/templates/50unattended-upgrades.erb Line: 8 Detail: undefined method `each' for "Puppetlabs:precise":String...21:53
clarkbooh bitrot21:54
*** e0ne has quit IRC21:54
ianwanteaya: is xenserver voting.  it's a minor problem, but i think it left a -1 vote on https://review.openstack.org/#/c/104125/ and then when i re-ran it, it was not removed21:54
clarkbgive me a couple minutes and I can fix it21:54
* anteaya looks at xenserver ci21:54
fungiclarkb: cool--thanks! ran into that debugging why stackforge/bandit wasn't getting a git repo created (which in turn was repeatedly tanking my image updates)21:55
*** JayJ____ has quit IRC21:55
fungii'm going to go grab some dinner before resuming, i think21:55
anteayaianw: it is in the voting third party group: https://review.openstack.org/#/admin/groups/91,members21:55
*** ChuckC has quit IRC21:56
anteayaianw: you did citrix recheck21:56
anteayayour patch says the syntax is xenserver: recheck21:56
*** bknudson has quit IRC21:56
salv-orlandorhsu, anteaya, jeblair: what I have done wrong this time?21:56
anteayathough recheck should fire jenkins plus your ci21:56
anteayasalv-orlando: well somehow your ci got voting on devstack-gate21:57
anteayasalv-orlando: and jeblair has no ability to tolerate that21:57
anteayasalv-orlando: so you are in comment only status at the moment21:57
clarkbfungi: oh wait I think I gave it the wrong type. that is simpler than I thought21:58
salv-orlandoanteaya: I’ve seen that. I think it was the right decision to make unvoting. But rhsu has a reliable CI system for nova. I don’t think voting on devstack-gate will ever happen again. I will provide more info as soon as I collect them from our CI server, but I’m fairly positive it has been a one-off triggered by a test which should have not picked a devstack-gate patch.21:58
openstackgerritClark Boylan proposed a change to openstack-infra/config: Unattended upgrades erb expects list of origins  https://review.openstack.org/12200121:59
anteayajeblair: can I make them voting again? your call here21:59
clarkbthat should do it21:59
anteayasalv-orlando: if you are incorrect you will just move to non-voting again, if jeblair doesn't disable you22:00
fungiclarkb: lgtm. heading out for a while, back later22:00
salv-orlandoanteaya: if I am not correct I will be happily sign up for hanging22:00
anteayasalv-orlando: I will take a chance on you22:01
anteayabut keep in mind jeblair may hang you22:01
anteayasalv-orlando: don't make a fool out of me22:01
SpamapSanteaya: oh maybe we should drop _most_ branches22:01
SpamapSanteaya: let me audit those22:01
* anteaya goes to move vmware nsx ci to voting22:01
*** kmartin has quit IRC22:01
anteayaSpamapS: thanks22:01
openstackgerritJordan Tardif proposed a change to openstack/requirements: Require requests module less than 2.4.0  https://review.openstack.org/12200222:02
*** marun_ has joined #openstack-infra22:02
anteayasalv-orlando: you are voting again, go forth and sin no more22:03
*** boden has joined #openstack-infra22:03
lifelessdstufft: hi, could you perhaps join #openstack-oslo? got a detective issue with the origin of 'METADATA' files in setuptools22:03
anteayaSpamapS: I only saw master and fiddling branches22:04
*** marcoemorais has quit IRC22:05
anteayaSpamapS: and given the nature of the name figured it was a candiate for deletion22:05
*** marcoemorais has joined #openstack-infra22:06
SpamapSanteaya: indeed, the rest can stay. :)22:06
SpamapSmeaning.. master. :)22:06
*** marcoemorais has quit IRC22:06
mattoliverauMorning22:06
*** marcoemorais has joined #openstack-infra22:06
*** marun_ has quit IRC22:07
anteayaSpamapS: awesome thank you22:07
anteayamorning mattoliverau22:07
anteayaclarkb: I'm trying to find a gerrit query to find patches ci's vote on22:08
*** boden has quit IRC22:08
anteayaclarkb: so far project:openstack-infra/devstack-gate label:Verified=-1, NOT jenkins is not returning any results22:08
anteayaoh well the -1 from vmware has been removed: https://review.openstack.org/#/c/120817/22:09
*** atiwari has quit IRC22:09
*** r-daneel__ has quit IRC22:09
salv-orlandoanteaya: there will be no sin. I’m 100% positive that the vote left on devstack-gate was a one off due to an experiment concerning the CI we’re developing for stackforge/congress. I have instructed to not post anything for those jobs until they are reliable. On a further not I am actually exploring alternatives to posting votes to gerrit.22:11
*** nelsnelson has quit IRC22:11
openstackgerritA change was merged to openstack-infra/gear: document how to configure gearman logging  https://review.openstack.org/9378422:12
*** dmsimard is now known as dmsimard_away22:12
anteayasalv-orlando: thanks22:12
anteayasalv-orlando: keep us informed of what you are considering22:13
clarkbanteaya: label:verified=-1,jenkins is how you search for jenkins22:13
clarkband the way to negate that is NOT label:verified=-1,jenkins iirc22:14
*** doug-fish has quit IRC22:14
*** doug-fish1 has joined #openstack-infra22:14
*** otherwiseguy has joined #openstack-infra22:14
anteayaclarkb: yes but that only gets you the +1 and 0 votes from jenkins, i tried it22:15
anteayaI want any vote that is not jenkins22:15
clarkbanteaya: gerrit may not have a way of expressing this then22:15
anteayaverfied vote22:15
anteayaclarkb: it may not, true22:15
clarkbinstead you could get all the verified=-1's then filter jenkins out with a script22:15
anteayasalv-orlando: have you seen this: http://www.rcbops.com/gerrit/reports/neutron-cireport.html22:15
anteayaclarkb: that is possible22:15
*** ChuckC has joined #openstack-infra22:15
anteayawas hoping to do it with a query22:15
anteayasalv-orlando: the source code is here: http://git.openstack.org/cgit/stackforge/radar/22:16
anteayaI have to get another walk in, be back in a while22:17
*** kmartin has joined #openstack-infra22:17
*** sballe has joined #openstack-infra22:17
openstackgerritA change was merged to openstack-infra/gear: Update from global requirements  https://review.openstack.org/10314422:17
salv-orlandoanteaya: this is along the lines of the results we should have to estimate the “correctness” of a driver or a 3rd party CI. However, I think that voting on upstream gerrit is frustating contributors that see -1s from 3rd party systems that they do not understand, annoys the infra team which should not spend time dealing with unruly 3rd party systems, and frustates 3rd party integrators which have to follow a lot of rules and feel like22:19
salv-orlandobeing policed.22:19
*** sballe_ has joined #openstack-infra22:20
*** armax has joined #openstack-infra22:20
*** armax has quit IRC22:21
*** melwitt1 has joined #openstack-infra22:23
*** craigbr has quit IRC22:23
*** sballe has quit IRC22:24
*** sballe has joined #openstack-infra22:24
*** melwitt has quit IRC22:24
*** craigbr has joined #openstack-infra22:25
*** melwitt1 is now known as melwitt22:25
*** david-lyle has quit IRC22:26
*** sballe_ has quit IRC22:26
*** gondoi is now known as zz_gondoi22:29
*** changbl_ has quit IRC22:29
*** jgrimm is now known as zz_jgrimm22:29
*** signed8bit_ZZZzz is now known as signed8bit22:30
*** bdpayne__ has joined #openstack-infra22:31
*** jcoufal has quit IRC22:32
*** otherwiseguy has quit IRC22:32
*** armax has joined #openstack-infra22:33
*** otherwiseguy has joined #openstack-infra22:33
*** bdpayne_ has quit IRC22:33
*** derekh has joined #openstack-infra22:34
*** lttrl has quit IRC22:35
derekhAny chance something has caused jenkins nodes to recently start getting reused instead of destroying them ? bug 137027522:35
uvirtbotLaunchpad bug 1370275 in tripleo "Jenkins nodes being reused" [High,Triaged] https://launchpad.net/bugs/137027522:36
*** jaypipes has joined #openstack-infra22:37
jeblairderekh: according to the nodepool logs, that's the only job that ever ran on that node.22:37
*** r-daneel__ has joined #openstack-infra22:39
*** jaypipes has quit IRC22:39
derekhjeblair: weird his is another one that started around the same time http://logs.openstack.org/30/109630/5/check-tripleo/check-tripleo-ironic-undercloud-precise-nonha/a5f0418/console.html22:40
derekhjeblair: ok,22:40
derekhjeblair: I only see the 222:40
openstackgerritmelanie witt proposed a change to openstack-dev/pbr: Support reading versions from wheels  https://review.openstack.org/12175122:40
*** psedlak has quit IRC22:41
*** reed has joined #openstack-infra22:42
jeblairderekh: since it uploaded some logs that shouldn't have existed and are, you hypothesize, from a previous run, we may be able to find the first copies of those logs.22:43
*** r-daneel__ has quit IRC22:44
jeblairderekh: all of the human readable logs should be renamed to have a .txt extension22:44
*** morgan_remote_ has joined #openstack-infra22:44
derekhjeblair: ok, noted, will do22:44
*** dannywilson has quit IRC22:44
jeblairderekh: can you point me at a seed log that is indexed in logstash so that we can use it to find other occurances of the same log?22:45
*** dannywilson has joined #openstack-infra22:45
*** dannywilson has quit IRC22:46
*** dannywilson has joined #openstack-infra22:46
openstackgerritA change was merged to openstack-infra/devstack-gate: Fix ssh key specification in the README.rst  https://review.openstack.org/11455322:46
*** signed8bit is now known as signed8bit_ZZZzz22:46
jeblairderekh: the only seed log i see in the logstash config is logs/seed_logs/os-collect-config.txt22:47
derekhjeblair: their not, as far as I know, the only one that is indexed is os-collect-config.log but it doesn't exist for that job because we're in a transition state https://review.openstack.org/#/c/120394/22:47
jeblairand i don't see that in the output of the job22:47
jeblairya22:47
clarkbharlowja: see comment on 12122122:49
harlowjaclarkb checking comment22:49
harlowjarebasing in progress22:49
*** signed8bit_ZZZzz is now known as signed8bit22:49
clarkbharlowja: basically don't want what is happening with zookeeper to happen with redis and memcached too22:51
harlowjafair enough22:51
*** dannywilson has quit IRC22:51
*** dannywilson has joined #openstack-infra22:51
*** baoli has quit IRC22:52
*** baoli has joined #openstack-infra22:52
*** armax has quit IRC22:52
openstackgerritJoshua Harlow proposed a change to openstack-infra/config: Ensure redis and memcached are on slaves (and running)  https://review.openstack.org/12122122:53
*** mwagner_lap has joined #openstack-infra22:53
harlowjadone22:53
*** craigbr has quit IRC22:53
clarkbthanks22:54
virmitiorequest for troubleshooting help:  getting an error from JJB when trying to use the !include, !include-raw, and !include-raw-escape tags22:54
clarkbjeblair: you should probably review 120480 as it is related to the zuul merger stuff22:55
clarkbvirmitio: can you paste the command you are running and the error?22:55
virmitiocmd:  jenkins-jobs test /usr/local/src/jenkins_jobs/job-01.yaml22:56
virmitioerror:  yaml.constructor.ConstructorError: could not determine a constructor for the tag '!include'22:56
jeblairclarkb: i think we fixed that by removing lxml from zuul22:56
*** r-daneel__ has joined #openstack-infra22:56
*** dannywilson has quit IRC22:56
clarkbjeblair: oh right, we moved it to test-requirements.txt22:56
clarkbvirmitio: how did you install jenkins job builder? I think this is a relatively new feature and if you didn't install via source may not get it22:57
*** prad has quit IRC22:59
virmitioclarkb: installed via pip on a ubuntu 14.04 box.  pip reports it as "jenkins-job-builder (0.8.1)".  online docs for JJB show they're for 0.8.1.95, but I don't know how to check previous versions of the docs to see when feature first appeared22:59
clarkbvirmitio: one sec23:00
clarkbvirmitio: yup it looks like that is a new feature since the 0.8.1 release23:00
*** nadya has joined #openstack-infra23:00
*** nadya is now known as Guest5347123:01
*** baoli has quit IRC23:01
*** Ryan_Lane has quit IRC23:01
clarkbvirmitio: if you want the feature you can install from source by cloning the repo `git clone https://git.openstack.org/openstack-infra/jenkins-job-builder` then using pip to install from that repo `cd jenkins-job-builder && pip install .`23:01
*** Ryan_Lane has joined #openstack-infra23:01
virmitioclarkb: well, that explains my issue.  any guess when a newer release may be available via pip?23:02
jeblairSpamapS: I approved 121776, but there's no zuul configuration so you won't be able to merge changes.23:02
clarkbvirmitio: no, I am not sure when they plan on pushing the next release23:03
virmitioclarkb: fair enough.  much appreciated23:03
jeblairSpamapS: you probably have about 10-20 minutes to push a new patchset if you change your mind :)23:03
*** dimtruck is now known as zz_dimtruck23:04
*** boden has joined #openstack-infra23:04
*** shashankhegde has quit IRC23:04
*** Guest53471 has quit IRC23:04
openstackgerritA change was merged to openstack-infra/config: Stop running non-voting oslo.db-python33 in gate  https://review.openstack.org/12083523:05
openstackgerritA change was merged to openstack-infra/config: Add missing merge-check and noop-jobs to layout.yaml  https://review.openstack.org/12035223:05
SpamapSjeblair: No zuul configuration?23:05
*** marcoemorais has quit IRC23:05
* SpamapS re-reads http://ci.openstack.org/stackforge.html23:05
*** marcoemorais has joined #openstack-infra23:05
jeblairSpamapS: a change to layout.yaml to tell zuul about it23:05
SpamapSdohh I missed the noop jobs thing23:06
SpamapSI went "oh see we don't need gate jobs"23:06
*** reed has quit IRC23:06
*** dims has quit IRC23:06
SpamapScurse me and my skimming ways23:06
*** dims has joined #openstack-infra23:06
*** marcoemorais has quit IRC23:08
clarkbfungi: does 119426 cause problems with the py34 migration? I guess we run under py34 and find out23:08
SpamapSjeblair: thanks for the heads up23:08
*** marcoemorais has joined #openstack-infra23:08
openstackgerritClint 'SpamapS' Byrum proposed a change to openstack-infra/config: Add tripleo-ansible to stackforge  https://review.openstack.org/12177623:08
*** boden has quit IRC23:08
*** marcoemorais has quit IRC23:08
*** mriedem has quit IRC23:08
*** marcoemorais has joined #openstack-infra23:09
clarkbfungi: I am running py34 for those too locally really quickly to see if there is anything to see23:09
*** timo__ has quit IRC23:11
*** dims has quit IRC23:11
*** Sukhdev has joined #openstack-infra23:11
*** AaronGr has quit IRC23:12
*** AaronGr has joined #openstack-infra23:12
openstackgerritA change was merged to openstack-infra/config: Add a docs job for manila  https://review.openstack.org/12158323:14
openstackgerritA change was merged to openstack-infra/config: Translate oslo.middleware  https://review.openstack.org/11965223:15
derekhjeblair: ok, I'm going to add something into the tripleo tests to detect if a node had been previously used,23:15
openstackgerritA change was merged to openstack-infra/config: Some third-party.rst cleanup  https://review.openstack.org/11989323:15
*** bookwar has quit IRC23:17
*** bookwar has joined #openstack-infra23:17
jeblairderekh: what are you going to do?23:18
*** virmitio has quit IRC23:18
openstackgerritA change was merged to openstack-infra/config: Update for new home for puppetboard module  https://review.openstack.org/11996623:19
clarkbfungi: oslo.log passes py34 but oslo.utils does not23:19
clarkbfungi: not sure if you want us to be adding voting py33 jobs while we try to move to py3423:19
*** sballe has quit IRC23:19
clarkb*voting py33 jobs that do not pass py3423:19
jeblairderekh: i ask because i think the scope of what you're looking for needs to be expanded.  considering the logs say that we did not re-use the node, we should consider things like "nova re-used the disk image" as well.23:21
jeblairbasically, anything should be on the table :)23:21
*** craigbr has joined #openstack-infra23:22
derekhjeblair: something like this https://review.openstack.org/#/c/122018/1/toci_devtest.sh,cm23:22
*** ryanpetrello has quit IRC23:23
derekhjeblair: yup, agreed it may be a nova problem either, I'll take a look through the nova logs aswell (probably in the morning at this stage) and see if I can spot anything weird going on23:24
*** jpich has joined #openstack-infra23:24
jeblairderekh: okay; i'm about to leave a comment on that change23:24
derekhjeblair: ok23:25
*** shashankhegde has joined #openstack-infra23:25
SpamapSjeblair: thanks for the quick re-review. :)23:26
*** craigbr has quit IRC23:27
jeblairderekh: commented23:28
*** dmsimard_away is now known as dmsimard23:32
*** rcarrillocruz has quit IRC23:34
*** marcoemorais has quit IRC23:36
*** otherwiseguy has quit IRC23:36
*** marcoemorais has joined #openstack-infra23:36
*** otherwiseguy has joined #openstack-infra23:37
*** praneshp has quit IRC23:37
*** rcarrillocruz has joined #openstack-infra23:37
*** hdd has joined #openstack-infra23:39
*** Guest72739 is now known as mgagne23:42
*** mgagne has joined #openstack-infra23:42
derekhjeblair: thanks, new version up, clocking off will pick it back up in the AM23:43
*** dangers is now known as dangers_away23:44
*** derekh has quit IRC23:45
*** bswartz has joined #openstack-infra23:46
*** rcarrillocruz has quit IRC23:47
*** tomoe has joined #openstack-infra23:48
*** ekarlso has quit IRC23:48
*** ekarlso has joined #openstack-infra23:48
*** rcarrillocruz has joined #openstack-infra23:51
*** yamahata has joined #openstack-infra23:56
*** groknix has quit IRC23:58
openstackgerritA change was merged to openstack-infra/devstack-gate: turn off clean log checking  https://review.openstack.org/11951223:59
openstackgerritA change was merged to openstack-infra/devstack-gate: Don't use ephemeral volumes for Ironic agent driver  https://review.openstack.org/12081723:59
openstackgerritA change was merged to openstack-infra/devstack-gate: Longer build timeout for Ironic agent driver  https://review.openstack.org/12160023:59
openstackgerritA change was merged to openstack-infra/devstack-gate: Install PyYaml for test-matrix.py  https://review.openstack.org/12092923:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!