Friday, 2015-03-06

mordredclarkb: however, if we upgrade novaclient and nodepool breaks, we can point our fingers at people and laugh00:00
mordredwhich is fun00:00
clarkbah ya that just won't make them on a new node but won't kill them on an existing host mtreinish I will just disable them on this host00:00
clarkbmordred: your idea of fun is funny00:00
clarkbmordred: also I could not get that image to break on any of the three hpcloud AZs00:00
clarkbthough az2 was consistently having scheduling errors00:00
*** timcline has quit IRC00:01
anteayagerrit is taking an inordinate amount of time to save my inline comment00:01
*** ivar-lazzaro has quit IRC00:01
mtreinishclarkb: ok thanks. FWIW, I checked cacti and the memory growth issue we were seeing the last time we ran 4 doesn't seem to be happening00:01
*** dmsimard is now known as dmsimard_away00:01
*** MarkAtwood has quit IRC00:02
*** ghostpl_ has quit IRC00:02
openstackgerritMonty Taylor proposed openstack-infra/puppet-nodepool: Upgrade dependent libraries when we upgrade  https://review.openstack.org/16195100:02
mordredjeblair, clarkb: I think we might want that ^^00:02
mtreinishmmedvede: hmm, ok. That differs from what I remember, but it's been almost a while so I'm probably just not remembering correctly00:03
openstackgerritMerged openstack-infra/project-config: Add Big Switch Networking library to StackForge  https://review.openstack.org/16033900:03
mmedvedemtreinish: I am not sure if that is the proper way to check though00:03
clarkbmtreinish: but we process quickly enough with one so might as well reduce the chance of having it explode00:03
openstackgerritIan Wienand proposed openstack-infra/nodepool: Add config validation option  https://review.openstack.org/16195200:03
*** achanda has quit IRC00:04
mtreinishclarkb: yeah, using 1 is the right way to do it. I was just pointing it out00:04
clarkbmtreinish: cool all done00:04
*** ayoung has quit IRC00:04
mtreinishclarkb: cool, thanks00:05
mtreinishclarkb: if you're looking at subunit2sql stuff, I've got the patch up to do the backups before we run upgrades00:06
openstackgerritIan Wienand proposed openstack-infra/nodepool: Add config validation option  https://review.openstack.org/16195200:08
clarkbianw: thats a lot of yaml :)00:09
devanandajogo: in test runs of services, are clients being installed from trunk or pip?00:09
clarkbmtreinish: does the db-manage script not use the same my.conf?00:10
devanandajogo: and is it different between master branch of services, and their stable branch(es)?00:10
clarkbmtreinish: I guess not since the manage script is all python/alembic00:10
mtreinishclarkb: no the formats are different. One's an oslo.config thing the others the mysql thing00:10
clarkbdevananda: pip for most jobs00:10
mtreinishthey're close but different00:10
clarkbdevananda: and thats not different for the branches00:10
devanandaclarkb: that's what I thought, thanks. any plans to change that? (please say no)00:10
*** ZZelle has quit IRC00:10
*** markvoelker has quit IRC00:11
*** ivar-lazzaro has joined #openstack-infra00:11
*** ZZelle has joined #openstack-infra00:11
clarkbdevananda: there are however jobs for the clients themselves that install from source00:11
clarkbdevananda: I don't think there are plans.00:11
devanandaclarkb: naturally00:11
devanandaok, great00:11
devanandathat's the sanity I was hoping for :)00:11
bkeroSo what exactly is the Monty fact? http://puppetboard.openstack.org/fact/monty00:11
ianwclarkb: just copies of the real config file ... though some might argue that's a lot of yaml too00:12
*** dangers is now known as dangers_away00:12
mtreinishmmedvede: that's probably as good as any other method. I assume that site just puts the inbound address in the response body00:12
SpamapSmordred: I'm thinking that with shade we might want to say if you don't pass cache_interval that we shouldn't do any caching. Thoughts?00:13
mordredSpamapS: that's an excellent idea00:13
*** Sukhdev has quit IRC00:13
openstackgerritMerged openstack-infra/subunit2sql: Add microsecond columns to the test_runs table  https://review.openstack.org/16005700:13
clarkbianw: it is I was mostly being silly, the initial diff numbers caught my eye00:14
SpamapSmordred: because otherwise we have a memory leak waiting for every casual user of shade. :-P00:14
mordredSpamapS: I would like to subscribe to your newsletter00:14
clarkbbkero: "blerg"00:14
* SpamapS writes patch00:14
bkeroclarkb: blerg, obviously00:14
openstackgerritJames E. Blair proposed openstack-infra/project-config: Normalize Gerrit projects.yaml  https://review.openstack.org/16195400:15
openstackgerritJames E. Blair proposed openstack-infra/project-config: Wrap projects.yaml at 80 characters  https://review.openstack.org/16195500:15
openstackgerritJames E. Blair proposed openstack-infra/project-config: Remove upstreams from projects.yaml  https://review.openstack.org/16195600:15
openstackgerritJames E. Blair proposed openstack-infra/project-config: Propose projects.yaml normalization daily  https://review.openstack.org/16195700:15
devanandagertty crashed. booo.00:15
jeblairdevananda: current master?00:15
clarkbmtreinish: commented00:15
devanandajeblair: latest pip00:15
devanandajeblair: it's a rare enough event that I felt it worth comment00:15
devanandaalso, it lost a several-paragraph review that I was working on...00:16
devanandajeblair: possibly because I touched my screen (it's a touch screen)00:16
jeblairdevananda: ok.  lots of crashing bugs are fixed in latest master; a few are still documented at https://storyboard.openstack.org/#!/project/69800:16
jogodevananda:AFAIK pip00:16
jogobut its easy to check in pip-freeze00:16
*** ivar-laz_ has joined #openstack-infra00:16
*** patri394 has quit IRC00:17
*** patri767 has quit IRC00:17
*** patri755 has quit IRC00:17
*** patri832 has quit IRC00:17
*** patri460 has quit IRC00:17
*** patri192 has quit IRC00:17
jeblairdevananda: (and i'm going to release a new version tomorrow, so it'll be on pip then)00:17
nibalizerjeblair: awesome00:17
devanandajogo: cool, thx00:17
*** ivar-lazzaro has quit IRC00:17
devanandajeblair: yay fixes!00:18
mtreinishmmedvede: I'm just really surprised they wouldn't be masquerading the outbound. It's not like it's routable from outside the firewall (unless you guys have changed how things are setup)00:18
mtreinishclarkb: thanks, I'll respin it quickly00:18
mmedvedemtreinish: I never asked that question until today00:19
*** patrickeast_ has joined #openstack-infra00:20
*** patrickeast__ has joined #openstack-infra00:20
*** patri851 has joined #openstack-infra00:20
*** patri601 has joined #openstack-infra00:20
*** patri466 has joined #openstack-infra00:20
*** pc_m has joined #openstack-infra00:20
*** patri796 has joined #openstack-infra00:20
*** dboik_ has quit IRC00:21
*** dboik has joined #openstack-infra00:22
openstackgerritMatthew Treinish proposed openstack-infra/puppet-subunit2sql: Add backup before running db migrations  https://review.openstack.org/16051400:23
mtreinishclarkb: ^^^00:23
*** bhunter71 has quit IRC00:23
mmedvedemtreinish: and yes, it is still not routable from outside the firewall00:23
clarkbmordred: can you maybe comment on 126747 with your ideas for how we will support that?00:24
clarkbmordred: I think it is heavily dependent on dib and shade things00:24
*** bhunter71 has joined #openstack-infra00:24
clarkbwoo for rereviewing your own code00:26
openstackgerritClark Boylan proposed openstack-infra/puppet-jenkins: Allow for plugin pinning  https://review.openstack.org/16054900:27
*** stevemar has quit IRC00:27
*** ZZelle_ has quit IRC00:28
SpamapSmordred: why does shade.OpenStackCloud.__init__ take kwargs or things that are really clearly just more kwargs that could be static?00:28
anteayamordred: does the gitorious information need to be updated in this patch? https://review.openstack.org/#/c/154132/00:28
*** claudiub has quit IRC00:29
mordredclarkb: yes. thanks for that reminder00:29
*** crc32 has quit IRC00:29
*** liusheng has quit IRC00:29
openstackgerritSpencer Krum proposed openstack-infra/project-config: Allow puppet-httpd to be publised to the forge  https://review.openstack.org/16102200:29
SpamapSmordred: there are a kind of ridiculous number of kwargs used exactly as they'd be if they were just listed.00:29
mordredSpamapS: we need to maybe sort out in shade how to list per-vendor info on image formats or something00:29
*** liusheng has joined #openstack-infra00:29
mordredSpamapS: most of the time it's when it's passthrough -00:29
mordredSpamapS: but it also could just be because I'm a bad person00:30
SpamapSmordred: auth, auth_token, service_type, service_name, endpoint, api_version, publicURL, private ... all just used with kwargs.get00:30
mordredSpamapS: have you seen my "clearn up parameters" patch?00:30
SpamapSmordred: yeah00:30
mordredk00:30
mordredSpamapS: so yeah, I support cleaning that up00:31
SpamapSOh!00:31
SpamapSsome of htose are there00:31
SpamapSdid not connect the dots00:31
mordredwe should land that00:31
mordredit'll make reasoning easier00:31
*** _shaps_ has quit IRC00:32
mordredclarkb: could you take a peek at 160184 and its parent 160183 ?00:32
clarkbmordred: sure00:32
SpamapSmordred: I'm going to rebase on that for my "don't cache if no interval is passed" patch.00:32
*** sdake_ has quit IRC00:33
SpamapSwe should also land unit tests and functional tests. :)00:33
openstackgerritMerged openstack-infra/project-config: Add bashate job for release-tools  https://review.openstack.org/16041900:34
*** sdake_ has joined #openstack-infra00:34
mordredSpamapS: ++00:35
mordredclarkb: also, yes, patch #2 has a breaking interface change. to my knowledge nothing has landed anywhere that consumes the old interface and I'd like to clean it before we can't00:35
clarkbpeople that know apache better than I should read 16116500:36
*** pc_m has quit IRC00:36
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Add expiremental plugin based ironic+ironicclient functional jobs  https://review.openstack.org/16148300:36
clarkbits funny that spamaps always pokes mordred for tests expecting mordred to even consider writing them00:40
openstackgerritClint 'SpamapS' Byrum proposed openstack-infra/shade: Disable dogpile.cache if cache_interval is None  https://review.openstack.org/16196300:41
*** achanda has joined #openstack-infra00:41
SpamapSclarkb: actually I wrote them and want to land them. :)00:41
clarkbalso I was told that shade is not being rigidly tested and as a reviewer I should assume the author has tested00:41
SpamapSclarkb: and rcarrillocruz also wrote some and wants to land them. :)00:41
clarkbSpamapS: those are orthogonal00:41
SpamapSclarkb: shade's growing up. :)00:41
clarkber orthogonal to ^00:42
SpamapSclarkb: oh I thought we were talking about my comment that we should also land unit tests and functional tests. :)00:42
clarkbno reviewing changes and you are poking for tests :)00:42
clarkbI don't think its a bad thing, but I also know mordred won't write them :)00:42
SpamapSI was poking you indirectly.00:42
clarkbI know00:43
SpamapSpoke poke00:43
SpamapSpoke00:43
clarkbI thought I had reviewed rcarrillocruz's stuff00:43
clarkband debugged d-g for him00:43
mordredclarkb: I am actually working on some os-client-config tests00:43
clarkbspeaking of I should show that fix to sdague to see if he knows what is going on there00:43
*** shivharis has quit IRC00:43
*** shivharis has joined #openstack-infra00:43
SpamapSoohhhh burn00:43
mordredclarkb: but mostly just to confuse you00:43
*** dims has quit IRC00:44
*** shivharis has quit IRC00:45
openstackgerritMerged openstack-infra/shade: Align cert, key, cacert and verify with requests  https://review.openstack.org/16018300:45
*** ivar-laz_ has quit IRC00:45
*** MarkAtwood has joined #openstack-infra00:45
*** ivar-lazzaro has joined #openstack-infra00:45
*** achanda has quit IRC00:46
devanandajogo, clarkb: either of you have a moment to sanity-check my statements about how microversions should interact with the gate? https://etherpad.openstack.org/p/oHVxTVRFix00:47
openstackgerritJames E. Blair proposed openstack-infra/project-config: Create a job template for tempest on devstack plugins  https://review.openstack.org/16196500:47
devanandabackground is folks disagreeing on whether python-ironicclient should, by default, send the greatest API version that was known about at the time it was released -- or if it should not send ANY header at all unless explicitly told to00:48
devanandathat will directly affect how we test the interaction between nova and Ironic. and, of course, users.00:48
clarkbshouldn't the client as the remote end instead of the other way around?00:48
*** ivar-lazzaro has quit IRC00:48
devanandaremote end supports a range of versions00:49
clarkbthats just me thinking out loud that it is better for clients to be forgiving to the servers00:49
clarkbdevananda: ya so it should return that to the client and client picks best one00:49
*** ivar-lazzaro has joined #openstack-infra00:49
clarkbso that we can avoid the glanceclient issue of "I'm gonna use v1 and only v1 and everything else is a user nightmare"00:49
devanandaor the client says to the server "here's the version I want" and if the server supports that, it adheres to it00:49
clarkbdevananda: ya thats what glanceclient does and it doesn't work00:50
devanandaand if the server does not support that, then the client interprets the headers and tries to ask for something the server does support00:50
mtreinishclarkb: version discovery is broken in more than just glance fwiw :)00:50
devanandaclarkb: so i'm talking about microversions, not /v1/ vs /v2/00:50
*** signed8bit is now known as signed8bit_ZZZzz00:50
clarkbdevananda: its the same difference really00:50
devanandaclarkb: not really00:51
clarkbdevananda: you are negotiating "a" version00:51
clarkbwhat type of version isn't really important00:51
mordredthe biggest question is whether you're asking the developer using the library to request a version00:51
mordredif you are doing that, you are losing00:51
devanandathough if the client defaults to always ask for v1.0, it is going to re-create the same problem00:51
clarkbmordred: yes that is what this would implicitly do00:51
mordredif you are not, and it's transparent - DONT CARE00:51
clarkbmordred: by installing a specific client you are requesting a specific version00:51
mordredwait - what?00:51
clarkbmordred: read scrollback00:52
mordredthere is someone suggesting that we install a specific client version to talk to a cloud?00:52
clarkbmordred: not quite00:52
devanandamordred: that is definitely true. also, if the client defaults to oldest version, it means we can't effectively ever expect any other openstack project to use a newer version00:52
devanandalike nova talking to ironic00:52
mordreddevananda: let me suggest something different00:52
devanandawhich is where the present debate (in our team) arose from00:52
*** rmcall has quit IRC00:52
clarkbmordred: but similar to how glanceclient works the idea here seems to be if you install fooclient version X it will use the corresponding api version Y to talk to server00:52
clarkbthough at least there seems to be an attempt to negotiate00:52
clarkbbut I think thats more work, client should say "what do you support" then pick00:53
devanandaclarkb: right - that's basically what I'm outlining in the 'pad I linked00:53
mordreddevananda: the client, by default, will always know the greatest API version that it knows about00:53
mordredbecause it can't not know that00:53
devanandamordred: right00:53
devanandalink again -- https://etherpad.openstack.org/p/oHVxTVRFix :)00:53
mordredit should request that, but be able to accept all previous versions that the server sends back00:53
devanandamordred: if you disagree with what i'm saying three, I'd like to know why00:53
mordredthen, if if the user tries to use a feature that the server does nto requests, there should be a "oh, hi, your server does not support this feature" error00:53
mordredbut if it's a feature that just had a different rest api semantic, the client should do the right thing00:54
adam_gdevananda, fwiw that testing plan in the pad looks sane to me. im actually putting together the barebones infra for the functional client testing now00:54
devanandaalso there is a lengthy spec up that descibes the different scenarios involved... since engineers want to describe every possible interaction :) https://review.openstack.org/#/c/161110/00:54
clarkbmy only request is that as a user I never have to tell my client to use a specific version or install a specific version of the client to get the previous result00:54
devanandaadam_g: k, thanks00:55
devanandaclarkb: ++00:55
mordredclarkb: ++00:55
clarkbas far as designing how that works I think you shove the logic in the client00:55
*** ddieterly has joined #openstack-infra00:55
mordredother than taht, I think microversions are a mistake00:55
mordredand I thnk they are developers wanking themselves00:55
devanandaclarkb: my goal, for a server and client within reasonable temporal distance (what ever we deem supportable) they will auto-negotiate, with as few round-trips as possible, the greatest common version00:55
mordredand not solving problems that users have00:55
jogodevananda: looking00:56
mordredAPI versions are bull00:56
mordredthey solve nothing00:56
mordredthey break users00:56
mordredthey are a waste of time00:56
mordredas is, btw, semver00:56
devanandamordred: they allow applications that talk to a versioned service to choose when to upgrade to the next version of the API00:56
clarkbdevananda: the only danger that immediately pops out with your test plan that I see is if you break that feature after merging it into the client but before releasing the client00:56
mordredmeh00:56
mordrednobody wants to upgrade to the next version of the API00:56
clarkbdevananda: there is a race there, I guess its up to ou to decide how dangerous it is00:56
mordredthey want to do work00:56
devanandaclarkb: good point. i think i'm willing to accept that00:57
mordredthe question is "does my cloud support me uploading an image"00:57
clarkbdevananda: I will note it on the etherpad for ease of findability00:57
mordredthe question is not "does my cloud run v1 or v2 or v2.1.1"00:57
devanandaclarkb: as it is, we aren't testing many things, because nova is effecetively pinned to the API that we had in stable/juno00:57
mordredliterally no developer ahs ever cared about the verion00:57
openstackgerritJames E. Blair proposed openstack-infra/project-config: Remove upstreams from projects.yaml  https://review.openstack.org/16195600:57
openstackgerritJames E. Blair proposed openstack-infra/project-config: Propose projects.yaml normalization daily  https://review.openstack.org/16195700:57
openstackgerritJames E. Blair proposed openstack-infra/project-config: Normalize Gerrit projects.yaml  https://review.openstack.org/16195400:57
openstackgerritJames E. Blair proposed openstack-infra/project-config: Wrap projects.yaml at 80 characters  https://review.openstack.org/16195500:57
devanandamordred: indeed00:57
mordredthe version is a very very poor way to map capabilities00:57
devanandamordred: the problem is "developers feel the need to change something in the API but dont want to break support for existing applications, however old they are"00:58
devanandamordred: "while enabling new apps to use new feature"00:58
mordreddevananda: the developers should just get over it00:58
mordredhonestly00:58
mordredand shoyld not change that thing00:58
* mordred points, once again, at libmysqlclient00:58
devanandamordred: so openstack should never grow new features? or we should only do fork-lift, wholesale changes to services. ...00:58
mordredgrow new features is great00:59
mordredthat doesn't break anything00:59
mordredchanging existing things should just not happen00:59
mordredOR00:59
mordredif it des00:59
mordredknow that it's the most painful thing that has ever happened00:59
mordredand no amount of increasing a version number will fix that00:59
*** ddieterly has quit IRC00:59
mordredif nova addeda  "create jellybean" api - I would not care at all01:00
mordredbut when glance v2 _removes_ the ability to upload directly via the v1 api call01:00
mordredNOW I have to write a bunch of "if v1: elif v2: crap"01:00
*** bhunter71 has quit IRC01:01
*** patri466 has quit IRC01:02
*** patri851 has quit IRC01:02
*** patri601 has quit IRC01:02
*** patri796 has quit IRC01:02
*** patrickeast_ has quit IRC01:02
*** patrickeast__ has quit IRC01:02
*** otter768 has joined #openstack-infra01:02
*** carl_baldwin has quit IRC01:02
mordredwhich leads to crap like this: http://git.openstack.org/cgit/openstack-infra/shade/tree/shade/__init__.py#n58301:02
mordredmy fear is taht microversions are going to make me write 100x that ^^01:02
mordredbecause they're goign to open the door to devs thinking they have freedom to change stuff because they told people about it01:03
anteayaclarkb: any reason I should not +A enable swift logs for python-jobs? https://review.openstack.org/#/c/156521/701:03
*** mikedillion has joined #openstack-infra01:03
anteayawe were holding off on team agreement which we got at last meeting01:03
anteayaany technical reason to time the merge?01:04
clarkbanteaya: we need the parent to merge first, then rebuild images then merge the child01:04
anteayaokay so I can +A https://review.openstack.org/#/c/158514/7 ?01:04
anteayayou are +201:04
mordreddevananda: (reading etherpad)01:05
clarkbanteaya: yes that one should be ready01:05
anteayavery good, thanks01:05
*** mikedillion has quit IRC01:05
openstackgerritAdam Gandelman proposed openstack-infra/project-config: Add expiremental plugin based ironic+ironicclient functional jobs  https://review.openstack.org/16148301:05
devanandamordred: if you have time, read the spec and the comments too01:06
*** mikedillion has joined #openstack-infra01:06
anteayaso after images build tomorrow then look again at the python-jobs patch01:06
anteayayes?01:06
*** otter768 has quit IRC01:06
*** MarkAtwood has quit IRC01:07
anteayawhy would we add python2.6 jobs to anything? https://review.openstack.org/#/c/160928/01:07
anteayaI had thought we were trying to get rid of them01:07
mordredanteaya: I think it's temporary01:08
mordredanteaya: since oslo still supports 2.6 and that library is used by oslo01:08
anteayayes that is what the commit message says01:08
anteayawe are okay with that?01:08
*** teoleg has joined #openstack-infra01:11
mordredfine by me - but I'm only but one voice01:12
jeblairanteaya: i think we're okay with it until the last openstack release supporting 2.6 goes away01:13
jeblair(if it's needed)01:13
jeblairharlowja: btw, you don't have to wrap your commit messages at 50 chars, feel free to use 80 for anything after the first line :)01:14
harlowjalol01:15
anteayamordred jeblair okay thanks01:15
*** ivar-laz_ has joined #openstack-infra01:15
harlowjai'm striving for 10 char lines01:15
harlowjalol01:15
*** ivar-laz_ has quit IRC01:15
*** markvoelker has joined #openstack-infra01:15
*** Jeba has quit IRC01:16
*** ivar-laz_ has joined #openstack-infra01:16
*** fandi has joined #openstack-infra01:16
anteayahave you tried kanji?01:16
harlowjahakiu u say01:16
harlowjau want me to write all my commit messages as haiku01:17
harlowjaok then01:17
harlowjalol01:17
*** rwsu is now known as rwsu-afk01:17
morganfainbergharlowja, can i get a bot that rewrites all commit messages as haikus?01:17
anteayaasselin: your reply to the mailing list thread went to the third party announce list only from what I can see01:17
morganfainbergor is haiku also the plural01:17
harlowjaprobabably not impossible, lol01:17
* morganfainberg should look01:17
anteayaasselin: the parent never should have included the -announce list01:17
*** bhunter71 has joined #openstack-infra01:18
morganfainbergharlowja, can i also get it wired up to a speak and spell? ;)01:18
harlowjasureeee01:18
morganfainbergharlowja, awesome!01:18
StevenKmorganfainberg: Pipe it to festival?01:18
asselinanteaya, hmmm...I just hit reply...01:18
anteayaasselin: oh no, it did go to -dev01:18
anteayayeah, the author of teh original post included -announce01:18
harlowjajust plug your bot into http://www.languageisavirus.com/interactive-haiku-generator.html and profit, lol01:18
anteayaso hitting reply would do the same01:18
*** ivar-lazzaro has quit IRC01:18
anteayaasselin: I'm going to deny the post to the -announce list and thought I would let you know01:19
asselinanteaya, ok, I canceled the request to send to annouch01:19
anteayathanks01:19
openstackgerritMerged openstack-infra/project-config: zuul-swift-upload support recursive globs  https://review.openstack.org/15851401:19
* morganfainberg goes back on the hunt for a portable laptop w/ 16GB of ram and real SSD space, and a usable screen. (portable means 13" or less)01:19
*** mtanino has quit IRC01:19
* morganfainberg wonders if anyone will make said laptop.01:19
asselinanteaya, it did go to the mailing list01:20
anteayayes, I see it now01:20
anteayajust took my client a minute, so I checked -dev in the meantime01:20
asselinanteaya, i will be more careful next time. thanks01:20
anteayano problem, the original issue didn't reside with you01:20
anteayathanks for cancelling01:20
*** markvoelker has quit IRC01:20
StevenKmorganfainberg: That would be nice.01:21
*** melwitt has quit IRC01:22
morganfainbergStevenK, right? i'm actually hoping the 12" MBA will have passible/usable hardware so i can use that whenever it comes out - use in this case means install linux as the main OS on it01:22
clarkbmtreinish: around? what is the name of devstack's imageid 42?01:23
clarkbmtreinish: is it m1.nano? and its the smallest one right?01:23
clarkbmtreinish: sorry not imageid, flavorid01:23
*** crinkle has quit IRC01:23
*** tsg has quit IRC01:26
openstackgerritMerged openstack-infra/project-config: Add python26 jobs to debtcollector  https://review.openstack.org/16092801:26
*** bcourt has quit IRC01:26
*** tsg has joined #openstack-infra01:26
*** teoleg has quit IRC01:27
*** Daisy_ has joined #openstack-infra01:28
*** ghostpl_ has joined #openstack-infra01:28
*** r-daneel has quit IRC01:31
*** otter768 has joined #openstack-infra01:32
*** bdpayne has quit IRC01:33
*** fandi has quit IRC01:33
*** fandi has joined #openstack-infra01:33
*** crinkle has joined #openstack-infra01:34
asselinI'm getting this error using DIB: "Package ruby1.8 is not available, but is referred to by another package." http://paste.openstack.org/show/189875/01:34
asselinWehre are those packages defined?01:35
asselinnevermind...I found the file: devstack-gate.d/dib-manifests/dib-manifest-dpkg-devstack-gate01:36
*** ghostpl_ has quit IRC01:36
anteayaare you building on trusty?01:36
asselinI using precide node to build trusty image01:37
anteayaah ha01:37
*** tsg has quit IRC01:37
anteayaprecise had ruby1.801:37
asselinbut ruby 1.8 isn't in that file....01:37
openstackgerritMerged openstack-infra/project-config: Fix swiftclient func tests to run using both auth versions  https://review.openstack.org/16084401:37
anteayaso that is what it would be looking for01:37
*** dannywilson has quit IRC01:38
*** ddieterly has joined #openstack-infra01:38
asselinbut should dib be using the trusy packages?01:38
anteayahttp://packages.ubuntu.com/trusty/ruby01:38
anteayahttp://packages.ubuntu.com/precise/ruby01:38
anteayaso the clue in what it is calling it01:38
anteayaprecise calls it ruby1.8 (even though it is ruby 1.9)01:39
anteayaand trusty calls it ruby01:39
jesusaurusasselin: i believe it bind-mounts the host deb cache. i dont think its possible to cross-build distros or distro versions01:39
* jesusaurus is not well-versed in dib01:39
greghaynesit is01:39
greghaynesboth things are possible for most combinations01:39
*** ddieterl_ has joined #openstack-infra01:40
*** ddieterly has quit IRC01:40
openstackgerritMerged openstack-infra/project-config: add python26 jobs back for compass-core  https://review.openstack.org/16025101:40
asselinis ruby the exception? anywya I see about printing some debug logging01:40
greghaynesasselin: youre building for a trusty target?01:40
asselingreghaynes, yes01:40
greghaynesasselin: huh, I cant find that 50-download-pkgs script, where did you find it?01:41
greghaynesoh nvm, found01:41
*** MarkAtwood has joined #openstack-infra01:44
greghaynesasselin: so http://git.openstack.org/cgit/openstack-dev/devstack/tree/files/debs is where its getting the packages to install AIUI01:44
greghaynesits a rather large collection01:45
greghaynesbut something in there obviously depends on ruby1.801:45
greghaynesI do wonder how the existing image builds dont have this issue though01:45
greghaynesclarkb: ^01:45
clarkbwe build on trusty?01:46
greghaynesits in the chroot so it matters what you target01:46
greghaynesI think01:46
*** achanda has joined #openstack-infra01:46
greghaynesyea01:46
clarkbalso its just downloading packages not installing anything01:47
clarkbso nothing should fail right?01:47
greghaynesapparently not01:47
greghaynesyou do build trusty targets though?01:47
clarkbyes01:47
clarkbcentos7, trusty, and precise01:47
asselinI see DIB is setting this up, not sure where it comes from: /home/stack/project-config/devstack-gate.d/dib-manifests01:48
greghaynesasselin: do you have a trusty vm handy?01:48
asselingreghaynes, this file has ruby in it: http://paste.openstack.org/show/189889/01:48
asselingreghaynes, yes01:48
asselingreghaynes, I think infra runs nodepool on trusty though. I'm running it on precise right now01:49
greghaynesasselin: If you edit openstack-infra/project-config/nodepool/elements/cache-devstack/install.d/50-download-pkgs you can echo out the list of packages it is attempting to install as it goes through the loop01:50
greghaynesthat should tell you what is failing01:50
greghayneser, attempting to download01:50
asselingreghaynes, yes I did that...just changed set +ex01:50
asselinrunning now01:50
clarkbSpamapS: ok I think I have reviewed all the shade test changes01:51
*** MarkAtwood has quit IRC01:52
openstackgerritRamy Asselin proposed openstack-infra/project-config: DIB project-config scripts requires DIB_RELEASE  https://review.openstack.org/16198901:53
*** rkukura has quit IRC01:57
mtreinishclarkb: yeah it's m1.nano01:59
mtreinishclarkb: https://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/tempest#n19301:59
*** ashleighfarnham has quit IRC02:00
clarkbmtreinish thanks02:00
*** mikedillion has quit IRC02:02
*** ChuckC has quit IRC02:04
*** tiswanso has joined #openstack-infra02:04
*** ChuckC has joined #openstack-infra02:05
*** tiswanso has quit IRC02:05
*** tiswanso has joined #openstack-infra02:05
*** stevemar has joined #openstack-infra02:08
*** tiswanso has quit IRC02:08
*** tiswanso has joined #openstack-infra02:09
*** rkukura has joined #openstack-infra02:09
*** tsg has joined #openstack-infra02:14
*** oomichi has quit IRC02:14
*** ivar-laz_ has quit IRC02:16
anteayagreghaynes: so I commented on https://review.openstack.org/#/c/157554/02:16
anteayagreghaynes: I am unable to see what benefit your patch is offering02:16
*** ivar-lazzaro has joined #openstack-infra02:17
*** markvoelker has joined #openstack-infra02:17
*** crc32 has joined #openstack-infra02:17
anteayabut it is expanding02:17
*** baoli has joined #openstack-infra02:17
*** signed8bit_ZZZzz is now known as signed8bit02:18
*** signed8bit has quit IRC02:19
*** weshay has quit IRC02:20
*** markvoelker has quit IRC02:21
openstackgerritMerged openstack-infra/project-config: skip rerunning requirements changes in devstack  https://review.openstack.org/15621402:22
*** vilobhmm2 has quit IRC02:23
*** vilobhmm1 has joined #openstack-infra02:23
*** ivar-lazzaro has quit IRC02:23
greghaynesanteaya: If export NODEPOOL_STATIC_NAMESERVER='1.2.3.4' then that value will be set for your resolv.conf rather than 8.8.8.802:24
*** ivar-lazzaro has joined #openstack-infra02:24
*** dmsimard_away is now known as dmsimard02:25
anteayaokay02:25
*** ivar-lazzaro has quit IRC02:25
greghaynesWhich is important if you want to, for example, use a mirror that only has dns entries on an internal dns server02:26
*** ivar-lazzaro has joined #openstack-infra02:26
anteayaokay02:27
*** yamamoto has joined #openstack-infra02:27
anteayaand 8.8.8.8 remains the default and can now be overridden02:27
greghaynesyep02:28
anteayaif you export a different value in the environment02:28
anteayaokay, i'm starting to see that02:28
greghaynesYea, the bash for it is pretty obtuse, but ${VAR:-default} expands to $VAR if VAR is set else default02:29
anteayaokay thanks02:29
anteayaI've commented again saying you have answered my questions02:30
greghaynes:)02:30
anteayabut I don't know enough about nodepool internals to vote on the patch02:30
anteayaso thanks02:30
anteaya:)02:30
*** dmsimard is now known as dmsimard_away02:35
*** camunoz is now known as camunoz_afk02:36
anteayaokay if you haven't seen this one yet: http://i.imgur.com/9hRi2jN.gifv02:38
openstackgerritBrian Hunter proposed openstack-infra/nodepool: Pretty Age in nodepool list  https://review.openstack.org/16199402:38
*** ddieterly has joined #openstack-infra02:38
*** tsg has quit IRC02:39
*** camunoz_afk is now known as camunoz02:41
*** ddieterl_ has quit IRC02:41
*** baoli has quit IRC02:43
*** ddieterl_ has joined #openstack-infra02:44
*** ddieterly has quit IRC02:45
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: zuul-swift-upload place mime icon into cell  https://review.openstack.org/16199502:48
*** jerryz_ has joined #openstack-infra02:50
*** vilobhmm1 has quit IRC02:55
*** ChuckC_ has joined #openstack-infra02:56
*** rfolco has joined #openstack-infra02:57
*** rmcall has joined #openstack-infra02:58
*** ChuckC has quit IRC02:59
*** ashleighfarnham has joined #openstack-infra03:00
*** nelsnelson has joined #openstack-infra03:01
*** baoli has joined #openstack-infra03:02
*** radez is now known as radez_g0n303:05
*** unicell has quit IRC03:07
*** rfolco has quit IRC03:09
*** yamahata has quit IRC03:12
*** ivar-lazzaro has quit IRC03:16
*** markvoelker has joined #openstack-infra03:18
*** ayoung has joined #openstack-infra03:19
*** rmcall has quit IRC03:20
*** ghostpl_ has joined #openstack-infra03:21
*** markvoelker has quit IRC03:22
*** ghostpl_ has quit IRC03:25
*** ChuckC_ is now known as ChuckC03:27
*** woodster_ has quit IRC03:30
*** dimtruck is now known as zz_dimtruck03:30
*** ashleighfarnham has quit IRC03:34
*** rkukura has quit IRC03:36
*** nelsnelson has quit IRC03:39
*** baoli has quit IRC03:42
*** david-lyle is now known as david-lyle_afk03:44
*** hdd_ has joined #openstack-infra03:44
*** ddieterl_ has quit IRC03:45
*** ddieterly has joined #openstack-infra03:45
*** Daisy_ has quit IRC03:46
*** nelsnelson has joined #openstack-infra03:46
*** ayoung has quit IRC03:49
*** fandi has quit IRC03:49
*** ddieterly has quit IRC03:50
*** oomichi has joined #openstack-infra03:51
*** baoli has joined #openstack-infra03:53
*** nelsnelson has quit IRC03:56
*** harlowja is now known as harlowja_away03:57
*** julim has quit IRC03:57
*** achanda has quit IRC03:57
*** vilobhmm has joined #openstack-infra03:58
*** esker has joined #openstack-infra04:04
*** Longgeek has joined #openstack-infra04:08
*** mriedem_away has quit IRC04:10
*** mriedem has joined #openstack-infra04:11
*** BharatK has joined #openstack-infra04:15
*** nelsnelson has joined #openstack-infra04:16
*** amotoki has joined #openstack-infra04:16
*** Daisy_ has joined #openstack-infra04:18
*** markvoelker has joined #openstack-infra04:19
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Refactor sources out of triggers  https://review.openstack.org/11899304:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add gerrit reviews into patchset approvals  https://review.openstack.org/9739004:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for negative requirements  https://review.openstack.org/10272604:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Configure triggers dynamically  https://review.openstack.org/11953404:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add support for 'connection' concept  https://review.openstack.org/12152804:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for triggers  https://review.openstack.org/11953204:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for sources  https://review.openstack.org/11953104:21
openstackgerritJoshua Hesketh proposed openstack-infra/zuul: Add base class for reporters  https://review.openstack.org/11953004:21
*** alexpilotti has quit IRC04:22
openstackgerritJoshua Hesketh proposed openstack-infra/project-config: zuul-swift-upload place mime icon into cell  https://review.openstack.org/16199504:22
*** gyee has quit IRC04:23
*** markvoelker has quit IRC04:25
*** tiswanso has quit IRC04:27
*** patrickeast has quit IRC04:28
*** otter768 has quit IRC04:28
*** baoli has quit IRC04:28
*** baoli has joined #openstack-infra04:31
*** achanda has joined #openstack-infra04:38
*** ddieterly has joined #openstack-infra04:46
*** wenlock has quit IRC04:48
*** koolhead17 has joined #openstack-infra04:49
*** ddieterly has quit IRC04:50
*** esker has quit IRC04:53
*** achanda has quit IRC04:54
*** esker has joined #openstack-infra05:00
*** vilobhmm1 has joined #openstack-infra05:01
openstackgerritSyed Ismail Faizan Barmawer proposed openstack-infra/project-config: Adding publish-to-pypi template to ironic-lib project  https://review.openstack.org/16201005:01
*** ujuc has joined #openstack-infra05:01
*** vilobhmm has quit IRC05:04
*** baoli has quit IRC05:06
purpc05:08
*** ghostpl_ has joined #openstack-infra05:09
*** garyh has quit IRC05:09
*** deepakcs has joined #openstack-infra05:11
*** nelsnelson has quit IRC05:13
*** ghostpl_ has quit IRC05:16
openstackgerritTimothy R. Chavez proposed openstack-infra/shade: Add 'rebuild' to shade  https://review.openstack.org/16106705:18
*** ashleighfarnham has joined #openstack-infra05:20
*** oomichi has quit IRC05:20
*** dannywilson has joined #openstack-infra05:21
*** markvoelker has joined #openstack-infra05:21
*** ashleighfarnham has quit IRC05:24
*** dannywilson has quit IRC05:25
*** markvoelker has quit IRC05:26
*** vilobhmm1 has quit IRC05:27
lifelessI just did a stupid - sorry. (released unittest2 1.0.0 on my friday)05:29
*** esker has quit IRC05:29
lifelesshowever its done. So if there is fallout, I'll make sure I can help.05:30
*** sdake_ has quit IRC05:32
*** esker has joined #openstack-infra05:33
*** bhunter71 has quit IRC05:41
*** ddieterly has joined #openstack-infra05:47
*** ddieterly has quit IRC05:51
*** achanda has joined #openstack-infra05:56
*** yfried has joined #openstack-infra05:56
*** esker has quit IRC06:00
*** resker has joined #openstack-infra06:02
*** _nadya_ has joined #openstack-infra06:03
*** michchap has quit IRC06:04
*** michchap has joined #openstack-infra06:04
*** xyang1 has quit IRC06:05
*** michchap has quit IRC06:05
*** michchap has joined #openstack-infra06:06
*** armax has quit IRC06:09
*** coolsvap_ has quit IRC06:11
*** Bsony has joined #openstack-infra06:13
*** ghostpl_ has joined #openstack-infra06:13
*** Bsony has quit IRC06:17
*** ghostpl_ has quit IRC06:18
*** HeOS has quit IRC06:18
*** mfink_ has joined #openstack-infra06:22
*** markvoelker has joined #openstack-infra06:22
*** unicell has joined #openstack-infra06:24
*** markvoelker has quit IRC06:27
openstackgerritMerged openstack-infra/project-config: Remove devstack-plugins group  https://review.openstack.org/16191706:28
*** otter768 has joined #openstack-infra06:29
*** mrmartin has joined #openstack-infra06:32
*** otter768 has quit IRC06:34
*** _nadya_ has quit IRC06:34
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Only run manuals niceness checks as needed  https://review.openstack.org/16203206:39
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Make manual checklang target voting  https://review.openstack.org/16203306:39
*** Longgeek has quit IRC06:39
*** mfink_ has quit IRC06:41
*** jyuso1 has joined #openstack-infra06:43
*** mrunge has joined #openstack-infra06:47
*** tnovacik has joined #openstack-infra06:48
openstackgerrityolanda.robla proposed openstack-infra/puppet-cgit: Add configurable Timeout settings on git vhost  https://review.openstack.org/16087106:48
*** ddieterly has joined #openstack-infra06:48
openstackgerrityolanda.robla proposed openstack-infra/puppet-gerrit: Add logging to fetchremotes call  https://review.openstack.org/16166306:51
*** hdd_ has quit IRC06:51
*** ddieterly has quit IRC06:52
*** unicell has quit IRC06:54
*** mrmartin has quit IRC06:54
*** vilobhmm has joined #openstack-infra06:54
*** phschwartz has quit IRC06:54
*** unicell has joined #openstack-infra06:54
*** phschwartz has joined #openstack-infra06:57
*** Bsony has joined #openstack-infra06:57
*** yamahata has joined #openstack-infra07:00
*** _nadya_ has joined #openstack-infra07:15
*** jamielennox is now known as jamielennox|away07:16
*** _nadya_ has quit IRC07:18
*** ivar-lazzaro has joined #openstack-infra07:18
*** Longgeek has joined #openstack-infra07:19
*** ivar-lazzaro has quit IRC07:22
*** mrmartin has joined #openstack-infra07:22
*** yamahata has quit IRC07:23
*** markvoelker has joined #openstack-infra07:24
*** scheuran has joined #openstack-infra07:27
*** markvoelker has quit IRC07:29
*** hichihara has quit IRC07:33
*** Bsony has quit IRC07:36
*** vilobhmm has quit IRC07:39
*** Longgeek has quit IRC07:43
*** teran_ has joined #openstack-infra07:44
*** achanda has quit IRC07:44
*** achanda has joined #openstack-infra07:44
*** teran has quit IRC07:46
*** mriedem has quit IRC07:47
*** mriedem has joined #openstack-infra07:47
GheRiveromorning all07:47
*** achanda has quit IRC07:48
*** ddieterly has joined #openstack-infra07:49
*** liusheng has quit IRC07:50
*** HeOS has joined #openstack-infra07:50
*** liusheng has joined #openstack-infra07:51
*** mpavone has joined #openstack-infra07:52
*** hamzy has quit IRC07:52
*** ddieterly has quit IRC07:53
*** camunoz has quit IRC07:54
*** mrmartin has quit IRC07:58
*** jerryz_ has quit IRC07:58
*** jerryz_ has joined #openstack-infra07:59
*** hamzy has joined #openstack-infra07:59
*** jcoufal has joined #openstack-infra08:01
*** zul has quit IRC08:01
*** markus_z has joined #openstack-infra08:01
*** ghostpl_ has joined #openstack-infra08:01
*** mrunge has quit IRC08:03
*** MaxV has joined #openstack-infra08:04
*** achanda has joined #openstack-infra08:05
*** teran has joined #openstack-infra08:07
*** ghostpl_ has quit IRC08:09
*** teran_ has quit IRC08:10
*** ashleighfarnham has joined #openstack-infra08:11
*** chlong has quit IRC08:11
*** mrunge has joined #openstack-infra08:11
*** mrunge_ has joined #openstack-infra08:11
*** MaxV has quit IRC08:11
*** mrunge_ has quit IRC08:12
*** mrunge has quit IRC08:12
*** mrunge has joined #openstack-infra08:13
*** SumitNaiksatam has quit IRC08:13
*** zul has joined #openstack-infra08:14
*** michchap has quit IRC08:16
*** SumitNaiksatam has joined #openstack-infra08:16
*** ashleighfarnham has quit IRC08:16
*** luqas__ has joined #openstack-infra08:16
*** MaxV has joined #openstack-infra08:18
*** krtaylor has quit IRC08:22
*** markvoelker has joined #openstack-infra08:25
*** unicell1 has joined #openstack-infra08:27
*** MaxV has quit IRC08:28
*** MaxV has joined #openstack-infra08:28
*** devvesa has joined #openstack-infra08:29
*** unicell has quit IRC08:29
*** notmyname has quit IRC08:30
*** devvesa has quit IRC08:30
*** otter768 has joined #openstack-infra08:30
*** markvoelker has quit IRC08:31
*** MaxV has quit IRC08:33
*** otter768 has quit IRC08:35
*** krtaylor has joined #openstack-infra08:35
*** afazekas has joined #openstack-infra08:39
*** mpaolino has joined #openstack-infra08:39
*** mpavone_ has joined #openstack-infra08:39
*** scheuran_ has joined #openstack-infra08:40
*** arxcruz has joined #openstack-infra08:40
*** stevemar has quit IRC08:42
openstackgerritSpencer Krum proposed openstack-infra/puppet-logstash: Flesh out the readme  https://review.openstack.org/15382008:43
*** mpavone has quit IRC08:43
*** scheuran has quit IRC08:43
*** jlanoux has joined #openstack-infra08:44
*** crc32 has quit IRC08:46
*** dtantsur is now known as dtantsur|bbl08:48
*** jgrimm is now known as zz_jgrimm08:49
*** ddieterly has joined #openstack-infra08:49
*** doude has joined #openstack-infra08:49
openstackgerritLin Yang proposed openstack-infra/zuul: Doc: fix minor typo  https://review.openstack.org/16206008:51
*** Ala has joined #openstack-infra08:53
*** ddieterly has quit IRC08:54
*** achanda has quit IRC08:54
*** che-arne has joined #openstack-infra08:54
*** krtaylor has quit IRC08:54
*** liusheng has quit IRC08:55
*** liusheng has joined #openstack-infra08:56
*** andreykurilin_ has joined #openstack-infra08:56
*** Hal has joined #openstack-infra08:57
*** achanda has joined #openstack-infra08:57
*** Hal is now known as Guest1681808:57
*** SumitNaiksatam has quit IRC08:58
*** MaxV has joined #openstack-infra08:59
*** bradm has quit IRC09:00
*** jistr has joined #openstack-infra09:02
*** hashar has joined #openstack-infra09:06
*** hashar is now known as hasharConf09:09
*** devvesa has joined #openstack-infra09:10
*** yamamoto has quit IRC09:10
*** krtaylor has joined #openstack-infra09:10
*** achanda has quit IRC09:12
*** achanda has joined #openstack-infra09:13
*** achanda has quit IRC09:14
*** achanda has joined #openstack-infra09:14
openstackgerritThomas Morin proposed openstack-infra/project-config: Add networking-bagpipe-l2 project  https://review.openstack.org/16129009:18
*** ghostpl_ has joined #openstack-infra09:19
*** achanda has quit IRC09:19
*** bradm has joined #openstack-infra09:22
openstackgerritGhe Rivero proposed openstack-infra/shade: Add cover to .gitignore  https://review.openstack.org/16206409:25
*** SumitNaiksatam has joined #openstack-infra09:26
*** markvoelker has joined #openstack-infra09:27
*** teran has quit IRC09:31
*** Longgeek has joined #openstack-infra09:32
*** dizquierdo has joined #openstack-infra09:33
*** markvoelker has quit IRC09:33
*** ihrachyshka has joined #openstack-infra09:33
*** AJaeger has joined #openstack-infra09:36
*** jistr is now known as jistr|biab09:36
*** AJaeger has quit IRC09:37
*** jp_at_hp has joined #openstack-infra09:40
*** mrmartin has joined #openstack-infra09:40
*** koolhead17 has quit IRC09:42
*** aarefiev has quit IRC09:42
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Ensure that gerritbot channels are in accessbot  https://review.openstack.org/16207009:43
*** Daisy_ has quit IRC09:45
*** aarefiev has joined #openstack-infra09:45
*** teran has joined #openstack-infra09:49
*** ddieterly has joined #openstack-infra09:50
*** dizquierdo has quit IRC09:51
*** teran has quit IRC09:53
*** BobBall_AWOL is now known as BobBall09:54
*** mpaolino has quit IRC09:54
*** ssam2 has joined #openstack-infra09:54
*** ddieterly has quit IRC09:55
kevinbentonanteaya: hey, how can i get added to the new networking-bigswitch-core group for the repo I requested?09:56
*** ihrachyshka has quit IRC09:57
*** mrmartin has quit IRC09:59
*** hasharConf is now known as hashar10:01
*** Longgeek has quit IRC10:06
*** mpaolino has joined #openstack-infra10:08
*** e0ne has joined #openstack-infra10:09
*** ghostpl_ has quit IRC10:11
*** ghostpl_ has joined #openstack-infra10:14
*** Longgeek has joined #openstack-infra10:14
*** andreykurilin_ has quit IRC10:18
*** andreykurilin__ has joined #openstack-infra10:18
*** skolekonov has joined #openstack-infra10:21
*** e0ne is now known as e0ne_10:22
*** liusheng has quit IRC10:23
*** liusheng has joined #openstack-infra10:24
openstackgerritValeriy Ponomaryov proposed openstack-infra/system-config: Enable logging of #openstack-manila  https://review.openstack.org/16120010:25
*** markvoelker has joined #openstack-infra10:29
*** otter768 has joined #openstack-infra10:31
*** _nadya_ has joined #openstack-infra10:32
*** e0ne_ has quit IRC10:33
*** teran has joined #openstack-infra10:33
*** markvoelker has quit IRC10:35
*** otter768 has quit IRC10:36
*** teran_ has joined #openstack-infra10:36
*** ZZelle has quit IRC10:37
*** andreykurilin__ has quit IRC10:38
*** teran has quit IRC10:39
*** e0ne has joined #openstack-infra10:39
*** jistr|biab is now known as jistr10:39
*** liusheng has quit IRC10:41
*** liusheng has joined #openstack-infra10:42
*** Guest16818 has quit IRC10:44
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Support translation of RST files  https://review.openstack.org/16209310:47
*** ddieterly has joined #openstack-infra10:51
*** ddieterly has quit IRC10:55
*** gmann_ has joined #openstack-infra10:59
*** dims has joined #openstack-infra11:00
*** hashar has quit IRC11:01
*** mpaolino has quit IRC11:07
*** e0ne is now known as e0ne_11:07
*** BobBall has quit IRC11:10
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Ensure that gerritbot channels are in accessbot  https://review.openstack.org/16207011:16
*** e0ne_ has quit IRC11:18
*** deepakcs has quit IRC11:19
*** yamamoto has joined #openstack-infra11:20
*** mpaolino has joined #openstack-infra11:21
*** chlong has joined #openstack-infra11:22
*** _nadya_ has quit IRC11:23
*** yamamoto has quit IRC11:24
*** ldnunes has joined #openstack-infra11:26
*** jcoufal_ has joined #openstack-infra11:28
*** markvoelker has joined #openstack-infra11:31
*** jcoufal has quit IRC11:31
openstackgerritIgor Gnatenko proposed openstack-infra/jenkins-job-builder: Add support of multiple credentials for ssh-agent-credentials wrapper  https://review.openstack.org/15668111:32
*** markvoelker has quit IRC11:36
*** dtantsur|bbl is now known as dtantsur11:39
*** ujuc has quit IRC11:39
*** BharatK has quit IRC11:42
openstackgerrityolanda.robla proposed openstack-infra/system-config: Expose jenkins_gitfullname and jenkins_gitemail  https://review.openstack.org/16169511:44
*** hashar has joined #openstack-infra11:46
*** e0ne has joined #openstack-infra11:47
*** flaper87 has quit IRC11:47
*** flaper87 has joined #openstack-infra11:47
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Ensure that gerritbot channels are in accessbot  https://review.openstack.org/16207011:51
*** ddieterly has joined #openstack-infra11:52
*** luqas__ has quit IRC11:52
*** BharatK has joined #openstack-infra11:54
*** ddieterly has quit IRC11:56
*** yamamoto has joined #openstack-infra11:58
*** pc_m has joined #openstack-infra12:01
openstackgerrityolanda.robla proposed openstack-infra/puppet-jenkins: Add configurable timeout for jjb update  https://review.openstack.org/16212112:01
openstackgerrityolanda.robla proposed openstack-infra/system-config: Install proper ruby1_9_1 packages for Debian Jessie  https://review.openstack.org/16212412:11
*** alexpilotti has joined #openstack-infra12:13
openstackgerritAleksey Ripinen proposed openstack-infra/storyboard: Tasks branches, milestones and projects ids  https://review.openstack.org/16212612:16
*** baoli has joined #openstack-infra12:16
*** sambetts has quit IRC12:17
*** sambetts has joined #openstack-infra12:17
*** adalbas has joined #openstack-infra12:17
*** gmann_ has quit IRC12:20
*** ZZelle has joined #openstack-infra12:25
*** rfolco has joined #openstack-infra12:26
*** BobBall has joined #openstack-infra12:26
*** Hal has joined #openstack-infra12:29
*** Hal is now known as Guest9419412:30
*** baoli_ has joined #openstack-infra12:31
*** markvoelker has joined #openstack-infra12:32
*** otter768 has joined #openstack-infra12:32
openstackgerritTimothy R. Chavez proposed openstack-infra/shade: Add 'rebuild' to shade  https://review.openstack.org/16106712:32
HenryGanteaya: ping12:33
*** baoli has quit IRC12:34
HenryGI am wondering why this change https://review.openstack.org/161569 does not seem to be taking effect in the gate? cc: clarkb12:35
HenryGhttp://logs.openstack.org/84/160884/2/check/check-neutron-dsvm-functional/590e659/console.html#_2015-03-05_22_48_03_12212:36
*** cdent has joined #openstack-infra12:36
*** markvoelker has quit IRC12:36
*** otter768 has quit IRC12:36
*** MaxV has quit IRC12:37
*** BobBall has quit IRC12:37
*** koolhead17 has joined #openstack-infra12:38
*** yamamoto has quit IRC12:38
*** aysyd has joined #openstack-infra12:39
*** ja has joined #openstack-infra12:39
*** yamamoto has joined #openstack-infra12:40
*** dhritishikhar has joined #openstack-infra12:41
openstackgerritPaul Michali proposed openstack-infra/project-config: Partial revert of func job setup for VPNaaS repo  https://review.openstack.org/16213512:44
*** cdent has quit IRC12:44
*** ssam2 has quit IRC12:45
*** ssam2 has joined #openstack-infra12:45
*** MaxV has joined #openstack-infra12:46
*** yamamoto has quit IRC12:46
jawhich distribution is infra using right now for zuul/jenkins/etc?  I'm in the process of planning a 3rd party CI system and would prefer using something known to work with the infra components we'd be reusing.  the only hint I've found so far is an example in the Ci automation pub at #link http://docs.openstack.org/infra/publications/ci-automation/#%2818%2912:49
*** yamamoto has joined #openstack-infra12:51
*** ddieterly has joined #openstack-infra12:52
*** ujuc has joined #openstack-infra12:54
*** ddieterly has quit IRC12:57
nibalizermost services run on ubuntu 14.0412:58
mordrednibalizer: why are you awake?12:59
mordrednibalizer: or are you _still_ awake?12:59
* mordred is impressed either way12:59
*** cdent has joined #openstack-infra13:03
*** mpavone_ has quit IRC13:04
nibalizerbecause gsl13:04
nibalizeri have to sleep before the gsl comes on13:04
nibalizerother wise this ends up happening13:04
nibalizerhttp://wiki.teamliquid.net/starcraft2/2015_Global_StarCraft_II_League_Season_1/Code_S13:04
*** markvoelker has joined #openstack-infra13:06
*** woodster_ has joined #openstack-infra13:06
*** yamamoto has quit IRC13:07
*** dizquierdo has joined #openstack-infra13:07
*** dprince has joined #openstack-infra13:07
janibalizer, thanks.  is there anything known to have problems on 14.04?  on 3rd party CI owner mentioned an indistinct problem "a while ago" that led them to use 12.04, be he was "pretty sure" it had been fixed by now hence my q here.13:08
jaif no known problems, I'll just plan on using 14.04 for the whole zuul/etc smash13:08
nibalizerja: well actaully13:09
nibalizeri would say most of it is going to 14.0413:09
nibalizera lot of it is still on 12.0413:09
nibalizersorry13:09
nibalizerja: check this out http://puppetboard.openstack.org/fact/lsbdistcodename13:10
nibalizerthat gives you an idea of where things are13:10
openstackgerritMonty Taylor proposed openstack-infra/shade: Small fixes found working on ansible modules  https://review.openstack.org/16214013:11
*** e0ne is now known as e0ne_13:12
openstackgerritLiKai proposed openstack/requirements: Update xstatic-font-awesome to 4.3.0  https://review.openstack.org/16214213:13
*** bobba has joined #openstack-infra13:13
*** bobba has quit IRC13:14
janibalizer, if you know offhand what is "Final" on that puppetboard?  clicking through its link isn't informative.  but thanks for the trusty vs precise answer.  so nodepool and puppetmaster can definitely cope with trusty from that list.  any of the others we're in less-charted territory with.13:15
*** bobba has joined #openstack-infra13:15
*** bobba is now known as Guest7064113:15
*** Guest70641 is now known as BobBall13:15
Kiallnibalizer: I'm intrigued - what's the difference between the two pie charts, just the orientation?13:15
Kiallja: lsbdistcodename Final / lsbdistdescription CentOS release 6.6 (Final)13:16
Kiallhttp://puppetboard.openstack.org/fact/lsbdistrelease might be a tad clearer ;)13:16
jathanks Kiall13:16
nibalizerKiall: kindof, the two charts got removed in later puppetboard13:17
nibalizerone orders by value and one orders by number of things in the slice13:17
*** mpavone_ has joined #openstack-infra13:17
*** ujuc has quit IRC13:18
nibalizerhttp://puppetboard.openstack.org/fact/physicalprocessorcount might show it better13:18
jaright, I see it; starting point is straight up; left one is by frequency, right alpha by name13:18
*** BobBall has quit IRC13:21
openstackgerritJulien Danjou proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack  https://review.openstack.org/16214613:22
*** BobBall has joined #openstack-infra13:22
*** mbacchi has joined #openstack-infra13:22
*** e0ne_ is now known as e0ne13:22
*** kgiusti has joined #openstack-infra13:23
openstackgerritMonty Taylor proposed openstack-infra/shade: Allow for passing cache class in as a parameter  https://review.openstack.org/16214813:24
openstackgerritJulien Danjou proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack  https://review.openstack.org/16214613:25
openstackgerritSyed Ismail Faizan Barmawer proposed openstack-infra/project-config: Adding publish-to-pypi template to ironic-lib project  https://review.openstack.org/16201013:25
openstackgerritJulien Danjou proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack  https://review.openstack.org/16214613:25
*** dkliban_afk is now known as dkliban13:27
*** dhritishikhar has quit IRC13:30
*** mwagner_lap has joined #openstack-infra13:30
nibalizerKiall: you can probably hit our internal puppetboard to see how far its come in the year or so since the upstream one went up13:34
*** vhoward has left #openstack-infra13:35
*** weshay has joined #openstack-infra13:35
*** HeOS has quit IRC13:37
Kiallnibalizer: If I knew the URL, I'd be able to! But I dont ;)13:37
nibalizerhipchat'd13:38
*** ashleighfarnham has joined #openstack-infra13:39
openstackgerrityolanda.robla proposed openstack-infra/puppet-jenkins: Add configuration for HipChat in jjb  https://review.openstack.org/16215213:39
*** dboik has quit IRC13:43
*** gordc has joined #openstack-infra13:43
*** ashleighfarnham has quit IRC13:44
*** radez_g0n3 is now known as radez13:46
*** nelsnelson has joined #openstack-infra13:47
*** SumitNaiksatam has quit IRC13:48
*** dustins has joined #openstack-infra13:49
*** mpavone_ has quit IRC13:49
openstackgerritSyed Ismail Faizan Barmawer proposed openstack/requirements: Add ironic-lib to project.txt and global-requirements.txt  https://review.openstack.org/16160313:52
*** julim has joined #openstack-infra13:52
*** sigmavirus24_awa is now known as sigmavirus2413:54
*** yamamoto has joined #openstack-infra13:54
*** HeOS__ has joined #openstack-infra13:56
sigmavirus24Is there an issue with nodes in zuul's pool using old versions of project-config? os-ansible-deployment keeps seeing http://logs.openstack.org/75/161975/3/check/os-ansible-deployment-dsvm-check-commit/06ecd42/console.html#_2015-03-06_10_01_58_615 pop up every once and a while which is the result of a version of project config from several days ago13:56
*** miqui has joined #openstack-infra13:56
*** HeOS_ has quit IRC13:56
*** mpavone has joined #openstack-infra13:57
*** BharatK has quit IRC13:57
*** gibi_ has joined #openstack-infra13:57
*** AJaeger has joined #openstack-infra13:59
*** luqas__ has joined #openstack-infra13:59
*** annegentle has joined #openstack-infra13:59
*** mattfarina has joined #openstack-infra14:00
*** mpavone has quit IRC14:00
*** annegentle has quit IRC14:00
*** annegentle has joined #openstack-infra14:01
*** ihrachyshka has joined #openstack-infra14:01
*** gibi has quit IRC14:01
*** Longgeek has quit IRC14:05
*** fifieldt_ has quit IRC14:05
Shrewsmordred: should shade's caching be 'opt-in' as well?14:06
*** zz_jgrimm is now known as jgrimm14:07
*** dboik has joined #openstack-infra14:07
Shrewsif os-client-config is used, the behaviour will change to 'opt-in' anyway with your recent o-c-c change if no cache is defined14:07
*** sc68cal_ has joined #openstack-infra14:07
*** sc68cal_ has quit IRC14:07
*** sc68cal_ has joined #openstack-infra14:07
*** dboik has quit IRC14:07
*** dboik_ has joined #openstack-infra14:08
*** bcourt has joined #openstack-infra14:08
*** sc68cal has quit IRC14:09
*** salv-orl_ has joined #openstack-infra14:09
*** salv-orlando has quit IRC14:09
*** bcourt has quit IRC14:10
Shrewsleft comment on the review14:11
mordredShrews: it should be opt-in with SpamapS's change14:12
Shrewsreview #?14:12
*** tnovacik has quit IRC14:13
Shrewsoh, the dependency14:13
*** dizquierdo has quit IRC14:14
*** ddieterly has joined #openstack-infra14:15
*** resker has quit IRC14:16
openstackgerritgordon chung proposed openstack-infra/project-config: add experimental ceilometer elasticsearch db jobs  https://review.openstack.org/16082414:16
*** garyh has joined #openstack-infra14:16
*** markus_z has quit IRC14:17
*** Longgeek has joined #openstack-infra14:20
*** signed8bit has joined #openstack-infra14:21
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add configurable url for OpenStack status  https://review.openstack.org/16216914:25
*** SumitNaiksatam has joined #openstack-infra14:28
*** peristeri has joined #openstack-infra14:29
*** Ala has quit IRC14:29
*** dansmith is now known as superdan14:30
*** annegentle is now known as superanne14:31
*** otter768 has joined #openstack-infra14:33
*** chlong has quit IRC14:34
*** dims has quit IRC14:37
*** otter768 has quit IRC14:37
*** dims has joined #openstack-infra14:38
*** bhunter71 has joined #openstack-infra14:38
*** tiswanso has joined #openstack-infra14:38
openstackgerritDavid Lyle proposed openstack-infra/project-config: Adding xstatic notifications to openstack-horizon  https://review.openstack.org/16217414:38
*** Ala has joined #openstack-infra14:40
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for cinder CLI race bug 1429151  https://review.openstack.org/16217614:40
openstackbug 1429151 in Cinder "SimpleReadOnlyCinderClientTest.test_cinder_encryption_type_list fails randomly" [Undecided,New] https://launchpad.net/bugs/142915114:40
*** ddieterly has quit IRC14:41
*** nelsnelson has quit IRC14:41
*** ddieterl_ has joined #openstack-infra14:41
*** ZZelle has quit IRC14:43
*** sdake_ has joined #openstack-infra14:43
dhellmannanteaya: I'm confused by your comment about the tarball site on https://review.openstack.org/#/c/161447/4 -- I thought that was used by publish-to-pypi?14:43
*** tonytan4ever has joined #openstack-infra14:44
*** alaski is now known as lascii14:44
jrollis there a jenkins thing to upload to swift?14:46
jrollor did y'all just write code to do things like that14:46
sigmavirus24jroll: code was written to do that14:47
sigmavirus24I believe clarkb wrote it14:47
*** sdake__ has joined #openstack-infra14:47
jrollok, thanks14:47
* jroll finds his swift client14:48
*** mrmartin has joined #openstack-infra14:48
*** sdake_ has quit IRC14:48
sigmavirus24jroll: I believe https://review.openstack.org/#/c/156788/ added it14:49
mordredjroll: we try to not have jenkins itself do things14:49
mordredjroll: because two reasons a) jenkins breaks b) we're working on getting rid of it14:50
jrollo.o14:50
jrollmordred: getting rid of jenkins in favor of?14:50
mordredjroll: b) follows from a) in a non-insignificant way14:50
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add configurable graphite url on zuul status  https://review.openstack.org/16217714:50
jrollright14:50
mordredjroll: zuul and nodepool directly orchestrating work14:50
jrollinteresting14:50
mordredjroll: you should check out jeblair's zuul v3 email if you haven't already14:51
*** mtanino has joined #openstack-infra14:51
jrollmordred: neat, thanks14:51
jrollI'm setting up a jenkins for our team to just build some containers and whatnot, so trying to keep it simple :)14:51
jrollvastly different scale :P14:51
mordredjroll: but before zuul v3 was hatched, the plan for getting rid of jenkins involved just running gearman workers on the nodes like turbohipster does14:51
mordredjroll: yah - I mean, jenkins is fine for the normal workgroup size stuff14:52
*** boris-42 has quit IRC14:52
mordredalthough we do hope that once we get zuul v3 going, it'll be easy to scale it _down_ to do simple tasks like that too14:52
*** rmcall has joined #openstack-infra14:52
openstackgerritDoug Hellmann proposed openstack-infra/project-config: import aeromancer to stackforge  https://review.openstack.org/16144714:52
openstackgerritDoug Hellmann proposed openstack-infra/project-config: Add detail to gerrit ACL normalization errors  https://review.openstack.org/16191414:52
*** amitgandhinz has joined #openstack-infra14:52
jrollmordred: cool14:53
dhellmannanteaya: let me know if ^^ looks more like what I should have if what I want is for the CI system to manage my releases14:53
jrollthanks for the info14:53
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Adding publish-to-pypi template to ironic-lib project  https://review.openstack.org/16201014:53
AJaegerihrachyshka: what's the problem with https://review.openstack.org/161569 ?14:54
ihrachyshkaAJaeger, I still see failures in e.g. neutron/juno. But need to recheck to make sure it's still the case.14:54
AJaegerihrachyshka: do you have a recent run as reference?14:55
*** e0ne is now known as e0ne_14:55
AJaegerdhellmann, jd__ : Could you review https://review.openstack.org/157472 and https://review.openstack.org/158379, please?14:55
ihrachyshkaAJaeger, https://review.openstack.org/#/c/161930/ ?14:56
openstackgerrityolanda.robla proposed openstack-infra/system-config: Add configurable graphite url on zuul status  https://review.openstack.org/16217714:56
openstackgerritMerged openstack-infra/elastic-recheck: Add query for cinder CLI race bug 1429151  https://review.openstack.org/16217614:56
openstackbug 1429151 in Cinder "SimpleReadOnlyCinderClientTest.test_cinder_encryption_type_list fails randomly" [High,Confirmed] https://launchpad.net/bugs/142915114:56
*** amotoki has quit IRC14:57
dhellmannAJaeger: +214:57
AJaegerdhellmann: thanks!14:57
AJaegerihrachyshka: looks like the old job definition was used. mordred, clarkb: Could you check http://logs.openstack.org/30/161930/1/check/check-neutron-dsvm-functional/7aa1c5b/console.html - it seems that patch 161569 was not applied14:59
ihrachyshkaAJaeger, maybe puppet not applied to jenkins? I don't know much about mechanics.15:00
*** e0ne_ has quit IRC15:01
*** persia has quit IRC15:01
AJaegerihrachyshka: that would explain it - that's why I need jeblair, mordred or clarkb to check...15:01
ihrachyshkaAJaeger, afaik we were not sure that it will work since the author is not fluent in project-config speak15:01
ihrachyshkait was more a wild guess?15:01
jd__AJaeger: sure15:02
AJaegerthanks, jd__ !15:02
ihrachyshkaah, missed the 2nd patchset there. then I guess it's more grounded15:02
*** yfried has quit IRC15:04
AJaegerjeblair, mordred, clarkb: https://review.openstack.org/161447 just failed jobs - and looking at the failures, I only get "file not found". Any problems with Jenkins/Zuul/gearman?15:05
AJaegerand two more failures just coming in. Something looks broken ;(15:06
jeblairhttps://jenkins02.openstack.org/job/gate-infra-puppet-apply-precise/1443/console15:06
jeblair2015-03-06 15:01:39.899 | ImportError: No module named glob215:07
jeblairi'm guessing a base image changed ?15:07
mordredyay!15:07
*** dims is now known as dimsum__15:07
jeblairthat was bare-precise-rax-dfw15:08
*** deepakcs has joined #openstack-infra15:08
*** mrmartin has quit IRC15:08
AJaegerjeblair: thanks for looking into this!15:08
jeblairfound one on rax-iad too15:09
*** achuprin_ has quit IRC15:10
jeblairwe started using new images there about 20 mins ago15:11
*** radez is now known as radez_g0n315:12
*** hdd_ has joined #openstack-infra15:13
*** persia_ has joined #openstack-infra15:13
*** persia_ has quit IRC15:13
*** persia_ has joined #openstack-infra15:13
*** jlanoux has quit IRC15:14
jeblairoh15:15
jeblairwe just merged a change to the script to use glob2, but did not add it to the venv15:15
jeblairthis may be a signal the script is getting a bit too big to be a standalone script.15:15
*** armax has joined #openstack-infra15:15
jeblairperhaps we should fold it into zuul.  we already install zuul to get cloner, so it would actually be simpler anywya.15:16
*** wenlock has joined #openstack-infra15:16
mordred++15:16
mordredjeblair: I agree 100%15:16
jrollmordred: is this thing awful or awesome? https://wiki.jenkins-ci.org/display/JENKINS/JClouds+Plugin15:17
mordredjroll: it's not usable by us15:17
jrollmordred: not for you15:17
jrollfor me15:17
mordredjroll: but we funded its creattion :)15:17
jroll:P15:17
mordredjroll: dunno15:17
mordredI've literally never been able to get it to work15:17
jrollhuh15:17
mordredbecause clouds are too flaky15:17
jrollprobably needs a real service catalog or something15:17
mordredthat's why we wrote nodepool15:17
jrollI just want to use the storage bits15:17
mordredno - it's just that handling all of the ways clouds fail is hard15:17
mordredoh - well that's probably fine15:18
jrollwhich shouldn't be super flaky15:18
jrollyeah15:18
jrollok, thanks :)15:18
*** wenlock has quit IRC15:19
AJaegerjeblair: argh, the swift uploader change15:19
*** changbl has quit IRC15:19
jeblairAJaeger: hrm, that change did come with a command to install glob215:20
jeblairAJaeger: so i'm back to not quite understanding what's wrong...15:20
*** esker has joined #openstack-infra15:20
*** mrmartin has joined #openstack-infra15:20
*** r-daneel has joined #openstack-infra15:20
*** timcline has joined #openstack-infra15:20
*** wenlock has joined #openstack-infra15:21
openstackgerritFilip Blaha proposed openstack-infra/project-config: Congress client repository clone fix  https://review.openstack.org/16218815:21
*** timcline_ has joined #openstack-infra15:22
jeblairoh!15:23
jeblairit's because puppet is stopped on nodepool.o.o15:23
*** marun has joined #openstack-infra15:24
jeblairso the image scripts are out of sync with the project_config repo that gets cloned on the image15:25
AJaegerjeblair: see above the discussion with ihrachyshka, where we looked into a job that looked it was an old job, so puppet did not update jenkins15:25
AJaegeris puppet disabled on the jenkins notes as well?15:26
*** timcline has quit IRC15:26
jeblairAJaeger: it shouldn't be15:26
jeblairAJaeger: except something is wrong with jenkins0415:26
jeblairAJaeger: http://puppetboard.openstack.org/15:26
jeblairAJaeger: so if it ran there, it would be outdated15:26
*** doude has quit IRC15:27
jeblairi think it's safe to re-enable puppet on nodepool15:27
*** dmsimard_away is now known as dmsimard15:28
*** jlanoux has joined #openstack-infra15:28
jeblairclarkb, AJaeger: i'm going to re-enable puppet on nodepool and delete all current bare- images15:28
*** hdd_ has quit IRC15:28
ihrachyshkaarmax, ^^15:28
*** ashleighfarnham has joined #openstack-infra15:29
AJaegerjeblair: seems it was run on jenkins03 - scroll back for discussion with ihrachyshka ...15:29
AJaegerjeblair: But let's first fix the images.15:29
armaxihrachyshka: but I am still confused15:29
jeblairmordred: you want to see why puppet isn't running on jenkins04?15:30
armaxihrachyshka: if I look at the expression [[ ! "$BRANCH_OVERRIDE" =~ ^stable/(icehouse|juno)$ ]] || [[ ! "$ZUUL_BRANCH" =~ ^stabl15:30
armaxe/(icehouse|juno)$ ]] as clarkb suggested15:30
armaxwe should still be getting in the if branch for stable branches even if we don’t want to15:30
ihrachyshkale sigh15:31
armaxihrachyshka: as a matter of fact, the aim of the fix should be ‘not to get into the if branch’ if we are stable15:31
ihrachyshkaarmax, remove '!'s?15:31
armaxihrachyshka: my bad for copying and pasting not paying any attention at what I was told :O15:31
armaxI think that we should not use BRANCH_OVERRIDE at all15:32
*** changbl has joined #openstack-infra15:32
armaxihrachyshka: it sounds to me that it is == default regardless of the branch, so what’s the point in using it to determine whether we are on a stable branch or not?15:32
*** e0ne has joined #openstack-infra15:32
armaxihrachyshka: but I have very little clue on how the infra stuff works15:33
armaxihrachyshka: apparently I am not alone :)(15:33
ihrachyshkaarmax, neither me, we're both blind15:33
*** tiswanso has quit IRC15:33
*** ashleighfarnham has quit IRC15:33
*** radez_g0n3 is now known as radez15:34
*** yamamoto has quit IRC15:34
*** zz_dimtruck is now known as dimtruck15:34
*** yamamoto has joined #openstack-infra15:34
*** tiswanso_ has joined #openstack-infra15:34
*** achuprin_ has joined #openstack-infra15:35
AJaegerarmax, ihrachyshka you confused me as well with this change ;)15:35
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Remove BRANCH_OVERRIDE from bool condition  https://review.openstack.org/16219915:36
armaxAJaeger: ^^15:36
clarkbno dont remove it ..15:36
ihrachyshkaAJaeger, so I guess it's both 'jenkins not updated' and 'code is wrong' issues15:36
*** tiswanso_ has quit IRC15:36
armaxclarkb: ok, can you elaborate why?15:36
clarkbshould || be && I may have flipped logic15:36
clarkbbecause if you are overridden you dont want to be in setting the projects...15:36
AJaegermight be, ihrachyshka. Note the job might fail due to the broken images.15:37
armaxclarkb: but I still don’t understand why we’d want it to determine a condition when the value doesn’t change from one branch to another15:37
clarkbthe idea is if icehouse or juno dont do this15:37
clarkbso if someone makes a job that says "this is icehouse" then you dont want to do that15:37
armaxso if branch_override is default regarless of whether we run master or juno15:37
clarkbits not necessarily15:37
armaxI beg to understand why we’d want to use it? in any condition OR or AND15:37
clarkbit may be icehouse or juno15:38
clarkbthats the whole point15:38
armaxclarkb: when?15:38
clarkbwhen people make jobs with the override15:38
*** dizquierdo has joined #openstack-infra15:38
mkj_wkI could use some guidance on normal Gerrit expectations, for my new feature changeset, now in its 20th revision... Would it be considered polite for me to +1 my own patch as an indication that I think it's ready for merge and would appreciate review/merge?15:39
lxsliHmm not sure this is the right place to raise this, but has anyone suggested colour-coding badges at Vancouver?15:39
armaxclarkb: this is utterly confusing15:39
lxsliPrimarily for the benefit of security, and hence decreasing queuing times15:39
armaxclarkb: so I can have BRANCH_OVERRIDE=default and ZUUL_BRANCH=juno?15:40
clarkbarmax yes15:40
armaxclarkb: what does that mean?15:40
jeblairmkj_wk: you could just leave a comment to that effect15:40
clarkbit means zuul has given you a change from juno so do the default behavior for juno15:40
AJaegermkj_wk: what's the URL for the patch?15:41
clarkbarmax but override juno branch master means this is a change from master but test it against juno15:41
*** ChuckC_ has joined #openstack-infra15:41
mkj_wkAJaeger: https://review.openstack.org/#/c/158877/15:41
armaxclarkb: for the sake of what it needs to be done, I still reckon that we don’t need BRANCH_OVERRIDE at all then15:41
*** ChuckC has quit IRC15:42
mkj_wkjeblair: thanks :)15:42
clarkbarmax then please completely remove it from your job.15:42
armaxclarkb: that said, what do you mean?15:42
clarkb162199 is still wrong regardless15:42
armaxlike remove the export?15:43
AJaegermkj_wk: ah, a patch for git-review. Let's ask clarkb and jeblair kindly for a review15:43
clarkband the template parameter15:43
*** tiswanso has joined #openstack-infra15:43
armaxclarkb: so the options are two, if I understand you correctly, flip the || to && or get rid of the BRANCH_OVERRIDE entirely?15:44
clarkbyes15:44
armaxclarkb: ok then, since I am a lazy moron, I am going to go for the easiest change15:44
mkj_wkAJaeger: that brings up the next politeness question -- does one normally add someone else as a reviewer to ones own patch, or just ping on IRC?15:44
openstackgerritArmando Migliaccio proposed openstack-infra/project-config: Flip the bool condition  https://review.openstack.org/16219915:45
armaxclarkb: assuming that ZUUL_BRANCH is a stable branch this should be good now15:46
*** ChuckC_ has quit IRC15:46
AJaegermkj_wk: I prefer pinging on IRC, people might not see the adding15:46
superanneanteaya: I'm trying to reach as many vendor driver writers as possible, can you think of a list that might be useful? I've been using -docs and -dev mostly.15:46
mkj_wkok, well I suspect jeblair and clarkb saw your mention as a ping then :)15:46
superanneanteaya: wondering about 3rd party list15:47
AJaegermkj_wk: hope so. But your patch is large, so reviewing needs time15:47
jeblairmkj_wk: we should ask mordred and/or fungi to look at it, though fungi is on vacation right now15:48
*** achuprin_ has quit IRC15:48
anteayasuperanne: this should get you started: https://review.openstack.org/#/admin/groups/270,members15:48
mkj_wkAJaeger: it was quite small when I started, but it's much better tested now15:48
superanneanteaya: perfect, thanks15:48
*** arxcruz has quit IRC15:48
anteayasuperanne: they also should all be listed here: https://wiki.openstack.org/wiki/ThirdPartySystems15:49
AJaegermkj_wk: ;) Please have patience, reviewing can take some time.15:49
mkj_wkjeblair: yeah, I saw fungi head off on vacation just as I was really getting to work on the feature15:49
anteayasuperanne: good luck15:49
mkj_wkAJaeger: oh, I'm patient. I just didn't want it to be stuck in limbo because I didn't realize I was supposed to ping someone15:49
mkj_wkAJaeger: I've been on the other end of this elsewhere15:49
anteayamind if I ask what your project is?15:49
superanneanteaya: will cross posting super annoy them?15:49
anteayacross posting what?15:49
superanneanteaya: it's informing them of the driver doc spec AJaeger completed15:49
superanneanteaya: basically "here's your template, expectations, and deadline for kilo"15:50
superanneanteaya: cross posting to 3rd party, -dev, and -docs15:50
AJaegermkj_wk: you don't have to ping, people will review at their time.15:50
openstackgerritJulien Danjou proposed openstack-infra/project-config: Move Gnocchi from Stackforge to OpenStack  https://review.openstack.org/16214615:50
mkj_wksuperanne: you might care to look at it, since I was attempting to implement a feature that you asked for almost two years ago. ☺15:50
*** skolekonov has quit IRC15:50
mkj_wkAJaeger: thanks for the clues :)15:51
anteayasuperanne: oh in that case, draft up and email and we can send it to third-party-announce mailing list15:51
anteayasince they all are expected to be subscribed to that15:51
anteayajeblair: any objections to sending the driver doc spec to -announce?15:52
superanneanteaya: ah nice. if I send it to -docs and -dev and send you a link is that what you need?15:52
anteayano don't spam dev with it15:52
* AJaeger will be back later, time for some cooking...15:52
anteayajust draft it in an etherpad so I can take a peek15:52
anteayathen we just send it to -announce and -docs at teh same time15:53
*** sdake__ has quit IRC15:53
*** AJaeger is now known as AJaeger_cooking15:53
jeblairanteaya: no15:54
anteayajeblair: okay15:54
anteayajeblair: what do you suggest?15:54
jeblairanteaya: that you send it to announce.  you asked if i had objections, and i said i had no objections.15:54
anteayaah15:55
anteayathank you, great15:55
sdaguejeblair: so on this https://review.openstack.org/#/c/161911/, is that actually what you want to do? The devstack plugin architecture was really more designed with the plugin being in the project repo.15:55
anteayasuperanne: okay give me a few minutes15:55
jeblairsdague: oh okay, are there projects that implement that?  (why are there devstack-plugin projects then?)15:56
sdaguethe only times we've built fully new repos is for stuff like glusterfs support, which don't really have a project repo to live in, as it's not a new project but a config tweak15:56
sdaguestackforge/ec2-api15:56
sdaguea bunch of the neutron plugins15:56
sdagueI think gnocchi as well15:56
*** ayoung has joined #openstack-infra15:57
sdagueI pushed it for tuskar recently, I don't remember if they landed it or not15:57
anteayasuperanne: okay so when you are ready just cc third-party-announce@lists.openstack.org when you send it out to -docs15:57
anteayasuperanne: keep in mind these folks are used to having deadlines and consequence15:57
anteayaif they don't have "this is waht will happen if you don't" they will just get confused and come in here and ask15:58
*** nelsnelson has joined #openstack-infra15:58
anteayaso my take on what happens if they don't is, they don't get included in the docs15:58
anteayawhich is fine with me15:58
jeblairsdague: okay, i see how that works, thanks.  i'll abandon it then.15:58
anteayasuperanne: the -announce list is moderated so I will keep an eye out for it and usher it through when I see it15:58
*** superanne has quit IRC15:59
jeblairsdague: does that means 161887 is a bad idea?15:59
*** achuprin_ has joined #openstack-infra16:00
*** superanne has joined #openstack-infra16:00
sdaguewell... I think it won't be used a ton16:00
openstackgerritgordon chung proposed openstack-infra/project-config: add experimental ceilometer elasticsearch db jobs  https://review.openstack.org/16082416:01
*** stevemar has joined #openstack-infra16:01
jeblairsdague: still do it, but just document that it should only be used if in-repo is not appropriate?16:01
sdagueyes, I think that makes sense16:01
openstackgerritBrian Hunter proposed openstack-infra/system-config: grammar updates for gerrit documentation  https://review.openstack.org/16018516:02
jeblairsdague: k, thx.  i'll write a patch to copy the devstack stuff into trove16:02
ihrachyshkaclarkb, just wondering, why would I want to run master patch with juno default?16:03
*** hashar is now known as hasharAway16:03
jeblairclarkb: since it's early still, i'd like to kick off bare-* image builds; how should i go about that?16:03
sdaguethe places I expect that we'll need it is where there is some 3rd party software outside the openstack space you need to configure for a thing. So filesystems like glusterfs and ceph (which we'll try to get into one of these as well).16:04
sdagueall the neutron stuff has driver trees now, so that should be fine16:04
clarkbfor branchless projects testing against other things16:04
clarkbfor example neutronclient16:04
sdagueor tempest16:04
*** maune has joined #openstack-infra16:04
clarkbjeblair bare-* are not dib so just image-update16:04
clarkbjeblair I like to run one command per region for parallel builds in screen16:05
jeblairclarkb: ok, thanks16:05
superdanhi all, the devstack patch merged that this depends on: https://review.openstack.org/#/c/160893/16:06
sdaguejeblair: unrelated, been playing with https://pypi.python.org/pypi/pycscope/ and emacs integration this morning. Thought you might appreciate it if you haven't seen it before.16:06
superdanwould love to get that in soon so I can start doing testing against devstack16:06
*** imcsk8 has quit IRC16:07
*** imcsk8 has joined #openstack-infra16:07
*** e0ne is now known as e0ne_16:07
*** superanne has quit IRC16:07
openstackgerritLucas Alvares Gomes proposed openstack-infra/project-config: Ironic: Move pxe_ipa-nv to check instead of experimental  https://review.openstack.org/16220416:07
sdaguesuperdan: +A16:08
superdansdague: thanks16:08
*** superanne has joined #openstack-infra16:08
*** carl_baldwin has joined #openstack-infra16:08
*** dizquierdo has quit IRC16:08
jeblairsdague: oh cool, i had not seen that, thanks.16:10
*** e0ne_ is now known as e0ne16:10
*** mpaolino has quit IRC16:10
*** mpaolino has joined #openstack-infra16:11
*** ChuckC_ has joined #openstack-infra16:12
*** deepakcs has quit IRC16:12
*** jaypipes has joined #openstack-infra16:14
*** mbacchi has quit IRC16:14
*** jaypipes is now known as leakypipes16:14
*** nelsnelson has quit IRC16:16
*** teran_ has quit IRC16:17
*** achuprin_ has quit IRC16:18
clarkbjeblair for dib images I do a single image-build the image-update per region16:19
clarkber16:19
clarkbimage-upload not image-update16:19
jeblairapparently we can't build centos6 images in rax16:19
jeblair2015-03-06 16:07:54,613 INFO nodepool.image.build.rax-dfw.bare-centos6:   setuptools must be installed to install from a source distribution16:19
jeblairwe have some that are 25 hours old16:20
*** rfolco has quit IRC16:20
*** dizquierdo has joined #openstack-infra16:20
*** sabari has quit IRC16:20
*** sabari has joined #openstack-infra16:23
*** krtaylor has quit IRC16:23
*** scheuran_ has quit IRC16:25
*** dannywilson has joined #openstack-infra16:27
*** garyk has quit IRC16:28
*** dannywilson has quit IRC16:28
*** ghostpl_ has quit IRC16:30
*** dannywilson has joined #openstack-infra16:32
*** ParsectiX has joined #openstack-infra16:32
*** amotoki has joined #openstack-infra16:33
*** chlong has joined #openstack-infra16:33
*** ghostpl_ has joined #openstack-infra16:34
*** otter768 has joined #openstack-infra16:34
*** krtaylor has joined #openstack-infra16:35
*** achuprin_ has joined #openstack-infra16:35
*** nelsnelson has joined #openstack-infra16:36
anteayaclarkb jeblair are we having more swift log upload issues? https://review.openstack.org/#/c/161447/416:36
anteayasorry I haven't read all the backscroll yet16:36
*** rfolco has joined #openstack-infra16:36
anteayalatest run has File Not Found instead of logs16:36
anteayaand the run prior to that as well16:37
*** otter768 has quit IRC16:38
jeblairanteaya: can you link to the failed run instead?16:38
anteayaand can someone add kevinbenton to https://review.openstack.org/#/admin/groups/632,members when they get a moment?16:38
jeblairanteaya: done16:39
anteayathanks16:39
anteayalink to the failed run? http://logs.openstack.org/47/161447/5/check/gate-infra-puppet-apply-centos6/16d57ef/16:39
anteayais that what you mean?16:39
anteayaand another: http://logs.openstack.org/47/161447/5/check/gate-infra-puppet-apply-precise/1444292/16:39
anteayaI've got more16:40
Ngmordred: are you using a patched vhd-util in your upload-rax.sh? (hence the docker madness in the script?)16:40
mordredNg: yes16:41
jeblairanteaya: that's from the same time we were having problems16:41
anteayaah16:41
anteayaso the way forward is recheck? or are we still looking for a solution?16:41
mordredNg: I took an initial stab at making debs here: https://launchpad.net/~mordred/+archive/ubuntu/vhd-util but haven't followed up16:41
*** sdake_ has joined #openstack-infra16:42
openstackgerritMerged openstack/requirements: Require openstack-doc-tools 0.23  https://review.openstack.org/15747216:42
jeblairanteaya: recheck. fixed a while ago16:42
anteayathanks16:42
openstackgerritMerged openstack/requirements: Update openstackdocstheme  https://review.openstack.org/15837916:42
*** unicell1 has quit IRC16:42
mordredjeblair: wtf on the centos6 in rax issue - aren't we using get-pip everywhere?16:42
clarkbwhat problems were we having?16:42
openstackgerritMerged openstack/requirements: Adds Autobahn to global requirements  https://review.openstack.org/15698216:43
openstackgerritMerged openstack/requirements: Add python-openstackclient  https://review.openstack.org/16088716:43
clarkboh!16:43
openstackgerritMerged openstack/requirements: Bump oslo.context version to 0.2.0  https://review.openstack.org/16186816:43
*** sdake__ has joined #openstack-infra16:43
jeblairclarkb: i pinged you on it earlier to fyi you.  puppet was stopped on nodepool, so the images prep scripts were not in sync with on-image copies of project-config, so zuul swift upload was missing glob216:43
clarkbya reading sb now16:43
jeblairclarkb: i deleted most recent images, re-enabled puppet, and started building new ones just cause16:44
clarkbcool16:44
jeblairbut yeah, i have no idea what's up with centos616:44
mordredjeblair: just cause is a great reason16:44
clarkbis devatack-fedora21-dib still failing to build with puppet enabled?16:44
jeblairclarkb, mordred: http://paste.openstack.org/show/190455/16:44
clarkbthat is an amazing error16:45
jeblairclarkb: yes, f21 seems to be in a tight fail loop16:45
jeblairclarkb: i think we should remove it from the config16:45
anteayaclarkb jeblair can you post when the new images are finished building? the enable swift logs on python-jobs patch is waiting on the image builds16:45
clarkbjeblair +116:45
jeblairanteaya: they are not building16:46
clarkbI wonder if setuptools released without a whl?16:46
mordredNg: https://github.com/emonty/deb-vhd-util <-- there's the packaging repo for it - in case you decide to punch that in the face instead of the docker16:46
Ngmordred: I'm going to be honest here, I just build a separate vhd-util binary and shoved it in ~/bin/ ;)16:46
Ng*built16:46
clarkbthere is a whl on pypi16:47
*** sdake_ has quit IRC16:47
Ngmordred: I am curious why the convert patch has been floating around since 2012 and apparently not made it into either debian or fedora16:47
openstackgerritJames E. Blair proposed openstack-infra/system-config: Remove devstack-fedora21-dib  https://review.openstack.org/16221716:47
clarkbbecause its terrible C16:47
jeblairclarkb: ^16:47
*** devvesa has quit IRC16:48
Ngclarkb: well that's no excuse!16:48
jeblairclarkb: i guess we're not building any nodes again?16:48
jeblairjeez16:48
anteayajeblair: oh16:48
*** rmcall has quit IRC16:48
*** rmcall has joined #openstack-infra16:49
openstackgerritDoug Hellmann proposed openstack-infra/devstack-gate: add oslo.versionedobjects  https://review.openstack.org/16221816:49
jeblairi am going to force-merge this patch, and then i am not going to review another nodepool code or config change again until this thing has some real testing.16:49
*** hdd_ has joined #openstack-infra16:49
clarkbok16:49
*** gyee has joined #openstack-infra16:49
openstackgerritMerged openstack-infra/system-config: Remove devstack-fedora21-dib  https://review.openstack.org/16221716:50
mordredNg: I have literally no idea16:52
jeblairnodes are being launched now16:52
clarkbmordred really its bad C iirc there was a review of it and no one worked to fix it16:52
*** yamahata has joined #openstack-infra16:52
*** boris-42 has joined #openstack-infra16:53
mordredclarkb: sure. but that has never stopped distros from using patches before16:53
*** notmyname has joined #openstack-infra16:54
*** sambetts has quit IRC16:55
*** purp is now known as purp_away16:55
*** sambetts has joined #openstack-infra16:57
greghaynesNg: Theres also https://review.openstack.org/#/c/156433/16:58
greghaynes(totally untested)16:58
*** jerryz_ has quit IRC16:58
mordredgreghaynes: nice16:58
jeblairgreghaynes: does that require the patched vhd-util?16:59
greghaynesyes16:59
cody-somervilleWhat version of setuptools is openstack currently using?17:00
mordredyah - but it'll allow us to tell dib to output a vhd and a qcow217:00
mordredcody-somerville: the latest17:00
mordredcody-somerville: don't ever attempt to pin setuptools, you will fail horriblyand die17:00
mordredcody-somerville: I mean, unless you want to :)17:00
anteaya2015-03-06T15:15:15  <jeblair> we just merged a change to the script to use glob2, but did not add it to the venv17:00
*** changbl has quit IRC17:00
anteayaso that was me17:00
anteayahelp me to learn what I should have done17:01
jeblairanteaya: that statement was wrong, keep reading17:01
Nggreghaynes: nice17:01
anteayaI'm reading17:02
zaromorning17:02
anteayaI see the problem was puppet was stopped on nodepool.o.o17:02
anteayaI still want to get better at doing the right thing though17:02
mordredgreghaynes: I know you said "completely untested" ... last time I tried a multi-format output, the second format failed because a variable got stepped IIRC17:02
mordredgreghaynes: we shoudl probably put "make sure format=qcow2,vhd works" on our TDL :)17:03
jeblairanteaya: okay, but you didn't do anything wrong, so good luck improving :)17:03
bhunter71quick question, sorry to butt in:   if I was to run my devstack-trusty at say 2 vcpus versus 4 - what would be the expected impact?    ie - are the tests CPU bound and concurrent/multiprocess?  would I notice significantly longer runs, do you think?    well, there's surely one way to find out!17:03
anteayathanks, it a daily thing17:04
openstackgerritMerged openstack-infra/project-config: Enable dsvm testing on oslo.versionedobjects  https://review.openstack.org/16089317:04
ihrachyshkaanteaya, please https://review.openstack.org/#/c/161303/17:05
jeblairclarkb: maybe there was not a wheel at the time?  i will kick off builds again and see17:05
greghaynesmordred: wait, you want to actually use the code? Clearly we should have designed it for this.17:05
*** jlanoux has quit IRC17:05
clarkbjeblair: ya either that or get-pip's pip is selecting the sdist for some reason. Maybe that wheel doesn't properly advertise support on say linux because it has to link against things?17:06
greghaynesbut yes, that actually could be a really simpe test case - build a few combinations of super minimal images17:06
jeblairclarkb: i just ran it on a not entirely clean vm and it got a wheel, which is what prompted me to think something different might happen17:06
greghaynessince im pretty sure the different dib output formats dont care if the image is basically empty17:07
jeblairclarkb: and yeah, one of the builds just got a wheel and passed that point17:07
*** Ala has quit IRC17:07
jeblairclarkb: so it seems temporal?17:07
anteayaihrachyshka: we have a bit of a backlog in zuul and I am still reading backscroll but I will get to it17:07
ihrachyshkathanks17:07
jeblairmordred: fyi centos6 pip issue was temporary lack of wheel from pypi17:07
clarkbjeblair: weird17:08
anteayaihrachyshka: and try to save pinging me for reviews for crisis things, because if you ping me for every patch you write, your pings will be lost to me17:08
mordredoh - like, "it was in the middle of being uploaded" or something?17:08
*** superanne has quit IRC17:08
ihrachyshkaanteaya, I don't ping for each review, only those that break branches for stable17:08
*** vilobhmm has joined #openstack-infra17:09
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds  https://review.openstack.org/16223717:09
clarkbjeblair: ^ that is the first test in response to this17:09
*** vilobhmm1 has joined #openstack-infra17:09
clarkbjeblair: that covers what you fixed17:09
anteaya2015-03-06T15:39:31  <lxsli> Hmm not sure this is the right place to raise this, but has anyone suggested colour-coding badges at Vancouver?17:10
*** ghostpl_ has quit IRC17:11
anteayalxsli: you might better email the foundation staff with your thoughts on summit workflows17:11
clarkblxsli anteaya summit@openstack.org17:11
clarkbforgot to use the dib-nodepool topic, will update17:12
*** vilobhmm has quit IRC17:13
asselingreghaynes, from yesterday, it does try to download the ruby1.8 package. Seem's like a bug in the package generation script. http://paste.openstack.org/show/190467/17:14
greghaynesasselin: nice!17:14
*** AJaeger_cooking is now known as AJaeger17:15
*** e0ne is now known as e0ne_17:16
*** e0ne_ is now known as e0ne17:16
anteayaclarkb: thanks17:16
jeblairclarkb: cool, so hopefully that will fail, yeah?17:17
clarkbjeblair: no it passes because your fix merged I think17:17
jeblairoh ok17:17
clarkbjeblair: nodepool daemon that was running failed because we need to restart the daemon17:17
clarkbyup your fix is the commit that test is based on17:18
jeblairclarkb: do you need the sleep(3) there?17:18
pleia2morning17:19
clarkblet me check, I mostly cargo culted the test just before it (which I also wrote) so it may be wrong too17:19
jeblairclarkb: also, what causes the dib image to fail?17:20
pleia2doh, need to remove old ssh key for ci-backup-rs-ord.openstack.org17:20
* pleia2 gets to it17:20
*** changbl has joined #openstack-infra17:20
anteayamorning pleia217:21
clarkbjeblair: in the yaml, I have specified an invalid base image name. The fake dib script that the tests runs exits 1 on that17:21
asselingreghaynes, so some bug here: https://github.com/openstack-infra/project-config/blob/master/nodepool/elements/cache-devstack/extra-data.d/55-cache-devstack-repos#L12017:21
clarkbjeblair: I can also add a specific assert that the dib image table is empty too and not just check nodes build17:21
jeblairclarkb: got it17:21
*** lucasagomes has joined #openstack-infra17:21
*** cdent has quit IRC17:22
*** armax has quit IRC17:22
lucasagomesHi all, I've a experimental job in Ironic and instead of building a ramdisk from scratch we download it during devstack run see: http://logs.openstack.org/44/158644/8/experimental/check-tempest-dsvm-ironic-pxe_ipa-nv/bd20459/logs/devstacklog.txt.gz#_2015-03-06_15_19_52_26717:22
lucasagomesis it possible to cache it somewhere?17:22
* lucasagomes goes and look for examples17:22
AJaegerpleia2: do you have a few minutes to discuss https://review.openstack.org/#/c/157626/ ?17:22
*** mpaolino has quit IRC17:23
pleia2AJaeger: if everyone is ok with it, that's fine, but it's still weird to me17:23
AJaegerpleia2: what is the weirdness? The descriptions or the check for it?17:23
anteayaAJaeger: me too17:24
anteayaI'm fine with adding descriptions17:24
anteayathe test seems weird to me17:24
pleia2AJaeger: that we're forcing projects to have descriptions when they chose not to17:24
pleia2I don't see the benefit17:24
anteayahave you looked at teh format check scripts jeblair introduced last night?17:24
anteayaI think checking for format is a better way to go17:24
pleia2and they don't know that a description has been added, so they won't know if they have to update it17:25
jeblairpleia2: i'm not sure that anyone has _chosen_ not to have a description17:25
*** vilobhmm1 has quit IRC17:25
AJaegeranteaya: I looked briefly but don't understand what you mean checking format17:25
AJaegeranteaya: jeblair's script can be enhanced to check for the description entry17:25
jeblairpleia2: more likely, cargo-culted something that happened to not have one17:25
pleia2jeblair: ah, just unaware of syntax?17:25
*** vilobhmm has joined #openstack-infra17:25
* pleia2 nods17:25
jeblairpleia2: i think very early versions of that file didn't have that, so many didn't get them originally17:26
AJaegerpleia2, anteaya: Most recent patches have descriptions in there - the cargo cult and docu works17:26
anteayaI'm fine checking format of what is there17:26
clarkbjeblair: the test does fail without the sleep, I think it may expose a potential real problem so let me debug before approving anything17:26
anteayawhitespace and removing '17:26
AJaegeranteaya: the patch adds missing descriptions and then checks for the future...17:26
clarkbjeblair: though the other tests seem to ru nwithout their sleeps, so we can probably clean the sleeps up17:26
AJaegeranteaya: yeah, I'm fine with that kind of format cleanup17:26
anteayaI'm fine with a test for format17:26
clarkblooks like waitForNodes should do the sleep for us17:26
*** e0ne is now known as e0ne_17:27
AJaegerFor me it's really usefull to have some kind of descriptions for repositories. There are far too many to know what each does...17:27
greghaynesasselin: http://git.openstack.org/cgit/openstack-dev/devstack/tree/files/debs/trema17:28
clarkbjeblair: I see what the problem is, we allocate nodes to fake-provider1 even when it does not have any images ready to build that node17:28
* asselin looking17:28
anteayaI have read some of the descriptions they don't help me in many cases understand any better what the project does17:28
clarkbjeblair: so waitForNodes() waits until it times out17:28
anteayabindep for instance17:28
greghaynesasselin: I am still very confused why infra doesnt hit this when building images for trusty17:28
anteayastill have no idea what that repo does17:28
*** unicell has joined #openstack-infra17:28
jeblairanteaya: er, you have an action item to adopt it into infra17:28
AJaegeranteaya: then let's ask for better descriptions merging it in.17:28
greghaynesasselin: are you building on a trusty build host or precise?17:29
AJaegeranteaya: and for bindep, you can surely improve it.17:29
asselingreghaynes, precise17:29
anteayajeblair: I do and have written the patches17:29
anteayaI know how to move a project17:29
greghayneshrm, well that kills my hypothesis then17:29
anteayabut I read the description it doesn't tell me what the project does17:29
anteayain language I understand17:29
*** luqas__ has quit IRC17:29
anteayabut that wasn't required to write the patch to move it17:30
AJaegeranteaya: http://git.openstack.org/cgit/stackforge/bindep/tree/README.rst17:30
*** dtantsur is now known as dtantsur|afk17:30
*** dtantsur|afk is now known as dtantsur|pto17:30
clarkbjeblair: and the sleep doesn't seem to affect that, I get it to pass with and without the sleep. I am thinking a race between allocation and image build failing17:30
AJaegeranteaya: binary dependencies tells me already a lot - but yes, it still leaves question open17:30
asselingreghaynes, why?17:30
clarkbjeblair: eg if we allocate before the image fails it thinks it will have an image and allocates 1, if we allocate after then it knows to give both nodes to provider217:31
greghaynesasselin: I was thinking infra didnt hit it because they build on precise but if youre doing that too then thats probably not the issue17:31
openstackgerritMerged openstack-infra/project-config: Flip the bool condition  https://review.openstack.org/16219917:31
asselingreghaynes, I think infra builds on trusty17:31
*** ddieterl_ has quit IRC17:31
*** ddieterly has joined #openstack-infra17:31
greghaynesah17:32
*** e0ne_ has quit IRC17:32
anteayaAJaeger: see I speak a different language17:32
anteayaso forcing someone to write a description that amounts to binary dependencies doens't help me17:32
AJaegeranteaya: I guess I did a bit more packaging than you did in my past...17:32
anteayano notice I didn't vote on teh patch17:32
anteayaI have done zero packaging17:33
*** jcoufal_ has quit IRC17:33
anteayaso if others want a test to force pepole to write stuff that amounts to binary dependanceies who am I to say no17:33
*** rwsu-afk is now known as rwsu17:33
anteayabut if you ask me does that add value for me? no, no it doesn't17:33
anteayaformating does17:33
*** claudiub has joined #openstack-infra17:34
AJaegeranteaya: Reading the README, what about as description for bindep: "Tool to check that binary packages need by applications and libraries exist"17:34
asselingreghaynes, for now I will work around by taking away the set -e.17:34
anteayaif you want17:35
AJaegeror "are available". It's basically like requirements.txt file but not for python packages but for binary (distribution) packages17:35
anteayamy point is a test for the presence of a description doesn't do anything for me17:35
*** mpaolino has joined #openstack-infra17:35
anteayaformating any description that is there does17:35
AJaegeranteaya: I agree that only the presence is not enough. We need to review better to get good descriptions in.17:35
anteayaokay17:36
greghaynesasselin: so my reading of devstack indicates that the trema dependencies file is never actually used17:36
AJaegerAnd I'm happy to +2 a formatting patch, I just need some more time to review it.17:36
anteayaI haven't bee reviewing for presence, just reasonableness if they put one there17:36
greghaynesasselin: which would explain why everyone isnt hitting this17:36
anteayaAJaeger: of course17:36
AJaegeranteaya: but I liked where it was going...17:36
anteayaAJaeger: your description of bindep makes sense to me17:36
anteayathanks17:36
*** dizquierdo has quit IRC17:36
greghaynesclarkb: is it possible that youall are pointing your nodepool-dib builds for ubuntu at a precise apt mirror?17:36
anteayaAJaeger: of course because you like to automate everything17:37
anteayaand i can see that17:37
clarkbgreghaynes: I don't think so17:37
AJaegeranteaya: ;)17:37
anteayabut somethings, like language choices aren't a good fit17:37
*** jistr has quit IRC17:37
clarkbgreghaynes: we use the ubuntu base images17:37
clarkbgreghaynes: no funny business17:37
anteayabut language format is17:37
greghaynesheh, ok17:37
*** pblaho has quit IRC17:38
AJaegeranteaya, pleia2: The patch has now 2 +2s on it. Since I didn't hear a veto, would you be ok if I approve it now - or do we need to discuss this in the infra-meeting some more?17:39
openstackgerritMauro Rodrigues proposed openstack-infra/subunit2sql: Convert tags metadata to string before persist it  https://review.openstack.org/14720517:40
*** ivar-lazzaro has joined #openstack-infra17:40
*** armax has joined #openstack-infra17:40
pleia2AJaeger: you can go ahead17:40
anteayaAJaeger: like I said if others feel there is value there, they can have it17:40
anteayaI see no value in it17:40
anteayaI'm not blocking it17:41
*** rkukura has joined #openstack-infra17:41
asselingreghaynes, ok looking here: https://github.com/openstack-infra/project-config/blame/master/nodepool/elements/cache-devstack/extra-data.d/55-cache-devstack-repos#L13717:41
*** ivar-lazzaro has quit IRC17:41
asselinshould pick either debs or apts? I see both...?17:41
clarkbjeblair: the allocation granting system does not seem to take into account image availability17:42
AJaegeranteaya, pleia2: understood - thanks17:42
*** ivar-lazzaro has joined #openstack-infra17:42
clarkbjeblair: probably something to try updating in a subsequent change?17:42
*** ivar-lazzaro has quit IRC17:42
AJaegeranteaya: speaking about checks, I added one that every channel in gerritbot is in accessbot as well: Do you see value in that? https://review.openstack.org/16207017:43
*** ivar-lazzaro has joined #openstack-infra17:43
*** david-lyle_afk is now known as david-lyle17:43
clarkbactually I think there may be a simple fix17:43
greghaynesasselin: I think its correct - I think apts is just a symlink to debs17:43
AJaegerclarkb: with 162070 in, all channels are cleaned up ;)17:44
greghaynesso worst case it would be double installing everything17:44
asselingreghaynes, ok I see17:44
*** hodos has joined #openstack-infra17:45
anteayaAJaeger: wasn't there some discussion you had with jeblair to create one bot file instead of many?17:45
asselingreghaynes, sorry going to a meeting17:45
*** armax has quit IRC17:45
anteayaso that getting channel services was one config entry rather than many?17:45
*** ashleighfarnham has joined #openstack-infra17:46
anteayaam I remembering correctly?17:46
*** AJaeger_ has joined #openstack-infra17:46
jeblairanteaya: i'm working on a plan, but don't have it ready yet17:47
*** Guest94194 has quit IRC17:47
anteayaokay thanks, just was wondering if I understood the direction correctly17:48
*** claudiub has quit IRC17:48
*** AJaeger has quit IRC17:48
*** afazekas has quit IRC17:49
anteayaAJaeger: and yes in the meantime ensuring that channels that recieve gerrit patch status events also have infra access is a good thing to have17:49
anteayaoh he left17:49
*** stevemar has quit IRC17:50
*** stevemar has joined #openstack-infra17:50
AJaeger_anteaya: I'm back17:52
*** bnemec is now known as beekneemech17:52
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds  https://review.openstack.org/16223717:52
openstackgerritClark Boylan proposed openstack-infra/nodepool: Don't allocate nodes to providers without images  https://review.openstack.org/16225517:52
clarkbjeblair: ^ I don't fully understand how that wasn't a 100% failure before, but this makes my test pass after 10 consecutive runs when before the fail rate was about 50%17:53
anteayaAJaeger_: oh okay great17:53
*** purp_away is now known as purp17:53
clarkbjeblair: not sure that that is the correct place to handle this case in the allocator either17:53
AJaeger_anteaya: having it in one file would be great - together with the eavesdrop, and then still we would need some checking. This is the best I came up with for now17:53
anteayaAJaeger_: understood17:53
anteayaand yes it has value for me17:54
anteayajust keep in mind that even when we disagree I admire your hard work17:54
anteayaand appreciate your perspective17:54
anteayaI just don't share it sometimes17:54
anteaya:)17:54
*** jcoufal has joined #openstack-infra17:54
clarkbjeblair: though this exposes a bunch of fails in the other tests :(17:55
AJaeger_anteaya: that's the spirit. Disagreement is fine if we handle it good as we do17:55
clarkbbecause they depend on a positive allocation leading to waiting for a node to be made17:55
clarkbhrm17:55
*** AJaeger_ is now known as AJaeger17:57
anteayaAJaeger: can you look at https://review.openstack.org/#/c/161447/5/zuul/layout.yaml for me17:58
anteayaperhaps I am confused17:58
anteayaI see tarballs and publish to pypi as separate things17:58
anteayadoes publish to pypi need tarballs?17:58
anteayaI didn't think it did17:58
*** ssam2 has quit IRC17:59
openstackgerritMerged openstack-infra/project-config: Add a lot of missing descriptions  https://review.openstack.org/15762618:02
jeblairclarkb: i feel like your code change is correct, but that means we need a different strategy for testing :/18:03
*** lucasagomes is now known as lucas-beer18:04
clarkbjeblair: ya I am working on that18:04
*** superanne has joined #openstack-infra18:04
jeblairpushed gertty 1.1.018:04
clarkbjeblair: I think the key is each test should explicitly state what resources it is waiting on early in a test18:05
jeblairclarkb: that sounds like a good solution18:05
clarkbjeblair: so waitForImage(X) waitForImage(Y) waitForNodes() then asserts18:05
jeblairclarkb: ++18:05
*** lucas-beer has left #openstack-infra18:06
*** yamahata has quit IRC18:07
clarkbjeblair: I think the sleeps were there to approximate this behavior in the past18:07
jeblairclarkb: the negative is still tricky there... you are working on a test where you don't expect the image to appear, and the dib image construction is asynchronous...18:07
clarkbjeblair: yup I will need to assert failed buidls some other way18:08
jeblairclarkb: perhaps it will generally work, but it doesn't seem foolproof18:08
jeblairclarkb: maybe record failed builds in something the test can inspect?18:08
clarkbjeblair: we do :) the log18:08
*** yamamoto has quit IRC18:09
*** SumitNaiksatam has quit IRC18:09
clarkbthough thats a bit fragile18:09
jeblairhttps://pypi.python.org/pypi/gertty/1.1.0 exists18:09
*** marcusvrn has quit IRC18:09
jlvillaljeblair: I got an exception about the database in gertty after I did a fresh git pull.  Is this a known issue?18:10
superanneanteaya: did you get a chance to read through the email, does it make sense for the 3rd party audience?18:11
jeblairjlvillal: can you paste the error?18:12
jlvillaljeblair: http://paste.openstack.org/show/190472/18:12
*** marcusvrn has joined #openstack-infra18:13
*** bdpayne has joined #openstack-infra18:13
jlvillaljeblair: It did immediately on startup.  So I deleted the ~/.gertty.db file  But then it died again.18:13
jlvillals/did/died/18:13
jeblairjlvillal: did it die with the same error the second time?18:14
jlvillaljeblair: The one I pasted is the 2nd time.  Let me look at scrollback for first error18:15
*** edmondk has left #openstack-infra18:15
jlvillaljeblair: Seems to be the same error.  I'll paste18:15
jlvillaljeblair: 1st run error: http://paste.openstack.org/show/190473/18:15
jeblairjlvillal: i believe it's an instance of this bug: https://storyboard.openstack.org/#!/story/200007318:16
jeblairmordred: what's the latest thinking on that?18:16
jeblairjlvillal: it seems to be somehowe related to pbr and the way gertty is installed from git18:17
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds  https://review.openstack.org/16223718:17
openstackgerritClark Boylan proposed openstack-infra/nodepool: Don't allocate nodes to providers without images  https://review.openstack.org/16225518:17
jeblairjlvillal: if you create a new venv, and run "pip install -d ." it should work.  or if you create a new venv and install from pypi with "pip install gertty"18:17
mordredjeblair: still unable to reproduce18:17
clarkbjeblair: ^ ok that should work for the first change, now I just need to sort out positively asserting that image builds failed in the new test18:17
jeblairjlvillal: do you have any idea how you originally installed it?18:18
jlvillaljeblair: I did the virtualenv setup18:18
jlvillaljeblair: Installed it18:18
jlvillaljeblair: Then I just installed it again after doing a git pull18:18
jeblairjlvillal: do you know what command you used to install it?18:18
*** purp is now known as purp_away18:18
clarkbjeblair: mordred the fix for that is git clean -xfd then reinstall18:18
jlvillaljeblair: pip install .18:19
clarkbthe stale egg tells pbr/setuptools it doesn't need to update the alembic files18:19
*** morganfainberg is now known as needscoffeebadly18:19
jeblairclarkb: so theoretically wou should be able to checkout an old version, pip install ., checkout a new version, pip install . and reproduce?18:19
jeblairs/wou/we/18:20
*** dolphm is now known as dolphin18:20
*** mpaolino has quit IRC18:20
*** dolphin is now known as Guest8273418:20
jeblairclarkb: so theoretically we should be able to checkout an old version, pip install ., start gertty, stop gertty, checkout a new version, pip install ., start gertty and reproduce?18:20
jeblairfixed ^ :)18:20
jlvillaljeblair: Did a brand new virtualenv.  activate.  'pip install .'  And crashed :(18:21
clarkbjeblair: yes I think so, just be sure you start with a clean slate otherwise your old version may also be affected by a stale egg18:21
jeblairjlvillal: what's the error this time?18:21
jlvillaljeblair: Same one18:21
jlvillaljeblair: sqlalchemy.exc.OperationalError: (OperationalError) no such column: ...18:22
*** esker has quit IRC18:22
*** needscoffeebadly is now known as CaptainMorgan18:22
*** e0ne has joined #openstack-infra18:22
pc_mclarkb: pping18:23
clarkbpc_m: pong18:23
*** achanda has joined #openstack-infra18:23
*** armax has joined #openstack-infra18:23
pc_mclarkb: Can you review (and hopefully approve) https://review.openstack.org/#/c/162135/?18:23
jlvillaljeblair: It happens when I hit the key 'L'18:23
pc_mclarkb: I goofed with the file filtering thing.18:23
*** e0ne has quit IRC18:24
clarkbpc_m: sure done18:24
pc_mclarkb: Thank you!18:24
jeblairjlvillal: i just did that and it worked for me.  are you sure you were in a completely new venv?18:24
*** tonytan4ever has quit IRC18:25
jlvillaljeblair: I'm sure.  I'll do it again :)18:25
jeblairjlvillal: best also remove the .gertty.db just to be sure18:25
*** jp_at_hp has quit IRC18:25
jlvillaljeblair: I did that.  Testing again.18:26
AJaegeranteaya: what's the problem with pypi?18:27
*** superanne has quit IRC18:27
AJaegeranteaya: the template generates the tarballs and uploads them18:27
*** jogo is now known as flashgordon18:27
jeblairclarkb: those steps don't reproduce :(18:28
*** salv-orl_ has quit IRC18:28
jeblair2015-03-06 10:32:44,963 Running upgrade 312cd5a9f878 -> 4a802b741d2f, add starred18:28
anteayaAJaeger: right so with publish to pypi we don't need any additional information for tarballs do we?18:28
jeblairclarkb: https://etherpad.openstack.org/p/LeqN5lpQ4d are the steps18:29
anteayaAJaeger: see my comments: https://review.openstack.org/#/c/161447/4/jenkins/jobs/projects.yaml18:29
*** coolsvap has joined #openstack-infra18:29
anteayasuperanne: yes, go ahead do send18:29
AJaegerah, dig a bit...18:30
anteayaAJaeger: am I wrong?18:30
AJaegeranteaya: jenkins/jobs/pypi-jobs.yaml uses tarball-site18:31
*** yamahata has joined #openstack-infra18:31
AJaegerso, the variable is needed18:31
AJaegerand the jjb template python-jobs includes the {name}-tarball job-group18:32
AJaegerI meant:18:32
AJaegerand the jjb  python-jobs job-group includes the {name}-tarball job18:32
jeblairclarkb: i don't see egg dirs in gertty when i do that; could an older version of pbr have created them?18:32
AJaegerAnd the Zuul publish-to-pypi template uses {name}-tarball job18:32
openstackgerritMichael Krotscheck proposed openstack-infra/storyboard-webclient: Added sane result set indicators.  https://review.openstack.org/16226518:33
AJaegerSo, if you remove the two liines from projects.yaml, our tests should fail because tarball-site is not set and the tarball job does not exist18:33
anteayaah okay18:34
anteayaso doug was right the first time and i"m wrong18:34
anteayathanks18:34
jeblairclarkb, mordred: i found that if i run python setup.py install with the old checkout, it creates an egg dir and triggers the problem18:34
mordredjeblair: AH. LOVELY18:34
*** otter768 has joined #openstack-infra18:34
jeblairmordred: see the steps in https://etherpad.openstack.org/p/LeqN5lpQ4d18:35
AJaegeranteaya: dhellmann used just ''{name}-tarball' in his most recent patch - so just the one job...18:35
anteayabut in the patch I commented on18:35
mordredjeblair: /me grumps18:35
openstackgerritMerged openstack-infra/project-config: Partial revert of func job setup for VPNaaS repo  https://review.openstack.org/16213518:36
anteayaAJaeger: did doug have it right in patchset 4?18:36
jeblairjlvillal: so clark's part of the solution is important.  you have to clean out the old egg dir from your git checkout (recreating the venv is not enough)18:36
*** Guest82734 is now known as dolphinator18:36
mordreddstufft: there isn't any way to detect if you're being run by pip vs. being run directy by python setup.py is there?18:36
jlvillaljeblair: Oh.  let me try that :)18:36
jeblairjlvillal: "the fix for that is git clean -xfd then reinstall"18:36
dstufftmordred: do you want the real answer or the ideal answer18:37
mordreddstufft: the real answer18:37
mordreddstufft: I've long since given up on idealism18:37
jlvillaljeblair: Woo hoo!!!  and woo hoo to clarkb :)  It works now.  Thanks.  Hopefully there can be a way to detect that for upgraders.18:38
mordredjeblair: (I'm thinking that if we can detect it - we can at least print a warning like "dude, don't run setup.py directly")18:39
AJaegeranteaya: setting tarball-site was right, using python-jobs included the tarball job but many other jobs as well - like many other include.18:39
*** markvoelker has quit IRC18:39
jeblairjlvillal: yeah, i think that's what mordred is working on.  but from now on, as long as you don't run "python setup.py install" and just do "pip install ." you shouldn't hit the error again, even on the next upgrade18:39
AJaegeranteaya: so it was good - but using only tarballs job is perfect like its now ;)18:39
*** otter768 has quit IRC18:40
*** markvoelker has joined #openstack-infra18:40
dstufftmordred: so pip calls setup.py in a specific way that's highly unlikely to be replicated by someone whose not pip, also I'm pretty sure that pip sets some environment variables too that you could check for. But none of those are promised things and may change in the future (and may not hold in the past)18:40
jlvillaljeblair: I only did 'pip install .'  as a note18:40
anteayaAJaeger: ah so the latest patchset is better, cool18:40
anteayaAJaeger: thanks for looking at it18:41
mordreddstufft: could I request that we add a thing that always sets a "RUNNING_IN_PIP" env or something? I'll even write the patch ...18:41
jeblairjlvillal: do you have a "gertty.egg-info" in your gertty directory now?18:41
*** BobBall is now known as BobBall_AWOL18:41
*** sarob_ has joined #openstack-infra18:41
anteayadhellmann: AJaeger likes your latest aeromancer patchset, and shortly I will too18:41
jlvillaljeblair: I don't see one18:41
*** thingee has joined #openstack-infra18:41
jeblairjlvillal: ok good.  i wonder if maybe an older version of pbr caused it to be created or something...18:42
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds  https://review.openstack.org/16223718:42
jeblairjlvillal: do you have your old virtualenv -- the one that had the initial problem?18:42
thingeejeblair: I noticed the #openstack-cinder channel topic changed. Not sure if I upset someone or if there was some global reset in topics.18:42
dstufftmordred: I'd need to think about that a little bit, I understand the desire to check for that, but I worry about the precidence it would set - We're trying to move towards a world where what installer you use doesn't matter18:42
jeblairthingee: global reset of topics with outdated info due to technical error.  sorry :(  feel free to change it back18:42
jlvillaljeblair: Sorry.  The git -xfd wiped it out.18:42
pleia2need to heads down for a bit to work on some slides+practice elasticon talk for next week, bbiab18:43
thingeejeblair: Ok just double checking I didn't do something wrong :)18:43
dstufftmordred: open a ticket for discussion?18:43
clarkbjeblair: I also see http://paste.openstack.org/show/190485/ occasionally when running these tests with --until-failure18:43
jeblairjlvillal: np.  we're a bit closer to understanding now, and i'm at least pretty sure that you should be all set going forward.18:43
jeblairjlvillal: thanks and sorry18:43
*** wenlock has quit IRC18:43
jlvillaljeblair: Thanks for the help :)18:43
mordreddstufft: fair concern... and yes, I shall do that18:43
*** markvoelker has quit IRC18:44
*** spzala has joined #openstack-infra18:44
openstackgerritRafael Folco proposed openstack-infra/system-config: Updates to running-your-own CI docs: Changes required  https://review.openstack.org/16226818:45
*** amitgandhinz has quit IRC18:45
anteayacan we get third party account 9751 IBM DB2 CI disabled please? http://paste.openstack.org/show/190486/ https://review.openstack.org/#/c/157158/18:45
jlvillaljeblair: On another note.  How hard would it be to have j/k scroll down/up ?  Or are those keys already used?  vim user here :)18:45
anteayakeystone has tried for 4 days to get them to fix it and they won't18:45
anteayaso CaptainMorgan has requested they be disabled18:45
CaptainMorgani also looked, they are not properly posting logs for success. and haven't been for a while.18:46
CaptainMorgan404 when you click the link.18:46
openstackgerritRafael Folco proposed openstack-infra/system-config: Updates to running-your-own CI docs: Changes required  https://review.openstack.org/16226818:47
CaptainMorgansince i haven't seen failures outside of merge conflict with no logs, i don't know if failures would have logs either.18:47
jeblairjlvillal: you can override keys in your local config file.  and if you create a "vim-style" map that others might like, we can add that as a built-in keymap too, so people can just say "keymap: vim" in their config files and get the same thing.18:47
jlvillaljeblair: Thanks.  I will check out the config file18:48
* mordred keeps wanting "u" to do the same thing as esc - but hasn't bothered doing it ...18:48
mordredalso, I have no idea why I want that ... my fingers just want to do it all the time18:48
mordredwas that an elm keystoke I wonder?18:48
jeblairjlvillal: i am not (much of) a vim user, so if you come up with the mapping, i'll be happy to help out with the patch18:48
jlvillaljeblair: Okay.  j/k are the things I naturally go for when I want to scroll up/down.  Not sure if any other keys come to mind as useful.18:49
*** stevemar is now known as stevedore18:50
jeblairjlvillal: it currently doesn't have support for multi-key commands, but probably could with a little work (so :q isn't an option yet).  i hope to add that at some point if someone doesn't beat me to it.18:51
jeblairjlvillal: (i would use it for c-x c-q of course ;)18:52
*** achanda has quit IRC18:52
jlvillaljeblair: :D18:52
jlvillaljeblair: I will get back to you and ask how to set it up in the config file.  Time for lunch now...18:52
*** ivar-lazzaro has quit IRC18:52
*** rkukura has quit IRC18:53
sabariI have a glance bp for k3 and it needs https://review.openstack.org/#/c/161878/.18:56
sabariIt would be great if someone can review.18:56
*** ddieterly has quit IRC18:56
anteayasabari: these are the folks you need to find: https://review.openstack.org/#/admin/groups/131,members18:57
*** achanda has joined #openstack-infra18:57
*** ddieterly has joined #openstack-infra18:57
*** sdake__ has quit IRC18:57
sabarianteaya: Thanks :). Will do.18:58
openstackgerrityolanda.robla proposed openstack-infra/system-config: Expose jenkins_gitfullname and jenkins_gitemail  https://review.openstack.org/16169518:58
*** rmcall has quit IRC18:58
*** sarob_ has quit IRC18:58
clarkbjeblair: "# granted but not requested?  shouldn't happen" comment around the raise for the KeyError I pasted above19:00
*** ivar-lazzaro has joined #openstack-infra19:00
clarkbjeblair: any thoughts on how that is happening would be awesome. I am going to see if it happens without my changes now19:00
*** dangers_away is now known as dangers19:00
*** amitgandhinz has joined #openstack-infra19:01
SpamapSclarkb: funny story, on your question "why did you remove the base class?" ... I actually didn't even look at the base class and figured that was just useless boiler plate. ;)19:01
* SpamapS will push patch up for that later19:01
*** markvoelker has joined #openstack-infra19:03
*** MaxV has quit IRC19:03
*** superanne has joined #openstack-infra19:04
clarkbjeblair: so far it hasn't happened without my change, going to let that run as much as it wants though while I step out for some food19:05
clarkbbut its likely my change has a bug19:05
*** rkukura has joined #openstack-infra19:06
clarkbCaptainMorgan: anteaya did that account get inactivated?19:06
clarkbI am reading scrollback to catch up19:06
cody-somervilleHave you ever seen a case where all the jobs have ran for a change but they just sit there in the queue?19:06
anteayaclarkb: I have not seen that it has19:07
clarkbanteaya: ok will do that now19:07
anteayaclarkb: thank you19:07
clarkbanteaya: CaptainMorgan all done19:07
*** melwitt has joined #openstack-infra19:07
clarkbcody-somerville: which queue?19:08
cody-somervilleclarkb: gate19:08
anteayaclarkb: thank you19:08
* anteaya composes email to -announce19:08
openstackgerritjose-idar proposed openstack-infra/project-config: Adding pep8 gate check to Opencafe  https://review.openstack.org/15964219:09
*** yamamoto has joined #openstack-infra19:09
clarkbcody-somerville: check your zuul event and results queue19:10
clarkbcody-somerville: if the results queue value is not 0 that means zuul is processing the results and finished jobs will hang around until zuul gets to them19:10
cody-somervilleclarkb: Queue lengths: 0 events, 0 results.19:10
clarkbcody-somerville: could it be that gerrit is taking a long time to merge the change?19:11
clarkbcody-somerville: zuul will sit and wait for either the successful or fail response19:12
clarkbjeblair: it almost looks like getNeededNodes is not properly reentrant19:12
clarkbjeblair: and both _run and waitForNodes call it19:13
openstackgerritMerged openstack-infra/subunit2sql: Convert tags metadata to string before persist it  https://review.openstack.org/14720519:13
clarkband reentrancy is broken by my change somehow, I will need to think this thorugh19:13
*** yamamoto has quit IRC19:14
cody-somervilleclarkb: Unlikely. It's been an hour and a half.19:15
cody-somervilleclarkb: its noop jobs fwiw19:15
cody-somervilleclarkb: is it because there are other changes in the change queue that aren't finished?19:15
clarkbcody-somerville: I don't think the jobs are at fault, it would be something zuul is doing19:16
clarkbcody-somerville: if it is a proper gating queue zuul always pops off the head19:16
clarkbcody-somerville: no changes can be removed elsewhere19:16
clarkbjeblair: I think I see it, this isn't an issue with the old code because it always requests consistently19:16
*** thingee has quit IRC19:17
clarkbjeblair: but now when we check the labels first there is a race between the label having an image ready and multiple getNeededNodes calls19:17
clarkbjeblair: except hrm, all of these objects appear to be on the stack19:18
jeblairclarkb: yeah, things are going to be more dynamic... things that might have done one node from each of 2 providers before could end up doing 2 nodes from 1 and none from another...19:19
cody-somervilleclarkb: but doesn't it do the grouping by job name? Why would noop jobs be in a change queue with a change that has no noop jobs?19:19
clarkbcody-somerville: yes its based on job but its transitive19:20
clarkbso if you have noop job in a project along side another gate job then that can happen19:20
nikhil_khey, we're looking to possibly retag glance_store version(s)19:20
*** dkranz has quit IRC19:20
nikhil_kwas hoping to find out the best way to do that in our setup19:21
cody-somervilleclarkb: they only have noop jobs then there is a cinder change with the standard cinder jobs19:21
jeblairnikhil_k: what do you mean by retag?19:21
nikhil_kjeblair: we've this issue where the release consisted of a feature addition but we released it with a bump on X in 0.0.X19:21
clarkbcody-somerville: and no other project has both noop and cinder jobs?19:21
jeblairdoesn't have to just be cinder19:22
nikhil_kjeblair: so, we are wondering if it is possible to retag 0.1.12 to 0.2.019:22
nikhil_kor have parallel tags, maybe?19:22
jeblairclarkb, cody-somerville: it could be, but it could also be noop + foo and foo + bar and bar + devstack-gate19:22
cody-somervilleand weird, clarkb jeblair: the noop jobs have finally disappeared but the change at the top of the queue is still running.19:22
jeblaircody-somerville: the status page tells you whether changes are in the same change queue or not19:23
cody-somervillejeblair: they were in the same change queue, yes.19:23
jeblaircody-somerville: if they are, then something is very, very broken.  if they aren't, it was probably just waiting for gerrit.19:23
cody-somervillethe list of projects for the change queue seemed to list almost all the projects we have19:23
*** superanne has quit IRC19:24
jeblaircody-somerville: then it seems likely that you have at least one project with noop and another job.19:24
*** ivar-lazzaro has quit IRC19:24
jeblaircody-somerville: but if they were in the same queue, it should not have been removed if it was not at the top.19:24
jeblairnikhil_k: that sounds terribly confusing :(19:24
nikhil_kumm19:25
*** coolsvap has quit IRC19:25
nikhil_kjeblair: one sec19:25
clarkbok I really need to step out for a bit, jeblair your thoughts on the nodepool change and possible bug much appreciated if you have time19:25
jeblairclarkb: yeah, i've been thinking about it, but i don't have a good answer yet19:26
nikhil_kjeblair: http://eavesdrop.openstack.org/irclogs/%23openstack-glance/%23openstack-glance.2015-03-06.log you would need to search for "2015-03-06T15:05:50"19:26
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Allow JJB to set connection timeout to jenkins server  https://review.openstack.org/15147219:27
nikhil_kjeblair: the basic issue is that a condition is asking for the release number to be 0.2.0 but we have a release of 0.1.1119:27
nikhil_kjeblair: we were hoping to see if that can be corrected and the release tag for 0.1.12 can be equivalent to 0.2.0 by any chance?19:28
jeblairnikhil_k: i think that's only going to make things worse.  i think sigmavirus24 is correct -- it's done, and we just need to deal with it.  releasing 0.2.0 will not make 0.1.12 go away, and 0.1.12 existing and adding new features is the part that causes problems.19:29
*** _shaps_ has joined #openstack-infra19:29
jeblairnikhil_k: so i think the best choice is just to deal with it (we may need to adjust stable branch version caps)19:29
*** thingee has joined #openstack-infra19:29
cody-somervillejesusaurus: ^^ can you check on that noop thing?19:30
openstackgerritIan Cordasco proposed openstack-infra/system-config: Add logging to #openstack-ansible  https://review.openstack.org/16141219:30
jeblairnikhil_k: if, for some reason, we really, really, really want to cause more pain -- you could release 0.1.13 which is equivalent to 0.1.11, and then release 0.2.0 as equivalent to 0.1.12.19:31
jeblairnikhil_k: i would recommend against that though, unless there's a really good reason19:31
nikhil_kjeblair: heh, that seems worse19:31
nikhil_kjeblair: yeah, ok. I will try to convince doug otherwise19:31
jeblairdhellmann: oh, you want a release of 0.2.0 that is == to 0.1.12?19:32
nikhil_kjeblair: he wanted for us to do it as we'r trying to bump up the cap https://review.openstack.org/#/c/161079/19:32
*** ayoung is now known as SugarAddy19:32
*** SugarAddy is now known as ayoung19:32
*** SumitNaiksatam has joined #openstack-infra19:32
*** ja is now known as zz_ja19:33
jeblairdhellmann: my thought is, 0.1.12 is already out there, and breaks any caps that people may have had.  releasing 0.2.0 doesn't fix that, but only makes things confusing for users (why are these versions the same?)19:33
jeblairdhellmann: releasing 0.1.13 as a revert of 0.1.12 (and then releasing 0.2.0) would "fix" people with version caps, but is complex.19:34
jeblairdhellmann: (and still slightly confusing to users, but only if they wonder what 0.1.12 was all about)19:34
jeblairdhellmann: whereas the state we are in now is a more comprehensible "oops -- semver fail" situation :)19:34
jeblairnikhil_k: ^19:34
jeblairso anyway, my recommendation would be (in order of preference): a) do nothing (except fix version caps where necessary); or b) release 0.1.13 and 0.2.0 as described above19:36
nikhil_kjeblair: I personally prefer to let things be as is and promise dhellmann to release 0.2.0 instead of 0.1.1319:36
nikhil_kjeblair: however, we really need that review merged https://review.openstack.org/#/c/161079/19:37
jeblairnikhil_k: yeah, i just don't see what releasing 0.2.0 without 0.1.13 solves19:37
jeblairit only creates confusion and solves no technical problems19:37
nikhil_kjeblair: :)19:38
nikhil_kjeblair: I was trying to resolve conflicts. What can we do to get that merged?19:38
*** tnovacik has joined #openstack-infra19:39
jeblairnikhil_k: i'm not going to review that change, so you should probably wait to see if i have convinced dhellman to change his mind and then do what he suggests.19:40
*** ivar-lazzaro has joined #openstack-infra19:41
*** ghostpl_ has joined #openstack-infra19:42
Nghmm, I am apparently being 17 kinds of stupid today19:43
nikhil_kjeblair: heh, ok.. appreciate it, nevertheless19:43
nikhil_kzhiyan: ^^19:43
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Make manual checklang target voting  https://review.openstack.org/16203319:44
Ngmordred: other than using rackspace-glanceclient, did you have to do some magic to be able to talk to rax glance? I seem to be talking fine to their nova, but glance is insisting on 401s19:45
*** thingee has quit IRC19:46
jeblairNg: at some point, their url wasn't what was actually in the catalog... i wonder if that got fixed?19:46
*** ddieterly has quit IRC19:48
*** dkranz has joined #openstack-infra19:49
*** ddieterl_ has joined #openstack-infra19:49
Ngjeblair: huh19:49
*** mrunge has quit IRC19:50
Ngwell this is just bizarre, keystone 401s too, but nova is very clearly working19:50
*** CaptainMorgan is now known as morganfainberg19:50
*** superanne has joined #openstack-infra19:50
morganfainbergNg, how is keystone 401ing19:50
morganfainbergor..19:50
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Normalize Gerrit projects.yaml  https://review.openstack.org/16195419:50
morganfainbergnothing i need to worry about ;)19:50
mordredNg: yes. please dont use rackspace-glanceclient19:50
mordredNg: use python-glanceclient19:51
mordredNg: you need to set an env var19:51
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Wrap projects.yaml at 80 characters  https://review.openstack.org/16195519:51
mordredNg: export OS_IMAGE_API_VERSION=219:51
Ngmordred: OS_IMAGE_API_VERSION?19:51
Ngk19:51
mordredyup19:51
mordredNg: it's ... bannanas19:51
*** superanne has quit IRC19:51
*** carl_baldwin has quit IRC19:52
morganfainbergoh19:52
*** salv-orlando has joined #openstack-infra19:52
morganfainbergglanceclient19:52
* morganfainberg goes back to identity-land19:52
*** superanne has joined #openstack-infra19:52
Nghmm, still 40119:53
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove upstreams from projects.yaml  https://review.openstack.org/16195619:53
jesusaurusclarkb: jeblair: oh the test name check for dependent pipelines is transitive?19:54
openstackgerritMerged openstack-infra/system-config: Add Wildfly Puppet module  https://review.openstack.org/14351219:54
jeblairjesusaurus: yes -- if project A affects B and project B affects C, then it follows that A affects C19:55
jesusaurusclarkb: jeblair: is there an easy way to get the list of projects grouped into a dependent pipeline? hovering over the list on the zuul status page gives a box too large to be useful19:56
*** superanne has quit IRC19:56
*** superanne has joined #openstack-infra19:57
jeblairjesusaurus: it should log all of the decision making about it at startup.  or you can grab the status.json directly.19:57
mrmartinAJaeger: are you here?19:58
AJaegermrmartin: yes, I am19:58
*** _et has joined #openstack-infra19:58
clarkbjeblair: I think I figured it out, there is a pool.allocation_history that is used19:59
clarkbjeblair: so we need to lock around access to that19:59
jeblairclarkb: just for testing?20:00
mtreinishjeblair: qq about depends on. Would it work if I had a tempest patch and linked it as depending on a stable/icehouse and stable/juno devstack patch?20:01
clarkbjeblair: yes, the only place getNeededNodes() is called in prod is the run loop20:01
sigmavirus24mtreinish: is there any way I can help with https://review.openstack.org/#/c/152224/ ?20:02
clarkbit might also be possible toget the info we need with calling the side effecting getNeededNodes20:02
jeblairmtreinish: tell me if this answers your question: http://docs.openstack.org/infra/manual/developers.html#multiple-changes20:02
mtreinishsigmavirus24: hmm, yeah if you want to pick that up and run with it that would be great20:03
mtreinishsigmavirus24: my next step was going to be tracking down which bugs are still there (which is causing the failures) and reopen them20:03
sigmavirus24mtreinish: heh okay20:03
*** che-arne has quit IRC20:03
*** bcourt has joined #openstack-infra20:03
sigmavirus24Elastic recheck caught a bunch of bugs it apparently recognized but let me rebase to see what's going on20:04
mtreinishjeblair: I think it does, so it should work as I would expect. Thanks20:05
jeblairmtreinish: woo!20:05
openstackgerritMerged openstack-infra/system-config: Add Zuul Gerrit Trigger Way & JJB  https://review.openstack.org/14903220:05
*** mriedem has quit IRC20:05
*** Longgeek has quit IRC20:06
mtreinishjeblair: the pending change is to set the tempest.conf on stable to disable an option which is being added (default enabled) to the config file20:06
mtreinishsigmavirus24: sure, I just assumed the bugs were still open, but there's also a good chance the tempest tests have bit rotted while skipped too20:07
jeblairmtreinish: perfect20:07
sigmavirus24mtreinish: yeah, also this isn't going to rebase cleanly (oh joy ;))20:07
mtreinishsigmavirus24: it might just make sense to abandon the single patch and split it into a bunch of separate patches for each skip being removed20:08
clarkbjeblair: commented on my change with the info20:08
clarkbjeblair: we might be able to get away with using two different history objects20:08
clarkbjeblair: though really we just need to record how many nodes we need to launch somewhere20:09
sigmavirus24mtreinish: makes sense20:09
jeblairclarkb: 162255?20:09
clarkbjeblair: yes20:09
sigmavirus24mtreinish: I'll start working on splitting it up. Thanks for the pointers. =)20:10
jeblairclarkb: how is that getting called twice?20:10
clarkbjeblair: _run() calls it and waitForNodes() in the tests call it20:11
jeblairclarkb: thx20:11
*** salv-orlando has quit IRC20:11
mtreinishsigmavirus24: sure, np. Thanks for taking it over. I haven't had a chance to pick it back up20:11
asselingreghaynes, another possibility of why -infra isn't running into the issue is that they already have the packages cached?20:11
*** ctlaugh_ has joined #openstack-infra20:11
*** esker has joined #openstack-infra20:11
sigmavirus24mtreinish: no worries. trying to turn on testing for a heat thing and noticed that review.20:11
mtreinishsigmavirus24: heh, have you seen: https://review.openstack.org/#/c/158536/20:13
Nggreghaynes: on a scale from zero to legal injunction, how offended would you be if I pushed some tweaks to your changeset for adding VHD support to DIB?20:13
*** mriedem has joined #openstack-infra20:13
sigmavirus24No but it's relevant to my interests heh20:13
*** rkukura has quit IRC20:14
*** bswartz has joined #openstack-infra20:14
*** bcourt has quit IRC20:14
anteayasuperanne: i see the post to -dev but not -announce yet20:16
superanneanteaya: ah yes, meant to ask, will teh same topics in the subject be useful on that list?20:16
anteayano, just the subject20:17
superanneok20:17
superannethanks anteaya20:17
anteayathank you!20:17
jeblairclarkb: if we made allocation_history a param to getNeededNodes, would it work?20:18
*** _shaps_ has quit IRC20:18
openstackgerritMerged openstack-infra/project-config: Normalize Gerrit projects.yaml  https://review.openstack.org/16195420:18
jeblairclarkb: nodepool could use its own allocation history, and the tests could use theirs?  the tests don't actually care about the history, they mostly just want to know whether demand is satisfied.20:18
clarkbjeblair ya I think that would work20:18
jeblairclarkb: or....20:19
jlvillaljeblair: In keymaps.  If I want to do:   cursor-down: ['j']   how would I also specify the arrow-up key?  Or do I need to?20:19
jeblairclarkb: actually i don't like my other idea, nevermind the "or...".  :)20:19
openstackgerritMerged openstack-infra/project-config: Wrap projects.yaml at 80 characters  https://review.openstack.org/16195520:20
jeblairclarkb: i did have one more idea which was to change waitfornodes from a general wait to a specific one (eg, wait for X nodes of this typo and Y nodes of that type -- like waitforimage)20:20
*** aysyd has quit IRC20:20
clarkbok I can update to do the first thing20:21
jlvillaljeblair: I didn't seem to need to have to specify arrow-up.  So I think I am good :)20:22
cineramahey umm pleia220:22
jeblairjlvillal: i don't think you need to20:22
*** rkukura has joined #openstack-infra20:22
*** aysyd has joined #openstack-infra20:22
jeblairjlvillal: yeah :)  but i think if you want to, you can say ['j', 'down'].20:23
jlvillaljeblair: Okay.  Thanks!20:23
jeblairjlvillal: but you should only need to do that if you want to change, say, down to up.  :)20:23
jlvillaljeblair: Is it just me or are the times being displayed in UTC?20:23
openstackgerritnikhil komawar proposed openstack/requirements: Bump glance_store to 0.2.0  https://review.openstack.org/16107920:23
jeblairjlvillal: they are in utc20:23
jlvillaljeblair: :(20:23
jlvillaljeblair: Is there a way to be in local time?20:23
jeblairjlvillal: see 12726020:23
nikhil_kjeblair: new release ^20:23
jeblairnikhil_k: did dhellmann continue to disagree?20:24
jlvillaljeblair: So I assume all the clocks in your house are set to UTC?  :P20:24
nikhil_kjeblair: I've passed along the info on ML too.20:24
*** mrmartin has quit IRC20:25
nikhil_kjeblair: we just figured this was the fastest way to resolve conflicts ...20:25
nikhil_kand hopefully it works with everyone!20:25
jeblairnikhil_k: not sure i follow -- dhellmann still isn't around to weigh in on this20:25
* dhellmann hears his name20:25
jeblairjlvillal: the one i use for work is, and all of fungi's clocks are :)  but anyway, i think it's a good idea, i just think it should be an option20:26
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Remove upstreams from projects.yaml  https://review.openstack.org/16195620:26
nikhil_kjeblair: ah, we just got a +2! \o\ \o/ /o/20:26
jeblairjlvillal: "20:26 < jeblair> jlvillal: the one i use for work is,"  :)20:26
nikhil_kplease do the honors if possible :)20:26
dhellmannjeblair, nikhil_k : sorry, I was at lunch for most of that earlier discussion.20:26
jlvillaljeblair: Okay and thanks.  Not sure I want to tackle that patch....   Hopefully the author will update with  your suggestions.20:27
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Propose projects.yaml normalization daily  https://review.openstack.org/16195720:27
nikhil_kdhellmann: no issues..20:27
nikhil_kglad we got it resolved!20:27
dhellmannjeblair, nikhil_k: Did the new release end up including things that were not in the earlier patch release?20:27
openstackgerritMatthew Treinish proposed openstack-infra/puppet-subunit2sql: Add option to disable uneccesary data migration  https://review.openstack.org/16228920:27
mtreinishclarkb, jeblair: ^^^ if you guys get a sec that branch is needed before I push the next release20:28
* mtreinish hopes to keep the proud tradition of Fri. releases20:29
*** tonytan4ever has joined #openstack-infra20:29
AJaegerjeblair: I loved your patch series and rebased and +2ed - thanks!20:30
openstackgerritAntoine Musso proposed openstack-infra/zuul: Fix testtools requiring unittest2  https://review.openstack.org/16229120:30
jeblairdhellmann: it looks like it has a few more commits in it, so at least it's not exactly the same20:30
dhellmannjeblair: ok. we've re-tagged libs in the past without anyone getting confused, so I really thought that was a simple solution.20:31
*** radez is now known as radez_g0n320:32
nikhil_kdhellmann: yes20:32
nikhil_kah, see that is figured out earlier..20:33
nikhil_kno issues with newer commits, they needed to go in soon anyways :)20:33
*** otter768 has joined #openstack-infra20:35
*** dmsimard has quit IRC20:36
*** achanda has quit IRC20:37
*** SumitNaiksatam has quit IRC20:37
marcusvrnasselin: ping20:38
*** ddieterl_ has quit IRC20:39
*** ddieterly has joined #openstack-infra20:39
*** ivar-lazzaro has quit IRC20:39
*** otter768 has quit IRC20:40
asselinhi20:41
*** rkukura has quit IRC20:41
asselinmarcusvrn, pong20:41
*** crc32 has joined #openstack-infra20:43
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Log info about updated and deleted jobs  https://review.openstack.org/16229320:46
marcusvrnasselin: hi! I was looking at my CI...and I saw that zuul is checking out the patches into its git repo (configured in zuul.conf)...but how can slaves use this repo to run the tempest against the code with that patch? Because I think my CI is running tempest against the devstack master, with no patch applied...20:46
*** sc68cal_ is now known as sc68cal20:46
lxslianteaya: clarkb: thanks20:47
asselinmarcusvrn, which os is zull running on?20:47
marcusvrnasselin: ubuntu20:47
asselinmarcusvrn, which version?20:47
marcusvrnasselin: ubuntu 12.04.520:48
asselinmarcusvrn, ok ...and are you using devstack-gate scripts?20:49
marcusvrnasselin: yes20:49
asselinmarcusvrn, do you have a link to one of the jobs in question?20:50
*** ivar-lazzaro has joined #openstack-infra20:51
*** ivar-lazzaro has quit IRC20:51
*** ivar-laz_ has joined #openstack-infra20:52
*** ivar-laz_ has quit IRC20:52
marcusvrnasselin: http://177.84.241.119:10000/28/159528/2/check/hds-hnas-iscsi/e332854/20:53
*** tnovacik has quit IRC20:54
asselinmarcusvrn, http://177.84.241.119:10000/28/159528/2/check/hds-hnas-iscsi/e332854/logs/devstack-gate-setup-workspace-new.txt.gz#_2015-02-28_08_28_03_19520:54
openstackgerritClark Boylan proposed openstack-infra/nodepool: Test that dib failures don't stop node builds  https://review.openstack.org/16223720:55
openstackgerritClark Boylan proposed openstack-infra/nodepool: Don't allocate nodes to providers without images  https://review.openstack.org/16225520:55
clarkbjeblair: ^ local testing seems to indicate that is working as expected20:55
clarkban I forgot to set the topic again :/20:55
clarkbbut consider that ready for review now20:55
erlonasselin: see the devstack.log  for https://review.openstack.org/#/c/161365/20:57
marcusvrnasselin: yep, this remote repo is the one that we have configured, but it's not the one that zuul is checking out and merging patches.20:58
erlonasselin: if you seek for 'git show --oneline', it says: "e34e6ae Add oslo.policy to global requirements", which is note the change in test20:58
asselinmarcusvrn, not sue this is right: http://177.84.241.119:10000/28/159528/2/check/hds-hnas-iscsi/e332854/console.html20:58
asselinZUUL_BRANCH=master20:58
asselinmarcusvrn, I'm looking at ours for that patch and we don't set it: http://15.126.198.151/65/161365/3/check/3par-iscsi-driver-master-client-pip-eos10-dsvm/7c788c8/console.html20:59
*** dprince has quit IRC20:59
marcusvrnasselin: do you have a local openstack git repo for your CI?21:00
*** achanda has joined #openstack-infra21:00
asselinmarcusvrn, we're using a mirror21:01
*** dimsum__ has quit IRC21:01
asselinmarcusvrn, you should see something like this which pulls it from zuul: http://15.126.198.151/65/161365/3/check/3par-iscsi-driver-master-client-pip-eos10-dsvm/7c788c8/logs/devstack-gate-setup-workspace-new.txt.gz#_2015-03-06_13_11_48_77021:01
marcusvrnasselin: how did you configure the repo to make zuul and slave use the same patch?21:03
asselinmarcusvrn, does your job look something like this? https://github.com/rasselin/os-ext-testing-data/blob/master/etc/jenkins_jobs/config/dsvm-cinder-driver.yaml.sample#L6721:03
asselinmarcusvrn, zuul sets up the change, passes the ref to the jenkins job which gets consumed by devstack-gate script21:04
asselinmarcusvrn, in jenkins, you can see all the environment variables passed in for completed jobs21:04
marcusvrnasselin: yes, my job is something like that you sent me21:05
clarkbjeblair: any word on the leaked ports? wondering if maybe we should just go ahead and dlete them21:06
asselinmarcusvrn, then you need to look at the env variales in jenkins. There should be a ZUUL_BRANCH set there21:06
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Log info about updated and deleted jobs  https://review.openstack.org/16229321:08
asselinmarcusvrn, e.g. this is what I see in mine. please paste yours: http://paste.openstack.org/show/190514/21:08
*** bcourt has joined #openstack-infra21:08
*** arosen has left #openstack-infra21:09
marcusvrnasselin: http://paste.openstack.org/show/190515/21:10
AJaegerclarkb: could you review https://review.openstack.org/162093 today, please? We want to translate RST files in openstack-manuals...21:10
clarkbAJaeger: uh oh bash arrays, this will be fun :)21:11
AJaegerclarkb: we use them already in openstack-manuals21:11
AJaegerclarkb: but I had to write a test script to check that everything works ;)21:11
clarkbya its fine I just have to lookup what the symbols mean each time21:11
*** yamamoto has joined #openstack-infra21:12
asselinmarcusvrn, strange. these lines seem to indicate nothing is set...? http://paste.openstack.org/show/190516/21:12
asselinmarcusvrn, are the variables getting unset anywehre?21:13
clarkbAJaeger: doc-tools-check-languages.conf does that exist in the repo(s) yet?21:13
marcusvrnasselin: I think I know what's going on...I have 2 jobs...one that handles the slave node VM and another one (triggered after the VM is ready) that runs the devstack-gate scripts..... the ZUUL env variables are on the master node, but it's not being passed to the slave node21:13
clarkbAJaeger: its set -e so will error if not21:13
AJaegerclarkb: it does21:14
asselinmarcusvrn, so the job doesn't get sent directly to the jenkins slave? I guess you have a different setup21:15
AJaegerclarkb: http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc-tools-check-languages.conf21:15
marcusvrnasselin: I think I have to pass those variables to the slave which will run the devstack-gate scripts21:15
*** ldnunes has quit IRC21:15
*** kgiusti has left #openstack-infra21:15
clarkbAJaeger: thank you21:15
AJaegerthe other repos don't use SPECIAL_BOOKS yet but have the file21:15
*** yamamoto has quit IRC21:16
marcusvrnasselin: yes, the job get sent to the slave that will handle the VM and then, this slave will trigger another slave to runs devstack-gate on the VM21:17
clarkbAJaeger: lgtm21:17
marcusvrnasselin: so, I think all that I have to do is to pass the env variables to that slave which will run the devstack-gate..21:18
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Change log level for job deletion and add logging for job creation  https://review.openstack.org/16230221:18
asselinmarcusvrn, yes21:18
AJaegerclarkb: thansk for the review21:18
*** rkukura has joined #openstack-infra21:18
*** thingee has joined #openstack-infra21:18
marcusvrnasselin: do you know which env variables the devstack-gate uses? Does it use only ZUUL variables?21:19
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Allow JJB to set connection timeout to jenkins server  https://review.openstack.org/15147221:20
asselinmarcusvrn, yes, definitely the ZUUL*. Not sure what else.21:21
marcusvrnasselin: and if I pass the variables to the slave, I will not need this (http://paste.openstack.org/show/190520/) anymore, will I?21:22
marcusvrnasselin: perfect, I'm going to fix it now!21:22
asselinmarcusvrn, actually, I removed those from my jobs already....21:22
marcusvrnasselin: nice21:23
openstackgerritAndreas Jaeger proposed openstack-infra/project-config: Support translation of RST files  https://review.openstack.org/16209321:24
AJaegerclarkb: yes, we can use the same way - just different directories ^21:24
*** rkukura has quit IRC21:24
marcusvrnasselin: Thanks!! You helped me a lot!21:25
asselinmarcusvrn, you're welcome21:25
*** ihrachyshka has quit IRC21:26
*** teran has joined #openstack-infra21:27
*** julim has quit IRC21:27
*** AJaeger has quit IRC21:28
*** jp_at_hp has joined #openstack-infra21:28
*** persia_ is now known as persia21:32
*** pc_m has quit IRC21:32
*** jp_at_hp has quit IRC21:33
*** harlowja_away has quit IRC21:34
*** salv-orlando has joined #openstack-infra21:36
bhunter71If one's gerrit account is Administrator group, and Administrator is granted 'Start Replication' - doesn't this mean I should be able to "gerrit replicate" over SSH?   [could have sworn this worked in the past -any ideas?]21:37
*** dboik_ has quit IRC21:37
bhunter71"gerrit: replicate: not found"21:37
*** dustins has quit IRC21:38
*** rfolco has quit IRC21:38
clarkbthey changed the command its `replication start` now21:38
*** jcoufal has quit IRC21:39
*** hodos|2 has joined #openstack-infra21:39
bhunter71hrm.  gerrit: replication: not found21:39
openstackgerritKhai Do proposed openstack-infra/jenkins-job-builder: Change log level for job deletion and add logging for job creation  https://review.openstack.org/16230221:40
clarkbit is a plugin maybe it isnt installed?21:40
bhunter71further, if I just say 'gerrit', I get a list of commands with nothing resembling replication21:40
zarobhunter71: yeah, depend on which version of Gerrit. but it's a core plugin in gerrit 2.921:41
*** hodos has quit IRC21:41
*** dimsum__ has joined #openstack-infra21:41
zaroclarkb: was wondering if you could help with this? https://review.openstack.org/#/c/158387/21:42
bhunter71interesting.  I still have 2.8.   maybe I downloaded a plugin in the past.21:42
*** koolhead17 has quit IRC21:42
bhunter71thanks zaro/clarkb21:42
clarkbzaro ya sorry keep getting distracted by fires21:42
*** superanne has quit IRC21:44
*** _shaps_ has joined #openstack-infra21:45
clarkbzaro wow that picture of the drawing I made21:45
marcusvrnasselin: I tested and now it's working! thanks again! :)21:46
asselinmarcusvrn, glad to hear! :)21:46
*** signed8bit has quit IRC21:46
*** marun has quit IRC21:47
*** bradm has quit IRC21:47
zaroclarkb: you must be looking at pleia2 slides.21:47
*** amotoki has quit IRC21:48
*** signed8bit has joined #openstack-infra21:48
clarkbzaro https://review.openstack.org/#/c/158387/5/images/ci_team_infrastructure_draft.png21:48
clarkbI approved21:48
mtreinishheh, I remember that21:48
*** baoli_ has quit IRC21:49
zaroclarkb: thanks, that's one of the openstack works of arts.  maybe should enter it into next tshirt contest :)21:51
clarkbha21:51
*** mattymo has quit IRC21:51
*** mattymo has joined #openstack-infra21:52
*** superanne has joined #openstack-infra21:52
flashgordonstill looking for reviews for  https://review.openstack.org/#/c/160548/ and https://review.openstack.org/#/c/160933/21:52
flashgordonexpected to shave 5 minutes of all devstack jobs21:52
jesusaurusdo zuul pipelines get starved by other pipelines with a higher precedence? will nodes be allocated to a normal queue over a low queue without regard to the length of time a job is waiting in a low queue?21:53
flashgordonand  over 1k dsvm jobs in in zuulat this very moment that is a lot of CPU hours21:53
flashgordonclarkb: ^21:54
flashgordonmordred: ^21:54
*** nelsnelson has quit IRC21:55
mordredflashgordon: looking21:55
zarojesusaurus: not sure, but that seems reasonable.  did you expect something else?21:55
clarkbflashgordon do all three devstack branches have that script?21:56
mordredflashgordon: I have a question you may not like21:56
jesusauruszaro: it does seem reasonable, but I was wondering if it might be a bit smarter for cases when zuul is under prolonged resource contention21:56
*** prad has joined #openstack-infra21:56
flashgordonclarkb: I don't think so but not 100% sure21:56
mordredflashgordon: how do you have the c library dependencies installed such that you can build the c extensions21:57
clarkbmordred hahaha good point21:57
mordredflashgordon: because normally those are installed by devstack21:57
flashgordonmordred: oh yeah ......21:57
flashgordonfuuuuuuuuu21:57
* mordred apologizes21:57
flashgordonck21:57
flashgordonmtreinish: ^21:57
clarkbmordred please leave review comment :)21:57
mordredclarkb: I will21:57
mordredI FULLY support the goal21:57
mordredbtw21:57
mtreinishmordred: because nodepool installs the apt list before we build the wheel21:58
mordredno it doesn't21:58
flashgordonit installs some things for sure not sure if its enough21:58
mordredit shoudl install none of the things21:58
mordredand if it does, it'sa  bug21:58
mtreinishthen I have no idea, let me check when devstack calls the wheel build script21:59
mtreinishbecause I though it did that before we run apt for anything21:59
jlvillaljeblair: gertty --print-keymap prints the keymaps that can be set.  But it doesn't print what they are set to.  Is there a way to get a list of the keymap settings?21:59
flashgordonmtreinish: I am running it inside an empty env now to test21:59
flashgordonmtreinish mordred: http://paste.openstack.org/show/190527/22:00
zarojesusaurus: so under prolonged resource contention you want low pri jobs to run before stuff in normal queue?22:00
mordredflashgordon: what caused that?22:00
flashgordon./tools/build_wheels.sh22:00
*** marcusvrn has quit IRC22:00
jesusauruszaro: under heavy resource contention i want low pri jobs to run more than 0% of the time, but less often than normal pri jobs22:01
flashgordonusing a base docker ubunut:14:04 image with just a few things isntalled22:01
mtreinishd'oh it's kinda obvious it's called right in the script22:01
mtreinishhttp://git.openstack.org/cgit/openstack-dev/devstack/tree/tools/build_wheels.sh#n5022:01
flashgordonmtreinish: doh22:01
mordredso - we coudl run this in a docker container, copy out the results and delete the docker container22:02
openstackgerritRamy Asselin proposed openstack-infra/nodepool: Fix Nodepool DIB Uploads to Glance w/o region_name  https://review.openstack.org/16231322:02
mordredjust as a way to get the wheelhouse pre-done but not pollute the machine22:02
clarkbor use our mirrors again22:02
mordredclarkb: or that22:02
clarkbI am now convinced this is what we should do22:02
flashgordonI still think the long term answer is: stack.sh ; unstack.sh22:03
mordredI'm fairly convinced of that - it would also lessen the frequency of rebuilds of things like lxml even more22:03
flashgordonand have a new job for devstack that runs on a clean env that didn't have that22:03
flashgordonbut that doesn't work with chroot ...22:04
*** tiswanso has quit IRC22:05
openstackgerritGene Snider proposed openstack-infra/zuul: Migrate zuul timer to apscheduler version 3.0  https://review.openstack.org/16231522:06
*** _shaps_ has quit IRC22:06
flashgordonbut install_prereqs.sh ; build_wheels.sh would get us pretty far22:06
*** jgrimm is now known as zz_jgrimm22:06
mordredflashgordon: yah - install_prereqs.sh ; build_wheels.sh in a chroot or a docker would be a very clean way to do it22:07
*** ddieterly has quit IRC22:07
*** ddieterly has joined #openstack-infra22:07
flashgordonright now there are some issues with dib though22:07
flashgordongreghaynes: spotted it22:07
mordredoh - right.22:08
mordredsorry - I forgot the dib part of this22:08
flashgordonit cleans out the apt-cache and links to the host by default22:08
flashgordonbut turns out the apt-get install bits aren't that slow22:08
flashgordonmaybe save a minute all together tops22:08
mordredand we can't build-host cache it because it's wheels22:08
*** radez_g0n3 is now known as radez22:08
*** mattfarina has quit IRC22:09
*** timcline_ has quit IRC22:09
jeblairclarkb, mordred: so...22:09
jeblairclarkb, mordred: the case i opened with hp public cloud says "Awaiting Customer Input"22:09
jeblairclarkb, mordred: but i can't actually see anywhere they have said or asked anything22:09
mordredjeblair: that's awesome22:09
*** baoli has joined #openstack-infra22:10
jeblairthe description field seems the same as i left it, and there is nothing in the comments section22:10
*** woodster_ has quit IRC22:10
mordredjeblair: have you tried the noc channel? or would you like me to?22:10
*** hdd_ has quit IRC22:10
jeblairmordred: i'll go there now22:10
openstackgerritJames Polley proposed stackforge/gertty: Expand sample keymaps to ameliorate OSX features  https://review.openstack.org/15748822:13
zarojesusaurus: from looking available zuul params it's unlikely that it does that.22:13
*** pblaho has joined #openstack-infra22:13
*** pblaho has quit IRC22:13
*** bradm has joined #openstack-infra22:14
*** dimsum__ has quit IRC22:16
*** dimsum__ has joined #openstack-infra22:17
jeblairmordred, clarkb: are we using any neutron api calls with hpcloud?22:17
mordredjeblair: not in nodepool22:17
mordredjeblair: we currently use no neutron calls at all22:18
mordred(although I want to change that wrt floating ips when we move to shade, because it's a more solid approach)22:18
jeblairright22:18
mordredbut that's future22:18
openstackgerritGene Snider proposed openstack-infra/zuul: Migrate zuul timer to apscheduler version 3.0  https://review.openstack.org/16231522:19
*** tnovacik has joined #openstack-infra22:20
*** hdd_ has joined #openstack-infra22:21
*** dimsum__ has quit IRC22:21
*** e0ne has joined #openstack-infra22:22
*** dimsum__ has joined #openstack-infra22:23
*** r-daneel has quit IRC22:23
jlvillaljeblair: I feel like this is a dumb question.  But when I'm making a comment on a line in a patch.  How do I get out of the comment mode?  Hitting <ESC> takes me out of diff mode.  I was hoping for a better method.22:23
jeblairclarkb: short version: they agree none of the instances are active, and they also are not in the deleted instances list which goes back to july 2014, and don't know what they are.  they had some basic followup questions which i have answered.22:23
jeblairjlvillal: hit up or down to move the cursor out of the box22:24
*** chlong has quit IRC22:24
*** andreykurilin_ has joined #openstack-infra22:24
jeblairjlvillal: i'm guessing the regular arrow keys are going to work better than j/k for this one? ;)  also, i'm hoping you can still type j/k in the comment box, is that correct?22:25
jlvillaljeblair: Oh.  Sounds simple! :)22:25
jlvillaljeblair: Yep j/k work in the comment box.  I have to admit I would think escape should exit the comment box22:25
jlvillaland not the whole diff mode22:25
jlvillaljeblair: Thanks!22:26
clarkbjeblair so we should delete them?22:27
jeblairclarkb: gimme a second, they pointed me at a jira ticket... so i'm vpning22:27
*** aysyd has quit IRC22:28
*** radez is now known as radez_g0n322:31
*** ddieterly has quit IRC22:31
tchaypojlvillal: i found that confusing at first as well22:32
tchaypoalso I’d love to hack in keybindings for deleting all the comment as well22:32
tchaypohaving to hold down backspace for half an hour is so tedious22:32
tchaypothen again maybe i should just write shorter comments22:32
*** marcusvrn has joined #openstack-infra22:33
*** dangers is now known as dangers_away22:36
*** otter768 has joined #openstack-infra22:36
jeblairclarkb: i don't think they have a good answer on the deletion question, so perhaps we try to stay on the safe side and maybe delete 20-50 of them?22:37
jeblairclarkb: also jesusaurus reports this internally two, so extra data points22:37
jeblairtoo even22:37
jeblairjlvillal, tchaypo: that's good to know (i love user feedback like that).... do you expect that to happen after you've typed a comment in, or only when you've selected a line to start a comment, but then changed your mind and want to leave no comment there after all?22:39
jeblairjlvillal, tchaypo: (referring to the esc thing)22:40
jlvillaljlvillal: I was hoping <ESC> would just exit the comment line, but not erase it.22:41
*** otter768 has quit IRC22:41
*** julim has joined #openstack-infra22:41
jeblairjlvillal: where should the cursor go in that case?22:41
tchaypoHrm.22:42
jlvillaljeblair: Hmmm.  Good question! :)22:42
tchaypoThinking back on the times when I’ve hit esc by mistake22:42
tchaypoI think it’s usually when I’m in the middle of writing something and decide I should comment on a different line, or that I don’t want to comment at all22:43
*** dimsum__ has quit IRC22:43
jlvillaljeblair: I was thinking that you had to press <enter> to start editing the comment.  But now I see that if there is a comment there that it automatically puts you into editing mode if you land on the line.22:43
*** dimsum__ has joined #openstack-infra22:43
jeblairjlvillal, tchaypo: the paradigm is that the cursor is always on a line, either ready to create a comment or edit an existing comment, so that makes it a bit hard to figure out what to do in that case....22:44
jlvillaljeblair: I was thinking that you wouldn't be in editing mode, and that maybe could do 'd' to delete the comment at that point.  And if you hit 'esc' while in editing mode it would leave it.22:44
*** tonytan4ever has quit IRC22:44
jeblairjlvillal, tchaypo: i think our options are to rethink the paradigm, so that it behaves more like what jlvillal just described; or perhaps disable esc while in comment editing, so at least you have to leave comment editing some other way before you can leave the screen (so it won't happen by accident)22:45
*** mriedem has quit IRC22:47
*** dimsum__ has quit IRC22:48
tchaypojlvillal: let me guess22:49
tchaypojlvillal: you use vim?22:49
tchaypo;)22:49
jlvillaltchaypo: Yep :)22:50
tchaypoI don’t think I like either of those options22:52
tchaypooften when I’m editing a comment I want to pop out to check another change or something22:52
jlvillalIs there an IRC channel for Stackalytics?  Engineer stats don't show up it seems: http://stackalytics.com/?release=kilo&module=ironic-group&metric=commits22:53
tchaypoif esc was disabled *thinks* I’d have to arrow up, then hit esc22:53
tchaypoi guess that’s not too bad22:53
*** salv-orlando has quit IRC22:53
jeblairjlvillal: i'm not certain, but NikitaKonovalov is in here and may be able to help22:53
tchaypobut I think I’d be confused about why esc didn’t work straight away22:53
*** peristeri has quit IRC22:54
jeblairtchaypo: yeah.  :|22:54
jlvillaltchaypo: jeblair: I think <ESC> should exit the comment editing, but that would require a paradigm change.  Or have it move up to the line being commented on.22:54
tchaypoone thing we could do to make this more clear22:55
tchaypowould be to edit the comment in a popup rather than inline22:55
tchaypobut i think that would be even more disruptive22:56
tchaypocan’t see the code you’re commenting on, or comments you’re responding to22:56
jlvillaljeblair: Also it would be nice when starting a comment if it defaulted to the right hand side, as I think that is the current patch revision.  It seems odd to default to the previous revision to start commenting on.22:56
jlvillalUnless I'm doing something wrong.22:57
jeblairjlvillal: oh interesting, that might work... it looks like you might currently be able to create multiple comments by selecting the same line multiple times, that seems buggy.  at any rate, i think your suggestion of having esc move the cursor back to the line being commented on would work....22:57
jeblairjlvillal: would you then want to have to hit enter on the line again to edit your comment, or is the 'move cursor into edit box' paradigm okay for that?22:58
*** hodos|2 has quit IRC22:58
tchaypoon a different note22:59
jlvillaljeblair: Either way is fine for me.  The only benefit of the <enter> paradigm.  Is then could add a way to delete a whole comment or other actions on the comment.22:59
tchaypoas lifeless mentioned yesterday22:59
*** hasharAway has quit IRC22:59
tchaypoI’m finding that gertty takes a really long time to sync, and seems to spend more time with the counter increasing than decreasing22:59
*** superanne has quit IRC22:59
openstackgerritKyle Mestery proposed openstack-infra/project-config: Add functional test job for python-neutronclient  https://review.openstack.org/16232922:59
jeblairjlvillal: and yes, i think it should default on the right. i note that once you leave a comment on the right, it defaults there.  i believe i was fighting urwid to get the initial focus on the right, but lost.  may be time for a rematch.  :)23:00
jlvillaljeblair: LOL :D23:00
*** ParsectiX has quit IRC23:00
*** ParsectiX has joined #openstack-infra23:00
openstackgerritKyle Mestery proposed openstack-infra/project-config: Add functional test job for python-neutronclient  https://review.openstack.org/16232923:00
jeblairtchaypo: if you run with verbose or debug, do log messages give you insight?  is it working but really slow?  is it piling up duplicate sync requests?  are some requests hanging and not timing out properly?23:01
tchaypoi seem to be subscribed to 69 projects23:01
*** r-daneel has joined #openstack-infra23:02
jeblairtchaypo: (also note that many sync actions are effectively noop -- i find that when my counter gets really high, it's really only about half as bad as it looks)23:02
tchaypoI’m running with verbose at the moment23:02
tchaypohttps://www.irccloud.com/pastebin/umEjgE8h23:02
*** achanda_ has joined #openstack-infra23:03
*** achanda has quit IRC23:03
tchaypoyeah, it does seem to eventually hit a point where all the remaining changes are noops and then it drops quickly, but it can take hours to get there23:03
*** Ryan_Lane has joined #openstack-infra23:04
tchaypolet me switch to debug23:04
*** andreykurilin_ has quit IRC23:04
*** cdent has joined #openstack-infra23:04
tchaypothe database lock messages usually have quite short timestamps - 0.03<x<0.1s23:04
tchayposo I think it’s mostly network latency23:05
*** claudiub has joined #openstack-infra23:05
tchaypohttps://www.irccloud.com/pastebin/ALjS1XPZ23:06
*** r-daneel_ has joined #openstack-infra23:06
jlvillaljeblair: Probably another dumb question.  Is it possible to set in the config the default sort method?  I searched for 'sort' in the reference example doc, but did not have success.  I would like it to default to reverse-sorted by change date.23:06
*** r-daneel has quit IRC23:07
jeblairtchaypo: i synced the same changes in your paste and the times in the "Synced change .. in .." messages are roughly similar for me23:07
*** achanda_ has quit IRC23:07
*** amitgandhinz has quit IRC23:07
*** esker has quit IRC23:08
tchayporight after i say the database locks tend to be fairly short, I find one that holds a lock for 14 seconds..23:08
tchaypoI’m interpreting the “revision x needs parent commit”, “…. synced” messages as meaning that it’s doing a git fetch of individual patchsets?23:09
*** adalbas has quit IRC23:09
*** achanda has joined #openstack-infra23:10
*** r-daneel_ has quit IRC23:10
*** r-daneel_ has joined #openstack-infra23:10
*** gordc has quit IRC23:10
*** tnovacik has quit IRC23:12
tchaypoReasons I know I’m a bad person: seeing that https://review.openstack.org/#/c/161047/2/specs/stable-requirements.rst has 37 comments makes me chuckle23:12
*** cdent has quit IRC23:12
*** sandywalsh has quit IRC23:12
*** crc32 has quit IRC23:13
jeblairtchaypo: actually, no network traffic is happening during those comments...23:13
*** sandywalsh has joined #openstack-infra23:13
jeblairer log entries23:13
*** yamamoto has joined #openstack-infra23:14
jeblairtchaypo: basically the result of all those "needs parent commit" lines is that later, after this sync is finished, it will go do some git fetches for those...23:14
tchaypoah23:14
tchaypoin that case23:14
jeblairtchaypo: most of what's going on for those 14 seconds is actually local db queries... so there might be something really unoptimized there23:15
tchaypowhy does it take so long? there’s like a second in between each comment23:15
tchaypoah23:15
tchaypoalso23:15
tchaypoI get the feeling that something is firing off a fresh sync before the previous sync is finished23:15
tchaypowhen i first fire up gertty it will quickly get up to about 256 or so23:15
tchaypoif it’s been synced recently that drops back to 0 almost immediately23:15
tchaypobut if it hasn’t that number just grows and grows - and it seems like every now and then another ~200 or so get added23:16
jeblairjlvillal: i believe the defaults are hard coded, but i think a config option for them would be a great idea.  if you don't want to write a patch, would you mind filing a story?23:16
*** prad has quit IRC23:16
tchaypoonce it hits 800 or so my usual remedy is to kill gertty and re-open it23:16
tchaypoin this case it was somewhere north of 800 when i killed and restarted to get debug logs, and now it’s fully synced23:17
openstackgerritJoe Gordon proposed openstack-infra/devstack-gate: force_config_drive always=>True  https://review.openstack.org/16233923:17
flashgordonmtreinish: have a chance to re-spin your patch?23:17
tchaypoin about 10 minutes; where before I’d had it open for >20 minutes and the count was still increasing23:17
jlvillaljeblair: I can investigate a patch.  If I did make it a config option.  What should the config option be called and where should it be placed?23:17
*** yamamoto has quit IRC23:18
jlvillaljeblair: As in top_level_name -> second_level_name -> etc...23:19
jeblairjlvillal: there's a top level "change-list-query" option, so probably "change-list-sort: <keywords: number, updated>" and "change-list-reverse: <bool, default False>" also at the top level23:19
tchaypowhile we’re talking feature requests23:19
jlvillaljeblair: Okay23:19
tchaypoI’d love to be able to switch between side-by-side and unified diff on the fly23:20
jeblairtchaypo: the periodic sync should wait until it's completely finished before it adds anything else to it... so the += 200 tasks should only be coming from things it's discovering it has to do while syncinc...23:20
mtreinishflashgordon: which patch? I have many23:21
mtreinishoh, probably the wheels one. What did I have to do, I missed the trail end of that discussion23:21
*** e0ne has quit IRC23:22
tchaypoflashgordon: so this might just be me being overly relaxed because it’s saturday23:22
jeblairtchaypo: interesting; i thought about that but didn't expect anyone to want to do that.  it should be possible, though would be a little bit of work.  out of curiousity, what would you get from that?  also -- literally toggle while you're already on a diff, or would going back to the change, switching, then hitting diff again be okay?23:22
tchaypoflashgordon: but I’m wondering if we should be thinking about adding features to pip23:22
tchaypoflashgordon: the ability to specify “here are the allowed packages and their allowed versions” seems like it would be useful in many contexts other than ours.23:23
jlvillaljeblair: And what do you think about unittests?23:23
tchaypojeblair: the latter would be fine23:24
tchaypowhen I’m looking at a modification of existing code, side-by-side is often helpful23:24
tchaypobut just now I was looking at a brand new file, so the left half of the window is entirely blank - unified would have used the space better23:24
*** ChuckC_ has quit IRC23:25
tchaypoalso, I often find myself splitting the pane in tmux and having gertty on the left with docs or some other code on the right, or even running the code. In that circumstance side-by-side often makes the columns too narrow to really see the change; unified would work better23:25
flashgordonmtreinish: a sudo or two, clarkb commented inline23:25
jeblairjlvillal: i think they might be cool for some things.  i really think the search could benefit from testing, for instance, keymaps, palettes, and maybe the sync stuff too.  i'm not sure what some of the ui behavioral stuff would look like unit tested.  understand that i'm skeptical of mocks and generally only want to test real behaviors.23:25
mtreinishflashgordon: oh, just that. I thought there was more to do. That's simple enough, one sec23:26
jlvillaljeblair: Okay.23:26
flashgordontchaypo: its not a bad idea, adam_g mentioned something similar23:26
jeblairtchaypo: got it, thanks :)23:26
flashgordontchaypo: I agree others have the same issues but on a smaller scale23:26
tchaypoflashgordon: maybe it’s a sign I’ve spent too much time in Enterprise environments though23:27
flashgordontchaypo: hehe23:27
tchaypoI think I’m thinking of an ITIL software librarby style thing23:27
*** armax has quit IRC23:27
flashgordonif you look at the bottom of the spec there is some related work there23:27
tchaypoA different way to manage this might be to run a devpi server and control which versions are available on it23:27
openstackgerritMatthew Treinish proposed openstack-infra/project-config: Build wheel house during nodepool devstack caching  https://review.openstack.org/16054823:28
mtreinishflashgordon: ^^^23:28
*** doug-fish has left #openstack-infra23:28
flashgordonclarkb:  ^^23:29
jeblairjlvillal: not sure how i missed this, but check out change 16103223:29
flashgordonmtreinish: so the best part of this, is we built numpy but don't even use it23:29
flashgordonwe still have the apt-get installed version in there23:29
mtreinishflashgordon: it's a 2nd level dep for something in nova23:30
mtreinishoh, heh I thought we removed the package, or is something else pulling it in23:30
jlvillaljeblair: :)  Just as I was doing it....23:30
* jlvillal goes to stash changes and cherry-pick patch23:30
flashgordonmtreinish: sigh23:31
*** stevedore has quit IRC23:31
*** signed8bit has quit IRC23:32
mtreinishflashgordon: nope, I'm wrong it looks like python-numpy is still in the apts lists for nova23:35
mtreinishif we want to use the wheel that's probably fair game to remove23:35
flashgordonmtreinish: so have a short term answer https://review.openstack.org/16234323:36
flashgordonwhich is the opposite23:36
flashgordonheh building numpy is single threaded23:37
flashgordonfail23:37
mtreinishflashgordon: well that only works until dtroyer's patches to use a venv for services lands23:37
flashgordonmtreinish: yup, as I said short term23:37
flashgordontiming a numpy wheel build on my box to get a rough estimate on how bad it is23:38
flashgordonmtreinish: how many dsvm jobs do we run a day?23:38
mtreinishflashgordon: I'm sure graphite can give you the complete answer. I can check subunit2sql for how many gate tempest/grenade dsvms we ran yesterady, one sec23:39
*** woodster_ has joined #openstack-infra23:39
*** mattfarina has joined #openstack-infra23:40
flashgordonwow 4 minutes23:41
flashgordonso yeah23:41
flashgordonIMHO worth temporary hacks just to get rid of that23:41
jlvillaljeblair: That patch worked for me :)23:41
flashgordonmtreinish: that is 4 out of the 20 minutes or so devstack takes ...23:41
jeblairjlvillal: cool, thanks23:42
flashgordonmtreinish: on zuul I see 675 dsvm jobs that are running or recently finished so this would save about 45 instance hours off of that23:42
*** ChuckC_ has joined #openstack-infra23:43
mtreinishflashgordon: accoring to the db we ran 836 gating jobs  between '2015-03-05 00:00:00' and '2015-03-06 00:00:00'23:44
mtreinishgating tempest or grenade23:44
*** ChuckC_ is now known as ChuckC23:44
jeblairmtreinish: not counting check?23:44
*** dmsimard has joined #openstack-infra23:45
mtreinishjeblair: nope the subunit2sql db only collects the streams from gate jobs23:45
flashgordonmtreinish: wow that is a lot of gate jobs23:45
jeblaircheck is much more23:45
flashgordonI assume that is only completed jobs too23:45
mtreinishhttp://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/logstash/jenkins-log-client.yaml#n1423:45
jeblairflashgordon, mtreinish: yesterday zuul launche 8766 jobs with 'dsvm' in their names23:47
flashgordonjeblair: wow!23:48
*** ghostpl_ has quit IRC23:48
mtreinishjeblair: heh, nice23:48
flashgordonmtreinish: we build numpy wheels 4,782 in the past 24 hours per logstash23:48
flashgordonmtreinish: or 318 hours of wall time23:49
flashgordonroughly23:49
jeblairflashgordon, mtreinish: two days ago 10671 dsvm jobs, but that may have been an abnormally heavy day due to the queue from the rax maint work.23:49
flashgordonout of that we are getting about 150-200 merges to openstack, openstack-infra, openstack-dev23:50
flashgordonhttps://github.com/openstack/openstack/graphs/commit-activity23:50
flashgordonthat is really impressive23:50
* flashgordon wonders how big travis CI is23:50
jeblairflashgordon, mtreinish: yesterday we launched 34841 jobs total23:51
mtreinishflashgordon: heh, I'm trying to get the run count for yesterday as meaninglessly large number to brag about. But the query is really slow...23:51
jeblairflashgordon: we are significantly larger than travisci23:52
jeblairmordred: ^ what was it last time you checked?23:52
flashgordonmtreinish: haha fail23:52
* mordred forgets23:52
mordredbut we were decently bigger23:52
flashgordonat 34k jobs I would be surprised if we are just 'decently' bigger23:53
jeblairflashgordon, mtreinish: 35300 total jobs launched 2 days ago.23:53
jeblairistr it was an integer factor larger23:53
mtreinishflashgordon: 627394 tempest tests as part of the gate queue yesterday23:54
mtreinish(that took 6 min 42.55 sec I probably need to make the schema better)23:54
mtreinishjeblair: hmm, 35300 over 2 days but 34841 yesterday23:56
mtreinishoh nm misread23:56
jeblairmtreinish: status:open (topic:enable_swift OR topic:dib-nodepool OR topic:zanata OR topic:downstream-puppet OR topic:askbot-site OR topic: gerrit-upgrade)23:56
jeblairgah23:56
jeblairmtreinish: 35300 on tuesday23:57
*** mattfarina has quit IRC23:57
jeblairmtreinish: 34841 on thursday23:57
jeblairfor UTC days23:57
mtreinishjeblair: yeah, I just misread what you put earlier23:58
*** dimsum__ has joined #openstack-infra23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!