Sunday, 2014-06-15

mordredyeah. I mean, there are MANY things I'd do different00:00
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add unit tests for SearchEngine.search  https://review.openstack.org/10001700:01
*** zzelle has quit IRC00:01
mordreddstufft: yup. it's a setuptools bug00:03
mordrederror: [Errno 2] No such file or directory: '/home/mordred/src/openstack/python-swiftclient/.tox/py27/local/lib/python2.7/site-packages/setuptools/script (dev).tmpl'00:03
mordredthe simple answer is right - there is, in fact, no such file or directory00:03
mordredthere are no .tmpl files installed into my virtualenv00:03
mordredclarkb: ^^00:04
clarkbnice00:04
dstufftis there a script (dev).py00:05
mordrednope00:05
dstuffthrm00:06
mordreddstufft: https://bitbucket.org/pypa/setuptools/issue/220/script-tmpl-files-missing-in-50100:07
dstufftmordred: fwiw shit you'd do differently feel free to email me00:07
mordreddstufft: oh - no, I meant if I ruled the world00:07
dstufftoh00:07
dstufftyea that00:08
mordreddstufft: like, everyone does it all wrong00:08
*** otherwiseguy has quit IRC00:08
mordredbtw - did I tell you my "have pbr generate a static setup.py file" idea?00:08
SukhdevI just filed a bug to report this failure https://bugs.launchpad.net/python-swiftclient/+bug/133014000:08
uvirtbotLaunchpad bug 1330140 in python-swiftclient "stack.sh fails because of python-swiftclient depenendencies" [Undecided,New]00:08
mordredSukhdev: yeah - there are some setuptools issues - I'm looking in to them00:09
SukhdevMost of the Third party Test systems have started to fail00:09
dstufftmordred: yes, I had a similar idea00:09
dstufftI made a repo for it, not sure i'll ever bother to do it00:09
*** salv-orlando has joined #openstack-infra00:09
mordreddstufft: I got halfway finished00:10
dstufftI don't want a setup.py at all00:10
dstufftexcep in the tarball00:10
mordredyup00:10
mordredthat's what I'd love to see00:10
mordredif you want to hack on openstack from the git repo, then you need to have pbr installed and you'll have to use it00:10
mordredif you want to just blindly consume - you need to consume a tarball00:10
Sukhdevmordred: Thanks - would you please update the bug, when you are able to figure out the fix/workaround - I am sure many guys are probably trying to figure out - as their CI systems are failing00:11
mordredI think there's about 5 things I need to get done before I can get that working00:11
dstufftbreaking -e kinda sucks00:11
dstufftnot sure how I feel about that yet00:11
dstufftin my own solution at least00:11
dstufftwhich exists entirely in my head00:11
dstufft:V00:11
mordredwell, there's nothing saying that, with pbr installed, you coudln't do that00:11
dstufftwell you'd need a shim setup.py00:11
mordredor you could just do "pbr develop" :)00:11
dstufftpip doesn't know to do that00:12
mordredit doesn't need to00:12
mordredpbr could then generate the thing and then pass off to pip00:12
dstufftit's kinda lame to have to do ``pbr develop``00:12
mordredyah.00:12
mordredyou want to keep pip install -e .00:12
dstufftbecause taken to the logical conclusion, not everyone wants to use pbr, so now I have to know what the correct "develop" command is based on what tool a person uses00:13
mordredbut also not have a setup.py in the git repo00:13
dstufftlong term probably there will be some sort of hook, that defines what you need to do00:13
dstufftsome file that defines "pbr.packagingthing:PackageMaker"00:13
mordredI yup00:14
dstufftthat pip can import and go "hey pbr, make me a package"00:14
dstufftthen chuck that package through the rest of stuff00:14
dstufftthat doesn't actually solve the -e case, but solves the direct from git case00:14
mordredalmost like this: http://git.openstack.org/cgit/openstack/neutron/tree/setup.cfg#n8800:14
dstufftbut could do something similar for -e, which just says "hey pbr, give me some metadata so I can add this path to the sys.path"00:15
dstufftall pip -e does (and setup.py develop) really is generate metadata and add add curdir to sys.path00:15
dstufftmetadata includes script wrappers00:15
dstufftdunno what to do about that00:16
dstufftnot sure I care about script wrappers, can just use python -m00:16
mordredyou know ...00:16
mordredit would be pretty trivial to make a patch to pip that looks for an entry in setup.cfg for "how should I run setup"00:17
mordredwich would default to "use the setup.py file stupid"00:17
mordredbut which, if defined, would be the definition of a callable that pip could callable00:17
dstufftit's trivial to do that, but we're not going to add that to pip without a PEP00:18
mordred:)00:18
dstufftpart of the problem of setuptools is that it just added whatever seemed useful, but did a shit job at documenting it and telling people how to consume shit without setuptools00:18
*** dims has joined #openstack-infra00:19
dstufftthis time around we want defined standards, this is what a X looks like00:19
dstufftso if you don't want to use the standard tooling, that's fine, it just has to understand the formats at play00:19
dstufftmordred: lifeless Nick said using + instead of ~ is OK with him00:20
dstufftso looks like that's what it'll be00:20
mordredwoot!00:21
mordreddstufft: thanks!00:21
mordredmy brain will hurt less that way00:22
dstufftyay PEP process00:22
dstufftPEP 440 isn't accepted yet, so IDK if y'all want to wait for your semver thing until it is or not00:23
dstufftI'm trying to get it accepted soonish00:23
*** dims has quit IRC00:23
mordredinteresting question. when lifeless get back, we should talk about whether or not we want to replace our .gXXXXXX with +gXXXXXX in anticipation of that00:28
mordredlifeless: stop enjoying spending time with your daughter and come talk to me about version numbers!00:29
* mordred hides00:30
dstufftmordred: oh and you asked earlier, metadata 2.0 still has a field to explicitly mark down what the "source version" is, which could be a git hash or whatever00:30
mordredyah. once metadata 2.0 is a thing, I'm totally game to use that00:30
dstufftpip won't use it or anything00:30
dstufftso it depends on what you want it for00:30
mordredyah. in our case, I think we'll _always_ put the source git sha00:31
mordredbecause that's a nice thing to know in general00:31
*** arnaud has joined #openstack-infra00:31
mordredbut if we do that, then I think adding the +$sha in the visible version string can probably stop00:31
dstufftif it's just to record what git hash somthing was, it's what you want, if you want to have something to differentiate between two different 1.0.dev4's from different git commits00:31
dstufftthen you'l still want +00:31
mordredyah00:31
dstufftthere is a defined order for +whatevers00:32
mordredI think I'm _less_ concerned about that00:32
dstufftso it's static00:32
mordredit's making sure that we have a dev3 and a dev400:32
dstufftbut it's just a best effort ordering kind of thing, it doens't have semantic meaning00:32
mordredif you're tracking not-even-trunk - I mean, there's only so much magic I can pull, right?00:32
dstufftbasically you just split on . and if it's all digits you order numerically and if it's alpha numeric it's ordered lexographically00:32
dstufftis how we order 1.0+foo and 1.0+bar00:33
dstufftbut that's just because we _need_ an order, not because foo is nescarily a higher version than bar00:33
*** Sukhdev has quit IRC00:38
*** mriedem has joined #openstack-infra00:39
*** otherwiseguy has joined #openstack-infra00:43
tchaypoSo, I was talking to lifeless on Friday and he's suggested I kick off a change to add a job to infra/config to test stuff on our CD cloud00:44
tchaypobut I'm not sure what "a job" means. I'm guessing a jenkins job..00:45
clarkbtchaypo: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config houses all of our jenkins jobs00:47
tchaypogood, that's the directory I'm looking in00:47
clarkbtchaypo: then in modules/openstack_project/files/zuul/layout.yaml you instruct zuul to run jobs00:47
clarkband yes I would interpret what you are saying to mean add a jenkins job and have zuul run it00:47
*** otherwiseguy has quit IRC00:47
tchaypoI have no knowledge of either jenkins or zuul, so this will be fun00:48
tchaypoi know what jenkins is - 4 years living with a Bamboo developer and then 3 years at Atlassian mean that I've heard it referenced many times00:49
clarkbthe jenkins job builder part should be relatively straight forward. you can cargo cult00:49
*** lttrl has quit IRC00:49
clarkbyou don't really need to know much about jenkins as much as knowing what you want to do and copy pasta00:49
tchaypoyeah, looking at the config files is enlightening00:49
clarkbtchaypo: http://ci.openstack.org/jenkins-job-builder/ docs for JJB if you end up being curious00:49
tchaypoI just ran across a comment_filter line and had a "oh, so that's how that gets triggered" moment00:50
clarkbthen the zuul layout.yaml basically maps gerrit stream events to projects and from there jobs00:50
clarkbgerrit says new patchset for project foo -> project foo section in layout.yaml -> run check jobs because that is what happens when new patchset is created00:50
tchaypoI'm guessing the jobs to test things on CD might be in tripleo.yaml00:50
clarkbI think so00:50
*** sarob_ has joined #openstack-infra00:51
*** arnaud has quit IRC00:53
clarkbtchaypo: feel free to ask questions but I need to run down the street to get dinner00:55
clarkbback later00:55
tchaypono problem00:55
*** sarob_ has quit IRC00:55
tchaypowhat i think I'll do is have a stab at something and push a change00:55
tchaypoand then people can tell me all the things I got wrong00:55
*** arnaud has joined #openstack-infra01:04
openstackgerritJames Polley proposed a change to openstack-infra/config: WIP: Add definitions for a job to check vlan config  https://review.openstack.org/10007601:07
*** dims has joined #openstack-infra01:08
*** otherwiseguy has joined #openstack-infra01:09
mordreddstufft: if you're still around, I'd love a pointer on submitting patches to projects on bitbucket01:10
*** Sukhdev has joined #openstack-infra01:11
tchaypoin the general case, or in the specific case of pypa projects?01:11
mordredtchaypo: in any case at all01:11
mordredtchaypo: I would like to fix a bug in setuptools that I have found01:12
*** mriedem has quit IRC01:12
mordredis the right thing to do to "fork" the repo, clone that locally, fix the problem, comit, push and then submit a merge request?01:12
tchaypoyeah, same as on github01:13
tchaypothat's what I did with my pip bug recently01:13
mordredok. I usually get lost as soon as I'm asked to fix something01:13
tchaypo"Don't I have minions for that?"01:13
mordredbut let's see if I can just get it right the firs ttime01:14
*** otherwiseguy has quit IRC01:14
tchaypoyou won't.01:14
tchaypothat's why there are comments on the merge request01:14
mordred:)01:15
*** arnaud has quit IRC01:18
*** arnaud has joined #openstack-infra01:18
mordredtchaypo: https://bitbucket.org/pypa/setuptools/pull-request/63/include-the-script-template-files-fixes/diff01:19
mordreddstufft: ^^ fwiw01:19
clarkbthe problem with the github model in bitbucket is you can't rebase with hg01:22
clarkbso its basically useless01:22
clarkba patch based workflow more like ours would be so much better01:23
tchaypobut that's irrelevant since the pypa repos all use git rather than hg01:23
*** arnaud has quit IRC01:23
clarkboh01:23
clarkbmy interaction has been with tox and other pythony things which are all hg01:23
dstufftmordred: lol submitting patches via bitbucket is terrible01:23
mordredtchaypo: yeah. they use hg to best of my knowledge01:23
mordreddstufft: oh - what's the better way?01:23
dstufftI don't think there is one for setuptools01:24
clarkbI still can't figure out the proper way to have a bitbucket fork where I maintain my own changes that may not be merged upstream without using branches01:24
dstufftbut bicket is terrible01:24
dstufftbitbucket01:24
clarkbbecause apparently you shouldn't use hg branches01:24
dstufftyou can use branches01:24
clarkbright but people don't likle them01:24
tchaypoalso, rebase has been a core hg module for ages now... it's just not turned on by default because it Goes Against Everything We Stand For and whatnot01:24
clarkbbecause they clutter history01:24
clarkbor something01:24
dstufftthere is also hg bookmarks01:24
dstufftwhich are like git branches01:24
dstufftexcept bitbucket doesn't properly support them01:24
dstufftbecause reasons01:24
mordredwell, see here - I run a project that requires people to submit patches to gerrit01:25
mordredso I'm not going to stand on principles01:25
mordredwhat I want is an actual clear explanation of the 3 steps I should do01:25
mordredthat don't involved 20 hg plugins that I may or may not have01:25
clarkbmordred: I don't understand what gerrit has to do with it. the ui might be bad but the worflow is easy to grojk01:25
mordredclarkb: what I'm saying is, I don't want to argue about the "right" way to do git or hg01:25
dstufftexcept for the whole, multiple patches == multiple review thing01:25
mordredthere are a billion opinions01:26
dstufftthat confused me for awhile01:26
dstufft:|01:26
* tchaypo giggles quietly01:26
clarkbdstufft: you have been polluted by githubs terrible way of doing it :P01:26
mordredI have a strong one on git - other people have other opinions01:26
mordredmy biggest grief with hg is that no one will ever tell me how to do simple things01:26
clarkbanyways I have derailed the conversation01:26
clarkbmordred: I agree01:26
* mordred throws wet cat at clarkb01:26
dstufftclarkb: no ;P gerrit should just do a squash merge unless you tell it you want it to not do that01:26
dstufftsame outcome, without the footgun01:26
clarkbmordred: and usually when they do tell you stuff it involves installing and configuring a half dozen plugins01:26
clarkbwhy is paging output not default?01:27
mordredwe could have the argument for ages01:27
clarkbwell there is a plugin or config for that01:27
tchaypoI'd have to take the hg side of the argument01:27
mordredcan we get back to ... how should I submit a patch to setuptools? did I do the right thing? will anyone see that?01:27
dstufftmordred: basically I don't think anyone actually knows the best way to do things with hg01:27
mordredbecause ALL OF PYTHON is currently broken01:27
tchayponot becuase i like it, but just to make the argument more interesting01:27
dstufftbecause hg is crazy01:27
mordreddstufft: at least it's not me01:27
tchayposo getting back to the original question01:28
dstufftmordred: that looks like a reasonableish thing01:28
mordred(seriously, python setup.py develop is broken for the world currently - it's not a pbr thing)01:28
mordreddstufft: awesome! thanks!01:28
tchaypodstufft: was filing a merge request on bitbucket the right thing to do?01:28
dstufftI know that jaraco pays attention to bitbucket01:28
mordredwoot01:28
mordred(he's not on IRC right now or I would  have bugged him)01:28
dstufftI love how the tests for everything are so bad01:28
dstufftit's not like this is critical infra or anything01:29
mordred:)01:29
mordreddstufft: if I ever get off my ass and implement the "hey man, that patch will break openstack" - it won't be perfect, but it'll hit a bunch of usage01:29
mordredof course, it'll make more people hate us01:29
dstufftI think my favorite was when I deleted something in pip, I can't recall what, but it was fairly complex and snaked throughout most of pip01:29
dstufftand after I deleted it01:29
dstufftone test failed01:29
dstufftjust one01:30
dstufft;(01:30
mordredhahaha01:30
mordredjeez01:30
dstufftit doesn't help that we support way too many ways of doing things01:30
dstufftin general01:30
mordred++01:30
mordredit's funny01:30
mordredbecause that was originally the difference between perl and python01:30
anteayabut apparently people like options01:30
anteayathat's what I heard anyway01:30
dstufftin the future there will be one path a package takes, and the only difference will be at what stage you get onto that path01:31
*** nati_ueno has joined #openstack-infra01:31
mordredexcept now I think that perl has more consistency01:31
dstufftat least that's my goal01:31
mordredanteaya: options are great. crazy is not01:31
dstufftoptions are kind of bad too01:31
dstufftI mean sometimes you need options01:31
mordredyeah. but crazy is worse than options01:31
dstufftbut the less you have of them the better01:31
mordredthis is amongst the reason that OpenStack has ONE way to contribute01:32
dstufftin my old age (of 26!) I'm becoming increasingly hostile to configuration01:32
mordredand if you'd prefer to do it differently, that's cool, except no01:32
anteayaokay, yeah, I can go with less crazy01:32
clarkbon that note I am seriously considering putting in an "OpenStack forms an opinion" summit session for paris01:32
mordredclarkb: +100000000001:32
clarkband everyone will be anfry01:32
clarkbbut seriously guys can we stop like "supporting" everything01:32
clarkbfrom the vendor side with third party tests whatever01:33
dstufftlol01:33
clarkbbut you don't need 5 different DBs and 5 different message queues01:33
mordredclarkb: you know I've been arguing to kill pg for forever01:33
anteayaoh the vendor side is always anfry with openstack01:33
clarkbwell I don't care if the answer is pg or mysql or mongodb01:33
anteayathey have to use not-windows01:33
dstufftsupporting everything is the worst idea ever and it's so hard to stop doing it once you do01:33
clarkbI just want there to be one01:33
dstufftbecause then you're breaking someones use case01:33
dstufftand people cry01:33
clarkbdstufft: well I am going to put both feet on the brake pedal if I can01:33
mordredclarkb: I know. I actually don't care either - but 95% of everyone uses mysql, so we should just go with that. supporting the one crazy holdout in the corner is not worth it01:34
mordredwe dont' support gentoo either, btw01:34
clarkbI think there is a major different between the infra black box peice and the end customer things01:34
dstufftthat's like with packaging right, I'm like "yea we have 4 different ways of doing this, all slightly different, let's just pick one and kill the others", and some random person will come up and be like this super edge case will break if you don't do that!!!01:34
mordred(and by everyone, I mean openstack deploys)01:34
clarkbyour customer does not care if you use mysql or pg01:34
clarkbthey do not care if you use qpid or rabbit01:34
clarkbor kafka or 0mq01:34
dstufftdon't use mongodb tho01:34
mordredwell no01:34
mordredunless you want coffee mugs01:34
dstufftI love coffee mugs01:35
mordredin which case, mongodb is awesome01:35
mordredI love all of my mongodb coffee mugs01:35
dstufftI'm pretty sure mongodb should just pivot as a coffee mug company01:35
mordredyup01:35
*** arnaud has joined #openstack-infra01:35
clarkbanyways I am ranting on a weekend :)01:35
dstufftonly on weekends?01:35
mordredclarkb: there was somethign I wanted to tell you ... but then you went for food01:35
clarkbdstufft: only. I am too busy during the week01:35
mordredwhat was it ...01:35
lifelessbackish01:36
lifelesslunch cooking etc but passing by more regularly than when swimming01:36
anteayayeah, irc while swimming is tough01:37
mordredclarkb: oh, I know what it was01:37
lifelessmordred: sure, + is fine.01:37
mordredclarkb: it was while you and tchaypo were talking about jjb01:37
dstufftlifeless: ok it's going to be +01:37
dstufftwe decided while you were swimming01:37
dstufft:D01:37
mordredclarkb: I was chatting way back in the scrollback with fungi about the remarkable similarity between jjb macros and ansible playbooks01:38
dstufftoh hey, I forgot about wolfenstein, time to go kill some nazi's01:38
clarkbdstufft: have fun!01:38
clarkbmordred: I saw that in passing but was only half awake01:38
mordredclarkb: compare: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/jenkins_job_builder/config/macros.yaml#n28801:39
mordredand https://review.openstack.org/#/c/100066/2/modules/openstack_project/files/ansible/run_mirror.yaml01:39
anteayaSukhdev: thanks for posting to the ml01:39
mordredclarkb: very simple example01:39
mordredit's different - but close enough that it makes me want to brainstorm about convergence01:39
clarkbmordred: wow they are very similar01:39
mordredright?01:39
mordredand if you think about it - they should be01:39
mordredgiven that they're each ways to encode a sequence of actions01:40
clarkbright we basically have an IR for instructing machines01:40
mordredone of them just gets intepreted out into some xml01:40
Sukhdevanteaya: I thought other CI maintainers might be investigating the issue as well - so avoid some frustration for them01:40
mordredand one turns into remote ssh calls01:40
mordredclarkb: and we've been talking about having turbo-hipster do direct local exec ... which ansible can already do01:40
mordredso I have no answers01:40
anteayaSukhdev: thanks for doing that, I really appreciate it01:40
mordredbut there may be a really nead opportunity to collaborate somewhere in here01:40
clarkbmordred: ++01:41
Sukhdevanteaya: NP01:41
clarkbmordred: if nothing else maybe we use their yaml parser01:41
mordredyah01:41
clarkbmordred: then you can pass around the files01:41
Sukhdevanteaya; I am hoping mordred will nail it for us01:41
mordredit would be super cool if there could be a yaml snippet that one could pass to jjb to get a jenkins job, to ansible to get remote ex and to turbo-hipster to get locally run on the other side of a gearman01:41
mordredSukhdev: uhoh. what am I doing?01:42
anteayaSukhdev: he is good at figuring out many of these sorts of issues01:42
anteayamordred: http://lists.openstack.org/pipermail/openstack-dev/2014-June/037713.html01:42
Sukhdevmordred: good to the community :-)01:42
*** yamahata has joined #openstack-infra01:42
mordredSukhdev: oh gosh. we're in trouble01:42
mordred;)01:43
anteayamordred: Sukhdev sent out an email to the -dev list for the third party ci folks01:43
anteayagiving them a heads up01:43
mordredI just submitted a patch to setuptools upstream01:43
anteayawhich I really appreciate01:43
openstackgerritJames Polley proposed a change to openstack-infra/config: WIP: Add definitions for a job to check vlan config  https://review.openstack.org/10007601:43
mordredlet me see if the two are related01:43
anteayamordred: yay01:43
tchayposee, this is way face-to-face meetups are awesome01:43
*** salv-orlando has quit IRC01:43
Sukhdevmordred: how can I pull your fix into my local system to test it out?01:44
tchaypowithout having met anteaya in person I wouldn't have the mental image of her cheering every time she says "yay"01:44
anteayayay!!01:44
anteaya:D01:44
Sukhdevtchaypo: +101:44
tchaypoMy mental image doesn01:44
tchaypot extend to anteaya saying "yay!!" though01:44
anteayajust yay01:44
tchaypothat implies more happiness than just "yay" but I can't imagine what that would look like01:44
mordredSukhdev: I _think_ you want to: hg clone https://bitbucket.org/mordred/fix-missing-files setuptools01:44
anteayawe have to work on that then01:45
mordredSukhdev: then "cd setuptools ; pip install ."01:45
tchaypoanteaya: at pycon-au, hopefully01:45
anteayaSukhdev: have you worked with mercurial before?01:45
Sukhdevanteaya: no01:45
anteayaSukhdev: hg is the mercurial command01:45
mordredSukhdev: but if devstack decides to re-install setuptools for us, we're still screwed01:45
tchaypoHg is the chemical symbol for mercury01:45
mordredSukhdev: don't worry - I don't know anything mercurial myself01:45
anteayait is a version control system but it isn't git01:45
tchaypoit's the lovechild of git and svn - it uses commands similar to SVN but achieves the results through similar means to git01:46
anteayayou have to install mercurial for that command to work, then just execute the command mordred gave you01:46
anteayaSukhdev: you _don't_ have to read the hg manual first01:46
anteayapoor Sukhdev, he had agreed to chair a meeting and thought he had to install meetbot somewhere01:47
anteayawe talked at the summit and I showed him the commands he could use the chair the meeting01:47
Sukhdevanteaya: Yes, you were a life savor there - I was totally lost01:47
tchaypoI can understand why, when I read the meetbot docs I had a similar impression01:47
anteayaokay so don't get upset about hg01:47
anteayajust install it and run the command mordred posted01:47
anteayatchaypo: and yes, hopefully at pycon-au01:48
anteayaI submitting a talk "How to Read the Logs"01:48
anteayawe'll see if it gets accepted01:48
Sukhdevanteaya mordred: let me try to install this beast and will report back in a few01:48
anteayaSukhdev: okay, don't get bogged down, ask for help if you need it01:48
Sukhdevanteaya: will do - thanks for the offer01:49
anteayakk01:49
anteayatchaypo: and yes there is a patch up for infra-manual repo creation01:50
anteayaand hopefully stuff like use the irc commands to run a meeting in an openstack-meeting channel find their way in there01:50
anteayawe need to get the inital commint in though01:51
anteayasorry the repo is created, the first commit is in review01:51
*** nati_ueno has quit IRC01:51
anteayaI am getting prompted to use openid to sign into gerrit: https://review.openstack.org/login/q/status:open,n,z01:52
anteayaI don't know as I have an openid01:52
tchaypolaunchpad01:52
anteayaand didn't know we had changed anything01:52
anteayayeah but I used to get a launchpad failure01:52
anteayathis is really confusing01:52
tchaypoheh01:52
tchaypoI'm getting the openid page there too01:52
anteayaespecially since we are moving to openid but we aren't there yet01:53
anteayadevs that get this will be really confused01:53
tchaypoWhat happens if you go to https://review.openstack.org/#/q/status:open,n,z ?01:54
tchaypo(that url comes from me being already logged in and searching for status:open)01:54
anteayathe page renders and I am not signed in01:54
anteayahere is the initial commit for the infra-manual repo: https://review.openstack.org/#/c/92475/01:54
anteayatchaypo: do review01:54
fungiSukhdev: just for the record, this happens to the devstack ci systems on average once every week or two (some new release of something we depend on is broken or not backward-compatible and we all end up scrambling for hours). i'd be surprised if it didn't happen to every one else's ci systems too01:55
anteayafungi: how goes the packing?01:56
Sukhdevfungi: It is failing many CI's01:56
fungianteaya: well enough. this weekend has mostly been spackling and sanding and washing walls01:56
clarkbfungi: at the new place?01:57
anteayaSukhdev: yeah, I think fungi's point is that keeping a close ear to the infra channel, like you are doing, is pretty much what is required for third party ci to be maintained01:58
anteayaSukhdev: so well done01:58
Sukhdevanteaya, mordred: OK - I have installed everything - getting ready to kick off the test01:58
anteayaSukhdev: great01:58
fungiclarkb: no, movers are showing up monday to load, and then i'm going to meet them at the new place on tuesday while christine stays behind to corral the move-out cleaners01:58
anteayafungi: you have to spackle and sand the old place?01:58
clarkbare you selling?01:59
clarkbI guess if you are selling that would make sense01:59
Alex_Gaynordstufft, mordred: my phone buzzed because of you two, anyone need me for anything?01:59
fungiclarkb: no, just making sure it's in good enough shape the owners only need to put a coat of paint on the walls01:59
anteayaAlex_Gaynor: what is the phone buzz keyword?01:59
Alex_Gaynoranteaya: my name :P01:59
mordredanteaya: WOAH02:00
mordredclarkb: WOAH02:00
mordredAlex_Gaynor: just looking at your python-swiftclient bug - but nothing urgent02:00
tchaypospeaking of openid02:00
anteayaAlex_Gaynor: ah, well if ever I need to ensure you know we are thinking about you02:00
clarkbmordred: woah?02:00
mordredclarkb: why is gerrit showing us an openid login page?02:00
tchaypoI try to log into the wiki and end up at https://wiki.openstack.org/w/index.php?title=Special:OpenIDLogin&returnto=TripleO02:00
clarkbmordred: aroo?02:00
Alex_Gaynormordred: okey doke02:00
mordredclarkb: go to gerrit. log out.02:00
mordredthen log back in02:00
mordredthen cry02:00
mordredW T F?02:00
* tchaypo repeats - not just gerrit02:00
tchaypohttps://wiki.openstack.org/w/index.php?title=Special:OpenIDLogin&returnto=TripleO02:00
mordredoh.02:01
* mordred didnt' see that02:01
mordredso - possibly launchpad openid provider is unhappy?02:01
clarkbI am guessing that launchpad is derpy02:01
tchaypo"An error occurred during verification of the OpenID URL. "02:01
clarkbwhich is what tchaypo said02:01
tchaypothis is why I repeated it :)02:01
anteayayeah, I can't log into anything02:01
clarkbmordred: I think this is the cue to drink02:01
fungiyeah, we've seen it fall over before02:01
mordredso, there was a time in the past when we were talking about switching the url to the ubuntu ssh instead of the launchpad one02:01
mordredsso02:01
Sukhdevanteaya mordred: While I am kicking off the test, can you verify that my installation was correct - http://paste.openstack.org/show/84073/02:01
*** arnaud has quit IRC02:01
tchaypoI've been drinking for the last hour. I think this is my cue to remember that it's sunday and put down the work laptop02:02
mordredSukhdev: yah. looks good02:02
clarkbtchaypo: yes I have been watching world cup with beer02:02
clarkbalso picked up a burger02:02
Sukhdevmordred: cool - thanks - will report shortly if this took care of the issue02:02
clarkband didn't wake up until almost noon. Its been a good day02:02
mordredclarkb: btw - japan is up!02:02
anteayatchaypo: it's sunday, put down the work laptop02:03
clarkbyup it went something like this "GOOOOOOOOO HONDA HONDA HONDA OOOOOOOOOOOOL!"02:03
anteayaha ha ha02:03
mordrednice02:03
clarkbI love univision02:04
tchaypobye all!02:04
anteayayou were saying in scrollback02:04
anteayabye tchaypo02:04
anteayahappy sunday02:04
clarkb"Inglaterra" it actually get me thinking about why we as humanity have a hard time calling people by the names they chose for themselves02:04
anteayawhat is inglaterra?02:04
clarkbanteaya: england02:05
mordredclarkb: it's a good question02:05
anteayaI must have missed something02:05
anteayaI'm not watching the sportsball02:05
clarkbanteaya: I am watching world cup on a spanish broadcast channel02:05
mordredI especially love that we feel it's important to rename Brasil as Brazil02:05
anteayaperhaps I need to be to connect the dots02:05
anteayaah02:05
clarkbmordred: right and its not just univision or espn doing it02:05
mordrednope. it's everybody02:05
clarkbmordred: everyone does it02:05
clarkbits weird02:05
lifelessclarkb: tchaypo: mordred: the LP SSO is the Ubuntu SSO.02:07
lifelessLaunchpad isn't being derpy because launchpad isn't involved in your SSO at all, and never has been since before OpenStack started.02:08
mordredlifeless: someone told me that by using the launchpad url we only got a subset of the available servers02:08
mordredalso - launchpad.net url is derping02:09
anteayaconfirmed02:09
mordredas is https://login.launchpad.net/+openid02:09
mordredas is login.ubuntu.com02:10
lifelessmordred: so that may be a Canonical DC outage, will poke in a sec02:11
*** david-lyle has joined #openstack-infra02:11
lifelessmordred: login.launchpad.net is the same server HA(ish) cluster as login.ubuntu.com02:11
lifelessmordred: bugs.launchpad.net etc are entirely separate infrastructure02:12
*** david-lyle has quit IRC02:12
anteayabugs.l.n is down for me as well02:12
*** david-lyle has joined #openstack-infra02:12
lifelessso web frontend is up02:12
lifelessI could speculate about cause, but the thing to do is check canonical-sysadmins02:13
anteayayeah, I get a message, please try again02:13
mordredlifeless: nod. well, it'll be up when it's up02:13
lifelesswhich says02:13
lifeless'Known issues: UK DC offline, staff en route '02:13
anteayais canonical-sysadmins a url?02:14
Sukhdevmordred: fix is looking good. It passed one of my local simple test. Now I am firing away some Gerritt events (real life test)02:15
mordredawesome02:15
anteayaSukhdev: well done02:15
mordredSukhdev: hopefully that patch will be merged soon and a new release of setuptools will be cut02:15
lifelessanteaya: #canonical-sysadmins - IRC channel for contacting them02:16
Sukhdevmordred: this will save lots of people tons of grief02:16
anteayaSukhdev: after you finish your tests you can post to your thread with the link to mordred's patch so others can follow the progress02:16
anteayalifeless: ah02:16
Sukhdevanteaya: I absolutely intend to do it…..help others save some grief….:-)02:17
*** david-lyle has quit IRC02:17
anteayaSukhdev: awesome, thank you02:17
anteayaand yes it will02:17
anteayaat the very least on Monday we can link confused third party ci folks to it so they can read it02:17
anteayaif this is still an issue, or if they want the history02:17
mordredanteaya: I doubt it will still be an issue on monday - the setuptools guy is very responsive02:18
anteayagreat02:18
anteayasaves us a pile-on02:18
*** yaguang has joined #openstack-infra02:19
*** harlowja_away has quit IRC02:20
mordredclarkb: woah. who thought THAT would happen02:23
lifelessmordred: que?02:27
lifelessmordred: clarkb: care to set a downtime alert about the SSO thing ?02:27
lifelessfungi: ^02:28
clarkb#status alert launchpad openid is down. login to openstack services will fail until launchpad openid is happy again02:29
openstackstatusclarkb: sending alert02:29
clarkbhows that?02:29
lifelessthanks02:29
lifelesswhats the acl on that?02:29
clarkbeventually it should hit all the channels02:29
clarkbits a list of users in a yaml file iirc02:30
anteayalifeless: something happened in the sportsball02:30
clarkbmust be identified with srvices02:30
anteayaIvory Coast scored02:30
-openstackstatus- NOTICE: launchpad openid is down. login to openstack services will fail until launchpad openid is happy again02:30
lifelessclarkb: how do I turn it off, in case you're asleep when it fixes?02:30
*** ChanServ changes topic to "launchpad openid is down. login to openstack services will fail until launchpad openid is happy again"02:30
clarkblifeless it is a '#status ok' but someone in the acl list will have to do it02:31
clarkbinfra core and ttx reed and a few others iirc02:31
tchaypoCan we review a change to the acl pronto?02:32
lifelessclarkb: I'm identified with services02:32
lifelessclarkb: which is what you said the acl was :)02:32
clarkbno its a list of users02:32
clarkbthey must be identified02:32
lifelessoh, I missed that line02:32
tchaypoI need to update the acl list controlling access to the NAB bank ATM machine02:32
lifelesssilly IRC highlight distracting eyes02:32
clarkbI note that because not being identified has confused folks before02:32
openstackstatusclarkb: finished sending alert02:32
lifelessclarkb: ok so I'd be happy to be a responsible person in my timezonish, should that be desired02:33
mordredclarkb: speaking of lifeless's timezone - did we add jhesketh to the list?02:33
clarkbmordred probably not :(02:34
clarkbwe should though02:34
fungiclarkb: lifeless: http://git.openstack.org/cgit/openstack-infra/config/tree/manifests/site.pp#n20602:35
fungino jhesketh there yet02:35
lifelessI hate vpns.02:36
fungiso nobody regularly in apac in that list yet02:36
mordredmaking a patch to add jhesketh02:36
fungiperfect02:36
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add jhesketh to the magic statusbot acl list  https://review.openstack.org/10008002:37
clarkbwoo 500 wall street is on fire according to police scanner02:38
clarkb(I am at 400)02:38
anteayaI can't sign in to +1 it02:38
anteayahow close is that to you?02:39
*** otherwiseguy has joined #openstack-infra02:39
fungii +2'd but am happy to approve if everyone else is unable to auth into gerrit02:39
openstackgerritMonty Taylor proposed a change to openstack-infra/config: Add lifeless to statusbot list  https://review.openstack.org/10008102:39
anteayacan you see smoke?02:39
Sukhdevmordred: can you please look at http://paste.openstack.org/show/84074/02:39
clarkbanteaya: one block02:39
anteayafungi: I like the patch as is, +1 from me02:39
anteayaclarkb: :(02:39
clarkbI don't see anything but lots of sirens and very fast moving vehicles02:39
mordredfungi: I'm not logged in - I'm happy for you to +A both02:40
anteayaclarkb: got a bag packed?02:40
Sukhdevmordred: while the original issue seems to be resolved, but, the tests are still failing because of this new dependency02:40
clarkbwoo "No sign of smoke or fire"02:40
clarkbso who knows what happened but lots of commotion02:40
anteayaclarkb: so a false alarm?02:41
clarkbanteaya: I guess? http://www2.seattle.gov/fire/realtime911/getRecsForDatePub.asp?action=Today&incDate=&rad1=des still shows it as active02:41
*** david-lyle has joined #openstack-infra02:42
anteayawell if you heard "no sign of smoke or fire" on the scanner that is a good start02:42
mordredSukhdev: this I'm not sure about02:42
anteayathey probably have to check the whole place top to bottom before they can change the alarm status02:43
clarkbya02:43
Sukhdevmordred: any hints/clues/guesses?02:43
clarkbanteaya: sounds like an open valve on a suppresion system may have triggered everything going off and the crazy response02:45
anteayaclarkb: yay, that means you are safe02:45
clarkbI am going to return to me evening of television. Will try to check launchpad and correct status if it is happy02:45
anteayaenjoy02:45
*** david-lyle has quit IRC02:47
openstackgerritA change was merged to openstack-infra/config: Add jhesketh to the magic statusbot acl list  https://review.openstack.org/10008002:50
openstackgerritA change was merged to openstack-infra/config: Add lifeless to statusbot list  https://review.openstack.org/10008102:51
*** sarob_ has joined #openstack-infra02:51
anteayaso I think I am done for the day02:51
*** cp16net has left #openstack-infra02:54
*** arnaud has joined #openstack-infra02:55
*** sarob_ has quit IRC02:55
Sukhdevcan anybody shed any light on this failure - 'amqplib' is not in global-requirements.txt02:55
fungijhesketh: lifeless: if you want to clear the current alert once launchpad sso returns to operation, assuming openstackstatus is still in channel, just say in the channel '#status ok' and it should update the wiki and put all the topics back to normal for you02:56
lifelessfungi: ack thanks02:58
lifelessI shall check in every hour or so02:58
*** arnaud has quit IRC03:03
*** openstackstatus has quit IRC03:05
*** openstackstatus has joined #openstack-infra03:06
*** ChanServ sets mode: +v openstackstatus03:06
*** Sukhdev has quit IRC03:11
*** Sukhdev has joined #openstack-infra03:13
Sukhdevfolks, I am stuck with this error - http://paste.openstack.org/show/84076/ - can anybody shed any light on this one?03:14
*** dims has quit IRC03:16
*** CaptTofu_ has joined #openstack-infra03:18
ianwls03:19
mordredSukhdev: did you update all of the repos on your test box?03:23
mordredSukhdev: a change recently went in to global-requirements which removed amqplib from the requirements03:23
mordredSukhdev: which was merged into neutron on 737fe322309364554df9ffea092183f783f4108403:24
mordredand global-requirements took the change june 403:24
mordredso i'd suggest that perhaps you have iether a stale copy of neutron, or a stale copy of global requirements03:25
Sukhdevmordred: I do sudo rm -rf /opt/stack and sudo rm -rf devstack and then pull fresh devstack and run stack.sh03:25
Sukhdevmordred: what else can I do to get rid of old stuff?03:25
mordredI'm not sure - this is now outside of the area where I'm helpful :(03:26
Sukhdevmordred: understood03:26
Sukhdevmordred: can you provide the link to the fix that you mentioned which modified global-requirements.txt03:27
*** arnaud has joined #openstack-infra03:28
*** arnaud has quit IRC03:29
fungiSukhdev: https://review.openstack.org/#/c/92149/03:35
*** Ryan_Lane has joined #openstack-infra03:35
Sukhdevfungi: thanks03:36
Sukhdevfungi: it is odd - this fix merged while back - but, the build is breaking as of this afternoon - I wonder what else changed which caused this?03:36
fungiSukhdev: what change are you testing which exhibits this error?03:37
Sukhdevfungi: how can I clean up everything to get rid of all the dependencies03:37
fungiSukhdev: i don't know what you're asking03:37
fungiSukhdev: at least for official ci, we use a fresh nova instance for every test, then delete it after03:38
*** Ryan_Lane has quit IRC03:38
fungier, for every job i mean03:39
Sukhdevfungi: I do not delete the vm, I clean it up - by ./clean.sh, ./unstack.sh, rm -rf /opt/stack, rm -rf devstack - and then pull fresh devstack and run ./stack.sh03:39
fungiSukhdev: the error message you're seeing suggests you're either testing neutron from more than a month ago or testing a change which reintroduces aqmplib to neutro03:40
fungin03:40
Sukhdevfungi: that is a good clue03:40
fungiSukhdev: anyway, almost midnight here, so i'm going to get some sleep. good luck on your bug hunt03:41
Sukhdevfungi: oh sorry to bug you so late in night03:41
Sukhdevfungi: have a good night03:41
fungithanks! hope the rest of your weekend goes better03:42
*** dims has joined #openstack-infra03:42
*** david-lyle has joined #openstack-infra03:43
*** david-lyle has quit IRC03:47
*** dims has quit IRC03:47
*** sarob_ has joined #openstack-infra03:51
*** sarob_ has quit IRC03:56
*** CaptTofu_ has quit IRC03:58
*** yaguang has quit IRC04:05
*** otherwiseguy has quit IRC04:17
*** yfried_ has quit IRC04:18
*** Sukhdev has quit IRC04:19
*** Sukhdev has joined #openstack-infra04:28
*** otherwiseguy has joined #openstack-infra04:31
*** Ryan_Lane has joined #openstack-infra04:35
*** david-lyle has joined #openstack-infra04:43
*** dims has joined #openstack-infra04:46
*** david-lyle has quit IRC04:48
*** sarob_ has joined #openstack-infra04:49
*** dims has quit IRC04:51
*** sarob__ has joined #openstack-infra04:51
*** sarob_ has quit IRC04:54
*** sarob__ has quit IRC04:56
*** HenryG has quit IRC04:56
*** HenryG has joined #openstack-infra04:58
*** nati_ueno has joined #openstack-infra04:59
*** nati_ueno has quit IRC04:59
*** basha has joined #openstack-infra05:08
*** yfried_ has joined #openstack-infra05:11
*** yaguang has joined #openstack-infra05:12
*** basha has quit IRC05:20
*** basha has joined #openstack-infra05:21
*** otherwiseguy has quit IRC05:24
*** praneshp has joined #openstack-infra05:42
*** david-lyle has joined #openstack-infra05:44
*** david-lyle has quit IRC05:49
*** _nadya_ has joined #openstack-infra05:49
*** praneshp_ has joined #openstack-infra05:50
*** _nadya_ has quit IRC05:51
*** praneshp has quit IRC05:52
*** praneshp_ is now known as praneshp05:52
*** Ryan_Lane has quit IRC05:54
*** basha has quit IRC05:59
*** Longgeek has joined #openstack-infra06:07
*** ihrachyshka has joined #openstack-infra06:07
*** basha has joined #openstack-infra06:15
*** basha has quit IRC06:19
*** srenatus has quit IRC06:23
*** ihrachyshka has quit IRC06:23
*** Sukhdev has quit IRC06:28
*** praneshp_ has joined #openstack-infra06:33
*** praneshp has quit IRC06:36
*** praneshp_ is now known as praneshp06:36
*** david-lyle has joined #openstack-infra06:45
*** david-lyle has quit IRC06:46
*** david-lyle has joined #openstack-infra06:47
*** sarob_ has joined #openstack-infra06:51
*** david-lyle has quit IRC06:51
*** yaguang has quit IRC06:54
*** sarob_ has quit IRC06:56
*** pcrews has quit IRC07:10
*** dosaboy has quit IRC07:21
lifeless#status ok07:23
openstackstatuslifeless: sending ok07:23
openstackstatuslifeless: finished sending ok07:25
*** nati_ueno has joined #openstack-infra07:29
*** alexpilotti has quit IRC07:39
*** alexpilotti has joined #openstack-infra07:40
*** alexpilotti has quit IRC07:45
*** david-lyle has joined #openstack-infra07:48
*** nati_ueno has quit IRC07:50
*** Ryan_Lane has joined #openstack-infra07:51
*** david-lyle has quit IRC07:52
*** e0ne has joined #openstack-infra07:53
*** e0ne has quit IRC07:55
*** e0ne has joined #openstack-infra07:55
*** e0ne has quit IRC07:57
*** Ryan_Lane1 has joined #openstack-infra08:03
*** Ryan_Lane1 has quit IRC08:04
*** andreykurilin_ has joined #openstack-infra08:04
*** alexpilotti has joined #openstack-infra08:06
*** penguinRaider has joined #openstack-infra08:08
*** andreykurilin_ has quit IRC08:15
*** andreykurilin_ has joined #openstack-infra08:17
*** e0ne has joined #openstack-infra08:17
*** Ryan_Lane has quit IRC08:18
*** koolhead17 has joined #openstack-infra08:23
*** alexpilotti has quit IRC08:26
*** e0ne has quit IRC08:26
*** e0ne has joined #openstack-infra08:27
*** e0ne has quit IRC08:36
*** e0ne has joined #openstack-infra08:37
*** roeyc has joined #openstack-infra08:38
*** david-lyle has joined #openstack-infra08:48
*** sarob_ has joined #openstack-infra08:51
*** david-lyle has quit IRC08:53
*** sarob_ has quit IRC08:56
*** Ryan_Lane has joined #openstack-infra09:00
*** Ryan_Lane has joined #openstack-infra09:02
*** Ryan_Lane has quit IRC09:03
*** Ryan_Lane has joined #openstack-infra09:03
*** Ryan_Lane has quit IRC09:05
*** Ryan_Lane has joined #openstack-infra09:05
*** jamespage_ has quit IRC09:08
*** Ryan_Lane has quit IRC09:09
*** e0ne has quit IRC09:14
*** e0ne has joined #openstack-infra09:15
*** maishsk has joined #openstack-infra09:21
*** zzelle has joined #openstack-infra09:24
*** srenatus has joined #openstack-infra09:33
*** koolhead17 has quit IRC09:34
*** maishsk has left #openstack-infra09:37
*** e0ne has quit IRC09:40
*** e0ne has joined #openstack-infra09:41
*** cnesa has joined #openstack-infra09:43
*** david-lyle has joined #openstack-infra09:49
*** sarob_ has joined #openstack-infra09:51
*** roeyc has quit IRC09:54
*** david-lyle has quit IRC09:54
*** sarob_ has quit IRC09:55
*** afazekas has joined #openstack-infra09:57
afazekaslooks like it is a new gate breaker pypi issue http://logs.openstack.org/49/87249/10/check/check-tempest-dsvm-full/c67435f/logs/devstacklog.txt.gz#_2014-06-15_09_34_11_38709:59
*** KiallM has joined #openstack-infra09:59
*** dosaboy has joined #openstack-infra10:02
boris-42afazekas ^ there is bug for that10:02
boris-42afazekas https://bugs.launchpad.net/devstack/+bug/133015110:02
uvirtbotLaunchpad bug 1330151 in devstack "swiftclient install: (dev).tmpl: No such file or directory" [Undecided,New]10:02
*** Ryan_Lane has joined #openstack-infra10:06
*** Ryan_Lane has quit IRC10:10
*** KiallM2 has joined #openstack-infra10:13
*** praneshp has quit IRC10:15
*** KiallM has quit IRC10:17
*** _nadya_ has joined #openstack-infra10:17
*** KiallM2 has quit IRC10:18
*** _nadya_ has quit IRC10:30
*** Longgeek has quit IRC10:32
*** Longgeek has joined #openstack-infra10:33
*** e0ne has quit IRC10:38
*** Longgeek has quit IRC10:48
*** david-lyle has joined #openstack-infra10:50
*** sarob_ has joined #openstack-infra10:51
*** Longgeek has joined #openstack-infra10:52
*** david-lyle has quit IRC10:55
*** sarob_ has quit IRC10:56
*** salv-orlando has joined #openstack-infra11:00
*** Ryan_Lane has joined #openstack-infra11:05
*** cnesa2 has joined #openstack-infra11:05
*** cnesa has quit IRC11:06
*** flaper87|afk is now known as flaper8711:09
*** Ryan_Lane has quit IRC11:09
*** e0ne has joined #openstack-infra11:17
*** e0ne has quit IRC11:19
*** andreykurilin_ has quit IRC11:20
*** rlandy has joined #openstack-infra11:21
*** KiallM2 has joined #openstack-infra11:23
fungiboris-42: afazekas: mordred put up a pull request to fix that in setuptools yesterday... i'll link the upstream activity for it in that bug11:23
boris-42fungi thanks11:24
fungiadded11:26
boris-42fungi bbl11:26
*** yaguang has joined #openstack-infra11:30
*** roeyc has joined #openstack-infra11:42
*** KiallM2 has quit IRC11:43
yfried_fungi: do you know what's up with the gate? tests fail before installation even start https://jenkins04.openstack.org/job/check-tempest-dsvm-full/12987/console11:45
*** sarob_ has joined #openstack-infra11:51
*** sarob_ has quit IRC11:56
*** roeyc has quit IRC12:01
*** roeyc has joined #openstack-infra12:02
fungiyfried_: yep, bug 133014012:06
uvirtbotLaunchpad bug 1330140 in python-swiftclient "stack.sh fails because of python-swiftclient depenendencies" [Undecided,New] https://launchpad.net/bugs/133014012:06
fungiyfried_: latest setuptools is broken, and we're waiting for the setuptools maintainer to wake up and patch/release another12:07
fungilooks like he merged mordred's request for that bug about 3 minutes ago12:08
yfried_fungi: tnx. is there an etherpad or something similar where I could see this without bugging you guys? some board of hot-critical errors that effect everyone and what's the status?12:08
fungiso i would expect a setuptools 5.0.2 at any time12:08
fungiyfried_: not really. this is a global python problem, not really an openstack infra problem12:09
yfried_fungi: well, it's an openstack problem. not an openstack solution though :)12:09
fungiyfried_: i would expect openstack's python developers, of whom there are a lot, to discuss these sorts of things on the openstack-dev mailing list12:09
yfried_fungi: as in - openstack gate is broken12:09
fungiyfried_: far from broken in fact, as it highlighted a problem with the latest setuptools which is now getting fixed12:10
fungiyfried_: i would say it is working extremely well in fact12:10
Kialland .. 5.0.2 is released...12:10
yfried_fungi: tnx for the info. I'll wait and then recheck12:10
fungikeep in mind that you not being able to merge code is a very minor problem on the scale up to "nobody can install openstack clients from pypi right now because setuptools is broken"12:11
*** yaguang has quit IRC12:11
fungiKiall: awesome! i'll trigger a mirror update asap12:12
yfried_fungi: I guess. but, being a selfish bastard, that's the only problem I care about :)12:12
fungiyfried_: yep. we're just collateral damage in all of this12:12
Kiallfungi: I wonder if you guys can convince the setuptools maintainer to release new versions under a different package name, say, 24 hours before the setuptools pkg itself is released12:13
Kiallthat would let us find a way to pre-test setuptools, and we seem to hit ALL THE BUGS, so they'd probably agree ;)12:13
fungiKiall: actually, we want to start running periodic jobs with trunk of upstream dependencies to spot these problems before they're released onto the rest of the unsuspecting python world12:14
KiallWell - that works too I suppose :)12:15
fungiwe're already acting as a defacto early warning system for low-level breakage throughout the python ecosystem, so i suppose we could just step that up a notch12:15
Kiallfungi: BTW - With designate incubated now, at what point do we move the repo from stackforge -> openstack? Whenever that happens, I'll be in for all sorts of fun with our internal build/ci/etc :(12:16
fungiKiall: yeah, you just put it on the agenda for the infra team weekly meeting and we discuss scheduling as a group and if necessary vote on a set of proposed maintenance windows for the gerrit outage12:18
KiallSounds good - I'll need to chat with mordred / clarkb about some matching internal changes too - so having a say in when it happens helps ;)12:19
fungiKiall: usually we also try to batch up other pending lower-priority project names and possibly any other work which we need to take down the ci systems to perform, so as to minimize disruption where possible12:22
fungis/names/renames/12:22
KiallYea - With Gerrit not supporting online renames that makes total sense12:23
fungianyway, i've got a manual mirror update going now, and will update the openstack-facing setuptools bug once 5.0.2 is in our mirror12:24
*** yaguang has joined #openstack-infra12:24
*** mriedem has joined #openstack-infra12:29
*** mriedem has quit IRC12:29
*** Alexei_987 has joined #openstack-infra12:34
*** davidhadas has quit IRC12:34
*** davidhadas_ has joined #openstack-infra12:36
*** sarob_ has joined #openstack-infra12:51
*** sarob_ has quit IRC12:56
*** yfried_ has quit IRC12:57
*** Ryan_Lane has joined #openstack-infra13:04
*** _nadya_ has joined #openstack-infra13:07
*** _nadya_ has quit IRC13:08
*** Ryan_Lane has quit IRC13:08
*** salv-orlando has quit IRC13:10
*** Longgeek has quit IRC13:12
*** Longgeek has joined #openstack-infra13:13
fungiwe had the new setuptools on our mirror at 12:36 utc, so should be all good now13:13
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: add query for setuptools 5.0.1 break  https://review.openstack.org/10008813:14
sdaguefungi: cool13:14
fungialso, rax-iad is accumulating nodes in a building state. some percentage of nodes nodepool builds there seem to just stay in that state indefinitely until they reach the 8 hour state timeout and get switched to delete. i cleared a bunch before i went to bed, and now there are 63 of them again13:16
fungithat's 63 which have been in a building state for between 1 and 10 hours13:17
openstackgerritA change was merged to openstack-infra/elastic-recheck: add query for setuptools 5.0.1 break  https://review.openstack.org/10008813:17
fungii guess if it's still going on tomorrow, someone should get phschwartz some instance uuids and see if he can determine what's breaking there13:18
sdagueyeh, I also feel that in general our network connectivity is far worse than it used to be. A lot of resets on getting to package mirrors and pypi.o.o13:18
fungistellar13:19
*** zhiyan_ is now known as zhiyan13:20
fungianyway, i'm clearing the stuck buiding nodes for now in case there are a lot of rechecks coming from work pileup during the setuptools, lp and more setuptools problem periods earlier13:20
fungisdague: 1330184 is a duplicate of 133014013:22
*** zhiyan is now known as zhiyan_13:22
sdagueok, I searched, and failed13:23
sdagueah, 140 had no actual reference to setuptools in it13:24
fungiin fact there was also another bug opened which got marked as a duplicate of that one as well13:25
sdaguefungi: when you update bugs like that in future, mind rewriting the title to something that's more descriptive of root cause. Otherwise it's tough to find for building ER queries13:25
fungisdague: yep, i'll try to remember13:25
sdagueI did a search for setuptools in openstack-ci before I filed my dup13:25
sdagueand the 2 hits were uber old13:25
fungialso i'll try to remember to add openstack-ci to things which someone might want to add an e-r query for, even if it's not an actual bug for us13:26
fungisince that one was only opened against devstack and swiftclient13:26
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: move bug to duplicate  https://review.openstack.org/10008913:28
sdaguefungi: ah, yeh13:28
fungiwhich is why a search in openstack-ci wouldn't have yielded that bug ;)13:29
sdague:)13:30
openstackgerritA change was merged to openstack-infra/elastic-recheck: move bug to duplicate  https://review.openstack.org/10008913:31
sdaguehmm... hp cloud is throwing us back a bunch of slow nodes as well - http://logstash.openstack.org/#eyJmaWVsZHMiOltdLCJzZWFyY2giOiJtZXNzYWdlOlwiV0FSTklORzogZGV2c3RhY2sgcnVuIHRvb2sgPiAxNSBtaW51dGVzXCIgQU5EIHRhZ3M6Y29uc29sZS5odG1sIiwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJvZmZzZXQiOjAsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sIm1vZGUiOiJzY29yZSIsImFuYWx5emVfZmllbGQiOiJidWlsZF9ub2RlIn0=13:31
sdagueok, waffles nearly up, I need to stop playing with computers :)13:32
*** sweston has joined #openstack-infra13:32
*** sweston has quit IRC13:36
*** pcrews has joined #openstack-infra13:37
*** sweston has joined #openstack-infra13:37
*** adalbas has joined #openstack-infra13:44
*** CaptTofu_ has joined #openstack-infra13:46
*** sarob_ has joined #openstack-infra13:51
*** Longgeek_ has joined #openstack-infra13:55
*** sarob_ has quit IRC13:55
*** Longgeek has quit IRC13:58
*** fabo has quit IRC14:03
*** pfallenop has quit IRC14:10
*** pfallenop has joined #openstack-infra14:15
anteayafungi or sdague can you get the irc 'ok' topics back again? lifeless did try to send #status ok which the bot took but the topic still needs to return to normal14:17
anteayanot sure what derped there14:17
*** yamahata has quit IRC14:31
*** roeyc has quit IRC14:35
mordredsdague: are any of the errors getting to pypi.o.o from teh DFW region?14:36
mordredsdague: (I'm asking because if intra-region network is ok, that would bolster the argument atht we should have per-region mirrors)14:36
anteayamordred: can you get statusbot to reset the channel topics?14:38
anteayalifeless tried but for some reason it wasn't effective14:39
mordred#status ok14:39
openstackstatusmordred: sending ok14:39
openstackstatusmordred: finished sending ok14:41
mordredthat's interessting openstackstatus - because I didn't see you change the status in here14:42
anteayayeah14:44
*** CaptTofu_ has quit IRC14:47
*** CaptTofu_ has joined #openstack-infra14:47
*** yfried_ has joined #openstack-infra14:50
*** sballe_ has joined #openstack-infra14:50
*** CaptTofu_ has quit IRC14:52
*** sballe has quit IRC14:53
*** CaptTofu_ has joined #openstack-infra14:56
mordredfungi, clarkb: ^^ statusbot seems to think that the launchpad openid status _IS_ the 'ok' status14:58
anteayait had disconnected and reconnected while that was the status14:59
anteayawould that have upset it?15:00
*** rcarrillocruz has joined #openstack-infra15:02
mordredyup15:03
mordredas best I can tell, there is no persisting of the saved topic dictionary to disk on restart15:03
anteayawell we have an explanation then15:03
anteayacool15:03
anteayawell all the channels are like that then15:03
anteayahow do we restore their individual topics?15:03
*** rcarrill` has quit IRC15:04
anteayatime for me to be outside with the twittering birds15:04
* anteaya runs away15:04
*** marun has quit IRC15:17
openstackgerritA change was merged to openstack-infra/config: Fix mirror build on py26  https://review.openstack.org/10006415:22
*** CaptTofu_ has quit IRC15:26
*** CaptTofu_ has joined #openstack-infra15:26
*** sweston has quit IRC15:27
mordred#status notice Launchpad is OK - statusbot lost the old channel statuses. They will need to be manually restored15:28
openstackstatusmordred: sending notice15:28
-openstackstatus- NOTICE: Launchpad is OK - statusbot lost the old channel statuses. They will need to be manually restored15:29
openstackstatusmordred: finished sending notice15:30
*** CaptTofu_ has quit IRC15:31
*** CaptTofu_ has joined #openstack-infra15:33
*** alexpilotti has joined #openstack-infra15:38
*** salv-orlando has joined #openstack-infra15:41
*** alexpilotti has quit IRC15:43
*** salv-orlando has quit IRC15:45
*** salv-orlando_ has joined #openstack-infra15:45
*** sarob_ has joined #openstack-infra15:51
openstackgerritFathi Boudra proposed a change to openstack-infra/jenkins-job-builder: builders: add 'publish over ssh' support as a build step  https://review.openstack.org/9843715:53
*** sarob_ has quit IRC15:56
*** fabo has joined #openstack-infra15:56
*** Guest47184 has joined #openstack-infra15:56
boris-42mordred hey there16:06
*** Longgeek_ has quit IRC16:06
*** salv-orlando_ has quit IRC16:15
*** salv-orlando has joined #openstack-infra16:19
*** rlandy has quit IRC16:19
*** CaptTofu_ has quit IRC16:26
*** CaptTofu_ has joined #openstack-infra16:26
*** salv-orlando has quit IRC16:29
*** CaptTofu_ has quit IRC16:31
*** _nadya_ has joined #openstack-infra16:32
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Make 'keep_descriptions' fully optional  https://review.openstack.org/10009816:34
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Use yaml local tags to support including files  https://review.openstack.org/4878316:35
*** yaguang has quit IRC16:35
*** alexpilotti has joined #openstack-infra16:36
*** _nadya_ has quit IRC16:41
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Make 'keep_descriptions' fully optional  https://review.openstack.org/10009816:45
openstackgerritDarragh Bailey proposed a change to openstack-infra/jenkins-job-builder: Use yaml local tags to support including files  https://review.openstack.org/4878316:45
*** sarob_ has joined #openstack-infra16:51
*** sarob_ has quit IRC16:56
*** cdub has quit IRC17:02
*** fbo_away is now known as fbo17:06
*** cody-somerville has joined #openstack-infra17:10
*** CaptTofu_ has joined #openstack-infra17:11
*** e0ne has joined #openstack-infra17:13
*** praneshp has joined #openstack-infra17:28
*** praneshp_ has joined #openstack-infra17:32
*** doude_ has joined #openstack-infra17:33
*** praneshp has quit IRC17:35
*** praneshp_ is now known as praneshp17:35
*** doude has quit IRC17:37
*** CaptTofu_ has quit IRC17:41
*** CaptTofu_ has joined #openstack-infra17:41
*** CaptTofu_ has quit IRC17:45
*** fbo is now known as fbo_away17:49
*** Guest47184 has quit IRC17:50
mordredgoooooooooooooooooooooooooooooool17:50
anteaya:D17:51
anteayawhich game is this now?17:51
*** sarob_ has joined #openstack-infra17:51
mordredswitzerland ecuador17:51
mordredamazing last second counter-attack goal17:52
anteayanice17:52
anteayaI'm listening to birds and watching the deer17:53
*** cody-somerville has quit IRC17:53
anteayathere will be many youtube vids of what you are seeing though17:53
SpamapSmordred: jesus. Switzerland Ecuador?17:55
SpamapSmordred: perhaps you need a hobby... ;)17:55
*** sarob_ has quit IRC17:55
*** e0ne has quit IRC17:59
SpamapSlaunchpad login is working17:59
SpamapSjust used it17:59
*** sweston has joined #openstack-infra17:59
*** e0ne has joined #openstack-infra17:59
*** CaptTofu_ has joined #openstack-infra18:00
*** zns has joined #openstack-infra18:00
*** Ryan_Lane has joined #openstack-infra18:03
*** _nadya_ has joined #openstack-infra18:05
mordredSpamapS: yes it is18:08
*** otter768 has joined #openstack-infra18:08
anteayaSpamapS: we can't get openstackstatus to set the topics back18:12
*** _nadya_ has quit IRC18:13
SpamapSanteaya: maybe it has to login too? ;)18:16
anteayaSpamapS: heh18:17
*** Ryan_Lane has quit IRC18:21
mordredSpamapS: it restarted and lost the old statues18:24
mordredstatuses18:24
boris-42mordred hi there18:24
mordredhey boris-42 !18:24
boris-42mordred how are you?)18:24
*** zns has joined #openstack-infra18:26
boris-42mordred actually I have some small question18:26
mordredboris-42: well! you?18:26
boris-42mordred good almost finished work on osprofiler library (need to publish new version)18:27
boris-42mordred have some issues with understanding infra config https://review.openstack.org/#/c/100052/18:27
boris-42mordred one question am I able to publish new versions without infra?18:27
boris-42mordred or it's better to use infra..18:27
SpamapSlost the old statuses?18:28
SpamapSI assume we have a bug open to write that stuff to disk?18:28
boris-42hehe seems like soccer time*18:30
boris-42=)18:30
*** arnaud__ has joined #openstack-infra18:30
*** andreykurilin_ has joined #openstack-infra18:35
*** arnaud__ has quit IRC18:44
mordredboris-42: looking18:44
boris-42mordred thanks18:45
mordredboris-42: well, you _could_ publish new versions without infra - but it's better to just use infra - it's extra work to not use us18:45
boris-42mordred I agree, I don't know how to work on plain github anymore =(18:46
boris-42=)18:46
mordredboris-42: :)18:46
*** CaptTofu_ has quit IRC18:47
*** CaptTofu_ has joined #openstack-infra18:47
*** CaptTofu_ has quit IRC18:51
*** e0ne has quit IRC18:51
*** sarob_ has joined #openstack-infra18:52
*** e0ne has joined #openstack-infra18:52
*** CaptTofu_ has joined #openstack-infra18:53
*** sarob_ has quit IRC18:56
*** e0ne has quit IRC18:58
*** e0ne has joined #openstack-infra18:59
*** Ryan_Lane has joined #openstack-infra19:04
*** e0ne has quit IRC19:07
*** Ryan_Lane has quit IRC19:09
*** arnaud__ has joined #openstack-infra19:16
*** alugovoi has joined #openstack-infra19:17
*** cdub has joined #openstack-infra19:20
openstackgerritafazekas proposed a change to openstack-infra/config: heat slow voting again  https://review.openstack.org/10010319:22
*** zns has quit IRC19:25
*** alugovoi has quit IRC19:26
*** e0ne has joined #openstack-infra19:38
*** e0ne_ has joined #openstack-infra19:39
*** e0ne has quit IRC19:39
openstackgerritA change was merged to openstack/requirements: Updated version requirement for pyghmi  https://review.openstack.org/9750819:42
*** alugovoi has joined #openstack-infra19:42
*** zns has joined #openstack-infra19:44
lifelessmordred: you note that status failed to send your 'its ok' as well19:44
*** sarob_ has joined #openstack-infra19:51
mordredlifeless: yes. for the reasons I've stated in scrollback19:52
*** sarob_ has quit IRC19:56
*** alugovoi has quit IRC19:57
lifelessmordred: I didn't see anything from you after that fail19:59
lifelessmordred: other than soccer disucssion19:59
*** _nadya_ has joined #openstack-infra19:59
*** CaptTofu_ has quit IRC20:00
mordredlifeless: statusbot restarted after the original #status alert - and it does not persist its dict of previous statuses across restarts20:00
*** CaptTofu_ has joined #openstack-infra20:00
mordredlifeless: so it currently does not know what the old status is, so it thinks the current status is correct, so it thinks there is nothing to do20:00
mordredlifeless: it's a bug20:00
lifelessmordred: so I got the non-persist, but I thought you tried to update the new status to explan the situation, and*that* didn't take, but that one is different to the current, so - differen tbug ?20:01
*** pcrews has quit IRC20:02
*** arnaud__ has quit IRC20:02
mordredlifeless: oh- so it did. hrm.20:03
mordredpossibly20:03
*** zns has quit IRC20:04
*** CaptTofu_ has quit IRC20:05
lifelessor perhaps statusbog didn't authenticate to services on reconnect20:05
lifelessor something20:05
*** _nadya_ has quit IRC20:06
*** arnaud__ has joined #openstack-infra20:15
*** praneshp has quit IRC20:17
*** otherwiseguy has joined #openstack-infra20:18
*** arnaud__ has quit IRC20:33
*** e0ne has joined #openstack-infra20:36
*** e0ne_ has quit IRC20:38
*** andreykurilin_ has quit IRC20:41
*** e0ne has quit IRC20:42
*** andreykurilin_ has joined #openstack-infra20:42
*** e0ne has joined #openstack-infra20:42
*** otter768 has quit IRC20:43
*** cnesa2 has quit IRC20:44
*** e0ne has quit IRC20:47
*** otter768 has joined #openstack-infra20:49
*** salv-orlando has joined #openstack-infra20:51
*** sarob_ has joined #openstack-infra20:51
*** sweston has quit IRC20:52
*** sarob_ has quit IRC20:56
*** sballe_ has quit IRC20:58
*** andreykurilin__ has joined #openstack-infra21:01
*** andreykurilin_ has quit IRC21:03
*** Ryan_Lane has joined #openstack-infra21:04
*** Ryan_Lane has quit IRC21:09
*** fifieldt__ has joined #openstack-infra21:20
*** fifieldt_ has quit IRC21:23
*** flaper87 is now known as flaper87|afk21:29
*** CaptTofu_ has joined #openstack-infra21:37
*** mrda-weekend is now known as mrda21:38
*** cnesa has joined #openstack-infra21:44
*** sandywalsh_ has joined #openstack-infra21:47
*** sandywalsh has quit IRC21:50
*** sarob_ has joined #openstack-infra21:51
*** sarob_ has quit IRC21:55
*** crc32 has joined #openstack-infra21:57
*** otherwiseguy has quit IRC22:05
*** praneshp has joined #openstack-infra22:06
*** sweston has joined #openstack-infra22:07
*** andreykurilin__ has quit IRC22:12
mattoliveraumordred: Fair enough, I only know what I've played with in my envornment, were we are using the same rules. We built up a zuul server using the same puppet rules for our turbo-hipster stuff. And with an empty sysadmins list, it didn't add the final 'root: <entires in sysadmins>' at the end of /etc/aliases. In exim the root user is on the disallowed list, meaning because the last line of alias's are22:17
*** CaptTofu_ has quit IRC22:17
mattoliveraumssing root ends up being the recipent of most messages.. causing an error that is logged  exim error log.22:17
mattoliveraumordred: if it's fine for you then sorry for any confusion.22:18
*** CaptTofu_ has joined #openstack-infra22:18
mattoliverauMorning all22:18
*** crc32 has quit IRC22:19
*** sweston_ has joined #openstack-infra22:22
*** CaptTofu_ has quit IRC22:22
mordred_phonemattoliverau: ah ... I grok now what you're sayinh22:23
mordred_phonemattoliverau: and I withdraw my disagreement with you22:23
*** sweston has quit IRC22:24
mordred_phoneIt seems we have a default value in our site.pp ... so we might want to go through and fix this everywhere22:24
*** zzelle has quit IRC22:24
*** salv-orlando has quit IRC22:27
morganfainbergmordred, i was thinking about the mod_Wsgi stuff, and yeah devstack should be doing that22:29
*** sweston_ has quit IRC22:30
morganfainbergmordred, unless jenkins itself needs mod_wsgi (why?) no reason.22:30
morganfainbergmordred, to install it *kicks enter key*22:30
*** alexpilotti has quit IRC22:30
*** camunoz has joined #openstack-infra22:30
*** salv-orlando has joined #openstack-infra22:35
*** sweston has joined #openstack-infra22:36
*** camunoz has quit IRC22:36
mordred_phonemorganfainberg: cool. I madez a patch22:40
mordredbtw - the corner that bosnia just kicked was about as bad as if _I_ had kicked it22:41
clarkbare you offering to help them?22:42
*** sweston has quit IRC22:42
*** morganfainberg is now known as morganfainberg_Z22:43
mordredclarkb: I mean, let's be honest...22:43
mordredclarkb: they're playing argentina22:43
mordredI _could_ help ... but I'm not sure it would make much difference in the outcome one way or the other22:43
clarkbI dunno messi is doing a lot of standing around22:44
mordredyah22:44
mordredI was actually just about to follow up with ... although argentina isn't exactly playing like costa rica or anything22:44
*** camunoz has joined #openstack-infra22:44
clarkbfair warning tomorrow afternoon is looking like it may be consumed by the US game22:45
mordredclarkb: jeblair will really enjoy, I think, our priorities over the next few weeks22:46
clarkb>_>22:47
mordredotoh - we get him back tomorrow - so woo22:47
*** sweston has joined #openstack-infra22:48
clarkbwell the next US game is on a sunday22:48
mordredclarkb: sure. but other interesting teams play as well22:48
clarkbso I doubt I will do the bar thing and be actually afk until the thursday game22:48
mordredyah. gotcha22:49
mordredI just sit on the couch where I normally work with the TV on instead of off22:49
clarkbthe other games will be viewed from couch22:49
mordredI believe I'm actually getting MORE work done22:49
mordredalthough I have been getting distracted by things as I try to work on the dib patch ...22:49
mordredI submitted three PRs to ansible earlier today :)22:49
clarkbgermany portugal tomorrow morning should be good22:50
mordredyah22:50
mordredI will be watching that22:50
*** sarob_ has joined #openstack-infra22:52
mordredclarkb: now, on the other hand - Iran v. Nigeria ...22:52
clarkbyeah and as I look deeper into the schedule the interesting games are more spread out22:53
clarkbthis opening week is dense though22:54
*** jhesketh__ has joined #openstack-infra22:54
*** sandywalsh_ has quit IRC22:55
*** jhesketh has quit IRC22:55
jhesketh__Morning22:55
*** jhesketh__ is now known as jhesketh22:55
*** sarob_ has quit IRC22:56
*** sandywalsh has joined #openstack-infra22:56
mattoliverauMorning jhesketh22:57
mordredmorning jhesketh23:01
*** pcrews has joined #openstack-infra23:03
*** yamahata has joined #openstack-infra23:03
*** asettle has joined #openstack-infra23:10
*** sweston has quit IRC23:10
*** alexandra has joined #openstack-infra23:10
*** asettle has quit IRC23:11
*** alexandra is now known as asettle23:11
*** zhiyan_ is now known as zhiyan23:14
*** zhiyan is now known as zhiyan_23:19
mordredgooooooooooooooooooooooooooooooooooooooooooooooooool23:23
*** sweston has joined #openstack-infra23:30
mattoliveraumordred: I'm looking at creating a patch for this sysadmin thing. To make it fool proof I was thinking in the exim module we could check to see if the sysadmin variable was empty, and if so create an 'admin' (or some other name)  user which no login. That way the mail system wont error. Another option is adding an if statment to the exim config to take root out of the disallowed list, but that would23:32
mattoliveraube loosening up security, which I don't want to do. Thoughts? (I know its your sunday, so happy to whip something together to get flamed down, so you can enjoy your Sunday and the rest of the game) :P23:32
bookwarhow do you set up jenkins authentication when install jenkins master via puppet?23:32
*** sunrenjie6 has joined #openstack-infra23:32
mordredmattoliverau: I think the 'create an admin user' approach sounds the best ... and probably the smallest amount of pain if it's the wrong choice23:33
mordredmattoliverau: jeblair is our ultimate expert on mail - so we'll ultimately want to get his opinion23:34
mattoliveraumordred: fair enough, I'll whip something together and wait for jeblair's flame :P Thanks23:34
mordredwhee!23:34
*** morganfainberg_Z is now known as morganfainberg23:35
*** arnaud__ has joined #openstack-infra23:36
bookwarmordred: do you own vuvuzela? :)23:38
mordredbookwar: haha. no :)23:39
morganfainbergi think i need to go camp at the coffee shop again tomorrow.23:40
morganfainbergi tend to get a ton of work done with the endless stream of caffeination (doesn't hurt that soccer is on)23:41
*** arnaud__ has quit IRC23:44
*** sweston has quit IRC23:45
*** sunrenjie6 has quit IRC23:46
*** jamielennox|away is now known as jamielennox23:47
*** sunrenjie6 has joined #openstack-infra23:47
mordredlifeless: are you blocked on me on any of those pbr patches? I think we're waiting on the spec review now, right?23:49
lifelessyes23:50
*** jhesketh has quit IRC23:50
*** salv-orlando has quit IRC23:51
*** sarob_ has joined #openstack-infra23:51
*** sunrenjie6 has quit IRC23:51
*** sarob_ has quit IRC23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!