| clarkb | reed: you can fetch the git ref and look there for committer vs author | 00:00 |
|---|---|---|
| reed | yeah, that makes the whole shebang even more complicated | 00:00 |
| * reed grows less and less happy as time passes | 00:00 | |
| jhesketh | reed: I think that's called getting older ;-) | 00:01 |
| reed | jhesketh, by this time I'll be a very grumpy old man by the age 43 | 00:01 |
| jhesketh | what are you working on? | 00:02 |
| *** sarob has quit IRC | 00:02 | |
| reed | jhesketh, https://bugs.launchpad.net/openstack-community/+bug/1195648 | 00:02 |
| uvirtbot | Launchpad bug 1195648 in openstack-community "Report separately first activities" [Wishlist,Fix committed] | 00:03 |
| reed | Activity Board | 00:03 |
| jhesketh | ah cool | 00:06 |
| openstackgerrit | A change was merged to openstack-infra/jeepyb: Revert "Remove tempest from list of projects that direct release." https://review.openstack.org/47921 | 00:08 |
| openstackgerrit | Jay Pipes proposed a change to openstack-infra/config: Enable gate on OpenStack Chef Repository https://review.openstack.org/47880 | 00:09 |
| *** michchap has quit IRC | 00:10 | |
| *** michchap has joined #openstack-infra | 00:11 | |
| Alex_Gaynor | clarkb: yes they pass (I never send patches like that without testing myself) I can leave a comment with shell output if it'd help | 00:12 |
| *** michchap has joined #openstack-infra | 00:14 | |
| *** dcramer_ has joined #openstack-infra | 00:14 | |
| clarkb | Alex_Gaynor: no, just confirmation from you that they work is good enough | 00:14 |
| *** dkehn_away is now known as dkehn | 00:15 | |
| openstackgerrit | A change was merged to openstack-infra/config: Check and gate requirements against PyPy https://review.openstack.org/47756 | 00:17 |
| clarkb | pip folks https://review.openstack.org/#/c/45555/7 <0.8 picks up 0.8b2 for example is that because of the silliness in tox and old pip? | 00:17 |
| clarkb | do we have a preferred way to deal with that? | 00:17 |
| lifeless | mordred: there is a little, from time to time. But there should not as a rule be any. | 00:17 |
| clarkb | mordred: ^ I think you would know | 00:18 |
| *** sarob_ has quit IRC | 00:19 | |
| Alex_Gaynor | clarkb: yes. That's because of old pip | 00:22 |
| mordred | clarkb: what? | 00:24 |
| mordred | clarkb: yes | 00:24 |
| mordred | clarkb: we need to update the tox.ini | 00:24 |
| clarkb | mordred: to specify the install command? | 00:24 |
| mordred | clarkb: the patch I posed to the mailing list this morning | 00:25 |
| mordred | clarkb: want me to make one real quick? | 00:25 |
| *** mrodden has joined #openstack-infra | 00:25 | |
| *** mrodden1 has quit IRC | 00:26 | |
| clarkb | mordred: no rush, I am just trying to flush my review queue and would like to do so properly (with an understanding of things) | 00:26 |
| clarkb | I left a +1 on that keystone change as <0.8 is correct but it does require new pip | 00:26 |
| mordred | clarkb: it's not the <0.8 that requires the new pip | 00:27 |
| mordred | clarkb: https://review.openstack.org/47942 | 00:27 |
| clarkb | mordred: it is though anyone with old pip will get hit by this issue | 00:28 |
| mordred | clarkb: it's possible we should have opened a project-wide bug on this | 00:28 |
| mordred | yes. but it's not due to sqla<0.8 | 00:28 |
| mordred | it's due to sphinx | 00:28 |
| mordred | and they should upgrade to pip>=1.4 | 00:28 |
| mordred | or else evil | 00:28 |
| clarkb | mordred: sqla<0.8 ionstalls 0.8b2 | 00:28 |
| clarkb | so you need ti for sqla too | 00:28 |
| mordred | AAAHHHHHHHHHH | 00:28 |
| clarkb | yes everyone should upgrade pip | 00:28 |
| mordred | fascinating | 00:29 |
| mordred | yes. you are corect | 00:29 |
| mordred | I'm 3 seconds from decreeing by fiat that openstack does not support pip < 1.4 in any way | 00:29 |
| mordred | I'm not sure I have that power | 00:29 |
| mordred | but I think I might do it anyway | 00:29 |
| clarkb | jog0: mtreinish I intend on cleaning up the puppet for elastic-recheck running on logstash.o.o tomorrow | 00:33 |
| clarkb | jog0: mtreinish hopefully we can get that running rsn | 00:34 |
| clarkb | and with that time to head home and find dinner | 00:34 |
| jog0 | clarkb: sounds good, still a bit more to do to make it pip able | 00:35 |
| jog0 | can we get this onto openstack-infra today? | 00:35 |
| jog0 | so I can send out an email | 00:35 |
| clarkb | I dont thibk so I need to afk for a bit then code review | 00:37 |
| clarkb | tomorrow may be possible. | 00:38 |
| jog0 | clarkb: no problem | 00:40 |
| mordred | clarkb: how's your review queue | 00:42 |
| openstackgerrit | A change was merged to openstack-infra/config: Enable gate on OpenStack Chef Repository https://review.openstack.org/47880 | 00:45 |
| *** pcm_ has quit IRC | 00:48 | |
| clarkb | mordred ~23 | 00:48 |
| jog0 | clarkb: haha try 141 | 00:48 |
| mordred | https://review.openstack.org/#/dashboard/2 | 00:50 |
| mordred | clarkb: is pypi-mirror in your review list? | 00:50 |
| mordred | fungi: ^^ how about you? | 00:50 |
| jog0 | mrodden: who was user #1 | 00:51 |
| mordred | jog0: jeblair | 00:51 |
| jog0 | haha | 00:51 |
| clarkb | mordred it isnt | 00:51 |
| mordred | just for the record - jenkins is #3 and Dolph is #4 :) | 00:51 |
| jog0 | is anyone here good at writting wiki pages that are short : https://wiki.openstack.org/wiki/GerritJenkinsGit#Test_Failures | 00:52 |
| clarkb | no :) | 00:53 |
| jog0 | not trying to blame actually wondering | 00:53 |
| mordred | we tend to like complete thoughts around here | 00:54 |
| *** anteaya has joined #openstack-infra | 00:54 | |
| jog0 | very complete | 00:54 |
| *** amotoki has joined #openstack-infra | 00:55 | |
| jog0 | I like the link to jenkins which is now wrong | 00:55 |
| *** UtahDave has quit IRC | 00:55 | |
| * anteaya having arrived home, and currently armed with a purring cat, serves notice to whatever virus has been dogging her movements | 00:55 | |
| mordred | anteaya: kitteh! | 00:57 |
| anteaya | :D | 00:58 |
| anteaya | I'm happy now, so is kitteh | 00:58 |
| anteaya | mordred, did pleia2 tell you we got plaques? | 01:02 |
| anteaya | now I need to get a mantle piece to put it on | 01:02 |
| anteaya | Ryan_Lane got one too | 01:03 |
| *** nosnos has joined #openstack-infra | 01:03 | |
| mordred | anteaya: nice! | 01:04 |
| *** dprince has joined #openstack-infra | 01:04 | |
| anteaya | I was very surprised | 01:04 |
| openstackgerrit | A change was merged to openstack-infra/config: Fix dependencies for activity-dev https://review.openstack.org/47708 | 01:04 |
| *** jerryz has quit IRC | 01:08 | |
| *** jhesketh has quit IRC | 01:09 | |
| *** morganfainberg has quit IRC | 01:11 | |
| *** morganfainberg has joined #openstack-infra | 01:12 | |
| *** Ryan_Lane has quit IRC | 01:14 | |
| jeblair | anteaya: better than plagues which is what i thought you said you got :) | 01:23 |
| *** melwitt has quit IRC | 01:24 | |
| jeblair | restarting zuul to pick up this morning's bugfix | 01:25 |
| *** yaguang has joined #openstack-infra | 01:25 | |
| jeblair | (eta 28 mins) | 01:25 |
| clarkb | hmm too late to merge Alex_Gaynor's change I guess | 01:26 |
| clarkb | thats ok it can go in next round | 01:26 |
| clarkb | jeblair: https://review.openstack.org/#/c/46869/ is the change I am referring to | 01:27 |
| clarkb | jeblair: I didn't approve as lifeless had a -1 and I tend to defer zuul approvals to you | 01:27 |
| lifeless | jeblair: did you have any feedback on the baremetal test plan ? | 01:28 |
| jeblair | lifeless: i have not had a chance to read it. i hope to before tomorrow's meeting. | 01:28 |
| jeblair | lifeless: thank you for sending it | 01:28 |
| lifeless | jeblair: ack, np. | 01:29 |
| openstackgerrit | Alex Gaynor proposed a change to openstack-infra/config: Handle when `id` is null. https://review.openstack.org/47953 | 01:29 |
| *** schwicht has quit IRC | 01:37 | |
| *** dcramer_ has quit IRC | 01:39 | |
| *** russellb has quit IRC | 01:43 | |
| *** russellb has joined #openstack-infra | 01:44 | |
| *** dprince has quit IRC | 01:53 | |
| *** rfolco has joined #openstack-infra | 01:57 | |
| *** jgriffith has joined #openstack-infra | 01:58 | |
| jgriffith | clarkb: ping | 01:58 |
| clarkb | jgriffith: pong (I am cooking dinner so won't be super responsive) | 02:02 |
| jgriffith | clarkb: go cook :) | 02:02 |
| jgriffith | clarkb: I'll post to the ML | 02:02 |
| jgriffith | Looking for a copyright expert | 02:02 |
| jgriffith | :) | 02:02 |
| clarkb | oh that isn't me. mordred and jeblair ^ | 02:02 |
| jgriffith | clarkb: ahh... thanks :) | 02:03 |
| jgriffith | enjoy your dinner | 02:03 |
| jgriffith | jeblair: mordred My question is WRT this: https://bugs.launchpad.net/keystone/+bug/1214176 | 02:03 |
| uvirtbot | Launchpad bug 1214176 in swift "Fix copyright headers to be compliant with Foundation policies" [Low,In progress] | 02:03 |
| jgriffith | jeblair: mordred going backand modifying all of the copyright headers doesn't seem kosher to me | 02:04 |
| *** morganfainberg has quit IRC | 02:08 | |
| *** morganfainberg has joined #openstack-infra | 02:08 | |
| clarkb | jgriffith: oh so in this particular case, the openstack LLC gave the openstack foundation copyright so this particular change is kosher | 02:09 |
| jgriffith | clarkb: hmm... okie dokie | 02:09 |
| mordred | what? | 02:09 |
| jgriffith | I stand corrected | 02:09 |
| mordred | looking | 02:09 |
| clarkb | adding openstack foundation copyright to files without original LLC copyright or files that haven't been written by foudantion employees is wrong | 02:09 |
| clarkb | now whether or not we need to change them wholesale in the fiels is questionable | 02:10 |
| jgriffith | personally my udnerstanding was that regardless of "giving" you don't modify the headers | 02:10 |
| mordred | well, we've been instructed by legal people in this case that we shoudl change the headers from LLC to Foundation | 02:10 |
| mordred | _I_ think that's crazy | 02:10 |
| mordred | because the files were (c) LLC back when that copyright was correct | 02:10 |
| jgriffith | mordred: I agree, but I typically feel that way when lawyers give directions :) | 02:10 |
| jgriffith | mordred: exactly!! | 02:11 |
| *** svarnau has quit IRC | 02:11 | |
| mordred | and then, a copyright of (c) $year Foundation should be _added_ | 02:11 |
| mordred | BUT | 02:11 |
| mordred | apparently that's not what happened or something | 02:11 |
| mordred | and I dont know | 02:11 |
| mordred | and lawyers lawyers | 02:11 |
| * mordred throws a shoe at the law student on the couch | 02:11 | |
| jgriffith | haha | 02:11 |
| jgriffith | mordred: careful, you'll be slapped with a law suit | 02:12 |
| mordred | wouldn't be the first time ... | 02:12 |
| anteaya | jeblair, agreed | 02:12 |
| jgriffith | So we believe that what should happen here is to "add" not search/replace then? | 02:13 |
| *** jerryz has joined #openstack-infra | 02:13 | |
| jgriffith | I'm fine with that | 02:13 |
| jgriffith | that makes sense | 02:13 |
| jgriffith | rewriting existing items makes no sense | 02:13 |
| * anteaya wonders if they even sell mantle pieces big enough for plagues | 02:14 | |
| mordred | jgriffith: no - lawyer people and the foundation have told us to replace them | 02:14 |
| mordred | my ranting earlier does not constitute legal opinion | 02:15 |
| mordred | as much as I'd love it to | 02:15 |
| mordred | anteaya: I think plagues are actually usually quite small | 02:15 |
| mordred | a tiny vial of some powder | 02:15 |
| *** emagana has quit IRC | 02:15 | |
| jgriffith | mordred: oh... ok | 02:15 |
| jgriffith | mordred: well if that's the case I suppose I'll +2/A | 02:16 |
| jgriffith | mordred: thanks | 02:16 |
| *** zyluo has joined #openstack-infra | 02:17 | |
| anteaya | mordred, oh okay guess I'm covered either way with a standard size mantle piece | 02:18 |
| *** rfolco has quit IRC | 02:30 | |
| *** jhesketh has joined #openstack-infra | 02:33 | |
| *** morganfainberg has quit IRC | 02:37 | |
| *** morganfainberg has joined #openstack-infra | 02:37 | |
| *** ryanpetrello has quit IRC | 02:45 | |
| *** dims has quit IRC | 02:48 | |
| *** Ryan_Lane has joined #openstack-infra | 02:50 | |
| *** dguitarbite has joined #openstack-infra | 03:18 | |
| *** jerryz has quit IRC | 03:20 | |
| *** blamar has quit IRC | 03:20 | |
| *** pcrews_ has quit IRC | 03:21 | |
| *** emagana has joined #openstack-infra | 03:26 | |
| *** jhesketh has quit IRC | 03:30 | |
| *** emagana has quit IRC | 03:31 | |
| *** pcrews_ has joined #openstack-infra | 03:32 | |
| *** anteaya has quit IRC | 03:33 | |
| *** morganfainberg has quit IRC | 03:39 | |
| *** morganfainberg has joined #openstack-infra | 03:40 | |
| *** jhesketh has joined #openstack-infra | 03:42 | |
| *** pcrews_ has quit IRC | 03:45 | |
| *** enikanorov-w has quit IRC | 03:48 | |
| *** enikanorov-w has joined #openstack-infra | 03:49 | |
| *** zehicle has joined #openstack-infra | 03:51 | |
| *** SergeyLukjanov has joined #openstack-infra | 04:00 | |
| openstackgerrit | Khai Do proposed a change to openstack-infra/config: Setup a private gerrit instance for security reviews https://review.openstack.org/47937 | 04:02 |
| *** dguitarbite has quit IRC | 04:09 | |
| *** DennyZhang has joined #openstack-infra | 04:13 | |
| *** morganfainberg has quit IRC | 04:16 | |
| *** morganfainberg has joined #openstack-infra | 04:19 | |
| *** hashar has quit IRC | 04:23 | |
| *** SergeyLukjanov has quit IRC | 04:24 | |
| *** vipul is now known as vipul-away | 04:27 | |
| *** morganfainberg has quit IRC | 04:35 | |
| *** morganfainberg has joined #openstack-infra | 04:35 | |
| *** DennyZhang has quit IRC | 04:36 | |
| *** DennyZhang has joined #openstack-infra | 04:36 | |
| *** reed has quit IRC | 04:43 | |
| *** boris-42 has joined #openstack-infra | 04:58 | |
| *** DennyZhang has quit IRC | 05:12 | |
| *** DennyZhang has joined #openstack-infra | 05:13 | |
| SpamapS | gah... gate fails eating my brain | 05:14 |
| *** DennyZhang has quit IRC | 05:15 | |
| *** DennyZhang has joined #openstack-infra | 05:17 | |
| *** DennyZhang has quit IRC | 05:21 | |
| *** DennyZhang has joined #openstack-infra | 05:21 | |
| *** nicedice has quit IRC | 05:30 | |
| *** blamar has joined #openstack-infra | 05:35 | |
| *** yaguang has quit IRC | 05:42 | |
| *** pblaho has joined #openstack-infra | 05:43 | |
| *** yaguang has joined #openstack-infra | 05:56 | |
| *** yolanda has joined #openstack-infra | 06:00 | |
| *** DennyZhang has quit IRC | 06:02 | |
| *** vipul-away is now known as vipul | 06:02 | |
| *** emagana has joined #openstack-infra | 06:22 | |
| jog0 | mtreinish: I updated the queries file | 06:33 |
| *** afazekas has joined #openstack-infra | 06:37 | |
| *** emagana has quit IRC | 06:44 | |
| *** fbo_away is now known as fbo | 06:46 | |
| *** yaguang has quit IRC | 06:54 | |
| *** yaguang has joined #openstack-infra | 06:54 | |
| *** dkehn_ has joined #openstack-infra | 06:57 | |
| *** flaper87|afk is now known as flaper87 | 06:58 | |
| *** dkehn has quit IRC | 07:01 | |
| *** jcoufal has joined #openstack-infra | 07:02 | |
| *** qba73 has joined #openstack-infra | 07:07 | |
| *** Ryan_Lane has quit IRC | 07:18 | |
| mikal | jog0: you around enough for a quick question? | 07:19 |
| *** hashar has joined #openstack-infra | 07:27 | |
| *** markmc has joined #openstack-infra | 07:28 | |
| *** rpodolyaka has joined #openstack-infra | 07:29 | |
| yaguang | need help , I push a patch half an hour ago, all Jenkins gate fails https://review.openstack.org/#/c/47965/ | 07:34 |
| *** SergeyLukjanov has joined #openstack-infra | 07:35 | |
| ttx | clarkb: looks like we have intended behavior from update-bug as of 4 hours ago. Haven't seen a "related bug" comment yet, though | 07:36 |
| *** tvb has joined #openstack-infra | 07:38 | |
| *** tvb is now known as Guest16638 | 07:39 | |
| *** Guest16638 is now known as tvb | 07:43 | |
| *** tvb has joined #openstack-infra | 07:43 | |
| *** annegentle has quit IRC | 07:44 | |
| *** annegentle has joined #openstack-infra | 07:45 | |
| jd__ | mordred: arf, happy to learn cloud archive was reverted :-/ | 07:45 |
| jd__ | mordred: that seems like a critical problem, is there a bug number about that nova issue? | 07:45 |
| *** Ryan_Lane has joined #openstack-infra | 07:49 | |
| *** yolanda has quit IRC | 07:51 | |
| *** dizquierdo has joined #openstack-infra | 07:51 | |
| *** SergeyLukjanov has quit IRC | 07:53 | |
| *** mkerrin has joined #openstack-infra | 07:53 | |
| *** kpavel has joined #openstack-infra | 07:56 | |
| *** Ryan_Lane has quit IRC | 07:56 | |
| *** SergeyLukjanov has joined #openstack-infra | 08:01 | |
| rpodolyaka | hmm... all tempest test seem to be constantly failing for new changesets | 08:14 |
| rpodolyaka | I don't see errors in console.log except "2013-09-24 08:06:52 ERROR: Module nbd does not exist in /proc/modules" | 08:15 |
| rpodolyaka | does anyone see something similar too? | 08:15 |
| *** qba73 has quit IRC | 08:16 | |
| *** Ryan_Lane has joined #openstack-infra | 08:24 | |
| *** Ryan_Lane has quit IRC | 08:28 | |
| *** zyluo has quit IRC | 08:34 | |
| afazekas | http://status.openstack.org/zuul/ all job failing | 08:39 |
| SpamapS | 2013-09-24 07:53:44.566 | Downloading/unpacking pyparsing>=2.0.1 (from cliff>=1.4.3->python-neutronclient>=2.3.0,<3->-r /home/jenkins/workspace/gate-heat-pep8/requirements.txt (line 20)) | 08:40 |
| afazekas | pep8: No distributions matching the version for pyparsing>=2.0.1 (from cliff>=1.4->-r /home/jenkins/workspace/gate-neutron-pep8/test-requirements.txt (line 3)) | 08:40 |
| SpamapS | 2013-09-24 07:53:44.566 | Could not find a version that satisfies the requirement pyparsing>=2.0.1 (from cliff>=1.4.3->python-neutronclient>=2.3.0,<3->-r /home/jenkins/workspace/gate-heat-pep8/requirements.txt (line 20)) (from versions: 1.5.6, 1.5.6, 1.5.7, 1.5.7) | 08:40 |
| SpamapS | fungi: here? | 08:41 |
| SpamapS | clarkb: ? | 08:41 |
| SpamapS | jeblair: ? | 08:41 |
| SpamapS | mordred: ? | 08:41 |
| * SpamapS looks for the big red panic button | 08:41 | |
| SpamapS | looks like a new version of cliff is pulling in an as-yet-un-synced pyparsing | 08:42 |
| *** therve has joined #openstack-infra | 08:43 | |
| ttx | hrm | 08:44 |
| *** raies_ has joined #openstack-infra | 08:45 | |
| ttx | SpamapS: we are on the dark side of the moon right now | 08:45 |
| raies_ | anyone can help me on build failure of https://review.openstack.org/#/c/47339/ | 08:45 |
| afazekas | raies_: all job failing http://status.openstack.org/zuul/ | 08:46 |
| raies_ | afazekas: then what I should do ? | 08:47 |
| ttx | raies_: at this point, wait a bit | 08:47 |
| raies_ | ok | 08:47 |
| SpamapS | yeah I think the mirror sync may fix it | 08:47 |
| * ttx tries to update infra status | 08:48 | |
| raies_ | I think I need not to upload further patches ?? | 08:48 |
| SpamapS | raies_: you can do a recheck on them later if you just want them up for review | 08:49 |
| afazekas | wait until the pep8 checks start working on the http://status.openstack.org/zuul/ , after that probably you can do a 'recheck no bug' | 08:49 |
| raies_ | thanks to all of you. :) | 08:50 |
| SpamapS | http://logs.openstack.org/periodic/periodic-mirror-python27/mirror27/44/console.html | 08:50 |
| SpamapS | looks like the mirror up 'n failed | 08:50 |
| SpamapS | https://jenkins.openstack.org/job/periodic-mirror-python27/ | 08:51 |
| ttx | where is te statusbot when you need it | 08:52 |
| ttx | ah. | 08:52 |
| SpamapS | ttx: any chance you have privileges on jenkins.o.o ? | 08:52 |
| ttx | SpamapS: I don't | 08:54 |
| SpamapS | Ok, because I think that job just has to be re-run | 08:55 |
| ttx | it also looks like I'm not whitelisted to openstackstatus bot | 08:55 |
| *** BobBall_Away is now known as BobBall | 08:58 | |
| SpamapS | ttx: for some reason I thought we had an EMEA infra person | 09:00 |
| ttx | SpamapS: we don't | 09:03 |
| SpamapS | So, IIRC, fungi and mordred are UTC-4 | 09:04 |
| SpamapS | (assuming mordred is actually at home.. big assumption) | 09:04 |
| SpamapS | jeblair and clarkb are UTC-7 | 09:04 |
| SpamapS | ttx: did you log in to jenkins and actually check that you don't have rights to start jobs? | 09:12 |
| SpamapS | ../manifests/site.pp: statusbot_auth_nicks => 'jeblair, ttx, fungi, mordred, clarkb, sdague', | 09:12 |
| chmouel | in the url http://logs.openstack.org/47/45447/9/check/gate-tempest-devstack-vm-large-ops/0cc80b5/ | 09:19 |
| chmouel | where is the 0cc80b5 come from? | 09:19 |
| chmouel | jeblair: ^ | 09:20 |
| chmouel | they seems to be random between all jobs (so not tight to commit) | 09:20 |
| chmouel | and have multiple of them | 09:20 |
| openstackgerrit | Julien Danjou proposed a change to openstack/requirements: Bump ceilometerclient dependency https://review.openstack.org/47994 | 09:22 |
| *** boris-42 has quit IRC | 09:27 | |
| *** dizquierdo has left #openstack-infra | 09:30 | |
| *** tvb has quit IRC | 09:35 | |
| *** tvb has joined #openstack-infra | 09:36 | |
| *** tvb is now known as Guest22854 | 09:36 | |
| *** yolanda has joined #openstack-infra | 09:36 | |
| *** Guest22854 is now known as tvb | 09:37 | |
| *** tvb has joined #openstack-infra | 09:37 | |
| *** jhesketh_ has joined #openstack-infra | 09:53 | |
| *** thomasbiege1 has joined #openstack-infra | 09:58 | |
| *** dizquierdo has joined #openstack-infra | 10:16 | |
| *** ndipanov has joined #openstack-infra | 10:19 | |
| ndipanov | hey infra folks - nova gate seems to be failing badly due to pip struggling with pyparsing versions | 10:22 |
| *** thomasbiege1 has quit IRC | 10:22 | |
| *** fbo is now known as fbo_away | 10:25 | |
| *** thomasbiege has joined #openstack-infra | 10:25 | |
| *** mordred has quit IRC | 10:29 | |
| *** jesusaurus has quit IRC | 10:30 | |
| *** mordred has joined #openstack-infra | 10:31 | |
| jd__ | ndipanov: ah there's not nova only actually | 10:33 |
| jd__ | I just hit it on ceilometerclient | 10:34 |
| ndipanov | jd__, is infra using some mirrors/caching for pypi? | 10:34 |
| ndipanov | jd__, since pyparsing seems to install fine on my laptop | 10:34 |
| ndipanov | with thw same pip args | 10:34 |
| *** schwicht has joined #openstack-infra | 10:39 | |
| *** thomasbiege has quit IRC | 10:40 | |
| *** thomasbiege has joined #openstack-infra | 10:40 | |
| *** thomasbiege has quit IRC | 10:47 | |
| *** alexpilotti has joined #openstack-infra | 10:48 | |
| *** jesusaurus has joined #openstack-infra | 10:49 | |
| *** yaguang has quit IRC | 10:55 | |
| *** boris-42 has joined #openstack-infra | 11:03 | |
| *** zhiyan has joined #openstack-infra | 11:12 | |
| zhiyan | adarazs_: ping | 11:12 |
| adarazs_ | zhiyan: pong | 11:13 |
| *** adarazs_ is now known as adarazs | 11:13 | |
| zhiyan | adarazs_: sorry to interrupt you, i have a quick ask | 11:13 |
| sdague | dhellmann seems to have released a new cliff which bumped the min version of pyparsing beyond what's in global requirements | 11:13 |
| sdague | so it's created a wedge | 11:13 |
| adarazs | zhiyan: okay, though I'm not sure you really want to ask me :) | 11:13 |
| *** pcm_ has joined #openstack-infra | 11:14 | |
| ttx | SpamapS: sorry, got sidetracked, French IRS surprise investigation | 11:14 |
| ttx | A great way to lose 3 hours of work. | 11:14 |
| zhiyan | adarazs: :) we have a change at https://review.openstack.org/#/c/47161/ , event it not get approve but it's a change on the right way, but as Jenkins said, the change break the test. | 11:15 |
| adarazs | zhiyan: check my name, I don't think I'm the nickname you're looking for :) | 11:15 |
| ttx | I /should/ be able to update statusbot but it's not really working | 11:15 |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: uncap pyparsing requirement https://review.openstack.org/48014 | 11:16 |
| zhiyan | adarazs: so i have a question is that, how can we landing this type change? seems it need disable some test case. and after this change landing we will change devstack to work with it (actually we did it at https://review.openstack.org/#/c/47552/ ) | 11:16 |
| adarazs | zhiyan: I don't know what you're talking about, I don't think you meant to talk to me. | 11:16 |
| zhiyan | adarazs: oh, sorry. | 11:17 |
| zhiyan | ok folks who can give some light to me? | 11:17 |
| *** dkranz has quit IRC | 11:17 | |
| zhiyan | fungi ^^ | 11:18 |
| sdague | zhiyan: you have to have the tool support deprecation, you can't just remove one set of commands and change them to something else | 11:18 |
| zhiyan | sdague: make sense... | 11:20 |
| zhiyan | sdague: thanks, will think about it. | 11:20 |
| zhiyan | sdague: btw, for more common scenario, if a change break the tempset test, but it's a right change, how can i handling it? just ask disable some test case to ask it land down? and then to change the test? is it a correct approach? | 11:26 |
| sdague | zhiyan: in the general case, it needs a bug, and the core team on the project to have at least one +2 on the patch you want to change. Then we can do a skip, land, and unskip with the new behavior. Once we get past release I'll write more of this down | 11:28 |
| sdague | anyone know how often the mirrors are build to infra? | 11:30 |
| sdague | fungi you up yet? | 11:30 |
| ttx | sdague: not everyone gets up at 7am like you do :) | 11:31 |
| sdague | :) | 11:31 |
| sdague | yep, but fungi is at least in my tz :) | 11:31 |
| sdague | so I think we actually have to do that cap instead | 11:32 |
| sdague | because the infra mirror takes time | 11:32 |
| ttx | let's all blame dhellmann! | 11:34 |
| * ttx fetches torches and pikes | 11:34 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: cap cliff to < 1.4.5 https://review.openstack.org/48019 | 11:35 |
| sdague | ok, there is the alternate patch | 11:35 |
| ttx | sdague: at this point in release it might actually make more sense. | 11:35 |
| sdague | yeh | 11:36 |
| sdague | I was torn on what to propose first | 11:36 |
| sdague | jd__: how do you feel about - https://review.openstack.org/#/c/48019/ ? | 11:37 |
| sdague | man, we need a .eu infra person :) | 11:37 |
| *** fbo_away is now known as fbo | 11:43 | |
| sdague | ug, there is another issue, even on the cliff pin | 11:44 |
| openstackgerrit | Mark McLoughlin proposed a change to openstack/requirements: Use oslo.config 1.2.0 https://review.openstack.org/48022 | 11:44 |
| sdague | https://jenkins02.openstack.org/job/gate-grenade-devstack-vm/10635/console - cliff isn't working with grizzly versions of python distutils | 11:45 |
| sdague | ok, this is going to be unwinding for hours I expect | 11:46 |
| *** ArxCruz has joined #openstack-infra | 11:47 | |
| boris-42 | sdague hi | 11:48 |
| boris-42 | sdague do you know something about http://logs.openstack.org/13/48013/2/check/gate-rally-pep8/5661b67/console.html why I am not able to install gevent ?) | 11:49 |
| *** thomasbiege has joined #openstack-infra | 11:51 | |
| rpodolyaka | boris-42: apparently, libevent1-dev package is not installed | 11:52 |
| *** thomasbiege has quit IRC | 11:52 | |
| boris-42 | rpodolyaka do you know it is possible to install it?) | 11:52 |
| rpodolyaka | boris-42: not sure, whether we have policy what libs can be installed on Infra VMs | 11:52 |
| boris-42 | yeah yeah.. | 11:53 |
| rpodolyaka | boris-42: so Infra guys should know better :) | 11:53 |
| boris-42 | fungi hi | 11:54 |
| sdague | why would be installing gevent in a pep8 run? | 11:54 |
| boris-42 | sdague actually I don't know why OpenStack CI does this=) | 11:54 |
| boris-42 | sdugue https://review.openstack.org/#/c/48013/ | 11:55 |
| boris-42 | here is the problem patch sdague ^ | 11:55 |
| *** yaguang has joined #openstack-infra | 11:57 | |
| sdague | your out of tree test suite is doing it | 11:57 |
| sdague | 2013-09-24 11:45:24.123 | Downloading/unpacking gevent==0.13.8 (from fuel-ostf-tests->-r /home/jenkins/workspace/gate-rally-pep8/requirements.txt (line 15)) | 11:57 |
| sdague | 2013-09-24 11:45:24.123 | Running setup.py egg_info for package gevent | 11:57 |
| sdague | that's one of the reasons you really shouldn't pip install random external urls | 11:57 |
| *** mriedem has joined #openstack-infra | 11:57 | |
| *** mriedem has quit IRC | 11:58 | |
| boris-42 | sdague it is not random url=) | 11:58 |
| sdague | it's not in our git... so I consider it random :) | 11:58 |
| boris-42 | sdague it is the future part of this system.. | 11:58 |
| *** amotoki has quit IRC | 11:59 | |
| boris-42 | sdague it will be soon in stackforge.. | 11:59 |
| sdague | yeh, then you should put it into a stackforge git tree | 11:59 |
| *** sandywalsh has quit IRC | 11:59 | |
| sdague | right, well that's the crux of your problem. The requirements in the out of tree tarball | 11:59 |
| jd__ | sdague: I think we can kick in the mirroring earlier, no? | 12:00 |
| * jd__ didn't read the backlog yet | 12:00 | |
| jd__ | ttx: sorry about IRS :] | 12:00 |
| boris-42 | sdague by the way how we will fix this problem? | 12:01 |
| boris-42 | sdague when we will be on stackforge.. | 12:01 |
| sdague | boris-42: don't know, and working on gate break now, so it will have to wait | 12:01 |
| boris-42 | sdague but it will be ok to add gevent in global requirements? | 12:02 |
| sdague | jd__: so.... even with the mirror update - http://logs.openstack.org/14/48014/1/check/gate-grenade-devstack-vm/386eb51/console.html seems not solvable | 12:02 |
| jd__ | sdague: what what what is happening! /o\ | 12:02 |
| *** afazekas has quit IRC | 12:02 | |
| jd__ | sdague: ok you got me, +2'ed | 12:03 |
| sdague | jd__: so grizzly devstack version of python tools apparently don't work with cliff 1.4.5 | 12:03 |
| *** dims has joined #openstack-infra | 12:08 | |
| *** thomasm has joined #openstack-infra | 12:10 | |
| zhiyan | sdague: got it, thanks. | 12:10 |
| *** sandywalsh has joined #openstack-infra | 12:12 | |
| *** mriedem has joined #openstack-infra | 12:17 | |
| *** dkranz has joined #openstack-infra | 12:18 | |
| sdague | so I think this - https://review.openstack.org/#/c/48032/ | 12:19 |
| sdague | and then cliff pin, will get the spice flowing again | 12:19 |
| *** afazekas has joined #openstack-infra | 12:20 | |
| openstackgerrit | Dolph Mathews proposed a change to openstack/requirements: remove oslo.config from dev-requirements exceptions https://review.openstack.org/48033 | 12:20 |
| openstackgerrit | Dolph Mathews proposed a change to openstack/requirements: remove oslo.config from dev-requirements exceptions https://review.openstack.org/48033 | 12:22 |
| *** weshay has joined #openstack-infra | 12:25 | |
| *** cppcabrera has joined #openstack-infra | 12:28 | |
| *** dims has quit IRC | 12:29 | |
| *** dims has joined #openstack-infra | 12:30 | |
| *** zhiyan has left #openstack-infra | 12:30 | |
| sdague | man... I hope dhellmann isn't on a plane today | 12:30 |
| *** dolphm has joined #openstack-infra | 12:31 | |
| *** dkliban has joined #openstack-infra | 12:33 | |
| *** adalbas has joined #openstack-infra | 12:34 | |
| *** EmilienM has quit IRC | 12:35 | |
| sdague | jd__: once this merges - https://review.openstack.org/#/c/48032/ - can you recheck the cliff capping review? | 12:36 |
| sdague | I'll probably be in transit | 12:36 |
| *** EmilienM has joined #openstack-infra | 12:36 | |
| jd__ | sdague: yes | 12:37 |
| dhellmann | good morning | 12:41 |
| dhellmann | jd__, sdague : that cliff release was supposed to fix the debian packaging issue with pyparsing | 12:42 |
| dhellmann | jd__, sdague : what's going on now? | 12:42 |
| jd__ | dhellmann: it can be installed with the Python env of Grizzly for whatever reason | 12:42 |
| jd__ | dhellmann: and good morning! ;-) | 12:42 |
| dhellmann | jd__: log? | 12:42 |
| jd__ | dhellmann: http://logs.openstack.org/14/48014/1/check/gate-grenade-devstack-vm/386eb51/console.html | 12:43 |
| dhellmann | jd__: that's the "setuptools tries to upgrade itself with distribute" problem | 12:44 |
| jd__ | ah, the grand classic | 12:44 |
| dhellmann | well, I don't see distribute in the log so maybe it's just setuptools trying to upgrade itself | 12:45 |
| jd__ | dhellmann: what's the solution to that? | 12:45 |
| dhellmann | find the thing that's trying to update setuptools as a dependency, and remove the dependency; then ensure the latest setuptools is installed at the start of the job | 12:45 |
| *** weshay has quit IRC | 12:46 | |
| *** mriedem has quit IRC | 12:46 | |
| *** tvb has quit IRC | 12:46 | |
| *** ndipanov has quit IRC | 12:46 | |
| *** jhesketh has quit IRC | 12:46 | |
| *** zul has quit IRC | 12:46 | |
| *** Reapster has quit IRC | 12:46 | |
| *** clayg has quit IRC | 12:46 | |
| *** locke105 has quit IRC | 12:46 | |
| *** jcooley has quit IRC | 12:46 | |
| *** jeremyb has quit IRC | 12:46 | |
| *** rockstar has quit IRC | 12:46 | |
| *** chmouel has quit IRC | 12:46 | |
| *** jcooley has joined #openstack-infra | 12:46 | |
| dhellmann | or just pin cliff in grizzly | 12:46 |
| *** rockstar has joined #openstack-infra | 12:46 | |
| *** clayg has joined #openstack-infra | 12:46 | |
| *** Reapster has joined #openstack-infra | 12:46 | |
| *** jeremyb has joined #openstack-infra | 12:46 | |
| dhellmann | that's probably the simplest solution | 12:46 |
| *** weshay has joined #openstack-infra | 12:46 | |
| *** tvb has joined #openstack-infra | 12:46 | |
| *** rockstar has quit IRC | 12:46 | |
| *** rockstar has joined #openstack-infra | 12:46 | |
| jd__ | that's what sdague did I think with https://review.openstack.org/#/c/48032/ | 12:46 |
| *** mriedem has joined #openstack-infra | 12:46 | |
| *** tvb has quit IRC | 12:46 | |
| *** tvb has joined #openstack-infra | 12:46 | |
| *** DennyZhang has joined #openstack-infra | 12:47 | |
| dhellmann | I guess I should have called that cliff 1.5 :-( | 12:47 |
| *** ndipanov has joined #openstack-infra | 12:47 | |
| *** zul has joined #openstack-infra | 12:47 | |
| *** ndipanov has quit IRC | 12:47 | |
| *** ndipanov has joined #openstack-infra | 12:47 | |
| *** chmouel has joined #openstack-infra | 12:47 | |
| dhellmann | that change was in master, though | 12:47 |
| zul | ugh new cliff? | 12:47 |
| * jd__ investigates | 12:48 | |
| *** jhesketh has joined #openstack-infra | 12:48 | |
| dhellmann | zul: just trying to fix a dependency issue on pyparsing, no code changes | 12:48 |
| *** locke105 has joined #openstack-infra | 12:48 | |
| *** jhesketh_ has quit IRC | 12:49 | |
| dhellmann | at least no important changes | 12:49 |
| zul | dhellmann: ah the pyparsing 2.0 thing? | 12:49 |
| dhellmann | zul: yeah, pyparsing 2.0.1 works with python 2 and 3, and cliff 1.4.5 takes advantage of that | 12:49 |
| *** jhesketh_ has joined #openstack-infra | 12:49 | |
| jd__ | pyparsing, sourceforge hmmm | 12:49 |
| zul | dhellmann: cool so i can drop a patch we carry :) | 12:49 |
| dhellmann | zul: patch? | 12:50 |
| zul | dhellmann: we are shipping pyparsing 2.0 so we had to force cliff to use pyparsing 2.0 | 12:50 |
| dhellmann | zul: how did you make that work on python 2? | 12:50 |
| sdague | dhellmann: so the issue is actually that new clients have to work with old servers, so clients pull in new cliff | 12:51 |
| *** rfolco has joined #openstack-infra | 12:51 | |
| sdague | which is actually what grizzly devstack tripped over | 12:51 |
| zul | dhellmann: patched the setup.py i think | 12:51 |
| dhellmann | sdague: ah, gotcha | 12:51 |
| dhellmann | sdague: should I do a 1.4.6 that undoes 1.4.5 and then a 1.5 that redoes it? | 12:51 |
| sdague | dhellmann: I think that might work | 12:52 |
| zul | dhellmann: like https://github.com/dreamhost/cliff/commit/6a35ba1b0d7b40eaa184769400ab7445427925dc | 12:52 |
| dhellmann | zul: maybe the issue was just with getting pyparsing 2 to install, then, but I thought it had python 3 code in the lib | 12:52 |
| sdague | though did you see the log in grenade with setuputils ? | 12:52 |
| zul | dhellmann: ah i dont necessarily care about python3 yet for cliff since i havent gotten to clients that use it yet | 12:52 |
| dhellmann | zul: I meant how did you get pyparsing 2 to work with python 2 | 12:52 |
| zul | dhellmann: nothing special tests pass with it | 12:53 |
| sdague | dhellmann: https://jenkins02.openstack.org/job/gate-grenade-devstack-vm/10644/console - as long as you think you can get a 1.4.6 to work where 1.4.5 failed there, I think we'll be good | 12:53 |
| jd__ | I can't seem to find what triggers that problem | 12:53 |
| sdague | then we can start unwinding the work arounds | 12:53 |
| dhellmann | sdague: that error usually means something tried to upgrade setuptools as a dependency | 12:54 |
| sdague | dhellmann: ok | 12:54 |
| *** kgriffs_afk is now known as kgriffs | 12:54 | |
| *** mkoderer_ is now known as mkoderer | 12:54 | |
| sdague | anyway, I need to move into the real office. Back on in 30 or so | 12:54 |
| jd__ | 2013-09-24 12:31:18.147 | 2013-09-24 12:31:18 File "setup.py", line 22, in <module> | 12:54 |
| jd__ | 2013-09-24 12:31:18.149 | 2013-09-24 12:31:18 pbr=True) | 12:54 |
| jd__ | so it's something using pbr, not cliff | 12:55 |
| jd__ | but the log talk about cliff just before that | 12:55 |
| jd__ | so I don't get how what's happening | 12:55 |
| dhellmann | sdague: ok, I'm going in this morning, too, so it will be a couple of hours before I can really work on this | 12:55 |
| *** tkammer has joined #openstack-infra | 12:56 | |
| dims | How do i annotate reviews busted by the cliff bug? is this summary ok? "Caused by bug 1229647. need to wait for https://review.openstack.org/#/c/48019/ to land." | 12:56 |
| uvirtbot | Launchpad bug 1229647 in grenade "TypeError" [Undecided,New] https://launchpad.net/bugs/1229647 | 12:56 |
| *** pcm_ has quit IRC | 12:56 | |
| *** dprince has joined #openstack-infra | 12:56 | |
| *** pblaho has quit IRC | 12:57 | |
| *** pblaho has joined #openstack-infra | 12:57 | |
| *** dolphm has quit IRC | 13:00 | |
| dhellmann | jd__: neutron client has cliff>=1.4.3, but no upper cap, so it wouldn't have mattered what version number I used for this release. :-/ | 13:01 |
| jd__ | dhellmann: I don't think it matters indeed | 13:02 |
| *** afazekas has quit IRC | 13:03 | |
| openstackgerrit | David Shrewsbury proposed a change to openstack-infra/config: Add #stackforge-libra to gerritbot https://review.openstack.org/48040 | 13:03 |
| *** tkammer has quit IRC | 13:03 | |
| dhellmann | jd__, sdague : I'm spinning up a grizzly dev env to try to reproduce this | 13:03 |
| *** yaguang has quit IRC | 13:04 | |
| *** tkammer has joined #openstack-infra | 13:04 | |
| *** dkliban has quit IRC | 13:06 | |
| *** yolanda has quit IRC | 13:07 | |
| *** yolanda has joined #openstack-infra | 13:08 | |
| *** julim has joined #openstack-infra | 13:12 | |
| fungi | wow, so much scrollback | 13:12 |
| *** pblaho has quit IRC | 13:13 | |
| mordred | fungi: I was just thinking the same thing | 13:14 |
| tkammer | hey guys, is there a known issue with zuul at the moment? | 13:15 |
| jd__ | gate's broken | 13:15 |
| *** jhesketh has quit IRC | 13:16 | |
| dims | staring at 48019,1 on http://status.openstack.org/zuul/ - that's probably the first one since https://review.openstack.org/48032 landed | 13:16 |
| *** jhesketh has joined #openstack-infra | 13:17 | |
| jd__ | mordred, fungi: if you missed my earlier message in this brouhaha, I wanna know more precisely why cloud archive has been reverted :( | 13:18 |
| * fungi looks to see if there's still new pyparsing badness in the gate | 13:18 | |
| *** dkehn_ is now known as dkehn | 13:19 | |
| mordred | jd__: I _think_ zul is looking at the nova/libvirt thing. I believe sdague is too | 13:20 |
| *** ryanpetrello has joined #openstack-infra | 13:21 | |
| mordred | SpamapS: even if ttx had perms to start jobs in jenkins, it doesn't work that way, you can't just run a single job in jenkins - it needs to be run by zuul | 13:21 |
| *** ryanpetrello has quit IRC | 13:21 | |
| fungi | sdague: and yeah, while i'm usually up earlier, i got talked into a few rounds of scrabble last night which ran unusually late | 13:21 |
| mordred | ndipanov: yes. we use a mirror | 13:21 |
| *** thomasbiege has joined #openstack-infra | 13:22 | |
| *** cthulhup has joined #openstack-infra | 13:22 | |
| dhellmann | jd__: it looks like the global requirements for stable/grizzly have pyparsing>=1.5.6,<2.0 -- is that what you see? devstack refused to install nova for me at all because of the dependency conflict | 13:23 |
| dhellmann | jd__: so I didn't even get to the point where the error in the logs occurred | 13:23 |
| mordred | ok. I gave up on reading all of the logs - | 13:23 |
| mordred | dhellmann: how can I be helpful re: the cliff stuff | 13:24 |
| dhellmann | mordred: http://logs.openstack.org/14/48014/1/check/gate-grenade-devstack-vm/386eb51/console.html | 13:24 |
| jd__ | dhellmann: weird, maybe grenade is not using it? | 13:24 |
| dhellmann | mordred: the error there looks like the "setuptools upgrading itself as a dependency" issue | 13:24 |
| mordred | yes. that looks exactly like that error | 13:24 |
| *** dizquierdo has quit IRC | 13:24 | |
| mordred | dhellmann: is that grenade run trying to upgrade to latest cliff? | 13:24 |
| dhellmann | but we can't figure out *what* caused the upgrade | 13:24 |
| mordred | gotcha | 13:25 |
| openstackgerrit | Julien Danjou proposed a change to openstack-infra/devstack-gate: Enable Ceilometer alarm services https://review.openstack.org/48046 | 13:25 |
| dhellmann | I don't know exactly what grenade does. is that the upgrade tester thing? | 13:25 |
| mordred | yes | 13:25 |
| mordred | it is | 13:25 |
| dhellmann | ok, I'm not sure how to reproduce that locally | 13:25 |
| mordred | dhellmann: it looks like it's still doing setup.py develop - and I think we've changed devstack to do pip install -e . | 13:25 |
| fungi | it's probably been said at this point, but problems encountered like that where a pip install of something tries to upgrade setuptools usually means something is incorrectly specifying a dependency on setuptools | 13:25 |
| dhellmann | what I've tried is running stable/grizzly devstack and that fails on a version conflict when it tries to install nova | 13:26 |
| dhellmann | fungi: yeah, I looked through the dependency chain for cliff and couldn't find anything like that | 13:26 |
| ndipanov | mordred, I assumed so, but couldn't see it specified in the pip arguments so I gave up and sent an email :) | 13:27 |
| dhellmann | fungi: and I don't see that happening in the logs, either, so I'm stumped | 13:27 |
| *** cthulhup has quit IRC | 13:27 | |
| dhellmann | mordred: is that substantially different? | 13:27 |
| ttx | mordred: in related news, I tried to /msg openstackstatus to warn everyone but it ignored me | 13:28 |
| mordred | dhellmann: yes | 13:28 |
| ttx | mordred: I'm in the whitelist though, dunno why it fails | 13:28 |
| * dhellmann sighs | 13:28 | |
| fungi | okay, i'm now caught up on scrollback (in this channel at least) | 13:28 |
| mordred | I'm about to propose a patch to devstack stable/grizzly | 13:28 |
| sdague | dhellmann: so 1.5 wouldn't have prevented this break, however it would have at least been signaling that something larger was comming | 13:28 |
| sdague | mordred: which piece of the fix are you running at stable/grizzly? | 13:28 |
| dhellmann | sdague: yeah. The way I read the semantic versioning instructions implied that a point release was ok. | 13:29 |
| *** changbl has quit IRC | 13:29 | |
| fungi | ttx: i'll dig up the statusbot docs for you in a sec, but it doesn't support /msg i don't think. we usually just do... | 13:30 |
| fungi | #status alert dependency problems in gating, currently under investigation... more news as it unfolds | 13:30 |
| openstackstatus | NOTICE: dependency problems in gating, currently under investigation... more news as it unfolds | 13:30 |
| *** ChanServ changes topic to "dependency problems in gating, currently under investigation... more news as it unfolds" | 13:30 | |
| dhellmann | sdague: this was a no-code, bug fix, change :-/ | 13:30 |
| *** markmcclain has quit IRC | 13:30 | |
| sdague | yeh, the dependency graph though really means like a merge commit, the impact is the entire dependency tree below you | 13:31 |
| ttx | fungi: oh! | 13:31 |
| ttx | fungi: I guess i'm spoiled by chanserv/nickserv | 13:32 |
| sdague | dhellmann: I guess we need to get on that automatically running after out dependencies thing | 13:32 |
| fungi | ttx: http://ci.openstack.org/irc.html#statusbot | 13:32 |
| fungi | (for future reference) | 13:32 |
| ttx | fungi: yes, I read that but assumed it required /msg | 13:32 |
| dhellmann | sdague: ok, I think I see what you mean there. | 13:33 |
| fungi | ttx: aha, got it. we should probably clarify in the doc. i agree it's vague on that point | 13:33 |
| dims | sdague, 48019,1 seems to have gotten through (looking at http://status.openstack.org/zuul/) | 13:33 |
| dhellmann | sdague: it seems like neutronclient should not be specifying the version of pyparsing they want, since they don't use it directly | 13:34 |
| dims | one thing still running (https://jenkins01.openstack.org/job/gate-requirements-install/284/console) | 13:34 |
| sdague | dhellmann: sorry, language not all there, still caffinating :) | 13:34 |
| dhellmann | sdague: me, too :-) | 13:34 |
| sdague | dims: yep, that's also a devstack run, so we need to make sure that works as well | 13:34 |
| jd__ | https://review.openstack.org/#/c/48019/ is almost ready to got but has timeout issue | 13:34 |
| dhellmann | sdague: let's just say I'll try to remember to bump the 2nd digit next time | 13:35 |
| dims | sdague, ouch, that one kicked the bucket | 13:35 |
| sdague | which it didn't.... | 13:35 |
| dims | :( | 13:35 |
| sdague | mordred: http://logs.openstack.org/19/48019/1/check/gate-requirements-install/9562d98/console.html this bug known to you? | 13:35 |
| sdague | that's a fail I don't recognize | 13:36 |
| sdague | dhellmann: works for me :) | 13:36 |
| fungi | wow, that job doesn't usually take so long | 13:36 |
| fungi | sdague: "Build timed out (after 30 minutes). Marking the build as failed." | 13:36 |
| sdague | yeh, exactly | 13:37 |
| fungi | sdague: the gate-requirements-install job is currently limited in jenkins/jjb to 30 minutes | 13:37 |
| dims | "timed out (after 30 minutes)" | 13:37 |
| fungi | so that's jenkins killing it at that point | 13:37 |
| fungi | bigger question is why it took so long--doesn't usually go so slowly | 13:37 |
| sdague | fungi: right, exactly | 13:38 |
| sdague | fungi: what does that job actually do? | 13:38 |
| *** matsuhashi has quit IRC | 13:38 | |
| sdague | we timing out on a network connection or something? | 13:38 |
| fungi | it tests installing everything listed in the global-requirements and dev requirements to make sure they specify valid existing package versions on pypi.python.org and that they're sanely coinstallable | 13:39 |
| fungi | so that we don't land conflicting requirements | 13:39 |
| *** matty_dubs|gone is now known as matty_dubs | 13:40 | |
| mordred | sdague: https://review.openstack.org/48049 | 13:40 |
| mordred | sdague: for one thing | 13:41 |
| dims | fungi, seems to have 4-5 min pauses in between | 13:41 |
| sdague | mordred: ok, cool | 13:41 |
| fungi | dims: it's possible there's something wrong with the slave where that job ran--i'm checking it out now | 13:41 |
| jd__ | sdague: fungi: this is not the first time that timeout issue happens today | 13:42 |
| sdague | jd__: interesting | 13:42 |
| jd__ | it just seems damn slow for an unknown reason | 13:43 |
| mordred | sdague: I don't know that that will get us out of the hole, but it certainly needs the backporting :) | 13:43 |
| sdague | diskimage-builder is sneaking in pypi urls directly? - https://jenkins02.openstack.org/job/gate-requirements-install/200/console | 13:44 |
| dims | fungi, watching this log now. saw one 5 min pause already - https://jenkins02.openstack.org/job/gate-requirements-install/200/console | 13:45 |
| fungi | i don't see anything concerning about the slave where that job ran, though i don't have a great way to check i/o performance (since i don't have a good baseline to compare against), but other system resources are basically clear and no noticeable packet loss/latency for it either | 13:46 |
| fungi | i'll hop on that one and see what it's doing exactly | 13:46 |
| fungi | sdague: gate-requirements-install hits pypi.python.org directly because it needs to test combinations which aren't on our mirror yet | 13:47 |
| sdague | mordred: yeh, the backport would be a good thing. If that passes check I'll merge it | 13:47 |
| mordred | sdague: actually, I just amended it to add the pip install/upgrade stuff | 13:47 |
| sdague | fungi: ok, which means network burps will kill us, right? | 13:48 |
| sdague | which seems to be what's happening | 13:48 |
| fungi | sdague: absolutely | 13:48 |
| mordred | yup | 13:48 |
| sdague | so we can't unfuxor ourselves.... | 13:48 |
| mordred | sdague: what's the unfuxor we need? | 13:48 |
| fungi | sdague: well, i don't see network problems with the slave, but possibly the fastly cdn in front of pypi has issues, or something on the route to it | 13:48 |
| * mordred trying to catch up on everything | 13:49 | |
| sdague | mordred: we have to cap cliff in global requirements | 13:49 |
| *** dkliban has joined #openstack-infra | 13:49 | |
| sdague | no code has merged on any project since cliff 1.4.5 hit the mirrors | 13:49 |
| mordred | is that just because of the grenade errors? | 13:50 |
| jd__ | sad day | 13:50 |
| mordred | or are there others? | 13:50 |
| sdague | nope | 13:50 |
| sdague | also because cliff 1.4.5 requires pyparsing >= 2.0.1 | 13:50 |
| sdague | which isn't in requirements | 13:50 |
| mordred | oh for the love of | 13:50 |
| mordred | the pyparsing version thing is goingto be the death of me | 13:50 |
| dhellmann | this was supposed to be the fix for that | 13:51 |
| mordred | yeah, I know | 13:51 |
| dhellmann | however, I'm seriously considering cliff 2.0 not using cmd2 or pyparsing | 13:51 |
| mordred | so, we can't get the mirror to update with new pyparsing because of pypi fail | 13:51 |
| mordred | and we can't get the requirements cap in because of, well, pypi fail | 13:51 |
| sdague | right, exactly | 13:51 |
| dhellmann | the mirror update failure isn't causing the grenade failure, is it? | 13:52 |
| mordred | no. that's setuptools upgrade love | 13:52 |
| sdague | dhellmann: no, the grenade failure was the old setuptools issue | 13:52 |
| sdague | and I've landed a grenade workaround for that | 13:52 |
| jd__ | dhellmann: nor Python :] | 13:52 |
| sdague | which just installs cliff upfront | 13:52 |
| dhellmann | oh, is that the pip change? | 13:52 |
| dhellmann | oh | 13:52 |
| sdague | dhellmann: https://review.openstack.org/#/c/48032/ | 13:53 |
| sdague | that is a very tactical work around | 13:53 |
| dims | dhellmann, latest run with pyparsing>=2.0.1 problem - https://jenkins02.openstack.org/job/gate-nova-python26/4127/console | 13:53 |
| sdague | to be removed as soon as the rest of the system comes together | 13:53 |
| mordred | I think we need a smarter mirror script thing | 13:54 |
| dhellmann | dims: ok, that's the mirror update failure breaking devstack | 13:54 |
| *** nosnos has quit IRC | 13:54 | |
| dhellmann | when I run stable/grizzly devstack I get an error installing nova because of a conflict on requirements for pyparsing -- how does grenade get past *that*? | 13:55 |
| sdague | dhellmann: actually, that's the wedge | 13:55 |
| sdague | dhellmann: because we install cliff upfront at 1.4.4 | 13:55 |
| dhellmann | duh, ok | 13:55 |
| *** nosnos has joined #openstack-infra | 13:55 | |
| sdague | the cliff requirement is actually >= 1.4.3 so nothing else installs newer cliff | 13:55 |
| * dhellmann drinks more caffeine | 13:56 | |
| dims | dhellmann, :) | 13:56 |
| dhellmann | mordred: smarter how? | 13:56 |
| dhellmann | mordred: it might be easier to just set up a full pypi mirror, and have the restricted mirror script pull from that | 13:57 |
| mordred | dhellmann: one that can respond to such events as "we need package X in the mirror please" or "this is the _change_ in reqs" | 13:57 |
| mordred | dhellmann: oy | 13:57 |
| mordred | dhellmann: full pypi mirror is hard and doesn't really work because none of the full mirroring software is worth anything | 13:57 |
| mordred | dhellmann: because it all respects the external links bs | 13:57 |
| fungi | yeah, stracing the pip install process, it does actually seem to get what i think are 404 errors and then tries several x.pypi.p.o mirrors on downloads to no avail, before ending up in a futex_wait_private/egain (resource temporarily unavailable) loop for a while | 13:58 |
| *** nosnos has quit IRC | 13:58 | |
| dhellmann | mordred: ok, well, that can be fixed. I thought the issue was with timeouts, not the logic. | 13:58 |
| mordred | it's both | 13:58 |
| fungi | so i think those are the pauses we're seeing | 13:58 |
| fungi | no child processes that i can see either | 13:58 |
| mordred | what I'm saying about smarter mirror script | 13:58 |
| mordred | is that it would e nice to be able to put in better retry loops | 13:58 |
| *** dolphm has joined #openstack-infra | 13:58 | |
| mordred | fungi: wait - we're trying the mirrors? | 13:59 |
| fungi | mordred: pip seemed to be | 13:59 |
| mordred | in our mirror build? we should turn those off | 13:59 |
| fungi | i saw references to a.pypi.python.org and b.pypi.python.org in the strace | 13:59 |
| mordred | ok. well, that's at least one thing we can do | 13:59 |
| fungi | mordred: in the gate-requirements-install job | 13:59 |
| mordred | fungi: where is that job running? | 14:01 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Tweak gate-requirements-install https://review.openstack.org/48055 | 14:02 |
| *** thomasbiege has quit IRC | 14:03 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Tweak gate-requirements-install https://review.openstack.org/48055 | 14:03 |
| mordred | fungi: ^^ | 14:03 |
| dims | mordred, there's one running here now - https://jenkins01.openstack.org/job/gate-requirements-install/285/console | 14:03 |
| fungi | mordred: http://paste.openstack.org/show/47426/ | 14:03 |
| mordred | btw - I'm not sure we should even keep gate-requirements-install | 14:03 |
| sdague | mordred: that's basically covered now that we have the global requirements sync in devstack, right? | 14:04 |
| mordred | since we actually have the pbr integration test running a more complete version of this | 14:04 |
| mordred | yeah | 14:04 |
| fungi | didn't we discuss that it's basically redundant now that devstack uses the requirements list? | 14:04 |
| fungi | oh, and pbr yes | 14:04 |
| *** EmilienM has quit IRC | 14:04 | |
| fungi | and you and sdague also just said those things | 14:04 |
| *** hashar has quit IRC | 14:04 | |
| mordred | so it's just more places for pypi to bork on us | 14:04 |
| mordred | uhm | 14:05 |
| mordred | why is openstack/requirements not triggering the other requirements tests? | 14:05 |
| mordred | GAH | 14:05 |
| fungi | that seems like an oversight | 14:06 |
| sdague | so what triggers the mirror update? | 14:07 |
| mordred | oh - nevermind. looking at wrong file | 14:07 |
| *** EmilienM has joined #openstack-infra | 14:07 | |
| *** yaguang has joined #openstack-infra | 14:08 | |
| *** thomasbiege has joined #openstack-infra | 14:08 | |
| sdague | mordred: so... the devstack stable/grizzly backport is hitting the issue that pyparsing 2.0.1 isn't in our mirror | 14:09 |
| fungi | sdague: three things: a daily timer, changes merging to the requirements project, and pypi uploads of some of our library projects which other projects of ours depend upon | 14:09 |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Remove gate-requirements-install https://review.openstack.org/48058 | 14:09 |
| sdague | fungi: so how do you handle the propose case? | 14:10 |
| mordred | sdague: we might have to ninja approve the requirements change then | 14:10 |
| fungi | sdague: proposed changes to the requirements project don't use our mirror | 14:10 |
| mordred | if the requirements change to cap it is hitting issues | 14:10 |
| sdague | fungi: ... the devstack jobs seem to be | 14:10 |
| fungi | or at least aren't supposed to | 14:10 |
| *** rcleere has joined #openstack-infra | 14:11 | |
| fungi | i believe the mirror selection script is supposed to set it to use pypi.python.org when it's testing a requirements project change | 14:11 |
| sdague | mordred: yeh, before ninja approving things, I think we should figure out how the jenga blocks are currently stacked, because I'm afraid of us wedging things later | 14:11 |
| * fungi looks | 14:11 | |
| mordred | sdague: TOTALLY AGREE | 14:11 |
| sdague | fungi: ok, I can tell you that's definitely not happening | 14:12 |
| mordred | this seems like an overly borked pile of jenga | 14:12 |
| sdague | fungi: https://review.openstack.org/#/c/48014/ being the example | 14:12 |
| mrodden | ^ phrase of the day | 14:12 |
| uvirtbot | mrodden: Error: "phrase" is not a valid command. | 14:12 |
| mrodden | i have many "overly borked piles of jenga" to fix today... | 14:13 |
| sdague | aren't emergent systems fun.... | 14:13 |
| fungi | sdague: the idea, at least, is encoded at http://git.openstack.org/cgit/openstack-infra/config/tree/modules/jenkins/files/slave_scripts/select-mirror.sh#n35 | 14:13 |
| mordred | ok. quick question. I think we've got too many workarounds for me to remember what our ultimate end-state is | 14:13 |
| fungi | now to figure out why it's not doing that | 14:13 |
| mordred | sdague: in a perfect world, do we _want_ to be at pyparsing >= 2.0.1 | 14:13 |
| mordred | and/or, would force-adding pyparsing 2.0.1 to our mirror by hand unstick us | 14:14 |
| sdague | mordred: sure, though I guess is there a reason we can't be at >= 1.5.7? | 14:14 |
| *** thedodd has joined #openstack-infra | 14:14 | |
| mordred | well, I more mean - adding 2.0.1 by hand to our mirror would not be hard | 14:14 |
| fungi | in fact, it would be too easy, and would feel like cheating | 14:14 |
| mordred | if we think that would get us past the log jam in a way that doesn't involve landing interrelated patches | 14:14 |
| sdague | so.... the only issue is stable | 14:15 |
| mordred | ok. what's that issue | 14:15 |
| sdague | so on stable, where we don't have a single req list, if we force the mirror to 2.0.1, is that going to break us? | 14:16 |
| sdague | given that stable is using master clients | 14:16 |
| mordred | is stable capped? | 14:16 |
| sdague | no | 14:16 |
| sdague | well, it's scattered | 14:16 |
| dhellmann | I'm 75% to the point of not needing pyparsing in cliff, if that helps. | 14:16 |
| sdague | I don't think we have a cap | 14:16 |
| *** vipul has quit IRC | 14:16 | |
| *** Ghe_Holidays has quit IRC | 14:17 | |
| mordred | I want to die | 14:17 |
| sdague | mordred: but I want to understand one other thing first | 14:17 |
| sdague | why when I uncapped pyparsing, didn't that trigger mirror updates in the check side/ | 14:17 |
| sdague | ? | 14:17 |
| *** dcramer_ has joined #openstack-infra | 14:17 | |
| sdague | and if I bump it to >= 2.0.1 will it? | 14:17 |
| mriedem | hey guys, i'm seeing flake8 failures getting through the gate in nova | 14:18 |
| mriedem | https://bugs.launchpad.net/nova/+bug/1229753 | 14:18 |
| uvirtbot | Launchpad bug 1229753 in nova "Several flake8 errors in new xenserver code" [Undecided,New] | 14:18 |
| sdague | mriedem: it's going to need to wait until we get the rest of the gate fixed | 14:18 |
| mordred | sdague: mirror updates only happen in the post queue | 14:18 |
| mriedem | sdague: ok, didn't know there were current issues, just joined. i'll just work on fixing the nova bug for now (whack a mole i guess) | 14:19 |
| mordred | mriedem: if it's xenserver code, is it in files that might need requirements that aren't normally instlaled? | 14:19 |
| mriedem | mordred: nope, pure flake8 failures | 14:19 |
| mriedem | fails locally for me | 14:19 |
| mordred | mriedem: k. will come back to it later | 14:19 |
| fungi | okay, so it doesn't look like any of our devstack jobs use select_mirror.sh, unless i'm missing it somewhere | 14:19 |
| mordred | fungi: facepalm | 14:20 |
| fungi | er, seelct-mirror.sh | 14:20 |
| mordred | fungi: wait - don't we do that in d-b? | 14:20 |
| mordred | d-g? | 14:20 |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: uncap pyparsing requirement https://review.openstack.org/48014 | 14:20 |
| fungi | yeah, looks like devstack-vm-gate-wrap.sh hard-sets mirrors | 14:20 |
| mordred | fungi: https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L188 | 14:21 |
| mordred | fungi: select_mirror.sh should be installed on d-g nodes, because they're slaves, yeah? | 14:21 |
| mordred | fungi: how about just having d-g call select-mirror.sh | 14:21 |
| mordred | so that we don't have duplicate logic | 14:22 |
| fungi | mordred: probably would make sense, yes | 14:22 |
| sdague | so, we should probably figure out how to get that status bot to update folks on the issue | 14:22 |
| mordred | oh - fungi and I can do that | 14:22 |
| fungi | sdague: nothing to figure out, but what's the new status since i last updated it? | 14:22 |
| * mordred always screws it up though | 14:22 | |
| mordred | sdague: fwiw, I've got a command locked and loaded to force in pyparsing 2.0.1 if we decide we want that | 14:23 |
| fungi | "changes being applied to unstick pyparsing dependency conflict in the gate, eta 1600 utc"? | 14:24 |
| sdague | maybe, we don't really have an eta yet, as we've not quite managed to figure out a version that gets us through | 14:24 |
| sdague | mordred: can you just jam that into the mirror? | 14:24 |
| mordred | sdague: yes | 14:24 |
| sdague | ok, lets do that | 14:25 |
| mordred | yeah - we've kind of hitting up against the systemic issue which is reuqirements in stable branches | 14:25 |
| sdague | yeh, well, I get voted down every time I want to cap things :) | 14:25 |
| mordred | sdague: this will fix that | 14:25 |
| fungi | sdague: we're going to cap stable branches rsn | 14:25 |
| mordred | ok. mirror is updated | 14:25 |
| *** markmcclain has joined #openstack-infra | 14:25 | |
| *** adalbas has quit IRC | 14:25 | |
| *** ryanpetrello has joined #openstack-infra | 14:26 | |
| mordred | sdague: retrying things that want pyparsing 2.0.1 should be better now | 14:26 |
| sdague | Alex_Gaynor: hey, +Aing changes that don't pass check don't actually help :P | 14:26 |
| *** markmcclain has quit IRC | 14:26 | |
| Alex_Gaynor | sdague: Ugh. Apparently I shouldn't review before breakfast, I misread that as passing check :/ Sorry 'bout that. | 14:26 |
| *** dolphm has quit IRC | 14:26 | |
| sdague | yeh, no worries | 14:27 |
| *** markmcclain has joined #openstack-infra | 14:27 | |
| *** julim has quit IRC | 14:27 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: change pyparsing to >= 2.0.1 https://review.openstack.org/48014 | 14:28 |
| *** markmc has quit IRC | 14:28 | |
| *** changbl has joined #openstack-infra | 14:28 | |
| *** yaguang has quit IRC | 14:28 | |
| openstackgerrit | Sean Dague proposed a change to openstack/requirements: cap cliff to < 1.4.5 https://review.openstack.org/48019 | 14:29 |
| sdague | this next change is gratuitous to kick it from the gate | 14:29 |
| sdague | or, last change I should say | 14:29 |
| *** markmcclain has quit IRC | 14:29 | |
| sdague | ok, so https://review.openstack.org/#/c/48014/ is where we are betting our money on | 14:30 |
| *** markmcclain has joined #openstack-infra | 14:31 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/devstack-gate: Use select-mirror in devstack-gate https://review.openstack.org/48064 | 14:32 |
| mordred | fungi: ^^ | 14:32 |
| dims | 2013-09-24 14:31:46.065 | 2013-09-24 14:31:46 Downloading/unpacking pyparsing>=2.0.1 (from cliff>=1.4.3->python-neutronclient==2.3.1.6.g89936fa) | 14:32 |
| dims | 2013-09-24 14:31:46.065 | 2013-09-24 14:31:46 Storing download in cache at /var/cache/pip/http%3A%2F%2Fpypi.openstack.org%2Fopenstack%2Fpyparsing%2Fpyparsing-2.0.1.tar.gz | 14:32 |
| mordred | woot. the force add worked :) | 14:32 |
| *** dizquierdo has joined #openstack-infra | 14:35 | |
| *** datsun180b has joined #openstack-infra | 14:35 | |
| *** DennyZha` has joined #openstack-infra | 14:36 | |
| *** DennyZhang has quit IRC | 14:36 | |
| dims | looks like other runs in 48014,3 are past the pyparsing/cliff problem, watching https://jenkins01.openstack.org/job/gate-requirements-install/288/console | 14:38 |
| *** thomasbiege has quit IRC | 14:38 | |
| *** vipul has joined #openstack-infra | 14:39 | |
| *** Ghe_Holidays has joined #openstack-infra | 14:40 | |
| fungi | mordred: hah, i was just in the process of figuring out how to write that | 14:40 |
| sdague | mordred: so we should create a bug to hang the rechecks off of | 14:40 |
| portante | folks, there are a lot of pent up jobs that at least the swift team would like to recheck, what is the best way to approach this? | 14:40 |
| sdague | portante: they aren't going to pass | 14:40 |
| sdague | please wait until we get the gate stable again | 14:41 |
| portante | certainly, just trying to think ahead to when they will be stable and how we should consider this | 14:41 |
| portante | sdague: thanks, and we are not taking any action until you give the go-ahead | 14:41 |
| sdague | yep, actually, I have the beginnings of a script for that, was just trying to figure out where it should live | 14:42 |
| *** dkliban has quit IRC | 14:42 | |
| *** salv-orlando has joined #openstack-infra | 14:42 | |
| dims | sdague, fungi - couple of 2 min pauses so far... | 14:42 |
| *** MoXxXoM has joined #openstack-infra | 14:43 | |
| *** dolphm has joined #openstack-infra | 14:44 | |
| dims | done | 14:44 |
| *** adalbas has joined #openstack-infra | 14:46 | |
| sdague | mordred: so while we wait for that requirements change to pass, gerrtlib release? | 14:46 |
| jeblair | mordred, fungi: good morning | 14:46 |
| mordred | morning jeblair ! | 14:46 |
| mordred | jeblair: welcome to a lovely morning of chaos | 14:46 |
| fungi | jeblair: morning | 14:47 |
| sdague | because then I could find a place to put my recheck script for gerrit | 14:47 |
| sdague | which people might appreciate today | 14:47 |
| * mordred is in favor of both a release and your script | 14:47 | |
| sdague | but it needs fixes in unreleased gerritlib | 14:47 |
| * mordred defers to jeblair - jeblair uses gerritlib in more things | 14:47 | |
| jeblair | mordred: while i do that, can you give me a quick summary? | 14:48 |
| fungi | jeblair: new cliff pinned a minimum version of pyparsing greater than our specified maximum version in requirements lists | 14:48 |
| mordred | jeblair: the mirror hit some intractable requirements issues exacerbated by pypi upstream going batty | 14:48 |
| *** davidhadas has joined #openstack-infra | 14:48 | |
| * dhellmann hangs head in shame | 14:48 | |
| mordred | jeblair: so the mirror update scripts couldn't get the mirror updated to contain the new version | 14:49 |
| fungi | and yes, slow pypi.p.o performance | 14:49 |
| sdague | also, clients on master, and servers on stable for grizzly devstack testing, created a fun grenade paradox | 14:49 |
| mordred | also - we discovered that d-g was not setting mirror == upstream for requirements changes | 14:49 |
| sdague | with the same issue | 14:49 |
| mordred | yup | 14:49 |
| sdague | burned down, fell over, then sank into a swamp | 14:49 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Do not build Network Admin Guide anymore https://review.openstack.org/48070 | 14:49 |
| mordred | AND - lack of stable/grizzly requirements pinning/coordination | 14:49 |
| sdague | but the 4th castle, that one stayed up laddy | 14:49 |
| mordred | :) | 14:50 |
| dhellmann | so we're working again? | 14:50 |
| fungi | enough of these events and we'll have the strongest castle in these parts | 14:50 |
| dhellmann | or at least the shallowest swamp | 14:50 |
| fungi | dhellmann: requirements change needs to finish landing first, i believe | 14:50 |
| sdague | dhellmann: so... requirements is in check | 14:50 |
| sdague | but looks promissing | 14:50 |
| sdague | then that goes to gate | 14:50 |
| dhellmann | ok, I'll hang out a little longer | 14:51 |
| mordred | things we did so far - forced pyparsing 2.0.1 into the mirror by copying it there by hand, proposed changes to backport pip changes from master devstack to stable/grizzly devstack, change to devstack-gate for using select-mirror.sh | 14:51 |
| sdague | then the spice will flow | 14:51 |
| fungi | our castle is built on a submerged foundation of previous crumbled castles | 14:51 |
| dims | 48014,3 - 2 devstack and 1 grenade run still going on | 14:51 |
| dhellmann | I ran into an issue with tab completion when I tried to remove cmd2/pyparsing from cliff, but I'll keep working on that. | 14:51 |
| dhellmann | and, sdague, that will be cliff 2.0 :-) | 14:51 |
| * portante but the third castle ... | 14:51 | |
| jeblair | sdague: wow, i didn't see https://review.openstack.org/#/c/47821/ | 14:52 |
| jeblair | sdague: i'm really curious about that, because your commit message basically reduces to "zuul does not work". | 14:53 |
| sdague | jeblair: so.... what were you using for ids? | 14:54 |
| jeblair | 1234,1 | 14:54 |
| *** reed has joined #openstack-infra | 14:54 | |
| sdague | so the gerrit -h says the id needs to come first | 14:54 |
| *** dkliban has joined #openstack-infra | 14:55 | |
| sdague | and I was having some issues with idempotent ids, and weird errors | 14:55 |
| sdague | which I may have worked around a different way | 14:55 |
| sdague | so the patch was basically just making it be what the gerrit help expected | 14:56 |
| fungi | when i reviewed that, i mainly +2'd it based on the parameter order matching other places where we were manually constructing gerrit review calls via the api, and also the order shown in the api docs | 14:56 |
| *** thomasbiege has joined #openstack-infra | 14:56 | |
| sdague | let me generate the fail case once the rest of this calmed down | 14:56 |
| jeblair | sdague: you've tested it, with supplying --message, and other params? | 14:57 |
| * mordred has to pop onto a call - but jeblair is awake now | 14:57 | |
| dims | sdague, 48014,3 - all green | 14:57 |
| *** julim has joined #openstack-infra | 14:57 | |
| mordred | jeblair, sdague: is sdague using gerrit 2.6? | 14:58 |
| sdague | no, this was against upstream | 14:58 |
| sdague | sorry, agains r.o.o | 14:58 |
| *** SergeyLukjanov has quit IRC | 14:58 | |
| sdague | ok... back to the gate wedge | 14:58 |
| mordred | ok. just making sure | 14:58 |
| sdague | requirements reviewers, please +2 / +A - https://review.openstack.org/#/c/48014/ | 14:59 |
| *** SergeyLukjanov has joined #openstack-infra | 14:59 | |
| *** dolphm has quit IRC | 15:00 | |
| mordred | looking | 15:00 |
| mordred | wow. that was quick | 15:00 |
| *** yportnova has quit IRC | 15:00 | |
| *** yportnova has joined #openstack-infra | 15:00 | |
| jeblair | sdague: ok, i mostly wanted to point out that i disagreed with the commit message's suggestion that the current implementation does not work. if that changes any of your thinking, i'll stand by. if you're sure that the new version works and are still ready for me to tag, i will do so. | 15:01 |
| *** markmc has joined #openstack-infra | 15:02 | |
| *** pcrews has joined #openstack-infra | 15:02 | |
| sdague | jeblair: so this - http://paste.openstack.org/show/47428/ (which is the first rev of my recheck script) | 15:03 |
| sdague | worked fine with those changes yesterday on a local set of changes | 15:03 |
| *** datsun180b has quit IRC | 15:05 | |
| *** datsun180b has joined #openstack-infra | 15:06 | |
| *** ruhe has joined #openstack-infra | 15:06 | |
| *** pcrews has left #openstack-infra | 15:07 | |
| * SpamapS reads the backscroll in fascination | 15:08 | |
| SpamapS | mordred: indeed my view of "that is what broke, so run it again" is entirely simplistic. Though I don't understand why zuul has to run a periodic job. :-P | 15:08 |
| jeblair | sdague: so you're happy? :) | 15:08 |
| sdague | yep | 15:08 |
| *** senk has joined #openstack-infra | 15:08 | |
| jeblair | SpamapS: who else would run it? | 15:08 |
| dhellmann | sdague: the gate jobs look happy, are we unstuck? | 15:10 |
| jeblair | sdague: tag pushed | 15:10 |
| *** prad has joined #openstack-infra | 15:11 | |
| SpamapS | jeblair: some moron who can run things in jenkins | 15:11 |
| mordred | SpamapS: we don't use jenkins to initiate jobs | 15:12 |
| mordred | SpamapS: it's too stupid | 15:12 |
| jeblair | SpamapS: are you volunteering? :) | 15:12 |
| mordred | SpamapS: we also have 3 jenkinses | 15:12 |
| *** julim has quit IRC | 15:12 | |
| *** morganfainberg has quit IRC | 15:13 | |
| SpamapS | I was more thinking that there's a need for a big red panic button than there is a need to let people run jenkins jobs. | 15:15 |
| mordred | SpamapS: I'm not sure what you mean? (not being snarky, want more thoughts) | 15:16 |
| *** morganfainberg has joined #openstack-infra | 15:16 | |
| sdague | dhellmann: yeh, gate should be good | 15:18 |
| *** wchrisj has joined #openstack-infra | 15:18 | |
| dhellmann | sdague: cool, I'm going to head to the office then | 15:18 |
| *** UtahDave has joined #openstack-infra | 15:18 | |
| *** ruhe has quit IRC | 15:19 | |
| jeblair | SpamapS: feel free to work on statusbot. i'm swamped, and updating it is not my highest priority. | 15:19 |
| mordred | SpamapS: ah - you mean a way for not-infra to hit the button and let people know that there may be an issue? | 15:20 |
| SpamapS | OTP | 15:20 |
| jeblair | SpamapS: statusbot updates a wiki page; and we can add banners to status.o.o and gerrit, etc, using some javascript. that bit just needs to be written. | 15:21 |
| jeblair | fungi, mordred: speaking of which, should we update status? | 15:22 |
| mordred | jeblair: yeah. I think we've back out of the woods, no? | 15:22 |
| *** senk has quit IRC | 15:25 | |
| fungi | did we have a bug number for rechecks/reverifies? | 15:27 |
| *** ruhe has joined #openstack-infra | 15:27 | |
| SpamapS | ok back | 15:28 |
| SpamapS | ok so "the big red button" is not a mechanical thing | 15:28 |
| SpamapS | It is a wiki page specifically designed to aid in dealing with any crisis. | 15:29 |
| SpamapS | it is in the topic of this channel, maybe others, linked to in gerrit, jenkins, etc. etc. | 15:29 |
| fungi | sdague: did we have (or need) a bug number for rechecks/reverifies? | 15:30 |
| sdague | no, not yet | 15:30 |
| sdague | sorry, otp, so slow to respond | 15:31 |
| *** rnirmal has joined #openstack-infra | 15:31 | |
| fungi | just didn't know if i should say recheck/reverify no bug in the status ok notice | 15:31 |
| *** jeremyb has quit IRC | 15:31 | |
| *** jeremyb has joined #openstack-infra | 15:31 | |
| sdague | yeh, we should have a bug | 15:31 |
| sdague | to track it | 15:31 |
| sdague | jeblair: you have opinions on where recheck tooling should good? | 15:32 |
| sdague | is that in config, or we have another repo that makes sense? I'd rather not spin up another repo if I can help it | 15:32 |
| *** mrodden has quit IRC | 15:33 | |
| *** markmcclain has quit IRC | 15:33 | |
| *** fbo is now known as fbo_away | 15:33 | |
| jeblair | sdague: this is for rechecking old changes? | 15:34 |
| sdague | jeblair: yes | 15:34 |
| jeblair | sdague: i think zuul should do that, actually | 15:35 |
| sdague | ok, so a set of clis in the zuul repo? | 15:35 |
| jeblair | sdague: i thought we had a bug about that, but i can't find it | 15:35 |
| sdague | I can do that | 15:35 |
| jeblair | sdague: er | 15:35 |
| jeblair | sdague: i mean, zuul should have a trigger that can do that; eg, it shouldn't be external | 15:36 |
| sdague | ok, well there are times when having external tooling for this is useful | 15:36 |
| jeblair | (we shouldn't need to leave comment spam for something like that) | 15:36 |
| sdague | like selecting on criteria that you want to drive rechecks, because you know it will be an issue | 15:36 |
| jeblair | sdague: yes, that i agree with | 15:36 |
| jeblair | sdague: like "we need to recheck all the keystone changes because ...." or "we need to recheck all the approved changes because..." | 15:37 |
| jeblair | sdague: but the "auto-recheck all changes older than X days" should be built into zuul | 15:37 |
| jeblair | sdague: i'd put it in config/tools for starters | 15:38 |
| jeblair | ("recheck no bug" is something specific to our configuration, so it should be close to that) | 15:39 |
| *** psedlak is now known as psedlak|afk | 15:39 | |
| *** dcramer_ has quit IRC | 15:39 | |
| sdague | jeblair: yeh, well we could take logic back into zuul | 15:39 |
| sdague | anyway, I wanted a prooving ground | 15:40 |
| jeblair | sdague: yeah, i think the nice way to do it would be to add it to the gerrit trigger in zuul as a pseudo event, so that we can add it as a trigger to the existing check pipeline | 15:41 |
| *** senk has joined #openstack-infra | 15:42 | |
| *** datsun180b has quit IRC | 15:43 | |
| *** datsun180b has joined #openstack-infra | 15:44 | |
| jeblair | probably also copying some of the logic from the timer trigger | 15:44 |
| sdague | yep | 15:44 |
| jeblair | it could also be its own trigger; and maybe if it _used_ the gerrit trigger (in the way that the gerrit reporter uses it), it wouldn't need to much code. actually, i'm starting to like that idea. | 15:45 |
| sdague | ok, so I'm still not sure where we land with where I should start hacking on cli tools for this :) | 15:45 |
| *** senk has quit IRC | 15:45 | |
| jeblair | need to _copy_ much code | 15:45 |
| jeblair | sdague: infra/config/tools | 15:45 |
| *** thomasbiege has quit IRC | 15:45 | |
| sdague | cool | 15:45 |
| jeblair | sdague: because "recheck no bug" is an openstack-ism, not a zuul-ism. | 15:46 |
| sdague | yep | 15:46 |
| *** zehicle_at_dell has joined #openstack-infra | 15:46 | |
| fungi | #status ok changes seem to be making it through the gate once more, and so it should be safe to "recheck bug 1229797" or "reverify bug 1229797" on affected changes as needed | 15:48 |
| openstackstatus | NOTICE: changes seem to be making it through the gate once more, and so it should be safe to "recheck bug 1229797" or "reverify bug 1229797" on affected changes as needed | 15:48 |
| *** ChanServ changes topic to "Discussion of OpenStack Project Infrastructure | Docs http://ci.openstack.org/ | Bugs https://launchpad.net/openstack-ci | Code https://git.openstack.org/cgit/openstack-infra/" | 15:48 | |
| uvirtbot | Launchpad bug 1229797 in openstack-ci "cliff 1.4.5 & pyparsing 2.0.1 cause sadness in the gate" [Undecided,New] https://launchpad.net/bugs/1229797 | 15:48 |
| fungi | duelling bots | 15:48 |
| jeblair | wham bam spam | 15:48 |
| *** mrodden has joined #openstack-infra | 15:48 | |
| *** senk has joined #openstack-infra | 15:48 | |
| SpamapS | maps? | 15:49 |
| * fungi has maps to the stars, but no maps to the spam | 15:49 | |
| *** senk has quit IRC | 15:50 | |
| *** atiwari has joined #openstack-infra | 15:52 | |
| SpamapS | fungi: follow the maps to the stars, then follow the scent of salted meat | 15:54 |
| SpamapS | Encino is just over the hill from Bel Aire.. you surely can smell the spam from there. :) | 15:54 |
| *** dcramer_ has joined #openstack-infra | 15:55 | |
| *** boris-42 has quit IRC | 15:55 | |
| jeblair | clarkb, mordred, lifeless: reviewing the triple-o test plan; overall i believe i understand it and agree in principle. when i try to think about the gritty details, i see things like one part says "Jenkins spawns a gate node instance" while another part says "make new nodepool base image definition"... | 15:57 |
| jeblair | clarkb, mordred, lifeless: is it worth my getting into that level of detail at this point, or should i just think in broad strokes? | 15:58 |
| mordred | jeblair: I think broad strokes - I believe that some of those things are things where we could go either way and might want to audible on it as we start moving forward | 15:58 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add first pass recheck script https://review.openstack.org/48087 | 15:59 |
| sdague | portante: so you guys should be good to recheck stuff now, sorry for not pinging you once the gate cleared | 15:59 |
| mordred | (like, we talked about re-using d-g nodes in seattle, then I realized we _could_ have an additional nodepool config script for these nodes) | 15:59 |
| *** markmc has quit IRC | 15:59 | |
| jeblair | ok. i think i understand then. | 16:00 |
| portante | no problem, thanks | 16:01 |
| dkranz | jeblair: Is the "triple-o test plan" somewhere I can look at it? | 16:02 |
| sdague | portante: actually we hit a reset on neutron | 16:02 |
| portante | what bug number should we use? | 16:02 |
| sdague | so the rechecks won't work yet | 16:02 |
| sdague | another 40 minutes probably | 16:02 |
| portante | okay | 16:02 |
| jeblair | dkranz: see email to openstack-infra list | 16:02 |
| sdague | https://bugs.launchpad.net/openstack-ci/+bug/1229797 | 16:02 |
| uvirtbot | Launchpad bug 1229797 in openstack-ci "cliff 1.4.5 & pyparsing 2.0.1 cause sadness in the gate" [Undecided,New] | 16:02 |
| *** senk has joined #openstack-infra | 16:02 | |
| sdague | that's the recheck bug for this | 16:02 |
| portante | thanks | 16:02 |
| dkranz | jeblair: OK, thanks. Behind on email :) | 16:02 |
| *** matty_dubs is now known as matty_dubs|lunch | 16:02 | |
| portante | notmyname: see above for recheck bug number to use | 16:02 |
| jeblair | dkranz: (it's at the level of "how do we actually run tests on something that needs bare metal" rather than "exactly what tests do we run") | 16:03 |
| *** gyee has joined #openstack-infra | 16:03 | |
| jeblair | dkranz: Subject: [OpenStack-Infra] Plan for testing nova baremetal and TripleO | 16:03 |
| jeblair | dkranz: i believe it will also be an infra meeting topic today | 16:03 |
| dkranz | jeblair: Great, thanks. | 16:03 |
| sdague | ok, off to lunch, but as soon as 48014,3 merges, rechecks should work again | 16:04 |
| *** anteaya has joined #openstack-infra | 16:04 | |
| *** rpodolyaka has left #openstack-infra | 16:05 | |
| *** rpodolyaka has quit IRC | 16:05 | |
| *** thomasbiege has joined #openstack-infra | 16:06 | |
| *** thomasbiege has quit IRC | 16:07 | |
| *** cody-somerville has quit IRC | 16:08 | |
| *** Ryan_Lane has joined #openstack-infra | 16:08 | |
| *** cody-somerville has joined #openstack-infra | 16:09 | |
| fungi | mordred: as an upshot of the debian tarball repacking conversation yesterday, what do you think about a configurable/cli option to pbr which makes changelogs like this? (reduces the nova changelog size by 75%) http://paste.openstack.org/show/47438 | 16:10 |
| *** nijaba_ is now known as nijaba | 16:11 | |
| fungi | not sure if anyone would find it useful, but i'm planning to do something similar for one of my non-openstack projects | 16:12 |
| *** boris-42 has joined #openstack-infra | 16:12 | |
| *** senk has quit IRC | 16:13 | |
| fungi | clearly it loses the full commit messages from every commit, but maybe it still captures enough to make it viable for a changelog | 16:13 |
| *** ryanpetrello has quit IRC | 16:13 | |
| *** kpavel has quit IRC | 16:14 | |
| *** zehicle has quit IRC | 16:14 | |
| jeblair | fungi: pretty | 16:14 |
| fungi | and also, i have to agree that a >5mib changelog in nova tarballs is kinda crazy | 16:14 |
| *** nijaba has quit IRC | 16:14 | |
| SpamapS | If I wanted to gather my thoughts on a "dealing with crisis" suggestion for infra, what ML shoudl I send that too? -dev? | 16:14 |
| mordred | fungi: I like it | 16:15 |
| *** tvb has quit IRC | 16:15 | |
| fungi | SpamapS: possibly either -dev or -infra depending on the kind of input you're looking for | 16:15 |
| mordred | fungi: and we're realy just using --gnu for no good reason at all | 16:15 |
| mordred | fungi: so I will accept that patch | 16:15 |
| fungi | mordred: well, the decorations (rst style) are mostly irrelevant to my point, which is wondering whether git shortlog grouped by reverse-ordering tags matching ^[0-9\.]\+$ are a sane approach | 16:16 |
| *** nijaba has joined #openstack-infra | 16:16 | |
| *** nijaba has quit IRC | 16:16 | |
| *** nijaba has joined #openstack-infra | 16:16 | |
| fungi | it could be re-massaged into something which looks like a traditional gnu changelog format without much effort | 16:17 |
| jd__ | fungi: that changelog looks pretty good | 16:18 |
| jd__ | I've been afk for a couple of hours and don't have the full backlog; what's the status on gate? | 16:18 |
| jeblair | jd__: https://wiki.openstack.org/wiki/Infrastructure_Status | 16:19 |
| fungi | mordred: also curious whether the shortshas and patch counts are wanted/useless, if it should not batch up the prerelease tags together into the releases covering them but instead break them out independently, et cetera | 16:19 |
| jd__ | jeblair: thanks :) | 16:19 |
| *** senk has joined #openstack-infra | 16:20 | |
| fungi | could also be made to treat unsigned tags the same way as prerelease tags and roll them into the next signed tag | 16:20 |
| mordred | fungi: I think it'sa great idea - I'm not sure it's _important_ to do it | 16:21 |
| mordred | fungi: as I really don't give a crap at all about 5M of changelog in a tarball - it's 2014. HOWEVER, I do think it's an improvement, so if you (or someone else) does want to make that prettier/more compact/better, I'm 100% all in for it | 16:22 |
| fungi | yup, agreed. was mainly just interesting to consider what could be done with the git log features available | 16:22 |
| mordred | ++ | 16:22 |
| jeblair | fungi: i'm going to check in my current atc code to infra/config/tools/atc | 16:22 |
| *** markmcclain has joined #openstack-infra | 16:23 | |
| mordred | jeblair: ++ | 16:23 |
| fungi | jeblair: okay, reed you're ready for us to generate the list immediately right? | 16:23 |
| *** dcramer_ has quit IRC | 16:23 | |
| jd__ | mordred: don't want to kill your enthusiasm, but it's still 2013 last time I checked | 16:23 |
| *** ryanpetrello has joined #openstack-infra | 16:23 | |
| fungi | mordred has circled the globe so many times he's actually communicating with us from 2014 | 16:24 |
| mordred | jd__: time has no meanning for me | 16:24 |
| fungi | time is an illusion | 16:25 |
| fungi | lunch time doubly so | 16:25 |
| fungi | speaking of which | 16:25 |
| * fungi is missing, presumed fed | 16:26 | |
| *** therve has quit IRC | 16:26 | |
| reed | fungi, yes | 16:29 |
| openstackgerrit | Flavio Percoco proposed a change to openstack-infra/config: Make PY3K a voting for python-marconiclient https://review.openstack.org/48089 | 16:30 |
| clarkb | morning | 16:30 |
| *** therve has joined #openstack-infra | 16:30 | |
| *** morganfainberg has quit IRC | 16:31 | |
| jeblair | clarkb, mordred: is there a heat plan i should review before the meeting? (I don't see an email about it) | 16:32 |
| jeblair | clarkb: good morning! | 16:32 |
| openstackgerrit | A change was merged to openstack/requirements: change pyparsing to >= 2.0.1 https://review.openstack.org/48014 | 16:32 |
| *** dolphm has joined #openstack-infra | 16:33 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/config: Initial checkin of ATC scripts https://review.openstack.org/48091 | 16:33 |
| jeblair | fungi: ^ it may also need the project list updated | 16:34 |
| mordred | jeblair: I think clarkb was going to work on writing up notes - the biggest todo-list task is handling caching of things needed by dib in the d-g caching process | 16:35 |
| mordred | jeblair: which may involve some d-g scripting, and also some additional features to dib | 16:35 |
| *** svarnau has joined #openstack-infra | 16:35 | |
| *** dcramer_ has joined #openstack-infra | 16:36 | |
| *** morganfainberg has joined #openstack-infra | 16:37 | |
| *** senk has quit IRC | 16:37 | |
| fungi | jeblair: okay, having a look | 16:38 |
| fungi | reed: thanks | 16:38 |
| clarkb | jeblair: yes I started an etherpad https://etherpad.openstack.org/testing-heat-trove-with-dib | 16:38 |
| clarkb | mordred: ^ | 16:39 |
| mordred | ossum | 16:39 |
| *** hashar has joined #openstack-infra | 16:40 | |
| *** hashar has quit IRC | 16:41 | |
| *** svarnau has joined #openstack-infra | 16:42 | |
| *** zehicle_at_dell has quit IRC | 16:44 | |
| *** dcramer_ has quit IRC | 16:46 | |
| *** boris-42 has quit IRC | 16:46 | |
| *** reed has quit IRC | 16:47 | |
| *** svarnau_ has joined #openstack-infra | 16:47 | |
| *** marun has joined #openstack-infra | 16:47 | |
| *** thomasbiege has joined #openstack-infra | 16:48 | |
| *** ndipanov is now known as ndipanov_gone | 16:49 | |
| *** svarnau has quit IRC | 16:50 | |
| clarkb | chmouel: I don't think your question was answered. That number is the first 7 characters of the hex representation of a 128bit random uuid | 16:53 |
| *** EmilienM has quit IRC | 16:53 | |
| chmouel | clarkb: ah thanks | 16:53 |
| clarkb | chmouel: each job is assigned a random uuid by zuul/jenkins and we use that instead of job numbers since we have multiple jenkins masters now | 16:53 |
| *** matty_dubs|lunch is now known as matty_dubs | 16:54 | |
| clarkb | (so job numbers can overlap now) | 16:54 |
| chmouel | clarkb: is there any way to get that info somewhere i.e: api | 16:54 |
| *** SergeyLukjanov has quit IRC | 16:54 | |
| *** EmilienM has joined #openstack-infra | 16:54 | |
| clarkb | chmouel: you might be able to get it out of jenkins if you query specific job info | 16:55 |
| *** tkammer has quit IRC | 16:59 | |
| chmouel | clarkb: i was doing that ssh -x -p 29418 review.openstack.org 'gerrit query --format=JSON --current-patch-set change:$CHANGE_ID' | 16:59 |
| chmouel | is there some other arguments to add | 16:59 |
| clarkb | chmouel: no, you are talking to gerrit there and it has no knowledge of the zuul job uuids | 17:00 |
| chmouel | clarkb: oh yeah, so does jenkins has an api to query those things? | 17:01 |
| clarkb | chmouel: jenkins has an API, it isn't well documented (or at least I have never been able to find good docs) so I am not sure what exactly it is able to do | 17:01 |
| *** dcramer_ has joined #openstack-infra | 17:01 | |
| clarkb | chmouel: is there something specific you are trying to accomplisH/ | 17:01 |
| clarkb | might be easier to think in terms of goals rather than capability | 17:02 |
| chmouel | just getting a link for the latest job to console.html.gz | 17:02 |
| chmouel | sure i can screenscrap | 17:02 |
| clarkb | didn't someone write that tool and post it to -dev? | 17:02 |
| chmouel | git-os-job? | 17:02 |
| chmouel | AFAICS it has different purpose | 17:03 |
| clarkb | maybe? that sounds familiar | 17:03 |
| clarkb | ah | 17:03 |
| chmouel | it goes to things like that http://logs.openstack.org/5a/5a5023bea0fad8e0c1461fe5763d3befc5bde514/ | 17:03 |
| chmouel | but i want to go there http://logs.openstack.org/47/45447/9/check/gate-tempest-devstack-vm-full/1f9814e/ | 17:03 |
| chmouel | :) | 17:03 |
| clarkb | ya so it is looking at the post merge jobs not premerge | 17:04 |
| *** dolphm has quit IRC | 17:04 | |
| jeblair | chmouel: you could go to http://logs.openstack.org/47/45447/9/check/gate-tempest-devstack-vm-full/ and use the index page there to decide to fetch 1f9814e/ | 17:04 |
| clarkb | chmouel: or get the comments from jenkins for the most recent patchset and do some minimal scraping there | 17:04 |
| jeblair | _slightly_ better than scraping gerrit | 17:04 |
| *** dkliban has quit IRC | 17:04 | |
| jeblair | maybe not better. just different. :) | 17:05 |
| chmouel | jeblair: yeah i was kind of avoiding that | 17:05 |
| chmouel | since I would want to get the latest | 17:05 |
| chmouel | but i guess this is what i would need to do | 17:05 |
| chmouel | and do date comparaison etc.. | 17:05 |
| jeblair | chmouel: it was easy when we had a stateful system with a single point of failure; it gets harder with stateless distributed systems. uuids everywhere. :/ | 17:06 |
| *** tvb has joined #openstack-infra | 17:06 | |
| *** tvb has joined #openstack-infra | 17:06 | |
| jeblair | chmouel: i think jog0 recently wrote something where he took clarkb's approach and scraped the jenkins comment | 17:06 |
| chmouel | jeblair: make sense | 17:06 |
| fungi | for that matter, logs.o.o doesn't really even know which one ran most recently, just which one got uploaded most recently (though that's probably close enough) | 17:07 |
| *** reed has joined #openstack-infra | 17:07 | |
| jeblair | fungi: true. that probably makes the gerrit/jenkins comment scraping slightly more correct. also, the synchronization is better (no race condition for partial uploads) | 17:07 |
| * fungi nods | 17:09 | |
| chmouel | fwiw this is getting me the latest curl -s 'http://logs.openstack.org/47/45447/9/check/gate-tempest-devstack-vm-full/?C=M;O=D' | sed -n '/folder.gif/ { s,.*a href="\([^/]*\)/.*,\1,;p; q;}' | 17:09 |
| fungi | however, if we end up moving logs into object storage and writing an index front-end, don't count on it being 100% backwards compatible to apache mod_autoindex | 17:10 |
| pleia2 | clarkb: when you have a moment, looking into replying to bug 1226855 and trying to figure out a sane way to do this strace if you have any thoughts | 17:10 |
| uvirtbot | Launchpad bug 1226855 in lxc "Cannot use open-iscsi inside LXC container" [Undecided,Incomplete] https://launchpad.net/bugs/1226855 | 17:10 |
| *** dizquierdo has quit IRC | 17:10 | |
| *** thomasbiege has quit IRC | 17:10 | |
| *** zehicle has joined #openstack-infra | 17:11 | |
| clarkb | pleia2: I think to do that you will want to start iscsid, connect strace to it, then run the iscsiadm command which should cause it to fail | 17:12 |
| pleia2 | clarkb: right, so I think the trouble with running that the command alone is that the baremetal target doens't exist so it may fail in a different way | 17:13 |
| *** ruhe has quit IRC | 17:13 | |
| clarkb | pleia2: I think the target is failing to exist as a result of this bug | 17:14 |
| clarkb | pleia2: so we should be able to capture it with strace (I could be missing something important though, far too much going on) | 17:14 |
| chmouel | fungi: yeah it's screenscraping :) screenscaping always need to be updated | 17:15 |
| portante | sdague are we good to go now rechecking gate jobs? | 17:15 |
| pleia2 | clarkb: ok, I'll give it a go | 17:15 |
| portante | clarkb, mordred, is it okay for swift to start rechecking gate jobs? | 17:16 |
| jeblair | portante: yes; see https://wiki.openstack.org/wiki/Infrastructure_Status for the bug # | 17:17 |
| chmouel | the script https://gist.github.com/chmouel/6688088 | 17:18 |
| chmouel | would show the console.html for the current review | 17:18 |
| chmouel | (first arg is the job name) | 17:18 |
| chmouel | without arg list the jobs | 17:18 |
| jeblair | chmouel: ah cool. well since you're doing a gerrit query anyway, i would recommend changing it to to scrape the jenkins comment (you can add --coments to the query command i think) | 17:20 |
| sdague | portante: yes, should be good | 17:20 |
| jeblair | chmouel: then that will save a network request | 17:20 |
| chmouel | cool | 17:21 |
| jeblair | yeah '--comments' | 17:21 |
| sdague | jeblair: so gerrit hasn't been reporting on nova changes recently, like this one - https://bugs.launchpad.net/nova/+bug/1224226 which I merged the code yesterday | 17:21 |
| uvirtbot | Launchpad bug 1224226 in nova "Modules with wrong file mode bits" [Medium,Fix committed] | 17:21 |
| sdague | anyone have the ability to take a look | 17:21 |
| fungi | sdague: looking into it | 17:22 |
| jeblair | fungi, sdague: https://review.openstack.org/#/c/41660/ my first thought is whether that bug syntax matches | 17:22 |
| jeblair | fungi, sdague: it certainly is not the documented syntax | 17:22 |
| fungi | it definitely does not | 17:23 |
| *** zehicle_at_dell has joined #openstack-infra | 17:23 | |
| pleia2 | clarkb: ok, so I'm pretty sure how this testing works is that the tripleo thing looks at the pool of mac addresses and starts booting a VM, then we use iscsi to do stuff, so we do indeed get a different error if the part where "start booting VM" isn't done and we just run the iscsiadm command http://paste.openstack.org/show/47444/ | 17:23 |
| clarkb | I did merge ttx's change as well, though he reported it looked good last night | 17:23 |
| fungi | and also, if all of them are marked partially-fixes then no one of them will cause it to be marked fix-committed either | 17:23 |
| pleia2 | (and actually #tripleo may be better for this chat, so I'll wander over there) | 17:23 |
| sdague | hmmm... but it matches the gerrit linkage. I thought it was the same regex for those? | 17:24 |
| clarkb | pleia2: cool, let me know if I need to do anyhting on my test vm? | 17:24 |
| fungi | sdague: no, linking to a bug and telling the commit hook which of several behaviors to follow based on properly-formed commit message headers is fairly divergent now | 17:24 |
| sdague | hmmmm... that's a bummer, I doubt folks realize that | 17:25 |
| fungi | sdague: we've sent several messages to the ml, but yeah for the most part people don't read | 17:25 |
| sdague | there is a lot of traffic on the ML :) | 17:25 |
| fungi | it tends to get lost in the other discussions going on, yes | 17:25 |
| sdague | so is there a reason we diverged them? | 17:26 |
| sdague | can we undiverge them? | 17:26 |
| fungi | sdague: well, based on a summit session it was decided that we needed different behavior for commits which completely fix bugs, commits which are partial fixes for bugs and commits which are merely related to a bug | 17:26 |
| fungi | sdague: however, yes, we could consider updating the gerrit link pattern to now only work if people use the preferred header syntax and ignore the limited backward-compat patterns the hook knows about | 17:27 |
| *** julim has joined #openstack-infra | 17:27 | |
| sdague | ok, that's fine, I'm just surprised that no reporting came in. | 17:27 |
| fungi | sdague: right, the patterns the hook recognizes for its various behaviors are maintained at http://git.openstack.org/cgit/openstack-infra/jeepyb/tree/jeepyb/cmd/update_bug.py#n346 | 17:29 |
| *** DennyZha` has quit IRC | 17:30 | |
| *** svarnau has joined #openstack-infra | 17:30 | |
| jeblair | fungi: ++ (better visual feedback there) | 17:30 |
| *** svarnau_ has quit IRC | 17:30 | |
| fungi | i suspect doing that would add some incentive to people to start using the new patterns | 17:31 |
| *** ruhe has joined #openstack-infra | 17:34 | |
| *** SergeyLukjanov has joined #openstack-infra | 17:36 | |
| *** kgriffs is now known as kgriffs_afk | 17:37 | |
| *** thomasbiege has joined #openstack-infra | 17:38 | |
| *** zehicle has quit IRC | 17:38 | |
| *** dkliban has joined #openstack-infra | 17:40 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Deploy elastic-recheck on logstash.openstack.org. https://review.openstack.org/47497 | 17:40 |
| clarkb | jog0: ^ if the infra cores are happy with that I will create a gerrit system user called elasticrecheck, update hiera with the key for that account, then we should be able to deploy it | 17:41 |
| openstackgerrit | A change was merged to openstack-infra/config: Put build UUID, log_url and project in log events. https://review.openstack.org/46590 | 17:43 |
| *** pblaho has joined #openstack-infra | 17:46 | |
| fungi | sdague: jeblair: i drafted up the link change while i was thinking about it, but it just dawned on me that the commentlink feature in gerrit is global to commit messages, inline code comments and review comments | 17:48 |
| svarnau | I am setting up an instance of openstack-infra for a different project, and have question about wiki.pp | 17:48 |
| fungi | we can't fine-tune it differently for commit messages than the others | 17:48 |
| svarnau | Are the SSL certs set-up to link with launchpad IDs, or is that a separate issue? | 17:49 |
| fungi | svarnau: it uses launchpad as an openid provider, but not in any way related to ssl certs | 17:49 |
| svarnau | okay, thanks. So the SSL cert config can just be the server certs? | 17:50 |
| fungi | svarnau: yes, the ssl cert config in wiki.pp is whatever you're using to serve https for that server | 17:50 |
| svarnau | fungi: thanks! | 17:51 |
| fungi | sdague: jeblair: comparing the docs for gerrit 2.6 it looks like that isn't changing in new versions either, though maybe it's a patch upstream might accept for 2.7 (differentiating scope for commentlink options) | 17:53 |
| fungi | sdague: i think the best we can hope for at the moment is to just continue to point people at https://wiki.openstack.org/wiki/GitCommitMessages#Including_external_references (and maybe clarify it further) | 17:54 |
| sdague | sure | 17:54 |
| jeblair | fungi: hrm. well, i'd be willing to lose the non-commit-message syntax (for comments, etc), but i don't feel strongly about that. | 17:55 |
| fungi | jeblair: mmm, well i'll go ahead and send it up for review and get opinions then | 17:55 |
| jeblair | ok | 17:55 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/config: Setup gerritbot for new Savanna repos https://review.openstack.org/48105 | 17:57 |
| *** KennethWilke has joined #openstack-infra | 17:58 | |
| *** markmcclain has quit IRC | 18:00 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Limit bug links in Gerrit to recommended headers https://review.openstack.org/48108 | 18:03 |
| hub_cap | hey where is the recheck bug XXX page that shows the rechecks and their assoc. bugs | 18:04 |
| *** sarob has joined #openstack-infra | 18:05 | |
| fungi | hub_cap: http://status.openstack.org/rechecks | 18:05 |
| hub_cap | <3 | 18:05 |
| fungi | hub_cap: one of the many useful tabs linked along the top of the status.o.o webpage | 18:05 |
| *** acabrera has joined #openstack-infra | 18:06 | |
| hub_cap | holy moly | 18:06 |
| hub_cap | why didnt i know of teh base status page | 18:06 |
| *** sarob has quit IRC | 18:06 | |
| hub_cap | its the base of all the links lol | 18:07 |
| fungi | carefully guarded secret? | 18:07 |
| hub_cap | that i already use | 18:07 |
| hub_cap | or stupid me | 18:07 |
| hub_cap | ill go w stupid me | 18:07 |
| *** sarob has joined #openstack-infra | 18:07 | |
| *** cppcabrera has quit IRC | 18:07 | |
| *** jerryz has joined #openstack-infra | 18:10 | |
| *** jerryz has quit IRC | 18:11 | |
| *** sarob has quit IRC | 18:11 | |
| *** jerryz has joined #openstack-infra | 18:11 | |
| *** kgriffs_afk is now known as kgriffs | 18:14 | |
| anteaya | UtahDave is going to be able to join us for the -infra meeting | 18:15 |
| anteaya | I look forward to hearing about the current Salt status | 18:15 |
| fungi | anteaya: yay! i have some more fun for him | 18:15 |
| anteaya | awesome | 18:15 |
| fungi | UtahDave: oh! you're here | 18:15 |
| UtahDave | yep! | 18:15 |
| *** kgriffs is now known as kgriffs_afk | 18:16 | |
| fungi | UtahDave: had a look last night after letting the master/minions mellow for a couple weeks... the centos minions are still working great but now the master has lost contact with all the precise minions... haven't started digging into logs yet to see if they say why or when it started | 18:16 |
| UtahDave | yeah, that will be interesting to see the logs. | 18:18 |
| UtahDave | fungi: does restarting the salt-master help? | 18:18 |
| fungi | UtahDave: haven't tested that yet since i don't want to disturb its state until i collect more details | 18:19 |
| *** markmcclain has joined #openstack-infra | 18:19 | |
| *** jhesketh_ has quit IRC | 18:20 | |
| UtahDave | fungi: ok, ping me when you have some time to troubleshoot it. | 18:21 |
| fungi | UtahDave: well, i'm digging into it now, but will ping you when i have some data | 18:22 |
| UtahDave | ok | 18:22 |
| *** mriedem has quit IRC | 18:27 | |
| *** melwitt has joined #openstack-infra | 18:28 | |
| fungi | UtahDave: so here's what i know... salt master and minion logs are empty since rotation on the 14th/15th. salt-minion processes were last restarted around the same timeframe, looks like. restarting the service on a nonresponsive minion causes it to be reachable by the master again | 18:29 |
| fungi | UtahDave: also, i have two ubuntu precise minions which did not go dark, even though they have similar start times in the process list | 18:30 |
| fungi | unlike previous issues, no unaccepted/pending minion keys, no excessive resource consumption, no mutiplicity of concurrent minion processes | 18:31 |
| fungi | just not responding to a salt ping and not running any commands requested by the master | 18:32 |
| *** jhesketh_ has joined #openstack-infra | 18:32 | |
| *** mriedem has joined #openstack-infra | 18:33 | |
| UtahDave | fungi: OK, let me go find out if we've documented this issue anywhere | 18:45 |
| mordred | sdague, jeblair: this morning's requirements issue ... | 18:46 |
| mordred | sdague, jeblair: I think we should perhaps revisit how we transactionally do some things in the mirror building | 18:47 |
| mordred | because if I'm grokking it all right- cliff released new version which had a transitive depend, and we got the new version of cliff but not the new version of pyparsing | 18:47 |
| mordred | which means there was a failure which left us half-synced | 18:47 |
| mordred | so while we want to have things break when upstream breaks us - that's not actually what happened here | 18:48 |
| UtahDave | fungi: can you pastebin the output of salt 'minion-name' test.versions_report from one of the previously unresponsive precise minions? | 18:48 |
| mordred | we created a state in the mirror which did not exist in the world at large because of a half-applied mirroring operation | 18:49 |
| mordred | I'm not 100% sure I've got a good idea of what the best next algorithm should be - but some combiation of "apply all of a given dep chain or none of it" and "be able to retry individual things that didn't work the first time" | 18:50 |
| *** hashar has joined #openstack-infra | 18:51 | |
| UtahDave | fungi: also do you know if Puppet is ever trying to reload the salt-minion? Due to some idiosyncrasies of how zeromq works Salt should always be restarted, not reloaded | 18:51 |
| clarkb | mordred: this has happened in the past and my understanding (after being very confused) was that this is the behavior we want | 18:51 |
| clarkb | mordred: because it very clearly shows us when the world has broken us | 18:51 |
| mordred | but the world did not break us | 18:51 |
| clarkb | mordred: it did cliff required >X we required <X | 18:52 |
| mordred | cliff's requirement of >X should have caused >X to make it in to the mirror, no? | 18:52 |
| clarkb | mordred: not necessarily as pip is poor at dependency resolution, <X may have been evaluated first | 18:53 |
| mordred | there _was_ a break that was valid | 18:53 |
| mordred | but there was also a break in what elements of the world were present in our mirror | 18:53 |
| mordred | I believe that we actually moved to agreeing with you tat the mirror should not attempt to impose additional enforcement of the reuqirment set | 18:53 |
| mordred | not that we have openstack/requirements in place and gating | 18:53 |
| jeblair | mordred: yeah, i think the actual issue is that the mirror script was changed to support partial success (if package a breaks, don't let that stop us from uploading package b) | 18:55 |
| jeblair | mordred: i believe this is an unintended consequence of that | 18:55 |
| jeblair | mordred: and perhaps the mirror script should go back to being all or none | 18:56 |
| lifeless | jeblair: thanks for the reply; on capacity, we may need to only check a subset of proposals, but gate all, if we don't have enough capacity at the start. Is that feasible? | 18:56 |
| mordred | jeblair: hrm. | 18:56 |
| mordred | jeblair: I thnk I'm beginning to think that the mirror system just needs to be a mix of the two | 18:56 |
| mordred | jeblair: and smarter | 18:56 |
| lifeless | jeblair: (naturally we wouldn't *start* as gate, I mean once reliability etc have been demonstrated) | 18:56 |
| mordred | jeblair: so that we can get the correct all or none and appropriate re-tries to catch transient crud | 18:57 |
| *** kgriffs_afk is now known as kgriffs | 18:57 | |
| mordred | jeblair: let me noodle on it more and come back with a real suggestion rather than brainstorming directly here | 18:57 |
| jeblair | mordred: ok | 18:57 |
| fungi | UtahDave: this is one which was unresponsive until i restarted the minion service a few minutes ago... http://paste.openstack.org/show/47451/ | 18:58 |
| jeblair | meeting time | 18:59 |
| UtahDave | fungi: wow, somehow zmq 2.1.11 is being installed. That's the problem. | 18:59 |
| UtahDave | fungi: I'll catch you after the meeting | 19:00 |
| fungi | UtahDave: awesome. and i'll find where it's coming in from | 19:00 |
| UtahDave | the saltstack ppa should provide the latest zmq | 19:00 |
| *** kgriffs is now known as kgriffs_afk | 19:02 | |
| fungi | it looks like i still have some puppetry issues around getting unattended upgrades on ubuntu to not ignore third-party repositories | 19:04 |
| fungi | one of the two which did not go on silent running was running a newer zeromq (ZMQ: 3.2.2, PyZMQ: 13.0.0) | 19:05 |
| *** kgriffs_afk is now known as kgriffs | 19:05 | |
| fungi | so this definitely supports that assertion | 19:05 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Deploy elastic-recheck on logstash.openstack.org. https://review.openstack.org/47497 | 19:06 |
| *** dizquierdo has joined #openstack-infra | 19:09 | |
| *** pycabrera has joined #openstack-infra | 19:12 | |
| *** pycabrera is now known as cppcabrera | 19:13 | |
| *** cppcabrera has quit IRC | 19:14 | |
| thomasm | Are the gate-tempest-devstack-* tests the same for all OS projects? Are they using the same hosts and orchestration? An update to global requirements isn't being applied completely for my grizzly patch: https://review.openstack.org/#/c/48039/ | 19:14 |
| thomasm | =[ | 19:14 |
| clarkb | thomasm: we are in our weekly meeting currently, but afterwards I am sure one of us will take a look | 19:15 |
| thomasm | clarkb, Oh, okay. Thanks. =] | 19:15 |
| *** acabrera has quit IRC | 19:15 | |
| *** ruhe has quit IRC | 19:18 | |
| *** Ryan_Lane has quit IRC | 19:29 | |
| fungi | thomasm: bug 1229797 is still not addressed in stable/grizzly i think | 19:30 |
| uvirtbot | Launchpad bug 1229797 in openstack-ci "cliff 1.4.5 & pyparsing 2.0.1 cause sadness in the gate" [Undecided,New] https://launchpad.net/bugs/1229797 | 19:30 |
| *** ruhe has joined #openstack-infra | 19:31 | |
| *** mrmartin has joined #openstack-infra | 19:32 | |
| *** pblaho has quit IRC | 19:33 | |
| thomasm | fungi, Ahh, right. The upstream comment. Thanks, fungi! =] | 19:34 |
| fungi | thomasm: you're welcome | 19:35 |
| *** sandywalsh has quit IRC | 19:35 | |
| hub_cap | fwiw jeblair mordred the weekly meeting is on my calendar now so ill be paying bettter attn, also in another rackspace meeting at this time lol | 19:37 |
| jeblair | hub_cap: from what i know of you, you should be able to handle 4 meetings simultaneously, easy. | 19:37 |
| hub_cap | heh its 1 cup of coffee / meeting jeblair | 19:40 |
| hub_cap | and ive only had 3 so far.... so really, the calendar just defines the # cups of coffee | 19:40 |
| *** vipul is now known as vipul-away | 19:41 | |
| *** yolanda has quit IRC | 19:42 | |
| *** dcramer_ has quit IRC | 19:44 | |
| *** markmc has joined #openstack-infra | 19:46 | |
| *** boris-42 has joined #openstack-infra | 19:46 | |
| *** sandywalsh has joined #openstack-infra | 19:47 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add first pass recheck script https://review.openstack.org/48087 | 19:47 |
| openstackgerrit | A change was merged to openstack/requirements: Use oslo.config 1.2.0 https://review.openstack.org/48022 | 19:59 |
| *** sarob has joined #openstack-infra | 19:59 | |
| mordred | w00t! | 20:00 |
| mtreinish | jeblair: I didn't even realize that was on the agenda :) | 20:01 |
| *** cppcabrera has joined #openstack-infra | 20:02 | |
| jeblair | mtreinish: me neither, though it was a good idea because there's a bit of a design discussion going on in https://review.openstack.org/#/c/47491/ | 20:03 |
| *** gyee has quit IRC | 20:03 | |
| *** cppcabrera has left #openstack-infra | 20:03 | |
| *** tvb has quit IRC | 20:03 | |
| mtreinish | jeblair: I was reading your comment earlier. So you're proposing we tag bugs (or leave comments with the queries) and have the bot pull that data from launchpad? | 20:04 |
| *** dprince has quit IRC | 20:04 | |
| *** rockyg has quit IRC | 20:06 | |
| *** nicedice has joined #openstack-infra | 20:08 | |
| jeblair | mtreinish: yes, that was the original idea that we brainstormed at the summit | 20:08 |
| fungi | have the bot retrieve comments from the bug, parse them to find the most recent containing an elastic recheck query... but yeah where do you tell elastic recheck which bugs to look at? integrate with initial gerrit recheck comments to build up a list? | 20:09 |
| jeblair | fungi: tag bugs with something and e-r searches for the tag | 20:09 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack-infra/config: Create new gates for openstack-manuals https://review.openstack.org/48128 | 20:09 |
| *** nicedice has quit IRC | 20:10 | |
| fungi | ahh, yep. search for all bugs with the tag. got it | 20:10 |
| jeblair | 'logstash-query' or something | 20:10 |
| *** armax has joined #openstack-infra | 20:10 | |
| anteaya | yay UtahDave will be in Hong Kong | 20:10 |
| *** nicedice has joined #openstack-infra | 20:10 | |
| fungi | anteaya: UtahDave: great! much collaboration will be had by all | 20:11 |
| anteaya | woohoo | 20:11 |
| clarkb | jeblair: fungi: I like the idea, I think we should start with the existing simple config file and add that feature later (I don't want to hold up deploying this thing) | 20:11 |
| armax | does anyone know how to deal with a gerrit 'Application Error: The page you requested was not found, or you do not have permission to view this page.', after having deleted a draft change? | 20:11 |
| armax | on review.openstack.org? | 20:11 |
| fungi | i didn't know you could delete a draft | 20:11 |
| UtahDave | fungi: Yep! I'm really looking forward to it. | 20:11 |
| jeblair | clarkb: yes, as i'm not writing the code, i'm willing to go with that since it does seem like a viable implementation. i tried to make it clear that was a 'helpful contribution' as opposed to a blocking core review. :) | 20:11 |
| fungi | armax: i really, really don't ever use drafts. my only exposure to them is helping others troubleshoot where they go wrong | 20:12 |
| *** vipul-away is now known as vipul | 20:12 | |
| armax | fungi: ok | 20:13 |
| fungi | armax: but not familiar with what happens when you delete them, no | 20:13 |
| mtreinish | jeblair, fungi: that sounds like it would definitely improve the workflow and I kind of like it. But, the one key advantage I see with the file now though is the verification step and review. With bug comments theres nothing stopping people from putting bad queries that catch everything | 20:13 |
| mtreinish | but I agree with clarkb we should start with what we've got now and work on that as a future item | 20:13 |
| armax | usually I publish them after they are ready for prime time | 20:13 |
| *** kokoisatard has joined #openstack-infra | 20:13 | |
| armax | this time I hitched for the wrong button, argh | 20:13 |
| *** kokoisatard has quit IRC | 20:13 | |
| clarkb | you can delete drafts? | 20:14 |
| mtreinish | (although now that I think about it we don't have it configured it to gate on running test_queries.py) | 20:14 |
| jeblair | mtreinish: agreed. though i feel that the negative impact of that would be small and outweigh the benefits (intentional vandalism is very unlikely, and accidental mistakes can be easily corrected by anyone). | 20:14 |
| fungi | mtreinish: though it's easy enough to untag or update a bug with a better query and assume people won't engage in unnecessary bugwars i think | 20:14 |
| armax | clarkb: apparently yes there are two buttons | 20:14 |
| jeblair | mtreinish: /outweigh/be outweighed by/ | 20:14 |
| armax | one says delete draft change | 20:14 |
| armax | the other delete draft patch set | 20:14 |
| armax | or something like that | 20:15 |
| fungi | clarkb: to my surprise as well | 20:15 |
| *** beagles has quit IRC | 20:15 | |
| clarkb | armax: the simplest thing may be to repush to gerrit | 20:16 |
| mtreinish | jeblair, fungi: I agree, that's why I think working on using launchpad for the queries is a good future step. (I was just pointing out an advantage of the json file) | 20:16 |
| armax | clarkb: I'll give it a whirl, thanks | 20:16 |
| jeblair | mtreinish: anyway, i'm all in favor of getting it going (after the acl typo is fixed) | 20:16 |
| mtreinish | jeblair: ok, let me take care of that right now | 20:17 |
| *** zehicle has joined #openstack-infra | 20:18 | |
| *** beagles has joined #openstack-infra | 20:18 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/config: Add elastic-recheck to openstack-infra https://review.openstack.org/47491 | 20:19 |
| *** dcramer_ has joined #openstack-infra | 20:21 | |
| *** rnirmal_ has joined #openstack-infra | 20:22 | |
| LinuxJedi | woot! according to Stackanalysis me and half my team don't work for HP :) | 20:22 |
| fungi | LinuxJedi: your wish came true? ;) | 20:22 |
| LinuxJedi | fungi: only if someone else can pay me the same ;) | 20:23 |
| *** rnirmal has quit IRC | 20:23 | |
| *** rnirmal_ is now known as rnirmal | 20:23 | |
| *** bnemec has quit IRC | 20:23 | |
| fungi | LinuxJedi: sorry, i can't spare my loose change--need to be able to do laundry | 20:24 |
| *** thomasbiege has quit IRC | 20:25 | |
| *** bnemec has joined #openstack-infra | 20:26 | |
| * anteaya heads out for a walk | 20:28 | |
| *** Ryan_Lane has joined #openstack-infra | 20:29 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/config: add min-sev parameter https://review.openstack.org/48131 | 20:30 |
| sdague | clarkb: give that a whirl & a look when you get a chance | 20:30 |
| *** krtaylor has quit IRC | 20:31 | |
| *** tvb has joined #openstack-infra | 20:33 | |
| *** rockyg has joined #openstack-infra | 20:34 | |
| clarkb | sdague: will do | 20:35 |
| sdague | it's lightly tested with a cinder log I have from a while ago for both html and/or text | 20:36 |
| clarkb | jeblair: fyi the zuul results queue is really long, watching it now to see if it falls at all | 20:36 |
| sdague | basically add min-sev=INFO and you only get INFO and up | 20:36 |
| clarkb | sdague: perfect | 20:36 |
| sdague | and it seems to go pretty fast | 20:36 |
| jeblair | clarkb: yeah, i think it may have been processing a reset | 20:38 |
| *** mikal is now known as annegentle_proxy | 20:38 | |
| *** annegentle_proxy is now known as mikal | 20:38 | |
| * fungi has been talked into popping out for some evening exercise while the weather's nice... back for more hacking in a while | 20:41 | |
| *** tvb has quit IRC | 20:42 | |
| *** rnirmal has quit IRC | 20:43 | |
| *** atiwari has quit IRC | 20:43 | |
| *** julim has quit IRC | 20:44 | |
| *** rcleere has quit IRC | 20:45 | |
| *** rcleere has joined #openstack-infra | 20:45 | |
| lifeless | is there a way to close all fix-committed bugs when a release is cut from a project ? | 20:47 |
| lifeless | or is it manual? | 20:47 |
| *** morganfainberg has quit IRC | 20:47 | |
| jeblair | ttx: see question from lifeless ^ | 20:48 |
| ttx | lifeless: it's a LP script | 20:48 |
| ttx | https://github.com/ttx/openstack-releasing/blob/master/process_bugs.py | 20:49 |
| SpamapS | ttx: but does it run with the git tag magic that makes tarballs and fresh cookies? | 20:49 |
| lifeless | or can it ? | 20:51 |
| *** ArxCruz has quit IRC | 20:54 | |
| *** rwsu has joined #openstack-infra | 20:54 | |
| ttx | lifeless: that would be manual at this point (two separate steps in the release process) | 20:55 |
| ttx | https://wiki.openstack.org/wiki/Release_Team/How_To_Release might be your friend (or not) | 20:55 |
| *** HenryG has joined #openstack-infra | 20:56 | |
| lifeless | SpamapS: ^ | 20:56 |
| SpamapS | ok seems like the git magic releaser could call the script. | 20:57 |
| SpamapS | lifeless: ty | 20:57 |
| jeblair | SpamapS: it could be made a jenkins job | 20:57 |
| *** thomasbiege has joined #openstack-infra | 20:57 | |
| mikal | Dear smart people. I need adult supervision. | 20:58 |
| mikal | I'm interested in either a) forming a boy band with you, or b) working out the right wat to fix https://review.openstack.org/#/c/47970/2 | 20:59 |
| lifeless | WAT | 20:59 |
| mikal | s/wat/way/ | 20:59 |
| lifeless | mikal: why isn't that a test? | 21:00 |
| mikal | Assuming we go with option number B, specifically I'm trying to decide the best way to tweak the module path to import the right version of nova.exceptions in the test script | 21:00 |
| mikal | lifeless: a unit test which walks all the exceptions? | 21:00 |
| mikal | lifeless: cause I didn't think of that? | 21:00 |
| lifeless | mikal: :) | 21:01 |
| jeblair | lifeless's suggestion++ | 21:01 |
| mikal | Its not a terrible idea though | 21:01 |
| *** rfolco has quit IRC | 21:01 | |
| lifeless | mikal: I have written this exact test back in bzr :) | 21:01 |
| * mikal will give it a go once he's finished watching these boy band videos | 21:01 | |
| lifeless | mikal: it works well | 21:01 |
| mikal | Priorities don't you know | 21:01 |
| jeblair | mikal: you better delegate in that case | 21:01 |
| * mordred wants to see the lifeless mikal boy band | 21:02 | |
| lifeless | mordred: you're going to keep on wanting for a loooong time | 21:02 |
| mikal | mordred: it would have to be down under themed. | 21:03 |
| adam_g | does the jenkins-job-builder currently support defining variables that are available globally to templates? | 21:08 |
| *** kgriffs is now known as kgriffs_afk | 21:08 | |
| *** hashar has quit IRC | 21:08 | |
| *** ericlin has joined #openstack-infra | 21:11 | |
| *** kgriffs_afk is now known as kgriffs | 21:12 | |
| *** krtaylor has joined #openstack-infra | 21:13 | |
| *** ericlin has quit IRC | 21:13 | |
| *** mrmartin has quit IRC | 21:14 | |
| *** gyee has joined #openstack-infra | 21:15 | |
| *** dkranz has quit IRC | 21:20 | |
| *** jcoufal has quit IRC | 21:21 | |
| *** thomasm has quit IRC | 21:22 | |
| *** morganfainberg has joined #openstack-infra | 21:23 | |
| clarkb | jeblair: I am updating the overview zuul simulation to show nnfi | 21:24 |
| clarkb | jeblair: I assume your javascript hasn't gotten that far yet | 21:24 |
| *** Ryan_Lane has quit IRC | 21:28 | |
| *** markmc has quit IRC | 21:28 | |
| *** thomasbiege has quit IRC | 21:32 | |
| *** dizquierdo has left #openstack-infra | 21:32 | |
| *** ryanpetrello has quit IRC | 21:34 | |
| *** tvb has joined #openstack-infra | 21:39 | |
| *** tvb has joined #openstack-infra | 21:39 | |
| *** zehicle_at_dell has quit IRC | 21:40 | |
| *** zehicle_at_dell has joined #openstack-infra | 21:41 | |
| *** tvb has quit IRC | 21:43 | |
| *** dkliban has quit IRC | 21:44 | |
| *** weshay has quit IRC | 21:46 | |
| *** wchrisj has quit IRC | 21:51 | |
| *** ruhe has quit IRC | 21:52 | |
| *** julim has joined #openstack-infra | 21:55 | |
| *** vipul is now known as vipul-away | 21:57 | |
| mordred | clarkb: I believe he's almost done | 22:02 |
| *** matty_dubs is now known as matty_dubs|gone | 22:03 | |
| *** amotoki has joined #openstack-infra | 22:03 | |
| clarkb | mordred: but he is doing js for status and not the talk simulation? | 22:04 |
| *** datsun180b has quit IRC | 22:06 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/publications: Update talk with new information and images. https://review.openstack.org/48151 | 22:09 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/publications: Add logstash slide. https://review.openstack.org/48152 | 22:09 |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/publications: Make zuul simulation reflect NNFI. https://review.openstack.org/48153 | 22:09 |
| clarkb | mordred: fungi jeblair ^ | 22:09 |
| *** tstevenson has quit IRC | 22:09 | |
| clarkb | mordred: http://docs.openstack.org/infra/publications/overview/#%2817%29 do you know what we are trying to express there? | 22:10 |
| *** jhesketh_ has quit IRC | 22:13 | |
| *** jhesketh has quit IRC | 22:13 | |
| *** vipul-away is now known as vipul | 22:13 | |
| mordred | clarkb: yes. we're showing the overview that you can get | 22:14 |
| mordred | with the various statuses things can be in and your view of the whole | 22:14 |
| clarkb | thanks | 22:15 |
| *** SergeyLukjanov has quit IRC | 22:17 | |
| *** mriedem has quit IRC | 22:19 | |
| *** markmcclain has quit IRC | 22:19 | |
| *** zehicle_at_dell has quit IRC | 22:25 | |
| *** jhesketh has joined #openstack-infra | 22:25 | |
| *** jhesketh_ has joined #openstack-infra | 22:26 | |
| clarkb | anyone else want to review https://review.openstack.org/#/c/47491/ (adds elastic-recheck to gerrit) before I approve? | 22:29 |
| clarkb | jog0: mtreinish is the source repo ready for the upstream to change? | 22:30 |
| *** changbl has quit IRC | 22:31 | |
| *** boris-42 has quit IRC | 22:32 | |
| *** Ryan_Lane has joined #openstack-infra | 22:34 | |
| mordred | clarkb: nope. go for it - if jog0 and mtreinish are ready | 22:34 |
| *** prad has quit IRC | 22:34 | |
| *** HenryG has quit IRC | 22:35 | |
| jog0 | clarkb mordred: Its ready enough go for it | 22:38 |
| clarkb | approved | 22:39 |
| *** rcleere has quit IRC | 22:40 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add elastic-recheck to openstack-infra https://review.openstack.org/47491 | 22:42 |
| mordred | woot | 22:43 |
| *** armax has left #openstack-infra | 22:46 | |
| *** KennethWilke has quit IRC | 22:46 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Deploy elastic-recheck on logstash.openstack.org. https://review.openstack.org/47497 | 22:47 |
| clarkb | and ^ should be ready for review | 22:47 |
| *** openstackgerrit has quit IRC | 22:52 | |
| *** openstackgerrit has joined #openstack-infra | 22:52 | |
| *** rwsu has quit IRC | 22:53 | |
| *** dims has quit IRC | 22:54 | |
| *** rwsu has joined #openstack-infra | 22:56 | |
| clarkb | ok I am going to AFK to prepare for the game tonight. Need to get dinner so that I don't have to eat safeco food | 22:58 |
| *** julim has quit IRC | 23:06 | |
| fungi | clarkb: awesome updates. another good todo on the overview preso is to refactor some of the devstack-gate slides so that we plug nodepool | 23:06 |
| *** dims has joined #openstack-infra | 23:07 | |
| clarkb | good idea | 23:07 |
| *** krtaylor has quit IRC | 23:09 | |
| *** tvb has joined #openstack-infra | 23:10 | |
| *** tvb has joined #openstack-infra | 23:10 | |
| *** thomasm has joined #openstack-infra | 23:12 | |
| *** tvb has quit IRC | 23:14 | |
| *** amotoki has quit IRC | 23:18 | |
| *** flaper87 is now known as flaper87|afk | 23:26 | |
| *** amotoki has joined #openstack-infra | 23:28 | |
| reed | guys, I have corrected the blog post http://www.openstack.org/blog/2013/09/openstack-governance-electing-technical-leaders-for-next-development-cycle/ but I don't think the copy on planet will update itself | 23:29 |
| reed | can you confirm or manually rebuild the planet? | 23:29 |
| *** Ryan_Lane has quit IRC | 23:29 | |
| *** Ryan_Lane has joined #openstack-infra | 23:34 | |
| *** vipul is now known as vipul-away | 23:39 | |
| *** tvb has joined #openstack-infra | 23:41 | |
| *** amotoki has quit IRC | 23:41 | |
| mordred | reed: you are righ t- it will not - we need a todo list item to make that work better | 23:43 |
| mordred | lemme go poke it with a big stick | 23:43 |
| reed | thanks mordred | 23:43 |
| *** UtahDave has quit IRC | 23:44 | |
| *** pabelanger has quit IRC | 23:45 | |
| *** pabelanger has joined #openstack-infra | 23:45 | |
| *** tvb has quit IRC | 23:45 | |
| mordred | reed: I have removed that entry from the cache | 23:45 |
| reed | cool | 23:45 |
| mordred | reed: so next time it runs, it should re-download that entry | 23:45 |
| *** vipul-away is now known as vipul | 23:46 | |
| *** notmyname has quit IRC | 23:47 | |
| *** uvirtbot has quit IRC | 23:47 | |
| *** crank has quit IRC | 23:47 | |
| *** hemnafk has quit IRC | 23:47 | |
| *** danger_fo_away has quit IRC | 23:47 | |
| *** Mithrandir has quit IRC | 23:47 | |
| *** Vivek has quit IRC | 23:47 | |
| *** mikal has quit IRC | 23:47 | |
| *** rahmu has quit IRC | 23:47 | |
| *** westmaas has quit IRC | 23:47 | |
| *** antonym has quit IRC | 23:47 | |
| *** belliott has quit IRC | 23:47 | |
| *** dansmith has quit IRC | 23:47 | |
| *** akscram has quit IRC | 23:47 | |
| *** Ryan_Lane has quit IRC | 23:47 | |
| *** marun has quit IRC | 23:47 | |
| *** cody-somerville has quit IRC | 23:47 | |
| *** ndipanov_gone has quit IRC | 23:47 | |
| *** jesusaurus has quit IRC | 23:47 | |
| *** schwicht has quit IRC | 23:47 | |
| *** salv-orlando has quit IRC | 23:47 | |
| *** mestery has quit IRC | 23:47 | |
| *** ianw has quit IRC | 23:47 | |
| *** Daviey has quit IRC | 23:47 | |
| *** dragondm has quit IRC | 23:47 | |
| *** SpamapS has quit IRC | 23:47 | |
| *** melwitt has quit IRC | 23:47 | |
| *** paul-- has quit IRC | 23:47 | |
| *** anteaya has quit IRC | 23:47 | |
| *** zul has quit IRC | 23:47 | |
| *** enikanorov-w has quit IRC | 23:47 | |
| *** lifeless has quit IRC | 23:47 | |
| *** bodepd has quit IRC | 23:47 | |
| *** Shrews has quit IRC | 23:47 | |
| *** MoXxXoM has quit IRC | 23:47 | |
| *** rainya has quit IRC | 23:47 | |
| *** EntropyWorks has quit IRC | 23:47 | |
| *** pabelanger has quit IRC | 23:47 | |
| *** gyee has quit IRC | 23:47 | |
| *** rockyg has quit IRC | 23:47 | |
| *** yongli has quit IRC | 23:47 | |
| *** dstufft has quit IRC | 23:47 | |
| *** klrmn has quit IRC | 23:47 | |
| *** kiall has quit IRC | 23:47 | |
| *** mtreinish has quit IRC | 23:47 | |
| *** soren has quit IRC | 23:47 | |
| *** LinuxJedi has quit IRC | 23:47 | |
| *** koolhead17 has quit IRC | 23:47 | |
| *** pleia2 has quit IRC | 23:47 | |
| *** openstackgerrit has quit IRC | 23:47 | |
| *** zehicle has quit IRC | 23:47 | |
| *** mrodden has quit IRC | 23:47 | |
| *** vipul has quit IRC | 23:47 | |
| *** mordred has quit IRC | 23:47 | |
| *** CaptTofu has quit IRC | 23:47 | |
| *** moted has quit IRC | 23:47 | |
| *** briancline has quit IRC | 23:47 | |
| *** cyeoh has quit IRC | 23:47 | |
| *** sdague has quit IRC | 23:47 | |
| *** jog0 has quit IRC | 23:47 | |
| *** fungi has quit IRC | 23:47 | |
| *** saschpe_ has quit IRC | 23:47 | |
| *** Mithrandir has joined #openstack-infra | 23:47 | |
| *** westmaas has joined #openstack-infra | 23:47 | |
| *** mikal has joined #openstack-infra | 23:47 | |
| *** dansmith has joined #openstack-infra | 23:47 | |
| *** danger_fo_away has joined #openstack-infra | 23:47 | |
| *** crank has joined #openstack-infra | 23:47 | |
| *** rahmu has joined #openstack-infra | 23:47 | |
| *** belliott has joined #openstack-infra | 23:47 | |
| *** Vivek has joined #openstack-infra | 23:47 | |
| *** antonym has joined #openstack-infra | 23:47 | |
| *** sarob has quit IRC | 23:48 | |
| *** akscram has joined #openstack-infra | 23:48 | |
| *** Vivek is now known as Guest98622 | 23:48 | |
| *** saschpe_ has joined #openstack-infra | 23:54 | |
| *** fungi has joined #openstack-infra | 23:54 | |
| *** jog0 has joined #openstack-infra | 23:54 | |
| *** sdague has joined #openstack-infra | 23:54 | |
| *** cyeoh has joined #openstack-infra | 23:54 | |
| *** briancline has joined #openstack-infra | 23:54 | |
| *** moted has joined #openstack-infra | 23:54 | |
| *** CaptTofu has joined #openstack-infra | 23:54 | |
| *** mordred has joined #openstack-infra | 23:54 | |
| *** vipul has joined #openstack-infra | 23:54 | |
| *** mrodden has joined #openstack-infra | 23:54 | |
| *** zehicle has joined #openstack-infra | 23:54 | |
| *** openstackgerrit has joined #openstack-infra | 23:54 | |
| *** notmyname has joined #openstack-infra | 23:54 | |
| *** Ryan_Lane has joined #openstack-infra | 23:54 | |
| *** Ryan_Lane1 has joined #openstack-infra | 23:54 | |
| *** hemnafk has joined #openstack-infra | 23:54 | |
| *** pabelanger has joined #openstack-infra | 23:54 | |
| *** gyee has joined #openstack-infra | 23:54 | |
| *** rockyg has joined #openstack-infra | 23:54 | |
| *** melwitt has joined #openstack-infra | 23:54 | |
| *** marun has joined #openstack-infra | 23:54 | |
| *** cody-somerville has joined #openstack-infra | 23:54 | |
| *** anteaya has joined #openstack-infra | 23:54 | |
| *** MoXxXoM has joined #openstack-infra | 23:54 | |
| *** salv-orlando has joined #openstack-infra | 23:54 | |
| *** zul has joined #openstack-infra | 23:54 | |
| *** ndipanov_gone has joined #openstack-infra | 23:54 | |
| *** jesusaurus has joined #openstack-infra | 23:54 | |
| *** schwicht has joined #openstack-infra | 23:54 | |
| *** enikanorov-w has joined #openstack-infra | 23:54 | |
| *** lifeless has joined #openstack-infra | 23:54 | |
| *** mestery has joined #openstack-infra | 23:54 | |
| *** ianw has joined #openstack-infra | 23:54 | |
| *** Daviey has joined #openstack-infra | 23:54 | |
| *** yongli has joined #openstack-infra | 23:54 | |
| *** bodepd has joined #openstack-infra | 23:54 | |
| *** dstufft has joined #openstack-infra | 23:54 | |
| *** Shrews has joined #openstack-infra | 23:54 | |
| *** klrmn has joined #openstack-infra | 23:54 | |
| *** dragondm has joined #openstack-infra | 23:54 | |
| *** rainya has joined #openstack-infra | 23:54 | |
| *** SpamapS has joined #openstack-infra | 23:54 | |
| *** paul-- has joined #openstack-infra | 23:54 | |
| *** kiall has joined #openstack-infra | 23:54 | |
| *** mtreinish has joined #openstack-infra | 23:54 | |
| *** EntropyWorks has joined #openstack-infra | 23:54 | |
| *** pleia2 has joined #openstack-infra | 23:54 | |
| *** koolhead17 has joined #openstack-infra | 23:54 | |
| *** LinuxJedi has joined #openstack-infra | 23:54 | |
| *** soren has joined #openstack-infra | 23:54 | |
| *** Ryan_Lane1 has quit IRC | 23:54 | |
| *** Ryan_Lane has quit IRC | 23:54 | |
| *** Ryan_Lane has joined #openstack-infra | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!