| *** unicell1 has quit IRC | 00:00 | |
| *** dangers is now known as dangers_away | 00:00 | |
| *** baoli has quit IRC | 00:00 | |
| heyongli | UTC 8:30 is china 4:30, UTC 11:30 is oure 7:00PM both ideal time slot | 00:01 |
|---|---|---|
| heyongli | if we across eu, us and china, we will be hard to find a good time slot. but let's try. anteaya | 00:02 |
| anteaya | heyongli: any objection to 7:00 utc? | 00:02 |
| anteaya | yeah we are just trying to accompdate eu and apac | 00:03 |
| jeblair | clarkb: answer to your question is "yes" :) | 00:03 |
| *** eharney has quit IRC | 00:03 | |
| heyongli | anteaya, it's very good for us, UTC 7:00 AM is 3:30PM it's very ok for us | 00:03 |
| *** melwitt has quit IRC | 00:04 | |
| heyongli | anteaya, this time slot we both on same day. | 00:06 |
| anteaya | heyongli: please reply to the email thread | 00:07 |
| heyongli | sure. | 00:07 |
| *** ddieterly has joined #openstack-infra | 00:07 | |
| *** tomoe_ has joined #openstack-infra | 00:08 | |
| *** david-lyle is now known as david-lyle_afk | 00:08 | |
| *** ZZelle_ has quit IRC | 00:08 | |
| clarkb | jeblair: yup thought about it more and couldn't figure why not. I have a scriptthat is basically ready to list things I will propose it to project-config/tools | 00:09 |
| *** mwagner_lap has joined #openstack-infra | 00:09 | |
| clarkb | jeblair: anteaya fungi I think 6 months is probably too long a time period should we iterate over all that anyways or trim it down to say a month? | 00:09 |
| anteaya | can you paste what you have, or is it too long? | 00:10 |
| *** dmsimard is now known as dmsimard_away | 00:10 | |
| *** Sukhdev has joined #openstack-infra | 00:10 | |
| anteaya | and yes 6 months is too long | 00:10 |
| anteaya | though we have at least one ci that has been disabled for at least a month I'd say | 00:11 |
| clarkb | ya I can push up what have | 00:11 |
| *** ivar-laz_ has joined #openstack-infra | 00:11 | |
| *** ddieterly has quit IRC | 00:11 | |
| anteaya | thanks just to take a peek | 00:12 |
| JayF | Would love to have your eyes, votes, comments, etc on https://review.openstack.org/#/c/134436/ if you have a moment :) Trying to get the IPA tempest job voting on IPA (even if not on Ironic) | 00:12 |
| *** mfink has joined #openstack-infra | 00:12 | |
| *** tomoe_ has quit IRC | 00:13 | |
| *** ivar-lazzaro has quit IRC | 00:14 | |
| *** tomoe_ has joined #openstack-infra | 00:14 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Tool to list accounts voting verified https://review.openstack.org/138585 | 00:14 |
| clarkb | anteaya: jeblair ^ that is the script | 00:14 |
| clarkb | http://paste.openstack.org/show/143863/ is the output | 00:15 |
| mordred | clarkb: does it do exciting things? | 00:15 |
| *** MarkAtwood has quit IRC | 00:15 | |
| openstackgerrit | Thanh Ha proposed openstack-infra/jenkins-job-builder: URL in Maven deploy is an optional field https://review.openstack.org/138586 | 00:15 |
| clarkb | mordred: currently just lists third party accounts that voted verified over the last day | 00:15 |
| clarkb | and maps them to projects | 00:15 |
| clarkb | I think there is at least one bug | 00:15 |
| clarkb | sometimes I don't get json back but not sure why that is yet | 00:15 |
| anteaya | clarkb: oh for the last day | 00:16 |
| mordred | that would be a bug I'd imagine | 00:16 |
| anteaya | I was going to say | 00:16 |
| clarkb | anteaya: I am testing small :) | 00:16 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for oslo.concurrency bug 1398624 https://review.openstack.org/138587 | 00:16 |
| uvirtbot | Launchpad bug 1398624 in oslo.concurrency "nova unit tests failing in the gate with "TypeError: acquire() got an unexpected keyword argument 'delay'"" [Undecided,New] https://launchpad.net/bugs/1398624 | 00:16 |
| anteaya | clarkb: good idea | 00:16 |
| clarkb | I am going to go big and see what 3 months looks like | 00:16 |
| *** MarkAtwood has joined #openstack-infra | 00:16 | |
| *** MarkAtwood has quit IRC | 00:16 | |
| anteaya | this will be a useful script for just checking status, as a quick check | 00:17 |
| clarkb | http://paste.openstack.org/show/143869/ is three months | 00:19 |
| clarkb | so the list isn't all that crazy (assuming there are no other unknown bugs in my script) | 00:20 |
| *** tnovacik has quit IRC | 00:20 | |
| clarkb | the output there is clean so no exceptions thrown for the json thing | 00:20 |
| heyongli | jogo,for testcases, after intel ci log server is ready, i try to find a way how we present our test case to infra team. or any guide for this? | 00:20 |
| *** otter768 has joined #openstack-infra | 00:21 | |
| anteaya | in three months only one thing returning on cinder? | 00:21 |
| clarkb | anteaya: for voting | 00:21 |
| anteaya | okay | 00:21 |
| patrickeast_ | yea cinder ci’s aren’t voting | 00:21 |
| clarkb | I mean we should cross check | 00:21 |
| clarkb | don't trust anything I write ;) | 00:21 |
| anteaya | sandbox we can ignore | 00:21 |
| clarkb | yup | 00:22 |
| *** shashankhegde has quit IRC | 00:22 | |
| heyongli | jogo, just post patches to tempest? | 00:22 |
| anteaya | we got jeblair and fungi in that list | 00:22 |
| anteaya | I wonder how that happened | 00:22 |
| *** ryanpetrello has quit IRC | 00:22 | |
| mordred | anteaya: they are CI systems themselves | 00:22 |
| clarkb | they voted in the verified column | 00:22 |
| clarkb | probably ninja approving | 00:22 |
| anteaya | clarkb: you too | 00:22 |
| anteaya | mordred: they are | 00:23 |
| zxiiro | how long does it take for a patch that has +2 to get merged usually? | 00:23 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for oslo.concurrency bug 1398624 https://review.openstack.org/138587 | 00:23 |
| uvirtbot | Launchpad bug 1398624 in oslo.concurrency "nova unit tests failing in the gate with "TypeError: acquire() got an unexpected keyword argument 'delay'"" [Undecided,New] https://launchpad.net/bugs/1398624 | 00:23 |
| viglesias | hey team ive got my zuul+jenkins+gerrit setup nailed barring 1 small thing :) | 00:23 |
| anteaya | the deprecated trove ci got in there | 00:23 |
| clarkb | zxiiro: it largely depends also you need the +1 approved too | 00:23 |
| clarkb | anteaya: ya there will be noise but this gets us 90% there | 00:23 |
| viglesias | my verified messages from zuul are not checking the verified column | 00:23 |
| anteaya | clarkb: it does | 00:23 |
| clarkb | anteaya: so next thing to do is cross check against current voting accounts | 00:23 |
| anteaya | and far better than I had invisioned | 00:24 |
| clarkb | whihc me and fungi and jeblair are not a part of | 00:24 |
| anteaya | envisioned | 00:24 |
| zxiiro | clarkb: I have this patch here https://review.openstack.org/#/c/136225/ just wondering if I need to do anything to get it moving? | 00:24 |
| anteaya | thank goodness | 00:24 |
| anteaya | zxiiro: reviewing other people's patches always helps | 00:24 |
| clarkb | zxiiro: yes it needs a second +2 and +1 workflow approval | 00:24 |
| clarkb | once it has that it will move through the gate queue and if it passes tests will merge | 00:24 |
| clarkb | viglesias: do you have voting permissions? | 00:25 |
| zxiiro | ah ok thanks! | 00:25 |
| clarkb | viglesias: most CI accounts can't vote in the Verified column | 00:25 |
| anteaya | where are we with the stackforge ci accounts on this | 00:25 |
| viglesias | clarkb: ahh that may be it! thanks again for the pointer clarkb | 00:25 |
| viglesias | i can see the comment posted | 00:25 |
| viglesias | but no check mark | 00:25 |
| clarkb | anteaya: we will do the same to them | 00:25 |
| anteaya | it will be painful to create a project-ci group for every single one of them | 00:25 |
| *** otter768 has quit IRC | 00:26 | |
| clarkb | anteaya: its not too bad, we get it done now and in the future they will be responsible for it | 00:26 |
| anteaya | okay | 00:26 |
| clarkb | since voting will require that that group exist and all that | 00:26 |
| anteaya | that is 10 if we group | 00:26 |
| viglesias | clarkb: is voting rights a project.config thing or something you can do via the ui | 00:26 |
| anteaya | as in fuel uses one and cookbooks is one group | 00:26 |
| clarkb | viglesias: its something you need the project to agree to, we are currently in the process of reworking how it all fits together so that the project release folks can do it through the gerrit UI | 00:27 |
| clarkb | anteaya: ya it probably won't be too bad | 00:27 |
| viglesias | clarkb: ack thanks | 00:27 |
| clarkb | I need to step away again, but I can clean up that list and propose a project-config change to add the ACLs tomorrow and once that is merged add the accounts into the various groups | 00:28 |
| *** stevemar has quit IRC | 00:28 | |
| clarkb | or anyone else can if they are interested. There should be no root access required for this and most of it is unauthenticated even | 00:29 |
| *** unicell has joined #openstack-infra | 00:31 | |
| anteaya | clarkb: I'm going to defer to you as I don't want to introduce confustion by mis-understanding | 00:31 |
| clarkb | kk | 00:31 |
| anteaya | I am however cheering you on and standing by to review | 00:31 |
| *** ryanpetrello has joined #openstack-infra | 00:31 | |
| *** dims_ has joined #openstack-infra | 00:31 | |
| *** emagana_ has quit IRC | 00:32 | |
| *** dims has quit IRC | 00:34 | |
| heyongli | anteaya, i repley the mail, but seems not in same mail thread , maybe my outlook is wierd. | 00:36 |
| *** yamamoto has joined #openstack-infra | 00:36 | |
| *** yamamoto has quit IRC | 00:40 | |
| anteaya | heyongli: it looks fine, thanks for adding your reply to the thread | 00:41 |
| anteaya | outlook is horrible for email threads | 00:41 |
| pleia2 | s/for email threads// | 00:41 |
| pleia2 | :D | 00:42 |
| anteaya | true that | 00:43 |
| *** Masahiro has joined #openstack-infra | 00:46 | |
| heyongli | it is -:) | 00:48 |
| *** Masahiro has quit IRC | 00:50 | |
| *** asettle has joined #openstack-infra | 00:51 | |
| *** ryanpetrello has quit IRC | 00:52 | |
| *** zz_dimtruck is now known as dimtruck | 00:56 | |
| *** shashankhegde has joined #openstack-infra | 00:58 | |
| openstackgerrit | Steven Weston proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 01:00 |
| pleia2 | sweston: thanks for tackling that, I wasn't sure where to start | 01:00 |
| sweston | pleia2: of course, reviews are welcome | 01:01 |
| *** emagana has joined #openstack-infra | 01:03 | |
| sweston | pleia2: I have my writing hat on for the next two hours ;-) | 01:04 |
| * anteaya reviews | 01:04 | |
| anteaya | thanks sweston | 01:04 |
| sweston | anteaya: you're always welcome | 01:05 |
| *** dannywilson has quit IRC | 01:05 | |
| openstackgerrit | Steven Weston proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 01:07 |
| clarkb | anteaya ok I will whip those change(s) up first thing tomorrow | 01:07 |
| anteaya | I'm wondering if the style and tone is a bit of a change from the paragraph prior | 01:07 |
| anteaya | clarkb: thank you | 01:08 |
| *** Masahiro has joined #openstack-infra | 01:08 | |
| *** emagana has quit IRC | 01:08 | |
| sweston | it probably is, but just a first draft | 01:08 |
| anteaya | it is and contains good information | 01:08 |
| *** ddieterly has joined #openstack-infra | 01:09 | |
| anteaya | my sense of things is that the paragraph is basically an expansion of one of the sentences in the prior section, "Once a change has been approved and passed the gate jobs, Gerrit automatically merges the latest patchset." | 01:09 |
| anteaya | sweston: not finding fault just thinking outloud | 01:09 |
| *** Sukhdev has quit IRC | 01:09 | |
| anteaya | feel free to disagree | 01:10 |
| sweston | anteaya: no worries, this is what reviews are all about | 01:10 |
| anteaya | well whitespaces | 01:11 |
| sweston | anteaya: I don't think the prior section goes into enough detail | 01:11 |
| anteaya | ah | 01:11 |
| sweston | yeah I fixed that in the next set already | 01:11 |
| anteaya | sorry | 01:11 |
| * anteaya updates | 01:11 | |
| sweston | can we move that to the gating section? IMHO that's really where it belongs | 01:12 |
| dims_ | clarkb: fungi: am seeing this - "HTTP error 404 while getting http://pypi.ORD.openstack.org/packages/py2.py3/o/oslotest/oslotest-1.3.0-py2.py3-none-any.whl#md5=8bbde802d5a2beb79efe5752b9349b22 (from http://pypi.ORD.openstack.org/simple/oslotest/)" | 01:14 |
| anteaya | move what to the gating section? | 01:14 |
| dims_ | clarkb: fungi: from https://jenkins02.openstack.org/job/gate-nova-pep8/22350/console | 01:15 |
| sweston | the sentence you mentioned in the prior section. | 01:15 |
| anteaya | well we have to keep in mind the target audience for this document, which are devs brand new to openstack | 01:15 |
| *** baoli has joined #openstack-infra | 01:15 | |
| anteaya | so they need to know enough to understand what it means to merge their code | 01:16 |
| sweston | anteaya: ok | 01:16 |
| anteaya | you can make any change you want in your patch, but I do think the above section accomplishes its goal fairly well | 01:16 |
| anteaya | and you are just getting my thoughts here | 01:17 |
| sweston | yes, thank you :-) | 01:17 |
| *** yamamoto_ has joined #openstack-infra | 01:18 | |
| *** yaguang has joined #openstack-infra | 01:19 | |
| *** Ryan_Lane has joined #openstack-infra | 01:19 | |
| sweston | So thinking more about this ... maybe merging should then be limited to the process which one follows when submitting code and doing reviews. The jobs which are run as part of the merge should be covered in gating. | 01:20 |
| *** jerryz has joined #openstack-infra | 01:21 | |
| sweston | I like the description of how a merge is applied to the repo, that is very nice | 01:22 |
| anteaya | I think that is more what we are looking for here yes | 01:23 |
| *** emagana has joined #openstack-infra | 01:23 | |
| anteaya | just enough of a taste to get a new dev up and running with our system | 01:23 |
| sweston | but there is going to be some crossover in the sections on merging and gating, since gating involves merging. | 01:23 |
| anteaya | agreed | 01:23 |
| *** teran has quit IRC | 01:24 | |
| *** Longgeek has joined #openstack-infra | 01:25 | |
| *** dimtruck is now known as zz_dimtruck | 01:28 | |
| *** hdd has quit IRC | 01:28 | |
| sweston | so when a system reports failure on a pipeline, and that system has voting privileges, how does it affect Jenkins? | 01:29 |
| anteaya | what do you mean a system | 01:29 |
| anteaya | a third party ci? | 01:30 |
| anteaya | third party cis have zero effect on jenkins | 01:30 |
| sweston | ah | 01:30 |
| *** Longgeek has quit IRC | 01:30 | |
| anteaya | and play no role in zuuls algorithm for merging | 01:30 |
| anteaya | the are totally in there own category | 01:30 |
| anteaya | they report to the gui | 01:30 |
| anteaya | that is it | 01:30 |
| sweston | gotcha | 01:30 |
| *** spzala has quit IRC | 01:31 | |
| anteaya | infra's system takes no information and makes no decisions based on third party ci | 01:31 |
| anteaya | humans can | 01:31 |
| anteaya | that is the hope | 01:31 |
| anteaya | but automated tools don't | 01:31 |
| *** Ryan_Lane has quit IRC | 01:31 | |
| sweston | anteaya: ok, thank you. I will fix it in the next patchset. | 01:32 |
| anteaya | great, thanks :D | 01:32 |
| sweston | np :-) | 01:33 |
| *** ryanpetrello has joined #openstack-infra | 01:34 | |
| *** markmcclain has quit IRC | 01:39 | |
| *** jaypipes has quit IRC | 01:39 | |
| *** arosen1 has joined #openstack-infra | 01:40 | |
| arosen1 | I've noticed if i restart jenkins it's no longer listening on 8888 for zmq events. Anyone else face this? | 01:40 |
| *** sputnik13 has quit IRC | 01:40 | |
| *** stevemar has joined #openstack-infra | 01:42 | |
| asselin | I've seen something like that | 01:42 |
| asselin | ps -ef | grep jenkins | 01:42 |
| asselin | see how many processes are running | 01:42 |
| arosen1 | asselin: http://paste.openstack.org/show/143885/ | 01:43 |
| asselin | arosen1, looks ok | 01:43 |
| arosen1 | 2i t looks like? | 01:43 |
| *** nelsnelson has quit IRC | 01:43 | |
| asselin | I've seen double sometimes | 01:44 |
| arosen1 | asselin: hrm it seems like it might be a problem with service jenkins restart; I just did service jenkins stop; service jenkins start; and it seems to be working again | 01:44 |
| arosen1 | asselin: thanks | 01:45 |
| *** dmellado has quit IRC | 01:46 | |
| *** yamamoto_ has quit IRC | 01:47 | |
| *** emagana has quit IRC | 01:48 | |
| *** ryanpetrello has quit IRC | 01:48 | |
| *** emagana has joined #openstack-infra | 01:48 | |
| *** dmellado has joined #openstack-infra | 01:49 | |
| openstackgerrit | Steven Weston proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 01:49 |
| anteaya | sweston: nicely done | 01:51 |
| sweston | anteaya: woot :-D | 01:51 |
| anteaya | just waiting on the docs to come back | 01:51 |
| anteaya | the info looks good and the style looks more in line with what we already have, nice | 01:52 |
| sweston | anteaya: thanks | 01:52 |
| *** emagana has quit IRC | 01:52 | |
| *** emagana_ has joined #openstack-infra | 01:52 | |
| *** rmcall has quit IRC | 01:53 | |
| sweston | anteaya: and thanks for organizing the sprint, as well. lots of progress has been made in the last two days! | 01:55 |
| anteaya | it worked out well didnt' it | 01:56 |
| *** nosnos has joined #openstack-infra | 01:56 | |
| anteaya | glad to see the manual get more content | 01:56 |
| anteaya | :D | 01:56 |
| sweston | yes, very well done :-D | 01:56 |
| *** emagana_ has quit IRC | 01:57 | |
| anteaya | so this is maybe useful and maybe not, but take a look at your paragraph and take a look at the rest of the guide: http://docs-draft.openstack.org/95/138595/3/check/gate-infra-manual-docs/851902e/doc/build/html/developers.html#merging | 01:57 |
| anteaya | don't read it just scroll up and down a bit | 01:57 |
| *** esker has quit IRC | 01:57 | |
| anteaya | any thoughts? | 01:58 |
| anteaya | it might just be me | 01:58 |
| *** sarob has quit IRC | 01:59 | |
| *** mriedem has quit IRC | 01:59 | |
| anteaya | sweston: any thoughts? | 02:00 |
| *** doug-fish has quit IRC | 02:03 | |
| *** r-daneel has quit IRC | 02:03 | |
| *** adalbas has quit IRC | 02:08 | |
| openstackgerrit | Justin Shepherd proposed openstack-infra/project-config: Adding openstack-ansible group https://review.openstack.org/138603 | 02:08 |
| sweston | anteaya: looking into it now ... | 02:11 |
| *** shashankhegde has quit IRC | 02:11 | |
| sweston | anteaya: yup, I will apply some formatting on the next patch | 02:12 |
| *** zz_dimtruck is now known as dimtruck | 02:13 | |
| mmedvede | anteaya: to answer you much earlier question, I am located in Texas, US | 02:14 |
| sweston | anteaya: I am about out of time for today, though. Ok if i push it first thing in the morning? | 02:14 |
| *** yamamoto_ has joined #openstack-infra | 02:14 | |
| *** shashankhegde has joined #openstack-infra | 02:15 | |
| anteaya | sweston: of course, and thank you | 02:16 |
| anteaya | mmedvede: you are in texas, why is 20:00 to 01:00 utc your perferred meeting times? | 02:17 |
| *** shashankhegde has quit IRC | 02:17 | |
| *** heyongli has quit IRC | 02:17 | |
| anteaya | s/is/are | 02:17 |
| *** Ryan_Lane has joined #openstack-infra | 02:17 | |
| *** ayoung has joined #openstack-infra | 02:17 | |
| *** Masahiro has quit IRC | 02:18 | |
| sweston | anteaya: You bet! | 02:18 |
| mmedvede | anteaya: that time window translates into time after my productive time is over :) | 02:19 |
| mmedvede | after 15:00 local time | 02:19 |
| anteaya | mmedvede: ah okay well the goal of that thread is to find a time that works for apac and eu folks | 02:19 |
| anteaya | folks that can't currently make a meeting during some interpretation of daytime for them | 02:20 |
| *** hdd has joined #openstack-infra | 02:20 | |
| *** Masahiro has joined #openstack-infra | 02:21 | |
| mmedvede | anteaya: yes, I understand. I just provided my input. I would go with whatever works best for them | 02:21 |
| *** otter768 has joined #openstack-infra | 02:21 | |
| *** patrickeast_ has quit IRC | 02:22 | |
| anteaya | okay thanks | 02:23 |
| *** ddieterly has quit IRC | 02:24 | |
| *** ddieterly has joined #openstack-infra | 02:25 | |
| *** teran has joined #openstack-infra | 02:25 | |
| *** hdd has quit IRC | 02:25 | |
| *** hdd has joined #openstack-infra | 02:25 | |
| *** dkranz has quit IRC | 02:26 | |
| *** otter768 has quit IRC | 02:26 | |
| *** ryanpetrello has joined #openstack-infra | 02:26 | |
| *** teran has quit IRC | 02:29 | |
| *** ddieterly has quit IRC | 02:29 | |
| *** ryanpetrello has quit IRC | 02:31 | |
| *** heyongli has joined #openstack-infra | 02:31 | |
| *** dims_ has quit IRC | 02:36 | |
| *** weshay has quit IRC | 02:36 | |
| *** ddieterly has joined #openstack-infra | 02:41 | |
| *** rushiagr_away is now known as rushiagr | 02:42 | |
| *** shashankhegde has joined #openstack-infra | 02:46 | |
| *** Masahiro has quit IRC | 02:52 | |
| *** wenlock1 has joined #openstack-infra | 02:52 | |
| *** Masahiro has joined #openstack-infra | 02:53 | |
| *** salv-orlando has quit IRC | 02:53 | |
| *** esker has joined #openstack-infra | 02:57 | |
| *** dimtruck is now known as zz_dimtruck | 02:57 | |
| *** Ryan_Lane has quit IRC | 02:57 | |
| *** shashankhegde has quit IRC | 02:58 | |
| *** Masahiro has quit IRC | 02:59 | |
| *** ryanpetrello has joined #openstack-infra | 03:02 | |
| *** Ryan_Lane has joined #openstack-infra | 03:03 | |
| *** ivar-laz_ has quit IRC | 03:03 | |
| *** ivar-lazzaro has joined #openstack-infra | 03:04 | |
| *** fandi has quit IRC | 03:04 | |
| *** ivar-lazzaro has quit IRC | 03:09 | |
| *** ryanpetrello has quit IRC | 03:12 | |
| *** Longgeek has joined #openstack-infra | 03:14 | |
| *** Longgeek has quit IRC | 03:19 | |
| *** mrodden has joined #openstack-infra | 03:20 | |
| *** Masahiro has joined #openstack-infra | 03:22 | |
| *** ivar-lazzaro has joined #openstack-infra | 03:24 | |
| *** teran has joined #openstack-infra | 03:25 | |
| *** ivar-lazzaro has quit IRC | 03:26 | |
| *** rushiagr is now known as rushiagr_away | 03:27 | |
| *** ivar-lazzaro has joined #openstack-infra | 03:27 | |
| *** avozza is now known as zz_avozza | 03:27 | |
| *** Sukhdev has joined #openstack-infra | 03:30 | |
| *** teran has quit IRC | 03:30 | |
| *** ryanpetrello has joined #openstack-infra | 03:31 | |
| *** david-ly_ has joined #openstack-infra | 03:32 | |
| *** zz_dimtruck is now known as dimtruck | 03:32 | |
| *** wenlock1 has quit IRC | 03:33 | |
| *** david-lyle_afk has quit IRC | 03:33 | |
| *** hdd has quit IRC | 03:33 | |
| *** dims has joined #openstack-infra | 03:36 | |
| *** changbl has joined #openstack-infra | 03:37 | |
| fungi | clarkb: as dims pointed out a little while ago and in his update to bug 1398500, oslotest-1.3.0-py2.py3-none-any.whl was missing on pypi.ord.openstack.org | 03:38 |
| uvirtbot | Launchpad bug 1398500 in openstack-ci "HTTP error 404 while getting oslotest-1.3.0" [High,Fix released] https://launchpad.net/bugs/1398500 | 03:38 |
| fungi | i've added it manually and started a full bandersnatch refresh there | 03:38 |
| *** nosnos has quit IRC | 03:39 | |
| *** mmaglana has quit IRC | 03:41 | |
| *** shashankhegde has joined #openstack-infra | 03:41 | |
| *** ivar-laz_ has joined #openstack-infra | 03:41 | |
| *** dims has quit IRC | 03:42 | |
| clarkb | so bandersnatch is randomly removing it then | 03:43 |
| clarkb | I checked prior to afking it was there... | 03:43 |
| *** jamielennox is now known as jamielennox|away | 03:44 | |
| *** achanda has joined #openstack-infra | 03:44 | |
| *** ivar-lazzaro has quit IRC | 03:45 | |
| *** ivar-laz_ has quit IRC | 03:46 | |
| *** harlowja_ is now known as harlowja_away | 03:47 | |
| clarkb | I think we dont want the full refresh | 03:47 |
| *** jaypipes has joined #openstack-infra | 03:48 | |
| *** koolhead17 has joined #openstack-infra | 03:49 | |
| *** koolhead17 has joined #openstack-infra | 03:49 | |
| *** AlexF has joined #openstack-infra | 03:49 | |
| *** _et has quit IRC | 03:51 | |
| *** rushiagr_away is now known as rushiagr | 03:53 | |
| *** ddieterly has quit IRC | 03:55 | |
| *** ddieterly has joined #openstack-infra | 03:56 | |
| *** shashankhegde has quit IRC | 03:56 | |
| *** AlexF has quit IRC | 03:59 | |
| *** ddieterly has quit IRC | 04:00 | |
| *** dannywilson has joined #openstack-infra | 04:01 | |
| *** hdd has joined #openstack-infra | 04:02 | |
| *** dannywilson has quit IRC | 04:05 | |
| *** nosnos has joined #openstack-infra | 04:19 | |
| *** Ryan_Lane has quit IRC | 04:20 | |
| *** jamielennox|away is now known as jamielennox | 04:20 | |
| *** otter768 has joined #openstack-infra | 04:22 | |
| *** thedodd has joined #openstack-infra | 04:26 | |
| *** ddieterly has joined #openstack-infra | 04:26 | |
| *** teran has joined #openstack-infra | 04:27 | |
| *** otter768 has quit IRC | 04:27 | |
| *** chandankumar has joined #openstack-infra | 04:28 | |
| *** ddieterly has quit IRC | 04:31 | |
| *** teran has quit IRC | 04:31 | |
| *** MaxV has joined #openstack-infra | 04:39 | |
| *** achanda has quit IRC | 04:43 | |
| *** achanda has joined #openstack-infra | 04:43 | |
| *** achanda has quit IRC | 04:46 | |
| *** achanda_ has joined #openstack-infra | 04:46 | |
| *** ryanpetrello has quit IRC | 04:47 | |
| *** david-ly_ has quit IRC | 04:48 | |
| *** dannywilson has joined #openstack-infra | 04:48 | |
| *** david-lyle_afk has joined #openstack-infra | 04:49 | |
| *** amotoki has joined #openstack-infra | 04:53 | |
| *** marcusvrn has quit IRC | 04:55 | |
| *** marcusvrn has joined #openstack-infra | 04:58 | |
| *** chandankumar has quit IRC | 05:00 | |
| *** Longgeek has joined #openstack-infra | 05:03 | |
| *** marcusvrn has quit IRC | 05:03 | |
| *** Guest79999 has quit IRC | 05:04 | |
| *** jerryz has quit IRC | 05:04 | |
| *** Longgeek has quit IRC | 05:08 | |
| *** fandi has joined #openstack-infra | 05:12 | |
| *** wenlock1 has joined #openstack-infra | 05:13 | |
| *** AlexF has joined #openstack-infra | 05:15 | |
| *** hdd has quit IRC | 05:15 | |
| *** wenlock2 has joined #openstack-infra | 05:16 | |
| *** jerryz has joined #openstack-infra | 05:17 | |
| *** wenlock2 has quit IRC | 05:18 | |
| *** wenlock1 has quit IRC | 05:18 | |
| *** marcusvrn has joined #openstack-infra | 05:19 | |
| *** shashankhegde has joined #openstack-infra | 05:19 | |
| *** AlexF has quit IRC | 05:20 | |
| *** wenlock has quit IRC | 05:22 | |
| openstackgerrit | Kashyap Kopparam proposed openstack-infra/project-config: EC2 compute driver that uses BOTO API to communicate with EC2 https://review.openstack.org/138629 | 05:22 |
| *** achanda_ has quit IRC | 05:23 | |
| *** talluri has joined #openstack-infra | 05:25 | |
| *** ddieterly has joined #openstack-infra | 05:27 | |
| *** teran has joined #openstack-infra | 05:27 | |
| *** fandi has quit IRC | 05:29 | |
| *** Sukhdev has quit IRC | 05:29 | |
| *** marun has quit IRC | 05:30 | |
| *** ddieterly has quit IRC | 05:32 | |
| *** teran has quit IRC | 05:33 | |
| *** Longgeek has joined #openstack-infra | 05:33 | |
| *** sarob has joined #openstack-infra | 05:36 | |
| *** stevemar has quit IRC | 05:36 | |
| *** pcrews has quit IRC | 05:37 | |
| *** thedodd has quit IRC | 05:38 | |
| openstackgerrit | Kashyap Kopparam proposed openstack-infra/project-config: EC2 compute driver that uses BOTO API to communicate with EC2 https://review.openstack.org/138629 | 05:39 |
| *** odyi has quit IRC | 05:43 | |
| *** pcrews has joined #openstack-infra | 05:45 | |
| *** shashankhegde has quit IRC | 05:47 | |
| *** unicell has quit IRC | 05:47 | |
| *** Sukhdev has joined #openstack-infra | 05:50 | |
| *** achanda has joined #openstack-infra | 05:51 | |
| *** rmcall has joined #openstack-infra | 05:52 | |
| *** rmcall has quit IRC | 05:52 | |
| *** teran has joined #openstack-infra | 05:52 | |
| *** Masahiro has quit IRC | 05:52 | |
| *** Masahiro has joined #openstack-infra | 05:53 | |
| *** rmcall has joined #openstack-infra | 05:54 | |
| *** achanda has quit IRC | 05:55 | |
| *** rmcall_ has joined #openstack-infra | 05:59 | |
| *** rmcall has quit IRC | 06:02 | |
| *** sarob has quit IRC | 06:02 | |
| *** rmcall_ has quit IRC | 06:04 | |
| *** rmcall has joined #openstack-infra | 06:04 | |
| *** Masahiro has quit IRC | 06:09 | |
| *** Masahiro has joined #openstack-infra | 06:10 | |
| *** Ark has joined #openstack-infra | 06:15 | |
| *** Ark is now known as Guest13608 | 06:15 | |
| *** shashankhegde has joined #openstack-infra | 06:16 | |
| *** Guest13608 has quit IRC | 06:19 | |
| *** Longgeek has quit IRC | 06:20 | |
| *** k4n0 has joined #openstack-infra | 06:20 | |
| *** Longgeek has joined #openstack-infra | 06:22 | |
| *** otter768 has joined #openstack-infra | 06:23 | |
| *** chandankumar has joined #openstack-infra | 06:26 | |
| *** harlowja_at_home has joined #openstack-infra | 06:26 | |
| *** ddieterly has joined #openstack-infra | 06:27 | |
| *** otter768 has quit IRC | 06:28 | |
| *** ddieterly has quit IRC | 06:32 | |
| *** rlandy has joined #openstack-infra | 06:32 | |
| *** gyee_ has quit IRC | 06:36 | |
| *** nosnos has quit IRC | 06:38 | |
| openstackgerrit | Mehdi Abaakouk proposed openstack-infra/project-config: Run ceilometer rally scenarios https://review.openstack.org/132650 | 06:39 |
| *** yaguang_ has joined #openstack-infra | 06:39 | |
| *** yaguang has quit IRC | 06:40 | |
| *** nosnos has joined #openstack-infra | 06:40 | |
| *** pcrews has quit IRC | 06:42 | |
| *** pcrews has joined #openstack-infra | 06:43 | |
| *** sputnik13 has joined #openstack-infra | 06:43 | |
| *** dannywilson has quit IRC | 06:45 | |
| *** sandywalsh_ has joined #openstack-infra | 06:46 | |
| *** simonmcc_ has quit IRC | 06:47 | |
| *** sandywalsh has quit IRC | 06:47 | |
| *** simonmcc_ has joined #openstack-infra | 06:47 | |
| *** simonmcc_ has quit IRC | 06:47 | |
| *** simonmcc_ has joined #openstack-infra | 06:47 | |
| *** Longgeek has quit IRC | 06:49 | |
| *** yfried is now known as yfried|afk | 06:54 | |
| *** _nadya_ has joined #openstack-infra | 06:54 | |
| *** shashankhegde has quit IRC | 06:55 | |
| *** vigneshvar has joined #openstack-infra | 06:55 | |
| *** teran has quit IRC | 07:01 | |
| *** Longgeek has joined #openstack-infra | 07:02 | |
| *** MaxV has quit IRC | 07:02 | |
| *** rmcall_ has joined #openstack-infra | 07:05 | |
| *** rmcall has quit IRC | 07:08 | |
| *** rmcall_ is now known as rmcall | 07:08 | |
| *** ildikov has quit IRC | 07:08 | |
| *** koolhead17 has quit IRC | 07:11 | |
| *** koolhead17 has joined #openstack-infra | 07:12 | |
| *** wuhg has joined #openstack-infra | 07:13 | |
| *** yfried|afk is now known as yfried | 07:14 | |
| *** david-ly_ has joined #openstack-infra | 07:15 | |
| *** david-lyle_afk has quit IRC | 07:16 | |
| *** koolhead17 has quit IRC | 07:16 | |
| *** teran has joined #openstack-infra | 07:17 | |
| *** rmcall has quit IRC | 07:19 | |
| *** pcrews has quit IRC | 07:20 | |
| *** teran has quit IRC | 07:21 | |
| *** jgallard has joined #openstack-infra | 07:23 | |
| *** mrunge has joined #openstack-infra | 07:25 | |
| *** mrunge has quit IRC | 07:25 | |
| *** Masahiro has quit IRC | 07:27 | |
| *** ddieterly has joined #openstack-infra | 07:27 | |
| *** belmoreira has joined #openstack-infra | 07:28 | |
| *** isaacb has joined #openstack-infra | 07:29 | |
| *** Masahiro has joined #openstack-infra | 07:31 | |
| *** ddieterly has quit IRC | 07:32 | |
| *** mrunge has joined #openstack-infra | 07:33 | |
| openstackgerrit | Merged openstack-infra/project-config: Add ironic-discoverd to #openstack-ironic IRC channel https://review.openstack.org/137136 | 07:34 |
| *** afazekas has joined #openstack-infra | 07:39 | |
| *** yfried is now known as yfried|afk | 07:41 | |
| *** amuller has joined #openstack-infra | 07:45 | |
| *** ildikov has joined #openstack-infra | 07:50 | |
| *** openstackgerrit has quit IRC | 07:50 | |
| *** openstackgerrit has joined #openstack-infra | 07:50 | |
| *** Sukhdev has quit IRC | 07:50 | |
| *** alla__ has joined #openstack-infra | 07:52 | |
| *** koolhead17 has joined #openstack-infra | 07:57 | |
| *** AlexF has joined #openstack-infra | 07:58 | |
| *** Masahiro has quit IRC | 07:59 | |
| *** yfried|afk is now known as yfried | 08:02 | |
| *** Masahiro has joined #openstack-infra | 08:02 | |
| *** teran has joined #openstack-infra | 08:03 | |
| *** _nadya_ has quit IRC | 08:07 | |
| *** armax has quit IRC | 08:10 | |
| *** AlexF has quit IRC | 08:18 | |
| *** AlexF has joined #openstack-infra | 08:18 | |
| *** MaxV has joined #openstack-infra | 08:20 | |
| *** harlowja_at_home has quit IRC | 08:21 | |
| *** AlexF has quit IRC | 08:23 | |
| *** otter768 has joined #openstack-infra | 08:24 | |
| *** salv-orlando has joined #openstack-infra | 08:25 | |
| *** tnovacik has joined #openstack-infra | 08:27 | |
| *** ddieterly has joined #openstack-infra | 08:27 | |
| *** vigneshvar_ has joined #openstack-infra | 08:28 | |
| *** ashaeron has joined #openstack-infra | 08:29 | |
| *** otter768 has quit IRC | 08:29 | |
| *** ddieterly has quit IRC | 08:32 | |
| *** vigneshvar has quit IRC | 08:32 | |
| *** Longgeek has quit IRC | 08:33 | |
| *** Hal has joined #openstack-infra | 08:33 | |
| *** Hal is now known as Guest42385 | 08:34 | |
| *** Longgeek has joined #openstack-infra | 08:34 | |
| *** ildikov has quit IRC | 08:34 | |
| *** jcoufal has joined #openstack-infra | 08:38 | |
| *** cristov_mac has joined #openstack-infra | 08:42 | |
| *** cristov_mac has quit IRC | 08:45 | |
| *** cristov__ has joined #openstack-infra | 08:46 | |
| *** cristov__ has quit IRC | 08:46 | |
| *** ihrachyshka has joined #openstack-infra | 08:47 | |
| *** Longgeek has quit IRC | 08:49 | |
| *** ildikov has joined #openstack-infra | 08:50 | |
| *** Longgeek has joined #openstack-infra | 08:53 | |
| *** jpich has joined #openstack-infra | 08:53 | |
| *** dims has joined #openstack-infra | 09:03 | |
| *** sarob has joined #openstack-infra | 09:03 | |
| *** markus_z has joined #openstack-infra | 09:04 | |
| *** ihrachyshka has quit IRC | 09:05 | |
| *** andreykurilin_ has joined #openstack-infra | 09:06 | |
| *** sarob has quit IRC | 09:08 | |
| *** dims has quit IRC | 09:08 | |
| *** groknix has joined #openstack-infra | 09:08 | |
| *** ociuhandu has quit IRC | 09:10 | |
| *** joa has quit IRC | 09:13 | |
| *** jlibosva has joined #openstack-infra | 09:13 | |
| *** koolhead17 has quit IRC | 09:15 | |
| *** koolhead17 has joined #openstack-infra | 09:15 | |
| *** mpaolino has joined #openstack-infra | 09:17 | |
| *** kkkkkkk has joined #openstack-infra | 09:17 | |
| *** kkkkkkk has left #openstack-infra | 09:17 | |
| jogo | heyongli: yeah post the patches to tempest. I am not sure if that is the correct home or not for those tests, but it will be easier to figure that out once the patches are up | 09:19 |
| jogo | heyongli: in fact you can start posting those before the CI server is ready | 09:19 |
| heyongli | jogo, thanks, i got | 09:19 |
| *** koolhead17 has quit IRC | 09:20 | |
| jogo | heyongli: once you post the patches let me know and I will take a look | 09:20 |
| heyongli | jogo, the log server is ready soon. sure , i will ping you when post it. | 09:20 |
| jogo | heyongli: great | 09:20 |
| heyongli | jogo, does third-party ci test case should be upstream? | 09:20 |
| jogo | heyongli: yes, I think you should upstream your PCI test case | 09:21 |
| heyongli | jogo, ok , we should find a way in 'vir' env , not running them. | 09:22 |
| jogo | heyongli: that way others can run it, we can see exactly what you are testing and others can collaborate | 09:22 |
| jogo | heyongli: 'vir' env? | 09:22 |
| grue_pm | heyongli: heya - hows your irc going? | 09:22 |
| heyongli | jogo, got it. 'virt' i mean the running in VM. | 09:22 |
| *** berendt has joined #openstack-infra | 09:23 | |
| heyongli | grue_pm, znc is working . irc , the log can not be high light, i'm fighting with it. | 09:23 |
| grue_pm | heyongli: ah either xchat or the terminal type thing | 09:24 |
| jogo | heyongli: ahh that is straight forward tempest has mechanisms to disable specific tests | 09:24 |
| *** e0ne has joined #openstack-infra | 09:24 | |
| jogo | heyongli: there are flags you can set in tempest to control what is run | 09:24 |
| grue_pm | heyongli: what are you running xchat in/from | 09:24 |
| heyongli | jogo, i'm not expert of tempest, but seems cool. | 09:24 |
| heyongli | grue_pm, i don't understand your question, i | 09:25 |
| heyongli | i running in linux | 09:25 |
| *** ZZelle has quit IRC | 09:25 | |
| heyongli | gui , grue_pm | 09:25 |
| grue_pm | Okay so what terminal type under? | 09:25 |
| jogo | heyongli: for now, I would recommend posting your tests as is to a tempest patch as a work in progress | 09:25 |
| *** jp_at_hp has joined #openstack-infra | 09:25 | |
| *** ZZelle has joined #openstack-infra | 09:26 | |
| heyongli | jogo, got it. | 09:26 |
| grue_pm | ah okay | 09:26 |
| jogo | heyongli: and we can discuss details of how to move forward from there | 09:26 |
| *** jedimike has joined #openstack-infra | 09:26 | |
| *** dizquierdo has joined #openstack-infra | 09:26 | |
| jogo | heyongli: if needed we can point you in the right direction for how to make it work in tempest | 09:26 |
| heyongli | jogo, you means the testcases we are going to post, i get it. | 09:26 |
| jogo | heyongli: yup | 09:27 |
| *** ddieterly has joined #openstack-infra | 09:27 | |
| heyongli | jogo, suppose we post 2 cases, and review and merged, then does it mean our ci can begin running the 2 test case? | 09:27 |
| heyongli | jogo, then we add more and more tests to tempest and to our third-party CI. does things working like this way? | 09:28 |
| *** alla__ has quit IRC | 09:29 | |
| jogo | heyongli: yes | 09:29 |
| heyongli | jogo, cool, i get it, even with 1 test cases, we still can online our CI, that's cool. | 09:30 |
| *** arxcruz has joined #openstack-infra | 09:30 | |
| grue_pm | heyongli: are you using xchat or the gnome verson XChat-GNOME? | 09:30 |
| heyongli | grue_pm, xchart, but not sure if it's XChat-GNOME | 09:30 |
| jogo | heyongli: yup, one test is better then 0 | 09:32 |
| *** ddieterly has quit IRC | 09:32 | |
| heyongli | jogo, cool to know. let's make it happen. | 09:32 |
| heyongli | and, thanks grue_pm , i need to leave now. but will available tomorrow. bye | 09:33 |
| *** chandan_kumar has joined #openstack-infra | 09:33 | |
| grue_pm | np I'm off too. | 09:33 |
| *** chandankumar has quit IRC | 09:36 | |
| *** ociuhandu has joined #openstack-infra | 09:36 | |
| *** BobBall_AWOL is now known as BobBall | 09:39 | |
| *** jcoufal has quit IRC | 09:40 | |
| *** alla__ has joined #openstack-infra | 09:40 | |
| *** yamamoto_ has quit IRC | 09:44 | |
| *** dannywilson has joined #openstack-infra | 09:46 | |
| *** dizquierdo is now known as dizquierdo_afk | 09:50 | |
| *** dannywilson has quit IRC | 09:50 | |
| *** belmorei_ has joined #openstack-infra | 09:54 | |
| *** _shaps_ has joined #openstack-infra | 09:56 | |
| *** belmoreira has quit IRC | 09:57 | |
| *** HeOS has joined #openstack-infra | 09:58 | |
| *** e0ne has quit IRC | 10:10 | |
| *** yamamoto_ has joined #openstack-infra | 10:11 | |
| *** alexpilotti has joined #openstack-infra | 10:12 | |
| *** e0ne has joined #openstack-infra | 10:12 | |
| *** andreykurilin_ has quit IRC | 10:14 | |
| *** ihrachyshka has joined #openstack-infra | 10:14 | |
| *** zz_avozza is now known as avozza | 10:15 | |
| *** mpaolino_ has joined #openstack-infra | 10:20 | |
| *** mpaolino_ has quit IRC | 10:20 | |
| *** mpaolino has quit IRC | 10:20 | |
| *** mpaolino has joined #openstack-infra | 10:21 | |
| *** redixin has joined #openstack-infra | 10:21 | |
| *** otter768 has joined #openstack-infra | 10:25 | |
| *** dizquierdo_afk has quit IRC | 10:26 | |
| *** pelix has joined #openstack-infra | 10:27 | |
| *** ddieterly has joined #openstack-infra | 10:27 | |
| *** Longgeek has quit IRC | 10:27 | |
| *** Masahiro has quit IRC | 10:29 | |
| jogo | https://jenkins03.openstack.org/job/gate-heat-pep8/3037/console | 10:29 |
| *** Longgeek has joined #openstack-infra | 10:30 | |
| jogo | http://pypi.dfw.openstack.org/packages/py2.py3/o/oslotest/oslotest-1.3.0-py2.py3-none-any.whl#md5=8bbde802d5a2beb79efe5752b9349b22 | 10:30 |
| jogo | SergeyLukjanov: ^ | 10:30 |
| *** otter768 has quit IRC | 10:30 | |
| *** ociuhandu has quit IRC | 10:30 | |
| *** ddieterly has quit IRC | 10:31 | |
| *** Longgeek has quit IRC | 10:35 | |
| jogo | sdague: ^ when you get up | 10:36 |
| *** jistr has joined #openstack-infra | 10:37 | |
| *** jistr is now known as jistr|trng | 10:38 | |
| jogo | ahh while a launchpad search didn't find anything google did: https://bugs.launchpad.net/openstack-ci/+bug/1398500 | 10:38 |
| uvirtbot | Launchpad bug 1398500 in openstack-ci "HTTP error 404 while getting oslotest-1.3.0" [High,Fix released] | 10:38 |
| *** ildikov has quit IRC | 10:38 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 10:39 | |
| *** redixin has quit IRC | 10:39 | |
| SergeyLukjanov | jogo, morning | 10:39 |
| *** belmorei_ has quit IRC | 10:39 | |
| jogo | SergeyLukjanov: looks like we are still seeing oslotest issues | 10:40 |
| *** Longgeek has joined #openstack-infra | 10:40 | |
| jogo | SergeyLukjanov: and its hitting the gate a lot. | 10:41 |
| SergeyLukjanov | jogo, heh, that's bad | 10:41 |
| *** yamamoto_ has quit IRC | 10:41 | |
| jogo | SergeyLukjanov: I was about to do a bunch of nova +Ws before I saw this | 10:41 |
| SergeyLukjanov | jogo, yeah, due to the http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiSFRUUCBlcnJvciA0MDQgd2hpbGUgZ2V0dGluZ1wiIEFORCBtZXNzYWdlOlwib3Nsb3Rlc3RcIiBBTkQgdGFnczpcImNvbnNvbGVcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiODY0MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDE3NTQ2MjU0NjQwfQ== | 10:41 |
| *** ldnunes has joined #openstack-infra | 10:42 | |
| SergeyLukjanov | jogo, I've been planning to do the same in sahara :) | 10:42 |
| *** jcoufal has joined #openstack-infra | 10:42 | |
| *** erlon has quit IRC | 10:43 | |
| jogo | looks like its been all in DFW since 2014-12-03T09:34:55.000 | 10:43 |
| SergeyLukjanov | jogo, yeah | 10:43 |
| jogo | SergeyLukjanov: is this related: https://review.openstack.org/#/c/138478/ | 10:44 |
| SergeyLukjanov | jogo, I've added the wheel manually | 10:45 |
| SergeyLukjanov | jogo, it should stop failing | 10:45 |
| jogo | SergeyLukjanov: thanks :) | 10:45 |
| SergeyLukjanov | jogo, but that's a very bad sign | 10:45 |
| SergeyLukjanov | fungi, clarkb, we have an issue with dfw mirror, oslotest 1.3.0 wheel 404 | 10:46 |
| SergeyLukjanov | I've added manually wheel and started full re-sync | 10:46 |
| * jogo goes back to doing some reviews while the gate is working and empty | 10:46 | |
| *** Kiall_ is now known as Kiall | 10:48 | |
| *** _nadya_ has joined #openstack-infra | 10:48 | |
| *** erlon has joined #openstack-infra | 10:48 | |
| SergeyLukjanov | jogo, and yeah, https://review.openstack.org/#/c/138478/ is related | 10:49 |
| *** derekh has joined #openstack-infra | 10:50 | |
| *** Longgeek has quit IRC | 10:50 | |
| *** Longgeek has joined #openstack-infra | 10:52 | |
| *** belmoreira has joined #openstack-infra | 10:55 | |
| *** jgallard has quit IRC | 10:56 | |
| openstackgerrit | Hugh Saunders proposed openstack-infra/project-config: Updating gating jobs for os-ansible repo https://review.openstack.org/138129 | 10:56 |
| *** teran has quit IRC | 10:59 | |
| *** teran has joined #openstack-infra | 11:01 | |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/system-config: Remove hardcoded variables from openstack_project module (Base) https://review.openstack.org/132250 | 11:02 |
| *** jp_at_hp has quit IRC | 11:02 | |
| *** e0ne has quit IRC | 11:02 | |
| *** e0ne_ has joined #openstack-infra | 11:02 | |
| *** e0ne_ has quit IRC | 11:02 | |
| *** e0ne has joined #openstack-infra | 11:03 | |
| *** dizquierdo has joined #openstack-infra | 11:08 | |
| openstackgerrit | Kashyap Kopparam proposed openstack-infra/project-config: EC2 compute driver that uses BOTO API to communicate with EC2 https://review.openstack.org/138629 | 11:11 |
| *** chandan_kumar has quit IRC | 11:11 | |
| *** Guest42385 has quit IRC | 11:17 | |
| *** alla__ has quit IRC | 11:19 | |
| *** mase_x200 has joined #openstack-infra | 11:21 | |
| *** andreaf has joined #openstack-infra | 11:22 | |
| *** k4n0 has quit IRC | 11:23 | |
| *** ddieterly has joined #openstack-infra | 11:27 | |
| *** alla__ has joined #openstack-infra | 11:28 | |
| *** tomoe__ has joined #openstack-infra | 11:28 | |
| *** sputnik13 has quit IRC | 11:29 | |
| *** Masahiro has joined #openstack-infra | 11:30 | |
| *** tomoe__ has quit IRC | 11:30 | |
| *** tomoe__ has joined #openstack-infra | 11:30 | |
| *** dims has joined #openstack-infra | 11:30 | |
| *** tomoe_ has quit IRC | 11:31 | |
| *** ddieterly has quit IRC | 11:32 | |
| *** adalbas has joined #openstack-infra | 11:33 | |
| *** Masahiro has quit IRC | 11:34 | |
| *** yamahata has joined #openstack-infra | 11:38 | |
| *** pc_m has joined #openstack-infra | 11:40 | |
| *** yamamoto has joined #openstack-infra | 11:42 | |
| *** _nadya_ has quit IRC | 11:46 | |
| *** yamamoto has quit IRC | 11:47 | |
| *** denis_makogon has joined #openstack-infra | 11:52 | |
| *** yamahata has quit IRC | 11:57 | |
| *** aysyd has joined #openstack-infra | 11:58 | |
| *** _nadya_ has joined #openstack-infra | 11:58 | |
| *** chandankumar has joined #openstack-infra | 12:05 | |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/system-config: Remove hardcoded variables from openstack_project module (Kerberos) https://review.openstack.org/138692 | 12:12 |
| *** asettle has quit IRC | 12:15 | |
| sdague | SergeyLukjanov: you might have enough zuul info to help with this | 12:17 |
| sdague | or maybe gerrit info | 12:17 |
| *** julim has joined #openstack-infra | 12:18 | |
| sdague | is it possible to make gerrit not dump the V column on a commit message change? | 12:18 |
| SergeyLukjanov | sdague, hm | 12:19 |
| SergeyLukjanov | sdague, it's on the gerrit side | 12:19 |
| sdague | ok | 12:19 |
| SergeyLukjanov | sdague, I'm not sure how this part is done | 12:19 |
| sdague | I remember that at one point it was configured to not reset CR values on commit message changes | 12:19 |
| sdague | but I know I'm guilty of leaving a bad commit message for a fix go in because losing another 2 hrs to add the bug reference isn't worth it | 12:20 |
| *** funzo has quit IRC | 12:20 | |
| *** otter768 has joined #openstack-infra | 12:26 | |
| *** ddieterly has joined #openstack-infra | 12:27 | |
| *** asettle has joined #openstack-infra | 12:28 | |
| *** ociuhandu has joined #openstack-infra | 12:28 | |
| openstackgerrit | Merged openstack-infra/zuul: Remove docutils pin https://review.openstack.org/128395 | 12:28 |
| *** Longgeek has quit IRC | 12:28 | |
| *** jistr|trng has quit IRC | 12:28 | |
| *** Longgeek has joined #openstack-infra | 12:30 | |
| *** otter768 has quit IRC | 12:31 | |
| *** ddieterly has quit IRC | 12:32 | |
| *** ociuhandu has quit IRC | 12:32 | |
| hughsaunders | hey, whats the etiquette for stackforge project mailing lists? use a subject tag on an existing openstack mailing list? new launchpad list? other? (relates to os-ansible-deployment) | 12:33 |
| *** koolhead17 has joined #openstack-infra | 12:33 | |
| *** dmsimard_away is now known as dmsimard | 12:34 | |
| *** ociuhandu has joined #openstack-infra | 12:34 | |
| *** jistr has joined #openstack-infra | 12:34 | |
| *** jistr is now known as jistr|trng | 12:35 | |
| openstackgerrit | Darren Birkett proposed openstack-infra/project-config: associate openstack-ansible with correct launchpad project https://review.openstack.org/138700 | 12:35 |
| mordred | hughsaunders: use openstack-dev and put in a tag at the front, like [ansible] | 12:36 |
| hughsaunders | mordred: thanks | 12:36 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check for valid entries in gerrit/projects.yaml https://review.openstack.org/138702 | 12:39 |
| *** funzo has joined #openstack-infra | 12:40 | |
| AJaeger | jhesketh, thanks for approving my zuul patch. Could you approve the nodepool version one as well, please? https://review.openstack.org/#/c/128394/ | 12:40 |
| *** nfedotov has joined #openstack-infra | 12:43 | |
| *** odyssey4me has joined #openstack-infra | 12:43 | |
| marcusvrn | Hi! we are asking to re-enable the HDS cinder CI in the mailing list. The cinder folks have approved it. Do you how when we can get it re-enabled? | 12:44 |
| *** mbacchi has joined #openstack-infra | 12:48 | |
| Kiall | Infra folks - Looks like the bandersnatch pypi mirror issue might be back: https://jenkins06.openstack.org/job/gate-designate-docs/206/console (RAX DFW mirror) | 12:49 |
| *** ddieterly has joined #openstack-infra | 12:50 | |
| *** doug-fish has joined #openstack-infra | 12:50 | |
| * mordred facepalms | 12:50 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check for valid entries in gerrit/projects.yaml https://review.openstack.org/138702 | 12:50 |
| mordred | anybody remember when the move from pypi-mirror to bandersnatch made things better? | 12:50 |
| Kiall | nope ;) | 12:51 |
| *** chandankumar has quit IRC | 12:52 | |
| *** unicell has joined #openstack-infra | 12:53 | |
| *** weshay has joined #openstack-infra | 12:57 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Support running script against local changes https://review.openstack.org/138710 | 12:59 |
| dims | mordred: approx July 1st? - http://markmail.org/message/uyqk5zqdlp6l3i32 | 13:01 |
| *** hdd has joined #openstack-infra | 13:03 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Support running script against local changes https://review.openstack.org/138710 | 13:05 |
| *** ryanpetrello has joined #openstack-infra | 13:07 | |
| *** MaxV has quit IRC | 13:08 | |
| *** Masahiro has joined #openstack-infra | 13:08 | |
| *** dprince has joined #openstack-infra | 13:09 | |
| *** rushiagr is now known as rushiagr_away | 13:10 | |
| *** AlexF has joined #openstack-infra | 13:11 | |
| *** tomoe__ has quit IRC | 13:12 | |
| *** Masahiro has quit IRC | 13:13 | |
| *** mfink has quit IRC | 13:13 | |
| *** AlexF has quit IRC | 13:13 | |
| *** AlexF_ has joined #openstack-infra | 13:13 | |
| *** radez_g0n3 is now known as radez | 13:16 | |
| *** bswartz has quit IRC | 13:16 | |
| *** AlexF_ has quit IRC | 13:17 | |
| *** mestery has quit IRC | 13:17 | |
| *** mpaolino has quit IRC | 13:19 | |
| *** pblaho has joined #openstack-infra | 13:19 | |
| *** bknudson1 has quit IRC | 13:21 | |
| jogo | SergeyLukjanov: https://jenkins02.openstack.org/job/gate-nova-pep8/22362/console | 13:23 |
| jogo | SergeyLukjanov: something is funny still | 13:23 |
| jogo | sdague: ^ | 13:23 |
| mordred | jogo: funny like haha? | 13:24 |
| jogo | mordred: no :( | 13:24 |
| *** amuller_ has joined #openstack-infra | 13:24 | |
| jogo | mordred: funny like https://bugs.launchpad.net/openstack-ci/+bug/1398500 keeps coming back | 13:24 |
| uvirtbot | Launchpad bug 1398500 in openstack-ci "HTTP error 404 while getting oslotest-1.3.0" [High,Fix released] | 13:24 |
| *** amuller has quit IRC | 13:24 | |
| *** jgallard has joined #openstack-infra | 13:24 | |
| jogo | hmm I wonder of that job is just too old | 13:25 |
| mordred | jogo: yeah. that doesn't sound funny | 13:25 |
| sdague | yeh, this is bandersnatch is crap problem | 13:25 |
| jogo | I don't think that its too old, SergeyLukjanov fixed it once | 13:26 |
| jogo | oh I see Kiall found it too | 13:26 |
| *** isviridov is now known as isviridov_away | 13:26 | |
| sdague | so, honestly, at this point bandersnatch seems less reliable than hitting pip upstream directly | 13:26 |
| Kiall | lol - wasn't it better for a while? Maybe revert to whatever version didn't have the bug? | 13:27 |
| jogo | sdague: well it fails more consistently which is nice | 13:27 |
| sdague | I guess | 13:28 |
| jogo | haha | 13:28 |
| *** tomoe_ has joined #openstack-infra | 13:28 | |
| *** HeOS_ has quit IRC | 13:28 | |
| jogo | this is fun, because nova unit tests are wedged still | 13:28 |
| openstackgerrit | Dmitry Teselkin proposed openstack-infra/zuul: Whitelist for gerrit events https://review.openstack.org/137634 | 13:28 |
| sdague | there was a long conversation yesterday about it. It's a known race, it didn't sound like anyone was working on it | 13:28 |
| sdague | though I think it's only whl files that are getting missed | 13:29 |
| jogo | sdague: clarkb has what appears to be a workaround patch | 13:29 |
| sdague | jogo: where? | 13:29 |
| Kiall | Don't we have a bunch of python developers who might be able to figure it out? ;) | 13:29 |
| jogo | 2014-12-03 12:40:36.702 | HTTP error 404 while getting http://pypi.DFW.openstack.org/packages/py2.py3/o/oslotest/oslotest-1.3.0-py2.py3-none-any.whl#md5=8bbde802d5a2beb79efe5752b9349b22 (from http://pypi.DFW.openstack.org/simple/oslotest/) | 13:29 |
| jogo | derp | 13:29 |
| jogo | https://review.openstack.org/#/c/138478/ | 13:29 |
| sdague | Kiall: not only that, the folks that work on this stuff are in channel | 13:29 |
| Kiall | ;) | 13:29 |
| jogo | sdague: ^ looks like it is related and should help (although haven't 100% confirmed with clarkb) | 13:30 |
| sdague | approved | 13:30 |
| jogo | sdague: looks like that should help future cases. but not get us out of the current jam | 13:31 |
| jogo | SergeyLukjanov tried manually uploading the wheel | 13:32 |
| jogo | and the failure went away for 2 or 3 hours | 13:32 |
| jogo | http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiZXJyb3IgNDA0XCIgQU5EIHRhZ3M6XCJjb25zb2xlLmh0bWxcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiMTQ0MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDE3NjEzNTQ2MDE3LCJtb2RlIjoiIiwiYW5hbHl6ZV9maWVsZCI6IiJ9 | 13:32 |
| jogo | so that is fun | 13:32 |
| *** hdd has quit IRC | 13:33 | |
| *** hdd has joined #openstack-infra | 13:33 | |
| jogo | fun in the way that makes mordred cringe | 13:33 |
| AJaeger | sdague, anteaya: Since I saw again a misspelled entry in gerrit/projects.yaml, I created https://review.openstack.org/138702 - this should help our reviews... | 13:34 |
| *** jaypipes is now known as jaypipes-afk | 13:34 | |
| *** ddieterly has quit IRC | 13:36 | |
| *** ddieterly has joined #openstack-infra | 13:36 | |
| *** ryanpetrello has quit IRC | 13:37 | |
| *** hogepodge has quit IRC | 13:37 | |
| *** hogepodge has joined #openstack-infra | 13:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Upload wheels before sdists https://review.openstack.org/138478 | 13:38 |
| dhellmann | hrm, the mirror issue with oslotest broke the job that was trying to publish the 1.5.0 release of oslo.messaging: http://logs.openstack.org/5e/5ed43ae50ad6ef3539d029e4f72dcee1da94a221/release/oslo.messaging-tarball/16258b9/console.html | 13:39 |
| *** yolanda has quit IRC | 13:39 | |
| *** avozza is now known as zz_avozza | 13:39 | |
| *** yolanda has joined #openstack-infra | 13:39 | |
| dhellmann | since it's a release queue job, it just failed silently. I wonder if we should have notification of those failures sent somewhere. | 13:39 |
| sileht | fungi, hi, can we restart this job manually ? | 13:39 |
| *** oomichi has quit IRC | 13:40 | |
| *** bknudson has joined #openstack-infra | 13:40 | |
| *** koolhead17 has quit IRC | 13:41 | |
| *** amuller_ has quit IRC | 13:41 | |
| *** ddieterly has quit IRC | 13:41 | |
| *** unicell has quit IRC | 13:41 | |
| *** koolhead17 has joined #openstack-infra | 13:41 | |
| *** zz_avozza is now known as avozza | 13:41 | |
| *** mfink has joined #openstack-infra | 13:42 | |
| *** kgiusti has joined #openstack-infra | 13:42 | |
| *** ryanpetrello has joined #openstack-infra | 13:46 | |
| *** mpaolino has joined #openstack-infra | 13:46 | |
| *** koolhead17 has quit IRC | 13:46 | |
| *** mfink has quit IRC | 13:46 | |
| *** ildikov has joined #openstack-infra | 13:48 | |
| *** amuller_ has joined #openstack-infra | 13:49 | |
| openstackgerrit | Dmitry Teselkin proposed openstack-infra/zuul: Whitelist for gerrit events https://review.openstack.org/137634 | 13:49 |
| *** amuller_ is now known as amuller | 13:49 | |
| *** _nadya_ has quit IRC | 13:50 | |
| *** Hal has joined #openstack-infra | 13:50 | |
| *** ryanpetrello has quit IRC | 13:50 | |
| *** Hal has joined #openstack-infra | 13:50 | |
| *** Hal is now known as Guest83375 | 13:50 | |
| *** markmcclain has joined #openstack-infra | 13:51 | |
| *** avozza is now known as zz_avozza | 13:55 | |
| openstackgerrit | Merged openstack-infra/project-config: associate openstack-ansible with correct launchpad project https://review.openstack.org/138700 | 13:55 |
| *** zz_avozza is now known as avozza | 13:55 | |
| sdague | dims: you about? | 13:55 |
| *** jcoufal has quit IRC | 13:55 | |
| sdague | I think that for https://review.openstack.org/#/c/138514/1 | 13:55 |
| flaper87 | AJaeger: hey, I replied to your comment here: https://review.openstack.org/#/c/138309/ | 13:55 |
| dims | sdague: pong | 13:55 |
| flaper87 | thanks for the review | 13:55 |
| *** esker has quit IRC | 13:56 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/storyboard-webclient: Read user preferences from backend https://review.openstack.org/137798 | 13:56 |
| AJaeger | flaper87, still wondering about this... | 13:56 |
| sdague | we should use an err trap instead I think | 13:56 |
| dims | sdague: will try ERR handler and rework it | 13:56 |
| *** esker has joined #openstack-infra | 13:56 | |
| *** jcoufal has joined #openstack-infra | 13:56 | |
| sdague | trap some_function ERR | 13:56 |
| sdague | and write the debug function | 13:56 |
| dims | sdague: do we still want pip freeze output after the thing runs? | 13:57 |
| AJaeger | sdague, regarding 138309: Is not running this in the gate fine? | 13:57 |
| dims | sdague: per current behavior | 13:57 |
| sdague | dims: honestly, I'm not sure it's useful in the success case | 13:57 |
| sdague | but maybe | 13:57 |
| sdague | actually... | 13:58 |
| sdague | lets do that instead | 13:58 |
| dims | sdague: it's useful as a record, so we can go back and look at good vs bad | 13:58 |
| sdague | turn of errexit | 13:58 |
| openstackgerrit | Merged openstack-infra/project-config: EC2 compute driver that uses BOTO API to communicate with EC2 https://review.openstack.org/138629 | 13:58 |
| openstackgerrit | Merged openstack-infra/project-config: Create new neutron service repositories https://review.openstack.org/138475 | 13:58 |
| sdague | capture $? after the tox run | 13:58 |
| openstackgerrit | Merged openstack-infra/project-config: Make run-tox a super set of run-unittests https://review.openstack.org/137433 | 13:58 |
| sdague | explicitly exit with it at the end | 13:58 |
| dims | sdague: that sounds way better | 13:58 |
| sdague | yeh | 13:58 |
| dims | sdague: will do | 13:59 |
| *** dustins has joined #openstack-infra | 13:59 | |
| sdague | thanks | 13:59 |
| flaper87 | AJaeger: but isn't that something the project should choose? | 13:59 |
| sdague | AJaeger: honestly, I'm fine with projects having more stuff in check than gate | 13:59 |
| flaper87 | AJaeger: what would be the immediate difference here ? | 13:59 |
| flaper87 | sdague: +1 | 13:59 |
| sdague | we just can't do the other thing | 13:59 |
| sdague | where tests are only run for the first time in the gate queue | 14:00 |
| sileht | sdague, hi, do you known how to restart the pypi publish job of oslo.messaging 1.5.0 ? | 14:00 |
| AJaeger | for most stuff, we run it twice since the tree might have changed. | 14:00 |
| *** dustins_ has joined #openstack-infra | 14:00 | |
| AJaeger | This is something I still need to get my head around sometime... | 14:01 |
| AJaeger | flaper87, most important is that it'S a concise decision to not do it, it could have been omission on your part as well. | 14:01 |
| *** dalgaaf has joined #openstack-infra | 14:02 | |
| flaper87 | AJaeger: +1 | 14:02 |
| *** bswartz has joined #openstack-infra | 14:02 | |
| flaper87 | AJaeger: actually, thanks for bringing this up :) | 14:02 |
| AJaeger | flaper87, +3 | 14:03 |
| sdague | AJaeger: so honestly, we do that for the case where the merge goes wrong | 14:03 |
| *** dustins has quit IRC | 14:03 | |
| sdague | or that multiple behaviors change all at once that make the tests not work after the merge | 14:03 |
| *** isviridov_away is now known as isviridov | 14:03 | |
| sdague | it's realistically a lot frequence occurance | 14:04 |
| AJaeger | sdague, it can also happen if you remove a function that is used by the added code or similar stuff... | 14:04 |
| sdague | so if a project wants to take the risk that they can land a break like that, I think it's fine. It just means they'll have to fix it in their next commit | 14:04 |
| sdague | right | 14:04 |
| AJaeger | sdague, but other tests should catch that ... | 14:04 |
| sdague | given that we are enforcing the clean check now | 14:05 |
| *** rushiagr_away is now known as rushiagr | 14:05 | |
| AJaeger | sdague, yeah, should be minimal chance | 14:06 |
| *** dpaterson has joined #openstack-infra | 14:06 | |
| *** mjturek has joined #openstack-infra | 14:08 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check for valid entries in gerrit/projects.yaml https://review.openstack.org/138702 | 14:08 |
| *** ddieterly has joined #openstack-infra | 14:09 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check for valid entries in gerrit/projects.yaml https://review.openstack.org/138702 | 14:09 |
| sdague | yolanda: you about? I have a slightly crazier idea for - https://review.openstack.org/#/c/130768/5 | 14:09 |
| sdague | why don't we just check that apt-get is installed | 14:09 |
| *** dkliban_afk is now known as dkliban | 14:09 | |
| AJaeger | dhellmann, could you review https://review.openstack.org/138702, please? You provided the original version. | 14:10 |
| yolanda | sdague, instead of checking /etc/*-release ? | 14:10 |
| *** mestery has joined #openstack-infra | 14:10 | |
| sdague | yeh | 14:10 |
| sdague | so change is_ubuntu to uses_debs | 14:10 |
| sdague | and just check that apt-get is installed | 14:11 |
| yolanda | would that be cheaper or safer? | 14:11 |
| dhellmann | sdague: trying to figure out if there's a sane way to merge that cap revert in https://review.openstack.org/#/c/138546 - we're getting errors because some of the projects merged the caps and some didn't and so grenade is failing. | 14:11 |
| sdague | well it means any other random debian derivitives would be supported | 14:11 |
| dhellmann | sdague: to unwedge, we could disable the test job on the requirements repo, ninja merge the patch, or maybe something less drastic? | 14:11 |
| dhellmann | AJaeger: looking | 14:12 |
| yolanda | sdague, or i can add a joint function instead of checking is_ubuntu and is_debian | 14:12 |
| sdague | because if you look a the d-g logic we're just using it for selection | 14:12 |
| yolanda | if we check for /etc/*-release and we query for "ubuntu" or "debian" at same time? what do you think? shall we miss some case? | 14:12 |
| sdague | mostly I kind of hate symbolic checking for features | 14:12 |
| sdague | because in this code we actually don't care if it's actually ubuntu or debian, we just want to know that apt-get is going to run | 14:12 |
| sdague | so lets actually test for the feature we are trying to use | 14:13 |
| mestery | sdague: Thanks for merging my new repository patches for neutron! Question now: Do you have to also create the gerrit groups for me, and add me into them so I can add others? | 14:13 |
| yolanda | sdague, not only checking for apt-get but there other differences, such as ntp, log directories.... | 14:13 |
| *** AlexF has joined #openstack-infra | 14:13 | |
| sdague | yolanda: sure, but in all cases they are the same here right? | 14:14 |
| dhellmann | AJaeger: do you want to check for any missing values that are required? are any actually required? | 14:14 |
| sdague | mestery: SergeyLukjanov needs to | 14:14 |
| *** cpowell has joined #openstack-infra | 14:14 | |
| sdague | or some other infra-core | 14:14 |
| mestery | sdague: Thank you sir! | 14:14 |
| sdague | dhellmann: oh, I have a grenade backport you need | 14:15 |
| sdague | one sec, I didn't backport the first yet | 14:15 |
| AJaeger | dhellmann, I'm not aware of required values besides "project" | 14:15 |
| dhellmann | sdague: oh, the "don't uninstall oslo.config" thing? | 14:15 |
| sdague | yeh | 14:15 |
| dhellmann | sdague: cool, apevec will be happy to hear it | 14:15 |
| yolanda | sdague, if that's 100% sure that apt-get will be installed in all scenarios, i guess that it's true, i don't mind changing that | 14:15 |
| sdague | yolanda: it errs on the side of acceptance | 14:16 |
| AJaeger | dhellmann, I would prefer description to have but that'S not even everywhere today... | 14:16 |
| mestery | SergeyLukjanov: Any chance you can create some new gerrit groups for me? Ping me if you're around. | 14:16 |
| yolanda | i'd like also keep the is_ubuntu and is_debian separate functions, in case we need to discriminate that in the future | 14:16 |
| sdague | so more software will tend to work | 14:16 |
| dhellmann | AJaeger: ok, that was the only thing I saw so I'll +1 | 14:16 |
| AJaeger | dhellmann, thanks! | 14:16 |
| sdague | yolanda: sure, though honestly getting away from that makes things simpler | 14:16 |
| *** hdd has quit IRC | 14:17 | |
| *** _nadya_ has joined #openstack-infra | 14:17 | |
| yolanda | sdague, ok, i'll rewrite the joint condition, and rename is_ubuntu_or_debian to something simpler | 14:17 |
| *** mfink has joined #openstack-infra | 14:17 | |
| AJaeger | dhellmann, let me add this to my cleanup list ;) | 14:17 |
| sdague | yolanda: thanks | 14:18 |
| yolanda | anytime | 14:18 |
| dhellmann | sdague: I'll be interested to see if https://review.openstack.org/#/c/138738/ fixes it, because I think we're hitting the problem now just from different projects having or not having the caps in their local requirements :-/ | 14:18 |
| *** AlexF has quit IRC | 14:19 | |
| sdague | dhellmann: so I don't think so | 14:19 |
| *** annegent_ has joined #openstack-infra | 14:19 | |
| sdague | because if we install from git, we overwrite their requirements files | 14:19 |
| dhellmann | sdague: http://logs.openstack.org/46/138546/2/check/check-grenade-dsvm/27d4eb4/logs/grenade.sh.txt.gz#_2014-12-02_22_04_49_875 | 14:20 |
| AJaeger | dhellmann, 110 out of 497 projects have no description ;( | 14:21 |
| *** yfried is now known as yfried|afk | 14:21 | |
| dhellmann | AJaeger: I'm sure we can find a lorem impsum generator to provide some | 14:22 |
| dhellmann | ipsum | 14:22 |
| straycat | hi | 14:22 |
| AJaeger | dhellmann, want to send a patch for oslo-specs, oslo.config, oslo.messaging, oslo.rootwrap, oslo.serialization, oslo.utils, oslo.version? ;) | 14:22 |
| dhellmann | AJaeger: pffft | 14:23 |
| *** vigneshvar_ has quit IRC | 14:25 | |
| sdague | dhellmann: yeh, but it gets downgraded the first time a library gets pulled in | 14:26 |
| openstackgerrit | Sergey Skripnick proposed openstack-infra/project-config: Add job that tests rally and tempest integration https://review.openstack.org/137232 | 14:26 |
| *** mriedem has joined #openstack-infra | 14:27 | |
| dhellmann | sdague: right, that's what I mean -- grenade is not uninstalling the lib, so I'm not sure your patch will fix this particular wedge issue | 14:27 |
| *** otter768 has joined #openstack-infra | 14:27 | |
| mestery | sdague: Sorry to bother you again, but one of my commits you approved failed Jenkins, and I'm not sure why. Any ideas? https://review.openstack.org/#/c/136854/ | 14:27 |
| mestery | sdague: Almost looks like some sort of transient issue to me, unrelated to my gerrit creation change. | 14:27 |
| sdague | dhellmann: this explosion happens later | 14:27 |
| dhellmann | sdague: ok, in this case we have some projects that want >=1.4 and some that want >=1.4,<1.5 and those differences are not being smoothed out by making the requirements files the same | 14:28 |
| sdague | so here is the fail sequence | 14:28 |
| dhellmann | and so I'm not sure we can actually land any changes | 14:28 |
| *** nosnos has quit IRC | 14:28 | |
| sdague | openstack-sdk installs o.c 1.5, because it installs first | 14:28 |
| sdague | the first time we get to a server project on old, keystone, we downgrade to 1.4.x because of requirements cap | 14:29 |
| *** Longgeek has quit IRC | 14:29 | |
| sdague | PREVIOUSLY, we uninstalled oslo.config | 14:29 |
| sdague | so then we did upgrades | 14:29 |
| dhellmann | oh, so if we don't uninstall then the sdk doesn't have to install anything and we'll have 1.4? | 14:29 |
| sdague | and the keystone upgrade said ... oh hey, I need oslo.config and it's not installed, install 1.5 | 14:30 |
| sdague | however.... | 14:30 |
| sdague | ceilometer has installed itself into swift's paste pipeline | 14:30 |
| *** _nadya_ has quit IRC | 14:31 | |
| sdague | and swift upgrades before ceilometer | 14:31 |
| *** odyssey4me has left #openstack-infra | 14:31 | |
| sdague | (for no particular reason, except we picked an order) | 14:31 |
| jkt | I wonder what memory usage I can expect from Zuul with two repos and <5 jobs | 14:31 |
| sdague | so swift new starts with ceilometer old, which pkg_resources have said only work with < 1.5 | 14:31 |
| sdague | so expode | 14:31 |
| jkt | because what I'm seeing is http://tinyurl.com/nvtam5e | 14:31 |
| fungi | dhellmann: sileht: i can retrigger the oslo.messaging 1.5.0 release pipeline jobs in a moment | 14:32 |
| jkt | which looks worrysome if I consider that as a trend | 14:32 |
| dhellmann | fungi: thanks | 14:32 |
| *** otter768 has quit IRC | 14:32 | |
| sdague | which, honestly, is really the crux of the problem | 14:32 |
| sdague | ceilometer needs to not install into other people's pipelines | 14:32 |
| *** yaguang_ has quit IRC | 14:32 | |
| dhellmann | sdague: the pieces ceilometer needs to install should be in a separate lib, at least | 14:32 |
| sdague | if there is something needed there, it should live in the swift tree | 14:32 |
| dhellmann | hahahahaha | 14:32 |
| dhellmann | yeah, so for now, to fix the wedge, where we can't change ceilometer, swift, or the global requirements, you think your patch will at least prevent this from happening with oslo.config? | 14:33 |
| sdague | this should bandaid us past the existing issue... *as long* as we don't end up with discontinous requirements from old => new | 14:34 |
| openstackgerrit | Merged openstack-infra/project-config: Run beaker-rspec tests on stackforge/puppet* https://review.openstack.org/137917 | 14:34 |
| *** xyang0 has joined #openstack-infra | 14:34 | |
| jkt | fungi: is the info at http://ci.openstack.org/zuul.html which says that a 2GB RAM is OK for OpenStack's Zuul still accurate? I'm not seeing any real warning yet, just a trend at http://tinyurl.com/nvtam5e | 14:34 |
| openstackgerrit | Merged openstack-infra/project-config: Add job to publish puppet modules to the forge https://review.openstack.org/134834 | 14:34 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: Support running script against local changes https://review.openstack.org/138710 | 14:34 |
| dhellmann | sdague: yeah, we're trying to land the change to uncap all of the requirements, so there shouldn't be a continuity problem (right now) | 14:35 |
| *** _nadya_ has joined #openstack-infra | 14:35 | |
| sdague | but this is a big part of why all this is just freaking hard, and I'm growingly of the opinion that it's not worth it. Just install all the services into venvs and be done with it | 14:35 |
| fungi | huh, so oslotest-1.3.0-py2.py3-none-any.whl is on the iad, ord and region-b.geo-1 mirrors now but not dfw | 14:36 |
| fungi | SergeyLukjanov: didn't you say you just fixed dfw? | 14:36 |
| SergeyLukjanov | fungi, yup, I've fixed it and it was working ok | 14:37 |
| openstackgerrit | Merged openstack-infra/project-config: Updating gating jobs for os-ansible repo https://review.openstack.org/138129 | 14:37 |
| SergeyLukjanov | fungi, but it sounds like it's broken again | 14:37 |
| *** yaguang_ has joined #openstack-infra | 14:37 | |
| openstackgerrit | Merged openstack-infra/project-config: Make zaqarclient's functional gate voting https://review.openstack.org/138309 | 14:37 |
| fungi | SergeyLukjanov: i'll redownload it into the mirror and see if i can figure out why it keeps gettign removed | 14:37 |
| SergeyLukjanov | mestery, do you still need gerrit groups? | 14:37 |
| mestery | SergeyLukjanov: Yes please! A have a handful I need created :) | 14:38 |
| fungi | okay, it's restored to the dfw mirror now | 14:38 |
| SergeyLukjanov | fungi, I've just donwloaded it and start the full sync | 14:38 |
| *** dimtruck is now known as zz_dimtruck | 14:38 | |
| mestery | SergeyLukjanov: Paste them here is ok? | 14:38 |
| mestery | SergeyLukjanov: neutron-lbaas-core neutron-fwaas-core neutron-vpnaas-core networking-odl-core networking-odl-release | 14:38 |
| SergeyLukjanov | fungi, I think full sync removes it | 14:38 |
| SergeyLukjanov | mestery, groups are auto-created I think, I'll add you to them | 14:38 |
| mestery | SergeyLukjanov: Thanks sir! The last two are not there yet, that merge failed Jenkins and is in re-check, I;ll ping you once that one lands. Tahnsk! | 14:39 |
| openstackgerrit | Darragh Bailey proposed openstack-infra/git-review: Use 'no_proxy' env variable in addition to uppercase https://review.openstack.org/132692 | 14:39 |
| openstackgerrit | Victor Stinner proposed openstack/requirements: Drop greenio dependency https://review.openstack.org/138748 | 14:39 |
| SergeyLukjanov | mestery, done | 14:40 |
| SergeyLukjanov | mestery, you're welcome | 14:40 |
| fungi | jkt: no longer accurate, no. we run the zuul scheduler daemon on a 30g instance at the moment, which is overkill but it has more than 2gb in used memory most of the time http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=392&rra_id=all | 14:40 |
| mestery | SergeyLukjanov: Thank you so much for the help! | 14:40 |
| *** btran has quit IRC | 14:40 | |
| jkt | fungi: :))), thanks | 14:40 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check for valid entries in gerrit/projects.yaml https://review.openstack.org/138702 | 14:41 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check that descriptions exists in gerrit/projects.yaml https://review.openstack.org/138749 | 14:41 |
| *** esker has quit IRC | 14:41 | |
| fungi | jkt: we also run 8 zuul merger daemons on separate 2gb instances | 14:41 |
| AJaeger | dhellmann, I fixed all the stackforge projects to have descriptions and get checked ^^^ | 14:41 |
| *** jaypipes-afk is now known as jaypipes | 14:41 | |
| *** dkranz has joined #openstack-infra | 14:41 | |
| *** yfried|afk is now known as yfried | 14:42 | |
| dhellmann | AJaeger: I assume we'll apply that rule to all projects at some point? | 14:42 |
| AJaeger | dhellmann, my idea is to add it everywhere besides the attic | 14:42 |
| *** signed8bit has joined #openstack-infra | 14:42 | |
| jkt | fungi: yup, the picture I showed was with a merger on another machine | 14:42 |
| dhellmann | AJaeger: sure | 14:42 |
| * AJaeger doesn't want to touch 110 entries at once - and stackforge adds the most repos ;) | 14:42 | |
| fungi | i can't find any mention in the bandersnatch logs of that oslotest wheel being deleted | 14:44 |
| jkt | fungi: I don't quite understand that early graph and cached memory -- does it maybe contain data from times where merger(s) were co-located on the same machine? | 14:44 |
| fungi | jkt: possibly | 14:44 |
| fungi | checking now | 14:45 |
| fungi | jkt: nope--rebooted two months ago | 14:45 |
| *** alla__ has quit IRC | 14:46 | |
| *** mestery has quit IRC | 14:46 | |
| *** alla__ has joined #openstack-infra | 14:46 | |
| *** thedodd has joined #openstack-infra | 14:46 | |
| jkt | fungi: thanks | 14:46 |
| *** hdd has joined #openstack-infra | 14:47 | |
| openstackgerrit | Terry Howe proposed openstack-infra/project-config: Add automated pypi release job for openstacksdk https://review.openstack.org/137159 | 14:47 |
| openstackgerrit | Victor Stinner proposed openstack/requirements: Add aioeventlet dependency https://review.openstack.org/138750 | 14:47 |
| fungi | jkt: it's possible that's from high volume debug logging | 14:47 |
| jkt | ah! | 14:47 |
| jkt | of course that explains it, thanks | 14:47 |
| jkt | or could explain, anyway | 14:48 |
| mriedem | sdague: did you have a link for that oslotest mirror fix? | 14:48 |
| mriedem | or is it something not in gerrit? | 14:48 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: print pip freeze even when flake8 fails https://review.openstack.org/138514 | 14:48 |
| fungi | jkt: when we turn on gearman debug logs we can generate gigabytes of log data an hour under load | 14:48 |
| sdague | mriedem: https://github.com/openstack-infra/project-config/commit/8193e0bb022a8d7d9191d146c6c6d31800a7e725 | 14:49 |
| openstackgerrit | Terry Howe proposed openstack-infra/project-config: Add automated pypi release job for openstacksdk https://review.openstack.org/137159 | 14:49 |
| *** Longgeek has joined #openstack-infra | 14:49 | |
| sdague | here is the non merge commit - https://github.com/openstack-infra/project-config/commit/b197db0c2d7127f12c689bdb49242ef616f38e7f | 14:49 |
| mriedem | sdague: thanks | 14:49 |
| *** mfink has quit IRC | 14:50 | |
| *** mfink has joined #openstack-infra | 14:50 | |
| *** wenlock has joined #openstack-infra | 14:52 | |
| *** mase_x200 has quit IRC | 14:53 | |
| *** alla__ has quit IRC | 14:54 | |
| *** annegent_ has quit IRC | 14:54 | |
| *** skolekonov has joined #openstack-infra | 14:54 | |
| fungi | okay, i've set a trap on all our pypi mirrors to detect as soon as oslotest-1.3.0-py2.py3-none-any.whl disappears again, so that i can hopefully correlate that timestamp to something in the logs | 14:55 |
| *** eharney has joined #openstack-infra | 14:55 | |
| openstackgerrit | Merged openstack-infra/project-config: Add networking-odl project to StackForge https://review.openstack.org/136854 | 14:55 |
| *** eharney is now known as Guest31501 | 14:55 | |
| *** cnesa has joined #openstack-infra | 14:55 | |
| *** Masahiro has joined #openstack-infra | 14:57 | |
| *** krtaylor has quit IRC | 14:57 | |
| *** pcrews has joined #openstack-infra | 14:57 | |
| *** alla__ has joined #openstack-infra | 14:58 | |
| *** david-ly_ is now known as david-lyle | 15:00 | |
| *** ayoung has quit IRC | 15:00 | |
| openstackgerrit | Merged openstack-infra/infra-manual: Add instructions to set team owners in LP https://review.openstack.org/138558 | 15:01 |
| *** Guest31501 is now known as eharney | 15:01 | |
| pleia2 | good morning (so early) | 15:02 |
| *** armax has joined #openstack-infra | 15:02 | |
| *** Masahiro has quit IRC | 15:02 | |
| esp | hello, can anyone help me figure out what’s wrong with the tempest tests for this patch please https://review.openstack.org/#/c/138190/ | 15:04 |
| *** dims has quit IRC | 15:04 | |
| *** erikmwilson has joined #openstack-infra | 15:04 | |
| *** adelinatuvenie has joined #openstack-infra | 15:04 | |
| *** dims has joined #openstack-infra | 15:04 | |
| *** rickerc has quit IRC | 15:06 | |
| *** rediskin has joined #openstack-infra | 15:07 | |
| *** rediskin has left #openstack-infra | 15:07 | |
| *** redixin has joined #openstack-infra | 15:08 | |
| *** rickerc has joined #openstack-infra | 15:09 | |
| *** yamamoto has joined #openstack-infra | 15:10 | |
| *** mestery has joined #openstack-infra | 15:10 | |
| *** cnesa has quit IRC | 15:10 | |
| *** berendt has quit IRC | 15:11 | |
| *** wenlock has quit IRC | 15:12 | |
| *** esker has joined #openstack-infra | 15:12 | |
| *** cnesa has joined #openstack-infra | 15:13 | |
| *** AlexF has joined #openstack-infra | 15:14 | |
| *** bswartz has quit IRC | 15:16 | |
| *** marun has joined #openstack-infra | 15:17 | |
| *** thedodd has quit IRC | 15:18 | |
| sdague | dhellmann: so... http://lists.openstack.org/pipermail/openstack-dev/2014-December/052121.html to get the conversation rolling about the paste thing | 15:18 |
| sdague | being in a library turns out not to be any more helpful... see keystone current issues | 15:19 |
| *** zz_dimtruck is now known as dimtruck | 15:19 | |
| *** AlexF has quit IRC | 15:19 | |
| *** belmoreira has quit IRC | 15:20 | |
| dhellmann | sdague: yeah, I like the idea of the proxy classes | 15:20 |
| *** r-daneel has joined #openstack-infra | 15:21 | |
| *** jerryz has quit IRC | 15:21 | |
| fungi | dhellmann: release job series retriggered successfully https://pypi.python.org/pypi/oslo.messaging | 15:22 |
| fungi | sileht: ^ | 15:22 |
| sileht | fungi, thanks ! | 15:22 |
| mestery | SergeyLukjanov: Can you add me to the gerrit groups "networking-odl-core networking-odl-release", the other change merged, thanks! | 15:23 |
| *** MaxV has joined #openstack-infra | 15:23 | |
| openstackgerrit | Valeriy Ponomaryov proposed openstack/requirements: Upgrade tempest-lib>=0.0.2 https://review.openstack.org/138758 | 15:23 |
| dhellmann | fungi: thanks! | 15:24 |
| *** teran has quit IRC | 15:26 | |
| *** doug-fish has left #openstack-infra | 15:28 | |
| *** bswartz has joined #openstack-infra | 15:28 | |
| *** nelsnelson has joined #openstack-infra | 15:30 | |
| *** hdd has quit IRC | 15:33 | |
| fungi | mestery: so the plan with the service layer split-out was to start with empty projects and have people dump new commits into them rather than starting from git filter-branch of code in the neutron repo to preserve commit history? | 15:33 |
| zul | Did you guys get your python3.4 bug fixed? | 15:33 |
| mestery | fungi: We looked at both options, yes, and we decided to go down this path. Did we make the wrong choice? | 15:34 |
| fungi | zul: no, i've prodded the bugs | 15:34 |
| zul | fungi: whats the bug number again? | 15:34 |
| fungi | mestery: i have no idea since i don't know the criteria you were weighing for that decision. just be aware that the current plan loses the original commit history for any existing code you're planning to copy into those empty projects | 15:35 |
| fungi | zul: bug 1348954, or bug 1367907 and bug 1382607 | 15:36 |
| uvirtbot | Launchpad bug 1348954 in python3.4 "update Python3 for trusty" [Undecided,Confirmed] https://launchpad.net/bugs/1348954 | 15:36 |
| uvirtbot | Launchpad bug 1367907 in python3.4 "Segfault in gc with cyclic trash" [High,Fix released] https://launchpad.net/bugs/1367907 | 15:36 |
| uvirtbot | Launchpad bug 1382607 in python3.4 "[SRU] Backport python3.4 logging module backward incompatibility fix." [High,Fix released] https://launchpad.net/bugs/1382607 | 15:36 |
| fungi | zul: those are basically preventing us from moving our python 3 testing to trusty | 15:37 |
| zul | fungi: k | 15:37 |
| *** amitgandhinz has joined #openstack-infra | 15:37 | |
| *** berendt has joined #openstack-infra | 15:38 | |
| mestery | fungi: OK, I'll make sure everyone is aware of that and onboard, thanks for looking out for us! | 15:38 |
| *** krtaylor has joined #openstack-infra | 15:38 | |
| fungi | mestery: you're welcome! just making sure you knew the implications | 15:38 |
| *** groknix has quit IRC | 15:39 | |
| *** zz_jgrimm is now known as jgrimm | 15:39 | |
| *** amotoki_ has joined #openstack-infra | 15:43 | |
| *** baoli has quit IRC | 15:44 | |
| *** odyssey4me has joined #openstack-infra | 15:45 | |
| *** asettle has quit IRC | 15:45 | |
| *** teran has joined #openstack-infra | 15:45 | |
| *** tonytan4ever has joined #openstack-infra | 15:46 | |
| odyssey4me | Hi everyone - I can't seem to submit code for review any more. It seems that my CLA disappeared, so I've re-signed it. It's still not working. When trying to complete my profile details (address, contact number, etc) I just get 'Server Error: Cannot store contact information'. Can someone assist? | 15:46 |
| anteaya | odyssey4me: hi, let's start with your gerrit username and gerrit account id | 15:47 |
| odyssey4me | thanks anteaya: username = jesse-pretorius; account-id = 6816 | 15:48 |
| sdague | dims: also... why do we have that pipe there? | 15:48 |
| dims | sdague: guess the original intent was to capture the pep output as a separate file from the console file? | 15:49 |
| sdague | yeh, that seems weird though | 15:49 |
| sdague | fungi: any ideas - https://review.openstack.org/#/c/138514/2 | 15:49 |
| sdague | is that pipe vestigial | 15:49 |
| *** Sukhdev has joined #openstack-infra | 15:50 | |
| *** talluri has quit IRC | 15:52 | |
| *** dangers_away is now known as dangers | 15:52 | |
| *** wenlock has joined #openstack-infra | 15:52 | |
| anteaya | odyssey4me: well gerrit knows about you: https://review.openstack.org/#/q/reviewer:jesse.pretorius%2540rackspace.co.uk+status:open,n,z and as far as I can tell from looking at the CLA gerrit group, it believes you have signed the CLA | 15:53 |
| *** rmcall has joined #openstack-infra | 15:53 | |
| anteaya | odyssey4me: is that your correct email address? | 15:53 |
| fungi | sdague: agreed | 15:53 |
| sdague | dims: ok, so want to respin without the pipe line | 15:54 |
| sdague | then we can get this through | 15:54 |
| dims | sdague: ack will do | 15:54 |
| sdague | dims: thanks much for fast turn arounds | 15:54 |
| odyssey4me | anteaya: yes, as well as this one as a secondary (although this used to be the primary): https://review.openstack.org/#/q/reviewer:jesse.pretorius%2540gmail.com+status:open,n,z | 15:55 |
| *** stevemar has joined #openstack-infra | 15:55 | |
| anteaya | did you review anything after changing your email address primary status? | 15:56 |
| fungi | odyssey4me: it doesn't look like you have any duplicate accounts in gerrit, however you definitely don't have contact info filed... when was the last time you pushed a patchset into gerrit for a cla-enforced (official openstack) project? | 15:56 |
| odyssey4me | fungi: ages ago - it didn't require the contact info back then | 15:56 |
| odyssey4me | which is why I'm trying to update it, and the updating of contact information gives a server error | 15:57 |
| *** rickerc has quit IRC | 15:57 | |
| fungi | odyssey4me: that explains it. in that case you need to make sure to join the foundation as an individual member and set your foundation profile primary e-mail to be the same as your preferred e-mail address in gerrit | 15:57 |
| *** jeblair changes topic to "Discussion of OpenStack Developer Infrastructure | docs http://ci.openstack.org | bugs https://launchpad.net/openstack-ci/ | https://git.openstack.org/cgit/openstack-infra/system-config/tree/" | 15:57 | |
| odyssey4me | anteaya: yes I did - both yesterday and today: https://review.openstack.org/#/q/reviewer:%22Jesse+Pretorius+%253Cjesse.pretorius%2540rackspace.co.uk%253E%22,n,z | 15:58 |
| *** asettle has joined #openstack-infra | 15:58 | |
| fungi | i sent out details about that behavior change about a year and a half ago when it went into effect, though i'd need to hunt to find the announcement | 15:58 |
| *** amitgandhinz has quit IRC | 15:58 | |
| openstackgerrit | Hugh Saunders proposed openstack-infra/project-config: Add Syntax job for os-ansible-deployment https://review.openstack.org/138768 | 15:58 |
| odyssey4me | fungi: I've just reviewed and updated all the foundation member info. I've ensured that the primary address in the membership is the same as the primary in gerrit. | 15:59 |
| *** amitgandhinz has joined #openstack-infra | 15:59 | |
| odyssey4me | fungi: the foundation profile does warn about that, which is why I fixed it | 15:59 |
| *** patrickeast has joined #openstack-infra | 15:59 | |
| *** bswartz1 has joined #openstack-infra | 15:59 | |
| odyssey4me | also anteaya did a nice little write-up: http://anteaya.info/blog/2013/03/03/fatal-cla-echosign-contributor-agreement-is-expired/ | 15:59 |
| *** isaacb has quit IRC | 15:59 | |
| sdague | fungi: is there a zuul restart planned in the future to get merge-check running agian? | 16:00 |
| *** rickerc has joined #openstack-infra | 16:00 | |
| fungi | odyssey4me: yeah, that got replicated to some extent into https://wiki.openstack.org/wiki/CLA-FAQ as well | 16:00 |
| fungi | odyssey4me: and are you registered with the foundation as an individual member, or just with a speaker account? | 16:00 |
| *** AlexF has joined #openstack-infra | 16:00 | |
| sdague | fungi: oh... and the thing I asked SergeyLukjanov this morning. Is it possible to make the whole system non reset V columns on commit message changes? | 16:00 |
| *** bswartz has quit IRC | 16:00 | |
| odyssey4me | fungi: how do I tell? | 16:00 |
| *** marcusvrn1 has joined #openstack-infra | 16:01 | |
| *** alla__ has quit IRC | 16:01 | |
| *** AlexF has quit IRC | 16:01 | |
| *** marcusvrn has quit IRC | 16:01 | |
| fungi | reed: ^ ? is there some indication of the difference in the ui? i've only ever checked that via database queries | 16:01 |
| *** dustins_ is now known as dustins | 16:02 | |
| reed | ? | 16:02 |
| *** scottda has joined #openstack-infra | 16:02 | |
| *** AlexF has joined #openstack-infra | 16:02 | |
| *** koolhead17 has joined #openstack-infra | 16:02 | |
| odyssey4me | fungi: the profile shows no legal agreements - does that matter? | 16:02 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/system-config: Redirect stackforge.html to infra manual https://review.openstack.org/138770 | 16:02 |
| *** asselin_ has joined #openstack-infra | 16:02 | |
| fungi | odyssey4me: yeah, i think that probably means it's not an individual member account then | 16:02 |
| reed | odyssey4me, I bet you're not an individual member of the foundation | 16:02 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: print pip freeze even when flake8 fails https://review.openstack.org/138514 | 16:02 |
| reed | the UI to change that yourself is still not ready | 16:03 |
| odyssey4me | fungi: ok, how do I get that sorted - strange, I do remember doing all that a while ago | 16:03 |
| clarkb | sdague: https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_copyAllScoresIfNoCodeChange it is but we have things that check the commit message so until hacking stops being silly I think we shouldn't do that | 16:03 |
| reed | odyssey4me, let me check... pm me your email address | 16:03 |
| *** yaguang_ has quit IRC | 16:03 | |
| fungi | sdague: i believe the gerrit option it to either reset all resettable votes, or not reset any of them, and then vote categories either reset on new patchsets or carry over, but you can't matrix that to have different per-vote behaviors on regular updates vs commit message updates vs trivial rebases | 16:04 |
| *** pblaho_ has joined #openstack-infra | 16:04 | |
| *** pblaho has quit IRC | 16:04 | |
| sdague | clarkb: so I'm totally happy deleting that from hacking | 16:04 |
| clarkb | me too :) | 16:04 |
| fungi | me three...thousand | 16:05 |
| clarkb | fungi: its per label | 16:05 |
| sdague | clarkb: it's per lable? | 16:05 |
| sdague | oh, it is, that's the awesome sauce | 16:05 |
| clarkb | ya so you can so don't reset verified but reset code review | 16:05 |
| scottda | AJaeger: We just voted at the cinder meeting to allow Cinder IRC to be archived. | 16:05 |
| clarkb | s/can so/can say/ | 16:05 |
| scottda | Is there anything else that needs to be done to get this going? | 16:06 |
| sdague | will zuul understand the event difference? | 16:06 |
| sdague | or do we need to code it on that side as well | 16:06 |
| *** rickerc has quit IRC | 16:06 | |
| clarkb | sdague: oh right new patchset events will likely still be emitted | 16:07 |
| clarkb | I would have to think about that | 16:07 |
| *** rickerc has joined #openstack-infra | 16:07 | |
| sdague | is there enough detail in them to know they didn't change? | 16:07 |
| fungi | clarkb: oh, you're totally right. i thought it was per upload type | 16:08 |
| fungi | clarkb: so, yeah, we still need to solve the hacking dependency on commit message updates though | 16:08 |
| clarkb | sdague: I think not in 2.8 but yes in 2.9 | 16:08 |
| sdague | fungi: so that's easy, we delete that from hacking | 16:08 |
| clarkb | iirc 2.9 added more info to say trivialrebase: true | 16:08 |
| sdague | clarkb: ah cool | 16:08 |
| sdague | is a 2.9 upgrade somewhere on the schedule? | 16:09 |
| fungi | oh, right also we still want to rerun jobs on trivial rebases | 16:09 |
| fungi | so carrying over the zuul vote on a patchset when only the commit message changes doesn't prevent zuul from rerunning check jobs anyway | 16:09 |
| clarkb | sdague: its a thing that zaro had in early stages of planning. there is an ssh bug that needs sorting first | 16:09 |
| sdague | ok | 16:10 |
| sdague | fungi: so I'd agree, trivial rebase is probably something we still want to run tests on | 16:10 |
| *** doug-fish has joined #openstack-infra | 16:11 | |
| sdague | now, that being said, it would also remove the 'pull from gate hack' | 16:14 |
| *** SumitNaiksatam has quit IRC | 16:14 | |
| *** Longgeek has quit IRC | 16:14 | |
| fungi | this is where an authenticated api endpoint to call a dequeue method would become a lot more useful | 16:14 |
| *** Longgeek has joined #openstack-infra | 16:16 | |
| jeblair | if you ignored a new patchset in the gate, the merge would fail (and surprise zuul causing a late gate reset), so we probably don't want to ignore it in gate | 16:17 |
| jeblair | i think it would need to be per-pipeline configurable | 16:17 |
| sdague | ok, good point | 16:17 |
| odyssey4me | anteaya, reed, fungi - thank you all - I'm able to submit patches again | 16:18 |
| sdague | mostly it would be really nice to have a patch submitted, someone say "fix the commit message" and not rerun the tests. Because a lot of time we aren't fixing the commit message because of the test time overhead. | 16:18 |
| jeblair | ya | 16:19 |
| anteaya | odyssey4me: yay | 16:20 |
| sdague | jeblair: any idea if we get an event related to only commit change? | 16:21 |
| *** rlandy has quit IRC | 16:21 | |
| jeblair | sdague: i'm looking in current master to see if we might in 2.9 | 16:22 |
| *** cnesa2 has joined #openstack-infra | 16:22 | |
| openstackgerrit | Ivan Udovichenko proposed openstack-infra/system-config: Remove hardcoded variables from openstack_project module (Lists) https://review.openstack.org/138777 | 16:23 |
| *** luqas has quit IRC | 16:23 | |
| mestery | Can someone on infra-core add me to the gerrit groups "networking-odl-core networking-odl-release" please? These are for a recent stackforge repository creation. Thanks! | 16:24 |
| tykeal | anteaya: I saw that Edgar gave a +2 on the mailing list for re-enabling the OpenDaylight CI. Is that enough to get us re-enabled? | 16:25 |
| *** marcusvrn1 has quit IRC | 16:25 | |
| *** marcusvrn has joined #openstack-infra | 16:26 | |
| *** esker has quit IRC | 16:27 | |
| *** melwitt has joined #openstack-infra | 16:27 | |
| clarkb | jeblair: can you see my edits on your plan https://etherpad.openstack.org/p/third-party-openid-accounts | 16:27 |
| clarkb | jeblair: I think if we do that then we don't need new scripting for group creation | 16:27 |
| sdague | can we ninja merge - https://review.openstack.org/#/c/138463/ | 16:27 |
| clarkb | jeblair: want to mke sure it looks good to you | 16:27 |
| *** luqas has joined #openstack-infra | 16:27 | |
| sdague | it's bounced off the gate 3 times for bandersnatch having bad hacking | 16:27 |
| sdague | and it's blocking all nova patches from passing | 16:27 |
| sdague | it's currently pep8 failing on bad bandersnatch again as well | 16:28 |
| *** otter768 has joined #openstack-infra | 16:28 | |
| clarkb | sdague: looks like oslotest not hacking | 16:28 |
| *** jpich has quit IRC | 16:28 | |
| clarkb | most recent failure at least | 16:28 |
| clarkb | which yay for pypi caches... | 16:29 |
| sdague | oh, ok, now it's oslotest | 16:29 |
| sdague | anyway | 16:29 |
| sdague | it's passed check 4 times now | 16:29 |
| sdague | and it's blocking all nova patches | 16:29 |
| mriedem | i'm seeing hacking failing | 16:29 |
| mriedem | not oslotest now | 16:29 |
| mriedem | https://jenkins02.openstack.org/job/gate-nova-pep8/22366/console | 16:29 |
| *** hdd has joined #openstack-infra | 16:29 | |
| *** andreaf has quit IRC | 16:29 | |
| *** luqas has quit IRC | 16:30 | |
| sdague | well, regardless, it's actually an infrastructure failure, not actually a valid test failure | 16:30 |
| clarkb | well | 16:30 |
| clarkb | sort of | 16:30 |
| clarkb | hacking should work in every release | 16:30 |
| *** mwagner_lap has quit IRC | 16:31 | |
| clarkb | and you know we should heed failures | 16:31 |
| clarkb | infra is just making that ^ more troublesome | 16:31 |
| *** melwitt has quit IRC | 16:31 | |
| sdague | so we can bump requirements to 0.9.4 then it will be an infra failure | 16:31 |
| sdague | it's pretty semantics though, we did a hacking release to address a bug | 16:31 |
| jeblair | clarkb, fungi, sdague: https://gerrit.googlesource.com/gerrit/+/master/gerrit-server/src/main/java/com/google/gerrit/server/change/ChangeKind.java | 16:32 |
| *** isaacb has joined #openstack-infra | 16:32 | |
| sdague | jeblair: awesome | 16:32 |
| fungi | ooh! | 16:32 |
| jeblair | clarkb, fungi, sdague: i believe that is included in patchset-created events in master | 16:32 |
| *** otter768 has quit IRC | 16:32 | |
| jeblair | so i think we'll be good | 16:33 |
| fungi | mestery: done | 16:33 |
| clarkb | sdague: honestly it isn't because hacking has very strict release requirements that we violated here | 16:33 |
| clarkb | sdague: hacking is not supposed to get more or less restrictive on a point release | 16:33 |
| pleia2 | alright, infra-manual sprint things are all done, stats collected and shared on mailing lists, blog, social media; now hopefully people can & will use it :) | 16:34 |
| openstackgerrit | Sean Dague proposed openstack/requirements: update hacking to 0.9.4 to deal with oslo namespace changes https://review.openstack.org/138778 | 16:34 |
| sdague | clarkb: it didn;'t | 16:34 |
| mestery | fungi: thanks sir! | 16:34 |
| sdague | there was a bug | 16:34 |
| clarkb | didn't we make hacking less restrictive rather than fix a bug? or did I miss the bug? | 16:34 |
| sdague | you missed a bug | 16:34 |
| *** arxcruz has quit IRC | 16:34 | |
| dhellmann | clarkb: we fixed a bug | 16:34 |
| clarkb | ah ok, I thought you basically said no more checking that | 16:34 |
| mriedem | https://jenkins02.openstack.org/job/gate-nova-pep8/22366/console | 16:35 |
| dhellmann | clarkb: hacking was not properly detecting things that are modules, so it was reporting errors for import lines that are valid | 16:35 |
| mriedem | oops | 16:35 |
| sdague | the H302 module detection didn't work correctly if you imported through a second module | 16:35 |
| mriedem | clarkb: that's the bug https://bugs.launchpad.net/hacking/+bug/1398472 | 16:35 |
| uvirtbot | Launchpad bug 1398472 in hacking "H302 isn't handling oslo_concurrency namespace change" [Undecided,Fix released] | 16:35 |
| sdague | which is why we backported to the 0.9.x branch to fix the issue in the stable release | 16:35 |
| jogo | it turns out python namespace module stuff is complex | 16:35 |
| mriedem | https://review.openstack.org/#/c/138462/ was the fix | 16:35 |
| mriedem | https://review.openstack.org/#/q/I4b42d081965b6d898b178cbe9232b47cfed17d8a,n,z | 16:36 |
| sdague | under the assumption that would put that into the system quicly | 16:36 |
| sdague | and every project wouldn't have to add custom ignores | 16:36 |
| *** baoli has joined #openstack-infra | 16:36 | |
| jeblair | clarkb: plan lgtm | 16:36 |
| sdague | clarkb: you should assume we're actually good actors on the hacking release | 16:37 |
| clarkb | I dunno it seems like we have been deleting things rather than fixing them lately | 16:38 |
| clarkb | anyways only hpcloud is missing new hacking not sure why yet | 16:38 |
| jeblair | sdague: should hacking have some non-voting jobs where it runs on some projects (eg nova)? | 16:38 |
| clarkb | all 3 rax mirrors are hacking happy | 16:38 |
| sdague | jeblair: the issue was actually that oslo.concurrency got overly agressive with #noqa | 16:39 |
| clarkb | subprocess.CalledProcessError: Command '['bandersnatch', 'mirror']' returned non-zero exit status 1 is a start | 16:39 |
| dhellmann | sdague: no, that wasn't it | 16:39 |
| sdague | it should have been caught there | 16:39 |
| sdague | dhellmann: yes it was | 16:39 |
| dhellmann | sdague: the noqa statements in oslo.concurrency had nothing to do with that | 16:39 |
| jogo | jeblair: we do that on hacking master already | 16:39 |
| dhellmann | I was never able to reproduce the failures there | 16:39 |
| *** Sukhdev has quit IRC | 16:39 | |
| *** markmcclain1 has joined #openstack-infra | 16:40 | |
| *** pblaho__ has joined #openstack-infra | 16:40 | |
| jogo | jeblair: but there clearly was a missing test somewhere | 16:40 |
| sdague | dhellmann: https://review.openstack.org/#/c/138451/ | 16:40 |
| *** pblaho_ has quit IRC | 16:40 | |
| sdague | dhellmann: it might be because you are on a mac | 16:40 |
| sdague | and import stuff is goofier there | 16:40 |
| dhellmann | I'm running the tests on ubuntu | 16:40 |
| sdague | dhellmann: https://review.openstack.org/#/c/138459/ | 16:41 |
| sdague | your removal of those #noqa lines triggered exactly the issue we should have caught | 16:41 |
| dhellmann | sdague: those tests passed for me locally. I can't explain the difference. | 16:41 |
| *** koolhead17 has quit IRC | 16:41 | |
| sdague | dhellmann: so I don't know why they pass for you locally | 16:42 |
| sdague | but they clearly fail in our system | 16:42 |
| sdague | and they fail here for me | 16:42 |
| clarkb | so I am going to suggset we go back to pypi-mirror. bandersnatch is all kinds of broken | 16:42 |
| *** markmcclain has quit IRC | 16:42 | |
| clarkb | mordred: ^ | 16:42 |
| *** koolhead17 has joined #openstack-infra | 16:42 | |
| clarkb | http://paste.openstack.org/show/144067/ | 16:42 |
| sdague | anyway, my opinion is the overuse of # noqa (reverted here - https://review.openstack.org/#/c/138459/ ) is how we got here | 16:42 |
| *** ihrachyshka has quit IRC | 16:43 | |
| *** sputnik13 has joined #openstack-infra | 16:43 | |
| *** annegent_ has joined #openstack-infra | 16:43 | |
| clarkb | at least put openstack/ back in as a thing and have bandersnatch best effort next to it | 16:44 |
| sdague | but regardless how we got here | 16:44 |
| sdague | nova's been blocked on merging patches for 24hrs now | 16:44 |
| clarkb | ok I killed generation and todo again on hpcloud and have restarted a full mirror sync | 16:44 |
| *** markmcclain1 has quit IRC | 16:44 | |
| sdague | pending merging the unit test fix for this | 16:44 |
| *** mpaolino has quit IRC | 16:45 | |
| sdague | which passed check 4 times, but has gotten nailed by being on an unlucky gate node 3 times | 16:45 |
| armax | fungi, whenever convenient for you, anteaya suggested me to ping you re: https://storyboard.openstack.org/#!/story/2000041 | 16:45 |
| fungi | clarkb: this is suspected as the pypi backend query race dstufft was plannign to fix with a transaction wrapper or lock, yeah? | 16:45 |
| anteaya | tykeal: yes I see it: http://lists.openstack.org/pipermail/third-party-announce/2014-December/000127.html | 16:45 |
| clarkb | fungi: that was the initial trigger of it all. I think bandersnatch full syncs just break sometimes | 16:45 |
| clarkb | fungi: eg I think its a different bug | 16:46 |
| *** Masahiro has joined #openstack-infra | 16:46 | |
| clarkb | sdague: so back to ninja merging. Until hacking is available everywhee you will have high incidence of failure anyways | 16:46 |
| clarkb | so merging does not directly help | 16:46 |
| clarkb | at least not until bandersnatch catches up in hpcloud | 16:46 |
| clarkb | which I can ninja so will do that now | 16:46 |
| *** koolhead17 has quit IRC | 16:46 | |
| *** rlrossit has joined #openstack-infra | 16:47 | |
| *** esker has joined #openstack-infra | 16:47 | |
| anteaya | when someone can re-enable 10386 odl-jenkins OpenDaylight CI odl-openstack@opendaylight.org I would be grateful | 16:47 |
| sdague | clarkb: but then at least we'd be down to 1 random fail | 16:47 |
| sdague | vs. all patches blocked | 16:47 |
| sdague | right now they are coupled | 16:47 |
| *** belmoreira has joined #openstack-infra | 16:48 | |
| fungi | armax: taking a look | 16:48 |
| fungi | anteaya: done | 16:48 |
| anteaya | fungi: thank you | 16:48 |
| armax | fungi: thanks, much appreciated | 16:48 |
| *** esker has quit IRC | 16:49 | |
| *** Masahiro has quit IRC | 16:50 | |
| clarkb | ok hpcloud mirror ninja'd | 16:50 |
| clarkb | sdague: I can promote the nova change | 16:51 |
| *** prad has joined #openstack-infra | 16:51 | |
| sdague | well, it might have lucked out on a pep8 node that won't fail | 16:51 |
| sdague | because heat got screwed instead | 16:52 |
| sdague | so it's probably closer to merge than a promote and restarting | 16:52 |
| clarkb | ok | 16:52 |
| *** ryanpetrello has joined #openstack-infra | 16:53 | |
| *** luqas has joined #openstack-infra | 16:53 | |
| *** achanda has joined #openstack-infra | 16:54 | |
| *** ashaeron has quit IRC | 16:54 | |
| *** annegent_ has quit IRC | 16:55 | |
| *** jgallard has quit IRC | 16:55 | |
| fungi | clarkb: we saw the oslotest wheel missing from dfw earlier this morning... i replaced it and started a full refresh there, but also set up background loops on all four mirror servers to log the second it disappears again so we can hopefully see if there's some correlation to other events | 16:56 |
| *** luqas has quit IRC | 16:56 | |
| clarkb | fungi: the full refreshes are what delete the file | 16:56 |
| clarkb | fungi: since pypi has cached the wrong version of the json file | 16:56 |
| fungi | clarkb: you found at what point in the refresh it does it? | 16:56 |
| clarkb | fungi: no but dstufft explained that | 16:56 |
| fungi | oh! right, it's the fastly cdn | 16:56 |
| *** cnesa2 has quit IRC | 16:56 | |
| clarkb | fungi: part of syncing is deleting files | 16:56 |
| fungi | which caches for up to a day | 16:56 |
| clarkb | ya | 16:56 |
| *** Sukhdev has joined #openstack-infra | 16:57 | |
| *** sputnik13 has quit IRC | 16:57 | |
| clarkb | I am actually going to run a purge against the oslotest files that show up in our logs | 16:57 |
| *** luqas has joined #openstack-infra | 16:57 | |
| dhellmann | clarkb, fungi : would this be fixed by me releasing a newer version of oslotest? | 16:57 |
| clarkb | dhellmann: not necessarily | 16:57 |
| fungi | dhellmann: possibly, or possibly made worse | 16:57 |
| clarkb | dhellmann: we could hit the same race again | 16:57 |
| fungi | dhellmann: the problem is that there is no enforced consistency between files cached on pypi's cdn even though there is a consistency expectation between some of them in the tools consuming pypi | 16:58 |
| dhellmann | can we fiddle things on pypi to purge whatever bad data it has? | 16:58 |
| fungi | dhellmann: yeah, when we identify what's returning stale/negative cache data we can send a purge request to fastly via a rest api call | 16:59 |
| dhellmann | k | 16:59 |
| *** melwitt has joined #openstack-infra | 16:59 | |
| clarkb | which apparently we don't log well in bandersnatch because bandersnatch things everything is ok | 16:59 |
| clarkb | dstufft: any chance you know what that url would be? | 16:59 |
| fungi | we even run a wrapper around bandersnatch that tries to do that automatically by parsing the logs from it, but it only catches a very obvious subset of these symptoms | 17:00 |
| fungi | clarkb: it's documented on ci.o.o isn't it? | 17:00 |
| *** esker has joined #openstack-infra | 17:00 | |
| clarkb | fungi: thats for a stale serial error which is logged | 17:00 |
| clarkb | fungi: in this case the serial isn't stale. the returned data is | 17:00 |
| fungi | oh | 17:00 |
| tykeal | anteaya: thank you! | 17:01 |
| fungi | so not `curl -X PURGE https://pypi.python.org/pypi/python-novaclient/json` | 17:01 |
| *** dustins has quit IRC | 17:01 | |
| clarkb | fungi: hrm actually it might be the same url | 17:01 |
| clarkb | before we purge it | 17:01 |
| clarkb | we should wget it, purge, wget it | 17:01 |
| *** asselin_ has quit IRC | 17:01 | |
| *** luqas has quit IRC | 17:02 | |
| clarkb | ok did that no diff in the json | 17:02 |
| clarkb | so I don't think that is the correct file | 17:02 |
| *** achanda has quit IRC | 17:02 | |
| clarkb | though it may have rotated its 24 hour cache by now | 17:03 |
| anteaya | tykeal: welcome, thanks for the diligence | 17:03 |
| anteaya | tykeal: in future, don't comment with ci logs on patches from your personal account | 17:03 |
| clarkb | I do see 1.3.0 sdist and whl in the returned json though, which is what we want | 17:03 |
| anteaya | tykeal: I didn't know you were doing that and that is not what I meant | 17:03 |
| *** esker has quit IRC | 17:04 | |
| tykeal | anteaya: *nod* I'll make sure to not set that up that way. I had just swapped out the CI's credentials with my own | 17:04 |
| fungi | clarkb: yeah, it's probably moot for that file now | 17:04 |
| jeblair | clarkb: so, hypothetically if i were not a week behind and we could use afs for pypi mirrors... could we validate the mirror before release to avoid this bug? i mean, is there some logic we could execute that would detect this condition? | 17:04 |
| *** adelinatuvenie has quit IRC | 17:04 | |
| tykeal | anteaya: hopefully we won't run into this ever again, it was sort of a perfect storm of issues :( | 17:04 |
| clarkb | jeblair: you could crawl the entire mirror looking for broken links | 17:04 |
| clarkb | jeblair: that would cover one issue. | 17:04 |
| clarkb | jeblair: the second issue is you as the mirrorer have no way of knowing that you may permanently avoided files | 17:05 |
| jeblair | clarkb: broken links == html anchor href pointing to file that is not on disk? | 17:05 |
| fungi | i think it would cover the main breaking problems we've experienced, though i wonder how quickly that would run | 17:05 |
| clarkb | jeblair: ya | 17:05 |
| anteaya | tykeal: well somedays I'm hopeful and some days I'm a realist, thanks for understanding and staying with it | 17:05 |
| *** MaxV has quit IRC | 17:05 | |
| clarkb | I think the only way you can know if you are broken is if an index file doesn't match what is on disk | 17:05 |
| tykeal | anteaya: it's my job ;) | 17:06 |
| clarkb | since pypi is claiming with serial X+1 that X does not exist | 17:06 |
| fungi | checking the md5sum to detect corrupted files would cover another less frequent class of problem we hit, but would definitely take a while | 17:06 |
| clarkb | fungi: I think bandersnatch cheks that at downloda time | 17:06 |
| anteaya | tykeal: yay! | 17:06 |
| clarkb | fungi: I suppose we could have bad disk | 17:07 |
| tykeal | well... part of my job | 17:07 |
| fungi | clarkb: oh, good. that may have only been a pypi-mirror era bug in that case | 17:07 |
| *** Sukhdev has quit IRC | 17:07 | |
| *** Guest83375 has quit IRC | 17:07 | |
| anteaya | tykeal: I hear that | 17:08 |
| jeblair | clarkb: i don't think i fully understand the second breakage pattern (permanently avoided files) | 17:08 |
| fungi | we did occasionally end up with interrupted/truncated downloads which would cause jobs pulling that broken file from the mirror to fail, but it's been so long since i recall seeing it, that probably predates bandersnatch implementation anyway | 17:08 |
| *** imcsk8 has quit IRC | 17:08 | |
| *** imcsk8 has joined #openstack-infra | 17:08 | |
| clarkb | jeblair: bandersnatch syncs in one direction only based on serial numbers. So if you get a serial number that says a file doesn't exist you won't sync that file | 17:09 |
| clarkb | jeblair: and since you are already synced to that serial you will never sync that file unless you start over | 17:09 |
| *** isaacb has quit IRC | 17:09 | |
| jeblair | clarkb: there's a way to say that a file doesn't exist? | 17:09 |
| clarkb | jeblair: ya if it isn't listed in that json url fungi pasted then the file "doesn't exist" and if it is on disk it will be deleted | 17:10 |
| *** achanda has joined #openstack-infra | 17:10 | |
| clarkb | jeblair: this is why resyncing was removing oslotest after we manually put it on the mirror | 17:10 |
| *** patrickeast has quit IRC | 17:10 | |
| *** pblaho__ has quit IRC | 17:10 | |
| fungi | even though the index file (which is apparently built separately from the json) contains a link to that file | 17:11 |
| mrodden | bandersnatch has a "no delete" option | 17:11 |
| mrodden | not sure if that affects the case there | 17:11 |
| fungi | it might be a viable workaround | 17:11 |
| clarkb | at least for our manually updating of the mirrors | 17:11 |
| *** jistr|trng has quit IRC | 17:11 | |
| fungi | though as a result we would wind up accumulating files which pypi actually deleted too | 17:11 |
| clarkb | but doens't help with not getting the file in the first place | 17:11 |
| mrodden | right | 17:12 |
| *** gyee_ has joined #openstack-infra | 17:12 | |
| clarkb | but I Think if we crawl looking for broken links we can detect this. not sure how fast that would be though | 17:12 |
| clarkb | stat all the files! | 17:12 |
| fungi | it seems to me like part of the problem is that the files and indexes are mirrored separately, rather than building the latter from the former | 17:13 |
| fungi | but that's obviously a much larger design consideration | 17:13 |
| clarkb | and it was shot down in the bug I submitted | 17:13 |
| clarkb | "bandersnatch is a simple tool and doesn't care about being consistent" | 17:13 |
| clarkb | is the tl;dr | 17:13 |
| clarkb | eventually consistent mirrors are ok | 17:13 |
| mrodden | doesn't sound like it will ever be consistent | 17:14 |
| jeblair | so the global index is built from scratch locally, i guess the per-project index comes from upstream? | 17:14 |
| clarkb | jeblair: yes it just copies what pypi.python.org/simple/hacking/index.html is for example | 17:15 |
| fungi | yes, it's downloaded verbatim from pypi.python.org | 17:15 |
| *** annegent_ has joined #openstack-infra | 17:15 | |
| *** amuller has quit IRC | 17:15 | |
| fungi | with all the raciness that implies | 17:15 |
| fungi | and as discussed yesterday, one of the issues is that it downloads the index after downloading the files rather than the other way around (i think dstufft was proposing that in a pull request to improve the situation slightly) | 17:16 |
| sdague | https://review.openstack.org/#/c/138474 more cells fine tuning of tests it can pass, if we can get another +2 from project-config-core | 17:16 |
| clarkb | ok hpcloud mirror claims to be up to date. I am running again to make sure it doesn't error | 17:16 |
| fungi | though downloading the index before downloading the files also has a risk if the file replacement isn't relatively atomic, since we'd be (briefly) linking files which were incomplete or nonexistent | 17:17 |
| clarkb | looks like it is happy | 17:17 |
| clarkb | fungi: the suggestion was download it first, grab sdist/whl, then write index file | 17:17 |
| fungi | yeah, so not put it into place until the related file downloads completed | 17:18 |
| clarkb | so the download happens first but it isn't part of the mirror until after the packages are grabbed | 17:18 |
| anteaya | sdague: +3 on 138474 | 17:18 |
| *** luqas has joined #openstack-infra | 17:18 | |
| fungi | sounds sane enough then | 17:18 |
| sdague | anteaya: thanks | 17:19 |
| anteaya | some day I hope to learn what cells are | 17:19 |
| mordred | anteaya: they are cells | 17:19 |
| anteaya | mordred: that clears it all up | 17:20 |
| anteaya | thanks | 17:20 |
| * mordred is helpful | 17:20 | |
| anteaya | you are indeed | 17:20 |
| * mrodden wonders what just happened. | 17:20 | |
| mordred | fungi, clarkb: there is still a race there because of deletes | 17:21 |
| mordred | what is really needed is transactions | 17:21 |
| mordred | because what we're talking about is transaction isolation problems | 17:21 |
| clarkb | mordred: yes that was explained | 17:21 |
| fungi | mordred: yeah, we discussed the fact that swapping the order races on deletes, but that's less critical and much, much less frequent | 17:21 |
| clarkb | mordred: but deletes are far less common | 17:21 |
| mordred | yes | 17:21 |
| mordred | agree | 17:21 |
| mordred | this is, btw, the same class of problems that exists with mirroring apt repos | 17:22 |
| jeblair | krotscheck: what's involved in obtaining an oauth bearer token? | 17:22 |
| *** mwagner_lap has joined #openstack-infra | 17:22 | |
| *** marcusvrn has quit IRC | 17:22 | |
| mordred | clarkb, fungi: another way to deal with the delete race is to not actually delete files on deletes, but mark them for delete for later | 17:22 |
| fungi | mordred: a major point of complication there is that we aren't really mirroring through an api, we're mirroring through a cache which can be internally incomsistent by as much as 24 hours | 17:22 |
| mrodden | so the workaround for now is just keep deleting the serial file, and re-running bandersnatch until its happy | 17:23 |
| mordred | so that "deleting" happens when the index is updated because now there are no links to the deleted file anymore | 17:23 |
| openstackgerrit | Merged openstack-infra/project-config: Filter floating/fixed ip tests for tempest-cells https://review.openstack.org/138474 | 17:23 |
| mordred | (in which case we'll have implemented mark and sweep garbage collection and reference counting, but who's keeping track) | 17:23 |
| *** skolekonov has quit IRC | 17:24 | |
| fungi | mrodden: well, and purging broken cached bits from pypi.python.org's cdn (exacerbated by a separate query race in pypi's backend right now) | 17:24 |
| mrodden | next up: openstack the programming language | 17:24 |
| mrodden | fungi, wow that sounds like a grand mess | 17:25 |
| mrodden | what happend to large static file apache2 farms and etags... | 17:26 |
| *** atan8 has joined #openstack-infra | 17:26 | |
| fungi | cdn happened | 17:27 |
| clarkb | anteaya: I have a list of current members in voting group that my script did not show as votimg. will paste that shortly so you can cross check | 17:28 |
| mrodden | i thought cdn was a fancy name for dns redirection to a large farm of apache2 servers with etags | 17:29 |
| openstackgerrit | Mehdi Abaakouk proposed openstack-infra/project-config: Run ceilometer rally scenarios https://review.openstack.org/132650 | 17:29 |
| anteaya | AJaeger: this expands the original test file, how would you feel about having this as an additional non-voting test for a while and then once we confirm it is stable, en-able it: https://review.openstack.org/#/c/138702/5 | 17:29 |
| anteaya | AJaeger: I like the idea just would like to discuss the implementation workflow | 17:29 |
| anteaya | clarkb: thanks | 17:29 |
| *** jcoufal has quit IRC | 17:30 | |
| pleia2 | ugh, guess what 4 days of rain does to a 4-stories-below-sea-level resident garage in downtown SF? :\ so far our storage units have avoided the flooding, but I'll be wandering off to check on things periodically | 17:30 |
| fungi | mrodden: a very large farm of apache servers which you can try to signal to invalidate critical content when you make changes but then your signaling doesn't always propagate and you end up with all manner of geographic inconsistencies you need to work around | 17:30 |
| *** dangers is now known as dangers_away | 17:31 | |
| *** avozza is now known as zz_avozza | 17:31 | |
| *** mattfarina has joined #openstack-infra | 17:32 | |
| *** ryanpetrello has quit IRC | 17:32 | |
| *** luqas has quit IRC | 17:32 | |
| mrodden | so the problem is updating/deleting resources at locations in a large farm | 17:33 |
| *** wuhg has quit IRC | 17:33 | |
| sdague | fungi: right, the issue really is the mirroring of the index. I think that's where all of these things get it wrong | 17:33 |
| mrodden | yeah | 17:34 |
| *** zz_avozza is now known as avozza | 17:34 | |
| mrodden | shouldn't care about static assets | 17:34 |
| sdague | you should never mirror the index, you should have local code that builds you an index based on the content you have | 17:34 |
| clarkb | anteaya: http://paste.openstack.org/show/144075/ thsoe accouts are part of the voting group but have not voted in the last 3 months according to my script | 17:34 |
| mrodden | need to update the indexes to point a new ones | 17:34 |
| clarkb | anteaya: would be good to see if we can at least confirm they haven't voted | 17:34 |
| dstufft | mrodden: fungi our CDN is a large farm of varnish servers | 17:34 |
| anteaya | clarkb: I will check what is happening and get back to you | 17:34 |
| anteaya | clarkb: I'll see what I can do | 17:34 |
| clarkb | anteaya: I am working on the first draft of the acl change now using the data I got though | 17:34 |
| anteaya | good idea | 17:35 |
| fungi | dstufft: yeah, i handwaved because mrodden said apache and i was trying to be glib... sorry! ;) | 17:35 |
| anteaya | easy to add any straglers after we get something up | 17:35 |
| clarkb | anteaya: I do note that no non members of the voting group have voted | 17:35 |
| clarkb | so gerrit acls must work :) | 17:35 |
| dstufft | fungi: techincally apache2 is used to terminate TLS :V | 17:36 |
| mrodden | always an apache2 somewhere... | 17:36 |
| mrodden | or nginx | 17:36 |
| *** rlrossit has left #openstack-infra | 17:36 | |
| dstufft | using varnish is nice, because it means you can cache dynamic pages | 17:37 |
| anteaya | clarkb: that is good news | 17:37 |
| fungi | (or ncsa httpd?) | 17:37 |
| dstufft | and Fastly purges are awesome, purges take like 500ms or so to distribute across the globe | 17:37 |
| *** teran has quit IRC | 17:37 | |
| *** sputnik13 has joined #openstack-infra | 17:37 | |
| fungi | except when one or more of their ndoes get stuck | 17:38 |
| fungi | s/ndoes/nodes/ | 17:38 |
| dstufft | fungi: there's that :/ | 17:38 |
| *** tykeal has left #openstack-infra | 17:38 | |
| dstufft | it used to have a problem where it's forget surrogate key associations | 17:39 |
| *** rmcall has quit IRC | 17:39 | |
| dstufft | PyPI is apparently a really great way to discover purge issues | 17:39 |
| jeblair | krotscheck: left some comments/questions on 105252 | 17:39 |
| krotscheck | jeblair: Thanks | 17:39 |
| krotscheck | jeblair: I’m still grading papers, will hopefully get to that by eod. | 17:39 |
| anteaya | clarkb: I think we are safe leaving off the deprecated, unknown and disabled named ci accounts | 17:40 |
| anteaya | clarkb: the problem I have is with knowing there are some deprecated ci accounts taht have voted in that time span | 17:41 |
| *** armax has quit IRC | 17:41 | |
| clarkb | dhellmann: question. For this per project third party group stuff (if you haven't followed it long story short is new groups for each project so that you can control who votes) should there be a single oslo group? or should oslo.vmware get one specifically for it? | 17:41 |
| clarkb | dhellmann: oslo.vmware is the only oslo project with third party ci voting right now best I can tell | 17:41 |
| clarkb | anteaya: jeblair fungi ^ | 17:41 |
| sdague | pleia2: you didn't set up an ipv6 vhost :) | 17:41 |
| clarkb | I lean towards a single oslo-ci group because there is a single oslo-release group | 17:41 |
| zaro | morning | 17:41 |
| anteaya | clarkb: I'm leaning with you | 17:42 |
| anteaya | morning zaro | 17:42 |
| clarkb | oh wait | 17:42 |
| clarkb | oslo.vmware has its own release group | 17:42 |
| clarkb | I think I want oslo.vmware-ci now. WIll start with that. It all gets reviewed anyways | 17:43 |
| *** marcusvrn has joined #openstack-infra | 17:43 | |
| anteaya | okay | 17:43 |
| pleia2 | sdague: ugh, yeah, ipv6 on this server has always been a bit meh, I probably disabled it when it went down once | 17:43 |
| *** rmcall has joined #openstack-infra | 17:44 | |
| pleia2 | hm, actually, it's *:80 | 17:44 |
| *** yamamoto has quit IRC | 17:45 | |
| pleia2 | I'll have to fix later | 17:45 |
| *** jaypipes has quit IRC | 17:45 | |
| *** HeOS has quit IRC | 17:46 | |
| *** ayoung has joined #openstack-infra | 17:46 | |
| *** hogepodge has quit IRC | 17:47 | |
| asselin | anyone seen this issue in nodepool image-update? TypeError: exec_command() got an unexpected keyword argument 'get_pty' | 17:47 |
| fungi | clarkb: it's probably a judgement call depending on how many oslo libraries are getting third-party ci votes | 17:48 |
| asselin | http://paste.openstack.org/show/144080/ | 17:48 |
| zaro | krotscheck: this there email support in storyboard yet? | 17:48 |
| zaro | *this/is | 17:49 |
| clarkb | fungi: ya right now its just oslo.vmware | 17:49 |
| *** hogepodge has joined #openstack-infra | 17:49 | |
| *** avozza is now known as zz_avozza | 17:50 | |
| *** armax has joined #openstack-infra | 17:50 | |
| pleia2 | sdague: hah, even better, my host changed my ipv6 address | 17:50 |
| * pleia2 grumbles and updates dns | 17:50 | |
| krotscheck | zaro: Nope. Been busy with bug hunting, caching, other things like that. | 17:51 |
| krotscheck | zaro: Oh, and migrating infra. | 17:51 |
| *** _nadya_ has quit IRC | 17:51 | |
| sdague | yeh, and it gets all kinds of fun with ssl | 17:51 |
| sdague | which you still have to bind by ip | 17:52 |
| zaro | krotscheck: i see that there's an email plugin in WIP status. how far along is that? | 17:52 |
| krotscheck | zaro: That one’s dead in the water because of work that I have to do on the cron plugin first. | 17:52 |
| krotscheck | zaro: And _that_ work is about 50% complete. | 17:52 |
| * krotscheck is also grading papers, so he has split brain problems. | 17:52 | |
| *** armax has quit IRC | 17:52 | |
| *** melwitt has quit IRC | 17:53 | |
| *** cpowell has quit IRC | 17:53 | |
| *** melwitt has joined #openstack-infra | 17:54 | |
| *** shayneburgess has joined #openstack-infra | 17:56 | |
| *** melwitt has quit IRC | 17:56 | |
| *** annegent_ has quit IRC | 17:56 | |
| *** melwitt has joined #openstack-infra | 17:56 | |
| *** adelinatuvenie has joined #openstack-infra | 17:57 | |
| *** shayneburgess has quit IRC | 17:57 | |
| *** tonytan4ever has quit IRC | 17:59 | |
| anteaya | clarkb: IBM PowerKVM CI is krtaylor's | 17:59 |
| *** _nadya_ has joined #openstack-infra | 18:00 | |
| anteaya | krtaylor: what is happening with IBM PowerKVM CI, our scrape shows it as non-voting for the past three months | 18:00 |
| anteaya | clarkb: it was three months, was it not? | 18:00 |
| *** derekh has quit IRC | 18:00 | |
| clarkb | anteaya: yup | 18:00 |
| anteaya | thanks | 18:00 |
| *** AJaeger_ has joined #openstack-infra | 18:01 | |
| *** mriedem has quit IRC | 18:01 | |
| *** Ryan_Lane has joined #openstack-infra | 18:01 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Add project specific CI acls https://review.openstack.org/138805 | 18:01 |
| * anteaya reviews | 18:02 | |
| clarkb | anteaya: jeblair fungi ^ that is the list of ACL changes according to my script | 18:02 |
| *** _nadya_ has quit IRC | 18:02 | |
| anteaya | clarkb: as a start, I agree with the files modified | 18:03 |
| anteaya | I find the xenapi-os-testing odd though | 18:03 |
| anteaya | I don't know what that repo is | 18:03 |
| *** jedimike has quit IRC | 18:04 | |
| *** harlowja_away is now known as harlowja_ | 18:04 | |
| *** dizquierdo has quit IRC | 18:05 | |
| pleia2 | I believe that's part of the the xen stuff that is all pretty old, they were doing third party testing before we had a name for it | 18:05 |
| *** odyssey4me has left #openstack-infra | 18:05 | |
| *** emagana has joined #openstack-infra | 18:05 | |
| *** AlexF has quit IRC | 18:05 | |
| fungi | clarkb: that set looks good to me | 18:05 |
| *** bdpayne has joined #openstack-infra | 18:05 | |
| anteaya | me too | 18:06 |
| *** rmcall has quit IRC | 18:06 | |
| clarkb | pleia2: anteaya they had votes in the last 3 months | 18:06 |
| clarkb | should probably ask BobBall | 18:06 |
| anteaya | mind if I give neutron, nova, horizon etc a chance to review? | 18:06 |
| fungi | they were non-voting, then voting, then disabled, and i don't think they're back up yet | 18:06 |
| *** Ryan_Lane has quit IRC | 18:06 | |
| *** Sukhdev has joined #openstack-infra | 18:06 | |
| *** tomoe_ has quit IRC | 18:06 | |
| anteaya | BobBall: what is the status of xenapi-os-testing? | 18:07 |
| anteaya | clarkb: can we leave it off this patch and then have a follow up patch if required? | 18:07 |
| *** rmcall has joined #openstack-infra | 18:07 | |
| *** dannywilson has joined #openstack-infra | 18:07 | |
| *** andreykurilin_ has joined #openstack-infra | 18:07 | |
| pleia2 | clarkb: oh, I think it's still active, just on the old side, so not doing the same thing as other CI systems | 18:08 |
| *** belmoreira has quit IRC | 18:08 | |
| *** dannywilson has quit IRC | 18:09 | |
| fungi | clarkb: did we narrow down how the ci manager groups were going to work? something like nova-ci-coordinators as the owner of the nova-ci group and then prepopulate it with the nova ptl, et cetera? | 18:10 |
| anteaya | mtreinish: this patch creates the permissions for devstack to adminsiter which third party accounts have voting rights, please review: https://review.openstack.org/#/c/138805/ | 18:10 |
| anteaya | dhellmann: and this for oslo.vmware too: https://review.openstack.org/#/c/138805/ | 18:11 |
| *** rmcall_ has joined #openstack-infra | 18:11 | |
| clarkb | fungi: etherpad says project-release owns it and I Think we start there | 18:11 |
| fungi | ahh, yep. that works | 18:11 |
| clarkb | fungi: it minimizes the amount of change and its easy to update later | 18:11 |
| *** rmcall has quit IRC | 18:12 | |
| *** rmcall_ is now known as rmcall | 18:12 | |
| anteaya | well mikal sent out an email to nova-drivers and the response was a majority in favour of reusing nova-drivers | 18:12 |
| anteaya | but tehre is nothing wrong with creating this group and populating it with nova-drivers as the members | 18:12 |
| *** dimtruck is now known as zz_dimtruck | 18:12 | |
| mtreinish | anteaya: so you're just adding *-ci groups that have perms for the verified column. Who is going to have group membership edit perms? | 18:13 |
| fungi | is there a nova-drivers in gerrit? | 18:13 |
| *** AlexF has joined #openstack-infra | 18:13 | |
| fungi | perhaps he meant nova-milestone? | 18:14 |
| clarkb | there is no nova-drivers | 18:14 |
| fungi | (only zuul) | 18:14 |
| clarkb | :) | 18:14 |
| fungi | mtreinish: tempest-release et cetera | 18:15 |
| *** mriedem has joined #openstack-infra | 18:16 | |
| *** dkranz has quit IRC | 18:16 | |
| anteaya | clarkb fungi nova-specs-core, sorry | 18:16 |
| *** markmcclain has joined #openstack-infra | 18:17 | |
| clarkb | can the owner of a group change the owner? | 18:17 |
| clarkb | probably not | 18:17 |
| fungi | clarkb: i believe they can if they are also a member of teh new owner group | 18:17 |
| *** hdd has quit IRC | 18:17 | |
| mtreinish | fungi: ok that's what I figured, just wanted to make sure | 18:17 |
| clarkb | oh then maybe -release is where we should start then they can update at will | 18:17 |
| jeblair | clarkb: ready for me to +3 the acl change? | 18:17 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 18:17 | |
| *** rushiagr is now known as rushiagr_away | 18:17 | |
| clarkb | jeblair: I am ready unless anteaya wants BobBall feedback on xen-os-thing first | 18:18 |
| fungi | mtreinish: basically some decision-making subset of each project, whether that be a core team or a release time or a specs approvers team or whatever | 18:18 |
| *** cpowell has joined #openstack-infra | 18:18 | |
| openstackgerrit | Doug Wiegley proposed openstack/requirements: Bump oslo.concurrency, as keystone is now importing oslo_concurrency https://review.openstack.org/138813 | 18:18 |
| jeblair | clarkb, anteaya: let's err on the side of status-quo and fix later if needed | 18:18 |
| clarkb | jeblair: sounds good | 18:18 |
| anteaya | very good | 18:18 |
| *** patrickeast has joined #openstack-infra | 18:18 | |
| anteaya | I shopped the change around and have feedback from horizon, cinder and neutron | 18:19 |
| *** zz_avozza is now known as avozza | 18:20 | |
| *** achanda has quit IRC | 18:20 | |
| mtreinish | fungi: for the qa projects I think each project's release group is fine | 18:20 |
| dhellmann | clarkb: if only oslo.vmware is doing 3rd party ci, let's create a group just for them. They're already a bit of an edge case w/in oslo. | 18:20 |
| *** atan8 has quit IRC | 18:20 | |
| clarkb | dhellmann: yup just them | 18:21 |
| clarkb | dhellmann: and that is how my proposal is set up so I think we are good to go | 18:21 |
| dhellmann | clarkb: ok, I'm just catching up on scrollbacak | 18:21 |
| *** achanda has joined #openstack-infra | 18:21 | |
| clarkb | one that gets in and is applied I will update group membership for each new group | 18:22 |
| dhellmann | anteaya: you said "too" did I miss the first thing? | 18:22 |
| anteaya | I had good feedback from nova last night so I think we are good to have this in place | 18:22 |
| *** markus_z has quit IRC | 18:22 | |
| anteaya | dhellmann: the too refered to the fact I directed to mtreinish first | 18:22 |
| anteaya | then added you in the line below | 18:23 |
| *** AlexF has quit IRC | 18:23 | |
| dhellmann | k | 18:23 |
| *** SumitNaiksatam has joined #openstack-infra | 18:23 | |
| *** AlexF has joined #openstack-infra | 18:23 | |
| *** esker has joined #openstack-infra | 18:26 | |
| morganfainberg | fungi, feature/hierarchical-multitenancy merged to master for keystone | 18:27 |
| fungi | morganfainberg: all set for branch deletion then? | 18:27 |
| morganfainberg | fungi, so we should be good to close the feature branch down. | 18:27 |
| morganfainberg | fungi, give me one second but sure. | 18:27 |
| * morganfainberg is just getting moving today. | 18:27 | |
| fungi | morganfainberg: great. whenever you're ready let me know | 18:28 |
| morganfainberg | fungi, yep | 18:28 |
| *** esker has quit IRC | 18:28 | |
| morganfainberg | fungi, the two outstanding reviews have been rebased to master. | 18:28 |
| morganfainberg | fungi, i'm abandoning them now. | 18:28 |
| morganfainberg | fungi, we're good | 18:28 |
| openstackgerrit | Merged openstack-infra/project-config: Add project specific CI acls https://review.openstack.org/138805 | 18:28 |
| fungi | morganfainberg: yeah, was about to point out https://review.openstack.org/#/q/status:open+branch:feature/hierarchical-multitenancy,n,z | 18:28 |
| *** jlibosva has quit IRC | 18:29 | |
| *** otter768 has joined #openstack-infra | 18:29 | |
| *** wenlock has quit IRC | 18:29 | |
| fungi | morganfainberg: i'll wait for you to do whatever you're going to do with those three | 18:29 |
| *** dkranz has joined #openstack-infra | 18:30 | |
| morganfainberg | fungi, all outstanding reviews on that branch are done | 18:30 |
| morganfainberg | fungi, abandoned | 18:30 |
| fungi | morganfainberg: confirmed. i've now deleted the feature/hierarchical-multitenancy branch previously at a98858e17087a6a1f3862d1a38c66f164fa15279 | 18:31 |
| *** hdd has joined #openstack-infra | 18:32 | |
| *** vigneshvar_ has joined #openstack-infra | 18:33 | |
| *** otter768 has quit IRC | 18:34 | |
| *** mriedem has quit IRC | 18:34 | |
| *** Masahiro has joined #openstack-infra | 18:35 | |
| *** pelix has quit IRC | 18:35 | |
| *** esker has joined #openstack-infra | 18:35 | |
| *** armax has joined #openstack-infra | 18:36 | |
| *** afazekas has quit IRC | 18:36 | |
| *** wenlock has joined #openstack-infra | 18:37 | |
| *** shashankhegde has joined #openstack-infra | 18:37 | |
| *** armax has quit IRC | 18:37 | |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config: print pip freeze even when flake8 fails https://review.openstack.org/138514 | 18:37 |
| *** armax has joined #openstack-infra | 18:38 | |
| *** Masahiro has quit IRC | 18:40 | |
| *** shashankhegde has quit IRC | 18:40 | |
| *** annegent_ has joined #openstack-infra | 18:40 | |
| *** isviridov is now known as isviridov_away | 18:42 | |
| *** dustins has joined #openstack-infra | 18:43 | |
| anteaya | clarkb: what did you decide to do with the accounts listed here? http://paste.openstack.org/show/144075/ | 18:44 |
| *** cpowell_ has joined #openstack-infra | 18:45 | |
| clarkb | anteaya: I think they are probably inactive and this move will smoke them out if not | 18:45 |
| anteaya | clarkb: brocade docker and ibm are the only ones that I think have a valid use case | 18:45 |
| anteaya | okay let's do that | 18:45 |
| *** yamahata has joined #openstack-infra | 18:45 | |
| *** yamamoto has joined #openstack-infra | 18:45 | |
| anteaya | I am surprised to see the ibm powerkvm on that list though | 18:45 |
| fungi | yeah, this acl doesn't prevent/enable those accounts commenting on whatever projects they want, so if they're not voting then this is a null change for them | 18:46 |
| *** wenlock1 has joined #openstack-infra | 18:46 | |
| fungi | remember that the upshot of this move is that we'll cease tracking non-voting third-party ci accounts | 18:46 |
| fungi | doesn't mean they don't exist, we just don't keep a list of them any longer | 18:47 |
| *** berendt has quit IRC | 18:47 | |
| anteaya | so they just bounce back to registered users? | 18:47 |
| clarkb | yup | 18:47 |
| *** cpowell has quit IRC | 18:47 | |
| anteaya | kk | 18:47 |
| *** zz_dimtruck is now known as dimtruck | 18:48 | |
| anteaya | hey so now I am a manager of nothing | 18:48 |
| *** annegent_ has quit IRC | 18:48 | |
| anteaya | so might as well delete that gerrit group | 18:48 |
| *** mriedem has joined #openstack-infra | 18:48 | |
| *** ivar-lazzaro has joined #openstack-infra | 18:48 | |
| *** dkranz has quit IRC | 18:48 | |
| anteaya | https://review.openstack.org/#/admin/groups/440,members | 18:49 |
| *** bknudson has quit IRC | 18:50 | |
| fungi | anteaya: can't actually delete gerrit groups. i just empty them out, switch the owner to administrators and set the hidden option so non-admins don't see them in the group list any longer | 18:51 |
| clarkb | yup, I will work on that cleanup once we have this all in place and working | 18:51 |
| *** achanda has quit IRC | 18:51 | |
| * clarkb checks if new groups exist yet | 18:51 | |
| *** yamamoto has quit IRC | 18:51 | |
| clarkb | they do \o/ I am populating them now | 18:51 |
| *** achanda has joined #openstack-infra | 18:52 | |
| *** achanda has quit IRC | 18:52 | |
| fungi | clarkb: i'd call this a massive exercise of https://review.openstack.org/136079 | 18:52 |
| *** achanda has joined #openstack-infra | 18:52 | |
| clarkb | fungi: and it seems ot have worked | 18:52 |
| clarkb | so my +2 still stands :) | 18:52 |
| fungi | maybe we should approve that before someone approves something else to jeepyb and my hand-patch on review.o.o gets undone | 18:52 |
| clarkb | jeblair: ^ you will probably wantto review that change | 18:53 |
| clarkb | fungi: ++ | 18:53 |
| anteaya | yes | 18:53 |
| anteaya | and emptying, switch owner to admin and hiding them works for me | 18:55 |
| *** sdake has quit IRC | 18:56 | |
| *** sdake has joined #openstack-infra | 18:57 | |
| morganfainberg | fungi, could i ask you to also remove the feature/key-dist topic branch from keystone/ | 18:57 |
| morganfainberg | fungi, ? if you wouldn't mind. | 18:57 |
| *** annegent_ has joined #openstack-infra | 18:57 | |
| fungi | morganfainberg: sure thing. i thought about asking but figured it was probably still active if you hadn't mentioned it | 18:57 |
| morganfainberg | it's definitely dead - kite moved over to barbican | 18:58 |
| fungi | morganfainberg: i've now deleted branch feature/key-dist previously at cf20a1a3d857d6f91a5591798eda5fbe6c47a2ff | 18:59 |
| *** teran has joined #openstack-infra | 18:59 | |
| *** esker has quit IRC | 18:59 | |
| *** emagana has quit IRC | 19:00 | |
| *** emagana has joined #openstack-infra | 19:00 | |
| clarkb | great there are two metaplugin CIs | 19:00 |
| *** dkranz has joined #openstack-infra | 19:01 | |
| *** Sukhdev has quit IRC | 19:01 | |
| *** emagana_ has joined #openstack-infra | 19:01 | |
| clarkb | anteaya: can you track down which of the two "Metaplugin CI" fullnames has voted on neutron? | 19:01 |
| *** shashankhegde has joined #openstack-infra | 19:01 | |
| *** esker has joined #openstack-infra | 19:01 | |
| *** bswartz1 has quit IRC | 19:02 | |
| clarkb | opendaylight too | 19:02 |
| clarkb | with the same email address even | 19:02 |
| clarkb | how does that happen? | 19:02 |
| *** baoli has quit IRC | 19:03 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Third party accounts need unique email https://review.openstack.org/138827 | 19:03 |
| fungi | clarkb: ^ | 19:03 |
| *** mwagner_lap has quit IRC | 19:03 | |
| * fungi is prescient | 19:04 | |
| *** emagana has quit IRC | 19:04 | |
| *** yamahata has quit IRC | 19:07 | |
| *** dannywilson has joined #openstack-infra | 19:07 | |
| *** Ryan_Lane has joined #openstack-infra | 19:08 | |
| *** baoli has joined #openstack-infra | 19:09 | |
| *** andreykurilin__ has joined #openstack-infra | 19:09 | |
| *** andreykurilin_ has quit IRC | 19:09 | |
| clarkb | ok only "Metaplugin CI" and "OpenDaylight CI" are problematic | 19:10 |
| clarkb | I will dig in and see if I can find which is the correct one | 19:10 |
| *** baoli has quit IRC | 19:10 | |
| clarkb | once that is done we can fire off announcement email and remove all-projects acl | 19:11 |
| *** MaxV has joined #openstack-infra | 19:11 | |
| clarkb | then I can cleanup the old groups | 19:11 |
| clarkb | ok was able to figure out metaplugin ci based on email for account in current old voting group | 19:12 |
| * clarkb digs in DB | 19:14 | |
| openstackgerrit | mbitard proposed openstack-infra/jenkins-job-builder: add slack notification support, largely inspired by hipchat module https://review.openstack.org/136627 | 19:15 |
| *** MaxV has quit IRC | 19:17 | |
| *** krtaylor has quit IRC | 19:17 | |
| fungi | dims: looking for existing examples for the purposes of consistency, i notice that run-tox.sh says in its top-level comment that it does a pip freeze, but does not actually run that in the script | 19:17 |
| *** weshay has quit IRC | 19:18 | |
| clarkb | fungi: I have a set of changes up to fix all that | 19:18 |
| *** weshay has joined #openstack-infra | 19:18 | |
| *** MaxV has joined #openstack-infra | 19:18 | |
| fungi | clarkb: you may want to weigh in on 138514 in that case | 19:19 |
| jeblair | clarkb, fungi; jeepyb change lgtm | 19:19 |
| jeblair | fungi: should i aprv? | 19:19 |
| fungi | jeblair: please do. it's been running successfully for a couple weeks now with no hiccup | 19:20 |
| fungi | through a couple dozen changes which create new groups | 19:20 |
| jeblair | nice that the code was already set up to facilitate this :) | 19:20 |
| fungi | indeedy | 19:20 |
| openstackgerrit | Merged openstack-infra/jeepyb: Reconnect to Gerrit database if needed https://review.openstack.org/136079 | 19:20 |
| clarkb | fungi: oh dims change looks fine mine only touches run tox and run unittests | 19:20 |
| jeblair | ah simple tests :) | 19:20 |
| clarkb | but dims is updating run pep8 | 19:21 |
| anteaya | clarkb: yeah I think you digging in the db to sort out opendaylight ci is the best route | 19:21 |
| jeblair | clarkb, fungi: for planning purposes, i do not think i will be able to look into gear/dib stuff this week :/ | 19:21 |
| anteaya | clarkb: it will be a higher quality of info that any I could access I think | 19:21 |
| dims | fungi: i see a pip freeze in run-tox.sh - https://github.com/openstack-infra/project-config/blob/master/jenkins/scripts/run-tox.sh#L40 | 19:21 |
| fungi | dims: aha, i bet i need to update my checkout | 19:22 |
| *** HeOS has joined #openstack-infra | 19:22 | |
| fungi | dims: ahh, yep that was it | 19:22 |
| fungi | now i see it in there | 19:22 |
| clarkb | jeblair: ok, I am pretty swamped too so thats fine | 19:22 |
| *** sarob has joined #openstack-infra | 19:23 | |
| fungi | dims: seems clarkb fixed that with https://review.openstack.org/137433 (merged just a few hours ago) | 19:24 |
| anteaya | clarkb: for opendaylight this is the account we just re-enabled: 10386 odl-jenkins OpenDaylight CI odl-openstack@opendaylight.org | 19:24 |
| anteaya | clarkb: what else do you have? | 19:25 |
| *** tonytan4ever has joined #openstack-infra | 19:26 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: Add .gitreview content to Creator's Guide https://review.openstack.org/138831 | 19:26 |
| clarkb | anteaya: I haven't been able to get the account id for the account in the voting group | 19:27 |
| clarkb | it is surprisingly difficult | 19:27 |
| clarkb | also I am about to redisable that account until they fix their stuff | 19:27 |
| *** sarob has quit IRC | 19:27 | |
| fungi | clarkb: on that note, it looks like 137433 merged _just_ before image updates kicked off, so unless we have broken image builds we should be safe to approve 137434 now | 19:27 |
| clarkb | fungi: someone should check :) | 19:28 |
| fungi | i am checking now | 19:28 |
| anteaya | clarkb: fix what stuff, what is out of order with them? | 19:28 |
| clarkb | anteaya: there are two accounts with the same name and same email address | 19:28 |
| clarkb | anteaya: which is currently making it almost impossible to know which account is which | 19:28 |
| anteaya | can I see a paste of what you are seeing? | 19:28 |
| *** bswartz has joined #openstack-infra | 19:29 | |
| clarkb | anteaya: i am trying to manually add "OpenDaylight CI" to the new neutron-ci group and I Have two options with no ids | 19:29 |
| anteaya | well from my understanding if they have two accounts, it is because somehow two accounts got created by infra | 19:29 |
| fungi | clarkb: if you make a rest api query for group membership, it will mention the associated account id | 19:29 |
| anteaya | is that not what it means? | 19:29 |
| *** AlexF has quit IRC | 19:30 | |
| *** krtaylor has joined #openstack-infra | 19:30 | |
| fungi | anteaya: not necessarily. they could have created one of those accounts via launchpad login/openid | 19:30 |
| anteaya | oh | 19:30 |
| fungi | anteaya: in fact, they would almost have to have done that | 19:30 |
| anteaya | so if we bounce it back to them what is the fix? | 19:30 |
| anteaya | man | 19:30 |
| clarkb | anteaya: they change the fullname and email address of one of the accounts | 19:31 |
| anteaya | we just spent a month getting them in order | 19:31 |
| clarkb | either we do it on the system account or they do it on the other one | 19:31 |
| anteaya | mestery: can you help us here? | 19:31 |
| fungi | anteaya: because the gerrit account create/change api call will refuse to associate a duplicate name or e-mail address, but gerrit will still happily import duplicates via openid login | 19:31 |
| cpowell_ | Greetings, I am attempting to save my offline contact information in Gerrit and am experiencing a server error. Does anyone know what could possibly be the issue? I am unable to contribute until this is resolved. | 19:31 |
| clarkb | fungi: oh right I Have that list already, forgot it came with account IDs | 19:31 |
| mestery | anteaya: On a call, trying to read the backscroll a bit. | 19:31 |
| mestery | clarkb: What can I heolp you with? | 19:32 |
| anteaya | mestery: opendaylight ci has two accounts | 19:32 |
| clarkb | cpowell_: most likely you are not a member of the openstack foundation and if you are either your primary email address in gerrit doesn't match your address on the foundation side or you have a special foundation account as a speaker or similar | 19:32 |
| mestery | Yikes! Let me ping tykeal and get him here for you | 19:32 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/system-config: Rework stackforge.html for infra manual changes https://review.openstack.org/138770 | 19:32 |
| anteaya | mestery: and we can't add them to the neutron-ci group as a result | 19:32 |
| anteaya | mestery: thank you | 19:32 |
| clarkb | anteaya: 10386 is the correct one to make votig | 19:33 |
| clarkb | I will add them now | 19:33 |
| anteaya | clarkb: yes | 19:33 |
| sdague | dhellmann: https://review.openstack.org/#/c/138507 should be prefered over my patch | 19:33 |
| cpowell_ | clarkb: ok, I will double check everything | 19:33 |
| *** MarkAtwood has joined #openstack-infra | 19:34 | |
| clarkb | ok thats done, all projects have populated groups now | 19:34 |
| anteaya | clarkb: also when you get around to owners of the neutron-ci group, they are mestery and emagana_ | 19:34 |
| clarkb | anteaya: its set to neutron-release | 19:34 |
| anteaya | clarkb: yay | 19:34 |
| mestery | clarkb: Thanks sir! | 19:34 |
| mestery | I pinged tykeal BTW, hope he pops on here soon | 19:34 |
| anteaya | clarkb: can we set it to mestery and emagana_ please? | 19:34 |
| clarkb | anteaya: not without creating a new group | 19:34 |
| clarkb | anteaya: I really want to avoid that | 19:34 |
| anteaya | well emagana_ does the lions share of the work here | 19:35 |
| anteaya | so he needs to be able to have permissions | 19:35 |
| clarkb | anteaya: then mestery can add emagana_ to the -release group? | 19:35 |
| anteaya | and mestery helps too but it is mostly emagana_ | 19:35 |
| anteaya | well I don't know as allowing emagana_ release permissions is what we should expect here | 19:35 |
| emagana_ | anteaya: leashing the lions here! | 19:35 |
| anteaya | unsuccessfully, as on most days | 19:36 |
| clarkb | anteaya: I don't know either. I just don't want to get ahead of ourselves | 19:36 |
| clarkb | anteaya: we haven't even gotten this officially going yet | 19:36 |
| anteaya | well neutron will be the busiest group | 19:36 |
| anteaya | and emagana_ does most of the work with third party | 19:36 |
| anteaya | so whether you want to do it today or not is up to you | 19:36 |
| anteaya | but that is what has been requested | 19:36 |
| anteaya | and it makes total sense to me | 19:37 |
| clarkb | anteaya: so I really don't want to create a special group for this unless we have to across the board | 19:37 |
| anteaya | since that is what I have seen | 19:37 |
| anteaya | why? | 19:37 |
| clarkb | I think optimizing now is the wrong time to start doing that | 19:37 |
| *** AlexF has joined #openstack-infra | 19:37 | |
| anteaya | if you like | 19:37 |
| clarkb | anteaya: because then I have to manually create this group or come up with some arbitrary ACL for it so that it gets automagically created | 19:37 |
| anteaya | I'm just telling you what neutron has requested | 19:37 |
| clarkb | or update manage-projects | 19:37 |
| *** AlexF has quit IRC | 19:37 | |
| anteaya | I'm for manual creation | 19:37 |
| clarkb | anteaya: and my response is that there is a simple way for them to get that functionality | 19:37 |
| anteaya | davidlyle is fine for horizon | 19:38 |
| clarkb | anteaya: if emagana_ is trusted managing groups I trust emagana_ to not push tags to neutronclient | 19:38 |
| clarkb | at least for now | 19:38 |
| *** avozza is now known as zz_avozza | 19:38 | |
| anteaya | I don't think that is fair to expect mestery to agree with that | 19:38 |
| clarkb | if we need to update this later I am fine with that | 19:38 |
| anteaya | in order to give emagana_ permissions | 19:38 |
| anteaya | but this is what they requested | 19:38 |
| anteaya | this is what works for them | 19:39 |
| fungi | one of these days i'm going to write a cli option for nodepool to spit out a "stale images" list | 19:39 |
| anteaya | no project does the same thing with third party ever | 19:39 |
| clarkb | anteaya: right all I am saying is we haven't even started on this yet | 19:40 |
| clarkb | anteaya: and before we start making exceptions and special cases I would really like to give it 5 minutes in the sun | 19:40 |
| anteaya | I'm letting you know what already works for neutron | 19:40 |
| jesusaurus | fungi: ++ but how stale is stale? | 19:40 |
| clarkb | no one had an issue with using the -release project in the proposals as initially proposed | 19:40 |
| fungi | jesusaurus: as stale as your additional command line parameter says | 19:40 |
| *** ddieterly has quit IRC | 19:41 | |
| fungi | jesusaurus: with maybe a default to one day | 19:41 |
| anteaya | that doesn't negate the fact that this is what they have requested | 19:41 |
| jesusaurus | one-day default seems reasonable | 19:41 |
| clarkb | and I have heard their requset and am saying "wait please" | 19:41 |
| anteaya | if you want to wait until tomorrow I can let them know | 19:41 |
| *** bknudson has joined #openstack-infra | 19:41 | |
| anteaya | fine | 19:41 |
| mestery | clarkb: I think CI enablement is different than pushing tags for client releases to be honest, I agree with anteaya here. | 19:41 |
| clarkb | mestery: yes understood | 19:41 |
| clarkb | mestery: but how often do you really change voting to not voting and back again? | 19:41 |
| mestery | But I have to take off for an appointment now, so I'll read the backscroll in an hour or so. | 19:41 |
| mestery | And I know where to find you all ;) | 19:41 |
| clarkb | this is what I am actually interested in | 19:41 |
| *** achanda has quit IRC | 19:41 | |
| mestery | clarkb: More than you think :) | 19:41 |
| mestery | We have a _lot_ of drivers and plugins :( | 19:42 |
| mestery | Anyways, going afk, back later | 19:42 |
| *** mestery is now known as mestery_afk | 19:42 | |
| *** achanda has joined #openstack-infra | 19:42 | |
| clarkb | anteaya: I want to wait until it has had a chance to work for a bit | 19:42 |
| clarkb | anteaya: we are not live yet | 19:42 |
| anteaya | clarkb: okay, just consider this information shared then | 19:43 |
| fungi | at the moment, the litmus test there is how often anteaya has been switching voting off and on for these accounts. i get the impression most of the urgent account actions have been gerrit account --inactive/--active rather than voting permission changes | 19:43 |
| *** sabeen2 has quit IRC | 19:43 | |
| anteaya | nova has requested nova-specs-core, neutron has requested mestery_afk and emagana_ and I haven't heard from cinder or horizon so make the ptl owner makes sense | 19:43 |
| *** sarob has joined #openstack-infra | 19:43 | |
| *** sarob has quit IRC | 19:44 | |
| *** mmaglana has joined #openstack-infra | 19:44 | |
| *** achanda has quit IRC | 19:46 | |
| clarkb | anteaya: right and all I am aksing is that we don't request anything just yet | 19:47 |
| *** armax has quit IRC | 19:47 | |
| clarkb | instead we make this go live, let it run for a bit then ask for suggsetions | 19:47 |
| *** rlandy has joined #openstack-infra | 19:47 | |
| clarkb | because really Idon't think we know what it is going to look like in a week | 19:47 |
| anteaya | okay, let's get this accnounced | 19:47 |
| anteaya | hopefully fewer things you and I need to take care of at the very least | 19:47 |
| clarkb | also, it is apparently possible for nova-release to switch to nova-specs-core on their own if they chose | 19:48 |
| anteaya | ah that I did not know | 19:48 |
| dhellmann | sdague: +2a | 19:49 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual: Add .gitreview content to Creator's Guide https://review.openstack.org/138831 | 19:49 |
| clarkb | I think https://etherpad.openstack.org/p/third-party-accounts-are-self-serve looks good | 19:50 |
| fungi | the current list of images which haven't built successfully in over 24 hours, with their respective ages: http://paste.openstack.org/show/144100 | 19:50 |
| *** armax has joined #openstack-infra | 19:51 | |
| fungi | we're doing very badly at building new images in hpcloud | 19:51 |
| anteaya | clarkb: I too am in agreement with the content of the etherpad | 19:51 |
| anteaya | since my name is at the bottom, I can send it | 19:51 |
| *** mwagner_lap has joined #openstack-infra | 19:51 | |
| clarkb | once that goes out I will be updating all-projects to stop the use of the old global groups | 19:51 |
| *** sabeen has joined #openstack-infra | 19:51 | |
| anteaya | fungi jeblair any last thoughts before we go live with third party ci account creation being self-serve? | 19:51 |
| anteaya | clarkb: awesome, thank you | 19:52 |
| fungi | anteaya: reviewing the announcement one last time | 19:52 |
| anteaya | fungi: thanks | 19:52 |
| *** wenlock1 has quit IRC | 19:52 | |
| *** Longgeek has quit IRC | 19:54 | |
| fungi | anteaya: one small clarification/addition there, but looks suitable to me | 19:55 |
| *** markvoelker has joined #openstack-infra | 19:55 | |
| fungi | don't want to give all projects the impression we created <project>-ci groups/acls for everything | 19:55 |
| clarkb | oh good point | 19:55 |
| * anteaya looks | 19:55 | |
| anteaya | agreed | 19:56 |
| anteaya | jeblair: any thoughts on https://etherpad.openstack.org/p/third-party-accounts-are-self-serve | 19:56 |
| *** tonytan4ever has quit IRC | 19:56 | |
| fungi | i think that was jeblair's text i just amended anyway | 19:56 |
| anteaya | great | 19:57 |
| fungi | clarkb: looks like we're also not successfully dib'ing for the past few days | 19:57 |
| anteaya | so should I wait for jeblair to weigh in again? or go ahead? | 19:57 |
| clarkb | we can wait for him since I need to eat soon | 19:57 |
| clarkb | >_> | 19:57 |
| *** armax has quit IRC | 19:57 | |
| *** ddieterly has joined #openstack-infra | 19:58 | |
| anteaya | okay | 19:58 |
| anteaya | I'm going to try a walk, the snow is making the land in the distance disappear | 19:58 |
| anteaya | so I might turn around early | 19:58 |
| anteaya | planning to be back in an hour | 19:59 |
| *** thedodd has joined #openstack-infra | 19:59 | |
| *** tykeal has joined #openstack-infra | 20:00 | |
| *** tonytan4ever has joined #openstack-infra | 20:00 | |
| fungi | hopefully pleia2 hasn't floated away in the great flood of '14 | 20:00 |
| clarkb | fungi: on the bright side they have needed rain | 20:00 |
| tykeal | anteaya: and clarkb mestery said you had a question about the ODL CI? | 20:00 |
| pleia2 | I saw the sun for a minute! | 20:00 |
| clarkb | tykeal: yes, you have two accounts with the same full name and email address | 20:00 |
| clarkb | tykeal: 10386 appears to be the one doing CI, I haven't figured out what the other one is yet | 20:01 |
| tykeal | clarkb: yeah.... I had been informed at one point that the display name needed to be changed so I had tried to create an account and do it... was a bad idea | 20:01 |
| tykeal | clarkb: the other one can be destroyed, deleted | 20:01 |
| clarkb | tykeal: can you change the details on that account so that it is less confusing? | 20:01 |
| clarkb | we cannot delete accounts | 20:01 |
| fungi | clarkb: i think if we --inactive the second account that will solve the issue too | 20:01 |
| tykeal | :-/ ok, I'll see what I can do | 20:01 |
| clarkb | fungi: we should do that once the data is up to date | 20:02 |
| fungi | yeah | 20:02 |
| clarkb | fungi: I mostly don't like have the same email addresses. The Full Name matching is less of an issue | 20:02 |
| esp | hello, anyone available to look into the failing tempest tests for this https://review.openstack.org/#/c/138190/ | 20:03 |
| clarkb | esp: http://logs.openstack.org/90/138190/3/check/gate-tempest-dsvm-src-django_openstack_auth-juno/5722c8f/logs/devstacklog.txt.gz#_2014-12-02_17_05_57_238 | 20:04 |
| *** dkranz has quit IRC | 20:04 | |
| esp | thx clarkb | 20:04 |
| fungi | /opt/stack/new/devstack/lib/horizon: line 151: _prepare_message_catalog_compilation: command not found | 20:04 |
| esp | and thx fungi | 20:04 |
| *** dprince has quit IRC | 20:04 | |
| openstackgerrit | Chad Roberts proposed openstack/requirements: Update python-saharaclient to 0.7.6 version https://review.openstack.org/138842 | 20:04 |
| tykeal | if I do a perma delete on the launchpad account will that do anything with the gerrit linkage? | 20:04 |
| clarkb | tykeal: no | 20:04 |
| tykeal | *sigh* | 20:05 |
| clarkb | tykeal: just log into the account and change the email address | 20:05 |
| tykeal | clarkb: it's using odl-openstack@opendaylight.org correct? | 20:05 |
| clarkb | ya | 20:05 |
| tykeal | just checking | 20:05 |
| clarkb | which is why it is ocnfusing because the other account uses the same address | 20:05 |
| *** e0ne has quit IRC | 20:05 | |
| tykeal | clarkb: should be odl-jenkins@opendaylight.org now... | 20:06 |
| clarkb | tykeal: yup thanks | 20:07 |
| clarkb | tykeal: and to confirm it isn't needed at all? I will mark the account inactive | 20:07 |
| *** mfink__ has joined #openstack-infra | 20:07 | |
| *** mfink has quit IRC | 20:07 | |
| tykeal | no, it's not needed. As I said, there had been some miscommunication to me and the account got created :-/ | 20:07 |
| clarkb | ok marking it inactive nowish | 20:07 |
| tykeal | clarkb: thanks | 20:08 |
| mtreinish | fungi: is there good time for when we can work through the mysql-proxy stuff? | 20:08 |
| *** esker has quit IRC | 20:08 | |
| *** annegent_ has quit IRC | 20:09 | |
| fungi | mtreinish: what's the roadmap for that? what step are you blocked on? | 20:09 |
| clarkb | tykeal: all done. thank you | 20:10 |
| tykeal | clarkb: sure thing | 20:10 |
| * tykeal waves this channel is super noisy for me ;) | 20:10 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/system-config: Replace link to GerritWorkflow https://review.openstack.org/138843 | 20:10 |
| *** tykeal has left #openstack-infra | 20:10 | |
| jeblair | anteaya, clarkb, fungi: etherpad lgtm; fire at will :) | 20:10 |
| *** yfried has quit IRC | 20:10 | |
| *** wenlock1 has joined #openstack-infra | 20:11 | |
| mtreinish | fungi: oh I can write one up on an etherpad, but there a couple hiera things, https://review.openstack.org/122311 , and then we need to add a "public" account to the subunit2sql db | 20:11 |
| fungi | mtreinish: no need for an etherpad unless you want. just trying to figure out what you need help with specifically | 20:12 |
| fungi | having a look at that review in a moment | 20:12 |
| mtreinish | I did test the puppet out beforehand this time :) It was a bit easier since it's mostly self contained | 20:13 |
| *** annegent_ has joined #openstack-infra | 20:13 | |
| *** hdd has quit IRC | 20:14 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: Fix layout-checks.py to work in virtualenv https://review.openstack.org/137679 | 20:14 |
| fungi | dhellmann: thanks! that's ^ something i've wanted for a while | 20:15 |
| *** markmcclain has quit IRC | 20:15 | |
| *** ldnunes has quit IRC | 20:15 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/system-config: Replace links to GerritWorkflow https://review.openstack.org/138843 | 20:15 |
| dhellmann | fungi: it's a rebase with 3 +2 already, so maybe you can approve it when the tests pass | 20:16 |
| *** Sukhdev has joined #openstack-infra | 20:17 | |
| fungi | dhellmann: yeah, it had somehow slipped my notice until now, but i'm trying it out myself in the background too for confirmation | 20:17 |
| *** dkranz has joined #openstack-infra | 20:17 | |
| dhellmann | fungi: it was blocked behind another patch related to removing stable-branch-compat jobs for oslo libs | 20:17 |
| *** hdd has joined #openstack-infra | 20:18 | |
| dhellmann | they're not actually related, I was just working on them at the same time | 20:18 |
| jeblair | meh, i went ahead and approved it; there's nothing else in the project-config queue for it to mess up :) | 20:18 |
| dhellmann | jeblair: cool, thanks | 20:18 |
| fungi | dhellmann: \o/ works for me on debian/testing | 20:19 |
| *** _nadya_ has joined #openstack-infra | 20:19 | |
| dhellmann | fungi: yeah, it was just a hard-coded path to /usr/bin/python so nothing fancy | 20:19 |
| *** tomhe has joined #openstack-infra | 20:20 | |
| fungi | dhellmann: right, i simply hadn't looked into why it didn't work because i was usually too busy testing some other patch with it and didn't want to unwind one step further to debug that | 20:20 |
| * dhellmann nods | 20:20 | |
| *** shashankhegde has quit IRC | 20:21 | |
| *** ChuckC has quit IRC | 20:21 | |
| fungi | mtreinish: so we need subunit2sql_db_and_port added to hiera, and mysql_proxy_admin_pass generated and added to hiera? | 20:22 |
| mtreinish | fungi: yep | 20:23 |
| fungi | mtreinish: what's the format for db_host_and_port | 20:23 |
| fungi | ? | 20:23 |
| *** Masahiro has joined #openstack-infra | 20:23 | |
| mtreinish | fungi: although we probably could be a bit more clever with what's in hiera for the subunit2sql db, because the full uri is already in there as another value | 20:23 |
| mtreinish | fungi: it should be hostname:port | 20:24 |
| *** ihrachyshka has joined #openstack-infra | 20:24 | |
| fungi | okay | 20:24 |
| tomhe | fungi: jeblair: Think I've found a problem in trigger/gerrit.py. We just added a new NoBlock label "Documentation-Consistent" in Gerrit and we get "can not merge" from Zuul. I'v tracked it down to an "else" with a "# IMPOSSIBLE" comment." | 20:24 |
| fungi | would be nice if we could decompose the existing dburi into multiple keys to avoid having duplicate data in there | 20:24 |
| *** dprince has joined #openstack-infra | 20:24 | |
| jeblair | tomhe: impossible! | 20:25 |
| jeblair | ;) | 20:25 |
| tomhe | :jeblair: Exactly my thought | 20:25 |
| mtreinish | fungi: yeah, that's what I was thinking, but I wasn't sure of how to do it | 20:25 |
| fungi | mtreinish: concatenate the different variables wherever the dburi is consumed? | 20:25 |
| *** bswartz has quit IRC | 20:26 | |
| * fungi is looking now to see if it's just in erb templates, in which case easy peasy | 20:26 | |
| jeblair | tomhe: can you paste the json of the submitrecord part of a query for that change? | 20:26 |
| openstackgerrit | Merged openstack-infra/project-config: Fix layout-checks.py to work in virtualenv https://review.openstack.org/137679 | 20:26 |
| mtreinish | fungi: yeah, I think it's just in the erb config file for subunit2sql | 20:27 |
| fungi | subunit2sql.conf.erb yep | 20:27 |
| *** Masahiro has quit IRC | 20:28 | |
| *** markmcclain has joined #openstack-infra | 20:29 | |
| *** otter768 has joined #openstack-infra | 20:30 | |
| fungi | so something like: connection = mysql://subunit2sql:<%= db_password %>@<%= db_host %>:3306/subunit2sql | 20:31 |
| mtreinish | fungi: yep that should work | 20:31 |
| fungi | or maybe more variables in there just sourced from the manifest instead of from hiera | 20:31 |
| fungi | add the building blocks we need, then compose the different formats you need for the various configs from them | 20:32 |
| *** hdd has quit IRC | 20:32 | |
| *** annegent_ has quit IRC | 20:32 | |
| fungi | i can go ahead and add the decomposed database hostname and password to hiera now | 20:32 |
| fungi | if you want to take that path | 20:32 |
| mtreinish | fungi: ok sure, I'll push out a patch to update the subunit2sql template to use the decomposed values | 20:33 |
| mtreinish | unless you wanted to | 20:33 |
| tomhe | jeblair: not sure how to get that json. Guess I should enable some logging? | 20:34 |
| *** xyang0 has quit IRC | 20:34 | |
| *** otter768 has quit IRC | 20:34 | |
| clarkb | tomhe: https://review.openstack.org/Documentation/cmd-query.html use that and query for a change id using the --submit-records and --format JSON options | 20:35 |
| fungi | mtreinish: i've included the key names i'm using in a review comment if you want to work on the new patchset part | 20:35 |
| mtreinish | fungi: ok sure | 20:36 |
| *** wenlock1 has quit IRC | 20:36 | |
| *** bswartz has joined #openstack-infra | 20:37 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for insufficient free space bug 1386249 https://review.openstack.org/132118 | 20:37 |
| uvirtbot | Launchpad bug 1386249 in tempest "Tests from package tempest.api.volume.admin sometimes fails due "insufficient free space"" [Undecided,Incomplete] https://launchpad.net/bugs/1386249 | 20:37 |
| fungi | mtreinish: oh, also "mysql_proxy_admin_pass" might be a little vague long-term | 20:38 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Check that descriptions exists in gerrit/projects.yaml https://review.openstack.org/138749 | 20:38 |
| nelsnelson | Greetings, channel. I've been trying to use the openstack-infra devstack-gate project to automate some tempest testing, but recently, the repo specified within the readme is no longer working. REPO_URL=https://review.openstack.org/p and ZUUL_PROJECT=openstack/nova and but the resulting https://review.openstack.org/p/openstack/nova repository is just a blank page with no errors. I'm sure I just missed a change announcement over the holiday. | 20:38 |
| fungi | mtreinish: do you mind changing it to subunit2sql_proxy_pass or something for the hiera key? | 20:38 |
| clarkb | nelsnelson: its not meant to be browsed by browsers, that is a location git can clone from | 20:39 |
| mtreinish | fungi: yeah that works, when I respin that patch I'll swap that too | 20:39 |
| *** hdd has joined #openstack-infra | 20:40 | |
| *** wenlock1 has joined #openstack-infra | 20:40 | |
| *** e0ne has joined #openstack-infra | 20:40 | |
| nelsnelson | clarkb, unfortunately, when I try to do a clone from either https://review.openstack.org/p/openstack/nova or https://review.openstack.org/p/openstack-infra/config, both fail. | 20:41 |
| nelsnelson | They were working a week ago. | 20:41 |
| fungi | mtreinish: thanks--noted that inline as well | 20:41 |
| clarkb | nelsnelson: openstack-infra/config was renamed to openstack-infra/project-config and openstack-infra/system-config | 20:41 |
| pleia2 | nelsnelson: I can't strictly speak to cloning from review.o.o, but config no longer exists, you probably want system-config | 20:42 |
| clarkb | nelsnelson: however nova should work (I am testing it locally and things are cloning) | 20:42 |
| * pleia2 too slow | 20:42 | |
| *** IvanBerezovskiy has quit IRC | 20:42 | |
| fungi | nelsnelson: i'm able to git clone from https://review.openstack.org/p/openstack/nova just fine | 20:43 |
| nelsnelson | I did start trying to use openstack-infra/system-config, but it only appears to work from the github.com source, and not the review.openstack.org/p source. I think that https://review.openstack.org/p/openstack-infra/devstack-gate has also been failing. | 20:43 |
| *** ihrachyshka has quit IRC | 20:43 | |
| openstackgerrit | Dmitry Teselkin proposed openstack-infra/zuul: Whitelist for gerrit events https://review.openstack.org/137634 | 20:43 |
| fungi | nelsnelson: though probably better to clone from git://git.openstack.org/openstack/nova.git rather than directly from our gerrit server (it's already loaded enough as it is and that /p subpath is just another replica anyway so there's nothing special about it) | 20:44 |
| nelsnelson | Okay, thanks fungi. Thanks clarkb. | 20:44 |
| *** vkmc has joined #openstack-infra | 20:44 | |
| openstackgerrit | Dmitry Teselkin proposed openstack-infra/zuul: Whitelist for gerrit events https://review.openstack.org/137634 | 20:44 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Switch subunit2sql to use decomposed uri https://review.openstack.org/138853 | 20:46 |
| mtreinish | fungi: ^^^ subunit2sql module switchover | 20:46 |
| vkmc | hi jeblair, clarkb hi! if you have a moment later... a Zaqar gate is failing and it seems to be something non related to us, the logs don't provide relevant information... is it possible that its something related to infra directly? | 20:48 |
| clarkb | vkmc: can you provide a link? | 20:48 |
| vkmc | clarkb, sure http://logs.openstack.org/05/138305/3/check/check-tempest-dsvm-redis-zaqar/ac56761/ | 20:48 |
| clarkb | vkmc: http://logs.openstack.org/05/138305/3/check/check-tempest-dsvm-redis-zaqar/ac56761/logs/devstacklog.txt.gz#_2014-12-03_15_37_40_554 | 20:49 |
| *** IvanBerezovskiy has joined #openstack-infra | 20:49 | |
| clarkb | http://logs.openstack.org/05/138305/3/check/check-tempest-dsvm-redis-zaqar/ac56761/logs/screen-zaqar-server.txt.gz | 20:49 |
| vkmc | clarkb, yeah, I checked those | 20:50 |
| vkmc | the screen shows that the server started | 20:50 |
| AJaeger_ | "ERROR] /opt/stack/new/devstack/lib/zaqar:195 Zaqar did not start" | 20:50 |
| clarkb | so zaqar isn't starting and isn't logging why it did not start | 20:50 |
| vkmc | yeah | 20:50 |
| clarkb | vkmc: well I think devstack is doing a check that it actually started | 20:50 |
| fungi | and also the initscript is exiting 0?!? | 20:50 |
| clarkb | fungi: ya something like that | 20:50 |
| fungi | fun times! | 20:50 |
| clarkb | I don't think this is an infra thing | 20:51 |
| fungi | agreed | 20:51 |
| vkmc | fungi, yeah, all the screens show the same | 20:51 |
| clarkb | timeout 60 sh -c 'while ! wget --no-proxy -q -O- http://127.0.0.1:8888/v1/health; do sleep 1; done' is failing | 20:51 |
| clarkb | maybe? anyways its likely something in your extras.d | 20:52 |
| fungi | that https://review.openstack.org/138305 change is a pretty massive refactor, so there be dragons | 20:52 |
| clarkb | or in zaqar itself that extras.d doesn't like | 20:52 |
| *** odyssey4me has joined #openstack-infra | 20:52 | |
| vkmc | clarkb, I'll try to dig in that direction | 20:52 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add mysql-proxy to enable read-only access to a db https://review.openstack.org/122311 | 20:52 |
| mtreinish | fungi: ^^^ | 20:52 |
| vkmc | fungi, yeah.. that's true, but its also happening in other patches | 20:52 |
| * anteaya is back from her walk, reads jeblair's post and prepares to email | 20:53 | |
| clarkb | anteaya: I am ready to update all-projects when ever you are | 20:53 |
| fungi | vkmc: oh, then yeah could be a change in devstack or in your job definition or in behavior of some dependency | 20:53 |
| vkmc | all right, thanks for your help clarkb, fungi | 20:53 |
| mtreinish | fungi, clarkb: one qq, nibalizer left a comment on an earlier review that I should have '@' in front of vars in the template files, should I add that to the subunit2sql conf while I'm doing the refactor? | 20:54 |
| nibalizer | only when you're using a variable injected by puppet | 20:55 |
| nibalizer | so if you're doing ip: <%= @ip %> then yes | 20:55 |
| nibalizer | but if you fancy stuff with defining your own variables in blocks then dont do it for those | 20:56 |
| mtreinish | nibalizer: https://review.openstack.org/#/c/138853/1/modules/subunit2sql/templates/subunit2sql.conf.erb | 20:56 |
| nibalizer | yep @'s on all of those | 20:56 |
| jeblair | tomhe: i'm about to be afk for a while; if you could open a bug on https://storyboard.openstack.org/#!/project/679 and paste that output, that would be nice | 20:56 |
| tomhe | jeblair: Got sidetracked, but here is the json: https://gist.github.com/tomhe/08b85ab9e5e8e15e251a | 20:56 |
| jeblair | hehe :) | 20:56 |
| tomhe | jeblair: still need me to open a bug? | 20:57 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Switch subunit2sql to use decomposed uri https://review.openstack.org/138853 | 20:57 |
| *** esker has joined #openstack-infra | 20:57 | |
| mtreinish | nibalizer: ok, thanks | 20:57 |
| anteaya | clarkb: thanks I wanted one last list before you nuke it | 20:58 |
| anteaya | clarkb: update as you are ready | 20:58 |
| *** AlexF has joined #openstack-infra | 20:58 | |
| anteaya | http://paste.openstack.org/show/144121/ | 20:58 |
| clarkb | vkmc: ya if you look at the script its doing that wget thing and saying it died | 20:58 |
| clarkb | vkmc: so wget isn't getting what it wants from the service | 20:58 |
| clarkb | anteaya: I will likely not remove people from those groups immediately | 20:59 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add mysql-proxy to enable read-only access to a db https://review.openstack.org/122311 | 20:59 |
| clarkb | anteaya: just update the acl on all-projects to start (that way we have the info for later if we need it) | 20:59 |
| clarkb | anteaya: start cleanup tomorrow probably | 20:59 |
| jeblair | tomhe: up to you; from looking at that and reading about noblock labels, i _think_ we just need to add 'MAY' to the 'OK' case. | 20:59 |
| vkmc | clarkb, yeah, that's right... the server doesn't start for some reason and the health retrieval fails | 20:59 |
| jeblair | tomhe: if you don't want to write that patch, would be a good idea to open a bug :) | 20:59 |
| vkmc | clarkb, thing is... why the server doesn't start | 20:59 |
| clarkb | vkmc: maybe redis isn't being started properly? | 20:59 |
| vkmc | clarkb, its possible yeah, it's harder to check because there is no error log in the screen | 21:00 |
| tomhe | jeblair: will do | 21:00 |
| clarkb | anteaya: jeblair fungi I am manually updating all-projects acl now | 21:01 |
| clarkb | probably just going to use the GUI | 21:01 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Switch subunit2sql to use decomposed uri https://review.openstack.org/138853 | 21:01 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add mysql-proxy to enable read-only access to a db https://review.openstack.org/122311 | 21:01 |
| anteaya | clarkb: cool | 21:01 |
| *** sarob has joined #openstack-infra | 21:02 | |
| *** esker has quit IRC | 21:03 | |
| clarkb | all done | 21:03 |
| *** armax has joined #openstack-infra | 21:03 | |
| *** odyssey4me has left #openstack-infra | 21:03 | |
| fungi | clarkb: awesome | 21:04 |
| fungi | i'm still working on figuring out why most of our images are not building in hpcloud | 21:05 |
| fungi | smart money says they changed one or more image names again | 21:05 |
| anteaya | clarkb: yay | 21:05 |
| anteaya | fungi: they do that? | 21:05 |
| clarkb | anteaya: all the time. | 21:05 |
| fungi | oh boy do they ever | 21:05 |
| anteaya | fungi: if I went into my hpcloud account would that help? | 21:06 |
| clarkb | mordred: has asked them to stop but they don't want to | 21:06 |
| anteaya | looked at their drop dowdn list? | 21:06 |
| anteaya | wow | 21:06 |
| mordred | sigh | 21:06 |
| clarkb | anteaya: you could compare the output of nova image-list against what we have in our nodepool.yaml | 21:06 |
| * anteaya no longer wants to ask why | 21:06 | |
| fungi | anteaya: you can compare it to the image names in nodepool.yaml.erb if you like | 21:06 |
| anteaya | I can try | 21:06 |
| * anteaya tries | 21:06 | |
| mordred | clarkb: sorry - I've been hoping that I could get us working base images across both clouds sooner than this | 21:06 |
| fungi | anteaya: it's an interesting exercise regardless | 21:06 |
| clarkb | mordred: me too | 21:06 |
| mordred | clarkb: but, well, all the reasons | 21:06 |
| clarkb | mordred: indeed | 21:06 |
| mtreinish | clarkb: do the id's at least stay constant, or are they changing too? | 21:07 |
| clarkb | mtreinish: the change | 21:07 |
| clarkb | mtreinish: I think they really do a add delete operation | 21:07 |
| fungi | they add updated images with modified names, and then eventually remove the older images | 21:07 |
| mtreinish | clarkb: sigh, yeah that's what they must be doing | 21:07 |
| mordred | well, the problem is that they also modify the old names | 21:07 |
| mordred | they add new images with new names | 21:08 |
| mordred | and also CHANGE the name of the old image | 21:08 |
| mordred | so that our stuff stops working | 21:08 |
| fungi | (just to make sure you notice!_ | 21:08 |
| mordred | they are doing this on purpose, btw | 21:08 |
| mordred | I have not yet convinced them to PLEASE FOR THE LOVE OF GOD STOP | 21:08 |
| jeblair | dib is the solution to this problem as well! | 21:09 |
| mordred | so I gave up and will just make our own image | 21:09 |
| fungi | because they're worried people will deploy from outdated images containing vulnerabilities and not update their servers? | 21:09 |
| mordred | yes | 21:09 |
| *** ihrachyshka has joined #openstack-infra | 21:09 | |
| mordred | there are SEVERAL problems we currently have that will all be solved by dib | 21:09 |
| *** hdd has quit IRC | 21:09 | |
| mordred | fungi: yes | 21:09 |
| fungi | mordred: in that case, you might want to have a look at why none of our dib images have updated for the past 2-3 days too ;) | 21:09 |
| fungi | it's on my list to look into next | 21:09 |
| clarkb | mordred: so you know how ou solve this? | 21:10 |
| clarkb | mordred: you do what rackspace does | 21:10 |
| mordred | yes | 21:10 |
| mordred | I know | 21:10 |
| clarkb | have a generic named thing that is alwas up to date | 21:10 |
| mordred | and that's what WE will do | 21:10 |
| clarkb | then hvae specific things in a corner | 21:10 |
| mordred | once we can make images for both | 21:10 |
| clarkb | that are hysterical | 21:10 |
| fungi | oh, also jenkins04 seems hella unhappy | 21:11 |
| mordred | in addition to nodepool dib - I fully intend for us to have our own dib-based base bare images | 21:11 |
| *** _nadya_ has quit IRC | 21:11 | |
| mordred | that do not change names | 21:11 |
| clarkb | oh right the jenkinses | 21:11 |
| *** hdd has joined #openstack-infra | 21:11 | |
| clarkb | I was going to look into making them work again today | 21:11 |
| mordred | once, you know, we can upload an image to rax | 21:11 |
| mordred | which I would finish now, but I'm going to go drink instead | 21:11 |
| nibalizer | does someone want to land this? its got a lot of positives and is a noop anyways https://review.openstack.org/#/c/136962/ | 21:12 |
| clarkb | holy crap 07 just let me login | 21:12 |
| fungi | it's responding, albeit slowly, via webui but nodepool is getting a fair number of api call timeouts to jenkins04 | 21:12 |
| * clarkb puts it into shutdown mode | 21:12 | |
| clarkb | or should I do that to 04 instaed? | 21:12 |
| *** vigneshvar_ has quit IRC | 21:12 | |
| fungi | so maybe yeah | 21:12 |
| fungi | jenkins04 could use your love first | 21:12 |
| clarkb | kk | 21:12 |
| *** asettle has quit IRC | 21:12 | |
| fungi | two birds with one stone that way | 21:13 |
| *** alex7376 has joined #openstack-infra | 21:13 | |
| *** alex7376 is now known as asettle | 21:13 | |
| clarkb | 04 won't let me login | 21:13 |
| clarkb | nonce verification failed | 21:14 |
| fungi | i was able to login, though slowly | 21:14 |
| fungi | might need a few tries | 21:14 |
| clarkb | date reports timestamp within seconds of my desktop | 21:14 |
| clarkb | like ~1 second difference at worst /me tries again | 21:15 |
| clarkb | oh wait it let me in | 21:15 |
| clarkb | huh | 21:15 |
| *** AJaeger_ has quit IRC | 21:15 | |
| mtreinish | clarkb: I was able to get in, it was just very slow... | 21:16 |
| clarkb | ok it is in shutdown mode now. there are a lot of offline nodes which I assume is beacuse nodepool is timing out trying to get those. I will try manually cleaning them up. If that fails will clean them up between stop and start of the service | 21:16 |
| *** MarkAtwood has quit IRC | 21:16 | |
| fungi | cool. i already cleaned up some stale nodes in nodepool and am deleting alien nodes now | 21:16 |
| clarkb | cacti doesn't show it as being in bad shape system wise | 21:17 |
| fungi | also removed a handful of disassociated floating ips in hpcloud, but we didn't really seem to be leaking there | 21:17 |
| clarkb | just jenkins itself being slow | 21:17 |
| *** mrunge has quit IRC | 21:17 | |
| *** ryanpetrello has joined #openstack-infra | 21:18 | |
| *** ChuckC has joined #openstack-infra | 21:18 | |
| *** mestery_afk is now known as mestery | 21:18 | |
| *** Ryan_Lane has quit IRC | 21:18 | |
| openstackgerrit | Merged openstack-infra/project-config: print pip freeze even when flake8 fails https://review.openstack.org/138514 | 21:19 |
| clarkb | trying to get /monitoring to load now | 21:19 |
| *** dangers_away is now known as dangers | 21:22 | |
| krtaylor | I just saw the Third-party self-serve account creation email - this is great news! | 21:23 |
| anteaya | so I am looking at the images via horizon and all the image names I see in the window appear to be what is in the nodepool.yaml.erb file | 21:23 |
| anteaya | including the dates in the image name | 21:23 |
| anteaya | would what is in horizon be different from what nova image list would give me? | 21:23 |
| clarkb | horizon won't show you the depreacted images iirc | 21:24 |
| clarkb | but that should be fnie for this check | 21:24 |
| *** dkliban is now known as dkliban_brb | 21:24 | |
| anteaya | okay well every file name I saw under hp for the nodepool.yaml.erb is available for me to select for instance creation | 21:25 |
| clarkb | thread list on 04 doesn't look too bad | 21:26 |
| clarkb | melody and the gearman manager thread have used the most cpu time | 21:26 |
| *** Ryan_Lane has joined #openstack-infra | 21:26 | |
| *** tomhe has quit IRC | 21:26 | |
| anteaya | note I didn't look at anything to do with tripleo | 21:27 |
| clarkb | anteaya: thats fine | 21:27 |
| fungi | anteaya: thanks! | 21:27 |
| anteaya | np | 21:27 |
| anteaya | now if I were to run nova image list would creating an instance do that for me? | 21:27 |
| fungi | i've got one of each of the failing image builds going on for hpcloud-b1 now to see if there is a commonality among them | 21:27 |
| anteaya | create an instance and run nova image list | 21:27 |
| anteaya | I can't see that working from a vm | 21:28 |
| *** emagana_ has quit IRC | 21:28 | |
| anteaya | fungi: awesome | 21:28 |
| fungi | a vm can definitely be configured to manage other instances via the nova api--that works just fine | 21:28 |
| *** emagana has joined #openstack-infra | 21:28 | |
| anteaya | okay | 21:28 |
| fungi | in fact we do it thousands of times every day! | 21:29 |
| fungi | clarkb: do we care about the devstack-centos7-dib images? | 21:29 |
| clarkb | no they cant build on precise | 21:30 |
| fungi | ahh, right, that old song | 21:30 |
| mtreinish | fungi: well sometimes there are network issues getting to the api server from a guest on a compute. (one of the config tools didn't allow this by default, I can't remember which one) | 21:30 |
| anteaya | been so long since I've created a vm | 21:30 |
| *** cpowell_ has quit IRC | 21:31 | |
| *** cpowell has joined #openstack-infra | 21:31 | |
| mtreinish | but I agree it definitely should be allowed, that was just a config bug | 21:31 |
| *** dmsimard is now known as dmsimard_away | 21:32 | |
| sdague | clarkb / fungi / jeblair - so this has been hanging out for a while - https://review.openstack.org/#/c/136512/ I think it's very minimal exposure, and saves a bunch of nodes for better things | 21:32 |
| *** adelinatuvenie has quit IRC | 21:33 | |
| *** emagana has quit IRC | 21:33 | |
| *** AlexF has quit IRC | 21:33 | |
| openstackgerrit | Merged openstack/requirements: Update to latest hacking https://review.openstack.org/138507 | 21:34 |
| *** shashankhegde has joined #openstack-infra | 21:34 | |
| *** baoli has joined #openstack-infra | 21:36 | |
| *** emagana has joined #openstack-infra | 21:37 | |
| openstackgerrit | Merged openstack-infra/project-config: only test libraries in a neutron config https://review.openstack.org/136512 | 21:38 |
| *** ociuhandu has quit IRC | 21:38 | |
| fungi | mtreinish: see inline comment on 122311 but i think you're missing a firewall rule update to make that proxy reachable | 21:40 |
| *** alexpilotti has quit IRC | 21:40 | |
| *** denis_makogon has quit IRC | 21:40 | |
| *** dmakogon_ has joined #openstack-infra | 21:40 | |
| mtreinish | fungi: ahh, yeah I figure that might have been the case, but I wasn't sure | 21:40 |
| mtreinish | by default it's actually the proxy runs on port 4040 | 21:40 |
| mtreinish | fungi: should I just copy one of the iptables rules already in logstash.pp and change the port? | 21:41 |
| *** Ryan_Lane has quit IRC | 21:41 | |
| mtreinish | oh, nm I'm dense, just add it to the list | 21:42 |
| *** jamielennox is now known as jamielennox|away | 21:42 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config: Add mysql-proxy to enable read-only access to a db https://review.openstack.org/122311 | 21:43 |
| mtreinish | fungi: ^^^ ok updated | 21:43 |
| *** AlexF has joined #openstack-infra | 21:45 | |
| fungi | mtreinish: that's what i had in mind. perfect | 21:46 |
| *** jgrimm is now known as zz_jgrimm | 21:46 | |
| *** dkliban_brb is now known as dkliban | 21:47 | |
| *** dpaterson has quit IRC | 21:48 | |
| anteaya | okay so after a confusing click path through the docs after searching for "openstack nova api" I end up on github: https://github.com/openstack/python-novaclient/ | 21:48 |
| anteaya | why are api docs on github? | 21:48 |
| annegentle | anteaya: hm | 21:48 |
| annegentle | where did you start your search? | 21:48 |
| * anteaya back tracks | 21:48 | |
| *** sputnik13 has quit IRC | 21:48 | |
| *** AlexF has quit IRC | 21:48 | |
| anteaya | started here: https://duckduckgo.com/?q=openstack+nova+api&t=canonical | 21:48 |
| *** amitgandhinz has quit IRC | 21:49 | |
| anteaya | selected top link nova.openstack.org | 21:49 |
| *** dprince has quit IRC | 21:49 | |
| anteaya | which resolves to: http://docs.openstack.org/developer/nova/ | 21:49 |
| anteaya | looked at http://docs.openstack.org/developer/nova/#api-extensions | 21:49 |
| anteaya | linked only link there | 21:49 |
| mtreinish | fungi: ok great, the other manual step needed for this was to add a user to mysql which has read access on the subunit2sql db that we can pass around with the proxy endpoint | 21:49 |
| *** adalbas has quit IRC | 21:50 | |
| anteaya | api.openstack.org which resolves to http://developer.openstack.org/ | 21:50 |
| anteaya | was confused | 21:50 |
| anteaya | clicked the api link in top right | 21:50 |
| anteaya | was redirected to further down the page | 21:50 |
| anteaya | http://developer.openstack.org/#api | 21:51 |
| annegentle | yup yup | 21:51 |
| fungi | mtreinish: and that's not represented in the puppet at all, right? | 21:51 |
| *** AlexF has joined #openstack-infra | 21:51 | |
| anteaya | guessed it wanted me to look at Command Line Interfaces | 21:51 |
| anteaya | so i picked compute and clicked | 21:51 |
| anteaya | github | 21:51 |
| annegentle | I can fix the nova link to api.openstack.org as it should go to developer.openstack.org/api-ref.html | 21:51 |
| *** Ryan_Lane has joined #openstack-infra | 21:51 | |
| mtreinish | fungi: nope, because I didn't think that we could manipulate the users on a trove db from puppet | 21:52 |
| anteaya | I will try dev.o.o/api-ref.html | 21:52 |
| anteaya | but what are we linking to anything on github? | 21:52 |
| annegentle | anteaya: I think we purposely go to github thinking people want the CLI itself not the docs for the CLI | 21:52 |
| annegentle | but that may be incorrect | 21:52 |
| anteaya | but why link to github? | 21:53 |
| fungi | mtreinish: i mean, it's not credentials used directly by the proxy, right? it's credentials someone would use to access the db via the proxy? | 21:53 |
| *** andreykurilin_ has joined #openstack-infra | 21:53 | |
| *** andreykurilin__ has quit IRC | 21:53 | |
| anteaya | if we want to link to source code, should be not link to git.o.o? | 21:53 |
| *** AlexF has quit IRC | 21:53 | |
| anteaya | *we | 21:54 |
| mtreinish | fungi: correct | 21:54 |
| *** dustins has quit IRC | 21:54 | |
| annegentle | anteaya: yep | 21:55 |
| fungi | mtreinish: what's the workflow you envision here... we just publish the read-only username/password for it? | 21:55 |
| annegentle | anteaya: I can patch that too, thanks for pointing it out | 21:55 |
| anteaya | annegentle: okay great, thanks | 21:55 |
| mtreinish | fungi: yeah pretty much or maybe we don't publish it, but just don't keep it a secret either | 21:56 |
| anteaya | ping me when you have a patch up, I'll review and thanks | 21:56 |
| *** _buzz_ has joined #openstack-infra | 21:56 | |
| *** bswartz has quit IRC | 21:56 | |
| fungi | mtreinish: okay, once that patch merges and we're ready to test it, i can add the read-only creds at that point | 21:56 |
| mtreinish | fungi: ok cool | 21:56 |
| *** apevec has joined #openstack-infra | 21:57 | |
| anteaya | and yes annegentle api-ref.html was what I was looking for, thank you | 21:57 |
| *** amitgandhinz has joined #openstack-infra | 21:57 | |
| mtreinish | eventually I don't think we'll need it, because we should have better mechanisms to get at the data. Like the dasboard mordred signed up to create | 21:57 |
| anteaya | shame it doesn't show up on a search | 21:57 |
| *** AlexF has joined #openstack-infra | 21:57 | |
| annegentle | anteaya: let's see how many weeks it takes for https://review.openstack.org/138871 to land | 21:58 |
| openstackgerrit | Doug Wiegley proposed openstack-infra/project-config: Add new neutron-lbaas repo to gerritbot for #openstack-lbaas https://review.openstack.org/138872 | 21:58 |
| mtreinish | but for the time being we need an easy way to get at the data, and I think it'll help with building tools on it | 21:58 |
| fungi | mtreinish: seems fine | 21:58 |
| *** _buzz has quit IRC | 21:58 | |
| *** Mithrandir has quit IRC | 21:59 | |
| anteaya | annegentle: well I reviewed so am tracking it as well | 21:59 |
| anteaya | annegentle: thanks | 22:00 |
| fungi | hrm, all the manual nodepool image updates in hpcloud-b1 are working, it seems. so maybe our thundering herd of simultaneously-triggered daily updates is simply overwhelming something there | 22:00 |
| *** prad has quit IRC | 22:00 | |
| *** sputnik13 has joined #openstack-infra | 22:00 | |
| *** AlexF has quit IRC | 22:01 | |
| *** melwitt has quit IRC | 22:01 | |
| jhesketh | Morning | 22:02 |
| *** melwitt has joined #openstack-infra | 22:02 | |
| clarkb | jhesketh: good morning. I am starting to try and prep for the jenkins restarts | 22:02 |
| clarkb | jhesketh: 04 is unhappy so starting there | 22:02 |
| mtreinish | fungi: I do have the first thing which will consume the data up WIP: https://review.openstack.org/#/c/136234/ | 22:02 |
| *** esker has joined #openstack-infra | 22:02 | |
| mtreinish | although from the testing clarkb and I did a while ago it didn't make much of a difference | 22:02 |
| *** tnovacik has quit IRC | 22:02 | |
| *** Mithrandir has joined #openstack-infra | 22:02 | |
| *** emagana has quit IRC | 22:02 | |
| jhesketh | clarkb: okay thanks. Let me know what I can do to help | 22:03 |
| *** emagana has joined #openstack-infra | 22:03 | |
| clarkb | jhesketh: at this point I just want to restart one jenkins msater asap so we can have a guinea pig before moving to the others later today | 22:03 |
| jhesketh | Yep, sounds sensible | 22:04 |
| anteaya | morning jhesketh | 22:04 |
| anteaya | jhesketh: so we seem to be converging on 0800 utc for a time, I'm thinking Tuesday | 22:05 |
| anteaya | jhesketh: your thoughts before I post to the mailing list? | 22:05 |
| *** odyssey4me has joined #openstack-infra | 22:06 | |
| anteaya | context: additional third party meeting time | 22:06 |
| *** odyssey4me has left #openstack-infra | 22:06 | |
| jhesketh | anteaya: sounds good to me | 22:07 |
| openstackgerrit | Adam Gandelman proposed openstack-infra/elastic-recheck: Add query for 1397796 https://review.openstack.org/138879 | 22:09 |
| anteaya | jhesketh: great thanks | 22:09 |
| anteaya | who does the nova docs | 22:09 |
| anteaya | they are fancy | 22:09 |
| *** Masahiro has joined #openstack-infra | 22:12 | |
| *** zxiiro has quit IRC | 22:13 | |
| *** esker has quit IRC | 22:13 | |
| *** mfink__ has quit IRC | 22:13 | |
| *** MaxV has quit IRC | 22:14 | |
| clarkb | I have been manually deleting slaves on 04 as I wait for jobs to finish because I was hopefully Iwould finish before the jobs | 22:14 |
| clarkb | this is a race I had no chance of winning :( | 22:14 |
| adam_g | clarkb, should we just cap alembic until neutron team can get a patch approved to fix the issue? | 22:14 |
| mattoliverau | Morning | 22:14 |
| clarkb | adam_g: we could do that too | 22:14 |
| clarkb | but then we wouldn't be testing that the fix fixes it. something to keep in mind | 22:15 |
| adam_g | clarkb, yeah | 22:15 |
| *** baoli has quit IRC | 22:15 | |
| *** zxiiro has joined #openstack-infra | 22:15 | |
| *** baoli has joined #openstack-infra | 22:16 | |
| *** Sukhdev has quit IRC | 22:17 | |
| *** cpowell has quit IRC | 22:17 | |
| *** Masahiro has quit IRC | 22:17 | |
| openstackgerrit | Adam Gandelman proposed openstack-infra/elastic-recheck: Add query for 1397796 https://review.openstack.org/138879 | 22:17 |
| openstackgerrit | Douglas Mendizábal proposed openstack-infra/project-config: Add Castellan project https://review.openstack.org/136140 | 22:17 |
| *** melwitt has quit IRC | 22:17 | |
| esp | clarkb and fungi can you direct me to where the tempest tests are written, trying to figure out why patches to django-openstack-auth has been failing last few days.. | 22:18 |
| clarkb | esp: openstack/tempest is the repo | 22:18 |
| esp | clarkb: yep, I atleast found that :) | 22:19 |
| *** yamamoto has joined #openstack-infra | 22:19 | |
| pleia2 | esp: folks in #openstack-qa can often help as well | 22:20 |
| *** melwitt has joined #openstack-infra | 22:20 | |
| esp | ah ok, thx pleia2 | 22:20 |
| *** armax has quit IRC | 22:20 | |
| fungi | okay, the remaining 37 stale images are being updated now and i'll retry them periodically for those that fail out on random errors | 22:21 |
| *** shashankhegde has quit IRC | 22:22 | |
| *** MaxV has joined #openstack-infra | 22:22 | |
| openstackgerrit | Adam Gandelman proposed openstack-infra/elastic-recheck: Add query for 1397796 https://review.openstack.org/138879 | 22:24 |
| *** mfink has joined #openstack-infra | 22:25 | |
| anteaya | morning mattoliverau | 22:25 |
| *** marun has quit IRC | 22:25 | |
| *** mfink has quit IRC | 22:26 | |
| *** shashankhegde has joined #openstack-infra | 22:26 | |
| *** esker has joined #openstack-infra | 22:26 | |
| *** ddieterly has quit IRC | 22:26 | |
| *** mfink has joined #openstack-infra | 22:26 | |
| *** dkranz has quit IRC | 22:26 | |
| fungi | oh! right we have mattoliverau again | 22:26 |
| *** ddieterly has joined #openstack-infra | 22:26 | |
| *** wenlock has quit IRC | 22:27 | |
| mattoliverau | Yeah, I decided to go exploring around the Europe post summit.. what can I say, I'm a slacker ;P | 22:27 |
| anteaya | jhesketh: http://lists.openstack.org/pipermail/openstack-dev/2014-December/052159.html | 22:28 |
| anteaya | and mattoliverau is always welcome to attend as well | 22:28 |
| mattoliverau | anteaya: cool :) | 22:29 |
| *** mattfarina has quit IRC | 22:30 | |
| *** otter768 has joined #openstack-infra | 22:30 | |
| *** ddieterly has quit IRC | 22:31 | |
| *** esker has quit IRC | 22:31 | |
| apevec | adam_g, clarkb - has alembic been capped in infra? This one has later timestamp 21:56 http://logs.openstack.org/62/138862/1/check/check-tempest-dsvm-neutron/41fc085/logs/ with alembic 0.7.0 in pip-freeze | 22:33 |
| *** tonytan4ever has quit IRC | 22:33 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for 1397796 https://review.openstack.org/138879 | 22:33 |
| apevec | vs 21:17 http://logs.openstack.org/80/137580/3/check/check-tempest-dsvm-neutron/e2776eb/logs/ with 0.7.1 | 22:33 |
| clarkb | apevec: we have 4 pypi mirrors | 22:33 |
| *** pc_m has quit IRC | 22:33 | |
| clarkb | they are not necessarily in sync with each other | 22:33 |
| fungi | though we would rather like them to be if possible | 22:34 |
| apevec | ah, more randomness to confuse us | 22:34 |
| clarkb | its not quite random | 22:34 |
| apevec | I mean, which job gets which | 22:34 |
| apevec | mirror | 22:34 |
| clarkb | that is determined by cloud region. each region has a mirror | 22:35 |
| *** otter768 has quit IRC | 22:35 | |
| *** shashankhegde has quit IRC | 22:35 | |
| *** ihrachyshka has quit IRC | 22:36 | |
| *** xyang8 has joined #openstack-infra | 22:38 | |
| mtreinish | clarkb: yeah, but which job goes to which region is random isn't it? | 22:39 |
| mtreinish | clarkb, jhesketh: when you get a sec https://review.openstack.org/138853 and https://review.openstack.org/122311 shouldn't be too bad to review :) | 22:39 |
| fungi | mtreinish: it depends on your use of the term "random" | 22:39 |
| *** MaxV has quit IRC | 22:40 | |
| fungi | mtreinish: it's more accurately "chaotic" | 22:40 |
| mtreinish | fungi: heh, fair enough :) | 22:40 |
| fungi | it's a choice being driven indirectly by a feedback loop from prior choices | 22:40 |
| fungi | hence effectively unpredictable | 22:41 |
| *** davideagnello has quit IRC | 22:41 | |
| clarkb | but the choice of mirror is not random I guess is what I am getting at | 22:42 |
| clarkb | that is deterministic based on where job ran | 22:42 |
| adam_g | zuul voodoo question: we currently need to merge a revert to projects requirements caps on client libraries to unwedge stable/juno, those are +A'd but cant merge till neutron alembic fix is backported to stable/juno. the latter cant merge without the former. are we jammed or will zuul test these together and merge if they are queued correctly and pass? | 22:42 |
| *** davideagnello has joined #openstack-infra | 22:43 | |
| clarkb | you are jammed | 22:43 |
| adam_g | rats | 22:43 |
| clarkb | which is sort of intentional as situations like this break continuous deployers | 22:43 |
| adam_g | clarkb, a temporary requirements change to exclude problematic alembic to get the reqs change through? | 22:43 |
| clarkb | adam_g: I think you uncap change should cap alembic | 22:44 |
| * adam_g checks | 22:44 | |
| clarkb | that will bypass the traffic jam | 22:44 |
| adam_g | naw | 22:44 |
| adam_g | https://review.openstack.org/#/c/138859/1/requirements.txt | 22:44 |
| clarkb | right in that same change cap alembic | 22:45 |
| clarkb | or rather in the uncapping requirements update do some capping :) | 22:45 |
| openstackgerrit | Khai Do proposed stackforge/python-jenkins: Handle job and node names that contain spaces https://review.openstack.org/138886 | 22:45 |
| adam_g | clarkb, thats an auto-proposal. we'd neeed to push the cap to requirements repo first | 22:45 |
| clarkb | yes | 22:45 |
| adam_g | k | 22:45 |
| clarkb | ok 04 is done running jobs finally I am going to stop it, clean slaves out, then start it | 22:45 |
| clarkb | jenkins refused to die by init script | 22:46 |
| clarkb | going to kill it with fire | 22:46 |
| *** yamamoto has quit IRC | 22:47 | |
| *** weshay has quit IRC | 22:47 | |
| *** shashankhegde has joined #openstack-infra | 22:47 | |
| mtreinish | clarkb: kill -s PWR? | 22:48 |
| clarkb | kill -9 | 22:50 |
| *** yamamoto has joined #openstack-infra | 22:50 | |
| clarkb | and its up again with no slaves | 22:50 |
| *** vkmc has quit IRC | 22:51 | |
| mtreinish | clarkb: ah, that's no fun. When you send sigpwr (it used to be -30) you get a nice msg "power fail" printed when it dies | 22:51 |
| *** apevec has quit IRC | 22:51 | |
| mtreinish | I used to have a lot of fun doing that to students on the clusters I managed when I was doing maint. | 22:51 |
| *** xyang8 has quit IRC | 22:51 | |
| clarkb | jhesketh: jenkins04 reports it has a postbuild script plugin at version 0.16. That look right? | 22:51 |
| *** krtaylor has quit IRC | 22:52 | |
| *** xyang8 has joined #openstack-infra | 22:52 | |
| jhesketh | clarkb: sounds right to me | 22:52 |
| *** Longgeek has joined #openstack-infra | 22:55 | |
| *** shashankhegde has quit IRC | 22:57 | |
| *** rlandy has quit IRC | 22:57 | |
| *** Ark has joined #openstack-infra | 22:58 | |
| *** Ark is now known as Guest63495 | 22:59 | |
| sweston | anteaya: regarding the project gating section, I am going to separate into paragraphs and include a few links to zuul docs, and add some details to make it flow better. sound okay? | 22:59 |
| *** Longgeek has quit IRC | 23:00 | |
| *** sarob has quit IRC | 23:00 | |
| *** shashankhegde has joined #openstack-infra | 23:01 | |
| clarkb | ok 04 is running jobs now | 23:04 |
| *** mjturek has quit IRC | 23:04 | |
| *** krtaylor has joined #openstack-infra | 23:04 | |
| *** mfink has quit IRC | 23:05 | |
| *** amitgandhinz has quit IRC | 23:07 | |
| *** jamielennox|away is now known as jamielennox | 23:09 | |
| *** armax has joined #openstack-infra | 23:11 | |
| *** e0ne has quit IRC | 23:12 | |
| *** pcrews has quit IRC | 23:13 | |
| *** cnesa has quit IRC | 23:16 | |
| clarkb | any opposition to me putting 07 and 06 into shutdown mode now? | 23:17 |
| *** odyssey4me has joined #openstack-infra | 23:17 | |
| *** xyang8 has quit IRC | 23:17 | |
| *** dimtruck is now known as zz_dimtruck | 23:17 | |
| *** emagana has quit IRC | 23:18 | |
| openstackgerrit | Steven Weston proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 23:18 |
| fungi | go for it | 23:18 |
| fungi | one straggler image left to retry, and then the run-tox.sh change should be safe | 23:18 |
| *** tomoe_ has joined #openstack-infra | 23:19 | |
| *** bknudson has quit IRC | 23:19 | |
| *** doug-fish has left #openstack-infra | 23:19 | |
| *** shardy is now known as shardy_z | 23:19 | |
| openstackgerrit | Steven Weston proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 23:20 |
| *** bswartz has joined #openstack-infra | 23:20 | |
| *** ryanpetrello has quit IRC | 23:24 | |
| *** julim has quit IRC | 23:27 | |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Move extended-choice support to parameters module https://review.openstack.org/116704 | 23:29 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Move extended-choice support to parameters module https://review.openstack.org/116704 | 23:29 |
| clarkb | huh two notices for the same thing? | 23:30 |
| clarkb | I am going to take a break while I wait for these jenkins to finish running jobs | 23:31 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Make the recipients argument optional https://review.openstack.org/135455 | 23:31 |
| *** dkliban is now known as dkliban_afk | 23:35 | |
| *** signed8bit has quit IRC | 23:37 | |
| jeblair | clarkb: glitch in the matrix | 23:37 |
| *** baoli has quit IRC | 23:37 | |
| fungi | wake up, neo | 23:37 |
| *** emagana has joined #openstack-infra | 23:37 | |
| openstackgerrit | Merged openstack-infra/subunit2sql: Remove unnecessary variable assignment https://review.openstack.org/130873 | 23:38 |
| *** marun has joined #openstack-infra | 23:38 | |
| *** baoli has joined #openstack-infra | 23:38 | |
| *** sputnik13 has quit IRC | 23:39 | |
| *** sarob has joined #openstack-infra | 23:47 | |
| fungi | yay! all relevant snapshot-based nodepool images are finally up to date. looking at the stale dib images now | 23:47 |
| *** marun has quit IRC | 23:48 | |
| fungi | weird... nodepool dib-image-list is oscillating between ready for 0 minutes and building for 0 minutes. that can't be right | 23:50 |
| *** ashp has joined #openstack-infra | 23:50 | |
| *** ryanpetrello has joined #openstack-infra | 23:51 | |
| clarkb | did we merge change to support to image types? | 23:51 |
| clarkb | maybe old db records didnt like that | 23:51 |
| *** dmsimard_away is now known as dmsimard | 23:51 | |
| clarkb | shouldve been compat though | 23:52 |
| fungi | hrm... `nodepool image-build devstack-precise-dib` is failing out very quickly | 23:52 |
| fungi | aha! | 23:53 |
| fungi | failed to create directory via template `/opt/dib_tmp/profiledir.XXXXXX': Too many links | 23:53 |
| fungi | i'll clean up | 23:53 |
| openstackgerrit | Steven Weston proposed openstack-infra/infra-manual: Adds project-gating description https://review.openstack.org/138595 | 23:53 |
| fungi | 31996 nodes in /opt/dib_tmp | 23:53 |
| fungi | basically all profiledir.XXXXXX cruft | 23:54 |
| *** sarob has quit IRC | 23:55 | |
| clarkb | ya new nodepool didnt have that problem. possible dib bug on precise | 23:55 |
| *** baoli has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!