| sdague | heh, tripleo repos basically are 95% recheck no bug | 00:00 |
|---|---|---|
| *** Ryan_Lane has joined #openstack-infra | 00:00 | |
| sdague | this old stats script is less useful than I remember | 00:00 |
| *** wenlock has joined #openstack-infra | 00:01 | |
| jogo | sdague: hehe | 00:01 |
| jogo | sdague: I haven't checked it in months at this point | 00:01 |
| jogo | I just look at the unclassified list instead | 00:01 |
| openstackgerrit | Elizabeth Krumbach Joseph proposed a change to openstack-infra/config: Replace Rechecks with elastic-recheck page https://review.openstack.org/93608 | 00:01 |
| sdague | jogo: oh, not rechecks | 00:02 |
| sdague | I wrote some local scripts to figure out how bad we were doing on 'recheck no bug' | 00:02 |
| jogo | sdague: ohhh cool | 00:02 |
| * jogo wonders off to find breakfast | 00:02 | |
| sdague | oh, right, japan | 00:02 |
| jhesketh | clarkb: will look shortly (re oslog patch) | 00:03 |
| jogo | sdague: yup, giving my (aka your) talk in a bit | 00:03 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/os-loganalyze: Add support to check swift for log files https://review.openstack.org/76796 | 00:05 |
| *** MarkAtwood has joined #openstack-infra | 00:06 | |
| *** matsuhashi has joined #openstack-infra | 00:07 | |
| clarkb | mordred: I have no idea why your revert revert of the cloud things isn't working | 00:07 |
| clarkb | mordred: I rebase it and git says it is a noop | 00:07 |
| clarkb | so I am going to hit it with a hammer and push a new patchset | 00:07 |
| clarkb | oh because its a revert revert | 00:07 |
| clarkb | silly git | 00:07 |
| jhesketh | sdague, jeblair, clarkb: https://review.openstack.org/#/c/76796 is rebased and ready to go | 00:09 |
| *** Sukhdev has joined #openstack-infra | 00:10 | |
| *** jergerber has quit IRC | 00:11 | |
| *** crc32 has quit IRC | 00:12 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Revert "Revert "Stop using hpcloud az1-az3 in favor of region-b"" https://review.openstack.org/94503 | 00:13 |
| clarkb | mordred: ^ there you go | 00:13 |
| *** dangers_away has quit IRC | 00:13 | |
| clarkb | jhesketh: +2 | 00:13 |
| clarkb | I am going to walk home now | 00:14 |
| jhesketh | thanks clarkb :-) | 00:14 |
| *** dangers_away has joined #openstack-infra | 00:14 | |
| *** sarob has quit IRC | 00:16 | |
| *** nati_uen_ has joined #openstack-infra | 00:21 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Check for rabbitmq before archiving https://review.openstack.org/93250 | 00:21 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Clean up sudoers archive https://review.openstack.org/93249 | 00:21 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add distro check functions for Ubuntu and Fedora https://review.openstack.org/92748 | 00:21 |
| *** MarkAtwood has quit IRC | 00:23 | |
| *** nati_ueno has quit IRC | 00:24 | |
| *** arnaud_ has quit IRC | 00:26 | |
| *** wenlock has quit IRC | 00:27 | |
| *** bhuvan has quit IRC | 00:28 | |
| *** msabramo has joined #openstack-infra | 00:29 | |
| *** wenlock has joined #openstack-infra | 00:30 | |
| *** arnaud_ has joined #openstack-infra | 00:32 | |
| *** HenryG_ has joined #openstack-infra | 00:37 | |
| *** dstanek is now known as dstanek_zzz | 00:37 | |
| *** praneshp has quit IRC | 00:38 | |
| *** HenryG_ has quit IRC | 00:38 | |
| *** lxsli has joined #openstack-infra | 00:39 | |
| *** alugovoi has quit IRC | 00:40 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Modify horizon log copy for Fedora https://review.openstack.org/93251 | 00:43 |
| *** MarkAtwood has joined #openstack-infra | 00:46 | |
| *** yaguang has joined #openstack-infra | 00:47 | |
| *** wenlock has quit IRC | 00:47 | |
| *** MarkAtwood has quit IRC | 00:47 | |
| *** wenlock has joined #openstack-infra | 00:50 | |
| *** ramashri has quit IRC | 00:50 | |
| jhesketh | if anybody has time to review this patch chain to zuul's status page that'd be great: https://review.openstack.org/#/c/91316/ | 00:52 |
| jhesketh | (that's the tail) | 00:52 |
| jhesketh | this is blocking having zuul's status overlay on gerrit | 00:52 |
| *** Ryan_Lane has quit IRC | 00:53 | |
| *** homeless_ has joined #openstack-infra | 00:53 | |
| *** homeless has quit IRC | 00:55 | |
| *** homeless has joined #openstack-infra | 01:01 | |
| *** homeless_ has quit IRC | 01:04 | |
| sdague | stat of the night, 1200 recheck bugs (with bug #) in the last 2w | 01:04 |
| *** homeless has quit IRC | 01:05 | |
| *** wenlock has quit IRC | 01:05 | |
| mtreinish | clarkb: ok, yeah that makes sense. I'll respin it now | 01:09 |
| *** wenlock has joined #openstack-infra | 01:09 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: enable heat in grenade https://review.openstack.org/87677 | 01:10 |
| *** resker has joined #openstack-infra | 01:15 | |
| *** malini_afk is now known as malini | 01:17 | |
| mrodden | nothing is more frustrating when people don't understand the point of regression tests... | 01:17 |
| mrodden | then when* | 01:17 |
| *** esker has quit IRC | 01:18 | |
| *** amcrn_ has quit IRC | 01:20 | |
| *** nati_uen_ has quit IRC | 01:21 | |
| *** dstanek_zzz is now known as dstanek | 01:23 | |
| sdague | apparently pasting 2.5k lines into paste.openstack.org is not ok | 01:23 |
| mrodden | heh | 01:23 |
| sdague | well, if anyone cares, here are all the rechecks entered by hand in the last 2 weeks in gerrit - https://dague.net/testing/rechecks.txt | 01:24 |
| sdague | I'll complete the bulk analysis in the morning | 01:24 |
| *** lxsli has quit IRC | 01:24 | |
| *** lxsli has joined #openstack-infra | 01:25 | |
| mrodden | the list of public shaming | 01:25 |
| mrodden | holy crap some of these are bad... | 01:25 |
| *** wenlock has quit IRC | 01:26 | |
| sdague | well, this wasn't for shaming | 01:27 |
| mrodden | right... | 01:27 |
| sdague | it was for figuring out if any data in there was useful beyond what we're doing with elastic recheck | 01:27 |
| mrodden | but there are some names in here that are like all recheck no bug | 01:27 |
| *** amcrn has joined #openstack-infra | 01:27 | |
| mrodden | :/ | 01:27 |
| sdague | yeh | 01:27 |
| sdague | feel free to go shame folks :) | 01:28 |
| sdague | it's all publicly accessible data | 01:28 |
| mrodden | actually some of them aren't surprising | 01:28 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Capture *.failure logs from $BASE/status/stack https://review.openstack.org/91965 | 01:28 |
| *** nati_ueno has joined #openstack-infra | 01:28 | |
| mtreinish | clarkb: ^^^ | 01:29 |
| *** wenlock has joined #openstack-infra | 01:29 | |
| *** nati_ueno has quit IRC | 01:29 | |
| mrodden | sdague: will you look at reviews if i add you to them in Gerrit or would you want me to ping them to you? | 01:29 |
| *** otherwiseguy has joined #openstack-infra | 01:29 | |
| *** nati_ueno has joined #openstack-infra | 01:30 | |
| mrodden | bash8 got pretty busy before i was ready | 01:30 |
| mrodden | so now i'm scrambling to add tests so we don't break devstack when it gets moved over... | 01:30 |
| *** paul--_ has quit IRC | 01:31 | |
| *** moted has quit IRC | 01:32 | |
| *** dstanek is now known as dstanek_zzz | 01:33 | |
| mtreinish | mrodden: just break devstack, then sdague will have to look at it | 01:33 |
| mrodden | mtreinish: makes sense to me :) | 01:34 |
| *** nati_ueno has quit IRC | 01:38 | |
| *** mriedem has joined #openstack-infra | 01:39 | |
| *** michchap has quit IRC | 01:40 | |
| *** aconrad has quit IRC | 01:41 | |
| *** ArxCruz has joined #openstack-infra | 01:44 | |
| *** wenlock has quit IRC | 01:45 | |
| *** thomasbiege1 has joined #openstack-infra | 01:47 | |
| *** dstanek_zzz is now known as dstanek | 01:47 | |
| *** thedodd has joined #openstack-infra | 01:48 | |
| *** thedodd has quit IRC | 01:50 | |
| *** thomasbiege has quit IRC | 01:51 | |
| *** wenlock has joined #openstack-infra | 01:51 | |
| *** terryw has joined #openstack-infra | 01:53 | |
| *** otherwiseguy has quit IRC | 01:53 | |
| *** paul--_ has joined #openstack-infra | 01:55 | |
| *** aveiga has quit IRC | 01:56 | |
| *** michchap has joined #openstack-infra | 01:56 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/nodepool: Make template and node hostnames configurable https://review.openstack.org/90883 | 01:56 |
| *** gus has quit IRC | 01:57 | |
| *** dstanek is now known as dstanek_zzz | 01:57 | |
| *** gus has joined #openstack-infra | 01:59 | |
| *** terryw has quit IRC | 02:00 | |
| *** wenlock_ has joined #openstack-infra | 02:00 | |
| *** saschpe has quit IRC | 02:00 | |
| *** saschpe has joined #openstack-infra | 02:02 | |
| *** signed8bit has quit IRC | 02:03 | |
| *** Sukhdev has quit IRC | 02:05 | |
| *** aconrad has joined #openstack-infra | 02:05 | |
| *** malini is now known as malini_afk | 02:05 | |
| *** markwash has quit IRC | 02:05 | |
| *** malini_afk is now known as malini | 02:09 | |
| *** lxsli has quit IRC | 02:10 | |
| *** yaguang has quit IRC | 02:10 | |
| *** otherwiseguy has joined #openstack-infra | 02:11 | |
| jogo | random question: do incubated projects go under stackforge or openstack? | 02:12 |
| jogo | re:barbacian | 02:14 |
| clarkb | openstack | 02:15 |
| *** alugovoi has joined #openstack-infra | 02:16 | |
| *** xchu has joined #openstack-infra | 02:19 | |
| jogo | ahh in that case, its not ther | 02:20 |
| jogo | clarkb: its still under stackforge | 02:20 |
| *** unicell has quit IRC | 02:21 | |
| *** aconrad has quit IRC | 02:22 | |
| *** HenryG has quit IRC | 02:22 | |
| clarkb | is it part of friday's renaming? | 02:23 |
| *** markwash has joined #openstack-infra | 02:23 | |
| *** bknudson has quit IRC | 02:24 | |
| *** paul--_ has quit IRC | 02:25 | |
| *** paul--_ has joined #openstack-infra | 02:25 | |
| jogo | clarkb: no idea | 02:26 |
| *** UtahDave has quit IRC | 02:27 | |
| jogo | clarkb: don't know how to check | 02:28 |
| *** wenlock_ has quit IRC | 02:31 | |
| *** amcrn has quit IRC | 02:35 | |
| *** gyee has quit IRC | 02:36 | |
| *** yaguang has joined #openstack-infra | 02:38 | |
| *** dstanek_zzz is now known as dstanek | 02:38 | |
| *** praneshp has joined #openstack-infra | 02:38 | |
| *** otter768 has joined #openstack-infra | 02:39 | |
| *** changbl has joined #openstack-infra | 02:41 | |
| *** yaguang has quit IRC | 02:44 | |
| *** james_li has joined #openstack-infra | 02:44 | |
| *** rwsu has quit IRC | 02:44 | |
| *** jgrimm has joined #openstack-infra | 02:46 | |
| *** thedodd has joined #openstack-infra | 02:48 | |
| *** unicell has joined #openstack-infra | 02:49 | |
| *** paul--_ has quit IRC | 02:49 | |
| *** james_li has quit IRC | 02:54 | |
| *** hipster has quit IRC | 02:56 | |
| *** thedodd has quit IRC | 02:56 | |
| *** yaguang has joined #openstack-infra | 02:57 | |
| *** markwash has quit IRC | 02:57 | |
| *** wenlock_ has joined #openstack-infra | 02:57 | |
| *** otter768 has quit IRC | 02:59 | |
| *** dims_ has joined #openstack-infra | 03:03 | |
| *** arnaud_ has quit IRC | 03:04 | |
| *** yaguang has quit IRC | 03:04 | |
| *** harlowja is now known as harlowja_away | 03:05 | |
| *** yaguang has joined #openstack-infra | 03:08 | |
| *** otter768 has joined #openstack-infra | 03:13 | |
| *** mriedem has quit IRC | 03:14 | |
| *** ArxCruz has quit IRC | 03:19 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Save sql logs https://review.openstack.org/94314 | 03:21 |
| *** yaguang has quit IRC | 03:21 | |
| *** yaguang has joined #openstack-infra | 03:22 | |
| *** otter768 has quit IRC | 03:22 | |
| jgriffith | jeblair: good evening | 03:33 |
| jgriffith | jeblair: wondered if you knew of a contact that succesfully setup the 3'rd party CI? | 03:33 |
| jgriffith | jeblair: I'm a bit stuck :( | 03:33 |
| *** yaguang has quit IRC | 03:36 | |
| clarkb | jgriffith: https://review.openstack.org/#/admin/groups/91,members is the voting third party ci group. in theory they all have working systme | 03:37 |
| clarkb | anything in particular bothering you? | 03:37 |
| jgriffith | clarkb: I can't seem to get gearman to connect | 03:38 |
| jgriffith | clarkb: it's rather annoying and there's not a ton of info in the logs | 03:38 |
| jgriffith | wow... that list is considerably longer than I had expected | 03:38 |
| clarkb | jgriffith: gearman between which two points? | 03:38 |
| jgriffith | clarkb: well, I'm just following Jay's doc | 03:39 |
| jgriffith | clarkb: installed the jenkins-master | 03:39 |
| *** otherwiseguy has quit IRC | 03:39 | |
| jgriffith | clarkb: the next step is enable gearman | 03:39 |
| jgriffith | clarkb: http://goo.gl/KICmcI | 03:40 |
| jgriffith | I'm down to the "Manage Jenkins" and "Configure System" portion | 03:40 |
| jgriffith | Select enable and the connection fails | 03:40 |
| clarkb | the test connection bit fails? | 03:41 |
| jgriffith | yep | 03:41 |
| jgriffith | I get a "Failed: Unable to Connect" | 03:41 |
| clarkb | jgriffith: to test that I would do `telnet fqdnoripofzuulhost 4730` and see if that connection works | 03:41 |
| jgriffith | ahhh | 03:42 |
| jgriffith | ok, same error | 03:42 |
| clarkb | if that doesn't work then we should make sure zuul is running and that it has two processes. The child is the geard service that should listen on 4730 | 03:43 |
| *** W00die has joined #openstack-infra | 03:43 | |
| clarkb | jgriffith: so either firewall filtering you or service isn't listening | 03:43 |
| jgriffith | makes sense | 03:43 |
| jgriffith | I suspect service is not running | 03:43 |
| jgriffith | but lemme double check firewall | 03:43 |
| jgriffith | Ok, so I just disabled ufw | 03:45 |
| jgriffith | I don't see a zuul process though | 03:45 |
| jgriffith | clarkb: you're the man! | 03:46 |
| jgriffith | clarkb: that was it, did a service restart of zuul and got past that point | 03:47 |
| clarkb | awesome | 03:47 |
| jgriffith | clarkb: thanks for the assist | 03:47 |
| clarkb | np | 03:47 |
| *** msabramo has quit IRC | 03:48 | |
| *** nosnos has joined #openstack-infra | 03:48 | |
| *** msabramo has joined #openstack-infra | 03:49 | |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Add slavescript to grab console log https://review.openstack.org/94520 | 03:49 |
| openstackgerrit | Joshua Hesketh proposed a change to openstack-infra/config: Add in JJB macros for pushing to swift https://review.openstack.org/94521 | 03:49 |
| *** malini is now known as malini_afk | 03:55 | |
| jgriffith | clarkb: well that's progress, I'll take a crack at finishing it up tomorrow | 03:56 |
| jgriffith | clarkb: thanks again for the help | 03:56 |
| jgriffith | clarkb: BTW, further thought/data you and jeblair were' correct, there's no reason to NOT do every Cinder commit | 03:56 |
| *** wenlock_ has quit IRC | 03:57 | |
| jgriffith | anywa... catch ya'll tomorrow | 03:57 |
| openstackgerrit | ayoung proposed a change to openstack/requirements: kerberos requires an additional requests library. Older versions break in py33 https://review.openstack.org/84740 | 03:58 |
| *** arnaud_ has joined #openstack-infra | 03:59 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Use pbr, pep8 checks & fixes, add "doc" tox target https://review.openstack.org/90455 | 04:06 |
| *** lxsli has joined #openstack-infra | 04:10 | |
| tchaypo | Does anyone know offhand if gerrit-powered-agenda is in use? | 04:10 |
| *** jaypipes has joined #openstack-infra | 04:11 | |
| tchaypo | If not I suspect ttx is still manually maintaing the calendar | 04:11 |
| ianw | clarkb: do you know if .conf files get indexed, or only .txt? i.e. do we really need to rename .conf to .txt ? | 04:12 |
| clarkb | tchaypo: I don't think it is in use yet | 04:12 |
| tchaypo | boo hiss. Shall have to nag ttx then | 04:13 |
| tchaypo | thanks clarkb | 04:13 |
| clarkb | ianw: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/logstash/jenkins-log-client.yaml that has the list of files | 04:13 |
| *** W00die has quit IRC | 04:14 | |
| ianw | clarkb: ah, ok, so i guess the rename of files such as in http://logs.openstack.org/48/93248/1/gate/gate-tempest-dsvm-full/7ea0a73/logs/etc/ceilometer/ is just so they get sent down to the browser as text you can see when you click? | 04:14 |
| clarkb | yes | 04:14 |
| ianw | clarkb: ok, got it, thanks | 04:15 |
| *** yfried has quit IRC | 04:17 | |
| *** w__ has quit IRC | 04:17 | |
| *** Sukhdev has joined #openstack-infra | 04:21 | |
| tchaypo | related question - we in #tripleo are about to spin up a new meeting with a new chair, but we're not sure if we need to do anything to add people to the meetbot ACL or if it will just work for anyone | 04:22 |
| tchaypo | I could just give it a try but I'd prefer not to create spurious logs of fake test meetings if I can avoid it | 04:22 |
| *** ihrachyshka has joined #openstack-infra | 04:23 | |
| clarkb | it will work for anyone | 04:23 |
| *** changbl has quit IRC | 04:23 | |
| tchaypo | cool. | 04:24 |
| tchaypo | thanks | 04:24 |
| *** yaguang has joined #openstack-infra | 04:26 | |
| *** ihrachyshka has quit IRC | 04:26 | |
| *** dims_ has quit IRC | 04:29 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add tox "coverage" target https://review.openstack.org/87382 | 04:31 |
| *** eharney has quit IRC | 04:35 | |
| zaro | clarkb: email draft looks good, one question though. | 04:42 |
| *** morganfainberg is now known as morganfainberg_Z | 04:42 | |
| zaro | clarkb: it seems to imply that you will change all drafts to WIP. i was think that you should be more explicit about that. | 04:44 |
| *** yaguang has quit IRC | 04:44 | |
| clarkb | I wont do that and it explicitly says I wont | 04:45 |
| clarkb | where are you reading the implication? | 04:45 |
| zaro | clarkb: "Draft patchsets will remain in that state until they are published". i thought 'that' meant WIP. | 04:48 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Support syslogs on Fedora https://review.openstack.org/93248 | 04:48 |
| clarkb | I can use a proper noun there | 04:48 |
| *** lxsli has quit IRC | 04:49 | |
| *** lxsli has joined #openstack-infra | 04:51 | |
| zaro | clarkb: you ever dvr nba games? | 04:51 |
| clarkb | nope I dont have a dvr | 04:54 |
| zaro | probably for the best. | 04:55 |
| *** jgrimm has quit IRC | 04:56 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-infra/devstack-gate: Remove rename call in log cleanup https://review.openstack.org/92981 | 04:57 |
| *** igor_ has quit IRC | 05:02 | |
| *** marcoemorais has joined #openstack-infra | 05:03 | |
| *** yaguang has joined #openstack-infra | 05:03 | |
| *** wenlock has quit IRC | 05:04 | |
| *** marcoemorais1 has joined #openstack-infra | 05:04 | |
| *** salv-orlando has quit IRC | 05:05 | |
| *** igor has joined #openstack-infra | 05:06 | |
| *** marcoemorais has quit IRC | 05:07 | |
| *** harlowja_at_home has joined #openstack-infra | 05:09 | |
| *** harlowja_at_home has quit IRC | 05:10 | |
| mordred | clarkb: ++ on disabling email | 05:10 |
| *** Sukhdev has quit IRC | 05:13 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/jenkins-job-builder: Add tox "coverage" target https://review.openstack.org/87382 | 05:21 |
| *** reed has quit IRC | 05:21 | |
| *** dstanek is now known as dstanek_zzz | 05:23 | |
| *** yfried has joined #openstack-infra | 05:25 | |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/config: Add non-voting jenkins-job-builder-coverage https://review.openstack.org/94531 | 05:30 |
| msabramo | zaro: Is this the right way to add a non-voting job to measure JJB coverage? https://review.openstack.org/94531 | 05:32 |
| *** wenlock has joined #openstack-infra | 05:33 | |
| msabramo | non-voting because it's going to fail until a new pbr is released by mordred with —coverage-package-name | 05:33 |
| *** lcheng_ has joined #openstack-infra | 05:43 | |
| *** paul--_ has joined #openstack-infra | 05:43 | |
| *** dstanek_zzz is now known as dstanek | 05:44 | |
| *** rlandy has joined #openstack-infra | 05:45 | |
| *** _nadya_ has joined #openstack-infra | 05:46 | |
| *** greghaynes has quit IRC | 05:50 | |
| *** pcrews_ has quit IRC | 05:52 | |
| *** dstanek is now known as dstanek_zzz | 05:54 | |
| *** lxsli has quit IRC | 05:55 | |
| *** e0ne has joined #openstack-infra | 05:55 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/devstack-gate: Save sql logs https://review.openstack.org/94314 | 05:58 |
| *** wenlock_ has joined #openstack-infra | 05:58 | |
| *** e0ne has quit IRC | 06:03 | |
| *** resker has quit IRC | 06:08 | |
| *** esker has joined #openstack-infra | 06:08 | |
| *** _nadya_ has quit IRC | 06:08 | |
| *** wenlock_ has quit IRC | 06:09 | |
| *** jlibosva has joined #openstack-infra | 06:13 | |
| *** _nadya_ has joined #openstack-infra | 06:22 | |
| *** wenlock has quit IRC | 06:28 | |
| *** doude has joined #openstack-infra | 06:31 | |
| *** paul--_ has quit IRC | 06:33 | |
| *** paul--_ has joined #openstack-infra | 06:34 | |
| praneshp | clarkb mordred: ping. Is there documentation on how I should enable bp/bug linking from commits in my stackforge prokect? | 06:41 |
| praneshp | for example, the link in https://review.openstack.org/#/c/94298/ should point to https://blueprints.launchpad.net/openstack-entropy/+spec/backend-abstraction but returns not found | 06:42 |
| *** dstanek_zzz is now known as dstanek | 06:45 | |
| *** masayukig has joined #openstack-infra | 06:48 | |
| *** lcheng_ has quit IRC | 06:49 | |
| *** yaguang has quit IRC | 06:50 | |
| *** vtapia has joined #openstack-infra | 06:50 | |
| *** lcheng_ has joined #openstack-infra | 06:50 | |
| *** dstanek is now known as dstanek_zzz | 06:55 | |
| openstackgerrit | Gaudenz Steinlin proposed a change to openstack-infra/elastic-recheck: Add e-r query for bug 1320670 https://review.openstack.org/94486 | 06:56 |
| uvirtbot | Launchpad bug 1320670 in grenade "404 on GET /v3/OS-SIMPLE-CERT/ca at grenade" [Undecided,In progress] https://launchpad.net/bugs/1320670 | 06:57 |
| *** pblaho has joined #openstack-infra | 06:59 | |
| *** flaper87|afk is now known as flaper87 | 07:02 | |
| *** jcoufal has joined #openstack-infra | 07:05 | |
| *** Longgeek has joined #openstack-infra | 07:06 | |
| *** e0ne has joined #openstack-infra | 07:09 | |
| *** harlowja_away has quit IRC | 07:10 | |
| *** marcoemorais1 has quit IRC | 07:11 | |
| *** resker has joined #openstack-infra | 07:11 | |
| *** greghaynes has joined #openstack-infra | 07:13 | |
| *** _nadya_ has quit IRC | 07:14 | |
| *** esker has quit IRC | 07:14 | |
| *** skolekonov has joined #openstack-infra | 07:18 | |
| *** paul--_ has quit IRC | 07:18 | |
| *** alugovoi has quit IRC | 07:21 | |
| *** _nadya_ has joined #openstack-infra | 07:25 | |
| *** jgallard has joined #openstack-infra | 07:31 | |
| *** yaguang has joined #openstack-infra | 07:36 | |
| *** Hal_ has joined #openstack-infra | 07:39 | |
| *** dkranz has quit IRC | 07:42 | |
| *** Hal_ has quit IRC | 07:44 | |
| *** e0ne has quit IRC | 07:44 | |
| *** arnaud_ has quit IRC | 07:45 | |
| *** dstanek_zzz is now known as dstanek | 07:46 | |
| *** e0ne has joined #openstack-infra | 07:48 | |
| *** pelix has joined #openstack-infra | 07:54 | |
| *** praneshp has quit IRC | 07:55 | |
| *** lcheng_ has quit IRC | 07:55 | |
| *** ominakov has joined #openstack-infra | 07:56 | |
| *** lcheng_ has joined #openstack-infra | 07:56 | |
| *** Hal_ has joined #openstack-infra | 07:59 | |
| *** jpich has joined #openstack-infra | 08:02 | |
| *** ihrachyshka has joined #openstack-infra | 08:03 | |
| *** ihrachyshka has joined #openstack-infra | 08:03 | |
| *** ihrachyshka has quit IRC | 08:04 | |
| *** rdopieralski has joined #openstack-infra | 08:11 | |
| *** lxsli has joined #openstack-infra | 08:11 | |
| *** rdopieralski has quit IRC | 08:12 | |
| *** rdopieralski has joined #openstack-infra | 08:12 | |
| *** dkranz has joined #openstack-infra | 08:13 | |
| *** derekh_ has joined #openstack-infra | 08:14 | |
| *** masayukig has quit IRC | 08:18 | |
| *** hashar has joined #openstack-infra | 08:18 | |
| *** doude has quit IRC | 08:25 | |
| *** NobodyCam has quit IRC | 08:26 | |
| *** NobodyCam has joined #openstack-infra | 08:28 | |
| *** doude has joined #openstack-infra | 08:32 | |
| *** jooools has joined #openstack-infra | 08:33 | |
| *** ihrachyshka has joined #openstack-infra | 08:36 | |
| *** dstanek is now known as dstanek_zzz | 08:38 | |
| *** yaguang has quit IRC | 08:40 | |
| *** lxsli has quit IRC | 08:40 | |
| *** lxsli has joined #openstack-infra | 08:41 | |
| *** jp_at_hp has joined #openstack-infra | 08:41 | |
| *** isviridov is now known as isviridov|away | 08:46 | |
| *** arnaud_ has joined #openstack-infra | 08:47 | |
| *** arnaud_ has quit IRC | 08:51 | |
| *** lcheng_ has quit IRC | 08:54 | |
| *** ominakov_ has joined #openstack-infra | 08:59 | |
| *** ominakov has quit IRC | 09:00 | |
| *** andreaf has joined #openstack-infra | 09:07 | |
| *** dizquierdo has joined #openstack-infra | 09:09 | |
| *** jcoufal has quit IRC | 09:10 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/config: Move back to 8G tripleo nodes https://review.openstack.org/94558 | 09:14 |
| derekh_ | If there is anybody around who can help approve ^^ it would be great, tripleo queue is getting longer and this should help | 09:15 |
| *** ominakov_ has quit IRC | 09:15 | |
| *** andreykurilin_ has joined #openstack-infra | 09:21 | |
| *** rlandy has quit IRC | 09:21 | |
| *** Hal_ has quit IRC | 09:26 | |
| *** ihrachyshka has quit IRC | 09:27 | |
| *** vogxn has joined #openstack-infra | 09:32 | |
| *** vogxn has quit IRC | 09:32 | |
| *** andreykurilin_ has quit IRC | 09:35 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard: Refresh token support https://review.openstack.org/94363 | 09:36 |
| *** vkdrao has joined #openstack-infra | 09:36 | |
| *** lxsli has quit IRC | 09:42 | |
| *** ominakov has joined #openstack-infra | 09:43 | |
| openstackgerrit | Chmouel Boudjnah proposed a change to openstack-infra/config: Add object-specs repository https://review.openstack.org/94140 | 09:44 |
| openstackgerrit | Chmouel Boudjnah proposed a change to openstack-infra/config: Add object-specs repository https://review.openstack.org/94140 | 09:50 |
| *** doude has quit IRC | 09:56 | |
| *** doude has joined #openstack-infra | 09:59 | |
| *** matsuhashi has quit IRC | 10:01 | |
| rcarrillocruz | anyone knows who maintains the tarballs.openstack.org ? i guess there must be a group of people with rights to generate tarballs and upload them there... | 10:01 |
| AJaeger | rcarrillocruz, there are uploaded automatically when certain actions are done | 10:05 |
| AJaeger | That's part of the release process. | 10:05 |
| AJaeger | rcarrillocruz, Which package do you need to update? | 10:05 |
| rcarrillocruz | hmm, what are those actions, could you link me to some documentation or wiki? | 10:05 |
| rcarrillocruz | thing is i was working on https://bugs.launchpad.net/openstack-ci/+bug/1011705 | 10:06 |
| uvirtbot | Launchpad bug 1011705 in openstack-ci "update_bug should use review owner instead of latest commiter" [Low,Confirmed] | 10:06 |
| rcarrillocruz | zaro put a patch on gerrit upstream | 10:06 |
| rcarrillocruz | but the date of the merge is past the latest gerrit war tarball | 10:06 |
| rcarrillocruz | thus the change is not there | 10:06 |
| rcarrillocruz | i'm not sure if i should work on the bug with upstream gerrit | 10:06 |
| rcarrillocruz | or await for openstack gerrit tarball to be updated, since i heard they will open a maintenance window this week to put some stuff in it | 10:07 |
| AJaeger | ah - sorry, can't help with that part, that's a different tar ball AFAIU | 10:07 |
| rcarrillocruz | nm thx, i'll wait for zaro in a couple of hours | 10:08 |
| AJaeger | rcarrillocruz, openstack-infra/config contains the file modules/openstack_project/files/jenkins_job_builder/config/gerrit.yaml - it mentions the gerrit war tarball | 10:09 |
| AJaeger | zuul should tell you when this is triggered | 10:09 |
| AJaeger | Sorry, no time to dig deeper - waiting for zaro might be better ;) | 10:09 |
| rcarrillocruz | y, no probs, thx | 10:09 |
| *** e0ne_ has joined #openstack-infra | 10:10 | |
| *** vkdrao has quit IRC | 10:11 | |
| *** e0ne has quit IRC | 10:13 | |
| *** jcoufal has joined #openstack-infra | 10:15 | |
| *** sahumada has left #openstack-infra | 10:16 | |
| rcarrillocruz | ah | 10:18 |
| *** jgallard has quit IRC | 10:18 | |
| rcarrillocruz | review.pp has this key/value: | 10:18 |
| rcarrillocruz | war => | 10:18 |
| rcarrillocruz | 'http://tarballs.openstack.org/ci/gerrit/gerrit-v2.8.4.15.6dc8444.war', | 10:18 |
| rcarrillocruz | i can just point it to upstream to pull a war that has the patch merged | 10:18 |
| *** _nadya_ has quit IRC | 10:22 | |
| *** ihrachyshka has joined #openstack-infra | 10:25 | |
| *** kmartin has quit IRC | 10:27 | |
| *** che-arne has joined #openstack-infra | 10:31 | |
| *** e0ne_ has quit IRC | 10:37 | |
| *** jcoufal has quit IRC | 10:39 | |
| *** jcoufal has joined #openstack-infra | 10:40 | |
| *** hipster has joined #openstack-infra | 10:41 | |
| *** hipster has quit IRC | 10:49 | |
| *** ArxCruz has joined #openstack-infra | 10:49 | |
| *** vkdrao has joined #openstack-infra | 11:02 | |
| *** e0ne has joined #openstack-infra | 11:07 | |
| *** lttrl has joined #openstack-infra | 11:09 | |
| *** e0ne_ has joined #openstack-infra | 11:09 | |
| *** e0ne__ has joined #openstack-infra | 11:11 | |
| *** e0ne_ has quit IRC | 11:11 | |
| *** e0ne has quit IRC | 11:12 | |
| *** e0ne__ has quit IRC | 11:16 | |
| *** vkdrao has quit IRC | 11:16 | |
| *** nosnos has quit IRC | 11:16 | |
| *** pcm_ has joined #openstack-infra | 11:26 | |
| *** mwagner_lap has quit IRC | 11:33 | |
| *** jgallard has joined #openstack-infra | 11:37 | |
| *** mbacchi has joined #openstack-infra | 11:41 | |
| *** lxsli has joined #openstack-infra | 11:43 | |
| *** jgallard has quit IRC | 11:46 | |
| *** salv-orlando has joined #openstack-infra | 11:48 | |
| *** lttrl has quit IRC | 11:55 | |
| *** lttrl has joined #openstack-infra | 11:55 | |
| *** lxsli has quit IRC | 12:01 | |
| *** lxsli has joined #openstack-infra | 12:04 | |
| *** rlandy has joined #openstack-infra | 12:06 | |
| *** skolekonov has quit IRC | 12:08 | |
| *** weshay has joined #openstack-infra | 12:08 | |
| *** salv-orlando has quit IRC | 12:12 | |
| *** pdmars has joined #openstack-infra | 12:12 | |
| *** pblaho has quit IRC | 12:15 | |
| *** pdmars has quit IRC | 12:17 | |
| *** unicell1 has joined #openstack-infra | 12:18 | |
| *** loetterle has joined #openstack-infra | 12:20 | |
| *** unicell has quit IRC | 12:20 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/zuul: Implement Depends-On: IdempotentID https://review.openstack.org/94365 | 12:22 |
| *** aysyd has joined #openstack-infra | 12:23 | |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/config: Add missing export -f post_test_hook in rally-cli https://review.openstack.org/94601 | 12:24 |
| *** salv-orlando has joined #openstack-infra | 12:25 | |
| *** smarcet has joined #openstack-infra | 12:29 | |
| *** sandywalsh_ has joined #openstack-infra | 12:29 | |
| *** pdmars has joined #openstack-infra | 12:32 | |
| *** pdmars has quit IRC | 12:33 | |
| *** pdmars has joined #openstack-infra | 12:33 | |
| *** pdmars has quit IRC | 12:34 | |
| *** pcm_ has quit IRC | 12:34 | |
| *** rfolco has joined #openstack-infra | 12:35 | |
| *** flaper87 is now known as flaper87|afk | 12:38 | |
| fungi | clarkb: https://etherpad.openstack.org/p/disabling-gerrit-drafts lgtm, though should we cram a wip option back into git-review and tag a new release between now and then? | 12:42 |
| *** Hal_ has joined #openstack-infra | 12:43 | |
| *** pdmars has joined #openstack-infra | 12:43 | |
| fungi | jeblair: clarkb: while i'm thinking about it, do we want to restart zuul during the project renames on friday so that we can put https://review.openstack.org/92683 back into effect? | 12:44 |
| sandywalsh_ | hey guys ... question about maintaining master and stable branches. | 12:45 |
| sandywalsh_ | should changes to stable go in as separate reviews just as master does, or is it ok to squash merge and do it in a big lump? | 12:46 |
| sandywalsh_ | I'm guessing best practice is to cherry pick and keep them separate | 12:46 |
| fungi | sandywalsh_: convention has been to cherry-pick master changes onto stable branches one-by-one, yes, though you might want to bring that question to the stable branch maintainers on the openstack-stable-maint@lists.openstack.org mailing list | 12:47 |
| fungi | they probably can point you to a much more detailed set of recommendations | 12:48 |
| sandywalsh_ | heh, yet another list I knew nothing about :) ... thank fungi | 12:48 |
| *** doug-fish has joined #openstack-infra | 12:48 | |
| fungi | http://lists.openstack.org/cgi-bin/mailman/listinfo ;) | 12:48 |
| *** HenryG has joined #openstack-infra | 12:49 | |
| *** mwagner_lap has joined #openstack-infra | 12:50 | |
| *** afazekas has joined #openstack-infra | 12:53 | |
| *** salv-orlando has quit IRC | 12:54 | |
| *** eharney has joined #openstack-infra | 12:54 | |
| *** doug-fish has quit IRC | 12:55 | |
| *** doug-fish has joined #openstack-infra | 12:56 | |
| *** aveiga has joined #openstack-infra | 12:56 | |
| *** dprince has joined #openstack-infra | 12:56 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Use pbr, pep8 checks & fixes, add "doc" tox target https://review.openstack.org/90455 | 12:57 |
| *** BobBallAway has quit IRC | 12:59 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/config: Remove old non-gz files in log archive maintenance https://review.openstack.org/94610 | 12:59 |
| *** resker has quit IRC | 13:01 | |
| *** BobBallAway has joined #openstack-infra | 13:01 | |
| *** esker has joined #openstack-infra | 13:02 | |
| *** mfer has joined #openstack-infra | 13:02 | |
| *** mriedem has joined #openstack-infra | 13:04 | |
| *** wenlock has joined #openstack-infra | 13:10 | |
| *** thomasbiege has joined #openstack-infra | 13:12 | |
| *** apevec has joined #openstack-infra | 13:14 | |
| anteaya | morning | 13:15 |
| anteaya | I'm away this afternoon | 13:15 |
| *** wenlock has quit IRC | 13:15 | |
| *** thomasbiege1 has quit IRC | 13:15 | |
| anteaya | I have two goals for today, get started learning the wiki meta page stuff - fungi - and kick off the conversation to decide when and where we are having the infra thing - reed | 13:15 |
| anteaya | after we decide when and where then maybe we can talk about what - jeblair | 13:16 |
| anteaya | though what may happen simultaneously | 13:16 |
| fungi | anteaya: some leisure reading... http://www.mediawiki.org/wiki/Help:Templates | 13:17 |
| fungi | anteaya: {{subst:Name}} is probably what we want for this purpose | 13:18 |
| fungi | anteaya: that will basically fill in the initial page with what was in the template, and then allow the author to edit the new page to "fill in the blanks" as it were | 13:18 |
| anteaya | awesome thank you | 13:20 |
| * anteaya eats breakfast and reads | 13:20 | |
| *** _nadya_ has joined #openstack-infra | 13:22 | |
| *** _nadya_ has quit IRC | 13:26 | |
| *** julim has joined #openstack-infra | 13:27 | |
| *** bknudson has joined #openstack-infra | 13:31 | |
| openstackgerrit | ayoung proposed a change to openstack/requirements: request-kerberos Kerberos requires an additional requests library. Older versions break in py33 https://review.openstack.org/84740 | 13:31 |
| *** zz_gondoi is now known as gondoi | 13:34 | |
| *** dhellmann_ is now known as dhellmann | 13:35 | |
| *** andreykurilin_ has joined #openstack-infra | 13:36 | |
| *** che-arne_afk has joined #openstack-infra | 13:37 | |
| *** homeless has joined #openstack-infra | 13:38 | |
| *** che-arne has quit IRC | 13:38 | |
| *** andreykurilin_ has quit IRC | 13:41 | |
| *** loetterle has quit IRC | 13:41 | |
| *** lttrl has quit IRC | 13:41 | |
| *** kashyap has quit IRC | 13:45 | |
| *** wenlock has joined #openstack-infra | 13:46 | |
| jeblair | anteaya: i think the main things to consider about that are: a) the qa team's meeting; b) oscon; c) our dev cycle is shifted (our quiet periods are openstack's busy periods) | 13:47 |
| *** signed8bit has joined #openstack-infra | 13:47 | |
| *** mihgen has joined #openstack-infra | 13:48 | |
| * anteaya nods | 13:48 | |
| *** hipster has joined #openstack-infra | 13:49 | |
| anteaya | mtreinish sdague do we have dates yet for the qa team's meeting? | 13:53 |
| *** Kiall has joined #openstack-infra | 13:53 | |
| anteaya | oscon is July 20-24 with a day or two of buffer at either end: http://www.oscon.com/oscon2014/public/content/home | 13:53 |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: Make Timeline configurable https://review.openstack.org/93433 | 13:53 |
| *** markwash has joined #openstack-infra | 13:54 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: Configure Timeline from Story details https://review.openstack.org/93563 | 13:54 |
| anteaya | and pyconau is aug 1-5 and I hope to be speaking at that, at least in the miniconf track: http://2014.pycon-au.org/ | 13:55 |
| *** _nadya_ has joined #openstack-infra | 13:56 | |
| *** salv-orlando has joined #openstack-infra | 13:56 | |
| mtreinish | anteaya: not yet, I'm driving back home this afternoon so I'll iron out everything tomorrow or friday | 13:56 |
| anteaya | mtreinish: great thanks | 13:57 |
| anteaya | you still in atlanta? | 13:57 |
| mtreinish | I'm actually in Baltimore right now :) | 13:57 |
| *** jgrimm has joined #openstack-infra | 13:57 | |
| fungi | any time i visit baltimore i get an overwhelming urge to watch john waters movies | 13:58 |
| *** prad_ has joined #openstack-infra | 13:59 | |
| *** otherwiseguy has joined #openstack-infra | 13:59 | |
| anteaya | I have never seen a john waters movie | 14:01 |
| anteaya | what a sheltered life I have lived | 14:01 |
| anteaya | you might call it a dirty shame | 14:01 |
| anteaya | fungi: so I found a sandbox: http://www.mediawiki.org/w/index.php?title=Project:Sandbox&oldid=456598 | 14:01 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add a tool for determining unaccounted for rechecks https://review.openstack.org/94624 | 14:02 |
| anteaya | if you feel so inclined to post a quick sample, I would be grateful | 14:02 |
| anteaya | at this point the docs seem to be speaking in circles, so I am looking for examples | 14:02 |
| *** terryw has joined #openstack-infra | 14:03 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: Configure Timeline from Story details https://review.openstack.org/93563 | 14:04 |
| sdague | jeblair: so there is more data in the old rechecks than I imagined | 14:05 |
| sdague | Summary | 14:05 |
| sdague | 2614 - Total Rechecks | 14:05 |
| sdague | 1233 - Total w/Bug | 14:05 |
| sdague | 574 - Total w/Bug and new recheck | 14:05 |
| sdague | that's 2 weeks | 14:05 |
| sdague | or, more importantly, reviews that have been active in the last 2 weeks | 14:05 |
| *** otherwiseguy has quit IRC | 14:06 | |
| *** wenlock has quit IRC | 14:06 | |
| sdague | actually... I should purge that further | 14:06 |
| sdague | as there might be old valid bugs that fell out of er | 14:06 |
| phschwartz | Morning | 14:09 |
| jeblair | sdague: also, the past 2 weeks haven't been our most active | 14:09 |
| *** Ryan_Lane has joined #openstack-infra | 14:10 | |
| anteaya | morning phschwartz | 14:10 |
| anteaya | hey Ryan_Lane | 14:10 |
| Ryan_Lane | howdy | 14:10 |
| Ryan_Lane | not here long :) | 14:10 |
| anteaya | kk | 14:10 |
| anteaya | fungi is teaching me wiki templates | 14:10 |
| anteaya | I might have some silly questions for you soon | 14:10 |
| anteaya | also, any movement on the bug to increase sso timeouts for our wiki? | 14:11 |
| anteaya | I seem to be piling on | 14:11 |
| *** Ryan_Lane has quit IRC | 14:12 | |
| anteaya | I have figured out how to call a template, but haven't yet figured out how to create the source template | 14:12 |
| *** pcrews_ has joined #openstack-infra | 14:12 | |
| *** zehicle_at_dell has joined #openstack-infra | 14:13 | |
| openstackgerrit | Nikita Konovalov proposed a change to openstack-infra/storyboard-webclient: Configure Timeline from Story details https://review.openstack.org/93563 | 14:14 |
| *** thedodd has joined #openstack-infra | 14:14 | |
| anteaya | seems I have to create a new wikipage with the name of the template as the title, hmmmm | 14:15 |
| *** rwsu has joined #openstack-infra | 14:15 | |
| *** changbl has joined #openstack-infra | 14:17 | |
| *** lcheng_ has joined #openstack-infra | 14:18 | |
| sdague | jeblair: fair, though that's all we have in ES | 14:19 |
| *** sandywalsh_ has quit IRC | 14:19 | |
| sdague | so we need to compare against that | 14:19 |
| jeblair | yeah | 14:19 |
| sdague | I actually bet these numbers will drop a lot once I filter by comment timestamp | 14:20 |
| *** _nadya_ has quit IRC | 14:21 | |
| *** msabramo has quit IRC | 14:21 | |
| *** mrodden has quit IRC | 14:21 | |
| sdague | we have these funny facts like some surprisingly large percentage of all time reviews have been active in the last 2 weeks based on how long patches take to land | 14:21 |
| *** _nadya_ has joined #openstack-infra | 14:22 | |
| *** lcheng_ has quit IRC | 14:23 | |
| sdague | jeblair: https://dague.net/testing/rechecks.txt | 14:23 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add a tool for determining unaccounted for rechecks https://review.openstack.org/94624 | 14:25 |
| jeblair | sdague: so that's about 30 bugs that er doesn't know about? | 14:25 |
| sdague | yeh | 14:26 |
| sdague | we are currently tracking 100 ish | 14:26 |
| sdague | which means I do think it's worth eventually mining this data | 14:27 |
| sdague | but I'll say it's lower priority | 14:27 |
| jeblair | er knows about 69 unclassified failures | 14:27 |
| *** BobBallAway is now known as BobBall | 14:27 | |
| anteaya | I'm brain dumping on this etherpad regarding the third party wiki templates: https://etherpad.openstack.org/p/third-party-wiki-templating | 14:27 |
| *** BobBall is now known as Guest74851 | 14:27 | |
| sdague | jeblair: 69 in gate | 14:28 |
| sdague | so you can't quite equate the 2 | 14:28 |
| *** lcheng_ has joined #openstack-infra | 14:29 | |
| jeblair | sdague: huh, i just spot checked the first one on the list: https://review.openstack.org/#/c/94506/ | 14:29 |
| jeblair | sdague: it has an er message | 14:29 |
| *** gokrokve has joined #openstack-infra | 14:29 | |
| sdague | jeblair: it does | 14:29 |
| *** medieval1 has quit IRC | 14:29 | |
| sdague | that wasn't the bug that was used to recheck | 14:29 |
| sdague | chmouel did a recheck bug 1155800 | 14:30 |
| uvirtbot | Launchpad bug 1155800 in nova "Cannot delete / confirm / revert resize an instance if nova-compute crashes after VERIFY_RESIZE" [High,Triaged] https://launchpad.net/bugs/1155800 | 14:30 |
| jeblair | sdague: oh there's two | 14:30 |
| sdague | 1155800 is definitely not in our signature list | 14:30 |
| jeblair | yeah, got it now | 14:31 |
| *** Guest74851 is now known as Bobba | 14:31 | |
| *** salv-orlando_ has joined #openstack-infra | 14:31 | |
| *** kgriffs|afk is now known as kgriffs | 14:32 | |
| *** salv-orlando has quit IRC | 14:32 | |
| *** salv-orlando_ is now known as salv-orlando | 14:32 | |
| *** msabramo has joined #openstack-infra | 14:32 | |
| sdague | my spot checks so far on the list seem to indicate that it's accurate, the code to gen it is in https://review.openstack.org/94624 if you think I'm missing something | 14:33 |
| jeblair | sdague: so why isn't that on the e-r unclassified list? | 14:33 |
| sdague | the review? | 14:33 |
| sdague | because it failed in check | 14:34 |
| sdague | not in gate | 14:34 |
| jeblair | it got a verified -2 | 14:34 |
| *** mrodden has joined #openstack-infra | 14:34 | |
| *** jgallard has joined #openstack-infra | 14:34 | |
| jeblair | this job http://logs.openstack.org/06/94506/2/gate/gate-grenade-dsvm/1b5348a at ... tz math ... 3:50AM eastern | 14:34 |
| sdague | where? | 14:35 |
| jeblair | https://review.openstack.org/#/c/94506/ | 14:35 |
| jeblair | sdague: it's the recheck bug 1155800 we've been discussing | 14:35 |
| uvirtbot | Launchpad bug 1155800 in nova "Cannot delete / confirm / revert resize an instance if nova-compute crashes after VERIFY_RESIZE" [High,Triaged] https://launchpad.net/bugs/1155800 | 14:35 |
| *** jpich has quit IRC | 14:35 | |
| sdague | jeblair: right, I don't see a Verified-2 anywhere in https://review.openstack.org/#/c/91788/ | 14:35 |
| sdague | oh, sorry, drifted | 14:36 |
| sdague | let me look | 14:36 |
| *** david-lyle has joined #openstack-infra | 14:37 | |
| sdague | that's a good question, I wonder if because a signature actually matches on the most recent run | 14:37 |
| derekh_ | hi, I'm trying to get the tripleo nodepool instances back to 8G, so I've create a new flavor on both clouds and submitted a patch to change config https://review.openstack.org/#/c/94558/ if anybody has a chance would be great to get sorted before the queue grows even longer | 14:37 |
| jeblair | derekh_: lgtm | 14:38 |
| jeblair | derekh_: and thanks! :) | 14:38 |
| fungi | jeblair: is that going to be sufficient to jiggle nodepool into picking the new flavor on the next image update? | 14:39 |
| fungi | or will it have already decided on the old flavor? | 14:39 |
| anteaya | git review is currently frozen for me | 14:39 |
| anteaya | no error yet | 14:39 |
| anteaya | still hung | 14:40 |
| jeblair | fungi: no, it will probably completely fail and we'll have to build new images. i'm assuming things are so bad that doesn't particularly bother us. :) | 14:40 |
| *** markmcclain has joined #openstack-infra | 14:40 | |
| fungi | k | 14:40 |
| *** markmcclain has quit IRC | 14:40 | |
| jeblair | fungi: i believe it will try to ask nova to boot 8g instances from images built from the 16g flavor and nova will refuse. | 14:40 |
| *** markmcclain has joined #openstack-infra | 14:40 | |
| anteaya | I did control-c and got http://paste.openstack.org/show/81065/ | 14:41 |
| *** medieval1 has joined #openstack-infra | 14:41 | |
| chmouel | have i done something bad with that recheck bug 1155800? | 14:41 |
| uvirtbot | Launchpad bug 1155800 in nova "Cannot delete / confirm / revert resize an instance if nova-compute crashes after VERIFY_RESIZE" [High,Triaged] https://launchpad.net/bugs/1155800 | 14:41 |
| *** wenlock has joined #openstack-infra | 14:41 | |
| jeblair | fungi: the "gentle" way to do this involves lots of patchsets and many hours, i think. probably best just to merge this and nodepool image-delete. | 14:41 |
| *** james_li has joined #openstack-infra | 14:42 | |
| jeblair | chmouel: no, we're just examining the recheck scripts and er-data and happened to pick that review to spot check some things | 14:42 |
| derekh_ | fungi: jeblair: R1 (our bigger cloud) doesn't seem to be getting used at all, has it been disabled? | 14:42 |
| derekh_ | fungi: jeblair I'm ok with loss of running jobs as is we can't keep up with them the way it is | 14:42 |
| jeblair | derekh_: i don't think we'll lose running jobs actually | 14:43 |
| fungi | derekh_: i see 4 nodes in a delete state and 6 in a used state in tripleo-test-cloud-rh1 | 14:43 |
| derekh_ | jeblair: ok great | 14:43 |
| derekh_ | fungi: hmm, when I check earlier we had 9 Active, looking | 14:44 |
| jeblair | anteaya: so that's the 'check if there's a newer version on pypi' code; presumably pypi is/was having problems, or you are/were having problems connecting to it | 14:44 |
| fungi | derekh_: these are nodepool states, not nova states. nodepool believes there are 10 nodes in that region | 14:44 |
| anteaya | k | 14:44 |
| jeblair | anteaya: check this url: https://pypi.python.org/pypi/git-review/ | 14:44 |
| anteaya | going to see what happens if I ssh -p | 14:44 |
| *** lcostantino has joined #openstack-infra | 14:45 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/config: The process for making a Third-Party CI voting needs docs https://review.openstack.org/93562 | 14:45 |
| *** jgallard has quit IRC | 14:45 | |
| derekh_ | fungi: ahh ok, I still see 9 active in nova and 1 error | 14:45 |
| *** doude has quit IRC | 14:45 | |
| fungi | sounds about right then | 14:46 |
| *** lcheng_ has quit IRC | 14:46 | |
| *** yamahata has joined #openstack-infra | 14:46 | |
| *** xchu has quit IRC | 14:46 | |
| anteaya | I have git-review 1.23 | 14:46 |
| anteaya | huh it worked the second time, hmmm | 14:47 |
| sdague | chmouel: nope, we're just trying to grok the recheck info to figure out if we need to incorporate comments for bugs not in ER | 14:47 |
| phschwartz | anteaya, fungi: I almost am ready to push out a new patchset to the one for vinz adding the things missed for the backend that will be needed. Just wanted to give you guys the heads up. | 14:47 |
| *** jpich has joined #openstack-infra | 14:48 | |
| anteaya | phschwartz: k, heads up not necessary, the bot will tell us | 14:48 |
| anteaya | thanks for thinking of us though | 14:48 |
| mordred | derekh_: R1 is not the redhat cloud, right? it's the HP cloud? | 14:48 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add a tool for determining unaccounted for rechecks https://review.openstack.org/94624 | 14:49 |
| mordred | derekh_, fungi: I also see no nodes in that cloud in the graphite graphs | 14:49 |
| derekh_ | monester: correct | 14:49 |
| jeblair | phschwartz: so if vinz ever becomes a full gerrit replacement, are you okay with morphing this backend project into that? or would you want to start a new project? | 14:49 |
| *** arnaud has joined #openstack-infra | 14:49 | |
| mordred | just making sure I was following | 14:49 |
| phschwartz | anteaya: I haven't noticed the bot giving notice of a new patchset on an existing review, I might be wrong. | 14:49 |
| sdague | jeblair: so other interesting fact, turns out that check-grenade-dsvm-icehouse is upgrading from icehouse -> icehouse | 14:49 |
| derekh_ | mordred: correct | 14:49 |
| derekh_ | monester: sorry | 14:49 |
| phschwartz | jeblair: That would be the exact intension. Morphing the project into exactly that. | 14:49 |
| anteaya | see timestamp 14:45:36 utc | 14:50 |
| jeblair | phschwartz: ok cool. mostly wanted to make sure we got the name right. :) | 14:50 |
| fungi | derekh_: oh, i had no idea you'd made up new terminology for region names not reflected in nodepool. we know them as tripleo-test-cloud and tripleo-test-cloud-rh1 | 14:50 |
| anteaya | that was me submitting a new patchset | 14:50 |
| phschwartz | jeblair: I was going to mirror storyboard and have vinz and vinz-webclient | 14:50 |
| fungi | mordred: i assumed r1 was short for rh1. my bad | 14:51 |
| jeblair | phschwartz: yep. sounds good. just wanted to make sure we didn't need to call this one 'vinz-temporary-backend-connector-thingy' or something. sounds like we don't and we're all good. :) | 14:51 |
| mordred | fungi: I did too - until I looked at my little graphs and they said the opposite, when I re-thought the input data | 14:51 |
| mordred | jeblair, phschwartz: so, I had a TERRIBLE idea last night while driving | 14:52 |
| *** andreaf has quit IRC | 14:52 | |
| phschwartz | mordred: you will be happy to note that I was messing with the database for a local install of gerrit and learned something interesting. There is a gerrit option to turn of the database caching in the jvm. Once that was off if I hand edited the database it showed in the gerrit ui the moment it was made. | 14:52 |
| * anteaya gets that if pypi is temporarily unavailable git-review may hang | 14:52 | |
| phschwartz | mordred: hit me with it. | 14:52 |
| mordred | jeblair, phschwartz which is that we could call the frontend vinz and the backend clortho | 14:52 |
| fungi | derekh_: in that case, i see 2 building and 38 delete in tripleo-test-cloud | 14:52 |
| phschwartz | LOLOLOL | 14:52 |
| derekh_ | fungi: sorry, we should change then in nodepool configs (I'll run that by lifeless), I knida started the new naming scheme to dissasosiate company names from blame (since its not really related) | 14:52 |
| fungi | derekh_: so possibly you changed something in that region to cause nodepool to no longer be able to launch images there | 14:52 |
| jeblair | oO | 14:52 |
| sdague | jeblair: https://github.com/openstack-infra/devstack-gate/blob/master/devstack-vm-gate-wrap.sh#L209-L215 should that be OVERRIDE_ZUUL_BRANCH ? | 14:53 |
| mordred | I should really not be allowed to drive for 3 hours at a time at night with a wife who is asleep | 14:53 |
| *** doude has joined #openstack-infra | 14:53 | |
| fungi | derekh_: er, to launch instances there | 14:53 |
| sdague | instead of ZUUL_BRANCH | 14:53 |
| derekh_ | fungi: I didn't change anything but maybe SpamapS did, he commented they other day when I asked about "turning it back on" | 14:53 |
| jeblair | sdague: HARD QUESTIONS BEFORE BREAKFAST! | 14:53 |
| anteaya | careful we might have Bill Murray show up and start serving tequilia | 14:53 |
| sdague | jeblair: heh | 14:53 |
| *** ihrachyshka has quit IRC | 14:53 | |
| fungi | derekh_: by "you" i meant "tripleo people" ;) | 14:54 |
| derekh_ | fungi: ok | 14:54 |
| mordred | anteaya: that's ok - I kinda want to rename nodepool to egon | 14:54 |
| phschwartz | mordred: +1 +1 +1, hell +20000 | 14:55 |
| jeblair | mordred: it never occurred to me to name things after the _good_ guys. | 14:55 |
| phschwartz | You know, nodepool could always be call the Containment Unit. ;) | 14:56 |
| derekh_ | fungi: ok, so according to nova R1 (tripleo-test-cloud) only has 5 nodepool instances running (you seeing 2 + 38 in nodepool would explain why its no longer being used) | 14:56 |
| openstackgerrit | Ben Nemec proposed a change to openstack-infra/reviewstats: Properly support WIP in Gerrit 2.8 https://review.openstack.org/94640 | 14:56 |
| fungi | derekh_: yep, seems like we probably have a leak where nodepool is trying (and failing) to delete instances which no longer exist in the provider for some reason | 14:57 |
| *** mburned is now known as mburned_out | 14:58 | |
| jeblair | phschwartz, mordred: there has to be something we can do with 'walter peck'. | 14:58 |
| anteaya | mordred: you are set on invoking the great one, aren't you | 14:58 |
| fungi | derekh_: keep in mind that nodepool doesn't periodically inventory the provider instance lists. it just keeps an internal list of instances it thinks still exist | 14:58 |
| anteaya | do you know I was in a movie with Bill Murray? | 14:58 |
| fungi | derekh_: and keeps trying to delete them | 14:58 |
| anteaya | uncredited | 14:58 |
| jeblair | fungi: nodepool _should_ figure that out if they aren't showing up in the instance list | 14:58 |
| jeblair | anteaya: which movie? | 14:59 |
| phschwartz | jeblair: he was a politician, could always rename the meetbot | 14:59 |
| anteaya | Meatballs | 14:59 |
| anteaya | I was 10 | 14:59 |
| openstackgerrit | Randy Coulman proposed a change to openstack-infra/jenkins-job-builder: Change default Reply-To header to $DEFAULT_REPLYTO. https://review.openstack.org/79331 | 14:59 |
| *** isviridov|away is now known as isviridov | 14:59 | |
| jeblair | sdague: btw, i really am working on your question. ;) | 14:59 |
| anteaya | http://en.wikipedia.org/wiki/Meatballs_%28film%29 | 14:59 |
| fungi | jeblair: what i've seen in the past with the tripleo nodes is that when you try to delete or query a nonexistent instance, the query times out | 14:59 |
| fungi | not sure if it's still doing that though | 14:59 |
| anteaya | filmed at Camp White Pine: http://en.wikipedia.org/wiki/Camp_White_Pine | 15:00 |
| rcarrillocruz | heh, nice | 15:00 |
| jeblair | anteaya: nice. also ivan reitman, and harold ramis. :) | 15:00 |
| derekh_ | fungi: got ya, if nobody is working on it I'd be willing to add a inventory check to nodepool (probably next week sometime)# | 15:00 |
| anteaya | never saw them | 15:00 |
| anteaya | they needed kids for the crowd | 15:00 |
| phschwartz | jeblair: or we could always write a bot in his name that watches for things that break the gate and have it spam the persons about it :) | 15:00 |
| anteaya | hardware was entertaining, and wheels | 15:00 |
| anteaya | he kept doing backfilps for us | 15:01 |
| *** Longgeek has quit IRC | 15:01 | |
| anteaya | I was in the basketball scene | 15:01 |
| jeblair | phschwartz: that would be elastic-recheck. which rhymes with walter peck. | 15:01 |
| *** Longgeek has joined #openstack-infra | 15:01 | |
| phschwartz | jeblair: so did I hear that we should rename elastic-recheck. | 15:01 |
| fungi | anteaya: that was one of my favorite films when i was younger | 15:01 |
| anteaya | :D | 15:01 |
| anteaya | it was a fun experience | 15:02 |
| jeblair | phschwartz: :) | 15:02 |
| anteaya | when I got older, I ended up being a camera trainee with the guy who was second camera assist on meatballs | 15:02 |
| sdague | jeblair: no worries :) | 15:02 |
| anteaya | ernie kestler | 15:02 |
| anteaya | anyway, enough of that | 15:02 |
| *** gokrokve has quit IRC | 15:04 | |
| *** pcm_ has joined #openstack-infra | 15:04 | |
| fungi | derekh_: looks like the cruft delete nodes have been stuck in that state for between 91 and 427 hours, so this seems to be something which has probably crept up over the course of a couple weeks | 15:04 |
| *** gokrokve has joined #openstack-infra | 15:04 | |
| sdague | jeblair: I'm going to disappear for a bit shortly for mid day ride and lunch. We can circle around that later. | 15:04 |
| *** gokrokve has quit IRC | 15:04 | |
| sdague | I'd also ask for some help on the zuul front with https://review.openstack.org/#/c/94365/ later today, because I feel like I've changed all the datastructures around dependencies in a way that should default to a noop, but still 2 tests are breaking | 15:04 |
| *** zehicle_at_dell has quit IRC | 15:05 | |
| sdague | and I got lost tracing them this morning | 15:05 |
| jeblair | sdague: ok, i still have like 100 changes to review though :/ | 15:05 |
| *** zehicle_at_dell has joined #openstack-infra | 15:05 | |
| fungi | derekh_: i tried forcing one to delete out of band with the nodepool client and it worked, so it appears nodepool is perhaps not retrying to delete these | 15:05 |
| phschwartz | anteaya: I have meatballs on dvd, I need to watch it and see if I can guess where you are in it :) | 15:07 |
| sdague | jeblair: ok, well I'm somewhat blocked on the zuul thing at this point without some expert advice. If you can get to it today, that means I'll not have lost context in diving into something else. If not, we'll have to pick it up again later. | 15:07 |
| *** changbl has quit IRC | 15:07 | |
| anteaya | phschwartz: I go by fast, stick to the basketball game if you want a fighting chance of spoting me | 15:08 |
| openstackgerrit | Brad P. Crochet proposed a change to openstack-infra/nodepool: Add cloud-user to non-root user list https://review.openstack.org/94643 | 15:08 |
| anteaya | spotting | 15:08 |
| *** moted has joined #openstack-infra | 15:09 | |
| derekh_ | fungi: ok, makes sense, watching to see if it starts an instance | 15:09 |
| mordred | jeblair: look! it's another non-root root user! | 15:09 |
| jeblair | mordred: now the world is three times as secure! | 15:10 |
| *** gondoi is now known as zz_gondoi | 15:10 | |
| *** moted has quit IRC | 15:10 | |
| fungi | taking one of the currently stuck delete nodes as an example, the last thing in see for it in the debug log is: | 15:10 |
| fungi | 2014-05-17 19:17:45,641 DEBUG nodepool.NodePool: Deleting server 067a84b9-f2e2-48b8-baee-4a499a64afca for node id: 4008105 | 15:10 |
| jeblair | sdague: i believe it should be override_zuul_branch if it exists, otherwise, fall back to zuul_branch. | 15:10 |
| *** pblaho has joined #openstack-infra | 15:11 | |
| fungi | taking a thread dump of nodepoold now to see if it's still spinning its wheels on that | 15:11 |
| jeblair | fungi: send it sigusr2 and... yeah. :) | 15:11 |
| *** zz_gondoi is now known as gondoi | 15:11 | |
| fungi | ;) | 15:11 |
| jeblair | fungi: i think 4008105 should be in the thread name | 15:11 |
| *** alugovoi has joined #openstack-infra | 15:11 | |
| *** moted has joined #openstack-infra | 15:12 | |
| *** gokrokve has joined #openstack-infra | 15:12 | |
| *** jgallard has joined #openstack-infra | 15:13 | |
| *** virmitio has joined #openstack-infra | 15:13 | |
| fungi | yup... Thread: NodeDeleter for 4008105 (139918793750272) | 15:14 |
| fungi | File "/usr/lib/python2.7/threading.py", line 243, in wait waiter.acquire() | 15:14 |
| fungi | that's spinning down inside listServers | 15:15 |
| *** kmartin has joined #openstack-infra | 15:16 | |
| *** dstanek_zzz is now known as dstanek | 15:16 | |
| *** ominakov has quit IRC | 15:16 | |
| fungi | so submitTask(ListServersTask()) isn't returning, apparently | 15:16 |
| phschwartz | The gates do testing of cells right? | 15:16 |
| *** ominakov has joined #openstack-infra | 15:17 | |
| fungi | phschwartz: i think it was finally turned off because of being completely broken... double-checking that assertion | 15:17 |
| fungi | phschwartz: yeah, seems we do run the jobs which test it, but they're entirely non-voting | 15:18 |
| phschwartz | fungi: hmm, I was under the impression that the broken-ness was fixed | 15:18 |
| phschwartz | fungi: So they run, just non-voting. I hope in the future that will change. But knowing they run is good. That means they work good in devstack and I can use that internal here as we use cells. | 15:19 |
| *** malini_afk is now known as malini | 15:19 | |
| *** alugovoi has quit IRC | 15:19 | |
| fungi | phschwartz: well, "run" just means we try to run the job. no bets on how well it's still working in devstack | 15:20 |
| phschwartz | fungi: I know who fixed them and he assured me it works so I will just have to bug him if it doesn't :) | 15:20 |
| *** changbl has joined #openstack-infra | 15:21 | |
| openstackgerrit | ayoung proposed a change to openstack/requirements: request-kerberos https://review.openstack.org/84740 | 15:21 |
| jeblair | fungi: Thread: tripleo-test-cloud (139922384090880) | 15:22 |
| jeblair | return self._sslobj.read(len) | 15:22 |
| sdague | jeblair: yeh, something still doesn't make sense to me on this devstack-gate thing... I'll sort it out once I get back | 15:22 |
| fungi | phschwartz: looks like it's been non-voting since being added in july of last year... https://review.openstack.org/35544 | 15:22 |
| *** reed has joined #openstack-infra | 15:25 | |
| *** che-arne_afk is now known as che-arne | 15:25 | |
| *** rockyg has joined #openstack-infra | 15:25 | |
| openstackgerrit | Marc Abramowitz proposed a change to stackforge/python-jenkins: Add support for measuring coverage https://review.openstack.org/94649 | 15:25 |
| jeblair | fungi: yeah, the tripleo-test-cloud manager has run nothing today, i think it's stuck. probably another dead connection | 15:25 |
| fungi | spinning in client.servers.list() | 15:26 |
| fungi | i see that. hrm... | 15:26 |
| jeblair | yup | 15:26 |
| jeblair | you know what? maybe instead of http for humans, we should use http for computers? that way they can detect error conditions and handle them. | 15:27 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: fix grenade branch selection https://review.openstack.org/94650 | 15:27 |
| fungi | derekh_: any idea if your api endpoint went offline there at some point in the past week or two? we've definitely seen nodepool not respond well to tcp sockets suddenly going dead with no tcp/rst or anything from the other end (i'd bet you have a very aggressive stateful firewall which drops packets rather than rejecting them) | 15:27 |
| fungi | i can try to kill the established socket on the nodepool end | 15:29 |
| fungi | looks like we have an open 138.35.77.16:13774 but no 138.35.77.16:13000 | 15:30 |
| jeblair | sdague: what's your zuul question? | 15:31 |
| jeblair | sdague: python -m testtools.run tests.test_scheduler.TestScheduler.test_check_queue_success | 15:34 |
| jeblair | sdague: AttributeError: 'IndependentPipelineManager' object has no attribute 'getFailingDependentItems' | 15:34 |
| *** sandywalsh_ has joined #openstack-infra | 15:34 | |
| derekh_ | fungi: I got no idea of api went offline but it is possible random packets didn't reach destination (as we have seen this in CI jobs intermittently) | 15:34 |
| *** mburned_out is now known as mburned | 15:34 | |
| Alex_Gaynor | If/when anyone has a second, I don't understand why https://review.openstack.org/#/c/93794/ provokes failures in the requirements gate at all. | 15:34 |
| jeblair | sdague: seems straightforward | 15:34 |
| *** Sukhdev has joined #openstack-infra | 15:38 | |
| *** krtaylor has quit IRC | 15:38 | |
| *** mjseger has joined #openstack-infra | 15:39 | |
| *** hashar has quit IRC | 15:40 | |
| mjseger | I was wondering if someone could tell me what I'm doing wrong. I tried to set up a new environment and git review tells me it can't find gerrit. I thought I did everything correctly but am clearly missing something. | 15:41 |
| *** msabramo has quit IRC | 15:41 | |
| fungi | jeblair: derekh_ after killing the old established 138.35.77.16:13774 socket with killcx, i see new connections to 138.35.77.16:13000 and 138.35.77.16:13774 | 15:42 |
| jeblair | Alex_Gaynor: there was a very early failure in that job | 15:42 |
| jeblair | Alex_Gaynor: 2014-05-15 20:14:47.047 | ConnectionError: HTTPConnectionPool(host='pypi.openstack.org', port=80): Max retries exceeded with url: /openstack/cliff/cliff-1.4.4.tar.gz (Caused by <class 'httplib.BadStatusLine'>: '') | 15:42 |
| Alex_Gaynor | jeblair: ahhhhhh, I totally missed that, I only saw the final explosion; just a recheck then you think? | 15:42 |
| jeblair | Alex_Gaynor: yeah, i think that caused it to fall through and end up with the apps installing their own dependencies, so everything was quite out of order. | 15:43 |
| Alex_Gaynor | jeblair: thanks. | 15:44 |
| jeblair | dtroyer, sdague: you may want to look into the failure scenario here: http://logs.openstack.org/94/93794/1/check/check-grenade-dsvm/9dc701c/console.html.gz | 15:44 |
| Alex_Gaynor | jeblair: Can I leave an explanator comment in the recheck comment? | 15:44 |
| fungi | Alex_Gaynor: yeah, openstack/requirements bypasses the mirror so that it can test new versions which we haven't mirrored, so we see this from time to time when a worker has trouble reaching pypi.python.org | 15:44 |
| fungi | Alex_Gaynor: no explanations in the recheck comment... have to be left as a second comment | 15:44 |
| jeblair | Alex_Gaynor: nope, it's very strict; but you can leave a second comment | 15:44 |
| Alex_Gaynor | fungi: it's unfrotunate that the internet doesn't work | 15:44 |
| fungi | Alex_Gaynor: you speak lies. the internet can do no wrong | 15:45 |
| jeblair | hpcloud-az2 | 15:45 |
| Alex_Gaynor | fungi: On another CI system I run, which is really really tiny compared to openstack's, we /still/ see stuff break due to the internet not working. (For a while we gated on all of the links in our docs being live links... that got turned off very quickly) | 15:45 |
| jeblair | i like to think of it as the "Web" not working. ;) | 15:46 |
| fungi | if only i had a webcam, so you could see how surprised i am | 15:46 |
| jeblair | fungi: i have one but it doesn't work well | 15:46 |
| *** crc32 has joined #openstack-infra | 15:47 | |
| *** dkranz has quit IRC | 15:48 | |
| derekh_ | fungi: ok, thanks, I see the api getting hit now | 15:48 |
| jeblair | derekh_, fungi: i think we need to do something about this -- we're spending way too much time babysitting connections to tripleo... | 15:49 |
| *** yfried has quit IRC | 15:49 | |
| *** yfried has joined #openstack-infra | 15:49 | |
| derekh_ | fungi: and lots of instance booting | 15:50 |
| *** andreykurilin_ has joined #openstack-infra | 15:50 | |
| *** msabramo has joined #openstack-infra | 15:50 | |
| *** gyee has joined #openstack-infra | 15:50 | |
| jeblair | derekh_: can you look into the firewall situation? if there's something you could do on your side to make the connection loss more obvious that would be great | 15:50 |
| *** igor has quit IRC | 15:50 | |
| *** kgriffs has left #openstack-infra | 15:50 | |
| jeblair | derekh_, fungi: otherwise, someone will need to fix requests so that it handles keepalive packets correctly | 15:50 |
| *** igor__ has joined #openstack-infra | 15:51 | |
| fungi | i'm going to take a look at what our other providers do when i send unsolicited tcp+ack packets to their api endpoints | 15:51 |
| mordred | jeblair: that would be nice in general | 15:51 |
| derekh_ | jeblair: yup agreed, I think all lot of this is down to the melenox issue we been having in the R1 region, which should be sorted one we redeploy on Trusty, | 15:51 |
| derekh_ | jeblair: I gotta rrun in a few minutes but will look into firewall either tonight or tomorrow morning | 15:52 |
| *** sandywalsh_ has quit IRC | 15:52 | |
| jeblair | derekh_: ok, thanks | 15:52 |
| *** arnaud has quit IRC | 15:53 | |
| malini | jeblair, SergeyLukjanov: can you plz take a look at https://review.openstack.org/#/c/93568/ when you have a moment? Its a one liner - trying to make marconi jobs voting | 15:54 |
| mjseger | can someone help me figure out what 'git review' really means when it says it can't connect to gerrit? | 15:54 |
| jeblair | mjseger: run "git review -v" and paste the output | 15:54 |
| *** jgallard has quit IRC | 15:55 | |
| mjseger | http://paste.openstack.org/show/81070/ | 15:56 |
| *** arnaud has joined #openstack-infra | 15:56 | |
| mjseger | I can ping review.openstack.org | 15:56 |
| *** derekh_ has quit IRC | 15:56 | |
| *** gondoi is now known as zz_gondoi | 15:56 | |
| *** yfried has quit IRC | 15:57 | |
| jeblair | mjseger: so one thing that is wrong is that the username is incorrect, but it should detect that and tell you; so i suspect there is still a connection problem | 15:58 |
| jeblair | mjseger: what happens when you run "ssh -x -p 29418 ubuntu@review.openstack.org gerrit ls-projects" ? | 15:59 |
| *** aconrad has joined #openstack-infra | 15:59 | |
| mjseger | that | 15:59 |
| zaro | morning | 15:59 |
| jeblair | mjseger: that what? | 15:59 |
| mjseger | I did try the ssh manually and it gave me a permission denied. so does that m,ean it didn'tlike my ssh key? | 15:59 |
| anteaya | jeblair: he updated the paste | 15:59 |
| *** zns has joined #openstack-infra | 15:59 | |
| anteaya | what is your gerrit username mjseger | 16:00 |
| mjseger | my gerrit username is markseger. is that part of the problem? my linx username is mjs | 16:00 |
| anteaya | sorry he didn't update the paste | 16:00 |
| jeblair | oh, there must be an 'ubuntu' user on gerrit which is probably defeating the wrong username check. | 16:00 |
| jeblair | so that's probably just it then. | 16:00 |
| anteaya | yes | 16:00 |
| anteaya | markseger@review.openstack.org | 16:00 |
| anteaya | try that | 16:00 |
| anteaya | reed you about yet? | 16:01 |
| jeblair | mjseger: so you can run "git config --add gitreview.username markseger" | 16:01 |
| mjseger | hmm, when I try the ssh with the username markseger, it tells me the agent admitted failued to sign using the key | 16:01 |
| reed | good day anteaya | 16:01 |
| anteaya | mjseger: paste? | 16:01 |
| anteaya | reed so we need to discuss time and place for the infra thing | 16:01 |
| anteaya | the thing we agreed to do in the id meeting | 16:02 |
| anteaya | we need to avoid oscon and the qa meeting | 16:02 |
| reed | the infra 'orchard' classes? | 16:02 |
| anteaya | don't know if classes is the word I would use | 16:02 |
| mjseger | anteaya: paste what? | 16:02 |
| anteaya | but yeah, that thing | 16:02 |
| reed | sure | 16:02 |
| anteaya | mjseger: can you paste your lateste error? | 16:02 |
| anteaya | reed: great | 16:02 |
| reed | mailing list or here? | 16:02 |
| anteaya | any thoughts yet on time and place | 16:03 |
| jeblair | anteaya: oh, that may or may not be the same as the infra mid-cycle meetup. | 16:03 |
| fungi | mjseger: "the agent admitted failued to sign using the key" means you're using ssh-agent and haven't added your key to it | 16:03 |
| jeblair | anteaya: i think we were talking about going to austin for that. | 16:03 |
| reed | I still need to look at my notes, let me grab them now | 16:03 |
| anteaya | jeblair: yes, this was the thing that solves reed's problem | 16:03 |
| *** andreykurilin_ has quit IRC | 16:03 | |
| anteaya | the problem of infra is hard | 16:03 |
| anteaya | jeblair: yes, austin was mentioned | 16:03 |
| mjseger | http://paste.openstack.org/show/81071/ | 16:03 |
| anteaya | just didn't want to assume it was agreed to yet | 16:03 |
| anteaya | fungi mjseger More than one value for the key gitreview.username: markseger | 16:04 |
| anteaya | fungi: looks like you might need to do some splunking | 16:04 |
| mjseger | yes, I saw that but don't know what it means | 16:04 |
| anteaya | means taht somehow our gerrit db has more than one account for you | 16:04 |
| anteaya | which fungi needs to investigate and fix manually | 16:05 |
| anteaya | mjseger: are you signed into gerrit now? | 16:05 |
| fungi | mjseger: that means you have multiple entries for your gerrit username now in your git config | 16:05 |
| anteaya | oh is that waht it means | 16:05 |
| mjseger | anteaya: yes, I'm signed in | 16:05 |
| anteaya | listen to fungi first | 16:05 |
| *** yamahata has quit IRC | 16:05 | |
| reed | anteaya, didn't we have an etherpad for that session? I can't find it | 16:06 |
| * anteaya looks | 16:07 | |
| *** vtapia has quit IRC | 16:07 | |
| *** zz_gondoi is now known as gondoi | 16:08 | |
| anteaya | reed: this was the session when it was discussed: https://etherpad.openstack.org/p/juno-openid | 16:08 |
| anteaya | I don't recall that our discussion about the physical meeting translated to any notes on the etherpad | 16:09 |
| fungi | mjseger: do 'git config --unset gitreview.username' and then 'git config --global --replace-all --add gitreview.username markseger' | 16:09 |
| *** alugovoi has joined #openstack-infra | 16:09 | |
| anteaya | I know we agreed that we need to have it, and that austin was mentioned | 16:09 |
| reed | anteaya, I remember that too | 16:10 |
| anteaya | okay so shall we begin there? | 16:10 |
| fungi | mjseger: er, that second one is slightly wrong. forget the --add. just 'git config --global --replace-all gitreview.username markseger' | 16:10 |
| reed | I'd like to have the objective of this training written down | 16:10 |
| anteaya | I agree | 16:10 |
| anteaya | would you like to create the etherpad for it? | 16:10 |
| *** rdopieralski has quit IRC | 16:10 | |
| reed | let's add some more details to that existing etherpad | 16:10 |
| anteaya | okay | 16:11 |
| mjseger | anteaya: cool!!! looks like git review -s works now. thanks | 16:11 |
| *** gokrokve has quit IRC | 16:11 | |
| anteaya | mjseger: yay, fungi did the heavy lifting here | 16:11 |
| anteaya | mjseger: glad you are on your way | 16:11 |
| mjseger | thanks again... | 16:11 |
| anteaya | and jeblair helped, as always | 16:11 |
| anteaya | happy contributing | 16:12 |
| anteaya | and welcome | 16:12 |
| *** gokrokve has joined #openstack-infra | 16:12 | |
| fungi | so, interestingly, rackspace's api endpoint silently drops my tcp+ack probe but hpcloud's replies with tcp/rst | 16:13 |
| jeblair | fungi: that suggests the rax api endpoint is rather reliable, yeah? | 16:14 |
| fungi | but rackspace's firewall may just be smarter than me. more testing needed | 16:14 |
| mjseger | anteaya: actually this still is a problem. I've been experimenting with my workstation AND a vm in the cloud as both were busted. the git config fixed the vm but my workstation is still busted | 16:14 |
| reed | anteaya, the objective looks good | 16:15 |
| fungi | mjseger: what error are you getting on your workstation? | 16:15 |
| anteaya | reed: great, we have a foundation | 16:15 |
| reed | i'm going to write to Collier today, I have to coordinate with him for this and other topics | 16:15 |
| anteaya | reed: sounds good | 16:16 |
| reed | i'll keep you posted | 16:16 |
| anteaya | mostly I just wanted to start the conversation | 16:16 |
| anteaya | thanks | 16:16 |
| *** NithyaG is now known as NithyaG_AFK | 16:16 | |
| mjseger | I guess THAT's the one that reported more than one value for username | 16:16 |
| anteaya | mtreinish said he should know by the end of the week about the timing for qa meetup | 16:16 |
| anteaya | I have to depart soon and will be gone for the afternoon, once we have a sense of things we can post to the ml | 16:16 |
| *** gokrokve has quit IRC | 16:17 | |
| anteaya | fwiw I think we should just pick dates and place and post them, since opening that up could take forever and we won't make everyone happy | 16:17 |
| jeblair | anteaya: would you mind either using a new etherpad or appending to a new section of the old one that makes it clear you are having a follow-up conversation? | 16:17 |
| mordred | jeblair: there is an 'ubuntu' user on gerrit? | 16:17 |
| anteaya | and there are alternates to attending in order to work with infra | 16:17 |
| *** marcoemorais has joined #openstack-infra | 16:17 | |
| anteaya | jeblair: sure | 16:17 |
| jeblair | anteaya: i refer to summit etherpads all the time to refresh my memory, and that works less well when they start changing. :) | 16:17 |
| anteaya | reed: can we do a new etherpad? | 16:18 |
| *** marcoemorais has quit IRC | 16:18 | |
| anteaya | jeblair: *nod | 16:18 |
| zaro | msabramo: commented on https://review.openstack.org/#/c/94531 | 16:18 |
| reed | anteaya, indeed, I just want to make sure that Mark is on board, too and that tipit will be able to join | 16:18 |
| anteaya | sure | 16:18 |
| zaro | rcarrillocruz: hi | 16:18 |
| *** marcoemorais has joined #openstack-infra | 16:18 | |
| reed | anteaya, we can, not sure we need to do it now, before we have a buy in from Collier | 16:18 |
| anteaya | reed: do you want to start one and transfer relevant info? | 16:18 |
| msabramo | zaro: thanks, will look shortly | 16:18 |
| anteaya | your call, but let's not edit that one anymore | 16:18 |
| anteaya | I have to depart for the afternoon, so do what works best for you and we will find each other tomorrow | 16:19 |
| reed | anteaya, yep, agreed: that one now feels like it has the basic content of our conversation (it was missing those bits) | 16:19 |
| anteaya | sound fair? | 16:19 |
| *** jpich has quit IRC | 16:19 | |
| anteaya | reed: *nod | 16:19 |
| reed | anteaya, safe trip | 16:19 |
| anteaya | thanks | 16:19 |
| anteaya | just errands | 16:19 |
| openstackgerrit | A change was merged to openstack/requirements: Add wrapt library https://review.openstack.org/87395 | 16:19 |
| anteaya | but I have to get ready for my pickup | 16:19 |
| zaro | rcarrillocruz: you will need to wait for us to upgrade to gerrit 2.9 or greater to implement fix for bug 1011705 | 16:19 |
| uvirtbot | Launchpad bug 1011705 in openstack-ci "update_bug should use review owner instead of latest commiter" [Low,Confirmed] https://launchpad.net/bugs/1011705 | 16:19 |
| *** olaph has joined #openstack-infra | 16:20 | |
| zaro | rcarrillocruz: it's because my gerrit upstream fix only got merged to 2.9 | 16:20 |
| fungi | mjseger: so what does 'git config --get gitreview.username' tell you? | 16:20 |
| mjseger | anteaya: I can see in my git config there are 2 entries for gitrevfiew.username. | 16:20 |
| mjseger | that must be the problem | 16:20 |
| fungi | mjseger: yep | 16:21 |
| mjseger | what's the best way to clean up? can I edit the config file or is there a remove commad? | 16:21 |
| *** Hal_ has quit IRC | 16:21 | |
| mjseger | ahh, can I just do an --unset --globalname ... | 16:21 |
| fungi | mjseger: you can edit it and remove the errant line, or you can use the --unset option to git config | 16:21 |
| fungi | yup | 16:21 |
| mjseger | ;) | 16:21 |
| *** UtahDave has joined #openstack-infra | 16:22 | |
| mjseger | hmm, --unset --global didn't do it. guess I shoudl edit the file? | 16:22 |
| fungi | mjseger: the main thing to keep in mind with git config is that you usually have a global ~/.gitconfig file and then local .git/config files in each repository you've cloned, so you sometimes need to check more than one place for a setting (but the git config command will figure that out for you) | 16:22 |
| jeblair | zaro: i think if it's merged upstream we can consider backporting it to our 2.8 branch | 16:22 |
| mordred | jeblair, zaro: agree | 16:23 |
| *** ramashri has joined #openstack-infra | 16:23 | |
| mjseger | ahhh! that also explains why when I remove and recreate a repoi remembers the last settings ;) | 16:23 |
| pleia2 | good morning | 16:24 |
| *** praneshp has joined #openstack-infra | 16:25 | |
| jeblair | pleia2: morning! | 16:26 |
| *** hemna_ is now known as hemna | 16:26 | |
| *** zns has quit IRC | 16:26 | |
| mjseger | anteaya: still no joy - http://paste.openstack.org/show/81072/ does anythign in my config jump out? | 16:28 |
| *** rockyg has left #openstack-infra | 16:29 | |
| openstackgerrit | A change was merged to openstack-infra/config: Move back to 8G tripleo nodes https://review.openstack.org/94558 | 16:30 |
| mjseger | anteaya: comparing the git configs between the 2 systems, the one that works has entries for gerrit.url and gerrit.fetch and the busted one has neither! I don't remember ever adding those manually | 16:31 |
| *** ominakov has quit IRC | 16:31 | |
| *** hogepodge has joined #openstack-infra | 16:35 | |
| zaro | jeblair, mordred : ok, i'll propose a change to our fork. thx | 16:36 |
| fungi | mjseger: git review adds them itself after a successful first connection | 16:39 |
| *** aconrad has quit IRC | 16:39 | |
| *** arborism has joined #openstack-infra | 16:40 | |
| *** gokrokve has joined #openstack-infra | 16:42 | |
| *** TheJulia has joined #openstack-infra | 16:42 | |
| fungi | mjseger: from the system exhibiting that error, try running 'ssh -p29418 markseger@review.openstack.org gerrit ls-projects' and see what error you get | 16:42 |
| mjseger | fungi: I made my config of my workstation look like the one on the VM that works correctly and now I'm getting a different type of failure:h ttp://paste.openstack.org/show/81074/ | 16:42 |
| mjseger | Agent admitted failure to sign using the key., followed by permission denied. does that mean I have a bad key? | 16:43 |
| fungi | mjseger: it means you're using ssh-agent and haven't added your key to the agent | 16:43 |
| mjseger | not sure what agent means in this context. I have added the key to my settings on github | 16:44 |
| fungi | mjseger: just run 'ssh-add' and it should make the ssh agent know to use that key | 16:45 |
| zaro | rcarrillocruz: i've patched our fork of gerrit, so you will only need to wait until our next upgrade.. https://review.openstack.org/#/c/94675 | 16:46 |
| *** gokrokve has quit IRC | 16:47 | |
| mjseger | fungi: nice - I think that did it! I don't get errors any more from git review -s. how do you remember all these wacko subleties? ;) | 16:47 |
| mjseger | thanks a bunch, both you and anteaya | 16:47 |
| fungi | mjseger: as a systems administrator, i have come to be intimately familiar with ssh | 16:47 |
| mjseger | indeed! | 16:47 |
| *** miqui has quit IRC | 16:47 | |
| fungi | mjseger: and as a developer, same for git | 16:47 |
| mjseger | ;) | 16:48 |
| *** esker has quit IRC | 16:48 | |
| mjseger | gotta run, but you definitely baled me out. thanks again... | 16:48 |
| *** esker has joined #openstack-infra | 16:48 | |
| openstackgerrit | ayoung proposed a change to openstack/requirements: request-kerberos https://review.openstack.org/84740 | 16:48 |
| *** esker has quit IRC | 16:49 | |
| *** miqui has joined #openstack-infra | 16:50 | |
| pleia2 | dprince: can I get your eyes on this one? https://review.openstack.org/#/c/94272/ (reviewday is broken because of it) | 16:51 |
| dprince | pleia2: looking... | 16:52 |
| *** afazekas has quit IRC | 16:53 | |
| *** jooools has quit IRC | 16:54 | |
| * JayF unsubscribed from os-dev for the third time due to other peoples' misconfigured servers :( | 16:56 | |
| *** vkdrao has joined #openstack-infra | 16:57 | |
| jeblair | JayF: is it time for us to start looking at blocking people with problem servers from subscribing? istr we were hoping that the problem might self-correct and stabilize (since presumably they are getting kicked too). | 16:58 |
| dprince | pleia2: looks good to me, feel free to send it | 16:58 |
| pleia2 | dprince: thank you | 16:58 |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Ensure mergeprop instantiation failures are not fatal https://review.openstack.org/94272 | 16:59 |
| dprince | pleia2: I may have just left that extra raise in there when debugging at some point | 16:59 |
| *** SumitNaiksatam has quit IRC | 16:59 | |
| * pleia2 nods | 16:59 | |
| openstackgerrit | A change was merged to openstack-infra/reviewday: Remove WIP reviews from reviewday report https://review.openstack.org/94266 | 16:59 |
| *** esker has joined #openstack-infra | 17:00 | |
| fungi | jeblair: clarkb: mordred: i'll be disappearing today around 20:00 utc (grabbing an early dinner, then going to see eddie izzard in durham) | 17:00 |
| *** esker has quit IRC | 17:00 | |
| *** jamielennox is now known as jamielennox|away | 17:00 | |
| fungi | got tickets months ago, and lucky too since it sold out almost instantly | 17:01 |
| *** dstanek is now known as dstanek_zzz | 17:01 | |
| jeblair | fungi: nice, where? | 17:01 |
| fungi | durham performing arts center | 17:01 |
| TheJulia | i heard tickets sold out insanely fast for that show | 17:02 |
| clarkb | morning | 17:02 |
| fungi | yep | 17:02 |
| clarkb | eddie izzard in NC? | 17:02 |
| *** mkoderer has quit IRC | 17:02 | |
| TheJulia | yup | 17:02 |
| jeblair | never been there -- they built it after i left | 17:02 |
| *** moted has quit IRC | 17:02 | |
| TheJulia | i think the show is tonight | 17:02 |
| *** Longgeek has quit IRC | 17:02 | |
| fungi | it is, ~7pm local | 17:03 |
| TheJulia | oh jeeze | 17:03 |
| clarkb | fungi: I don't think we need to cram anything into git review | 17:03 |
| * TheJulia makes mental note to go park in the deck at ATCO by DPAC well before 7 then | 17:04 | |
| fungi | TheJulia: yeah, we're going to try to park by 5pm and then eat somewhere in atd, probably cuban revolution | 17:04 |
| clarkb | zaro: I just updated the etherpad about drafts to make it clear that drafts will remain as drafts | 17:04 |
| TheJulia | fungi: i've always found cuban revolution to be really slow, but the food is good | 17:05 |
| clarkb | fungi: jeblair what do you think about the date ni that email? should we do it sooner? | 17:05 |
| fungi | TheJulia: yeah, but we'll have a couple hours there so slow is fine. oh, and 7pm is when the doors open, show starts later, but the crowds... the crowds | 17:05 |
| *** dstanek_zzz is now known as dstanek | 17:05 | |
| *** Bobba has quit IRC | 17:05 | |
| pleia2 | TheJulia: yay, welcome! | 17:06 |
| openstackgerrit | Brad P. Crochet proposed a change to openstack-infra/nodepool: Add cloud-user to non-root user list https://review.openstack.org/94643 | 17:06 |
| JayF | jeblair: that's why I mentioned it. If peoples' mailservers are not behaving nicely with a list then maybe they shouldn't be allowed to use those mailservers for list email. | 17:06 |
| fungi | clarkb: that date wfm | 17:06 |
| *** harlowja has joined #openstack-infra | 17:06 | |
| TheJulia | fungi: yeah... Until yesterday I worked in Crowe on the ATCO campus, so I've seen the masses of people :\ | 17:06 |
| TheJulia | pleia2: thank you! | 17:06 |
| *** changbl has quit IRC | 17:06 | |
| fungi | JayF: jeblair: agreed. detecting them may be a bit of a chore though, unless we just remove on request when people point out the issue | 17:07 |
| *** _nadya_ has quit IRC | 17:07 | |
| mordred | TheJulia: o hai! | 17:07 |
| *** Bobba has joined #openstack-infra | 17:07 | |
| fungi | TheJulia: so you're a triangle local, eh? we'll have to make sure you get to know the other locals soon | 17:08 |
| clarkb | mordred: thoughts on https://etherpad.openstack.org/p/disabling-gerrit-drafts ? | 17:08 |
| clarkb | or did you already +1? | 17:08 |
| *** Bobba is now known as Guest67947 | 17:08 | |
| *** saschpe has quit IRC | 17:08 | |
| mordred | clarkb: I +1'd in IRC | 17:08 |
| fungi | clarkb: the main reason i asked about shoehorning -W back in was so that people who like -D still have a viable alternative | 17:08 |
| clarkb | fungi: I think the issues are orthogonal | 17:09 |
| TheJulia | fungi: I'm actually in the Triad, but I've been known to spend some time in the Triangle. | 17:09 |
| clarkb | drafts are broken and -D isn't a viable thing | 17:09 |
| mordred | I think that we should do both things | 17:09 |
| openstackgerrit | Roger Luethi proposed a change to openstack-dev/pbr: Fix typos in docs https://review.openstack.org/94678 | 17:09 |
| fungi | TheJulia: oh fun, that was one heck of a commute then | 17:10 |
| *** saschpe has joined #openstack-infra | 17:10 | |
| TheJulia | fungi: When I did it, it was painful. I was lucky to not have to do it every day... unless I wanted tylers for lunch | 17:12 |
| *** bhuvan has joined #openstack-infra | 17:12 | |
| *** aconrad has joined #openstack-infra | 17:12 | |
| *** terryw has quit IRC | 17:13 | |
| fungi | TheJulia: well, regardless, welcome to the north carolina contingent. clarkb accuses us of being robots | 17:13 |
| *** nati_ueno has joined #openstack-infra | 17:13 | |
| TheJulia | mordred: greetings | 17:14 |
| TheJulia | fungi: Thanks :) | 17:14 |
| *** otherwiseguy has joined #openstack-infra | 17:16 | |
| clarkb | mordred: where are we on removing hpcloud 1.0? | 17:16 |
| *** SumitNaiksatam has joined #openstack-infra | 17:16 | |
| *** mriedem has quit IRC | 17:16 | |
| mordred | clarkb: we just need to land the patch - I'm aroudn all day, so I can watch it | 17:17 |
| fungi | mordred: got a link? i'm handy with my +2 stamp | 17:18 |
| *** mriedem has joined #openstack-infra | 17:18 | |
| clarkb | fungi: jeblair maybe you guys can take a quick look then modrred and I babysit | 17:18 |
| *** markmcclain has quit IRC | 17:18 | |
| mordred | fungi: https://review.openstack.org/#/c/94503/ | 17:18 |
| mordred | jeblair: ^^ | 17:18 |
| JayF | question re: keysigning: All parties I've been to before, the process has been download from keyserver, sign, re-upload to same keyserver, but I'm getting loads of emails with my signed key | 17:19 |
| jeblair | clarkb, mordred: wcpgw? | 17:19 |
| JayF | is it a preference thing or would be it bad juju to re-upload other peoples' keys? | 17:19 |
| fungi | JayF: caff e-mails them for security reasons. if you received the signature and can decrypt it successfully (because it's encrypted to your key) then that proves you really are in control of both the address and the key | 17:20 |
| *** moted has joined #openstack-infra | 17:21 | |
| JayF | fungi: honestly I've never used a tool to do this, just a for loop, gpg, and the paper with the fingerprints on it | 17:21 |
| clarkb | jeblair: everything :) | 17:21 |
| *** changbl has joined #openstack-infra | 17:21 | |
| mordred | jeblair: bah. it's only a cloud | 17:21 |
| JayF | but if people prefer the emailing method, I can look into caff and do the signings this weekend :) | 17:22 |
| fungi | JayF: right, so basically if you're just uploading your signatures of someone's key to a keyserver, then you don't really have proof they're in control of the uids or the master key with which they're associated. it's a matter of personal preference, but it's a good additional check in my opinion | 17:23 |
| *** arnaud has quit IRC | 17:24 | |
| JayF | I understand the extra step for verification when it's put that way. I can appreciate the paranoia. | 17:24 |
| JayF | Is there a tool on the reverse side? aka an easier way than downloading and importing the attachments to reflect the signature? | 17:24 |
| *** james_li has quit IRC | 17:25 | |
| jeblair | JayF: you appreciate the paranoia; that's why we like you :) | 17:25 |
| JayF | well, it's annoying. but the scale usually has 'secure + annoying' on one side and 'insecure + convienient' on the other | 17:25 |
| fungi | JayF: in mutt, i just v to view attachments, then select the signature attachment and |gpg2 --import | 17:25 |
| fungi | done and done | 17:25 |
| JayF | I usually pick a place towards the middle, but I'm OK with going further towards annoying security if it's the group decision ;) | 17:26 |
| * JayF has never used mutt | 17:26 | |
| mordred | in thunderbirg with enigmail, I click on the attachment and select "import key" | 17:26 |
| fungi | JayF: long form, save the attachment to your homedir and then run gpg2 --import on it | 17:26 |
| notmyname | mordred: jeblair: clarkb: can you give this plan a once-over, please? https://gist.github.com/notmyname/7521817bd1027adc35a7 | 17:26 |
| fungi | JayF: and then send your key to the keyserver, which will update the published set of sigs on teh keyserver network | 17:26 |
| notmyname | mordred: jeblair: clarkb: this is also the FYI on what's coming next week in Swift-landia | 17:27 |
| JayF | yeah that's what I was going to, just is going to be onerous doing it 40 times * 4 UIDs | 17:27 |
| notmyname | mordred: jeblair: clarkb: that's just a rough draft, and I'll be sending out the final version to the -dev and -operators mailing list | 17:27 |
| zaro | msabramo: how about setting this one to WIP? https://review.openstack.org/#/c/87382 | 17:27 |
| fungi | JayF: you could save them all to your homedir, then dump them all to --import at once and send your key when that's done | 17:28 |
| mordred | notmyname: ++ looks great | 17:28 |
| notmyname | mordred: makes sense as to what's happening? the important parts are the dependent patches and the freeze on master | 17:29 |
| * sc68cal wishes there was a syntax plugin for gerrit for reStructured Text | 17:30 | |
| clarkb | notmyname: mordred: is there a reason to not propsoe a merge change to master? presumably the feature branch changes were already reviewed and tested | 17:30 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Revert "Stop using hpcloud az1-az3 in favor of region-b"" https://review.openstack.org/94503 | 17:30 |
| mordred | here comes the madness | 17:31 |
| *** esker has joined #openstack-infra | 17:31 | |
| fungi | notmyname: that plan seems okay | 17:31 |
| notmyname | clarkb: two reasons. first, they patches on feature/ec aren't fully reviewed and second, there's a lot of history in that branch which makes it harder to understand what's happening where and why | 17:32 |
| notmyname | fungi: thanks | 17:32 |
| clarkb | notmyname: ok so its a sanity thing. wfm | 17:32 |
| *** lcheng_ has joined #openstack-infra | 17:32 | |
| notmyname | clarkb: ya. and that the review guidelines were somewhat relaxed for inclusion on feature/ec | 17:32 |
| notmyname | clarkb: thanks | 17:32 |
| notmyname | clarkb: mordred: jeblair: mostly, I just don't want to take you by surprise at what's going on, and also make sure I'm not missing an important -infra step | 17:33 |
| fungi | seems like more fodder for the "people want feature branches until they realize the implications of developing on one while master continues to churn" | 17:33 |
| notmyname | fungi: I'm not yet ready to comment on the success or not of the feature/ec branch :-) | 17:33 |
| jeblair | notmyname: wfm. an alternative would be a second feature branch where you do your plan then merge that into master. just throwing that out there. | 17:33 |
| fungi | notmyname: fair enough ;) | 17:34 |
| clarkb | jeblair: once the cloud changes settle in we should get jesusaurus' change in | 17:34 |
| jeblair | notmyname: it's understandable though if you want to go with the freeze. there are many technical, and even more non-technical reasons to do that. :) | 17:34 |
| clarkb | otherwise it will continue to need rebases | 17:34 |
| clarkb | jesusaurus: are you able to babysit today? | 17:34 |
| *** aconrad has quit IRC | 17:35 | |
| pleia2 | notmyname: hey, how are you feeling? | 17:35 |
| notmyname | jeblair: right. I considered that. I think it would work too (and have some advantages), but I was under the impression it would be trickier to get another feature branch set up rather than do what I proposed there | 17:35 |
| notmyname | jeblair: and yes. the main reasons for the master freeze are non-technical :-). so that would happen no matter if another feature branch or not | 17:35 |
| notmyname | pleia2: well, my shoulder hurts :-) | 17:36 |
| fungi | notmyname: on the infra side, adding another feature branch is as simple as one of us branching feature/foo in gerrit from whatever sha you request | 17:36 |
| mordred | if it would happen either way, then I'd say the extra branch would be overhead | 17:36 |
| pleia2 | notmyname: I bet! | 17:36 |
| mordred | s/it/the freeze/ | 17:36 |
| jeblair | notmyname: oh, i think we just click a button; right fungi? of course we'd naturally have to call it "ec2". | 17:36 |
| notmyname | pleia2: waiting until after the swift team meeting to take the powerful pain meds :-) | 17:36 |
| fungi | hee | 17:36 |
| notmyname | mordred: ack | 17:36 |
| notmyname | jeblair: lo | 17:36 |
| jeblair | mordred: ya | 17:36 |
| notmyname | l | 17:36 |
| fungi | i even almost typed feature/ec2 without noticing the, er, implication ;) | 17:36 |
| jeblair | notmyname: now you have to choose between "achieves development objectives" and "would be really funny". good luck. :) | 17:37 |
| * mordred votes for humor | 17:37 | |
| notmyname | jeblair: tough call :-) | 17:37 |
| notmyname | jeblair: let's wait until after I take the percocet to make that call :-) | 17:38 |
| *** Sukhdev has quit IRC | 17:38 | |
| jeblair | notmyname: you'll be ready to write some infra patches too. | 17:38 |
| *** jlibosva has quit IRC | 17:39 | |
| *** dizquierdo has quit IRC | 17:40 | |
| *** apevec has quit IRC | 17:41 | |
| *** vkdrao has quit IRC | 17:43 | |
| *** gokrokve has joined #openstack-infra | 17:43 | |
| *** lakshmiS has joined #openstack-infra | 17:44 | |
| *** dangers_away is now known as dangers | 17:45 | |
| *** gokrokve has quit IRC | 17:47 | |
| jesusaurus | clarkb: yeah, i can babysit | 17:48 |
| jesusaurus | jeblair: my big change needs another review after a rebase yesterday | 17:48 |
| *** e0ne has joined #openstack-infra | 17:49 | |
| *** aconrad has joined #openstack-infra | 17:50 | |
| *** Ivoz has quit IRC | 17:50 | |
| openstackgerrit | Matt Farina proposed a change to openstack-infra/config: Adding openstack-sdk-dotnet and openstack-cli-powershell to gerritbot config. https://review.openstack.org/94683 | 17:52 |
| *** sarob has joined #openstack-infra | 17:53 | |
| *** e0ne has quit IRC | 17:55 | |
| *** nati_uen_ has joined #openstack-infra | 17:56 | |
| *** nati_uen_ has quit IRC | 17:56 | |
| *** e0ne has joined #openstack-infra | 17:56 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/jeepyb: Wait for ACL creation in manage-projects https://review.openstack.org/94684 | 17:57 |
| fungi | sdague: is that ^ what you were thinking we needed? | 17:57 |
| *** arborism is now known as amcrn | 17:57 | |
| *** bhuvan_ has joined #openstack-infra | 17:58 | |
| mordred | clarkb: ok. the nodepool.yaml is updated on nodepool and there are only three nodes in hpcloud-az* that are in the building state (were from before the config landed) | 17:58 |
| *** nati_uen_ has joined #openstack-infra | 17:58 | |
| mordred | clarkb: there are 83 nodes building in 1.1 - so, I think it's possibly safe to say it's live | 17:58 |
| *** nati_ueno has quit IRC | 17:59 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Python2: fixed UnicodeEncodeError https://review.openstack.org/88828 | 17:59 |
| *** e0ne has quit IRC | 17:59 | |
| clarkb | mordred: cool. now we monitor error rates | 17:59 |
| *** TheJulia has quit IRC | 18:00 | |
| *** markmcclain has joined #openstack-infra | 18:00 | |
| mordred | yup | 18:00 |
| *** bhuvan has quit IRC | 18:00 | |
| *** bhuvan_ is now known as bhuvan | 18:00 | |
| *** TheJulia has joined #openstack-infra | 18:01 | |
| *** pblaho has quit IRC | 18:03 | |
| *** lcostantino has quit IRC | 18:05 | |
| *** jp_at_hp has quit IRC | 18:07 | |
| *** james_li has joined #openstack-infra | 18:10 | |
| clarkb | mordred: fungi zaro can we discuss the git review thing re drafts being disabled | 18:10 |
| clarkb | are we saying that we should have git review ready first? | 18:10 |
| *** e0ne has joined #openstack-infra | 18:10 | |
| clarkb | because I really think they are independent problems and I don't want to delay this change if possible | 18:10 |
| *** arnaud has joined #openstack-infra | 18:11 | |
| jeblair | clarkb: what's the issue? | 18:15 |
| *** lcheng_ has quit IRC | 18:16 | |
| clarkb | jeblair: zaro mordred and fungi have suggested we cram a -W into git review as part of this change | 18:16 |
| clarkb | it isn't clear to me if they think this hsould happen first as a prereq | 18:16 |
| jeblair | clarkb: i think it's a good idea but doesn't need to be a pre-req | 18:18 |
| clarkb | fungi: 79125 would be another good one to look at. mordred didn't approve so that you could give it a go | 18:19 |
| mordred | I dont' think it's a pre-req | 18:20 |
| mordred | I do think we should do it | 18:20 |
| *** _nadya_ has joined #openstack-infra | 18:21 | |
| *** lcostantino has joined #openstack-infra | 18:23 | |
| *** _nadya_ has quit IRC | 18:26 | |
| *** nati_uen_ has quit IRC | 18:29 | |
| *** matsuhas_ has joined #openstack-infra | 18:29 | |
| reed | if anybody has time to look at this today, I'd appreciate it https://review.openstack.org/#/c/94342/ | 18:29 |
| *** pelix has quit IRC | 18:30 | |
| zaro | clarkb: i concur with jeblair | 18:30 |
| *** Ivo has joined #openstack-infra | 18:31 | |
| clarkb | ok so it sounds like I can send that email today and continue with the plan but should look into extending git revie | 18:31 |
| *** lcheng_ has joined #openstack-infra | 18:31 | |
| zaro | clarkb: i can help with the -W if you need. | 18:32 |
| zaro | afk, bb in about 2hrs | 18:32 |
| *** igor__ has quit IRC | 18:34 | |
| *** nati_ueno has joined #openstack-infra | 18:35 | |
| *** igor_ has joined #openstack-infra | 18:35 | |
| SpamapS | fungi: reading backscroll.. did you figure out if we broke our cloud or if nodepool just needed a bump? | 18:36 |
| SpamapS | fungi: I see tons of stuff running, so I assume you got it going. | 18:36 |
| clarkb | mordred: looks like a slight increase in error rate in hpcloud 1.1 | 18:36 |
| clarkb | mordred: but nothing to be worried about yet | 18:36 |
| sdague | jeblair: test_check_queue_success isn't one of my failing tests | 18:38 |
| clarkb | mordred: there is also a sharpish increase in the number of nodes building | 18:38 |
| sdague | jeblair: this is what I've gotten down to with the latest patch - http://logs.openstack.org/65/94365/3/check/gate-zuul-python27/4c237a6/testr_results.html.gz | 18:38 |
| jerryz | clarkb: could you share with me what you are using to monitor the nodepool? or just command line? | 18:39 |
| sdague | oh, sorry, I reintroduced something in the last patch | 18:39 |
| clarkb | jerryz: the graphs at http://status.openstack.org/zuul/ mostly | 18:40 |
| jerryz | clarkb: but that just shows the overall statistics. what about a particular cloud provider? | 18:41 |
| fungi | clarkb: yeah, i was mostly just pointing out that some people do use -D and so may welcome a working -W as an alternative after -D ceases working. i was more curious what others thought, and wasn't suggesting that we necessarily need to have that feature in a released git-review prior to turning off drafts in gerrit | 18:42 |
| *** gokrokve has joined #openstack-infra | 18:43 | |
| clarkb | gotcha | 18:44 |
| clarkb | jerryz: we try to be nice to the providers and don't advertise those numbers | 18:44 |
| mordred | clarkb: http://goo.gl/Ur3PDL this is what I'm looking at right now | 18:44 |
| jerryz | fungi: will submitting a wip patch trigger check jobs? | 18:45 |
| clarkb | mordred: ya, I have a hunch we will be reverting again | 18:45 |
| fungi | jerryz: yes | 18:45 |
| clarkb | but am willing to give it a bit of burn in | 18:45 |
| openstackgerrit | Marc Abramowitz proposed a change to openstack-infra/config: Add non-voting jenkins-job-builder-coverage https://review.openstack.org/94531 | 18:45 |
| openstackgerrit | A change was merged to openstack/requirements: Add pyca/cryptography to global requirements https://review.openstack.org/93794 | 18:46 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/zuul: Implement Depends-On: IdempotentID https://review.openstack.org/94365 | 18:46 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: fix grenade branch selection https://review.openstack.org/94650 | 18:46 |
| sdague | jeblair: so that's back down to the 2 failures that I'm stumped on | 18:46 |
| fungi | SpamapS: my guess is that something "happened" to packets for the established connection to the api endpoint for your cloud such that the firewall in front of it started dropping any further packets coming from nodepool. i killed the established socket and everything sprang back to life again | 18:46 |
| mfer | anyone know why something wouldn't be listed on git.openstack.org but would be listed on github as the mirror? | 18:46 |
| mfer | specifically https://github.com/stackforge/python-openstacksdk | 18:47 |
| jerryz | fungi: is it supposed to be like that? in wip, what if the patch is not supposed to pass because it is not quite there yet? | 18:47 |
| mfer | it is at https://git.openstack.org/cgit/stackforge/python-openstacksdk ... but not on the overview | 18:47 |
| *** gokrokve has quit IRC | 18:47 | |
| *** ominakov has joined #openstack-infra | 18:49 | |
| jerryz | clarkb: but you do have the tool, whose numbers are not for the public, right? because we are looking to contributing a nodepool to the community in the future, it would be nice to have such metric. | 18:50 |
| fungi | jerryz: the amount of resources we'd waste in that situation is relatively trivial. and it's not as if we intentionally didn't test draft changes, we simply had no choice since they weren't advertised in the event stream nor visible to zuul's gerrit account. i can recall numerous occasions where devs tried to get their drafts tested by subscribing the account used by our automation (clearly they | 18:50 |
| fungi | should have been using wip instead) | 18:50 |
| openstackgerrit | Matt Farina proposed a change to openstack-infra/config: Adding openstack-sdk-dotnet, openstack-cli-powershell, openstackclient, and python-openstacksdk to gerritbot config. https://review.openstack.org/94683 | 18:50 |
| fungi | mfer: it might have failed to get created on one or more of our cgit servers... checking it now | 18:50 |
| jerryz | fungi: got you. | 18:51 |
| mfer | fungi i'm dufus who looked past pagenation | 18:51 |
| fungi | mfer: oh, yes. i misread your comment | 18:51 |
| mfer | i'll go back to coding where i'm protected by git. it's safer than thinking today | 18:51 |
| fungi | it's definitely there | 18:51 |
| fungi | mfer: i think we previously "disabled" pagination in cgit by setting it to an "absurdly high" value :/ | 18:52 |
| fungi | clearly the definition of absurd varies over time | 18:52 |
| mfer | well, we're successful enough it wasn't high enough! | 18:52 |
| *** matsuhas_ has quit IRC | 18:54 | |
| *** krtaylor has joined #openstack-infra | 18:55 | |
| *** jcoufal has quit IRC | 18:56 | |
| fungi | mfer: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/git/cgitrc#n71 if you want to propose something even more absurd ;) | 18:56 |
| clarkb | teh cgit servers do cache pages for ~5 minutes iirc | 18:56 |
| *** _nadya_ has joined #openstack-infra | 18:56 | |
| *** jcoufal has joined #openstack-infra | 18:56 | |
| clarkb | so if you are constantly asking for athing it may not show up for a bit | 18:56 |
| *** jamielenz has joined #openstack-infra | 18:57 | |
| *** ianw- has joined #openstack-infra | 18:57 | |
| mfer | fungi oooo... i like choosing things... like absurd numbers. thanks | 18:57 |
| clarkb | mordred: do you want to point internal folks at this live? looking at zuul graphs I think we won't be able to sit here for long | 18:58 |
| *** ianw-- has joined #openstack-infra | 18:59 | |
| fungi | eee, that's almost a 50% error rate now | 18:59 |
| mordred | clarkb: I have - looking just a few more minutes | 19:00 |
| *** aconrad has quit IRC | 19:00 | |
| *** jamielennox|away has quit IRC | 19:00 | |
| *** jamielennox|away has joined #openstack-infra | 19:00 | |
| mordred | clarkb: woah. are we down to 0 ready nodes? | 19:00 |
| *** rickerc has joined #openstack-infra | 19:00 | |
| clarkb | mordred: yes its pretty bad | 19:01 |
| *** ianw has quit IRC | 19:01 | |
| clarkb | we are basically 50% building 25% in use and 25% deleting | 19:01 |
| *** rickerc has quit IRC | 19:01 | |
| mordred | ok. let's revert the revert revert | 19:01 |
| *** rickerc has joined #openstack-infra | 19:01 | |
| *** ianw- has quit IRC | 19:01 | |
| *** jamielenz has quit IRC | 19:02 | |
| openstackgerrit | Monty Taylor proposed a change to openstack-infra/config: Revert "Revert "Revert "Stop using hpcloud az1-az3 in favor of region-b""" https://review.openstack.org/94697 | 19:02 |
| clarkb | well I am not sure we need to yet if it is valuable to keep it going for a bit | 19:02 |
| *** ianw-- is now known as ianw | 19:02 | |
| clarkb | I think we will need to eventually though | 19:02 |
| mordred | there's the change for when we do | 19:02 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add e-r query for bug 1320670 https://review.openstack.org/94486 | 19:02 |
| uvirtbot | Launchpad bug 1320670 in grenade "404 on GET /v3/OS-SIMPLE-CERT/ca at grenade" [Undecided,In progress] https://launchpad.net/bugs/1320670 | 19:02 |
| openstackgerrit | Matt Farina proposed a change to openstack-infra/config: Increasing the number of repos displayed on git.openstack.org to remove the need to paginate. https://review.openstack.org/94698 | 19:03 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/jeepyb: Wait for ACL creation in manage-projects https://review.openstack.org/94684 | 19:03 |
| *** rickerc_ has joined #openstack-infra | 19:04 | |
| *** harlowja is now known as harlowja_away | 19:04 | |
| clarkb | fungi: nice | 19:04 |
| *** nati_ueno has quit IRC | 19:04 | |
| clarkb | I wish gerrit were a little more synchronous around things like project creation | 19:04 |
| fungi | clarkb: no idea if that's an actual problem, sdague suggested we might have a race condition there, so... | 19:05 |
| clarkb | yeah I wouldn't be surprised | 19:05 |
| sdague | fungi: it was a guess | 19:05 |
| clarkb | considering the other race prone stuff we have found | 19:05 |
| sdague | just based on the failing | 19:05 |
| *** nati_ueno has joined #openstack-infra | 19:05 | |
| *** msabramo has quit IRC | 19:05 | |
| sdague | I have no idea if it's real | 19:05 |
| fungi | sdague: yep, it seems possible at any rate | 19:05 |
| *** zns has joined #openstack-infra | 19:06 | |
| *** rickerc_ has quit IRC | 19:07 | |
| *** dims has joined #openstack-infra | 19:09 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/jeepyb: Wait for ACL creation in manage-projects https://review.openstack.org/94684 | 19:10 |
| clarkb | fungi: ^ lgtm | 19:13 |
| *** nati_ueno has quit IRC | 19:13 | |
| *** sarob has quit IRC | 19:13 | |
| *** aconrad has joined #openstack-infra | 19:13 | |
| clarkb | I need to grab lunch now. Going to review revert revertrevert but not approve it | 19:14 |
| clarkb | I think others can approve when it is appropriate | 19:14 |
| *** melwitt has joined #openstack-infra | 19:15 | |
| jeblair | mordred: do you think 1.1 can handle more than 200? (but obviously less than 600)? | 19:15 |
| clarkb | ok +2'd now to find lunch back in a bit | 19:15 |
| SpamapS | | free_ram_mb | -185044 | | 19:16 |
| SpamapS | hmmmmm | 19:16 |
| SpamapS | something tells me.. | 19:16 |
| SpamapS | thats a problem | 19:16 |
| *** marcoemorais has quit IRC | 19:17 | |
| fungi | SpamapS: probably because we need new images after derekh's nodepool config update | 19:17 |
| *** arnaud has quit IRC | 19:17 | |
| fungi | SpamapS: i'll delete the old images so we don't try to launch anything with them | 19:17 |
| fungi | which should hopefully also get nodepoold building new images with the right (smaller) flavor | 19:18 |
| SpamapS | fungi: perhaps.. but why would my hypervisors be reporting such a skew.. it is causing scheduling failure | 19:18 |
| fungi | SpamapS: he bumped the min-ready count when he reduced the min-ram | 19:19 |
| SpamapS | oohhh | 19:19 |
| SpamapS | so we haven't gotten rid of the old big ones yet? | 19:19 |
| fungi | SpamapS: i'm killing the images associated with them now | 19:19 |
| mordred | jeblair: maybe? should we try? | 19:19 |
| clarkb | maybe go to 300? | 19:20 |
| clarkb | that bisects the range and we can do a binary search for something optimalish | 19:20 |
| *** lxsli has quit IRC | 19:20 | |
| sdague | fungi: how would you feel about putting your patch on top of this one - https://review.openstack.org/#/c/94196/4/jeepyb/cmd/manage_projects.py , as I think that's now got all the feedback incorporated | 19:21 |
| sdague | just to avoid a rebase | 19:21 |
| mordred | it's also been suggested that we do something with having more than one neutron router | 19:21 |
| mordred | but that's a patch to nodepool | 19:21 |
| fungi | something seems mildly wrong with trying different values of quota usage on a provider to figure out where the "sweet spot" is between working and their cloud falling over | 19:21 |
| fungi | sdague: seems like a grand idea to me. doing now | 19:22 |
| mordred | yeah. I mean, we alraedy see launch-time spikes even at our old values | 19:22 |
| mordred | I think it might be worthwhile teaching nodepool about neutron and being able to configure instances-per-network as a general tunable | 19:22 |
| mordred | but I don't know when I'll be able to get to that | 19:22 |
| SpamapS | fungi: well.. cloud isn't falling over, it's just refusing to schedule more. :) | 19:23 |
| SpamapS | oh thats somebody else's cloud falling over? | 19:23 |
| SpamapS | for once? | 19:23 |
| clarkb | mordred: so I am all for teaching it how to do that, but there is clearly a massive gap between odl cloud and new cloud | 19:23 |
| fungi | SpamapS: i was talking about a different cloud ;) | 19:23 |
| fungi | SpamapS: for once ;) | 19:23 |
| clarkb | mordred: and the other side of the conversation doesn't appreciate that which is a bit scary | 19:23 |
| mordred | clarkb: I'm not sure the other side doesn't appreciate it | 19:24 |
| mordred | I think it might be due to the difference between nova-network and neutron | 19:24 |
| jeblair | mordred: http://tinyurl.com/kppgnvm | 19:24 |
| jeblair | mordred: i added some things to your graph | 19:24 |
| clarkb | mordred: it may be, but that isn't documented aiui and we are being forced to move a known and understood workload and get shrugs when it doesn't work | 19:24 |
| jeblair | the delete line is impressive | 19:25 |
| clarkb | now I appreciate that nova network and neutron do not have feature parity and that sucks for everyone | 19:26 |
| mordred | clarkb: sure. but I'm not really interested in fixing the social systems inside of a company, as I am in trying to make workloads work on it | 19:26 |
| fungi | jeblair: i think that link is to the original graph parameters. i don't see a delete line | 19:26 |
| mordred | and to-be-fair to neutron, the have a TON of code to work-around deficiencies in clouds that we've learned over the ages | 19:26 |
| *** msabramo has joined #openstack-infra | 19:26 | |
| jeblair | fungi: oops | 19:26 |
| mordred | jeblair: I agree with fungi | 19:26 |
| mordred | :) | 19:26 |
| clarkb | mordred: if this wasn't a forced move I would agree with you | 19:27 |
| *** harlowja_away is now known as harlowja | 19:27 | |
| jeblair | http://tinyurl.com/lubbrxu | 19:27 |
| jeblair | mordred: fungi ^ | 19:27 |
| jeblair | so be careful, we've got 2 lines which are rates, and 4 lines which are absolute values | 19:28 |
| * fungi gets nostalgic for calculus | 19:28 | |
| mordred | jeblair: yah | 19:29 |
| rcarrillocruz | zaro: thx for heads up regarding the gerrit patch, i'll await then | 19:29 |
| jeblair | what do you think about dropping 1.1 but not increasing 1.0? | 19:30 |
| fungi | mordred: jeblair: clarkb: it pretty clearly illustrates that once we started trying to build additional nodes, the number of existing nodes in use there fell fairly sharply | 19:30 |
| fungi | though in recent minutes, some recovery it seems, once the building spike subsided | 19:31 |
| clarkb | so how do we teach nodepool about routers. Do we want a network for say every 128 nodes and let nodepool spin up and down those networks+routers as necessary? | 19:31 |
| mordred | clarkb: yah, that's what I was thinking | 19:31 |
| clarkb | mordred: a quick check on my personal account shows my quota for routers is 1 | 19:32 |
| clarkb | so first step is probably going to be router and network quota adjustments | 19:32 |
| *** arnaud has joined #openstack-infra | 19:33 | |
| mfer | clarkb https://community.hpcloud.com/article/hp-cloud-default-account-limits-limit-increase-form-135 | 19:33 |
| *** arnaud has quit IRC | 19:33 | |
| jeblair | mordred: if i reduce the summary interval to 600s (from 1h) i can see that the error line went up when we jumped from 25 to 75 building servers | 19:33 |
| clarkb | mfer: ya I know the process | 19:33 |
| jeblair | that's probably on a bit of a delay too | 19:34 |
| mfer | clarkb i've just done this stuff so many times now... i even have the url handy | 19:34 |
| *** yfried has joined #openstack-infra | 19:34 | |
| openstackgerrit | A change was merged to openstack-infra/config: Adding Greek Language mailing list in the corresponding puppet module https://review.openstack.org/94342 | 19:34 |
| jeblair | so actually it probably couldn't even handle the 50 server increase | 19:34 |
| openstackgerrit | A change was merged to openstack-infra/git-review: "git review --setup" failed in Chinese locale https://review.openstack.org/79125 | 19:34 |
| clarkb | ok I really need to find food | 19:34 |
| clarkb | if someone wants to do rough numbers for what our quota should look like I can submit a request | 19:35 |
| *** rlandy has quit IRC | 19:35 | |
| jeblair | clarkb: 10 routers? strictly speaking we need 5. | 19:35 |
| mordred | well, if 50 is the pain point ... | 19:36 |
| mordred | and we want a quota of 500-ish servers | 19:36 |
| krtaylor | anteaya, are you doing an etherpad for wiki/ThirdPartySystems template? or is the plan to just jump on the wiki page? | 19:37 |
| *** arnaud has joined #openstack-infra | 19:37 | |
| mordred | then we should really have 10 routers and 10 networks yeah? also, I think we need to make sure we have port headroom, as each router and network takes a port too | 19:37 |
| mordred | jeblair: does the "teach nodepool about neutron" plan sound overly crazy? | 19:37 |
| jeblair | mordred: hrm. well, we seem to be able to handle 200 servers on a network, yeah? | 19:38 |
| fungi | smarcet: is http://www.codeyellow.nl/identifier-sqli.html a concern in the current openstackid implementation? | 19:38 |
| jeblair | mordred: i was suggesting that the pain point might be less than 50 servers beyond what we were doing, so somewhere between 200 and 250) | 19:38 |
| jeblair | clarkb: threw out 128 as a number, which is nicely about half that. | 19:39 |
| jeblair | mordred: does nodepool already know how to use neutron to specify a network? | 19:40 |
| jeblair | mordred: eg, the stuff that tripleo is using? | 19:40 |
| jeblair | mordred: (assuming that the network is already created and can be specified in the config file) | 19:41 |
| clarkb | yes | 19:41 |
| clarkb | I think the scaling bits are more of what would be new | 19:41 |
| jeblair | mordred, clarkb: because if so, we could probably make up N number of providers inside of nodepool, all of which are actually 1.1 | 19:41 |
| jeblair | and manually configure networks for each of them | 19:42 |
| clarkb | hrm, ugly but a quick fix | 19:42 |
| jeblair | clarkb: yup | 19:42 |
| jeblair | clarkb: it would have the benefit of testing the idea before we implemented network auto-scaling in nodepool | 19:42 |
| clarkb | ya | 19:42 |
| mordred | ++ | 19:42 |
| mordred | in fact | 19:43 |
| *** marcoemorais has joined #openstack-infra | 19:43 | |
| smarcet | fungi: i dont think so but i would open a chili ticket and do an exhaustive investigation on that | 19:43 |
| mordred | given that we see the launch-time spikes even at our normal usage | 19:43 |
| *** gokrokve has joined #openstack-infra | 19:43 | |
| mordred | even just adding a single extra network would probably give us _some_ data | 19:43 |
| clarkb | ok food happening, I trust you guys to figure it ouy :) | 19:43 |
| fungi | smarcet: okay, great. mainly just wanted to make sure it's on your radar since it just hit the oss-security mailing list a little while agi | 19:44 |
| fungi | ago | 19:44 |
| jeblair | clarkb: mordred: i think next step is to increase quota to (at least) 10 routers and 10 networks. i'm assuming clarkb is volunteering for that unless mordred wants to. | 19:44 |
| smarcet | fungi: sure np :) | 19:45 |
| *** gokrokve_ has joined #openstack-infra | 19:45 | |
| jeblair | fungi, mordred: i think we're probably ready to revert the revert revert | 19:45 |
| mordred | jeblair: I agree | 19:46 |
| mordred | let's revert the revert revert | 19:46 |
| jeblair | revert revert reverted | 19:46 |
| *** signed8bit is now known as signed8bit_ZZZzz | 19:46 | |
| *** signed8bit_ZZZzz is now known as signed8bit | 19:46 | |
| *** gokrokve has quit IRC | 19:48 | |
| *** dprince has quit IRC | 19:49 | |
| *** wenlock_ has joined #openstack-infra | 19:50 | |
| *** gokrokve_ has quit IRC | 19:50 | |
| *** esker has quit IRC | 19:50 | |
| mordred | jeblair: maybe one day it'll get to challenging the revert revert revert revert revert we did that time | 19:51 |
| *** Sukhdev has joined #openstack-infra | 19:51 | |
| *** marcoemorais has quit IRC | 19:54 | |
| *** marcoemorais has joined #openstack-infra | 19:54 | |
| sdague | mordred: are you going to fail hacking because the first line is too long? | 19:55 |
| mordred | sdague: nope. this is an infra project :) | 19:56 |
| sdague | :) | 19:56 |
| * mordred introduces sdague to jeblair | 19:56 | |
| *** doude has quit IRC | 19:56 | |
| jeblair | o/ | 19:57 |
| sdague | I did manage to fail a tempest revert because of that once | 19:57 |
| fungi | shorten to rererererererevert | 19:57 |
| jeblair | though actually that is one of the few style-ish things we semi-enforce, since really long subject lines screw up the gerrit interface | 19:58 |
| sdague | jeblair: so, sorry that I'd left the zuul patch in the higher failure state. The last one is down to a single failure that is opaque to me. | 19:58 |
| jeblair | but it's more like by the time you get to 100 characters i might mention it. :) | 19:58 |
| sdague | revert^5 | 19:58 |
| jeblair | sdague: ok, i'll check it out now | 19:58 |
| sdague | it has to do with the gerrit cache still having content in it, when that's unexpected, during tear down | 19:59 |
| fungi | TheJulia: not sure if you have a local osug out in the triad, but looks like our rtp osug is having its next meeting on the 29th if you're interested in joining... http://www.meetup.com/Triangle-OpenStack-Meetup/ | 20:00 |
| *** marcoemorais has quit IRC | 20:01 | |
| fungi | Shrews: olaph: we should consider grabbing drinks after that ^ if you're both free | 20:01 |
| jeblair | sdague: actually there's two errors, the real one is listed second, of course | 20:01 |
| jeblair | sdague: | 20:01 |
| *** marcoemorais has joined #openstack-infra | 20:01 | |
| jeblair | 2014-05-21 19:35:42.710 | File "tests/test_scheduler.py", line 3603, in test_client_promote_dependent | 20:01 |
| jeblair | 2014-05-21 19:35:42.710 | self.assertTrue(self.job_has_changes(self.builds[0], B)) | 20:01 |
| jeblair | 2014-05-21 19:35:42.711 | File "/usr/lib/python2.7/unittest/case.py", line 420, in assertTrue | 20:01 |
| openstackgerrit | A change was merged to openstack-infra/config: Revert "Revert "Revert "Stop using hpcloud az1-az3 in favor of region-b""" https://review.openstack.org/94697 | 20:01 |
| jeblair | yay! ^ | 20:01 |
| *** marcoemorais has quit IRC | 20:01 | |
| *** marcoemorais has joined #openstack-infra | 20:02 | |
| jeblair | sdague: so a change we expected to be in the series wasn't | 20:02 |
| jeblair | (change B, specifically) | 20:02 |
| Shrews | fungi: who? what? where? | 20:04 |
| * Shrews reads sb | 20:04 | |
| Shrews | fungi: ah yeah. i'm all in | 20:04 |
| fungi | now you just have to drag olaph away from the cow and chickens | 20:05 |
| sdague | jeblair: ok, I had thought I'd done a pretty good job at defaulting back in the noop case. I must have missed something. | 20:05 |
| jeblair | and GOATS! | 20:05 |
| fungi | indeed. and goats | 20:05 |
| sdague | from the patch, is anything sticking out at you about where that might be missed | 20:05 |
| *** _nadya_ has quit IRC | 20:06 | |
| *** arnaud has quit IRC | 20:07 | |
| TheJulia | fungi: cool, I've put it on my calendar. | 20:07 |
| jeblair | sdague: honestly, i'm not in the right headspace to fully read and understand it. i mean, if i could answer that in 5 minutes i would have just written the patch in 5 minutes a long time ago. :) | 20:07 |
| sdague | heh | 20:07 |
| *** marcoemorais has quit IRC | 20:07 | |
| *** marcoemorais has joined #openstack-infra | 20:08 | |
| * fungi is disappearing for the night | 20:09 | |
| jeblair | sdague: probably the best bet near-term is to keep poking at it, or see if hesketh has some cycles to look at it. i'll try to get to it, but i'm currently trying to get to things from 3 weeks ago | 20:09 |
| jeblair | fungi: have fun1 | 20:09 |
| fungi | jeblair: thanks! | 20:09 |
| TheJulia | fungi: enjoy the show | 20:09 |
| jeblair | dhellmann: you still want oslo.test renamed to oslotest? | 20:11 |
| *** e0ne has quit IRC | 20:11 | |
| *** e0ne has joined #openstack-infra | 20:11 | |
| jeblair | markwash: do you want glance-specs renamed to image-specs on friday? | 20:13 |
| markwash | sure | 20:13 |
| *** gema has quit IRC | 20:13 | |
| jeblair | markwash: ok | 20:13 |
| jeblair | jgriffith: would you like cinder-specs renamed to volume-specs on friday? | 20:14 |
| *** gema has joined #openstack-infra | 20:15 | |
| *** jeblair has quit IRC | 20:16 | |
| *** e0ne has quit IRC | 20:16 | |
| *** jeblair has joined #openstack-infra | 20:16 | |
| jeblair | i don't suppose anyone knows if kyle mestery is on irc and what his nick is...? | 20:17 |
| * mestery waves to jeblair! | 20:18 | |
| jeblair | mestery: hey there! | 20:18 |
| mestery | jeblair: Howdy! | 20:18 |
| jeblair | mestery: would you be interested in having neutron-specs renamed on friday? | 20:19 |
| mestery | jeblair: I think that makes sense. Will infra migrate existing reviews, or will folks have to propose them to the new repository? | 20:19 |
| mestery | jeblair: Or it's all just handled under the covers by the rename? | 20:19 |
| jeblair | mestery: all handled by the rename | 20:20 |
| jeblair | i'm not entirely sure what the name should be; networking-specs? network-specs? netconn-specs? | 20:20 |
| mestery | jeblair: OK, cool! Lets do the rename Friday then. | 20:20 |
| mestery | jeblair: I suggest networking-specs | 20:20 |
| *** wenlock_ has quit IRC | 20:20 | |
| jeblair | sounds good | 20:20 |
| *** james_li has quit IRC | 20:21 | |
| mestery | jeblair: Thanks for the help! Want me to send an email to openstack-dev after it's done, or will you do that? | 20:21 |
| *** marcoemorais has quit IRC | 20:21 | |
| *** ominakov has quit IRC | 20:22 | |
| jeblair | mestery: i'm about to send out the mail announcing the renames and will send a followup after it's done, but you might want to send one (or more) too with the neutron tag; couldn't hurt | 20:22 |
| *** marcoemorais has joined #openstack-infra | 20:22 | |
| mestery | jeblair: Roger sir, thank you! | 20:22 |
| jeblair | mikal: so we have a rename outage coming up on friday; do you want to rename nova-specs then? or do you want to wait for a later one (it will probably be many weeks). | 20:23 |
| *** arnaud has joined #openstack-infra | 20:23 | |
| *** james_li has joined #openstack-infra | 20:23 | |
| *** marcoemorais has quit IRC | 20:24 | |
| *** marcoemorais has joined #openstack-infra | 20:24 | |
| *** e0ne has joined #openstack-infra | 20:26 | |
| *** arnaud has quit IRC | 20:26 | |
| *** arnaud has joined #openstack-infra | 20:29 | |
| *** e0ne has quit IRC | 20:29 | |
| *** e0ne has joined #openstack-infra | 20:29 | |
| *** zns has quit IRC | 20:29 | |
| *** arnaud has quit IRC | 20:30 | |
| *** gema has quit IRC | 20:30 | |
| *** arnaud has joined #openstack-infra | 20:30 | |
| *** pcm_ has quit IRC | 20:31 | |
| *** arnaud has quit IRC | 20:31 | |
| *** e0ne_ has joined #openstack-infra | 20:31 | |
| *** arnaud has joined #openstack-infra | 20:31 | |
| *** gema has joined #openstack-infra | 20:32 | |
| *** e0ne has quit IRC | 20:32 | |
| *** eharney has quit IRC | 20:33 | |
| *** markwash_ has joined #openstack-infra | 20:33 | |
| *** markwash has quit IRC | 20:34 | |
| *** markwash has joined #openstack-infra | 20:34 | |
| *** zns has joined #openstack-infra | 20:35 | |
| *** r-daneel has joined #openstack-infra | 20:35 | |
| *** zns has quit IRC | 20:35 | |
| *** cyli has joined #openstack-infra | 20:35 | |
| *** zns has joined #openstack-infra | 20:36 | |
| *** weshay has quit IRC | 20:36 | |
| *** _nadya_ has joined #openstack-infra | 20:36 | |
| jeblair | dhellmann: so yesterday at the project meeting most of the ptls agreed that the specs repos should be named "program-spec"; i'm not sure what that should be for oslo-specs. library-specs? common-library-specs? ...? | 20:36 |
| dhellmann | jeblair: I saw your email about that and wondered the same thing | 20:37 |
| *** yfried has quit IRC | 20:37 | |
| *** markwash_ has quit IRC | 20:37 | |
| dhellmann | jeblair: is it likely that we would create a competing "common library" project? or incubator? there's not one "thing" called "oslo", it's sort of short-hand for the program itself | 20:37 |
| dhellmann | unlike say nova | 20:38 |
| dhellmann | jeblair: yes, please rename oslo.test oslotest | 20:38 |
| *** wenlock_ has joined #openstack-infra | 20:38 | |
| jeblair | well, i just discovered that in governance/programs.yaml the programs themselves actually have codenames. i did not realize that yesterday. i thought the codenames were just for the projects. | 20:39 |
| jeblair | dhellmann: and indeed, the common library program has a codename of oslo. | 20:39 |
| arnaud | hi jeblair | 20:40 |
| *** e0ne_ has quit IRC | 20:40 | |
| jeblair | but i guess as programs expand a bit more, we're getting into a grey area | 20:40 |
| arnaud | a quick question | 20:40 |
| *** e0ne has joined #openstack-infra | 20:40 | |
| *** UtahDave has quit IRC | 20:41 | |
| arnaud | we have a group for glance https://review.openstack.org/#/admin/groups/342,members and https://review.openstack.org/#/c/90461/6/modules/openstack_project/files/gerrit/acls/openstack/glance-specs.config | 20:41 |
| arnaud | and somehow we are not able to +2 https://review.openstack.org/#/c/94464/ | 20:41 |
| arnaud | do you have any idea what could be wrong? | 20:41 |
| dhellmann | jeblair: how are programs going to map to storyboard? I called it oslo-specs at least in part because we have a bunch of other oslo-* stuff, including the launchpad project and teams, but if that's going to change I'll go along with it | 20:41 |
| mordred | jeblair: naming sucks | 20:42 |
| *** e0ne has quit IRC | 20:43 | |
| clarkb | woo just had my first real post trusty upgrade breakage | 20:43 |
| jeblair | dhellmann: we'll have project groups; i don't think we've dived into what they would be named; but the main thing is that projects themselves will map directly to git repos, and the project groups will be entirely a storyboard db concept | 20:43 |
| *** gokrokve has joined #openstack-infra | 20:43 | |
| *** unicell1 has quit IRC | 20:43 | |
| jeblair | dhellmann: so i don't think it'll be a big factor | 20:43 |
| clarkb | coming back from suspend desktop related things went downhill fast | 20:43 |
| jeblair | dhellmann: i assume we can call the oslo project group whatever we want | 20:43 |
| dhellmann | jeblair: ok, I'm just thinking through other similar cases | 20:44 |
| jeblair | dhellmann: it's a good one | 20:44 |
| clarkb | arnaud: make suer you are logged in as that user? | 20:44 |
| dhellmann | jeblair: if our program name is officially "Common Libraries" then I guess we should be "common-libraries-specs" | 20:45 |
| jeblair | dhellmann: i guess the question is would you like to go to storyboard and look at the stories for the "oslo" group or the "common libraries" group? and what should show up on index pages versus urls... | 20:45 |
| arnaud | yeah both markwash and I tried | 20:45 |
| dhellmann | jeblair: right, I would want it to be consistent with what the other programs are doing | 20:46 |
| jeblair | dhellmann: i feel bad because we're shortening some others, like 'image' and 'volume'. i can't think of anything shorter than 'common-libraries' that's not confusing. and even then it's arguably slightly more confusing than oslo. | 20:46 |
| krotscheck | jeblair, dhellmann: Related aside: there’s actually been a request for projects that aren’t associated with git repos from the UX team. | 20:47 |
| clarkb | arnaud: ok I am doing some testing my side | 20:47 |
| arnaud | ok thank you clarkb :) | 20:47 |
| jeblair | krotscheck: what ux work doesn't relate to any project at all? | 20:47 |
| openstackgerrit | Nikhil Manchanda proposed a change to openstack/requirements: Remove openstack/database-api from list of projects https://review.openstack.org/94722 | 20:47 |
| *** afazekas has joined #openstack-infra | 20:47 | |
| clarkb | arnaud: it looks like I haev +2 for some reason :) | 20:48 |
| *** gokrokve has quit IRC | 20:48 | |
| arnaud | clarkb: really on this https://review.openstack.org/#/c/94464/ patch? | 20:48 |
| krotscheck | jeblair: Talk to them about it, I’m just the messenger. | 20:48 |
| clarkb | arnaud: yes | 20:48 |
| clarkb | arnaud: but I am also a gerrit admin so occasionally I have weird extra access that I shouldn't | 20:48 |
| clarkb | arnaud: so doing investigating | 20:49 |
| arnaud | I see | 20:49 |
| arnaud | ok | 20:49 |
| *** krotscheck has left #openstack-infra | 20:49 | |
| *** nithya_g has joined #openstack-infra | 20:49 | |
| *** pelix has joined #openstack-infra | 20:49 | |
| *** krotscheck has joined #openstack-infra | 20:49 | |
| openstackgerrit | A change was merged to openstack-infra/git-review: Remove parsing of --help and variants from our code https://review.openstack.org/90335 | 20:50 |
| krotscheck | Sometimes, using hotkeys while having the wrong window selected can be embarrassing. | 20:50 |
| *** afazekas is now known as afazekas_wfp | 20:50 | |
| clarkb | so I did add my self to the core group really quickly, had +2 there, removed myself and still had +2 | 20:51 |
| clarkb | krotscheck: jeblair can you see if you have +2 perms on 94464? | 20:51 |
| pelix | have a change that's been up for a while if anyone has any questions about it and would like to poke - https://review.openstack.org/#/c/48783/ | 20:51 |
| *** gema has quit IRC | 20:51 | |
| jeblair | clarkb: i do not | 20:52 |
| clarkb | jeblair: ok so maybe this is a cache thing? | 20:52 |
| * clarkb will flush caches and see if my +2 goes away | 20:52 | |
| *** SumitNaiksatam has quit IRC | 20:52 | |
| zaro | pelix: what are your thoughts on review jjb review rate? | 20:53 |
| *** gema has joined #openstack-infra | 20:53 | |
| jeblair | krotscheck: is there any artifact of this ux request that i can consult? a story, mailing list poist, irc meeting log, etc? | 20:53 |
| clarkb | huh I still haev +2 after flushing cache | 20:54 |
| clarkb | s | 20:54 |
| pelix | zaro: it's up and down, I can't complain too much as I've been a bit lack over the last few weeks in reviewing myself :/ | 20:55 |
| pelix | s/lack/lax/ | 20:55 |
| jesusaurus | clarkb: jeblair are we going to push the jenkins refactor patch through today? | 20:55 |
| zaro | looking for some help debugging this failure http://logs.openstack.org/75/94675/1/check/check-gerrit-buck/36482a0/console.html | 20:56 |
| clarkb | arnaud: curious to know if it works for you now though as I flushed gerrit's caches | 20:56 |
| arnaud | let me check | 20:56 |
| dhellmann | jeblair: I'm not too worried about the length. Between tab-completion and browser bookmarks it's not something I expect to type very often. | 20:56 |
| arnaud | clarkb: no still -1..+1 | 20:56 |
| *** fifieldt has joined #openstack-infra | 20:57 | |
| *** zns has quit IRC | 20:57 | |
| clarkb | weird | 20:57 |
| clarkb | I also tried logging in and out to invalidate my credentials but that didn't help | 20:57 |
| arnaud | could you double check that https://review.openstack.org/#/c/90461/ is how it is supposed to be? | 20:57 |
| clarkb | arnaud: yeah the acls looked ok | 20:58 |
| arnaud | I checked/compared with nova-specs, that looks the same | 20:58 |
| clarkb | yup I don't see anything wrong about the ACL list | 20:58 |
| *** e0ne has joined #openstack-infra | 20:58 | |
| *** gema has quit IRC | 20:58 | |
| jeblair | i'm going to edit the acl ttw | 20:58 |
| *** e0ne has quit IRC | 20:58 | |
| zaro | clarkb: is there anything exclusive going on ? | 20:58 |
| *** e0ne has joined #openstack-infra | 20:58 | |
| clarkb | zaro: I don't think so | 20:58 |
| jeblair | arnaud: try again now? | 20:59 |
| arnaud | yeah | 20:59 |
| arnaud | works jeblair! :) | 20:59 |
| *** e0ne has quit IRC | 20:59 | |
| *** afazekas_wfp has quit IRC | 20:59 | |
| jeblair | arnaud: yay | 20:59 |
| clarkb | jeblair: curious to know what yo udid? | 20:59 |
| jeblair | clarkb: :( | 20:59 |
| *** gema has joined #openstack-infra | 20:59 | |
| jeblair | clarkb: i just added an acl, saved, removed it, saved. | 20:59 |
| *** wenlock_ has quit IRC | 20:59 | |
| clarkb | wow | 21:00 |
| jeblair | clarkb: suggesting something about manage-projects acl push borked | 21:00 |
| *** e0ne has joined #openstack-infra | 21:00 | |
| clarkb | except gerrit showed the correct data | 21:00 |
| jeblair | yup. | 21:00 |
| arnaud | awesome thanks jeblair clarkb | 21:00 |
| *** e0ne has quit IRC | 21:00 | |
| clarkb | :/ | 21:01 |
| clarkb | ok /me goes back to figuring out neutron for nodepool | 21:01 |
| *** e0ne has joined #openstack-infra | 21:01 | |
| zaro | pelix: ok, good to hear. one of jjb core reviewers is having a big spike in his workload | 21:01 |
| *** mburned is now known as mburned_out | 21:01 | |
| *** e0ne has quit IRC | 21:02 | |
| cyli | Hi folks! Apologies for the very basic question - I've volunteered to set up a *-specs repo for heat, so I wanted to ask how I was to do this. Should I just submit a patch against http://git.openstack.org/cgit/openstack-infra/config/, copying the same setup as glance-specs or nova-specs? Is there anything else involved? | 21:02 |
| *** zns has joined #openstack-infra | 21:03 | |
| jeblair | cyli: that's about it. should probably be called 'orchestration-specs' as we're trying to converge on program-name for these | 21:03 |
| clarkb | hrm nova quota-show isn't working with my personal account | 21:04 |
| clarkb | mordred: ^ do you know how to make that work with hpcloud? | 21:04 |
| zaro | clarkb: got a moment to help? | 21:04 |
| clarkb | zaro: sure | 21:04 |
| *** gokrokve has joined #openstack-infra | 21:04 | |
| zaro | need help figuring out why this is failing.. http://logs.openstack.org/75/94675/1/check/check-gerrit-buck/36482a0/console.html | 21:04 |
| mordred_ | clarkb: I do not. mfer ? | 21:04 |
| jeblair | clarkb: i find 'nova absolute-limits' helpful | 21:05 |
| zaro | clarkb: worked on my local dev env | 21:05 |
| clarkb | looks like neutron quota-show works though | 21:05 |
| clarkb | and that has the quota's I am interested in so nova quota's not important | 21:05 |
| cyli | jeblair: ok, sounds good, thanks! Is building the docs the best way to see if I've done stuff correctly? | 21:05 |
| clarkb | jeblair: that works thanks | 21:05 |
| *** mrodden has quit IRC | 21:05 | |
| *** gema has quit IRC | 21:05 | |
| clarkb | zaro: prologcafe download failed | 21:06 |
| jesusaurus | clarkb: there is also `nova rate-limits` which is also useful and might help explain the name | 21:06 |
| *** dmsimard has joined #openstack-infra | 21:06 | |
| dmsimard | Is there a way to delete a paste on paste.openstack.org ? | 21:06 |
| *** arnaud has quit IRC | 21:07 | |
| *** sarob has joined #openstack-infra | 21:07 | |
| *** otherwiseguy has quit IRC | 21:07 | |
| *** gema has joined #openstack-infra | 21:07 | |
| zaro | clarkb: why? it didn't fail on my machine. | 21:07 |
| clarkb | zaro: could be just a network burp | 21:08 |
| clarkb | zaro: is the failure consistent? | 21:08 |
| zaro | clarkb: i've rechecked and got same failure | 21:08 |
| *** aysyd has quit IRC | 21:08 | |
| mfer | mordred clarkb i dunno. i would use neutron if that works. | 21:08 |
| zaro | clarkb: i can try another recheck? | 21:08 |
| clarkb | zaro: it may be worthwhile especially if there is a time delta | 21:09 |
| clarkb | zaro: could be more systemic though like location of the slaves | 21:09 |
| *** arnaud has joined #openstack-infra | 21:09 | |
| clarkb | since your test machien isn't in rackspace iirc | 21:09 |
| * zaro attempts another one. | 21:09 | |
| *** lcostantino has quit IRC | 21:09 | |
| zaro | clarkb: i got same failure on 1st PS, that one ran on hpcloud | 21:10 |
| *** amcrn_ has joined #openstack-infra | 21:10 | |
| clarkb | ok we have 610 ports and 600 floating IPs. | 21:10 |
| *** otherwiseguy has joined #openstack-infra | 21:10 | |
| clarkb | if we add 10 routers that is 10 more ports and 600 + 10 = 610 | 21:10 |
| *** pdmars has quit IRC | 21:11 | |
| clarkb | so I think the only thing we need to do is go from 5 networks to 10 and 1 router to 10 | 21:11 |
| clarkb | whipping up a request now | 21:11 |
| *** arnaud has quit IRC | 21:11 | |
| *** bknudson has quit IRC | 21:11 | |
| mordred_ | dmsimard: not really | 21:11 |
| *** arnaud has joined #openstack-infra | 21:11 | |
| *** arnaud has quit IRC | 21:11 | |
| dmsimard | mordred: /sadpanda | 21:11 |
| *** arnaud has joined #openstack-infra | 21:12 | |
| jeblair | lifeless: interested in renaming tripleo-specs to deployment-specs on friday? | 21:12 |
| *** michchap has quit IRC | 21:12 | |
| jeblair | mordred_: you have a tail | 21:13 |
| *** arnaud has quit IRC | 21:13 | |
| *** gema has quit IRC | 21:13 | |
| *** amcrn has quit IRC | 21:13 | |
| *** nati_ueno has joined #openstack-infra | 21:13 | |
| *** gema has joined #openstack-infra | 21:14 | |
| lifeless | jeblair: whenever suits for you | 21:15 |
| krotscheck | phschwartz: https://storyboard.openstack.org/#!/story/69 | 21:15 |
| *** reed has quit IRC | 21:18 | |
| *** mfer has quit IRC | 21:18 | |
| *** reed has joined #openstack-infra | 21:18 | |
| *** arnaud has joined #openstack-infra | 21:18 | |
| *** arnaud has quit IRC | 21:19 | |
| *** lxsli has joined #openstack-infra | 21:19 | |
| *** krotscheck has quit IRC | 21:19 | |
| *** krotscheck has joined #openstack-infra | 21:20 | |
| *** jcoufal has quit IRC | 21:21 | |
| *** nati_ueno has quit IRC | 21:26 | |
| clarkb | jeblair: mordred_ fungi quota request sent, you have been CC'd | 21:26 |
| *** smarcet has quit IRC | 21:26 | |
| *** nati_ueno has joined #openstack-infra | 21:26 | |
| mordred | jeblair: sometimes I have a tail. sometimes I do not have a tail. | 21:26 |
| *** mrodden has joined #openstack-infra | 21:29 | |
| *** michchap has joined #openstack-infra | 21:31 | |
| *** UtahDave has joined #openstack-infra | 21:31 | |
| *** julim has quit IRC | 21:32 | |
| *** nithya_g has quit IRC | 21:32 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add a tool for determining unaccounted for rechecks https://review.openstack.org/94624 | 21:32 |
| *** marcoemorais has quit IRC | 21:34 | |
| *** marcoemorais has joined #openstack-infra | 21:35 | |
| *** marcoemorais has quit IRC | 21:35 | |
| *** eglynn has joined #openstack-infra | 21:35 | |
| *** marcoemorais has joined #openstack-infra | 21:35 | |
| * pelix pokes mordred | 21:36 | |
| pelix | read your email much :) | 21:36 |
| *** lxsli has quit IRC | 21:36 | |
| *** dmsimard has left #openstack-infra | 21:37 | |
| * mordred writes on pelix's wall | 21:37 | |
| mordred | pelix: almost never | 21:37 |
| mordred | sup? | 21:37 |
| pelix | hah | 21:37 |
| pelix | thought so | 21:37 |
| pelix | just saying hello, now that I'm 6 weeks back from travelling :) | 21:37 |
| eglynn | I'm gonna get myself banned from this channel | 21:39 |
| eglynn | ... if I keep whining about infra patches sitting in the review queue for a *whole* day! ;) | 21:40 |
| * eglynn coughs and points to https://review.openstack.org/94044 | 21:40 | |
| eglynn | ... seriously it would be really nice to get that new specs repo created before the ceilo team meeting tomorrow | 21:41 |
| * eglynn fears he'll miss the boat on the wave of post-summit energy to file BPs | 21:41 | |
| *** andreaf has joined #openstack-infra | 21:41 | |
| mordred | pelix: hello! welcome back from being a travelling gypsy | 21:42 |
| *** eharney has joined #openstack-infra | 21:43 | |
| clarkb | mordred: can you abandon 58099? I think we are caching that now | 21:43 |
| mordred | clarkb: on it | 21:46 |
| *** SumitNaiksatam has joined #openstack-infra | 21:47 | |
| mordred | done | 21:47 |
| *** dstanek is now known as dstanek_zzz | 21:47 | |
| *** marcoemorais has quit IRC | 21:47 | |
| *** mwagner_lap has quit IRC | 21:47 | |
| clarkb | pleia2: you around? in 60375, does the puppet file resource need a recurse => true? | 21:48 |
| *** ominakov has joined #openstack-infra | 21:48 | |
| *** bknudson has joined #openstack-infra | 21:48 | |
| *** smarcet has joined #openstack-infra | 21:48 | |
| clarkb | Hunner: nibalizer: if I have a file resource that uses a dir as the source do I get all of the files in that dir one level below if not using recurse => true? | 21:48 |
| *** ominakov has quit IRC | 21:49 | |
| *** SumitNaiksatam has quit IRC | 21:49 | |
| *** michchap has quit IRC | 21:49 | |
| nibalizer | I doubt it, but ionno | 21:49 |
| *** Sukhdev has quit IRC | 21:51 | |
| *** _nadya_ has quit IRC | 21:52 | |
| dhellmann | it looks like when the oslo-specs repo was set up, no members were added to oslo-specs-core -- could someone add me? | 21:53 |
| anteaya | krtaylor: https://etherpad.openstack.org/p/third-party-wiki-templating | 21:53 |
| anteaya | this is my brain dump of what needs to be templated, working on figuring out templating | 21:53 |
| clarkb | dhellmann: sure, one sec | 21:53 |
| clarkb | dhellmann: done | 21:54 |
| *** eglynn has quit IRC | 21:54 | |
| dhellmann | clarkb: thanks! | 21:54 |
| *** hipster has quit IRC | 21:55 | |
| zaro | clarkb: tried recheck again 94675, it failed again this time on hpcloud slave. what do suggest i do? | 21:55 |
| clarkb | zaro: can you reproduce in hpcloud? | 21:56 |
| clarkb | zaro: I am guessing that upstream is broken | 21:56 |
| clarkb | and prologcafe has moved or gone away | 21:56 |
| clarkb | it probably works in your normal test env because maven has that package cached locally | 21:56 |
| * krotscheck cautiously peers at storyboard’s 1.1 feature tasks, and _thinks_ he might be able to get back into code. | 21:56 | |
| anteaya | I can't stay awake, I have to sleep some more, I will be back tomorrow morning | 21:56 |
| jeblair | i have sent 3 group nominations to the -dev list | 21:57 |
| *** smarcet has quit IRC | 21:58 | |
| * clarkb respodns to the non storyboard nominations | 21:58 | |
| *** gema has quit IRC | 21:59 | |
| pleia2 | clarkb: looking | 22:01 |
| *** gema has joined #openstack-infra | 22:01 | |
| *** zns has quit IRC | 22:01 | |
| pleia2 | clarkb: ah that review, that's the one I never did get around selinux issue for | 22:02 |
| clarkb | ah | 22:02 |
| clarkb | ok | 22:02 |
| mattoliverau | Morning | 22:02 |
| *** msabramo has quit IRC | 22:02 | |
| clarkb | pleia2: I am tackling my review list from oldest to enwest in gertty and that oen was in there | 22:02 |
| *** rfolco has quit IRC | 22:04 | |
| Hunner | clarkb: I think you don't get any files if not using recurse => true. If you want to just get the files one below, check out `recurselimit` or `ignore` | 22:04 |
| clarkb | Hunner: thanks | 22:05 |
| *** krtaylor has quit IRC | 22:05 | |
| *** michchap has joined #openstack-infra | 22:05 | |
| openstackgerrit | Ying Li proposed a change to openstack-infra/config: Create orchestration-specs git repo https://review.openstack.org/94735 | 22:07 |
| *** malini is now known as malini_afk | 22:07 | |
| *** dstanek_zzz is now known as dstanek | 22:08 | |
| *** msabramo has joined #openstack-infra | 22:08 | |
| clarkb | fungi: confirmed that gerrit ls-members does not do recursive group membership lookup | 22:09 |
| *** SumitNaiksatam has joined #openstack-infra | 22:12 | |
| *** eglynn has joined #openstack-infra | 22:12 | |
| *** gokrokve has quit IRC | 22:12 | |
| *** SumitNaiksatam has quit IRC | 22:12 | |
| jeblair | krotscheck: there are already more active storyboard-core members who are non-hp than hp. :) | 22:12 |
| jeblair | (though continuing in that direction is certainly good) | 22:13 |
| krotscheck | jeblair: Eeehhh… I was under the impression that most of those are infra-core and are basically grandfathered in. | 22:13 |
| *** gondoi is now known as zz_gondoi | 22:14 | |
| krotscheck | But I could be wrong. | 22:14 |
| * krotscheck is arguing with sphinx right now | 22:14 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/infra-specs: New specification for StoryBoard UX overhaul https://review.openstack.org/94499 | 22:14 |
| jeblair | krotscheck: i'm not sure "grandfathered" is the word i would use. storyboard is an infra project and infra-core rightfully have +2 there. but even so, i was only counting the _active_ reviewers. | 22:14 |
| jeblair | krotscheck: the top reviewers are you, nikita, thierry, sergey, myself, and mordred | 22:16 |
| pelix | any concerns about using mock.patch to test the JJB command line behaviour? | 22:16 |
| jeblair | krotscheck: those are the ones that are "above the line" for what i would consider appropriate for core membership regardless of their current status | 22:16 |
| krotscheck | jeblair: Righto | 22:17 |
| *** dims has quit IRC | 22:18 | |
| *** dstanek is now known as dstanek_zzz | 22:18 | |
| clarkb | 87384 is a tricky one | 22:18 |
| clarkb | jeblair: mordred fungi et al is the topic of what downstream things should go in openstack stuff something I should add to next week's meeting? | 22:19 |
| *** _nadya_ has joined #openstack-infra | 22:19 | |
| clarkb | I feel like we are running into this a lot without a good handle on how to generally deal with these problems | 22:19 |
| *** yamahata has joined #openstack-infra | 22:19 | |
| clarkb | we want the openstack-infra/config tooling to be reconsumable but imo openstack_project is for openstack things | 22:19 |
| jeblair | clarkb: i ignored it because it said it needed a new patchset. | 22:20 |
| *** sarob has quit IRC | 22:20 | |
| mordred_ | clarkb: I agree, but I also think we have an overly broad definition of OpenStack things | 22:20 |
| clarkb | jeblair: ya Iam less concerned about the change itself and more the type of change | 22:20 |
| clarkb | mordred_: I don't think we have a definition at all :) | 22:21 |
| mordred_ | yup | 22:21 |
| *** sarob has joined #openstack-infra | 22:21 | |
| *** marcoemorais has joined #openstack-infra | 22:21 | |
| *** pelix has left #openstack-infra | 22:21 | |
| *** sarob_ has joined #openstack-infra | 22:22 | |
| * clarkb adds a thing to the meetin gagenda | 22:22 | |
| *** markmcclain has quit IRC | 22:22 | |
| *** markmcclain has joined #openstack-infra | 22:23 | |
| *** mbacchi has quit IRC | 22:23 | |
| *** sarob has quit IRC | 22:25 | |
| krotscheck | Anyone know why “tox -evenv -- python setup.py build_sphinx” would fail the first time its run, but pass if you run it immediately a second time? | 22:25 |
| krotscheck | (in infra-specs) | 22:25 |
| clarkb | nope, I can try to reproduce shortly | 22:26 |
| *** sarob_ has quit IRC | 22:26 | |
| mordred | clarkb: I would consider that a bug :) | 22:26 |
| *** dstanek_zzz is now known as dstanek | 22:27 | |
| clarkb | mordred: that == the tox thing? | 22:27 |
| clarkb | I haev updated the meeting agenda | 22:27 |
| mordred | yup. and sorry. krotscheck: I would consider that a bug | 22:27 |
| clarkb | well tere isn't anything in infra specs yet sooo.... | 22:28 |
| clarkb | krotscheck: what change are you using as a base? | 22:28 |
| krotscheck | 94499 | 22:28 |
| *** pcrews__ has joined #openstack-infra | 22:28 | |
| *** fbo_away has joined #openstack-infra | 22:28 | |
| * mordred blames little bouncing leprechauns | 22:29 | |
| jhesketh | Morning | 22:29 |
| *** shardy is now known as shardy_zz | 22:29 | |
| *** jerryz_ has joined #openstack-infra | 22:29 | |
| jhesketh | sdague: what did I need to spend some cycles on? | 22:29 |
| *** wenlock_ has joined #openstack-infra | 22:29 | |
| *** shardy_zz has quit IRC | 22:29 | |
| *** james_li has quit IRC | 22:29 | |
| *** _nadya__ has joined #openstack-infra | 22:30 | |
| *** dkliban1 has joined #openstack-infra | 22:30 | |
| mordred | jhesketh: all the things | 22:30 |
| *** jeremyb_ has joined #openstack-infra | 22:30 | |
| jhesketh | woo | 22:30 |
| clarkb | krotscheck: your storyboard ux rst file isn't included by a toctree | 22:30 |
| clarkb | krotscheck: you need to include it somewhere otherwise it is a broken link | 22:31 |
| *** pfallenop has joined #openstack-infra | 22:31 | |
| clarkb | I think it doesn't error the second time because the file is already built | 22:31 |
| *** Adri2000_ has joined #openstack-infra | 22:31 | |
| *** Adri2000_ has quit IRC | 22:31 | |
| *** Adri2000_ has joined #openstack-infra | 22:31 | |
| clarkb | and isn't built again thus doesn't trigger warning | 22:31 |
| *** mancdaz_ has joined #openstack-infra | 22:31 | |
| *** cgoncalv1s has joined #openstack-infra | 22:31 | |
| *** rcarrillocruz has quit IRC | 22:32 | |
| *** zns has joined #openstack-infra | 22:32 | |
| *** signed8bit has quit IRC | 22:32 | |
| krotscheck | clarkb: Yes, that’s what the error says the first time I run it. | 22:32 |
| krotscheck | clarkb: So, is my change to index.rst not sufficient? | 22:32 |
| krotscheck | That makes sense. | 22:32 |
| *** SumitNaiksatam has joined #openstack-infra | 22:32 | |
| *** portante has quit IRC | 22:33 | |
| zaro | clarkb: you were right about the gerrit build problem, upstream bug. | 22:33 |
| *** msabramo1 has joined #openstack-infra | 22:33 | |
| *** michchap_ has joined #openstack-infra | 22:33 | |
| *** notmyname_ has joined #openstack-infra | 22:33 | |
| *** james_li has joined #openstack-infra | 22:33 | |
| *** pcrews_ has quit IRC | 22:33 | |
| *** fbo has quit IRC | 22:33 | |
| *** notmyname has quit IRC | 22:33 | |
| *** mancdaz has quit IRC | 22:33 | |
| *** aconrad has quit IRC | 22:33 | |
| *** fbo_away is now known as fbo | 22:33 | |
| *** mancdaz_ is now known as mancdaz | 22:33 | |
| *** notmyname_ is now known as notmyname | 22:34 | |
| clarkb | krotscheck: under toctree in index.html you need to list your file | 22:34 |
| clarkb | I think | 22:34 |
| *** cyli_ has joined #openstack-infra | 22:34 | |
| *** phschwartz_ has joined #openstack-infra | 22:34 | |
| clarkb | oh you have a glob max depth 2 | 22:34 |
| clarkb | maybe 2 isn't deep enough? | 22:34 |
| *** daenney has quit IRC | 22:34 | |
| *** igor__ has joined #openstack-infra | 22:34 | |
| *** johnthetubaguy_ has joined #openstack-infra | 22:34 | |
| krotscheck | clarkb: Hrm. I guess I assumed that :glob: and :maxdepth: 2 would map the specs/* to include things? | 22:35 |
| krotscheck | I tried 3 | 22:35 |
| *** timrc has quit IRC | 22:35 | |
| krotscheck | clarkb: I wonder if the globbing pattern needs to be specs/**/* | 22:35 |
| clarkb | krotscheck: maybe /me reads docs | 22:35 |
| jhesketh | thanks for the nomination jeblair et al :-) | 22:35 |
| *** timrc has joined #openstack-infra | 22:35 | |
| *** msabramo has quit IRC | 22:35 | |
| * jhesketh will brb | 22:35 | |
| *** portante has joined #openstack-infra | 22:35 | |
| *** morgabra_ has joined #openstack-infra | 22:36 | |
| *** jhesketh has quit IRC | 22:36 | |
| *** hughsaunders_ has joined #openstack-infra | 22:36 | |
| *** amcrn_ has quit IRC | 22:36 | |
| krotscheck | clarkb: There’s a ** in there, let me try that | 22:36 |
| krotscheck | Well, that didn’t error. | 22:36 |
| zaro | clarkb: https://groups.google.com/d/msg/repo-discuss/csdj1_OErO8/2jLCtXrllscJ | 22:37 |
| krotscheck | Ok, that did it | 22:37 |
| *** dansmith_ has joined #openstack-infra | 22:37 | |
| openstackgerrit | Michael Krotscheck proposed a change to openstack-infra/infra-specs: New specification for StoryBoard UX overhaul https://review.openstack.org/94499 | 22:37 |
| *** nico_ has joined #openstack-infra | 22:37 | |
| *** Adri2000 has quit IRC | 22:38 | |
| *** _nadya_ has quit IRC | 22:38 | |
| *** pfalleno1 has quit IRC | 22:38 | |
| *** jeremyb has quit IRC | 22:38 | |
| *** johnthetubaguy has quit IRC | 22:38 | |
| *** cyli has quit IRC | 22:38 | |
| *** dkliban has quit IRC | 22:38 | |
| *** phschwartz has quit IRC | 22:38 | |
| *** hughsaunders has quit IRC | 22:38 | |
| *** morgabra has quit IRC | 22:38 | |
| *** michchap has quit IRC | 22:38 | |
| *** asettle has quit IRC | 22:38 | |
| *** anteaya has quit IRC | 22:38 | |
| *** greghaynes has quit IRC | 22:38 | |
| *** cgoncalves has quit IRC | 22:38 | |
| *** reed has quit IRC | 22:38 | |
| *** exz has quit IRC | 22:38 | |
| *** dansmith has quit IRC | 22:38 | |
| *** igor_ has quit IRC | 22:38 | |
| *** nico_ is now known as exz | 22:38 | |
| *** hughsaunders_ is now known as hughsaunders | 22:38 | |
| *** johnthetubaguy_ is now known as johnthetubaguy | 22:38 | |
| *** dansmith_ is now known as dansmith | 22:38 | |
| zaro | pelix : considering that there are no test for jjb cmd, i would support that option. | 22:39 |
| *** asettle has joined #openstack-infra | 22:39 | |
| *** greghaynes has joined #openstack-infra | 22:39 | |
| *** jgrimm has quit IRC | 22:39 | |
| *** cyli_ is now known as cyli | 22:40 | |
| *** reed has joined #openstack-infra | 22:40 | |
| *** rcarrillocruz has joined #openstack-infra | 22:40 | |
| *** r-daneel has quit IRC | 22:41 | |
| *** mriedem has quit IRC | 22:41 | |
| clarkb | jeblair: are you cool with rally jobs if the project ptl +1's them? | 22:41 |
| clarkb | jeblair: or would you like to hold off more on that | 22:41 |
| *** gokrokve has joined #openstack-infra | 22:43 | |
| *** jamielennox|away is now known as jamielennox | 22:43 | |
| *** UtahDave has quit IRC | 22:44 | |
| *** dripton_ has joined #openstack-infra | 22:45 | |
| *** Hefeweiz1n has joined #openstack-infra | 22:47 | |
| *** che-arne_afk has joined #openstack-infra | 22:47 | |
| *** gokrokve has quit IRC | 22:47 | |
| *** Bobba has joined #openstack-infra | 22:47 | |
| *** Bobba is now known as Guest12451 | 22:48 | |
| clarkb | Alex_Gaynor: where are we on 91799? | 22:49 |
| *** michchap_ has quit IRC | 22:49 | |
| *** eharney has quit IRC | 22:49 | |
| *** krotscheck has quit IRC | 22:49 | |
| *** ianw has quit IRC | 22:49 | |
| *** lakshmiS has quit IRC | 22:49 | |
| *** rwsu has quit IRC | 22:49 | |
| *** cody-somerville has quit IRC | 22:49 | |
| *** isviridov has quit IRC | 22:49 | |
| Alex_Gaynor | clarkb: Waiting for the new images to get rolled with the updated package from the PPA; the issue has been fixed there (and the latest pip also includes a workaround) | 22:49 |
| *** ianw has joined #openstack-infra | 22:49 | |
| *** aconrad has joined #openstack-infra | 22:49 | |
| Alex_Gaynor | clarkb: https://review.openstack.org/#/c/94373/ can be approved whenever that happens I believe | 22:50 |
| clarkb | Alex_Gaynor: ok, how new would images need to be to have the PPA? | 22:50 |
| clarkb | I can check and see if we are updated yet | 22:50 |
| *** morgabra_ is now known as morgabra | 22:50 | |
| Alex_Gaynor | clarkb: I think the PPA update dyesterday or the day before | 22:50 |
| *** morgabra has quit IRC | 22:50 | |
| *** morgabra has joined #openstack-infra | 22:50 | |
| Alex_Gaynor | 32 hours ago it looks like | 22:50 |
| clarkb | looks like a couple more images need kicking, I will dothat now | 22:51 |
| *** mburned_` has joined #openstack-infra | 22:52 | |
| *** mburned_` has joined #openstack-infra | 22:52 | |
| Alex_Gaynor | Awesome, thanks! | 22:52 |
| clarkb | hpcloud az2 and rax iad are the two regions. images are building in both now | 22:52 |
| notmyname | clarkb: what else is needed to get the object-specs set up? https://review.openstack.org/#/c/94140/ | 22:53 |
| *** otherwiseguy has quit IRC | 22:53 | |
| jeblair | clarkb: fine with rally | 22:54 |
| jeblair | notmyname: more core reviewers? :) | 22:54 |
| *** mburned_out has quit IRC | 22:54 | |
| *** Guest67947 has quit IRC | 22:54 | |
| *** phschwartz_ has quit IRC | 22:54 | |
| *** matrohon has quit IRC | 22:54 | |
| *** phschwartz has joined #openstack-infra | 22:54 | |
| clarkb | notmyname: I am a bit confused, that change adds a job but doesn't use it | 22:55 |
| *** jlk has quit IRC | 22:55 | |
| clarkb | looks like anteaya was questioning that too? | 22:55 |
| *** grantbow has quit IRC | 22:55 | |
| *** Hefeweizen has quit IRC | 22:55 | |
| openstackgerrit | A change was merged to openstack-infra/jenkins-job-builder: add overview info to pypi https://review.openstack.org/94434 | 22:55 |
| *** med_ has quit IRC | 22:55 | |
| *** grantbow has joined #openstack-infra | 22:55 | |
| *** grantbow has quit IRC | 22:55 | |
| *** grantbow has joined #openstack-infra | 22:55 | |
| notmyname | clarkb: what job? gate-{name}-docs? | 22:55 |
| *** jlk has joined #openstack-infra | 22:56 | |
| *** jlk has quit IRC | 22:56 | |
| *** jlk has joined #openstack-infra | 22:56 | |
| clarkb | notmyname: ya that one | 22:56 |
| *** krotscheck has joined #openstack-infra | 22:56 | |
| *** anteaya has joined #openstack-infra | 22:56 | |
| krotscheck | We’ve got another UI iteration on the storyboard dashboard: http://invis.io/D3UN5LSU | 22:56 |
| notmyname | clarkb: ah ok. so if we have a noop job that's used, then that line and the one above it should simply be removed? | 22:57 |
| clarkb | notmyname: yup, otherwise you create a job on all the jenkins masters and never use it | 22:57 |
| *** che-arne has quit IRC | 22:57 | |
| clarkb | so not broken, just confusing | 22:57 |
| notmyname | clarkb: ok. the intention for now is to have no jobs and noop gate | 22:58 |
| clarkb | notmyname: ok | 22:58 |
| clarkb | I will leave a commetn about that | 22:58 |
| notmyname | clarkb: better yet push a commit over it? ;-) | 22:58 |
| clarkb | well then you lose a potential +2 | 22:59 |
| *** dripton has quit IRC | 22:59 | |
| clarkb | depending on how correct we want to be about review | 22:59 |
| clarkb | notmyname: also quick question, you want to use the existing swift core group for spec approvals? | 22:59 |
| notmyname | clarkb: yes. actually I feel very strongly about that. same group | 23:00 |
| clarkb | I an push a patchset shortly if you confirm that is the desire. | 23:00 |
| openstackgerrit | A change was merged to openstack-infra/config: Add check-rally-dsvm-cinder job https://review.openstack.org/90402 | 23:00 |
| clarkb | ok | 23:00 |
| notmyname | clarkb: I can :-) | 23:00 |
| *** otherwiseguy has joined #openstack-infra | 23:00 | |
| jeblair | notmyname: please use the docs job | 23:01 |
| clarkb | oh /me holds up | 23:01 |
| jeblair | i think that should be a minimum requirement for specs repos | 23:01 |
| notmyname | jeblair: ok. what is the docs job? the specs repo will have a bunch of text files in it. what docs will there be? | 23:01 |
| Alex_Gaynor | clarkb: let me know if there's anything else you need / I can do on this | 23:02 |
| *** dangers is now known as dangers_away | 23:02 | |
| jeblair | notmyname: there's a cookiecutter-specs repo that will set up a sphinx based doc repo, that way we can consistently publish the specs for all the projects | 23:02 |
| clarkb | Alex_Gaynor: I don't think there is, just need to get these image builds going, they can be flaky | 23:02 |
| *** virmitio has quit IRC | 23:02 | |
| jeblair | notmyname: http://git.openstack.org/cgit/openstack-dev/specs-cookiecutter/tree/ | 23:02 |
| notmyname | jeblair: ok, that looks harmless enough. just as long as we can have our own template if needed. or dir structure under the specs repo | 23:04 |
| jeblair | notmyname: yeah, totally free form there. basically, it'll make our lives a lot easier if we can just use the usual docs/publish jobs as entry points. my interesst doesn't extend beyond that. | 23:05 |
| notmyname | jeblair: then our interests are non-overlapping for the time being :-) | 23:05 |
| notmyname | clarkb: so it seems the jobs need to have some docs thing. but I don't know how to spell that in zuul-talk | 23:06 |
| clarkb | notmyname: I can update the change to do that instead | 23:06 |
| notmyname | thanks | 23:06 |
| clarkb | jeblair: any chance we can get jesusaurus' change in today? | 23:06 |
| clarkb | the one to reorg jenkins? | 23:06 |
| *** mrodden has quit IRC | 23:06 | |
| notmyname | clarkb: as far as "proper" review procedure, I'll leave that to y'all. it's your repo, so you get to choose what you do :-) | 23:07 |
| notmyname | clarkb: jeblair: but let me know when it's set up and I'll push the initial repo (mirrored from cookiecutter) | 23:07 |
| *** jhesketh_ has joined #openstack-infra | 23:08 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/config: Add object-specs repository https://review.openstack.org/94140 | 23:08 |
| *** jhesketh_ is now known as jhesketh | 23:08 | |
| jeblair | clarkb: i would be fine with that, but i haven't had time to review stuff today, and i have to run out for a bit right now, so couldn't act on a revert request from jesusaurus. | 23:08 |
| * jhesketh returns | 23:08 | |
| clarkb | jeblair: ok, maybe we can plan for tomorrow | 23:09 |
| *** nosnos has joined #openstack-infra | 23:10 | |
| openstackgerrit | Matthieu Baerts proposed a change to openstack-infra/git-review: Topic: do not use '(detached' when detached https://review.openstack.org/90466 | 23:11 |
| *** lakshmiS has joined #openstack-infra | 23:12 | |
| *** cody-somerville has joined #openstack-infra | 23:12 | |
| *** rwsu has joined #openstack-infra | 23:12 | |
| *** michchap_ has joined #openstack-infra | 23:13 | |
| *** eharney has joined #openstack-infra | 23:13 | |
| *** isviridov has joined #openstack-infra | 23:13 | |
| SpamapS | fungi: sorry if you've said this 3 ways already.. given the flavor change on tripleo's CI .. we just need to wait for jobs to finish on the old flavor, and then new images will be built and smaller flavors (but more instances) will start running, right? | 23:14 |
| *** dangers_` has joined #openstack-infra | 23:14 | |
| *** mwagner_lap has joined #openstack-infra | 23:14 | |
| *** KurtMartin has joined #openstack-infra | 23:14 | |
| *** dangers_away has quit IRC | 23:14 | |
| *** kmartin has quit IRC | 23:14 | |
| *** homeless_ has joined #openstack-infra | 23:15 | |
| *** markwash has quit IRC | 23:15 | |
| *** homeless has quit IRC | 23:15 | |
| *** thedodd has quit IRC | 23:16 | |
| openstackgerrit | Chris Hoge proposed a change to openstack-infra/config: Adds stackforge/puppet-openstack-specs project https://review.openstack.org/94743 | 23:18 |
| *** terryw has joined #openstack-infra | 23:18 | |
| *** homeless_ has quit IRC | 23:19 | |
| mordred | SpamapS: yes | 23:19 |
| *** homeless has joined #openstack-infra | 23:19 | |
| hogepodge | here: we’re stealing good ideas from the core projects and moving our specification reviews to Gerrit. ^^ | 23:19 |
| clarkb | oh I need to send the draft chagnes announcement /me does that | 23:20 |
| hogepodge | If anyone from core here wants to take a poke at it (clarkb and fungi always seems to be the most reliably pokable) | 23:20 |
| SpamapS | mordred: mmk | 23:20 |
| SpamapS | mordred: I will stop my OCD refresh of hypervisor-stats then | 23:21 |
| *** otherwiseguy has quit IRC | 23:21 | |
| mordred | SpamapS: keep it up | 23:21 |
| mordred | SpamapS: it makes your cloud run betterer | 23:21 |
| SpamapS | | POST http://localhost:35357/v2.0/tokens | 0.252480983734 | | 23:22 |
| SpamapS | | GET https://ci-overcloud.tripleo.org:13774/v2/625f9c5cf9624ffaa68e679c1c9a85ee/os-hypervisors/statistics | 0.0898640155792 | | 23:22 |
| SpamapS | seems legit | 23:22 |
| *** andreaf has quit IRC | 23:22 | |
| *** terryw has quit IRC | 23:25 | |
| clarkb | hogepodge: we are still trying t ocatch up on post summit things | 23:25 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Re-add fingerprint for 1285833 https://review.openstack.org/94744 | 23:26 |
| clarkb | my review queue is a little over 100 deep right now | 23:26 |
| clarkb | and that is after spedning a good amount of time today and yesterday reviewing things | 23:26 |
| *** david-lyle has quit IRC | 23:26 | |
| *** zns has quit IRC | 23:26 | |
| hogepodge | thanks clarkb, no problem | 23:28 |
| hogepodge | It will give us time to nail down the exact language of our policies and template anyway. | 23:28 |
| hogepodge | Nobody gets a vacation post summit. :-) | 23:29 |
| *** yamahata has quit IRC | 23:29 | |
| openstackgerrit | Adam Gandelman proposed a change to openstack-infra/elastic-recheck: Add a fingerprint for bug 1321494 https://review.openstack.org/94746 | 23:29 |
| uvirtbot | Launchpad bug 1321494 in ironic "NodeLocked causing random test failures" [High,Triaged] https://launchpad.net/bugs/1321494 | 23:29 |
| clarkb | jeblair: note next time zuul is restarted it will pick up swift configuration again | 23:30 |
| *** nati_ueno has quit IRC | 23:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: add gerritbot configs for new oslo libs https://review.openstack.org/92617 | 23:30 |
| *** eglynn has quit IRC | 23:30 | |
| openstackgerrit | A change was merged to openstack-infra/config: Add puppet-storyboard as a unique project https://review.openstack.org/92231 | 23:31 |
| *** bookwar has quit IRC | 23:32 | |
| *** moted has quit IRC | 23:32 | |
| *** bookwar has joined #openstack-infra | 23:32 | |
| *** harlowja has quit IRC | 23:33 | |
| *** harlowja_ has joined #openstack-infra | 23:33 | |
| *** moted has joined #openstack-infra | 23:34 | |
| *** lttrl has joined #openstack-infra | 23:35 | |
| openstackgerrit | Chris Hoge proposed a change to openstack-infra/config: Adds stackforge/puppet-openstack-specs project https://review.openstack.org/94743 | 23:35 |
| *** james_li has quit IRC | 23:35 | |
| *** lxsli has joined #openstack-infra | 23:36 | |
| *** john3213 has joined #openstack-infra | 23:36 | |
| clarkb | jogo: awake yet? whats with the large ops drop from 175 to 100? | 23:37 |
| *** james_li has joined #openstack-infra | 23:37 | |
| openstackgerrit | Chris Hoge proposed a change to openstack-infra/config: Update puppet-openstack gerritbot configuration https://review.openstack.org/94748 | 23:37 |
| clarkb | jogo: has my +2 but curious if yo udebugging got anywhere | 23:38 |
| *** unicell has joined #openstack-infra | 23:38 | |
| *** chuckC has joined #openstack-infra | 23:39 | |
| jogo | clarkb: morning | 23:40 |
| jogo | clarkb: so still catching up from the summit but this is where things were left off: | 23:40 |
| jogo | * 175 didn't work with ceilometer. If 100 works with ceilometer then I think we should go with that | 23:41 |
| jogo | * Haven't tested neutron out yet with 175 so that is a wild card | 23:41 |
| *** james_li has quit IRC | 23:41 | |
| *** john3213 has left #openstack-infra | 23:41 | |
| *** morganfainberg_Z is now known as morganfainberg | 23:42 | |
| jhesketh | if anybody has time to review this patch chain to zuul's status page that'd be great: https://review.openstack.org/#/c/91316/ | 23:43 |
| jhesketh | (that's the tail) | 23:43 |
| jhesketh | this is blocking having zuul's status overlay on gerrit | 23:43 |
| *** gokrokve has joined #openstack-infra | 23:44 | |
| openstackgerrit | A change was merged to openstack-infra/config: Fetch content from groups-static-pages repository https://review.openstack.org/92664 | 23:48 |
| jeblair | dhellmann: can you weigh in on 92910 | 23:48 |
| *** gokrokve has quit IRC | 23:49 | |
| *** openstackgerrit has quit IRC | 23:49 | |
| *** nati_ueno has joined #openstack-infra | 23:50 | |
| *** openstackgerrit has joined #openstack-infra | 23:51 | |
| jeblair | fungi: you are being paged to 93108 | 23:51 |
| *** dstanek is now known as dstanek_zzz | 23:52 | |
| SpamapS | hm, no nodes running check-tripleo-overcloud-precise ... | 23:52 |
| *** chuckC has quit IRC | 23:52 | |
| jeblair | jogo: if 93127 is just an experiment, why don't you propose a WIP devstack-gate change that forces the value you want to test? | 23:53 |
| *** rfolco has joined #openstack-infra | 23:53 | |
| jeblair | jogo: (i'm concerned about dropping that value on a production job in case it lets things get worse) | 23:53 |
| jogo | well large-ops-testing isn't a production job | 23:53 |
| jeblair | oh that was -testing... | 23:54 |
| jogo | jeblair: but I am not opposed to doing this as a WIP patch | 23:54 |
| *** dstanek_zzz is now known as dstanek | 23:54 | |
| SpamapS | ah wait I see one running | 23:54 |
| *** mburned_` is now known as mburned | 23:54 | |
| jeblair | jogo: i was momentarily confused since i had just reviewed the neutron-testing patch; for some reason that made me think this was the normal job | 23:54 |
| jogo | the patch I really need is the neutron one | 23:55 |
| jeblair | jogo: they have both been approved | 23:55 |
| SpamapS | but zuul status not showing it.. https://jenkins04.openstack.org/job/check-tripleo-overcloud-precise/642/console .. https://review.openstack.org/#/c/94523/ .. I don't see 94523 as running check-tripleo-overcloud-precise | 23:56 |
| *** mburned is now known as mburned_out | 23:56 | |
| jeblair | dtroyer, sdague: can you look at 93161 ? | 23:57 |
| jogo | jeblair: thanks | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!