Monday, 2013-08-26

*** alexpilotti has quit IRC00:21
*** michchap has joined #openstack-infra00:24
*** changbl has joined #openstack-infra00:25
*** sdake_ has joined #openstack-infra00:28
clarkbI am going to start puppet on git.o.o now00:35
jeblairclarkb: ++00:35
jeblairclarkb: i got close but not quite there on nodepool; i'll finish that early in the week.00:36
clarkbok00:36
clarkbI plan on working towards a new git.o.o early in the week00:36
clarkbthen hopefully back to backups afterwards00:38
clarkbhaproxy did restart00:38
clarkball done. git.o.o seems happy00:39
jeblairgood time to do it then.  :)00:39
jeblairthe ramp-up seems to just be starting.00:39
clarkbyup00:40
clarkbI thought I had more tiem but realized the australians were working00:40
*** michchap has quit IRC00:41
*** michchap has joined #openstack-infra00:41
jeblairclarkb: my plan for zuul this week is to get some merger improvents in place, along with a fix to the graceful restart bug.00:42
clarkbg-g-p takes under a minute now on the py26 jobs I have checked. so much better00:42
clarkbjeblair: merger improvements to support zuul merging code instead of gerrit?00:43
jeblairclarkb: not that yet --00:43
jeblairclarkb: we lost the git merge option that we used to have in ggp that causes cgit to use approximately the same strategy that gerrit uses00:43
jeblairclarkb: so remember how markmc was complaining about how every few patches in his series, one would refuse to be merged by gerrit?00:44
clarkbyeah00:44
clarkbgoing to try and catch that early if possible?00:44
jeblairclarkb: apparently when i moved the merge testing from ggp to zuul, i missed that option00:44
jeblairclarkb: so zuul has been doing plain 'git merge'.00:44
clarkbjeblair: https://review.openstack.org/#/c/42785/ if that could get reviewed early in the week that would be awesome00:44
jeblairclarkb: so this should make zuul's merging closer to gerrit and kick things out earlier (resulting in fewer resets)00:45
jeblairclarkb: secondarily, i have been looking into making the merger more efficient, but atm, a nova change takes a full minute to be enqueued.00:46
jeblairclarkb: both of those should help with our load, but they are also not super-urgent, so that's why i want to try to get them in some time this week, but it's not a rush.00:46
clarkbI see00:46
clarkbmaking merges more efficient is a fun problem00:47
jeblairi think we somehow ended up doing 'git remote update' twice for every change.00:47
jeblairlots of opportunity for improvement here.  :)00:47
*** sdake_ has quit IRC00:48
*** dina_belova has joined #openstack-infra00:49
jeblairclarkb: i reviewed that change.  i suggested to mrmartin that we go ahead and use a cloud db from the start for the user groups server00:50
jeblairclarkb: basically because it looked like we were about ready to start doing backups of trove dbs (at least, we should be ready by the time we need it)00:51
clarkbyes, I think so00:51
clarkbwe will just need to feed in credentials instead of using the defaults file00:51
openstackgerritA change was merged to openstack-infra/config: Make mysql backup crons quiet.  https://review.openstack.org/4278500:52
*** dims has joined #openstack-infra00:53
*** dina_belova has quit IRC00:54
openstackgerritA change was merged to openstack-infra/gear: fix project dependencies  https://review.openstack.org/4219400:59
jheskethHowdy. I'm having trouble with git-review. Primarily I'm trying to submit a change that depends on an already existing change but git-review insists on submitting them both and fails because the dependency has no new changes01:05
jheskethany ideas?01:05
clarkbare you rebasing so that dependency gets a new sha?01:06
jheskethno01:07
jheskethalthough I could try that01:08
clarkbif you don't have a new sha it shouldn't try pushing the dependency01:08
openstackgerritA change was merged to openstack-infra/devstack-gate: Conditionally run the slow heat tests  https://review.openstack.org/4350001:08
jeblairyou may have ended up with a new sha but no actual change01:08
jeblairyou might try rebasing on the original sha01:09
jeblair(sorry have to run now)01:09
jheskethmy sha matches gerrit's01:09
clarkbjhesketh: can you paste the output of git log -2 and git review?01:11
*** michchap_ has joined #openstack-infra01:11
jheskethclarkb: would you like that from my branch containing the problematic patchset or the one that depends on it?01:11
clarkbassuming you are talking to review.o.o01:11
jheskethclarkb: yes01:11
clarkbjhesketh: I would like it from your branch01:12
clarkbjhesketh: sorry01:12
clarkbjhesketh: I want it from whichever commit you are trying to push01:12
clarkbthe -2 should give me your commit and the dependency01:12
*** jamiem has quit IRC01:12
jheskethclarkb: http://paste.openstack.org/show/45120/01:13
jheskethI'm tempted to work around it by making a cosmetic change to generate a new sha101:13
*** jamiem has joined #openstack-infra01:14
*** jamiem has quit IRC01:14
*** michchap has quit IRC01:14
clarkbweird01:17
jheskethindeed01:18
jheskethclarkb: do you know how git-review tracks sha's with gerrit?01:19
jheskethis it creating refs somewhere?01:19
clarkbit does a remote update I think01:19
clarkbjhesketh: it will do a rebase for you it is possible it is doing that and it is causing problems (it will only do that if you conflict with the target branch)01:20
clarkbjhesketh: give git review the -v argument and see if it does anything funny?01:20
jheskethperhaps but I rebased myself and it's uptodate01:20
jheskethI've also tried submitting with -R01:20
jheskethverbose doesn't show anything unusual01:21
jheskethI think I might just work around it for now01:22
jheskeththat seemed to work01:24
jheskethI think I got into the mess somehow by trying to submit a new patchset for an abandoned change (without restoring it first) which had dependencies01:27
clarkbpushing to an abandoned change won't work. that may have been the trouble01:28
jheskethyeah so the abandoned one failed but the dependency went up broken somehow01:29
*** Prince_Africanus has joined #openstack-infra01:33
*** yaguang has joined #openstack-infra01:36
openstackgerritA change was merged to openstack-infra/devstack-gate: Skip devstack/exercises by default  https://review.openstack.org/4208201:40
*** Prince_Africanus has quit IRC01:41
*** sjing has joined #openstack-infra01:43
*** dkehn has quit IRC01:46
*** dina_belova has joined #openstack-infra01:50
*** dina_belova has quit IRC01:55
*** dkehn has joined #openstack-infra01:57
*** sdake_ has joined #openstack-infra02:19
*** sdake_ has joined #openstack-infra02:19
*** erfanian has quit IRC02:26
*** rcleere has joined #openstack-infra02:40
*** dina_belova has joined #openstack-infra02:51
*** dina_belova has quit IRC02:55
*** xchu has joined #openstack-infra03:04
*** dims has quit IRC03:05
*** UtahDave has joined #openstack-infra03:07
*** SergeyLukjanov has joined #openstack-infra03:19
*** markmcclain has quit IRC03:43
*** dina_belova has joined #openstack-infra03:51
*** dina_belova has quit IRC03:55
*** vogxn has joined #openstack-infra03:58
*** afazekas has joined #openstack-infra04:10
Alex_GaynorSo, am I correct in thinking from here through September 4th, the pace of commits is going to continue to rise?04:18
*** dguitarbite has joined #openstack-infra04:18
clarkbAlex_Gaynor: definitely for the gate queue04:19
Alex_Gaynorclarkb: has check queue peaked then?04:19
Alex_Gaynor(I assume it's a plateau)04:19
clarkbI think the major rush happened as people had to propse their changes. Now they will refine them through review so it will probably still be busy04:20
clarkbAlex_Gaynor: http://tinyurl.com/kkgtr6z04:28
clarkbI don't expect that same kind of week over week growth with next week04:28
clarkbbut will still be busy04:28
Alex_Gaynorjesus, 11k jobs at hour, that's insane04:30
clarkbAlex_Gaynor: thats per day04:30
clarkbstill crazy04:30
clarkbdid I not change the title? whoops04:30
Alex_Gaynorit's considerably less insane :) down to "only" 500 jobs an hour on average04:31
*** dina_belova has joined #openstack-infra04:52
*** dina_belova has quit IRC04:57
*** sjing has quit IRC05:03
*** sjing has joined #openstack-infra05:04
*** Dr01d has joined #openstack-infra05:05
*** nicedice_ has quit IRC05:09
*** SergeyLukjanov has quit IRC05:13
*** afazekas_ has joined #openstack-infra05:34
*** afazekas has quit IRC05:47
*** sdake_ has quit IRC05:51
*** dina_belova has joined #openstack-infra05:52
*** afazekas_ has quit IRC05:55
*** dina_belova has quit IRC05:57
*** xchu has quit IRC06:02
jheskethjeblair: ping06:10
*** mikal has quit IRC06:11
*** mikal has joined #openstack-infra06:13
*** jfriedly has quit IRC06:13
*** xchu has joined #openstack-infra06:18
*** dmakogon__ has joined #openstack-infra06:35
*** nayward has joined #openstack-infra06:36
*** jpich has joined #openstack-infra06:39
*** yolanda has joined #openstack-infra06:49
*** dina_belova has joined #openstack-infra06:53
*** dina_belova has quit IRC06:56
*** UtahDave has quit IRC07:00
*** jhesketh has quit IRC07:06
*** jhesketh has joined #openstack-infra07:06
*** hashar has joined #openstack-infra07:32
*** vogxn has quit IRC07:34
*** nayward has quit IRC07:38
*** dina_belova has joined #openstack-infra07:53
*** dina_belova has quit IRC07:59
*** fbo_away is now known as fbo08:01
*** shardy is now known as shardy_afk08:02
*** afazekas has joined #openstack-infra08:04
*** thomasbiege has joined #openstack-infra08:05
*** thomasbiege has quit IRC08:10
*** dguitarbite has quit IRC08:13
*** sjing has quit IRC08:13
*** sjing has joined #openstack-infra08:13
*** derekh has joined #openstack-infra08:25
*** thomasbiege has joined #openstack-infra08:30
*** yolanda has quit IRC08:32
*** vogxn has joined #openstack-infra08:32
*** dguitarbite has joined #openstack-infra08:42
*** xchu has quit IRC08:52
*** dina_belova has joined #openstack-infra08:54
*** dina_belova has quit IRC08:58
*** xchu has joined #openstack-infra09:04
*** DennyZhang has joined #openstack-infra09:24
*** alexpilotti has joined #openstack-infra09:25
*** ruhe has joined #openstack-infra09:29
*** mikal has quit IRC09:32
*** DennyZha` has joined #openstack-infra09:33
*** sjing has quit IRC09:34
*** DennyZhang has quit IRC09:35
*** DennyZha` has quit IRC09:36
*** jhesketh has quit IRC09:41
*** thomasbiege has quit IRC09:41
*** DennyZhang has joined #openstack-infra09:42
*** vogxn has quit IRC09:45
*** yolanda has joined #openstack-infra09:47
*** thomasbiege has joined #openstack-infra09:47
*** hashar has quit IRC09:48
*** DennyZhang has quit IRC09:50
*** DennyZhang has joined #openstack-infra09:50
*** thomasbiege has quit IRC09:51
*** dina_belova has joined #openstack-infra09:55
*** dguitarbite has quit IRC09:55
*** dina_belova has quit IRC09:59
*** afazekas has quit IRC10:03
*** vogxn has joined #openstack-infra10:03
*** afazekas has joined #openstack-infra10:04
*** fbo is now known as fbo_away10:05
*** xchu has quit IRC10:07
*** ruhe has quit IRC10:08
*** ruhe has joined #openstack-infra10:10
*** pcm_ has joined #openstack-infra10:11
*** pcm_ has quit IRC10:11
*** pcm_ has joined #openstack-infra10:11
*** ruhe has quit IRC10:12
*** yaguang has quit IRC10:22
*** mikal has joined #openstack-infra10:22
*** morganfainberg is now known as morganfainberg|a10:25
*** afazekas_ has joined #openstack-infra10:27
*** DennyZhang has quit IRC10:28
*** afazekas has quit IRC10:28
*** jhesketh has joined #openstack-infra10:34
*** vogxn has quit IRC10:35
*** dims has joined #openstack-infra10:40
*** chuckieb has joined #openstack-infra10:45
*** ruhe has joined #openstack-infra10:48
*** dina_belova has joined #openstack-infra10:55
*** dina_belova has quit IRC10:59
*** sgviking has quit IRC11:01
*** sgviking has joined #openstack-infra11:02
*** weshay has joined #openstack-infra11:07
*** weshay has quit IRC11:08
*** weshay has joined #openstack-infra11:08
*** dkranz has quit IRC11:10
*** hashar has joined #openstack-infra11:14
*** fbo_away is now known as fbo11:16
*** lcestari has joined #openstack-infra11:20
*** adalbas has quit IRC11:21
*** weshay has quit IRC11:35
*** rfolco has joined #openstack-infra11:39
*** adalbas has joined #openstack-infra11:44
*** lcestari has quit IRC11:46
*** thomasbiege has joined #openstack-infra11:47
*** mugsie has joined #openstack-infra11:49
*** lcestari has joined #openstack-infra11:50
mugsiehi, just a quick question about the mailing list11:50
mugsiehow do we register as a topic for the openstack-dev mailing list?11:50
mugsie(We were discussing it in #openstack-dns11:50
mugsie)*11:50
*** apcruz has joined #openstack-infra11:54
*** dina_belova has joined #openstack-infra11:56
*** dina_belova has quit IRC12:00
*** dprince has joined #openstack-infra12:01
*** dims has quit IRC12:03
*** dkranz has joined #openstack-infra12:13
*** dolphm has joined #openstack-infra12:13
*** jjmb has joined #openstack-infra12:20
*** boris-42 has joined #openstack-infra12:20
*** vogxn has joined #openstack-infra12:27
*** thomasbiege has quit IRC12:29
openstackgerritDolph Mathews proposed a change to openstack/requirements: pin keyring to < 2.0  https://review.openstack.org/4369012:31
openstackgerritDolph Mathews proposed a change to openstack/requirements: pin keyring to < 2.0  https://review.openstack.org/4369012:32
*** mikal has quit IRC12:37
*** saper_ is now known as saper12:38
*** mikal has joined #openstack-infra12:38
*** mikal has quit IRC12:44
*** mikal has joined #openstack-infra12:45
*** woodspa has joined #openstack-infra12:46
*** jamiem has joined #openstack-infra12:47
*** sandywalsh has joined #openstack-infra12:52
*** jamiem has quit IRC12:54
*** dina_belova has joined #openstack-infra12:56
*** beagles has quit IRC12:57
*** weshay has joined #openstack-infra12:58
*** beagles has joined #openstack-infra12:58
*** anteaya has joined #openstack-infra12:59
*** dina_belova has quit IRC12:59
*** jergerber has joined #openstack-infra13:06
*** jergerber has quit IRC13:06
*** CaptTofu has joined #openstack-infra13:07
*** jamiem has joined #openstack-infra13:07
*** jergerber has joined #openstack-infra13:09
*** odyssey4me has quit IRC13:10
*** mriedem has joined #openstack-infra13:12
anteayahi mugsie, to the best of my understanding, noone is registered as a topic in the openstack-dev mailing list. When the first post in a new thread is composed if no tags are in the subject [openstack-dev] is placed as a prefix in the subject and that is it. All other tags in a topic subject line are typed in by the originator of the first post. You will see here that both [glance] and [Glance] have been used: http://lists.op13:16
anteayaenstack.org/pipermail/openstack-dev/2013-August/subject.html13:16
anteayalink again: http://lists.openstack.org/pipermail/openstack-dev/2013-August/subject.html13:16
*** CliMz has joined #openstack-infra13:19
*** CliMz has quit IRC13:22
*** Bada has joined #openstack-infra13:22
*** cppcabrera has joined #openstack-infra13:25
*** cppcabrera has left #openstack-infra13:25
*** jog0-away is now known as jog013:30
*** dims has joined #openstack-infra13:31
*** zul has quit IRC13:32
*** yolanda has quit IRC13:32
anteayawe had very heavy thunderstorms lasting a surprising long time twice this weekend, forecast says chances of more today - if I disappear quickly, that would be me saving my electronics13:35
*** arezadr has joined #openstack-infra13:35
*** blamar has joined #openstack-infra13:35
*** CaptTofu has quit IRC13:35
*** thomasbiege has joined #openstack-infra13:36
*** CaptTofu has joined #openstack-infra13:36
*** thomasbiege has quit IRC13:36
*** thomasbiege has joined #openstack-infra13:37
*** dina_belova has joined #openstack-infra13:37
mugsieanteaya: cheers!13:38
*** SergeyLukjanov has joined #openstack-infra13:38
anteayamugsie: :D13:38
*** yolanda has joined #openstack-infra13:39
*** CaptTofu has quit IRC13:40
*** ruhe has quit IRC13:40
*** jhesketh has quit IRC13:41
*** hashar has quit IRC13:42
*** hashar has joined #openstack-infra13:42
*** burt has joined #openstack-infra13:42
*** thomasbiege has quit IRC13:42
*** dkehn is now known as dkehn_away13:43
*** datsun180b has joined #openstack-infra13:44
*** yolanda has quit IRC13:46
*** hashar has quit IRC13:51
openstackgerritDavid Kranz proposed a change to openstack-infra/config: Add slow heat job  https://review.openstack.org/4351113:51
*** HenryG_ has quit IRC13:52
*** HenryG has joined #openstack-infra13:52
jog0Random question: what do people think of moving all the git checks in hacking into a post-commit git hook?13:52
*** ruhe has joined #openstack-infra13:53
jog0https://bugs.launchpad.net/hacking/+bug/121692513:55
uvirtbotLaunchpad bug 1216925 in hacking "Pull out git checks to work as post-commit git hook" [Undecided,New]13:55
*** dina_belova has quit IRC13:56
*** rustlebee is now known as russellb13:56
*** dkliban has joined #openstack-infra13:57
anteayamorning jog013:57
anteayaI have no opinion13:57
*** michchap_ has quit IRC13:57
*** michchap has joined #openstack-infra13:58
jog0anteaya: morning, from the east coast13:58
anteayame too13:58
anteaya:D13:58
dkranzmtreinish: Never mind. I did not realize the zuul change was also in the config project. Submitted new patch.13:59
*** dims has quit IRC14:00
*** rcleere has quit IRC14:02
mtreinishdkranz: did you want to add the heat slow job to heat also?14:04
*** yolanda has joined #openstack-infra14:04
dkranzmtreinish: Yes, once it is working.14:10
dkranzmtreinish: But that gives us an asymetrical gate.14:10
dkranzmtreinish: What if some change in nova breaks this job?14:10
dkranzmtreinish: Perhaps it needs to go on all projects that are related in any way, or just all projects?14:11
mtreinishdkranz: we should put it on all the projects one we switch it voting14:11
dkranzmtreinish: We had a long discussion about this at the last summit and came down on the "gate everything" side I think.14:12
dkranzmtreinish: OK, yes I agree.14:12
dkranzmtreinish: I wanted to see it work on one project before putting on others.14:12
mtreinishdkranz: I'd just put it nonvoting everywhere up front, then it makes the switch to voting patch a one liner14:13
mtreinishalso, why don't you just test it locally?14:14
dkranzmtreinish: By "working" I mean flowing through the whole gate process. I don't know how to test that locally.14:15
openstackgerritMark McLoughlin proposed a change to openstack/requirements: Allow use of oslo.messaging 1.2.0a11  https://review.openstack.org/4371214:18
*** yolanda has quit IRC14:19
dkranzmkoderer: ping14:21
mkodererdkranz: pong14:21
dkranzmkoderer: What's with line 36 of https://review.openstack.org/#/c/43444/3/tempest/stress/run_stress.py,unified14:22
mkodererhhhh14:22
dkranzmkerrin: leftover from testing?14:23
mkodererthat was for debugging14:23
mkodereryes :(14:23
dkranzmkerrin: OK14:23
mkodererdkranz: give me a -1 I will fix it14:23
dkranzmkerrin: Sorry, meant mkoderer14:23
*** hashar has joined #openstack-infra14:24
*** rnirmal has joined #openstack-infra14:25
mkodererdkranz: that's why reviews are always a good idea :)14:25
*** zul has joined #openstack-infra14:28
*** michchap has quit IRC14:32
anteayalook at that, the gate is empty - wasn't expecting to see that today14:32
*** jpeeler has quit IRC14:33
*** jpeeler has joined #openstack-infra14:33
*** rcleere has joined #openstack-infra14:33
*** ryanpetrello has joined #openstack-infra14:34
openstackgerritA change was merged to openstack-dev/hacking: Update requirements  https://review.openstack.org/4345714:37
*** jpich has quit IRC14:39
*** jpeeler has quit IRC14:40
*** jpeeler has joined #openstack-infra14:41
*** jpich has joined #openstack-infra14:41
*** prad has joined #openstack-infra14:47
*** dmakogon__ has quit IRC14:47
*** prad has quit IRC14:48
*** prad has joined #openstack-infra14:48
*** dkehn_away is now known as dkehn14:51
*** jpich has quit IRC14:53
*** jpich has joined #openstack-infra14:55
*** krtaylor has joined #openstack-infra14:56
*** dina_belova has joined #openstack-infra14:57
*** boris-42 has quit IRC14:59
*** dguitarbite has joined #openstack-infra15:00
*** CaptTofu has joined #openstack-infra15:00
*** dina_belova has quit IRC15:01
*** michchap has joined #openstack-infra15:02
*** dina_belova has joined #openstack-infra15:07
*** michchap has quit IRC15:10
*** dina_belova has quit IRC15:11
*** nati_ueno has joined #openstack-infra15:12
*** nati_ueno has quit IRC15:14
*** nati_ueno has joined #openstack-infra15:15
*** Dr01d has quit IRC15:21
*** Dr01d has joined #openstack-infra15:22
*** afazekas_ has quit IRC15:22
*** markmcclain has joined #openstack-infra15:24
*** CaptTofu has quit IRC15:28
*** CaptTofu has joined #openstack-infra15:29
*** Bada has quit IRC15:30
*** CaptTofu_ has joined #openstack-infra15:31
*** CaptTofu has quit IRC15:31
*** marun has joined #openstack-infra15:32
*** jpich has quit IRC15:34
*** _TheDodd_ has joined #openstack-infra15:36
*** yaguang has joined #openstack-infra15:38
*** rnirmal has quit IRC15:39
*** rnirmal has joined #openstack-infra15:41
jog0I am unable to create branches for hacking15:42
jog0any superadmins around that can help15:42
anteayapleia2: when you are around - does the current config with cgit use an .htaccess file? I can't seem to find one, but it is possible it exists and I have overlooked it.15:44
*** UtahDave has joined #openstack-infra15:46
*** branen_ has joined #openstack-infra15:46
*** jswarren has quit IRC15:48
jog0jeblair: ^15:49
*** zul has quit IRC15:50
jeblairjog0: a post commit hook is a fine idea, but local installation of commit hooks is voluntary.  of course if you also run it in the style-check gate job, it's not a big deal if they don't run it locally.15:52
jeblairjog0: what kind of branch do you need?15:52
jeblairmugsie: at the bottom of http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev15:54
jeblairmugsie: you will see "OpenStack-dev list run by thierry at openstack.org, stefano at openstack.org"15:54
openstackgerritJason Meridth proposed a change to openstack-dev/hacking: Adds ability to ignore hacking validations with noqa  https://review.openstack.org/4171315:54
*** yaguang has quit IRC15:55
jeblairmugsie: so you can email them directly or email openstack-dev-owner@lists.openstack.org with your question about topics15:55
jog0jeblair: a jackign 0.7.x branch15:55
branen_Hi guys, I'm trying to get this global requirement approved. https://review.openstack.org/#/c/42991/  Any help would be greatly appreciated.15:56
jog0jeblair: hacking*15:56
jog0jeblair: starting from the 0.7.0 tag (sha1 6e1404cb221190645c7ef66e8d77e3402cba01eb )15:56
jeblairjog0: why does hacking need branches?15:56
jog0jeblair: and as for the post commit, cool15:56
jog0jeblair: because we cannot cut new 0.7.x releases from master as there is a new check already15:57
jog0jeblair: and to make sure we can bump dependencies, do bug fixes without forcing a new check on people15:57
jog0too often15:57
jeblairjog0: ok, i think i understand the branch model15:59
jog0jeblair: cool, thanks15:59
jog0last time I tried to create a branch (0.5.x) I hit similar issues about not being able to create one15:59
jeblairjog0: is hacking part of oslo?16:02
*** nati_ueno_2 has joined #openstack-infra16:03
jog0jeblair: yes16:03
jog0jeblair: but I am not16:03
jog0but I am part of hacking16:03
*** thomasbiege has joined #openstack-infra16:03
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Grant branch-creation to hacking-ptl  https://review.openstack.org/4373116:04
jeblairjog0: ^16:04
jog0that group doesn't exist16:06
*** nati_ueno has quit IRC16:06
jeblairjog0: nope, but it will when the change merges.  we can add you and oslo-ptl then.16:06
jog0jeblair: ahh cool16:06
jog0so I am partof hacking-core so why couldn't i create a branch before?16:07
jeblairjog0: the top line adds create branch perms; the other part of the change is just cleanup16:07
*** thomasbiege has quit IRC16:07
jeblairjog0: we like to keep tag permissions limited to a small group of people because it's easy to mess up pushing tags16:07
jeblairjog0: and even if core==ptl today, that sort of thing should limit who you decide to give code review permission to16:08
*** dina_belova has joined #openstack-infra16:08
jeblair"shouldn't limit"!16:08
jog0jeblair: ahh makes sense to me16:08
jog0thanks16:08
openstackgerritA change was merged to openstack-infra/nodepool: Fix typo in node check method  https://review.openstack.org/4359516:12
openstackgerritA change was merged to openstack-infra/nodepool: Fix image delete logic  https://review.openstack.org/4362316:12
*** hashar has quit IRC16:12
*** dina_belova has quit IRC16:13
openstackgerritA change was merged to openstack-infra/config: Fix join/leave sounds in the conference  https://review.openstack.org/4239616:13
openstackgerritA change was merged to openstack-infra/config: Make oslo.messaging gating on devstack/tempest  https://review.openstack.org/4261316:15
anteayapleia2: nevermind I found the httpd.conf.erb file: https://git.openstack.org/cgit/openstack-infra/config/tree/modules/cgit/templates/httpd.conf.erb16:16
*** mikal has quit IRC16:17
*** zul has joined #openstack-infra16:18
*** mikal has joined #openstack-infra16:18
*** nicedice_ has joined #openstack-infra16:19
*** ruhe has quit IRC16:20
*** cppcabrera has joined #openstack-infra16:24
annegentleclarkb: or jeblair or others who know how translations work? Around?16:25
annegentleThis patch definitely fails to do what we wanted: https://review.openstack.org/#/c/39846/16:26
*** ruhe has joined #openstack-infra16:26
annegentleI have a hunch it's because the Post job is against the wrong repo16:26
annegentlebut now that I'm working on a patch, I'm not sure what pom.xml to point to.16:26
annegentleI'll try a patch and see what you all think.16:26
*** ruhe has quit IRC16:26
cppcabreraHey, guys. I was looking to get eavesdrop support in for the openstack-marconi channel. I have a patch pending that would address that: https://review.openstack.org/#/c/42956/16:27
cppcabreraCan I get approval/eyes on that? :)16:27
*** reed has joined #openstack-infra16:27
jeblairannegentle: if it's not urgent, yeah, why don't you propose a patch and lets look at it when clarkb is around16:27
annegentlejeblair: sure sounds good.16:27
*** nati_ueno_2 has quit IRC16:29
openstackgerritAnne Gentle proposed a change to openstack-infra/config: Update Japanese Ops Guide build job.  https://review.openstack.org/4373416:29
jeblaircppcabrera: we'll get to that soon, we're working on a backlog with very few infra-core people around right now.  but one pre-req for that is to give me founder access to the channel, at least until https://bugs.launchpad.net/openstack-ci/+bug/1190296 is fixed16:29
uvirtbotLaunchpad bug 1190296 in openstack-ci "IRC bot to manage official channel settings" [Medium,In progress]16:29
*** nati_ueno has joined #openstack-infra16:29
annegentle^^ that's my first attempt. For sure the zuul layout has to change. But what I'm unsure of is whether there's a "generated" pom.xml file.16:29
uvirtbotannegentle: Error: "^" is not a valid command.16:29
jeblaircppcabrera: and you may want to add it to the channel list in the wiki if it isn't already16:29
jeblaircppcabrera: /msg chanserv access #openstack-marconi add jeblair +AFRfiorstv16:30
cppcabrerathanks, jeblair. Sounds like things are really busy atm!16:30
*** dolphm has quit IRC16:30
jeblaircppcabrera: yep16:30
pleia2anteaya: anything specific you were looking for that I can help with?16:32
*** moted has quit IRC16:32
*** moted has joined #openstack-infra16:33
anteayapleia2: well the blog post talked about using an .htaccess file to redirect requests for .rst files to .html output - I am trying to read how one does redirects and can pipe .html to output using the httpd.conf16:33
pleia2anteaya: gotcha, an .htaccess file is just a local mini version of the httpd.conf, designed so that web people without control over httpd.conf could still make *some* changes to apache via the .htaccess, so yeah it's all doable in httpd.conf and the syntax will be similar16:35
* pleia2 has a look at the blog post again16:36
jeblairjd__: i left a 0-score comment on https://review.openstack.org/#/c/42642/  let me know if you have questions16:37
anteayapleia2: grand, yes that was what I was understanding - thanks for your confirmation about .htaccess and httpd.conf16:37
cppcabrerajeblair: Just made sure - openstack-marconi is on the channel list on the OS IRC wiki. I'll check with kgriffs later regarding getting you founder access on the channel. Thanks again!16:38
anteayaI am just going over the httpd.conf file now - trying to find the right section to introduce a setting16:38
jd__jeblair: :-(16:38
pleia2anteaya: so you can do something like <Directory /var/lib/git> and put the rewrite rules within that </Directory>16:38
jd__jeblair: I'll build patches tomorrow for caching then!16:39
anteayapleia2: what section - global? virtual? I'm guessing virtual16:39
*** markmcclain has left #openstack-infra16:39
jeblairjd__: cool, it shouldn't be too hard.  it's just a little scary because we don't have a good way of testing proposed patches to those.  but i just had an idea of how we could do that (but probably won't be able to implement it for a while)16:39
* pleia2 has another look at the httpd.conf too :)16:40
*** boris-42 has joined #openstack-infra16:41
pleia2anteaya: actually, we probably want to put it in http://git.openstack.org/cgit/openstack-infra/config/tree/modules/cgit/templates/git.vhost.erb16:41
jd__jeblair: don't worry I'll as usual when working on infra: http://i.imgur.com/GNmZpEH.gif16:41
jeblairjd__: :)16:42
jd__:-)16:42
*** dina_belova has joined #openstack-infra16:42
*** pcrews has quit IRC16:42
*** SergeyLukjanov has quit IRC16:43
anteayapleia2: great, now that I have a proposed home file I can cobble together a rudimentary beginning and we can flesh it out16:43
*** dina_belova has quit IRC16:43
*** ^d has joined #openstack-infra16:43
*** ^d has joined #openstack-infra16:43
pleia2anteaya: sounds good16:44
anteayapleia2: let me know how that .html file looks on cgit so we can figure out if using rst2html even gives us what we want: https://review.openstack.org/#/c/42961/16:45
*** SergeyLukjanov has joined #openstack-infra16:45
pleia2anteaya: ah yes, I'll toss this up in a repo in my test instance16:46
anteayathanks16:47
*** fbo is now known as fbo_away16:47
cppcabrerajeblair: kgriffs just let me know you've got founder access on openstack-marconi. :)16:49
clarkbmorning16:51
jeblairclarkb: good morning16:51
*** SergeyLukjanov has quit IRC16:52
anteayamorning clarkb16:54
* anteaya makes some lunch16:54
*** derekh has quit IRC16:57
pleia2anteaya: just tossed it in a test repo here http://15.185.127.146/cgit/testing/project/tree/17:01
anteayapleia2: that doesn't render .html for me17:04
pleia2anteaya: nope, it does if you click on "plain" though17:05
anteayathat renders an .html file in plain text when I view it17:05
anteayaah17:05
* anteaya clicks17:05
anteayacan we have the plain setting as default for .html files?17:05
anteayahow many people are going to know to click on plain?17:05
*** sdake has joined #openstack-infra17:07
pleia2great question, I don't know :)17:07
anteayahaving to spend time telling people to click on plain - where on earth could we put that as a direction that is discoverable? - kind of defeats the purpose of the exercise17:07
pleia2yep17:07
anteayaI wonder if time is better spend working on getting sphnix generated docs up and linking wiki articles to sphnix docs17:07
pleia2I haven't looked into the magic they use to make the About pages show up17:07
anteayaseems longer, but the result is more effective17:08
*** Ryan_Lane has joined #openstack-infra17:08
anteayathere is a setting re: about pages I have found17:08
anteayalet's see if I can find it again17:08
anteayahttps://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/git/cgitrc#n4417:09
anteayalooks like it might be as simple as uncommenting out line #45 and provided said file17:09
*** sarob_ has joined #openstack-infra17:09
*** leifmadsen has quit IRC17:11
pleia2anteaya: yeah, so that enables it for the about page on each repo, I'm just not sure how that works internally and how/if we could make it work for other .html files that are floating around17:11
pleia2(and eventually, .rst)17:11
anteayayeah17:11
anteayawould be awesome if we could17:11
*** dina_belova has joined #openstack-infra17:12
openstackgerritA change was merged to openstack-infra/config: Grant branch-creation to hacking-ptl  https://review.openstack.org/4373117:12
anteayabut since sphnix docs are part of the long game anyway, I will redirect to sphnix docs17:12
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use "sort -V" for tags when building docs  https://review.openstack.org/4373917:12
anteayaI have confidence I should get an acceptable result with the direction17:12
*** cppcabrera has left #openstack-infra17:12
anteayabut yeah, if we could figure out how cgit renders the about.html files, perhaps we can get it to render the other types too - across the repos17:13
anteayapleia2: any objection to me abandoning 42961 now?17:13
anteayaI don't think we need it anymore17:13
anteayado you?17:13
*** mriedem has quit IRC17:14
pleia2anteaya: go for it17:14
anteayathanks17:14
* anteaya abandons17:14
*** Dr01d has quit IRC17:15
clarkbpleia2: jeblair and I would like to replace git.o.o with an 8GB node (down from 30GB) that only runs haproxy and doesn't do the git + cgit stuff17:16
*** sandywalsh has quit IRC17:16
clarkbpleia2: I can whip that up but was wondering if you had an opinion on whether or not we should keep haproxy stuff in the cgit module or just lift it into the openstack_projects manifest for git.openstack.org17:16
anteayaokay, now lunch17:17
openstackgerritA change was merged to openstack-infra/config: Use "sort -V" for tags when building docs  https://review.openstack.org/4373917:18
pleia2clarkb: I'm thinking we put cgit back to being basic module for cgit and split out haproxy to openstack_projects17:20
jeblairannegentle: can you weigh in on https://review.openstack.org/#/c/42988/217:20
jeblairpleia2, clarkb: sgtm17:20
*** SergeyLukjanov has joined #openstack-infra17:22
*** mriedem has joined #openstack-infra17:22
*** dolphm has joined #openstack-infra17:28
openstackgerritA change was merged to openstack-infra/jenkins-job-builder: Adding AnsiColor Support  https://review.openstack.org/4312117:30
*** sarob_ has quit IRC17:30
*** sarob has joined #openstack-infra17:31
*** vogxn has quit IRC17:31
*** jfriedly has joined #openstack-infra17:31
*** ruhe has joined #openstack-infra17:32
*** sandywalsh has joined #openstack-infra17:32
*** sarob_ has joined #openstack-infra17:34
*** rfolco has quit IRC17:34
*** pcrews has joined #openstack-infra17:34
*** sarob has quit IRC17:35
openstackgerritA change was merged to openstack-infra/config: Add the ability to filter Zuul Status on multiple (comma-delimited) terms.  https://review.openstack.org/4238217:37
*** sandywalsh has quit IRC17:38
jeblairclarkb: i don't understand your comment in https://review.openstack.org/#/c/43596/17:41
*** dolphm has left #openstack-infra17:41
jeblairclarkb: (i've been using the new graph for a couple of days locally)17:41
openstackgerritClark Boylan proposed a change to openstack-infra/config: Split git.o.o haproxy out of cgit module.  https://review.openstack.org/4374317:42
dkranzjeblair: Thanks for the review of https://review.openstack.org/#/c/43511/ but you said there was an (unspecifiede) race.17:42
clarkbjeblair: pleia2 ^ something like that? it needs testing, thankfully we need a new node node anyways so that fits right in17:42
dkranzjeblair: Can you say what needs to be fixed? This is intended to go voting as soon as it works.17:42
jeblairdkranz: i was responding to the comment about including both jjb and zuul in the same change17:42
clarkbjeblair: I took the new string there and pasted it into the old jobs launched graph and got an image that said no data17:42
clarkbjeblair: let me try again as I may have typoed something17:43
jeblairdkranz: nothing needs to be fixed17:43
openstackgerritA change was merged to openstack-infra/config: Disable salt master and minions globally  https://review.openstack.org/4303017:43
dkranzjeblair: Ah. So it would have been better to do it as two patches?17:43
jeblairdkranz: zuul might get the new config before jjb, for a few minutes, in which case the job will be marked as LOST, but it won't affect anything because it is non-voting.17:43
dkranzjeblair: OK, thanks.17:43
jeblairdkranz: it's fine as one patch because it's non-voting; if it were voting, 2 patches would be better17:43
jeblairclarkb: also compare to http://tinyurl.com/kg93m7r17:44
*** dprince has quit IRC17:45
clarkbjeblair: yup it works and matches your tinyurl17:45
*** vipul is now known as vipul-away17:46
jeblairclarkb: whew17:46
*** burt has quit IRC17:46
*** sarob_ has quit IRC17:47
*** sarob has joined #openstack-infra17:47
*** sdake_ has joined #openstack-infra17:47
*** sdake_ has joined #openstack-infra17:47
pleia2clarkb: right now, what is the difference between git.o.o server and the others? (trying to figure out exactly what we ended up with in our current load balancing scheme)17:50
*** sandywalsh has joined #openstack-infra17:51
clarkbpleia2: right now git.o.o runs haproxy, apache, git-daemon and is synced to by gerrit. So the only difference is it layers haproxy over the top of everything17:52
pleia2clarkb: so are the other servers running haproxy at all?17:53
clarkbpleia2: they are not17:54
pleia2or just straight up git/http(s) on the default ports17:54
clarkbthey run on git/http(s) on the non default ports17:55
pleia2ah ok, got it17:55
pleia2thanks17:55
*** dims has joined #openstack-infra17:57
*** weshay has quit IRC18:01
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use stacked area test node graph  https://review.openstack.org/4356518:01
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Add sparklines to pipeline headers  https://review.openstack.org/4356718:01
openstackgerritJames E. Blair proposed a change to openstack-infra/config: Use more efficient jobs/hour graph  https://review.openstack.org/4359618:01
*** sarob has quit IRC18:02
*** sarob has joined #openstack-infra18:03
*** morganfainberg|a has quit IRC18:04
*** vipul-away is now known as vipul18:04
clarkbpleia2: I debated switching back to the default ports, but I think that should happen in a subsequent change if we do that18:05
clarkbpleia2: fewer moving parts to juggle that way18:05
*** morganfainberg|a has joined #openstack-infra18:05
*** morganfainberg|a is now known as morganfainberg18:06
*** sarob has quit IRC18:07
pleia2clarkb: given how flexible we are in the config with default/non-default ports (nice job!), I think it's ok to keep them non-default, but I agree18:07
*** ruhe has quit IRC18:10
*** sdake_ has quit IRC18:10
*** melwitt has joined #openstack-infra18:13
reedfungi, are you back on dry land?18:14
clarkbreed: I think he is on the boat until Friday18:15
reedI need help with solving this issue: https://bugs.launchpad.net/bugs/117294418:15
uvirtbotLaunchpad bug 1172944 in openstack-community "Misattributed commit" [High,New]18:15
reedclarkb, lucky man...18:15
openstackgerritTomas Sedovic proposed a change to openstack-infra/config: Add Jenkins jobs for Tuskar  https://review.openstack.org/4346118:16
reedthat issue is about matching identities across different systems18:17
reedlaunchpad to gerrit, especially18:17
clarkbyeah, I think there are two ways around this. The first is what fungi describes which requires elevated gerrit sql privs. Or just use the info in the git commit18:18
reedclarkb, explain the git commit thing please, how would that work?18:19
clarkbreed: each git commit has an author with a name and email address18:19
clarkbreed: you could use that name and email address in an attempt to lookup people in launchpad18:19
clarkbreed: the most reliable thing would be getting that user's openid from gerrit though18:19
clarkbjeblair: ^18:19
reedwhat if launchpad user has no email address?18:20
clarkbreed: fall back on name search?18:20
clarkbthe second option I describe isn't perfect18:20
reedmaybe we should just drop the connection to launchpad ... it's not that useful, we only use it for bugs and blueprints anyway18:21
*** dkehn has quit IRC18:21
reedi mean, if we find a match with the email then fine, if not... too bad18:21
jeblairreed: yeah, we've tried really hard to make that connection as loose as possible because things were always breaking when it was tighter.18:22
jeblairreed: as you'd probably expect, admin access isn't really an option.  we could consider a periodic data dump (that's a bit of work and kind of fragile.  also, note that it would likely take many hours, if not days, to actually run)18:23
reedok, thanks so far, I'll suggest JP to loosen up that connection and see how that goes18:23
jeblair(those lp openid lookups are really slow)18:23
reedjeblair, I knew it :)18:23
clarkbpleia2: thank you for the review. I missed those things18:24
reedand we're not going to rely on LP for OpenID for much longer hopefully anyway :)18:24
*** dkehn has joined #openstack-infra18:24
clarkbjeblair: did you catch that a couple zuul changes failed in the gate with python26 failures?18:29
*** zul has quit IRC18:31
annegentlejeblair: yep I'm good with the OpenSUSE addition for install.18:34
jeblairclarkb: yes, i suspect it's because we're now actually running 26 tests and there may be race conditions in the tests18:35
openstackgerritMarton Kiss proposed a change to openstack-infra/config: Add groups-dev.openstack.org node  https://review.openstack.org/4260818:36
jeblairclarkb: it's about #4 on my list of zuul things to look at18:36
jeblairclarkb: see -dev18:39
*** dprince has joined #openstack-infra18:39
clarkbjeblair: mordred did that once. It should prevent the release jobs from running correctly but nothing will go sideways18:48
jeblairclarkb: i believe it was released successfully18:49
jeblairclarkb: it's very wrong though.  git describe (and likely pbr) will not identify the correct version18:49
clarkbhmm I wonder if that commit is available to gerrit somewhere then18:50
clarkbjeblair: when mordred did it I believe the fix was to push an update to the tag18:50
jeblairclarkb: it's in the repo; it got pushed with the tag18:51
jeblairthat's why pushing tags is so dangerous18:51
clarkbjeblair: ya ok. When mordred did it the local commit was local only18:51
jeblairand we only let people who, ideally, know what they are doing do it.18:51
jeblairclarkb: i'm not sure how mordred could have pushed a tag without a commit18:51
jeblairclarkb: it does not make sense for a repo to have a tag that points to a commit that is not in the repo18:51
jeblairclarkb: maybe you're remembering something else?  perhaps pushing an unsigned tag or something?  or maybe the job failed for some other reason, since 'corrected'?18:52
clarkbjeblair: I think pushing the tag pushes the commit18:52
jeblairclarkb: i agree18:52
jeblairclarkb: which is why markwash's error 'worked'.  he pushed a commit that isn't in the repo.  i imagine he could have pushed an entire tree if he wanted.18:53
clarkbjeblair: so mordred's local commit ended up in gerrit with the tag. But g-g-p in the jobs couldn't checkout the tag because by default we don't fetch all tags18:53
jeblair(that wasn't in the repo)18:53
clarkbjeblair: we only fetch things reachalbe by the branches18:53
clarkbwhich mordred's change was not18:53
jeblairclarkb: that sounds likely; i agree.  i wonder why it could check this out.18:54
jeblairclarkb: perhaps because it was cloned18:54
jeblairclarkb: that was the case for the docs job18:54
clarkbThat may do it18:54
jeblairwe really need code review for tags18:55
jeblairpeople are just not able to follow directions18:55
*** mrmartin has joined #openstack-infra18:55
*** xBsd has joined #openstack-infra18:55
*** rfolco has joined #openstack-infra18:56
*** burt has joined #openstack-infra18:58
clarkb++19:00
*** vipul is now known as vipul-away19:02
*** pentameter has joined #openstack-infra19:04
*** hashar has joined #openstack-infra19:05
openstackgerritClark Boylan proposed a change to openstack-infra/config: Split git.o.o haproxy out of cgit module.  https://review.openstack.org/4374319:06
clarkbpleia2: ^ that addresses your comments. Tahnks19:06
jog0how can I create a new devstack test, where the only change is the config?19:07
jog0the nova.conf file taht is19:07
clarkbjog0: nova.conf is updated by devstack. If the update you want to make is already supported by devstack then all you need is an update to devstack-gate to toggle that change and a new job that toggles devstack-gate19:09
clarkbjog0: if devstack does not already support that toggle I think you need to update devstack as well19:09
*** nati_ueno has quit IRC19:09
jog0clarkb: right, how do I add a new job?19:09
*** nati_ueno has joined #openstack-infra19:09
clarkbjog0: edit openstack-infra/config/modules/openstack_project/files/jenkins_job_builder/config/devstack.yaml19:10
clarkbjog0: http://ci.openstack.org/jjb.html describes what goes into those files and has links into more in depth info19:10
clarkbjog0: then update openstack-infra/config/modules/openstack_project/files/zuul/layout.yaml to tell zuul to run the job. http://ci.openstack.org/zuul.html has info on that and links into more info19:11
*** sarob has joined #openstack-infra19:11
jog0clarkb: awesome thanks19:11
jog0I have two new checks i want to run for nova19:11
jog0one is should already be green (spin up 25 vms at once)19:12
jog0the other will fail at first19:12
clarkbdo they need to be new jenkins jobs or can they just run as part of existing tests?19:12
clarkbI suppose something that isn't passing can't run as part of the gate but the passings bits could19:12
*** sarob has quit IRC19:12
*** sarob has joined #openstack-infra19:13
*** vipul-away is now known as vipul19:13
jog0clarkb: they both need seperate nova.conf values so AFAIK we need different jobs19:13
jog0anyway will do this right after lunch19:14
*** nati_ueno has quit IRC19:14
clarkbjog0: you can't restart nova after tempest runs?19:14
clarkbwith the new config?19:14
clarkbyou might find the post test hook in devstack-gate useful as well19:15
jog0clarkb: that could work, be back in 3019:15
*** sarob has quit IRC19:17
*** nati_ueno has joined #openstack-infra19:19
openstackgerritAnita Kuno proposed a change to openstack-infra/config: Creating/adding the openstack/governance repository  https://review.openstack.org/4300219:19
*** sdake_ has joined #openstack-infra19:22
*** sdake_ has quit IRC19:22
*** sdake_ has joined #openstack-infra19:22
*** nati_ueno has quit IRC19:23
*** nati_ueno has joined #openstack-infra19:24
*** fbo_away is now known as fbo19:25
*** nati_ueno_2 has joined #openstack-infra19:26
*** nati_ueno has quit IRC19:27
*** sarob has joined #openstack-infra19:27
*** mrodden has quit IRC19:28
*** mrodden has joined #openstack-infra19:28
*** nati_ueno_2 has quit IRC19:31
*** nati_ueno has joined #openstack-infra19:31
*** nati_ueno has quit IRC19:35
*** morganfainberg has left #openstack-infra19:38
*** SergeyLukjanov has quit IRC19:38
*** morganfainberg has joined #openstack-infra19:38
*** afazekas has joined #openstack-infra19:39
*** Dr01d has joined #openstack-infra19:39
jog0clarkb: I think at least for now keeping the test seperate will make things easier19:45
*** nati_ueno has joined #openstack-infra19:46
*** nati_ueno has quit IRC19:47
*** nati_ueno has joined #openstack-infra19:48
*** redbeard2 has joined #openstack-infra19:51
anteayaI let the salt guys know about: https://review.openstack.org/#/c/43030/ which disables salt masters and minions - they know we are busy and are trying to see if they can get a better sense of the issue so they can offer some support19:51
anteayajeblair or clarkb if either of you could run `salt --versions-report` I think they would be grateful to access the output - if we can still run it if they are disabled19:52
*** nati_ueno has quit IRC19:52
jeblairanteaya: ok; we're not really frustrated or anything, it's mostly that we don't have a lot of time to be good users (if i were to ask for help or report a problem, i'd at least want to be able to articulate the problem better and have thought about it a little myself)19:53
jeblairanteaya: on the master?19:53
anteayaredbeard2: on the master?19:54
anteayajeblair: yes that was my sense of it and what I tried to convey19:54
jeblairhttp://paste.openstack.org/show/45161/19:54
anteayathanks19:54
redbeard2anteaya and jeblair: both master and minion is ideal19:55
redbeard2oh, 0.15.3, that's a little old19:55
anteayaredbeard2: what is current?19:55
redbeard2i think we're on 0.16.3 or 0.16.4 right now19:55
redbeard2UtahDave ^19:56
UtahDavejeblair: Yeah, 0.16.3  is the latest. and we're getting ready to cut 0.16.419:57
openstackgerritAntoine Musso proposed a change to openstack-infra/jenkins-job-builder: Revert "Adding AnsiColor Support"  https://review.openstack.org/4377119:58
*** svarnau has joined #openstack-infra19:59
UtahDavejeblair: and zmq 2.2.0 is basically just broken19:59
*** harlowja has quit IRC19:59
*** harlowja has joined #openstack-infra20:00
*** xBsd has quit IRC20:04
*** wenlock has joined #openstack-infra20:06
*** vipul is now known as vipul-away20:06
clarkbjog0: the reason I ask is we do have an upper bound on the number of slaves available at any one time and he seem to hit that periodically. So two new tests per nova patchset takes us from 7 d-g slaves to 920:09
jog0clarkb: ahh20:11
jog0can we just run these periodically on trunk for now?20:11
clarkbwe can do that20:11
jog0well one of them is tempest.scenario.test_large_ops.TestLargeOpsScenario.test_large_ops_scenario20:11
jog0in tempest20:11
jog0and that can be periodic20:11
jog0at least for now20:11
openstackgerritStefano Maffulli proposed a change to openstack-infra/config: Added list for Track Chairs Committee for Summit Nov 2013  https://review.openstack.org/4377320:11
jog0and the other I want to add is a compute rpcapi  backwards compat test20:12
jeblairclarkb: when would we make them gating?20:14
clarkbjeblair: if one is a tempest test it can be made gating in the same way other tempest tests are added to the mix20:15
clarkbinless that is the one that requires a different nova config20:15
jeblairclarkb: you may have misunderstood my question; by what criteria would one determine when it should be gating as opposed to periodic?20:16
jeblairclarkb: and yes, that is the one that requires a different nova config20:16
clarkbjeblair: when it passes regularly20:16
clarkbjeblair: but I think we need to get away from the practice of non voting check tests for random things that don't pass20:17
jeblairclarkb: then silent or the experimental pipeline would be good choices?20:17
clarkbjeblair: I like mordred's experimental queue idea20:17
jeblairwhat happened to that change?20:17
clarkbjeblair: good question. I know it needed work, maybe it got auto abandoned?20:17
jeblairclarkb: i think i mostly agree with you; non-voting should be the last thing that happens before it becomes voting; check shouldn't end up being a test graveyard -- but neither should periodic20:18
clarkbjeblair: I agree20:18
clarkblet me find the experimental queue change and resurrect it if necessary20:19
jeblairclarkb: here it is https://review.openstack.org/#/c/41954/20:19
clarkbjeblair: I restored it20:20
jeblairclarkb: so let's say expiremental is for "i'm trying to get this thing working", silent is for "i think it works but it needs lots of testing to get data", check/gate non-voting is for "this is about to gate"20:21
clarkb++20:21
anteayaUtahDave tells me zmq 2.2.0 had memory leak issues20:25
jog0clarkb: one more question, i don't want to run all of tempest just 'tox -eall tempest.scenario.test_large_ops'20:26
jog0and change a the tempest config20:26
anteayaafter feature freeze is over, he says there will be more salt support in channel20:26
anteayaI suggested after feature freeze20:26
anteayas/had/has20:27
*** mrmartin has quit IRC20:27
anteayahe said once zmq is on 3.2.x most of those issues go away20:27
clarkbjog0: you may need to write your own test running hook for d-g then. As by default it will do the regular tempest stuff20:27
clarkbjog0: in devstack.yaml the swift functional and pbr install tests do this20:27
openstackgerritJames E. Blair proposed a change to openstack-infra/zuul: Fetch specific refs on ref-updated events  https://review.openstack.org/4377520:28
jog0clarkb: thanks20:28
jeblairjog0: unless there's a tox env in tempest for it?20:28
jeblair(i'd lean toward putting that in devstack-gate so it's self20:29
jeblair-gating)20:29
jog0'tox -eall tempest.scenario.test_large_ops'20:30
* anteaya goes for a walk20:30
jeblairand so it fits in with the rest of the devstack-gate tempest tests20:30
*** nati_ueno has joined #openstack-infra20:30
jog0jeblair: it needs a seperate nova.conf (fake virt driver)20:30
*** weshay has joined #openstack-infra20:31
jeblairjog0: i know, i'm just saying that if this is going to run alongside the other 6 or so devstack tempest tests, it would be nice if it didn't have a completely different way of being run20:31
jeblairjog0: so you can add the 'tox -eall tempest.scenario.test_large_ops' to devstack-gate20:31
hasharI am on vacations for 3 weeks and you guys managed to colorize the success,failure and lost messages reported in Gerrit… crazy :)20:31
jog0jeblair: it can be done with a 'unstack; change localrc; stack; tempest tox20:31
clarkbhashar: jeblair has been really busy. I don't know how he does it20:31
jeblairjog0: that's not what i'm trying to say20:32
jeblairjog0: i'm saying that clarkb suggested that you launch the test in a way that is completely different than the way the other devstack-gate tempest tests are launched, and i'm not certain it's appropriate20:32
jeblairjog0: the heat slow tests are probably a good pattern20:33
dkranzclarkb: Any chance you could approve https://review.openstack.org/#/c/43511/ to get the heat jobs going?20:33
jeblairspeaking of :)20:33
jog0I was looking at cells code as a reference but will checkou t the heat code20:34
jog0jeblair: thanks20:34
clarkbdkranz: jeblair: 43511 reopens the experimental vs silent vs non voting discussion20:34
jeblairjog0: cells works too; they both do the same thing20:34
clarkbI don't think that test belongs there unless we expect to make it voting in the next couple weeks20:35
hasharclarkb: I suspect some javascript voodoo :D20:35
*** nati_ueno has quit IRC20:35
jeblairclarkb: if we had an alternative in place, i might agree with you.20:35
clarkbjeblair: I can rebase and cleanup mordred's change if you want to try and get experimental pipeline in soon20:36
jeblairclarkb: keep in mind, heat is just being added to tempest20:36
jeblairhashar: actually, just a gerrit commentlink and some css20:36
jeblairhashar: (and a teeny change to zuul)20:36
hasharI thought Gerrit was escaping html20:37
jog0dumb question20:37
jeblairclarkb: i'd be happy to merge that if you get it in shape.20:37
jog0for https://review.openstack.org/#/c/43511/2/modules/openstack_project/files/jenkins_job_builder/config/devstack-gate.yaml20:37
*** vipul-away is now known as vipul20:37
clarkbjeblair: great. Let me see what I can do here really quick20:37
dkranzclarkb: The intent is to turn it voting as soon as it is demonstrated to be working.20:37
jeblairhashar: it does! and the commentlink actually matches the generated html.  it's crazy.  :)20:37
jog0how does that run heat slow tests?20:38
clarkbdkranz: I understand, but that is the intent with the other non passing jobs as well20:38
jeblairjog0: corresponding change to devstack-gate repo20:38
jog0jeblair: ohh20:38
dkranzjog0: And see the heat section in https://github.com/openstack/tempest/blob/master/tox.ini20:38
hasharjeblair: smart trick :-]20:39
dkranzclarkb: The other non-passing jobs are there because no one has yet fixed them20:39
dkranzclarkb: I think we need to push these new heat tests through the infrastructure before letting them block merges. No?20:40
dkranzclarkb: that's what we did with parallel testr as well.20:41
clarkbdkranz: we do, but we shouldn't be running them on every patchset by default20:41
dkranzclarkb: I'm not sure what you mean20:41
clarkbdkranz: mordred has proposed an experimental queue that would make it easy to push tests through on specific patchsets with manual nudging. So that you could run the tests on patchsets that address test issues20:41
clarkbdkranz: then when the problems are sorted out we can promote you to non voting then gating20:42
clarkbdkranz: the trouble with non voting by default is we have in the past run a ton of tests that do nothing for us20:42
dkranzclarkb: So there are a bunch of heat tests waiting on this patch to run. What do you suggest we do?20:43
clarkbdkranz: I am fixing mordred's change to add an experimental queue so that you can run your tests there20:43
dkranzclarkb: OK. Is this something general that you will announce to the dev list?20:44
dkranzmtreinish:   ^^^^20:45
*** weshay has quit IRC20:45
*** Dr01d has quit IRC20:45
*** emagana has joined #openstack-infra20:48
marunlifeless: Hoping you can take a look at this review that adds postmortem support to neutron's tests: https://review.openstack.org/#/c/43776/20:49
marunlifeless: Please let me know if you think this is the way to go or whether the functionality should be added to testtools directly.20:50
*** amotoki has joined #openstack-infra20:55
*** nicedice__ has joined #openstack-infra20:56
*** nati_ueno has joined #openstack-infra20:57
openstackgerritClark Boylan proposed a change to openstack-infra/config: Move non-voting jobs to an experimental queue  https://review.openstack.org/4195420:58
clarkbdkranz: jeblair ^20:58
clarkbdkranz: I don't know that it needs an announcement to -dev but it can definitely be documented somewhere20:58
*** nicedice_ has quit IRC21:00
openstackgerritClark Boylan proposed a change to openstack-infra/config: Move non-voting jobs to an experimental queue  https://review.openstack.org/4195421:01
dkranzclarkb: So how do you invoke a job in this experimental queue?21:02
*** rfolco has quit IRC21:02
clarkbdkranz: you leave a comment with "check experimental" as the only content in the review21:02
clarkbdkranz: that string can be modified and is on line 83 of the layout.yaml file in my change21:03
dkranzclarkb: OK, works for me.21:03
*** boris-42 has quit IRC21:03
jeblairclarkb: can you add the instructions to the pipeline description?21:04
clarkbjeblair: yes21:04
*** dprince has quit IRC21:04
*** apcruz has quit IRC21:04
openstackgerritClark Boylan proposed a change to openstack-infra/config: Move non-voting jobs to an experimental queue  https://review.openstack.org/4195421:05
dkranzjeblair, clarkb : I will resubmit the heat patch with experimental as soon as 41954 goes in.21:05
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Add support for large_ops tempest test  https://review.openstack.org/4377921:05
jog0first of my patches for a new tests: https://review.openstack.org/#/c/43778/21:06
*** pcrews has quit IRC21:06
dkranzjeblair: BTW, I could not figure out from the docs how to decide whether some job should go in check, gate, or both. Is this explained somewhere?21:07
dkranzjeblair: I couldn't tell from the examples in the yaml file either.21:07
jeblairdkranz: they should be 1:1, especially now with experimental21:07
dkranzjeblair: Well, that's easy :)21:08
openstackgerritA change was merged to openstack-infra/config: Move non-voting jobs to an experimental queue  https://review.openstack.org/4195421:08
jeblairdkranz: pretty much at this point the only jobs that should be in check but not gate are informative ones, like we have a job that tells us if a change alters the jenkins configuration21:08
jeblairdkranz: but that's not something we gate on (sometimes it is okay to alter the jenkins config!)21:08
openstackgerritKhai Do proposed a change to openstack-infra/gear: Update gear docs to include gearman server daemon  https://review.openstack.org/4378021:09
jeblairdkranz: there should never be a gate job that isn't also in check21:09
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add gate-tempest-devstack-vm-large-ops-full job  https://review.openstack.org/4378121:09
*** pcrews has joined #openstack-infra21:09
jog0jeblair: here are the patches https://review.openstack.org/#/q/status:open++branch:master+topic:bug/1199433,n,z21:10
jog0clarkb: ^21:10
reedjeblair, pleia2: modified https://review.openstack.org/#/c/42998/2, when you have time please give me feedback21:12
clarkbjog0: d-g change lgtm. The openstack-infra/config change is mostly there (see comments). I can give you a +1 on the devstack change but do not have core there21:15
jog0clarkb: thanks21:15
clarkbjeblair: I am going to noop apply https://review.openstack.org/#/c/43743/ to the existing git nodes to make sure that I didn't miss anything too horrible and if that is mostly clean next step is spinning up a new git.o.o with the development env21:17
jeblairclarkb: roger; i'll try to review it soon21:20
*** dina_belova has quit IRC21:20
clarkbjeblair: https://bugs.launchpad.net/openstack-ci/+bug/121709721:21
uvirtbotLaunchpad bug 1217097 in openstack-ci "LaunchPad Core developer lists are out of date" [Undecided,Confirmed]21:21
*** dina_belova has joined #openstack-infra21:21
ttxclarkb: that's a dupe21:21
ttxfor 116027721:22
clarkbttx: thanks. I will update21:22
ttxbug 116027721:22
uvirtbotLaunchpad bug 1160277 in openstack-ci "Groups have similar names in LP and gerrit but are no longer synced" [Medium,In progress] https://launchpad.net/bugs/116027721:22
clarkbdone21:22
ttxI should just jump and remove them, but I keep using them as subscriptoin groups for private bugs21:23
ttxwe should actually replace them with a ~nova-dev-interested-in-security group21:23
clarkbttx: Maybe update the group descriptions?21:23
clarkbor that21:24
clarkbI think that the only bug here is one of documentation21:24
ttxbecause subscribing PROJ-core is not really efficient. Lost in the noise21:24
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add gate-tempest-devstack-vm-large-ops-full job  https://review.openstack.org/4378121:24
ttxclarkb: last time I looke we could not remove because quantum-core had a ML and LP is picky about removing groups with MLs21:25
ttxbut now the ML should be removed and fogotten (takes a few days/weeks to remove)21:25
ttxso will fix it when I come back21:25
*** dina_belova has quit IRC21:25
jog0clarkb: ^ I have a hunch I still borked something there21:26
mtreinishdkranz: one sec reading backlog21:26
*** nati_ueno has quit IRC21:28
*** nati_ueno has joined #openstack-infra21:28
*** dkranz has quit IRC21:28
clarkbjog0: that is really close. Just one little thing noted inline21:29
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add gate-tempest-devstack-vm-large-ops-full job  https://review.openstack.org/4378121:30
jog0clarkb: thanks21:30
clarkbjeblair: the git change is relatively clean with --noop. No errors and only expected changes happen21:32
mtreinishclarkb: now that we've got an experimental queue how about we push this in: https://review.openstack.org/#/c/41797/ and drop the testr-full job too :)21:32
*** mriedem has quit IRC21:32
clarkbjeblair: is it possible to nova boot a second git.openstack.org in the same AZ?21:33
clarkbmtreinish: wfm21:34
clarkbttx: ^ do you have an opinion?21:34
ttxclarkb: missing a bit of context21:35
ttx$21:35
clarkbttx: mtreinish is proposing we run most tempest tests with testr an run them in parallel21:35
mtreinishall but neutron21:36
ttxclarkb: right, was wondering about the "experimental queue"21:36
clarkbttx: I think that is just noise for this particular discussion21:36
anteayattx if you want to discuss the goverance repo patch in tc meeting tomorrow: https://review.openstack.org/#/c/43002/421:36
clarkbttx: the experimental queue gives us a much nicer place to develop new tests with low impact against jenkins and zuul21:37
ttxclarkb: it's the last window to do it. I don't think we'll have the freedom to get creative afterwards (untli FF is done)21:37
clarkbttx: tempest testr is beyond that point and is ready to go into gating21:37
clarkbjeblair: any opinion on merging tempest testr?21:38
ttxclarkb: so if infra feels comfortable with it, I'd say go for it21:38
clarkbI am comfortable with it. Let us see what jeblair has to say21:38
clarkbjog0: one more comment. I missed it earlier sorry21:39
*** reed has quit IRC21:41
openstackgerritJoe Gordon proposed a change to openstack-infra/config: Add gate-tempest-devstack-vm-large-ops job  https://review.openstack.org/4378121:42
*** thomasbiege has joined #openstack-infra21:42
mtreinishjog0: on https://review.openstack.org/#/c/43779 why don't we just add a large ops tox job to tempest21:43
jog0mtreinish: we can21:44
*** nati_uen_ has joined #openstack-infra21:44
mtreinishjog0: ok, lets do it that way then. I like having a separate tox job for each jenkins job.21:45
jeblairclarkb, mtreinish: i defer to the tempest folks on that; i have been preoccupied and haven't kept up with the stats; and i am less likely to be called upon to help devs if it's flaky21:45
*** nati_ueno has quit IRC21:46
mtreinishalthough it does add a third patch to that series of getting it into zuul21:46
jog0mtreinish: thats why I didn't do it21:46
clarkbjeblair: that is how I feel. From the infrastructures perspective I don't think there is any reason not to now that we are mostly stable21:46
jog0but happt to do it if you want21:46
clarkbmtreinish: ^21:46
mtreinishjeblair, clarkb: I think it's as good as it's going to get right now with just me looking at things21:46
mtreinishthere are still a couple of flaky fails I've seen but there're not that common21:47
mtreinishand there is a requirments pinning that should fix the most common one21:47
mtreinishbut the flaky rate will temporarily increase a bit21:47
jeblairmtreinish: ok.  you know the drill, flaky will get us into trouble around the milestone, but if you think you can wrangle people to help fix the remaining issues, it's probably worth it.21:48
*** nati_ueno has joined #openstack-infra21:48
jeblairmtreinish: i suppose this can always be fairly easily backed out too.  :)21:48
*** nati_uen_ has quit IRC21:48
clarkbjeblair: will the launch script allow me to boot a second git.openstack.org?21:49
clarkbjeblair: or do I need to use some other name then change it over later?21:49
mtreinishjeblair: yeah it'll just be a revert. :) As soon as I push it through I'll send another note out to the list announcing the change and asking people to watch the recheck list and help with the debugging.21:49
jeblairclarkb: it will; don't run the cert gen step and use the existing cert21:49
*** redbeard2 has quit IRC21:49
clarkbjeblair: ok, doing that now21:49
*** pcm_ has quit IRC21:49
jeblairmtreinish: cool.  need anything from us, or is it just that tempest change at this point?21:49
mtreinishjeblair: it's just a tempest change to switch the tox jobs over. I guess there aren't any other tempest cores around now though21:50
jeblairclarkb: the experimental pipeline messed up the status page layout i was used to.  :(  maybe we should call it 'testing' :)21:50
mtreinishI'll just self approve :)21:50
*** nati_uen_ has joined #openstack-infra21:50
clarkbjeblair: I am fine with a name change21:50
anteayawhat is the priority for the experimental pipeline? The priority that I know about is gate, post, check - where does experimental fit?21:50
clarkbjeblair: the behavior is what is important to me :)21:51
anteayasoon to be called something else21:51
*** nati_uen_ has quit IRC21:51
clarkbanteaya: it is low21:51
clarkbanteaya: so shares resources with check21:51
anteayaafter check then?21:51
jeblairclarkb: or we could encode the order into the status page: (check, gate, everything else in alpha order)21:51
anteayaah the same as check21:51
lifelessmarun: thanks for the ping; commented in the review21:51
clarkbjeblair: either way. I defer to you as I don't know how difficult the status change would be21:51
jeblairclarkb: it's just javascript; how hard could it be?  ;)21:52
clarkbjeblair: 8GB for git.o.o?21:52
jeblairclarkb: t21:52
*** nati_uen_ has joined #openstack-infra21:52
clarkbserver is being launched21:52
*** nati_ueno has quit IRC21:53
*** thomasbiege has quit IRC21:53
mtreinishjeblair: actually if you guys could look at https://review.openstack.org/#/c/43690/ that will probably fix one of the flaky fails on the keystone cli tests21:53
clarkblookig21:54
*** lcestari has quit IRC21:55
*** nati_uen_ has quit IRC21:55
*** nati_ueno has joined #openstack-infra21:56
openstackgerritJames E. Blair proposed a change to openstack-infra/zuul: Don't reset the local repo  https://review.openstack.org/4379221:58
notmynamejeblair: clarkb: in gerrit, is the "needed by" flag not working for the base of a series of patches?21:58
jeblairnotmyname: it hides it by default in the ui; click 'dependencies' or hit 'd' to see it21:58
jeblairnotmyname: isn't that helpful? :)21:59
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Add support for large_ops tempest test  https://review.openstack.org/4377921:59
notmynamejeblair: ya. thanks. so if it's a child patch, it's shown by default, else it isn't21:59
jeblairnotmyname: yep.  no idea why they decided that was a good idea.21:59
notmynamejeblair: and also apparently I'm blind ;-)21:59
notmynamejeblair: this is actually the second thing (in about as many minutes) that22:00
notmynamejeblair: this is actually the second thing (in about as many minutes) that's been right in front of me and I haven't seen ;-)22:00
notmynamesorry for the trouble22:00
clarkbstill stuck at BUILD 0 ....22:00
jeblairnotmyname: i dunno about you, but i'm a little frazzled atm.  :)22:00
clarkbjeblair: 43690 is a simple change22:01
*** ^d has quit IRC22:02
jeblairclarkb: should there be an open bug about that so we don't lose track of it?22:02
jeblairi'm just more than usually worried about pinning something called 'keyring' and missing security updates.22:03
clarkbjeblair: probably not a bad idea as the bunch referenced is for ubuntu22:03
jeblairclarkb: i'll open one and then leave a +2 on that22:04
clarkbjeblair: at the very least we have a link to that bug, but having a distinct bug for how it affects us will help us track it22:04
*** thomasbiege has joined #openstack-infra22:05
clarkbjeblair: new git.o.o errored. trying again22:07
*** woodspa has quit IRC22:08
*** hashar has quit IRC22:09
*** thomasbiege has quit IRC22:09
*** nati_uen_ has joined #openstack-infra22:11
*** nati_uen_ has quit IRC22:11
*** portante is now known as portante|afk22:11
*** nati_uen_ has joined #openstack-infra22:12
openstackgerritJames E. Blair proposed a change to openstack-infra/zuul: Don't reset the local repo  https://review.openstack.org/4379222:13
openstackgerritJames E. Blair proposed a change to openstack-infra/zuul: Fetch specific refs on ref-updated events  https://review.openstack.org/4377522:13
marunlifeless: thank you for the review feedback.  It would be great if if pm debugging support could be added directly to testtools, but given that testtools doesn't necessarily run tests (and therefore can't be relied upon to configure debugging), would support be limited to defining the appropriate exception handler so it can be reused?22:13
jeblairclarkb: i'm hopeful that those 2 changes will do a great deal to reduce zuul's gerrit trigger backlog.22:14
*** nati_ueno has quit IRC22:14
*** burt has quit IRC22:14
*** datsun180b has quit IRC22:15
clarkbhmm pulling the salt stuff broke the launch script. I will make that a flag22:15
jeblairclarkb, jog0, mtreinish: i ran 'check experimental' on https://review.openstack.org/43791 to see it work.22:16
jeblairclarkb: oh how exciting.22:17
jeblairclarkb: we're about to test (in production) the result of having only non-voting jobs in an independent pipeline.22:18
clarkbcool22:18
jeblairclarkb: if you say so.22:18
jeblairclarkb: it's entirely possible it could wedge zuul.22:18
jeblairclarkb: i sort of expected jobs in the experimental pipeline to be voting.22:19
clarkbjeblair: oh that issue22:19
clarkbjeblair: hmm22:19
jeblairclarkb: not issue, just unknown.22:19
clarkbjeblair: I guess they can be voting as long as they don't live in two pipelines22:20
jeblairclarkb: if i had to guess, it would be: that it will work, but report overall failure regardless of the outcome.22:21
*** ryanpetrello has quit IRC22:21
openstackgerritA change was merged to openstack-infra/config: Use stacked area test node graph  https://review.openstack.org/4356522:24
openstackgerritA change was merged to openstack-infra/config: Add sparklines to pipeline headers  https://review.openstack.org/4356722:24
openstackgerritA change was merged to openstack-infra/config: Use more efficient jobs/hour graph  https://review.openstack.org/4359622:24
openstackgerritKhai Do proposed a change to openstack-infra/gear: Update gear docs to include gearman server daemon  https://review.openstack.org/4378022:25
openstackgerritClark Boylan proposed a change to openstack-infra/config: Make salt key management on node launch optional.  https://review.openstack.org/4379422:26
clarkbjeblair: ^ what do you think of that as a workaround for the salt stuff?22:26
openstackgerritA change was merged to openstack-infra/config: Fix releastestatus dependency  https://review.openstack.org/4351622:27
*** nati_ueno has joined #openstack-infra22:28
*** _TheDodd_ has quit IRC22:28
openstackgerritA change was merged to openstack-infra/config: Make salt key management on node launch optional.  https://review.openstack.org/4379422:31
*** dina_belova has joined #openstack-infra22:31
*** nati_uen_ has quit IRC22:32
clarkblaunchig new git.o.o now22:32
*** ^demon has joined #openstack-infra22:33
*** ^demon is now known as ^d22:33
*** jhesketh has joined #openstack-infra22:34
*** nati_ueno has quit IRC22:34
clarkbjeblair: https://review.openstack.org/43791 experimental tests were removed form the pipeline but I don't see a comment22:34
clarkbjeblair: to have force-message work do we need to provide a non default message or is this prossible related to the non voting?22:34
*** nati_ueno has joined #openstack-infra22:34
*** vipul has quit IRC22:35
*** vipul has joined #openstack-infra22:35
*** dina_belova has quit IRC22:36
mtreinishclarkb: do you think I should pull the testr-full job or should I wait until after the milestone? (in case we have to revert back to serial?)22:37
clarkbmtreinish: I think you can go ahead and pull it22:37
clarkbmtreinish: updating zuul configs is really easy these days22:37
mtreinishclarkb: ok, will do22:38
mtreinishI'll leave the devstack-gate and tempest bits lying around though22:38
jeblair2013-08-26 22:33:15,152 DEBUG zuul.IndependentPipelineManager: Change <Change 0x15fb5d0 43791,2> is a new version of <Change 0x7f123c57a610 43791,1>, removing <Change 0x7f123c57a610 43791,1>22:38
jeblairclarkb: ^ :)22:38
anteayanice logging22:40
jeblairclarkb: we could accept that behavior, or set that pipeline _not_ to remove changes.22:40
jeblairanteaya: thanks; "what was zuul thinking?" is a common and important enough question that it records pretty much every decision it makes.22:41
jeblairbiab22:41
anteayak22:41
clarkbjeblair: I am fine with that behaviour22:41
anteayavery friendly to read22:41
clarkbjeblair: as gerrit will not let you merge an old patchset. It is possible the old data may be useful, but typically you will want info for new patchsets22:42
clarkbmtreinish: yeah those bits are harder to remove or add back in22:42
anteayais the zuul DEBUG info publicly consumable? if so, I haven't recognized it if I have seen it22:44
clarkbanteaya: it isn't22:44
anteayathat would explain why I haven't seen it22:44
anteayaso it appears that "check experimental" puts the patch in question into the experimental pipeline _instead of_ the check pipeline, not _as well as_22:46
clarkbanteaya: I think it can do both. The behavior we saw is related to pushing a new patchset22:46
clarkbanteaya: typically new patchset == no need for old tests22:47
*** fbo is now known as fbo_away22:47
clarkbjeblair: 192.237.223.224 is the new git.o.o haproxy host22:47
anteayaclarkb: oh okay, I will continue to observe22:48
clarkbjeblair: it seems to be happy with the proposed puppet change. If you like it I think we should go ahead and lower the DNS ttl, swap dns, merge the puppet change, then restart gerrit to pick up new replication config22:48
*** redbeard2 has joined #openstack-infra22:49
anteayaclarkb: umm, but jeblair ran "check experimental" on patchset 1 of 42791, so none of the check tests has run on it yet22:49
anteayapatchset 2 is currently in the check pipeline22:50
clarkbanteaya: no, patchset 2 kicked out any tests for patchset122:50
anteayaah ha22:50
openstackgerritMatthew Treinish proposed a change to openstack-infra/config: Remove testr-full job  https://review.openstack.org/4379722:50
anteayamakes sense now22:50
clarkbanteaya: we do that because gerrit doesn't let you revert to an old patchset. So there is little value in tests belonging to an older patchset when you push a new one22:51
clarkbpleia2: you may be intrested in 192.237.223.224 as well22:51
anteayamtreinish: can we run "check experimental" on 43797?22:51
anteayaclarkb: makes sense22:51
clarkbanteaya: there are no experimental jobs for that change22:51
clarkbs/change/project/22:51
anteayaclarkb: not much point in running them then22:51
*** pcrews has quit IRC22:52
*** nati_ueno has quit IRC22:52
*** nati_ueno has joined #openstack-infra22:53
clarkbmtreinish: good announcement22:55
mtreinishclarkb: looks like the 4th time we ran tempest we hit a flaky fail: https://jenkins01.openstack.org/job/gate-tempest-devstack-vm-postgres-full/4875/console . Not the most promising start...22:55
mtreinishclarkb: thanks. Hopefully people will look at things a bit more closely now.22:56
mtreinishI know I'm guilty of just rechecking and moving on all the time22:56
clarkbmtreinish: that looks like a cli keystone test. Perhaps the requirements change is behind it?22:56
clarkbhmm no I don't see the requirements change in the list22:56
mtreinishclarkb: yeah I'm hoping the requirments change will fix it22:56
mtreinishjeblair: only +2'd no +a...22:57
clarkboh22:57
* clarkb approves22:57
mtreinishclarkb: well I'm gonna head home for the day now (it's getting kind of late here) I'll be back tomorrow morning.22:58
clarkbok, I have approved the keyring pinning. Should have a better idea of what that fixes tomorrow22:58
mtreinishok cool22:59
*** dims has quit IRC23:00
*** wenlock has quit IRC23:02
anteayalook at the pretty zuul sparklines23:03
*** dims has joined #openstack-infra23:04
clarkbjeblair: I went ahead and lowered the TTL on git.openstack.org A/AAAA records to 300 seconds23:04
openstackgerritKhai Do proposed a change to openstack-infra/gear: Update gear docs to include gearman server daemon  https://review.openstack.org/4378023:05
clarkbjeblair: also rax will have potential network outages for up to 10 minutes on wednesday according to their maintenance page23:06
openstackgerritKhai Do proposed a change to openstack-infra/gear: Update gear docs to include gearman server daemon  https://review.openstack.org/4378023:08
*** dims has quit IRC23:08
*** dolphm has joined #openstack-infra23:10
openstackgerritKhai Do proposed a change to openstack-infra/gear: Update gear docs to include gearman server daemon  https://review.openstack.org/4378023:11
*** nicedice__ has quit IRC23:12
*** nicedice_ has joined #openstack-infra23:12
*** dolphm has quit IRC23:14
*** nati_ueno has quit IRC23:15
*** pentameter has quit IRC23:17
clarkbjeblair: ping me when you get back. I think we can do the git.o.o switch fairly quickly this evening23:19
*** dims has joined #openstack-infra23:22
*** sarob has quit IRC23:30
*** sarob has joined #openstack-infra23:31
*** rnirmal has quit IRC23:31
*** dina_belova has joined #openstack-infra23:32
*** sarob has quit IRC23:35
*** pcrews has joined #openstack-infra23:36
*** sarob has joined #openstack-infra23:37
*** dina_belova has quit IRC23:37
clarkbjeblair: I am going to head home now. I am still on board with switching git.o.o if you are.23:40
jeblairclarkb: ++23:41
jeblairheh, it looks like if a change is kicked out of a pipeline, the graphite gauge for the pipeline isn't updated.23:43
*** zul has joined #openstack-infra23:47
openstackgerritJoe Gordon proposed a change to openstack-infra/devstack-gate: Add support for large_ops tempest test  https://review.openstack.org/4377923:49
*** dims has quit IRC23:53
*** dims has joined #openstack-infra23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!