Saturday, 2014-10-18

fungiyeah. or maybe i'll just read more e-mail00:00
mordredfungi: funny, I was thinking about drinking cachaça and watching the teevee00:00
*** hemna is now known as hemnass00:00
mordredI suppose the cachaça involvement makes that unamerican00:00
fungithe television more than makes up for the lack of americanness of the drink00:00
*** jergerber has quit IRC00:02
*** gyee has quit IRC00:04
*** dimsum_ has joined #openstack-infra00:06
*** patrickeast has quit IRC00:08
*** sarob has quit IRC00:08
*** rwsu has quit IRC00:14
*** mmaglana has quit IRC00:15
*** dimsum_ has quit IRC00:16
*** dimsum_ has joined #openstack-infra00:16
*** ivar-laz_ has joined #openstack-infra00:17
*** ivar-laz_ has quit IRC00:17
*** ryanpetrello has quit IRC00:17
*** timrc is now known as timrc-afk00:18
*** ryanpetrello has joined #openstack-infra00:18
*** ivar-laz_ has joined #openstack-infra00:18
*** ivar-lazzaro has quit IRC00:20
*** dimsum_ has quit IRC00:21
*** ivar-laz_ has quit IRC00:22
*** ivar-lazzaro has joined #openstack-infra00:23
*** r-daneel has joined #openstack-infra00:23
*** r-daneel has quit IRC00:29
*** mmedvede has quit IRC00:39
*** hdd has quit IRC00:44
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Flush out the db api documenation  https://review.openstack.org/12940500:45
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Reorganize the python api documentation  https://review.openstack.org/12940600:45
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add section on the basic data model  https://review.openstack.org/12940700:45
*** r-daneel has joined #openstack-infra00:46
*** dimtruck is now known as zz_dimtruck00:50
*** bdpayne has quit IRC00:54
*** hdd has joined #openstack-infra00:54
openstackgerritMatthew Treinish proposed a change to openstack-infra/project-config: Add docs jobs for subunit2sql  https://review.openstack.org/12940900:58
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Flush out the db api documenation  https://review.openstack.org/12940501:02
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Reorganize the python api documentation  https://review.openstack.org/12940601:02
openstackgerritMatthew Treinish proposed a change to openstack-infra/subunit2sql: Add section on the basic data model  https://review.openstack.org/12940701:02
lifelessmtreinish: btw it looks like you've made subunit2sql subunitv1 based01:05
lifelessmtreinish: I think it would be a really good idea to fix that asap01:06
*** ivar-lazzaro has quit IRC01:07
*** ivar-lazzaro has joined #openstack-infra01:07
*** ivar-lazzaro has quit IRC01:07
*** ivar-lazzaro has joined #openstack-infra01:08
*** ivar-lazzaro has quit IRC01:09
*** ivar-lazzaro has joined #openstack-infra01:10
jeblairlooks like https://review.openstack.org/129396 has merged and all the jjb runs have finished01:13
anteayaivar-lazzaro: check your client01:13
anteayajeblair: good01:14
ivar-lazzaroanteaya: yeah I think it should be fine now01:14
anteayaivar-lazzaro: thanks01:15
ivar-lazzaroanteaya: :)01:15
*** dimsum_ has joined #openstack-infra01:19
*** dimsum_ has quit IRC01:19
*** dimsum_ has joined #openstack-infra01:19
*** groknix has quit IRC01:21
*** hdd has quit IRC01:25
fungiyep01:28
fungiseems we're all set01:28
*** packet has joined #openstack-infra01:36
*** r-daneel has quit IRC01:36
*** packet has quit IRC01:36
*** pc_m has quit IRC01:37
mtreinishlifeless: shouldn't be, I thought I did everything using subunit v201:40
*** dimsum_ has quit IRC01:40
*** salv-orlando has quit IRC01:41
*** tomoe has joined #openstack-infra01:43
mtreinishlifeless: I might have screwed part of it up, but my intent was to make it v2 only01:45
*** cristov_mac has joined #openstack-infra01:45
*** cristov_mac has quit IRC01:45
*** tomoe has quit IRC01:45
*** cristov_mac has joined #openstack-infra01:46
*** cristov_mac has quit IRC01:46
*** cristov_mac has joined #openstack-infra01:46
*** cristov_mac has quit IRC01:46
*** cristov_mac has joined #openstack-infra01:47
*** cristov_mac has quit IRC01:47
*** cristov_mac has joined #openstack-infra01:47
*** cristov_mac has quit IRC01:48
*** cristov_mac has joined #openstack-infra01:48
*** cristov_mac has quit IRC01:48
*** cristov_mac has joined #openstack-infra01:49
*** cristov_mac has quit IRC01:49
*** cristov_mac has joined #openstack-infra01:49
*** harlowja has quit IRC01:55
*** spzala has quit IRC01:55
*** harlowja has joined #openstack-infra01:56
*** captainmorgan is now known as morganfainberg01:59
*** cody-somerville_ has joined #openstack-infra02:04
openstackgerritMatthew Treinish proposed a change to openstack-infra/project-config: Add docs jobs for subunit2sql  https://review.openstack.org/12940902:05
*** Longgeek has joined #openstack-infra02:08
*** mriedem has quit IRC02:12
*** amcrn has quit IRC02:12
*** leakypipes has quit IRC02:12
*** Longgeek has quit IRC02:12
*** koolhead17 has joined #openstack-infra02:15
*** koolhead17 has joined #openstack-infra02:15
*** achanda has joined #openstack-infra02:18
*** hdd has joined #openstack-infra02:21
*** ashp has quit IRC02:23
*** cody-somerville_ has quit IRC02:23
*** ashp has joined #openstack-infra02:23
*** Ryan_Lane has quit IRC02:27
*** baoli has joined #openstack-infra02:28
*** andreaf has quit IRC02:40
*** andreaf has joined #openstack-infra02:41
*** dimsum_ has joined #openstack-infra02:41
*** liusheng has quit IRC02:41
*** liusheng has joined #openstack-infra02:42
*** dimsum_ has quit IRC02:46
*** slagle has quit IRC02:53
*** achanda_ has joined #openstack-infra03:01
*** hdd has quit IRC03:03
*** achanda__ has joined #openstack-infra03:04
*** achanda has quit IRC03:05
*** achanda_ has quit IRC03:07
*** ivar-laz_ has joined #openstack-infra03:16
*** ivar-laz_ has quit IRC03:16
*** hdd has joined #openstack-infra03:18
*** ivar-lazzaro has quit IRC03:19
*** achanda has joined #openstack-infra03:23
*** SumitNaiksatam has quit IRC03:25
*** SumitNaiksatam has joined #openstack-infra03:25
*** SumitNaiksatam has quit IRC03:26
*** achanda__ has quit IRC03:26
*** SumitNaiksatam has joined #openstack-infra03:27
*** achanda has quit IRC03:36
*** achanda has joined #openstack-infra03:38
*** achanda has quit IRC03:39
*** baoli has quit IRC03:41
*** HeOS has joined #openstack-infra03:46
*** mikedillion has joined #openstack-infra03:47
*** mikedillion has quit IRC03:54
*** Longgeek has joined #openstack-infra03:57
*** HeOS has quit IRC03:59
*** Longgeek has quit IRC04:01
*** hdd has quit IRC04:04
*** sabeen1 has joined #openstack-infra04:07
*** changbl has joined #openstack-infra04:08
*** garyh has quit IRC04:10
*** harlowja is now known as harlowja_away04:15
*** pcrews has quit IRC04:19
*** zz_dimtruck is now known as dimtruck04:22
*** andreaf has quit IRC04:22
*** andreaf has joined #openstack-infra04:22
*** chandankumar has joined #openstack-infra04:39
*** garyh has joined #openstack-infra04:40
*** changbl has quit IRC04:41
*** chandankumar has quit IRC04:45
*** andreaf has quit IRC04:49
*** andreaf has joined #openstack-infra04:50
*** zns has joined #openstack-infra04:52
*** zns has quit IRC04:52
*** cristov_mac has quit IRC05:02
*** mikedillion has joined #openstack-infra05:08
*** ildikov has quit IRC05:09
*** cristov_mac has joined #openstack-infra05:11
*** cristov_mac has quit IRC05:11
*** cristov_mac has joined #openstack-infra05:12
*** cristov_mac has quit IRC05:12
*** cristov_mac has joined #openstack-infra05:12
*** cristov_mac has quit IRC05:13
*** cristov_mac has joined #openstack-infra05:13
*** mikedillion has quit IRC05:14
*** cristov_mac has quit IRC05:16
*** mikedill_ has joined #openstack-infra05:17
*** cristov_mac has joined #openstack-infra05:17
*** stevemar has quit IRC05:19
*** davidlenwell has quit IRC05:19
*** davidlenwell has joined #openstack-infra05:20
*** cristov_mac has quit IRC05:21
*** andreaf has quit IRC05:23
*** andreaf has joined #openstack-infra05:24
*** SumitNaiksatam_ has joined #openstack-infra05:32
*** SumitNaiksatam has quit IRC05:35
*** SumitNaiksatam_ is now known as SumitNaiksatam05:35
*** hdd has joined #openstack-infra05:36
*** pcrews has joined #openstack-infra05:38
*** doude has joined #openstack-infra05:44
*** parimala has joined #openstack-infra05:46
*** pcrews has quit IRC05:49
*** parimala has quit IRC05:50
*** ildikov has joined #openstack-infra05:51
*** MarkAtwood has joined #openstack-infra05:52
*** ildikov_ has joined #openstack-infra05:52
*** mikedill_ has quit IRC05:56
*** Longgeek has joined #openstack-infra05:58
*** tchaypo has quit IRC06:01
*** tchaypo has joined #openstack-infra06:01
*** Longgeek has quit IRC06:06
*** mikedillion has joined #openstack-infra06:07
*** chandankumar has joined #openstack-infra06:08
*** sabeen1 has quit IRC06:16
*** yolanda has joined #openstack-infra06:17
*** dimsum_ has joined #openstack-infra06:21
*** Longgeek has joined #openstack-infra06:25
*** mikedillion has quit IRC06:25
*** dimsum_ has quit IRC06:25
*** ildikov_ has quit IRC06:31
*** ildikov_ has joined #openstack-infra06:35
*** chandankumar has quit IRC06:37
*** doude has quit IRC06:40
openstackgerritLonggeek proposed a change to openstack-infra/puppet-apparmor: Add Rakefile file and update puppet coding style  https://review.openstack.org/10279406:41
*** MarkAtwood has quit IRC06:42
*** dimtruck is now known as zz_dimtruck06:48
*** ildikov_ has quit IRC06:50
*** ildikov has quit IRC06:57
*** Longgeek has quit IRC06:59
*** Longgeek has joined #openstack-infra07:03
*** liam__ has joined #openstack-infra07:05
*** chandankumar has joined #openstack-infra07:05
*** liam__ has quit IRC07:05
*** hdd has quit IRC07:09
*** ChuckC___ has joined #openstack-infra07:12
*** ChuckC___ has quit IRC07:18
*** chandankumar has quit IRC07:42
*** e0ne has joined #openstack-infra07:52
*** liam_ has joined #openstack-infra07:53
*** liam_ is now known as Guest3080907:53
*** e0ne has quit IRC07:53
*** e0ne has joined #openstack-infra07:54
*** Guest30809 has quit IRC07:56
*** e0ne has quit IRC07:56
*** e0ne has joined #openstack-infra08:00
*** e0ne has quit IRC08:02
*** e0ne has joined #openstack-infra08:06
*** e0ne has quit IRC08:08
*** fandi has joined #openstack-infra08:08
*** fandi has quit IRC08:10
*** fandi has joined #openstack-infra08:10
*** e0ne has joined #openstack-infra08:12
*** fandi has quit IRC08:13
*** fandi has joined #openstack-infra08:13
*** fandi has quit IRC08:16
*** e0ne has quit IRC08:16
*** fandi has joined #openstack-infra08:17
*** fandi has quit IRC08:19
*** fandi has joined #openstack-infra08:20
*** e0ne has joined #openstack-infra08:20
*** e0ne has quit IRC08:21
*** fandi has quit IRC08:22
*** fandi has joined #openstack-infra08:23
*** fandi has quit IRC08:25
*** fandi has joined #openstack-infra08:26
*** fandi has quit IRC08:28
*** fandi has joined #openstack-infra08:29
*** fandi has quit IRC08:31
*** vb has quit IRC08:34
*** vb has joined #openstack-infra08:34
*** MaxV has joined #openstack-infra08:39
*** koolhead17 has quit IRC08:48
*** jkt has joined #openstack-infra08:53
jkthi there, I'm trying to duplicate your CI config for kde.org, and I wonder how you ended up with Jenkins08:54
jktI mean, I assume that you "only" use the CI for automated voting on incoming changes and for project gating08:55
*** che-arne has quit IRC08:56
*** mdbooth has quit IRC09:03
*** mdbooth has joined #openstack-infra09:08
*** andreykurilin_ has joined #openstack-infra09:12
*** andreykurilin_ has quit IRC09:28
*** MaxV has quit IRC09:42
*** enikanorov has joined #openstack-infra09:43
*** chandankumar has joined #openstack-infra09:44
*** andreykurilin_ has joined #openstack-infra09:53
*** koolhead17 has joined #openstack-infra09:54
*** andreaf has quit IRC09:55
*** andreaf has joined #openstack-infra09:55
*** koolhead17 has quit IRC09:59
*** andreykurilin_ has quit IRC10:24
*** andreykurilin_ has joined #openstack-infra10:26
*** koolhead17 has joined #openstack-infra10:28
*** koolhead17 has quit IRC10:29
*** koolhead17 has joined #openstack-infra10:30
*** koolhead17 has joined #openstack-infra10:30
*** andreykurilin_ has quit IRC10:33
*** chandan_kumar has joined #openstack-infra10:36
*** chandankumar has quit IRC10:36
*** dimsum_ has joined #openstack-infra10:41
*** dimsum_ has quit IRC10:48
*** koolhead17 has quit IRC10:50
*** jamespage_ has joined #openstack-infra10:59
*** chandan_kumar has quit IRC11:15
*** chandan_kumar has joined #openstack-infra11:18
*** chandan_kumar has quit IRC11:22
*** koolhead17 has joined #openstack-infra11:25
*** lttrl has quit IRC11:25
*** andreaf has quit IRC11:29
*** andreaf has joined #openstack-infra11:29
*** dimsum_ has joined #openstack-infra11:39
*** HeOS has joined #openstack-infra11:40
*** qba73 has joined #openstack-infra11:42
*** qba73 has quit IRC11:50
*** HeOS has quit IRC11:50
*** bcourt has quit IRC12:05
*** jamespage_ has quit IRC12:15
*** koolhead17 has quit IRC12:32
*** zaphod42 has joined #openstack-infra12:34
*** dimsum_ has quit IRC12:44
*** dimsum_ has joined #openstack-infra12:44
*** andreaf has quit IRC12:46
*** andreaf has joined #openstack-infra12:46
*** mase_x200 has joined #openstack-infra12:46
*** dimsum_ has quit IRC12:48
*** hdd has joined #openstack-infra12:52
*** Guest51022 has joined #openstack-infra12:54
*** yaguang has joined #openstack-infra12:56
*** yamahata has joined #openstack-infra13:06
*** koolhead17 has joined #openstack-infra13:06
fungijkt: we used a lot more of its extensible features early on but have slowly been weaning our project off of it to the point where it's just a go-between got running and collecting logs from jobs now13:12
clarkband in fact implemented a form of gating with the jenkins trigger plugin13:13
fungijkt: projects like zuul and nodepool have filled in the gaps for some of the more complex plug-ins we used previously, and added new features we never could have really implemented within the context of jenkins13:13
clarkber gerrit trigger plugin13:13
fungijkt: once we get log collection and brokering of jobs to workers reimplemented, we basically no longer need jenkins for anything13:14
*** e0ne has joined #openstack-infra13:15
*** dimsum_ has joined #openstack-infra13:15
fungijkt: like with any complex system, our implementation is alive and evolving continuously, so duplicating what we do ultimately means that you either continue to follow suit on major changes we make or you rapidly diverge from what we're doing13:15
fungi(also it means our documentation has an unfortunate tendency to lag behind our tools somewhat, though we do try to avoid it lapsing into uselessness)13:16
clarkbthough with zuul speaking gearman one side of that evolution is thankfully pretty well defined (rackspace does use a non jenkins test runner today)13:17
fungijkt: anyway, i'd be very excited to see kde.org to reuse any of our tools or workflows... reusability outside of our community is a primary design goal for them. so please, feel free to ask questions if you run into any issues at all with them13:17
*** dimsum_ has quit IRC13:22
*** dimsum_ has joined #openstack-infra13:23
*** zaphod42 has quit IRC13:24
*** zul has quit IRC13:27
*** dimsum_ has quit IRC13:27
*** zul has joined #openstack-infra13:28
*** alexpilotti has joined #openstack-infra13:33
jktfungi (and others), thanks a lot13:42
jktwe already have gerrit up and running in some testing mode with just three repos, and no gating (or even CI) right now13:42
jktthat history was exactly what I was looking for; I didn't mean to imply that you're doing wrong things, just asking to help me understand what were the reasons behind what I do not currently understand13:43
jktone quick question -- the "test what is going to end in the repo" is not completely bitwise-accurate, right? I mean, zuul will use the same git algorithm (merge commit or a fast forward) for each change, same as gerrit, and will apply that in-order, but the actual commits might be slightly different (timestamps, commiter, etc), right?13:44
fungijkt: correct. one plan for solving that is to have zuul start doing the merging into gerrit's repositories rather than simply triggering gerrit's submit api call13:46
jktI heard that gerrit isn't happy when you push a change underneath13:46
jktdoes it require some magic trick?13:46
fungijkt: you push it through gerrit, but don't let gerrit/jgit calculate the merge commit for you13:46
jktso is that essentially a direct push of the changes, including the merge commit, into refs/head/$targetbranch?13:47
fungijkt: at least that's the plan. gerrit deals with pushed merge commits fine (and from what we've seen so far, potentially better than making them itself--we have definitely seen bugs and behavior differences between its merge algorithm and what cgit gives you)13:47
fungijkt: yep13:48
jktah13:48
jktI read somewhere that doing that won't mark the change as submitted13:48
jktI should probably try it before asking, though :)13:48
jktand another one, "zuul merger" is in charge of doing these merges ahead of time, right? What is the reason for that, i.e. why isn't that a part of the build job itself? Is that performance, i.e. "don't merge more often than necessary"? Could that be solved by splitting the build pipeline into one merge and a ton of tests?13:48
fungiyeah, give the push idea a try and see what you get... we may find that there's an additional api call we have to make once that's done so it updates its database accordingly... but also gerrit's been getting better over the course of its development as it tries to track less in a database and rely more on what it sees in its repos13:49
jktyup13:50
fungijkt: the zuul merger is abstracted away from the job because in a dependent pipeline you may want to recalculate that same merge over and over13:50
jktgot an example? my intuitive understand would be exactly the other way round, I might want to aovid doing that IMHO13:51
fungijkt: also it makes it easier for the job since it can just check out a ref fed into it as an envvar13:51
fungijkt: actually, even in an independent pipeline, if you have a dozen jobs for a change the zuul merger calculates that once and all teh jobs pull it, rather than every job recalculating that for itself13:52
jktok, understood13:52
jktit makes sense to separate "stuff with git" vs. "stuff with build", I guess13:53
*** tomoe has joined #openstack-infra13:53
fungijkt: also fair to mention that this is optimized for integration testing, so we can have one common refname which many tens of projects all get cloned which represents the common state across all of them which we want to test together13:53
clarkbfungi and that there is the killer feature of the merger imo13:54
fungiagreed. integration testing is where it particularly shines13:54
clarkbmakes it super easy to grab one state of the world and test it (even locally)13:54
jktis there some reason for sticking with gerrit 2.8 instead of 2.9, or is that just a matetr of not having time/whatever for update?13:55
fungijkt: we've found that the least disruptive time for our developer community to upgrade major tools is the first part of the release cycle13:56
fungiand 2.9 came out while we were rushing toward the release we just wrapped up two days ago13:56
fungiso i expect to be looking hard at another major gerrit upgrade for us in eth coming weeks13:56
jktunderstood :)13:56
fungithere are some communities already reusing our tools with gerrit 2.9 so i think it probably mostly works, but i can't say for certain13:57
jktthat test matrix UI thingy, is that a plugin? I didn't find it in the docs13:57
jkt(assuming that you aren't using a variant of https://gerrit-review.googlesource.com/58283 already)13:58
clarkbjeepyb needs to be patched for the fake group stuff13:58
clarkbthere is a change up for that but it drops 2.4 support13:58
fungijkt: are you talking about the job results table you see on changes at review.openstack.org?13:58
jktfungi: yes13:59
fungijkt: it's a javascript overlay we've bolted on. i'll get you the url13:59
jktimpressive -- so I guess you gave your jenkins a negative "can mail reviewers" to reduce bot spam as well14:00
fungijkt: http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/gerrit/hideci.js14:00
clarkbwe should probably tag jeepyb, then drop 2.4 support and merge the 2.9 compat change14:00
fungijkt: we actually haven't done much to reduce bot spam triggered through gerrit, other than recently standardizing most of the bot account names so that people can filter on them more easily14:01
fungijkt: we have a swarm of roughly 100 different bots operated by third parties reporting on viability of proposed changes in various projects, it's been a bit of a struggle to keep order14:01
*** yolanda has quit IRC14:03
*** hdd has quit IRC14:04
fungijkt: for example, this change https://review.openstack.org/126189 is currently showing 12 different external ci systems reporting on it in a pattern which that javascript overlay can recognize and display in the table (along with the results from our upstream ci)14:04
fungijkt: but if you hit the "toggle ci" button down at the bottom of the page you'll see it exposes comments from lots of ci systems, some of which aren't using a consistent reporting format (and so don't appear in the summary table up top)14:05
*** ZZelle has quit IRC14:06
*** ZZelle has joined #openstack-infra14:06
jktfungi: but that's only for the old change screen, right?14:07
fungijkt: oh, right. i keep forgetting there's a new change screen14:07
jkt:)14:07
fungion the whole our community has not been too fond of the new change screen14:07
clarkbit should work in new change screen too14:07
*** sandywalsh_ has joined #openstack-infra14:07
fungithat's my understanding, though i don't think i've tried it myself14:07
*** sandywalsh has quit IRC14:08
fungithen again, i'm usually looking at gertty anyway, not the webui14:08
clarkbdansmith uses it and is noisy :)14:08
jktclarkb: it doesn't, for me, using firefox 32.0.314:08
clarkbjkt ok so thats a bug then. it did work at one point14:08
jktI'm asking because I saw some work at upstream for supporting these kinds of status matrix; I thought it's your (openstack-infra's) work14:09
fungiit's possible sdad's recent patch isn't compatible with the new change screen (i didn't think to test that when i reviewed it)14:09
fungijkt: the interest it it has been driven from our community, among others, but we try not to run a fork of gerrit any longer so we're patiently waiting to be able to use that14:09
clarkbupstream is like a mystical beast to us. zaro seems to be able to communicate well with them. the rest of us have all various levels of rage quitting I think14:10
fungijkt: we operated a fork of gerrit 2.4 with custom patches for quite a while, and it became exceedingly painful14:10
fungiand yeah, the one patch i got into upstream gerrit was enough for me. then again, java makes me want to scream14:11
* jkt only has doc patches so far, that was reasonably easy to get right14:11
clarkbI have yet to get any code in. not even my bug fixes get merged14:11
jktI'm still waiting for them to review that change which increases the length of HTTP passwords14:11
clarkband now my cla is no longer valid? fungi did you run into that?14:12
jktbut I dislike Java as much as any other guy :)14:12
clarkb(another issue with CLAs in general)14:12
*** dimsum_ has joined #openstack-infra14:13
jktdo you guys have any opinion on buildbot? from the very quick doc scraping I did, it appears to have an openstack frontend, and looks to be much more lightweight than full-blown jenkins14:13
fungiclarkb: i would assume my google cla has expired as well14:13
clarkbI know mordred refuses to buildbot14:14
clarkbiirc it is impossible to debug failures14:14
fungii can't recall the issues he pointed out now (and maybe they're no longer relevant)14:15
*** yaguang has quit IRC14:15
clarkbit has to do with debugging iirc. you get some twisted stacktrace and have fun14:15
clarkbwhich may be better now14:15
jktmordred: if you have some time, I would appreciate to hear what you think of buildbot, people say you have some bad memories14:15
fungiahh, right, so there's also something else worth pointing out... our systems are built around a particular need to test changes to the software underlying a cloud environment, so as a result we need to do all sorts of heinous things to test machines, granting root access to completely untrusted proposed code. as a result we have a lot of heavyweight solutions in place so that we can do that safely14:17
fungiand still be able to debug failures reliably14:17
clarkband mozilla is/was ditching buildbot for jenkins so there may be more info from them too14:17
jktclarkb: didn't they ditch tinderbox and not buildbot?14:17
* jkt checks14:17
clarkbI thought it was buildbot14:18
clarkbbkero would know but isnt here14:19
*** tomoe has quit IRC14:19
*** MaxV has joined #openstack-infra14:21
*** MaxV has quit IRC14:22
*** MaxV has joined #openstack-infra14:22
*** dimsum_ has quit IRC14:23
*** dimsum_ has joined #openstack-infra14:23
clarkbultimately a decision was made ~4.5 years ago and we have grown from there14:24
*** mase_x200 has quit IRC14:24
clarkbfor us buildbot wouldnt make much sense today as we would have to rewrite zuul in the buildbot framework14:24
clarkbsince zuul and the buildbot master straddle some similar functionality14:25
clarkbthough I suppose we could put a gearman trigger thing in the buildbot master like we have with jenkins14:26
jktright, that trigger is what I'm thinking14:27
clarkbbut thats more work than just having gearman workers since you can do this today (rackspace does)14:27
jktbut I do see that there's a certain overlap in functionality, such as the dependencies on multiple projects14:27
*** dimsum_ has quit IRC14:27
jktclarkb: got a link? I tried to google for it earlier today, but didn't have much luck14:28
clarkbhttps://github.com/stackforge/turbo-hipster14:28
jktah, that thing14:28
clarkbthat is the worker they use talking directly to zuul14:28
fungione of the more colorfully-named projects around here ;)14:28
* jkt will look at it with a generic eye and disregard the purpose in its description14:28
clarkbit runs database migration tests against real db data which they post results back to gerrit14:29
jktyes, that's what I read weeks ago, and somehow it didn't occur to me that it could be a generic tool on its own14:29
clarkbbut iirc can run arbitrary shell/thingsyoucanfork14:30
*** yamahata has quit IRC14:30
jktall right, thanks a lot, I guess I have my homework now; I'll ask once I get stuck on something14:30
fungiplease do!14:30
*** lttrl has joined #openstack-infra14:31
clarkbjhesketh would know more but its early sunday night for hime now14:31
jktare you guys going to Paris btw? I should probably bring some Czech beer with me to express how much I like your stuff14:31
clarkbyes I will be there :)14:31
fungiyeah, i'll be there for owf the week before (presenting and on a couple panels) and then all week for the openstack summit14:31
fungilet's definitely catch up14:32
* jkt takes a note to not travel with a carry-on only14:32
clarkber late saturday night for jhesketh. timezones are hard14:33
*** qba73 has joined #openstack-infra14:34
*** e0ne has quit IRC14:35
*** qba73 has quit IRC14:36
*** harlowja_at_home has joined #openstack-infra14:46
fungithat reminds me. i should really write my talk for owf14:47
fungiit's a lead-in to a panel on whether free software projects really need to run their own infrastructure or should just consume hosted saas14:49
*** yamahata has joined #openstack-infra14:49
fungii think i can come up with a least a few reasons some projects might need to run their own ;)14:51
*** Longgeek has quit IRC14:51
clarkbI am sure ttx would say control at release time14:53
*** dimsum_ has joined #openstack-infra14:54
fungiwell, there's that14:58
fungii think transifex is actually a great example of saas gone wrong14:58
clarkb++14:58
*** dimsum_ has quit IRC14:59
clarkbsimilar to that I would say you dont get something like gating if you hand the reigns over to another group14:59
clarkb(so it becomes part of your decision around the value in testing)14:59
fungii think that just falls into the "you give up flexibility to tailor your tools to your workflow, and must instead tailor your workflow to your tools" category15:00
clarkbya15:01
fungigranted, that's probably the single biggest reason to at least retain the ability to run your own15:02
jeblairjkt: in the gerrit 2.4 days, i did verify that git pushing to a change marked it as merged in the gerrit db.  i have not reverified that with 2.8/2.9.15:03
*** alexpilotti has quit IRC15:03
fungii also think vps hosting providers are actually a great middle ground/compromise between the traditional "find somewhere to rack your donated servers and plug them in" situation a number of larger established projects' infrastructure teams are still struggling to support15:03
*** chandan_kumar has joined #openstack-infra15:04
*** baoli has joined #openstack-infra15:04
fungiat least compared to the more recent model of "run your project exclusively through github/bitbucket"15:04
clarkbya though orgs like osuosl help fill the rack gap pretty well15:05
fungiright, but having personally been stuck remote-managing fleets of physical servers over the years, i can say now that i've been spoiled by virtual hosting providers i wouldn't consider going back to that15:06
clarkb++15:06
jeblair2014-10-18 15:06:08,805 p=818 u=root |  fatal: [groups.openstack.org] => SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue15:06
fungithat's ansible complaining?15:07
clarkbfungi did you rebuild that server?15:07
jeblairya15:07
fungii did rebuild it and switch dns, haven't deleted the old one yet15:07
clarkbprobably need to accept host key?15:07
fungiaha--i totally forgot to do that15:07
fungifixing15:07
fungishould be all better now15:08
jeblairyaay15:08
fungiremoved the old conflicting key root had cached on the puppetmaster and accepted the new one15:08
*** baoli has quit IRC15:08
fungii would have just copied the host key to the new machine had i been thinking straight15:09
fungibut don't feel like booting the old server back up to do that at this point15:09
jktjeblair: thanks, I'll give it a try, then15:09
clarkbya no biggy15:09
*** zz_dimtruck is now known as dimtruck15:09
*** lttrl has quit IRC15:10
clarkbfungi your beach is on my television. apparently some brothers flew an airplane nearby once long ago15:13
jeblairclarkb: i assume if fungi's beach is visible, you can see him lounging with a drink (and a wearable computer)15:15
fungior lounging with a computer and a wearable drink!15:15
clarkbjeblair: unfortunately it was not live15:15
*** harlowja_at_home has quit IRC15:16
jeblairfungi: you have a drinking problem! ;)15:16
fungii did in fact walk to the memorial for those two brothers just a couple hours ago15:16
fungiit's one of our more frequent training routes15:16
*** dimtruck is now known as zz_dimtruck15:16
clarkbneat15:16
fungii live about two miles south of there on foot15:17
jeblairfungi: how many miles if you fly?15:17
fungishorter as the seagull flies, but we wander through a warren-like maze of small neighborhood streets to get to the back entrance... less traffic15:17
fungiroughly 1.5 miles straight line on the map15:18
fungialso we can walk three miles south to the tallest sand dune on the island chain15:19
clarkbdo they do reenactment flight things?15:19
clarkbprobably not osha compliant15:19
fungivery popular with hang gliders and kite fliers (and model airplane and quadrotor drone hobbyists)15:19
fungithere is in fact an airstrip right behind the memorial15:20
fungiwhich is the back entrance we take into the park... usually 2-5 small craft parked in the parking lot adjacent to it15:20
*** mdbooth has quit IRC15:21
fungiin 2003 they did a reenactment there of the wright brothers' longest flight for the 100th anniversary, but it didn't work out so well15:22
fungihttp://kittyhawk.airshowjournal.com/2003/15:24
*** Guest51022 has quit IRC15:24
*** _nadya__ has joined #openstack-infra15:25
jktjeblair: ok, I did some testing. On Gerrit 2.9.1, it appears that Gerrit does the "right thing", including updating the refs/notes/review15:26
clarkbI love that the president noped out of there15:26
clarkbthe air force one flyover would have been fun to see though15:26
jktif an unmodified change gets direct-pushed, the existing reviews are recorded in there15:27
jktif there's a modification, the reviews are not applied15:27
fungijkt: you mean recorded as attached git notes?15:27
jktand in any case the Submitted-By is not added15:28
*** mdbooth has joined #openstack-infra15:28
fungijkt: i think if you push a merge commit that should allow you to preserve the original commit sha (and thus the notes will be attached correctly)15:28
jktfungi: yes15:28
jkthttp://fpaste.org/143125/13646058/15:28
jeblairjkt: cool!15:28
jktthe UI looks a bit funky though, https://gerrit.vesnicky.cesnet.cz/r/#/c/100/15:28
fungithat's excellent news. means when we get to that point it probably won't be too hard to implement15:28
*** fandi has joined #openstack-infra15:29
jktthe git-kde-org-pusher is the name of the service account which is used for incoming replication from KDE's git; in this case, from Gerrit's POV it's the account which pushed that thing into gerrit15:29
jktit would be interesting to extend this so that a "proper" submitted-by or maybe Direct-pushed-by is added15:30
jktor maybe it's not worth the effort15:30
clarkbfungi owf looks fun15:31
fungijkt: well, since we use a separate workflow label to indicate change approval, that's basically our human proxy for the eventual submit (or eventually maybe push) performed by zuul anyway15:31
jktthere's also no change-merged event, that's something one should probably keep in mind15:32
*** mdbooth has quit IRC15:32
jkt(my modified irc bot filters out ref-updated or-how-it's-called, but reports on change-merged, and was silent during this testing)15:33
fungiright, i think zuul would probably have to use some internal event trigger for that, but it does also mean if you're soupling other tools to gerrit watching for that event there may be workarounds to consider15:33
jeblairjkt: good point.  we use ref-updated currently for all of our post-merge jobs (we do that because change-merged doesn't include the merge commit sha, so we can't actually build the correct tarball without a lot of work; whereas ref-updated includes the new commit sha of the ref even after a merge commit)15:34
fungis/soupling/coupling/15:34
jeblairoh yeah, our gerritbot watches change-merged15:34
*** yamahata has quit IRC15:34
fungii was just pulling up the source for it to confirm, and yes it does15:35
fungiwe'd likely need to adjust that to work off ref-updated15:35
*** e0ne has joined #openstack-infra15:35
jeblairheh, so we have the inverse of the problem -- given a commit sha for a merge commit, find the change15:36
fungid'oh!15:36
jeblairat least the consequences for getting it wrong are less disturbing for an irc bot15:36
clarkbI think gerrit api supports that lookup15:36
jeblairclarkb: probably so15:36
fungii wonder how it does that15:36
fungithe merge commit itself just has parent git shas, both of which might be changes in gerrit15:37
clarkbfungi one side should always be the change you want15:37
clarkbso if not merge use current sha else pick sha on correct side of merge15:38
*** stevemar has joined #openstack-infra15:38
fungioh, good point. it's the second parent of the merge15:38
fungiat least the way gerrit's performing merges currently15:38
fungiand in theory we'd preserve that order in zuul15:39
*** mdbooth has joined #openstack-infra15:39
*** e0ne has quit IRC15:41
*** harlowja_at_home has joined #openstack-infra15:44
*** e0ne has joined #openstack-infra15:45
*** e0ne has quit IRC15:48
*** e0ne has joined #openstack-infra15:52
clarkbI need to find apple juice to ferment. getting 5 gallons of fresh pressed juice has been more difficult thanI expectes15:53
*** e0ne has quit IRC15:53
jeblairclarkb: got a farmers market with a farmer you can ask?15:54
clarkbya that is my next place to check (I drove up to the gorge last weekend but all the fruit stands had intact apples only)15:55
clarkbjeblair: I think I may be a little early too?15:57
clarkbI get a feeling that juicing happens later in the season with the less desireable left behind apples. but I dunno15:57
*** e0ne has joined #openstack-infra15:57
*** e0ne has quit IRC15:57
fungii'm starting to think part of my python 3.4 problem in heatclient is actually http://git.openstack.org/cgit/openstack/oslo-incubator/tree/openstack/common/apiclient/fake_client.py#n8815:59
jeblairi dunno either :/15:59
clarkbfungi the url parameter being the troublemaker?16:00
fungiclarkb: yeah, again it's comparisons against predetermined sequences of url query parameters16:02
fungiclarkb: for cider you need to be sure to pick the right kinds of apples16:02
fungiyou don't want sweet varieties of "sauce" apples16:03
clarkbI am told any apple will work but traditionally they are apples you dont eat16:03
fungiyeah. the town i grew up near in the nc mountains grew apples as its primary cash crop16:04
fungii'm trying to remember what months the orchards operated their cider presses16:04
fungibut i think they ran all through the fall16:04
clarkbworst case I just brew more beer >_>16:05
fungiback home, ciderfest is going on while we're in paris16:07
fungibut no idea if the appalachian cider season is in the same timeframe as the cascades16:08
clarkbya I think paris timeframe is about right16:08
*** MaxV has quit IRC16:08
clarkbwhich may be part of my problem16:09
*** jamespage_ has joined #openstack-infra16:09
*** MaxV has joined #openstack-infra16:09
jeblairyou know you want one of these: http://www.sausagemaker.com/31170appleandfruitcrusher.aspx16:10
fungitempting!16:10
*** bcourt has joined #openstack-infra16:12
*** MaxV has quit IRC16:14
*** paulmillar has quit IRC16:26
*** mmaglana has joined #openstack-infra16:28
*** mmaglana has quit IRC16:30
*** dmellado has quit IRC16:31
*** paul-- has joined #openstack-infra16:33
*** _nadya__ has quit IRC16:40
*** HeOS has joined #openstack-infra16:42
*** harlowja_at_home has quit IRC16:47
*** e0ne has joined #openstack-infra16:52
*** e0ne has quit IRC16:53
*** SumitNaiksatam_ has joined #openstack-infra16:59
*** SumitNaiksatam has quit IRC17:02
*** SumitNaiksatam_ is now known as SumitNaiksatam17:02
*** zz_dimtruck is now known as dimtruck17:05
*** harlowja_at_home has joined #openstack-infra17:10
*** leifmadsen_ has joined #openstack-infra17:13
*** andreaf has quit IRC17:14
*** andreaf has joined #openstack-infra17:14
*** leifmadsen has quit IRC17:15
*** leifmadsen_ is now known as leifmadsen17:17
*** _nadya_ has joined #openstack-infra17:19
*** koolhead17 has quit IRC17:26
jlkanybody around? Looking to confirm something in my head. nova-api is the only nova service that listens on an API for direct connections, all other nova services read messages from a message bus. Is this generally true?17:27
clarkbjlk the nova channel may have the best info but I think that is true17:30
mordredjlk: that is my understanding ... but also what clarkb said17:30
clarkbthe sched cond cpu et al services all use the message bus17:30
jlkexcellent. Makes rolling that much easier.17:32
*** chandan_kumar has quit IRC17:33
*** andreykurilin_ has joined #openstack-infra17:34
*** harlowja_at_home has quit IRC17:37
clarkbmordred you see that harvin trade?17:39
*** sunil__ has quit IRC18:00
*** ihrachyshka has joined #openstack-infra18:02
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Limit python26 jobs to icehouse/juno for server projects  https://review.openstack.org/12943318:03
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Remove python26 jobs from all stackforge projects  https://review.openstack.org/12943418:03
openstackgerritAndreas Jaeger proposed a change to openstack-infra/project-config: Remove python26 jobs from various projects  https://review.openstack.org/12943518:03
*** changbl has joined #openstack-infra18:05
*** lttrl has joined #openstack-infra18:05
*** chandan_kumar has joined #openstack-infra18:06
*** rkukura has quit IRC18:20
*** MaxV has joined #openstack-infra18:23
*** ihrachyshka has quit IRC18:24
*** MaxV has quit IRC18:27
*** yolanda has joined #openstack-infra18:45
*** ihrachyshka has joined #openstack-infra18:53
*** mikedillion has joined #openstack-infra18:57
*** stevemar has quit IRC19:00
*** paul-- has quit IRC19:01
*** jhesketh has quit IRC19:01
*** jhesketh has joined #openstack-infra19:01
*** paul-- has joined #openstack-infra19:02
*** _nadya_ has quit IRC19:07
*** marun has joined #openstack-infra19:07
*** koolhead17 has joined #openstack-infra19:12
*** koolhead17 has joined #openstack-infra19:12
*** MaxV has joined #openstack-infra19:23
*** MaxV has quit IRC19:28
*** salv-orlando has joined #openstack-infra19:35
*** andreaf has quit IRC19:37
*** andreaf has joined #openstack-infra19:38
*** chandan_kumar has quit IRC19:39
*** pc_m has joined #openstack-infra19:39
*** pc_m has joined #openstack-infra19:40
*** dimtruck is now known as zz_dimtruck19:41
*** marun has quit IRC19:51
*** marun has joined #openstack-infra19:57
*** sandywalsh_ has quit IRC20:01
*** fifieldt_ has joined #openstack-infra20:06
*** e0ne has joined #openstack-infra20:08
*** fifieldt has quit IRC20:09
*** e0ne has quit IRC20:11
*** marun has quit IRC20:12
*** ihrachyshka has quit IRC20:21
*** MaxV has joined #openstack-infra20:24
*** yolanda has quit IRC20:24
*** MaxV has quit IRC20:29
*** andreaf has quit IRC20:31
*** andreaf has joined #openstack-infra20:31
*** HeOS has quit IRC20:35
*** HeOS has joined #openstack-infra20:35
*** HeOS has quit IRC20:45
openstackgerritJeremy Stanley proposed a change to openstack-infra/reviewstats: Sync the infra subprojects to the governance list  https://review.openstack.org/12944020:45
openstackgerritClark Boylan proposed a change to openstack-infra/system-config: Bail out in run_all.sh runs when failures happen  https://review.openstack.org/12944120:46
clarkbfungi: I see you too are working on this fine weekend :)20:46
funginot really20:46
clarkbanyways 129441 fixes a thing that bugged me late on friday20:46
clarkbwe really shouldn't have to be careful about that stuff. the code should do it for us20:46
fungijust wanted to update the reviewstats and governance repos20:46
*** andreaf has quit IRC20:47
fungiideally reviewstats, if someone starts to actively maintain it, should pull from the governance repo list and use something like the who-approves script i've got more or less working in https://review.openstack.org/12731520:47
fungialso, for the record, the config->system-config rename for programs.yaml in governance is https://review.openstack.org/#/c/12943820:48
clarkb++ to the reviewstats using governance thing20:49
fungii believe the stackalytics peeps said they consume programs.yaml now20:50
*** mdbooth has quit IRC20:51
fungioh, also i used some creative filtering from zuul's current parsing of layout.yaml to map to the machine-generated list of projects with stable/havana and stable/juno branches and came up with 23 we'll need to pin20:53
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Havana branches are dead, remove havana from d-g  https://review.openstack.org/12897520:53
fungii'll get krotscheck's patch amended to do that20:53
openstackgerritClark Boylan proposed a change to openstack-infra/devstack-gate: Add kilo to features.yaml  https://review.openstack.org/12897620:53
clarkband that should keep the juno grenade train rolling20:53
clarkbfungi: I think Ajaeger proposed that stuff atop krotscheck's change?20:54
clarkbhe stacked a few things on it20:54
fungioh, maybe i missed it20:54
clarkbbut I didn't look too closey20:54
fungiwell, if nothing else it will make a good cross-audit20:54
clarkbso for monday I think my plan is to keep pushing on the make grenade grok juno stuff and to update our gearman plugin everywhere20:55
clarkbthen switch back to dibification20:55
*** mdbooth has joined #openstack-infra20:56
clarkbfungi: do you have a change up for your edits to the wiki upgrade process?20:56
clarkbfungi: I wanted to look at that if you have it ready20:57
*** koolhead17 has quit IRC20:57
fungioh, not yet20:57
clarkb(curious to know what the solution to elasticsearch stuff was. did you do a full reindex? what is that process)20:57
fungithough it was just going to be a link to the upgrading section of https://git.wikimedia.org/blob/mediawiki%2Fextensions%2FCirrusSearch.git/HEAD/README20:57
fungii used 1a in that list of options20:58
fungiit took about 10 minutes20:58
clarkbgotcha. I was a bit annoed that that doc doesn't really describe when you might choose each of the options20:58
clarkbbut 1a should be universal so good enough for me20:58
*** HeOS has joined #openstack-infra20:59
fungiand also, easy21:05
fungiand also, search was already broken, so having it down for 10 more minutes was not especially impacting21:06
clarkb++21:06
*** mikedillion has quit IRC21:09
*** emagana has joined #openstack-infra21:20
*** emagana has quit IRC21:20
openstackgerritJeremy Stanley proposed a change to openstack-infra/system-config: Document rebuilding the MediaWiki search index  https://review.openstack.org/12944321:49
fungiclarkb: ^ as requested21:49
*** emagana has joined #openstack-infra21:49
*** zz_dimtruck is now known as dimtruck21:51
*** pc_m has quit IRC21:52
*** dimtruck is now known as zz_dimtruck21:53
*** bknudson has quit IRC21:53
*** baoli has joined #openstack-infra21:55
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Add Kerberos config  https://review.openstack.org/12944421:56
*** ivar-lazzaro has joined #openstack-infra21:59
*** baoli has quit IRC21:59
*** ivar-lazzaro has quit IRC22:01
*** ivar-lazzaro has joined #openstack-infra22:01
*** andreykurilin_ has quit IRC22:04
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Add Kerberos config  https://review.openstack.org/12944422:05
*** bknudson has joined #openstack-infra22:08
*** cody-somerville_ has joined #openstack-infra22:14
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Add Kerberos config  https://review.openstack.org/12944422:24
openstackgerritMonty Taylor proposed a change to openstack-infra/system-config: Add Kerberos config  https://review.openstack.org/12944422:24
*** pc_m has joined #openstack-infra22:36
jeblairclarkb: ping22:47
jeblairclarkb: es07 has an ssd and a sata drive in its volume group.  we're only using the ssd.  which one do you want to be using?  (ie, do you want it to stay on the ssd, or do you want to move the data to sata?)22:48
jeblairfungi: ^ if you happen to remember?22:49
jeblairi'm restarting zuul22:51
clarkbI think the ssd experiment is mostly meh23:04
*** otter768 has joined #openstack-infra23:04
clarkbjeblair: ssd is what is currently used so probably best to leave it and remove normal disk23:04
clarkbthen I should go back and ssd the rest23:05
clarkbbut it didnt appear to have tremendous impact23:05
jeblairif it's meh, why not leave everything on sata?23:05
jeblairit's "cheaper" so presumably if we don't have an opinion we might as well go with that :)23:06
clarkbthats fair too. I guess I wasnt considering the "cheaper" angle23:06
clarkbin that case we should migrate data back to sata then remove ssd23:06
clarkbout of curiousity why the zuul restart?23:07
jeblairok cool23:08
jeblairwe're at cinder quota, so i recovered a mostly unused volume from zuul (was holding logs)23:08
jeblairwe can get another 1g by recovering the extra volume from es07, but we should probably ask for a quota bump next week23:08
clarkbsounds good23:09
jeblairgraphite is just about out of space, so that's probably spoken for once we recover it23:09
jeblairmordred and i might be setting up an afs cell because that's how we weekend23:10
*** baoli has joined #openstack-infra23:10
clarkbI saw the kerberos change go by and figured something afs related was happening23:10
mordredclarkb: oh, it's happening baby23:11
jeblairya, we have a kerberos realm now.  ;)  afs is up next.23:11
mordredclarkb: you can add a principal for yourself if you like23:11
mordredclarkb: kerberos is operational23:11
clarkbI am about to go procure chili ingredients for chili23:12
mordredclarkb: that's probably more fun23:12
jeblairchili++23:12
clarkbthe real fun doesnt start until tomorrow morning when I roast the peppers and brown the meat23:14
*** michchap_ has quit IRC23:20
*** emagana has quit IRC23:28
*** emagana has joined #openstack-infra23:29
*** emagana has quit IRC23:34
*** cody-somerville_ has quit IRC23:38
*** baoli has quit IRC23:44
*** baoli has joined #openstack-infra23:44
*** pc_m has quit IRC23:47
*** cody-somerville_ has joined #openstack-infra23:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!