Wednesday, 2015-09-23

*** jerrygb_ has joined #openstack-swift00:05
*** jerrygb_ has quit IRC00:08
*** minwoob has quit IRC00:09
*** jerrygb has quit IRC00:09
*** mahatic has joined #openstack-swift00:26
*** garthb has quit IRC00:30
*** mahatic has quit IRC00:37
*** mfalatic has quit IRC00:40
*** gyee has quit IRC00:49
*** peluse has quit IRC00:57
*** km has quit IRC01:03
*** km has joined #openstack-swift01:04
*** DericHorn-HP has joined #openstack-swift01:07
*** peluse has joined #openstack-swift01:15
*** DericHorn-HP has quit IRC01:17
*** DericHorn-HP has joined #openstack-swift01:27
*** rohit_ has quit IRC01:40
*** jerrygb has joined #openstack-swift01:44
*** jerrygb has quit IRC01:45
*** jerrygb has joined #openstack-swift01:45
*** haomaiwang has joined #openstack-swift01:58
*** haomaiwang has quit IRC02:01
*** haomaiwang has joined #openstack-swift02:01
*** DericHorn-HP has quit IRC02:04
*** DericHorn-HP has joined #openstack-swift02:05
*** DericHorn-HP has quit IRC02:06
*** DericHorn-HP has joined #openstack-swift02:07
wbhuberclayg: u in?02:16
*** darrenc is now known as darrenc_afk02:21
*** DericHorn-HP has quit IRC02:34
*** DericHorn-HP has joined #openstack-swift02:37
*** early has quit IRC02:47
mattoliverauwbhuber: haven't seen him since I've been up. But he may pop up cause he keeps some wierd times :)02:47
wbhubermattoliverau: thanx.02:48
*** darrenc_afk is now known as darrenc02:49
*** sanchitmalhotra has joined #openstack-swift02:52
*** early has joined #openstack-swift02:55
*** wbhuber has quit IRC02:56
*** haomaiwang has quit IRC03:01
*** haomaiwa_ has joined #openstack-swift03:01
*** DericHorn-HP has quit IRC03:04
*** DericHorn-HP has joined #openstack-swift03:05
*** david-lyle has joined #openstack-swift03:13
*** peluse has quit IRC03:21
notmynamegood evening03:25
openstackgerritMerged openstack/swift: Fix proxy handling of EC client disconnect  https://review.openstack.org/22535703:29
notmynameyay03:29
notmynameI'm working on a list of prioritized reviews tonight. that's one of them I can take off now :-)03:30
*** DericHorn-HP has quit IRC03:38
mattoliveraunotmyname: welcome back :)03:40
*** DericHorn-HP has joined #openstack-swift03:41
*** jrichli has joined #openstack-swift03:49
notmynameclayg: on your note on outstanding patches, you suggested that https://bugs.launchpad.net/swift/+bug/1484598 was potentially less that a critical bug for release (and patch https://review.openstack.org/#/c/213147/)03:54
openstackLaunchpad bug 1484598 in OpenStack Object Storage (swift) "Proxy server ignores additional fragments on primary nodes" [High,In progress] - Assigned to paul luse (paul-e-luse)03:54
notmynamewhy would it be anything other than critical for the release? seems pretty important03:54
*** hrou has quit IRC03:56
*** jerrygb has quit IRC03:56
*** mahatic has joined #openstack-swift03:56
*** garthb has joined #openstack-swift03:57
mahaticgood morning03:59
*** DericHorn-HP has quit IRC03:59
*** haomaiwa_ has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:01
*** silor has joined #openstack-swift04:08
mattoliveraumahatic: morning :)04:09
notmynamehttps://wiki.openstack.org/wiki/Swift/PriorityReviews is updated with a "Liberty patches" section. it's got a bunch of EC stuff in it right now04:10
notmynameC = critical, H = high, L = low04:10
notmynamefor priority04:10
notmynamethe critical and high ones are starred in the review dashboard04:11
mahaticmattoliverau: hello04:12
notmynamealso, this will be the focus of the meeting this week04:12
zaitcevhmm, 215276 is WIP04:13
mattoliverauKk04:13
notmynamezaitcev: ah, thanks. I'll ad that note04:13
*** jrichli has quit IRC04:14
notmynameyeah, we need clayg to figure out what's going on and what's needed04:14
notmynamethe other big deal (related, though not directly, to EC) is https://bugs.launchpad.net/swift/+bug/145243104:17
openstackLaunchpad bug 1452431 in OpenStack Object Storage (swift) "some parts replicas assigned to duplicate devices in the ring" [Critical,Confirmed] - Assigned to Samuel Merritt (torgomatic)04:17
notmynameah, wait. that patch 222799 is already referenced :-)04:17
patchbotnotmyname: https://review.openstack.org/#/c/222799/04:17
notmynamewell, if we can get that patch, then we can likely have a little bit of breathing room for really solving the ring placement bug04:18
notmynameclayg: link to the error in the ring validator patch for py27 tests in the gate: http://logs.openstack.org/99/222799/4/check/gate-swift-python27/a8ca52c/console.html#_2015-09-22_23_40_46_86604:19
notmynameacoles_: clayg: tomorrow at the meeting here's what I'd love to see:04:21
*** zaitcev has quit IRC04:22
notmynamewe've got a good list of patches and bug that need closing. but getting a bigger picture of it would be helpful. for me, especially the multiple versions of a FA and what to do in that case would be nice to know04:23
notmynamethere are a few patches and comments in gerrit that leave me wondering which way the current plan is for04:23
notmynameie for optimistic gets04:25
*** silor1 has joined #openstack-swift04:25
* mahatic will be right back04:26
*** silor has quit IRC04:26
*** silor1 is now known as silor04:26
*** mahatic has quit IRC04:26
claygnotmyname: the duplicate parts on one node issue takes two things 1) a part on a handoff 2) a rebalance causing that part to be a primary - after minwoo's patch 213147 we'll have tests that validate the reconstructor can dig it self out of this situation, and if it can keep up with the ring's min_part_hours we won't ever have more than a single unavailable frag while it does - meaning there's no risk to availability and dura04:47
patchbotclayg: https://review.openstack.org/#/c/213147/04:47
notmynameclayg: ack04:50
claygnotmyname: so I don't think lp bug #1484598 is really much of an issue now that we have etag buckets and use handoffs correctly to work around overwrite04:50
openstackLaunchpad bug 1484598 in OpenStack Object Storage (swift) "Proxy server ignores additional fragments on primary nodes" [High,In progress] https://launchpad.net/bugs/1484598 - Assigned to paul luse (paul-e-luse)04:50
claygnotmyname: i'm cool improving on the proxy behavior - but it should fall out of the work on optomistic GET's - where we're evaluating different mixtures of available data and potentially making mutltiple requests to a single node *already* - rather than be an independent behavior04:51
notmynameclayg: and because of the etag buckets and handoffs on overwrites, that's why the stuff you and acoles_ have been doing around the optimistic gets is also listed as WIP--basically, not sure yet if it's needed. patch 181407 and patch 17487504:53
patchbotnotmyname: https://review.openstack.org/#/c/181407/04:53
patchbotnotmyname: https://review.openstack.org/#/c/174875/04:53
notmynameand maybe even patch 21527604:54
patchbotnotmyname: https://review.openstack.org/#/c/215276/04:54
claygnotmyname: well, there was another one that's already landed - paul started it - ended up splitting node_iter into a NodeIter class - it helped a lot04:54
clayg... but it merged awhile ago (shortly after Austin)04:54
*** garthb has quit IRC04:55
claygnotmyname: i briefly talked to acoles_ about patch 181407 and patch 174875 - they're not critical - some of the bad behaviors they were trying to address have been addressed in subsequent patches (we had better ideas) - so they need to be compleatly re-evaluated and there's no open bugs related to the issues they set out to address currently open (issues they were trying to expose have been apparently mostly fixed in other w04:56
patchbotclayg: https://review.openstack.org/#/c/181407/04:56
patchbotclayg: https://review.openstack.org/#/c/174875/04:56
notmynameok04:57
notmynameclayg: I took your gist and put it into the wiki at https://wiki.openstack.org/wiki/Swift/PriorityReviews04:57
notmynameso we have a common place to look04:57
claygpatch 215276 is a beast - thank god for acoles_ - but he's probably going to need some help and I'm frankly thinking that aside from maybe some cleanup on the wire-protocol-timeout-disconnect stuff I'll be mainly working on rings04:57
patchbotclayg: https://review.openstack.org/#/c/215276/04:57
claygyeah I saw that on the wiki page - looks nice04:57
clayg... but - I have an idea on the rings!04:58
notmynameyay!04:58
claygwell see - i'm going back to my debugger04:58
*** haomaiwang has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:01
*** zhill has joined #openstack-swift05:03
*** mahatic has joined #openstack-swift05:04
*** klrmn1 has quit IRC05:05
*** ppai has joined #openstack-swift05:07
*** trifon has joined #openstack-swift05:18
*** dipe has joined #openstack-swift05:27
*** proteusguy__ has quit IRC05:33
*** zhill has quit IRC05:40
*** proteusguy__ has joined #openstack-swift05:45
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-swift06:01
*** jerrygb has joined #openstack-swift06:08
*** jerrygb has quit IRC06:12
*** SkyRocknRoll has joined #openstack-swift06:12
*** sanchitmalhotra1 has joined #openstack-swift06:29
*** sanchitmalhotra has quit IRC06:31
*** sanchitmalhotra has joined #openstack-swift06:45
*** sanchitmalhotra1 has quit IRC06:47
*** haomaiwang has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** rledisez has joined #openstack-swift07:07
*** mahatic has quit IRC07:11
*** dipe has quit IRC07:33
*** sayali has quit IRC07:34
*** sayali has joined #openstack-swift07:38
*** jordanP has joined #openstack-swift07:41
*** KLuka_ has left #openstack-swift07:48
*** geaaru has joined #openstack-swift07:51
*** sanchitmalhotra1 has joined #openstack-swift07:57
*** acoles_ is now known as acoles07:58
*** sanchitmalhotra has quit IRC07:59
openstackgerritClay Gerrard proposed openstack/swift: Validate against duplicate part replica device assignment  https://review.openstack.org/22279908:00
openstackgerritClay Gerrard proposed openstack/swift: wip: figure out when you're going to have a bad time  https://review.openstack.org/22664508:00
*** haomaiwang has quit IRC08:01
*** haomaiwang has joined #openstack-swift08:01
*** haomaiwang has quit IRC08:10
*** haomaiwa_ has joined #openstack-swift08:12
*** mahatic has joined #openstack-swift08:13
*** aix has joined #openstack-swift08:15
*** sanchitmalhotra has joined #openstack-swift08:17
*** sanchitmalhotra1 has quit IRC08:20
*** mac_ified has quit IRC08:27
openstackgerritMerged openstack/swift: Change fake_footers to replace response headers instead of append  https://review.openstack.org/22295308:37
openstackgerritMerged openstack/swift: Minor keymaster cleanup  https://review.openstack.org/22653108:51
openstackgerritAlistair Coles proposed openstack/python-swiftclient: Make more assertions in client unit tests  https://review.openstack.org/22665408:57
acolestimburke: see what you started ^^ :)08:57
*** sanchitmalhotra1 has joined #openstack-swift08:59
*** haomaiwa_ has quit IRC09:01
*** 16WAAB8UT has joined #openstack-swift09:01
acolesnotmyname: i agree with clayg re  patch 181407 and patch 174875. there are some optimisations in them but they need rework and imho not critical for liberty.09:02
patchbotacoles: https://review.openstack.org/#/c/181407/09:02
patchbotacoles: https://review.openstack.org/#/c/174875/09:02
*** sanchitmalhotra has quit IRC09:02
acolesclayg: thanks for review on 215276, yes its a beast.09:05
*** jerrygb has joined #openstack-swift09:08
*** jerrygb has quit IRC09:13
*** joeljwright has joined #openstack-swift09:18
*** ChanServ sets mode: +v joeljwright09:18
*** joeljwright has quit IRC09:19
*** sanchitmalhotra has joined #openstack-swift09:27
*** sanchitmalhotra1 has quit IRC09:30
*** tsubic has quit IRC09:44
*** tsubic has joined #openstack-swift09:44
*** 16WAAB8UT has quit IRC10:01
*** 6JTAB3MXI has joined #openstack-swift10:01
*** Kennan_Vacation has quit IRC10:01
*** Kennan_Vacation has joined #openstack-swift10:02
*** km has quit IRC10:14
*** ctennis has quit IRC10:41
*** ctennis has joined #openstack-swift10:41
*** mahatic has quit IRC10:47
*** 6JTAB3MXI has quit IRC11:01
*** haomaiwang has joined #openstack-swift11:01
*** aix has quit IRC11:03
openstackgerritAlistair Coles proposed openstack/python-swiftclient: Add tests and param definitions for headers parameter  https://review.openstack.org/22623611:09
*** kei_yama has quit IRC11:20
openstackgerritAlistair Coles proposed openstack/python-swiftclient: Add tests and param definitions for headers parameter  https://review.openstack.org/22623611:22
*** haomaiwang has quit IRC11:22
*** shakamunyi has quit IRC11:25
*** aix has joined #openstack-swift11:34
*** mahatic has joined #openstack-swift11:34
*** jerrygb has joined #openstack-swift11:35
*** ccavanna has quit IRC11:36
openstackgerritDonagh McCabe proposed openstack/swift: Disallow cross-reseller COPY  https://review.openstack.org/22670711:38
*** jerrygb has quit IRC11:39
*** sanchitmalhotra1 has joined #openstack-swift11:42
*** sanchitmalhotra has quit IRC11:43
*** jerrygb has joined #openstack-swift11:48
*** sayali has quit IRC11:50
*** sayali has joined #openstack-swift11:55
tamizh_geekclayg: Hello, when you have time can you please review this for me? https://review.openstack.org/#/c/225499/11:58
*** sanchitmalhotra has joined #openstack-swift12:07
*** sanchitmalhotra1 has quit IRC12:08
*** agomez has joined #openstack-swift12:12
*** mahatic has quit IRC12:21
*** ccavanna has joined #openstack-swift12:24
*** mahatic has joined #openstack-swift12:39
*** agomez has quit IRC12:47
*** sayali has quit IRC12:47
*** sayali has joined #openstack-swift12:50
*** ppai has quit IRC12:58
*** peluse has joined #openstack-swift12:58
*** janonymous has quit IRC13:00
*** mac_ified has joined #openstack-swift13:01
*** mac_ified has quit IRC13:01
*** hrou has joined #openstack-swift13:01
*** jkugel has joined #openstack-swift13:02
*** haomaiwang has joined #openstack-swift13:08
hrouacoles, thanks for reviewing / merging that crypto patch, yea I'm willing to bet 12 hours is a record !  But careful now you're setting a precedence ; )13:17
*** ccavanna has quit IRC13:19
*** silor has quit IRC13:22
*** wbhuber has joined #openstack-swift13:25
*** dustins has joined #openstack-swift13:28
*** SkyRocknRoll has quit IRC13:36
*** sanchitmalhotra1 has joined #openstack-swift13:39
*** ccavanna has joined #openstack-swift13:40
*** sanchitmalhotra has quit IRC13:42
*** mahatic has quit IRC13:44
*** sanchitmalhotra has joined #openstack-swift13:53
peluseFYI the perf data I posted yesterday has some issues with the bar charts at the end - the 50/50 numbers are all about 50% of what they should be.  Mistake in compiling the data, will post a new one later today13:56
*** delatte has quit IRC13:56
*** sanchitmalhotra1 has quit IRC13:56
*** delattec has quit IRC13:58
*** kairo has quit IRC13:59
*** kairo has joined #openstack-swift13:59
*** haomaiwang has quit IRC14:01
*** haomaiwang has joined #openstack-swift14:01
*** sanchitmalhotra has quit IRC14:02
*** jerrygb has quit IRC14:03
*** trifon has quit IRC14:04
*** jerrygb has joined #openstack-swift14:07
*** breitz has quit IRC14:10
*** breitz has joined #openstack-swift14:10
openstackgerritOpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements  https://review.openstack.org/8925014:12
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873614:13
*** mahatic has joined #openstack-swift14:13
*** jlhinson has joined #openstack-swift14:17
*** hemanthm has left #openstack-swift14:18
*** jrichli has joined #openstack-swift14:19
*** jkugel has left #openstack-swift14:23
*** pberis has joined #openstack-swift14:31
*** cdelatte has joined #openstack-swift14:36
*** delattec has joined #openstack-swift14:36
claygacoles: i hope you mean that thank you - it was just my first impression - you know this one -> http://www.osnews.com/story/19266/WTFs_m14:42
claygtamizh_geek: oh yeah!  I remeber that one!14:42
claygwait... no that's a new one - but it does the same thing as the old one?14:43
*** peluse has quit IRC14:43
acolesclayg: lol14:43
claygit took a long time to load in my head - so it was acctually a pretty low rate14:44
acolesclayg: it was genuine!14:44
claygkudos!14:44
acolesi am discarding discard :)14:44
claygoh yeah that - we need a glossary!14:45
acolesclayg: 'DiskFile for Dummies' available at any good bookshop14:45
*** delattec has quit IRC14:46
*** cdelatte has quit IRC14:46
claygacoles: that's funny14:47
*** mahatic has quit IRC14:47
*** haomaiwang has quit IRC14:47
*** mahatic has joined #openstack-swift14:47
*** cdelatte has joined #openstack-swift14:48
*** delattec has joined #openstack-swift14:48
*** garthb has joined #openstack-swift14:50
notmynameacoles: does the table on https://wiki.openstack.org/wiki/Swift/PriorityReviews look reasonable to you?14:51
acolesnotmyname: mostly. i think https://review.openstack.org/#/c/213147 is going to be redundant once i am done with patch 21527614:55
patchbotacoles: https://review.openstack.org/#/c/215276/14:55
acolesnotmyname: so i think we could avoid parallel/redundant effort there14:55
*** itlinux has joined #openstack-swift14:55
notmynameok, great14:55
acolesthe optimistic gets will handle alt frags as a side effect of fetching durable frags when necessary14:56
notmynameshall I drop patch 213147 references then?14:56
patchbotnotmyname: https://review.openstack.org/#/c/213147/14:56
acolesi should say.. "IMHO"14:56
notmyname:-)14:56
claygnotmyname: that's the same thing I said last night14:57
acolesright now patch 215276 on gerrit doesn't do alt frags but it will14:57
patchbotacoles: https://review.openstack.org/#/c/215276/14:57
claygone patch with two closes-bug is better than two patches with one closes-bug each14:57
acolesalso i think clayg and peluse and i agree that alt frags is a less likely scenario14:57
acolesclayg: i actually implemented frag tracking in the proxy today and discovered there are SO many tests that don't send a frag index with faked responses :/14:58
acolesso my poor proxy code amassed lots of frag index=None14:59
notmynamepriority reviews and dashboard updated14:59
notmynamehttps://review.openstack.org/#/c/225357/ merged last night. does that affect the prioritization of https://review.openstack.org/#/c/211338 ?15:02
*** pberis has quit IRC15:04
tamizh_geekclayg: It fixes the same bug 1182155, but this will be a bigger change touching all over the code base. I'm just starting, but wanted to confirm if the direction I'm heading is the right one?15:05
openstackbug 1182155 in OpenStack Object Storage (swift) "Error responses ignore accept header" [Undecided,In progress] https://launchpad.net/bugs/1182155 - Assigned to Azhagu Selvan SP (tamizhgeek)15:05
*** pberis has joined #openstack-swift15:05
*** lpabon has joined #openstack-swift15:06
*** minwoob has joined #openstack-swift15:06
*** ahale has quit IRC15:07
*** ahale has joined #openstack-swift15:07
notmynameclayg: I'm not getting any local unittest failures for https://review.openstack.org/#/c/222799 (I assume like you). but the gate certainly doesn't like it15:12
notmynametamizh_geek: I think there's a few things you can do to help out reviewers looking at your patches. first, make sure that the standard tests are passing.15:15
notmynametamizh_geek: if you get stuck on a particular test that's failing, that's a great thing to ask people for guidance or help on15:15
*** itlinux_ has joined #openstack-swift15:16
notmynametamizh_geek: and then when you have the tests passing, a specific question like "I was considering A and B, I think A is better because..., what do you think?" is going to get a better review response than "did I do this whole thing right"15:16
notmynametamizh_geek: but in general, as we talked about previously, the general idea of what you're working on is good15:16
notmynametamizh_geek: thanks for tackling it15:17
tamizh_geeknotmyname: Yes sorry, I should have fixed them. Just thought of validating it with more than one core dev. Thanks for the suggestions.15:17
*** itlinux has quit IRC15:18
notmynametamizh_geek: right now we're all working on the last set of priority patches that need to land for the next release (by the end of next week). So if you don't get a lot of attention on your patch yet, don't feel bad :-)15:18
notmynametamizh_geek: and if you want to help out with getting those last patches working, take a look at the stuff listed on https://wiki.openstack.org/wiki/Swift/PriorityReviews15:18
tamizh_geeknotmyname: Ah okay!15:18
tamizh_geekwill sure do :)15:20
*** garthb has quit IRC15:20
*** garthb has joined #openstack-swift15:20
*** kevinc___ has joined #openstack-swift15:25
*** pberis has quit IRC15:29
*** pberis has joined #openstack-swift15:31
*** delattec has quit IRC15:31
*** cdelatte has quit IRC15:31
tamizh_geeknotmyname: https://review.openstack.org/#/c/177195 I see this in merge conflict and required for Liberty release(If I'm right), can I fix it and send patch to the same patchset? Or should I start a new one?15:32
notmynametamizh_geek: if you can resolve the merge conflict, then yes, send it to the same change-id (same patch, new patchset in gerrit)15:33
*** alejandrito has joined #openstack-swift15:33
*** trifon has joined #openstack-swift15:37
*** klrmn1 has joined #openstack-swift15:38
*** klrmn1 has quit IRC15:39
openstackgerritTim Burke proposed openstack/swift: Add the ability to specify ranges for SLO segments  https://review.openstack.org/21172615:44
*** sayali has quit IRC15:55
*** esker has joined #openstack-swift15:56
notmynameclayg: locally on my saio, if I use a seed of 1, I get the exact same results as CI sees with a seed of 2 on http://logs.openstack.org/99/222799/5/check/gate-swift-python27/dc1c354/console.html#_2015-09-23_08_06_35_11115:57
openstackgerritTim Burke proposed openstack/swift: Add the ability to specify ranges for SLO segments  https://review.openstack.org/21172616:00
*** garthb_ has joined #openstack-swift16:00
*** garthb has quit IRC16:02
*** itlinux_ has quit IRC16:06
*** esker has quit IRC16:06
*** sayali has joined #openstack-swift16:08
*** resker has joined #openstack-swift16:12
*** itlinux has joined #openstack-swift16:13
*** rledisez has quit IRC16:14
*** resker has quit IRC16:16
*** chsc has joined #openstack-swift16:17
*** peluse has joined #openstack-swift16:17
*** amit213 has quit IRC16:17
*** amit213 has joined #openstack-swift16:17
openstackgerritJohn Dickinson proposed openstack/swift: Add the ability to specify ranges for SLO segments  https://review.openstack.org/21172616:24
*** klrmn1 has joined #openstack-swift16:24
*** klrmn1 has quit IRC16:24
notmynametimburke: I rebased on top of master so the coverage thing is picked up and CI (or at least the 3rd party CI) passes ^16:24
*** pberis has quit IRC16:28
*** lpabon has quit IRC16:28
*** janonymous has joined #openstack-swift16:28
*** pberis has joined #openstack-swift16:29
claygtamizh_geek: notmyname is correct - but I generally like the "push early" strategy for code review - it's always better to get some early validation before you're invested if you're unsure16:29
*** gyee has joined #openstack-swift16:29
*** jordanP has quit IRC16:29
*** garthb_ has quit IRC16:30
claygtamizh_geek: that said - we barely keep up with all the code that *is* ready for review - much less stuff that's still WIP - so specific questions help alot - "I pushed up change xyz, it's not ready, what do you think about <link to hunk of diff> WRT doing a instead of b"16:30
*** garthb has joined #openstack-swift16:30
claygi stared the change - but I won't get a chance to look at it until off hours16:30
notmynameclayg: I'd like to help out on the ring validation, but I'm not really sure where to start with getting the CI unit test to pass16:32
claygnotmyname: kota says it fails if you run it with tox?16:35
*** pberis has quit IRC16:35
notmynameyeah, I'm doing that now. passes with ./.unitttests16:35
claygnotmyname: he also had a follow up change that with smaller seeds that he said was better?16:35
*** pberis has joined #openstack-swift16:36
claygnotmyname: yeah I mean those two tests that fail are storta stupid - I think the pre-existing one was trying to validate the printing of the balance warning and the exit code - but with a valid ring it wasn't unbalanced anymore16:36
notmynametox just passed for me16:36
claygnotmyname: if there was a way to make a ring that would be for sure unbalanced we could probably just fix the test to do that and remove the other.16:37
notmynametest_warn_at_risk and test_no_warn_when_balanced? I'm only seeing the failure in test_no_warn_when_balanced16:37
claygyeah - something like that - they were right next to each other and they do basically the same thing with a different seed16:39
claygit was probably a bad idea - but it WOMM and had nothing to do with what I was acctually trying to change.16:40
notmynameya16:40
clayg... it probably could use an explicit test for failing validation :\16:40
claygnotmyname: anyway - one of them pre-existed - and like I said it was really there I think to validate the error-printing and exit code16:41
pelusectennis: you around?16:41
notmynameyeah16:41
clayg... if it's not obvious how to create a ring that will absolutely be un-balanced (at risk) we should probably mock it up16:41
ctennispeluse: y16:42
*** pberis has quit IRC16:43
*** sanchitmalhotra has joined #openstack-swift16:43
*** pberis has joined #openstack-swift16:44
notmynameclayg: ok, I'll work on that16:45
*** pberis has quit IRC16:45
*** pberis has joined #openstack-swift16:46
*** delattec has joined #openstack-swift16:47
*** cdelatte has joined #openstack-swift16:47
openstackgerritJohn Dickinson proposed openstack/swift: Validate against duplicate part replica device assignment  https://review.openstack.org/22279916:49
*** delattec has quit IRC16:50
*** cdelatte has quit IRC16:50
notmynamethat has kota's patch squashed into it to unblock CI. still the ugly tests, but perhaps this can let us move forward (perfect is the enemy of good and all that)16:50
claygoh - does that work?16:51
notmynamewell the tests passed for jenkins in gerrit, so I think so16:51
notmynamealso passes locally16:51
*** aix has quit IRC16:53
*** pberis has quit IRC16:55
*** pberis has joined #openstack-swift16:56
*** mfalatic has joined #openstack-swift17:02
*** mfalatic has quit IRC17:03
*** mfalatic has joined #openstack-swift17:04
*** amoturi has joined #openstack-swift17:05
*** esker has joined #openstack-swift17:15
*** xnox has quit IRC17:17
*** pberis has quit IRC17:20
*** pberis has joined #openstack-swift17:21
*** peluse has quit IRC17:21
*** pberis has quit IRC17:23
*** pberis has joined #openstack-swift17:23
*** klrmn1 has joined #openstack-swift17:26
*** garthb has quit IRC17:26
*** garthb has joined #openstack-swift17:26
*** peluse has joined #openstack-swift17:29
pelusex17:30
*** pberis has quit IRC17:35
*** pberis has joined #openstack-swift17:36
*** xnox has joined #openstack-swift17:36
*** geaaru has quit IRC17:40
*** pberis has quit IRC17:40
*** pberis has joined #openstack-swift17:40
*** bill_az has joined #openstack-swift17:41
*** esker has quit IRC17:43
notmynameclayg: gate is unblocked for https://review.openstack.org/#/c/222799/17:44
*** esker has joined #openstack-swift17:50
*** pberis has quit IRC17:52
openstackgerritTim Burke proposed openstack/python-swiftclient: Make LengthWrappers resettable if their _readable is  https://review.openstack.org/22689717:53
*** pberis has joined #openstack-swift17:54
*** Fin1te has joined #openstack-swift18:02
*** thurloat_isgone is now known as thurloat18:02
*** esker has quit IRC18:03
*** garthb has quit IRC18:03
*** itlinux_ has joined #openstack-swift18:03
*** garthb has joined #openstack-swift18:03
*** itlinux has quit IRC18:04
*** annegentle has joined #openstack-swift18:13
*** itlinux has joined #openstack-swift18:23
*** delattec has joined #openstack-swift18:24
*** cdelatte has joined #openstack-swift18:24
*** itlinux_ has quit IRC18:25
*** pberis has quit IRC18:27
*** pberis has joined #openstack-swift18:28
*** robefran has joined #openstack-swift18:32
*** pberis has quit IRC18:32
*** aix has joined #openstack-swift18:33
*** pberis has joined #openstack-swift18:33
*** pberis has quit IRC18:37
*** pberis has joined #openstack-swift18:38
robefranDoes anyone know how to access account_info from within object-server middleware ? Specifically I'm trying to find the bytes used in the account  X-Account-Bytes-Used . I know this is a dumb idea, but I am not allowed to add middleware to the proxy-server pipeline in my deployment. I am however allowed to add middleware to a specific set of object-servers, but I can't figure out how to make an account HEAD request from the18:41
robefranobject-server pipeline..18:42
*** amoturi has left #openstack-swift18:42
*** esker has joined #openstack-swift18:44
*** pberis has quit IRC18:45
*** pberis has joined #openstack-swift18:46
*** ccavanna has quit IRC18:46
*** pberis has quit IRC18:48
*** pberis has joined #openstack-swift18:49
jrichlirobefran: you might take a look at swift/common/internal_client.py:    An internal client that uses a swift proxy app to make requests to Swift.18:50
*** CaioBrentano has joined #openstack-swift18:50
jrichlirobefran: and there is the method swift/proxy/controllers/base.py:def get_account_info(env, app, swift_source=None)18:50
jrichlibut that is on the proxy18:50
robefranjrichli: what I'm struggling with is the app, when I am using the object-server pipeline instead of the proxy-server pipeline18:52
robefranjrichli: for InternalClient, I've only found uses in object-expirer and object-reconciler. I haven't figured out how to invoke it from object-server middleware18:54
*** delattec has quit IRC18:54
*** cdelatte has quit IRC18:54
robefranjrichli: trying to get account_info from the object-server is likely a big mistake, and I may need to convince my deployment to let me add proxy-server middleware, but I wanted to first see if it could be done in the object-server pipeline18:55
*** layzee has joined #openstack-swift18:55
robefranjrhichl: oops, that should be container-reconciler, not object-reconciler18:57
jrichlirobefran: this has exceeded my knowledge.  I can poke around to see what I can find.18:59
*** pberis has quit IRC18:59
robefranjrichli: the object-expirer.conf and container-reconciler.conf refer to app:proxy-server, I may be able to fake up my own .conf for my middleware to refer to app:proxy-server, and then use InternalClient from within my middleware.19:00
*** pberis has joined #openstack-swift19:01
hrourobefran, yea we use the IC (internal client) from storlet middleware.  In theory our code can reside on both the object and proxy (that is the middleware lives on both), but I'm not entirely sure it'll work (I'll check):  https://github.com/openstack/storlets/blob/master/Engine/swift/storlet_gateway/storlet_docker_gateway.py#L469    The key is the proxy conf file's existence on the obj server.19:01
claygare we having fun yet?  I can't tell :\19:01
claygwhat's that middleware that spits out storage node urls from the ring?19:01
claygendpoints?19:02
claygsomeone should do a version of IC that can use that api instead of needed real rings on disk - basically a python "APIRing" class that supports all the iter_nodes stuff that IC needs19:02
robefranhrou: do I need the entire proxy-server.conf, or can I just use a partial fake .conf for the InternalClient invoked by my middleware?19:02
*** cdelatte has joined #openstack-swift19:03
*** delattec has joined #openstack-swift19:03
hrouclayg, robefran - interesting !   Yea I wonder if that's a good approach instead.19:03
clayghrou: yeah that'd be sweet19:04
clayghrou: it could get even better with like signed requests to the storage nodes and the ability to fill in stuff like error limiting data19:04
hrourobefran, nah I think the key is you need the proxy.conf and the ring files to know where to look ; )19:04
clayghrou: and a pony19:04
hroulist_endpoints19:05
robefranhrou: ahh.. the ring files, yes you need all the proxy stuff19:05
*** kairo has quit IRC19:08
notmynameif you're going to the tokyo summit and for some reason you haven't registered yet, do it now. prices are about to go up, and the discount codes are about to expire19:09
*** thurloat is now known as thurloat_isgone19:11
*** janonymous has quit IRC19:19
*** sanchitmalhotra has quit IRC19:20
*** esker has quit IRC19:22
*** cdelatte has quit IRC19:22
*** delattec has quit IRC19:22
robefranhrou: object-expire on the object nodes uses InternalClient, so everything is there for InternalClient19:23
hrourobefran, yea that's a really good point.19:24
robefranhrou: I'm going to try IC from within my middleware. The only problem may be the time taken by the loadapp() within InternalClient __init__19:25
*** delattec has joined #openstack-swift19:25
*** cdelatte has joined #openstack-swift19:25
*** pberis has quit IRC19:26
*** pberis has joined #openstack-swift19:27
*** pberis has quit IRC19:30
*** pberis has joined #openstack-swift19:32
*** annegentle has quit IRC19:32
*** annegentle has joined #openstack-swift19:32
*** pberis has quit IRC19:36
*** pberis has joined #openstack-swift19:37
*** trifon has quit IRC19:38
*** pberis has quit IRC19:46
*** pberis has joined #openstack-swift19:47
*** pberis has quit IRC19:51
*** pberis has joined #openstack-swift19:52
*** trifon has joined #openstack-swift19:53
*** pberis has quit IRC19:53
*** pberis has joined #openstack-swift19:54
*** delattec has quit IRC19:57
*** cdelatte has quit IRC19:57
*** delattec has joined #openstack-swift19:58
*** cdelatte has joined #openstack-swift19:58
*** esker has joined #openstack-swift19:59
*** esker has quit IRC20:02
*** pberis has quit IRC20:08
*** pberis has joined #openstack-swift20:09
openstackgerritAlistair Coles proposed openstack/swift: Enable object server to return non-durable data  https://review.openstack.org/21527620:10
acolespeluse: ^^ this now does alt frags on same node, which means you are a co-author!20:10
claygacoles: nice work!20:10
acolesclayg: and you, fab review thanks, but more great ideas than i could implement today20:11
claygacoles: no worries - you're a machine!20:11
*** pberis has quit IRC20:13
*** pberis has joined #openstack-swift20:14
acolesnotmyname: i'm just heading home, may be a few mins late for meeting20:17
*** kevinc___ has quit IRC20:17
acolesthere is ameeting right?20:17
*** acoles is now known as acoles_20:20
*** acoles_ is now known as acoles20:20
*** acoles is now known as acoles_20:21
*** delattec has quit IRC20:22
*** cdelatte has quit IRC20:22
*** Fin1te has quit IRC20:23
notmynameacoles_: yup. there's a meeting20:23
*** mahatic has quit IRC20:25
*** pberis has quit IRC20:26
*** cdelatte has joined #openstack-swift20:26
*** delattec has joined #openstack-swift20:26
*** robefran has quit IRC20:38
*** Fin1te has joined #openstack-swift20:44
claygwhen is the meeting?20:47
*** zhill has joined #openstack-swift20:48
*** dustins has quit IRC20:50
*** ho has joined #openstack-swift20:51
notmynameclayg: 9 minutes20:52
*** cutforth has joined #openstack-swift20:52
*** kota_ has joined #openstack-swift20:53
*** ChanServ sets mode: +v kota_20:53
notmynamein the meeting we'll be going over the patches on https://wiki.openstack.org/wiki/Swift/PriorityReviews20:53
kota_good morning20:54
hogood morning!20:54
notmynameclayg: I just learned earlier today that (in the old gerrit view at least) if you hover your mouse over a vote, it will tell you who voted for it20:54
notmynamekota_: ho: hello!20:54
honotmyname: kota_: hello and good morning20:55
notmynamemeeting time (1 minute)20:59
mattoliveraumorning21:00
*** agomez has joined #openstack-swift21:03
*** acoles_ is now known as acoles21:03
*** hrou has quit IRC21:04
acolesnotmyname: in the old gerrit view you can set an option to have it show you the reviewer's name by the vote21:05
*** Fin1te has quit IRC21:12
*** delattec has quit IRC21:15
*** cdelatte has quit IRC21:15
acolesnotmyname: settings->preferences->'Display Person Name In Review Category'21:17
*** cdelatte has joined #openstack-swift21:19
*** esker has joined #openstack-swift21:20
notmynameyeah, I had that on for a while. made the display too noisy, I thought21:21
*** trifon has quit IRC21:23
*** agomez has quit IRC21:25
openstackgerritpaul luse proposed openstack/swift: Better error handling for EC PUT path when client goes away  https://review.openstack.org/21133821:31
claygkota_: I think the tests in https://review.openstack.org/#/c/220059 might be easier to follow if you wrote them with specific object server responses - like in unit.proxy.controllers.test_obj.class TestECObjController(BaseObjectControllerMixin, unittest.TestCase):21:39
claygkota_: TestECObjController I ment21:39
*** minwoob_ has joined #openstack-swift21:42
acoleskota_: so in patch 215276 i need to add some checks that the responses we collect in a bucket are conistent e.g. same timestamp, same etag, same status code maybe...so would probably be a natural place to check the ec scheme21:42
patchbotacoles: https://review.openstack.org/#/c/215276/21:42
acoles*consistent21:42
*** minwoob has quit IRC21:43
acoleskota_: the ECGetResponseBucket can get smarter21:43
kota_great comments, reading.21:46
*** minwoob_ is now known as minwoob21:46
*** cdelatte has quit IRC21:53
acoleskota_: i think the 'hard' part with adding more to 215276 will be writing good tests. there are so many variables and i want to be sure we cover them all. ec scheme would be one more.21:54
*** esker has quit IRC21:55
acolespeluse: clayg just want to sanity check something with you...21:55
* mattoliverau is off to find breakfast21:55
peluseacoles: sure, I'm totally not sane if thats th e question21:55
notmynamehttps://wiki.openstack.org/wiki/Swift/PriorityReviews is updated21:56
kota_acoles: agreed. writing good tests is sometimes hard.21:56
kota_so21:56
acolesas a simplification (and boy did i need some) I am only making the obj server report alt frags on same node that are *durable*21:56
claygacoles: why is that simpler?21:57
acolesi'm assuming that multiple frags on same node with no durable is very unlikely21:57
peluseyeah, I had some similar decisions to make in the original one and opted for "hey, alt frags is sorta of a best effort thing so KISS rules for sure"21:57
claygacoles: if the gather_* methods return a list of all the frags the object server could "easily" just return x-alternative-timestamp-frags: ts#, ts#, ts#21:57
* notmyname goes afk for a bit21:58
peluseclayg: yeah, I thought you were looking to see that a while ago btu wondered why it never materialized, or maybe I dreamed about it21:58
acolesclayg: you're right, tbh i got hung up on how to best ship all that info back to the proxy - like would a header with a json dict keyed with timestamp and having lists of frag indexes be OK?21:58
claygpeluse: well it wasn't so clear in my head how it would work until I started to see acoles cahnge come to be21:58
*** esker has joined #openstack-swift21:59
acolesclayg: and then in proxy i think it was simpler if i could assume all alt frags were durable so the existence of one made a bucket 'viable'21:59
claygacoles: it'd be ok - but I think a simple list of serailized ts# would be ok - basicallyt he file names "%(timestamp)s#%(frag_index)"21:59
acoles^^ thats a bit hazy i admit21:59
claygacoles: I was sorta thinking when you poop out the gather'd sorted file list - before you go into "select the most recent bestish set of this crap that would make up a file" - you could just sorta pop out the ts#'s that the proxy says it already has streams setup for22:00
claygso then when you look at the databag the only thing you could possibly come up with is something that be useful?22:00
claygso step 1) sort out everything I have 2) remove what the proxy already has 3) return the most useful looking thing from what I left if any22:01
acolesclayg: sendng filenames exposes diskfile implementation all the way to the proxy22:01
claygacoles: nah - it's just a serialization format for the idea of (timestamp, frag_index)22:01
* peluse has to join another meeting for a bit...22:02
acoleshmmm, ok, maybe22:02
claygif you like {<timestamp>: [<frag_index>, ...], ...} like *way* better than a csv that's ok I think22:02
*** bhakta has quit IRC22:02
claygacoles: sorry to poop on things - but I really am thinking that the whole idea of durable just doesn't belong in the GET path22:03
clayglike if you PUT a replicated object and only one copy survies to disk - you get a 500 - then if you GET and you happen to get that data - GREAT - if not - GREAT!22:03
peluseoh man22:03
claygit's not crazy to imagine that if that one and only one copy of data that was saved and returned you a 500 - disk' DIES - that next time you GET you get something else22:03
claygit's like the 500'd PUT never happened22:04
*** esker has joined #openstack-swift22:04
acolesclayg: ? so if we get ndata frags and none have a durable we go ahead and return the data?22:04
claygeven if you were able to successfully read it ONCE that ONE time22:04
kota_Ah, yeah, I got it about the ECGetResponseBucket22:04
*** proteusguy__ has quit IRC22:04
claygso on a GET to EC data the proxy and object server can ignore the whole consistency engine problem and return the best thing they can make up - durable or not really doesn't matter that much - trust the consistency engine will make the object readable if it can22:05
*** proteusguy__ has joined #openstack-swift22:05
*** bhakta has joined #openstack-swift22:05
kota_hmm, quick look at 215276, seems easy way to make another validation.22:05
acolesclayg: but if there is no durable then that object will eventualy be deleted, no?22:05
claygI mean you could look at if it's durable or not - but if you can rebuild it - it *should* be durable - we're working hard on the PUT path to make sure that we never return 200 on an object that we're not going to be able to reconstruct - so let's use that simplicity on GET22:05
MooingLemurclayg: found it!  I see why my db replicators kept coming back with 0 containers/accounts to sync22:06
claygacoles: but that's what i'm saying - same thing in the replciated case - if you only have one copy on disk you can serve it - not so crazy to think that disk may die and you loose it22:06
*** hrou has joined #openstack-swift22:06
acolesclayg: true22:06
MooingLemurclayg: I was binding to a long-form IPv6 address instead of canonical22:06
claygdo we really think that we'll get ndata frags down and not have a single durable out there?22:06
acolesno22:06
acolesnot often22:06
claygacoles: well not often == sometimes at scale - but I acctually have a hunch that we may be closer to practically never if you consider only the cases where we responded 201 on the PUT22:07
claygMooingLemur: good catch!22:07
claygacoles: so... sorry22:08
claygacoles: you can ignore me22:08
claygacoles: but I think the list-of-things-that-is-not-helpful is a good strategy in general over make-specific-request-for-something-that-i-think-this-one-time-might-have-helped22:08
acolesclayg: no. i wouldn't have asked if it wasn't something i'd been considering.22:08
MooingLemurclayg: and whataremyips from swift.common.utils was using the literal bind_ip value rather than normalizing it.  Somehow this wasn't a problem before 2.4.0.  I could also solve this by having the container/account servers bind to :: rather than a specific IP.22:09
claygMooingLemur: amazing - so you're having some sucess running swift on all ipv6 deployment - that's sort of exciting!22:09
*** garthb has quit IRC22:10
MooingLemurclayg: yeah, everything swift-related is IPv6 enabled and all talking IPv6 except for its memcache client :/22:10
*** garthb has joined #openstack-swift22:10
claygMooingLemur: sweet!22:10
claygMooingLemur: what's memcache's problem :P22:10
openstackgerritKota Tsuyuzaki proposed openstack/swift: Fix missing container update  https://review.openstack.org/18673522:11
MooingLemurclayg: memcached supports listening on v6.. was doing that 7 years ago.  memcache.py simply doesn't appear to support IPv6.22:11
claygtorgomatic: so 3 tiers one tier has twice as much as the other two - 1, 1, 2 - what's the needed overload? I still don't intuit this very well :\22:12
claygMooingLemur: oh - well that's totally fixable!22:12
torgomaticclayg: I think it's 0.522:12
claygMooingLemur: you gunna fix it!22:12
torgomaticer, wait... 1.0?22:12
torgomaticyeah, I'll go with that22:13
acolesclayg: so would proxy only send list-of-things-that-is-not-helpful to nodes that have something helpful i.e. something not on that list?22:13
MooingLemurclayg: I mean swift.common.memcached.  it even has AF_INET sprinkled around in it.  I suppose I could give it a try.22:14
*** annegent_ has joined #openstack-swift22:14
claygacoles: idk, i was thinking tactically - instead of stuffing the headers in the alt_node dict - maybe somehow the ResponseGatheringBrain could get a callback hook into the "make_backend_headers"22:14
claygand basically ever time you fire a request - you should down the current most up to date list-of-things-that-are-not-helpful22:15
*** esker has quit IRC22:15
claygthen it doesn't matter if it's a priarmy, or a handoff or an alt frag - as long as it's not responding with something I already have a stream for22:15
kota_mattoliverau: new patch set uploaded on 18673522:16
*** openstackgerrit has quit IRC22:16
acolesclayg: yes. i'd also been thinking about making the alt node thing a callback so proxy does the right thing according to current state rather then state when it dumped out the alt nodes22:16
*** openstackgerrit has joined #openstack-swift22:16
acolesclayg: got it.22:16
clayg:\22:16
claygyeah - sorry - seriously - i feel like such an armchair qb even suggesting all this :\22:17
acolesno its cool22:17
claygs/armchair qb/heckler from peanut gallery/ if that makes more sense22:17
acolesbackseat driver?22:18
claygmaybe there's a way to convey my guilt w/o euphemism even, not sure22:18
claygacoles: yes thank you!22:18
*** annegentle has quit IRC22:18
*** esker has joined #openstack-swift22:19
claygtorgomatic: so but overload is amplified per device somehow like (1, 1), (1, 1), (2, 2) doesn't end up needing as much overload as (1,), (1,), (2,) somehow?22:19
torgomaticit's how much extra each device is willing to take... to balance out a 1-1-2 situation, you need to boost it up to a 2-2-2 so nobody is more than 1/3 of the weight22:20
*** jlhinson has quit IRC22:20
claygso but then (1, 1), (1, 1), (2, 2) *should* still need 1.0 to work :\22:21
acolesclayg: ok i'll go look at that.22:21
acolesclayg: another thing i have in mind is not assuming that responses with same timestamp implies same etag. I think I need to make the buckets cope with multiple etags just to be safe. Or do we just let the decoder blow up?22:23
claygacoles: heh22:23
*** esker has quit IRC22:24
claygacoles: I don't think anyone would notice if you grouped by timestamp instead of etag - we all seemed to think that was the easiest thing to reason about22:25
acolesclayg: i am grouping by timestamp now.22:25
claygacoles: in some ways the decoder would be happiest grouped by etag - but we might all loose our minds - so maybe it's better to use timestamp and ignore consistency issue22:25
claygacoles: ok, try not think about the craziness then22:26
acolesbut i was thinking i can group by (timestamp, etag)22:26
acolesat least for real responses, not for alt frags22:26
acolesbut then my mind drifts to (timestamp, etag, status code)22:27
acolesand kota_ wants to add ec scheme :P22:27
kota_acoles: exactly :\22:28
acolesclayg: ok its late here, watch this space, your X-Don't-Be-Unhelpful list thing is growing on me.22:28
claygacoles: kota_: wfm - tuples can be pretty longish22:28
claygacoles: have a good night - thanks for being awesome22:29
acolesclayg: can a header name have a ' ? :P22:29
acolesas in Don't22:29
acolesjk22:29
acolesgood night22:29
kota_acoles: good night!22:30
acoleskota_: sorry, good morning for you!22:30
*** acoles is now known as acoles_22:30
*** wbhuber has quit IRC22:31
*** jamielennox is now known as jamielennox|away22:35
*** alejandrito has quit IRC22:36
mattoliverauMorning (I'm back)22:39
mattoliverauacoles: night22:39
mattoliveraukota_: cool will take a look22:40
*** annegent_ has quit IRC22:49
*** jerrygb has quit IRC22:55
*** km has joined #openstack-swift23:03
*** annegentle has joined #openstack-swift23:04
*** annegentle has quit IRC23:05
*** kota_ has quit IRC23:06
*** agomez has joined #openstack-swift23:12
*** chsc has quit IRC23:12
*** garthb has quit IRC23:13
*** kei_yama has joined #openstack-swift23:24
*** _hrou_ has joined #openstack-swift23:46
*** minwoob has quit IRC23:48
*** hrou has quit IRC23:49
*** jamielennox|away is now known as jamielennox23:52
*** peluse has quit IRC23:54
*** peluse has joined #openstack-swift23:55

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!