Thursday, 2015-08-06

*** aerwin has quit IRC00:06
*** annegentle has quit IRC00:18
openstackgerritMerged openstack/swift: Add extra_header_count to document and config.  https://review.openstack.org/20952500:25
openstackgerritMerged openstack/swift: do container listing updates in another (green)thread  https://review.openstack.org/18908000:27
openstackgerritMerged openstack/swift: Cleanup nits from container_update_timeout  https://review.openstack.org/20481300:27
claygwho look out - puts just got faster!00:30
*** tsg has quit IRC00:33
*** bapalm has joined #openstack-swift00:37
notmynamewoohoo00:37
*** bapalm has quit IRC00:39
*** minwoob has quit IRC00:40
*** gyee has quit IRC00:41
*** nakagawamsa has joined #openstack-swift00:55
*** robefran has joined #openstack-swift00:57
*** bill_az has quit IRC01:06
*** nakagawamsa has quit IRC01:10
*** chsc has quit IRC01:14
*** annegentle has joined #openstack-swift01:14
*** kota_ has quit IRC01:27
*** robefran has quit IRC01:30
*** logan2 has quit IRC02:08
*** logan2 has joined #openstack-swift02:09
*** darrenc is now known as darrenc_afk02:12
*** nakagawamsa has joined #openstack-swift02:21
openstackgerritThiago da Silva proposed openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20964902:24
*** hezhiqiang has joined #openstack-swift02:24
*** km has quit IRC02:28
*** km has joined #openstack-swift02:28
*** tsg has joined #openstack-swift02:31
*** tsg has quit IRC02:36
*** aagrawal has quit IRC02:43
*** jrichli has joined #openstack-swift02:50
*** km has quit IRC03:03
*** km has joined #openstack-swift03:04
*** silor has joined #openstack-swift03:07
*** sanchitmalhotra has joined #openstack-swift03:07
*** logan2 has quit IRC03:14
*** sanchitmalhotra1 has joined #openstack-swift03:24
*** dmorita has quit IRC03:25
*** dmorita has joined #openstack-swift03:25
*** sanchitmalhotra has quit IRC03:26
pelusecharz, you there?03:28
*** sakaYK has joined #openstack-swift03:29
charzpeluse: yeah,03:29
*** breitz has joined #openstack-swift03:29
peluseahh, just added a note to the bug :)03:30
pelusequestion actually, take a quick look03:30
pelusebut how do you end up with just one token that works for both PACOs? You cmd line has just one right and don't you need to get one from each PACO?03:30
*** logan2 has joined #openstack-swift03:31
charzpeluse: I think the memcached will take that part. proxies query memcached to see the token is expired or not.03:32
peluseso still don't quite understand, you have 2 totally independnt PACO nodes right?03:33
pelusebasically 2 clusters03:33
charzpeluse: no, that's one cluster with 2 PACO nodes.03:33
peluseohhhh03:33
charzLOL03:34
peluseso the proxy service isn't being used on one of them?03:34
peluseif that's the case then how does ssbench talk to the other node?  I'm so confused :)03:35
*** marzif__ has joined #openstack-swift03:35
*** darrenc_afk is now known as darrenc03:36
charzlet me explain it more clear. here is a cluster with two PACOs node that include proxy/account/container/object servers are running on it.03:36
peluseok, go on...03:36
charzAnd I setup two benchmark nodes(other VMs), ssbench installed on both node.03:37
pelusewhoa03:37
charzI get a token from one proxy of PACOs node, and then trigger ssbench-master to against two proxies(PACOs).03:39
charzI skip the load balancer in front of cluster.03:39
peluseyeah, I get that there's no LB03:40
charzAnd here is one ssbench-master in each benchmark node.03:41
*** dmorita has quit IRC03:42
charzI uses the command I post in gist.03:42
peluseI think the thing I'm missing here is that I've not manually setup a cluster w/>1 proxy on it before.  I either just use SAIO or the SwiftStack product.  Guess I need to RTFM03:43
pelusesomy current setup is just 2 single node clusters which is why I'm confused by having just one token03:43
pelusegotta run, will get my stuff fixed up sometime between now and the hackathon I hope.  Have a busy 2 days before heading out there...03:45
pelusethanks!03:45
charzpeluse: yeah, I'll confuse that too.03:45
mattoliverauWhen I used ssync I had to set up keystone so all the proxies could use the same token03:45
charzpeluse: np, just ping me if you have problems.03:46
*** aagrawal has joined #openstack-swift03:50
*** ppai has joined #openstack-swift04:03
mattoliverau*ssbench (lol too much swift fingers autocomplete)04:03
*** zaitcev has quit IRC04:10
*** early has quit IRC04:16
*** early has joined #openstack-swift04:18
*** sakaYK has quit IRC04:19
*** sakaYK has joined #openstack-swift04:20
*** dmorita has joined #openstack-swift04:24
openstackgerritHisashi Osanai proposed openstack/swift: Restrict account_autocreate behavior with allow_account_management  https://review.openstack.org/20975004:25
*** tsg has joined #openstack-swift04:25
*** sakaYK has quit IRC04:25
*** sakaYK has joined #openstack-swift04:26
*** breitz has quit IRC04:35
honotmyname: thanks for the explanation. I created a patch for it. :-)04:40
notmynameho: interesting. thanks04:40
*** annegentle has quit IRC04:41
notmynameho: I say "interesting" because I'm not sure that it's what is expected04:41
notmynamei'll take a look04:41
*** jrichli has quit IRC04:41
notmyname(later)04:41
*** tsg has quit IRC04:43
*** tsg has joined #openstack-swift04:44
*** ppai has quit IRC04:50
charzho: the closes-bug link is broken or that's incorrect bug id(1482041)?04:52
hocharz: it looks correct. https://bugs.launchpad.net/swift/+bug/148204105:02
openstackho: Error: malone bug 1482041 not found05:02
*** ppai has joined #openstack-swift05:04
honotmyname: thanks!05:09
*** itlinux has quit IRC05:09
StevenKcharz: That bug is most likely private05:10
*** hrou has quit IRC05:10
StevenKWhich means the bot can't do anything to it05:10
charzStevenK: yeah, ho just told me. That's private bug.05:10
charzStevenK: thx05:11
*** SkyRocknRoll_ has quit IRC05:14
*** sakaYK has quit IRC05:14
*** sanchitmalhotra has joined #openstack-swift05:20
*** sanchitmalhotra1 has quit IRC05:22
*** sanchitmalhotra1 has joined #openstack-swift05:29
*** sanchitmalhotra has quit IRC05:31
*** ppai has quit IRC05:33
*** ppai has joined #openstack-swift05:45
*** ho has quit IRC05:46
*** ho has joined #openstack-swift05:46
openstackgerritMerged openstack/swift: Fix TypeError if backend response doesn't have expected headers  https://review.openstack.org/20937005:51
openstackgerritMerged openstack/swift: Having said H, I, J, we ought to say K  https://review.openstack.org/20967705:56
openstackgerritMerged openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20964205:56
openstackgerritMerged openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20964305:57
*** mahatic has joined #openstack-swift05:57
*** sakaYK has joined #openstack-swift06:13
*** sanchitmalhotra has joined #openstack-swift06:14
*** sanchitmalhotra1 has quit IRC06:16
*** sakaYK has quit IRC06:19
*** SkyRocknRoll has joined #openstack-swift06:20
*** sakaYK has joined #openstack-swift06:21
*** dmorita has quit IRC06:22
*** sakaYK has quit IRC06:25
*** sakaYK has joined #openstack-swift06:26
*** marzif__ has quit IRC06:39
*** sakaYK has quit IRC06:44
*** tsg has quit IRC06:45
*** sakaYK has joined #openstack-swift06:45
*** sakaYK has quit IRC06:50
*** sakaYK has joined #openstack-swift06:51
*** sakaYK has quit IRC06:57
*** sakaYK has joined #openstack-swift06:58
*** sakaYK has quit IRC07:03
*** sakaYK has joined #openstack-swift07:04
openstackgerritMatthew Oliver proposed openstack/swift-specs: Update sharding spec, notably CountingTrie and shrinking  https://review.openstack.org/20100007:06
mattoliverau^ latest SPEC update (or braindump) now including initial version of shrinking sharded containers (collapsing).07:07
openstackgerritChristian Schwede proposed openstack/swift: Make swift-ring-builder filename usage more consistent  https://review.openstack.org/20978707:16
*** sanchitmalhotra1 has joined #openstack-swift07:21
*** sanchitmalhotra has quit IRC07:23
*** sakaYK has quit IRC07:23
*** sakaYK has joined #openstack-swift07:24
*** sakaYK has quit IRC07:30
*** sakaYK has joined #openstack-swift07:30
*** proteusguy has quit IRC07:34
*** sakaYK has quit IRC07:36
*** sanchitmalhotra has joined #openstack-swift07:36
*** sakaYK has joined #openstack-swift07:37
*** sanchitmalhotra1 has quit IRC07:38
openstackgerritJamie Lennox proposed openstack/python-swiftclient: Add some logging debug for authentication  https://review.openstack.org/20980007:42
*** sakaYK has quit IRC07:43
*** sakaYK_ has joined #openstack-swift07:43
*** sakaYK_ has quit IRC07:48
*** sakaYK has joined #openstack-swift07:49
openstackgerritJamie Lennox proposed openstack/python-swiftclient: Add some logging debug for authentication  https://review.openstack.org/20980007:50
*** proteusguy has joined #openstack-swift07:52
*** sakaYK has quit IRC07:53
*** sakaYK has joined #openstack-swift07:54
*** geaaru has joined #openstack-swift07:54
*** sakaYK has quit IRC07:56
*** jordanP has joined #openstack-swift07:57
*** sakaYK has joined #openstack-swift07:57
*** sakaYK has quit IRC08:00
*** sakaYK has joined #openstack-swift08:00
*** sakaYK has quit IRC08:05
*** sakaYK has joined #openstack-swift08:05
*** ttrumm has joined #openstack-swift08:09
*** saltsa_ has quit IRC08:13
*** jistr has joined #openstack-swift08:14
openstackgerritkenichiro matsuda proposed openstack/swift: Fix shebang of commands  https://review.openstack.org/20981108:15
*** SkyRocknRoll has quit IRC08:39
*** SkyRocknRoll has joined #openstack-swift08:39
*** marzif__ has joined #openstack-swift08:54
*** joeljwright has joined #openstack-swift08:54
*** ChanServ sets mode: +v joeljwright08:54
*** mahatic has quit IRC08:57
*** SkyRocknRoll has quit IRC08:59
*** mahatic has joined #openstack-swift08:59
*** SkyRocknRoll has joined #openstack-swift08:59
mahaticacoles_away: morning!09:00
acoles_awaymahatic: hi09:01
* acoles_away notices that his away-nick bot is broken09:01
mahaticah! was wondering if this is a good time to ask. thanks for that info!09:01
*** acoles_away is now known as acoles09:02
acolesmahatic: ok i have adopted my alter-ego09:02
mahaticacoles: :) I'm trying to figure out a way to move forward on those probe tests. For what jrichli asked on etherpad "Is there any use case (aside from testing) that we would need to decrypt what is received from direct_get_object?  Also, is it ok to skip those tests altogether when using encryption"09:04
mahaticacoles: from my knowledge, I don't see a need to test on direct_get_object since it's for swift backend09:04
*** sakaYK has quit IRC09:05
mahaticacoles: to test encryption on* If you think otherwise, please let me know and maybe I could go in that direction09:06
*** sanchitmalhotra1 has joined #openstack-swift09:08
mahaticmaybe it is needed somewhere, but I'm not familiar with such a usecase09:10
*** sanchitmalhotra has quit IRC09:10
acolesmahatic: hmmm, i have thought a bit about that and still not sure how best to proceed. probe tests assume a certain configuration (basically, the SAIO setup). Question is, does encyption become part of that 'standard' setup?09:12
*** sakaYK has joined #openstack-swift09:13
acolesmahatic: so far my thinking is 'no', at least not encryption enabled by default. I could be wrong, but my guess we will not move to SAIO encrypting by default.09:14
acolesmahatic: some probe tests are conditional e.g. erasure code specific tests which Skip if no EC policy is found.09:14
mahaticacoles: oh, that's interesting - conditional probe tests09:15
acolesmahatic: but i'm not a fan of making all existing probe tests conditional on encryption NOT being enabled. So I still think that having a way to dynamically enable encryption for specific (new perhaps) probe tests is a better way forwards.09:16
mahaticacoles: that sounds right09:17
acolesi.e. for testing, crypto middleware could be in the proxy pipeline but not enabled by default. that way all probe existing tests should pass.09:17
acolesthen we have a mechanism for a test to enable crypto eg by setting a header that our trivial keymaster would detect.09:18
mahaticright09:18
acolesBut then I  am wondering how that sits w.r.t. functional testing - does that mean that functional tests have to set that header? yuk :/09:19
mahaticwhy not? coz there will be too many?09:20
*** haomaiwang has quit IRC09:21
acolesmahatic: just need take a call. bbiab09:21
*** haomaiwang has joined #openstack-swift09:21
* mahatic looksup bbiab - got it09:22
* mahatic is only used to brb ;)09:22
acolesmahatic: sorry, back now09:29
mahaticacoles: okay, np09:29
acolesmahatic: We do now have a way for functional tests to target a specific policy, so if we made out triv keymaster only enable crypto for a specific policy then that might be a way.09:29
acolesthen crypto specific probe tests use that policy or skip if its not found.09:30
mahaticacoles: okay. we will still need to tests specific to encryption, in a case where encryption is enabled at the account level irrespective of the policy I think?09:31
mahaticneed tests*09:31
mahatics/encryption/crypto09:33
mahaticacoles: I mean how do we test those?09:35
acolesmahatic: not sure i follow. probe or functional test?09:35
mahaticacoles: crypto could be enabled not just based on policy, correct?09:36
mahatic"Vinz also checks the account metadata for a metadata item35009:38
mahatic   'X-Account-Sysmeta-Vinz-Encrypt: always' that a sys admin may have set. If35109:38
mahatic   present Vinz will specify object encryption regardless of the container35209:38
mahatic   policy."09:38
mahaticacoles: excuse those line numbers09:38
mahaticacoles: do I make sense?09:39
acolesoic. yes. so let me clarify - func tests should pass against *any* cluster. some func tests skip if the cluster doesnt have certain features. So yes, whatever keymaster your cluster uses, func tests should pass.09:40
mahaticokay09:41
acolesWhat i was trying to address is when you have an SAIO with our triv keymaster that we dont want enabled *all the time* (cos it would break probe tests) - then how do we force func tests to exercise crypto?09:41
acolesAnd I was thinking we could do that by running the func tests using a specific policy, and the policy would trigger the trivial keymaster to enable crypto.09:42
mahaticright, I think I moved a lil away from it09:42
mahaticcorrect09:42
acolesSo I could e.g. func test my SAIO without crypto and also with crypto enabled by choosing the func test policy.09:43
mahaticacoles: yes, basically it does (should) cover all of that I mentioned. thanks for the explanation!09:44
acolesmahatic: anyway, in terms of existing probe tests, i dont think we should rush to change them to work with crypto enabled. If we write new crypto specific probe tests we should make them skip if crypto middleware is not installed.09:45
mahaticacoles: okay. sounds right09:46
acolesmahatic: i still advocate adding ciphertext checking to the unit tests in the near term09:46
acolesas we discussed before, that would be great.09:47
mahaticacoles: ah. so in the same unit test that I refered? or as you suggested later, a new test?09:47
*** hezhiqia_ has joined #openstack-swift09:47
*** hezhiqi__ has joined #openstack-swift09:48
acolesmahatic: i think jrichli said the existing test mocked the crypto engine, so yeah maybe a new unit test class that would be similar but use a real instance of the crypto engine.09:49
*** hezhiqiang has quit IRC09:49
acolesmahatic: or look at why the cryto engine needs to be mocked in existing? is that necessary? (I need to go study that some more)09:49
openstackgerritHiroshi Miura proposed openstack/python-swiftclient: change deprecated assertEquals to assertEqual  https://review.openstack.org/20833109:50
acolesmahatic: i'll add some notes to ether pad based on what we just discussed.09:50
mahaticacoles: sure, I haven't looked into the why part yet, will do that09:50
mahaticacoles: sure, that'd be great09:51
acolescool09:51
acolesmahatic: are you coming to austin next week?09:51
mahaticacoles: nope! :( but for any discussion around this, I'd love jump in through skype if that's possible09:52
*** hezhiqia_ has quit IRC09:52
acolesmahatic: ah, shame. yes we should try to make that happen.09:53
*** silor has quit IRC09:54
*** sanchitmalhotra has joined #openstack-swift09:58
*** sanchitmalhotra1 has quit IRC10:01
*** jamespage has joined #openstack-swift10:01
*** joeljwright has quit IRC10:04
*** hezhiqiang has joined #openstack-swift10:05
*** hezhiqi__ has quit IRC10:06
*** hezhiqia_ has joined #openstack-swift10:07
*** hezhiqiang has quit IRC10:10
*** kei_yama has quit IRC10:15
*** joeljwright has joined #openstack-swift10:16
*** ChanServ sets mode: +v joeljwright10:16
*** hezhiqia_ has quit IRC10:19
*** sakaYK_ has joined #openstack-swift10:21
*** sakaYK has quit IRC10:22
*** ho has quit IRC10:23
*** iepupp_ has joined #openstack-swift10:24
iepupp_why https://review.openstack.org/#/q/topic:bp/object-patch-support,n,z was Abandoned?10:26
*** eandersson has joined #openstack-swift10:26
*** sakaYK_ has quit IRC10:42
*** SkyRocknRoll has quit IRC11:08
mattoliverauiepupp: first one because back in 2013, if there was a negative score and no action for a few weeks it gets auto abandoned, the other 2 were abandoned by the author (a long time ago)11:13
*** SkyRocknRoll has joined #openstack-swift11:20
*** km has quit IRC11:29
*** km has joined #openstack-swift11:29
*** eandersson has quit IRC11:35
*** eandersson has joined #openstack-swift11:36
openstackgerritJamie Lennox proposed openstack/python-swiftclient: Add some logging debug for authentication  https://review.openstack.org/20980011:36
*** iepupp_ has quit IRC11:41
*** silor has joined #openstack-swift11:45
zigohttps://buildd.debian.org/status/fetch.php?pkg=python-pyeclib&arch=arm64&ver=1.0.8-1&stamp=143877026011:50
zigoany idea what's going on?11:50
*** mahatic_ has joined #openstack-swift11:57
*** mahatic has quit IRC11:57
*** mahatic_ has quit IRC11:59
*** mahatic_ has joined #openstack-swift12:00
*** bill_az has joined #openstack-swift12:07
jordanPzigo, you are packaging python-pyeclib-1.0.8 but you have a system with liberasurecode-dev (1.0.5-2)12:17
jordanPzigo, this looks wrong to me12:18
*** yuanzz has quit IRC12:34
*** bapalm has joined #openstack-swift12:44
*** zul has joined #openstack-swift12:46
zigojordanP: Makes sense! Good catch. I'll fix the build-deps.12:49
*** marzif__ has quit IRC12:53
*** marzif__ has joined #openstack-swift12:54
*** nakagawamsa has quit IRC12:55
*** km has quit IRC12:56
tdasilvajust ran into a similar issue with liberasurecode-dev: http://ur1.ca/ndblr12:58
tdasilvaclayg, peluse ^^^12:58
*** aagrawal has quit IRC13:02
*** tsg has joined #openstack-swift13:05
portantetdasilva, notmyname: check out https://github.com/elastic/packetbeat, for those that want to integrate with ElasticSearch, might be cool for swift13:06
*** mahatic_ has quit IRC13:12
*** robefran has joined #openstack-swift13:15
*** ppai has quit IRC13:19
*** hrou has joined #openstack-swift13:21
*** dustins has joined #openstack-swift13:22
*** thumpba has joined #openstack-swift13:23
*** thumpba_ has joined #openstack-swift13:26
*** vinsh_ has joined #openstack-swift13:27
*** doxavore has joined #openstack-swift13:29
*** vinsh has quit IRC13:29
doxavoreis there any documentation on what causes quarantined objects (as reported by swift-recon)?13:30
*** thumpba has quit IRC13:30
*** marcusvrn has joined #openstack-swift13:30
*** thumpba_ has quit IRC13:30
*** petertr7_away is now known as petertr713:31
*** thumpba has joined #openstack-swift13:36
*** rledisez has joined #openstack-swift13:36
*** proteusguy has quit IRC13:37
openstackgerritEran Rom proposed openstack/swift: Container-Sync to iterate only over synced containers  https://review.openstack.org/20580313:37
*** vinsh has joined #openstack-swift13:39
rledisezhi onovy13:39
rledisezonovy: i have a review ongoing (https://review.openstack.org/#/c/192186/) and clayg added a comment about yours13:39
rledisezonovy: basically, he's suggesting to add the rsync module name in the ring, like you did for the port13:40
rledisezonovy: i was wondering what is your use case? because it could match mine too :)13:41
*** vinsh_ has quit IRC13:42
*** SkyRocknRoll has quit IRC13:45
*** breitz has joined #openstack-swift13:49
*** proteusguy has joined #openstack-swift13:53
*** thurloat_isgone is now known as thurloat13:53
tsubicare there any open source application examples of swift usage? I would like to just take a look at a programming model for using swift. Thanks13:57
*** breitz has quit IRC13:58
*** dustins has quit IRC13:59
jamespagejordanP, matching the liberasurecode versions appears to improve the unit testing situation in Debian - thanks for that pointer14:00
jordanPjamespage, that's good to know14:00
*** mahatic has joined #openstack-swift14:01
jamespagejordanP, I think we're seeing some endian type issues on two archs - just waiting to confirm that14:01
jamespagewell at least tests fail on two big-endian architectures14:01
jamespagehttps://buildd.debian.org/status/package.php?p=python-pyeclib&suite=sid14:02
jamespagefor reference14:02
*** jrichli has joined #openstack-swift14:05
*** tsg has quit IRC14:10
*** joeljwright has quit IRC14:11
*** jlhinson has joined #openstack-swift14:14
*** dustins has joined #openstack-swift14:17
*** ttrumm has quit IRC14:21
*** saltsa has joined #openstack-swift14:22
*** joeljwright has joined #openstack-swift14:25
*** ChanServ sets mode: +v joeljwright14:25
openstackgerritBill Huber proposed openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20965214:30
*** dmorita has joined #openstack-swift14:34
*** jasondot_ is now known as jasondotstar14:38
*** breitz has joined #openstack-swift14:41
*** breitz has joined #openstack-swift14:41
*** marzif__ has quit IRC14:43
*** marzif__ has joined #openstack-swift14:44
*** dustins_ has joined #openstack-swift14:45
*** annegentle has joined #openstack-swift14:45
*** MVenesio has joined #openstack-swift14:45
*** dustins has quit IRC14:47
MVenesioHi guys, i'm running swift "Juno" version in 4 Ubuntu Boxes with 8 disks each. And i'm seeing  [xfsaild(sdx] cousing 90% of I/O in all disks, while swift process are generating just 20% of I/O14:48
*** delattec has joined #openstack-swift14:48
MVenesioAny idea about this ?14:48
*** bill_az has quit IRC14:54
notmynamegood morning14:56
*** bill_az has joined #openstack-swift14:57
*** Fin1te has joined #openstack-swift14:58
MVenesiogood morning14:59
*** bapalm_ has joined #openstack-swift15:00
*** dustins_ is now known as dustins15:00
*** bapalm has quit IRC15:01
openstackgerritBill Huber proposed openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20997615:02
*** bill_az has quit IRC15:04
*** lpabon has joined #openstack-swift15:04
*** bill_az has joined #openstack-swift15:05
notmynameMVenesio: I don't knwo about that. quick google searches don't reveal much. perhaps ahale or donagh might have seen something like that and have ideas15:05
MVenesionotmyname: perfect i'll ask them, thanks15:06
*** zul has quit IRC15:10
*** gyee has joined #openstack-swift15:14
*** geaaru has quit IRC15:21
*** breitz has quit IRC15:22
*** breitz has joined #openstack-swift15:22
*** lcurtis has joined #openstack-swift15:30
*** minwoob has joined #openstack-swift15:31
*** blmartin has joined #openstack-swift15:33
mahaticacoles: re do we need to mock crypto in enginer test_encrypter: It makes a call to Encrypter (class) and mocks crypto engine, so there isn't a way the actual encryption/decryption is tested15:33
*** janonymous_ has joined #openstack-swift15:34
mahaticjrichli: why exactly should the ecrypter be ignorant of crypto engine?15:34
mahaticin test_encrypter15:34
mahaticacoles: jrichli: I don't particularly see a need to mock it, when we might call the real crypto and do real testing of encryption/decryption?15:35
*** bill_az has quit IRC15:35
*** bill_az has joined #openstack-swift15:36
*** nadeem has joined #openstack-swift15:39
*** tongli has joined #openstack-swift15:44
*** lpabon has quit IRC15:44
*** silor has quit IRC15:47
lcurtishello all...i swapped out a disk yesterday and am seeing  ERROR Container update failed (saving for async update later): 404 response from x.x.x.x:6001/disk-x15:49
*** sanchitmalhotra1 has joined #openstack-swift15:50
lcurtiscontainer server is quite busy iowait of 2-3....would that be overall reason?15:51
jrichlimahatic: we do want to add a test that integrates with the real crypto.  but we want to do this in such a way that a different crypto algorithm could be used later, if desired, with no modifications to the encrypter test.15:51
jrichlimahatic: but we can easily do this.15:51
*** sanchitmalhotra has quit IRC15:52
*** petertr7 is now known as petertr7_away15:52
mahaticjrichli: but what exactly are testing with this test_encrypter that won't be tested when crypto engine is not mocked? Even if there was a diff crypto algo used later, we'll have new tests per requirement right?15:53
mahaticare we*15:54
*** janonymous_ has quit IRC15:54
*** nadeem has quit IRC15:55
*** SkyRocknRoll has joined #openstack-swift15:57
jrichlimahatic:  I don't think we don't need a new test_encrypter class.  Just add a new test case in the current file that integrates with the real crypto.  You might have to change how the mock is specified, since it is globally applied currently.15:58
jrichlimahatic: yes, somebody could always modify the encrypter tests when crypto changes, but that isn't making crypto very modular.15:58
openstackgerritRomain LE DISEZ proposed openstack/swift: Allows the replicator to rsync to a device-based module  https://review.openstack.org/19218615:59
*** rledisez has quit IRC15:59
jrichlimahatic:  it is a best practice to keep items such as the crypto alg (that you predict might change later) modularized15:59
jrichlimahatic:  I dont think it will be difficult to keep it modular16:01
mahaticjrichli: um that's true. my only question is, what are we specifically testing by mocking crypto engine? If it is only making sure that a call to Encrypter goes fine, can we not integrate it with real crypto itself?16:01
jrichlimahatic: the encrypter is a pretty large class with logic about how the headers are to be encrypted.  There is a lot that can be unit tested in just that class alone.16:03
*** marzif__ has quit IRC16:03
jrichlimahatic: currently, all the tests there are asserting behavior in just the encrypter class.  trust me, they can fail if the encyrpter is not donig its job - has happened to me for sure!16:03
mahaticjrichli: yes I did go through the whole thing, although it does include the logic about how the headers are to be encrypter, we're still faking the encryption part16:04
mahaticjrichli: ah alright16:04
acolesjrichli: but the current crypto engine can't ever go away, correct? so how does it hurt to incorporate it in unit tests?16:04
jrichlimahatic: it does not hurt.  we are suggesting that a test case be added to incorporate it.16:05
mahaticacoles: exactly16:05
mahaticjrichli: do you mean remove the mock part and incorporate the real crypto?16:06
acolesjrichli: it was me that typed that :P16:06
mahatic:D16:06
jrichliacoles: yes, i just noticed that.  :-)16:06
acolesand yes that is what i meant16:06
acolesthen if another engine comes along it may get some unit tests of its own but the existing tests have to still work16:07
jrichlimahatic, acoles: i am a bit of a unit test purist.  I like having tests that test one thing - like encrypter.  I think the integrated test case is a good idea to have in addition16:07
hrouacoles, mahatic, I think the idea is indeed to incorporate it while keeping the encrypter unit tests independent, best of both worlds ? : )16:07
jrichlithe crypto alg has unit tests of its own16:08
jrichliwe did talk about *someday* somebody *might* want to have the crypto alg be configurable.  Not to bring up a bid debate,16:08
mahaticjrichli: acoles: in addition? meaning we're gonna have to repeat all the unit tests with real crypto, correct?16:08
jrichlibut it would be good to not make decisions that will lead to that being impossible16:09
*** potato_farmer has joined #openstack-swift16:09
acolesjrichli: hrou: maybe. i like testing the "whole" as much as possible. its too easy to build modules that are happy with their own tests but diverge from working together.16:09
potato_farmerHi. I am trying to remove a swift storage node. I have balance, partitions, and weight down to 0. I have waited several hours between each try. But anytime I try to remove the IP, I get: "No partitions could be reassigned. Either none need to be or none can be due to min_part_hours [1]." Has anyone run into this before?16:10
jrichlimahatic: no, I figured just a basic PUT test case should be enough to test the integration.16:10
mahaticjrichli: oh I see16:10
hrouacoles, true, it really depends on the attitude you take to unit tests, sometimes folks strive to keep them as independent as possible.  They're fairly isolated here though !  Probably can just start by adding a direct crypto algo test (and I agree this is already fairly module, if for example in the future we support 3DES in the swift engine)16:11
lcurtispotato_farmer what is your min_part_hours set to16:11
*** jistr has quit IRC16:12
potato_farmerlcurtis: I am still pretty new to this, how can I check?16:14
potato_farmerlcurtis: based on the error, isn't it [1] ?16:16
jrichlimahatic acoles: I am open to changing the tests to all be more integrated, if you think that is the best direction16:17
acoleshrou: i just see existing tests where we could easily verify that stuff got encrypted/decrypted correctly, except the engine is mocked out. So instead we'll have to write another set of test that would either duplicate or not have complete coverage.16:17
acolesjrichli: happy to debate next week. i know folks have different views on test strategies.16:18
*** jordanP has quit IRC16:18
mahaticacoles: jrichli I won't be around next week! :\ debate here next week, on IRC :P16:19
*** doxavore has quit IRC16:19
jrichliacoles mahatic: no debate necessary on my end.  if you all think that it's best to have all the test cases be integrated, I am on board.16:19
jrichliand that way, mahatic can start making the mods16:20
mahaticthat's true though16:21
hrouacoles, back in my large software dev days (DB2) we were beaten with sticks when unit tests were not 100% isolated even if the end result was 100% more unit tests ; )  But its probably easy to change this all going forward either way !16:21
*** nadeem has joined #openstack-swift16:22
hroumahatic, speaking of encryption, quick question for you - did you have to play around with extra_header_count to achieve the test results cited on the etherpad ?16:22
acolesjrichli: mahatic : so a first step would be to remove the crypto mock and see what breaks in existing tests?16:22
acoleshrou: yep, understand, its just what you perceive to be a unit and i'm not convinced of the separation of the encrypter and crypto engine.  The current engine isn't going to go away even if others come along, so I see it as the built in default if you like.16:24
openstackgerritBill Huber proposed openstack/swift: Replace python print operator with print function (pep H233, py33)  https://review.openstack.org/21001416:24
lcurtispotato_farmer run the swift-ring-builder command against your ring file:   swift-ring-builder /etc/swift/object.builder16:25
lcurtisu should see: The minimum number of hours before a partition can be reassigned is X16:25
potato_farmerlcurtis: It shows "1" for account, container, and object16:26
*** hezhiqiang has joined #openstack-swift16:27
lcurtispotato_farmer so, something is odd then...hopefully one of the gurus that hang out here can assist16:27
mahaticacoles: I could do that16:27
mahaticjrichli: ^16:28
jrichlimahatic: sounds good16:29
*** chsc has joined #openstack-swift16:30
potato_farmerlcurtis: I agree. I have removed nodes in the past without issues.16:30
mahatichrou: oh I have had multiple issues, majorly pip, for whatever reason (I couldn't figure out all of that). But just downloading the code and running the tests should give you those results, did it not happen that way?16:31
hrouacoles, yep I agree the current engine will likely encapsulate the other algorithms (as it hard codes AES per say today) and maybe an account / policy setting dictates what to use.  Either approach will work, I guess someone can argue if you had independent crypto algo unit tests you'd only have to run this X times (where X is the different algo's supported), as opposed to everything.  Its a different story if the crypto was full16:31
hroufledge middleware outside of swift. Maybe the answer is just do both : ) integrate and separate.  But minor either way.16:31
hroumahatic, thanks !  Yep so I actually had to set extra_header_count to some large number, question - did you run the tests fairly recently, I think only as of the last week or so (maybe 2) the account metadata encryption was sent for review ?16:33
mahatichrou: a week and a half ago I believe. Sorry I can't correlate with the review you mentioned, will need to look16:34
hroumahatic, there was a func test that set a bunch of metadata account headers => we add a bunch of headers in the keymaster, and my theory that was the cause.16:34
*** hezhiqiang has quit IRC16:34
hroumahatic, oh no worries at all, let me dig into it more, maybe its just me ; )  Thanks a bunch though !16:34
acoleshrou: interesting. maybe we bloat the backend headers enough to exceed the max header count. thanks for mentioning that!16:35
jrichlihrou: I know that I have never set this.  But maybe setting it would fix something that has always failed for me, I don't know.16:35
jrichlihrou: you said that your failures match those that I have posted - that was even after you changed this config?16:36
mahatichrou: I could try and reproduce this in another vm (tomm sometime)16:36
hroujrichli, acoles - yep exactly, that was my thought, and on paper it definitely seems possible - by default we only allow 122 headers (this was recent change to the HTTP python lib)16:36
hrouacoles, if you recall clayg (I think, sorry if I'm wrong !) delivered a patch to main that allowed you to increase this through swift.conf, using  extra_header_count and that did the trick for me.16:37
jrichlihrou: what test failed before that now succeeds?  or do we have a test that is asserting this behavior?16:38
hroumahatic, sure that'd be great, make sure to pull down all the recent reviews and rebase to latest (I did it manually) there was only a minor conflict, let me know if you have any issues.16:38
*** annegentle has quit IRC16:38
jrichlihrou mahatic: I have not yet rebased, so maybe it was from master that somebody added a test to assert this.16:38
acoleshrou yeah it was Christian I think who fixed up the header count and added the extra_header_count option16:38
hroujrichli, I'm going to check now (I'll rerun with it off), it wasn't directly testing it from what I could see, we were just hitting the defined limit.  I actually noticed it in many places.16:38
jrichlimahatic: I think you can make progress in the unit tests without rebasing.  you probably want to wait on that, because i will be making changes you will want to pick up in a day anyway16:40
mahatichrou: could you point me to that patch of extra_header_count?16:40
hroujrichli, mahatic - 100% agreed to the above, don't bother with that !  You may run into other issues.  Or if you're using a VM, clone it first : )16:40
mahaticjrichli: sure, I was actually going to do this in a diff VM - which initially gave me many errors afte downloading the ecnryption code - and i didn't delete the VM yet, so I'm just curious and ready to verify quickly :)16:41
*** Fin1te has quit IRC16:42
* jrichli is off to lunch16:43
mahaticjrichli: enjoy your lunch!16:44
acolesmahatic: patch 19594016:44
patchbotacoles: https://review.openstack.org/#/c/195940/16:44
hroujrichli, first time I hit: test_bad_metadata3 (test.functional.test_account.TestAccount) ... FAIL16:44
hrouraise HTTPException("got more than %d headers" % _MAXHEADERS)#012HTTPException: got more than 122 headers (txn: tx60e4e93309cb47b888068-0055c38e84)16:44
mahaticacoles: thanks!16:45
hrouthanks acoles !  Yep its that one, but keep in mind this didn't introduce the issue, this just added a workaround, so we should in theory have seen it before depending on your version of python16:45
acoleshrou sure16:46
acolesmahatic: hrou: also note that the doc of extra_header_count was wrong until recently, fixed here patch 20952516:50
patchbotacoles: https://review.openstack.org/#/c/209525/16:50
acolesjust in case you get confused ;)16:50
mahaticacoles: cool, thanks16:52
*** saltsa has quit IRC16:56
hrouacoles,  thanks !  I'm actually not convinced there still isn't an issue ; ) I found updating test.conf only didn't do the trick I needed to update the swift.conf : ) - this is just for the func tests to pass.  I'll double check though.16:57
*** saltsa has joined #openstack-swift16:58
*** cazino has joined #openstack-swift17:01
*** cazino has left #openstack-swift17:01
*** bapalm_ has quit IRC17:01
acoleshrou: that makes sense - its the number of header from proxy->backend servers that could be exceeded by the crypto m/ware adding extra sysmeta headers.17:02
acolesso you'd need to increase itin swift.conf and restart17:02
hrouacoles, ah, that could be it, but that makes we wonder why we bother documenting it in test.conf17:03
acoleshrou: was just thinking the same :D17:04
hrouacoles, ; - )  I figure the other parameters are more dynamic in nature ? (e.g. max header size)17:05
*** gyee has quit IRC17:05
mahatichrou: you're right, thanks for brining this up! I increased extra_header_count (directly in swift/common/constraints.py) and it worked17:07
hroumahatic, by that do you mean it was failing before hand ?17:07
hroumahatic, i.e. you had failing tests before making that change ?17:07
mahatichrou: unit, func, probe are all giving the expected results. earlier I was getting the same headers issue17:07
hroumahatic, perfect !17:08
hroumahatic, thanks !  be back in a bit.17:08
*** petertr7_away is now known as petertr717:08
mahatichrou: yes, the logs give the same error too -  "more than 122 headers" or the like17:08
*** chlong has quit IRC17:14
openstackgerritBill Huber proposed openstack/swift: Python3: do not use im_self/im_func/func_closure  https://review.openstack.org/21003917:16
* mahatic calls it a day17:16
*** mahatic has quit IRC17:17
*** SkyRocknRoll has quit IRC17:20
*** dmorita has quit IRC17:20
*** joeljwright has quit IRC17:21
notmynamegood morning (again)17:27
hrouacoles, so in terms maximum headers, should we be more prudent then requiring someone to set extra_header_count, not sure its possible but potentially by detecting if the keymaster is in the pipeline on startup and add a fudge factor - really all this archives is passing testing - we can always hit the limit with the trivial keymaster, I'd like to understand more about this limit in httplib, I'll look into it (add an issue to the17:34
hrouetherpad)17:34
*** petertr7 is now known as petertr7_away17:34
*** petertr7_away is now known as petertr717:35
acoleshrou  yes it definitely seems like we will need to consider either increasing the default max_header_count or documenting the need to increase the extra_header_count if using the crypto middlewares.17:36
acoleshrou: good catch btw!17:37
hrouacoles, thanks!  Yea in practice this likely isn't a big deal, but at the very least we'd want to spell out that encryption implies X additional headers per user defined header (regardless of the keymaster or algo in use).17:43
*** marcusvrn has quit IRC17:43
*** eandersson has quit IRC17:47
*** acoles is now known as acoles_17:53
*** thumpba has quit IRC17:54
*** zul has joined #openstack-swift17:56
*** thumpba has joined #openstack-swift17:57
*** breitz has quit IRC17:58
*** bapalm has joined #openstack-swift18:04
*** thumpba_ has joined #openstack-swift18:14
*** nadeem has quit IRC18:15
*** nadeem has joined #openstack-swift18:16
*** thumpba has quit IRC18:17
jrichlihrou mahatic acoles_: I haven't hit this yet because I hadn't rebased.  I am performing rework on a patch earlier in the chain right now.18:21
*** Fin1te has joined #openstack-swift18:21
*** balajir_ has joined #openstack-swift18:22
hroujrichli, thanks that makes sense !  Though is rather interesting, because it makes me wonder what the old httplib max header default was before this change, that is it mentions python 2.7.9+ but I believe you are on an earlier version ?  So I wonder if the older limit was much higher, but then this change breaks it even on older versions of python.18:26
hroujrichli, I say this because your using the same crypto code we are (implying we're both generating the same # of headers).18:26
jrichlihrou: the "extra_header_count" was added on master recently, it seems.  It was added before the patch that acoles pointed out, although patch 209525 does modify that functionality.18:29
patchbotjrichli: https://review.openstack.org/#/c/209525/18:29
jrichlihrou: I assume the test may have been modified to assert that number?  But i have not verified that.  All that I know is that the "bad metadata" tests pass for me right now.18:30
hroujrichli, heh that second patch was just a doc fix up, alcoes mentioned both.18:30
jrichliwhat was the first?18:30
hroujrichli, I don't believe the tests were intentionally stressing that actually, it was just a side effect (many failed).18:31
hroujrichli, first patch that did the real change was: https://review.openstack.org/#/c/195940/18:31
openstackgerritBill Huber proposed openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20997618:36
*** mtreinish has quit IRC18:37
*** mtreinish has joined #openstack-swift18:39
*** gyee has joined #openstack-swift18:40
jrichlihrou:  I haven't looked too deep, but I would guess the change that did this is the following line:18:41
jrichlihttplib._MAXHEADERS = constraints.MAX_HEADER_COUNT18:41
hroujrichli, well see that's the fix ; )  The real issue is when they introduced _MAXHEADERS into httplib in the first place, that was done through this: http://bugs.python.org/issue16037#msg18280318:41
hrouspecifically: http://bugs.python.org/file29201/issue16037_py27.patch18:42
jrichlijust sayin - i think that is why the test didn't fail before, even tho the number of headers was still over18:43
*** bill_az has quit IRC18:48
*** bill_az has joined #openstack-swift18:49
hroujrichli, yep!  I added the issue to the etherpad.18:53
jrichlihrou: thx!19:01
potato_farmerHi. I am trying to remove a swift storage node. I have balance, partitions, and weight down to 0. I have waited several hours between each try. But anytime I try to remove the IP, I get: "No partitions could be reassigned. Either none need to be or none can be due to min_part_hours [1]." Has anyone run into this before?19:02
*** annegentle has joined #openstack-swift19:09
*** zhill_desktop has quit IRC19:14
*** jrichli has quit IRC19:15
*** breitz has joined #openstack-swift19:32
*** thumpba_ has quit IRC19:34
*** potato_farmer has quit IRC19:34
*** thumpba has joined #openstack-swift19:35
*** bill_az has quit IRC19:37
openstackgerritEran Rom proposed openstack/swift: Add process level concurrency to container sync  https://review.openstack.org/21009919:37
*** fifieldt_ has joined #openstack-swift19:40
*** amoturi has joined #openstack-swift19:40
*** fifieldt has quit IRC19:42
*** thumpba has quit IRC19:48
*** thumpba has joined #openstack-swift19:48
*** thumpba has quit IRC19:53
*** bill_az has joined #openstack-swift19:54
*** thumpba has joined #openstack-swift19:55
*** thumpba has quit IRC19:56
*** thumpba has joined #openstack-swift19:57
*** jamespage_ has joined #openstack-swift20:00
*** blmartin has quit IRC20:02
*** alanvitor has joined #openstack-swift20:06
*** _hrou_ has joined #openstack-swift20:09
*** robefran_ has joined #openstack-swift20:10
*** daemontool_ has joined #openstack-swift20:10
*** robefran has quit IRC20:11
*** ajiang has quit IRC20:12
openstackgerritJanie Richling proposed openstack/swift: FakeFooters middleware  https://review.openstack.org/16551720:12
*** ajiang_ has joined #openstack-swift20:12
*** Fin1te has quit IRC20:12
*** hrou has quit IRC20:12
*** logan2 has quit IRC20:12
*** marzif has quit IRC20:12
*** logan2 has joined #openstack-swift20:12
*** thumpba_ has joined #openstack-swift20:12
*** cazino has joined #openstack-swift20:12
*** thumpba has quit IRC20:16
*** marzif has joined #openstack-swift20:17
*** DCWilliams_VA has joined #openstack-swift20:21
*** jrichli has joined #openstack-swift20:23
*** NM has joined #openstack-swift20:25
*** DCWilliams_VA has quit IRC20:26
*** NM1 has joined #openstack-swift20:30
*** NM has quit IRC20:32
*** tongli has quit IRC20:34
*** jamespage_ has quit IRC20:34
*** NM has joined #openstack-swift20:39
*** NM1 has quit IRC20:41
*** sanchitmalhotra has joined #openstack-swift20:48
*** logan2 has quit IRC20:48
openstackgerritThiago da Silva proposed openstack/swift-specs: PACO single-process spec.  https://review.openstack.org/21011720:50
*** bapalm has quit IRC20:50
*** sanchitmalhotra1 has quit IRC20:50
tdasilva^just because we don't have enought specs to review20:50
tdasilvaeikke, bill_az: ^^^ would like to hear your feedback20:51
*** NM has quit IRC20:51
*** dustins has quit IRC20:59
*** thurloat is now known as thurloat_isgone21:02
*** petertr7 is now known as petertr7_away21:06
*** thumpba_ has quit IRC21:08
*** robefran_ has quit IRC21:09
pgbridgeoh that's an interesting spec21:15
*** annegentle has quit IRC21:18
*** jrichli has quit IRC21:21
*** MVenesio has quit IRC21:24
openstackgerritMerged openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20965221:25
*** zhill has joined #openstack-swift21:25
*** zhill has quit IRC21:26
*** zhill has joined #openstack-swift21:27
*** sanchitmalhotra1 has joined #openstack-swift21:38
*** sanchitmalhotra has quit IRC21:40
*** zul has quit IRC21:45
*** nadeem has quit IRC21:52
openstackgerritTim Burke proposed openstack/python-swiftclient: Update mock to get away from env markers  https://review.openstack.org/20515021:52
*** robefran has joined #openstack-swift21:53
*** zul has joined #openstack-swift21:55
*** delattec has quit IRC21:58
*** marzif has quit IRC22:07
mattoliverauMorning!22:17
notmynamehi22:17
mattoliverauLast day before flying out to Austin... I should probably think about packing :)22:18
*** sanchitmalhotra has joined #openstack-swift22:22
*** sanchitmalhotra1 has quit IRC22:24
*** zul has quit IRC22:35
*** alanvitor has quit IRC22:38
*** _hrou_ has quit IRC22:41
*** wbhuber has quit IRC22:41
*** marzif has joined #openstack-swift22:42
*** chsc has quit IRC22:48
*** km has joined #openstack-swift23:00
*** jlhinson has quit IRC23:01
*** marzif has quit IRC23:01
*** logan2 has joined #openstack-swift23:06
*** sc68cal has quit IRC23:09
*** breitz has quit IRC23:12
*** sc68cal has joined #openstack-swift23:14
openstackgerritMerged openstack/python-swiftclient: make Connection.get_auth set url and token attributes on self  https://review.openstack.org/20567223:14
openstackgerritMerged openstack/python-swiftclient: make Connection.get_auth set url and token attributes on self  https://review.openstack.org/20567223:14
openstackgerritMerged openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20964623:15
openstackgerritMerged openstack/swift: pep8 fix: assertEquals -> assertEqual  https://review.openstack.org/20964923:15
lcurtisquestion all...I am seeing network throughput of 25mbps to my container servers but more than 10x that amount written to disk via iostat23:16
lcurtiswhat would cause that kind of write amplification to the sqlitedbs?23:16
*** otoolee has quit IRC23:25
*** otoolee has joined #openstack-swift23:26
*** sanchitmalhotra1 has joined #openstack-swift23:31
*** morganfainberg is now known as morgan_50323:31
*** sanchitmalhotra has quit IRC23:34
notmynameif you propose or review a security patch (in any openstack project), here's how to manage the patches: https://security.openstack.org/#how-to-propose-and-review-a-security-patch23:38
*** breitz has joined #openstack-swift23:38
*** lcurtis has quit IRC23:40
*** zaitcev has joined #openstack-swift23:45
*** ChanServ sets mode: +v zaitcev23:45
mattoliveraunotmyname: good info, thanks :)23:47
notmynameand I just sent that to the ML too :-)23:48
*** ho has joined #openstack-swift23:50
notmynameho: good morning23:50
honotmyname: hello-23:52
tdasilvanotmyname: nice blog post!23:55
notmynamethanks23:55
mattoliverauho: morning23:58
homattoliverau: morning!23:59
*** tsg has joined #openstack-swift23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!