Wednesday, 2015-12-02

*** siva_krishnan has joined #openstack-swift00:00
*** diogogmt has joined #openstack-swift00:00
*** siva_krishnan has quit IRC00:04
*** zhill has joined #openstack-swift00:08
*** blmartin has joined #openstack-swift00:09
*** jerrygb has joined #openstack-swift00:16
*** ho has joined #openstack-swift00:20
hogood morning!00:22
mattoliverauho: morning00:22
homattoliverau: morning!00:23
openstackgerritOpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements  https://review.openstack.org/8925000:39
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873600:39
kota_morning00:42
notmynamehello mattoliverau ho kota_. all you people for whom it's already wednesday00:42
hokota_: notmyname: good morning & hello00:45
mattoliveraunotmyname: o/00:54
*** diogogmt has quit IRC00:55
*** zhill has quit IRC00:55
asettleOhhhh notmyname I was just thinking I needed to talk to you!01:00
notmynameasettle: hi!01:01
asettleHello!01:01
notmynameasettle: if it's about that swiftclient docs stuff, that's what I'm doing now ;-)01:01
asettleIt is! It's like you know me :p haha. I've been snowed under with RPCO stuff since we got back from Tokyo. But my workload is freeing up... so keen to get started. I'm just about to review Joel's latest patch too :)01:02
notmynameasettle: very good!01:02
notmynameasettle: I've been in the same boat, and this week I'm getting that stuff the attention it needs01:02
asettleSplendid! Well. Keep me updated and let me know where I need to be and what I can do to help :)01:03
notmynameasettle: looking at my notes...01:03
notmynameI've put together a rough outline. I want to clean that up some and then send it to you01:04
asettleGreat :) thank you! Looking forward to seeing what you come up with :)01:04
notmynamebut that leads to the real question: what's the actual real thing we need to make progress?01:04
notmynamesometimes I feel like I keep talking about getting started without actually getting started. outlines are good, I guess, but then what? :-)01:04
asettleWell it depends how you want to do it. After we have an agreed upon outline - do we want to put it to the swift community? If so, blueprints ahoy. If we just want to get started, then what I'll do is implement the whole change in one patch (i'm talking place holders, new TOC, all that good stuff) and from there we can then have it approved, merged, and we can work on that. Although we'd have to ensure it's not in maste01:07
asettler so that we don't override the existing documentation.01:07
notmynamethe "good thing" (only in the context of this exact statement) is that based on what we have now (http://docs.openstack.org/developer/python-swiftclient/) even if we overwrite everything we don't really lose anything ;-)01:08
notmynameasettle: I think we need to land the ToC in the repo and then work from that as a TODO list01:09
notmynamesome parts should be docstrings. some should be new .rst files01:09
asettlePerfect. Well, if you're not afraid of losing anything really... we can start working on it pretty much straight away. I am happy to implement the placeholders for info dumps and ... yep, perfect.01:09
asettleExactly that.01:09
notmynameok, I'll clean up my outline and send it over ASAP (if I haven't in 24 hours, you should call me on it)01:10
asettleHahahaha, okay! Well, my release isn't due for another 8 days. So my head will be in that sand bucket, but I should be able to get back to you soon! :)01:10
*** aerwin has quit IRC01:25
*** jerrygb has quit IRC01:50
*** diogogmt has joined #openstack-swift01:54
*** jerrygb has joined #openstack-swift01:55
*** blmartin has quit IRC02:02
*** omkarjoshi has left #openstack-swift02:13
*** wuhg has joined #openstack-swift02:15
openstackgerritMerged openstack/swift: Merge branch 'master' into feature/crypto  https://review.openstack.org/25202402:28
*** david-lyle has joined #openstack-swift02:31
*** haomaiwa_ has joined #openstack-swift02:42
*** sanchitmalhotra has joined #openstack-swift02:47
openstackgerritMerged openstack/swift: Fix crash when a SLO repeats a segment.  https://review.openstack.org/25009902:49
*** haomaiwa_ has quit IRC02:52
*** haomaiwang has joined #openstack-swift02:53
openstackgerritJanie Richling proposed openstack/swift: WIP - Support for http footers - Replication and EC  https://review.openstack.org/23739302:57
*** haomaiwang has quit IRC02:57
*** sgundur has joined #openstack-swift03:00
*** david-lyle has quit IRC03:00
*** sgundur has left #openstack-swift03:00
*** miurahr has joined #openstack-swift03:03
*** venkat_p has joined #openstack-swift03:09
*** changbl has joined #openstack-swift03:13
*** gyee has quit IRC03:16
*** SkyRocknRoll has quit IRC03:16
*** miurahr has quit IRC03:20
*** jerrygb has quit IRC03:34
*** jerrygb has joined #openstack-swift03:34
*** tsg has quit IRC03:37
*** jerrygb has quit IRC03:39
*** ppai has joined #openstack-swift03:40
*** links has joined #openstack-swift03:43
*** mahatic has joined #openstack-swift03:54
*** sanchitmalhotra has quit IRC04:01
*** sanchitmalhotra has joined #openstack-swift04:04
*** km has joined #openstack-swift04:06
*** km is now known as Guest9024604:06
*** Guest75007 has quit IRC04:06
*** kei_yama_ has joined #openstack-swift04:07
*** kei_yama has quit IRC04:08
*** badari has quit IRC04:10
*** badari has joined #openstack-swift04:19
*** jome has joined #openstack-swift04:24
jomeHi. Can anyone help me with this scenario. I'd like to have multiple proxy servers configured behind a load balancer that talk to a single object and account servers. Is this even possible?04:26
jomeFrom documentation that I have gathered, this seems to be possible but I've not seen anything docs that explain hos to configure this.04:27
*** badari has quit IRC04:27
openstackgerritMahati Chamarthy proposed openstack/swift: Modify unit tests to include real crypto  https://review.openstack.org/21145104:33
*** venkat_p has quit IRC04:33
*** venkat_p has joined #openstack-swift04:33
*** SkyRocknRoll has joined #openstack-swift04:34
mattoliveraujome: yeah that's possible, but a single storage server isn't great for object durability. But yeah, the rings just have to container the storage server(s) and all the proxies have the same rings.04:43
mattoliveraualso the load balancer would allow you to have many connections, but they will all go to the one storage node, so you might have a bottle neck there too.04:44
*** david-lyle has joined #openstack-swift04:44
jomeOkay. maybe I did not make it clear. I do have three storage nodes currently04:45
jomeHow do I configure multiple proxy servers in that scenario.04:46
mattoliveraujome: ok, well yeah you can have as many proxies as you want, just set them up as normal, and make sure they have a copy of the same rings.04:47
mattoliveraujome: you can then use the healthcheck middleware in the proxies so the load balancer can check proxy health04:47
mattoliveraujome: the rings are what tell a proxy where to connect. So long as your proxies have access to the storage nodes and have the rings, that's it.04:49
jomeThat simple??? :)04:50
mattoliveraujome: should be, so long as you know how to setup 1 proxy, you can set up n proxies :)04:51
mattoliveraujome: if you had multiple regions then the only difference might be proxy read/write affinity settings, but really there just proxy services with rings that tell them where they can go.04:52
jomeSo if i set up a proxy on a different machine in the same network for example, all i do is copy the rings from the original server and I'm good to go04:53
mattoliveraujome: yup :)04:54
jomeFantastic. I'll give it a spin now04:55
jomeThanks.04:56
mattoliveraujome: that may you can tune your swift cluster to what ever you need, more connections, scale up the proxies, need more redundently/durabiliy or storage, scale add more storage nodes.04:56
openstackgerritMahati Chamarthy proposed openstack/swift: Modify unit tests to include real crypto  https://review.openstack.org/21145104:56
mattoliveraus/may/way/04:56
mattoliveraujome: have fun ;)04:57
jomeWhat about auth though? That would have to be one machine, right?04:57
mattoliveraudepends on what auth your using04:57
mattoliverauif your using keystone, then it's just middles ware configuration which can be the same.04:58
mattoliverauif it's tempauth.. then you need to have it configured on both, then use something like puppet or ansible to manage (not that tempauth should be used for a production cluster).04:59
*** kei_yama_ has quit IRC05:00
*** Guest90246 has quit IRC05:01
*** kei_yama has joined #openstack-swift05:01
*** km has joined #openstack-swift05:02
*** km is now known as Guest6431305:03
*** darrenc is now known as darrenc_afk05:10
*** thumpba has joined #openstack-swift05:14
*** trifon has joined #openstack-swift05:22
*** klrmn1 has quit IRC05:22
*** jome has quit IRC05:27
*** tsg has joined #openstack-swift05:28
*** jome has joined #openstack-swift05:28
*** darrenc_afk is now known as darrenc05:32
*** thumpba has quit IRC05:37
*** jome has quit IRC05:43
*** links has quit IRC05:43
*** jome has joined #openstack-swift05:44
*** rcernin has joined #openstack-swift05:51
*** thumpba has joined #openstack-swift05:52
*** links has joined #openstack-swift05:54
*** thumpba has quit IRC05:59
*** SkyRocknRoll has quit IRC06:03
openstackgerritMahati Chamarthy proposed openstack/swift: Modify unit tests to include real crypto  https://review.openstack.org/21145106:15
*** trifon has quit IRC06:17
*** SkyRocknRoll has joined #openstack-swift06:20
*** rcernin has quit IRC06:25
openstackgerritShu Muto proposed openstack/python-swiftclient: Delete python bytecode before every test run  https://review.openstack.org/25221006:32
*** ChubYann has quit IRC06:38
*** tsg has quit IRC06:40
*** silor has joined #openstack-swift06:57
*** mahatic has quit IRC07:05
*** silor1 has joined #openstack-swift07:06
*** silor has quit IRC07:08
*** silor1 is now known as silor07:08
*** rcernin has joined #openstack-swift07:09
*** jerrygb has joined #openstack-swift07:21
*** jerrygb has quit IRC07:25
openstackgerritChristian Schwede proposed openstack/swift: Add functional test for repeated SLO segments  https://review.openstack.org/25190907:58
*** haomaiwang has joined #openstack-swift08:04
*** arnox has joined #openstack-swift08:10
*** kei_yama has quit IRC08:11
*** kei_yama has joined #openstack-swift08:17
*** shakamunyi has quit IRC08:27
*** jmccarthy has quit IRC08:27
*** jmccarthy has joined #openstack-swift08:27
*** jome_ has joined #openstack-swift08:29
*** jome has quit IRC08:32
*** eranrom has joined #openstack-swift08:33
*** ho has quit IRC08:41
*** trifon has joined #openstack-swift08:44
openstackgerritrenminmin proposed openstack/python-swiftclient: Fix the http request headers logging overwrited  https://review.openstack.org/25226108:46
*** jmccarthy has quit IRC08:48
*** jome_ has quit IRC08:49
*** haomaiwang has quit IRC08:54
*** jome has joined #openstack-swift08:57
*** haomaiwa_ has joined #openstack-swift08:57
*** mahatic has joined #openstack-swift09:06
*** jmccarthy has joined #openstack-swift09:08
*** raginbajin has joined #openstack-swift09:17
*** aix has joined #openstack-swift09:21
*** jodah has quit IRC09:23
*** hogepodge has quit IRC09:38
*** hogepodge has joined #openstack-swift09:41
*** jordanP has joined #openstack-swift09:43
*** hseipp has joined #openstack-swift09:47
*** hseipp has quit IRC09:47
*** hseipp has joined #openstack-swift09:48
*** jistr has joined #openstack-swift09:50
*** haomaiwa_ has quit IRC09:51
*** kei_yama has quit IRC10:00
*** jome has quit IRC10:06
*** Guest64313 has quit IRC10:07
*** jmccarthy has quit IRC10:08
*** jmccarthy has joined #openstack-swift10:20
*** weihan has joined #openstack-swift10:39
*** venkat_p has quit IRC10:45
*** jmccarthy has quit IRC10:49
openstackgerritChristian Schwede proposed openstack/python-swiftclient: Fix debug and info option parsing  https://review.openstack.org/25232810:56
*** acoles_ is now known as acoles10:58
*** venkat_p has joined #openstack-swift10:58
openstackgerritChristian Schwede proposed openstack/python-swiftclient: Fix debug and info option parsing  https://review.openstack.org/25232810:59
*** jmccarthy has joined #openstack-swift11:08
openstackgerritChristian Schwede proposed openstack/python-swiftclient: Fix debug and info option parsing  https://review.openstack.org/25232811:15
*** aix has quit IRC11:34
*** venkat_p has quit IRC11:36
*** asettle has quit IRC11:43
*** asettle has joined #openstack-swift11:45
*** venkat_p has joined #openstack-swift11:48
*** lpabon has joined #openstack-swift11:59
*** hseipp has quit IRC12:01
*** trifon has quit IRC12:02
*** trifon has joined #openstack-swift12:02
*** aix has joined #openstack-swift12:04
*** ppai has quit IRC12:12
*** LiuNanke has joined #openstack-swift12:16
*** LiuNanke has quit IRC12:22
*** hseipp has joined #openstack-swift12:23
*** hseipp has quit IRC12:23
*** hseipp has joined #openstack-swift12:23
*** ppai has joined #openstack-swift12:26
*** LiuNanke has joined #openstack-swift12:28
*** LiuNanke has quit IRC12:29
*** jome has joined #openstack-swift12:32
*** silor has quit IRC12:34
*** silor has joined #openstack-swift12:37
*** proteusguy has quit IRC12:56
*** haomaiwa_ has joined #openstack-swift12:57
*** mahatic_ has joined #openstack-swift12:57
*** mahatic has quit IRC12:59
*** hseipp has quit IRC13:01
*** hseipp has joined #openstack-swift13:05
*** mahatic_ has quit IRC13:05
*** jmccarthy has quit IRC13:08
*** proteusguy has joined #openstack-swift13:09
acoleseranrom: hi, i have a question about your container sync patch 20580313:09
patchbotacoles: https://review.openstack.org/#/c/205803/ - Container-Sync to iterate only over synced containers13:09
*** hseipp has quit IRC13:12
*** tsg has joined #openstack-swift13:16
*** venkat_p has quit IRC13:16
*** lpabon has quit IRC13:17
*** jmccarthy has joined #openstack-swift13:25
*** mahatic_ has joined #openstack-swift13:26
*** mahatic_ has quit IRC13:37
*** thumpba has joined #openstack-swift13:39
*** tsg has quit IRC13:52
*** links has quit IRC13:53
*** sanchitmalhotra has quit IRC13:54
*** jmccarthy has quit IRC13:56
*** venkat_p has joined #openstack-swift13:57
*** venkat_p has quit IRC13:58
*** breitz has quit IRC14:05
*** breitz has joined #openstack-swift14:05
*** thumpba has quit IRC14:07
*** jerrygb has joined #openstack-swift14:10
*** haomaiwa_ has quit IRC14:13
*** petertr7_away is now known as petertr714:23
*** haomaiwang has joined #openstack-swift14:25
*** SkyRocknRoll has quit IRC14:30
*** ppai has quit IRC14:40
*** badari has joined #openstack-swift14:41
openstackgerritRico Lin proposed openstack/python-swiftclient: Remove py26 support  https://review.openstack.org/25243514:54
*** blmartin has joined #openstack-swift14:55
*** zaitcev has joined #openstack-swift14:55
*** ChanServ sets mode: +v zaitcev14:55
openstackgerritRico Lin proposed openstack/python-swiftclient: Remove py26 support  https://review.openstack.org/25243514:56
*** jmccarthy has joined #openstack-swift14:57
petertr7Hello! I was wondering if anyone was working on integrating the creation of static large objects into python-swiftclient. Was this something the community would be interested in?14:59
petertr7I've created a tool that split of a large file into segments, uploads it to swift and creates a manifest file. I was just wondering if potentially contributing this to python-swiftclient would be welcome. The one caveat is that I chose to create pseudo directories on swift to hold the segments. Not sure how you guys feel about that15:01
petertr7Of course I could always add additional flags to let the user decide where segments are stored. (like in a separate container if they wanted or a specific pseudo folder)15:01
*** jordanP has quit IRC15:03
openstackgerritRico Lin proposed openstack/python-swiftclient: Remove py26 support  https://review.openstack.org/25243515:03
*** dustins has joined #openstack-swift15:04
petertr7Hmm just realized perhaps people in the west coast are no awake yet.15:08
openstackgerritRico Lin proposed openstack/python-swiftclient: Remove py26 support  https://review.openstack.org/25243515:09
sjmc7petertr7, i’m not a swift dev but maybe take a look at http://docs.openstack.org/developer/swift/overview_large_objects.html ?15:12
petertr7Thanks sjmc715:12
petertr7This is what I'm referring to15:12
petertr7The tool I've been working on does the steps outlined in the above doc to upload a SLO15:13
sjmc7ok. the first section mentions support in the swift CLI15:13
petertr7Ohh, I didn't catch that15:13
petertr7Thanks, I never realized this15:14
sjmc7sure15:14
petertr7Sorry, I just realized, I did know this already15:15
petertr7The reason I created my own tool15:15
petertr7was because if the upload of a SLO failed half way through, you'd have to restart15:15
petertr7My tool does a quick check to pick up where the previous failed upload stopped15:15
*** haomaiwang has quit IRC15:15
*** raginbajin has quit IRC15:16
petertr7The swift cli I believe does checksum on the segments before attempting to upload it15:16
sjmc7ah, ok. that sounds like a good addition to have, but you’d be best off talking to a real swift dev :)15:16
*** delattec has joined #openstack-swift15:16
petertr7haha okay. but thanks for pointing it out15:16
sjmc7maybe submit a spec and ask for input?15:16
*** raginbajin has joined #openstack-swift15:17
petertr7I think I'll do that. But I also need to rethink - there is probably a reason why the swift cli does these checksums. Anyways I'll have to think about it more15:17
*** delatte has joined #openstack-swift15:19
*** delattec has quit IRC15:19
*** cdelatte has quit IRC15:19
*** jordanP has joined #openstack-swift15:21
*** thumpba has joined #openstack-swift15:25
*** siva_krishnan has joined #openstack-swift15:25
jrichlipetertr7: if you already have the code that does this, its probably best to simply upload the patch as opposed to creating a spec.  Specs in swift are typically for large new feature developments.15:26
*** petertr7 is now known as petertr7_away15:28
*** thumpba has quit IRC15:28
*** petertr7_away is now known as petertr715:29
*** weihan has quit IRC15:30
*** ChanServ has quit IRC15:34
*** raginbajin has quit IRC15:34
*** elligottmc has quit IRC15:34
*** janonymous has quit IRC15:34
*** takashi has quit IRC15:34
*** hogepodge has quit IRC15:34
*** trifon has quit IRC15:34
*** changbl has quit IRC15:34
*** diogogmt has quit IRC15:34
*** tamizh_geek has quit IRC15:34
*** sjmc7 has quit IRC15:34
*** patchbot has quit IRC15:34
*** stevemar has quit IRC15:34
*** sileht has quit IRC15:34
*** chlong has quit IRC15:34
*** amit213 has quit IRC15:34
*** notmyname has quit IRC15:34
*** kota_ has quit IRC15:34
*** okdas has quit IRC15:34
*** peluse has quit IRC15:34
*** jerrygb has quit IRC15:34
*** breitz has quit IRC15:34
*** aix has quit IRC15:34
*** david-lyle has quit IRC15:34
*** chmouel has quit IRC15:34
*** anteaya has quit IRC15:35
*** _fortis has quit IRC15:35
*** dosaboy has quit IRC15:35
*** amandap has quit IRC15:35
*** mattoliverau has quit IRC15:35
*** timur has quit IRC15:35
*** dfg has quit IRC15:35
*** charz has quit IRC15:35
*** hrou has quit IRC15:35
*** lifeless has quit IRC15:35
*** bsdkurt has quit IRC15:35
*** swifterdarrell has quit IRC15:35
*** ndk has quit IRC15:35
*** cschwede has quit IRC15:35
*** glange has quit IRC15:35
*** joearnold has quit IRC15:35
*** alpha_ori has quit IRC15:35
*** a1|away has quit IRC15:35
*** hugokuo has quit IRC15:35
*** portante has quit IRC15:35
*** dabukalam has quit IRC15:35
*** mandarine has quit IRC15:35
*** eikke has quit IRC15:35
*** torgomatic has quit IRC15:35
*** StevenK has quit IRC15:35
*** dmsimard has quit IRC15:35
*** clayg has quit IRC15:35
*** ir2ivps8_ has quit IRC15:35
*** mtreinish has quit IRC15:35
*** bapalm has quit IRC15:35
*** jistr has quit IRC15:35
*** pdardeau has quit IRC15:35
*** MooingLemur has quit IRC15:35
*** bwall has quit IRC15:35
*** zhiyan has quit IRC15:35
*** Guest49572 has quit IRC15:35
*** bkmz has quit IRC15:35
*** onovy has quit IRC15:35
*** ekarlso has quit IRC15:35
*** klrmn has quit IRC15:35
*** fbo_ has quit IRC15:35
*** noqa_v_g1ovnie has quit IRC15:35
*** cppforlife_ has quit IRC15:35
*** zul has quit IRC15:35
*** onder has quit IRC15:35
*** blair has quit IRC15:35
*** briancline has quit IRC15:35
*** acoles has quit IRC15:35
*** tdasilva has quit IRC15:35
*** nottrobin has quit IRC15:35
*** briancurtin has quit IRC15:35
*** jordanP has quit IRC15:35
*** delatte has quit IRC15:35
*** proteusguy has quit IRC15:35
*** asettle has quit IRC15:35
*** arnox has quit IRC15:35
*** gmmaha has quit IRC15:35
*** flaper87 has quit IRC15:35
*** saltsa has quit IRC15:35
*** petertr7 has quit IRC15:35
*** darrenc has quit IRC15:35
*** urth has quit IRC15:35
*** Lickitysplitted_ has quit IRC15:35
*** ejat has quit IRC15:35
*** McMurlock has quit IRC15:35
*** clyps___ has quit IRC15:35
*** j_king_ has quit IRC15:35
*** csmart has quit IRC15:35
*** remix_tj has quit IRC15:35
*** kragniz has quit IRC15:35
*** baffle has quit IRC15:35
*** cebruns has quit IRC15:35
*** balajir has quit IRC15:35
*** swat30 has quit IRC15:35
*** ahale_ has quit IRC15:35
*** jrichli has quit IRC15:35
*** sw3 has quit IRC15:35
*** HenryG has quit IRC15:35
*** zigo has quit IRC15:35
*** zacksh has quit IRC15:35
*** logan- has quit IRC15:35
*** AbyssOne has quit IRC15:35
*** chrisnelson has quit IRC15:35
*** mlanner has quit IRC15:35
*** wbhuber has quit IRC15:35
*** anderstj has quit IRC15:35
*** ctennis has quit IRC15:35
*** timburke has quit IRC15:35
*** EmilienM has quit IRC15:35
*** redbo_ has quit IRC15:35
*** treyd has quit IRC15:35
*** jlhinson has quit IRC15:35
*** wolsen has quit IRC15:35
*** acorwin_ has quit IRC15:35
*** Anticimex has quit IRC15:35
*** jeblair has quit IRC15:35
*** bobby2 has quit IRC15:35
*** hugespoon has quit IRC15:35
*** serverascode has quit IRC15:35
*** sudorandom has quit IRC15:35
*** philipw has quit IRC15:35
*** jroll has quit IRC15:35
*** jamielennox has quit IRC15:35
*** wasmum has quit IRC15:35
*** donagh has quit IRC15:35
*** early has quit IRC15:35
*** CrackerJackMack has quit IRC15:35
*** torgomatic has joined #openstack-swift15:37
*** StevenK has joined #openstack-swift15:37
*** dmsimard has joined #openstack-swift15:37
*** clayg has joined #openstack-swift15:37
*** ir2ivps8_ has joined #openstack-swift15:37
*** wolfe.freenode.net sets mode: +vv torgomatic clayg15:37
*** trifon has joined #openstack-swift15:39
*** changbl has joined #openstack-swift15:39
*** diogogmt has joined #openstack-swift15:39
*** sjmc7 has joined #openstack-swift15:39
*** tamizh_geek has joined #openstack-swift15:39
*** patchbot has joined #openstack-swift15:39
*** stevemar has joined #openstack-swift15:39
*** sileht has joined #openstack-swift15:39
*** chlong has joined #openstack-swift15:39
*** amit213 has joined #openstack-swift15:39
*** notmyname has joined #openstack-swift15:39
*** kota_ has joined #openstack-swift15:39
*** okdas has joined #openstack-swift15:39
*** peluse has joined #openstack-swift15:39
*** wolfe.freenode.net sets mode: +v notmyname15:39
*** jistr has joined #openstack-swift15:39
*** pdardeau has joined #openstack-swift15:39
*** MooingLemur has joined #openstack-swift15:39
*** bwall has joined #openstack-swift15:39
*** zhiyan has joined #openstack-swift15:39
*** Guest49572 has joined #openstack-swift15:39
*** bkmz has joined #openstack-swift15:39
*** onovy has joined #openstack-swift15:39
*** ekarlso has joined #openstack-swift15:39
*** klrmn has joined #openstack-swift15:39
*** fbo_ has joined #openstack-swift15:39
*** noqa_v_g1ovnie has joined #openstack-swift15:39
*** cppforlife_ has joined #openstack-swift15:39
*** zul has joined #openstack-swift15:39
*** onder has joined #openstack-swift15:39
*** blair has joined #openstack-swift15:39
*** raginbajin has joined #openstack-swift15:40
*** zigo has joined #openstack-swift15:40
*** zacksh has joined #openstack-swift15:40
*** logan- has joined #openstack-swift15:40
*** AbyssOne has joined #openstack-swift15:40
*** chrisnelson has joined #openstack-swift15:40
*** mlanner has joined #openstack-swift15:40
*** wbhuber has joined #openstack-swift15:40
*** anderstj has joined #openstack-swift15:40
*** ctennis has joined #openstack-swift15:40
*** timburke has joined #openstack-swift15:40
*** EmilienM has joined #openstack-swift15:40
*** redbo_ has joined #openstack-swift15:40
*** treyd has joined #openstack-swift15:40
*** jlhinson has joined #openstack-swift15:40
*** wolsen has joined #openstack-swift15:40
*** acorwin_ has joined #openstack-swift15:40
*** Anticimex has joined #openstack-swift15:40
*** jeblair has joined #openstack-swift15:40
*** bobby2 has joined #openstack-swift15:40
*** wolfe.freenode.net sets mode: +v timburke15:40
*** mtreinish has joined #openstack-swift15:42
*** bapalm has joined #openstack-swift15:42
*** elligottmc has joined #openstack-swift15:42
*** janonymous has joined #openstack-swift15:42
*** takashi has joined #openstack-swift15:42
*** briancline has joined #openstack-swift15:42
*** acoles has joined #openstack-swift15:42
*** tdasilva has joined #openstack-swift15:42
*** nottrobin has joined #openstack-swift15:42
*** briancurtin has joined #openstack-swift15:42
*** wolfe.freenode.net sets mode: +vv acoles tdasilva15:42
*** hogepodge has joined #openstack-swift15:42
*** hugespoon has joined #openstack-swift15:43
*** serverascode has joined #openstack-swift15:43
*** sudorandom has joined #openstack-swift15:43
*** philipw has joined #openstack-swift15:43
*** jroll has joined #openstack-swift15:43
*** jamielennox has joined #openstack-swift15:43
*** wasmum has joined #openstack-swift15:43
*** donagh has joined #openstack-swift15:43
*** early has joined #openstack-swift15:43
*** CrackerJackMack has joined #openstack-swift15:43
*** trifon has quit IRC15:49
*** aix has joined #openstack-swift15:51
*** _fortis has joined #openstack-swift15:52
*** wshao has joined #openstack-swift15:57
*** proteusguy has joined #openstack-swift15:59
*** chmouel has joined #openstack-swift15:59
*** jerrygb has joined #openstack-swift15:59
*** breitz has joined #openstack-swift15:59
*** david-lyle has joined #openstack-swift15:59
*** eikke has joined #openstack-swift15:59
*** anteaya has joined #openstack-swift15:59
*** dosaboy has joined #openstack-swift15:59
*** amandap has joined #openstack-swift15:59
*** mattoliverau has joined #openstack-swift15:59
*** timur has joined #openstack-swift15:59
*** dfg has joined #openstack-swift15:59
*** charz has joined #openstack-swift15:59
*** lifeless has joined #openstack-swift15:59
*** bsdkurt has joined #openstack-swift15:59
*** swifterdarrell has joined #openstack-swift15:59
*** ndk has joined #openstack-swift15:59
*** glange has joined #openstack-swift15:59
*** cschwede has joined #openstack-swift15:59
*** wolfe.freenode.net sets mode: +vvvv mattoliverau swifterdarrell glange cschwede15:59
*** joearnold has joined #openstack-swift15:59
*** alpha_ori has joined #openstack-swift15:59
*** a1|away has joined #openstack-swift15:59
*** hugokuo has joined #openstack-swift15:59
*** portante has joined #openstack-swift15:59
*** dabukalam has joined #openstack-swift15:59
*** mandarine has joined #openstack-swift15:59
*** rcernin has quit IRC15:59
*** zul has quit IRC16:00
*** dustins has quit IRC16:01
*** kragniz has joined #openstack-swift16:03
*** jmccarthy has quit IRC16:07
openstackgerritChristian Schwede proposed openstack/python-swiftclient: Fix debug and info option parsing  https://review.openstack.org/25232816:07
*** proteusguy has quit IRC16:09
*** chmouel has quit IRC16:09
*** jerrygb has quit IRC16:09
*** breitz has quit IRC16:09
*** david-lyle has quit IRC16:09
*** anteaya has quit IRC16:09
*** dosaboy has quit IRC16:09
*** amandap has quit IRC16:09
*** mattoliverau has quit IRC16:09
*** timur has quit IRC16:09
*** dfg has quit IRC16:09
*** charz has quit IRC16:09
*** lifeless has quit IRC16:09
*** bsdkurt has quit IRC16:09
*** swifterdarrell has quit IRC16:09
*** ndk has quit IRC16:09
*** cschwede has quit IRC16:09
*** glange has quit IRC16:09
*** joearnold has quit IRC16:09
*** a1|away has quit IRC16:09
*** alpha_ori has quit IRC16:09
*** hugokuo has quit IRC16:09
*** portante has quit IRC16:09
*** dabukalam has quit IRC16:09
*** mandarine has quit IRC16:09
*** eikke has quit IRC16:09
*** diogogmt has quit IRC16:17
*** wshao has quit IRC16:31
*** jome has quit IRC16:33
*** wshao has joined #openstack-swift16:34
*** HenryG has joined #openstack-swift16:34
*** sw3 has joined #openstack-swift16:34
*** jrichli has joined #openstack-swift16:34
*** ahale_ has joined #openstack-swift16:34
*** swat30 has joined #openstack-swift16:34
*** balajir has joined #openstack-swift16:34
*** cebruns has joined #openstack-swift16:34
*** baffle has joined #openstack-swift16:34
*** remix_tj has joined #openstack-swift16:34
*** csmart has joined #openstack-swift16:34
*** j_king_ has joined #openstack-swift16:34
*** clyps___ has joined #openstack-swift16:34
*** McMurlock has joined #openstack-swift16:34
*** ejat has joined #openstack-swift16:34
*** Lickitysplitted_ has joined #openstack-swift16:34
*** urth has joined #openstack-swift16:34
*** darrenc has joined #openstack-swift16:34
*** petertr7 has joined #openstack-swift16:34
*** saltsa has joined #openstack-swift16:34
*** flaper87 has joined #openstack-swift16:34
*** gmmaha has joined #openstack-swift16:34
*** arnox has joined #openstack-swift16:34
*** asettle has joined #openstack-swift16:34
*** delatte has joined #openstack-swift16:34
*** jordanP has joined #openstack-swift16:34
*** hrou has joined #openstack-swift16:34
*** tsg has joined #openstack-swift16:34
*** hseipp has joined #openstack-swift16:34
*** sgundur has joined #openstack-swift16:34
*** diazjf has joined #openstack-swift16:34
*** zul has joined #openstack-swift16:34
*** jmccarthy1 has joined #openstack-swift16:34
*** proteusguy has joined #openstack-swift16:34
*** chmouel has joined #openstack-swift16:34
*** jerrygb has joined #openstack-swift16:34
*** breitz has joined #openstack-swift16:34
*** david-lyle has joined #openstack-swift16:34
*** eikke has joined #openstack-swift16:34
*** anteaya has joined #openstack-swift16:34
*** dosaboy has joined #openstack-swift16:34
*** amandap has joined #openstack-swift16:34
*** mattoliverau has joined #openstack-swift16:34
*** timur has joined #openstack-swift16:34
*** dfg has joined #openstack-swift16:34
*** charz has joined #openstack-swift16:34
*** lifeless has joined #openstack-swift16:34
*** wolfe.freenode.net sets mode: +v mattoliverau16:34
*** bsdkurt has joined #openstack-swift16:34
*** swifterdarrell has joined #openstack-swift16:34
*** ndk has joined #openstack-swift16:34
*** glange has joined #openstack-swift16:34
*** cschwede has joined #openstack-swift16:34
*** joearnold has joined #openstack-swift16:34
*** alpha_ori has joined #openstack-swift16:34
*** a1|away has joined #openstack-swift16:34
*** hugokuo has joined #openstack-swift16:34
*** portante has joined #openstack-swift16:34
*** dabukalam has joined #openstack-swift16:34
*** mandarine has joined #openstack-swift16:34
*** wolfe.freenode.net sets mode: +vvv swifterdarrell glange cschwede16:34
*** jome has joined #openstack-swift16:35
notmynamegood morning16:36
*** klrmn1 has joined #openstack-swift16:44
*** jome has quit IRC16:44
*** zul has quit IRC16:45
*** jmccarthy1 has quit IRC16:45
*** proteusguy has quit IRC16:45
*** chmouel has quit IRC16:45
*** jerrygb has quit IRC16:45
*** breitz has quit IRC16:45
*** david-lyle has quit IRC16:45
*** anteaya has quit IRC16:45
*** dosaboy has quit IRC16:45
*** amandap has quit IRC16:45
*** mattoliverau has quit IRC16:45
*** timur has quit IRC16:45
*** dfg has quit IRC16:45
*** charz has quit IRC16:45
*** lifeless has quit IRC16:45
*** bsdkurt has quit IRC16:45
*** swifterdarrell has quit IRC16:45
*** ndk has quit IRC16:45
*** cschwede has quit IRC16:45
*** glange has quit IRC16:45
*** joearnold has quit IRC16:45
*** a1|away has quit IRC16:45
*** alpha_ori has quit IRC16:45
*** hugokuo has quit IRC16:45
*** portante has quit IRC16:45
*** dabukalam has quit IRC16:45
*** mandarine has quit IRC16:45
*** eikke has quit IRC16:45
*** diazjf has quit IRC16:45
*** sgundur has quit IRC16:45
*** hseipp has quit IRC16:45
*** tsg has quit IRC16:45
*** hrou has quit IRC16:45
*** jordanP has quit IRC16:45
*** delatte has quit IRC16:45
*** asettle has quit IRC16:45
*** arnox has quit IRC16:45
*** gmmaha has quit IRC16:45
*** flaper87 has quit IRC16:45
*** saltsa has quit IRC16:45
*** petertr7 has quit IRC16:45
*** darrenc has quit IRC16:45
*** urth has quit IRC16:45
*** Lickitysplitted_ has quit IRC16:45
*** ejat has quit IRC16:45
*** McMurlock has quit IRC16:45
*** clyps___ has quit IRC16:45
*** j_king_ has quit IRC16:45
*** csmart has quit IRC16:45
*** remix_tj has quit IRC16:45
*** baffle has quit IRC16:45
*** cebruns has quit IRC16:45
*** balajir has quit IRC16:45
*** swat30 has quit IRC16:45
*** ahale_ has quit IRC16:45
*** jrichli has quit IRC16:45
*** sw3 has quit IRC16:45
*** HenryG has quit IRC16:45
*** klrmn1 has quit IRC16:45
*** mandarine has joined #openstack-swift16:46
*** dabukalam has joined #openstack-swift16:46
*** portante has joined #openstack-swift16:46
*** hugokuo has joined #openstack-swift16:46
*** alpha_ori has joined #openstack-swift16:46
*** joearnold has joined #openstack-swift16:46
*** cschwede has joined #openstack-swift16:46
*** glange has joined #openstack-swift16:46
*** ndk has joined #openstack-swift16:46
*** swifterdarrell has joined #openstack-swift16:46
*** bsdkurt has joined #openstack-swift16:46
*** lifeless has joined #openstack-swift16:46
*** charz has joined #openstack-swift16:46
*** dfg has joined #openstack-swift16:46
*** timur has joined #openstack-swift16:46
*** mattoliverau has joined #openstack-swift16:46
*** wolfe.freenode.net sets mode: +vvvv cschwede glange swifterdarrell mattoliverau16:46
*** amandap has joined #openstack-swift16:46
*** dosaboy has joined #openstack-swift16:46
*** anteaya has joined #openstack-swift16:46
*** eikke has joined #openstack-swift16:46
*** david-lyle has joined #openstack-swift16:46
*** breitz has joined #openstack-swift16:46
*** jerrygb has joined #openstack-swift16:46
*** chmouel has joined #openstack-swift16:46
*** proteusguy has joined #openstack-swift16:46
*** jmccarthy1 has joined #openstack-swift16:46
*** zul has joined #openstack-swift16:46
*** diazjf has joined #openstack-swift16:46
*** sgundur has joined #openstack-swift16:46
*** hseipp has joined #openstack-swift16:46
*** tsg has joined #openstack-swift16:46
*** hrou has joined #openstack-swift16:46
*** jordanP has joined #openstack-swift16:46
*** delatte has joined #openstack-swift16:46
*** asettle has joined #openstack-swift16:46
*** arnox has joined #openstack-swift16:46
*** gmmaha has joined #openstack-swift16:46
*** flaper87 has joined #openstack-swift16:46
*** saltsa has joined #openstack-swift16:46
*** petertr7 has joined #openstack-swift16:46
*** darrenc has joined #openstack-swift16:46
*** urth has joined #openstack-swift16:46
*** Lickitysplitted_ has joined #openstack-swift16:46
*** ejat has joined #openstack-swift16:46
*** McMurlock has joined #openstack-swift16:46
*** clyps___ has joined #openstack-swift16:46
*** j_king_ has joined #openstack-swift16:46
*** csmart has joined #openstack-swift16:46
*** remix_tj has joined #openstack-swift16:46
*** baffle has joined #openstack-swift16:46
*** cebruns has joined #openstack-swift16:46
*** balajir has joined #openstack-swift16:46
*** swat30 has joined #openstack-swift16:46
*** ahale_ has joined #openstack-swift16:46
*** jrichli has joined #openstack-swift16:46
*** sw3 has joined #openstack-swift16:46
*** HenryG has joined #openstack-swift16:46
notmynameseems freenode is under some sort of ddos, thus the netsplits16:47
notmynamefor those who are here, patch 252308 drops py26 testing from swiftclient. at the last meeting we talked about how that was going to happen16:48
patchbotnotmyname: https://review.openstack.org/#/c/252308/ - Remove py26 support from swiftclient16:48
*** aix has quit IRC16:48
*** aix has joined #openstack-swift16:48
notmynameso I propose doing a python-swiftclient release (2.6.1 or 2.7.0) with what's currently HEAD of master16:49
openstackgerritAlistair Coles proposed openstack/python-swiftclient: Fix debug and info option parsing  https://review.openstack.org/25232816:51
jrichlinotmyname: hrou was talkin smack while you cores were not connected16:51
*** jordanP has quit IRC16:52
*** chmouel has quit IRC16:52
jrichliand its not in the online log either!16:52
hrouAnd apparently there was no history so I guess no one will ever know ;)16:52
jrichlithat is, if the 30 people connected at the time can keep the secret ;-)16:53
notmynameheh16:53
gmmahahehe16:53
jrichligmmaha was on16:54
hrouI trust that group of folks ;)  They'll stay quiet16:54
gmmahaheheh16:54
notmynamehrou: that's ok. while you weren't connected I laid down some new rules for contributing. if you don't follow them to the letter, your code won't be reviewed ;-)16:54
gmmahai was def on!!16:54
*** jordanP has joined #openstack-swift16:54
* gmmaha goes to save the hexchat log to file for the future16:54
jrichligmmaha lol16:55
jrichlinotmyname: hrou and I were connected most of the time (still not sure why that is).16:55
hrounotmyname, lol (noo I guess there goes the symlink work ;) jk16:56
notmynameprobably just depends on which freenode server you're actually connected to16:56
hrouyea I think that was the key16:56
jrichliright, makes sense16:56
notmynamewhew. full meeting agenda today16:57
*** jistr has quit IRC17:01
*** pgbridge has joined #openstack-swift17:02
*** ChanServ has joined #openstack-swift17:04
*** wolfe.freenode.net sets mode: +o ChanServ17:04
*** nadeem has joined #openstack-swift17:05
*** chmouel has joined #openstack-swift17:06
*** rcernin has joined #openstack-swift17:11
*** wshao has quit IRC17:11
*** gyee has joined #openstack-swift17:12
*** diazjf has quit IRC17:19
timburkepetertr7: we always appreciate better fault tolerance in swiftclient, so i'd be interested in what you had in mind. i also wonder if patch 226897 would address the issue for you17:24
patchbottimburke: https://review.openstack.org/#/c/226897/ - Make LengthWrappers resettable if their _readable ...17:24
timburke(oh, and notmyname: that^^ might be a good candidate to get in before dropping py26 support. and maybe patch 246040? though i have no idea what version of keystoneclient you would need to have py26 support; maybe it would be old enough that it wouldn't emit the warning)17:32
patchbottimburke: https://review.openstack.org/#/c/246040/ - Stop passing attr to keystoneclient when there's n...17:32
notmynametimburke: pretty much for anything that's not landed on master yet, we'll need to have explicit reviewer testing and affirmation that it works on py2617:33
notmynametechnically, that would be after patch 252308 lands, but i'm working under the assumption that it's a done deal17:33
patchbotnotmyname: https://review.openstack.org/#/c/252308/ - Remove py26 support from swiftclient17:33
notmynametimburke: so yeah, we could get a few more things in, but it will require more work to do so17:33
*** a1|away has joined #openstack-swift17:34
timburkethat's fair. just thought i'd point them out as potentially useful enough to warrant having them in that next release17:35
notmynameack :-)17:35
*** NM has joined #openstack-swift17:47
*** changbl has quit IRC17:54
*** arnox has quit IRC17:57
*** blmartin has quit IRC17:58
zaitcevI don't understand why encryption wants updateable metadata. Do we have an explanation written down anywhere?18:01
*** wuhg has quit IRC18:01
*** klrmn1 has joined #openstack-swift18:04
hrouzaitcev, what do you mean be updatable metadata ?  Are you referring to patch 245885 by any chance ?18:05
*** jmccarthy1 has quit IRC18:05
patchbothrou: https://review.openstack.org/#/c/245885/ - Append crypto-meta to user-meta18:05
*** jordanP has quit IRC18:06
*** changbl has joined #openstack-swift18:07
zaitcevIf I look at the spec (specs/in_progress/at_rest_encryption.rst), it lists a bunch of stuff that it chucks into sysmeta, where it's at risk of being lost if any middleware screws up the update. So 1) I see why we need something stored, 2) I suspect a reason why we want it persist, but we never claim the reason-outcome connection.18:15
zaitcevIt looks like it would be possible to live without Massmeta18:16
*** dustins has joined #openstack-swift18:19
hrouzaitcev, In terms what is being stored each piece of user meta has associated crypto meta, namely an IV, this is what needs to be persisted.  In terms of the problem with sysmeta, (and the patch above actually does touch on this for fast-post) - the semantics of POST don't allow sysmeta to be updated, and we'd need to set that for the new headers (err new is a bad choice of words, for the headers period  given the granularity of user meta).18:24
*** sjmc7 has left #openstack-swift18:26
*** jome has joined #openstack-swift18:28
*** hseipp has quit IRC18:35
*** jome has quit IRC18:39
*** jome has joined #openstack-swift18:42
claygheyoh!18:46
*** jome has quit IRC18:54
*** sgundur has quit IRC18:57
claygtorgomatic: nice work on the slo segment fix - thanks everyone who reviewe!18:57
clayganyone have a handle for "Lorcan Browne" ???18:58
clayghe said he was running ring tests on patch 241571 but gave no indication of how they're going :'(18:58
patchbotclayg: https://review.openstack.org/#/c/241571/ - Put part-replicas where they go18:58
acolesclayg: don't think he hangs out here, he's in my team though18:58
acolesclayg: and they're going ok i believe18:59
*** siva_krishnan has quit IRC18:59
*** siva_krishnan has joined #openstack-swift18:59
*** sgundur has joined #openstack-swift19:00
claygacoles: phew19:00
claygacoles: is ec multi frag gee-double-oh-dee-to-oh-gee-oh?19:00
*** sgundur has quit IRC19:02
*** klrmn1 has quit IRC19:03
acolesclayg: if you mean would i like you to review patch 231121 again, then abso-el-triple-o-tee-lee19:04
patchbotacoles: https://review.openstack.org/#/c/231121/ - Make ECDiskFile report all fragments found on disk19:04
claygacoles: oh - and pick a follow up - there's three patches that depend on that one - i'm not going to get to all of them today19:04
acolesclayg: sure, i am just rebasing them actually.19:05
claygi'm leaning towards patch 232684, but patch 181407 also seems pretty smart19:05
patchbotclayg: https://review.openstack.org/#/c/232684/ - Don't ssync data when only a durable is missing19:05
patchbotclayg: https://review.openstack.org/#/c/181407/ - EC: Avoid conflicts when ssync'ing fragments19:05
*** klrmn1 has joined #openstack-swift19:07
acolesclayg: 232684 then 181407 is probably a good order to choose19:08
* clayg got something right for a change!19:08
claygacoles: is really is annoying that you can't use bisect on a reverse sorted list with a custom key :'(19:09
openstackgerritAlistair Coles proposed openstack/swift: Enable object server to return non-durable data  https://review.openstack.org/21527619:09
claygacoles: OMG I love _split_gt[e]_timestamp!!!19:09
acolesclayg: i have a philosophy that is a review comment has gone past 3 or 4 lines then the code probably needs changing19:10
claygit's like a 10 second grok - gah - so freaking good19:10
acolesclayg: so i am looking at Lorcan's spreadsheet, which needs some cleansing before we can share. there's a couple of case where he sees balance 999.99 with the new builder19:12
claygacoles: I think your philosophy services you well - I also have this nose for kwargs to methods that made it do two drastically different things - most of the time IME it means it'd look better as two methods19:12
claygacoles: that might happen when there's only a single device in a server/zone for sure19:13
acolesclayg specifically when setting weight to zero on a bunch of devices19:13
claygoic19:13
clayglike not all parts get shed - that's not great - unless min_part_hours is just keeping them from moving?  which would be fine/expected19:13
acolesi'm pretty sure he used the trick to force min_part_hours to zero (there's some kind of flag for that i think?) cos he's run a gazillion scenarios in last day or so19:16
claygacoles: awesome!  ok so maybe there's an issue - that's great!  I wonder how we can tease out the minimum viable test case..19:17
claygtorgomatic: in your work with fuzz or the ring analyzier did you go over anything with removing devices and checking balance?19:18
acolesclayg: so he is stepping weight down to zero before then removing the devices, and when they are removed balance is 0, its just weight zero where he sees 99919:20
*** aix has quit IRC19:20
claygacoles: sure does *sound* like min_part_hours?19:20
acolesi can ask him to repeat to double check19:20
claygacoles: I think if weight is 0 even one part on the devices is going to say the balance is *terrible* - and if min_part_hours is locking it up there's not much to be done19:21
acolesmakes sense19:21
claygOTOH it's quite possible those last few parts locked on the zero wieght device are just in the "swapped part" scenario - and the new gathers aren't picking them up because they think they won't have anywhere else to go19:21
acolesok so will ask him to check min_part_hours for those. and if it repeats than we could look at sharing the ring data.19:22
claygbut the removed device case is different - removed devices just have their parts gobbled up no matter what - we could do something similar for zero weight devices - leaving the only condition that would leave a part on a dev with zero weight as min_part_hours19:22
claygit depends why you're reducing the weight to zero operationally I guess - maybe - it's probably better to reassign them - I think we could write a test for this19:23
claygs/anywhere else/anywhere better/g19:23
claygbut maybe any other device is better than a zero weight device19:24
claygacoles: if these rings are artificial the easiest thing might be to share one with me via google drive (or a tempurl)19:24
openstackgerritAlistair Coles proposed openstack/swift: Don't ssync data when only a durable is missing  https://review.openstack.org/23268419:25
acolesclayg: ^^ rebased19:25
claygif it's the swapped-part-problem (there's a couple of tests in the change that talk about it) I won't be able to say for sure unless I can see the replica2part2dev19:25
claygacoles: and tell this cat to get him some IRC!?19:26
openstackgerritChristian Schwede proposed openstack/python-swiftclient: Fix debug and info option parsing  https://review.openstack.org/25232819:26
acolesclayg: i'll speak with lorcan tomorrow. in some case the rings are from production.19:26
claygacoles: good for him!19:26
acolesclayg: gotta go get son from geetaar lesson, back for meeting time later19:27
claygacoles: k, awesome19:27
*** acoles is now known as acoles_19:27
*** briancurtin has quit IRC19:33
*** serverascode has quit IRC19:33
*** nottrobin has quit IRC19:33
*** zhiyan has quit IRC19:33
jrichlizaitcev: you bring up a good point, though.  the spec is a bit out of date.  Crypto is no longer adding 'massmeta'.  It is making use of sysmeta, though, to store data required for the decryption.19:34
claygyay sysmeta!19:37
claygRIP massmeta19:38
zaitcevThat means... Alasdair is going to make it so sysmeta persists across POST somehow? Right?19:38
zaitcevOr we're just asking all middleware to be careful about not deleting encryption sysmeta?19:39
claygzaitcev: it sorta already does - it's stored with the .datafile and bleeds through .meta files - on POST-AS-COPY it'll slurp up any sysmeta it finds and keep it with the new datafile19:39
zaitcevclayg: I figured that out, but I wanted to review fast-post19:39
claygzaitcev: middleware should namespace it's sysmeta, there's no protection against middleware an operator puts in the pipeline being incompatible (stoping on each others keys)19:40
claygthe *encryption* middleware has to be internally consistent about storing encyrption data that goes wiht the data and metadata in deifferent keys19:40
claygIIRC19:40
claygI'm should just let jrichli explain it ;)19:40
jrichliclayg: you are doing fine :-) zaitcev: there was one small change made to sysmeta behavior.  one sec19:44
jrichlizaitcev: in proxy/controllers/obj, there is a place that says "# post-as-copy: ignore new sysmeta, copy existing sysmeta"19:46
notmynameI've got another meeting about to start that will last right up to the swift meeting. might be one or two minutes late starting19:46
jrichliit used to do "remove_items(sink_req.headers, condition)", crypto branch has removed that action19:46
jrichlinotmyname: sounds like a fun afternoon :-)19:47
jrichlioh, of course, the meeting is at 3 now.  i was thinking 419:48
zaitcevjrichli: thanks19:48
jrichlizaitcev: np19:48
zaitcevSince we're at it, does anyone actually use the encryption already?19:48
openstackgerritTim Burke proposed openstack/python-swiftclient: Stop passing attr to keystoneclient when there's no filter_value  https://review.openstack.org/24604019:49
zaitcevNot afraid of setting the middleware into pipelines and then one day upgrade the cluster to Mitaka and not being able to decrypt anything?19:49
jrichlizaitcev: no, nobody's using it since not all the functests pass yet (big things like COPY wouldnt work).19:50
jrichlizaitcev: as far as future-proofing, we have had talks about what would happen if crypto is taken in and out of the pipeline.  We are trying to make the best decisions for these types of scenarios19:51
jrichlicluster upgrades would only be an issue if we make a change at some point that is not backwards incompatible.19:52
zaitcevjrichli: obviously taking it out is going to ruin GETs (although perhaps not DELETEs). I'm more concerned about better sysmeta being added/removed and such.19:52
jrichlizaitcev: ah, so the effects of changing the sysmeta behavior?  I would think acoles is the best person to speak with about those concerns19:54
zaitcevokay19:55
zaitcevI'll just go along with fast-post as is, then. If you need changes in the future, we'll take them in when crypto branch gets in, I suppose.19:55
jrichlizaitcev: there are issues with crypto + fast-post right now that pchng is working on.  So, again, the spec is out of date, and does not capture some of the latest details of how all the crypto-meta will actually work.19:57
jrichlizaitcev: some details are here https://trello.com/c/018BLfj1/58-fix-errors-when-using-fast-post19:58
*** cppforlife_ has quit IRC19:59
*** diazjf has joined #openstack-swift20:01
hroujrichli, for fast post the patch 245885 - some interesting discussions there.20:06
patchbothrou: https://review.openstack.org/#/c/245885/ - Append crypto-meta to user-meta20:06
jrichlihrou: thanks. I had seen all the comments there except for your recent ones.  The first couple comments there are what sparked the updates to the trello card.20:08
jrichlihrou: i will read your comments now20:08
*** sgundur has joined #openstack-swift20:09
hroujrichli, and don't read too much to them, just some random thoughts :)20:09
*** nottrobin has joined #openstack-swift20:11
*** silor has quit IRC20:11
*** blmartin has joined #openstack-swift20:15
jrichlihrou: for the case you describe here "you can POST an object that is plain txt after enabling crypto and we'll want to be just encrypting that header. ", the body crypto-meta being absent should cause override flag to be set, so no encrypting of hdrs.20:16
*** serverascode has joined #openstack-swift20:17
jrichlihrou: that is something i mentioned in the trello card - that it is sort of a requirement that we have unintentionally put onto a keymaster, i think20:17
hroujrichli, oh so in that scenario a plain txt object, that you POST (with fast-post or not) will never be encrypted ?20:18
jrichlihrou: IIRC.  I need to make sure we have a test that asserts this behavior.20:18
*** zhiyan has joined #openstack-swift20:19
hroujrichli, interesting as I think that was acoles_ concern " ... But we cannot assume that e.g. where there are existing objects that pre-date crypto being enabled, or when crypto has been selectively applied. ..."20:19
jrichlihrou: I believe acoles concerns for this are all involving a scenario where crypto middleware is taken in and out of the pipeline20:20
*** tsg has quit IRC20:22
jrichlihrou: so, the problem is more about if you have an existing encrypted thing, then POST when crypto is not present, then put crypto back in and GET.20:22
*** tsg has joined #openstack-swift20:22
*** klrmn1 has quit IRC20:22
jrichlihrou: but maybe i need to read all this again later tonight and think deeply :-)20:23
hroujrichli, its funny I was about to type the very same thing as I didn't think that was the original concern, but mind you I see why that's an issue too : )20:23
jrichlihrou: I think that its never a problem if an existing object body is unencrypted, because we have always planned for that20:24
hroujrichli, less fast post ?20:24
jrichlihrou: I guess what I mean is that by design, if its an unencrypted body, there is no way anything for that object will ever be encrypted20:25
jrichliwith any feature20:25
jrichlionly because the keymaster looks to the body crypto-meta to make the decision20:25
jrichliso ... again, it seems like a requirement of the keymaster to check that20:26
*** cppforlife_ has joined #openstack-swift20:26
*** briancurtin has joined #openstack-swift20:27
eranromacoles: ok, gotcha. I got convinced that the update can take place from PUT/POST - just call the update after doing broker.update_metadata20:30
eranromacoles: please ignore my last comment over the patch, I will post a new one with your suggestion soon.20:31
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873620:32
claygeranrom: don't forget to take the extra replicate step out of the probe test for container sync?20:32
claygor at least - it *should* work without it ;)20:32
eranromclayg: Yeah I was just thinking about it.20:35
jrichlifor those following the crypto talk, hrou pointed out that the post as copy is a GET + PUT.  So, we have both remembered the reason crypto will still leave an unencrypted thing alone is that keymaster needs to look for PUT to be from a copy.20:37
*** ho has joined #openstack-swift20:55
* notmyname is hoping this meeting finishes in the next 3 minutes20:57
claygnotmyname: drop the mic20:57
kota_mornin20:58
openstackgerritPeter Chng proposed openstack/swift: Add round-trip encrypter/decrypter unit tests  https://review.openstack.org/25160620:58
mattoliveraumorning20:58
notmynameanalyst briefing about swift and swiftstack20:58
notmynameclayg: "swift is awesome. /me drops mic"20:58
jrichlithe meeting before us is still going on anyway20:59
*** acoles_ is now known as acoles20:59
jrichlithey ended now21:00
notmynameright on time, my other meeting ended too21:00
notmynamelet me move back to my desk and we'll do our swift meeting21:00
*** ober37 has joined #openstack-swift21:01
*** ober37 has left #openstack-swift21:01
*** ober37 has joined #openstack-swift21:01
*** cutforth has joined #openstack-swift21:02
notmynameall right21:02
notmynamemeeting time in #openstack-meeting21:02
openstackgerritPeter Chng proposed openstack/swift: Add round-trip encrypter/decrypter unit tests  https://review.openstack.org/25160621:04
acoleseranrom: ack, sounds good, look forward to next patchset21:05
acoleszaitcev: you are asking great questions! fully updateable sysmeta on object POSTs is tricky. they have something on feature/crypto that meets their needs, but IIRC I was nervous about it.21:08
*** klrmn1 has joined #openstack-swift21:12
*** diazjf1 has joined #openstack-swift21:15
zaitcevacoles: I was just wondering... Once we decree that parts of metadata (user or system) persist, then previously unnecessary explicit delete becomes necessary. The usual approach is to overload with e.g. empty key: post with A= causes A to be deleted. Then a compatibility issue possibly arises. I'm probably naive considering this21:17
*** diazjf has quit IRC21:17
zaitcevacoles: we may say that IV and cipher are never to be deleted anyway, so the fact that persistent meta cannot be deleted (only updated) is of no concern21:17
acoleszaitcev: problem with posting A= to delete A=old to an object is that we can't go back and delete from the object file, so we either remember A= for ever, or at some point A=old comes back to haunt21:22
acoleswhich is ok if a piece of middleware can be reliedon to always set some value for A, even if it is empth (deleted)21:24
acolesempty*21:25
*** diazjf1 has quit IRC21:31
notmynametsg: #openstack-meeting for pyeclib update21:31
acoleszaitcev: btw thanks for looking at the fast post stuff, i hope to update the patch soon, when i get time21:31
notmynametsg: thanks21:32
claygacoles: how quick do you gotta run off after the meeting?  I'm looking at _verify_on_disk_files using the '.ts' keys and am totally confused?21:37
claygacoles: this doesn't bode well => https://gist.github.com/clayg/82411b8076303a2cd1a021:38
acolesclayg: i can hang around for a bit21:39
acolesi hate those verify methods21:40
claygacoles: heheheh21:40
claygacoles: i just wanted to confirm that a) .ts .meta .data files in the results dict are gone (diff sorta verifies that) and b) see if you have any pointers to tests that *should* be raising asserts for invalid files on disk that apparenlty aren't hitting the code right21:41
*** takashi_k has joined #openstack-swift21:42
acolesclayg: oh thats bad :/ just shows why its such a bad idea to pass untyped dict things around in a language rather than use proper objects with like real interface :021:44
* acoles goes off to rant21:44
claygacoles: tdd says even with strong types your code is broken unless you test it21:45
claygacoles: but... i feel ya21:45
claygmakes refactors harder :\  - but bags of dicts are just so damn handy!21:46
claygacoles: why didn't you make the file_info results a class if you were so sure strong types are better!  :P21:46
claygacoles: py3 has some loosey goosey type support you might be into21:46
acolesclayg: you know i NEARLY did !!21:48
acolesclayg: but yeah, there's a hole in tests for sure21:48
torgomaticI love strongly typed languages with interfaces and contracts and compiler-enforced correctness that *still* let every reference be null and blow up your code at runtime21:48
claygacoles: well if you have an idea where to look i'm all ears - otherwise I'll keep digging and leave you a crumb with my comments21:49
clayggo go code review!21:49
claygtorgomatic: I think you said that because you *don't* mean it?21:49
claygtorgomatic: sometimes in this line of work we forget what love is21:50
acolesclayg: i'll look for the test that should cover it after meeting21:50
claygacoles: *hugs*21:52
acolesclayg: i said i'd look, i didn't say i'd find ;)21:53
clayglol21:54
hoacoles: thanks for reviewing of patch 202411. i have a question about your expectation value for the test. now it takes 70s in your env.22:00
patchbotho: https://review.openstack.org/#/c/202411/ - Add functional test for access control (RBAC) with...22:00
notmynamenext meeting time. be back in 30-4522:01
acolesho: hi!22:02
*** NM has quit IRC22:02
*** ober37 has quit IRC22:03
timburkespeaking of swiftclient (from the meeting), i'm interested in people's opinions on patch 18495722:03
patchbottimburke: https://review.openstack.org/#/c/184957/ - Add --decode-content option22:03
timburkebit of background: it used to be that if you uploaded for example a .tar.gz file and included a Content-Encoding: gzip header, swiftclient would (1) decode the content, leaving you with just a tar file, and (2) claim that the download failed, because the MD5 (of the *decoded* content) no longer matched the ETag (from the *encoded* content)22:04
acolesclayg: so re. (a) yes .ts .meta and .data keys should be gone from results dict22:04
timburkethis was fixed in patch 184659, but it occurred to me that some users may have been expecting the decode-it-for-me behavior and just ignored the error (since the file would still get written). so i wrote 184957 to add an option to restore the previous behavior sans error22:04
patchbottimburke: https://review.openstack.org/#/c/184659/ - Stop decoding object content (MERGED)22:04
timburkeso i have two questions for people: first, does this sound like something you would actually want and use? or do you feel more like "*shrug* seems cool; couldn't hurt"?22:04
timburkesecond, what are your opinions on swiftclient's role here? should it act primarily as a transport layer and just move bytes between server and client, or should it try to interpret those bytes? if the latter, under what circumstances?22:04
hoacoles: hi, i would like to know targeted time. 35s?22:05
*** petertr7 is now known as petertr7_away22:06
*** NM has joined #openstack-swift22:06
pdardeaupeluse: thx for reviewing 23953422:07
pelusepdardeau: welcome!22:08
acolesho: i don't have a target time in mind. i think we just have to try our best to make the tests as time-efficient as possible. for some of the scenarios i thought it would be ok to re-use the same resource and that did seem to save significant time.22:08
jrichliacoles zaitcev: I know acoles is gonna turn into a pumpkin soon, so I dont want to stir too much discussion.  but I was wondering if we could designate a sub-set of sysmeta ... like designate "persisted" sysmeta22:10
hoacoles: i see. i prefer to not having dependency b/w cases so my idea is using random.samples() to pick up small num of cases based on normal distribution matrix.22:11
hoacoles: what do you think?22:11
zaitcevjrichli: we could, the problem is that I can't imagine what the long-term consequences are going to be.22:12
jrichlizaitcev: the list would be carefully controlled.  only things that must persist like crypto-meta necessary for decrypting22:12
pdardeautimburke: re swiftclient's role, i think by default it should just be transport only. only interpret (do value-add processing) if requested (by whichever mechanism)22:13
acolesho: i see two problems with randomizing test scenarios - 1) can you be confident that a patch should merge if not all scenarios were tested? and 2) its hard to reproduce a test failure if the next run chooses different scenarios.22:13
*** dustins has quit IRC22:15
pdardeautimburke: i wouldn't expect any value-add processing just by setting Content-Encoding header22:15
*** takashi_k has quit IRC22:16
*** jome has joined #openstack-swift22:16
hoacoles: i see. if i have pooling resources it lose a chance to execute the test parallel with other way (testr)22:20
timburkepdardeau: that was basically where i arrived while fixing the original bug; we *definitely* shouldn't decode by default. but should the client even have options to do decoding? or should we intentionally limit our scope, and leave such interpretation to the user/application?22:20
pdardeautimburke: if i'm writing a swift application, i'd expect to handle it myself. not to say it shouldn't be there, but i think it opens pandora's box22:22
acolesjrichli: zaitcev its ok to "persist" object sysmeta is it is set in every POST. The problem comes when you try to keep an old sysmeta item and 'merge' it with a new set of metadata from a new POST. you risk violating the principle that every piece of state MUST be maintained with the timestamp at which it was created. account/container servers do just that. It is possible that object servers could but its imho a lot of wo22:22
acolesrk. See https://review.openstack.org/#/c/109314/1/specs/swift/updateable-obj-sysmeta.rst22:22
hrouacoles, and that's only problematic with fast post (with none fast post we're just getting a new set of everything) ?22:24
acolesho: I don't think the func tests suite can run in parallel without a lot of changes since the tests shares account and rely on setting up state in those accounts (maybe even shared containers).22:24
acoleshrou: correct22:24
zaitcevacoles: That's what I don't get about what you said earlier, about A=. If we do _not_ have updateable metadata, then every metadata has its own timestamp (except Content-Type), and therefore lasting A= is unnecessary. May simply remove it. But you said "A=old comes back to haunt". How?22:25
zaitcevacoles: If we do, then perfect, the stuck A= means "deleted" in both API and implementation.22:25
timburkepdardeau: that's my fear. i wrote 184957 because i saw that 184659 potentially broke existing behavior for people, but i'm happy to abandon it if we don't want to set a precedent of pushing application logic into the client22:26
acoleszaitcev: if I have A=old in to.data file and A=new in t1.meta, GET will show me A=new. If I now replace t1.meta with t2.meta that has no A, GET will show me A=old :( Unless I always copy the complete set of metadata from to.data to t1.meta, then from t1.meta to t2.meta and never read metadata from t0.data. But to do that I would need to tag every piece of metadata with its timestamp when copying into tx.meta. Which is do-22:29
acolesable.22:29
acolesBUT... t2.meta on server x could have different content from t2.meta on server y if there have been failed requests to one or other server.22:29
acolesso they are inconsistent, but their names are the same. So that spec linked above tried to deal with that.22:30
zaitcev*scribble scribble*22:30
acoleszaitcev: now, if middleware *always* tells object server the value of A in every POST then object server never needs to copy it from an old .meta to a new .meta and that works just fine.22:31
acoles...until one day the middleware goes missing and a post creeps in with no A...how does the object server know NOT to use the old A value in the .data file?22:32
acoles...if as jrichli suggests we declare a sysmeta namespace that will only be returned in GETs if it is in the latest .meta file then that would work...which is pretty much what the massmeta proposal was !22:33
*** cutforth has quit IRC22:33
hoacoles: in the test cases, it is necessary to configure acl on containers for each test. if i share the container, it's neccesary to remove acl and put acl ... the framework became complex and is it really fast than creating new container with acl???22:35
jrichliacoles: I do realize this suggestion is like massmeta, but I thought maybe it was more palatable to clayg since it wasn't a new namespace?22:35
jrichlior, i mean ... designate a subset somehow - i a way that clayg is more comfortable wiht22:35
*** ianbrown has joined #openstack-swift22:36
hrouacoles, jrichli - so instead of a new namespace, there's just a list of meta prefixed (e.g. crypto) you treat differently ?  In the sense of their lack of presence in the newest meta implies they're not set?22:36
acolesclayg: ok, back to testing the verify methods - on reflection, I don't there are tests that cause the _verify_ondisk_files asserts to blow up, because I *think* they only blow up if there is a bug in the code, where bug could be an ondisk file scenario that we haven't thought of and therefore haven't dealt with in get_ondisk_files.22:36
hrouThat was more a jrichli question.22:36
acolesclayg: make sense? I never liked it but I guess it does mean we might get a log if there is a scenario we haven't coded for.22:36
jrichlihrou: something like that.  i haven't thought it out fully yet22:37
acolesclayg: of course, the current patchset is bad. but i'm not sure how to test other than to call the method directly and pass in a dict with both ts_file and data_file set etc22:38
acolesactually thats quite reasonable ^^22:38
jrichlime thinks clayg went to the meeting with notmyname22:39
acolesjrichli: well you better hope clayg isn't following along or he'll spot the flanking manouver :-)22:39
hoacoles: thank for this conversation. I will think it more. thanks!22:39
hrouacoles, lol (its renamed thats all that matters ;)22:40
acolesho: i agree that with ACLs the resource needs to be setup22:40
* jrichli looks up flanking maneuver22:40
jrichliah, nice22:40
notmynameback from meetings22:41
acolesho: no problem, have a good day!22:41
acolesjrichli: maybe its out-flanking, idk22:42
jrichlinotmyname: you missed out on acoles having 3 discussions or more at once22:42
acolesjrichli: and thats just in irc!22:42
jrichliwow!22:42
jrichliyou should feel very needed :-)22:43
acolesjrichli: actually there is nothing else happening22:44
jrichliacoles: lol :-)22:44
acolesclayg: ok i was going to push rebased patch 181407 but there's no point since 231121 needs fixing22:47
patchbotacoles: https://review.openstack.org/#/c/181407/ - EC: Avoid conflicts when ssync'ing fragments22:47
*** eranrom has quit IRC22:47
acolesclayg: lemme know what you think about adding a test to directly call the _verify_ondisk_files method with crazy dicts.22:49
*** lyrrad has joined #openstack-swift22:51
*** ianbrown has quit IRC22:54
*** sgundur has quit IRC23:00
claygacoles: not pushing the rebase yet is ok by me23:02
claygacoles: I have some comments on patch 231121 too - it's getting late for you yeah?23:02
patchbotclayg: https://review.openstack.org/#/c/231121/ - Make ECDiskFile report all fragments found on disk23:02
claygacoles: if you think calling _verify directly is the best test maybe that's ok - but I was thinking it be better to have a [(<<list_of_files_on_disk>, <{expected_attributes}|Exception>), ...] test23:04
hotdasilva: around?23:07
acolesclayg: understand, but there *ought* to be no list of files on disk that could generate the assertion error23:10
* acoles now waits to be proven badly wrong23:11
*** km has joined #openstack-swift23:11
*** km is now known as Guest3691423:12
*** dmorita has joined #openstack-swift23:16
*** acoles has quit IRC23:20
*** dmorita has quit IRC23:22
*** dmorita has joined #openstack-swift23:23
*** acoles_ has joined #openstack-swift23:27
*** ChanServ sets mode: +v acoles_23:27
*** acoles_ is now known as acoles23:28
*** kei_yama has joined #openstack-swift23:33
*** siva_krishnan has quit IRC23:42
asettlenotmyname just got your email! Thank you :) i'll review today :)23:52
notmynamethanks23:53
asettleNo, thank you :)23:53
*** cryptsting has joined #openstack-swift23:56
*** nadeem has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!