| *** saschpe has quit IRC | 00:00 | |
| *** Trozz has joined #openstack-swift | 00:03 | |
| *** saschpe has joined #openstack-swift | 00:07 | |
| openstackgerrit | A change was merged to openstack/swift: Make initialization test failure more explicit https://review.openstack.org/82688 | 00:13 |
|---|---|---|
| *** krtaylor has joined #openstack-swift | 00:20 | |
| *** d89 has joined #openstack-swift | 00:26 | |
| *** matsuhashi has joined #openstack-swift | 00:27 | |
| *** h6w has left #openstack-swift | 00:28 | |
| *** piousbox has quit IRC | 00:32 | |
| *** h6w has joined #openstack-swift | 00:35 | |
| openstackgerrit | John Dickinson proposed a change to openstack/swift: Bump up sleep when expecting a timeout https://review.openstack.org/82664 | 00:35 |
| * h6w thinks that that patch could be applied to life in general. ^^ :-p | 00:36 | |
| h6w | Aside from specifying affinity, is there any difference between adding a proxy server locally and adding a proxy server globally? I'm specifically concerned about traffic between the geographically-disparate proxies. | 00:39 |
| *** mlipchuk has quit IRC | 00:39 | |
| *** zaitcev has quit IRC | 00:45 | |
| *** shri has quit IRC | 00:49 | |
| *** byeager has quit IRC | 00:52 | |
| h6w | Hmmm. Maybe I've done this a bit wrong. Can a storage node bind to more than one IP? | 00:52 |
| *** byeager has joined #openstack-swift | 00:52 | |
| h6w | I have local and VPN-based networks with the local proxy only being on the local network in each case. But this means that each storage node needs a local IP and VPN IP. | 00:54 |
| *** occupant has joined #openstack-swift | 00:56 | |
| *** byeager has quit IRC | 00:57 | |
| h6w | oic, maybe. The "ip" should be the local IP and the "replication_ip" should be the VPN IP? | 00:57 |
| *** godb has joined #openstack-swift | 01:04 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Add object-reconciler daemon https://review.openstack.org/80730 | 01:04 |
| godb | hi, how can i use "rexpirer.py" script? it is not operation | 01:04 |
| *** changbl has joined #openstack-swift | 01:12 | |
| *** matsuhashi has quit IRC | 01:12 | |
| godb | i want delete dummy .ts file | 01:13 |
| godb | how can i delete that? | 01:14 |
| *** matsuhashi has joined #openstack-swift | 01:14 | |
| clayg | is this related to bug # https://bugs.launchpad.net/swift/+bug/1257330 | 01:15 |
| clayg | godb: ^ | 01:15 |
| godb | clayg : hi, thx your answer. | 01:19 |
| godb | clayg : i did check it. and see https://gist.github.com/gholt/7777637 (external script) | 01:19 |
| godb | but i don't know how can i running this scirpts. | 01:20 |
| godb | when i running this scripts. but, there is no response | 01:21 |
| godb | my swift version 1.13 previous version 1.10 (last week upgrade) | 01:23 |
| *** nosnos has joined #openstack-swift | 01:27 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: additional functional tests for account acls https://review.openstack.org/80073 | 01:30 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Clean up Account ACL sysmeta in AccountController https://review.openstack.org/80140 | 01:30 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Harden v2 common.middleware.acl interface https://review.openstack.org/80188 | 01:30 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: additional functional tests for account acls https://review.openstack.org/80073 | 01:31 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Clean up Account ACL sysmeta in AccountController https://review.openstack.org/80140 | 01:31 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Harden v2 common.middleware.acl interface https://review.openstack.org/80188 | 01:31 |
| clayg | sorry, forgot to amend the commit msg | 01:31 |
| clayg | the script eats logs via stdin | 01:32 |
| clayg | you need to find the logs from your expirer while the expirer was effected and feed it to them via stdin | 01:32 |
| *** krtaylor has quit IRC | 01:34 | |
| clayg | godb: ^ | 01:34 |
| *** csd has quit IRC | 01:42 | |
| godb | clayg : unfortunately, i was deleted last log file. | 01:45 |
| godb | the day before yesterday. | 01:45 |
| clayg | yeah that sucks, we really need a patch for the auditor zbf that looks for the x-delete-at metadata and just purges data files outside of a window | 01:46 |
| clayg | it's sorta trixy though :\ | 01:46 |
| godb | then.. if i try to delete tombstone file then swift all clear? | 01:47 |
| clayg | you could probably suss out the object list from the deleted rows in the .expiring_objects container dbs - or just walk the objects dir "by hand" and pull out expired diskfiles | 01:47 |
| *** wer_mocha is now known as wer | 01:47 | |
| clayg | godb: well the bug was that the tombstone never got created, the .data files were left around on disk expired but the rows in the .expiring_object's containers were removed so they weren't retried | 01:48 |
| godb | .expiring_object is exist container db?? | 01:49 |
| clayg | .expiring_objects is a system account, it'll have some containers under it, the rows in the containers represent objects to be expired | 01:50 |
| *** mkollaro has quit IRC | 01:51 | |
| *** d89 has quit IRC | 01:51 | |
| godb | hum.. if i delete expire file (.ts file and directory) directly. i worried confilct sqlite3 database .. there any no problems?? | 01:52 |
| godb | or.. need to change any value in the sqlite3 database ? | 01:53 |
| clayg | godb: yeah expired object effected by that bug still had a row in their container databases IIRC | 01:53 |
| clayg | godb: probably better just to make the delete with the correct x-if-delete-at header via internal_client and let the container updates propogate like normal | 01:54 |
| clayg | the trick is finding the .data files that are expired weren't deleted | 01:54 |
| clayg | *expired _but_ weren't deleted | 01:55 |
| godb | that's good. i try it. | 01:55 |
| godb | and. i saw empth director in the container partition | 01:56 |
| godb | can i delete these?? | 01:56 |
| godb | empty eirectory | 01:56 |
| clayg | if you get a chance you might grep around for 'Object audit (ALL) "forever" mode completed' - and see what the cycle time is on your auditors - it be a good proxy for how long it'd take to find expired .data files walking the tree directly | 01:56 |
| clayg | the directory name was just like a 3 char hex string? (a suffix) | 01:57 |
| godb | no it is number | 01:57 |
| godb | like this /sdb1/containers/153130 | 01:58 |
| clayg | so /srv/node/sdb/containers/<part> where part is an ... | 01:58 |
| clayg | yeah so thats' the partition | 01:58 |
| godb | can i delete this empty dir? | 01:58 |
| clayg | wellllll yeah if it's empty you can - but that sort of cleanup is normally the replicators job - lemme look at some code | 01:59 |
| godb | thx.:) .. last one more question. | 01:59 |
| godb | my swift version is 1.13. | 02:00 |
| *** matsuhashi has quit IRC | 02:00 | |
| godb | the bug that replicator can't delete expire object (.ts file and director) is fix cleraly? in 1.13 version? | 02:01 |
| clayg | hrm... yeah it looks like the container replicator doesn't clear up the partition dirs? | 02:02 |
| godb | yeah. | 02:02 |
| clayg | the expiring object but that caused expired objects to not be deleted is fixed in 1.13 | 02:03 |
| clayg | but it couldn't retroactivly issues delete's for the previously expired but not deleted objects | 02:03 |
| godb | that very good job!! .. my swift suffer from many inode .. :( | 02:04 |
| clayg | i'll have to file a bug on container-replicators not cleaning up handoff partitions... seems very strange | 02:04 |
| godb | me too. the count of dir in the contaeinrs partition is 283 | 02:05 |
| godb | but empty dirs are 214 | 02:05 |
| godb | total : 283 .. empty 214 | 02:06 |
| *** d89 has joined #openstack-swift | 02:07 | |
| *** matsuhashi has joined #openstack-swift | 02:08 | |
| godb | i will check more that issue. | 02:08 |
| clayg | godb: https://bugs.launchpad.net/swift/+bug/1297580 | 02:11 |
| godb | clayg : thx. i will more check it. | 02:12 |
| *** saschpe has quit IRC | 02:24 | |
| godb | !Æ®·¹ÀÌ | 02:25 |
| godb | ¢Æ Zero IRC ¢Æ < Æ®·¹ÀÌ»óÅ·ΠÀüȯÇÕ´Ï´Ù! > | 02:25 |
| openstack | godb: Error: "Æ®·¹ÀÌ" is not a valid command. | 02:25 |
| *** saschpe has joined #openstack-swift | 02:27 | |
| *** krtaylor has joined #openstack-swift | 02:38 | |
| portante | clayg, notmyname, dfg, others: the in-process functional tests were very helpful in finding the proxy-logging bug today, I did there work on that branch first and then moved it to master | 02:42 |
| portante | I think it will be a very useful addition to help debug these situations | 02:43 |
| clayg | portante: did you fix the thing where they don't work for that one test that was broken? | 02:43 |
| portante | clayg: I believe I did, but let me double check | 02:43 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Fix PUT to existing container w/o specific storage policy https://review.openstack.org/82590 | 02:45 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Container Confict on obj update with wrong Storage-Policy-Index https://review.openstack.org/79731 | 02:45 |
| portante | clayg: all the functional tests in-process are OKs, except 3 SKIPs, the test_tempauth_account_acls and the two test_file_size_limit runs | 02:48 |
| portante | I think notmyname's patch fixes the tempauth one, right? | 02:48 |
| clayg | no | 02:49 |
| portante | okay, I'll track down why they are skipped | 02:49 |
| clayg | well i mean when I was testing the in-process tests I just fixed test_tempauth_account_acls by hand (as far as the container_info call goes) because I'd already fixed in the account_acl func test patch and it was still broken because it was re-reading the test.conf from disk and going and talking to my real servers or some crazyness - it was terrible | 02:50 |
| clayg | just make your change dependent on notmynames fix and work from there, I think you just need to say something like get_config... well I don't remember - it's whatever I ended up doing in the acct acl func test patch | 02:50 |
| clayg | oh ptpthht - which i'm fixing a pep8 issue on now | 02:51 |
| portante | okay, thanks, now I remember us discussion that, I'll do that | 02:51 |
| openstackgerrit | Yuan Zhou proposed a change to openstack/swift: Update swift-get-nodes to use storage_directory util func https://review.openstack.org/82968 | 02:52 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: additional functional tests for account acls https://review.openstack.org/80073 | 02:53 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Clean up Account ACL sysmeta in AccountController https://review.openstack.org/80140 | 02:53 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Harden v2 common.middleware.acl interface https://review.openstack.org/80188 | 02:53 |
| *** gyee has quit IRC | 02:56 | |
| wer | what does it mean when an object gets .temp added to it? I fear bad. | 02:56 |
| clayg | probably just a cancled upload | 02:57 |
| clayg | yeah nm, i thought those went in /srv/node/device/tmp anyway... what do you mean ".temp added to it" | 02:58 |
| wer | I ran swift upload some_container foo.xml and it 500'd :/ And now I have some_container/foo.xml.temp | 03:00 |
| wer | The cluster has a lot of writes happening right now... but I have never 500'd when attempting to upload something :/ | 03:00 |
| clayg | not sure on that one, swift won't create an object name like that - and I don't see how swiftclient would do it either | 03:02 |
| clayg | normally a 500 just ends up looking the object upload never happened | 03:03 |
| wer | ok maybe it is unrelated.... it possible that file is legit or something. | 03:03 |
| clayg | sometimes (if one of the backend nodes was successful) you can get a 500 and still have the object exist - but it has to be on the very last byte, so it's pretty rare | 03:03 |
| clayg | either way it ends up having the same name - or it's not there at all (common case) | 03:03 |
| wer | ok. I must not be chasing the right thing. | 03:04 |
| clayg | you could try and find the trasnaction id of the 500'd request and dig for more info wrt what happened on the backend | 03:04 |
| clayg | ChunkWriteTimeout or some such most likly | 03:04 |
| wer | yeah clarkb I think I saw some of those. I am worried I am in a weird state. | 03:05 |
| wer | Object HEAD failed is what repeated attempts to upload gives me. Ill dig in the logs. | 03:06 |
| wer | it's just that one object :) I can't seem to stat it from any of the nodes so far. | 03:10 |
| wer | ok I found the tx where it got deleted. | 03:12 |
| wer | I see an object-server 204 the DELETE and the proxy 204. Shouldn't I see two other object server DELETE requests if I have replication at 3? | 03:14 |
| openstackgerrit | Yuan Zhou proposed a change to openstack/swift: Indicate default storage policy in /info response https://review.openstack.org/82969 | 03:14 |
| *** matsuhashi has quit IRC | 03:15 | |
| clayg | yuan: json has a boolean type bro ;) | 03:26 |
| yuan | clayg: ahh, thanks. will update it. | 03:28 |
| *** nosnos has quit IRC | 03:28 | |
| Anju1 | notmyname, clayg : hii | 03:35 |
| Anju1 | does swift provide any any api to create and delete metadata ata the same time ? | 03:36 |
| clayg | Anju1: like create/update one key and remove a diferent key in the same request - yes | 03:37 |
| Anju1 | clayg: hii , https://github.com/openstack/tempest/blob/master/tempest/api/object_storage/test_account_services.py#L336 | 03:38 |
| Anju1 | can you please check this test | 03:38 |
| openstackgerrit | Yuan Zhou proposed a change to openstack/swift: Indicate default storage policy in /info response https://review.openstack.org/82969 | 03:40 |
| clayg | Anju1: where is self.account_client.create_and_delete_account_metadata defined at? | 03:41 |
| Anju1 | https://github.com/openstack/tempest/blob/master/tempest/services/object_storage/account_client.py#L94 | 03:41 |
| Anju1 | clayg: ^^^ | 03:42 |
| dfg | portante: i just meant to time from the proxy logging middleware to the proxy server itself- to see the overhead of all the middleware we have by itself. anyway- nm. | 03:48 |
| clayg | Anju1: yeah i just tested it and it works for me - i stared at the test case for awhile but nothing jumps out at me | 03:48 |
| Anju1 | clayg: https://github.com/openstack/tempest/blob/master/tempest/services/object_storage/account_client.py#L94 did you check the method | 03:49 |
| portante | dfg: I think that would be interesting and easy to do with this change. | 03:51 |
| clayg | Anju1: yeah it looked fine at first glance, i'd have to step through it - but at lest you know to keep digging -> https://gist.github.com/clayg/9776706 | 03:53 |
| madhuri | clayg: Thanks for reviewing :) | 04:09 |
| clayg | does that container-info change conflict with the account-info change? | 04:09 |
| clayg | i feel like they both wanted to update the man pages or something | 04:09 |
| clayg | same nit on account-info RE "Account_hash" btw | 04:09 |
| madhuri | no | 04:12 |
| madhuri | clayg: Yes, I noticed that | 04:12 |
| madhuri | clayg: what happens when more than 1024 arrives at proxy server at a time? | 04:24 |
| *** matsuhashi has joined #openstack-swift | 04:25 | |
| madhuri | Does it start discarding the requests? | 04:25 |
| openstackgerrit | MORITA Kazutaka proposed a change to openstack/swift: Remove spaces between hacking rules https://review.openstack.org/82984 | 04:26 |
| clayg | you mean a single worker - and no eventlet.wsgi just stops accepting the sockets - eventually the tcp layer in linux probably backs up | 04:26 |
| Anju1 | clayg: if we can use the remove header for removing the metadata , | 04:30 |
| Anju1 | then what is the usability of passing a metadata without any value | 04:31 |
| clayg | i think the -remove was added before curl supported the -H 'header-value;' syntax for sending empty headers | 04:32 |
| clayg | Anju1: so the -remove thing was just a friendly thing for clients that didn't like to send empty headers... | 04:32 |
| clayg | Anju1: both will be supported | 04:32 |
| openstackgerrit | MORITA Kazutaka proposed a change to openstack/swift: Update swift3 repository https://review.openstack.org/82985 | 04:33 |
| *** nosnos has joined #openstack-swift | 04:36 | |
| clayg | we just forked fujita? - hopefully he an update his README to reflect the new project home and we can get some original authorship credits for him on the stackforge project | 04:38 |
| *** Manish_ has joined #openstack-swift | 04:39 | |
| Manish_ | Hi Clayg.....I have one query here | 04:39 |
| * clayg feels like the late night swift support team - hugokuo where you at my man? :P | 04:40 | |
| clayg | Manish_: what's up? | 04:41 |
| Manish_ | if somehow all the request are coming for one partition , so after some time that partition will be full and that partition will not be able to handle more data. What will happen in this scenario and Is there a mechanism to avoid this scenario? | 04:41 |
| hugokuo | wasup | 04:41 |
| clayg | if a storage node gets full it returns 507, the proxy will error limit the node, and new data will be placed on handoff | 04:42 |
| *** creiht has quit IRC | 04:43 | |
| clayg | generally speaking (and emperically verified with monitoring data on most well operated clusters) the data per drive (typically with 100's or 1000's of partitions) is well distributed within 5-10% | 04:43 |
| Manish_ | So, we have handled this scenario but haven't define a way to avoid this. Right? | 04:43 |
| * hugokuo Manish_: am curious about the partition in your definition. Do you mean a physical disk or the logical partition number in Swift ? | 04:44 | |
| clayg | the indroduction of the hash_path_suffix stuff very early on (pre-opensource IIRC) was added specifically to avoid being able to pre-calculate the partition for an object without the secret | 04:44 |
| Manish_ | partition here is a logival directory on a disk | 04:45 |
| Manish_ | ok.. | 04:46 |
| hugokuo | k | 04:46 |
| Manish_ | So handoff is for storage node...but there is a possibility(although very-very thin possibility) that one partition gets all the requests accidently..right? | 04:47 |
| Manish_ | all request ...means sufficient number of requests to full that partition | 04:48 |
| hugokuo | hmm.... It's possible. But the chance is quite low tho. At least not in my life. If the average object size is relevant large, then the chance is higher. Large I mean to contract the disk size. | 04:52 |
| clayg | if you have small disks or you're running with a small part power you can tweek your max object size to force your clients to help you get your varience back in check | 04:53 |
| * hugokuo if there're 20 100GB disks in your cluster. And the average object size about 4GB. Then it has higher chance to full a single drive. | 04:53 | |
| * hugokuo clayg you are really my dude..... | 04:54 | |
| * clayg hoping hugokuo is going to be in ALT? | 04:54 | |
| hugokuo | So if any object larger than the max_object_size, jut to ask clients using manifest feature for uploading larger object. You will have better distribution rate. | 04:56 |
| hugokuo | clayg: I think yes. A session was accepted ... | 04:56 |
| * hugokuo manifest means SLO or DLO... | 04:56 | |
| Manish_ | Ok...so if one storage node is full, new data will be placed on handoff node. Now in this case ,for object GET it will have to read object from handoff node.. thus there must be some special handling.. | 04:56 |
| clayg | the same error limiting applied on the PUT will apply to the GET - proxy'll find the handoffs pretty quick | 04:57 |
| hugokuo | Same philosophy .... proxy-server is super ... | 04:57 |
| clayg | but I'm worried that you're worrying about this? Are you trying to run with a really low part power or do you not really follow that swift's distribution making for effectively event drive utilization isn't like a property of good luck or something? | 04:59 |
| Manish_ | I am just curious about these corner cases...:) | 05:00 |
| clayg | oic | 05:00 |
| clayg | make a ring with a really small part power and upload more than 2 ** part_power objects - about half small and half large | 05:01 |
| Manish_ | Sorry..if i am troubling much.... | 05:01 |
| clayg | then upload another 2 ** part_power objects with a similar split and watch your utilization per disk start to drift apart | 05:01 |
| clayg | if you keep at you'll be able to get one of them filled up and you start to poke at swift's error limiting and stable handoffs | 05:02 |
| clayg | vagrant-swift-all-in-one let's you play around with different numbers of devices and part_power pretty easily | 05:03 |
| Manish_ | ok... | 05:03 |
| *** Midnightmyth has joined #openstack-swift | 05:04 | |
| Manish_ | So, if my PUT request has gone to Handoff node and later on i am trying GET that object....So does ring directly provides me the address of Handsoff node or is there some other mechanism? | 05:04 |
| clayg | Manish_: it's no big deal, I struggled with it for awhile as well, even distribution of the number of objects per partition didn't seem like it would be enough to satisfy even drive utilization, but with a reasonablly size part_power I've come to grok that it does - because math | 05:05 |
| clayg | yeah it's in the ring - play with swift-get-nodes | 05:05 |
| hugokuo | swift-get-nodes /etc/swift/object.ring.gz AUTH_OH yeah swift.obj | 05:07 |
| Manish_ | Ok..thanks alot for the help Clayg | 05:07 |
| clayg | you should do the small part power upload lots of objects thing - I think it would be really interesting to see a write up on it if you took some notes | 05:08 |
| *** ppai has joined #openstack-swift | 05:18 | |
| *** Longgeek_ has joined #openstack-swift | 05:19 | |
| *** Longgee__ has joined #openstack-swift | 05:20 | |
| *** Longgeek_ has quit IRC | 05:24 | |
| Manish_ | ok | 05:33 |
| *** Longgee__ has quit IRC | 05:39 | |
| *** Diddi has joined #openstack-swift | 06:08 | |
| *** Midnightmyth has quit IRC | 06:10 | |
| *** bvandenh has quit IRC | 06:13 | |
| wer | clayg: BTW that 500 problem was all me. I did it :) | 06:17 |
| *** saju_m has joined #openstack-swift | 06:19 | |
| *** mmcardle has joined #openstack-swift | 06:26 | |
| *** mmcardle has quit IRC | 06:31 | |
| *** nshaikh has joined #openstack-swift | 06:32 | |
| *** matsuhashi has quit IRC | 06:33 | |
| *** matsuhashi has joined #openstack-swift | 06:35 | |
| *** psharma has joined #openstack-swift | 06:37 | |
| *** matsuhas_ has joined #openstack-swift | 07:03 | |
| *** matsuhas_ has quit IRC | 07:04 | |
| *** matsuhas_ has joined #openstack-swift | 07:06 | |
| *** matsuhashi has quit IRC | 07:07 | |
| openstackgerrit | A change was merged to openstack/swift: Added swift-container-info tool. https://review.openstack.org/81968 | 07:07 |
| *** saju_m has quit IRC | 07:13 | |
| openstackgerrit | A change was merged to openstack/swift: Indicate default storage policy in /info response https://review.openstack.org/82969 | 07:32 |
| *** matsuhas_ has quit IRC | 07:59 | |
| *** matsuhashi has joined #openstack-swift | 08:04 | |
| *** bvandenh has joined #openstack-swift | 08:07 | |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Add object-reconciler daemon https://review.openstack.org/80730 | 08:11 |
| openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Add functional tests for python-swiftclient https://review.openstack.org/76355 | 08:14 |
| godb | !Æ®·¹ÀÌ | 08:29 |
| godb | ¢Æ Zero IRC ¢Æ < Æ®·¹ÀÌ»óÅ·ΠÀüȯÇÕ´Ï´Ù! > | 08:29 |
| openstack | godb: Error: "Æ®·¹ÀÌ" is not a valid command. | 08:29 |
| *** chandankumar_ has joined #openstack-swift | 08:42 | |
| *** nshaikh has quit IRC | 08:44 | |
| *** mlipchuk has joined #openstack-swift | 08:52 | |
| *** saschpe has quit IRC | 08:54 | |
| *** saschpe has joined #openstack-swift | 09:03 | |
| *** mmcardle has joined #openstack-swift | 09:18 | |
| *** mmcardle1 has joined #openstack-swift | 09:19 | |
| *** mmcardle has quit IRC | 09:19 | |
| *** godb has quit IRC | 09:20 | |
| *** Trozz has quit IRC | 09:28 | |
| *** Trozz has joined #openstack-swift | 09:29 | |
| *** Trozz_ has joined #openstack-swift | 09:30 | |
| *** Trozz_ has quit IRC | 09:30 | |
| *** Trozz has quit IRC | 09:30 | |
| *** Trozz has joined #openstack-swift | 09:31 | |
| *** d89 has quit IRC | 09:31 | |
| *** Trozz_ has joined #openstack-swift | 09:31 | |
| *** mich5225 has joined #openstack-swift | 09:32 | |
| *** Trozz has quit IRC | 09:33 | |
| *** Trozz_ has quit IRC | 09:33 | |
| *** mich5225 has quit IRC | 09:33 | |
| *** Trozz has joined #openstack-swift | 09:33 | |
| *** mmcardle1 has quit IRC | 09:36 | |
| *** mmcardle has joined #openstack-swift | 09:39 | |
| *** Trozz has quit IRC | 09:39 | |
| *** Trozz has joined #openstack-swift | 09:40 | |
| *** Trozz has quit IRC | 09:40 | |
| *** gvernik has joined #openstack-swift | 09:41 | |
| *** Trozz has joined #openstack-swift | 09:43 | |
| *** Trozz has quit IRC | 09:45 | |
| *** Manish_ has quit IRC | 10:08 | |
| *** matsuhashi has quit IRC | 10:09 | |
| *** dmorita has joined #openstack-swift | 10:09 | |
| *** Trixboxer has joined #openstack-swift | 10:10 | |
| *** chandankumar_ has quit IRC | 10:22 | |
| *** nacim has joined #openstack-swift | 10:24 | |
| *** mkollaro has joined #openstack-swift | 10:25 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/swift: Add containeralias middleware https://review.openstack.org/62494 | 10:31 |
| *** Trozz has joined #openstack-swift | 10:31 | |
| *** Trozz has quit IRC | 10:34 | |
| *** matsuhashi has joined #openstack-swift | 10:39 | |
| *** Trozz has joined #openstack-swift | 10:41 | |
| openstackgerrit | A change was merged to openstack/swift: fix a skipped account ACLs functional test https://review.openstack.org/82922 | 10:41 |
| *** Trozz has quit IRC | 10:42 | |
| *** Trozz has joined #openstack-swift | 10:44 | |
| *** Trozz has quit IRC | 10:45 | |
| *** mmcardle1 has joined #openstack-swift | 10:54 | |
| *** mkollaro has quit IRC | 10:54 | |
| *** mmcardle has quit IRC | 10:57 | |
| *** saju_m has joined #openstack-swift | 11:20 | |
| *** Trozz has joined #openstack-swift | 11:23 | |
| *** Trozz has quit IRC | 11:23 | |
| *** Trozz has joined #openstack-swift | 11:29 | |
| *** Trozz has quit IRC | 11:36 | |
| *** Diddi has quit IRC | 11:40 | |
| *** nshaikh has joined #openstack-swift | 11:42 | |
| *** Trozz has joined #openstack-swift | 12:02 | |
| *** ppai has quit IRC | 12:02 | |
| *** Trozz has quit IRC | 12:03 | |
| *** mlipchuk has quit IRC | 12:05 | |
| *** matsuhashi has quit IRC | 12:10 | |
| *** matsuhashi has joined #openstack-swift | 12:11 | |
| *** mmcardle1 has quit IRC | 12:13 | |
| *** matsuhas_ has joined #openstack-swift | 12:13 | |
| *** matsuhashi has quit IRC | 12:13 | |
| *** mmcardle has joined #openstack-swift | 12:13 | |
| *** mkollaro has joined #openstack-swift | 12:15 | |
| *** ppai has joined #openstack-swift | 12:16 | |
| *** mmcardle has quit IRC | 12:16 | |
| *** mkollaro has quit IRC | 12:21 | |
| *** mlipchuk has joined #openstack-swift | 12:23 | |
| *** JuanManuelOlle has joined #openstack-swift | 12:24 | |
| *** mkollaro has joined #openstack-swift | 12:36 | |
| *** ppai has quit IRC | 12:39 | |
| *** mmcardle has joined #openstack-swift | 12:44 | |
| *** mmcardle has quit IRC | 12:48 | |
| *** ppai has joined #openstack-swift | 12:52 | |
| *** dmorita has quit IRC | 12:54 | |
| *** mmcardle has joined #openstack-swift | 13:00 | |
| *** JuanManuelOlle has quit IRC | 13:01 | |
| *** tanee-away is now known as tanee | 13:01 | |
| *** mlipchuk has quit IRC | 13:01 | |
| *** tanee is now known as tanee-away | 13:01 | |
| *** matsuhas_ has quit IRC | 13:07 | |
| *** jairo has quit IRC | 13:08 | |
| *** tanee-away is now known as tanee | 13:16 | |
| *** mlipchuk has joined #openstack-swift | 13:16 | |
| *** ppai has quit IRC | 13:20 | |
| *** ppai has joined #openstack-swift | 13:23 | |
| *** nacim has quit IRC | 13:34 | |
| *** nosnos has quit IRC | 13:35 | |
| openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Add functional tests for python-swiftclient https://review.openstack.org/76355 | 13:46 |
| *** Trixboxer has quit IRC | 13:49 | |
| *** nacim has joined #openstack-swift | 13:51 | |
| *** Midnightmyth has joined #openstack-swift | 13:53 | |
| *** dmsimard has joined #openstack-swift | 13:56 | |
| *** creiht has joined #openstack-swift | 13:57 | |
| *** ChanServ sets mode: +v creiht | 13:57 | |
| *** Trixboxer has joined #openstack-swift | 14:01 | |
| *** gvernik has left #openstack-swift | 14:04 | |
| *** ppai has quit IRC | 14:07 | |
| *** Midnightmyth has quit IRC | 14:07 | |
| *** tdasilva has joined #openstack-swift | 14:26 | |
| *** mlipchuk has quit IRC | 14:37 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Add Storage Policy Support to Account HEAD https://review.openstack.org/73747 | 14:37 |
| *** psharma has quit IRC | 14:38 | |
| *** piyush1 has joined #openstack-swift | 14:39 | |
| *** chandan_kumar has quit IRC | 14:42 | |
| *** chandankumar_ has joined #openstack-swift | 14:43 | |
| *** chandankumar_ has quit IRC | 14:46 | |
| *** chandan_kumar has joined #openstack-swift | 14:47 | |
| *** chandan_kumar has quit IRC | 14:48 | |
| *** chandan_kumar has joined #openstack-swift | 14:48 | |
| *** dvas has joined #openstack-swift | 14:51 | |
| *** zaitcev has joined #openstack-swift | 14:54 | |
| *** ChanServ sets mode: +v zaitcev | 14:54 | |
| *** mlipchuk has joined #openstack-swift | 14:54 | |
| *** chandan_kumar has quit IRC | 14:55 | |
| *** chandan_kumar has joined #openstack-swift | 14:56 | |
| notmyname | good morning | 14:57 |
| *** dvas has quit IRC | 14:58 | |
| notmyname | so evault says that swift is awesome (12 9s of durability) https://www.evault.com/uncategorized/how-safe-is-your-data-with-openstack-swift/ | 14:58 |
| *** dvas_ has joined #openstack-swift | 14:59 | |
| portante | nice | 15:02 |
| *** bvandenh has quit IRC | 15:02 | |
| *** chandan_kumar has quit IRC | 15:03 | |
| *** JuanManuelOlle has joined #openstack-swift | 15:04 | |
| notmyname | clayg: portante: swift3 is now officially homed in stackforge, using gerrit and openstack-ci for tests | 15:06 |
| *** dvas_ has quit IRC | 15:06 | |
| *** chandan_kumar has joined #openstack-swift | 15:09 | |
| openstackgerrit | Yuan Zhou proposed a change to openstack/swift: Update container storage_policy_index when auditing https://review.openstack.org/83093 | 15:12 |
| portante | oh, nice | 15:13 |
| *** dvas_ has joined #openstack-swift | 15:13 | |
| *** tanee is now known as tanee-away | 15:17 | |
| *** JuanManuelOlle has quit IRC | 15:22 | |
| notmyname | reminder that there is a swift team meeting in 3.5 hours | 15:29 |
| notmyname | acoles_: (is eamonn in IRC?) can you bug eamonn about rebasing https://review.openstack.org/#/c/59778/ | 15:36 |
| acoles_ | notmyname: will do | 15:37 |
| notmyname | acoles_: thanks :-) | 15:37 |
| acoles_ | otoolee^^ | 15:37 |
| notmyname | otoolee: oh hi! | 15:37 |
| acoles_ | notmyname: i have other means too... :) | 15:38 |
| notmyname | I just updated https://wiki.openstack.org/wiki/Swift/PriorityReviews | 15:40 |
| *** tanee-away is now known as tanee | 15:42 | |
| *** saju_m has quit IRC | 15:43 | |
| otoolee | notmyname: Sorry, I was off IRC for a few days. I'll rebase. | 15:45 |
| notmyname | otoolee: no worries. actually I didn't know you were on IRC before now. :-) | 15:46 |
| notmyname | (you can't hide anymore!) | 15:46 |
| *** mlipchuk1 has joined #openstack-swift | 15:48 | |
| *** mlipchuk has quit IRC | 15:48 | |
| *** nshaikh has quit IRC | 15:53 | |
| *** dvas_ has quit IRC | 15:57 | |
| *** piyush1 has quit IRC | 15:59 | |
| *** Ankit has joined #openstack-swift | 16:17 | |
| *** piyush1 has joined #openstack-swift | 16:21 | |
| *** leews has joined #openstack-swift | 16:22 | |
| openstackgerrit | A change was merged to openstack/swift: Update swift3 repository https://review.openstack.org/82985 | 16:23 |
| *** piyush2 has joined #openstack-swift | 16:24 | |
| *** piyush1 has quit IRC | 16:25 | |
| *** madhuri_ has joined #openstack-swift | 16:27 | |
| madhuri_ | clayg: ping? | 16:28 |
| *** nacim has quit IRC | 16:30 | |
| openstackgerrit | Eamonn O'Toole proposed a change to openstack/swift: Parallel object auditor https://review.openstack.org/59778 | 16:33 |
| *** jasondotstar has joined #openstack-swift | 16:33 | |
| openstackgerrit | Jenkins proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/81207 | 16:34 |
| otoolee | notmyname: Rebase finished and submitted. | 16:34 |
| notmyname | otoolee: thanks | 16:36 |
| notmyname | anyone want an invite to http://keybase.io ? | 16:43 |
| Ankit | Is django swiftbrowser work properly for managing swift cluster??? | 16:45 |
| *** JuanManuelOlle has joined #openstack-swift | 16:46 | |
| *** Ankit has quit IRC | 16:46 | |
| *** jasondotstar has quit IRC | 16:50 | |
| *** JuanManuelOlle1 has joined #openstack-swift | 16:51 | |
| *** JuanManuelOlle has quit IRC | 16:51 | |
| *** erlon has joined #openstack-swift | 16:52 | |
| *** pconstantine has joined #openstack-swift | 16:53 | |
| pconstantine | has anybody ever seen this one? | 16:54 |
| pconstantine | File "swift/swift/common/wsgi.py", line 133, in parsetype | 16:54 |
| pconstantine | orig_parsetype(self) | 16:54 |
| pconstantine | RuntimeError: maximum recursion depth exceeded while calling a Python object | 16:54 |
| pconstantine | proxy: Account HEAD returning 503 for [] | 16:54 |
| pconstantine | proxy: 127.0.0.1 127.0.0.1 26/Mar/2014/16/51/52 PUT /v1/a/5_oc_ascii/5_o_non_ascii%25C2%25A3 HTTP/1.0 412 - - t - 92 - txa77e9e65fbba435797378-00533305a8 - 0.0472 - - 1395852712.002628088 1395852712.049833059 | 16:54 |
| pconstantine | it starts appearing when I add more tests to the unit/proxy/test_server.py | 16:55 |
| pconstantine | even copy-pasting old tests and renaming them causes this error | 16:56 |
| pconstantine | happens exactly on adding 13th test to test_proxy.py | 16:57 |
| pconstantine | how ironic | 16:57 |
| *** chandan_kumar has quit IRC | 16:57 | |
| *** mlipchuk1 has quit IRC | 17:01 | |
| *** JuanManuelOlle has joined #openstack-swift | 17:06 | |
| notmyname | pconstantine: ok, that seems odd | 17:07 |
| *** shri has joined #openstack-swift | 17:07 | |
| *** JuanManuelOlle1 has quit IRC | 17:08 | |
| *** piyush2 has quit IRC | 17:09 | |
| *** gvernik has joined #openstack-swift | 17:13 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Add Storage Policy Support to Account HEAD https://review.openstack.org/73747 | 17:14 |
| notmyname | to all my ops friends: https://twitter.com/jongold/status/448815087785365504 | 17:14 |
| *** Midnightmyth has joined #openstack-swift | 17:19 | |
| *** dvas_ has joined #openstack-swift | 17:27 | |
| pconstantine | notmyname: oh, God, monkey_path_mime_tools() is called from TestObjectController.setUp() | 17:30 |
| pconstantine | commenting it out seem to fix things | 17:30 |
| pconstantine | looks like it's called for each test run | 17:31 |
| notmyname | pconstantine: so what you are saying is that it's the monkey's fault? | 17:32 |
| pconstantine | I dunno, but will it take a month of code review to get it into core? :) | 17:39 |
| notmyname | lol | 17:39 |
| *** gyee has joined #openstack-swift | 17:40 | |
| pconstantine | notmyname: removing monkey caused unit tests to run noticeably faster... | 17:49 |
| pconstantine | hmm, maybe it's my imagination... | 17:50 |
| *** saju_m has joined #openstack-swift | 17:57 | |
| notmyname | pconstantine: isn't there a mechanism to make a module-level setup? or why not just put in in that class's constructor? | 17:58 |
| pconstantine | notmyname: it's already in module setup | 17:58 |
| pconstantine | it was run once on setup of the module, and then each time on setup of each test | 17:59 |
| notmyname | well why's it in the class setUp then? ;-) | 17:59 |
| pconstantine | I've removed it from test setup | 17:59 |
| pconstantine | why? I think because somebody left it there | 18:00 |
| pconstantine | :) | 18:00 |
| notmyname | git blame is weird on that. the (big) patch is all taking out bare excepts. and adding that one line | 18:01 |
| notmyname | we can blame early-2011 dfg | 18:01 |
| notmyname | ;-) | 18:01 |
| dfg | huh? | 18:02 |
| *** tanee is now known as tanee-away | 18:02 | |
| notmyname | heh | 18:02 |
| notmyname | the module-level setup was done (by redbo) just a few days before the class-level setup. so I'm guessing that both of them patched something in a non-conflicting way and then they both landed | 18:03 |
| *** madhuri_ has quit IRC | 18:05 | |
| notmyname | dfg: you and redbo both fixed something in early 2011, but it turns out they interact in fun and exciting ways. they are monkey-patching each other, so you get a recursion-depth error when you add more tests | 18:05 |
| redbo | sounds like monkey_patch_mimetools needs an "already patched" flag | 18:05 |
| notmyname | pconstantine: so ya, it looks like the fix is to remove the call in TestObjectController | 18:05 |
| notmyname | redbo: ya | 18:05 |
| *** JuanManuelOlle1 has joined #openstack-swift | 18:08 | |
| notmyname | redbo: eg https://gist.github.com/notmyname/8a7d5fd44aea0ce8bbdf ? | 18:09 |
| *** JuanManuelOlle has quit IRC | 18:09 | |
| redbo | I think it needs a "not" on that if-check | 18:10 |
| *** dvas_ has quit IRC | 18:10 | |
| notmyname | redbo: ah yes. thanks :-) | 18:10 |
| *** JuanManuelOlle has joined #openstack-swift | 18:11 | |
| openstackgerrit | John Dickinson proposed a change to openstack/swift: don't repatch mimetools again. and again. again. https://review.openstack.org/83156 | 18:11 |
| notmyname | pconstantine: can you try that ^ | 18:11 |
| *** dvas_ has joined #openstack-swift | 18:11 | |
| *** JuanManuelOlle1 has quit IRC | 18:13 | |
| dfg | wow- that was a long time ago | 18:13 |
| redbo | like 2 kids ago for you | 18:13 |
| pconstantine | notmyname: slightly later, I need to rewrite a lot of tests right now... | 18:13 |
| notmyname | pconstantine: ok | 18:14 |
| *** dvas_ has quit IRC | 18:16 | |
| *** mmcardle has quit IRC | 18:20 | |
| *** mmcardle has joined #openstack-swift | 18:23 | |
| *** mlipchuk has joined #openstack-swift | 18:23 | |
| *** piyush has joined #openstack-swift | 18:25 | |
| *** piyush1 has joined #openstack-swift | 18:27 | |
| *** piyush has quit IRC | 18:30 | |
| creiht | clayg: ping | 18:32 |
| creiht | clayg: so I'm trying to understand this | 18:32 |
| creiht | and feel a bit stupid | 18:33 |
| *** mmcardle has quit IRC | 18:33 | |
| creiht | but I updated the review about the tox stuff | 18:33 |
| clayg | you're not stupid - IT's stupid | 18:33 |
| creiht | tox is too much magic for me | 18:33 |
| creiht | heh | 18:33 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Add Storage Policy Support to Account HEAD https://review.openstack.org/73747 | 18:34 |
| *** lpabon has joined #openstack-swift | 18:35 | |
| creiht | clayg: so what I'm doing wrong, or what am I missing? | 18:35 |
| creiht | sorry that this has been more of a pain then you were probably expecting | 18:35 |
| *** IRTermite has left #openstack-swift | 18:36 | |
| *** JuanManuelOlle has quit IRC | 18:40 | |
| peluse | torgomatic: good to go in my book the the acct rollup, nice call on the new test - rest assured I did that manually of course before checking in though :) | 18:41 |
| clayg | creiht: i'm guessing that tox -e pep8 doesn't work for you on master either yeah? | 18:42 |
| torgomatic | peluse: naturally :) | 18:42 |
| creiht | clayg: not it doesn't | 18:42 |
| clayg | creiht: if you go into the .tox/pep8 dir and source bin/activate - what do you get from flake8 --version | 18:42 |
| creiht | $ flake8 --version | 18:43 |
| creiht | 2.0 (pep8: 1.4.5, mccabe: 0.2.1, pyflakes: 0.7.3, GitCheckCommitTitleBug: 0.0.1, GitCheckCommitTitleLength: 0.0.1, GitCheckCommitTitlePeriodEnding: 0.0.1, hacking.core: 0.0.1, ProxyChecker: 0.0.1) | 18:43 |
| notmyname | peluse: when I saw your comment, I thought you had "manually cursed" before checking it in :-) | 18:43 |
| *** mkollaro has quit IRC | 18:44 | |
| clayg | and if you "flake8 ~/swift" do you get the same erros as the tox run? | 18:44 |
| dfg | irc | 18:44 |
| creiht | clayg: yup | 18:44 |
| torgomatic | peluse: okay, in that case, I'm shipping it | 18:45 |
| peluse | rock n roll | 18:45 |
| clayg | creiht: crazy... so like what are they? EXXX | 18:45 |
| peluse | notmyname: that too! | 18:46 |
| *** mkollaro has joined #openstack-swift | 18:48 | |
| creiht | clayg: http://paste.openstack.org/show/74381/ | 18:48 |
| creiht | that's the first couple of lines | 18:49 |
| *** lpabon has quit IRC | 18:49 | |
| clayg | creiht: lol - that's not even our code? | 18:49 |
| clayg | swift/build ? I think pip did something stupid - delete that shit | 18:49 |
| creiht | hrm | 18:50 |
| creiht | lol | 18:50 |
| * creiht blames pbr :) | 18:50 | |
| clayg | YES! ^ that | 18:50 |
| torgomatic | huh... I saw something like that in swiftclient, and I fixed it by making tox's pep8 only check certain dirs | 18:50 |
| *** nacim has joined #openstack-swift | 18:51 | |
| clayg | torgomatic: don't try and not blame pbr | 18:51 |
| creiht | clayg: heh and there it goes | 18:51 |
| torgomatic | :) | 18:51 |
| torgomatic | I never did figure out who was putting crap in build; could have been pbr for all I know | 18:52 |
| creiht | clayg: well to be fair, for some reason I had a pbr .egg dir in my swift dir :/ | 18:52 |
| creiht | clayg: and whatever happened to the pbr removal stuff? ;) | 18:53 |
| notmyname | that's on me | 18:54 |
| creiht | notmyname: then go direct some technology and get it done! :) | 18:54 |
| peluse | meeting in 5 min right? | 18:54 |
| notmyname | peluse: yes | 18:54 |
| creiht | clayg: ok thanks for helping me figure out what the heck was going on | 18:55 |
| *** JuanManuelOlle has joined #openstack-swift | 18:56 | |
| zaitcev | meeing? | 18:56 |
| notmyname | normal weekly swift team meeting | 18:57 |
| notmyname | in 3 minutes in #openstack-meeting | 18:57 |
| creiht | notmyname: I will have to jet in 30 mins | 18:59 |
| notmyname | creiht: ok. I'm hoping it won't take that long :-) | 18:59 |
| notmyname | we'll see | 18:59 |
| creiht | hehe | 18:59 |
| torgomatic | man, I always forget about the meeting until it's too late to go get lunch | 19:00 |
| creiht | heh | 19:00 |
| notmyname | me too | 19:01 |
| creiht | torgomatic: I usually forget until I'm halfway through lunch and realize that I'm missing it | 19:01 |
| torgomatic | creiht: I'm gonna try that next time :) | 19:02 |
| *** mkollaro has quit IRC | 19:03 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Only run flake8 on Swift code https://review.openstack.org/83177 | 19:10 |
| *** nacim has quit IRC | 19:22 | |
| notmyname | torgomatic: now lunch :-) | 19:24 |
| torgomatic | seriously :) | 19:24 |
| creiht | torgomatic: wait before you go... | 19:24 |
| torgomatic | yes? | 19:24 |
| peluse | clayg: see my note on the 409 patch? I was unable to repro what you mentioned using swift client. Maybe you can give it one more shot... since its not 2am now :) | 19:24 |
| creiht | torgomatic: enjoy lunch :) | 19:25 |
| notmyname | portante: let me know what I missed on the priority review page | 19:25 |
| torgomatic | :) | 19:25 |
| zaitcev | I was so happy about the extension that I hardly paid attention to meeting. So, we have patches to review, check. Priority list, check. | 19:25 |
| creiht | notmyname: you should call it the importante review page :) | 19:25 |
| notmyname | :-) | 19:25 |
| peluse | nice! | 19:25 |
| clayg | peluse: so this is the container recreate patch - not the container object put patch | 19:26 |
| notmyname | zaitcev: tldr is that we need an RC next week | 19:26 |
| zaitcev | I think I'll focus on in-process functests. They increase confidence in PBE | 19:26 |
| * peluse thinks he used the link provided in the email - will double check | 19:26 | |
| peluse | clayg: yeah, I used the right one. I meant 409 in the context of the error you were getting, not the patch title. sorry | 19:27 |
| notmyname | oh, also, the schedule for the conference part has been published at http://openstacksummitmay2014atlanta.sched.org | 19:27 |
| notmyname | actually a lot of swift-related talks this time | 19:27 |
| portante | notmyname: okay | 19:28 |
| peluse | clayg: tested both its parent, the patch itself and what depends on it. First fails, 2nd two pass as expected | 19:28 |
| clayg | peluse: wait - how long is the patch chain? But the thing I said was broken on feature/ec (a put w/o sp to an existing non-default storage policy returns 409) - you verified it's broken and fixed by the first patch in the chain? | 19:29 |
| peluse | clayg: correct | 19:29 |
| * clayg is pretty sure that should be the case and would like feature/ec to not have this behavior any longer | 19:29 | |
| clayg | k | 19:30 |
| peluse | clayg: yeah, I think we're ready to move on this:https://review.openstack.org/#/c/79731/ | 19:30 |
| peluse | clayg: well, that chain - you know what I mean :) | 19:30 |
| clayg | yeah i'ma click go on https://review.openstack.org/#/c/82590/ | 19:30 |
| clayg | thanks for the double check | 19:31 |
| peluse | that's the ticket... awesome | 19:31 |
| *** JuanManuelOlle has quit IRC | 19:31 | |
| clayg | peluse: so https://review.openstack.org/#/c/79731/ just may not be the way to go if the keep-per-object-spi ends up being the way to go | 19:32 |
| *** JuanManuelOlle has joined #openstack-swift | 19:32 | |
| peluse | clayg: is that "if" part of a reconciler discussion? | 19:33 |
| clayg | peluse: did you and yuan verify anything with the forever un-migrated legacy sp0 databases? I saw he did that patch for the auditor. | 19:33 |
| peluse | clayg: I haven't, no. I will do that today though and close w/you guys on trello | 19:34 |
| *** piyush1 has quit IRC | 19:34 | |
| * peluse will do so now actually before he forgets | 19:35 | |
| *** gvernik has quit IRC | 19:37 | |
| *** nacim has joined #openstack-swift | 19:46 | |
| openstackgerrit | A change was merged to openstack/python-swiftclient: Make bin/swift testable part 1 https://review.openstack.org/76487 | 19:47 |
| peluse | clayg: OK, you're right - some changes have been made there since I originally coded it and it doesn't work the way it used to so having the auditor do it is good | 19:51 |
| *** nacim has quit IRC | 19:52 | |
| *** jergerber has joined #openstack-swift | 19:59 | |
| *** shri has quit IRC | 20:11 | |
| *** saju_m has quit IRC | 20:11 | |
| clayg | peluse: sweet - thanks for verifying! | 20:20 |
| openstackgerrit | A change was merged to openstack/swift: don't repatch mimetools again. and again. again. https://review.openstack.org/83156 | 20:21 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: minor cleanups for swift-container-info https://review.openstack.org/83201 | 20:26 |
| openstackgerrit | A change was merged to openstack/swift: Bump up sleep when expecting a timeout https://review.openstack.org/82664 | 20:28 |
| *** saju_m has joined #openstack-swift | 20:29 | |
| *** IRTermite has joined #openstack-swift | 20:33 | |
| openstackgerrit | A change was merged to openstack/swift: Fix PUT to existing container w/o specific storage policy https://review.openstack.org/82590 | 20:34 |
| *** saju_m has quit IRC | 20:37 | |
| openstackgerrit | Peter Portante proposed a change to openstack/swift: Invert which proxy logging middleware instance logs https://review.openstack.org/82909 | 20:37 |
| portante | notmyname, dfg: unit tests for the proxy logging fix, 100% coverage | 20:37 |
| *** gordc has joined #openstack-swift | 20:41 | |
| dfg | portante: cool. i'll check it out. thanks for taking care of it | 20:41 |
| portante | welcome | 20:41 |
| *** shri has joined #openstack-swift | 20:47 | |
| *** gordc has left #openstack-swift | 20:48 | |
| openstackgerrit | A change was merged to openstack/swift: Add Storage Policy Support to Account HEAD https://review.openstack.org/73747 | 20:49 |
| *** piyush has joined #openstack-swift | 20:50 | |
| openstackgerrit | Florian Hines proposed a change to openstack/swift: Write out ring.gz's in a safer fashion https://review.openstack.org/83209 | 20:51 |
| openstackgerrit | Peter Portante proposed a change to openstack/swift: Invert which proxy logging middleware instance logs https://review.openstack.org/82909 | 20:52 |
| *** piyush has left #openstack-swift | 20:52 | |
| *** piyush has joined #openstack-swift | 20:52 | |
| portante | dfg: probably too verbose, but the commit message did not describe the problem well, or how it was fixed | 20:53 |
| *** lpabon has joined #openstack-swift | 20:55 | |
| openstackgerrit | Florian Hines proposed a change to openstack/swift: Write out ring.gz's in a safer fashion https://review.openstack.org/83209 | 20:55 |
| clayg | pepAAAAAAAAATEEEE! | 20:56 |
| *** JuanManuelOlle has quit IRC | 20:58 | |
| dfg | portante: it looks fine to me. so does the code- just need to load it up and run it. | 21:00 |
| *** tdasilva has left #openstack-swift | 21:04 | |
| pandemicsyn | clayg: so you have it in meme form for the future - http://img.ronin.io/pep8.jpg ;) | 21:04 |
| *** lpabon has quit IRC | 21:05 | |
| portante | dfg: great, thanks for taking a look | 21:06 |
| portante | anybody else want to get some eyes on the proxy logging change? | 21:07 |
| clayg | chmouel: http://img.ronin.io/pep8.jpg < from pandemicsyn for openstackreactions | 21:08 |
| openstackgerrit | A change was merged to openstack/swift: Update tox.ini so flake8 works even without hacking https://review.openstack.org/77072 | 21:11 |
| notmyname | portante: ya, I'm going to take a loot | 21:14 |
| notmyname | (but don't let that stop anyone else) | 21:14 |
| portante | hope it is good loot | 21:14 |
| notmyname | take a loot? | 21:14 |
| portante | and don't get caught | 21:14 |
| notmyname | heh | 21:14 |
| openstackgerrit | A change was merged to openstack/swift: Removed hard coded location of ring https://review.openstack.org/82742 | 21:17 |
| notmyname | portante: please take a look at http://docs.openstack.org/developer/swift/middleware.html#module-swift.common.middleware.proxy_logging | 21:18 |
| notmyname | portante: which comes from the docstring at the top of proxy_logging.py | 21:19 |
| dfg | portante: this is existing behavior (and not a huge deal) but there doesn't seems to be any bytes-transferred on the log line put for the manifest itself. http://paste.openstack.org/show/74399/ . shouldn't be fixed with this patch and has probably been that way the whole time | 21:24 |
| dfg | but prob should be fixed later on | 21:24 |
| dfg | other than that I'll +2 once notmyname is happy with the docs. | 21:25 |
| notmyname | dfg: ah. interesting. should be the actual bytes of the SLO manifest there | 21:27 |
| notmyname | (right?) | 21:27 |
| dfg | notmyname: i'd think so ya. | 21:28 |
| dfg | but- i'm not terribly worried about it | 21:28 |
| notmyname | agreed. also that it should be a later patch | 21:29 |
| *** piyush1 has joined #openstack-swift | 21:31 | |
| *** piyush2 has joined #openstack-swift | 21:32 | |
| *** piyush has quit IRC | 21:33 | |
| *** piyush1 has quit IRC | 21:35 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Fix broker.storage_policy_index after .set_storage_policy_index https://review.openstack.org/83228 | 21:43 |
| *** dencaval has quit IRC | 21:44 | |
| openstackgerrit | A change was merged to openstack/swift: Tests for storage policies in /info https://review.openstack.org/82906 | 21:45 |
| portante | notmyname, dfg: yes, the slo middleware passes along the initial GET or HEAD request, line 354 of slo.py, which is not marked as an internal request, and is using the same environment dictionary, so it won't get logged | 21:48 |
| *** pberis has quit IRC | 21:49 | |
| portante | in a separate commit, we can make that request as a proper subrequest, and then that will be logged | 21:50 |
| portante | however | 21:50 |
| portante | the logging does not know ahead of time if it is a real manifest, so this kind of change would mean two log entries for SLO objects that are not manifests | 21:51 |
| dfg | portante: i was talking about on the PUT of the slo | 21:54 |
| portante | I think it does the same thing, checking | 21:54 |
| portante | yes, it does | 21:55 |
| portante | if you look for self._app_call(req.environ), that is a place where the middleware "passes" along the external request, without touching it, so there is no way for the second middleware to know what to do with it | 21:56 |
| dfg | on PUT in doesn't do that does it? it builds a whole new body- right? | 21:57 |
| portante | dfg: hmm, it looks like it is just passing the original request along | 21:59 |
| portante | but maybe I am not reading the code right, though the behaviors you are reporting seem to support the way I am reading the code | 21:59 |
| notmyname | hmm...just ran it on my SAIO. seemed to work | 22:00 |
| portante | wait, so the PUT request is logged twice? or just once marked as SLO? or just once no markign? | 22:01 |
| portante | or are you talking abou the GET request | 22:01 |
| portante | ? | 22:01 |
| notmyname | PUT so far :-) | 22:02 |
| *** piyush2 has quit IRC | 22:02 | |
| portante | okay, so then I am not reading the code right, that is good, nothing to change! | 22:02 |
| portante | ;) | 22:02 |
| portante | notmyname: | 22:02 |
| notmyname | interestingly, the auth for the HEADs is logged, but not the actual HEAD subrequests | 22:02 |
| notmyname | https://gist.github.com/notmyname/25e4077230cb746f5664 | 22:03 |
| dfg | notmyname: grep -v 'swift_proxy: Use' | 22:04 |
| portante | not hitting the cache? | 22:04 |
| dfg | :) | 22:04 |
| notmyname | dfg: :-) | 22:04 |
| notmyname | dfg: but I've got 1490 bytes logged there | 22:05 |
| dfg | notmyname: ya | 22:05 |
| portante | I'm gonna have to head out in a few minutes, and will be back on line a bit later. | 22:05 |
| notmyname | portante: dfg: the GET to the SLO looks awesome https://gist.github.com/notmyname/363cd5291ecf4fe34a87 | 22:06 |
| notmyname | dfg: you were dong SLO and not DLO, right? | 22:06 |
| dfg | haha | 22:07 |
| dfg | notmyname: ya sorry- i was switching between them with swift client. | 22:08 |
| dfg | ok nm then | 22:08 |
| notmyname | yay! not changes needed | 22:09 |
| notmyname | portante: except to the docstring ;-) | 22:09 |
| portante | for the docstring? | 22:09 |
| portante | meaning, you want a doc string change to the proxy logging module? | 22:09 |
| notmyname | portante: yes. because that's what goes to swift.openstack.org (and also because I think it's wrong now) | 22:10 |
| portante | hmm | 22:10 |
| notmyname | portante: oh, also looks like jenkins went all http://img.ronin.io/pep8.jpg on it too | 22:11 |
| portante | hmm | 22:11 |
| portante | notmyname: not sure what is wrong with the doc string, can you point out what you are seeing? | 22:12 |
| notmyname | (see this is why I didn't just push over it ;-) | 22:12 |
| portante | it seems to me that it is now documenting the right behavior | 22:12 |
| notmyname | "the subrequest will be logged by the rightmost middleware (with a swift.source set) and the outgoing request (with body overridden) will be logged by leftmost middleware." | 22:12 |
| notmyname | or maybe I just have right and left mixed up (again) | 22:13 |
| portante | yes, that should be the case for staticweb | 22:13 |
| notmyname | so that is correct now? | 22:13 |
| notmyname | and was incorrect before this patch? | 22:13 |
| portante | if staticweb properly marks the subrequest separate from the client request | 22:13 |
| dfg | i think its correct now | 22:13 |
| portante | yes | 22:13 |
| notmyname | ok, good | 22:13 |
| portante | heading out, bbiab | 22:15 |
| notmyname | https://gist.github.com/notmyname/7b58811f097997c6a957 <-- easy pep8 tester | 22:17 |
| *** Midnightmyth has quit IRC | 22:17 | |
| notmyname | dfg: running last tests now, then I'm going to push the pep8 fix over it and +2 it | 22:21 |
| *** erlon has quit IRC | 22:22 | |
| *** dmsimard has quit IRC | 22:24 | |
| openstackgerrit | Florian Hines proposed a change to openstack/swift: Write out ring.gz's in a safer fashion https://review.openstack.org/83209 | 22:27 |
| *** jergerber has quit IRC | 22:28 | |
| notmyname | ...hmmm. that's weird | 22:30 |
| chmouel | clayg: ahaha | 22:34 |
| notmyname | dfg: portante: I get a functional test skipped with the logging patch. not on master. I'm looking for the particular test now | 22:35 |
| torgomatic | notmyname: try rebasing the logging patch on master? there was that one test that was getting skipped | 22:36 |
| notmyname | torgomatic: yup. it's clean | 22:37 |
| clayg | can we just update a container's created_at time when we recreated a container that was deleted - or should we just make the status and status_changed_at fields work? | 22:40 |
| torgomatic | clayg: seems like the created_at would be sufficient, but I don't know your use case | 22:41 |
| clayg | torgomatic: well I don't know the use case of the field to begin with - it gets set once when the db is initialized and doesn't change even if the container is deleted and then recreated | 22:42 |
| torgomatic | clayg: oh really? that's crap... it seems like it should be the container creation time, so if you PUT DELETE PUT, it should be at the time of the *second* PUT | 22:42 |
| clayg | put_timestamp OTOH gets updated everytime a PUT comes in | 22:42 |
| torgomatic | clayg: including PUT container object? | 22:43 |
| clayg | torgomatic: yeah well, it's not | 22:43 |
| clayg | so a) change that b) fix the other stupid/broken time tracking field to work like you might imagine it should that currently noone looks at cause it's basically broken | 22:43 |
| torgomatic | clayg: well, shoot... the container replicator is going to need something that behaves like what I described so it can work with storage policies | 22:44 |
| clayg | torgomatic: you and me both brother | 22:44 |
| clayg | gholt: creiht: do you have strong feelings on what the created_at field in the db's should mean? Do you have any strong feelings about the long term fate of status_changed_at and status? | 22:45 |
| openstackgerrit | John Dickinson proposed a change to openstack/swift: Invert which proxy logging middleware instance logs https://review.openstack.org/82909 | 22:46 |
| notmyname | dfg: ^ | 22:46 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Merge branch 'master' into feature/ec https://review.openstack.org/83234 | 22:50 |
| openstackgerrit | Clay Gerrard proposed a change to openstack/swift: Add object-reconciler daemon https://review.openstack.org/80730 | 22:50 |
| openstackgerrit | Greg Lange proposed a change to openstack/swift: Unify backend logging https://review.openstack.org/83237 | 23:02 |
| h6w | Morning! | 23:02 |
| clayg | damn you fixed length replication args!!! | 23:02 |
| clayg | new rule - when you write a wire protocol that accepts a fixed width number of arguments - trim the width of the trasmitted arguments to match the length you expect so you can ignore future updates | 23:03 |
| clayg | ^ that goes for you too future me! | 23:06 |
| h6w | When I have two co-lo proxies, should the proxies be using each other to talk to the local storage nodes, or be going direct to the storage nodes? (i.e. Client->Local Proxy->VPN->Remote Proxy->Remote SN or Client->Local Proxy->VPN->Remote SN) | 23:07 |
| torgomatic | h6w: proxies only talk to storage nodes, not other proxies | 23:08 |
| * clayg goes to write middleware that talks to other proxies | 23:09 | |
| *** Trozz has joined #openstack-swift | 23:10 | |
| h6w | torgomatic: Then how does fallback work with global clusters? E.g. 404 = call the other proxy | 23:10 |
| torgomatic | h6w: load balancer --> proxies --> storage nodes | 23:10 |
| h6w | Ah. The 404 is at the storage level? | 23:11 |
| torgomatic | yup | 23:11 |
| h6w | So the affinity is only local. | 23:11 |
| torgomatic | ...maybe? | 23:11 |
| torgomatic | affinity stuff affects the order in which the proxy picks storage nodes to talk to | 23:12 |
| h6w | This may be a complex question, but how do the proxies then resolve a conflicting balancing situation? | 23:12 |
| torgomatic | I don't understand what you mean by 'conflicting balancing situation' | 23:13 |
| *** mlipchuk has left #openstack-swift | 23:13 | |
| clayg | torgomatic: you know the situation where your balances are in conflict! | 23:14 |
| h6w | Well, lets say that, for the same zone and ring, two proxies want to balance data differently across the local and remote nodes. Both proxies may issue conflicting storage instructions, no? | 23:14 |
| torgomatic | h6w: you mean the proxies are using write affinity, so they'll prefer to route object PUTs to different storage nodes? | 23:15 |
| h6w | Yes, but maybe I'm thinking storage nodes are smarter than they are. | 23:17 |
| torgomatic | yeah, the object server processes don't really talk to each other; replication and such will communicate between storage nodes, but the HTTP stuff is just proxy -> storage nodes -> disks in a tree | 23:20 |
| h6w | Ok. After a bit of a chat here, my question has to do with the situation where there's a Venn-diagram type storage node allocation. So the proxies have some nodes they share, but they each have individual copies somewhere. | 23:37 |
| h6w | If something gets written via a local proxy to the uniquely-available storage node, how does the other proxy know that the data has changed? | 23:38 |
| h6w | uniquely-available = only locally available | 23:38 |
| torgomatic | short answer: it doesn't | 23:38 |
| h6w | It could first be replicated to the other local storage nodes that *are* shared, and then the second proxy would know. | 23:39 |
| torgomatic | you have to wait for object replication to move the object | 23:39 |
| h6w | Ok, so if the system is set up particularly badly, there could result some situations where the replication is delayed to the point where a write conflict may occur? | 23:40 |
| *** Trozz has quit IRC | 23:40 | |
| torgomatic | sure, in which case the last write wins, just like always | 23:40 |
| h6w | How's that established? If the transaction IDs are unique to each proxy, then is it just the timestamp? So we have to have really accurate clock synchronisation? | 23:42 |
| torgomatic | yes, you need your clocks synced up | 23:43 |
| torgomatic | any Swift cluster needs its clocks synced up | 23:44 |
| *** _sEBAs_ has joined #openstack-swift | 23:53 | |
| h6w | True, but in a delayed write situation, where the local proxy reads an object before the remote proxy has been able to transmit it, does that mean we'll end up with the version before the write? | 23:55 |
| h6w | Since it's HTTP, I'm guessing that there's no "oh but it's changed" communication. | 23:56 |
| torgomatic | h6w: yep, that's eventual consistency for you | 23:58 |
| torgomatic | same thing goes if you start reading version N of an object while you're uploading N+1 | 23:59 |
| torgomatic | or sometimes even after you're done uploading N+1 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
