| *** dmorita has joined #openstack-swift | 00:24 | |
| *** DisneyRicky has joined #openstack-swift | 00:25 | |
| *** DisneyRicky has quit IRC | 00:50 | |
| *** DisneyRicky has joined #openstack-swift | 01:08 | |
| *** DisneyRicky has quit IRC | 01:25 | |
| *** charz_ is now known as charz | 01:36 | |
| *** nosnos has joined #openstack-swift | 01:37 | |
| *** miqui_ has quit IRC | 02:11 | |
| *** robsparker has quit IRC | 02:28 | |
| *** bkopilov has quit IRC | 02:40 | |
| *** robsparker has joined #openstack-swift | 02:41 | |
| *** ho has joined #openstack-swift | 02:50 | |
| *** mrsnivvel has joined #openstack-swift | 03:07 | |
| *** aswadr has joined #openstack-swift | 03:09 | |
| *** nosnos has quit IRC | 03:25 | |
| *** Andrew___ has joined #openstack-swift | 03:39 | |
| *** zhiyan_ is now known as zhiyan | 03:40 | |
| Andrew___ | hello~ | 03:41 |
|---|---|---|
| Andrew___ | I think there is something wrong with the name and description of funtion weight_of_one_part in RingBuilder class. Is there someone who think like me? | 03:45 |
| Andrew___ | I think that it is not weight of one part but part of one weight. right? | 03:47 |
| *** bkopilov has joined #openstack-swift | 03:59 | |
| *** nosnos has joined #openstack-swift | 04:02 | |
| *** ppai has joined #openstack-swift | 04:08 | |
| *** zhiyan is now known as zhiyan_ | 04:20 | |
| mattoliverau | Andrew___: Its attempting to get the wieght of a partition based of the total weights of all the devices. So I think its named correctly. The part isn't 'part' its a swift partition, an intergral part of the swfit modified consistant hashing ring. See http://docs.openstack.org/developer/swift/overview_ring.html | 04:20 |
| *** elambert has joined #openstack-swift | 04:51 | |
| *** ajc_ has joined #openstack-swift | 04:52 | |
| *** Andrew___ has quit IRC | 05:16 | |
| *** psharma has joined #openstack-swift | 05:26 | |
| *** anand_ts has joined #openstack-swift | 05:33 | |
| *** chandan_kumar has joined #openstack-swift | 05:34 | |
| *** Andrew has joined #openstack-swift | 06:33 | |
| openstackgerrit | Matthew Oliver proposed a change to openstack/swift: Swift configuration parameter audit https://review.openstack.org/104760 | 07:03 |
| *** haomaiwa_ has joined #openstack-swift | 07:05 | |
| *** chandan_kumar is now known as chandankumar | 07:23 | |
| *** mkerrin has quit IRC | 07:41 | |
| *** mitz_ has quit IRC | 07:41 | |
| *** nacim has joined #openstack-swift | 07:42 | |
| *** mkerrin has joined #openstack-swift | 07:45 | |
| *** jyoti-ranjan has joined #openstack-swift | 07:49 | |
| *** mmcardle has joined #openstack-swift | 08:14 | |
| *** mariusv has quit IRC | 08:17 | |
| *** mariusv has joined #openstack-swift | 08:19 | |
| *** mitz_ has joined #openstack-swift | 08:23 | |
| *** mmcardle has quit IRC | 08:26 | |
| *** zhiyan_ is now known as zhiyan | 08:30 | |
| *** andyandy has joined #openstack-swift | 08:34 | |
| *** Andrew has quit IRC | 09:07 | |
| *** markd has quit IRC | 09:17 | |
| *** DisneyRicky has joined #openstack-swift | 09:22 | |
| *** DisneyRicky has quit IRC | 09:37 | |
| *** andrew has joined #openstack-swift | 09:51 | |
| *** bvandenh has joined #openstack-swift | 10:02 | |
| *** DisneyRicky has joined #openstack-swift | 10:34 | |
| *** haomaiwa_ has quit IRC | 10:38 | |
| *** haomaiwa_ has joined #openstack-swift | 10:38 | |
| anand_ts | anyone have idea on how to give Access control list to users using Swift service | 10:46 |
| anand_ts | users are having admin access when I give admin or Swift operator access, As a normal user I need to give only read only access to the containers. ie. view and download objects, not to upload | 10:47 |
| anand_ts | which file I need to edit? | 10:47 |
| *** DisneyRicky has quit IRC | 10:47 | |
| *** haomaiwa_ has quit IRC | 10:57 | |
| *** haomai___ has joined #openstack-swift | 10:57 | |
| *** DisneyRicky has joined #openstack-swift | 10:58 | |
| *** ppai has quit IRC | 11:00 | |
| *** andrew has quit IRC | 11:10 | |
| *** ppai has joined #openstack-swift | 11:13 | |
| *** DisneyRicky has quit IRC | 11:17 | |
| *** ho has quit IRC | 11:31 | |
| *** mmcardle has joined #openstack-swift | 11:32 | |
| *** DisneyRicky has joined #openstack-swift | 11:45 | |
| *** dmorita has quit IRC | 11:46 | |
| *** DisneyRicky has quit IRC | 12:01 | |
| *** Anticimex has quit IRC | 12:17 | |
| *** Anticimex has joined #openstack-swift | 12:24 | |
| *** pandemicsyn has quit IRC | 12:28 | |
| *** zz_wasmum has quit IRC | 12:28 | |
| *** serverascode has quit IRC | 12:28 | |
| *** Alex_Gaynor has quit IRC | 12:28 | |
| *** swills has quit IRC | 12:29 | |
| *** zz_wasmum has joined #openstack-swift | 12:29 | |
| *** nottrobin has quit IRC | 12:29 | |
| *** rpedde has quit IRC | 12:29 | |
| *** dfg_ has joined #openstack-swift | 12:30 | |
| *** rpedde has joined #openstack-swift | 12:30 | |
| *** swills has joined #openstack-swift | 12:30 | |
| *** gholt has quit IRC | 12:30 | |
| *** zacksh has quit IRC | 12:30 | |
| *** zacksh has joined #openstack-swift | 12:30 | |
| *** swills has quit IRC | 12:30 | |
| *** swills has joined #openstack-swift | 12:30 | |
| *** nottrobin_ has joined #openstack-swift | 12:30 | |
| *** joearnold has quit IRC | 12:30 | |
| *** hugokuo has quit IRC | 12:30 | |
| *** cschwede has quit IRC | 12:30 | |
| *** dfg has quit IRC | 12:31 | |
| *** Alex_Gaynor_ has joined #openstack-swift | 12:31 | |
| *** nottrobin_ has quit IRC | 12:31 | |
| *** nottrobin_ has joined #openstack-swift | 12:31 | |
| *** Alex_Gaynor_ has quit IRC | 12:31 | |
| *** Alex_Gaynor_ has joined #openstack-swift | 12:31 | |
| *** pandemicsyn has joined #openstack-swift | 12:31 | |
| *** serverascode has joined #openstack-swift | 12:31 | |
| *** nottrobin_ is now known as nottrobin | 12:32 | |
| *** cschwede has joined #openstack-swift | 12:32 | |
| *** joearnold has joined #openstack-swift | 12:32 | |
| *** ajc_ has quit IRC | 12:33 | |
| *** gholt has joined #openstack-swift | 12:33 | |
| *** ChanServ sets mode: +v gholt | 12:33 | |
| *** hugokuo has joined #openstack-swift | 12:33 | |
| *** mmcardle has quit IRC | 12:46 | |
| *** bkopilov has quit IRC | 12:53 | |
| *** nosnos has quit IRC | 13:02 | |
| *** chandan_kumar has joined #openstack-swift | 13:10 | |
| *** mmcardle has joined #openstack-swift | 13:12 | |
| *** chandankumar has quit IRC | 13:15 | |
| *** ppai has quit IRC | 13:15 | |
| *** chandan_kumar is now known as chandankumar | 13:15 | |
| *** miqui has joined #openstack-swift | 13:17 | |
| *** zhiyan is now known as zhiyan_ | 13:37 | |
| *** psharma has quit IRC | 13:44 | |
| *** sandywalsh has joined #openstack-swift | 13:47 | |
| *** tdasilva has joined #openstack-swift | 13:48 | |
| *** anand_ts has quit IRC | 13:55 | |
| *** swat30 has quit IRC | 14:01 | |
| *** thurloat has quit IRC | 14:01 | |
| *** lpabon has joined #openstack-swift | 14:06 | |
| *** swat30 has joined #openstack-swift | 14:07 | |
| *** tellesnobrega has left #openstack-swift | 14:10 | |
| *** thurloat has joined #openstack-swift | 14:13 | |
| *** bkopilov has joined #openstack-swift | 14:16 | |
| *** annegent_ has joined #openstack-swift | 14:20 | |
| *** tsg has joined #openstack-swift | 14:36 | |
| *** mmcardle has quit IRC | 14:45 | |
| *** pberis has joined #openstack-swift | 14:46 | |
| *** chandankumar has quit IRC | 14:51 | |
| *** kevinc_ has joined #openstack-swift | 14:54 | |
| *** elambert has quit IRC | 14:55 | |
| *** physcx has joined #openstack-swift | 14:56 | |
| *** sandywalsh has quit IRC | 15:01 | |
| *** annegent_ has quit IRC | 15:02 | |
| *** annegent_ has joined #openstack-swift | 15:02 | |
| *** sandywalsh has joined #openstack-swift | 15:03 | |
| *** sandywalsh_ has joined #openstack-swift | 15:05 | |
| *** chandan_kumar has joined #openstack-swift | 15:06 | |
| *** sandywalsh has quit IRC | 15:08 | |
| *** kevinc_ has quit IRC | 15:34 | |
| *** kevinc_ has joined #openstack-swift | 15:39 | |
| *** zaitcev has joined #openstack-swift | 15:40 | |
| *** ChanServ sets mode: +v zaitcev | 15:40 | |
| *** aswadr has quit IRC | 15:42 | |
| *** sandywalsh_ has quit IRC | 15:57 | |
| *** sandywalsh has joined #openstack-swift | 15:59 | |
| *** annegent_ has quit IRC | 16:07 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/swift: Merge tag '2.0.0' https://review.openstack.org/105222 | 16:08 |
| *** kevinc_ has quit IRC | 16:08 | |
| *** mwstorer has joined #openstack-swift | 16:19 | |
| openstackgerrit | Donagh McCabe proposed a change to openstack/swift-specs: Handling X-Service-Token in keystoneauth middleware https://review.openstack.org/105228 | 16:27 |
| *** elambert has joined #openstack-swift | 16:30 | |
| notmyname | FYI, Swift 2.0 final is out. http://tarballs.openstack.org/swift/swift-2.0.0.tar.gz there will be some press releases and blog posts tomorrow (thanks cschwede!) | 16:34 |
| notmyname | please don't make a big deal about it today. make a big deal about it tomorrow | 16:34 |
| clayg | notmyname: can i just not make a big deal about it at all? | 16:35 |
| *** cpen has joined #openstack-swift | 16:35 | |
| physcx | is there a 2.0 feature list or change log? | 16:36 |
| notmyname | physcx: official changelog at https://github.com/openstack/swift/blob/master/CHANGELOG | 16:39 |
| *** kevinc_ has joined #openstack-swift | 16:40 | |
| physcx | clayg: got a minute to talk about Buffer DiskfileWriter writes? | 16:46 |
| *** shri has joined #openstack-swift | 16:49 | |
| *** ChanServ changes topic to "Swift 2.0 https://launchpad.net/swift/juno/2.0.0 | Swift Review Dashboard: http://bit.ly/1iVBigF" | 16:52 | |
| *** annegent_ has joined #openstack-swift | 16:53 | |
| *** mlipchuk has joined #openstack-swift | 16:57 | |
| *** mlipchuk has left #openstack-swift | 16:57 | |
| *** erlon has joined #openstack-swift | 16:58 | |
| *** jyoti-ranjan has quit IRC | 17:16 | |
| *** Alex_Gaynor_ is now known as Alex_Gaynor | 17:37 | |
| *** Alex_Gaynor has quit IRC | 17:37 | |
| *** Alex_Gaynor has joined #openstack-swift | 17:37 | |
| *** gyee has joined #openstack-swift | 17:41 | |
| *** miqui has quit IRC | 17:45 | |
| *** andyandy has quit IRC | 17:48 | |
| *** miqui has joined #openstack-swift | 17:49 | |
| *** Tyger has joined #openstack-swift | 17:51 | |
| *** kevinc_ has quit IRC | 17:52 | |
| *** infotection has quit IRC | 18:10 | |
| *** infotection has joined #openstack-swift | 18:16 | |
| notmyname | idea for someone looking for something to do: in python-swiftclient, replace the dependency on keystoneclient with the rest api calls. that way we can support auth v2+ out of the box and also not get all the transitive dependencies of keystoneclient | 18:31 |
| *** miqui has quit IRC | 18:32 | |
| *** foexle has joined #openstack-swift | 18:32 | |
| *** CaioBrentano has joined #openstack-swift | 18:40 | |
| *** miqui has joined #openstack-swift | 18:41 | |
| *** occup4nt is now known as occupant | 18:42 | |
| *** gyee has quit IRC | 18:42 | |
| *** thurloat has quit IRC | 18:45 | |
| *** swat30 has quit IRC | 18:46 | |
| clayg | physcx: sure | 18:46 |
| anticw | notmyname: +1 | 18:54 |
| *** foexle has quit IRC | 19:02 | |
| *** kevinc_ has joined #openstack-swift | 19:06 | |
| *** chandan_kumar has quit IRC | 19:06 | |
| *** gyee has joined #openstack-swift | 19:06 | |
| notmyname | anticw: any chance you could put a dev on it? ;-) | 19:06 |
| *** annegent_ has quit IRC | 19:07 | |
| *** annegent_ has joined #openstack-swift | 19:08 | |
| *** annegent_ has quit IRC | 19:13 | |
| anticw | notmyname: actually, i do have a list of things and this was already on it ... timing is always tricky | 19:22 |
| *** kevinc_ has quit IRC | 19:22 | |
| anticw | notmyname: multiple-endpoints is higher on the list | 19:22 |
| notmyname | great | 19:22 |
| notmyname | what's multiple endpoints? (I mean, I know what it is. I just want to make sure you do ;-) ) | 19:22 |
| anticw | zaitcev: btw, this reminds me ... swiftclient 1.13.1 in RDO is broken ... i forget how but i have an image of what i hacked up | 19:23 |
| anticw | notmyname: v2+ auth responses can specifiy multiple storage URLs ... which last i checked swiftclient doesn't deal with | 19:23 |
| notmyname | anticw: ah, cool | 19:24 |
| anticw | actually, few things deal with this well ... some things use first, some last ... some just get confused and cry | 19:24 |
| zaitcev | anticw: so... needs 2.0 something? | 19:24 |
| anticw | zaitcev: i recall now .... there code adds content-length headers in some cases | 19:24 |
| anticw | and because it no longer uses python-request or something ... they are added by the lower layers as well | 19:24 |
| anticw | so you get two of these ... swift is fine with thatm nginx gives a 400 | 19:25 |
| anticw | YES I KNOW NGINX IS BALLS :-) | 19:25 |
| zaitcev | interesting | 19:25 |
| zaitcev | Well, I still use Pound, so I didn't notice. | 19:25 |
| anticw | using --os-storage-url=http://something/with/netcat/listening i captured headers and see this | 19:26 |
| anticw | it's probably upstream swiftclient as well in all fairness, not just rdo | 19:26 |
| zaitcev | version_string = str(pbr.version.VersionInfo('python-swiftclient')) | 19:27 |
| zaitcev | I hate all this automation | 19:28 |
| * anticw hates pbr | 19:28 | |
| *** mkerrin1 has joined #openstack-swift | 19:28 | |
| zaitcev | >>> swiftclient.version.version_string | 19:28 |
| zaitcev | '2.0.2' | 19:28 |
| anticw | zaitcev: ok, upstream from github has the issue there ... so if you want i'll just put up a patch for you to eyeball | 19:29 |
| zaitcev | okay. We probably want to upgrade to 2.0.2 in RDO, at least in Icehouse | 19:29 |
| *** mkerrin has quit IRC | 19:29 | |
| zaitcev | wait a moment, http://koji.fedoraproject.org/koji/buildinfo?buildID=499626 | 19:29 |
| zaitcev | But it's not pushed to RDO? | 19:29 |
| anticw | zaitcev: oh, one more things .... swift-proxy has a dependancy on swift3 ... it should be optional | 19:30 |
| anticw | swift3 in rdo is old and cruft (ie. breaks) | 19:30 |
| anticw | s/cruft/crufty/ | 19:30 |
| zaitcev | anticw: I can't help it. It's the same idiotic policy that add dependency on python-keystoneclient. You may file a bug if you want. | 19:31 |
| anticw | zaitcev: but it's NOT a requirement ... and it means replacing just swift3 is painful | 19:31 |
| anticw | in the end i just rm -r the crap underneath rpm | 19:31 |
| zaitcev | anticw: I have an idea: /join #rdo, or /quote jruzicka | 19:33 |
| zaitcev | Bet he'll ask you to file a bug, because we're responsible for working on bugs | 19:33 |
| anticw | will do | 19:33 |
| *** pberis has quit IRC | 19:34 | |
| *** pberis has joined #openstack-swift | 19:34 | |
| zaitcev | BTW, in addition to Swift, they asked me to work on Manila. Actually, "focus on Manila". Because Swift has too few bugs, so I have nothing to do. | 19:35 |
| creiht | lol | 19:35 |
| notmyname | wow | 19:35 |
| anticw | notmyname: maybe too late, but for 2.0.0 can we take all the middleware mangling magic out? | 19:35 |
| anticw | and require it be explicit? | 19:35 |
| notmyname | anticw: nope. too late. that would have been better to address about 3 weeks ago | 19:36 |
| *** annegent_ has joined #openstack-swift | 19:36 | |
| anticw | can we have 3.0.0 next month then? :) | 19:36 |
| notmyname | of course. the good things about release numbers is there is always a bigger number :-) | 19:36 |
| *** annegent_ has quit IRC | 19:41 | |
| *** bvandenh has quit IRC | 19:41 | |
| anticw | ℵ_0 | 19:46 |
| zaitcev | or 2.0.0.pl1 | 19:49 |
| *** physcx has quit IRC | 19:52 | |
| *** CaioBrentano has quit IRC | 19:55 | |
| *** CaioBrentano has joined #openstack-swift | 19:57 | |
| *** kevinc_ has joined #openstack-swift | 20:00 | |
| *** sandywalsh has quit IRC | 20:03 | |
| *** sandywalsh has joined #openstack-swift | 20:06 | |
| *** tdasilva has quit IRC | 20:11 | |
| *** peluse_ has quit IRC | 20:19 | |
| *** peluse_ has joined #openstack-swift | 20:19 | |
| *** infotection has quit IRC | 20:25 | |
| torgomatic | someone in here earlier was talking about the chunked-upload buffering patch; who was that? | 20:26 |
| torgomatic | (I saw it on my phone, and that IRC client doesn't keep logs. Sorry.) | 20:26 |
| *** mwstorer has quit IRC | 20:30 | |
| *** infotection has joined #openstack-swift | 20:33 | |
| *** mwstorer has joined #openstack-swift | 20:36 | |
| *** erlon has quit IRC | 20:39 | |
| *** foexle has joined #openstack-swift | 20:40 | |
| *** infotection has quit IRC | 20:41 | |
| anticw | torgomatic: kr4zy (sp?) was asking about it in the context of apache mod_wsgi | 20:47 |
| torgomatic | anticw: thanks; I'll keep any eye out and see if that user shows up again | 20:48 |
| *** infotection has joined #openstack-swift | 20:48 | |
| *** miqui has quit IRC | 20:49 | |
| *** kevinc_ has quit IRC | 20:57 | |
| *** kevinc_ has joined #openstack-swift | 21:08 | |
| TaiSHi | Hello everyone | 21:10 |
| *** tsg has quit IRC | 21:11 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Zero-copy object-server GET responses with splice() https://review.openstack.org/102609 | 21:13 |
| notmyname | TaiSHi: hello | 21:17 |
| TaiSHi | Hey | 21:18 |
| TaiSHi | I'm planning on using a series of webservers that need to host same data (roughly 100k files) which is updated dynamically (user content) | 21:18 |
| TaiSHi | I've heard that swift manages metadata pretty well, which wouldn't screw up when listing folders with a big amount of files | 21:19 |
| TaiSHi | And to add to the matter, aside from the 'fixed' webservers, new ones will be fired up on-demand and would need to access the same data | 21:19 |
| *** foexle has quit IRC | 21:31 | |
| notmyname | TaiSHi: sounds similar to what swift can do | 21:34 |
| notmyname | TaiSHi: but the first thign to understand (and I want to make clear) is that swift isn't a shared filesystem | 21:34 |
| TaiSHi | notmyname: yeah, you told me, which is why I've been around | 21:35 |
| notmyname | :-) | 21:35 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Zero-copy object-server PUT requests with splice() https://review.openstack.org/104705 | 21:35 |
| notmyname | TaiSHi: ok, so that being covered, think of swift as a pool of storage. and if you spin up new webservers, they can access the pool as well | 21:35 |
| notmyname | no probems | 21:35 |
| TaiSHi | Hmm, regarding the shared fs, I (tried to) use gluster. I don't know much about network filesystems other than ceph used as a block storage | 21:36 |
| notmyname | TaiSHi: have you used systems like AWS S3 or rackspace cloud files? | 21:37 |
| TaiSHi | None deeply. It has escaped my needs so far | 21:38 |
| notmyname | that's ok. just looking for a common frame of reference | 21:39 |
| TaiSHi | It's not mounting the same NFS over multiple nodes perhaps ? | 21:39 |
| notmyname | no, not at all | 21:39 |
| TaiSHi | Great (me and NFS have had our differences in the past) | 21:39 |
| notmyname | a swift cluster has an API enpoint. that endpoint is one or more "proxy servers", and those proxy servers talk to "storage nodes". the storage nodes have drives and read/write data. (in a high-level, close-enough sort of way) | 21:40 |
| TaiSHi | So all the traffic would have to go through the proxy, not directly to the nodes, right ? | 21:41 |
| notmyname | correct. but there can be an arbitrary number of proxies. no shared state, horizontally scalable | 21:42 |
| TaiSHi | What bugs me is that the data will probably be hosted at the web server nodes themselves initially. I could propose a centralized fs (which would be GREAT) | 21:45 |
| notmyname | well, think of a centralized object storage system (rather than an FS) | 21:46 |
| notmyname | ie all the webservers talking to the same object storage pool | 21:46 |
| notmyname | or better yet, the webservers simply serving links for clients to directly access the storage pool (eg that's what wikipedia does for all their images. everything is stored in their swift cluster) | 21:47 |
| TaiSHi | Sorry, I have to start incorporating the terms object/block storage | 21:47 |
| TaiSHi | Given the fact that swift handles metadata really well, I do like the solution but as I said, initially webservers would also be storage nodes | 21:48 |
| TaiSHi | I know it's not ideal, but well, the customer is still unsure about this "cloud migration" (they currently use 1 server that handles ~700mB/s traffic) | 21:48 |
| notmyname | what do you mean by "swift handles metadata really well"? what is the metadata you're referring to? | 21:48 |
| TaiSHi | I might be using some internet adopted terms :P, my previous experience with an object storage (gluster) resulted in almost exploding my nodes due to a big amount of files (~100k) | 21:50 |
| notmyname | 100k files is not a big number | 21:50 |
| notmyname | (from a swift perspective) | 21:50 |
| TaiSHi | Figured as mercadolibre and wikipedia use it | 21:53 |
| notmyname | ok, so you've got a bunch of files created by your users that you need to have available on any of your many webservers, right? | 21:55 |
| TaiSHi | Exactly | 21:55 |
| notmyname | so the way you do that with swift is to have your webservers serving your app (with links to the swift cluster) and a separate set of machines for your swift cluster. | 21:56 |
| notmyname | size each for their specific needs (webservers for the app traffic and swift for the network transfer and storage) | 21:57 |
| *** mkerrin1 has quit IRC | 21:57 | |
| notmyname | your users can upload and download directly to the swift cluster without needing to go through the webservers | 21:57 |
| *** mkerrin has joined #openstack-swift | 21:57 | |
| TaiSHi | But they upload their images through the web app, so they'd still need to go through the webservers | 21:58 |
| TaiSHi | (I might be missing something here) | 21:58 |
| notmyname | well, they wouldn't have to | 21:58 |
| notmyname | eg if you can get your users to send a PUT or POST request to a swift URL, then you can bypass the webservers | 21:59 |
| notmyname | but that could be something you do later | 21:59 |
| TaiSHi | Yeah I could talk to the dev, he's an open guy when it comes to techs | 21:59 |
| notmyname | for now, maybe the simplest thing is to use your webservers to pass all traffic between the user and swift | 21:59 |
| notmyname | TaiSHi: have you read up on any of the swift architecture docs? | 22:00 |
| TaiSHi | I found some docs but they're for rc2.x and packages aren't as up to date on my distro | 22:00 |
| notmyname | TaiSHi: check out https://swiftstack.com/openstack-swift/architecture/ | 22:01 |
| TaiSHi | Actually "some docs" are the official openstack | 22:01 |
| TaiSHi | Ok, let me do some reading | 22:01 |
| TaiSHi | Thanks | 22:01 |
| notmyname | TaiSHi: actually, the parent page there https://swiftstack.com/openstack-swift/ has a lot of links to blog posts and some videos too | 22:01 |
| TaiSHi | Ok, I'm going to do some reading | 22:02 |
| TaiSHi | Before I go, I have a couple questions | 22:02 |
| notmyname | ok | 22:03 |
| TaiSHi | I know that ideally I should use separate servers according to the needs, but would it be much of a hassle to use the webservers as nodes for now ? | 22:03 |
| notmyname | depends on what sort of resource overhead you have on the webservers. ie cpu, ram, disk | 22:04 |
| TaiSHi | They're digitalocean's VMs, the disk seems to perform fairly well | 22:06 |
| notmyname | hmm... | 22:07 |
| notmyname | so in general, I don't recommend running swift in a virtualized environment. you don't want eg digital ocean to suddenly rehome your VM and either cause a lot of data movement or, worse, place 2 of your VMs on the same host, thus raising your exposure to a single hardware failure | 22:08 |
| TaiSHi | I understand what you mean, sadly it is what I have at my disposal with this customer | 22:09 |
| TaiSHi | (I just recalled / saw the topic that 2.0 went live on ubuntu) | 22:10 |
| TaiSHi | swift looks great but perhaps I'm trying to overkill | 22:14 |
| notmyname | TaiSHi: would it help if you could install it on a VM locally and play with using the APi? | 22:15 |
| TaiSHi | notmyname: I'm actually free to install a lot of VMs on DO directly, I tend to test stuff that way | 22:15 |
| TaiSHi | For now and given that the first site I'm migrating is rather small, I wont implement swift, but I'll read the architecture documentation and see if it's doable | 22:16 |
| notmyname | TaiSHi: https://github.com/swiftstack/vagrant-swift-all-in-one uses vagrant and virtual box. if you've got those, it's a really fast and simple way to get going | 22:16 |
| TaiSHi | Oh, that is simpler lol | 22:16 |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Zero-copy object-server PUT requests with splice() https://review.openstack.org/104705 | 22:16 |
| TaiSHi | My job's computer is crappy, I'll probably do it at home | 22:16 |
| TaiSHi | I need to understand how to use the API and how to provide objects to the webservers | 22:17 |
| notmyname | torgomatic: my aren't you pushing a lot of splice patches today | 22:17 |
| TaiSHi | notmyname: thank you for your patience, seriously | 22:17 |
| torgomatic | notmyname: now if only I can get one of them right ;) | 22:17 |
| notmyname | TaiSHi: sure. happy to help | 22:17 |
| TaiSHi | I am a completely newb on this, glad you pointed out a couple stuff | 22:18 |
| notmyname | torgomatic: as long as tests pass, right? | 22:18 |
| torgomatic | notmyname: yeah, that was actually the problem... | 22:18 |
| torgomatic | old kernel on the py2.6 Jenkins boxes; stuff broke | 22:18 |
| bgmccollum | is there a correlation to which container-server updates which account-server when a container is created? like, is it 1:1, 2:2, 3:3 from get-nodes of both, or shuffle x:x, y:y, z:z? | 22:19 |
| *** CaioBrentano has quit IRC | 22:19 | |
| notmyname | bgmccollum: yes* | 22:19 |
| bgmccollum | i assume the * is for failure modes | 22:20 |
| notmyname | bgmccollum: * assuming that you have the same number of account and container replicas, it's basically [(x,y) for x in account_nodes for y in container_nodes] | 22:20 |
| notmyname | err...s/account/object/ | 22:21 |
| bgmccollum | so if i were piecing log messages together, to generate a graph...id have to look at the order from get-nodes | 22:21 |
| TaiSHi | notmyname: when you mentioned arbitrary number of proxy servers, that means that I could make each and one of the nodes a proxy as well and make them access themselves ? | 22:21 |
| TaiSHi | (that sounded fishy) | 22:21 |
| notmyname | ie remember that each partition has replica_count nodes listed in the ring. although order doesn't matter, the first object replica goes to the first container replica, etc | 22:22 |
| notmyname | TaiSHi: ya. you can run all the swift services on the same box (proxy, account, container, and object) | 22:22 |
| mattoliverau | Morning all | 22:22 |
| notmyname | bgmccollum: are you building a graph from log messages? cause that would be cool | 22:23 |
| notmyname | bgmccollum: like https://github.com/notmyname/logflow maybe? | 22:23 |
| TaiSHi | notmyname: great that would solve the single-node failure and actually it would improve performance a bit | 22:23 |
| bgmccollum | it would be cool ;) -- no, more of a representation of steps when showing various commands | 22:23 |
| notmyname | TaiSHi: ya, swift is designed to automatically work around hardware failure. | 22:23 |
| bgmccollum | notmyname: well hot damn | 22:24 |
| TaiSHi | notmyname: great, going to study (exam in a week) and keep reading on swift | 22:24 |
| notmyname | bgmccollum: it's not perfect (I can't yet tie the message to a particular swift process), but it works really well as a logical diagram | 22:25 |
| bgmccollum | notmyname: its def interesting... | 22:25 |
| Tyger | notmyname: I notice you have that map obj-server to object-server for the referrer - so there's one place that looks at the referrer | 22:26 |
| notmyname | bgmccollum: eg compare http://d.not.mn/log_flow.png to http://d.not.mn/log_flow2.png | 22:27 |
| notmyname | bgmccollum: what I want is almost like the first one. I don't have enough info in the logs to make it jsut right though | 22:27 |
| notmyname | Tyger: in that logflow repo? ya. but I don't care if that breaks ;-) | 22:28 |
| Tyger | notmyname: It looks like it actually wouldn't break, and the output wouldn't even change at all. | 22:28 |
| notmyname | Tyger: the concern would be if someone else has done something similar. sorry i haven't had a chance to look yet. overall I think the patch is probably just fine :-) | 22:28 |
| Tyger | notmyname: Right. And I'm not in a rush to get it submitted, it's not like it's blocking anything. | 22:29 |
| notmyname | bgmccollum: but I'm imagining a day where I can take a swift log file (a big one) and then map out all the different interactions. I think that would be very cool | 22:29 |
| bgmccollum | notmyname: yeah, that would be awesome | 22:30 |
| notmyname | bgmccollum: it would be pretty cool for after-the-fact usage analysis. | 22:30 |
| notmyname | bgmccollum: I /think/ if the log file had the pid in it then I could get all the right info. | 22:31 |
| bgmccollum | the log file does have the PID in it | 22:31 |
| notmyname | and that would be simple* to add | 22:31 |
| bgmccollum | container-server 1697 | 22:31 |
| notmyname | the filename | 22:31 |
| notmyname | (* funny story about "simple". when I was at rackspace, he who called a problem simple by default volunteered to write the solution. I have since stopped calling things "simple") | 22:32 |
| bgmccollum | i usually jump to calling things impossible...then it ends up being like 3 lines | 22:32 |
| *** pberis has quit IRC | 22:32 | |
| notmyname | hmm..looks like I need to update that code anyway since the log fields were fixed upstream (account and container used to have two log fields swapped) | 22:33 |
| *** pberis has joined #openstack-swift | 22:34 | |
| *** CaioBrentano has joined #openstack-swift | 22:35 | |
| notmyname | ah. just comments. values aren't actually used (yet) | 22:36 |
| *** CaioBrentano has quit IRC | 22:36 | |
| notmyname | bgmccollum: actually, what I think I want is that the PID is added to the log line itself. that way it works with multiple log files or just a single log file | 22:37 |
| bgmccollum | im seeing PIDs as part of the user agent in my logs | 22:37 |
| Tyger | notmyname: You mean the PID of the server processing the request? | 22:38 |
| notmyname | Tyger: ya | 22:38 |
| bgmccollum | oh wait...thats the referrer UA / PID | 22:38 |
| notmyname | ya, and that's good | 22:39 |
| notmyname | and useful | 22:39 |
| notmyname | bgmccollum: but look at http://d.not.mn/log_flow.png again | 22:39 |
| notmyname | bgmccollum: what I want is that I can know that the proxy is taking to which one of the 3 object servers with pids (and same with object->container) | 22:39 |
| bgmccollum | yeah i see now | 22:41 |
| *** pberis has quit IRC | 22:43 | |
| bgmccollum | you know there were some puts from a particular proxy to a set of object servers, but you dont have PIDS for those object server | 22:43 |
| *** pberis has joined #openstack-swift | 22:45 | |
| notmyname | bgmccollum: playing around with something | 22:49 |
| *** tsg has joined #openstack-swift | 22:49 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Use except x as y instead of except x, y https://review.openstack.org/96610 | 22:50 |
| torgomatic | did we lose the auto-abandon bot at some point? | 22:54 |
| notmyname | bgmccollum: http://d.not.mn/sample_target.png <-- that's what I want | 22:55 |
| torgomatic | I'm pretty sure that patches with negative feedback used to get automatically-abandoned at some point, but that doesn't seem to be happening any more | 22:55 |
| notmyname | bgmccollum: I hand-added a pid to the end of each log line | 22:55 |
| notmyname | torgomatic: after one week, used to | 22:55 |
| torgomatic | notmyname: well, it's not any more; see https://review.openstack.org/#/c/91729/ for proof | 22:56 |
| notmyname | torgomatic: seems hugokuo doesn't like it anymore anyway. I'll abandon it | 22:57 |
| torgomatic | notmyname: thanks; I'll go ask in -infra and see what happened | 22:58 |
| notmyname | torgomatic: thanks | 22:58 |
| *** kevinc_ has quit IRC | 22:58 | |
| openstackgerrit | Samuel Merritt proposed a change to openstack/swift: Zero-copy object-server PUT requests with splice() https://review.openstack.org/104705 | 23:10 |
| *** cpen has quit IRC | 23:23 | |
| *** pberis has quit IRC | 23:23 | |
| *** occupant has quit IRC | 23:23 | |
| *** pberis has joined #openstack-swift | 23:26 | |
| TaiSHi | notmyname: I feel stalked <3 | 23:29 |
| notmyname | TaiSHi: ;-) | 23:29 |
| notmyname | TaiSHi: actually, it's kinda nice that you called out digital ocean about supporting swift. I'd love to see that happen :-) | 23:30 |
| TaiSHi | I'm very interested on this and I do enjoy testing/implementing new stuff | 23:31 |
| *** occupant has joined #openstack-swift | 23:33 | |
| zaitcev | http://digitalocean.net/ -- Inspiring Action For Ocean Sustainability | 23:34 |
| notmyname | #swift on freenode <- students with an interest in the future of technology | 23:36 |
| *** ho has joined #openstack-swift | 23:41 | |
| notmyname | zaitcev: FWIW, https://www.digitalocean.com is the new hotness in hosting these days | 23:44 |
| zaitcev | notmyname: I know | 23:44 |
| notmyname | but they aren't cool yet, IMO, because they don't run swift (like RAX and HP and others) | 23:45 |
| notmyname | ;-) | 23:45 |
| *** elambert has quit IRC | 23:45 | |
| notmyname | yet | 23:45 |
| zaitcev | I'm going to stay with Linode for now. | 23:45 |
| ctennis | doubtful that will happen notmyname | 23:46 |
| notmyname | zaitcev: they aren't cool either, for the same reason ;-) | 23:47 |
| TaiSHi | Linode is a bit more expensive and has load balancers | 23:47 |
| TaiSHi | I'm currently using a homebrew one :P | 23:47 |
| TaiSHi | Good thing is that DO doesn't charge for bw as of yet | 23:48 |
| TaiSHi | I was able to pull out 600MB/s out of a 5$ vm :P | 23:53 |
| *** markd has joined #openstack-swift | 23:54 | |
| *** tsg has quit IRC | 23:56 | |
| *** tsg has joined #openstack-swift | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
