| mattoliverau | ho: good morning | 00:01 |
|---|---|---|
| ho | mattoliverau: morning! today japan has a game in AFC but it will be started from 18:00 jst. it is a bit early for me. :) | 00:07 |
| mattoliverau | So you can get some work in before it starts :) | 00:09 |
| ho | mattoliverau: hahaha. | 00:10 |
| *** atan8 has joined #openstack-swift | 00:16 | |
| openstackgerrit | paul luse proposed openstack/swift: New hashes.pkl format and other reconstructor changes https://review.openstack.org/131872 | 00:20 |
| peluse | clayg, ^^ some fun stuff to talk about... will try and catch up w/you tomorrow afternoon. fun stuff!!! | 00:23 |
| *** dmorita has joined #openstack-swift | 00:28 | |
| *** atan8 has quit IRC | 00:44 | |
| *** Nadeem has quit IRC | 00:45 | |
| *** elambert has quit IRC | 00:48 | |
| *** addnull has joined #openstack-swift | 01:02 | |
| *** lcurtis has quit IRC | 01:08 | |
| *** addnull has quit IRC | 01:14 | |
| *** abhirc has quit IRC | 01:15 | |
| *** addnull has joined #openstack-swift | 01:15 | |
| *** addnull has quit IRC | 01:18 | |
| *** addnull has joined #openstack-swift | 01:19 | |
| *** addnull has quit IRC | 01:22 | |
| *** lcurtis has joined #openstack-swift | 01:23 | |
| *** addnull has joined #openstack-swift | 01:25 | |
| *** addnull has quit IRC | 01:26 | |
| *** addnull has joined #openstack-swift | 01:26 | |
| *** echevemaster has joined #openstack-swift | 01:42 | |
| *** echevemaster has quit IRC | 01:42 | |
| *** abhirc has joined #openstack-swift | 01:47 | |
| *** fandi has quit IRC | 02:44 | |
| *** addnull has quit IRC | 03:04 | |
| *** elambert has joined #openstack-swift | 03:37 | |
| *** bill_az has quit IRC | 03:52 | |
| *** Masahiro has quit IRC | 03:55 | |
| *** Masahiro has joined #openstack-swift | 03:58 | |
| *** elambert has quit IRC | 03:59 | |
| *** lcurtis has quit IRC | 04:15 | |
| *** chlong has quit IRC | 04:19 | |
| *** addnull has joined #openstack-swift | 04:21 | |
| *** chlong has joined #openstack-swift | 04:28 | |
| *** ppai has joined #openstack-swift | 04:53 | |
| *** linuxgeek_ has quit IRC | 05:01 | |
| *** bkopilov has joined #openstack-swift | 05:05 | |
| *** david-lyle has joined #openstack-swift | 05:15 | |
| ho | notmyname: I would like you to solve an issue regarding to https://review.openstack.org/#/c/89568/. | 05:24 |
| ho | notmyname: The issue is I would like to have this functionality but it seems to be no progress from Apr 29, 2014 in the gerrit | 05:25 |
| ho | notmyname: therefore I set an email in Dec 12 to the owner but couldn't get any response. | 05:25 |
| ho | notmyname: I would like to take it over to realize the functionality as an official one. | 05:25 |
| ho | notmyname: Could you please handle this issue? | 05:25 |
| notmyname | ho: hello | 05:33 |
| ho | notmyname: hi! | 05:33 |
| notmyname | ho: have you looked at the cloudwatt things listed in the final gerrit comment on that patch? | 05:33 |
| ho | notmyname: i checked it. | 05:34 |
| notmyname | is it what you're looking for? ie the middleware on stackforge | 05:35 |
| ho | notmyname: i think keystoneauth.py has to have RBAC only swift doesn't have this func. in openstack components means swift is better to support RBAC with keystone officially | 05:36 |
| * notmyname looks | 05:38 | |
| notmyname | seems that the cloudwatt middleware does depend on keystone. I'm guessing you need the functionality apart from keystone? | 05:39 |
| notmyname | are there any docs for this rbac anywhere? | 05:40 |
| notmyname | my first try was https://github.com/openstack/oslo.policy but that's an empty repo (yay!) | 05:41 |
| ho | notmyname: i think to support RBAC with Keystone we use oslo-incubator https://github.com/openstack/oslo-incubator (https://github.com/openstack/oslo-incubator/tree/master/openstack/common/policy.py) | 05:42 |
| notmyname | ah, thanks | 05:43 |
| notmyname | "Policies can be expressed in one of two forms: A list of lists, or a string written in the new policy language." | 05:44 |
| notmyname | ho: you need it with other auth besides keystone? | 05:47 |
| notmyname | ho: or are you using keystone but you jsut want this to be in-tree? | 05:47 |
| *** ppai has quit IRC | 05:48 | |
| ho | notmyname: my idea is integrate this policy func into current keystoneauth.py | 05:48 |
| notmyname | ho: instead of being separate middleware? | 05:49 |
| ho | notmyname: yes. | 05:49 |
| notmyname | ok. just trying to understand your motivation here | 05:49 |
| ho | notmyname: thanks for asking this. | 05:50 |
| mattoliverau | It's a notmyname! | 05:50 |
| notmyname | where? | 05:51 |
| StevenK | In a mirror. | 05:52 |
| notmyname | fortunately, I have not mirrors handy | 05:52 |
| notmyname | ho: do you know what it will take (ie level of code changes) to get the functionality in the middleware into keystone auth? | 05:52 |
| notmyname | ie have you scoped the work yet? | 05:52 |
| ho | notmyname: from last week i started prototyping. i think i can write it until the hack-a-thon. | 05:54 |
| notmyname | cool | 05:54 |
| notmyname | ho: here's my thoughts | 05:55 |
| ho | notmyname: include oslo-incubator steps. my estimation is 3000 steps. | 05:55 |
| notmyname | heh | 05:56 |
| notmyname | ho: nobody has looked at it (the current patch) because it's marked WIP. it should probably be abandoned | 05:56 |
| notmyname | ho: the oslo incubator part is a show-stopper. we aren't going to put oslo incubator code into the swift source tree. however, since there's at least an empty repo for it, perhaps it's being moved soon | 05:57 |
| notmyname | ho: overall the functionality looks interesting. as in, I can see how it offers flexibility. I'm not too keen on the "whole new policy language" + "default allow" rules. but, on the other hand, someone else has already written it, so that means we don't have to invent it from scratch | 05:58 |
| ho | notmyname: i think so too. most difficult thing is keeping backward comatibility :) | 05:59 |
| notmyname | ho: you should talk to cschwede about it. he's closely involved with cloudwatt. and should be at the hackathon | 05:59 |
| ho | notmyname: thanks! i will discuss this with cshwede. | 06:00 |
| notmyname | ho: your changes to incorporate into keystone auth would require compat with the existing stuff. ie whatever other openstack projects are using. so that means we'd have to use the oslo library | 06:01 |
| notmyname | ho: do you think there are any compat issues with the cloudwatt middleware? | 06:01 |
| *** ppai has joined #openstack-swift | 06:01 | |
| ho | notmyname: it is as a middleware so there is no compat problem but it can not support 'allow_names_in_acls' | 06:05 |
| ho | sorry s/it is as a middleware/it is implemented as a middleware/ | 06:07 |
| notmyname | ho: what's that? something in the cloudwatt middleware or something in the policy that swift cant' support? | 06:07 |
| notmyname | (the allow_names_in_acls) | 06:08 |
| *** SkyRocknRoll has joined #openstack-swift | 06:08 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:08 | |
| ho | notmyname: a configuration setting of keystoneauth. this setting for backcompat when keystoneauth supports keystone v3 | 06:10 |
| notmyname | ah ok | 06:10 |
| *** zaitcev has quit IRC | 06:10 | |
| ho | notmyname: behavior of is_admin looks different. | 06:16 |
| ho | notmyname: thanks a lot :) | 06:22 |
| clayg | torgomatic: I had this nightmare that the way the rebalance code works it'll run the algo multiple times until like the number of parts stops changing or something... then I was thinking that I added the _build_dispersion_graph inside of that multi-rebalance loop instead of outside | 06:22 |
| clayg | torgomatic: do you think I just imagined that? | 06:23 |
| clayg | phew, it does rebalance multiple times, but _build_dispersion_graph is only called at the end | 06:24 |
| clayg | or... maybe I should say rebalance calls gather/reassign multiple times | 06:24 |
| *** lihkin has quit IRC | 06:24 | |
| *** silor has joined #openstack-swift | 06:29 | |
| *** nshaikh has joined #openstack-swift | 06:43 | |
| *** david-lyle has quit IRC | 06:57 | |
| openstackgerrit | Daisuke Morita proposed openstack/swift: Improvement of logging for PolicyError https://review.openstack.org/141238 | 07:07 |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Allow rsync to use compression https://review.openstack.org/148454 | 07:19 |
| *** sileht has quit IRC | 07:25 | |
| *** fandi has joined #openstack-swift | 07:28 | |
| *** addnull has quit IRC | 07:29 | |
| *** addnull has joined #openstack-swift | 07:30 | |
| *** Masahiro has quit IRC | 07:31 | |
| *** david-lyle has joined #openstack-swift | 07:31 | |
| *** Masahiro has joined #openstack-swift | 07:31 | |
| *** fandi has quit IRC | 07:32 | |
| openstackgerrit | Koert van der Veer proposed openstack/swift: Allow default reseller prefix in domain_remap middleware https://review.openstack.org/142067 | 07:33 |
| *** fandi has joined #openstack-swift | 07:33 | |
| *** sileht has joined #openstack-swift | 07:33 | |
| *** fandi has quit IRC | 07:35 | |
| *** david-lyle has quit IRC | 07:35 | |
| *** fandi has joined #openstack-swift | 07:37 | |
| *** ppai has quit IRC | 07:38 | |
| *** chlong has quit IRC | 07:41 | |
| *** david-lyle has joined #openstack-swift | 07:42 | |
| *** david-lyle has quit IRC | 07:47 | |
| *** ppai has joined #openstack-swift | 07:50 | |
| *** Masahiro has quit IRC | 07:52 | |
| *** miqui_ has quit IRC | 07:54 | |
| *** david-lyle has joined #openstack-swift | 08:01 | |
| *** david-lyle has quit IRC | 08:05 | |
| *** rledisez has joined #openstack-swift | 08:22 | |
| *** oomichi has quit IRC | 08:42 | |
| *** jordanP has joined #openstack-swift | 08:51 | |
| *** ppai has quit IRC | 08:54 | |
| *** nellysmitt has joined #openstack-swift | 09:00 | |
| *** acoles_away is now known as acoles | 09:05 | |
| *** ppai has joined #openstack-swift | 09:05 | |
| *** david-lyle has joined #openstack-swift | 09:19 | |
| *** foexle has joined #openstack-swift | 09:23 | |
| *** nellysmitt has quit IRC | 09:25 | |
| *** chlong has joined #openstack-swift | 09:27 | |
| *** jistr has joined #openstack-swift | 09:35 | |
| openstackgerrit | Daisuke Morita proposed openstack/swift: Show each policy's information on quarantined files in recon https://review.openstack.org/137769 | 09:42 |
| *** fandi has quit IRC | 10:05 | |
| *** dmorita has quit IRC | 10:11 | |
| *** nellysmitt has joined #openstack-swift | 10:15 | |
| openstackgerrit | Yuan Zhou proposed openstack/swift: Update contianer sync to use internal client https://review.openstack.org/143791 | 10:20 |
| *** fandi has joined #openstack-swift | 10:25 | |
| *** addnull has quit IRC | 10:29 | |
| *** ahonda has quit IRC | 10:45 | |
| *** rledisez has quit IRC | 10:53 | |
| openstackgerrit | Donagh McCabe proposed openstack/swift: Add multiple reseller prefixes and composite tokens https://review.openstack.org/137086 | 10:55 |
| *** jamieh_ has joined #openstack-swift | 11:02 | |
| *** jamieh_ has quit IRC | 11:03 | |
| *** jamieh_ has joined #openstack-swift | 11:03 | |
| *** jamieh_ has joined #openstack-swift | 11:03 | |
| *** jamiehan_ has joined #openstack-swift | 11:04 | |
| *** addnull has joined #openstack-swift | 11:14 | |
| *** ho has quit IRC | 11:25 | |
| *** mahatic has joined #openstack-swift | 11:25 | |
| *** aix has joined #openstack-swift | 11:33 | |
| *** tellesnobrega_ has quit IRC | 11:42 | |
| ekarlso | hey guys, does python-swiftclient support keystone sessions ? | 12:00 |
| *** ppai has quit IRC | 12:07 | |
| *** mahatic has quit IRC | 12:08 | |
| *** tellesnobrega_ has joined #openstack-swift | 12:08 | |
| *** mahatic has joined #openstack-swift | 12:08 | |
| *** km has quit IRC | 12:15 | |
| *** pcaruana has joined #openstack-swift | 12:17 | |
| *** tellesnobrega has quit IRC | 12:46 | |
| *** fandi has quit IRC | 12:51 | |
| *** tellesnobrega has joined #openstack-swift | 12:52 | |
| openstackgerrit | Donagh McCabe proposed openstack/swift-specs: Minor updates to composite token spec https://review.openstack.org/138771 | 12:55 |
| *** addnull has quit IRC | 12:57 | |
| acoles | ekarlso: if you are referring to keystoneclient Session class, afaik swiftclient does not use that | 13:01 |
| ekarlso | acoles: *sadfacE* | 13:02 |
| *** lpabon has joined #openstack-swift | 13:07 | |
| *** Anticimex has quit IRC | 13:08 | |
| *** Anticimex has joined #openstack-swift | 13:09 | |
| *** EmilienM|afk is now known as EmilienM | 13:21 | |
| *** bill_az has joined #openstack-swift | 13:32 | |
| *** abhirc_ has joined #openstack-swift | 13:58 | |
| *** abhirc has quit IRC | 14:00 | |
| *** jamiehan_ has quit IRC | 14:07 | |
| *** SkyRocknRoll has quit IRC | 14:18 | |
| *** abhirc_ has quit IRC | 14:20 | |
| *** abhirc has joined #openstack-swift | 14:21 | |
| *** abhirc has quit IRC | 14:36 | |
| *** nshaikh has left #openstack-swift | 14:36 | |
| *** nottrobin_ is now known as nottrobin | 14:43 | |
| *** abhirc has joined #openstack-swift | 14:55 | |
| *** lihkin has joined #openstack-swift | 14:58 | |
| *** abhirc has quit IRC | 14:59 | |
| *** tellesnobrega_ has quit IRC | 15:00 | |
| *** abhirc has joined #openstack-swift | 15:00 | |
| *** lihkin has quit IRC | 15:01 | |
| openstackgerrit | Donagh McCabe proposed openstack/swift: Add multiple reseller prefixes and composite tokens https://review.openstack.org/137086 | 15:01 |
| *** abhirc has quit IRC | 15:02 | |
| *** lihkin has joined #openstack-swift | 15:05 | |
| openstackgerrit | Merged openstack/swift: Fix missing content length of Response https://review.openstack.org/147049 | 15:08 |
| *** zaitcev has joined #openstack-swift | 15:11 | |
| *** ChanServ sets mode: +v zaitcev | 15:11 | |
| *** bkopilov has quit IRC | 15:33 | |
| *** gyee has joined #openstack-swift | 15:38 | |
| *** bkopilov has joined #openstack-swift | 15:46 | |
| *** tdasilva has joined #openstack-swift | 15:53 | |
| *** bkopilov has quit IRC | 16:13 | |
| *** lcurtis has joined #openstack-swift | 16:16 | |
| *** Anticimex has quit IRC | 16:23 | |
| *** tellesnobrega_ has joined #openstack-swift | 16:29 | |
| notmyname | good morning | 16:39 |
| *** tellesnobrega_ has quit IRC | 16:39 | |
| *** abhirc has joined #openstack-swift | 16:40 | |
| *** Anticimex has joined #openstack-swift | 16:40 | |
| mahatic | notmyname, good morning | 16:45 |
| notmyname | mahatic: hi! how are things? | 16:47 |
| mahatic | notmyname, things are okay. Confusing mock tests! | 16:47 |
| saltsa | helo, not sure if this is right channel to ask but: Is there really any (good) alternatives for Swift? :) Requirements are that it should be able to store millions of <100MB objects with total storage some hunder terabytes. | 16:49 |
| glange | saltsa: ceph, cassandra are other options | 16:51 |
| notmyname | saltsa: heh. I think that's a funny question to ask in here | 16:51 |
| notmyname | saltsa: just to get the obvious out of the way: no, of course there aren't any other good alternatives! ;-) | 16:51 |
| saltsa | how about riak cs? | 16:52 |
| saltsa | yeah, i know it's funny. But just exploring the alternatives :) How would I say, I don't feel Swift very stable and it's not packaged so nicely than some others. Basicly it must be installed from the git repo. | 16:53 |
| notmyname | ouch | 16:53 |
| notmyname | tell me about your concerns around stability in swift? | 16:54 |
| glange | http://docs.openstack.org/developer/swift/development_saio.html <-- swift installed on one server from a git repo :) | 16:54 |
| glange | oh, you don't like that it's installed from a git repo | 16:55 |
| saltsa | yeah, and that's why I _feel_ it's not stable. Dunno whether it's or not :) | 16:55 |
| glange | it's stable as anything else | 16:56 |
| saltsa | and that guide is for Ubuntu. I would prefer Centos/RHEL based installation and packages .) | 16:56 |
| notmyname | saltsa: it's got a very stable API and has always supported live upgrades from one version to the next. it's running at massive scale all over the world | 16:56 |
| notmyname | saltsa: for packaging, if you aren't building your own packages, then you're going to need packages that someone else builds. and normally, that's going to be whatever distro you choose. so you'll be subject to their update cycles | 16:57 |
| notmyname | saltsa: while I'll admit packaging is hard (I hate it), you can find a lot of help in here. and rackspace publishes their packages they use for prod (.deb, not rpms) | 16:59 |
| notmyname | saltsa: and there are commercial solutions too, like swiftstack (disclosure, that's where I work). swiftstack has a free trial you can use and makes installing really easy | 17:00 |
| *** mkerrin has quit IRC | 17:02 | |
| notmyname | saltsa: but there are a lot of people in here who have deployed swift. I think you can get good feedback from them on swift's stability and help with packaging ( acoles, ahale, fifieldt_, cschwede, briancline, ctennis, thurloat, and I'm sure many others) | 17:02 |
| thurloat | hey! someone said my name. | 17:03 |
| swat30 | thurloat, ping | 17:03 |
| openstackgerrit | Alistair Coles proposed openstack/swift-specs: Updating containers on object fast-POST https://review.openstack.org/102592 | 17:03 |
| acoles | clayg: ^^ i added you as co author | 17:04 |
| notmyname | thurloat: hi! saltsa is looking at swift and could probably learn from your experience (see above about "I don't feel Swift [is] very stable" and hard to deploy (lack of current upstream packages) | 17:05 |
| saltsa | one important need is the disaster recovery: let's assume the entire swift-software brokes. Is it possible to just crawl through the filesystems and find the objects based on the metadata or name easily? | 17:06 |
| notmyname | saltsa: yes. data is stored as files on disk and metadata is in xattrs on those files | 17:07 |
| thurloat | saltsa: I can certainly vouch for swift's stability. We have it deployed in a public cloud setting, so it gets exercised quite well. Both large objects (100s of GB each) and buckets filled with tiny objects. | 17:07 |
| saltsa | notmyname: is the file content same as the object content? | 17:07 |
| notmyname | saltsa: yes | 17:07 |
| saltsa | okey, and then: probably you're using xfs as the filesystem? Any experience of btrfs and its stability? | 17:08 |
| notmyname | saltsa: I haven't looked at btrfs. and yes, xfs is what's recommended | 17:09 |
| thurloat | swat30 and can answer any specific questions you might have as well | 17:09 |
| thurloat | our object servers are using xfs as well | 17:09 |
| zaitcev | saltsa: I never install Swift from git repo. I always do "yum install openstack-swift" and that does the job. | 17:13 |
| saltsa | zaitcev: are you running fedora or centos/rhel= | 17:14 |
| zaitcev | saltsa: RHEL 6, and yes, I know that it's obsolete | 17:15 |
| saltsa | okey, and that package is from epel or? | 17:16 |
| saltsa | what i was looking it was a quite old version from the rpm compared to the latest tag in git repo? | 17:17 |
| notmyname | zaitcev: aren't you the one who builds the red hat packages? | 17:21 |
| zaitcev | It is old, yes. Well, actually, I had 2.1 repackaged... Just dropped the tarball in. | 17:21 |
| zaitcev | hold on, lemme switch the hats here | 17:21 |
| notmyname | lol | 17:21 |
| zaitcev | okay, fedora on | 17:22 |
| zaitcev | notmyname: not really, Alan Pevec mostly does it for RDO and Lon Hohberger builds and pushes RHEL OSP. | 17:22 |
| notmyname | ah ok | 17:22 |
| zaitcev | you'd think I'd have way more time to do upstream reviews properly | 17:23 |
| zaitcev | with a help like that | 17:23 |
| *** Nadeem has joined #openstack-swift | 17:24 | |
| *** bkopilov has joined #openstack-swift | 17:29 | |
| tdasilva | saltsa: I just put this together last weekend in case you want to try SAIO on Fedora: https://github.com/thiagol11/swift-aio | 17:35 |
| saltsa | tdasilva: well, currently i have centos7 nodes, but could change that easily :) | 17:36 |
| saltsa | but thanks, will look at it | 17:37 |
| *** jordanP has quit IRC | 17:38 | |
| tdasilva | saltsa: it currently installs from git repo thou... | 17:39 |
| *** shri has joined #openstack-swift | 17:44 | |
| *** aix has quit IRC | 17:55 | |
| *** lcurtis has quit IRC | 17:57 | |
| *** pcaruana has quit IRC | 18:00 | |
| clayg | acoles: thanks I'll give it a read! | 18:04 |
| clayg | notmyname: I wonder if someone adds the swiftstack platform repos just download the latest packaged swift if they could then break/uninstall ssnoded and never claim it? | 18:05 |
| clayg | notmyname: and configure it manually.. | 18:05 |
| clayg | notmyname: saltsa: besides doesn't RDO package for centos/rhel? zaitcev <<< | 18:06 |
| *** acoles is now known as acoles_away | 18:06 | |
| clayg | zaitcev: oh sorry, looks like I wasn't quite done with my scroll back... | 18:06 |
| zaitcev | clayg, saltsa: Yeah well... I forgot that I was cheating with rebuilds of up-to-date RPMs. I usually steal those from RDO and graft them onto my ancient RHEL. Yes, we have 2.x coming down for RHEL users, Any Day Now | 18:07 |
| clayg | there's a centos7!? | 18:07 |
| zaitcev | yes, of course | 18:08 |
| clayg | shit | 18:08 |
| *** chlong has quit IRC | 18:08 | |
| zaitcev | It's forked off F19 and follows RHEL 7. Gird your loins for SystemD, baby. | 18:08 |
| clayg | oh, maybe that's why i haven't seen it - seems most of the folks I'm on the phone with have 6.5 or 6.5 rhel/centos | 18:10 |
| *** dmsimard_away is now known as dmsimard | 18:10 | |
| *** jistr has quit IRC | 18:14 | |
| *** ChanServ changes topic to "Priority Reviews: https://wiki.openstack.org/wiki/Swift/PriorityReviews | Review Dashboard: http://goo.gl/r2mxbe | Ideas: https://wiki.openstack.org/wiki/Swift/ideas" | 18:15 | |
| zaitcev | You'd be amazed how much folks like to cling to their old RHEL. We have 7 year support now, you just didn't see it in Swift because Swift itself is younger, so RHEL 4 was never supported with OpenStack. But, ugh. | 18:16 |
| *** shri has quit IRC | 18:16 | |
| zaitcev | The fun part is, Microsoft has it worse. The Windows XP is actually _older_ than RHEL 4 by about 2 years. | 18:17 |
| zaitcev | Or was it older than RHEL 2.1, I don't remember | 18:17 |
| *** silor has quit IRC | 18:17 | |
| zaitcev | And their users expect that to work on Internet. It's insane. | 18:17 |
| *** ChanServ changes topic to "Hackathon: https://www.eventbrite.com/e/openstack-swift-hack-a-thon-sponsored-by-swiftstack-tickets-15226147818 | Priority Reviews: https://wiki.openstack.org/wiki/Swift/PriorityReviews | Review Dashboard: http://goo.gl/r2mxbe" | 18:19 | |
| saltsa | zaitcev: okey, thanks :) | 18:21 |
| notmyname | hackathon link is now in the topic message | 18:22 |
| notmyname | hackathon link is https://www.eventbrite.com/e/openstack-swift-hack-a-thon-sponsored-by-swiftstack-tickets-15226147818 | 18:22 |
| tdasilva | notmyname: cool, looking forward to it | 18:23 |
| notmyname | all the new meeting rooms in the new swiftstack office are named after openstack releases | 18:30 |
| *** gyee has quit IRC | 18:30 | |
| mahatic | cool! | 18:32 |
| *** tellesnobrega_ has joined #openstack-swift | 18:33 | |
| zaitcev | you're gonna need a bigger building | 18:34 |
| tdasilva | notmyname: is that hackathon meeting in the Kilo room? | 18:35 |
| notmyname | tdasilva: austin bexar and kilo, actually | 18:37 |
| notmyname | :-) | 18:37 |
| mahatic | torgomatic, lol at the comment. Thanks for simply removing a line from the stupid test :D | 18:38 |
| *** lcurtis has joined #openstack-swift | 18:38 | |
| mahatic | torgomatic, thanks much for removing duplicacy in my code. I knew it was raw, I was too caught up with the test to pretty up the code. Callable is working good! | 18:39 |
| clayg | that torgomatic is such a nice guy | 18:42 |
| lcurtis | swift object server and replicator consumings tons of memory in my beta environment | 18:43 |
| lcurtis | is that due to size of partitions or some other formula | 18:43 |
| notmyname | lcurtis: well maybe. it's not just buffer space (linux reports a lot of memory used that is available to the system. eg for page cache) | 18:44 |
| notmyname | lcurtis: umm..that was supposed to be a question | 18:45 |
| notmyname | lcurtis: can you check that it's not just buffer space...? | 18:45 |
| lcurtis | just looking at ps | 18:53 |
| lcurtis | and virt | 18:53 |
| lcurtis | lots of object replicator pids and object server pids | 18:54 |
| lcurtis | each using 1.5gb | 18:54 |
| lcurtis | does that seem normal> | 18:55 |
| ahale | is virt size like that a big deal? isnt rss more realistic view (in as close as you can get to seeing a processes memory use) | 18:56 |
| ahale | like mine are "swift 4980 9.4 0.0 1561044 37836" for an obj server in ps which isnt really worrying | 18:57 |
| lcurtis | interesting | 18:58 |
| lcurtis | mine seem very similar | 18:58 |
| ahale | i can never remember the details of memory use - VSS/RSS - always have to read up | 18:59 |
| lcurtis | me too! | 18:59 |
| *** gvernik has joined #openstack-swift | 19:00 | |
| lcurtis | i do have quite a few processes though | 19:01 |
| ahale | have you been restarting/reloading a bit ? | 19:01 |
| ahale | you might have some orphan processes | 19:01 |
| lcurtis | no not at all | 19:01 |
| swifterdarrell | lcurtis: ahale: ya, RSS + actual swap activity is more interesting than VSS; also, slabtop can reveal another place physical RAM goes (kernel dentry/inode caching, etc.). | 19:01 |
| lcurtis | swiftdarrell: great tip | 19:02 |
| swifterdarrell | lcurtis: ahale: and for pre-fork servers, copy-on-write shared memory can also make RSS overly pessimistic (though I've heard Python does a good job of just dirtying all the pages again anyway, heh) | 19:03 |
| lcurtis | thanks for that | 19:03 |
| ahale | heh yeah - trusting the numbers to be precise never works | 19:03 |
| swifterdarrell | lcurtis: ahale: to get a better idea in the past, I wrote something that actually parsed /proc/$PID/maps or something, which gives you more accurate info on shared RSS pages | 19:03 |
| swifterdarrell | lcurtis: ahale: and rolled that up into domain-specific application groups interesting in our product (this was a previous company) | 19:04 |
| ahale | I'm normally happy if none of the processes have hit D state and teh swap is only maybe a few kb | 19:04 |
| lcurtis | is there a certain number of processes that should be spun up at any given time? | 19:04 |
| lcurtis | ie, swift processes | 19:04 |
| ahale | ah nice - i've done stuff with pmap as well , but really it all just ends up best guessing | 19:05 |
| swifterdarrell | lcurtis: ahale: to make the most of RAM with a very large Perl codebase, Apache+mod_perl started first, then forked off all the other background daemons from there so mod_perl apache workers and 10 to 20 background daemons of various ilk still shared a ton of RAM | 19:05 |
| ahale | theres the worker counts in confs that should limit it | 19:05 |
| lcurtis | ah...okay | 19:05 |
| swifterdarrell | lcurtis: ahale: in that env, RSS was completely useless | 19:05 |
| *** lihkin has quit IRC | 19:06 | |
| ahale | sounds like some of the horrors I saw working support | 19:06 |
| lcurtis | really great insight | 19:06 |
| lcurtis | definitely have to do more homework | 19:06 |
| *** lihkin1 has joined #openstack-swift | 19:07 | |
| *** gvernik has quit IRC | 19:27 | |
| *** mahatic has quit IRC | 19:29 | |
| *** tellesnobrega_ has quit IRC | 19:30 | |
| *** david-lyle has quit IRC | 19:37 | |
| *** gvernik has joined #openstack-swift | 19:40 | |
| *** mahatic has joined #openstack-swift | 19:44 | |
| *** openstackstatus has joined #openstack-swift | 19:46 | |
| *** ChanServ sets mode: +v openstackstatus | 19:46 | |
| -openstackstatus- NOTICE: Gerrit restarted to address likely memory leak leading to server slowness. Sorry if you were caught in the restart | 19:54 | |
| *** nellysmitt has quit IRC | 20:02 | |
| *** nellysmitt has joined #openstack-swift | 20:02 | |
| *** erlon has joined #openstack-swift | 20:03 | |
| clayg | ctennis: torgomatic: I just think it's kinda weird that the stats end up being so exspensive; after a restart I'd expect most of the stat info on the parts to be cached - maybe it gets thrown out once the replicator starts chewing on the suffixes - anyway https://review.openstack.org/#/c/140178/ looks great - priority review or not! | 20:09 |
| *** gyee has joined #openstack-swift | 20:10 | |
| ctennis | clayg: I mean, in general it's fast, the main issue is on systems that are way far behind on replication, you have 10x partitions living on a disk that aren't supposed to be there | 20:11 |
| ctennis | they can never catch up! | 20:11 |
| *** tellesnobrega_ has joined #openstack-swift | 20:11 | |
| ctennis | clayg: in this particular customer's case, 10x replicas across 36 disks. replicator took over 4 hours from start before it would do anything. | 20:12 |
| openstackgerrit | Mahati proposed openstack/swift: Implement OPTIONS verb for storage nodes. https://review.openstack.org/140103 | 20:18 |
| mahatic | notmyname, can you take a look at it? I have a test scenario that does not work for that | 20:19 |
| *** nellysmitt has quit IRC | 20:20 | |
| notmyname | ctennis: computers are hard | 20:22 |
| *** tellesnobrega_ has quit IRC | 20:27 | |
| *** silor has joined #openstack-swift | 20:29 | |
| *** sk91 has joined #openstack-swift | 20:33 | |
| notmyname | hmm..looks like the cluster isn't set to silent and is failing tests | 20:35 |
| notmyname | hugokuo: charz: ^^ anything you can do about that? | 20:35 |
| sk91 | Hi,im installing openstack juno for Ubuntu 14.04 from ( http://docs.openstack.org/juno/install-guide/install/apt/content/swift-install-storage-node.html ) | 20:35 |
| sk91 | im using virtualization, and all my nodes are virtual machines,at the installing swift node parts, it refers to /dev/sdb and /dev/stc , doest it mean that i have to set up 2 virtual machines with ubuntu on them one for /dev/sdb and one /dev/sdc? | 20:37 |
| *** silor has quit IRC | 20:53 | |
| *** lihkin1 has quit IRC | 20:57 | |
| *** lihkin has joined #openstack-swift | 20:58 | |
| *** sk91 has quit IRC | 21:01 | |
| *** gvernik has quit IRC | 21:09 | |
| *** sk91 has joined #openstack-swift | 21:12 | |
| mattoliverau | Morning | 21:14 |
| *** tellesnobrega_ has joined #openstack-swift | 21:22 | |
| sk91 | would u mind helping me with block storage (swift) partition table's details? what i understood is that swift got /dev/sda, /dev/sdb and /dev/sdc which all formats could be ex4 and /dev/sda is where node's os ( i.e ubuntu 14.04 ) is installed on | 21:23 |
| *** dosaboy has quit IRC | 21:26 | |
| *** tellesnobrega_ has quit IRC | 21:32 | |
| *** sk91 has quit IRC | 21:36 | |
| mattoliverau | sk91: you need to tell the storage node the location of the drives (the mount point), these drives need to be xfs or ext4. If your just testing you can use loop mounts or partitions. | 21:39 |
| *** dosaboy has joined #openstack-swift | 21:41 | |
| *** tellesnobrega_ has joined #openstack-swift | 21:45 | |
| *** jeblair has quit IRC | 21:51 | |
| *** sk91 has joined #openstack-swift | 21:52 | |
| *** sk91 has quit IRC | 22:24 | |
| notmyname | what did setting up my SAIO for EC change that makes probe tests not work? | 22:48 |
| *** gyee has quit IRC | 22:53 | |
| *** gyee has joined #openstack-swift | 22:54 | |
| *** foexle has quit IRC | 22:58 | |
| *** tdasilva has quit IRC | 23:04 | |
| *** gyee has quit IRC | 23:06 | |
| *** lihkin has quit IRC | 23:08 | |
| *** lihkin has joined #openstack-swift | 23:10 | |
| peluse | notmyname, the fact that its not 3x on the ring and not 4 devices | 23:10 |
| peluse | notmyname, see abandonded change https://review.openstack.org/#/c/143002/ for more info | 23:12 |
| notmyname | peluse: ya, I think I've got that. policy 0 is default. 3 replicas. 4 devices | 23:13 |
| *** abhirc has quit IRC | 23:30 | |
| *** oomichi has joined #openstack-swift | 23:30 | |
| *** abhirc has joined #openstack-swift | 23:30 | |
| *** abhirc has quit IRC | 23:35 | |
| *** chlong has joined #openstack-swift | 23:40 | |
| *** chlong has quit IRC | 23:45 | |
| *** chlong has joined #openstack-swift | 23:46 | |
| *** dmsimard is now known as dmsimard_away | 23:47 | |
| briancline | sorry to dig up ancient history, but has this seen enough testing to ensure it's not a concern for xattrs through normal API usage alone? https://bugs.launchpad.net/bugs/cve/2012-4406 | 23:51 |
| peluse | just FYI, in the midst of a merge master-->EC. Cleaning up a minor conflict. Should be up soon | 23:51 |
| torgomatic | briancline: the xattrs part isn't a concern, and the memcache thing was fixed | 23:56 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!