*** km___ has joined #openstack-swift | 00:48 | |
*** km has quit IRC | 00:49 | |
*** kei_yama has quit IRC | 00:50 | |
*** kei_yama has joined #openstack-swift | 00:51 | |
*** ChanServ sets mode: +v peluse | 01:27 | |
*** remix_tj has quit IRC | 01:52 | |
*** remix_tj has joined #openstack-swift | 01:52 | |
*** Vinsh has joined #openstack-swift | 01:56 | |
*** remix_tj has quit IRC | 01:57 | |
*** Vinsh has quit IRC | 02:00 | |
*** proteusguy has quit IRC | 02:37 | |
notmyname | hello mandarine | 02:55 |
---|---|---|
*** david-lyle has quit IRC | 02:55 | |
notmyname | also, mattoliverau | 02:58 |
*** jamielennox is now known as jamielennox|away | 02:58 | |
mattoliverau | notmyname: hey hey | 02:59 |
notmyname | mattoliverau: in my morning tomorrow, I want to get a handle on the essential patches for RC2. any input you have is invaluable | 03:02 |
mattoliverau | notmyname: k, I'll do a review of our reviews and send you my suggested additions :) | 03:03 |
*** jamielennox|away is now known as jamielennox | 03:03 | |
notmyname | :-) | 03:04 |
notmyname | thanks | 03:04 |
notmyname | I just updated https://wiki.openstack.org/wiki/Swift/PriorityReviews with a section | 03:07 |
notmyname | there's a section for RC2 | 03:07 |
notmyname | mattoliverau: so if you have stuff to add (or remove), that's the place | 03:07 |
notmyname | stuff for an RC2 needs to be stuff that fixes a big issue in RC1 and already has a patch to master | 03:07 |
*** silor has joined #openstack-swift | 03:13 | |
*** kei_yama has quit IRC | 03:14 | |
mattoliverau | k | 03:27 |
*** km___ has quit IRC | 03:36 | |
*** kei_yama has joined #openstack-swift | 03:37 | |
peluse | notmyname, I don't see the patch from clayg on the ECrecon that goes with the new pyeclib (he posted a github version of it last week). Maybe you should ping him tomorrow about that one and add it to the list. | 03:40 |
peluse | I'll be off/on all week (technically on vacation) | 03:40 |
*** kei_yama has quit IRC | 03:54 | |
*** kei_yama has joined #openstack-swift | 03:58 | |
*** Vinsh has joined #openstack-swift | 04:06 | |
*** Vinsh has quit IRC | 04:13 | |
*** kota_ has joined #openstack-swift | 04:53 | |
notmyname | kota_: hello! | 05:33 |
notmyname | kota_: I'm about to go to bed, but tomorrow I'll be organizing the RC2 patches. anything from you to take advantage of pyeclib 1.0.7 needs to be submitted asap (I'll also be talking to clayg about it tomorrow) | 05:34 |
kota_ | nomyname: hello :) | 05:34 |
kota_ | notmyname: yup, pyeclib 1.0.7 thing for global requirement seems to be blocked, right? | 05:36 |
notmyname | kota_: not for technical reasons afaik. I think we'll be able to get it in | 05:37 |
notmyname | at least I'm hoping until I'm told otherwise) | 05:37 |
notmyname | kota_: for now, assume it will land. | 05:38 |
kota_ | notmyname: ok I'll look at the status again. | 05:39 |
kota_ | notmyname: thanks to care about that! | 05:39 |
notmyname | :-) | 05:40 |
notmyname | have a good rest of your day. I'm going to bed | 05:40 |
kota_ | notmyname: have a nice sleep, good night ;) | 05:41 |
*** zhill has joined #openstack-swift | 05:54 | |
*** kota_ has quit IRC | 06:03 | |
*** SkyRocknRoll has joined #openstack-swift | 06:14 | |
*** SkyRocknRoll has joined #openstack-swift | 06:14 | |
*** kei_yama has quit IRC | 06:19 | |
*** kei_yama has joined #openstack-swift | 06:19 | |
*** zhill has quit IRC | 06:37 | |
*** jamielennox is now known as jamielennox|away | 06:52 | |
*** Vinsh has joined #openstack-swift | 07:13 | |
*** remix_tj has joined #openstack-swift | 07:15 | |
*** Vinsh has quit IRC | 07:18 | |
*** geaaru has joined #openstack-swift | 07:27 | |
*** km_ has joined #openstack-swift | 07:29 | |
*** chlong has quit IRC | 07:29 | |
*** bkopilov has quit IRC | 07:32 | |
*** bkopilov has joined #openstack-swift | 07:32 | |
openstackgerrit | Christian Schwede proposed openstack/swift-bench: Fix bug when using --log-level option https://review.openstack.org/175285 | 07:36 |
openstackgerrit | Christian Schwede proposed openstack/swift-bench: Fix bug when using --log-level option https://review.openstack.org/175285 | 07:38 |
acoles | notmyname: re patch 174875 , not sure re RC2, probably clayg's call based on whether he believes there is serious enough failure scenario to backport | 07:44 |
patchbot | acoles: https://review.openstack.org/#/c/174875/ | 07:44 |
*** jistr has joined #openstack-swift | 07:47 | |
*** jordanP has joined #openstack-swift | 08:04 | |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Bump PyECLib version from 1.0.3 to 1.0.7 https://review.openstack.org/174225 | 08:23 |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Use reconstruct insetad of decode/encode https://review.openstack.org/175299 | 08:24 |
*** kota_ has joined #openstack-swift | 08:25 | |
*** chlong has joined #openstack-swift | 08:50 | |
*** proteusguy has joined #openstack-swift | 08:52 | |
*** foexle has joined #openstack-swift | 09:13 | |
*** chlong has quit IRC | 09:20 | |
*** pcaruana has quit IRC | 09:27 | |
*** pcaruana has joined #openstack-swift | 09:30 | |
*** chlong has joined #openstack-swift | 09:32 | |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Validates the number of data/parity on EC https://review.openstack.org/175325 | 09:46 |
*** silor has quit IRC | 09:46 | |
openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Validates the number of data/parity on EC https://review.openstack.org/175325 | 09:48 |
*** aix has joined #openstack-swift | 10:01 | |
*** haomaiwa_ has quit IRC | 10:03 | |
*** haomaiwa_ has joined #openstack-swift | 10:04 | |
*** haomaiwa_ has quit IRC | 10:09 | |
*** haomaiwang has joined #openstack-swift | 10:18 | |
straycat | Does anyone know if there's any particular reason this guide http://docs.openstack.org/juno/install-guide/install/apt/content/swift-install-storage-node.html suggests mounting the xfs partitions with nobarrier, from what I've read in the man page this doesn't seem particularly safe | 10:53 |
portante_ | straycat: isn't barriers for when you are working with shared storeage? | 11:00 |
portante_ | Are you sharing disks between swift nodes? | 11:00 |
straycat | portante_, i don't know, i will do some reading, but no i'm not sharing storage devices | 11:04 |
straycat | portante_, ahh no it seems drives can reorder writes held in their cache | 11:06 |
*** silor has joined #openstack-swift | 11:14 | |
straycat | https://lwn.net/Articles/283161/ explains it nicely | 11:20 |
*** astellwa1 has joined #openstack-swift | 11:23 | |
*** astellwa1 has quit IRC | 11:23 | |
*** km_ has quit IRC | 11:29 | |
*** kota_ has quit IRC | 11:35 | |
*** acoles is now known as acoles_away | 11:48 | |
*** kei_yama has quit IRC | 11:53 | |
*** jistr has quit IRC | 11:57 | |
*** chlong has quit IRC | 11:58 | |
*** haomaiwang has quit IRC | 12:01 | |
*** jistr has joined #openstack-swift | 12:03 | |
*** dbay has joined #openstack-swift | 12:10 | |
*** dbay has quit IRC | 12:10 | |
*** acoles_away is now known as acoles | 12:11 | |
portante_ | straycat: you might want to read XFS related info: http://xfs.org/index.php/XFS_FAQ#Q:_How_can_I_address_the_problem_with_the_disk_write_cache.3F | 12:17 |
*** dencaval has joined #openstack-swift | 12:24 | |
*** jamielennox|away is now known as jamielennox | 12:31 | |
*** jamielennox is now known as jamielennox|away | 12:32 | |
*** erlon has joined #openstack-swift | 12:34 | |
*** esker has quit IRC | 12:52 | |
*** bkopilov has quit IRC | 13:27 | |
*** krykowski has joined #openstack-swift | 13:32 | |
*** bkopilov has joined #openstack-swift | 13:35 | |
*** annegentle has joined #openstack-swift | 13:35 | |
*** jistr_ has joined #openstack-swift | 13:35 | |
acoles | tsg: you around? | 13:37 |
*** jistr has quit IRC | 13:38 | |
*** esker has joined #openstack-swift | 13:40 | |
*** bkopilov has quit IRC | 13:40 | |
*** bkopilov has joined #openstack-swift | 13:42 | |
*** jistr_ has quit IRC | 13:44 | |
*** jistr has joined #openstack-swift | 13:44 | |
*** bkopilov has quit IRC | 13:49 | |
*** SkyRocknRoll has quit IRC | 13:52 | |
*** imkarrer has joined #openstack-swift | 14:01 | |
*** Vinsh has joined #openstack-swift | 14:04 | |
*** bkopilov has joined #openstack-swift | 14:04 | |
*** Vinsh has quit IRC | 14:06 | |
*** minwoob_ has joined #openstack-swift | 14:11 | |
*** admin6 has joined #openstack-swift | 14:17 | |
*** Vinsh has joined #openstack-swift | 14:20 | |
*** Vinsh_ has joined #openstack-swift | 14:22 | |
*** pberis has joined #openstack-swift | 14:24 | |
*** Vinsh has quit IRC | 14:25 | |
*** timburke has quit IRC | 14:25 | |
*** timburke has joined #openstack-swift | 14:26 | |
*** Vinsh_ has quit IRC | 14:26 | |
*** winggundamth has joined #openstack-swift | 14:28 | |
winggundamth | hi. anyone know about rsyncd service in Swift? I want to know what exactly it is doing and how port 873 is important to this? | 14:31 |
*** jrichli has joined #openstack-swift | 14:33 | |
ctennis | winggundamth: if object data needs to be copied from one machine to another, rsync is used. The rsyncd daemon listens on port 873 for those connections. | 14:33 |
winggundamth | is it including container and account sync as well? | 14:34 |
ctennis | yes | 14:34 |
winggundamth | if I change rsyncd daemon listen port. Which config that I need to change? | 14:34 |
*** acoles is now known as acoles_away | 14:36 | |
winggundamth | and what happen if no rsyncd run? | 14:37 |
ctennis | I'm not sure where to change the port value, and if it doesn't run then your data will not be able to copy around when you make storage changes or have drive failures. | 14:38 |
winggundamth | so rsync will important when only storage changed like remove or add new storage right? | 14:41 |
winggundamth | not for normal object, container, account operation | 14:42 |
*** mahatic has joined #openstack-swift | 14:43 | |
*** vinsh has joined #openstack-swift | 14:46 | |
*** acoles_away is now known as acoles | 14:47 | |
*** annegentle has quit IRC | 14:57 | |
*** annegentle has joined #openstack-swift | 14:59 | |
*** pberis has quit IRC | 15:02 | |
*** annegentle has quit IRC | 15:04 | |
*** annegentle has joined #openstack-swift | 15:05 | |
*** annegentle has quit IRC | 15:06 | |
*** annegentle has joined #openstack-swift | 15:10 | |
*** jistr has quit IRC | 15:14 | |
jordanP | hi guys. I know that an object can have a forward slash '/' in its name to support "pseudo folder". My question is, is it possible/valid to have two consecutive forward slash in an object name ? | 15:16 |
notmyname | jordanP: yes | 15:16 |
notmyname | jordanP: /v1/AUTH_foo/barcontainer/x/////y <-- totally valid path in swift | 15:17 |
jordanP | notmyname, thanks ! | 15:17 |
winggundamth | just wonder if python -c 'from swift.common.utils import whataremyips; print whataremyips()' return | 15:18 |
winggundamth | ['127.0.0.1', '::1', '10.10.10.1', 'fe80::74c1:1ff:fe0d:521b', '172.17.0.122', 'fe80::42:acff:fe11:7a'] | 15:18 |
*** annegentle has quit IRC | 15:18 | |
winggundamth | which ip that Swift will use? | 15:18 |
notmyname | winggundamth: all of them | 15:21 |
notmyname | winggundamth: https://github.com/openstack/swift/blob/master/swift/common/ring/utils.py#L233 | 15:22 |
*** minwoob_ has quit IRC | 15:22 | |
notmyname | winggundamth: the my_ips parameter is normally the output of whataremyips | 15:22 |
*** krykowski has quit IRC | 15:23 | |
winggundamth | so what is dev_ip and dev_port? is it specify in .conf file? | 15:25 |
notmyname | winggundamth: those would be from the ring | 15:25 |
notmyname | winggundamth: the idea most of the time is in replication to see if the target location (according to the ring) is local or remote | 15:25 |
notmyname | winggundamth: eg don't try to sync to a remote location. like in https://github.com/openstack/swift/blob/master/swift/obj/replicator.py#L467 | 15:27 |
notmyname | winggundamth: the ips parameter there is the output of whataremyips() | 15:27 |
*** foexle has quit IRC | 15:27 | |
notmyname | winggundamth: that method is building a list of stuff that needs to be replicated to some other place in the cluster | 15:27 |
notmyname | winggundamth: so it filters out anything that's "local" | 15:28 |
*** jistr has joined #openstack-swift | 15:28 | |
notmyname | jordanP: I gotta be careful! I say something in IRC and then see it quoted 5 minutes later in an update email from github ;-) | 15:28 |
winggundamth | notmyname: ok. trying to understand what happen if the ring is public ip but node itself is private ip that doing NAT right now | 15:30 |
notmyname | winggundamth: that sounds...unusual | 15:31 |
winggundamth | notmyname: so it will fail on this method right https://github.com/openstack/swift/blob/master/swift/common/ring/utils.py#L233 | 15:31 |
*** lpabon has joined #openstack-swift | 15:31 | |
winggundamth | Yes I know. Trying to understand so I can fix this | 15:32 |
notmyname | acoles: re https://review.openstack.org/#/c/174875/ the idea (as i understand from clayg) is that we probably don't ever want multiple FAs on one drive because that _reduces_ durability (less disperse across failure domains) | 15:32 |
winggundamth | Can I create another private ip interface on each node that can connect together and use that? | 15:33 |
winggundamth | like proxy and nodes having private ip to connect each other and the way to go outside is only proxy | 15:33 |
notmyname | winggundamth: the idea is that all of the storage nodes need to be routable to each other | 15:34 |
jordanP | notmyname, yup ! You are the source of truth :) | 15:34 |
hurricanerix | good morning | 15:34 |
notmyname | winggundamth: and yes, that's very common. the proxy is the only thing that should have "public" access (ie outside the cluster) | 15:34 |
notmyname | hurricanerix: hello! | 15:34 |
winggundamth | hurricanerix: 22:34 here :) | 15:34 |
winggundamth | notmyname: ok. I'll try that way. So I need to config the ring with only private ip right? | 15:35 |
acoles | notmyname: thx i left a comment there earlier, i guess i'm confused why we did all the work to support it | 15:36 |
notmyname | winggundamth: yes. and all of the IPs in it need to be routable to each other (and of course the proxy servers) | 15:36 |
notmyname | acoles: oops? | 15:36 |
notmyname | ;-) | 15:36 |
acoles | notmyname: maybe. and like we never enabled it to ever happen :/ | 15:37 |
notmyname | yeah | 15:37 |
notmyname | clayg was fixing it and then realized the fix would probably be worse than the problem. and the problem probably isn't actually a problem | 15:37 |
notmyname | acoles: re pyeclib 1.0.7. the thing is that the issue is in liberasurecode. BUT, on pypi liberasurecode is bundled with pyeclib. so therefore, bumping pyeclib1.0.7 is what's required. and since you've installed them independently you see the difference | 15:38 |
acoles | notmyname: yup. i think i just about caught up with that today. oh well!! | 15:38 |
acoles | notmyname: so why doesn't pip install pyeclib==1.0.7 also install the latest liberasurecode? | 15:39 |
notmyname | I thought it did | 15:39 |
acoles | not for me | 15:40 |
acoles | it made noise but didn't install, but then i have been very confused today :) | 15:40 |
notmyname | this is why golang's "static link everything into one binary" might be the best thing about the language ;-) | 15:41 |
notmyname | acoles: mostly it's at this point I make disparaging comments about packages and dependencies and ask tsg to tell me what's really going on ;-) | 15:41 |
acoles | notmyname: :) | 15:42 |
notmyname | acoles: ok, now I'm trying to make sense of patches | 15:42 |
acoles | notmyname: i have to leave early, kid stuff, have a good day | 15:43 |
notmyname | acoles: https://review.openstack.org/#/c/173973/ vs https://review.openstack.org/#/c/174138 | 15:43 |
notmyname | acoles: ah, ok | 15:43 |
notmyname | i'll get it sorted later today | 15:43 |
acoles | i hope i haven't confused things! | 15:43 |
notmyname | no worries :-) | 15:43 |
*** acoles is now known as acoles_away | 15:44 | |
* notmyname thinks, "I'll just log in while I drink coffee and check on what's going on." Hours later...."I should probably actually get dressed and go to the office" | 15:44 | |
*** gyee has joined #openstack-swift | 15:45 | |
*** jistr_ has joined #openstack-swift | 15:53 | |
egon | notmyname: that's often my morning. | 15:54 |
notmyname | :-) | 15:55 |
egon | actually, that's my morning right now. | 15:55 |
notmyname | if everyone else wouldn't get up so many hours before me, I wouldn't have this problem! | 15:55 |
notmyname | what's with these "time zones"? | 15:55 |
notmyname | I guess it's my fault for living on the western end of the global time zones ;-) | 15:56 |
*** jistr has quit IRC | 15:56 | |
egon | true, plus, it's not like SF gets up early anyway. | 15:58 |
notmyname | I have small kids. I get up early no matter what SF culture says ;-) | 15:59 |
egon | LOL. truth. | 15:59 |
*** jistr_ is now known as jistr | 16:05 | |
*** minwoob has joined #openstack-swift | 16:07 | |
*** wbhuber has joined #openstack-swift | 16:15 | |
winggundamth | notmyname: thanks for you suggestion. now I can fix my problem :) | 16:16 |
*** winggundamth has quit IRC | 16:16 | |
notmyname | ok, I've talked to ttx and have a good idea on the process needed for RC2 stuff. that's my priority today (ie done byt eh time I go to sleep) | 16:19 |
notmyname | my goal is to have a good list of stuff that needs to be in RC2 on https://wiki.openstack.org/wiki/Swift/PriorityReviews (and landed ASAP, today preferably) | 16:20 |
notmyname | the reguirements change has been approved for pyeclib 1.0.7 in global requirements for master and stable/kilo, so work can progress there | 16:20 |
notmyname | time to get ready for the day and go to the office | 16:21 |
clayg | morning | 16:24 |
*** jistr has quit IRC | 16:24 | |
notmyname | clayg: hi. see above for RC2 patches. acoles_away left a note on the priority reviews page with a question. and the other stuff needs to be tracked there | 16:26 |
clayg | acoles_away: I think we justified the work often because of the percieved need for a disk to be able to hold two indexes without loosing data | 16:26 |
clayg | but I'm not sure if we did anything we didn't need to - fi in datapath tracking suffixes hashes per fi | 16:27 |
clayg | if we ever found we want to use rsync for revert we'll be very glad that having mutli fi's is supported | 16:28 |
notmyname | ok, now I'm really, really going to not be online | 16:29 |
clayg | acoles_away: I had the same issue with liberasurecode - I think something that was supposed to check liberasure version in pyeclib doesn't work - it wasn't an issue when I uninstalled everything and reinstalled newest pyeclib 1.0.7 from scratch | 16:29 |
imkarrer | Good afternoon, I am looking for some help with integrating Ceilometer with Swift. Does anyone have experience setting up the middleware which is responsible for the notifications? Which yum package contains the middleware for swift? I would prefer not to get the source from github as indicated in the Manual. I have set up an SAIO with Ceilometer using the source from github, but I would like to use yum packages instead. | 16:37 |
*** zaitcev has joined #openstack-swift | 16:40 | |
*** ChanServ sets mode: +v zaitcev | 16:40 | |
clayg | zaitcev: ^ | 16:40 |
*** admin6 has left #openstack-swift | 16:41 | |
zaitcev | clayg: Sorry, but I joined too late to see the message you're pointing at. | 16:41 |
imkarrer | clayg: were you pointing to my message or your message to acoles concerning liberasure? | 16:42 |
clayg | zaitcev: imkarrer wasn't to use yum packages on a saio to test out ceilometer | 16:42 |
zaitcev | wasn't? so he has become now? what? | 16:43 |
imkarrer | zaitcev: I am trying to setup ceilometer with swift for production. I would prefer to install it via yum instead of cloning from github. | 16:44 |
zaitcev | You're IRCing from an iPhone today, aren't you :-) | 16:44 |
clayg | zaitcev: yeah that was typo s/wasn't/wants | 16:44 |
clayg | it's just not enough coffee | 16:44 |
zaitcev | imkarrer: Okay, that makes sense. So, what is the issue? It's not one of those permissions thing, is it? | 16:44 |
imkarrer | zaitcev: I figured out the permissions stuff by cloning via git, oh what fun. I am trying to install the source via yum, but there are several ceilometer packages and I have yet to find the one which provides the middleware as an egg. | 16:46 |
zaitcev | The most common problem I see is that Ceilometer has a separate user ID (with a name like "ceilometer"). But its middleware runs in Swift under "swift". So, people run yum install openstack-ceilometer, edit the pipeline, and then everything looks nice, but can't work. | 16:46 |
zaitcev | Wait, what? Install "the source" via yum? | 16:46 |
imkarrer | zaitcev: Maybe I am mis-using terminology. I need to install the middelware via yum and not clone from git. You indicate that openstack-ceilometer should provide the middelware. Will try more with that package. | 16:47 |
zaitcev | hold on, let me look | 16:49 |
zaitcev | Actually, I have no idea about the details of Ceilomoter. I'll need to rely on you doing something like "rpm -ql openstack-ceilometer| grep middleware" | 16:50 |
imkarrer | zaitcev: the command you request returns nothing. The packages I have to choose from are openstack-ceilometer-{alarm, api, central, collector, common, compute} let me try that command on the common package | 16:52 |
*** haomaiwang has joined #openstack-swift | 16:55 | |
imkarrer | zaitcev, openstack-ceilometer-common appears to give me the egg i need. Thanks for the insight. Hitting some distribution not found issues, ill will manually install the missing packages, hopefully there are no version conflicts with swift :D | 16:56 |
imkarrer | zaitcev: I must run to a meeting, thanks again for your help today. | 16:57 |
zaitcev | np | 16:57 |
*** thumpba has joined #openstack-swift | 16:58 | |
*** thumpba has quit IRC | 16:58 | |
openstackgerrit | Clay Gerrard proposed openstack/swift: Use reconstruct insetad of decode/encode https://review.openstack.org/175299 | 17:04 |
*** thumpba has joined #openstack-swift | 17:05 | |
*** thumpba_ has joined #openstack-swift | 17:06 | |
*** thumpba has quit IRC | 17:10 | |
notmyname | good morning again | 17:33 |
*** zhill has joined #openstack-swift | 17:40 | |
*** lcurtis has joined #openstack-swift | 17:46 | |
*** geaaru has quit IRC | 17:48 | |
*** jordanP has quit IRC | 17:50 | |
clayg | notmyname: so I finished going over all of acoles_away notes' on the 409/revert-to-handoff patches - and I think he'd agree we're probably better off doing nothing in RC2 than either of the suggested patches | 18:01 |
*** annegentle has joined #openstack-swift | 18:01 | |
clayg | notmyname: I think there's something I can pull out of patch 175076 that might still be useful - i'm going to work on that | 18:02 |
patchbot | clayg: https://review.openstack.org/#/c/175076/ | 18:02 |
clayg | the fix for revert will probably end up being a combination of acoles_away work and some additional stuff to avoid fragment index duplication when it's reasonable to do so | 18:03 |
clayg | *maybe* for now we could make the early and loud failure mode on revert when the primary node index is unavailable a little more on purpose than a unhnadled KeyError in RC2, but meh. | 18:05 |
clayg | I think with the etag pop in place the only other need-to-have is the pyeclib/reconstruct stuff! | 18:05 |
clayg | acoles_away and I will continue to debate the "ideal error handling when dealing with failures while recovering from failure" on master unless we find something far worse than we have so far | 18:08 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 18:10 |
clayg | the whole thing reminding me of the scene from some star trek episode about the *secondary backup* - found the transcript -> https://gist.githubusercontent.com/clayg/3f84c2563754433b4247/raw/183498fdb2dae9b09a59f5e4c80a4aa80a211438/gistfile1.txt | 18:14 |
torgomatic | belt, suspenders, duct tape | 18:15 |
*** tsg has joined #openstack-swift | 18:17 | |
tsg | notmyname, looks like we are close to master merge for the pyeclib update to 1.0.7 .. will go ahead and submit a review for stable/kilo | 18:21 |
tsg | (unless someone else has already submitted the same - I didn't see one) | 18:22 |
clayg | i'm heading in | 18:22 |
clayg | tsg: i didn't see it on the wiki | 18:22 |
tsg | clayg: ok - will submit the stable/kilo review now | 18:23 |
clayg | tsg: oh wait - yeah I did - https://review.openstack.org/#/c/174171/ | 18:23 |
clayg | oh that's the rquirements repo? | 18:23 |
tsg | yea .. don't have one yet for swift | 18:23 |
tsg | have one only for swift master | 18:23 |
clayg | tsg: is this it? https://review.openstack.org/#/c/174225/3 | 18:24 |
clayg | idk, you'll have to ask notmyname | 18:24 |
tsg | clayg: for whatever reason, my topic got set to "stable/kilo" :) but that one is actually for master | 18:24 |
clayg | tsg: patch 175299 is being tracked on the prioirty reviews page (https://wiki.openstack.org/wiki/Swift/PriorityReviews) | 18:25 |
patchbot | clayg: https://review.openstack.org/#/c/175299/ | 18:25 |
clayg | so as long as that gets merged to stable/kilo it should al workout | 18:25 |
tsg | yep. clayg: https://review.openstack.org/#/c/175525/ | 18:26 |
imkarrer | zaitcev: So installing the ceilometer middleware from openstack-ceilometer-common (Juno) leads to package resolution and conflict issues. First it cannot find pysnmp. After installing that there is conflict with oslo.rootwrap. After upgrading rootwrap, no matter what i do (uninstall, rm -rf, pip install, easy_install), the module six cannot be found. Do you know anyone with experience setting up Ceilometer in production for Juno? | 18:32 |
zaitcev | imkarrer: This just makes no sense. And once you're deep enough to use rm(1), what's the point of using yum to begin with? | 18:33 |
*** mahatic has quit IRC | 18:33 | |
zaitcev | imkarrer: Ending with "package resolution and conflict issues" is bizzarre. | 18:35 |
imkarrer | zaitcev: All of our other packages are installed via yum. I figure it should be trivial to install the ceilometer middleware in the same manner the keystone middleware is installed. | 18:35 |
*** mahatic has joined #openstack-swift | 18:35 | |
zaitcev | imkarrer: well, yeah... What's the OS? | 18:36 |
imkarrer | zaitcev: RHEL6.5 | 18:36 |
zaitcev | imkarrer: okay, I have that. Let's see | 18:36 |
zaitcev | go go gadget ssh | 18:37 |
notmyname | tsg: thanks for the requirements stuff. I talked to ttx this morning and got it taken care of upstream. so yeah our next step is doing it on stable/kilo and master in swift | 18:37 |
zaitcev | [root@rhev-a24c-01 ~]# cat /etc/redhat-release | 18:37 |
zaitcev | Red Hat Enterprise Linux Server release 6.6 (Santiago) | 18:37 |
imkarrer | [root@wdc01sft003ccz020 ~]# cat /etc/redhat-release | 18:38 |
imkarrer | Red Hat Enterprise Linux Server release 6.5 (Santiago) | 18:38 |
notmyname | clayg: looks like you updated priority reveiws page | 18:38 |
notmyname | clayg: the DO is the one we need in RC2 and the DO NOT is the one we don't? | 18:38 |
zaitcev | so, running "yum install openstack-ceilometer-common openstack-ceilometer-api python-ceilometerclient" | 18:39 |
*** mmcardle has joined #openstack-swift | 18:39 | |
zaitcev | okay, so installation had no issues... The middleware is actually in the shared code | 18:44 |
zaitcev | [root@rhev-a24c-01 ~]# rpm -qf /usr/lib/python2.6/site-packages/ceilometer/objectstore/swift_middleware.py | 18:44 |
zaitcev | python-ceilometer-2014.1.2-1.el6.noarch | 18:44 |
zaitcev | now to configure | 18:44 |
*** MVenesio has joined #openstack-swift | 18:45 | |
imkarrer | So the middleware comes from python-ceilometer? Okay, thats probably much more lightweight than ceilometer-common, less packages to confict with... | 18:47 |
zaitcev | You're thinking about it all wrong. There should be no conflicts period | 18:49 |
zaitcev | Don't even go there | 18:49 |
zaitcev | The python-ceilometer is installed automatically as a dependency. And if you use the corrent repositories for the basic distro, there will be no conflicts of any kind. It's all tested together by the QA. | 18:50 |
zaitcev | Anyway, since I never installed Ceilometer before, I have no idea what I'm doing here | 18:51 |
notmyname | zackmdavis wrote an awesome blog post on how erasure codes work: https://swiftstack.com/blog/2015/04/20/the-foundations-of-erasure-codes/ | 18:51 |
imkarrer | zaitcev: python-ceilometer gets installed as a dependency of swift? | 18:51 |
zaitcev | imkarrer: no, openstack-ceilometer pulls it | 18:52 |
imkarrer | zaitcev: I am more confused now. I do not have access in my repo list openstack-ceilometer package, only openstack-ceilometer-common. | 18:53 |
*** pberis has joined #openstack-swift | 18:55 | |
zaitcev | imkarrer: sorry, I was imprecise. The openstack-ceilometer is built as a bunch of subpackages, which include -common, -api, etc. I presume you'll need the -api one for sure, because it contains the REST server. | 18:55 |
zaitcev | Ah-ha, here's the first problem - oslo.config.cfg.ConfigFilesNotFoundError: Failed to read some config files: /usr/share/ceilometer/ceilometer-dist.conf,/etc/ceilometer/ceilometer.conf | 18:57 |
imkarrer | zaitcev: I already have ceilometer api running on a separate node. I am trying to configure the ceilometer middleware so that I can see the three notifications which swift sends back to ceilometer. The other ceilometer statistics (pollsters) are trivial. | 18:58 |
zaitcev | imkarrer: right, so I guess only openstack-ceilometer-common is needed and then some kind of agent or whatnot. | 18:59 |
zaitcev | imkarrer: I added swift to ceilometer group for now. ceilometer:x:166:ceilometer,swift | 18:59 |
zaitcev | imkarrer: also had to do chgrp ceilometer /var/log/ceilometer && chmod 775 /var/log/ceilometer | 19:01 |
imkarrer | zaitcev: so what yum repos have you install thus far? using the openstack-ceilometer-common led to package confict issues for me. | 19:02 |
zaitcev | oh, great - Apr 20 15:02:27 rhev-a24c-01 ceilometer.openstack.common.rpc.common: AMQP server on localhost:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 9 seconds. | 19:02 |
imkarrer | So you are trying to hit the rabbitmq :D | 19:03 |
imkarrer | I was able to get that far in my dev environment, but not with the yum packages | 19:03 |
zaitcev | imkarrer: http://paste.openstack.org/show/204911/ | 19:04 |
*** mmcardle has quit IRC | 19:04 | |
*** silor1 has joined #openstack-swift | 19:05 | |
zaitcev | imkarrer: no, wait, that's useless. all the fake repo names | 19:05 |
imkarrer | zaitcev: I mean packages | 19:06 |
*** silor has quit IRC | 19:06 | |
*** pberis has quit IRC | 19:07 | |
*** pberis has joined #openstack-swift | 19:08 | |
zaitcev | imkarrer: here is "yum repolist -v": http://paste.openstack.org/show/204912/ There's a lot of historic junk there, but really what you need is 1) RHEL 6, 2) EPEL, 3) RDO. Together they should never cause any conflicts. | 19:08 |
imkarrer | zaitcev: but what package did you install to get the ceilometer middleware? | 19:10 |
zaitcev | imkarrer: I just did " 1125 yum search ceilometer | 19:12 |
zaitcev | 1126 yum install openstack-ceilometer-common openstack-ceilometer-api python-ceilometerclient" | 19:12 |
zaitcev | imkarrer: result is like http://paste.openstack.org/show/204913/ | 19:13 |
imkarrer | Thank you for all of your help today zaitcev, I really appreciate another set of eye on this. | 19:13 |
imkarrer | eyes* :D | 19:13 |
zaitcev | The eagle-eyed among us might notice that I have openstack-swift-2.2.2-1.el6.noarch, which is not officially available (yet). It's a backport that I built from a Fedora RPM. However, I'm pretty sure that ceilometer middleware ought to work with the official Swift 1.13.1. | 19:14 |
*** tsg has quit IRC | 19:14 | |
zaitcev | Honestly, I have no clue what I'm doing with Ceilometer. But all OpenStack work about the same way... There's a REST front-end, a scheduler or some deamonistic thing, maybe a service daemon or two, and a bunch of agents. | 19:16 |
*** krtaylor has left #openstack-swift | 19:16 | |
zaitcev | The RPMs for them typically are built from one SRPM. That thing has a bunch of shared code, that ends in something like python-manila. Every derived subpackage depends on that and only contains things like sample configs, docs, and maybe binaries. | 19:17 |
zaitcev | Swift is (of course, haha) different in that there's no python-swift RPM. We keep all our common stuff in openstack-swift. | 19:18 |
*** pberis has quit IRC | 19:21 | |
*** pberis has joined #openstack-swift | 19:22 | |
*** stevemar has joined #openstack-swift | 19:22 | |
stevemar | whats up with notifications and swift? seems like it doesn't use oslo messaging | 19:23 |
*** pberis has quit IRC | 19:25 | |
*** pberis has joined #openstack-swift | 19:26 | |
*** Fin1te has joined #openstack-swift | 19:26 | |
zaitcev | notifications about what? | 19:27 |
*** minwoob has quit IRC | 19:29 | |
*** aix has quit IRC | 19:29 | |
*** imkarrer__ has joined #openstack-swift | 19:31 | |
stevemar | zaitcev, event notifications, and use auth_token and audit middleware | 19:32 |
*** wbhuber has quit IRC | 19:33 | |
stevemar | but i guess cause it's still used as a standalone, so it's not using oslo messaging and other keystone bits | 19:33 |
*** imkarrer has quit IRC | 19:34 | |
cschwede | stevemar: there are no event notifications in swift? | 19:35 |
*** annegentle has quit IRC | 19:36 | |
*** bill_az has joined #openstack-swift | 19:36 | |
stevemar | cschwede, well you can add keystonemiddleware's audit code to nova/cinder/etc, and it'll send off a notifications using oslo messaging | 19:38 |
stevemar | i think for all api requests that are in the mapping configuration (in pycadf) | 19:39 |
*** pberis has quit IRC | 19:43 | |
*** pberis has joined #openstack-swift | 19:44 | |
*** pberis has quit IRC | 19:45 | |
*** pberis has joined #openstack-swift | 19:46 | |
*** lpabon has quit IRC | 19:48 | |
*** pberis has quit IRC | 19:50 | |
clayg | notmyname: I was trying to highlight there's some contraversary between patch 174875 and patch 175076 | 19:56 |
patchbot | clayg: https://review.openstack.org/#/c/174875/ | 19:56 |
*** dencaval has quit IRC | 19:56 | |
clayg | notmyname: we don't either for rc2 - or said another way - we need some combination of both | 19:56 |
*** tsg has joined #openstack-swift | 19:56 | |
clayg | notmyname: I'm going to try and pull out the non-contraversial fixes in patch 175076 and resubmit | 19:57 |
patchbot | clayg: https://review.openstack.org/#/c/175076/ | 19:57 |
clayg | notmyname: depending on how that goes I may try and combine some sane combination of patch 174875 with what's left | 19:57 |
patchbot | clayg: https://review.openstack.org/#/c/174875/ | 19:57 |
clayg | notmyname: if I was trying to manage all these balls up in the air I'd focus on pyeclib 1.0.7 and the reconstruct patch - and leave the others alone - i'll catch you up with the issues acoles identified with my revert fix next time we're high bandwidth | 19:58 |
*** silor1 has quit IRC | 20:08 | |
*** tsg has quit IRC | 20:12 | |
*** pberis has joined #openstack-swift | 20:19 | |
*** tsg has joined #openstack-swift | 20:28 | |
*** imkarrer_ has joined #openstack-swift | 20:34 | |
*** tsg has quit IRC | 20:35 | |
*** imkarrer__ has quit IRC | 20:38 | |
openstackgerrit | Samuel Merritt proposed openstack/swift: Make RingBuilders deep-copy-able https://review.openstack.org/175567 | 20:39 |
openstackgerrit | Clay Gerrard proposed openstack/swift: Exclude local_dev from sync partners on failure https://review.openstack.org/175076 | 20:47 |
*** proteusguy has quit IRC | 20:52 | |
*** annegent_ has joined #openstack-swift | 20:55 | |
*** stevemar has quit IRC | 20:58 | |
*** mahatic has quit IRC | 21:07 | |
*** mmcardle has joined #openstack-swift | 21:29 | |
*** mmcardle has quit IRC | 21:29 | |
*** Fin1te has quit IRC | 21:33 | |
*** Guest46134 has joined #openstack-swift | 21:35 | |
*** Guest46134 is now known as _alejandrito_ | 21:39 | |
_alejandrito_ | hi everyone, is there a chance to make a container PUBLIC WRITABLE on swift JUNO / KILO without using tempURL ??? | 21:41 |
*** erlon has quit IRC | 21:41 | |
swifterdarrell | _alejandrito_: it actually surprised a couple of us, but you can set a container write ACL to ".r:*" to allow public writes | 21:42 |
wer | hrm, I had to add that ability. Last time I checked that wasn't a supported permission. But I'm running an old version of swift now. | 21:42 |
clayg | swifterdarrell: so we have a doc bug? "Also, .r designations aren’t allowed in headers whose names include the word ‘write’." | 21:43 |
swifterdarrell | _alejandrito_: I mean, your auth middleware may or may not like it, but it's worked at least once with some auth middleware, YMMV, etc etc | 21:44 |
swifterdarrell | clayg: yes? | 21:44 |
_alejandrito_ | swifterdarrell, clayg so ... i didnt found any documentation about doing a container writable | 21:44 |
clayg | Container POST failed: http://saio:8080/v1/AUTH_test/test 400 Bad Request Referrers not allowed in write ACL: '.r:*' | 21:44 |
swifterdarrell | clayg: not 100% sure what swift version klrmn was using and/or whether tempauth or something that filters .r for write ACLs wasn't active... | 21:45 |
swifterdarrell | _alejandrito_: sounds like it may not work to set a ".r:*" write ACL...but do try it and see | 21:46 |
_alejandrito_ | swifterdarrell, clayg wow ok, so i can try it in juno NOW, you say it might work ? | 21:46 |
swifterdarrell | clayg: ya, that's disallowed in swift.common.middleware.acl.clean_acl() | 21:47 |
swifterdarrell | _alejandrito_: sure, try it and it might work, but it looks like at least some code tries to make it NOT work, so... who knows?! | 21:48 |
swifterdarrell | _alejandrito_: your own experimentation will trump what anyone says, in any case :) | 21:48 |
_alejandrito_ | swifterdarrell, the header would be X-Container-Write ? | 21:48 |
*** tsg has joined #openstack-swift | 21:49 | |
*** MVenesio_ has joined #openstack-swift | 21:51 | |
*** MVenesio has quit IRC | 21:54 | |
*** kota_ has joined #openstack-swift | 21:54 | |
kota_ | good morning | 21:55 |
_alejandrito_ | swifterdarrell, clayg Referrers not allowed in write ACL: '.r:*' | 21:59 |
swifterdarrell | _alejandrito_: bummer :( | 21:59 |
swifterdarrell | _alejandrito_: I checked and our auth middleware was changed to allow it to let swift3 middleware support a canned ACL of public-read-write on a "bucket" | 22:00 |
mattoliverau | Morning | 22:02 |
_alejandrito_ | swifterdarrell, clayg so, it was an in-house mod you did ? | 22:02 |
_alejandrito_ | swifterdarrell, can you share it to me ? its exactly what i need | 22:02 |
_alejandrito_ | swifterdarrell, ;D | 22:02 |
swifterdarrell | _alejandrito_: no, but the idea is to just modify swift.common.middleware.acl.check_acl() to not disallow it | 22:03 |
notmyname | kota_: mattoliverau: hello | 22:03 |
kota_ | nomyname, mattoliverau: hi :) | 22:04 |
_alejandrito_ | swifterdarrell, swift.common.middleware.acl.check_acl() doesnt exists on juno | 22:06 |
swifterdarrell | _alejandrito_: is "def check_acl" somewhere else in the code tree? | 22:07 |
swifterdarrell | _alejandrito_: maybe it moved or something | 22:07 |
_alejandrito_ | swifterdarrell, nope, it doesnt exists in junor nor kilo | 22:08 |
openstackgerrit | Samuel Merritt proposed openstack/swift-specs: Symlinks in Swift https://review.openstack.org/173609 | 22:09 |
swifterdarrell | _alejandrito_: Perhaps search for some portion of the response string you get? | 22:10 |
swifterdarrell | _alejandrito_: e.g. "grep -r 'not allowed in write' ." | 22:10 |
swifterdarrell | _alejandrito_: surely, somewhere, there is code that says "not allowed in write" | 22:10 |
notmyname | swifterdarrell: _alejandrito_: clean_acl, not check_acl | 22:10 |
notmyname | right? | 22:10 |
swifterdarrell | _alejandrito_: ^^^^^^ | 22:10 |
swifterdarrell | _alejandrito_: ya, I misspoke? | 22:11 |
swifterdarrell | _alejandrito_: grep is 100% better than me | 22:11 |
swifterdarrell | notmyname: thx! | 22:11 |
notmyname | kota_: do you mind if I propose https://review.openstack.org/#/c/175299/ to stable/kilo for RC2 and manage that (instead of you doing it)? | 22:11 |
kota_ | notmyname: yes, right now, I'm looking at acoles question | 22:12 |
notmyname | kota_: ok. yeah if there are oustanding questions, I'd like to see them resolved first | 22:13 |
kota_ | notmyname: clayg already pushed the unittest but the unittest passed with v1.0.6 :P | 22:13 |
_alejandrito_ | notmyname, so if i want for clean_acl to allow me to set PUBLIC WRITE ACL show i remove this ones ? https://github.com/openstack/swift/blob/stable/juno/swift/common/middleware/acl.py#L101-L103 | 22:13 |
notmyname | _alejandrito_: yeah, looks like it | 22:13 |
kota_ | notmyname: and I found the change on the unittest we need (maybe) and I'll push it ASAP | 22:13 |
notmyname | kota_: ok, thanks | 22:13 |
_alejandrito_ | notmyname, swifterdarrell commenting out this 3 lines https://github.com/openstack/swift/blob/stable/juno/swift/common/middleware/acl.py#L101-L103 worked like a charm | 22:16 |
swifterdarrell | _alejandrito_: sweet! | 22:16 |
*** imkarrer_ has quit IRC | 22:20 | |
notmyname | RC2 patch list updated on https://wiki.openstack.org/wiki/Swift/PriorityReviews | 22:21 |
notmyname | https://review.openstack.org/#/c/175299/ needs to land on master, then be proposed to stable/kilo | 22:21 |
egon | Anyone here looked at the EMC Vipr thing? | 22:22 |
notmyname | then the 4 patches listed (the 3 on the wiki page plus the one that needs to be proposed) can land on stable/kilo | 22:22 |
notmyname | I'll need to update the commit messages for them to add a LP bug reference (Which I'll also need to create for each of them) | 22:22 |
notmyname | summary is, review https://review.openstack.org/#/c/175299/ and land it on master, and then i'll take care of the button clicking for backports | 22:23 |
notmyname | also, please ping me ASAP if there is something else that needs to be in RC2 for the release | 22:23 |
notmyname | egon: if you've got access to one, I'd love to see what happens when you run the swift functional tests against it | 22:24 |
*** tsg has quit IRC | 22:25 | |
kota_ | oops, my assumption seemed wrong, now confuesed as same as acoles. now thinking... | 22:26 |
openstackgerrit | Minwoo Bae proposed openstack/swift: Included step in development_saio.rst for installing dependencies in requirements.txt. https://review.openstack.org/175594 | 22:28 |
dfg | q | 22:31 |
dfg | oops | 22:31 |
kota_ | notmyname: ok, I confirm clayg's patch is good (the test pass was my mistake to install liberasurecode) so I think the newest patch set is ready to land on path 175299 | 22:38 |
kota_ | s/path/patch/ | 22:38 |
*** jrichli has quit IRC | 22:41 | |
*** _alejandrito_ has quit IRC | 22:46 | |
*** pberis has quit IRC | 22:47 | |
*** km has joined #openstack-swift | 22:50 | |
kota_ | I'm leaving a while for breakfast. | 22:52 |
*** kota_ has quit IRC | 22:52 | |
notmyname | kota_: I'm about to look at that. thanks for pushing another copy | 22:52 |
notmyname | * patch set | 22:52 |
*** annegent_ has quit IRC | 22:57 | |
notmyname | clayg: did you write the tests on https://review.openstack.org/#/c/175299 ? | 22:57 |
notmyname | hmm...so my turn now | 23:08 |
notmyname | what's the right way to upgrade liberasurecode once you've had it installed via pyeclib before? | 23:09 |
*** lcurtis has quit IRC | 23:22 | |
*** esker has quit IRC | 23:27 | |
*** chlong has joined #openstack-swift | 23:34 | |
clayg | notmyname: uninstall pyeclib and delete /usr/local/lib/liberasure.* | 23:35 |
clayg | then reinstall pyeclib | 23:35 |
clayg | notmyname: ^ WOMM | 23:35 |
clayg | notmyname: yeah i wrote the test - what of it! | 23:35 |
notmyname | yeah, that's what I did too. just wondered if there was a better day | 23:35 |
*** kei_yama has joined #openstack-swift | 23:35 | |
notmyname | clayg: you've got some magic numbers in there. wondering what they are. also, maybe should have a co-authored-by line | 23:36 |
clayg | notmyname: i don't want anyone to pin that cappy code on me - much better in my mind if the blame kota! | 23:36 |
*** MVenesio_ has quit IRC | 23:36 | |
notmyname | lol | 23:36 |
clayg | yeah I think the test name sorta hints at the magic - reconstruct *parity* with *data* node down | 23:37 |
clayg | so the broken_body is a parity index (because the schema is 10+4) and the error_index is the down'd data node (randomint 0, ec_ndata-1) | 23:38 |
notmyname | clayg: what about [:-454] | 23:40 |
clayg | oh i just try not to make it easy for fragment or segment size to align with the client data | 23:41 |
clayg | I started doing that in all my tests - it's just triming some random number of bytes | 23:41 |
clayg | the test I copied was doing 777 I think | 23:41 |
notmyname | oh | 23:41 |
notmyname | mostly it was thet (the -454) that was weird to me. the pop(-4) makes sense now | 23:43 |
notmyname | thanks | 23:43 |
clayg | notmyname: might could all use comments - let me know if there's a good time to add them - or if kota can do it | 23:43 |
clayg | notmyname: more than likely after it gets merged no one will look at that test again unless it fails :\ | 23:44 |
*** david-ly_ has joined #openstack-swift | 23:44 | |
clayg | notmyname: that's how good tests work right?! | 23:44 |
*** jamielennox|away is now known as jamielennox | 23:45 | |
*** tsg has joined #openstack-swift | 23:45 | |
*** david-ly_ is now known as david-lyle | 23:46 | |
openstackgerrit | Samuel Merritt proposed openstack/swift: SAIO instructions: ensure ~/bin exists before copying into it https://review.openstack.org/175613 | 23:47 |
openstackgerrit | John Dickinson proposed openstack/swift: Use reconstruct insetad of decode/encode https://review.openstack.org/175299 | 23:49 |
notmyname | clayg: I added comments ^ | 23:49 |
*** annegentle has joined #openstack-swift | 23:57 | |
*** openstackgerrit has quit IRC | 23:58 | |
*** openstackgerrit has joined #openstack-swift | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!