| notmyname | swift-bench --saio fails spectacularly | 00:01 |
|---|---|---|
| *** km has joined #openstack-swift | 00:03 | |
| *** km__ has quit IRC | 00:03 | |
| *** km__ has joined #openstack-swift | 00:05 | |
| *** km has quit IRC | 00:07 | |
| notmyname | fucntional test results: | 00:08 |
| *** breitz has quit IRC | 00:08 | |
| notmyname | Ran 340 tests in 117.440s | 00:08 |
| notmyname | FAILED (SKIP=12, errors=4, failures=8) | 00:08 |
| *** breitz has joined #openstack-swift | 00:08 | |
| notmyname | looks like some container listing stuff, some utf8 stuff, and some cross-policy stuff | 00:09 |
| notmyname | ah, looks like my original swift-bench issue was because I had an EC policy as default | 00:10 |
| clayg | notmyname: the docs *said* not to use storage policies | 00:11 |
| notmyname | :-) | 00:11 |
| swifterdarrell | who reads docs? | 00:11 |
| clayg | swifterdarrell: not swift devs :\ | 00:11 |
| swifterdarrell | clayg: touché | 00:12 |
| notmyname | swift-init <server> <action> vs hummingbird <action> <server> | 00:14 |
| clayg | swift-init can do *either* | 00:14 |
| notmyname | oh really? TIL | 00:14 |
| notmyname | why would I try something new if the old way works ;-) | 00:15 |
| clayg | notmyname: for sure | 00:15 |
| clayg | i just kept typing it backwards and both ways - so i made the code read my mind | 00:15 |
| notmyname | so my initial tests on my virtualized SAIO (so TOTALLY exactly like production) shows that hummingbird is faster | 00:16 |
| notmyname | https://gist.github.com/notmyname/5219bb5f9120846cd68d | 00:16 |
| clayg | heh | 00:16 |
| clayg | i got about 100% on PUT | 00:16 |
| notmyname | ok, gotta run. wife calling, kids crazy, etc | 00:16 |
| clayg | yeah so I got to trace some code through hummingbird/main.go - grep found the definition in common/utils - and code explained what I was doing wrong - this will totally work | 00:18 |
| clayg | redbo: I wish you didn't dump the whole thing out at once, it's going to take a long while to digest 8K lines | 00:20 |
| clayg | redbo: like the objectserver has a REPLICATE verb? | 00:22 |
| *** gyee has quit IRC | 00:22 | |
| clayg | oh, that's just the hashes | 00:23 |
| clayg | oh it has SYNC | 00:23 |
| clayg | ok, i'm quitting out for this afternoon - thanks for the code dump redbo! | 00:28 |
| *** dmorita has joined #openstack-swift | 00:29 | |
| clayg | dfg_: hurricanerix: scotticus: great work! | 00:29 |
| *** tsg has quit IRC | 00:30 | |
| *** shri has left #openstack-swift | 01:03 | |
| *** kota_ has joined #openstack-swift | 01:23 | |
| kota_ | morning, again. | 01:23 |
| *** asettle is now known as asettle-gym | 01:38 | |
| mattoliverau | kota_: morning (again) | 01:45 |
| *** tsg has joined #openstack-swift | 01:50 | |
| kota_ | mattoliverau: :) | 01:53 |
| *** tsg has quit IRC | 02:05 | |
| *** tsg has joined #openstack-swift | 02:25 | |
| *** jrichli has joined #openstack-swift | 02:26 | |
| *** bkopilov has quit IRC | 02:36 | |
| *** fifieldt has joined #openstack-swift | 02:36 | |
| *** ir2ivps8_ has quit IRC | 02:48 | |
| *** ir2ivps8 has joined #openstack-swift | 02:48 | |
| jrichli | notmyname: I created an etherpad for the encryption fishbowl session. Should we use that one for the working session also? | 02:57 |
| *** asettle-gym is now known as asettle | 02:58 | |
| *** erlon has quit IRC | 03:11 | |
| *** jrichli has quit IRC | 03:29 | |
| *** tsg has quit IRC | 03:34 | |
| *** gvernik has joined #openstack-swift | 03:36 | |
| *** jrichli has joined #openstack-swift | 03:41 | |
| *** ho has quit IRC | 03:42 | |
| jrichli | notmyname: nm. I think I want a separate etherpad for them both now that I think about it. | 03:42 |
| notmyname | jrichli: glad to help ;-) | 03:44 |
| jrichli | :-) | 03:44 |
| *** thumpba has joined #openstack-swift | 03:46 | |
| *** jrichli has quit IRC | 03:53 | |
| *** tsg has joined #openstack-swift | 03:57 | |
| *** bkopilov has joined #openstack-swift | 04:25 | |
| *** zhill has joined #openstack-swift | 04:38 | |
| *** zhill has quit IRC | 04:38 | |
| *** gvernik has quit IRC | 04:44 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Remove confusable query string on post as copy https://review.openstack.org/178927 | 04:52 |
| *** zhill has joined #openstack-swift | 04:54 | |
| *** zaitcev has quit IRC | 04:57 | |
| *** ppai has joined #openstack-swift | 05:19 | |
| *** openstackgerrit has quit IRC | 05:22 | |
| *** openstackgerrit has joined #openstack-swift | 05:22 | |
| *** esker has quit IRC | 05:28 | |
| *** esker has joined #openstack-swift | 05:33 | |
| *** zhill has quit IRC | 05:36 | |
| mattoliverau | sharding update: Don't want to jinx anything.. But sharing in the POC seems to be sharding. Now hopefully I've correctly plumbed the replicators to replicate the shard nodes table in the DB.. Think I need a break before the next round of debugging :p | 05:38 |
| mattoliverau | Soon comes testing large containers and tuning/fixing bottlenecks | 05:39 |
| *** silor has joined #openstack-swift | 05:42 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Remove confusable query string on post as copy https://review.openstack.org/178927 | 05:58 |
| *** joeljwright has left #openstack-swift | 06:43 | |
| *** zul has joined #openstack-swift | 06:53 | |
| *** tsg has quit IRC | 07:02 | |
| *** tsg_ has joined #openstack-swift | 07:05 | |
| *** zul has quit IRC | 07:12 | |
| *** zul has joined #openstack-swift | 07:12 | |
| *** thumpba has quit IRC | 07:13 | |
| *** thumpba has joined #openstack-swift | 07:14 | |
| *** thumpba has quit IRC | 07:18 | |
| *** geaaru has joined #openstack-swift | 07:20 | |
| *** jordanP has joined #openstack-swift | 07:22 | |
| *** tsg_ has quit IRC | 07:33 | |
| *** chlong has quit IRC | 07:42 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Remove confusable query string on post as copy https://review.openstack.org/178927 | 07:44 |
| *** geaaru has quit IRC | 07:52 | |
| kota_ | hi, matthow, still here? | 07:57 |
| kota_ | no response but that's okay I'll ask someone at aonther timing. | 07:59 |
| *** jistr has joined #openstack-swift | 08:04 | |
| mattoliverau | kota_ what's up? | 08:07 |
| kota_ | mattoliverau: ah, nice | 08:11 |
| kota_ | mattoiverau: jast for confirmation, now I'm looking at EC code and I'm wondering the behavior of reconstructor. | 08:11 |
| kota_ | mattoiverau: current reconstructor seems to select an avaialble handoff nodes from primary nodes, does it correct? | 08:12 |
| kota_ | mattoliverau: AFAIK, replicator will pick up the handoff nodes from get_more_nodes (i.e. different nodes from primaries) | 08:13 |
| kota_ | mattoliverau: Do you know the reason for the EC behavior? | 08:14 |
| kota_ | mattoliverau: on the case partner nodes unmounted the disk. | 08:15 |
| mattoliverau | Yup, when replicating we want to make sure we have the right number of replicas out there for data retention purposes | 08:15 |
| mattoliverau | The EC recomstructor can only rebuild using EC parts (to recreate missing bits) | 08:16 |
| mattoliverau | I'd need to look closely, but if a primary is missing on reconstruction (primary down, then it'll use a hand off node).. I hope (assume) | 08:17 |
| mattoliverau | Not in front of a computer at the moment but will confirm when I am :) | 08:18 |
| kota_ | mattoliverau: hah | 08:19 |
| kota_ | mattoliverau: thanks for going to look at :) | 08:19 |
| kota_ | mattoliverau: I also test the behavior more, and I think we should use the handoff nodes in the reconstruction case too, right? | 08:20 |
| kota_ | mattoliverau: for data retention and some reason like that.... (i.e. one disk have to keep one fragment archive) | 08:21 |
| mattoliverau | Yeah, I'd hope so.. When a primary node can't be contacted. | 08:22 |
| kota_ | mattoliverau: ok thanks, I'm happy to hear your opinion ;) | 08:23 |
| *** krykowski has joined #openstack-swift | 08:30 | |
| *** early has quit IRC | 08:31 | |
| *** geaaru has joined #openstack-swift | 08:32 | |
| *** zul has quit IRC | 08:38 | |
| *** acoles_away is now known as acoles | 08:38 | |
| acoles | kota_: hi | 08:42 |
| kota_ | acoles: hi | 08:42 |
| acoles | there are two kinds of reconstructor jobs - 'sync' and 'revert'... | 08:44 |
| kota_ | acoles: like as update and update_deleted on replicator | 08:44 |
| mattoliverau | Yay its acoles! | 08:44 |
| acoles | kota_: a sync job picks partner primary nodes and attempts to rebuild any fragment that is missing on the partner | 08:44 |
| acoles | mattoliverau: hi! | 08:45 |
| acoles | kota_: yes similar | 08:46 |
| acoles | a 'revert' job attempts to move fragments to their 'correct' primary node | 08:46 |
| acoles | where the fragment exists but is on the wrong node | 08:46 |
| acoles | a 'revert' will first try the correct primary for the frag index, then try another handoff that may be a better handoff | 08:47 |
| kota_ | acoles: yes, _revert calls get_more_nodes for destination. | 08:48 |
| acoles | kota_: but at the moment the reverts to a handoff don't work - there is a KeyError raised in reconstructor because get_more_nodes don't have an 'index' key | 08:48 |
| acoles | kota_: there's a patch to fix that, one min... | 08:48 |
| *** early has joined #openstack-swift | 08:49 | |
| acoles | kota_: https://review.openstack.org/#/c/176403/ | 08:49 |
| kota_ | acoles: oh..I've seen the fix, perhaps. | 08:49 |
| kota_ | acoles: nice | 08:50 |
| kota_ | acoles: reading... | 08:51 |
| acoles | kota_: right now clayg and i are uncertain how to proceed on that - its possible that if you revert to another handoff you can end up with two copies of the same frag on two handoffs - the probe test in that patch shows that happening | 08:51 |
| acoles | kota_: so basically there is some more work required1 | 08:52 |
| acoles | s/1/!/ !) | 08:52 |
| kota_ | I see... | 08:53 |
| kota_ | acoles: for the summary, we are now working to achieve that reconstructor places the frag to the handoff with revert | 08:55 |
| kota_ | acoles: right? | 08:55 |
| acoles | kota_: yes...but...:) ...there's another 'open question' which is do we want to revert a fragment to another handoff if that handoff already has another fragment for same object ie do we want to deliberately have t0#1.data and t0#2.data on same handoff? | 08:56 |
| acoles | kota_: i *think* clayg has opinion that that is not good from a durability point of view | 08:57 |
| acoles | kota_: and i *think* i agree with clayg ;) | 08:57 |
| acoles | kota_: so my current thinking is to work towards being able to revert a fragment from one handoff to a better handoff IFF the better handoff does not already have another fragment of same object, and IFF it can be done without accidentally ending up with two copies of same fragment. phew! | 08:59 |
| *** km__ has quit IRC | 08:59 | |
| acoles | kota_: the second part (IFF it can be done without accidentally ending up with two copies of same fragment) needs some changes to ssync which i have in my head but not on gerrit :P | 08:59 |
| mattoliverau | Can we review and sent patches to acoles head ( the new gerrit ) :p | 09:01 |
| kota_ | acoles: yeah, i *think* i agree with you we shouldn't allow the handoff nodes to have more than one fragment of the same object. | 09:01 |
| acoles | mattoliverau: i think that could prove even less reliable than the real gerrit :P | 09:02 |
| mattoliverau | Lol | 09:03 |
| mattoliverau | FI on any handoff node (even more then one) is better then on no handoff nodes.. Though yes, better to keep FI on handoffs seperate | 09:04 |
| acoles | kota_: yeah, so i see two reasons to avoid that - (1) right now the proxy won't read both frags from one node (i think) (2) if one fragment has bit rot the auditor will quarantine them both :/ | 09:04 |
| mattoliverau | FI == FA :p | 09:04 |
| kota_ | acoles: right, maybe my collegue hit the behavior. | 09:05 |
| acoles | mattoliverau: oh, yes, agree. question is whether the reconstructor chooses to move an FA to cohabit with another FA of same object | 09:05 |
| kota_ | acoles: I'm not sure but he reported GET object failed with in the PyECLib layer said "no enough fragments to decode" | 09:06 |
| acoles | mattoliverau: heh, 'cohabiting' FAs sounds real friendly | 09:07 |
| kota_ | acoles: I didn't done to understand whole codes but it seems to allow duplicated fragments like as | 09:07 |
| acoles | kota_: interesting | 09:07 |
| mattoliverau | Lol | 09:07 |
| acoles | mattoliverau: but the have a death pact :/ | 09:07 |
| acoles | they | 09:08 |
| kota_ | acoles: nstream_decode[0, 0, 1, 2 ,3] (the number means FI) on k=5 and m=2 or so on.... | 09:08 |
| kota_ | acoles: *maybe* | 09:08 |
| acoles | kota_: so i am interested if you/colleague have seen two FAs land on same node - I don't yet know a way for that to happen | 09:08 |
| mattoliverau | Well they do all share the same tombstone :p | 09:09 |
| *** silor has quit IRC | 09:09 | |
| acoles | mattoliverau: it says 'Rest In Pieces' | 09:09 |
| acoles | kota_: huh, interesting | 09:09 |
| kota_ | acoles: not finished to look at yet but I'm thinking reconstructor building the fragment and pushing to other primaries | 09:10 |
| acoles | kota_: but that could be same fragment on different nodes, yes? | 09:10 |
| kota_ | acoles: on no failure case, we don't have same fragment on different nodes i think. | 09:11 |
| mattoliverau | Lol! Love it | 09:11 |
| kota_ | acoles: so that point I'm still wondering and found the reconstrucor might select the destination nodes from primaries. | 09:11 |
| kota_ | acoles: on the sync job. but I'm not sure :\ | 09:12 |
| acoles | kota_: hmmm, i see | 09:12 |
| kota_ | acoles: that's why I asked you and mattoliverau. | 09:12 |
| kota_ | acoles: currently I'm testing whether the different nodes have a possibility to have the same fragment of the same object | 09:13 |
| kota_ | acoles: (maybe we need "probe" test for that :/) | 09:14 |
| acoles | kota_: so tbh clayg is probably more up to speed on this, but maybe there is a case where frag 0 lands on a handoff and then a sync job also rebuilds frag 0 on its primary, so there are two copies of frag 0 until the reconstructor runs on the handoff and should clean up its copy. | 09:14 |
| acoles | kota_: probe test would be great! | 09:15 |
| kota_ | acoles: oh, yes. correct, we could feed the same fragment to decode and goes to fail unfortunately. | 09:16 |
| acoles | kota_: right, if during that window another primary fails then the proxy may bet frag 0 from two nodes?? | 09:17 |
| kota_ | acoles: for that case, maybe, we have to validate the frag-index before starting to decode. | 09:17 |
| acoles | s/bet/get/ | 09:17 |
| kota_ | acoles: nice assumption, I'll also try to make the test and code improvement. | 09:19 |
| kota_ | acoles: Thanks, my head is becoming to be clear. | 09:20 |
| acoles | kota_: if you can reproduce with a probe test then even post only that to gerrit and we can discuss how to fix in comments. i know that clayg has thought about this stuff perhaps more than me | 09:23 |
| kota_ | acoles: ok :) | 09:24 |
| kota_ | acoles, mattoliverau: FYI, (*unfortunately*) I have a week holidays since tommorow untill thursday next week because there are many continious national holidays in the term in Japan :( | 09:24 |
| kota_ | maybe I'll be watching on IRC and gerrit during the week :P | 09:25 |
| mattoliverau | kota_: not unfortuantly, go have fun :) We leave you a bunch of work to do when you get back :P | 09:26 |
| acoles | kota_: sounds good - have a good holiday! we have Monday as holiday in UK but I like the sound of 'continuous national holidays' :D | 09:26 |
| mattoliverau | +1 | 09:26 |
| kota_ | mattoliverau, acoles: lol, thanks ;) | 09:27 |
| *** zul has joined #openstack-swift | 09:38 | |
| *** dmorita has quit IRC | 09:44 | |
| kota_ | leaving from office for going back to home and dinner. | 10:06 |
| *** kota_ has quit IRC | 10:07 | |
| openstackgerrit | Merged openstack/swift: Fix tempauth acl checks when simplejson has no speedups https://review.openstack.org/159530 | 10:12 |
| *** openstackgerrit_ has joined #openstack-swift | 10:15 | |
| *** knl has joined #openstack-swift | 10:21 | |
| *** silor has joined #openstack-swift | 10:23 | |
| *** krykowski has quit IRC | 10:26 | |
| *** krykowski_ has joined #openstack-swift | 10:26 | |
| *** openstackgerrit_ has quit IRC | 10:31 | |
| *** zul has quit IRC | 10:35 | |
| *** krykowski_ has quit IRC | 10:37 | |
| *** krykowski_ has joined #openstack-swift | 10:38 | |
| *** mwheckmann has quit IRC | 10:40 | |
| *** xnox has joined #openstack-swift | 10:45 | |
| xnox | https://lists.ubuntu.com/archives/technical-board/2015-April/002100.html | 10:45 |
| xnox | "We respectfully ask that you voluntarily remove and not republish the libraries JErasure 2.0 and GF_Complete, as well as any other packages or releases that incorporate them or depend on them. We believe these packages and releases include: libjerasure-dev, libjerasure2, libgf-complete-dev, libgf-complete1, liberasurecode-dev, liberasurecode1, Pyeclib, CEPH (see ceph/src/erasure-code/jerasure) and | 10:45 |
| xnox | Swift 2.3.0. | 10:45 |
| xnox | " | 10:45 |
| xnox | ?! | 10:45 |
| xnox | ah old news | 10:48 |
| xnox | https://www.techdirt.com/articles/20141115/07113529155/patent-troll-kills-open-source-project-speeding-up-computation-erasure-codes.shtml | 10:48 |
| *** krykowski_ has quit IRC | 11:02 | |
| *** krykowski has joined #openstack-swift | 11:02 | |
| *** jamielennox is now known as jamielennox|away | 11:20 | |
| *** aix has joined #openstack-swift | 11:21 | |
| *** krykowski has quit IRC | 11:34 | |
| *** knl has quit IRC | 11:48 | |
| *** zul has joined #openstack-swift | 11:50 | |
| *** dencaval has joined #openstack-swift | 11:51 | |
| *** kota_ has joined #openstack-swift | 12:04 | |
| *** ppai has quit IRC | 12:05 | |
| *** chuck__ has joined #openstack-swift | 12:08 | |
| kota_ | hmm...sync job will reconstruct the FA with the target fragment index. It makes me sense the reason the reconstructor sets only part_nodes to dest_nodes for now. | 12:09 |
| *** zul has quit IRC | 12:10 | |
| *** krykowski has joined #openstack-swift | 12:27 | |
| *** bkopilov has quit IRC | 12:31 | |
| *** panbalag has joined #openstack-swift | 12:35 | |
| *** knl has joined #openstack-swift | 12:35 | |
| *** jistr has quit IRC | 12:39 | |
| *** ahale has joined #openstack-swift | 12:44 | |
| kota_ | FWIW, I'm a little worried the revert job for tombstones would fan a new .data came after building jobs out to primaries. | 12:46 |
| kota_ | not sure and need to look at more deeply :/ | 12:47 |
| *** chuck__ has quit IRC | 12:53 | |
| kota_ | hmm...I need more fresh head to think clealy so stop here today. | 12:54 |
| *** kota_ has quit IRC | 12:54 | |
| *** jistr has joined #openstack-swift | 12:54 | |
| *** jistr is now known as jistr|biab | 12:55 | |
| *** geaaru has quit IRC | 12:57 | |
| *** thumpba has joined #openstack-swift | 12:58 | |
| *** thumpba_ has joined #openstack-swift | 13:00 | |
| *** thumpba has quit IRC | 13:04 | |
| *** fifieldt has quit IRC | 13:04 | |
| eikke | are Swift 2.2.0 functional tests supposed to pass on a Python 2.6 system? | 13:13 |
| *** esker has quit IRC | 13:18 | |
| *** esker has joined #openstack-swift | 13:18 | |
| *** esker has quit IRC | 13:19 | |
| eikke | notmyname: ^^ | 13:20 |
| *** cdelatte has joined #openstack-swift | 13:33 | |
| *** zul has joined #openstack-swift | 13:35 | |
| *** ppai has joined #openstack-swift | 13:36 | |
| *** zul has quit IRC | 13:40 | |
| *** krykowski has quit IRC | 13:44 | |
| *** jrichli has joined #openstack-swift | 13:49 | |
| *** jistr|biab is now known as jistr | 13:53 | |
| *** proteusguy has joined #openstack-swift | 13:55 | |
| *** chlong has joined #openstack-swift | 13:56 | |
| *** erlon has joined #openstack-swift | 13:56 | |
| *** mwheckmann has joined #openstack-swift | 13:58 | |
| *** wbhuber has joined #openstack-swift | 13:59 | |
| jrichli | eikke: how many failures are you getting? Have you tried a resetswift and then run again? swift still supports 2.6 | 13:59 |
| *** zul has joined #openstack-swift | 14:00 | |
| *** esker has joined #openstack-swift | 14:02 | |
| jordanP | jrichli, we see more than 60 failures, related to Unicode Support in the swift client | 14:11 |
| *** ppai has quit IRC | 14:13 | |
| acoles | eikke: jordanP : i just ran functests ok in a py26 venv on my SAIO | 14:14 |
| *** ppai has joined #openstack-swift | 14:14 | |
| *** vinsh has quit IRC | 14:23 | |
| *** NM has joined #openstack-swift | 14:24 | |
| *** zul has quit IRC | 14:26 | |
| *** JelleB is now known as a1|away | 14:30 | |
| *** vinsh has joined #openstack-swift | 14:41 | |
| *** wbhuber__ has joined #openstack-swift | 14:41 | |
| *** lpabon has joined #openstack-swift | 14:42 | |
| *** wbhuber has quit IRC | 14:43 | |
| *** zul has joined #openstack-swift | 14:49 | |
| *** chlong has quit IRC | 14:53 | |
| *** ppai has quit IRC | 14:59 | |
| *** aerwin has joined #openstack-swift | 15:00 | |
| peluse | xnox, Swift core project does not include EC libraries. They are external to the project and users are free to choose from multiple options like jerasure or ISA-L from Intel | 15:01 |
| peluse | xnox, but thanks and yes we are all well aware of the info you passed on :) | 15:01 |
| *** annegentle has joined #openstack-swift | 15:02 | |
| xnox | peluse: right. i'll go back to ignoring trolls. | 15:03 |
| peluse | that's always the best option for sure! | 15:03 |
| notmyname | :-) | 15:17 |
| notmyname | peluse: thanks | 15:17 |
| notmyname | yeah, swift does not depend on jerasure | 15:17 |
| notmyname | eikke: and jrichli told you correctly. current swift should still work under py26 | 15:17 |
| lpabon | notmyname: hi... what's the story with hummingbird? Should I spend some time reviewing the Go code, or should I wait for it to stabilize? | 15:20 |
| notmyname | lpabon: take a look, see what you think. vancouver will be a great place to have a good discussion | 15:21 |
| lpabon | notmyname: will do | 15:21 |
| notmyname | right now it's an idea in a feature branch, and between now and the tokyo summit I'd love to explore the idea of "compiled language object server" | 15:21 |
| lpabon | lpabon: Sweeet.. plus.. Go is really awesome | 15:22 |
| portante | lpabon: are you talking to your self about Go again? ;) | 15:25 |
| lpabon | portante: you know it.. I want to shout it from the mountain tops | 15:25 |
| *** jordanP has quit IRC | 15:25 | |
| dmsimard | peluse: So, I couldn't sleep until I was satisfied last night. I got up to 8 Gbps | 15:28 |
| dmsimard | Could probably do more but that's where I went to sleep | 15:28 |
| peluse | dmsimard, great, what did ya do? | 15:28 |
| peluse | just lots of coffee? | 15:29 |
| dmsimard | :D | 15:29 |
| dmsimard | So long story short, there were two things | 15:29 |
| dmsimard | 1) I wasn't pushing ssbench enough so I boosted the scenarios in amount of files, their size, the amount of operations but especially the user_count and the worker concurrency | 15:29 |
| dmsimard | Once I had done that, haproxy was choking under the load so I looked into how I could solve that | 15:30 |
| dmsimard | Turns out a single haproxy process wasn't enough - I put tentatively nbproc at 4 and the traffic it was able to pull was greatly increased | 15:31 |
| peluse | hmm, interesting. don't know what ours was setup for but will check | 15:31 |
| dmsimard | With that done, I was getting a lot of CPU %sys and Soft IRQs so it looks like the next bottleneck is the network and SSL TPS processing | 15:31 |
| dmsimard | We've had to do some network card config tweaking before due to the high amount of pps, by default the processing will only be done on one core but you can "spray" the packets across multiple cores for improved throughput | 15:33 |
| peluse | how are your card configured - num links, bonded, etc? | 15:34 |
| dmsimard | Load balancers are 2x 10 Gbps in LACP, hash layer 2+3 - that same bond handles both WAN and LAN connectivity (storage nodes are behind in a LAN) | 15:35 |
| peluse | OK, IM'ing with our benchmark guy and he said they didn't have the cards balanced properly accross cpus and are going to try again today | 15:37 |
| peluse | BTW, same config here on the NICs in the LB | 15:38 |
| peluse | glad to hear you made such great progress! | 15:39 |
| *** annegentle has quit IRC | 15:53 | |
| *** EmilienM is now known as EmilienM|afk | 15:55 | |
| *** mahatic has joined #openstack-swift | 16:01 | |
| *** lpabon has quit IRC | 16:01 | |
| *** delattec has joined #openstack-swift | 16:06 | |
| *** delatte has joined #openstack-swift | 16:06 | |
| *** gyee has joined #openstack-swift | 16:07 | |
| *** cdelatte has quit IRC | 16:09 | |
| *** zaitcev has joined #openstack-swift | 16:13 | |
| *** ChanServ sets mode: +v zaitcev | 16:13 | |
| openstackgerrit | John Dickinson proposed openstack/swift: Merge branch 'stable/kilo' into master https://review.openstack.org/179148 | 16:20 |
| notmyname | ^ that one will make the tags work out for versioning | 16:21 |
| hurricanerix | morning | 16:31 |
| *** mahatic has quit IRC | 16:33 | |
| *** gyee has quit IRC | 16:37 | |
| *** guitarzan has joined #openstack-swift | 16:39 | |
| *** knl has quit IRC | 16:40 | |
| *** zul has quit IRC | 16:40 | |
| notmyname | mailing list post: http://lists.openstack.org/pipermail/openstack-dev/2015-April/063019.html | 16:40 |
| notmyname | good mornign hurricanerix | 16:41 |
| *** mahatic has joined #openstack-swift | 16:42 | |
| eikke | notmyname: thats what we thought | 17:05 |
| eikke | notmyname: seems related to #1190190, on centos6 | 17:05 |
| eikke | notmyname: will investigate next week (sorry for multi-message, on a train, terrible latencies) | 17:06 |
| peluse | notmyname, FYI I just added a py3.0 perf discussion topic for the Fri session - what we talked about before about working with our compiler/interpreter optimization team on improvements in 3.0 | 17:08 |
| notmyname | cool | 17:08 |
| * eikke also *very* interested in the whole language/platform thing, but not exactly a Go fan | 17:12 | |
| *** mahatic has quit IRC | 17:14 | |
| peluse | go go gadget... something | 17:15 |
| *** cutforth has quit IRC | 17:21 | |
| *** zhill has joined #openstack-swift | 17:22 | |
| *** tsg_ has joined #openstack-swift | 17:29 | |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Fix --skip-identical to allow identical xLO to replace non-xLO https://review.openstack.org/179175 | 17:30 |
| *** jrichli_ has joined #openstack-swift | 17:37 | |
| *** thumpba has joined #openstack-swift | 17:37 | |
| *** jrichli_ has quit IRC | 17:40 | |
| *** BAKfr has quit IRC | 17:40 | |
| *** jrichli_ has joined #openstack-swift | 17:40 | |
| *** jrichli_ has quit IRC | 17:41 | |
| *** haomaiw__ has joined #openstack-swift | 17:41 | |
| *** anticw_ has joined #openstack-swift | 17:42 | |
| *** guitarza1 has joined #openstack-swift | 17:42 | |
| *** jeblair_ has joined #openstack-swift | 17:42 | |
| openstackgerrit | Merged openstack/swift: Merge branch 'stable/kilo' into master https://review.openstack.org/179148 | 17:45 |
| *** dmsimard_ has joined #openstack-swift | 17:45 | |
| *** aerwin has quit IRC | 17:46 | |
| *** jrichli has quit IRC | 17:47 | |
| *** thumpba_ has quit IRC | 17:47 | |
| *** haomaiwang has quit IRC | 17:47 | |
| *** anticw has quit IRC | 17:47 | |
| *** dmsimard has quit IRC | 17:47 | |
| *** omame has quit IRC | 17:47 | |
| *** jistr has quit IRC | 17:47 | |
| *** jeblair has quit IRC | 17:47 | |
| *** a1|away has quit IRC | 17:47 | |
| *** guitarzan has quit IRC | 17:47 | |
| *** delattec has quit IRC | 17:47 | |
| *** morganfainberg has quit IRC | 17:47 | |
| *** alpha_ori has quit IRC | 17:47 | |
| *** jd__ has quit IRC | 17:47 | |
| *** redbo has quit IRC | 17:47 | |
| *** tanee has quit IRC | 17:47 | |
| *** dmsimard_ is now known as dmsimard | 17:47 | |
| *** delattec has joined #openstack-swift | 17:49 | |
| *** morganfainberg has joined #openstack-swift | 17:49 | |
| *** alpha_ori has joined #openstack-swift | 17:49 | |
| *** jd__ has joined #openstack-swift | 17:49 | |
| *** redbo has joined #openstack-swift | 17:49 | |
| *** tanee has joined #openstack-swift | 17:49 | |
| *** sendak.freenode.net sets mode: +v redbo | 17:49 | |
| *** bkopilov has joined #openstack-swift | 17:54 | |
| *** a1|away has joined #openstack-swift | 17:55 | |
| *** BAKfr has joined #openstack-swift | 18:02 | |
| *** acoles is now known as acoles_away | 18:05 | |
| *** annegentle has joined #openstack-swift | 18:05 | |
| *** aix has quit IRC | 18:07 | |
| *** EmilienM|afk is now known as EmilienM | 18:11 | |
| *** jrichli has joined #openstack-swift | 18:16 | |
| jlk | Hey all, I'm back with more tempurl problems | 18:22 |
| *** bkopilov has quit IRC | 18:23 | |
| jlk | I can't seem to make tempurl auth work | 18:23 |
| jlk | and I can't convince swift-proxy to tell me why it's 401ing the url | 18:23 |
| jlk | This is the log entry in swift-proxy I'm getting https://gist.github.com/j2sol/5825c107eb10c910f83f | 18:26 |
| *** bkopilov has joined #openstack-swift | 18:26 | |
| jlk | I generated the Meta Temp-Url-Key, and used swift tempurl to create a GET url | 18:27 |
| jlk | swift tempurl GET 200 /v1/AUTH_tempest4/test1/upgrade.retry password1 | 18:27 |
| jlk | "tempest4" is the name of the tenant my user belongs in | 18:27 |
| jlk | Can somebody help me debug what's going on here? | 18:28 |
| notmyname | jlk: do a HEAD to your account (AUTH_tempest4) to make sure that the tempurl key is actually set | 18:32 |
| jlk | what would that URL look like? | 18:32 |
| jlk | or would a simple swift stat suffice? | 18:32 |
| jlk | swift stat shows: | 18:32 |
| jlk | Meta Temp-Url-Key: password1 | 18:32 |
| notmyname | ok | 18:32 |
| jlk | for the AUTH_ part, should I be using tenant name, tenant uuid, user uuid? | 18:34 |
| jlk | I see mixed examples on the 'net | 18:34 |
| notmyname | it's not something like not escaping the & on the command line, right? | 18:34 |
| jlk | well, I used swift tempurl to make it, which appears to properly escape all the parts of the url | 18:35 |
| *** bkopilov has quit IRC | 18:35 | |
| jlk | it spat out: | 18:35 |
| *** esmute has quit IRC | 18:35 | |
| notmyname | jlk: that depends on what your auth system says. the only hard rule, is that you use the storage url that the auth system gives you | 18:35 |
| *** bkopilov has joined #openstack-swift | 18:35 | |
| *** esmute has joined #openstack-swift | 18:35 | |
| jlk | er, where would my auth system say that? I'm using keystone for auth with swift, although in the swift-proxy I have tempurl listed before authtoken or keystoneauth | 18:36 |
| jlk | and the tempurl filter just lists use = egg:swift#tempurl | 18:37 |
| jlk | Here's a more complete output https://gist.github.com/j2sol/32858d7e6a9c2e16a957 | 18:39 |
| torgomatic | jlk: did you set allow_overrides = true in your keystoneauth config? or delay_auth_decision or whatever it is? | 18:41 |
| torgomatic | hang on | 18:41 |
| jlk | yeah, delay_auth_decision is set | 18:41 |
| torgomatic | okay, that's good | 18:41 |
| jlk | Apr 30 18:38:34 localhost.localdomain proxy-server: Invalid user token - deferri | 18:41 |
| jlk | ng reject downstream | 18:41 |
| jlk | Apr 30 18:38:34 localhost.localdomain proxy-server: Authorizing from an overridi | 18:42 |
| jlk | ng middleware (i.e: tempurl) (txn: tx0844182784cc4f6f874c5-00554276aa) | 18:42 |
| torgomatic | so the tempurl middleware won't ever return a 401 like that; the body will say "Temp URL invalid" or something similar | 18:42 |
| torgomatic | jlk: okay, so that looks good | 18:42 |
| jlk | torgomatic: if I do a straight curl instead of a HEAD, I get 401 Unauthorized: Temp URL invalid | 18:43 |
| *** bkopilov has quit IRC | 18:43 | |
| torgomatic | jlk: the gist you sent does not include the body of the response; could you post a new one with the entire failing curl? | 18:43 |
| *** guitarza1 is now known as guitarzan | 18:45 | |
| jlk | Is this what you meant? https://gist.github.com/j2sol/2f49d0831caafb43de1a | 18:45 |
| jlk | re-ran with -v https://gist.github.com/j2sol/2f49d0831caafb43de1a | 18:46 |
| zaitcev | y u not -v | 18:47 |
| zaitcev | oh | 18:47 |
| clayg | what is going on! | 18:47 |
| torgomatic | jlk: okay, and how about the output of "swift stat"? that looks to me like your temp url key isn't set properly | 18:48 |
| jlk | https://gist.github.com/j2sol/eda188e46458f0e2991a | 18:48 |
| openstackgerrit | Merged openstack/python-swiftclient: Compare each chunk of large objects when uploading https://review.openstack.org/161043 | 18:49 |
| jlk | odd that stat is showing 0 objects even though I've uploaded an object into my container | 18:49 |
| torgomatic | jlk: eh, it's eventually consistent | 18:49 |
| torgomatic | jlk: so that says your account is 7bb348216c144eb69cae5d7777cb604f | 18:51 |
| torgomatic | your tempurl is for AUTH_tempest4 | 18:52 |
| jlk | yeah, one is the name, the other is the UUID | 18:52 |
| torgomatic | use the uuid | 18:52 |
| jlk | awesome | 18:52 |
| torgomatic | sure | 18:52 |
| *** annegentle has quit IRC | 18:52 | |
| torgomatic | /v1/<uuid>/container/object | 18:52 |
| jlk | not even AUTH_<uuid> ? | 18:53 |
| torgomatic | doesn't look like it | 18:53 |
| torgomatic | use the thing that comes after "Account: " in the output of `swift stat`, because that's the account's name on disk | 18:53 |
| jlk | curl "https://bbg-staging-swift.openstack.blueboxgrid.com:8090/v1/7bb348216c144eb69cae5d7777cb604f/test1/upgrade.retry?temp_url_sig=6fdb33b55daca3d02d2938b173a675385fd2f692&temp_url_expires=1430420461" | 18:53 |
| jlk | 401 | 18:53 |
| torgomatic | jlk: did you regenerate the tempurl with the new name? | 18:54 |
| jlk | (after generating a new one) | 18:54 |
| jlk | oh wait | 18:54 |
| jlk | geez, now it works | 18:54 |
| torgomatic | :) | 18:54 |
| jlk | so the docs for this... really not good | 18:55 |
| *** bkopilov has joined #openstack-swift | 18:56 | |
| jlk | and tempest passes now, because my catalog was bad. | 19:01 |
| *** cdelatte has joined #openstack-swift | 19:01 | |
| *** delattec has quit IRC | 19:01 | |
| *** delatte has quit IRC | 19:02 | |
| *** delattec has joined #openstack-swift | 19:02 | |
| *** bkopilov has quit IRC | 19:03 | |
| *** jd__ has quit IRC | 19:04 | |
| *** jd__ has joined #openstack-swift | 19:06 | |
| jlk | torgomatic: notmyname: thank you for the help | 19:09 |
| torgomatic | jlk: you're quite welcome | 19:09 |
| *** thumpba has quit IRC | 19:13 | |
| *** bkopilov has joined #openstack-swift | 19:15 | |
| *** bkopilov has quit IRC | 19:16 | |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Actually function under python 3 https://review.openstack.org/178793 | 19:18 |
| *** bkopilov has joined #openstack-swift | 19:20 | |
| *** bkopilov has quit IRC | 19:29 | |
| zaitcev | jlk: I filed a bug to remind ourselves about docs https://bugs.launchpad.net/python-swiftclient/+bug/1450606 | 19:32 |
| openstack | Launchpad bug 1450606 in python-swiftclient "the tempurl subcommand is missing from the man page" [Undecided,New] | 19:32 |
| jlk | oh thanks! | 19:32 |
| *** omame has joined #openstack-swift | 19:32 | |
| clayg | has anyone tried to get pyeclib installed on a mac? | 19:32 |
| clayg | it's starting to bum me out that I can't import swift in a repl from a virtualenv on my mac | 19:33 |
| *** a1|away has quit IRC | 19:33 | |
| *** wolsen has quit IRC | 19:33 | |
| *** d0ugal has quit IRC | 19:33 | |
| *** wolsen_ has joined #openstack-swift | 19:33 | |
| mwheckmann | quit | 19:33 |
| zaitcev | God, no. Linux was trouble enough. | 19:33 |
| *** mwheckmann has quit IRC | 19:33 | |
| clayg | mwheckmann: no way man - stick with it! | 19:33 |
| clayg | zaitcev: :P | 19:34 |
| *** a1|away has joined #openstack-swift | 19:34 | |
| *** d0ugal has joined #openstack-swift | 19:34 | |
| *** bkopilov has joined #openstack-swift | 19:35 | |
| *** esker has quit IRC | 19:36 | |
| jlk | ooooh! | 19:37 |
| jlk | wrong win | 19:37 |
| *** bkopilov has quit IRC | 19:41 | |
| jrichli | lol. I just know I am gonna post one of my passwords one of these days. | 19:42 |
| *** bkopilov has joined #openstack-swift | 19:42 | |
| clayg | tsg_: https://gist.github.com/clayg/df95a411a6f5db552550 | 19:48 |
| clayg | got a compile error on my mac - looks like -Werror might be just being mean to me? | 19:48 |
| clayg | tsg_: I don't recall seeing anything like this on my linux box - but I *did* just pull master? | 19:48 |
| *** bkopilov has quit IRC | 19:49 | |
| clayg | idk, it sure does look like we're testing if an unsigned int is less than 0 - that does *seem* silly? | 19:51 |
| clayg | is a 900+ line Makefile big? feels kinda epic | 19:53 |
| peluse | clayg, our internal IM shows tsg offline for the last 12 hrs, he might be travelling | 19:58 |
| *** jd__ has quit IRC | 19:58 | |
| clayg | peluse: I try to pretend that i can talk to people on irc even if they're not here :\ | 19:59 |
| clayg | how is that after make I don't have any .so's in my liberasurecode folder | 19:59 |
| clayg | does freebsd/mac not call shared object's .so? | 20:00 |
| *** tsg_ has quit IRC | 20:05 | |
| *** tsg_ has joined #openstack-swift | 20:06 | |
| *** vinsh_ has joined #openstack-swift | 20:06 | |
| tsg_ | clayg: noticed your comment above on libec error | 20:07 |
| tsg_ | clayg: let me check the latest Kota commits | 20:07 |
| clayg | tsg_: dont' worry about it right just yet - i fixed that | 20:07 |
| clayg | so I have a make that finishes on my mac - but I don't see the .so's? | 20:07 |
| tsg_ | clayg: you should have .dylibs? | 20:08 |
| * clayg was never good a building c projects :'( | 20:08 | |
| clayg | tsg_: YES! | 20:08 |
| clayg | you rock | 20:08 |
| clayg | ah ./src/.libs | 20:08 |
| tsg_ | yep | 20:08 |
| clayg | didn't notice the dot-dir | 20:08 |
| tsg_ | that's libtool! | 20:08 |
| clayg | tsg_: so I'm trying to avoid a `sudo make install` and would rather just tell my swift venv how to find the liberasure libs it needs | 20:09 |
| tsg_ | clayg: you can do "make install DESTDIR=<dir>"? | 20:09 |
| clayg | not that my mac is so "pristine" or anything - but I couldn't even really tease out from the Makefile what 'install' would *do* - it wasn't obvious to me that it would "do the right thing" on a mac | 20:09 |
| tsg_ | and add that dir to ldconfig? | 20:09 |
| *** vinsh has quit IRC | 20:10 | |
| tsg_ | (or the equivalent on mac .. ) | 20:10 |
| clayg | mac says ldconfig not found - do i really need to brew install that? I thought if I added something to some env somewhere the pyeclib build could find it? | 20:10 |
| tsg_ | clayg: DYLD_LIBRARY_PATH | 20:11 |
| clayg | tsg_: ok that's good - turns out that pyeclib install is trying to do the download and compile trick | 20:12 |
| * notmyname is convinced that tsg_ has an encyclopedic knowledge of every platform's build tooling and environment | 20:12 | |
| clayg | and it's blowing up on the same error with the unsigned int thing - so maybe I should have just let you look at that :P | 20:12 |
| clayg | tsg_: but since i have managed to get liberasure build in another dir - it seems like i should be able to convince the pyeclib install to just use that | 20:13 |
| clayg | i set the DYLD_LIBRARY_PATH but either i did it wrong or it didn't buy it | 20:13 |
| clayg | DYLD_LIBRARY_PATH=/Users/clayg/Workspace/liberasurecode/src/.libs | 20:14 |
| * tsg_ checking "man dyld" | 20:14 | |
| *** erlon is now known as erlon_awaY | 20:19 | |
| tsg_ | clayg: btw, the Makefile is generated w/ automake so is full of GNU'isms .. thus LONG :) | 20:22 |
| tsg_ | clayg: and I checked DYLD_LIBRARY_PATH should have worked :| | 20:22 |
| tsg_ | clayg: let me look at the -Werror thing first | 20:23 |
| clayg | tsg_: awesome - super helpful thanks! | 20:23 |
| *** dencaval has quit IRC | 20:27 | |
| tsg_ | clayg: are you running Yosemite? what gcc version? | 20:28 |
| clayg | Apple LLVM version 6.1.0 (clang-602.0.49) (based on LLVM 3.6.0svn) | 20:29 |
| clayg | ^ so you know - total garbage! | 20:29 |
| tsg_ | clayg: ah clang .. :) ok gcc didn't complain | 20:29 |
| clayg | heh :P | 20:29 |
| clayg | well... | 20:29 |
| clayg | yeah i mean fixing my environment so i can compile without errors during the pyeclib install seems like a reasonable work-around? | 20:29 |
| tsg_ | clayg: nah .. it is a legitimate warning - let me push a fix | 20:30 |
| tsg_ | clayg: tells me I need to try it building manually with clang (kept thinking Eric Lambert had a Jenkins job testing with clang .. need to check with him) | 20:31 |
| ekarlso | notmyname: are you at rackspace atm or smth since you mentioned cloudfiles team ? :P | 20:31 |
| notmyname | ekarlso: no, I've worked at swiftstack for 3 years now | 20:35 |
| ekarlso | notmyname: ok :D | 20:36 |
| *** jd__ has joined #openstack-swift | 20:41 | |
| tsg_ | clayg: tried with clang, also added a "-Wall" - looks like the uint comparison was the only leftover warning | 20:41 |
| clayg | tsg_: right! so it's easy | 20:42 |
| tsg_ | pushed a fix (along with the -Wall addition) | 20:43 |
| tsg_ | clayg: are you still seeing the DYLD_LIBRARY_PATH issue? | 20:44 |
| clayg | hrmm... so I tried with "gcc (Homebrew gcc 4.9.2_1) 4.9.2" and got a different "no such instruction: `vmovdqa (%rdi,%rax), %xmm0'" error | 20:44 |
| clayg | tsg_: I was kicking around trying to let pyeclib build libearusre on install with a different compiler | 20:45 |
| tsg_ | clayg: stale binaries perhaps? | 20:45 |
| clayg | who's stale now? | 20:45 |
| tsg_ | clayg: :) | 20:45 |
| tsg_ | clayg: assuming you are building in the same tree as the clang build, does a "make mrproper && ./configure && make" help? | 20:46 |
| tsg_ | sorry | 20:46 |
| tsg_ | make distclean | 20:46 |
| * tsg_ has been building linux kernels too often lately! (that's where the mrproper came from) | 20:47 | |
| clayg | well yeah so the make works with clang after the fix | 20:47 |
| clayg | the undefined symbol thing was trying to build with homebrew gcc | 20:47 |
| clayg | so - like whole different direction | 20:47 |
| tsg_ | clayg: unless the gcc build was in a completely separate tree, I was thinking there were perhaps some leftover binaries from the clang build that gcc tried to use .. | 20:48 |
| clayg | oh oh oh - yeah it was in a totally seperate tree - because it was a tempdir that pip pulled down | 20:48 |
| tsg_ | clayg: ok .. so might be something to do with the gcc on Mac (I have little experience with that) - but I know that Kevin has been using gcc on Mac | 20:49 |
| tsg_ | let me ask him what rev | 20:49 |
| clayg | sure - i'd honestly be happier building it seperatly and making pip pyeclib install able to find the liberasure i already built | 20:50 |
| clayg | but yeah there the dyld thing didn't seem to be doing jack | 20:50 |
| clayg | maybe I could install pyeclib by hand and let the swift install just find that | 20:50 |
| tsg_ | clayg: pip install may not be able to find liberasurecode if the DYLD_LIBRARY_PATH isn't global | 20:51 |
| tsg_ | something that might work - "DYLD_LIBRARY_PATH=<path> pip install pyeclib" | 20:52 |
| clayg | yeah that's *sorta* what I did | 20:52 |
| clayg | nah it's still trying to download and install liberasurecode | 20:53 |
| tsg_ | clayg: let me find the Mac guy | 20:53 |
| *** annegentle has joined #openstack-swift | 21:00 | |
| clayg | tsg_: making progress installing pyeclib from source -> ld: library not found for -lerasurecode | 21:00 |
| clayg | this is where I would *think* the DYLD_LIBRARY_PATH think would help | 21:00 |
| tsg_ | clayg: Just tried these steps on a Mac and seemed to work | 21:02 |
| tsg_ | cd liberasurecode.git; ./configure | 21:02 |
| tsg_ | ; make; | 21:02 |
| tsg_ | export DYLD_LIBRARY_PATH="src/.libs" | 21:02 |
| tsg_ | ; sudo pip install pyeclib | 21:02 |
| tsg_ | clayg: sorry about the extra ';'s .. didn't know they'd turn into newlines :) | 21:03 |
| clayg | ;) | 21:03 |
| clayg | tsg_: wait... is https://bitbucket.org/kmgreen2/pyeclib.git the remote origin for pyeclib - i pulled master and got 1.0.5? | 21:04 |
| tsg_ | you should have gotten 1.0.7m | 21:04 |
| clayg | well it doesn't matter | 21:05 |
| clayg | it worked :) | 21:05 |
| clayg | er... what you told me to do worked | 21:05 |
| tsg_ | cool! :) | 21:05 |
| clayg | the only think I had to add was C_INCLUDE_PATH= | 21:06 |
| clayg | zaitcev: see it wasn't so bad - like *barely* three pages of scrollback | 21:06 |
| zaitcev | clayg: meanwhile I decompiled my CRUSH map and set a parameter there "step chooseleaf firstn 0 type osd", and now my "CAIO" works | 21:08 |
| zaitcev | also took a bit of hand-holding on IRC | 21:09 |
| clayg | yay IRC! | 21:09 |
| zaitcev | yay! | 21:09 |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - working on the encryption feature. https://review.openstack.org/157907 | 21:10 |
| zaitcev | oh, speaking of those include paths | 21:11 |
| tsg_ | clayg: what's with C_INCLUDE_PATH? | 21:12 |
| clayg | tsg_: i got an error couldn't find the erasurecode.h or some such | 21:13 |
| zaitcev | clayg, tsg_: I eventually built liberasurecode like so: http://www.zaitcev.us/things/liberasurecode-1.0.7-cflags.patch | 21:13 |
| zaitcev | A smart guy suggested me to trick the code by placing all the right flags into CPPFLAGS | 21:14 |
| clayg | zaitcev: that doesn't seem so unreasonable - why not push back? | 21:14 |
| zaitcev | But I chose to mod the source instead. It's open source, right. | 21:14 |
| zaitcev | Our build environment passes CFLAGS down, which includes mandatory things in Fedora, like -fsomething-something-stack-protector | 21:15 |
| tsg_ | zaitcev: ah ok .. I was about to ask | 21:15 |
| zaitcev | But without that patch, liberasurecode ignores CFLAGS | 21:16 |
| tsg_ | zaitcev: you mean overrides it? | 21:16 |
| clayg | tsg_: no he just *adds* to it | 21:16 |
| zaitcev | tsg_: that's a good verb too | 21:16 |
| zaitcev | like so - https://bugzilla.redhat.com/show_bug.cgi?id=1208695#c4 | 21:17 |
| openstack | bugzilla.redhat.com bug 1208695 in Package Review "Review Request: liberasurecode - Erasure Code API library written in C with pluggable backends" [Medium,Assigned] - Assigned to ppisar | 21:17 |
| tsg_ | zaitcev: ok .. I will double check the top-level automake/autoconf setup | 21:17 |
| *** jrichli has quit IRC | 21:18 | |
| tsg_ | zaitcev: thank you - as you mentioned in comment#5 there, please do submit the patch upstream | 21:19 |
| tsg_ | zaitcev: also the m4 cleanup is almost done (Kevin and I are working on finalizing that) | 21:19 |
| zaitcev | tsg_: thanks, guys | 21:19 |
| *** vinsh has joined #openstack-swift | 21:20 | |
| tsg_ | zaitcev: we are debating whether to support multiple cross-build targets (for SSE, AVX, NEON etc), or do the cpu capabilities detection at runtime | 21:21 |
| tsg_ | zaitcev: in the former, we'd end up creating multiple liberasurecode subpackages | 21:21 |
| zaitcev | tsg_: Kevin e-mails me that it's inconvenient to make everything runtime. I trust you guys to figure it out... As long as I can build something on x86_64 that works on both Intel and AMD. | 21:22 |
| *** vinsh_ has quit IRC | 21:23 | |
| tsg_ | zaitcev: :) OK .. we'll figure out something that works cross platform | 21:23 |
| *** mragupat has joined #openstack-swift | 21:25 | |
| *** esker has joined #openstack-swift | 21:33 | |
| *** annegentle has quit IRC | 21:42 | |
| *** annegentle has joined #openstack-swift | 21:42 | |
| *** bkopilov has joined #openstack-swift | 21:48 | |
| *** annegentle has quit IRC | 21:48 | |
| mattoliverau | Morning | 21:52 |
| peluse | morning | 22:07 |
| peluse | zaitcev, wait.. works on what? :) | 22:07 |
| *** annegentle has joined #openstack-swift | 22:08 | |
| *** gyee has joined #openstack-swift | 22:13 | |
| *** jamielennox|away is now known as jamielennox | 22:16 | |
| *** erlon_awaY has quit IRC | 22:21 | |
| *** mragupat has quit IRC | 22:22 | |
| zaitcev | peluse: That's nothing. You should see the OpenStack Manila RPMs that I have built today. | 22:35 |
| zaitcev | It made me love Swift's modest requirements even more | 22:35 |
| zaitcev | How would you like to bug maintainers to build python-oslo-concurrency >= 1.8.0, python-oslo-config >= 1.7.0, python-oslo-db >= 1.7.1, python-oslo-i18n >= 1.5.0, python-oslo-messaging >= 1.3.0-0.1.a9, python-oslo-serialization >= 1.4.0, python-oslo-utils >= 1.4.0 | 22:36 |
| *** chlong has joined #openstack-swift | 22:36 | |
| zaitcev | None which I have access rights to build myself | 22:36 |
| zaitcev | They also require the following transients: python-netifaces >= 0.10.4, python-alembic >= 0.7.2, python-migrate >= 0.9.5, python-stevedore >= 1.3.0 | 22:36 |
| zaitcev | I probably meant "transitives" there, if that is a word. | 22:37 |
| *** esker has quit IRC | 22:42 | |
| *** annegentle has quit IRC | 22:43 | |
| *** NM has quit IRC | 22:48 | |
| InAnimaTe | hey all. im finding myself on a trusty box but needing to install grizzly (1.8.0) | 22:54 |
| InAnimaTe | adding the ppa obv yells at me | 22:54 |
| InAnimaTe | besides tracking down the dpkg's manually, any ideas? | 22:54 |
| notmyname | InAnimaTe: don't install from the PPAs. otherwise it should work | 22:54 |
| InAnimaTe | ok | 22:54 |
| notmyname | worst case, you could install from source | 22:54 |
| notmyname | ie if you can't find packages | 22:55 |
| InAnimaTe | yeah im looking for them right now | 22:55 |
| *** wbhuber__ has quit IRC | 22:56 | |
| InAnimaTe | https://launchpad.net/~ubuntu-cloud-archive/+archive/ubuntu/grizzly-staging/+build/4730136 | 22:57 |
| InAnimaTe | ^that should do :) | 22:57 |
| *** annegentle has joined #openstack-swift | 22:58 | |
| InAnimaTe | although those are i386 | 22:58 |
| notmyname | I'd suspect you'd be able to get packages from today's kilo release (2.3.0) from canonical | 23:01 |
| notmyname | oh, you need to use 1.8.0 | 23:02 |
| InAnimaTe | yeah | 23:02 |
| InAnimaTe | well | 23:02 |
| InAnimaTe | those package links are *_all.deb | 23:02 |
| notmyname | just to state the thing that of course is always said by the maintainers of a project: you should upgrade :-) | 23:02 |
| InAnimaTe | ehh, its not an option | 23:02 |
| InAnimaTe | trust me, i pushed for it | 23:02 |
| InAnimaTe | mgmt ftl | 23:02 |
| notmyname | heh. like I said, it's just my duty to say that | 23:02 |
| InAnimaTe | lol thx | 23:02 |
| *** annegentle has quit IRC | 23:03 | |
| notmyname | tell them "some guy on the internet said I should upgrade". management loves that | 23:03 |
| InAnimaTe | wtf | 23:03 |
| InAnimaTe | were these packages only built i386 | 23:04 |
| InAnimaTe | i cant seem to find them for 64bit | 23:04 |
| *** annegentle has joined #openstack-swift | 23:08 | |
| InAnimaTe | ahh | 23:11 |
| InAnimaTe | Architecture: all | 23:11 |
| InAnimaTe | so its just blanket supports everything | 23:12 |
| InAnimaTe | SWEEET | 23:12 |
| zaitcev | Well it's Python | 23:22 |
| *** vinsh has quit IRC | 23:34 | |
| *** annegentle has quit IRC | 23:36 | |
| *** ho has joined #openstack-swift | 23:42 | |
| ho | good morning | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!