*** haomaiwa_ has quit IRC | 00:01 | |
*** haomaiwang has joined #openstack-swift | 00:02 | |
clayg | we're the highest of A's | 00:06 |
---|---|---|
*** nadeem has quit IRC | 00:08 | |
clayg | After g = itemgetter(2, 5, 3), the call g(r) returns (r[2], r[5], r[3]) | 00:12 |
clayg | ^ I DID **NOT** KNOW THAT!? | 00:13 |
clayg | peluse: ^ who found that!? | 00:14 |
*** ccavanna has joined #openstack-swift | 00:19 | |
*** ccavanna_ has joined #openstack-swift | 00:22 | |
*** dmorita has joined #openstack-swift | 00:25 | |
*** ccavanna has quit IRC | 00:26 | |
peluse | you mean my use of itemgetter in that patch? | 00:29 |
peluse | if so that'd be me (with a little help from my friend google) | 00:31 |
*** jasondotstar has quit IRC | 00:33 | |
clayg | peluse: I think I could use some unittests in unit.obj.test_server that describe the use and behiavor of this object server api wrt to the new x-backend-preferred-frag-index request header and the x-backend-obj-alt-frag-index response header | 00:35 |
clayg | peluse: I know that ultimately it's the diskfile that's doing all the work and the object server is just brokering them in - but I think it'd be a little easier to dive in if I could make a little black box model of the object server in my head before I dive into the proxy handling | 00:36 |
peluse | clayg, OK, will work on that tomorrow | 00:36 |
clayg | peluse: e.g. if an object server has 2, but the proxy says prefer 1 - it will respond 404 - but does that 404 include the alt-frag-indexes? | 00:37 |
peluse | gotcha | 00:37 |
clayg | well i say it will 404, last time I checked a diskfile passed a frag_index would raise if it doesn't have the frag | 00:37 |
peluse | I can add some tests there pretty easily I think... gotta go flip my chicken on the grill now though :) | 00:38 |
clayg | it's probably also worth considering if we should add the headers for HEAD requests - just for sanity and uniformity and out of band poking | 00:38 |
clayg | sure np - thanks | 00:39 |
*** bill_az_ has quit IRC | 00:40 | |
notmyname | stopping in for a bit. looks like I missed some excitement this afternoon | 00:54 |
notmyname | peluse: I certainly hope they consider swift to be HA. | 00:58 |
notmyname | si1v3r_: what ctennis said about exposing the raw devices to swift is important. that's one of the main reasons to not use RAID volumes in a swift cluster | 00:59 |
notmyname | si1v3r_: the other big reason is that RAID rebuild times can take a really long time and seriously affect performance while it's happening | 01:00 |
*** haomaiwang has quit IRC | 01:01 | |
notmyname | si1v3r_: other reasons include the fact that swift doesn't lots of small, random reads and writes, and that can be pretty bad for RAID | 01:01 |
notmyname | si1v3r_: and once you have a whole RAID volume fail, then Swift has to work around it anyway, just with a larger failure domain and thus more data movement. so keeping the volumes smaller (ie a single drive), you smooth ou the hit from failed volumes | 01:02 |
*** haomaiwa_ has joined #openstack-swift | 01:02 | |
*** jasondotstar has joined #openstack-swift | 01:03 | |
notmyname | si1v3r_: also, since drives (and storage servers) tend to dominate the cost of a cluster, it doesn't generally make sense to add another $500 per server to the cluster. that ends up being a noticeable percentage of cost | 01:03 |
notmyname | si1v3r_: but beyond all that, sure you could use RAID in swift. there's nothing that would break. I think you'd do better without RAID, though | 01:05 |
peluse | notmyname, yeah that's what I was thining too but since I didn't see it mentioned I figured I'd better not "assume" :) | 01:05 |
notmyname | peluse: I saw "swift" mentioned in my buffer playback in one of the meeting channels. seems there's an HA group or something. but I couldn't find what they were referring to as the "Swift stuff" | 01:06 |
notmyname | peluse: how did that come to your attention? | 01:16 |
peluse | notmyname, a mktg guy here is working on a white paper about HA and openstack and asked me for some feedback, it only had cinder stuff mentioned. but its a paper on 'what needs work' so... | 01:27 |
peluse | and he referenced that PDF I linked above in the paper and I noticed it didn't mention Swift either | 01:27 |
notmyname | peluse: so in general, I don't know anything about what that doc is, who produced it, or who's it for. I only know that there is a group that occasionally has meetings (based on my IRC client highlighting it this afternoon) | 01:28 |
peluse | heh | 01:29 |
peluse | nothing to sweat, clearly :) | 01:29 |
notmyname | well, it's openstack, so these things tend to turn into "Rules You Must Now Follow or Explain Why You are Not" sort of things. | 01:30 |
notmyname | so I don't know if it's something to sweat or not ;-) | 01:30 |
peluse | we could rename the project to Swift-HA :) | 01:38 |
notmyname | or "Ha! Swift!" | 01:39 |
peluse | yes!! | 01:39 |
notmyname | ok, I gota run. I'll be back online later tonight | 01:39 |
peluse | later, I'm done for the eve... | 01:40 |
*** kota_ has joined #openstack-swift | 01:43 | |
*** ChanServ sets mode: +v kota_ | 01:43 | |
kota_ | good morning | 01:44 |
*** jkugel has joined #openstack-swift | 01:51 | |
*** haomaiwa_ has quit IRC | 02:01 | |
*** haomaiwang has joined #openstack-swift | 02:02 | |
openstackgerrit | janonymous proposed openstack/swift: Adding bandit for security static analysis testing in swift https://review.openstack.org/196395 | 02:08 |
*** bapalm has joined #openstack-swift | 02:09 | |
mattoliverau | kota_: morning | 02:14 |
kota_ | mattoliverau: morning :) | 02:14 |
*** jrichli has joined #openstack-swift | 02:16 | |
*** bapalm has quit IRC | 02:19 | |
*** openstackgerrit has quit IRC | 02:31 | |
*** openstackgerrit has joined #openstack-swift | 02:31 | |
*** hrou has joined #openstack-swift | 02:38 | |
kota_ | tdasilva: around? | 02:49 |
kota_ | oh no, EDT is already 10:50 p.m. I'm too late. | 02:50 |
kota_ | k, let's take a lunch to refresh my head. I might miss something. | 02:53 |
*** jasondotstar has quit IRC | 02:58 | |
*** sanchitmalhotra has joined #openstack-swift | 02:58 | |
*** jkugel has quit IRC | 02:58 | |
*** haomaiwang has quit IRC | 03:01 | |
*** haomaiwang has joined #openstack-swift | 03:02 | |
*** sakaYK has joined #openstack-swift | 03:11 | |
*** proteusguy has joined #openstack-swift | 03:14 | |
*** sanchitmalhotra1 has joined #openstack-swift | 03:19 | |
*** sanchitmalhotra has quit IRC | 03:21 | |
*** jrichli has quit IRC | 03:25 | |
*** h00327910__ has quit IRC | 03:28 | |
*** sanchitmalhotra has joined #openstack-swift | 03:36 | |
*** sanchitmalhotra1 has quit IRC | 03:38 | |
*** proteusguy has quit IRC | 03:41 | |
notmyname | hello, world | 03:51 |
*** sanchitmalhotra1 has joined #openstack-swift | 03:57 | |
*** sanchitmalhotra has quit IRC | 03:59 | |
peluse | notmyname, is that your first program? | 03:59 |
*** haomaiwang has quit IRC | 04:01 | |
*** haomaiwang has joined #openstack-swift | 04:02 | |
*** ktsuyuzaki has joined #openstack-swift | 04:04 | |
*** kota_ has quit IRC | 04:06 | |
*** kota_ has joined #openstack-swift | 04:07 | |
*** ChanServ sets mode: +v kota_ | 04:07 | |
*** haigang has joined #openstack-swift | 04:07 | |
*** ktsuyuzaki has quit IRC | 04:09 | |
notmyname | redbo: there was going to be a "ginger pride" day in san francisco, but it was cancelled http://www.breitbart.com/california/2015/07/30/redheads-gather-in-san-francisco-to-celebrate-ginger-pride-day/ | 04:09 |
*** kota_ has quit IRC | 04:11 | |
*** ho has quit IRC | 04:16 | |
*** ho has joined #openstack-swift | 04:19 | |
*** ho has quit IRC | 04:22 | |
*** sanchitmalhotra has joined #openstack-swift | 04:31 | |
*** ppai has joined #openstack-swift | 04:31 | |
*** sakaYK has quit IRC | 04:32 | |
*** sanchitmalhotra1 has quit IRC | 04:33 | |
*** nakagawamsa has joined #openstack-swift | 04:34 | |
openstackgerrit | Merged openstack/swift: Swift documentation for first-time contributors. https://review.openstack.org/206861 | 04:52 |
openstackgerrit | Merged openstack/python-swiftclient: swiftclient: add short options to help message https://review.openstack.org/202902 | 04:54 |
openstackgerrit | Merged openstack/swift: Remove error_suppression_interval, error_suppression_limit options. https://review.openstack.org/207296 | 04:54 |
*** sanchitmalhotra1 has joined #openstack-swift | 04:58 | |
*** haomaiwang has quit IRC | 05:01 | |
*** sanchitmalhotra has quit IRC | 05:01 | |
*** haomaiwang has joined #openstack-swift | 05:02 | |
*** sanchitmalhotra has joined #openstack-swift | 05:05 | |
*** sanchitmalhotra1 has quit IRC | 05:07 | |
*** SkyRocknRoll has joined #openstack-swift | 05:18 | |
*** SkyRocknRoll has joined #openstack-swift | 05:18 | |
*** kota_ has joined #openstack-swift | 05:19 | |
*** ChanServ sets mode: +v kota_ | 05:19 | |
*** ig0r_ has joined #openstack-swift | 05:24 | |
redbo | I had no idea there was going to be a ginger pride day. That's what I get for skipping the meetings. | 05:40 |
*** hrou has quit IRC | 06:00 | |
*** haomaiwang has quit IRC | 06:01 | |
*** 18WAAC7EF has joined #openstack-swift | 06:02 | |
*** silor has joined #openstack-swift | 06:03 | |
*** ig0r__ has joined #openstack-swift | 06:11 | |
*** ig0r_ has quit IRC | 06:12 | |
*** ppai has quit IRC | 06:19 | |
*** ig0r_ has joined #openstack-swift | 06:24 | |
openstackgerrit | Merged openstack/swift: Fix pep8 E265 warning of hacking 0.10 https://review.openstack.org/207242 | 06:24 |
openstackgerrit | Merged openstack/swift: Fix warning pep8 E128 warning of hacking 0.10 https://review.openstack.org/207243 | 06:24 |
openstackgerrit | Merged openstack/swift: Fix warning pep8 E128 warning of hacking 0.10 https://review.openstack.org/207243 | 06:24 |
*** ig0r_ has quit IRC | 06:24 | |
openstackgerrit | Merged openstack/swift: Fix pep8 E warning for hacking 0.10 https://review.openstack.org/207237 | 06:24 |
openstackgerrit | Merged openstack/swift: Update hacking to 0.10.0 https://review.openstack.org/205977 | 06:24 |
*** mahatic has joined #openstack-swift | 06:24 | |
*** ppai has joined #openstack-swift | 06:28 | |
*** sakaYK has joined #openstack-swift | 06:29 | |
*** sanchitmalhotra1 has joined #openstack-swift | 06:43 | |
*** sakaYK has quit IRC | 06:43 | |
*** sanchitmalhotra has quit IRC | 06:45 | |
*** sakaYK has joined #openstack-swift | 06:54 | |
*** SkyRocknRoll has quit IRC | 06:56 | |
openstackgerrit | Merged openstack/swift: Support keystone v3 domains in swift-dispersion https://review.openstack.org/195131 | 06:59 |
*** 18WAAC7EF has quit IRC | 07:01 | |
*** haomaiwang has joined #openstack-swift | 07:01 | |
*** sanchitmalhotra has joined #openstack-swift | 07:07 | |
*** sanchitmalhotra1 has quit IRC | 07:09 | |
*** rledisez has joined #openstack-swift | 07:11 | |
*** silor has quit IRC | 07:11 | |
*** zaitcev has quit IRC | 07:24 | |
*** geaaru has joined #openstack-swift | 07:27 | |
*** SkyRocknRoll has joined #openstack-swift | 07:27 | |
onovy | hi, https://review.openstack.org/#/c/207322/ can someone explain me why py3 gate is failing? | 07:42 |
*** sakaYK has quit IRC | 07:45 | |
haypo | onovy: it looks like a test failed on Python 3, http://logs.openstack.org/22/207322/2/check/gate-python-swiftclient-python34/4e8af41/testr_results.html.gz | 07:50 |
haypo | onovy: the call with prefix_b was not done | 07:51 |
*** sakaYK has joined #openstack-swift | 07:56 | |
*** marzif_ has joined #openstack-swift | 07:57 | |
*** mahatic has quit IRC | 07:59 | |
*** haomaiwang has quit IRC | 08:01 | |
*** haomaiwa_ has joined #openstack-swift | 08:02 | |
*** sanchitmalhotra1 has joined #openstack-swift | 08:08 | |
onovy | but i'm not changing that file at all | 08:09 |
onovy | hmm, file is same, but function not | 08:09 |
*** marzif_ has quit IRC | 08:09 | |
*** sanchitmalhotra has quit IRC | 08:10 | |
acoles | good morning | 08:13 |
*** joeljwright has joined #openstack-swift | 08:16 | |
*** ChanServ sets mode: +v joeljwright | 08:16 | |
*** jasondotstar has joined #openstack-swift | 08:19 | |
kota_ | acoles: good morning | 08:29 |
acoles | hi kota_ | 08:30 |
*** jistr has joined #openstack-swift | 08:32 | |
*** ppai has quit IRC | 08:37 | |
*** marzif_ has joined #openstack-swift | 08:40 | |
*** marzif_ has quit IRC | 08:41 | |
*** sakaYK has quit IRC | 08:42 | |
*** marzif_ has joined #openstack-swift | 08:42 | |
*** ppai has joined #openstack-swift | 08:45 | |
*** ppai has quit IRC | 08:54 | |
*** sakaYK has joined #openstack-swift | 08:58 | |
*** jordanP has joined #openstack-swift | 08:59 | |
*** mahatic has joined #openstack-swift | 08:59 | |
*** haomaiwa_ has quit IRC | 09:01 | |
*** aix has joined #openstack-swift | 09:02 | |
*** haomaiwang has joined #openstack-swift | 09:02 | |
*** dmorita has quit IRC | 09:07 | |
*** ppai has joined #openstack-swift | 09:07 | |
acoles | peluse: cool. i don't know how to change 'owner' of patch 207165 back to you. but you and clayg are still the authors afaict | 09:09 |
patchbot | acoles: https://review.openstack.org/#/c/207165/ | 09:09 |
*** sakaYK has quit IRC | 09:12 | |
openstackgerrit | Nakagawa Masaaki proposed openstack/swift: SAIO on Ubuntu 14.04 can not pass unittest. https://review.openstack.org/207790 | 09:16 |
*** sanchitmalhotra has joined #openstack-swift | 09:20 | |
*** sanchitmalhotra1 has quit IRC | 09:22 | |
*** sakaYK has joined #openstack-swift | 09:23 | |
acoles | notmyname: looks like there is a 'gap' in the review dashboard, reviews with a +2 but no + vote from jenkins disappear?? e.g. patch 207322 | 09:24 |
patchbot | acoles: https://review.openstack.org/#/c/207322/ | 09:24 |
*** sanchitmalhotra1 has joined #openstack-swift | 09:26 | |
*** sanchitmalhotra has quit IRC | 09:28 | |
tamizh_geek | Hey cschwede, can you please take a look at the comments in https://review.openstack.org/#/c/204196/ ? | 09:37 |
*** joeljwright1 has joined #openstack-swift | 09:40 | |
*** joeljwright1 has quit IRC | 09:40 | |
*** joeljwright has quit IRC | 09:42 | |
onovy | don't get it. patch 207322 after recheck, another gate failed :) | 09:50 |
patchbot | onovy: https://review.openstack.org/#/c/207322/ | 09:50 |
*** akle has joined #openstack-swift | 09:51 | |
*** ppai has quit IRC | 09:51 | |
acoles | onovy: looks like same test failed but in py26 rather than py34 job? | 09:54 |
acoles | onovy: the failing test expects two calls to a method, and cursory inspection suggests those calls are made in separate threads. Clearly nothing to do with your patch! | 09:56 |
acoles | onovy: maybe an intermittent race bug? i wonder if timburke can help when he wakes up? | 09:57 |
cschwede | tamizh_geek: done :) | 09:57 |
acoles | timburke: patch 207322 failing test_upload_delete_dlo_segments http://logs.openstack.org/22/207322/2/check/gate-python-swiftclient-python26/0dda652/testr_results.html.gz | 09:58 |
patchbot | acoles: https://review.openstack.org/#/c/207322/ | 09:58 |
tamizh_geek | cschwede: Thanks. Now, do I need to wait for another +2 from a core dev? | 09:58 |
*** silor has joined #openstack-swift | 09:59 | |
mahatic | acoles: Hello! | 09:59 |
acoles | mahatic: hi! | 10:00 |
cschwede | tamizh_geek: yes, the second core is then going to add another +2 and a +A (approval). of course only if he is fine with the patch too ;) | 10:00 |
tamizh_geek | cschwede: yeah yeah. thanks :) will be my first patch to openstack! | 10:01 |
mattoliverau | Yay, its cschwede, acoles, and mahatic! Good evening all | 10:01 |
cschwede | tamizh_geek: great, so welcome and thanks for the contribution to openstack! | 10:02 |
cschwede | mattoliverau: Hello Matthew! | 10:02 |
mahatic | acoles: I see that there are unit tests for encrypter/decrypter on PUT POST etc. Could you tell me what the probe test, for instance on PUT object supposed to be doing? ^ | 10:02 |
mahatic | mattoliverau: good evening! | 10:03 |
acoles | mattoliverau: hey you are still alive? | 10:03 |
mahatic | :D | 10:03 |
mattoliverau | Yeah, feeling great as of this arvo, been quite a week :) | 10:04 |
acoles | mattoliverau: are you watching the ashes???? | 10:04 |
acoles | mattoliverau: that will cheer you up :P | 10:04 |
mattoliverau | acoles: don't remind me, you guys are doing too well this test | 10:04 |
mahatic | lol | 10:05 |
acoles | mahatic: so an idea for a probe test for a PUT with encryption would be to PUT an object then get the object file direct from backend and check that it is encrypted, and check all the metadata is encrypted etc. | 10:06 |
*** ppai has joined #openstack-swift | 10:06 | |
acoles | mattoliverau: it'll swing back the other way next game, and this one ain't over yet! | 10:06 |
acoles | mahatic: BUT any such probe test would rely on the encryption middleware being in the proxy pipeline so you'd need to make it check the /info and skip if no encrypter/decrypter | 10:07 |
acoles | mattoliverau: anyways glad you're feeling better | 10:08 |
mahatic | acoles: After setting up encryption, I'm finally seeing the errors and failures that are expected as jrichli pointed! | 10:08 |
mahatic | acoles: okay. I'll look into that | 10:08 |
acoles | mahatic: great ! (?) ;) | 10:08 |
mahatic | acoles: :D well, yes. yay, same failures and errors! | 10:09 |
acoles | mahatic: so at some point we'll need to think about some kind of "default" or sample config for testing crypto e.g. keys, so that a probe test could independently calculate a cipher text that it expects to see on backend obj servers | 10:10 |
acoles | mahatic: consistency is a good thing!) | 10:10 |
*** moo41 has joined #openstack-swift | 10:11 | |
*** sakaYK has quit IRC | 10:12 | |
moo41 | Hi all. Is anybody here who can give me a hint, how I can make an exception within my middleware component visible? I just get a HTTP 500 back and cannot see anything in the log.... | 10:12 |
moo41 | I have already added catch_errors to the begin of the pipeline without any change | 10:13 |
*** silor1 has joined #openstack-swift | 10:13 | |
onovy | acoles, thanks | 10:14 |
*** SkyRocknRoll has quit IRC | 10:16 | |
*** silor has quit IRC | 10:16 | |
*** silor1 is now known as silor | 10:16 | |
mahatic | acoles: I see. So the current unit test, "test_basic_put_req" (in test_encrypter.py) isn't really testing encryption. That is on a TODO | 10:17 |
mahatic | acoles: When I add encrypt/decrypt check on probe test, it will be similar to unit test with the TODO I think. Or am I missing something completely here? | 10:19 |
acoles | mahatic: which patch in that unit test in? | 10:20 |
mahatic | acoles: https://review.openstack.org/#/c/203454 | 10:20 |
acoles | mahatic: k, looking | 10:21 |
mahatic | okay | 10:21 |
*** jasondotstar has quit IRC | 10:22 | |
acoles | mahatic: yes. looks like that test is checking nothing blows up but the TODO needs to be implemented - maybe by having FakeApp stash the object body and then comparing that with the expected value. | 10:24 |
mahatic | acoles: okay. So that goes into the unit test? and not probe? Should I be looking into adding that check in the unit test first? | 10:25 |
acoles | mahatic: jrichli may think otherwise but yes implementing that check in unit tests first would seem a good thing to do and will be easier (i'd hope!) than a probe test | 10:28 |
acoles | mahatic: but check with jrichli, she is more familiar with the code (i'm trying to catch up!) | 10:29 |
mahatic | acoles: :) sure. I also think that would be a relatively easier step to begin with. But will check with her | 10:32 |
*** joeljwright has joined #openstack-swift | 10:34 | |
*** ChanServ sets mode: +v joeljwright | 10:34 | |
onovy | zigo, hi, are you there? | 10:34 |
*** jasondotstar has joined #openstack-swift | 10:40 | |
openstackgerrit | Victor Stinner proposed openstack/swift: Fix tox -e py34 https://review.openstack.org/199034 | 10:42 |
haypo | WARNING: Python 3 is coming! ------------^ | 10:44 |
haypo | notmyname: ok, i think that we know have all requirements to push this stuff ^^ | 10:44 |
haypo | notmyname: FYI i contributed to PyEClib to port it to Python 3, but also to fix their embedded version of liberasurecode to ease installation (fix how it's installed and detected in setup.py) | 10:45 |
haypo | notmyname: they are running last tests on PyEClib 1.0.8 and they found a last bug on Centos (Tushar has a fix) | 10:46 |
haypo | so the tempoary hack in tox.ini for PyEClib will go away quickly | 10:46 |
*** kota_ has quit IRC | 10:49 | |
*** sakaYK has joined #openstack-swift | 10:54 | |
*** sakaYK has quit IRC | 10:57 | |
*** sakaYK has joined #openstack-swift | 10:58 | |
*** jasondotstar has quit IRC | 11:01 | |
*** jasondotstar has joined #openstack-swift | 11:03 | |
*** akle|2 has joined #openstack-swift | 11:08 | |
*** kei_yama has quit IRC | 11:13 | |
*** ig0r__ has quit IRC | 11:15 | |
*** akle has left #openstack-swift | 11:15 | |
*** jasondotstar has quit IRC | 11:17 | |
*** nakagawamsa has quit IRC | 11:17 | |
*** sakaYK has quit IRC | 11:18 | |
*** ig0r_ has joined #openstack-swift | 11:18 | |
*** ig0r__ has joined #openstack-swift | 11:19 | |
*** kei_yama has joined #openstack-swift | 11:19 | |
*** akle|2 has quit IRC | 11:22 | |
*** robefran has quit IRC | 11:29 | |
onovy | thinking about one thing: why not put packagging information directly into official openstack-swift git repo? for example 'debian' directory | 11:29 |
*** sanchitmalhotra1 has quit IRC | 11:38 | |
openstackgerrit | Merged openstack/swift: go: fix expecttransport memory usage https://review.openstack.org/206276 | 11:46 |
*** marzif_ has quit IRC | 12:05 | |
*** joeljwright has quit IRC | 12:06 | |
*** marcusvrn has joined #openstack-swift | 12:06 | |
*** joeljwright has joined #openstack-swift | 12:21 | |
*** ChanServ sets mode: +v joeljwright | 12:21 | |
*** ig0r__ has quit IRC | 12:22 | |
*** jasondotstar has joined #openstack-swift | 12:23 | |
*** haigang has quit IRC | 12:25 | |
*** silor has quit IRC | 12:27 | |
*** ig0r__ has joined #openstack-swift | 12:27 | |
*** thumpba has joined #openstack-swift | 12:28 | |
openstackgerrit | Merged openstack/swift: go: make replicateDevice more defensive https://review.openstack.org/207217 | 12:29 |
*** thumpba has quit IRC | 12:33 | |
*** bapalm has joined #openstack-swift | 12:42 | |
*** marzif_ has joined #openstack-swift | 12:54 | |
*** jasondotstar has quit IRC | 12:55 | |
*** marzif_ has quit IRC | 12:56 | |
*** marzif_ has joined #openstack-swift | 12:56 | |
*** hrou has joined #openstack-swift | 12:57 | |
*** jasondotstar has joined #openstack-swift | 12:59 | |
*** km has quit IRC | 13:06 | |
*** kei_yama has quit IRC | 13:09 | |
*** jkugel1 has joined #openstack-swift | 13:16 | |
*** marzif_ has quit IRC | 13:17 | |
openstackgerrit | Merged openstack/swift: Refactor diskfile https://review.openstack.org/198429 | 13:22 |
openstackgerrit | Merged openstack/swift: Fix string formatting in dispersion cli command https://review.openstack.org/205326 | 13:23 |
*** annegentle has joined #openstack-swift | 13:33 | |
*** ig0r__ has quit IRC | 13:34 | |
*** ppai has quit IRC | 13:41 | |
*** jrichli has joined #openstack-swift | 13:42 | |
*** jlhinson has joined #openstack-swift | 13:48 | |
haypo | acoles, cschwede : hi. i'm going to holiday, i would be so happy to see this patch merged before leaving! https://review.openstack.org/#/c/199034/ "Fix tox -e py34" | 14:07 |
haypo | you can read the long history, or just notice the short "gate-swift-python34: SUCCESS" which is explicit enough, no? | 14:07 |
haypo | i had to cut a whole forest of bugs to prepare this one :-p (port PyEClib, upgrade hacking to 0.10 and so fix pep8 issues, upgrade pbr to 1.3, add dnspython3 dependency, etc.) | 14:09 |
cschwede | haypo: looking | 14:09 |
cschwede | haypo: so thee hack in https://review.openstack.org/#/c/199034/11/tox.ini for pyeclib is still required? | 14:11 |
haypo | cschwede: it's still needed | 14:12 |
cschwede | ok | 14:12 |
haypo | cschwede: i harass PyEClib developers everydays, and i sent many patches to fix their issues (not related to py3 anymore, more to packaging/deploy/install code) | 14:12 |
haypo | cschwede: i expect a release in a few days, maybe it will take 1 month | 14:13 |
haypo | cschwede: "We believe that the core functionality of the library is stable (i.e. erasure coding), but dealing with platform-specific stuff is always tricky and fickle." https://bitbucket.org/kmgreen2/pyeclib/issues/69/need-a-release-108-with-python-3-fixes | 14:14 |
mahatic | jrichli: good morning! | 14:18 |
jrichli | mahatic: good morning! | 14:19 |
jrichli | mahatic: did you get past the pip issues? | 14:20 |
mahatic | jrichli: did you happen to see the scrollback? I was wondering if I could start with | 14:20 |
jrichli | mahatic: no, i just got in a bit ago, and was going through email. scrollback next :-) | 14:20 |
mahatic | sure! :) | 14:21 |
mahatic | lemme wait then ;) | 14:21 |
haypo | cschwede: so do you have an opinion on this change? :) | 14:22 |
*** bapalm_ has joined #openstack-swift | 14:23 | |
*** h00327910__ has joined #openstack-swift | 14:23 | |
*** Fin1te has joined #openstack-swift | 14:24 | |
cschwede | haypo: yes. just added my +2 | 14:25 |
cschwede | :) | 14:25 |
*** annegentle has quit IRC | 14:26 | |
*** bapalm has quit IRC | 14:26 | |
haypo | cschwede: cool! | 14:27 |
haypo | cschwede: "For the skipped rfc822 import on Python 3, see for example the change Ia5ee2ead67e36e8c6416183667f64ae255887736 which replaces the rfc822 module with the email module." see my comment before yours ;) | 14:28 |
cschwede | haypo: i saw that :) | 14:28 |
jrichli | mahatic: the unit test for the encrypter has a mocked encryption. The TODO is to verify that that the encryption was envoked, but it is not integrated with the real encryption. | 14:28 |
jrichli | acoles: was part of the intent of the probetest to test with the real encryption? | 14:28 |
*** tdasilva has quit IRC | 14:29 | |
*** acoles has quit IRC | 14:29 | |
mahatic | jrichli: so that unit test is not supposed verify the real encryption? and any real encryption test goes into probe? | 14:30 |
*** moo41 has quit IRC | 14:31 | |
jrichli | mahatic: the first statement there is correct. the other one - not sure what is typical. I would have thought maybe a functest for that integration, but maybe its a probetest. waiting on acoles to express an opinion :-) | 14:31 |
mahatic | jrichli: yes, okay :) | 14:32 |
jrichli | mahatic: there is a unit test for the real crypto class itself. | 14:32 |
hrou | jrichli, mahatic - the difference between probe / func confused me as well ; ) From what I gather probe are more end to end usage scenario (maybe more like SVT testing, for those coming from a more traditional dev model), whereas FVT tends to target a very specific use case. But I could be off base. | 14:35 |
jrichli | hrou - my understanding is that the functests are testing a flow from an API call. The other big thing is that the functests run with your SAIO. So, if we add a PUT for testing encrypter, it would be skipped if that is not in the pipeline. | 14:38 |
jrichli | hrou - my understanding of the probetests is that they test more complicated things under the hood like replication. they require more than the SAIO and will start and stop processes | 14:39 |
*** lcurtis has joined #openstack-swift | 14:39 | |
jrichli | so, encryption tests can fit in either case. but I wanted acoles's opinion in this particular case because he had written a trello card for a probetest | 14:40 |
hrou | jrichli, thanks ! That makes sense, that's a good point re: SAIO by default it wouldn't run anything related to encryption (so I think there's a precedence there as I think there are other examples of middelware that aren't in the pipeline by default, but have func tests). | 14:41 |
mahatic | jrichli: right, I'm looking at it (the crypto class). and thanks! ( for the info tests) | 14:44 |
mahatic | info on tests* | 14:44 |
jrichli | mahatic: I haven't yet looked into the errors given on the existing probetests. If you wanted to start digging into those, it would be a big help. Record your findings on the etherpad. | 14:45 |
mahatic | jrichli: okay. But now that you mention probe tests require more than an SAIO, it got me thinking ;) | 14:46 |
jrichli | mahatic: I may have over-summarized there - you don't need to setup more than the SAIO for them to run. | 14:47 |
mahatic | jrichli: okay! will look into them | 14:48 |
*** pgbridge has quit IRC | 14:49 | |
*** breitz has quit IRC | 14:55 | |
*** breitz has joined #openstack-swift | 14:56 | |
*** annegentle has joined #openstack-swift | 14:57 | |
*** chlong has quit IRC | 14:57 | |
*** jasondotstar has quit IRC | 15:03 | |
*** pgbridge has joined #openstack-swift | 15:07 | |
*** thurloat is now known as thurloat_isgone | 15:08 | |
*** chlong has joined #openstack-swift | 15:10 | |
*** joeljwright has quit IRC | 15:18 | |
*** joeljwright has joined #openstack-swift | 15:19 | |
*** ChanServ sets mode: +v joeljwright | 15:19 | |
*** joeljwright has quit IRC | 15:19 | |
*** joeljwright has joined #openstack-swift | 15:19 | |
*** ChanServ sets mode: +v joeljwright | 15:19 | |
*** acoles_away has joined #openstack-swift | 15:26 | |
*** jasondotstar has joined #openstack-swift | 15:26 | |
*** tdasilva has joined #openstack-swift | 15:26 | |
*** acoles_away is now known as acoles | 15:26 | |
*** ChanServ sets mode: +v acoles | 15:26 | |
*** zaitcev has joined #openstack-swift | 15:28 | |
*** ChanServ sets mode: +v zaitcev | 15:28 | |
notmyname | good morning | 15:28 |
tdasilva | good morning! | 15:30 |
mahatic | good morning! | 15:31 |
tdasilva | notmyname: still in Seattle? | 15:44 |
notmyname | yup | 15:44 |
tdasilva | cool! you got some nice pics from there! | 15:45 |
notmyname | the weather has been very clear and warm, so it's been easy :-) | 15:46 |
notmyname | I think we may drive towards rainier tomorrow to see it up close | 15:46 |
tdasilva | nice! enjoy! | 15:52 |
notmyname | acoles: I think that patch isn't showing up (at the moment) because it got the +A vote, so it's not in the "needs another vote to land" section | 16:01 |
notmyname | but it is on the summary/overview dashboard (2nd link in the topic) | 16:01 |
notmyname | in the "in the gate queue" section | 16:01 |
haypo | notmyname: hey! nice pics on twitter yeah :) | 16:01 |
*** joeljwright has quit IRC | 16:01 | |
notmyname | but I think I can tweak that rul a little | 16:01 |
*** rledisez has quit IRC | 16:01 | |
notmyname | haypo: thanks | 16:01 |
haypo | notmyname: just before i leave for ~1 month of holiday, can i get a +2a here? :-D https://review.openstack.org/#/c/199034/ | 16:02 |
haypo | notmyname: my patch to fix py34 gate | 16:02 |
haypo | well, in fact, there is no major urgency, but someone may have to update my patch later | 16:04 |
zaitcev | it's okay | 16:04 |
haypo | (i hate timezones, i don't understand why the whole world don't live in my timezone :-D) | 16:04 |
haypo | notmyname: do you live at seattle? or is it just for holiday? | 16:04 |
notmyname | haypo: when are you leaving? | 16:04 |
zaitcev | My god, that sed | 16:04 |
haypo | i'm going to seattle next month for the KVM Forum with my Red Hat team | 16:05 |
haypo | zaitcev: \o/ | 16:05 |
notmyname | haypo: no, I live in san francisco. I'm here for the week because my son is going to an FC Barca soccer camp | 16:05 |
haypo | notmyname: ah, soccer, cool :) it's less popular in the USA than in europe | 16:05 |
haypo | people are crazy about soccer here. our "local" team in the the "OM", Olympique de Marseille | 16:06 |
haypo | (not as popular as Barca? well, i don't know) | 16:06 |
*** jasondotstar has quit IRC | 16:06 | |
haypo | zaitcev: the sed command should not stay for long. PyEClib 1.0.8 is "almost" there | 16:06 |
haypo | zaitcev: i sent my pull request in april. it takes times for some projects to merge pull requests and then cook a release. the sed is a compromise to not have to wait for an external dependency | 16:08 |
notmyname | I'm working on (ie it's on my todo list) to get pyeclib more manageable for us | 16:08 |
haypo | notmyname: they dropped jerasure & gf-complete tarballs, but there is still a liberasurecode tarball | 16:10 |
haypo | i don't like embedding dependencies :-( but at least, i have to make it work correctly on my fedora 22 | 16:10 |
haypo | oops | 16:10 |
haypo | i helped* to make it | 16:10 |
haypo | sorry, i really have to go. see you maybe later. bye | 16:14 |
*** nadeem has joined #openstack-swift | 16:15 | |
*** nadeem has quit IRC | 16:16 | |
*** jlhinson_ has joined #openstack-swift | 16:16 | |
*** nadeem has joined #openstack-swift | 16:16 | |
*** jlhinson has quit IRC | 16:17 | |
*** mahatic has quit IRC | 16:18 | |
*** Fin1te has quit IRC | 16:20 | |
*** ig0r__ has joined #openstack-swift | 16:21 | |
notmyname | on the review dashboard, do you want to see your own patches that only need one more +2 or do you want those filtered out? currently those are filtered out | 16:22 |
notmyname | acoles: ^ | 16:22 |
*** Fin1te has joined #openstack-swift | 16:28 | |
*** jlhinson_ has quit IRC | 16:37 | |
acoles | notmyname: no. having own patches filtered is cool. i have my own dashboard ;) | 16:38 |
notmyname | :-) | 16:38 |
acoles | notmyname: re the 'missing' one - I hadn't +A'd that patch when i noticed it was missing on dashboard, it had +2 and -1 from jenkins | 16:38 |
notmyname | ah, ok | 16:38 |
acoles | it no big deal | 16:38 |
notmyname | I see that filter. I've taken it out | 16:38 |
acoles | ok cool | 16:39 |
acoles | jrichli: yes re probe test | 16:39 |
acoles | jrichli: but can't a unit test also test 'real' encryption? by grabbing the body output from the encrypter? | 16:40 |
acoles | jrichli: mahatic was asking about the TODO in an encrypter unit test saying 'somehow verify encryption' or words to that effect | 16:40 |
*** robefran has joined #openstack-swift | 16:41 | |
jrichli | acoles: i was just thinking that maybe it should check that the crypto method had been called. but that uses a mock for the crypto class since it has its own unit tests | 16:42 |
acoles | jrichli: oic | 16:42 |
*** jistr has quit IRC | 16:42 | |
tdasilva | notmyname: cool! didn't know your son was a soccer fan...did he get to hangout with barca players? | 16:42 |
tdasilva | or just coaches? | 16:43 |
acoles | jrichli: imho the more coverage in unit tests the better since we don't run probe tests in a jenkins job | 16:43 |
acoles | notmyname: didn't barca play man utd in CA recently? | 16:43 |
jrichli | acoles: ok. but i think it will be good to keep the encrypter test ignorant of the crypto algorithm being used | 16:43 |
*** briancline has joined #openstack-swift | 16:43 | |
notmyname | acoles: yup, they did | 16:44 |
acoles | jrichli: fair enough. so maybe another unit test that combines real encrypter and real crypto modules and verified the output? | 16:44 |
notmyname | tdasilva: this is part of the FCB escola thing (their training camp). there's a few coaches from spain here that are running the camp | 16:45 |
* portante brazil vs germany, 1-7 | 16:45 | |
acoles | notmyname: i just realised than man utd are back in champions league this season and swift meetings now conflict with the end of the matches :/ | 16:45 |
jrichli | acoles: ok. we will need to think of a way to verify the real crypto without being crytpo alg specific | 16:45 |
acoles | portante: hi! | 16:45 |
portante | acoles: hello | 16:45 |
portante | just poking jabs at tdasilva here | 16:46 |
portante | ;) | 16:46 |
acoles | portante: they don't play soccer in brazil do they ? ;) | 16:46 |
tdasilva | :P | 16:46 |
portante | =) | 16:46 |
tdasilva | portante: 5 | 16:46 |
notmyname | jrichli: assertFalse(any(word in word_list for words in cyphertext.split())) | 16:46 |
*** marzif_ has joined #openstack-swift | 16:47 | |
acoles | jrichli: right | 16:48 |
jrichli | notmyname: sounds good! | 16:48 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 16:48 |
notmyname | jrichli: tests pass! ship it! | 16:48 |
jrichli | notmyname: and if they dont pass, just remove the test ;-) | 16:49 |
jrichli | bad test! | 16:49 |
*** amit213 has quit IRC | 16:50 | |
*** amit213 has joined #openstack-swift | 16:50 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 16:50 |
openstackgerrit | janonymous proposed openstack/swift: bin directory: :Replace python print operator with print function (pep H233, py33) https://review.openstack.org/206602 | 16:55 |
notmyname | Clint Dempsey just walked by the kids' soccer camp and they all got a picture with him | 16:58 |
acoles | who? | 16:59 |
notmyname | (for those who don't follow soccer, he was the captain of the US team in the world cup) | 16:59 |
acoles | ;P | 16:59 |
notmyname | "...the first American player to score a hat-trick in the English Premier League" | 16:59 |
notmyname | https://en.wikipedia.org/wiki/Clint_Dempsey | 16:59 |
hrou | jrichli, yea validating something is encrypted is always fun, we had a similar issue with DB2 when we did our encryption work and used a similar solution to what notmyname described, but if you use a random IV and key .. ; ) | 16:59 |
tdasilva | acoles: lol | 16:59 |
notmyname | the soccer camp is at the same place where the seattle sounders practice | 17:00 |
* tdasilva notmyname just keeps setting himself up for more acoles sarcasm | 17:00 | |
acoles | tdasilva: notmyname nah my knife is actually sharpened for mattoliverau re cricket | 17:01 |
notmyname | acoles: oh, that's the game that takes 4 days to play and has tea-time breaks, right? | 17:01 |
*** Fin1te has quit IRC | 17:01 | |
acoles | mattoliverau: btw pycharm tip - to update the pep8 version pycharm runs you need to copy pep8.py into the helpers dir | 17:02 |
acoles | mattoliverau: at least that wfm on 3.4.1 community ed. ymmv | 17:02 |
acoles | mattoliverau: thats if you want pycharm to moan the same way that tox -e pep8 does with the newer hacking version | 17:03 |
acoles | notmyname: thats the game that has 5.5 ounces of hard ball hit you at >80mph if you dither | 17:03 |
acoles | but the teas are good! | 17:03 |
acoles | notmyname: https://s1.yimg.com/bt/api/res/1.2/kpfrfPafsRY2Pkc12ZiTRg--/YXBwaWQ9eW5ld3M7Zmk9aW5zZXQ7aD0zNTM7aWw9cGxhbmU7cT03NTt3PTYzMA--/http://media.zenfs.com/en_us/News/Reuters/2014-08-09T130609Z_1243708996_GM1EA891MIJ01_RTRMADP_3_CRICKET-INDIA.JPG | 17:05 |
notmyname | acoles: https://www.youtube.com/watch?v=dEH4ahCCrJo | 17:06 |
openstackgerrit | Alistair Coles proposed openstack/swift: Add POST capability to ssync for .meta files https://review.openstack.org/138498 | 17:06 |
tdasilva | acoles: acoles you just reminded me of this: https://www.youtube.com/watch?v=fzqsqBXMy7g | 17:07 |
acoles | lol | 17:07 |
acoles | tdasilva: but he's wearing a glove! | 17:08 |
tdasilva | acoles: would you do that wearing a glove? | 17:08 |
*** chlong has quit IRC | 17:09 | |
*** annegentle is now known as theannegentle | 17:09 | |
*** jrichli has quit IRC | 17:09 | |
acoles | clayg: i finally got patch 138498 updated & ready for review (ssync/post) | 17:10 |
patchbot | acoles: https://review.openstack.org/#/c/138498/ | 17:10 |
*** jrichli has joined #openstack-swift | 17:11 | |
acoles | tdasilva: https://www.youtube.com/watch?v=E_ANBewmvhA | 17:12 |
tdasilva | acoles: wow! touché! | 17:13 |
*** SkyRocknRoll has joined #openstack-swift | 17:17 | |
*** theannegentle has quit IRC | 17:19 | |
*** theannegentle has joined #openstack-swift | 17:20 | |
*** acoles is now known as acoles_away | 17:24 | |
*** ig0r__ has quit IRC | 17:24 | |
openstackgerrit | Thiago da Silva proposed openstack/swift: versioned writes middleware https://review.openstack.org/134347 | 17:25 |
*** jasondotstar has joined #openstack-swift | 17:25 | |
tdasilva | clayg: around? when you have a chance please checkout latest comments on patch 134347. I think upgrade path got a bit more complicated | 17:26 |
patchbot | tdasilva: https://review.openstack.org/#/c/134347/ | 17:26 |
*** ChanServ changes topic to "Review Dashboard: https://goo.gl/eqeGwE | Summary Dashboard: https://goo.gl/jL0byl | Hackathon topics: https://etherpad.openstack.org/p/swift-midcycle-aug-2015 | https://etherpad.openstack.org/p/swift_encryption_issues | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/" | 17:27 | |
openstackgerrit | Merged openstack/python-swiftclient: Test auth params together with --help option. https://review.openstack.org/207322 | 17:33 |
*** theannegentle has quit IRC | 17:45 | |
*** aix has quit IRC | 17:54 | |
*** robefran has quit IRC | 18:03 | |
*** theannegentle has joined #openstack-swift | 18:11 | |
*** breitz has quit IRC | 18:12 | |
*** Fin1te has joined #openstack-swift | 18:14 | |
*** jordanP has quit IRC | 18:21 | |
*** geaaru has quit IRC | 18:22 | |
*** marzif_ has quit IRC | 18:25 | |
clayg | hey-oh | 18:49 |
clayg | tdasilva: no, i'll only look at if you say it got *better* | 18:49 |
* haypo is back | 18:50 | |
haypo | timburke: for pbr, it's possible to specify the minimum pbr version in setup.py | 18:50 |
timburke | haypo: yes. we don't currently, but if we're going to use env markers, we should | 18:51 |
haypo | timburke: but i don't know what's the best option | 18:51 |
haypo | lifeless is our local pbr expert | 18:51 |
* haypo invokes lifeless | 18:51 | |
haypo | lifeless lifeless lifeless | 18:51 |
zaitcev | I run git-review on Python 3 and man, the side effects are hilarious. Can't wait to see Swift doing that. | 18:53 |
haypo | zaitcev: what are the side effects? | 18:54 |
zaitcev | haypo: git-review -v tracebacks immediately, because py3 has different exception variable scope | 18:55 |
zaitcev | UnboundLocalError: local variable 'no_git_dir' referenced before assignment | 18:55 |
haypo | zaitcev: ah? i'm using git review regulary in .tox/py34 venv | 18:55 |
zaitcev | haypo: git-review -l prints this: b'37781' b'master' b'Cache check mount calls' | 18:55 |
haypo | i mean, i'm running git review everydays, multiple times, in various projects | 18:56 |
haypo | and i never got your error | 18:56 |
zaitcev | of course | 18:56 |
haypo | zaitcev: i have git-review 1.25.0 | 18:56 |
zaitcev | only a complete n00b would run it without arguments and outside of git repository | 18:56 |
haypo | it's installed for python 2, not python 3 | 18:57 |
zaitcev | as long as you either run it in a repo, or provide it meaningful arguments, it will not traceback | 18:57 |
zaitcev | haypo: anyway, see https://review.openstack.org/200371 | 18:57 |
*** Fin1te has quit IRC | 18:57 | |
haypo | "or provide it meaningful arguments" which command makes failing git review? | 18:58 |
haypo | zaitcev: but yeah, it would be nice to have git-review working on python 3 ;) | 18:59 |
haypo | zaitcev: py26 & py27 failed on your patch :p) | 19:00 |
zaitcev | haypo: They failed because of mock 1.1 | 19:00 |
haypo | zaitcev: 1.1? mock 1.3 is out! | 19:01 |
zaitcev | haypo: well, the py26 did. The py27 failed because of something called "Gerrit WAR". | 19:01 |
zaitcev | haypo: well then, let's have infra guys install it and then my patch may go in | 19:01 |
haypo | zaitcev: i'm not sure that mock is in the part. it's installed by tox using requirements, no? | 19:02 |
zaitcev | haypo: I linked this when we discussed it previously, but https://bugs.launchpad.net/openstack-gate/+bug/1473286 | 19:04 |
openstack | Launchpad bug 1473286 in OpenStack-Gate "mock fails with SyntaxError in py26" [Undecided,Invalid] | 19:04 |
zaitcev | haypo: And either your or lifeless explained that they fixed it up in 1.4 | 19:05 |
zaitcev | because someone volunteered to do it | 19:05 |
haypo | zaitcev: i didn't work on mock | 19:06 |
haypo | but i helpd to fix swift (and some oslo projects) for mock 1.1 | 19:06 |
haypo | because mock become more strict and helped to find bugs in tets | 19:07 |
haypo | tests* | 19:07 |
zaitcev | I know. But that only worked because we do not support py26 anymore. 1.1 being dead in py26 is not something you can fix in a project code. The only thing a project can do is this: https://review.openstack.org/200860 | 19:08 |
openstackgerrit | Victor Stinner proposed openstack/swift: Fix tox -e py34 https://review.openstack.org/199034 | 19:11 |
haypo | timburke, zaitcev : ^^ i ran "tox -e update ../swift" in requirements. it added the requirements pbr>=1.3 in setup.py | 19:11 |
haypo | timburke: is it better? | 19:11 |
*** jasondotstar has quit IRC | 19:12 | |
*** ig0r__ has joined #openstack-swift | 19:12 | |
*** SkyRocknRoll has quit IRC | 19:14 | |
*** jasondotstar has joined #openstack-swift | 19:19 | |
timburke | haypo: hmm. maybe that wasn't sufficient. i expected http://paste.openstack.org/show/406627/ to Just Work, but despite installing a newer pbr, it doesn't seem to use it | 19:21 |
openstackgerrit | Michael Barton proposed openstack/swift: go: fix some panics in unpickle https://review.openstack.org/208033 | 19:22 |
*** jasondotstar has quit IRC | 19:25 | |
haypo | timburke: i heard that this is a ckicken-egg issue | 19:25 |
*** ig0r_ has quit IRC | 19:26 | |
haypo | timburke: but "tox -r -e py34" works for me, whats the use case? | 19:26 |
timburke | haypo: i believe it. but i still want a way to upgrade from a previous install | 19:26 |
haypo | timburke: the way is simple: tox -r -e py34 | 19:27 |
haypo | timburke: or rm -rf .tox | 19:27 |
haypo | timburke: does it work if you upgrade pbr to 1.3? pip install -U 'pbr>=1.3' | 19:28 |
haypo | timburke: you should only have to do this once | 19:28 |
haypo | but yeah, you dev environment may need some manual steps if you don't want to use tox -r | 19:29 |
tdasilva | zaitcev: hi, have you seen this error before: http://ur1.ca/nbh25 | 19:29 |
haypo | tdasilva: do you have Jerasure installed? | 19:29 |
timburke | haypo: no, i mean given an old deployment of swift, i should be able to install the new version. requiring users to upgrade pbr out-of-band feels busted to me | 19:29 |
haypo | tdasilva: oops, you don't have Jerasure :) you should install it | 19:30 |
timburke | haypo: it's not tox that i'm worried about. tox can sort itself out | 19:30 |
haypo | timburke: Red Hat packages don't use pbr :) | 19:30 |
zaitcev | tdasilva: Swift in Rawhide is super fragile and I think I made it build just once. Tons of issues like that. | 19:30 |
haypo | timburke: who deploy using raw source code? | 19:30 |
haypo | in a package, you have a full control on requirements | 19:31 |
zaitcev | haypo: Agile DevOps ninja rockstars all do. | 19:31 |
haypo | better than our fragile setup.py, no? | 19:31 |
timburke | haypo: fair enough. maybe it's my thinking that's busted | 19:31 |
haypo | zaitcev: lol | 19:31 |
haypo | i heard that people don't really upgrade clouds | 19:31 |
haypo | you spawn a new one and migrate VMs | 19:32 |
haypo | i was surprised, but well, it makes sense :) | 19:32 |
lifeless | zaitcev: but thats wrong | 19:32 |
haypo | sorry, i don't have any experience in deployment. or well, small experience | 19:32 |
*** openstack has joined #openstack-swift | 19:35 | |
*** ig0r__ has quit IRC | 19:35 | |
lifeless | timburke: so that pastebin isn't pbr | 19:35 |
lifeless | timburke: http://paste.openstack.org/show/406627/ - its setuptools being too old | 19:36 |
lifeless | timburke: you can add a setup_requires versioned dep on setuptools>=17.1 | 19:36 |
lifeless | timburke: but it won't make it automatically work, because setuptools doesn't handle upgrades to itself | 19:37 |
haypo | lifeless: ah yes, that's the chicken-egg issue | 19:37 |
lifeless | I'd like to do a lot better | 19:38 |
lifeless | and I have some prototype code to do so, but its still working through the experimental stage | 19:38 |
*** NM1 has joined #openstack-swift | 19:39 | |
haypo | lifeless: what's the impact for swift developers and swift deployers of upgrading pbr from 0.11 to 1.3? | 19:39 |
*** ig0r_ has joined #openstack-swift | 19:39 | |
haypo | developers should recreate their virtualenv, that's all, no? | 19:40 |
timburke | lifeless: ah, ok. so install setuptools>=17.1 and pbr<1.0, *then* running setup.py should use the deps it's installed and work? | 19:42 |
clayg | timburke: does that work? | 19:44 |
*** ig0r_ has quit IRC | 19:46 | |
openstackgerrit | Michael Barton proposed openstack/swift: go: fix timing issue in ring test https://review.openstack.org/208041 | 19:47 |
timburke | clayg: nope :( http://paste.openstack.org/show/406628/ | 19:48 |
timburke | lifeless: no, setuptools is apparently up-to-date (18.0.1) | 19:49 |
lifeless | timburke: so yeah, thats likely just another symptom of the inability of setuptools to 'fix' unsatisfied setup_requires :( | 19:51 |
lifeless | timburke: but - what is your setup.py here? | 19:51 |
lifeless | timburke: cn you pastebinit, or is there a review I can pull down? | 19:51 |
timburke | lifeless: https://review.openstack.org/#/c/199034/12/setup.py - basically, add a min pbr version to setup_requires | 19:52 |
ccavanna_ | Hi. I have a question about git. Hoping anyone can help me. I have a local branch I've been working on for a few days now (and commited changes). | 19:52 |
ccavanna_ | I am addressing some of the reviews at the moment. | 19:53 |
ccavanna_ | As part of that, I decided to run: git pull origin master | 19:53 |
ccavanna_ | (to get everything up to date) | 19:53 |
haypo | lifeless: ah, there is a question for you (me?) on https://review.openstack.org/#/c/199034/12/setup.py | 19:53 |
ccavanna_ | I changed my file (just a doc) | 19:53 |
haypo | zaitcev, lifeless : i don't know how pbr requirements should be specified, so i copied what was done in nova. in nova, pbr>=1.3 is in setup.py and requirements.txt | 19:54 |
haypo | i guess that it's required because of the chicken-egg issue | 19:54 |
zaitcev | haypo: I see. Okay | 19:54 |
clayg | timburke: but I don't understand - can you just pip install --upgrade setuptools pbr and then it works? | 19:54 |
openstackgerrit | Minwoo Bae proposed openstack/swift: Error handling of DiskFileNoSpace() https://review.openstack.org/198909 | 19:57 |
lifeless | timburke: oh btw - does pip install . do any better? | 20:01 |
haypo | zaitcev, timburke, clayg : i'm sorry, i didn' follow the discussion closely, i'm doing something else at the same time. did you fix you issues with my "tox -e py34" patch? https://review.openstack.org/#/c/199034/ | 20:03 |
clayg | lol - timburke oh yeah i forgot that part - you can't use python setup.py install anymore - it's all pip install . | 20:04 |
clayg | oh my favorite pip install -e . | 20:05 |
clayg | i can't even type python setup.py develop anymore - but I guess my build scripts are going to be pissed if I can't get python setup.py sdist to work | 20:05 |
clayg | I don't know how to spell sdist in pip - but i bet it's possible | 20:05 |
lifeless | python setup.py sdist :) | 20:09 |
lifeless | pip knows how to wheel | 20:09 |
lifeless | and how to install from an sdist | 20:10 |
lifeless | but not how to make an sdist | 20:10 |
lifeless | once we have declarative deps, it may make sense to make an sdist thing | 20:10 |
lifeless | or at least an install-my-build-deps | 20:10 |
lifeless | thing | 20:10 |
haypo | clayg: i never call setup.py myself, i always rely on tox. you don't use tox? | 20:11 |
clayg | to install the code in my development environment and virtualenvs? for swift or like other python things I work on? | 20:11 |
clayg | I used tox; but using it doesn't make me not run things like 'pythno setup.py sdist' or 'pip install -e .' sometimes | 20:11 |
haypo | clayg: you need maybe to write a "pbr survivor guide" and share it with other developers :) | 20:15 |
*** zhill has joined #openstack-swift | 20:16 | |
redbo | I'm glad it add so much value | 20:16 |
redbo | add* | 20:16 |
haypo | i still don't understand why we use pbr :) | 20:16 |
redbo | ack what's wrong with my s key | 20:16 |
haypo | but i'm not interested to work on this :-D | 20:16 |
redbo | nobody knows why we need it, but we're not allowed to get rid of it | 20:17 |
haypo | the nice thing is setup.cfg | 20:17 |
*** robefran has joined #openstack-swift | 20:17 | |
*** robefran has quit IRC | 20:17 | |
haypo | redbo: not allowed? | 20:17 |
redbo | I know! | 20:18 |
haypo | for my understanding, there is is not good packaging tool, pbr is maybe the least worst tool :) | 20:18 |
haypo | reminder: right now, i only care of python3 :) | 20:19 |
redbo | what does it really add? git versioning is about it as far as I can tell. | 20:19 |
haypo | redbo: but lifeless can maybe reply to your questions. he knows well pip, pbr, setuptools, tox, etc. | 20:19 |
haypo | redbo: it adds support for setup.cfg, packaging stuff written in a configure file, instead of a python program | 20:20 |
clayg | redbo: pbr was trying to work around shit that setuptools-0.6c did that was stupid I'm pretty sure | 20:20 |
haypo | redbo: it's better to use a configuration file than executing python code | 20:20 |
redbo | doesn't setup.cfg come from distuils? | 20:20 |
redbo | distutils | 20:20 |
haypo | redbo: no | 20:20 |
haypo | redbo: it comes from distutils2 which died in pain | 20:20 |
lifeless | setup.cfg is used by setuptools, but only minimally | 20:20 |
haypo | (and two other developers :-D) | 20:20 |
lifeless | its also used by wheel | 20:20 |
lifeless | and d2 | 20:21 |
lifeless | and pbr | 20:21 |
lifeless | => its a common config file | 20:21 |
openstackgerrit | Carlos Cavanna proposed openstack/swift: Improving statistics sent to Graphite. https://review.openstack.org/202657 | 20:21 |
redbo | okay | 20:22 |
clayg | redbo: lol! | 20:22 |
clayg | ^ AND THAT IS HOW THE MAGIC HAPPENS FOLKS | 20:22 |
redbo | I still don't think that's worth all the pain its put us through | 20:22 |
lifeless | redbo: pbr gets us versions in docs; declarative dependencies that can be machine introspected; git versioning; support for PEP-426 markers in fields that setuptools doesn't support yet | 20:22 |
haypo | redbo: you're free to propose something else | 20:22 |
redbo | I've tried | 20:22 |
clayg | lol | 20:23 |
haypo | redbo: but i don't think that you want to know the dark side of python packaging | 20:23 |
haypo | redbo: most developers who tried lost their mind, we lost them forever | 20:23 |
haypo | they are now ermits in high mountains | 20:23 |
clayg | haypo: that's also true! | 20:23 |
haypo | hum. maybe i exagerate sometimes | 20:24 |
redbo | Well you say that, but there's like 20,000 python libraries that don't use pbr and manage to get installed | 20:24 |
clayg | no no no - it's all true | 20:24 |
haypo | clayg: haha | 20:24 |
haypo | clayg: tarek ziade and eric araujo wrote distutils2. both left the python project | 20:24 |
haypo | eric also left france | 20:24 |
clayg | yeah yeah tarek - that's the guy | 20:25 |
haypo | i didn't see tarek since more than 2 years | 20:25 |
haypo | i'm not sure that he still exists! | 20:25 |
zaitcev | I went to Monty's talk once like 3 summits ago in Atlanta or Sun Antonio and basically he listed some issues with packaging that prompted PBR. But none of them applied to Fedora, so I forgot all of it. | 20:25 |
haypo | zaitcev :) | 20:25 |
clayg | sorry this is *so* therapeutic for a Friday | 20:26 |
haypo | ah, tarek is still alive :) http://blog.ziade.org/ | 20:26 |
lifeless | redbo: theres more like 10 projects with the developer base of swift | 20:26 |
lifeless | redbo: and maybe 2 with the developer base of openstack as a whole | 20:26 |
haypo | i just didn't met him since many years, he doesn't go to pycon fr anymore | 20:26 |
redbo | I'm sure the django project is about to fall apart with the overhead of maintaining their setup.py file | 20:27 |
haypo | redbo: for me, using a configuration file (setup.cfg) is more secure, more portable, and may be faster | 20:28 |
swifterdarrell | redbo: I think they still put the version in a file. like Neanderthals | 20:28 |
clayg | oh god - no it's too much - please stop | 20:28 |
clayg | ROFL | 20:28 |
haypo | redbo: i also like wheel packages. pip 7 helped a lot, it now builds automatically wheel transparently | 20:29 |
redbo | Okay, let's say we need *something* there. Does it have to be something that causes endless pain just to translate .cfg files to setup() arguments? | 20:31 |
haypo | redbo: ok, to discuss more seriouly, _all_ packaging tools are still based on distutils. pip is based on setuptools which is based on distutils | 20:32 |
clayg | redbo: I think you're transfering some of pain from new features in setuptools and pip to pbr - it *started* with pbr - but now thye're fixing *everything* - it's a new world! | 20:33 |
haypo | redbo: it became impossible to fix the underlying layer (distutils) because it would break too much things | 20:33 |
clayg | I remember back when python developers used to be into "move slow and just work" | 20:33 |
haypo | so we are f***ed | 20:33 |
clayg | move fast and break things | 20:33 |
* clayg pops off another six shooter | 20:33 | |
haypo | said differently, nobody wants to start a new fresh "distutils" project from scratch | 20:33 |
redbo | clayg: that may be true. Any time running seutp.py completely screws up my machine, I kind of blame pbr. | 20:34 |
clayg | redbo: yeah :) it was downhill from dist is not a Distribution | 20:35 |
haypo | clayg: it's impressive how much bugs and new cool features are now added to pip | 20:35 |
haypo | oops | 20:35 |
haypo | i mean, bugs are fixed, new features are added, sorry :-D | 20:35 |
haypo | redbo: pbr is a small piece. there are pip, setuptools and distutils behind | 20:36 |
zaitcev | Subject: Re: Transitioning Ceph from Autotools to CMake | 20:36 |
zaitcev | There is no escape | 20:36 |
redbo | clayg: Oh man, it still hasn't really even been all that long since I saw dist is not a Distribution. Then I google for it, and I just see your post to distutils mailing list or whatever. | 20:36 |
haypo | from my point of view, it's only become better | 20:36 |
clayg | haypo: every day is a little better than the next | 20:37 |
lifeless | redbo: running setup.py should never screw up your machine | 20:37 |
lifeless | redbo: pbr or not | 20:37 |
redbo | I agree | 20:38 |
lifeless | redbo: but sadly, easy_install pretty much guarantees that running setup.py directly will do so | 20:38 |
lifeless | :/ | 20:38 |
openstackgerrit | David Goetz proposed openstack/swift: Speed up reaper for a big account delete and some better error handling https://review.openstack.org/208057 | 20:38 |
*** bapalm_ has quit IRC | 20:38 | |
redbo | I don't know, lots of python stuff, I run setup.py and nothing too bad happens. | 20:39 |
zaitcev | I simply never run anything with sudo (don't even have sudo installed), and I never run setup.py as root. | 20:40 |
*** michaeljk has joined #openstack-swift | 20:40 | |
*** zhill has quit IRC | 20:41 | |
clayg | zaitcev: is stright up baller | 20:44 |
haypo | two years ago, i dind't understand the purpose of virtualenv | 20:45 |
haypo | now i'm only using it to develop | 20:45 |
haypo | it like being install to install whatever, modify code of libraries, hack pip, hack everyhing | 20:45 |
haypo | and then simply destroy it to rebuild a fresh virtualenv | 20:46 |
*** aix has joined #openstack-swift | 20:47 | |
redbo | But it's kind of like complaining about the weather. | 20:47 |
clayg | redbo: well if the weather is shitty all the time | 20:48 |
redbo | I know, it's a perfect metaphor | 20:48 |
haypo | don't forgot to review my patch :) python3 is not the future, but the present! | 20:49 |
clayg | hrmm.... it seems like there is an alternative future we could go with | 20:49 |
clayg | if the weather is so crappy then why don't you just move? | 20:50 |
lifeless | for instance.. https://bitbucket.org/kmgreen2/pyeclib/issues/62/pip-install-uninstall-install-errors-with | 20:50 |
clayg | too much fun | 20:50 |
redbo | I did! | 20:50 |
lifeless | thats the sort of thing setuptools/pip/pbr get blamed for | 20:50 |
zaitcev | "go with" rimshot | 20:50 |
lifeless | but nothing to do with them | 20:50 |
lifeless | (except insofar that they don't force real isolation etc... postels tarpit) | 20:50 |
redbo | as soon as there's a gbr, I'll just start over again in rust or something. | 20:52 |
* clayg snickers | 20:52 | |
haypo | lifeless: PyEClib setup.py is a little bit strange. it installs manually a C library (liberasurecode) using ./configure --prefix=/usr/local && make && make install | 20:53 |
clayg | acoles_away: thanks for doing the v3 dispersion change | 20:53 |
haypo | lifeless: did you notice my comment? it should now be fixed | 20:53 |
openstackgerrit | Michael Barton proposed openstack/swift: go: fix timing issue in ring test https://review.openstack.org/208041 | 20:54 |
lifeless | haypo: perhaps. easy enough to test | 20:54 |
haypo | lifeless: I added /usr/include/liberasurecode to paths used to locate C headers, and I modified --prefix=/usr/local with --prefix=/usr | 20:55 |
haypo | lifeless: (and embedded liberasurecode was upgrade from 1.0.7 to 1.0.8) | 20:55 |
lifeless | haypo: its not clear to me why that would fix it | 20:56 |
lifeless | haypo: anyhow - as i said, easy enough to test | 20:56 |
haypo | lifeless: it worked for me :) | 20:57 |
haypo | lifeless: the error is "erasurecode.h: No such file or directory". liberasurecode installs its headers in /usr/include/liberasurecode, but /usr/include/liberasurecode was not in "include dirs" | 20:59 |
*** jrichli has quit IRC | 21:00 | |
clayg | tdasilva: so i'm not 100% on what I need to look at on versioned writes - upgrades are working according to your fpaste yeah? | 21:01 |
haypo | lifeless: but i'm not sure that i understood your usecase :) | 21:02 |
clayg | tdasilva: oic - removing the container option blocks old containers - bah | 21:02 |
lifeless | haypo: I need to go get food n stuff | 21:04 |
*** michaeljk has left #openstack-swift | 21:06 | |
haypo | i have to go away from computers :) | 21:07 |
haypo | bye | 21:08 |
*** haypo has left #openstack-swift | 21:08 | |
*** changbl has quit IRC | 21:09 | |
*** NM1 has quit IRC | 21:10 | |
clayg | dfg_: does shard mean - each replica of the account will process the containers modulo? | 21:23 |
dfg_ | clayg: ya. shard is a word right? | 21:25 |
dfg_ | its basically my solution to everything. that and ratelimiting | 21:27 |
*** bapalm_ has joined #openstack-swift | 21:31 | |
hrou | Hey All, after picking up haypo's change to tox.ini (essentially changed it from a white list to a black list) anyone else seeing a bunch of failures in PEP8 with 'H402 one line docstring needs punctuation' ? | 21:34 |
*** DericHorn-HP has joined #openstack-swift | 21:41 | |
*** mahatic has joined #openstack-swift | 21:42 | |
hrou | We didn't use to check H402 so that would make sense, but I wagered someone would have ran into this already by now so maybe there's more to it : ) i.e. something I'm missing. | 21:44 |
*** mahatic has quit IRC | 21:44 | |
lifeless | hrou: did you also update your hacking ? | 21:51 |
lifeless | hrou: since that check came and went IIRC | 21:51 |
clayg | dfg_: yeah totally nice work! | 21:52 |
clayg | dfg_: I looked it over and had a few first impressions - but I haven't reviewed it yet because I have another stupid change checked out | 21:52 |
clayg | dfg_: do you want me to push my comments - or do you want me to wait till I run it? | 21:52 |
clayg | dfg_: it'll probably just be a little bit | 21:53 |
hrou | lifeless, ah that's probably it I'm on 0.8.1 | 21:55 |
*** bapalm_ has quit IRC | 21:55 | |
clayg | hrou: sorry bro - I think pip install --upgrade -r test-requirements.txt should get you goin? | 21:55 |
hrou | clayg, haypo, thanks a bunch, indeed it does ! Given its in test-requirments (i.e. the newer hacking version) that did the trick, err silly me | 21:57 |
clayg | yeah that + rebase on master should get you going - weee fun times! | 21:58 |
clayg | move fast and break things! | 21:59 |
clayg | yee haw! | 21:59 |
*** jkugel1 has left #openstack-swift | 22:03 | |
clayg | so cschwede's comment about "this test should have to change" got me thinking on patch 204196 | 22:17 |
patchbot | clayg: https://review.openstack.org/#/c/204196/ | 22:17 |
hrou | An FYI for all ( I should add this to that new doc btw ; )) ... you want to also do: "tox -r" to recreate the VM | 22:17 |
clayg | the only reason that test has to change is because it bypasses some swob machinery - if it just called it like anyone else swob would cook up a request if it didn't get one -> https://gist.github.com/clayg/1e8cbd0b90eb66eb3ae0#file-request-is-should-patch-L26 | 22:18 |
clayg | but then I was like - oh hrmm... swob is cooking up a request when you call it if you don't get one - i bet we do that all the time! -> https://gist.github.com/clayg/1e8cbd0b90eb66eb3ae0#file-request-is-should-patch-L10 | 22:18 |
clayg | then I was like "we should audit everywhere we return a Request and..." and basicalloy passed out from exahustion | 22:19 |
clayg | can ask the author of this stupid^Wnew feature to do this work - or do we just have to get ready for all the "this *one* swift error *doesn't* return json when I want it to!" bugs :'( | 22:20 |
clayg | well - idk what to tell him - but I don't want to +2 it anymore either :'( | 22:21 |
clayg | i'll do a +1 and see what happens | 22:21 |
*** lcurtis has quit IRC | 22:25 | |
*** nadeem has quit IRC | 22:25 | |
*** hrou has quit IRC | 22:33 | |
*** NM1 has joined #openstack-swift | 22:36 | |
*** minwoob_ has quit IRC | 22:37 | |
*** theannegentle has quit IRC | 22:48 | |
*** NM1 has quit IRC | 22:50 | |
clayg | what's the python for taking a list and grouping it into tuples? | 22:56 |
clayg | i know there's an itertools that does this... | 22:57 |
clayg | it's like the oppose of chain | 22:57 |
clayg | pairwise recipe I think! | 22:59 |
StevenK | zip() ? | 23:01 |
*** theannegentle has joined #openstack-swift | 23:01 | |
StevenK | Nope, that's for two lists. :-/ | 23:02 |
clayg | bah - no pair wise isn't it | 23:04 |
clayg | it's like a zip of an islice after you tee it | 23:04 |
*** theannegentle has quit IRC | 23:06 | |
clayg | zip(*[iter(info_lines)] * 2) | 23:09 |
clayg | yeah so making the list that all have the same reference to the iter for zip is pretty cleaver | 23:10 |
clayg | points go to torgomatic for pointing me at grouper | 23:10 |
clayg | don't forget to call iter tho - that trick doesn't work with lists | 23:10 |
clayg | (obvs.) | 23:10 |
*** DericHorn-HP has quit IRC | 23:22 | |
*** DericHorn-HP has joined #openstack-swift | 23:25 | |
*** jrichli has joined #openstack-swift | 23:42 | |
*** hrou has joined #openstack-swift | 23:43 | |
*** marcusvrn has quit IRC | 23:43 | |
openstackgerrit | Merged openstack/swift: Fix regression in WSGI server SIGHUP behavior https://review.openstack.org/207637 | 23:50 |
*** jrichli_ has joined #openstack-swift | 23:52 | |
*** jrichli has quit IRC | 23:53 | |
*** theannegentle has joined #openstack-swift | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!