| *** vint_bra has quit IRC | 00:33 | |
| *** sams-gleb has joined #openstack-swift | 00:51 | |
| zhengyin | mattoliverau:morning | 01:23 |
|---|---|---|
| *** sams-gleb has quit IRC | 01:23 | |
| mattoliverau | zhengyin: morning | 01:26 |
| kota_ | morning | 01:31 |
| kota_ | clayg: i didn't yet file it but just thinking do today :P | 01:32 |
| kota_ | notmyname: thanks for adding pyeclib/liberasurecode to swift review dashboard which is very useful. | 01:32 |
| *** david_c_ has joined #openstack-swift | 02:00 | |
| *** sams-gleb has joined #openstack-swift | 02:21 | |
| *** trananhkma has joined #openstack-swift | 02:27 | |
| *** takashi has joined #openstack-swift | 02:52 | |
| *** sams-gleb has quit IRC | 02:54 | |
| *** bkopilov has quit IRC | 02:57 | |
| mahatic | good morning! | 03:42 |
| *** sams-gleb has joined #openstack-swift | 03:52 | |
| *** links has joined #openstack-swift | 03:55 | |
| *** trananhkma has quit IRC | 03:57 | |
| *** winggundamth has quit IRC | 04:03 | |
| *** trananhkma has joined #openstack-swift | 04:15 | |
| *** bkopilov has joined #openstack-swift | 04:17 | |
| *** sams-gleb has quit IRC | 04:24 | |
| *** chlong has joined #openstack-swift | 04:47 | |
| kota_ | mahatic: good morning! | 04:50 |
| openstackgerrit | Charles Hsu proposed openstack/python-swiftclient: Add additional headers for HEAD/GET/DELETE requests. https://review.openstack.org/372656 | 05:18 |
| *** Jeffrey4l has joined #openstack-swift | 05:21 | |
| *** sams-gleb has joined #openstack-swift | 05:21 | |
| openstackgerrit | Merged openstack/liberasurecode: Fix clang compile time error https://review.openstack.org/324083 | 05:27 |
| *** SkyRocknRoll has joined #openstack-swift | 05:39 | |
| *** sams-gleb has quit IRC | 05:54 | |
| *** rcernin has joined #openstack-swift | 06:03 | |
| *** sams-gleb has joined #openstack-swift | 06:51 | |
| *** tesseract has joined #openstack-swift | 07:06 | |
| *** tesseract is now known as Guest71310 | 07:06 | |
| *** ChubYann has quit IRC | 07:16 | |
| *** pcaruana has joined #openstack-swift | 07:17 | |
| *** Jeffrey4l has quit IRC | 07:18 | |
| *** sams-gleb has quit IRC | 07:24 | |
| *** two_tired has quit IRC | 07:25 | |
| *** sams-gleb has joined #openstack-swift | 07:36 | |
| *** hseipp has joined #openstack-swift | 07:59 | |
| *** klrmn has quit IRC | 08:05 | |
| *** Jeffrey4l has joined #openstack-swift | 08:17 | |
| *** rledisez has joined #openstack-swift | 08:26 | |
| *** amoralej|off is now known as amoralej | 08:41 | |
| *** chlong has quit IRC | 08:41 | |
| -openstackstatus- NOTICE: Gerrit is going to be restarted due to slowness and proxy errors | 08:47 | |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** openstackgerrit has joined #openstack-swift | 08:48 | |
| *** winggundamth has joined #openstack-swift | 09:32 | |
| *** sams-gleb has quit IRC | 10:02 | |
| *** sams-gleb has joined #openstack-swift | 10:03 | |
| *** asettle has joined #openstack-swift | 10:05 | |
| *** sams-gleb has quit IRC | 10:06 | |
| *** sams-gleb has joined #openstack-swift | 10:06 | |
| *** rickflare has quit IRC | 10:07 | |
| *** sureshj has joined #openstack-swift | 10:09 | |
| sureshj | hii all i am uploading 3 objects of size 5.1GB with segment size 512MB at a time | 10:09 |
| sureshj | but all of them are failed with 503 service unavialable | 10:10 |
| *** sams-gleb has quit IRC | 10:10 | |
| sureshj | plese someone help on this | 10:10 |
| *** sams-gleb has joined #openstack-swift | 10:10 | |
| *** sams-gleb has quit IRC | 10:10 | |
| *** asettle has quit IRC | 10:11 | |
| *** asettle has joined #openstack-swift | 10:11 | |
| *** rickflare has joined #openstack-swift | 10:11 | |
| *** acoles_ is now known as acoles | 10:12 | |
| acoles | sureshj: take a look in your proxy logs for any error messages container string "returning 503", they may give some hint as to what is happening - could be backend connection timeouts | 10:42 |
| sureshj | acoles: here is some part of my log http://paste.openstack.org/show/588230/ | 10:44 |
| cschwede | Hello everyone! | 10:51 |
| cschwede | acoles: i’m looking into the pyeclib testing in the gate we discussed in Barcelona | 10:52 |
| cschwede | acoles: but i’m no longer sure what is really missing here? at the end of the day we need a job to run probetests, right? | 10:53 |
| acoles | cschwede: remind me what the discussion was? (I remember discussing upgrade tests and tiering tests and the need for multi-policy func tests) | 10:57 |
| acoles | cschwede: did we discuss running swift tests in gate for pyeclib??? | 10:58 |
| cschwede | acoles: we discussed running EC probetests on the gate with John, but looking into the current gate setup it either looks much more simple to me as it sounded in Barcelona or I am missing something | 10:58 |
| acoles | cschwede: oic. hmmm, TBH I have never looked into why we do not run probe tests in gate, I just assumed there was a good reason that we could not. | 11:01 |
| acoles | probe tests make assumptions about the SAIO deployment | 11:01 |
| acoles | like tempauth is required IIRD, but devstack has that | 11:01 |
| cschwede | acoles: well, there is one. it requires more resources, because we need more than 1 replica | 11:01 |
| acoles | cschwede: does devstack use a single replica policy? | 11:02 |
| cschwede | acoles: yes, but one can override this, and use 3 for example to rest replication | 11:03 |
| cschwede | acoles: and my assumption is that we can simply increase that for EC, modify the swift.conf and run these tests | 11:03 |
| cschwede | there are already tests for pyeclib itself, for example: https://review.openstack.org/#/c/390149/ | 11:04 |
| patchbot | patch 390149 - pyeclib - Remove Ryuta Kon from NTT shss reference (MERGED) | 11:04 |
| cschwede | (last patch on master branch) | 11:04 |
| cschwede | acoles: anyhow - i will continue testing on devstack how to enable EC probetests on the gate. i’ll ping you again soon ;) | 11:05 |
| acoles | cschwede: so currently we have no gate *functional* test using EC policy, that would be a good addition. I had thought it could be achieved with the in-process framework by adding an EC policy to that. | 11:06 |
| acoles | IDK about probe tests in the gate, if it can happen then great! | 11:07 |
| cschwede | hmm, looking into that oo | 11:07 |
| cschwede | s/oo/too | 11:07 |
| acoles | sureshj: your logs are showing that the backend PUT requests are timing out after 10 secs - you could try increasing the timeout value (node_timeout in proxy-server section of proxy-server.conf) | 11:10 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Unset random seed after rebalancing ring https://review.openstack.org/371564 | 11:11 |
| *** sams-gleb has joined #openstack-swift | 11:16 | |
| *** chlong has joined #openstack-swift | 11:16 | |
| *** sams-gleb has quit IRC | 11:45 | |
| *** zhengyin has quit IRC | 11:45 | |
| *** silor has joined #openstack-swift | 12:02 | |
| *** tdasilva has quit IRC | 12:06 | |
| *** tdasilva has joined #openstack-swift | 12:16 | |
| *** bkopilov has quit IRC | 12:21 | |
| *** kei_yama has quit IRC | 12:27 | |
| *** SkyRocknRoll has quit IRC | 12:28 | |
| *** rcernin has quit IRC | 12:43 | |
| *** rcernin has joined #openstack-swift | 12:44 | |
| *** sams-gleb has joined #openstack-swift | 12:44 | |
| timss | Hi. Does the containers/objects created with swift-dispersion-populate use any significant amount of disk space? Is the dispersion_coverage just about partition coverage, or can it grow out of control if there's many partitions? | 12:54 |
| *** takashi has quit IRC | 13:00 | |
| *** jordanP has joined #openstack-swift | 13:07 | |
| *** amoralej is now known as amoralej|lunch | 13:11 | |
| *** sams-gleb has quit IRC | 13:15 | |
| *** adb5a56 has quit IRC | 13:17 | |
| *** vint_bra has joined #openstack-swift | 13:38 | |
| *** mvk has quit IRC | 13:38 | |
| *** links has quit IRC | 13:40 | |
| *** bkopilov has joined #openstack-swift | 13:52 | |
| *** amoralej|lunch is now known as amoralej | 13:53 | |
| *** Guest71310 has quit IRC | 13:58 | |
| *** klamath has joined #openstack-swift | 14:01 | |
| *** klamath has quit IRC | 14:01 | |
| *** klamath has joined #openstack-swift | 14:01 | |
| *** mvk has joined #openstack-swift | 14:11 | |
| *** Guest71310 has joined #openstack-swift | 14:12 | |
| *** sams-gleb has joined #openstack-swift | 14:14 | |
| *** Guest71310 has quit IRC | 14:18 | |
| *** tesseract has joined #openstack-swift | 14:19 | |
| *** tesseract is now known as Guest41117 | 14:19 | |
| *** Guest41117 has quit IRC | 14:22 | |
| *** _JZ_ has joined #openstack-swift | 14:27 | |
| *** tesseract- has joined #openstack-swift | 14:27 | |
| *** thurloat has joined #openstack-swift | 14:31 | |
| *** chlong has quit IRC | 14:34 | |
| *** rvasilets___ has joined #openstack-swift | 14:45 | |
| *** sams-gleb has quit IRC | 14:46 | |
| *** mvk has quit IRC | 15:02 | |
| *** sams-gleb has joined #openstack-swift | 15:04 | |
| *** tuan_luong has joined #openstack-swift | 15:12 | |
| *** geaaru has joined #openstack-swift | 15:14 | |
| *** StraubTW has joined #openstack-swift | 15:16 | |
| openstackgerrit | Mathias Bjoerkqvist proposed openstack/swift: WIP: Storing encryption root secret in Barbican https://review.openstack.org/364878 | 15:17 |
| *** sgundur has joined #openstack-swift | 15:17 | |
| *** mvk has joined #openstack-swift | 15:18 | |
| openstackgerrit | Nandini Tata proposed openstack/swift: Allow custom swift configuration directory https://review.openstack.org/393952 | 15:24 |
| *** Renich has joined #openstack-swift | 15:29 | |
| *** Renich has quit IRC | 15:31 | |
| *** links has joined #openstack-swift | 15:34 | |
| tdasilva | do we have documented anywhere all the environment variables that could be set for swift and swift dev? | 15:35 |
| tdasilva | ntata: ^ maybe you know this... | 15:36 |
| *** serverascode has quit IRC | 15:41 | |
| *** kmARC has quit IRC | 15:41 | |
| *** kmARC has joined #openstack-swift | 15:42 | |
| *** sgundur has quit IRC | 15:42 | |
| *** cppforlife_ has quit IRC | 15:42 | |
| *** cargonza has quit IRC | 15:42 | |
| *** kozhukalov has quit IRC | 15:42 | |
| *** amit213 has quit IRC | 15:42 | |
| *** zhiyan has quit IRC | 15:42 | |
| *** raginbaj- is now known as raginbajin | 15:42 | |
| *** philipw has quit IRC | 15:43 | |
| *** madorn has quit IRC | 15:43 | |
| *** rsFF has quit IRC | 15:43 | |
| *** fungi has quit IRC | 15:43 | |
| *** zacksh has quit IRC | 15:43 | |
| *** zacksh has joined #openstack-swift | 15:44 | |
| *** zhiyan has joined #openstack-swift | 15:45 | |
| *** cppforlife_ has joined #openstack-swift | 15:46 | |
| openstackgerrit | Merged openstack/swift: update urls to newton https://review.openstack.org/388196 | 15:46 |
| *** cargonza has joined #openstack-swift | 15:46 | |
| *** amit213 has joined #openstack-swift | 15:47 | |
| *** kozhukalov has joined #openstack-swift | 15:48 | |
| *** serverascode has joined #openstack-swift | 15:48 | |
| *** rsFF has joined #openstack-swift | 15:52 | |
| *** sgundur has joined #openstack-swift | 15:54 | |
| *** rcernin has quit IRC | 15:55 | |
| *** philipw has joined #openstack-swift | 15:55 | |
| *** fungi has joined #openstack-swift | 15:55 | |
| *** Renich has joined #openstack-swift | 15:56 | |
| *** madorn has joined #openstack-swift | 15:56 | |
| *** tuan_luong has quit IRC | 15:58 | |
| *** sgundur has quit IRC | 16:01 | |
| *** pcaruana has quit IRC | 16:12 | |
| *** david_c_ has quit IRC | 16:15 | |
| *** klrmn has joined #openstack-swift | 16:17 | |
| *** stradling has joined #openstack-swift | 16:28 | |
| *** tesseract- has quit IRC | 16:31 | |
| *** david_c_ has joined #openstack-swift | 16:34 | |
| *** sgundur has joined #openstack-swift | 16:36 | |
| *** sgundur has quit IRC | 16:38 | |
| *** sgundur has joined #openstack-swift | 16:39 | |
| ntata | tdasilva, good question. Well, I couldn't find anything that talks about all the possible environment variables at a single place | 16:47 |
| tdasilva | ntata: yeah, i was just wondering, it would probably be helpful to do that at some point | 16:52 |
| ntata | tdasilva, agree, I can pool them together (to deployment guide maybe?) unless you're planning to.. | 16:55 |
| *** diogogmt has joined #openstack-swift | 17:02 | |
| tdasilva | ntata: feel free to go ahead | 17:05 |
| *** klrmn has quit IRC | 17:07 | |
| *** Renich has quit IRC | 17:10 | |
| *** mvk has quit IRC | 17:10 | |
| *** arch-nemesis has joined #openstack-swift | 17:19 | |
| *** links has quit IRC | 17:20 | |
| *** hseipp has quit IRC | 17:21 | |
| notmyname | good morning | 17:22 |
| notmyname | well this is interesting http://lists.openstack.org/pipermail/openstack-dev/2016-November/106877.html | 17:22 |
| notmyname | I haven't read the referenced blog post yet | 17:22 |
| *** mvk has joined #openstack-swift | 17:22 | |
| clayg | yay mondays! | 17:23 |
| *** amoralej is now known as amoralej|off | 17:30 | |
| *** diogogmt has quit IRC | 17:33 | |
| *** asettle has quit IRC | 17:33 | |
| notmyname | well, while I'm piling on the mondays, there's this too https://review.openstack.org/#/c/365590/ | 17:34 |
| patchbot | patch 365590 - governance - Add "Assume Good Faith" to OpenStack principles | 17:34 |
| *** cbartz has joined #openstack-swift | 17:34 | |
| *** ukaynar has joined #openstack-swift | 17:37 | |
| *** ukaynar has quit IRC | 17:38 | |
| *** jordanP has quit IRC | 17:39 | |
| *** diogogmt has joined #openstack-swift | 17:43 | |
| openstackgerrit | Donagh McCabe proposed openstack/swift: Document access control lists (ACLs) https://review.openstack.org/374215 | 17:43 |
| *** rledisez has quit IRC | 17:45 | |
| *** cbartz has left #openstack-swift | 17:45 | |
| tdasilva | notmyname: i read the blog post and i think it seems sensible, i couldn't point out anything that's necessary wrong the the ideas (principles), just not sure I agree with some of the specifics... | 17:46 |
| notmyname | tdasilva: yeah, the "team asking for golang must first create goslo.{config,messaging,db} and auth" seems ... well, I'm not sure what ;-) | 17:47 |
| tdasilva | notmyname: yeah, i think asking 'us' to help contribute to common code is sensible, defining what that common code is will be the challenge | 17:48 |
| notmyname | tdasilva: in our specific case, most of those don't make sense. config might, but the requirement (no different in python) is that existing configs must continue to work | 17:49 |
| tdasilva | notmyname: agree...honestly, top of my head i can only think of config and logging which is not even on his list | 17:50 |
| tdasilva | i'm not sure authentication makes sense at the moment as we are not proposing changes to user facing services, but maybe other services have authentication in the backend services??? i have no idea | 17:51 |
| notmyname | could be. same with db and messaging, too | 17:51 |
| openstackgerrit | Christian Hugo proposed openstack/swift: Use direct_get_suffix_hashes in the reconstructor https://review.openstack.org/394551 | 17:52 |
| *** cdelatte has joined #openstack-swift | 17:54 | |
| *** dmorita has joined #openstack-swift | 18:00 | |
| *** stradling has quit IRC | 18:01 | |
| *** klrmn has joined #openstack-swift | 18:02 | |
| *** mvk has quit IRC | 18:02 | |
| *** stradling has joined #openstack-swift | 18:03 | |
| *** sgundur has quit IRC | 18:15 | |
| *** sgundur has joined #openstack-swift | 18:22 | |
| *** ChubYann has joined #openstack-swift | 18:23 | |
| *** rvasilets___ has quit IRC | 18:28 | |
| clayg | i can't get half way through a sentence without having to delete it - and I'm normally pretty lippy | 18:35 |
| openstackgerrit | Merged openstack/swift: Fix signal handling for daemons with InternalClient https://review.openstack.org/259347 | 18:36 |
| openstackgerrit | Christian Hugo proposed openstack/swift: object-replicator cleanup https://review.openstack.org/391617 | 18:46 |
| openstackgerrit | Christian Hugo proposed openstack/swift: Use direct_get_suffix_hashes in the reconstructor https://review.openstack.org/394551 | 18:46 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 18:48 |
| *** stradling has quit IRC | 19:00 | |
| *** stradling has joined #openstack-swift | 19:02 | |
| *** geaaru has quit IRC | 19:02 | |
| openstackgerrit | Christian Hugo proposed openstack/swift: Use direct_get_suffix_hashes in the reconstructor https://review.openstack.org/394551 | 19:23 |
| *** sgundur has quit IRC | 19:28 | |
| *** d0ugal has quit IRC | 19:30 | |
| *** d0ugal has joined #openstack-swift | 19:31 | |
| *** d0ugal has quit IRC | 19:31 | |
| *** d0ugal has joined #openstack-swift | 19:31 | |
| clayg | jrichli: nice response on lp bug #1319096 | 19:32 |
| openstack | Launchpad bug 1319096 in OpenStack Object Storage (swift) "Include object metadata in container list" [Undecided,In progress] https://launchpad.net/bugs/1319096 | 19:32 |
| *** sgundur has joined #openstack-swift | 19:32 | |
| jrichli | clayg: I looked for a nick that might match Christian Hugo to see if we could chat here, but no luck. | 19:33 |
| jrichli | I see he has uploaded a few different patches lately | 19:34 |
| clayg | jrichli: yeah! would love to see him get more engaged! ... see if there's anything we can do to help/support? | 19:36 |
| openstackgerrit | Ondřej Nový proposed openstack/swift: object-replicator cleanup https://review.openstack.org/391617 | 19:36 |
| *** sgundur has quit IRC | 19:40 | |
| *** sgundur has joined #openstack-swift | 19:40 | |
| *** acoles is now known as acoles_ | 19:50 | |
| notmyname | fungi: torgomatic: tdasilva: patch 394600 and patch 394601 set up a testing environment that allows for checking xattrs in swift's tests, thus enabling patch 336323 to land (I think) | 19:53 |
| patchbot | https://review.openstack.org/#/c/394600/ - openstack-infra/project-config - enable xfs for swift in-process functests | 19:53 |
| patchbot | https://review.openstack.org/#/c/394601/ - openstack-infra/project-config - add python-jobs-with-xfs for swift | 19:53 |
| patchbot | https://review.openstack.org/#/c/336323/ - swift - Add checksum to object extended attributes | 19:53 |
| notmyname | fungi: I'd definitely appreciate your insight on the 2 infra patches | 19:53 |
| notmyname | I spearated them to make it more clear about the 2 things I'm doing, but i'm happy to combine if that's what you'd prefer | 19:53 |
| fungi | notmyname: yep, saw 394601 and its parent scroll by in the infra channel and already have them pulled up | 19:53 |
| notmyname | thanks :-) | 19:54 |
| fungi | i'll look closer once the jobs report back on them in a few minutes | 19:54 |
| *** sgundur has quit IRC | 19:57 | |
| *** sgundur has joined #openstack-swift | 19:57 | |
| *** Renich has joined #openstack-swift | 20:00 | |
| *** silor has quit IRC | 20:00 | |
| clayg | timburke: I'm on vbox -> 5.0.14r105127 and vagrant -> 1.8.4.1 | 20:03 |
| clayg | obvs no problems mounting file systems - could totally be a guest agent thing tho - i'd be happy to upgrade if you're on newer-er versions and see i can duplicate your pain | 20:04 |
| timburke | clayg: hmm... i *am* on newer, though when i first hit trouble, i was on older (at least for vbox) | 20:05 |
| timburke | currently on 5.0.28 & 1.8.5.3 | 20:05 |
| timburke | but the USERNAME tip got me off & running on trusty, so w/e | 20:05 |
| notmyname | fungi: ah, I'm guessing the errors are from the consolidation I did to https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/projects.yaml#L14043-L14055 | 20:09 |
| clayg | timburke: but xenial is *so* much better! | 20:09 |
| notmyname | fungi: I don't see why those need to be different sections (one with trusty+xenial and one with just xenial). | 20:09 |
| notmyname | fungi: from git history, it seems that they got split as part of some trusty->xenial migration in infra. certainly not intended to only work in one or the other from the swift perspective | 20:10 |
| timburke | clayg: eh, may as well be on a platform that my customers might actually be using | 20:10 |
| timburke | (i should really look into adding centos support for vsaio...) | 20:10 |
| notmyname | fungi: but I'll admit to not really understanding all the layout. fwiw, the error is at http://logs.openstack.org/00/394600/1/check/gate-project-config-layout/65bcf33/console.html#_2016-11-07_19_57_01_190485 | 20:12 |
| notmyname | oh! because layout.yaml doesn't reference the new name! (I think) | 20:12 |
| fungi | notmyname: sorry, had to step afk for a moment but i'm taking a look now | 20:15 |
| clayg | timburke: yes | 20:17 |
| clayg | timburke: however, there was *some* reason I needed to move to xenial? some skip tests if you have an old kernel maybe? | 20:17 |
| notmyname | fungi: pushed new revisions | 20:18 |
| clayg | timburke: I think it was honestly because the haproxy that comes with precise can't do ssl termination and I wanted to play with the proxy protocol stuff for the ip address handling patch/story? | 20:18 |
| fungi | notmyname: the error i see there is "jenkins_jobs.errors.JenkinsJobsException: Duplicate definitions for job 'swift-branch-tarball' specified" which i think is because you have the python-jobs and python-jobs-with-xfs-tmp job-groups instantiated on swift in jenkins/jobs/projects.yaml, and each of those job groups defines some of the same jobs | 20:18 |
| notmyname | fungi: oh? can't reference the same job names in 2 different job groups? | 20:18 |
| fungi | notmyname: if the plan is to use both those job-groups together, you probably just need to have gate-{name}-python27-xfs-tmp-{node} and gate-{name}-python34-xfs-tmp in the python-jobs-with-xfs-tmp group | 20:19 |
| timburke | dammit! that was on my shortlist of things to go +A, just as soon as i could get around to func testing it... | 20:19 |
| notmyname | fungi: that doesn't seem to align with python-db-jobs | 20:20 |
| fungi | notmyname: yeah, in this case it's that jjb isn't smart enough to know that you're instantiating, say, {name}-branch-tarball twice with the same set of parameters (if it were smarter it might learn to dedupe those). instead it worries that you're instantiating templates that result in jobs with identical names (which as far as it knows might be different jobs) | 20:20 |
| notmyname | fungi: ...which seems to be a copy of python-jobs but with a couple of names changed | 20:20 |
| clayg | timburke: you're going to need xenial | 20:20 |
| clayg | :P | 20:21 |
| notmyname | fungi: oh, do you prefer these to be one commit or the 2 separate ones I've proposeD? | 20:21 |
| clayg | oh... wait - were you talking about the proxy protocl patch or the other xfs-gate thing | 20:21 |
| timburke | clayg: the PROXY proto patch | 20:22 |
| clayg | timburke: ah | 20:22 |
| fungi | notmyname: right, the projects which use the python-db-jobs group don't use the normal python-jobs group. honestly that's probably an oversight and they should use both but factor out the jobs common between them (so that nova can't accidentally make it impossible to run their tox env without mysql and postgres both installed) | 20:23 |
| fungi | i'll write that up as a separate change | 20:23 |
| notmyname | fungi: TBH, I'd be totally ok with not running the normal python-jobs group and running this one instead, but I've been given the impression that wouldn't fly with the TC and the common testing interface (or "swift being different") | 20:24 |
| fungi | notmyname: yep, i'm writing this one up for parity with the swift change. either they're both needed or neither is but i definitely want a level playing field here whichever way things go | 20:25 |
| notmyname | fungi: ok. I'll change mine to just have the two xfs jobs in the -with-xfs-tmp group | 20:25 |
| notmyname | fungi: but do you want me to keep my 2 patches or squash them together? | 20:26 |
| *** d0ugal has quit IRC | 20:26 | |
| *** dfisher has joined #openstack-swift | 20:26 | |
| *** sgundur has quit IRC | 20:27 | |
| *** d0ugal has joined #openstack-swift | 20:27 | |
| *** d0ugal has quit IRC | 20:27 | |
| *** d0ugal has joined #openstack-swift | 20:27 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Always set swift processes to use UTC https://review.openstack.org/331369 | 20:27 |
| fungi | notmyname: i'm not opposed to keeping them separate changes. probably slightly easier to review this way | 20:28 |
| fungi | i see 394600 failed (different) tests. i was looking at the error on 394601 | 20:29 |
| fungi | and yeah, it looks like the job failure on 394600 is due to needing to also update the gate-swift-tox-bandit-ubuntu-(trusty|xenial) regex in zuul/layout.yaml | 20:30 |
| *** sams-gleb has quit IRC | 20:31 | |
| notmyname | fungi: ok, thanks. new version pushed | 20:32 |
| *** sams-gleb has joined #openstack-swift | 20:32 | |
| *** sams-gleb has quit IRC | 20:36 | |
| mattoliverau | Morning | 20:41 |
| fungi | notmyname: for reference, here's my first attempt at the strawman for making sure all projects using custom db setup in unit tests are also tested without: https://review.openstack.org/394620 | 20:48 |
| patchbot | patch 394620 - openstack-infra/project-config - Make sure opportunistic DB use in unit tests works | 20:48 |
| fungi | i agree that's basically the same case as expecting someone to make an xfs filesystem available when running unit tests | 20:48 |
| fungi | notmyname: it still doesn't look like 394600 patchset 3 updated the gate-swift-tox-bandit-ubuntu-(trusty|xenial) regex in zuul/layout.yaml | 20:58 |
| *** sgundur has joined #openstack-swift | 21:09 | |
| jrichli | timburke: I just glanced at patch 331369 and am wondering what the impacts are. Is this change visible to the user? | 21:12 |
| patchbot | https://review.openstack.org/#/c/331369/ - swift - Always set swift processes to use UTC | 21:12 |
| *** clu_ has joined #openstack-swift | 21:14 | |
| *** abalfour has joined #openstack-swift | 21:15 | |
| *** jamielennox is now known as jamielennox|away | 21:19 | |
| abalfour | So... I could use some advice. It appers that the swift ring files are architecture dependent. I.e. if you use swift-ring-builder to create a ring file on an little endian machine, and then copy that ring file to a big endian machine, you're going to have a bad time. For example: http://paste.openstack.org/show/588317/ | 21:24 |
| abalfour | The obvious problem being on the endianess the file wasn't built on, the device index of 256 isn't going to work for indexing into the r2p2d array... Should we be creating the ring files locally on each architecture, or should the file format be endinan agnostic? | 21:26 |
| *** sgundur has quit IRC | 21:29 | |
| *** vint_bra has quit IRC | 21:32 | |
| *** sams-gleb has joined #openstack-swift | 21:34 | |
| *** Jeffrey4l has quit IRC | 21:35 | |
| *** stradling has quit IRC | 21:46 | |
| pdardeau | abalfour: ouch! i don't have a BE environment to verify, but you might try hacking https://github.com/openstack/swift/blob/master/swift/common/ring/ring.py#L76 | 21:48 |
| pdardeau | maybe change 'H' to '!H' | 21:49 |
| pdardeau | abalfour: it's possible it might be written to file ok, and then converted improperly on BE load | 21:49 |
| dfisher | abalfour had to step away. here's another possible solution http://paste.openstack.org/show/588319/ | 21:52 |
| clayg | abalfour: I did not know that! | 21:53 |
| dfisher | i don't really grok array/struct python objects to comment on 'H' vs '!H' | 21:53 |
| notmyname | interesting | 21:53 |
| pdardeau | dfisher: https://docs.python.org/2/library/struct.html#byte-order-size-and-alignment | 21:53 |
| timburke | jrichli: ideally, not really. the trouble i was running into was that some middleware would try to parse some user-provided timestamp with our stdlib in a crazy split-brain, so the result would be hours off. this lead to some authz errors in swift3 (which i suppose is user-visible), but to my knowledge we always report timestamps in UTC anyway | 21:54 |
| clayg | can i make a vm with backwards default platform bits? and why would python even allow me to serialize in a platform dependent way. | 21:54 |
| pdardeau | the table in 7.3.2.1 shows the architecture specific modifiers | 21:54 |
| pdardeau | and table in 7.3.2.2 shows data types | 21:54 |
| clayg | that's so annoying | 21:55 |
| notmyname | fungi: thanks. back from lunch, and I'm looking now | 21:55 |
| *** sgundur has joined #openstack-swift | 21:55 | |
| dfisher | well, i can certainly try to hammer the '!H' in and see what happens | 21:56 |
| timburke | huh. i find it a little disconcerting that there's no reference to endianness in https://docs.python.org/2/library/array.html | 21:58 |
| timburke | for that matter, "!H" isn't referenced either :-( | 21:59 |
| *** jamielennox|away is now known as jamielennox | 22:02 | |
| jrichli | timburke: re: timestamps - which user-provided timestamp? i guess that it wasn't a Unix Epoch timestamp? | 22:04 |
| timburke | jrichli: peak at the... third related change. we were parsing a user-provided header (either x-amz-date or, if that wasn't present, Date) | 22:06 |
| *** sams-gleb has quit IRC | 22:07 | |
| jrichli | ok, thx | 22:07 |
| pdardeau | timburke: dfisher: and just because '!H' works for struct packing/unpacking doesn't mean it's valid for array :/ | 22:12 |
| dfisher | still poking at it. results in a sec... | 22:12 |
| notmyname | fungi: https://review.openstack.org/#/c/394600/ and https://review.openstack.org/#/c/394601/ have passed the checks! | 22:13 |
| patchbot | patch 394600 - openstack-infra/project-config - enable xfs for swift in-process functests | 22:13 |
| patchbot | patch 394601 - openstack-infra/project-config - add python-jobs-with-xfs for swift | 22:13 |
| notmyname | fungi: is there any way to run swift through those without having them land in infra first? | 22:17 |
| timburke | hmm. i guess we need to make a call to byteswap()? looks like array.array wants a character, not a string | 22:28 |
| *** mvk has joined #openstack-swift | 22:29 | |
| dfisher | yeah, changing to !H didn't seem to make a difference | 22:31 |
| dfisher | sparc: 'replica2part2dev_id': [array('H', [0, 0, 256, 256, 0, 256, | 22:31 |
| dfisher | x86: 'replica2part2dev_id': [array('H', [0, 0, 1, 1, 0, 1, 1, 0, | 22:31 |
| patchbot | Error: Missing "]". You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands. | 22:31 |
| patchbot | Error: Missing "]". You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands. | 22:31 |
| dfisher | ^ ha | 22:31 |
| notmyname | dfisher: someday I'll rewrite patchbot to be smarter | 22:32 |
| dfisher | seems pretty smart to me. | 22:32 |
| dfisher | smarter than I am, anyway :) | 22:33 |
| jrichli | por patchbot gets no luv :-( | 22:33 |
| dfisher | timburke: my paste.openstack.org has the byteswap() patch in it. http://paste.openstack.org/show/588319/ | 22:35 |
| pdardeau | &patchbot-not-so-fussy; don't freak out about my non-matching [ | 22:36 |
| patchbot | Error: Missing "]". You may want to quote your arguments with double quotes in order to prevent extra brackets from being evaluated as nested commands. | 22:36 |
| fungi | notmyname: you could run the script from mount-tmp-with-xfs before running `tox -e py27` | 22:40 |
| torgomatic | notmyname: what if you made the xattrs patch depend on those infra ones? I'm totally guessing here; I have no idea how any of that works. | 22:40 |
| torgomatic | I mean, I know some things... but "will a pending job-specifier patch get used if it's a dependency of a Swift patch" is not anywhere in there | 22:41 |
| fungi | notmyname: my main concern with this is that if our underlying job infrastructure relies on files places in /tmp prior to starting the job payload, it could get confused once it can no longer find them later. i'll try to find out if that will pose a problem | 22:41 |
| notmyname | fungi: yeah, that seems like a totally reasonable concern. TBH I was a little surprised linux let me do it :-) | 22:42 |
| clayg | timburke: wow so https://github.com/eventlet/eventlet/pull/354 is acctaully pretty epic | 22:51 |
| timburke | was it? idk. i had a much worse time with that UTC/stdlib-doesn't-know-what-timezone-it's-in thing | 22:53 |
| clayg | timburke: first learning for me was that eventlet has some dynamic patching of stuff like getaddrinfo if you have dnspython installed -> oh crap or at least it *used* to (!?) https://github.com/eventlet/eventlet/blob/4872be77001bde7b393f8973779a15c65ce36086/eventlet/green/socket.py#L22 | 22:53 |
| clayg | now it just bundles dnspython!? | 22:53 |
| timburke | clayg: you're welcome? https://github.com/eventlet/eventlet/pull/341 | 22:55 |
| clayg | timburke: second learning for me at least is that dnspython compleatly reimplements stuff like getaddrinfo - which is like... glibc stuff | 22:55 |
| clayg | probably just a sign that I need to do better reading the eventlet change lots | 22:58 |
| clayg | *changelogs | 22:58 |
| *** sgundur has quit IRC | 23:00 | |
| dfisher | so, here's a random question that i've wondered about for awhile … why does swift use rsync to keep data nodes updated instead of something like libtorrent? | 23:03 |
| *** sams-gleb has joined #openstack-swift | 23:04 | |
| clayg | dfisher: i'm not really sure there's a direct comparison to be made between the rsync client/server and the torrent *library* - rsync was choosen as the basis for data movement back in ~09 because it was broadly comfortable for the devops that would be running cloud files - always assuming it would be replaced when needed | 23:10 |
| * dfisher nods | 23:11 | |
| clayg | dfisher: and in fact SSYNC and now REPCON (hummingbird) are going down that path | 23:11 |
| dfisher | cool | 23:11 |
| dfisher | just figured that the idea of torrents for data would be more efficient than rsync, especially as more storage nodes are brought online | 23:12 |
| clayg | dfisher: FWIW no one then or now really likes the rsync *protocol* - but since swift only moves whole files i'm not sure it matters that much | 23:12 |
| clayg | dfisher: but both SSYNC and REPCONN are still following the same basic "push" model that the rsync replicators employ | 23:12 |
| clayg | dfisher: I'm not sure anyone has experiemented much to determine if an entire partition could be "pulled" from multiple replica peers in some fashsion that'd be more "efficient" along an axis where there's a need to optomise | 23:13 |
| dfisher | sure. there's probably an inflection point where if you < X nodes, rsync is probably as good if not better than a torrent-esque solution (ignoring things like network speeds for now) | 23:14 |
| dfisher | if you have* < X nodes | 23:14 |
| clayg | dfisher: mostly you just have three primary nodes - the "swarm" effect (lots of hosts/nics/disks all "participating" in the filling of new capacity) mostly comes from the placement algorightm and doing a good job managing incoming streams | 23:15 |
| clayg | dfisher: yeah I suppose if your cluster had an asymmetric link it would become more important for all three replicas of a partition to some how "coordinate" to push subsets of the partition during a rebalance if you don't want to way for a 1:1 push of each part - but again since you're taking 100-1000-10K's of parts onto a single node - you can normally mostly overwhelm whatever resource you're aiming to overwhelm pretty easil | 23:18 |
| *** abalfour1 has joined #openstack-swift | 23:18 | |
| *** dfflanders has joined #openstack-swift | 23:20 | |
| clayg | a big part of what the hummingbird replicator is trying to tackle is reducing disk reads that are only tanginially related to the transfer of data during rebalance | 23:20 |
| abalfour1 | sorry, had to run to get eyeballs checked. so it appears the general consensus is that the ring file should be endian agnostic, correct? so I should file a bug? | 23:22 |
| notmyname | abalfour1: yes you should file a bug so that we can track it. but I'm not sure that the resolution is to make the ring byte-order agnostic | 23:24 |
| clayg | abalfour1: yeah I think it's a bug - how critical is this for you? | 23:24 |
| abalfour1 | well, I have a horrible hack fix that I think dfisher pasted in here. So I can get it to work for me now. :) | 23:24 |
| *** StraubTW has quit IRC | 23:24 | |
| clayg | abalfour1: oh that worked!? | 23:24 |
| abalfour1 | the byteswap(), yep. | 23:25 |
| clayg | oh... | 23:25 |
| notmyname | (eg the resolution might be "don't do that" and document and print warnings about building a ring on a different architecture than prod machines) | 23:25 |
| abalfour1 | it's just the check to see if we need to do the byteswap() is gross. | 23:25 |
| abalfour1 | both archiretcures are prod, btw. | 23:25 |
| abalfour1 | we have a swift cluster with x86 and sparc nodes. | 23:26 |
| abalfour1 | we figured we get cute and pregenerate the ring files and have puppet plop them on all the nodes. | 23:26 |
| abalfour1 | and it failed in humerous ways. :) | 23:27 |
| clayg | abalfour1: that's really awesome | 23:28 |
| abalfour1 | thanks. | 23:28 |
| notmyname | hmm...if you build the ring on each architecture with the same seed value, it would give the same effective results. but md5 checks would be different | 23:28 |
| abalfour1 | correct | 23:28 |
| notmyname | (right?) | 23:28 |
| clayg | abalfour1: can something in sys. tell us the byte of the machine? | 23:28 |
| abalfour1 | sys.byteorder | 23:28 |
| abalfour1 | but, we can't tell what byteorder the file was generated with, I don't think. | 23:29 |
| clayg | abalfour1: I would be fine with the ring file growing a field to say which one it is is - and on load it byteswaps based on != my.byteorder | 23:29 |
| abalfour1 | oh, cool. | 23:29 |
| notmyname | clayg: yeah, that's it | 23:29 |
| clayg | great all over but the typing | 23:29 |
| abalfour1 | ok, I'll file the bug, code it up and submit the patch. | 23:29 |
| abalfour1 | thanks for looking at it! | 23:29 |
| clayg | abalfour1: NICE! | 23:29 |
| notmyname | clayg right now: https://i.imgflip.com/kzxw3.jpg | 23:30 |
| notmyname | abalfour1: thanks :-) | 23:30 |
| abalfour1 | now that's just mean. :) | 23:30 |
| dfisher | abalfour as management is a *terrifying* thought | 23:30 |
| clayg | virtualbox is shit - i bet vmware player or whatever the kids use can toally make a big endian sparc cpu | 23:31 |
| dfisher | i've worked with him for 13 years now. that's the last thing ANYBODY wants. | 23:31 |
| *** Jeffrey4l has joined #openstack-swift | 23:31 | |
| dfisher | clayg: ebay a T2000 + solaris 11.3 ;) | 23:31 |
| notmyname | clayg: what's funny is that the same company that makes virtualbox also owns sparc ;-) | 23:31 |
| clayg | abalfour1: forgive notmyname - he sorta "is" upper management - he doesn't realize it's mostly insulting to the rest of us | 23:31 |
| notmyname | lol | 23:31 |
| dfisher | sorry, abalfour and I work for Sun. | 23:32 |
| clayg | notmyname: ;) | 23:32 |
| clayg | rofl | 23:32 |
| * dfisher sobs quietly | 23:32 | |
| clayg | this is #$%^&ing rich - open source is fun | 23:32 |
| notmyname | "sun" | 23:32 |
| dfisher | my badge says Sun. | 23:32 |
| * dfisher continues to sob | 23:32 | |
| *** cdelatte has quit IRC | 23:33 | |
| *** sams-gleb has quit IRC | 23:37 | |
| *** kei_yama has joined #openstack-swift | 23:41 | |
| *** Renich_ has joined #openstack-swift | 23:44 | |
| openstackgerrit | Merged openstack/swift: EC: reconstruct using non-durable fragments https://review.openstack.org/376630 | 23:45 |
| notmyname | torgomatic: tdasilva: just remounting /tmp as XFS int he gate won't work. turns out a lot of the stuff infra does puts stuff in /tmp, so we can't blow it away | 23:46 |
| torgomatic | notmyname: woo | 23:46 |
| torgomatic | notmyname: can you get an XFS filesystem mounted somewhere else and set $TMPDIR? | 23:46 |
| notmyname | torgomatic: tdasilva: new plan is to mount the xfs loopback somehwere else, set that to TMPDIR and pass TMPDIR through to the tests. it should probably work | 23:46 |
| notmyname | yeah :-) | 23:46 |
| torgomatic | notmyname: the good news is that TMPDIR is in tox's internal list of environment variables to allow through, so hopefully you won't need to mess with the passenv directive at all | 23:47 |
| notmyname | oh cool | 23:47 |
| notmyname | and so you'll probably be wondering, "are we good devs and we're just using the tempfile module/methods, or did we ever hard code /tmp somehwere?" | 23:47 |
| *** Renich has quit IRC | 23:47 | |
| notmyname | hint: it's not the good one | 23:48 |
| mattoliverau | lol | 23:49 |
| notmyname | it's probably fine, though | 23:49 |
| *** _JZ_ has quit IRC | 23:49 | |
| pdardeau | notmyname: if you could just go ahead and send out the memo on that /tmp report that would be great! mmkay? | 23:50 |
| notmyname | let me rephrase. it's not really fine, but it isn't likely to break test in the same way that not having xattrs does. tests should all work if we redefine TMPDIR | 23:51 |
| notmyname | but there's a bit of stuff that should change | 23:54 |
| clayg | notmyname: i bet timburke could come up with a creative python/sed script that will walk the testdir and rewrite and class TestFoo(unitest.TestCase) as TestFoo(unit.test.BaseTestCase) if you need that | 23:54 |
| *** Renich_ has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
