Tuesday, 2015-11-10

*** haomaiwang has quit IRC00:01
*** haomaiwang has joined #openstack-swift00:01
*** cdelatte has quit IRC00:04
openstackgerritSivasathurappan Radhakrishnan proposed openstack/swift: Added unit test cases for server.py  https://review.openstack.org/24230000:05
*** tsg has quit IRC00:06
*** sanchitmalhotra has joined #openstack-swift00:16
*** sanchitmalhotra1 has quit IRC00:18
*** jlhinson_ has quit IRC00:23
*** siva_krishnan has quit IRC00:31
*** siva_krishnan has joined #openstack-swift00:32
*** breitz has quit IRC00:32
*** rohit_ has joined #openstack-swift00:33
*** dmorita has joined #openstack-swift00:33
*** jamielennox is now known as jamielennox|away00:33
*** nadeem has joined #openstack-swift00:37
*** siva_krishnan has quit IRC00:40
*** m_kazuhiro has joined #openstack-swift00:41
*** wbhuber has quit IRC00:55
*** wbhuber has joined #openstack-swift00:55
*** wbhuber has quit IRC00:59
*** haomaiwang has quit IRC01:01
*** 20WACGE4T has joined #openstack-swift01:01
*** nadeem has quit IRC01:13
*** rjaiswal has quit IRC01:15
*** thumpba has joined #openstack-swift01:16
*** thumpba has quit IRC01:18
*** garthb_ has quit IRC01:19
*** nadeem has joined #openstack-swift01:23
*** nadeem has quit IRC01:25
*** pdardeau has joined #openstack-swift01:25
*** mzhou has quit IRC01:27
clayganyone know of something that's like update but does set default instead?01:31
*** badari has quit IRC01:32
clayglike updated_example = default.update(example) but doesn't modify default?01:33
*** jrichli has joined #openstack-swift01:35
*** diogogmt has quit IRC01:37
*** diazjf has joined #openstack-swift01:40
torgomatic_clayg: not in python; you'd have to be like updated_example = default.copy(); updated_example.update(more_stuff)01:42
claygtorgomatic_: well that's BS01:42
claygI bet go has it01:43
torgomatic_Ruby has that as Hash#merge (versus Hash#merge! that modifies its invocant) and I miss it sometimes01:43
*** badari has joined #openstack-swift01:45
*** zhill has quit IRC01:56
*** Guest33023 has quit IRC01:59
*** pdardeau has quit IRC01:59
*** km has joined #openstack-swift01:59
*** km is now known as Guest6531802:00
*** 20WACGE4T has quit IRC02:01
*** pdardeau has joined #openstack-swift02:01
*** haomaiwang has joined #openstack-swift02:01
*** diazjf has quit IRC02:02
*** haomaiwang has quit IRC02:09
*** badari has quit IRC02:12
*** zhill has joined #openstack-swift02:14
*** klrmn1 has quit IRC02:14
*** diazjf has joined #openstack-swift02:17
*** asettle is now known as asettle-afk02:18
*** Guest65318 has quit IRC02:18
*** zhill has quit IRC02:18
*** badari has joined #openstack-swift02:19
*** pdardeau has quit IRC02:24
lifelessclayg: torgomatic_: you can do it with pyrsistent02:24
*** badari has quit IRC02:25
*** diogogmt has joined #openstack-swift02:31
*** jamielennox|away is now known as jamielennox02:34
openstackgerritjanonymous proposed openstack/swift: Eventlet green threads not released back to pool  https://review.openstack.org/22785502:43
*** rohit_ has quit IRC02:46
*** badari has joined #openstack-swift02:48
*** haomaiwang has joined #openstack-swift02:51
*** bapalm has quit IRC02:54
*** thumpba has joined #openstack-swift02:56
*** haomaiwang has quit IRC03:01
*** badari_ has joined #openstack-swift03:01
*** haomaiwang has joined #openstack-swift03:01
openstackgerritCatherine Northcott proposed openstack/swift: Add support for storage policies to have more than one name  https://review.openstack.org/24197803:01
*** thumpba has quit IRC03:02
*** rjaiswal has joined #openstack-swift03:02
*** bapalm has joined #openstack-swift03:02
*** badari has quit IRC03:04
*** klrmn1 has joined #openstack-swift03:14
*** badari_ has quit IRC03:15
*** zaitcev has quit IRC03:19
*** venkat_p has joined #openstack-swift03:21
openstackgerritvenkatesh proposed openstack/swift: remove unused parameters in the method  https://review.openstack.org/24340103:24
*** asettle-afk is now known as asettle03:27
*** badari has joined #openstack-swift03:28
*** haomaiwang has quit IRC03:38
*** haomaiwang has joined #openstack-swift03:38
jrichliclayg: around?  found a new issue with encryption we might need to solve03:40
jrichli(if my deduction is correct)03:41
*** badari has quit IRC03:41
jrichliacoles_: you are correct.  'swift_bytes' is a problem for the content_type.  We cannot have this part encrypted, because the container server needs to parse it.03:47
jrichlithere are no functional tests that include 'swift_bytes' in the content-type, AFAIK.  This is probably how it was missed.03:48
*** sanchitmalhotra1 has joined #openstack-swift03:53
*** sanchitmalhotra has quit IRC03:55
*** diazjf has quit IRC03:59
*** links has joined #openstack-swift03:59
*** haomaiwang has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:01
*** david-lyle has joined #openstack-swift04:07
jrichliclayg: sorry, no worries.  I sent mail to acoles_ and we can chat later about the issue if needed.04:08
*** jrichli has quit IRC04:09
*** sanchitmalhotra has joined #openstack-swift04:12
*** diazjf has joined #openstack-swift04:13
*** sanchitmalhotra1 has quit IRC04:14
notmynamegood evening04:16
*** diogogmt has quit IRC04:17
honotmyname: good evening!04:20
*** thumpba has joined #openstack-swift04:22
*** thumpba has quit IRC04:26
*** jerrygb has quit IRC04:28
claygnotmyname: we need to get the intel & ibm-ers some bouncer action04:31
*** ChanServ changes topic to "Review Dashboard: https://goo.gl/iLt6Ld | Get an IRC bouncer at http://znc.in | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/"04:32
notmynameclayg: topic updated ;-)04:32
claygnotmyname: nice04:33
claygacoles_: jrichli: i get slo's sticking bytes in the content-type is an problem for fast-POST too!04:34
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873604:34
clayg^ watch it be pyeclib>=1.0.7 :P04:34
clayglol - totally is04:35
claygnotmyname: we never did chat about the pyeclib chain04:36
*** jerrygb has joined #openstack-swift04:36
notmynameyeah. I was out longer than I expected and I needed to get home04:36
notmynameclayg: I want to look at the patches out there again (tonight or more likely in the am) before we dive into it04:38
claygnotmyname: I still don't understand how patch 238926 is ever supposed to land if the gate's upper-constraints is pinning pyeclib==1.0.704:38
patchbotclayg: https://review.openstack.org/#/c/238926/ - EC: Use ec_type=liberasurecode_rs_vand in unit tests04:38
claygnotmyname: ok, np04:38
notmynameclayg: yeah, there's a way or a reason :-)04:38
notmynamethat's what I want to wrap my head around first04:38
*** zhill has joined #openstack-swift04:38
notmynamein the meantime, I've got https://www.masterofmalt.com/tequila/la-altena/tapatio-excelencia-gran-reserva-extra-anejo-tequila/ in a glass. let's not ruin it with dependency resolutions ;-)04:39
*** itlinux has joined #openstack-swift04:40
openstackgerritMerged openstack/swift: DiskFileNoSpace check during object delete  https://review.openstack.org/23764804:40
*** notmyname sets mode: -b Zyric*!*@*04:44
*** notmyname sets mode: -b *!~catherine@203-97-119-226.cable.telstraclear.net04:45
*** sanchitmalhotra1 has joined #openstack-swift04:48
*** nadeem has joined #openstack-swift04:48
*** sanchitmalhotra has quit IRC04:50
*** klrmn1 has quit IRC04:52
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873604:53
*** haomaiwang has quit IRC04:55
*** klrmn1 has joined #openstack-swift04:55
*** haomaiwang has joined #openstack-swift04:55
*** chsc has quit IRC04:56
*** haomaiwang has quit IRC05:01
*** jamielennox is now known as jamielennox|away05:01
*** 18WAA337V has joined #openstack-swift05:01
*** klrmn1 has quit IRC05:03
*** nadeem has quit IRC05:07
*** jerrygb has quit IRC05:20
*** jerrygb has joined #openstack-swift05:23
*** jerrygb has quit IRC05:23
*** garthb has joined #openstack-swift05:24
*** venkat_p has quit IRC05:28
*** badari has joined #openstack-swift05:29
*** asettle is now known as asettle-afk05:29
*** badari has quit IRC05:33
*** ppai has joined #openstack-swift05:35
*** itlinux has quit IRC05:37
*** sanchitmalhotra has joined #openstack-swift05:40
*** sanchitmalhotra1 has quit IRC05:42
*** trifon has joined #openstack-swift05:44
*** garthb has quit IRC05:45
*** gyee has quit IRC05:52
*** trifon has quit IRC05:53
*** SkyRocknRoll has joined #openstack-swift05:58
*** 18WAA337V has quit IRC06:01
*** 20WACGGAC has joined #openstack-swift06:01
*** ig0r__ has quit IRC06:05
openstackgerritJohn Dickinson proposed openstack/swift: suppress warning output in a unit test  https://review.openstack.org/24341906:09
*** ig0r__ has joined #openstack-swift06:09
notmynameclayg: that tiny patch suppresses the write_buider warning in unit tests06:09
*** venkat_p has joined #openstack-swift06:21
*** itlinux has joined #openstack-swift06:22
*** jerrygb has joined #openstack-swift06:24
*** thumpba has joined #openstack-swift06:24
*** jerrygb has quit IRC06:28
*** thumpba has quit IRC06:29
*** mahatic_ has joined #openstack-swift06:29
*** mahatic has joined #openstack-swift06:29
*** mahatic_ has quit IRC06:29
* mahatic realizes her IRC client ditched her with proxy issues06:31
mahaticclayg: IRC bouncer seems impossible with an intel proxy standing in between :\06:33
notmynameyou could use The Cloud. I hear it's pretty popular06:33
mahaticirccloud you mean?06:34
*** rjaiswal has quit IRC06:35
mahaticnotmyname: ^06:35
notmynameno. like rackspace or softlayer or ec2 or etc ;-)06:37
mahaticah yeah, i just figured you meant that ;)06:37
*** diazjf has quit IRC06:38
notmynamepeluse seems to have it figured out06:38
notmynameie the intel + stay on IRC thing06:38
mahaticyeah, he has another desktop at work apart from his laptop, whereas me just got a desktop *screen* :P06:39
mahatic(IIRC)06:40
*** km has joined #openstack-swift06:41
*** km is now known as Guest3832306:42
*** August1914 has quit IRC06:43
*** itlinux has quit IRC06:45
*** SkyRocknRoll has quit IRC06:54
*** trifon has joined #openstack-swift06:57
*** sanchitmalhotra1 has joined #openstack-swift06:57
*** sanchitmalhotra has quit IRC06:58
*** 20WACGGAC has quit IRC07:01
*** pchng_ghost has joined #openstack-swift07:01
*** 7GHABMC07 has joined #openstack-swift07:01
*** pchng_ has quit IRC07:04
*** jerrygb has joined #openstack-swift07:25
*** thumpba has joined #openstack-swift07:25
*** jerrygb has quit IRC07:29
*** thumpba has quit IRC07:29
*** itlinux has joined #openstack-swift07:34
*** openstackgerrit has quit IRC07:46
*** openstackgerrit has joined #openstack-swift07:46
*** zhill has quit IRC07:50
*** itlinux_ has joined #openstack-swift07:52
*** itlinux has quit IRC07:53
*** itlinux_ has quit IRC07:55
*** itlinux has joined #openstack-swift07:56
*** 7GHABMC07 has quit IRC08:01
*** haomaiwang has joined #openstack-swift08:01
*** zigo has quit IRC08:01
*** arnox has joined #openstack-swift08:01
*** zigo has joined #openstack-swift08:03
*** itlinux has quit IRC08:04
*** itlinux has joined #openstack-swift08:06
*** rledisez has joined #openstack-swift08:08
*** peterlisak has joined #openstack-swift08:08
*** haomaiwang has quit IRC08:11
*** haomaiwang has joined #openstack-swift08:11
*** itlinux has quit IRC08:13
*** haomaiwang has quit IRC08:16
*** 18WAA3459 has joined #openstack-swift08:18
*** ChanServ sets mode: +v cschwede08:18
peterlisakacoles_, hi, I have a question about patch 241342. What would change your mind to give +2 to patch set 2? I made also patch set 3, but it looks like it's not as convenient as previous patch set 2.08:23
patchbotpeterlisak: https://review.openstack.org/#/c/241342/ - Add content-type in list of container content08:23
*** itlinux has joined #openstack-swift08:25
*** jerrygb has joined #openstack-swift08:26
*** jerrygb has quit IRC08:31
*** openstackgerrit has quit IRC08:31
*** papercup has quit IRC08:31
*** openstackgerrit has joined #openstack-swift08:31
*** SkyRocknRoll has joined #openstack-swift08:36
*** SkyRocknRoll has quit IRC08:40
*** jordanP has joined #openstack-swift08:41
*** geaaru has joined #openstack-swift08:51
*** 18WAA3459 has quit IRC09:01
*** sanchitmalhotra has joined #openstack-swift09:01
*** haomaiwa_ has joined #openstack-swift09:02
*** sanchitmalhotra1 has quit IRC09:03
*** o_o has joined #openstack-swift09:04
*** o_o is now known as Guest9462809:05
*** donagh has quit IRC09:06
*** jmccarthy has joined #openstack-swift09:11
*** SkyRocknRoll has joined #openstack-swift09:15
*** itlinux has quit IRC09:15
*** donagh has joined #openstack-swift09:16
openstackgerritPeter Lisák proposed openstack/swift: swift-init return codes  https://review.openstack.org/23035209:19
*** itlinux has joined #openstack-swift09:19
*** SkyRocknRoll has quit IRC09:20
charzI fixed "pbr>=1.6" issue in vm-saio-probe jobs, if you see this error in your patch, please add comment "recheck" in your patch. Community jenkins will rerun it.09:21
openstackgerritCatherine Northcott proposed openstack/swift: Add support for storage policies to have more than one name  https://review.openstack.org/24197809:23
*** jerrygb has joined #openstack-swift09:27
*** jerrygb has quit IRC09:32
*** ho has quit IRC09:41
*** jistr has joined #openstack-swift09:45
*** kei_yama has quit IRC09:45
*** sanchitmalhotra1 has joined #openstack-swift09:56
*** sanchitmalhotra has quit IRC09:58
*** haomaiwa_ has quit IRC10:01
*** haomaiwa_ has joined #openstack-swift10:01
*** can8dnSix has joined #openstack-swift10:04
*** aix has joined #openstack-swift10:04
*** sanchitmalhotra1 has quit IRC10:07
*** jamielennox|away is now known as jamielennox10:24
openstackgerritPeter Lisák proposed openstack/swift: swift-init return codes  https://review.openstack.org/23035210:26
*** jerrygb has joined #openstack-swift10:28
*** pchng_ has joined #openstack-swift10:32
*** jerrygb has quit IRC10:34
*** pchng_ghost has quit IRC10:35
*** ig0r__ has quit IRC10:40
*** sanchitmalhotra has joined #openstack-swift10:42
*** sanchitmalhotra has quit IRC10:45
*** ppai has quit IRC10:51
*** itlinux has quit IRC10:52
*** itlinux has joined #openstack-swift10:58
*** haomaiwa_ has quit IRC11:01
*** haomaiwang has joined #openstack-swift11:01
*** peterlisak13 has joined #openstack-swift11:02
*** marcusvrn_ has joined #openstack-swift11:04
*** peterlisak has quit IRC11:04
*** ppai has joined #openstack-swift11:05
*** jmccarthy1 has joined #openstack-swift11:09
*** jmccarthy has quit IRC11:09
*** npf has joined #openstack-swift11:10
npfhello11:11
npfi have an install of openstack swift (2.5.0) and i have lots of tombstones with more than 7 days. How can i debug the issue in object-replicator?11:11
npf# find /srv/node/ -iname \*ts -mtime +7 | wc -l    => shows 22779 files11:13
npfreading the metadata from on of the files i get:11:14
npfdatetime.datetime.fromtimestamp(float(pickle.loads(xattr.getxattr('/srv/node/s1-sata1/objects/1551/c3a/3079500de295d4e63b3b83a887893c3a/1446413903.75113.ts','user.swift.metadata'))['X-Timestamp']))11:14
npfdatetime.datetime(2015, 11, 1, 21, 38, 23, 751130)11:14
*** dmorita has quit IRC11:20
npfany ideas?11:27
*** itlinux has quit IRC11:30
*** jmccarthy1 has quit IRC11:32
*** aix has quit IRC11:33
*** aix has joined #openstack-swift11:33
*** jmccarthy has joined #openstack-swift11:36
*** jmccarthy has quit IRC11:39
cschwedenpf: did you set your reclaim age to a lower value? the default is 604800 seconds, ie 4 weeks (https://github.com/openstack/swift/blob/master/etc/object-server.conf-sample#L209-L210)11:49
*** venkat_p has quit IRC11:49
*** can8dnSix has quit IRC11:53
*** itlinux has joined #openstack-swift11:55
*** jmccarthy has joined #openstack-swift11:57
*** haomaiwang has quit IRC12:01
*** haomaiwang has joined #openstack-swift12:01
npfcschwede: hum.. 604800/3600/24 = 7 days12:02
npfnot 4 weeks..12:03
npfthat's why it did the find with mtime +712:03
cschwedenpf: er, you’re absolutely right. *me needs more coffee*12:04
*** m_kazuhiro has quit IRC12:04
npf;)12:04
* npf is hoping for an easy fix...12:05
*** cdelatte has joined #openstack-swift12:05
*** Guest38323 has quit IRC12:21
*** CaioBrentano has joined #openstack-swift12:24
cschwedenpf: permissions are correct? ie the same as the user that is running the object-replicator?12:29
*** jerrygb has joined #openstack-swift12:32
*** jerrygb has quit IRC12:37
*** arnox has quit IRC12:38
npfyes.. i think i found where the problem lies...12:40
npfall permissions are ok..12:40
npfthis is a seldom used cluster (we are in the process of installing it)12:40
npfbut because there wasn't any "real" updates(inserts, updates, deletes) in those containers, all the hashes.pkl remained the same12:41
npfafter i deleted all the hashes.pkl file.. the replicator started sweeping the filesystem and reconstructing hashes.pkl and in the process cleaning up tombstones12:42
*** mahatic_ has joined #openstack-swift12:42
npfbut this was a last resort12:42
*** fbo has quit IRC12:44
*** mahatic has quit IRC12:46
*** fbo has joined #openstack-swift12:47
*** arnox has joined #openstack-swift12:49
*** arnox has quit IRC12:49
npfhttps://answers.launchpad.net/swift/+question/21828512:50
npfi think is related to this12:50
*** haomaiwang has quit IRC12:51
*** jmccarthy has quit IRC12:51
*** haomaiwang has joined #openstack-swift12:52
*** haomaiwang has quit IRC12:57
*** trifon has quit IRC12:58
*** arnox has joined #openstack-swift12:59
*** jmccarthy1 has joined #openstack-swift13:09
*** jmccarthy1 has quit IRC13:09
*** jmccarthy1 has joined #openstack-swift13:09
*** wbhuber has joined #openstack-swift13:11
*** peterlisak has joined #openstack-swift13:11
*** peterlisak13 has quit IRC13:14
*** wbhuber has quit IRC13:15
*** ppai has quit IRC13:16
*** tsg has joined #openstack-swift13:21
*** jerrygb has joined #openstack-swift13:24
*** daemontool has quit IRC13:25
*** jerrygb has quit IRC13:29
*** trifon has joined #openstack-swift13:38
*** haomaiwang has joined #openstack-swift13:50
*** ahale_ is now known as _ahale13:55
*** haomaiwang has quit IRC14:01
*** haomaiwang has joined #openstack-swift14:01
*** dustins has joined #openstack-swift14:03
openstackgerritvenkatamahesh proposed openstack/swift: OpenStack typo  https://review.openstack.org/24360814:04
*** links has quit IRC14:08
*** thumpba has joined #openstack-swift14:09
*** diogogmt has joined #openstack-swift14:13
*** petertr7_away is now known as petertr714:14
openstackgerritvenkatamahesh proposed openstack/swift: Change stackforge repo to openstack repo  https://review.openstack.org/24361714:17
*** daemontool has joined #openstack-swift14:18
*** CaioBren_ has joined #openstack-swift14:26
*** pdardeau has joined #openstack-swift14:27
*** CaioBrentano has quit IRC14:27
*** siva_krishnan has joined #openstack-swift14:31
*** tsg has quit IRC14:33
*** dustins has quit IRC14:41
*** barker_ has joined #openstack-swift14:44
*** barker_ has quit IRC14:44
*** jmccarthy1 has quit IRC14:45
*** jmccarthy1 has joined #openstack-swift14:45
*** ujjain has quit IRC14:46
*** ujjain has joined #openstack-swift14:48
*** ujjain has joined #openstack-swift14:48
*** badari has joined #openstack-swift14:52
*** barker_ has joined #openstack-swift14:53
*** barker_ has quit IRC14:56
*** barker_ has joined #openstack-swift14:57
*** daemontool has quit IRC14:58
openstackgerritNiall Bunting proposed openstack/python-swiftclient: Retry download of object body  https://review.openstack.org/15920814:59
*** haomaiwang has quit IRC15:01
*** links has joined #openstack-swift15:01
*** haomaiwang has joined #openstack-swift15:01
*** chsc has joined #openstack-swift15:04
*** barker_ has quit IRC15:04
*** breitz has joined #openstack-swift15:04
*** links has quit IRC15:05
*** diogogmt has quit IRC15:06
*** barker has joined #openstack-swift15:06
*** wbhuber has joined #openstack-swift15:12
*** siva_krishnan has quit IRC15:14
*** ajiang has quit IRC15:15
*** bhakta has quit IRC15:16
*** hseipp has joined #openstack-swift15:17
*** siva_krishnan has joined #openstack-swift15:18
openstackgerritvenkatamahesh proposed openstack/swift: Changed .md files to .rst files  https://review.openstack.org/24365515:22
*** tsg has joined #openstack-swift15:27
*** rjaiswal has joined #openstack-swift15:31
*** jerrygb has joined #openstack-swift15:33
*** dustins has joined #openstack-swift15:33
*** siva_krishnan has quit IRC15:34
*** jerrygb has quit IRC15:37
*** pdardeau has quit IRC15:39
*** garthb has joined #openstack-swift15:41
notmynamegood morning15:43
tdasilvagood morning15:43
*** pdardeau has joined #openstack-swift15:47
*** npf has quit IRC15:49
*** barker has quit IRC15:49
mahatic_good morning15:49
pelusegood morning to all 3 of you15:50
*** blmartin has joined #openstack-swift15:50
tdasilvahonestly, I don't get this: patch 24360815:53
patchbottdasilva: https://review.openstack.org/#/c/243608/ - OpenStack typo15:53
*** peterlisak has quit IRC15:53
*** jrichli has joined #openstack-swift15:54
*** diogogmt has joined #openstack-swift15:55
*** proteusguy__ has quit IRC15:55
pelusehmmm, yeah15:55
peluseoh wait.  Hmm, yeah.15:56
notmynamelol15:56
peluselooks like the other projects are being 'scrubbed' for the same thing and accepting them as 'sure, why no' kinda thing...15:58
pelusenot15:58
*** diogogmt has quit IRC15:59
*** jmccarthy1 has quit IRC16:00
*** wbhuber_ has joined #openstack-swift16:00
*** haomaiwang has quit IRC16:01
*** jmccarthy has joined #openstack-swift16:01
*** haomaiwang has joined #openstack-swift16:01
*** dustins has quit IRC16:01
*** diogogmt has joined #openstack-swift16:02
*** daemontool has joined #openstack-swift16:03
openstackgerritGanesh Maharaj Mahalingam proposed openstack/swift: Unit tests for account/backend.py  https://review.openstack.org/24061316:04
*** wbhuber has quit IRC16:04
*** aix has quit IRC16:04
*** dustins has joined #openstack-swift16:06
*** jerrygb has joined #openstack-swift16:07
*** jmccarthy has quit IRC16:07
*** proteusguy__ has joined #openstack-swift16:08
*** jmccarthy has joined #openstack-swift16:08
*** itlinux has quit IRC16:09
*** mzhou has joined #openstack-swift16:10
tdasilvawow, patch 238771 still hasn't merged!16:10
patchbottdasilva: https://review.openstack.org/#/c/238771/ - Bump PyECLib version to >=1.0.716:10
*** jerrygb has quit IRC16:12
notmynametdasilva: I'm working on it ;-)16:12
*** jerrygb has joined #openstack-swift16:13
notmynametdasilva: for that typo patch, please don't spend a lot of time on it. if you think it's good, approve it.16:13
tdasilvanotmyname: yeah, i see it on the gate, but tsg already had to add 4 'rechecks' to it :/16:13
tsgnotmyname, tdasilva, something's up with the neutron gate jobs16:14
notmynametdasilva: I was going to get my head around it again this morning. summary is that it's more complicated in part because of requirements16:14
notmynametsg: ah ok16:14
tsgchecks succeed but they fail at gate16:14
tdasilvatsg: yeah16:14
tsgI mentioned to lifeless earlier on -infra16:14
*** jerrygb_ has joined #openstack-swift16:15
*** jerrygb has quit IRC16:17
*** esker has joined #openstack-swift16:18
openstackgerritGanesh Maharaj Mahalingam proposed openstack/swift: Add unit tests for direct_client  https://review.openstack.org/24129116:19
jrichliclayg notmyname: bouncer is on my todo list.  hrou and pchng_ have been doing work towards making this easier for ibm'ers :-)  constant connection coming soon16:22
tsghaypo: ping16:22
notmynamejrichli: :-)16:22
* notmyname gets off the computer to commute to the office16:22
tdasilvajrichli: just fyi, acoles_ pointed me to panicbnc and that's been working well for me16:23
tdasilvamahatic_: ^^^16:23
*** pchng_ is now known as pchng16:24
mahatic_tdasilva: thanks, will check it out16:24
*** diazjf has joined #openstack-swift16:25
*** pdardeau has quit IRC16:25
*** minwoob has joined #openstack-swift16:26
onovyi'm using ZNC on my home server, working fine too :)16:26
*** minwoob has quit IRC16:26
*** donagh has quit IRC16:26
*** pchng_ has joined #openstack-swift16:28
*** jamielennox has quit IRC16:28
*** minwoob has joined #openstack-swift16:28
onovynotmyname, hi, can we talk about: https://www.mail-archive.com/openstack-infra@lists.openstack.org/msg03339.html pls?16:31
*** pchng has quit IRC16:31
*** trifon has quit IRC16:32
*** mzhou has quit IRC16:33
*** August1914 has joined #openstack-swift16:37
claygjrichli: oh wow, do you use irc logs on http://eavesdrop.openstack.org/irclogs/%23openstack-swift/ to catch up or something?16:37
jrichliyup16:39
jrichlibeen doing that since last Feb :-)16:39
claygonovy: I think timburke & kota_ & chmouel recently went thorugh this moving swift3 from fugita's repo to under openstack16:40
onovydon't need help with "how to"16:41
claygonovy: now swift3 is a like a real thing, people talk about it at design summits, we're working on better cross project gating jobs16:41
onovybut where :]16:41
clayg"where" ?16:41
clayg*here*16:41
clayg:P16:41
onovyso as subproject?16:41
*** jamielennox has joined #openstack-swift16:41
onovy(just created openstack-swauth irc channel :)16:41
onovyit's possible to set different gerrit ACL for subprojects?16:42
claygheh16:42
claygoh yeah for sure - swift3 has different social review expectations16:42
claygand of course a different set of people who *can* acctually click +A16:43
onovyperfect, so subproject is better16:43
claygI dont' think they have their own irc channel tho :P16:43
onovynp, will remove it16:44
tdasilvathey do16:44
tdasilva#swift316:44
claygI don't know if swift3 calls itself a swift subproject - it's just an openstack associated project - I think they used to call them stackforge projects16:44
claygtdasilva: nice!16:44
claygI should hang out in there16:44
*** jlhinson has joined #openstack-swift16:44
claygi bet thats where all the cool kids are16:44
tdasilvaclayg, onovy: right, it's not a subproject, but just an openstack project associated with swift16:44
tdasilvafor example, #swiftonfile is another one16:44
*** siva_krishnan has joined #openstack-swift16:45
onovypython-swiftclient too?16:45
tdasilvait has its own list of cores, maintainers, etc...so you can set up your own project16:45
claygonovy: well there you go - talk to tdasilva16:45
claygI need to get the kids ready for school16:45
onovyclayg, :]16:45
tdasilvaclayg: sorry, for jumping in the middle16:45
onovypeter is following this: http://docs.openstack.org/infra/manual/creators.html#give-openstack-permission-to-publish-releases16:46
claygtdasilva: no it's cool - i'm clueless on openstack "regulations" - i just pretend they're all normal opensource projects and try to do what makes sense16:46
*** donagh has joined #openstack-swift16:46
tdasilvaclayg: yep! and you are right16:46
jrichliacoles_ is still away ... I am wondering if that means he was selected16:47
*** haomaiwang has quit IRC16:48
*** itlinux has joined #openstack-swift16:49
*** peterlisak has joined #openstack-swift16:49
jlhinsonjrichli: may the odds be ever in his favor16:50
jrichlijlhinson: :-p16:50
tdasilvaonovy: hrou just went through the process of setting up the storlets project too, so he might be able to provide some tips too16:51
tdasilvaonovy: https://github.com/openstack/storlets16:52
onovyyep, there was presenation on summit16:52
*** gyee has joined #openstack-swift16:52
*** August1914 has quit IRC16:53
*** onovy has quit IRC17:00
*** onovy has joined #openstack-swift17:00
mahatic_torgomatic_: hello, I was wondering if you could address a comment on this patch https://review.openstack.org/#/c/173497/ - I know itś been long since this was merged, but could you take a look please?17:00
*** daemontool has quit IRC17:04
*** mzhou has joined #openstack-swift17:08
*** hseipp has quit IRC17:09
*** pdardeau has joined #openstack-swift17:09
*** itlinux has quit IRC17:10
*** jmccarthy has quit IRC17:12
*** garthb_ has joined #openstack-swift17:14
*** garthb has quit IRC17:15
*** mzhou has quit IRC17:17
*** mzhou has joined #openstack-swift17:18
*** gyee has quit IRC17:25
*** pdardeau has quit IRC17:25
notmynameonder: peterlisak: I was wondering about the swauth thing you created. what is that?17:26
*** gyee has joined #openstack-swift17:28
*** badari has quit IRC17:29
notmynamewow. I just got this email. Subject: "How Hyperconvergence Affects App Deployment" Title in the body: "The IT Social Dynamics of Hyperconverged Infrastructure"17:29
tsgnotmyname: http://tinyurl.com/ne3ex4v (gate failure rate)17:29
notmynameis there a phrase for buzzwords so dense it turns into a black hole?17:30
tdasilvalol17:30
notmynameonder: oh, interesting. it seems I missed one of your earlier comments in the channel about the swauth thing17:32
*** rledisez has quit IRC17:33
tdasilvanotmyname: do you mean onovy ?17:33
notmynameoh yeah17:33
notmynameonovy:17:33
notmynamethanks17:33
*** petertr7 is now known as petertr7_away17:34
onder:)17:34
*** esker has quit IRC17:35
notmynameonovy: it seems from your email that you've talked to gholt about it. I'm a little curious about what your plans are for swauth (and why that's possible inside the openstack project namespace and not possible now)17:37
notmynameonovy: but in general, if you are moving it over, then I'd suggest doing it similar to swift3: as a separate team/project that is in the ecosystem but not one of the official swift deliverables17:38
*** arnox has quit IRC17:45
tdasilvahas anybody run into this problem?17:46
tdasilva[vagrant@server0 swift]$ sudo python setup.py develop17:46
tdasilvaerror in setup command: Invalid environment marker: (python_version>='3.0')17:46
notmynametdasilva: yeah, you gotta upgrade pbr17:46
tdasilvawhen setting up a new saio dev VM17:46
tdasilvaoh17:46
tdasilvai have Version: 1.8.117:47
*** jistr has quit IRC17:48
*** jmccarthy has joined #openstack-swift17:49
notmynamehmm. IIRC it was pbr v1 that added support for the python version specifiers17:50
tdasilvammmm :/17:51
*** jmccarthy has quit IRC17:54
notmynametsg: looks like sdague just sent an email to the ML about the neutron failures17:55
*** pdardeau has joined #openstack-swift17:58
tdasilvanotmyname: ok, had to also upgrade setuptools18:00
tdasilvanotmyname: rpm available on centos is pretty old18:01
notmynameah ok18:01
*** garthb_ has quit IRC18:02
*** garthb_ has joined #openstack-swift18:02
*** zhill has joined #openstack-swift18:04
*** jordanP has quit IRC18:05
tdasilvanotmyname: thanks for the tip18:05
*** willsama has joined #openstack-swift18:06
*** jerrygb has joined #openstack-swift18:06
*** jerrygb_ has quit IRC18:09
*** klrmn1 has joined #openstack-swift18:13
*** blmartin has quit IRC18:13
*** diazjf has quit IRC18:14
*** tsg has quit IRC18:14
*** tsg has joined #openstack-swift18:17
*** tsg has quit IRC18:18
*** tsg has joined #openstack-swift18:19
*** pdardeau has quit IRC18:19
*** diazjf has joined #openstack-swift18:20
openstackgerritMerged openstack/swift: Change stackforge repo to openstack repo  https://review.openstack.org/24361718:21
*** jlhinson has quit IRC18:21
*** lcurtis has joined #openstack-swift18:23
*** tsg has quit IRC18:23
*** tsg has joined #openstack-swift18:23
*** siva_krishnan has quit IRC18:26
*** siva_krishnan1 has joined #openstack-swift18:26
*** garthb__ has joined #openstack-swift18:31
*** garthb_ has quit IRC18:31
onovynotmyname, plan is simple: gholt don't have time for maintaining of swauth. we are using it, and openstack ecosystem is fine for project maintaining.18:36
onovyso we (seznam.cz) will maintain it using openstack ecosystem (gerrit, ...)18:37
onovynotmyname, so, you are suggesting follow: http://docs.openstack.org/infra/system-config/stackforge.html + http://docs.openstack.org/infra/manual/creators.html right?18:38
onovyand not as "official openstack project", but as stackforge18:39
*** CaioBren_ has quit IRC18:39
notmynameyeah, I think the second like is right18:39
notmynamestackforge (as a name) doesn't exist any more18:40
onovyrelated project?18:40
*** CaioBrentano has joined #openstack-swift18:41
notmynameyeah18:41
notmynameit won't be an "official openstack project". but it will live int eh openstack namespace and use the CI18:41
onovyperfect, thats your goal18:41
onovy*our18:41
claygtorgomatic_: I don't have a good plan on where to put words about replicanths18:42
claygtorgomatic_: my hope was that after the design summit in Toyko everyone would have an intuative sense of how I was describing part-replica placement - but I've thought about it more and I think I could put together a paragraph that wouldn't to egregiously bastardize math words that have real meanings18:43
clayg... I just don't know where to put them?18:43
claygtorgomatic_: would commit message be ok for now?  Or do you think there's a high level doc string that could speak to the concept?18:43
onovynotmyname, and it will *not* be "swift official" (http://stackalytics.com/?module=swift-group), right?18:43
notmynameclayg: http://docs.openstack.org/developer/swift/overview_ring.html ?18:43
claygnotmyname: good call!18:43
notmynameonovy: correct18:44
dfgonovy: are you planning on updating swauth? cool :) what are you planning on doing?18:44
onovydfg, first of all: code cleanup and unit tests18:45
onovyand first of first of all: move it to gerrit :)18:45
dfgonovy: neat. one of our guys has been thinking about maybe some updates to it recently but i'll let him know so he can coordinate with you. thx!18:47
onovydfg, perfect! #openstack-swauth channel is up and working for dicussion18:48
notmynameonovy: also briancline might be interested since they currently use swauth in their clusters (although have plans to move off of it)18:48
*** jlhinson has joined #openstack-swift18:48
clayglifeless: tsg: i doesn't look like patch 238771 is going in?!18:49
patchbotclayg: https://review.openstack.org/#/c/238771/ - Bump PyECLib version to >=1.0.718:49
tsgclayg: neutron broke gate18:49
lifelessclayg: there's a 35% failure in neutron in the combined gate at the moment18:49
lifelessclayg: see sdague's email in the 'learning to debug the gate' thread18:49
lifeless[perhaps not the most obvious thread to put it in]18:50
claygoh... well looking at the failures on that openstack/requirements change it didn't seem to be nuetron specific but rather something quite specific with the requirements-resolver18:50
claygnotmyname: what gives with patch 243655 - are our tld docs markdown or restructured text?18:51
patchbotclayg: https://review.openstack.org/#/c/243655/ - Changed .md files to .rst files18:51
lifelessclayg: the resolver job is non-voting18:52
*** petertr7_away is now known as petertr718:53
clayglifeless: oh how strange... so it's the tempest-nuetron-full failure that's eating it's lunch afterall - mybad18:54
torgomatic_clayg: commit message is fine; I feel like I used to know what it meant but now I don't, so I just need a little refresher18:57
torgomatic_preferably in Gerrit so the other reviewer can read it too18:57
*** tsg has quit IRC19:00
notmynameclayg I have no idea why the md->rst patch was done. all the rest of our docs are rst, so maybe that's why?19:00
claygtorgomatic_: notmyname suggested the ring overview in tree doc - which I thought sounded pretty good?  Would an implementation detail as trivial as that be inappropriate to document in a highlevel over view meant for consumption by a board audience - or do you agree developer docs is a good spot?19:01
claygif I put it in the diff it'll be in gerrit for reviewers ;)19:01
claygtorgomatic_: thanks for reading the diff BTW, I normally go over my changes after I push for exactly that kind of crap - but I was so exhausted I couldn't bare to look at it for a couple of days - so... much appreciated!19:01
*** jerrygb_ has joined #openstack-swift19:01
notmynameis venkatamahesh here in IRC?19:02
claygnotmyname: what is the name of the editor you use?19:02
claygnotmyname: hipster-text or something like ... ah sublime!19:02
notmynameyeah, that one :-)19:02
notmynameI was using it before it was cool (of course)19:02
tdasilvalol19:03
*** jerrygb has quit IRC19:04
*** jerrygb has joined #openstack-swift19:05
*** tsg has joined #openstack-swift19:05
*** geaaru has quit IRC19:06
*** jerrygb_ has quit IRC19:08
torgomatic_Sublime Text always sounds like a good editor for writing vaporware.19:10
notmynametorgomatic_: the adjective definition is much better than the verb ;-)19:11
notmyname(if perhaps a little pretentious)19:11
*** blmartin has joined #openstack-swift19:13
clayganyone know Peter Lisak's handle?19:19
tdasilvapeterlisak:19:20
claygpeterlisak: hey I need your builder!19:20
onovyit's 8:23 pm here (where peter lives) and he is not in work now :)19:24
onovybut if you want builder for from our production cluster, i can send it to you19:24
onovy*builder file19:24
tdasilvaonovy: are you guys in Brno?19:25
onovycan someone with better english help me with this? "An alternate authentication system for Swift": Alternate or Alternative?19:25
onovytdasilva, yep19:26
diazjfnotmyname, jrichli, I have added an etherpad for what is need from Castellan in order to include it in the keymaster. https://etherpad.openstack.org/p/swifjt-keymaster-with-castellan Can we plan a discussion on this sometime :)19:26
tdasilvaonovy: have you heard of devconf: http://devconf.cz/ ?19:27
jrichlithanks, diazjf19:27
onovytdasilva, no19:27
*** siva_krishnan1 has quit IRC19:27
onovytdasilva, oh, sry, yes19:28
openstackgerritMerged openstack/swift: Skip functional tests when test account4 is unauthorized  https://review.openstack.org/23844119:28
*** siva_krishnan has joined #openstack-swift19:28
diazjfjrichli no worries :)19:28
onovytdasilva, i will be there :)19:29
tdasilvaonovy: cool! you should propose a talk19:29
tdasilvatalk about your swift experiences :)19:29
onovytdasilva, with my horrible english? no thanks :)19:30
tdasilvaonovy: i'm planning to be there too..we'll see19:30
onovyBUT can i send some of my colleage19:30
tdasilvaonovy: i'm sure you would do just fine19:31
peterlisakclayg, hi, builder can be created by http://paste.openstack.org/show/478422/ ... or onovy will send it19:31
openstackgerritBill Huber proposed openstack/swift: ObjectControllers return application errors as 499 on bad read  https://review.openstack.org/23600719:31
onovypeterlisak, you are working? NOW? 20:31? REALY? :)19:32
*** lpabon has joined #openstack-swift19:32
*** tsg has quit IRC19:34
openstackgerritMerged openstack/swift: Compare Swift config checksum in swift-recon --all  https://review.openstack.org/24072219:34
peterlisakonovy, no, just trying your bouncer ;)19:34
peterlisakonovy, and I have seen my name ...19:35
*** subscope has joined #openstack-swift19:39
peterlisaktdasilva, notmyname, timburke, mattoliverau hi, sorry to bother you. I made a patch 241342, there are two solutions: patch set 2 (extend current list output) and patch set 3 (keep current list output and add new option to extend it). What patch set do you suggest more convenient? thx19:42
patchbotpeterlisak: https://review.openstack.org/#/c/241342/ - Add content-type in list of container content19:42
onovypeterlisak, https://review.openstack.org/#/c/243795/19:43
peterlisakonovy, is it for related project?19:49
onovypeterlisak, yep19:49
brianclineI would just like to take a moment to proclaim how much I love pbr19:50
claygwee!19:50
claygpeterlisak: well I don't think it's the topology that's the problem ( in the comments you even said when I start from scratch it works)19:51
claygpeterlisak: I need the replica2part2dev table19:51
claygpeterlisak: so yeah, if onovy could send it that would be amazing19:51
*** daemontool has joined #openstack-swift19:51
onovyclayg, peterlisak: .builder file from production, right?19:52
*** tsg has joined #openstack-swift19:52
peterlisakonovy, yes19:52
claygwhich ever one fails the change!19:52
notmynameok, clayg and I just "discussed" the pyeclib stuff. there was shouting, wall-hitting, and we were asked to move to a separate room. but we have a plan now! :-)19:52
clayglol ^ all true!19:52
notmynametsg: ^19:52
onovypeterlisak, ^^ but we don't have problem with production rings19:52
onovyi'm missing something? :)19:53
claygnotmyname: althoguth I think until neutron is fixed we're all just spinning our wheels19:53
onovypeterlisak, i think we only have scenario which fails19:53
notmynameclayg: well yeah, but at least when that's fixed we won't be waiting19:53
onovyhttps://etherpad.openstack.org/p/tokyo-swift-rings // scenario here19:53
tdasilvanotmyname, clayg: that must have been entertaining ;)19:53
claygtdasilva: meh, typical clayg rant19:54
peterlisakonovy, object.builder ... I tried to rebalance our prod. builder with patch 241571 today and it failed19:54
patchbotpeterlisak: https://review.openstack.org/#/c/241571/ - Put part-replicas where they go19:54
onovypeterlisak, ah!19:54
notmynamehere's the new plan: clayg will rebase patch 243316 against master. that will make patch 238926 unnecessary. then we'll have code that will work with pyeclib 1.0.7 or greater19:55
patchbotnotmyname: https://review.openstack.org/#/c/243316/ - Update code to work with requirements19:55
patchbotnotmyname: https://review.openstack.org/#/c/238926/ - EC: Use ec_type=liberasurecode_rs_vand in unit tests19:55
notmynamethen we'll update g-r with >=1.0.7, then swift with >=1.0.7, then g-r with >=1.1.1, then swift with >=1.1.1 and remove jerasure19:55
onovyhttp://ondrej.org/object.builder19:56
onovyhere it is19:56
notmynametsg: so very very similar. but now without swift's requirements lying to the world about what works (just in case there are any gaps)19:56
notmynameIMO this is how we'd have normally done it anyway (with any dependency) but we got to the other weird g-r dance first because of asking how to update 2 repos in lockstep19:57
*** diazjf has quit IRC19:58
*** jlhinson_ has joined #openstack-swift20:01
*** jlhinson has quit IRC20:01
*** diazjf has joined #openstack-swift20:02
tsgnotmyname: agreed20:03
notmynamealways fun to read (even if historically not a good fit for swift) https://www.backblaze.com/blog/cloud-storage-hardware/20:04
onovynotmyname, hmm nice hardware20:06
minwoobclayg: So the issue with the tests I added in 196848 (and the other tests in test/probe/test_reconstructor_revert.py) seems to be that we aren't actually verifying whether the fragment was reverted back to its location, or if actually the reconstructor rebuilt it.20:07
*** zhill is now known as zhill_afk20:08
minwoobclayg: So in 196848 the change made to the reconstructor itself is to abort the rebuild if the fragment is found via one of the responses.20:08
tsgclayg: in your patch 243316, let's leave "liberasurecode_rs_vand" out until we up the requirement to 1.1.120:08
patchbottsg: https://review.openstack.org/#/c/243316/ - Update code to work with requirements20:08
onovynotmyname, why it's not good for swift?20:08
minwoobclayg: Therefore, it looks to me that it does not matter whether the added probe tests pass before and after the change, since it makes sense that they would pass before the change as well (fragment is rebuilt and placed in the proper location, as opposed to being reverted there).20:09
notmynameonovy: not sure about the 5.0 version, but earlier versions were very oversubscribed on IO bandwidth. that worked ok with the backblaze archive/backup model, but not for swift clusters where there's a lot of disk activity (even with background processes)20:09
onovynotmyname, ah20:10
notmynamehowever, with their new B2 service, the newer pods are probably getting better for that20:10
minwoob(cschwede as well).20:10
*** pdardeau has joined #openstack-swift20:11
notmynamethis one has much better CPU and more memory. still looks like it's using sata port multipliers20:12
tsgclayg: never mind20:12
notmynameonovy: would be interesting to compare this one to an avoton or xeon-d SoC box. you can get those with 12 drives for 1U, so you'll end up with 3 more drives in 4U. would be interesting to compare price and performance20:13
minwoobclayg: cschwede: The fix that came before this one made sure that the reconstructor does not rebuild using an existing fragment (if it is found), but it will still proceed with the rebuild using the other fragments.20:13
*** klrmn1 has quit IRC20:13
*** CaioBrentano has quit IRC20:14
claygsorry I was afk20:15
claygonovy: peterlisak: so I saw a comment that said "no home for device" -> I want the ring that raised that error20:15
claygicanhaz!?20:15
onovynotmyname, yes really interesting. we are using dell/ibm 2u servers with 24 drives and really: it's not cheap :)20:16
tdasilvanotmyname: what software does blackblaze use ?20:16
*** lpabon has quit IRC20:16
minwoobclayg: cschwede: Which makes me think that these probe tests should have been with the previous fix, and the current one should test whether revert vs. rebuild occurred.20:17
*** itlinux has joined #openstack-swift20:17
peterlisakclayg, here http://ondrej.org/object.builder ... onovy posted it few minutes ago20:18
tsgclayg: I abandoned patch 238926, patch 243316 is next now :)20:18
patchbottsg: https://review.openstack.org/#/c/238926/ - EC: Use ec_type=liberasurecode_rs_vand in unit tests20:18
patchbottsg: https://review.openstack.org/#/c/243316/ - Update code to work with requirements20:18
*** gyee has quit IRC20:18
tsgclayg: next in* is what I meant :) all yours20:19
openstackgerritClay Gerrard proposed openstack/swift: EC: Use best available ec_type in unittests  https://review.openstack.org/24331620:19
claygtsg: notmyname: ^ I think that's right?  hopefully it'll work20:20
mahatic_torgomatic_: ping20:21
mahatic_or clayg20:21
claygmahatic_: pong20:21
mahatic_or anyone else: https://review.openstack.org/#/c/173497/5/swift/proxy/controllers/base.py I wonder if the condition in 1069 is ever true since content-type i always set in Response20:22
mahatic_is*20:22
mahatic_is ever false** (damn!)20:23
claygminwoob: oic, so is it not *possible* to write a probe test to demonstrate the behavior of "skipping the rebuild when frag is on other node" - or just that these tests don't do it?20:23
claygminwoob: I mean there *is* a behavior change introduced in the change - so it seems like it would be *possible* to construct a scearnio that would demonstrate the behavior - or else - why change the behavior if it's un-observable?20:23
claygmahatic_: it looks defensive to me20:25
claygmahatic_: I think in *our* httplib.Response we should never find an object server that forgot to put in *something* for the Content-Type header20:25
*** lifeless has quit IRC20:26
claygmahatic_: but in practice you *may* have a valid HTTP response that does not include a content-type (e.g. most 3XX responses do not have a Content-Type)20:26
*** badari has joined #openstack-swift20:26
*** rjaiswal has quit IRC20:26
*** rjaiswal has joined #openstack-swift20:27
claygmahatic_: there's also unittests mocking httplib to return some faked up thing that maybe didn't include a content-type header (i.e. not a good fake for the real object server)20:27
clayg*generally* it's totally reasonable to track down those tests and fix them to be a better fake by acting more like the system they're faking - but otoh coding for the web has taught us to embrance liberal in what you accept - so putting in the defensive code in your response handler also feels natural20:28
claygmahatic_: is it causing you some greif?20:28
*** moravec has left #openstack-swift20:28
glange*clayg*20:29
claygtsg: I think having the liberasure in the ec-type-preferences list makes sense as soon as we say >=1.0.7 - so we can add it in that change or now - i'm game either way?20:29
mahatic_clayg: sorry, was afk. Yeah, I mean it is called in every GET and PUT which means every content-type is set to ´multipart/byteranges´ ?20:29
claygmahatic_: wat?20:30
mahatic_(not afk - was in a vm and IRC client is outside of it)20:30
claygis_multipart = content_type == 'multipart/byteranges'20:30
*** CaioBrentano has joined #openstack-swift20:30
mahatic_if ct:is_multipart = content_type == 'multipart/byteranges'20:30
mahatic_clayg: yeah20:30
claygso is_multipart depends on if the content_type of the response is equal to 'multipart/byteranges' ???20:30
mahatic_yeah, thatś precisely my question20:31
*** mzhou has quit IRC20:32
claygsorry that was a rehtorical (???) as in "a monad is just a monoid in the category of endofunctors, what's the problem"?20:32
*** mzhou has joined #openstack-swift20:32
claygso if not ct is probbably mostly true, and is_multipart is either true or false dendending on the content-type of the response being equal to 'multipart/byteranges'20:33
pelusemahatic_:  so I just noticed that LOC that clay posted up there.  earlier you were asserting that anything with a content_type would be considered multipart, that's not what that LOC does20:33
claygI don't see where is_multipart is effecting the response that get's sent to the client...20:33
claygmahatic_: type in x = 1 == 2 and then print x20:34
pelusemahatic_: so you get it that is_multipart is only set if content_type is 'multipart/byteranges' right (and not set just if any old content type is present)?20:34
mahatic_clayg: that makes sense :D20:34
claygwerd20:34
minwoobclayg: Basically, that these tests don't do it. I'm not sure of a way to discern whether it was rebuilt vs. reverted, based solely on outcome alone.20:34
claygminwoob: oic - crap20:35
mahatic_clayg: thanks for the example (stupid me duh)20:35
minwoobclayg: It would require verifying which code path was taken in the code itself -- if that's appropriate for a probe test, then yes it's possible.20:35
mahatic_peluse: yeah, it now does20:35
pelusemahatic_:  cool, I didn't see that line earlier, sorry :)20:35
*** pdardeau has quit IRC20:36
mahatic_peluse: ah, no problem! and I am losing my eye sight (no diff between = and ==)20:36
*** pdardeau has joined #openstack-swift20:36
pelusemahatic_: lost mine year ago!  well, never really had it to being with but WTF...20:36
mahatic_:D20:36
claygminwoob: i had convinced myself that we could verify it simply by controlling which node (conf) was holding which frag and running them indipendenly in a controlled fashion20:36
claygmahatic_: don't call yourself stupid - you learned something - that's smart as all get out - most people are terrible at absorbing new information20:37
claygminwoob: ok, well leave that change for now - i'll start it - i want to take a stab at the probe test - if I find something I like I'll push it up and we can brig cschwede around ;)20:38
mahatic_heh, sure20:38
claygmahatic_: you could also refactor it be more clear, there's lots of places where we 'return x == y' instead of 'return True if x == y else False' - i'm not sure the former is as readable20:40
claygit's less characters20:40
minwoobclayg: Sounds good. Let me know if you need more info about it.20:40
mahatic_clayg: actually the former is kinda readable i think; but = and == together blinded me!20:41
claygminwoob: thanks for getting it this far20:41
claygdoh!20:41
minwoobclayg: Np.20:41
clayg:P20:41
minwoobclayg: And also, thanks for the help on it.20:41
claygphththt - it's nothing20:41
mahatic_x == y is ok if one has programmed a little; seems = and == is not :P or itś just me. Iĺl get some sleep20:43
mahatic_good night. Thanks clayg !20:44
mahatic_and peluse !20:44
claygg'night20:44
*** mahatic_ has quit IRC20:44
*** tsg has quit IRC20:46
*** klrmn1 has joined #openstack-swift20:47
*** blmartin has quit IRC20:48
*** jlhinson_ has quit IRC20:50
*** jerrygb has quit IRC20:51
claygso for weeks I was using {t[-1]: r for (t, r) in wr.items() if len(t) == 2} but reducing all the failure domains in a tier to just their last digit doesn't work out so well when you have duplicates20:54
claygdid you know you can have z1 in r1 and r2 - because I apparently have selective memory20:54
*** CaioBrentano has quit IRC20:54
*** jlhinson has joined #openstack-swift20:57
claygonovy: anyway - that's not the problem with your ring20:57
*** CaioBren_ has joined #openstack-swift20:57
onovyclayg, which is good or bad news? :)20:57
claygonovy: peterlisak: *acctually* there's nothing wrong with your ring - thanks for catching this bug!20:57
onovyah, good news20:58
onovyperfect20:58
claygonovy: this is great node!  we'll merge code that's better than it would have been other wise!  You're like a hero!20:58
claygonovy: thanks!20:58
onovy:)))20:58
onovypeterlisak, ^^ this is for you, not me20:58
onovyi just sent one file20:58
*** jerrygb has joined #openstack-swift20:59
*** jerrygb has quit IRC20:59
claygyeah - but like - you sent it *really* good20:59
clayg;)20:59
onovy:))))20:59
claygbah - i acctually don't know how i'm going to fix this yet - but still better that we know about it than not - i'm sure I'll come up with something20:59
*** jerrygb has joined #openstack-swift20:59
*** diazjf has quit IRC20:59
clayganyway - here's the skinny - https://gist.github.com/clayg/d9d908c1aaa37ac66321 - four zones and four replicas + floating point rounding21:03
notmynamelal @ the buffer playback21:03
notmynamehmm... "a" isn't anywhere near "o" on my keyboard21:04
*** jlhinson has quit IRC21:04
claygyou were going for lawl21:04
claygurban dictionary is so much fun - I've never heard of "pronouncing" ROFL as "waffles" but I'm totally going to do that now21:06
claygalso - my new title is Cheif Executive Captain Bringdown21:06
*** lifeless has joined #openstack-swift21:10
*** CaioBren_ has quit IRC21:12
*** jerrygb_ has joined #openstack-swift21:12
*** jlhinson has joined #openstack-swift21:12
openstackgerritPaul Dardeau proposed openstack/swift: Added unit tests for ringbuilder command-line utility  https://review.openstack.org/24007621:12
*** klrmn1 has quit IRC21:15
*** klrmn1 has joined #openstack-swift21:15
*** jerrygb has quit IRC21:16
*** klrmn1 has quit IRC21:16
*** jerrygb_ is now known as jerrygb21:17
*** klrmn1 has joined #openstack-swift21:17
peluseclayg: hey I got a new title this weekend too (not a joke though), CSCS.... http://www.nsca.com/CSCS_Certification_2/21:18
notmynameclayg: yeah, looks good. running tests21:19
jrichlipeluse: congrats!21:19
pelusejrichli: thanks, it was a lot harder than you might think!!21:20
*** klrmn1 has quit IRC21:22
*** klrmn1 has joined #openstack-swift21:23
*** jamielennox is now known as jamielennox|away21:25
notmynamepeluse: it doesn't look easy ;21:26
peluseso if there are any pro athletes in the channel who need a trainer.... :)21:27
notmynameheh21:27
pelusetook 5 months of studying and a 4.5 hr test at the end21:27
peluseif acoles EC patches were only that easy!21:28
peluseacoles_: JK of course, finally reviewing them... sorry it took me so long to get to them21:29
wbhuber_peluse: congrats!  must be tuff to find time to study and take the course!21:31
pelusewbhuber_: yeah, well, that's what weekends and vacation time if for right?  heh21:31
*** jbonjean has joined #openstack-swift21:32
wbhuber_right.  since the field is broad, you'd need to tackle every possible sport - running, tri-athetes, cycling, etc.  quite impressive!21:33
pelusewbhuber_: yeah, the test included everything from rowing technique to cycling to football.  Main emphasis was on science of strength and conditioning but developing a program for different sports requires a decent knowledge of the sport as well...21:38
*** diazjf has joined #openstack-swift21:39
openstackgerritSivasathurappan Radhakrishnan proposed openstack/swift: Added unit test cases for server.py  https://review.openstack.org/24230021:39
*** esker has joined #openstack-swift21:42
*** petertr7 is now known as petertr7_away21:42
*** itlinux has quit IRC21:44
*** itlinux has joined #openstack-swift21:44
*** dustins has quit IRC21:46
*** zhill_afk is now known as zhill21:47
*** itlinux has quit IRC21:47
notmynametdasilva: I just ran into the exact same problem21:49
notmynameI had to install virtualenv21:50
claygpeluse: right on bro that's awesome! (Re: CSCS)21:50
*** ChanServ changes topic to ""you didn't blow away your entire deployment and install everything fresh from the very latest version?! wow.""21:51
*** esker has quit IRC21:51
notmynameclayg: ok, *now* I'll actually run tests on that patch21:52
peluseclayg: so at the next hackathon I'll setup like an obstacle course or something for everyone or something like that for our team event :)21:52
*** esker has joined #openstack-swift21:52
claygnotmyname: it passed the gate!  patch 24331621:53
patchbotclayg: https://review.openstack.org/#/c/243316/ - EC: Use best available ec_type in unittests21:53
notmynamemerge it!21:53
*** thumpba has quit IRC21:54
jrichlipeluse: like the ninja warrior course?  that'd be cool :-)21:54
pelusejrichli: heack yeah, but with my course there will be a differet shot flavor before each obstacle that one must consume... oh yeah baby!21:55
claygtdasilva: you're on a review tear - thanks!21:56
* clayg goes back into his hoe to play with rings21:56
notmynamepeluse: the first thing clayg said on friday at the summit was "Paul tried to kill me"21:56
*** esker has quit IRC21:56
*** mzhou_ has joined #openstack-swift21:56
*** mzhou has quit IRC21:57
tdasilvanotmyname: here's how i fixed it: https://github.com/thiagodasilva/ansible-saio/commit/3231d4683e72f04bc259adef92a6bcc599fe26aa21:57
claygpeluse: I had a coffee in both hands and I'm sure I looked like death - I'm not 100% sure my death was your ultimate goal - but I do know if I'd have tried to keep up with you much longer I wouldn't have made it home21:57
*** mzhou_ is now known as mzhou21:57
tdasilvaclayg: trying to keep up with you guys ;)21:57
*** jerrygb has quit IRC21:57
peluseclayg: ha!  yeah, that was pretty rough fr sure but I'm proud of you son, you did well :)21:58
notmynamelol21:58
tdasilvapeluse: clayg did look pretty funny on Friday morning, his first words were: peluse tried to kill me last night and the room burst out laughing21:59
clayg:D21:59
claygpeluse: I probably just need more training21:59
peluseclayg: it's all about practice for sure.  i actually didn't feel too bad, was going to head over but fell asleep since I didn't get back to the room til, well, lets leave it at that....22:01
*** jlhinson_ has joined #openstack-swift22:05
*** jlhinson has quit IRC22:08
*** jamielennox|away is now known as jamielennox22:24
claygso you know what's great about floats?  sum(x for i in range(46)) * 2 == sum(x for i in range(92)) is only true for most values of x!22:28
*** Zyric1 has joined #openstack-swift22:30
jrichlifun!22:32
peluseclayg: are you using an old pentium?22:32
peluseI think I have a keychin with one of those procs in it :)22:33
*** diazjf has quit IRC22:33
*** pdardeau has quit IRC22:35
jrichliacoles_ and others interested in crypto:  I have been digging into the content-type issue recently brought up in channel.22:35
jrichliThe content-type crypto-meta only gets chopped-off when update_data_record is called, which is only container listings.  That is because this is the only place that swift will split on ';' in content-type to look for swift_bytes, which is only inserted for SLO manifests.22:36
jrichliSo, that is one issue, we need to maybe use a different delimiter than what is used for swift_bytes.22:36
jrichliBut, to answer the question: why didn't we see this issue earlier?22:36
jrichlitest/functional/tests.py:TestSlo.test_slo_overwrite_segment_with_manifest is a case where content-type in encrypter is application/octet-stream;swift_bytes=2097152, but there is no container listing performed, so it passes.22:36
jrichliSo, we need a functest that combines using SLO manifest and container listings22:36
*** km has joined #openstack-swift22:37
*** km is now known as Guest9397122:38
claygjrichli: how did we not get a functional test for that silly bytes feature of slo's into tree when we merged it!?22:39
claygdfg: ^22:39
*** tongli has quit IRC22:39
jrichliclayg: there were unit tests created :-).  I did dig up the review that added that feature.22:40
torgomatic_what would the functest look like? a HEAD request that gets the bytes right?22:40
claygtorgomatic_: nah this is the thing that makes "bytes" returned in the container listing be different from the size of the file for slo manifests22:40
torgomatic_clayg: oh right; it's jammed in the content type or something22:41
claygtorgomatic_: yup22:41
notmynameand it works on master, right? it's just that the parsing wasn't accounted for in crypto because there wasn't a test for it22:41
jrichlinotmyname: I assume so.   I didn't test by hand.22:42
claygnotmyname: yeah works on master22:43
notmynamejrichli: and so the part that breaks is that bytes is in the content type. and you're encrypting content-type so you can't get the "real" bytes22:44
claygcontent_type = application/octet-stream;swift_bytes=531133122:44
notmynameclayg: wait. that's in the DB or that's in the response?22:44
claygit's in the db so that it can "fix" the response22:45
notmynameright22:45
jrichlinotmyname: well, the first step for us to do is use a different delimiter for the crypto-meta.  it doesn't even get that far now22:45
claygi was wondering where content-type was acctually getting the thing added - but didn't know what to grep for22:45
claygturns out swift_bytes is in the container GET (object listing) and middleware.slo22:46
jrichlinotmyname: after that, I would suggest having encrypter split on ';' as well to get the real content-type before encrypting - similar unwinding behavior for decrypt22:46
claygI wish acoles_ was around to chat with him about fast-POST with content-type update to SLO probably messes with that thing22:47
claygjrichli: *maybe* - I think the slo middleware maybe shouldn't be munging this quite so early22:47
*** asettle-afk is now known as asettle22:48
claygif we just passed X-SLO-Bytes round trip to the container update we could have the only person that knows about the akward munging of content-type and slo-manifest-bytes be the container server (and it's just an implementation detail that he decides to cram it into content-type like a mad man)22:48
jrichlilol22:49
jrichlisounds like a good idea.  I just laugh b/c of the mad man ref22:49
claygjrichli: well lots of good ideas fall apart in practice22:50
claygi don't know if the async updates will already sync arbitrary X-Backend headers22:50
claygi don't know if changing proxies to write to alt header instead of pure content-type will work in an upgrade (I mean I know it won't just "work" I've just not thought enough of who needs to be backwards compatible to make it work or if that's possible)22:51
clayglike I'd guess the container server could be updated to to eat the header if it's there and do the content-type munge on a PUT - but if a proxy with updated middleware ended up making slo-manifest PUTs to container servers that don't know how to do that - those container server might just not write swift_bytes into the content-type - so we could do both?22:53
claygnotmyname: don't you have a change that fixes the warning in unittests?22:54
claygnotmyname: can we +A that?22:54
notmynameclayg: yup https://review.openstack.org/#/c/243419/22:55
claygnotmyname: pfththt kinda weak - you could have just mocked stderr and then written another test to validate no warning is printed when new min_part_hours is passed22:57
claygnotmyname: but it WOMM22:57
notmynamepfththt mine's a 6 character patch ;-)22:57
notmynameand yes, I did think about another patch that would explicitly test both branches to see if the warning was printed at the right time22:58
*** klrmn1 has quit IRC22:59
notmynameand then i didn't do that because I just wanted the warning to go away and the test printing it is unaffected by the warning22:59
notmyname(actually, the test is only testing "this thing ran and didn't crash")22:59
*** klrmn1 has joined #openstack-swift22:59
notmynamebut yeah, having another test that does the check for the warning is the right way to go, i think23:00
*** rjaiswal has quit IRC23:05
notmynameclayg: how were you testing patch 243316?23:05
patchbotnotmyname: https://review.openstack.org/#/c/243316/ - EC: Use best available ec_type in unittests23:05
claygi started to code it up and got so sleepy - we have that RunSwiftRingBuilderMixin but it uses self.tempfile instead of self.tmpfile - what a kludge23:06
*** tsg has joined #openstack-swift23:06
notmynameoh, nice. tsg is back23:06
claygwho was I giving grief about making launchpad bugs for test cleanup because I totally want ot make on now :'(23:07
notmynameclayg: lol23:07
claygnotmyname: i don't think i tested the latest rev - is it broken?23:07
notmynameclayg: https://bugs.launchpad.net/swift/+bug/151483123:07
openstackLaunchpad bug 1514831 in OpenStack Object Storage (swift) "OpenStack Typo" [Undecided,In progress] - Assigned to venkatamahesh (venkatamaheshkotha)23:07
claygyeah that guy - but that's not even test clean up - and he had the patch all ready to go - just weird23:07
*** jerrygb has joined #openstack-swift23:08
claygno I think other paul was trying to add some coverage to recon - and I was like "gah this test module is whack!?"  and he was like "well should we open a bug" and I was like "a bug for a test module seems silly" and then peluse was like "use trello!"23:08
claygat least *I* think that's how it happened23:09
claygbut if I don't write down all the things I wanted fixed in test modules how am I ever going to trick someone else into doing it for me!?23:11
notmynameclayg: tsg: I started with master (test pass), applied patch 243316 (tests pass), upgraded pyeclib to 1.1.1 (tests pass), uninstalled pyeclib and liberasurecode and reinstalled only 1.1.1 (test fail with a jerasure reference)23:12
patchbotnotmyname: https://review.openstack.org/#/c/243316/ - EC: Use best available ec_type in unittests23:12
notmynamewhile upgrading to pyeclib1.1.1 the first time, I manually changed swift's requirements.txt to be >=1.0.723:13
claygtorgomatic_: my brain isn't working to day - this is simple data structure manipulation - but I can't click to the obvious way to spell it so it's not itereating over all the devices and all their tiers all the time -> https://gist.github.com/clayg/edf91dbdec109a01c81123:13
claygtorgomatic_: this is the fix for onovy's rings23:13
tsgnotmyname: I thought we were testing only for the ==1.0.7 case :)23:13
*** siva_krishnan has quit IRC23:13
notmynametsg: I'm testing clay's patch that should make it work for both. ie what should be an upgrade path according to the requirements file23:14
claygnotmyname: upgrade worked but uninstall/reinstall didn't work?23:14
claygnotmyname: sounds like a PEBCAK23:14
notmynameclayg: right23:14
notmynameand likely23:14
*** siva_krishnan has joined #openstack-swift23:14
claygnotmyname: ok ok wait23:14
notmynameexcept that when you upgrade, you get 2 versions of liberasure code23:15
*** wbhuber_ has quit IRC23:15
claygso when you uninstalled did you remove your requirements.txt change?23:15
notmynameno23:15
claygso you have swift requires >= 1.0.7 and pyeclib 1.1.123:15
claygshould work23:15
notmynameright23:15
notmynameuninstall was with pip uninstall and manually rm'ing the .so files23:15
notmynameNov 10 23:11:39 saio liberasurecode[17757]: liberasurecode_backend_open: dynamic linking error libJerasure.so: cannot open shared object file: No such file or directory23:15
*** esker has joined #openstack-swift23:16
peluseoh man23:17
claygwAT? O.o23:17
notmynamewait a minute23:17
claygnp23:17
notmynameI may have found the problem23:17
notmynamesomewhere between keyboard and chair ;-)23:17
tsg:)23:18
notmynameyeah. all good23:20
notmynametsg: and you're actually listed as the Author. clayg is only co-author in the commit message23:20
tsgoh sorry23:20
*** esker has quit IRC23:20
tsgnotmyname: thanks for pointing out23:20
tsgand clayg: thanks23:21
notmynametsg: only know it because I had the exact same thought ;-)23:21
notmynamecool. and peluse already +2'd it, so therefore +A23:21
notmynameyay, progress!23:21
pelusei'm hot today23:21
tsgnotmyname, clayg: did you guys see an issue "error in setup command: Invalid environment marker: (python_version>='3.0')" after haypo's latest commit23:22
*** garthb__ has quit IRC23:22
notmynametsg: yeah, you have to upgrade everything23:22
notmynametsg: see the topic message ;-)23:22
notmynametsg: tdasilva and I both saw it too23:22
tsgLOL23:23
notmynameupgrade pbr, setupptools, and pip. probably other stuff. in fact, you aren't running Arch and installing everything from nightlies?! wow.23:23
*** siva_krishnan has quit IRC23:23
tsgnotmyname: this is a brand new trusty container23:23
tsgah23:23
tsgI may need to upgrade pip23:23
*** garthb__ has joined #openstack-swift23:23
notmynamecontainer?! like docker/lxc/rkt? you're already way ahead!23:24
* notmyname needs to dial back the snark a little23:24
tsgdocker :)23:25
*** chsc has quit IRC23:26
tsgnotmyname: cheaper than vagrant*23:26
claygtorgomatic_: oh goodness thank you for talking me out of doing something stupid23:29
claygtorgomatic_: this new better plan works great23:30
claygoh crap - now I need a test23:30
claygsigh23:30
claygcan I just add onovy's builder to our test suite?  it's only 5M23:30
tsgnotmyname: turns out it was in fact setuptools that needed to be upgraded23:30
notmynametsg: ah ok. same issue tdasilva saw23:31
claygtest_this_ring_works test_that_ring_works test_another_ring_works test_??? <- totally scalable!23:31
claygtsg: it's either pip, pbr, setuptools, virtualenv, or something else - everytime23:31
*** minwoob has quit IRC23:33
tsgclayg: I see :) I tested in a brand new trusty instance - and looks like this time it was setuptools.  Good to put a note in SAIO23:33
tsgclayg: based on https://github.com/testing-cabal/mock/issues/261, and https://github.com/nikolas/pylti/commit/2c556f7cd90c6cb9075a5e975ba2310102059057, we can probably tell that the minimum setuptools version needed is 17.123:34
*** jrichli has quit IRC23:38
*** lcurtis has quit IRC23:42
*** ho has joined #openstack-swift23:44
*** jlhinson_ has quit IRC23:49
clayghow gross is .format(**locals()) like it *feels* gross - but is it really worse than .format(some_var=some_var, other_thing=other_thing)23:50
claygruby has native string interpolation with the fancy "#{some_var} is #{other_thing}" and it's pretty terse23:51
claygtsg: it doesn't matter how often you update the docs - it'll be broken in again in a few weeks - see topic23:53
hogood morning!23:53
torgomatic_clayg: it's pretty gross; it basically makes the variable grep-proof23:57
* torgomatic_ isn't a big fan of ruby's string interpolation either, tbh23:58
* briancline gags23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!