| *** haomaiwang has quit IRC | 00:01 | |
| *** haomaiwang has joined #openstack-swift | 00:01 | |
| notmyname | onovy: yeah, I think that looks good. much better option with this patch set. thanks | 00:06 |
|---|---|---|
| *** Vinsh has quit IRC | 00:23 | |
| *** Vinsh has joined #openstack-swift | 00:24 | |
| *** zhill has joined #openstack-swift | 00:31 | |
| *** Jeffrey4l__ has quit IRC | 00:39 | |
| *** rminmin has joined #openstack-swift | 00:46 | |
| kota_ | hello | 00:54 |
| peluse | NICE! | 00:54 |
| *** diogogmt has joined #openstack-swift | 00:56 | |
| *** haomaiwang has quit IRC | 01:01 | |
| *** haomaiwang has joined #openstack-swift | 01:01 | |
| *** nakagawamsa has joined #openstack-swift | 01:06 | |
| *** rminmin has quit IRC | 01:09 | |
| mattoliverau | kota_: o/ | 01:13 |
| kota_ | mattoliverau: o/ | 01:13 |
| *** garthb has quit IRC | 01:27 | |
| *** km_ has joined #openstack-swift | 01:28 | |
| *** km__ has quit IRC | 01:29 | |
| *** badari has quit IRC | 01:32 | |
| *** Vinsh has quit IRC | 01:32 | |
| *** Vinsh has joined #openstack-swift | 01:33 | |
| *** zhill has quit IRC | 01:38 | |
| *** haomaiwang has quit IRC | 02:01 | |
| *** haomaiwang has joined #openstack-swift | 02:01 | |
| *** diazjf has joined #openstack-swift | 02:03 | |
| *** doxavore has joined #openstack-swift | 02:04 | |
| kota_ | nakagawamasa: still? | 02:12 |
| *** klrmn has quit IRC | 02:12 | |
| kota_ | nakagawamasa: https://bitbucket.org/kmgreen2/pyeclib/commits/4bff4a92ea2ab336081737f9f1acc1a3c949af78 might save you quickly on patch 207790 | 02:13 |
| patchbot | kota_: https://review.openstack.org/#/c/207790/ - Update SAIO setup for Ubuntu 14.04 | 02:13 |
| kota_ | nakagawamasa: newest pyeclib doesn't include liberasurecode tar ball so an user should install via OS packaging system like apt or yum. | 02:15 |
| *** zhill has joined #openstack-swift | 02:29 | |
| *** zhill has quit IRC | 02:32 | |
| *** zhill has joined #openstack-swift | 02:32 | |
| *** zhill has quit IRC | 02:36 | |
| *** badari has joined #openstack-swift | 02:42 | |
| *** Vinsh_ has joined #openstack-swift | 02:46 | |
| nakagawamsa | kota_: thanks infomation! I'll retry unittest with "sudo apt-get install liberasurecode-dev" | 02:48 |
| *** Vinsh has quit IRC | 02:50 | |
| *** haomaiwang has quit IRC | 03:01 | |
| *** haomaiwang has joined #openstack-swift | 03:01 | |
| *** badari has quit IRC | 03:01 | |
| *** diazjf has quit IRC | 03:05 | |
| openstackgerrit | Janie Richling proposed openstack/swift: Fix POST-as-COPY mess up the DLO manifest https://review.openstack.org/252791 | 03:11 |
| *** Jeffrey4l has joined #openstack-swift | 03:11 | |
| jrichli | ^^ rebase only | 03:11 |
| *** diazjf has joined #openstack-swift | 03:12 | |
| *** diazjf has quit IRC | 03:13 | |
| *** klrmn has joined #openstack-swift | 03:19 | |
| openstackgerrit | yapeng Yang proposed openstack/swift: Replace assertEqual(None, *) with assertIsNone https://review.openstack.org/263547 | 03:24 |
| *** Jeffrey4l has quit IRC | 03:26 | |
| *** km_ has quit IRC | 03:26 | |
| *** robefran has quit IRC | 03:26 | |
| *** km_ has joined #openstack-swift | 03:27 | |
| *** Jeffrey4l has joined #openstack-swift | 03:28 | |
| *** sanchitmalhotra has joined #openstack-swift | 03:28 | |
| *** Jeffrey4l has quit IRC | 03:42 | |
| *** Jeffrey4l has joined #openstack-swift | 03:43 | |
| *** doxavore has quit IRC | 03:44 | |
| *** venkat has joined #openstack-swift | 03:46 | |
| *** links has joined #openstack-swift | 03:52 | |
| *** haomaiwang has quit IRC | 04:01 | |
| *** haomaiwang has joined #openstack-swift | 04:01 | |
| *** trifon has joined #openstack-swift | 04:05 | |
| *** haomaiwang has quit IRC | 04:13 | |
| *** Jeffrey4l has quit IRC | 04:21 | |
| *** ppai has joined #openstack-swift | 04:24 | |
| *** suresh_ has joined #openstack-swift | 04:34 | |
| *** Jeffrey4l has joined #openstack-swift | 04:34 | |
| suresh_ | I am getting warning NOTE: Dispersion of 64.062500 indicates some parts are not optimally dispersed. | 04:35 |
| suresh_ | while doing swift-ring-builder account.builder rebalnce | 04:36 |
| suresh_ | please some one help | 04:36 |
| *** trifon has quit IRC | 04:39 | |
| *** tsg_ has quit IRC | 04:42 | |
| *** zhill has joined #openstack-swift | 04:48 | |
| *** haomaiwang has joined #openstack-swift | 04:53 | |
| *** SkyRocknRoll has joined #openstack-swift | 04:54 | |
| openstackgerrit | yapeng Yang proposed openstack/swift: Replace assertEqual(None, *) with assertIsNone https://review.openstack.org/263547 | 04:56 |
| *** haomaiwang has quit IRC | 05:01 | |
| *** haomaiwang has joined #openstack-swift | 05:01 | |
| *** PsionTheory has quit IRC | 05:03 | |
| *** miurahr has joined #openstack-swift | 05:11 | |
| *** zhill has quit IRC | 05:13 | |
| *** yatin has joined #openstack-swift | 05:27 | |
| *** ppai has quit IRC | 05:29 | |
| openstackgerrit | yapeng Yang proposed openstack/swift: Use assertTrue/False instead of assertEqual(T/F) https://review.openstack.org/263560 | 05:32 |
| *** ppai has joined #openstack-swift | 05:42 | |
| *** klrmn has quit IRC | 05:43 | |
| *** yarkot has joined #openstack-swift | 05:55 | |
| *** km_ has quit IRC | 05:55 | |
| *** km_ has joined #openstack-swift | 05:57 | |
| *** haomaiwang has quit IRC | 06:01 | |
| *** haomaiwang has joined #openstack-swift | 06:01 | |
| *** gyee has quit IRC | 06:01 | |
| *** trifon has joined #openstack-swift | 06:05 | |
| *** joker_ has quit IRC | 06:06 | |
| *** peterlisak has joined #openstack-swift | 06:11 | |
| *** lcurtis has quit IRC | 06:11 | |
| openstackgerrit | yapeng Yang proposed openstack/swift: Replace assertEqual(None, *) with assertIsNone https://review.openstack.org/263547 | 06:13 |
| *** joker_ has joined #openstack-swift | 06:18 | |
| *** venkat has quit IRC | 06:19 | |
| *** yarkot has quit IRC | 06:20 | |
| openstackgerrit | yapeng Yang proposed openstack/swift: Use assertTrue/False instead of assertEqual(T/F) https://review.openstack.org/263560 | 06:22 |
| *** blmartin has joined #openstack-swift | 06:25 | |
| *** venkat has joined #openstack-swift | 06:31 | |
| *** Jeffrey4l has quit IRC | 06:36 | |
| suresh_ | I am getting warning NOTE: Dispersion of 64.062500 indicates some parts are not optimally dispersed. while doing swift-ring-builder account.builder rebalnce | 06:36 |
| suresh_ | please some one help | 06:37 |
| *** Jeffrey4l has joined #openstack-swift | 06:37 | |
| *** ChubYann has quit IRC | 06:44 | |
| kota_ | suresh_: docs are here, http://docs.openstack.org/developer/swift/overview_ring.html#dispersion | 06:45 |
| kota_ | suresh_: for that case, it would be good to review device weight in your ring or overload value. | 06:46 |
| kota_ | suresh_: docs about overload is a bit above in the dispersion document. | 06:47 |
| *** blmartin has quit IRC | 06:55 | |
| *** km_ has quit IRC | 06:58 | |
| *** km_ has joined #openstack-swift | 06:59 | |
| *** haomaiwang has quit IRC | 07:01 | |
| *** 18WABPMMU has joined #openstack-swift | 07:01 | |
| openstackgerrit | yapeng Yang proposed openstack/swift: Use assertTrue/False instead of assertEqual(T/F) https://review.openstack.org/263560 | 07:22 |
| *** Jeffrey4l has quit IRC | 07:30 | |
| *** venkat has quit IRC | 07:40 | |
| *** zaitcev has quit IRC | 07:42 | |
| *** Jeffrey4l has joined #openstack-swift | 07:51 | |
| *** venkat has joined #openstack-swift | 07:52 | |
| *** 18WABPMMU has quit IRC | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** arnox has joined #openstack-swift | 08:03 | |
| *** eranrom has joined #openstack-swift | 08:08 | |
| *** rledisez has joined #openstack-swift | 08:12 | |
| *** Jeffrey4l has quit IRC | 08:18 | |
| *** Jeffrey4l has joined #openstack-swift | 08:21 | |
| *** eranrom has quit IRC | 08:30 | |
| *** eranrom has joined #openstack-swift | 08:39 | |
| *** m_kazuhiro has joined #openstack-swift | 08:45 | |
| *** m_kazuhiro has quit IRC | 08:46 | |
| *** haomaiwang has quit IRC | 08:53 | |
| *** haomaiwang has joined #openstack-swift | 08:53 | |
| *** jsuchome has joined #openstack-swift | 08:59 | |
| *** haomaiwang has quit IRC | 09:01 | |
| *** haomaiwang has joined #openstack-swift | 09:01 | |
| *** jordanP has joined #openstack-swift | 09:05 | |
| openstackgerrit | Merged openstack/swift: Port swift.common.utils.StatsdClient to Python 3 https://review.openstack.org/237005 | 09:14 |
| openstackgerrit | Merged openstack/swift: Fix AttributeError for LogAdapter https://review.openstack.org/261619 | 09:15 |
| *** jistr has joined #openstack-swift | 09:21 | |
| *** km_ has quit IRC | 09:21 | |
| *** janonymous has joined #openstack-swift | 09:23 | |
| *** joeljwright has joined #openstack-swift | 09:29 | |
| *** ChanServ sets mode: +v joeljwright | 09:29 | |
| eranrom | cschwede: Have you abondoned the notification imddleware? Going over the comments seems like it got stuck over the problems raised with the need to be reliable form the Swift side. Any news? | 09:31 |
| cschwede | eranrom: it’s not (yet) abandoned: https://review.openstack.org/#/c/196755/ however, i’m currently not working on it. reliability is one challenge (though doable, imo), but there are other missing parts. have a look at the last two comments from Sam in the review | 09:37 |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Last-Modified header support on HEAD/GET container https://review.openstack.org/198633 | 09:42 |
| *** silor has joined #openstack-swift | 09:44 | |
| eranrom | cschwede: thanks. Yes, as far as I understand one remark has to do with the reliability problem - which if I get right it says keep all in memory and forget about being 100% relaible. the other one seems specific for queues that enforce authorization. Which IMHO cover only part of the use case. | 09:45 |
| cschwede | eranrom: well, to work with Zaqar authentication is required IIRC, so this is a hard problem. Just sending out notifications to a MQ (instead of sending it to Zaqar) is not that useful for public providers, though it might be enough for a private cloud with specific requirements. but then the question raised which MQ should be supported? | 09:52 |
| *** jmccarthy has quit IRC | 09:54 | |
| *** silor1 has joined #openstack-swift | 09:54 | |
| *** jmccarthy has joined #openstack-swift | 09:55 | |
| *** silor has quit IRC | 09:55 | |
| *** silor1 is now known as silor | 09:55 | |
| jsuchome | kota_: hi, any reason for keeping this waiting? https://review.openstack.org/#/c/189258/ | 09:58 |
| *** haomaiwang has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| eranrom | cschwede,: Anything wrong with the ability to just configure a URL for POSTs? Do you think all the problems must be solved before this can land? I can dig deeper into IBM's use case, but seems to me there is a lot of value here even without solving all of the problems. | 10:18 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 10:31 |
| openstackgerrit | Ondřej Nový proposed openstack/swift: swift-init: New option kill-after-timeout https://review.openstack.org/229532 | 10:34 |
| *** hseipp has joined #openstack-swift | 10:37 | |
| *** hseipp has quit IRC | 10:37 | |
| *** hseipp has joined #openstack-swift | 10:37 | |
| kota_ | jsuchome: sorry just dropped from my heads. Will see tomorrow. | 10:40 |
| kota_ | jsuchome: thx, for bringing it up to me. | 10:41 |
| jsuchome | ok | 10:42 |
| cschwede | eranrom: well, we need an agreement what the middleware should be doing. just sending a POST (without a valid Keystone token) is not enough for Zaqar, thus making the middleware useless except when you build your own receiving consumer. and this looks to specialized to me - if we ship a notification middleware with swift itself, it should be usable with some existing broker | 10:57 |
| cschwede | eranrom: however, we could modify the middleware to use an own account to send notifications to zaqar (ie, the middleware needs to authenticate itself to keystone, fetch a token and use that token instead of forwarding a user token) | 10:58 |
| cschwede | eranrom: and if there is no user/password defined for the middleware, no token is sent. that might be a suitable solution?+ | 10:58 |
| *** haomaiwang has quit IRC | 11:01 | |
| *** haomaiwang has joined #openstack-swift | 11:01 | |
| *** aix has quit IRC | 11:25 | |
| *** acoles_ is now known as acoles | 11:34 | |
| *** daemontool has joined #openstack-swift | 11:35 | |
| openstackgerrit | Tomáš Matlocha proposed openstack/swift: Additional error log message for drive-audit https://review.openstack.org/263691 | 11:37 |
| *** Jeffrey4l has quit IRC | 11:48 | |
| *** haomaiwang has quit IRC | 12:01 | |
| *** haomaiwang has joined #openstack-swift | 12:01 | |
| *** yatin has quit IRC | 12:11 | |
| *** nakagawamsa has quit IRC | 12:55 | |
| *** haomaiwang has quit IRC | 13:01 | |
| *** haomaiwang has joined #openstack-swift | 13:01 | |
| *** ppai has quit IRC | 13:02 | |
| *** kei_yama has quit IRC | 13:04 | |
| *** links has quit IRC | 13:10 | |
| *** haomaiwang has quit IRC | 13:12 | |
| *** aix has joined #openstack-swift | 13:14 | |
| *** proteusguy has quit IRC | 13:17 | |
| joker_ | ı am working on this bug https://bugs.launchpad.net/swift/+bug/1509429 | 13:21 |
| openstack | Launchpad bug 1509429 in OpenStack Object Storage (swift) "X-Timestamp missing from object PUT/COPY response headers" [Low,Confirmed] - Assigned to Mustafa (celikmustafa89) | 13:21 |
| joker_ | ı complete x-timestamp headers for account and container operations | 13:21 |
| *** yatin has joined #openstack-swift | 13:22 | |
| joker_ | but for object operation ı have problem with POST and PUT operations | 13:22 |
| joker_ | if someone has experience about this i am gone give the details | 13:23 |
| *** venkat has quit IRC | 13:25 | |
| *** proteusguy has joined #openstack-swift | 13:29 | |
| *** yatin has quit IRC | 13:36 | |
| *** haomaiwang has joined #openstack-swift | 13:40 | |
| *** haomaiwang has quit IRC | 13:45 | |
| *** dslevin has quit IRC | 13:45 | |
| *** suresh_ has quit IRC | 13:49 | |
| acoles | eranrom: fyi i am mid way through reviewing patch 205803 so you may want to wait before addressing mattoliverau's review comments | 13:55 |
| patchbot | acoles: https://review.openstack.org/#/c/205803/ - Container-Sync to iterate only over synced containers | 13:55 |
| *** SkyRocknRoll has quit IRC | 13:57 | |
| *** robefran has joined #openstack-swift | 14:01 | |
| *** dslevin_ has quit IRC | 14:02 | |
| *** 18VAALI1N has joined #openstack-swift | 14:07 | |
| openstackgerrit | LiuNanke proposed openstack/swift: Replace assertTrue(isinstance()) with assertIsInstance() https://review.openstack.org/263742 | 14:08 |
| *** SkyRocknRoll has joined #openstack-swift | 14:08 | |
| *** petertr7_away is now known as petertr7 | 14:13 | |
| *** pdion891 has joined #openstack-swift | 14:21 | |
| *** links has joined #openstack-swift | 14:24 | |
| *** ccontini has joined #openstack-swift | 14:26 | |
| *** ccontini has quit IRC | 14:26 | |
| openstackgerrit | LiuNanke proposed openstack/swift: Replace assertTrue(isinstance()) with assertIsInstance() https://review.openstack.org/263742 | 14:26 |
| *** ccontini has joined #openstack-swift | 14:28 | |
| *** tsg has joined #openstack-swift | 14:29 | |
| *** Vinsh_ has quit IRC | 14:30 | |
| *** Vinsh has joined #openstack-swift | 14:30 | |
| *** badari has joined #openstack-swift | 14:31 | |
| ccontini | Hi all, I have a question about the replication in swift (2.2.0) | 14:34 |
| ccontini | We have a ring built like this | 14:35 |
| ccontini | 33554432 partitions, 3.000000 replicas, 1 regions, 4 zones, 139 devices, 0.00 balance | 14:35 |
| ccontini | The minimum number of hours before a partition can be reassigned is 1 | 14:35 |
| *** Vinsh_ has joined #openstack-swift | 14:36 | |
| *** Vinsh has quit IRC | 14:36 | |
| ccontini | One of the object servers has been having a lot of issues and had a lot of drives changed + was rebooted several times | 14:36 |
| ccontini | Now, I see that in the logs: Jan 5 09:32:31 swift04 object-replicator: 86955/6597974 (1.32%) partitions replicated in 65319.46s (1.33/sec, 1358h remaining) | 14:37 |
| ccontini | It looks like replication is taking forever | 14:37 |
| ccontini | Is there any way we can speed up the process ? | 14:37 |
| ccontini | I know the min_part_hours in the ring should be increased (a lot in our case) but I don't understand exactly what it does | 14:38 |
| *** diogogmt has quit IRC | 14:43 | |
| ahale | the min part stuff is really just to protect against doing crazy stuff, to avoid having the ring move more than one copy of a partition during successive rebalances. i think the idea is to set it around your replication pass time | 14:47 |
| ahale | that seems like quite a high part power for that device count, maybe theres a ton of slow directory walking going on.. i don't know - im not great at making replication go fast | 14:48 |
| ccontini | But from the log it looks like replication takes over 1358h right ? | 14:49 |
| *** links has quit IRC | 14:49 | |
| ccontini | That looks incredibly slow | 14:49 |
| *** Jeffrey4l has joined #openstack-swift | 14:54 | |
| *** lcurtis has joined #openstack-swift | 14:57 | |
| *** blmartin has joined #openstack-swift | 14:58 | |
| *** 18VAALI1N has quit IRC | 15:01 | |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| *** Vinsh_ is now known as Vinsh | 15:04 | |
| *** breitz has quit IRC | 15:12 | |
| *** breitz has joined #openstack-swift | 15:12 | |
| *** jmccarthy1 has joined #openstack-swift | 15:13 | |
| *** jmccarthy1 has quit IRC | 15:13 | |
| *** jmccarthy1 has joined #openstack-swift | 15:13 | |
| *** jmccarthy1 has quit IRC | 15:14 | |
| *** jmccarthy1 has joined #openstack-swift | 15:15 | |
| *** barker has joined #openstack-swift | 15:16 | |
| *** jmccarthy has quit IRC | 15:16 | |
| *** Jeffrey4l has quit IRC | 15:19 | |
| *** SkyRocknRoll has quit IRC | 15:19 | |
| *** changbl has quit IRC | 15:27 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: Modify unit tests to include real crypto https://review.openstack.org/211451 | 15:28 |
| *** barker_ has joined #openstack-swift | 15:28 | |
| *** barker has quit IRC | 15:29 | |
| *** breitz has quit IRC | 15:29 | |
| *** breitz has joined #openstack-swift | 15:30 | |
| *** Jeffrey4l has joined #openstack-swift | 15:31 | |
| *** barker_ has quit IRC | 15:33 | |
| mahatic | acoles: jrichli ^ | 15:37 |
| acoles | mahatic: ack | 15:37 |
| jrichli | mahatic: great, I will take a look | 15:37 |
| *** SkyRocknRoll has joined #openstack-swift | 15:40 | |
| *** Jeffrey4l has quit IRC | 15:41 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 15:45 | |
| *** yarkot has joined #openstack-swift | 15:49 | |
| *** petertr7 is now known as petertr7_away | 15:51 | |
| *** Jeffrey4l has joined #openstack-swift | 15:53 | |
| *** jsuchome has left #openstack-swift | 15:53 | |
| *** dslevin has joined #openstack-swift | 15:54 | |
| *** doxavore has joined #openstack-swift | 15:54 | |
| *** aix has quit IRC | 15:55 | |
| doxavore | If one would like to reduce the I/O load during replication (/rebalancing) to reduce impact on request latency, at the cost of longer replication times, would rsync's max connections be a good place to do that? | 15:55 |
| doxavore | and if so, is there a recommended range you stay between? | 15:56 |
| *** petertr7_away is now known as petertr7 | 15:58 | |
| *** haomaiwang has quit IRC | 16:01 | |
| *** mragupat has joined #openstack-swift | 16:01 | |
| *** haomaiwang has joined #openstack-swift | 16:01 | |
| *** dslevin has quit IRC | 16:06 | |
| *** garthb has joined #openstack-swift | 16:09 | |
| *** yatin has joined #openstack-swift | 16:13 | |
| *** shakamunyi has joined #openstack-swift | 16:14 | |
| *** barra204 has quit IRC | 16:15 | |
| *** diogogmt has joined #openstack-swift | 16:17 | |
| *** diazjf has joined #openstack-swift | 16:21 | |
| *** PsionTheory has joined #openstack-swift | 16:25 | |
| eranrom | acoles: thanks I am waiting :-) | 16:30 |
| acoles | eranrom: its looking good. i just have some suggestions to cleanup some of tests. | 16:31 |
| eranrom | acoles: thanks. awaiting patiently :-) | 16:32 |
| *** zhill has joined #openstack-swift | 16:33 | |
| *** robefran has quit IRC | 16:36 | |
| *** robefran has joined #openstack-swift | 16:38 | |
| *** aix has joined #openstack-swift | 16:43 | |
| *** diazjf has quit IRC | 16:56 | |
| *** ccontini has quit IRC | 16:57 | |
| acoles | eranrom: over to you :) | 16:58 |
| *** diazjf has joined #openstack-swift | 16:59 | |
| *** haomaiwang has quit IRC | 17:01 | |
| *** haomaiwang has joined #openstack-swift | 17:01 | |
| *** jmccarthy1 has quit IRC | 17:02 | |
| openstackgerrit | Stuart McLaren proposed openstack/python-swiftclient: Retry download of object body https://review.openstack.org/159208 | 17:02 |
| *** SkyRocknRoll has quit IRC | 17:03 | |
| *** gyee has joined #openstack-swift | 17:08 | |
| *** jistr has quit IRC | 17:19 | |
| mahatic | acoles: I think master needs to be merged into crypto branch. patch 211451 failed jenkins, looks like it is not recognizing nosetests | 17:22 |
| patchbot | mahatic: https://review.openstack.org/#/c/211451/ - Modify unit tests to include real crypto | 17:22 |
| *** petertr7 is now known as petertr7_away | 17:23 | |
| acoles | mahatic: the unit test failed with AttributeError: 'module' object has no attribute 'get_crypto' | 17:25 |
| mahatic | acoles: ouch | 17:26 |
| mahatic | looking | 17:27 |
| *** silor has quit IRC | 17:27 | |
| mahatic | oh I don't have those tests with me yet, the round trip tests | 17:28 |
| *** petertr7_away is now known as petertr7 | 17:29 | |
| *** yarkot has quit IRC | 17:30 | |
| *** yatin has quit IRC | 17:31 | |
| *** trifon has quit IRC | 17:32 | |
| *** rledisez has quit IRC | 17:35 | |
| *** joeljwright has quit IRC | 17:36 | |
| *** aix has quit IRC | 17:38 | |
| *** arnox has quit IRC | 17:45 | |
| *** mragupat has quit IRC | 17:56 | |
| *** mragupat has joined #openstack-swift | 17:56 | |
| *** jordanP has quit IRC | 17:59 | |
| openstackgerrit | Mahati Chamarthy proposed openstack/swift: Modify unit tests to include real crypto https://review.openstack.org/211451 | 18:00 |
| *** haomaiwang has quit IRC | 18:01 | |
| *** haomaiwang has joined #openstack-swift | 18:01 | |
| *** klrmn has joined #openstack-swift | 18:08 | |
| *** zaitcev has joined #openstack-swift | 18:09 | |
| *** ChanServ sets mode: +v zaitcev | 18:09 | |
| *** changbl has joined #openstack-swift | 18:12 | |
| *** hseipp has quit IRC | 18:15 | |
| *** yarkot has joined #openstack-swift | 18:21 | |
| *** Jeffrey4l has quit IRC | 18:24 | |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Add functional test for object PUT with raw iterator https://review.openstack.org/263838 | 18:34 |
| notmyname | good morning | 18:35 |
| gmmaha | good morning notmyname | 18:35 |
| notmyname | not much happened in here during my overnight | 18:39 |
| acoles | notmyname: we're all hard at work while you sleep :) | 18:40 |
| notmyname | right | 18:40 |
| notmyname | ;-) | 18:40 |
| * acoles will try to make more noise tomorrow | 18:40 | |
| notmyname | lol | 18:40 |
| *** acoles is now known as acoles_ | 18:44 | |
| *** pdion891 has quit IRC | 18:54 | |
| *** haomaiwang has quit IRC | 19:01 | |
| *** haomaiwang has joined #openstack-swift | 19:01 | |
| *** yarkot has quit IRC | 19:05 | |
| *** yarkot has joined #openstack-swift | 19:12 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Fixed few misspellings in comments https://review.openstack.org/263870 | 19:21 |
| *** dslevin has joined #openstack-swift | 19:25 | |
| openstackgerrit | Ondřej Nový proposed openstack/python-swiftclient: Fixed few misspellings in comments https://review.openstack.org/263872 | 19:29 |
| *** sarafraj has joined #openstack-swift | 19:34 | |
| *** minwoob has joined #openstack-swift | 19:38 | |
| onovy | notmyname: thanks :] | 19:38 |
| *** daemontool has quit IRC | 19:39 | |
| *** sarafraj has quit IRC | 19:41 | |
| *** daemontool has joined #openstack-swift | 19:41 | |
| *** ChubYann has joined #openstack-swift | 19:45 | |
| *** daemontool has quit IRC | 19:47 | |
| *** daemontool has joined #openstack-swift | 19:48 | |
| *** daemontool has quit IRC | 19:51 | |
| *** sarafraj has joined #openstack-swift | 19:57 | |
| *** sarafraj has left #openstack-swift | 19:58 | |
| *** haomaiwang has quit IRC | 20:01 | |
| *** haomaiwang has joined #openstack-swift | 20:01 | |
| Zyric | Good Morning | 20:01 |
| onovy | morning? it's 9pm :) | 20:09 |
| *** zhill has quit IRC | 20:09 | |
| *** bjkeller has joined #openstack-swift | 20:18 | |
| *** tsg has quit IRC | 20:24 | |
| *** gyee has quit IRC | 20:25 | |
| *** djc_ has joined #openstack-swift | 20:26 | |
| *** djc_ has quit IRC | 20:30 | |
| *** djc_ has joined #openstack-swift | 20:30 | |
| djc_ | I am getting a message when doing a swift stat: Account HEAD failed: https://x.x.x.x:8443/v1/AUTH_yyyy 503 Internal Server Error | 20:30 |
| djc_ | not sure what to make of it. | 20:31 |
| jrichli | djc_: I would check the logs if you could to see what the internal error was | 20:35 |
| *** joeljwright has joined #openstack-swift | 20:37 | |
| *** ChanServ sets mode: +v joeljwright | 20:37 | |
| djc_ | jrichli: I checked the proxy logs and same type of message: http://pastebin.com/2se1bXED | 20:40 |
| *** yarkot has quit IRC | 20:41 | |
| djc_ | jrichli: my proxy server is actually a lxc container. Not sure if that makes a difference. It shouldn't. | 20:43 |
| jrichli | djc_: check the storage logs | 20:44 |
| *** dslevin has quit IRC | 20:45 | |
| notmyname | djc_: take the trasaction id (txdf0d5292847e403f99ffa-00568c1fbf) and grep all the logs you have for it | 20:48 |
| notmyname | well, you know it will be on your .215 and .214 server | 20:49 |
| *** dslevin has joined #openstack-swift | 20:50 | |
| djc_ | notmyname: .214 and .215 are my storage nodes. I did grep the transaction id from /var/log/messages and it returned nothing. I am not seeing anything unusual on the storage nodes log other than a possible bad disk. | 20:52 |
| openstackgerrit | Bill Huber proposed openstack/swift: Re-format the SLO manifest file on new multipart-manifest GET call https://review.openstack.org/263902 | 20:56 |
| *** mragupat has quit IRC | 20:57 | |
| notmyname | djc_: ah ok, then. for some reason your proxy server wasn't able to get a response from the account servers | 20:58 |
| djc_ | notmyname: I am a relative novice to swift and I am reading the swift book you recommended a couple months back. The swift.conf and *.gz files is the same on all nodes | 20:58 |
| notmyname | djc_: are you running with 3 replicas? | 20:58 |
| *** mragupat has joined #openstack-swift | 20:58 | |
| djc_ | notmyname: 4 replicas | 20:58 |
| djc_ | 4 zones | 20:59 |
| notmyname | hmm | 20:59 |
| *** mragupat has quit IRC | 20:59 | |
| notmyname | interestingly, you don't have the same transaction id on those log messages | 21:00 |
| *** haomaiwang has quit IRC | 21:01 | |
| *** haomaiwang has joined #openstack-swift | 21:01 | |
| *** mragupat has joined #openstack-swift | 21:01 | |
| djc_ | notmyname: looks like my ports are right for all the objects: http://pastebin.com/jY1iZ3Db | 21:02 |
| notmyname | you can automate that check with `swift-recon --validate-servers` | 21:02 |
| Guest79971 | clayg: why u no like my blueprint?! | 21:03 |
| Guest79971 | :D | 21:03 |
| Guest79971 | whoops... | 21:03 |
| *** Guest79971 is now known as scotticus | 21:03 | |
| scotticus | thats better. | 21:03 |
| notmyname | oh hi scotticus ;-) | 21:03 |
| djc_ | notmyname: same with account server: http://pastebin.com/RGMae6Ay | 21:04 |
| notmyname | scotticus: are you referring to dev_id limits? | 21:04 |
| scotticus | the 2^16 limit on devices, yes. | 21:05 |
| *** mragupat has quit IRC | 21:05 | |
| *** mragupat has joined #openstack-swift | 21:05 | |
| *** mragupat has quit IRC | 21:06 | |
| *** mragupat has joined #openstack-swift | 21:06 | |
| notmyname | scotticus: did you see any of the scrollback from yesterday when we talked about it in here? | 21:07 |
| notmyname | (if not I can summarize) | 21:07 |
| scotticus | nope. | 21:07 |
| notmyname | there were a few points | 21:07 |
| scotticus | the summary i was relayed was clayg didn't think it was as valuable as fixing the rebalance. | 21:08 |
| scotticus | that it "wasn't valuable" | 21:08 |
| notmyname | ah | 21:08 |
| scotticus | at least thats what was relayed to me. | 21:08 |
| notmyname | first is that the 65k limit is being hit (from what we're told) because of "holes" in the ring from older devices. so that should be fixed first instead of a 2**16 limit | 21:09 |
| notmyname | that would allow more dev ids with less changes, and something that will benefit more deployers overall | 21:09 |
| scotticus | so that would bandaid the device limit temporarily... | 21:09 |
| scotticus | by freeing up some null devices. | 21:09 |
| notmyname | right | 21:09 |
| scotticus | so since we're probably 1/2 way to that limit, i thought why not start working on somethign like this now. | 21:10 |
| notmyname | second IIRC is that there are some other things that would need to be worked on too (like rebalancing) to support these larger rings, and maybe that should be focused on first (trying to correctly summarize what clayg was saying) | 21:10 |
| *** mragupat has quit IRC | 21:11 | |
| scotticus | whats the timeline for the rebalance changes? | 21:12 |
| notmyname | IMO (me, not clayg) I think it would be great to remove the 16 bit limit (and any limit). but it's a question of effort vs reward. eg filling the holes might be easier first. however pdardeau already is well along on removing the limit, so that's good | 21:12 |
| scotticus | is someone actively working on that? | 21:12 |
| notmyname | no, not AFAIK. | 21:12 |
| scotticus | is there a blueprint for the rebalance stuff? | 21:13 |
| notmyname | I think the discussion was not about if removing the limit was valuable, but what was valuable to do first. that was my takeaway | 21:13 |
| notmyname | I haven't seen pdardeau's patch yet, so I don't know what it involves. and I certainly want to ensure that larger policies keep working | 21:14 |
| scotticus | his patch is for removing the device size limit? | 21:14 |
| notmyname | yeah | 21:14 |
| notmyname | but basically having 30k+ drives in a ring comes with other problems, and the question was about those other things | 21:14 |
| notmyname | so IMO we should have a patch to fill the holes in the ring. and we should plan for removing the device limit altogether. | 21:15 |
| notmyname | IIRC clayg was only asking about the "why now" part. not if it was actually a good thing. just if it is a good thing now | 21:15 |
| notmyname | which I think was what got back to you | 21:15 |
| scotticus | probably better now than later. | 21:16 |
| *** zhill has joined #openstack-swift | 21:17 | |
| pdardeau | notmyname, scotticus: o/ | 21:18 |
| notmyname | so that gets back around to the question that wasn't answered yesterday: how pressing is this? is it a Right Now thing? sometime this year? if current growth continues... | 21:18 |
| notmyname | pdardeau: did all that I just spewed make sense? line up with what you remember from yesterday? | 21:18 |
| pdardeau | notmyname: yes, nice synopsis | 21:19 |
| pdardeau | notmyname, scotticus: i can finish up device limit patch (a few more hours at most) and make it available for review | 21:19 |
| pdardeau | then you guys can get a better feel for whether it's worth accepting at this time | 21:20 |
| *** mragupat has joined #openstack-swift | 21:20 | |
| notmyname | heh, if code's already written, it's much easier to accept ;-) | 21:20 |
| pdardeau | ha, good point! | 21:21 |
| *** chlong has quit IRC | 21:22 | |
| openstackgerrit | Eran Rom proposed openstack/swift: Container-Sync to iterate only over synced containers https://review.openstack.org/205803 | 21:22 |
| notmyname | however, it's not like "put 100k devices in a ring" is likely to be a thing that only this patch is preventing. what about rebalancing? what about part power adjustments? what about ring-of-rings for better placements? | 21:22 |
| pdardeau | notmyname: that's almost like a disturbance in the force, or something ;-) | 21:24 |
| notmyname | scotticus: does all that make sense? I don't remember anything from yesterday about removing or raising the limit being not valuable. just curious about why now and any plans around the other stuff that comes with rings that big | 21:24 |
| scotticus | all of it is good. apparently pdardeau has done what the blueprint calls for. the other stuff would absolutely need to get done at some point. | 21:25 |
| notmyname | and, hey. if pdardeau's got a patch, that's awesome. working code votes ;-) | 21:27 |
| *** Vinsh has quit IRC | 21:29 | |
| *** petertr7 is now known as petertr7_away | 21:29 | |
| *** Vinsh has joined #openstack-swift | 21:30 | |
| *** mragupat has quit IRC | 21:30 | |
| *** gyee has joined #openstack-swift | 21:30 | |
| *** mragupat has joined #openstack-swift | 21:30 | |
| onovy | doxavore: still here? | 21:31 |
| *** mragupat_ has joined #openstack-swift | 21:32 | |
| onovy | doxavore: https://github.com/openstack/swift/blob/master/etc/object-server.conf-sample#L172 this options do "shaping" on source side | 21:32 |
| onovy | doxavore: https://github.com/openstack/swift/blob/master/etc/rsyncd.conf-sample#L19 this on destination side | 21:33 |
| *** Vinsh_ has joined #openstack-swift | 21:33 | |
| *** gyee has quit IRC | 21:34 | |
| *** mragupat has quit IRC | 21:35 | |
| *** Vinsh has quit IRC | 21:37 | |
| mattoliverau | Morning | 21:39 |
| djc_ | notmyname: got any more advice for me? | 21:39 |
| *** gyee has joined #openstack-swift | 21:41 | |
| *** yarkot has joined #openstack-swift | 21:41 | |
| notmyname | djc_: at this point you need to look on the account server and figure out why it's not responding to the proxy. you could test with a known object that goes to that server. or directly to the account server with curl | 21:47 |
| notmyname | djc_: definitely need to make sure that the disk 14 is ok on that server | 21:47 |
| notmyname | djc_: so I dont' have anything specific. just start going through the system to isolate the issue so you can figure out what it is | 21:48 |
| djc_ | notmyname: ok thanks. | 21:48 |
| *** petertr7_away is now known as petertr7 | 21:50 | |
| *** robefran has quit IRC | 21:52 | |
| *** dslevin has quit IRC | 21:54 | |
| *** cdelatte has quit IRC | 21:56 | |
| *** dslevin has joined #openstack-swift | 21:59 | |
| *** haomaiwang has quit IRC | 22:01 | |
| *** haomaiwang has joined #openstack-swift | 22:01 | |
| *** dslevin has quit IRC | 22:04 | |
| *** dslevin has joined #openstack-swift | 22:09 | |
| *** blmartin has quit IRC | 22:11 | |
| onovy | can someone look to https://review.openstack.org/#/c/253038/ pls? | 22:22 |
| *** haomaiwang has quit IRC | 22:23 | |
| *** haomaiwang has joined #openstack-swift | 22:28 | |
| hrou | clayg, thanks a bunch for the comments re: symlink - I just wanted to validate: So are we roughly on the same page that is post-as-copy is just about equally terrible with posting a regular object and a symlink (where we are redirecting the post to the target object i.e. the desired behavior) ... in the sense that we're not really introducing any additional consistency issues ? If so we're going to push forward on evaluating fast-post in sy | 22:37 |
| hrou | (as I think there are some issues there we may need to resolve). | 22:37 |
| *** petertr7 is now known as petertr7_away | 22:37 | |
| *** djc_ has quit IRC | 22:39 | |
| *** mragupat has joined #openstack-swift | 22:41 | |
| *** robefran has joined #openstack-swift | 22:41 | |
| mattoliverau | hey hrou, happy new year! Haven't had a chance so say it to you yet :) | 22:43 |
| hrou | mattoliverau, thanks a lot, and you too ! | 22:43 |
| *** mragupat_ has quit IRC | 22:44 | |
| openstackgerrit | Alan Erwin proposed openstack/swift: Refactoring the expiring objects feature https://review.openstack.org/252085 | 22:44 |
| notmyname | mattoliverau: just got my flights for LCA | 22:50 |
| notmyname | I'm not looking forward to LAX->MEL (15h45m) | 22:51 |
| notmyname | do you take that one when you go home? | 22:51 |
| doxavore | onovy: i'll work with those settings tonight and see where it lands. thanks! | 22:52 |
| mattoliverau | notmyname: \o/ | 22:58 |
| mattoliverau | notmyname: anything thrrough LAX is annoying I prefer via SFO. Having said that, the last few times through LAX has actually been pretty good. I think it's the smart gates that has speed it up.. at least everytime I've been there lately | 22:59 |
| mattoliverau | but yes, from SAT and YVR I came back on LAX > MEL | 23:00 |
| gmmaha | howdy mattoliverau | 23:00 |
| notmyname | yikes | 23:00 |
| *** daemontool has joined #openstack-swift | 23:00 | |
| mattoliverau | gmmaha: hey man | 23:00 |
| mattoliverau | gmmaha: happy new year | 23:01 |
| *** haomaiwang has quit IRC | 23:01 | |
| gmmaha | mattoliverau: happy new year to you too.. :) | 23:01 |
| *** minwoob has quit IRC | 23:01 | |
| *** haomaiwang has joined #openstack-swift | 23:01 | |
| mattoliverau | notmyname: you coming in on Saturday before LCA or earlier? | 23:02 |
| notmyname | actually, just sending you an email about that :-) | 23:02 |
| *** bjkeller has left #openstack-swift | 23:02 | |
| *** doxavore has quit IRC | 23:03 | |
| mattoliverau | notmyname: great, I will reply to the said email once I have recieved and read it :P | 23:04 |
| *** joeljwright has quit IRC | 23:08 | |
| *** mragupat has quit IRC | 23:09 | |
| *** chlong has joined #openstack-swift | 23:12 | |
| *** diazjf has quit IRC | 23:12 | |
| *** chlong has quit IRC | 23:14 | |
| *** robefran has quit IRC | 23:15 | |
| *** kei_yama has joined #openstack-swift | 23:23 | |
| clayg | aerwin3_: did you *read* any of my comments on patch 252085 - you didn't respond to any of them? | 23:27 |
| patchbot | clayg: https://review.openstack.org/#/c/252085/ - Refactoring the expiring objects feature | 23:27 |
| *** chlong has joined #openstack-swift | 23:30 | |
| mattoliverau | he may hav a clayg greese monkey filter :P | 23:34 |
| *** daemontool has quit IRC | 23:40 | |
| clayg | mattoliverau: oh you published that!? | 23:40 |
| mattoliverau | Lol | 23:41 |
| *** robefran has joined #openstack-swift | 23:45 | |
| *** badari has quit IRC | 23:47 | |
| *** diogogmt has quit IRC | 23:54 | |
| *** ho has joined #openstack-swift | 23:58 | |
| *** ho is now known as Guest14447 | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!