| *** nolwenn has quit IRC | 00:09 | |
| openstackgerrit | Timur Alperovich proposed openstack/python-swiftclient master: Turn stdin uploads into SLO above 10MB. https://review.openstack.org/478611 | 00:12 |
|---|---|---|
| *** vinsh has joined #openstack-swift | 00:19 | |
| *** psachin has joined #openstack-swift | 00:28 | |
| *** tovin07_ has joined #openstack-swift | 00:31 | |
| *** kiennt has joined #openstack-swift | 00:40 | |
| timburke | timur: you fixed the things i was *less* worried about :-/ | 00:45 |
| timburke | :P | 00:45 |
| kota_ | good morning | 01:20 |
| kota_ | oh, yeah. today is the meeting | 01:21 |
| kota_ | notmyname: ack | 01:21 |
| notmyname | kota_: hello :-) | 01:21 |
| notmyname | kota_: have you thought about the meeting agenda at all yet? | 01:21 |
| kota_ | not yet, starting to look up the wiki page since now | 01:22 |
| notmyname | ok | 01:22 |
| kota_ | do you have something? | 01:22 |
| *** lucasxu has joined #openstack-swift | 01:22 | |
| kota_ | notmyname: ^^ | 01:23 |
| notmyname | we have to have a tag for swiftclient by the end of my wednesday this week. and I'm hoping for a swift tag at the end of this week | 01:23 |
| notmyname | otherwise, bug triage work (from the etherpad) | 01:23 |
| notmyname | and a reminder of the PTG topics | 01:23 |
| *** lucasxu has quit IRC | 01:24 | |
| kota_ | ok, I'll announce that. | 01:24 |
| notmyname | but if there are other topics or patches you'd like to bring up, or that others want to be discussed, that's good to | 01:24 |
| kota_ | kk | 01:24 |
| kota_ | ok, the release scheduled was added to the meeting wiki | 01:29 |
| * kota_ is recapping the meeting commands looking at the previous meeting log | 01:30 | |
| notmyname | thanks. please ask if you have any questions. I've got an evening meeting tonight, so I'll be around for a while | 01:31 |
| kota_ | notmyname: thanks! | 01:33 |
| openstackgerrit | Thiago da Silva proposed openstack/swift master: Remove left over param https://review.openstack.org/487255 | 01:55 |
| tdasilva | notmyname: thanks for merging patch 344693 | 01:55 |
| patchbot | https://review.openstack.org/#/c/344693/ - swift - Fix swiftdir option and usage of storage policy al... (MERGED) | 01:55 |
| tdasilva | timburke: interesting, i did notice the need for the multiple ^D too | 01:57 |
| *** csmart has joined #openstack-swift | 01:59 | |
| csmart | \o | 02:02 |
| mattoliverau | csmart: o/ | 02:04 |
| csmart | I think I've finally recovered enough to re-connect with #openstack land | 02:05 |
| mattoliverau | Lol, welcome back :) | 02:05 |
| csmart | mattoliverau: thanks | 02:05 |
| tdasilva | mattoliverau: congrats on the new job!!! | 02:06 |
| mattoliverau | tdasilva: thanks man! It means your stuck with me a while longer ;) now just need to convince SUSE to let me spend more and more time on Swift ;) | 02:06 |
| tdasilva | hehehe, that's cool, what else are you going to be working on? | 02:08 |
| *** tonanhngo has quit IRC | 02:10 | |
| *** itlinux_ has quit IRC | 02:14 | |
| *** noxdafox has quit IRC | 02:19 | |
| *** noxdafox has joined #openstack-swift | 02:20 | |
| *** MooingLemur has quit IRC | 02:20 | |
| *** tonanhngo has joined #openstack-swift | 02:23 | |
| *** MooingLemur has joined #openstack-swift | 02:25 | |
| mattoliverau | Not 100% sure yet. Apparently we break into virtual teams and work on defined milestones that last a few months, so would get some choice and probably play with a bunch of things... At least that's how it's been described. Will tell you next week once I start :) | 02:25 |
| *** tonanhngo has quit IRC | 02:28 | |
| tdasilva | mattoliverau: cool, planning to be at the ptg? | 02:33 |
| openstackgerrit | Merged openstack/swift master: ready yet? nope, please wait! https://review.openstack.org/486745 | 02:36 |
| mattoliverau | tdasilva: probably not, I assume I'd be to new to have budget to travel :( but will ask ;) | 02:37 |
| notmyname | hello mattoliverau and csmart | 02:44 |
| *** JimCheung has quit IRC | 02:47 | |
| *** JimCheung has joined #openstack-swift | 02:48 | |
| *** JimCheung has quit IRC | 02:52 | |
| mattoliverau | notmyname: o/ | 03:00 |
| mahatic | good morning | 03:29 |
| mahatic | mattoliverau: congrats! that's great news! | 03:30 |
| mattoliverau | mahatic: thanks :) | 03:30 |
| *** links has joined #openstack-swift | 03:30 | |
| *** links has quit IRC | 03:30 | |
| *** gkadam has joined #openstack-swift | 03:32 | |
| *** kei_yama has quit IRC | 03:34 | |
| *** kei_yama has joined #openstack-swift | 03:40 | |
| csmart | notmyname: \o hey, hope you're well | 03:50 |
| notmyname | csmart: aside from late-evening meetings? yeah :-) | 03:56 |
| csmart | hehe good | 03:56 |
| *** two_tired has joined #openstack-swift | 04:01 | |
| *** tonanhngo has joined #openstack-swift | 04:02 | |
| *** tonanhngo has quit IRC | 04:07 | |
| *** kei_yama has quit IRC | 04:24 | |
| *** deep-book-gk_ has joined #openstack-swift | 04:26 | |
| *** deep-book-gk_ has left #openstack-swift | 04:30 | |
| * kota_ is on patch 477000 again | 04:41 | |
| patchbot | https://review.openstack.org/#/c/477000/ - swift - Ring rebalance respects co-builders' last_part_moves | 04:41 |
| *** tonanhngo has joined #openstack-swift | 04:45 | |
| *** tonanhngo has quit IRC | 04:49 | |
| *** Venkata has joined #openstack-swift | 05:10 | |
| *** tonanhngo has joined #openstack-swift | 05:15 | |
| *** tonanhngo has quit IRC | 05:20 | |
| *** JimCheung has joined #openstack-swift | 05:21 | |
| *** JimCheung has quit IRC | 05:25 | |
| *** skudlik has joined #openstack-swift | 05:30 | |
| *** kei_yama has joined #openstack-swift | 05:32 | |
| notmyname | good night | 05:37 |
| notmyname | kota_: I'll read the meeting transcript in the morning. thank you for chairing | 05:37 |
| kota_ | notmyname: have a good night o/ | 05:38 |
| *** kiennt has quit IRC | 05:42 | |
| *** two_tired has quit IRC | 05:48 | |
| *** cshastri has joined #openstack-swift | 05:53 | |
| *** kiennt has joined #openstack-swift | 06:00 | |
| *** rcernin has joined #openstack-swift | 06:06 | |
| *** tonanhngo has joined #openstack-swift | 06:09 | |
| *** tonanhngo has quit IRC | 06:13 | |
| *** thurloat has quit IRC | 06:27 | |
| *** geaaru has joined #openstack-swift | 06:27 | |
| *** xrb has joined #openstack-swift | 06:30 | |
| *** thurloat has joined #openstack-swift | 06:35 | |
| *** bkopilov_ has quit IRC | 06:39 | |
| *** tonanhngo has joined #openstack-swift | 06:42 | |
| *** ChubYann has quit IRC | 06:42 | |
| *** tonanhngo has quit IRC | 06:46 | |
| *** cschwede_ has joined #openstack-swift | 06:55 | |
| acoles | good morning | 06:57 |
| kota_ | acoles: morning | 06:58 |
| kota_ | reminder, swift 0700 meeting will be #openstack-meeting in 2 minutes | 06:58 |
| acoles | mattoliverau: excellent news! congratulations | 06:59 |
| *** ukaynar has quit IRC | 07:06 | |
| *** ukaynar has joined #openstack-swift | 07:07 | |
| *** hseipp has joined #openstack-swift | 07:07 | |
| *** ukaynar has quit IRC | 07:11 | |
| *** tonanhngo has joined #openstack-swift | 07:14 | |
| *** tonanhngo_ has joined #openstack-swift | 07:17 | |
| *** tonanhngo has quit IRC | 07:19 | |
| *** klrmn has quit IRC | 07:20 | |
| *** tonanhngo_ has quit IRC | 07:22 | |
| *** PavelK has quit IRC | 07:41 | |
| *** jaosorior has quit IRC | 07:43 | |
| * kota_ is trying to find new cup of coffee | 07:50 | |
| *** PavelK has joined #openstack-swift | 07:53 | |
| kota_ | oh, I forgot to introduce the session voting is now open for Sydney summit in the meeting | 07:56 |
| kota_ | https://www.openstack.org/summit/sydney-2017/vote-for-speakers#/ | 07:57 |
| *** jaosorior has joined #openstack-swift | 07:57 | |
| *** jaosorior has quit IRC | 07:58 | |
| *** jaosorior has joined #openstack-swift | 07:59 | |
| *** jaosorior has quit IRC | 08:02 | |
| *** jaosorior has joined #openstack-swift | 08:03 | |
| *** jaosorior has quit IRC | 08:03 | |
| *** hseipp has quit IRC | 08:04 | |
| *** jaosorior has joined #openstack-swift | 08:04 | |
| *** tonanhngo has joined #openstack-swift | 08:11 | |
| *** jaosorior has quit IRC | 08:13 | |
| *** ukaynar has joined #openstack-swift | 08:15 | |
| *** tonanhngo has quit IRC | 08:16 | |
| *** bkopilov_ has joined #openstack-swift | 08:19 | |
| *** cshastri has quit IRC | 08:26 | |
| *** cshastri has joined #openstack-swift | 08:27 | |
| *** ukaynar has quit IRC | 08:37 | |
| *** ukaynar has joined #openstack-swift | 08:38 | |
| *** ukaynar has quit IRC | 08:42 | |
| *** kei_yama has quit IRC | 08:45 | |
| openstackgerrit | Merged openstack/swift master: Remove left over param https://review.openstack.org/487255 | 08:49 |
| *** PavelK has quit IRC | 08:51 | |
| *** jaosorior has joined #openstack-swift | 08:53 | |
| *** oshritf has joined #openstack-swift | 08:58 | |
| *** jaosorior has quit IRC | 09:03 | |
| *** PavelK has joined #openstack-swift | 09:04 | |
| *** tonanhngo has joined #openstack-swift | 09:05 | |
| *** tonanhngo has quit IRC | 09:10 | |
| *** dr_gogeta86_ has quit IRC | 09:15 | |
| *** dja has quit IRC | 09:16 | |
| *** timburke has quit IRC | 09:20 | |
| *** timburke has joined #openstack-swift | 09:20 | |
| *** dr_gogeta86 has joined #openstack-swift | 09:20 | |
| *** jaosorior has joined #openstack-swift | 09:21 | |
| *** jarbod__ has quit IRC | 09:21 | |
| *** oshritf has quit IRC | 09:21 | |
| *** ChanServ sets mode: +v timburke | 09:21 | |
| *** dr_gogeta86 has quit IRC | 09:21 | |
| *** dr_gogeta86 has joined #openstack-swift | 09:21 | |
| *** dja_ has joined #openstack-swift | 09:21 | |
| *** jarbod__ has joined #openstack-swift | 09:22 | |
| *** hseipp has joined #openstack-swift | 09:24 | |
| *** dancn has joined #openstack-swift | 09:25 | |
| *** mvk has quit IRC | 09:31 | |
| *** PavelK has quit IRC | 09:50 | |
| *** tovin07_ has quit IRC | 10:01 | |
| *** mvk has joined #openstack-swift | 10:04 | |
| *** kiennt has quit IRC | 10:05 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift master: Small fixup for CooperativeRingBuilder https://review.openstack.org/487354 | 10:14 |
| kota_ | acoles: done for https://review.openstack.org/#/c/477000/ | 10:16 |
| patchbot | patch 477000 - swift - Ring rebalance respects co-builders' last_part_moves | 10:16 |
| kota_ | and | 10:16 |
| * kota_ is going to a pub :P | 10:17 | |
| acoles | kota_: thank you! | 10:17 |
| acoles | kota_: if i could I would buy your beer :) | 10:17 |
| kota_ | acoles: np, and nice work! | 10:17 |
| acoles | kota_: the review made it much better | 10:18 |
| *** tonanhngo has joined #openstack-swift | 10:28 | |
| *** jaosorior has quit IRC | 10:38 | |
| *** clarkb has quit IRC | 10:45 | |
| *** clarkb has joined #openstack-swift | 10:45 | |
| *** tonanhngo has quit IRC | 11:13 | |
| openstackgerrit | Merged openstack/swift master: Ring rebalance respects co-builders' last_part_moves https://review.openstack.org/477000 | 11:45 |
| *** psachin has quit IRC | 11:51 | |
| *** tonanhngo has joined #openstack-swift | 11:54 | |
| openstackgerrit | Alistair Coles proposed openstack/swift master: Clarify usage of replicator and reconstructor override options https://review.openstack.org/487383 | 11:57 |
| *** tonanhngo has quit IRC | 11:58 | |
| *** tonanhngo has joined #openstack-swift | 11:58 | |
| openstackgerrit | Alistair Coles proposed openstack/swift master: Add multiple worker processes strategy to reconstructor https://review.openstack.org/478416 | 11:59 |
| *** tonanhngo has quit IRC | 12:02 | |
| *** bkopilov_ has quit IRC | 12:02 | |
| *** Dinesh_Bhor is now known as Dinesh_Bhor|away | 12:08 | |
| *** Dinesh_Bhor|away is now known as Dinesh_Bhor | 12:09 | |
| *** Venkata has quit IRC | 12:12 | |
| tdasilva | good morning | 12:15 |
| *** chlong has joined #openstack-swift | 12:16 | |
| acoles | tdasilva: o/ | 12:20 |
| openstackgerrit | Alistair Coles proposed openstack/swift master: Small fixup for CooperativeRingBuilder https://review.openstack.org/487354 | 12:21 |
| *** catintheroof has joined #openstack-swift | 12:21 | |
| mahatic | tdasilva: o/ | 12:28 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 12:35 |
| *** JimCheung has joined #openstack-swift | 12:36 | |
| *** JimCheung has quit IRC | 12:40 | |
| *** gkadam has quit IRC | 12:41 | |
| *** xrb has quit IRC | 12:52 | |
| *** xrb has joined #openstack-swift | 12:53 | |
| *** xrb has quit IRC | 12:53 | |
| *** xrb has joined #openstack-swift | 12:53 | |
| *** noxdafox has quit IRC | 12:56 | |
| *** noxdafox has joined #openstack-swift | 13:02 | |
| *** saint_ has joined #openstack-swift | 13:17 | |
| *** caiobrentano has joined #openstack-swift | 13:22 | |
| *** jaosorior has joined #openstack-swift | 13:24 | |
| openstackgerrit | Merged openstack/swift master: Small fixup for CooperativeRingBuilder https://review.openstack.org/487354 | 13:25 |
| *** jistr is now known as jistr|mtg | 13:31 | |
| *** dja_ is now known as dja | 13:40 | |
| *** xrb has quit IRC | 13:43 | |
| *** catintheroof has quit IRC | 13:44 | |
| *** catintheroof has joined #openstack-swift | 13:46 | |
| *** jistr|mtg is now known as jistr | 13:47 | |
| *** xrb has joined #openstack-swift | 13:47 | |
| *** catintheroof has quit IRC | 13:48 | |
| *** catintheroof has joined #openstack-swift | 13:48 | |
| *** catintheroof has quit IRC | 13:51 | |
| *** catintheroof has joined #openstack-swift | 13:52 | |
| *** rcernin has quit IRC | 13:54 | |
| *** SkyRocknRoll has joined #openstack-swift | 13:55 | |
| *** SkyRocknRoll has quit IRC | 13:59 | |
| *** SkyRocknRoll has joined #openstack-swift | 14:00 | |
| *** ukaynar has joined #openstack-swift | 14:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 14:08 |
| clayg | kota_: timburke: acoles: great work on composite ring cobuilding! We should +A https://review.openstack.org/#/c/486263/ asap too! | 14:10 |
| patchbot | patch 486263 - swift - Add ever_rebalanced property to RingBuilder | 14:10 |
| *** saint_ has quit IRC | 14:11 | |
| *** vinsh has quit IRC | 14:14 | |
| *** vinsh has joined #openstack-swift | 14:16 | |
| *** bkopilov_ has joined #openstack-swift | 14:31 | |
| openstackgerrit | Alistair Coles proposed openstack/swift master: WIP fix is_healthy() to detect ring changes https://review.openstack.org/487448 | 14:36 |
| *** JimCheung has joined #openstack-swift | 14:48 | |
| *** cdelatte has joined #openstack-swift | 14:52 | |
| *** JimCheung has quit IRC | 14:53 | |
| *** SkyRocknRoll has quit IRC | 15:03 | |
| *** mhks has joined #openstack-swift | 15:14 | |
| *** SkyRocknRoll has joined #openstack-swift | 15:15 | |
| *** klrmn has joined #openstack-swift | 15:17 | |
| *** rcernin has joined #openstack-swift | 15:22 | |
| *** itlinux has joined #openstack-swift | 15:22 | |
| *** jaosorior has quit IRC | 15:28 | |
| *** rcernin has quit IRC | 15:34 | |
| notmyname | good morning | 15:43 |
| notmyname | kota_: thanks for running the meeting | 15:47 |
| *** mvk has quit IRC | 15:49 | |
| notmyname | acoles: any suggestions for the changelog entry for https://review.openstack.org/#/c/477000/ | 15:50 |
| patchbot | patch 477000 - swift - Ring rebalance respects co-builders' last_part_moves (MERGED) | 15:50 |
| clayg | acoles: I'm not surprised there's some jank on the ring reloading. But fwiw it did seem to work in practice? | 15:50 |
| clayg | Maybe parent process gets lucky because it's not in a reconstruct loop doing periodic ring checks? | 15:51 |
| clayg | Or because I only had one ring? | 15:51 |
| acoles | clayg: not for me, but my workers were doing little work | 15:51 |
| clayg | And they'd just miss the update entirely? Yuk. | 15:52 |
| acoles | clayg: sometimes it seems yes, but worse even when is_healthy spots the change, the ring may not be reloaded before new worker args are generated, so it just restarts same set of workers :/ | 15:54 |
| acoles | clayg: of course, I may be going crazy ;) | 15:54 |
| clayg | That does sound crazy. | 15:55 |
| clayg | Actually the whole configurable ring check interval is crazy - it doesn't actually get plumbed into the ring reload time. | 15:55 |
| *** hseipp has quit IRC | 15:56 | |
| acoles | clayg: so Ring.has_changed() returns True - that's good - but the ring is not reloaded by the get_worker_args() AFAICT | 15:56 |
| acoles | clayg: yes, the existing ring check stuff seems a bit wonky | 15:57 |
| clayg | Wonky!!! | 15:57 |
| clayg | Still. I don't love seeing so much of that ring internals lifted into just this one daemon. | 15:58 |
| clayg | Maybe get_worker_args should just always reload to play it safe? | 15:59 |
| *** gkadam has joined #openstack-swift | 16:00 | |
| clayg | Then the check_ring nonsense is just a pre-existing bug-ish? Maybe? I might not understand the failure. | 16:00 |
| acoles | no I don't like it either :/ | 16:00 |
| clayg | I guess I wasn't changing *devices* - just part assignments. Which the worker will refresh each pass eventually. | 16:01 |
| acoles | yup, i was adding/removing devs | 16:01 |
| clayg | So... maybe not existing bug. And maybe only effects workers. Lame! | 16:01 |
| *** tristanC has quit IRC | 16:03 | |
| *** rcernin has joined #openstack-swift | 16:03 | |
| clayg | acoles: yeah. I'm realizing my earlier statement of "works in practice" referred to ring reloading for living workers - I could say "I thought I saw device changing worker reload work once in dev probably" | 16:04 |
| clayg | But if it's racy enough it might only work sometimes. I wasn't paying that close of attention. Good check! | 16:05 |
| clayg | Catch. | 16:05 |
| acoles | I think it is *possible* that a race might mean that the ring reloads itself before the get_worker_args does its thing - it depends on when the ring's next_check is due vs. when the reconstructor next_check fired | 16:07 |
| *** chlong has quit IRC | 16:08 | |
| acoles | clayg: welcome back by the way ! :) | 16:08 |
| *** itlinux has quit IRC | 16:11 | |
| *** Sukhdev_ has joined #openstack-swift | 16:15 | |
| *** klrmn has quit IRC | 16:18 | |
| *** mvk has joined #openstack-swift | 16:23 | |
| *** Sukhdev_ has quit IRC | 16:24 | |
| *** Sukhdev_ has joined #openstack-swift | 16:28 | |
| *** JimCheung has joined #openstack-swift | 16:30 | |
| *** xrb has quit IRC | 16:31 | |
| *** xrb has joined #openstack-swift | 16:31 | |
| timburke | good morning | 16:34 |
| *** xrb has quit IRC | 16:36 | |
| *** gkadam has quit IRC | 16:36 | |
| timburke | hmmm... i should probably hop in -infra at some point and advocate for patch 379805... | 16:37 |
| patchbot | https://review.openstack.org/#/c/379805/ - openstack-infra/meetbot - Make #links links | 16:37 |
| openstackgerrit | Alistair Coles proposed openstack/swift master: WIP fix is_healthy() to detect ring changes https://review.openstack.org/487448 | 16:39 |
| *** ChubYann has joined #openstack-swift | 16:40 | |
| *** tristanC has joined #openstack-swift | 16:48 | |
| acoles | notmyname: I'll leave a suggestion on the changelog patch | 16:49 |
| *** SkyRocknRoll has quit IRC | 16:52 | |
| *** cshastri has quit IRC | 17:02 | |
| *** noxdafox has quit IRC | 17:02 | |
| *** Sukhdev_ has quit IRC | 17:03 | |
| *** itlinux_ has joined #openstack-swift | 17:04 | |
| *** SkyRocknRoll has joined #openstack-swift | 17:05 | |
| *** noxdafox has joined #openstack-swift | 17:08 | |
| *** itlinux_ has quit IRC | 17:09 | |
| *** ukaynar has quit IRC | 17:11 | |
| *** itlinux has joined #openstack-swift | 17:11 | |
| *** ukaynar has joined #openstack-swift | 17:11 | |
| *** ukaynar has quit IRC | 17:16 | |
| *** klrmn has joined #openstack-swift | 17:21 | |
| acoles | I'm looking at patch 475038 | 17:23 |
| patchbot | https://review.openstack.org/#/c/475038/ - python-swiftclient - Allow for uploads from standard input. | 17:23 |
| *** chlong has joined #openstack-swift | 17:25 | |
| *** chlong has quit IRC | 17:27 | |
| *** chlong has joined #openstack-swift | 17:28 | |
| timburke | acoles: good spot on the ring-reloading... seems like a pre-existing issue, but probably still worth fixing? | 17:28 |
| acoles | timburke: yes. before the ring would be reloading itself after a change so the next reconstructor cycle would pick up dev changes, but now we use the devs to set up workers so adding a dev might never be noticed | 17:31 |
| *** SkyRocknRoll has quit IRC | 17:44 | |
| timburke | acoles: true. but like, https://github.com/openstack/swift/blob/2.14.0/swift/obj/reconstructor.py#L1017-L1020 has *always* been worthless, yeah? | 17:56 |
| acoles | timburke: unreliable at best - has_changed is not a reliable way for an external observer to determine if the ring has changed since some point in history - AFAICT it only tells you if the ring has changed since it last detected a change and reloaded itself. | 18:14 |
| *** mhks has quit IRC | 18:18 | |
| *** JimCheung has quit IRC | 18:18 | |
| *** JimCheung has joined #openstack-swift | 18:19 | |
| *** geaaru has quit IRC | 18:22 | |
| timburke | so it seems like we need to either (1) make Ring._mtime a public thing that external observers don't need to feel weird using or (2) store some separate "time of last reload" so external observers can track `start_time = time.time()` and later ask `if Ring.has_changed_since(start_time)` | 18:22 |
| timur | acoles: if you're testing out the patches, consider looking at the follow-on SLO patch as well | 18:28 |
| *** Sukhdev_ has joined #openstack-swift | 18:28 | |
| clayg | zohno! we need to fixup patch 390781 | 18:31 |
| patchbot | https://review.openstack.org/#/c/390781/ - swift - Replace replication_one_per_device by custom count | 18:31 |
| clayg | cc rledisez | 18:31 |
| acoles | timburke: ack. I won't get chance today I'm afraid | 18:33 |
| timburke | acoles: no worries... just kinda thinking out loud. does either of those seem preferable to you? | 18:34 |
| timburke | i can play with an alternative to patch 487448 today i think | 18:35 |
| patchbot | https://review.openstack.org/#/c/487448/ - swift - WIP fix is_healthy() to detect ring changes | 18:35 |
| acoles | timburke: I thought about making has_changed take an optional time to test against but it tests equality so you wouldn't be able to pass in any time, would need to also be able to fetch mtime at start | 18:37 |
| acoles | I do prefer a method rather than leaking out mtime | 18:37 |
| acoles | timburke: also considered ring incrementing a counter on each reload but then we have to assume the ring instance is never re-instantiated | 18:38 |
| acoles | timburke: just to be clear: as I see it there are two issues: 1. has_changed() is not reliable test for reconstructor to use and 2. we need to ensure the ring data is reloaded when get_worker_args is called (so far I used _reload) | 18:39 |
| timburke | k. i'll play with that. track a _last_reload_time, then expose something like `has_changed_since(ref_time): has_changed() or _last_reload_time > ref_time` | 18:40 |
| timburke | and yeah, still need to think about how best to handle part 2 | 18:40 |
| acoles | timur: I see a stack trace when I use stdin upload with --skip-identical | 18:41 |
| timur | acoles: cool -- I'll investigate now | 18:42 |
| acoles | timur: but I think it is an existing behaviour - i.e. would be same with any stream source | 18:42 |
| timur | acoles: oh... | 18:42 |
| timur | that's less exciting :) | 18:43 |
| acoles | timur: path is set to None and then is_identical tries to open path ! | 18:43 |
| acoles | timur: lol | 18:43 |
| acoles | timur: https://github.com/openstack/python-swiftclient/blob/638d7c789cf3ccab61bf6af6fcab6e6d79b9e0a4/swiftclient/service.py#L1845-L1849 | 18:43 |
| timur | acoles: right. What do you think? We should definitely fix it, but should it be fixed in this change? Streaming standard input uploads definitely make it easier to hit it | 18:47 |
| timur | I'll try to write up the fix and see what, if any, tests I have to fix up for it and possibly update the patchset? | 18:47 |
| *** thurloat has quit IRC | 18:47 | |
| acoles | timur: could be trivial to fix? do you have time? | 18:51 |
| timur | acoles: I do -- going to get lunch and try to push it in | 18:52 |
| timur | so probably within the next 1-2 hours | 18:52 |
| *** cschwede_ has quit IRC | 18:52 | |
| acoles | timur: compare line 1879 to 1889 - just needs similar if path is not None for the skip_identical case | 18:52 |
| timur | I'm also curious about the ctrl-D behavior. Would like to understand better what's happening there | 18:52 |
| acoles | https://github.com/openstack/python-swiftclient/blob/638d7c789cf3ccab61bf6af6fcab6e6d79b9e0a4/swiftclient/service.py#L1879-L1889 | 18:53 |
| acoles | timur: not sure the ctrl-d thing is a blocker though | 18:53 |
| acoles | timur: I'll be around later (2-3pm PST for upstream meeting) so can revisit - otherwise it looks great | 18:54 |
| acoles | so great that I +2'd, oops | 18:54 |
| acoles | timur: this seems to fix it but TBH I didn't stare at code for too long https://gist.github.com/alistairncoles/62b0e7f39fa96af2e7c0e7723591fb16 | 18:57 |
| acoles | timburke: just a thought - given the timing, perhaps fixing the existing check_ring behaviour could wait i.e. 'inline ring checking' and we just get is_healthy working? | 18:59 |
| *** rcernin has quit IRC | 19:27 | |
| *** thurloat has joined #openstack-swift | 19:43 | |
| *** chlong has quit IRC | 19:43 | |
| *** chlong has joined #openstack-swift | 19:49 | |
| timburke | timur: i'm guessing it's something to do with cpython trying to read until we get an empty string -- like, the first ^D cuts short the 8k read it wants to do, but then it goes back *again* | 19:51 |
| timburke | see https://github.com/python/cpython/blob/2.7/Lib/httplib.py#L850-L856 | 19:51 |
| *** chlong has quit IRC | 19:52 | |
| *** thurloat has quit IRC | 19:52 | |
| *** chlong has joined #openstack-swift | 19:52 | |
| *** Sukhdev_ has quit IRC | 20:05 | |
| *** ukaynar has joined #openstack-swift | 20:07 | |
| *** thurloat has joined #openstack-swift | 20:14 | |
| *** vint_bra has quit IRC | 20:16 | |
| *** vint_bra has joined #openstack-swift | 20:17 | |
| timur | acoles: great -- I will try to incorporate your gist/validate it -- doing that now | 20:24 |
| *** caiobrentano has quit IRC | 20:26 | |
| *** caiobrentano_ has joined #openstack-swift | 20:30 | |
| notmyname | mattoliverau: is suse going to send you to the ptg? or is it too late for that? | 20:32 |
| mattoliverau | notmyname: I'm assuming it's too late... But I have asked my manager.. just in case :) | 20:38 |
| notmyname | :-) | 20:38 |
| notmyname | good luck! | 20:38 |
| *** caiobrentano_ has quit IRC | 20:39 | |
| *** caiobrentano has joined #openstack-swift | 20:40 | |
| *** caiobrentano has quit IRC | 20:40 | |
| *** caiobrentano has joined #openstack-swift | 20:41 | |
| *** caiobrentano has quit IRC | 20:45 | |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient master: moved cli doc to the right place for new links process https://review.openstack.org/487582 | 20:48 |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient master: moved cli doc to the right place for new links process https://review.openstack.org/487582 | 20:50 |
| kota_ | good morning | 20:56 |
| *** ukaynar_ has joined #openstack-swift | 20:58 | |
| notmyname | meeting time | 21:00 |
| *** tonanhngo has joined #openstack-swift | 21:01 | |
| *** catintheroof has quit IRC | 21:01 | |
| *** ukaynar has quit IRC | 21:02 | |
| *** joeljwright has joined #openstack-swift | 21:02 | |
| *** ChanServ sets mode: +v joeljwright | 21:02 | |
| *** ukaynar_ has quit IRC | 21:03 | |
| *** ukaynar has joined #openstack-swift | 21:03 | |
| *** tonanhngo has quit IRC | 21:05 | |
| openstackgerrit | Timur Alperovich proposed openstack/python-swiftclient master: Turn stdin uploads into SLO above 10MB. https://review.openstack.org/478611 | 21:07 |
| openstackgerrit | Timur Alperovich proposed openstack/python-swiftclient master: Allow for uploads from standard input. https://review.openstack.org/475038 | 21:07 |
| *** ukaynar has quit IRC | 21:08 | |
| timburke | notmyname: any way we could get https://docs.openstack.org/developer/python-swiftclient/cli.html redirecting there, too?currently, that just goes to https://docs.openstack.org/python-swiftclient/latest/ | 21:09 |
| notmyname | timburke: dunno | 21:09 |
| timburke | (it's actually not so bad for client, since it's a pretty shallow tree. I *really* want to fix all the overview links for server) | 21:10 |
| *** chlong has quit IRC | 21:14 | |
| *** caiobrentano has joined #openstack-swift | 21:26 | |
| *** ukaynar has joined #openstack-swift | 21:33 | |
| clayg | rledisez: y u no #openstack-meeting !?? | 21:34 |
| notmyname | clayg: he's on vacation, I think | 21:37 |
| clayg | sapchcpahcpah - who takes vacation!? 😁 | 21:38 |
| notmyname | the French! | 21:38 |
| clayg | N.B. clayg will be out next week for vacation | 21:38 |
| joeljwright | I'm always too slow to go add a second +2 | 21:39 |
| acoles | joeljwright: nothing to stop you adding the third :) | 21:40 |
| joeljwright | :) | 21:40 |
| kota_ | I'm planning to get vacation *after* ptg :P | 21:40 |
| *** ukaynar has quit IRC | 21:42 | |
| *** ukaynar has joined #openstack-swift | 21:43 | |
| *** ukaynar has quit IRC | 21:53 | |
| notmyname | kota_: good idea :-) | 21:57 |
| acoles | notmyname: more jury duty ? do you not get a few years before being eligible again? | 21:59 |
| notmyname | once a year | 21:59 |
| acoles | wow | 21:59 |
| notmyname | I don't think it was last year I had it | 21:59 |
| notmyname | but if feels like it was recent | 21:59 |
| notmyname | just got the letter in the mail yesterday | 21:59 |
| acoles | :( | 22:01 |
| acoles | good night | 22:02 |
| *** joeljwright has quit IRC | 22:03 | |
| openstackgerrit | Merged openstack/swift master: Add ever_rebalanced property to RingBuilder https://review.openstack.org/486263 | 22:07 |
| *** caiobrentano has quit IRC | 22:38 | |
| timburke | acoles: how reliably were the workers not getting cleaned up? i'm wondering if we just need to make sure the orchestrator guy needs to do the ring checks *before* writing aggregates (since that's where it loops over devices), or maybe have it save off the set of expected devices for aggregates (cc clayg) | 22:42 |
| timburke | i *think* that's the only place it really touches rings? | 22:43 |
| *** tonanhngo has joined #openstack-swift | 22:49 | |
| *** tonanhngo has quit IRC | 22:53 | |
| clayg | acoles: you maybe had your stats_interval cranked down super low? | 22:55 |
| *** vint_bra has quit IRC | 22:56 | |
| *** skudlik has quit IRC | 22:57 | |
| clayg | yes, that is definitely the only place | 23:00 |
| clayg | looking at all_local_devices and get_policy2devices it's not super obvious why I ended up pulling those methods apart like that | 23:00 |
| clayg | it's really policy2local_devices and all_local_devices is just a stupid wrapper to turn a map of policy => list_of_local_devs into a set | 23:01 |
| *** ^andrea^ has quit IRC | 23:03 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 23:05 |
| clayg | but roughly I like the idea that aggregate_recon_update uses a cached self._all_devices and every ring_check_interval we ask we calc old_devices, self._all_devices = self._all_devices, <inline self.all_local_devices>; return old_devices == self._all_devices ??? | 23:07 |
| *** JimCheung has quit IRC | 23:14 | |
| *** JimCheung has joined #openstack-swift | 23:15 | |
| *** ukaynar has joined #openstack-swift | 23:30 | |
| *** kei_yama has joined #openstack-swift | 23:41 | |
| timburke | clayg: acoles: something like http://paste.openstack.org/show/616638/ seems to make things happy... by changing the stats_interval to a float, i could drop it down to like .05 and super-reliably have issues with ring changes (curiously it was still a bit hit-or-miss at 1? don't need to keep that part, anyway) | 23:47 |
| *** Sukhdev has joined #openstack-swift | 23:54 | |
| clayg | timburke: I think that's the idea | 23:55 |
| openstackgerrit | Tim Burke proposed openstack/swift master: Add multiple worker processes strategy to reconstructor https://review.openstack.org/478416 | 23:55 |
| clayg | cool! | 23:56 |
| openstackgerrit | Merged openstack/python-swiftclient master: moved cli doc to the right place for new links process https://review.openstack.org/487582 | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!