| *** wanghao has joined #openstack-zaqar | 00:11 | |
| openstackgerrit | Nguyen Van Trung proposed openstack/zaqar master: Use generic user for both zuul v2 and v3 https://review.openstack.org/515639 | 01:11 |
|---|---|---|
| flwang | yangzhenyu: wxy_: ping re the delayed queue spec | 01:15 |
| wxy_ | flwang: yeah | 01:15 |
| flwang | wxy_: I think there is a copy-paste issue in https://review.openstack.org/#/c/512919/11/specs/queens/delayed-queues.rst ? | 01:16 |
| flwang | it's copying words from my original spec, but it's still using 'delay' as the named of the new attribute | 01:17 |
| flwang | but I can't see it's consistent with the code in patch https://review.openstack.org/508669 | 01:17 |
| wxy_ | Yeah. The patch has more work to do. | 01:18 |
| wxy_ | But I think the spec is OK, except you have other idea? | 01:18 |
| wanghao | flwang: ping | 01:19 |
| flwang | wanghao: pong | 01:20 |
| wanghao | flwang: wxy_: just want your guys' idea, did you think it's worth to add V3 API for remove group? | 01:20 |
| wanghao | flwang: http://paste.openstack.org/show/624608/ | 01:20 |
| flwang | wanghao: i mentioned in my previous comments | 01:20 |
| flwang | v3 is overkill | 01:20 |
| flwang | so NO | 01:20 |
| wxy_ | ++ | 01:20 |
| wanghao | flwang: yeah, me too | 01:20 |
| wanghao | wxy_: yes | 01:20 |
| wxy_ | V2 is still in progress IMO. :) | 01:21 |
| flwang | if we have to add a new version to remove group, then i even would keep it until we have a better idea | 01:21 |
| wanghao | sure, so my idea is we support two behaviors in V2 at Pike, and remove the old one in S | 01:22 |
| flwang | wxy_: as for the delayed queue patch, I mean the latest patch is saying a new attribute 'delay', but in code commit, it's using '_default_message_delay_ttl' | 01:23 |
| flwang | which should be updated | 01:23 |
| wxy_ | wanghao: that is a correct way. | 01:23 |
| *** gecong has joined #openstack-zaqar | 01:23 | |
| flwang | 'delay' is the old name I mentioned in my original spec | 01:23 |
| wxy_ | flwang: No. "delay" is for message. "_default_message_delay_ttl" is for queue. | 01:24 |
| wxy_ | message object has a new property called "delay". queue object has a new reserved property called "_default_message_delay_ttl" | 01:24 |
| wanghao | wxy_, flwang: em sure, will update the spec and see others' idea. | 01:25 |
| flwang | wxy_: see line 35 https://review.openstack.org/#/c/512919/11/specs/queens/delayed-queues.rst | 01:26 |
| wxy_ | flwang: Oh. It's a mistake. | 01:27 |
| wxy_ | flwang: I think yangzhenyu forget to change it. It should be "message" | 01:28 |
| flwang | https://review.openstack.org/#/c/508669/27/zaqar/storage/mongodb/claims.py | 01:29 |
| flwang | and I can't see any related change in claims.py | 01:29 |
| *** wanghao_ has joined #openstack-zaqar | 01:29 | |
| flwang | I don't like current change in claims.py which is introducing a call to queue to get metadata | 01:30 |
| flwang | skip my last comments :) | 01:31 |
| flwang | it's not introducing a new call to queue but change the location | 01:31 |
| wxy_ | flwang: lol | 01:32 |
| flwang | wxy_: yangzhenyu is still using "_default_message_delay_ttl" to determine if it's a delayed queue | 01:32 |
| *** wanghao has quit IRC | 01:33 | |
| wxy_ | flwang: he think it's an option property for the queue. If it's not set, it's a normal queue. | 01:34 |
| *** wanghao_ has quit IRC | 01:36 | |
| *** wanghao has joined #openstack-zaqar | 01:36 | |
| flwang | wxy_: ok, i will leave my comments on the spec | 01:38 |
| wxy_ | flwang: Yes, please. | 01:39 |
| flwang | and I don't think current code can meet the scenarios #2 | 01:40 |
| wxy_ | flwang: Yes. The patch has much work to do. | 01:41 |
| flwang | and see this http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-delay-queues.html | 01:41 |
| flwang | To set delay seconds on individual messages, rather than for an entire queue, use message timers. If you send a message with a message timer, Amazon SQS uses the message timer's delay seconds value instead of the delay queue's delay seconds value. For more information, see Amazon SQS Message Timers. | 01:42 |
| flwang | the scenario #2 could be a little bit hard based on current design what I can see from the code | 01:42 |
| wxy_ | Can this work: message_delay_ttl = message_create_request_body.get("message_delay_ttl", queue_meta.get("default_delay_ttl")) | 01:48 |
| yangzhenyu | flwang, wxy_ I will update the spec for "delay". | 01:52 |
| *** wanghao_ has joined #openstack-zaqar | 01:52 | |
| *** wanghao has quit IRC | 01:55 | |
| yangzhenyu | flwang, messages = wsgi_utils.sanitize | 01:57 |
| yangzhenyu | flwang, this can work | 01:57 |
| flwang | it could work, but | 01:57 |
| flwang | if user send a msg with delayed ttl to a normal queue | 01:58 |
| flwang | how can you filter it? | 01:58 |
| flwang | let me reword | 01:58 |
| flwang | could we filter it without any performance impatc? | 01:58 |
| *** yangzhenyu_ has joined #openstack-zaqar | 02:00 | |
| yangzhenyu_ | flwang, queue_delay_ttl = queue_meta.get('_default_message_delay_ttl', | 02:02 |
| yangzhenyu_ | None) | 02:02 |
| yangzhenyu_ | flwang, when user sent a 'delay_ttl' message for a normal queue, this var is a None | 02:03 |
| yangzhenyu_ | if queue_delay_ttl is not None: | 02:04 |
| yangzhenyu_ | message_post_spec += (('delay_ttl', int, queue_delay_ttl),) | 02:04 |
| flwang | yep, that's my question | 02:04 |
| flwang | when user do claim, if it's a normal queue but with delayed messages, it's a little bit hard to get it, right? | 02:05 |
| yangzhenyu_ | flwang, If the queue is a normal queue, the message's delay_expires is 0 | 02:06 |
| flwang | i think we're talking about the #2 scenario in the spec | 02:07 |
| flwang | see http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-delay-queues.html | 02:08 |
| flwang | To set delay seconds on individual messages, rather than for an entire queue, use message timers. If you send a message with a message timer, Amazon SQS uses the message timer's delay seconds value instead of the delay queue's delay seconds value. For more information, see Amazon SQS Message Timers. | 02:08 |
| flwang | so in SQS, set delayed seconds on individual message will need another feature named message timer | 02:09 |
| flwang | so we may need to skip this scenario in current spec | 02:09 |
| *** gengchc2 has joined #openstack-zaqar | 02:18 | |
| gengchc2 | ping flwang | 02:19 |
| gengchc2 | I am working on v2.0 for supporting group and no group, if there is no problem, we do not need to add v3.0 interface, we can support two programs coexist. Later we can completely removed group. I code completely,and test now. | 02:24 |
| gengchc2 | ping wanghao_ | 02:25 |
| flwang | gengchc2: so could you please update the spec so that we can be on the same page about the design? | 02:33 |
| yangzhenyu_ | flwang, | 02:35 |
| yangzhenyu_ | Oh, yes, this implementation is not exactly the same as SQS, but I do not think it does affect the implementation of case 2. When the delay message is sent to the delay queue, the message's delay_expires attribute is calculated based on the message's delay_ttl instead of using the queue's meta attribute _default_message_delay_ttl, this should be the priority to use the message delay attribute. | 02:35 |
| gengchc2 | ok | 02:36 |
| flwang | yangzhenyu: i understand. but that means you can't do things like https://review.openstack.org/#/c/508669/27/zaqar/storage/mongodb/claims.py | 02:37 |
| flwang | can you see my point? | 02:37 |
| gecong | feilong, could you please review https://review.openstack.org/#/c/513594/ ,we can finish the polices in code now, thanks | 02:37 |
| flwang | I mean you can't easily know if the queue is a delayed queue or not | 02:37 |
| flwang | gecong: sure, sorry for the late review ;) | 02:38 |
| flwang | gengchc2: i will be off to Sydney at this Friday | 02:38 |
| flwang | so won't be online much since this Friday until 13 Nov | 02:38 |
| flwang | if anyone in Sydney during the summit, drop me a message in Wechat | 02:39 |
| yangzhenyu | flwang, Oh, are you worried about this? My idea is that if there is no _default_message_delay_ttl attribute in the queue, it is not a delay queue. This attribute is only used for delay queues. It is not a reserved attribute. Must be set by the user, if set to mean that it is a delay queue. | 02:45 |
| yangzhenyu | The logic of my code is the same, if you do not set this metadata, it is not a delay queue | 02:46 |
| *** wanghao_ has quit IRC | 02:47 | |
| flwang | yangzhenyu: i see, then my question is, if you're sending a delayed message to a normal (non-delayed) queue, how can you get the message list with delayed messages? | 02:47 |
| flwang | just list all of them? | 02:47 |
| *** wanghao has joined #openstack-zaqar | 02:48 | |
| yangzhenyu | yes, normal queue has no delay message, I think. | 02:48 |
| yangzhenyu | flwang, the case 3 of the spec. | 02:50 |
| flwang | i'm talking about the scenario #2 | 02:51 |
| yangzhenyu | flwang, Because to ensure that does not affect the performance of the normal queue, so there is a choice. But I think this is also considered normal, the delay queue has a switch. | 02:51 |
| flwang | yangzhenyu: i totally understand, but I'm trying to ask if there is any way in your mind to get the function without performance issue | 02:52 |
| *** wanghao_ has joined #openstack-zaqar | 02:58 | |
| *** wanghao has quit IRC | 02:59 | |
| yangzhenyu | wxy_, I am still more inclined to use _default_message_delay_ttl, but I will add a default value for this property. what do you think? | 03:17 |
| wxy_ | yangzhenyu: Actually, it depends on "what's the reserved metadata mean?" | 03:21 |
| wxy_ | yangzhenyu: Now we have "_max_messages_post_size" and "_default_message_ttl", they both are config options. So If you add "_default_message_delay_ttl", should it be a config option as well? | 03:23 |
| yangzhenyu | wxy_, no, it is not be config | 03:24 |
| wxy_ | yangzhenyu: Yes, So IMO, Whether a queue is a delay queue should not depend on the config option. it's a queue's property. | 03:24 |
| wxy_ | yangzhenyu: So this is my concern about the "_XXX" usage. | 03:24 |
| yangzhenyu | wxy_, flwang I think we need to make sure what property is reserved, we understand are not the same. | 03:26 |
| yangzhenyu | wxy_, flwang xiyuan that the retention attribute is configurable, feilong that the reserved attribute is code can be called. | 03:28 |
| wxy_ | Can we just add a new property for "queue"? Our usage now are mainly focus on "metadata" that I don't think it is good. | 03:28 |
| wxy_ | More reserved metadata, harder for maintenance. | 03:29 |
| yangzhenyu | wxy_, flwang yes, I am more inclined to not a reserved attribute, just a common attribute. | 03:30 |
| wxy_ | My opinion: Be careful for reserved metadata usage. Anyway, if you guys decide to use it here, I don't want to -2. :) | 03:30 |
| yangzhenyu | wxy_, But can not normal attributes be put into metadata? | 03:32 |
| wxy_ | yangzhenyu: It depends on "what the metadata mean?" https://en.wikipedia.org/wiki/Metadata: descriptive metadata, structural metadata, and administrative metadata | 03:40 |
| wxy_ | yangzhenyu: So now, we combined descriptive and administrative usage. I think it's a little obscure. | 03:41 |
| wxy_ | yangzhenyu: For other projects, Like Nova, Cinder, they use metadata just for description. But in Zaqar, we give it more usage. Some for description, some for reservation. Now we'll complicate it more: some are from config option, some are not. Well, do you think it's suitable? | 03:46 |
| *** rcernin has quit IRC | 03:47 | |
| *** rcernin_ has joined #openstack-zaqar | 03:47 | |
| yangzhenyu | wxy_, Yes, according to the current mechanism, with the addition of features, metadata will be more and more. But we need to have a specification, where the new attribute is added. | 03:51 |
| yangzhenyu | wxy_, | 03:53 |
| yangzhenyu | Now often use the metadata attribute, it may not start from its definition, but on the metadata processing logic are ready, add up more convenient. | 03:53 |
| *** flwang1 has quit IRC | 03:56 | |
| wxy_ | yangzhenyu: have to go for launch now. Leaved my comment in the spec. Please review it. Thanks. | 03:57 |
| chandankumar | flwang: Hello | 05:48 |
| chandankumar | flwang: please have a look on this https://review.openstack.org/#/c/504899/ | 05:48 |
| chandankumar | flwang: in rdo side it is fixed | 05:48 |
| *** yangzhenyu_ has quit IRC | 05:52 | |
| *** yangzhenyu_ has joined #openstack-zaqar | 05:53 | |
| *** Kevin_Zheng has quit IRC | 06:01 | |
| *** jtomasek has joined #openstack-zaqar | 06:26 | |
| *** idmvxiwqmc has joined #openstack-zaqar | 06:35 | |
| *** gecong has quit IRC | 06:55 | |
| *** spectr has joined #openstack-zaqar | 07:02 | |
| *** rcernin_ has quit IRC | 07:17 | |
| *** tesseract has joined #openstack-zaqar | 07:33 | |
| *** rcernin has joined #openstack-zaqar | 08:22 | |
| chandankumar | flwang: as this patch https://review.rdoproject.org/r/#/c/10320/ will get merged in RDO only if https://review.openstack.org/504899 | 09:02 |
| chandankumar | gets merged | 09:02 |
| *** wanghao_ has quit IRC | 09:34 | |
| *** wanghao has joined #openstack-zaqar | 09:34 | |
| *** wanghao has quit IRC | 09:35 | |
| *** wanghao has joined #openstack-zaqar | 09:35 | |
| *** wanghao has quit IRC | 09:36 | |
| *** wanghao has joined #openstack-zaqar | 09:36 | |
| *** gengchc2 has quit IRC | 09:50 | |
| -openstackstatus- NOTICE: Zuul has been restarted due to an unexpected issue. Please recheck any jobs that were in progress | 10:47 | |
| *** pcaruana has joined #openstack-zaqar | 10:57 | |
| *** jtomasek has quit IRC | 11:21 | |
| *** jtomasek has joined #openstack-zaqar | 11:51 | |
| *** spectr has quit IRC | 11:59 | |
| *** spectr has joined #openstack-zaqar | 12:00 | |
| *** rcernin has quit IRC | 12:58 | |
| *** aputtur_ has joined #openstack-zaqar | 13:10 | |
| *** jtomasek_ has joined #openstack-zaqar | 13:31 | |
| *** aputtur_ has quit IRC | 13:31 | |
| *** aputtur_ has joined #openstack-zaqar | 13:31 | |
| *** jtomasek has quit IRC | 13:32 | |
| *** aputtur_ has quit IRC | 13:32 | |
| *** aputtur_ has joined #openstack-zaqar | 13:47 | |
| *** aputtur_ has quit IRC | 13:47 | |
| *** aputtur_ has joined #openstack-zaqar | 14:02 | |
| *** Eva-i has joined #openstack-zaqar | 15:07 | |
| *** spectr has quit IRC | 16:02 | |
| *** tesseract has quit IRC | 16:55 | |
| *** flwang1 has joined #openstack-zaqar | 17:06 | |
| *** harlowja has joined #openstack-zaqar | 17:19 | |
| *** Eva-i has quit IRC | 18:02 | |
| *** Eva-i has joined #openstack-zaqar | 18:44 | |
| *** Eva-i has quit IRC | 18:56 | |
| *** flwang1 has quit IRC | 19:05 | |
| *** Eva-i has joined #openstack-zaqar | 19:08 | |
| *** Eva-i has quit IRC | 19:17 | |
| *** Eva-i has joined #openstack-zaqar | 19:33 | |
| *** Eva-i has quit IRC | 20:26 | |
| *** flwang1 has joined #openstack-zaqar | 20:27 | |
| *** Eva-i has joined #openstack-zaqar | 20:33 | |
| flwang | chandankumar: cool, thanks for your great job | 20:38 |
| *** aputtur_ has quit IRC | 20:48 | |
| *** jtomasek_ has quit IRC | 20:58 | |
| *** pcaruana has quit IRC | 21:15 | |
| *** rcernin has joined #openstack-zaqar | 21:52 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-zaqarclient master: Updated from global requirements https://review.openstack.org/500036 | 22:16 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-zaqarclient master: Updated from global requirements https://review.openstack.org/500036 | 22:22 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-zaqarclient master: Updated from global requirements https://review.openstack.org/500036 | 22:22 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!