Tuesday, 2015-12-08

*** betherly has quit IRC00:32
*** jjardon has quit IRC00:32
*** jjardon has joined #storyboard00:36
*** betherly has joined #storyboard01:20
*** coolsvap has quit IRC01:23
*** coolsvap has joined #storyboard01:25
*** jtomasek has joined #storyboard07:21
pedroalvarezmorning!09:13
pedroalvarezI had a debugging session yesterday and I found useful things for the task '"task_assignee_changed" event is not being used when changing assignee ' of the story https://storyboard.openstack.org/#!/story/200043509:14
pedroalvarezI added this debugging information: https://github.com/palvarez89/storyboard/commit/1b3e531f463822b1a2a8e302adac35ec03209a6809:14
pedroalvarezand got the result: http://paste.openstack.org/show/481123/09:14
pedroalvarezI don't know yet how to solve it, that's why I'm sharing this here.09:16
ZaraI've discovered a bug in pagination. I can already hear the sound of SotK quietly weeping.10:09
Zarasetting limit doesn't work when you're logged out, so you can't see more than 10 entries per page10:09
Zaraunless you set limit, then browse to another page, then back to the first page10:10
Zaraso it updates it, but not on the page you're looking at10:10
Zarayou can test this by going between projects and stories.10:10
Zara(that log is a mystery to me atm, but thanks for sharing it)10:12
* SotK quietly weeps10:25
Zara=D10:26
SotKpedroalvarez: I'm sorry for my nonsensical reviews yesterday :)10:43
pedroalvarezhah, no worries, they made sense to me until I tried to fix my patches10:44
SotKI thought "oh what about that pop-up", went and looked at it, decided the boxes weren't there, looked at the code, still didn't notice them already being there10:45
* SotK was very sleepy yesterday10:45
pedroalvarezI learnt the word thanks to that10:48
pedroalvarezthe word "cog"*10:48
ZaraSotK: happy to +1 https://review.openstack.org/#/c/251566/ ? I haven't merged it yet since it was updated after your last +1, but I don't think you oppose it.10:50
SotKdone :)10:51
Zarahehe10:54
openstackgerritMerged openstack-infra/storyboard: Add story_title to tag events.  https://review.openstack.org/25156610:59
* pedroalvarez dances11:01
Zara:P11:01
* SotK wonders if Zara will +1 https://review.openstack.org/#/c/251605/ ? :P11:02
pedroalvarezfirst database migration script in my life :)11:02
*** openstackgerrit has quit IRC11:02
*** openstackgerrit has joined #storyboard11:02
Zaraonce I've tested it :P11:05
ZaraI want to get event notificatons set up on my instance at some point; might be worth doing that first11:06
Zaranow there are some docs11:06
* SotK expects that setting up event notifications will be useful for testing a patch to fix the event notifications :)11:15
Zarayup11:16
ZaraI just got distracted by a rogue storyboard instance, but I'm back again11:16
pedroalvarezhehe, note, after getting them working you might find more bugs :)11:18
Zarathat's my super-power! :D11:19
SotKugh, that pagination bug was a regression caused by the fix for default preferences11:21
Zaragah11:21
openstackgerritAdam Coldrick proposed openstack-infra/storyboard-webclient: Fix setting preferences when logged out  https://review.openstack.org/25470211:33
Zararecent events seem to be working for me now \o/11:37
SotK\o/11:37
pedroalvarezquick quick, tag a story and un-tag it :)11:37
Zarayup, I can now see the blank event. now to look at the patches...11:40
openstackgerritMerged openstack-infra/storyboard: Document manually setting up notifications  https://review.openstack.org/25359111:42
Zarawhile looking at tags patches, have found that subscriptions sometimes seem to be removed when recentevents list changes. trying to work out what's going on. I don't think it's related to the tags patches.12:05
Zararemoving a tag can remove a subscription12:07
Zarawhyyyyyyyyyyyyyyyyy12:07
pedroalvarezthat is something I haven't hit..12:08
pedroalvarezI wonder if this is something you can reproduce elsewhere, (openstack's or sotk's instances)12:09
Zaraokay, removing a tag stops a subscription from displaying (that is, it shows an empty star). but tag removal still comes up in the list of recent events, suggesting that either the user is still subscribed, or recentevents isn't paying attention to a user's subscriptions12:10
Zarathis happens on master branch, so am confident it's not to do with the tags template patch, anyway12:12
Zaraidk why it would be but am now certain :P12:12
pedroalvarezfailed to reproduce it so far12:13
Zarayeah, doesn't seem to be an issue on stories list on storyboard.openstack.org. going to see if it's just a problem with the dashboard, otherwise it might just be my instance (though still not sure why)12:15
Zarahm, happens on my instance on the story list, so suggests it's different to storyboard.openstack.org12:16
Zaramy instance is running master...12:17
Zaraso either some strangeness in my db, or possibly an issue from a patch merged recently that hasn't made it to production yet.12:19
SotKo.O12:31
SotKyou always find the weirdest things12:31
Zarahm. on my instance, adding a tag PUTs the tag, then GETs events and the relevant user. deleting the tag deletes the tag and GETs the events. On storyboard.openstack.org, it's the same, but it doesn't GET the user.12:31
Zaraor maybe it does, but in a different order12:32
SotKmy test instance doesn't GET the user12:33
Zarahttp://paste.openstack.org/show/481153/12:34
ZaraI have user-related errors when I search for things, since I don't think I added fulltext stuff properly for users12:34
Zaraso maybe it's related?12:34
SotKoh hm, it did lose the subscription star though12:34
Zarahm, permanently? sometimes it's just slow to load12:35
Zarabut if you refresh and it's gone, then :/12:35
SotKyeah, permanently12:36
Zaraoh, look, it's that time of the day where I make tea and hide.12:36
SotKbut I'm still subscribed to it12:37
SotK(at least, I'm still getting notified of changes/new comments)12:38
SotKor am I?12:38
pedroalvarezI'm still failing to reproduce it :/12:38
SotKaha, I got the notification because I'm subscribed to the project too12:39
SotKpedroalvarez: I just added a tag to a story I was subscribed to, deleted the tag, then refreshed and the star next to the title was empty12:39
Zaraah, maybe that's why I was getting notifications12:42
ZaraI definitely don't get the issue on storyboard.openstack.org. if it starts, we'll know it's from a recently-merged patch.12:44
Zaraif not, yet another mystery?12:44
pedroalvarezstill can't reproduce :S12:45
pedroalvarezwe could try to reproduce the error without that patch merged12:45
* SotK has reproduced it without pedroalvarez's patch12:45
pedroalvarezphew12:46
Zara\o/ though is that also rolling back the db? I don't know why it would be an issue but I'm already confused.12:46
SotKyep12:48
Zaragah. can we see what patch production is currently running from?12:49
Zaramaybe we have to go back further?12:49
Zaraotherwise maybe we've both done something strange with our databases12:49
* SotK tries again and this time it does GET the user12:52
SotKand doing it again doesn't12:53
* SotK shrugs12:53
Zara\o/12:53
ZaraI can't say that it definitely does it every time for me12:53
Zarait was an immediate difference I spotted, might not be relevant at all12:54
Zaralooks like it's GETting the same things on my instance whether or not there's a tag. maybe we should investigate what goes on when a tag is deleted...13:01
* SotK is doing so13:01
Zara:P13:01
SotKI think I've discovered the cause of the issue13:15
* SotK double checks13:16
Zara:013:17
SotKindeed I have13:17
Zaraoh my!13:18
SotKthe worker is deleting the subscription because it sees that the http method is DELETE13:20
Zaraomg13:20
SotKI'm not sure why it isn't happening for pedroalvarez or in production, maybe we are doing something weird when running it manually13:21
pedroalvarezmaybe I need to restart the workers...?13:21
pedroalvareznot sure..13:21
Zaraand here I was going to joke that it was the docs patch that caused it.13:21
pedroalvareznope, not happening here13:22
*** tlbr has quit IRC13:22
* SotK investigates some more13:23
Zaradeleting a task doesn't seem to affect the subscription13:24
Zaraoh, actually, hang on, maybe it would for the related project13:25
Zarano, still seems fine13:26
SotKnope, that would do the right thing looking at it (delete the subscription to the task, if one exists)13:26
Zarahm,  my worker is reporting errors13:27
SotKcan you paste them somewhere please?13:27
Zarahttp://paste.openstack.org/show/481160/13:27
SotKhmm, I've never seen that13:28
SotKwhat did you do that caused it?13:28
ZaraI'm not positive, might have been from adding and deleting a task in a subscribed resource, going by the timestamp13:28
Zarawill try to narrow it down13:29
* SotK just did that without issue... weird13:29
Zarayeah, seemed to work fine13:31
Zaraand now I can't reproduce the error13:31
Zaramanaged to get it again; think it's something to do with navigating to a page before another page has loaded properly (maybe)13:34
ZaraI didn't get it when refreshing each page bfore doing anything13:34
SotKSo the deletion problem is that the notifications hook intercepts the DELETE request and creates an event with method = 'DELETE', which is fine, except that the path in the request for deleting tags is "/v1/stories/1/tags".13:36
SotKThat gets parsed such that the affected resource is a "story" with id 1, and the affected sub_resource is a "tag" with id None...13:37
SotKand the check for handling deletions (which is done so you don't end up with subscriptions to non-existent things) is `if method == 'DELETE' and not sub_resource_id:`13:38
Zarahaha13:38
SotKtl;dr: the worker is taking `DELETE /v1/stories/1/tags` to mean the story was deleted, and unsubscribing from it13:38
ZaraI wondered if deleting worked like that but didn't get why it would only be the case there, since I was missing the notification-hook-intercepting piece of the puzzle13:39
SotKme too, since the creation of the "tag deleted" event is handled by a different thing13:40
SotKwhich creates an event with method='POST'13:40
Zara(btw, I think the weird error I got might involve a race-condition, though I'm not sure. I only seem to be able to reproduce it by accident)13:41
pedroalvarezI wonder if workers are not being upgraded on my instance13:42
SotKI guess I should send a patch to fix it13:55
Zaraprobably wise.13:55
ZaraI got thoroughly distracted from testing the tag templates patch13:55
Zarawell, tags have now had a thorough testing.13:56
SotK\o/13:57
SotKhah, I've seen that error now13:59
Zara:D14:01
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Don't delete subscription if sub_resource is set  https://review.openstack.org/25477014:06
* pedroalvarez destroys his insance in another attempt to reproduce the error14:08
Zarapatch doesn't seem to be fixing the problem for me. Should I restart the worker?14:10
SotKyes :)14:10
* Zara will try that anyway14:10
Zarahahah14:10
Zaraah, all seems well14:10
Zarathanks14:10
openstackgerritMerged openstack-infra/storyboard-webclient: Add templates for tags added/deleted subscription events  https://review.openstack.org/25160514:11
Zara\o/14:13
*** tlbr has joined #storyboard14:14
SotKZara: I've never had to refresh the story page when changing the preferences, unless you mean they won't auto-update if you change them on the preferences screen in a different window14:31
Zarayeah, sorry, that's what I meant14:32
ZaraI had it open in a different tab14:32
Zarajust mentioned it because I spent a brief moment wondering why it wasn't working.14:33
openstackgerritMerged openstack-infra/storyboard-webclient: Add checkboxes for tag events on preferences page  https://review.openstack.org/25416214:45
pedroalvareztriple dance today14:49
openstackgerritZara proposed openstack-infra/storyboard-webclient: Align comment timestamps on the right on dashboard  https://review.openstack.org/25480214:52
ZaraI have patches from a couple of weeks back, though not many :P https://review.openstack.org/#/q/owner:zara.zaimeche%2540codethink.co.uk+status:open,n,z14:54
pedroalvarezdamn, fresh deployment, and I still can't reproduce the removing-subscriptions error14:58
SotK:/15:02
Zaraare the rabbitmq settings different? gah15:02
SotKpedroalvarez: can you add some debugging code to the handle function in storyboard/plugin/subscription/base.py, to print out the method, path, and resource of each request?15:03
pedroalvarezSotK: http://paste.openstack.org/show/481178/15:10
SotKhmm, I wonder if its because of the "Attempted to send payload with no destination resource." thing15:12
SotKdoes the regex on line 126 of storyboard/notifications/notification_hook.py match the path properly?15:14
pedroalvarezyou nailed it15:16
pedroalvarezso, I found a solution some days ago for the "Attempted to send payload with no destination resource."15:16
pedroalvarezbut given that I don't know if that is really a solution, I didn't send a patch for that15:17
pedroalvarezif I apply the solution, the bug happens15:17
SotKI wonder if that issue is happening in production too15:18
pedroalvarezhere the log: http://paste.openstack.org/show/481179/15:18
SotKsince the api is at https://storyboard.openstack.org/api/v1/ there too15:19
SotKthat log is the same as I see on my test instance15:19
pedroalvarezand here the fix: http://paste.openstack.org/show/481181/15:20
pedroalvarezbut I guess that the fix will break your development environments15:20
SotKindeed it will15:20
pedroalvarezI also wonder, what is that for :)15:21
pedroalvarezgiven that everything works nice without it15:21
pedroalvarezheh15:21
pedroalvarezand if it's needed maybe we can:15:22
pedroalvarezurl_pattern = re.match("^(\/api)?\/v1\/([a-z_]+)\/?([0-9]+)?"15:22
pedroalvareznot sure if that is an expression here, tbh15:22
SotKpedroalvarez: what is what for?15:25
pedroalvarezI.. I don't know how to even call it. I mean, the code that is broken in production environments, and working in development environment15:27
pedroalvarezthis notification_hook15:27
SotKit gets called each time there is a request to the API, and if its a PUT, POST, or DELETE request it publishes a message to rabbitmq describing the request, which the event worker can then handle however it wants to (eg. creating a subscription_event, or sending an email)15:30
SotKthe notifications work in production despite that not working because I think all of the timeline event changes also publish a message with information about the event, and that is what actually gets used to create subscription_events AIUI15:31
* SotK isn't sure how much sense that made15:32
pedroalvarezmakes some sense :) thanks for the explanation15:32
*** tlbr has quit IRC15:41
pedroalvarezI can say now that your patch works15:43
Zara\o/15:48
* pedroalvarez fails to fix the regex15:56
pedroalvarez:)15:56
SotKwhat went wrong?15:56
pedroalvarezmy ability of doing regexes15:57
pedroalvarezmy suggestion didnt work15:58
pedroalvarezurl_pattern = re.match("^(\/api)?\/v1\/([a-z_]+)\/?([0-9]+)?"15:58
pedroalvarezalso, I'm not sure if it makes sense15:58
openstackgerritMerged openstack-infra/storyboard: Don't delete subscription if sub_resource is set  https://review.openstack.org/25477015:58
*** tlbr has joined #storyboard16:05
*** tlbr has quit IRC16:10
*** tlbr has joined #storyboard16:13
SotKpedroalvarez: I managed to get it working with that regex, but I had to increase all the indices used later in the function by 116:21
pedroalvarezlet me know if I can help16:23
openstackgerritAdam Coldrick proposed openstack-infra/storyboard: Parse path correctly when it begins with /api  https://review.openstack.org/25485916:46
*** openstackgerrit has quit IRC16:47
*** openstackgerrit has joined #storyboard16:47
pedroalvarezoh yes, now I understand16:49
*** yarkot has joined #storyboard17:26
*** yarkot has quit IRC17:58
*** openstackgerrit has quit IRC19:02
*** openstackgerrit has joined #storyboard19:03
*** yarkot has joined #storyboard20:18
*** yarkot has quit IRC20:49
*** jtomasek has quit IRC21:03
*** yarkot has joined #storyboard21:24
*** yarkot has quit IRC22:39

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!