Wednesday, 2015-01-07

« Tuesday, 2015-01-06 Index Thursday, 2015-01-08 »
*** david-lyle is now known as david-lyle_afk00:01
*** ajo has quit IRC00:04
boris-42dims__: around?00:06
boris-42dhellmann: any chance to get new release of oslo.config with that patch? =)00:19
dims__boris-42: pong00:24
boris-42dims__: happy new year =) it's not too late lol00:24
boris-42dims__: so we (Rally team) has broken gates because of some changes in oslo.config00:25
boris-42dims__: this patch fixes https://review.openstack.org/#/c/145214/ them*00:25
boris-42dims__: could you please take a look at it thank you00:25
*** andreykurilin has quit IRC00:25
dims__boris-42: will do in just a little bit00:26
*** zzzeek has quit IRC00:28
boris-42dims__: thank you00:28
*** _amrith_ is now known as amrith01:30
*** harlowja_at_home has quit IRC01:43
*** tsekiyam_ has joined #openstack-oslo01:49
*** tsekiya__ has joined #openstack-oslo01:52
*** tsekiyama has quit IRC01:52
*** amrith is now known as _amrith_01:52
*** tsekiyam_ has quit IRC01:53
*** mtanino has quit IRC01:54
*** tsekiya__ has quit IRC01:56
*** harlowja_at_home has joined #openstack-oslo02:04
openstackgerritMerged openstack/oslo.config: Fix of wrong cli opts unregistration  https://review.openstack.org/14521402:49
*** tsekiyama has joined #openstack-oslo03:02
*** tsekiyama has quit IRC03:06
openstackgerritSridhar Gaddam proposed openstack/oslo.utils: Utility API to generate EUI-64 IPv6 address  https://review.openstack.org/13777403:13
*** harlowja_at_home has quit IRC03:33
*** Nakato has quit IRC04:16
*** Nakato has joined #openstack-oslo04:16
*** Nakato has quit IRC04:18
*** Nakato has joined #openstack-oslo04:19
*** harlowja_at_home has joined #openstack-oslo04:36
openstackgerritJoshua Harlow proposed openstack/taskflow: Use monotonic time when/if available  https://review.openstack.org/14421204:50
openstackgerritJoshua Harlow proposed openstack/taskflow: Use monotonic time when/if available  https://review.openstack.org/14421204:50
*** arnaud__ has joined #openstack-oslo04:51
openstackgerritMerged openstack/taskflow: Rework the in-memory backend  https://review.openstack.org/12455204:52
*** harlowja_at_home has quit IRC04:58
*** dims__ has quit IRC05:50
*** dims__ has joined #openstack-oslo05:50
*** amotoki has joined #openstack-oslo05:53
*** dims__ has quit IRC05:54
*** liusheng has joined #openstack-oslo06:00
openstackgerritOpenStack Proposal Bot proposed openstack/oslo.messaging: Imported Translations from Transifex  https://review.openstack.org/14325606:11
*** rpodolyaka2 has joined #openstack-oslo06:27
*** mtanino has joined #openstack-oslo06:34
*** noelbk has joined #openstack-oslo06:41
*** liusheng has quit IRC06:46
*** e0ne has joined #openstack-oslo06:50
*** mtanino has quit IRC06:57
*** stevemar has quit IRC07:01
*** noelbk has quit IRC07:10
*** ajo has joined #openstack-oslo07:14
*** zz_sabari is now known as sabari07:16
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Ensure kombu channels are closed  https://review.openstack.org/14523207:23
*** e0ne has quit IRC07:23
*** ajo has quit IRC07:26
*** ajo has joined #openstack-oslo07:28
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Ensure kombu channels are closed  https://review.openstack.org/14523207:30
*** arnaud__ has quit IRC07:43
*** arnaud__ has joined #openstack-oslo07:44
*** arnaud__ has quit IRC07:48
*** sabari is now known as zz_sabari07:51
*** subscope has joined #openstack-oslo07:57
*** pcaruana|afk| is now known as pcaruana08:04
*** zz_sabari is now known as sabari08:19
*** dulek has joined #openstack-oslo08:24
*** dtantsur|afk is now known as dtantsur08:42
*** pcaruana has quit IRC08:59
*** sabari is now known as zz_sabari09:00
*** pcaruana has joined #openstack-oslo09:12
*** GheRivero has quit IRC09:23
*** GheRivero has joined #openstack-oslo09:23
*** rpodolyaka2 has quit IRC09:31
*** rpodolyaka2 has joined #openstack-oslo09:31
*** boris-42 has quit IRC09:53
*** andreykurilin has joined #openstack-oslo10:09
*** ihrachyshka has joined #openstack-oslo10:10
*** freyes has quit IRC10:22
*** freyes has joined #openstack-oslo10:24
*** yassine has joined #openstack-oslo10:48
*** andreykurilin has quit IRC10:52
*** andreykurilin has joined #openstack-oslo10:52
*** rpodolyaka2 has quit IRC10:53
*** boris-42 has joined #openstack-oslo11:06
*** andreykurilin has quit IRC11:10
*** amotoki has quit IRC11:12
*** GheRivero has quit IRC11:15
openstackgerritJulien Danjou proposed openstack/tooz: Add support for an optional redis-sentinel  https://review.openstack.org/14499911:17
*** GheRivero has joined #openstack-oslo11:21
*** dims__ has joined #openstack-oslo11:30
*** GheRivero has quit IRC11:33
*** GheRivero has joined #openstack-oslo11:34
*** GheRivero has quit IRC11:44
*** GheRivero has joined #openstack-oslo11:50
*** alexpilotti has joined #openstack-oslo11:57
*** david-lyle_afk is now known as david-lyle11:58
*** e0ne has joined #openstack-oslo11:59
*** jaosorior has joined #openstack-oslo12:10
*** vigneshvar has joined #openstack-oslo12:15
*** salv-orlando has quit IRC12:29
*** _amrith_ is now known as amrith12:30
openstackgerritDavanum Srinivas (dims) proposed openstack/oslo.config: Support i18n messages in config generator  https://review.openstack.org/14528012:31
*** miqui has quit IRC12:36
*** pblaho_ has joined #openstack-oslo12:38
*** salv-orlando has joined #openstack-oslo12:43
*** pblaho_ has quit IRC12:43
*** rpodolyaka2 has joined #openstack-oslo12:49
*** rushiagr_away is now known as rushiagr12:51
*** GheRivero has quit IRC12:53
*** GheRivero has joined #openstack-oslo12:53
openstackgerritMerged openstack/tooz: Add support for an optional redis-sentinel  https://review.openstack.org/14499912:55
jd__could someone add me in Gerrit so I can release tooz ? (dims__, dhellmann?)13:05
*** david-lyle is now known as david-lyle_afk13:08
*** rushiagr is now known as rushiagr_away13:09
dims__jd__: tooz-release is empty!13:11
jd__dims__: this is so sad13:12
dims__can you check on infra? at least oslo-release should be in it13:13
dims__s/check/please check/13:13
*** e0ne has quit IRC13:13
dims__or we can wait for dhellmann, he may be able to do the magic13:14
*** bknudson has joined #openstack-oslo13:14
*** GheRivero has quit IRC13:20
*** amrith is now known as _amrith_13:20
openstackgerritSean Dague proposed openstack/oslo.concurrency: make time format for processutils match lockutils  https://review.openstack.org/14549713:21
*** andreykurilin has joined #openstack-oslo13:27
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: TEST  https://review.openstack.org/14549813:29
*** pblaho has joined #openstack-oslo13:30
*** rpodolyaka2 has quit IRC13:32
*** jeckersb is now known as jeckersb_gone13:38
*** pblaho has quit IRC13:45
*** pblaho_ has joined #openstack-oslo13:45
*** e0ne has joined #openstack-oslo13:47
*** e0ne has quit IRC13:50
*** kbyrne has quit IRC13:50
*** e0ne has joined #openstack-oslo13:52
*** jeckersb_gone is now known as jeckersb13:54
*** sigmavirus24_awa is now known as sigmavirus2413:55
*** kgiusti has joined #openstack-oslo13:59
*** dulek_ has joined #openstack-oslo14:01
*** dulek_ has quit IRC14:04
*** dulek has quit IRC14:04
*** dulek has joined #openstack-oslo14:06
openstackgerritOpenStack Proposal Bot proposed openstack/tooz: Updated from global requirements  https://review.openstack.org/14478614:14
*** dulek has quit IRC14:16
*** dulek_ has joined #openstack-oslo14:16
*** mtanino has joined #openstack-oslo14:19
*** mriedem has joined #openstack-oslo14:23
*** dulek has joined #openstack-oslo14:27
*** dulek_ has quit IRC14:28
*** rpodolyaka2 has joined #openstack-oslo14:35
*** salv-orlando has quit IRC14:41
*** jecarey has joined #openstack-oslo14:49
*** _amrith_ is now known as amrith14:51
*** dulek has quit IRC14:51
*** noelbk has joined #openstack-oslo14:51
*** noelbk has quit IRC14:51
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: rabbit: fix timeout timer when duration is None  https://review.openstack.org/14552114:52
*** kbyrne has joined #openstack-oslo14:52
*** dulek has joined #openstack-oslo14:52
*** gordc has joined #openstack-oslo14:54
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: rabbit: fix timeout timer when duration is None  https://review.openstack.org/14552114:56
*** dulek has quit IRC14:57
*** dulek has joined #openstack-oslo14:58
*** mtanino has quit IRC14:58
*** dulek has quit IRC14:59
*** dulek has joined #openstack-oslo14:59
*** salv-orlando has joined #openstack-oslo15:07
*** dulek has quit IRC15:16
*** pradk has joined #openstack-oslo15:20
*** salv-orlando has quit IRC15:23
*** rpodolyaka2 has quit IRC15:23
*** rpodolyaka2 has joined #openstack-oslo15:24
*** zz_jgrimm is now known as jgrimm15:36
silehtHi guys, I have a issue with oslo.utils.strutils.mask_password15:42
silehtoslo.messaging largejobs fail gate, I have investigated, and have I found that nova-conductor looks stuck and use 100% cpus15:43
*** subscope has quit IRC15:44
silehtand the reason is that mask_password do a lot of re.XXXX on the rpc message payload15:44
silehtwhen the large jobs run, it start 100 vm, then nova conductor got 100 db update to do, but all oslo.messaging threads stuck in a regex15:46
sileht(I haven't found which one yet)15:46
dhellmanndims__, jd__ : someone from infra will have to fix the tooz release group for us15:50
jd__dhellmann: want me to ask?15:51
dhellmannjd__: please15:51
dhellmannsileht: ew, that's gross15:51
dims__Whoa!15:52
dhellmannsileht: is oslo.messaging calling mask_password because it is logging, or before it sends the message over the rpc connection?15:54
*** mtanino has joined #openstack-oslo15:54
silehtdhellmann, just for logging15:54
dhellmannsileht: can you tell me the line and file where the log call is?15:55
silehthttps://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/common.py#L16415:55
silehtand15:55
silehtfor each message received: https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_drivers/amqpdriver.py#L10015:56
dhellmannso the driver is passing a dictionary to _safe_log(), which converts it to a string, then uses a bunch of regexes to sanitize it?15:57
silehtdhellmann, yes15:57
dhellmannI wonder if it would be faster to deep copy the dict, sanitize it as a data structure, then create the string and log it15:57
dhellmannwe could also just not do that debug logging for now15:58
silehtdhellmann, previously it was the case15:58
dhellmannuntil we can figure out a more efficient way to do it15:58
*** rpodolyaka2 has quit IRC15:58
dhellmannhave we had patterns added to the regex list lately? or have message structures changed in some way that makes the regex more expensive to use?15:58
*** yamahata has joined #openstack-oslo15:59
bknudsonjust wondering if the problem you're discussing is causing http://logs.openstack.org/51/145051/1/gate/gate-tempest-dsvm-neutron-large-ops/2e149e0/console.html#_2015-01-07_13_37_51_452 ?15:59
bknudsonit's "FAIL: tearDownClass (tempest.scenario.test_large_ops.TestLargeOpsScenario)"15:59
silehtbknudson, if the job use master branch of oslo.messaging yes16:00
*** stevemar has joined #openstack-oslo16:00
bknudsonI don't think it does.16:00
silehtdhellmann, I have recently replaced the custom oslo.msg implementation of mask_password by the strutils one16:01
bknudsonI'll keep diggin16:01
dhellmannsileht: ah, so oslo.messaging used to do something different for mask_password? I wonder if it was more efficient16:01
silehtdhellmann, yes it was more efficient but doesn't parse the whole payload...16:02
dhellmannright16:02
*** andreykurilin has quit IRC16:02
dims__dhellmann: jd__: fungi added oslo-release to tooz-release groups and i've added jd__ to tooz-release16:03
dhellmanndims__: thanks16:03
dhellmannjd__: please use the oslo_release.sh script in openstack-infra/release-tools to prepare the release16:04
jd__dhellmann: I was when I got rejected :(16:05
jd__trying again then :)16:06
dhellmannjd__: ok, just making sure :-)16:07
jd__done16:07
jd__dhellmann: does that looks normal http://paste.openstack.org/show/155859/ ? especially the end16:07
jd__oh maybe that's because the LP project is python-tooz not tooz16:08
dhellmannyeah, that argument needs to be the launchpad project name16:08
dhellmannthat bites me every time with stevedore16:08
dhellmannjd__: you can rerun the script, and it will skip tagging the repo because it will find the existing tag16:09
* dhellmann wonders which library to release this morning16:09
openstackgerritRossella Sblendido proposed openstack/oslo.db: Retry query if db deadlock error is received  https://review.openstack.org/10954916:10
jd__dhellmann: cool it worked!16:10
dhellmannjd__: woo! make sure you create a new next-kilo milestone for new bugs16:11
* jd__ looking for the button16:12
openstackgerritgordon chung proposed openstack/oslo.messaging: Implements notification-dispatcher-filter  https://review.openstack.org/7788616:13
silehtthe consume re are:16:14
sileht                      r'([\'"].*?%(key)s[\'"]\s*:\s*u?[\'"]).*?([\'"])',16:14
sileht                      r'([\'"].*?%(key)s[\'"]\s*,\s*\'--?[A-z]+\'\s*,\s*u?[\'"]).*?([\'"])',16:14
jd__dhellmann: there's no LP API to create a milestone automatically?16:16
dhellmannjd__: apparently not16:16
jd__booh16:17
*** tsekiyama has joined #openstack-oslo16:17
dhellmannsileht: try replacing some of those .*? patterns that are searching for strings between quotes with [^\"']*? to see if that helps16:18
dhellmannsileht: we should probably put together a performance test for these things to make them easier to test16:18
silehtdhellmann, I'm considering cutting the payload to limit the time consumption of the logging16:20
* sileht writting the bug report16:20
dhellmannsileht: cutting it how? a max length?16:20
dhellmannor something smarter that logs some metadata about the message but not the entire message?16:20
silehtdhellmann, yes16:20
dhellmannsileht: yes, which? :-)16:22
silehtdhellmann, perhaps logging, the msg_id, ctxt and a part of payload16:22
dhellmannsileht: ok, I like that, especially if we can be smart about the "part of payload" piece to ensure that it never includes passwords16:23
dhellmannalthough I suppose things like tokens appear in the context, so we have to be careful there, too. Maybe we can pick out a few identifiers instead of logging the whole context16:24
silehtdhellmann, the size of the conductor message that cause the issue 66k16:25
dhellmannholy wow16:25
silehtdhellmann, it's a bit huge to log that ...16:25
dhellmannyes, that's not going to be useful to anyone16:25
*** jgrimm is now known as zz_jgrimm16:25
dhellmannis that an entire copy of the database or something?16:25
silehtlol16:25
silehthttp://paste.openstack.org/show/155864/16:27
silehtthe format is odd, I have taken the message directly in rabbit16:28
dhellmannsileht: what do you think about dropping the endpoints values?16:29
dhellmannI'm not sure if that's considered useful or not16:29
silehthum, the paste looks truncated16:30
dhellmannmaybe because it's one long line?16:30
dhellmannI also wonder if it would make sense to pass the context values in through extra so they will be pulled into the message format string, instead of embedding them in the log message directly.16:30
dhellmannwait, is this your new proposed format or is this an example of the 66k messages we're logging now?16:31
silehtit's the (truncated) 66k message16:32
dhellmannok16:34
dhellmannyeah, this includes way more info than I think is really useful16:34
*** arnaud__ has joined #openstack-oslo16:34
dhellmannsileht: another optimization we can make is to only call mask_password if the message is going to be emitted. To do that we'll need to change _safe_log to take a logger and a level, rather than the log function, so we can test the level ourselves16:36
dhellmannthere's no sense in building a safe message if we're not logging debug content16:37
sdaguesileht: just noticing the regex thing16:37
silehtdhellmann, yes but gate use debug level16:37
dhellmannsileht: true, but it would help people in production who don't16:37
sdague.*? is pretty expensive, especially once you put 2 of them in 1 regex (as it creates a ton of backtracking)16:38
sdagueis there a way to trim that bit up to something more specific16:38
dhellmannyeah, I think we can replace some of those with [^"']* but we're also looking at just not logging the whole message16:38
dhellmannsileht: maybe we should start by releasing an oslo.message that doesn't log every message at all, and then we can make it more efficient and turn it back on?16:39
*** dtantsur is now known as dtantsur|afk16:41
*** salv-orlando has joined #openstack-oslo16:43
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Don't log each received messages.  https://review.openstack.org/14554116:54
dhellmannsileht: I'm working on the regexes to see if I can speed them up16:54
silehtdhellmann, ok16:55
dims__dhellmann: are these the ones amrith added?16:55
dhellmanndims__: yeah16:55
dhellmannthe tests are a bit of a mess, so I'm looking at those first16:56
dims__amrith: ping! :)16:56
*** ihrachyshka has quit IRC16:56
dhellmannsileht: a few comments on that patch16:58
*** noelbk has joined #openstack-oslo16:59
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Don't log each received messages.  https://review.openstack.org/14554117:06
silehtdhellmann, thanks, fixed17:06
openstackgerritZhiQiang Fan proposed openstack/oslo-incubator: prints --insecure if verify=False and debug=True  https://review.openstack.org/14417617:10
*** gordc has quit IRC17:15
*** gordc has joined #openstack-oslo17:16
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: rabbit: fix timeout timer when duration is None  https://review.openstack.org/14552117:16
*** noelbk has quit IRC17:21
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Don't log each received messages.  https://review.openstack.org/14554117:23
jamespageanyone know where there are cross project plans to switch from posix-ipc to sysv-ipc as python-tooz has done?17:23
jamespagewhere/whether17:23
dhellmannsileht: that version looks good to me17:24
dhellmannjamespage: I don't know of any such plans17:24
jamespagedhellmann, ack17:25
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Don't log each received messages  https://review.openstack.org/14554117:27
silehtdhellmann, sorry, one more time ^ (stupid ending period...)17:29
silehtbnemec, ^17:29
bnemecsileht: Already +2'd :-)17:30
dhellmannsileht: +2a, let's see what jenkins says17:30
silehtdhellmann, this should fix the largeops job, but I wonder if the timeout issue of the gate-tempest-dsvm-neutron-src-oslo.messaging is due to same bug.17:33
bnemecposix-ipc - the api redesign gone so wrong that it can no longer be used for its intended purpose17:33
*** e0ne has quit IRC17:35
*** dims__ has quit IRC17:43
jamespagejd__, as ceilometer is currently the only user of tooz, do you happen to know which driver it uses? if it's not the ipc driver, I'm tempted to just disable the ipc tests for the Ubuntu package build for now17:43
*** dims__ has joined #openstack-oslo17:44
*** dims__ has quit IRC17:45
*** dims__ has joined #openstack-oslo17:46
*** zz_sabari is now known as sabari17:46
*** david-lyle_afk is now known as david-lyle17:47
dhellmannsileht: I need a copy of that long message string and paste.openstack.org is erroring out, can you put it on another paste server?17:48
dhellmannsileht: oh, nevermind, it's back up and working now17:50
silehtdhellmann, I will generate it17:50
dhellmannemacs is a little unhappy that I pasted that into my source file17:51
*** zzzeek has joined #openstack-oslo17:54
*** cburgess_ is now known as cburgess17:58
silehtdhellmann, the uncutted message is 304k !17:58
dhellmannsileht: yeah, I saved it to a separate file for now17:58
silehtdhellmann, just in case: http://dl.sileht.net/public/payload.json.gz18:01
*** e0ne has joined #openstack-oslo18:01
sileht(the whole message in json format)18:01
*** e0ne has quit IRC18:03
*** sabari is now known as zz_sabari18:08
*** pblaho_ has quit IRC18:16
*** stevemar has quit IRC18:16
*** stevemar has joined #openstack-oslo18:16
*** yamahata has quit IRC18:20
jd__jamespage: it uses whatever the user wants to use18:21
jd__jamespage: though IPC is less likely to be used so it'd be OK to disable it18:21
jd__jamespage: is it just a problem for tests or a problem to package IPC on Ubuntu?18:21
*** subscope has joined #openstack-oslo19:05
*** yamahata has joined #openstack-oslo19:09
*** zz_jgrimm is now known as jgrimm19:22
*** raildo has joined #openstack-oslo19:25
*** zz_sabari is now known as sabari19:26
*** sabari is now known as zz_sabari19:27
*** rpodolyaka2 has joined #openstack-oslo19:29
openstackgerritDoug Hellmann proposed openstack/oslo.utils: Improve performance of strutils.mask_password  https://review.openstack.org/14556219:32
dhellmannsileht, dims__, sdague: ^^19:33
sdagueyep, looks very sane19:35
*** zz_sabari is now known as sabari19:35
dhellmannsdague: I'm a little scared by how much faster it is19:35
sdagueis that a bunch of runs averaged?19:36
sdaguehonestly the double .*? is a big anti pattern in regex, because it keeps adjusting back and forth by one character searching for the matches19:36
dhellmannI think it's just one rep, but the results were pretty consistent across several runs of the test19:36
dhellmannyeah19:36
dhellmannwell, I'm also only looking for the patterns where we see the key in the input string using "in" first, and so that cuts down on the number of regexes applied to any given input19:37
dhellmannthat "in" test for a literal should be a lot faster than the regex19:37
*** vigneshvar has quit IRC19:38
dhellmannsdague: if I add a bunch of reps, I get pretty consistent numbers19:39
sdagueyeh, though now that you scoped the regex correctly, they are mostly clocking in at a few miliseconds even for large output19:40
dhellmanntrue19:46
openstackgerritJames Carey  proposed openstack/oslo.i18n: Improve fixture documentation  https://review.openstack.org/14537519:47
*** andreykurilin has joined #openstack-oslo19:50
*** rpodolyaka2 has quit IRC19:57
*** jaypipes is now known as jaypipes-afk20:00
dims__dhellmann: looking20:11
dhellmanndims__: pep8 failure of some sort :-(20:12
dhellmannoh, hrm, timeout downloading a package20:13
dims__"The remote end hung up unexpectedly" << old style phone? :)20:13
* dhellmann removes the modem dialing instructions from his changeset20:14
dims__haha20:16
openstackgerritMerged openstack/oslo.messaging: Don't log each received messages  https://review.openstack.org/14554120:21
*** rpodolyaka2 has joined #openstack-oslo20:22
dims__dhellmann: scratching my head on this one - (<%(key)s>)[^<]*(</%(key)s>20:23
dims__dhellmann: the other one replacing .*? with [^\"\'] i can understand20:24
dims__oops [^\"\']*20:24
dhellmannit's an xml pattern, looking for the content between two tags <key> and </key>20:25
dhellmannthe content can't have a < in it, because that wouldn't be valid xml (it'll be &lt;)20:25
dims__right, does it mean the string in between cannot have a < sign?20:25
dhellmannright, but it already can't have that and be valid xml20:26
dims__right &lt;20:26
dhellmannright, the < has to be written as an entity escape sequence20:26
dims__cool. lgtm! thanks20:26
dhellmannnp, thanks for the close review :-)20:27
dims__i missed the backtracking issue when i was doing this with amrith :(20:28
openstackgerritMin Pae proposed openstack/taskflow: Fix for job consumption example using wrong object  https://review.openstack.org/14557420:28
*** e0ne has joined #openstack-oslo20:28
dhellmannwith the small test strings it wouldn't have mattered, and I don't know that I would have thought about it either20:28
dims__right20:29
bnemecThis is why I cringe every time I review something with a regex20:29
dims__bnemec: yep, true!20:29
dhellmannwell, I don't think anyone anticipated a 310K log message, either20:30
dims__:)20:30
bnemecThere's always _something_ with regexes though.  I don't know that I've ever written one that worked completely right on the first try. :-)20:31
bknudsonunit tests20:32
bknudsonthey're a good target for tdd20:32
bnemecYeah, as long as you can come up with all of the edge cases.  Like 310k input strings. ;-)20:33
dhellmanngordc: I'm thinking of releasing oslo.middleware with the namespace package changes. Do you have anything you want to wait for?20:34
gordcdhellmann: nope. was going to ask you exact same question but i figured you were busy with other oslo namespace changes.20:34
dhellmanngordc: ok, I'll do it in a few minutes20:34
gordcdhellmann: cool cool.20:35
dhellmanngordc: 0.3.0 would be the next number, unless you think we're ready for 1.020:35
dhellmannI'll just use 0.3.0, we can bump to 1.0 later20:36
gordc0.3.0 works for me.20:36
bknudsonhaving unit tests sure makes it easier to verify that performance fixes are valid20:37
bknudsonit looks like the replace code was looping through the string multiple times & replacing.20:37
bnemecHmm, some of the patterns are escaping \" and some aren't.  I guess that's probably fine though.20:39
openstackgerritDoug Hellmann proposed openstack/oslo.middleware: Fix bug tracker link in readme  https://review.openstack.org/14557820:40
*** amrith is now known as _amrith_20:40
dhellmannbnemec: yeah, I was trying to change as little as possible, but I don't think a lot of those escapes are needed20:40
dhellmannbknudson: it is, using different patterns20:40
bnemecdhellmann: Yeah, it's an existing problem, not something to block this change.20:41
*** subscope has quit IRC20:41
dhellmanngordc: that readme fix ^^ would be good to get in if you have a few minutes20:41
dhellmannbnemec: yeah, we could probably clean that up further in a couple of ways20:42
gordcdhellmann: done.20:43
dhellmanngordc: thanks20:44
dhellmannoslo.middleware 0.3.0 released20:46
*** ajo has quit IRC20:47
openstackgerritDoug Hellmann proposed openstack/oslo.rootwrap: Add cross-testing script  https://review.openstack.org/14558220:49
dhellmannttx: how do you feel about an oslo.rootwrap release?20:49
dims__dhellmann: i saw this request from jogo for rootwrap - https://bugs.launchpad.net/oslo.rootwrap/+bug/140807321:00
dims__to make grenade better/easier?21:01
dhellmannI'm not sure what to do with that. The config settings are up to the app, not the lib, so I don't think we can put them in the lib repository.21:01
dims__y, jogo are you saying that newer rootwrap is looking for a rootwrap specific config file now?21:03
*** sabari is now known as zz_sabari21:05
*** openstackgerrit has quit IRC21:05
*** openstackgerrit has joined #openstack-oslo21:05
*** salv-orl_ has joined #openstack-oslo21:06
*** salv-or__ has joined #openstack-oslo21:07
*** e0ne has quit IRC21:08
*** salv-orlando has quit IRC21:10
*** salv-orlando has joined #openstack-oslo21:11
*** salv-orl_ has quit IRC21:11
*** salv-or__ has quit IRC21:12
*** noelbk has joined #openstack-oslo21:24
mriedemdhellmann: i'm running tox for nova on stable/juno and hitting the billion deprecation warnings which stevedore 1.2.0 fixed on master (i thought?), but still seeing a ton of these:21:24
mriedem      return pkg_resources.EntryPoint.parse("x=" + s).load(False)21:24
mriedem    /home/mriedem/git/nova/.tox/py27/local/lib/python2.7/site-packages/paste/deploy/loadwsgi.py:22: DeprecationWarning: Parameters to load are deprecated.  Call .resolve and .require separately.21:24
mriedemchecked my venv and i have stevedore 1.2.021:24
mriedemany ideas?21:24
dhellmannmriedem: looking at the path, those calls are coming from paste21:24
mriedemah crap21:25
dhellmannit looks like we need to get the paste folks to update their code, or convince the setuptools folks to drop the warning21:25
mriedemyup http://logs.openstack.org/74/145374/1/check/gate-nova-python27/6201323/console.html#_2015-01-06_23_34_35_47721:25
mriedem20736 hits of that21:25
mriedemsdague: ^ :(21:25
sdagueoh, that's down in paste21:26
dhellmanndid we really not figure out how to make those deprecation warnings only show up once?21:26
sdaguepaste deploy is basically stalled right?21:26
dhellmannthat might be a change to paste, too21:26
sdaguedhellmann: so... I tried the thing that was suggested21:27
sdaguebut it didn't seem to work21:27
dhellmannI really don't know. That was my impression, given the lack of python 3 support a couple of years ago. It's possible that's changed and paste is more active now.21:27
dhellmannah, bummer21:27
sdaguehttps://review.openstack.org/#/c/145037/1/nova/test.py,cm21:27
mriedemoo someone is using new gerrit21:27
sdaguethe docs indicate that that should log once, but it doesn't seem to21:28
dhellmannsdague: I wonder if that didn't work because we have deprecations wired up to the logging?21:28
sdaguethis isn't going through logging21:28
sdagueit's going to stderr directly from python warnings module21:28
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: rabbit: fix timeout timer when duration is None  https://review.openstack.org/14552121:30
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Ensure kombu channels are closed  https://review.openstack.org/14523221:30
dhellmannsdague: oh, I thought we were capturing them21:30
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Add a info log when a reconnection occurs  https://review.openstack.org/13841121:30
sdaguedhellmann: the issue is they are in the stderr stream21:30
sdaguewhich goes out to the console and the subunit logs, because it captures that21:30
dhellmannsdague: maybe try the more complex filter specification?21:31
sdaguemaybe21:31
dhellmannsdague: or https://docs.python.org/2/library/logging.html#logging.captureWarnings21:31
dhellmannthat would cause them to go through logging, so they would at least not be going to stderr21:31
*** zz_sabari is now known as sabari21:31
sdaguetrue21:32
bknudsonin keystone I changed the tests so that deprecations are only printed from keystone.*21:32
dhellmannbknudson: how did you do that? we're having trouble making nova do something similar21:32
mriedemhttps://review.openstack.org/#/c/143183/2/keystone/tests/core.py21:32
bknudsonmriedem: that's the wrong one.21:32
mriedemhttps://review.openstack.org/#/c/144810/2/keystone/tests/core.py21:32
bknudsonthat one caused everything to fail once paste started calling deprecated code21:33
sdaguebknudson: ok, but you are still getting the output right?21:33
bknudsony, it's module='^keystone\\.')21:33
sdaguejust not as an error21:33
dhellmannyeah, that turns some of the deprecations into errors, but doesn't ignore the others21:33
sdagueI can try to work with that tomorrow though21:33
sdagueI need to leave shortly for lug meeting21:34
dhellmannsdague: maybe adding a module filter to the ignore line you have would make it take precedence21:34
sdaguedhellmann: yeh21:34
bknudsonthe filters are applied in order... the first one that matches causes it to stop21:34
sdaguehonestly, something later might be reseting it as well21:34
dhellmanntrue21:34
dhellmannoh, you know, we did have a module messing with the filters at some point21:34
dhellmannmaybe moving the call to happen in the test setup would help21:34
sdagueyeh, that might21:34
dhellmannwe could also make the oslotest base class call logging.captureWarnings for the same reason21:35
sdagueok, I'll dive in in the morning on it if now one has fixed it21:35
mriedemi can pokey pokey21:35
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Declare DirectPublisher exchanges with passive=True  https://review.openstack.org/10937321:35
sdaguemriedem: you might want to build it as another fixture in the nova test class, I was pretty close to extracting everything as fixtures there anyway21:36
bknudsonthe output of keystone tests is different for some reason... we don't see the output unless there's a failure in the test.21:36
mriedembknudson: that's something sdague added to nova from tempest21:36
sdaguethe bigger issue, if this is deprecated, at some point it's going to change21:36
*** rpodolyaka2 has quit IRC21:36
sdaguewhich mean paste deploy is going to break21:36
sdagueand this is dragging in from pip, so we're not really going to have the ability to pin it21:37
mriedembknudson: https://review.openstack.org/#/c/145340/21:37
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Implements notification-dispatcher-filter  https://review.openstack.org/7788621:38
*** arnaud__ has quit IRC21:38
*** arnaud__ has joined #openstack-oslo21:39
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: Move each drivers options into its own group  https://review.openstack.org/12003821:40
dims__dhellmann: does overriding warnings.showwarning help with avoiding stderr flood?21:42
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: NotifyPublisher need handle amqp_auto_delete  https://review.openstack.org/13872321:43
*** arnaud__ has quit IRC21:43
sdaguedims__: in theory the warnings module has facilities for what we want, just, so far just having gotten the right incantation21:44
dims__sdague: ah cool21:45
*** ajo has joined #openstack-oslo21:48
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: RabbitMQ heartbeat implementation  https://review.openstack.org/12633021:48
*** e0ne has joined #openstack-oslo21:52
*** e0ne has quit IRC21:52
*** jeckersb is now known as jeckersb_gone21:57
*** jaypipes-afk is now known as jaypipes21:58
*** jecarey has quit IRC21:58
jamespagejd__, both sysv and posix ipc are packaged, just getting a bit of pushback from the Ubuntu MIR team on having both supported in Ubuntu main22:11
openstackgerritMehdi Abaakouk proposed openstack/oslo.messaging: RabbitMQ heartbeat implementation  https://review.openstack.org/12633022:12
openstackgerritJames Page proposed openstack/oslo.messaging: zmq: support Cinder multi-backend server targets  https://review.openstack.org/12918522:15
openstackgerritJames Page proposed openstack/oslo.messaging: zmq: Refactor test case shared code  https://review.openstack.org/14495422:16
jogodims__: the issue here is if you add a new line to the rootwrap config22:16
jogodims__: and upgrade but leave your /etc/ behind you are stuck with an old rootwrap config and new code fails22:16
dhellmannjogo: do we consider the rootwrap config file to be something the deployer needs to customize, or is it "part of the code"?22:18
*** shardy is now known as shardy_z22:19
jogodhellmann: IMHO part of the code22:19
jogodhellmann: maybe distros need to customize? but not sure why a user would need to22:19
dhellmannjogo: in that case I would expect the file to be updated when the rest of the code is updated, right?22:19
dhellmannmaybe that means we're putting it in a bad place?22:19
dhellmannttx: ^^22:19
jogodhellmann: that is my train of thought as well22:20
*** harlowja_at_home has joined #openstack-oslo22:29
*** stevemar has quit IRC22:37
*** mriedem has quit IRC22:45
*** pradk has quit IRC23:01
*** gordc has quit IRC23:08
openstackgerritMichael Bayer proposed openstack/oslo.db: Ensure mysql_sql_mode is set for MySQLOpportunisticTests  https://review.openstack.org/14562223:10
openstackgerritMichael Bayer proposed openstack/oslo.db: Ensure DBConnectionError is raised on failed revalidate  https://review.openstack.org/13973823:10
*** yassine has quit IRC23:18
*** stevemar has joined #openstack-oslo23:24
*** jgrimm is now known as zz_jgrimm23:26
*** dims__ has quit IRC23:30
openstackgerritMerged openstack/oslo.utils: Improve performance of strutils.mask_password  https://review.openstack.org/14556223:38
dhellmannI'll cut a release of oslo.utils first thing in the morning with ^^ unless someone else wants to do it and hang around for a while to make sure it doesn't break things worse than they are now. :-)23:42
dhellmanncall it 14 hrs23:42
*** zzzeek has quit IRC23:47
*** andreykurilin has quit IRC23:50
*** gordc has joined #openstack-oslo23:53
« Tuesday, 2015-01-06 Index Thursday, 2015-01-08 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!