*** crc32 has quit IRC | 00:01 | |
*** prad has quit IRC | 00:05 | |
*** crc32 has joined #openstack-oslo | 00:12 | |
*** achanda has quit IRC | 00:15 | |
*** achanda has joined #openstack-oslo | 00:16 | |
*** achanda has quit IRC | 00:18 | |
*** achanda has joined #openstack-oslo | 00:18 | |
*** sputnik13 has quit IRC | 00:23 | |
*** tsekiyam_ has joined #openstack-oslo | 00:27 | |
*** mtanino has quit IRC | 00:30 | |
*** achanda has quit IRC | 00:30 | |
*** tsekiyama has quit IRC | 00:30 | |
*** tsekiyam_ has quit IRC | 00:31 | |
*** ChuckC has quit IRC | 00:53 | |
*** salv-orl_ has quit IRC | 01:04 | |
openstackgerrit | Joshua Harlow proposed openstack/taskflow: Always return scope walker instances from `fetch_scopes_for` https://review.openstack.org/165262 | 01:05 |
---|---|---|
*** stevemar has joined #openstack-oslo | 01:47 | |
*** dims has quit IRC | 01:59 | |
*** harlowja_ is now known as harlowja_away | 02:06 | |
*** yamahata has quit IRC | 02:09 | |
*** crc32 has quit IRC | 02:11 | |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: ZeroMQ deployment guide https://review.openstack.org/130943 | 02:16 |
*** salv-orlando has joined #openstack-oslo | 02:18 | |
*** crc32 has joined #openstack-oslo | 02:20 | |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: ZeroMQ deployment guide https://review.openstack.org/130943 | 02:20 |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: ZeroMQ deployment guide https://review.openstack.org/130943 | 02:27 |
*** zzzeek has joined #openstack-oslo | 02:35 | |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: ZeroMQ deployment guide https://review.openstack.org/130943 | 02:44 |
*** exploreshaifali has joined #openstack-oslo | 02:55 | |
*** exploreshaifali has quit IRC | 03:00 | |
*** zzzeek has quit IRC | 03:01 | |
openstackgerrit | Fei Long Wang proposed openstack/oslo.messaging: Minor improvement https://review.openstack.org/165278 | 03:07 |
*** crc32 has quit IRC | 03:27 | |
*** crc32 has joined #openstack-oslo | 03:29 | |
*** salv-orlando has quit IRC | 03:29 | |
*** salv-orlando has joined #openstack-oslo | 04:28 | |
*** rushiagr_away is now known as rushiagr | 04:37 | |
*** boris-42 has joined #openstack-oslo | 04:47 | |
*** dims has joined #openstack-oslo | 04:52 | |
*** dims has quit IRC | 04:57 | |
*** stevemar has quit IRC | 04:58 | |
*** stevemar has joined #openstack-oslo | 04:59 | |
*** BrianShang_ has joined #openstack-oslo | 05:04 | |
*** achanda has joined #openstack-oslo | 05:06 | |
*** BrianShang has quit IRC | 05:06 | |
*** stevemar has quit IRC | 05:11 | |
*** stevemar has joined #openstack-oslo | 05:12 | |
*** sputnik13 has joined #openstack-oslo | 05:15 | |
*** xek has quit IRC | 05:18 | |
*** sputnik13 has quit IRC | 05:23 | |
*** sputnik13 has joined #openstack-oslo | 05:25 | |
*** salv-orlando has quit IRC | 05:31 | |
*** rushiagr is now known as rushiagr_away | 05:36 | |
openstackgerrit | Joshua Harlow proposed openstack/taskflow: Always return scope walker instances from `fetch_scopes_for` https://review.openstack.org/165262 | 05:44 |
*** subscope_ has joined #openstack-oslo | 05:44 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/oslo.db: Imported Translations from Transifex https://review.openstack.org/164579 | 06:01 |
*** rushiagr_away is now known as rushiagr | 06:03 | |
*** sreshetn1 has joined #openstack-oslo | 06:06 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/oslo.messaging: Imported Translations from Transifex https://review.openstack.org/164404 | 06:10 |
*** sreshetn1 has quit IRC | 06:10 | |
*** achanda has quit IRC | 06:25 | |
*** achanda has joined #openstack-oslo | 06:27 | |
*** sputnik13 has quit IRC | 06:31 | |
*** liusheng has quit IRC | 06:31 | |
*** liusheng has joined #openstack-oslo | 06:31 | |
*** inc0 has joined #openstack-oslo | 06:31 | |
openstackgerrit | Mehdi Abaakouk proposed openstack/tooz: fix lock concurrency issues with certain drivers https://review.openstack.org/164642 | 06:32 |
*** salv-orlando has joined #openstack-oslo | 06:33 | |
*** subscope_ has quit IRC | 06:56 | |
openstackgerrit | Masaki Matsushita proposed openstack-dev/pbr: Add truncated lincense text https://review.openstack.org/165319 | 07:10 |
*** hogepodge has quit IRC | 07:16 | |
*** salv-orlando has quit IRC | 07:33 | |
*** achanda has quit IRC | 07:35 | |
*** achanda has joined #openstack-oslo | 07:39 | |
*** salv-orlando has joined #openstack-oslo | 07:42 | |
*** stevemar has quit IRC | 07:50 | |
*** shardy has joined #openstack-oslo | 08:04 | |
*** rushiagr is now known as rushiagr_away | 08:05 | |
*** dulek has joined #openstack-oslo | 08:09 | |
*** achanda has quit IRC | 08:10 | |
*** rushiagr_away is now known as rushiagr | 08:11 | |
*** exploreshaifali has joined #openstack-oslo | 08:12 | |
*** dtantsur|afk is now known as dtantsur | 08:28 | |
*** ndipanov has joined #openstack-oslo | 08:33 | |
*** uschreiber_ has joined #openstack-oslo | 08:38 | |
*** uschreiber_ has quit IRC | 08:39 | |
*** i159 has joined #openstack-oslo | 08:40 | |
*** uschreiber_ has joined #openstack-oslo | 08:41 | |
*** uschreiber_ has quit IRC | 08:42 | |
*** uschreiber_ has joined #openstack-oslo | 08:42 | |
*** uschreiber_ has quit IRC | 08:42 | |
openstackgerrit | Victor Sergeyev proposed openstack/oslo.db: Provide working SQLA_VERSION attribute https://review.openstack.org/165166 | 08:44 |
*** ajo has joined #openstack-oslo | 08:51 | |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: heartbeat implementation https://review.openstack.org/146047 | 09:05 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: Improves logging https://review.openstack.org/165336 | 09:05 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 09:05 |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: Fix deleting keys during iteration in matchmaker heartbeat https://review.openstack.org/164972 | 09:07 |
*** sreshetn1 has joined #openstack-oslo | 09:07 | |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: Fix changing keys during iteration in matchmaker heartbeat https://review.openstack.org/164972 | 09:09 |
*** sreshetn1 has quit IRC | 09:11 | |
*** boris-42 has quit IRC | 09:12 | |
*** andreykurilin_ has joined #openstack-oslo | 09:15 | |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: Improves logging https://review.openstack.org/165336 | 09:16 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 09:16 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: heartbeat implementation https://review.openstack.org/146047 | 09:16 |
*** e0ne has joined #openstack-oslo | 09:19 | |
*** mfedosin has quit IRC | 09:21 | |
*** hogepodge has joined #openstack-oslo | 09:22 | |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 09:22 |
*** mfedosin has joined #openstack-oslo | 09:26 | |
openstackgerrit | Julien Danjou proposed openstack/oslo-incubator: Remove timeutils.strtime() usage https://review.openstack.org/165340 | 09:28 |
*** pblaho has joined #openstack-oslo | 09:36 | |
*** dims_ has joined #openstack-oslo | 09:43 | |
dims_ | sileht: Mehdi, good morning | 09:44 |
sileht | dims_, hi | 09:44 |
dims_ | sileht: did you see a set of logs i sent last night my time? | 09:44 |
dims_ | i sent it to sileht AT sileht.net email | 09:46 |
sileht | dims_, I have just open it | 09:46 |
dims_ | thanks, it has some __getitem__ kind of tracebacks and some others too | 09:46 |
sileht | very wierd, looks like a pyamqp bug | 09:47 |
*** alexpilotti has joined #openstack-oslo | 09:47 | |
dims_ | sileht: ok, also, i had to switch a bunch of LOG.exception to LOG.error to stop tracebacks from being printed in the logs - http://paste.openstack.org/show/192895/ | 09:49 |
dims_ | please see if those are appropriate | 09:49 |
dims_ | back in a bit, it's way early here, need to wake up kids for school :) | 09:49 |
sileht | dims_, I have rewritten the log thing | 09:49 |
sileht | dims_, https://review.openstack.org/#/c/165336/ | 09:49 |
sileht | dims_, and discover a new (very old) bug during my testing: https://review.openstack.org/#/c/165337/ | 09:50 |
dims_ | sileht: i saw one spot | 09:50 |
dims_ | sileht: the paste above has 4 more spots for LOG.exception -> LOG.error | 09:50 |
dims_ | ah | 09:50 |
sileht | dims_, I have changed the code a bit to completly remove the LOG.exception when this is not needed | 09:51 |
dims_ | AH, did not see the other reviews | 09:51 |
dims_ | thanks! | 09:51 |
sileht | dims_, so unexpected exception should continue to LOG.exception and recoverable are silencly muted | 09:52 |
sileht | dims_, last thing I would like to add is a 'id' for each log message that represented the connection object | 09:53 |
dims_ | yes, that would be great | 09:56 |
dims_ | one more from me, is there a switch to switch off heartbeat? | 09:56 |
sileht | dims_, with timeout of 0 | 09:56 |
dims_ | cool | 09:57 |
sileht | dims_, I will add that to the oslo.config option doc string | 09:57 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: Improves logging https://review.openstack.org/165336 | 09:58 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 09:58 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: heartbeat implementation https://review.openstack.org/146047 | 09:58 |
dims_ | sileht: y for heartbeat_timeout_threshold | 10:00 |
*** crc32 has quit IRC | 10:01 | |
dims_ | sileht: do we print message id when we are unable to send something? and it gets dropped on the floor? | 10:02 |
sileht | dims_, I don't think so, because everything is already serialized when we got this kind of error | 10:05 |
sileht | dims_, but you can follow log because just before serialization we LOG.debug the UNIQUE_ID of the message | 10:06 |
sileht | dims_, when the message is received we print it too | 10:07 |
*** inc0 has quit IRC | 10:08 | |
*** dtantsur is now known as dtantsur|bbl | 10:08 | |
*** xek has joined #openstack-oslo | 10:12 | |
*** e0ne is now known as e0ne_ | 10:15 | |
dims_ | sileht: ack, thanks. making sure we can trace them in case of issues | 10:16 |
*** inc0 has joined #openstack-oslo | 10:16 | |
sileht | dims_, do you known the version of pyamqp that klindgren_ use ? It looks installed with pip, so I guess 1.4.6 ? | 10:17 |
dims_ | dims@dims-mac:~/openstack/oslo/oslo.messaging$ pip freeze | grep amqp | 10:18 |
dims_ | amqp==1.4.6 | 10:18 |
dims_ | amqplib==1.0.2 | 10:18 |
dims_ | those are from my box | 10:18 |
*** exploreshaifali has quit IRC | 10:21 | |
sileht | dims_, that looks a amqp bug: https://github.com/celery/py-amqp/blob/v1.4.6/amqp/connection.py#L202 | 10:22 |
sileht | self.channels is set to None instead of {} | 10:22 |
*** ihrachyshka has joined #openstack-oslo | 10:24 | |
*** e0ne_ has quit IRC | 10:26 | |
dims_ | sileht: override _do_close till they fix it? | 10:26 |
sileht | dims_, I can't reproduce it locally I got the expected ChannelError, we I try to trigger that code | 10:28 |
dims_ | sileht: y i did not get that when i tried against a rabbitmq cluster, only when i went against haproxy fronting the 2 rabbitmq instances | 10:29 |
dims_ | anyway, i can submit a patch later if i still see it with latest changes from you | 10:29 |
*** andreykurilin_ has quit IRC | 10:34 | |
*** e0ne has joined #openstack-oslo | 10:38 | |
openstackgerrit | Li Ma proposed openstack/oslo.messaging: Fix changing keys during iteration in matchmaker heartbeat https://review.openstack.org/164972 | 10:39 |
*** crc32 has joined #openstack-oslo | 10:40 | |
sileht | dims_, I still get 'NotFound: Exchange.declare: (404) NOT_FOUND - no exchange ...' with haproxy, I will setup a rabbit cluster with haqueue, perhaps related to combination of rabbit HA+haproxy | 10:44 |
sileht | I will follow exactly the procedure of krislindgren | 10:45 |
* sileht will grab some foods first | 10:45 | |
sileht | bbl | 10:45 |
*** exploreshaifali has joined #openstack-oslo | 10:45 | |
*** boris-42 has joined #openstack-oslo | 10:51 | |
*** rushiagr is now known as rushiagr_away | 11:01 | |
*** dulek has quit IRC | 11:02 | |
*** dtantsur|bbl is now known as dtantsur | 11:02 | |
*** dulek has joined #openstack-oslo | 11:02 | |
*** inc0_ has joined #openstack-oslo | 11:02 | |
*** sputnik13 has joined #openstack-oslo | 11:02 | |
*** inc0 has quit IRC | 11:06 | |
*** amotoki has joined #openstack-oslo | 11:10 | |
openstackgerrit | Julien Danjou proposed openstack/oslo-incubator: Remove timeutils.strtime() usage https://review.openstack.org/165340 | 11:17 |
*** cdent has joined #openstack-oslo | 11:29 | |
*** e0ne is now known as e0ne_ | 11:41 | |
*** achanda has joined #openstack-oslo | 11:45 | |
*** achanda has quit IRC | 11:49 | |
*** rushiagr_away is now known as rushiagr | 11:51 | |
*** e0ne_ has quit IRC | 11:51 | |
*** e0ne has joined #openstack-oslo | 11:59 | |
*** david-lyle_afk is now known as david-lyle | 12:00 | |
*** ajo has quit IRC | 12:03 | |
*** ajo has joined #openstack-oslo | 12:03 | |
*** stpierre has joined #openstack-oslo | 12:09 | |
*** kgiusti has joined #openstack-oslo | 12:22 | |
*** dims_ has quit IRC | 12:22 | |
*** dims_ has joined #openstack-oslo | 12:22 | |
*** jaosorior has joined #openstack-oslo | 12:38 | |
openstackgerrit | Doug Hellmann proposed openstack/oslo-incubator: Move the release_notes.py script to release-tools https://review.openstack.org/161274 | 12:40 |
*** sputnik13 has quit IRC | 12:40 | |
*** dulek has quit IRC | 12:46 | |
*** rushiagr is now known as rushiagr_away | 12:49 | |
*** geguileo has quit IRC | 12:50 | |
*** geguileo has joined #openstack-oslo | 12:51 | |
*** jecarey has joined #openstack-oslo | 12:54 | |
*** geguileo has quit IRC | 12:56 | |
*** salv-orlando has quit IRC | 12:56 | |
dhellmann | dims_: it would be good to get https://review.openstack.org/#/c/164993/ in before neutron's freeze | 12:57 |
dims_ | eezhova: ping | 12:57 |
dims_ | dhellmann: ack | 12:57 |
*** salv-orlando has joined #openstack-oslo | 12:57 | |
*** inc0_ has quit IRC | 12:59 | |
*** gordc has joined #openstack-oslo | 12:59 | |
*** sputnik13 has joined #openstack-oslo | 13:00 | |
*** sputnik13 has quit IRC | 13:01 | |
*** boris-42 has quit IRC | 13:02 | |
*** gordc has quit IRC | 13:02 | |
*** pblaho has quit IRC | 13:02 | |
*** amotoki has quit IRC | 13:03 | |
*** gordc has joined #openstack-oslo | 13:04 | |
dims_ | dhellmann: i asked eezhova if SIGHUP can be sent multiple times and the code works fine for that scenario. can you tell from the code that it will work fine? | 13:04 |
dhellmann | dims_: great question, let me look again | 13:05 |
*** amotoki has joined #openstack-oslo | 13:05 | |
*** bknudson has quit IRC | 13:06 | |
dhellmann | dims_: I think it should be fine. The handlers are registered once at startup and then called when the signal is received. Whether a given handler works when called more than once is out of our control, but as far as I can tell they are called properly. | 13:06 |
dhellmann | dims_: did you see something that made you worry about that? maybe I'm missing it. | 13:07 |
dims_ | _set_signals_handler(signal.SIG_DFL) | 13:07 |
dims_ | i see that in a couple of spots dhellmann | 13:07 |
dhellmann | aren't those calls in places where the service is shutting down? | 13:09 |
* dhellmann looks again | 13:09 | |
dhellmann | oh, hmm, no | 13:10 |
dhellmann | ok, so I think that may be a completely different issue than the one the OP was trying to fix | 13:10 |
dims_ | true, but the scenario is that they want to be able to repeatedly change log levels... | 13:11 |
dhellmann | right | 13:11 |
dhellmann | what I would really like to see is a general callback registry for signal handlers | 13:11 |
dhellmann | rather than messing with sets of functions on a class like this | 13:11 |
dims_ | agree dhellmann | 13:12 |
dhellmann | with the freeze tomorrow, we might just have to punt this fix to liberty :-/ | 13:13 |
dims_ | exactly | 13:13 |
dhellmann | the change in process right now is valid, it just doesn't solve their problem completely | 13:13 |
dims_ | ok let me +2 it anyway | 13:13 |
*** rushiagr_away is now known as rushiagr | 13:20 | |
openstackgerrit | Merged openstack/oslo-incubator: Store ProcessLauncher signal handlers on class level https://review.openstack.org/164993 | 13:20 |
sdague | dhellmann: is there an oslo specific log capture fixture? The one in fixtures doesn't let you specific our extra formatting params, which makes debugging some of the unit/functional tests with multiple requests a bit problematic | 13:23 |
dhellmann | dims_: https://github.com/j4mie/simplesignals looks interesting | 13:23 |
*** mriedem_away has quit IRC | 13:24 | |
dhellmann | sdague: looking... | 13:24 |
*** bknudson has joined #openstack-oslo | 13:27 | |
*** prad has joined #openstack-oslo | 13:27 | |
dims_ | dhellmann: yes it does, will look in further when i get a chance. trying to work on the oslo.messaging heartbeat and a neutron/nova problem :) | 13:27 |
dhellmann | sdague: the code in the oslotest base class sets up the FakeLogger using a format string. Is that not what you want to do? or do you want to use an oslo.log ContextFormatter with the FakeLogger? | 13:27 |
dhellmann | dims_: oh, that's definitely a higher priority :-) | 13:27 |
dhellmann | dims_: I was just trying to answer the "does this already exist" question :-) | 13:28 |
*** ChuckC has joined #openstack-oslo | 13:28 | |
sdague | right, that format string is only allowed to have things defined in python logging | 13:28 |
dims_ | dhellmann: ack :) | 13:28 |
sdague | which is the problem | 13:28 |
dhellmann | sdague: "extra" is one of those things, right? | 13:28 |
dhellmann | or maybe it's "extras" | 13:29 |
sdague | so, I guess the answer is, yes, I want to use ContextFormatter | 13:29 |
dhellmann | sdague: what's verifying that the format string only includes python logger variables? | 13:29 |
*** e0ne is now known as e0ne_ | 13:30 | |
dhellmann | ok, we can take the logging stuff out of oslotest and put it in a fixture in oslo.log | 13:30 |
sdague | I don't remember now, the last time I tried this I got a pretty deep level of explode | 13:30 |
dhellmann | I'll leave a note to myself on the spec to refactor oslotest's base class in https://review.openstack.org/#/c/158788/ | 13:30 |
sdague | honestly, we have our own fixtures for this stuff in nova, mostly I want to know if there is a way to do it | 13:30 |
dhellmann | there's not right now. I would be happy to have those fixtures in oslo.log, if that's appropriate | 13:32 |
* dhellmann wishes folks would think about sharing more test tooling | 13:32 | |
*** salv-orlando has quit IRC | 13:34 | |
*** zzzeek has joined #openstack-oslo | 13:34 | |
*** alexpilotti has quit IRC | 13:35 | |
*** crc32 has quit IRC | 13:38 | |
sdague | Well, that's a longer discussion. But I'm generally pretty anti externalizing test code because you really *have* to know the details in that code to actually know that you are testing the correct thing | 13:40 |
*** e0ne_ has quit IRC | 13:40 | |
*** boris-42 has joined #openstack-oslo | 13:41 | |
*** mriedem has joined #openstack-oslo | 13:43 | |
openstackgerrit | Merged openstack/oslo-incubator: Move the release_notes.py script to release-tools https://review.openstack.org/161274 | 13:44 |
dhellmann | sdague: fixtures like what you're describing seem very sharable | 13:46 |
openstackgerrit | Merged openstack/oslo.db: Provide working SQLA_VERSION attribute https://review.openstack.org/165166 | 13:46 |
*** cdent has quit IRC | 13:47 | |
*** enikanorov_ has joined #openstack-oslo | 13:47 | |
sdague | I've personally found myself needing to change our in tree fixtures quite often to address functional testing cases | 13:47 |
*** enikanorov has quit IRC | 13:49 | |
*** dulek has joined #openstack-oslo | 13:49 | |
*** cdent has joined #openstack-oslo | 13:49 | |
*** zzzeek has quit IRC | 13:52 | |
*** e0ne has joined #openstack-oslo | 13:54 | |
*** zz_jgrimm is now known as jgrimm | 13:57 | |
*** jecarey has quit IRC | 13:59 | |
*** dulek has quit IRC | 13:59 | |
*** dulek has joined #openstack-oslo | 14:00 | |
*** alexpilotti has joined #openstack-oslo | 14:03 | |
*** sigmavirus24_awa is now known as sigmavirus24 | 14:08 | |
*** inc0 has joined #openstack-oslo | 14:11 | |
*** mfedosin has quit IRC | 14:13 | |
*** mtanino has joined #openstack-oslo | 14:13 | |
*** mfedosin has joined #openstack-oslo | 14:14 | |
*** alexpilotti has quit IRC | 14:14 | |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: Improves logging https://review.openstack.org/165336 | 14:15 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 14:15 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: heartbeat implementation https://review.openstack.org/146047 | 14:15 |
*** alexpilotti has joined #openstack-oslo | 14:15 | |
openstackgerrit | Ken Giusti proposed openstack/oslo.messaging: Provide better detection of failures during message send https://review.openstack.org/126065 | 14:18 |
openstackgerrit | Ken Giusti proposed openstack/oslo.messaging: Create a unique transport for each server in the functional tests https://review.openstack.org/155476 | 14:18 |
*** inc0 has quit IRC | 14:28 | |
*** exploreshaifali has quit IRC | 14:31 | |
*** tsekiyama has joined #openstack-oslo | 14:36 | |
klindgren_ | <sileht> dims_, do you known the version of pyamqp that klindgren_ use ? It looks installed with pip, so I guess 1.4.6 ? | 14:45 |
klindgren_ | python-amqplib-1.0.2-7.el7.noarch | 14:45 |
klindgren_ | python-amqp-1.4.6-0.el7.centos.noarch | 14:45 |
dims_ | klindgren_: thanks for confirming | 14:46 |
sileht | klindgren_, thanks, I try to reproduce, but without success :( | 14:46 |
dims_ | sileht: give me a few, i may be able to try your latest patch series in my environment | 14:47 |
*** achanda has joined #openstack-oslo | 14:47 | |
klindgren_ | silent what do you need from me? | 14:48 |
klindgren_ | sileht, * | 14:48 |
sileht | klindgren_, you give enough information I guess, I need to figure out what happen now | 14:49 |
sileht | "specialy about this backtrace: "'TypeError: 'NoneType' object has no attribute '__getitem__'" | 14:50 |
sileht | After ready amqp code, that looks like something have closed the connection while we publish a message | 14:51 |
klindgren_ | kk - here is the config we are using btw: https://gist.github.com/krislindgren/3361463ff9a3d081cc22 | 14:51 |
*** achanda has quit IRC | 14:52 | |
*** sputnik13 has joined #openstack-oslo | 14:53 | |
dims_ | sileht: the __getitem__ logs were from me, not klindgren_ :) | 14:54 |
*** stevemar has joined #openstack-oslo | 14:54 | |
*** sputnik13 has quit IRC | 15:02 | |
*** sputnik13 has joined #openstack-oslo | 15:04 | |
*** amotoki has quit IRC | 15:06 | |
*** salv-orlando has joined #openstack-oslo | 15:09 | |
*** jecarey has joined #openstack-oslo | 15:10 | |
*** jaypipes has quit IRC | 15:12 | |
*** amotoki has joined #openstack-oslo | 15:16 | |
*** salv-orlando has quit IRC | 15:17 | |
*** pcaruana has joined #openstack-oslo | 15:19 | |
openstackgerrit | Mehdi Abaakouk proposed openstack/tooz: fix lock concurrency issues with certain drivers https://review.openstack.org/164642 | 15:22 |
openstackgerrit | Mehdi Abaakouk proposed openstack/tooz: fix lock concurrency issues with certain drivers https://review.openstack.org/164642 | 15:24 |
*** crc32 has joined #openstack-oslo | 15:26 | |
*** crc32 has quit IRC | 15:30 | |
openstackgerrit | Doug Hellmann proposed openstack/oslo.messaging: fix up verb tense in log message https://review.openstack.org/165462 | 15:34 |
*** SridharGaddam has quit IRC | 15:37 | |
*** amotoki has quit IRC | 15:38 | |
*** dulek has quit IRC | 15:44 | |
*** dulek has joined #openstack-oslo | 15:47 | |
*** _amrith_ is now known as amrith | 15:57 | |
*** salv-orl_ has joined #openstack-oslo | 15:58 | |
*** enykeev has quit IRC | 15:58 | |
openstackgerrit | Roman Prykhodchenko proposed openstack/cliff: Allow using custom help and complete commands https://review.openstack.org/157368 | 15:59 |
*** dulek has quit IRC | 16:00 | |
*** yamahata has joined #openstack-oslo | 16:01 | |
*** enykeev has joined #openstack-oslo | 16:04 | |
*** exploreshaifali has joined #openstack-oslo | 16:06 | |
*** jgrimm_ has joined #openstack-oslo | 16:07 | |
openstackgerrit | amrith proposed openstack/oslo.utils: Allow users to add new patterns to mask with mask_password https://review.openstack.org/165478 | 16:08 |
amrith | dhellmann, I uploaded https://review.openstack.org/#/c/165478/. I believe it addresses your comment in the bug. thx! | 16:09 |
*** jgrimm- has joined #openstack-oslo | 16:09 | |
*** jgrimm has quit IRC | 16:10 | |
*** i159 has quit IRC | 16:11 | |
*** jgrimm_ has quit IRC | 16:13 | |
*** sreshetn1 has joined #openstack-oslo | 16:13 | |
klindgren_ | sileht, dims_ let me know if you need anything else from me. Thanks for you guys work on this. | 16:16 |
dims_ | klindgren_: thanks will do. wrapping up something else | 16:16 |
dhellmann | amrith: is there any chance that a project would call add_pattern() and then send a message to another service that doesn't have that same pattern added? | 16:19 |
amrith | dhellmann, I believe that this is possible but if it sends a message that has been sanitized, then all is good. that's the intended use-case. | 16:20 |
dhellmann | so we sanitize messages before we send them? | 16:21 |
amrith | well, a sanitized message may not be of any use to a recipient | 16:21 |
dhellmann | I'm not sure why you consider it more difficult to add things to the list in oslo.utils than to a consuming project. | 16:21 |
amrith | dhellmann, adding to oslo.utils is easy | 16:22 |
amrith | but there's then a version dependency | 16:22 |
amrith | and when one gets to see the new oslo.utils with the change is independent of the project requesting the change. | 16:22 |
dhellmann | we don't generally update dependencies for bug fixes like this | 16:22 |
amrith | hence the request to add dynamically, does that make sense? | 16:23 |
dhellmann | no, there's no need to update the dependency to get the fix, so I don't think we have the problem you're trying to solve | 16:24 |
amrith | actually we do | 16:24 |
amrith | I'm adding code to a module that is to be released out of band with oslo.utils | 16:24 |
amrith | a downstream change | 16:24 |
amrith | in one of our repositories | 16:25 |
dhellmann | "our"? | 16:25 |
amrith | our (tesora) | 16:25 |
amrith | we release this more frequently than either openstack or oslo | 16:25 |
dhellmann | more often than weekly? | 16:25 |
amrith | more often than 6 monthly | 16:26 |
amrith | is weekly the oslo.utils release schedule? | 16:26 |
dhellmann | oslo libraries are released weekly | 16:26 |
amrith | ok. | 16:26 |
dhellmann | as needed | 16:26 |
amrith | so assume that we release a change (in this case to mask the string pswd) | 16:26 |
amrith | we'd then have to bump our oslo.utils dependency | 16:26 |
amrith | I was hoping to avoid that. | 16:26 |
dhellmann | otoh, if this is some sort of third-party thing that isn't part of an openstack project, then I don't know if we'd want that patch | 16:26 |
amrith | it isn't, it is trove | 16:27 |
amrith | code that will be upstreamed | 16:27 |
dhellmann | ok | 16:27 |
amrith | but upstream to trove would take a while to get into a released version | 16:27 |
*** dtantsur is now known as dtantsur|afk | 16:27 | |
amrith | like 6 months | 16:27 |
amrith | the other option (which I don't like) is to write a downstream wrapper on strutils.mask_password() | 16:28 |
amrith | and bury these strings in that. | 16:28 |
amrith | then make that part of trove | 16:28 |
amrith | and upstream that | 16:28 |
amrith | that way, we could release a new trove with this change (generating a message with psdw: dhellmanns-password) and then mask it with a change in the trove wrapper on mask_password() | 16:29 |
dhellmann | amrith: how often do you expect to be adding new strings that need to be masked? | 16:29 |
amrith | I honestly don't know the answer | 16:30 |
amrith | but I assume that over time it will reduce | 16:30 |
amrith | consider how long it has been since the last time I wanted to add a pattern, at least five months. | 16:30 |
amrith | dhellmann, I think you've talked me out of this approach ;) | 16:32 |
amrith | I think it makes more sense to add the string into the list | 16:32 |
amrith | and wait for the next version of strutils | 16:32 |
dhellmann | amrith: ok :-) | 16:32 |
amrith | and if this turns out to be a *real* problem, then we can revisit with a *real* use-case. | 16:32 |
dhellmann | yeah, that sounds good | 16:32 |
amrith | Done, I'll repurpose the same patch and push a change for the pswd string. | 16:32 |
openstackgerrit | amrith proposed openstack/oslo.utils: Add a new string to the list of masked patterns https://review.openstack.org/165478 | 16:37 |
*** zzzeek has joined #openstack-oslo | 16:38 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 16:42 | |
*** ihrachyshka has quit IRC | 16:57 | |
*** openstackgerrit has quit IRC | 16:59 | |
*** openstackgerrit has joined #openstack-oslo | 16:59 | |
*** alexpilotti has quit IRC | 17:02 | |
*** alexpilotti_ has joined #openstack-oslo | 17:02 | |
*** kgiusti has quit IRC | 17:03 | |
*** achanda has joined #openstack-oslo | 17:04 | |
openstackgerrit | Monty Taylor proposed openstack-dev/pbr: Don't avoid egg-info quite so often https://review.openstack.org/165511 | 17:06 |
*** kgiusti has joined #openstack-oslo | 17:08 | |
*** e0ne is now known as e0ne_ | 17:10 | |
*** sigmavirus24_awa is now known as sigmavirus24 | 17:10 | |
*** e0ne_ is now known as e0ne | 17:13 | |
*** exploreshaifali has quit IRC | 17:15 | |
*** e0ne has quit IRC | 17:18 | |
*** salv-orl_ has quit IRC | 17:19 | |
*** salv-orlando has joined #openstack-oslo | 17:25 | |
*** harlowja_away is now known as harlowja_ | 17:27 | |
*** sreshetn1 has quit IRC | 17:28 | |
dims_ | sileht: still around? | 17:32 |
sileht | dims_, yes | 17:32 |
dims_ | please see latest run (email on its way) | 17:33 |
openstackgerrit | Joshua Harlow proposed openstack/taskflow: Always return scope walker instances from `fetch_scopes_for` https://review.openstack.org/165262 | 17:35 |
dims_ | sileht: i am inclined to push the series in and handle any issues as new reviews | 17:35 |
sileht | dims_, yes, it will be easier to review | 17:35 |
sileht | dims_, looks like a typo and a missing test to improve coverage | 17:37 |
dims_ | want to fix typo? i'll +2A after that | 17:37 |
*** sreshetn1 has joined #openstack-oslo | 17:38 | |
dims_ | self._log_publisher_send_error(msg_id, exc) - right? | 17:39 |
sileht | dims_, I have missed 'as exc' in the catch all except statements | 17:39 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: rabbit: Improves logging https://review.openstack.org/165336 | 17:40 |
openstackgerrit | Mehdi Abaakouk proposed openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 17:40 |
sileht | dims_, https://review.openstack.org/#/c/165336/4..5/oslo_messaging/_drivers/impl_rabbit.py | 17:40 |
sileht | dims_, what you kombu version and rabbit server version ? | 17:41 |
sileht | dims_, unfortunatly this typo hide the interesting error | 17:42 |
sileht | dims_, if you get 'exchange NOT found' that goods, if you get anything else that bad | 17:42 |
dims_ | server - {running_applications,[{rabbit,"RabbitMQ","3.2.4"}, | 17:43 |
dims_ | dims@dims-mac:~/openstack/oslo/oslo.messaging$ pip freeze | grep -i -e kombu -e amqp | 17:44 |
dims_ | amqp==1.4.6 | 17:44 |
dims_ | amqplib==1.0.2 | 17:44 |
dims_ | kombu==3.0.24 | 17:44 |
dims_ | so, can i pull the trigger on the first review? | 17:46 |
dims_ | sileht: | 17:46 |
sileht | dims_, if we are not sure about heartbeat stuff, we can disable it by default, but I'm confident | 17:47 |
sileht | lets operator play with it and report bugs | 17:48 |
*** ajo has quit IRC | 17:48 | |
*** ChuckC has quit IRC | 17:51 | |
*** ChuckC_ has joined #openstack-oslo | 17:51 | |
*** ChuckC_ is now known as ChuckC | 17:52 | |
dims_ | sileht: it's all in | 17:53 |
sileht | dims_, thanks a lot ! | 17:53 |
*** sheeprine has quit IRC | 17:53 | |
*** ndipanov has quit IRC | 17:55 | |
*** e0ne has joined #openstack-oslo | 17:59 | |
*** sheeprine_ has joined #openstack-oslo | 18:00 | |
*** exploreshaifali has joined #openstack-oslo | 18:00 | |
*** crc32 has joined #openstack-oslo | 18:00 | |
*** tsufiev_ has left #openstack-oslo | 18:01 | |
*** sheeprine_ has quit IRC | 18:07 | |
*** sheeprine has joined #openstack-oslo | 18:08 | |
*** sreshetn1 has quit IRC | 18:12 | |
openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.messaging: Publish tracebacks only on debug level https://review.openstack.org/165544 | 18:20 |
openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.messaging: Publish tracebacks only on debug level https://review.openstack.org/165544 | 18:22 |
dims_ | sileht: dhellmann: https://review.openstack.org/#/c/165544/ | 18:22 |
openstackgerrit | Merged openstack/oslo.messaging: rabbit: heartbeat implementation https://review.openstack.org/146047 | 18:44 |
*** rushiagr is now known as rushiagr_away | 18:46 | |
*** achanda has quit IRC | 18:48 | |
*** jaosorior has quit IRC | 18:52 | |
*** stevemar has quit IRC | 18:53 | |
*** stevemar has joined #openstack-oslo | 18:53 | |
*** achanda has joined #openstack-oslo | 19:01 | |
*** alexpilotti_ has quit IRC | 19:04 | |
*** liusheng has quit IRC | 19:05 | |
*** andreykurilin_ has joined #openstack-oslo | 19:06 | |
*** liusheng has joined #openstack-oslo | 19:06 | |
*** cburgess has quit IRC | 19:10 | |
*** cburgess has joined #openstack-oslo | 19:11 | |
*** amrith is now known as _amrith_ | 19:11 | |
*** exploreshaifali has quit IRC | 19:13 | |
*** _amrith_ is now known as amrith | 19:13 | |
openstackgerrit | Merged openstack/oslo.messaging: fix up verb tense in log message https://review.openstack.org/165462 | 19:24 |
*** salv-orlando has quit IRC | 19:24 | |
*** andreykurilin_ has quit IRC | 19:27 | |
*** salv-orlando has joined #openstack-oslo | 19:30 | |
*** boris-42 has quit IRC | 19:37 | |
*** lbragstad has quit IRC | 19:39 | |
*** lbragstad has joined #openstack-oslo | 19:40 | |
*** boris-42 has joined #openstack-oslo | 19:42 | |
*** crc32 has quit IRC | 19:53 | |
*** devlaps has joined #openstack-oslo | 19:55 | |
*** zigo__ has joined #openstack-oslo | 19:56 | |
*** zigo has quit IRC | 19:56 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 19:57 | |
*** subscope_ has joined #openstack-oslo | 19:58 | |
*** flwang has quit IRC | 19:59 | |
*** achanda has quit IRC | 20:00 | |
*** flwang has joined #openstack-oslo | 20:01 | |
*** openstackgerrit has quit IRC | 20:14 | |
*** openstackgerrit has joined #openstack-oslo | 20:15 | |
*** exploreshaifali has joined #openstack-oslo | 20:16 | |
*** kiseok7_ has quit IRC | 20:16 | |
*** sigmavirus24_awa is now known as sigmavirus24 | 20:22 | |
*** prad has quit IRC | 20:23 | |
*** ihrachyshka has joined #openstack-oslo | 20:26 | |
*** BrianShang has joined #openstack-oslo | 20:40 | |
*** BrianShang_ has quit IRC | 20:43 | |
*** achanda has joined #openstack-oslo | 20:48 | |
*** exploreshaifali has quit IRC | 20:56 | |
*** alexpilotti has joined #openstack-oslo | 20:56 | |
jungleboyj | dhellmann: ping | 21:01 |
*** spyyy has joined #openstack-oslo | 21:15 | |
*** kgiusti has left #openstack-oslo | 21:15 | |
*** ihrachyshka has quit IRC | 21:16 | |
bknudson | why no import_opt on http://docs.openstack.org/developer/oslo.config/cfg.html#module-oslo_config.cfg ? | 21:18 |
*** ChuckC has quit IRC | 21:19 | |
jungleboyj | bknudson: Good question. It is in the code. | 21:19 |
bknudson | I'll add it to my list o' things to look at. | 21:20 |
jungleboyj | bknudson: It is looking to me like it isn't able to import from libraries though. | 21:20 |
bknudson | jungleboyj: what does it do? | 21:21 |
bknudson | jungleboyj: do you have an example of a call? | 21:22 |
bknudson | http://git.openstack.org/cgit/openstack/oslo.config/tree/oslo_config/cfg.py#n2022 are the docs that don't show up in the generated docs. | 21:23 |
bknudson | there's also an import_group! | 21:23 |
jungleboyj | When I try to do CONF.import_opt('policy_file', 'oslo_policy') I get test cases that fail with 'NoSuchOptError: no such option: policy_file' | 21:23 |
jungleboyj | bknudson: ^ | 21:24 |
bknudson | oslo_policy is supposed to be the module that defines the option on import | 21:24 |
bknudson | on the global CONF object. | 21:24 |
jungleboyj | Right, policy_file is defined in oslo.policy | 21:24 |
jungleboyj | oslo_policy/opts.py | 21:24 |
bknudson | oslo_policy registers options when you call _register(conf) | 21:25 |
bknudson | actually, you would create an Enforcer with your conf: http://git.openstack.org/cgit/openstack/oslo.policy/tree/oslo_policy/policy.py#n300 | 21:25 |
bknudson | then that will call _register. | 21:25 |
bknudson | at which point your options are available. | 21:26 |
jungleboyj | Oh ... oh ... ok. | 21:26 |
jungleboyj | Doh. So the problem is that I am not first initializing oslo_policy so the option really isn't there yet. | 21:27 |
bknudson | oslo.policy could use better docs. | 21:27 |
spyyy | Hi there. Is there a way for oslo.messaging to specify a timeout for RPC call on a server side (using rabbit transport)? I want to handle the situation when a client stops waiting for the result (because of client timeout), but the server continue and hanging and working on the procedure. | 21:27 |
bknudson | in this area | 21:27 |
*** e0ne has quit IRC | 21:28 | |
dhellmann | jungleboyj: hi | 21:28 |
*** mriedem is now known as mriedem_away | 21:28 | |
jungleboyj | dhellmann: Hey. Was just trying to figure out how to use import_opt to set policy_file in olso_policy. | 21:29 |
dhellmann | jungleboyj: you want to set the option value from the outside? | 21:30 |
dhellmann | jungleboyj: see oslo_policy.opts.set_defaults() | 21:30 |
*** jamielennox is now known as jamielennox|away | 21:31 | |
dhellmann | jungleboyj: keep in mind, options are subject to move and aren't part of the API, so if you modify the config directly your code may break later if we change that option definition | 21:31 |
jungleboyj | For our test cases: https://github.com/openstack/cinder/blob/master/cinder/tests/conf_fixture.py#L24 | 21:32 |
*** stpierre has quit IRC | 21:32 | |
jungleboyj | dhellmann: Oh, I vaguely remember using set_defaults somewhere else. | 21:33 |
bknudson | jungleboyj: you should be able to call oslo_policy.opts.set_defaults(policy_file='cinder.policy') there. | 21:33 |
dhellmann | jungleboyj: right. You want to combine oslo_config.fixture.Config() to clean up a config object, with oslo_policy.opts.set_defaults() to change the default | 21:33 |
bknudson | oslo_policy.opts.set_defaults(CONF, policy_file='cinder.policy') | 21:34 |
dhellmann | bknudson: right | 21:34 |
*** ihrachyshka has joined #openstack-oslo | 21:34 | |
jungleboyj | dhellmann bknudson Sweet. Let me try. | 21:34 |
dhellmann | jungleboyj: although I don't know if that reset does anything to reset the default value of the option | 21:35 |
* dhellmann looks at code | 21:35 | |
dhellmann | oh, good, it does | 21:35 |
jungleboyj | Hey, I think I am making progress. | 21:43 |
dhellmann | dims_, sileht : should we be planning to backport these heartbeat changes to the kilo stable branch? | 21:43 |
*** devlaps has quit IRC | 21:43 | |
*** flaper87 has quit IRC | 21:50 | |
stevemar | i need a notification for oslo.policy mentions | 21:50 |
jungleboyj | stevemar: Oh, you are the policy man? | 21:51 |
stevemar | jungleboyj, maybe, whats it to you? | 21:51 |
openstackgerrit | Merged openstack/oslo.messaging: rabbit: Improves logging https://review.openstack.org/165336 | 21:51 |
jungleboyj | stevemar: Trying to get Cinder moved from incubator to oslo_policy. | 21:51 |
*** sigmavirus24 is now known as sigmavirus24_awa | 21:51 | |
*** yamahata has quit IRC | 21:52 | |
stevemar | jungleboyj, dhellmann and sigmavirus24_awa know stuff too | 21:52 |
jungleboyj | :-) | 21:52 |
*** yamahata has joined #openstack-oslo | 21:52 | |
stevemar | jungleboyj, heres the glance patch: https://review.openstack.org/#/c/162368/ | 21:53 |
* dhellmann disavows any knowledge of anything | 21:53 | |
*** flaper87 has joined #openstack-oslo | 21:53 | |
stevemar | jungleboyj, and the keystone patch: https://review.openstack.org/#/c/148624/ | 21:53 |
stevemar | dhellmann, that's a good strategy, but you're too well known | 21:53 |
* dhellmann makes a note to buy a new irc nick | 21:54 | |
stevemar | my favorite phrase at work it 'i know nothing' | 21:54 |
stevemar | is* | 21:54 |
openstackgerrit | Joshua Harlow proposed openstack/taskflow: Just let the future executors handle the max workers https://review.openstack.org/164352 | 21:57 |
*** boris-42 has quit IRC | 22:02 | |
*** bknudson has quit IRC | 22:09 | |
*** jamielennox|away is now known as jamielennox | 22:12 | |
*** sputnik13 has quit IRC | 22:13 | |
*** sputnik13 has joined #openstack-oslo | 22:14 | |
*** stevemar has quit IRC | 22:15 | |
*** shardy is now known as shardy_z | 22:20 | |
openstackgerrit | Merged openstack/oslo.messaging: cleanup connection pool return https://review.openstack.org/165337 | 22:29 |
*** andreykurilin_ has joined #openstack-oslo | 22:48 | |
*** jecarey has quit IRC | 22:51 | |
dims_ | harlowja_: here's the issue...if i do LOG.exception we print the message and the traceback. I would like to sqaush the traceback. i can do this by using LOG.error. however *if* needed someone like an operator or developer can turn on debug which would print the stack traces too. i know it's convoluted but the thing here is we need to find a balance between what we log and what we don't. one stupid traceback and there ar | 22:56 |
dims_ | e 10 people screaming at you to look see what failed :) | 22:56 |
harlowja_ | dims_ all i was saying is that i think the exc_info should either be a tuple (from sys.exc_info or equivalent) or true :-P | 22:56 |
dims_ | dhellmann: ack on the error vs warning will respin in a bit. thanks. | 22:56 |
harlowja_ | instead it appears to be the exception object itself | 22:57 |
dims_ | harlowja_: aha :) | 22:57 |
dims_ | will fix. | 22:57 |
harlowja_ | * which is not in the set of exc_info tupe, boolean... that i think the function expects | 22:57 |
harlowja_ | thats all | 22:57 |
harlowja_ | k | 22:57 |
dims_ | thanks :) | 22:58 |
harlowja_ | np | 22:58 |
harlowja_ | all logger.exception does is the following | 22:58 |
harlowja_ | https://hg.python.org/releasing/2.7.9/file/753a8f457ddc/Lib/logging/__init__.py#l1188 | 22:58 |
harlowja_ | kwargs['exc_info'] = 1 | 22:58 |
harlowja_ | self.error(msg, *args, **kwargs) | 22:58 |
harlowja_ | lol | 22:58 |
*** gordc has quit IRC | 22:58 | |
* harlowja_ thought thats funny | 22:59 | |
harlowja_ | ha | 22:59 |
dims_ | haha | 22:59 |
*** zzzeek has quit IRC | 22:59 | |
*** ihrachyshka has quit IRC | 23:03 | |
*** subscope_ has quit IRC | 23:07 | |
openstackgerrit | Joshua Harlow proposed openstack/taskflow: Avoid the writing to special task detail https://review.openstack.org/165645 | 23:13 |
*** sputnik13 has quit IRC | 23:14 | |
*** sputnik13 has joined #openstack-oslo | 23:21 | |
*** sputnik13 has quit IRC | 23:24 | |
*** andreykurilin_ has quit IRC | 23:25 | |
openstackgerrit | Clint 'SpamapS' Byrum proposed openstack/oslo.messaging: Add a gearman/gearstore/gearhorn driver https://review.openstack.org/165646 | 23:25 |
SpamapS | morganfainberg: ^^ I _might_ have gotten obssessed and forgotten that you did some work there already.. | 23:26 |
*** sputnik13 has joined #openstack-oslo | 23:26 | |
morganfainberg | SpamapS: ahh no worries. I've been stupid busy. | 23:26 |
morganfainberg | Happy to see it. | 23:27 |
SpamapS | morganfainberg: note that it relies heavily on the other things I've been poking at, gearstore and gearhorn | 23:27 |
SpamapS | I should be working on nodepool testing | 23:27 |
SpamapS | but this is just ... captivating ;) | 23:27 |
harlowja_ | put that into kombu so the rest of the world can use it :-P | 23:28 |
SpamapS | harlowja_: kombu is actually too demanding. I looked at that. | 23:29 |
harlowja_ | what means demanding, lol | 23:29 |
SpamapS | harlowja_: I'd have to implement even more corners | 23:29 |
SpamapS | also AFAICT, there is no oslo.messaging "kombu" driver | 23:29 |
SpamapS | so i'd have to write two things | 23:29 |
*** dims__ has joined #openstack-oslo | 23:29 | |
harlowja_ | replace impl_rabbit with impl_kombu, lol | 23:30 |
SpamapS | nothankyou | 23:30 |
SpamapS | all the amqp crapola .. gah | 23:30 |
*** boris-42 has joined #openstack-oslo | 23:30 | |
*** sputnik13 has quit IRC | 23:31 | |
*** dims_ has quit IRC | 23:33 | |
harlowja_ | hmmm, ok, me still thinks that u could just have something like https://github.com/celery/kombu/blob/master/kombu/transport/memory.py (which is not doing that much) and u could then have a working transport for kombu | 23:33 |
harlowja_ | but to each there own | 23:33 |
SpamapS | harlowja_: btw, the kombu base class is HORRIBLY documented, which is another reason I ran for the hills. | 23:34 |
harlowja_ | cough cough, #celery channel (asksol author is in there) | 23:35 |
SpamapS | harlowja_: but really, biggest problem is there's just no impl_kombu in oslo.messaging, so it doubles the amount of code I have to do. :-/ | 23:35 |
harlowja_ | ya, thats a different conundrum | 23:35 |
harlowja_ | since the impl_rabbit in oslo.messaging is like some weird hybrid of kombu, rabbit... | 23:36 |
harlowja_ | and likely with some massaging impl_rabbit could turn into impl_kombu (and maybe it should?) | 23:36 |
harlowja_ | and then the impl_qpid maybe can go away also | 23:36 |
harlowja_ | tech debt fun, lol | 23:37 |
SpamapS | yeah there's a mountain of it | 23:37 |
SpamapS | harlowja_: honestly, if you show me a "how to write a driver for kombu" page, I'll poke at it next. | 23:37 |
*** sputnik13 has joined #openstack-oslo | 23:38 | |
*** sputnik13 has quit IRC | 23:38 | |
SpamapS | harlowja_: but really, if you are going to suggest we do kombu drivers.. first step is getting oslo.messaging to be a thin layer in front of kombu. :) | 23:38 |
SpamapS | OR aboloshing oslo.messaging. | 23:38 |
harlowja_ | http://paste.ubuntu.com/10624084/ | 23:39 |
harlowja_ | lol | 23:39 |
harlowja_ | there i make u a page | 23:39 |
SpamapS | aw snap there it is ok I did all the steps now what? | 23:39 |
SpamapS | it's failing travis-ci | 23:39 |
harlowja_ | lol | 23:39 |
harlowja_ | https://github.com/celery/kombu/issues/430 | 23:39 |
harlowja_ | that might intersest u | 23:39 |
harlowja_ | typically i jump on #celery and bug asksol if i have quesetions, i think the docs could be improved | 23:40 |
harlowja_ | but idk asksol's priorities/job/anything, lol | 23:40 |
harlowja_ | * https://github.com/ask (that guy) | 23:40 |
klindgren_ | aren't the rabbit people saying dont use kombu/celery anymore but use some other driver? | 23:40 |
harlowja_ | they somehow told the stacktach people to use a driver thats not maintained afaik | 23:41 |
harlowja_ | seems questionable, lol | 23:41 |
harlowja_ | oh, nm, seems somewhat upated | 23:41 |
harlowja_ | https://pypi.python.org/pypi/pika | 23:41 |
harlowja_ | * https://github.com/pika/pika | 23:42 |
harlowja_ | and maybe we really should just use that, how many people really run things other than rabbitmq? | 23:43 |
harlowja_ | idk | 23:43 |
*** e0ne has joined #openstack-oslo | 23:43 | |
harlowja_ | if 99% of the people run rabbitmq, then imho use pika, stop kidding ourselves, move on... | 23:43 |
harlowja_ | *of the operators/deployers... | 23:44 |
SpamapS | harlowja_: the point of what I'm doing is to replace rabbitmq because honestly it is _awful_ to administer. | 23:44 |
harlowja_ | understandable, i haven't used gearman much, but i know others at y! have | 23:44 |
harlowja_ | although i will sorta stick to my guns, and say that if 99% of the operators are using it, and have gotten used to its quirks, then meh, the grass on the other side of the fence is always less awful :-P | 23:46 |
harlowja_ | until u get there and its brown, lol | 23:46 |
* harlowja_ thinks postively! | 23:46 | |
harlowja_ | lol | 23:46 |
openstackgerrit | Joshua Harlow proposed openstack/taskflow: By default use a in memory backend (when none is provided) https://review.openstack.org/165654 | 23:46 |
SpamapS | harlowja_: the feedback from the ops meetup is that everybody is spending a ton of time dealing with those quirks. | 23:46 |
harlowja_ | u trying to take the operators jerb, lol | 23:47 |
SpamapS | harlowja_: meanwhile in infra we've been using gearman for messaging for quite a while and are very comfortable running our cloud with it. | 23:47 |
harlowja_ | taking there jerb | 23:47 |
SpamapS | tryin'na take mah jerrb | 23:47 |
harlowja_ | lol | 23:47 |
lifeless | mmm | 23:49 |
lifeless | so the operators use rabbit because we built this city on rock and roll | 23:49 |
harlowja_ | jimmy hendrix stack | 23:49 |
harlowja_ | lol | 23:49 |
lifeless | if we want to pick better plumbing, lets do it. gear isn't appreciably better in the ways that matter /here/ AFAICT. | 23:49 |
lifeless | we can't assess whats better on the basis of whats deployed, rather on the basis of how well it matches what deployers need. | 23:50 |
SpamapS | lifeless: gear is dead dumbass simple which is one reason it's my preference. | 23:50 |
lifeless | E.g. reliability. Low operational headaches. Efficiency. | 23:50 |
lifeless | SpamapS: I'm not dissing gear. | 23:50 |
harlowja_ | lifeless sure, its a tricky situation, to move the needle, to maybe something better, always a risk | 23:50 |
lifeless | SpamapS: RPC over mq in general is just daft. I think replacing rabbit with gear is a forest-for-trees situation | 23:50 |
harlowja_ | although me personally, would just like stuff to all be http, ha | 23:50 |
harlowja_ | but maybe thats cause i'm brain washed at yahoo where most stuff is | 23:51 |
harlowja_ | ^ totally possible, lol | 23:51 |
lifeless | SpamapS: gear is lovely to work with, and as its actually an RPC system, its a better fit for RPC problems than rabbit | 23:51 |
SpamapS | rabbitmq problems: clustering loses messages on failover, connection heartbeats aren't reliable enough to keep from causing spurious errors. | 23:51 |
openstackgerrit | Davanum Srinivas (dims) proposed openstack/oslo.messaging: Publish tracebacks only on debug level https://review.openstack.org/165544 | 23:51 |
lifeless | SpamapS: but (IIRC) its synchronous on each channel, which may have significant implications. Contrast that with e.g. grpc | 23:51 |
lifeless | SpamapS: yes, rabbits terrible. | 23:52 |
SpamapS | lifeless: no, gear is sync or async | 23:52 |
SpamapS | lifeless: background=(True|False) | 23:52 |
lifeless | SpamapS: ok, good. | 23:52 |
lifeless | SpamapS: AIUI you're hacking on a fan-out approximation though | 23:52 |
lifeless | SpamapS: As a thunk thats reasonable, but we should (IMO) be focusing on an actual RPC model | 23:53 |
SpamapS | lifeless: I did create one yes. https://github.com/SpamapS/gearhorn | 23:53 |
lifeless | rather than RPC on MQ model on RPC system on MQ emulator | 23:53 |
lifeless | :) | 23:53 |
SpamapS | lifeless: and a durable worker too https://github.com/SpamapS/gearstore | 23:53 |
lifeless | I have to take C to gym | 23:53 |
harlowja_ | whose gonna take A and B ? | 23:53 |
SpamapS | we MUST, I think, have more ways to define which things need durable messaging vs. which don't in oslo.messaging btw | 23:53 |
lifeless | don't have an A or B | 23:53 |
SpamapS | 90% of the time you probably don't need durable. | 23:54 |
lifeless | SpamapS: right,a nd you don't for RPC :) | 23:54 |
lifeless | SpamapS: so I guess I'm really saying, I think the direct path (targeted RPC) is more valuable than a gear backend for oslo.messaging. | 23:54 |
lifeless | would love to chat more, but later. | 23:54 |
SpamapS | lifeless: yeah, as long as you're gonna hang around and wait for replies, no sense making things durable. | 23:54 |
lifeless | SpamapS: (and targeted RPC with gear is entirely doable and makes sense) | 23:54 |
SpamapS | lifeless: I am going to have to deploy a cloud soon, so I am trying to not have to deploy RabbitMQ. :) | 23:55 |
SpamapS | stripping away all the wasteful durable messaging will be an optimization at that point. :) | 23:55 |
SpamapS | the fanouts are actually pretty performant. | 23:55 |
SpamapS | and I don't see them as a reliability problem in most cases. | 23:56 |
SpamapS | the main place that uses fanout is neutron dhcp | 23:58 |
SpamapS | and there are some other bits like compute sending updates about available resources | 23:58 |
SpamapS | neither of those have many subscribers | 23:58 |
SpamapS | anyway, I have to run too | 23:59 |
* SpamapS disappears | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!