| *** gcb has joined #openstack-oslo | 00:15 | |
| *** zhangguoqing has joined #openstack-oslo | 00:28 | |
| *** dave-mccowan has joined #openstack-oslo | 01:06 | |
| *** harlowja has quit IRC | 01:13 | |
| *** yamamoto has joined #openstack-oslo | 02:17 | |
| *** yamamoto has quit IRC | 02:38 | |
| *** yamamoto has joined #openstack-oslo | 02:48 | |
| *** chhavi has joined #openstack-oslo | 03:37 | |
| *** chhavi has quit IRC | 03:53 | |
| *** hieulq has quit IRC | 03:57 | |
| *** hieulq has joined #openstack-oslo | 03:57 | |
| *** yamamoto has quit IRC | 03:57 | |
| *** links has joined #openstack-oslo | 03:58 | |
| *** yamamoto has joined #openstack-oslo | 03:59 | |
| *** dave-mccowan has quit IRC | 04:01 | |
| *** yamamoto has quit IRC | 04:04 | |
| *** yamamoto has joined #openstack-oslo | 04:05 | |
| *** zhangguoqing has quit IRC | 04:07 | |
| *** zhangguoqing has joined #openstack-oslo | 04:08 | |
| *** Dinesh_Bhor has joined #openstack-oslo | 04:11 | |
| *** zhangguoqing has quit IRC | 04:14 | |
| *** yamamoto has quit IRC | 04:14 | |
| *** Dinesh_Bhor has quit IRC | 04:14 | |
| *** Dinesh_Bhor has joined #openstack-oslo | 04:15 | |
| *** yamamoto has joined #openstack-oslo | 04:15 | |
| *** yamamoto has quit IRC | 04:21 | |
| *** harlowja has joined #openstack-oslo | 04:31 | |
| *** yamamoto has joined #openstack-oslo | 04:44 | |
| *** harlowja has quit IRC | 04:47 | |
| *** lpetrut__ has joined #openstack-oslo | 04:51 | |
| *** zhangguoqing has joined #openstack-oslo | 05:03 | |
| *** nicolasbock has joined #openstack-oslo | 05:11 | |
| openstackgerrit | ChangBo Guo(gcb) proposed openstack/tooz master: Revert "Make sure Lock.heartbeat() returns True/False" https://review.openstack.org/485035 | 05:14 |
|---|---|---|
| openstackgerrit | ChangBo Guo(gcb) proposed openstack/tooz master: Revert "etcd3: replace custom lock code by more recent etcd3 lock code" https://review.openstack.org/485036 | 05:15 |
| *** zhangguoqing has quit IRC | 05:15 | |
| *** Dinesh_Bhor has quit IRC | 05:17 | |
| *** zhangguoqing has joined #openstack-oslo | 05:21 | |
| *** zhangguoqing has quit IRC | 05:27 | |
| *** zhangguoqing has joined #openstack-oslo | 05:39 | |
| *** zhangguoqing has quit IRC | 05:39 | |
| *** harlowja has joined #openstack-oslo | 05:46 | |
| *** rcernin has joined #openstack-oslo | 05:50 | |
| *** lpetrut__ has quit IRC | 06:04 | |
| openstackgerrit | Merged openstack/oslo.config master: fix an issue with looking up deprecated option names in code https://review.openstack.org/484897 | 06:13 |
| openstackgerrit | liuyamin proposed openstack/oslo.db master: Replace six.iteritems() with .items() https://review.openstack.org/485055 | 06:15 |
| *** nicolasbock has quit IRC | 06:23 | |
| *** nicolasbock has joined #openstack-oslo | 06:25 | |
| *** nicolasbock has quit IRC | 06:37 | |
| *** nicolasbock has joined #openstack-oslo | 06:39 | |
| *** harlowja has quit IRC | 06:42 | |
| *** Dinesh_Bhor has joined #openstack-oslo | 06:50 | |
| *** Dinesh_Bhor has quit IRC | 06:50 | |
| *** lpetrut has joined #openstack-oslo | 07:06 | |
| *** zhangguoqing has joined #openstack-oslo | 07:07 | |
| *** shardy has joined #openstack-oslo | 07:11 | |
| *** zhangguoqing has quit IRC | 07:11 | |
| *** zhangguoqing has joined #openstack-oslo | 07:12 | |
| *** zhangguoqing has quit IRC | 07:15 | |
| *** zhangguoqing has joined #openstack-oslo | 07:15 | |
| *** tesseract has joined #openstack-oslo | 07:17 | |
| openstackgerrit | Tovin Seven proposed openstack/osprofiler master: [WiP] OSprofiler with OpenTracing compatible https://review.openstack.org/480018 | 07:18 |
| openstackgerrit | Tovin Seven proposed openstack/osprofiler master: [WiP] OSprofiler with OpenTracing compatible https://review.openstack.org/480018 | 07:21 |
| *** zhangguoqing has quit IRC | 07:32 | |
| *** ralonsoh has joined #openstack-oslo | 07:46 | |
| *** zhangguoqing has joined #openstack-oslo | 07:47 | |
| *** lpetrut has quit IRC | 07:55 | |
| *** lucas-afk is now known as lucasagomes | 08:18 | |
| *** shardy is now known as shardy_afk | 08:21 | |
| *** aselius has quit IRC | 08:32 | |
| *** ralonsoh has quit IRC | 08:33 | |
| *** ralonsoh_ has joined #openstack-oslo | 08:33 | |
| sileht | /1/29 | 08:33 |
| *** shardy_afk is now known as shardy | 08:36 | |
| *** openstackgerrit has quit IRC | 08:49 | |
| *** yamamoto has quit IRC | 09:06 | |
| *** pblaho has joined #openstack-oslo | 09:07 | |
| *** yamamoto has joined #openstack-oslo | 09:10 | |
| *** yamamoto has quit IRC | 09:10 | |
| *** yamamoto has joined #openstack-oslo | 09:14 | |
| *** zhangguoqing has quit IRC | 09:17 | |
| *** zhangguoqing has joined #openstack-oslo | 09:18 | |
| *** yamamoto has quit IRC | 09:36 | |
| *** yamamoto has joined #openstack-oslo | 09:36 | |
| *** sdague has joined #openstack-oslo | 09:37 | |
| *** sambetts|afk is now known as sambetts | 09:42 | |
| *** lpetrut has joined #openstack-oslo | 09:50 | |
| *** zhangguoqing has quit IRC | 10:00 | |
| *** Dinesh_Bhor has joined #openstack-oslo | 10:27 | |
| *** boden has joined #openstack-oslo | 10:38 | |
| *** lpetrut_ has joined #openstack-oslo | 10:54 | |
| *** yamamoto has quit IRC | 10:55 | |
| *** gcb has quit IRC | 10:55 | |
| *** lpetrut has quit IRC | 10:56 | |
| *** gcb has joined #openstack-oslo | 10:57 | |
| *** zhangguoqing has joined #openstack-oslo | 11:12 | |
| *** zhangguoqing has quit IRC | 11:15 | |
| *** zhangguoqing has joined #openstack-oslo | 11:19 | |
| *** zhangguo_ has joined #openstack-oslo | 11:25 | |
| *** dave-mccowan has joined #openstack-oslo | 11:26 | |
| *** zhangguoqing has quit IRC | 11:29 | |
| *** yamamoto has joined #openstack-oslo | 11:40 | |
| *** ansmith has quit IRC | 11:56 | |
| *** kgiusti has joined #openstack-oslo | 11:58 | |
| *** raildo has joined #openstack-oslo | 12:01 | |
| *** yamamoto has quit IRC | 12:03 | |
| *** lucasagomes is now known as lucas-hungry | 12:04 | |
| *** hoonetorg has joined #openstack-oslo | 12:09 | |
| *** yamamoto has joined #openstack-oslo | 12:21 | |
| *** ralonsoh_ has quit IRC | 12:32 | |
| *** MVenesio has joined #openstack-oslo | 12:37 | |
| *** ansmith has joined #openstack-oslo | 12:48 | |
| *** gcb has quit IRC | 12:54 | |
| *** zhangguo_ has quit IRC | 13:01 | |
| *** ralonsoh has joined #openstack-oslo | 13:19 | |
| *** ralonsoh has quit IRC | 13:20 | |
| *** ralonsoh has joined #openstack-oslo | 13:20 | |
| *** bknudson has joined #openstack-oslo | 13:28 | |
| *** bknudson has quit IRC | 13:29 | |
| *** zhangguoqing has joined #openstack-oslo | 13:31 | |
| *** bknudson has joined #openstack-oslo | 13:33 | |
| *** zhangguoqing has quit IRC | 13:41 | |
| *** janzian has joined #openstack-oslo | 13:42 | |
| jd_ | sileht: dims: any of you took a look at the tooz/Cinder issue? Another pair of eyes could be useful | 13:55 |
| jd_ | dims: I know you mostly copy/pasted the etcd3gw driver but still… :) | 13:55 |
| dims | jd_ : y it's on my TODO, i did spend some time yesterday but could not figure out. will look again. my finger was on this change in etcd3gw - https://review.openstack.org/#/c/478202/2/tooz/drivers/etcd3gw.py | 13:57 |
| jd_ | dims: yeah definitely I think that's the one | 13:59 |
| dims | jd_ : do you expect things to fail if we drop the _exclusive_access lock in there? | 13:59 |
| jd_ | dims: oh yeah. | 13:59 |
| dims | keep the True/False, but drop the lock | 13:59 |
| jd_ | dims: heartbeat will fail renewing the lock because of a race condition with release() and the heartbeat thread | 14:00 |
| jd_ | dims: we've the same mechanism in Redis that we used for months/year in telemetry | 14:00 |
| jd_ | and I fixed it a few months ago, it's working AFAIK | 14:01 |
| dims | jd_ : if we don't do anything the current tooz version in g-r/u-c still has that problem | 14:01 |
| jd_ | dims: yeah I know | 14:02 |
| jd_ | that's just hiding problems under the carpet | 14:02 |
| dims | we can't make any more tooz releases unless we figure out why the _exclusive_access fails intermittently or we yank that code for now | 14:02 |
| dims | right | 14:02 |
| jd_ | it only fails with Cinder / eventlet | 14:02 |
| dims | jd_ : no one else is using it yet just cinder | 14:03 |
| dims | i mean etcd3gw | 14:03 |
| jd_ | dims: … telemetry? | 14:03 |
| dims | i meant the etcd3gw :) | 14:03 |
| dims | you are not using that...are you? | 14:03 |
| jd_ | not in the gate but it's possible operators deploy with it | 14:03 |
| dims | ack jd_ | 14:04 |
| dims | jd_ : did any of the tracebacks show a need to use RLock(s)? | 14:08 |
| *** yamamoto has quit IRC | 14:09 | |
| jd_ | dims: you mean RLock rather than Lock? | 14:09 |
| dims | y | 14:09 |
| *** alegacy has quit IRC | 14:10 | |
| jd_ | ah good question, I don't think any traceback showed that | 14:10 |
| jd_ | and we only used Locks in Redis without having any problem | 14:10 |
| *** alegacy has joined #openstack-oslo | 14:11 | |
| dims | ack. back in a bit jd_ | 14:11 |
| jd_ | dims: so no a RLock would make sense if you were trying to hearbeat() and release() from the same thread, but that's not what happens here | 14:16 |
| *** links has quit IRC | 14:18 | |
| *** gordc has joined #openstack-oslo | 14:35 | |
| *** lucas-hungry is now known as lucasagomes | 14:47 | |
| *** aselius has joined #openstack-oslo | 14:59 | |
| sileht | jd_, dims dhellmann: https://bugs.launchpad.net/cinder/+bug/1694509/comments/19 | 15:01 |
| openstack | Launchpad bug 1694509 in tooz "eventlet errors and cannot switch to a different thread" [High,Confirmed] | 15:01 |
| *** hoonetorg has quit IRC | 15:02 | |
| *** yamamoto has joined #openstack-oslo | 15:09 | |
| *** yamamoto has quit IRC | 15:19 | |
| *** gcb has joined #openstack-oslo | 15:32 | |
| *** rajathagasthya has joined #openstack-oslo | 15:46 | |
| *** Rockyg has joined #openstack-oslo | 15:52 | |
| *** gcb has quit IRC | 15:54 | |
| *** gcb has joined #openstack-oslo | 15:55 | |
| *** aarefiev is now known as aarefiev_afk | 16:04 | |
| gcb | dims, dhellmann, jd_, sileht: thanks for working together to address Cinder's issue | 16:07 |
| dims | w00t! | 16:07 |
| jd_ | gcb: are we sure it works? | 16:08 |
| *** janzian has quit IRC | 16:08 | |
| *** rcernin has quit IRC | 16:09 | |
| gcb | jd_: that's the best solution for me :-), sileht pointed out the root cause, that's one key step to fix the issue | 16:09 |
| gcb | jd_: let's wait for the test result | 16:10 |
| *** rcernin has joined #openstack-oslo | 16:23 | |
| *** lucasagomes is now known as lucas-afk | 16:24 | |
| *** janzian has joined #openstack-oslo | 16:24 | |
| *** gcb has quit IRC | 16:50 | |
| *** harlowja has joined #openstack-oslo | 17:25 | |
| *** harlowja has quit IRC | 17:25 | |
| *** harlowja has joined #openstack-oslo | 17:26 | |
| *** lpetrut_ has quit IRC | 17:33 | |
| *** ralonsoh has quit IRC | 17:42 | |
| *** tesseract has quit IRC | 17:46 | |
| *** Rockyg has quit IRC | 17:54 | |
| *** bknudson1 has joined #openstack-oslo | 18:00 | |
| *** bknudson has quit IRC | 18:02 | |
| *** jd_ has left #openstack-oslo | 18:02 | |
| *** boris-42___ has joined #openstack-oslo | 18:17 | |
| *** u_nuSLASHkm8 has joined #openstack-oslo | 18:26 | |
| *** u_nuSLASHkm8 has left #openstack-oslo | 18:27 | |
| *** bknudson1 has quit IRC | 18:52 | |
| *** emccormick has joined #openstack-oslo | 18:54 | |
| *** bknudson has joined #openstack-oslo | 18:55 | |
| emccormick | Hi everyone! Does anybody know how one would go about modifying prlimits in oslo-concurency? I"m having some trouble spawning instnaces from a very large image and it seems to be related. | 18:56 |
| emccormick | thinking that cpu-time needs increasing from 8 for this particular situation, but can't quite see where to do it. | 18:56 |
| *** rajathagasthya has quit IRC | 19:03 | |
| emccormick | Looks like it's nova that actually sets it. Sorry :) | 19:27 |
| *** emccormick has left #openstack-oslo | 19:27 | |
| *** bknudson has quit IRC | 19:33 | |
| *** tinyurl_comSLASH has joined #openstack-oslo | 19:38 | |
| *** tinyurl_comSLASH has left #openstack-oslo | 19:41 | |
| *** rajathagasthya has joined #openstack-oslo | 19:46 | |
| *** lpetrut_ has joined #openstack-oslo | 19:52 | |
| *** jdwidari has joined #openstack-oslo | 20:02 | |
| *** jdwidari has quit IRC | 20:03 | |
| *** nicolasbock has quit IRC | 20:06 | |
| *** ujjain has quit IRC | 20:06 | |
| *** rajathagasthya has quit IRC | 20:07 | |
| *** ujjain has joined #openstack-oslo | 20:20 | |
| *** ujjain has quit IRC | 20:20 | |
| *** ujjain has joined #openstack-oslo | 20:20 | |
| *** raildo has quit IRC | 20:21 | |
| *** rajathagasthya has joined #openstack-oslo | 20:27 | |
| *** kgiusti has left #openstack-oslo | 20:31 | |
| *** ansmith has quit IRC | 20:36 | |
| *** MVenesio has quit IRC | 20:41 | |
| kevinbenton | harlowja: yo, oslo messaging question for you | 20:44 |
| harlowja | sup | 20:44 |
| kevinbenton | harlowja: http://logs.openstack.org/85/480185/4/check/gate-grenade-dsvm-ironic-inspector-ubuntu-xenial/6ef1e75/logs/new/screen-q-svc.txt.gz?level=ERROR#_2017-07-19_12_53_37_858 | 20:44 |
| kevinbenton | harlowja: NotFound: Basic.publish: (404) NOT_FOUND - no exchange 'q-agent-notifier-port-delete_fanout' in vhost '/' | 20:44 |
| kevinbenton | so that happens after we have no more subscribers | 20:45 |
| kevinbenton | and I want to know if there is a way to just ignore it | 20:45 |
| kevinbenton | it seems to ignore the condition where there were no subscribers to begine with | 20:45 |
| kevinbenton | just fine | 20:45 |
| kevinbenton | but after upgrade we have nobody listening on that legacy topic anymore | 20:45 |
| kevinbenton | so it starts to throw those errors | 20:46 |
| harlowja | hmmm | 20:46 |
| harlowja | it looks like something tried to publish to it | 20:47 |
| harlowja | ya? | 20:47 |
| kevinbenton | harlowja: yes | 20:47 |
| kevinbenton | after port delete event the server sends out a message to it that the agents picked up | 20:48 |
| kevinbenton | the new Pike agents no longer use that | 20:48 |
| harlowja | try: catch NotFound: pass :-P | 20:48 |
| harlowja | i guess u want something lower? | 20:48 |
| harlowja | (lower in the stack?) | 20:48 |
| kevinbenton | well that works for an individual one, but there are quite a few | 20:48 |
| kevinbenton | i was thinking something to be set on the notifier option | 20:49 |
| kevinbenton | one sec | 20:49 |
| harlowja | kk | 20:49 |
| harlowja | i may be multi-tasking (badly) so might be slow | 20:49 |
| harlowja | lol | 20:49 |
| harlowja | to many chats | 20:49 |
| harlowja | lol | 20:49 |
| harlowja | ETOOMANYCHAT | 20:49 |
| harlowja | lol | 20:49 |
| harlowja | bufferoverflow | 20:50 |
| harlowja | lol | 20:50 |
| kevinbenton | harlowja: :) | 20:50 |
| kevinbenton | harlowja: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/rpc.py#L394 | 20:50 |
| kevinbenton | basically something to put there to make it ignore casts if the exchange doesn't exist | 20:50 |
| *** sambetts is now known as sambetts|afk | 21:13 | |
| *** shardy has quit IRC | 21:13 | |
| harlowja | right, seems like a decent idear | 21:19 |
| kevinbenton | harlowja: what's not clear to me is why it doesn't throw exceptions when there are no agents to begin with | 21:27 |
| *** ihrachys has quit IRC | 21:29 | |
| *** ansmith has joined #openstack-oslo | 21:36 | |
| harlowja | :-/ | 21:38 |
| kevinbenton | sileht: is a notfound exception supposed to be something propagated back to an olso messaging user? | 21:39 |
| kevinbenton | sileht: http://logs.openstack.org/85/480185/4/check/gate-grenade-dsvm-ironic-inspector-ubuntu-xenial/6ef1e75/logs/new/screen-q-svc.txt.gz?level=ERROR#_2017-07-19_12_53_37_858 | 21:39 |
| kevinbenton | on a missing exchange? | 21:39 |
| *** janzian has quit IRC | 21:40 | |
| *** boris-42___ has quit IRC | 21:41 | |
| kevinbenton | harlowja: i'm not even sure we can catch that exception without import amqp directly, which isn't in our requirements.txt | 21:42 |
| harlowja | ya, let's see if sileht has any ideas, it does seem odd | 21:42 |
| kevinbenton | harlowja, sileht: https://bugs.launchpad.net/oslo.messaging/+bug/1705351 | 21:50 |
| openstack | Launchpad bug 1705351 in neutron "agent notifier getting amqp NotFound exceptions propagated up to it" [High,Confirmed] | 21:50 |
| *** boden has quit IRC | 22:03 | |
| *** lpetrut_ has quit IRC | 22:04 | |
| *** gordc has quit IRC | 22:16 | |
| *** sdague has quit IRC | 22:33 | |
| *** feefifofum has joined #openstack-oslo | 22:53 | |
| feefifofum | hello | 22:53 |
| feefifofum | i'm trying to write a script that imports configs from the various openstack services | 22:54 |
| feefifofum | one of the issue i'm running into is global state with oslo_config | 22:54 |
| feefifofum | i'm trying to clear oslo_config.CONF between reads, but it seems impossible to completely clear it out | 22:55 |
| feefifofum | any suggestions on what to do? | 22:57 |
| *** yamamoto has joined #openstack-oslo | 23:04 | |
| *** yamamoto has quit IRC | 23:18 | |
| *** rcernin has quit IRC | 23:34 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!