| *** rcernin has quit IRC | 00:17 | |
| *** rcernin has joined #openstack-cinder | 00:17 | |
| *** tosky has quit IRC | 00:29 | |
| *** baojg has joined #openstack-cinder | 01:08 | |
| *** Liang__ has joined #openstack-cinder | 01:15 | |
| *** ircuser-1 has quit IRC | 01:17 | |
| *** ociuhandu has joined #openstack-cinder | 01:25 | |
| *** ociuhandu has quit IRC | 01:32 | |
| *** nanzha has joined #openstack-cinder | 01:56 | |
| *** ociuhandu has joined #openstack-cinder | 02:02 | |
| *** ociuhandu has quit IRC | 02:06 | |
| *** smcginnis has joined #openstack-cinder | 02:10 | |
| *** ChanServ sets mode: +o smcginnis | 02:10 | |
| *** ociuhandu has joined #openstack-cinder | 02:19 | |
| *** ociuhandu has quit IRC | 02:24 | |
| *** zhengMa has joined #openstack-cinder | 02:38 | |
| *** brinzhang has joined #openstack-cinder | 03:07 | |
| *** ociuhandu has joined #openstack-cinder | 03:33 | |
| *** ociuhandu has quit IRC | 03:40 | |
| *** vesper11 has quit IRC | 03:45 | |
| *** vesper11 has joined #openstack-cinder | 03:47 | |
| *** davee__ has joined #openstack-cinder | 04:03 | |
| *** davee_ has quit IRC | 04:05 | |
| *** brinzhang_ has joined #openstack-cinder | 04:11 | |
| *** brinzhang has quit IRC | 04:14 | |
| *** ociuhandu has joined #openstack-cinder | 04:14 | |
| *** ociuhandu has quit IRC | 04:22 | |
| *** ociuhandu has joined #openstack-cinder | 04:22 | |
| *** ociuhandu has quit IRC | 04:23 | |
| *** ociuhandu has joined #openstack-cinder | 04:25 | |
| *** ociuhandu has quit IRC | 04:34 | |
| *** rchurch has quit IRC | 04:35 | |
| *** ociuhandu has joined #openstack-cinder | 04:36 | |
| *** nanzha has quit IRC | 04:40 | |
| *** nanzha has joined #openstack-cinder | 04:41 | |
| *** ociuhandu has quit IRC | 04:41 | |
| *** ociuhandu has joined #openstack-cinder | 04:54 | |
| *** ociuhandu has quit IRC | 04:59 | |
| *** nanzha has quit IRC | 05:00 | |
| *** nanzha has joined #openstack-cinder | 05:02 | |
| *** brinzhang has joined #openstack-cinder | 05:08 | |
| *** brinzhang_ has quit IRC | 05:10 | |
| *** udesale has joined #openstack-cinder | 05:27 | |
| *** ociuhandu has joined #openstack-cinder | 05:30 | |
| *** ociuhandu has quit IRC | 05:35 | |
| openstackgerrit | Abhishek Kekane proposed openstack/cinder-specs master: Image snapshot co-location https://review.opendev.org/695630 | 05:37 |
|---|---|---|
| *** pots has quit IRC | 05:39 | |
| *** pots has joined #openstack-cinder | 05:39 | |
| *** brinzhang_ has joined #openstack-cinder | 05:40 | |
| *** brinzhang has quit IRC | 05:43 | |
| *** whfnst has joined #openstack-cinder | 05:48 | |
| *** nanzha has quit IRC | 06:00 | |
| *** brinzhang has joined #openstack-cinder | 06:10 | |
| *** nanzha has joined #openstack-cinder | 06:12 | |
| *** brinzhang_ has quit IRC | 06:13 | |
| *** nanzha has quit IRC | 06:24 | |
| *** nanzha has joined #openstack-cinder | 06:36 | |
| *** dpawlik has joined #openstack-cinder | 06:47 | |
| *** brinzhang_ has joined #openstack-cinder | 06:56 | |
| *** brinzhang has quit IRC | 06:59 | |
| *** rcernin has quit IRC | 07:09 | |
| *** lpetrut has joined #openstack-cinder | 07:16 | |
| *** awalende has joined #openstack-cinder | 07:16 | |
| *** awalende has quit IRC | 07:21 | |
| openstackgerrit | Abhishek Kekane proposed openstack/cinder master: Support multiple stores of Glance https://review.opendev.org/661676 | 07:21 |
| *** nanzha has quit IRC | 07:22 | |
| *** sahid has joined #openstack-cinder | 07:25 | |
| openstackgerrit | Abhishek Kekane proposed openstack/cinder master: Support multiple stores of Glance https://review.opendev.org/661676 | 07:31 |
| *** ociuhandu has joined #openstack-cinder | 07:38 | |
| *** ociuhandu has quit IRC | 07:44 | |
| *** zhengMa has quit IRC | 07:55 | |
| *** zhengMa has joined #openstack-cinder | 08:00 | |
| *** zhengMa has left #openstack-cinder | 08:00 | |
| *** nanzha has joined #openstack-cinder | 08:00 | |
| *** tkajinam has quit IRC | 08:06 | |
| *** tesseract has joined #openstack-cinder | 08:16 | |
| *** tosky has joined #openstack-cinder | 08:21 | |
| *** awalende has joined #openstack-cinder | 08:30 | |
| *** awalende has quit IRC | 08:34 | |
| *** ociuhandu has joined #openstack-cinder | 08:35 | |
| *** ociuhandu has quit IRC | 08:40 | |
| *** awalende has joined #openstack-cinder | 08:43 | |
| *** ociuhandu has joined #openstack-cinder | 08:50 | |
| *** brinzhang has joined #openstack-cinder | 08:59 | |
| *** nanzha has quit IRC | 09:00 | |
| *** nanzha has joined #openstack-cinder | 09:01 | |
| *** brinzhang_ has quit IRC | 09:01 | |
| *** mns96 has joined #openstack-cinder | 09:06 | |
| *** e0ne has joined #openstack-cinder | 09:10 | |
| *** Lucas_Gray has joined #openstack-cinder | 09:21 | |
| *** ociuhandu has quit IRC | 09:40 | |
| *** martinkennelly has joined #openstack-cinder | 09:42 | |
| *** Liang__ has quit IRC | 09:48 | |
| *** ociuhandu has joined #openstack-cinder | 09:49 | |
| *** ociuhandu has quit IRC | 09:54 | |
| openstackgerrit | Dincer Celik proposed openstack/cinder stable/train: Increase cpu limit for image conversion https://review.opendev.org/693382 | 09:55 |
| *** takamatsu has quit IRC | 10:05 | |
| *** takamatsu has joined #openstack-cinder | 10:06 | |
| *** davidsha has joined #openstack-cinder | 10:27 | |
| *** ociuhandu has joined #openstack-cinder | 10:31 | |
| *** nanzha has quit IRC | 10:34 | |
| *** ociuhandu has quit IRC | 10:36 | |
| *** nanzha has joined #openstack-cinder | 10:38 | |
| *** brinzhang_ has joined #openstack-cinder | 10:44 | |
| *** brinzhang has quit IRC | 10:47 | |
| *** brinzhang has joined #openstack-cinder | 10:49 | |
| *** brinzhang_ has quit IRC | 10:53 | |
| *** awalende_ has joined #openstack-cinder | 10:59 | |
| *** awalende has quit IRC | 11:02 | |
| *** awalende_ has quit IRC | 11:03 | |
| *** awalende has joined #openstack-cinder | 11:04 | |
| *** nanzha has quit IRC | 11:06 | |
| *** nanzha has joined #openstack-cinder | 11:06 | |
| *** awalende has quit IRC | 11:11 | |
| *** brinzhang_ has joined #openstack-cinder | 11:13 | |
| *** pcaruana has joined #openstack-cinder | 11:14 | |
| *** brinzhang has quit IRC | 11:16 | |
| *** Lucas_Gray has quit IRC | 11:19 | |
| *** fgonzales3246 has joined #openstack-cinder | 11:23 | |
| *** ociuhandu has joined #openstack-cinder | 11:23 | |
| *** openstack has joined #openstack-cinder | 11:55 | |
| *** ChanServ sets mode: +o openstack | 11:55 | |
| *** awalende has joined #openstack-cinder | 11:56 | |
| *** raghavendrat has joined #openstack-cinder | 12:03 | |
| raghavendrat | hi geguileo: are you online? | 12:03 |
| geguileo | raghavendrat: I am | 12:03 |
| *** nanzha has quit IRC | 12:03 | |
| *** nanzha has joined #openstack-cinder | 12:04 | |
| raghavendrat | geguileo: this is regarding a backport patch https://review.opendev.org/#/c/694743/ | 12:04 |
| raghavendrat | whenever you get time, may i please request you to have a look. thanks. | 12:04 |
| geguileo | raghavendrat: done | 12:07 |
| raghavendrat | geguileo: thanks for quick response. | 12:08 |
| geguileo | np | 12:08 |
| *** brinzhang has joined #openstack-cinder | 12:14 | |
| *** brinzhang_ has quit IRC | 12:17 | |
| *** nanzha has quit IRC | 12:18 | |
| *** brinzhang_ has joined #openstack-cinder | 12:29 | |
| *** nanzha has joined #openstack-cinder | 12:30 | |
| *** dave-mccowan has joined #openstack-cinder | 12:30 | |
| *** brinzhang has quit IRC | 12:32 | |
| *** raghavendrat has left #openstack-cinder | 12:34 | |
| *** dave-mccowan has quit IRC | 12:35 | |
| *** nanzha has quit IRC | 12:36 | |
| *** nanzha has joined #openstack-cinder | 12:37 | |
| *** brinzhang has joined #openstack-cinder | 12:43 | |
| *** Lucas_Gray has joined #openstack-cinder | 12:44 | |
| *** brinzhang_ has quit IRC | 12:46 | |
| *** brinzhang_ has joined #openstack-cinder | 12:47 | |
| *** brinzhang has quit IRC | 12:48 | |
| *** lpetrut has joined #openstack-cinder | 12:53 | |
| *** nanzha has quit IRC | 13:12 | |
| *** brinzhang has joined #openstack-cinder | 13:14 | |
| *** brinzhang_ has quit IRC | 13:16 | |
| *** nanzha has joined #openstack-cinder | 13:17 | |
| *** mvkr has quit IRC | 13:19 | |
| *** rosmaita has joined #openstack-cinder | 13:24 | |
| *** nanzha has quit IRC | 13:24 | |
| awalende | Imagine your are running cinder with ceph rbd as backend and you test your service with creating 30 volumes parallel and deleting 30 volumes parallel and you suddenly see something like this in the logs: | 13:25 |
| awalende | pymysql.err.InternalError: (1205, 'Lock wait timeout exceeded; try restarting transaction') | 13:26 |
| *** lseki has joined #openstack-cinder | 13:26 | |
| awalende | Would you blame cinder-wsgi not releasing a lock or the database for being inresponsive? | 13:26 |
| rosmaita | awalende: wouldn't be cinder-wsgi, it hands off volume creation to cinder-volume (eventually) | 13:28 |
| rosmaita | cinder-volume does use locking, but i don't think it uses DB locks directly | 13:28 |
| awalende | errr yes, sorry. I saw the errors in cinder-volume.log | 13:28 |
| rosmaita | so i would suspect the database | 13:28 |
| awalende | Also HAproxy gives us gateway timeout on volume creation/deletion. Assuming that the api is not done in time | 13:29 |
| awalende | It's like 60 seconds already :/ | 13:29 |
| *** nanzha has joined #openstack-cinder | 13:30 | |
| rosmaita | well, both of those should return 202 fairly quickly, because they don't actually do the operation | 13:31 |
| awalende | I see | 13:32 |
| rosmaita | although, create would have to contact the DB to create a volume record | 13:32 |
| rosmaita | i'm pretty sure that's the case, give me a minute to check | 13:33 |
| awalende | Oooff, will be hard to find the culprit of this. The controller nodes are not running on 100% load, neither does the db. | 13:33 |
| rosmaita | yeah, sounds like some kind of race condition where everyone is waiting on something | 13:34 |
| awalende | The first 5 volumes of the 30 are spawning quickly and fine, the following seem to be stumbling on something | 13:34 |
| openstackgerrit | Merged openstack/cinder stable/train: iSCSI driver initialization should fail for Primera backend https://review.opendev.org/694743 | 13:34 |
| rosmaita | awalende: what release are you using? | 13:35 |
| awalende | rocky (production cloud) and stein (dev-cluster) | 13:35 |
| awalende | Have the problems on both | 13:35 |
| awalende | HASetup with 3 beefy controller nodes running galera also in HA | 13:36 |
| rosmaita | awalende: looks like the REST api doesn't need to contact the DB for create or delete, that's handled later | 13:43 |
| awalende | It must be some kind of race condition... | 13:44 |
| awalende | Either RabbitMQ or DB | 13:45 |
| awalende | Or is there something on cinder side I can increase? The only thing I found were the cinder-wsgi spawns in the apache2 config file | 13:45 |
| rosmaita | awalende: not sure, TBH. you'll want to look through what's available in cinder.conf. https://docs.openstack.org/cinder/stein/configuration/index.html and https://docs.openstack.org/cinder/stein/_static/cinder.conf.sample | 13:52 |
| awalende | Mhh okay, I'll play around on our dev cluster. Fingers crossed! | 13:53 |
| awalende | Thanks for the help :) | 13:53 |
| *** KeithMnemonic has joined #openstack-cinder | 13:54 | |
| rosmaita | not sure i was much help, but good luck! | 13:54 |
| awalende | I appreciate every bit :D | 13:54 |
| *** nanzha has quit IRC | 13:56 | |
| rosmaita | awalende: you could ask around in #openstack and see if any operators have suggestions | 13:56 |
| *** nanzha has joined #openstack-cinder | 13:59 | |
| *** brinzhang_ has joined #openstack-cinder | 14:02 | |
| openstackgerrit | Benny Kopilov proposed openstack/cinder-tempest-plugin master: Add testcase to verify disk size after snapshot created https://review.opendev.org/695817 | 14:03 |
| *** brinzhang has quit IRC | 14:04 | |
| *** brinzhang_ has quit IRC | 14:07 | |
| *** jcosmao has joined #openstack-cinder | 14:09 | |
| *** jdillaman has joined #openstack-cinder | 14:16 | |
| *** Lucas_Gray has quit IRC | 14:27 | |
| *** KeithMnemonic has quit IRC | 14:33 | |
| openstackgerrit | Benny Kopilov proposed openstack/cinder-tempest-plugin master: Add testcase to verify disk size after snapshot created https://review.opendev.org/695817 | 14:37 |
| smcginnis | rosmaita: Hey! Just wanted to point out the "PTG and Summit Meeting Summaries" section here - https://wiki.openstack.org/wiki/Cinder#Etherpads | 14:37 |
| smcginnis | Up to you if you want to maintain that and add a link to your Ussuri one. | 14:38 |
| rosmaita | smcginnis: ty, i will add a link | 14:38 |
| openstackgerrit | Helen Walsh proposed openstack/cinder master: PowerMax Driver - retype attached replication fix https://review.opendev.org/686429 | 14:51 |
| *** sfernand has joined #openstack-cinder | 14:55 | |
| rosmaita | Hello everyone ... virtual PTG starting in https://bluejeans.com/3228528973 in about 4 minutes (1500 UTC) | 14:56 |
| *** eharney has joined #openstack-cinder | 14:57 | |
| tosky | rosmaita: sound only or video too? | 14:57 |
| rosmaita | tosky: both | 14:58 |
| rosmaita | hopefully | 14:58 |
| jungleboyj | rosmaita: Have a meeting running long. Will join as soon as I can. | 15:00 |
| rosmaita | ok, sounds good | 15:00 |
| *** mvkr has joined #openstack-cinder | 15:06 | |
| * e0ne will connect to the virtual PTG in ~10 minutes | 15:09 | |
| *** ociuhandu has quit IRC | 15:10 | |
| *** awalende has quit IRC | 15:11 | |
| *** awalende has joined #openstack-cinder | 15:16 | |
| *** awalende has quit IRC | 15:17 | |
| *** jdillaman has quit IRC | 15:19 | |
| *** ociuhandu has joined #openstack-cinder | 15:58 | |
| *** ociuhandu has quit IRC | 16:03 | |
| *** openstackstatus has joined #openstack-cinder | 16:14 | |
| *** ChanServ sets mode: +v openstackstatus | 16:14 | |
| *** nanzha has quit IRC | 16:18 | |
| *** ircuser-1 has joined #openstack-cinder | 16:37 | |
| *** lpetrut has quit IRC | 16:51 | |
| *** dklyle has quit IRC | 16:53 | |
| *** dklyle has joined #openstack-cinder | 16:54 | |
| *** udesale has quit IRC | 16:57 | |
| smcginnis | eharney: Based on past comments, you may like this - http://lists.openstack.org/pipermail/openstack-discuss/2019-November/011030.html | 17:01 |
| smcginnis | ;) | 17:01 |
| smcginnis | rosmaita: Not sure about others, but lost your audio as you were wrapping up. | 17:01 |
| smcginnis | I hear jungleboyj typing fine. | 17:01 |
| rosmaita | yeah, recording has now stopped | 17:02 |
| jungleboyj | Thanks Brian. Good meeting. | 17:02 |
| rosmaita | will be interesting to see how much of me talking to myself shows up there | 17:02 |
| *** e0ne has quit IRC | 17:03 | |
| jungleboyj | :-) | 17:03 |
| openstackgerrit | Benny Kopilov proposed openstack/cinder-tempest-plugin master: Add testcase to verify disk size after snapshot created https://review.opendev.org/695817 | 17:03 |
| davee___ | stream of consciousness meeting lol | 17:03 |
| *** sahid has quit IRC | 17:08 | |
| *** jcosmao has left #openstack-cinder | 17:09 | |
| *** rkuschel has quit IRC | 17:10 | |
| *** fgonzales3246 has quit IRC | 17:12 | |
| *** awalende has joined #openstack-cinder | 17:12 | |
| *** jcosmao has joined #openstack-cinder | 17:15 | |
| *** awalende has quit IRC | 17:17 | |
| *** takamatsu has quit IRC | 17:27 | |
| *** FlorianFa has quit IRC | 17:29 | |
| *** takamatsu has joined #openstack-cinder | 17:32 | |
| *** tesseract has quit IRC | 17:57 | |
| *** davidsha has quit IRC | 18:09 | |
| *** martinkennelly has quit IRC | 18:30 | |
| openstackgerrit | Brian Rosmaita proposed openstack/cinder master: Remove Python 2 support from testing and gate jobs https://review.opendev.org/695317 | 18:31 |
| *** mns96 has quit IRC | 18:36 | |
| *** ociuhandu has joined #openstack-cinder | 18:45 | |
| *** cmorpheus is now known as cmurphy | 18:50 | |
| *** ociuhandu has quit IRC | 18:51 | |
| *** ociuhandu has joined #openstack-cinder | 18:52 | |
| openstackgerrit | Benny Kopilov proposed openstack/cinder-tempest-plugin master: Remove duplicate code - base inherited from tempest https://review.opendev.org/695970 | 18:53 |
| openstackgerrit | Eric Harney proposed openstack/cinder stable/stein: Increase cpu limit for image conversion https://review.opendev.org/693610 | 19:04 |
| *** eharney has quit IRC | 19:18 | |
| *** ociuhandu has quit IRC | 19:32 | |
| *** ociuhandu has joined #openstack-cinder | 19:33 | |
| *** ociuhandu has quit IRC | 19:38 | |
| *** jdillaman has joined #openstack-cinder | 19:38 | |
| *** e0ne has joined #openstack-cinder | 19:55 | |
| *** whfnst has quit IRC | 20:15 | |
| *** sfernand has quit IRC | 20:24 | |
| *** eharney has joined #openstack-cinder | 20:32 | |
| *** e0ne has quit IRC | 20:36 | |
| *** e0ne has joined #openstack-cinder | 20:36 | |
| *** e0ne has quit IRC | 20:48 | |
| *** ociuhandu has joined #openstack-cinder | 20:50 | |
| *** e0ne has joined #openstack-cinder | 21:02 | |
| *** ociuhandu has quit IRC | 21:11 | |
| *** ociuhandu has joined #openstack-cinder | 21:12 | |
| *** spatel has joined #openstack-cinder | 21:15 | |
| *** ociuhandu has quit IRC | 21:17 | |
| *** e0ne has quit IRC | 21:25 | |
| *** spatel has quit IRC | 21:26 | |
| *** e0ne has joined #openstack-cinder | 21:30 | |
| *** e0ne has quit IRC | 21:31 | |
| *** rcernin has joined #openstack-cinder | 21:31 | |
| *** pcaruana has quit IRC | 21:38 | |
| *** ntpttr has joined #openstack-cinder | 21:48 | |
| *** e0ne has joined #openstack-cinder | 22:08 | |
| *** KeithMnemonic has joined #openstack-cinder | 22:41 | |
| *** e0ne has quit IRC | 22:52 | |
| *** tkajinam has joined #openstack-cinder | 23:09 | |
| *** ociuhandu has joined #openstack-cinder | 23:30 | |
| *** kaisers1 has quit IRC | 23:31 | |
| *** kaisers has joined #openstack-cinder | 23:34 | |
| *** n-saito has joined #openstack-cinder | 23:34 | |
| *** ociuhandu has quit IRC | 23:35 | |
| *** tosky has quit IRC | 23:36 | |
| *** jcosmao has left #openstack-cinder | 23:42 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!