| *** markvoelker has quit IRC | 00:35 | |
| *** zhurong has joined #openstack-glance | 00:37 | |
| *** markvoelker has joined #openstack-glance | 00:47 | |
| *** gcb has joined #openstack-glance | 01:44 | |
| *** gyee has quit IRC | 02:02 | |
| *** bkopilov has quit IRC | 02:12 | |
| *** udesale has joined #openstack-glance | 03:35 | |
| *** bkopilov has joined #openstack-glance | 03:37 | |
| *** links has joined #openstack-glance | 04:02 | |
| *** hoonetorg has quit IRC | 04:40 | |
| *** hoonetorg has joined #openstack-glance | 04:55 | |
| *** mabrams has joined #openstack-glance | 05:02 | |
| *** ratailor has joined #openstack-glance | 05:17 | |
| *** abhishekk has joined #openstack-glance | 05:21 | |
| openstackgerrit | Dinesh Bhor proposed openstack/glance master: Optimise: Use sqlalchemy to build table sequence https://review.openstack.org/423958 | 05:23 |
|---|---|---|
| *** rcernin has joined #openstack-glance | 05:40 | |
| openstackgerrit | Bhagyashri Shewale proposed openstack/python-glanceclient master: Remove redundant information from error message https://review.openstack.org/475351 | 06:15 |
| *** d0ugal has quit IRC | 06:33 | |
| *** pcaruana has joined #openstack-glance | 06:52 | |
| *** tesseract has joined #openstack-glance | 06:56 | |
| *** d0ugal has joined #openstack-glance | 07:09 | |
| *** d0ugal has quit IRC | 07:18 | |
| *** tshefi has joined #openstack-glance | 07:19 | |
| *** nicolasbock has joined #openstack-glance | 07:24 | |
| *** nicolasbock has quit IRC | 07:50 | |
| *** aavraham has joined #openstack-glance | 07:53 | |
| *** markvoelker has quit IRC | 08:08 | |
| *** belmoreira has joined #openstack-glance | 08:31 | |
| *** e0ne has joined #openstack-glance | 08:38 | |
| *** d0ugal has joined #openstack-glance | 08:41 | |
| *** d0ugal has quit IRC | 09:15 | |
| *** mvk has quit IRC | 09:28 | |
| abhishekk | andreykurlin: hi you around? | 09:35 |
| abhishekk | andreykurilin: hi you around? | 09:36 |
| abhishekk | andreykurilin: hi | 09:37 |
| andreykurilin | hi | 09:37 |
| andreykurilin | yes | 09:37 |
| abhishekk | have you checked https://bugs.launchpad.net/devstack/+bug/1701088 | 09:40 |
| openstack | Launchpad bug 1701088 in devstack "glance under uwsgi fails on uploads" [Medium,Fix released] - Assigned to Matthew Treinish (treinish) | 09:40 |
| abhishekk | will it solve your issue? | 09:40 |
| andreykurilin | abhishekk: Right after swithing to glance+uswgi, our gates were failing with 500 error, as far as I remember | 09:42 |
| andreykurilin | and after a following patch was merged, the errors cheched | 09:42 |
| andreykurilin | note: we are using master of devstack and master of glance in out gates | 09:42 |
| abhishekk | ok | 09:43 |
| andreykurilin | *error changed | 09:43 |
| abhishekk | i will have a further look | 09:43 |
| andreykurilin | thanks | 09:44 |
| abhishekk | https://bugs.launchpad.net/glance/+bug/1703856 is similar what you are seeing in rally right | 09:45 |
| openstack | Launchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [Undecided,New] | 09:45 |
| andreykurilin | yes | 09:50 |
| abhishekk | ok | 09:50 |
| *** mvk has joined #openstack-glance | 09:55 | |
| *** markvoelker has joined #openstack-glance | 10:09 | |
| *** nicolasbock has joined #openstack-glance | 10:25 | |
| abhishekk | andreykurilin: could you please share glance-api logs? | 10:25 |
| andreykurilin | sure | 10:27 |
| andreykurilin | abhishekk: http://logs.openstack.org/73/482473/6/check/gate-rally-dsvm-rally-ubuntu-xenial/ed186ad/logs/ | 10:27 |
| andreykurilin | all logs are here | 10:27 |
| abhishekk | thank you | 10:27 |
| andreykurilin | abhishek: also, here you can find timestamps of errors - http://logs.openstack.org/73/482473/6/check/gate-rally-dsvm-rally-ubuntu-xenial/ed186ad/console.html#_2017-07-12_22_21_33_772749 | 10:30 |
| *** mvk has quit IRC | 10:34 | |
| abhishekk | ok | 10:35 |
| *** udesale has quit IRC | 10:36 | |
| *** markvoelker has quit IRC | 10:43 | |
| *** zhurong has quit IRC | 10:43 | |
| *** mvk has joined #openstack-glance | 10:50 | |
| *** gcb has quit IRC | 10:52 | |
| *** rosmaita has joined #openstack-glance | 10:52 | |
| *** bkopilov has quit IRC | 10:57 | |
| *** mabrams has quit IRC | 11:04 | |
| *** mvk has quit IRC | 11:13 | |
| *** smatzek has joined #openstack-glance | 11:24 | |
| *** mvk has joined #openstack-glance | 11:26 | |
| *** e0ne_ has joined #openstack-glance | 11:37 | |
| *** e0ne has quit IRC | 11:39 | |
| abhishekk | andreykurilin: still around? | 11:39 |
| andreykurilin | Yup | 11:40 |
| *** markvoelker has joined #openstack-glance | 11:41 | |
| *** d0ugal has joined #openstack-glance | 11:41 | |
| *** sudhirag098 has joined #openstack-glance | 11:46 | |
| *** d0ugal has quit IRC | 11:47 | |
| abhishekk | have you tried increasing socket-timeout in /etc/glance/glance-uwsgi.ini? | 11:51 |
| *** sudhirag098 has left #openstack-glance | 11:54 | |
| *** ratailor has quit IRC | 11:57 | |
| *** udesale has joined #openstack-glance | 12:01 | |
| andreykurilin | abhishekk: I thought it should be already done by mtreinish's patch | 12:01 |
| *** sudhirag098_ has joined #openstack-glance | 12:02 | |
| abhishekk | andreykurilin: as a workaround try setting it to higher value, he has set it to 30 | 12:03 |
| andreykurilin | abshishekk: shoould I propose a patch to devstack? | 12:04 |
| andreykurilin | *abhishekk | 12:04 |
| andreykurilin | sorry | 12:04 |
| abhishekk | i guess we should discuss this with mtreinish | 12:08 |
| abhishekk | but for workaround could you please try this, I have teseted this in my environment | 12:08 |
| abhishekk | https://bugs.launchpad.net/glance/+bug/1703856/comments/1 | 12:08 |
| openstack | Launchpad bug 1703856 in Glance "502 Bad gateway error on image-create" [High,Confirmed] | 12:08 |
| abhishekk | andreykurilin: ^^^^ | 12:09 |
| andreykurilin | abhishekk: we do not use custom ci for checking glance scenarios. It is just regular infra ci. so to apply proposed change we need to modify devstack or project-config | 12:10 |
| abhishekk | you can run rally scenarios in your local environment right? | 12:11 |
| *** markvoelker has quit IRC | 12:13 | |
| andreykurilin | yes, but I do not have a VM now to install rally with OpenStack | 12:13 |
| abhishekk | ohhh | 12:13 |
| abhishekk | we cannot propose patch as we can not determine the exact value for socket-timeout | 12:14 |
| abhishekk | I have tried it with 10GB only, not tested it for higher size because of limited resources :( | 12:15 |
| *** bkopilov has joined #openstack-glance | 12:24 | |
| andreykurilin | abhishekk: In gates the size of image is quite small, I think it is much smaller that 10gb | 12:25 |
| andreykurilin | because the whole hd is 10gb )) | 12:25 |
| andreykurilin | abhishekk: do you know, does it possible to propose a change to devstack and make a patch to rally project with depends-on to devstack change? will infra apply c change from devstack ? | 12:26 |
| abhishekk | andreykurilin: ok, but definitely this is the problem of uwsgi socket | 12:26 |
| abhishekk | not sure, ideally it should do | 12:27 |
| andreykurilin | abhishekk: I'm not saying that the problem in glance:) even more, I mentioned that all these years we did not have any problems with glance till now | 12:27 |
| rosmaita | andreykurilin: yes, and we appreciate that | 12:28 |
| rosmaita | we're just a little sensitive | 12:28 |
| kairat | abhishekk, will you able to create empty images? | 12:28 |
| abhishekk | wait, I am running rally and it is failing even with higher socket-timeout value | 12:28 |
| rosmaita | abhishekk: ty for being on top of this | 12:28 |
| abhishekk | kairat: yest | 12:28 |
| abhishekk | rosmaita: no problem | 12:29 |
| kairat | so i mean only image upload request is failimn at the current moment | 12:29 |
| kairat | i wondering if it is related to http-keepalive | 12:29 |
| abhishekk | kairat: no it is not creating empty image | 12:29 |
| abhishekk | http-keepalive is false in uwsgi | 12:30 |
| andreykurilin | rosmaita: everyone is sensitive about their project ;) | 12:30 |
| kairat | abhishekk, so any request fails with 502& | 12:30 |
| kairat | *502 error | 12:31 |
| kairat | i am asking because when I tried to deploy glance under uwsgi it was not supporting chunked requests | 12:32 |
| *** markvoelker has joined #openstack-glance | 12:32 | |
| abhishekk | sorry, my reply might be late as I am travelling to home now | 12:32 |
| kairat | np | 12:32 |
| abhishekk | it does not create zero size image-file when it fails with 502 error | 12:35 |
| *** abhishekk has quit IRC | 12:35 | |
| kairat | need to know if you able glance image-create without file | 12:36 |
| *** abhishekk_ has joined #openstack-glance | 12:37 | |
| *** gcb has joined #openstack-glance | 12:39 | |
| *** deep-book-gk_ has joined #openstack-glance | 12:48 | |
| *** deep-book-gk_ has left #openstack-glance | 12:49 | |
| *** d0ugal has joined #openstack-glance | 12:55 | |
| *** catintheroof has joined #openstack-glance | 12:56 | |
| *** links has quit IRC | 12:57 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/glance master: Updated from global requirements https://review.openstack.org/483354 | 12:59 |
| *** MVenesio has joined #openstack-glance | 13:06 | |
| *** d0ugal has quit IRC | 13:07 | |
| *** d0ugal has joined #openstack-glance | 13:18 | |
| *** d0ugal has quit IRC | 13:18 | |
| *** d0ugal has joined #openstack-glance | 13:18 | |
| *** d0ugal has quit IRC | 13:20 | |
| *** d0ugal has joined #openstack-glance | 13:20 | |
| *** mabrams has joined #openstack-glance | 13:27 | |
| *** gabor_antal_ is now known as gabor_antal | 13:28 | |
| *** d0ugal has quit IRC | 13:29 | |
| *** sudhirag098_ has quit IRC | 13:35 | |
| *** smatzek has quit IRC | 13:44 | |
| *** mabrams has quit IRC | 13:58 | |
| rosmaita | Courtesy meeting reminder on #openstack-meeting-4: ativelkov, cpallares, flaper87, flwang1, hemanthm, jokke_, kragniz, lakshmiS, mclaren, mfedosin, nikhil_k, Nikolay_St, Olena, pennerc, rosmaita, sigmavirus24, sabari, TravT, ajayaa, GB21, bpoulos, harshs, abhishek, bunting, dshakhray, wxy, dhellmann, kairat, aavraham, alex_bash, smcginnis | 13:58 |
| smcginnis | o/ | 13:59 |
| kairat | o/ | 14:00 |
| kairat | smcginnis, looks like meeting has started in meeting channel openstack-meeting-4 | 14:01 |
| smcginnis | kairat: Yep, I see you there. :) | 14:01 |
| *** belmoreira has quit IRC | 14:02 | |
| *** smatzek has joined #openstack-glance | 14:06 | |
| *** abhishekk has joined #openstack-glance | 14:08 | |
| *** abhishekk_ has quit IRC | 14:11 | |
| *** abhishekk has quit IRC | 14:14 | |
| *** abhishekk has joined #openstack-glance | 14:15 | |
| *** e0ne_ has quit IRC | 14:16 | |
| *** aarefiev is now known as aarefiev_afk | 14:21 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-glanceclient master: Updated from global requirements https://review.openstack.org/483416 | 14:23 |
| *** e0ne has joined #openstack-glance | 14:24 | |
| *** TravT has joined #openstack-glance | 14:28 | |
| *** lucasxu has joined #openstack-glance | 14:46 | |
| rosmaita | hello everyone | 14:50 |
| efried | rosmaita https://etherpad.openstack.org/p/glance-store-priority-reviews-pike | 14:50 |
| rosmaita | let's have an impromptu glance_store prioritization | 14:50 |
| rosmaita | using the etherpad efried has so kindly created for us | 14:50 |
| efried | Pasted in that link that IRC split up three times. | 14:51 |
| efried | Can use it as a start. | 14:51 |
| efried | Looks like there's really not that many. | 14:52 |
| efried | Should be able to knock out those "Needs final +2" pretty quick, nah? | 14:52 |
| rosmaita | sorry, was looking | 14:54 |
| rosmaita | yes, that';s a good place to start | 14:54 |
| *** efried is now known as efried_bbiab | 15:00 | |
| rosmaita | jokke_: how do you feel about https://review.openstack.org/#/c/430641/ ? | 15:00 |
| rosmaita | gone through 31 patch sets since feb 8 | 15:00 |
| *** abhishekk has quit IRC | 15:00 | |
| *** abhishekk has joined #openstack-glance | 15:01 | |
| *** e0ne has quit IRC | 15:01 | |
| jokke_ | looking | 15:02 |
| rosmaita | you don't have to assess patch quality atm, just whether the topic is a good one to include | 15:02 |
| jokke_ | yes I think that would be super helpful to include ... also I have to disagree smcginnis "hole" is the only well known term for those holes in sparse files I've ever stumbled upon to in my stroage career so I would not hold it up for using it ... I just haven't had time to actually test that it works | 15:05 |
| rosmaita | ok, cool | 15:05 |
| rosmaita | i think fei long is still ceph driver maintainer, so i will shoot him an email to look too | 15:05 |
| jokke_ | I'll ping our ceph guys if someone would have time to have a look on that | 15:07 |
| smcginnis | jokke_: No worries, I've certainly spent more time on the block side than file. Just hadn't heard that commonly used over the last 15 years in storage. | 15:08 |
| rosmaita | what's the policy about backports? do we need to worry about that now? asking about https://review.openstack.org/#/c/410547/ | 15:08 |
| *** e0ne has joined #openstack-glance | 15:09 | |
| *** lucasxu has quit IRC | 15:09 | |
| jokke_ | rosmaita: glance_store follows normal stable policy | 15:10 |
| rosmaita | so would now be a good time to backport a fix, or wait? | 15:11 |
| jokke_ | rosmaita: I'm not 100% sure if ending up with bigger volume than requested is critical enough to backport to newton | 15:12 |
| *** abhishek_k has joined #openstack-glance | 15:12 | |
| rosmaita | good enough, let's not worry about it | 15:12 |
| jokke_ | I'd focus atm. to get the Pike release together and look into other backport after that | 15:12 |
| rosmaita | sean raises a good point on this one: https://review.openstack.org/#/c/448316/1 | 15:13 |
| jokke_ | we can have a look of backports and request stable releases the week after | 15:13 |
| rosmaita | namely, we need the patch that actually removes the deprecated options | 15:13 |
| rosmaita | i'll pick up this patch and add the option removal to it | 15:13 |
| rosmaita | will also look over whether there's anything else deprecated that's supposed to go in this cycle | 15:14 |
| *** abhishekk has quit IRC | 15:16 | |
| jokke_ | ;) | 15:17 |
| jokke_ | indeed | 15:17 |
| *** gcb has quit IRC | 15:23 | |
| jokke_ | I need to get something to eat ... back in few min | 15:23 |
| rosmaita | ok | 15:23 |
| *** rcernin has quit IRC | 15:27 | |
| *** udesale has quit IRC | 15:31 | |
| *** aavraham has quit IRC | 15:34 | |
| *** pcaruana has quit IRC | 15:38 | |
| *** efried_bbiab is now known as efried | 15:38 | |
| *** MVenesio has quit IRC | 15:38 | |
| *** MVenesio has joined #openstack-glance | 15:39 | |
| *** lucasxu has joined #openstack-glance | 15:49 | |
| *** e0ne has quit IRC | 15:55 | |
| *** tshefi has quit IRC | 15:55 | |
| *** lucasxu has quit IRC | 16:04 | |
| *** gyee has joined #openstack-glance | 16:09 | |
| efried | rosmaita So what about the bug-topic ones with -1s? Should we follow up and try to drive them to completion? | 16:13 |
| rosmaita | use your judgement on those depending on what's causing the -1 | 16:15 |
| *** tesseract has quit IRC | 16:19 | |
| *** jamielennox has quit IRC | 16:19 | |
| *** lucasxu has joined #openstack-glance | 16:21 | |
| rosmaita | efried: i'll try to get some -1s prioritized, but it will be later in the day | 16:24 |
| *** chlong has joined #openstack-glance | 16:30 | |
| *** chlong has quit IRC | 16:32 | |
| *** chlong has joined #openstack-glance | 16:33 | |
| abhishek_k | rosmaita, jokke_: ran couple of scenarios with higher socket-timeout value, http://paste.openstack.org/show/615295/ | 16:36 |
| jokke_ | abhishek_k: so it really looks like it's just socket timeout issue? | 16:37 |
| abhishek_k | i will try to configure uswgi logs and find out if I can get any helpful logs from uswgi side, http://uwsgi-docs.readthedocs.io/en/latest/Logging.html | 16:37 |
| abhishek_k | jokke_: yes | 16:38 |
| abhishek_k | I have tried couple of options to enable uswgi logs but after making changes in glance-uswgi.ini api service fails to start | 16:39 |
| rosmaita | bummer | 16:40 |
| rosmaita | abhishek_k: what's the time unit, is it 300 sec? | 16:41 |
| *** zzzeek_ has quit IRC | 16:41 | |
| abhishek_k | yes | 16:41 |
| rosmaita | your results are interesting, the concurrent uploads seem to indicate that its not a data saturation problem, probably is just a timeout | 16:42 |
| abhishek_k | rosmaita: yes, to confirm that I have ran concurrent uploads | 16:43 |
| *** zzzeek has joined #openstack-glance | 16:43 | |
| rosmaita | yeah, that was a good iea | 16:43 |
| rosmaita | *idea | 16:43 |
| *** ChanServ changes topic to "OpenStack Glance" | 16:53 | |
| *** ChanServ changes topic to "OpenStack Glance | priorities: http://lists.openstack.org/pipermail/openstack-dev/2017-July/119673.html" | 16:53 | |
| *** ChanServ changes topic to "OpenStack Glance | priorities: http://lists.openstack.org/pipermail/openstack-dev/2017-July/119673.html | logs at http://eavesdrop.openstack.org/irclogs/%23openstack-glance/" | 16:53 | |
| *** rosmaita has left #openstack-glance | 16:54 | |
| *** e0ne has joined #openstack-glance | 17:09 | |
| *** harlowja has quit IRC | 17:27 | |
| *** abhishek_k has quit IRC | 17:47 | |
| *** abhishekk has joined #openstack-glance | 17:50 | |
| abhishekk | jokke_: I will continue digging tomorrow morning my time, need to find the cause why uwsgi is closing the socket intermittently | 17:52 |
| *** abhishekk has quit IRC | 17:54 | |
| *** aavraham has joined #openstack-glance | 17:55 | |
| *** aavraham has left #openstack-glance | 17:55 | |
| *** mvk has quit IRC | 18:01 | |
| openstackgerrit | Merged openstack/python-glanceclient master: Updated from global requirements https://review.openstack.org/483416 | 18:07 |
| *** e0ne has quit IRC | 18:16 | |
| *** mvk has joined #openstack-glance | 18:35 | |
| *** jamielennox has joined #openstack-glance | 18:37 | |
| *** pcaruana has joined #openstack-glance | 18:38 | |
| *** harlowja has joined #openstack-glance | 18:54 | |
| *** nicolasbock has quit IRC | 19:04 | |
| *** aavraham has joined #openstack-glance | 19:13 | |
| *** aavraham has quit IRC | 19:27 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/glance_store master: Updated from global requirements https://review.openstack.org/483523 | 19:29 |
| *** lucasxu has quit IRC | 19:50 | |
| *** pcaruana has quit IRC | 19:52 | |
| *** mtanino__ has joined #openstack-glance | 20:08 | |
| *** harlowja has quit IRC | 20:19 | |
| *** smatzek has quit IRC | 20:21 | |
| *** harlowja has joined #openstack-glance | 20:22 | |
| *** TravT has quit IRC | 20:23 | |
| *** rosmaita has joined #openstack-glance | 20:34 | |
| *** MVenesio has quit IRC | 20:51 | |
| *** chlong has quit IRC | 20:52 | |
| *** smatzek has joined #openstack-glance | 21:15 | |
| *** smatzek has quit IRC | 21:20 | |
| *** zzzeek has quit IRC | 21:22 | |
| *** zzzeek has joined #openstack-glance | 21:24 | |
| *** rosmaita has left #openstack-glance | 22:48 | |
| *** catintheroof has quit IRC | 23:03 | |
| *** mtanino__ has quit IRC | 23:05 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!