| *** andrebeltrami has quit IRC | 00:54 | |
| *** hongbin has joined #openstack-meeting-4 | 01:12 | |
| *** yamamoto has joined #openstack-meeting-4 | 01:35 | |
| *** yamamoto has quit IRC | 01:36 | |
| *** yamamoto has joined #openstack-meeting-4 | 01:36 | |
| *** vishalmanchanda has joined #openstack-meeting-4 | 01:51 | |
| *** diablo_rojo has quit IRC | 02:13 | |
| *** yamamoto has quit IRC | 02:20 | |
| *** rosmaita has left #openstack-meeting-4 | 02:28 | |
| *** yamamoto has joined #openstack-meeting-4 | 02:35 | |
| *** hongbin has quit IRC | 03:03 | |
| *** yamamoto has quit IRC | 03:19 | |
| *** Xuchu_ has joined #openstack-meeting-4 | 03:20 | |
| *** Xuchu has quit IRC | 03:22 | |
| *** Xuchu_ is now known as Xuchu | 03:22 | |
| *** yamamoto has joined #openstack-meeting-4 | 03:24 | |
| *** ricolin_ has joined #openstack-meeting-4 | 03:31 | |
| *** ricolin_ has quit IRC | 03:33 | |
| *** ricolin_ has joined #openstack-meeting-4 | 03:34 | |
| *** psachin has joined #openstack-meeting-4 | 03:37 | |
| *** yamamoto has quit IRC | 03:44 | |
| *** yamamoto has joined #openstack-meeting-4 | 03:45 | |
| *** evrardjp has quit IRC | 05:35 | |
| *** evrardjp has joined #openstack-meeting-4 | 05:36 | |
| *** links has joined #openstack-meeting-4 | 05:58 | |
| *** Luzi has joined #openstack-meeting-4 | 06:02 | |
| *** gcheresh has joined #openstack-meeting-4 | 06:22 | |
| *** e0ne has joined #openstack-meeting-4 | 06:28 | |
| *** yamamoto has quit IRC | 06:40 | |
| *** yamamoto has joined #openstack-meeting-4 | 06:42 | |
| *** yamamoto has quit IRC | 06:50 | |
| *** yamamoto has joined #openstack-meeting-4 | 06:55 | |
| *** LiangFang has quit IRC | 07:02 | |
| *** smcginnis has quit IRC | 07:03 | |
| *** smcginnis has joined #openstack-meeting-4 | 07:04 | |
| *** Liang__ has joined #openstack-meeting-4 | 07:04 | |
| *** e0ne has quit IRC | 07:10 | |
| *** roman_g has quit IRC | 07:11 | |
| *** e0ne has joined #openstack-meeting-4 | 07:19 | |
| *** e0ne has quit IRC | 07:23 | |
| *** slaweq has joined #openstack-meeting-4 | 08:02 | |
| *** e0ne has joined #openstack-meeting-4 | 08:34 | |
| *** k_mouza has joined #openstack-meeting-4 | 08:44 | |
| *** k_mouza has quit IRC | 08:49 | |
| *** ralonsoh has joined #openstack-meeting-4 | 08:58 | |
| *** ktibi has joined #openstack-meeting-4 | 09:11 | |
| *** k_mouza has joined #openstack-meeting-4 | 09:34 | |
| *** Xuchu_ has joined #openstack-meeting-4 | 09:49 | |
| *** Xuchu has quit IRC | 09:50 | |
| *** Xuchu_ is now known as Xuchu | 09:50 | |
| *** e0ne_ has joined #openstack-meeting-4 | 10:50 | |
| *** e0ne has quit IRC | 10:51 | |
| *** vishalmanchanda has quit IRC | 11:00 | |
| *** andrebeltrami has joined #openstack-meeting-4 | 11:08 | |
| *** ktibi has quit IRC | 11:10 | |
| *** ktibi has joined #openstack-meeting-4 | 11:11 | |
| *** yamamoto has quit IRC | 11:22 | |
| *** yamamoto has joined #openstack-meeting-4 | 11:24 | |
| *** yamamoto has quit IRC | 11:27 | |
| *** eharney has joined #openstack-meeting-4 | 11:52 | |
| *** yamamoto has joined #openstack-meeting-4 | 11:59 | |
| *** yamamoto has quit IRC | 12:05 | |
| *** enriquetaso has joined #openstack-meeting-4 | 12:09 | |
| *** Luzi has quit IRC | 12:10 | |
| *** yamamoto has joined #openstack-meeting-4 | 12:14 | |
| *** yamamoto has quit IRC | 12:15 | |
| *** psachin has quit IRC | 12:47 | |
| *** yamamoto has joined #openstack-meeting-4 | 12:52 | |
| *** eharney has quit IRC | 12:53 | |
| *** yamamoto has quit IRC | 12:57 | |
| *** psachin has joined #openstack-meeting-4 | 13:12 | |
| *** Xuchu has quit IRC | 13:16 | |
| *** Xuchu has joined #openstack-meeting-4 | 13:17 | |
| *** k_mouza has quit IRC | 13:21 | |
| *** yamamoto has joined #openstack-meeting-4 | 13:24 | |
| *** Xuchu has quit IRC | 13:24 | |
| *** gcheresh has quit IRC | 13:29 | |
| *** eharney has joined #openstack-meeting-4 | 13:38 | |
| *** yamamoto has quit IRC | 13:39 | |
| *** gcheresh has joined #openstack-meeting-4 | 13:39 | |
| abhishekk | #startmeeting glance | 14:00 |
|---|---|---|
| abhishekk | #topic roll call | 14:00 |
| openstack | Meeting started Thu Mar 12 14:00:05 2020 UTC and is due to finish in 60 minutes. The chair is abhishekk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| abhishekk | #link https://etherpad.openstack.org/p/glance-team-meeting-agenda | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: glance)" | 14:00 | |
| openstack | The meeting name has been set to 'glance' | 14:00 |
| *** openstack changes topic to "roll call (Meeting topic: glance)" | 14:00 | |
| *** nao-shark has joined #openstack-meeting-4 | 14:00 | |
| abhishekk | #link https://etherpad.openstack.org/p/glance-team-meeting-agenda | 14:00 |
| abhishekk | o/ | 14:00 |
| jokke_ | o/ | 14:00 |
| abhishekk | lets wait for 2-3 minutes | 14:01 |
| smcginnis | o/ | 14:01 |
| *** jraju__ has joined #openstack-meeting-4 | 14:01 | |
| nao-shark | o/ | 14:02 |
| abhishekk | lets start | 14:02 |
| *** links has quit IRC | 14:02 | |
| abhishekk | #topic release/periodic jobs update | 14:02 |
| *** openstack changes topic to "release/periodic jobs update (Meeting topic: glance)" | 14:02 | |
| *** dave-mccowan has joined #openstack-meeting-4 | 14:02 | |
| abhishekk | We are in last month of Ussuri cycle, some of the important dates are; | 14:03 |
| abhishekk | Final release for non-client libraries - 2 weeks | 14:03 |
| abhishekk | Milestone 3 - 3 weeks | 14:03 |
| abhishekk | So guys please review all remaining and important patches | 14:03 |
| abhishekk | Surprisingly, Periodic jobs all green | 14:03 |
| *** yamamoto has joined #openstack-meeting-4 | 14:03 | |
| abhishekk | for last couple of days | 14:03 |
| *** rosmaita has joined #openstack-meeting-4 | 14:04 | |
| smcginnis | Makes me wonder if it's actually working. :D | 14:04 |
| rosmaita | sorry, lost track of time | 14:04 |
| abhishekk | I have local patch do suppress some deprecation warning messages | 14:04 |
| abhishekk | will submit it tomorrow | 14:04 |
| abhishekk | smcginnis, likewise :D | 14:04 |
| abhishekk | rosmaita, NP | 14:04 |
| abhishekk | moving ahead; | 14:05 |
| abhishekk | #topic S3 support for glance_store | 14:05 |
| *** openstack changes topic to "S3 support for glance_store (Meeting topic: glance)" | 14:05 | |
| abhishekk | This one is important change and we agree to get it in at the start of the cycle | 14:05 |
| abhishekk | specs and patch are both in good shape, also we have etherpad which tells us how to test this change | 14:06 |
| abhishekk | spec, #link https://review.opendev.org/687390 | 14:06 |
| abhishekk | Implementation, #link https://review.opendev.org/695844 | 14:06 |
| abhishekk | nao-shark, do you want to share anything on this? | 14:07 |
| nao-shark | yes | 14:07 |
| abhishekk | please carry on | 14:07 |
| nao-shark | i also summarized how to configure s3 driver on etherpad | 14:08 |
| nao-shark | and i know deadline of glance_store's final release is approaching | 14:08 |
| jokke_ | I'll try to find time to review that | 14:08 |
| nao-shark | so kindly review above patches | 14:08 |
| nao-shark | thanks. | 14:09 |
| jokke_ | nao-shark: thanks for the very good documentation how to utilize this and sorry for neglegting it for this long | 14:09 |
| abhishekk | jokke_, thank you | 14:09 |
| abhishekk | moving ahead | 14:10 |
| abhishekk | #topic Delete image from single store | 14:10 |
| *** openstack changes topic to "Delete image from single store (Meeting topic: glance)" | 14:10 | |
| abhishekk | #link https://review.opendev.org/#/c/698049 | 14:10 |
| abhishekk | Need review on this as it is a important change for us | 14:11 |
| abhishekk | rosmaita, jokke_ smcginnis kindly reveiw | 14:11 |
| abhishekk | I have added my findings on the patch and made changes accordingly | 14:11 |
| rosmaita | ok | 14:11 |
| abhishekk | I hope we will find middle way here (fingers crossed)) | 14:11 |
| abhishekk | rosmaita, thank you | 14:12 |
| abhishekk | moving ahead | 14:12 |
| abhishekk | #topic Transition Rocky to EM | 14:12 |
| *** openstack changes topic to "Transition Rocky to EM (Meeting topic: glance)" | 14:12 | |
| abhishekk | #link https://review.opendev.org/#/c/709888/2 | 14:12 |
| abhishekk | #link | 14:12 |
| abhishekk | https://etherpad.openstack.org/p/glance-stable-rocky-em | 14:12 |
| abhishekk | this etherpad tells about some patches are pending to be merged against stable/rocky | 14:13 |
| abhishekk | but IMO those are not important and not as per the backport standards | 14:13 |
| *** eharney has quit IRC | 14:13 | |
| abhishekk | I have given my approval for it, kindly suggest if you think otherwise | 14:13 |
| smcginnis | Yeah, looks like maybe some useful stuff in openstack/glance, but nothing too critical. | 14:15 |
| jokke_ | abhishekk: if those backports are not approriate by the standards, are they blocked? | 14:15 |
| abhishekk | jokke_, no as of now | 14:15 |
| smcginnis | And we can still continue to merge fixes after we transition, just can't do an official community release. | 14:15 |
| jokke_ | as in -2'd or -W'd | 14:15 |
| jokke_ | ok, we might want to do that and abandon after | 14:15 |
| abhishekk | jokke_, not ye | 14:15 |
| abhishekk | jokke_, ack | 14:15 |
| jokke_ | kk | 14:15 |
| abhishekk | will do this later tonight | 14:15 |
| abhishekk | also if you have time, kindly have a look | 14:16 |
| *** thgcorrea has joined #openstack-meeting-4 | 14:16 | |
| abhishekk | smcginnis, makes sense | 14:16 |
| abhishekk | Cool, moving ahead; | 14:17 |
| abhishekk | #topic Open discussion | 14:17 |
| *** openstack changes topic to "Open discussion (Meeting topic: glance)" | 14:17 | |
| abhishekk | So as per community goal we were missing some pieces in contributors guide | 14:17 |
| abhishekk | I have added the same | 14:17 |
| abhishekk | kindly have a look; | 14:17 |
| abhishekk | #link https://review.opendev.org/712236 | 14:18 |
| abhishekk | thank you to rosmaita for this :P | 14:18 |
| rosmaita | what did i do? | 14:18 |
| abhishekk | you submitted same for cinder, I took reference from there | 14:19 |
| rosmaita | ok, ok, as long as it was something good | 14:19 |
| rosmaita | thought maybe you were being sarcastic | 14:19 |
| abhishekk | haha | 14:19 |
| abhishekk | Reload is broken under py3 | 14:20 |
| abhishekk | #link https://bugs.launchpad.net/glance/+bug/1855708 | 14:20 |
| openstack | Launchpad bug 1855708 in Glance "Reload tests broken in Py3" [Critical,Triaged] - Assigned to Khuong Luu (organic-doge) | 14:20 |
| abhishekk | We are in serious trouble | 14:20 |
| abhishekk | jokke_, stage is yours | 14:20 |
| jokke_ | ok, so this turned from "flaky test" to actually quite nasty bug | 14:21 |
| jokke_ | This was originally bundled to same pile ith our broken ssl test and at least I thought it was test issue with python 3. feshfood took a proper spin on it and we looked his findings yesterday as he couldn't figure out why the parent process disappeared under him in the middle of the test | 14:22 |
| jokke_ | well looks like py3 multithreading changes actually properly broke our reload. So current situation is that if you try to HUP the glance-api service to pick up config changes on flight, you end up loosing the parent process and leaking orphaned child processes | 14:24 |
| jokke_ | So big hand to Khuong who spent time on this and provided me enough details to draw initial conclusions, but this is nasty and we really need to figure it out | 14:24 |
| *** dave-mccowan has quit IRC | 14:25 | |
| abhishekk | yes, this is very critical or else we will be missing out on big feature of reload | 14:25 |
| jokke_ | and very easy to reproduce, just run glance-ap with bunch of workers, sighup ip (even without config changes) and you see the outcome | 14:25 |
| jokke_ | Yes so one thing which makes this super critical and nasty is that we have been marketing the multistore feature being very easy to add new stores without downtime as you can just add the store in configs and reload and api should pick them up. Well, not exactly | 14:27 |
| rosmaita | i think our code was done before oslo.config supported reloadable config | 14:27 |
| rosmaita | but now it does | 14:27 |
| abhishekk | rosmaita, yes | 14:27 |
| rosmaita | but i think you have to mark the reloadable items explicitly or something | 14:27 |
| abhishekk | not able to recollect it, but there was some limitations on some config parameters which cannot be reloaded | 14:29 |
| *** yamamoto has quit IRC | 14:29 | |
| jokke_ | we do allow only subset of configs too, what looks like the problem is, that iiuc py3 multithreading works in a way where signal handlers are persisten and only processed in the main thread, and we overload different handlers to child and parent which might actually cause us to gracefully die on parent instead of doing any of the reload logic | 14:29 |
| smcginnis | The only issue is where we store off the config item rather than reading it from oslo.config. So "self.value = CONF.value" during init, then just always referencing "self.value". | 14:30 |
| jokke_ | as the child hup handler is just "finish requests in flight and die" | 14:30 |
| rosmaita | smcginnis: thanks for that clarification | 14:31 |
| jokke_ | I don't know how the oslo_config reload works but we actually most of the cases maintain the socket and just respun all the workers with the new config | 14:31 |
| jokke_ | or that's how it worked in py27 | 14:31 |
| bnemec | You do also have to explicitly mark opts as mutable or they won't change on a config reload. | 14:32 |
| *** dave-mccowan has joined #openstack-meeting-4 | 14:32 | |
| bnemec | If there's code you need to run on a reload there's a hook for that too. | 14:32 |
| abhishekk | rosmaita, we don't use oslo.services so oslo.config reload will not work for us | 14:32 |
| jokke_ | glance used to support even changing the bind on the flight where we released the socket and rebind if that was changed, but indeed that's fully separate topic as abhishekk said ^^ | 14:33 |
| abhishekk | bnemec, AFAIK, this will work if and only if you are using oslo.services, right? | 14:33 |
| jokke_ | that's what I thought as well | 14:34 |
| bnemec | You get it for essentially free in oslo.service, but it should be possible to mutate config without it. | 14:35 |
| rosmaita | i guess the question is how much re-architecting we will have to do, and if so, what direction it should take | 14:35 |
| jokke_ | yeah, I do not know yet. I'm still trying to find the actual rot cause, I know the symptoms and likely origin for them | 14:36 |
| jokke_ | root | 14:36 |
| smcginnis | Rot may be applicable too. :) | 14:36 |
| abhishekk | I will also spend some time around this | 14:36 |
| abhishekk | haha | 14:36 |
| jokke_ | mhm | 14:37 |
| jokke_ | I'm just happy that we found out now, not when someone had production failing under them due to this | 14:37 |
| rosmaita | yes indeed | 14:37 |
| abhishekk | +1 | 14:37 |
| jokke_ | but this needs some cycles sooner than later | 14:37 |
| rosmaita | yeah, because with this release its no longer possible to run under py27 | 14:38 |
| jokke_ | indeed | 14:38 |
| abhishekk | yes | 14:38 |
| jokke_ | and we claim py3 support | 14:38 |
| bnemec | If you're using cotyledon it looks like that supports mutable config too, FYI. | 14:38 |
| jokke_ | cotyledon? | 14:39 |
| abhishekk | need to roll our sleeves for this | 14:39 |
| rosmaita | so, may need to send something to the ML that it's not a good idea to run train, etc under py3 if you intend to use the reload config via sighup feature | 14:39 |
| jokke_ | rosmaita: yeah, will do that | 14:39 |
| bnemec | It was a non-eventlet replacement for oslo.service. | 14:39 |
| jokke_ | bnemec: ah, well we're running eventlet+oslo_config :P | 14:39 |
| jokke_ | I think we're on some modified incubator stage of oslo.service before it became oslo.service | 14:40 |
| abhishekk | yes | 14:40 |
| jokke_ | like we've been on lots of those incubator thingies | 14:40 |
| bnemec | Ah, yuck. :-/ | 14:40 |
| *** ricolin_ has quit IRC | 14:41 | |
| abhishekk | I remember I have migrated glance to oslo.service but that was rejected at that time :P | 14:41 |
| *** ricolin_ has joined #openstack-meeting-4 | 14:41 | |
| jokke_ | it was craploads of refactoring and broke about dozen core things :P | 14:41 |
| jokke_ | I remember you looking into it at the time | 14:42 |
| abhishekk | yeah | 14:42 |
| jokke_ | but that's it from me, at this point just awareness and shout out to be careful and have a look if you have spare cycles | 14:43 |
| abhishekk | I think jokke_ and I should co-ordinate with each other to share the findings | 14:44 |
| jokke_ | I'll keep my focus on those couple of critical reviews, this and the uncompress plugin for now | 14:44 |
| abhishekk | ack, I will try to spend some time on it as don't have anything big atm | 14:44 |
| abhishekk | apart from delete from store thing, which is already in good shape | 14:45 |
| jokke_ | it was in good shape 3 months ago :P | 14:45 |
| abhishekk | haha | 14:46 |
| rosmaita | cyril ran into something yesterday ... looks like that gate job cinderclient was using to make sure it still worked with v1 API wasn't actually testing patches until *after* they were merged to master already | 14:46 |
| rosmaita | we didn't notice because we never broke v1 compatability, apparently | 14:47 |
| jokke_ | nice | 14:47 |
| abhishekk | rosmaita, yes, he has backported some patches to stable/train yesterday | 14:47 |
| *** ricolin_ has quit IRC | 14:48 | |
| rosmaita | yeah, so my point is, don't use that zuul config as a model if you need to do something like that in the future! | 14:48 |
| jokke_ | ;) | 14:49 |
| jokke_ | good point, thnx | 14:49 |
| abhishekk | last 10 minutes | 14:50 |
| jokke_ | I'm good | 14:52 |
| *** psachin has quit IRC | 14:52 | |
| abhishekk | rosmaita, smcginnis ? | 14:52 |
| rosmaita | nothing from me | 14:53 |
| abhishekk | cool, wrapping it up for today | 14:54 |
| abhishekk | guys keep reviewing important patches | 14:54 |
| abhishekk | Thank you all | 14:54 |
| jokke_ | Thanks all! | 14:54 |
| abhishekk | #endmeeting | 14:55 |
| *** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/" | 14:55 | |
| openstack | Meeting ended Thu Mar 12 14:55:00 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:55 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/glance/2020/glance.2020-03-12-14.00.html | 14:55 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/glance/2020/glance.2020-03-12-14.00.txt | 14:55 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/glance/2020/glance.2020-03-12-14.00.log.html | 14:55 |
| *** nao-shark has quit IRC | 14:55 | |
| *** rosmaita has left #openstack-meeting-4 | 14:55 | |
| *** Xuchu has joined #openstack-meeting-4 | 15:01 | |
| *** danielarthurt has joined #openstack-meeting-4 | 15:01 | |
| *** yamamoto has joined #openstack-meeting-4 | 15:02 | |
| *** enriquetaso has quit IRC | 15:02 | |
| *** enriquetaso has joined #openstack-meeting-4 | 15:02 | |
| *** yamamoto has quit IRC | 15:07 | |
| *** Xuchu has left #openstack-meeting-4 | 15:16 | |
| *** gcheresh has quit IRC | 15:43 | |
| *** eharney has joined #openstack-meeting-4 | 15:49 | |
| *** gcheresh has joined #openstack-meeting-4 | 15:49 | |
| *** diablo_rojo has joined #openstack-meeting-4 | 15:58 | |
| *** diablo_rojo has quit IRC | 16:06 | |
| *** roman_g has joined #openstack-meeting-4 | 16:07 | |
| *** diablo_rojo has joined #openstack-meeting-4 | 16:10 | |
| *** yamamoto has joined #openstack-meeting-4 | 16:20 | |
| *** gcheresh has quit IRC | 16:30 | |
| *** Xuchu has joined #openstack-meeting-4 | 16:34 | |
| *** gcheresh has joined #openstack-meeting-4 | 16:37 | |
| *** enriquetaso has quit IRC | 16:38 | |
| *** gcheresh has quit IRC | 16:50 | |
| *** e0ne_ has quit IRC | 17:06 | |
| *** jraju__ has quit IRC | 17:16 | |
| *** evrardjp has quit IRC | 17:35 | |
| *** evrardjp has joined #openstack-meeting-4 | 17:36 | |
| *** kozhukalov has joined #openstack-meeting-4 | 17:46 | |
| *** andrebeltrami has quit IRC | 18:03 | |
| *** ralonsoh has quit IRC | 18:29 | |
| *** e0ne has joined #openstack-meeting-4 | 18:48 | |
| *** e0ne has quit IRC | 18:52 | |
| *** gmann is now known as gmann_lunch | 19:00 | |
| *** danielarthurt has quit IRC | 19:01 | |
| *** kozhukalov has quit IRC | 19:13 | |
| *** gcheresh has joined #openstack-meeting-4 | 19:23 | |
| *** e0ne has joined #openstack-meeting-4 | 19:36 | |
| *** trident has quit IRC | 19:51 | |
| *** trident has joined #openstack-meeting-4 | 19:59 | |
| *** gmann_lunch is now known as gmann | 20:01 | |
| *** gcheresh has quit IRC | 20:11 | |
| *** gcheresh has joined #openstack-meeting-4 | 20:18 | |
| *** ktibi has quit IRC | 20:45 | |
| *** gcheresh has quit IRC | 21:09 | |
| *** e0ne has quit IRC | 21:19 | |
| *** enriquetaso has joined #openstack-meeting-4 | 21:28 | |
| *** enriquetaso has quit IRC | 21:32 | |
| *** senrique_ has joined #openstack-meeting-4 | 21:32 | |
| *** senrique_ has quit IRC | 21:38 | |
| *** slaweq has quit IRC | 22:28 | |
| *** dave-mccowan has quit IRC | 22:50 | |
| *** dave-mccowan has joined #openstack-meeting-4 | 23:10 | |
| *** liuyulong has quit IRC | 23:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!